I keep going back and forth on this feeling, but lately I find myself thinking "F it, I'm going to do what I'm going to do that interests me".
Today I'm working on doing the unthinkable in an AI-world: putting together a video course that teaches developers how to use Phlex components in Rails projects and selling it for a few hundred bucks.
One way of thinking about AI is that it puts so much new information in front of people that they're going to need help from people known to have experience to navigate it all and curate it. Maybe that will become more valuable?
Who knows. That's the worst part at this moment in time—nobody really knows the depths or limits of it all. We'll see breakthroughs in some areas, and others not.
From all of my observations, the impact of LLMs on human thought quality appears largely corrosive.
I’m very glad my kid’s school has hardcore banned them. In some class they only allow students to turn in work that was done in class, under the direct observation of the teacher. There has also been a significant increase in “on paper” work vs work done on computer.
Lest you wonder “what does this guy know anyways?”, I’ll share that I grew up in a household where both parents were professors of education.
Understanding the effectiveness of different methods of learning (my dad literally taught Science Methods) were a frequent topic. Active learning (creating things using what you’re learning about) is so much more effective than passive, reception oriented methods. I think LLMs largely are supporting the latter.
Anyone who has learned a second language can tell you that you aren't proficient just by memorizing vocabulary and grammar. Having a conversation and forming sentences on the fly just feels different- either as a different skill or using a different part of the brain.
I also don't think the nature of LLMs being a negative crutch is new knowledge per se; when I was in school, calculus class required a graphing calculator but the higher end models (TI-92 etc) that had symbolic equation solvers were also banned, for exactly the same reason. Having something that can give an answer for you fundamentally undermines the value of the exercise in the first place, and cripples your growth while you use it.
Same with drawing which is easy to teach, but hard to master because of the coordination between eyes and hand. You can trace a photograph, but that just bypass the whole point and you don’t exercise any of the knowledge.
Another case in point is that memorizing vocabularies and grammar, although could seem like an efficient way to learn a language, is incredibly unrewarding. I've been learning japanese from scratch, using only real speech to absorb new words, without using dictionaries and anything else much. The first feeling of reward came immediately when I learned that "arigatou" means thanks (although I terribly misheard how the word sounded, but hey, at least I heard it). Then after 6 month, when I could catch and understand some simple phrases. After 6-7 years I can understand about 80% of any given speech, which is still far, but I gotta say it was a good experience.
With LLM's giving you ready-made answers I feel like it's the same. It's not as rewarding because you haven't obtained the answer yourself. Although it did feel rewarding when I was interrogating an LLM about how CSRF works and it said I asked a great question when I asked whether it only applies to forms because it seems like fetch has a different kind of browser protection.
Mostly anime. Surprisingly, not that much, I think somewhere in the ballpark of 100 titles. In the beginning I was also watching some grammar tutorials on YouTube to get started with grammar quicker (Otherwise convergence on solution would be too slow).
Contrary to what I said I actually did use dictionaries, but the point I was trying to make is rather than memorizing phrases in advance, I used it to translate something I thought I heard.
If you used subtitles over audio then why would you avoid dictionaries too ? Purely for the reward of treating it as a puzzle ? (Since you would have to figure out which word corresponds to a which concept in a phrase.)
Comparing extracting a square root my hand is rather different in scope than reducing / simplifying equations entirely. The TI-92 could basically do all of your coursework for you up to college level, if memory serves.
The real question isn't "is it okay to use a tool" but "how does using a tool affect what you learn".
In the cases of both LLMs and symbolic solving calculators, I believe the answer is "highly detrimental".
Needs are all fabricated, Ludwig Wittgenstein said "the limits of my language are the limits of my world", the same thing happens with logical thinking and all its tools including math.
Well, I think one should be able to come up with a way of doing so on the fly just from knowing that the square root function is monotonically increasing, and knowing binary search? Of course, doing it another way might be more efficient.
The manual methods are also the foundation for higher approaches involving approximation and iterative solutions. These are widely used in engineering and science.
Pressing a calculator key doesn’t give the same insight.
Memorization and manual methods also help develop intuition about what is a plausible answer and what is not. It helps build a brain that is able to sanity-check what it is being told.
Using a tool like that is opposite to mastering the skill. There's no royal road to mastery and never will be. One does not have to master all skills, of course, and may do well not mastering any (or mastering dark ones).
Yes they are dumber because of it, not in the mental retardation kind of way but a more nuanced way, among others the mental work you put into trying to find another simpler way than the one the professor is teaching you, and the understanding about numbers such attempts can give you, even if they are unsuccessful.
> No one to day learns that anymore. The vast, vast majority have no idea and I don’t think people are dumber because of it.
Arguably, the kind of person who was helped by learning to do that by hand still learns to do it by hand, but because of curiosity rather than because a teacher told them to.
I remember being thirteen and trying to brute force methods for computing the square root. I didn’t have the tools yet to figure out how to do it in any systematic way, and the internet wasn’t at a point yet where it would have even occurred to me to just search online. Wikipedia wouldn’t exist for another two years.
I probably finally looked it up at some point in high school. I’m not sure exactly when, but I remember spending a lot of time practicing doing a few iterations in my head as a parlor trick (not that I ever had the opportunity to show it off).
If I were thirteen and curious about that now, I’d probably just ask ChatGPT. Then I’d have a whole follow up conversation about how it was derived. It would spit a lot of intimidating LaTeX at me, but unlike with Wikipedia, I’d be able to ask it to explain what those things meant.
This is the thing I don’t get when people talk about LLMs’ impact on education. Everybody focuses on cheating, like learning is inherently a chore that all students hate and must be carefully herded into doing despite themselves.
But that’s a problem with school, not learning. If your actual, self-motivated goal is to learn something, LLM’s are an incredible tool, not a hindrance.
Any school's #1 job is to motivate learning. Schools clearly suck at this.
LLMs are a fascinating effective learning tool. early learning would be better off embracing it as such.
i imagine a combination of a video of a good tutor explaining a concept followed up with an llm to quiz and explain the concept seems far better than what we have today.
I think we need to find a way to teach without grades. That is, we do know how to teach without grades at younger levels, and many schools do it successfully.
The problem is that eventually you need to measure for placement, and Goodhart’s Law kicks in and destroys the enjoyment of learning. It’s very hard to be intrinsically motivated when the external pressure dominates.
The fact that “study for the test” is basically synonymous with “study” for most people is indicative that we’re doing something extremely wrong in education.
> The problem is that eventually you need to measure for placement
Why do you need to be "placed"? I know the answer - resources.
But why can't each individual have sort of their own bespoke governess tutoring them all through school? Pay Gilbert Strang a million dollars to do a 32 week course in LinAlg. Yes I know a normal course is shorter, expand it to cover everything in exquisite detail, so even I can keep up. Alan Kay, Donald Knuth, and John Carmack can teach computer science. Continue on for each subject.
Then let students go as fast as they want. I'd have finished some subjects of HS by the time I was in 9th grade, and others I might still be trying. ¯\_(ツ)_/¯
Where you'd still need to be "placed" is social development. Which I think should actually be a subject.
Feels different, comes naturally, without conscious thought, just like we don't focus on beating our hearts.
And agree about learning by practicing a skill being best. But you and I both know the school system has worked on rote memorisation for hundreds of years at least and still is now.
> the higher end models (TI-92 etc) that had symbolic equation solvers were also banned
I'm surprised it was a problem in the first place. Don't equation solving exercises require you to leave intermediary steps, and you can't just put a "x=5" as a one liner answer ?
We used TI-Nspire CX CAS at my university but it was banned in the first half of the calculus 1 course. It would really make no sense to allow you to bring one to an exam where many questions are one liners, or close to it (eg. "find the derivative of f(x)=7x^5+10"). IIRC, they also banned it in a calc 2 exam. Almost all other courses allowed them and they exams where designed for it (eg. 8 big problems instead of 2).
I am waiting for the day (maybe it's already here) when I can talk to an LLM to practice my 2nd language. It can correct everything I say, it can talk forever, it can challenge me to use new grammar or vocabularly. Note: I can speak all day in my 2nd language with friends but I wouldn't give a business presentation nor could I explain, as a native, how something technical works. If I watch a TV show I might understand 30%-99% but the more lawyers/military/goverment/science parts there are the more it's beyond my current level.
Getting exposure there is hard. Talking to friends just means more practice with what I already know but an LLM could help me practice things outside that area.
This reminds me how back in my school days I was not allowed to use the internet to prepare research on some random topics (e g. history essay). It was the late 90s when the internet started to spread. Anyway teachers forced us to use offline libraries only.
Later in the university I was studying engineering. And we were forced to prepare all the technical drawings manually in the first year of study. Like literally with pencil and ruler. Even though computer graphics were widely used and we're de facto standard.
Personally I don't believe hardcore ban will help with any sort of thing. It won't stop the progress either. It's much better to help people learn how to use things instead of forcing them to deal with "old school" stuff only.
I was expecting some response like this, because schools have “banned” things in the past.
While this is superficially similar, I believe we are talking about substantially different things.
Learning (the goal) is a process. In the case of an assignment, the resulting answer / work product, while it is what is requested, is critically not the goal. However, it is what is evaluated, so many confuse it with the goal (“I want to get a good grade”)
Anything which bypasses the process makes the goal (learning) less likely to be achieved.
So, I think it is fine to use a calculator to accelerate your use of operations you have already learned and understand.
However, I don’t think you should give 3rd graders calculators that just give them the answer to a multiplication or division when they are learning how those things work in the first place.
Similarly, I think it’s fine to do research using the internet to read sources you use to create your own work.
Meanwhile, I don’t think it’s fine to do research using the internet to find a site where you can buy a paper you can submit as your own work.
Right now, LLMs can be used to bypass a great deal of process, which is why I support them not being used.
It’s possible, maybe even likely that we’ll end up with a “supervised learning by AI” approach where the assignment is replaced by “proof of process”, a record of how the student explored the topic interactively. I could see that working if done right.
You can learn a lot from LLMs though, same with, say, Wikipedia. You need curiosity. You need the desire to learn. If you do not have it, then of course you will get nowhere, LLMs or no LLMs.
> if it really is laziness, and not something else, e.g. a mental health issue.
Kids optimize. When I was in high school I was fully capable of getting straight F's in a class I didn't care about and straight A's in a class I enjoyed.
Why bother learning chemistry when you could instead spend that time coding cool plugins and websites in PHP that thousands of internet strangers are using? I really did build one of the most popular phpBB plugins and knew I was gonna be a software engineer. Not that my chemistry professor cared about any of that or even understood what I'm talking about.
How is it irrelevant? Kids will always cheat their way through classes they feel are a distraction. Even the super smart Type A kids.
Hell, all humans do that. You use every resource available to get out of dealing with things that are not your priority. This means you will never be good at those things and that’s fine. You can’t be good at everything.
They will, but we were talking about the will or motivation to learn. If someone has a curious mind, and actually wants to learn, then they can definitely use LLMs to do that.
Realistically, putting them into trades sooner could almost be a good thing. Kids who don’t want to learn end up dragging down the class and distracting those who do.
But, these are kids… Hard to argue that adults should selectively deny education when it is their responsibility to do otherwise.
We don’t neglect the handicapped because it is inconvenient to provide them with assistance.
Maybe my comment was a little harsh. I do believe we should have the structure in place for people to learn from as they wish. We should show them it's available, and the benefits of it
But the lazy, uncurious person that wants to shortcut everything, that you described? They're not even trying to help themselves. Maybe the best thing to do there is to let a little time pass and for them to see the result of their actions. You can lead a horse to water..
> Maybe the best thing to do there is to let a little time pass and for them to see the result of their actions
This is most kids, myself included. I’m curious and have done well, but if I could have had essays written for me as a kid for sure I’d have tried. And I’d be off worse for it.
Kids also just want to eat sugar and play video games. As parents and a society, we both provide and force alternatives because as adults we know better.
You can lead a horse to water, sure, but you could also lead it to something filled with their worst instincts. Let’s avoid the later.
You can learn a lot, if you want to. I can ask it a question with regarding to pharmacodynamics of some medication, and I can ask more and more questions, and learn. Similarly, I could pick up a book on pharmacology, but LLMs can definitely make learning easier.
You can learn an incredible amount. I do quite a bit of research as a core part of my job, and LLMs are amazing at helping me find relevant research to help me explore ideas. Something like "I'm thinking of X. Does this make sense and do you know of any similar research?" I also mentor some students whose educational journey has been fundamentally changed by them.
Like any other tool, it's more a question of how they're used. For example, I've seen incredible results for students who use ChatGPT to interrogate ideas as they synthesize them. So, for example, "I'm reading this passage PASSAGE and I'm confused about phrase X. The core idea seems similar to Y, which I am familiar with. if I had to explain X, I'd put it like this ATTEMPT Can you help me understand what I'm missing?"
The results are very impressive. I'd encourage you to try it out if you haven't.
“The irony is that I now know more than I ever would have before AI. But I feel slightly dumber. A bit more dull. LLMs give me finished thoughts, polished and convincing, but none of the intellectual growth that comes from developing them myself. The output from AI answers questions. It teaches me facts. But it doesn’t really help me know anything new.”
I think the thesis is that with AI there is less need and incentive to “put the work in” instead of just consuming what the AI outputs, and that in consequence we do the needed work less and atrophy.
I know, that is why you need the desire, the will to learn. I have been using LLMs for this, so I know it is possible. I understand what you are saying though, and it is indeed a sad state of affairs, but then again, this was the case due to search engines, Wikipedia, and so forth, long before LLMs.
Again, you can truly learn a lot using LLMs, but you have to approach it properly. It does not have to be just "facts", and sometimes, even learning "facts" is learning.
I can use LLMs and learn nothing, but I can use LLMs to learn, too!
Yes, but previously you didn’t need the desire that much, because you were more forced to it, there was no easy way. The fact that now you need that internal motivation means that it will happen less, where previously it happened by default.
Honestly, I doubt that LLMs are great for learning. Too often, they output plausible-sounding things that turn out to be completely wrong. I know Wikipedia can have its problems with factuality, but this is on an entirely different level. (And yes, they do this even when they're allowed to do web searches and "reason".)
The effort of verifying everything it claims may or may not outweigh the effort of other means of learning.
We are talking about learning. You can learn much more from LLMs than Wikipedia, because if you do not understand something, you can always ask an LLM about it, and it would reply to you in any way you want; whatever helps you learn better.
> I’m very glad my kid’s school has hardcore banned them.
What does that mean, I’m curious?
The schools and university I grew up in had a “single-sanction honor code” which meant if you were caught lying or cheating even once you would be expelled. And you signed the honor code at the top of every test.
My more progressive friends at other schools who didn’t have an honor code happily poo-pooed it as a repugnantly harsh old fashioned standard. But I don’t see a better way today of enforcing “don’t use AI” in schools, than it.
I’m not sure how LLMs output is indistinguishable from Wikipedia or World Book.
Maybe? and if the question is “did the student actually write this?” (which is different than “do they understand it?” there are lots of different ways to assess if a given student understands the material…that don’t involve submitting typed text but still involve communicating clearly.
If we allow LLMs- like we allow calculators, just how poor LLMs are will become far more obvious.
Do you really not see the problem? A student who pastes an essay prompt into an input box and copies out the response has learned nothing. Even direct plagiarism from Wikipedia would typically need to be reworked; there will rarely be a Wikipedia page corresponding to your teacher's specific essay prompt.
Students are also poor writers. Often LLM-generated essays can be spotted in elementary school because they write too well for that grade level. A good student will surpass a chatbot, but not if they use it as a crutch while it's still a stronger writer than they are.
The school has an academic honestly policy which explicitly bans it, under “Cheating”, which includes:
“Falsifying or inventing any academic work, including the use of AI (ChatGPT, etc)”
Additionally, as mentioned, the school is taking actions to change how work is done to ensure students are actually doing their own work - such as requiring written assignments be completed during class time, or giving homework on physical paper that is to be marked up by hand and returned.
Apparently this is the first year they have been doing this, as last year they had significant problems with submitted work not being authored by students.
This is in an extremely competitive Bay Area school, so there can be a lot of pressure from parents on students to make top grades, and sometimes that has negative side effects.
Asking as a non-American non-school-pupil-parent: what does it mean for a school to be competitive in this context? Competitive entry into a school I understand, but that threshold has been cleared. Isn’t US college admission based on essays and standardised tests like GMAT, SAT, GRE?
Among the many ways that AI causes me existential angst, you've reminded me of another one. That is, the fact that AI pushes you towards the most average thoughts. It makes sense, given the technology. This scares me because creative thought happens at the very edge. When you get stuck on a problem, like you mentioned, you're on the cusp of something novel that will at the very least grow you as a person. The temptation to use AI could rob you of the novelty in favor what has already been done.
I feel there's a glaring counter point to this. I have never felt more compelled to try out whatever coding idea that pops into my head. I can make Claude write a poc in seconds to make the idea more concrete. And I can write into a good enough tool in a few afternoons. Before this all those ideas would just never materialize.
I mean I get the existential angst though. There's a lot of uncertainty about where all this is heading. But, and this is really a tangent, I feel that the direction of it all is in the intersection between politics, technology and human nature. I feel like "we the people" leave walkover to powerful actors if we do not use these new powerful tools in service of the people. For one - to enable new ways to coordinate and organise.
Good point. It's not that AI is "pushing us" towards anything. AI can be a muse that elevates our creativity. IF we use it that way. But do we use it that way? I think there will be some who do.
The majority of users seem to want convenience at any expense. Most are unconcerned with a loss of agency, almost enthusiastic about it if it removes the labor of thinking.
Agency only goes away if control of AI is ultimately centralized. If we end up in a world where anyone can run good enough models on consumer devices and we can install our own models into off the shelf humanoid robots I don't see that we have lost agency.
That's interesting point. But here is the thing: you are supposed to drive. Not AI god. Look at it as at an assistant whom you can interrupt, instruct, correct, ask to redo. While focusing on 'what' you can delegate it some 'how' problems.
I for one, thing directing subordinates to do something I could be doing kinda … sucks? Like, I get that’s how you have to work with LLMs, but it isn’t a fun thing to do for me.
Non hierarchical collaboration is the option you are excluding. Where you accept pushback and feedback because you know it comes with creative vision and perspective you lack. You can do creative things with other creative people.
It depends how much. I used LLM to do simple web development for me. I even don't want to learn HTML, JS that much. I'm already hitting the limits. For the same reasons writing complex ffmpeg command line (can do simple myself), using some libs just ones. There are many small things makes no sense to learn too deep. Easier to offload. For narrow tasks in big companies it's probably different, but open ended hobby robotics covers almost everything. I can't be an expert in all topics. Now with LLM assistant one persone alone can work like a small team.
Yeah this is a fair point. In honesty, the attempts I have made to have GPT help me think creatively has usually left me disappointed and feeling like it was picking safe, middle-of-the-road solutions. That could be on my prompting skills but also I tend to view LLMs as more of a fuzzy information retrieval tool than a creative/reasonable one. It just hasn’t shown me original ideas that have seemed compelling to me yet (maybe I just need to beg it to be more “original”).
It takes practice. LLM can't read your mind, put it in prompt if you have some specific ideas. Usually helps to ask for options and overview first. You may get some ideas here. I think communication barrier is one of biggest problems when working with LLM. You can often see posts like "I asked it for XX" and it gave me some crap in return. Well, it depends on 'how' you ask. Also you need to consciously stay in control. At takes some efforts and usually LLM is arrogant, bossy, self confident, and sometimes manipulative. Just like human 'experts' on forums.
I was playing around with AI autocomplete, and found it to be good for a month or two. Then, it suggested I upgrade the model to match my new computer’s increased performance. It’s useless now. The worse model was usable for creative writing and chatrooms; the new models are fit strictly for business professional communications.
I've noticed something like this as well. A suggestion is to write/build for no one but yourself. Really no one but yourself.
Some of my best writing came during the time that I didn't try to publicize the content. I didn't even put my name on it. But doing that and staying interested enough to spend the hours to think and write and build takes a strange discipline. Easy for me to say as I don't know that I've had it myself.
Another way to think about it: Does AI turn you into Garry Kasparov (who kept playing chess as AI beat him) or Lee Sedol (who, at least for now, has retired from Go)?
If there's no way through this time, I'll just have to occasionally smooth out the crinkled digital copies of my past thoughts and sigh wistfully. But I don't think it's the end.
Yeah there is the personal passion and then the points/likes driven which sucks the joy out
I experienced this when I was younger with my rc planes, I joined some forum and I felt like everything I did had to be posted/liked to have value. I'd post designs/fantasy and get the likes then lose interest/not actually do it after I got the ego bump
I think that's why some writers refuse to talk about their work in progress. If they do, it saps the life out of it. Your ego bump example kind of fits into that.
I just wrote a paper a few days ago arguing that "manual thinking" is going to become a rare and valuable skill in the future. When you look around you, everyone is finding ways to be better using AI, and they're all finding amazing successes – but we're also unsure about the downsides. I hedge that my advantage in ten years will be that I chose not to do what everyone else did. I might regret it, we will see.
If AI is going to be as economically efficient as promised, there is going to be no way to avoid using it altogether. So the trick will be to keep your thinking skills functional, while still using AI for speedup. Like focus in the age of Internet is a rare skill, but not using Internet is not an option either.
Not all processes are the same, though. I strongly suspect any efficiency improvements will come in processes that didn't require much "thinking" to begin with. I use it daily, but mostly as a way to essentially type faster—I can read much faster than I can type, so I mostly validate and correct the autocomplete. All of my efforts to get it to produce trustworthy output beyond this seem to trail behind the efficiency of just searching the internet.
Granted, I'm blessed to not have much busywork; if I need to produce corporate docs or listicles AI would be a massive boon. But I also suspect AI will be used to digest these things back into small bullet points.
I figure if AI gets as efficient as people seem to think. Then spending a bunch of effort getting good at using it now is kind of pointless, because it’s just going to get easier and easier to use.
I've been finding a lot of fulfillment in using AI to assist with things that are (for now) outside of the scope of one-shot AI. For example, when working on projects that require physical assembly or hands-on work, AI feels more like a superpower than a crutch, and it enables me to tackle projects that I wouldn't have touched otherwise. In my case, this was applied to physical building, electronics, and multimedia projects that rely on simple code that are outside of my domain of expertise.
The core takeaway for me is that if you have the desire to stretch your scope as wide as possible, you can get things done in a fun way with reduced friction, and still feel like your physical being is what made the project happen. Often this means doing something that is either multidisciplinary or outside of the scope of just being behind a computer screen, which isn't everyone's desire and that's okay, too.
Yeah I haven't found the right language for this yet, but it's something like: I'm happy and optimistic about LLMs when I'm the one doing something, and more anxious about them when I'm supporting someone else in doing something. Or: It makes me more excited to focus on ends, and less excited to focus on means.
Like, in the recent past, someone who wanted to achieve some goal with software would either need to learn a bunch of stuff about software development, or would need to hire someone like me to bring their idea to life. But now, they can get a lot further on their own, with the support of these new tools.
I think that's good, but it's also nerve-wracking from an employment perspective. But my ultimate conclusion is that I want to work closer to the ends rather than the means.
Interesting, I just replied to this post recommending the exact opposite: to focus on means vs ends.
The post laments how everything is useless when any conceivable "end state" a human can do will be inferior to what LLMs can do.
So an honest attention toward the means of how something comes about—the process of the thinking vs the polished great thought—is what life is made of.
Another comment talks about hand-made bread. People do it and enjoy it even though "making bread is a solved problem".
I saw that and thought it was an interesting dichotomy.
I think a way to square the circle is to recognize that people have different goals at different times. As a person with a family who is not independently wealthy, I care a lot about being economically productive. But I also separately care about the joy of creation.
If my goal in making a loaf of bread is economic productivity, I will be happy if I have a robot available that helps me do that quickly. But if my goal is to find joy in the act of creation, I will not use that robot because it would not achieve that goal.
I do still find joy in the act of creating software, but that was already dwindling long before chatgpt launched, and mostly what I'm doing with computers is with the goal of economic productivity.
But yeah I'll probably still create software just for the joy of it from time to time in the future, and I'm unlikely to use AIs for those projects!
But at work, I'm gonna be directing my efforts toward taking advantage of the tools available to create useful things efficiently.
ooh I like this take. We can change the framing. In the frame of one's livelihood we need to be concerned with economic productively, philosophy be damned.
This is only a problem if one is writing/thinking on things which have already been written about without creating a new/novel approach in one's writing.
An AI is _not_ going to get awarded a PhD, since by definition, such are earned by extending the boundaries of human knowledge:
I agree, frustration steps in earlier than before because AIs tell you very eagerly that unique thought is already part of its training data. Sometimes I wish one could put an AI on drugs and filter out some hallucinations that'll become main stream next week.
I’m so so glad at the end it said “Written entirely by a human” because all the way through I was thinking - absolutely no way does AI come up with great ideas or insights and it definitely would not write this article- it holds together and flows too well (if this turns out to be the joke I’ll admit we’re all now fucked)
Having said that I am very worried about kids growing up with AI and it stunting their critical thinking before it begins - but as of right this moment AI is extremely sub par at genuinely good ideas or writing.
It’s an amazing and useful tool I use all the time though and would struggle to be without.
I used to feel the same way about AI, but my perspective has completely changed.
The key is to treat AI as a tool, not as a magic wand that will do everything for you.
Even if AI could handle every task, leaning on it that way would mean surrendering control of your own life—and that’s never healthy.
What works for me is keeping responsibility for the big picture—what I want to achieve and how all the pieces fit together—while using AI for well-defined tasks. That way I stay fully in control, and it’s a lot more fun this way too.
The article mentions that spell and grammar checking AI was used to help form the article. I think there is a spectrum here, with spell and grammar checking on one end, and the fears the article mentions on the other end (AI replacing our necessity to think). If we had a dial to manually adjust what AI works on, this may help solve the problems mentioned here. The issue is that all the AI companies are trying too hard to achieve AGI, and thus making the interfaces general and without controls like this.
I don't think LLMs replace thinking, but rather elevate it. When I use an LLM, I’m still doing the intellectual work, but I’m freed from the mechanics of writing. It’s similar to programming in C instead of assembly: I’m operating at a higher level of abstraction, focusing more on what I want to say than how to say it.
The writing is the work, though.
The words on paper (or wherever) are the end product,
but they are not the point.
See chapter 5 of Ahrens, Sönke. 2017. How to take smart notes: one simple technique to boost writing, learning and thinking - for students, academics and nonfiction book writers.,
for advice on how writing the ideas in your own words is the primary task and improves not only writing,
but all intellectual skills,
including reading and thinking.
C. Wright Mills in his 1952 essay, "“On Intellectual Craftsmanship" says much the same thing.
Stating the ideas in your own words is thinking.
You can also use a microwave to bloom spices, or thaw frozen veggies from your home garden, or steam things, or thicken sauces, …
The microwave is a tool with certain useful aspects and certain limitations. It is also a tool which can lead to faster outcomes of things you need to do if the tool didn’t exist. At what point should a chef draw the line in the tools they use? Should I forgo microwaves? What about pressure cookers? Ovens? Surely knives are fair game? Maybe I should knap flint and butcher meat with it and cook over an open campfire — then truly no one can claim I am not a chef.
> nothing I make organically can compete with what AI already produces—or soon will.
No LLM can ever express your unique human experience (or even speak from experience), so on that axis of competition you win by default.
Regurgitating facts and the mean opinion on topics is no replacement for the thoughts of a unique human. The idea that you're competing with AI on some absolute scale of the quality of your thought is a sad way to live.
More generally, prior to LLMs, you were competing with 8 billion people alive (plus all of our notable dead). Any novel you could write probably had some precedent. Any personal story you could tell probably happened to someone else too. Any skill you wanted to develop, there probably was another person more capable of doing the same.
It was never a useful metric to begin with. If your life goal is to be #1 on the planet, the odds are not in your favor. And if you get there, it's almost certainly going to be unfulfilling. Who is the #1 Java programmer in the world? The #1 topologist? Do they get a lot of recognition and love?
But inevitably you lose in the flood of enshitified creations made with LLMs.
I think we will come back to roots, the simple in person creation: pen and paper, declamation, theatre, live performance, hand painting, improvisation, handmade work.
Maybe not everybody but it will be for (mentally) free people.
I used to write open source a lot but lately, I don't see the point. Not because I think LLMs can produce novel code as good code as me or will be able to in the near future. But because any time I come up with a new solution to something, it will be stolen and used without my permission, without giving me credit or without giving users the rights I give them. And it will be mangled just enough that I can't prove anything.
Large corporations were so anal about copyright that people who ever saw Microsoft's code were forbidden from contributing to FOSS alternatives like wine. But only as long as copyright suited them. Now abolishing copyright promises the C-suite even bigger rewards by getting rid of those pesky expensive programmers, if only they could just steal enough code to mix and match it with enough plausible deniability.
And so even though _in principle_ anybody using my AGPL code or anything that incorporates my AGPL code has the right to inspect and modify said code; yet now tine fractions of my AGPL code now have millions or potentially billions of users but nobody knows and nobody has the right to do anything about it.
And those who benefit the most are those who already have more money than they can spend.
Agreed. I keep seeing posts where people claim the output is all that really matters (particularly with code), and I think that's missing something deeply fundamental about being human.
Think Bigger. As the LLM's capability expands use it to push yourself outside your comfort zone every now and then. I have done some great fun projects recently I would have never thought of tackling before
This person should probably read pre-Internet books, discover or rediscover that the bar for passable expression in text is very low compared to what it was.
Most of that 'corpus' isn't even on the Internet so it is wholly unknown to our "AI" masters.
AI is far better in style than in substance. That is, an AI-written solution will have all the trappings of expertise and deep thought, but frankly is usually at best mediocre. It's sort of like when you hear someone make a very eloquent point and your instinct is to think "wow, that's the final word on the subject, then!" ... and then someone who actually understands the subject points out the flaw in the elegant argument, and it falls down like a house of cards. So, at least for now, don't be fooled! Develop expertise, be the person who really understands stuff.
What happens to conversation in this case? When groups of people are trained to use LLMs as a crutch for thinking, what happens when people get together and need to discuss something. I feel like the averageness of the thinking would get compounded so that the conversation as a whole becomes nothing but a staging ground for a prompt. Would an hour long conversation about the intricacies of a system architecture become a ten minute chatter of what prompts people would want to ask? Does everyone then channel their LLM to the rest of the group? In the end, would the most probable response to which all LLMs being used agree with be the result of the conversation?
I want to see people who used LLMs all their conscious life enter the workforce. It's going to be amazing. I mean soon we'll see people who are essentially raised by LLMs.
What we need is mental gyms. In modern society there is no need for physical labor but we go to gyms just to keep ourselves healthy.
Similarly in future we will not need mental "labor" but to keep ourselves sharp we need engage in mental exercises. I am thinking of picking up chess again just for this reason.
I've had the same experience but with drawing. What's the point when AI can generate perfect finished pieces in seconds? Why put all that effort in to learning and drawing something. It's always been hard for me but it used to feel worth it for the finished piece, but now you can bring a piece of computer art into being with a simple word prompt.
I still create, I just use physical materials like clay and such, to make things that AI can't yet replicate.
AI won't create something perfect or even better. And to the extent we enjoy creating for its own sake that's still there. But it's true that real creators will have a harder time being seen by others when there's an ocean of slop being shoved down our throats by parties with endless budgets.
You are in a maze of twisty little passages that are not the same. You have to figure out which ones are not. Try this. The world is infinitely complex. AI is very good at dealing with the things it knows and can know. It can write more accurately than I can, spell better. It just takes stuff I do and learns from my mistakes. I'm good with that. But here is something to ask AI:
Name three things you cannot think about because of the language you use?
Or "why do people cook curds when making cheese."
Or how about this:
"Name three things you cannot think about because of the language you use?"
AI is at least to some extent a artificial regurgitarian. It can tell you about things that have been thought. Cool. But here is a question for you. Are there things that you can think about that have not been thought about before?
The reason people cook curds is because the goal of cheese making was to preserve milk, not to make cheese.
> The fun has been sucked out of the process of creation because nothing I make organically can compete with what AI already produces—or soon will.
So the fun, all along, was not in the process of creation itself, but in the fact that the creator could somehow feel superior to others not being able to create? I find this to be a very unhealthy relationship to creativity.
My mixer can mix dough better than I can, but I still enjoy kneading it by hand. The incredibly good artisanal bakery down the street did not reduce my enjoyment of baking, even though I cannot compete with them in quality by any measure. Modern slip casting can make superior pottery by many different quality measures, but potters enjoy throwing it on a wheel and producing unique pieces.
But if your idea of fun is tied to the "no one else can do this but me", then you've been doing it wrong before AI existed.
Yeah, I think you're onto something. I'm not sure the performative motivation is necessarily bad, but def different
Maybe AI is like Covid, where it will reveal that there were subtle differences in the underlying humans all along, but we just never realized it until something shattered the ability for ambiguity to persist.
I'm inclined to so that this is a destabilising thing, regardless of my thoughts on the "right" way to think about creativity. Multiple ways could coexist before, and now one way no longer "works".
Let's frame it more generously: The reward is based on being able to contribute something novel to the world - not because nobody else can but because it's another contribution to the world's knowledge.
If the core idea that was intended to be broadcasted to the world was a "contribution", and LLM simply expanded on it, then I would view LLMs simply a component in that broadcasting operation (just as the internet infrastructure would be), and the author's contribution would still be intact, and so should his enjoyment.
But his argument does not align with that. His argument is that he enjoys the act of writing itself. If he views his act of writing (regardless of the idea being transmitted) as his "contribution to world's knowledge", then I have to say I disagree - I don't think his writing is particularly interesting in and of itself. His ideas might be interesting (even if I disagree), but he obviously doesn't find the formation of ideas enjoyable enough.
If one merely presses a button (the same button, not choosing what button to push based on context), I don’t see what it is that one has contributed? One of those tippy bird toys can press a button.
It sounds to me like you are maybe agreeing with me but thought that I was expressing the opposite of what I did, and so are phrasing it as if it were disagreement?
Or maybe you are just agreeing, and did understand that my point was that I don’t think pressing a button is a contribution.
If you are disagreeing with my comment, can you explain how this is disagreeing?
The primary motivation should be wisdom. No one can become wise for you. You don't become any wiser yourself that way. And a machine isn't even capable of being wise.
So while AI might remove the need for human beings to engage in certain practical activities, it cannot eliminate the theoretical, because by definition, theory is done for its own sake, to benefit the person theorizing by leading them to understanding something about the world. AI can perhaps find a beneficial place here in the way books or teachers do, as guides. But in all these cases, you absolutely need to engage with the subject matter yourself to profit from it.
It sounds as if the reward is primarily monetary in this case.
As some others have commented, you can find rewards that aren't monetary to motivate you, and you can find ways to make your work so unique that people are willing to pay for it.
Technology forces us to use the creative process to more creatively monetize our work.
If that's the source of author's existential crisis, they may possibly find it interesting to meditate on the idea that there's no thinker behind the thought, and the impermanence of "self".
Even if they don't buy all the way into the whole hard incompatiblism thing, the idea is that they may find some value in the process.
I think you articulated the actual point of the OP. It isn’t so much about creating something better than anyone else, but it is a feeling that your contribution and world means something.
AI can somehow cause one to react with a feeling of futility.
Engaging in acts of creation, and responding to others acts of creation seems a way out of that feeling.
Self-actualisation should be about doing the things that only you can. Not better than anyone else, but more like, the specific things that ony you, with the same of your experience, expertise, values and constraints can do.
Knowing that what I do anyone can do, no matter how well I'll do it, is discouraging. Because then, what is my purpose? What can I say that I'm good at?
It's interesting that you name those examples, because Lee Sedol, the all-time great Go player, retired shortly after losing to Alpha Go, saying:
"Even if I become the number one, there is an entity that cannot be defeated... losing to AI, in a sense, meant my entire world was collapsing... I could no longer enjoy the game. So I retired." [1, 2]
So for some, yes. It is of course also true that many people derive self-worth and fulfillment from contributing positively to the world, and AI automating the productive work in which they specialize can undermine that.
I am in no way disputing that some people would feel that way because of AI, just as some performing classical musicians felt that way in the advent of the audio recorder.
What I am saying is that (1) I regard this as an unhealthy relationship to creativity (and I accept that this is subjective), and (2) that most people do not feel that way, as can be confirmed by the fact that chess, go, and live music performances are all still very much practiced.
That's a deeper question that only you can answer. I can only say that your thinking based on how you phrased it doesn't really lead to happiness in general
In case that would happen - welp, I'm out of job, out of something I went to school and studied for a few years, now without purpose or "real" skills (as LLMs are on the same level as I).
"The fun has been sucked out of the process of preparing food because nothing I make organically can compete with what restaurants/supermarkets already produces—or soon will."
> So the fun, all along, was not in the process of creation itself, but in the fact that the creator could somehow feel superior to others not being able to create? I find this to be a very unhealthy relationship to creativity.
People realize this at various points in their life, and some not at all.
In terms the author might accept, the metaphor of the stoic archer comes to mind. Focusing on the action, not the target is what relieves one of the disappointment of outcome. In this cast, the action is writing while the target is having better thoughts.
Much of our life is governed by the success at which we hit our targets, but why do that to oneself? We have a choice in how we approach the world, and setting our intentions toward action and away from targets is a subtle yet profound shift.
A clearer example might be someone who wants to make a friend. Let's imagine they're at a party and they go in with the intention of making a friend, they're setting themselves up for failure. They have relatively little control over that outcome. However, if they go in with the intention of showing up authentically - something people tend to appreciate, and something they have full control over - the changes of them succeeding increase dramatically.
Choosing one's goals - primarily grounded in action - is an under-appreciated perspective.
>> Focusing on the action, not the target is what relieves one of the disappointment of outcome.
The primary reason is not that it relieves us of the disappointment, but that worrying about the outcome increases our anxiety and impacts our action which hampers the outcome.
> Focusing on the action, not the target is what relieves one of the disappointment of outcome.
This is true, but the tough part is it's not the whole story.
First, obviously along some dimensions of life, targets matter. If we need to grow food to eat, the pleasant feeling of working in the garden isn't going to be sufficient; if we need to strengthen a dike to prevent the town from being inundated, the sensation of swinging a hammer isn't going to cut it.
> However, if they go in with the intention of showing up authentically - something people tend to appreciate, and something they have full control over - the changes of them succeeding increase dramatically.
That is true, but it's also possible for a person to feel like they are being authentic (and even to be correct about that), yet still seem off-putting to others, perhaps for reasons they aren't aware of. Even if they're not focused on the "target" of making a friend, there are intermediate targets like "interact with other people in a way that they (not just I) enjoy", and if those targets aren't met, eventually a reckoning must come.
So the second point is that evaluating the "action" is an internal perspective that can become out of sync with reality even in cases where the result isn't so critical. We may not want to be focused on "end goals" but we need some amount of focus on external calibrators of some sort, to keep us from descending into solipcism.
Then the third thing is that (maybe because of the first two), people have a tendency to extend their results-oriented mindset more and more, and even if an individual resists this, they have to deal with the fact that everyone around them may be doing it. So even if you take the view that writing is a human activity that should be valued for the gusto and AI writing is missing the point, if everyone around you stops writing and starts using AI instead, a lot of important stuff in the penumbra of the activity can be weakened. Like it becomes harder to put together a writing club/workshop etc., maybe even to buy books. And in particular it can become harder to straddle the line between target and action in terms of employment and generally meeting your material needs. There are plenty of people who have artistic skill and have a job where they get to use it to some extent (e.g., graphic design), and even though it may have some distasteful commercial aspects, they can still get some of that "action satisfaction" from their job. But if AI eats all the graphic design jobs, now you have to spend all your work hours doing something that gives you none of that satisfaction, and cram all the satisfying artistic action into your free time.
The same is true for technical tasks. A lot of the dismay over the use of AI for programming arises because people used to be able to get paid for doing things that also gave them a sense of satisfaction for engaging in a sort of problem-solving task that they enjoyed as an action. Now it's harder to do that, but everyone still has to eat, so they have to give up some of the satisfaction they used to get because they can't get paid for it anymore.
I agree that, for an individual, shifting the mindset to action can be helpful. But we as individuals live in the world, and the more an individual's mindset becomes out of step with that of his society, the harder it becomes to live in accordance with that mindset. So I think we also need to apply pressure to create a societal mindset that values and supports the kinds of individual mindsets we want people to have.
This is a very millennial style of thinking (myself included). It feels like people can't just have a hobby, they have to be great at it. The sense of greatness, the sense of accomplishment is not merely doing a thing, but getting to an outcome which is measurable and/or which we can tell others or put on social media. I thought it was only me, but turns out this is all around me. I started gardening, spending 15 mins a day, I talk to a friend around it. They tell me about this gardening insta page, tips, and community. The community has people doing things at a better pace / rate than me. Putting in more effort than me. I suddenly feel that rush to have some competition. Then it becomes boring because the fun was the fifteen minutes i spent in there, not the part where it occupied rest of my day. Side projects, writing, painting, I somehow see people doing this all the time. Picking the wrong goals, or expecting a dopamine hit from wrong places.
Choosing the right goals is the great way to put that in perspective. I don't know what happened with hobbies, but it's not there anymore. (so much that i dont tell people i do xyz things on the side)
Yeah, there’s something this person needs to embrace about the process rather than being some kind of modern John Henry, comparing themselves to a machine. There’s still value in the things a person creates despite what AI can derive from its training model of Reddit comments. Find peace in the process of making and you’ll continue to love it.
We need to start taking a leaf of advice from spiritual knowledge that "You are not the doer." You were never the doer. The doing happened on its own. You were merely a vessel, an instrument. A Witness. Observe your inner mechanisms of mind, and you will quickly come to this realisation.
I think the article is getting at the fact that in a post-AGI world, human skill is a depreciating asset. This is terrifying because we exchange our physical and mental labor for money. Consider this: why would a company hire me if - with enough GPU and capital - they can copy-and-paste 1,000 of AI agents much smarter to do the work?
With AGI, Knowledge workers will be worth less until they are worthless.
While I'm genuinely excited about the scientific progress AGI will bring (e.g. curing all diseases), I really hope there's a place for me in the post-AGI world. Otherwise, like the potters and bakers who can't compete in the market with cold-hard industrial machines, I'll be selling my python code base on Etsy.
This is only terrifying because of how we’ve structured society. There’s a version of the trajectory we’re on that leads to a post-scarcity society. I’m not sure we can pull that off as a species, but even if we can, it’s going to be a bumpy road.
the barrier to that version of the trajectory is that "we" haven't structured society. what structure exists, exists as a result of capital extracting as much wealth from labor as labor will allow (often by dividing class interests among labor).
agreed on the bumpy road - i don't see how we'll reach a post-scarcity society unless there is an intentional restructuring (which, many people think, would require a pretty violent paradigm shift).
I think we think of it as 'extracting' because people are coerced into jobs that they hate. I think AI can help us exit the paradigm of working as extracting. Basically, passion economy (ai handles marketing, internet distribution). Allows you to focus on what you actually like, but it can actually make money this time.
to be trite, we've been promised a world where AI will help to alleviate the menial necessities so that we're free to pursue our passions. in reality, what we're getting is AI that replaces the human component of passion projects (art, music, engineering as craft), leaving the "actually-hard-to-replace" "low-class" roles (cashiering, trash collection, housekeeping, farming, etc) to humans who generally have few other economic options.
without a dramatic shift in wealth distribution (no less than the elimination of private wealth and the profit motive), we can't have a post-scarcity society. capitalism depends entirely upon scarcity, artificial or not.
That seems like a very narrow perspective. For one, it is neither clear we will end up with AGI at all—we could have reached or soon reach a plateau with the possibilities of the LLM technology—or whether it’ll work like what you’re describing; the energy requirements might not be feasible, for example, or usage is so expensive it’s just not worth applying it to every mundane task under the sun, like writing CRUD apps in Python. We know how to build flying cars, technically, but it’s just not economically sustainable to use them. And finally, you never know what niches are going to be freed up or created by the ominous AGI machines appearing on the stage.
> With AGI, Knowledge workers will be worth less until they are worthless.
"Knowledge workers" being in charge is a recent idea that is, perhaps, reaching end of life. Up until WWII or so, society had more smart people than it had roles for them. For most of history, being strong and healthy, with a good voice and a strong personality, counted for more than being smart. To a considerable extent, it still does.
In the 1950s, C.P. Snow's "Two Cultures" became famous for pointing out that the smart people were on the way up.[1] They hadn't won yet; that was about two decades ahead. The triumph of the nerds took until the early 1990s.[2] The ultimate victory was, perhaps, the collapse of the Soviet Union in 1991. That was the last major power run by goons. That's celebrated in The End of History and the Last Man (1992).[3] Everything was going to be run by technocrats and experts from now on.
But it didn't last. Government by goons is back. Don't need to elaborate on that.
The glut of smart people will continue to grow. Over half of Americans with college educations work in jobs that don't require a college education. AI will accelerate that process. It doesn't require AI superintelligence to return smart people to the rabble. Just AI somewhat above the human average.
> With AGI, Knowledge workers will be worth less until they are worthless.
The article you've linked fundamentally relies on the assumption that "the tasks can be done better/faster/cheaper by AIs". (Plus, of course, the idea that AGI would be achieved, but without this one the whole discussion would be pointless as it would lack the subject, so I'm totally fine with this one.)
Nothing about AGI (as in "a machine that can produce intelligent thoughts on a given matter") says that human and non-human knowledge workers would have some obvious leverage over each other. Just like my coworkers' existence doesn't hurt mine, a non-human intelligence is of no inherent threat. Not by definition.
Non-intelligent industrial robotics is well-researched and generally available, yet we have plenty of sweatshops because they turn out to be cheaper than robot factories. Not fun, not great, I'm no fond of this, but I'm merely taking it as a fact, as it is how it currently is. So I really wouldn't dare to unquestionably assume that "cheaper" would be true.
And then "better" isn't obvious either. Intelligence is intelligence, it can think, it can make guesses, it can make logical conclusions, and it can make mistakes too - but we've yet to see even the tiniest hints of "higher levels" of it, something that would make humans out of the league of thinking machines if we're ranking on some "quality" of thinking.
I can only buy "faster" - and even that requires an assumption that we ignore any transhumanist ideas. But, surely, "faster" alone doesn't cut it?
I think the point is that part of the value of a work of art to this point is the effort or lack of effort involved in its creation. Evidence of effort has traditionally been a sign of the quality of thought put into a work as a product of time spent in its creation. LLMs short circuit this instinct in evaluation making some think works generated by AI are better than they are while simultaneously making those who create work see it as devaluation of work (which is the demotivator here).
I'm curious why so any people see creators and intellectuals as competitive people trying to prove they're better than someone else. This isn't why people are driven to seek knowledge or create Art. I'm sure everyone has their reasons for this, but it feels like insecurity from the outside.
Looking at debates about AI and Art outside of IP often brings out a lot of misunderstandings about what makes good Art and why Art is a thing man has been compelled to make since the beginning of the species. It takes a lifetime to select techniques and thought patterns that define a unique and authentic voice. A lifetime of working hard on creating things adds up to that voice. When you start to believe that work is in vain because the audience doesn't know the difference it certainly doesn't make it feel rewarding to do.
To put it another way: if we made a machine that could instantly create a baby, how would that effect the notion of motherhood? Sure children are adopted or born to surrogacy but the connection formed during gestation and that time itself is a huge part of our notion of the connection between mother and child. Being an Artist is the same thing, an identity bred from gestation proved by the ends.
Before the rise of Western culture, ancient cultures didn't attribute an artist to a work. Think Ancient Greece or Egypt. These cultures still produced Art because the culture valued it, but in society these creators were seen as tradesmen or they were slaves. AI used in this way both reduces cultural value and removes or reduces the social status of the creator.
I find it telling that LLMs are quite adept at mash-ups and decisions based on data analysis which in my experience is what most business managers do. Why are we not using AI to replace worthless middle management? After all they are lower skilled and higher paid than many developers. I'd argue that anyone who thinks you can replace a job with AI is not doing that job as a career. AI devs who think LLM can replace Java web developers are not Java web developers. Internet trolls who think LLM can replace Artists are not Artists. I think this moment we're in is revealing that we've become so siloed that we have lost our curiosity about each other and cultural history. It's frightening to see how we're changing our culture to accommodate a technology at the expense of people and just how blase we are about it.
> But now, when my brain spontaneously forms a tiny sliver of a potentially interesting concept or idea, I can just shove a few sloppy words into a prompt and almost instantly get a fully reasoned, researched, and completed thought. Minimal organic thinking required.
No offense, but I've found that AI outputs very polished but very average work. If I am working on something more original, it is hard to get AI to output reasoning about it without heavy explanation and guidance. And even then, it will "revert to the mean" and stumble back into a rut of familiar concepts after a few prompts. Guiding it back onto the original idea repeatedly quickly uses up context.
If an AI is able to take a sliver of an idea and output something very polished from it, then it probably wasn't that original in the first place.
458 comments
[ 0.20 ms ] story [ 156 ms ] threadToday I'm working on doing the unthinkable in an AI-world: putting together a video course that teaches developers how to use Phlex components in Rails projects and selling it for a few hundred bucks.
One way of thinking about AI is that it puts so much new information in front of people that they're going to need help from people known to have experience to navigate it all and curate it. Maybe that will become more valuable?
Who knows. That's the worst part at this moment in time—nobody really knows the depths or limits of it all. We'll see breakthroughs in some areas, and others not.
From all of my observations, the impact of LLMs on human thought quality appears largely corrosive.
I’m very glad my kid’s school has hardcore banned them. In some class they only allow students to turn in work that was done in class, under the direct observation of the teacher. There has also been a significant increase in “on paper” work vs work done on computer.
Lest you wonder “what does this guy know anyways?”, I’ll share that I grew up in a household where both parents were professors of education.
Understanding the effectiveness of different methods of learning (my dad literally taught Science Methods) were a frequent topic. Active learning (creating things using what you’re learning about) is so much more effective than passive, reception oriented methods. I think LLMs largely are supporting the latter.
I also don't think the nature of LLMs being a negative crutch is new knowledge per se; when I was in school, calculus class required a graphing calculator but the higher end models (TI-92 etc) that had symbolic equation solvers were also banned, for exactly the same reason. Having something that can give an answer for you fundamentally undermines the value of the exercise in the first place, and cripples your growth while you use it.
With LLM's giving you ready-made answers I feel like it's the same. It's not as rewarding because you haven't obtained the answer yourself. Although it did feel rewarding when I was interrogating an LLM about how CSRF works and it said I asked a great question when I asked whether it only applies to forms because it seems like fetch has a different kind of browser protection.
Contrary to what I said I actually did use dictionaries, but the point I was trying to make is rather than memorizing phrases in advance, I used it to translate something I thought I heard.
No one to day learns that anymore. The vast, vast majority have no idea and I don’t think people are dumber because of it.
That is to say, I think it’s not cut-and-dried. I agree you need to learn something, but something’s it’s okay use a tool.
I tried to encapsulate that to some degree when writing something (perhaps poorly?) recently actually - https://smcleod.net/2025/03/the-democratisation-paradox-what...
The real question isn't "is it okay to use a tool" but "how does using a tool affect what you learn".
In the cases of both LLMs and symbolic solving calculators, I believe the answer is "highly detrimental".
Pressing a calculator key doesn’t give the same insight.
Arguably, the kind of person who was helped by learning to do that by hand still learns to do it by hand, but because of curiosity rather than because a teacher told them to.
I remember being thirteen and trying to brute force methods for computing the square root. I didn’t have the tools yet to figure out how to do it in any systematic way, and the internet wasn’t at a point yet where it would have even occurred to me to just search online. Wikipedia wouldn’t exist for another two years.
I probably finally looked it up at some point in high school. I’m not sure exactly when, but I remember spending a lot of time practicing doing a few iterations in my head as a parlor trick (not that I ever had the opportunity to show it off).
If I were thirteen and curious about that now, I’d probably just ask ChatGPT. Then I’d have a whole follow up conversation about how it was derived. It would spit a lot of intimidating LaTeX at me, but unlike with Wikipedia, I’d be able to ask it to explain what those things meant.
This is the thing I don’t get when people talk about LLMs’ impact on education. Everybody focuses on cheating, like learning is inherently a chore that all students hate and must be carefully herded into doing despite themselves.
But that’s a problem with school, not learning. If your actual, self-motivated goal is to learn something, LLM’s are an incredible tool, not a hindrance.
Any school's #1 job is to motivate learning. Schools clearly suck at this.
LLMs are a fascinating effective learning tool. early learning would be better off embracing it as such.
i imagine a combination of a video of a good tutor explaining a concept followed up with an llm to quiz and explain the concept seems far better than what we have today.
The problem is that eventually you need to measure for placement, and Goodhart’s Law kicks in and destroys the enjoyment of learning. It’s very hard to be intrinsically motivated when the external pressure dominates.
The fact that “study for the test” is basically synonymous with “study” for most people is indicative that we’re doing something extremely wrong in education.
But why can't each individual have sort of their own bespoke governess tutoring them all through school? Pay Gilbert Strang a million dollars to do a 32 week course in LinAlg. Yes I know a normal course is shorter, expand it to cover everything in exquisite detail, so even I can keep up. Alan Kay, Donald Knuth, and John Carmack can teach computer science. Continue on for each subject.
Then let students go as fast as they want. I'd have finished some subjects of HS by the time I was in 9th grade, and others I might still be trying. ¯\_(ツ)_/¯
Where you'd still need to be "placed" is social development. Which I think should actually be a subject.
And agree about learning by practicing a skill being best. But you and I both know the school system has worked on rote memorisation for hundreds of years at least and still is now.
I'm surprised it was a problem in the first place. Don't equation solving exercises require you to leave intermediary steps, and you can't just put a "x=5" as a one liner answer ?
Hilarious miscalculation.
Getting exposure there is hard. Talking to friends just means more practice with what I already know but an LLM could help me practice things outside that area.
Later in the university I was studying engineering. And we were forced to prepare all the technical drawings manually in the first year of study. Like literally with pencil and ruler. Even though computer graphics were widely used and we're de facto standard.
Personally I don't believe hardcore ban will help with any sort of thing. It won't stop the progress either. It's much better to help people learn how to use things instead of forcing them to deal with "old school" stuff only.
While this is superficially similar, I believe we are talking about substantially different things.
Learning (the goal) is a process. In the case of an assignment, the resulting answer / work product, while it is what is requested, is critically not the goal. However, it is what is evaluated, so many confuse it with the goal (“I want to get a good grade”)
Anything which bypasses the process makes the goal (learning) less likely to be achieved.
So, I think it is fine to use a calculator to accelerate your use of operations you have already learned and understand.
However, I don’t think you should give 3rd graders calculators that just give them the answer to a multiplication or division when they are learning how those things work in the first place.
Similarly, I think it’s fine to do research using the internet to read sources you use to create your own work.
Meanwhile, I don’t think it’s fine to do research using the internet to find a site where you can buy a paper you can submit as your own work.
Right now, LLMs can be used to bypass a great deal of process, which is why I support them not being used.
It’s possible, maybe even likely that we’ll end up with a “supervised learning by AI” approach where the assignment is replaced by “proof of process”, a record of how the student explored the topic interactively. I could see that working if done right.
The problem is really about how to evaluate performance or incentivize students to actually work on their exercise.
So most are not curious. So what do you do for them?
Plus, many kids fail school not because of laziness, but because of their toxic environment.
Kids optimize. When I was in high school I was fully capable of getting straight F's in a class I didn't care about and straight A's in a class I enjoyed.
Why bother learning chemistry when you could instead spend that time coding cool plugins and websites in PHP that thousands of internet strangers are using? I really did build one of the most popular phpBB plugins and knew I was gonna be a software engineer. Not that my chemistry professor cared about any of that or even understood what I'm talking about.
As for what you said, yeah, I got 1s (Fs) because I was too busy coding and reading books on philosophy, as a 14 years old.
Hell, all humans do that. You use every resource available to get out of dealing with things that are not your priority. This means you will never be good at those things and that’s fine. You can’t be good at everything.
I don't disagree with you though.
But, these are kids… Hard to argue that adults should selectively deny education when it is their responsibility to do otherwise.
We don’t neglect the handicapped because it is inconvenient to provide them with assistance.
But the lazy, uncurious person that wants to shortcut everything, that you described? They're not even trying to help themselves. Maybe the best thing to do there is to let a little time pass and for them to see the result of their actions. You can lead a horse to water..
This is most kids, myself included. I’m curious and have done well, but if I could have had essays written for me as a kid for sure I’d have tried. And I’d be off worse for it.
Kids also just want to eat sugar and play video games. As parents and a society, we both provide and force alternatives because as adults we know better.
You can lead a horse to water, sure, but you could also lead it to something filled with their worst instincts. Let’s avoid the later.
Like any other tool, it's more a question of how they're used. For example, I've seen incredible results for students who use ChatGPT to interrogate ideas as they synthesize them. So, for example, "I'm reading this passage PASSAGE and I'm confused about phrase X. The core idea seems similar to Y, which I am familiar with. if I had to explain X, I'd put it like this ATTEMPT Can you help me understand what I'm missing?"
The results are very impressive. I'd encourage you to try it out if you haven't.
I try to use AI to automate things I already know and force myself to learn things I don't know.
It takes discipline/curiosity but it can be a net positive.
They should just simply try it. Start with something you actually know to see how useful it might be to you with your prompts.
“The irony is that I now know more than I ever would have before AI. But I feel slightly dumber. A bit more dull. LLMs give me finished thoughts, polished and convincing, but none of the intellectual growth that comes from developing them myself. The output from AI answers questions. It teaches me facts. But it doesn’t really help me know anything new.”
I think the thesis is that with AI there is less need and incentive to “put the work in” instead of just consuming what the AI outputs, and that in consequence we do the needed work less and atrophy.
Again, you can truly learn a lot using LLMs, but you have to approach it properly. It does not have to be just "facts", and sometimes, even learning "facts" is learning.
I can use LLMs and learn nothing, but I can use LLMs to learn, too!
The effort of verifying everything it claims may or may not outweigh the effort of other means of learning.
What does that mean, I’m curious?
The schools and university I grew up in had a “single-sanction honor code” which meant if you were caught lying or cheating even once you would be expelled. And you signed the honor code at the top of every test.
My more progressive friends at other schools who didn’t have an honor code happily poo-pooed it as a repugnantly harsh old fashioned standard. But I don’t see a better way today of enforcing “don’t use AI” in schools, than it.
I’m not sure how LLMs output is indistinguishable from Wikipedia or World Book.
Maybe? and if the question is “did the student actually write this?” (which is different than “do they understand it?” there are lots of different ways to assess if a given student understands the material…that don’t involve submitting typed text but still involve communicating clearly.
If we allow LLMs- like we allow calculators, just how poor LLMs are will become far more obvious.
Students are also poor writers. Often LLM-generated essays can be spotted in elementary school because they write too well for that grade level. A good student will surpass a chatbot, but not if they use it as a crutch while it's still a stronger writer than they are.
“Falsifying or inventing any academic work, including the use of AI (ChatGPT, etc)”
Additionally, as mentioned, the school is taking actions to change how work is done to ensure students are actually doing their own work - such as requiring written assignments be completed during class time, or giving homework on physical paper that is to be marked up by hand and returned.
Apparently this is the first year they have been doing this, as last year they had significant problems with submitted work not being authored by students.
This is in an extremely competitive Bay Area school, so there can be a lot of pressure from parents on students to make top grades, and sometimes that has negative side effects.
(Also, typing was only appropriate for only some classes anyway.)
I'm at some uni in Poland, not top tier, but at the same time - not bad either, slighly above average.
The amount of cheating I saw - it's almost mundane. Teachers know this, so do we...
And it gave students a sense of pride in their education
I have a huge respect for you and your university.
Schools will ban anything they think of as sinister.
English teachers even recommend grammarly..
Students are given a “prompt” for writing.
I wish other schools had the conviction you describe…
Students were always given a “prompt” for writing.
That’s why tech companies used that term; rather than the other way around.
What do you think "prompt" mean?
Or you're saying the students are asked to mimic AI's style?
citation needed
I mean I get the existential angst though. There's a lot of uncertainty about where all this is heading. But, and this is really a tangent, I feel that the direction of it all is in the intersection between politics, technology and human nature. I feel like "we the people" leave walkover to powerful actors if we do not use these new powerful tools in service of the people. For one - to enable new ways to coordinate and organise.
The majority of users seem to want convenience at any expense. Most are unconcerned with a loss of agency, almost enthusiastic about it if it removes the labor of thinking.
That's interesting point. But here is the thing: you are supposed to drive. Not AI god. Look at it as at an assistant whom you can interrupt, instruct, correct, ask to redo. While focusing on 'what' you can delegate it some 'how' problems.
Some of my best writing came during the time that I didn't try to publicize the content. I didn't even put my name on it. But doing that and staying interested enough to spend the hours to think and write and build takes a strange discipline. Easy for me to say as I don't know that I've had it myself.
Another way to think about it: Does AI turn you into Garry Kasparov (who kept playing chess as AI beat him) or Lee Sedol (who, at least for now, has retired from Go)?
If there's no way through this time, I'll just have to occasionally smooth out the crinkled digital copies of my past thoughts and sigh wistfully. But I don't think it's the end.
I experienced this when I was younger with my rc planes, I joined some forum and I felt like everything I did had to be posted/liked to have value. I'd post designs/fantasy and get the likes then lose interest/not actually do it after I got the ego bump
Granted, I'm blessed to not have much busywork; if I need to produce corporate docs or listicles AI would be a massive boon. But I also suspect AI will be used to digest these things back into small bullet points.
The core takeaway for me is that if you have the desire to stretch your scope as wide as possible, you can get things done in a fun way with reduced friction, and still feel like your physical being is what made the project happen. Often this means doing something that is either multidisciplinary or outside of the scope of just being behind a computer screen, which isn't everyone's desire and that's okay, too.
Like, in the recent past, someone who wanted to achieve some goal with software would either need to learn a bunch of stuff about software development, or would need to hire someone like me to bring their idea to life. But now, they can get a lot further on their own, with the support of these new tools.
I think that's good, but it's also nerve-wracking from an employment perspective. But my ultimate conclusion is that I want to work closer to the ends rather than the means.
The post laments how everything is useless when any conceivable "end state" a human can do will be inferior to what LLMs can do.
So an honest attention toward the means of how something comes about—the process of the thinking vs the polished great thought—is what life is made of.
Another comment talks about hand-made bread. People do it and enjoy it even though "making bread is a solved problem".
I think a way to square the circle is to recognize that people have different goals at different times. As a person with a family who is not independently wealthy, I care a lot about being economically productive. But I also separately care about the joy of creation.
If my goal in making a loaf of bread is economic productivity, I will be happy if I have a robot available that helps me do that quickly. But if my goal is to find joy in the act of creation, I will not use that robot because it would not achieve that goal.
I do still find joy in the act of creating software, but that was already dwindling long before chatgpt launched, and mostly what I'm doing with computers is with the goal of economic productivity.
But yeah I'll probably still create software just for the joy of it from time to time in the future, and I'm unlikely to use AIs for those projects!
But at work, I'm gonna be directing my efforts toward taking advantage of the tools available to create useful things efficiently.
An AI is _not_ going to get awarded a PhD, since by definition, such are earned by extending the boundaries of human knowledge:
https://matt.might.net/articles/phd-school-in-pictures/
So rather than accept that an LLM has been trained on whatever it is you wish to write, write something which it will need to be trained on.
Having said that I am very worried about kids growing up with AI and it stunting their critical thinking before it begins - but as of right this moment AI is extremely sub par at genuinely good ideas or writing.
It’s an amazing and useful tool I use all the time though and would struggle to be without.
The key is to treat AI as a tool, not as a magic wand that will do everything for you.
Even if AI could handle every task, leaning on it that way would mean surrendering control of your own life—and that’s never healthy.
What works for me is keeping responsibility for the big picture—what I want to achieve and how all the pieces fit together—while using AI for well-defined tasks. That way I stay fully in control, and it’s a lot more fun this way too.
The microwave is a tool with certain useful aspects and certain limitations. It is also a tool which can lead to faster outcomes of things you need to do if the tool didn’t exist. At what point should a chef draw the line in the tools they use? Should I forgo microwaves? What about pressure cookers? Ovens? Surely knives are fair game? Maybe I should knap flint and butcher meat with it and cook over an open campfire — then truly no one can claim I am not a chef.
No LLM can ever express your unique human experience (or even speak from experience), so on that axis of competition you win by default.
Regurgitating facts and the mean opinion on topics is no replacement for the thoughts of a unique human. The idea that you're competing with AI on some absolute scale of the quality of your thought is a sad way to live.
It was never a useful metric to begin with. If your life goal is to be #1 on the planet, the odds are not in your favor. And if you get there, it's almost certainly going to be unfulfilling. Who is the #1 Java programmer in the world? The #1 topologist? Do they get a lot of recognition and love?
James Gosling, of course[1]. Next question...
> The #1 topologist?
I'm not a mathematician, but... maybe Akshay Venkatesh, who won the Fields Medal in 2018?
[1] https://news.ycombinator.com/item?id=44005008
I think we will come back to roots, the simple in person creation: pen and paper, declamation, theatre, live performance, hand painting, improvisation, handmade work.
Maybe not everybody but it will be for (mentally) free people.
I used to write open source a lot but lately, I don't see the point. Not because I think LLMs can produce novel code as good code as me or will be able to in the near future. But because any time I come up with a new solution to something, it will be stolen and used without my permission, without giving me credit or without giving users the rights I give them. And it will be mangled just enough that I can't prove anything.
Large corporations were so anal about copyright that people who ever saw Microsoft's code were forbidden from contributing to FOSS alternatives like wine. But only as long as copyright suited them. Now abolishing copyright promises the C-suite even bigger rewards by getting rid of those pesky expensive programmers, if only they could just steal enough code to mix and match it with enough plausible deniability.
And so even though _in principle_ anybody using my AGPL code or anything that incorporates my AGPL code has the right to inspect and modify said code; yet now tine fractions of my AGPL code now have millions or potentially billions of users but nobody knows and nobody has the right to do anything about it.
And those who benefit the most are those who already have more money than they can spend.
Most of that 'corpus' isn't even on the Internet so it is wholly unknown to our "AI" masters.
Similarly in future we will not need mental "labor" but to keep ourselves sharp we need engage in mental exercises. I am thinking of picking up chess again just for this reason.
I still create, I just use physical materials like clay and such, to make things that AI can't yet replicate.
Name three things you cannot think about because of the language you use?
Or "why do people cook curds when making cheese."
Or how about this:
"Name three things you cannot think about because of the language you use?"
AI is at least to some extent a artificial regurgitarian. It can tell you about things that have been thought. Cool. But here is a question for you. Are there things that you can think about that have not been thought about before?
The reason people cook curds is because the goal of cheese making was to preserve milk, not to make cheese.
So the fun, all along, was not in the process of creation itself, but in the fact that the creator could somehow feel superior to others not being able to create? I find this to be a very unhealthy relationship to creativity.
My mixer can mix dough better than I can, but I still enjoy kneading it by hand. The incredibly good artisanal bakery down the street did not reduce my enjoyment of baking, even though I cannot compete with them in quality by any measure. Modern slip casting can make superior pottery by many different quality measures, but potters enjoy throwing it on a wheel and producing unique pieces.
But if your idea of fun is tied to the "no one else can do this but me", then you've been doing it wrong before AI existed.
Maybe AI is like Covid, where it will reveal that there were subtle differences in the underlying humans all along, but we just never realized it until something shattered the ability for ambiguity to persist.
I'm inclined to so that this is a destabilising thing, regardless of my thoughts on the "right" way to think about creativity. Multiple ways could coexist before, and now one way no longer "works".
But his argument does not align with that. His argument is that he enjoys the act of writing itself. If he views his act of writing (regardless of the idea being transmitted) as his "contribution to world's knowledge", then I have to say I disagree - I don't think his writing is particularly interesting in and of itself. His ideas might be interesting (even if I disagree), but he obviously doesn't find the formation of ideas enjoyable enough.
Where is the line drawn?
Is me sneezing a contribution to the world of art, since art is all about interpretation™®© and some smarmy critic will do a piece on how my sneeze is a visceral physical performative art illustrating the downfall of the modern world where technology binds us and we spend too much time inside surrounded by screens and dust and co2.
Nah, I just sneezed. That's all.
Or maybe you are just agreeing, and did understand that my point was that I don’t think pressing a button is a contribution.
If you are disagreeing with my comment, can you explain how this is disagreeing?
So while AI might remove the need for human beings to engage in certain practical activities, it cannot eliminate the theoretical, because by definition, theory is done for its own sake, to benefit the person theorizing by leading them to understanding something about the world. AI can perhaps find a beneficial place here in the way books or teachers do, as guides. But in all these cases, you absolutely need to engage with the subject matter yourself to profit from it.
As some others have commented, you can find rewards that aren't monetary to motivate you, and you can find ways to make your work so unique that people are willing to pay for it.
Technology forces us to use the creative process to more creatively monetize our work.
Even if they don't buy all the way into the whole hard incompatiblism thing, the idea is that they may find some value in the process.
AI can somehow cause one to react with a feeling of futility.
Engaging in acts of creation, and responding to others acts of creation seems a way out of that feeling.
Self-actualisation should be about doing the things that only you can. Not better than anyone else, but more like, the specific things that ony you, with the same of your experience, expertise, values and constraints can do.
So for some, yes. It is of course also true that many people derive self-worth and fulfillment from contributing positively to the world, and AI automating the productive work in which they specialize can undermine that.
[1] https://en.yna.co.kr/view/AEN20191127004800315
[2] https://www.nytimes.com/2024/07/10/world/asia/lee-saedol-go-...
What I am saying is that (1) I regard this as an unhealthy relationship to creativity (and I accept that this is subjective), and (2) that most people do not feel that way, as can be confirmed by the fact that chess, go, and live music performances are all still very much practiced.
In case that would happen - welp, I'm out of job, out of something I went to school and studied for a few years, now without purpose or "real" skills (as LLMs are on the same level as I).
People realize this at various points in their life, and some not at all.
In terms the author might accept, the metaphor of the stoic archer comes to mind. Focusing on the action, not the target is what relieves one of the disappointment of outcome. In this cast, the action is writing while the target is having better thoughts.
Much of our life is governed by the success at which we hit our targets, but why do that to oneself? We have a choice in how we approach the world, and setting our intentions toward action and away from targets is a subtle yet profound shift.
A clearer example might be someone who wants to make a friend. Let's imagine they're at a party and they go in with the intention of making a friend, they're setting themselves up for failure. They have relatively little control over that outcome. However, if they go in with the intention of showing up authentically - something people tend to appreciate, and something they have full control over - the changes of them succeeding increase dramatically.
Choosing one's goals - primarily grounded in action - is an under-appreciated perspective.
The primary reason is not that it relieves us of the disappointment, but that worrying about the outcome increases our anxiety and impacts our action which hampers the outcome.
This is true, but the tough part is it's not the whole story.
First, obviously along some dimensions of life, targets matter. If we need to grow food to eat, the pleasant feeling of working in the garden isn't going to be sufficient; if we need to strengthen a dike to prevent the town from being inundated, the sensation of swinging a hammer isn't going to cut it.
> However, if they go in with the intention of showing up authentically - something people tend to appreciate, and something they have full control over - the changes of them succeeding increase dramatically.
That is true, but it's also possible for a person to feel like they are being authentic (and even to be correct about that), yet still seem off-putting to others, perhaps for reasons they aren't aware of. Even if they're not focused on the "target" of making a friend, there are intermediate targets like "interact with other people in a way that they (not just I) enjoy", and if those targets aren't met, eventually a reckoning must come.
So the second point is that evaluating the "action" is an internal perspective that can become out of sync with reality even in cases where the result isn't so critical. We may not want to be focused on "end goals" but we need some amount of focus on external calibrators of some sort, to keep us from descending into solipcism.
Then the third thing is that (maybe because of the first two), people have a tendency to extend their results-oriented mindset more and more, and even if an individual resists this, they have to deal with the fact that everyone around them may be doing it. So even if you take the view that writing is a human activity that should be valued for the gusto and AI writing is missing the point, if everyone around you stops writing and starts using AI instead, a lot of important stuff in the penumbra of the activity can be weakened. Like it becomes harder to put together a writing club/workshop etc., maybe even to buy books. And in particular it can become harder to straddle the line between target and action in terms of employment and generally meeting your material needs. There are plenty of people who have artistic skill and have a job where they get to use it to some extent (e.g., graphic design), and even though it may have some distasteful commercial aspects, they can still get some of that "action satisfaction" from their job. But if AI eats all the graphic design jobs, now you have to spend all your work hours doing something that gives you none of that satisfaction, and cram all the satisfying artistic action into your free time.
The same is true for technical tasks. A lot of the dismay over the use of AI for programming arises because people used to be able to get paid for doing things that also gave them a sense of satisfaction for engaging in a sort of problem-solving task that they enjoyed as an action. Now it's harder to do that, but everyone still has to eat, so they have to give up some of the satisfaction they used to get because they can't get paid for it anymore.
I agree that, for an individual, shifting the mindset to action can be helpful. But we as individuals live in the world, and the more an individual's mindset becomes out of step with that of his society, the harder it becomes to live in accordance with that mindset. So I think we also need to apply pressure to create a societal mindset that values and supports the kinds of individual mindsets we want people to have.
Choosing the right goals is the great way to put that in perspective. I don't know what happened with hobbies, but it's not there anymore. (so much that i dont tell people i do xyz things on the side)
With AGI, Knowledge workers will be worth less until they are worthless.
While I'm genuinely excited about the scientific progress AGI will bring (e.g. curing all diseases), I really hope there's a place for me in the post-AGI world. Otherwise, like the potters and bakers who can't compete in the market with cold-hard industrial machines, I'll be selling my python code base on Etsy.
No Set Gauge had an excellent blog post about this. Have a read if you want a dash of existential dread for the weekend: https://www.nosetgauge.com/p/capital-agi-and-human-ambition.
agreed on the bumpy road - i don't see how we'll reach a post-scarcity society unless there is an intentional restructuring (which, many people think, would require a pretty violent paradigm shift).
without a dramatic shift in wealth distribution (no less than the elimination of private wealth and the profit motive), we can't have a post-scarcity society. capitalism depends entirely upon scarcity, artificial or not.
I wouldn’t worry too much yet.
"Knowledge workers" being in charge is a recent idea that is, perhaps, reaching end of life. Up until WWII or so, society had more smart people than it had roles for them. For most of history, being strong and healthy, with a good voice and a strong personality, counted for more than being smart. To a considerable extent, it still does.
In the 1950s, C.P. Snow's "Two Cultures" became famous for pointing out that the smart people were on the way up.[1] They hadn't won yet; that was about two decades ahead. The triumph of the nerds took until the early 1990s.[2] The ultimate victory was, perhaps, the collapse of the Soviet Union in 1991. That was the last major power run by goons. That's celebrated in The End of History and the Last Man (1992).[3] Everything was going to be run by technocrats and experts from now on.
But it didn't last. Government by goons is back. Don't need to elaborate on that.
The glut of smart people will continue to grow. Over half of Americans with college educations work in jobs that don't require a college education. AI will accelerate that process. It doesn't require AI superintelligence to return smart people to the rabble. Just AI somewhat above the human average.
[1] https://en.wikipedia.org/wiki/The_Two_Cultures
[2] https://archive.org/details/triumph_of_the_nerds
[3] https://en.wikipedia.org/wiki/The_End_of_History_and_the_Las...
The article you've linked fundamentally relies on the assumption that "the tasks can be done better/faster/cheaper by AIs". (Plus, of course, the idea that AGI would be achieved, but without this one the whole discussion would be pointless as it would lack the subject, so I'm totally fine with this one.)
Nothing about AGI (as in "a machine that can produce intelligent thoughts on a given matter") says that human and non-human knowledge workers would have some obvious leverage over each other. Just like my coworkers' existence doesn't hurt mine, a non-human intelligence is of no inherent threat. Not by definition.
Non-intelligent industrial robotics is well-researched and generally available, yet we have plenty of sweatshops because they turn out to be cheaper than robot factories. Not fun, not great, I'm no fond of this, but I'm merely taking it as a fact, as it is how it currently is. So I really wouldn't dare to unquestionably assume that "cheaper" would be true.
And then "better" isn't obvious either. Intelligence is intelligence, it can think, it can make guesses, it can make logical conclusions, and it can make mistakes too - but we've yet to see even the tiniest hints of "higher levels" of it, something that would make humans out of the league of thinking machines if we're ranking on some "quality" of thinking.
I can only buy "faster" - and even that requires an assumption that we ignore any transhumanist ideas. But, surely, "faster" alone doesn't cut it?
I'm curious why so any people see creators and intellectuals as competitive people trying to prove they're better than someone else. This isn't why people are driven to seek knowledge or create Art. I'm sure everyone has their reasons for this, but it feels like insecurity from the outside.
Looking at debates about AI and Art outside of IP often brings out a lot of misunderstandings about what makes good Art and why Art is a thing man has been compelled to make since the beginning of the species. It takes a lifetime to select techniques and thought patterns that define a unique and authentic voice. A lifetime of working hard on creating things adds up to that voice. When you start to believe that work is in vain because the audience doesn't know the difference it certainly doesn't make it feel rewarding to do.
Before the rise of Western culture, ancient cultures didn't attribute an artist to a work. Think Ancient Greece or Egypt. These cultures still produced Art because the culture valued it, but in society these creators were seen as tradesmen or they were slaves. AI used in this way both reduces cultural value and removes or reduces the social status of the creator.
I find it telling that LLMs are quite adept at mash-ups and decisions based on data analysis which in my experience is what most business managers do. Why are we not using AI to replace worthless middle management? After all they are lower skilled and higher paid than many developers. I'd argue that anyone who thinks you can replace a job with AI is not doing that job as a career. AI devs who think LLM can replace Java web developers are not Java web developers. Internet trolls who think LLM can replace Artists are not Artists. I think this moment we're in is revealing that we've become so siloed that we have lost our curiosity about each other and cultural history. It's frightening to see how we're changing our culture to accommodate a technology at the expense of people and just how blase we are about it.
No offense, but I've found that AI outputs very polished but very average work. If I am working on something more original, it is hard to get AI to output reasoning about it without heavy explanation and guidance. And even then, it will "revert to the mean" and stumble back into a rut of familiar concepts after a few prompts. Guiding it back onto the original idea repeatedly quickly uses up context.
If an AI is able to take a sliver of an idea and output something very polished from it, then it probably wasn't that original in the first place.