To his point: personally, I find it shifts 'where and when' I have to deal with the 'cognitive load'. I've noticed (at times) feeling more impatient, that I tend to skim the results more often, and that it takes a bit more mental energy to maintain my attention..
This list of things not to use AI for is so quaint. There's a story on the front page right now from The Atlantic: "Film students who can no longer sit through films". But why? Aren't they using social media, YouTube, Netflix, etc responsibly? Surely they know the risks, and surely people will be just as responsible with AI, even given the enormous economic and professional pressures to be irresponsible.
The “lump of cognition” framing misses something important. it’s not about how much thinking we do, but which thinking we stop doing. A lot of judgment, ownership, and intuition comes from boring or repetitive work, and outsourcing that isn’t free. Lowering the cost of producing words clearly isn’t the same as increasing the amount of actual thought.
My fundamental argument: The way the average person is using AI today is as "Thinking as a Service" and this is going to have absolutely devastating long term consequences, training an entire generation not to think for themselves.
Thinking developed naturally as a tool that helps our species to stay dominant on the planet, at least on land. (Not by biomass but by the ability to control.)
If outsourcing thought is beneficial, those who practice it will thrive; if not, they will eventually cease to practice it, one way or another.
Thought, as any other tool, is useful when it solves more problems than it creates. For instance, an ability to move very fast may be beneficial if it gets you where you want to be, and detrimental, if it misses the destination often enough, and badly enough. Similarly, if outsourced intellectual activities miss the mark often enough, and badly enough, the increased speed is not very helpful.
I suspect that the best results would be achieved by outsourcing relatively small intellectual acts in a way that guarantees very rare, very small errors. That is, AI will become useful when AI becomes dependable, comparable to our other tools.
The interesting axis here isn’t how much cognition we outsource, it’s how reversible the outsourcing is. Using an LLM as a scratchpad (like a smarter calculator or search engine) is very different from letting it quietly shape your writing, decisions, and taste over years. That’s the layer where tacit knowledge and identity live, and it’s hard to get back once the habit forms.
We already saw a softer version of this with web search and GPS: people didn’t suddenly forget how to read maps, but schools and orgs stopped teaching it, and now almost nobody plans a route without a blue dot. I suspect we’ll see the same with writing and judgment: the danger isn’t that nobody thinks, it’s that fewer people remember how.
I still read the LLMs output quite critically and I cringe whenever I do. LLMs are just plain wrong a lot of the time. They’re just not very intelligent. They’re great at pretending to be intelligent. They imitate intelligence. That is all they do. And I can see it every single time I interact with them. And it terrifies me that others aren’t quite as objective.
A lot of this stuff depends on how a person chooses to engage, but my contrarian take is that actually throughout history whenever anyone said X technology will lead to the downfall of humanity for y reasons, that take was usually correct.
The article he references gives this example:
“Is it lazy to watch a movie instead of making up a story in your head?”
Yes, yes it is, this was a worry when we transitioned from oral culture to written culture, and I think it was probably prescient.
For many if not most people cultural or technological expectations around what skills you _have_ to learn probably have an impact on total capability. We probably lost something when Google Maps came out and the average person didn’t have to learn to read a map.
When we transitioned from paper and evening news to 24 hour partisan cable news, I think more people outsourced their political opinions to those channels.
How many of you know how to do home improvement? Fix your own clothes? Grow your own food? Cook your own food? How about making a fire or shelter? People used to know all of those things. Now they don't, but we seem to be getting along in life fine anyway. Sure we're all frightened by the media at the dangers lurking from not knowing more, but actually our lives are fine.
The things that are actually dangerous in our lives? Not informing ourselves enough about science, politics, economics, history, and letting angry people lead us astray. Nobody writes about that. Instead they write about spooky things that can't be predicted and shudder. It's easier to wonder about future uncertainty than deal with current certainty.
What I am worried about (and it's something about regular internet search that has worried me for the past ~10 years or so) is that, after they've trained a generation of folks to rely on this tech, they're going to start inserting things into the training data (or whatever the method would be) to bias it towards favoring certain agendas wrt the information it presents to the users in response to their queries.
Ever since Google experimented LLM in Gmail it bothers me alot. I firmly believe every word and the way you put them together portrays who you are. Using LLM for direct communication is harmful to human connections.
IMHO, the real problem is that they create an even greater dissonance between online life and IRL.
Think about dating apps, pictures could be fake, and now words exchanged can be fake too.
You thought you were arguing with a gentle and smart colleague by chat and mails, too bad, when you meet then at a conference or at a restaurant you find them very unpleasant.
One perspective I’m circling right now about this topic is that maybe we’re coming to realize as a society that what we considered intelligence (or symbolic intelligence whatever you wanna call that thing that we measure with traditional IQ tests, verbal fluency, etc) is actually a far less essential cognitive aspect to us as humans then we had previously assumed and is in fact, far more mechanical in nature than we had formerly believed.
This ties with how I sometimes describe current generation AI as a form of mechanized intelligence: like Babbage’s calculating machine, but scaled up to be able to represent all kinds of classes of things.
And in this perspective that I’m circling these days where I’m currently coming down on it is maybe the effect of this realization will be something like the dichotomy outlined in the Dune series: namely, that between mechanized intelligence embodied by mentats and the more intuitive and prescient aspects of cognition embodied by the Benni Jessarit and Paul’s lineage.
A simple but direct way to describe this transition in perspective may be that we come to see what we formally thought of as intelligence in the West/reductive tradition as a form of mechanized calculation that it’s possible to outsource to automatic non-biological processes, and we start to lean in more deeply to the more intuitive and prescient aspects of cognition.
One thing I’m reminded of is how Indian yogic texts describe various aspects of mind.
I’m not sure if it’s a one-to-one mapping because I’m not across that material but merely the idea of distinguishing between different aspects of mind is something with precedent; and central to that is the idea of removing association between self identity and the aspects of mind.
And so maybe one of the effects for us as a society will be something akin to that.
One bothersome aspect of generative assistance for personal and public communication not mentioned is that it introduces a lazy hedge, where a person can always claim that "Oh, but that was not really what I meant" or "Oh, but I would not express myself in that way" - and use it as a tool to later modify or undo their positions - effectively reducing honesty instead of increasing it.
This is something I noticed myself. I let AI handle some of my project and later realized I didn't even understand my own project well enough to make decisions about it :)
The author says it's too long. So let's tighten it up.
A criticism of the use of large language models (LLMs) is that it can deprive us of cognitive skills.
Are some kinds of use are better than others? Andy Masley's blog says "thinking often leads to more things to think about", so we shouldn't worry about letting machines do the thinking for us — we will be able to think about other things.
My aim is not to refute all his arguments, but to highlight issues with "outsourcing thinking".
Masley writes that it's "bad to outsource your cognition when it:"
- Builds tacit knowledge you'll need in future.
- Is an expression of care for someone else.
- Is a valuable experience on its own.
- Is deceptive to fake.
- Is focused in a problem that is deathly important to get right, and where you don't totally trust who you're outsourcing it to.
How we choose to use chatbots is about how we want our lives and society to be.
That's what he has to say. Plus some examples, which help make the message concrete. It's a useful article if edited properly.
36 comments
[ 2.7 ms ] story [ 42.9 ms ] threadTo his point: personally, I find it shifts 'where and when' I have to deal with the 'cognitive load'. I've noticed (at times) feeling more impatient, that I tend to skim the results more often, and that it takes a bit more mental energy to maintain my attention..
https://gwern.net/doc/fiction/science-fiction/2012-10-03-yva...
My fundamental argument: The way the average person is using AI today is as "Thinking as a Service" and this is going to have absolutely devastating long term consequences, training an entire generation not to think for themselves.
If outsourcing thought is beneficial, those who practice it will thrive; if not, they will eventually cease to practice it, one way or another.
Thought, as any other tool, is useful when it solves more problems than it creates. For instance, an ability to move very fast may be beneficial if it gets you where you want to be, and detrimental, if it misses the destination often enough, and badly enough. Similarly, if outsourced intellectual activities miss the mark often enough, and badly enough, the increased speed is not very helpful.
I suspect that the best results would be achieved by outsourcing relatively small intellectual acts in a way that guarantees very rare, very small errors. That is, AI will become useful when AI becomes dependable, comparable to our other tools.
We already saw a softer version of this with web search and GPS: people didn’t suddenly forget how to read maps, but schools and orgs stopped teaching it, and now almost nobody plans a route without a blue dot. I suspect we’ll see the same with writing and judgment: the danger isn’t that nobody thinks, it’s that fewer people remember how.
Notice something subtle.
Early inventions extend coordination. Middle inventions extend memory. Later inventions extend reasoning. The latest inventions extend agency.
This suggests that human history is less about tools and more about outsourcing parts of the mind into the world.
The article he references gives this example:
“Is it lazy to watch a movie instead of making up a story in your head?”
Yes, yes it is, this was a worry when we transitioned from oral culture to written culture, and I think it was probably prescient.
For many if not most people cultural or technological expectations around what skills you _have_ to learn probably have an impact on total capability. We probably lost something when Google Maps came out and the average person didn’t have to learn to read a map.
When we transitioned from paper and evening news to 24 hour partisan cable news, I think more people outsourced their political opinions to those channels.
The things that are actually dangerous in our lives? Not informing ourselves enough about science, politics, economics, history, and letting angry people lead us astray. Nobody writes about that. Instead they write about spooky things that can't be predicted and shudder. It's easier to wonder about future uncertainty than deal with current certainty.
Think about dating apps, pictures could be fake, and now words exchanged can be fake too.
You thought you were arguing with a gentle and smart colleague by chat and mails, too bad, when you meet then at a conference or at a restaurant you find them very unpleasant.
This ties with how I sometimes describe current generation AI as a form of mechanized intelligence: like Babbage’s calculating machine, but scaled up to be able to represent all kinds of classes of things.
And in this perspective that I’m circling these days where I’m currently coming down on it is maybe the effect of this realization will be something like the dichotomy outlined in the Dune series: namely, that between mechanized intelligence embodied by mentats and the more intuitive and prescient aspects of cognition embodied by the Benni Jessarit and Paul’s lineage.
A simple but direct way to describe this transition in perspective may be that we come to see what we formally thought of as intelligence in the West/reductive tradition as a form of mechanized calculation that it’s possible to outsource to automatic non-biological processes, and we start to lean in more deeply to the more intuitive and prescient aspects of cognition.
One thing I’m reminded of is how Indian yogic texts describe various aspects of mind.
I’m not sure if it’s a one-to-one mapping because I’m not across that material but merely the idea of distinguishing between different aspects of mind is something with precedent; and central to that is the idea of removing association between self identity and the aspects of mind.
And so maybe one of the effects for us as a society will be something akin to that.
A criticism of the use of large language models (LLMs) is that it can deprive us of cognitive skills. Are some kinds of use are better than others? Andy Masley's blog says "thinking often leads to more things to think about", so we shouldn't worry about letting machines do the thinking for us — we will be able to think about other things.
My aim is not to refute all his arguments, but to highlight issues with "outsourcing thinking".
Masley writes that it's "bad to outsource your cognition when it:"
- Builds tacit knowledge you'll need in future.
- Is an expression of care for someone else.
- Is a valuable experience on its own.
- Is deceptive to fake.
- Is focused in a problem that is deathly important to get right, and where you don't totally trust who you're outsourcing it to.
How we choose to use chatbots is about how we want our lives and society to be.
That's what he has to say. Plus some examples, which help make the message concrete. It's a useful article if edited properly.
Apparently he think of planning a vacation as some artistic expression.