I've always had the feeling that AI researchers want to build their own human without having to change diapers being part of the process. Just skip to adulthood please, and learn to drive a car without having experience in bumping into things and hurting yourself.
> Language doesn't just describe reality; it creates it.
I wonder if this is a statement from the discussed paper or from the blog author. Haven't found the original paper yet, but this blog post very much makes me want to read it.
> The primary counterargument can be framed in terms of Rich Sutton's famous essay, "The Bitter Lesson," which argues that the entire history of AI has taught us that attempts to build in human-like cognitive structures (like embodiment) are always eventually outperformed by general methods that just leverage massive-scale computation
This reminds me Douglas Hofstadter, of the Gödel, Escher, Bach fame. He rejected all of this statistical approaches towards creating intelligence and dug deep into the workings of human mind [1]. Often, in the most eccentric ways possible.
> ... he has bookshelves full of these notebooks. He pulls one down—it’s from the late 1950s. It’s full of speech errors. Ever since he was a teenager, he has captured some 10,000 examples of swapped syllables (“hypodeemic nerdle”), malapropisms (“runs the gambit”), “malaphors” (“easy-go-lucky”), and so on, about half of them committed by Hofstadter himself.
>
> For Hofstadter, they’re clues. “Nobody is a very reliable guide concerning activities in their mind that are, by definition, subconscious,” he once wrote. “This is what makes vast collections of errors so important. In an isolated error, the mechanisms involved yield only slight traces of themselves; however, in a large collection, vast numbers of such slight traces exist, collectively adding up to strong evidence for (and against) particular mechanisms.”
I don't know when, where, and how the next leap in AGI will come through, but it's just very likely, it will be through brute-force computation (unfortunately). So much for fifty years of observing Freudian slips.
This article seems to fall straight into the trap it aims to warn us about. All this talk about "true" understanding, embodiment, etc. is needless antropomorphizing.
A much better framework for thinking about intelligence is simply as the ability to make predictions about the world (including conditional ones like "what will happen if we take this action"). Whether it's achieved through "true understanding" (however you define it; I personally doubt you can) or "mimicking" bears no relevance for most of the questions about the impact of AI we are trying to answer.
I think the Stochastic Parrots idea is pretty outdated and incorrect. LLMs are not parrots, we don't even need them to parrot, we already have perfect copying machines. LLMs are working on new things, that is their purpose, reproducing the same thing we already have is not worth it.
The core misconception here is that LLMs are autonomous agents parroting away. No, they are connected to humans, tools, reference data, and validation systems. They are in a dialogue, and in a dialogue you quickly get into a place where nobody has ever been before. Take any 10 consecutive words from a human or LLM and chances are nobody on the internet stringed those words the same way before.
LLMs are more like pianos than parrots, or better yet, like another musician jamming together with you, creating something together that none would do individually. We play our prompts on the keyboard and they play their "music" back to us. Good or bad - depends on the player at the keyboard, they retain most control. To say LLMs are Stochastic Parrots is to discount the contribution of the human using it.
Related to intelligence, I think we have a misconception that it comes from the brain. No, it comes from the feedback loop between brain and environment. The environment plays a huge role in exploration, learning, testing ideas, and discovery. The social aspect also plays a big role, parallelizing exploration and streamlining exploitation of discoveries. We are not individually intelligent, it is a social, environment based process, not a pure-brain process.
Searching for intelligence in the brain is like searching for art in the paint pigments and canvas cloth.
> But that still leaves a crucial question: can we develop a more precise, less anthropomorphic vocabulary to describe AI capabilities? Or is our human-centric language the only tool we have to reason about these new forms of intelligence, with all the baggage that entails?
I don't get the problem with this really. I think LLM's "reasoning" is a very fair and proper way to call it. It takes time and spits out tokens that it recursively uses to get a much better output than it otherwise would have. Is it actually really reasoning using a brain like a human would? No. But it is close enough so I don't see the problem calling it "reasoning". What's the fuss about?
The problem is fuzzy language can make debate poor and about the definition of words rather than about reality. The answer I think it to avoid that and find things that you can be clear about. A famous example is the Turing test. Rather than debates on whether machines can think getting bogged down in endless variation of how people define thinking, Turing looked at if the machines could be told apart from humans which he discussed in his paper.
I would add a fifth fallacy: assuming what we humans do can be reduced to “intelligence”. We are actually very irrational. Humans are driven strongly by Will, Desire, Love, Faith, and many other irrational traits. Has an LLM ever demonstrated irrational love? Or sexual desire? How can it possibly do what humans do without these?
For all its advanced capabilities, the LLM remains a glorified natural language interface. It is exceptionally good at conversational communication and synthesizing existing knowledge, making information more accessible and in some cases, easier to interact with. However, many of the more ambitious applications, such as so-called "agents," are not a sign of nascent intelligence. They are simply sophisticated workflows—complex combinations of Python scripts and chained API calls that leverage the LLM as a sub-routine. These systems are clever, but they are not a leap towards true artificial agency. We must be cautious not to confuse a powerful statistical tool with the dawn of genuine machine consciousness.
I don't understand why people can't handle metaphors to explain things in AI so much.
The same terms exist in other fields. Physics has things that want to go to a lower energy level, the ball wants to fall but the table is holding it up. Electrons don't like being near each other, The hugs boson puts on little bunny ears and goes around giving mass to all the other good particles.
None of these are said in any way as a suggestion that these things have any form of intention.
They also don't in AI. When scientists really think those abilities are there in a provable way (or even if they suspect), I can assure you that they will be prepared to make it crystal clear that this is what they are claiming. Critisising the use of metaphor is kind of a pre-emptive attack against claims that might be made in the future.
Some AI scientists believe that there is a degree of awareness in recent models. They may be right or wrong but the ones who believe this are outright saying so.
I'm also inclined, if you'll excuse the term, to be critical of anything suggesting the assumption of smooth progress when they declare something to be the first step. Steps are not smooth. That's a good example of ignoring the what of the metaphor.
I don't really know what to make about the embodiment position, it feels like it's trying to hide dualism behind a practical limitation. Once you start drilling down into the why/why not and what do you mean by that, I wouldn't be at all surprised to see the expectation that you can't train an AI because it doesn't have a soul
> The primary counterargument can be framed in terms of Rich Sutton's famous essay, "The Bitter Lesson," which argues that the entire history of AI has taught us that attempts to build in human-like cognitive structures (like embodiment) are always eventually outperformed by general methods that just leverage massive-scale computation.
That's not what it says, but that hand-made heuristics are defeated by general methods. There is no reason why the same methods should not perform even better when informed by data through interacting with the world.
>...the most important fallacy. It's the deep-seated assumption that intelligence is, like software, a form of pure information processing that can be separated from its body.
I think he gets into a muddle on that one. If something online can provide smarter thinking and answers to questions than I can then I figure it's intelligent and it doesn't matter if it's an LLM, a human or a disembodied spirit that somehow happens to be online.
He kind of gets that from human minds not being disembodied from their brains but that's a different thing.
It’s true that much of the debate around AI swings between extremes — utopian promises on one side, dystopian collapse on the other. But institutions don’t operate well in extremes.
What matters is how we design governance that acknowledges uncertainty while still enabling progress. In practice, that means imperfect but adaptive frameworks — guardrails that evolve as technology and society evolve.
Instead of asking “which fallacy is right,” we might ask: how do we build systems that remain trustworthy even when our assumptions about AI turn out to be wrong?
22 comments
[ 4.5 ms ] story [ 45.2 ms ] threadSomeone should let Waymo, Zoox, Pony.ai, Apollo Go, and even Tesla know!
> Language doesn't just describe reality; it creates it.
I wonder if this is a statement from the discussed paper or from the blog author. Haven't found the original paper yet, but this blog post very much makes me want to read it.
This reminds me Douglas Hofstadter, of the Gödel, Escher, Bach fame. He rejected all of this statistical approaches towards creating intelligence and dug deep into the workings of human mind [1]. Often, in the most eccentric ways possible.
> ... he has bookshelves full of these notebooks. He pulls one down—it’s from the late 1950s. It’s full of speech errors. Ever since he was a teenager, he has captured some 10,000 examples of swapped syllables (“hypodeemic nerdle”), malapropisms (“runs the gambit”), “malaphors” (“easy-go-lucky”), and so on, about half of them committed by Hofstadter himself.
>
> For Hofstadter, they’re clues. “Nobody is a very reliable guide concerning activities in their mind that are, by definition, subconscious,” he once wrote. “This is what makes vast collections of errors so important. In an isolated error, the mechanisms involved yield only slight traces of themselves; however, in a large collection, vast numbers of such slight traces exist, collectively adding up to strong evidence for (and against) particular mechanisms.”
I don't know when, where, and how the next leap in AGI will come through, but it's just very likely, it will be through brute-force computation (unfortunately). So much for fifty years of observing Freudian slips.
[1]: https://www.theatlantic.com/magazine/archive/2013/11/the-man...
A much better framework for thinking about intelligence is simply as the ability to make predictions about the world (including conditional ones like "what will happen if we take this action"). Whether it's achieved through "true understanding" (however you define it; I personally doubt you can) or "mimicking" bears no relevance for most of the questions about the impact of AI we are trying to answer.
Question for the author: how are SOTA LLM models not common sense machines?
The core misconception here is that LLMs are autonomous agents parroting away. No, they are connected to humans, tools, reference data, and validation systems. They are in a dialogue, and in a dialogue you quickly get into a place where nobody has ever been before. Take any 10 consecutive words from a human or LLM and chances are nobody on the internet stringed those words the same way before.
LLMs are more like pianos than parrots, or better yet, like another musician jamming together with you, creating something together that none would do individually. We play our prompts on the keyboard and they play their "music" back to us. Good or bad - depends on the player at the keyboard, they retain most control. To say LLMs are Stochastic Parrots is to discount the contribution of the human using it.
Related to intelligence, I think we have a misconception that it comes from the brain. No, it comes from the feedback loop between brain and environment. The environment plays a huge role in exploration, learning, testing ideas, and discovery. The social aspect also plays a big role, parallelizing exploration and streamlining exploitation of discoveries. We are not individually intelligent, it is a social, environment based process, not a pure-brain process.
Searching for intelligence in the brain is like searching for art in the paint pigments and canvas cloth.
I don't get the problem with this really. I think LLM's "reasoning" is a very fair and proper way to call it. It takes time and spits out tokens that it recursively uses to get a much better output than it otherwise would have. Is it actually really reasoning using a brain like a human would? No. But it is close enough so I don't see the problem calling it "reasoning". What's the fuss about?
they don't need to reach equal human intelligence, the just need to reach an acceptable of intelligence so corporation can reduce labor cost
sure it bad at certain things but you know what ??? most of real world job didn't need a genius either
That is also a fallacy from being too immersed in a professional environment filled with deep reasoning and a deep rooted tradition of logic.
In the greater human civilization you will find an abundance of individuals lacking both reasoning and common sense.
The same terms exist in other fields. Physics has things that want to go to a lower energy level, the ball wants to fall but the table is holding it up. Electrons don't like being near each other, The hugs boson puts on little bunny ears and goes around giving mass to all the other good particles.
None of these are said in any way as a suggestion that these things have any form of intention.
They also don't in AI. When scientists really think those abilities are there in a provable way (or even if they suspect), I can assure you that they will be prepared to make it crystal clear that this is what they are claiming. Critisising the use of metaphor is kind of a pre-emptive attack against claims that might be made in the future.
Some AI scientists believe that there is a degree of awareness in recent models. They may be right or wrong but the ones who believe this are outright saying so.
I'm also inclined, if you'll excuse the term, to be critical of anything suggesting the assumption of smooth progress when they declare something to be the first step. Steps are not smooth. That's a good example of ignoring the what of the metaphor.
I don't really know what to make about the embodiment position, it feels like it's trying to hide dualism behind a practical limitation. Once you start drilling down into the why/why not and what do you mean by that, I wouldn't be at all surprised to see the expectation that you can't train an AI because it doesn't have a soul
I agree with xkcd 1425 though.
That's not what it says, but that hand-made heuristics are defeated by general methods. There is no reason why the same methods should not perform even better when informed by data through interacting with the world.
I think he gets into a muddle on that one. If something online can provide smarter thinking and answers to questions than I can then I figure it's intelligent and it doesn't matter if it's an LLM, a human or a disembodied spirit that somehow happens to be online.
He kind of gets that from human minds not being disembodied from their brains but that's a different thing.
What matters is how we design governance that acknowledges uncertainty while still enabling progress. In practice, that means imperfect but adaptive frameworks — guardrails that evolve as technology and society evolve.
Instead of asking “which fallacy is right,” we might ask: how do we build systems that remain trustworthy even when our assumptions about AI turn out to be wrong?