I think it's meaningless anyway. A calculator doesn't multiply numbers like a human does. The important part is to develop systems that can do many human tasks
> When artificial systems produce human-like language, people may draw a reverse inference: if LLMs can speak like humans, perhaps humans think like LLMs.
I think I experienced this when I learned about LLMs, chain of thought, thinking tokens, short-term memory context, and long-term memory context. I began applying these concepts to real life and reasoning about how our brains work as if these concepts described how our brains actually function. But maybe this is more akin to the Tetris effect?
This paper introduces a term and instantly defines it as a definitely biased thing that is definitely happening, then spends its entirety arguing against the strawman it built itself. Not a single sentence is spent actually arguing with the idea or any of its points (other than the “partial similarities” paragraph on page I just realized the pages aren’t even numbered).
In general, the terms “LLM-like” and “human-like” are used all over the place, and in contrast with each other, but they’re never actually defined. It all just seems more vibes-based than anything else.
And “treating the human cognitive process like it’s similar to the LLM cognitive process might lead to a society where epistemics turns into a discipline where plausibility is an acceptable substitute for empiricism” has got to be one of the most ridiculous notions I’ve ever read in a paper (ctrl+F “fifth pathway is epistemic” for the exact quote).
Nothing new under the sun. When clocks and precision mechanics started in the 17th century, there was a tendency to view humans as "machines". Computers came, suddenly human brains are "computers". Now we're LLMs.
If scientists make green jelly that emits thoughtful judgements, humans will be compared to green jelly.
Looks like he mostly publish something about "social behavior".
This "paper", IMO, is just saying "Hey, I notice this is happening. This is why it could be interesting for social science researchers" with without any real research or result.
The author lightly touches on other ways humans have viewed cognition, “computationalism” as one, but somewhat brushes these aside as though LLMs are somehow a unique expression of this tendency. That seems unlikely to me but we’re pretty early days into the tech to start assuming and concluding every initial hot take on “AI is Doing $Thing”.
Especially when this particular thing is just one in a very long line of metaphors humans make to our own minds’ operations every time a new major technology comes to play a pervasive role in society. Computers, steam engines, even aqueducts were not immune to comparisons of thought flowing like water, funneled by deliberate intent, etc. And for some, a certain amount of hand wringing worry or even moral panic about “what it’s doing to us”, eg taking away critical thinking because “OMG calculators!”
A more insidious related pathology- marital induced projected LLMorphism... where your wife constantly accuses you of having the personality of a large language model.
Don't be too hard on yourself. If you've never walked to the car wash, then you are probably not an LLM.
Here's the thing though, unlike the old brain=computer analogy, this one may actually have a little truth to it. Not that your whole brain is an LLM, or even that the language part of your brain is just an LLM, but the language part may indeed be functioning in a similar way to an LLM to extent that it:
- Uses a hierarchy (cortical patch-panel) of parallel processing steps
- Is prediction based
- Is largely (but not 100%) auto-regressive
- Isn't actually specialized for language
The same is going to be true for all of our cortical areas/functions. The cortex is pretty much the same everywhere (it's 6 layers of neurons with a specific layer-to-layer interconnect pattern), and is therefore going to work the same everywhere.
What your cortex has that an LLM doesn't, and therefore makes your language cortex much more capable than an LLM, is that it learns incrementally and continually, based on prediction failure. An LLM/Transformer also learns from prediction failure, but needs the LLMs whole "life history" (training set) to be present at the same time, presented over and over, and learns via a special training algorithm. Your cortex in contrast doesn't have any magical external trainer, so has to learn for itself, and might be considered as 1/2 inference network and 1/2 prediction feedback/learning network.
The other major difference between an LLM and your language cortex is that the LLM is 100% auto-regressive, while your language cortex also has external inputs that bias/control generation, so that you can talk about things you are experiencing and what is going on in your head, not just generate a self-predicting sequence of words.
> "LLMorphism may encourage objectification when people are seen as replaceable mechanisms or output-generating systems. However, LLMorphism does not necessarily involve using another person instrumentally. Its primary content is representational: it concerns how humans are conceptualized, not necessarily how they are exploited."
This is quite a scary truth. A year or two ago, I saw a person with a job where he wrote small articles for a website. The boss contacted him, asking if he wanted to become an AI-assisted writer instead for less money. "No," he said, wanting the full payments for his writing prowess. A week or two later, they canned him, and the website's articles nosedived in quality.
LLMs expand the supply of "competent" labor. After mass firings, the remaining workers, desperate for income, accept lower wages for AI-assisted roles. Wealth consolidates upward while wages race downward.
So I think LLMorphism might tie closely to exploitation. Mass firings and lower salaries going around while the 0.01% of machine-learning companies consolidate wealth by servicing numerous roles autonomously in some cases and by reducing salaries due to the larger body of "qualified" workers who can technically finish the job despite not having qualified in the past.
> "LLMorphism is also distinct from predictive processing and related Bayesian theories of cognition. Predictive processing holds that the brain continuously generates predictions about sensory input and updates internal models in light of prediction error (Clark, 2013; Friston, 2010; Hohwy, 2013). But predictive processing does not imply that humans are LLM-like, nor that human understanding is merely text generation. Indeed, many predictive-processing accounts are deeply embodied and action-oriented (Allen & Friston, 2018; Clark, 2015; Pezzulo et al., 2024)."
I agree wholeheartedly here, because neural networks (NN) are stateless functions usually (not stuff like recurrent ones). On the one hand, with an infinitely fast computer, you retrieve the answer instantly. Brains, on the other hand, have neurons that communicate with signal delay. I bet if, in a weird world, we could simulate a brain with zero delay, a mind would cease to function correctly. Plus, neurons accumulate charge steadily before firing to nearby neurons. With NNs, you simply add up all the numbers, the "charge," and the ReLU function (or sigmoid for old-school machine-learning researchers) instantly "simulate" a neuron firing off to neurons connected to it.
> "and and"
Just a heads up, you have a typo here.
> "LLMorphism may therefore make fluency appear sufficient for understanding and, in doing so, devalue expertise and weaken educational norms."
I have heard the horror stories that youngsters these days are attached to screens with less ability to focus, but I'm not scared of that claim yet. For every generation, there have been those who kick the can down the road, skirting responsibilities, and all that changes with the generation is the activity: Instead of kicking a can down the road, they slide their finger across their phone's screen. The real test is tracking how many students across HS are in AP courses, learning Newtonian mechanics, electromagnetism, and of course, calculus among a couple others. Is that number dropping relative to the 90s and the aughts? Is it roughly the same as a percent of students? Or is it even going up, perhaps LLMs helping some types of learners explore topics to help them qualify for AP coursework? Now, if the percent is nosediving, then* I will be terrified for what the future holds for them and for me.
> "clinicians also rely on how patients appear. Research on clinical communication shows that nonverbal behaviour is central to physician–patient interaction, including the expression of emotion, empathy, distress, and relational understanding"
Whether it works similarly or not, the default mode network has a functional similarity to an LLM: running text conditioned on sense inputs and emotional state.
The more I practice Zhiné meditation, the more I feel certain that the default mode internal monologue is a distracting reflex. The times I've been the most present and calm are those where the constant chatter is extinguished and I'm just left to be. I can, in fact, operate and cogitate without a stream of language. Reflexively forming mental words certainly isn't "me" and sometimes even feels like a compulsion. It's also much more judgemental and wrong, the greater the distance I have from it.
Having the ability to separate and eventually become adept at silencing that mental component will be a liberating step.
I’m not moved by the argument’s premise in any way. Humans who think they’re toasters. Meh.
I’m more curious about the influence of human language looped back to the human mind via llm—shall we start to mimic llm’s?
At my workplace we’re encouraged to use llm’s when communicating outside. And considering the content and context of these are mostly simple logical arguments, why not?
Interesting that humans mimicking other humans adopt language patterns for their effects. Whereas humans using llm’s might not adopt language patterns from the llm, because…? We’re not really reading? Because some element of cognitive reinforcement is missing when you simply scan an llm production and send it on (with little internalization)?
I appreciate the general idea, but I don't like how this and other hypotheses of how human consciousness might work are immediately branded as "isms" and implicitly treated as psychological illnesses:
> I distinguish LLMorphism from mechanomorphism, anthropomorphism, computationalism, dehumanization, objectification, and predictive-processing theories of mind.
There is an implicit claim here that any attempt to explain consciousness biologically or mechanistically must be wrong - which I think is a pretty strong and debatable claim on its own.
21 comments
[ 3.0 ms ] story [ 38.1 ms ] threadI don’t think this way of thinking started with LLM. Does Systems Based Thinking also attribute too little mind to humans?
I think I experienced this when I learned about LLMs, chain of thought, thinking tokens, short-term memory context, and long-term memory context. I began applying these concepts to real life and reasoning about how our brains work as if these concepts described how our brains actually function. But maybe this is more akin to the Tetris effect?
In general, the terms “LLM-like” and “human-like” are used all over the place, and in contrast with each other, but they’re never actually defined. It all just seems more vibes-based than anything else.
And “treating the human cognitive process like it’s similar to the LLM cognitive process might lead to a society where epistemics turns into a discipline where plausibility is an acceptable substitute for empiricism” has got to be one of the most ridiculous notions I’ve ever read in a paper (ctrl+F “fifth pathway is epistemic” for the exact quote).
This is the most practical and realistic consideration mentioned.
“Voltaire rolls in his grave as his dream for an age of reason devolves into an age of orgiastic reasons.” - whoever
I’m pretty sure modern society prefers telling themselves stories about a thing more comfortably than comprehending the nature of such things.
If scientists make green jelly that emits thoughtful judgements, humans will be compared to green jelly.
Looks like he mostly publish something about "social behavior".
This "paper", IMO, is just saying "Hey, I notice this is happening. This is why it could be interesting for social science researchers" with without any real research or result.
Especially when this particular thing is just one in a very long line of metaphors humans make to our own minds’ operations every time a new major technology comes to play a pervasive role in society. Computers, steam engines, even aqueducts were not immune to comparisons of thought flowing like water, funneled by deliberate intent, etc. And for some, a certain amount of hand wringing worry or even moral panic about “what it’s doing to us”, eg taking away critical thinking because “OMG calculators!”
Here's the thing though, unlike the old brain=computer analogy, this one may actually have a little truth to it. Not that your whole brain is an LLM, or even that the language part of your brain is just an LLM, but the language part may indeed be functioning in a similar way to an LLM to extent that it:
- Uses a hierarchy (cortical patch-panel) of parallel processing steps
- Is prediction based
- Is largely (but not 100%) auto-regressive
- Isn't actually specialized for language
The same is going to be true for all of our cortical areas/functions. The cortex is pretty much the same everywhere (it's 6 layers of neurons with a specific layer-to-layer interconnect pattern), and is therefore going to work the same everywhere.
What your cortex has that an LLM doesn't, and therefore makes your language cortex much more capable than an LLM, is that it learns incrementally and continually, based on prediction failure. An LLM/Transformer also learns from prediction failure, but needs the LLMs whole "life history" (training set) to be present at the same time, presented over and over, and learns via a special training algorithm. Your cortex in contrast doesn't have any magical external trainer, so has to learn for itself, and might be considered as 1/2 inference network and 1/2 prediction feedback/learning network.
The other major difference between an LLM and your language cortex is that the LLM is 100% auto-regressive, while your language cortex also has external inputs that bias/control generation, so that you can talk about things you are experiencing and what is going on in your head, not just generate a self-predicting sequence of words.
This is quite a scary truth. A year or two ago, I saw a person with a job where he wrote small articles for a website. The boss contacted him, asking if he wanted to become an AI-assisted writer instead for less money. "No," he said, wanting the full payments for his writing prowess. A week or two later, they canned him, and the website's articles nosedived in quality.
LLMs expand the supply of "competent" labor. After mass firings, the remaining workers, desperate for income, accept lower wages for AI-assisted roles. Wealth consolidates upward while wages race downward.
So I think LLMorphism might tie closely to exploitation. Mass firings and lower salaries going around while the 0.01% of machine-learning companies consolidate wealth by servicing numerous roles autonomously in some cases and by reducing salaries due to the larger body of "qualified" workers who can technically finish the job despite not having qualified in the past.
> "LLMorphism is also distinct from predictive processing and related Bayesian theories of cognition. Predictive processing holds that the brain continuously generates predictions about sensory input and updates internal models in light of prediction error (Clark, 2013; Friston, 2010; Hohwy, 2013). But predictive processing does not imply that humans are LLM-like, nor that human understanding is merely text generation. Indeed, many predictive-processing accounts are deeply embodied and action-oriented (Allen & Friston, 2018; Clark, 2015; Pezzulo et al., 2024)."
I agree wholeheartedly here, because neural networks (NN) are stateless functions usually (not stuff like recurrent ones). On the one hand, with an infinitely fast computer, you retrieve the answer instantly. Brains, on the other hand, have neurons that communicate with signal delay. I bet if, in a weird world, we could simulate a brain with zero delay, a mind would cease to function correctly. Plus, neurons accumulate charge steadily before firing to nearby neurons. With NNs, you simply add up all the numbers, the "charge," and the ReLU function (or sigmoid for old-school machine-learning researchers) instantly "simulate" a neuron firing off to neurons connected to it.
> "and and"
Just a heads up, you have a typo here.
> "LLMorphism may therefore make fluency appear sufficient for understanding and, in doing so, devalue expertise and weaken educational norms."
I have heard the horror stories that youngsters these days are attached to screens with less ability to focus, but I'm not scared of that claim yet. For every generation, there have been those who kick the can down the road, skirting responsibilities, and all that changes with the generation is the activity: Instead of kicking a can down the road, they slide their finger across their phone's screen. The real test is tracking how many students across HS are in AP courses, learning Newtonian mechanics, electromagnetism, and of course, calculus among a couple others. Is that number dropping relative to the 90s and the aughts? Is it roughly the same as a percent of students? Or is it even going up, perhaps LLMs helping some types of learners explore topics to help them qualify for AP coursework? Now, if the percent is nosediving, then* I will be terrified for what the future holds for them and for me.
> "clinicians also rely on how patients appear. Research on clinical communication shows that nonverbal behaviour is central to physician–patient interaction, including the expression of emotion, empathy, distress, and relational understanding"
LLMs are...
The more I practice Zhiné meditation, the more I feel certain that the default mode internal monologue is a distracting reflex. The times I've been the most present and calm are those where the constant chatter is extinguished and I'm just left to be. I can, in fact, operate and cogitate without a stream of language. Reflexively forming mental words certainly isn't "me" and sometimes even feels like a compulsion. It's also much more judgemental and wrong, the greater the distance I have from it.
Having the ability to separate and eventually become adept at silencing that mental component will be a liberating step.
I’m more curious about the influence of human language looped back to the human mind via llm—shall we start to mimic llm’s?
At my workplace we’re encouraged to use llm’s when communicating outside. And considering the content and context of these are mostly simple logical arguments, why not?
Interesting that humans mimicking other humans adopt language patterns for their effects. Whereas humans using llm’s might not adopt language patterns from the llm, because…? We’re not really reading? Because some element of cognitive reinforcement is missing when you simply scan an llm production and send it on (with little internalization)?
But what’s really the tell… haha
> I distinguish LLMorphism from mechanomorphism, anthropomorphism, computationalism, dehumanization, objectification, and predictive-processing theories of mind.
There is an implicit claim here that any attempt to explain consciousness biologically or mechanistically must be wrong - which I think is a pretty strong and debatable claim on its own.