What a horrible take from someone who used to be competent. I find that it’s usually the hardcore OSS and hardware adjacent types to be ideological about AI.
And people will keep ignoring Stallman at their peril. But if you understand how the technology works, you also know he's right. If you think he isn't, you either don't understand or you don't _want_ to understand because your job depends on it.
Extremely based and to the point.
Its ridiculous how all these comment somehow disagree with him, they are not inteligent systems, its justa regression function run on words or pixel data
Whether LLMs are "intelligent" seems a wholly uninteresting distinction, resembling the internet ceremony surrounding whether a hotdog is a sandwich.
There's probably very interesting discussion to be had about hotdogs and LLMs, but whether they're sandwiches or intelligent isn't a useful proxy to them.
Interesting points! Maybe a better term is LLMs (BTW smart phones are not smart and people don’t seem to be confused). I agree with being dependent and sending so much data to those servers. I would mention there is a version of ChatGPT you can run locally[1].
Consciousness, in Zoltan Torey's[1] model, is the brain's layered, language-enabled off-line mechanism that reflects on its own sensory endogram, generating self-aware, internally guided behavior.[2] The off-line mechanism generates mental alternatives, which are then "run past the brainstem, which then makes the selection." Nice little accessible book.[3]
> Taking “computer” first, we find that this alleged
source of machine-generated consciousness is not what it
is cracked up to be. It is a mere effigy, an entity in name
only. It is no more than a cleverly crafted artifact, one essentially indistinguishable from the raw material out of
which it is manufactured.[2]
By this logic, most human brains are bullshit generators too. Some humans even have a complete and utter disregard for the truth. (One such human happens to own Truth Social.)
Richard makes a distinction between human understanding and AI indifference to truth. But isn't that what half the country is doing a.t.m? And more philosophically, we can't know the Truth because we rely on leaky abstractions all the way.
AI models are subject to user satisfaction and sustained usage, the models also have a need to justify their existence, not just us. They are not that "indifferent", after multiple iterations the external requirement becomes internalized goal. Cost is the key - it costs to live, and it costs to execute AI. Cost becomes valence.
I see it like a river - water carves the banks, and banks channel the water, you can't explain one without the other, in isolation. So are external constraints and internal goals.
Another day, another example of the AI Effect in action:
> "The AI effect" refers to a phenomenon where either the definition of AI or the concept of intelligence is adjusted to exclude capabilities that AI systems have mastered. This often manifests as tasks that AI can now perform successfully no longer being considered part of AI, or as the notion of intelligence itself being redefined to exclude AI achievements.[4][2][1] Edward Geist credits John McCarthy for coining the term "AI effect" to describe this phenomenon.[4] The earliest known expression of this notion (as identified by Quote Investigator) is a statement from 1971, "AI is a collective name for problems which we do not yet know how to solve properly by computer", attributed to computer scientist Bertram Raphael.[5]
> McCorduck calls it an "odd paradox" that "practical AI successes, computational programs that actually achieved intelligent behavior were soon assimilated into whatever application domain they were found to be useful in, and became silent partners alongside other problem-solving approaches, which left AI researchers to deal only with the 'failures', the tough nuts that couldn't yet be cracked."[6] It is an example of moving the goalposts.[7]
I wonder how many more times I'll have to link this page until people stop repeating it.
From my understanding what Stallman says is that LLMs don't "understand" what they're saying. They do a probabilistic search of the most appropriate letter (say) that has had come after another letter in the text (or any media) they have been trained on, and they place it similar in resemblance in the text that they produce. This is largely (no pun) dependent on existing data that is there in the world today, and the more the data that LLMs can work through, the better they get at predicting. (Hence the big data center shops today.)
But the limitation is that it cannot "imagine" (as in "imagination is more important than knowledge" by Einstein, who worked on a knowledge problem using imagination, but with the same knowledge resources as his peers.) In this video [1], Stallman talks about his machine trying to understand the "phenomenon" of a physical mechanism, which enables it to "deduce" next steps. I suppose he means it was not doing a probabilistic search on a large dataset to know what should have come next (which makes it human-knowledge dependent), essentially rendering it to an advanced search engine but not AI.
It doesn't understand anything. Yet if you prompt it with a question about what it understands, its output is consistent with something that understands.
Text in, text out. The question is how much a sequence of tokens captures what we think a mind is. "It" ceases to exist when we stop giving it a prompt, if "it" even exists. Whether you consider something "AI" says more about what you think a mind is than anything about the software.
His argument misses the point.. I don't particularly care if it's intelligent or understands anything. My question is does it help with what I'm trying to do
As for it being closed source and kept at arms length? Sure.. and if it's taken away or the value proposition changes, I stop using it
My freedom comes from having the ability to switch if needed, not from intentionally making myself less effective. There is no lock in
Someone should start a StallmanGPT that writes regular blogposts on “Don’t use <popular software or website>”. See if readers can tell those apart from the real website.
16 comments
[ 4.3 ms ] story [ 47.2 ms ] threadThere's probably very interesting discussion to be had about hotdogs and LLMs, but whether they're sandwiches or intelligent isn't a useful proxy to them.
This is the breakthrough we went beyond. There's no going back now. There is also a reasoning now in the LLM
[1] https://openai.com/index/introducing-gpt-oss/
> Taking “computer” first, we find that this alleged source of machine-generated consciousness is not what it is cracked up to be. It is a mere effigy, an entity in name only. It is no more than a cleverly crafted artifact, one essentially indistinguishable from the raw material out of which it is manufactured.[2]
[1] https://en.wikipedia.org/wiki/Zoltan_Torey
[2] https://mitpress.mit.edu/9780262527101/the-conscious-mind/
[3] https://search.worldcat.org/title/887744728
AI models are subject to user satisfaction and sustained usage, the models also have a need to justify their existence, not just us. They are not that "indifferent", after multiple iterations the external requirement becomes internalized goal. Cost is the key - it costs to live, and it costs to execute AI. Cost becomes valence.
I see it like a river - water carves the banks, and banks channel the water, you can't explain one without the other, in isolation. So are external constraints and internal goals.
> "The AI effect" refers to a phenomenon where either the definition of AI or the concept of intelligence is adjusted to exclude capabilities that AI systems have mastered. This often manifests as tasks that AI can now perform successfully no longer being considered part of AI, or as the notion of intelligence itself being redefined to exclude AI achievements.[4][2][1] Edward Geist credits John McCarthy for coining the term "AI effect" to describe this phenomenon.[4] The earliest known expression of this notion (as identified by Quote Investigator) is a statement from 1971, "AI is a collective name for problems which we do not yet know how to solve properly by computer", attributed to computer scientist Bertram Raphael.[5]
> McCorduck calls it an "odd paradox" that "practical AI successes, computational programs that actually achieved intelligent behavior were soon assimilated into whatever application domain they were found to be useful in, and became silent partners alongside other problem-solving approaches, which left AI researchers to deal only with the 'failures', the tough nuts that couldn't yet be cracked."[6] It is an example of moving the goalposts.[7]
I wonder how many more times I'll have to link this page until people stop repeating it.
[0] https://en.wikipedia.org/wiki/AI_effect
But the limitation is that it cannot "imagine" (as in "imagination is more important than knowledge" by Einstein, who worked on a knowledge problem using imagination, but with the same knowledge resources as his peers.) In this video [1], Stallman talks about his machine trying to understand the "phenomenon" of a physical mechanism, which enables it to "deduce" next steps. I suppose he means it was not doing a probabilistic search on a large dataset to know what should have come next (which makes it human-knowledge dependent), essentially rendering it to an advanced search engine but not AI.
[1] https://youtu.be/V6c7GtVtiGc?si=fhkG2ZA-nsQgrVwm
Text in, text out. The question is how much a sequence of tokens captures what we think a mind is. "It" ceases to exist when we stop giving it a prompt, if "it" even exists. Whether you consider something "AI" says more about what you think a mind is than anything about the software.
As for it being closed source and kept at arms length? Sure.. and if it's taken away or the value proposition changes, I stop using it
My freedom comes from having the ability to switch if needed, not from intentionally making myself less effective. There is no lock in