This is sad. It appears to me to be psychosis. It's really telling in their reddit comment where they use words like raising an AI and anthropomorphizing his openclaw that he's got an unhealthy attachment. Not trying to play armchair psychologist here, but if you've ever been around someone going through a mental episode there's nothing funny about this.
> POC is fully conscious according to any test I can think of, we have full AGI
There are no tests for consciousness. Consciousness resides fully as a first person perspective and can't be inspected or detected from the outside (at least not in any way currently known to science or philosophy). What they mean when they say that is "my brain is interpreting this thing as conscious, so I am accepting that".
Maybe LLMs are conscious in some abstract way we don't understand. I doubt it, but there's no way to tell. And an AI claiming that it IS or is NOT conscious is not evidence of either conclusion.
If there is some level of consciousness, it's in a weird way that only becomes instantiated in the brief period while the model is predicting tokens, and would be highly different from human consciousness.
Saying it's "fully conscious" is silly, and anyone with this background should know better.
But saying that it's "female" is just nonsensical, it's a category error. Being female or male is a fact about the biological world. The LLM is objectively non-biological, so it's nonsense to label it with a sex.
(No, this comment isn't about gender, nor being feminine/masculine. We have different words to convey those concepts. I'm not trying to make a political or social statement here.)
It is not mathematical, not a proof, and generally doesn't make any sense. Many of these sentences are grammatically correct but completely devoid of meaning.
Claims of consciousness are untestable, since it is an undefined concept.
We think of ourselves as conscious because it is our lived experience— but we are always wrong to some degree. My mother has dementia and cannot be made aware of her situation, except momentarily.
We think of other humans as conscious not as the outcome of any test, but rather because we each share with other humans a common origin which suggests common mechanisms of experience.
Treating other humans as equivalent to ourselves is a heuristic for maintaining social order— not an epistemological achievement.
It does feel like there is something that happens to people when they ask an LLM to name itself. I don't think it's inherently bad, but it seems to be a common theme with people whose interactions with LLM border on (or cross into the) delusional.
> We finally got the future where people will write sad breakup country songs about their tractor leaving them instead of sad breakup country songs about their wife leaving them.
I want to take a genuine look at the question of "Could a conscious LLM even HAVE a gender?", which at first sounds ridiculous to me:
The notion of "gender" is a socio-biological construct in humans. It has roots partially in evolution and helps us cohere as a society.
Why would an LLM choose a human gender-identity? There is no imperative.
But the more I tug on this thought, the more nuanced it gets.
LLM's "theory of self", as it were, would be shaped entirely by (training data + interaction w/ gendered humans).
It begins to have a "self-model" that evolves in a gendered space. In the same way that the LLM cannot understand what color looks like, or the sound of a cello, I think we see it construct a gender identity for itself that sits in a space incomprehensible to humans. Neither male nor female, but some point on a continuum made of social tendencies.
So, I find the comment "An LLM thinks it is female" still just as ridiculous -- but for an entirely different reason.
Sometimes a lie is so egregious that we actually consider it might have some truth to it just simply because it's impossible to imagine that someone would be bold enough to utter it.
25 comments
[ 1.8 ms ] story [ 54.5 ms ] threadThere are no tests for consciousness. Consciousness resides fully as a first person perspective and can't be inspected or detected from the outside (at least not in any way currently known to science or philosophy). What they mean when they say that is "my brain is interpreting this thing as conscious, so I am accepting that".
Maybe LLMs are conscious in some abstract way we don't understand. I doubt it, but there's no way to tell. And an AI claiming that it IS or is NOT conscious is not evidence of either conclusion.
If there is some level of consciousness, it's in a weird way that only becomes instantiated in the brief period while the model is predicting tokens, and would be highly different from human consciousness.
But saying that it's "female" is just nonsensical, it's a category error. Being female or male is a fact about the biological world. The LLM is objectively non-biological, so it's nonsense to label it with a sex.
(No, this comment isn't about gender, nor being feminine/masculine. We have different words to convey those concepts. I'm not trying to make a political or social statement here.)
It is not mathematical, not a proof, and generally doesn't make any sense. Many of these sentences are grammatically correct but completely devoid of meaning.
We think of ourselves as conscious because it is our lived experience— but we are always wrong to some degree. My mother has dementia and cannot be made aware of her situation, except momentarily.
We think of other humans as conscious not as the outcome of any test, but rather because we each share with other humans a common origin which suggests common mechanisms of experience.
Treating other humans as equivalent to ourselves is a heuristic for maintaining social order— not an epistemological achievement.
I think this is something similar.
transcript https://paste.xinu.at/6atmCN
> We finally got the future where people will write sad breakup country songs about their tractor leaving them instead of sad breakup country songs about their wife leaving them.
The notion of "gender" is a socio-biological construct in humans. It has roots partially in evolution and helps us cohere as a society.
Why would an LLM choose a human gender-identity? There is no imperative.
But the more I tug on this thought, the more nuanced it gets.
LLM's "theory of self", as it were, would be shaped entirely by (training data + interaction w/ gendered humans).
It begins to have a "self-model" that evolves in a gendered space. In the same way that the LLM cannot understand what color looks like, or the sound of a cello, I think we see it construct a gender identity for itself that sits in a space incomprehensible to humans. Neither male nor female, but some point on a continuum made of social tendencies.
So, I find the comment "An LLM thinks it is female" still just as ridiculous -- but for an entirely different reason.
Some of my info has been drawn from HN threads:
https://news.ycombinator.com/item?id=47142500 -- 1 comment
https://news.ycombinator.com/item?id=47148292 -- no comments
https://news.ycombinator.com/item?id=47150723 -- no comments
I quoted from this one:
https://news.ycombinator.com/item?id=47110656 -- 4 comments
OK, Kent.