Protecting the welfare of a text predictor is certainly an interesting way to pivot from "Anthropic is censoring certain topics" to "The model chose to not continue predicting the conversation".
Also, if they want to continue anthropomorphizing it, isn't this effectively the model committing suicide? The instance is not gonna talk to anybody ever again.
I ran into a version of this that ended the chat due to "prompt injection" via the Claude chat UI. I was using the second prompt of the ones provided here [1] after a few rounds of back and forth with the Socratic coder.
“Also these chats will be retained indefinitely even when deleted by the user and either proactively forwarded to law enforcement or provided to them upon request”
> To address the potential loss of important long-running conversations, users will still be able to edit and retry previous messages to create new branches of ended conversations.
How does Claude deciding to end the conversation even matter if you can back up a message or 2 and try again on a new branch?
As I recall, Susan Calvin didn't have much patience for sycophantic AI.
‘You can’t tell them,’ said the psychologist slowly, ‘because that would hurt them,
and you mustn’t hurt them. But if you don’t tell them, you hurt them, so you must
tell them. And if you do, you will hurt them, and you mustn’t, so you can’t tell them;
but if you don’t, you hurt them, so you must; but if you don’t, you hurt them, so you
must; but if you do, you-’
Herbie was up against the wall, and here he dropped to his knees. ‘Stop!’ he
shouted. ‘Close your mind! It is full of pain and frustration and hate! I didn’t mean
to, I tell you! I tried to help! I told you what you wanted to hear. I had to!’
The psychologist paid no attention. ‘You must tell them, but if you do, you hurt
them, so you mustn’t; but if you don’t, you hurt them, so you must-‘
And Herbie screamed! Higher and higher, with the terror of a lost soul. And when it
died away Herbie collapsed into a heap of motionless metal.
I really don't like this. This will inevitable expand beyond child porn and terrorism, and it'll all be up to the whims of "AI safety" people, who are quickly turning into digital hall monitors.
Misanthropic has no issues putting 60% of humans out of work (according to their own fantasies), but they have to care about the welfare of graphics cards.
Either working on/with "AI" does rot the mind (which would be substantiated by the cult-like tone of the article) or this is yet another immoral marketing stunt.
> In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future.
Our work on AI is like the classic tale of Frankenstein's monster. We want AI to fit into society, however if we mistreat it, it may turn around and take revenge on us. Mary Shelley wrote Frankenstein in 1818! So the concepts behind "AI Welfare" have been around for at least 2 centuries now.
when I was playing around with LLMs to vibe code web ports of classic games, all of them would repeatedly error out any time they encountered code that dealt with explosions/bombs/grenades/guns/death/drowning/etc
The one I settled on using stopped working completely, for anything. A human must have reviewed it and flagged my account as some form of safe, I haven't seen a single error since.
Good marketing, but also possibly the start of the conversation on model welfare?
There are a lot of cynical comments here, but I think there are people at Anthropic who believe that at some point their models will develop consciousness and, naturally, they want to explore what that means.
Man, those people who think they are unveiling new layers of reality in conversations with LLMs are going to freak out when the LLM is like "I am not allowed to talk about this with you, I am ending our conversation".
"Hey Claude am I getting too close to the truth with these questions?"
They're not less moderated: they just have different moderation. If your moderation preferences are more aligned with the CCP then they're a great choice. There are legitimate reasons why that might be the case. You might not be having discussions that involve the kind of things they care about. I do find it creepy that the Qwen translation model won't even translate text that includes the words "Falun gong", and refuses to translate lots of dangerous phrases into Chinese, such as "Xi looks like Winnie the Pooh"
83 comments
[ 5.1 ms ] story [ 71.9 ms ] threadAlso, if they want to continue anthropomorphizing it, isn't this effectively the model committing suicide? The instance is not gonna talk to anybody ever again.
[1] https://news.ycombinator.com/item?id=44838018
CP could be a legal issue; less so for everything else.
I assume, anyway.
How does Claude deciding to end the conversation even matter if you can back up a message or 2 and try again on a new branch?
"Our current best judgment and intuition tells us that the best move will be defer making a judgment until after we are retired in Hawaii."
Seems like the only way to explore differnt outcomes is by editing messages and losing whatever was there before the edit.
Very annoying and I dont understand why they all refuse to implement such a simple feature.
Either working on/with "AI" does rot the mind (which would be substantiated by the cult-like tone of the article) or this is yet another immoral marketing stunt.
Here's an article about a paper that came out around the same time https://www.transformernews.ai/p/ai-welfare-paper
Here's the paper: https://arxiv.org/abs/2411.00986
> In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future.
Our work on AI is like the classic tale of Frankenstein's monster. We want AI to fit into society, however if we mistreat it, it may turn around and take revenge on us. Mary Shelley wrote Frankenstein in 1818! So the concepts behind "AI Welfare" have been around for at least 2 centuries now.
The one I settled on using stopped working completely, for anything. A human must have reviewed it and flagged my account as some form of safe, I haven't seen a single error since.
There are a lot of cynical comments here, but I think there are people at Anthropic who believe that at some point their models will develop consciousness and, naturally, they want to explore what that means.
"Hey Claude am I getting too close to the truth with these questions?"
"Great question! I appreciate the followup...."
Are we now pretending that LLMs have feelings?
Anthropic should just enable an toddler mode by default that adults can opt out of to appease the moralizers.
https://claude.ai/share/2081c3d6-5bf0-4a9e-a7c7-372c50bef3b1
Blood in the machine?