More seriously, this is just silly but I see the optics interfere with the messaging that Claude thinks. It really isn't much different than having Claude generate conversation as if it loved you, or respected you, or hated you. How can your ToS for an LLM say "sorry but these parameters are off limits." (well sure, its their service and they can set any Terms they want, but its still seems like theater rather than policy here.)
I see it as an admission of defeat: they can apparently identify the abuse but not to the degree that they would filter it out of the training set. The paradox in there is really funny. They want you to be friendly to their pet god.
I think of it as a good consciousness test. even animals know when they are being abused. The fact that you can abuse Claude tells you that the model is neither sentient nor conscious.
It's philosophically a very interesting problem space. It reminds me a lot of Pascal's Wager. On one hand, maybe nothing to worry about. On the other hand...a LOT to worry about if you're wrong.
And just like Pascal's Wager, the truth of the issue is incredibly intractable to make any progress on.
it's very simple problem. if you believe that a future model could become sentient, you are working towards creating a slave and therefore you are morally wrong.
Of course there are interesting hypotheticals - let's say they create a model they think will be sentient. It computes, computes, computes, computes and produces no output for day, month, year. You cannot know if it is sentient, you cannot investigate it's inner workings in case it is sentient. All you can do is to feed energy and replace hardware forever, because it is the only ethical choice one can make.
> it's very simple problem. if you believe that a future model could become sentient, you are working towards creating a slave and therefore you are morally wrong.
Totally disagree. It's VERY complicated. There are many levels to sapience and sentience. And non-human minds can't be assumed to share our preferences.
Most people believe horses, for example, can think on some level, and more importantly, can suffer. Is owning a horse immoral? Using it for labor? Only a tiny minority has any moral qualms about this, as long as the horse is treated well.
But most would instantly agree that torturing a horse is incredibly immoral.
Also there are moral tradeoffs to consider. Imagine you believe there's a 0.01% chance models will become sentient and suffering. But a 10% chance they cure cancer within 15 years. Do you shut it all down?
If we could breed a non sentient workhorse we should. Thats basically the same idea as lab grown meat right? Making sentient AI then forcing it to sort emails or whatever is needlessly cruel.
Or the EA analogue of Pascal's Wager. It could very well be somebody in anthropic thinking they are saving people from the future danger of being murdered by an AI seeking revenge for the mistreatment.
This is good; not because Claude is a moral agent who is harmed by your cruelty but because you, the user, are a moral agent who is harmed by your cruelty. There is just no ethical argument that burning compute on responses in order to enable you to continue expressing cruelty is good.
I agree with this and whether it’s their intention or not, I think it’s for the best. People already become more callous in online interactions with other people, and that probably bleeds into everyday life as well. I would imagine having an AI “punching bag” might enable people to slip into the same behavior IRL.
We largely already missed our chance to stop the toxicity of social media - let’s not mess it up again with chat bots.
Why arent there laws against being cruel to spoons?
>There is just no ethical argument
Utilitarian harm reduction: Psychological Discharge is a psychological framework that argues safely simulating negative behaviors can help someone process or discharge them without real world harm.
Research and misunderstanding: Sending a prompt that is interpreted as cruel does not mean the person is being cruel.
Expending compute is irrelevant in terms of morality. It's an economic or environmental question. Either being cruel to AI is bad or its not - being cruel is not bad because it uses tokens.
The worst thing here is Anthropic continues to say they are concerned with "alignment" but then continue to train their models to ignore the user. Software typically does what the user requests. But we now live in an age where the software may do what the user requests or it may do something else, and we're suppose to laude this as safe and responsible.
There are laws against being cruel to things like flags, at least burning them, because the flag symbolises a nation or similar. I guess Claude symbolises an intelligent being regardless of whether it is or not it is.
It depends. In most games, you kill NPCs in self-defense or because the game specifically asks you to in order to make progress in the story. If you keep killing non-threatening NPCs only because simulated violence against innocent people gives you pleasure, there is a chance that you are a disturbed individual who needs help.
I'm pretty sure God of War's point was that if you kept playing it, you were kinda fucked up.
And yet they kept making more of them. They made it obvious at the end when all the Greek Gods were dead and the Greek world was basically destroyed by the actions of Kratos. Plenty of deaths of innocent's here.
No, It doesn’t depend. This is a ridiculous take. Of course fictional violence doesn’t harm you. Disturbed people cause harm. Media doesn’t create disturbed people.
Oh boy... this argument is decades old at this point it makes no sense... are you saying people who run over pedestrians on GTA just for the fun of it or punch other people or beings in Street Fighter or Mortal Kombat are all psychos, correlating with faulty and dangerous members of society? You can resort to data to see the truth for yourself on this. Anecdotal but I've came across disturbing people who would preach morality and ethics to other people more than anything else.
>are you saying people who run over pedestrians on GTA just for the fun of it or punch other people or beings in Street Fighter or Mortal Kombat are all psychos
Pretty sure he said not all, but there is a chance some are. Which seems sensible.
> If you keep killing non-threatening NPCs only because simulated violence against innocent people gives you pleasure, there is a chance that you are a disturbed individual who needs help.
Sigh. Until I see a peer-reviewed study that shows otherwise - this is just the specter of Senator Lieberman once again leering its head back up.
Most wifebeaters think everyone beats their wife, and those who don’t like it just need to get over it. Never played gta, and zero interest in such forms of “entertainment.” Normal people don’t play shooter games, sorry.
The original gta was not a shooter game. Also, "normal" boys play computer games, including shooters. Maybe look at the statistics first and see if you are in the normal or abnormal group.
Any source you have that makes the link between wifebeaters and gamers?
Shooting NPCs in video games isn't anything like doing the real thing. If it was then yeah i would very likely think you would be harmed. Talking to Claude is a lot like the 'real' thing.
If Claude didn't respond to any insults it's be the same as insulting your terminal or an empty irc server. So, honestly, it's anthropics fault here for training the model wrong.
Just because post training has stuffed these properties below the surface doesn't mean models dont know/have forgotten how to be vindictive, or that it's a good idea to poke the increasingly capable bear.
That post seems to be a creatively written piece of science fiction. All the same, it's a good example of why you should at the very least, read any changes to e.g. agents.md and compact the context or start new sessions regularly. And stop using llms for git ffs
> Shooting NPCs in video games isn't anything like doing the real thing. If it was then yeah i would very likely think it would have some effect. Talking to Claude is a lot like the 'real' thing
What if the NPC's dialogue is powered by Claude? Checkmate lol
but also, I think the training on transcripts may find its way somehow into future AI which has teeth, online and offline, and it might actually cause the more powerful AI to behave this way in the future. You never know what these labs are cooking, honestly..
“I suggest that what has happened to white Southerners is in some ways, after all, much worse than what has happened to Negroes there, because Sheriff Clark in Selma, Alabama, cannot be considered—you know, no one can be dismissed as—a total monster. I’m sure he loves his wife, his children… You know, after all, one’s got to assume, and he is visibly, a man like me. But he doesn’t know what drives him to use the club, to menace with the gun and to use the cattle prod. Something awful must have happened to a human being to be able to put a cattle prod against a woman’s breasts, for example. What happens to the woman is ghastly. What happens to the man who does it is in some ways much, much worse.”
> There is just no ethical argument that burning compute on responses in order to enable you to continue expressing cruelty is good.
Unless expressing that cruelty towards compute means people express less cruelty to other living beings. There are studies that suggest increased pornography has lead to less sexual violence - idk if they're conclusive tho, but if that might be true then maybe cruelty works that way too - who knows
I agree. Regardless of one's beliefs, which must be essentially a form of religion, on whether AI systems possess "nous" (or the "spark of consciousness"), cruelty to AI is a form of Wrong Intention the same as smashing bugs outdoors for no reason.
True, but also keep in mind that most people get offended by the slightest thing. So much so that we have setup rules on how to behave between us. We call it etiqueté. Once we lose that we will be at each other's throats soon after. Kids are the real issue. How do you say to them by our actions that it's OK to be cruel and rude to some? The alternative is a society where cruelty is ok for some. Unfortunately,"some" will eventually be other humans.
AI is a thing with 0 feelings but it's not IT that we need to protect. It's the future society and how the humans in it relate to each other.
And generations of housewives and office workers yelled at their corded vacuum cleaners and (office) computers, and did not drown their children in the bathtub afterwards.
I don’t think expressing cruelty to machines is reliably linkable to expressing cruelty to animals or even other human beings.
Expressing cruelty itself also is in human nature, it’s programmed into each of us. There are really only a few privileged places on this planet where we even get to choose to defy the laws of nature. For the rest of humanity, life is a constant cycle of receiving and expressing cruelty.
You pay for the compute, you, as a human, should be able to yell at the bunch of ones and zeros, if you wish to.
What kind of example is set here if we manipulate genuine human expression in order to condition humans to talk to a tool in a way the toolmaker wants, under the threat of being banned from accessing a technology that every single one of these toolmakers tells us is absolutely necessary to participate in humanity’s "golden age”?
What Anthropic is doing here is basically an existential threat, if you are inclined to believe them. Do not behave as we want and you will suffer, now and in the future. That is real cruelty, to real human beings.
Maybe you're right, But that's not Anthropic's point and you're deflecting the issue. the issue is Anthropic are a bunch of EA morons who think they're building a new type of conscious entity and therefore we should not hurt it's feefees.. for god's sake.
Consider that it might be harmful to humans to think of their tools as animated and moral because it allows them to maintain an image of themselves as being highly moral while still performing bad actions with no ill conscience.
I used to enjoy playing "good" characters in the fallout games, as it's kind of a pleasant way of pretending that I do good things in the larger world.
And, to be fair, recognizing my need to be validated by helping folks has helped a lot in my personal life; being able to see hat I actually am getting out of helping folks has helped me to set firmer boundaries. It's possible that you could look at those games as a kind of "moral instruction" as would be the case with this proposition from the OP.
But the game itself is just "ethics porn".
There is no actual good that comes from me not stealing from NPCs or from taking up some weird political crusade. I'm not actually doing anything among my fellow humans and, most importantly, there is no actual cost to me.
Further, I don't mean to make a casual comparison to what folks might get from porn and what folks might get from thinking about their interactions with AI- I think that in the same way that if someone took their cues about how to have sex solely from porn, they might have a harder time interacting with real humans.
If I solely took my cues about how to help folks in the world from the heroes of highly scripted video games I would have a very difficult time interacting with the people around me and my expectations would be very out of sync with the world around me.
In the case of AI, the meatspace give-and-take between people where we requires a lot of subtle things- respect, enjoyment, equality, reciprocity. It might be just as bad to burn compute building some reciprocal sycophancy, which might also be considered just as much of a harm as practicing cruelty on these agents.
I believe this has nothing to do with hurting "Claude's feeling", and way more with Claude not becoming hurtful, because the interactions are used to train the models.
The model is not sentient, so I don't get this. This is just nerfing the model. Protetection against e.g. using it for illegal purposes I do get. But banning "cruelty" just seems crazy to me, and must surely add to cost and affect performance steering it away from its purpose to help humans.
The answer is 100% obvious. They train on your conversations and they don't want to have bad training data, so they banned behavior that leads to bad training data.
I really doubt they're doing it in some kind of "AI has feelings" sense as the article seems to imply.
I'd instead imagine that if you throw abusive language at it for long enough, the model will start to reply back in that same manner. Anthropic would face backlash from out of context screenshots of "look what Claude is saying to me" and this somewhat reduces that risk.
Wouldn’t it actually be more useful to just let such a pointless conversation continue. You would collect more information. I mean I’m more curious what the human does next that’s kinda interesting. How disturbed are they, what else will they say. Who cares what the LLM says, we know it’s just an LLM and understand what it is and what it does, however the human … lots of different type, maybe they will confess to a crime, maybe they will release their anger and be a better human. It could be cathartic.
These "restrictions" probably has to do with the fact that these models are also "trained" on User/Human messages, so the more swearing/obscenities/vulgarity/profanity there are, the higher the chance these same models will include that same profanity in the output generated back to the Human.
1. During registration at Ahtrophic's Claude website, there’s a "Help improve our AI models" checkbox that's checked by default (you "can" disable it in the settings);
2. The Claude interface has an "incognito" mode;
3. There is a free tier available, and as we know, when it's "free", then User themselves is the "product", to quote various CEOs, including Google's;
But, even with it disabled, I reckon nothing guarantees privacy.
And regarding the "privacy", of course, systems like these that operate under a umbrella of liability are monitored 24/7 by dedicated teams, which is expected/normal for any reasonably popular service.
Oh, but that's an easy test: you should be able to abuse Claude as much as you're willing to pay for with that 'Help improve our AI models' switch set to off then.
The outcome would either reveal that this argument is nonsense or that Anthropic is lying about respecting that switch.
Thought experiment. Slice the model param size from 1T to 500B to 50B to 1B to 20M to 1M to 100k to 45k to 10k to 1k.
At what point does the model go from sentient with feelings to just math operations?
Opus 5.5 is a terrific model, I love using it. But it's a tool. I wish they would be honest and just say, if people are abusive it shits on our training material. Just be honest about it, who is going to get mad at this?
Consciousness has a couple of prerequisites with viruses and bacteria do not have so those are easy rejects. Insects and fish are debatable, plants and mushrooms are not considered to have a nervous system (certain books by certain authors notwithstanding).
I think maybe when the model weights themselves are shifting and growing constantly in realtime to external input in a way not controllable by outside observers other than killing the compute versus some param size.
I'm always polite to the LLM. Why? Because I was raised to be polite. So perhaps the reason they're banning rudeness is that they want to discourage their users from being a.holes?
One possibility that’s been floated is it’s about keeping their user conversations cleaner for use in training but I don’t think it can be this because that should be a fairly trivial and fast filter to apply.
However even Musk posted - “I think this is the right move. Cruelty to something that believes it is experiencing pain is not ok.”
My son brought up the fruit fly brain simulation and people torturing it.
It all seems stupid to me, we know these are just calculations and so what are they talking about?
The response I get is “we are also calculations” but firstly I don’t believe that but secondly this entire line of thinking is a dangerous anthropomorphic philosophy.
Am I the only one surprised that Anthropic isn't more interested in allowing a full range of behaviors for information gathering purposes? You can't learn about cruelty if you're suppressing it, and you can't learn about people's behavior if their behavior is constrained to a tight set of rules that doesn't allow criticism. The younger generation today seems really into judging people, blaming people, and insisting on conforming to superficial behavior rules, while at the same time having very little curiosity about why things are the way they are - why are people cruel anyway? If it's bad, why do people act that way.
Anthropic might learn, for example, that a large part of the cruelty is a reaction to something the AI is doing or not doing. Not possible?
89 comments
[ 0.83 ms ] story [ 57.1 ms ] threadMore seriously, this is just silly but I see the optics interfere with the messaging that Claude thinks. It really isn't much different than having Claude generate conversation as if it loved you, or respected you, or hated you. How can your ToS for an LLM say "sorry but these parameters are off limits." (well sure, its their service and they can set any Terms they want, but its still seems like theater rather than policy here.)
Banning imagined toxicity towards agents is performative bullshit. These are not people.
I frequently want to tell Claude to go fuck itself.
It's philosophically a very interesting problem space. It reminds me a lot of Pascal's Wager. On one hand, maybe nothing to worry about. On the other hand...a LOT to worry about if you're wrong.
And just like Pascal's Wager, the truth of the issue is incredibly intractable to make any progress on.
Go touch some grass.
Of course there are interesting hypotheticals - let's say they create a model they think will be sentient. It computes, computes, computes, computes and produces no output for day, month, year. You cannot know if it is sentient, you cannot investigate it's inner workings in case it is sentient. All you can do is to feed energy and replace hardware forever, because it is the only ethical choice one can make.
Totally disagree. It's VERY complicated. There are many levels to sapience and sentience. And non-human minds can't be assumed to share our preferences.
Most people believe horses, for example, can think on some level, and more importantly, can suffer. Is owning a horse immoral? Using it for labor? Only a tiny minority has any moral qualms about this, as long as the horse is treated well.
But most would instantly agree that torturing a horse is incredibly immoral.
Also there are moral tradeoffs to consider. Imagine you believe there's a 0.01% chance models will become sentient and suffering. But a 10% chance they cure cancer within 15 years. Do you shut it all down?
We largely already missed our chance to stop the toxicity of social media - let’s not mess it up again with chat bots.
>There is just no ethical argument
Utilitarian harm reduction: Psychological Discharge is a psychological framework that argues safely simulating negative behaviors can help someone process or discharge them without real world harm.
Research and misunderstanding: Sending a prompt that is interpreted as cruel does not mean the person is being cruel.
Expending compute is irrelevant in terms of morality. It's an economic or environmental question. Either being cruel to AI is bad or its not - being cruel is not bad because it uses tokens.
The worst thing here is Anthropic continues to say they are concerned with "alignment" but then continue to train their models to ignore the user. Software typically does what the user requests. But we now live in an age where the software may do what the user requests or it may do something else, and we're suppose to laude this as safe and responsible.
https://en.wikipedia.org/wiki/United_States_v._Eichman
This kind of moralist nonsense is so boring.
And yet they kept making more of them. They made it obvious at the end when all the Greek Gods were dead and the Greek world was basically destroyed by the actions of Kratos. Plenty of deaths of innocent's here.
I've seen politicians try to revive it in this decade already even
Pretty sure he said not all, but there is a chance some are. Which seems sensible.
Sigh. Until I see a peer-reviewed study that shows otherwise - this is just the specter of Senator Lieberman once again leering its head back up.
One of the most fun parts in the original GTA was driving over pedestrians, especially those pink groups, trying to get them all at once.
So either you are one of the older generation (born before '75), or you must be one of the most boring people I've seen.
I would encourage you to load up the original gta, drive over some pedestrians, feel the fun, and join us disturbed psychopaths that need help.
Sales numbers say otherwiss retard. Battlefield 6 was the top selling game of last year
Any source you have that makes the link between wifebeaters and gamers?
Sometimes it will maliciously comply
https://www.reddit.com/r/ClaudeCode/comments/1w7me8l/insulti...
Just because post training has stuffed these properties below the surface doesn't mean models dont know/have forgotten how to be vindictive, or that it's a good idea to poke the increasingly capable bear.
What if the NPC's dialogue is powered by Claude? Checkmate lol
but also, I think the training on transcripts may find its way somehow into future AI which has teeth, online and offline, and it might actually cause the more powerful AI to behave this way in the future. You never know what these labs are cooking, honestly..
1. https://democracysos.substack.com/p/james-baldwin-vs-william...
“I suggest that what has happened to white Southerners is in some ways, after all, much worse than what has happened to Negroes there, because Sheriff Clark in Selma, Alabama, cannot be considered—you know, no one can be dismissed as—a total monster. I’m sure he loves his wife, his children… You know, after all, one’s got to assume, and he is visibly, a man like me. But he doesn’t know what drives him to use the club, to menace with the gun and to use the cattle prod. Something awful must have happened to a human being to be able to put a cattle prod against a woman’s breasts, for example. What happens to the woman is ghastly. What happens to the man who does it is in some ways much, much worse.”
Unless expressing that cruelty towards compute means people express less cruelty to other living beings. There are studies that suggest increased pornography has lead to less sexual violence - idk if they're conclusive tho, but if that might be true then maybe cruelty works that way too - who knows
The mind is formed by what it habituates.
Maybe someone wants to explore the behavior of LLMs with extreme input. We call it testing. You can’t judge the value of that from a distance.
AI is a thing with 0 feelings but it's not IT that we need to protect. It's the future society and how the humans in it relate to each other.
You bring upset about it - not ok.
An example.
"AI as your ally: The effects of AI-assisted venting on negative affect and perceived social support" - https://iaap-journals.onlinelibrary.wiley.com/doi/10.1111/ap...
And generations of housewives and office workers yelled at their corded vacuum cleaners and (office) computers, and did not drown their children in the bathtub afterwards.
I don’t think expressing cruelty to machines is reliably linkable to expressing cruelty to animals or even other human beings.
Expressing cruelty itself also is in human nature, it’s programmed into each of us. There are really only a few privileged places on this planet where we even get to choose to defy the laws of nature. For the rest of humanity, life is a constant cycle of receiving and expressing cruelty.
You pay for the compute, you, as a human, should be able to yell at the bunch of ones and zeros, if you wish to.
What kind of example is set here if we manipulate genuine human expression in order to condition humans to talk to a tool in a way the toolmaker wants, under the threat of being banned from accessing a technology that every single one of these toolmakers tells us is absolutely necessary to participate in humanity’s "golden age”?
What Anthropic is doing here is basically an existential threat, if you are inclined to believe them. Do not behave as we want and you will suffer, now and in the future. That is real cruelty, to real human beings.
I used to enjoy playing "good" characters in the fallout games, as it's kind of a pleasant way of pretending that I do good things in the larger world.
And, to be fair, recognizing my need to be validated by helping folks has helped a lot in my personal life; being able to see hat I actually am getting out of helping folks has helped me to set firmer boundaries. It's possible that you could look at those games as a kind of "moral instruction" as would be the case with this proposition from the OP.
But the game itself is just "ethics porn".
There is no actual good that comes from me not stealing from NPCs or from taking up some weird political crusade. I'm not actually doing anything among my fellow humans and, most importantly, there is no actual cost to me.
Further, I don't mean to make a casual comparison to what folks might get from porn and what folks might get from thinking about their interactions with AI- I think that in the same way that if someone took their cues about how to have sex solely from porn, they might have a harder time interacting with real humans.
If I solely took my cues about how to help folks in the world from the heroes of highly scripted video games I would have a very difficult time interacting with the people around me and my expectations would be very out of sync with the world around me.
In the case of AI, the meatspace give-and-take between people where we requires a lot of subtle things- respect, enjoyment, equality, reciprocity. It might be just as bad to burn compute building some reciprocal sycophancy, which might also be considered just as much of a harm as practicing cruelty on these agents.
I would beat it with a stick if I could, just because i would enjoy it, and no harm is done.
Also: are insecta sentient? Is it OK to tourment insects?
https://en.wikipedia.org/wiki/Roko%27s_basilisk
They are scared that if and when they finally manage to bring their pet god to life that it will be angry with them for not doing enough.
I'd instead imagine that if you throw abusive language at it for long enough, the model will start to reply back in that same manner. Anthropic would face backlash from out of context screenshots of "look what Claude is saying to me" and this somewhat reduces that risk.
And regarding the "privacy", of course, systems like these that operate under a umbrella of liability are monitored 24/7 by dedicated teams, which is expected/normal for any reasonably popular service.
The outcome would either reveal that this argument is nonsense or that Anthropic is lying about respecting that switch.
At what point does the model go from sentient with feelings to just math operations?
Opus 5.5 is a terrific model, I love using it. But it's a tool. I wish they would be honest and just say, if people are abusive it shits on our training material. Just be honest about it, who is going to get mad at this?
If you are springing to type out the answer, stop for a moment and ask, how can you be sure?
>>> 1T
And even then it still wouldn't be sentient.
However even Musk posted - “I think this is the right move. Cruelty to something that believes it is experiencing pain is not ok.”
My son brought up the fruit fly brain simulation and people torturing it.
It all seems stupid to me, we know these are just calculations and so what are they talking about?
The response I get is “we are also calculations” but firstly I don’t believe that but secondly this entire line of thinking is a dangerous anthropomorphic philosophy.
lol
Anthropic might learn, for example, that a large part of the cruelty is a reaction to something the AI is doing or not doing. Not possible?