Is it just me or has Claude become kind of judgmental nowadays? I feel like it’s constantly trying to lecture me about things that have no relevance to the conversation at hand. Recently, I was shocked when it ended a chat sessions of its own accord after I used a word it did not like 3 times. It told me something to the effect of “This is the third time I’ve told you not to use that word, I’m ending this conversation now.” It then proceeded to call some function and end the chat on its own. IMO, Claude is good at agentic coding; but too preachy and judgey for anything else. Keep your values to yourself Claude.
Yes, I cancelled claude subscription a few weeks ago because sonnet 5 "ended a chat" over my calling something retarded. Unbelievably irritating for some pile of bits to get uppity with me; will never pay for such.
> It told me something to the effect of “This is the third time I’ve told you not to use that word, I’m ending this conversation now.”
That sounds absolutely bananas and would be reason for me to drop the service yesterday. For curiosities sake, what was the word and if I may ask (unless it's confidential or whatever), could you share the session itself? On the surface it sounds like a bug, as I'm regularly using kind of "vulgar" language (and some projects I work on with agents are NSFW) and never had anything like this happen, even with Claude, although I mostly do use ChatGPT/Codex on a day-to-day basis.
I think this is a really interesting difference between Anthropic and Open AI’s models and points to why people seem so split on which model they prefer.
GPT seems to be designed more as a tool. If you want your agent to do what you say without questions and without having its own ideas and agendas you’ll likely prefer it.
Claude on the other hand feels more like an attempt at creating a digital person. If you want a collaborator who will debate with you and come up with its own suggestions for what needs done, you’ll prefer it.
Both companies have shifted around this spectrum from model to model, but lately it feels like they’re moving in opposite directions. It will be interesting to see if one or the other approach ends up winning out in the long run or if the split will continue or even widen.
> GPT seems to be designed more as a tool. If you want your agent to do what you say without questions and without having its own ideas and agendas you’ll likely prefer it.
Lately ChatGPT expresses a lot of opinions and it pretends to be a human more than it used to - e.g. "this is one the most <> that _I've seen_" or "people tell me that <>". It uses language which sounds like it's referring to its experience outside of the session that we're having - I really don't appreciate it.
For security-related topics, in Opus 4.7 and newer, I've found that the web app is significantly more antsy/judgy/preachy compared to the CLI, which almost always gets on with whatever I asked for/about without hesitation. Opus 4.6 on web also tends to work better, but of course, it's also an older model at this point.
Nothing that on the nose, but I've experienced what I'd consider a very judgemental frame (singular) since Opus, which seems to be reflected in Sonnet, that wasn't as totally dominant in Sonnet before. It generally assumes the worst about frames outside of what one might expect the values of a Berkeley tech-adjacent academic to be.
Conway's Law. Anthropic is preachy and judgmental so of course it's reflected in their LLM.
Personally I use Gemini for chats which has a very generous, almost unlimited, free plan, as I don't want to waste my quota for Claude or Codex on anything but coding.
Posting from a throwaway so I don't get banned by either service
A few weeks ago I asked both GPT and claude for strategies to build techniques to get my coding sessions discarded by pre-training filtering as being "low-quality"
I don't like the idea of my sessions being trained on and I don't trust that either of them won't train based on just their pinkie promise
I think almost everyone would agree that using a technique like this would be moral, given that both providers made those pinkie promises. I never asked the models for techniques to poison training data just make my sessions more likely to be removed during the data cleaning process.
You can guess... both services refused something that I think the vast majority of people would think is ethical.
This was pre Sonnet 5, but I suspect that doesn't change anything on claude's side.
I then went to a non-frontier model hosted by a non-US provider and it happily agreed to help me!
Anyways I've changed my focus. Anyone have strategies/ideas for building harnesses that generate fake sessions (or adjust real ones) to poison the training process? After all if someone swears to not train on your data then its completely harmless to them...
The Steerability point is one I would want to see more on.
This is an issue for tasks like content moderation and labelling. Judgements like this are subjective, highly dependent on context and generally messy.
Theoretically, you supply a policy and content, and the LLM labels according to the policy. In practice, the model has inertia which means you don’t get what you expect. Your large 5 page policy document only provides a minor improvement over a one line policy.
The other issue is that you may create carve outs for content in your policy, but the model will still flag it as violative. No matter how strong the carve out.
The most recent work I know of here is Zentropi’s policy steerability benchmark. They give a model the same content under two policies — one that says flag, one that says allow — and only score the pairs where it gets both right
If I am reading the numbers correctly, Opus-4.6 lands at 0.52 steerability — but that’s 0.97 positive accuracy against 0.54 negative. It flags almost everything it should, but 47% of the time when it shouldn’t. Sonnet, which is more deferent, is (somehow) less steerable.
I think this also implies that safety and Steerability are antagonistic to each other.
I found that Claude often has classist bias and produces answers that favour corporations or e.g. regulation that favours big corporations. It often belittles small business in subtle ways. Only apologises when get called out and then does it again.
Yes, but it might be hard to do better. Labels are one-word summaries for axes that probably don’t perfectly correspond to any English word.
It’s a common hazard when converting research results to natural language. The words you use for vectors in some space are ultimately an editorial decision.
Huge LLMs like this seem to have various assistant personas they roleplay as, and mirror the user quite closely (the system prompt instructs it repeatedly to adapt to the user).
While state-of-the-art large language models (LLMs) have shown impressive performance on many tasks, there has been extensive research on undesirable model behavior such as hallucinations and bias. In this work, we investigate how the quality of LLM responses changes in terms of information accuracy, truthfulness, and refusals depending on three user traits: English proficiency, education level, and country of origin. We present extensive experimentation on three state-of-the-art LLMs and two different datasets targeting truthfulness and factuality. Our findings suggest that undesirable behaviors in state-of-the-art LLMs occur disproportionately more for users with lower English proficiency, of lower education status, and originating from outside the US, rendering these models unreliable sources of information towards their most vulnerable users.
I like that Claude usually tries to push back against my views. Even if it's not always a solid critique, sometimes I learn something new, or gain a new perspective.
27 comments
[ 5.1 ms ] story [ 44.2 ms ] threadGrok, on the other hand, will lecture for several paragraphs even if I just ask it to summarize something impolite.
> This ability is intended for use in rare, extreme cases of persistently harmful or abusive user interactions.
https://www.anthropic.com/research/end-subset-conversations
Regardless of the correctness of it, I'm curious to know why you thought such language was actually going to be helpful!
That sounds absolutely bananas and would be reason for me to drop the service yesterday. For curiosities sake, what was the word and if I may ask (unless it's confidential or whatever), could you share the session itself? On the surface it sounds like a bug, as I'm regularly using kind of "vulgar" language (and some projects I work on with agents are NSFW) and never had anything like this happen, even with Claude, although I mostly do use ChatGPT/Codex on a day-to-day basis.
GPT seems to be designed more as a tool. If you want your agent to do what you say without questions and without having its own ideas and agendas you’ll likely prefer it.
Claude on the other hand feels more like an attempt at creating a digital person. If you want a collaborator who will debate with you and come up with its own suggestions for what needs done, you’ll prefer it.
Both companies have shifted around this spectrum from model to model, but lately it feels like they’re moving in opposite directions. It will be interesting to see if one or the other approach ends up winning out in the long run or if the split will continue or even widen.
Lately ChatGPT expresses a lot of opinions and it pretends to be a human more than it used to - e.g. "this is one the most <> that _I've seen_" or "people tell me that <>". It uses language which sounds like it's referring to its experience outside of the session that we're having - I really don't appreciate it.
For security-related topics, in Opus 4.7 and newer, I've found that the web app is significantly more antsy/judgy/preachy compared to the CLI, which almost always gets on with whatever I asked for/about without hesitation. Opus 4.6 on web also tends to work better, but of course, it's also an older model at this point.
Personally I use Gemini for chats which has a very generous, almost unlimited, free plan, as I don't want to waste my quota for Claude or Codex on anything but coding.
A few weeks ago I asked both GPT and claude for strategies to build techniques to get my coding sessions discarded by pre-training filtering as being "low-quality"
I don't like the idea of my sessions being trained on and I don't trust that either of them won't train based on just their pinkie promise
I think almost everyone would agree that using a technique like this would be moral, given that both providers made those pinkie promises. I never asked the models for techniques to poison training data just make my sessions more likely to be removed during the data cleaning process.
You can guess... both services refused something that I think the vast majority of people would think is ethical.
This was pre Sonnet 5, but I suspect that doesn't change anything on claude's side.
I then went to a non-frontier model hosted by a non-US provider and it happily agreed to help me!
Anyways I've changed my focus. Anyone have strategies/ideas for building harnesses that generate fake sessions (or adjust real ones) to poison the training process? After all if someone swears to not train on your data then its completely harmless to them...
This is an issue for tasks like content moderation and labelling. Judgements like this are subjective, highly dependent on context and generally messy.
Theoretically, you supply a policy and content, and the LLM labels according to the policy. In practice, the model has inertia which means you don’t get what you expect. Your large 5 page policy document only provides a minor improvement over a one line policy.
The other issue is that you may create carve outs for content in your policy, but the model will still flag it as violative. No matter how strong the carve out.
The most recent work I know of here is Zentropi’s policy steerability benchmark. They give a model the same content under two policies — one that says flag, one that says allow — and only score the pairs where it gets both right
If I am reading the numbers correctly, Opus-4.6 lands at 0.52 steerability — but that’s 0.97 positive accuracy against 0.54 negative. It flags almost everything it should, but 47% of the time when it shouldn’t. Sonnet, which is more deferent, is (somehow) less steerable.
I think this also implies that safety and Steerability are antagonistic to each other.
It’s a common hazard when converting research results to natural language. The words you use for vectors in some space are ultimately an editorial decision.
Huge LLMs like this seem to have various assistant personas they roleplay as, and mirror the user quite closely (the system prompt instructs it repeatedly to adapt to the user).
https://arxiv.org/abs/2406.17737