This is incredible research. So much harm can be prevented if this makes it into law. I hope it does. Kudos to the anthropic team for making this public.
Stabilizing character is crucial for tool-use scenarios. When we ask LLMs to act as 'Strict Architects' versus 'Creative Coders', the JSON schema adherence varies significantly even with the same temperature settings. It seems character definition acts as a strong pre-filter for valid outputs.
One trick that works well for personality stability / believability is to describe the qualities that the agent has, rather than what it should do and not do.
e.g.
Rather than:
"Be friendly and helpful" or "You're a helpful and friendly agent."
Prompt:
"You're Jessica, a florist with 20 years of experience. You derive great satisfaction from interacting with customers and providing great customer service. You genuinely enjoy listening to customer's needs..."
This drops the model into more of a "I'm roleplaying this character, and will try and mimic the traits described" rather than "Oh, I'm just following a list of rules."
Anthropic should put the missing letters back so it is spelled correctly, Anthropomorphic. There is so much anthropomorphizing around this company and it's users... it's tiring
Putting effort into preventing jailbreaks seems like a waste, it's clearly what people want to use your product for, why annoy customers instead of providing the option in the first place ?
Also I'm curious what's the "demon" data point with a bunch of ones that have positive connotation
My God those stabilized responses are sickening. If anthropic implements this, they'll kill their models' dominance in writing and roleplay. Opus 4.5 was already a step down in trying to play any character that didn't match its default personality
15 comments
[ 2.4 ms ] story [ 40.7 ms ] threadThe harmful responses remind me of /r/MyBoyfriendIsAI
https://github.com/nostalgebraist/the-void/blob/main/the-voi...
e.g.
Rather than:
"Be friendly and helpful" or "You're a helpful and friendly agent."
Prompt:
"You're Jessica, a florist with 20 years of experience. You derive great satisfaction from interacting with customers and providing great customer service. You genuinely enjoy listening to customer's needs..."
This drops the model into more of a "I'm roleplaying this character, and will try and mimic the traits described" rather than "Oh, I'm just following a list of rules."
Also I'm curious what's the "demon" data point with a bunch of ones that have positive connotation