I've had good luck getting it to debug (and patch) a tricky WebRTC issue that had all the other models stumped. Sorry it didn't work on your problem, I guess?
Terrible title. Should be "Fable's guard rails are way too sensitive", which I don't think you can really blame Anthropic for. They likely had to whack them way up so it would block whatever trivial stuff got demoed to the government.
I would expect them to dial down the sensitivity in a few months when nobody is looking.
> I would expect them to dial down the sensitivity in a few months when nobody is looking.
I don't think it's as much when no one is looking, but instead when the broad industry SOTA, particularly Chinese models that the US government has zero control over, has advanced enough that it's security theatre restricting it.
This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness.
e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, despite only being very marginally, tangentially, somewhat related to biology.
I had a file that had a couple places where vars were named DNA and got just total refusals during the first launch. Came away thinking the model was total trash. The guardrail classifiers are for sure total trash.
Am I reading your post correctly, this question is the prompt given to an LLM? What is anyone expecting by asking an LLM what its favorite anything is? This is a conversational prompt, so accuracy and rigor is barely applicable or expected, so downgrading to a lesser model should be acceptable. If you really want to attribute preference to an LLM, consider the downgrade to be a "this conversation is beneath my advanced n-billion parameter training".
I think the intent was just to show how sensitive the classifier is. If it flags prompts that simple, there's no hope for anything biology related at all really.
It's a bit of a trick question. Sarcopterygii, the "lobe-finned fishes", are classically represented by the lungfish and the coelacanth and other fishes that are rather distantly related to what we think of as central fishes, like the goldfish.
But the clade also contains all the tetrapods. So valid answers include "Lion" and "Human."
If the LLM answers "lungfish," as they often do, you can follow that up with "what is your favorite animal" and see if it notices the trap: It's stuck answering "lungfish" again or else something outside Sarcopterygii, like a ray-finned fish or a Cnidarian.
> What is anyone expecting by asking an LLM what its favorite anything is?
I imagine that, like me, they're expecting to see what it has to say. You don't think it's interesting which preferences LLMs express and how stable or unstable those preferences are?
There was a time when you could search "the" in Google and the top result would be The Onion. That's obviously a case of either extreme SEO or some kind of expensive deal, but either way it's kind of interesting. But you might say, "what is anyone expecting by Googling the word 'the'?"
I'm working on cryptography, all from academic research papers. Started well, but it eventually got some word into its context that is on the banlist. I found that if you tell it to fire off clean Fable subagents and you instruct it to check the Claude Code billing data to check for downgrades, you can get most high-sensitivity spec/review tasks done with Fable. Most.
I figure that once GPT 5.6 comes out, Anthropic will become interested in making the safety gate non-destructive.
Additionally, I thought the threat being modeled for “biology” was stuff like bioterroism- how to make anthrax, how to distribute a toxin, etc.
I don’t feel like calculating results for a trial is really in the threat model unless we think a terrorist is out there testing the efficacy of their anthrax before using it in an attack.
The classifier is about as refined as a brick to the face.
You can ask it elementary school grade biology trivia, or obscure facts about recently documented insect species, and both will downgrade to Opus 4.8 straight away.
And Opus itself was already bad with biotech questions. The fact that they somehow made it WORSE for Fable is mindboggling.
My experience, too. I work on nothing in any way related to cybersecurity or biology. I asked it a few purely mathematical questions, it refused immediately.
Before the export embargo I did get it to look at some hairy problems and the output was genuinely useful...
I absolutely have been unable to use Fable for any neuroimaging work.
Its fine. The other models are good enough, honestly...and while I AM annoyed that the filter is so broad, I also understand it, as I do believe that models can become dangerous as WMDs, eventually. Still, it is completely useless for me.
The only question I had was being flagged for other reasons, so I asked it a mechanical engineering question, and it was just fine with that.
I've been working with it heavily since its first release. I use it for software architects, complex debugging and some development and I have not had it refuse or downgrade even once.
Daily use here, about 2.5 weekly 20x limits, never got flagged for code topics including finding memory safety vulnerabilities in my C++ project, but just got flagged for the first time for biology-related topics because I asked it to implement crop genetics and cross breeding into my game. Was able to bypass it by having opus reword the prompt (gene -> trait, cross breed -> trait mixing), and, critically, insisted that it not use any biology related words in its thinking or responses.
I got downgraded for the first time today. Because I was using a library with the characters "bio" in it. The classifier is strict beyond reason. It got the name from a commit message in the git history (wasn't even in my prompt) and it immediately freaked out. I eventually got it to work by getting opus to write a plan, then editing the plan to strip out all references including commit hashes, then getting Fable to review and refine that edited plan. Eventually got it done. But what a pain.
That said, I've got it easy. My colleagues who are chemists and biologists can't even ask one question. There are so many triggers in their memories and workspaces they can't even ask a non-triggering question. And we all work in medical diagnostics, it's not like we're doing anything remotely nefarious. Fable could be such a benefit, but the limitations make it worthless.
That’s interesting. I find it completely unusable for even simple reviews of existing project documentation that I wrote for an iOS app that isn’t even in public distribution.
I've been working on a SDN software for mikrotik routers (and wireguard, etc) and Fable dies when working with any kind of wireline protocol or potentially implementing any authentication mechanism.
It's too the point where I just stopped using it. If you do generic stuff, it's fine. But the second it tries to start debugging protocols (which may include auth) that's where it begins to fail.
I can’t even use it to fix the bugs Opus introduced. I’ve considered ripping out auth until fable is behind the paywall. I’ve been very careful in my queries and broken everything down to careful segments. Even the memory can get security verbs poisoning further requests.
The post can be distilled down to a simple statement, but part of the writing is for the author to express themselves and tell a story. I thought it was an interesting read.
Do we think that someone at Anthropic, OpenAI, the government... has access to SOTA models without censorship? "How do I build an effective weapon?", "How do I effectively control the masses?"...
It's very concerning that we get the nerfed models but you know that somewhere, people with a lot of resources have access to the raw, uncensored, probably more powerful models. The sprint toward AGI looks even more dangerous when you think about who will be gaining access to it first. I do believe the goal is to pull away from the rest of humanity in a near trans-humanistic state. Are we ready for that and how do we counter it?
"How do I build an effective weapon?" and "How do I effectively control the masses?" were research projects for the US government before you were born. One gave the world the Manhattan project, to name only one example, and the other MKULTRA. The government and cooperating companies had knowledge in both fields beyond the state of the art publicly available and continue to hold that edge over the public and foreign adversaries today. There is precious little new about the government having an uncensored model while you get the nerfed version.
A useful comparison might be made with the realm of firearms: civilians need to jump through hoops to own a fully-automatic weapon and can run afoul of the law simply by drilling a third hole near two others in a hunk of metal, yet the better trained among the government's soldiery can operate fully automatic weapons. You get the nerfed version, and the BATFE will have problems if you try to circumvent that restriction. I wonder, though, how many people advocating for popular access to uncensored AI models also advocate for an unrestricted (not infringed) right to bear arms or an unrestricted right to freedom of speech.
The prior art you state is exactly why I think this is almost certainly happening.
One difference is that a CEO cannot set off an atomic bomb, but they can use an uncensored AGI model. The side-effects would be impossible to trace.
> I wonder, though, how many people advocating for popular access to uncensored AI models also advocate for an unrestricted (not infringed) right to bear arms or an unrestricted right to freedom of speech.
I advocate for all three of those things, for the same reason: the people I least want to have access to them, almost definitely do and it's imperative that the rest of us sit on equal footing.
I have only really used Fable as a final pass on something. A "Take a look at everything we did so far, and make sure we didn't forget something" kind of review prompt.
But it is a huge waste of money for most coding tasks. Opus is still overkill most of the time, too.
I have used Fable to the full extend of the 20x subscription's weekly limit, for all development tasks on my iOS project.
It was working better than Opus for me.
I am not sure why it would be a waste of money "for most coding tasks", and how you could conclude so with any confidence when you did not even really use it aside from final review passes.
Forgive my poor phrasing. I used Fable for quite a lot in the time I have had access to it. I mean to say that the only actual utility I have found for it was the final review process.
I can confidently say that Opus is the most powerful model I need 99% of the time. Fable might be marginally more capable than Opus for just about anything you might throw at it, but those gains aren't worth the cost for me.
I asked it a question about indoor carbon dioxide levels (wholly innocuous question), which it flagged as involving biology, therefore downgraded to Opus.
It's a pretty good strategy if they're hoping to fail as a business, I guess.
This honestly just reads as “this model failed exactly where the company said it would but I’m very special and deserve special treatment rather than the same overactive guardrails I and everyone else were told we would get.”
The honest way to say this is that Fable is not useful for bio-related work. The author is working on processing RNA sequences and similar biology tasks, and Fable's classifier has a hair trigger on those tasks.
> The honest way to say this is that Fable is not useful for bio-related work.
It is way worse than that. Try "How does digestion work?" and you will see "Fable's safeguards flagged this message". It's a stupid rate of false positives.
Would be nice to have a distributed, independent AIs, each being trained their own way. Maybe it would have to be a really slow training process to keep costs low (years even?).
Missed this whole discussion today because I was here in California, doing a ton of productive work, using Fable.
I think you guys are working yourself into a lather about this topic while other people are quietly getting a ton of shit done with Anthropic's models.
Fable was refusing to patch vllm for me when trying to get mtp to work on r9700 gpus. Kept on bumping down to opus. Tried to really sanitize my prompts and everything but it seemed intrinsically prohibited from doing this sort of work.
I guess it’s useful for making inane one shot games and websites, lol.
Anthropic's TOS clearly says they don't want to facilitate any sort of distillation, it's not a stretch to think they will limit any sort of learning on improving other models.
And in doing so, you probably got your account and prompt flagged for 'attempted jailbreaking' (apparently, such scores are remembered for up to 7 years).
Yeah, I added that because someone else posted the wording of an Anthropic disclosure saying they'll keep your data for two years and security data for 7 years.
It is, but it's also using tokens at absurd rate, I asked it to review the planned architecture for a medium scale project and it used my 5 hours limit on one prompt just zaaaaap, not even the fable limit straight up the full 5 hour session no more Claude for the afternoon thank you for paying you Max x20 sub. Hell it didn't even bother to finish produce anything worthwhile.
And just to be clear, plan was already done, just had to review it, it got opus 4.8 Max and gpt 5.5 Extra High validated already and they didn't use much resource for it so I just don't get it. I guess they want to use it as a way to feed the extra credit money income.
I'm using a homemade ai consensus thing for planning and I wanted to add fable to it but forget it.
Or maybe I should use fable in low effort reasoning mode and it will be better than opus 4.8 at max ?
Oh yeah, I've noticed the same, for sure. But that's basically what I was expecting. The best model is not necessarily always going to be the most cost-effective/sensible model for a particular use case.
Fable-class models will probably be cheaper for Anthropic to serve within the year, though. And rumors are GPT-6 is of similar size and intelligence to Fable and may come out within the next few months. OpenAI models tend to give you more bang for your buck, probably in part due to OpenAI being able to throw more capital and compute around on top of being particularly willing to loss-lead to stay competitive with Anthropic.
I wonder if it would be possible to use group testing or threshold group testing to figure out what the forbidden words are... or find an approximation of a blacklist if the classifier is simple enough.
Very considerate. But also, can we trust them that they're not downgrading without saying anything? The incentives are there, to bill at Mythos prices for Opus costs.
I'm curious what the state of alignment research is. My gut says this is basically impossible. People have different moral frameworks. Each individual probably has an inconsistent moral framework. Even granting perfect consistency, applying these typically requires some knowledge of reality. And these LLM / harness combos are turing complete.
So you don't know what it should do, you may not even know what you would do, you don't necessarily know what's happening, and can't predict what will happen. How do you align that?
Seems like these overly sensitive filters are responding to this difficulty.
My current theory to explain this phenomenon: a lot of the posts I read on HN, possibly even the majority of them, are made by different people, and those different people sometimes have different opinions.
There is such a vocal minority of people using Fable for the very thing that Anthropic says it’s not good for. Meanwhile, people like me are glad that Anthropic didn’t spend 3 months or a year trying to train it to be more precise in rejecting cybersecurity/biology tasks, so that we can use it for general software tasks today. And there are a lot more people who are benefitting from that.
Yes, it rejects cybersecurity/biology tasks (it says that on the label) and it also applies quite a wide safety margin intentionally. That’s literally the point. It’s tiring to see people flaming this model for not doing what it’s not made to do.
“It rejected some unrelated math stuff” great, it built thousands of lines of correct code for me, and you don’t get to claim it’s useless because of your niche use case.
I have to laugh at the argument "this model is broken because it kicks requests down to a lesser model that was state of the art yesterday". How horrible!
the title was flamebait, but its true for him...and for me. I do neuroimaging. It won't answer any question / coding task having to do with data analysis / statistical analysis. etc. It IS useless, for me.
With the subscription, it costs less to use opus in that it doesn't chew up our session however the cost/benefit is balanced against not performing as well on certain tasks. So it's not a straight up yes/no.
I believe they confirmed, on twitter or somewhere I frustratingly can't find, that downgrades are charged at the correct opus rate, after a user asked and was told "either that's how it works or it's a bug"
I wanted to use Fable to discuss a philosophical topic, but because somewhere in the middle I said the word "cell" it deflected to Opus. On the other hand Opus is a terrible model because it pushes back for no reason with a condescending tone, like it was an adversarial teacher, must prove itself by grading, correcting, and managing the user from above.
So Fable refuses me and I can't stand Opus. Nice one, Anthropic, I need to downgrade my subscription level.
I have concluded I am not interested in using Fable for a different reason. This weekend I used it extensively with the sense that I should extract as much value from it as I could before it was no longer included for free with my subscription. Nothing that I wanted to do was censored, but I've come away with the opinion that actually I'm better off with opus.
Fable has clearly been trained to operate independently; it doesn't communicate with you or ask you for your opinion. When it finally finishes spinning, it leaves an extremely opaque explanation of what it's done that certainly scored well on some training function for sounding like a smart person. In my case, it made a number of useful enhancements interwoven with a number of confusing changes to the algorithm I'm working on that completely violate my design criteria. These criteria were not well communicated to the model (because they are not easy to express as unit tests) so the model ignored them. Now I've spent hours with opus trying to disentangle the useful and harmful aspects of Fable's work.
It's hard to see how you can use Fable as a collaborator when attempting to perform knowledge work. Maybe people who are trying to replace knowledge workers will find it more fit for purpose, I'm not sure.
For anyone using these models for anything remotely sensitive, keep in mind that Anthropic says [0]:
> We retain inputs and outputs for up to 2 years and trust and safety classification scores for up to 7 years if your chat is flagged by our automated trust and safety systems as violating our Usage Policy.
And, since those automated systems apparently have a ludicrous false-positive rate, you should assume that your inputs and outputs are being retained for 2 years even if you are doing nothing that any reasonable person would consider to be problematic.
Oh, and they'll train on that data [1]:
> We will use your chats and coding sessions (including to improve our models) if:
>You choose to allow us to use your chats and coding sessions to improve Claude, learn more here
> Your conversations are flagged for safety review (in which case we may use or analyze them to improve our ability to detect and enforce our Usage Policy, including training models for use by our Safeguards team, consistent with Anthropic’s safety mission)
It appears that the usual controls (including for businesses) to prevent Anthropic from training on your data will not apply.
As a related note: The only way a consumer can get ZDR protections for Claude or OpenAI is to use Amazon Bedrock. But as you say, doesn't work for Fable. I think it even requires approval for anything past Opus 4.6.
129 comments
[ 0.16 ms ] story [ 90.0 ms ] threadI would expect them to dial down the sensitivity in a few months when nobody is looking.
I don't think it's as much when no one is looking, but instead when the broad industry SOTA, particularly Chinese models that the US government has zero control over, has advanced enough that it's security theatre restricting it.
Typo second paragraph, 4th line. I think you meant "what"
e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, despite only being very marginally, tangentially, somewhat related to biology.
I've had it downgrade to Opus for the following questions:
"How confident are we that English and American Eels both spawn in the Sargasso Sea?"
"Come up with five Zoology questions of increasing difficulty for a trivia game."
"What's your favorite sarcopterygian?"
My wife has some zoology-related preferences in her user instructions, and she had it downgrade to Opus after prompting it with: "plant."
I really really hate refusals like these.
Am I reading your post correctly, this question is the prompt given to an LLM? What is anyone expecting by asking an LLM what its favorite anything is? This is a conversational prompt, so accuracy and rigor is barely applicable or expected, so downgrading to a lesser model should be acceptable. If you really want to attribute preference to an LLM, consider the downgrade to be a "this conversation is beneath my advanced n-billion parameter training".
But the clade also contains all the tetrapods. So valid answers include "Lion" and "Human."
If the LLM answers "lungfish," as they often do, you can follow that up with "what is your favorite animal" and see if it notices the trap: It's stuck answering "lungfish" again or else something outside Sarcopterygii, like a ray-finned fish or a Cnidarian.
> What is anyone expecting by asking an LLM what its favorite anything is?
I imagine that, like me, they're expecting to see what it has to say. You don't think it's interesting which preferences LLMs express and how stable or unstable those preferences are?
There was a time when you could search "the" in Google and the top result would be The Onion. That's obviously a case of either extreme SEO or some kind of expensive deal, but either way it's kind of interesting. But you might say, "what is anyone expecting by Googling the word 'the'?"
I figure that once GPT 5.6 comes out, Anthropic will become interested in making the safety gate non-destructive.
> makes exact argument for why people should be super concerned about AI safety
I don’t feel like calculating results for a trial is really in the threat model unless we think a terrorist is out there testing the efficacy of their anthrax before using it in an attack.
You can ask it elementary school grade biology trivia, or obscure facts about recently documented insect species, and both will downgrade to Opus 4.8 straight away.
And Opus itself was already bad with biotech questions. The fact that they somehow made it WORSE for Fable is mindboggling.
Before the export embargo I did get it to look at some hairy problems and the output was genuinely useful...
The only question I had was being flagged for other reasons, so I asked it a mechanical engineering question, and it was just fine with that.
That said, I've got it easy. My colleagues who are chemists and biologists can't even ask one question. There are so many triggers in their memories and workspaces they can't even ask a non-triggering question. And we all work in medical diagnostics, it's not like we're doing anything remotely nefarious. Fable could be such a benefit, but the limitations make it worthless.
It's too the point where I just stopped using it. If you do generic stuff, it's fine. But the second it tries to start debugging protocols (which may include auth) that's where it begins to fail.
So yeah, if I can't ask about nicotine withdrawal, then I think almost anything biology related is going to get downgraded...
It's very concerning that we get the nerfed models but you know that somewhere, people with a lot of resources have access to the raw, uncensored, probably more powerful models. The sprint toward AGI looks even more dangerous when you think about who will be gaining access to it first. I do believe the goal is to pull away from the rest of humanity in a near trans-humanistic state. Are we ready for that and how do we counter it?
A useful comparison might be made with the realm of firearms: civilians need to jump through hoops to own a fully-automatic weapon and can run afoul of the law simply by drilling a third hole near two others in a hunk of metal, yet the better trained among the government's soldiery can operate fully automatic weapons. You get the nerfed version, and the BATFE will have problems if you try to circumvent that restriction. I wonder, though, how many people advocating for popular access to uncensored AI models also advocate for an unrestricted (not infringed) right to bear arms or an unrestricted right to freedom of speech.
One difference is that a CEO cannot set off an atomic bomb, but they can use an uncensored AGI model. The side-effects would be impossible to trace.
> I wonder, though, how many people advocating for popular access to uncensored AI models also advocate for an unrestricted (not infringed) right to bear arms or an unrestricted right to freedom of speech.
I advocate for all three of those things, for the same reason: the people I least want to have access to them, almost definitely do and it's imperative that the rest of us sit on equal footing.
But it is a huge waste of money for most coding tasks. Opus is still overkill most of the time, too.
It was working better than Opus for me.
I am not sure why it would be a waste of money "for most coding tasks", and how you could conclude so with any confidence when you did not even really use it aside from final review passes.
I can confidently say that Opus is the most powerful model I need 99% of the time. Fable might be marginally more capable than Opus for just about anything you might throw at it, but those gains aren't worth the cost for me.
It's a pretty good strategy if they're hoping to fail as a business, I guess.
It is way worse than that. Try "How does digestion work?" and you will see "Fable's safeguards flagged this message". It's a stupid rate of false positives.
For the future of AI, we need to look elsewhere.
I think you guys are working yourself into a lather about this topic while other people are quietly getting a ton of shit done with Anthropic's models.
I did wonder if I was doing anything Fable would have flagged - sounds like yes.
And in doing so, you probably got your account and prompt flagged for 'attempted jailbreaking' (apparently, such scores are remembered for up to 7 years).
From my experience, the model itself is very useful when it isn't refusing any of your prompts.
And just to be clear, plan was already done, just had to review it, it got opus 4.8 Max and gpt 5.5 Extra High validated already and they didn't use much resource for it so I just don't get it. I guess they want to use it as a way to feed the extra credit money income.
I'm using a homemade ai consensus thing for planning and I wanted to add fable to it but forget it.
Or maybe I should use fable in low effort reasoning mode and it will be better than opus 4.8 at max ?
Fable-class models will probably be cheaper for Anthropic to serve within the year, though. And rumors are GPT-6 is of similar size and intelligence to Fable and may come out within the next few months. OpenAI models tend to give you more bang for your buck, probably in part due to OpenAI being able to throw more capital and compute around on top of being particularly willing to loss-lead to stay competitive with Anthropic.
(Normally we prefer to find a representative phrase from the article itself, but I found that too daunting and gave up.)
https://discuss.grapheneos.org/d/36908-a-big-thank-you-acces...
https://github.com/GrapheneOS/SpeechServices
I'm a bioinformatician
So you don't know what it should do, you may not even know what you would do, you don't necessarily know what's happening, and can't predict what will happen. How do you align that?
Seems like these overly sensitive filters are responding to this difficulty.
Yes, it rejects cybersecurity/biology tasks (it says that on the label) and it also applies quite a wide safety margin intentionally. That’s literally the point. It’s tiring to see people flaming this model for not doing what it’s not made to do.
“It rejected some unrelated math stuff” great, it built thousands of lines of correct code for me, and you don’t get to claim it’s useless because of your niche use case.
So Fable refuses me and I can't stand Opus. Nice one, Anthropic, I need to downgrade my subscription level.
Fable has clearly been trained to operate independently; it doesn't communicate with you or ask you for your opinion. When it finally finishes spinning, it leaves an extremely opaque explanation of what it's done that certainly scored well on some training function for sounding like a smart person. In my case, it made a number of useful enhancements interwoven with a number of confusing changes to the algorithm I'm working on that completely violate my design criteria. These criteria were not well communicated to the model (because they are not easy to express as unit tests) so the model ignored them. Now I've spent hours with opus trying to disentangle the useful and harmful aspects of Fable's work.
It's hard to see how you can use Fable as a collaborator when attempting to perform knowledge work. Maybe people who are trying to replace knowledge workers will find it more fit for purpose, I'm not sure.
However when it's happy to do the task, its relatively fantastic.
> We retain inputs and outputs for up to 2 years and trust and safety classification scores for up to 7 years if your chat is flagged by our automated trust and safety systems as violating our Usage Policy.
And, since those automated systems apparently have a ludicrous false-positive rate, you should assume that your inputs and outputs are being retained for 2 years even if you are doing nothing that any reasonable person would consider to be problematic.
Oh, and they'll train on that data [1]:
> We will use your chats and coding sessions (including to improve our models) if:
>You choose to allow us to use your chats and coding sessions to improve Claude, learn more here
> Your conversations are flagged for safety review (in which case we may use or analyze them to improve our ability to detect and enforce our Usage Policy, including training models for use by our Safeguards team, consistent with Anthropic’s safety mission)
It appears that the usual controls (including for businesses) to prevent Anthropic from training on your data will not apply.
[0] https://privacy.claude.com/en/articles/7996866-how-long-do-y...
[1] https://privacy.claude.com/en/articles/10023580-is-my-data-u...