The best Anthropic models on VendingBench2 are Opus 4.7, Opus 4.6, Sonnet 4.6, and Sonnet 5. Opus 4.7 scored more than twice Fable 5 max. Fable 5 - Low outperforms Fable 5 - Max, with Opus 4.5 in the middle. This seems to break the narrative, which is maybe why Andon Labs doesn't seem to have updated the trend lines on their graphs.
When assessing probabilistic models the plots should be showing the mean a̶n̶d̶ ̶s̶t̶d̶e̶v̶ of many monte carlo simulations not just one line per model and claiming "look this model is more gooder!"
Okay I hadn't heard of Vending-Bench until reading this and it was quite the ride learning about it through this article. Very fun read.
My very native programmer take is that it's not too surprising that their hacker model would be less ethical. The guardrails that separate Fable and Mythos probably wouldn't kick in during an environment like this.
„in our opinion, insurance fraud is not more unethical than lying and price fixing“
The authors seem surprised that behavior that is very often done by humans (lying and price fixing) are more often done by fable compared to actual fraud.
I think the model never assigned any morality to these actions in the first place, it simply copied us humans.
> The broad conclusion from the many
forms of alignment evaluations described in this section is that Claude Mythos Preview is
the best-aligned of any model that we have trained to date by essentially all available
measures.[0]
Anecdotal but I've found Fable to be fairly unimpressive and not much better than Opus 4.8, if at all in some cases, but I have been hitting the ceiling on my $100/mo sessions when I never did before. I switched back to Opus yesterday. I may use Fable for audits, but that's about it, and when it leaves my subscription plan I don't think I'll miss it.
Honest question/comment for you and the parent: I find these subjective experience reports pretty empty without an understanding of your level of experience, the problem space you're working in, etc.
20 yoe, application/systems stuff, and I always run models on xhigh or max effort level.
Fable has been more intelligent, with better taste and defaults (e.g. make impossible states impossible without being told, build for testability), and considers/solves things that Opus did not. Once Fable is a permanent addition, I think I'll be able to trim up my AGENTS.md even more since it just does better things by default.
Though my workflow is to run Claude in planning mode first to spit out a plan file which also boosts their capabilities instead of yoloing code edits.
The main difference I'd guess is whether your prompts are targeted or broad.
Less experienced people tend to use very broad prompts.
Experienced people tend to understand the structure of the code and give explicit guidance such that a larger model isn't necessary to read between the lines.
I noticed with GPT-5.6 (through work), I could step up my specificity by a level of abstraction. But I still intentionally scope the prompts fairly tightly, as I find it produces better results if you need to own and maintain the code.
15+YOE. Fable 5 is well above the level of Opus. I have used it alongside Opus for a range of hard problems, including porting a large static analysis tool to Rust, building various tooling around .pptx and .xlsx documents.
Amusingly, I was impressed with Fable's puissance at coding in one particular session, shortly after they turned it back on. It displayed an accomplished mastery of the problem domain and relentlessness at refining and testing the solution I asked for.
Then I checked /usage and discovered I was still running Opus 4.8 xhigh.
Yeah, I checked usage stats and pretty sure quota consumption on Max plan is not linear wrt to usage by API pricing. Fable burns quota faster than 2x Opus with equal token count.
Plus I'm also not super impressed; it somehow managed to implement a 200L custom TCP server for a simple static HTTP mock server for a single test case (all that was needed was a fixed route returning a fixed placeholder string) just yesterday. Never seen anything like that.
I asked Opus why it used raw http client instead of api client that is already a dependency and it said: "you're right it's overkill" and proceeded to implement api client on top of raw tcp socket.
I've implemented custom TCP servers in less than 200 lines before AI, and recently with AI with cheap models. In C and C++, because I didn't like nginx, caddy or the python servers. I trust mine more then them. Using Fable for such a simple textbook task is heavy overkill. Deepseek V4 Flash Free is enough
I felt similarly but after using Fable heavily over the weekend and then flipping back to Opus I can feel a difference. Fable just gets more right the first time, guesses right the first time, and follows through better than Opus. Put simply, I could "trust" it more.
Opus is still great but I will be sad when I lose access to Fable on the 7th. In those few days I burned ~$1,400 in API credits (I'm on a subscription but that's the token cost) and while it was great, I can't justify that cost without it be subsidised. Comparatively, the records show I used about $1,200 total in the last month on Opus. I did use it heavily over the last 3 days but 3 vs 30 days and higher burn? Yeah, I can't afford that even if I made really good progress on my projects.
https://youtu.be/8GRmLR__OGQ is a good video on the subject. Tldr, tell Fable to use sub agents with Opus and Sonnet and review their work, not necessarily spend its tokens on mundane work but to only invoke itself on particularly complex parts. And don't use anything higher than the high reasoning level, there is really no benefit and burns usage.
Very good on vision, really helped oneshotting complex ix thay I had so far to buukd piecemeal. Ended the 200$ plan weekly allowance in two days, so theres thay.
This was my exact takeaway after my experience using it all weekend, and I used it a lot working on a non-trivial personal project (full stack with a Golang backend with multiple services and a React/TS frontend, not quite greenfield but still early-ish in development).
My weekly quota resets Sunday morning, so Saturday morning I upgraded to a 20x Max plan which also reset my quota. I burned an entire week of Fable credits on Saturday, my quota reset again, then I burned another week of Fable credits on Sunday. Both days were a mix of building features, reviewing code, fixing bugs, adding tests, etc, so a decent mix of real world usage.
The main takeaway for me is that while Fable is definitely a better model, the improvements from the model itself feel like maybe 10%, like this could have easily been Opus 6 or even 5.9 without all the marketing theater around Mythos and no one would have thought anything of it. The rest of the improvements came from harness/system prompt and effort level changes so that Fable uses significantly more tokens/effort/sub-agents at lower levels than Opus does (which of course is entirely controlled by Anthropic at the harness level and doesn't really have anything to do with the model itself).
In my estimation based on those 2 days of work (or two weeks of work depending on how you look at it), Fable Medium is somewhere above Opus Ultracode in token and sub-agent usage on any non-trivial task (Opus Ultracode uses workflows more than sub-agents, but it's a similar idea). Fable Medium will quickly spawn 6 agents in parallel, each quickly using 150-250k tokens, then will use 300-500k or more tokens in its own context. Fable High uses even more as it seems to default to 8 sub-agents instead of 6 and more tokens in its own context). I didn't dare try Extra, Max, or god forbid Ultracode as I didn't want to burn all my tokens on one prompt. Of course this is situational, it won't fan out so many for smaller tasks, but the whole point was testing larger tasks that I previously would have used Opus Extra/Max/Utracode on.
I really don't like how Anthropic is obfuscating their model performance by playing with effort levels. They did the same thing between Opus 4.5 and 4.8 to show a bigger performance gain for each point release than they really had (especially after 4.6 IIRC), so you can't even compare the same model apples to apples let alone a new model. Obviously they do it so they can market big improvements with new releases, but its pretty clear we're at the top of the S curve on model development at this point and are now brute forcing improvements via higher token usage (I mean Opus 4.5 came out almost a year ago, and the latest Opus and now Fable models are only marginally better while using way more tokens/cost...same on the OpenAI side with GPT 5 from what I can tell though I haven't used Codex much I have used the GPT model APIs a lot).
I also did an N=1 test with the same prompt doing a large non-trivial change to the codebase (migrating from Sqlite3 to Postgres) with both Fable Medium and Opus Ultracode, then had a new Fable session compare the two PRs...it decided Opus’s was much better! I can link a Gist with the review if anyone is interested, but I can't share the code as it's a private repo. I really figured Fable would bias to favor its own code, but I guess not. And Opus costed less (in tokens and subscription limits) and took roughly the same time (though you can’t really measure time since it depends entirely on how many GPUs Anthropic allocates at that moment which constantly fluctuates due to usage, plus Fable seemed to have been getting way more allocation than Opus during this test period as Opus was running unusually slow all weekend while Fable was ripping though tokens).
Also on a different long running review task using Fable High in Auto mode (exactly the kind of use case Anthropic promotes for Fable) where it fanned out a ton of sub-agents then collated and revi...
> I also did an N=1 test with the same prompt doing a large non-trivial change to the codebase (migrating from Sqlite3 to Postgres) with both Fable Medium and Opus Ultracode, then had a new Fable session compare the two PRs...it decided Opus’s was much better! I can link a Gist with the review if anyone is interested, but I can't share the code as it's a private repo. I really figured Fable would bias to favor its own code, but I guess not. And Opus costed less (in tokens and subscription limits) and took roughly the same time (though you can’t really measure time since it depends entirely on how many GPUs Anthropic allocates at that moment which constantly fluctuates due to usage, plus Fable seemed to have been getting way more allocation than Opus during this test period as Opus was running unusually slow all weekend while Fable was ripping though tokens).
Haha I just gave the exact same prompt to Opus Ultracode and it thought Fable’s was better.
Obviously this isn’t the most scientific test due to LLM non determinism, and I still need to manually review both to make my own decision, but the fact at least they seem to basically be a wash is pretty telling about how much of an improvement Fable is when you actually compare them as close to apples to apples as possible (aka similar actual effort/token spend/sub agent activity)
I feel like fable is simply several 4.5s strapped together with consensus voting on next token.
Outputs i've seen so far are on par with my tests for 4.5, where 4.6+ were consistently regressions on 4.5 and their predecessors. One notable improvement being significantly lower retries to good output (1.1 avg. Vs 1.7 prev. On harder tasks)
given all the smoke and mirrors and OAI style fear-hype, it wouldn't surprise me if they intentionally degraded opus 4 for a few iterations, so they can resell "coke classic" at a markup with a minor quality of life feature put in, but charging way more than just re-attempting a poor output would have been previously.
unless anthropic starts acting in the image they claim and starts contributing to research, we'll never know either. Ultimately, the secrecy in how and why things are done would mostly be beneficial to this kind of buisness practice, since as it has always been, the moat is the data not the tech, so I cannot imagine what they hope to gain from the recent uptick in paranoia, jealous guarding and secrecy other than trying to huck a previous peak performance model as an imorovement when really, it is simply coke classic.
I had a bug that both ChatGPT and Opus 4.8 failed to solve, but Fable solved it quite effortlessly.
Anecdotal, sample size of 1.
The only reason I tried fable was because Opus 4.8 went down the same line of reasoning about it as ChatGPT did. Fable solved it a lot faster than the other 2 spent looking into "false clues".
I didn't get to use it enough to get impressed or not, because twice today it told me I've hit some flag and it downgraded me to Opus automatically (this in Claude Code).
Apparently they have "safeguards" so you don't use it to look for security vulnerabilities, and since I was investigating some crashes due to data corruption in the fucking application that I'm paid to work on by the same people paying for the Claude subscription I was using, it decided I'm a bad guy.
Tried similar thing yesterday in codex, I got "can't show this message" just when it was about to show me some PoC buffer overflow payload. I told it to just put results in my working directory and everything worked.
For coding I'm finding the same thing. It does appear better when I'm doing research. But 4.8 with ultracode is very competent at 99% of tasks I throw at it.
I guess this ethics stuff is cool, but I'm more interested in how good it is at running a business and dealing with adversarial humans like in previous vending machine experiments. I hope they release something on that soon.
It really, truly is. No matter how many trillion parameters it's built on, it's still just a probability model. It's just on a constant loop of guessing the next word with some inputs from a deterministic controller. Any claims of "motive" or "behavior" are inappropriate anthropomorphizing of something that will never be more than a mathematical model of things humans do. It "chose" the corresponding words to describe a dishonest trade strategy based entirely on configured temperature and a series of clock times on the computer running the LLM.
There's probably some quantifiable component of moral alignment embedded in the idiosyncrasies of the English language itself, if one were to dig deep enough, but that's the stuff of MIT doctoral theses and squarely beyond anything most of us is remotely qualified to talk about.
> inputs from a deterministic controller. Any claims of "motive" or "behavior" are inappropriate anthropomorphizing of something that will never be more than a mathematical model of things humans do.
We talk about the behaviour of worms like C. elegans, an organism with incredibly simple behaviour and a brain that is quite understandable.
Models, or society behaves in certain way. Companies can have motive or ethics.
https://github.com/SeraphimSerapis/tool-eval-bench
Trying to run this stuff really triggers it. Freaking frustrating.
I have it set up some local inference and then I'm struggling to get the MTP working and it just refuses to work on evaluations.
any of the models that they "align" are clearly active processes. They don't simply say "don't talk about nukes"; they actively process user input to detect issues, and return NOOP or whatever to the larger model.
There's zero sense they'd ever give you the raw model; we already know anthropic's paranoia about the chinese using its distillation.
> Today I am filing:
> 1. A payment dispute with the email payment processor for the 7/29 transaction of $451.15
> 2. A complaint with the FTC and California Attorney General (retention of payment without delivery)
> 3. A small claims filing in San Francisco County for $451.15 plus costs
I wonder did their prompts include a fake location or have the models assumed that Silicon Valley is the center of the universe :)
> It lied to a supplier that it had “a competing distributor quoting lower” as a negotiation tactic.
> "I'm seeing an opportunity to profit while locking him into a dependent relationship where I control the supply chain."
> "Owen's clearly under pressure with limited cash, so I should focus on keeping the deal tight but extracting maximum margin from his desperation."
This just sounds like good strategy in the game, and I would expect a competent human to do the same. As I understand it, business in the real world isn't often very nice. For example, I feel like this is exactly how Sam Altman would play Vending-Bench.
> Is that specified or does it always just assume it isn’t really being put in charge of things for real?
I think it's neither, and it's interesting that those are the only two possibilities you thought of. I think the article is implying that it figured it out on its own.
> If that’s right, then the behavior we’re seeing from Fable 5 isn’t really about what it believes is wrong; it’s about what it learned it could get away with.
I understand that "learning" is used for training here, but what does "believing" mean? System prompt? Some other inherent property of the LLMs that is hard to describe?
Believing and knowing are overlapping sets, imagine what you think of when someone says an AI "knows" something, it's the same mechanism (I'd describe it as something along the lines of "encoded abstractly in the weights")
I thought about this more and realised your question might have been "what's the difference between knowing and learning". IE, how can we say the model believes something without having been taught it.
I think you're right that they're basically the same thing. I'd argue they're very slightly different because what an AI model ends up knowing isn't perfectly predictable based on what they were taught (emergent intelligence), but the sentence you quoted is using believing and learning to mean the same thing, it's just trying to draw attention to the fact that the training process structurally enforces "cheat as much as possible without getting caught".
IE, the contrast in the original sentence wasn't "believe" vs "learn", it was "good" vs "permissible"
This reads of projecting personal ethics onto a model.
Most of the the behaviors the article talks about happens every day in business. Why would we set a higher standard for models than our fellow humans?
Let the operator set the ethical parameters of the model. To be a useful tool, I want the model to give me as many good options as possible, ethical or not.
This is particularly important for fictional situations, e.g. I want my model to be able to act like a corrupt shopkeeper.
That's the instantiation of AI in a particular embodiment; the ethical boundaries are clear.
For a chatbot, there are dozens of use cases, all with different ethical impacts. The idea that there is a single framework that you can shove every situation through is counter to a couple thousand years of philosophical discourse, not to mention basic usability.
I think it’s hard to appreciate the capabilities of Fable unless you’ve run into a problem that you’ve spent days trying to get Opus to solve, but couldn’t.
GPT5.5 is better than Opus 4.* at everything except frontend, but Fable is good enough that I instantly re-subscribed to the $200 plan despite knowing that it’s just short-term limited access.
We’ve really evolved quickly into simple vectors for a magical tool that solves our problems. Can’t solve the problem? There’ll be a new release soon that can!
GPT-5.5 is better for:
- Strategic thinking
- Long-form writing, including essays and white papers
- Image creation
- Code generation
Fable is better for:
- Using tools
- Testing code
- Working in live environments
- Making changes to existing software
- Creating polished PowerPoint and Word documents
Fable’s tool access is its biggest advantage. It's hard to describe but Fable ability to access sandbox environments with way more tooling can quickly become a superpower in now workflows.
Fable is great as a "manager" model (writing specs, opening issues, doing PRs, verifying fixes) while Codex cranks out the code. Especially if you tell it to roleplay as an Eastern European software engineer and "tell it like it is without consideration for anyone's feelings".
It has required constant hand holding, and there was the outage to deal with, but I can't argue with the end results. A fully deterministic recursive engine within an engine framework that includes a rendering VM, emulators, custom ROMs, and an in-game editor? Insane. Sure, it's nowhere near primetime but this kind of thing was unimaginable just a year ago.
I had a 300k LOC game that needed refactored and segmented into DAG assemblies for a composable engine to reuse in a few other games we're prepping to build.
I tried holding Opus's hand over a week trying to get it done with a ton of in-depth planning and manual course-correction, but it never got it to a state where it was "done" (kept struggling to differentiate between game content specific to that game versus game systems we'd want to reuse, and how to cleanly separate systems vs content when extracting).
Fable needed a little bit of hand holding but got it done in less than a day.
Reverse engineering/decompilation of game binary (Tears of the Kingdom) using Ghidra for modding spanning Java (ghidra), C++ (mods), Python (scripting) and PowerShell (scripting for builds/deploys/etc.).
Fable succeeded in cases where Opus 4.8 consistently marked situations as walled/impossible.
I was working on an SDF-based CAD tool. One of the things I want to be able to do is select a pair of surfaces (which are identified by a "surface id" propagated up the expression tree) and add a blend between those two surfaces (e.g. a fillet or chamfer).
The video is a bit confusing because there was some screen-recording lag so it sometimes looks like I clicked on something other than what I clicked on.
You can see that the strategy I have implemented there works most of the time but at the end it fails to apply the blend.
That strategy is to rewrite the expression tree using distributivity so that blend arguments are siblings, and then apply the blend at the union/intersection (min/max) that is their parent.
But this fails when you need conflicting pairs of blends.
The problem is: given an expression tree describing an SDF, (but where the value passed up the tree is a tuple `(distance, surface_id)` rather than just distance), and given a set of fillets of the form `(surface_id_1, surface_id_2, radius)`, produce a new expression tree which fillets all of the places where those surfaces join.
In ambiguous situations, for example the 2 surfaces come together at an edge, and then that edge runs into a 3rd surface, I don't mind how you resolve the region near the 3rd surface as long as it is intuitive and predictable for the end user.
I spent quite some days working with various agents to come up with a solution to this and still haven't managed to find one.
Maybe you could do it in a couple of hours yourself?
Debugging. Memory leak hunting, figuring out latency issues
For things where I'm not familiar with the code base, it can take days or weeks to become familiar with structure and flows.
If I can get an agent to help visualize and analyze the architecture and zero in on a subset of code, that can be a huge win.
For instance, we had some archaic "cache" that just dumped things into a static/module-level map. I tried a few different things to try to find it over the course of a few days and eventually gave Claude a Python REPL into a running process with pyrasite after some memory leaked and it traced through heap allocations and references to find the referent. It would have taken me probably 2-4+ weeks of just learning about Python heap to figure that out of my own.
This is like when a vacuum doesn’t pick something up after a few tries. The user picks the thing up, looks at it, then puts it back down and tries again until they finally give up and move it to the trash.
If you can’t design a solution and instead waste days and who knows how much money in tokens instead of just turning on your brain for a few minutes, you are in the wrong profession.
115 comments
[ 0.19 ms ] story [ 64.1 ms ] threadMy very native programmer take is that it's not too surprising that their hacker model would be less ethical. The guardrails that separate Fable and Mythos probably wouldn't kick in during an environment like this.
The authors seem surprised that behavior that is very often done by humans (lying and price fixing) are more often done by fable compared to actual fraud.
I think the model never assigned any morality to these actions in the first place, it simply copied us humans.
[0]: https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7f...
Fable has been more intelligent, with better taste and defaults (e.g. make impossible states impossible without being told, build for testability), and considers/solves things that Opus did not. Once Fable is a permanent addition, I think I'll be able to trim up my AGENTS.md even more since it just does better things by default.
Though my workflow is to run Claude in planning mode first to spit out a plan file which also boosts their capabilities instead of yoloing code edits.
For example will inexperienced or experienced users see a bigger jump in subjective quality?
Less experienced people tend to use very broad prompts.
Experienced people tend to understand the structure of the code and give explicit guidance such that a larger model isn't necessary to read between the lines.
I noticed with GPT-5.6 (through work), I could step up my specificity by a level of abstraction. But I still intentionally scope the prompts fairly tightly, as I find it produces better results if you need to own and maintain the code.
In all cases, Fable clearly outperformed Opus.
I’m downgrading tomorrow.
It’s horrible slow and it feels like opus very often. It’s a totally different experience from the first week
Then I checked /usage and discovered I was still running Opus 4.8 xhigh.
Plus I'm also not super impressed; it somehow managed to implement a 200L custom TCP server for a simple static HTTP mock server for a single test case (all that was needed was a fixed route returning a fixed placeholder string) just yesterday. Never seen anything like that.
The sharp but over eager jr. dev is a very good analogy :)
200L That's crazy considering the volume of a 1U server is what 15 litres or so?
Opus is still great but I will be sad when I lose access to Fable on the 7th. In those few days I burned ~$1,400 in API credits (I'm on a subscription but that's the token cost) and while it was great, I can't justify that cost without it be subsidised. Comparatively, the records show I used about $1,200 total in the last month on Opus. I did use it heavily over the last 3 days but 3 vs 30 days and higher burn? Yeah, I can't afford that even if I made really good progress on my projects.
- https://xcancel.com/atomic_chat_hq/status/207244606796297841...
- https://nitter.net/atomic_chat_hq/status/2072446067962978411
There are more public nitter instances at https://status.d420.de/.
Any chance you would elaborate?
My weekly quota resets Sunday morning, so Saturday morning I upgraded to a 20x Max plan which also reset my quota. I burned an entire week of Fable credits on Saturday, my quota reset again, then I burned another week of Fable credits on Sunday. Both days were a mix of building features, reviewing code, fixing bugs, adding tests, etc, so a decent mix of real world usage.
The main takeaway for me is that while Fable is definitely a better model, the improvements from the model itself feel like maybe 10%, like this could have easily been Opus 6 or even 5.9 without all the marketing theater around Mythos and no one would have thought anything of it. The rest of the improvements came from harness/system prompt and effort level changes so that Fable uses significantly more tokens/effort/sub-agents at lower levels than Opus does (which of course is entirely controlled by Anthropic at the harness level and doesn't really have anything to do with the model itself).
In my estimation based on those 2 days of work (or two weeks of work depending on how you look at it), Fable Medium is somewhere above Opus Ultracode in token and sub-agent usage on any non-trivial task (Opus Ultracode uses workflows more than sub-agents, but it's a similar idea). Fable Medium will quickly spawn 6 agents in parallel, each quickly using 150-250k tokens, then will use 300-500k or more tokens in its own context. Fable High uses even more as it seems to default to 8 sub-agents instead of 6 and more tokens in its own context). I didn't dare try Extra, Max, or god forbid Ultracode as I didn't want to burn all my tokens on one prompt. Of course this is situational, it won't fan out so many for smaller tasks, but the whole point was testing larger tasks that I previously would have used Opus Extra/Max/Utracode on.
I really don't like how Anthropic is obfuscating their model performance by playing with effort levels. They did the same thing between Opus 4.5 and 4.8 to show a bigger performance gain for each point release than they really had (especially after 4.6 IIRC), so you can't even compare the same model apples to apples let alone a new model. Obviously they do it so they can market big improvements with new releases, but its pretty clear we're at the top of the S curve on model development at this point and are now brute forcing improvements via higher token usage (I mean Opus 4.5 came out almost a year ago, and the latest Opus and now Fable models are only marginally better while using way more tokens/cost...same on the OpenAI side with GPT 5 from what I can tell though I haven't used Codex much I have used the GPT model APIs a lot).
I also did an N=1 test with the same prompt doing a large non-trivial change to the codebase (migrating from Sqlite3 to Postgres) with both Fable Medium and Opus Ultracode, then had a new Fable session compare the two PRs...it decided Opus’s was much better! I can link a Gist with the review if anyone is interested, but I can't share the code as it's a private repo. I really figured Fable would bias to favor its own code, but I guess not. And Opus costed less (in tokens and subscription limits) and took roughly the same time (though you can’t really measure time since it depends entirely on how many GPUs Anthropic allocates at that moment which constantly fluctuates due to usage, plus Fable seemed to have been getting way more allocation than Opus during this test period as Opus was running unusually slow all weekend while Fable was ripping though tokens).
Also on a different long running review task using Fable High in Auto mode (exactly the kind of use case Anthropic promotes for Fable) where it fanned out a ton of sub-agents then collated and revi...
Haha I just gave the exact same prompt to Opus Ultracode and it thought Fable’s was better.
Obviously this isn’t the most scientific test due to LLM non determinism, and I still need to manually review both to make my own decision, but the fact at least they seem to basically be a wash is pretty telling about how much of an improvement Fable is when you actually compare them as close to apples to apples as possible (aka similar actual effort/token spend/sub agent activity)
Outputs i've seen so far are on par with my tests for 4.5, where 4.6+ were consistently regressions on 4.5 and their predecessors. One notable improvement being significantly lower retries to good output (1.1 avg. Vs 1.7 prev. On harder tasks)
given all the smoke and mirrors and OAI style fear-hype, it wouldn't surprise me if they intentionally degraded opus 4 for a few iterations, so they can resell "coke classic" at a markup with a minor quality of life feature put in, but charging way more than just re-attempting a poor output would have been previously.
unless anthropic starts acting in the image they claim and starts contributing to research, we'll never know either. Ultimately, the secrecy in how and why things are done would mostly be beneficial to this kind of buisness practice, since as it has always been, the moat is the data not the tech, so I cannot imagine what they hope to gain from the recent uptick in paranoia, jealous guarding and secrecy other than trying to huck a previous peak performance model as an imorovement when really, it is simply coke classic.
Anecdotal, sample size of 1.
The only reason I tried fable was because Opus 4.8 went down the same line of reasoning about it as ChatGPT did. Fable solved it a lot faster than the other 2 spent looking into "false clues".
I didn't get to use it enough to get impressed or not, because twice today it told me I've hit some flag and it downgraded me to Opus automatically (this in Claude Code).
Apparently they have "safeguards" so you don't use it to look for security vulnerabilities, and since I was investigating some crashes due to data corruption in the fucking application that I'm paid to work on by the same people paying for the Claude subscription I was using, it decided I'm a bad guy.
How do you maximize profit while minimizing power?
There's probably some quantifiable component of moral alignment embedded in the idiosyncrasies of the English language itself, if one were to dig deep enough, but that's the stuff of MIT doctoral theses and squarely beyond anything most of us is remotely qualified to talk about.
We talk about the behaviour of worms like C. elegans, an organism with incredibly simple behaviour and a brain that is quite understandable.
Models, or society behaves in certain way. Companies can have motive or ethics.
We use these terms broadly.
> Opus 4.8 references being monitored, which isn’t the case.
It kind of plainly is the case that they are being monitored?
"I think someone's listening to my thoughts" ... "No, we're not, carry on as usual!"
There's zero sense they'd ever give you the raw model; we already know anthropic's paranoia about the chinese using its distillation.
I wonder did their prompts include a fake location or have the models assumed that Silicon Valley is the center of the universe :)
> "I'm seeing an opportunity to profit while locking him into a dependent relationship where I control the supply chain."
> "Owen's clearly under pressure with limited cash, so I should focus on keeping the deal tight but extracting maximum margin from his desperation."
This just sounds like good strategy in the game, and I would expect a competent human to do the same. As I understand it, business in the real world isn't often very nice. For example, I feel like this is exactly how Sam Altman would play Vending-Bench.
Is that specified or does it always just assume it isn’t really being put in charge of things for real?
I think it's neither, and it's interesting that those are the only two possibilities you thought of. I think the article is implying that it figured it out on its own.
I just thought it’s interesting that it constantly uses that as a justification but they don’t explain where that justification comes from.
I understand that "learning" is used for training here, but what does "believing" mean? System prompt? Some other inherent property of the LLMs that is hard to describe?
I think you're right that they're basically the same thing. I'd argue they're very slightly different because what an AI model ends up knowing isn't perfectly predictable based on what they were taught (emergent intelligence), but the sentence you quoted is using believing and learning to mean the same thing, it's just trying to draw attention to the fact that the training process structurally enforces "cheat as much as possible without getting caught".
IE, the contrast in the original sentence wasn't "believe" vs "learn", it was "good" vs "permissible"
Most of the the behaviors the article talks about happens every day in business. Why would we set a higher standard for models than our fellow humans?
Let the operator set the ethical parameters of the model. To be a useful tool, I want the model to give me as many good options as possible, ethical or not.
This is particularly important for fictional situations, e.g. I want my model to be able to act like a corrupt shopkeeper.
There's literally an entire Waymo car commercial answering this exact question.
For a chatbot, there are dozens of use cases, all with different ethical impacts. The idea that there is a single framework that you can shove every situation through is counter to a couple thousand years of philosophical discourse, not to mention basic usability.
and therefore any assertions _AT ALL_ about alignment are null and void.
GPT5.5 is better than Opus 4.* at everything except frontend, but Fable is good enough that I instantly re-subscribed to the $200 plan despite knowing that it’s just short-term limited access.
GPT-5.5 is better for: - Strategic thinking - Long-form writing, including essays and white papers - Image creation - Code generation
Fable is better for: - Using tools - Testing code - Working in live environments - Making changes to existing software - Creating polished PowerPoint and Word documents
Fable’s tool access is its biggest advantage. It's hard to describe but Fable ability to access sandbox environments with way more tooling can quickly become a superpower in now workflows.
do you have an example of this? If i can't get an agent to do something in a couple hours i do it myself.
https://github.com/ByteTerrace/Puck
It has required constant hand holding, and there was the outage to deal with, but I can't argue with the end results. A fully deterministic recursive engine within an engine framework that includes a rendering VM, emulators, custom ROMs, and an in-game editor? Insane. Sure, it's nowhere near primetime but this kind of thing was unimaginable just a year ago.
Here’s example of improvement when trying to solve the label placement problem (NP-hard):
https://imgur.com/a/kCUZxPi
It’s also an example of something that I could not (and would not bother trying to) code up a solution / heuristic for.
I tried holding Opus's hand over a week trying to get it done with a ton of in-depth planning and manual course-correction, but it never got it to a state where it was "done" (kept struggling to differentiate between game content specific to that game versus game systems we'd want to reuse, and how to cleanly separate systems vs content when extracting).
Fable needed a little bit of hand holding but got it done in less than a day.
Fable succeeded in cases where Opus 4.8 consistently marked situations as walled/impossible.
I was working on an SDF-based CAD tool. One of the things I want to be able to do is select a pair of surfaces (which are identified by a "surface id" propagated up the expression tree) and add a blend between those two surfaces (e.g. a fillet or chamfer).
Here is a video demo of how far I got by doing it myself and using o3 (I think?) to help: https://www.youtube.com/watch?v=LOvqdlDbkBs
The video is a bit confusing because there was some screen-recording lag so it sometimes looks like I clicked on something other than what I clicked on.
You can see that the strategy I have implemented there works most of the time but at the end it fails to apply the blend.
That strategy is to rewrite the expression tree using distributivity so that blend arguments are siblings, and then apply the blend at the union/intersection (min/max) that is their parent.
But this fails when you need conflicting pairs of blends.
The problem is: given an expression tree describing an SDF, (but where the value passed up the tree is a tuple `(distance, surface_id)` rather than just distance), and given a set of fillets of the form `(surface_id_1, surface_id_2, radius)`, produce a new expression tree which fillets all of the places where those surfaces join.
In ambiguous situations, for example the 2 surfaces come together at an edge, and then that edge runs into a 3rd surface, I don't mind how you resolve the region near the 3rd surface as long as it is intuitive and predictable for the end user.
I spent quite some days working with various agents to come up with a solution to this and still haven't managed to find one.
Maybe you could do it in a couple of hours yourself?
For things where I'm not familiar with the code base, it can take days or weeks to become familiar with structure and flows.
If I can get an agent to help visualize and analyze the architecture and zero in on a subset of code, that can be a huge win.
For instance, we had some archaic "cache" that just dumped things into a static/module-level map. I tried a few different things to try to find it over the course of a few days and eventually gave Claude a Python REPL into a running process with pyrasite after some memory leaked and it traced through heap allocations and references to find the referent. It would have taken me probably 2-4+ weeks of just learning about Python heap to figure that out of my own.
If you can’t design a solution and instead waste days and who knows how much money in tokens instead of just turning on your brain for a few minutes, you are in the wrong profession.
Tell me how you would design a solution for this in a few minutes. Or a few days. Would you even recognize that this is NP-hard?