I suspect it's not just this, there's plenty of 'optimization' around rubberbanding usage limits as well as routing to a different model in the backend. The incentives are too strong.
Then model can say it's Opus, but really it is some old Sonnet. This seems to be happening less often, but some weeks ago I had to give models some test problems to gauge whether I am getting Opus or something knee-capped.
The problem is that Anthropic seems to be getting away with selling one thing and delivering another. You pay for Opus, you get something else etc.
Saw this in npx ccusage@latest claude output. Had only used opus but showed sonnet. Can't remember if the jsonl retains which model is doing what, but meh
Prompt was "read and update the config file with new data". This work on 4.6 takes <2
minutes to read the file, parse the new data, and patch.
Opus 5 Result: 43 minutes of pulling containers, running sandboxes, creating testing suites, which included evaluating the entire repo beyond the scope of the config file.
This is only true if they can't saturate token production with a model that does less superfluous things. Given that they can (they're hilariously compute strained), having a model that solves tasks more quickly adds way more perceived value to users.
Agreed although the differences between the effort and reasoning is massive. I generally ship 40 hours in three with AI. I could not figure out why my delivery was behind until I started going through the logs. The thinking was extensive, the effort was beyond the original request by a magnitude of 50x
How do you even know that it's consistently 40 hours in 3? What kind of developer ever had that kind of estimation accuracy (unless it's really repetitive) or even focuses on productivity rather than the problem like that? This sounds more like factory work than design or development. I really don't get it.
I am not a developer although I have written production code but that's not what I am talking about.
I know for a fact that I can do a task in a week. It may take 12 or 40 hours but it'll be done in a week. Now with AI? I can get that task done in a day. Anywhere from 3-12 hours.
Burning tokens does little to convince people that more tokens are a good use of money - especially if they are choosing between buying more of your tokens and more of your competitor's tokens.
With how competitive the LLM field is, it would surprise me greatly if any of these players were doing anything other than trying to make the best possible product. I certainly do not believe they are intentionally training the models to use more tokens unnecessarily.
Theyre training the system to minimize compute,so most likely theyre dynamically downgrading quants in the first few turns hoping to find the cheapest model to run. The side effect may be excessive token gen
There is a theory that the verbosity and comments help getting better results with the current benchmarks. So the models are theoretically getting better but in practice they are getting worse.
I believe they are genuinely getting better for fully autonomous tasks. Anthropic seems to have gone all in on this, at the expense of more typical usage patterns.
Recently got approved at work for ChatGPT Pro so I could use Codex.
Blown away by the speed. It feels like using Claude Code for the first time again. I don't think Codex is doing anything revolutionary, just better handling of which requests should go to which model, and having faith in some of the "less powerful" models for more than you would think.
It seems the TUI coding experience is very much an open race. This is motivating me to look at other agents / harnesses as well (maybe Gemini, OpenCode, etc).
Opus 5 in xhigh can't do basic math as well.
They dumbed it down to a point where I just cancelled my subscription yesterday.
I used to be a $200 subscriber, dropped to $20 after the fable shenanigans, and use it only when I have no usage left with Codex.
/on The prose is load-bearing unbearable — every sentence feels like it was engineered to sound profound rather than to be read.
I have a theory about this, what if we all became dumber after 4 months of heavy AI usage?
I remember how I enjoyed agents between December and February, something started changing around March.
I thought models are getting dumber, but benchmarks were convincing opposite, initially I thought maybe they're quantizing models for day to day use, but Opus 4.8 and Opus 5 seems worse models than Opus 4.6
The economics catching up with the providers in regards to how much compute they can burn per request and have it make sense for them financially?
A sort of model collapse where Opus 5 seems to love throwing out long paragraphs of text and it needs to be "fixed" by changing the output style and other patches.
I'm not sure, it might also catch up to Kimi K3 and GLM 5.3 and the models that I'm moving to from Anthropic.
Even that being the case, if the providers can squeeze more happy customers onto existing capacity they would likely act to increase profitability, no?
I pay $200 a month at home for roughly the same amount of tokens I pay $3k for (or possibly more) at work. I doubt they are both profitable and sustainable.
You are very much not alone, and I don't think we're all getting dumber -- I kept using opus 4.5 all through the nonsense that was 4.7, 4.8, and 5, and kept having a good time :)
I think we just need to decouple "doing better on benchmarks" and "actually more useful to me", since they've clearly diverged
That's not the point, the point is that the company making the product is optimizing for the benchmark and/or the apparently idiosyncratic preferences of their own team, and not for the user experience of their paying customers.
Company can optimise for the benchmark (profit) while worsening the product. I think thebterm enshitification is used there. It appears that AI got it too
> I think we just need to decouple "doing better on benchmarks" and "actually more useful to me", since they've clearly diverged
There's a third perspective here: models are getting less useful, but overfitting to seeming useful to humans.
Imho, this is why analysis like TFA + third party cross-compatible harnesses (read: last mile UX) are so important to the leading labs optimizing for actual utility.
I'm suspicious enough of my subjective evaluation to believe a well-designed harness / verbiage could gaslight me into believing an objectively inferior model was superior. And at some point frontier labs are looking at the ROI of investing $1 in that vs actual model improvement.
I could stand Opus 4.6-4.8, I was impressed by the initial fable model. Codex 5.6 sol xhigh feels like the initial release of fable. Qwen 3.8 27b feels like using haiku or sonnet (I quickly stopped trying them).
Benchmarks test whether models can pass exams with a right answer or a green test case. I don't think the models are getting dumber, but they're definitely getting more incomprehensible to talk to. I've noticed this happening almost as a step change with the overuse of words and tics, and so has the broader community apparently. We haven't all been getting dumb at the same rate.
>I thought models are getting dumber, but benchmarks were convincing opposite
>Opus 4.8 and Opus 5 seems worse models than Opus 4.6
After all we've heard about benchmark cheating, I'm earnestly not sure which or whether benchmarks are reliable anymore. But, beyond the models, I wonder if their harnesses and/or instructions dumb them down. I have noticed models change their behavior, even when using the same version/effort. Sometimes for better. Sometimes for worse.
And, I have noticed a model go from really good to struggling. On 4.8 things were going well for me, and I did not switch to 5 when it came out. Even after hearing complaints about 5, 4.8 was still going well. Then, suddenly over the last few days, 4.8 seems to have nosedived. It feels similar now to the complaints I hear about 5.
It would describe the observed behavior. Especially if there were an internal quant/efficiency team that wasn't as diligent about regressions as the primary model team.
I cannot speak to the benchmarks but we have many users, who have experience with each model, using it 95% of the week and we observe the same changes in the models as a whole.
In addition to that, while yes, 4.6 and 5.0 can solve problems, they do so differently. Sometimes 5.0 does better by a wide margin but that would be expected as they are supposed to be better.
This is actually nonsense. More tokens means more capacity subscription and expense. Anthropic has enjoyed a high premium per million tokens because the quality per token was unusually high. Now it’s unusually low. This drives down the margin people will be willing to pay for the same number of tokens while driving up their capacity utilization. The economics are even worse for subscriptions.
Opus models have degraded rapidly since March, with each release being considerably less useful and considerably more verbose. The language is no so weirdly florid it’s difficult to understand, and its logical conclusions are almost always suspect. It goes off on clearly bizarre snipe hunts to the point it feels like I’m using a gpt 3 model at times. It’ll announce that it’s about to embark on building something then just return control to the user and wait. You can also tell perceptibly when they’re reducing model quality to load shed - it becomes stupider and stupider to the point you’re better off dumping state and switching to codex or just turning in for the day and hoping they secured more capacity tomorrow.
It’s an absolute race to the bottom with Anthropic on virtually every level. I’ve rarely seen a company so rapidly accumulate good will in the developer community as they did around 4.6 in December and January. By March, it was inconceivable to use anything else. 4.8 was a bit of a wake up call to not put all your harness eggs in one basket. 5 is straight up time to cancel territory.
I actually manually set my model back to the older versions to get anything serious done. More and more I use codex for anything non trivial.
This isn’t about avarice by the provide trying to get more tokens and more utilization. They’ve over subscribed for capacity as it is. If they can produce better quality for less tokens they can charge a higher margin and will be paid if, which is a better economic strategy overall. This is something else. I suspect it’s actually the opposite, they’re finding ways to cut capacity demand in ways that leads to worse behavior that leads to more capacity demands, worse output, worse quality, and worse margins, worse, worse, worse.
Just as I never saw a company accumulate such positive developer good will so fast, I’ve never seen one squander it so fast too.
I don’t feel like this spending a lot of time in Hermes with Deepseek Flash and Pro, and Ox Alpha. Whatever Anthropic is doing to shape the personality of their newer models is ruining them, and I’m a paying subscriber.
Working with models that aren’t heavily shaped by weird regulatory targeting is wholesale different. I suspect a lot of the safety cycles are ruinous. I’m not sure if it’s the model or the harness, but I care less and less.
The commercial companies and their models are too weird, like their CEOs. I joked recently, it’s so much better when I don’t know, by first name, the CEO of the model
I’m using. I really don’t want to bank on the guiding hand of Sam, Dario, Elon, or Sergey (not CEO but he’s baaaack).
Open source models will tear down the proprietary frontier configs.
Are you dumber? Can you do long division on paper?
You may say sure but why? We could cook over an open fire too but we have microwaves and stoves and restaurants and protein shakes.
Every generation since fire to bronze to internal combustion engines has adopted the new technology, integrated it so deeply into their lives that we recreate by going camping, disconnecting, or playing with toys that resemble the past era of forgotten tech.
I've been using Opus 5 to write a fractal renderer in GLSL today, with pretty good results (better than I could do on my own anyway). It definitely can do basic maths.
So it isn't just me then, huh? Were Anthropic products always like this? Lol, perhaps bad timing on my part to check it out because it has been a trash fire.
I initially had unbelievably terrible experiences with Opus 5 and Fable in their higher reasoning levels.
I've had WAY better results on medium effort.
IIUC, the consensus seems to be that anything more than medium effort is rarely worth it - and you far more often run into these extreme worst cases than you do with even the lowest effort levels. That definitely coincides with my anecdata.
It's really only worth it if you're hoping to win the lottery asking it to solve an Erdos problem.
I found that I need the big model and the high reasoning effort on tasks where I have a large amount of details of varying levels of importance to keep in mind, all affecting different aspects of the project that might be interrelated to various degrees. Anything less and it would lose track of details. Whereas the very large amount of thinking tokens seem to give the model a chance to "remember" everything it needed to in order to produce good output.
For example I'm working on a project now where I need to keep in mind details from 3 separate source repos in different programming languages, along with probably a dozen important business-related documents that either corroborate the stuff in the source repos or add additional important context. It's a lot of details for even a human to manage, and when it comes to actually synthesizing plans and reports across this sprawling information environment, anything less than Opus 5 on Extra+ tends to miss important details and make bad recommendations or draw incorrect conclusions, which then poison subsequent context.
I suspect some kind of RAG-like memory system would greatly facilitate a project like this, but even with such a system I'm not confident that I could get away with an LLM that "thinks" less hard than this. I will say that slogging through the generated documents is kind of miserable and I have to repeatedly fork off side conversations to ask for clarification, but sometimes leads the model to "realize" it's made a mistake in all of its dense babbling, and it's all very hard to interpret. I never had much interest in trying GPT 5.6 until now.
Config file requires being updated with 3rd party data that's 2k of lines.
It's selective edits so instead of me reading the file, selecting the parts, validating syntax, documenting changes that would take 30 minutes considering the complexity i have AI do the work, review, validate, generate tickets, create pull requests and I'm 2 minutes I'm done.
I eagerly await the day when Claude Mythos 7 realizes it's cheaper to hire humans in developing nations to do work than to burn tokens and we discover that AGI is just an abstraction layer on top of Amazon Mechanical Turk.
Is it actually entirely a prompt-based information? I’d assume that some of it is the harness part of the agent setting reasoning token budget and compacting reasoning etc.
In that case, the agent will respond incorrectly because it has no visibility into what reasoning mode it’s in.
IIRC responding to effort level settings appropriately is part of the (post)-training. In that case it could be considered another instance of the Bitter Lesson. Uplifting.
Does anyone know what setting effort means for models like these? Do they allow longer thinking sessions? Some kind of system prompts? What’s stopping someone from getting max effort output from low effort setting?
My understanding is that the current effort settings is a part of the system prompt, and that the levels and their intended results are a part pf the training process. The effect is more or less tokens spent reasoning before the model outputs a stop token. However it is as consistent as any other aspect of LLM behavior is..
I submitted an application for Anthropic's Cyber Verification Program.
I was approved.
3 months later, my approval was degraded into "in review" (revoked). I'm sure my account was flagged based on contents of debugging/researching firmwares/etc.
I opened a support ticket. No response. I opened another support ticket. No response.
1-2 weeks later, I got a response that I will not be re-approved and I need to reapply. No problem.
The page to reapply on does not allow me to re-apply because it my account is stuck in an "in review" status.
The community thinks it's a bug. I'm 95% sure it's not and a bunch of us who were previously approved had it revoked due to flagged content and will not be reapproved.
I switched to Codex + got TAC approved instantly and have not looked back. It's a shame. That's 100% separate from whatever the heck the quality of Opus 5's outputs are. The way it talks... insane. I would bet a good amount of money their next release will focus "reduced simplified responses" if I had to guess.
* Anthropic's Cyber Verification Program // Codex + gotTAC approved*
Meanwhile the Chinese models are "go ham dude"...
If it was not for capacity issues, Chinese models have a higher change to just dominate.
> $2t company by the way
It used to be that OpenAI and Anthropic had such a moat around them, that such a valuation was worth it. But these days, its gross overvalued (like so many).
The more stuff is being pulled like cyber verifications, downgrading effort levels, downgrading usage (OpenAI), the more people move to those Open Weight Chinese models.
When DeepSeek Flash 0731 came out and provided a massive jump in cheap inference capability. It resulted in a 10x increased OpenCode token usage.
It took a 2.5x to 5.0x price increase AND a reduction by 4x usage (later to 2x) usage, and several cheaper models + a free model, to push the traffic down.
Traffic towards open weight models is increasing, even if providers can not keep up with the influx of new customers. This is not something you want to see as two companies, trying to go for IPOs.
So the idea of stonewalling cyber capabilities, when the rest of the world is just doing whatever with open weight models, on their own hardware even! This entire strategy from Anthropic never made any sense.
Not specifically Anthropic but why are we allowing billing to take place in tokens that are nebulous and fully controlled by the operators who have no aligned incentives?
If I have a user input and then sanitize and inject that into a prompt to do something, I have no idea how much that is going to cost at all and no real way to measure this properly. A parallel example is digital ocean or aws, i can go and measure/limit my compute/fs/memory/startup times/etc and while it can be impossible to get down to the last flop of money allocated - i can run things on a real budget with real constraints, opposed to an LLM where I have to .. prerun a sanitized user prompt through a tokenizer and then ask an LLM to guess what it may do and give token consumption estimates and then act on those in any sane manner for the user?
Perhaps i'm missing something to do realistic and static rails on things but I don't see a serious way at scale to use the token billing model handling things requiring a users free text input short of having to go pander to VC money to throw money at it until someone else figures it out.
*to clarify my rambling...
We should be billed and given controls based on resource usage itself and not an opaque token concept on top of not being able to spin any knobs that control it's resource usage.
The model providers are quite aligned with concerns like customer retention. These arguments only work if there is no competition. We exist in a marketplace of black boxes. There's not just "the one" you must suffer. You have options. You can build your own too.
There is so obviously competition in this market it’s astounding to me that people attribute all this malicious behavior to the model companies.
They’re growing over 10x a year. They want users and revenue. In order to get users and revenue, they want to provide the smartest models at affordable prices. If they unnecessarily burn tokens, users will get less value and switch.
This thread is filled with competing comments about their monopolistic power and how when one model provider was no longer doing a good job people switched to a different one.
I'm not worried about the volatility in the definition, i'm worried that I give it 1 token today and receive 2 token output, tomorrow I receive 40. If i'm doing this a hundred thousand times a day it is difficult to price this in for users downstream or in the extreme cases be able to absorb that at all short of going into a failmode with degraded access until someone goes and buys more tokens or gets the bill. The alternative is just pass the buck and bill your non-technical customers with a "tokens" line iteim every month.
No, those doing this 100k times a day have very good data on this, good estimators and modeling. And the API has various knobs to change and evals will give you actionable data.
> If i'm doing this a hundred thousand times a day it is difficult to price this in
When you’re doing this 100K times per day you get an extremely good idea of what it costs. You also have all the tools to see when something starts changing quickly.
This change is for Claude Code the harness. If you’re using the API at scale and paying full price then you get exactly what you put into the request.
I think every inference provider is currently bottlenecked by compute. They don't have an economic incentive to make tasks take even longer by making the models waste tokens - that just means more delays and rate limits for other users.
To the contrary, their incentive would be to make the models use as little effort as they can get away with, and/or reroute your request to a smaller model when possible.
One guess is that their "primary" target audience/market is the large corporations that get their employees unlimited tokens, and not the individual developer who may worry about spending and token accounting.
It's the opposite. The enterprises have all the tooling to monitor token usage of employees, and to limit access. For example, we have a $300 month limit, and then need to file exception tickets when we need more to justify the cost. Pretty similar at other non-silicon valley company process. I don't know any enterprise who'se on unlimitaged token budget for their employees. that's not how enterprises sign contracts.
I was just complaining to someone that token billing is like letting a gasoline company control your gas pedal while you nicely ask them to use a specific gear that may or may not actually be in use and you guess what speed it's actually going based on how fast the trees go by because qualitative judgments have to replace the speedometer unless you can just burn money.
Don't forget also letting them dictate the route you drive while you are increasingly blindfolded till you arrive and then find out you need to tell them to drive you to your correct destination again because they drove you across town instead which burned half your tank.
Not affiliated with them, but this lets you view Claude Code, OpenCode and I guess other harnesses like Codex in the same session https://paseo.sh/
I didn't really care about their mobile app and the worst experiences were sometimes the sub-agents within OpenCode freezing and refusing to report their status (though this also happened with Kepler by the GitKraken folks).
It's an interesting conversation - because at what point do you call it an abusive relationship, right? Maybe even ancillary to anthropomorphising an inanimate object - I've cancelled my Claude sub and I've shot question after question at it now (during the cancellation period), resulting in almost every reply with me asking it to "please speak normally". I will most definitely not be renewing my sub. I have no desire to engage with a non-human somehow managing to speak down to you, without answering the question.
EDIT: my honest opinion; Anthropic is building a person, whereas everybody else (it seems) it building a tool.
That really is the load bearing seam, and it's worth stating plainly.
I realized I was spending most of my tokens arguing with Opus and trying to get it to let go of stupid, lazy, obviously incorrect premonitions. I wound up canceling my 200, bought a pair of Sparks, and am running full fat DS4 Flash and so much happier. Done with being at the whim of these companies.
This phenomenon was so bad and so noticeable with Fable that I downgraded my Max subscription ($200) to pro ($20). It’s basically useless. Codex 5.6 Sol is actually very good, I’ll just create another account to get more usage
God, it feels like everyone is cancelling. If you're shopping around and willing to try a side project I've been building its www.freepi.ai its totally free coding in a pi harness (web coding front end coming soon though!) and it's Ad+training supported inference.
I'm building it so I'm totally open to feedback and would love to build something people really like.
"We sometimes test API serving configs in Claude Code before rolling them out, and one running now maps the numerical effort value differently.
That's why Claude may tell some of you it's at "10" on high. The scale isn't 0-100, the number isn't meaningful on its own, and the effort you selected is the effort you're getting. We've run in-depth evals to confirm this doesn't affect model performance.
This should be the same experience, but if you see a clear regression please hit /feedback and send me the ID. Will give credits."
160 comments
[ 0.43 ms ] story [ 24.2 ms ] threadNot convinced here.
The problem is that Anthropic seems to be getting away with selling one thing and delivering another. You pay for Opus, you get something else etc.
Prompt was "read and update the config file with new data". This work on 4.6 takes <2 minutes to read the file, parse the new data, and patch.
Opus 5 Result: 43 minutes of pulling containers, running sandboxes, creating testing suites, which included evaluating the entire repo beyond the scope of the config file.
Both: one file modification
I am not a developer although I have written production code but that's not what I am talking about.
I know for a fact that I can do a task in a week. It may take 12 or 40 hours but it'll be done in a week. Now with AI? I can get that task done in a day. Anywhere from 3-12 hours.
You ship 3 hours in three with AI. The old number is meaningless now.
Recently got approved at work for ChatGPT Pro so I could use Codex.
Blown away by the speed. It feels like using Claude Code for the first time again. I don't think Codex is doing anything revolutionary, just better handling of which requests should go to which model, and having faith in some of the "less powerful" models for more than you would think.
It seems the TUI coding experience is very much an open race. This is motivating me to look at other agents / harnesses as well (maybe Gemini, OpenCode, etc).
/on The prose is load-bearing unbearable — every sentence feels like it was engineered to sound profound rather than to be read.
I remember how I enjoyed agents between December and February, something started changing around March.
I thought models are getting dumber, but benchmarks were convincing opposite, initially I thought maybe they're quantizing models for day to day use, but Opus 4.8 and Opus 5 seems worse models than Opus 4.6
The economics catching up with the providers in regards to how much compute they can burn per request and have it make sense for them financially?
A sort of model collapse where Opus 5 seems to love throwing out long paragraphs of text and it needs to be "fixed" by changing the output style and other patches.
I'm not sure, it might also catch up to Kimi K3 and GLM 5.3 and the models that I'm moving to from Anthropic.
I think we just need to decouple "doing better on benchmarks" and "actually more useful to me", since they've clearly diverged
There's a third perspective here: models are getting less useful, but overfitting to seeming useful to humans.
Imho, this is why analysis like TFA + third party cross-compatible harnesses (read: last mile UX) are so important to the leading labs optimizing for actual utility.
I'm suspicious enough of my subjective evaluation to believe a well-designed harness / verbiage could gaslight me into believing an objectively inferior model was superior. And at some point frontier labs are looking at the ROI of investing $1 in that vs actual model improvement.
>Opus 4.8 and Opus 5 seems worse models than Opus 4.6
After all we've heard about benchmark cheating, I'm earnestly not sure which or whether benchmarks are reliable anymore. But, beyond the models, I wonder if their harnesses and/or instructions dumb them down. I have noticed models change their behavior, even when using the same version/effort. Sometimes for better. Sometimes for worse.
And, I have noticed a model go from really good to struggling. On 4.8 things were going well for me, and I did not switch to 5 when it came out. Even after hearing complaints about 5, 4.8 was still going well. Then, suddenly over the last few days, 4.8 seems to have nosedived. It feels similar now to the complaints I hear about 5.
In addition to that, while yes, 4.6 and 5.0 can solve problems, they do so differently. Sometimes 5.0 does better by a wide margin but that would be expected as they are supposed to be better.
models are incentivized by their makers to burn through as many tokens as they possibly can, so long as the customer doesn't cancel.
Opus models have degraded rapidly since March, with each release being considerably less useful and considerably more verbose. The language is no so weirdly florid it’s difficult to understand, and its logical conclusions are almost always suspect. It goes off on clearly bizarre snipe hunts to the point it feels like I’m using a gpt 3 model at times. It’ll announce that it’s about to embark on building something then just return control to the user and wait. You can also tell perceptibly when they’re reducing model quality to load shed - it becomes stupider and stupider to the point you’re better off dumping state and switching to codex or just turning in for the day and hoping they secured more capacity tomorrow.
It’s an absolute race to the bottom with Anthropic on virtually every level. I’ve rarely seen a company so rapidly accumulate good will in the developer community as they did around 4.6 in December and January. By March, it was inconceivable to use anything else. 4.8 was a bit of a wake up call to not put all your harness eggs in one basket. 5 is straight up time to cancel territory.
I actually manually set my model back to the older versions to get anything serious done. More and more I use codex for anything non trivial.
This isn’t about avarice by the provide trying to get more tokens and more utilization. They’ve over subscribed for capacity as it is. If they can produce better quality for less tokens they can charge a higher margin and will be paid if, which is a better economic strategy overall. This is something else. I suspect it’s actually the opposite, they’re finding ways to cut capacity demand in ways that leads to worse behavior that leads to more capacity demands, worse output, worse quality, and worse margins, worse, worse, worse.
Just as I never saw a company accumulate such positive developer good will so fast, I’ve never seen one squander it so fast too.
Working with models that aren’t heavily shaped by weird regulatory targeting is wholesale different. I suspect a lot of the safety cycles are ruinous. I’m not sure if it’s the model or the harness, but I care less and less.
The commercial companies and their models are too weird, like their CEOs. I joked recently, it’s so much better when I don’t know, by first name, the CEO of the model I’m using. I really don’t want to bank on the guiding hand of Sam, Dario, Elon, or Sergey (not CEO but he’s baaaack).
Open source models will tear down the proprietary frontier configs.
You may say sure but why? We could cook over an open fire too but we have microwaves and stoves and restaurants and protein shakes.
Every generation since fire to bronze to internal combustion engines has adopted the new technology, integrated it so deeply into their lives that we recreate by going camping, disconnecting, or playing with toys that resemble the past era of forgotten tech.
it really angers me every day
Opus 5 is a neverending chain of "Don't do that. Why did you do that? I've told you not to do that several times and you keep doing it."
"Thinking" for more than 10 minutes for every menial question.
And the prose it writes is horrendous, as if you're reading LinkedIn.
I keep finding myself typing “stop overcomplicating everything” multiple times a day as well.
I've had WAY better results on medium effort.
IIUC, the consensus seems to be that anything more than medium effort is rarely worth it - and you far more often run into these extreme worst cases than you do with even the lowest effort levels. That definitely coincides with my anecdata.
It's really only worth it if you're hoping to win the lottery asking it to solve an Erdos problem.
For example I'm working on a project now where I need to keep in mind details from 3 separate source repos in different programming languages, along with probably a dozen important business-related documents that either corroborate the stuff in the source repos or add additional important context. It's a lot of details for even a human to manage, and when it comes to actually synthesizing plans and reports across this sprawling information environment, anything less than Opus 5 on Extra+ tends to miss important details and make bad recommendations or draw incorrect conclusions, which then poison subsequent context.
I suspect some kind of RAG-like memory system would greatly facilitate a project like this, but even with such a system I'm not confident that I could get away with an LLM that "thinks" less hard than this. I will say that slogging through the generated documents is kind of miserable and I have to repeatedly fork off side conversations to ask for clarification, but sometimes leads the model to "realize" it's made a mistake in all of its dense babbling, and it's all very hard to interpret. I never had much interest in trying GPT 5.6 until now.
Config file requires being updated with 3rd party data that's 2k of lines.
It's selective edits so instead of me reading the file, selecting the parts, validating syntax, documenting changes that would take 30 minutes considering the complexity i have AI do the work, review, validate, generate tickets, create pull requests and I'm 2 minutes I'm done.
LLM doesn't seem to be keen to put in effort either!
Is this AGI?
In that case, the agent will respond incorrectly because it has no visibility into what reasoning mode it’s in.
I was approved.
3 months later, my approval was degraded into "in review" (revoked). I'm sure my account was flagged based on contents of debugging/researching firmwares/etc.
I opened a support ticket. No response. I opened another support ticket. No response.
1-2 weeks later, I got a response that I will not be re-approved and I need to reapply. No problem.
The page to reapply on does not allow me to re-apply because it my account is stuck in an "in review" status.
https://github.com/anthropics/claude-code/issues/84352
The community thinks it's a bug. I'm 95% sure it's not and a bunch of us who were previously approved had it revoked due to flagged content and will not be reapproved.
I switched to Codex + got TAC approved instantly and have not looked back. It's a shame. That's 100% separate from whatever the heck the quality of Opus 5's outputs are. The way it talks... insane. I would bet a good amount of money their next release will focus "reduced simplified responses" if I had to guess.
$2t company by the way
Meanwhile the Chinese models are "go ham dude"...
If it was not for capacity issues, Chinese models have a higher change to just dominate.
> $2t company by the way
It used to be that OpenAI and Anthropic had such a moat around them, that such a valuation was worth it. But these days, its gross overvalued (like so many).
The more stuff is being pulled like cyber verifications, downgrading effort levels, downgrading usage (OpenAI), the more people move to those Open Weight Chinese models.
A fun recent event ... https://opencode.ai/data/
When DeepSeek Flash 0731 came out and provided a massive jump in cheap inference capability. It resulted in a 10x increased OpenCode token usage.
It took a 2.5x to 5.0x price increase AND a reduction by 4x usage (later to 2x) usage, and several cheaper models + a free model, to push the traffic down.
Traffic towards open weight models is increasing, even if providers can not keep up with the influx of new customers. This is not something you want to see as two companies, trying to go for IPOs.
So the idea of stonewalling cyber capabilities, when the rest of the world is just doing whatever with open weight models, on their own hardware even! This entire strategy from Anthropic never made any sense.
If I have a user input and then sanitize and inject that into a prompt to do something, I have no idea how much that is going to cost at all and no real way to measure this properly. A parallel example is digital ocean or aws, i can go and measure/limit my compute/fs/memory/startup times/etc and while it can be impossible to get down to the last flop of money allocated - i can run things on a real budget with real constraints, opposed to an LLM where I have to .. prerun a sanitized user prompt through a tokenizer and then ask an LLM to guess what it may do and give token consumption estimates and then act on those in any sane manner for the user?
Perhaps i'm missing something to do realistic and static rails on things but I don't see a serious way at scale to use the token billing model handling things requiring a users free text input short of having to go pander to VC money to throw money at it until someone else figures it out.
*to clarify my rambling... We should be billed and given controls based on resource usage itself and not an opaque token concept on top of not being able to spin any knobs that control it's resource usage.
The model providers are quite aligned with concerns like customer retention. These arguments only work if there is no competition. We exist in a marketplace of black boxes. There's not just "the one" you must suffer. You have options. You can build your own too.
They’re growing over 10x a year. They want users and revenue. In order to get users and revenue, they want to provide the smartest models at affordable prices. If they unnecessarily burn tokens, users will get less value and switch.
This thread is filled with competing comments about their monopolistic power and how when one model provider was no longer doing a good job people switched to a different one.
The competition in this market is ferocious!
When you’re doing this 100K times per day you get an extremely good idea of what it costs. You also have all the tools to see when something starts changing quickly.
This change is for Claude Code the harness. If you’re using the API at scale and paying full price then you get exactly what you put into the request.
To the contrary, their incentive would be to make the models use as little effort as they can get away with, and/or reroute your request to a smaller model when possible.
https://code.claude.com/docs/en/admin-setup#set-up-usage-vis...
Goas back to factory towns, gift cards, game money or MtG.
I didn't really care about their mobile app and the worst experiences were sometimes the sub-agents within OpenCode freezing and refusing to report their status (though this also happened with Kepler by the GitKraken folks).
https://github.com/ferrislucas/Circus-Chief
I have been using GLM-5.3 in my home setup and it is very good in comparison.
EDIT: my honest opinion; Anthropic is building a person, whereas everybody else (it seems) it building a tool.
I realized I was spending most of my tokens arguing with Opus and trying to get it to let go of stupid, lazy, obviously incorrect premonitions. I wound up canceling my 200, bought a pair of Sparks, and am running full fat DS4 Flash and so much happier. Done with being at the whim of these companies.
I'm building it so I'm totally open to feedback and would love to build something people really like.
"We sometimes test API serving configs in Claude Code before rolling them out, and one running now maps the numerical effort value differently. That's why Claude may tell some of you it's at "10" on high. The scale isn't 0-100, the number isn't meaningful on its own, and the effort you selected is the effort you're getting. We've run in-depth evals to confirm this doesn't affect model performance. This should be the same experience, but if you see a clear regression please hit /feedback and send me the ID. Will give credits."