Yeah. And weird pricing seems like it's winding down.
It's interesting to compare it to electricity. Basically Anthropic was selling a flat fee electricity subscription, and when someone started connecting expensive washing machines (OpenClaw) to their subscriptions, instead of changing the pricing model, they banned washing machines...
I wonder if we will get to "electricity" style pricing for AI. What makes electricity predictable is relatively constant average usage over time + price is manageable. I'm just not buying electrical house heating and manage my electricity spending within some bounds.
With AI the problem is that we are only now getting to useful AI, and for now it's still too expensive to be useful, so they subsidize until they can stabilize at "cheap enough and smart enough" level. But it feels like that's still 2 years away while they are stopping to subsidize now. Will be interesting.
meh - by this logic, every new tech and startup ever is a "scam"
The truth is that the AI companies are gambling that inference cost will continue following a hyper version of Moore's Law, e.g. Google TurboQuant.
The countervailing thesis is that frontier models are consuming more and more compute.
The deepest truth: you often don't need a frontier model to get commercially acceptable results from AI. Thus, bring on the true pricing! and I'll just switch models to something financially sustainable.
The general problem the average user has with a metered instead of provisioned billing model for computer services is you can’t easily control for cost overruns. From the old days customers getting stung for hosting costs when slashdotted or DOSed, to last decades microservice shock horror of the CI retry loop that burns money overnight to today’s AI that you basically have no idea how efficient the AI will be while it ponders your question, you are just setting yourself up for disappointment and cost overruns and a feeling that you’re not getting the value for money you got last week etc.
>At some point, the incredible, toxic burn-rate of generative AI is going to catch up with them, which in turn will lead to price increases, or companies releasing new products and features with wildly onerous rates (..) that will make even stalwart enterprise customers with budget to burn unable to justify the expense.
I pray this happens soon, but I feel I've been hearing some version of it for a while.
There's a few major problems with the article. The most obvious is that frontier labs are not charging remotely close to the cost of tokens; afaik most estimate north of 80% profit margins. As a reference, providers are profitably providing Kimi K2.6 for $4/1Mtok out. Is that as good as Opus? No, but it's probably at least Sonnet level, so that's ~4x cheaper than Sonnet while still being profitable to serve on the margin. So you aren't plausibly getting into actual subsidization territory until you're over 5:1 sub to nameplate token costs.
How many tokens can you realistically burn through in one chat session? Opus and many other frontier models do maybe 60tok/s, less 250k/hr out. In you can use more, but in most cases cache is 5-10:1 cheaper than new input. Say you average 500ktok in, 90% cache, per request. That amounts to 100-150ktok in new input-equivalent costs, which in most cases is ~20-30ktok in output-equivalent costs. Do a request every minute, that's a total of about 1.5-2Mtok/hr. At API prices that's $50/hr for Opus, but really it probably only costs Anthropic $10/hr to serve that.
That said, even if a developer is burning $50/hr, many, many employees at large companies cost more than $100k/yr to employ all costs considered, so making them say 20-30% more productive can easily make that worth it for most. If the labs shave their margins ultimately to more like 20-30%, you'd have ~$15/hr in costs to use the services, and nearly every white collar job is way over 30k/yr to employ. If your salary is 80k, you probably cost the company 200k all in, so making you 15% more productive offsets the $15/hr cost.
So first party providers are not in a horrifying position or anything from a subsidization standpoint. The people in bad shape are Cursor and Perplexity, who don't have frontier models and are dependent on the open source community, which is typicly 6-12 months behind the frontier. They have to pay full freight API costs at 80% margin for the big boys to serve their harnesses, which is indeed untenable, and they'll have to either force users to use open source models and/or in house models they can serve at-cost or they will have to charge vastly more.
Gemini, Claude, and ChatGPT first-party services like Antigravity, Codex, and Claude Code are not in serious trouble though.
Problem with this math is it always assumes some ridiculous baseline compensation (or costs, in this case) as a matter of fact. There's an entire world of developers not costing 200k to their employers.
Truth of the matter in most companies large enough is if you make your devs 30% more productive, then that'd mean 30% more code going through "change management" hell for months. You're not even paying to stand still, you're just pushing even more down a bottleneck. The price most people are willing to pay to make things worse is close to zero.
> providers are profitably providing Kimi K2.6 for $4/1Mtok out.
Do you perchance have a source for this? Is the profitability assessment comprehensive, including hardware amortization? I've found it hard to track down actual hard numbers for the cost of inference.
It make sense if you account for cost of intelligence getting cheaper every year. Most of the models per unit of intelligence is getting far cheaper. We get better hardware, architecture, training techniques, inference optimizations and caching. All those improvements add up. In in early 2022 you were getting 10x cheaper annually now is closer to 2x - 5x cheaper annually. The cost is still dropping where as Uber can only get the cost down by so much.
The entire basis of this article is that generating tokens is a variable cost and that that cost will not decrease over time.
> On an economic basis, a monthly subscription only makes sense with relatively static costs.
Running a data center is a fixed expense. Whether or not people use that data center to it's capacity doesn't change how much the operator pays (electricity use factors into this, since a GPU running at 100% will use more watts than an idle one, but it doesn't move the needle much on other fixed and variable costs of a data center).
> They also assumed, I imagine, that the cost of tokens would come down over time, versus what actually happened — while prices for some models might have come down, newer “reasoning” models burn way more tokens, which means the cost of inference has, somehow, gotten higher over time.
This is backwards. When the cost of something goes down, people use it more. This is basic supply and demand. Inference has gotten cheaper already, and will continue to do so.
Companies subsidizing costs for growth happens all the time. Yes, switching to usage-based pricing instead of subscriptions sucks for customers, but enterprises will continue to pay.
I am a paying subscriber to Ed Zitron and I enjoy his writing a lot. He should at some point admit that not everything is bullshit and there is definitely a business model to it. It is fun to read, though
The moves from “the subscription model for AI isn’t working given these parameters” to “a subscription model for AI can never work” to “the model was deliberately deceptive” to “it’s a fucking ripoff” is not logical. AI companies are feeling the need to get hold of spiraling costs by increasing prices and limitations. Inference hasn’t gotten cheap enough fast enough, and for some reason they feel they can’t wait longer. That doesn’t mean a subscription service can’t work: only that it will be expensive, maybe vastly so, and will need tiers based on usage with some fluidity for users to move between tiers in a given month. The model is something like HP’s “instant ink” service. Sure, there’s a question whether the moves companies are making now are worth the cost in the eyes of customers. But that’s a question of economics and timing, not a fundamental blow to monthly subscriptions as a model. The article doesn’t deal with these considerations fairly. It’s too much in the direction of a rant, with conspiracy theories thrown in.
I'm just flabbergasted at the massive inefficient usage of tokens. What are people doing to spend 500 usd/day in tokens. I just don't understand what you could possibly be doing that would be not complete spagetti at the end if you run something in an autoloop.
Reading this piece, I'm reminded of a podcast I heard some years ago where they were interviewing an early google marketing employee who was talking about the economics of google search. They said they'd done some surveys and concluded that they determined that the average user would get something like $20/year of value, and so that was the most they could realistically charge for search. Meanwhile, they could make something like $500/user in Q4 alone for advertising. So, of course, advertising.
I just don't think that LLM business models can survive the allure of advertising dollars, any more than Search could, or TV, or Radio, or Movies. Ignoring the talk of copilot putting ads into pull requests, there is just no way that publicly hosted LLMs will not end up inserting ads into the output.
And why would they? Google has Gemini even if nothing else happens it is patently obvious that the best current LLM will capture a multi trillion dollar advertising market zero sum. That right there is more than enough to justify Google continuing investment for the next decade since they simply cannot afford to lose that market. I wish we lived in a sane country were you can't invest in your competition but whatever.
Zitron misunderstands the economics of models. Inference costs have dropped 99% in less than 2 years. Models are being commoditized faster than any technology in history.
A $20 subscription 2 years ago is not providing the same level of intelligence you're getting today.
Every major lab knows open source models are 6 months behind (See Google's "We have no moat") and none of them plan to make money on inference. Companies are subsidizing users to create moats that persist when models are essentially free for most everyday use.
Do we know the breakdown of revenue from API vs subscriptions for OAI/Anthropic? That seems very relevant, since this entire article seems to be on the premise that users are only willing to pay for a subsidized subscription and would never pay the 'true' token cost.
The internet seems to be saying that 70%+ of Anthropic revenue is per-token metered API, which would largely invalidate the article, but I can't find a solid source.
Customer: “I don’t want to pay more than $100/mo for my website”
Developer: “What are your goals?”
Customer: “1M daily visits, 1,000 monthly signups.”
And we've spent the past 25 years offering serverless compute, auto-scaling, pay-as-you-go for AWS and Internet infrastructure. And the economics are still a hard sell.
I thought this burning of cash was all an excuse for the exponential growth we saw in the last 6 years.
They went from GPT 2 a text only, goldfish-esque memory at a 8th grade reading level to what we have today, GPT 5, multimodality + a token window encompassing a enclyopedia and a Doctorate/Masters level of mastery in major subjects.
The economics are probably betting on this exponential growth to continue, which if it fails, the cash would burn.
I think there's another route this goes. At $7k a year or more per eng in token use, I think it's very reasonable to buy engineers machines with obscene GPUs and RAM and run models locally. And if it doesn't make sense now, someone will figure it out and save companies $10k+/eng over 3 years.
If you only want/need the kind of model output that can be served on a machine costing single digit thousands, aren’t cheaper cloud-served models available? (And as the sister comment points out, sharing hardware allows greater utilization and lower costs per user.)
I imagine there are companies forming now with their entire business model being building "prosumer" inference machines and farms running everything from Qwen 3.6 27b up to GLM 5.1 and everything in between, packaged perfectly for companies to make one-time investments in with the assumption that open models will be getting both more efficient and better over time.
I've sort of lost some respect for ed that I had early on in the hype cycle - he's still right about some things, but I can see him slowly and subtly retreating from his strong position, held even a few months ago, that these things will never ever be useful for anything and it's all a scam because they don't actually do anything at all except burn money. He would say it like 8 times a monologue. I remember one podcast maybe ~6 months ago he brought a developer skeptic on, and was trying to get him to say it wasn't actually useful for coding, and the dev was like "maybe not as advertised, but I definitely use it and it is useful to me" and he pivoted off the topic very quickly.
It seems he realizes he was wrong about that and has pivoted slowly to, "well, maybe they work sometimes, but the cost isn't justified." Which is a reasonable question! I just find his style of never admitting when he is wrong off putting and the way he presents things as absolute fact, when he's guessing like the rest of us. He was right about a lot, wrong about a lot, it's okay to admit that, I don't think his fan base would care.
Ed's writing style is often off-putting, repetitive and sometimes gives almost "desperate" vibes. But he does raises questions no one in the industry is seriously entertaining and exploring. What if those monsters are indeed unprofitable, now what? So while I stopped reading him regularly, I visit once a quarter just to read something not about our inevitable benevolent apocalyptic LLM gods and their Prophet St. Sam, prophesying a complete job loss and despair.
This reminds me of a Bitfinexed blog situation. That guy researched and proved Tether token scam for years and he was right. But he didn't account for a tiny nuance - Tethers are useful for financial crime and are propped by that public regardless of the financial viability or rejection by every decent financial institution. Turns out you can have a hundred billion of unbacked tokens, if they are "alternatively backed" instead. I suspect LLM monsters may turn out the same way (or not).
Serious question - are there any LLM bubble critics with more sane and to the point style of writing and not just posting unsubstantiated hype for views like most on YT?
I am sympathetic to his view because I also considered the whole AI hype train a complete scam until pretty recently. When I saw enough people validating that coding agents were actually legitimately ok and sometimes good at things, I decided to spend $50 on one to test it out.
I have been pleasantly surprised at its utility knocking out grunt work. It's not super smart, but it's great at things like writing a python script to edit characteristics of a jsonl file or sorting structured data. I didn't ever expect it to be useful beyond extremely limited output and it's actually kinda good when you know how to narrowly target the tasks. The constraints of code make it a more suitable category than all the other stuff.
It's still a bs hype machine with Elon saying it might save all of humanity in court today. That's pretty unlikely.
All subscription models are subsidized by users who don't use much. The fact that somebody on a $20 sub might get $50 in value isn't crazy if there are 3 people who only get $10 in value. This isn't some sign that the model is broken, it's the intended outcome.
Also, I didn't read this whole thing, but I have yet to see Zitron respond to the strongest AI financials claim, which is that the models themselves are profitable on a life-cycle basis, even if the companies are not profitable on an annual basis due to capital expenditure. Dario made this claim exactly, and it more or less blows all of Zitron's financials arguments up.
I subscribed to Claude for a month. I sat down with it for a few sessions, but in each case I ran into a limit before I achieved anything worthwhile. And that was with me babysitting it the whole time to try to get the most out of it. I'm not sure it's possible to use it less (so that others can use it more) and get anything meaningful done.
48 comments
[ 0.19 ms ] story [ 65.9 ms ] threadIt's interesting to compare it to electricity. Basically Anthropic was selling a flat fee electricity subscription, and when someone started connecting expensive washing machines (OpenClaw) to their subscriptions, instead of changing the pricing model, they banned washing machines...
I wonder if we will get to "electricity" style pricing for AI. What makes electricity predictable is relatively constant average usage over time + price is manageable. I'm just not buying electrical house heating and manage my electricity spending within some bounds.
With AI the problem is that we are only now getting to useful AI, and for now it's still too expensive to be useful, so they subsidize until they can stabilize at "cheap enough and smart enough" level. But it feels like that's still 2 years away while they are stopping to subsidize now. Will be interesting.
The truth is that the AI companies are gambling that inference cost will continue following a hyper version of Moore's Law, e.g. Google TurboQuant.
The countervailing thesis is that frontier models are consuming more and more compute.
The deepest truth: you often don't need a frontier model to get commercially acceptable results from AI. Thus, bring on the true pricing! and I'll just switch models to something financially sustainable.
I pray this happens soon, but I feel I've been hearing some version of it for a while.
How many tokens can you realistically burn through in one chat session? Opus and many other frontier models do maybe 60tok/s, less 250k/hr out. In you can use more, but in most cases cache is 5-10:1 cheaper than new input. Say you average 500ktok in, 90% cache, per request. That amounts to 100-150ktok in new input-equivalent costs, which in most cases is ~20-30ktok in output-equivalent costs. Do a request every minute, that's a total of about 1.5-2Mtok/hr. At API prices that's $50/hr for Opus, but really it probably only costs Anthropic $10/hr to serve that.
That said, even if a developer is burning $50/hr, many, many employees at large companies cost more than $100k/yr to employ all costs considered, so making them say 20-30% more productive can easily make that worth it for most. If the labs shave their margins ultimately to more like 20-30%, you'd have ~$15/hr in costs to use the services, and nearly every white collar job is way over 30k/yr to employ. If your salary is 80k, you probably cost the company 200k all in, so making you 15% more productive offsets the $15/hr cost.
So first party providers are not in a horrifying position or anything from a subsidization standpoint. The people in bad shape are Cursor and Perplexity, who don't have frontier models and are dependent on the open source community, which is typicly 6-12 months behind the frontier. They have to pay full freight API costs at 80% margin for the big boys to serve their harnesses, which is indeed untenable, and they'll have to either force users to use open source models and/or in house models they can serve at-cost or they will have to charge vastly more.
Gemini, Claude, and ChatGPT first-party services like Antigravity, Codex, and Claude Code are not in serious trouble though.
Truth of the matter in most companies large enough is if you make your devs 30% more productive, then that'd mean 30% more code going through "change management" hell for months. You're not even paying to stand still, you're just pushing even more down a bottleneck. The price most people are willing to pay to make things worse is close to zero.
Do you perchance have a source for this? Is the profitability assessment comprehensive, including hardware amortization? I've found it hard to track down actual hard numbers for the cost of inference.
> On an economic basis, a monthly subscription only makes sense with relatively static costs.
Running a data center is a fixed expense. Whether or not people use that data center to it's capacity doesn't change how much the operator pays (electricity use factors into this, since a GPU running at 100% will use more watts than an idle one, but it doesn't move the needle much on other fixed and variable costs of a data center).
> They also assumed, I imagine, that the cost of tokens would come down over time, versus what actually happened — while prices for some models might have come down, newer “reasoning” models burn way more tokens, which means the cost of inference has, somehow, gotten higher over time.
This is backwards. When the cost of something goes down, people use it more. This is basic supply and demand. Inference has gotten cheaper already, and will continue to do so.
Companies subsidizing costs for growth happens all the time. Yes, switching to usage-based pricing instead of subscriptions sucks for customers, but enterprises will continue to pay.
I just don't think that LLM business models can survive the allure of advertising dollars, any more than Search could, or TV, or Radio, or Movies. Ignoring the talk of copilot putting ads into pull requests, there is just no way that publicly hosted LLMs will not end up inserting ads into the output.
This looks like what I remember. https://freakonomics.com/podcast/is-google-getting-worse/
A $20 subscription 2 years ago is not providing the same level of intelligence you're getting today.
Every major lab knows open source models are 6 months behind (See Google's "We have no moat") and none of them plan to make money on inference. Companies are subsidizing users to create moats that persist when models are essentially free for most everyday use.
The internet seems to be saying that 70%+ of Anthropic revenue is per-token metered API, which would largely invalidate the article, but I can't find a solid source.
[0]: https://www.wheresyoured.at/why-are-we-still-doing-this/
Customer: “I don’t want to pay more than $100/mo for my website” Developer: “What are your goals?” Customer: “1M daily visits, 1,000 monthly signups.”
And we've spent the past 25 years offering serverless compute, auto-scaling, pay-as-you-go for AWS and Internet infrastructure. And the economics are still a hard sell.
They went from GPT 2 a text only, goldfish-esque memory at a 8th grade reading level to what we have today, GPT 5, multimodality + a token window encompassing a enclyopedia and a Doctorate/Masters level of mastery in major subjects.
The economics are probably betting on this exponential growth to continue, which if it fails, the cash would burn.
It seems he realizes he was wrong about that and has pivoted slowly to, "well, maybe they work sometimes, but the cost isn't justified." Which is a reasonable question! I just find his style of never admitting when he is wrong off putting and the way he presents things as absolute fact, when he's guessing like the rest of us. He was right about a lot, wrong about a lot, it's okay to admit that, I don't think his fan base would care.
This reminds me of a Bitfinexed blog situation. That guy researched and proved Tether token scam for years and he was right. But he didn't account for a tiny nuance - Tethers are useful for financial crime and are propped by that public regardless of the financial viability or rejection by every decent financial institution. Turns out you can have a hundred billion of unbacked tokens, if they are "alternatively backed" instead. I suspect LLM monsters may turn out the same way (or not).
Serious question - are there any LLM bubble critics with more sane and to the point style of writing and not just posting unsubstantiated hype for views like most on YT?
I have been pleasantly surprised at its utility knocking out grunt work. It's not super smart, but it's great at things like writing a python script to edit characteristics of a jsonl file or sorting structured data. I didn't ever expect it to be useful beyond extremely limited output and it's actually kinda good when you know how to narrowly target the tasks. The constraints of code make it a more suitable category than all the other stuff.
It's still a bs hype machine with Elon saying it might save all of humanity in court today. That's pretty unlikely.
Because, comparing vs GPUs
~16k–17k tokens/second per user
<1ms latency
10x power efficiency
20x cheaper production
Model to Si ~ 60 to 90 days
We have every reason to believe SW_to_Si will facilitate improving economics
Also, I didn't read this whole thing, but I have yet to see Zitron respond to the strongest AI financials claim, which is that the models themselves are profitable on a life-cycle basis, even if the companies are not profitable on an annual basis due to capital expenditure. Dario made this claim exactly, and it more or less blows all of Zitron's financials arguments up.