I don't want to argue here, but once models are trained, using them is quite cheap. Also in the last few months most improvements I've seen are in the smaller sized models, which might make using models even cheaper.
There's this weird tension where small models are getting a lot better, but the gap between the frontier and "what I can run on my gaming GPU" at home isn't necessarily getting smaller. That said, Qwen-3.8-27B-Q3 runs on my 9700 XT and performs well enough for tool calling in agents, which is a critical floor in local model performance imo.
Do you use qwen for any coding? Do you make a frontier model orchestrate it? Are you running with llama.cpp? Sorry for the barrage of questions, but I've been interesting in local AI for a while, yet I've struggled to set up anything usable.
If the models are cheap to run, while good for customers, isn't that a problem for hyperscalers and all the investment poured into the AI data center build out?
If you use opencode for your AI work or becomes affordable. To the point where you would reasonably build applications with intelligence the LLM version. I think in the near future the intelligence will be free and there is no business in free. I know it is a big statement but if you look around there is already a large amount of free models of gradually increasing strength. For most applications you don't need a gpt6
Another thing to pay attention to is the margins of hardware companies like nvidia or micron. Hyperscalers are spending huge amount of money but the money mostly goes to the profits of hardware companies.
11 comments
[ 2.7 ms ] story [ 853 ms ] threaduntil the other guys comes out with a slightly better model, then using yours wont be cheap enough.