Very impressive score for the size, though token use is higher than k3 and far higher than proprietary models, and its price to performance isn't all that far ahead of k3 as a result
Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.
yes, but the GLM subscription gives you barely more usage than you'd get just paying API rates. it's not subsidized like the claude or chatgpt subscriptions are.
I use the $200 plan w/ Anthropic and run out of tokens half way through the week and supposedly they are progressively reducing the limits on all their subs even further.
At some point I will switch, $200 buys a lot of tokens on OpenRouter.
Is the conventional wisdom that the subscription price/token is better than the API price/token not valid any more? Or is access to model diversity worth the increased per token costs?
If anything, it's going to be more expensive. Price/performance ratio isn't there yet for frontier open weight models.
But regardless, you definitely should use a harness where switching models on the fly is easy. There's a reason why Anthropic uses their own proprietary formats/conventions anywhere they can - to lock you in when inference eventually commoditizes.
FYI, you can use your Claude subscription pricing with OpenCode via Meridian[0], which also makes it easier to try out other models when they come out. You can also use your other subscriptions in OpenCode with CLIProxyAPI[1]. The switching cost was relatively high, mostly from claude code plugins but completely worth it. I'm now mostly using GLM-5.3 and Codex models via OpenCode and barely using Claude which seemed unfathomable less than two months ago.
I'm sometimes tempted to use these sorts of proxies, but I always worry that the hacks they have to use to wrap the upstream tools and APIs is going to mess with my downstream harness/agent.
I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics:
Tested muse spark 1.2 because it was rated so high on design arena, and I've missed a model that can do nice UI in the hands of an operator with no UI skills.
It produced worse UI mockups than GPT and GPT models are already the bottom of the barrel here. The only model that performed well was Kimi K3 - insanely good, but expensive.
This matches my experience with Sol. Read and thought for a while, and edited files, tested, edited again, then ran out of budget in a relatively short time. But its solution was very good and was done quickly, so all things equal I prefer that over something much more verbose like Deepseek.
for over 1 billion real world users living in China, they don't have the option of paying $1.52 per task to use Opus 5, they are banned doing that due to US politics.
The Chinese models also like to cut corners on stuff like science. Their scores on stuff like biotech and scientific knowledge is far from ChatGPT unfortunately. (Claude is pretty good but it just refuses all prompts).
I've tested GLM 5.3 on the release day and Artificial Analysis is spot on. It's a really gold model.
But my main takeaway was something else. I've used closed weight models for long enough that I've forgotten how good it feels to see reasoning tokens.
With GPT/Claude, you kind of hope that intent was captured well, that agent had all the information, all the tools it needed, because you won't see "hmmm it seems like nix flake isn't available here and I shouldn't install something globally" until it slopped out millions of tokens and wasted hundreds of dollars for 8 hours. With GLM and the likes, you just stop the disease right where it begins.
Yes, not necessary often but being able to stop something that is going off the rails is super useful. Especially if the root cause is prompt ambiguity - inject a clarification & it recovers
It's also starting to go beyond reasoning and it's becoming much more problematic. Reasoning is one thing, but codex, for example now encrypts agent-to-agent messages as well, and compaction. I've no idea what subagents are instructed to do, or what they reported back in native codex.
The only thing that's keeping me is the value $200 subscription provides. If that value disappears, I see no reason why not to switch to something that isn't a black box.
Generally, are closed sourced models hiding their traces? I was making an agent to develop and deploy apps and fed the traces to dispel time-consuming detours and made it a few times faster.
I understand that running these benchmarks can get expensive, but it would be really nice to see AA include more benchmarks of models at reasoning settings other than the maximum, at least for the biggest releases. They have that nice graph of cost vs. composite benchmark score with the Pareto frontier line, but who knows if those are actually the optimal choices? There are already a few non-max-reasoning models on the Pareto line, among the few that were tested.
Yes, they have multiple levels of Claude, GPT, Gemini, and Kimi, but not the other top models (I would put GLM, Qwen, Muse, Grok, and Deepseek in that bucket).
56 comments
[ 0.21 ms ] story [ 6.6 ms ] threadGLM sets effort to max by default historically.
They're only a better value if you're paying API rates
At some point I will switch, $200 buys a lot of tokens on OpenRouter.
But regardless, you definitely should use a harness where switching models on the fly is easy. There's a reason why Anthropic uses their own proprietary formats/conventions anywhere they can - to lock you in when inference eventually commoditizes.
[0] https://github.com/rynfar/meridian
[1] https://github.com/router-for-me/CLIProxyAPI
edit: reworded for clarity
It produced worse UI mockups than GPT and GPT models are already the bottom of the barrel here. The only model that performed well was Kimi K3 - insanely good, but expensive.
It's hard to trust benchmarks these days.
generally I choose models by their intelligence and then personal preference from direct experience.
for over 1 billion real world users living in China, they don't have the option of paying $1.52 per task to use Opus 5, they are banned doing that due to US politics.
But my main takeaway was something else. I've used closed weight models for long enough that I've forgotten how good it feels to see reasoning tokens.
With GPT/Claude, you kind of hope that intent was captured well, that agent had all the information, all the tools it needed, because you won't see "hmmm it seems like nix flake isn't available here and I shouldn't install something globally" until it slopped out millions of tokens and wasted hundreds of dollars for 8 hours. With GLM and the likes, you just stop the disease right where it begins.
The only thing that's keeping me is the value $200 subscription provides. If that value disappears, I see no reason why not to switch to something that isn't a black box.
For web dev is just a must to have, and offloading that part to a secondary model doesn't work really well in my experience.
https://sumedh.info/models/glm-5-3