I've never used worst smartphones than anything from Xiaomi, bloated ad infested borderline malware territory fork of Android. Maybe just me but whenever I see them on HN I just can't think anything good about this company.
It's funny, I have the exact opposite reaction. This is probably misguided on my part, but Xiaomi is one of the very few major tech companies that I don't have an immediate strong negative reaction to. Everything I've bought from them, from robot vacuum to mobile phone, has been reasonably well designed, didn't break, and was priced fairly. I also think their car looks badass.
I'm sure they're doing all kinds of terrible things, like all major companies. I just can't help but like them. Also, this model looks great, and I'll give their subscription a shot next month.
I'm in Europe, and here the options for home appliances are usually German (e.g. Philips), Balkan (e.g. Gorenje), or Xiaomi. Xiaomi is the best by far, and it's honestly not even close.
Their home appliances are so rock-solid that they actually still surprise me. I've gone from, e.g., having to replace electric water kettles every six months to buying one from Xiaomi and never replacing it. (Nigh on three years now.)
I still have a xiaomi mi 11 lite, my wife has a 15t. The cameras are the best for the price. The way they chove ads down your throat at every opportunity should be illegal though
This is a big week. Probably getting next OpenAI and Anthro models, Grok 4.7, Mimo, etc. These open source model releases are why I can't take the "slow down" crowd seriously. I pitted older Mimo, qwen, step, gpt-oss, and other models against each other playing games like Werewolf and Sketch.io-like games where I let them talk shit while they played against each other. Mimo was by far pareto frontier of game-playing for the models that were <$0.15/m input tokens on OpenRouter. Qwen was pareto frontier in the shit talking game though. Qwen's hilarious. https://www.tiktok.com/@clankerfights/video/7642862917582425...
ah, would you look at that. I was wondering why mimo 2.5 became "dumber" the last weeks. I was speculating they are probably about to release a new version of the model. because the model really acted out a lot. especially the last two weeks. dont know, was just a feeling, highly speculative.
Looking at the frontend design examples; why do these models seem to love the "01 - UPPERCASE TEXT" motif. It's everywhere now (see https://try.cloudflare.com/, which has '01 · QUICK TUNNELS', but no "02" anywhere).
The extraneous small-caps labels are one of the main idiosyncrasies of AI generated markup. I wonder how much of this is a "scaffolding" technique to help the model build stable designs. But was it reinforced in RLHF or an emergent behavior of the models?
I know we have strong views on what a truly open model is (open weights, open training data, open training code etc.) but I really like how transparent they’ve been about the training of this model.
The dashboard they shared during training (https://mimo.xiaomi.com/rl/) was an incredible learning and teaching tool for me, and they’ve been unusually comprehensive in sharing details about their methodology (check out that tech report - it's got lots of clever behind the scene tricks like Google or Deepseek writeups) and benchmark scores (even the stuff they didn’t do well on).
If you’re releasing an open model going forward, please consider offering the community more of this transparency!
Thanks so much for sharing this. As someone who mostly watches from the sideline, can you share what you can see in this dashboard that someone like me can't see? Is it the metrics themselves that they measure (the metrics tab is absurdly detailed), something in the notices, or something else I missed?
The best thing they did is being open about all the setbacks they had to deal with. They logged every restart with a reason, talked about dropping a cyber dataset after it degraded coding benchmarks. Also published real time training loss, benchmark scores after every checkpoint and running cost estimates.
Really the only thing missing was dataset descriptions, the dashboard only had random IDs like "dataset-zrso". I guess it's their lawyers fault.
It'd be great to see a description of even just a subset of training datasets. It feels very much under-reported how much expense is worth investing in preparing and selecting training datasets versus just using masses of random quality unprepared training data.
For example, if they were to train on Wikipedia dumps, do they consider every article to be the same quality across each language, or have they done more work beyond Wikipedia's own article quality ratings to make training decisions such as "Ignore cebwiki it's machine-generated spam" and "Treat dewiki articles with coordinates within Germany as being higher quality (weight it higher) than their equivalent enwiki articles".
And let's say one of the datasets is all the source code of packages in the Gentoo package repository. Not every software package is a good example of how to write code. You perhaps wouldn't want to train your LLM on 1990s era PHP web application source code as an example of how to write code in 2026. Instead, you'd possibly want to use such PHP web application source code as a negative training example of what _not_ to write. But when training an LLM to detect software bugs, maybe outdated PHP source code is good for training.
Similarly for translation, perhaps UN treaty documents translated into 4+ languages are good translation examples because of high accuracy needed, professional translators being used, and bigger budgets. However this training data would perhaps be a negative training example towards translating chat messages, movie subtitles, etc because it doesn't use everyday slang and could result in output of nonsense such as "Pending Your Excellency's response, please accept, Your Excellency, my sincere greetings." for a prompt asking to write a birthday card for a child.
Preparing training data and deciding how to best use it for training I assume would be the largest expense (cost of labour -- mostly expert labour too) and also the greatest opportunity in the future for LLMs to improve. It seems to me somewhat irrelevant if the dashboard indicates a compute expense of $1m or $5m if good training datasets (prepared by experts in their fields) cost $10m/y to maintain. For example, hiring expert software developers to tag 1000's of open source software packages according to their quality, on different metrics, such as human readability, performance optimisation with choice of algorithms, reasonable trade-off between coherence and coupling in the software architecture, currency with state of the art programming trends/preferred dependencies/operating system APIs, etc. And keeping that metadata continually updated rather than a rapidly obsolete once off tagging project completed in 2005.
I might turn this into a blogpost if folks are interested, but my god there is so much clever info in that dashboard.
Here is one really neat bit:
A cutting edge training idea is on-policy RL, basically, it's not enough to say "here is an end to end agentic sequence (including tool calls etc.) that is perfect" you want to say "here is a sequence you might actually have generated that turns out to be correct".
Basically, it's more training efficient for models to improve with small tweaks to what they already do than from some perfect oracular answer (if you've ever tried to each people new skills, you’ve probably noticed this too :) )
When you do that, you care about how far the model you are updating (improving) has deviated from the one being used to generate rollouts (because agentic rollouts are painfully slow, so you can't keep redeploying every tweak).
Lo and behold, the dashboard literally has:
partial/avg_staleness (likely the measure of how many micro iterations the "generate answers" model is behind the "improving based on the occasional right answer" model)
train_infer_diff/new_infer/kl (a more direct KL divergence based way of measuring how much the two models have diverged)
Hold on, isn't that just standard practice for post-training LLMs for agentic use? Give task, generate n rollouts, grade rollouts (either at termination or after each tool call)? Or is the difference that the rollouts are generated ahead of time and then graded? (Of course, then it's not really on-policy.)
maybe this is why Dario want to slow down AI development and all the big AI labs in the USA is signing the same song. they are afraid of Chinese good enough LLM model killing their margin. we already have story about US companies switch some task to use cheaper Chinese model hosted on Neoclouds. this might be the same play that China have done with EV and solar panel. they flood the market with cheaper alternative and slowly gain market share.
Please explain how putting an upper bound on how good the strongest models can be prevents cheaper less strong models from catching up, rather than enabling it. I do not understand this argument at all.
> Please explain how putting an upper bound on how good the strongest models can be prevents cheaper less strong models from catching up, rather than enabling it. I do not understand this argument at all.
They are not proposing to regulate only the strongest models. They are proposing to regulate all models. If they are already on top, regulation may stop them from proceeding further, but it also stops the cheaper alternatives from catching up.
If they feel they have reached the asymptote of the curve, then regulation doesn't affect them, it affects those who have yet to reach the asymptote.
Particularly, the route they seem to want to go is "safety".
My guess is that Anthropic and OpenAI will push for "safety" regulations which require byzantine testing that, shocker, Anthropic and OpenAI can pass but the chinese models cannot. The route they'll take is import bans and potentially even general bans on products producing or using "unsafe" models.
They'll further likely try and push AI "safety" treaties from the US to other nations to further lock in their lead.
That's why, IMO, we've been seeing so many "OMG, AI will destroy the world and these AI researchers are so scared" articles.
Dario has always wanted the AI development to slow down and be more careful. Safer AI development was a core reason that Anthropic split off from OpenAI.
What's different today is that now all the big LLM firms want to slow down AI development. When men like Musk and Altman (both known for habitually shooting their mouths off and saying whatever they need to whoever needs to hear it regardless of truth) suddenly agree with Amodei, that's when things start to smell off.
The reason is money. They want regulation to make it harder for new competitors and competitors from other countries.
They invested billions into training the models but there is no competitive advantage, we see that within a couple of months everyone catches up. There is no way to profitability unless they get some policies to shields them against competitors that can't comply with the regulatory requirements.
That is also why there are things like Claude, Codex and Cursor. They are trying hard to build a customer relationship with a higher switching cost that hopefully sticks.
But the problem is that the AI buildout has become a large percentage of GDP. So obviously the government wants to keep it going because these companies are pumping enormous amounts of money into the economy.
> But the problem is that the AI buildout has become a large percentage of GDP. So obviously the government wants to keep it going because these companies are pumping enormous amounts of money into the economy.
They are pumping enormous amounts of money into each other. Hardly any of that is making its way to people, it's all going to highly automated construction and to energy use.
Seriously, how many jobs did the $1t in venture capital fund?
And, in this case, the dollar-amount increase in GDP serves as a virtual quantitative proxy for the increase in mowed lawns (and the value thereof). In other words, the participants in this economy are collectively ~$200 richer with their mowed lawns than they were without them.
If everyone goes around mowing lawns for each other, the economy is richer in lawn mowing at the expense of all the other things that would have been funded had everyone mowed their own lawns and purchased different services instead.
I am confused with this, if "everyone mowed their own lawns" then the net result will be exactly the same, everyone will be busy the same and not poorer, just without money movement.
This is not the same. If everyone wants mowed lawns, and everyone is busy working on that, there is no opportunity cost, everyone is working on their top priorities. The broken window fallacy is a fallacy because the headline gdp figure doesn't account for the destruction of the window which cancels out the benefit. In the grass mowing analogy nothing has been destroyed, useful and priority work has been done all around.
If the pricing is fair and at arms' length. What's happening in reality is as if they are mowing each others' lawns at wink wink nudge nudge $1000. Not a good proxy for actual value created.
In the real world, you have to pay taxes. So people are incentivized to claim less value for the lawns mowed, or even just do it themselves, instead of benefiting from the division of labour.
But also importantly the government of the residents' country is about 39% ($78) richer, if say the participants are honest in reporting this and the country is the UK and the participants are people like you and me in the tech industry who frequent HN and would think to do something like this.
> Hardly any of that is making its way to people, it's all going to highly automated construction and to energy use.
How do we know that? How automated is the construction really?
In any case, the Fed and other central banks can print as much money as they want in order to hit any aggregate spending or inflation target they have for the economy.
I’m still not at all sure about the “billions” invested claim. How much of that is cloud running the models? How much is pre and post training (which may or may not be part of what we’d want to include in accounting). Etc. Does anyone have links to good reporting about this: not blind recitations of numbers, but analysis and thought mixes with investigation?
Unless something has shifted, “everyone catches up” is because these bleeding edge models are distilled. You don’t see this happening with other European and US labs and the problem isn’t something being ignored. I’m not convinced this pattern will continue indefinitely.
I don’t follow. Fair use is a copyright defense, and nobody is suggesting distillation attacks are just a copyright violation are they?
Aren’t they alleging these other companies directly entered into a contract and violated the terms, and in cases where question, answer pairs were obtained without such agreement, it was accomplished by outright wire fraud or theft?
I did no such moral claim. I just noted that the foundation labs are working on technical hurdles to thwart distillation efforts and the cost and quality of Chinese models isn’t likely to keep up with the 6 month lag time everyone has assumed.
I don't understand on what you are basing this reasoning? If a well educated workforce can produce Fable then why couldn't a well educated workforce produce MiMo?
Besides, the latter actually published and open-sourced its RL stack to make it reproducible, which would in fact make it more trustworthy than the models you are speculating were distilled.
No chinese lab has caught up yet. They've tried to fake it by distilling and overfitting on benchmarks to make their models look better than they are, the 'best' models available from chinese labs right now (GLM 5.3 and Kimi K3) fall apart completely when you try to do real work with them. K3 is especially embarrassing because it is larger than Mythos yet performs worse than opus 5 and 5.6 sol in benchmarks they haven't been able to fake yet.
Well, if you are right, I just hope their protectionism will only affect the American market, and they leave us unAmericans free to get our models from wherever.
On the other hand (OTOH), China is desperate to keep up and keeps pushing open models (rightfully so), as they understand how far ahead from everyone the US is, and that whoever gets this right first basically is going to become an alien compared to others.
But even with all the open models the US is just insanely ahead in AI buildout and capital allocation (as usual).
Is it a bubble? Is it like the race for the-first-to-the-nuclear bomb? Both?
Does anyone know what are the proposed regulations? Controlling software is impossible, so the only option is banning hardware ownership. No more mac studio.
If you pay attention to how these US CEOs talk, it'll be "safety". If I were to guess, they'll try and require a lot of testing, validation, certification before a model is legally allowed to be used in the US or on US products.
It won't be a great moat, they'll probably try and get trade treaties setup to try and expand the moat. But ultimately it won't slow down chinese model development, just limit who can legally use them.
From the frontier labs, the only publicly stated one seemed to be to give them an exception from anti-trust laws to form a cartel and place - incidentally friendly - regulators in charge of monitoring everyone's work.
From politicians like Bernie Sanders, we've had proposals like 20 year imprisonment for anyone researching "ASI".
To me it’s clear that OAI and Anthropic are forced to release a better model every x months otherwise the Chinese ones will not only be cheaper but also better. So how could Dario show the investor that very nice profit chart representing profit = revenue excluding training costs and fees paid to Amazon aws?
They want to sell the same model for longer(a kind of software subscription where the cost of running /inference is cheap) but the Chinese doesn’t let them do it. That’s the gist of it. You can see already how they nerf the models just a week or so after release and try all kind of tricks to deliver you shitty performance for the same money. I think it’s part of the same issue of costs and enshitification plan
In a recent Dwarkesh podcast Dylan Patel breaks down how little compute the chinese labs actually have- not even the fact that they don't have access to new Nvidia chips and they're stealing them through shell companies- just that, even if they have cheap electricity, the compute just doesn't compare. Maybe even two orders of magnitude less. They couldn't get it even if they had the money. And if you look at how much more efficient newer chips are, that cuts the effective compute in half again. The conclusion was that they are at least 2-3 years behind.
For frontier labs the current compute seems to be driving model progress (in training) at least to some degree, even without true RSI, and this seems like it'll continue to keep any chinese model from drawing even with the frontier labs, at least for the foreseeable future.
Inevitably the chinese government will drive more funding in chip fab technology and the money will come around to build chinese data centers, but who knows how far off that is. A few different things in the tech tree need to fall into place. It doesn't seem like it'll be next year.
The counterpoint to that, though, is that the Chinese companies have to figure out how to be competitive, regardless of their significant compute deficit. And, as far as I can tell, they're actually doing that. They're trailing the frontiers in model effectiveness, but not by years. It's single digit months.
If there is no upper bound how how these things scale with compute, and if China does really begin to catch up to Nvidia (and they're probably not going to feel encumbered by US patents for domestic AI hardware, given how important AI seems to be to the Chinese government), there will come a day when China leapfrogs the US on AI.
This is definitely part of it. I think the reports/PR over the past month ended up being a serious unforced error.
Chinese models are increasingly closer to the frontier, while being able to run on much cheaper hardware than what US frontier models run on.
On top of that, both Anthropic and OpenAI showed that they can't really be trusted on data security.
Even if US companies can be forced to not use Chinese models, the rest of the world is going to see the risks and the availability of good enough open weight models for their purposes and be more likely to lean in favor of self-hosted Chinese models or local inference clouds.
> and the availability of good enough open weight models for their purposes
it is childish to believe Chinese are going to give up profits to provide such open weight models forever. the whole idea of being "open" is not compatible with the Chinese culture.
This almost racist read of other cultures has always seemed so bizarre to me. Even moreso when said as a counterargument to completely closed competitors, some outright seeking to ban open weights.
Chinese companies will continue to provide open weight models as long as it is profitable to do so. Chinese companies are on the more open end in many other industries despite the lack of meaningful foreign competition (for one, 3d printing) so there's plenty of reason to be optimistic as far as I'm concerned.
I mean DSv4.1 Flash and GLM 5.3 kept in check by a supervising frontier like Astra or Fable already in my experience clowns massively on ever using Opus or Sonnet. Opus 5 in particular has been such a stinker that they have to know that they're going to get smoked outside the halo models.
> got lots of clever behind the scene tricks like Google or Deepseek writeups) and benchmark scores
Xiaomi MiMo is led by Luo Fuli, a former Alibaba & DeepSeek employee. Perhaps it is due to Luo just how similar Xiaomi's tech & GTM approach is to DeepSeek's.
I wonder if this waters down the “distillation attack” claims by Anthropic. They have their own RL environments! I guess the caveat is that the RL datasets are still opaque, nothing is really proved.
I was absolutely mind blown when I saw how they were publishing that training dashboard while US models publish 100s of pages of reports (just provide a "copy as MD" button, folks, in the future). I was thinking about doing something similar but did not know how to show it, and this is a perfect example for someone who wants to show whatever they are training, for me it was local training on a consumer GPU.
My dream is to see this like a dashboard for a model trained across distributed machines, like Bitcoin mining, where minted coins are given to people whose machines were used for training. I don't know if they are worth it, but bragging rights alone, like a tag they can put on a website or social media, will be good enough for me.
This dashboard is almost certainly built on verl (https://github.com/verl-project/verl), which comes with a bunch of dashboarding capabilities built in (that doesn't look too dissimiliar to these dashboards).
Not an expert on this but I think the RL runs need to work sequentially? I wonder what the opportunities for distributed execution would be... Maybe parallelizing the benchmark task or inference
I think all of these can be addressed by being able to understand which machines contributed what to the model. I know that's messy, but if I can state "this information has come from Alice and the consensus is that it's good and right as it aligns with other information from Bob and others, meanwhile this information came from Mallory and stands out as being incongruent with the rest of the information I have in the model", we can identify malicious violations. If we're then able to state either Alice or Bob is one of the copyright holders, and one of them gets paid a little more, the other a little less as the confirmation agent, well, the economics of all of this changes a little.
At the moment we have Annas Archive being paid by frontier labs and rare/second hand books being destroyed in order to support the training regime. If instead we could just pay the publishers and they could distribute royalties to authors...
For (C) -- I think it could additional create jobs funded by government, philanthropic and other private institutions. For example, a government funded museum may already be participating in Wikimedia GLAM projects (e.g. uploading historical images to Wikimedia Commons with complete metadata). Perhaps this type of open source contribution may increase if organisations realise their mission can be better accomplished by contributing this same open data into LLMs, in addition to Wikimedia Commons. If the museum's mission is to educate the public on the history of life in ACMEville, having LLMs be able to provide historical information and images to a prompt of "What is the history of ACMEville?" may be a good pursuit.
I'm sceptical though whether use of LLMs would encourage creation of data that doesn't already exist. For example, if you ask an LLM "What are the top 100 most prevalent flora endemic to ACME National Park", this data may not currently exist _at all_, and to collect, would require paying botanists to do an extensive field survey. If no one has done this work yet--why? Is it relevant to the scientific community, to making government decisions, etc, or just an obscure academic curiosity. There are certainly some journal articles on _other_ national parks describing some of their common endemic flora, but perhaps there was a reason for this. Such as a scientist funded by a one-off government program trying to determine how to preserve or even create habitat for a specific endangered species.
Consider for the prompt of: "What are the top 100 most prevalent flora endemic to ACME National Park"
An LLM may reply: "I couldn't find any journal article or other prior work that may answer this question. Typically such survey field work may cost $X to complete, require expert botanists, and take 6-12 months to complete. Let me know if you want further information on how to find and select a company to conduct such a botanical field survey."
Would this type of LLM response grow the industry of botanical field surveys, or do nothing, perhaps because anyone likely to fund botanical field surveys is already doing so regardless of whatever is happening with AI.
I don't work with large language models, but out of curiosity: mimo's score on DeepSWE keeps going up, so why are they stopping training early? Is it due to budget constraints?
It looks like the curve is flattering, and the current state actually looks slightly cherry picked (it matches a previous spike that looks a bit of an outlier before the result went down). The longer they train, the more they risk getting scooped by another release by someone else. Etc etc it's a judgement call based on all of these factors (and more, including cost/occupying a big cluster as you mention)
+1 at some point, you need to expect to train a much better base model using everything you've learnt. At the least, you probably want to bring on line the next 10 clever RL environments and ideas your team has been cooking up (which will pipeline into v2.7 etc.)
Because the world is conditioned to distrust chinese models (pick your reason here), I believe this is critical for them in order to kill any arguments outside the actual merits. They probably spent a lot of time making this call and might pay off on the long run.
Whatever well-founded/or not distrust people have in Chinese models, this dashboard proves/shows nothing that can make them trust it more or less. It's like providing the journalctl logs of your HTTP server on your website and claim this proves NSA isn't listening or something.
Yeah, I agree it's probably fixable, but I think a naive interpretation of the metadata is probably the source of this bug (which HF has had for as long as I can remember)
Been playing around with both of these since last night. So far, when enabling reasoning (which is binary on/off), it seems to me like Flash either is less "token efficient" or just likes to think more, or I'm doing something else wrong, because most prompts I send to both, Flash reasons more and for longer than Pro, which is the opposite of my expectations.
Any lower (or broken) quantization could do that, not what I'm talking about though. They work fine for their size, as far as I can tell. Just surprised the Flash would reason for longer than the Pro.
All these new models are such tease for us folks with 128GB of shared memory. Buying another unit now to expand to 256GB is a mortgage payment but it’s getting tempting…
MiMo-V2.6-Flash-310B-A15B roughly GPT-5.6 Luna / Claude 4.9 according to benchmarks
MiMo-V2.6-Pro-1.02T-A42B roughly GPT-5.6 Sol / Opus 5 according to benchmarks
I really liked MiMo 2.5, it was really affordable and actually had vision, unlike DeepSeek. (DeepSeek has only recently added it)
Just tried 2.6 flash on a really niche topic I specialise in and it has done a really good job. They’ve definitely polluted their training data with claudeslop, but looking past the slop there is a decent model.
Can we afford to look past it? If/when claudeslop starts infecting every new model to such an extent, that model will produce its own slop, infecting new models... At what point do we lose all reliable methods for establishing "truth"? This is epistemic collapse waiting to happen. I honestly thought it would take longer... holding out for a coherent shared reality in 2030 seems optimistic.
In the chart they use "Pareto Line", which I think is wrong. Pareto is 20% effort leading to 80% results. Which could be interpreted as models costing 20% having 80% of peak intelligence, but that’s not what it looks like to me.
It looks like the "Frontier Line" to me, which is also often misinterpreted. frontier does not mean the best models. It means all models that are not strictly dominated, meaning in most cases: Not same price or cheaper and more intelligent.
I personally would like the word frontier to be used with more criterias: Open Weights, per use-case, etc etc. This would make model selection easier, but I understand it’s not an easy thing to do.
This is the Pareto Front [1], rather than the Pareto principle. It's the idea that anything that's more intelligent is more expensive and anything that's less expensive is less intelligent.
There are two (or more) concepts named after the same person:
- Pareto efficiency/Pareto curves: Basically the convex hull of points along the edge of a graph, indicating the best tradeoff between the axes. This is what the post is talking about.
- Pareto principle: this is the 80/20 rule you're talking about
Absolutely! Chinese models are both cheaper and more capable in many cases, compared to the American models and their makers continuously fumbling or reducing model capability with each update. Deepseek decreased costs when they released Flash 4.1 you would not see any American company do this, in reverse they would try charge you more.
> OpenAI reduced prices and Anthropic increased weekly usage limits.
As a Max x20 and Pro x20 subscriber, can tell you that it doesn't matter since they continually move the baseline of token use So in practice you feel that you're continually getting less from your subscription.
While it never happened to me in the past, i reached my weekly limit within 3 days using Opus 5. And the Open AI weekly limit essentially is a Claude Max x20 5-hour limit. Not even talking about the baseline in intelligence : on release day Astra was so good that it lead me to move to Pro x20. Now it's dumb af and token use is insane.
Deepseek 4.1 Flash has been a lifeboat for me, finally able to work without being constrained/distracted by limits and with what is in my view even better intelligence than Opus 5 for a fraction of the costs. DS is not messing up my brain with load-bearing pseudo jargon in every sentence. It respects coding guidelines, and completes even the most complex tasks most of the time in one shot.
DS 4.1 had been able to add complex features to my repo without breaking a sweat (330k lines of F# + 4M circa lines of an Angular frontend). Writes very idiomatic F# and respects our guidelines and style perfectly. Just completed an extensive UI/UX research and implementation work.
I am ditching both x20 subs and will only keep a Pro x5 because wife does a lot of design work and needs solid image generation capabilities.
The analogy makes little sense. The USA was not in front of the USSR and Sputnik merely showed that. It is at this point that the Americans woke up, put a lot of effort and finally were able to surpass the Soviets during the Apollo missions.
China was never ahead of the USA in AI. So perhaps a more proper analogy is the Moon landing. In real history the side that lost the race never got its mojo back...
Yep, I'm trending in that direction, and I'm someone with Claude stickers all over my laptop. My main app dev work is still going to Claude, but everything else is going to China even at API rates now.
One simple task: I needed an LLM to go through and clean up a few thousand page descriptions and titles in my personal search engine index, where the human web page authors had put in no effort sigh. I did a shoot out between Claude, Luna, GLM 5.3 Flash and Deepseek. Despite the high cost, Claude's descriptions were terrible, and even Opus warned me that the descriptions coming back from Haiku were "generalized, not accurate". I expected I would choose Luna because of price, and occasionally it did have wonderful descriptions (one captured emotion in a way no other model did). But in the end, the GLM 5.3 Flash descriptions were the easiest to read, they flow well while also being accurate & including necessary keywords, and being highly affordable. So it won out. It's a task that is nowhere near frontier, but a task where somehow China is better than frontier.
API rates still aren’t quite competitive with the OpenAI x20 accounts, but they are definitely getting close with deepseek 4.1 flash. I spent a few days with only 4.1 and was very impressed.
Absolutely! DeepSeek-V4-Flash-0731 has become my daily driver. It's pretty amazing what it can do for what it costs at deepinfra.com (I don't use deepseek as a provider since they train on your data [at least their honest about it]). GLM-5.1 was my daily driver before that and Kimi K2.5 before that.
My primary use is AI coding agent. Its vastly cheaper than Kimi K3 and I haven't found a scenario where I really need Kimi K3 versus smaller models. GLM-5.3 Flash is good but there is series of bugs in the vllm middleware that prevent GLM models from getting all of their reasoning content returned to them that impairs inference quality. A lot of inference providers use vllm which makes it hard to find a good provider for GLM. I've been using friendli.ai but using GLM-5.3 Flash from them is more expensive then using DS V4 Flash from deepinfra.com simply because deepinfra.com is so cheap. The DS V4 Flash cost at together.ai is similar to the GLM-5.3 Flash from friendli.ai or at least that's what I found in my benchmarks a week ago: https://www.linkedin.com/posts/joshheitzman_i-ran-a-fuller-r...
4.1 consistently surprises me in capability for the price. And I don't think I'm the only one. It's been dominating the leaderboard at OpenRouter, and I just got an email today from Fireworks saying they were _raising_ the price by about 30%. I'll probably switch, because their infra doesn't support being the highest-cost, but it's still telling.
I tried it a few times and liked the speed, but often found it ended up looping, i.e. repeating the same token sequence (e.g. the same sequence of 5 paragraphs) over and over again until it hit the max output limit. This doesn't end up happening every session, but does every now and then.
My impression of DSv4.1-flash was very positive aside from this. But that was enough for me to stick with GLM-5.3(-flash), which both gave me consistently great results
I was using a vibe coded bare bones harness.
I was wondering if this was normal from DSv4.1-flash, or if its my harnesses fault.
I've had that looping issue with open models too. But never 4.1. I wonder if it's a model + harness combo? But yeah, one loop issue and I'm done with a model forever.
I haven't tried 4.1 flash as I'm assuming its a preview. I did not get good results from the preview version of 4.0 flash (i.e. the one that did not include the month and date of release in its name).
I did try to use Chinese open models, but for my production work they simply couldn't cope at all; both GLM 5.3 and Deepseek v4 went into infinite loop and wasted my tokens until my OpenRouter wallet reached 0; good thing I didn't enable the auto topup. US models, by contrast, breezed past them.
Even for simpler tasks, Chinese models took long time to complete, and I needed to supervise closely. The price , in the end, didn't come cheap, mainly because too much time wasted on thinking.
So maybe one day Chinese models will squeeze out the American ones, but today is not that day.
So no, I am not excited about Chinese models ( just because its open weight and not American).
Not too be "that guy" (e.g. "you're using it wrong"), I just want to humbly ask — have you tried blacklisting "underperformers" in OpenRouter config?
Here on HN was a post few days ago titled like "so you want to use openrouter", there was a benchmark in capabilities between providers which showed some aggressively quantize and basically break models and tool calling.
I am in no way a professional power user, but I frequently suffered from "call fails" (e.g. unclosed tags, broken agent loop, broken thinking blocks), so I had to babysit agent on it's loop. After I blacklisted like 20 providers (I think most broken were Nebius and DigitalOcean) these issues completely went away. I had several agents work on my small tasks for 18+ hours with no issues.
The cost issue is obviously of prime importance, but I'd also argue that the transparency of the innovations creates a tremendous cross-pollination, and not only within the Chinese communities but in the US/Europe as well. How many of us are learning the practical aspects of actually running and building AI based primarily on open models? As an example - how far would the work of vLLM or SGLang or even NVIDIA itself (all random examples) be without these models and the challenges they pose?
American models are on the frontier of capability. Chinese models are on the frontier of efficiency. The problem for American labs is that Chinese models are more than capable enough for the vast majority of applications that people care about at this point, so efficiency is more interesting for people.
it's really weird to me at the moment because both OpenAI and Anthropic seem to be competing in an extreme benchmaxxing contest on super intelligence that actually nobody cares about. I haven't really cared about model intelligence since about Opus 4.8. It is by far not my biggest problem. I don't need to replace or support Einstein in my production workflow. I just need basic intelligence that can equal a routine office worker - safely and reliably. What they doing - chasing super-intelligence but dramatically escalating risk - is actively what I don't need.
I really think they have drunk too much of their own kool aid and become completely detached from what the market wants.
Affordability is derivative of control which is really what I care about.
I'm just not going to build long term infra that depends on something that another person can and will - objectively based on experience - take away from me at some unknown point in the future.
The biggest benefit of open models is they keep all the other players honest. The extent to which they feel they can dictate terms is directly set by the threshold where they feel people will take the trade to run open models instead.
They match my experience. Astra and Fable I rate below Sonnet. They are incredibly poor. They were excellent for a couple of days after release and then plummeted.
Maybe I am being routed to more quantised versions or less capable models with system prompt to fake Astra or Fable.
Yeah, for better or worse, writing style is practically uncorrelated with agentic performance, which is all the rage right now and the thing that most popular benchmarks currently prioritize.
Yeah it’s a shame a lot of these benchmarks are behind. My favourite was ‘SlopCodeBench’ [1] as I’m most interested in ai reinforcing its own bad decisions, but it’s not even up to current gen oai
We're past the one model fits them all kind of LLM. Most of the recent release actually regress on world knowledge for example, but optimize for something different: tool usage, thinking process, and agentic approach. And yes, in my own usage, some usecases Opus beats Fable.
I think we are still far from nailing down good LLM benchmarks, because the more general-purpose your software the harder the question of what makes it good becomes. Is Python a good programming language? Is Java? Is C? I think it's a similar class of problem. You can benchmark rudimentary things like execution speed similar to how you can benchmark tokens/second, but these metrics don't tell the whole story.
A sentence that I would not have expected to read on HN as recently as last year, but that makes perfect sense today.
Jokes asides, @simonw any plan to include a 3D model version (make a 3D model of a Pelican riding a bicycle in Blender)?
Given that Astra seems to have improved a lot in 3D modeling capabilities (and that matches my experience) I'm actually quite interested to see if/when other models catch up and how they stack against it.
Or if anyone knows what would be a useful existing benchmark for that skill.
I've not committed to that as a benchmark yet because you really need a full coding agent configured to run Blender, and I like benchmarks I can run as a single prompt/response through the appropriate API.
I have been intending to get more of a coding agent benchmark going though, so maybe this should be part of it.
Making SVGs is coding (of a very specific markup language). So it's not actually absurd. We've also seen that more capable models are generally better at everything minus benchmark contamination (benchmaxxing) anomalities.
can you update this website, I just wish the entire layout wouldn't shift when the page gets loaded and the timestamps in the title look very ugly and take up a lot of space.
Not sure if it is the same thing they're referring, but the transition from loading the prompt from the gist to the result is indeed jarring: https://files.catbox.moe/c56eyl.mp4
I don't believe it is that important, but I guess you might've just hit some people's pet peeve with that. If anything, it goes to show as to why skeletons are popular on modern sites
A different pelican on a different bicycle direction(R2L) finally. Wondering why MiMo V2.6 Pro choose R2L and Flash choose L2R for the bicycle direction.
pro model was having fun on minimal effort with that fish lmao, hilarious.
out of curiosity, is it the first time we're seeing a fish there? I don't remember seeing that detail before.
The moat for OAI and anthropic seems to be very quickly shrinking. Chinese labs are now using RSI-like approaches and even without resorting to heavy distillation they're catching up in a couple of months vs. what would have been 6-12 months a year prior.
And as these models get better the pace of training is quickly speeding up too.
This doesn't bode particularly well for anthropic/OAI after they go public.
I don't know if we will look at OpenAI and Anthropic as moating on frontier models & selling tokens.
They are banking on the application layer and accumulated business and end user context. They have to quickly make that systems integrated value out weigh the model choice price value in the broader market
Leaning into what it cost to train is hilarious and an obvious shot at US frontier labs spending tens to hundreds of millions or more to train their models.
351 comments
[ 0.35 ms ] story [ 52.3 ms ] threadSome features of the release I like:
- Demonstration of diverse tasks, such as using a DAW
- Graphs from various benchmarks and price ranges
- Real world use of the model in scientific environments
Curious if Verizon / ATT still force apps on your phone, eg. NFL and Amazon apps, Fi service is subpar
I'm sure they're doing all kinds of terrible things, like all major companies. I just can't help but like them. Also, this model looks great, and I'll give their subscription a shot next month.
I'm in Europe, and here the options for home appliances are usually German (e.g. Philips), Balkan (e.g. Gorenje), or Xiaomi. Xiaomi is the best by far, and it's honestly not even close.
Their home appliances are so rock-solid that they actually still surprise me. I've gone from, e.g., having to replace electric water kettles every six months to buying one from Xiaomi and never replacing it. (Nigh on three years now.)
I really have a very positive impression of them.
I ask because my wife has the 15T and the camera is better than my iPhone 17 Pro. And while toying around with it I didn't notice any bloat.
Plus hers support native split screen which I kinda need to multitask on the go.
I'm so pissed at how bad Siri is compared to her android phone that I'm thinking about selling the iPhone to get a Huawei Pura Ultra.
but now I got my "proof".
The dashboard they shared during training (https://mimo.xiaomi.com/rl/) was an incredible learning and teaching tool for me, and they’ve been unusually comprehensive in sharing details about their methodology (check out that tech report - it's got lots of clever behind the scene tricks like Google or Deepseek writeups) and benchmark scores (even the stuff they didn’t do well on).
If you’re releasing an open model going forward, please consider offering the community more of this transparency!
Really the only thing missing was dataset descriptions, the dashboard only had random IDs like "dataset-zrso". I guess it's their lawyers fault.
It'd be great to see a description of even just a subset of training datasets. It feels very much under-reported how much expense is worth investing in preparing and selecting training datasets versus just using masses of random quality unprepared training data.
For example, if they were to train on Wikipedia dumps, do they consider every article to be the same quality across each language, or have they done more work beyond Wikipedia's own article quality ratings to make training decisions such as "Ignore cebwiki it's machine-generated spam" and "Treat dewiki articles with coordinates within Germany as being higher quality (weight it higher) than their equivalent enwiki articles".
And let's say one of the datasets is all the source code of packages in the Gentoo package repository. Not every software package is a good example of how to write code. You perhaps wouldn't want to train your LLM on 1990s era PHP web application source code as an example of how to write code in 2026. Instead, you'd possibly want to use such PHP web application source code as a negative training example of what _not_ to write. But when training an LLM to detect software bugs, maybe outdated PHP source code is good for training.
Similarly for translation, perhaps UN treaty documents translated into 4+ languages are good translation examples because of high accuracy needed, professional translators being used, and bigger budgets. However this training data would perhaps be a negative training example towards translating chat messages, movie subtitles, etc because it doesn't use everyday slang and could result in output of nonsense such as "Pending Your Excellency's response, please accept, Your Excellency, my sincere greetings." for a prompt asking to write a birthday card for a child.
Preparing training data and deciding how to best use it for training I assume would be the largest expense (cost of labour -- mostly expert labour too) and also the greatest opportunity in the future for LLMs to improve. It seems to me somewhat irrelevant if the dashboard indicates a compute expense of $1m or $5m if good training datasets (prepared by experts in their fields) cost $10m/y to maintain. For example, hiring expert software developers to tag 1000's of open source software packages according to their quality, on different metrics, such as human readability, performance optimisation with choice of algorithms, reasonable trade-off between coherence and coupling in the software architecture, currency with state of the art programming trends/preferred dependencies/operating system APIs, etc. And keeping that metadata continually updated rather than a rapidly obsolete once off tagging project completed in 2005.
Here is one really neat bit:
A cutting edge training idea is on-policy RL, basically, it's not enough to say "here is an end to end agentic sequence (including tool calls etc.) that is perfect" you want to say "here is a sequence you might actually have generated that turns out to be correct".
Basically, it's more training efficient for models to improve with small tweaks to what they already do than from some perfect oracular answer (if you've ever tried to each people new skills, you’ve probably noticed this too :) )
When you do that, you care about how far the model you are updating (improving) has deviated from the one being used to generate rollouts (because agentic rollouts are painfully slow, so you can't keep redeploying every tweak).
Lo and behold, the dashboard literally has:
partial/avg_staleness (likely the measure of how many micro iterations the "generate answers" model is behind the "improving based on the occasional right answer" model)
train_infer_diff/new_infer/kl (a more direct KL divergence based way of measuring how much the two models have diverged)
How cool is that?!
They are not proposing to regulate only the strongest models. They are proposing to regulate all models. If they are already on top, regulation may stop them from proceeding further, but it also stops the cheaper alternatives from catching up.
If they feel they have reached the asymptote of the curve, then regulation doesn't affect them, it affects those who have yet to reach the asymptote.
My guess is that Anthropic and OpenAI will push for "safety" regulations which require byzantine testing that, shocker, Anthropic and OpenAI can pass but the chinese models cannot. The route they'll take is import bans and potentially even general bans on products producing or using "unsafe" models.
They'll further likely try and push AI "safety" treaties from the US to other nations to further lock in their lead.
That's why, IMO, we've been seeing so many "OMG, AI will destroy the world and these AI researchers are so scared" articles.
What's different today is that now all the big LLM firms want to slow down AI development. When men like Musk and Altman (both known for habitually shooting their mouths off and saying whatever they need to whoever needs to hear it regardless of truth) suddenly agree with Amodei, that's when things start to smell off.
not all, just a few American ones, there are other big American LLM developers (notables include Nvidia and Meta) that do not agree
They invested billions into training the models but there is no competitive advantage, we see that within a couple of months everyone catches up. There is no way to profitability unless they get some policies to shields them against competitors that can't comply with the regulatory requirements.
That is also why there are things like Claude, Codex and Cursor. They are trying hard to build a customer relationship with a higher switching cost that hopefully sticks.
But the problem is that the AI buildout has become a large percentage of GDP. So obviously the government wants to keep it going because these companies are pumping enormous amounts of money into the economy.
They are pumping enormous amounts of money into each other. Hardly any of that is making its way to people, it's all going to highly automated construction and to energy use.
Seriously, how many jobs did the $1t in venture capital fund?
If everyone goes around mowing lawns for each other, the economy is richer in lawn mowing at the expense of all the other things that would have been funded had everyone mowed their own lawns and purchased different services instead.
Person B owns shares in Person A.
> eru
Tolkien fan?
Value was created!
How do we know that? How automated is the construction really?
In any case, the Fed and other central banks can print as much money as they want in order to hit any aggregate spending or inflation target they have for the economy.
When eg I snap a picture of my dog, that's not derived from anything. But I still get intellectual property rights for the photograph.
Aren’t they alleging these other companies directly entered into a contract and violated the terms, and in cases where question, answer pairs were obtained without such agreement, it was accomplished by outright wire fraud or theft?
Besides, the latter actually published and open-sourced its RL stack to make it reproducible, which would in fact make it more trustworthy than the models you are speculating were distilled.
But even with all the open models the US is just insanely ahead in AI buildout and capital allocation (as usual).
Is it a bubble? Is it like the race for the-first-to-the-nuclear bomb? Both?
It won't be a great moat, they'll probably try and get trade treaties setup to try and expand the moat. But ultimately it won't slow down chinese model development, just limit who can legally use them.
From politicians like Bernie Sanders, we've had proposals like 20 year imprisonment for anyone researching "ASI".
They want to sell the same model for longer(a kind of software subscription where the cost of running /inference is cheap) but the Chinese doesn’t let them do it. That’s the gist of it. You can see already how they nerf the models just a week or so after release and try all kind of tricks to deliver you shitty performance for the same money. I think it’s part of the same issue of costs and enshitification plan
For frontier labs the current compute seems to be driving model progress (in training) at least to some degree, even without true RSI, and this seems like it'll continue to keep any chinese model from drawing even with the frontier labs, at least for the foreseeable future.
Inevitably the chinese government will drive more funding in chip fab technology and the money will come around to build chinese data centers, but who knows how far off that is. A few different things in the tech tree need to fall into place. It doesn't seem like it'll be next year.
If there is no upper bound how how these things scale with compute, and if China does really begin to catch up to Nvidia (and they're probably not going to feel encumbered by US patents for domestic AI hardware, given how important AI seems to be to the Chinese government), there will come a day when China leapfrogs the US on AI.
For instance a Chinese EUV machine seems like it's very far away. Even if they have (steal/borrow) the necessary IP.
Chinese models are increasingly closer to the frontier, while being able to run on much cheaper hardware than what US frontier models run on.
On top of that, both Anthropic and OpenAI showed that they can't really be trusted on data security.
Even if US companies can be forced to not use Chinese models, the rest of the world is going to see the risks and the availability of good enough open weight models for their purposes and be more likely to lean in favor of self-hosted Chinese models or local inference clouds.
it is childish to believe Chinese are going to give up profits to provide such open weight models forever. the whole idea of being "open" is not compatible with the Chinese culture.
Chinese companies will continue to provide open weight models as long as it is profitable to do so. Chinese companies are on the more open end in many other industries despite the lack of meaningful foreign competition (for one, 3d printing) so there's plenty of reason to be optimistic as far as I'm concerned.
We, as consumers, the majority of the world, can make deals that benefit us most only when there is enough competition on the other side.
Xiaomi MiMo is led by Luo Fuli, a former Alibaba & DeepSeek employee. Perhaps it is due to Luo just how similar Xiaomi's tech & GTM approach is to DeepSeek's.
- How Luo Fuli Keeps an Earthy Touch as she Soars Through the AI World, https://newsen.pku.edu.cn/news_events/news/people/15385.html (https://archive.vn/I8Pmu).
- Luo Fuli, the 30-year-old ‘AI genius girl’ behind DeepSeek’s success?, https://e.vnexpress.net/news/tech/personalities/who-is-luo-f... (https://archive.vn/sb3B6).
I wonder if this waters down the “distillation attack” claims by Anthropic. They have their own RL environments! I guess the caveat is that the RL datasets are still opaque, nothing is really proved.
My dream is to see this like a dashboard for a model trained across distributed machines, like Bitcoin mining, where minted coins are given to people whose machines were used for training. I don't know if they are worth it, but bragging rights alone, like a tag they can put on a website or social media, will be good enough for me.
A) How to prevent malicious injection of bad training data?
B) How to handle copyright violations, will participants be responsible and will they have to pay the creators?
C) Can this create an income stream for content creators and how to avoid abuse, eg feeding with AI content?
So many more, but let’s focus on these before we break things fast because we didn’t think about them.
At the moment we have Annas Archive being paid by frontier labs and rare/second hand books being destroyed in order to support the training regime. If instead we could just pay the publishers and they could distribute royalties to authors...
I'm sceptical though whether use of LLMs would encourage creation of data that doesn't already exist. For example, if you ask an LLM "What are the top 100 most prevalent flora endemic to ACME National Park", this data may not currently exist _at all_, and to collect, would require paying botanists to do an extensive field survey. If no one has done this work yet--why? Is it relevant to the scientific community, to making government decisions, etc, or just an obscure academic curiosity. There are certainly some journal articles on _other_ national parks describing some of their common endemic flora, but perhaps there was a reason for this. Such as a scientist funded by a one-off government program trying to determine how to preserve or even create habitat for a specific endangered species.
Consider for the prompt of: "What are the top 100 most prevalent flora endemic to ACME National Park"
An LLM may reply: "I couldn't find any journal article or other prior work that may answer this question. Typically such survey field work may cost $X to complete, require expert botanists, and take 6-12 months to complete. Let me know if you want further information on how to find and select a company to conduct such a botanical field survey."
Would this type of LLM response grow the industry of botanical field surveys, or do nothing, perhaps because anyone likely to fund botanical field surveys is already doing so regardless of whatever is happening with AI.
Xiaomi Mimo 2.6 live post-training dashboard - https://news.ycombinator.com/item?id=49732270 - Sept 2026 (155 comments)
no idea why I laughed so hard at this, feel so familiar
I think you mean mostly the US
Pro [2]:, 1.02T total / 42B activated parameters
[1]: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL
[2]: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL
https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
Are others seeing the same thing?
This is also true for Deepseek 4(.1) .
Close but yet so far. Nobody has released a Q3 that fits so far. I think 95-115gb is the sweet spot
so weird to acknowledge someone being on the front edge, but not name it
MiMo-V2.6-Flash-310B-A15B roughly GPT-5.6 Luna / Claude 4.9 according to benchmarks MiMo-V2.6-Pro-1.02T-A42B roughly GPT-5.6 Sol / Opus 5 according to benchmarks
Perhaps with IQ2 flash will run on 128G M5?
Just tried 2.6 flash on a really niche topic I specialise in and it has done a really good job. They’ve definitely polluted their training data with claudeslop, but looking past the slop there is a decent model.
It looks like the "Frontier Line" to me, which is also often misinterpreted. frontier does not mean the best models. It means all models that are not strictly dominated, meaning in most cases: Not same price or cheaper and more intelligent.
I personally would like the word frontier to be used with more criterias: Open Weights, per use-case, etc etc. This would make model selection easier, but I understand it’s not an easy thing to do.
[1]: https://en.wikipedia.org/wiki/Pareto_front
- Pareto efficiency/Pareto curves: Basically the convex hull of points along the edge of a graph, indicating the best tradeoff between the axes. This is what the post is talking about.
- Pareto principle: this is the 80/20 rule you're talking about
What you call "frontier line" is also called "Pareto frontier" https://en.wikipedia.org/wiki/Pareto_front
Your description of it is basically correct though
OpenAI reduced prices and Anthropic increased weekly usage limits.
As a Max x20 and Pro x20 subscriber, can tell you that it doesn't matter since they continually move the baseline of token use So in practice you feel that you're continually getting less from your subscription.
While it never happened to me in the past, i reached my weekly limit within 3 days using Opus 5. And the Open AI weekly limit essentially is a Claude Max x20 5-hour limit. Not even talking about the baseline in intelligence : on release day Astra was so good that it lead me to move to Pro x20. Now it's dumb af and token use is insane.
Deepseek 4.1 Flash has been a lifeboat for me, finally able to work without being constrained/distracted by limits and with what is in my view even better intelligence than Opus 5 for a fraction of the costs. DS is not messing up my brain with load-bearing pseudo jargon in every sentence. It respects coding guidelines, and completes even the most complex tasks most of the time in one shot.
DS 4.1 had been able to add complex features to my repo without breaking a sweat (330k lines of F# + 4M circa lines of an Angular frontend). Writes very idiomatic F# and respects our guidelines and style perfectly. Just completed an extensive UI/UX research and implementation work.
I am ditching both x20 subs and will only keep a Pro x5 because wife does a lot of design work and needs solid image generation capabilities.
debatable if a turn around is possible before '29
China was never ahead of the USA in AI. So perhaps a more proper analogy is the Moon landing. In real history the side that lost the race never got its mojo back...
I expect this to happen within 12-18 months, the differentiation has shrunk, many models are now sufficiently capable for most tasks
One simple task: I needed an LLM to go through and clean up a few thousand page descriptions and titles in my personal search engine index, where the human web page authors had put in no effort sigh. I did a shoot out between Claude, Luna, GLM 5.3 Flash and Deepseek. Despite the high cost, Claude's descriptions were terrible, and even Opus warned me that the descriptions coming back from Haiku were "generalized, not accurate". I expected I would choose Luna because of price, and occasionally it did have wonderful descriptions (one captured emotion in a way no other model did). But in the end, the GLM 5.3 Flash descriptions were the easiest to read, they flow well while also being accurate & including necessary keywords, and being highly affordable. So it won out. It's a task that is nowhere near frontier, but a task where somehow China is better than frontier.
I just had like four big sessions going today, paid about $8 in tokens. I see no reason to pay more, this is more than I need for intelligence.
My impression of DSv4.1-flash was very positive aside from this. But that was enough for me to stick with GLM-5.3(-flash), which both gave me consistently great results
I was using a vibe coded bare bones harness. I was wondering if this was normal from DSv4.1-flash, or if its my harnesses fault.
So if you use MCP a lot, simplify the params, be more lenient on validation and rework the errors.
It is quite good with shell.
glm 5.3 flash is a tad slower but a bit more capable and way more token efficient.
Source: self hosted tested on rented GB200 node at 8bit.
Months ago I switched entirely to use Chinese model. Mostly DeepSeek and MiMo, although I recently started to play with GLM as well.
The models are excellent and in many ways I prefer them to Claude.
I see no difference in terms of capability, but the fact that they are cheap frees me to experiment.
So maybe one day Chinese models will squeeze out the American ones, but today is not that day.
So no, I am not excited about Chinese models ( just because its open weight and not American).
Here on HN was a post few days ago titled like "so you want to use openrouter", there was a benchmark in capabilities between providers which showed some aggressively quantize and basically break models and tool calling.
I am in no way a professional power user, but I frequently suffered from "call fails" (e.g. unclosed tags, broken agent loop, broken thinking blocks), so I had to babysit agent on it's loop. After I blacklisted like 20 providers (I think most broken were Nebius and DigitalOcean) these issues completely went away. I had several agents work on my small tasks for 18+ hours with no issues.
I really think they have drunk too much of their own kool aid and become completely detached from what the market wants.
I'm just not going to build long term infra that depends on something that another person can and will - objectively based on experience - take away from me at some unknown point in the future.
The biggest benefit of open models is they keep all the other players honest. The extent to which they feel they can dictate terms is directly set by the threshold where they feel people will take the trade to run open models instead.
Maybe Terminal Bench 4.0 and ExploitGym are reasonable.
Terminal Bench 4.0
GPT 6 Astra 59.6 Claude Fable 5.1 55.1 Claude Opus 5 49.0 MiMo-V2.6-Pro 34.9 MiMo-V2.6-Flash 28.8 DeepSeek V4.1 Flash 26.8 MiMo-V2.5-Pro 1.5
ExploitGym
GPT 6 Astra 42.4 Claude Fable 5.1 30.4 Claude Opus 5 22.1 MiMo-V2.6-Pro 17.8 MiMo-V2.6-Flash 6.0 MiMo-V2.5-Pro 0.1
DeepSWE v1.1
DeepSeek V4.1 Flash 74.2 Claude Opus 5 74.0 GPT 6 Astra 74.0 MiMo-V2.6-Pro 71.9 Claude Fable 5 70.0 MiMo-V2.6-Flash 67.9 MiMo-V2.5-Pro
Maybe I am being routed to more quantised versions or less capable models with system prompt to fake Astra or Fable.
1 - https://bench.killswitch-lang.org/
TerminaBench, DeepSwe sites are out of date.
1: https://www.scbench.ai/
Pelicans for Pro: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Jokes asides, @simonw any plan to include a 3D model version (make a 3D model of a Pelican riding a bicycle in Blender)?
Given that Astra seems to have improved a lot in 3D modeling capabilities (and that matches my experience) I'm actually quite interested to see if/when other models catch up and how they stack against it.
Or if anyone knows what would be a useful existing benchmark for that skill.
I've not committed to that as a benchmark yet because you really need a full coding agent configured to run Blender, and I like benchmarks I can run as a single prompt/response through the appropriate API.
I have been intending to get more of a coding agent benchmark going though, so maybe this should be part of it.
Qwen3.8-27b pelican was amazing on Mac.
https://www.nudgehost.com/dpjn3uwe
https://simonwillison.net/2026/Sep/18/the-creative-spirit-of...
I don't believe it is that important, but I guess you might've just hit some people's pet peeve with that. If anything, it goes to show as to why skeletons are popular on modern sites
It's because offpeak electricity is cheaper?
And as these models get better the pace of training is quickly speeding up too.
This doesn't bode particularly well for anthropic/OAI after they go public.
They are banking on the application layer and accumulated business and end user context. They have to quickly make that systems integrated value out weigh the model choice price value in the broader market
Unknown if they will be able to pull that off.