VRAM aint cheap, Sam Altman ruined the cost of memory, Nvidia doesnt make enough consumer GPUs letting the market go insane over them, I still have friends on 1070s or 1070 TIs because GPUs have been severely overpriced for too long. I remember when a gaming PC was only $1000.
Even so why would anyone not sleep on a model they cannot run?
seriously, if a single politician stepped forward and said "i'll bring down ram prices" they could then shoot a puppy and call me a slur and I'd still go out and doorknock for them.
You won't see it because this politician wouldn't survive one rally before getting shot, and even so it would be hard to bring prices down. There would have to be major investigation into ram vendors forming cartel in order to break them, and major investments to increase production, because this is also a supply shortage fulled inflation
I don't think RAM vendors have formed a cartel and I think this is knee-jerk anger without any thought. RAM is a commodity product with massive upfront capex costs, and those always have boom-and-bust cycles. At various points in the 2010s and 2020s RAM vendors were getting eaten alive by a supply glut, this would not have happened if they were a cartel.
Is it really so hard to believe that RAM prices are up because demand is simply exceeding supply, especially in a market where additional supply takes years and billions of dollars to come online? There's no need to posit cartel behavior and a fair amount of evidence that there is none.
The AI boom started in 2022. Prices rose THREE years later after 2025 sanction and tariff style legislation to protect the market during a price hike.
I got a 4090 in 2023 for 1600, a 5090 in 2025 for 2000 with 256 DDR5 for about $1,000 ... and then, after some protectionist legislation passed, these prices quickly shot to the moon.
Man I think you're just spewing word salad and a lot of what you've written is either wrong or not even wrong. The AI hype really got started in 2022, but hype on social media doesn't mean anything for RAM prices, only real buildouts do that. They rose pretty steadily until OpenAI revealed their shenanigans re: locking up a ton of supply from two different vendors with secret contracts, and that's when the takeoff really started. This is definitely scummy behavior from OpenAI (big surprise), and I actually think they arguably should see an antitrust investigation for that (not that that will ever happen), but OpenAI is a buyer; that's not the same thing as the vendors forming a cartel.
You can't say "connect the dots" at the end of a raving, mostly-incorrect post and act like you've made an ironclad argument.
I mean, just because OpenAI started it doesn't mean the vendors didn't form a cartel afterwards (or conspire together) to ensure maximum profits in a "crazy high demand" situation..
Micron has 3 brand new fabs currently under construction, 2 Boise, 1 in New York as the first of 4 planned for a campus.
Plus expanding other existing facilities.
These things take ~3-5 years from breaking ground to full production. You'd have had to anticipate the current demand years before it happened in order to be bringing production on-line before 2030 or so.
Samsung and HK Hynix also have fabs under construction and planned.
CXMT started 11 years ago and only now is reaching any real volume. If they decided a year ago to react to the current demand cycle they'd be 6-7 years out.
Not much you can really do to wish for more fabrication to exist on any timeline not measured in fractional decades.
Could they do more and react quicker? Probably, but everything I've read on the subject seems to point to 3 years is absolute bare minimum if you happen to have a shovel ready project with the land bought, local permitting completed, infrastructure extended to the site, and a skilled workforce already in place. They could suspend buy-backs/dividends today and dump it all into building production and there would be no material impact until around 2030.
> The Micron CEO just recently said this is the exact plan
CEO simply stated the demand pressure will not go away through 2027, and supply will not increase until around 2028 when currently under construction fabs start shipping volume. The article does not support your statement.
You can see the SKUs they base the prices on by hovering over a data point, and the products just don't generally represent the market. Oftentimes the whole market is represented by some bottom of the barrel legacy memory, single stick in some warehouse. During the last years DDR3 and DDR4 haven't gotten as expensive as the current generations, and I think the memory compatible with current systems should represent the market rather than a Toshiba Satellite 4 GB extension kit.
Capitalism eats itself this way. Second and third order effects will collapse the demand.
You need to keep the market healthy, not some insane Bitcoin style HODL pump - that's how you get wrecked.
I mean I'm not a neoclassicalist but I've read all of them. I'm in consensus with them here. There's a bunch of theories on what a healthy market is but what we're currently seeing matches none of them.
It's short term profitable but long term disastrous, especially in a world where new mathematics and techniques could literally collapse the demand overnight.
Imagine if some paper hits arxiv and the 256 GB requirement for some model now becomes 64. Woops!
Some clever trick about how attention heads and context Windows work could potentially slash a bunch of requirements by giant margins and all they're doing is firing the starting gun at that global race with every obscenely priced unit they sell.
But if prices were reasonable, this wouldn't be an apocalypse. It'd be fine. Consumers wouldn't rush to 64GB, they'd say " Cool I can multitask now at 256" or " great I can do horizontal scalability' or something else.
But no they created the market conditions so now what would happen is the consumer will immediately flip the 192GB they don't need on eBay, hoping to snatch a profit before the prices tank and the second hand market will be flooded the rug will be pulled out from the luxury pricing and everyone will get screwed.
This has happened in electronics markets before. Many times.
When Engels talked about the grave diggers of capitalism they were looking at it through a 19th century labor/manufacturing lens but arguably this same dynamic is at play here.
> Imagine if some paper hits arxiv and the 256 GB requirement for some model now becomes 64. Woops!
If you were a DRAM manufacturer, isn't this exactly the kind of thing that would make you think twice about investing years and $billions in new fab construction?
The second Micron boise fab hasn't even broken ground yet, they are still working on the first one. So don't expect these things to be completed in parallel.
Some of my family is pretty happy, though, with the job security as they are pretty convinced these projects are all going to take much longer than what's being stated publicly. Micron is saying the first chip from the new fab will be in 2027... though they also predicted it'd be 2026. The date seems pretty slippy.
If memory prices cool, in about 2 years, Micron will stop new projects, they have done it before.
Especially given CXMT has been able to scale up much faster than what most people expected, only reason their isn't a bigger impact is modern HBM is hard to CXMT even today.
We are likely to see supply double in the next 3 years, but demand even out with optimizations, cooling of data center demand, and most importantly moving some of the dram to flash demand instead which is much easier to produce and scale.
> CXMT started 11 years ago and only now is reaching any real volume. If they decided a year ago to react to the current demand cycle they'd be 6-7 years out.
It's taken them this long to catch up to the DDR5 standard. They've only recently been through qualifications to be a DDR5 supplier for the big boys.
> Every Major Motherboard Maker Now Validates CXMT DDR5
I'm fully open to the idea of having to fly to China to buy hardware. If we're seeing $25,000 US prices versus like $4,000 in China, might as well go visit the great wall and have a vacation.
RAM manufacturers are bidding against NVIDIA and everyone else for the same constrained supply of EUV machines. And it takes years to build more fabs. Micron has multiple fabs coming online in 2027 and 2028.
If we take some time to understand how HBM memory is manufactured (with particular focus on yield risk for final packaging steps), we will hopefully learn that the current capacity crisis is not bullshit.
I guarantee Micron & friends are not intentionally orchestrating their business such that they would suffer a massively reduced chance of yielding on a per-die basis. Unless someone is actually buying HBM devices, they are not going to be making them. These are not a commodity that can be speculatively manufactured in any economically rational way.
>Seriously, if a single politician stepped forward and said "i'll bring down ram prices" they could then shoot a puppy and call me a slur and I'd still go out and doorknock for them.
How many people, outside of tech geeks and megacorps care about RAM prices? And how gullible would you be to BELIEVE the politician they could actually make it happen, and even if they did, that it would extend to the average person, and not JUST megacorps/megadonors?
Phone companies have been differentiating their models based on RAM for a decade. As have laptop and desktop sellers. The reason your router sometimes randomly crashes could very well be a result of not enough memory. The reason it takes such a long time to launch some programs repeatedly is because you don't have enough memory to cache it. Swapped from your browser to an app on your phone, but when you go back to the browser the site has reset and you lost everything you were working on? Not enough memory. Etc.
I think a lot of people care about the downstream effects of memory prices, but I agree with you that they may not realize that they happen because of memory prices.
>The reason your router sometimes randomly crashes could very well be a result of not enough memory. The reason it takes such a long time to launch some programs repeatedly is because you don't have enough memory to cache it. Swapped from your browser to an app on your phone, but when you go back to the browser the site has reset and you lost everything you were working on? Not enough memory. Etc.
That might be true at micro level, but at the macro level more memory just means developers get more lazy with their optimizations, causing apps to get more bloated, eating up any gains in extra memory. There's no reason why slack needs 1+GB to run, yet people are perfectly happy to put up with it.
> How many people, outside of tech geeks and megacorps care about RAM prices?
They don’t care about RAM prices, but they do care about the price of things that have RAM in them (or even NAND), and all of them are increasing way faster than inflation.
> Seriously, if a single politician stepped forward and said "i'll bring down ram prices"
Or abolished VAT (the meaning of VAT is that you pay a "rent" for all the infrastructure used to produce the thing) and import taxes (protect your market) on stuff we don't produce in our markets anyway.
The situation is actually much worse and the long term consequences will start materializing soon. The wholesale theft of humanities soul is in progress. It won't be a pretty sight in supposedly civil first world countries, when the human spirit awakens. Currently we are still pressing that snooze button hard and repeatedly, as I think most are keenly and deeply aware of what needs to happen but that too will cost our souls.
I keep arguing that memory needs more competition, and people keep pointing out that it's too slow and expensive to ramp up. But if the threshold to enter that market is so steep, that means it cannot function as a free market and requires regulation.
In this case I think investment in more production is the only option, and it needs to happen even if it is expensive and slow.
> Seriously, if a single politician stepped forward and said "i'll bring down ram prices" they could then shoot a puppy and call me a slur and I'd still go out and doorknock for them.
He doesn't ruin the cost of memory. Advances in memory size and speed are now in full speed mode. Expect drastic increase in the upcoming years. Big factories are in the making and planned. Gigalab in the US and many others in the east.
Since 2010 we have computers with 16gb as being normal. Finally we are moving into a new era where the standard will be 64gb next year and 128 in 2028. Hopefully we reach 1tb in 2030.
There's no BF16, original full quality weights are quantized already and 510GB.
Then good portion of those weights are n-grams (~200GB) that don't need to be in VRAM.
Then KV cache of that model is super lightweight at ~1GB per 1M tokens. If HBF succeeds, then accelerator with 16GB of VRAM and 1TB HBF/NAND is probably all you need (?).
GLM 5.3 Flash runs fine on two Sparks and Qwen 3.8 Flash Next on one is indeed incredible! I made this 3D game with it in two days using Qwen Code as agent:
Having Flash Next local at 150 t/s with 250k context is a joy. It’s as good as Sonnet 5. It will spaz out but it was less eager compared to DS Flash 4.1. Both are good but I find I prefer Flash Next. This and Qwen 27B are the models people should be freaking out about.
I reimplemented most of the features of the Deepseek v4.1 flash paper (apart from quantization aware training which doesn't make sense because my implementation uses float32 precision anyways)
I'm currently learning how to distill reasoning traces (check my other github repositories) but I think that a locally selfhostable deepseek is possible with my mixture of experts sharding mechanism. I decided to optimize everything for CPU parallelization, with the idea that the KV cache and meta model have to run from CPU RAM anyways, so the experts can also be loaded/unloaded at runtime if needbe, to save more RAM.
Anyways, would love to see someone train this on their own datasets. Currently my pipeline is kinda optimized for parquet and zim files.
The article isn't just about running locally though. The author is saying it's super cheap to run the model through Opencode Go (and presumably OpenRouter etc.) Personally I'm always most excited by models I can actually run locally, but even these huge open source models open up the competitive landscape for companies to let you call models via an API or just lease compute. And they don't have to charge you to offset research, training, huge staffs of the best minds in the world, crazy PR etc. I think that's a big win for customers and buts competitive pressure on the frontier labs as well.
You can run it locally for the price of a decent car, or run it on somebody else's hardware at vast.ai or a similar provider for much less. What's not to like?
No, you won't get frontier-level intelligence on a 1070Ti. Yes, it should be illegal to do what Altman did. Since we clearly don't live in the best of all possible worlds, we need to settle, and DS4.1 Flash is a good place to do that.
For tasks that don't require vision to support, I personally like the NVFP4 quant of GLM 5.3 from Local Inference Lab better, but they are both well beyond awesome.
Allow a question from someone who’s only got a very vague idea of how this kind of stuff works behind the scenes: say I rent usage of this model through one of the many LLM hosting providers out there, and let‘s assume I use it extensively through something like Pi or OpenCode and vibe code away all the time, keeping the hosted model occupied as much as I can, happily burning my credits.
Does that mean that there is a hardware cluster as described by you above that is crunching away just for me?
So at FP16, I alone keep a 1,664 GiB system occupied all the time?
No, a cluster can server multiple users at the same time, providers cap the tok/s so that one cluster can run inference on multiple inputs at the same time. OpenAI with their new ultrafast mode is probably reserving the whole cluster or prioritizing requests of ultrafast users above others with a higher tok/s hence the high price and high speed. There's many other knobs providers tweak that they don't show the users, for example I doubt many providers are hosting the full FP16 version.
The "expensive part" of generating the next token is streaming in the model weights from memory. The computations are relatively simple, which is called a "low arithmetic intensity" in industry jargon.
So what they do is batch multiple chats together and compute the neuron activations for all of them together.
This is vaguely similar to how some database engines work, where if multiple users need to run a "whole table scan" query, the additional users "join" the streaming workload of the first query mid-way, then loop back around to complete the first part that they missed. The AI accelerators don't do this looping, but the concept is the same: amortize the expensive I/O over multiple computations running in parallel.
The "turbo mode" token rate thing is almost certainly your query getting sent to slower or faster hardware, like B200 vs newer B300 kit.
It depends hugely on what "rent usage of this model through one of the many LLM hosting providers" means. If you're asking them to host the model privately then yes, all of that 1.6T of RAM is likely in use holding weights, activations and KV cache by an inference engine that's only getting/answering requests from you alone. When you aren't actively using the model the hosting process is still active and waiting with all of that memory still wired to it.
As background: For the most part VRAM oversubscription/paging/swapping isn't a thing in the same way that RAM for a VM often is. There are some approaches to it, but (to my knowledge) not at that sort of scale.
There are some systemic reasons for this, but very broadly speaking the GPU vendors are building toward the highest bandwidth and lowest latency possible, and the overhead/complexity of something like protected memory modes serves neither of those priorities.
Projects like DwarfStar https://github.com/antirez/ds4 really lower the hardware bar a lot so Deepseek 4.1 flash and other mixture of expert models can run on consumer hardware. There are also other inference providers who make their money serving openweight models. Services like OpenRouter make it all too easy to utilize these models. Access to these models isn't hard. The hardware moat is becoming pretty easy to bridge.
More concretely DwarfStar M5 128GB Deepseek 4.1 flash 1K tokens @ 29s, 5K tokens + reasoning @ 147s, 10k token prompt @ 463 tokens/s = 22s. Hardware buy-in USD$7K / AUD$8.5K / EUR€6.8K. At typical workloads, ROI is still poor vs. current-era subsidies, but owning hardware is good for privacy/longevity/connectivity independence. Whether you actually consider Apple hardware 'owned' is a valid and thought provoking question.
"1070s or 1070 TIs because GPUs have been severely overpriced for too long" ... ."
1070ti launch MSRP was $450 ish.
5070 could be had in the last year for 5xx-6xx range easily.
All things considered - (inflation being about 30%~ (guess)) between these two timelines. You are looking at 300% performance difference at a cost dollar for dollar that is cheaper then when they purchased their cards.
Might be a bit of a stretch blaming it on "severely overpriced for too long..."
Are you counting the n-gram/PLE as part of the model weights there? They can go in host memory. Would be good to show your working. Also the released weights are pre-quantised and presumably QATed, so your "Full Precision" and INT8 are simply not a version of the model that actually exists.
Edit: I went and checked for you. The LM backbone is 307.2 GB (286.1 GiB), straight from DeepSeek's upload. The n-gram table is 203.1 GB (189.1 GiB), which goes in host RAM. Note the embeddings are higher precision than the expert tensors, so it's a larger fraction of the bytes than it is of the parameters.
DeepSeek V4.1 Flash is mixed MXFP4/MXFP8 so all but the INT4 calculation is wrong here and that's still wrong because you can run it on < 400GB VRAM. The n-gram table is MXFP8, but can be offloaded to RAM or disk without too much of a performance hit. Really, you could probably cram it onto < 300GB VRAM if you're willing to apply a small quant to certain parts of the model considering how little VRAM is dedicated to kv cache.
I know my comment is a little nit picky because it's still pretty expensive to run, but it's not quite as bad as this comment makes it out to be. Really, if you're VRAM constrained, take a look at GLM 5.3 Flash or Qwen 3.8 Flash Next before you worry about this model as all three models perform pretty similarly.
In nvfp4, it's about 300 gigs once you offload n-grams, 491 without offloading, you can run it pretty well on 4x DGX Sparks, which last I checked was about $20k. So, it's definitely runnable.
Or you can just use any of the neoclouds' shared hosting. The thing for them to be freaked out is that these models are getting good enough very quickly, and all the shared hosting providers can run them for a tiny fraction of what the frontier model companies charge.
That is opposite to my experience so far, can you describe your coding tasks? Mine are systems level code, utilities, operating system code, networking and real time control stuff.
I also prefer GLM-5.3-flash to DS-4.1-flash, but it's close. Since Z.ai has been offering essentially free GLM-5.3-flash tokens on their coding plan between 8am-6pm pdt I've been using it a lot... though that ends on Oct 10 IIRC.
Opencode Go gives only 6300 requests for glm and 23 000 for deepseek. And, if I wanted to, I would be able to do all my work on $10 plan with deepseek. It’s very cheap.
Unfortunately OpenCode Go is garbage now. Charm Hyper provides deepseek-4.1-flash at $0.33/$1.31/$0.03/M/cache, and glm-5.3-flash at $0.16/$0.54/$0.03. The rate limit is the same for all models, just the cost; not like the absurd multiple-levels-of-price-rate-limit OpenCode Go pricing, nor their terrible performance.
People tend to conflate the question "is AI a useful technology?" with "are the AI companies going to do well?" but they're surprisingly separated in practice, with either one able to be true while the other is false. There is a lot of money tied up in a lot of hardware with a lot of loans made against that hardware as collateral all based on the assumption that AIs are going to need more and more and more and more hardware and whoever has the hardware wins. If a much better model comes out that requires vastly less hardware, or even more accurately, merely charges vastly less than the current AI companies, then to a first approximation (barring Jevon's paradox, and bearing in mind there's no timeline guarantee on that) all that hardware becomes much less valuable for being grotesquely oversupplied relative to what is necessary, and even though that would generally make AI objectively more useful than it was before, it would cause mass financial chaos in the markets.
The markets need a very particular rate of progress. It isn't entirely clear to me that it's even a possible rate of progress, it may be overconstrained, but they certainly don't have plans for the AI models to get commoditized on the timeframes of these vast, vast array of loans being made against hardware as collateral. Spend a metric shit ton of money to kill all your competition then charge monopoly rent on the one thing absolutely everyone needs doesn't work if you can't economically "kill all your competition" because the economics favor them in the spending spree.
And then, based on the fact that this is not even remotely complicated logic, there are plenty of people who are fully aware that they have a lot of money tied up in not running around telling everyone how wonderful the cheap models have become.
It's also unclear whether those who approved those loans understand GPUs depreciation. In any case, progress in software but also hardware could bring chaos and ruin their house of cards.
> It's also unclear whether those who approved those loans understand GPUs depreciation.
This also assumes heavy utilization, though. If there's heavy utilization, it might mean they're doing well. If they're all spinning, it's time to raise prices.
Agree that people leaving the big two companies is going to be hard to keep a pulse on prior to IPO.
Anthropic and OpenAi are in the news, so they get the press and people go and try out their product. Large enterprise businesses are going to make larger, longer-term contracts with them and are only going to pivot if they think switching costs are easy or if they think the provider won't deliver.
The other inference producers are less well known or you need to get your cloud sales rep to tell you how to switch to them as a provider rather than Anthropic or OpenAI.
I use OpenRouter, I know switching is easy, but larger businesses tend to work in yearly cycles. DeepSeek v4 Flash came out in late April.
I agree OpenAI and Anthropic are going to struggle when the median price of running a smart-enough model keeps falling.
While using DeepSeek v4.1 Flash I was architecting a system and I made a mistake of drawing the RPC boundaries at a wrong place that did cost me in so many ways.
I realized that mistake and guided DeepSeek where it should be.
Next I fired Fabble 5.5 set to high to check if the hype is real about Fabble. It exhausted 89% of quota and came up with NOTHING that DeepSeek hadn't flagged itself already in its notes.
Do you mean Fable 5.1? Or Opus 5.5? I'm not sure what you're working on but for me DS 4.1 flash isn't nearly at their level. For the price it's obvious very impressive, though Luna 6.0 is excellent too.
The problem with benchmarks and proprietary models is that one day a model is best at doing X, another day that's not so sure. And anyway, we are throwing the same X.
I've found supposedly smaller and, less performant models do better on certain tasks. I end up using several models, sticking to what my unconscious statistical observations tell me to use for the kind of task at end.
The question seems rhetorical but I think there are two reasons in some combination. First is it there is some awareness lag here. That lag can be on the producer and consumer side. Software enterprises are pretty slow to adopt new things and slow to try new things so they might only be aware of openai and Claude as options. Plus there are some scariness because deep seek is a Chinese model and therefore export restricted - never mind that there are American in European providers.
The other reason is more interesting. Maybe the frontier providers think that price performance is irrelevant in light of very powerful frontier models that can start the RSI loop and or a huge displacement of work and a winner take all economic situation. After all if frontier providers earn everyone's money then you won't have any money to spend on any model 100x cheaper or not.
I think it also has a bit to do with the AI sector of tech still moving at lightning speed.
Theres already models that outdo DS 4.1 flash in cost/performance. Luna 6 on max effort for example. Luna also doesn't care what time of the day it is for cost calculation.
And I'm sure by the time people ask why Luna 6 is being slept on there will be another cost/performance king
Luna is very slow and bad at agentic tasks. DS runs circles around it and there are US providers providing cheaper rates no matter the time of the day.
I've been waiting for non-preview release of v4.1 flash. I'm assuming this one is a preview like the v4.0 ones without dates in their name. Maybe its not a preview, but after how v4.0 was handled I'm assuming it is.
I guess what we're seeing is selection bias - people clicking on this HN story will be those who are interested in DeepSeek. And those people who are invested in DeepSeek may not like the facts that you presented.
It's annoying that social networks work this way. The upvote should be for high-quality content and the downvote should be for low-quality content. But .. well.. human nature and tribal dynamics always seem to win.
HN just hates the big labs, so any story about some open weight model "eating their lunch" has gotten huge traction for the last year, while the big labs have grown their revenue 10x. HN is like the Jim Cramer of market adoption of tech.
I had the same experience using ds 4.1 last couple of weeks. It’s insanely good for the price. I’m doing mostly web dev it excels at everything I throw at it. The pricing is ridiculous. I canceled my gpt subscription and haven’t looked back hope the pricing stays like that. I almost never need a better model. I still keep my Claude 20$ sub for now but I feel like one more iteration and I won’t need even that anymore I hardly use it
I think the interesting provider to cross-check this assumption with here is Meta, who is clearly freaking out, and is currently providing Muse 1.3 even cheaper so long as you are willing to share data with them
It's still lacking llama.cpp support, and the work on that isn't moving very fast either. Looks more like a general community issue, where this model isn't drawing much interest.
Because the version officially released by DS is an extremely quantized version, and there are many new things in the architecture, Unsloth needs time to handle this, just as was the case with the previous DS V4 Flash.
Oh man. v4.1-flash has been an sbolute game changer for us. We run all our Personal Assistants now on flash (thinking high) by default and it works incredibly well. There is really no need for basic agentic tasks that might require Kimi K.3 or GLM-5.3 levels.
Once its gets juicier, we let flash launch specialized subagents with specific models. GLM-5.3 for coding or Kimi K.3 for research and critique.
But as a main driver. I love flash. And it brought our bill down by A LOT :D
> Today's models are now good enough for high-quality unattended tasks. Chasing the latest and greatest is silly. It is fun to see the new Fable capabilities, but the tasks we throw at them are usually ridiculous (maybe even insulting) if you believe in LLM sentience. It's like asking a math PhD to organize the files on your desktop.
I'm using DS V4.1 Flash as my main model since their release and it works great for all my coding tasks. My setup is OpenCode Go subscription and obra/superpowers skill.
The only times I try to change models are on general planning tasks (like research this codebase for tech debt mitigation opportunities) or if I need deep research which would benefit from searching the web, in which I still think Gemini is still the best because of the speed and access to google search index. But these are not even 20% of my daily tasks.
I have had good luck rotating between the three tiers of GPT 5.6 with occasional jumps up to Astra or Fable. Most of my work is with GPT-5.6-Sol. Simple tasks like data extraction or trivial refactors (rename this variable etc) I often push down to Terra or Luna, or even to self-hosted Gemma4:31B. Very tricky stuff, like planning a new feature, design review, or code review of a complex change across multiple repositories is where I leverage Astra.
I've had middling success with models like DS V4.1 Flash and free Gemini. They tend to be pretty good at very easy stuff ... but they're more likely to go down rabbit holes, confidently assert falsehoods, or fix bugs with changes to my test harness rather than my code.
I asked about comparison to the well-known SotA models specifically because I use either Astra or Sol for ~85% of my daily tasks. When I try to use smaller/cheaper models, I have had very mixed success. Sometimes it's perfect, while other times it fails in subtle and hard to catch ways.
Just use a provider hosting it in your country especially if your country has major data centers then its the same as using Anthropic or GPT of GCP Model Garden or AWS Bedrock
nobody here is talking about running frontier level intelligence locally so if you’re Chinaphobic and prefer layers of corporations siphoning your data in between you and the party there are plenty of options instead of directly to the party
Not shilling for them but Ollama cloud hosts domestically with ZDR afaik. I run 95% of my open weight inference through them. The rest goes through Opencode Go $10 plan (which is enough to run 3 hermes agents on DSF 4.1 and leave plenty of left to experiment with when new models drop).
should you not be worried more about sending data to providers of your own country? I think they are equally bad personally, but if you think one of them is acceptable, why do you prefer it to be your own government that has more ties to your life?
What is the cost of access like for DeepSeek-v4.1-flash, compared to GLM-5.3-flash via ZAI's Coding Plan? Because that's what I use; and often hit the "wait". I wouldn't mind trying a new model subscription or even API access which hits around glm-5.3-flash level weight class (which seem to be enough for me; with quite some human suprvision and nudging) but gives muuuuuuch moooore tokens for the same price.
Technically yes, but has been reported to be quite benchmaxxed. In practice Deepseek Flash 4.1 and GLM 5.3 might therefore still outperform Mimo 2.6 pro.
I’ve been using it a lot and it’s performing really well. Not GLM 5.3 levels but it beats Deepseek for my use. I’ve used it on long running coding tasks, though mostly prototyping, but it’s done a great job at very low cost.
if you have a legitimate coding application, it isn't very good. if you have some kind of inauthentic activity, which could be what it is trained for for all sorts of reasons...
What is your evidence? Deepseek v4.1 Flash is by far the most popular coding model on openrouter, having processed 38.7T tokens in just the last 7 days, over 3x the usage of the 2nd rank model.
So I ask again, what are you basing your assertion on?
my own usage of deekseep v4.1 flash, and that among the dozens of great programmers i know, not a single person is using it
BUT. they are employed to do / deciding-to-do authentic (if often meaningless) stuff.
here's a short list of inauthentic activity that claude and openai refuse to do:
- chat services that, when you ask them, say they are not chatbots when they are
- code to work around software licenses or DRM
- code to scrape or download copyrighted material
- directly cheating on homework
- adopting a persona in social media that spreads misinformation or propaganda
this is but a short list. but ask me, "are there enough inauthentic activity demands such that someone who CANNOT USE claude or gpt as the LLM would use dsv4.1 on openrouter instead?" yes. i mean there are whole countries right now where the culture can be summarized as, "bottom to top, inauthentic activity." i am surprised it is not more usage!
So your argument is that the 38T of tokens used in the last 7 days is by moron programmers or people doing "inauthentic" tasks? You think the person who made this post is also an idiot?
Do you realize how incredibly delusional/self-centered you sound?
DeepSeek 4.1 Flash 0910 is perfect for M5 Ultra 256GiB. Running it fully resident in RAM, prefill at ~2500 tok/s and decode at ~40 tok/s. Probably tons of room to improve from there.
792 comments
[ 0.30 ms ] story [ 93.8 ms ] threadVRAM & Memory Requirements by Precision
• FP16 (Full Precision): Requires ~1,664 GB of VRAM (e.g., an 8x B300 288GB cluster).
• INT8 Quantization: Requires ~832 GB of VRAM (e.g., 8x H200 141GB).
• INT4 Quantization: Requires ~416 GB of VRAM (e.g., 8x A100 80GB)
VRAM aint cheap, Sam Altman ruined the cost of memory, Nvidia doesnt make enough consumer GPUs letting the market go insane over them, I still have friends on 1070s or 1070 TIs because GPUs have been severely overpriced for too long. I remember when a gaming PC was only $1000.
Even so why would anyone not sleep on a model they cannot run?
Is it really so hard to believe that RAM prices are up because demand is simply exceeding supply, especially in a market where additional supply takes years and billions of dollars to come online? There's no need to posit cartel behavior and a fair amount of evidence that there is none.
I got a 4090 in 2023 for 1600, a 5090 in 2025 for 2000 with 256 DDR5 for about $1,000 ... and then, after some protectionist legislation passed, these prices quickly shot to the moon.
Don't be dumb.
You can't say "connect the dots" at the end of a raving, mostly-incorrect post and act like you've made an ironclad argument.
wonder what voting would be like?
gamer vote ++
datacenter hater vote --
datacenter lobby ++
micron lobby --
Micron has 3 brand new fabs currently under construction, 2 Boise, 1 in New York as the first of 4 planned for a campus.
Plus expanding other existing facilities.
These things take ~3-5 years from breaking ground to full production. You'd have had to anticipate the current demand years before it happened in order to be bringing production on-line before 2030 or so.
Samsung and HK Hynix also have fabs under construction and planned.
CXMT started 11 years ago and only now is reaching any real volume. If they decided a year ago to react to the current demand cycle they'd be 6-7 years out.
Not much you can really do to wish for more fabrication to exist on any timeline not measured in fractional decades.
Could they do more and react quicker? Probably, but everything I've read on the subject seems to point to 3 years is absolute bare minimum if you happen to have a shovel ready project with the land bought, local permitting completed, infrastructure extended to the site, and a skilled workforce already in place. They could suspend buy-backs/dividends today and dump it all into building production and there would be no material impact until around 2030.
> The Micron CEO just recently said this is the exact plan
CEO simply stated the demand pressure will not go away through 2027, and supply will not increase until around 2028 when currently under construction fabs start shipping volume. The article does not support your statement.
Costs did go nuts, but there are signs of easing in the market of late. CXMT is starting to have an impact and priced will probably fall in 2027.
You need to keep the market healthy, not some insane Bitcoin style HODL pump - that's how you get wrecked.
I mean I'm not a neoclassicalist but I've read all of them. I'm in consensus with them here. There's a bunch of theories on what a healthy market is but what we're currently seeing matches none of them.
It's short term profitable but long term disastrous, especially in a world where new mathematics and techniques could literally collapse the demand overnight.
Imagine if some paper hits arxiv and the 256 GB requirement for some model now becomes 64. Woops!
Some clever trick about how attention heads and context Windows work could potentially slash a bunch of requirements by giant margins and all they're doing is firing the starting gun at that global race with every obscenely priced unit they sell.
But if prices were reasonable, this wouldn't be an apocalypse. It'd be fine. Consumers wouldn't rush to 64GB, they'd say " Cool I can multitask now at 256" or " great I can do horizontal scalability' or something else.
But no they created the market conditions so now what would happen is the consumer will immediately flip the 192GB they don't need on eBay, hoping to snatch a profit before the prices tank and the second hand market will be flooded the rug will be pulled out from the luxury pricing and everyone will get screwed.
This has happened in electronics markets before. Many times.
When Engels talked about the grave diggers of capitalism they were looking at it through a 19th century labor/manufacturing lens but arguably this same dynamic is at play here.
If you were a DRAM manufacturer, isn't this exactly the kind of thing that would make you think twice about investing years and $billions in new fab construction?
Some of my family is pretty happy, though, with the job security as they are pretty convinced these projects are all going to take much longer than what's being stated publicly. Micron is saying the first chip from the new fab will be in 2027... though they also predicted it'd be 2026. The date seems pretty slippy.
Especially given CXMT has been able to scale up much faster than what most people expected, only reason their isn't a bigger impact is modern HBM is hard to CXMT even today.
We are likely to see supply double in the next 3 years, but demand even out with optimizations, cooling of data center demand, and most importantly moving some of the dram to flash demand instead which is much easier to produce and scale.
It's taken them this long to catch up to the DDR5 standard. They've only recently been through qualifications to be a DDR5 supplier for the big boys.
> Every Major Motherboard Maker Now Validates CXMT DDR5
https://www.techtimes.com/articles/321572/20260725/every-maj...
After their recent IPO, they have everything they need to ramp up in a major way.
It's just a matter of time.
I know it's just a figure of speech, but damn. I laughed out aloud in public just reading this.
I guarantee Micron & friends are not intentionally orchestrating their business such that they would suffer a massively reduced chance of yielding on a per-die basis. Unless someone is actually buying HBM devices, they are not going to be making them. These are not a commodity that can be speculatively manufactured in any economically rational way.
How many people, outside of tech geeks and megacorps care about RAM prices? And how gullible would you be to BELIEVE the politician they could actually make it happen, and even if they did, that it would extend to the average person, and not JUST megacorps/megadonors?
I think a lot of people care about the downstream effects of memory prices, but I agree with you that they may not realize that they happen because of memory prices.
That might be true at micro level, but at the macro level more memory just means developers get more lazy with their optimizations, causing apps to get more bloated, eating up any gains in extra memory. There's no reason why slack needs 1+GB to run, yet people are perfectly happy to put up with it.
They don’t care about RAM prices, but they do care about the price of things that have RAM in them (or even NAND), and all of them are increasing way faster than inflation.
Or abolished VAT (the meaning of VAT is that you pay a "rent" for all the infrastructure used to produce the thing) and import taxes (protect your market) on stuff we don't produce in our markets anyway.
In this case I think investment in more production is the only option, and it needs to happen even if it is expensive and slow.
Priorities
How? Increase production? The time needed to scale up the production is longer than one election cycle.
I spit out my coffee laughing when I read this
1660 ti, 4790k, 16gb ddr3
Then good portion of those weights are n-grams (~200GB) that don't need to be in VRAM.
Then KV cache of that model is super lightweight at ~1GB per 1M tokens. If HBF succeeds, then accelerator with 16GB of VRAM and 1TB HBF/NAND is probably all you need (?).
It has a set of n-gram tables which you can stream from system RAM or even NVMe
That said it’s still quite big! I can’t fit it on my DGX Spark, though I believe you can if you have two?
https://github.com/christopherowen/spark-ds41f
I’m quite spoiled with how good Qwen 3.8 Flash Next is on a single spark though: shocking how good local models are getting on attainable-ish hardware
https://blog.jonathanpage.com/
GLM 5.3 Flash runs fine on two Sparks and Qwen 3.8 Flash Next on one is indeed incredible! I made this 3D game with it in two days using Qwen Code as agent:
https://games.jonathanpage.com/
https://www.storagereview.com/review/dgx-station-gb300-clust...
I reimplemented most of the features of the Deepseek v4.1 flash paper (apart from quantization aware training which doesn't make sense because my implementation uses float32 precision anyways)
I'm currently learning how to distill reasoning traces (check my other github repositories) but I think that a locally selfhostable deepseek is possible with my mixture of experts sharding mechanism. I decided to optimize everything for CPU parallelization, with the idea that the KV cache and meta model have to run from CPU RAM anyways, so the experts can also be loaded/unloaded at runtime if needbe, to save more RAM.
Anyways, would love to see someone train this on their own datasets. Currently my pipeline is kinda optimized for parquet and zim files.
[1] https://github.com/cookiengineer/gonano
Because it's an open model so providers compete on price.
No, you won't get frontier-level intelligence on a 1070Ti. Yes, it should be illegal to do what Altman did. Since we clearly don't live in the best of all possible worlds, we need to settle, and DS4.1 Flash is a good place to do that.
For tasks that don't require vision to support, I personally like the NVFP4 quant of GLM 5.3 from Local Inference Lab better, but they are both well beyond awesome.
Allow a question from someone who’s only got a very vague idea of how this kind of stuff works behind the scenes: say I rent usage of this model through one of the many LLM hosting providers out there, and let‘s assume I use it extensively through something like Pi or OpenCode and vibe code away all the time, keeping the hosted model occupied as much as I can, happily burning my credits.
Does that mean that there is a hardware cluster as described by you above that is crunching away just for me?
So at FP16, I alone keep a 1,664 GiB system occupied all the time?
The "expensive part" of generating the next token is streaming in the model weights from memory. The computations are relatively simple, which is called a "low arithmetic intensity" in industry jargon.
So what they do is batch multiple chats together and compute the neuron activations for all of them together.
This is vaguely similar to how some database engines work, where if multiple users need to run a "whole table scan" query, the additional users "join" the streaming workload of the first query mid-way, then loop back around to complete the first part that they missed. The AI accelerators don't do this looping, but the concept is the same: amortize the expensive I/O over multiple computations running in parallel.
The "turbo mode" token rate thing is almost certainly your query getting sent to slower or faster hardware, like B200 vs newer B300 kit.
As background: For the most part VRAM oversubscription/paging/swapping isn't a thing in the same way that RAM for a VM often is. There are some approaches to it, but (to my knowledge) not at that sort of scale.
There are some systemic reasons for this, but very broadly speaking the GPU vendors are building toward the highest bandwidth and lowest latency possible, and the overhead/complexity of something like protected memory modes serves neither of those priorities.
He gave demand signal so many times years ago and was mocked for it and now we have the consequences of industry not taking him seriously.
https://news.ycombinator.com/item?id=15781499
1070ti launch MSRP was $450 ish. 5070 could be had in the last year for 5xx-6xx range easily.
All things considered - (inflation being about 30%~ (guess)) between these two timelines. You are looking at 300% performance difference at a cost dollar for dollar that is cheaper then when they purchased their cards.
Might be a bit of a stretch blaming it on "severely overpriced for too long..."
Edit: I went and checked for you. The LM backbone is 307.2 GB (286.1 GiB), straight from DeepSeek's upload. The n-gram table is 203.1 GB (189.1 GiB), which goes in host RAM. Note the embeddings are higher precision than the expert tensors, so it's a larger fraction of the bytes than it is of the parameters.
So,
> Call me crazy but:
You're crazy.
I know my comment is a little nit picky because it's still pretty expensive to run, but it's not quite as bad as this comment makes it out to be. Really, if you're VRAM constrained, take a look at GLM 5.3 Flash or Qwen 3.8 Flash Next before you worry about this model as all three models perform pretty similarly.
Or you can just use any of the neoclouds' shared hosting. The thing for them to be freaked out is that these models are getting good enough very quickly, and all the shared hosting providers can run them for a tiny fraction of what the frontier model companies charge.
it rips with just 64 ram and a 9070xt
Is OpenAI coming in $20B under a sign of "freaking out"?
People tend to conflate the question "is AI a useful technology?" with "are the AI companies going to do well?" but they're surprisingly separated in practice, with either one able to be true while the other is false. There is a lot of money tied up in a lot of hardware with a lot of loans made against that hardware as collateral all based on the assumption that AIs are going to need more and more and more and more hardware and whoever has the hardware wins. If a much better model comes out that requires vastly less hardware, or even more accurately, merely charges vastly less than the current AI companies, then to a first approximation (barring Jevon's paradox, and bearing in mind there's no timeline guarantee on that) all that hardware becomes much less valuable for being grotesquely oversupplied relative to what is necessary, and even though that would generally make AI objectively more useful than it was before, it would cause mass financial chaos in the markets.
The markets need a very particular rate of progress. It isn't entirely clear to me that it's even a possible rate of progress, it may be overconstrained, but they certainly don't have plans for the AI models to get commoditized on the timeframes of these vast, vast array of loans being made against hardware as collateral. Spend a metric shit ton of money to kill all your competition then charge monopoly rent on the one thing absolutely everyone needs doesn't work if you can't economically "kill all your competition" because the economics favor them in the spending spree.
And then, based on the fact that this is not even remotely complicated logic, there are plenty of people who are fully aware that they have a lot of money tied up in not running around telling everyone how wonderful the cheap models have become.
This also assumes heavy utilization, though. If there's heavy utilization, it might mean they're doing well. If they're all spinning, it's time to raise prices.
Anthropic and OpenAi are in the news, so they get the press and people go and try out their product. Large enterprise businesses are going to make larger, longer-term contracts with them and are only going to pivot if they think switching costs are easy or if they think the provider won't deliver.
The other inference producers are less well known or you need to get your cloud sales rep to tell you how to switch to them as a provider rather than Anthropic or OpenAI.
I use OpenRouter, I know switching is easy, but larger businesses tend to work in yearly cycles. DeepSeek v4 Flash came out in late April.
I agree OpenAI and Anthropic are going to struggle when the median price of running a smart-enough model keeps falling.
I realized that mistake and guided DeepSeek where it should be.
Next I fired Fabble 5.5 set to high to check if the hype is real about Fabble. It exhausted 89% of quota and came up with NOTHING that DeepSeek hadn't flagged itself already in its notes.
I've found supposedly smaller and, less performant models do better on certain tasks. I end up using several models, sticking to what my unconscious statistical observations tell me to use for the kind of task at end.
The other reason is more interesting. Maybe the frontier providers think that price performance is irrelevant in light of very powerful frontier models that can start the RSI loop and or a huge displacement of work and a winner take all economic situation. After all if frontier providers earn everyone's money then you won't have any money to spend on any model 100x cheaper or not.
Theres already models that outdo DS 4.1 flash in cost/performance. Luna 6 on max effort for example. Luna also doesn't care what time of the day it is for cost calculation.
And I'm sure by the time people ask why Luna 6 is being slept on there will be another cost/performance king
Opus 5.5: TIME 9.3m COST / $1.99 / SCORE 99/100 https://jonclegg.github.io/pacman-bakeoff/#claude-opus-5-5
Deepseek 4.1 Flash: TIME 2.8m / COST $1.89 / SCORE 72/100 https://jonclegg.github.io/pacman-bakeoff/dev/#deepseek-v4.1...
It's annoying that social networks work this way. The upvote should be for high-quality content and the downvote should be for low-quality content. But .. well.. human nature and tribal dynamics always seem to win.
And as long as I pay as little for claude opus 5.5 i do right now, i'm using it.
But yes i'm glad that we have alternatives.
I don't think so.
Would be cool if they added it.
There are some quirks if your harness use unsupported features of course.
Once its gets juicier, we let flash launch specialized subagents with specific models. GLM-5.3 for coding or Kimi K.3 for research and critique.
But as a main driver. I love flash. And it brought our bill down by A LOT :D
> Today's models are now good enough for high-quality unattended tasks. Chasing the latest and greatest is silly. It is fun to see the new Fable capabilities, but the tasks we throw at them are usually ridiculous (maybe even insulting) if you believe in LLM sentience. It's like asking a math PhD to organize the files on your desktop.
I'm using DS V4.1 Flash as my main model since their release and it works great for all my coding tasks. My setup is OpenCode Go subscription and obra/superpowers skill.
The only times I try to change models are on general planning tasks (like research this codebase for tech debt mitigation opportunities) or if I need deep research which would benefit from searching the web, in which I still think Gemini is still the best because of the speed and access to google search index. But these are not even 20% of my daily tasks.
I've had middling success with models like DS V4.1 Flash and free Gemini. They tend to be pretty good at very easy stuff ... but they're more likely to go down rabbit holes, confidently assert falsehoods, or fix bugs with changes to my test harness rather than my code.
I asked about comparison to the well-known SotA models specifically because I use either Astra or Sol for ~85% of my daily tasks. When I try to use smaller/cheaper models, I have had very mixed success. Sometimes it's perfect, while other times it fails in subtle and hard to catch ways.
are you worried about sending all your data to third parties, especially if they're in different countries?
nobody here is talking about running frontier level intelligence locally so if you’re Chinaphobic and prefer layers of corporations siphoning your data in between you and the party there are plenty of options instead of directly to the party
I cant ofc be fully sure because i don’t own the chain end 2 end.
(And what are the preferred providers?)
Can't you just say "shrank to 1/437th the size"? It's not that hard.
see https://artificialanalysis.ai/models/releases/comparisons?co...
It’s disgustingly good value. I find it capable of doing anything I want.
Obviously can’t use it at work, but for home projects it’s awesome.
I do wonder how long it'll be before a us-hosted offering is available via bedrock, copilot, etc.
if you have a legitimate coding application, it isn't very good. if you have some kind of inauthentic activity, which could be what it is trained for for all sorts of reasons...
So I ask again, what are you basing your assertion on?
BUT. they are employed to do / deciding-to-do authentic (if often meaningless) stuff.
here's a short list of inauthentic activity that claude and openai refuse to do:
- chat services that, when you ask them, say they are not chatbots when they are
- code to work around software licenses or DRM
- code to scrape or download copyrighted material
- directly cheating on homework
- adopting a persona in social media that spreads misinformation or propaganda
this is but a short list. but ask me, "are there enough inauthentic activity demands such that someone who CANNOT USE claude or gpt as the LLM would use dsv4.1 on openrouter instead?" yes. i mean there are whole countries right now where the culture can be summarized as, "bottom to top, inauthentic activity." i am surprised it is not more usage!
Do you realize how incredibly delusional/self-centered you sound?