208 comments

[ 6.0 ms ] story [ 84.4 ms ] thread
I don't have a WSJ subscription, but was there actually anything weird about the deals? I mean it seems pretty normal.
It can be normal, legal and still morally bankrupt. Fuck these parasites
Forcing your customers to buy additional hardware they don't need to inflate the prices of your own product is a crime in the united states of america.

It get worse when we find out they did it to increase nvidia production for the purpose of being able to redirect more output to china.

The whole entire AI bubble was actually jensen's smuggling operation to china. These large data centers have no purpose and cost more money in power grid upgrades than any company will ever make operating them.

The only winners look to be jensen's bank account, the bank accounts of the CEOs that signed the deals to buy unneeded hardware, and the chinese buisnesses that were able to buy more nvidia graphics cards.

It is not a crime for these CEOs to drive their companies into the ground to make themselves richer. Most CEOs do this today. It is a crime if we find out all of this was really a chinese smuggling operation and that would make it the largest crime per monetary value in US history.

Everyone who made money on this smuggling ring has criminal liability. Just like with madoff, the people who withdrew gains had to pay it back because the gains weren't real. Not paying it back would make you an accomplice.

Forcing your customers to buy additional hardware

Ok that sounds interesting and it's not in this article. Where can I learn about that?

Ed Zitron has been saying it. Some people love him and some people hate him.

Satya Nadella said last year they have GPUs sitting that they can't use because the data centers don't exist.

https://redmondmag.com/blogs/generationai/2025/12/microsoft-...

That sounds like horrible planning. Build the data center and when it is 90% complete then order the GPUs. The theory is that Nvidia says buy now or get nothing later.

What the Wall St Journal is saying is that Nvidia basically has an oversupply of GPUs that they are committed to buying from TSMC. The hyperscale companies are designing their own AI chips so they don't want as many Nvidia chips. So Nvidia is giving startups money to buy Nvidia chips in exchange for equity in the startup.

> The theory is that Nvidia says buy now or get nothing later.

The RAM makers were doing similar, too.

"Nvidia was in talks to provide billions of dollars of guarantees to companies that couldn’t afford its chips. That included deals led by Huang’s son-in-law, Nico Caprez, who had been offering upstart cloud providers and data-center developers financing in the hopes of turning them into long-term customers."

...

"Amid the rollout of the program, Kress raised questions including how to decide which deals to pursue, how to value the guarantees, and how to calculate the return on investment, a person familiar with the matter said. Behind the scenes, she tried to rein in Huang’s son-in-law, who had been promoted to a vice president in January, people who worked with her said."

----------

-- https://archive.is/gx14x

It sounds like there may have been a little bit of Nepotism. This article also suggests that Huang himself was uncomfortable with the deals Nvidia and the data centres it had invested in were making, although he appeared to shift blame onto those running the data-centres.

-------------

"In Berlin during a conference, Huang, accompanied by his top sales executives, broadcast a new message to a half-dozen CEOs of companies that are major buyers of Nvidia chips who were seated around a circular table. He opened with a story that sounded like a fable, according to people who attended.

Huang had once found a soup vendor in Japan, he said, with a long line of customers. He joined the line, and after a long wait, enjoyed one of the best bowls of soup he’d ever had. When he asked for another serving, the soup-seller told him no—he had too many customers waiting and not enough soup for all of them.

“He’s a better businessperson than all of you,” Huang told the cloud computing CEOs. The soup-man doesn’t sell out all his inventory to one or two customers, but instead keeps prices high and his customer base diverse, the CEO said.

“You are not establishing the value of my products,” he said."

>Huang had once found a soup vendor in Japan, he said, with a long line of customers. He joined the line, and after a long wait, enjoyed one of the best bowls of soup he’d ever had. When he asked for another serving, the soup-seller told him no—he had too many customers waiting and not enough soup for all of them.

OMG this is delusional left brain thinking[0] to conceive that the man was spreading his supply out for profit motives. Not everything is about capitalism. Some people make decisions for humanitarian reasons.

Our way of life is being destroyed by this limited perception that everything is about self-interest.

[0]: https://www.cbc.ca/radio/ideas/neuroscientist-argues-the-lef...

Maybe the secret ingredient in the soup was snake venom with one bowl containing just a bit less than a dose that would cause medical issues. Give somebody two bowls and the jig is up!

But seriously, somebody with normal, human priorities is not likely to become a billionaire CEO. It's why we shouldn't elect billionaires or let them buy power. Their success at what they do is a demonstration of their unsuitability for greater trust.

As if we didn’t suspect this. Same with RAM makers not upping production.

It’s like the oil companies whining and fleecing us too.

What’s the move though?

Remember when Microsoft got in years of antitrust lawsuits over bundling a browser with their OS? Bring back that level of legal scrutiny to antitrust, and sever the absurdly close ties the US government has with the top 100 companies in the country.
Sure, but not gonna happen with the current admin. We'll see after 2028.
Ram makers are more understandable since they went through this already in the past. The Koreans learned it the hard way.
Strip them of their IP, it's now public domain and any upstart competitor can use it.
> Same with RAM makers not upping production.

Why is this false claim so persistent?

Companies are expanding productions everywhere. https://techcrunch.com/2026/06/29/south-korean-tech-giants-c...

Building fabs and getting them running takes years.

Do some people think that RAM production is a big knob that gets turned up at the factory and instantly we have more RAM chips?

Ram makers not upping production makes sense. Same as supermarket in the Netherlands refusing to overstock toilet paper on shelves during the pandemic. After a few days of it being back on shelves at normal rate people calmed down. This is the extreme version. They see the buildout of dcs slowing down and a memory fab is not a medium term investment.
The invisible hand of the free market ;)
I believe we should start working towards a "fair" market. Free, is almost impossible to attain IMO.
People are acting like vendors extending financing to their customers is somehow irregular or illegal.

It’s not. It happens ALL THE TIME. This isn’t “circular financing” which is a great meme term but is not factual.

That's what these allegations of circular financing are about? Extending credit? Lmao
No, it's that it goes in a larger circle, involving many companies and gets counted as revenue multiple times
Revenue is counted per company so it makes sense
Vendor financing is not illegal but in this case it may have created significant non-recurring revenue. (And maybe amplified scarcity and artificially high chip prices in the industry at large.)

And the part that sounds shady to people is that part of Nvidia’s recorded cash flow is its own cash coming back around.

Nvidia should be broken up.
Especially with the buyout of SchedMD, who support Slurm for these large HPC clusters doing the AI training.
Into what exactly? They have a single product that makes them money.
Nvidia East, Nvidia West, Nvidia North, Nvidia South
The Baby Nvidias.
> Into what exactly? They have a single product that makes them money.

Mellanox ([1]) acquisition by NVIDIA was a monopolistic move and they should be forced to spin out Mellanox as an independent public company.

The same applies to Hugging Face. That deal should have never been approved.

1. https://en.wikipedia.org/wiki/Mellanox_Technologies

I agree on principle but there are many alternatives for networking hardware on the market, them owning Mellanox is nowhere as bad.
Mellanox, GeForce and whatever you want to call the server part.
No, competition needs to get their heads together.
(comment deleted)
I don't think they need to be broken up. Nvidia needs to be investigated.

There are stories of huge amounts of sold chips sitting in warehouses unused. This could be channel stuffing which is illegal

There are stories of Nvidia threatening that customers have to buy these chips now or they will be cut off and not able to buy the next gen. That may be illegal

Since when is channel stuffing illegal? It can happen for any number of reasons. There's often a difference between sell-in figures and sell-through figures.
I say force them to FOSS their microcodes, firmwares, and drivers. Same thing for Intel/AMD. Same for all UEFI code. ARM doesn't get to escape either. No more 'appliance' Universal Machines; if you sell a Universal Machine as or in your product then you must allow the owner or primary leasee full access to it and its capabilities. This includes segmentation switches, which must not be permanent fuses. If a chip is segmented due to failure of manufacture, then the owner/leasee unsegmenting it should be at owner/leasee's risk.
I love to see when capitalism works great and free market is efficient.
ironically, it would seem a free (as in free of rents) market seems to be harder to accomplish in a free (as in rules/regulations) market...
(comment deleted)
(comment deleted)
Jensen Huang made mistakes that gave opportunities to his enemies. But the bigger mistake was his incredible greed, he should have been a happy supplier of an extremely valuable product, and accept competition and that his quasi-monopoly was only transient. But he wanted the entire cake for himself.

Greed and hubris. I think the turbulences will be painful to some, especially those who counted on Nvidia stock to finance their retirement.

But, big picture, this is good news.

Not a monopoly. 3dfx is going to drop the voodoo 6 soon
PowerVR will be back on top just in time for Dreamcast 2 too.
I want a Number Nine Imagine 128
Wow this was a nostalgia bomb. I owned each of these 3 at one point or another. I hadn't thought of Number Nine in a long long time.
Yes, for use with the SGI 1600sw screen
But they don't have a monopoly. Their marketshare is sinking, down to perhaps 45% of estimated TSMC CoWoS supply next year (down from close to 70% last year).
75% gross margin. For a market this big. This not healthy free-market competition working as expected.
(comment deleted)
Nobody made AMD Radeon shit the bed, see: geohot’s experience with getting model training and inference working on AMD

Nvidia is right to cash in on the benefits of its long term strategy, including making CUDA available on grad students’ lowly gaming GPUs.

US capital markets are the envy of the world because winners get to take all (from the world, not just the US), instead of getting smited into oblivion for being head and shoulders better than the competition

I don’t get why he just didn’t use ‘discreet services’ which I’m absolutely certain are available to rich people like him. Why would you even bother with an employee? It just seems like so much unnecessary hassle.
> especially those who counted on Nvidia stock to finance their retirement

well it's no secret that this sort of stock was not that good, I don't think that many people counted on nvidia for their retirement

At least some gamers who bought in a decade ago can sell to finance a 5090.
I’m not being sarcastic, but isn’t that the exact mandate of a CEO?
The job of the CEO is to manage the company as well as they can, keeping business good today and ensuring the company has a tomorrow. It is not hard to intuit that this involves both short and long term thinking.

The whole "it's to maximise shareholder value" refrain is not actually a thing, AFAIK, despite it being something people like to keep repeating.

It absolutely is a thing. It’s the fundamental thing. What you’re describing is just the first in a tall pyramid of answers to the question “how do we maximize shareholder value?”
Whatever kind of thing it is is not a legal obligation.
It's not law that corporations have to maximize shareholder value. Boards have plenty of room to balance the interests of employees, customers, communities, and long-term company health.
It is not the thing in Japan. Japanese corporations are primarily sustained to continue existing, not to return shareholder value. They are very good at that.

Returning shareholder value wasn’t even a thing in America until 1970s. Corporations were originally formed to limit liability in the 1600s. I can’t say that is better or worse. There is a lot of novel approaches that come from this, but it also drives enshittification.

Nothing forces a company to cede control of its values to venture capitalists or to go public. And I suspect there’s a lot of private companies that own their destiny and are making moral choices at the cost of revenue.

The legality is an irrelevant distraction. The shareholders have a voice with, if nothing else, their money.

It feels like people are surprised that a public corporation is seeking to do whatever makes people believe it’s worth more. The people who buy or sell are voting on how they believe the company ought to be behaving.

Why would investors invest, if a company declares that it has no interest in returning shareholder value? I'm sure that many people do get a kick from supporting companies that they believe in, but generally investment is not done for purely altruistic reasons.
Perhaps there is something between maximization excluding all else and no interest at all.
Ironically the case that established this principle is one where the investor was trying to purposefully sabotage the company from expanding into the market as they also owned a competitor (Dodge v Ford). So actually this “return shareholder value” was used precisely to put Ford into a straight jacket in terms of expansion while the Dodge brothers grew their business.
I imagine that if the company's goals were disclosed, potential shareholders could make that determination for themselves. Not to be snarky but even among investors, who are possibly more a monolith than most groups of people, they are not necessarily all motivated by the idea of "maximize".
Delaware corps have a duty to maximize shareholder value when selling the business. But no us jx require a corporation to unifocus on shareholder value to exclusion of all or any other interest. Nor does maximizing shareholder interest as it’s understood by practitioners ask 7 Deadly Sins performances of greed of its CEOs.
Why don't they pay dividends?
(comment deleted)
Stable long term growth is valuable. An excessively greedy CEO that gets himself ousted creates uncertainty, which is less ideal.
Ensuring a diverse field of customers seems like the stable long term growth strategy to me.
hmmm, greed?

follow up nvidia's history, Jensen never was a guy motivated for money, they are doing the best the can to compete in a market where just staying alive doesn't cut it

check Intel and IBM

If someone expects Nvidia stock to single handedly fund their retirement then they're greedy as well.
Looks like this claim was coming from Ed Zitron

https://respawnfirst.com/multiple-reports-say-nvidia-manufac...

I honestly find his podcast a little grating and his personality needlessly abrasive/smug at times, but at the end of the day I still read and listen here and there because the dude does nail it often enough to warrant some of his attitude.
meh, he's like Burry predicting doom and gloom no matter what. Burry was right ONCE and then pretty much never again. I don't pretend to have the truth but life taught me to chill it when I see stuff that doesn't make sense TO ME because markets have this way of staying irrational way longer than I can stay solvent.
I would agree with you except that Zitron has been correct more than once.
The claim is coming from the Wall Street Journal

Zitron already has a video about it but he is just reaffirming what the article already said.

This is just feels really misleading. Nvidia invested in startups that would turn around and help the market of companies that would buy from them grow.

That's not "deliberately keeping GPU prices high" and it's really weird to frame it this way.

It is good that Nvidia focused on supporting startups!

They did what car companies do.

They finance the selling of their inventory.

all the pitch forks are coming out all at the same time, what a coincidence.
We had a huge thread about this yesterday

Nvidia allegedly engineered scarcity by funding AI startups to buy its GPUs (techpowerup.com)

https://news.ycombinator.com/item?id=50493251

"engineering scarcity" is bullshit. It's called "demand generation" and every single enterprise company that can do so does it (car companies, cloud providers... they all either provide financing or credits / volume discounts).
You can call it whatever you want but read the WSJ article. The CFO Kress was against this plan but Jensen pushed it ahead. Now he's out and the CFO Kress is the interim CEO. She will probably slow down or stop the vendor financing. That will have a huge impact on the AI industry. What will the new CEO do whenever they are hired?
while China pours billions over billions in catching up.

neutrally, taking Jensen out is a proverbial shoot in the foot

> taking Jensen out is a proverbial shoot in the foot

Probably should have kept his dick in his pants then, and avoided even the impression that he might be complicit in skirting export controls.

FWIW, I don't even agree with the export controls, but if they exist then everyone should follow the law equally.

It's really amazing how many people on here think that Huang being incredibly unethical is a small price to pay for the current AI market being the way it is, when pretty much no one likes the current state of the AI market.

Being a founder or generating tons of value doesn't let you dodge export controls, introduce sexual relations into your corporate culture, or lie to the board about how many times you had done that.

> The CFO Kress was against this plan

That is why she should not be a CEO. And frankly speaking she shouldn't even be a CFO in the first place.

It's a bit more troublesome when they've generated so much demand that they've blown up the global market for RAM and NAND...
Everyone please just read the Top comment of that thread.
>circular financing

It was an open secret that 'LLM inferencing as-a-Service' was a license to print money until very recently, and still is if you like CoreWeave, Crusoe, Xai etc. can get LLM inferencing deals from OpenAI and Anthropic.

Nvidia invested in genuinely investment attractive businesses. The "circular financing" accusation just seems like wishcasting from the anti-AI crowd.

one way to spot that this makes no sense is to look at GPU rental prices [1]

if there were some cabal buying them to keep purchase prices high, then rental prices would be falling; it would be unprofitable to buy them in order to rent them out. but actually the opposite is true

[1]: https://semianalysis.com/gpu-pricing-index/

> if there were some cabal buying them to keep purchase prices high, then rental prices would be falling

What?

Scarce things are scarce. Scarce things cost more.

Are you assuming that this cabal is buying GPUs to rent them and thus there would be an over-supply that puts downward price pressure?

A lot of people are dismissive of this as if it was obvious

The circular financing meme was always about the other frontier labs and Mag 7, not about smaller private players that Nvidia was buying to purchase its GPUs

Thats a huge difference and the scale of this is not necessarily known

Meeting analyst targets to buoy the share prices is a margin of like 1-5% in revenue

This could really no longer justify the share prices, the forward guidance, the margins on gpus and a lot more. The true demand not being known

The whole semiconductor supply chain is inflated because of this fictional off balance sheet demand

People didn't want AMD chips, and still don’t, but they try to make it work based on scarcity and premium of Nvidia chips, people are resurrecting DDR3 because all more advanced memory is scarce

If none of that is true or necessary, grab your shorts and look out below!

  > People didn't want AMD chips, and still don’t, but they try to make it work based on scarcity
it might be good in the long run though if amd and others can benefit from that and finally get good competition with cuda et al
AMD business model is being the best second place on a market with two players. They will promptly waste the opportunity and shoot them selves in the foot again.

Just like intel, AMD might only move if extremely threatened.

Lots of discussions about the whole strategy of chips, which are super important, but only after the basic is done: make faster chips people actually want.

Not speaking about the possible knowledge of smuggling in contravention of law, marital infidelity, and possibility getting mixed up with an intelligence agent, but I don't see the circular financing thing as a problem. Maybe I'm misinformed.

But weren't they were investing money in companies to help them grow and in turn buy more of their product. Of course Nvidia wants to stimulate demand of their product. I don't think this was nefarious. There is no free lunch, the risk of them never getting that money back is real with a lot of these pie in the sky AI companies. How is this anticompetitive? It's all of these SPVs that meta, google, and others are using that I think needs to be investigated and punished by wall street.

If you think that's wrong, USE YOUR WORDS, I'm open to reason. Down voting with no response or making jeers is not effective communication.

I don't see the issue either. You could argue that it's not the wisest idea, but that's NVIDIA's call to make. I don't think it's inherently anti-competitive; or at least only in the sense that making a better product is anti-competitive. You're giving your customers what they want.
Doubly weird because Nvidia "artificially keeping their prices high" is something they could also do without the circular financing. Nvidia doesn't have to sell GPUs to people at any price, they can choose not to sell them if the price is lower than they like. Which "artificially" (whatever that is supposed to mean) keeps the price high.

They don't even have to sell the GPU at consistent prices, they can just given them away to some people and sell them at high prices to others. Which is what the circular financing accusation would be equivalent to. As long as they're transparent about what they are doing there doesn't seem to be any sort of problem. If someone thinks NVidia GPU are overpriced then they can go buy from AMD or whatever.

“Circular financing” is a financially illiterate meme. How many months has this concept been circulating Reddit, LinkedIn, HN? You think some Nvidia investors might not have been tipped off to this by now if it was an actual concern and not just “le economy is collapsing” fud that has been a dime a dozen for the last 5 years (minimum)?

If you are buying into this let me tell you now: your cynicism and desire to feel like you’re somehow more informed than the entire market (because everyone else is just stupid/evil/both), this is doing nothing good for you.

True. It would be the first time ever if investors got fleeced choosing to ignore something too good to be true. Never happened before ever.
Investors who trust the market to correctly price assets and who maintain their investment strategy over long time horizons despite daily tiktok-tier financial misinformation generally do much better than those who look for every reason to stay out of the market or are always waiting for a “bubble” to pop.
Is this opinion in conflict with my post? Or are you saying the market has never mispriced an asset before?
Circular financing might be illiterate, but I thought some of the allegations were circular _accounting_, which is actually a thing, without commenting on if it’s occurring here
I hope a few more heads will roll in Nvidia leadership/board and in the Trump administration as well. Someone said it right on https://x.com/ gnoble79/status/2121233927352250754, Jensen ran an arms-dealer for both camps (US and China).
This is such a weird take. The demand was there, so they capitalized on it by financing the purchases.

This kind of thing happens EVERYWHERE. You ever notice how essentially every single carmaker has a credit arm? Practically nobody can just outright buy a car at the sticker price, so auto loans exist. Eventually, the carmakers realized they could make money on the backend if they got into the game of providing the loans to buy their own products. There are TONS of industries like this. Practically every single industry around capital intensive machinery and equipment operates like this. Farming, manufacturing, shipping, oil & gas, etc. etc. The biggest players who can afford to do it try to verticalize the debt/credit market for their own products because it makes complete sense for them to do so. Now that semiconductors have gone from personal computing to “industrial equipment”, this transition in the chain of financing would be a completely banal evolution.

Also, consider the other massive glaring hole in the narrative… NVIDIA has hungry competitors who would absolutely love to eat away at their dominant marketshare. Google, AMD, Intel, Qualcomm, etc. could have come in with their own purported “avalanche of chips” at comparatively bargain basement prices to dismantle NVIDIA’s grip on the market. Notably, they did not, because there is no actual supply glut presently.

NVIDIA financing the purchase of their own products is a demand-side subsidy. Demand-side subsidies increase prices when demand is inelastic. Hence, the demand was there and NVIDIA did the same thing that GM, GE, John Deere, Airbus, etc. etc. etc. all do… all. the. time.

Your commentary around automotive is a good one. I'll extend it to add that it's reasons like this that auto has gone bankrupt so many times. You can over finance to keep people going only as long as they can pay the bill. Then you are holding the bag.
Yes, that’s one of the risks of becoming a lender. You’re leveraged, and thus the downside is amplified. It is however… not a scandal.

Airbus finances purchasing of their own aircraft. They won’t lend you money to buy a Boeing.

The difference between car companies/john Deere is that Nvidia is coming in with equities and guarantees in a junior position with highly correlated customers who have never made a dime on “capital equipment” with a micro depreciation schedule and thin resale market. These are not separate financing agreements that can be surfaced or segmented but footnotes in financing. Not here to argue that “purchaser financing is necessarily a moral panic” but just to point out that Nvidia’s exposure is very different than other durable goods manufacturers.
i can fairly support the circular financing deals as a way to provide GPU infra for the small upto large sized with some sort deals that make sense for them. which seems a way to compensate the other end
If the price of the RTX 6000 (96gb blackwell gpu) drops to $2000 (original MSRP $8000, currently $16,000 usd due to "scarcity") then, all of a sudden, "we have ai at home" becomes real and why would you ever pay for a subscription for your company/family when you can run Qwen Flash Next (which runs great on consumer GPU via software like Strata) forever for free?

Engineering types and power users need not apply,but For Everyone Else, the "normies", Qwen Flash Next is probably "640k is enough for anyone" territory.

IPO, valuation etc depend heavily on frontier, or "good enough" AI to need to run on commercial grade GPUs. If Qwen Flash Next is good enough and runs on a $2000 GPU, why would you pay $20/30/100/200 per month in perpetuity?

I wouldn't pick Qwen Flash Next to add kernel namespacing and cgroups v2 to my custom OS kernel, but it will happily implement the Plan.md that Fable 5.2 wrote for a dollar.

You would need several RTX 6000 to run the large open weight models, and even then, the price of electricity would probably exceed API prices.
MoE next token prediction is higher than 80%; you can store the other experts in RAM and the engrams can be streamed off very mediocre nvme. It's been this way since September. I'm running it at home (although heavily quantized, there are limits) on 48gb ram and 12gb gpu. We run it at work on the 96GB RTX at a much better quant.

See also: https://github.com/niko1221/strata

You aren't hitting almost 2x decode with speculation on a MoE model, but it's irrelevant. Electricity will cost most for most people than API will, even with free hardware. And 4-bit qwen next will be a major competitive disadvantage compared to larger models.
Qwen 3.8 is decent enough for half of what my family would ask it to do.
If RTX 6000 drops to $2000...

That's at or below cost. Nvidia would just stop making them. Of course used cards will reach that price ten years from now.

An RTX 6000 at $2000 is crazy talk. When was the last time the premium Nvidia workstation GPU retailed at $2000? Probably a decade or longer.
> Engineering types and power users need not apply,but For Everyone Else, the "normies", Qwen Flash Next is probably "640k is enough for anyone" territory.

Why would any of those non-power-users buy a $2000 GPU that doubles as a space heater (seriously, try sitting next to one during long runs in the summer) and thousands more for a computer to host it, plus all of the trouble of setting it up, instead of a $20/month subscription?

I do have the hardware to run Qwen3.8-Flash-Next at home, and I still heavily use my subscriptions even for simple tasks. The experience of using the apps on my phone on the go is great and I always have the option to scale the model up to the task.

The people who are going to pay thousands of dollars up front to avoid $20 are power users and people who have different priorities and motivations, like independence or ultimate privacy. GPU prices dropping won’t change that much.

Also, you don’t need an RTX6000 for Q3.8FN. There are much cheaper options right now at lower speeds.

> The experience of using the apps on my phone on the go is great and I always have the option to scale the model up to the task.

Just sidenote, this experience can be replicated with software like Paseo or some other options.

I usually have a frontier model orchestrating between dumber frontier, Chinese, and local models, using Paseo subagents, from my phone or whatever hardware I'm near.

Though I don't lean too much on the local models, I find them to simply be too stupid. They're fun for experiments though like quizzing them against Wikipedia or making them debate philosophy in loops.

Partly, it's much faster than the cloud models. Also, those cards aren't that hot, the Max-Qs are rated at 300W, and can be limited to 250W.
Tokens per second is slower on most local setups than cloud models.
Even if you own a RTX6000 anyway it's not obvious that it's cheaper to run Q3.8FN on it than buying tokens. Google tells me that it can produce ~1 million tokens per hour and uses 450W of power, which for me comes out to 18¢ per million tokens. That's very similar to the price on OpenRouter. Maybe during the winter when all the waste heat cuts your heating bill a bit?
The workstation edition defaults to 600 W, Max-Q to 300 W. 1M tokens/hr is 277 per second, which is roughly right for a single request at a time for a lot of models, not sure about Q3.8FN specifically, but these things scale really well to 16+ concurrents, I typically get a couple thousand tokens per second on Deepseek Flash when the cards are heavily loaded. MTP (a small network that predicts a few tokens which get checked in parallel against the full net, kind of like CPU branch prediction) makes a huge difference in single stream throughput.
it's probably not cheaper, but economics are only one concern at play
I did similar math for my card (9070 XT) and it is close to break-even on electricity cost alone nevermind the amortized cost and requirement to have a desktop for it to go into. The parallelization and faster speed of the cloud makes it feel more expensive but it's just doing the same work more quickly.

The strongest case for local AI is for anything requiring privacy/discretion but I don't think individuals will collectively prioritize this. Convenience is king for the majority followed closely by cost. Businesses who care about protecting their competitive edge seem like the more likely candidates for local AI but only if ZDR providers aren't good enough.

Local AI will probably end up where it is now: in the hands of hobbyists and/or privacy-minded people who already own a GPU for gaming.

This assumes status quo. The privacy part will become more important as subscriptions become more invasive, ad-ridden, and hostile, as it always seems to go. Local will catch on as hardware availability improves and the community converges on common solutions.
Local models can still surprise us, a small 27B could be much stronger in 2-3 years than today, while capability to run it would exist on more devices.
If a company (Apple?) can provide the illusion of privacy and seamless AI integration then they could capture the market of people that are at least aware of the issues.

Despite everything we know about the attention economy the self-hosted community is still a niche group of tech-minded hobbyists. Running anything locally is a commitment and adding expensive hardware to the mix is an even higher barrier.

I want people to care about privacy, technology, and interoperability but I'm just not seeing it happen.

> The privacy part will become more important as subscriptions become more invasive, ad-ridden, and hostile

I would bet good money that my $20/month subscriptions are not going to become ad-ridden.

If someone wants to use the free tiers, then nobody is going to be surprised if they see ads.

But these claims that my $20/month plan are going to become “ad-ridden” are at odds with what is happening. I’ve had at least $20/month plans from OpenAI and Anthropic for about as long as they’ve been available, and the quality and amount of inference I get from even the lowest tier are increasing all the time.

At this point, I can run Luna/xhigh on the $20 plan almost constantly (with brief breaks for tool calls doing actual work, not just token generation for fun) and it feels endless. In another 24 months we’ll be doing even higher quality outputs with smaller hosted models on faster cloud hardware and it’s going to be available from many different providers.

The doomsaying about “ad ridden” should be kept to the free plans. It’s just not going to happen on $20/month

> Why would any of those non-power-users buy a $2000 GPU that doubles as a space heater

Because, unlike the $20 Claude subscription, your own hardware will not respond "sorry, but I cannot do that" when you try to hack a device or game that you own.

Ah yes, all those none-power-user-hackers. Fact is, the average users LLM usage can be served for literal pennies per month in inference via cloud (doesn't matter whether open weight or closed like GPT-6 Luna), their use akin to advanced web search and simple back-and-forth chatting can easily be covered by advertising.

Meanwhile, any of the reverse engineering and recomp efforts we have seen will not work with Qwen3.8-Flash-Next, so not sure how either case applies.

> instead of a $20/month subscription

And that $20 subscription isn't a good comparison. That gets you Astra or Opus. Better comparison is OpenAI's $8/month plan, which gets you Luna (which scores the same as Qwen on benchmarks). At that price, most of the cost isn't even the model; it's just hosting the infrastructure.

With all of the money these companies owe I don’t think that $20 or even $100 monthly subs will stick around forever, but who knows.
This is exactly correct imho.

The same thing happened with uber when they needed to start making money. It's only competition that stopped it from increasing more.

Ironically, going bankrupt would allow them to wipe out the debt, keep the GPUs, and offer cheaper subscriptions.
Who are we talking about? Pretty sure you’d have to liquidate your GPUs in this case
Same level of understanding as thinking going to the town square and screaming "I declare bankruptcy!" frees you from your debt.
I don't think so? A Chapter 11 is only possible if you can demonstrate clearly to the court that the underlying business would be viable without the debts. It also wipes out the investors.

AI startups are characterized by:

1) Market dumping. They are burning investor capital to buy market share.

2) A need to constantly raise money to pay for (1).

A Chapter 11 bankruptcy would make no sense in this situation. They haven't demonstrated that their business model is viable, so if a court ordered a Chapter 11 these firms would just immediately go bankrupt again, and that's usually a criteria to require a different and harsher kind of bankruptcy.

But even if for some reason a court did allow it, they'd still need to do emergency raises because their outgoings are so high. Debt has nothing to do with that, it's purely a matter of fixed costs and (in Anthropic's case at least) huge locked in contracts. But no investor is going to give money to a company that just permanently defaulted on them and ran to court to zero out the value of those shares. So they would be unable to raise and would immediately go bankrupt again.

Then you just move once they're more expensive. They have essentially no lock-in.
Anthropic has so much cash they earn a billion dollars per year in interest.

The reason they’re entering compute contracts is because every dollar they spend on compute for serving their plans (inference) produces more dollars of revenue.

Models are getting better. Serving hardware is getting faster. Fabs are being produced to make more hardware.

Things will only get cheaper and more plentiful from here.

I live in northern EUrope. I wouldn’t mind a space heater.
No one other than engineering types and power users want to run their own server. Everyone else just wants their phones and laptops to connect to the cloud and do the magic for them.
Well the $200 plan will cost $20 instead because they have the same input costs, by your theory, or they don't, because energy isn't getting cheaper. Running your cluster at home in California can easily cost $900/mo
> add kernel namespacing and cgroups v2 to my custom OS kernel

deepseek can run locally on my strix halo and handles this kind of thing just fine. it really is good enough already for hands off programming, and hands on looping for slightly more complex stuff.

> the "normies"

The normies are not going to pay $2000 upfront just to use AI lol. Why would they?

Engineering types should be plenty happy with this setup too
>Qwen Flash Next is probably "640k is enough for anyone" territory.

This is a weird comparison to make, because 640k is known mostly because of how wrong this prediction was. It turned out that people wanted more, and the more RAM was available the more RAM hungry applications were created. So probably the opposite of what you meant (that Qwen will be enough for most).

It was mostly a nod to the fact that most predictions about technology turn out being wrong, but at that point in time seem perfectly valid.
> to add kernel namespacing and cgroups v2 to my custom OS kernel

To be fair, you wouldn't use an LLM for this.

I use LLMs for code review a lot, didn't try Qwen Flast Next but the bigger models do give better reviews so just having this case probably won't apply to me.

This is a crazy take. No one pays $200/m and uses a single agent at a time, and at $20/m it would take 100 months to pay back, and we don’t know whether frontier then will be bigger, similar, or smaller models yet.

The subscriptions (or rather, the model training) are subsidized at the moment, so all of this could change in a heartbeat. But e.g. Anthropic subscriptions seem to be terrific value AFAICT.

And that’s at $2000, which is never going to happen until these cards are several generations old. That’s probably close to or below cost.

If it drops to 2000, then it will take 100 months of $20/mo AI subscription to amortize out, not including the time value of money or the cost of running the machine. I don't think this is a good trade for most people.
I can’t argue with your math or conclusion, but the long term price will not be $20/mo when the companies need to start turning a profit.

And if they manage to build a moat the sub price will be even more.

That’s what people keep saying. I have my doubts. Open weight models and their lower prices will forever create a ceiling on what AI companies can charge for a given level of intelligence.
$20 a month subscription won't get you very far with current models, but yeah buying hardware doesn't make sense financially unless you've got it utilized 24/7.
But this doesn’t include token usage? For $20/m there are serious caps on usage limits. I mean I can cap out the usage limit in about an hour and have to wait 4 hours to prompt again then there is the weekly limit. So with local usage your unbound on these time constraints but like others have said there is a power cost.
Flash next sucks compared to Opus 5.5 let alone fable
50 something SWE here. I’m happy with a 2015 MacBook Pro. Don’t need Ai at home. Ever. I think you have a misaligned view of the world.
> why would you ever pay for a subscription for your company/family

For convenience mostly.

Why do the majority still continue paying for Netflix et al. when you can have your home media server at home with automatic downloads?

It's worth noting that media piracy is illegal and therefor complicated in most places, requires a lot of manual setup, and people remember seeing fines of tens or hundreds of thousands of dollars.

Installing ollama or lmstudio on a computer with a GPU is about as complicated as installing Netflix. Computers come from the store with GPUs installed and an OS.

Outside of dev people typically don't use LLMs on a desktop computer.

To be competitive with frontier offerings, it must be multidevice, available on any network, and integrated with all the apps people are used to use AI with.

Most tech enthusiasts don't self host super trivial things like a local DNS resolver, Their web presence, or email despite the huge incentives and the super low computing requirements, what makes you think they would build and run at least 12/24 a day a dedicated fan buzzing computer (to be useful for the whole family, it can't be the same as the gaming machine) in their home to be available for whatever request their family members may want to do to a model?

I think that AI appliances will become a commodity at some point and people will run this at home.

Things are moving too fast right now, so it’s mostly hobbyists and tech enthusiasts.

at the very least where i come from, engineering types are those who most value privacy, and carry the greatest disdain for direct-to-customer saas in their own personal lives. diy ethic and that. QFN is as dumb as local is ever going to get, and you can run it at acceptable speed on a machine that gets good frames in a ue5 game on high and was built with enough ram for a virtual machine or two.
$200 / month is getting you access to frontier models that can't run on a $2000 GPU and let's you run lots of things in parallel. I'll very often run swarms of 10-20 agents that I couldn't possibly run locally without likely $100k-$1m of local compute?

Essentially is the same as "why use the Cloud" argument.

I could 100% see myself wanting local inference for some local use-cases: continuously monitoring my network for intrusions, checking my security cameras every few seconds, running long running video generation models, etc.

> Essentially is the same as "why use the Cloud" argument.

The case is much stronger. You can run pretty much every "cloud service" out there at home on hardware that is practically e-junk. It really does not take much to serve webpages, host email, or a fileserver. In all cases you get something better than a commercial solution.

And if you use your AI to set that up - because people complain the level of friction to keep such solutions working, then you can make your AI costs back in cloud costs.
Maybe say "Qwen Flash Next" a few more times for emphasis.
Putting 2k in a 3% savings account already pays for three $20 months. Include the rest of the hardware around the gpu and electricity costs and you never save money by running your own hardware. Which makes perfect sense, since cloud providers have cheaper electricity and can use hardware more efficiently by running it at higher load. The reasons to run inference locally will never be economic. Perhaps we will reach the point where it becomes cheap enough not to matter so that nonmonetary considerations win.
If you run it at home, it won't be realistically doing it 24/7. 8 hours per day is already 33% utilization. If the model is actually running half of that time, it's already 15%.

So, there will be always business for someone to host one in a data center, and sell the tokens at a markup, that will still be cheaper than buying something you will only use 15% of the time.

I don't think it's currently possible to calculate a reasonable valuation for an AI company using conventional techniques. It is too new, it is still rapidly evolving, nobody knows how it will reshape the industry. It's more of a poker game where the underwriters are betting on how much the investors are willing to throw into a perceived once-in-20-years oportunity.

There's a lot of use cases where you may not be personally interacting constantly but the LLM is running a lot of the time. A consumer-hostable model like qwen3.8 flash next take longer than opus or better models to complete software builds, and I set mine off before bed, before work, and over lunch to take the next stage of some feature. It is running pretty consistently, just over more time than if it ran on the more powerful datacenter hardware that we use at work.
The math still doesn't add up.

Let's say, you get 80 tokens/sec, so 80 * 3600 * 24 = ~7M tokens or $3.3/day at the current OpenRouter token pricing. That's ~2 years to break even at $2000.

That's also about 7kWh/day assuming 300W, so net $2/day or ~3 years. Given how fast the hardware changes, it makes no sense at all.

Also, in my experience, Qwen 3.8 heavily relies on reasoning to produce meaningful results. So where Gemini-Flash would just give you a usable result in like 5 seconds, Qwen would stumble on a trivial technicality (like completely forget '+' and '-' in a patch), then reason for a minute or two on "hey, but what about that requirement, maybe it is related"? and eventually come to the right answer. But the speed and the token use is abysmal.

I mean, it's a cool model and it's a big step, and it's crucial to have open-weight AI, but currently it's not practical beyond proof-of-concept.

I get quite all my work done with Claude Pro, which is around $20 a month, even using it at work. I could subscribe to this for 10 years until breaking even with your fictitious $2000 RTX 6000. Economically this just doesn't make sense.
Incredible what one can achieve with fantasy math and leaps in logic
If it dropped to 2,000, I'd rebuild my server all over again with 384GB vram.

DSV4.1F is enough.

I known 0 "normies" that work on a desktop computer, or that would even remotely care to invest 2k upfront to not pay for ai subscription, or have a tiniest effort cpaacity to learn how to install it.

Why do you think everyone still pays for icloud or google drive instead of just self gosting a foss on nas at home? Hell, even me as an engineer know how to do it and know i would save money but i just can't be bothered because of convenience

Isn't the lesson of "640k is enough for anyone" that people will end up wanting more? Nobody today will buy a computer with 640KB of RAM, even though surely this would be cheaper.
With Strata and the other inference engines designed for these moderate sized MOE models its more about RAM than it is the GPU. Qwen Flash Next will run on a fairly typical 12-16GB consumer card at 50 tok/s but you need 64GB to hold the weights of the experts in the model with NGRAM coming off the SSD. A blackwell GPU will run it faster but you can get a good experience with Qwen flash next at 2-4 bit quant mostly on RAM.
Agreed, almost nobody needs the ability to run 60 gpu workloads concurrently, unfortunately nvidia refuses to sell a competing product with 64, 96, 128gb vram
Thinking that the RTX 6000 can go from $8000 to $2000 just like that is beyond any reality and shows a lack of basic grasp of how (GPU) prices evolve. Even with almost perfect economic conditions, it will take a couple of years... And we're way off even average economic conditions.
It’s the other way around- the engineering types know the limits of a smaller model and will coordinate their tasks accordingly.

A normal user, on the other hand, knows nothing and will be disappointed after finding out that a local model doesn’t keep up with a month-long conversation, sucks at searching the web, makes way more logical errors and requires better directions than chat gpt.

Coding is way easier for a smaller open-weight model than navigating real-life issues

I don't think there is legitimate story behind artificial scarcity claim. I'm in the industry and I know that the labs and AI are companies are buying up any compute available. No one is even blinking about the cost. You don't have to be an insider, look at that lengths both the frontier labs are going to and Sam's public comments.