It sure is tempting to make it seem like a plan. Nvidia had been through cyclical financial situations before, most recently with crypto mining. Jensen had had to know he was riding a tiger. I doubt he planned his exit, but he must be a little relieved. It will be interesting to see what Elon does. I don't think Elon can let go.
> In an August meeting, Huang belittled Kress for questioning the deals, calling her “a deer in the headlights”. To justify why Nvidia needed to financially backstop OpenAI and smaller cloud providers to keep buying its GPUs, Huang admitted that Nvidia was caught in a massive squeeze.
Are these two sentences related? Is it saying Jensen questioned her on the circular financing? Or was she questioning him? A bit lost on that
There seems to be some evidence of "GPUs sitting in warehouses awaiting to be plugged in". But it seems to be mostly/only Ed Zitron pushing the agenda that it's not as planned and that there is oversupply of GPUs.
All hyperscalers have in the last 2 months stated that they have more demand for AI compute (I guess GPUs & their own chips) than they can supply.
Nvidia might have been trying to spread the demand over more providers (neoclouds, and seemingly China) and provided financing to enable that.
It's not clear that this means there is an oversupply of reserved wafer capacity and Nvidia GPUs. But if datacenter construction slows down it might affect the GPU demand as they can't be plugged in.
> [...] pushing the agenda that it's not as planned and that there is oversupply of GPUs.
I don't read that it is not as planned, there is high demand from everyone but it appears to be similar to the over recruitment of engineers. They have FOMO, and they also want to block competitors.
> All hyperscalers have in the last 2 months stated that they have more demand for AI compute (I guess GPUs & their own chips) than they can supply.
I certainly have not had any issues getting as much access as I would like. It just costs. Which would imply that the scarcity is manufactured to drive up pricing rather than availability.
> if datacenter construction slows down it might affect the GPU demand as they can't be plugged in.
I think this is the crux of the matter from Nvidia's point of view, that's if they weren't already impacted by the power limitations for customer installations.
> I certainly have not had any issues getting as much access as I would like. It just costs.
It just costs, because demand is high, supply is low, if it was opposite, it would have been like in 2018s, you could get 128Gb RAM server from Hetzner for 50-80 EUR, now it starts at 170 EUR (3x)
There was a post about how growth in the data center space was slowing, so AI must be over. The evidence seems to be that we've reached whatever growth capacity existed or was planned, rather than any drop off in demand.
I could never track Zitrons argument. I would not be surprised if there are gpus sitting around, we have known for at least the past year that hyperscalers have been unable to build enough data centers. They continue to state demand exceeding supply, that’s kind of why the neoclouds have even become a thing. It does not indicate to me that those gpus are not going to be used or even if companies overpaid for them.
Explanation not endorsement: The argument is that while end user demand for gpu compute may exceed currently online supply, for Nvidia to be able to keep the revenue up, the demand will have to exceed not just currently online supply but also the supply of gpus that have been manufactured and sold but not installed. Otherwise you get some point where neoclouds etc say “ok we have enough gpus for a long time we’re going to spend our money installing them, not buying more. And then suddenly Nvidia’s revenue drops.
Definitely get the idea what never clicks is he always uses it as a gotcha. I am not convinced there is pervasive circular financing. I am certain some entities have overcommit but based on current demand I don’t think it will be an issue for most entities. So maybe Nvidia juiced their earnings for a year or two, they will pay the price. The capacity will still come online.
I thought the argument was that Nvidia were requiring buyers to take immediate delivery of their purchase and wouldn't let them secure future parts or something like that. This was forcing hyperscalars to store their GPUs until they actually had data centers online.
This whole buildout is one of the largest construction projects ever - there will definitely be oversupply hotspots even for items that are in super short supply. I could easily imagine $1B+ of top-tier chips just sitting around in a warehouse for months on end because of difficult planning problems. That's not a sign of any sort of crisis.
> All hyperscalers have in the last 2 months stated that they have more demand for AI compute (I guess GPUs & their own chips) than they can supply.
That just means the price point is wrong. They might say they want to buy more racks to increase supply and increase profit, but turning away customers is not what they're doing in a market where prices turn on a dime.
Of course they were. They bought up literally as much as they could manage, because anything they didn't would be bought up by someone else. It's literally a use it or lose it market right now. This is not a problem. Just stockpile it.
"It's literally a use it or lose it market right now."
Are you sure? The point of this stockpiling was to create that impression. Now that you have data that that foundation behind that impression is manufactured and faked, you need to reconsider everything that was resting on that foundation. The demand is just so incredible needs to be reexamined from scratch now in light of the active attempts to forge that signal.
It is also worth remembering that this is a prime example of a situation that is not black or white. The absolute size of the demand, however large it may be, is not the question. It's the size relative to the supply, supply which includes all the pumping shenanigans that are coming into the light. The "AI bubble" has never been if there is demand or if the tech has real use, it has always been a relative question about if the build out could possibly make economic sense.
signed closed source drivers for hardware that can restrict which algorithms can be run on it locally... temporally...
edit: i'd willingly bet that AMD has a backroom deal to tank their performance and options to encourage this homogenous deployment overseas.
swallow the hook.
Given that you practically cannot buy memory or storage right now because all of the capacity is taken up by hyperscalers... even if it was a self-fulfilling prophecy, it's still currently fulfilled
That just represents the inertia of the whole system operating on old assumptions, given that it's only still early in the process of coming to light in a public way how much of the frantic purchasing has literally been sitting in a warehouse, depreciating. I mean, I've been pretty suspicious of this for literally years now, but it's been a "conspiracy theory" that this amazing demand is manufactured by a variety of financial hacks rather than just pure demand that the system as a whole has not paid attention to because number was going up, do not question number going up lest the number stop going up.
You need to apply first-principles thinking to this sort of scenario. Beliefs like the demand is crazy do not just exist floating through space; they exist on top of a specific set of facts and deductions from those facts, which can be very easy to forget when a fact seems to be true for a long period of time. When the set of facts and deductions a conclusion rests on are found to be faulty, the conclusions need to be re-examined from scratch. It is not generally impossible that a fact could still end up being true, just for different reasons.
However in this specific case, it strikes me as extraordinarily unlikely. If the shortage has been manufactured, then all assumptions around demand and possible profitability have to be completely re-examined. Once the system digests this, because these things do take some time, it can cease to be a "use it or lose it" market very, very quickly, and turn incredibly quickly into a "liquidate everything you can while it still has some sort of value" because of massive, massive oversupply. What is the value of a rack of nVidia processors that nobody can turn on? What is the value of a rack of nVidia processors that nobody can turn on when there are warehouses' worth of the things? Anyone who blind operates on "it's a use it or lose it market" will lose their shirts on the transactions. "It's a use it or lose it market" is not a foundation; it is contingent on a lot of things, some of which are being shown to be false.
To be fair I did say "right now". The demand hasn't stopped quite yet.
I look forward to the next R710 where you can just buy enterprise grade servers for a few hundred bucks and as long as you have somewhere to put a hurricane level of noise, you can Just Have terabytes of memory at home. Or many-GPU in this case.
I'm sure Nvidia would have no issue selling this "avalanche" if they just dropped prices a bit
We went with AMD for our $300k AI server, then after two months of delays were told it will be delayed another 10 months. Getting the same amount of VRAM from Nvidia remains out of our budget
Meanwhile "reasonably-sized" cards like a RTX 6000 Pro Blackwell or even a consumer RTX 5080 still trade for twice what they cost in January
Honestly agreed. I would gladly switch back from AMD if Nvidia lost their multiplier over how it was a few years ago (and got rid of that new power connector). But my few thousand is nothing to Oracle and co's hundreds of billions. Even your $300k. So if the circular financing does pay off, it'll pay off big.
Can they afford to drop the prices without causing investor panic? Feels like Nvidia needs to constantly exceed all expectations to justify their share value. And if their share value drops, it might have a negative impact on their ability to finance their own customers, and if their customers run out of funding, they’re gonna have even more trouble shifting units.
Yes this is 100% about stock price/perception. Every single piece of Nvidia hardware is being scalped at the moment; they must be averaging 2x MSRP, upping supply is not real problem.
Radeon AI PRO R9700 32G gpu with the recent CUDA-free LLM software... now punches way above its budget class. They are cheap as most folks haven't figured out the recent performance hacks yet.
They are still everywhere, and who cares if it takes 14 hours instead of 6 hours if the task gets done. =3
Nobody ever listens to the CFO and they should. The one person who actually keeps a company on track is the person who looks at the actual numbers. I have seen this over and over it’s a freaking meme.
This article inaccurately paraphrases actual journalism at the WSJ. That article does not use the word “excess” once. They can have an avalanche of wafers coming at them without it being excessive. You can make that inference but that exact statement cannot be attributed to Huang, per the actual reporting.
The actual quote from the actual article: “ Then he went on to describe the situation the company was in. Major cloud providers were running away from Nvidia, making plans to use alternative chips and lessen their reliance on the firm. Even worse, Huang said, the vast supply of chip wafers and memory Nvidia had committed to buying were coming at him like an "avalanche."”
A) The point is mainly that if someone is reporting on something they need to separate facts from inference. If he did not say excess, you cannot say he did, and there is no support from the WSJ source article for this claim.
B) Huang has been clear that he needs to diversify his customer base, in part bc of the threat posed by in house accelerator programs. He could see urgency in finding customers beyond those key accounts without necessarily seeing excess supply across the industry. He is also famous for operating with high urgency (the “our company is 30 days away from going out of business” quotes come to mind).
In general it’s important to zero in on what is known vs inferences made to avoid getting caught up in faulty media narratives. I’m not making a judgment on supply excess, just pointing out that small wording changes can nudge groups of people into weakly substantiated conclusions. That can then build into a consensus view that is not supported by fact. This is why people hate journalists and analysts but also why we need good ones.
My hot take is that the avalanche of wafers and memory is actually Nvidia's secret advantage, because that avalance is coming to Nvidia and not its competitors and customers! (But I repeat myself.)
I suspect that this massive supply-chain advantage, locked in by Nvidia's massive monopoly cash flow, is a huge factor in keeping other competitors like TPUs and Trainium etc catching on; they simply cannot acquire the necessary volume of memory at a competitive enough price-point.
40 comments
[ 2.3 ms ] story [ 23.4 ms ] threadAre these two sentences related? Is it saying Jensen questioned her on the circular financing? Or was she questioning him? A bit lost on that
All hyperscalers have in the last 2 months stated that they have more demand for AI compute (I guess GPUs & their own chips) than they can supply.
Nvidia might have been trying to spread the demand over more providers (neoclouds, and seemingly China) and provided financing to enable that.
It's not clear that this means there is an oversupply of reserved wafer capacity and Nvidia GPUs. But if datacenter construction slows down it might affect the GPU demand as they can't be plugged in.
I don't read that it is not as planned, there is high demand from everyone but it appears to be similar to the over recruitment of engineers. They have FOMO, and they also want to block competitors.
> All hyperscalers have in the last 2 months stated that they have more demand for AI compute (I guess GPUs & their own chips) than they can supply.
I certainly have not had any issues getting as much access as I would like. It just costs. Which would imply that the scarcity is manufactured to drive up pricing rather than availability.
> if datacenter construction slows down it might affect the GPU demand as they can't be plugged in.
I think this is the crux of the matter from Nvidia's point of view, that's if they weren't already impacted by the power limitations for customer installations.
It just costs, because demand is high, supply is low, if it was opposite, it would have been like in 2018s, you could get 128Gb RAM server from Hetzner for 50-80 EUR, now it starts at 170 EUR (3x)
That just means the price point is wrong. They might say they want to buy more racks to increase supply and increase profit, but turning away customers is not what they're doing in a market where prices turn on a dime.
Are you sure? The point of this stockpiling was to create that impression. Now that you have data that that foundation behind that impression is manufactured and faked, you need to reconsider everything that was resting on that foundation. The demand is just so incredible needs to be reexamined from scratch now in light of the active attempts to forge that signal.
It is also worth remembering that this is a prime example of a situation that is not black or white. The absolute size of the demand, however large it may be, is not the question. It's the size relative to the supply, supply which includes all the pumping shenanigans that are coming into the light. The "AI bubble" has never been if there is demand or if the tech has real use, it has always been a relative question about if the build out could possibly make economic sense.
edit: i'd willingly bet that AMD has a backroom deal to tank their performance and options to encourage this homogenous deployment overseas. swallow the hook.
You need to apply first-principles thinking to this sort of scenario. Beliefs like the demand is crazy do not just exist floating through space; they exist on top of a specific set of facts and deductions from those facts, which can be very easy to forget when a fact seems to be true for a long period of time. When the set of facts and deductions a conclusion rests on are found to be faulty, the conclusions need to be re-examined from scratch. It is not generally impossible that a fact could still end up being true, just for different reasons.
However in this specific case, it strikes me as extraordinarily unlikely. If the shortage has been manufactured, then all assumptions around demand and possible profitability have to be completely re-examined. Once the system digests this, because these things do take some time, it can cease to be a "use it or lose it" market very, very quickly, and turn incredibly quickly into a "liquidate everything you can while it still has some sort of value" because of massive, massive oversupply. What is the value of a rack of nVidia processors that nobody can turn on? What is the value of a rack of nVidia processors that nobody can turn on when there are warehouses' worth of the things? Anyone who blind operates on "it's a use it or lose it market" will lose their shirts on the transactions. "It's a use it or lose it market" is not a foundation; it is contingent on a lot of things, some of which are being shown to be false.
I look forward to the next R710 where you can just buy enterprise grade servers for a few hundred bucks and as long as you have somewhere to put a hurricane level of noise, you can Just Have terabytes of memory at home. Or many-GPU in this case.
We went with AMD for our $300k AI server, then after two months of delays were told it will be delayed another 10 months. Getting the same amount of VRAM from Nvidia remains out of our budget
Meanwhile "reasonably-sized" cards like a RTX 6000 Pro Blackwell or even a consumer RTX 5080 still trade for twice what they cost in January
They are still everywhere, and who cares if it takes 14 hours instead of 6 hours if the task gets done. =3
CEOs make the call, and I expect they do listen to the CFO.
It is all utter nonsense.
The actual quote from the actual article: “ Then he went on to describe the situation the company was in. Major cloud providers were running away from Nvidia, making plans to use alternative chips and lessen their reliance on the firm. Even worse, Huang said, the vast supply of chip wafers and memory Nvidia had committed to buying were coming at him like an "avalanche."”
Ouch! Sounds like they have an avalanche of excess chips coming their way and no customers to buy them.
B) Huang has been clear that he needs to diversify his customer base, in part bc of the threat posed by in house accelerator programs. He could see urgency in finding customers beyond those key accounts without necessarily seeing excess supply across the industry. He is also famous for operating with high urgency (the “our company is 30 days away from going out of business” quotes come to mind).
In general it’s important to zero in on what is known vs inferences made to avoid getting caught up in faulty media narratives. I’m not making a judgment on supply excess, just pointing out that small wording changes can nudge groups of people into weakly substantiated conclusions. That can then build into a consensus view that is not supported by fact. This is why people hate journalists and analysts but also why we need good ones.
But far simpler explanation is just supply and demand. There is no need to craft some elaborate theories when the simplest one checks all the boxes.
I think it should not.
I am also hoping companies would move a bit away from AI and start hiring more developers instead.
(yes I am unemployed)
I suspect that this massive supply-chain advantage, locked in by Nvidia's massive monopoly cash flow, is a huge factor in keeping other competitors like TPUs and Trainium etc catching on; they simply cannot acquire the necessary volume of memory at a competitive enough price-point.