Valuations of private unicorns like OpenAi and Anthropic must be in free fall. DeepSeek spends $6 million in old H800 hardware to develop open source model that overtakes ChatGPT.
AI gets better, but profit margins sink with strong competition.
Consider that the chinese might be misrepresenting their costs. A newsletter was implying that they might do it to undermine the sanctions justifications.
Agree that the AI bubble should pop though and the earlier, the better.
Much easier to identify the incentives of the people who just lost a lot of money who were betting on the idea that it was their money that was going to make artificial intelligence intelligent.
Everyone’s already begun trying this recipe in-house. Either it works with much less compute, or it doesn’t.
For instance, HKUST just did an experiment where small weak base models trained with DeepSeek’s method beat stronger small base models being trained with much more costly RL methods. Already this seems like it is enough to upend the low end models niche market, things like haiku and 4o-mini.
Be really skeptical why the people who should be making tons of money by realizing actually it was all a mirage and that they can now get the real stuff for even cheaper, would spend so much effort shouting about this, in order to undercut their own profitability..
They express their cost in terms of GPU hours, then convert that to USD based on market GPU rental rates, so it's not affected by subsidies. It's possible however they lied about GPU hours, but if that was the case an expert should be able to show they lied by working out how many flops are needed to train based on the amount of tokens they say they used vs the flops of the GPUs they say they used.
Total training FLOPs can be deduced from model architecture (which they can't hide since they released weights) and how many tokens they trained on. With total training FLOPs and GPU hours you can calculate MFU. And the MFU of their deepseek-v3 train is around 40%, which sounds right. Both Google and Meta reported higher MFU. So the GPU hours should be correct. The only thing they could have lied is on how many tokens they trained the model on. DeepSeek reported 14T which is also similar to what Meta did so nothing crazy here.
tl;dr all numbers check up and the winnings come from the model architecture innovations they made.
I did a quick search for "llama" and didn't find anywhere they outright state they just fine-tuned some llama weights.
Is it possible that they based their model architecture on the llama model architecture? Rather than just fine-tuned already training llama weights? In that case, they'd still have to do "bottoms up" training.
> DeepSeek spends $6 million in old H800 hardware to develop open source model that overtakes ChatGPT.
DeepSeek claims that's what they spent. They're under a trade embargo, and if they had access to any more than that it would have been obtained illegally.
They might be telling the truth, but let's wait until someone else replicates it before we fully accept it.
That's 8 (not 4), on a NVIDIA platform board to start with.
You can't buy them as "GPU"s and integrate them to your system. NVIDIA sells your the platform (GPUs + platform board which includes switches and all the support infra), and you integrate that behemoth of a board to your server, as a single unit.
So that open server and the wrapped ones at the back are more telling than it looks.
I remember a year ago I was hoping that in a decade from now it would be great to run GPT4-class models on my own hardware. The reality seems to be far more exciting.
PRC companies breaking US export control laws is legal (for PRC companies). Maybe they're trying to avoid US entity listing, lot's of PRC companies keep mum about growing capabilites to do so. But the mere fact Deepseek is publicizing means they're unlikely to care about the political heat that is coming and the ramifications. If anything, getting on US entity list probably locks in their employees with Deepseek on resume into PRC.
Hard to think they plan to, PRC strategic companies that gets competitive gets entity listed anyway. And CEO seems mission driven for AGI - if US going to limit hardware inevitably then nothing to do but go gloves off, and try to dunk on competition. At this point US can take deep seek off appstores but what's the point except to look petty. Eitherway, more technical ppl have pointed out some of the R1 optimizations _only_ make sense if Deepseek was constrained to older hardware, i.e. engineer at PTX level to circumvent H800 limitations to perfrom more like H100s.
Throwing this model out also gives US allies soverign AI a launchpad... reducing US dependency is step 1 to not being US allies.
Which allies? The ones the current US president is threatening in all sorts of manner?
I actually hope he doubles down. I would love for EU to rely less on the US. It would also reduce the reach of the silly embargoes that benefit no one but the US.
If they sell software and build devices in China and then people from the US or our allies have to break our laws to import it, it seems like an us problem.
Depending on how the law is written this may be legal even under US law.
For instance if the law bans US companies from exporting/selling some chips to Chinese companies and that's it then it is unclear to me whether a Chinese company would do anything illegal under US law by buying such chips as it would be for the American seller to refuse.
Anyway, usually this sort of things takes place through intermediaries in third countries so it is difficult to track but obviously it would be stupid to brag about it if that happened.
All of the western AI companies trained on illegally obtained data, they barely even bother to deny it. This is an industry where lies are normalised. (Not to contradict your point about this specific number)
It's legally a grey area. It might even be fair use. Facts themselves are not protected by copyright. If there's no unauthorized reproduction/copying then it's not a copyright issue. (Maybe it's a violation of terms of services of course.)
We don't know what LLMs encode because we don't know what the model weights represent.
On the second point it depends how the models were made to reporduce text verbatim. If i copy-paste someone's article in MS word i technically made word reproduce the text verbatim., obviously that's not Word's fault. If i asked an LLM explicitly to list the entire Bee Movie script it would probably do it, which means it was trained on it, but that's through a direct and clear request to copy the original verbatim.
> If i copy-paste someone's article in MS word i technically made word reproduce the text verbatim., obviously that's not Word's fault. If i asked an LLM explicitly to list the entire Bee Movie script it would probably do it, which means it was trained on it, but that's through a direct and clear request to copy the original verbatim.
But that clearly means that the LLM already has the Bee Movie script inside it (somehow), which would be a copyright violation. If MS word came with an "open movie script" button that let you pick a movie and get the script for it, that would clearly be a copyright violation. Of course if the user inputs something then that's different - that's not the software shipping whatever it is.
> If i asked an LLM explicitly to list the entire Bee Movie script it would probably do it, which means it was trained on it, but that's through a direct and clear request to copy the original verbatim.
Huh? The "request" part doesn't matter. What you describe is exactly like if someone ships me a hard drive with a file containing "the entire Bee Movie script" that they were not authorized to copy: it's copyright infringement before and after I request the disk to read out the blocks with the file.
I mean, it is IP law, this stuff was all invented to help big corps support their business models. So, it is impossible to predict what any of it means until we see who is willing to pay more to get their desired laws enforced. We’ll have to wait for more precedent to be purchased before us little people can figure out what the laws are.
Copies are made in the formation of the training corpus and in the memory of the computers during training so there's definitely a copyright issue. Could be fair use though.
No, the DMCA amended the law to give search engines (and automated caches and user generated content sites) safe harbor from infringement if they follow the takedown protocol.
Of course they are overhyped but in spite of this Altman is always asking for more money. And we know financially they are just burning money. So when someone finally brings a cheap but good model for the masses, this is where money should go. (This will also help all small AI startups.)
The bigger correction will be in tech stocks that are overly exposed to datacenter investments to accommodate for ever rising AI demands. MSFT, AMZN, META they are all exposed
It's kind of silly. It's not like MSFT and the other hyper-scalers dont need the capacity build out for other reasons too. This should be an easy pivot if DeepSeek turns out to be as good as promised.
This is not exactly right, they said they spent $6M on training V3, there aren't numbers out there related to the training of R1, I can feel it will be cheaper than o1, but it's hard to tell how much cheaper. I can guess that overall deepseek spent way less than openai to release the model, because I have the feeling that the R&D part was cheaper too, but we don't have the numbers yet. Anyway, we can assume that deepseek and Alibaba will try to get the most out of their current GPUs however.
The issue here is not that DeepSeek exists as a competitor to GPT, Claude, Gemini,...
The issue is that DeepSeek have shown that you don't need that much raw computing power to run an AI, which means that companies including OpenAI may focus more on efficiency than on throwing more GPUs at the problem, which is not good news for those in the business of making GPUs. At least according to the market.
One of the questions about this is that of the US’s human capital, i.e. does the US (still) have enough capable tech people in order to make that happen?
Lol, yes. The US is still very much at the forefront of this stuff. DeepSeek have presented some neat optimizations, but there have been many such papers and optimizations get implemented quickly once someone has proven them out.
> The US is still very much at the forefront of this stuff
Doesn't look like it, because the some of the biggest US tech companies now active (including Meta and Alphabet) couldn't come up with what this much-smaller Chinese company has. Which begs the question, what is that companies like Meta, Alphabet and the like do with the (already) hundreds of billions of dollars that they invested in this space?
Best guess is that they were all caught up in the arms race to try and make a better model, at whatever the cost. And if you work in this space you were probably getting thrown fistfuls of money to join in on it. I read somewhere on reddit that anyone trying to push for efficiency at these places was getting ignored or pushed aside. DeepSeek had an incentive to focus on efficiency because of the chip embargo. So I don't think this is necessarily a knock on US AI capabilities. It is just that the incentives were different and when stock prices are going to the moon regardless of how much capex was getting spent, it was easy for everyone to just go along with it.
With that said, I think all of these companies are capable of learning from this and implementing these efficiency improvement. And I think the arms race is still on. The goal is to achieve super human level of intelligence, and they have a ways to go to get there. It is possible that these new efficiency improvements might even help them take the next step as they can now do a lot more with a lot less.
I see no reason to believe they couldn't have done so. Rather, this is the typical pattern we see across industry: the west focuses on working out what the next big thing is, and China is in a fast-follow-and-optimize mode.
> You can ban the company but are you going to ban any US company from using the open model and running it on their own hardware [1]?
Just for the people who might not have been around the last time, this has precedent :) US government (and others) have been trying to outlaw (open source) cryptography, for various reasons, for decades at this point: https://en.wikipedia.org/wiki/Crypto_Wars
The vast majority of what the US government has tried to ban was export of cryptography tools. However, as your own link makes clear, they stopped doing that in 2000.
Furthermore, what was restricted was not "open source cryptography"; it was cryptography that they could not break. The only way that open source comes into it is that that is what made it abundantly clear that the cat was out of the bag and there was no going back.
Please try to at least attempt to consider nuance. Do you seriously think that would happen? What is your point here? Do you think people in favor of restricting one thing are in favor of restricting everything?
> Please try to at least attempt to consider nuance. Do you seriously think that would happen? What is your point here? Do you think people in favor of restricting one thing are in favor of restricting everything?
The restriction on TikTok was blatantly because it's a Chinese product outcompeting American products, everything else was the thinnest of smokescreens. Yes, I think people in favour of it are in favour of slapping whatever tariffs or bans they can get away with on everything that China makes.
People are trying to spur up “we shouldn’t use Chinese AI because our data is going to be stolen” discussions. But after TikTok debacle, no serious person is willing to bite. It’s just a big coping strategy for everyone who’s been saying how western AI is years ahead.
I believe that NVIDIA is overvalued, but if DeepSeek really is as great as has been said, then it'll be even greater when scaled up to OpenAI sizes, and when you get more out you have more reason to pay, so this should if it pans out lead to more demand for GPUs-- basically Jevon's paradox.
But surely it can be scaled up, or is this compression thing something making the approach good only for small models (I haven't read the Deepseek papers (can't allocate time to it))?
If the top-tier premium GPUs aren't the difference-maker they were thought to be then that will hurt NVIDIA's margins, even if they make some of it up on volume.
If, as some companies claim, these models truly possess emergent reasoning, their ability to handle imperfect data should serve as a proof of that capability.
Have you read about this specific model we're talking about?
My understanding is that the whole point of R1 is that it was surprisingly effective to train on synthetic data AND to reinforce on the output rather than the whole chain of thought. Which does not require so much human-curated data and is a big part of where the efficiency gain came from.
If we expect that the demand for GPT-5 in AI compute is 100x of that of GPT-4 then if GPT-4 was trained in months on 10k of H100 then you would need years with 100k of H100 or maybe again months with 100k of GB200.
See, there is your answer. The issue is the compute of GPUs is way to low yet for GPT-5 if they continue parameter scaling as they used to do.
GPT3 took months on 10k A100s. 10k H100 would have done it in a fraction of a time. Blackwell could train GPT4 in 10 days with same amount of GPUs as Hopper which took months.
Don't forget GPT3 is just 2.5 years old. Training is obviously waiting for the next step up in large clusters of training speed increasement. Don't be fooled, the 2x Blackwell vs. Hopper is only chip vs. chip. 10k of Blackwell including all networking speedup is easily 10x or more faster than the same amount of Hopper. So building a 1 million Blackwell cluster means 100x more training compute compared to a 100k Hopper cluster.
Nobody starts a model training if it takes years to finish... too much risk in that.
Transfer model was introduced in 2017 and ChatGPT came out 2022. Why? Because they would have needed millions of Volta GPUs instead of thousands of Ampere GPUs to train it.
It is a possibility, but my understanding of what OpenAI has said is that GPT-5 is delayed because of the apparent promise of RL trained things like o1, etc. and that they've simply decided to train those instead of training a bigger base model training on better data, and I think this is plausible.
For Oracle (another Stargate recipient) it was reversion to the mean. For Nvidia, it's a big loss - I imagine they might have predicated their revenue based on the continued need for compute - and now that's in question.
That's arguable, though. I mean it's much cheaper and reasonably competitive which is almost the same but IMHO DeepSeek seems to get stuck in random loops and hallucinates more frequently than o1.
It's funny the bubble seems popping because the tech can be made better or cheaper instead due to loss of enthusiasm for the tech itself. AI is here to stay I suppose then.
Meta should actually go up from this. If deep seek is perfect they don't need to pay for an expensive Llama team. And even if deep seek isn't perfect, the low cost training strategies they've invented could be used by Meta to reduce the cost of Llama training.
Although Meta develops models they don't sell them. So a world where foundation models are free is fine for them.
No, it seems to me like a stopgap measure against others, rather than anything for themselves. If they were out after "open source goodwill" they'd actually release the models as open source (like let people use it without signing a license/terms agreement, and use models for whatever). As it stands today, they're tricking people into "open source goodwill" but it will eventually catch up with them.
From Meta’s perspective, AI could be incredibly profitable in the context of generating adverts or interactive chat bots for businesses.
They just don’t want to use OpenAI/Google models because they fear being screwed over by them with anti-advert terms of service or price increases. Similar to what they suffered with Apple.
It's like everyone forgets that App Tracking Transparency (ATT) was supposed to put Meta out of business. By many accounts, Meta's ad targeting is even better now than before ATT. It's been reported that AI is what saved their ad targeting.
The OSS goodwill is just a side effect and a way to undermine companies who are not using AI to effectively make profits today.
Cheaper/more efficient is absolutely great for Meta. If they can lower their capex it would be an instant bump to their bottom line.
Thanks, but I don't really see how these articles support the claim that their ad network is more efficient. As you note, the first article has a single anecdata point about it actually being 10-15% worse, while the second one basically says 'trust me bro'. Also of note from the second article is the fact that the ad spend would actually increase.
Of course, if businesses are gullible enough to believe facebook when it fudges up some brand lift metrics without having a real impact on conversions, that's their choice. Trusting facebook to report any analytics is how you take your business behind the barn and help it pivot to video. https://en.wikipedia.org/wiki/Pivot_to_video
But now (presumably) they don't need to spend 50 billion? e.g. 5 billion or whatever might be enough which makes it even easier for them to justify this.
Meta's main use of AI is in their own products, I don't think they should be affected. I would be more worried about companies that want to sell AI models and are not being efficient.
I don't follow. Meta has been the only US big dog that released open-whatever variants of their models. They did that intending to minimise the gap between them and other big dogs. Their stated goal is to give open access to the community, while at the same time develop the models for internal uses (on their many platforms).
Meta doesn't sell API access. They are not losing on "cheaper" anything. If anything, they get to implement whatever others release under open terms into their stacks. And they still have all the GPUs to further train and serve on whatever improved stack comes next.
I don't see how meta loses here. In fact I think it is one of the only big players in this space that will come out better.
Good time to buy then, I don’t understand how stupid some traders can be.
A more efficient model is better for NVIDIA not worse. More compute is still better given the same model. And as more efficient models proliferate it means more edge computing which means more customers with lower negotiating power than Meta and Google…
This is like thinking that if people need to dig only 1 day instead of an entire month to get to their nugget of gold in the midst of a gold rush the trading post will somehow sell fewer shovels…
Nvidia is way too overvalued regardless of deepseek or the success of AI. This is just some correction (not even too big even considering the current bubble), these traders are not stupid.
NVIDIA has a P/E ratio of 56 that’s double that of the S&P 500 but half that of AMD and the same one as Meta.
And whether it’s overvalued or not isn’t relevant that selling a stock because the product the company produces is now even more effective is mind bogglingly stupid.
It's arguable how good a strategy it is to check against other P/E. During the tech bubble people would say X is cheap, because Y is trading at 100P/E instead of 200
Again that may reasonable but it’s a completely different argument. Whether there is a bubble or not and whether NVDA is overvalued is irrelevant to the subject at hand.
If it’s cheaper to train models it means far more customers that will try their luck.
If you reduce training requirement from a 100,000 GPUs to a 1000 you’ve now opened the market to 1000’s and 1000’s of potential players instead of like the 10 that can afford dumping so much money into a compute cluster.
the holy grail is to not have a separate train and inference steps. when the model can be updated while it is inferencing is where we're headed. deepseek only accelerates the need for more compute, not less
THIS is the only correct statement in all of this.
The goal for AGI and ASI MUST BE to train, inference, train, inference and so on and that all on the fly in fractions of a second from every token produced.
Now good luck calculating the compute and hard work in algorithms to get there.
Not possible? Then AGI won't ever work because how can AGI beat a human if it can't learn on the fly? Not to mention ASI lol.
P/E alone is useless anyway. A growth company is likely not making a profit as they are reinvesting. But not a profit doesn't implies good either of course.
AMD does not have a PE double of NVIDIA. PE is high because of amortisation of an acquisition. People on hackernews talk a lot but have no idea what they talk about. You might know how to write javascript or some other language but clearly you have not read the earnings reports or financials of AMD and probably alot of the other companies you talk about. So please stop spreading nonsense.
Well the price has a built in presumption that the earnings will keep growing. That's why PER is not that relevant for them, it's been over 50-70 since forever, but the stock went up 10x, which means earnings went up as well. DeepSeek might be good for their business overall, but it might mean earnings will not continue growing exponentially like they have been for the past two years. So it's time to bail.
You shouldn't underestimate the fact that a large amount of these trades are on margin. Sometimes you can't wait it out because you'll get margin called and if you can't pony up additional cash you're basically getting caught with your pants down.
Disclaimer: I am not a trader, so could be way off
Why? The compute requirements would still continue to grow the more efficient and more capable the models become.
If it’s cheaper to inference you end up using the model for more task, it it’s cheaper to train you train more than models. And if you now need only 1000’s of GPUs instead of 10’s or 100’s of thousands you’ve just unlocked a massive client base of those who can afford to invest high six to low seven figures instead of 100’s of millions or billions into to try their luck.
Doesn't this situation also imply to some degree that China is focused on beating the US on AI and probably they will develop a competitor to NVIDIA that will cause margins to drop significantly?
They have a lot of very smart people and the will to do it, seems like a matter of time before they succeed.
It could be, but maybe the feeling is the investments now are already massive and everyone has jumped on the AI train. If you are suddenly 10x efficient, and everyone gets 10x more efficient, there's less room to grow than before. What you're saying makes a lot of sense, but it's one thing to write it on a message board and another to use it to back up your decision that affects billions of dollars you have in your fund.
The proof is in the pudding, you're welcome to prove "everyone" wrong.
> selling a stock because the product the company produces is now even more effective is mind bogglingly stupid.
No it isn't. Investors are most likely expecting there will be less demand for Nvidia's product long-term due to these alleged increased training efficiencies.
There is afaict no inherent limit to expand on the bottom end of the market. My gut feeling is lower training costs will expand the market for hardware horizontaly far faster than any vertical scaling by a select 2 digit of mega corps could.
This is hackernews, not some boilerroom pump n dump forum. Please use more professional language and take your confidence down a notch. Try to learn and add to the discussion.
You seem to believe that the more inference or training value per piece of tech the more demand there will be for that piece of tech full stop when there are multiple forces at play. As a simple example, you can think of this as a supply spike; while you can make the bet that the demand will follow there could be a lag on that demand spike due to the time it takes to find use cases with product/market fit. That could collapse prices over the near term which could in turn decrease revenue. As a reminder the stock value isn't a bet on whether "the gold trader" will sell more gold or not, it's a bet on whether the net future returns of the gold trader will occur in line with expectations, expectations that are sky high and have zero competition built in.
Have you priced in the extremely limited freedom to operate they have? There is an extreme systemic risk to being a monopoly in a strategic position. It's an extreme beneficial position to be in, until it isn't.
ASML for now has a monopoly on cutting edge EUV. Since this is considered a strategic technology, the US dictates what they can sell to whom. This places ASML in a pincer. The US will develop a competitor as soon as they can if they can't get enough control over ASML, and at that point ASML would still be forbidden to sell to BRICS while losing the 'western' market as well.
So they're in a plushy seat, until the US decides they aren't.
I agree with Aswath Damodaran here. NVDA is priced for perfection in AI, but also whatever is next.
In addition, IMO NVDA’s margins are a gift and a curse. They look great to investors, but also mean all their customers are aggressively looking to produce their own GPUs.
Exactly.
gpu's have become too profitable and of strategic importance, to not see several deep pocketed existing technology companies invest more and try and acquire market share.
there is a mini moat here with cuda and existing work, but some the start of commodification must be on the <10 year horizon
They are also priced on the idea that nothing will challenge them. If AMD, Intel, or anyone else comes out with a challenger for their top GPUs at competitive prices, that’s a problem.
The biggest challengers are likely the hyperscalers and companies like Meta. It sort of flew under the radar when Meta released an update on their GPU plans last year and said their cluster would be as powerful as X NVDA GPUs, and not that it would have X NVDA GPUs [1].
Also, I should add that Deepseek just showed the top GPUs are not necessary to deliver big value.
This announcement is one step in our ambitious infrastructure roadmap. By the end of 2024, we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s.
NVDA is a totally manipulated stock. The company beat earnings in the last three quarters and the stock dropped 15% to 20% immediately after the results release.
In economics, the Jevons paradox occurs when technological progress increases the efficiency with which a resource is used, but the falling cost of use induces increases in demand enough that resource use is increased, rather than reduced.
Yes but it will also mean that people wouldn't need cutting edge NVIDIA chips - it will be able to run on older node chips. Or from different manufacturers. So NVIDIA wouldn't be able to command the margins they do now.
It may be great news for VRAM manufacturers tough.
While everything you say may be true, this shows a fundamental misunderstanding about how the modern stock market functions. How much value a company creates is at best tangential and often completely orthogonal to how much the stock is worth. The stock market has always been a Keynesian beauty contest, but in the past few decades, strongly shaped and morphed by attention economies. A good example of this is DJT, a company which functionally doesn't do much anything, but in the last year has traded at wildly differing prices. P/E, EBITDA, etc, are all useless metrics in trying to explain this phenomenon.
In other words, NVIDIA is in the red not because the company is suddenly doing worse, but because traders think other traders think it will trade down. That is a self fulfilling prophecy, but only so long as there is sufficient attention to drive that. The same works the other way around as well, so long as there is sufficient attention to drive the AI hype train upwards, related stocks will do well as well.
>In other words, NVIDIA is in the red not because the company is suddenly doing worse, but because traders think other traders think it will trade down.
Well put. People need to unterstand that some stocks are basically one giant casino poker table. There was a comment with a link here that a lot of Nvidia buyers don't even know what products Nvidia is making and they don't care, they just want to buy low and sell high. Insert old famous comment abut shoe shine boy giving investment advice to Wall Street stock traders.
It's a reflection of expectations about the future economy. Obviously, such expectations are not always accurate because humans are quite fallible when trying to predict the future. This is even more true when there is a lot of hype about a certain product.
Yesterdays price of (say) NVidia was based on the expectation that companies would need to buy N billion of USD of GPUs per year. Now Deepseek comes out and makes a point that N/10 would be enough. From there it can go two ways:
- NVidia's expected future sales drop by 90%.
- The reduced price for LLMs should allow companies to push AI into markets that were previously not cost effective. Maybe this can 10x the total available market, but since the estimated total available market was already ~everything (due to hype) that seems unlikely.
- NVidia finds another usecase for GPUs to offset the reduced demand from AI companies.
In practice, it will probably be some combination of all three. The real problems are not caused for the "shovel sellers" but for companies like OpenAI and Anthropic, who now suddenly have to compete against a competitor that can produce the same product at (apparently) a fraction of the price.
I think it is more expectation about expectation. You buy/sell based on whether you expect other people to expect earn or lose. It is self-referential, hence irrational. If a new play enters and peoples expectations shift, that affects your expectation of value even though the companies involved are not immediately or directly affects.
As already mentioned elsewhere, Jevon's Paradox will increase demand subsequent to improved efficiency. Yes, will not can.
So if the stock market was reflective of the economy (future or the present) then stocks should go up, instead they're going down. Why? Because the stock market is not reflective of the economy.
The stock market is essentially a reflection of societal perception. DJT which was brought up earlier is a great example, because the price of DJT has next to nothing to do with Trump's businesses and almost everything to do with how he is perceived (and remember there is no such thing as bad publicity).
Personally I think the fall will be momentary and followed shortly by a climb to recovery and beyond, but who really knows.
If you don't want to lose your money: Don't let the sensationalist financial journalists and pundits get to you, don't let big red numbers in your portfolio scare you, ignore traders (they all lose their money), don't sell your stocks unless you actually need that money for something right now, re-read your investment manifesto if you have one, and maybe buy the dip for shits and giggles if you have some spare cash laying around.
I agree that it will improve demand for AI services. There's no hard rule that the demand increase will be larger than the efficiency increase though, and so total sales of GPUs may still decrease as a result.
> OpenAI and Anthropic, who now suddenly have to compete against a competitor that can produce the same product at (apparently) a fraction of the price.
OpenAI and Anthropic can react by adopting DeepSeek's compute enhancements and using them to build even better models. AI training is still very clearly compute-limited from their POV (they have more data than they know what to do with already, and training "reasoning"/chains-of-thought requires a lot of reinforcement learning which is especially hard) so any improvement in compute efficiency is great news no matter where it comes from.
You can buy the dip if you want, as long as you're aware that you're not betting that "stupid" traders are undervaluing NVIDIA's fundamentals. Rather, you're betting that "stupid" traders will again rally NVIDIA's share price significantly above this dip, and you will be a smart trader who will know when to sell what you bought. Good luck.
And not just that, but even if AI's future is indeed as bright as the hype says (i.e. that NVIDIA's fundamentals are solid & that the market will eventually acknowledgment that after the fluctuations) they may still be wrong about the timeline.
In the .com bust you could have "bought the dip" in the early 00s right after the crash started and still taken 5 years before you weren't in the red even on "good" (in hindsight) stocks like amazon, ebay, microsoft, etc. The big hype there was eCommerce - it turned out to be true! We use eCommerce all the time now, but it took longer than predicted during the .com boom (same for broadband internet enabling "rich web experience" - it came true, but not fast enough for some hyped companies in '00).
And if you bought some of the darling stocks back then like Yahoo or Netscape that ended up not so great in hindsight you may have never recouped your losses.
It's not just the usual herding behavior though. There's a convex response to news like this because people look at higher order effects like the growth of growth for stocks. Basically the DeepSeek story is about needing 40x fewer compute resources to run inference if their benchmarks are true. The dip doesn't mean that NVidia is now doomed, it simply means that if DeepSeek is legit, you need much less NV hardware to run the same amount of inference as before. Will the demand rise to still use up all the built hardware? Probably, but we went from a very stratospheric supply constraint to a slightly less stratospheric one, and this is reflected in the prices. Generally these moves are exaggerated initially, and it takes a bit of time for people to digest the information and the price to settle. It's an oscillating system with many feedback loops.
As someone who bought NVDA in early 2023 and sold in late 2024 I can say this is wrong.
There was never a question of if NVDA hardware would have high demand in 2025 and 2026. Everyone still expects them to sell everything they make. The reason the stock is crashing is because Wall St believed that companies who bought 50B+ of NVDA hardware would have a moat. That was obviously always incorrect, TPUs and other hardware was eventually going to be good enough for real world use cases. But Wall St is run by people who don't understand technology.
Loving the absolute 100% confidence there and the clear view into all the traders' minds that are trading it this morning.
If they'll sell everything they make and it's all about the moat of their clients, why is NVDA still down 15% premarket? You could quote correlation effects and momentum spillover, but that is still just the higher order effects I mentioned about people's expectations being compounded and thus reactions to adverse news being convex.
Presumably because backorders will go down, production volume and revenue won't grow as fast, Nvidia will be forced to decrease their margins due to lower demand etc. etc.
Selling everything you make is an extremely low bar relative to Nvidia's current valuation because it assumes that Nvidia will be able to grow at a very fast pace AND maintain obscene margins for the next e.g. ~5 years AND will face very limited competition.
That's literally what I wrote in my post, which the parent disagreed with. You could disagree with the part that it is because inference is now cheaper - but again I'd argue that's just a different way of saying there's no moat.
People owned NVDA because they believed that huge NVDA hardware purchases was the ONLY way to get a AI replacement for a Mid Level software engineer or similar functionality.
That's basically what I wrote: "it simply means that if DeepSeek is legit, you need much less NV hardware to run the same amount of inference as before."
So I still don't understand what it is that you are so strongly disagreeing with, and I also don't understand how having owned NVidia stock somehow lends credence to your argument.
We are in agreement that this won't threaten NVidia's immediate bottom line, they'll still sell everything they build, because demand will likely rise to the supply cap even with lower compute requirements. There are probably a multitude of reasons why the very large number of people who own NVidia stock have decided to de-lever on the news, and a lot of it is simple uneducated herding.
But we are fundamentally dealing with a power law here - the forward value expectations for NVidia have exponential growth baked in to the hilt, combined with some good old fashioned tulip mania, and when that exponential growth becomes just slightly less exponential, that results in fairly significant price oscillations today - even though the basic value proposition is still there. This was the gist of my comment - you disagree with this?
Up until recently there was a belief by some investors that OpenAI was going to "cure cancer" or something as big as that. They assumed that the money flowing into OpenAI would 10x, under the assumption that no one else could catch up with them after that event and a lot of that would flow to NVDA.
Now is looks like that 10x of flow of money into OpenAI will no longer exist. There will be competition and compodiditzation, which causes the value of the tokens to drop way more than 40x.
I think the message everyone now accepts is: "there is no moat". It is plain stupid to think big models can be magically copy-protected - they are simply arrays of numbers and all components one need to create such arrays are free and well established. This is unlike the whole infrastructure, processes, social connections, hardware and storage, one need say to recreate a service like YouTube or Facebook. Large models are different - you don't need all of that - the future of LLMs is Open Source like Linux.
Everything above the street level and physical economy is becoming gambling.
There has always been a component of gambling to all investing, but that component now seems to utterly eclipse everything else. Merit doesn’t even register. Fundamentals don’t register.
I think it's because the media coverage is all focused on how this means the big AI players have lost their competitive advantage, rather than the other side of the equation.
But that's also dumb, because "huge leap forward in training efficiency" is not exactly bad news for the major players in even the medium term. Short term, it means their models are less competitive, but I don't see any reason that they can't leverage e.g. these new mixed precision training techniques on their giant GPU farms and train something even bigger and smarter.
There seems to be this weird baked in assumption that AI is at a permanent (or at least semi-permanent) plateau, and that open source models catching up is the end of the game. But this is an arms race, and we're nowhere near the finish line.
Good time to buy is % of income via tax efficient methods into the in SP500 for most.
But I agree in the sense that Deepseek just creates more demand. Because people desire to get AI to do more work. This makes bang for buck greater opening new opportunities.
This sell off is like selling Intel in 2010 because of a new C compiler.
maybe Nvidia is fine but I don't understand this logic. suppose it turns out GPUs are unnecessary at all, but they still provide a performance boost, but you can do everything you can do right now with CPUs w.r.t performance. would that be good or bad for Nvidia?
unless it can be said we need more performance than is currently possible, e.g. new demand, it would be catastrophic. it is unclear that throwing more compute actually expands what is possible. if that is not the case, efficiency is bad for nvidia because it simply results in less demand.
I could see arguments be made in both ways here. If GPUs end up being more efficient/powerful (like today) it could induce even more demand, but also if CPU gets within ~20% of how fast you can do something with a GPU, people might start opting for something like Macs with unified memory instead of GPUs.
Today a CPU setup is still nowhere near as fast as a GPU setup (for ML/AI), but who knows how it looks like in the future.
> it is unclear that throwing more compute actually expands what is possible
Wasn't that demonstrated to be true already in the GPT1/2 days? AFAIK, LLMs became a thing very much because OpenAI "discovered" that "throwing more compute (and training data) at the problem/solution expands what is possible"
> I don’t understand how stupid some traders can be.
90% of traders lose money, so that's a data point...
You're trying to apply rational thinking but that's not how markets work. In the end valuations are more about narratives in the collective mind than technological merit.
You answered your own question. People do not dig in the Sacramento right anymore for gold, because, it is gone. If you can train models for 1/100 the cost, and you sell model training chips, you probably are not going to sell as many chips.
That's why the shovel maker from back then are selling mining machines today.
Everyone here thinks Nvidia is dommed because of training efficiency.
But what has Nvidia been doing for the past decade? Correct increasing training and inferencing efficiency by magnitudes.
Try to train GPT4 on 10k of Volta, Ampere, Hopper and then Blackwell.
What has happened since then? Nvidia has increased their sales in magnitudes.
Why? Because thanks to improvement in data, in algorithms, compute efficiency ChatGPT was possible in the first place.
Imagine Nvidia wouldn't exist. When do you think the ChatGPT moment would happen on CPUs? LOL
Going back to my first sentence. Nvidia started also with small shovels which were GeForce cards with CUDA. Today Nvidia is selling huge GPU clusters (mining machines, yes pun intended ^^).
nVidia is also about HPC in general, not just AI. It's remarkably silly that the stock would plunge 13% just because someone made a more compute-efficient LLM.
> Good time to buy then, I don’t understand how stupid some traders can be.
Likely a "how solid is the technical moat" evaluation - this could be a one-off or could be that there are an avalanche of advancements to continue along the efficiency side of the process.
Given the style and hype of logic in the AI space, I fully believe resources are not well allocated in compute and _actual_ thinking as to how they are spent.
Deepseek's apparent 10x more efficient per inference token... implies a lot of other hardware meets the general use-case. We also know that reasoning should be about 10W for human speed-of-thought... maybe another 1-2 orders of power efficiency.
"Pre-Training: Towards Ultimate Training Efficiency
We design an FP8 mixed precision training framework and, for the first time, validate the feasibility and effectiveness of FP8 training on an extremely large-scale model.
Through co-design of algorithms, frameworks, and hardware, we overcome the communication bottleneck in cross-node MoE training, nearly achieving full computation-communication overlap.
This significantly enhances our training efficiency and reduces the training costs, enabling us to further scale up the model size without additional overhead.
At an economical cost of only 2.664M H800 GPU hours, we complete the pre-training of DeepSeek-V3 on 14.8T tokens, producing the currently strongest open-source base model. The subsequent training stages after pre-training require only 0.1M GPU hours." [1]
There is a point where there are enough shovels circulating that the demand for new shovels falters, even with zero drawback in the rush. And if so much gold was being mined that it overwhelmed the market and reduced the commodity price, the value of better shovels is reduced.
DeepSeek and friends basically reduce the commodity value of AI (and to be fair, Facebook, Microsoft et al are trying to do the same thing with their open source models, trying to chop the legs out of the upstart AI cos). If AI is worth less, there are going to be fewer mega capitalized AI ventures buying a trillion dollars worth of rapidly-depreciating GPUs in hopes at eeking out some minor advantage.
I wouldn't short nvidia stock, but at the same time there is a point where the spend of GPUs just isn't rational anymore.
>And as more efficient models proliferate it means more edge computing which means more customers with lower negotiating power than Meta and Google
Edge compute has infinitely more competition than the data center.
Agree. Apple should be as well. The only con I can think of would be their (Meta's) data center investments but seems this will make them more efficient?
Not sure I understand how exactly open source plays a key role here in terms of project development.
Looking at https://github.com/deepseek-ai, those repos have a bunch of of contributors but unless I'm wrong I don't see any significant contributions. What am I missing?
One could argue that by Meta and other companies releasing open weights and detailed got us to where we are now with R1. Even if it wasn't your race car that crossed the line first, everyone can now get a copy.
I'm not so sure about that. Deepseek puts their LLM (Llama) even further behind. It's basically at the back of the pack, signaling to the market that they don't have the top minds in the industry on board. Second, I'm not sure how a massive trove of misinformation is of much use or how it's of more use to them than it is to others. Can you elaborate on that?
What kind of quality of content can Meta theoretically mine with all the vast data that they collect? They have a good advertising and a content algorithms but does it approximate general intelligence? And the content you see on Facebook, Instagram, Whatsapp, that's just a lot of average content that might actually be great for AGI.
The $500 billion data center can now be $50 billion. That is excellent news, unless you were the company that was expected to sell $450 billion of GPUs with a 95% gross margin to that project.
$50 billion can be afforded by WAY WAY more sites and companies so Nvidia will simply delivered to >10x more data centers. Or instead of shipping 100k GPUs to Meta, they will ship 100k GPUs to 10 different customers.
For Nvidia, it is great news because now finally, the concentration of GPUs at Hyperscalers will end and every Fortune company can finally get their local data center to train their AI Models.
Because if training AI models becomes more efficient and easier then the ones being that business are at risk so basically Big Tech. Nvidia isn't in that business but in the business of providing tools to train.
Fortunately, Big Tech can easily do something to prevent ANYONE for competing. They simply buy all available GPUs. Oh wait, haven't they been doing it for years? Excatly!
People really don't get what an arms race and market competition race is.
How do I prevent disruption? I simply buy all the tools the competition needs to disrupt me.
See, if Fortune 500 companies want to build large data centers but can't because all Hyperscalers buy the GPUs then eventually they will rent from cloud as otherwise they can't get the GPUs.
> $50 billion can be afforded by WAY WAY more sites and companies
Spending $50 billion to do $6 million worth of Ai training seems like a good way to trigger a golden parachute and "spend more time with your family" as a CEO.
Inference can be more easily offloaded to other kinds of processing units, which also probably are more efficient, like TPUs.
That makes both NVDA stock and big AI infrastructure spending less compelling, as those needs are scaled down via software efficiency and chip alternatives.
The whole AI valuation was based on being able to rent-seek a significant chunk of all white collar salaries, with a permanent monopoly moat because nobody else would pay hundreds of billions to train models.
And yet again a cheaper Chinese product turns up and everyone loses their minds. Expect a ban incoming to preserve the AI valuations.
Banning wouldn't work imo - now that the cat's out of the bag, with the architecture being open source, anyone can replicate their results to a compete with them for a relatively small investment.
The people writing market commentary are simply making it up. The news about DeepSeek is not new and doesn't reduce the value of ASML. People are selling now because they are scared because the number went down.
Andrej Karpathy was tweeting about DeepSeek a month (!) ago.
"DeepSeek (Chinese AI co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget (2048 GPUs for 2 months, $6M)."
Yes exactly. The actual impetus this time was the article I posted here and how it got echoed and amplified by massive X accounts like Chamath and Naval.
Yes! and this applies to all market commentary. Market goes down .7% and you have talking heads saying "fear of tariffs" "middle east tensions" "Hurricane season" whatever, next day market goes up .6% "talk of tax cuts" "Jobs numbers" whatever. There's no way anyone knows why "the market" behaves the way it does. The free market is the OG "Decentralized" project, it's 1 billion different decisions being made in a day each with their personal reasons. Yes, sometimes it's fairly obvious that something caused it (plane blows up, stock goes down) but that still doesn't explain the entirety of it
ASML paper value is determined by equipment sales from projected compute supply/demand. CHIPS building redundant global fabs = glut from more excess capacity = less future sales. Stargate = excess demand from everyone spending 100s of billions of compute = need even more fabs = more future sales. Then DeepSeek = suddenly no need for that much future compute... if number of future compute demand relative to short term fab overcapacity is going down, then it's reasonable to sell. Relatively predictable Semiconductor market cycles due to cost of capex and time to build fabs / increase new wafers output to match future demand is a thing.
This would be an excellent explanation if DeepSeek had announced its model over the last weekend rather than weeks ago, and if R1 wasn't a COT reasoning model which needs a lot more inference time compute than other SOTA models like llama.
Information lag, especially with respect to PRC developments and technical developments. Taking 1-2 week for info to be shared and passed down info chain not unusual. IRC COT typically increase inference 1-5x depending on task complexity, i.e. instead of scaling down compute demand by 50x, it's 10x, which is still substantial. Could investors be panicking? Sure, but there's rational basis for doing so.
DeepSeek V3 is a 671B parameter MOE model? I am not sure why it's 50x cheaper at inference time than other models. We don't know what the cost of running o1 is, but I doubt it has 50x as many params as R1. Most the advantages of MOE are reduced when using reasonable branch sizes so that wouldn't make R1 cheaper in practice either. I think people might be seeing a lower markup from DeepSeek and confusing it with cheaper inference?
If DeepSeek is side/pet project for PRC quants whose fine with subsisting on low markup then that's market price competitors have to calibrate return on investment and future capex. DeepSeek also appear open and performative enough to drive cheaper inference on commodity hardware with very different margins for variety of use cases, including existing hardware. At least short term there's going to be % of LLM use cases that has to compete on China prices or previously considered depreciated hardware. IMO snowballing interests undoubtly getting investors to pay attention and deep dive to related developments, i.e. Bank of China's 1T AI fund, and DeepSeek CEO just met PRC premiere a few days ago. AFAIK DeepSeek hasn't ever gotten this much domestic political attention before, they're potentially going to be elevated as domestic champion and it's likely to open a lot of more doors, i.e. significantly more compute. Hard to tell how that will effect western models business models short term.
Nothing. Apple (and Meta) are not directly impacted by AI even if the cost of training these AI models becomes cheaper and $0 free models get better.
It actually affects the frontier AI companies (OpenAI, Anthropic, etc) who directly make money from their closed models AND spend hundreds of millions on training these models.
Why pay $3/per million tokens (Claude 3.5 Sonnet) when DeepSeek R1 offers $0.14 / per million tokens and the model is on par with (OpenAI o1) and R1 itself is released for free?
$0 free AI models are eating closed AI models lunch.
Nothing, and I think that was the point GP was making. Since not much of Apple's valuation is tied to AI hype they won't suffer (at least not close to as much) from the bubble bursting.
I don't think there has to be an AI bubble, but valuations overall have to come down to something in accordance with the interest rates and expected long-term profit rates.
Wow, literally bought more Nvidia shares last week. Just goes to show the stock market is 80% gambling. Me believing in the value of a company is overshadowed by the hype of growth and "future valuation".
If this is gambling, what isn’t gambling? Sometimes things just don’t go the way you want.
A Chinese company coming up with a cheaper alternative to a cutting-edge technology out of nowhere, is an outcome that is hard to predict.
In hindsight, betting on Nvidia maintaining its monopoly on a resource crucial for such an important technology as AI, might not be the best of ideas, but then again, who knows.
I find it interesting because the DeepSeek stuff, while very cool, doesn't seem invalidate that more compute wouldn't translate to even _higher_ capabilities?
It's amazing what they did with a limited budget, but instead of the takeaway being "we don't need that much compute to achieve X", it could also be, "These new results show that we can achieve even 1000*X with our currently planned compute buildout"
But perhaps the idea is more like: "We already have more AI capabilities than we know how to integrate into the economy for the time being" and if that's the hypothesis, then the availability of something this cheap would change the equation somewhat and possibly justify investing less money in more compute.
The stock market is not the economy, Wall Street is not Main Street. You need to look at this more macroscopically if you want to understand this.
Basically: China tech sector just made a big splash, traders who witnessed this think other traders will sell because maybe US tech sector wasn't as hot, so they sell as other traders also think that and sell.
The fall will come to rest once stocks have fallen enough that traders stop thinking other traders will sell.
Investors holding for the long haul will see this fall as stocks going on sale and proceed to buy because they think other investors will buy.
Meanwhile in the real world, on Main Street, nothing has really changed.
Bogleheads meanwhile are just starting the day with their coffee, no damns given to the machinations of the stock market because it's Monday and there's work to be done.
s&p500 was still up by normal amounts during 2023 and 2024 if you exclude big tech. definitely they are an outsize portion of the index but that doesn't mean the rest of the economy isn't growing. https://www.inc.com/phil-rosen/stock-market-outlook-sp500-in...
Is it really related to China's tech sector as such, though? If this is true then Openai, Google or even many magnitudes smaller companies etc. can just easily replicate similar methods in their processes and provide models which are just as good or better. However they'll need way less Nvidia GPUs and other HW to do that than when training their current models.
> doesn't seem invalidate that more compute wouldn't translate to even _higher_ capabilities?
That's how i understand it.
And since their current goal seems to be 'AGI' and their current plan for achieving it seems to be scaling LLMs (network depth wise and at inference time prompt wise), i don't see why it wouldn't hold.
Probably not. If the price of Nvidia is dropping, it's because investors see a world where Nvidia hardware is less valuable, probably because it will be used less.
You can't do the distill/magnify cycle like you do with alphago. LLM models have basically stalled in their base capabilities, pre training is basically over at this point, so the news arms race will be over marginal capability gains and (mostly) making them cheaper and cheaper.
But inference time scaling, right?
A weak model can pretend to be a stronger model if you let it cook for a long time. But right now it looks like models as strong as what we have aren't going to be very useful even if you let them run for a long, long time. Basic logic problems still tank o3 if they're not a kind that it's seen before.
Basically, there doesn't seem to be a use case for big data centers that run small models for long periods of time, they are in a danger zone of both not doing anything interesting and taking way too long to do it.
The AI war is going to turn into a price war, by my estimations. The models will be around as strong as the ones we have, perhaps with one more crank of quality. Then comes the empty, meaningless battle of just providing that service for as close to free as possible.
If Openai's agents panned out we might be having another conversation. But they didn't, and it wasn't even close.
This is probably it. There's not much left in the AI game
We don't know for example what a larger model can do with the new techniques DeepSeek is using for improving/refining it. It's possible the new models on their [own] failed to show progress but a combination of techniques will enable that barrier to be crossed.
We also don't know what the next discovery/breakthrough will be like. The reward for getting smarter AI is still huge and so the investment will likely remain huge for some time. If anything DeepSeek is showing us that there is still progress to be made.
Pending me getting an understanding of what those advances were, maybe?
But making things smaller is different than making them more powerful, those are different categories of advancement.
If you've noticed, models of varying sizes seem to converge on a narrow window of capabilities even when separated by years of supposed advancement. This should probably raise red flags
Your implication is that we have unlimited compute and therefore know that LLMs are stalled.
Have you considered that compute might be the reason why LLMs are stalled at the moment?
What made LLMs possible in the first place? Right, compute! Transformer Model is 8 years old, technically GPT4 could have been released 5 years ago. What stopped it? Simple, the compute being way too low.
Nvidia has improved compute by 1000x in the past 8 years but what if training GPT5 takes 6-12 months for 1 run based on what OpenAI tries to do?
What we see right now is that pre-training has reached the limits of Hopper and Big Tech is waiting for Blackwell. Blackwell will easily be 10x faster in cluster training (don't look on chip performance only) and since Big Tech intends to build 10x larger GPU clusters then they will have 100x compute systems.
Let's see then how it turns out.
The limit on training is time. If you want to make something new and improve then you should limit training time because nobody will wait 5-6 months for results anymore.
It was fine for OpenAI years ago to take months to years for new frontier models. But today the expectations are higher.
There is a reason why Blackwell is fully sold out for the year. AI research is totally starved for compute.
The best thing for Nvidia is also that while AI research companies compete with each other, they all try to get Nvidia AI HW.
The age of pre-training is basically over, I think everyone acknowledged this and it's not to do with not having a big enough cluster. The bull argument on AI is that inference time scaling will pull us to the next step
Except o3 benchmarks are, seemingly, pretty solid evidence that leaving LLM'S on for the better part of a day and spending a million dollars gets you... Nothing. Passing a basic logic test using brute force methods and which falls apart on a marginally easier test that it just wasn't trained on.
The returns on computer and data seem to be diminishing with more and more exponential increases in inputs returning geometric increases in quality, and we're out of quality training data so that is now much worse even if the scaling wasn't plateauing.
All this, and the scale that got us this far seems to have done nothing to give us real intelligence, there's no planning or real reasoning and this is demonstrated every time it tries to do something out of distribution, or even in distribution but just complicated. Even if we got another crank or two out of this, we're still at the bottom of the mountain here. We haven't started and we're already out of gas
Scale doesn't fix this any more than building a mile tall fence stops the next break in. If it was going to work we would have seen to work already. LLM's don't have much juice left in the squeeze, imo
if you've tried to get o1 to give you outputs in a specific format, it often just tells you to take a hike. It's a stubborn model, which implies a lot
This is speculation, but it seems that the main benefit of reasoning models is that they provide a dimension along which RL can be applied to make them better at math and maybe coding, things with verifiable outputs.
Reasoning models likely don't learn better reasoning from their hidden reasoning tokens, they're 1) trying to find a magic token which when raised to its attention make it more effective (basically give it room to say something that jogs its memory) or 2) it is trying to find a series of steps which do a better job of solving a specific class of problem than a single pass does, making it more flexible in some senses but more stubborn along others
Reasoning data as training data is a poison pill, in all likelihood, and just makes a small window of RL vulnerable problems easier to answer (when we have systems that don't better). It doesn't really plan well, doesn't truly learn reasoning, etc
Maybe seeing the actual output of o3 will change my mind but I'm horrifically bearish on reasoning models
It really doesn't lol. Those laws are like Moore's law, an observation rather than something Fundamental like laws in physics
The scaling has been plateauing, and half that equation is quality training data which is totally out at this point.
Maybe reasoning models will help produce synthetic data but that's still to be seen. So far the only benefit reasoning seems to bring is fossilizing the models and improving outputs along a narrow band of verifiable answers that you can do RL on to get correct
Synthetic data maybe buys you time, but it's one turn of the crank and not much more
He seems adamant that there are no diminishing returns to scaling AI.
I don’t want to stir up conspiracy theories but I do think that currently all the big AI players have a vested interest in the message that the current scaling paradigm is the right one, and that this is a supremacy issue wrt China. It drives so much investment and valuation that I doubt they can truly be objective.
> I don’t want to stir up conspiracy theories but I do think that currently all the big AI players have a vested interest in the message that the current scaling paradigm is the right one, and that this is a supremacy issue wrt China. It drives so much investment and valuation that I doubt they can truly be objective.
500 Billion is a lot of money. Expect even crimes to be commited in order to make it happen.
The issue is it feels like we came to a stop but Hyperscalers are simply waiting for Blackwell. That's all.
Why buy 100k Hoppers if 20k Blackwell offer the same compute so then it's better to buy 100k Blackwells right?
Backwell will increase cluster scaling easily by 10x performance and if you buy 10x of them then your compute on a cluster will be 100x than before. If it takes you to wait 6-12 months for that then so be it. You will easily make up the time in the end with the speed up.
I think part of the problem was many/most previous models coming out of China were trained on eval data to cheat in the rankings. Quite an uphill battle for Deepseek.
Assuming that is true, which I don't have a reason to believe otherwise, only gave an excuse to the average reader to disregard without reading more than the title.
You can see a similar bias in academia with work originating outside EU/USA.
before someone thinks something strange regarding me, I can only tell you I'm not chinese, but Argentinian :)
Not sure what the fuss is. I tried Deepseek earlier today for the first time and it was even worse than o1 when it came to reasoning skills and following my requests for how I wanted to engage with it.
o1 at least gives it to me straight. When I ask it to engage in more back and forth before assuming what I'm after, it tends to follow through. Deepseek seemed immediately eager to (very slowly) feed me a bunch of made up information thinking that's what I wanted.
I feel as though a lot of people get hung up on these sort of "micro benchmarks" whereas trying to get practical work done is severely under tested. I'm not a fan of openai at all but I don't have the spare compute to run anything locally so o1 suffices for now.
Still don't see how this is anything but a win for Nvidia though.
The excitement isn't the capabilities of the model, it's how efficiently it was created. One of the major lessons in AI in the last couple of years was that scale mattered - you would want to throw more and more compute at a problem and that has turned into incredible share prices for Nvidia and incredible investments in data centre and energy generation. If it turns out that actually we didn't need quite such incredible scale to get these results and actually we were just missing some really quite basic efficiency optimizations then the entire investment cycle into Nvidia, data centres and energy generation is going to whipsaw in an incredible way.
Essentially, Deepseek is showing that there is a lot of room for improvement with AIs. To paraphrase Orwell, AIs are a lot more like Alarm Clocks and a lot less like Manhattan Projects.
o1 does not show the reasoning trace at this point. You may be confusing the final answer for the <think></think> reasoning trace in the middle, it's shown pretty clearly on r1.
I wasn't really referring much to the UI as I was the fact that it does it to begin with. The thinking in deepseek trails off into its own nonsense before it answers, whereas I feel openai's is way more structured.
Reassessing directives
Considering alternatives
Exploring secondary and tertiary aspects
Revising initial thoughts
Confirming factual assertions
Performing math
Wasting electricity
... and other useless (and generally meaningless) placeholder updates. Nothing like what the <think> output from DeepSeek's model demonstrates.
As Karpathy (among others) has noted, the <think> output shows signs of genuine emergent behavior. Presumably the same thing is going on behind the scenes in the OpenAI omni reasoning models, but we have no way of knowing, because they consider revealing the CoT output to be "unsafe."
R1 is the first model I've used that one-shotted a full JavaScript tetris with all the edge-case keyboard handling and scoring. It also one-shotted an AI snake game. With the right prompts I've found it consistently better than o1 and Claude 3.5 Sonnet.
This is an example of a recurrent phenomenon in politics. Trump comes in with big plans, this and that, threatening countries, deporting people, a clear agenda to take America to the hard right. Isolationism, crush China, screw every other country over, etc. etc.
And then - something completely unexpected, a total curveball - arrives a week into office and everything changes. Your agenda collapses and you enter reactive mode.
AI bubble could pop. Valuations drop. Crypto might get hit. Where is Project Stargate now? It might become a joke.
A more cynical observer might suggest that _the timing was no coincidence_.
I said in other threads about how the establishment passed Trump a ticking time bomb – if he were smart, he should've made it blow up early himself to avoid becoming the next Herbert Hoover
> Your agenda collapses and you enter reactive mode.
Where has Trump's agenda collapsed? I might have missed the press release.
And why would a curveball on AI throw off an agenda around trade, immigration and military engagement? I don't follow.
China could take the lead on AI and I don't see it would impact any of those things. Isn't DeepSeek open source? The US already has access to it, so what leverage could China possible have?
> A more cynical observer might suggest that _the timing was no coincidence_.
Do you feel the Chinese government closely controls AI research and timed this response?
> Where has Trump's agenda collapsed? I might have missed the press release.
Well, it's too early to say if it has happened in this case. But we've seen it happen again and again, so it will not be a surprise if it happens.
> And why would a curveball on AI throw off an agenda around trade, immigration and military engagement? I don't follow.
Well, trade restrictions against China have just backfired spectacularly. So further trade restrictions may not seem as good an idea. And to start trade wars, you need a strong economy. And at the moment America's economy is entirely driven by the AI bubble, as it is the value of the AI stocks that has separated the economy from the European trend.
It is very likely that military engagement will be driven by, or affected by, developments in AI. And there's no doubt that Taiwan's situation is heavily affected by chip production.
> Do you feel the Chinese government closely controls AI research and timed this response?
No evidence, but I am not naive enough to think it's out of the range of possibility. I don't follow your reasoning, more likely the timing of the release is all they had to modify - which is quite trivial for any government. To not question that would be very naive. They have _every motive_.
> Well, it's too early to say if it has happened in this case
Ah ok, so you’re just guessing. You wrote it as if it had already happened.
And I think you’re confusing the stock market with the broader economy. The economy as a whole is unaffected by AI at this point.
And the trade war with China involved hundreds of industries beyond AI. Most of it is manufacturing. I’m not sure how failure of AI sanctions (questionable conclusion) somehow means the trade issues around machine parts needs to be abandoned.
I agree China has motive but I’ve heard so many claims that the Chinese government doesn’t control research or businesses like TikTok.
> Ah ok, so you’re just guessing. You wrote it as if it had already happened.
That's not a generous reading. I'm saying this is what has happened historically.
Edit: the use of the words "might" and "could" made this pretty clear.
> The economy as a whole is unaffected by AI at this point.
True in terms of no marked effect on GDP. But the stock market and the broader economy are very much linked. See 2008. And the stock market is high on AI.
> I’ve heard so many claims that the Chinese government doesn’t control research or businesses like TikTok.
Even western countries tell tech companies what to do. Do you think an authoritarian government is going to do that more or less? There's a reason they didn't want to sell TikTok.
H800s were made to match Biden's export restrictions. They were banned in late 2023 but a lot were sold to China. Having 2k is quite small compared to the bigger players like BABA (200k employees) and Tencent (100k employees). And those sure have access to the few H100s that were smuggled. But unlikely for a tiny company like High-Flyer/Deepseek (160 employees).
1,112 comments
[ 3.2 ms ] story [ 470 ms ] threadValuations of private unicorns like OpenAi and Anthropic must be in free fall. DeepSeek spends $6 million in old H800 hardware to develop open source model that overtakes ChatGPT. AI gets better, but profit margins sink with strong competition.
Chinese AI startup DeepSeek overtakes ChatGPT on Apple App Store https://news.ycombinator.com/item?id=42839656
Edit: Nvidia now -15% in Frankfurt.
Agree that the AI bubble should pop though and the earlier, the better.
Even if they are heavily government subsidized for energy and hardware, I don see how the cost of training in the US would be more than double.
Not saying they are lying, but there incentives.
Everyone’s already begun trying this recipe in-house. Either it works with much less compute, or it doesn’t.
For instance, HKUST just did an experiment where small weak base models trained with DeepSeek’s method beat stronger small base models being trained with much more costly RL methods. Already this seems like it is enough to upend the low end models niche market, things like haiku and 4o-mini.
Be really skeptical why the people who should be making tons of money by realizing actually it was all a mirage and that they can now get the real stuff for even cheaper, would spend so much effort shouting about this, in order to undercut their own profitability..
tl;dr all numbers check up and the winnings come from the model architecture innovations they made.
Is it possible that they based their model architecture on the llama model architecture? Rather than just fine-tuned already training llama weights? In that case, they'd still have to do "bottoms up" training.
DeepSeek claims that's what they spent. They're under a trade embargo, and if they had access to any more than that it would have been obtained illegally.
They might be telling the truth, but let's wait until someone else replicates it before we fully accept it.
You can't buy them as "GPU"s and integrate them to your system. NVIDIA sells your the platform (GPUs + platform board which includes switches and all the support infra), and you integrate that behemoth of a board to your server, as a single unit.
So that open server and the wrapped ones at the back are more telling than it looks.
Replications of small models indicate that they don't lie any significant amount. The architecture is cheap to train.
Berkeley Researchers Replicate DeepSeek R1's Core Tech for Just $30: A Small Model RL Revolution https://xyzlabs.substack.com/p/berkeley-researchers-replicat...
I remember a year ago I was hoping that in a decade from now it would be great to run GPT4-class models on my own hardware. The reality seems to be far more exciting.
Asking as someone who honestly only superficially followed the developments since the end of 2023 or so
PRC companies breaking US export control laws is legal (for PRC companies). Maybe they're trying to avoid US entity listing, lot's of PRC companies keep mum about growing capabilites to do so. But the mere fact Deepseek is publicizing means they're unlikely to care about the political heat that is coming and the ramifications. If anything, getting on US entity list probably locks in their employees with Deepseek on resume into PRC.
So long as they don't plan to do any business with the US or any of their allies I guess.
Throwing this model out also gives US allies soverign AI a launchpad... reducing US dependency is step 1 to not being US allies.
They already are. You can make a paid account and use their API from most countries around the world. This is what doing business looks like.
I actually hope he doubles down. I would love for EU to rely less on the US. It would also reduce the reach of the silly embargoes that benefit no one but the US.
For instance if the law bans US companies from exporting/selling some chips to Chinese companies and that's it then it is unclear to me whether a Chinese company would do anything illegal under US law by buying such chips as it would be for the American seller to refuse.
Anyway, usually this sort of things takes place through intermediaries in third countries so it is difficult to track but obviously it would be stupid to brag about it if that happened.
But don't LLMs encode language, not facts?
> If there's no unauthorized reproduction/copying then it's not a copyright issue.
I'm pretty sure copyright holders have gotten the models to regurgitate their copyright works verbatim, or nearly so.
On the second point it depends how the models were made to reporduce text verbatim. If i copy-paste someone's article in MS word i technically made word reproduce the text verbatim., obviously that's not Word's fault. If i asked an LLM explicitly to list the entire Bee Movie script it would probably do it, which means it was trained on it, but that's through a direct and clear request to copy the original verbatim.
But it has to have that copy, verbatim, to produce it, as you acknowledge.
If dropbox was hosting and serving IP from paramount, paramount would be able to submit a DCMA request to get that data removed.
Not only can you not submit a DMCA request to chatGPT, they can't actually obey one.
But that clearly means that the LLM already has the Bee Movie script inside it (somehow), which would be a copyright violation. If MS word came with an "open movie script" button that let you pick a movie and get the script for it, that would clearly be a copyright violation. Of course if the user inputs something then that's different - that's not the software shipping whatever it is.
Huh? The "request" part doesn't matter. What you describe is exactly like if someone ships me a hard drive with a file containing "the entire Bee Movie script" that they were not authorized to copy: it's copyright infringement before and after I request the disk to read out the blocks with the file.
The issue is that DeepSeek have shown that you don't need that much raw computing power to run an AI, which means that companies including OpenAI may focus more on efficiency than on throwing more GPUs at the problem, which is not good news for those in the business of making GPUs. At least according to the market.
Doesn't look like it, because the some of the biggest US tech companies now active (including Meta and Alphabet) couldn't come up with what this much-smaller Chinese company has. Which begs the question, what is that companies like Meta, Alphabet and the like do with the (already) hundreds of billions of dollars that they invested in this space?
With that said, I think all of these companies are capable of learning from this and implementing these efficiency improvement. And I think the arms race is still on. The goal is to achieve super human level of intelligence, and they have a ways to go to get there. It is possible that these new efficiency improvements might even help them take the next step as they can now do a lot more with a lot less.
The cat is out of the bag and there is no going back.
[1] https://apxml.com/posts/gpu-requirements-deepseek-r1
Just for the people who might not have been around the last time, this has precedent :) US government (and others) have been trying to outlaw (open source) cryptography, for various reasons, for decades at this point: https://en.wikipedia.org/wiki/Crypto_Wars
The vast majority of what the US government has tried to ban was export of cryptography tools. However, as your own link makes clear, they stopped doing that in 2000.
Furthermore, what was restricted was not "open source cryptography"; it was cryptography that they could not break. The only way that open source comes into it is that that is what made it abundantly clear that the cat was out of the bag and there was no going back.
Hm. Kind of like this situation.
The restriction on TikTok was blatantly because it's a Chinese product outcompeting American products, everything else was the thinnest of smokescreens. Yes, I think people in favour of it are in favour of slapping whatever tariffs or bans they can get away with on everything that China makes.
I believe that NVIDIA is overvalued, but if DeepSeek really is as great as has been said, then it'll be even greater when scaled up to OpenAI sizes, and when you get more out you have more reason to pay, so this should if it pans out lead to more demand for GPUs-- basically Jevon's paradox.
After some readjustment we can expect AI companies to start using the new method to deliver more. Science fiction might happen sooner than expected.
Buy the dip.
If, as some companies claim, these models truly possess emergent reasoning, their ability to handle imperfect data should serve as a proof of that capability.
There are no perpetual motion machines.
Humans certainly did. We did not inherit our physics and poetry books from some aliens.
LLMs can not reason - many people seen to believe that they can.
My understanding is that the whole point of R1 is that it was surprisingly effective to train on synthetic data AND to reinforce on the output rather than the whole chain of thought. Which does not require so much human-curated data and is a big part of where the efficiency gain came from.
Simple question: where is GPT-5?
See, there is your answer. The issue is the compute of GPUs is way to low yet for GPT-5 if they continue parameter scaling as they used to do.
GPT3 took months on 10k A100s. 10k H100 would have done it in a fraction of a time. Blackwell could train GPT4 in 10 days with same amount of GPUs as Hopper which took months.
Don't forget GPT3 is just 2.5 years old. Training is obviously waiting for the next step up in large clusters of training speed increasement. Don't be fooled, the 2x Blackwell vs. Hopper is only chip vs. chip. 10k of Blackwell including all networking speedup is easily 10x or more faster than the same amount of Hopper. So building a 1 million Blackwell cluster means 100x more training compute compared to a 100k Hopper cluster.
Nobody starts a model training if it takes years to finish... too much risk in that.
Transfer model was introduced in 2017 and ChatGPT came out 2022. Why? Because they would have needed millions of Volta GPUs instead of thousands of Ampere GPUs to train it.
I wouldn't give what they say to much credence, and will only believe the results I see.
But it can still make sense for a state, even if it doesn't make sense for investors though.
Some consider this to be spurious/conspiracy.
Edit: I assumed that the model was distillation, that is apparently not true.
That's arguable, though. I mean it's much cheaper and reasonably competitive which is almost the same but IMHO DeepSeek seems to get stuck in random loops and hallucinates more frequently than o1.
That's gonna be a looong nap
Although Meta develops models they don't sell them. So a world where foundation models are free is fine for them.
They just don’t want to use OpenAI/Google models because they fear being screwed over by them with anti-advert terms of service or price increases. Similar to what they suffered with Apple.
The OSS goodwill is just a side effect and a way to undermine companies who are not using AI to effectively make profits today.
Cheaper/more efficient is absolutely great for Meta. If they can lower their capex it would be an instant bump to their bottom line.
Could you please provide any sources for this claim?
https://archive.is/8bRBH
https://medium.com/@omarkorim/is-meta-really-moving-beyond-t...
Of course, if businesses are gullible enough to believe facebook when it fudges up some brand lift metrics without having a real impact on conversions, that's their choice. Trusting facebook to report any analytics is how you take your business behind the barn and help it pivot to video. https://en.wikipedia.org/wiki/Pivot_to_video
I don't follow. Meta has been the only US big dog that released open-whatever variants of their models. They did that intending to minimise the gap between them and other big dogs. Their stated goal is to give open access to the community, while at the same time develop the models for internal uses (on their many platforms).
Meta doesn't sell API access. They are not losing on "cheaper" anything. If anything, they get to implement whatever others release under open terms into their stacks. And they still have all the GPUs to further train and serve on whatever improved stack comes next.
I don't see how meta loses here. In fact I think it is one of the only big players in this space that will come out better.
A more efficient model is better for NVIDIA not worse. More compute is still better given the same model. And as more efficient models proliferate it means more edge computing which means more customers with lower negotiating power than Meta and Google…
This is like thinking that if people need to dig only 1 day instead of an entire month to get to their nugget of gold in the midst of a gold rush the trading post will somehow sell fewer shovels…
And whether it’s overvalued or not isn’t relevant that selling a stock because the product the company produces is now even more effective is mind bogglingly stupid.
If it’s cheaper to train models it means far more customers that will try their luck.
If you reduce training requirement from a 100,000 GPUs to a 1000 you’ve now opened the market to 1000’s and 1000’s of potential players instead of like the 10 that can afford dumping so much money into a compute cluster.
The goal for AGI and ASI MUST BE to train, inference, train, inference and so on and that all on the fly in fractions of a second from every token produced.
Now good luck calculating the compute and hard work in algorithms to get there.
Not possible? Then AGI won't ever work because how can AGI beat a human if it can't learn on the fly? Not to mention ASI lol.
Just for those that clearly have no idea https://old.reddit.com/r/AMD_Stock/comments/1d2okn1/when_wil...
You shouldn't underestimate the fact that a large amount of these trades are on margin. Sometimes you can't wait it out because you'll get margin called and if you can't pony up additional cash you're basically getting caught with your pants down.
Disclaimer: I am not a trader, so could be way off
If it’s cheaper to inference you end up using the model for more task, it it’s cheaper to train you train more than models. And if you now need only 1000’s of GPUs instead of 10’s or 100’s of thousands you’ve just unlocked a massive client base of those who can afford to invest high six to low seven figures instead of 100’s of millions or billions into to try their luck.
They have a lot of very smart people and the will to do it, seems like a matter of time before they succeed.
The proof is in the pudding, you're welcome to prove "everyone" wrong.
No it isn't. Investors are most likely expecting there will be less demand for Nvidia's product long-term due to these alleged increased training efficiencies.
You seem to believe that the more inference or training value per piece of tech the more demand there will be for that piece of tech full stop when there are multiple forces at play. As a simple example, you can think of this as a supply spike; while you can make the bet that the demand will follow there could be a lag on that demand spike due to the time it takes to find use cases with product/market fit. That could collapse prices over the near term which could in turn decrease revenue. As a reminder the stock value isn't a bet on whether "the gold trader" will sell more gold or not, it's a bet on whether the net future returns of the gold trader will occur in line with expectations, expectations that are sky high and have zero competition built in.
So they're in a plushy seat, until the US decides they aren't.
In addition, IMO NVDA’s margins are a gift and a curse. They look great to investors, but also mean all their customers are aggressively looking to produce their own GPUs.
I’m surprised they haven’t yet.
Also, I should add that Deepseek just showed the top GPUs are not necessary to deliver big value.
[1] https://engineering.fb.com/2024/03/12/data-center-engineerin...
This announcement is one step in our ambitious infrastructure roadmap. By the end of 2024, we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s.
Every single time...
https://en.wikipedia.org/wiki/Jevons_paradox
In economics, the Jevons paradox occurs when technological progress increases the efficiency with which a resource is used, but the falling cost of use induces increases in demand enough that resource use is increased, rather than reduced.
It may be great news for VRAM manufacturers tough.
In other words, NVIDIA is in the red not because the company is suddenly doing worse, but because traders think other traders think it will trade down. That is a self fulfilling prophecy, but only so long as there is sufficient attention to drive that. The same works the other way around as well, so long as there is sufficient attention to drive the AI hype train upwards, related stocks will do well as well.
Well put. People need to unterstand that some stocks are basically one giant casino poker table. There was a comment with a link here that a lot of Nvidia buyers don't even know what products Nvidia is making and they don't care, they just want to buy low and sell high. Insert old famous comment abut shoe shine boy giving investment advice to Wall Street stock traders.
Yesterdays price of (say) NVidia was based on the expectation that companies would need to buy N billion of USD of GPUs per year. Now Deepseek comes out and makes a point that N/10 would be enough. From there it can go two ways:
- NVidia's expected future sales drop by 90%.
- The reduced price for LLMs should allow companies to push AI into markets that were previously not cost effective. Maybe this can 10x the total available market, but since the estimated total available market was already ~everything (due to hype) that seems unlikely.
- NVidia finds another usecase for GPUs to offset the reduced demand from AI companies.
In practice, it will probably be some combination of all three. The real problems are not caused for the "shovel sellers" but for companies like OpenAI and Anthropic, who now suddenly have to compete against a competitor that can produce the same product at (apparently) a fraction of the price.
So if the stock market was reflective of the economy (future or the present) then stocks should go up, instead they're going down. Why? Because the stock market is not reflective of the economy.
The stock market is essentially a reflection of societal perception. DJT which was brought up earlier is a great example, because the price of DJT has next to nothing to do with Trump's businesses and almost everything to do with how he is perceived (and remember there is no such thing as bad publicity).
Personally I think the fall will be momentary and followed shortly by a climb to recovery and beyond, but who really knows.
If you don't want to lose your money: Don't let the sensationalist financial journalists and pundits get to you, don't let big red numbers in your portfolio scare you, ignore traders (they all lose their money), don't sell your stocks unless you actually need that money for something right now, re-read your investment manifesto if you have one, and maybe buy the dip for shits and giggles if you have some spare cash laying around.
OpenAI and Anthropic can react by adopting DeepSeek's compute enhancements and using them to build even better models. AI training is still very clearly compute-limited from their POV (they have more data than they know what to do with already, and training "reasoning"/chains-of-thought requires a lot of reinforcement learning which is especially hard) so any improvement in compute efficiency is great news no matter where it comes from.
Maybe part of the growth was also "stupidness", and in that case buying the dip is a mistake because the "merit" price (value) is still way below.
In the .com bust you could have "bought the dip" in the early 00s right after the crash started and still taken 5 years before you weren't in the red even on "good" (in hindsight) stocks like amazon, ebay, microsoft, etc. The big hype there was eCommerce - it turned out to be true! We use eCommerce all the time now, but it took longer than predicted during the .com boom (same for broadband internet enabling "rich web experience" - it came true, but not fast enough for some hyped companies in '00).
And if you bought some of the darling stocks back then like Yahoo or Netscape that ended up not so great in hindsight you may have never recouped your losses.
There was never a question of if NVDA hardware would have high demand in 2025 and 2026. Everyone still expects them to sell everything they make. The reason the stock is crashing is because Wall St believed that companies who bought 50B+ of NVDA hardware would have a moat. That was obviously always incorrect, TPUs and other hardware was eventually going to be good enough for real world use cases. But Wall St is run by people who don't understand technology.
If they'll sell everything they make and it's all about the moat of their clients, why is NVDA still down 15% premarket? You could quote correlation effects and momentum spillover, but that is still just the higher order effects I mentioned about people's expectations being compounded and thus reactions to adverse news being convex.
Presumably because backorders will go down, production volume and revenue won't grow as fast, Nvidia will be forced to decrease their margins due to lower demand etc. etc.
Selling everything you make is an extremely low bar relative to Nvidia's current valuation because it assumes that Nvidia will be able to grow at a very fast pace AND maintain obscene margins for the next e.g. ~5 years AND will face very limited competition.
So I still don't understand what it is that you are so strongly disagreeing with, and I also don't understand how having owned NVidia stock somehow lends credence to your argument.
We are in agreement that this won't threaten NVidia's immediate bottom line, they'll still sell everything they build, because demand will likely rise to the supply cap even with lower compute requirements. There are probably a multitude of reasons why the very large number of people who own NVidia stock have decided to de-lever on the news, and a lot of it is simple uneducated herding.
But we are fundamentally dealing with a power law here - the forward value expectations for NVidia have exponential growth baked in to the hilt, combined with some good old fashioned tulip mania, and when that exponential growth becomes just slightly less exponential, that results in fairly significant price oscillations today - even though the basic value proposition is still there. This was the gist of my comment - you disagree with this?
Now is looks like that 10x of flow of money into OpenAI will no longer exist. There will be competition and compodiditzation, which causes the value of the tokens to drop way more than 40x.
There has always been a component of gambling to all investing, but that component now seems to utterly eclipse everything else. Merit doesn’t even register. Fundamentals don’t register.
But that's also dumb, because "huge leap forward in training efficiency" is not exactly bad news for the major players in even the medium term. Short term, it means their models are less competitive, but I don't see any reason that they can't leverage e.g. these new mixed precision training techniques on their giant GPU farms and train something even bigger and smarter.
There seems to be this weird baked in assumption that AI is at a permanent (or at least semi-permanent) plateau, and that open source models catching up is the end of the game. But this is an arms race, and we're nowhere near the finish line.
But I agree in the sense that Deepseek just creates more demand. Because people desire to get AI to do more work. This makes bang for buck greater opening new opportunities.
This sell off is like selling Intel in 2010 because of a new C compiler.
unless it can be said we need more performance than is currently possible, e.g. new demand, it would be catastrophic. it is unclear that throwing more compute actually expands what is possible. if that is not the case, efficiency is bad for nvidia because it simply results in less demand.
Today a CPU setup is still nowhere near as fast as a GPU setup (for ML/AI), but who knows how it looks like in the future.
> it is unclear that throwing more compute actually expands what is possible
Wasn't that demonstrated to be true already in the GPT1/2 days? AFAIK, LLMs became a thing very much because OpenAI "discovered" that "throwing more compute (and training data) at the problem/solution expands what is possible"
90% of traders lose money, so that's a data point...
You're trying to apply rational thinking but that's not how markets work. In the end valuations are more about narratives in the collective mind than technological merit.
You answered your own question. People do not dig in the Sacramento right anymore for gold, because, it is gone. If you can train models for 1/100 the cost, and you sell model training chips, you probably are not going to sell as many chips.
Everyone here thinks Nvidia is dommed because of training efficiency.
But what has Nvidia been doing for the past decade? Correct increasing training and inferencing efficiency by magnitudes.
Try to train GPT4 on 10k of Volta, Ampere, Hopper and then Blackwell.
What has happened since then? Nvidia has increased their sales in magnitudes.
Why? Because thanks to improvement in data, in algorithms, compute efficiency ChatGPT was possible in the first place.
Imagine Nvidia wouldn't exist. When do you think the ChatGPT moment would happen on CPUs? LOL
Going back to my first sentence. Nvidia started also with small shovels which were GeForce cards with CUDA. Today Nvidia is selling huge GPU clusters (mining machines, yes pun intended ^^).
No. The stock is still x10 after this dip from 2 years ago and x40 from a few years ago.
Likely a "how solid is the technical moat" evaluation - this could be a one-off or could be that there are an avalanche of advancements to continue along the efficiency side of the process.
Given the style and hype of logic in the AI space, I fully believe resources are not well allocated in compute and _actual_ thinking as to how they are spent.
Deepseek's apparent 10x more efficient per inference token... implies a lot of other hardware meets the general use-case. We also know that reasoning should be about 10W for human speed-of-thought... maybe another 1-2 orders of power efficiency.
"Pre-Training: Towards Ultimate Training Efficiency
We design an FP8 mixed precision training framework and, for the first time, validate the feasibility and effectiveness of FP8 training on an extremely large-scale model. Through co-design of algorithms, frameworks, and hardware, we overcome the communication bottleneck in cross-node MoE training, nearly achieving full computation-communication overlap. This significantly enhances our training efficiency and reduces the training costs, enabling us to further scale up the model size without additional overhead. At an economical cost of only 2.664M H800 GPU hours, we complete the pre-training of DeepSeek-V3 on 14.8T tokens, producing the currently strongest open-source base model. The subsequent training stages after pre-training require only 0.1M GPU hours." [1]
[1] https://huggingface.co/deepseek-ai/DeepSeek-V3
I believe the saying is "The market can stay wrong for longer than you can stay solvent."
There is a point where there are enough shovels circulating that the demand for new shovels falters, even with zero drawback in the rush. And if so much gold was being mined that it overwhelmed the market and reduced the commodity price, the value of better shovels is reduced.
DeepSeek and friends basically reduce the commodity value of AI (and to be fair, Facebook, Microsoft et al are trying to do the same thing with their open source models, trying to chop the legs out of the upstart AI cos). If AI is worth less, there are going to be fewer mega capitalized AI ventures buying a trillion dollars worth of rapidly-depreciating GPUs in hopes at eeking out some minor advantage.
I wouldn't short nvidia stock, but at the same time there is a point where the spend of GPUs just isn't rational anymore.
>And as more efficient models proliferate it means more edge computing which means more customers with lower negotiating power than Meta and Google
Edge compute has infinitely more competition than the data center.
I think Meta is a big winner from this - they still control the content and now mining it for value has been proven even cheaper by DeepSeek.
The startups that are in trouble are the ones that have been screaming for "AGI" and buying up GPUs and close-sourcing their models.
Open source was always going to win the race to the bottom. [0]
[0] https://news.ycombinator.com/item?id=41671582
This shows how effective open source and open development can be.
Best way to get a correct answer is ... https://www.phind.com/search?cache=ws3qq1xl8hj4yx1izd5oeda0
Looking at https://github.com/deepseek-ai, those repos have a bunch of of contributors but unless I'm wrong I don't see any significant contributions. What am I missing?
For Nvidia, it is great news because now finally, the concentration of GPUs at Hyperscalers will end and every Fortune company can finally get their local data center to train their AI Models.
Because if training AI models becomes more efficient and easier then the ones being that business are at risk so basically Big Tech. Nvidia isn't in that business but in the business of providing tools to train.
Fortunately, Big Tech can easily do something to prevent ANYONE for competing. They simply buy all available GPUs. Oh wait, haven't they been doing it for years? Excatly!
People really don't get what an arms race and market competition race is.
How do I prevent disruption? I simply buy all the tools the competition needs to disrupt me.
See, if Fortune 500 companies want to build large data centers but can't because all Hyperscalers buy the GPUs then eventually they will rent from cloud as otherwise they can't get the GPUs.
Spending $50 billion to do $6 million worth of Ai training seems like a good way to trigger a golden parachute and "spend more time with your family" as a CEO.
Everyone is hung up on the cost of training R1 but it’s 685 billion parameters. We still need all of those GPUs to actually use the model.
That makes both NVDA stock and big AI infrastructure spending less compelling, as those needs are scaled down via software efficiency and chip alternatives.
And yet again a cheaper Chinese product turns up and everyone loses their minds. Expect a ban incoming to preserve the AI valuations.
"DeepSeek (Chinese AI co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget (2048 GPUs for 2 months, $6M)."
https://x.com/karpathy/status/1872362712958906460
Are we talking about the same forum? HN commenters have been raving about DeepSeek v3 for at least a month.
One could argue by extension that ASML is also cyclical.
Everyone except for you, right?
Everyone else who over-leveraged into the private AI companies (Anthropic, OpenAI) are going to have their valuations under scrutiny.
It actually affects the frontier AI companies (OpenAI, Anthropic, etc) who directly make money from their closed models AND spend hundreds of millions on training these models.
Why pay $3/per million tokens (Claude 3.5 Sonnet) when DeepSeek R1 offers $0.14 / per million tokens and the model is on par with (OpenAI o1) and R1 itself is released for free?
$0 free AI models are eating closed AI models lunch.
But yeah agree with you!
cheaper spam and impersonation engines (LLMs) have a direct impact on the ad network that is strategically downsizing its moderation efforts
For some time it's been clear that there's an AI bubble and this may be what finally pops it.
The dot com bubble was a real thing, and yet the internet has gone on to be one of humanities most valuable innovations.
A Chinese company coming up with a cheaper alternative to a cutting-edge technology out of nowhere, is an outcome that is hard to predict.
In hindsight, betting on Nvidia maintaining its monopoly on a resource crucial for such an important technology as AI, might not be the best of ideas, but then again, who knows.
meta uses amd mi300x for all inference and which bought 173.000 gpu ( 40% of their total gpu count ) according to that report https://www.cnbc.com/2023/12/06/meta-and-microsoft-to-buy-am...
It's amazing what they did with a limited budget, but instead of the takeaway being "we don't need that much compute to achieve X", it could also be, "These new results show that we can achieve even 1000*X with our currently planned compute buildout"
But perhaps the idea is more like: "We already have more AI capabilities than we know how to integrate into the economy for the time being" and if that's the hypothesis, then the availability of something this cheap would change the equation somewhat and possibly justify investing less money in more compute.
Basically: China tech sector just made a big splash, traders who witnessed this think other traders will sell because maybe US tech sector wasn't as hot, so they sell as other traders also think that and sell.
The fall will come to rest once stocks have fallen enough that traders stop thinking other traders will sell.
Investors holding for the long haul will see this fall as stocks going on sale and proceed to buy because they think other investors will buy.
Meanwhile in the real world, on Main Street, nothing has really changed.
Bogleheads meanwhile are just starting the day with their coffee, no damns given to the machinations of the stock market because it's Monday and there's work to be done.
The Magnificent Seven are the only thing propping up the whole US economy.
If they go down, you go down.
That's how i understand it.
And since their current goal seems to be 'AGI' and their current plan for achieving it seems to be scaling LLMs (network depth wise and at inference time prompt wise), i don't see why it wouldn't hold.
You can't do the distill/magnify cycle like you do with alphago. LLM models have basically stalled in their base capabilities, pre training is basically over at this point, so the news arms race will be over marginal capability gains and (mostly) making them cheaper and cheaper.
But inference time scaling, right?
A weak model can pretend to be a stronger model if you let it cook for a long time. But right now it looks like models as strong as what we have aren't going to be very useful even if you let them run for a long, long time. Basic logic problems still tank o3 if they're not a kind that it's seen before.
Basically, there doesn't seem to be a use case for big data centers that run small models for long periods of time, they are in a danger zone of both not doing anything interesting and taking way too long to do it.
The AI war is going to turn into a price war, by my estimations. The models will be around as strong as the ones we have, perhaps with one more crank of quality. Then comes the empty, meaningless battle of just providing that service for as close to free as possible.
If Openai's agents panned out we might be having another conversation. But they didn't, and it wasn't even close.
This is probably it. There's not much left in the AI game
We also don't know what the next discovery/breakthrough will be like. The reward for getting smarter AI is still huge and so the investment will likely remain huge for some time. If anything DeepSeek is showing us that there is still progress to be made.
But making things smaller is different than making them more powerful, those are different categories of advancement.
If you've noticed, models of varying sizes seem to converge on a narrow window of capabilities even when separated by years of supposed advancement. This should probably raise red flags
Have you considered that compute might be the reason why LLMs are stalled at the moment?
What made LLMs possible in the first place? Right, compute! Transformer Model is 8 years old, technically GPT4 could have been released 5 years ago. What stopped it? Simple, the compute being way too low.
Nvidia has improved compute by 1000x in the past 8 years but what if training GPT5 takes 6-12 months for 1 run based on what OpenAI tries to do?
What we see right now is that pre-training has reached the limits of Hopper and Big Tech is waiting for Blackwell. Blackwell will easily be 10x faster in cluster training (don't look on chip performance only) and since Big Tech intends to build 10x larger GPU clusters then they will have 100x compute systems.
Let's see then how it turns out.
The limit on training is time. If you want to make something new and improve then you should limit training time because nobody will wait 5-6 months for results anymore.
It was fine for OpenAI years ago to take months to years for new frontier models. But today the expectations are higher.
There is a reason why Blackwell is fully sold out for the year. AI research is totally starved for compute.
The best thing for Nvidia is also that while AI research companies compete with each other, they all try to get Nvidia AI HW.
Except o3 benchmarks are, seemingly, pretty solid evidence that leaving LLM'S on for the better part of a day and spending a million dollars gets you... Nothing. Passing a basic logic test using brute force methods and which falls apart on a marginally easier test that it just wasn't trained on.
The returns on computer and data seem to be diminishing with more and more exponential increases in inputs returning geometric increases in quality, and we're out of quality training data so that is now much worse even if the scaling wasn't plateauing.
All this, and the scale that got us this far seems to have done nothing to give us real intelligence, there's no planning or real reasoning and this is demonstrated every time it tries to do something out of distribution, or even in distribution but just complicated. Even if we got another crank or two out of this, we're still at the bottom of the mountain here. We haven't started and we're already out of gas
Scale doesn't fix this any more than building a mile tall fence stops the next break in. If it was going to work we would have seen to work already. LLM's don't have much juice left in the squeeze, imo
are you sure? people are saying that there’s an analogous cycle where you use o1-style reasoning to produce better inputs to the next training round
if you've tried to get o1 to give you outputs in a specific format, it often just tells you to take a hike. It's a stubborn model, which implies a lot
This is speculation, but it seems that the main benefit of reasoning models is that they provide a dimension along which RL can be applied to make them better at math and maybe coding, things with verifiable outputs.
Reasoning models likely don't learn better reasoning from their hidden reasoning tokens, they're 1) trying to find a magic token which when raised to its attention make it more effective (basically give it room to say something that jogs its memory) or 2) it is trying to find a series of steps which do a better job of solving a specific class of problem than a single pass does, making it more flexible in some senses but more stubborn along others
Reasoning data as training data is a poison pill, in all likelihood, and just makes a small window of RL vulnerable problems easier to answer (when we have systems that don't better). It doesn't really plan well, doesn't truly learn reasoning, etc
Maybe seeing the actual output of o3 will change my mind but I'm horrifically bearish on reasoning models
The scaling has been plateauing, and half that equation is quality training data which is totally out at this point.
Maybe reasoning models will help produce synthetic data but that's still to be seen. So far the only benefit reasoning seems to bring is fossilizing the models and improving outputs along a narrow band of verifiable answers that you can do RL on to get correct
Synthetic data maybe buys you time, but it's one turn of the crank and not much more
https://en.m.wikipedia.org/wiki/Chernoff_bound
I agree with you that they require data
https://m.youtube.com/watch?v=7LNyUbii0zw
He seems adamant that there are no diminishing returns to scaling AI.
I don’t want to stir up conspiracy theories but I do think that currently all the big AI players have a vested interest in the message that the current scaling paradigm is the right one, and that this is a supremacy issue wrt China. It drives so much investment and valuation that I doubt they can truly be objective.
500 Billion is a lot of money. Expect even crimes to be commited in order to make it happen.
Why buy 100k Hoppers if 20k Blackwell offer the same compute so then it's better to buy 100k Blackwells right?
Backwell will increase cluster scaling easily by 10x performance and if you buy 10x of them then your compute on a cluster will be 100x than before. If it takes you to wait 6-12 months for that then so be it. You will easily make up the time in the end with the speed up.
You can see a similar bias in academia with work originating outside EU/USA.
before someone thinks something strange regarding me, I can only tell you I'm not chinese, but Argentinian :)
people are just more focusing on the political side of it
it's always 'but tiananmen square'
o1 at least gives it to me straight. When I ask it to engage in more back and forth before assuming what I'm after, it tends to follow through. Deepseek seemed immediately eager to (very slowly) feed me a bunch of made up information thinking that's what I wanted.
I feel as though a lot of people get hung up on these sort of "micro benchmarks" whereas trying to get practical work done is severely under tested. I'm not a fan of openai at all but I don't have the spare compute to run anything locally so o1 suffices for now.
Still don't see how this is anything but a win for Nvidia though.
The business is doing good, the wannabe traders not so much
As Karpathy (among others) has noted, the <think> output shows signs of genuine emergent behavior. Presumably the same thing is going on behind the scenes in the OpenAI omni reasoning models, but we have no way of knowing, because they consider revealing the CoT output to be "unsafe."
And then - something completely unexpected, a total curveball - arrives a week into office and everything changes. Your agenda collapses and you enter reactive mode.
AI bubble could pop. Valuations drop. Crypto might get hit. Where is Project Stargate now? It might become a joke.
A more cynical observer might suggest that _the timing was no coincidence_.
Where has Trump's agenda collapsed? I might have missed the press release.
And why would a curveball on AI throw off an agenda around trade, immigration and military engagement? I don't follow.
China could take the lead on AI and I don't see it would impact any of those things. Isn't DeepSeek open source? The US already has access to it, so what leverage could China possible have?
> A more cynical observer might suggest that _the timing was no coincidence_.
Do you feel the Chinese government closely controls AI research and timed this response?
Any evidence for that?
Well, it's too early to say if it has happened in this case. But we've seen it happen again and again, so it will not be a surprise if it happens.
> And why would a curveball on AI throw off an agenda around trade, immigration and military engagement? I don't follow.
Well, trade restrictions against China have just backfired spectacularly. So further trade restrictions may not seem as good an idea. And to start trade wars, you need a strong economy. And at the moment America's economy is entirely driven by the AI bubble, as it is the value of the AI stocks that has separated the economy from the European trend.
It is very likely that military engagement will be driven by, or affected by, developments in AI. And there's no doubt that Taiwan's situation is heavily affected by chip production.
> Do you feel the Chinese government closely controls AI research and timed this response?
No evidence, but I am not naive enough to think it's out of the range of possibility. I don't follow your reasoning, more likely the timing of the release is all they had to modify - which is quite trivial for any government. To not question that would be very naive. They have _every motive_.
Ah ok, so you’re just guessing. You wrote it as if it had already happened.
And I think you’re confusing the stock market with the broader economy. The economy as a whole is unaffected by AI at this point.
And the trade war with China involved hundreds of industries beyond AI. Most of it is manufacturing. I’m not sure how failure of AI sanctions (questionable conclusion) somehow means the trade issues around machine parts needs to be abandoned.
I agree China has motive but I’ve heard so many claims that the Chinese government doesn’t control research or businesses like TikTok.
That's not a generous reading. I'm saying this is what has happened historically.
Edit: the use of the words "might" and "could" made this pretty clear.
> The economy as a whole is unaffected by AI at this point.
True in terms of no marked effect on GDP. But the stock market and the broader economy are very much linked. See 2008. And the stock market is high on AI.
> I’ve heard so many claims that the Chinese government doesn’t control research or businesses like TikTok.
Even western countries tell tech companies what to do. Do you think an authoritarian government is going to do that more or less? There's a reason they didn't want to sell TikTok.
Maybe this was the point of creating this model all along ?