It should be very clear that Nvidia is not interested in selling you GPUs to run models. But it’s very interested in selling data centers that you can use to run models in a way that everybody on their end “accepts”.
DGX Spark, RTX Spark, Pro Blackwells, etc. They are absolutely interested in selling local hardware you own. Just at the similar margins to the rest of their main business now.
Back in the day, things like these were outlawed and the government tried to ensure competition in the markets:
> The major film studios owned the theaters where their motion pictures were shown, either in partnerships or outright. Thus, specific theater chains showed only the films produced by the studio that owned them. The studios created the films, had the writers, directors, producers and actors on staff (under contract), owned the film processing and laboratories, created the prints and distributed them through the theaters that they owned: In other words, the studios were vertically integrated, creating a de facto oligopoly. [...] Ultimately, this issue of the studios' then-alleged (and later upheld) illegal trade practices led to all the major movie studios being sued in 1938 by the U.S. Department of Justice.
As usual, Wikipedia has it backward. There were very few theaters in existence able to show films produced by the studios. They invested in theaters and found local businessmen to operate them, both as sole proprietors and as partners.
The investment was out of necessity, not interest in monopolies.
Writers and directors craved an audience, so they appealed to studios for funding. That’s opportunity meeting, not monopolistic practices.
It’s just government intervention in business it knows little, and can only know little, about.
Only a few years earlier none of that existed. Then it did, proved highly successful, then government, which knows not what it is doing, nor can it, comes along and gets involved in something it had no interest in, and contributed little to nothing to.
The Sarbanes-Oxley Act of 2002 could be considered the worst regulation of 21 century. It helped created behemoth corporations, where a single person can misallocate hundred of billon of dollars (Oh, hi Mark), they can buy out any competition.
Right now US economy is basically a centrally planned economy.
I've seen people use those three words to make a dig at zuckerberg before, but I never put 2 and 2 together and realized the quote until just now. I think the comma was essential, made my brain automatically read it correctly... Wiseau voice and all, lol.
More on-topic: I remain amazed by how the past quarter century of American legislation has included one overwhelming boon for corporations after another. Once corporations could give an infinitly higher amount of political contributions when compared to individual citizens, it became clear that the overall well-being and happiness of corporations was the only real political focus. Anything that doesnt directly or indirectly benefit a group of corporations is almost always designed to manipulate the electorate through invented/perceived/embiggened/non-cromulent "social issues".
There is no mechanism. Finance and startup Bros want to dump their overvalued companies with deceitful accounting on public investors to make a quick profit and socialize the inevitable losses. SOX makes it harder for them to do that.
To whatever extent SOX compliance makes it more complex to go public, it has no meaningful effect on legitimate successful companies - if you think going public will allow you to raise the most money, you'll go public; if ZIRP meant you could indefinitely raise private money, you'd stay private. Finance and startup bros want to blame companies staying private on SOX, but it has everything to do with either (1) the companies being deeply questionable from an accounting perspective, or (2) the companies being able to raise whatever investments they wanted in private without taking any of the costs of an IPO (e.g. the IPO pop, which could (simplisticaly) be thought of as money being made by the banks underwriting the IPO rather than by the existing owners/investors).
Reflection AI was founded in 2024. Is it really consolidation when new companies spawn out of nothing and get acquired 2 years later? That's just normal M&A churn.
This is one of those completely factual statements that is not factually complete.
Obviously not all (tech) companies will prove to be successful.
Although, if we define success as “was acquired, investors / founders came out financial better off” then both absorbed and successful can be true while consolidation never was a concern.
if they aren't successful they should go under, financially viable organizations should be rewarded with survival; there should be more incentive to build successful sustainable businesses and more competition to drive down costs and raise quality for customers than the quick exit we see a lot of the time
So their assets/ brand/ people are worth hundreds of millions or billions to strategic buyers like nvidia but we should just make them go bankrupt? Why?
if they go bankrupt, someone will buy their assets or end up in sale to someone anyways, and if they couldn't make enough money to continue operations maybe its not actually worth "hundreds of millions or billions"?
bankruptcy also gives companies a chance to redo things and get stuff in order to pay the debtors; it doesn't always end in liquidation.
in any case, its better longterm to incentivize survival of the fittest through competition than through golden exit as it is now...
Huh? new tech companies have proliferated in the US the last few years. M&A is just one side of the coin. Without an exit path -- IPO, acquisition or private credits -- there is very little appetite for new businesses.
Did you know that 151 companies were founded by SpaceX's alumni? Compare to 5 they've acquired (not including XAI)
Insane conspiracy theory. There's no incentive whatsoever for Nvidia to release weak models. If you bothered to pay any attention, they are aggressively trying to catch up. Whether they succeed, that's of course a separate question.
similar to how eventually AWS started making their own chips for data centers, and Apple did that for their hardware, it's not a ridiculous thing to plan for the AI companies to start making their own chips to optimize for their use cases and cut out the middleman for margins.
Dog fooding their own product. This isn’t new for Nvidia. For a long time they have been selling GPU cards, and also licensing the chips for other manufacturers to make their own cards. Similar for their automotive products, Shield, and DGX Spark.
I believe NVIDIA's long term strategy will be to pivot from the data center to the public, and the public will utilize open models on NVIDIA hardware at home. This will come after the RAMpocalypse completes (when the new fabrication plants (China, Tesla/SpaceXAI/Intel) fully ramp up and start selling their RAM for cheap in the next few years). Data centers will be for training mostly.
But, not really at the current technologies. Kimi and GLM are fucking awesome, but I don’t have 3TB of VRAM to run them, and I don’t expect to even when ram prices drop.
So now you’re back to the scaling issue before talking about power and compute distribution.
Do you think you'll realistically need 3 TB RAM to run a sufficiently good model 1 to 2 years from now? I certainly don't. Considering what can already be done with 128 GB of relatively slow unified memory, imagine if efficiency improvements continue apace, the memory becomes 128 GB of HBM, the flash device becomes capable of sequential throughput matching today's DDR5, and such a system was affordable as a routine purchase for the average person.
Don't discount improvement in the ability of models to search for and ingest external data - either web or purpose built databases. I really don't think 3 TB will be necessary with the right model architecture and training environment.
It is cheaper to subscribe to AI for $500 a year for the rest of your life than to buy a machine capable of running the a current frontier model with no subagents.
Almost all the non-tech people I know acknowledge concerns about privacy but have no realistic alternative available to them. I think they won't want to self host until and unless it's easy to deploy, there's nothing to administrate, and the hardware cost is in the same ballpark as a new laptop was prior to the apocalypse.
> Why do you think the public wants to self host models over using a cheaper solution hosted in the cloud?
you aren't truly in control of your compute in the cloud, and if you came to become reliant on ai, the cloud company could simply turn it off for you and you'd be shit outta luck. Unlike a regulated utility like electricity and water, where you cannot be discriminated against based on how you use said utility.
Therefore, having the ability to run local ai is a good thing. Currently, hardware constraints have kept it out of most hobbyist's hand, but i am hoping that will change soon enough (within 2-3 yrs).
The mega corps are looking to regulate ai such that individuals cannot run their private ai themselves, so that they can become more effective a monopoly. That's why it is crucial that ai is not regulated, but companies providing ai.
Personally, the biggest reason is that if I'm investing in my business, and the business is going to rely on AI models, I need them hosted locally on my own hardware in a replicable way. The reason being is, if the AI subscription service changes the model, or drops support for it, or worse (goes out of business), you're done.
Have you used Nemotron-3.5-lightening? I don’t use it as much as Poolside’s (excellent!!) Laguna XS 2.1 6bit, but the new Nemotron model is good.
I think NVIDIA does want small open models running on-prem to explode as a market! Lots of smaller GPU installations for companies who wisely want on-prem inference.
Of course NVIDIA will also keep making a ton of money selling to hyper scalers, but not forever: Chinese chips are getting better, Google, Microsoft, Amazon, etc. designing their own inference chips.
I think the idea that they’d purposely spend company time and resources making a bad model is an extraordinary claim, requiring extraordinary evidence. The more likely explanation is that they aren’t willing to distill from their own customers, so they are at a disadvantage.
Nvidia is trying to increase its customer base. Look at the Mag 7, Amazon, Google, Microsoft, and even Meta are all working on their own inference chips. I don't think they can completely ditch Nvidia for LLM training, but they can make their own chips for inference. I believe that's also why OpenAI made its own.
> Which is what I thought kept everyone using Nvidia.
Once an organization is big enough to have AI-platform or AI-infra teams, you stop being hung-up on details like CUDA; resources will be invested to make whatever abstractions the AI researchers and engineers rely on (e.g. Torch) will be made to work, and work well.
Yeah but if you have the capability why not write that software for existing hardware alternatives and abstract hardware vendor away for your self? It seems like instead your choosing to do two hard things instead of just one
The big tech companies are cash-rich, can afford to hire (and pay) top people and are looking for things for those people to do.
In other words their motivations and your and my motivations are different. I have limited cash, and my primary goal is expanding my ability to get more cash.
By contrast, the Google's of this world have too much cash. Their primary goal is finding things to depend the cash on (billions at a time) that can return, or at least potentially return, either mountains of more cash, or a competitive advantage.
So yes, for them, hard problems, the harder the better, are desirable.
Both Google and Meta have done that years ago, with Tensorflow/JAX and PyTorch?
But a software abstraction only helps to reduce lock in. It does not unlock efficiencies on itself. For that new and/or hardware hardware might be needed.
If Nvidia was actually smart they'd increased production enough that these companies don't wanna do all this. Instead they're producing the conditions for 50 new competitors to not only rise up but be able to to print money. And the gaming community they are burning in the process will happily jump ship.
I think almost the exact opposite. They want open models. Without credible open models, they only have a few customers, and those customers have leverage against nvidia. With open models, they have tons of customers, and nvidia has all the leverage.
Smaller customers are also less able to develop their own hardware and threaten NVidia's business.
Sure but it also means these companies nvidia buys don’t work with AMD and other competitors anymore. There are myriad motivations and it’s not all one or the other but this is a classic component of Silicon Valley acquisition strategies.
They can't ride the hyperscaler gravy train forever; at some point between Google, AMD, and Apple NVIDIA is going to lose its monopoly on serving large customers.
At that point, it would be useful if a few open models existed which were only a couple of months behind the frontier.
But it's very important that the open models never be TOO good, because the AI companies are buying compute on the assumption that their software will add value. If it becomes a commodity business with frontier open models, NVIDIA won't be able to get away with such a crazy markup.
If NVIDIA is forced to compete on price, they're screwed unless they can find more customers. They aren't the only game in town for accelerators any more.
Because NVIDIA is competing against AMD, Google, and Apple just to name a few, and more dangerously soon they'll be competing against their clients.
Apple is working on an enterprise server. Google TPU revenue growth is crazy. Even AMD is finally usable for ML. All the frontier labs have chip teams and it's not to keep buying from Nvidia.
If it wants to avoid being dependent on its biggest clients it needs to make sure the hyperscalers have competition, and preferably competition that doesn't want to get into the chip design business.
> Without credible open models, they only have a few customers, and those customers have leverage against nvidia. With open models, they have tons of customers, and nvidia has all the leverage.
I wonder if Nvidia is also trying to cover themselves against a market crash the bankrupts the AI labs but leaves the tech standing? (Similar to the dotcom crash or the big railroad crash back in the day.) If Anthropic and OpenAI struggle, Nvidia can always sell local inference hardware. But local inference hardware often has a lower utilization, so you need more GPUs for the same number of tokens.
Indeed the hardware-inefficiency of local inference is in principle good, not bad, for hardware vendors. But until the manufacturing capacity exists to ship local-inference systems with enough VRAM to enough customers then inference is going to keep taking place at (more-)efficient centralised providers and nVidia will have to keep trying to sell what it makes to them to remain an AI player.
This is the correct take. NVidia did the same thing with the hyperscalers - when GOOG/MSFT/AMZN were all building their own chip competitors, NVidia began funding the neoclouds to fight down the margin on their competitors.
I doubt Nvidia wants to be fully dependent on the success of two highly unprofitable companies that could implode at any moment. It makes much more sense for them to commoditize LLMs so that their target market grows to every mid-sized or larger company.
I think this is a classic [Commoditize Your Complement](https://news.ycombinator.com/item?id=17047348) - Nvidia wants open models because their business is hardware and it's complement is AI models, so they want AI models to be commoditized so that hardware is the industry with leverage. OpenAI/Anthropic/etc want closed models so that the AI model development/data has leverage over the hardware providers.
This is one of the sillier conspiracies when you think about the economics.
nVidia sells hardware. They want open models everywhere because it encourages people and companies to buy more of their hardware, which will sit idle most of the time, instead of using data center services which are ruthlessly optimized to maximize token throughput per hardware unit.
Acquiring companies that are good at training models is exactly what you would do if you were bad at training models.
Their strategy is to prevent open models from proliferating, so their massive investments in these AI frauds are not completely unwound.
This makes no sense. It's pretty clear that they want open models to proliferate as a counter to OpenAI and Anthropic dominance.
If the market is concentrated into 2 players, those 2 players will control both the models and the hardware. Both of them already have custom AI chip ambitions.
Nvidia wants open models to matter because it allows them to sell hardware to OpenAI, Anthropic, and any company that wants to run open models.
Yes, NV took $800M of a $2B round in Reflection.ai.
NV's long-term strategic incentive in funding a semi-open model provider like Reflection.ai is to ensure competitive frontier models which fully leverage the NV proprietary stack (chips, interconnects, servers, CUDA) continue to be widely available to all customer classes and continue to offer better performance worth a higher price to the most profitable customer classes.
NV's ~75% margins on hardware(!)) at >$100B/yr are historically unprecedented and still increasing, creating tectonic pressure on NV's largest customers (hyperscalers and frontier labs) to escape the "NV Tax" by gaining access to competitive frontier chips, servers, and/or middleware at lower margins. As Jeff Bezos famously said, "Your margin is my opportunity." NV's biggest, best customers are also now their biggest long-term existential threat.
While NV's moats to significant competition are currently almost unimaginably deep, on a decadal time-scale, the literal trillions of dollars at stake can make unimaginable things possible. It's forcing ALL the major participants into playing 3D strategic chess on multiple time horizons at once, simultaneously working with, investing in and hedging against each other. And under the feet of the giants, smaller players are scurrying to buy temporary protection by either partnering with a giant temporary favorprotection or
Yes, NV took $800M of a $2B round in Reflection.ai.
NV's long-term strategic incentive in funding a semi-open model provider like Reflection.ai is to ensure competitive frontier models which fully leverage the NV proprietary stack (chips, interconnects, servers, CUDA) continue to be widely available to all customer classes and continue to offer better performance worth a higher price to the most profitable customer classes.
NV's ~75% margins on hardware(!)) at >$100B/yr are historically unprecedented and still increasing, creating tectonic pressure on NV's largest customers (hyperscalers and frontier labs) to escape the "NV Tax" by gaining access to competitive frontier chips, servers, and/or middleware at lower margins. As Jeff Bezos famously said, "Your margin is my opportunity." NV's biggest, best customers are also now their biggest long-term existential threat.
At the moment, NV's moats to significant competition are almost unimaginably deep, but on a decadal time-scale, literal trillions of dollars are at stake. That can make the once-unimaginable not only possible but probable. It's forcing all the big companies into playing 3D strategic chess on multiple time horizons at once, simultaneously working with, investing in and hedging against each other. This can create strange bedfellows overnight. I wouldn't be surprised to see some of NV's biggest customers, who compete fiercely against each other, pooling resources with NV's competitors to create a viable alternative to NV. This is the stuff Jensen has nightmares about, waking up in a cold sweat and soaked black leather pajamas.
Every ex-Google Deep mind should start a start up, get funding from VCs and Nvidia. Sell the startup back to Nvidia. That's the best play for those who are capable of this kind of AI startup.
I like that US Big-AI companies are M&A'ing startups left and right, while the Chinese competitors are hounding them incessantly. Just don't know how this will work out financially in the long term.
That's precisely the opposite of what Nvidia is trying to do. Nvidia wants open and local to succeed so that their customer base is as large and diverse as possible. Some commenters here can't see that because they're blinded with cynicism.
DeepSeek drops CUDA kernels for fast inference, if users care about having day-0 fast inference, you'll have to play catch up every time.
Also, I've heard many western labs use open source CUDA kernels (usually from Chinese labs), IMO it makes little sense to move away from CUDA right now unless you're a big player that can afford a patchy ecosystem with internal development (e.g. Anthropic with JAX/TPU).
Precisely. But that might change. Once Tenstorrent releases a high VRAM card that only does linear algebra then the price might genuinely start to make sense.
Not really a fan of pure GDDR, lower bandwidth means you pay in latency when pulling the weights/KV-cache, on top of that you need to up the batches to be compute bound, so you really need to have really high capacity (since requests are routed to a specific GPU).
AFAIK you're much more limited in capacity with GDDR compared to HBM, as you're limited by the number of GPU memory channels, so you can't vertically scale much.
I don't think it needs higher VRAM, the whole point is horizontally scaling.
Most models have moved to MoE and expert routing which is done over infiniband, so tensor parallelism where Tenstorrent was really weak at isn't used much, I can see it as a viable alternative where latency isn't a concern but cost is.
Also even if they wanted to increase the capacity on their GDDR, they wouldn't be able to since they are limited by the GPU memory channels.
What are the ways to make models work on AMD, Intel, etc faster? Do AMD, Intel, Tenstorrent, Qualcomm (due to their Modular.ai acquisition) combine forces to take on CUDA?
This matches a clear pattern in Nvidia’s latest 10-Q. In H1, they spent $42.4B buying equity in other companies—nearly matching the $45.3B returned to shareholders. Owning a piece of your customers isn't the same business as just selling them GPUs.
104 comments
[ 5.0 ms ] story [ 25.0 ms ] thread> The major film studios owned the theaters where their motion pictures were shown, either in partnerships or outright. Thus, specific theater chains showed only the films produced by the studio that owned them. The studios created the films, had the writers, directors, producers and actors on staff (under contract), owned the film processing and laboratories, created the prints and distributed them through the theaters that they owned: In other words, the studios were vertically integrated, creating a de facto oligopoly. [...] Ultimately, this issue of the studios' then-alleged (and later upheld) illegal trade practices led to all the major movie studios being sued in 1938 by the U.S. Department of Justice.
https://en.wikipedia.org/wiki/United_States_v._Paramount_Pic....
Nowadays it seems like "vertically integrated" is something most companies openly aim for.
The investment was out of necessity, not interest in monopolies.
Writers and directors craved an audience, so they appealed to studios for funding. That’s opportunity meeting, not monopolistic practices.
Only a few years earlier none of that existed. Then it did, proved highly successful, then government, which knows not what it is doing, nor can it, comes along and gets involved in something it had no interest in, and contributed little to nothing to.
Right now US economy is basically a centrally planned economy.
More on-topic: I remain amazed by how the past quarter century of American legislation has included one overwhelming boon for corporations after another. Once corporations could give an infinitly higher amount of political contributions when compared to individual citizens, it became clear that the overall well-being and happiness of corporations was the only real political focus. Anything that doesnt directly or indirectly benefit a group of corporations is almost always designed to manipulate the electorate through invented/perceived/embiggened/non-cromulent "social issues".
Adam Tooze and Cameron Abadi did a series of episodes last year on a number of them
https://www.youtube.com/playlist?list=PLYRt3SPnw6gzpIl6aE325...
What's the mechanism for this? Genuine question: I read an overview over Sarbanes-Oxley and it seems like a reasonable idea.
To whatever extent SOX compliance makes it more complex to go public, it has no meaningful effect on legitimate successful companies - if you think going public will allow you to raise the most money, you'll go public; if ZIRP meant you could indefinitely raise private money, you'd stay private. Finance and startup bros want to blame companies staying private on SOX, but it has everything to do with either (1) the companies being deeply questionable from an accounting perspective, or (2) the companies being able to raise whatever investments they wanted in private without taking any of the costs of an IPO (e.g. the IPO pop, which could (simplisticaly) be thought of as money being made by the banks underwriting the IPO rather than by the existing owners/investors).
Obviously not all (tech) companies will prove to be successful.
Although, if we define success as “was acquired, investors / founders came out financial better off” then both absorbed and successful can be true while consolidation never was a concern.
if they aren't successful they should go under, financially viable organizations should be rewarded with survival; there should be more incentive to build successful sustainable businesses and more competition to drive down costs and raise quality for customers than the quick exit we see a lot of the time
bankruptcy also gives companies a chance to redo things and get stuff in order to pay the debtors; it doesn't always end in liquidation.
in any case, its better longterm to incentivize survival of the fittest through competition than through golden exit as it is now...
Did you know that 151 companies were founded by SpaceX's alumni? Compare to 5 they've acquired (not including XAI)
https://www.alumnifounders.com/stats
Obviously my feelings about things I know little about, games I have no skin in, are important.
If you say anything to the contrary the group-think will downvote you.
Shut up bigot.
/s
Say government intervenes and prevents this merger, what’s the most likely outcome?
Company won’t exist in 6 minutes.
In all likelihood this company was spun up with the intention it would be purchased by a ”competitor.”
That's my take, at least.
Nemotron is TERRIBLE, and purposefully so. It must be.
They cannot be THAT BAD at training AI models. I don't believe it.
similar to how eventually AWS started making their own chips for data centers, and Apple did that for their hardware, it's not a ridiculous thing to plan for the AI companies to start making their own chips to optimize for their use cases and cut out the middleman for margins.
But, not really at the current technologies. Kimi and GLM are fucking awesome, but I don’t have 3TB of VRAM to run them, and I don’t expect to even when ram prices drop.
So now you’re back to the scaling issue before talking about power and compute distribution.
in two years, yes I expect us to have much better models. Perhaps many times more efficient. But not 100 times.
you aren't truly in control of your compute in the cloud, and if you came to become reliant on ai, the cloud company could simply turn it off for you and you'd be shit outta luck. Unlike a regulated utility like electricity and water, where you cannot be discriminated against based on how you use said utility.
Therefore, having the ability to run local ai is a good thing. Currently, hardware constraints have kept it out of most hobbyist's hand, but i am hoping that will change soon enough (within 2-3 yrs).
The mega corps are looking to regulate ai such that individuals cannot run their private ai themselves, so that they can become more effective a monopoly. That's why it is crucial that ai is not regulated, but companies providing ai.
I think NVIDIA does want small open models running on-prem to explode as a market! Lots of smaller GPU installations for companies who wisely want on-prem inference.
Of course NVIDIA will also keep making a ton of money selling to hyper scalers, but not forever: Chinese chips are getting better, Google, Microsoft, Amazon, etc. designing their own inference chips.
NVIDIA is handling this brilliantly.
No one wants to pay the Nvidia tax
Once an organization is big enough to have AI-platform or AI-infra teams, you stop being hung-up on details like CUDA; resources will be invested to make whatever abstractions the AI researchers and engineers rely on (e.g. Torch) will be made to work, and work well.
In other words their motivations and your and my motivations are different. I have limited cash, and my primary goal is expanding my ability to get more cash.
By contrast, the Google's of this world have too much cash. Their primary goal is finding things to depend the cash on (billions at a time) that can return, or at least potentially return, either mountains of more cash, or a competitive advantage.
So yes, for them, hard problems, the harder the better, are desirable.
But a software abstraction only helps to reduce lock in. It does not unlock efficiencies on itself. For that new and/or hardware hardware might be needed.
Smaller customers are also less able to develop their own hardware and threaten NVidia's business.
They can't ride the hyperscaler gravy train forever; at some point between Google, AMD, and Apple NVIDIA is going to lose its monopoly on serving large customers.
At that point, it would be useful if a few open models existed which were only a couple of months behind the frontier.
But it's very important that the open models never be TOO good, because the AI companies are buying compute on the assumption that their software will add value. If it becomes a commodity business with frontier open models, NVIDIA won't be able to get away with such a crazy markup.
Of course they would. You need hardware to run the model. nVidia sets the floor.
If NVIDIA is forced to compete on price, they're screwed unless they can find more customers. They aren't the only game in town for accelerators any more.
Yes nVidia is not the only game in town, but that's no more or less true regardless of whether there are a bunch of competing providers.
Apple is working on an enterprise server. Google TPU revenue growth is crazy. Even AMD is finally usable for ML. All the frontier labs have chip teams and it's not to keep buying from Nvidia.
If it wants to avoid being dependent on its biggest clients it needs to make sure the hyperscalers have competition, and preferably competition that doesn't want to get into the chip design business.
I wonder if Nvidia is also trying to cover themselves against a market crash the bankrupts the AI labs but leaves the tech standing? (Similar to the dotcom crash or the big railroad crash back in the day.) If Anthropic and OpenAI struggle, Nvidia can always sell local inference hardware. But local inference hardware often has a lower utilization, so you need more GPUs for the same number of tokens.
nVidia sells hardware. They want open models everywhere because it encourages people and companies to buy more of their hardware, which will sit idle most of the time, instead of using data center services which are ruthlessly optimized to maximize token throughput per hardware unit.
Acquiring companies that are good at training models is exactly what you would do if you were bad at training models.
Have you seen who composes most of Nvidia's revenue?
The Nemotron models are just a by-product of this strategy.
If the market is concentrated into 2 players, those 2 players will control both the models and the hardware. Both of them already have custom AI chip ambitions.
Nvidia wants open models to matter because it allows them to sell hardware to OpenAI, Anthropic, and any company that wants to run open models.
NV's long-term strategic incentive in funding a semi-open model provider like Reflection.ai is to ensure competitive frontier models which fully leverage the NV proprietary stack (chips, interconnects, servers, CUDA) continue to be widely available to all customer classes and continue to offer better performance worth a higher price to the most profitable customer classes.
NV's ~75% margins on hardware(!)) at >$100B/yr are historically unprecedented and still increasing, creating tectonic pressure on NV's largest customers (hyperscalers and frontier labs) to escape the "NV Tax" by gaining access to competitive frontier chips, servers, and/or middleware at lower margins. As Jeff Bezos famously said, "Your margin is my opportunity." NV's biggest, best customers are also now their biggest long-term existential threat.
While NV's moats to significant competition are currently almost unimaginably deep, on a decadal time-scale, the literal trillions of dollars at stake can make unimaginable things possible. It's forcing ALL the major participants into playing 3D strategic chess on multiple time horizons at once, simultaneously working with, investing in and hedging against each other. And under the feet of the giants, smaller players are scurrying to buy temporary protection by either partnering with a giant temporary favorprotection or
NV's long-term strategic incentive in funding a semi-open model provider like Reflection.ai is to ensure competitive frontier models which fully leverage the NV proprietary stack (chips, interconnects, servers, CUDA) continue to be widely available to all customer classes and continue to offer better performance worth a higher price to the most profitable customer classes.
NV's ~75% margins on hardware(!)) at >$100B/yr are historically unprecedented and still increasing, creating tectonic pressure on NV's largest customers (hyperscalers and frontier labs) to escape the "NV Tax" by gaining access to competitive frontier chips, servers, and/or middleware at lower margins. As Jeff Bezos famously said, "Your margin is my opportunity." NV's biggest, best customers are also now their biggest long-term existential threat.
At the moment, NV's moats to significant competition are almost unimaginably deep, but on a decadal time-scale, literal trillions of dollars are at stake. That can make the once-unimaginable not only possible but probable. It's forcing all the big companies into playing 3D strategic chess on multiple time horizons at once, simultaneously working with, investing in and hedging against each other. This can create strange bedfellows overnight. I wouldn't be surprised to see some of NV's biggest customers, who compete fiercely against each other, pooling resources with NV's competitors to create a viable alternative to NV. This is the stuff Jensen has nightmares about, waking up in a cold sweat and soaked black leather pajamas.
Also, I've heard many western labs use open source CUDA kernels (usually from Chinese labs), IMO it makes little sense to move away from CUDA right now unless you're a big player that can afford a patchy ecosystem with internal development (e.g. Anthropic with JAX/TPU).
AFAIK you're much more limited in capacity with GDDR compared to HBM, as you're limited by the number of GPU memory channels, so you can't vertically scale much.
Most models have moved to MoE and expert routing which is done over infiniband, so tensor parallelism where Tenstorrent was really weak at isn't used much, I can see it as a viable alternative where latency isn't a concern but cost is.
Also even if they wanted to increase the capacity on their GDDR, they wouldn't be able to since they are limited by the GPU memory channels.