In a sense, I think everyone knew it was coming. Historically speaking, in US at least, there initially may be a notable number of upstarts, but those numbers quickly dwindle to form oligopolies. That is how we end up with 3 million cereal types, but 2-3 companies controlling them. The same repeats throughout most industries with an annoying consistency.
I actually use both gpt and local qwen and I personally think the question tends to be of what one needs ( speed? privacy? lack of corporate silliness? ).
Still, it is sad that we can predict the future so well based merely on what happened before.
Edit: I certainly hope I don't have to explain to you why the above is relevant. No market participant would dare to argue this if the competition was healthy.
And this is why socialism never works: only you know the value of your RAM. When you "own nothing and be happy," somebody else decides that you are using too much RAM.
What has socialism to do here lmao, you literally have the PEAK capitalist in the world right now doing the exact thing you complain about and bring an hypothetical SoCiAlIsM as the problem. Man american propaganda is one hell of a drug, I'm surprised you didn't manage to blame cHiNa in the same sentence
I wonder if there's a side business in a service which for a fee plants enough trees to pay off the 'oxygen debt' of a person the customer deems to be an oxygen thief?
It would be a decidedly positive outlet for negative interpersonal feelings, we've all heard of spite houses and spite walls but spite trees would be a rather more ecological form of the concept. It is also a fairly practical way to insult someone, since the victim cannot rankle at it without looking churlish towards a charitable act.
All well and good for now until the free lunch of free models ends, and the ones you can "freely" download have weights that tell you to drink Ovaltine or whatever other messaging gets sold.
Two statements can be true and he stated one of them. Sam is not wrong.
1. Local inference can cause a large under-utilization of resources from users that do not run local inference hardware at capacity over the hardwares lifespan. This is under-utilization and under-utilization is not optimal from an economics perspective.
2. The concentration of inference hardware to singular providers can aggregate power to single systems or organizations, and their incentives may be at odds with individual users.
The same power dynamic and economics struggle has existed nearly everywhere for all of history. For example, if you want to decide to own or rent a piece of heavy construction machinery for business, you will find a long list of do-not's on the rental contractual terms, but you are more than welcome to buy your own and operate your own machinery at several times the price for the hours used and use it however you want.
Few people will argue with the logic of whether it is technically accurate ( assuming actual deployment will be optimal, because that is hardly a given ). It is just amusingly self-serving. It is almost as if McD got to decide you can only eat there.
Weirdly, underutilization of hardware was never a problem before. In fact, wastefulness is so ingrained in programming culture that the previous hype - blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.
> Two statements can be true
As I general point, because I keep seeing this line on HN: Of course they can in the general case, but more often than not, the reason why this line is written is because there is some tension or contradiction between the statements. In that case, understanding where the tension lies is IMO more useful than shrugging and deciding to "hold two contradictory thoughts in your head" like in 1984.
Sometimes it feels like a desperate rhetoric device to "save" an opinion when the opposite opinion is supported by obvious evidence.
> ... blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.
A hard rule of life is, if there are resources available, something will consume them. It's interesting that this also ended up being true for compute!
99.67% of seawater can't be used for fusion. Some very poor napkin math, it would drop the sea level something like 35 feet (not including rises from global warming)?
Underutilization was indeed a problem before. The anecdotes you've presented are not broadly applicable.
The entire reason SETI@Home started was because people had unused hardware sitting idle at home.
Every company I've ever been in has had slides on resource utilization for procured hardware.
In fact not only could the two statements be true, in fact both are true. People want to ignore otherwise because doing so will make local LLM inference sound like the only option when only one is considered.
It can still be optimal because some activity A might provide more utility at 10% utilization rate than activity B at 100% utilization rate . Otherwise shared cars would be vastly superior to individual owned cars and same for houses in which 2 or 3 families live in different shifts ... we actually had that before in some areas in early stages of the industrialisation.
Users apparently don't need to be thought about at all if a few companies can secure loans large enough to purchase all dram capacity, and in the process close that market for consumers.
It seems to me it's economically optimal for inference providers to intentionally close markets that would allow local inference at all, even if that's bad for users as a whole.
>Weirdly, underutilization of hardware was never a problem before. In fact, wastefulness is so ingrained in programming culture that the previous hype - blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.
Same with AI. AI use is a colossal waste, for all kinds of frivolous purposes (e.g. some person's ai girfriend chat, or someone asking about K-pop lore, or slop image generation, or slop "dark internet posts"), but also to facilitate "serious" business that we'd be better if it wasn't done at all.
> Weirdly, underutilization of hardware was never a problem before.
Fixing exactly this has been the constant drumbeat of IT since the early 00's at least. It started with virtualization, to get capacity factor of hardware up in the enterprise. The average server back before VMWare (and HyperV/etc.) became a major thing was probably sitting at 10% utilization or less 95% of the time. Hypervisors were invented to fix this as their primary use case. You'd take a 5-10% performance hit on the raw server performance to gain 90% efficiency back.
Next came the cloud. Which was virtualization at scale. The larger the scale, the better the efficiency.
It's basically almost all anyone has actually accomplished in IT over the past decade or two. Everything else has been largely stagnant compared to the 80's and 90's.
AI inference is in it's infancy, but I imagine we will see more or less the exact same thing play out yet again. Lots of initial on-prem stuff due to cost and freedom, and eventually it becomes virtualized, commoditized, customer segregated, etc. Just like average cloud workloads are today and all the various "small" megawatt-scale builds moved back into the hyperscaler type providers within the next 5-10 years.
> How many companies are building datacenter commons and sharing with other companies to optimize resource usage?
Quite a few that I've seen. Rented inference/token factories are the current new hotness. Plenty of speculative builds being done by hundreds of smalltime providers. These will consolidate per usual and everyone but the last few top percent will remain as the winners. Just like the first hosting providers. Virtualization and fancy schedulers to maximize hardware utilization across a number of end-user customers are a huge topic in this space at the moment. Heck, a lot of these small time players are providing raw compute for the giant AI labs and even hyperscalers.
I don't like that Altman is right on this btw. I much preferred the highly inefficient days of every midsize corporate office having a 200-500kw datacenter in the basement. AWS and their ilk are quite obviously orders of magnitude more hardware efficient though.
> This is under-utilization and under-utilization is not optimal from an economics perspective.
The problem is that this sneaks in a postulate by negating "optimal", which is that underutilization is the opposite of that which is desirable, that it is implicitly undesirable. If we don't take that premise, say that it's value neutral and neither desirable nor undesirable by itself, then the argument doesn't really stand. This neutral valuation is far more reasonable to me. Anybody who works in logistics intuitively already understands this. We don't have cargo ships bow to stern across every single shipping lane, yet this is "underutilization" of those shipping lanes. Sure they're bottlenecked at the ports, but the ports can always be bigger, more numerous and possibly even work faster. The slop turns out doesn't matter. Actually it's beneficial for a few reasons, but my point here is sufficient I think.
I agree with him. And I'll gladly donate my 128GB of DDR5 if it means OpenAI and Anthropic will be nationalized to ensure equal access to resources. Central planning will surely create the most efficient system
3. Some applications are going to work just fine with smaller models on local hardware. If people still have machines with the memory to run those models laying around, that will cost OpenAI business. Also, once RAM suppliers catch up and Moore's law resumes operation, the capability of local hardware is going to steadily improve. Were it not for the AI build-out, typical configurations for new PC's would likely have moved on from 32GB by now.
4. Everything is training data. If that data isn't going to OpenAI, they're not training on it.
Also, it's important to remember OpenAI's failure might actually be the best thing for everyone else in the AI field. e.g. Some new innovation/approach could come out of left field and eat OpenAI's lunch. They've bet the farm (and a lot of other peoples' farms) on a brute force approach. Altman needs your RAM because, if he doesn't control the brute force, the brute force approach might not work only for him.
It would be interesting if there were some kind of holographic LLM. You could send part of the encoded prompt and receive generated tokens that only you can decode. Then we could use the distributed capacity of the world and everyone could be paid for marginal token generation capacity. Even open models are profitable to run on almost all electricity.
Altman: You are welcome to invest a little bit of your fortune in making major software developers write more efficient code that stops wasting our RAM. Start with internet browsers and the Windows 11 app that wastes one gigabyte showing the weather.
He's right in a such a narrow space on this particular matter that it's just a perfect example of what's wrong with a lot of things lately.
OpenAI triggered this ordeal with its order of 900K DRAM wafers per month on october 2025, making everyone panic because it was around 40% of the world production at the time.
We all know that memory providers know what near bankruptcy feels like when they over invest, but right now SK Hynix have 80% margins (ie TSMC ~60%) and choose to pay their shareholders this year the quivalent of ALL their revenues in 2025. It's now common knowledge thanks to Samsung that the rate of factory buidling is not maxmium to portect their margins. And memory providers have already been fined in 2000 for price fixing.
Supply is short because they all choose to make it that way.
Seems like micron is pulling out all the stops to get their new line open. Source is someone who lives near the micron plant and talked about their insane hiring and work right now.
Everything that has happened is 100% aligned with ECON 101.
Demand has increased for high-bandwidth memory. That increases prices and quantity.
The increase in quantity has happened. Quantity sold has increased. Fabs have increased production and shifted away from other products including consumer products.
Prices increase when there is an increase in price of a substitute in production.
New fabs are planned, consistent with profit-maximizing behavior increasing production in response to increased prices, but they take time.
Yes, what smart RAM manufacturers (or who've been longer in the business) are doing: ramp up production capacity, but do so gradually.
Those ramp-up timelines tend to be longer than the markets' boom & bust cycles. So they'll be looking at longER-term trends to decide what to invest in & when.
I mean, he's right: individuals hoarding scarce resources is less efficient that everyone holding them in common. The problem is, how to ensure we all get a fair share of them. We can't leave that up to commercial enterprises, because they also have an incentive to hoard and control them for their own profit. Perhaps what we need is some kind of government that holds them in common for the benefit of the entire populace and. But what to call it? Common-ism...?
He is obviously correct about the specific narrow statement made here and it’s so annoying everyone has to be memey mad about it like a bad reddit thread.
No he isn't. Most AI "utilization" is from idiots asking stupid stuff and doing stupid projects. They have all the resources they need for serious stuff. Fuck them.
There is a huge loophole in the reasoning: Sam Altman assumes, without stating it clearly, that RAM is used only for AI use, and the only question is "should AI inference being run in the cloud or on our computer".
But that's not true. Computers, including RAM, are used for many different things that do not involve AI at all. video games, office work, communications, access to culture, etc.
So, what Altman really says is "AI is the only thing that matter for computing, and thus, for the success of his company, we must allocate all the RAM to his company, and forbid any people from accessing it".
And, IMO, we should not accept that kind of discourse :s
I have a vehicle in my driveway, but I am not driving it all day, therefore I am wasting it and should give it to a corporation to rent it back to me. You will own nothing and be happy
I understand what he's saying. Sam Altman thinks that HE, with his data centers, will make better use of the potential of VRAM (or whatever word designates computing capacity) compared to the local market.
In theory, I don't disagree. He needs it, and he will use it, that's for sure. Moreover, his logical argument should be viewed purely through the lens of his own self-interest: "I need it, I don't have it, it's your fault, and anyway, you don't even use it properly."
But the question I ask myself is: can he use VRAM for something other than what is requested millions of times?
"GPT, make me an app to boil eggs,"
"GPT, make me an app to compare prices,"
"GPT, make me an app to alert me during Black Friday..."
So many resources used thousands of times... With his VRAM... So the counter-argument we can give him is: Stop thinking like an American picking flowers with a bulldozer (no offense, it's just a metaphor) and learn to conserve your resources.
The most fascinating thing about this is how american billionaires can speak in public and advocate for abolition of private [computing] property (means of production), to give full control over these resources to a handful of megacorporations (the party), in effect letting them control and command the economy (central planning). It's the communist boogeyman wearing a trench coat under the banner of capitalism!
Guy with the most DRAM in the world thinks it's terrible he doesn't have all of it.
It would be more efficient to ban cars in favor of mass transit. It would be more efficient if people only lived in apartment complexes rather than standalone houses. Tons of things can be more "efficient" if you take away people's desire for privacy and ownership.
64 comments
[ 1.6 ms ] story [ 17.7 ms ] threadI actually use both gpt and local qwen and I personally think the question tends to be of what one needs ( speed? privacy? lack of corporate silliness? ).
Still, it is sad that we can predict the future so well based merely on what happened before.
Edit: I certainly hope I don't have to explain to you why the above is relevant. No market participant would dare to argue this if the competition was healthy.
This is, very literally, capitalism at work. It's the CEO of a massively valuable company jawboning his self-interest.
These massively valuable companies are the ones who want you to "own nothing and to be happy". That's the core tenet of the rental/"sharing" economy.
It would be a decidedly positive outlet for negative interpersonal feelings, we've all heard of spite houses and spite walls but spite trees would be a rather more ecological form of the concept. It is also a fairly practical way to insult someone, since the victim cannot rankle at it without looking churlish towards a charitable act.
Sam Altman can eat my entire ass.
Careful what you wish for, Sam may actually be willing to do it.
mainly because the thing about open weights is folks will just immediately strip it back out.
But I can guarantee you that openai/anthropic/other will absolutely jump at the chance to monetize users by feeding them paid recommendations.
Can you please not do this here? It's not what HN is for, and destroys what it is for.
https://news.ycombinator.com/newsguidelines.html
1. Local inference can cause a large under-utilization of resources from users that do not run local inference hardware at capacity over the hardwares lifespan. This is under-utilization and under-utilization is not optimal from an economics perspective.
2. The concentration of inference hardware to singular providers can aggregate power to single systems or organizations, and their incentives may be at odds with individual users.
The same power dynamic and economics struggle has existed nearly everywhere for all of history. For example, if you want to decide to own or rent a piece of heavy construction machinery for business, you will find a long list of do-not's on the rental contractual terms, but you are more than welcome to buy your own and operate your own machinery at several times the price for the hours used and use it however you want.
> Two statements can be true
As I general point, because I keep seeing this line on HN: Of course they can in the general case, but more often than not, the reason why this line is written is because there is some tension or contradiction between the statements. In that case, understanding where the tension lies is IMO more useful than shrugging and deciding to "hold two contradictory thoughts in your head" like in 1984.
Sometimes it feels like a desperate rhetoric device to "save" an opinion when the opposite opinion is supported by obvious evidence.
A hard rule of life is, if there are resources available, something will consume them. It's interesting that this also ended up being true for compute!
The entire reason SETI@Home started was because people had unused hardware sitting idle at home. Every company I've ever been in has had slides on resource utilization for procured hardware.
In fact not only could the two statements be true, in fact both are true. People want to ignore otherwise because doing so will make local LLM inference sound like the only option when only one is considered.
What they did was to say "hey, we can leverage your idle computer to help process this".
Different perspective.
It seems to me it's economically optimal for inference providers to intentionally close markets that would allow local inference at all, even if that's bad for users as a whole.
Same with AI. AI use is a colossal waste, for all kinds of frivolous purposes (e.g. some person's ai girfriend chat, or someone asking about K-pop lore, or slop image generation, or slop "dark internet posts"), but also to facilitate "serious" business that we'd be better if it wasn't done at all.
Fixing exactly this has been the constant drumbeat of IT since the early 00's at least. It started with virtualization, to get capacity factor of hardware up in the enterprise. The average server back before VMWare (and HyperV/etc.) became a major thing was probably sitting at 10% utilization or less 95% of the time. Hypervisors were invented to fix this as their primary use case. You'd take a 5-10% performance hit on the raw server performance to gain 90% efficiency back.
Next came the cloud. Which was virtualization at scale. The larger the scale, the better the efficiency.
It's basically almost all anyone has actually accomplished in IT over the past decade or two. Everything else has been largely stagnant compared to the 80's and 90's.
AI inference is in it's infancy, but I imagine we will see more or less the exact same thing play out yet again. Lots of initial on-prem stuff due to cost and freedom, and eventually it becomes virtualized, commoditized, customer segregated, etc. Just like average cloud workloads are today and all the various "small" megawatt-scale builds moved back into the hyperscaler type providers within the next 5-10 years.
> How many companies are building datacenter commons and sharing with other companies to optimize resource usage?
Quite a few that I've seen. Rented inference/token factories are the current new hotness. Plenty of speculative builds being done by hundreds of smalltime providers. These will consolidate per usual and everyone but the last few top percent will remain as the winners. Just like the first hosting providers. Virtualization and fancy schedulers to maximize hardware utilization across a number of end-user customers are a huge topic in this space at the moment. Heck, a lot of these small time players are providing raw compute for the giant AI labs and even hyperscalers.
I don't like that Altman is right on this btw. I much preferred the highly inefficient days of every midsize corporate office having a 200-500kw datacenter in the basement. AWS and their ilk are quite obviously orders of magnitude more hardware efficient though.
Not to mention, I bet users will be open to "rent" their hardware for proper utilization, provided he can pay for it.... real question is, will he?
The problem is that this sneaks in a postulate by negating "optimal", which is that underutilization is the opposite of that which is desirable, that it is implicitly undesirable. If we don't take that premise, say that it's value neutral and neither desirable nor undesirable by itself, then the argument doesn't really stand. This neutral valuation is far more reasonable to me. Anybody who works in logistics intuitively already understands this. We don't have cargo ships bow to stern across every single shipping lane, yet this is "underutilization" of those shipping lanes. Sure they're bottlenecked at the ports, but the ports can always be bigger, more numerous and possibly even work faster. The slop turns out doesn't matter. Actually it's beneficial for a few reasons, but my point here is sufficient I think.
also, its a bit weird to say a supply chain fails if you simply order much more than they have.
in stock:100 'i want a million, the supply chain is failing and others need to stop buying so i can get a million'
4. Everything is training data. If that data isn't going to OpenAI, they're not training on it.
Also, it's important to remember OpenAI's failure might actually be the best thing for everyone else in the AI field. e.g. Some new innovation/approach could come out of left field and eat OpenAI's lunch. They've bet the farm (and a lot of other peoples' farms) on a brute force approach. Altman needs your RAM because, if he doesn't control the brute force, the brute force approach might not work only for him.
OpenAI triggered this ordeal with its order of 900K DRAM wafers per month on october 2025, making everyone panic because it was around 40% of the world production at the time.
We all know that memory providers know what near bankruptcy feels like when they over invest, but right now SK Hynix have 80% margins (ie TSMC ~60%) and choose to pay their shareholders this year the quivalent of ALL their revenues in 2025. It's now common knowledge thanks to Samsung that the rate of factory buidling is not maxmium to portect their margins. And memory providers have already been fined in 2000 for price fixing.
Supply is short because they all choose to make it that way.
Demand has increased for high-bandwidth memory. That increases prices and quantity.
The increase in quantity has happened. Quantity sold has increased. Fabs have increased production and shifted away from other products including consumer products.
Prices increase when there is an increase in price of a substitute in production.
New fabs are planned, consistent with profit-maximizing behavior increasing production in response to increased prices, but they take time.
When e.g. memory manufacturers don't collude, they end up on synchronized boom-and-bust cycles.
But it's also illegal for them for form a cartel, which IIUC would have the benefit of smoothing the supply:demand ratio over time.
Is there any truly good solution to this?
(I'm not an economist, so it's quite possible that my question isn't even cogent.)
Yes, what smart RAM manufacturers (or who've been longer in the business) are doing: ramp up production capacity, but do so gradually.
Those ramp-up timelines tend to be longer than the markets' boom & bust cycles. So they'll be looking at longER-term trends to decide what to invest in & when.
I use my RAM how the hell I want and nobody else better damn well think about telling me otherwise.
But that's not true. Computers, including RAM, are used for many different things that do not involve AI at all. video games, office work, communications, access to culture, etc.
So, what Altman really says is "AI is the only thing that matter for computing, and thus, for the success of his company, we must allocate all the RAM to his company, and forbid any people from accessing it".
And, IMO, we should not accept that kind of discourse :s
In theory, I don't disagree. He needs it, and he will use it, that's for sure. Moreover, his logical argument should be viewed purely through the lens of his own self-interest: "I need it, I don't have it, it's your fault, and anyway, you don't even use it properly."
But the question I ask myself is: can he use VRAM for something other than what is requested millions of times? "GPT, make me an app to boil eggs," "GPT, make me an app to compare prices," "GPT, make me an app to alert me during Black Friday..."
So many resources used thousands of times... With his VRAM... So the counter-argument we can give him is: Stop thinking like an American picking flowers with a bulldozer (no offense, it's just a metaphor) and learn to conserve your resources.
It would be more efficient to ban cars in favor of mass transit. It would be more efficient if people only lived in apartment complexes rather than standalone houses. Tons of things can be more "efficient" if you take away people's desire for privacy and ownership.