Samsung is already building another fab (which was originally suspended due to low memory prices years ago). Hopefully when it goes live in a few years it's not all dedicated to HBM for AI.
Seemingly the AI bulls are either not sufficiently optimistic, or not sufficiently wealthy(?!), to build new fabs as joint ventures with the manufacturers in which they agree to assume most of the downside risk?
They signed long term contracts for the hardware delivery over years. AI investment is approaching 1 trillion per year and expected to grow to well over 1 trillion per year. [1]
Even over their many years, projects like the Manhattan Project, the Apollo Program, or the U.S. Interstate Highway System never added up to that. [2]
Is your argument that there is insufficient AI spending at present as they aren’t also taking on building their own fabs?
If the AI bubble bursts, the DRAM manufacturers switch back to making conventional DRAM and pent-up business and consumer demand results in a surge of purchases, cushioning the fall considerably.
Look into the time frames a bit more, this isn’t a switch they flip. If the bubble bursts today it will be years before those downstream effects add up.
That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will want those models instead.
This + your ROI in $10k device will be always lower than busy datacenter, its literally math. They sell free compute to others when you dont use it, you will never sell at that level or even you magically sell home compute, you will not compete at price
That greater efficiency only benefits the LLM SaaS providers as long as the hardware manufacturers, probably especially the VRAM manufacturers, remain supply constrained, since the high-efficiency users are the ones who can pay top dollar. But the hardware guys' dream is presumably to get parts into millions of laptops which remain on standby for 19 hours a day, not to bargain with SaaS providers who obsessively optimise their memory consumption.
We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.
Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
I’ll grant that for specialized applications like coding agents and mathematics, but even there I suspect that most the real gains are actually taking place in the harness.
But I suspect returns may have already diminished into negative territory for at least some other use cases. One of my least favorite job responsibilities in this brave new era is figuring out how to avoid performance and behavior regressions when an older model were using for some application reaches end of life. It’s getting uncommon for me to look at our benchmark results and say, “Oh, good, it does better on one of the newer models!”
One thing that I really want to know - the better models from today vs a year ago - what has changed. They have already pre-trained on all available public data. Scooping up the last percentage of archaic texts which were never digitized is not going to move the needle.
Is it just that the providers are generating tons of synthetic datasets on coding tasks so that the models get more exposure to the right thing to do? Every time someone points out an LLM stupidity they add some training data to patch over the weakness (trivial to generate "there are two 'l's in llama")?
And you apparently only read the title and ignored the content.
It’s also poor reasoning to you only pick one thing you think supports your view, and ignore vastly more things not supporting it, all from the same useful criteria.
Now if only you’d carefully read the report you chose, and spend equal time looking at ample presented evidence, you’d develop a more accurate understanding.
It’s a constant tension in computing that has been around since mainframes and clients… Neither is going to disappear. My general feeling is normal people care more about how thin and light something is than their privacy, so if data center powered LLMs will have a strong future.
Edge based storage is very very viable, and very very cheap, people still use Apple / Google Photos (consumers don’t want any friction).
Coorperate people likely just want “the best” and are happy to pay a lot for it to make their workforce more efficient.
It’s hard for me to guess where the data center buildout will go, eventually there will be an oversupply, but we haven’t hit that yet, and at the moment the companies owning these high capacity data centers appear to be making lots of very real money in leases to hyper scalers, hard to imagine this premium continuing, but we need to see where demand for cloud based LLMs tops out.
I’m not so sure about that. Already AI vendors are back to cutting prices to try and keep customers from cutting back on their usage. My own employer is working hard at pivoting to much smaller fine-tuned models for established use cases, and seeing model performance improvement in addition to large inference cost reductions. Being able to run them locally hasn’t exactly been a disaster for devex, either.
It may turn out that demand for SOTA frontier models isn’t so limitless after all.
Every product follows demand curves. At a price of 0 you could find infinite usage. This has nearly zero relation to how much it costs to provide the product.
Except of course it relates. All else being equal, we will prefer $X COGS over $2X COGS because that helps us with both profit margins and price competition.
Framing it in terms of the price people might be willing to pay for a single product in isolation frames the point I was making, which was about price competition, right out of the picture.
Maybe I'd be willing to pay $10 for product A if I had other options. But if there's a product B for $3 that's not quite as nice but still ticks all my boxes, then product instantly becomes a lot less attractive.
It's going to happen very soon, which is why these frontier labs are scrambling to shut down open source language models. There's an existential risk threatening their obscene returns.
But he gave a reason for that. "Before that happens the frontier will move". Why do you think it will happen anyway? Do you think the frontier will not move fast enough that local models are unable to catch up, or do you think people will prefer local models at a point. Or something else?
Because everything is converging on a backlog of huge efficiency gains established in research, waiting to be combined. Looped transformers, a whole host of diffusion techniques and new quantization techniques, maturation of ternary distillation and new ways to separate logic from stuff that can be looked up. It would surprise me if most frontier models were actually even that big at that point in terms of active params. I highly doubt it.
I feel like cost competitiveness of local has been going down, if anything, not up. API providers can use hardware more and have scale efficiencies I assume. Do you see any reason this will reverse?
You don’t need frontier-level performance for every task. That’s why companies hire both junior and senior developers. I suspect a decent percentage of people using Fable would probably be fine with Opus.
Also, the frontier can’t keep advancing at this rate forever. Eventually the low-hanging fruit will all be gone and advances will slow down.
What if the AI companies start selling the hardware with their models?
I.e., you get a locked down device with access to their AI and maybe a way to run third party apps. Of course the developers of those apps will have to pay a percentage of their revenue.
During the Covid global chip shortage Intel announced new factories to address the production bottleneck, before the factories even started to be constructed, the shortage was long over and the projects were eventually canceled.
About 2 years ago I bought some SDRAM or something like that, DDR4 or DDR5, don't recall offhand. A few months ago I looked at the price today, and it was over
3x as high. That's just insane. Governments need to do something against this abuse system that AI amplified here.
Yeah the 300 already felt like a splurge - it's the max. 96 was less than 200 I felt kinda stupid not just going with that. Imagine 96 of anything now for less than 200
In one sense, nothing, in another, everything. It is DRAM, but the bandwidth requirements mean it’s paired to a processor, i.e. no DIMMs. Not 100% sure but things like MacBooks and the Framework tower, where you have fixed RAM for the device lifetime, have ~0 tradeoff.
It's not that there's a blocker. It's that it takes roughly 3x the manufacturing capacity to produce an HBM package at the same storage capacity as DRAM. We are sacrificing total bytes for bandwidth.
How so? FEOL is pretty much the same, BEOL is almost the same save TSVs, the packaging tech is different and more advanced, but not exactly 1:1 comparable. Do TSVs really occupy 3x the area of DDR IO's?
It doesn't really say that it needs to use 3x more area, but that 3x more gets consumed due to "advanced packaging and manfuacturing complexity". Which doesn't properly explain why it consumes 3x more and could simply mean they have a bad yield and 2/3 produced is garbage.
This might not be far off the mark. You are irreversibly linking the fates of these devices after a certain stage of manufacturing. If something goes wrong at final packaging time, you lose all dies instead of one.
You make HBM by stacking a whole bunch of dies on top of each other. The signals from the upper dies need to pass through vias in the lower dies to get out - taking up valuable die space in a way which simply isn't needed with regular DDR. Similarly, HBM has a far wider bus, so each individual die has, say, 16 banks of depth 32, rather than 4 banks of depth 128. That's more control area needed per byte of memory.
Those two combined already result in a huge reduction in bytes per mm2, so with the same wafer processing capacity you're producing far less byte of memory. Add to that a complicated chain of HBM-specific packaging steps, and you're now also losing a decent bunch of perfectly-fine dies because rather than putting it into DDR you tried making a HBM sandwich and screwed up.
Even if the memory cells are the same and have an absolutely identical yield, HBM will always end up having a significantly lower output. That's just the cost of stacking, but some people are willing to pay the per-gigabyte price penalty in return for the higher bandwidth.
Classic DRAM stacks up to four wafers on top of each other and then is packaged with BGAs. The manufacturer can check the DRAM chips independently.
Soldering the DRAM onto a PCB is such a reliable process that there is almost zero risk of defects and even if a defect occurs the damage is limited. If the DRAM is soldered onto a DIMM the risk of a defect on the non memory hardware is non-existent. If the DRAM is soldered straight onto an SBC or GPU, then the DRAM can be removed to save the precious SoC or GPU chips.
Meanwhile HBM is the ultimate nightmare scenario. You stack up to 16 DRAM wafers on top of each other. One defect and the whole stack is worthless and that was actually the easy part.
In stage two things get even worse. You now have your accelerator chip and you must place the HBM on that chip. E.g. Blackwell GB300 has eight HBM stacks and the accelerator chip has a bigger area than the HBM. You must get the packaging right eight times in a row or you have wasted not only the DRAM silicon, but also the accelerator silicon because HBM cannot be removed and defects are permanent.
The issue here isn't just the yield of the HBM (which is obviously lower if you have taller stacks) but rather the yield of the combined HBM-based product, which is why doesn't make sense to say it needs more area but it is completely correct to say that HBM leads to more silicon being consumed. Hence it doesn't make sense to talk about yield of the HBM itself, because it is always part of an integrated product.
I don't fully understand the source of the "total bytes" constraint, but a major factor may be because HBM4 / HBM4E can only make use of the footprint directly above the processor/logic die (or in direct vicinity of its interconnect), while traditional DRAM can be placed further away where there's lots of real estate on the motherboard.
I gather a practical max ceiling today is a stack of 16 chips in height yielding 64GB?
These chips have a massive bus size of 2048 bits, instead of the 64 or 128 bits (dual channel) used by DDR5. That's what gives them their order-of-magnitude bandwidth speedup. But even though they technically pack in more capacity per square millimeter of motherboard, I gather they take up more space than older technologies once you account for the vias and interconnects to route all those signals.
I think people might prefer the lower idle power consumption from lpddr over the better bandwidth in hbm in battery powered stuff. That said right now the price is definitely preventing us from finding out.
If they're talking about production capacity, that is some product of die area and process steps, right? It doesn't have to be 3x die area, just 3x lower factory throughput for the same number of functioning memory bits.
The advantage of HBM over regular non-stacked DRAM is memory bandwidth, which also requires a super-wide memory bus - 2048 bits wide for HBM4. Compare that to the 128 bit wide bus of a modern CPU.
So to take advantage of it on the desktop, or anywhere else, you need that 2048 bit wide bus, and a processor capable of consuming 2-3 TB of data per second!
These are not normal requirements, other than for a GPU.
Mainly bus width. Afaik HBM is like 1024 bits vs DDRs 64 so you need lots of transfers in parallel to saturate the bus, and CPUs kinda want 64 bytes of data as that's the size of a cache line ASAP. So you need a ton of in flight transfers which isn't a thing CPUs provide, maybe multicore workloads.
Buy the way you win with CPUs is with latency, and not bandwidth, which is why Apple M series actually uses DDR with lower latency because of the stacking.
Even if the cost were the same of the RAM itself, you'd need much bigger and more expensive CPU to deal with it.
Running 17 chrome tabs doesn't benefit at all from that HBM and all the additional hardware+software complexities that come with it. You want a specialized coprocessor to handle specialized workloads. The GPU exists separately from the CPU for a reason.
HBM isn't dramatically better in every dimension. You get huge bandwidth and good energy efficiency per bit transferred, but not necessarily a meaningful latency improvement and capacity expansion becomes tied to the package
On a related note, I was reading yesterday that apparently the real bottleneck for Chinese production of AI accelerators is HBM production, not processors or ASML equipment.
The lack of ASML EUV machines certainly hurts, and pushing DUV so hard results in abysmal yields of good chips, but you can compensate by running more wafers or making smaller chips, and the net result is that Huawei's Ascend production volume is limited by CXMT's HBM capacity not processor dies.
The problem is that HBM manufacture requires many steps (die thinning, via drilling, plating, alignment) where the equipment used by everyone else (Samsung, SK Hynix, Micron) is also blocked by sanctions, so the Chinese are having to develop all of this themselves too, which they have, but yields are currently low, even when using shorter HBM stacks.
According to SemiAnalysis, both inference and post-training (RLVR) is mostly memory bandwidth bound. Only pre-training is compute bound, but it now only takes a small share of overall data center capacity.
Not quite, it's got quite a bit more complicated with agentic use cases.
Prefill (input tokens) is heavily compute bound. And the ratio of input to output continues to rise, as typically in agentic sessions you have a few tokens output for a tool call and (many) thousands of input from the tool result.
Then you have cached input tokens, which is a totally different issue, system RAM or NVMe bound.
Obviously output tokens is VRAM memory bandwidth bound, but this is less and less of the bottleneck these days for overall agentic speed.
I can easily use close to 100 million input tokens a day. A few million output tokens but at the end of the day maybe a thousand or so lines of code get written.
This is why I was careful to specify tokens per second not time to first token which is the prefill step you’re talking about. Clearly to run these models well you need both but as I said adding more chips or compute units can give you more latency where as overall memory bandwidth (throughput) is limited by access to fast memory.
The two challenges there are firstly - that analog design has been a separate electrical engineering school for most of a century, so there are few who could design it - and secondly - that every single chip will have subtle variations in its computations, necessitating some sort of model finetuning per chip. Possibly the chip could be characterised at the factory, and ship with the characterisation data burned into a controller rom or something, but if that doesn’t pan out the whole thing is likely a non-starter.
If it could be made to work, you could run a fable-grade model in tens of watts.
This made me smile. A true test of human-like intelligence. Maybe they could take some entry level jobs (“I can make a nice Wordpress site”) and build a CV?
The only thing the West is achieving here is that sooner or later China will be able to compete on their own terms rather than ours. It may buy some time but the end result is very predictable.
Over the last 20 years the economy has become dysfunctional. It no longer really resembles a free market; monopolies have established barriers to entry everywhere. And the biggest investors are awash with helicopter money that's been doled out for favors by the political class.
So those investors have tons of cash to burn and surprisingly few opportunities. Even within Silicon Valley/VC there are surprisingly few who seem to really understand the fundamental economics of software. Or perhaps those economics just aren't that important when you have billions of dollars on hand and cash is obviously not going to get you a return. Any whisper of possible exponential growth is worth throwing money at. Crypto? Why not. AI? Why not. Datacenters? Why not. Tulips? Why not.
This is by all definitions an empire in decline. Everything is broken or fake. Everyone is afraid to do what needs to be done. Power forbids it. So we're all just waiting for the other shoe to drop. Our secret police aren't as bad as the late stage USSR's yet, but hold Uncle Sam's beer...
(That's not a recommendation to try and time the nadir, by the way, as it could easily be 50 years away.)
It started way before the last 20 years, the past 40 years have created the conditions for this to happen.
Overfinancialisation of the economy while removing safeguards to keep markets functional, like anti-trust enforcement, are the main forces behind this erosion. Through finance the focus of companies is completely shifted away from producing good products and services, and into how to extract more paper wealth from existing structures to the detriment of products.
Not enforcing anti-trust and letting behemoths to form which cannot be competed against since with their amount of capital they can either buy their competition outright or just price dump for long enough to make competition non-viable.
It's the failure of neoliberalism, and that agenda has been pushed into Western countries since the 1980s-1990s, it made enormous wealth for the few at the top while eroding whole societies, economically and socially, it's all dysfunctional.
I feel like my point is being over extrapolated a bit. All I was saying is that people in 2020 believed AI companies had much more of a moat then they actually do - that other companies would struggle to obtain enough data (y’know, downloading the internet to train models wasn’t legal back then) or resources to compete
I replied to the extrapolation, not to your comment directly, I think your point might be more fitting under that comment than mine, I just drilled down from the points above mine.
The initial growth of the internet was a boom bust cycle, and as you see the internet is still here. This is more about the economic paradigm that is being used to grow a technology versus the economic paradigm the technologies rate of growth can support over time.
Just to give some rough numbers in the past 5 years the amount of GPU compute installed in FLOPS is somewhere over 5 times all the CPU flops that have ever existed.
This has nothing to do with AI being good or bad or being able to produce things and economic value. It is by far the largest and fastest growth of any technology ever and we have zero clue about the economic stability of this grand experiment we're performing.
The internet is a much more appropriate comparison than crypto or tulips, which bring no or negligible value.
Some differences to the market at that time seems to be that during the dotcom bubble, many of the companies had little to no revenue.
Leading AI labs already generate enormous revenue. The investments into capacity are needed to address the current demand.
The situation seems somewhat less speculative.
That said, I cannot predict how AI capabilities will develop and how demand will respond.
Should capabilities plateau hard and soon, maybe the demand will not be there for the compute investments.
If it does not, and instead AI applications in robotics, science, and self driving expand, chances seem reasonably good that the demand will be there, no?
And as for the economic stability, much of the investment comes from existing giants like Microsoft, Alphabet, Amazon, and Meta, who have the necessary cash flow.
These companies are less likely to collapse than some of the ones during the dotcom bubble.
Eh, I'd say it's closer to something like internet + tulips.
It's the total amount of money in the economy that's been invested toward a potential outcome. AI represents the largest amount of money, and largest fractional part of the economy invested ever.
Because of this AI could be the biggest economic boon ever, yet still not recover the full amount invested. This will have deep economic impacts that affect everyone and everything. At this point AI must achieve all its stated economic impacts or there will still be a huge economic crash that kills off any company that is over invested and cannot make a profit.
Worse, the many of the perceived economic impacts of AI are not for humans like you or I, but the huge companies you listed. Even if they economically win, everyone else made out of meat could still lose.
They say history doesn't repeat, but it does rhyme. This, at least to me, sounds like a mixtape of "internet" + "tulips" + "1920s financial world leading to global political instability".
Every potential outcome I see occurring pushes us closer to further instability, even if the economics on it work out on paper.
Out of this number how much of this goes to payroll that goes to humans versus how much of this goes to non-human infrastructure costs. The numbers suggest this is anywhere from 60 to 90%, with 70% being a reasonable average figure. Just under a trillion dollars paid directly to humans would disappear if AI somehow captured all of this market.
Moreso the AI industry needs to capture these gains very fast or they will be crushed by interest payments on the massive debts they've accrued.
>ven though Amodei hopes to cure most major disease in the next 5-10 years
The average development time for medicine is 10 to 15 years before a single dollar is earned from said medicine. AI will shave very little off that as human testing and our stupidly complex bodies introduce all kinds of problems. Furthermore running head long into blindly using AI medicine is how you produce X risks from super intelligence AI.
The speed at which AI has to develop in which to get profitability is the biggest risk, a potentially catastrophic risk at that.
You can theorize all you like, and many people like yourself are expecting to see GDP growth pick up as a result of AI, maybe if only because of the boom in datacenter construction etc (regardless of whether AI ends up actually boosting the economy), BUT ...
The reality is that so far there has been no sign of GDP growth picking up. It is basically flat at 2.5% +/- over the last few years.
In a similar vein one might have expected that the internet (think of all the e-commerce and efficiencies!) might have shifted GDP growth into a higher gear, but it did not, although in that case there was at least a significant boost in the 1996-2000 "dot com" era when the build out was happening (then to be followed by the crash and all the unused dark fiber etc).
So, maybe the hoped-for AI boom will be just as much of a dud (as it appears to be so far) as the internet boom. New day, different tools, same growth.
It's perhaps odd that we're not even seeing datacenter/etc build out register on GDP, but perhaps the scale of it is not as large as the internet build out?
>but perhaps the scale of it is not as large as the internet build out?
In the past 5 years 5 times as much GPU compute capacity has been installed as total CPU capacity that has ever existed. When it comes to capital intensity the growth of AI has been one of, if not the largest capital expenditure for a class of project in this short of time frame. Things like the Manhattan project which was absolutely monstrous expense at the time is dwarfed by this.
Of course measuring GDP growth is difficult when you have an idiot in chief trying to destroy the global economy with the dumbest set of actions contrived by a human ever. Disentangling this factors will take a lot of work on someones part.
I feel like my point is being over extrapolated a bit. All I was saying is that people in 2020 believed AI companies had much more of a moat then they actually do - that other companies would struggle to obtain enough data (y’know, downloading the internet to train models wasn’t legal back then) or resources to compete
No longer strangling their own economy would be a good start. Take a look at the magnificance that was the ESRS. A 300 A4 pages long list of yearly reporting duties. And the best thing is: Just because they're no longer nicely summarized in one place, doesn't mean the underlying duties went away.
> It baffles me how shortsighted the policymaking here is. Like, what did they expect to happen?
As someone who is myself 'old af' by tech standards (I'll be 53 very soon) the main problem is that the policymakers are all 70+ years old.
Their concept of what China even is is forever stuck in pre-2004 thinking from back when they still had significant neuroplasticity.
Vote these fucking fossils out, kids, before it is entirely too late.
(And, yeah, before someone inevitably brings it up, some exceptions-that-prove-the-rule older policymakers/politicians do avoid this trap of getting stuck in the past, but most don't)
As other comments pointed out, that was always China's plan, the value here is really that when the Chinese overlord annillate the world's industrial competition, they'll have the moral high ground of being out of self-reliance and hostility by western monopolists.
But none of them had a government-mandated one-child policy (maximum, not minimum) for over three decades. That's a pretty major difference between China and neighboring countries.
And yet all of them have an even lower fertility rate than China.
On the one hand, you're proving my point. On the other hand, if you think that China's headed for a demographic disaster, so are all of those countries, and your thesis of blaming the CCP for everything... Falls a bit flat.
China has extremely low fertitily rate given that they haven't yet reached the status of a developed country (which Taiwan, SK and Singapore have reached).
Developed countries tend to have lower fertility rates (i.e. less kids per family) than undeveloped ones. Comfortably off, well educated people tend to put thought into whether to have kids or not. In poor countries people tend to have large families so they can send the kids out to work.
Having a high fertility rate isn't "high status" and a sign of a country's progress, but rather the exact opposite.
Of course this is separate from this issue of having a shrinking population, which is a sign that your citizens are not optimistic about the future, not a positive thing.
China is headed for a demographic disaster, though. High schools are incredibly packed, middle schools, not so much and primary schools and kindergartens are kind of empty.
It's hard to say that it's stupidity (at least not country-specific stupidity) - it seems that globally, presumably due to the state of the world along many axis, many people are choosing to have fewer children, and it's happening on a large enough scale to have major consequences for things like the size of your tax base, trying to keep your economy growing, demographic concerns over an aging population with no-one to look after or pay for them etc, etc.
Different countries are handling this in different ways, with encouraging immigration being the major one. Japan is facing a major lack of workers, with many businesses having to shut down as a result, but have decided to tackle it in a different way relying on advanced robotics as an alternative to immigration.
There are very few developed countries NOT having this problem. Most of the population growth is in poorer parts of the world like Africa.
Well, if there are >0 developed countries that have solved this (Israel, the GCC, and dare I say fairly well-off countries like Jordan and Algeria) then this proves wrong the hypothesis that modern hyper-development must inherently mean population collapse.
Going purely by the evolution theory and natural selection, having so few children that there is a serious risk a population will disappear in a century or two, means losing. Needless to say, this is stupid. From the entire American continent, to Europe, India more recently, Thailand, China, the Koreas, Japan.
Africans, Arabs, and more broadly Muslims will carry the torch. Much respect for not falling into the trap.
> then this proves wrong the hypothesis that modern hyper-development must inherently mean population collapse.
Honestly you're reading the data completely wrong. You cannot look at it at the country level. In Israel for example the populations that are producing the most population are highly religious and less educated than those that are producing less population.
It's more about memes. Religion is a meme driven breeding cult. This is why said religions tend to have very long term staying power. The particular problem here is these religions come at a cost of having to ignore a lot of reality to get to that point. Because they are faith based anything that presents a direct challenge to that faith must be setup as an enemy or it may risk the religion. Yes, religions can incorporate new information but it tends to be a slow process as it needs to involve slowly changing peoples mind to avoid instability.
What this looks like in the long term is large populations of highly religious that have a fundamental disconnect from the technology that's keeping them alive. A new Late Bronze Age collapse.
So? How does the population being religious disprove the fact that they are developed and having children? Not to mention that I have listed around ~10 countries after Israel.
Uh... Meme-based? This is such an unserious argument I don't even know what I can possibly say to that.
How are they disconnected with the technology keeping them alive? All people, in GCC countries especially, are working to make 1st world development happen.
Implying that religious Jews or Muslims are stuck in the Bronze Age is ignoring reality. Most European development itself came from men of science who were Christian, I thought this apparent conflict of ideas was something taught in schools.
First, you seems to think a meme is something unserious. Read Richard Dawkins The Selfish Gene so you know what meme actually stands for rather than the thing you see on Reddit.
GCC countries are resource economies mostly. They must weave a very narrow path to success to non-oil based means of income or the entire region collapses. I will say they do realize this is a problem and are using oil money to diversify, but resource curses are very hard to break and collapse can happen very quickly.
"My grandfather rode a camel, my father rode a camel, I drive a Mercedes, my son drives a Land Rover, his son will drive a Land Rover, but his son's son will ride a camel." --Sheikh Rashid bin Saeed Al Maktoum
>Most European development itself came from men of science who were Christian
This is just stating that that in 1700 you were a Christian or you got hung (hell, pre-independence America was founded by people that thought that too few people were being hung). As we see when you take out the hanging or social shame part of the religion people tend to flee them as they are oppressive structures. You could say them being men of science either had no bearing to their religion, or was in spite of their religion.
I am saying, for example, that the Abrahamic religions are bronze age (ok, sorry, Iron age) social structures. I don't think studied scholars are going to disagree on that point very much. People saw how people worked and could be sheppereded in societies without formal education and turned it into a series of stories that were effective in modifying peoples behaviors. Kings later came along and added their stories so they could further control the population. Over time a semi-coherent story about how people worked was formed below a set of rather incoherent stories regarding things that didn't happen that people are supposed to accept with faith.
Notably, it is hilarious how ‘modern western society’ seems to believe that people somehow work differently than they always have, while clearly mimicking the same rough structures they are deriding.
(Comic book superheros, cough cough. The references to Canon in what are essentially liturgical disagreements - I mean power scaling arguments - are particularly hilarious.)
Not me of course, I’m different and unique. Just like everyone else.
Algeria and Morocco hardly seem "fairly well-off" countries, considering how many of their citizens prefer moving to the low-birth rate European countries you loathe. High birth rate of Israel is also unsustainable from economic perspective, as it relies mostly on orthodox Jews who on average are much poorer, less educated, and contribute less to the Israeli economy.
I'm not saying fast fall in birth rates isn't a problem, it obviously is. However, nearly all countries that don't have this problem offer much worse quality of life than those which do.
Young southern europeans are in turn emigrating north. Are Spain, Italy, Portugal not well off?
There is a saying. More is better. Morocco has issues with tourists and diaspora pricing out locals from houses to food even, but Algeria is an honestly good country all around. With its issues of course. But there is more to life than earning $$$.
Westerners forget the tiny detail that these countries are extremely young unlike them, hence why it feels like there are so "many" people.
Edit: oops, I glossed over the part about Israel. Trust me, it's still a billion times better than nothing, which is the case for the entire West.
> this proves wrong the hypothesis that modern hyper-development must inherently mean population collapse
I don't know the exact correlation but even in Morocco, total fertility rate is declining very rapidly, down from 6 children in 1980s to now around 1.7 in urban areas. UAE (Dubai) has similar TFR to Belgium or Portugal.
Still, a stupid governmental policy cost them at least 100 million people compared to the expected spontaneous development. Probably more. That is the sort of punishment that omnipotent governments sometimes visit upon their folk.
I don't believe that even the CCP itself, in hindsight, considers the one-child policy to have been a good decision. At the very least it should have been ended some 15 years earlier.
Well, I've been reading on The Economist how China is seriously struggling with youth unemployment right now. Lots of fresh university graduates are sitting unemployed, or doing manual labor. In my home country (Finland) we have had way below replacement birth rates for a long while now, yet youth unemployment is at all time high.
The problem with low birth rates is assumed to be that there are less workers to take care of the pensioners. But right now it seems like even the few young workers available often aren't needed anymore, thanks to increasing automation of industries. I predict youth unemployment will be made even worse by AI soon enough. So, why do we need children, if capitalist economies don't need workers? Unemployed people do not pay much taxes to sustain the non-working age population.
I believe low birth rates might become a blessing soon enough. It just means less people for governments to take care of when robots and AI take away most jobs.
No, a remarkably farsighted policy that prevented PRC from having to deal with additional 200m-400m subsistent farmers. If anything it wasn't done sooner/aggressive enough. Like you can literally wipe away 200-400m from PRC button quantile and PRC would be net better off due to increase per capita resources and wealth. This hindsight will be historically obvious in 20-30 years when you see the outcome of India with 1.6B people with 1.2B poor, vs PRC with 1.4B people with 400m poor. The drag of having disproportionate, bluntly, worthless bodies is not linear for governance, and it's not because India hasn't tried aggressive family planning of their own, they simply failed unlike PRC. Who very much regards family planning as largely successful demographic management system that enabled proper allocation of resources away from generating peasants to building industry and the largest pool of high end human talent in recorded history, where frankly any tertiary dividend is worth 50 farmers in terms of productivity.
The official one child policy didn't actually have much of an impact. Birth rates had already fallen a lot, despite the official "lots-a-children" policy.
The one child policy is gone now but birth rates haven't really moved upwards.
So China has similar family sizes as the US that had no one child policy.
It's difficult to near impossible here to determine which counterfactuals would actually hold true. Based on other world wide trends it's very likely that Chinese family sizes now would only be slightly higher than if the policy didn't exist. In the modern world there are just a huge number of different factors working together to decrease family sizes.
Of course it was devasting for some families. That is never the same thing as saying that it affected China's birth rate much (which it didn't).
There were forced abortions (sometimes very late), there was forced contraception, forced sterilization (I think), and lots of punishments for people who broke the policy.
There were also others who got around it fairly easily by paying a bribe and by not registering their "surplus" child/children (especially if they were girls). That of course caused some problems down the road.
That still doesn't mean that it affected China's birth rate much. It was already low (and had fallen steadily) when the policy was instituted. It also had plenty of exceptions for rural China and ethnic minorities.
Eh. Industrial policy in a big country is always the output of politics between stakeholders, anthropomorphizing it to a "guy with a plan", especially an "overlord" looking to "annihilate" is probably getting it wrong.
A friend of a friend is married to a member of the national academy of sciences in China. I heard at dinner a few years ago that he had been lobbying for more onshoring of chip production for years before the Nvidia restrictions, and always heard "you're right, maybe next year we'll be able to fund that". Then the restrictions came in and everything changed.
Same in the EU, it is not just laws it is companies changing their dependencies from US partners. We do not have the same pressure as China feels. The US administrations have always been more national navel gazers, but the Trump civil war of America really made it worse.
And they are more about sounding good than about reality. They also affect a much smaller share of the economy than they used to -- which is very good! The original five year plans caused nothing but misery and starvation.
I’m pointing out it’s disenguous to say ‘it’s not like there was a plan’, when there was explicitly a plan with that title on it. Especially if the plan didn’t work out well.
It's disingenuous to characterize "don't anthropomorphize the political process" as saying no planning documents exist as part of that process.
And, to the point I was responding to originally, we can check those documents! They don't speak of "annihilating" other countries' industrial capacities.
>They don't speak of "annihilating" other countries' industrial capacities.
Why would they have to?
When China talked about their 5 and 10 year plans on solar panels there was no need to talk about putting the rest of the competitors out of business. That was a highly probable end goal of what they were doing, no need to state it.
It would be like Walmart stating "We're going to put out massive loss leaders in sporting goods to put X out of business". Instead they'll talk about growing market share and long term customer loyalty.
More of, China is smart enough to use the benefits of both capitalism and authoritarianism to grow their country while western governments are busy stealing the furniture and fixtures off the walls.
False dichotomy. the problem is far deeper than that, and wanting to simplify the problem to those two extremes belies a belief that the world could and should be able to be simplified into two power dynamic. It is not. The world, and the powers that fight within in, are far more dynamic than a simple communist or capitalist spectrum, and attempts to simplify the world along those lines is only going to hurt you.
That just means your stakeholders argue twice: once when drafting the five year plan and then again, later, saying their policy position is more aligned with the spirit of the plan.
> [...] that was always China's plan, [...] they'll have the moral high ground
It's a possibility. It is also possible that they will steal yesterday's technology and still pretend to have moral high ground.
I do not wish China and Chinees wrong but most probably communism will work as it always do - by faking thing and bringing self-harm to their own citizens.
It’s the same in communism/socialism except people are prevented from stopping everyone else from stealing their stuff, in the name of the ‘greater good’, and if they try to get more than everyone else get guilt tripped into giving it up under the ‘to everyone what they need, from everyone what they can’.
Which incentivizes pretending you need a ton, and can give nothing. You think being useful at work in Capitalism is bad? Wait until you are accidentally useful in a communist/socialist economy.
There is usually less pretending under capitalism, but people are people, so it all the same stories still play out.
People are shitty, if given the chance and influenced in specific ways. Capitalism or not. We’re dealing with the current bullshit in the west because somehow people bought the line that everyone is a good person - and we totally can’t actually see or do anything if we see someone doing something bad! - and we need to always follow ‘the rules’ even when someone bad clearly isn’t.
> In free markets, everyone steals everything they can, and stops everyone else stealing from them as much as they can. This is normal and expected. The ideology says that it will even out and converge to a fair situation.
This is all nonsense. Capitalism isn't an ideology, it's just a slight formalisation of what naturally arises. People are free to choose; they can own things; they can make agreements; they must honour the agreements or pay a penalty. Capitalism in no way is about stealing.
Our LLMs were trained on the widespread theft of copyright content, with clearly defined and well tested law. But they made capitalists too much money so it became defacto legal.
Next you'll be telling me fairy tales like "laws are real".
If you're paying money for something you could get for free, you're a sucker. Ordinarily the person who is selling it makes sure you can't get it for free.
China's plan to be self-sufficient in semiconductors was a failure before western sanctions, because all the Chinese chipmakers found it too easy to buy western chipmaking equipment or to fab through TSCM. No Chinese company wanted to buy from other Chinese companies. Nobody was incentivized to develop self-sufficiency. "Because the government wants it" was not enough incentive.
Western sanctions have succeeded in achieving China's self-sufficiency goals where their own government failed.
people have been claiming Nvidia will get competition too for years but you guys seem to be forgetting that the current leaders are not asleep at the wheel in the meantime.
Well Soviet Union tried to be self-sufficient, but the truth is such strategy loses to global free trade. Whether we want it or not, global trade is what keeps the world from WW3 for now, not the nuclear MAD.
Yep, exactly. US is in the active process of shooting itself in the foot. My point is that China is very dependent on resources imports, so there's no way for them (and any other country) to be self-sufficient or they will stagnate.
And China is very dependent on imports of almost everything. Maybe some rare elements, but e.g. oil is a big part of it (they agressively diversify off it, but it's still a large piece). Global interdependency is a good thing.
Sanctions can absolutely create the incentive to build a domestic supply chain, but incentive doesn't automatically translate into catching up on the same timeline
What the West is doing is forcing China into a situation which is impossible to sustain when their economic dividend runs out. Especially considering these investments are fuelled by an extremely high level of debt.
And it’s not a given that they’ll catch up. I think it’s fair to say that their development of jet engines is on track to be obsolete before they’re competitive.
They may in principle have established the technologies to be self sustained but that’s only relevant and sustainable if they develop an economy based on domestic consumption rather than exports. Which they are struggling with as well.
i dont think it will - they still have a chokehold on battery production (which implies dominance in the EV sector), and they also control the vast majority of all rare earths. The chinese gov't can subsidize chip research/development with these other sectors that are dominating.
The west is the one that need to watch out for their own economic dividend running out from the past century.
"Economic dividend" really means "Demographic dividend". It's when your fertility rate decreases but your total population isn't decreasing yet, so you get more people to work more as they spend less time as parents. This of course is a self perpetuating cycle and ends in demographic collapse. China's population is now decreasing. The West's population is still growing. China is in the most literal sense dying. It's what you get for having a one child policy for 50 years. But hey, for a beautiful moment in time they really created a lot of shareholder value.
What makes you think they don't have a domestic market?
The Chinese domestic EV market is the largest in the world.
Chinese factories are full of Chinese robotics.
Companies like Huawei are struggling to meet the domestic demand for AI accelerators, while growing volume very fast year over year.
The Chinese domestic AI market is massive, but we just don't hear much about it outside of the models that are also sold here. Can you even name the models that ByteDance (China's Meta) makes?
As far as exports go, China is a low cost producer for many things. If the US doesn't want to buy high quality EV's costing 1/2 the price of a Tesla, then there are plenty of other countries happy to do so. The US is only 5% of the global population - there are lots of other consumers out there.
China has had a very strong incentive to recreate lithography and fab equipment for years, but a lot of the specialized HBM packaging chain was probably much lower priority until relatively recently
The "abysmal" yield using DUV is an overblown statement by western commentators who don't look closer at the development. Certainly yield is worse than with EUV, but after multiple iterations of development, yield has become pretty good, within economically acceptable bounds, still making scaling possible. Volume is still ramping up. A lot of capacity will come online in 2027. The removal of western middlemen such as Mediatek actually improved the economics. Further yield improvements are still coming. Volume is already so high that domestically made phones and NPUs are making a real impact, yet are also selling like hot cakes.
It depends on what you are trying to make, but there are workarounds for most things, and it becomes more of a cost and volume issue than a showstopper.
If your yield is low and you still want the volume, then run more wafers, but then you need access to more DUV machines and more wafers. If your defect density is too high, then design smaller chips with a higher chance of avoiding defects, which is what Hauwei are doing - split processor into multiple chiplets.
HBM3E usually has 5-6000 vias, but doing this with multi-pattern DUV without defects is tough, so CXMT currently drop that to 3000, at a cost of some loss of thermal and voltage stability.
HBM:processor production ratios aren't what Huawei would like, so they mitigate this at system level by putting an optical memory bus on the GPU chip and sharing memory across the system.
Not everything is a huge LLM - smaller models like recommendation systems don't have so many parameters and need so much memory, so why waste HBM on them? ByteDance use their SeedChip acelerator for this, currently made by TSMC (using an older sanctions-approved 28mm process), which instead etches dense RRAM on die beside the processor.
There is also a time dynamic to this, with China still using pre-sanctions equipment and chips as their domestic alternatives ramp up to replace them. One interesting part of this is HBM ... HBM is very demanding to make, and everyone had problems with it, with initially only SK Hynix being successful. Samsung and Micron took a year or so to catch up, and during this time Samsung had made a ton of HBM that didn't meet NVidia's specifications, so ended up, pre-sanctions, selling it to China, where it has acted as a stockpile to carry them over as CXMT's domestic capacity ramps up.
China seems to be doing fine. Of course they would like sanctions lifted, moreso for HBM than anything else, but all that sanctions have really achieved is accelerating their semiconductor independence. They are still building 1T+ SOTA models, standing up 100K clusters of domestic AI accelerators, etc, etc.
This competition will be great for us consumers for a while. Until China undercuts everybody else for long enough to essentially monopolize the market. This pattern has repeated itself across many markets already.
The solution would seem to be preventing that "Buyer (AI) has crazy money". They already have borrowed billions and are only generating losses so the money should run out. It just needs to be made sure that nobody (taxpayers) will rescue them.
I'm sure you can make it work with HBM chips on adapter boards for PC. And there should be lots of HBM chips from scavengers taking apart bankrupt data centers.
HBM dies are designed to be physically mounted on top of other dies in the same package, they can have up to 2048 data pins (compared to 8-64 on consumer dies which are in their own packaging) - to do what you propose you'd need an interposer chip of some type which would push the price back up
I’ve often wondered about die thinning. I mean just looking at sd cards and micro sd etc, it was clear that there had to be a thinning step (or many steps) and it seemed amazing to me that such a complex step could be economically viable. And yet it clearly is. All a matter of being able to operate at scale I guess.
It’s not a step I’ve seen discussed very much, so it was kind of neat to see it discussed front and center in a lay article like this.
Yeah, the scale is kind of mind-bending. You're taking something that's already extremely fragile, grinding most of its thickness away, handling it through several more process steps, and somehow doing that cheaply enough for mass-market products.
The critical back end step for HBM is thermal compression bonding - where they actually compress 13 very thin die separated by solder balls. Cracking is a major issue.
We are getting so many technologies like this that are so insanely specialized and require such deep knowledge of physical processes that some event like a 1918 flu that kills 10% of the population could take out enough people that we'd be unable to make things like this for years as we fill in the missing knowledge workers.
It's just crazy how far down the scale we're tickling atoms these days.
Not really at all. Between those dates and now the world has become fully global and concentration of knowledge has massively occurred. The past world was much more highly distributed, far less technical, and far more redundant.
Not dissimilar to lens or mirror making, the process of grinding to a precise flatness. The key step here is bonding it to a glass carrier, presumably because otherwise it gets too thin to support itself.
See also the "reduced kerf diamond wire saw": technology critical to cheap solar panels is the ability to saw the boule into thinner and thinner pieces with less waste at each cut, like a big ham slicer.
It matters so that average people have a chance to own their means of production, and that the current hardware-grab by entrenched capital doesn't get to charge such high rent.
Frontier AI labs are customers just like the average Joe down the street. The only difference is who can bid more. Average Joe probably doesn't have millions or wont buy millions of chips.
There are no privileged consumers that have "dedicated" amount of production, no matter the demand.
I mean... You can buy a stick of RAM with Samsung chips on it and use it in your own machine to run a local model vs. paying for an Anthropic subscription. I assume that's what they're referring to, not owning Samsung stock.
I have read an interview with Acer's CEO claiming that the shortage is already over.
He alleged memory maker's public announcements of the shortage lasting throughout the end of 2027 are attempts to covertly signal the others on pricing strategy.
With additional production capacity ramping up this year and next, this seems somewhat plausible.
If manufacturer's inventories are filling since they advanced orders when prices were rising, there could be an oversupply of memory when they don't place new orders at the higher current price.
This could quickly result in a price war while capacity expansion and yield improvements create irreversible oversupply.
To me, the most idiotic thing is that the world currently cannot build power plants fast enough to even use all the HBM that is being produced. That is, a large portion of HBM and cards are sitting in storage, waiting for the data centers and the power plants to run them. The FOMO in the AI world isn't just in the stock market
I was wondering how much financial value they put in one truck. Presumably the value density of this stuff is incredibly high (although not really to anyone else, so security probably isn't a huge issue). At what point do you split the shipment between multiple trucks to avoid huge losses if there's a crash?
When I worked on satellites we had some electronics which used a radiation hardened FPGA which cost something like $500k per chip. Carrying that around was nerve wracking.
I once had to keep a $20M (technically; a million download codes for the same $20 movie for a promotion) thumbdrive in my home safe because it arrived too late to put in the safety deposit box like it was supposed to be.
I had to remind myself a few times the practical value to anyone was about $22.
Enterprisey client chain-of-custody requirements. It was couriered to us, and the courier was late. Cue panicked phone calls to get an exemption to the contractual requirement for overnight storage at the bank.
I worked at a [Not to be named Country]'s monetary authority. Part of everyone's duty was to receive shipments of newly printed cash from De La Rue and get it into the vault before it was distributed to banks. Seeing millions of dollars packaged in cardboard boxes being moved from the street to the vault with men posted up in the bushes with machine guns was something I will never forget.
It's funny how rote and regularized "playing" with money gets after a while though.
I used to work for a financial institution small enough to sometimes need someone from the back office to help fill in as a warm body for dual-control when we received cash shipments.
It only took a couple sessions in a vault to get entirely bored with having a half million dollars in front of you.
My first serious software was for a sizable goldsmith factory. They keep an incredibly low profile visible from outside, and they get really good at it. Once inside, the amount of gold that's moving around is jaw dropping. After a while, most of the time it feels like aluminum, or any other material. All you see is cuts to be made, welding to be done, carvings, sandblast, etc... It's work and responsibility and everyone is just trying to get it done and make it the problem of the next person.
It's treated the same way as any other generic shipment of cargo, really. It's fine throwing $20 million in one truck.
Security is an issue, though, these are treated as a high value load and typically would be followed by unmarked, plain clothes armed security guards tailing around the truck. (That's how Apple often does warehouse deliveries of trucks full of iPhones. It wouldn't surprise me if there was going to be some rolling lead soon following some trucks full of Duos...)
They probably have insurance which means it doesn’t matter if it is all in one car or not. And even if they didn’t Samsung is big enough where a loss of a couple million dollars of components isn’t gonna bankrupt them.
Ha - I still remember when I worked on the IBEX-Lo Sensor, we had to hand carry it to Switzerland. One lucky member of our team got to fly first class with it because it wouldn't fit in a coach seat!
My buddies company shuffles chips around the west coast, some deliveries are a few million dollars. They shove the chips into the truck, then shove a giant movable wall, leaving about an inch or two gap around the perimeter.
The truck has multiple drivers and private security.
I had a cousin that worked private security, and for a few years (I think it was early 2000's) his main gig was to work as a team of armed security, and follow trucks from the two Intel campuses in Oregon to the airport in unmarked vehicles, get on the cargo plane with the cargo, and fly to the location where they did the packaging of the wafers (I want to say it was the Philippine Islands) and then fly back. Lots of overtime.
Almost 10 years ago at my university in Chile. My professor of Computer Networks invited a PhD Student that was working on the ALMA Observatory [0] doing some research on signal processing.
What stayed with me from that talk was that all the data that was captured from those massive antennas was packaged in boxes, loaded in a cargo van and travel 1.600 kilometers to Santiago to be uploaded into the systems.
I wanted to mention AWS Snowmobile [0], but after looking it up I found that apparently it was discontinued [1]. It seems that nowadays there are more economical/efficient alternatives (I guess theoretically you could still beat these alternatives in certain metrics, e.g. by using more trucks at once to get more bandwidth).
Right now, every company that can is ramping up their RAM production as much as humanly possible. There is such outrageous and growing demand, that it's like printing money if you can manufacture more RAM chips. And that massive financial incentive is driving huge investment in manufacturing capacity.
But someday, that demand will stop rising.
At some point, most people will have enough RAM. And they just won't need to buy more.
The inventories of RAM will become overfull, in warehouses and stores. And suddenly the manufacturers, middlemen, and even retailers will all find the price they can sell at keeps dropping.
But you can't shut down a huge manufacturing facility overnight. You can't shut down a years-in-the-making retail distribution system overnight, either. You can't even shut down a facility that is halfway finished with construction. They'll keep producing and selling RAM at lower and lower profit margins, until the day comes they finally accept they're losing money with each sale.
And when that happens, we'll all have more RAM than we know what to do with. It'll be cheaper than we have all ever experienced in our lifetimes.
I don't expect this to happen soon. But demand cannot keep exponentially rising forever. There will come a point, probably in the next 5 years, when demand at least levels off, even as production capacity is still rising.
And it will be interesting to see what's possible at that point. We'll be able to create software that is not doable right now, because it will no longer be constrained by memory. And even cheap laptops will massive amounts.
When you commoditize intelligence, you dramatically lower the cost to create. The amount of electronics being created is... incredible. We've really seem to have hit the asymtote part of the growth and i don't really see it coming back.
Everything is moving to edge compute. From smart rings and watches, to robotics and drones, iphones and humanoids, and no end in sight to the scaling improvements in AI... It does seem that you can continue to get more emergent behavior the more compute you expend in pre/ post training. There's no reason we can't scale up to a million data centers in space, as long as you get dramatically improved intellegence.
289 comments
[ 2.2 ms ] story [ 90.1 ms ] threadLong-term contracts, not building new fabs - they're in full on hedging mode right now.
Even over their many years, projects like the Manhattan Project, the Apollo Program, or the U.S. Interstate Highway System never added up to that. [2]
Is your argument that there is insufficient AI spending at present as they aren’t also taking on building their own fabs?
[1] https://www.pwc.com/gx/en/news-room/press-releases/2026/glob...
[2] https://www.aljazeera.com/news/2026/2/19/visualising-ai-spen...
We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.
Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
But I suspect returns may have already diminished into negative territory for at least some other use cases. One of my least favorite job responsibilities in this brave new era is figuring out how to avoid performance and behavior regressions when an older model were using for some application reaches end of life. It’s getting uncommon for me to look at our benchmark results and say, “Oh, good, it does better on one of the newer models!”
Is it just that the providers are generating tons of synthetic datasets on coding tasks so that the models get more exposure to the right thing to do? Every time someone points out an LLM stupidity they add some training data to patch over the weakness (trivial to generate "there are two 'l's in llama")?
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C23&q=llm...
https://proceedings.iclr.cc/paper_files/paper/2026/hash/3b4e...
It’s also poor reasoning to you only pick one thing you think supports your view, and ignore vastly more things not supporting it, all from the same useful criteria.
Now if only you’d carefully read the report you chose, and spend equal time looking at ample presented evidence, you’d develop a more accurate understanding.
Who is so emotionally invested into random comment sections being purely positive about their pet.. uuuuuuuh.. tech?
Very weird.
But I also don't believe things just because I want them to be true.
I agree that datacenters are not going to go away, but I have doubts that the buildup that has happened is really going to pay off for most operators.
Coorperate people likely just want “the best” and are happy to pay a lot for it to make their workforce more efficient.
It’s hard for me to guess where the data center buildout will go, eventually there will be an oversupply, but we haven’t hit that yet, and at the moment the companies owning these high capacity data centers appear to be making lots of very real money in leases to hyper scalers, hard to imagine this premium continuing, but we need to see where demand for cloud based LLMs tops out.
It may turn out that demand for SOTA frontier models isn’t so limitless after all.
Maybe I'd be willing to pay $10 for product A if I had other options. But if there's a product B for $3 that's not quite as nice but still ticks all my boxes, then product instantly becomes a lot less attractive.
Why? You can't just assert it. There are very good reasons to think it won't happen soon, and you've given no reasons to think it will happen soon.
Also, the frontier can’t keep advancing at this rate forever. Eventually the low-hanging fruit will all be gone and advances will slow down.
I.e., you get a locked down device with access to their AI and maybe a way to run third party apps. Of course the developers of those apps will have to pay a percentage of their revenue.
Sound familiar?
Bastards.
People will have to get used to buying a fixed amount of RAM with their CPU but thats unlikely to be a problem.
This might not be far off the mark. You are irreversibly linking the fates of these devices after a certain stage of manufacturing. If something goes wrong at final packaging time, you lose all dies instead of one.
Those two combined already result in a huge reduction in bytes per mm2, so with the same wafer processing capacity you're producing far less byte of memory. Add to that a complicated chain of HBM-specific packaging steps, and you're now also losing a decent bunch of perfectly-fine dies because rather than putting it into DDR you tried making a HBM sandwich and screwed up.
Even if the memory cells are the same and have an absolutely identical yield, HBM will always end up having a significantly lower output. That's just the cost of stacking, but some people are willing to pay the per-gigabyte price penalty in return for the higher bandwidth.
Soldering the DRAM onto a PCB is such a reliable process that there is almost zero risk of defects and even if a defect occurs the damage is limited. If the DRAM is soldered onto a DIMM the risk of a defect on the non memory hardware is non-existent. If the DRAM is soldered straight onto an SBC or GPU, then the DRAM can be removed to save the precious SoC or GPU chips.
Meanwhile HBM is the ultimate nightmare scenario. You stack up to 16 DRAM wafers on top of each other. One defect and the whole stack is worthless and that was actually the easy part.
In stage two things get even worse. You now have your accelerator chip and you must place the HBM on that chip. E.g. Blackwell GB300 has eight HBM stacks and the accelerator chip has a bigger area than the HBM. You must get the packaging right eight times in a row or you have wasted not only the DRAM silicon, but also the accelerator silicon because HBM cannot be removed and defects are permanent.
The issue here isn't just the yield of the HBM (which is obviously lower if you have taller stacks) but rather the yield of the combined HBM-based product, which is why doesn't make sense to say it needs more area but it is completely correct to say that HBM leads to more silicon being consumed. Hence it doesn't make sense to talk about yield of the HBM itself, because it is always part of an integrated product.
I gather a practical max ceiling today is a stack of 16 chips in height yielding 64GB?
These chips have a massive bus size of 2048 bits, instead of the 64 or 128 bits (dual channel) used by DDR5. That's what gives them their order-of-magnitude bandwidth speedup. But even though they technically pack in more capacity per square millimeter of motherboard, I gather they take up more space than older technologies once you account for the vias and interconnects to route all those signals.
HBM is meant to be integrated into the same package as the CPU, so no more DIMM sockets. It also has higher latency apparently.
> These chips have a massive bus size of 2048 bits, instead of the 64 or 128 bits (dual channel) used by DDR5.
You make HBM instead of DDR and the number of bits you're making goes down.
It's really that simple.
The advantage of HBM over regular non-stacked DRAM is memory bandwidth, which also requires a super-wide memory bus - 2048 bits wide for HBM4. Compare that to the 128 bit wide bus of a modern CPU.
So to take advantage of it on the desktop, or anywhere else, you need that 2048 bit wide bus, and a processor capable of consuming 2-3 TB of data per second!
These are not normal requirements, other than for a GPU.
But in reality we also already fact unified memory architecture systems, integrated graphics etc.
Or 256-512 bits on medium to high end consumer CPUs if you're apple.
The advantage in going wide is transferring cache lines rapidly, not the CPU bus interface.
Buy the way you win with CPUs is with latency, and not bandwidth, which is why Apple M series actually uses DDR with lower latency because of the stacking.
Running 17 chrome tabs doesn't benefit at all from that HBM and all the additional hardware+software complexities that come with it. You want a specialized coprocessor to handle specialized workloads. The GPU exists separately from the CPU for a reason.
The lack of ASML EUV machines certainly hurts, and pushing DUV so hard results in abysmal yields of good chips, but you can compensate by running more wafers or making smaller chips, and the net result is that Huawei's Ascend production volume is limited by CXMT's HBM capacity not processor dies.
The problem is that HBM manufacture requires many steps (die thinning, via drilling, plating, alignment) where the equipment used by everyone else (Samsung, SK Hynix, Micron) is also blocked by sanctions, so the Chinese are having to develop all of this themselves too, which they have, but yields are currently low, even when using shorter HBM stacks.
https://x.com/EugeneNg/status/2099315982959616369
Prefill (input tokens) is heavily compute bound. And the ratio of input to output continues to rise, as typically in agentic sessions you have a few tokens output for a tool call and (many) thousands of input from the tool result.
Then you have cached input tokens, which is a totally different issue, system RAM or NVMe bound.
Obviously output tokens is VRAM memory bandwidth bound, but this is less and less of the bottleneck these days for overall agentic speed.
If it could be made to work, you could run a fable-grade model in tens of watts.
As I understand it, yields are different for all three:
25% for standard 8-Hi HBM3E
45-50% for HiZQ (used for Huawei's Ascend 960DT training/decode chip)
75-80% for HiBL (used for Huawei's Ascend 950PR prefill chip)
HiBL yield is best because it only users 4 layers, resulting in lower memory bandwidth (prefill is more dependent on compute than memory).
HiZQ uses 8 layers same as their HBM3E, but is higher yield due to using a Huawei specific base logic die that relaxes a lot of the specifications.
Over the last 20 years the economy has become dysfunctional. It no longer really resembles a free market; monopolies have established barriers to entry everywhere. And the biggest investors are awash with helicopter money that's been doled out for favors by the political class.
So those investors have tons of cash to burn and surprisingly few opportunities. Even within Silicon Valley/VC there are surprisingly few who seem to really understand the fundamental economics of software. Or perhaps those economics just aren't that important when you have billions of dollars on hand and cash is obviously not going to get you a return. Any whisper of possible exponential growth is worth throwing money at. Crypto? Why not. AI? Why not. Datacenters? Why not. Tulips? Why not.
This is by all definitions an empire in decline. Everything is broken or fake. Everyone is afraid to do what needs to be done. Power forbids it. So we're all just waiting for the other shoe to drop. Our secret police aren't as bad as the late stage USSR's yet, but hold Uncle Sam's beer...
(That's not a recommendation to try and time the nadir, by the way, as it could easily be 50 years away.)
Overfinancialisation of the economy while removing safeguards to keep markets functional, like anti-trust enforcement, are the main forces behind this erosion. Through finance the focus of companies is completely shifted away from producing good products and services, and into how to extract more paper wealth from existing structures to the detriment of products.
Not enforcing anti-trust and letting behemoths to form which cannot be competed against since with their amount of capital they can either buy their competition outright or just price dump for long enough to make competition non-viable.
It's the failure of neoliberalism, and that agenda has been pushed into Western countries since the 1980s-1990s, it made enormous wealth for the few at the top while eroding whole societies, economically and socially, it's all dysfunctional.
Comparing this to crypto or turnips makes me doubt the merit of any other opinions your comment offers.
Just to give some rough numbers in the past 5 years the amount of GPU compute installed in FLOPS is somewhere over 5 times all the CPU flops that have ever existed.
This has nothing to do with AI being good or bad or being able to produce things and economic value. It is by far the largest and fastest growth of any technology ever and we have zero clue about the economic stability of this grand experiment we're performing.
Some differences to the market at that time seems to be that during the dotcom bubble, many of the companies had little to no revenue.
Leading AI labs already generate enormous revenue. The investments into capacity are needed to address the current demand.
The situation seems somewhat less speculative.
That said, I cannot predict how AI capabilities will develop and how demand will respond.
Should capabilities plateau hard and soon, maybe the demand will not be there for the compute investments.
If it does not, and instead AI applications in robotics, science, and self driving expand, chances seem reasonably good that the demand will be there, no?
And as for the economic stability, much of the investment comes from existing giants like Microsoft, Alphabet, Amazon, and Meta, who have the necessary cash flow.
These companies are less likely to collapse than some of the ones during the dotcom bubble.
It's the total amount of money in the economy that's been invested toward a potential outcome. AI represents the largest amount of money, and largest fractional part of the economy invested ever.
Because of this AI could be the biggest economic boon ever, yet still not recover the full amount invested. This will have deep economic impacts that affect everyone and everything. At this point AI must achieve all its stated economic impacts or there will still be a huge economic crash that kills off any company that is over invested and cannot make a profit.
Worse, the many of the perceived economic impacts of AI are not for humans like you or I, but the huge companies you listed. Even if they economically win, everyone else made out of meat could still lose.
They say history doesn't repeat, but it does rhyme. This, at least to me, sounds like a mixtape of "internet" + "tulips" + "1920s financial world leading to global political instability".
Every potential outcome I see occurring pushes us closer to further instability, even if the economics on it work out on paper.
I had a look at the numbers.
The investment into data centers is estimated around $800B/year currently.
OpenAI + Anthropic combined had ARR of >$100B in July.
Global labor income is around $65T/year, the US GDP $32T, the global one $126T.
Global software spending: $1.4T/year.
I am no financial analyst, but I don't think all the stated AI impacts have to be met just to recover the investments.
A 1% productivity gain on global labor income represents $660B/year. At 5% we would look at $3.3T/year.
They don't need to cure cancer to justify the investment, even though Amodei hopes to cure most major disease in the next 5-10 years.
>Global software spending: $1.4T/year.
Out of this number how much of this goes to payroll that goes to humans versus how much of this goes to non-human infrastructure costs. The numbers suggest this is anywhere from 60 to 90%, with 70% being a reasonable average figure. Just under a trillion dollars paid directly to humans would disappear if AI somehow captured all of this market.
Moreso the AI industry needs to capture these gains very fast or they will be crushed by interest payments on the massive debts they've accrued.
>ven though Amodei hopes to cure most major disease in the next 5-10 years
The average development time for medicine is 10 to 15 years before a single dollar is earned from said medicine. AI will shave very little off that as human testing and our stupidly complex bodies introduce all kinds of problems. Furthermore running head long into blindly using AI medicine is how you produce X risks from super intelligence AI.
The speed at which AI has to develop in which to get profitability is the biggest risk, a potentially catastrophic risk at that.
The reality is that so far there has been no sign of GDP growth picking up. It is basically flat at 2.5% +/- over the last few years.
In a similar vein one might have expected that the internet (think of all the e-commerce and efficiencies!) might have shifted GDP growth into a higher gear, but it did not, although in that case there was at least a significant boost in the 1996-2000 "dot com" era when the build out was happening (then to be followed by the crash and all the unused dark fiber etc).
So, maybe the hoped-for AI boom will be just as much of a dud (as it appears to be so far) as the internet boom. New day, different tools, same growth.
It's perhaps odd that we're not even seeing datacenter/etc build out register on GDP, but perhaps the scale of it is not as large as the internet build out?
That's 0.6%. Subtract last year's investment and then we're talking yearly growth.
Turns out it's not the earth shattering capital investment the comment before mine was making it out to be.
And about productivity gains, AI became good at software development in spring this year.
I'm not sure what productivity miracles reflecting in GDP growth you already expect to see by now.
In the past 5 years 5 times as much GPU compute capacity has been installed as total CPU capacity that has ever existed. When it comes to capital intensity the growth of AI has been one of, if not the largest capital expenditure for a class of project in this short of time frame. Things like the Manhattan project which was absolutely monstrous expense at the time is dwarfed by this.
Of course measuring GDP growth is difficult when you have an idiot in chief trying to destroy the global economy with the dumbest set of actions contrived by a human ever. Disentangling this factors will take a lot of work on someones part.
As someone who is myself 'old af' by tech standards (I'll be 53 very soon) the main problem is that the policymakers are all 70+ years old.
Their concept of what China even is is forever stuck in pre-2004 thinking from back when they still had significant neuroplasticity.
Vote these fucking fossils out, kids, before it is entirely too late.
(And, yeah, before someone inevitably brings it up, some exceptions-that-prove-the-rule older policymakers/politicians do avoid this trap of getting stuck in the past, but most don't)
Was the Great Leap Forward part of the plan too?
No, that's generally what happens when countries stop being poor. Taiwan, Singapore, Korea, all have similar fertility rates.
On the one hand, you're proving my point. On the other hand, if you think that China's headed for a demographic disaster, so are all of those countries, and your thesis of blaming the CCP for everything... Falls a bit flat.
Developed countries tend to have lower fertility rates (i.e. less kids per family) than undeveloped ones. Comfortably off, well educated people tend to put thought into whether to have kids or not. In poor countries people tend to have large families so they can send the kids out to work.
Having a high fertility rate isn't "high status" and a sign of a country's progress, but rather the exact opposite.
Of course this is separate from this issue of having a shrinking population, which is a sign that your citizens are not optimistic about the future, not a positive thing.
There is a handful of developed countries where this is not an issue.
Different countries are handling this in different ways, with encouraging immigration being the major one. Japan is facing a major lack of workers, with many businesses having to shut down as a result, but have decided to tackle it in a different way relying on advanced robotics as an alternative to immigration.
There are very few developed countries NOT having this problem. Most of the population growth is in poorer parts of the world like Africa.
Going purely by the evolution theory and natural selection, having so few children that there is a serious risk a population will disappear in a century or two, means losing. Needless to say, this is stupid. From the entire American continent, to Europe, India more recently, Thailand, China, the Koreas, Japan.
Africans, Arabs, and more broadly Muslims will carry the torch. Much respect for not falling into the trap.
Honestly you're reading the data completely wrong. You cannot look at it at the country level. In Israel for example the populations that are producing the most population are highly religious and less educated than those that are producing less population.
It's more about memes. Religion is a meme driven breeding cult. This is why said religions tend to have very long term staying power. The particular problem here is these religions come at a cost of having to ignore a lot of reality to get to that point. Because they are faith based anything that presents a direct challenge to that faith must be setup as an enemy or it may risk the religion. Yes, religions can incorporate new information but it tends to be a slow process as it needs to involve slowly changing peoples mind to avoid instability.
What this looks like in the long term is large populations of highly religious that have a fundamental disconnect from the technology that's keeping them alive. A new Late Bronze Age collapse.
Uh... Meme-based? This is such an unserious argument I don't even know what I can possibly say to that.
How are they disconnected with the technology keeping them alive? All people, in GCC countries especially, are working to make 1st world development happen.
Implying that religious Jews or Muslims are stuck in the Bronze Age is ignoring reality. Most European development itself came from men of science who were Christian, I thought this apparent conflict of ideas was something taught in schools.
GCC countries are resource economies mostly. They must weave a very narrow path to success to non-oil based means of income or the entire region collapses. I will say they do realize this is a problem and are using oil money to diversify, but resource curses are very hard to break and collapse can happen very quickly.
"My grandfather rode a camel, my father rode a camel, I drive a Mercedes, my son drives a Land Rover, his son will drive a Land Rover, but his son's son will ride a camel." --Sheikh Rashid bin Saeed Al Maktoum
>Most European development itself came from men of science who were Christian
This is just stating that that in 1700 you were a Christian or you got hung (hell, pre-independence America was founded by people that thought that too few people were being hung). As we see when you take out the hanging or social shame part of the religion people tend to flee them as they are oppressive structures. You could say them being men of science either had no bearing to their religion, or was in spite of their religion.
I am saying, for example, that the Abrahamic religions are bronze age (ok, sorry, Iron age) social structures. I don't think studied scholars are going to disagree on that point very much. People saw how people worked and could be sheppereded in societies without formal education and turned it into a series of stories that were effective in modifying peoples behaviors. Kings later came along and added their stories so they could further control the population. Over time a semi-coherent story about how people worked was formed below a set of rather incoherent stories regarding things that didn't happen that people are supposed to accept with faith.
(Comic book superheros, cough cough. The references to Canon in what are essentially liturgical disagreements - I mean power scaling arguments - are particularly hilarious.)
Not me of course, I’m different and unique. Just like everyone else.
And if that fails, we've got "Idiocracy" to save the human species as the dumb-asses pump out babies as fast as you like.
I'm not saying fast fall in birth rates isn't a problem, it obviously is. However, nearly all countries that don't have this problem offer much worse quality of life than those which do.
There is a saying. More is better. Morocco has issues with tourists and diaspora pricing out locals from houses to food even, but Algeria is an honestly good country all around. With its issues of course. But there is more to life than earning $$$.
Westerners forget the tiny detail that these countries are extremely young unlike them, hence why it feels like there are so "many" people.
Edit: oops, I glossed over the part about Israel. Trust me, it's still a billion times better than nothing, which is the case for the entire West.
I don't know the exact correlation but even in Morocco, total fertility rate is declining very rapidly, down from 6 children in 1980s to now around 1.7 in urban areas. UAE (Dubai) has similar TFR to Belgium or Portugal.
I don't believe that even the CCP itself, in hindsight, considers the one-child policy to have been a good decision. At the very least it should have been ended some 15 years earlier.
The problem with low birth rates is assumed to be that there are less workers to take care of the pensioners. But right now it seems like even the few young workers available often aren't needed anymore, thanks to increasing automation of industries. I predict youth unemployment will be made even worse by AI soon enough. So, why do we need children, if capitalist economies don't need workers? Unemployed people do not pay much taxes to sustain the non-working age population.
I believe low birth rates might become a blessing soon enough. It just means less people for governments to take care of when robots and AI take away most jobs.
The one child policy is gone now but birth rates haven't really moved upwards.
It's difficult to near impossible here to determine which counterfactuals would actually hold true. Based on other world wide trends it's very likely that Chinese family sizes now would only be slightly higher than if the policy didn't exist. In the modern world there are just a huge number of different factors working together to decrease family sizes.
It was insanely disruptive.
There were forced abortions (sometimes very late), there was forced contraception, forced sterilization (I think), and lots of punishments for people who broke the policy.
There were also others who got around it fairly easily by paying a bribe and by not registering their "surplus" child/children (especially if they were girls). That of course caused some problems down the road.
That still doesn't mean that it affected China's birth rate much. It was already low (and had fallen steadily) when the policy was instituted. It also had plenty of exceptions for rural China and ethnic minorities.
A friend of a friend is married to a member of the national academy of sciences in China. I heard at dinner a few years ago that he had been lobbying for more onshoring of chip production for years before the Nvidia restrictions, and always heard "you're right, maybe next year we'll be able to fund that". Then the restrictions came in and everything changed.
Funny part is people do not seem to care.
And, to the point I was responding to originally, we can check those documents! They don't speak of "annihilating" other countries' industrial capacities.
Why would they have to?
When China talked about their 5 and 10 year plans on solar panels there was no need to talk about putting the rest of the competitors out of business. That was a highly probable end goal of what they were doing, no need to state it.
It would be like Walmart stating "We're going to put out massive loss leaders in sporting goods to put X out of business". Instead they'll talk about growing market share and long term customer loyalty.
More of, China is smart enough to use the benefits of both capitalism and authoritarianism to grow their country while western governments are busy stealing the furniture and fixtures off the walls.
It's a possibility. It is also possible that they will steal yesterday's technology and still pretend to have moral high ground.
I do not wish China and Chinees wrong but most probably communism will work as it always do - by faking thing and bringing self-harm to their own citizens.
Which incentivizes pretending you need a ton, and can give nothing. You think being useful at work in Capitalism is bad? Wait until you are accidentally useful in a communist/socialist economy.
There is usually less pretending under capitalism, but people are people, so it all the same stories still play out.
People are shitty, if given the chance and influenced in specific ways. Capitalism or not. We’re dealing with the current bullshit in the west because somehow people bought the line that everyone is a good person - and we totally can’t actually see or do anything if we see someone doing something bad! - and we need to always follow ‘the rules’ even when someone bad clearly isn’t.
Until we find our balls, good luck.
This is all nonsense. Capitalism isn't an ideology, it's just a slight formalisation of what naturally arises. People are free to choose; they can own things; they can make agreements; they must honour the agreements or pay a penalty. Capitalism in no way is about stealing.
https://www.britannica.com/money/capitalism
Next you'll be telling me fairy tales like "laws are real".
Western sanctions have succeeded in achieving China's self-sufficiency goals where their own government failed.
Whereas China is developing a powerful network of rapidly developing trading partners like Africa and the Middle East?
And it’s not a given that they’ll catch up. I think it’s fair to say that their development of jet engines is on track to be obsolete before they’re competitive.
They may in principle have established the technologies to be self sustained but that’s only relevant and sustainable if they develop an economy based on domestic consumption rather than exports. Which they are struggling with as well.
i dont think it will - they still have a chokehold on battery production (which implies dominance in the EV sector), and they also control the vast majority of all rare earths. The chinese gov't can subsidize chip research/development with these other sectors that are dominating.
The west is the one that need to watch out for their own economic dividend running out from the past century.
but... not the western govs
The Chinese domestic EV market is the largest in the world.
Chinese factories are full of Chinese robotics.
Companies like Huawei are struggling to meet the domestic demand for AI accelerators, while growing volume very fast year over year.
The Chinese domestic AI market is massive, but we just don't hear much about it outside of the models that are also sold here. Can you even name the models that ByteDance (China's Meta) makes?
As far as exports go, China is a low cost producer for many things. If the US doesn't want to buy high quality EV's costing 1/2 the price of a Tesla, then there are plenty of other countries happy to do so. The US is only 5% of the global population - there are lots of other consumers out there.
If your yield is low and you still want the volume, then run more wafers, but then you need access to more DUV machines and more wafers. If your defect density is too high, then design smaller chips with a higher chance of avoiding defects, which is what Hauwei are doing - split processor into multiple chiplets.
HBM3E usually has 5-6000 vias, but doing this with multi-pattern DUV without defects is tough, so CXMT currently drop that to 3000, at a cost of some loss of thermal and voltage stability.
HBM:processor production ratios aren't what Huawei would like, so they mitigate this at system level by putting an optical memory bus on the GPU chip and sharing memory across the system.
Not everything is a huge LLM - smaller models like recommendation systems don't have so many parameters and need so much memory, so why waste HBM on them? ByteDance use their SeedChip acelerator for this, currently made by TSMC (using an older sanctions-approved 28mm process), which instead etches dense RRAM on die beside the processor.
There is also a time dynamic to this, with China still using pre-sanctions equipment and chips as their domestic alternatives ramp up to replace them. One interesting part of this is HBM ... HBM is very demanding to make, and everyone had problems with it, with initially only SK Hynix being successful. Samsung and Micron took a year or so to catch up, and during this time Samsung had made a ton of HBM that didn't meet NVidia's specifications, so ended up, pre-sanctions, selling it to China, where it has acted as a stockpile to carry them over as CXMT's domestic capacity ramps up.
China seems to be doing fine. Of course they would like sanctions lifted, moreso for HBM than anything else, but all that sanctions have really achieved is accelerating their semiconductor independence. They are still building 1T+ SOTA models, standing up 100K clusters of domestic AI accelerators, etc, etc.
And the worst thing is that is the best possible strategy for them. It's essentially win-win for everyone but consumers.
- Buyer (AI) has crazy money, so will pay whatever
- Seller doesn't have to build anything new, as the buyer is willing to pay whatever
- Generate ridiculous profits from crazy money
- No oversupply risk in case of reversal
- Return from producing HBM to DDR5 in a single quarter if reversal does happen.
Hold on to your existing hardware, people, and be on lookout for your local deals.
oof.
It absolutely is not a great situation at all. This won't make things better.
The big problem is that this will not depress the consumer DRAM market and mean we can all afford DRAM again
It’s not a step I’ve seen discussed very much, so it was kind of neat to see it discussed front and center in a lay article like this.
Zero chance you could do it if a human had to touch the die/chip directly during those steps.
But clean room + vacuum pick and place machine? A matter of fine tuning, and it should work just fine.
It's just crazy how far down the scale we're tickling atoms these days.
https://resources.pcb.cadence.com/blog/2023-the-planarizatio...
Not dissimilar to lens or mirror making, the process of grinding to a precise flatness. The key step here is bonding it to a glass carrier, presumably because otherwise it gets too thin to support itself.
See also the "reduced kerf diamond wire saw": technology critical to cheap solar panels is the ability to saw the boule into thinner and thinner pieces with less waste at each cut, like a big ham slicer.
All supply is supply, despite what certain folks seem to believe.
There are no privileged consumers that have "dedicated" amount of production, no matter the demand.
You can own stock, but other than that?
It's all GDDR6X or 7X
He alleged memory maker's public announcements of the shortage lasting throughout the end of 2027 are attempts to covertly signal the others on pricing strategy.
With additional production capacity ramping up this year and next, this seems somewhat plausible.
If manufacturer's inventories are filling since they advanced orders when prices were rising, there could be an oversupply of memory when they don't place new orders at the higher current price.
This could quickly result in a price war while capacity expansion and yield improvements create irreversible oversupply.
"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."
When I worked on satellites we had some electronics which used a radiation hardened FPGA which cost something like $500k per chip. Carrying that around was nerve wracking.
I had to remind myself a few times the practical value to anyone was about $22.
I used to work for a financial institution small enough to sometimes need someone from the back office to help fill in as a warm body for dual-control when we received cash shipments.
It only took a couple sessions in a vault to get entirely bored with having a half million dollars in front of you.
I definitely know what those small brown boxes which ship UPS ground look like. Very normal.
Security is an issue, though, these are treated as a high value load and typically would be followed by unmarked, plain clothes armed security guards tailing around the truck. (That's how Apple often does warehouse deliveries of trucks full of iPhones. It wouldn't surprise me if there was going to be some rolling lead soon following some trucks full of Duos...)
The truck has multiple drivers and private security.
https://www.wired.com/story/the-worst-ive-ever-seen-cargo-th...
Likely ending up on a container to China, export bans be damned.
What stayed with me from that talk was that all the data that was captured from those massive antennas was packaged in boxes, loaded in a cargo van and travel 1.600 kilometers to Santiago to be uploaded into the systems.
[0] https://www.almaobservatory.org
https://www.supplychain247.com/article/how_do_you_get_all_of...
[0] https://aws.amazon.com/blogs/aws/aws-snowmobile-move-exabyte...
[1] https://www.cnbc.com/2024/04/17/aws-stops-selling-snowmobile...
But someday, that demand will stop rising.
At some point, most people will have enough RAM. And they just won't need to buy more.
The inventories of RAM will become overfull, in warehouses and stores. And suddenly the manufacturers, middlemen, and even retailers will all find the price they can sell at keeps dropping.
But you can't shut down a huge manufacturing facility overnight. You can't shut down a years-in-the-making retail distribution system overnight, either. You can't even shut down a facility that is halfway finished with construction. They'll keep producing and selling RAM at lower and lower profit margins, until the day comes they finally accept they're losing money with each sale.
And when that happens, we'll all have more RAM than we know what to do with. It'll be cheaper than we have all ever experienced in our lifetimes.
I don't expect this to happen soon. But demand cannot keep exponentially rising forever. There will come a point, probably in the next 5 years, when demand at least levels off, even as production capacity is still rising.
And it will be interesting to see what's possible at that point. We'll be able to create software that is not doable right now, because it will no longer be constrained by memory. And even cheap laptops will massive amounts.
Everything is moving to edge compute. From smart rings and watches, to robotics and drones, iphones and humanoids, and no end in sight to the scaling improvements in AI... It does seem that you can continue to get more emergent behavior the more compute you expend in pre/ post training. There's no reason we can't scale up to a million data centers in space, as long as you get dramatically improved intellegence.
I wonder how many layers theirs is. So many layers (ok, 8, 12, or 16 for different fabs). So much less dram for the rest of us.