Ask HN: Is Moore's Law over, or not?
Every once in a while I see an article that seems to claim that Moore's Law is over, or slowing down, or about to be over. Then I see some counter-claim, that no, if you account for added cores, or GPUs, or some other third thing, that actually it's still right on track. This cycle has repeated every year for like the past 10 years, but the last few years feel like things have really started to slow down. Maybe that was partially illusory with the chip slowdown from the pandemic, but I figure now that we're several years out we should be able to say for sure.
It also seems like a pretty important question to answer because it has big implications for the advancement of AI technology which has everyone so freaked out.
So what's the consensus around here? Is Moore's Law actually over yet, or not?
93 comments
[ 4.9 ms ] story [ 152 ms ] threadEdit: I have no idea why anyone would downvote a link to this article. It directly answers the question with a decent level of technical detail. We are nowhere near single atom sized features yet, despite what node names might lead you to believe. There's still quite a ways to go.
I’m excited about photon-based processors, but until that’s a reality we still have a ton of headway for application-specific scaling.
If you rip specific loops out of a general purpose CPU, there are still plenty of gains to be made!
People claimed this - “single core is over” - years ago when Intel was stuck doing Skylake refreshes and AMD’s Ryzen only matched the older Haswell CPUs.
A general-purpose CPU requires that a program is transformed (via compilation or interpretation) into a set of basic instructions, ones that the processor knows how to handle. This means that almost every program requires many cycles to complete, even if the underlying logic itself could theoretically be done within a single cycle (or within no cycle at all!).
On the other end of the spectrum are FPGAs and ASICs, programmable or dedicated circuits that allow you to create specialized logic that corresponds directly to a specific need.
Bringing this back to the discussion at hand: Moore's law cares nothing about general-purpose CPUS, and is just focused on number of transistors on an IC doubling. With that said, transistors can only get so small (due to the laws of physics), and so one can presume we'll see an end to scaling eventually.
There are changes we can make to improve general-purpose CPU architecture, regardless of transistor count, and there are changes we can make to how we run programs (moving dedicated logic to dedicated circuits). Forcing any logic into a generic set of steps that run a in a loop will always be less efficient than wiring up the logic itself.
The questions has always been whether to wait for the machine to get faster or to create the dedicated logic yourself. The former has been true since the beginning of computation, and has been closely associated with Moore's law. With that said, it doesn't mean that the literal end of Moore's law is the end of computational efficiency gains.
Intel, by contrast, says that Moore's Law is still alive. But Intel is technologically behind, and it is easier to improve when there is someone to learn from, so maybe there is a wall that they haven't yet hit.
Regardless, it is a very different law than when I was young, when code just magically got faster each year. Now we can run more and more things at once, but the timing for an individual single-threaded computation hasn't really improved in the last 15 years.
Jensen aims to charge more for more GPU computing power into the future.
This is because Nvidia has close to monopoly power this is able to break Moores Law single handedly.
Unlike in gaming in the data center initial cost + performance per watt are the only thing that really matter (besides software, Nvidia has a huge moat there..). And in relation to how much Nvidia is charging per GPU total power costs are close to zero.. So 4 ‘worse’ but much cheaper chips might be a better deal than buying an A/H100 etc.
That's not true from a hardware perspective either. You can't just plug in 4 worse cards in the same rack. The savings on the graphics cards become less significant if you need to double/quadruple all other hardware to increase the number of racks. A 1U blade can easily cost $10000 without a graphics card.
For anything datacenter related, customers are very sensitive to price per performance. And datacenters are happy to oblige.
Of course data centers are happy to oblige to customer demands, but initial cost per GPU and performance per watt are certainly not the only relevant factors.
IMO if Intel stay in the game it’ll be sorted out within a few years.
Nvidia may have good software but people like paying less money.
Paying less will win out.
otoh it can be said that gpu's just look like they improved longer than cpu's because their workload is vastly more susceptible to parallelism.
The clock speeds haven't really gone up anymore, but computations still got considerably faster. From an i7 2700k (2011) to an i7 13700k single core benchmark scores went up 131%
https://cpu.userbenchmark.com/Compare/Intel-Core-i7-2700K-vs...
Second, over that time period we've had a lot of changes in tooling. Some make code faster. Most make code slower. (Examples include the spread of containerization, and adoption of slow languages like Python.) The result is that programs to do equivalent things might wind up actually faster or slower, no matter what a CPU benchmark shows.
This is interesting to me, how did they end up like this?
On desktop, anyway. Mobile is a different store, back in the mid 2000s they had everything to dominate the market for the next 10+ years (e.g. the fastest ARM chips) yet choose not to due to reasons..
And well Nvidia is almost a monopoly at this point so they have barely any incentives to continue innovating as opposed trying to extract as much money as possible from their clients.
On the other hand look at what happened with CPUs over the last few years. Huge improvements in efficiency (including Intel)
> hasn't really improved in the last 15 years.
I don’t think that’s even close to being the case in almost all use cases. Increasing complexity/bloat has obfuscated that to a large degree though.
We'll probably have a couple more innovations and might get to making a transistor out of a single atom (silicon atom is 0.262nm; carbon atom is 0.3).
5nm / (2*2*2*2) =~ 0.3
So I don't think we're done making faster hardware just yet, but we're certainly getting to the boundaries of what appears to be physically possible.
Its a marketing term
https://www.tsmc.com/english/news-events/blog-article-201908...
https://en.m.wikipedia.org/wiki/Intel_8087
The problem is memory bandwidth. This is also a place we’ve been to before :)
https://www.tomshardware.com/news/no-sram-scaling-implies-on...
> While there were a great number of interesting papers from both academia and industry, it was the one by TSMC that brought frighteningly bad news: whereas logic is still scaling more-or-less along the historical trendline, SRAM scaling appears to have completely collapsed.
https://fuse.wikichip.org/news/7343/iedm-2022-did-we-just-wi...
People are starting to move to getting performance improvements by increasing chip sizes and power budgets - part of the reason why GPUs are more expensive than they used to be.
"AI's Rule: Just as Moore's Law unfolds, the language models might expand, doubling the size and inference capability every [insert timeframe], revolutionizing communication and comprehension in unprecedented ways." (generated by ChatGPT)
[1] "a cliché and phrasal template that can be used and recognized in multiple variants", https://en.wikipedia.org/wiki/Snowclone
https://www.youtube.com/live/oIG9ztQw2Gc?feature=share
This isn’t the best recording on YouTube but it’s late and I couldn’t quickly find the other one.
[0]: https://upload.wikimedia.org/wikipedia/commons/0/00/Moore%27...
But "number of transistors" is like "number of lines of code": it's a cost, not a benefit, and if it feels otherwise it's only because that cost is the cost we have to pay for some benefit we care about.
And the claim that's increasingly commonly made these days isn't "transistor density has stopped improving" (though I think that's slowed down somewhat since Moore's time?) but more like "performance has stopped improving".
If we are putting more and more transistors on our chips (hence, larger die area, lower yields, more cost, more heat produced, more expensive cooling required) but not getting corresponding performance improvements in the tasks we actually value, then the thing everyone actually valued about Moore's law is dead, regardless of the status of the literal words of Moore's claim.
Moore's law, strictly, is about the growth of transistor density. Koomey's law, strictly, is about the improvement in computation per unit energy.
Those are both interesting, but frequently people care about something different from either, which is something like "computation per second available in hardware of reasonable size, power consumption and cost". Call this "effective performance".
This can increase even if Moore fails (e.g., we find good ways to exploit parallelism, and build larger devices with more cores). It can fail to increase even if Moore holds (e.g., we can put more cores on a device of the same size, but we aren't good enough at exploiting parallelism so real performance doesn't improve).
It can increase even if Koomey fails (e.g., we find ways to make our hardware faster; there's a corresponding increase in power consumption but we are still able to cool things well enough so we just accept that). It can fail to increase even if Koomey holds (e.g., we can't make anything faster but we find a way to maintain existing speeds at lower power; very nice but no performance improvement unless power consumption is the current bottleneck).
It used to be that effective performance increased exponentially at a fairly consistent rate. This increase has slowed but not stopped; it's not obvious (to me, anyway) what we should expect it to do in the nearish future.
The consistent exponential increase in effective performance had a name, in popular discourse. It was called "Moore's law". It's unfortunate that strictly speaking it isn't what Moore was originally describing, leading to an ambiguity when people refer to "Moore's law" between a law about density and a law about effective performance.
(I unfortunately lack the ability to read minds, so I can't be sure what OP had in mind. But given the statement that "it has big implications for the advancement of AI technology", it looks to me more like effective performance than density.)
People have been predicting its end for a long time.
Note also that it's about transistor cost, not about cpu performance - people sometimes think it is because performance used to be more correlated with transistor count.
This graph to me show that while yes technically Moore's law of doubling transistor per "thing Intel or AMD sells you" is still holding, it has ended for single threaded workloads. Moore's law is only holding due to core count increase.
For everyday use of users running multiple simple programs/apps, that's fine. But for truly compute heavy workloads (think a CAD software or any other heavy processing), developers turned to GPUs to get the compute power improvements.
Writing amazing programs taking full advantage of the core count increase is simply impossible (see Amdahl's law). So even if one wanted to rearchitect programs to take full advantage of the overall transistor count from ~2005 to now, they won't be able to.
Compare with pre-2005, where one just had to sit & wait too see their CPU-heavy workloads improve... It's definitely a different era of compute improvements
Do you need theoretical full?
Practical full is enough and still huge improvement over single core
I don't see this as true at all, using Wikipedias example -
"a program needs 20 hours to complete using a single thread, but a one-hour portion of the program cannot be parallelized, therefore only the remaining 19 hours execution time can be parallelized, then regardless of how many threads the minimum execution time is always more than 1 hour"
Just increase "19 hours" to "400 hours" execution time, it's possible to increase output as far as you want.
If you're mind set can't get past the "one hour" matters sure or change your mind set to the "19 hours" part of the calculation is what matters.
I don't fully get Gustafson's Law, but maybe it touches on this.
Yeah, sure. Just give up on the thing you wanted to do and instead do something entirely different that is more parallelizable. That'll keep your CPU utilization high, but it won't actually accomplish your goal (presuming you were using software for some purpose other than maximizing CPU utilization).
I don]t think that's the goal right now, though - it's about parallelizing multiple separate tasks/programs/etc: IOW, you can run 50 applications on a single 64 core CPU that used to take 50 individual servers
https://github.com/rui314/mold
"The complexity for minimum component costs has in creased at a rate of roughly a factor of two per year (see graph on next page). Certainly over the short term this rate can be expected to continue, if not to increase."
I remember reading in a magazine when I was a kid that Pentium 4 Extreme failed to reach 4.0 GHz in 2003 or 2004.
Since then, it took Intel quite some years to hit 4.0 GHz. Instead, the industry shifted to multi-core CPU, starting with the Core 2 series.
Does multi-core CPU count? I would say it's a bit of a stretch. It's more about horizontal scaling, where multi-CPU or even cluster also work in similar ways - there's no hard limit on how many CPUs you can add as long as you can cool them down. You can also make it much larger and sparse then put it in a large box to deal with the heating problem.
P.S. From the perspective of programming paradigm, people would then find "share nothing" and "message passing" is the way to harness concurrent and multi-core programming, after getting burned again and again with shared memory. These disciplines of not sharing RAM would further make multi-core more like programming on multi-CPU or clusters.
We still have Moore's law, which gives us more transistors. We just can't use them all at the same time and the individual transistors aren't getting faster (much).
For a while, we were able to use those extra transistors to wring out more performance out of sequential instruction streams by creating ever more complex out-of-order execution engines to figure out parallelism dynamically at run time. That also appears to have run its course.
Now we can use those extra transistors to add more cores, more cache and more specialised execution engines.
https://ourworldindata.org/grapher/transistors-per-microproc...
(the stagnation of 2019-2020 has nothing to do with technology; it's COVID)
- Transistor count doubles every ~24 months (Moore's law) - still going strong
- Total power stays ~constant (Dennard Scaling) - no longer holds
- Total cost stays ~constant - don't know if there is a name for this, but it no longer holds either
The real magic was when you had all three trends working together, you would get more transistors for the same power and same total cost. As the last two trends have fallen off, the benefit of Moore's law by itself is reduced.
There is still some effect of the last two trends, power per transistor and cost per transistor are still dropping, just at a slower rate than density is increasing. So power budgets and costs continue to grow. Hence 450W GPUs and 400W CPUs.
Moore posited in 1965 that the the amount of transistor per chip will roughly double every year - something he himself called "a wild extrapolation" in a later interview.
Actual development speed proved slower than that, so in 1975 he revised his prediction to transistors doubling every two years - so, the original "Moore's Law" was already dead by then. The second revision of his prediction proved more long-lived, in part because manufacturers were actively planning their development and releases around fulfilling that expectation - making it sorta a self-fulfilling prophecy.
There was another slow down in 2010 though - with actual development falling behind "schedule" since then.
But neither the "doubling" - nor the "year" or "two years" were ever anywhere near precise to begin with, so the question "is it dead" depends highly on how much leeway you are willing to give.
If you demand strict doubling every year - that's been dead since before 1975.
If you demand roughly doubling every two years - that's probably mostly dead since 2010.
If you allow for basically exponential growth but with a tiny bit of slow down over time - then it's still alive and well.
There can be no precise answer to the question - since the whole prediction was so imprecise to begin with. I don't think there's any benefit to getting hung up on drawing a supposedly precise line in the sand...
So don't fall for software vendors that want to convince you that you need a faster CPU every x year. You don't.
Today's 3nm processes use 3 dimensional gates that have film thicknesses on the order of 5 to 8 atoms thick and the features size is smaller than the wavelength of light used to measure expose wafers' different mask reticles that rely on using light slit interference to make features smaller than the EUV wavelength of around 10nm.
To get much smaller than 1nm using these techniques is going to run into fundamental physical limits in a decade and probably that limit will be around .5nm feature size.
The next frontier in silicon will be building three dimensional chips and IBM is a pioneer in 3D stacking of CMOS gates.