And to some extent it did end there. Before that time it was assumed by many that Moore's law indicated the sequential processing power would double every two years as well, which ended around that time. After that most increase in processing power have been in form of more cores, which means that you have to parallelize your programs to get performance.
Of cause Moore's law is about the number of transistors on a chip, and for that it has continued to be true. Physics do mandate that it will end at some point, when the size reach sub-atomic sized, so it won't be 100.
Ten years is over six of Moore's law 18 month periods, so that 'should' give a factor of over 64 (ignoring the difference between transistor density and speed)
No, that's not how trends work. You can't establish a trend from just two data points. And, given a trend, you can't just pick two points and say they should be so far apart. The only way that would work is if all the data points were very close to the trendline. And the only way that would happen is if CPU development happened very smoothly; which it doesn't. You have spurts of performance increase as new architectures are released.
The only thing I was showing with those two figures was that it's clearly not true that speed stopped increasing in 2005. If you want to show there's a different trend, then you need multiple data points and compare the recent trend with the longer term trend to see if they're significantly different.
You could pick data points to make either case. However if you plot transistor density, performance, clock speed, flops/watt, or other metric people attribute to moore's law you'll find the curve isn't holding. Improvements aren't stopping, but aren't doubling every 18 months either.
Take for instance 3 generations of intel chips (Sandy Bridge, Ivy bridge and Haswell), Nvidia GPUs, or Arm chips. Sure a new feature might do encryption, floating point, or some unusual graphic feature twice as well, but that's small fraction of the normal use case for that chip.
It's gotten so bad that nvidia often just renamed the chip, not actually revising the silicon between generations. Arm's new 64-bit generation of CPUS (cortex a53 and a57) aren't improving much at all over last years arm cores. Intel's new GPUs (one of the most parallel friendly workloads) aren't much better than 2 generations ago. Compare the best of the HD6000 intel GPUs to the old HD4000 2 genrations ago.
So by exactly what metric can support the idea that Moore's law isn't dead? Clockspeed? Performance? Transistor budget? Number of cores?
What today (april 2015) is 8 times better than 4.5 years ago?
Not a fair comparison, you are comparing a single-threaded benchmark vs a multi-threaded one, most software today is still single-threaded (unfortunately).
That i7 has 6 cores. So all things being equal 13,638/6 = 2,273 PassMark score per core. Considering the FX-57 is single core, we're really comparing 731 to 2,273, which is about 3x the performance per core in TEN YEARS!
Power usage is more impressive and the ability to create and sell multi-core CPU's certainly is, but shoving more and more cores into chip is cheating. Its clear we've hit some wall here and the only way to keep up with performance demand is keep making chips with lots of mediocre cores.
Moore's law is dead. No biggie. It happened and no one noticed and those who did, didn't seem to care.
> Power usage is more impressive and the ability to create and sell multi-core CPU's certainly is, but shoving more and more cores into chip is cheating.
No, its not. Shoving more components into a single IC is what Moore's Law is about. Performance per core was one manifestation of that, but a "core" isn't the fundamental unit the Law addresses.
On a practical level it certainly matters for single threaded applications. They're just not benefiting like they should.
Formally, Moore's law is just about transistor density not performance, so this is all academic. Historically, density was tied to performance. That's just not true anymore and hasn't been for at least a decade.
> Formally, Moore's law is just about transistor density not performance
Formally, it was about number of components per integrated circuit, not density of components (though related observations about density and performance were made very early on, and the 18 month, rather than two year, version actually is about performance, but wasn't from Moore.)
> Historically, density was tied to performance.
Its still tied to overall performance (computations performable b the chip in a given period of time). Its not tied to per core (i.e., single-threaded) performance when the larger number of components is divided among multiple cores, and that does make it harder to make full use of the available performance in some applications. But that doesn't mean Moore's Law is dead, it just means that more programming skills (possibly, outsourced to the creators of better programming languages/tools) are needed to fully take advantage of the available performance.
So, in other words, performance did not stop improving in 2005, as it is now "about 3x the performance per core"?
Personally, I think that it's too soon to say whether computing power is beginning to plateau. Certainly no-one has shown any good evidence for it so far.
>An object of 14 nanometers in size is smaller than a typical virus cell, and about the equivalent to the thickness of the outer cell wall of a typical germ.
I've been studying that very problem for a few years and I currently hold the view about "general intelligence" that it is untranslatable into software, or at least, not an algorithm.
I use the term algorithm because it's a pretty fixed definition, and "software" is relatively woolly.
This isn't a spooky, defeatist or handwavy argument. To be "generally" intelligent you need to be intimately physically linked to your environment in a way that the "input -> processing -> output" character of an algorithm, even when realised on a microchip, is not.
At the 90's, the MIT published a lower bound estimate of the processing capacity of a person's brain. We are currently entering the era when "all the computers, added" have reached this capacity.
I simply have no idea where did you got that "resound yes" from.
You didn't provide the source of your MIT study but that's okay because:
- I didn't provide any source either.
- But more importantly, I'm well aware that there are studies out there that grossly overestimate the information processing capacity of a human brain.
In the petascale section, the top supercomputer as of today Tianhe-2 (~33 petaflops) is listed right next to the computing power of human brain (~36 petaflops).
But it lists many mutually disagreeing estimates. About half of the estimates indicate we have already achieved the capability (in the form of the top supercomputer, not necessarily in the form of a sub-$1000 computer).
I should point out the accepted answer in particular. I have spent some time on the RIKEN simulation story (1 second of activity of 1% of human brain took 40 minutes on a 10 petaflops RIKEN supercomputer) in the past and I'm beginning to believe it's nothing more than a marketing gimmick:
- They never released a comprehensive scientific paper, detailing the methods they used.
- 1% refers to ~1.73 billion neurons, which is really 2% of the whole CNS (~86 billion neurons). But calling the whole CNS human brain is nonsense. The brain only has about ~19-23 billion neurons. So in reality it was ~10% of human brain.
- 'Simulation' could have vastly different definitions, and associated computational complexities. If they were solving differential equations of human brain's biochemistry then I wouldn't be surprised it took 40 minutes to simulate 1 second of activity of 1.73 billion neurons. But that is computationally expensive, and a wasteful way of doing brain simulation, because we're mostly concerned with the information-processing nature of the neurons, not their biochemistry.
Furthermore:
The top supercomputer as of today, Tianhe-2, is almost two years old. Do you know how much it costs today to buy GPUs worth 33 petaflops? AMD's R9 290X is a 6 teraflop GPU costing $250. You do the math. (Hint: It's less than $1.5 million! Tianhe-2 was a $390 million project, Titan $97 million at ~18 petaflops).
Finally:
If you plan to run a dedicated piece of software on your hardware, FPGA blows GPU out of the water, and an ASIC blows FPGA out of the water. If someone today implements a deep learning system (just giving an example, not claiming it's the end-all-be-all of strong AI) on an ASIC, I wouldn't be suprised if one such card performs equivalent of between 100 and 1000 teraflops of a GPU's performance.
And I'm talking about 28nm process. 22nm would only sweeten the deal.
So yes, Moore's law has been a great help in its leg in the "relay race" lasting ~50 years. Now it's time to pass the baton to the next leg, optimize the architecture and put some strong AI software in it.
In a sense Moore's law is already over. It's not only about transistor density, there's an economic side too: the original paper talk about the density of the least cost process. So the law ends either when we can't increase the density, or when we can't do it without raising the cost.
Now look what Samsung says (from http://www.eetimes.com/document.asp?doc_id=1326369): "The cost per transistor has increased in 14nm FinFETs and will continue to do so, Low said".
Now I know Intel has a different take on this, but a few months ago a JP Morgan report said that Intel cast wast TSMC selling price... They have more margin to reduce costs.
And that's only the marginal cost of transistors. The NRE costs to design a new chip (design & tests, masks) are increasing with each new process nodes. For massive volume players (Intel, Apple, Qualcomm...) that's acceptable. For low to mid volume players these NRE costs must be factored in, and they penalize new nodes. A lot of smaller players stop way before the latest 14/16FF. 40nm is common, and 28 will see a lot of people stop there (the last node with no double patterning).
BTW, the "least cost" part is very important. The cost of developing and putting in production each new process node is increasing. But as long as the cost per transistor (with NRE amortized ;) goes down, the accessible market is increasing and can bear the increasing cost.
When costs start to increase, you need a reason to pay more for the newer node. That's performance (whether speed, power efficiency or a mix). And not everybody will accept to pay for better, some are ok with the "old" nodes. So the market base is not increasing, it's reducing. At some point it won't bear the massive R&D needed.
This is not a nice though for all the experts involved in this expensive game.
Sure there are many problems in moore's law, and it's definetly slowing(not sure yet about stopping, because maybe euv lithography will work, or maybe monolithic 3d will - quallcomm is said to sells such chips in 2016, or maybe something else). Also there are interesting innovations in the higher levels - how to build much smaller SRAM cells, the use of memristors, etc - which could still support a declining cost for electronics.
But i don't agree that the market base is declining.Two major factors in the market base($$) are - the value of uses of computing, and how much of that value chip manufacturers extract.And there are some really killer apps with plenty of value in the future(virtual reality, AI in all it's forms, new large data sets to work with, emerging markets, etc). Also chip makers in many markets get a small share of the created value, and might get more.
As for the problem of small/large players and design costs - the solution is platforms, whether GPU/FPGA/CPU or more exotic programmable platform like specialized computer vision chips, or whether it's using chip where some of the layers are already created(and standard) and just creating the rest(like easic is selling) ,which can lead to great design cost reduction and relatively good efficiency.
So in general i tend to agree with some guy from darpa who said that we're near the end of moore's law, but we might still be able to get a total of 50x improvement from a combination of things.
> Two major factors in the market base($$) are - the value of uses of computing, and how much of that value chip manufacturers extract.
You are defining the market as the entire market for chips, while the GP is talking about the market for next generation chips only.
Your definition is useless for a company deciding if they invest in R&D for improving their chips or not. The GP definition is the one Moore's law depends upon.
For most of the markets i've defined lower transistor costs and lower power are critical factors and assuming the next generation provides both, they must improve their chips to compete.
In what sense isn't it dead already? Transistors per chip, clock rate, and performance haven't doubled for several generations already. Take Nvidia GPUs, Intel CPUs, or Arm CPUs for instance.
What's important is that they are increasing by a smaller ratio every generation; generations are taking longer and longer; and price/transistor is increasing steadily.
41 comments
[ 5.9 ms ] story [ 111 ms ] threadhttp://www.fark.com/comments/7215487/Moores-Law-no-more
http://www.slate.com/articles/technology/technology/2005/12/...
Of cause Moore's law is about the number of transistors on a chip, and for that it has continued to be true. Physics do mandate that it will end at some point, when the size reach sub-atomic sized, so it won't be 100.
Meanwhile, in 2015, we have the Intel Core i7-5930K which scores: 13,638.
Your numbers show a speed up of less than 20.
No, that's not how trends work. You can't establish a trend from just two data points. And, given a trend, you can't just pick two points and say they should be so far apart. The only way that would work is if all the data points were very close to the trendline. And the only way that would happen is if CPU development happened very smoothly; which it doesn't. You have spurts of performance increase as new architectures are released.
The only thing I was showing with those two figures was that it's clearly not true that speed stopped increasing in 2005. If you want to show there's a different trend, then you need multiple data points and compare the recent trend with the longer term trend to see if they're significantly different.
You could pick data points to make either case. However if you plot transistor density, performance, clock speed, flops/watt, or other metric people attribute to moore's law you'll find the curve isn't holding. Improvements aren't stopping, but aren't doubling every 18 months either.
Take for instance 3 generations of intel chips (Sandy Bridge, Ivy bridge and Haswell), Nvidia GPUs, or Arm chips. Sure a new feature might do encryption, floating point, or some unusual graphic feature twice as well, but that's small fraction of the normal use case for that chip.
It's gotten so bad that nvidia often just renamed the chip, not actually revising the silicon between generations. Arm's new 64-bit generation of CPUS (cortex a53 and a57) aren't improving much at all over last years arm cores. Intel's new GPUs (one of the most parallel friendly workloads) aren't much better than 2 generations ago. Compare the best of the HD6000 intel GPUs to the old HD4000 2 genrations ago.
So by exactly what metric can support the idea that Moore's law isn't dead? Clockspeed? Performance? Transistor budget? Number of cores?
What today (april 2015) is 8 times better than 4.5 years ago?
https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E5-2699+...
Power usage is more impressive and the ability to create and sell multi-core CPU's certainly is, but shoving more and more cores into chip is cheating. Its clear we've hit some wall here and the only way to keep up with performance demand is keep making chips with lots of mediocre cores.
Moore's law is dead. No biggie. It happened and no one noticed and those who did, didn't seem to care.
No, its not. Shoving more components into a single IC is what Moore's Law is about. Performance per core was one manifestation of that, but a "core" isn't the fundamental unit the Law addresses.
Formally, Moore's law is just about transistor density not performance, so this is all academic. Historically, density was tied to performance. That's just not true anymore and hasn't been for at least a decade.
Formally, it was about number of components per integrated circuit, not density of components (though related observations about density and performance were made very early on, and the 18 month, rather than two year, version actually is about performance, but wasn't from Moore.)
> Historically, density was tied to performance.
Its still tied to overall performance (computations performable b the chip in a given period of time). Its not tied to per core (i.e., single-threaded) performance when the larger number of components is divided among multiple cores, and that does make it harder to make full use of the available performance in some applications. But that doesn't mean Moore's Law is dead, it just means that more programming skills (possibly, outsourced to the creators of better programming languages/tools) are needed to fully take advantage of the available performance.
Personally, I think that it's too soon to say whether computing power is beginning to plateau. Certainly no-one has shown any good evidence for it so far.
You can tell this guy's not a medical journalist.
And IMO the answer is a resounding 'yes'.
I use the term algorithm because it's a pretty fixed definition, and "software" is relatively woolly.
This isn't a spooky, defeatist or handwavy argument. To be "generally" intelligent you need to be intimately physically linked to your environment in a way that the "input -> processing -> output" character of an algorithm, even when realised on a microchip, is not.
I simply have no idea where did you got that "resound yes" from.
- I didn't provide any source either.
- But more importantly, I'm well aware that there are studies out there that grossly overestimate the information processing capacity of a human brain.
Please see this link: http://en.wikipedia.org/wiki/Computer_performance_by_orders_...
In the petascale section, the top supercomputer as of today Tianhe-2 (~33 petaflops) is listed right next to the computing power of human brain (~36 petaflops).
Also this link is relevant: http://cs.stackexchange.com/questions/20016
But it lists many mutually disagreeing estimates. About half of the estimates indicate we have already achieved the capability (in the form of the top supercomputer, not necessarily in the form of a sub-$1000 computer).
I should point out the accepted answer in particular. I have spent some time on the RIKEN simulation story (1 second of activity of 1% of human brain took 40 minutes on a 10 petaflops RIKEN supercomputer) in the past and I'm beginning to believe it's nothing more than a marketing gimmick:
- They never released a comprehensive scientific paper, detailing the methods they used.
- 1% refers to ~1.73 billion neurons, which is really 2% of the whole CNS (~86 billion neurons). But calling the whole CNS human brain is nonsense. The brain only has about ~19-23 billion neurons. So in reality it was ~10% of human brain.
- 'Simulation' could have vastly different definitions, and associated computational complexities. If they were solving differential equations of human brain's biochemistry then I wouldn't be surprised it took 40 minutes to simulate 1 second of activity of 1.73 billion neurons. But that is computationally expensive, and a wasteful way of doing brain simulation, because we're mostly concerned with the information-processing nature of the neurons, not their biochemistry.
Furthermore:
The top supercomputer as of today, Tianhe-2, is almost two years old. Do you know how much it costs today to buy GPUs worth 33 petaflops? AMD's R9 290X is a 6 teraflop GPU costing $250. You do the math. (Hint: It's less than $1.5 million! Tianhe-2 was a $390 million project, Titan $97 million at ~18 petaflops).
Finally:
If you plan to run a dedicated piece of software on your hardware, FPGA blows GPU out of the water, and an ASIC blows FPGA out of the water. If someone today implements a deep learning system (just giving an example, not claiming it's the end-all-be-all of strong AI) on an ASIC, I wouldn't be suprised if one such card performs equivalent of between 100 and 1000 teraflops of a GPU's performance.
And I'm talking about 28nm process. 22nm would only sweeten the deal.
So yes, Moore's law has been a great help in its leg in the "relay race" lasting ~50 years. Now it's time to pass the baton to the next leg, optimize the architecture and put some strong AI software in it.
http://www.forbes.com/2005/04/15/cx_ah_0415tentech.html
Now look what Samsung says (from http://www.eetimes.com/document.asp?doc_id=1326369): "The cost per transistor has increased in 14nm FinFETs and will continue to do so, Low said". Now I know Intel has a different take on this, but a few months ago a JP Morgan report said that Intel cast wast TSMC selling price... They have more margin to reduce costs.
And that's only the marginal cost of transistors. The NRE costs to design a new chip (design & tests, masks) are increasing with each new process nodes. For massive volume players (Intel, Apple, Qualcomm...) that's acceptable. For low to mid volume players these NRE costs must be factored in, and they penalize new nodes. A lot of smaller players stop way before the latest 14/16FF. 40nm is common, and 28 will see a lot of people stop there (the last node with no double patterning).
For an interesting analysis of the cost factors in semiconductors, here's a nice blog: https://www.semiwiki.com/forum/content/4522-moore%92s-law-de...
BTW, the "least cost" part is very important. The cost of developing and putting in production each new process node is increasing. But as long as the cost per transistor (with NRE amortized ;) goes down, the accessible market is increasing and can bear the increasing cost. When costs start to increase, you need a reason to pay more for the newer node. That's performance (whether speed, power efficiency or a mix). And not everybody will accept to pay for better, some are ok with the "old" nodes. So the market base is not increasing, it's reducing. At some point it won't bear the massive R&D needed. This is not a nice though for all the experts involved in this expensive game.
But i don't agree that the market base is declining.Two major factors in the market base($$) are - the value of uses of computing, and how much of that value chip manufacturers extract.And there are some really killer apps with plenty of value in the future(virtual reality, AI in all it's forms, new large data sets to work with, emerging markets, etc). Also chip makers in many markets get a small share of the created value, and might get more.
As for the problem of small/large players and design costs - the solution is platforms, whether GPU/FPGA/CPU or more exotic programmable platform like specialized computer vision chips, or whether it's using chip where some of the layers are already created(and standard) and just creating the rest(like easic is selling) ,which can lead to great design cost reduction and relatively good efficiency.
So in general i tend to agree with some guy from darpa who said that we're near the end of moore's law, but we might still be able to get a total of 50x improvement from a combination of things.
You are defining the market as the entire market for chips, while the GP is talking about the market for next generation chips only.
Your definition is useless for a company deciding if they invest in R&D for improving their chips or not. The GP definition is the one Moore's law depends upon.
In other words, Moore's Law is dead already.