That's because software comes from the realm of biology, not physics, improvements are path-dependent, historically determined and can quite easily get stuck at local maxima rather than finding globally optimum states.
None of my multi-GHz, multi-core, multi-gigabyte machines boot up anything like as fast as my old AppleII or Osbourne portable. None of my text editors launches as quickly as WordStar used too. Moore's Law has _never_ quite kept up with the speed that software slows down.
People like to use this sort of thing to bemoan how bad software is, but to be really accurate, we should change it to "Software gets MORE than hardware gets faster."
How long does it take to boot an AppleII? My system boots in around 15 seconds (i7 +SSD), but you can boot to a command prompt in less time.
PS: It actually look less time to install windows 7 ~15 minutes than it took to boot one of my old windows 95 machines with far to little ram and a antivirus software.
>Isn't there a "reverse Moore's Law" for software? None of my multi-GHz, multi-core, multi-gigabyte machines boot up anything like as fast as my old AppleII or Osbourne portable.
This might be true for machines that held the OS in ROM, but most current computers load much faster than an IBM PC loading the OS from a floppy disk.
These comparisons don't mean much, because the software can do thousands of new things that it could't, like connect to the internet, wirelessly, at high speeds, play multimedia at millions of colors and huge sampling rates, etc.
It's not like the exact same programs got somehow slower, just the same category of programs. A modern text editor is so much more advanced that WordStart that it's not even funny.
And you might not remember that loading a simple game in a 8-bit home computer from tape (a cassette tape, such as used by Amstrad CPC, Commodore 64, ZX Spectrum, etc) took ages (5-20 minutes), and involved 1/1000000 of the data of a modern game.
None of my text editors launches as quickly as WordStar used too.
I wouldn't say biology so much as psychology. Simply put hardware has laws of physics to bound it. Software doesn't such strictly defined limits. It's only bounds are metaphors and underlying mechanism to implement it.
Moore's Law has largely been sustained because we've just been been getting as much as we can out of one overarching idea (silicon and transistors), and incremental progress along that direction is inevitable when there's no stimulus to force exploration of a completely new direction altogether.
Like olefoo said, software is an entirely different game. In addition to the above point, the relative progress of software is also impeded by the fact that it has to cater to millions of different applications in the real world and perform under different constraints in each one. When it covers such a broad range of application, its hard to really see what the "best way" is,and there probably isn't one.
Final (and most important IMO) point - there's no way to simply characterize the progress of software into one number and see if it follows Moore's law. Given that the open source movement, social networking, sites based on user-generated content, and tons of other things have all been created in the last decade, we may not be doing that bad.
> Final (and most important IMO) point - there's no way to simply characterize the progress of software into one number and see if it follows Moore's law.
Sure there is. 1) SLOC reflects amount of software, and 2) Amount of data created per year (or whatever time) reflects how much that software is being used.
And before you say "flawed etc" well so is Moore's Law and just counting the # of transistors.
Decades ago, image-editing software looked like MS Paint (or worse). Nowadays we have the ability to create fancy renderings in something like Blender and add all sorts of effects with Photoshop. Is that the sort of progress a "Moore's Law for Software" would be measuring?
One of the intrinsic problems with something like MLfS is that software "progress" is much harder to measure than transistor density. What if image editors make great progress but text editors are stagnant for a decade? What about search relevance improving steadily through the 1990s, then content farms cluttering results in the 2000s?
I agree the problem is in the definition of the phrase. But taking it a step further, it may be that this is not an apt comparison to begin with. "Moore's Law for Software" It sounds witty and fantastic, and I had this tingly "Ooh, aah, interesting point" feeling when I first read it, but it's hiding a lot underneath all that brevity.
We can look at Moore's law and get and easy metric: The layman may know it as "The speed of computers doubles every X years" (where X changes depending on the person retelling it). Those more familiar know it as "The number of transistors we can cram into the same space doubles every 18 months." We get something concrete to look for, something empirical to measure: Transistors.
With software it's far less clear not only what we're measuring, but how we measure it. And even if we ("we" being necessarily a mere subset of software users) settle on a measurement, it's almost certain that any one measurement will not matter to all people in the same ways.
Moore's law for Software. Does that mean Moore's law for the usefulness of Software to its end users? Does it mean efficiency of achieving some result? Does it mean the capacity of the software to be intuitive? Enjoyable? It's ability to spawn a series of companies that create new jobs? We might as well be in a social sciences seminar debating definitions of "utility". Put into an example: A redesigned UI for an internal tool at a major corporation could lead to hundreds of people doing their jobs twice as fast. Maybe even thousands. That's Moore's Law-esque doubling, but is it really what we want to measure? If it is, where did it come from, and how do we continue to reproduce that improvement in the manner that the word "law" would require?
I won't try and go any further down this rabbit hole, but I just wanted to elaborate a bit on the points you opened up.
The pace of innovation in software is both staggering and obvious to anyone who takes a serious look at what you can do with a computer 10, 20, or 30 years in the past. We've done quite a bit with the advances Moore's law has given us, if you ask me. And software is hard and complicated because the problems it's tackling are hard and complicated. A basic UI, or a basic web page, is easier to make now than it's ever been. But the thing that keeps me comfortably in business is that no matter how much I produce, everybody still wants more stuff, faster and cheaper.
Knowing a bit about Alan Kay's work I have to say that some posts here are missing the point. In my view when he says that there isn't a Moore's Law for software he is asking why haven't we abstracted software development to the level where we can double our productivity every few years or months.
Agreed. Been wondering where the fork in the road happened. Was it going OOP instead of FP? Should we be doing domain-driven design & using DSLs everywhere? Should we be using macros to write code that writes code as a routine practice. Was Dylan the way to go instead of Java?
Not sure at all. We are still looking for a silver bullet which may be the wrong path. Pushing the analogy a bit further, if software had behaved like hardware we would have had a constant stream of small improvements over time. If this had happenned C (or Fortran for that matter)would have morphed into a "modern" language (whatever that means).
I think Alan Kay is best interpreted as saying "there has not been a Moore's law level of progress for software", not "there can be no Moore's Law for software".
I can think of two metrics by which software progress can be measured.
Measure 1: Consider a software to be a sequence of operations which generates some arbitrary particular string of bits. A measure of progress can be, for any random string of bits whose generation we view as useful, how far is the totality of code (libraries, OS portions, etc) from the true Kolmogorov Complexity? Then Moore's Law can be, is this difference between true and actual complexity decreasing exponentially? Put another way, is the level to which we can compress the code specification of software exponentially approaching it's limit? I don't think so. But I also don't think this measure is useful as its too abstract.
Measure 2: For any given piece of software from the past, is the total number of man years required to replicate it decreasing exponentially as a function of time? Again I don't think so, not uniformly. I think it's more polynomial.
Of course there is a theoretical manner by which Exponential progress can be made in software. But some people view it as a dangerous existential threat. Either way, achieving Moore's law for software is not something that should be stepped into blindly.
There has been some progress on this front. Average programmer productivity has been roughly constant over the past several decades in terms of Source Lines of Code. (SLOC - Yes, it's a rotten metric, but it's good enough in huge aggregates.) SLOC per unit of functionality for mainstream languages stopped decreasing when we got to Smalltalk, and hasn't moved much since.
So to progress from here, we either have to:
- reduce SLOC per unit of functionality
- increase reuse of existing code
- increase the rate at which programmers can write correct code
I'm not sure the 1st is a good idea. When languages get too terse, they get harder to understand, and maintenance and debugging are a big deal.
The 2nd has been happening more and more. The way communities share good code has indeed progressed and increased the power of the individual programmer in the past two decades.
The 3rd one is quite a complex issue. I suspect functional programming can play some kind of role here, but it will have to be packaged in a way which is palatable to the programming mainstream.
Much could be done by a programming language that can enable the application of functional programming in much the same spirit as John Carmack's post here:
Just as OO became mainstream through "impure" implementations, FP could become mainstream through an impure implementation enabling programmers to easily incorporate it into their existing projects.
How about an FP-Lint for popular programming languages that can flag non-FP code in regions that have been designated FP?
Alan Kay's current project [1] is facinating. One of its accomplishments is [2]. One tool is a PEG parser embedded in a scheme like language and stacking DSL on DSL to succinctly express solutions. This sounds like a possible way to achieve Moore's law for software.
I think there's still a lot of potential improvement to be gained by increasing reuse. It seems to me what we need is to take the CPAN to the next level: to have a readily available, heavily tested repository of libraries that work with all major programming languages. Think of all the effort that's currently duplicated over and over again, once for each existing platform...
Mind you, I have no idea how to do it. It's one of the goals Parrot was aiming for a decade ago, but Parrot doesn't seem to be particularly close to being a good answer yet.
I would argue that Open Source software has offered a lot to your second point. Look at what powers web software now, from the servers, OSs, databases, and languages. It's easier than ever to "plug and play" different pieces of technology together with a little code glue.
25 comments
[ 3.0 ms ] story [ 57.7 ms ] threadThat's because software comes from the realm of biology, not physics, improvements are path-dependent, historically determined and can quite easily get stuck at local maxima rather than finding globally optimum states.
None of my multi-GHz, multi-core, multi-gigabyte machines boot up anything like as fast as my old AppleII or Osbourne portable. None of my text editors launches as quickly as WordStar used too. Moore's Law has _never_ quite kept up with the speed that software slows down.
PS: It actually look less time to install windows 7 ~15 minutes than it took to boot one of my old windows 95 machines with far to little ram and a antivirus software.
This might be true for machines that held the OS in ROM, but most current computers load much faster than an IBM PC loading the OS from a floppy disk.
These comparisons don't mean much, because the software can do thousands of new things that it could't, like connect to the internet, wirelessly, at high speeds, play multimedia at millions of colors and huge sampling rates, etc.
It's not like the exact same programs got somehow slower, just the same category of programs. A modern text editor is so much more advanced that WordStart that it's not even funny.
And you might not remember that loading a simple game in a 8-bit home computer from tape (a cassette tape, such as used by Amstrad CPC, Commodore 64, ZX Spectrum, etc) took ages (5-20 minutes), and involved 1/1000000 of the data of a modern game.
None of my text editors launches as quickly as WordStar used too.
Try an SSD drive and you'd be surprised.
Like olefoo said, software is an entirely different game. In addition to the above point, the relative progress of software is also impeded by the fact that it has to cater to millions of different applications in the real world and perform under different constraints in each one. When it covers such a broad range of application, its hard to really see what the "best way" is,and there probably isn't one.
Final (and most important IMO) point - there's no way to simply characterize the progress of software into one number and see if it follows Moore's law. Given that the open source movement, social networking, sites based on user-generated content, and tons of other things have all been created in the last decade, we may not be doing that bad.
Sure there is. 1) SLOC reflects amount of software, and 2) Amount of data created per year (or whatever time) reflects how much that software is being used.
And before you say "flawed etc" well so is Moore's Law and just counting the # of transistors.
One of the intrinsic problems with something like MLfS is that software "progress" is much harder to measure than transistor density. What if image editors make great progress but text editors are stagnant for a decade? What about search relevance improving steadily through the 1990s, then content farms cluttering results in the 2000s?
We can look at Moore's law and get and easy metric: The layman may know it as "The speed of computers doubles every X years" (where X changes depending on the person retelling it). Those more familiar know it as "The number of transistors we can cram into the same space doubles every 18 months." We get something concrete to look for, something empirical to measure: Transistors.
With software it's far less clear not only what we're measuring, but how we measure it. And even if we ("we" being necessarily a mere subset of software users) settle on a measurement, it's almost certain that any one measurement will not matter to all people in the same ways.
Moore's law for Software. Does that mean Moore's law for the usefulness of Software to its end users? Does it mean efficiency of achieving some result? Does it mean the capacity of the software to be intuitive? Enjoyable? It's ability to spawn a series of companies that create new jobs? We might as well be in a social sciences seminar debating definitions of "utility". Put into an example: A redesigned UI for an internal tool at a major corporation could lead to hundreds of people doing their jobs twice as fast. Maybe even thousands. That's Moore's Law-esque doubling, but is it really what we want to measure? If it is, where did it come from, and how do we continue to reproduce that improvement in the manner that the word "law" would require?
I won't try and go any further down this rabbit hole, but I just wanted to elaborate a bit on the points you opened up.
I can think of two metrics by which software progress can be measured.
Measure 1: Consider a software to be a sequence of operations which generates some arbitrary particular string of bits. A measure of progress can be, for any random string of bits whose generation we view as useful, how far is the totality of code (libraries, OS portions, etc) from the true Kolmogorov Complexity? Then Moore's Law can be, is this difference between true and actual complexity decreasing exponentially? Put another way, is the level to which we can compress the code specification of software exponentially approaching it's limit? I don't think so. But I also don't think this measure is useful as its too abstract.
Measure 2: For any given piece of software from the past, is the total number of man years required to replicate it decreasing exponentially as a function of time? Again I don't think so, not uniformly. I think it's more polynomial.
Of course there is a theoretical manner by which Exponential progress can be made in software. But some people view it as a dangerous existential threat. Either way, achieving Moore's law for software is not something that should be stepped into blindly.
There has been some progress on this front. Average programmer productivity has been roughly constant over the past several decades in terms of Source Lines of Code. (SLOC - Yes, it's a rotten metric, but it's good enough in huge aggregates.) SLOC per unit of functionality for mainstream languages stopped decreasing when we got to Smalltalk, and hasn't moved much since.
So to progress from here, we either have to:
I'm not sure the 1st is a good idea. When languages get too terse, they get harder to understand, and maintenance and debugging are a big deal.The 2nd has been happening more and more. The way communities share good code has indeed progressed and increased the power of the individual programmer in the past two decades.
The 3rd one is quite a complex issue. I suspect functional programming can play some kind of role here, but it will have to be packaged in a way which is palatable to the programming mainstream.
Much could be done by a programming language that can enable the application of functional programming in much the same spirit as John Carmack's post here:
http://www.altdevblogaday.com/2012/04/26/functional-programm...
Just as OO became mainstream through "impure" implementations, FP could become mainstream through an impure implementation enabling programmers to easily incorporate it into their existing projects.
How about an FP-Lint for popular programming languages that can flag non-FP code in regions that have been designated FP?
[1] http://www.vpri.org/
[2] http://news.ycombinator.com/item?id=846028
Mind you, I have no idea how to do it. It's one of the goals Parrot was aiming for a decade ago, but Parrot doesn't seem to be particularly close to being a good answer yet.