I'm not sure if anyone agrees with me, but I feel like software development will outweigh the need for significantly faster chips over the next 5-10 years. It seems like all the buzz has been about machine learning, signal processing, software defined networking and online education rather than the latest and greatest from AMD/Intel. That's not to say new hardware development isn't necessary, it just seems like there's greater potential for software development in the coming years.
Not really, sorting Algorithms as a classic example have been mostly stagnant for the last 50 years. There have been plenty of advances in a wide range of areas, but 'Algorithms' is way to generic a concept to fit any sort of progress curve.
Sorry, I have made 200x speedups in a specific code over the course of a few weeks that does not mean I am millions of times faster than mores law, just that I spent more time solving that problem. If you take the performance on that data set use it on a benchmark on the same hardware it's not going to be 43,000 times as fast in another 15 years due to 'better' algorithms. After all it's been 2 years is the code 10 times as fast on today's hardware?
PS: Not to mention architecture specific improvements such as ever increasing L3 cache sizes that make may algorithms a lot faster on today's hardware without showing similar speedups on hardware that's 10 years old.
No, they really do mean that on the same hardware, solving the same problems, we have algorithms that will solve those problems 43,000 times faster compared to fifteen years ago...at least for the problems they looked at. And that progress is continuing.
The assumption here is that at any given time, you're reading the latest published papers and implementing the best known algorithms, in a field that uses complicated algorithms on big problems.
It's not just because of faster chips that Google has a self-driving car now.
(As for sorting, it's been proven that the best you can do is O(n log n), and John von Neumann achieved that with mergesort in 1945.)
With a large enough data set O(n^2) vs O(n^1.99999) be 43,000 times faster or 43,000,000 times faster. Which is one of the reasons that metric is next to meaningless. Looking back you can find plenty of great speedups but as I said it's in no way steady progress across all fields.
PS: Also Radix sorting is O(n) operation under conditions where mergesort is O (n log n). (AKA in practice merge sorting distinct strings is 0 (n * (log n) * (log n) as you need to look at an ever increasing number of symbols, or if your strings have a fixed max length or if your theoretical computer can compare infinitely long strings in a single step then Radix sorting is also O(n).)
I should have specified "for sorting algorithms that depend on comparing elements to each other."
You're right that it's difficult to measure algorithm progress, but using problem sizes that you're interested in running in practice is probably a decent heuristic. "I wanted to run this problem but it would have taken 80 years. But then I caught up on recent published algorithms and look, I can do it in an hour on the same machine."
I don't think I claimed the progress was steady. Then again, Moore's Law might get more jumpy as silicon wafers reach their limits, and we transition to memristors, spin devices, or whatever else they come up with.
But machine learning, signal processing, software defined networking are exactly the fields that use non intel/amd processors, and see a lot of development effort that goes into unique hardware architectures.
The early and mid-20th centuries looked like they were on the way to the overpower future: if trends in power generation continued as they had, by now we'd all have access to the dedicated output of a few nuclear power plants for a hundred dollars a month. It is interesting to speculate on what technology would arise at that point.
Instead power generation and storage turned out to be some combination of harder than expected and less desirable than thought. We got the infotech future instead, the path that wasn't foreseen, but is probably the better one from the point of view of living standards, since it drives (a) medicine, and (b) technological development as a whole.
The point here is that people still live in the 50s when they knee-jerk about the technologies they consider progress - they think about things that are on the line of increasing power rather than on the line of increasing data.
With every good comes a bad, and our optimism always needs to be grounded. Synthetic biology could lead to new forms of bioterrorism; surveillance technologies—which are becoming ever more sophisticated—already provide governments more information than Big Brother ever dreamed about; no guidelines have yet been developed for ethics in the exponential era. My worry: will humanity evolve fast enough to fulfill its increasing responsibilities?
I've learned in several years of writing opinion articles and reading them that there is easy optimism and there is easy pessimism. Both have benefits and drawbacks. Neither adequately describe where we're at as a civilization.
Back in the 1940s, the A-bomb caused a lot of really smart people to ask "Yes, we've invented it, but are we ready as a species and a society to handle it? Do we have the morals and culture in place to know what to do with this kind of change?"
It was an incredible advancement in application of raw power -- but it was at the state level, and states have some mechanisms already in place for managing big things that kill people.
What's happening now is that things with more and more impact are being made available to smaller and smaller groups of people. There are no mechanisms in place to deal with that. There is no culture, no common morality, no historical tradition to draw on. It's like four thousand years of philosophy and history have prepared us for a huge exam -- one in which I am not sure we have studied enough.
This is the reason you see the state trying to intrude in on so many areas of the Information Age -- huge amounts of social and information power is being widely distributed in ways that never have happened before, and they feel that they should rightly be in control of it all. (I disagree strongly, but looking at history it makes sense why they would think so.)
Having said that, I'm mildly optimistic short-term (1-40 years), very pessimistic medium-term (40-400 years), and agnostic long-term (after the singularity, if it happens). We'll see. To address the post directly, yes, the next decade will continue to show improvement just as the past one has. It will also keep surfacing things trends that we've never seen before but haven't reached crisis levels yet. It is definitely an incredible time to be alive.
As a counter to this, humans have been doing small groups more or less effectively for... well forever. There are certain controls built into small groups that aren't at state scale:
* stronger pressure to fit the ruleset/norms
* stronger "altruistic" motives - people within the group getting the group ahead is a stronger drive in small groups than at a state level (generally as a trend, yes there are counter examples)
* more people, in a smaller group, the members are all people, and you know them, compared to at a state level, where most people are not actual people, but 'others' (again general trend, also, other is a specific term, look it up).
* small groups are more constrained geographically. They can only exclusively occupy small spaces, or must be interspersed geographically with other groups. This necessitates certain considerations states don't have for the 'nuclear' option - e.g. sure we could nuke them, and kill all our crops and kids too, that might be dumb.
I'm not saying this is a full counter, and renders the stuff you are saying incorrect, nor am I suggesting a full anarchist solution to the future. I just am pointing out that there is some balances to the problem you present. I see the points I mentioned above as bringing less state authoritarianism, and a general aggregate good for the majority of people as small groups have the ability to affect change for themselves. Of course, some of the concerns you bring up are very scary and very real. The trick for us (humanity) is to find a sane balance and strike it.
Thanks for the counter-points. I believe your argument is in some way a version of "but just like the A-bomb example, we already have mechanisms in place that can help deal with this"
This is true. Note that the A-bomb situation may very well still spiral out of control over the next 20-40 years. Too often we get lost in arguing immediate benefits or drawbacks and lose the plot on the bigger picture.
Suffice it to say that I don't understand how these local group dynamics will do anything except exacerbate the state's interest in controlling what the population knows and thinks. I imagine "good" effects, like promoting altruism, will be encouraged. "Bad" effects, like pointing out gaping problems with civil liberties and calling for massive peaceful political change, will be stifled.
Also note that traditional ideas of small groups have them all being in the same geographical place. This is no longer true, and this alone might have greater ramifications than any of the other technological changes do. We could very well see (medium-term) a re-introduction of the clan and lots of associated desultory and internecine warfare. Or just general chaos and instability. Beats me, I'm just an internet commenter. You should know what kind of track record we have :)
It's interesting that you frame the next forty years as something you're mildly optimistic about and beyond that pessemistic (as do I). I wonder if our cognitive dissonance is involved (personally and societally) because we expect to live out a reasonable amount of what's left of our lives ok, but beyond that?
We've got some real challenges stacked against us as a species. I personally think they're probably insurmountable, but then I never guessed growing up that I'd be talking to you by tapping on a little glass screen while sitting on my own in my living room.
The point I'm trying to make isn't really associated with an exact period of 40 years. Looking back on these kinds of discussions, many times people who watched trends and worried about underlying philosophical problems got it right -- but only after decades or perhaps centuries have passed. Usually things come along and all the old folks say it's terrible, all the young folks think it's awesome, and not much is accomplished. Fundamental changes also fit into this pattern, but the problems never show up in any reasonable amount of time for people to remember who gave the warnings. So we forget. I could use a lot of examples, but the arguments against the ratification of the Constitution of the United States come to mind. All of the arguments against the structure were panned as being wild absurd fantasies: that there would be a standing army, that the government would consistently overspend, that the federal government would take more and more control from the states, and so on.
They all turned out to be true, but the "winners" of these arguments lost in the short term and weren't around to see how prescient they were. As these trends continue it's the rest of us, many generations later, that have to deal with the problems that were easily foretold at the beginning.
So it might not be 40 years. Might be 100. As long as we agree that these predictions are real problems whether they happen next week or in 2200 we can carry the discussion forward.
"There is no culture, no common morality, no historical tradition to draw on."
There is a hundred centuries of culture, moral evolution, historical tradition, and also philosophy, art, literature, and science to draw from, if not more. It is not uniform, it is not monolithic, but it is there. If we took a random collection of human beings from all over the Earth and we dumped them on a planet with nothing they would develop society, culture, ethics, and government within a short matter of time.
Yes it's scary, and it's going to get scarier. And yes there is still a lot we need to do to make sure that things go well. But this isn't the first time that the world has been rocked by titanic power shifts into the hands of "unprepared" individuals (see: the automobile, the printing press, firearms, industrialization). Ultimately we're already past the point where we could stop it, so we best figure out how to deal with it.
Seems like you really want to take issue with my post but I have no idea what the hell you're going on about. Sorry for the language, but this isn't the first time, so I'm a bit frustrated.
Yes, we must deal with it. Yes, there are examples in various formats from the past but nothing directly applicable. Drawing inferences and finding a common ontology would be difficult in the best of times. No, it is not especially scary. I said I was pessimistic, not that I was afraid of anything. So we're living in the latter days of the Roman Empire. It's just a thing that is part of our existence. No, this isn't the first time the world has been rocked with change but the change today is occurring at a much greater rate than ever before.
Like I said, I'm not sure we disagree on much. Apologies if it wasn't clear from my comment what my point was. Historical analogies are much more difficult to draw than at any time in our past, philosophical underpinnings of the way governments and groups of people work need to be applied at the principles level and not the policy level, and dozens of trends kick off each decade which will have major impacts down the road. Nothing to fear; this is just where we are. Understanding the problem and talking about it helps gain a common understanding for the required adaptation that will eventually arrive. Evolutionary adaptation can be very painful to experience.
I find it hard to believe that any decade will be more innovative than the 1960s, which had massive structural changes across the board: to both hard and soft sciences; international politics and the routine appearance of new countries; explosion of mass communication; major shifts in art and music; widespread experimentation with wildly different religions; huge increases in foreign travel; significant strides forward in civil rights and self determination across the globe... it's hard to think of a field that didn't have strong if not fundamental changes in the 1960s. I guess we have another eight years before we'll really know...
I doubt there would be, because historically it has been far more common to assume that the past was a Golden Age from which we've fallen due to our sins (or other appropriate word). Optimism about the present is a fairly recent phenomenon.
"In this and the next decade, we will begin to make energy and food abundant, inexpensively purify and sanitize water from any source, cure disease, and educate the world’s masses."
I may just be cynical, but I don't see how that automatically follows from rapid tech advances. I thought most of the problems of distributing food and education were "people problems" that won't just go away because you have fancy gadgets.
I'm getting a little tired of articles making claims about the rate of technological progress (arguing either that it's speeding up or slowing down) that don't set any objective qualifications for where we should measure a given innovation as falling on the timeline.
Every science mentioned in this article has been the product of decades of research. All have some practical applications already in progress. All have more promising applications on the horizon.
But if you provide no measurements or qualifications from which we can judge to which decade they belong, you can simply drop goalposts where-ever you like to make any argument you like.
unless part of learning how to do something is learning how difficult it is. You don't obtain a degree in a difficult field and then feel you are further behind because you understand the challenges better.
But yeah, the perception of how far we have to go works that way.
Agreed. This article, and those like it, essentially support their thesis with a bunch of examples, which doesn't really measure or prove anything.
I can think of a number of circumstantial arguments that would indicate technological progress is growing. For example, that world population is at an all-time high, and therefore most likely more people are working on research than ever before. However this doesn't actually prove anything either.
The next few decades will be a time of progress & disparity. No doubt that technology and innovation will flourish, but access to that technology and its benefits will not be well distributed.
Can anybody list any truly significant scientific or technological advances in the past 40 years? Say, starting from late 60-s, the time when ARPANET was developed?
I think writing a novel genome on a computer, injecting it into a cell and 'rebooting' the cell as an organism like no other and descended from no other before it is significant.
Eh, opinion pieces like this typically annoy me. They are about as bad as your daily horoscope: They talk about the future as if it is something that will happen to us and is already largely predetermined rather than something we can activly participate in creating and shaping. We all make choices every single day which help shape the future. I think the world would be a better place if we talked more about what we want and how we can make it happen and less about imaginary scenarios of this sort, whether they are gloom-and-doom or pie-in-the-sky.
39 comments
[ 3.2 ms ] story [ 97.5 ms ] threadhttp://agtb.wordpress.com/2010/12/23/progress-in-algorithms-...
PS: Not to mention architecture specific improvements such as ever increasing L3 cache sizes that make may algorithms a lot faster on today's hardware without showing similar speedups on hardware that's 10 years old.
The assumption here is that at any given time, you're reading the latest published papers and implementing the best known algorithms, in a field that uses complicated algorithms on big problems.
It's not just because of faster chips that Google has a self-driving car now.
(As for sorting, it's been proven that the best you can do is O(n log n), and John von Neumann achieved that with mergesort in 1945.)
PS: Also Radix sorting is O(n) operation under conditions where mergesort is O (n log n). (AKA in practice merge sorting distinct strings is 0 (n * (log n) * (log n) as you need to look at an ever increasing number of symbols, or if your strings have a fixed max length or if your theoretical computer can compare infinitely long strings in a single step then Radix sorting is also O(n).)
You're right that it's difficult to measure algorithm progress, but using problem sizes that you're interested in running in practice is probably a decent heuristic. "I wanted to run this problem but it would have taken 80 years. But then I caught up on recent published algorithms and look, I can do it in an hour on the same machine."
I don't think I claimed the progress was steady. Then again, Moore's Law might get more jumpy as silicon wafers reach their limits, and we transition to memristors, spin devices, or whatever else they come up with.
Instead power generation and storage turned out to be some combination of harder than expected and less desirable than thought. We got the infotech future instead, the path that wasn't foreseen, but is probably the better one from the point of view of living standards, since it drives (a) medicine, and (b) technological development as a whole.
The point here is that people still live in the 50s when they knee-jerk about the technologies they consider progress - they think about things that are on the line of increasing power rather than on the line of increasing data.
I've learned in several years of writing opinion articles and reading them that there is easy optimism and there is easy pessimism. Both have benefits and drawbacks. Neither adequately describe where we're at as a civilization.
Back in the 1940s, the A-bomb caused a lot of really smart people to ask "Yes, we've invented it, but are we ready as a species and a society to handle it? Do we have the morals and culture in place to know what to do with this kind of change?"
It was an incredible advancement in application of raw power -- but it was at the state level, and states have some mechanisms already in place for managing big things that kill people.
What's happening now is that things with more and more impact are being made available to smaller and smaller groups of people. There are no mechanisms in place to deal with that. There is no culture, no common morality, no historical tradition to draw on. It's like four thousand years of philosophy and history have prepared us for a huge exam -- one in which I am not sure we have studied enough.
This is the reason you see the state trying to intrude in on so many areas of the Information Age -- huge amounts of social and information power is being widely distributed in ways that never have happened before, and they feel that they should rightly be in control of it all. (I disagree strongly, but looking at history it makes sense why they would think so.)
Having said that, I'm mildly optimistic short-term (1-40 years), very pessimistic medium-term (40-400 years), and agnostic long-term (after the singularity, if it happens). We'll see. To address the post directly, yes, the next decade will continue to show improvement just as the past one has. It will also keep surfacing things trends that we've never seen before but haven't reached crisis levels yet. It is definitely an incredible time to be alive.
* stronger pressure to fit the ruleset/norms
* stronger "altruistic" motives - people within the group getting the group ahead is a stronger drive in small groups than at a state level (generally as a trend, yes there are counter examples)
* more people, in a smaller group, the members are all people, and you know them, compared to at a state level, where most people are not actual people, but 'others' (again general trend, also, other is a specific term, look it up).
* small groups are more constrained geographically. They can only exclusively occupy small spaces, or must be interspersed geographically with other groups. This necessitates certain considerations states don't have for the 'nuclear' option - e.g. sure we could nuke them, and kill all our crops and kids too, that might be dumb.
I'm not saying this is a full counter, and renders the stuff you are saying incorrect, nor am I suggesting a full anarchist solution to the future. I just am pointing out that there is some balances to the problem you present. I see the points I mentioned above as bringing less state authoritarianism, and a general aggregate good for the majority of people as small groups have the ability to affect change for themselves. Of course, some of the concerns you bring up are very scary and very real. The trick for us (humanity) is to find a sane balance and strike it.
This is true. Note that the A-bomb situation may very well still spiral out of control over the next 20-40 years. Too often we get lost in arguing immediate benefits or drawbacks and lose the plot on the bigger picture.
Suffice it to say that I don't understand how these local group dynamics will do anything except exacerbate the state's interest in controlling what the population knows and thinks. I imagine "good" effects, like promoting altruism, will be encouraged. "Bad" effects, like pointing out gaping problems with civil liberties and calling for massive peaceful political change, will be stifled.
Also note that traditional ideas of small groups have them all being in the same geographical place. This is no longer true, and this alone might have greater ramifications than any of the other technological changes do. We could very well see (medium-term) a re-introduction of the clan and lots of associated desultory and internecine warfare. Or just general chaos and instability. Beats me, I'm just an internet commenter. You should know what kind of track record we have :)
We've got some real challenges stacked against us as a species. I personally think they're probably insurmountable, but then I never guessed growing up that I'd be talking to you by tapping on a little glass screen while sitting on my own in my living room.
The point I'm trying to make isn't really associated with an exact period of 40 years. Looking back on these kinds of discussions, many times people who watched trends and worried about underlying philosophical problems got it right -- but only after decades or perhaps centuries have passed. Usually things come along and all the old folks say it's terrible, all the young folks think it's awesome, and not much is accomplished. Fundamental changes also fit into this pattern, but the problems never show up in any reasonable amount of time for people to remember who gave the warnings. So we forget. I could use a lot of examples, but the arguments against the ratification of the Constitution of the United States come to mind. All of the arguments against the structure were panned as being wild absurd fantasies: that there would be a standing army, that the government would consistently overspend, that the federal government would take more and more control from the states, and so on.
They all turned out to be true, but the "winners" of these arguments lost in the short term and weren't around to see how prescient they were. As these trends continue it's the rest of us, many generations later, that have to deal with the problems that were easily foretold at the beginning.
So it might not be 40 years. Might be 100. As long as we agree that these predictions are real problems whether they happen next week or in 2200 we can carry the discussion forward.
Everyone should check out John Robb's blog, it covers this kind of thing: http://globalguerrillas.typepad.com
http://globalguerrillas.typepad.com/globalguerrillas/2008/02...
http://globalguerrillas.typepad.com/globalguerrillas/2011/05...
http://globalguerrillas.typepad.com/globalguerrillas/2007/01...
http://globalguerrillas.typepad.com/globalguerrillas/2010/03...
There is a hundred centuries of culture, moral evolution, historical tradition, and also philosophy, art, literature, and science to draw from, if not more. It is not uniform, it is not monolithic, but it is there. If we took a random collection of human beings from all over the Earth and we dumped them on a planet with nothing they would develop society, culture, ethics, and government within a short matter of time.
Yes it's scary, and it's going to get scarier. And yes there is still a lot we need to do to make sure that things go well. But this isn't the first time that the world has been rocked by titanic power shifts into the hands of "unprepared" individuals (see: the automobile, the printing press, firearms, industrialization). Ultimately we're already past the point where we could stop it, so we best figure out how to deal with it.
Yes, we must deal with it. Yes, there are examples in various formats from the past but nothing directly applicable. Drawing inferences and finding a common ontology would be difficult in the best of times. No, it is not especially scary. I said I was pessimistic, not that I was afraid of anything. So we're living in the latter days of the Roman Empire. It's just a thing that is part of our existence. No, this isn't the first time the world has been rocked with change but the change today is occurring at a much greater rate than ever before.
Like I said, I'm not sure we disagree on much. Apologies if it wasn't clear from my comment what my point was. Historical analogies are much more difficult to draw than at any time in our past, philosophical underpinnings of the way governments and groups of people work need to be applied at the principles level and not the policy level, and dozens of trends kick off each decade which will have major impacts down the road. Nothing to fear; this is just where we are. Understanding the problem and talking about it helps gain a common understanding for the required adaptation that will eventually arrive. Evolutionary adaptation can be very painful to experience.
Is there a term for this perception?
I may just be cynical, but I don't see how that automatically follows from rapid tech advances. I thought most of the problems of distributing food and education were "people problems" that won't just go away because you have fancy gadgets.
Every science mentioned in this article has been the product of decades of research. All have some practical applications already in progress. All have more promising applications on the horizon.
But if you provide no measurements or qualifications from which we can judge to which decade they belong, you can simply drop goalposts where-ever you like to make any argument you like.
The biggest thing happening how much easier and faster it is to share information. The full impact of that is probably not yet apparent.
But yeah, the perception of how far we have to go works that way.
I can think of a number of circumstantial arguments that would indicate technological progress is growing. For example, that world population is at an all-time high, and therefore most likely more people are working on research than ever before. However this doesn't actually prove anything either.
Greg Bear's "Queen of Angels" (and "Slant") set up a great example of this world: http://en.wikipedia.org/wiki/Queen_of_Angels_(novel)