> We didn't "get" physics 50 times better in 1990 compared to 1940
We don't have to. The fundamental rules are discovered slowly. The frontier has been expanded & there's plenty of room in the spaces between. We can now utilize these rules to create & build. The rules of physics are being assimilated by the rest of the culture & it's properties are being utilized more than ever. This trend is accelerating.
There is plenty of room for exploration, "cheap" growth (scaling), & turning over new leaves in biotech, tech, biology, medicine, robotics, ai, software development patterns, etc.
The nice thing about the globalization of the internet is there's an unprecedented amount, quality, & interconnections of minds to advance our global knowledge. We are also undergoing cultural change that fosters collaboration and decentralized intelligence. There's also a lower barrier of entry for one to contribute to global knowledge (i.e. you don't need a PhD nor do you have to be a white male from a privileged background).
>After all the total quantity of intelligence and hard work available around is millionfold what you can provide –you're just a drop of water in the ocean. Rather use your imagination, the one thing that makes you a beautiful unique snowflake. Intelligence and hard work should be merely at the service of our imagination. Think outside of the box. Break out. Shake the axioms.
Given this sort of nonsense, it's no surprise the author is (Probably, since they all are) some kind of vitalist. He probably doesn't think brain emulations are people!
There are very, very few people who are capable of arguing against an idea on its merit if that idea clashes strongly with their worldview. Usually the cognitive dissonance of a differing worldview eclipses understanding to the point that anything with a wildly different context is just "insane."
"Intelligence" as we have come to understand it is not some magical thing, not some exotic element that's impossible to fabricate. In fact, it's probably something mundane and insultingly simple when we get to the core of it.
Reproducing intelligence is mostly a matter of finding the correct formula, the right structures, the right technology. After that it will be boring, ordinary, even disposable. This doesn't sit well with some people who reject that on an emotional level even if they can't figure out any concrete reason why. It just can't be.
We already have enough hardware to fake intelligence, to make a believable Turing Test candidate that could win, but we've yet to figure out how. That much will become clear in the coming decades, guaranteed.
If you look at the progression in computer chess programs, where they were pathetic for the longest time until things came together and the performance of them rose exponentially, matching and then eclipsing grandmasters, it's inevitable that the same pattern will play out in the artificial intelligence space.
> "Intelligence" as we have come to understand it is not some magical thing
I agree that intelligence can be replicated. My preferred definition of intelligence is broad. Intelligence is a system that has i/o & a feedback cycle. Thus, I think all matter & concepts have intelligence. Of course, there are higher forms of intelligence, such as human intelligence (which is not limited to the brain as we have a complete nervous system & more generally, autonomous cells).
I still think it's magical, however. Even if we understand all of the rules & procedures, the experience of it all coming together fills me with awe. It's like playing a game where you understand the rules, yet you can achieve zen & a higher consciousness when playing. Another example is Fractals. The driving equation is simple, yet infinite complexity & beauty is created.
> Reproducing intelligence is mostly a matter of finding the correct formula, the right structures, the right technology. After that it will be boring, ordinary, even disposable. This doesn't sit well with some people who reject that on an emotional level even if they can't figure out any concrete reason why. It just can't be.
Animals & possibly lower life forms have evolved emotions. It's an important part of what drives us to do whatever we do. I would not be surprised if AI will need emotions to reach higher intelligence.
I have also been driven to honor all life & intelligence. I see the interconnections within the system of existence & aspire to respect all of it's parts. When I hold such a mindset, I'm at my best.
Emotion may be necessary for human-like intelligent systems, however I think you've missed his point. He's not saying that emotion will not be created or emerge in synthetic intelligent systems; rather he's saying that emotional arguments, rather than logical arguments, are rubbish.
In other words, "I don't want this to be true, because I don't like the implications, therefore I think that it is not true" is not sound. Whether or not emotion plays an important role in intelligence is irrelevant.
> he's saying that emotional arguments, rather than logical arguments, are rubbish.
That's true if the system in question is entirely based on logic. That means it's probably a static (non-creative) system, as emotions often are involved in creative systems. Note that creating a logical system is itself a creative system.
Side note: part of arguments are about framing the system. Often times, disagreements arise because the parties aren't even talking about the same system. A party will often be focused on a subsystem or an entirely different system.
And what about concepts like art, love, beauty, morality? Are these also simply "computations" of some sort?
I believe your argument holds true for all intelligence that relies on logic. However, I would put forth that the full definition of "intelligence" includes logic, and also something else, let's call it "beyond logic".
For example, see Gödel's incompleteness theorems [1].
You will find that there are statements that can be "true but unprovable". Statements that are "true but unprovable" cannot be computed. But that doesn't make them untrue.
Therefore, there are limits to artificial intelligence.
>And what about concepts like art, love, beauty, morality? Are these also simply "computations" of some sort?
Yes. I would argue that they are no more abstract than any other pattern recognition and the fact that they can be differentiated by perception makes them computable.
You fundamentally ask the "Qualia" question, which for my part at least is well understood to be an emergent property of having perception.
The Godel question is always brought up as though it is some kind of trump card - though it is not. It simply states that, we don't think that the universe can simulate itself without inconsistency. Call that a limit if you want but AI is complete well before we have a consistent universal model.
If I were you I would be more worried about the münchhausen trilemma
The point isn't that everything can be proven, the point is that human brains have no special abilities that can't be emulated by computers. If you emulate every neuron and neurotransmitter in the brain with a sufficiently powerful computer, you will get an intelligence that has the same perception of love, beauty, blah blah blah that people think somehow set them apart from all other matter in the universe.
In addition, once you have it modeled, its possible invent a computer that loves more deeply than any human ever could. Who understands and appreciates the human concept of beauty better than any human ever could etc.
We know these things are emergent properties of very slow meat processors, and therefore can be recreated again on very fast silicon processors.
Are you talking about sensory perception, hormones from the rest of the body affecting cognition and emotion? Any of those effects could be replicated as well.
> And what about concepts like art, love, beauty, morality? Are these also simply "computations" of some sort?
Yes, all that we are is contained within our skulls, in some sort of mess of chemical reactions and computations built ontop of those chemical reactions. Our detailed understanding of those reactions and relationships is limited at this point, however there has been nothing to suggest that anything that "is us" is operating outside of the laws of physics.
In other words, we are materialistic.
That leaves the question of if we are algorithmic. According to the Church-Turing-Deutsch principle, we are. Some people, most notably Roger Penrose, disagree.
Penrose's objection to the algorithmic nature of the mind is rooting in Gödel's incompleteness theorems (I suspect you are familiar with his objections, maybe read his books on the topic?). However it is important to remember that Penrose does not argue against a materialistic mind, merely the algorithmic mind. He believes that there are quantum phenomenon in your wetware that are essential to the human mind, and which allow a violation of the Church-Turing-Deutsch principle.
If he is correct (and I believe that he is not. evidence for such a essential quantum phenomenon is absent), then it doesn't really matter. What it would mean is that any computer constructed to run a human-like intelligence would need to take advantage of those same quantum phenomenon. Anything a materialistic (although not algorithmic) mind can do, a sufficiently advanced materialistic computer can do as well.
The issue I have with your statement is that I don't see why we are special? What about our machinery makes us any more fit to be intelligent than a hunk of doped silicon? I mean, it all seems to come down to electrons flowing through (meat or metal) circuits. Of course, whether or not you feel the same way might partially depend on your religious beliefs. I guess I just don't see anything special about our wetware that couldn't be reproduced on silicon (eventually).
There may be limits, depending on your definition of "logic", but there is no part of the incompleteness theorem that exempts the particular type of computer that happened to evolve over the last few millions of years. There is no spark that is fundamentally unreplicable, despite being replicated every time humans reproduce. Art, love, beauty, and morality are parts of our desires that don't correspond neatly to the idea of self-interested maximization of material wealth or whatever, but there is no rule that an AI must have a simple objective function: one could be built with such goals or without them, with useless ones (paperclip maximizer) or weird ones (e.g. complete disregard for the well-being of the self).
Hopefully if we ever do build some super AI (I'm not opining on any particular time frame in which this might happen, just on theoretical possibility), it will share a variety of our values, and will therefore want to include such values in any AIs it builds, and so on. But values are fundamentally arbitrary - look at the variety of values in human cultures, and consider how much more alien we should expect a nonhuman intelligence to be.
You are spewing complete nonsense. So far as anyone knows the human brain is not capable of doing anything that can't be done by a general purpose computer. There is absolutely no reason to suspect otherwise.
Gödel's incompleteness theorem (and everything else) apply just as strongly to biological brains as to silicon ones.
> If you look at the progression in computer chess programs, where they were pathetic for the longest time until things came together and the performance of them rose exponentially, matching and then eclipsing grandmasters, it's inevitable that the same pattern will play out in the artificial intelligence space.
Chess programs started beating grandmasters when they built computers big enough to recurse a few dozen levels down the dynamic programming tree. Other than a few brainy programming optimizations and some well-chosen search heuristics (like e.g. learning all of Kasparov's opening repertoire) it was all brawn as far as I know. With its countable state space and discrete, turn-based dynamics, chess actually strikes me as one of the easier AI problems you can pose.
If you want a sense of how truly far away we are from real AI go check out what they are doing in computer vision. Classification performance on tasks that a four-year-old could execute without error is in the single digits in some cases (http://image-net.org/challenges/LSVRC/2012/analysis/). It is indeed difficult to imagine a human-like AI in this domain without some sort of singularity occurring, but no one I talk to thinks that such a thing is near.
I disagree strongly. He was using chess as an example of how progress can increase exponentially, not how artificial intelligence is close.
Object recognition is another good example with major breakthroughs in recent years leading to massive improvements. They are already very close to, and in some cases exceeding, human ability.
Let's say I accept that higher, human-like, intelligence is reproducible.
It still doesn't follow that beyond that is some sort of yet in-conceived 'super intelligence'; that intelligent machines can outstrip humanity in any meaningful way and build even more intelligent machines than themselves and on and on in a virtuous cycle as the Singularity predicts.
Given that we have assumed human-like intelligence is reproducible, let's assume only slightly further that intelligence can be represented as some pattern, P, that can be reproduced in media other than human brains (I'm assuming you are meaning "reproducible" in terms of "reproducible within a machine", not the uninteresting meaning of "Every human born has reproduced intelligence").
If P exists, we have all kinds of faster, better, more durable strata we could encode it into than the one evolution happened to hit upon, and that encoding alone could be considered 'super intelligence' in the sense that it would easily out-think you and I. That may not necessarily imply that P can be improved upon exponentially (but if I were your stock advisor---or for that matter, licensed to be so---I'd humbly suggest that wagering against the _total possibility_ that something can be improved tends to be a losing wager). But it wouldn't take too many improvements to get orders of magnitude past our slower-than-a-speeding-bullet organic thought machinery.
It's worth noting that Kurzweil's book doesn't necessarily predict infinite, unbounded growth in intelligence (I'd assume that what we'd call "intelligence" is susceptible to the S-curve of diminishing returns like other technologies). But one doesn't need infinite, unbounded intelligence growth to build something that fundamentally transforms the world we live in.
> encoding alone could be considered 'super intelligence' in the sense that it would easily out-think you and I.
>But it wouldn't take too many improvements to get orders of magnitude past our slower-than-a-speeding-bullet organic thought machinery.
These are HUGE leaps to make.
Not to mention that thinking faster =\= thinking better, which is to say that there is no reason to assume that faster intelligence would not be understandable to the human brain.
I think we're debating definitional points. In this context, I was equating thinking faster and thinking better; if you and I would---when faced with a problem---arrive at conclusion Q, and the AI would arrive at Q twice as fast, it's thinking better.
So if we distinguish thinking faster (I've doubled the rate at which the algorithm runs) from thinking better (I've refined the algorithm so it only takes half as many "brain cycles" or what have you), I don't disagree that there's no guarantee that a faster AI would improve its own algorithm---but I would disagree that a human-level AI with twice the thought speed of a human wouldn't "outstrip humanity." It stands to reason that it should be able to outstrip humanity on any problem domain requiring cognition, simply by definition of measurement (if the machine is coming up with answers twice as fast as its human counterparts, it's "outstripping humanity").
I also hold to the assertion that it wouldn't take too many improvements to get orders-of-magnitude past organic brains. The fastest nerve impulses are clocked in the range of 120 m/s. If you took the pattern of a human brain and replaced the meat with electronic switching, you'd be improving the raw rate of flow by about a million. Given that you've already conceded hypothetically that P is reproducible, electronics alone (without such niceties as optic fiber for any really big jumps), you'd be looking at the processes of P occurring at an (upper-bound) million times the speed it does in the human brain. Does this prove a lower-bound improvement? No; after all, it's not raw speed but switching and parallelization that matters if we're replicated a human brain.
But I submit to you that the lower bound of improvement by transitioning from meat to electronic is probably more than '2'. ;)
I wish I could reply to your last comment directly. HNs comment system is sometimes beyond my understanding.
I make the better vs. faster distinction because, if merely faster, the creative works and solutions to problems such an intelligence could cook up would be fathomable to humans, whereas if they are thinking better somehow, that is not a given. Kurzweil, Vinge et al predict intelligence that would leave ordinary humans behind.
It could be that the problem space (reality) and 'algorithms' (for lack of a better term) used to approach that problem space in what we would term creative and intelligent ways, are as such that even a million fold increase in raw speed would not help very much.
All that being said, I hope we do create durable intelligence at least up to our own capabilities. I personally think it is our responsibility to seed our part of the universe with intelligence and that is really the only practical way to extend intelligence into the wider cosmos.
For one machines could simply be faster than humans. Transistors are thousands to millions of times faster than neurons. In the time it takes a human to solve a problem, an AI could solve thousands. Not to mention improvements in the algorithm itself, or increasing the size of a brain.
I see little justification in the article for the assertion that creativity/imagination^ is something unique and distinct from other aspects of intelligence.
Any system that explores solutions will need to speculate on which branch of decisions will have the greatest chances of yielding a desirable outcome. Any system with enough power to fully explore all possibilities will have enough power to discover other possibilities that it lacks the power to fully explore within it's SLAs. Different intelligent systems that employ different pruning techniques or heuristics will have different "creativities". The more sophisticated the intelligent system, the greater the range of 'creative' thought.
I've seen little reason to think that creativity is some sort of "special sauce" that would not become commoditized as the rest of intelligence is commoditized. Why shouldn't it?
^ Imagination without creativity is merely simulation, something that nobody doubts is mechanical. I therefore conclude that the author is actually identifying creativity as special.
There's a couple of ways I see to go with this discussion. I'll explore both...
1. We are in a temporal frame. The evolution of intelligence has gotten us to this point. Right now, high creativity is unique to humans. I agree that the possibility exists for humans to create creative beings.
2. The definition of creativity is broad at the moment. Pursuits that are defined as "creative" are also changing. As we learn more about creativity (zoom into the definition), we will identify more nuances of creativity. Indeed, it may loop back to a fundamental notion of cognition which itself may be related to a fundamental notion of self-perpetuating systems creating the best possible outcome for it's lineage.
I suspect that creativity is akin to a driving force. So in a fractal system, "creativity" or "emotional drive" is the equation which creates/generates the fractal over time.
I think that it is easiest (and therefore best ;) to define creativity by what we can observe about it, without any internal knowledge about it. So basically "creativity" is some sort of black-box that generates novel outcomes (for a broad sense of outcomes).
On the face of it, I think that for this definition of creativity, I think you could do by simply logically exploring problem spaces that have not been explored before, or by exploring branches in problem spaces that have not been explored before. Maybe throw in a source of randomness when deciding what to explore, to explore possibilities that may not otherwise make sense but could nevertheless lead to interesting outcomes. (The equivalent of a chemist randomly fucking around in his backyard shed).
Some of my friends and I practice a sort of very limited mechanical creativity in the form of a drinking game. We invent new "startup ideas" by ad libbing into the phrase "The ____ of ____", where both blanks are existing companies. We then explore that idea and speculate on what such a company would actually entail. "The Twitter of Githubs", maybe that's a company build around the idea of sharing code snippets, like gists? Not a very interesting idea, but I'm sober at the moment. ;)
Well he said if you kept giving the AI exponentially more computing resources it would get linear growth.
I think it's fairly obvious that providing 1 computer exponentially more resources isn't much different from creating exponentially more brain simulations...
This is not a good Hacker News comment. It's snarky, resorts to name-calling, and presents no substantive view. Thus its signal/noise ratio is poor.
Several people have observed that HN threads are sensitive to initial conditions. Threads seem to go more off the rails when a dismissive one-liner becomes the top comment early. I'm not sure how one could test this, but it does strike me as valid.
All: Please don't post dismissive one-liners. They don't just lower the signal/noise ratio a little—they add a significant risk of lowering it a lot. It's better to post nothing.
When you do comment, please do so "in the spirit of colleagues cooperating in good faith to figure out the truth about something". That's a pg phrase which captures the desired quality of HN so well that we intend to add it to the site guidelines.
This author rails about how scientific progress isn't exponential; but that's not even relevant. The idea of accelerating returns was only ever meant to apply to information technologies. Though they are related on various levels, fundamentally, science != technology. Only when scientific processes are converted into information technologies do they start becoming exponential (as was the case with genome sequencing).
Why the author decouples imagination from intelligence is odd to me, especially as he claims to be an AI researcher. Artificial Imagination and the like has been researched basically since the beginning of AI research.
That is not to say that imagination is a solved problem within AI, far from it, but it is as covered in theory (which this article is about after all) as any other AI problem.
>Rather use your imagination, the one thing that makes you a beautiful unique snowflake.
My guess is that the author thinks there is something special or non-material about being human.
To clarify this point: at the time I meant "imagination" in a human context, as the ability to create new thought systems to operate in, as opposed to doing research within an existing thought framework (typically inherited from your thesis advisor, etc.).
This is analogous to creating a new market with a completely new type of product vs. entering an existing market, for companies.
No, I do not believe there is anything special about human intelligence. Artificial consciousness is among my research interests. As for artificial imagination, it is not exactly "covered in theory" by our current understanding of AI or the human mind --the only kind of AI we know are AIs that operate within rule systems determined by humans. Let's take a simple example, the most general AI we know of, genetic algorithms: if you launch a genetic algorithm to find a solution to some problem, it will merely search through a search space you will have to determine yourself. Artificial imagination would the capability to expand or modify that search space based on previous findings...
>at the time I meant "imagination" in a human context, as the ability to create new thought systems to operate in, as opposed to doing research within an existing thought framework (typically inherited from your thesis advisor, etc.).
I think that is a good enough definition for imagination universally - and I would still not differentiate. How we do it is not known, but it is not unknowable. At least I don't think so. So stating that it will never happen is a bridge too far and I think built on shaky rafters.
>Artificial imagination would the capability to expand or modify that search space based on previous findings...
Check out some of the evolved antennas that came out of AIAA a few years ago. Would that not meet this definition? There is also some work with computer vision that modify prior outputs for new inputs in searches, so that would meet the definition you give.
That said, I won't argue that we have good examples of working imaginations in practice or research - though I bet there are some that I haven't seen - but at least there are papers on the theory. I even have a few of my own ideas that I just haven't put on paper.
> it will merely search through a search space you will have to determine yourself
What makes you think "human imagination" is any different, qualitatively at least? Yes, our minds have a humongous search space at their disposal: what comes through our sense organs, the processed memories of our entire lives and the ability to consume information in formats that are very hard for current AI systems to make sense of (ambiguous written sources, like most of the world's literature, ambiguous communication in form of conversations, art etc.). Add this with the fact that we can all communicate and the search space becomes as big as the minds of the entire human beings alive (thoug admitedly, you have very pool bandwith and availability of access to this amount of information), plus the transmitted pieces from the ones before, and that this space is ever expanding, and you get the "miracle of human imagination".
...but a human level AI will have access to the same thing (by definition, because otherwise we wouldn't call it "human level AI"), so the same kind of "imagination" but augmented by 10...0x times better connectivity and access to it and everything related.
Also, the search space is fixed for human minds too, it's called the universe, and at least if we accept the scientific-objective world view, we don't make it any larger by imagining things (if I imagine a new particle, it doesn't just pop into existence out of my imagination). The only way to defend a difference between "human imagination" and "artificial imagination" is to go against the science based and objective world view.
EDIT: typed "intelligence" instead of "imagination" at the end, sorry...
As an aside, although I completely agree with you, for an alternative fictional perspective I can recommend Neal Stephenson's Anathem - there is one particular section about the mind modelling the universe that I am completely entranced by (and as a bit of background I worked for years as an AI researcher working on systems modelling).
>Intelligence is just a skill, more precisely a meta-skill that defines your ability to get new skills. But imagination is a fucking superpower.
Isn't this the core inconsistency? Intelligence explosion advocates obviously believe that imagination is not an irreducible superpower, but part of the same meta-skill that defines your ability to get new skills.
If you assume that paradigm shifts can be farmed at the same rate of discoveries, it seems like you'd get explosive effects again. This also assumes that fields don't proliferate; i.e., that a single field couldn't produce multiple paradigm shifts that each open up a space of new low-hanging fruit.
Still, good treatment of the question. Much more convincing than a lot of the explanations for.
But if science is becoming exponentially harder and we are making linear scientific progress, then wouldn't that imply that scientific endeavors are increasing exponentially?
Here I say a better model for the Universe is not matter but information, an information architecture of the universe. Matter is buggy, less elegant, event-driven (whereas emergence (acausality) might help).
We're investigating exponentially more if we remain vigilant in preserving the materialist's conceptual scheme.
I am always amazed that the same people who are proponents of the scientific explanation (simple immaterial matter turning into self-aware human beings without any designer. At the same time talk about a very small part of evolution as something unique and magical as if it required a designer.
Why is it so hard to believe that computers created and designed by human beings who themselves didn't even have a designer, can become self-aware?
We also see this in the quantum physics field where some people insist on the "hidden variable" theory.
> That this deadline would arrive just in time to save the proponents of the Singularity from old age is just a weird coincidence that ought to be ignored.
> ...due to the "exponential" rate of progress of science.
I don't remember any great futurist referring to the "exponential rate of progress in science", but rather the accelerating advances in regards to information processing technology specifically.
Most of the article seems to be attacking a straw man argument saying science does not progress exponentially hence no singularity. The arguments for the singularity do not generally involve the advance of science. They are based largely on computing getting faster which is down to technology and economics and does not necessarily require any new scientific discoveries although there may well be some. The basic argument is that is computers keep getting more powerful in the way they have for the last century (see https://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg) then at some point they will have processing power greater than a human brain and assuming software development keeps up that they will be able to exhibit human level intelligence. The hardware getting up to human equivalent levels seems a near certainty and software development while less certain is coming along in a promising manner with Google's self driving cars and the like.
1. Moore's Law is not an ironclad law, it's an observation about improvements in integrated circuit transistor density. Given that there are not that many node shrinks remaining before we hit the fundamental limits of our current technology, scientific discoveries will be necessary to continue increasing transistor density into the far future. It is absolutely not a given that Moore's Law has to continue until the Singularity can occur, although it isn't impossible that it'll be around for several more decades either.
2. The most powerful hardware in the world is useless without software. Improvements in AI theory don't follow Moore's law, and there is no reason they should. Maybe your super-fast computer can run machine learning algorithms on a huge dataset, but it's not clear machine learning alone is sufficient to produce something on the scale of the Singularity. Anyways, strong AI is a very difficult problem, but in no case I know of has the limiting factor ever been the power of the hardware (except maybe for neuronal simulations trying to emulate the activity of a human brain).
1 - you don't need Moore's law, just the observation that computers have got steadily faster for decades and will probably continue.
2 >I know of has the limiting factor ever been the power of the hardware
One example would be computer chess:
"In 1990, futurist Ray Kurzweil predicted that a computer would win a game of chess against a human by 1998. His prediction came true on May 11, 1997, when IBM’s Deep Blue defeated the world chess champion Garry Kasparov."
that prediction was done pretty much entirely on figuring the computational power needed and projecting when we'd get there. Not really rocket science, just some back of the envelope calculations. His 2029 prediction for the turing test is the same stuff.
But will it, and why? This is little more than an article of faith. Computers are getting faster according to a given model, a model whose underlying technological process is reaching fundamental (scientific, not engineering) limits. We have no idea if whatever replaces that model will be anywhere as scalable as Moore's Law.
> computer chess
Computers play chess very differently from humans (they do what amounts to a brute force search of the state space and choose the best option). This is basically taking a well-known algorithm and figuring out how fast a computer needs to be in order to solve it in a tractable manner - cryptographers do the same thing when they decide that a cryptosystem can't be brute-force cracked in a reasonable amount of time. There is no well-defined algorithm for passing the Turing test, and given how so many of the 'interesting' problems in AI are outside the domain of highly logical, rules-based algorithms, using the same metric to measure them is not that useful.
Kurzweil actually postulates that human evolution is exponential in many facets. His paper goes about demonstrating that Moore's law is actually a specialization of a more generic exponential evolution law.
> scientific progress has really been linear. We didn't "get" physics 50 times better in 1990 compared to 1940
It's very hard to quantify scientific progress. The main reason is that we first pick the "low hanging fruits" and advancements get progressively harder. Just think of particle physics: While the first steps could be done with basic lab bench equipment, today we need to build gigantic accelerators to push the boundaries even a little bit. The same is true in other areas: The first steps are comparatively easy, but the more you know the harder it gets to solve the remaining problems, because you already got the easy ones.
So even if perceived progress is linear, considering the increasing difficulty we may have exponential scientific progress.
That is the OP's main point, except turned around. He was measuring progress by results, not by effort expended, and saying that it takes exponential effort to reach linear results.
But considering that the total amount of scientific knowledge is limited, linear progress will find "everything" in a manageable amount of time. Of course we don't know how big the percentage of our total knowledge is, but it's quite possible that we are already getting close to know "everything".
The singularity doesn't promise certain invention (for example it's possible that there is no and will never be a way to go faster than the speed of light) it only promises exponential growth which will bring the "end of science" into the foreseeable future. So if we found maybe 80% of all physics in the last 200 years, with linear progress we would reach the end of physics in 40 years.
I'm of the opinion that the singularity has been here for a while. There have been self-modifying, self-improving biorobots terraforming the Earth to augment themselves physically and mentally since before my grandparents were born.
The model assumes that researchers can predict the impact and effort of their discoveries. If that's the case, then of course progress slows down as earlier researchers pick the low-hanging tasty fruit.
But if we really knew what the research would show, we wouldn't have to do the research.
A smaller point is that it's not really true that progress in algorithms is slowing down. According to a study a couple years ago, in the time it took hardware to improve by a factor of 1000, algorithms improved by a factor of 43,000, for a total speedup of 43 million.
I've long thought that among the chief promises of a self-improving AI is not that it will exponentially make new discoveries in Science. But that it will be able to synthesize and combine existing knowledge in a manner too complex for most humans.
The problem with current knowledge is that humans just aren't capable of handling more than a tiny fraction of it. A functional AI (even of the weaker variety) is capable of handling orders of magnitude more information and arriving at novel combinations of existing knowledge.
Part of the reason the impact-per-researcher has fallen so much is that there is simply so much more to know and consider than there was a century ago. And Computers are really, really good at dealing with volumes of information that would overwhelm a person.
And while a self-improving AI may someday slow down it's rate of self improvement, in the short term its rate of self-improvement would appear exponential as it groks scales of knowledge humans can't cope with.
This is a pretty valuable insight. Even without some kind of innovation to intelligence itself, simply taking the human cognitive process and mapping it onto a strata with several factors more bandwidth (an expectation that isn't unrealistic, given the current implementation is low-speed potential flows and chemical squirting) could significantly push science forward.
This one is an easy bet, since we're already doing this; modern researchers (even the professionals ;) ) use Google and more specialized search tools, which are little more than machines that can read, digest, and correlate symbols at speeds much, _much_ faster than a human (though in a far more narrowly-scoped domain than a human).
It's really a question of how quickly it gets difficult to improve an arbitrary AI algorithm, in terms of the problem-solving capabilities unlocked by having an AI algorithm of that strength. Once you have given beliefs about these quantities, you've got a belief about the result of AI research.
I think that the AI problem gets hard really quickly. It's less clear how much problem solving ability having an AI algorithm gives you. There's three ways I'd use an AI to do AI research better - AI(improve_AI_algorithm), AI(get_more_computing_power), and AI(improve_AI_research_process).
The first is pretty much a run-once sort of thing for a fixed gain. The second is a lot less clear to me - I'm not really familiar with the constraints of chip design, and convincing people to put more resources into AI research hits diminishing returns. The third is much more promising at the current level of AI sophistication, for doing things like hire/fire decisions, scheduling and resource allocation, etc.
Anyhow, my model is roughly a sideways S, with low-hanging fruit getting more reachable at first until it gets exhausted in fairly short order. There's a rather broad distribution of where it ends up afterwards and how quickly the easier improvements get found. The low end wouldn't get classified as a singularity, but the high end would.
I came to the conclusion a while back that humans are just fundamentally religious. Even most people who don't believe in "gods" seem to end up largely clinging to some over-simplified, basically religious model of the world.
As examples:
Modern liberalism = New Testament Christian ethics with the government replacing god.
"Universe is a simulation" = Gnostic/platonic belief system (our universe as a sub-creation created by a flawed sub-creator), with a technological guise.
Belief in the Singularity probably appeals to some people in the same way that an imminent Rapture or Second Coming appeals to others. Especially with the uploading consciousness to computers and living eternally ...
... except for all the new stuff, of course. The fact that Da Vinci dreamed of flying machines doesn't diminish the significance of the actual invention of the airplane or modern commercial air travel.
I have read a lot of thoughts on the supposed "singularity", and something I still have yet to see is for someone to realize that every person who believes in the singularity disregards economics.
They all seem to think that this intelligent machine will improve it's algorithm or find ways to make smaller, more efficient, and better designed chips. On the software side, there is only so much one can do to maximize the intelligent aspect of the algorithm. At one point or another, the program will be maximally efficient at solving whatever problem the algorithm is designed to solve.
On the hardware side, the singularitists sometimes argue that the program will design every better, bigger, and more efficient chip designs as the news chips would be used to build even better chips.
However, who is going to pay for the manufacturing, energy, and maintenance costs of this self improving machine? Eventually, this machine will have to quit trying to design more intelligent machines and start solving HUMAN PROBLEMS. Why? Cause humans will actually pay for the solutions to problems that this intelligent machine can solve.
I'm not saying that this type of self improving machine would never be built. I just think that we will end up splitting the line between devoting resources to improving the machine to encourage future growth, and making money by solving problems that people are willing to pay for.
Also note that as this machine gets more and more powerful, the problem of making itself better will get more and more complex, and as I think we can all agree on, complexity is our greatest enemy.
"I have read a lot of thoughts on the supposed "singularity", and something I still have yet to see is for someone to realize that every person who believes in the singularity disregards economics."
I don't think that's accurate. LessWrong, one of the largest online communities talking about (friendly) AI and the Intelligence Explosion, was founded by Eliezer Yudkowky. He was a co-blogger with Economist Robin Hanson, also a contributor to LessWrong. That's not to say that Robin Hanson agrees with all the views, but they are aware of economics.
In fact, Eliezer Yudkowsky wrote a paper specifically on the economics of the Intelligence Explosion (https://intelligence.org/files/IEM.pdf). He also regularly writes about economic matters.
As to your specific points:
"However, who is going to pay for the manufacturing, energy, and maintenance costs of this self improving machine? Eventually, this machine will have to quit trying to design more intelligent machines and start solving HUMAN PROBLEMS. Why? Cause humans will actually pay for the solutions to problems that this intelligent machine can solve."
"I'm not saying that this type of self improving machine would never be built. I just think that we will end up splitting the line between devoting resources to improving the machine to encourage future growth, and making money by solving problems that people are willing to pay for."
For a very advanced intelligence, creating its own resources isn't a problem. E.g. humans. We make our own resources. The AI doesn't have to rely on humans necessarily. Maybe only in the beginning.
"Also note that as this machine gets more and more powerful, the problem of making itself better will get more and more complex, and as I think we can all agree on, complexity is our greatest enemy."
That sounds reasonable. But the limit that a superior intelligence reaches can be many orders of magnitude above where humans are on the spectrum. E.g., things that are overly complex for monkeys have been achieved by humans, despite the fact that they are much more complex.
Interesting text, but flawed in many ways. A couple of the more important flaws:
- Exponentials often look linear in short x axis observations. Science at t-50 and t-100 may be too short an observation period. Looking at the past 3000 years the exponential characteristic is quite obvious.
- Measuring scientific progress as defined in the paper is obviously simplistic. You don't have a limited set of equal valued discoveries to be found randomly. You have a limited set of increasingly valuable[a] discoveries, with dependencies, to be found. The value rate of discoveries is ignored in the text, but is relevant. Say, observing the periodicity of star movement was important, but the gravitic theory built on that to produce even more valuable information. One allows you to count time, the other is fundamental in travelling to the moon. One can be discovered by an uneducated farmer, the other requires calculus knowledge.
[a] The definition of value is itself difficult. Impact value, as in practical application value, is a metric. Foundational value, as in which doors are now open for research is another. The utility function is probably a mix of both. I'd wager more recent discoveries are better than older ones in both counts (antibiotics vs immune system programming for example). Do not get fooled by the relative impact (you get a division by zero error in the first scientific discovery ever made)
Exponential self-improvement may or may not happen. I'm of the opinion that it will. The first AI is likely to be vastly suboptimal and just thrown together. The very first thing that actually works, but not the best possible thing (same is true of humans.) Therefore there are likely to be lots of optimizations and improvements that can be made - low hanging fruit. So lots of progress will happen very quickly until it starts to approach the actual limits.
But those limits don't have to be anything near humans. Transistors are many thousands of times faster than neurons. It will have massively superior serial processing power, which is good at doing optimization tasks (i.e. the kind of tasks we want it to do or that will be useful in improving itself.) They are general purpose and can be reprogrammed for specific tasks, unlike our brains. And we can create infinitely many instances of them to work on every conceivable problem in parallel, and share resources (human brains are quite limited at that.)
First, let's separate "Intelligence Explosion" from "Singularity" as to many people those terms mean different things.
A lot of the people talking about an "Intelligence Explosion" do not agree that the progress of science has been getting exponentially faster. What they are talking about, is that once we create an AI that can improve its own intelligence in a way that humans can't, then the improved intelligence will know how to further improve its own intelligence, and so on.
Even if there is a limit to the number of discoveries possible in any field, if you're affecting the base rate at which you're making new discoveries, you can (potentially) continue making progress.
More importantly, all of this is besides the point - it could easily be that all the "low hanging fruit" in improving intelligence can make an intelligence that is vastly more intelligent than a human, at which point the so-called "singularity" will have been reached. Or in the words of the "Intelligence Explosion" camp, we will have reached a world where we are no longer the dominant intelligence, possibly by a very wide gap.
I see plenty of reason to think that human intelligence is not a good benchmark for "the maximum intelligence you can get by picking all the low-hanging fruit", since there are many, many easy improvements we can imagine that will almost certainly improve human intelligence (e.g., perfect memory, ability to partially reprogram yourself to do stuff, like for example not to have "bad" desires like eating unhealthily, etc.).
The biggest flaw was the articles strawman that science laws as in physics knowledge as opposed to information technologies like millions of instructions per second or memory capacity are exponentially increasing. Funny hiw he arrogantly "objects your honor" on science exponentially ibcreasing while missing this.
67 comments
[ 4.3 ms ] story [ 120 ms ] threadWe don't have to. The fundamental rules are discovered slowly. The frontier has been expanded & there's plenty of room in the spaces between. We can now utilize these rules to create & build. The rules of physics are being assimilated by the rest of the culture & it's properties are being utilized more than ever. This trend is accelerating.
There is plenty of room for exploration, "cheap" growth (scaling), & turning over new leaves in biotech, tech, biology, medicine, robotics, ai, software development patterns, etc.
The nice thing about the globalization of the internet is there's an unprecedented amount, quality, & interconnections of minds to advance our global knowledge. We are also undergoing cultural change that fosters collaboration and decentralized intelligence. There's also a lower barrier of entry for one to contribute to global knowledge (i.e. you don't need a PhD nor do you have to be a white male from a privileged background).
Given this sort of nonsense, it's no surprise the author is (Probably, since they all are) some kind of vitalist. He probably doesn't think brain emulations are people!
Reproducing intelligence is mostly a matter of finding the correct formula, the right structures, the right technology. After that it will be boring, ordinary, even disposable. This doesn't sit well with some people who reject that on an emotional level even if they can't figure out any concrete reason why. It just can't be.
We already have enough hardware to fake intelligence, to make a believable Turing Test candidate that could win, but we've yet to figure out how. That much will become clear in the coming decades, guaranteed.
If you look at the progression in computer chess programs, where they were pathetic for the longest time until things came together and the performance of them rose exponentially, matching and then eclipsing grandmasters, it's inevitable that the same pattern will play out in the artificial intelligence space.
I agree that intelligence can be replicated. My preferred definition of intelligence is broad. Intelligence is a system that has i/o & a feedback cycle. Thus, I think all matter & concepts have intelligence. Of course, there are higher forms of intelligence, such as human intelligence (which is not limited to the brain as we have a complete nervous system & more generally, autonomous cells).
I still think it's magical, however. Even if we understand all of the rules & procedures, the experience of it all coming together fills me with awe. It's like playing a game where you understand the rules, yet you can achieve zen & a higher consciousness when playing. Another example is Fractals. The driving equation is simple, yet infinite complexity & beauty is created.
> Reproducing intelligence is mostly a matter of finding the correct formula, the right structures, the right technology. After that it will be boring, ordinary, even disposable. This doesn't sit well with some people who reject that on an emotional level even if they can't figure out any concrete reason why. It just can't be.
Animals & possibly lower life forms have evolved emotions. It's an important part of what drives us to do whatever we do. I would not be surprised if AI will need emotions to reach higher intelligence.
I have also been driven to honor all life & intelligence. I see the interconnections within the system of existence & aspire to respect all of it's parts. When I hold such a mindset, I'm at my best.
In other words, "I don't want this to be true, because I don't like the implications, therefore I think that it is not true" is not sound. Whether or not emotion plays an important role in intelligence is irrelevant.
That's true if the system in question is entirely based on logic. That means it's probably a static (non-creative) system, as emotions often are involved in creative systems. Note that creating a logical system is itself a creative system.
Side note: part of arguments are about framing the system. Often times, disagreements arise because the parties aren't even talking about the same system. A party will often be focused on a subsystem or an entirely different system.
I believe your argument holds true for all intelligence that relies on logic. However, I would put forth that the full definition of "intelligence" includes logic, and also something else, let's call it "beyond logic".
For example, see Gödel's incompleteness theorems [1].
You will find that there are statements that can be "true but unprovable". Statements that are "true but unprovable" cannot be computed. But that doesn't make them untrue.
Therefore, there are limits to artificial intelligence.
[1] https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...
Yes. I would argue that they are no more abstract than any other pattern recognition and the fact that they can be differentiated by perception makes them computable.
You fundamentally ask the "Qualia" question, which for my part at least is well understood to be an emergent property of having perception.
The Godel question is always brought up as though it is some kind of trump card - though it is not. It simply states that, we don't think that the universe can simulate itself without inconsistency. Call that a limit if you want but AI is complete well before we have a consistent universal model.
If I were you I would be more worried about the münchhausen trilemma
In addition, once you have it modeled, its possible invent a computer that loves more deeply than any human ever could. Who understands and appreciates the human concept of beauty better than any human ever could etc.
We know these things are emergent properties of very slow meat processors, and therefore can be recreated again on very fast silicon processors.
But I also think that "human" == "brain" is far from a given, and that is an implicit assumption of this viewpoint.
If you're talking about a non-materialistic view of the mind: http://youtu.be/Juriylw7B0g
Yes, all that we are is contained within our skulls, in some sort of mess of chemical reactions and computations built ontop of those chemical reactions. Our detailed understanding of those reactions and relationships is limited at this point, however there has been nothing to suggest that anything that "is us" is operating outside of the laws of physics.
In other words, we are materialistic.
That leaves the question of if we are algorithmic. According to the Church-Turing-Deutsch principle, we are. Some people, most notably Roger Penrose, disagree.
Penrose's objection to the algorithmic nature of the mind is rooting in Gödel's incompleteness theorems (I suspect you are familiar with his objections, maybe read his books on the topic?). However it is important to remember that Penrose does not argue against a materialistic mind, merely the algorithmic mind. He believes that there are quantum phenomenon in your wetware that are essential to the human mind, and which allow a violation of the Church-Turing-Deutsch principle.
If he is correct (and I believe that he is not. evidence for such a essential quantum phenomenon is absent), then it doesn't really matter. What it would mean is that any computer constructed to run a human-like intelligence would need to take advantage of those same quantum phenomenon. Anything a materialistic (although not algorithmic) mind can do, a sufficiently advanced materialistic computer can do as well.
Hopefully if we ever do build some super AI (I'm not opining on any particular time frame in which this might happen, just on theoretical possibility), it will share a variety of our values, and will therefore want to include such values in any AIs it builds, and so on. But values are fundamentally arbitrary - look at the variety of values in human cultures, and consider how much more alien we should expect a nonhuman intelligence to be.
Gödel's incompleteness theorem (and everything else) apply just as strongly to biological brains as to silicon ones.
Chess programs started beating grandmasters when they built computers big enough to recurse a few dozen levels down the dynamic programming tree. Other than a few brainy programming optimizations and some well-chosen search heuristics (like e.g. learning all of Kasparov's opening repertoire) it was all brawn as far as I know. With its countable state space and discrete, turn-based dynamics, chess actually strikes me as one of the easier AI problems you can pose.
If you want a sense of how truly far away we are from real AI go check out what they are doing in computer vision. Classification performance on tasks that a four-year-old could execute without error is in the single digits in some cases (http://image-net.org/challenges/LSVRC/2012/analysis/). It is indeed difficult to imagine a human-like AI in this domain without some sort of singularity occurring, but no one I talk to thinks that such a thing is near.
Object recognition is another good example with major breakthroughs in recent years leading to massive improvements. They are already very close to, and in some cases exceeding, human ability.
Also see https://en.wikipedia.org/wiki/AI_effect
It still doesn't follow that beyond that is some sort of yet in-conceived 'super intelligence'; that intelligent machines can outstrip humanity in any meaningful way and build even more intelligent machines than themselves and on and on in a virtuous cycle as the Singularity predicts.
If P exists, we have all kinds of faster, better, more durable strata we could encode it into than the one evolution happened to hit upon, and that encoding alone could be considered 'super intelligence' in the sense that it would easily out-think you and I. That may not necessarily imply that P can be improved upon exponentially (but if I were your stock advisor---or for that matter, licensed to be so---I'd humbly suggest that wagering against the _total possibility_ that something can be improved tends to be a losing wager). But it wouldn't take too many improvements to get orders of magnitude past our slower-than-a-speeding-bullet organic thought machinery.
It's worth noting that Kurzweil's book doesn't necessarily predict infinite, unbounded growth in intelligence (I'd assume that what we'd call "intelligence" is susceptible to the S-curve of diminishing returns like other technologies). But one doesn't need infinite, unbounded intelligence growth to build something that fundamentally transforms the world we live in.
>But it wouldn't take too many improvements to get orders of magnitude past our slower-than-a-speeding-bullet organic thought machinery.
These are HUGE leaps to make.
Not to mention that thinking faster =\= thinking better, which is to say that there is no reason to assume that faster intelligence would not be understandable to the human brain.
So if we distinguish thinking faster (I've doubled the rate at which the algorithm runs) from thinking better (I've refined the algorithm so it only takes half as many "brain cycles" or what have you), I don't disagree that there's no guarantee that a faster AI would improve its own algorithm---but I would disagree that a human-level AI with twice the thought speed of a human wouldn't "outstrip humanity." It stands to reason that it should be able to outstrip humanity on any problem domain requiring cognition, simply by definition of measurement (if the machine is coming up with answers twice as fast as its human counterparts, it's "outstripping humanity").
I also hold to the assertion that it wouldn't take too many improvements to get orders-of-magnitude past organic brains. The fastest nerve impulses are clocked in the range of 120 m/s. If you took the pattern of a human brain and replaced the meat with electronic switching, you'd be improving the raw rate of flow by about a million. Given that you've already conceded hypothetically that P is reproducible, electronics alone (without such niceties as optic fiber for any really big jumps), you'd be looking at the processes of P occurring at an (upper-bound) million times the speed it does in the human brain. Does this prove a lower-bound improvement? No; after all, it's not raw speed but switching and parallelization that matters if we're replicated a human brain.
But I submit to you that the lower bound of improvement by transitioning from meat to electronic is probably more than '2'. ;)
I make the better vs. faster distinction because, if merely faster, the creative works and solutions to problems such an intelligence could cook up would be fathomable to humans, whereas if they are thinking better somehow, that is not a given. Kurzweil, Vinge et al predict intelligence that would leave ordinary humans behind.
It could be that the problem space (reality) and 'algorithms' (for lack of a better term) used to approach that problem space in what we would term creative and intelligent ways, are as such that even a million fold increase in raw speed would not help very much.
All that being said, I hope we do create durable intelligence at least up to our own capabilities. I personally think it is our responsibility to seed our part of the universe with intelligence and that is really the only practical way to extend intelligence into the wider cosmos.
Any system that explores solutions will need to speculate on which branch of decisions will have the greatest chances of yielding a desirable outcome. Any system with enough power to fully explore all possibilities will have enough power to discover other possibilities that it lacks the power to fully explore within it's SLAs. Different intelligent systems that employ different pruning techniques or heuristics will have different "creativities". The more sophisticated the intelligent system, the greater the range of 'creative' thought.
I've seen little reason to think that creativity is some sort of "special sauce" that would not become commoditized as the rest of intelligence is commoditized. Why shouldn't it?
^ Imagination without creativity is merely simulation, something that nobody doubts is mechanical. I therefore conclude that the author is actually identifying creativity as special.
1. We are in a temporal frame. The evolution of intelligence has gotten us to this point. Right now, high creativity is unique to humans. I agree that the possibility exists for humans to create creative beings.
2. The definition of creativity is broad at the moment. Pursuits that are defined as "creative" are also changing. As we learn more about creativity (zoom into the definition), we will identify more nuances of creativity. Indeed, it may loop back to a fundamental notion of cognition which itself may be related to a fundamental notion of self-perpetuating systems creating the best possible outcome for it's lineage.
I suspect that creativity is akin to a driving force. So in a fractal system, "creativity" or "emotional drive" is the equation which creates/generates the fractal over time.
On the face of it, I think that for this definition of creativity, I think you could do by simply logically exploring problem spaces that have not been explored before, or by exploring branches in problem spaces that have not been explored before. Maybe throw in a source of randomness when deciding what to explore, to explore possibilities that may not otherwise make sense but could nevertheless lead to interesting outcomes. (The equivalent of a chemist randomly fucking around in his backyard shed).
Some of my friends and I practice a sort of very limited mechanical creativity in the form of a drinking game. We invent new "startup ideas" by ad libbing into the phrase "The ____ of ____", where both blanks are existing companies. We then explore that idea and speculate on what such a company would actually entail. "The Twitter of Githubs", maybe that's a company build around the idea of sharing code snippets, like gists? Not a very interesting idea, but I'm sober at the moment. ;)
I think it's fairly obvious that providing 1 computer exponentially more resources isn't much different from creating exponentially more brain simulations...
Several people have observed that HN threads are sensitive to initial conditions. Threads seem to go more off the rails when a dismissive one-liner becomes the top comment early. I'm not sure how one could test this, but it does strike me as valid.
All: Please don't post dismissive one-liners. They don't just lower the signal/noise ratio a little—they add a significant risk of lowering it a lot. It's better to post nothing.
When you do comment, please do so "in the spirit of colleagues cooperating in good faith to figure out the truth about something". That's a pg phrase which captures the desired quality of HN so well that we intend to add it to the site guidelines.
I stopped reading right there. The model that the author is looking for to express his ideas is "logistic growth", or "S curves".
Indeed!
Why the author decouples imagination from intelligence is odd to me, especially as he claims to be an AI researcher. Artificial Imagination and the like has been researched basically since the beginning of AI research.
That is not to say that imagination is a solved problem within AI, far from it, but it is as covered in theory (which this article is about after all) as any other AI problem.
>Rather use your imagination, the one thing that makes you a beautiful unique snowflake.
My guess is that the author thinks there is something special or non-material about being human.
This is analogous to creating a new market with a completely new type of product vs. entering an existing market, for companies.
No, I do not believe there is anything special about human intelligence. Artificial consciousness is among my research interests. As for artificial imagination, it is not exactly "covered in theory" by our current understanding of AI or the human mind --the only kind of AI we know are AIs that operate within rule systems determined by humans. Let's take a simple example, the most general AI we know of, genetic algorithms: if you launch a genetic algorithm to find a solution to some problem, it will merely search through a search space you will have to determine yourself. Artificial imagination would the capability to expand or modify that search space based on previous findings...
I think that is a good enough definition for imagination universally - and I would still not differentiate. How we do it is not known, but it is not unknowable. At least I don't think so. So stating that it will never happen is a bridge too far and I think built on shaky rafters.
>Artificial imagination would the capability to expand or modify that search space based on previous findings...
Check out some of the evolved antennas that came out of AIAA a few years ago. Would that not meet this definition? There is also some work with computer vision that modify prior outputs for new inputs in searches, so that would meet the definition you give.
That said, I won't argue that we have good examples of working imaginations in practice or research - though I bet there are some that I haven't seen - but at least there are papers on the theory. I even have a few of my own ideas that I just haven't put on paper.
What makes you think "human imagination" is any different, qualitatively at least? Yes, our minds have a humongous search space at their disposal: what comes through our sense organs, the processed memories of our entire lives and the ability to consume information in formats that are very hard for current AI systems to make sense of (ambiguous written sources, like most of the world's literature, ambiguous communication in form of conversations, art etc.). Add this with the fact that we can all communicate and the search space becomes as big as the minds of the entire human beings alive (thoug admitedly, you have very pool bandwith and availability of access to this amount of information), plus the transmitted pieces from the ones before, and that this space is ever expanding, and you get the "miracle of human imagination".
...but a human level AI will have access to the same thing (by definition, because otherwise we wouldn't call it "human level AI"), so the same kind of "imagination" but augmented by 10...0x times better connectivity and access to it and everything related.
Also, the search space is fixed for human minds too, it's called the universe, and at least if we accept the scientific-objective world view, we don't make it any larger by imagining things (if I imagine a new particle, it doesn't just pop into existence out of my imagination). The only way to defend a difference between "human imagination" and "artificial imagination" is to go against the science based and objective world view.
EDIT: typed "intelligence" instead of "imagination" at the end, sorry...
http://en.wikipedia.org/wiki/Anathem
But that's not the most general AI we know of. Goedel Machines are the most general AI we know of.
Isn't this the core inconsistency? Intelligence explosion advocates obviously believe that imagination is not an irreducible superpower, but part of the same meta-skill that defines your ability to get new skills.
If you assume that paradigm shifts can be farmed at the same rate of discoveries, it seems like you'd get explosive effects again. This also assumes that fields don't proliferate; i.e., that a single field couldn't produce multiple paradigm shifts that each open up a space of new low-hanging fruit.
Still, good treatment of the question. Much more convincing than a lot of the explanations for.
We're investigating exponentially more if we remain vigilant in preserving the materialist's conceptual scheme.
Why is it so hard to believe that computers created and designed by human beings who themselves didn't even have a designer, can become self-aware?
We also see this in the quantum physics field where some people insist on the "hidden variable" theory.
Simply don't get it.
It's also not a true coincidence: http://intelligence.org/files/PredictingAI.pdf It look like it was just an artifact of the 10 or so predictions Kelly compiled.
I don't remember any great futurist referring to the "exponential rate of progress in science", but rather the accelerating advances in regards to information processing technology specifically.
2. The most powerful hardware in the world is useless without software. Improvements in AI theory don't follow Moore's law, and there is no reason they should. Maybe your super-fast computer can run machine learning algorithms on a huge dataset, but it's not clear machine learning alone is sufficient to produce something on the scale of the Singularity. Anyways, strong AI is a very difficult problem, but in no case I know of has the limiting factor ever been the power of the hardware (except maybe for neuronal simulations trying to emulate the activity of a human brain).
2 >I know of has the limiting factor ever been the power of the hardware
One example would be computer chess: "In 1990, futurist Ray Kurzweil predicted that a computer would win a game of chess against a human by 1998. His prediction came true on May 11, 1997, when IBM’s Deep Blue defeated the world chess champion Garry Kasparov." that prediction was done pretty much entirely on figuring the computational power needed and projecting when we'd get there. Not really rocket science, just some back of the envelope calculations. His 2029 prediction for the turing test is the same stuff.
But will it, and why? This is little more than an article of faith. Computers are getting faster according to a given model, a model whose underlying technological process is reaching fundamental (scientific, not engineering) limits. We have no idea if whatever replaces that model will be anywhere as scalable as Moore's Law.
> computer chess
Computers play chess very differently from humans (they do what amounts to a brute force search of the state space and choose the best option). This is basically taking a well-known algorithm and figuring out how fast a computer needs to be in order to solve it in a tractable manner - cryptographers do the same thing when they decide that a cryptosystem can't be brute-force cracked in a reasonable amount of time. There is no well-defined algorithm for passing the Turing test, and given how so many of the 'interesting' problems in AI are outside the domain of highly logical, rules-based algorithms, using the same metric to measure them is not that useful.
It's very hard to quantify scientific progress. The main reason is that we first pick the "low hanging fruits" and advancements get progressively harder. Just think of particle physics: While the first steps could be done with basic lab bench equipment, today we need to build gigantic accelerators to push the boundaries even a little bit. The same is true in other areas: The first steps are comparatively easy, but the more you know the harder it gets to solve the remaining problems, because you already got the easy ones.
So even if perceived progress is linear, considering the increasing difficulty we may have exponential scientific progress.
The singularity doesn't promise certain invention (for example it's possible that there is no and will never be a way to go faster than the speed of light) it only promises exponential growth which will bring the "end of science" into the foreseeable future. So if we found maybe 80% of all physics in the last 200 years, with linear progress we would reach the end of physics in 40 years.
But if we really knew what the research would show, we wouldn't have to do the research.
A smaller point is that it's not really true that progress in algorithms is slowing down. According to a study a couple years ago, in the time it took hardware to improve by a factor of 1000, algorithms improved by a factor of 43,000, for a total speedup of 43 million.
http://bits.blogs.nytimes.com/2011/03/07/software-progress-b...
The problem with current knowledge is that humans just aren't capable of handling more than a tiny fraction of it. A functional AI (even of the weaker variety) is capable of handling orders of magnitude more information and arriving at novel combinations of existing knowledge.
Part of the reason the impact-per-researcher has fallen so much is that there is simply so much more to know and consider than there was a century ago. And Computers are really, really good at dealing with volumes of information that would overwhelm a person.
And while a self-improving AI may someday slow down it's rate of self improvement, in the short term its rate of self-improvement would appear exponential as it groks scales of knowledge humans can't cope with.
This one is an easy bet, since we're already doing this; modern researchers (even the professionals ;) ) use Google and more specialized search tools, which are little more than machines that can read, digest, and correlate symbols at speeds much, _much_ faster than a human (though in a far more narrowly-scoped domain than a human).
I think that the AI problem gets hard really quickly. It's less clear how much problem solving ability having an AI algorithm gives you. There's three ways I'd use an AI to do AI research better - AI(improve_AI_algorithm), AI(get_more_computing_power), and AI(improve_AI_research_process).
The first is pretty much a run-once sort of thing for a fixed gain. The second is a lot less clear to me - I'm not really familiar with the constraints of chip design, and convincing people to put more resources into AI research hits diminishing returns. The third is much more promising at the current level of AI sophistication, for doing things like hire/fire decisions, scheduling and resource allocation, etc.
Anyhow, my model is roughly a sideways S, with low-hanging fruit getting more reachable at first until it gets exhausted in fairly short order. There's a rather broad distribution of where it ends up afterwards and how quickly the easier improvements get found. The low end wouldn't get classified as a singularity, but the high end would.
As examples:
Modern liberalism = New Testament Christian ethics with the government replacing god.
"Universe is a simulation" = Gnostic/platonic belief system (our universe as a sub-creation created by a flawed sub-creator), with a technological guise.
Belief in the Singularity probably appeals to some people in the same way that an imminent Rapture or Second Coming appeals to others. Especially with the uploading consciousness to computers and living eternally ...
... except for all the new stuff, of course. The fact that Da Vinci dreamed of flying machines doesn't diminish the significance of the actual invention of the airplane or modern commercial air travel.
They all seem to think that this intelligent machine will improve it's algorithm or find ways to make smaller, more efficient, and better designed chips. On the software side, there is only so much one can do to maximize the intelligent aspect of the algorithm. At one point or another, the program will be maximally efficient at solving whatever problem the algorithm is designed to solve.
On the hardware side, the singularitists sometimes argue that the program will design every better, bigger, and more efficient chip designs as the news chips would be used to build even better chips.
However, who is going to pay for the manufacturing, energy, and maintenance costs of this self improving machine? Eventually, this machine will have to quit trying to design more intelligent machines and start solving HUMAN PROBLEMS. Why? Cause humans will actually pay for the solutions to problems that this intelligent machine can solve.
I'm not saying that this type of self improving machine would never be built. I just think that we will end up splitting the line between devoting resources to improving the machine to encourage future growth, and making money by solving problems that people are willing to pay for.
Also note that as this machine gets more and more powerful, the problem of making itself better will get more and more complex, and as I think we can all agree on, complexity is our greatest enemy.
I don't think that's accurate. LessWrong, one of the largest online communities talking about (friendly) AI and the Intelligence Explosion, was founded by Eliezer Yudkowky. He was a co-blogger with Economist Robin Hanson, also a contributor to LessWrong. That's not to say that Robin Hanson agrees with all the views, but they are aware of economics.
In fact, Eliezer Yudkowsky wrote a paper specifically on the economics of the Intelligence Explosion (https://intelligence.org/files/IEM.pdf). He also regularly writes about economic matters.
As to your specific points:
"However, who is going to pay for the manufacturing, energy, and maintenance costs of this self improving machine? Eventually, this machine will have to quit trying to design more intelligent machines and start solving HUMAN PROBLEMS. Why? Cause humans will actually pay for the solutions to problems that this intelligent machine can solve."
"I'm not saying that this type of self improving machine would never be built. I just think that we will end up splitting the line between devoting resources to improving the machine to encourage future growth, and making money by solving problems that people are willing to pay for."
For a very advanced intelligence, creating its own resources isn't a problem. E.g. humans. We make our own resources. The AI doesn't have to rely on humans necessarily. Maybe only in the beginning.
"Also note that as this machine gets more and more powerful, the problem of making itself better will get more and more complex, and as I think we can all agree on, complexity is our greatest enemy."
That sounds reasonable. But the limit that a superior intelligence reaches can be many orders of magnitude above where humans are on the spectrum. E.g., things that are overly complex for monkeys have been achieved by humans, despite the fact that they are much more complex.
- Exponentials often look linear in short x axis observations. Science at t-50 and t-100 may be too short an observation period. Looking at the past 3000 years the exponential characteristic is quite obvious.
- Measuring scientific progress as defined in the paper is obviously simplistic. You don't have a limited set of equal valued discoveries to be found randomly. You have a limited set of increasingly valuable[a] discoveries, with dependencies, to be found. The value rate of discoveries is ignored in the text, but is relevant. Say, observing the periodicity of star movement was important, but the gravitic theory built on that to produce even more valuable information. One allows you to count time, the other is fundamental in travelling to the moon. One can be discovered by an uneducated farmer, the other requires calculus knowledge.
[a] The definition of value is itself difficult. Impact value, as in practical application value, is a metric. Foundational value, as in which doors are now open for research is another. The utility function is probably a mix of both. I'd wager more recent discoveries are better than older ones in both counts (antibiotics vs immune system programming for example). Do not get fooled by the relative impact (you get a division by zero error in the first scientific discovery ever made)
But those limits don't have to be anything near humans. Transistors are many thousands of times faster than neurons. It will have massively superior serial processing power, which is good at doing optimization tasks (i.e. the kind of tasks we want it to do or that will be useful in improving itself.) They are general purpose and can be reprogrammed for specific tasks, unlike our brains. And we can create infinitely many instances of them to work on every conceivable problem in parallel, and share resources (human brains are quite limited at that.)
There is Plenty of Room Above Us: http://intelligenceexplosion.com/2011/plenty-of-room-above-u...
First, let's separate "Intelligence Explosion" from "Singularity" as to many people those terms mean different things.
A lot of the people talking about an "Intelligence Explosion" do not agree that the progress of science has been getting exponentially faster. What they are talking about, is that once we create an AI that can improve its own intelligence in a way that humans can't, then the improved intelligence will know how to further improve its own intelligence, and so on.
Even if there is a limit to the number of discoveries possible in any field, if you're affecting the base rate at which you're making new discoveries, you can (potentially) continue making progress.
More importantly, all of this is besides the point - it could easily be that all the "low hanging fruit" in improving intelligence can make an intelligence that is vastly more intelligent than a human, at which point the so-called "singularity" will have been reached. Or in the words of the "Intelligence Explosion" camp, we will have reached a world where we are no longer the dominant intelligence, possibly by a very wide gap.
I see plenty of reason to think that human intelligence is not a good benchmark for "the maximum intelligence you can get by picking all the low-hanging fruit", since there are many, many easy improvements we can imagine that will almost certainly improve human intelligence (e.g., perfect memory, ability to partially reprogram yourself to do stuff, like for example not to have "bad" desires like eating unhealthily, etc.).