Predictions about something not being possible are difficult, because you can never be right. You are either proven wrong our 'we don't know yet'. Therefore these quotes are rather meaningless I believe. (I see SAs point of encouraging people to go for the moonshots, but still.)
His point is that many experts have been wrong when assuming the limitations of technology in the fields that they have mastered. He is simply trying to mute the argument that predictions from AI experts are sufficient in dismissing AI concerns.
The Einstein/Wright Brothers quotes really hit this home for me.
Maybe the quotes are meaningless to you because you already agree. But they might persuade others that would otherwise assume discussing AI impacts on humanity is a waste of time.
I have doubts about the Einstein quote. Atoms had been split at will for decades by then.
Szilard hadn't yet proposed a theory of nuclear chain reactions, but according to some cites of the quote Einstein didn't say it until 1934 - which was after Szilard.
I don't have a problem with the possibility that suprahuman intelligence may be possible. I do have a problem with the fact that currently we have no idea what the concept may even mean - and right now, more immediate cybersecurity issues are being neglected.
Computers are already better than humans at many activities. From playing chess to landing planes to learning how to play a video game - a computer with the right software is much better at these than an average human, and is often at least good as the best humans.
Take that to the black corner, and worms and botnets are already a serious problem.
We don't need to wait for the Internet to become sentient and start talking to us in a deep echoey robot voice to worry about cyberthreats.
There's more than enough to deal with already. And if you're going to try to regulate and contain a future AI, making current systems as secure as possible seems like a realistic place to start.
"I have doubts about the Einstein quote. Atoms had been split at will for decades by then."
Not in a chain reaction. When Szilárd described the concept of a chain reaction to Einstein, Einstein was shocked. He said "I never thought of that!"
Until then, nuclear physics was purely an academic enterprise. There were few applications for radioactive materials. Radioactive decay just happened at its own slow pace, and not much could be done with it. X-rays could be used to pump the process, but less energy came out than what was put in. Suddenly the nuclear physicists realized they had a tiger by the tail. This was going to change the world, not necessarily for the better.
Like @TheOtherHobbes above, I had disbelief about the Einstein quote ("Wasn't Einstein presciently aware of where nuclear fission technology was going?").
But, poking around a bit, I came to the same understanding you have. Here's some more of the time line:
The quote in the OP (which I can't find online; the Einstein archives at Caltech are, alas, not indexed) about Einstein's skepticism about nuclear energy is dated 1932. The first demonstrations of nuclear fission were years later, in late 1938 and into 1939. And as you said, Einstein is reported to have said, "I had not thought of that." -- regarding the chain reaction.
The fabled Einstein-Szilard letter to Franklin Roosevelt, warning about the Nazis getting the atomic bomb, was written in August 1939 (http://en.wikipedia.org/wiki/Einstein–Szilárd_letter), and then relayed to Roosevelt in October after the flurry of activity due to the Nazis invading Poland had died out.
Many good points. I can't argue for or against you about resource allocation because I have no idea what resources are available. I can't even argue for regulation, because I know so little about the current landscape. But I can say that AI is possible, people are working on it, therefore people should be encouraged to discuss the potential threats and safeguards for it.
My opinions currently stop at this is an important topic and anyone who is interested should be exploring it via their chosen medium.
Do many "otherwise smart" people actually believe "superhuman machine intelligence is prima facie ridiculous"? I'd like to see some citations :-). I think smart people tend to have much more nuanced views.
See the reaction of tech industry after Musk donated 10M USD to AI research. It sort of divided into two groups, one saying that it's great choice and another claiming that he's an idiot and AI is a hoax (for the record, I'm in the former group).
>Do many "otherwise smart" people actually believe "superhuman machine intelligence is prima facie ridiculous"?
I don't know how "otherwise smart" I am, but I wonder how we would be able to tell that a machine intelligence was "superhuman" as opposed to "buggy".
For example, suppose we build a super-AI and ask it, "Is Shinichi Mochizuki's proof of the ABC conjecture correct" [1]. What would we do if it said "yes"?
(Of course, if "superhuman" just means "able to do things humans already know how to do and verify, but lots faster", then we're already there).
What if we ask a super-AI for a proof of the ABC conjecture, and the result is something too complicated for humans to verify?
My point, if I have one, is that when I read about "superhuman machine intelligence", sometime people seem to mean "capable of knowledge that humans couldn't figure out on their own but that humans can understand once they see it"; and sometimes they seem to mean "capable of knowledge that is beyond human capacity to even verify".
I think development machine intelligence of the first kind is extremely likely, but I'm more skeptical about the second kind.
I am, by most measures, pretty smart, and I agree with Dijkstra that the question of whether a computer can think is as interesting as whether a submarine can swim.
The Strong AI hypothesis assumes a mechanistic universe, if not necessarily a materialistic one, and I think that condition is false.
Sam's last post on Machine Intelligence, and the worries regarding it, received a lot of dismissal here on HN from people who thought that the idea is completely unfounded and implausible.
Some patents are classified each year. A clerk who has seen many classified patents would have a unique opinion on "blue oceans" for investment opportunities, especially if they knew how to prevent new patents from being classified, by avoiding certain areas of research, http://fas.org/sgp/othergov/invention/ . In TRIZ terminology, they would have different psychological inertia.
Is there a yearly list of declassified patents over the last few decades? This would be similar to lists of expired patents or books which enter the public domain in some countries.
"I believe that in about fifty years' time it will be possible, to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning." - You-Know-Who, in 1950.
It's a bit ironic that Sam missed this one quote that would have be very relevant to the post, both in term of the conclusion and the topic at hand (it's hard to predict stuffs) ;). It's interesting to note that the prediction was wrong on both counts: our machine is like a gazillion times more powerful than the quote, and we're doing nowhere near the capability of the prediction.
From the last HN comments on the topic, while it's true that there are certain people who believe that strong AI is impossible, the stance I've got from AI/robotics researchers were that it's just silly to talk about general AI at this point: it's so far away. This post from Karpathy would be one, for example: http://karpathy.github.io/2012/10/22/state-of-computer-visio... .
Because of that, any talk and concern on the topic is a bit silly, especially about regulation of the subject. I mean, can you imagine people in the 1600 trying to figure out how to regulate the air traffic that we have right now? Or should we talk about space travel regulation now? Because seriously, chance are that will happen before we have singularity/ strong AI.
If we're really concern about the danger of strong AI, then I'm more in favored dealing with it by the way of Eliezer Yudkowsky and the Singularity Institute (by making sure that whatever we're researching toward is "Friendly"). Even though I'm also thinking that they're too optimistic. I'm not against immortality in my lifetime though.
Now, a talk about more sophisticated automated/ autonomous system that do funny things (in a good or bad way), or their risk, that's something worth discussing about.
For a fun (philosophical) remark, if the robots are to become our overlords, it may be a bad idea trying to regulate them! Google "Roko's basilisk" for more details
I'm glad someone is challenging this incredibly glib argument in support of taking away freedoms and smothering innovation under regulation, because of poorly-defined fears that experts mostly discount.
Sam's proposal for regulation depends on a prediction too. It is no less a prediction than those who disagree with him. Not every technology we can dream up is destined to come true. Is strong, hostile AI the same kind of tech as warp drives and teleporters and flying cars? Who can say? So to me this post argues against Sam's goal of regulations, because why should we take away freedoms for the sake of such a dubious prediction?
Unlike your warp drive or teleporter examples, we're pretty sure human-level AI is possible because human-level natural intelligence exists. The brain isn't magic. Eventually, people will figure out the algorithms running on it, then improve them. After that, there's nothing to stop the algorithms from improving themselves. And they can be greatly improved. Current brains are nowhere near the pinnacle of possible intelligences.
> Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization—a niche we filled because we got there first, not because we are in any sense optimally adapted to it.
— Nick Bostrom. Superintelligence: Paths, Dangers, Strategies[1]
It seems to me merely figuring out the algorithms isn't enough; you have to have a similar scale of processing power, and the brain has some 100 trillion connections, which cannot easily be replicated with silicon chips. So for all intents and purposes, the brain might as well be magic.
I suppose someday we'll be able to grow brain tissue and use it in a computing environment. Maybe that will allow us to approach true AI.
Thank you for your response! I agree that unlike a warp drive we can at least say that intelligence exists. But I think that is begging the question. For as long as we have records, people have experienced the mind as something sui generis compared to the physical. Maybe it's not magic, but it's far from understood. I think it remains to be seen if it is replicatable by physical processes.
"only way to build computers that can interpret scenes like we do is to allow them to get exposed to all the years of (structured, temporally coherent) experience we have"
This may appear daunting until you realize robots can share memories. Five robots running around for a year is equivalent to one robot running around for five years. Does not Google have 25 cars driving around experiencing the world right now?
I also see skeptics running to computer vision as an example of how far we are from human level AI. Is that just the hardest problem to solve? Is it the most useful problem to solve?
That seems reasonable, until you factor in that language and communication are itself part of intelligent life. Sharing knowledge and cooperation are fundamental to learning and intelligence, keeping in mind that these learned strategies are fundamentally asymmetric (and thus cannot be shared by simple copying).
We need to appreciate that intelligence is not an individual trait, but part of a shared strategy, utilizing diversity to be able to react quickly to changing demands.
For example, lets say we have a group of people with a shared task of moving a (large) set of boxes from one source to one destination. When performing this task initially, different strategies are tried and a winning strategy is chosen, without one person coordinating the group and without each individual having total knowledge of the strategy. However, when a similar task is presented, the group will quickly perform the winning strategy again. Who possesses the intelligence? Would we gain anything when all the knowledge would be shared? (Given limited time and space, the answer is no for most strategies)
You can transfer large quantities of data to an intelligent computer system nearly instantaneous. It seems plausible that this data could encapsulate said years of experience. What is missing is that ability to create a computer than can process said data and create consciousness with it.
Sure in the first truly AI-capable systems it will most likely be easier training them over the years in human-time, but it seems to me very unlikely to be needed when AI becomes established and at that point you should be able to create X copies of intelligence(s) at will.
And for all this talk about AI terminator doomsdays, these seems to be much less talk about what can be accomplished with it.
Let's say you create AI system, it lives in a air-gapped system, the system is carefully crafted to establish reality for the AI(s) that exists solely in the virtual world. Then you create a scientist AI, mathematician AI, engineer AI, etc. Then you have a hard problem you want to solve, great, spin up 1000 scientist AI's, 10000 engineer AI's, X project coordinator AI's, etc. Let's just say they have roughly the same capabilities as their human counterparts and work at similar speed, but do not sleep, grow tired, form unions, nor do anything other than work on the task assigned. Create a system (API?) that allows them to somehow interact with our physical reality but without understanding it whatsoever to allow them to do experiments and test results. How long would it take for such a system to recreate all of google's infrastructure, or develop the next space shuttle, or cure cancer?
I think it's important to note the a true AI(however you define it) does not have to be self aware. It doesn't even have to be aware of our physical reality. Once we reach the point where we understand consciousness well enough to recreate it, it seems likely that will be able to tune it however which way we'd like: remove self-consciousness making it act on more what we would consider instinct, configure it's reward pathways in whatever way the directs the agent to whatever task the AI designer wishes and yes, even improve it. It will be very interesting when the system spins on not 1 average human level engineer but something like a Einstein-Newton hybrid that works several orders of magnitude faster than human time. I would guess the danger there would be less from the AI(as you could isolate it from our physical reality by isolating in a virtual world) and more the extremely advanced knowledge/technology gained from such a system.
yep beam transportation a'la star trek is just round the corner. The fools that say it will take a thousands years are just as misguided as einstein was on nuclear energy.
We need to urgently enact regulation for beaming, otherwise bad androids might start beaming themselves into my bedroom.
> our machine is like a gazillion times more powerful than the quote
Really? Around 2000, storage capacity of 10^9 sounds pretty reasonable. Given that this is a prediction that's trying to hit an exponential development on a half-century time scale, I find it pretty impressive.
Yeah, I fairly-arbitrarily picked an advertisement from the January 2000 issue of PC Magazine, and it looks like $750 would buy you a machine with 10^9 bits of RAM and 10^11 bits of hard disk space, so depending on how you define "storage capacity" it's uncannily accurate.
I wasn't thinking of personal computer when I wrote the comment. I was thinking about more of a state-of-the-art super computer type (in 2000 that would have been ASCI RED), which would have probably be the same magnitude in term of physical size with "computer" in Turing time! Prediction should be judged based on "the best it could be". Otherwise, it seems like an arbitrary pick (why personal computer and not tablet?), doesn't it? Anyway, since it's an exponential development, the line between "supercomputer" to "big server" to "personal computer" is only a few years off. So yes, it was impressive.
But that wasn't the point I was trying to make. I meant that even with access to computing power a gazillion more powerful than mentioning in the quote (again, not personal computer), we still couldn't pass the Turing test. The prediction was off on both the necessary computing resources, as well as the difficulty of the test itself.
> For a fun (philosophical) remark, if the robots are to become our overlords, it may be a bad idea trying to regulate them! Google "Roko's basilisk" for more details
Eh... not everyone should look up Roko's basilisk. If you're the sort that takes thought experiments really seriously sometimes, like if you hyperventilate just by imagining certain barely-possible possible worlds, then you probably shouldn't Google it.
That probably isn't going to prevent anyone from actually doing it I guess, but just know that there are a particular set of neuroses that would indicate you should not be exposed to the idea. You've been warned :-)
Though if instead of saying that X will never be invented, they had said that if X is invented then its inventors won't make a substantial sum of money from their invention, then they would have been right almost every time.
Bozo the Clown was a huge innovation: a franchised, national brand of kids entertainment, made possible by TV. The creators laughed all the way to the bank.
I will appreciate bold predictions about the future that may turn out to be wrong over today's writings where it's all about super safe post hoc analysis after the fact. Things like "Why X succeeded!" or "X failed because of these 5 reasons" don't impressive me one bit. I wish more writers wrote about bold predictions and explain why you predict that way.
Like every other technology, AI has risks and benefits. The main issue here, is that we are letting fear dominate the discussion, and that fear is not based on any supporting facts or evidence.
Meanwhile, we are completely missing the discussion we need to have about the realistic, short and medium-term dangers associated with the development of AI. Intelligent automation is about to disrupt economic production and existing power balances, much like software has done not too long ago, with significant social consequences. This is what we need to be talking about, and preparing for. We need to plan for a smooth transition to a post-AI world.
Regulating AI for fear that it may take over makes about as much sense as outlawing space travel for fear of aliens. Can we start having a sane discussion about AI now?
I'm not sure why it hasn't come up, but my biggest fear is that a powerful nation will use AI for their own advantage, mostly in the form of propaganda and mass manipulation through Internet. That's at least my fear, and I think that "AI taking over" is complete bullshit. Although, if let loose, AI could even accidentally manipulate humanity into any non-positive state. We know that being easily manipulable is probably the biggest vulnerability of humans (and there's lots of history to prove that).
What comes to regulations, I think it should be required by law that AI identifies themselves as AI on the Internet discussion boards and social networks. I'm not sure how it's possible to enforce this requirement (without needing proof from everyone that you're human). Captchas are already too difficult (only bots get in these days). Popular captchas (such as Google's ReCaptcha) are also centralized, which is dangerous because it gives the captcha-owner a controlling position.
> my biggest fear is that a powerful nation will use AI for their own advantage, mostly in the form of propaganda and mass manipulation through Internet.
Chances are it is already happening. More advanced AI will give governments and corporations the ability to do the same in a much more effective way, and on a larger scale. Large-scale intelligent data mining will make it possible to use people's data to build actionable models of what they think, what they will do next, and how to affect what they think and do. Better than humans could.
It doesn't even need to occur through sockpuppets, so the anti-sockpuppet regulation you propose would be not only highly intrusive but also ineffective. Here's an example: Facebook can manipulate your emotional state by selecting what goes into your newsfeed [1].
Some military research centers may have developed something that is far more advanced than whats available in public. (Wouldn't be surprising, considering their ridiculous budget.)
Perhaps this talk about dangerous AI in the past year is to acclimate the public to thinking about this issue before the reveal.
Secret government agency with hyper-advanced technology (AI, fusion, gravity control, etc.) keeps scientific advances from the general public and gradually trickles them out once they feel they can control the response properly.
I found the bitcoin intruding on the others, it is pretty impressive how this "digital money" has grown. I remember a time where it was 20$ a bitcoin and no one would have believed it if I had tell them: in 6 months a bitcoin will be worth more than 1000$.
I think it felt really out of place because many "otherwise smart people" have been shouting it'll reach both zero and the sky soon and its future is still really unclear.
"Machine superintelligence may or may not be controllable. If we do nothing to regulate it, or to prevent horrible outcomes, we will with X > [too big] probability find ourselves doomed.
We need to find a way to reduce X. I propose regulation is at least not likely to be counter-productive, and may be strictly incrementally useful."
Sure, predictions are hard. We often get them wrong on the upside (over-hype) and the downside. As a result, "AI isn't going to happen" really is not a good argument against discussing the potential risks of it. (... and I say that as someone who is an "AI is around the corner" skeptic.)
Yet that doesn't change the basic risk calculus. In his previous post, Sam advocated imposing draconian licensing and observation requirements on what in practice would be the majority of non-trivial CS research. He advocated this on the basis of the potential risk that as-yet-to-be-developed hypothetical AIs might pose to human beings.
In addition to what I wrote there, I think that the risk of dramatically slowing progress in CS/AI also has to be taken into account. There is risk of doing and there is the risk of not doing.
The problem is that we currently face a number of existential risks -- like catastrophic economic collapse due to fossil fuel depletion -- where the majority of the risk is in the "risk of not doing" category. We know with total certainty that if we continue business as usual with no change, our civilization will collapse. It's simple physics and high school math -- exponential growth in consumption of a finite resource without any substitution or path to replacement can only end in one way.
Smart computers might help us crack tough problems like fusion, safe and scalable fission, better batteries to make renewable energy more practical, etc. That in turn might help us avoid an absolutely real, tangible, non-hypothetical, definite existential risk. I see no reason to hamstring that kind of progress to defend against extremely hypothetical low-probability risks.
That's why I consider Sam's suggestions to regulate CS research more dangerous than any risk posed by speculative AI scenarios.
I am not opposed to all regulation, but I am opposed to regulations based on extremely hypothetical hand-wavey risks. I'm also opposed to regulations that are virtually impossible to define accurately or enforce fairly. Regulations should be clear, objective, rationally justified by tangible problems or risks, and minimal. We should have regulations around, say, the use of nuclear materials, but that's because we know for an absolute fact that it is dangerous. We should have financial regulations because we know financial fraud has happened and will continue to happen without them. ... etc. But I positively cringe at the imposition of ill-defined broad regulations based on fear-mongering and "precautionary principle" thinking -- a.k.a. institutionalized paranoia and cowardice. Such regulations can do nothing other than halt progress in the name of vague paranoia.
Make no mistake: Sam's proposal in his previous post would halt all non-trivial CS research, or at least would slow it to such a crawl that it would effectively stop. It would also cause a mass exodus from the field, since nobody wants to operate under that kind of nonsense. Given that CS is the primary driver now of progress in other fields, that would also likely halt major progress in energy, materials, propulsion, transportation, etc.
If you read my blog post above, I take this in almost a conspiracy direction and speculate that this is some sort of political power play to lock down the field. The reason for this is that I find it hard to believe that someone of Sam's intellect and education would not realize the implications of what he's suggesting.
I don't want to go all Clinton here, but, please, lets first define "prediction". Here are some predictions that have general consensus-
1. You will die someday ( so will I )
2. The Bay Area will experience an earthquake in next decade
3. A few islands will go under due to sea level rise.
I wouldn't like to call the above predictions - they are too sun-will-rise-in-the-east obvious. Its like looking at the dumbarton bridge & predicting - someday that bridge will fall. That's a biblical prediction - all standing things must fall, bridge is a standing thing, ergo, given enough wear and tear, it too will crumble and fall.The oldest standing bridge in the world is like 2800 years old & is in a much more geologically stable place than the Bay area, so what chance does dumbarton have ?
Now here are what I call predictions -
1. qqq $200 by 2018
2. esn replaces fizzbuzz in 2019 :)
3. cnn,rnn,esn become middle school curriculum in 2020
I mean, here you have a reasonable level of confusion. Yet, if you plot the probability over time for each of those predictions, the slope is definitely positive. qqq has doubled in the past 3 years, give it another 3 years & it'll probably double again. Given this pervasive SMI fetish, it's only logical that startups replace their fizzbuzz with "in the next 20 minutes, code up an echo state network in haskell". And if sama's actually right, cnns & rnns are going to get so commonplace society is going to want middle schoolers to ace their exams with questions on "ten key differences between the recurrent neural net & the convolution neural net" instead of the pedestrian garbage we teach them now - "on a z3, if 1+2=3 and 1+1=2, how much is 3+3 ?" So the poor kids instead of sweating bullets & laboring through convoluted reasoning like "since 1 + 2 is 3, so 2 +1 is 3 per abelian, and 1 + 1 is 2, that means 3 + 1 per cayley unique column entries must be 1, which implies 3 is the identity, so 3 + 3 must be 3 as well. Ergo 3+3=3. Voila!" can actually make useful technological predictions about which esn based startup will cross a trillion dollar market cap by the time the kid hits puberty.
There are about 7 billion living testimonies that provide the non-guarantee of #1. Statistically, not everyone has died. Religiously, would Jesus' second coming be to an empty earth?
The Navy revealed the embryo of an electronic computer today that it expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence. [...] Later Perceptrons will be able to recognize people and call out their names and instantly translate speech in one language to speech and writing in another language.
- The New York Times in 1958 after a press conference with Rosenblatt. ("New Navy Device Learns By Doing; Psychologist Shows Embryo of Computer Designed to Read and Grow Wiser")
We now have walking, talking, object recognizing, writing, self-replicating, face-detecting, text-to-speech converting, and translating computers. All at a scale and accuracy surpassing us mere mortals. We do not know enough about "being conscious of our existence" to measure this in other animals and digital life forms. Perhaps "humans predicting future predictive capability of machines" is fundamentally flawed. Perhaps the above article drew an unnecessary amount of ire and criticism. Probably a fuzzy combination of the two.
> A self-replicating machine is a construct that is capable of reproducing itself autonomously using raw materials found in the environment, thus exhibiting self-replication in a way analogous to that found in nature.
“In the future, robots must be able to solve tasks in deep mines on distant planets, in radioactive disaster areas, in hazardous landslip areas and on the sea bed beneath the Antarctic. These environments are so extreme that no human being can cope. Everything needs to be automatically controlled. Imagine that the robot is entering the wreckage of a nuclear power plant. It finds a staircase that no-one has thought of. The robot takes a picture. The picture is analysed. The arms of one of the robots is fitted with a printer. This produces a new robot, or a new part for the existing robot, which enables it to negotiate the stairs.”
-2014 Kyrre Glette "Using 3D printers to print out self-learning robots"
Exactly my thoughts. A hint of condescension and lack of self-awareness - the camp Sam Altman seems to be a part of ("regulate AI, it's an existential threat to humanity!") is just as much a prediction of the future as anything else. Yet, somehow, he seems to be subtly implying that he's "more" correct.
Marc Andreessen has been relatively level-headed about the topic of AI recently on Twitter, and it would be nice to see other industry figureheads be less emotionally involved and more scientifically rigorous in their assessment of the industry. The debate is devolving into an ego battle (especially with a post like this!), and it's rather unfortunate.
Edit: Additionally, Altman appears to be primarily attacking a strawman with this article. "Superhuman" intelligence already exists. The emergent intelligence (via technological amplification) of society is, by definition, super-human. What's less realistic is anticipating a human-like artificial intelligence that would, in any way, represent an existential threat to the human race. There are many, many problems with the latter argument. (From a technological, philosophical, economic, and evolutionary perspective.)
129 comments
[ 4.2 ms ] story [ 197 ms ] threadIf the world should blow itself up, the last audible voice would be that of an expert saying it can't be done. - Peter Ustinov
Popular Mechanics did a retrospective which is an absolute circus of hits and misses: http://www.popularmechanics.com/flight/g462/future-that-neve...
that one seems particularly relevant
The Einstein/Wright Brothers quotes really hit this home for me.
Maybe the quotes are meaningless to you because you already agree. But they might persuade others that would otherwise assume discussing AI impacts on humanity is a waste of time.
Szilard hadn't yet proposed a theory of nuclear chain reactions, but according to some cites of the quote Einstein didn't say it until 1934 - which was after Szilard.
I don't have a problem with the possibility that suprahuman intelligence may be possible. I do have a problem with the fact that currently we have no idea what the concept may even mean - and right now, more immediate cybersecurity issues are being neglected.
Computers are already better than humans at many activities. From playing chess to landing planes to learning how to play a video game - a computer with the right software is much better at these than an average human, and is often at least good as the best humans.
Take that to the black corner, and worms and botnets are already a serious problem.
We don't need to wait for the Internet to become sentient and start talking to us in a deep echoey robot voice to worry about cyberthreats.
There's more than enough to deal with already. And if you're going to try to regulate and contain a future AI, making current systems as secure as possible seems like a realistic place to start.
Not in a chain reaction. When Szilárd described the concept of a chain reaction to Einstein, Einstein was shocked. He said "I never thought of that!"
Until then, nuclear physics was purely an academic enterprise. There were few applications for radioactive materials. Radioactive decay just happened at its own slow pace, and not much could be done with it. X-rays could be used to pump the process, but less energy came out than what was put in. Suddenly the nuclear physicists realized they had a tiger by the tail. This was going to change the world, not necessarily for the better.
But, poking around a bit, I came to the same understanding you have. Here's some more of the time line:
The quote in the OP (which I can't find online; the Einstein archives at Caltech are, alas, not indexed) about Einstein's skepticism about nuclear energy is dated 1932. The first demonstrations of nuclear fission were years later, in late 1938 and into 1939. And as you said, Einstein is reported to have said, "I had not thought of that." -- regarding the chain reaction.
The fabled Einstein-Szilard letter to Franklin Roosevelt, warning about the Nazis getting the atomic bomb, was written in August 1939 (http://en.wikipedia.org/wiki/Einstein–Szilárd_letter), and then relayed to Roosevelt in October after the flurry of activity due to the Nazis invading Poland had died out.
My opinions currently stop at this is an important topic and anyone who is interested should be exploring it via their chosen medium.
Also, Einstein's quote originates from 1934, not 1932.
I don't know how "otherwise smart" I am, but I wonder how we would be able to tell that a machine intelligence was "superhuman" as opposed to "buggy".
For example, suppose we build a super-AI and ask it, "Is Shinichi Mochizuki's proof of the ABC conjecture correct" [1]. What would we do if it said "yes"?
(Of course, if "superhuman" just means "able to do things humans already know how to do and verify, but lots faster", then we're already there).
[1] http://www.newscientist.com/article/dn26753-mathematicians-a...
Yeah, that would work :)
Maybe the question I should have asked, is:
What if we ask a super-AI for a proof of the ABC conjecture, and the result is something too complicated for humans to verify?
My point, if I have one, is that when I read about "superhuman machine intelligence", sometime people seem to mean "capable of knowledge that humans couldn't figure out on their own but that humans can understand once they see it"; and sometimes they seem to mean "capable of knowledge that is beyond human capacity to even verify".
I think development machine intelligence of the first kind is extremely likely, but I'm more skeptical about the second kind.
The Strong AI hypothesis assumes a mechanistic universe, if not necessarily a materialistic one, and I think that condition is false.
Some patents are classified each year. A clerk who has seen many classified patents would have a unique opinion on "blue oceans" for investment opportunities, especially if they knew how to prevent new patents from being classified, by avoiding certain areas of research, http://fas.org/sgp/othergov/invention/ . In TRIZ terminology, they would have different psychological inertia.
Is there a yearly list of declassified patents over the last few decades? This would be similar to lists of expired patents or books which enter the public domain in some countries.
It's a bit ironic that Sam missed this one quote that would have be very relevant to the post, both in term of the conclusion and the topic at hand (it's hard to predict stuffs) ;). It's interesting to note that the prediction was wrong on both counts: our machine is like a gazillion times more powerful than the quote, and we're doing nowhere near the capability of the prediction.
From the last HN comments on the topic, while it's true that there are certain people who believe that strong AI is impossible, the stance I've got from AI/robotics researchers were that it's just silly to talk about general AI at this point: it's so far away. This post from Karpathy would be one, for example: http://karpathy.github.io/2012/10/22/state-of-computer-visio... .
Because of that, any talk and concern on the topic is a bit silly, especially about regulation of the subject. I mean, can you imagine people in the 1600 trying to figure out how to regulate the air traffic that we have right now? Or should we talk about space travel regulation now? Because seriously, chance are that will happen before we have singularity/ strong AI.
If we're really concern about the danger of strong AI, then I'm more in favored dealing with it by the way of Eliezer Yudkowsky and the Singularity Institute (by making sure that whatever we're researching toward is "Friendly"). Even though I'm also thinking that they're too optimistic. I'm not against immortality in my lifetime though.
Now, a talk about more sophisticated automated/ autonomous system that do funny things (in a good or bad way), or their risk, that's something worth discussing about.
For a fun (philosophical) remark, if the robots are to become our overlords, it may be a bad idea trying to regulate them! Google "Roko's basilisk" for more details
Sam's proposal for regulation depends on a prediction too. It is no less a prediction than those who disagree with him. Not every technology we can dream up is destined to come true. Is strong, hostile AI the same kind of tech as warp drives and teleporters and flying cars? Who can say? So to me this post argues against Sam's goal of regulations, because why should we take away freedoms for the sake of such a dubious prediction?
> Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization—a niche we filled because we got there first, not because we are in any sense optimally adapted to it.
— Nick Bostrom. Superintelligence: Paths, Dangers, Strategies[1]
1. http://www.amazon.com/Superintelligence-Dangers-Strategies-N...
I suppose someday we'll be able to grow brain tissue and use it in a computing environment. Maybe that will allow us to approach true AI.
http://www.fhi.ox.ac.uk/brain-emulation-roadmap-report.pdf
"only way to build computers that can interpret scenes like we do is to allow them to get exposed to all the years of (structured, temporally coherent) experience we have"
This may appear daunting until you realize robots can share memories. Five robots running around for a year is equivalent to one robot running around for five years. Does not Google have 25 cars driving around experiencing the world right now?
I also see skeptics running to computer vision as an example of how far we are from human level AI. Is that just the hardest problem to solve? Is it the most useful problem to solve?
We need to appreciate that intelligence is not an individual trait, but part of a shared strategy, utilizing diversity to be able to react quickly to changing demands.
For example, lets say we have a group of people with a shared task of moving a (large) set of boxes from one source to one destination. When performing this task initially, different strategies are tried and a winning strategy is chosen, without one person coordinating the group and without each individual having total knowledge of the strategy. However, when a similar task is presented, the group will quickly perform the winning strategy again. Who possesses the intelligence? Would we gain anything when all the knowledge would be shared? (Given limited time and space, the answer is no for most strategies)
You can transfer large quantities of data to an intelligent computer system nearly instantaneous. It seems plausible that this data could encapsulate said years of experience. What is missing is that ability to create a computer than can process said data and create consciousness with it.
Sure in the first truly AI-capable systems it will most likely be easier training them over the years in human-time, but it seems to me very unlikely to be needed when AI becomes established and at that point you should be able to create X copies of intelligence(s) at will.
And for all this talk about AI terminator doomsdays, these seems to be much less talk about what can be accomplished with it.
Let's say you create AI system, it lives in a air-gapped system, the system is carefully crafted to establish reality for the AI(s) that exists solely in the virtual world. Then you create a scientist AI, mathematician AI, engineer AI, etc. Then you have a hard problem you want to solve, great, spin up 1000 scientist AI's, 10000 engineer AI's, X project coordinator AI's, etc. Let's just say they have roughly the same capabilities as their human counterparts and work at similar speed, but do not sleep, grow tired, form unions, nor do anything other than work on the task assigned. Create a system (API?) that allows them to somehow interact with our physical reality but without understanding it whatsoever to allow them to do experiments and test results. How long would it take for such a system to recreate all of google's infrastructure, or develop the next space shuttle, or cure cancer?
I think it's important to note the a true AI(however you define it) does not have to be self aware. It doesn't even have to be aware of our physical reality. Once we reach the point where we understand consciousness well enough to recreate it, it seems likely that will be able to tune it however which way we'd like: remove self-consciousness making it act on more what we would consider instinct, configure it's reward pathways in whatever way the directs the agent to whatever task the AI designer wishes and yes, even improve it. It will be very interesting when the system spins on not 1 average human level engineer but something like a Einstein-Newton hybrid that works several orders of magnitude faster than human time. I would guess the danger there would be less from the AI(as you could isolate it from our physical reality by isolating in a virtual world) and more the extremely advanced knowledge/technology gained from such a system.
We need to urgently enact regulation for beaming, otherwise bad androids might start beaming themselves into my bedroom.
Really? Around 2000, storage capacity of 10^9 sounds pretty reasonable. Given that this is a prediction that's trying to hit an exponential development on a half-century time scale, I find it pretty impressive.
https://books.google.com/books?id=UEgKC-lu4_IC&lpg=PA32&dq=p...
But that wasn't the point I was trying to make. I meant that even with access to computing power a gazillion more powerful than mentioning in the quote (again, not personal computer), we still couldn't pass the Turing test. The prediction was off on both the necessary computing resources, as well as the difficulty of the test itself.
Eh... not everyone should look up Roko's basilisk. If you're the sort that takes thought experiments really seriously sometimes, like if you hyperventilate just by imagining certain barely-possible possible worlds, then you probably shouldn't Google it.
That probably isn't going to prevent anyone from actually doing it I guess, but just know that there are a particular set of neuroses that would indicate you should not be exposed to the idea. You've been warned :-)
https://en.wikipedia.org/wiki/Bozo_the_Clown
[1] http://en.wikipedia.org/wiki/Panama_Canal
Ahem: http://en.wikipedia.org/wiki/Drake_Passage
Remember Magellan?
Meanwhile, we are completely missing the discussion we need to have about the realistic, short and medium-term dangers associated with the development of AI. Intelligent automation is about to disrupt economic production and existing power balances, much like software has done not too long ago, with significant social consequences. This is what we need to be talking about, and preparing for. We need to plan for a smooth transition to a post-AI world.
Regulating AI for fear that it may take over makes about as much sense as outlawing space travel for fear of aliens. Can we start having a sane discussion about AI now?
What comes to regulations, I think it should be required by law that AI identifies themselves as AI on the Internet discussion boards and social networks. I'm not sure how it's possible to enforce this requirement (without needing proof from everyone that you're human). Captchas are already too difficult (only bots get in these days). Popular captchas (such as Google's ReCaptcha) are also centralized, which is dangerous because it gives the captcha-owner a controlling position.
Chances are it is already happening. More advanced AI will give governments and corporations the ability to do the same in a much more effective way, and on a larger scale. Large-scale intelligent data mining will make it possible to use people's data to build actionable models of what they think, what they will do next, and how to affect what they think and do. Better than humans could.
It doesn't even need to occur through sockpuppets, so the anti-sockpuppet regulation you propose would be not only highly intrusive but also ineffective. Here's an example: Facebook can manipulate your emotional state by selecting what goes into your newsfeed [1].
[1] http://www.theguardian.com/technology/2014/jun/29/facebook-u...
Perhaps this talk about dangerous AI in the past year is to acclimate the public to thinking about this issue before the reveal.
[1] http://www.amazon.com/Influx-Daniel-Suarez/dp/0525953183
-- TL;DR Synopsis --
Secret government agency with hyper-advanced technology (AI, fusion, gravity control, etc.) keeps scientific advances from the general public and gradually trickles them out once they feel they can control the response properly.
It would be nice if there were either (a) two newlines between each quote or (b) only one newline between the quote and attribution.
- Delusional VC
A genuinely sympathetic paraphrase might be:
"Machine superintelligence may or may not be controllable. If we do nothing to regulate it, or to prevent horrible outcomes, we will with X > [too big] probability find ourselves doomed.
We need to find a way to reduce X. I propose regulation is at least not likely to be counter-productive, and may be strictly incrementally useful."
Yet that doesn't change the basic risk calculus. In his previous post, Sam advocated imposing draconian licensing and observation requirements on what in practice would be the majority of non-trivial CS research. He advocated this on the basis of the potential risk that as-yet-to-be-developed hypothetical AIs might pose to human beings.
I did a short post on it here: http://adamierymenko.com/did-sam-altman-of-y-combinator-just...
In addition to what I wrote there, I think that the risk of dramatically slowing progress in CS/AI also has to be taken into account. There is risk of doing and there is the risk of not doing.
The problem is that we currently face a number of existential risks -- like catastrophic economic collapse due to fossil fuel depletion -- where the majority of the risk is in the "risk of not doing" category. We know with total certainty that if we continue business as usual with no change, our civilization will collapse. It's simple physics and high school math -- exponential growth in consumption of a finite resource without any substitution or path to replacement can only end in one way.
Smart computers might help us crack tough problems like fusion, safe and scalable fission, better batteries to make renewable energy more practical, etc. That in turn might help us avoid an absolutely real, tangible, non-hypothetical, definite existential risk. I see no reason to hamstring that kind of progress to defend against extremely hypothetical low-probability risks.
That's why I consider Sam's suggestions to regulate CS research more dangerous than any risk posed by speculative AI scenarios.
I am not opposed to all regulation, but I am opposed to regulations based on extremely hypothetical hand-wavey risks. I'm also opposed to regulations that are virtually impossible to define accurately or enforce fairly. Regulations should be clear, objective, rationally justified by tangible problems or risks, and minimal. We should have regulations around, say, the use of nuclear materials, but that's because we know for an absolute fact that it is dangerous. We should have financial regulations because we know financial fraud has happened and will continue to happen without them. ... etc. But I positively cringe at the imposition of ill-defined broad regulations based on fear-mongering and "precautionary principle" thinking -- a.k.a. institutionalized paranoia and cowardice. Such regulations can do nothing other than halt progress in the name of vague paranoia.
Make no mistake: Sam's proposal in his previous post would halt all non-trivial CS research, or at least would slow it to such a crawl that it would effectively stop. It would also cause a mass exodus from the field, since nobody wants to operate under that kind of nonsense. Given that CS is the primary driver now of progress in other fields, that would also likely halt major progress in energy, materials, propulsion, transportation, etc.
If you read my blog post above, I take this in almost a conspiracy direction and speculate that this is some sort of political power play to lock down the field. The reason for this is that I find it hard to believe that someone of Sam's intellect and education would not realize the implications of what he's suggesting.
More interesting failed technology predictions.
I predict Bitcoin to $10k sometime in the next 5 years. Prima facie of course!
1. You will die someday ( so will I )
2. The Bay Area will experience an earthquake in next decade
3. A few islands will go under due to sea level rise.
I wouldn't like to call the above predictions - they are too sun-will-rise-in-the-east obvious. Its like looking at the dumbarton bridge & predicting - someday that bridge will fall. That's a biblical prediction - all standing things must fall, bridge is a standing thing, ergo, given enough wear and tear, it too will crumble and fall.The oldest standing bridge in the world is like 2800 years old & is in a much more geologically stable place than the Bay area, so what chance does dumbarton have ?
Now here are what I call predictions -
1. qqq $200 by 2018
2. esn replaces fizzbuzz in 2019 :)
3. cnn,rnn,esn become middle school curriculum in 2020
I mean, here you have a reasonable level of confusion. Yet, if you plot the probability over time for each of those predictions, the slope is definitely positive. qqq has doubled in the past 3 years, give it another 3 years & it'll probably double again. Given this pervasive SMI fetish, it's only logical that startups replace their fizzbuzz with "in the next 20 minutes, code up an echo state network in haskell". And if sama's actually right, cnns & rnns are going to get so commonplace society is going to want middle schoolers to ace their exams with questions on "ten key differences between the recurrent neural net & the convolution neural net" instead of the pedestrian garbage we teach them now - "on a z3, if 1+2=3 and 1+1=2, how much is 3+3 ?" So the poor kids instead of sweating bullets & laboring through convoluted reasoning like "since 1 + 2 is 3, so 2 +1 is 3 per abelian, and 1 + 1 is 2, that means 3 + 1 per cayley unique column entries must be 1, which implies 3 is the identity, so 3 + 3 must be 3 as well. Ergo 3+3=3. Voila!" can actually make useful technological predictions about which esn based startup will cross a trillion dollar market cap by the time the kid hits puberty.
Location, time, and magnitude on all earthquake predictions, please.
- The New York Times in 1958 after a press conference with Rosenblatt. ("New Navy Device Learns By Doing; Psychologist Shows Embryo of Computer Designed to Read and Grow Wiser")
We now have walking, talking, object recognizing, writing, self-replicating, face-detecting, text-to-speech converting, and translating computers. All at a scale and accuracy surpassing us mere mortals. We do not know enough about "being conscious of our existence" to measure this in other animals and digital life forms. Perhaps "humans predicting future predictive capability of machines" is fundamentally flawed. Perhaps the above article drew an unnecessary amount of ire and criticism. Probably a fuzzy combination of the two.
> A self-replicating machine is a construct that is capable of reproducing itself autonomously using raw materials found in the environment, thus exhibiting self-replication in a way analogous to that found in nature.
http://www.apollon.uio.no/video/a_robot_e.mp4
And the press release:
“In the future, robots must be able to solve tasks in deep mines on distant planets, in radioactive disaster areas, in hazardous landslip areas and on the sea bed beneath the Antarctic. These environments are so extreme that no human being can cope. Everything needs to be automatically controlled. Imagine that the robot is entering the wreckage of a nuclear power plant. It finds a staircase that no-one has thought of. The robot takes a picture. The picture is analysed. The arms of one of the robots is fitted with a printer. This produces a new robot, or a new part for the existing robot, which enables it to negotiate the stairs.”
-2014 Kyrre Glette "Using 3D printers to print out self-learning robots"
Marc Andreessen has been relatively level-headed about the topic of AI recently on Twitter, and it would be nice to see other industry figureheads be less emotionally involved and more scientifically rigorous in their assessment of the industry. The debate is devolving into an ego battle (especially with a post like this!), and it's rather unfortunate.
Edit: Additionally, Altman appears to be primarily attacking a strawman with this article. "Superhuman" intelligence already exists. The emergent intelligence (via technological amplification) of society is, by definition, super-human. What's less realistic is anticipating a human-like artificial intelligence that would, in any way, represent an existential threat to the human race. There are many, many problems with the latter argument. (From a technological, philosophical, economic, and evolutionary perspective.)