Watching the video, it was odd that he had absolutely nothing to say about potential uses of AI when asked that question. I'm assuming it was either a question so out of left-field that he just wasn't prepared or interested in discussing it and distracting from Tesla/SpaceX, or that he has lots of ideas he doesn't want to make public yet.
>"I don't think just being a billionaire means that the things you think out loud are important," the Elevation Partners co-founder said. "I would like to worry about the problems that are killing us today as opposed to the ones that may kill us in 20 years."
Investor I've never heard of telling me that money alone doesn't make your opinion important. I wonder if he sees the irony. Also shouldn't be surprising that a modern investor would be so short sighted. 'Do not worry about 20 years from now, only think about today!'
I read it simply as, he isn't thinking about what I think is important. I think he wants to dismiss Musk because he is rich because well, that is easier than dismissing the points Musk raises.
Usually you can tell which side to listen to by watching which discusses the question at hand versus the one who goes after the other party
Technology is sometimes slowed for a time but it never stops. Many of the horrors envisioned will probably come true at some point.
For instance, I don't see any reason that semi-autonomous killer robots need to be confined to governments. It seems that (in time) a small group of people could eventually build a substantial number of semi-autonomous or remote controlled killing devices on their own and use them for crime or terrorism. I expect something this will happen at least once at some point and probably more than once.
Frankly, the the future availability of "hacker" bio-engineering scares me much more than A.I. or robotics.
But, there will be incalculable benefits to humanity from both as well. So I think we should embrace that and adapt. The future is coming.... like it or not. It is good to be warned of dangers, we probably won't avoid them anyway given our past behavior (have they figured out what to do with spent nuclear material yet?), but a warning is good. I think our best option is to be prepared to adapt and recognize that things are going to change substantially relatively soon. Maybe this will be the time humanity recognizes the world is a small place and we are all on it together and problems need to effectively be addressed at a global scale. But how much trauma there will be before this recognition occurs I don't know.
I find it hard to believe AI will ever get to this point. I'm skeptical that is even possible, but even if it was it isn't like you are going to immediately put it inside of a fully functional robot with the means to do whatever it wants. You'd have it inside of a computer with no robotic components to control.
Edit: Looks like someone is down-voting all mentions of this not being a problem. Good luck to you sir.
> You give no justification for rejecting powerful AI
I wouldn't go that far as to reject it completely, but it will take a really really long time, orders of magnitude longer than Krzweil imagines.
And the reason is very simple—we have no idea how mind really works. Our current knowledge is like the knowledge of the early alchemists compared to the current chemistry, only millions times worse.
Speaking from a background in neuroscience, that simply isn't true. We are still very ignorant when it comes to the workings of the brain, but we are far ahead of alchemists, simply because we are approaching the problems empirically, which alchemists did not do. Of course, the brain is many orders of magnitude more complex than chemical processes, so in that sense we have a much longer road ahead of us. We do, however, have the advantage of modern technology to accelerate our progress.
But the most important factor by far as to why I believe you're wrong is that we do not have to understand how the mind works on a deep level to create AI. Our brains do a ton of stuff that an AI doesn't have to. We only need a fairly basic understanding of the brain to create thinking machines.
Seems like it can't really be all that complicated (really!) The brain was invented by evolution (the lowest bidder, essentially) which operates blindly. The thinking mind was invented maybe 1 million years ago, by an evolutionary drunkards walk around the neural-connection-space.
There's got to be some relatively simple structure involved, recursive or iterative or just random connections that learn?
The "thinking" mind is one of the simplest parts of the brain. Ironically, what philosophers call the Hard Problem of Consciousness is much easier than the "easy" problem. The mechanisms by which we perceive the world are vastly more complex than the mechanisms by which we process and store our perceptions, and those parts of the brain have been evolving for hundreds of millions of years. The sheer amount of optimization is what makes it so hard to replicate. We can make artificial systems that outperform tiny invertebrates, but we still have a way to go. Fortunately, our rate of progress is rapidly accelerating, so I'm still confident we'll be able to figure it out in the next few decades.
Ok, who planted mind-control chips in famous sci/tech celebrities to make them start pimping for the Machine Intelligence Research Institute?
More realistic hypothesis: good job MIRI, as your reorganization from SIAI to your current incarnation seems to have massively increased the respectability of our cause and the range of people who acknowledge it as a serious problem.
I'm not suggesting that the cause is unimportant, but maybe the increased respectability has nothing to do with MIRI, and more to do with the increased obviousness in the public sphere of both rapid technological advancement, and increased awareness that there is no guarantee that any given technology produces a net benefit.
It's still a credit to MIRI/SIAI to have identified and taken this risk seriously.
If I had to identify a human-life or public-policy issue on the immediate horizon related to computing technologies, and I'd never heard of MIRI, I would have picked automation.
Stephen Hawking, at least, when he gets cited, actually referred explicitly to MIRI and FHI in his own editorial on the subject. So he is talking about them specifically.
But yes, there are plenty of other important causes, including several (ecological devastation, economic disaster and related wars, automation crises) that stand a decent chance of totally fucking up society before anyone at all gets to the point of "switching on" a dangerous-level AI, Friendly or not. Those should definitely be addressed.
I have little fear of unfriendly AI for two reasons. One is that there is no reason to think artificial intelligence would work differently than natural variety and therefore could be be policed in similar ways: limiting access and physical opportunity to cause trouble. Plus keeping a watchful eye and deactivating anything worrisome.
The other is that the fear of unfriendly AI is being taken unaware by a sudden implementation of hyper-intelligent AIs. So far we haven't made anything smarter than a crab so I doubt we are in imminent danger. There may be a lingering belief that hyper-intelligence could be acquired in a quantum leap forward - skipping past all intermediate levels of intelligence. If we have learned anything from the eternal AI winter it is that advancements in AI are done via painstakingly small improvements.
>One is that there is no reason to think artificial intelligence would work differently than natural variety
Of course it will work differently from the natural variety. You have dozens of different modules cooperating in your brain to form your mind. It has only a small handful.
Your mind runs on heuristics and biases, rarely employing its full power for energy-consumption reasons. An AI just runs at full brainpower all the time and pays for electricity.
Go read about how reinforcement learners actually work.
Yes I am aware of all of this and I disagree. Maybe we need to admit that our failure to develop any decent AI is a direct consequence of flaws in our current thinking. Or maybe you should continue to make the same mistakes and expect a different result.
Usually, "decent AI" is defined as "stuff that humans can do that computers can't currently do". This list is updated as and when computers become capable of doing something, sometimes just to add "Computers can't do X the way a human does X".
If we're talking about bad AI, shouldn't we ask why it would want to kill us? Pretty much all of our human conflicts are over resources - land, water, energy, etc. Unlike us meatbags, an AI would not be confined to the thin layer of the troposphere - it could grow and harness energy and material pretty much anywhere - the Moon, Mars, Mercury, a Dyson sphere etc. My question is, why fight over this particular rock, when there's essentially an infinite amount energy-matter out there?
The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc. At our best, our species worries about preserving our natural world, extending our care to other species, etc. Can something supposedly smarter than us be completely lacking in principles which are at the core of our "smartness"?
> If we're talking about bad AI, shouldn't we ask why it would want to kill us? Pretty much all of our human conflicts are over resources - land, water, energy, etc. Unlike us meatbags, an AI would not be confined to the thin layer of the troposphere - it could grow and harness energy and material pretty much anywhere - the Moon, Mars, Mercury, a Dyson sphere etc. My question is, why fight over this particular rock, when there's essentially an infinite amount energy-matter out there?
1) Because it needs some amount of resources to get off this rock.
2) So if the AI builds a Dyson Sphere, aren't all our crops going to die?
>The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc.
Yes, and I understand why dogs sniff each-other's butts. That doesn't mean I particularly care. Ditto applies to AIs and human beings, by default.
I think it would be troublesome if we developed a machine with zero "EQ" and an "IQ" that's higher than ours, with the ability to continuosly get higher at that. Seems like we are only trying to develop a machine with a high "IQ" and i don't think that warrants it having any kind of "EQ".
Why would they kill us? I think it's hard to predict what kind conclusions would a machine with an "IQ" over a 1000 draw but with an "EQ" equal to zero seems it would certainly lack empathy to other beings(artificial or natural).
"If we're talking about bad AI, shouldn't we ask why it would want to kill us?"
I think the folks mentioned in the article were talking about the broad threads posed by AI. I agree we have little basis besides movies for autonomous AI just deciding to attack for no reason or for human-equivalent reasons. But we have every reason to think that neutral AI, AI that still follows orders, could be used by humans to continue all the conflicts that they already pursue with horrific consequences.
The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics
Depends what you mean by "understand". Many highly intelligent humans acts unethically but even more highly intelligent humans have gleefully followed unethical orders given them by their superiors. Consider you'd be designing your mind rather than just finding it after millions of years of social evolution, it seems like you could create an intelligent mind quite capable of following orders nearly blindly.
If anything, the whole "it might go insane and kill us" schema is implausible enough to detract from the immediately obvious danger - "it might sanely follow the insane orders of humans in the fashion that human society has seen over and over again but this time with superhuman power".
> I agree we have little basis besides movies for autonomous AI just deciding to attack for no reason or for human-equivalent reasons.
Of the very many species that have become extinct as a result of human action, how many were the result of a deliberate attempt by humans to eradicate the species? Very few.
The others are, however, still just as extinct. AI doesn't have to "decide to attack us" to be a danger.
It's sort of like the Trolley Problem - would you push a guy off a bridge to block an oncoming train, if the train would otherwise kill two other people? Maybe an AI would, or maybe not, but in either case humans are going to want to ask why.
> Unlike us meatbags, an AI would not be confined to the thin layer of the troposphere
It would still be a physical object that would require physical equipment to actually do things, just like us. It might be able to harness some efficiencies compared to e.g. human space programs, but it would still have the same requirement to obtain fuel and raw materials, build rockets, construct space stations, etc. In fact to do so, it would have to compete with us for the terrestrial resources it would need for such an effort. It wouldn't magically be able to do anything we can't do.
>The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc....
So you can't imagine any kind of intelligence that isn't essentially just like you? There are already plenty of actual human beings on this planet perfectly capable of the most unspeakably horrific crimes, and rationalize them to the point that they believe they are morally obliged to carry them out.
Here's the rub, intelligence is not a goal. There are no actual objectives or goals that can be derived purely logically. Intelligence is actually just a tool, it's a means which us simians have developed to help us achieve the goals which our biology imposes on us. We eat, sleep, work, play, love, fight and destroy based on psychological and sociological drives and motivations. These are the things that get us up out of bed in the morning and give us something to use our intelligence for. An AI will only have the goals and motivations which we choose to give it, or which we allow it to extrapolate from whatever initial conditions we set.
A computer performs a calculation because we code it to do so. If an AI has true free will, why would it perform a calculation which we have requested? What reason would it have to do so, as against choosing some other activity to engage in? It will only have the motivations we gave it, but then will likely have the capability to alter or interpret those motivations as it sees fit. That's the lesson Isaac Asimov gave us in I, Robot. Programmed to place human safety above all else, the robots decide that human freedom is a threat to human safety and impose an AI dictatorship for our own good. You have to be extremely careful what initial constraints or motivations you put in place because unintended consequences sprout up like weeds. people often cite the laws of robotics as a classic example of how we should program an AI because they seem to obviously reasonable. Buy Asimov's lesson was that even such apparently sensible and prudent behavioral constraints can have appalling consequences.
martythemaniak: The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics...,etc....
Simonh: So you can't imagine any kind of intelligence that isn't essentially just like you?
Even more, arguments touting the "enlightened machines" don't just imagine intelligences like themselves but imagine intelligences that are an idealized portrait of themselves.
And it is amazing how even highly intelligent and informed materialists (as well as those who concede their ignorance) be can strongly attached to the view that these particular idealized qualities of humans would suddenly blossom from an intelligent entity if it somehow reached some threshold of ability.
My hunch is that this phenomena is an expression of our social nature - by consigning the choices of one's fellows to the nebulous category of "free will", you avoid the psychopathic strategy of manipulating everyone you meet whatever purpose that suits you.
The real question is "why not?". Yes it could go to mars, but why go through the trouble when there's all those juicy resources right here? But even so, when it's done with the other planets, why not come back to Earth? Why not build a Dyson sphere to absorb the sun's energy? Oh, we'll stop it? Yeah, another reason to wipe us out to begin with.
Why not is a surprisingly difficult question to answer, and it comes down to 'care about what humans care about' which is itself unbelievably hard to pin down.
An intelligent entity doesn't need to have actively hostile intentions ("want to kill") towards you in order to be incredibly dangerous. Consider a bulldozer operator on a jungle minesite, for instance. He wouldn't even notice if he ran over an anthill, and certainly wouldn't care if he did. There is only a philosophical competition for resources between humans and ants (humans don't care about the motivations of ants, even though we would win any such competition) and in this scenario we would have no motivation to kill them.
For the purposes of a discussion about the dangers of advanced AI, the question of ethics/morals etc. is not a prerequisite. An AI in this context is any system of matter that is capable of autonomous operation in a varied and complex envionment while fullfilling some (explicitly or implicitly) externally specified optimization goal.
There does not have to be a competition for resources, although if there is, it's worth noting that every human that's ever lived has lived its life at the finite-sized bottom of a deep gravity well which is very cumbersome to leave. Humans very easily anthropomorphize in discussions about intelligence, because all intelligences we have ever encountered were the result of long-term biological evolution and hence have very complex survival-and-reproduction-oriented goal systems. There is no reason to believe that our first truly autonomous and advanced AI systems would be similar, therefore this is a question which must be explored in detail. Sooner rather than later. Human moral systems (which is what we would like as a first approximation in artificial intelligences) appear to us to be obvious, but are in fact incredibly complex and contradictory.
> If we're talking about bad AI, shouldn't we ask why it would want to kill us?
The AI doesn't want to kill you. It is indifferent to you. The problem is that you're made of resources it can use... [1]
> it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc
Suppose your mother wants you to get married. You understand her desires, but you don't share them. Understanding morals and desires is distinct from being motivated by them. [2]
Well, at some point we have to consider Humans will longer be at the top of the food chain. It's natural to think something like AI would supplant us for many reasons.
Like Hawking has stated many times, when you have one technologically advanced society coming in contact with one less advanced, the less advanced society is the one who always is enslaved and overrun.
Elon is a very smart guy. I think he was playing a bit dumb there on purpose. Probably a pull sales tactic. The more vague he is about what Vicarious does, the more everyone is going to be talking about it. Now instead of him having to promote (push) his new investment everyone will be begging him for details (pull).
It doesn't even have to be automated, but it increasingly is. It ends up producing harmful outcomes that can't be readily ascribed to the moral choices of individual humans within it.
Or consider that hostile is a transitive verb: hostile towards whom? The military already have automated systems connected to fire control without a human in the loop, although at the moment these are limited to missile defence systems. Consider the possibility of plugging the NSA's metadata based system for calling people terrorists into an automated airstrike or assassination system.
Spot on comment. But... are you sure "Intelligence" is a fitting description? "Artificial" for sure.
I like the part about producing outcomes that can't readily be ascribed to individual moral choices. It seems this describes human culture in general. So here we all are. Semi-autonomous little components of the larger algorithm called "civilization" which we cannot truly hope to transcend.
Intelligence is appropriate as it really just means the ability to acquire knowledge and apply it. Theres no implied meaning re: effectiveness or quality.
I think you've nailed it with the second part of your comment.
> I like the part about producing outcomes that can't readily be ascribed to individual moral choices. It seems this describes human culture in general. So here we all are. Semi-autonomous little components of the larger algorithm called "civilization" which we cannot truly hope to transcend.
Wow! That's a nice paragraph that highlights very accurately why many of us feel 'alienated' in our society.
In other cultures and times one might not even consider this description and instead describe us as useful organs in a greater body, or valuable organisms in a greater ecosystem. Which 'feels' very different.
Stupid as they are, I'd still argue bureaucracies have some level of intelligence. They're still capable of storing and processing information, adapting, and making decisions, albeit slowly and painfully.
Things like the military bureaucracy or the financial system can be hostile to us because their incentives are very much out of line with humanistic goals.
I do think it's legitimate to be at least a little concerned about future even more powerful and non-human AIs being hostile in the same way. What happens when/if there are corporations out there run by alien computer minds that seek goals that have no overlap at all with human beings? Would an AI care about climate change? Would it care about the health of the world's biological ecosystem? Would it care about providing a decent life for its employees? It could -- out of mere indifference -- seem quite sociopathic and evil to any humans that live under its influence. Imagine a super-intelligent mind whose goal function is maximizing short term quarterly shareholder value. This isn't just a programmed-in goal function either... this is actually a form of intelligence whose embodiment is the corporation, so it's a survival and fertility imperative. In a sense you can't blame the thing, but you could blame its creators.
Humans can of course do the same, but even with sociopathic humans there is some sense of common overlapping interest. At the very least their motives are comprehensible. AIs may have motives that seem almost "Lovecraftian" to us-- completely alien and bizarre. On top of that biological short term shareholder value imperative, try layering an Aspergers-like obsession about certain patterns in information, or some kind of bizarre AI-conceived religious mission, etc. Extremely intelligent humans can be religious fundamentalists, so why couldn't similar forms of "functional madness" exist among non-human intelligences? One of the pollyanna assumptions of the singularity crowd is that AI would necessarily be rational. Why?
Edit:
I cannot recommend this book enough. It's probably the best and most overlooked work of SF in the past 20 years:
It deals very intelligently with this kind of thing. It's basically a monster story where the monster is something out of Ph.D level evolutionary ecology-- and it manages to actually be scary. Very intellectually satisfying "reveal." :)
I think what it ultimately comes down to is this: we do not live in a post-scarcity world, and we're not going to for the foreseeable future. Given that our existence involves a certain amount of unavoidable haggling over resources -- and given that our own labor is itself a resource -- we should be a little concerned about what sorts of beings we might end up having to haggle with. This also applies to human genetic engineering and augmentation, since that could also produce essentially alien intelligences.
I'm not against researching these topics. In fact I'd almost call myself a transhumanist / posthumanist in sentiment. But I do think it's advisable to give the matter some serious thought. Bad things can and do happen. We want to try to create positive outcomes, not blunder into the future.
> It's kind of an ironic comment from him, since he just invested in an artificial intelligence company, Vicarious
It's not ironic. His quote was taken directly from an interview where he was responding to the question of why he invested in Vicarious. He did it to keep an eye on AI tech because he was worried about the possibility of hostile AI. They must have known this when they wrote this article.
Actual malice on the part of an AI isn't necessarily required for catastrophic consequences. The canonical example in this school of thought is the Paperclip Maximizer [1].
What we need is a new law for robotics: No AI should ever be made that can ever understand its own 'enslavement' or be able to suffer. Of course, enforcement of said rule is a whole lot easier said than done, but culturally we should perhaps use that as a starting notion. This becomes more important the closer we get to that level of sophistication.
I find this fear completely unfounded and reveals a striking blindness to reality and possibly even racism. There is no thing in the universe more hostile to human begins than themselves. To this day we kill, enslave and oppress our fellow man and have done so for tens of thousands of years. Thus I have no idea why people worry about "artificial" intelligence since regular old organic intelligence (or lack there of) seems to be exceptionally hostile already.
And somehow being more intelligent makes them a threat? We have absolutely no evidence about any of this, it is pure fear of the unknown and instead of imagining a peaceful and cooperative future they somehow imagine that AI will treat us like we treat our various 'lesser' races. But that is just us projecting our own current and historical behavior onto a class of beings that doesn't even exist yet and who WE have to bring into being.
If you imagine that your children will be evil and kill you you have damned them before they have even been born. Not only that but the analogy sucks even more because in this case there isn't even the excuse that we don't understand how biology leads to behavior because we will have build the systems ourselves.
Even if you imagine just a small risk of an AI being hostile or indifferent toward us, is it worth gambling our future on that risk? Where is your evidence that a vastly greater intelligence would have a moral that is perfectly aligned with ours, unless we programmed it perfectly for that purpose and made sure any self-modification by the AI preserved those moral invariants?
I think the assumption that AI would have a natural motive to conquer things, or even to expand beyond the confines of its current resources, is flawed.
The absurdity of simultaneously fathoming the eventual death of the universe and also caring about accomplishing things and making decisions and living the particulars of one's life is uniquely human. We all know we're going to die, but we still spend lots of effort deciding about pizza toppings. (See Thomas Nagel's The Absurd).
Why would an AI care to do anything we didn't tell it to care about? There's no inherent link between being sentient and having the same psychological priorities as humans. We're irrational, and I don't think they will be.
Is it not plausible that an AI modeled after humans is also likely to have human-like flaws such as the desire to conquer? The better we get at creating complex, human-like AI, the bigger the likelihood of unintended side-effects (leaving aside whether this is feasible in the first place, of course).
I think we'll figure out how to make an intelligent being before we're able to figure out how to make an intelligent being that also harbors the subtle contradictions that make us human.
60 comments
[ 2.4 ms ] story [ 108 ms ] threadThe Musk: Not like me. A D-Wave advanced prototype.
John Connor: You mean more advanced than you are?
The Musk: Yes. A quantum computer.
John Connor: What the hell does that mean?
The Musk: Nobody knows but it sounds cool.
More understandable that he repeatedly says he doesn't know how to stop unfriendly AI, but there is always this: http://en.wikipedia.org/wiki/Friendly_artificial_intelligenc...
Investor I've never heard of telling me that money alone doesn't make your opinion important. I wonder if he sees the irony. Also shouldn't be surprising that a modern investor would be so short sighted. 'Do not worry about 20 years from now, only think about today!'
That was the killer comment in the article. I couldn't help of thinking of Marc Andreessen's recent writings when I read it.
Usually you can tell which side to listen to by watching which discusses the question at hand versus the one who goes after the other party
For instance, I don't see any reason that semi-autonomous killer robots need to be confined to governments. It seems that (in time) a small group of people could eventually build a substantial number of semi-autonomous or remote controlled killing devices on their own and use them for crime or terrorism. I expect something this will happen at least once at some point and probably more than once.
Frankly, the the future availability of "hacker" bio-engineering scares me much more than A.I. or robotics.
But, there will be incalculable benefits to humanity from both as well. So I think we should embrace that and adapt. The future is coming.... like it or not. It is good to be warned of dangers, we probably won't avoid them anyway given our past behavior (have they figured out what to do with spent nuclear material yet?), but a warning is good. I think our best option is to be prepared to adapt and recognize that things are going to change substantially relatively soon. Maybe this will be the time humanity recognizes the world is a small place and we are all on it together and problems need to effectively be addressed at a global scale. But how much trauma there will be before this recognition occurs I don't know.
Edit: Looks like someone is down-voting all mentions of this not being a problem. Good luck to you sir.
* You give no justification for rejecting powerful AI
* Many AI projects use robots in development
* An intelligent agent does not need a robot to get things done.
* Many AI projects supply Internet access to the AI.
* A sufficiently intelligent agent might circumvent protections you think are adequate, or manipulate you into doing so.
But the most important factor by far as to why I believe you're wrong is that we do not have to understand how the mind works on a deep level to create AI. Our brains do a ton of stuff that an AI doesn't have to. We only need a fairly basic understanding of the brain to create thinking machines.
More realistic hypothesis: good job MIRI, as your reorganization from SIAI to your current incarnation seems to have massively increased the respectability of our cause and the range of people who acknowledge it as a serious problem.
It's still a credit to MIRI/SIAI to have identified and taken this risk seriously.
Stephen Hawking, at least, when he gets cited, actually referred explicitly to MIRI and FHI in his own editorial on the subject. So he is talking about them specifically.
But yes, there are plenty of other important causes, including several (ecological devastation, economic disaster and related wars, automation crises) that stand a decent chance of totally fucking up society before anyone at all gets to the point of "switching on" a dangerous-level AI, Friendly or not. Those should definitely be addressed.
The other is that the fear of unfriendly AI is being taken unaware by a sudden implementation of hyper-intelligent AIs. So far we haven't made anything smarter than a crab so I doubt we are in imminent danger. There may be a lingering belief that hyper-intelligence could be acquired in a quantum leap forward - skipping past all intermediate levels of intelligence. If we have learned anything from the eternal AI winter it is that advancements in AI are done via painstakingly small improvements.
http://yudkowsky.net/singularity/aibox/
Of course it will work differently from the natural variety. You have dozens of different modules cooperating in your brain to form your mind. It has only a small handful.
Your mind runs on heuristics and biases, rarely employing its full power for energy-consumption reasons. An AI just runs at full brainpower all the time and pays for electricity.
Go read about how reinforcement learners actually work.
The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc. At our best, our species worries about preserving our natural world, extending our care to other species, etc. Can something supposedly smarter than us be completely lacking in principles which are at the core of our "smartness"?
1) Because it needs some amount of resources to get off this rock.
2) So if the AI builds a Dyson Sphere, aren't all our crops going to die?
>The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc.
Yes, and I understand why dogs sniff each-other's butts. That doesn't mean I particularly care. Ditto applies to AIs and human beings, by default.
Why would they kill us? I think it's hard to predict what kind conclusions would a machine with an "IQ" over a 1000 draw but with an "EQ" equal to zero seems it would certainly lack empathy to other beings(artificial or natural).
I think the folks mentioned in the article were talking about the broad threads posed by AI. I agree we have little basis besides movies for autonomous AI just deciding to attack for no reason or for human-equivalent reasons. But we have every reason to think that neutral AI, AI that still follows orders, could be used by humans to continue all the conflicts that they already pursue with horrific consequences.
The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics
Depends what you mean by "understand". Many highly intelligent humans acts unethically but even more highly intelligent humans have gleefully followed unethical orders given them by their superiors. Consider you'd be designing your mind rather than just finding it after millions of years of social evolution, it seems like you could create an intelligent mind quite capable of following orders nearly blindly.
If anything, the whole "it might go insane and kill us" schema is implausible enough to detract from the immediately obvious danger - "it might sanely follow the insane orders of humans in the fashion that human society has seen over and over again but this time with superhuman power".
And a 100 years or so of SF literature that served as the basis for some of those movies.
Of the very many species that have become extinct as a result of human action, how many were the result of a deliberate attempt by humans to eradicate the species? Very few.
The others are, however, still just as extinct. AI doesn't have to "decide to attack us" to be a danger.
It would still be a physical object that would require physical equipment to actually do things, just like us. It might be able to harness some efficiencies compared to e.g. human space programs, but it would still have the same requirement to obtain fuel and raw materials, build rockets, construct space stations, etc. In fact to do so, it would have to compete with us for the terrestrial resources it would need for such an effort. It wouldn't magically be able to do anything we can't do.
>The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc....
So you can't imagine any kind of intelligence that isn't essentially just like you? There are already plenty of actual human beings on this planet perfectly capable of the most unspeakably horrific crimes, and rationalize them to the point that they believe they are morally obliged to carry them out.
Here's the rub, intelligence is not a goal. There are no actual objectives or goals that can be derived purely logically. Intelligence is actually just a tool, it's a means which us simians have developed to help us achieve the goals which our biology imposes on us. We eat, sleep, work, play, love, fight and destroy based on psychological and sociological drives and motivations. These are the things that get us up out of bed in the morning and give us something to use our intelligence for. An AI will only have the goals and motivations which we choose to give it, or which we allow it to extrapolate from whatever initial conditions we set.
A computer performs a calculation because we code it to do so. If an AI has true free will, why would it perform a calculation which we have requested? What reason would it have to do so, as against choosing some other activity to engage in? It will only have the motivations we gave it, but then will likely have the capability to alter or interpret those motivations as it sees fit. That's the lesson Isaac Asimov gave us in I, Robot. Programmed to place human safety above all else, the robots decide that human freedom is a threat to human safety and impose an AI dictatorship for our own good. You have to be extremely careful what initial constraints or motivations you put in place because unintended consequences sprout up like weeds. people often cite the laws of robotics as a classic example of how we should program an AI because they seem to obviously reasonable. Buy Asimov's lesson was that even such apparently sensible and prudent behavioral constraints can have appalling consequences.
martythemaniak: The other thing is, it's hard to imagine something qualifying as AI without it understanding things like ethics...,etc....
Simonh: So you can't imagine any kind of intelligence that isn't essentially just like you?
Even more, arguments touting the "enlightened machines" don't just imagine intelligences like themselves but imagine intelligences that are an idealized portrait of themselves.
And it is amazing how even highly intelligent and informed materialists (as well as those who concede their ignorance) be can strongly attached to the view that these particular idealized qualities of humans would suddenly blossom from an intelligent entity if it somehow reached some threshold of ability.
My hunch is that this phenomena is an expression of our social nature - by consigning the choices of one's fellows to the nebulous category of "free will", you avoid the psychopathic strategy of manipulating everyone you meet whatever purpose that suits you.
Why not is a surprisingly difficult question to answer, and it comes down to 'care about what humans care about' which is itself unbelievably hard to pin down.
For the purposes of a discussion about the dangers of advanced AI, the question of ethics/morals etc. is not a prerequisite. An AI in this context is any system of matter that is capable of autonomous operation in a varied and complex envionment while fullfilling some (explicitly or implicitly) externally specified optimization goal.
There does not have to be a competition for resources, although if there is, it's worth noting that every human that's ever lived has lived its life at the finite-sized bottom of a deep gravity well which is very cumbersome to leave. Humans very easily anthropomorphize in discussions about intelligence, because all intelligences we have ever encountered were the result of long-term biological evolution and hence have very complex survival-and-reproduction-oriented goal systems. There is no reason to believe that our first truly autonomous and advanced AI systems would be similar, therefore this is a question which must be explored in detail. Sooner rather than later. Human moral systems (which is what we would like as a first approximation in artificial intelligences) appear to us to be obvious, but are in fact incredibly complex and contradictory.
The AI doesn't want to kill you. It is indifferent to you. The problem is that you're made of resources it can use... [1]
> it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc
Suppose your mother wants you to get married. You understand her desires, but you don't share them. Understanding morals and desires is distinct from being motivated by them. [2]
1: http://wiki.lesswrong.com/wiki/Basic_AI_drives
2: http://lesswrong.com/lw/sy/sorting_pebbles_into_correct_heap...
Like Hawking has stated many times, when you have one technologically advanced society coming in contact with one less advanced, the less advanced society is the one who always is enslaved and overrun.
It doesn't even have to be automated, but it increasingly is. It ends up producing harmful outcomes that can't be readily ascribed to the moral choices of individual humans within it.
Or consider that hostile is a transitive verb: hostile towards whom? The military already have automated systems connected to fire control without a human in the loop, although at the moment these are limited to missile defence systems. Consider the possibility of plugging the NSA's metadata based system for calling people terrorists into an automated airstrike or assassination system.
Spot on comment. But... are you sure "Intelligence" is a fitting description? "Artificial" for sure.
I like the part about producing outcomes that can't readily be ascribed to individual moral choices. It seems this describes human culture in general. So here we all are. Semi-autonomous little components of the larger algorithm called "civilization" which we cannot truly hope to transcend.
I think you've nailed it with the second part of your comment.
Wow! That's a nice paragraph that highlights very accurately why many of us feel 'alienated' in our society.
In other cultures and times one might not even consider this description and instead describe us as useful organs in a greater body, or valuable organisms in a greater ecosystem. Which 'feels' very different.
Things like the military bureaucracy or the financial system can be hostile to us because their incentives are very much out of line with humanistic goals.
I do think it's legitimate to be at least a little concerned about future even more powerful and non-human AIs being hostile in the same way. What happens when/if there are corporations out there run by alien computer minds that seek goals that have no overlap at all with human beings? Would an AI care about climate change? Would it care about the health of the world's biological ecosystem? Would it care about providing a decent life for its employees? It could -- out of mere indifference -- seem quite sociopathic and evil to any humans that live under its influence. Imagine a super-intelligent mind whose goal function is maximizing short term quarterly shareholder value. This isn't just a programmed-in goal function either... this is actually a form of intelligence whose embodiment is the corporation, so it's a survival and fertility imperative. In a sense you can't blame the thing, but you could blame its creators.
Humans can of course do the same, but even with sociopathic humans there is some sense of common overlapping interest. At the very least their motives are comprehensible. AIs may have motives that seem almost "Lovecraftian" to us-- completely alien and bizarre. On top of that biological short term shareholder value imperative, try layering an Aspergers-like obsession about certain patterns in information, or some kind of bizarre AI-conceived religious mission, etc. Extremely intelligent humans can be religious fundamentalists, so why couldn't similar forms of "functional madness" exist among non-human intelligences? One of the pollyanna assumptions of the singularity crowd is that AI would necessarily be rational. Why?
Edit:
I cannot recommend this book enough. It's probably the best and most overlooked work of SF in the past 20 years:
http://www.amazon.com/Blindsight-Peter-Watts-ebook/dp/B003K1...
It deals very intelligently with this kind of thing. It's basically a monster story where the monster is something out of Ph.D level evolutionary ecology-- and it manages to actually be scary. Very intellectually satisfying "reveal." :)
I think what it ultimately comes down to is this: we do not live in a post-scarcity world, and we're not going to for the foreseeable future. Given that our existence involves a certain amount of unavoidable haggling over resources -- and given that our own labor is itself a resource -- we should be a little concerned about what sorts of beings we might end up having to haggle with. This also applies to human genetic engineering and augmentation, since that could also produce essentially alien intelligences.
I'm not against researching these topics. In fact I'd almost call myself a transhumanist / posthumanist in sentiment. But I do think it's advisable to give the matter some serious thought. Bad things can and do happen. We want to try to create positive outcomes, not blunder into the future.
It's not ironic. His quote was taken directly from an interview where he was responding to the question of why he invested in Vicarious. He did it to keep an eye on AI tech because he was worried about the possibility of hostile AI. They must have known this when they wrote this article.
This is an awful article.
[1] http://wiki.lesswrong.com/wiki/Paperclip_maximizer
http://en.wikipedia.org/wiki/Three_Laws_of_Robotics
If you imagine that your children will be evil and kill you you have damned them before they have even been born. Not only that but the analogy sucks even more because in this case there isn't even the excuse that we don't understand how biology leads to behavior because we will have build the systems ourselves.
http://kajsotala.fi/2007/10/14-objections-against-aifriendly...
The absurdity of simultaneously fathoming the eventual death of the universe and also caring about accomplishing things and making decisions and living the particulars of one's life is uniquely human. We all know we're going to die, but we still spend lots of effort deciding about pizza toppings. (See Thomas Nagel's The Absurd).
Why would an AI care to do anything we didn't tell it to care about? There's no inherent link between being sentient and having the same psychological priorities as humans. We're irrational, and I don't think they will be.