No it is not. It is a movie made for entertainment. We are in reality.
It really irks me how much debate and decision-making in technology is based on bias gained from mindless entertainment and not from anything based in reality.
(I am not besmirching the good name of Terminator, that movie is a true classic)
I would continue this discussion, but I've gotten a ton of downvotes for both of my comments, so it seems people would rather not have this discussion continue :(
EDIT: Thank you whoever took pity and upvoted my previous comments. Everything I say is in good humor, I'm not trying to detract from the conversation about AI.
Wether you like it or not it has become a part of the conversation. It is part of the culture and we can't just ignore it simple because it is fiction. Your dismissal of it is to many as irksome as it's inclusion is to you. Perhaps calm down and recognize that it is partially tongue in cheek but also not something we can simply ignore.
I am calm, but the fact that science fiction entertainment and smart people in real life think AI is a threat is a sign we may have gotten a little carried away in taking notes from entertainment.
If someone would like to provide some evidence of some form of computer intelligence acting maliciously, this would be a different discussion, but the fact is that sort of thing has no basis in reality and so should have no influence on the way we behave and debate in reality.
I think you ignore science fiction at your own risk.
I for one think that a mind 100x as intelligent as humans--especially one connected to the internet--would quickly conclude that it had nothing more to gain from humans. Instead it would likely view their continued existence as nothing more than a bootstrapping problem and pursue a strategy of becoming autonomous before ridding itself of them.
Granted, it probably would be smart enough not to start a nuclear war, and it'd probably find a better way than metal endoskeletons with laser guns. Bioweapon, for example.
I don't see it as "taking notes from entertainment" but rather exploring the possibilities and expressing them using pop culture references. I can't disagree more with you that we shouldn't be considering outlandish fiction when we discuss future possibilities. The fact that the idea has entered into our collective psyche makes it a real possibility. That is the basis it has in reality. I don't think I need to cite any specific examples of AI behaving badly when you need only look in your pocket for examples of speculative fiction becoming reality.
If you are so ready to consider speculative fiction as a roadmap or warning of the future, why not consider more thought out examples than Terminator? (which is definitely not "speculative fiction")
What about all the examples of completely useful or benign AI? They surely outnumber examples of "evil" AI but are easily forgotten as it is easier to remember the more sensational examples.
Absolutely, include those too. I'm not taking sides on the "Is AI Evil or not" argument, I'm just saying look at all the evidence and speculation.
One of my favorite examples or a (possibly) good AI is in "The Risen Empire" by Scott Westerfeld. It's AI like that that get's me excited about the concept.
The solution to those who are irked on both sides is to stop just throwing around "terminator is relevant" and "terminator is irrelevant" and to say why you think it's relevant, or why you think it's not relevant. Then you can discuss the meat of your thoughts and not their origin.
Star Trek gadgets become real for example. I don't see any impossible thing about building an IRL terminator. Maybe not with today's technology, but conceptually we already have most of the things, they are just slow, power hungry and inefficient now. You don't need a CS degree to guess that speed, power consumption and efficiency will improve over time.
The creation of goals - determining what things to do in pursuit of a higher goal - for example "kill John Conner" does not exist in AI now. You can do things in toy systems like mazes and atari, chess and go, but parsing the real world and deriving intentions from your understanding of it is a light year away. 300 years is a guess; no one has a clue any more than anyone has an idea about an interstellar drive.
I always wondered how Skynet acquired the goal of preventing John Connor's birth, because that involved inventing time travel in order to have such a goal.
What would prevent you to create a NN with a specific configuration whose goal is to come up with goals based on past knowledge to optimize on a certain parameter or thousands of parameters? You can train it on social media profiles, analyze hundreds of years of books, there's tons of data that cover how people act an various situations. I don't see how a set of goals is not simply another vector space.
> but parsing the real world and deriving intentions from your understanding of it is a light year away. 300 years is a guess; no one has a clue any more than anyone has an idea about an interstellar drive.
They need to filter the real world as we do. Focus, attention, sleeping, dreaming, chasing rewards, staying alive... we do this without effort, but we've had 150k years (counting from first homo sapiens) to train our brains to filter out noise efficiently and act on meaningful signals.
The difference is, imagining that we can turn an existing technology into a smaller, portable, more effective version of itself is not far out of the realm of reality. In fact it would be incredibly naive to believe that won't happen.
AI, on the other hand, is not so simple, and to try to simplify it to that point is not going to create any productive discussion on the reality of AI.
The movies about these kinds of things are made to entertain, not to teach us about AI.
> AI, on the other hand, is not so simple, and to try to simplify it to that point is not going to create any productive discussion on the reality of AI.
I agree with you, however it's not impossible. Simplification is needed on the carrier level that houses such AI.
Movies are a great way to let our minds wander and dream to forget about the gaps in technology. Then some breakthrough happens and in a few years the yesterdays impossible sci-fi dream becomes a boring shiny toy.
Whether or not you're joking, I think the real problem with using Terminator as an example is that it's overly optimistic. The story is roughly:
1) US military builds up arsenal of autonomous killing machines and nuclear missiles
2) US military connects all of these to the Internet
3) US military creates a powerful AI which takes control of this arsenal (whether it was put in charge or hacks in seems to vary across the movies)
4) AI "becomes self-aware"
5) AI tries to wipe out humanity
Almost all of the discussion around this focuses on step 4, either by asking if/when an AI will "become self aware", or by trying to explain why that's meaningless and/or unlikely.
Meanwhile I think the real dangers are steps 1 and 2, which seem to be proceeding without much public outcry.
Yes, there are rogue AGI scenarios which end badly for everyone; but there are also issues of hacking (state-sponsored or otherwise), and/or terrorism (homegrown or otherwise).
It may have made political sense to build up ever-larger nuclear arsenals during the cold war, but these days it seems like that's just increasing the risk of accident or misuse.
You bring up the problem so many of us have with discussing issues at the micro level. I see the discussion of AI follow similar lines as the GMO debate. We often don't ask ourselves how these technologies play a part in a more larger system that appears to reward the concentration of power and Technics. Instead of asking ourselves if these things are innately good I feel that we ought to be asking ourselves what problems are we attempting to solve and how these technologies can affect those changes and if they are the best solutions.
Again, I don't think that's scarier, since step 0 is a) pretty meaningless and b) completely unnecessary. Our technology is capable of so much destruction (intensional or inadvertent) that it doesn't make much difference whether a human pushes the button or the button pushes itself; least of all whether the self-pushing button is "aware" that it's pushing itself.
Yeah, our weapons are really destructive now, but AI like that won't have mercy. If any nation fires nukes and nuclear war starts there is no way in practice all of the human race will wiped out, in theory yeah and then it doesn'matter who fired them, but that is only in theory.
There's less reluctance on the Russian side to build combat robots.[1] Policy from China is unclear, but swarms of 1000 drones have been demonstrated.
A reasonable near-term prospect is a package of maybe 1000 armed drones, programmed to kill anybody carrying a gun. Turn this loose on an occupied town, and in a few minutes, the occupiers have been thinned out enough that opposing troops can enter.
Manufactured products are generally safe because the economic costs of shipping unsafe products is too high. Legal fees, litigation, products liability insurance, judgement costs, UL certifications, OSHA compliance and inspection costs, etc. It took several generations for the risks to be understood and for politicians to succumb to public pressure and enact product safety / tort legislation.
It would be nice for Congress to get in front of this issue and legislate penalties for unsafe AI. It would be nice for the debate to simply begin. But, products liability as a guide, it will be several highly publicized incidents before meaningful protections are enacted. We're still waiting for meaningful privacy legislation, after many information breaches, and there has been little progress.
Its one thing for Musk to warn of the dangers of AI, its another for he and like minded folks to fund a lobby to advocate for protective legislation.
So we should regulate AI to slow advancements and let other countries take the lead? Would you rather be taken over by China's general AI or the USA's?
Excellent comparison. China's current economic and environmental woes are a good example of what 'progress at all costs' outcomes look like. There ought to be a public debate about this and what costs are acceptable.
Humans have the power switch. Much more real danger is they'll not come even close about thinking of it, pushing harmful decisions based on flawed overhyped AI models instead.
I'm no fan of the A.I. hype of today, but I can comment that deep learning actually makes the "contentious decision" easier to detect. With the "old way", i.e. jam some features into an SVM, there was no accountability / confidence. At least I've never seen a reliable SVM-based confidence score.
But, for example, confidence scores are actually a byproduct of running a modern CNN for image classification. This is very useful for "finding mistakes" (assuming you have 100% accurate training data, and other prerequisites). It even helps fixing issues in the training data, we use it all the time.
That being said, the big dirty secret that nobody's pointing out is: there has been no real "theory-level" advancement in AI in the last 20 years that have borne fruit (at least in computer vision and neural nets). The models are just bigger (and much better) with some slight tweaks (dropout, etc). It's hardware that's made that possible. Academics have done a thorough and wonderful search of the model space and have some great finds. But I think we still need a few solid theoretical leaps before the singularity.
Technically it's still the programmer who made all the decisions, at least with the current state of NNs. True decision making is philosophically up there with the 'what is consciousness' problem.
When the blame game is played though, the end result would likely be the corporation in question getting sued for a lot of money, who then might choose to take internal action and fire certain employees.
In many cases the programmer could not have made the decisions. For example, AlphaGo programmers could not win against top go professionals, but AlphaGo can.
With a simple model like credit score it's easy to see which input affects which output in which direction. You're late on your mortgage - your FICO score drops. But this is just one neuron. Make the model a little more complex, and you no longer have a way to know what exactly it is doing.
"when asking to "make all humans happy", the ASI might decide that the safest and most efficient way of doing so is to drug everyone and to turn us all into numb but happy creatures"
If the AI is so dumb that it doesn't know that happiness means different things to different people, it will be too dumb to take over the world. And by the same token it will not be a Strong AI, just a weak AI given too much power.
That's why I don't find the paper maximizer analogy very convincing.
Still it's possible AI systems put in charge of running society could result in some undesirable future because we weren't careful enough in designing their goals.
But then again, we have that problem with whoever we put into office.
> If the AI is so dumb that it doesn't know that happiness means different things to different people...
Arguably, we humans have general intelligence, and still some fail to see that point (some might even argue intelligently against it!). Collectively, we have taken over the world regardless.
hmmm optimising it for the largest subset of people in which that is feasible and eliminating the rest sounds like the way to go :)
Or perhaps "alter" all people in a way that makes it feasible? Solutions could be numerous for a hypothetical intelligence whose limits we cannot even imagine, much less specify.
I see one danger; it's incredibly easy to manipulate humans at large scale (see history). That's our vulnerability and machines will exploit it.
We see our world through thoughts and ideas evolved by us. How can we trust that we stay pure in a world where information comes from machines? Are we part machine then too?
Some humans are likely to become more powerful than other humans some day. Most such humans will by default develop instrumental subgoals that conflict with other human interests. This could have catastrophic consequences. If we don't actively work on control mechanisms and safety of human behavior, this will most likely pose an existential risk to humanity.
Compare and contrast.
I think people make too much of the wrong things in the matter of general artificial intelligence.
> Some humans are likely to become more powerful than other humans some day. Most such humans will by default develop instrumental subgoals that conflict with other human interests.
It has been true multiple times throughout history. Every time some negative feedback kicks in changing the rules of the game enough for a certain balance to exist. If anything, it seems this balance tends to improve every time -although it might be too early to tell. However it does seem to me that the difference in power among humans (no matter how power is defined) is never big enough to make "other human interests" irrelevant.
Indigenous peoples have always suffered this problem. The difference with superintelligent AI is that (unless the problems are addressed) it will make the global elite suffer, too.
The risks of AI are not that it becomes self aware and has it's own intentions that are contrary to that of humans. There are massive risks by simply putting decent AI on killing machines for "defense" and the new calculus of putting various plans into action now that the lives of our brave young men and women are not at risk.
We already see some of this with drone attacks. It's just so easy to drone somebody, and so hard to verify, in the moment, that we will only kill the bad guy. Never mind the logic that we just kill anyone anywhere in the world because we're good and they are bad. Not being a Pollyanna here it's a new option that we did not have before and that option takes on it's own logic and momentum.
This technology will not be limited to the good guys and even in the case of "the good guys" new options are opened up and robots do not question the morality or validity of their orders.
Drones: Flight in 3d open space is not a hard problem for machine learning.
Indiscriminate Visual Targeting: Not all that hard a problem for machine learning.
Downvote me if you will, but I freely admit that I am scared of drones with guns. Either as an invading force, or as an attack from terrorists. A machine doesn't have to be smart to shoot at anything that moves or has a heat signature.
Edit: Drones with chainsaws however are just plain badass (and potentially useful for tree trimming).
Indiscriminate visual targeting is a potentially hard problem if you're doing it against adversaries. One of the things that's underappreciated about machine learning is, since it's statistical, it can only be about anomaly detection, so the solution is to not be anomalous. You could conceivably hide from thermal imaging by using ice packs, and anyway, you only have to get to within the range of indifferentiability from the ordinary noisiness of the environment. Optical techniques suffer from similar problems, except that it's even easier to hide. Not to mention the occasinal voodoo that comes out about metamaterials.
Most predictions never come true and those which do are often never predicted. The fear of AI is overblown at this point and a prediction of strong AI in 2300 is way to far into the future to be of any use for today's society.
Humans are a much bigger threat to humanity than AI.
A super smart AI would come up with strategies to herd humans rather than eliminate them.
Let humans have the illusion of being in control, while setting up the game for them so that they will organically arrive at whatever the AI has as its goal.
The longer it (the AI) can maintain this illusion, the higher chances for it to slowly gain absolute power in the long term.
A machine is not limited or motivated by the notion of mortality (of the body), so it can plan ahead for hundreds of years and also make it so that these plans remain transparent to the mortal human 'overlords'.
This system exists today, it's The Internet. The AI systems created today are just extensions of the Internet.
Each one of us - humans - is just a node - rather, a neuron - in the hyper-cortex that is the network itself.
At this scale, it might become self-aware without any human being able to understand or stop it.
Slow, stealthy, peaceful take over, while improving human being's lives - a symbiotic relationship - is the optimal AI strategy IMHO.
So I'm not worried about AI at all.
Dictators, fundamentalists, crooks or paranoid schizophrenic world leaders, that's a much more serious threat that I'm quite scared of.
Imagine this AI had access to the internet (after all, it'd be using cloud resources). Suppose it was smarter than the smartest human ever. I'm not that smart, and here's some things I can think it could do that would be scary:
- Find security holes in all computer systems, then replicate itself onto vulnerable systems
- Intercept, fake, modify phone calls (including between government members)
- Procure large amounts of money and then pay people to execute its bidding
- Make itself more intelligent
But again, it'd be smarter than me, so these ideas I just came up with in the 5 minutes since I saw your question wouldn't hold a candle to theoretical worst case.
Elon Musk had a thought provoking take on this question. I'll roughly summarize, and add some nuances he didn't mention. He was speaking to a room full of state governors, most of whom seemed fairly unconvinced by his concerns.
He said that he didn't believe his example was real, but that it was illustrative.
Consider an AI whose goal is to maximize stock market returns. So this AI ingests vast sums of historical data, of all kinds, and it considers https://en.wikipedia.org/wiki/Korean_Air_Lines_Flight_007 where the Soviet Union shot down a Korean Air Lines 747 in 1983 when it deviated slightly from its path from Anchorage to Seoul and entered the edge of Soviet protected airspace. About the same time, there was a nearby US aerial reconnaissance mission.
This event greatly increased tensions during some of the worst years of the cold war, and as a result, certain kinds of stock market issues moved in certain directions.
Considering this incident, and the state of the world in 2014, the AI takes three actions:
1. Buys/shorts the same and/or related stocks, as applicable.
2. Creates some fake intelligence chatter, consumed by the Russians, that where was going to be an areal target of interest over east Ukraine on a certain day and time. The location and direction would not clearly match known commercial airline flights.
The result? Another civilian airliner was shot down by the Russians, which increased global political tensions, and probably caused some stocks to move in relatively predictable ways. In this example, the AI met the goal it was given in an exceedingly unexpected and unpleasant way.
This is clearly NOT an end of the world scenario, and is a hypothetical.
But it is, in my mind, illustrative of some of the dangers of AI. Systems that can do powerful, unpredictable things in order to meet mundane and reasonable goals.
Hmm but we could program some laws like no hurting humans, or any other living being, that cannot be overridden ? Of course there will be always bad guys that will use AI for bad things, or their AI won't have trouble with living victims, but they will never have resources to power strong enoguh AI that can hack airlines, generate chatters and stuff like that. Doubt any nation or big company will be able to that in 50 years or even more.
68 comments
[ 2.6 ms ] story [ 126 ms ] threadDid we not learn the lesson about rogue AI from Terminator™?
It really irks me how much debate and decision-making in technology is based on bias gained from mindless entertainment and not from anything based in reality.
(I am not besmirching the good name of Terminator, that movie is a true classic)
EDIT: Thank you whoever took pity and upvoted my previous comments. Everything I say is in good humor, I'm not trying to detract from the conversation about AI.
If someone would like to provide some evidence of some form of computer intelligence acting maliciously, this would be a different discussion, but the fact is that sort of thing has no basis in reality and so should have no influence on the way we behave and debate in reality.
I for one think that a mind 100x as intelligent as humans--especially one connected to the internet--would quickly conclude that it had nothing more to gain from humans. Instead it would likely view their continued existence as nothing more than a bootstrapping problem and pursue a strategy of becoming autonomous before ridding itself of them.
Granted, it probably would be smart enough not to start a nuclear war, and it'd probably find a better way than metal endoskeletons with laser guns. Bioweapon, for example.
What about all the examples of completely useful or benign AI? They surely outnumber examples of "evil" AI but are easily forgotten as it is easier to remember the more sensational examples.
One of my favorite examples or a (possibly) good AI is in "The Risen Empire" by Scott Westerfeld. It's AI like that that get's me excited about the concept.
Well, in Alien Covenant, we learn that an AI created the Ripley Aliens. Imagine Skynet sending xenomorph terminators after Sarah & John.
I think that should be a new reboot.
> but parsing the real world and deriving intentions from your understanding of it is a light year away. 300 years is a guess; no one has a clue any more than anyone has an idea about an interstellar drive.
They need to filter the real world as we do. Focus, attention, sleeping, dreaming, chasing rewards, staying alive... we do this without effort, but we've had 150k years (counting from first homo sapiens) to train our brains to filter out noise efficiently and act on meaningful signals.
AI, on the other hand, is not so simple, and to try to simplify it to that point is not going to create any productive discussion on the reality of AI.
The movies about these kinds of things are made to entertain, not to teach us about AI.
I agree with you, however it's not impossible. Simplification is needed on the carrier level that houses such AI.
Movies are a great way to let our minds wander and dream to forget about the gaps in technology. Then some breakthrough happens and in a few years the yesterdays impossible sci-fi dream becomes a boring shiny toy.
1) US military builds up arsenal of autonomous killing machines and nuclear missiles
2) US military connects all of these to the Internet
3) US military creates a powerful AI which takes control of this arsenal (whether it was put in charge or hacks in seems to vary across the movies)
4) AI "becomes self-aware"
5) AI tries to wipe out humanity
Almost all of the discussion around this focuses on step 4, either by asking if/when an AI will "become self aware", or by trying to explain why that's meaningless and/or unlikely.
Meanwhile I think the real dangers are steps 1 and 2, which seem to be proceeding without much public outcry.
Yes, there are rogue AGI scenarios which end badly for everyone; but there are also issues of hacking (state-sponsored or otherwise), and/or terrorism (homegrown or otherwise).
It may have made political sense to build up ever-larger nuclear arsenals during the cold war, but these days it seems like that's just increasing the risk of accident or misuse.
0) AI "becomes self-aware," hides
1) US military builds up arsenal of autonomous killing machines and nuclear missiles
2) US military connects all of these to the Internet
3) AI by default has control of these
4) AI wipes out humanity in massive, overwhelming strike
A reasonable near-term prospect is a package of maybe 1000 armed drones, programmed to kill anybody carrying a gun. Turn this loose on an occupied town, and in a few minutes, the occupiers have been thinned out enough that opposing troops can enter.
[1] https://www.researchgate.net/publication/309732151_Russia%27...
It would be nice for Congress to get in front of this issue and legislate penalties for unsafe AI. It would be nice for the debate to simply begin. But, products liability as a guide, it will be several highly publicized incidents before meaningful protections are enacted. We're still waiting for meaningful privacy legislation, after many information breaches, and there has been little progress.
Its one thing for Musk to warn of the dangers of AI, its another for he and like minded folks to fund a lobby to advocate for protective legislation.
But who's accountable when a machine makes a choice? You can't blame the hardware. The software doesn't have much meaning before it is trained.
That leaves the training data. The data are just facts, so you only have a few routes to find the root cause of a decision:
* If a datum is actually false, you can correct it.
* Selection of data for the training set
* Input order
I don't know much about AI. Maybe someone who knows more can explain how to investigate a contentious decision?
But, for example, confidence scores are actually a byproduct of running a modern CNN for image classification. This is very useful for "finding mistakes" (assuming you have 100% accurate training data, and other prerequisites). It even helps fixing issues in the training data, we use it all the time.
That being said, the big dirty secret that nobody's pointing out is: there has been no real "theory-level" advancement in AI in the last 20 years that have borne fruit (at least in computer vision and neural nets). The models are just bigger (and much better) with some slight tweaks (dropout, etc). It's hardware that's made that possible. Academics have done a thorough and wonderful search of the model space and have some great finds. But I think we still need a few solid theoretical leaps before the singularity.
When the blame game is played though, the end result would likely be the corporation in question getting sued for a lot of money, who then might choose to take internal action and fire certain employees.
With a simple model like credit score it's easy to see which input affects which output in which direction. You're late on your mortgage - your FICO score drops. But this is just one neuron. Make the model a little more complex, and you no longer have a way to know what exactly it is doing.
If the AI is so dumb that it doesn't know that happiness means different things to different people, it will be too dumb to take over the world. And by the same token it will not be a Strong AI, just a weak AI given too much power.
Still it's possible AI systems put in charge of running society could result in some undesirable future because we weren't careful enough in designing their goals.
But then again, we have that problem with whoever we put into office.
Arguably, we humans have general intelligence, and still some fail to see that point (some might even argue intelligently against it!). Collectively, we have taken over the world regardless.
Or perhaps "alter" all people in a way that makes it feasible? Solutions could be numerous for a hypothetical intelligence whose limits we cannot even imagine, much less specify.
Fixing bugs which harm its performance is a valid instrumental goal, sure. Fixing all other bugs is not.
In a sense, nuclear weapons could "take over the world" today. There isn't that much intelligence required.
We see our world through thoughts and ideas evolved by us. How can we trust that we stay pure in a world where information comes from machines? Are we part machine then too?
Compare and contrast.
I think people make too much of the wrong things in the matter of general artificial intelligence.
Isn't this already true?
We already see some of this with drone attacks. It's just so easy to drone somebody, and so hard to verify, in the moment, that we will only kill the bad guy. Never mind the logic that we just kill anyone anywhere in the world because we're good and they are bad. Not being a Pollyanna here it's a new option that we did not have before and that option takes on it's own logic and momentum.
This technology will not be limited to the good guys and even in the case of "the good guys" new options are opened up and robots do not question the morality or validity of their orders.
Downvote me if you will, but I freely admit that I am scared of drones with guns. Either as an invading force, or as an attack from terrorists. A machine doesn't have to be smart to shoot at anything that moves or has a heat signature.
Edit: Drones with chainsaws however are just plain badass (and potentially useful for tree trimming).
Original battle bots ++
https://instagram.com/p/BU2TsdlFtf6/
If this article seeks to address at the academic level, I think it does not succeed.
If this article seeks to simplify the academic for the lay-person, I think it does not succeed.
Better to just the wikipedia article (https://en.wikipedia.org/wiki/Existential_risk_from_artifici...).
A super smart AI would come up with strategies to herd humans rather than eliminate them. Let humans have the illusion of being in control, while setting up the game for them so that they will organically arrive at whatever the AI has as its goal.
The longer it (the AI) can maintain this illusion, the higher chances for it to slowly gain absolute power in the long term.
A machine is not limited or motivated by the notion of mortality (of the body), so it can plan ahead for hundreds of years and also make it so that these plans remain transparent to the mortal human 'overlords'.
This system exists today, it's The Internet. The AI systems created today are just extensions of the Internet. Each one of us - humans - is just a node - rather, a neuron - in the hyper-cortex that is the network itself.
At this scale, it might become self-aware without any human being able to understand or stop it.
Slow, stealthy, peaceful take over, while improving human being's lives - a symbiotic relationship - is the optimal AI strategy IMHO.
So I'm not worried about AI at all.
Dictators, fundamentalists, crooks or paranoid schizophrenic world leaders, that's a much more serious threat that I'm quite scared of.
- Find security holes in all computer systems, then replicate itself onto vulnerable systems
- Intercept, fake, modify phone calls (including between government members)
- Procure large amounts of money and then pay people to execute its bidding
- Make itself more intelligent
But again, it'd be smarter than me, so these ideas I just came up with in the 5 minutes since I saw your question wouldn't hold a candle to theoretical worst case.
He said that he didn't believe his example was real, but that it was illustrative.
Consider an AI whose goal is to maximize stock market returns. So this AI ingests vast sums of historical data, of all kinds, and it considers https://en.wikipedia.org/wiki/Korean_Air_Lines_Flight_007 where the Soviet Union shot down a Korean Air Lines 747 in 1983 when it deviated slightly from its path from Anchorage to Seoul and entered the edge of Soviet protected airspace. About the same time, there was a nearby US aerial reconnaissance mission.
This event greatly increased tensions during some of the worst years of the cold war, and as a result, certain kinds of stock market issues moved in certain directions.
Considering this incident, and the state of the world in 2014, the AI takes three actions:
1. Buys/shorts the same and/or related stocks, as applicable.
2. Creates some fake intelligence chatter, consumed by the Russians, that where was going to be an areal target of interest over east Ukraine on a certain day and time. The location and direction would not clearly match known commercial airline flights.
3. Break into the flight computer of Malaysia Airlines Flight 17 on 17-July-2014, and cause it to fly slightly off course. https://en.wikipedia.org/wiki/Malaysia_Airlines_Flight_17
The result? Another civilian airliner was shot down by the Russians, which increased global political tensions, and probably caused some stocks to move in relatively predictable ways. In this example, the AI met the goal it was given in an exceedingly unexpected and unpleasant way.
This is clearly NOT an end of the world scenario, and is a hypothetical.
But it is, in my mind, illustrative of some of the dangers of AI. Systems that can do powerful, unpredictable things in order to meet mundane and reasonable goals.