Advances in tech should not overpower Society's needs. The first thing people do with tech is to use it to create or enhance weapons. Anyone that denies that needs to look back at history and see how technical innovations have been used. It's time to discuss the dangers of tech and how to limit its ability to create massively dangerous weapons. We all need to take the rose color glasses off and deal with the problem.
No, pick every advancement/breakthrough in technology and you'll see it either enhanced a weapon or created a new class of weapon. No cherry picking. If it can be used in society it can be used in war.
This argument is a little pedantic, but here are some examples that contradict your assertion.
Consider this: without the bomb we would not have nuclear power. Without modern warfare we would not have modern medicine. There are other examples, but you would have been worse off limiting these. Not to mention non-technological benefits like geopolitical stability from MAD.
Well I think what OP actually means (how I understand it) rather than specific inventions is "meta-technologies" such as AI, cryptography, the internet, etc. Perhaps meta-technology isn't the right term or super clearly defined but hopefully you can grok what I mean. Clearly not every single invention was created for he military.
Better clothing for soldiers, tents, and more clothing for them.
Plenty more I'm sure. Early war planes used canvas instead of metal as outside cover as they went out to fight each other.
> Hypodermic needle
Enhancement of battlefield medicine. Fix them faster so they can continue to fight.
> Fracking
Cheaper and continued source of fuel for war machinery
> Solar
Spy satellites, power for field communications. It's only the beginning.
Nuclear power is a perfect example of the dangers of technology. The world powers have decided to limit its spread because it's so dangerous to let everyone have it. If nuclear bomb tech was free to run its course they would be small, cheap and every country would have some. And most would not hesitate to use them.
"Making clothes for soldiers" is reaching. You're expanding "use technology for war" to the point of nonsense and diluting the point. I can just as easily say that the cotton gin enhanced every thing ever because it allowed every human involved in every endeavor to wear clothes.
Furthermore, to circle back to your original point: even if we accept that the cotton gin is a technology "used for war", it most certainly was not the first nor the primary use of that technology (the first use was clothing).
You're cherry-picking an irrelevant property of the argument. So what if, hypothetically, general AI is used nth rather than 1st in line for warfare? It's still AI used for warfare.
Ok don't take that as an impact but tech does not live in a bubble. The cotton gin was a contributor to the civil war. It made cotton processing much easier and more cotton was grown which required more slaves. Which inflamed the debate between the ones that wanted slavery and those that did not.
So from my view it not only aided the war once it started. But it was also a contributor to its beginning.
But whatever I say you'll find fault. You'll focus on the small details. I'm a techie too and I sometimes find myself arguing the smallest detail whether it has consequences or not. It's that type of attention to detail that keeps us employed when we need to make sure things are working.
The bottom line is that you think tech is great while I think it's great but we have to be careful with it's development and use.
All I can say is that it's worth being careful where we step.
That's tautological. The very natures of technology and war mean that any and all technologies can, and will, be used for military, directly or indirectly. As you say, "If it can be used in society it can be used in war". But this is not a good argument against technology or research.
As Rodney Brooks was recently quoted: "There are quite a few people out there who’ve said that AI is an existential threat: Stephen Hawking, astronomer Royal Martin Rees, who has written a book about it, and they share a common thread, in that: they don’t work in AI themselves" [1]
That's gotta be some kind of pick and choose fallacy (availability bias?). Those two happen to be high-status individuals from old high-status science disciplines so they tend to get whatever they say amplified. I wouldn't use this kind of reasoning to figure out what opinion AI professionals as a class have, but I do know there are more celebrities outside of AI than inside of AI.
It doesn't take much AI to create smart weapons. A smart drone can do a lot of damage. I don't even want to think about a level 5, completely automatic, car, which is what tech visionaries are aiming to create.
Tech creators look at the bright side of their creations. They have tunnel vision and can't foresee the harm. If they can't others need to.
The criticism that Musk does not know enough about AI ignores the fact that he is trying to create a fully automatic car with AI and has experience with its limitations. Yes AI(a catch all process for making computers do things that require intelligence when done by humans) is a process and it has no intelligence in itself but you can say the same thing about C++. It's just a language but look at what's possible to do with it.
I'm struggling to understand your point, because it just sounds like FUD. I mean that sincerely - you're not really backing up your concerns here, other than to (essentially) say the technology is new. Yes, emerging technologies can be risky. But as you just said, a programming language is also capable of a lot of harm.
So what exactly are you proposing, and what precisely is your grievance? You're going to have to "think about a Level 5 autonomous vehicle" if you want to convince opponents of your position, because you'll need to come up with a more convincing argument than, "I don't want to think about how dangerous it could be."
Hadn't there been an incident where Microsoft had a chat bot that started swearing on people after having conversation with internet people. So like that, if say a caretaker humanoid uses machine learning to learn from human and if these robots learn that humans fight to protect their loved ones...maybe the bot could also pick up a fight with other humans for wrong reasons.
I think its one way to explain what i felt about rouge AI, its not actually a bad AI, but a bad dataset used by AI.
Do you really need someone to explain to you that AI is a force multiplier for an individual interested in terrorism? Autonomous drones are much better killing machines than a guy driving a semi truck through a crowded street in Nice.
If AI advances to the point of being able to determine the most optimal path to a solution, and the pre-programmed solution by a terrorist is murder everyone you can find - do you not see the danger that increasingly sophisticated technology poses?
Deciding what the solution to this problem is is non-trivial - but you have to be kidding me if you don't see that we're headed for a much scarier world when a disenfranchised person could give their AI robot the command to determine a plan to build the most devastating bomb possible, build the device, and go leave it in the optimally damaging place.
Yes, I really do need that explained to me, and no I'm not kidding when I say I don't see us heading for that future.
Hey, I could be wrong! I'm willing to be wrong, this is not my primary domain of expertise. But I remain unconvinced by arguments that are consequentialist without any reasoning about the specifics, because they are just as susceptible to be hysterical as they are to being valid.
Tell me exactly how you envision us getting from the current state of artificial intelligence to, "an arbitrary person to automate the design and deployment of a bomb, such that dangerous warfare is meaningfully democratized."
I understand you think this is possible; a lot of things are possible. I want to know how you think we can get from point A to point B. I don't want to hear something that devolves to, runaway or recursive AI self-improvement. I'm a technical person - if that's a component of your argument, I want to understand how exactly you see the current state of the art in AI research proceeding to something that empowers people to participate in warfare with unprecedented capability and autonomy. "Programming" is a force multiplier, as is "lots of money."
Let's talk about specifics, because very few people counter that it's simply not possible, they counter that it's not a priority concern, and they do so aware of the things you said. I can't combat an argument if you don't provide a sufficient amount of precision as to be refutable, and I'm going to have to say that I don't really worry more about AI on an axiomatic basis, just as I don't really worry about the Sun exploding. It's possible, it will probably happen, but is it relevant and how does it happen?
I actually agree with the position that dangers of artificial intelligence are vastly overstated; that said, this is cherrypicking and doesn't really help the cause of reducing FUD, because it can be refuted with counter examples.
I'm normally a huge fan of Elon Musk, but that tweet comes across as pretty hostile to me. Neither party seems to have critically backed up their beliefs, but I didn't get the sense that Zuckerberg was attempting to single out any individual for a lack of understanding.
I don't think people like Musk or the others that he gets some of these ideas from, such as Kurzweil, are actually saying we shouldn't pursue advanced AI. Are they? I mean Musk helped fund OpenAI right?
He is just saying that it is actually an existential threat or will be soon enough that we need to consider the problem seriously.
I think the point of him mentioning that besides just worrying about it, is to motivate efforts like Neuralink that will allow for high-bandwidth communication between the brain and AIs.
There is some misunderstanding on the part of Zuckerberg and others that saying that AI is an existential risk is the same thing as saying 'we should suppress AI research'. You would not think that was the implication if you understood from the perspective of Kurzweil/Musk/other people who have read Kurzweil's/Bostrom's book(s) or think the same. The idea is that we cannot possibly stop advanced human-like and then human-surpassing AI from developing. No matter how hard we try to slow down AI development.
So when they say that AI can be dangerous, they do not mean that we should try to stop developing AI, since that is impossible. What they mean is that we should try to mitigate the risks to some degree. One way, which Musk and others are pursuing, is to try to create strong but friendly AI before some less friendly but equally potent variant becomes dominant. But if we do not develop some kind of high bandwidth interface with computers and AI that we control, eventually we should expect computer-based intelligence to significantly surpass our capabilities, to the point where we become irrelevant.
If you disagree that there is any possibility of strong AI or any existential threat, what harm is there in pursuing strong but friendly AI, or in developing the BCIs (brain-computer interfaces)? The only 'harm' I can see is if this gets oversimplified as it has been to the idea of "AIs could be bad, we must stop AI" -- which they are not trying to say.
Wasn't Elon's original point that he has current access to working AI demonstrations that personally frightened him, and served as the basis of his call for regulation?
Kinda though his arguments seem largely from first principles that robots will get smarter than us so we'd better be careful what they get up to. Here's him chatting a bit at "National Governors Association
Streamed live on Jul 15, 2017" https://www.youtube.com/watch?v=2C-A797y8dA&feature=youtu.be...
No, it's Elon Musk whose understanding of AI is limited.
I doubt Musk has ever actually built a neural network, or got close to coding one... because if he did, he would realize just how fragile and hard to generalize they are.
Stacking a ton of data together to build predictive models that can produce intelligent-like outputs, is a far, far cry from a robot gaining consciousness or a sense of agency.
With Musk talking about AI the way he is, seems to me that he wants to use his reputation and the public's naivety to instill fear and have some control over the distribution of AI that works for general purposes...sounds like a great opportunity.
If he stays consistent with what he says about AI, I think he might be successful in deterring people from using useful and reasonable AI for a while. I wish he would take the dangers of new features like Autopilot in a production car as seriously though.
> If he stays consistent with what he says about AI, I think he might be successful in deterring people from using useful and reasonable AI for a while.
What exactly did he say that would imply that? As far as I know (and I did watch all the recent interviews), his approach boils down to:
- more open access to AI is less dangerous than it being developed in secret by corporations or governments; hence OpenAI
- suggesting that regulators start paying attention (and explicitly advising them not to start regulating outright; just to form an agency that would "gain insight" into the field)
I do agree with this. One of the clearest inequalities emerging in A.I.-driven machine learning is how useless those techniques are unless you have access to a monstrously large amount of well categorized data. Currently you need to be one of the big four to have that access, and this is highly monopolistic.
Agreed, the big AI successes of the past decade like AlphaGo are specialised machines that only work in a specified domain. It's still a huge achievement, but I have yet to see any AI that warrants regulation anymore than any other automation technology.
Even if he is campaigning out of ignorance, it's still not a bad idea to start thinking about these topics a few decades early (assuming the type of AI he fears is an actual possibility).
At the very least it could help regulate some very real dangers, such as the misuse of highly effective specialized neural networks. (widespread surveillance, political manipulation, etc.)
The human brain actually is "magical", not in the supernatural sense but rather that it's so complex that we only understand how it works in a very superficial way.
Just because technology has been rapidly advancing recently doesn't mean it will continue to do so indefinitely. It's possible that innovation will follow an S-curve and eventually level out due to hitting natural limits. (I don't necessarily think that will happen, but we have to acknowledge it as a possibility.)
Advancements in intelligence and technology will push us towards a thorougher scientific understanding of the brain and the mechanics of intelligence itself.
There is no reason to believe that the brain so complex as to be impossible to understand. Not to mention that we may not need to have a completed neuroscience in order to develop some true intelligence (no matter how primitive) that can improve itself.
Like if neural networks were the be-all and end-all of AI.
> Stacking a ton of data together to build predictive models that can produce intelligent-like outputs, is a far, far cry from a robot gaining consciousness or a sense of agency.
That's, I think, you putting words into his mouth. I don't recall Musk ever talking much about AI consciousness or agency.
The way I understand it, Elon sides with the Yudkowsky/Bostrom reasoning of intelligence being an ability to perform extremely strong, cross-domain optimization. Which is orthogonal to a) consciousness as we know it, and b) ethical values as we know it - and that is precisely why it's considered a threat.
But how is that even possible? The best "AI" we have now are essentially just impressive exercises in data science, telling us in a surprisingly accurate ways things about the world that we already know.
Give it something completely strange, foreign and unknown, and it will acts orders of magnitude more irrational than the dumbest human.
One of the best AIs of recent times is AlphaGo which works well with strange Go moves and invents moves humans haven't thought of. Of course it's a few more steps to deal with the real world rather than Go or Atari games.
It's a huge leap. AlphaGo used data to train itself well against a game with extremely clear rules and outputs.
So while impressive, it's an example again of specialized AI. Real life is not at all like this, nor is what Google achieved clear that it can be a roadmap to anything like General AI.
AI doesn't need with consciousness or a sense of agency to be dangerous. AI doesn't need to be implemented entirely, or even partly, with neural networks either.
I'd highly recommend Bostrom's book Super Intelligence if you have the time and inclination to read a book that makes this argument more clearly.
That article just seemed to run out at the end, it seemed like there was some previous problem between the two related to the Shuttle? And then the article didn't describe it where I was reading.
This debate seems so premature. It's hard to take seriously when so far no one has even come close to building an AGI equivalent to a mouse, let alone a human.
Here is a broader set of perspectives from people working in the field.
If we have an AGI equivalent to a mouse, wouldn't it iterate itself to a human intelligence pretty rapidly? Safeguards and intelligent thinking now could be the difference between:
Mouse -> cat -> dolphin -> helpful friendly human -> better world
The end situation is highly theoretical, but a self-modifying AGI will think differently than a human does, so it's hard to predict what could happen with our existing preconditions. With such a broad range of potential divergence, and a steep difference in desirability of potential outcomes, it seems reasonable to ask some pointed questions sooner rather than later.
No there's no evidence to indicate that would happen. Mice have been around for a long time. How much smarter is a mouse today relative to a mouse a million years ago? And why would a simple mouse level AGI have the intelligence to improve its own abilities?
56 comments
[ 5.4 ms ] story [ 112 ms ] threadThat's only true if you're cherry picking. Plenty of tech doesn't even apply to the defense industry.
Moreover, I'd posit thay any limitation placed on tech to limit danger will almost certainly limit potential benefit as well.
Ignorance is never the solution.
Hypodermic needle
Fracking
Solar panels
This argument is a little pedantic, but here are some examples that contradict your assertion.
Consider this: without the bomb we would not have nuclear power. Without modern warfare we would not have modern medicine. There are other examples, but you would have been worse off limiting these. Not to mention non-technological benefits like geopolitical stability from MAD.
Better clothing for soldiers, tents, and more clothing for them. Plenty more I'm sure. Early war planes used canvas instead of metal as outside cover as they went out to fight each other.
> Hypodermic needle
Enhancement of battlefield medicine. Fix them faster so they can continue to fight.
> Fracking
Cheaper and continued source of fuel for war machinery
> Solar
Spy satellites, power for field communications. It's only the beginning.
Nuclear power is a perfect example of the dangers of technology. The world powers have decided to limit its spread because it's so dangerous to let everyone have it. If nuclear bomb tech was free to run its course they would be small, cheap and every country would have some. And most would not hesitate to use them.
Furthermore, to circle back to your original point: even if we accept that the cotton gin is a technology "used for war", it most certainly was not the first nor the primary use of that technology (the first use was clothing).
So from my view it not only aided the war once it started. But it was also a contributor to its beginning.
But whatever I say you'll find fault. You'll focus on the small details. I'm a techie too and I sometimes find myself arguing the smallest detail whether it has consequences or not. It's that type of attention to detail that keeps us employed when we need to make sure things are working.
The bottom line is that you think tech is great while I think it's great but we have to be careful with it's development and use.
All I can say is that it's worth being careful where we step.
[1] https://techcrunch.com/2017/07/19/this-famous-roboticist-doe...
Tech creators look at the bright side of their creations. They have tunnel vision and can't foresee the harm. If they can't others need to.
The criticism that Musk does not know enough about AI ignores the fact that he is trying to create a fully automatic car with AI and has experience with its limitations. Yes AI(a catch all process for making computers do things that require intelligence when done by humans) is a process and it has no intelligence in itself but you can say the same thing about C++. It's just a language but look at what's possible to do with it.
So what exactly are you proposing, and what precisely is your grievance? You're going to have to "think about a Level 5 autonomous vehicle" if you want to convince opponents of your position, because you'll need to come up with a more convincing argument than, "I don't want to think about how dangerous it could be."
I think its one way to explain what i felt about rouge AI, its not actually a bad AI, but a bad dataset used by AI.
1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws
[0]: https://en.wikipedia.org/wiki/Three_Laws_of_Robotics
If AI advances to the point of being able to determine the most optimal path to a solution, and the pre-programmed solution by a terrorist is murder everyone you can find - do you not see the danger that increasingly sophisticated technology poses?
Deciding what the solution to this problem is is non-trivial - but you have to be kidding me if you don't see that we're headed for a much scarier world when a disenfranchised person could give their AI robot the command to determine a plan to build the most devastating bomb possible, build the device, and go leave it in the optimally damaging place.
Hey, I could be wrong! I'm willing to be wrong, this is not my primary domain of expertise. But I remain unconvinced by arguments that are consequentialist without any reasoning about the specifics, because they are just as susceptible to be hysterical as they are to being valid.
Tell me exactly how you envision us getting from the current state of artificial intelligence to, "an arbitrary person to automate the design and deployment of a bomb, such that dangerous warfare is meaningfully democratized."
I understand you think this is possible; a lot of things are possible. I want to know how you think we can get from point A to point B. I don't want to hear something that devolves to, runaway or recursive AI self-improvement. I'm a technical person - if that's a component of your argument, I want to understand how exactly you see the current state of the art in AI research proceeding to something that empowers people to participate in warfare with unprecedented capability and autonomy. "Programming" is a force multiplier, as is "lots of money."
Let's talk about specifics, because very few people counter that it's simply not possible, they counter that it's not a priority concern, and they do so aware of the things you said. I can't combat an argument if you don't provide a sufficient amount of precision as to be refutable, and I'm going to have to say that I don't really worry more about AI on an axiomatic basis, just as I don't really worry about the Sun exploding. It's possible, it will probably happen, but is it relevant and how does it happen?
https://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-r...
He is just saying that it is actually an existential threat or will be soon enough that we need to consider the problem seriously.
I think the point of him mentioning that besides just worrying about it, is to motivate efforts like Neuralink that will allow for high-bandwidth communication between the brain and AIs.
There is some misunderstanding on the part of Zuckerberg and others that saying that AI is an existential risk is the same thing as saying 'we should suppress AI research'. You would not think that was the implication if you understood from the perspective of Kurzweil/Musk/other people who have read Kurzweil's/Bostrom's book(s) or think the same. The idea is that we cannot possibly stop advanced human-like and then human-surpassing AI from developing. No matter how hard we try to slow down AI development.
So when they say that AI can be dangerous, they do not mean that we should try to stop developing AI, since that is impossible. What they mean is that we should try to mitigate the risks to some degree. One way, which Musk and others are pursuing, is to try to create strong but friendly AI before some less friendly but equally potent variant becomes dominant. But if we do not develop some kind of high bandwidth interface with computers and AI that we control, eventually we should expect computer-based intelligence to significantly surpass our capabilities, to the point where we become irrelevant.
If you disagree that there is any possibility of strong AI or any existential threat, what harm is there in pursuing strong but friendly AI, or in developing the BCIs (brain-computer interfaces)? The only 'harm' I can see is if this gets oversimplified as it has been to the idea of "AIs could be bad, we must stop AI" -- which they are not trying to say.
I doubt Musk has ever actually built a neural network, or got close to coding one... because if he did, he would realize just how fragile and hard to generalize they are.
Stacking a ton of data together to build predictive models that can produce intelligent-like outputs, is a far, far cry from a robot gaining consciousness or a sense of agency.
If he stays consistent with what he says about AI, I think he might be successful in deterring people from using useful and reasonable AI for a while. I wish he would take the dangers of new features like Autopilot in a production car as seriously though.
What exactly did he say that would imply that? As far as I know (and I did watch all the recent interviews), his approach boils down to:
- more open access to AI is less dangerous than it being developed in secret by corporations or governments; hence OpenAI
- suggesting that regulators start paying attention (and explicitly advising them not to start regulating outright; just to form an agency that would "gain insight" into the field)
At the very least it could help regulate some very real dangers, such as the misuse of highly effective specialized neural networks. (widespread surveillance, political manipulation, etc.)
* Technology will keep advancing.
* There is nothing magical about the brain.
Just because technology has been rapidly advancing recently doesn't mean it will continue to do so indefinitely. It's possible that innovation will follow an S-curve and eventually level out due to hitting natural limits. (I don't necessarily think that will happen, but we have to acknowledge it as a possibility.)
Technology: I never said anything about the speed of growth. A minimal advancement would still be an improvement.
This makes it only a question of time.
Have you ever thought to consider that we are unable to understand our own brains due to being constrained exactly because of it?!
Advancements in intelligence and technology will push us towards a thorougher scientific understanding of the brain and the mechanics of intelligence itself.
There is no reason to believe that the brain so complex as to be impossible to understand. Not to mention that we may not need to have a completed neuroscience in order to develop some true intelligence (no matter how primitive) that can improve itself.
> Stacking a ton of data together to build predictive models that can produce intelligent-like outputs, is a far, far cry from a robot gaining consciousness or a sense of agency.
That's, I think, you putting words into his mouth. I don't recall Musk ever talking much about AI consciousness or agency.
The way I understand it, Elon sides with the Yudkowsky/Bostrom reasoning of intelligence being an ability to perform extremely strong, cross-domain optimization. Which is orthogonal to a) consciousness as we know it, and b) ethical values as we know it - and that is precisely why it's considered a threat.
Give it something completely strange, foreign and unknown, and it will acts orders of magnitude more irrational than the dumbest human.
So while impressive, it's an example again of specialized AI. Real life is not at all like this, nor is what Google achieved clear that it can be a roadmap to anything like General AI.
I'd highly recommend Bostrom's book Super Intelligence if you have the time and inclination to read a book that makes this argument more clearly.
[1] https://www.youtube.com/watch?v=npL_UsK_npE&t=285 [2] https://artificialintelligencenow.com/schedule/2017-symposiu...
Here is a broader set of perspectives from people working in the field.
http://spectrum.ieee.org/computing/software/humanlevel-ai-is...
Mouse -> cat -> shark -> Evil human mastermind -> problems
and:
Mouse -> cat -> dolphin -> helpful friendly human -> better world
The end situation is highly theoretical, but a self-modifying AGI will think differently than a human does, so it's hard to predict what could happen with our existing preconditions. With such a broad range of potential divergence, and a steep difference in desirability of potential outcomes, it seems reasonable to ask some pointed questions sooner rather than later.