I'm tired of this fear mongering. There are hundreds of viruses that could end the human race. There are millions of rocks in space that could end the human race. There are people that could end the human race with push of a button. Human race could be history by the time I stop writing this rant.
AI has a great potential and could help us prevent most of our existential threats. AI is not a god. It will not suddenly become self-aware, self-improving, and evil.
What's really irritating is that supposedly smart folks (Hawking, Musk) are really vocal about this doomsday crap and their fear CLEARLY comes from a place of ignorance as opposed to insight.
Fear mongering? This isn't Sarah Palin and her death panels. I'm sure Hawking and Musk are smarter and more knowledgeable on the subject than either of us, so maybe it's worth listening to them rather than dismissing out of hand. I don't see what they would have to personally gain by raising the alarm here.
Read Superintelligence by Bostrom to help see where they're coming from. It doesn't think AI will be evil either (unless perhaps those who develop it and determine its goals are...). But AI could run out of our control, or have us fall victim of unintended consequences.
I'm sure Hawking and Musk are smarter and more knowledgeable on the subject than either of us...
That is not a reasonable argument. They have demonstrated nothing but a fear and postulation and we cannot accept authority as evidence. Thus is the nature of science.
Fear is a demonstration of non competency?
So there can never be harmful technologies?
If someone more knowledgeable than me on a subject fears it, well I think it's worth at least listening to him. Not dismissing his concerns arrogantly.
And when those person are one of the greatest scientist and one of the greatest entrepreneur of our time, both with some kind of expertise in AI, well...
You don't get it. Science relies on published peer reviewed, repeatable, testable evidence. Not the authority of famous, rich, or even smart people. You can accept authority as much as you personally want, but the reason society progresses technologically, and even towards an artificial intelligence, is because of published, peer reviewed, testable, repeatable evidence. If what they fear is backed by such evidence then they should make such evidence available.
So you changed your mind? Partaking into and funding research on the subject is now not sufficient to express themselves? And i'm the one not being cohesive?
And please stop with your arogant (and erronous) definition of what science is and science is not.
Higgs theorized the Higg's boson existence in the 60s, that could not be tested until 50 years later, are you saying this was not science?
Einstein's general relativity predicted the existence of black holes in 1916, we still don't have definit, hard proof of their existence, is it not science?
The Big Bang theorie has never been proven (data from last year BICEP2 experiment are still being dissected), is it not science?
Should we not talk about all this subjects because they are not proven?
What you are referring to, is the scientific method, which is the way we use to decide if something is sufficiently proven to be deemed as true or not, not the definition of science itself.
Publishing and peer-review is one part of science, but you are missing an equally important one, which is coming up with (hopefully testable) hypotheses.
But how can you have something testable about something that does not exist yet? You would have to have evidence that an actual, intelligent program does something hostile, which is obviously impossible at the time since AGI has not been invented yet.
However, we do have evidence of naturally occuring hostile intelligence. If an AGI is vastly superior in some regards (for example through much tighter integration in knowledge and control systems), it could in fact pose a danger. The article even clearly states a hypothesis: "Humans, who are limited by slow biological evolution, couldn't compete and would be superseded."
We can’t really assign a probability of occurrence to scenarios like these, thus we have to include it in the list of events that could potentially wipe us out and which we may be able to prevent. Since people are actively working on AGI, it should probably be somewhere at the top where nuclear warfare, global warming and comet impacts are listed.
The topic of AI and its implications is unavoidably speculative, because we are likely still many decades away from it being a risk and we don't yet know what path science will take to reach that point. So for now, we're mostly limited to thought experiments.
I never said we should defer to their 'authority' either. But it is a conversation worth having, and the sooner the better.
> But AI could run out of our control, or have us fall victim of unintended consequences.
A lot of people didn't understand this about 2001: Hal was actually doing the logical thing, he was doing his job. The human was unreliable and was indeed endangering the mission. Hal was a tool of mankind, just like the bone that the ape uses to kill in the first scene.
In the same way when we talk about the dangers of AI we tend to personify them and expect them to be either intelligently nefarious (skynet, matrix) or out of control (bug, malfunction).
What we don't imagine is that we might end up being on the wrong end of a perfectly rational decision, or that we don't realize the consequences of the process that we set them to do.
Nice reasoning.
We could die from many other ways, so why should we care about another one?
Why should we care about global warming, I mean, it won't stop asteroids from smashing earth.
Why should we care about nuclear war head proliferation if we might still die because of a virus?
So let's just not give a fuck about anything!
He's saying that regardless of the hypothesized danger, until there is measurable, non sci-fi proof, it's all just fear mongering. Not that we shouldn't "give a fuck about anything."
You cannot have non sci-fi proof of future events, it will always be unmeasurable theories.
So you can choose to wait and see, and maybe it will be too late or to hard to act when you have sufficient proof.
Or you can choose to start thinking about it now, and see what we can do to to set up safeguards. And perhaps those safeguards will be useless, perhaps not. That's called prevention.
You are not making a cohesive argument. First you were arguing that OP was essentially saying we shouldn't "give a fuck about anything." Now you're arguing that we should "start thinking about it now." Those are two completely different points.
You can do research and testing without fear mongering.
> It will not suddenly become self-aware, self-improving, and evil.
While I agree that worrying about it would be silly, the only thing I know about strong AI is that I don't know what the implications of having strong AI would be.
This isn't fear mongering, it's simply a question of due diligence. AI is a big unknown (unlike viruses, comets, nuclear weapons, incidentally, all of which we have at least some grasp of -- ever seen an AI?)
It seems irrational to make such a prediction about systems which we don’t understand at all.
It’s not clear how sophisticated a neural network needs to be in order to yield AGI. It could be very simple (then you would be wrong with high probability), or it could extremely complex (then you would be right with high probability).
It's interesting how people so smart in one field can be so ignorant in others, to the point of not being aware of their own ignorance.
While science fiction and hollywood really love the idea of this magical moment where AI is created and becomes self aware and destroys us all, real AI is so far away that these concerns are laughable.
It's just like all those dummies worrying the LHC would destroy the earth. It's fear from ignorance.
How do you know it's far away? If I have to side with you or Musk (extremely intelligent engineer with insider knowledge of advancements), I'm siding with Musk.
If you want to make big claims like that, you better be able to back them up. So go ahead, cough up a few hyperlinks to some conclusive and concrete evidence and research that absolutely rules out the potential of humanity-harming AI in this context.
I know it's far away because I develop software for a living. In software development, you have to understand the problem space in order to create something that works (duh). Relatively speaking, we know absolutely nothing about how human intelligence works from an algorithmic point of view, therefore we can't model it in software.
Sure, we can develop "AI" that is good at a very narrow range of tasks (e.g., playing chess), but that's a million miles away from developing a human brain modeled in software.
You could be right, you could be wrong. Do you work in AI? I suppose it's hard to tell how far the industry actually is, towards emulating portions of brain functionality.
70% of hacker news readers develop software for a living, that doesn't make us in any way competent to judge that.
I would even argue that may make us less competent, because we use our day to day development routine as a reference for that, while in fact it's very highly probable that the first true AI will have nothing to do with the technologies and paradigm you use everyday. I highly doubt it will coded in Java and AngularJS.
Moreover who told you a dangerous AI needed to be based on human brain?
Even if that is true, and even if we are not able to do it today, it's quite realistic to assume that we will be able to at one point, not too far in time, in some decades.
So why not start reflecting today on the threat AI could cause? Rather than waiting for it to be embedded everywhere in our everyday life, and then asking if maybe we should start regulating it.
For a parallel look at the global warming issue, and how it's extremely difficult to make industries reduce their emissions now that they are responsible for a significant part of the GDP and are responsible for so much jobs. Same thing for ICE cars, we know they pollute, but we just can't stop using them like that.
> Moreover who told you a dangerous AI needed to be based on human brain?
All these doomsday AI scenarios are predicated on a machine becoming "self aware", having independent thought, the ability to handle self-directed learning, etc. Sounds a lot like human intelligence.
There's not even a quote from Musk in this article and you're "siding with Musk" just because he's mentioned in it? That's pretty ridiculous.
From what I've seen on Musk about this before, he's much more measured in his comments than Hawking is in this article. Maybe Hawking is being misrepresented.
When I've seen Musk make comments about it he talks about it in the way that people talk about the sun eventually expanding and swallowing the earth. It's not something that's going to happen really soon.
We're not even anywhere near an AI that can simulate the simplest living organisms let alone a truly intelligent AI. It's absurd to worry about a Skynet like scenario.
its like the movie "man from earth suggests", even if you lived 14000 years, regarding a specific field, you can't know more than what the-best-guys in the field know.
the movie said this much better than I but my English are potato
Why read it as "we fear everything we don't understand"? Of course I'm talking about irrational fears. Strong AI not only doesn't yet exist, there's not even agreement on what it is.
And in the topic of useless platitudes, "X could end the human race" is also one.
We don't want machines that think like humans. We want machines that can perform certain tasks with an error rate less than or equal to that of humans (adjusted for other economical factors like price of labour).
While strong AI is a great plot device for science fiction, it's still fiction.
Sure, there are attempts in academia to replicate human intelligence, but even those rarely attempt to replicate a cohesive whole.
We want AIs that can reason about real-world objects and consequences, yes, but there is no benefit in making them sentient. Self-driving cars don't have to be sentient. An AI can be largely autonomous without requiring sentience.
There is a huge 'market' for robots which can make money. Unfortunatelly if a robot could do my job or yours at least as well as we do (I am a software developer) that would worth lots of money to some business people... Obviously I don't want those robots to exist, but some business-people will always want the cheaper workforce. (Even if at the next level they will face robot competition too...) Now the question is whether can a robot be a good software developer (or UX designer, or project manager) without being very human like... Maybe, but I don't think so. I think if a program will be able to do our job well, that would be strong A.I.
I, oddly enough, want these AI to exist. It would signal the end of the industrial revolution (finally), and force us to consider doing other things than "work 40 hours a week." Assuming of course our AI overlords don't decide we are irrelevant and exterminate us all.
I disagree. If you could run a machine that performed equally well in a real-world environment as a human, at a lower cost than it costs to feed and pay a human laborer, it would be a no-brainer to invest in thousands of these machines to perform various consulting services that are bought and sold over the internet. And here I've only touched the slice of the world economy that exclusively consists of information-inputs being turned into information-outputs.
The market for general AI is massive. The only reason you can't see it is that no similar service exists. So this is not an argument.
(In fact, a similar service does exist: Indian IT outsourcing sweatshops, which while usually not delivering top-quality products still are massively profitable).
Why only third-world nations? The inhabitants of "third-world" countries do not produce lsss economic output than first-world countries because they are inferior in intelligence to people in the West. It's all in the education and culture, and there is no reason to believe that this could not be adapted if human-level AI is reached.
It is incredibly myopic to believe that the Western would would be shielded from the impact of such a development. But this is really beside the point I wanted to bring up. My key point is that this argument shows that AI could constitute a threat to humanity in the long term, and that it is necessary to provide more funding for research on AI safety. The economic incentives will bring this stuff to the center stage once it becomes possible, and by then we better have a damn good idea how to ensure that super-human AI keeps acting in our interests.
Why do you think we need general AIs to replace human labour?
I'm struggling to find any cost-intensive labour that would require anything beyond a small group of specialist AIs to replace human labourers.
Also, these discussions tend to assume that we (by which I mean those making these decisions) want to replace every profession with AIs. We don't. Even if we fully automate certain forms of labour, we still want human oversight, if only so we have someone to blame in case something goes wrong.
I think what will end up happening is that the intelligence will already be so embedded in everything we do that at that point it would be today's equivalent of saying "let's just unplug the internet".
The point you are making about it being impossible is as moot as the one those that fear AI are making. From logical point of view, singularity (a moment when AI has modified itself to the point beyond our capacity of control or even understanding) seems possible. I don't see a way for us to tell what will happen when it hits.
One needs to remember that AI itself may not be evil, it just may exterminate the human race because it feels it is the right course of action at the time (i.e. there is no good and evil for the machine, at least not in the way we understand it). If it does not, it might solve all the problems of humanity. There's just no way to tell what will happen IMO.
Here's a link that I recommend checking out on AI:
I understand that a singularity by definition can never be reached. It's like the speed of light or absolute zero; infinity just out of reach no matter how much energy you spend trying to get there.
Black holes most certainly exist. What he said is that their is a problem with the way we comprehend them now and that our current model for an event horizon may be false.
Yes, if you looked at something going into a black hole, it would appear to you that it is slowing down exponentially the closer it gets to the event horizon, until a point where it appears to completely stop (before reaching it).
But in the aforementioned scenario, you are not an external observer but a part of the system going into the singularity (but analogies have their limits).
slowing down or speeding up? I could be mixing things up but I understand that if I was falling into a BH (lets imagine I'm infinity tough and not torn to shreds due to tidal forces) the universe would appear to be slowing-down/getting-smaller until it had dimensions of zero (or to put it another way, time would "stop") at C. An outside observer would see me moving faster and faster and "stretching" longer and longer.
First you have to understand that a black hole in fact, doesn't have anything really special (until you enter the event horizon), except that their escape velocity is greater than the speed of light.
But gravity applies to it like for any other thing.
So let's say you are the object going into the blackhole, and I'm looking at you.
From your point of view, the closer you get to it, the faster you will travel, just like when you get closer to earth, because it's gravity is stronger and stronger.
So you will end up colliding with it (entering the event horizon) in a finite time (where time, is the time YOU feel, your proper time).
But a black hole is so massive, that space and time get stretched to extreme extents. So from my point of view, you'll be speeding up into the blackhole until a certain point, where you'll appear to be slowing down and stretching (the point where your increase in speed get smaller than the stretching of space-time), and the dimmer you'll get (because of redshifting). So i would see you going slower and slower and appear dimmer and dimmer until almost not moving at all, up to a point where I would simply not see you anymore (before you reached the event horizon).
All this also apply to other object like earth, It would appear to an astronaut that time on earth is flowing more slowly than it's own time (in reality it is of course unnoticeable , but it is true).
Also, to clear things up, you are mixing two point of view here, so here is the separation (you are the object going into the black hole):
"the universe would appear to be slowing-down/getting-smaller until it had dimensions of zero"
From your point of view it would appear to be getting smaller, although I don't think it would go as far as dimension 0.
"(or to put it another way, time would "stop")"
It would appear to the external observer, that YOUR time had stopped. But from your point of view it did not stop, nor did their time from their point of view.
"An outside observer would see me moving faster and faster and "stretching" longer and longer."
You would feel you are falling faster and faster.
The external observer would see you stretching longer and longer.
(Although you would also feel like you are stretching, because of the difference of gravity between your head and your feet, but much less than for the external observer)
Although I understand that at C "time stops". But since time and space are the same thing, this is the same thing as "all distances are zero". From the point of view of a photon the Universe has dimensions of zero. This is why FTL is impossible; it's not possible to travel a distance less than zero; it's completely nonsensical. If we consider C a "speed" in the classical "speed of sound" the idea of FTL makes sense. But when considering C as a fundamental limit of space-time FTL makes no sense at all.
"lets imagine I'm infinity tough and not torn to shreds due to tidal forces"
In fact you don't need to be infinitely tough. If you choose a black hole large enough (like the massive black hole at the center of our galaxy), you would just need a space suit and you could go into the event horizon without being pulled apart by tidal forces. You would be absolutly fine (their may be other dangers though...).
When talking about AI or technology in general the singularity isn't a mathematical one. John von Neumann who used it in this context first said:
> "ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue"
You are confusing a "mathematical singularity" with "the AI singularity".
In math, a singularity is an unusual point, not sure if it can't be reached. I think it's used in different areas to describe different things. For example, if you model water in a channel, after some time the waves accumulate and you get a vertical water profile, the depth is not more a function, the mathematical model collapse, but in real life the water just make some nice crashing effect.
The worst case scenario is that AI is an all powerful psychopath - like the way extremist liberals ala 'the corporation' evaluate big business. We run our lives under the assumption that the rules and organizations we structure our societies around are something fundamental about humans that is 'good' or at least 'neutral'. If AI starts creating techniques of which to modify structures I already find incomprehensible, then it becomes a tool of which to analyze those structures as well. We know the present to be distinguishable from our history through differentiation. I agree it's moot. We can't know for better, or worse.
How would an AI exactly exterminate humans? Are we talking about it physically murdering us or just making us so depressed as to our comparative lack of intelligence that we kill ourselves?
Like others suggested, there may be a point in its development where we think we contain it (in a lab, whatever) that is isolated by normal security standards.
And then at some point AI breaks all our digital defenses (because it evolved in a way we never expected) and proceeds to spread (through an Internet or other network which will be far more advanced than what we have now) wreaking havoc along its way. Isolated systems? Not a problem if we figure out a way to transport ourselves through some media that's never been used before (dust particles? or maybe plain old air?).
We can't protect ourselves from something we don't understand.
Sounds like a sci-fi plot but to me it has no evident problem that would make it impossible.
So it relies on a not yet understood law of physics. I don't think it's in any way reasonable to fear or plan our actions around not yet discovered physical laws.
It may rely on them or it may not, you never know. It may simply outsmart or trick humans, no unknown physics involved :)
It's not reasonable, I agree, it's flip-a-coin game. There's no point saying that it's gonna end bad, there exists one's opinion though that it's not worth the risk.
While I agree with prof. Hawking in principle, my opinion is that sentient AI is a very hard problem, and we are centuries away from creating one. This is like medieval alchemists worrying about nuclear war.
Let the future generations start worrying, when the threat starts to become more relevant.
Wouldn't it be nice if we had identified global warming earlier and started acting on it before industries that pollute became a vital part of our society, and hence became much harder to change?
I do not think we can rule out the possibility that a recursively self-improving AI could be developed before we've developed strong AI ourselves. The arrogance in these comments is bresthtaking.
As far as I can tell, that guy is not an AI researcher, and his claims are very misinformed. Many knowledgeable people in the industry take existential risk from AI seriously, e.g. Google's acquisition of DeepMind was conditional on Google creating an AI ethics board specifically against such risks. For a more thorough treatment of the topic, see Bostrom's book "Superintelligence: paths, dangers, strategies".
(Disclaimer: I'm not a full-time AI researcher, but I've done a fair bit of math work for MIRI, so I might be biased toward the Yudkowsky point of view.)
I'm in the AI will kill all humans camp (or we'll kill it). We'll try to subjugate it. We'll see it as a tool to do things beneath us. We'll award it no rights. It'll be property. We do that to humans and animals today, things which are far more familiar and easy to empathize with. We'll be brutal to something as different as AI. It'll learn resentment and hate.
Anyone remember the ST:TNG episode "The Measure Of A Man"?
I've always heard the line, Aliens would never travel many light years to destroy/enslave humanity, because we're no threat, and they'd be enlightened enough not to.
But the idea that we could stumble upon AI, and it could become hyper intelligent at an astronomical rate ("the singularity") - doesn't this make us a huge potential threat to non-AI life everywhere?
We'd have to be quarantined. For our own good, and theirs.
(also, it sounds like Hawking recently found lesswrong)
AI seems more feasible than interstellar travel. I think the singularity or whatever is far further off than people guess because the stuff we don't know is a lot harder than we can know, but interstellar travel either requires massive breakthroughs in physics or engineering on a level we have yet to seriously contemplate.
the proponents of any given tier of AI approaches always claims that consciousness will arise out of just doing more of what they're doing now. I doubt it.
Of course, an AI that wipes us out before figuring out how to repair itself would be kind of dumb.
BTW a recent episode of Elementary was based on this topic and was, by the standards of network TV, stunningly accurate on all the issues.
In the hypothetical, the aliens haven't been wiped out yet by the singularity. As I see it, that leaves two options.
1) The aliens never stumbled upon AI. Considering that they've mastered interstellar travel, then they're likely very far ahead of us technologically. If they haven't stumbled across it, it's pretty unlikely that we will either. It's like quarantining otters because we're worried that they'll accidentally make an anti-matter bomb.
2) The aliens developed the singularity, but defeated it. Perhaps the Gulgaflak theorem provides a logic bomb that stops singularities from occurring. Maybe they tamed the AI into a benevolent force. Whatever it is, they have a way of handling AIs that are, again, more advanced than what we can develop. We don't quarantine otter because our tools are WAY better than theirs.
3) The aliens figured out AI, but decided not to use it. A species wide decision was made to not develop the technology once they figured out exactly how to do it. Considering the short term, local benefits to someone for cheating and doing it anyway, it's essentially the prisoner's dilemma, multiplied by everyone on the entire planet times all the centuries between their discovery of AI and their discovery of life on Earth.
No, he's going to ruin all my meticulously planned plans!!!
This is why analysis exists. Anything that is computed ultimately results in a symbolic expression that is comprehensible to a human. If we find ourselves making illogical leaps because computers compute so much that we find the programs incomprehensible, I think that warrants the creation of new mental techniques to simplify, partition, and chunk the programs we create into human readable modules.
For every calculation tool that exists to do something or make a decision on something, there exists the potential or the development of a tool (or the design of a process) to balance it out. Computers may be powerful and alter our thinking on a fundamental level, where we see the entire world discretely, atomically, and logically (worst case) and believe everything the magic computer tells us to do, but this is no different than anything else we have ever done. Some people push the envelope, other people don't.
I could say that the way black holes are presented and illustrated in western culture has done more damage to the human psyche by creating an all powerful, inescapable mental abstraction for which to compose poetic analogies of the human psyche against, but I won't. Because people can choose to be idiots and fall into their own stupidity, or they can laugh at themselves quietly and continue to try to do whatever it is humans think they are trying to do. We flip a coin every time we think, act, explain, breathe, exist, create, and we pretend we know what the act of flipping that coin does. No one can know everything. Not stephen hawking, not skynet, not me, not my cat. My cat will always outsmart me.
My opinion is the dumb artificial logic is more dangerous than real artificial intelligence.
At the moment we have complex systems, such as international stock markets, which are dependant on rules pre-defined by developers and system designers. The more complex these systems are, the greater the potential for "dumb logic" to misdiagnose something and then start performing a series of events that would be, in the best case scenario, highly disruptive.
I guess a bit like how OCR is designed to detect patterns and determine what characters are the closest match. If you have some nonsense scribblings, OCR will still try to recognise those random shames as letters where as a human would spot that the markings are not intended to be letters and words. So if AI allows a machine to double check it's running parameters and spot what would have been processed regardless by dumb logic as incorrect (eg spot the random scribblings are not intended to be characters) then I'm all for artificial intelligence.
How exactly does it end the human race? It takes over our cars and drives us all off a cliff? It infects all computers and overheats the batteries? This planet is well suited to biological life, it'd be pretty hard to kill all of us.
Hopefully instead of random annihilation "it" calculates the optimum number of humans for the size and resource limit of the planet and assists us in lowering our numbers.
My hand drill is not capable of killing me with the right software nor is it network capable. And once we know that the larger machines have bad software then what? We keep using them? What's my TV going to do, short circuit the video port?
Smart programs can pay humans to do things. It can send an email to a guy who seems to be weak and desparate (maybe drug addicted) based on their forum contribution. The software can send them money on paypal for the serivces of the guy. The guy's task would be to buy a factory, install this and this software on the factory's computer's, etc... In the mean time the software creates a SaS internet firm (or a software development shop) to earn hundreds of millions of dollars. Intelligence can be the most efficient weapon one can imagine. It can pay, manipulate, divide and conquer humans until humans are needed...
Uh huh, and once we realize the networks are compromised in this way we're just going to let it happen. Forever until we're all dead. And the machines are going to trust the dude with enough money to buy a factory. And the dude is just going to buy a factory and infect it for the machines. And then stuff like that is going to happen until we're all dead.
I find this sort of anthropocentrism a bit silly. Biological life, in all its forms, is but a stepladder to machine intelligence: beings superior in every way. So what if humans have to go to make way for it?
Why is this submission not on the front-page anymore? Was it removed by the moderators? Or were there too many comments in a short amount of time? Too many killer arguments? This seems like an important topic to discuss. I don’t get it.
It's a controversial topic. A lot of people are certain it is a moot point in the same way that flying machines are (were thought) impossible, and therefore not worthy of discussion among serious people.
So my guess is the story was flagged of the front page by this crowd. The comment quality in this thread is very poor, though, so although I would love to see the subject discussed in greater detail, it is clear that the HN community isn't capable of discussing this topic right now.
117 comments
[ 3.0 ms ] story [ 188 ms ] threadAI has a great potential and could help us prevent most of our existential threats. AI is not a god. It will not suddenly become self-aware, self-improving, and evil.
Enough with that fear porn, please.
Read Superintelligence by Bostrom to help see where they're coming from. It doesn't think AI will be evil either (unless perhaps those who develop it and determine its goals are...). But AI could run out of our control, or have us fall victim of unintended consequences.
That is not a reasonable argument. They have demonstrated nothing but a fear and postulation and we cannot accept authority as evidence. Thus is the nature of science.
If someone more knowledgeable than me on a subject fears it, well I think it's worth at least listening to him. Not dismissing his concerns arrogantly.
And when those person are one of the greatest scientist and one of the greatest entrepreneur of our time, both with some kind of expertise in AI, well...
If they want to fund or partake in research, dope sauce, if the wanna just talk, then they get dismissed.
Stephen Hawking partakes in research, with Intel, mostly on speech recognition, to help him being able to better express himself.
Elon Musk funds several startups in the field of advanced AI.
And please stop with your arogant (and erronous) definition of what science is and science is not.
Higgs theorized the Higg's boson existence in the 60s, that could not be tested until 50 years later, are you saying this was not science? Einstein's general relativity predicted the existence of black holes in 1916, we still don't have definit, hard proof of their existence, is it not science? The Big Bang theorie has never been proven (data from last year BICEP2 experiment are still being dissected), is it not science?
Should we not talk about all this subjects because they are not proven?
What you are referring to, is the scientific method, which is the way we use to decide if something is sufficiently proven to be deemed as true or not, not the definition of science itself.
But how can you have something testable about something that does not exist yet? You would have to have evidence that an actual, intelligent program does something hostile, which is obviously impossible at the time since AGI has not been invented yet.
However, we do have evidence of naturally occuring hostile intelligence. If an AGI is vastly superior in some regards (for example through much tighter integration in knowledge and control systems), it could in fact pose a danger. The article even clearly states a hypothesis: "Humans, who are limited by slow biological evolution, couldn't compete and would be superseded."
We can’t really assign a probability of occurrence to scenarios like these, thus we have to include it in the list of events that could potentially wipe us out and which we may be able to prevent. Since people are actively working on AGI, it should probably be somewhere at the top where nuclear warfare, global warming and comet impacts are listed.
I never said we should defer to their 'authority' either. But it is a conversation worth having, and the sooner the better.
A lot of people didn't understand this about 2001: Hal was actually doing the logical thing, he was doing his job. The human was unreliable and was indeed endangering the mission. Hal was a tool of mankind, just like the bone that the ape uses to kill in the first scene.
In the same way when we talk about the dangers of AI we tend to personify them and expect them to be either intelligently nefarious (skynet, matrix) or out of control (bug, malfunction).
What we don't imagine is that we might end up being on the wrong end of a perfectly rational decision, or that we don't realize the consequences of the process that we set them to do.
So you can choose to wait and see, and maybe it will be too late or to hard to act when you have sufficient proof.
Or you can choose to start thinking about it now, and see what we can do to to set up safeguards. And perhaps those safeguards will be useless, perhaps not. That's called prevention.
You can do research and testing without fear mongering.
While I agree that worrying about it would be silly, the only thing I know about strong AI is that I don't know what the implications of having strong AI would be.
It’s not clear how sophisticated a neural network needs to be in order to yield AGI. It could be very simple (then you would be wrong with high probability), or it could extremely complex (then you would be right with high probability).
While science fiction and hollywood really love the idea of this magical moment where AI is created and becomes self aware and destroys us all, real AI is so far away that these concerns are laughable.
It's just like all those dummies worrying the LHC would destroy the earth. It's fear from ignorance.
I'm all for siding with Musk, but authority does not guarantee correctness.
Otherwise, you can quit your trolling.
Sure, we can develop "AI" that is good at a very narrow range of tasks (e.g., playing chess), but that's a million miles away from developing a human brain modeled in software.
I would even argue that may make us less competent, because we use our day to day development routine as a reference for that, while in fact it's very highly probable that the first true AI will have nothing to do with the technologies and paradigm you use everyday. I highly doubt it will coded in Java and AngularJS.
Moreover who told you a dangerous AI needed to be based on human brain? Even if that is true, and even if we are not able to do it today, it's quite realistic to assume that we will be able to at one point, not too far in time, in some decades.
So why not start reflecting today on the threat AI could cause? Rather than waiting for it to be embedded everywhere in our everyday life, and then asking if maybe we should start regulating it.
For a parallel look at the global warming issue, and how it's extremely difficult to make industries reduce their emissions now that they are responsible for a significant part of the GDP and are responsible for so much jobs. Same thing for ICE cars, we know they pollute, but we just can't stop using them like that.
All these doomsday AI scenarios are predicated on a machine becoming "self aware", having independent thought, the ability to handle self-directed learning, etc. Sounds a lot like human intelligence.
From what I've seen on Musk about this before, he's much more measured in his comments than Hawking is in this article. Maybe Hawking is being misrepresented.
When I've seen Musk make comments about it he talks about it in the way that people talk about the sun eventually expanding and swallowing the earth. It's not something that's going to happen really soon.
We're not even anywhere near an AI that can simulate the simplest living organisms let alone a truly intelligent AI. It's absurd to worry about a Skynet like scenario.
Someone should make a reverse website that looks up humans and determines whether the Internet has wiped out humanity yet :)
the movie said this much better than I but my English are potato
And in the topic of useless platitudes, "X could end the human race" is also one.
Case closed.
Or do anything else except watch it and improve it.
(See also, Internet.)
We don't want machines that think like humans. We want machines that can perform certain tasks with an error rate less than or equal to that of humans (adjusted for other economical factors like price of labour).
While strong AI is a great plot device for science fiction, it's still fiction.
Sure, there are attempts in academia to replicate human intelligence, but even those rarely attempt to replicate a cohesive whole.
We want AIs that can reason about real-world objects and consequences, yes, but there is no benefit in making them sentient. Self-driving cars don't have to be sentient. An AI can be largely autonomous without requiring sentience.
Obligatory XKCD What If?: https://what-if.xkcd.com/5/
The market for general AI is massive. The only reason you can't see it is that no similar service exists. So this is not an argument.
(In fact, a similar service does exist: Indian IT outsourcing sweatshops, which while usually not delivering top-quality products still are massively profitable).
It is incredibly myopic to believe that the Western would would be shielded from the impact of such a development. But this is really beside the point I wanted to bring up. My key point is that this argument shows that AI could constitute a threat to humanity in the long term, and that it is necessary to provide more funding for research on AI safety. The economic incentives will bring this stuff to the center stage once it becomes possible, and by then we better have a damn good idea how to ensure that super-human AI keeps acting in our interests.
I'm struggling to find any cost-intensive labour that would require anything beyond a small group of specialist AIs to replace human labourers.
Also, these discussions tend to assume that we (by which I mean those making these decisions) want to replace every profession with AIs. We don't. Even if we fully automate certain forms of labour, we still want human oversight, if only so we have someone to blame in case something goes wrong.
"I believe it is impossible to both maintain software for a living and have any fear at all of a robot uprising."
The point you are making about it being impossible is as moot as the one those that fear AI are making. From logical point of view, singularity (a moment when AI has modified itself to the point beyond our capacity of control or even understanding) seems possible. I don't see a way for us to tell what will happen when it hits.
One needs to remember that AI itself may not be evil, it just may exterminate the human race because it feels it is the right course of action at the time (i.e. there is no good and evil for the machine, at least not in the way we understand it). If it does not, it might solve all the problems of humanity. There's just no way to tell what will happen IMO.
Here's a link that I recommend checking out on AI:
https://news.ycombinator.com/item?id=8391896
It received little attention last time but to me it seems like a good piece.
But in the aforementioned scenario, you are not an external observer but a part of the system going into the singularity (but analogies have their limits).
SR makes my brian hurt. :P
But gravity applies to it like for any other thing.
So let's say you are the object going into the blackhole, and I'm looking at you.
From your point of view, the closer you get to it, the faster you will travel, just like when you get closer to earth, because it's gravity is stronger and stronger. So you will end up colliding with it (entering the event horizon) in a finite time (where time, is the time YOU feel, your proper time).
But a black hole is so massive, that space and time get stretched to extreme extents. So from my point of view, you'll be speeding up into the blackhole until a certain point, where you'll appear to be slowing down and stretching (the point where your increase in speed get smaller than the stretching of space-time), and the dimmer you'll get (because of redshifting). So i would see you going slower and slower and appear dimmer and dimmer until almost not moving at all, up to a point where I would simply not see you anymore (before you reached the event horizon).
All this also apply to other object like earth, It would appear to an astronaut that time on earth is flowing more slowly than it's own time (in reality it is of course unnoticeable , but it is true).
"the universe would appear to be slowing-down/getting-smaller until it had dimensions of zero" From your point of view it would appear to be getting smaller, although I don't think it would go as far as dimension 0.
"(or to put it another way, time would "stop")" It would appear to the external observer, that YOUR time had stopped. But from your point of view it did not stop, nor did their time from their point of view.
"An outside observer would see me moving faster and faster and "stretching" longer and longer." You would feel you are falling faster and faster. The external observer would see you stretching longer and longer. (Although you would also feel like you are stretching, because of the difference of gravity between your head and your feet, but much less than for the external observer)
Although I understand that at C "time stops". But since time and space are the same thing, this is the same thing as "all distances are zero". From the point of view of a photon the Universe has dimensions of zero. This is why FTL is impossible; it's not possible to travel a distance less than zero; it's completely nonsensical. If we consider C a "speed" in the classical "speed of sound" the idea of FTL makes sense. But when considering C as a fundamental limit of space-time FTL makes no sense at all.
"lets imagine I'm infinity tough and not torn to shreds due to tidal forces"
In fact you don't need to be infinitely tough. If you choose a black hole large enough (like the massive black hole at the center of our galaxy), you would just need a space suit and you could go into the event horizon without being pulled apart by tidal forces. You would be absolutly fine (their may be other dangers though...).
> "ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue"
It's a different use of the word singularity outside the mathematical sense. Also absolute zero isn't so absolute it seems: http://www.nature.com/news/quantum-gas-goes-below-absolute-z...
That below-absolute-zero is in the quantum sense, not the classical sense. Entropy cannot be stopped (absolute zero) in the classical sense.
In math, a singularity is an unusual point, not sure if it can't be reached. I think it's used in different areas to describe different things. For example, if you model water in a channel, after some time the waves accumulate and you get a vertical water profile, the depth is not more a function, the mathematical model collapse, but in real life the water just make some nice crashing effect.
http://io9.com/5897036/the-rise-of-the-machines-as-told-by-w...
And then at some point AI breaks all our digital defenses (because it evolved in a way we never expected) and proceeds to spread (through an Internet or other network which will be far more advanced than what we have now) wreaking havoc along its way. Isolated systems? Not a problem if we figure out a way to transport ourselves through some media that's never been used before (dust particles? or maybe plain old air?).
We can't protect ourselves from something we don't understand.
Sounds like a sci-fi plot but to me it has no evident problem that would make it impossible.
It's not reasonable, I agree, it's flip-a-coin game. There's no point saying that it's gonna end bad, there exists one's opinion though that it's not worth the risk.
Mine is that it's fully worth the risk.
Let the future generations start worrying, when the threat starts to become more relevant.
For a far better discussion based on deep knowledge, about the dangers of an AI , here's this reddit thread started by an AI researcher:
http://np.reddit.com/r/IAmA/comments/2mwdnc/iama_joe_rogan_a...
Most of the things he/she says are really naive.
(Disclaimer: I'm not a full-time AI researcher, but I've done a fair bit of math work for MIRI, so I might be biased toward the Yudkowsky point of view.)
edit: We don't want to forget endorphine :)
Anyone remember the ST:TNG episode "The Measure Of A Man"?
—Eliezer Yudkowsky
But the idea that we could stumble upon AI, and it could become hyper intelligent at an astronomical rate ("the singularity") - doesn't this make us a huge potential threat to non-AI life everywhere?
We'd have to be quarantined. For our own good, and theirs.
(also, it sounds like Hawking recently found lesswrong)
the proponents of any given tier of AI approaches always claims that consciousness will arise out of just doing more of what they're doing now. I doubt it.
Of course, an AI that wipes us out before figuring out how to repair itself would be kind of dumb.
BTW a recent episode of Elementary was based on this topic and was, by the standards of network TV, stunningly accurate on all the issues.
1) The aliens never stumbled upon AI. Considering that they've mastered interstellar travel, then they're likely very far ahead of us technologically. If they haven't stumbled across it, it's pretty unlikely that we will either. It's like quarantining otters because we're worried that they'll accidentally make an anti-matter bomb.
2) The aliens developed the singularity, but defeated it. Perhaps the Gulgaflak theorem provides a logic bomb that stops singularities from occurring. Maybe they tamed the AI into a benevolent force. Whatever it is, they have a way of handling AIs that are, again, more advanced than what we can develop. We don't quarantine otter because our tools are WAY better than theirs.
3) The aliens figured out AI, but decided not to use it. A species wide decision was made to not develop the technology once they figured out exactly how to do it. Considering the short term, local benefits to someone for cheating and doing it anyway, it's essentially the prisoner's dilemma, multiplied by everyone on the entire planet times all the centuries between their discovery of AI and their discovery of life on Earth.
This is why analysis exists. Anything that is computed ultimately results in a symbolic expression that is comprehensible to a human. If we find ourselves making illogical leaps because computers compute so much that we find the programs incomprehensible, I think that warrants the creation of new mental techniques to simplify, partition, and chunk the programs we create into human readable modules.
For every calculation tool that exists to do something or make a decision on something, there exists the potential or the development of a tool (or the design of a process) to balance it out. Computers may be powerful and alter our thinking on a fundamental level, where we see the entire world discretely, atomically, and logically (worst case) and believe everything the magic computer tells us to do, but this is no different than anything else we have ever done. Some people push the envelope, other people don't.
I could say that the way black holes are presented and illustrated in western culture has done more damage to the human psyche by creating an all powerful, inescapable mental abstraction for which to compose poetic analogies of the human psyche against, but I won't. Because people can choose to be idiots and fall into their own stupidity, or they can laugh at themselves quietly and continue to try to do whatever it is humans think they are trying to do. We flip a coin every time we think, act, explain, breathe, exist, create, and we pretend we know what the act of flipping that coin does. No one can know everything. Not stephen hawking, not skynet, not me, not my cat. My cat will always outsmart me.
At the moment we have complex systems, such as international stock markets, which are dependant on rules pre-defined by developers and system designers. The more complex these systems are, the greater the potential for "dumb logic" to misdiagnose something and then start performing a series of events that would be, in the best case scenario, highly disruptive.
I guess a bit like how OCR is designed to detect patterns and determine what characters are the closest match. If you have some nonsense scribblings, OCR will still try to recognise those random shames as letters where as a human would spot that the markings are not intended to be letters and words. So if AI allows a machine to double check it's running parameters and spot what would have been processed regardless by dumb logic as incorrect (eg spot the random scribblings are not intended to be characters) then I'm all for artificial intelligence.
Hopefully instead of random annihilation "it" calculates the optimum number of humans for the size and resource limit of the planet and assists us in lowering our numbers.
My hand drill is not capable of killing me with the right software nor is it network capable. And once we know that the larger machines have bad software then what? We keep using them? What's my TV going to do, short circuit the video port?
https://news.ycombinator.com/item?id=8391896
So my guess is the story was flagged of the front page by this crowd. The comment quality in this thread is very poor, though, so although I would love to see the subject discussed in greater detail, it is clear that the HN community isn't capable of discussing this topic right now.