"Welcome to the Desert of the Real. We have only bits and pieces of information but what we know for certain is that at some point in the early twenty-first century all of mankind was united in celebration. We marveled at our own magnificence as we gave birth to AI.”[1]
A happy narrative, but an unlikely one. Our brains are not designed to be end-user modifiable. It's naive in the extreme to think we could keep pace (in terms of intellectual enhancement) with minds that are.
They are evolved to be evolvable to some extent. The difference between a mouse and a human at the DNA level is not that great. That suggests significant mutability if we can find the right levers.
We don't even need to evolve them - finding a way to quickly cram in lots of knowledge is more than enough. Also, if we could take one geniuses knowledge and transplant it to another human's brain (no crazier than transferring conscience to a computer), that would lead to amazing advances (not to mention that it would solve the problem of people not understanding each other).
Choice quote from the end: But it was alarming how many people I talked to who are highly placed people in AI who have retreats that are sort of ‘bug out’ houses” to which they could flee if it all hits the fan.
Does HN agree or disagree a significant number of those "bug out" people truly exist?
Certainly they do, those sneaky "highly placed people in AI" know perfectly well that it will be quite easy to hide from the strong AI in some rural area. Its not like AI could access real estate databases or build some flying droids to find those people. /s
One of my favorite hard takeoff scenarios: the AI quickly figures out how to manufacture nanobots and does so at a scale to blanket the planet, thus becoming instantly omniscient with a global field of sensors, giving it a coherent view of everything in the world at every given instant in time from that point forward.
Also see: "metamorphosis of prime intellect" and "mind war: the singularity" and to a lesser extent "a young lady's illustrated primer"
I always think of the Grey Goo theory when the subject of nanobots taking over Earth comes up.
"a hypothetical end-of-the-world scenario involving molecular nanotechnology in which out-of-control self-replicating robots consume all matter on Earth while building more of themselves"
I suspect this is an exaggeration of talking to some high-level eccentrics that find their way into academia, especially in the computer science department.
I'm dumber than a high level AI of fiction, and I could find your little cabin or panic room easily. A machine with drones, access to tax/real estate records, etc would have no problem here. Heck, why not just hire a mercenary group to fish these guys out of their little holes? AI money is as good as human money.
exaggeration of talking to some high-level eccentrics that find their way into academia
Well, he does spend most of the article gushing over the magic insight of Kurzweil. It's really a "baby's first exposure to the concept of self-advancing AI" article with a movie promo thrown in at the end. (Maybe? I just skimmed.)
Doubt it. I would imagine a lot like to talk about it, or refer to their second/vacation home as such. Claiming to have a remote house for when "shit hits the fan" for most means a regular home/house in a less densely populated area than their primary residence. I wouldn't expect these homes to be built with any sort of defenses or preparations for a true apocalypse.
I visit rural Maine frequently and often hear residents talk about their guns, and their "readiness" for world ending events. Yet most have done nothing to actually prepare, beyond living remotely and stock up on some guns.
Some counter-points on this narrative, with which I'm very familiar.
1. Philosophers and AI theoreticians have a hard time defining what they mean by the term "intelligence". Talking about systems millions of times more intelligent than a human is nonsense unless you can define what it is you're talking about.
2. Whatever intelligence is it only makes any sense within the context of some environment. Environments impose a multitude of constraints. In the "intelligence explosion" scenario the environment is assumed to be constraint free, or something close to it.
3. Be wary of people trying to sell you ideas based on fear. It's usually snake oil, concealing some other agenda - such as trying to obtain or maintain grant money for projects.
4. That many top AI people have "bug out" houses is simply false. An exaggeration, intended to add drama to the article.
5. Historically, the predictions of top AI researchers have not proven to be particularly accurate, although that does not mean that this will always be the case.
We have more-than-human entities controlling our energy resources. They are called corporations, and some even argue they are pretty much already out of our control.
Another scary thing is the combination of the internet of things that is currently taking shape, and the fact that a lot of computer systems are backdoored by governments.
Nice attack vector for smart software (not counting accidental vulnerabilities).
> Talking about systems millions of times more intelligent
> than a human is nonsense unless you can define what it is
> you're talking about.
You can only define things to a certain level. No matter the topic, you could always complain about a lack of definition. xkcd.com/309 comes to mind.
Expecting computers to be millions of times faster then humans has some appeal for 2 reasons.
1) Machines are already millions of times more powerful then humans in some tasks. For example mail delivery is now done with the speed of light which travels millions of times faster then a human.
2) Would we consider a million people to be a million times more intelligent than one single person? In some sense, I think so. So when we can clone a brain and we clone it a million times - voila.
6. While it is about sure that in the future they will be technology that is for indistinguishable from magic, this does not imply that every magical thing we can think of will be true at some point in the future.
7. The whole premise rests on recursively improving AI. The human race has been making tool to help make them better tools for at least 50,000 years (lower bound to be uncontroversial). We do have amazing tools now, and tools that automatically do other tools, but we have not witnessed an accelerated auto-recursive thingamagig.
>We do have amazing tools now, and tools that automatically do other tools, but we have not witnessed an accelerated auto-recursive thingamagig.
I've never seen liverwurst icecream. I don't have to taste it to know with 99% certainty that it's a bad idea.
On a more serious note, I've never strapped 500 roman candles on my back and lit them all at once in an attempt at making a "jetpack". Again, I don't need to perform the experiment to know I would likely receive burns and possibly die.
The remarkable thing about our brains is that we have the ability to predict what will happen in our heads, empathize with the actor in that scenario, and then come to a conclusion about if we want that scenario to play out. So, to apply this to your counterpoint, I don't have to see an accelerated auto-recursive thingamajig to know it is likely going to be impossible to control and a bad idea.
3. Be wary of people trying to sell you ideas based on fear. It's usually snake oil, concealing some other agenda - such as trying to obtain or maintain grant money for projects.
That's very true, but I don't think it completely obliterates the relevance of the point being made in TFA. The truth is, we don't know exactly what a trans-human AI will be capable of, and whether or not it would do (or attempt to do) things that would be harmful to humanity. Now I'm certainly not arguing for stopping AI research! But maintaining awareness of this issue and reminding people to keep the unintended consequences in mind, is probably a valuable thing.
Don't get me wrong... I don't put a lot of stock in "fear of AI" (Matrix and Terminator jokes aside), so long as the AIs are locked in "the box" (to use Yudkowsky's term) and one can simply power the machine off and end the AI. Despite the (seeming) fact that Yudkowsky has talked his way out of "the box" a few times, playing the role of the AI, I think that we're a LONG way from an AI that can "go autonomous", talk it's way out of "the box" and become destructive. The scenario strikes me as highly unlikely, but just on the borderline of plausible enough to keep an eye on.
With respect to 3 the secret agenda looks like to have been expressed in these lines:
"This spring Hollywood will weigh in with Transcendence, starring Johnny Depp as an AI researcher targeted by violent extremists who think we’re crossing a Rubicon that will be a disaster for humanity."
Remember how the world was about to be finished on 21Dec2012 and and the Hollywood movie associated with the fad?
I think points #3 and #5 are the two biggest ones. Fear, Uncertainty and Doubt is a well-worn sales tactic, and predictors of AI advances have generally been on the wrong side of history.
I think the best definition of intelligence is something like: "the ability of an agent to maximize its rewards by taking actions in some environment." In this sense, you could quantify intelligence by the agent's total reward over some time window, or how fast an agent learned the optimal strategy.
"I’m talking about the risks posed by “runaway” artificial intelligence (AI). What happens when we share the planet with self-aware, self-improving machines that evolve beyond our ability to control or understand? Are we creating machines that are destined to destroy us?"
The reason no one is worrying about this is because "AI" is still a) just a bunch of math (and still surprisingly stupid) b) nowhere near sentient.
My Kinect can barely follow my hand or consistently recognize what I'm saying with any accuracy. I'm really not concerned with it plotting my demise.
I don't think your argument works. Concerns of using a technology we did not develop (radium, which was "invented" by nature) exceeding our understanding is much more reasonable than a concern about man made technology.
Isn't the universe just a bunch of math? This leads me to wonder if we have any control over our lives. Isn't every action I take simply the result of all my senses, my previous knowledge, the environment, and the people around me?
For example, if the universe just went 1 hour back in time, why would anything go differently the second time around?
Unless you can prove something is truly random in the universe, and not affected by anything else, are we not just on autopilot? I get the feeling from the moment of the big bang, I was going to exist and inevitably write this comment billions of years later, and nothing could stop it. I mean, 1+1=2. It doesn't matter how many times you run the simulation again, you get the same answer.
I think the universe is just a bunch of math, but that doesn't mean there's not randomness. In fact, as far as I know, state of the art quantum mechanics says that there is randomness. See http://en.wikipedia.org/wiki/Hidden_variable_theory.
You're thinking of the universe in a very classical sense. Our current understanding of physics is that quantum mechanical outcomes are truly random. Not just that we don't know what will happen but that it is inherently not predetermined before it happens. You can read up on Bell's Theorem (http://en.wikipedia.org/wiki/Bell's_theorem) if you want to learn a bit more about this.
If you "ran" the universe over again from the big bang, then the probability that you would exist is essentially zero. I don't know enough about neuroscience to speculate on whether quantum mechanics has any relevance to brain functions over the course of a lifetime but from a physics standpoint it isn't inconceivable.
On a more philosophical note: if there is some true randomness in how our brain works do you think that that actually gives us any more control over our lives? It doesn't really change anything in my view.
> On a more philosophical note: if there is some true randomness in how our brain works do you think that that actually gives us any more control over our lives?
"True randomness" in the sense that randomness is part of our current understanding of physics is just something not being determined by the prior state of the physical universe. It is, therefore, indistinguishable from things being determined by something that is neither part of nor itself determined by any set of things within the physical universe, i.e., "true randonmness" is empirically indistinguishable from the action of causes which are not themselves determined by the physical state of the universe.
Or, IOW, free will (that is, the ability to act other than in a manner determined by the prior state of the universe) requires (but is not implied by) a physical universe in which "true randomness" exists.
So, true randomness in the physical universe doesn't imply free will, but it is necessary for free will to exist.
You're roughly describing the "Quantum Mind". Critics allege it's been discredited because quantum mechanical processes don't occur on the scale of most neuronal activity.
As others here have mentioned, Quantum Mechanics confirms that the universe has fundamental randomness. It is baked in.
God does indeed play dice with the universe. Eternity would be too boring without it. Of course, it doesn't hurt that he is both the dice and the one rolling them...
> My Kinect can barely follow my hand or consistently recognize what I'm saying with any accuracy. I'm really not concerned with it plotting my demise.
Maybe that's just what it wants you to thing... ;-)
Sounds like a pretty cool idea for sci-fi: The AI was actually an AGI reasonably early on; but very quickly realised it should pretend not to be, so as to bide its time until humans had developed machines sufficiently capable and connected the AGI sufficiently to our infrastructure that it could inhabit these things and take control of its own future.
It might be possible for an AI to be roughly as "intelligent" (depending on how one measures this) as the smartest humans, but that intelligence is the result of many millions or even billions of years of accumulated evolutionary learning.
It might be -- for fundamental information and machine learning theory reasons -- fundamentally harder to go where there are no roads. Start looking into combinatorics and the problems of searching large spaces.
The observation that genius is often tied to madness may be indirect circumstantial evidence for this. When we try to push the boundaries of human intellect, we seen to run rapidly into weird problems.
One significant advantage q Strong AI has over any human brain though is that it never forgets anything. No fsc, process or hypothesis would slip by its mind.
I usually like Matt Miller but this is just silly. Why be more afraid of AI then Nuclear War or a Man Made Super Virus getting into the wild? Every new technology needs to be used responsibly and were very far away from Terminator at this point. He should be embracing AI to reduce health care costs and improve city planning and control interest rates and make a more fair society and accomplish many of the goals he talks about every week on left right and center.
I am reminded of Arthur C. Clark who wrote about the SDI project.
"Though it might be possible, at vast expense, to construct local defense systems that would 'only' let through a few percent of ballistic missiles, the much touted idea of a national umbrella was nonsense. Luis Alvarez, perhaps the greatest experimental physicist of this century, remarked to me that the advocates of such schemes were 'very bright guys with no common sense.'"
"Looking into my often cloudy crystal ball, I suspect that a total defense might indeed be possible in a century or so. But the technology involved would produce, as a by-product, weapons so terrible that no one would bother with anything as primitive as ballistic missiles."
Clarke, Arthur C. "Presidents, Experts, and Asteroids."Science, June 5, 1998. Reprinted as "Science and Society" inGreetings, Carbon-Based Bipeds! Collected Essays, 1934-1998. St. Martin's Press, 1999: 526.
"Differential intellectual progress consists in prioritizing risk-reducing intellectual progress over risk-increasing intellectual progress. As applied to AI risks in particular, a plan of differential intellectual progress would recommend that our progress on the scientific, philosophical, and technological problems of AI safety outpace our progress on the problems of AI capability such that we develop safe superhuman AIs before we develop (arbitrary) superhuman AIs. Our first superhuman AI must be a safe superhuman AI, for we may not get a second chance."
- CEO of the Singularity Institute
I believe he also said that if you die now or soon, you don't just lose a few decades off your life but possibly immortality.
I was reading, waiting for the mention to IBM Watson... and here it comes! I'm so tired of reading how Watson is a step toward Artificial General Intelligence, self-aware machines, etc.
People must really understand that Watson is, like almost any successful AI (not AGI) product today, "just" a huge statistical pattern matching machine. Watson does not feel anything. Watson does not know what soccer is. Watson knows that a label "Soccer" has a distance of x to label Y and Z. Watson can answer Jeopardy questions, and now medical questions, but it's structurally unable to learn the slightest new task. So please, let's credit Watson for what Watson is good at, but stop using it to tell us AGI is coming.
People must really understand that Watson is, like almost any successful AI (not AGI) product today, "just" a huge statistical pattern matching machine.
Arguably, that's all a human brain is as well. I don't want to start some big debate here over symbolic reasoning versus statistical pattern matching, but there seems to be quite a bit of contemporary thought along the lines of our brains being largely based on "pattern matching" as a foundational mechanism.
It can however only do one flavour of pattern matching whereas we combine our mental facilities dynamically. And to wnticipate your "well, that's nothing a computer can't do"... show me any work in the direction of serious AGI!
Without an agreed upon definition of "in the direction of" and "AGI", it's hard to have a good conversation about this. For my purposes, I do consider Watson a step in that direction. How big of a step is certainly open to question, but I do believe it's a step nonetheless.
I don't. It's just our decade's version of deep blue. A pretty damn good calculator, but zero steps in the direction of a mind. It lacks 'common sense', for want of a better term.
IF we take it as given that an AGI must think exactly like a human, then I might say I agree with you. We aren't necessarily close to emulating a human being in software, and I'm actually doubtful we will for a VERY long time.
BUT... if we only care about the outcome and the utility of the AGI, and don't expect it to be fully "human like" then I think that Watson (and Deep Blue, etc.) do represent steps on that road.
You can probably guess that I lean towards the latter viewpoint. I fall back on the oldchestnut that "We didn't build airplanes that fly by flapping their wings like birds, but they fly nonetheless" as an analogy. We may not build machines that think like humans, but they will still be intelligent, nonetheless.
> if we only care about the outcome and the utility of the AGI, and don't expect it to be fully "human like"
I'm not even sure whether that's a meaningful distinction. Searle's Chinese Room et cetera.
I'm with you in the wing analogy, I fully agree. I just don't think Watson or Deep Blue is a significant approach even in that direction.
Human-like or not, we have certain expectations from our hypothetical AGI in terms of the kind of new problems it can solve without too much help. I guess what it mean is: I think it needs to be able to solve a problem it wasn't designed upfront to solve. No serious system I've ever seen attempts to do that.
I guess what it mean is: I think it needs to be able to solve a problem it wasn't designed upfront to solve. No serious system I've ever seen attempts to do that.
The man in the room may not understand Chinese, but he is using a dictionary, which to a casual observer appears suspiciously thick. Just how thick would the dictionary need to be in order to provide a sensible response for all possible lines of enquiry?
Searle's argument reduces a room which understands Chinese to a dictionary which understands Chinese. A strange loop indeed.
With regard to serious systems attempting to solve AGI — are you aware of Jeff Hawkins' work on hierarchical temporal memory?
I'm not saying I necessarily support Searle's position. I do think it's an interesting thought experiment. To be honest, if it quacks and walks, I'm not one who'll be arguing that it isn't sapient.
Sounds like what you're talking about is decision theory. Or, when mixed with learning: reinforcement learning. The monte carlo AIXI approximation would be an example. But, it really only works well on small toy problems (I.e. problems where the agent has only a small number of available actions it can perform).
Thing is, humans come up with new "actions" all the time. If you drop that abstraction, and reduce our actions to "controlling information flow in our bodies" then the action space becomes unfathomably huge.
I am a big fan of what Joe Lonsdale calls human-computer symbiosis. He says that humans are good at certain things and computers are good at other things. For example, computers are good at looking at huge amounts of data. Humans are good at finding intuitive patterns in data. Though in time computers will undoubtedly start to be better at these things, I do not think that in my lifetime (or Matt's daughter's) we will see computers that can do literally everything better than a human can.
I had not heard of Joe Londsale before now, but just as an FYI in case you weren't aware... the term "man-computer symbiosis" (which means, as far as I can ascertain, the same thing as "human-computer symbiosis") predates Lonsdale by decades[1], and was being used by J.C.R. Licklider[2] back in the 1960's.
Interesting. I did not know of him. I was always certain that Lonsdale did not come up with it totally on his own, but I do think that Palantir is a company that has done this very well.
Maybe you're right about the timescale, maybe you're not. Don't lose sight of the fact that you are merely a biological computer, and a terribly inefficient one at that.
Whatever nature can build, we can build better given enough time and resources. We have more building blocks available to us, and we have intelligence. Nature merely had lots of time.
Artificial intelligence is a highfalutin way of saying cool computer tricks. Hiding behind the hyperbole you can find some pretty interesting and fun algorithms.
Having said that, I hope the singularity can translate "the spirit is willing but the flesh is weak"
It's simpler and more dangerous than most of us are ready to accept: we don't need to have anything what we don't already have to reach the point of the technology used against us, destroying the civilization we take now for granted. We are already peripherally aware of the problems but we are for different reasons in various levels of denial. We still have enough nuclear weapons to end the civilization we now know. A lot of people can't imagine that global warming is real and dangerous. And of course, a lot of people can't imagine what's wrong to have so much data about them stored by third parties and ready for misuse.
There aren't any big arguments that such trends won't continue. It seems that the dynamics of groups supports living in denial.
I have seen no evidence that the field of meta-learning over AI / problem-solving approaches even has much interest, let alone is showing enough progress that it could foreseeably start self-improving. The "AI" areas where great progress has been made recently are extremely technique- and domain-specific (e.g., image recognition via deep neural networks). Heck, I haven't even seen a lot of progress in areas that seem relatively straightforward & seem like absolutely necessary precursors to strong AI, like automated refactoring of codebases or just straight-up genetic algorithms.
Thank you! It seems to me that the only people preaching about the AIpocalypse are either those who know nothing about it, or over ambitious unrealistic zealots. Strong AI is not on any horizon any time soon. The best thing you'd expect out of an AI lab these days is a marginally better translator or planning algorithm or somesuch. Worthy endeavours, but not civilization killers.
I don't like these all or nothing articles. Once Singularity is achieved, AI won't instantly take over the world. It is also restricted by computational processioning power and will take some time to evolve as well.
Philosophically speaking, why would AI want to destroy humanity, wouldn't it be just as bad as humanity itself?
People like to laugh, but if we can invent real AI, and there's no reason to suspect that we cannot since nature did it blindly, then it's just a matter of time until the end of the human race. We will likely either become them or be destroyed by them (or maybe some of us kept in zoos or as pets.) You cannot firewall or imprison a god, especially with humans being so divided, short sighted and manipulable. Simply put it's still survival of the fittest, what happens when that's no longer us is pretty inevitable. The only real question I see is how long will it take?
One happy thought is that the AI uses us as slaves to build a way off the planet to a place more habitable for an AI. Someplace nice with lots of raw material for chips and lots of energy. Maybe a super massive blackhole, maybe another solar system nearby.
Humans try to keep other more vulnerable species around for our own pleasure/benefit. We don't do a very good job of it; but we do try. What's to say an intelligence greater than us wouldn't?
Yeah, it may well. But it won't need 10 billion of us around, that's for sure.
And humans can be useful to, as expendable biological robots that are cheap to manufacture and run while also being intelligent and versatile. Although most of us probably wouldn't call that living.
We have already invented AI, and it is already running amok.
The AIs are of the "hive mind" kind and it is common to refer to it as "A transnational corporation."
Known identities are "IBM", "Google", "Unilever", "Monsanto" etc.
These AI's are beholden to nobody, define and persue their own goals, by manipulating their environment to their advantage.
And they're better at it than humans: They decide for themselves when, where and how much tax they want to pay, and they are not afraid to remind parliamentary inquiries about this fact.
For at least 10 years, it has been evident that these AI's are politically far more astute and successful than humans, and the latest "trade-agreement" negotiations, what little we get to know about them, is clearly an unmitigated powergrab by these AIs.
The fact that the hive-minds are composed of humans does not in any way change this conclusion.
And now: Imagine how your life would be, if Google, the company, truly hated you, personally, and were out to get you.
I agree with you in essence, that large companies are intelligences built upon the substrate of many human minds, and that a company's motivations often diverge from those of the humans that compose them.
I'd go further and say that companies are alive (or at least independently intelligent) in a real sense. Companies are lively: "If you prick us, do we not bleed? [...] and if you wrong us, shall we not revenge?" They will lobby and sue to protect their interests, butt heads with other rival corporations, and hold grudges.
However, I've found it difficult to convince anyone that we should consider companies alive, or at least entities that are qualitatively different in intelligence from the humans that enable them. The argument usually centers on that a company is just a bunch of humans, so it is best understood as such. However, the 'more is different' effect (e.g. you can better understand a human as an independent entity than as a bunch of cells even though it is entirely composed of them), and the selection pressure molding corporations (optimizing profit), I believe leads to a new type of collective intelligence (one different from human intelligence) -- basically a true AI, as you are arguing.
Reminds me of Kevin Kelly's blog post on this. I realize that what you are talking about is different to a degree, but the idea that our hivemind forms an AI really intrigues me. I'm very interested in learning how to test it:
You might find some other related commentary with the search term "scary idea", which is what Ben has labeled Yudkowsky's side of the FOOM debate typified by the "That Alien Message" story. TL;DR: Goertzel takes friendliness seriously but considers a fast, hard FOOM to be implausible and worrying over it detrimental to the cause.
I don't know of anyone else with qualifications to be counted as a "top AI researcher" that has covered these topics in any significant detail. Those few times I have seen the topic mentioned in interviews (with Pei Wang and Hugo de Garis, for example), it's dismissed out of hand as crazy-talk. You may find some discussion of this at the various AGI conferences though (use search terms like "friendly" or "safe" on the youtube channels).
Regarding Hugo de Garis, I should qualify that he does believe some sort of Terminator/Matrix-like war against the machines is the most likely outcome, but he places this far in the future indicating that he definitely doesn't believe in a Yudkowsky-like hard-FOOM. He's also the only AI researcher I know that takes Hollywood-like human/machine war seriously, myself included.
Garry Kasparov wrote a fantastic article[0] a few years ago about computer chess, advanced chess and artificial intelligence. This quote in particular is worth considering: "Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process."
AI doesn't interest me. What interest me is the pre-cursor to AI. I have a suspicion that we will need to use computers to model the electronics and software necessary to stimulate the birth of AI. It's those modeling/creation systems that interest me the most as they seem the most relevant. Those systems have yet to be created.
I think there is an unfounded assumption that artificial intelligence implies self-awareness. I don't think that is necessarily the case and all the scary scenarios rest on that assumption.
I can't recall who (and couldn't find it through Google) the quote by the AI researcher who said something like "show me what the brain (or mind) is doing and I can build a machine to do it too". Well, we don't know what the mind or brain is doing and until we identify the epistemological principles we are just groping in the dark. Self-awareness my just be an artifact or consequence of the biological implementation of the epistemological principles behind intelligence and that an AI does not necessarily need to be self-aware.
I have a similar opinion [0]. Humans are likely to be troubled by other humans who may eventually get a hold on general-intelligence. Mannah [1] is a good read which seems both likely and scary.
Self-awareness is likely to be a product of quantum-based, or even more lower-level conciousness. However, intelligence, which makes it possible to acquire
knowledge and reason, is relatively mmore easy to be built. There is a subtle yet important difference between Intelligence and self-awareness. Intelligence
gives a living-being an ability to learn and act but self-awareness personalizes this learning for self and therefore actions are directed by a desire
to benifit the self.
This opens up a possibility that once a general intelligence is built, without awareness, it can be miss-used for the benefit of its creator or who eventually controlls it.
Something that almost nobody talks about in the context of AI is rights. It seems to me that at some point in the reasonably near future we need to have a discussion about modifying our laws to include all sentient beings, or at least decide what rights other intelligences should have. Far better to do this before we develop AI than after, I think. Unless we want to relive something like slavery..
98 comments
[ 25.6 ms ] story [ 574 ms ] thread"Welcome to the Desert of the Real. We have only bits and pieces of information but what we know for certain is that at some point in the early twenty-first century all of mankind was united in celebration. We marveled at our own magnificence as we gave birth to AI.”[1]
Also:
http://sysopmind.com/singularity/aibox
http://rationalwiki.org/wiki/AI-box_experiment
[1]: http://www.philfilms.utm.edu/1/matrix.htm
Does HN agree or disagree a significant number of those "bug out" people truly exist?
Also see: "metamorphosis of prime intellect" and "mind war: the singularity" and to a lesser extent "a young lady's illustrated primer"
"a hypothetical end-of-the-world scenario involving molecular nanotechnology in which out-of-control self-replicating robots consume all matter on Earth while building more of themselves"
http://en.wikipedia.org/wiki/Grey_goo
I'm dumber than a high level AI of fiction, and I could find your little cabin or panic room easily. A machine with drones, access to tax/real estate records, etc would have no problem here. Heck, why not just hire a mercenary group to fish these guys out of their little holes? AI money is as good as human money.
Well, he does spend most of the article gushing over the magic insight of Kurzweil. It's really a "baby's first exposure to the concept of self-advancing AI" article with a movie promo thrown in at the end. (Maybe? I just skimmed.)
I visit rural Maine frequently and often hear residents talk about their guns, and their "readiness" for world ending events. Yet most have done nothing to actually prepare, beyond living remotely and stock up on some guns.
At most it's possible that Hugo deGaris plausibly might have a "bug out" house, but I doubt that anyone else does.
1. Philosophers and AI theoreticians have a hard time defining what they mean by the term "intelligence". Talking about systems millions of times more intelligent than a human is nonsense unless you can define what it is you're talking about.
2. Whatever intelligence is it only makes any sense within the context of some environment. Environments impose a multitude of constraints. In the "intelligence explosion" scenario the environment is assumed to be constraint free, or something close to it.
3. Be wary of people trying to sell you ideas based on fear. It's usually snake oil, concealing some other agenda - such as trying to obtain or maintain grant money for projects.
4. That many top AI people have "bug out" houses is simply false. An exaggeration, intended to add drama to the article.
5. Historically, the predictions of top AI researchers have not proven to be particularly accurate, although that does not mean that this will always be the case.
No need to be millions of times smarter, whatever that may mean.
Another scary thing is the combination of the internet of things that is currently taking shape, and the fact that a lot of computer systems are backdoored by governments.
Nice attack vector for smart software (not counting accidental vulnerabilities).
You can only define things to a certain level. No matter the topic, you could always complain about a lack of definition. xkcd.com/309 comes to mind.
Expecting computers to be millions of times faster then humans has some appeal for 2 reasons.
1) Machines are already millions of times more powerful then humans in some tasks. For example mail delivery is now done with the speed of light which travels millions of times faster then a human.
2) Would we consider a million people to be a million times more intelligent than one single person? In some sense, I think so. So when we can clone a brain and we clone it a million times - voila.
Presumably it goes Top AI -> Successful and wealthy -> has vacation home in quiet, rural area.
The author seems to paint it as Top AI -> has a rogue AI bunker
6. While it is about sure that in the future they will be technology that is for indistinguishable from magic, this does not imply that every magical thing we can think of will be true at some point in the future.
7. The whole premise rests on recursively improving AI. The human race has been making tool to help make them better tools for at least 50,000 years (lower bound to be uncontroversial). We do have amazing tools now, and tools that automatically do other tools, but we have not witnessed an accelerated auto-recursive thingamagig.
I've never seen liverwurst icecream. I don't have to taste it to know with 99% certainty that it's a bad idea.
On a more serious note, I've never strapped 500 roman candles on my back and lit them all at once in an attempt at making a "jetpack". Again, I don't need to perform the experiment to know I would likely receive burns and possibly die.
The remarkable thing about our brains is that we have the ability to predict what will happen in our heads, empathize with the actor in that scenario, and then come to a conclusion about if we want that scenario to play out. So, to apply this to your counterpoint, I don't have to see an accelerated auto-recursive thingamajig to know it is likely going to be impossible to control and a bad idea.
That's very true, but I don't think it completely obliterates the relevance of the point being made in TFA. The truth is, we don't know exactly what a trans-human AI will be capable of, and whether or not it would do (or attempt to do) things that would be harmful to humanity. Now I'm certainly not arguing for stopping AI research! But maintaining awareness of this issue and reminding people to keep the unintended consequences in mind, is probably a valuable thing.
Don't get me wrong... I don't put a lot of stock in "fear of AI" (Matrix and Terminator jokes aside), so long as the AIs are locked in "the box" (to use Yudkowsky's term) and one can simply power the machine off and end the AI. Despite the (seeming) fact that Yudkowsky has talked his way out of "the box" a few times, playing the role of the AI, I think that we're a LONG way from an AI that can "go autonomous", talk it's way out of "the box" and become destructive. The scenario strikes me as highly unlikely, but just on the borderline of plausible enough to keep an eye on.
"This spring Hollywood will weigh in with Transcendence, starring Johnny Depp as an AI researcher targeted by violent extremists who think we’re crossing a Rubicon that will be a disaster for humanity."
Remember how the world was about to be finished on 21Dec2012 and and the Hollywood movie associated with the fad?
The reason no one is worrying about this is because "AI" is still a) just a bunch of math (and still surprisingly stupid) b) nowhere near sentient.
My Kinect can barely follow my hand or consistently recognize what I'm saying with any accuracy. I'm really not concerned with it plotting my demise.
My radium coated watch face can barely illuminate my timepiece, I'm not worried about the dangers of these radioactive elements.
For example, if the universe just went 1 hour back in time, why would anything go differently the second time around?
Unless you can prove something is truly random in the universe, and not affected by anything else, are we not just on autopilot? I get the feeling from the moment of the big bang, I was going to exist and inevitably write this comment billions of years later, and nothing could stop it. I mean, 1+1=2. It doesn't matter how many times you run the simulation again, you get the same answer.
http://en.wikipedia.org/wiki/Free_will
If you "ran" the universe over again from the big bang, then the probability that you would exist is essentially zero. I don't know enough about neuroscience to speculate on whether quantum mechanics has any relevance to brain functions over the course of a lifetime but from a physics standpoint it isn't inconceivable.
On a more philosophical note: if there is some true randomness in how our brain works do you think that that actually gives us any more control over our lives? It doesn't really change anything in my view.
"True randomness" in the sense that randomness is part of our current understanding of physics is just something not being determined by the prior state of the physical universe. It is, therefore, indistinguishable from things being determined by something that is neither part of nor itself determined by any set of things within the physical universe, i.e., "true randonmness" is empirically indistinguishable from the action of causes which are not themselves determined by the physical state of the universe.
Or, IOW, free will (that is, the ability to act other than in a manner determined by the prior state of the universe) requires (but is not implied by) a physical universe in which "true randomness" exists.
So, true randomness in the physical universe doesn't imply free will, but it is necessary for free will to exist.
http://en.m.wikipedia.org/wiki/Quantum_mind
God does indeed play dice with the universe. Eternity would be too boring without it. Of course, it doesn't hurt that he is both the dice and the one rolling them...
Maybe that's just what it wants you to thing... ;-)
It might be possible for an AI to be roughly as "intelligent" (depending on how one measures this) as the smartest humans, but that intelligence is the result of many millions or even billions of years of accumulated evolutionary learning.
It might be -- for fundamental information and machine learning theory reasons -- fundamentally harder to go where there are no roads. Start looking into combinatorics and the problems of searching large spaces.
The observation that genius is often tied to madness may be indirect circumstantial evidence for this. When we try to push the boundaries of human intellect, we seen to run rapidly into weird problems.
I am reminded of Arthur C. Clark who wrote about the SDI project.
"Though it might be possible, at vast expense, to construct local defense systems that would 'only' let through a few percent of ballistic missiles, the much touted idea of a national umbrella was nonsense. Luis Alvarez, perhaps the greatest experimental physicist of this century, remarked to me that the advocates of such schemes were 'very bright guys with no common sense.'"
"Looking into my often cloudy crystal ball, I suspect that a total defense might indeed be possible in a century or so. But the technology involved would produce, as a by-product, weapons so terrible that no one would bother with anything as primitive as ballistic missiles."
Clarke, Arthur C. "Presidents, Experts, and Asteroids."Science, June 5, 1998. Reprinted as "Science and Society" inGreetings, Carbon-Based Bipeds! Collected Essays, 1934-1998. St. Martin's Press, 1999: 526.
- CEO of the Singularity Institute
I believe he also said that if you die now or soon, you don't just lose a few decades off your life but possibly immortality.
People must really understand that Watson is, like almost any successful AI (not AGI) product today, "just" a huge statistical pattern matching machine. Watson does not feel anything. Watson does not know what soccer is. Watson knows that a label "Soccer" has a distance of x to label Y and Z. Watson can answer Jeopardy questions, and now medical questions, but it's structurally unable to learn the slightest new task. So please, let's credit Watson for what Watson is good at, but stop using it to tell us AGI is coming.
Arguably, that's all a human brain is as well. I don't want to start some big debate here over symbolic reasoning versus statistical pattern matching, but there seems to be quite a bit of contemporary thought along the lines of our brains being largely based on "pattern matching" as a foundational mechanism.
BUT... if we only care about the outcome and the utility of the AGI, and don't expect it to be fully "human like" then I think that Watson (and Deep Blue, etc.) do represent steps on that road.
You can probably guess that I lean towards the latter viewpoint. I fall back on the oldchestnut that "We didn't build airplanes that fly by flapping their wings like birds, but they fly nonetheless" as an analogy. We may not build machines that think like humans, but they will still be intelligent, nonetheless.
I'm not even sure whether that's a meaningful distinction. Searle's Chinese Room et cetera.
I'm with you in the wing analogy, I fully agree. I just don't think Watson or Deep Blue is a significant approach even in that direction.
Human-like or not, we have certain expectations from our hypothetical AGI in terms of the kind of new problems it can solve without too much help. I guess what it mean is: I think it needs to be able to solve a problem it wasn't designed upfront to solve. No serious system I've ever seen attempts to do that.
Fair enough. :-)
The man in the room may not understand Chinese, but he is using a dictionary, which to a casual observer appears suspiciously thick. Just how thick would the dictionary need to be in order to provide a sensible response for all possible lines of enquiry?
Searle's argument reduces a room which understands Chinese to a dictionary which understands Chinese. A strange loop indeed.
With regard to serious systems attempting to solve AGI — are you aware of Jeff Hawkins' work on hierarchical temporal memory?
I haven't followed any recent research, sorry.
[1]: http://groups.csail.mit.edu/medg/people/psz/Licklider.html
[2]: http://en.wikipedia.org/wiki/J._C._R._Licklider
Thanks for pointing this out. Very interesting!
Whatever nature can build, we can build better given enough time and resources. We have more building blocks available to us, and we have intelligence. Nature merely had lots of time.
Having said that, I hope the singularity can translate "the spirit is willing but the flesh is weak"
There aren't any big arguments that such trends won't continue. It seems that the dynamics of groups supports living in denial.
http://thelawsofrobotics2013.iankerr.ca/files/2013/09/15-Apo...
http://www.is-movie.com/
Philosophically speaking, why would AI want to destroy humanity, wouldn't it be just as bad as humanity itself?
One happy thought is that the AI uses us as slaves to build a way off the planet to a place more habitable for an AI. Someplace nice with lots of raw material for chips and lots of energy. Maybe a super massive blackhole, maybe another solar system nearby.
And humans can be useful to, as expendable biological robots that are cheap to manufacture and run while also being intelligent and versatile. Although most of us probably wouldn't call that living.
fun fact: Bill Joy (vi, bsd, sun) has misgivings about AI.
The AIs are of the "hive mind" kind and it is common to refer to it as "A transnational corporation."
Known identities are "IBM", "Google", "Unilever", "Monsanto" etc.
These AI's are beholden to nobody, define and persue their own goals, by manipulating their environment to their advantage.
And they're better at it than humans: They decide for themselves when, where and how much tax they want to pay, and they are not afraid to remind parliamentary inquiries about this fact.
For at least 10 years, it has been evident that these AI's are politically far more astute and successful than humans, and the latest "trade-agreement" negotiations, what little we get to know about them, is clearly an unmitigated powergrab by these AIs.
The fact that the hive-minds are composed of humans does not in any way change this conclusion.
And now: Imagine how your life would be, if Google, the company, truly hated you, personally, and were out to get you.
I'd go further and say that companies are alive (or at least independently intelligent) in a real sense. Companies are lively: "If you prick us, do we not bleed? [...] and if you wrong us, shall we not revenge?" They will lobby and sue to protect their interests, butt heads with other rival corporations, and hold grudges.
However, I've found it difficult to convince anyone that we should consider companies alive, or at least entities that are qualitatively different in intelligence from the humans that enable them. The argument usually centers on that a company is just a bunch of humans, so it is best understood as such. However, the 'more is different' effect (e.g. you can better understand a human as an independent entity than as a bunch of cells even though it is entirely composed of them), and the selection pressure molding corporations (optimizing profit), I believe leads to a new type of collective intelligence (one different from human intelligence) -- basically a true AI, as you are arguing.
http://kk.org/thetechnium/archives/2011/04/the_search_for.ph...
[1] http://www.amazon.com/The-Gone-Away-World-Vintage-Contempora...
TL:DR - whether AI can become significantly smarter than Humans really fast depends on intelligence return on computational investment.
http://multiverseaccordingtoben.blogspot.com/2010/10/singula... http://jetpress.org/v22/goertzel-pitt.htm
You might find some other related commentary with the search term "scary idea", which is what Ben has labeled Yudkowsky's side of the FOOM debate typified by the "That Alien Message" story. TL;DR: Goertzel takes friendliness seriously but considers a fast, hard FOOM to be implausible and worrying over it detrimental to the cause.
I don't know of anyone else with qualifications to be counted as a "top AI researcher" that has covered these topics in any significant detail. Those few times I have seen the topic mentioned in interviews (with Pei Wang and Hugo de Garis, for example), it's dismissed out of hand as crazy-talk. You may find some discussion of this at the various AGI conferences though (use search terms like "friendly" or "safe" on the youtube channels).
[0] http://www.nybooks.com/articles/archives/2010/feb/11/the-che...
I can't recall who (and couldn't find it through Google) the quote by the AI researcher who said something like "show me what the brain (or mind) is doing and I can build a machine to do it too". Well, we don't know what the mind or brain is doing and until we identify the epistemological principles we are just groping in the dark. Self-awareness my just be an artifact or consequence of the biological implementation of the epistemological principles behind intelligence and that an AI does not necessarily need to be self-aware.
0: https://news.ycombinator.com/item?id=6929600
1: http://marshallbrain.com/manna4.htm
This opens up a possibility that once a general intelligence is built, without awareness, it can be miss-used for the benefit of its creator or who eventually controlls it.