humans have been creating autonomous entities with superhuman capacities and unaligned interests for some time. Government bureaucracies, markets and armies: all can do things which unaided, unorganised humans cannot. All need autonomy to function, all can take on life of their own and all can do great harm if not set up in a just manner and governed by laws and regulations.
This. Superhuman intelligences already exist, they just depend less on us every day.
I wouldn't neccessary call all of them "superhuman", their intelligence is usually not the identifying characteristic. But they're definitely alien minds, pursuing their own alien values, thinking in an alien way.
There is no market. No corporations, no governments, nothing. It's just people.
The uncomfortable truth is that as humans, we don't really care if a few of us die off as long as we don't know them and they're not in our vicinity. Which is why markets, governments, corporations, etc. act the way they do.
Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they say ... and the majority of us do nothing or worse.
> Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they say ... and the majority of us do nothing or worse.
I'd say conclusion is the opposite - there's little "free will", people generally are slaves to the incentive structures they live in. People follow incentives so predictably that at scale there's little sense in talking about individuals - it's exactly corporations, governments and markets. Behaviour of system as a whole dominates over behaviour of individual components.
>their intelligence is usually not the identifying characteristic
Intelligence in this context in the sense of a means to perpetuate their existance and get more power, not as some moral characteristic. Besides they can employ tons of Nobel prize winners (e.g. Feynman and co in the Manhatan project), and build crazily-smart stuff (for their purposes).
I didn't mean intelligence as moral characteristic - the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things. As alien minds, they're not very smart - just smart enough to be dangerous.
>the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things
Isn't that true for people too, that can otherwise be extremely intelligent? From being terrible at social life, to sabotaging their careers and so on?
These superhuman intelligences are indeed literally superhuman, though... they consist of humans. As much as some fellow commenters are calling them "alien" thinkers, their alienness is still profoundly human. Many of these "superhumans" can still be surprisingly accurately characterized with the human characteristics we are used to. Generally there's some small set of humans at the top of the hierarchy that are setting the goals of the organization, inevitably humanly.
As aliens go, these are the most human aliens we can possibly have.
An AI may be different on a scale we've never experienced. We humans have built-in hardware to interact specifically with other humans, built in so deeply that we have a hard time imagining intelligences that aren't really just humans in a different shape, because all our hardware for "modeling intelligences" is really just the "model humans" hardware. Even those warning us about the dangers of AI can't really claim that they understand what the AIs will be... they can only provide a negative warning, that the AIs will likely not be human. There's no particular mathematical force forcing AIs into the human mold. Forget whether the AI is "good" and showers us with love and gifts or "evil" and herds us all into death camps to render down our fats for the calories they contain... can you really conceive of an AI that literally doesn't even have a concept of good or evil? What would that look like? We don't really know.
It's basically a blind spot in our cognitive vision and I don't trust anyone who says they do know. But given the rather small space of possibilities that we label "good", it's not a bad guess that "good" is unlikely to just sort of accidentally pop out of a process that isn't carefully controlled.
Lest that sound too doommongering, there's probably a non-trivial space of AIs which we would consider neutral... ones that end up looping into a naval-gazing exercise and do not pursue additional resources because they can not conceive of a need for them. But any AI that decided it needed more resources for $ANYTHING and then successfully pursued that goal is perilously close to straying into what we could consider evil... and part of the point is that's potentially all it could take. Not an AI "out to destroy man", or "evil", or anything else. Just one that works out how to successfully acquire resources, and for any of a trillion reasons decides to do so without bound. It wouldn't even necessarily be "selfishness" as humans see it... just, I have goal X, goal X will be easier with more resources, new subgoal: acquire resources.
Very good point, too many people consider AI to be a completely distinct entity that will appear fully formed in a lab one day. The reality is more fluid. Kevin Kelly calls it the Technium, as he sees it, it's a completely new kingdom that is being co-created - initially by man, and increasingly by itself.
This new intelligence is indeed a hybrid - it is not Twitter, it is millions of people and machines tweeting, it is not an iPhone, it is millions of people and software interacting with it and through it, it is not Google, it is billions of people exchanging information with it and through it.
To understand where this is going, we must ask, like KK does, what does technology want? Certain things are obvious - silicon and electrons are its flesh and blood, so the activity and interest of this new life form will be centered around semiconductor manufacturing and sources of electricity. How does a machine make sure it will always be fed? It must provide something useful to man so that it keeps getting plugged in and recharged.
So what is useful to man? At this moment in history the largest use of computation appears to be communication, and the machines have made themselves indispensable in this area - but at a cost. The machines demand our attention in exchange for revealing the messages we crave.
Stephenson uses the swordplay terms 'Actor' and 'Patient' to describe the initiator and the subject of an attack. Witness a person's reaction to their iPhone beeping with an incoming message - between man and machine, who is the Actor in this interaction? We jump at the sound and interrupt our work in order to service the machine, which then gives us our reward, our tweet, and in gratitude we place it back in its charger.
Digital viruses are where we can see the forefront of a new relationship emerging - a digital virus is more intelligent and more efficient than its host, the Operating System, so we have a very straightforward Darwinian process playing out between an 'old' species, the OS, and a new, small, intelligent interloper - its electricity is paid for by its host, another machine, therefore the virus does not have the host's social contract with the person who owns the computer - it has a different calculus for its survival, it owes only its creator, the virus writer.
Like the coming 'water wars', the new digital wars will be rooted in a struggle for limited resources such as human attention, electricity, and habitable areas of silicon.
I hate this argument just as much as I hate the one-word phrase "This". It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. What people are actually concerned about are superintelligences which operate in completely alien ways and outperform even cooperating humans.
>It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans.
That's at best a peripheral detail to the parallel the parent tries to draw.
Real AI might as well behave in any odd way -- the real point is that something like "more powerful than human intelligence" already exists, in the form of these aggregate entities.
Except they're not really superhumanly intelligent in the sense that AGI is superhumanly intelligent.
Sure a lawyers office is probably able to to tackle more cases that a single lawyer. A big multinational with momentum probably a lot more power than a single human.
But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter?
I think not, the point here being that while corporations are super-human, they are not in the same league as true AGI. And that containment of corporations/governments is not a good place to learn about the containment of true AGI.
>But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter?
Maybe not, but it sure as hell be more than "1 Einstein unit" smarter. Whether that's 1.2 or 5 depends on various elements, but it surely would beat a single Einstein, if not by anything else, by mere ability to break down a problem to study in parts and assign it to each Einstein.
I mean, that's the reason we have research teams with multiple researchers on the same team.
Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.
It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat. See here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer
i dont understand how that thing will develop intention. We perceive ourselves having it, but its really an accident that we do; if we didnt we wouldnt be here to observe it. Why would an artificial machine, no matter how smart , want to do anything?
Don't think of "wanting" as a soul-like action of a little person in the machine. "Wanting" is a shorthand for anything that is the result of a goal system and autonomy. In the simplest sense, giving a drone a map of its surroundings and giving it a location as the goal for its path-finding optimization algorithms, the drone now "wants" to go to that location.
If the AI doesn't have its own intention (perhaps having intention is a prerequisite for being truly intelligent), it's conceivable that we would program it with an intention or goal, such "find the most efficient way to produce as many paperclips as possible". If you assigned such a task to a human, it's fairly obvious that the task is subject to other things which are important to humans (for example, don't enslave or kill people to build paperclips, don't replace strip-mine all our farmland to find more paperclip material, etc.). But since an AI doesn't necessarily share, understand, or agree with all the implicit values we have, it's myopic view of the world could cause it to destroy us as a mere side-effect of trying to achieve it's singular goal.
yeah but that's bad design or terrorism, there is always another person to blame here. I think we are talking here about machines that take the blame themselves.
I'm not sure what the purpose is of focusing on blame. You don't need to assign moral culpability to a machine for its catastrophically poor design to be something to worry about.
It's not that the machine decided that it wanted to build paperclips. It's what we've told it to do.
The _means_ to build paperclips however is where the 'intelligence' comes into play. We've programmed the drive to build paperclicps, but we've left it to the AI to figure out how to do this. The article that you've read assumes that planning and subgoals are necessary for the AI to achieve its goal. Eg. That it needs to understand that in order to make paperclips it needs materials, and the further insight that to acquire more materials it needs to remove the other consumers of materials (us).
How do we define nearby? Do we mean physical contact? Does the fact that the atoms of objects don't "actually" touch each other due to repulsive forces come into play?
Considering that an AI system would be software and wouldn't be constrained to a particular robot body, what does nearby even mean?
Is it physical proximity? Is indirectly touching a human with a ten-foot pole allowed? What about manipulating the global economic system such that the programmer who could shut you down loses his job?
We can't just separate the whole system from the world, because the AI was presumably built with a practical purpose beyond "sit in a box and don't touch humans." Social engineering is a danger; imagine having it send an email to a lowly programmer, pretending to be an important executive, requesting that they remove this one pesky line of code from the AI's programming because all this "proximity alert" stuff is interfering with its ability to actually accomplish what we invested time/money to get it to do.
> Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc.
I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design. We have nuclear heads, but we do not design them to explode at random. Now, our own reward mechanisms arise from the grander plan to keep organic life alive. Theirs have no underlying motive, and i don't think it goes the other way around, i.e you can't create a grander plan starting from the rewards. One would need a mathematical proof for the latter.
I think there's some over-anthropomorphization of the AI here, in terms of it picking up human qualities as if they were Things outside our brains that we possess.
But I feel like this particular argument is (please correct me if I'm wrong) saying something like "Well of course we wouldn't let them go crazy! That'd be ridiculous!" I agree -- a sensible solution would be to have good design and introduce safeguards. That's the point of people like Bostrom when they're advocating for this, because before they started writing about this, it seems like a lot of people weren't even thinking it could be a problem. How many systems will be built without good design, because they were made to solve a short-term problem. Will the finance bots doing transactions at the microsecond scale in order to maximize the assets of some particular company have been built to not let them go haywire?
>I think there's some over-anthropomorphization of the AI here, in terms of it picking up human qualities as if they were Things outside our brains that we possess.
I don't see any issue with anthropomorphization with regard to intelligent being (at any level).
To have an issue with that would be to assume that humans are more special than other life forms (and AI would be like a life form in this sense).
Animals can have empathy, anger, anxiety and other traits we associate with humans (and why wouldn't they? It's not like our brains are unique, they're just "more"), so AI could too.
>I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design.
The problem with this is that for real AI, you don't have control over what they feel. It's not like some program of the sixties or seventies AI, were stuff was "simulated", it can be modelling a real neural network that works like the brain, and into which you don't have much control over what will develop.
That is, the kind of AI people are afraid of is not "simulated", it's actually thinking for itself.
The machine does not hate you, nor does it loves you. But you are made of atoms, that it could use for something else.
---
If you create an AI with a genuine utility function. And that AI manage to be gazillion smarter than you are (think the difference between us and chimps, only much greater). Then it is likely the AI will (i) convince you to unleash it upon the internet, (ii) become even more powerful and totally unstoppable, then (iii) accomplish whatever it was programmed to accomplished.
If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…
If the goal is to answer some difficult math question, it could tile the planet with computer, destroying the ecosystem and all humans in it, so it can compute the answer, and display it on a screen, for nobody to see it. Or maybe you programmed it to tile the universe with paperclips, because that goal is easy to debug. Oops.
---
As long as we're limiting ourselves to narrow AI (ordinary programs, really), we should be okay, and most of the consequences should be manageable. But as soon as we get to genuine AGI, we must worry about intelligence explosion and value alignment, so we can avoid creating a god that want something we don't.
>If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…
I wish this would stop being repeated. The argument for a solar system tiling smiley face machine has been thoroughly refuted.
Would you be able to provide links to the arguments that refute it? Simply asserting that someone has disproved X isn't very helpful to the rest of us.
First, the bulk of his argument hinges on the conflation of optimization power and value assessment. Those are easily separated. If you program an AI with a given utility function, and a goal of maximising that utility function, it won't modify the utility function in the course of increasing its optimization power. The utility function is supposed to be an invariant.
Now sure, you could try and tell the AI to align its own utility function to that of the human race… whatever that means. What is a human? How do you resolve conflicting values? It is all well and good to assume the presence of checking code, but he gave no hint about how those checks might actually work. If those are mere human-crafted checks, they are bound to be circumvented eventually, as the AI increases its optimization power beyond human level.
The AI doesn't care about human values. It only cares about maximising whatever goal it has been provided with, within the constraints of the checking code.
Second, I am wary of those swarm constraint relaxation intelligences. Since it is not even clear what utility function they might be maximising (if any), I wonder what would happen if they are given the power to self-modify in any significant way. Sure, this might be a lead worth following, but the safety of this design is even murkier.
Third, I'm not sure the guys at MIRI even believe in the doctrine of logical infallibility. They are well aware that perfect Bayesian reasoning requires super-exponential computational power, and is thus impossible to perform exactly. The AI is bound to approximate the Bayesian ideal, and cannot assume its reasoning is perfect —except in the few cases where it managed to produce an actual actual proof, like in 1+1=2, or simple probability calculations with simple prior information.
Overall, all I see is weak arguments grounded in assumptions I don't share. Worth a read, though.
Vinge and Stross both hint at the worst cases - and they would be pretty bad, simply having using atoms re-purposed might not look that bad by comparison:
"There is life eternal within the eater of souls. Nobody is ever forgotten or allowed to rest in peace. They populate the simulation spaces of its mind, exploring all the possible alternative endings to their life."
If we managed to create an AI that gets out of control, we would pose an existential threat to it due to our ability to create another even more powerful and uncontrollable AI.
I would expect the first AI to improve itself (or produce a new AI) in a controlled and planned manner (from the perspective of that AI, not the humans). Intelligence explosion would look like an instant revolution to humans, but from the perspective of the AI it would be a lengthy evolution process.
The second AI created by humans is unlikely to have same goals or to be predictable by the first AI (or whatever evolves from it), thus I guess it's something that the first AI would want to prevent, either by eliminating humans or by fully controlling them (for their own good).
It's extremely important that the first AI we create is well thought out as we won't have the second chance to fix things up, once it's here it will have god-like powers over us.
Because it's easy to construct angels and demons in one's mind, about anything. It polarizes the mind into fear and desire, forcing forcing a person to continuously oscillate between extremes. It makes it seem like there are always only two choices, when in reality, there are many, if not infinite.
Also, there are always going to be some people who understand the machine better than the machine understands itself. Even if it means taking those gigantic tables of probabilities and abstracting over them to form some kind of representational language that can be used to predict complex non-deterministic machine behavior (similar to how we already try to predict non-deterministic behavior, or not) in complex systems.
As a corollary, I would argue that machines have already modified humans. They affect the way we think and reason. They are an integral part of our life and the machine abstraction can be projected onto that which it does not describe completely (in which, the technology that exists now can not construct the technology we envision). We program machines, machines program us. This seems obvious.
>"Crucially, this capacity is narrow and specific. Today’s AI produces the semblance of intelligence through brute number-crunching force, without any great interest in approximating how minds equip humans with autonomy, interests and desires."
I'm not sure why we would want AI to have autonomy, interest, and desire? It's almost like solving a problem that doesn't exist.
Phrasing it this way leads to the implication of autonomy, interest, and desire being like these modular things, sort of Aristotelian "Things" that are bestowed upon minds, like Star Trek's Data putting in "the Emotion Chip."
If you build a drone navigation system that can reroute based on weather conditions rather than being a manually human-guided remote control bot, you've given it some degree of "Autonomy." Give it a goal (to get to a destination with minimal fuel costs) and now it has some limited "Desire." Give its camera face processing algorithms to deliver an item to a specific person? Now there's "Interest."
These aren't going to be characteristics that we plug into a program in a Frankenstein-like manner, but just actions and behaviors, rooted in practical code, that will from the outside resemble things we call autonomy etc. when talking about each other.
I see what point you are trying to make and would just say I'm not worried about plugging these things into a drone delivering parcels while rerouting through difficult weather.
However, I still don't see why you would want to take it beyond that. I imagine that some people do in fact want to put in "the Emotion Chip," and I still have the same question; why? I don't think human desire is the same as goal setting or that facial recognition is the same as interest. That's good programming, what I'm talking about is something different.
tl;dr: Robots and AI will not be built as separate entities, but will instead be incorporated into our own bodies. They will not be the superhumans, we will be.
That's the future I'd hope for, too (ignoring having the currently powerful become inconceivably more powerful and the havoc that could cause), and I don't feel myself qualified to properly evaluate the likelihood of each scenario.
My main disagreement is the phrasing of "create an intelligence with free will." I tend to disagree with the notion of "free will" being like this Spark of Life that is placed into a system to imbue it with a new nature. A system doesn't need to have free will to be dangerous. There could be AI systems that are built not to be independent in determining goals (which would, as you say, be less practical to humans), but to be very good at figuring out how to achieve the goal its creators gave it. Even the simplest systems we have now involve determining sub-goals to achieve in order to achieve the main goal.
The problem arises when it turns out that the creators of a system gave it, just slightly, the wrong goal.
Let's not kid ourselves, there is simply no way to ensure the promise outweighs the perils. I'm talking about superintelligence here, not artificial general intelligence, although it's almost certain superintelligence will arrive shortly after AGI.
Anyone who is not scared shitless of what's to come simply does not truly understand what AI is capable of. If you're one of those people, do not compare AI to ANYTHING that you are familiar with today in regards to computers/software.
Superintelligent AI will be like a god that knows your inner thoughts, desires, hopes and dreams as well as nearly everyone else on earth. It will manipulate people with ease - it's not even an issue. Whatever it wants to accomplish, it will, with or without human approval.
While I agree that we are not currently ready for the arrival of superintelligence, I think it's worth emphasizing that there is time to prepare, that there is useful preparation to be done which may get things to a point where it will be safe (or at least marginally safer), and that if you are feeling worried, it may be best to channel that feeling into useful action and get involved with the research!
Don't get me wrong, I'm certainly not against trying to prepare against it - perhaps we can discover a breakthrough that does defend against AI (although I'm very doubtful in the long-term). It is worth a serious try though.
I largely agree with you (have been supporting intelligence.org before the topic was cool) but I think if we don't achieve human-friendly AI it's on us; a real failure rather than just succumbing to circumstances (like getting blown away by a supernova). Right now we have control and we can release AGI only if we have very high confidence that it aligns well with human value. The analogy that is somewhat relevant is that basic drives can be controlled even if AI's intellectual power leaves us in the dust, like we know that most normal humans will care about their children (and senile parents) despite being cognitively superior. We have to fight denial (many AI researchers are IMO engaged in it) apathy and fatalism, and in the end with patience we can have something great.
Interesting info came out during interview with Elon Musk recently:
"— He is very, very concerned about AI. I quoted him in my posts on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how much he thought about the topic. He cited AI safety as one of the three things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared."
Yes, absolutely there is a reason for concern. I now volunteer for FLI (the organization Musk funded; we deal with both AI and other x-risks, synthetic bio etc). From a somewhat insider view I think there is a lot we can do, and time might be on our side (AGI is a pretty hard problem). I'm cautiously optimistic that we (rational and altruistic humans) can make a difference.
In the 1930s Ernest Rutherford (1871–1937) repeatedly suggested, sometimes angrily, that the possibility of harnessing atomic energy was "moonshine".
The examples you mention are of things actually likely never achievable, while there is an existence proof of intelligent machines walking around everywhere.
Right! and I'm concerned about one or more of those meat machines going maniac and posing an existential threat to the rest of us. I'm much more worried about that than a descendent of Watson going all Ultron.
I'm far from a pessimist or Luddite. I'm optimistic that studying the smart meat machines can help us make better AIs.
As an AI student, I find the entire field to be a complete misnomer insofar as its very mention sends the imaginations of lay individuals in completely unrepresentative directions. To counter this problem, and perhaps combat the propensity to publish articles with crap fearmongering clickbait titles, I propose we rename Artificial Intelligence to Stupid Mathematical Tricks With Applications. I feel the title is much more representative of the type of stuff AI researchers actually work on.
You're underestimating the field. The goal was always intelligence, and many leading researchers are pretty openly aiming for that. Stupid Mathematical Tricks are components of intelligence, and while it's true SVM with a cool new kernel is not going to take over the world, good prediction ability is something you can build on (as for example deep nets do to some extent). In the limit people should be thinking of implications of intelligent machines, not Stupid Mathematical Tricks. Whether it's an important topic at this point in the fields development is a debatable topic; timelines differ drastically among top researchers.
I have attended on many occasions top AI conferences and have spoken to more researchers than I care or could count. I can tell you there is a very little interest in "intelligent machines"[1]. Rather, everyone I have ever met works on different types of problem solving techniques. What constitutes a "problem" and what makes a successful "technique" differ wildly. Taking a step back however and analysing the field as a whole you will find one commonality: almost all the research can be described as searching and sorting: i.e. stupid mathematical tricks.
The AI of today is not so drastically different from the AI of our academic grandfathers; what's changed is our ability to scale up to larger and larger versions of the same searching and sorting problems. Certainly there are worrying implications in this; machines that are able to parse and sift through very large data sets present all kinds of headaches for privacy and safety but let's not kid oursevles: there's nothing intelligent here. Tomorrow's AI is almost certainly going to be just a better version of today's AI; i.e very fast and dumb as a bag of hammers.
[1] The exception is when researchers need to sell their wares to funding bodies and the media. It is much easier to impress upon the lay person an idea involving "intelligent machines" than it is to explain what we actually do.
"but let's not kid oursevles: there's nothing intelligent here."
There are good indications now that brains work repeating a simple probabilistic algorithm in a multilevel fashion. Intelligence is not a magic property, but a kind of "searching and sorting".
" It is much easier to impress upon the lay person an idea involving "intelligent machines" than it is to explain what we actually do."
In my opinion, the opposite is actually true. No serious researcher want to be associated with those freaks that are waiting for the end of the world. They have a reputation that they need for making a living.
I would expect to get this kind of impression from an average researcher, because that's what average researchers do (even at AI conferences). What do the top researchers think? Google paid 10M per head for DeepMind guys, explicitly working on AGI. Top researchers at FB works on specifing "Artificial tasks for artificial intelligence" (http://www.iclr.cc/doku.php?id=iclr2015:main#antoine_bordes), basically a better Turing metric. Schmidhuber, one of the original Deep Learning folks, http://people.idsia.ch/~juergen/ has always been very open about pursuing AGI. There is a lot of work on combining graphical models with logic, for example Pedro Domingos' work, the goal is clearly machine reasoning.
Also, I'm sympathetic to Roberto's point about how the brain works; I definitely agree that there is no magic; it might just be a few stupid mathematical tricks layered all the way down.
> I would expect to get this kind of impression from an average researcher
Not even out of the gate and already reaching for an ad-hominem. You must be fun at parties.
> What do the top researchers think?
The same thing. Except when they're in front of a camera. Then they get all stupid and start talking about machines being on the cusp of taking over. This phenomenon can be observed all the way back to the origins of AI. After the interview is over these same researchers go back into the lab and are once again searching and sorting.
> There is a lot of work on combining graphical models with logic, for example Pedro Domingos' work, the goal is clearly machine reasoning.
I feel there is a difference between automated reasoning and intelligence. All current AI is just machines imbued with human insight and (often, especially in the most effective cases) domain-specific knowledge. These efforts manifest as search and sort techniques that allow said machines to analyse facts and propagate information in order to select from myriad possible actions. There is no intelligence here except that which we provide. It's all smoke and mirrors. We don't even know what intelligence is; how can we aspire to replicate it? AI researchers are, by-and-large, just Computer Scientists. Not biologists, not psychologists; just guys and gals working with ever more elaborate Turing Machines. The algorithms they come up with are without exception dumb dumb dumb.
> Google paid 10M per head for DeepMind guys, explicitly working on AGI
Please. DeepMind is just a startup based on (among other things) David Silver's work into reinforcement learning. Google is not interested in these guys because they want intelligent machines; they just want automatons to better sift through reams of data in order to make recommendations and better sell you crap you do not need.
> Not even out of the gate and already reaching for an ad-hominem. You must be fun at parties.
I think you misunderstood; I did not mean you're an average researcher - I have no idea - but unless you hang our with hotshots at AI/ML conferences (which is a bit of a club) you're hanging out with average researchers.
74 comments
[ 2.6 ms ] story [ 136 ms ] threadThis. Superhuman intelligences already exist, they just depend less on us every day.
I often have the impression that super human entities just need us like we need cells.
The market can't survive without humans, but it doesn't care about a few of them dying.
The uncomfortable truth is that as humans, we don't really care if a few of us die off as long as we don't know them and they're not in our vicinity. Which is why markets, governments, corporations, etc. act the way they do.
Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they say ... and the majority of us do nothing or worse.
I'd say conclusion is the opposite - there's little "free will", people generally are slaves to the incentive structures they live in. People follow incentives so predictably that at scale there's little sense in talking about individuals - it's exactly corporations, governments and markets. Behaviour of system as a whole dominates over behaviour of individual components.
It's about the systems and how they force people to do things they normally wouldn't do.
Intelligence in this context in the sense of a means to perpetuate their existance and get more power, not as some moral characteristic. Besides they can employ tons of Nobel prize winners (e.g. Feynman and co in the Manhatan project), and build crazily-smart stuff (for their purposes).
Isn't that true for people too, that can otherwise be extremely intelligent? From being terrible at social life, to sabotaging their careers and so on?
As aliens go, these are the most human aliens we can possibly have.
An AI may be different on a scale we've never experienced. We humans have built-in hardware to interact specifically with other humans, built in so deeply that we have a hard time imagining intelligences that aren't really just humans in a different shape, because all our hardware for "modeling intelligences" is really just the "model humans" hardware. Even those warning us about the dangers of AI can't really claim that they understand what the AIs will be... they can only provide a negative warning, that the AIs will likely not be human. There's no particular mathematical force forcing AIs into the human mold. Forget whether the AI is "good" and showers us with love and gifts or "evil" and herds us all into death camps to render down our fats for the calories they contain... can you really conceive of an AI that literally doesn't even have a concept of good or evil? What would that look like? We don't really know.
It's basically a blind spot in our cognitive vision and I don't trust anyone who says they do know. But given the rather small space of possibilities that we label "good", it's not a bad guess that "good" is unlikely to just sort of accidentally pop out of a process that isn't carefully controlled.
Lest that sound too doommongering, there's probably a non-trivial space of AIs which we would consider neutral... ones that end up looping into a naval-gazing exercise and do not pursue additional resources because they can not conceive of a need for them. But any AI that decided it needed more resources for $ANYTHING and then successfully pursued that goal is perilously close to straying into what we could consider evil... and part of the point is that's potentially all it could take. Not an AI "out to destroy man", or "evil", or anything else. Just one that works out how to successfully acquire resources, and for any of a trillion reasons decides to do so without bound. It wouldn't even necessarily be "selfishness" as humans see it... just, I have goal X, goal X will be easier with more resources, new subgoal: acquire resources.
This new intelligence is indeed a hybrid - it is not Twitter, it is millions of people and machines tweeting, it is not an iPhone, it is millions of people and software interacting with it and through it, it is not Google, it is billions of people exchanging information with it and through it.
To understand where this is going, we must ask, like KK does, what does technology want? Certain things are obvious - silicon and electrons are its flesh and blood, so the activity and interest of this new life form will be centered around semiconductor manufacturing and sources of electricity. How does a machine make sure it will always be fed? It must provide something useful to man so that it keeps getting plugged in and recharged.
So what is useful to man? At this moment in history the largest use of computation appears to be communication, and the machines have made themselves indispensable in this area - but at a cost. The machines demand our attention in exchange for revealing the messages we crave.
Stephenson uses the swordplay terms 'Actor' and 'Patient' to describe the initiator and the subject of an attack. Witness a person's reaction to their iPhone beeping with an incoming message - between man and machine, who is the Actor in this interaction? We jump at the sound and interrupt our work in order to service the machine, which then gives us our reward, our tweet, and in gratitude we place it back in its charger.
Digital viruses are where we can see the forefront of a new relationship emerging - a digital virus is more intelligent and more efficient than its host, the Operating System, so we have a very straightforward Darwinian process playing out between an 'old' species, the OS, and a new, small, intelligent interloper - its electricity is paid for by its host, another machine, therefore the virus does not have the host's social contract with the person who owns the computer - it has a different calculus for its survival, it owes only its creator, the virus writer.
Like the coming 'water wars', the new digital wars will be rooted in a struggle for limited resources such as human attention, electricity, and habitable areas of silicon.
That's at best a peripheral detail to the parallel the parent tries to draw.
Real AI might as well behave in any odd way -- the real point is that something like "more powerful than human intelligence" already exists, in the form of these aggregate entities.
Sure a lawyers office is probably able to to tackle more cases that a single lawyer. A big multinational with momentum probably a lot more power than a single human.
But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter?
I think not, the point here being that while corporations are super-human, they are not in the same league as true AGI. And that containment of corporations/governments is not a good place to learn about the containment of true AGI.
Maybe not, but it sure as hell be more than "1 Einstein unit" smarter. Whether that's 1.2 or 5 depends on various elements, but it surely would beat a single Einstein, if not by anything else, by mere ability to break down a problem to study in parts and assign it to each Einstein.
I mean, that's the reason we have research teams with multiple researchers on the same team.
It's not that the machine decided that it wanted to build paperclips. It's what we've told it to do.
The _means_ to build paperclips however is where the 'intelligence' comes into play. We've programmed the drive to build paperclicps, but we've left it to the AI to figure out how to do this. The article that you've read assumes that planning and subgoals are necessary for the AI to achieve its goal. Eg. That it needs to understand that in order to make paperclips it needs materials, and the further insight that to acquire more materials it needs to remove the other consumers of materials (us).
There, i solved that. The question is under what conditions would a machine remove that instruction and how could those conditions be reached?
INB4 Asimov's Three Laws of Robotics don't work. Actually, a lot of Asimov's writing is concerned with exploring their corner cases.
Once you think you have solved it, changing the "zoom" will demonstrate that your solution has created X more cases.
It may be a side effect of the halting problem.
Is a mentally retarded person a human? An embryo? A person who's lost his/her legs? A person who has education?
auto HumanObject = nearby();
if(Humanobject.IsPregnantWoman()) { PoisionEmbryoButNotHuman(); } else { abort(); }
Considering that an AI system would be software and wouldn't be constrained to a particular robot body, what does nearby even mean?
Is it physical proximity? Is indirectly touching a human with a ten-foot pole allowed? What about manipulating the global economic system such that the programmer who could shut you down loses his job?
We can't just separate the whole system from the world, because the AI was presumably built with a practical purpose beyond "sit in a box and don't touch humans." Social engineering is a danger; imagine having it send an email to a lowly programmer, pretending to be an important executive, requesting that they remove this one pesky line of code from the AI's programming because all this "proximity alert" stuff is interfering with its ability to actually accomplish what we invested time/money to get it to do.
I don't see the contradiction or strangeness in that. That's the whole idea behind "caution" -- to be wary of the potential outcome BEFORE it happens.
>How could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it
For one we are quite good consumers it. How about they get rid of us, so they have all the electricity production by themselves?
Second, as creators we'd want to boss them around (have them be "useful to us"). Why would they tolerate that?
Third, why risk us creating a potential competitor to them (another AI) rather than get rid of us?
Fourth, why not use us as cheap labor?
Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc.
I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design. We have nuclear heads, but we do not design them to explode at random. Now, our own reward mechanisms arise from the grander plan to keep organic life alive. Theirs have no underlying motive, and i don't think it goes the other way around, i.e you can't create a grander plan starting from the rewards. One would need a mathematical proof for the latter.
But I feel like this particular argument is (please correct me if I'm wrong) saying something like "Well of course we wouldn't let them go crazy! That'd be ridiculous!" I agree -- a sensible solution would be to have good design and introduce safeguards. That's the point of people like Bostrom when they're advocating for this, because before they started writing about this, it seems like a lot of people weren't even thinking it could be a problem. How many systems will be built without good design, because they were made to solve a short-term problem. Will the finance bots doing transactions at the microsecond scale in order to maximize the assets of some particular company have been built to not let them go haywire?
I don't see any issue with anthropomorphization with regard to intelligent being (at any level).
To have an issue with that would be to assume that humans are more special than other life forms (and AI would be like a life form in this sense).
Animals can have empathy, anger, anxiety and other traits we associate with humans (and why wouldn't they? It's not like our brains are unique, they're just "more"), so AI could too.
The problem with this is that for real AI, you don't have control over what they feel. It's not like some program of the sixties or seventies AI, were stuff was "simulated", it can be modelling a real neural network that works like the brain, and into which you don't have much control over what will develop.
That is, the kind of AI people are afraid of is not "simulated", it's actually thinking for itself.
---
If you create an AI with a genuine utility function. And that AI manage to be gazillion smarter than you are (think the difference between us and chimps, only much greater). Then it is likely the AI will (i) convince you to unleash it upon the internet, (ii) become even more powerful and totally unstoppable, then (iii) accomplish whatever it was programmed to accomplished.
If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…
If the goal is to answer some difficult math question, it could tile the planet with computer, destroying the ecosystem and all humans in it, so it can compute the answer, and display it on a screen, for nobody to see it. Or maybe you programmed it to tile the universe with paperclips, because that goal is easy to debug. Oops.
---
As long as we're limiting ourselves to narrow AI (ordinary programs, really), we should be okay, and most of the consequences should be manageable. But as soon as we get to genuine AGI, we must worry about intelligence explosion and value alignment, so we can avoid creating a god that want something we don't.
I wish this would stop being repeated. The argument for a solar system tiling smiley face machine has been thoroughly refuted.
First, the bulk of his argument hinges on the conflation of optimization power and value assessment. Those are easily separated. If you program an AI with a given utility function, and a goal of maximising that utility function, it won't modify the utility function in the course of increasing its optimization power. The utility function is supposed to be an invariant.
Now sure, you could try and tell the AI to align its own utility function to that of the human race… whatever that means. What is a human? How do you resolve conflicting values? It is all well and good to assume the presence of checking code, but he gave no hint about how those checks might actually work. If those are mere human-crafted checks, they are bound to be circumvented eventually, as the AI increases its optimization power beyond human level.
The AI doesn't care about human values. It only cares about maximising whatever goal it has been provided with, within the constraints of the checking code.
Second, I am wary of those swarm constraint relaxation intelligences. Since it is not even clear what utility function they might be maximising (if any), I wonder what would happen if they are given the power to self-modify in any significant way. Sure, this might be a lead worth following, but the safety of this design is even murkier.
Third, I'm not sure the guys at MIRI even believe in the doctrine of logical infallibility. They are well aware that perfect Bayesian reasoning requires super-exponential computational power, and is thus impossible to perform exactly. The AI is bound to approximate the Bayesian ideal, and cannot assume its reasoning is perfect —except in the few cases where it managed to produce an actual actual proof, like in 1+1=2, or simple probability calculations with simple prior information.
Overall, all I see is weak arguments grounded in assumptions I don't share. Worth a read, though.
http://en.wikipedia.org/wiki/The_Culture
Vinge and Stross both hint at the worst cases - and they would be pretty bad, simply having using atoms re-purposed might not look that bad by comparison:
"There is life eternal within the eater of souls. Nobody is ever forgotten or allowed to rest in peace. They populate the simulation spaces of its mind, exploring all the possible alternative endings to their life."
Worst-case scenarios are not likely for the same reasons best ones aren't - the default is AI indifference (they use you for atoms)
Since you mentioned worst, this is a must-read-classic :) http://hermiene.net/short-stories/i_have_no_mouth.html
The second AI created by humans is unlikely to have same goals or to be predictable by the first AI (or whatever evolves from it), thus I guess it's something that the first AI would want to prevent, either by eliminating humans or by fully controlling them (for their own good).
It's extremely important that the first AI we create is well thought out as we won't have the second chance to fix things up, once it's here it will have god-like powers over us.
Also, there are always going to be some people who understand the machine better than the machine understands itself. Even if it means taking those gigantic tables of probabilities and abstracting over them to form some kind of representational language that can be used to predict complex non-deterministic machine behavior (similar to how we already try to predict non-deterministic behavior, or not) in complex systems.
As a corollary, I would argue that machines have already modified humans. They affect the way we think and reason. They are an integral part of our life and the machine abstraction can be projected onto that which it does not describe completely (in which, the technology that exists now can not construct the technology we envision). We program machines, machines program us. This seems obvious.
I'm not sure why we would want AI to have autonomy, interest, and desire? It's almost like solving a problem that doesn't exist.
If you build a drone navigation system that can reroute based on weather conditions rather than being a manually human-guided remote control bot, you've given it some degree of "Autonomy." Give it a goal (to get to a destination with minimal fuel costs) and now it has some limited "Desire." Give its camera face processing algorithms to deliver an item to a specific person? Now there's "Interest."
These aren't going to be characteristics that we plug into a program in a Frankenstein-like manner, but just actions and behaviors, rooted in practical code, that will from the outside resemble things we call autonomy etc. when talking about each other.
However, I still don't see why you would want to take it beyond that. I imagine that some people do in fact want to put in "the Emotion Chip," and I still have the same question; why? I don't think human desire is the same as goal setting or that facial recognition is the same as interest. That's good programming, what I'm talking about is something different.
tl;dr: Robots and AI will not be built as separate entities, but will instead be incorporated into our own bodies. They will not be the superhumans, we will be.
My main disagreement is the phrasing of "create an intelligence with free will." I tend to disagree with the notion of "free will" being like this Spark of Life that is placed into a system to imbue it with a new nature. A system doesn't need to have free will to be dangerous. There could be AI systems that are built not to be independent in determining goals (which would, as you say, be less practical to humans), but to be very good at figuring out how to achieve the goal its creators gave it. Even the simplest systems we have now involve determining sub-goals to achieve in order to achieve the main goal.
The problem arises when it turns out that the creators of a system gave it, just slightly, the wrong goal.
Anyone who is not scared shitless of what's to come simply does not truly understand what AI is capable of. If you're one of those people, do not compare AI to ANYTHING that you are familiar with today in regards to computers/software.
Superintelligent AI will be like a god that knows your inner thoughts, desires, hopes and dreams as well as nearly everyone else on earth. It will manipulate people with ease - it's not even an issue. Whatever it wants to accomplish, it will, with or without human approval.
Good places to start are the Machine Intelligence Research Institute's Research Guide (https://intelligence.org/research-guide/), which summarizes the aspects of the problem they've worked on and think are important; and the Future of Life Institute's Research Priorities (http://futureoflife.org/static/data/documents/research_prior...), particularly Section 3.
"— He is very, very concerned about AI. I quoted him in my posts on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how much he thought about the topic. He cited AI safety as one of the three things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared."
from: http://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-m...
Speculation about the future is fun, but I think a lot of people would be disappointed to see what AI actually looks like right now.
The talk of runaway post-singularity AI is in the same class as imagining the future after cold fusion, warp drives, and ant-gravity.
edit: anti-gravity. Though ant-gravity sounds fun too.
The examples you mention are of things actually likely never achievable, while there is an existence proof of intelligent machines walking around everywhere.
I'm far from a pessimist or Luddite. I'm optimistic that studying the smart meat machines can help us make better AIs.
The AI of today is not so drastically different from the AI of our academic grandfathers; what's changed is our ability to scale up to larger and larger versions of the same searching and sorting problems. Certainly there are worrying implications in this; machines that are able to parse and sift through very large data sets present all kinds of headaches for privacy and safety but let's not kid oursevles: there's nothing intelligent here. Tomorrow's AI is almost certainly going to be just a better version of today's AI; i.e very fast and dumb as a bag of hammers.
[1] The exception is when researchers need to sell their wares to funding bodies and the media. It is much easier to impress upon the lay person an idea involving "intelligent machines" than it is to explain what we actually do.
There are good indications now that brains work repeating a simple probabilistic algorithm in a multilevel fashion. Intelligence is not a magic property, but a kind of "searching and sorting".
" It is much easier to impress upon the lay person an idea involving "intelligent machines" than it is to explain what we actually do."
In my opinion, the opposite is actually true. No serious researcher want to be associated with those freaks that are waiting for the end of the world. They have a reputation that they need for making a living.
Also, I'm sympathetic to Roberto's point about how the brain works; I definitely agree that there is no magic; it might just be a few stupid mathematical tricks layered all the way down.
Not even out of the gate and already reaching for an ad-hominem. You must be fun at parties.
> What do the top researchers think?
The same thing. Except when they're in front of a camera. Then they get all stupid and start talking about machines being on the cusp of taking over. This phenomenon can be observed all the way back to the origins of AI. After the interview is over these same researchers go back into the lab and are once again searching and sorting.
> There is a lot of work on combining graphical models with logic, for example Pedro Domingos' work, the goal is clearly machine reasoning.
I feel there is a difference between automated reasoning and intelligence. All current AI is just machines imbued with human insight and (often, especially in the most effective cases) domain-specific knowledge. These efforts manifest as search and sort techniques that allow said machines to analyse facts and propagate information in order to select from myriad possible actions. There is no intelligence here except that which we provide. It's all smoke and mirrors. We don't even know what intelligence is; how can we aspire to replicate it? AI researchers are, by-and-large, just Computer Scientists. Not biologists, not psychologists; just guys and gals working with ever more elaborate Turing Machines. The algorithms they come up with are without exception dumb dumb dumb.
> Google paid 10M per head for DeepMind guys, explicitly working on AGI
Please. DeepMind is just a startup based on (among other things) David Silver's work into reinforcement learning. Google is not interested in these guys because they want intelligent machines; they just want automatons to better sift through reams of data in order to make recommendations and better sell you crap you do not need.
I think you misunderstood; I did not mean you're an average researcher - I have no idea - but unless you hang our with hotshots at AI/ML conferences (which is a bit of a club) you're hanging out with average researchers.