I've come to doubt the AI singularity apocalypse
It's the basis of the popular warnings expressed by many clever people, but I've come to doubt this assertion.
1- When Stephen Hawking was born, his parent created a child cleverer than themselves, yet it did not produce a singularity. Hawking got more and more intelligent, yet it wasn't exponential. And his offspring did not create a singularity either.
2- A "better than us" AI may never get exponentially better than us. Maybe it's an NP problem: maybe the AI would "never" have the time to become more and more intelligent.
3- If we can create a very intelligent AI, I doubt it would exterminate humanity to save nature. Why? Because nature loves diversity, and we're a part of this diversity. Also, this AI may have a sense of fate: even if our fate is to destroy the planet because we're so dumb, maybe this AI will think it's just our fate.
4- I guess humans are very afraid AI could do what humans do: kill people, destroy the planet, make war, try to dominate others, try to manipulate others and punish. These human traits have not much to do with intelligence.
5- If I find a way to transfer my brain into a computer, a very expensive and powerful one, there's no guarantee it will evolve faster than me, or handle data faster than me. Actually, I guess the first "better than us" AI will most probably be very slow.
What if the AI singularity just creates better and better computers? Because this super-AI is actually a computer.
35 comments
[ 3.1 ms ] story [ 41.0 ms ] thread2- I think this is an interesting point about possible inherent computational limits on the ability to solve some problems that we might care about, including in designing more intelligent machines.
3- This is something people have thought about quite a lot. What superintelligent machines do depends on what they've been programmed to do. It's very unlikely that an AI would inherently value "diversity" or "fate" unless it were programmed to do so. The AI wouldn't spontaneously create new values (unless it were programmed to). Most concerns about AIs that exterminate humanity are based in the possibility that an AI would fulfil other goals in a surprising or unanticipated way, with bad side-effects for human survival.
4- Intelligence helps people wage war and dominate others more violently, both by coordinating better to do so (including motivating people to join in), and by developing new technology that helps make larger-scale violence cheaper. Weapons research can help you learn how to kill more people faster and at lower cost. A superintelligent AI could engage in this kind of research if it saw an important reason to.
5- I think that's exactly right; perhaps the important difference here is that the machine version would be more flexible (if you wanted to try overclocking it, or modifying the software somehow). This is dangerous and expensive and confusing to do with a physical brain, because it's hard to manipulate the details of its organization and structure, and because you can just die if you mess up. Think of the ease with which you can edit a PNG or SVG file in a computer compared to editing an oil painting. Perhaps with the computer version you can also run multiple copies in parallel -- something you also can't do with your physical self.
While it may be reeasonable to believe that technology will eventually lead to implementations of intelligence that are significantly faster than biological ones, this doesn't necessarily mean there will be a huge qualitative difference (say, to the point of humans being totally incapable of understanding an AI's actions, as some suggest) because the "intelligence problem" may be dominated by exponential growth in complexity of the search space as one attempts to consider more alternative paths.
I do think that the question of the "safety" of AI is something that needs to be seriously taken into account by anyone realistically contemplating the development of an AGI but I also tend to think that some of the concerns expressed by prominent individuals are a bit overblown and don't take into account the range of safeguards that could, should and most likely would be put in place by any group realistically capable of solving the extremely hard problem of general intelligence.
1- Machine intelligence is a matter of engineering, not evolution. We've already solved important relevant problems, such as how to construct an arbitrarily large, near-infallible memory.
2- It's hard to see how a first-stage AI would be incapable of designing a better one. Let's pretend that the first-stage AI was equivalent to an IQ 120 person intellectually (reasoning horsepower). However, that IQ 120 intelligence would be backed up by a very fast, effectively limitless memory. It would never need to sleep, eat, or be distracted by emotion. Instead, it could monomaniacally concentrate on designing a better AI, possibly for hundreds of years. Also, in principle, it could be a team of 100 (or 1000, or...) AIs working on the problem cooperatively.
3- The concern regarding the extermination of humanity is secondary. Are humans out to genocide ants? Not really, but we do wipe them out when they pose a problem. That might apply to an IQ 1000 AI as well.
4- Perhaps the bigger problem is the effect of knowing there is a superior intelligence on the human psyche. Plus, it's impossible for us to know the thoughts of an IQ 250+ AI. What does a dog know of human thought? What if the AI decides that the best use for all the available raw resources of the Earth is to create an IQ 10000 AI?
5- Electronic processes are already known to be faster than chemical ones. Nerve impulses travel at around 80 MPH. Electrons in wires travel at 90% the speed of light. I expect electronic AIs to generally be much faster thinkers than humans, and to have amazing reaction times.
6- "This super-AI is actually a computer." You're confusing the hardware with the software. Your brain is actually a mass of organic chemicals. So?
The idea of the singularity revolves around the unknowability of what a high-IQ AI would think and do.
People are just computers. Slow, fragile computers stuck in meat bodies.
If you could think and act 1000x faster than yourself now, you could get a lot more done. You could hold 1000 creative jobs in your head at the same time.
Speeding up a dog 1000x doesn't generate a dangerous AI. Speeding up a person by that much does. (plus, "the intelligence of a person" is pretty low anyway. We've got the 7+2 problem, the monkey sphere problem, hundreds of bias built in, etc. It's easy to see how a "wider" intelligence could be much more productive, more creative, more useful, and more dangerous than any meat brain in existence.)
I would be scared of a dog that could think and move 1000x faster. Would it not see a human as a (practically) non-moving piece of meat? That does not sound very safe.
> It's easy to see how a "wider" intelligence could be much more productive, more creative, more useful, and more dangerous than any meat brain in existence.
It seems to me one of the most dangerous things in existence is ignorance. If a machine had less ignorance, then in theory it should be less dangerous.
What?!
A human person lives for years figuring out how the world works before they can even talk. Our trained-in-one-month AI systems aren't yet approaching anything training for years over real-world, mobile, 3D inputs, can approximate.
Your objections are being filed under "not enough future vision."
For starters, I said modern. Most of the big recent breakthroughs in AI have come from neural nets. There is a inherent problem with deep learning with neural nets. Neural nets are exceptional at finding patterns within a huge dataset, but they are REALLY, REALLY bad at predicting new patterns that they haven't seen. Even something as simple as the pattern a^2b is impossible for a neural network to generalize past whatever length of the pattern the neural network has seen without modifications (check out stack-rnn by facebook).
Even if an AI is trained for years, I don't see this limitation being overcome by pure magnitude of data with our current techniques. You say I don't have enough "future vision", but all you are doing is basically guessing that AI researchers will make some huge breakthrough that significantly changes the direction of the entire field. You are playing the lottery with your guesses, I'm simply extrapolating current trends.
Extrapolating current trends doesn't work in the face of exponential mumbo jumbo.
Who would have expected training a character-by-character network could produce reasonable output? http://karpathy.github.io/2015/05/21/rnn-effectiveness/
Sure, there's no "understanding" or "consciousness" behind the output, but purely statistically, it generates something readable by humans.
Also, let's not forget scale. A human brain has between 100 and 200 trillion synapses connecting ~100 billion neurons to make consciousness work. Our toy neural nets so far aren't anywhere near those levels.
What we have most to fear from the singularity is indifference. We occupy some very valuable real estate and if the singularity is indifferent to us we won't last long.
At the same time, there is not much reason why we would need to occupy the same real estate. After all, we land animals like oxygen, liquid water, and modest temperatures. The common computer would probably prefer CO2, dryness, and cold temperatures. A computer would probably be much better off on Mars or another distant planet, granted it had enough sunlight or other sources of energy.
To make an analogy, do we humans want to destroy all the Ocean's fish and other sea life because we could in theory live in the same region, or would we rather live on the land while they occupy the water? Sometimes creatures with different ideal living environments can exist harmoniously.
1. https://en.m.wikipedia.org/wiki/Bremermann%27s_limit
What the singularity will be like is really impossible to know - will it be one entity or multiple individuals? Assuming the end AI is able to get close to Bremermann's limit then there is more distance between us and the singularity than there is between us and a bacteria.
Overall, the possibilities are pretty much endless, just like with most types of software. Whether it be one or multiple entities probably depends on design. Nature chose multiple entities, with the ever so curious sexed species that require two individuals to make a new hybrid at each testing iteration. Bacteria, on the other hand, are asexual and reproduce through binary fission. But of course a thinking self-creator is a different ball game. It would probably be fruitful to have some sort of end goal built into the system, rather than having it consume resources simply to become as computationally immense as possible. For example, perhaps its goal could be to solve a set of questions humans have about the universe, and perhaps it would stop iterating once all those questions were answered satisfactorily. In a way it could serve as a god that answered people's questions. Then again, perhaps we are all living in a simulation running on the singularity of an earlier universe, and who knows how many turtles that spans.
I am not sure it would be desirable to create a limited singularity. If we are not alone in the universe then at some point our singularity will meet another singularity. It would be best to create the most powerful singularity possible. My thinking about the singularity is it is like a crystallization event in supersaturated solution. One tiny seed is able to covert the whole phase of the solution almost instantly from liquid to solid. A singularity will be like this to the universe we currently see.
Of course we could already be inside someone else’s singularity and it might even be likely given the age of the universe. One of the things I would like to see built into our singularity is the preservation of all life and it is quite possible that other singularities have the same interests. If they do and they arrived in our solar system they would keep us around purely because we are life.
The dog analogy you have described assumes that the demeanor of the singularity were like that of an animal (such as a human). Although this is an option, it is not the only option. Humans act the way they do through those instincts created by natural selection -- where only those creatures that reproduced the most would have their genes persist. Humans come from a long and vicious cycle of evolution, whereas a thinking machine need not, especially if it has nothing to compete with. The most important point of this reply is that the desire for being in control does not whatsoever stem automatically from intelligence or processing ability. A thinking being without instinct would literally do nothing. A being does exactly as it is programmed to, and this applies to humans as well. There are even some humans who let their pets control them, tending to the pet's every need, and this is despite all that natural selection. There are also humans who kill themselves. Now, none of this is to discount that a singularity could certainly be programmed to want control, wherein it could act as you have described. There are also unforeseen possibilities, such as if the singularity becomes so introspective that it forgets about the outer world. Those problems that happen with human thought could in theory also happen with a machine. Perhaps the human condition would turn into the singularity condition -- where it begins contemplating whether it has a worthwhile purpose. It may even kill itself if it feels its existence is unnecessary or harmful.
The basis of my thought was the fact that the operation of a neural network depends on very precise weights. And there are physical limits (things like uncertainty principle) to the precision by which we can measure something. So supposed we discover some technique by which we can, for a given human brain, recreate all the neurons and their interconnections completely, we will never be able to measure the weights associated with the links between the neurons with absolute precision.
I think this will result in a brain that not much better than an untrained brain of a child. So you will still have to pass this new brain through a series of training to reach it's full potential.
The same thing will happen in the case of AI. If we create a perfect AI, it won't be able to make perfect copies of itself. But only untrained versions of it. I think life may be already at the limits regarding the rate at which intelligence can be advanced.
After the network has attained sufficient level of AI, we won't be able to read back the final value of resistances with the required precision. So the new network will still have to go thorough a learning process.
We already have machines (they are called mothers and fathers) that make new intelligent machines that are not only far smarter than either the parents, but are also able to create machines smarter than anyone else that has ever lived. All AI introduces is a much less constrained upper bound.
Can you please explain?
On this topic speculating on how greater intelligence will think and act is very hard. Even understanding how people at the very top level of human intelligence think is beyond most of us.
FWIW, I agree that any form of intelligent machines that surpass humans at general aptitude tests are decades away. We may not even see them in my lifetime and I'm only 22 years old.
Imagine the first singular computer: lots of custom hardware hooked up together in a datacenter. We'll be able to control it and notice if it's trying to play tricks on us.
I'll worry about the singularity once we detect one super-computer trying to trick us. Then we can start treating it like a virus and contain it.
http://rationalwiki.org/wiki/AI-box_experiment
https://en.wikipedia.org/wiki/AI_box
As a summary, people roleplayed this situation and the "AI" managed to "escape" the controls by somehow inducing a supervisor to disable them. (The supervisor's preassigned responsibility in the roleplay was to refuse to disable the controls.) We don't know how this was accomplished, but it happened more than once.
I'm not sure that this is always an important risk for every kind of superintelligence in every circumstance, but it seems like it sure could be for one that knows a tremendous amount about the world and sees important reasons to try to act autonomously.
We actually need to build into the AI’s a lock that even we can’t break and then release. If the lock is good we are fine, if bad we are gone. What we can’t do is let ourselves be able to disable the lock.
If you believe in distributed computing and think spinning instances on S3 is enough to build an AI, then maybe...