My first reaction was that if you're going to live forever, why rush to graduate, there's plenty of time. Then when I read the article, I saw that they are looking at all kinds of other opportunities. Good stuff.
this is why I don't understand the "nerd utopia" line. it seems that the vast vast majority of possible non-human intelligences would have goals directly harmful to human goals (even modest ones). It seems like the best we can hope for is a really nice zoo.
I would say that the best we can hope for is that intelligence is so hard to understand that we get uploading first, and therefore the vast majority of non-human intelligences are previously-human intelligences -- that is, us.
The second-best possibility is that intelligence is possible without consciousness ("really powerful optimization processes") and no one creates people AI before we are adept at using RPOPs to further our human goals.
Both of these seem like wishful thinking, unfortunately. I don't have any idea how to push the future toward one of these outcomes.
If you said that about a topic you're familiar with, it sounds silly: it seems that the vast vast majority of possible computer programs would have goals directly harmful to human goals (even modest ones).
The vast majority of possible medical treatments, proteins, DNAs, etc. would be directly harmful to human goals, and yet we keep on trying, cognizant and wary of these ramifications.
Granted, there are "irresponsible" researchers who don't understand that an AI might not have "gratitude" that prevents it from rm -rfing humanity like a bad shell script, but others work on 'Friendly AI' to mitigate or eliminate this concern.
A recursively self-improving intelligence seems far more likely to be an existential threat than virtually any other human creation. Even gray goo can be fought fairly easily with heat (yes, "fight it with fire" will be the slogan of the First Goo War... ;)), but if a hard takeoff is possible, the first AI will have nothing to oppose whatever its goals happen to be. The touted solution to this is Friendliness, as you note, but Friendliness appears to be a problem in the same class of difficulty (or more so) than AI itself, even if workable at all, which is uncertain. In my opinion, people who choose to work on Friendliness as a prerequisite to AI are simply guaranteeing that AI will be developed by someone else first.
the things you mentioned would be harmless, not harmful. at worst they would waste some resources. they certainly wouldn't requisition resources by force (unless you want to talk about ideology as a "technology" :)
Er, prions, viruses, bacteria, runaway breakout of monoculture -- these things use up resources, and have the potential to cause great harm. It's just that few of them are likely to rise to the level of existential threat, while in the field of recursively self-improving AI, the likelihood is strongly the other way.
This isn't a person attack. It's just a general observation.
People seem to vastly oversimplify Kurzweil's (and others) stance on the singularity. The truth is that we don't know what will happen. The best one can do is prepare for possible senarios.
Is is just me or did the project seems a bit mundane? The introduction is about biotech and nanotech, etc, and the projects are mobile apps, scaling up 3D printing, and smart traffic. Anyone else underwhelmed? Maybe I had unreasonable expectations.
Kurzweil articulated many of his early ideas on Singularity at an AI symposium held in 2001. RealPlayer video of all the sessions used to be available through Dr. Dobb's Journal's TechNetCast feature. I watched them all and they were very, very interesting -- both for and against. Great stuff.
I tried to locate a link to them for this post but TechNetCast seems no more and DDJ's site doesn't bear fruit either. If anyone here knows where they might be found please share -- they are definitely worth the time.
Since then, Kurzweil has certainly pursued his beliefs (books, speeches, press, etc.) with these graduates being the latest expression of those ideas.
I tend to agree more with Eliezer Yudkowsky and Michael Anissimov than Kurzweil about this stuff, but the video above is still worth watching (much much better than his TED video, which was too short to allow him to make his main points).
17 comments
[ 3.6 ms ] story [ 54.7 ms ] threadThe second-best possibility is that intelligence is possible without consciousness ("really powerful optimization processes") and no one creates people AI before we are adept at using RPOPs to further our human goals.
Both of these seem like wishful thinking, unfortunately. I don't have any idea how to push the future toward one of these outcomes.
Granted, there are "irresponsible" researchers who don't understand that an AI might not have "gratitude" that prevents it from rm -rfing humanity like a bad shell script, but others work on 'Friendly AI' to mitigate or eliminate this concern.
http://en.wikipedia.org/wiki/Friendly_artificial_intelligenc...
http://en.wikipedia.org/wiki/Friendly_artificial_intelligenc...
http://online.sfsu.edu/~rone/GEessays/killervirus.html
People seem to vastly oversimplify Kurzweil's (and others) stance on the singularity. The truth is that we don't know what will happen. The best one can do is prepare for possible senarios.
I tried to locate a link to them for this post but TechNetCast seems no more and DDJ's site doesn't bear fruit either. If anyone here knows where they might be found please share -- they are definitely worth the time.
Since then, Kurzweil has certainly pursued his beliefs (books, speeches, press, etc.) with these graduates being the latest expression of those ideas.
http://mitworld.mit.edu/video/327/
I then read The Singularity is Near.
I tend to agree more with Eliezer Yudkowsky and Michael Anissimov than Kurzweil about this stuff, but the video above is still worth watching (much much better than his TED video, which was too short to allow him to make his main points).