I was so sure this would be about the paperclip maximizer:
"The paperclip maximizer is the canonical thought experiment showing how an artificial general intelligence, even one designed competently and without malice, could ultimately destroy humanity."
Yeah, but the paper-clip maximizer theory is pretty awfully flawed. It rests on the idea that a generally human-level of intelligence necessarily leads to a super-human explosion of intelligence.
>It rests on the idea that a generally human-level of intelligence necessarily leads to a super-human explosion of intelligence.
Why must it be necessary? If there's a large chance, that's still a problem. Besides, the claim is generally that a human-created intelligence is unlikely to be in the same range as humans (because there's no reason to think that the human range on the spectrum is unique), so if it's not dumber, then it's most likely going to significantly smarter. There's also the point that if you have a human-level AI, simply throwing more computing power at it makes it strictly faster than humans.
How does the existence of a paperclip maximizer rest on the assumption of recursive self-improvement? Making nanotech weapons to stop humans from shutting you down and thereby reducing the expected number of paperclips doesn't require exponentially increasing amounts of intelligence. It only requires a fixed amount of intelligence.
Also you should justify "this is clearly [1] not the case". You're probably taking the "necessarily" phrasing really literally, instead of as "with non-negligible risk", and thinking of corporations or evolution as "superhuman"? On the other hand, evolution and corporations respectively did and plan to produce smarter entities so that's not so clear cut.
10 comments
[ 699 ms ] story [ 1164 ms ] threadThis YouTube video [0] seems to be the source of the GIF, or at least the full version. Maybe the mods should change the link to it?
[0]: https://www.youtube.com/watch?v=OsDdmDFDYHA (skip to 2:10)
"The paperclip maximizer is the canonical thought experiment showing how an artificial general intelligence, even one designed competently and without malice, could ultimately destroy humanity."
http://wiki.lesswrong.com/wiki/Paperclip_maximizer
beware!
This is clearly not the case.
Why must it be necessary? If there's a large chance, that's still a problem. Besides, the claim is generally that a human-created intelligence is unlikely to be in the same range as humans (because there's no reason to think that the human range on the spectrum is unique), so if it's not dumber, then it's most likely going to significantly smarter. There's also the point that if you have a human-level AI, simply throwing more computing power at it makes it strictly faster than humans.
See http://intelligenceexplosion.com/en/2011/plenty-of-room-abov... and http://www.nickbostrom.com/superintelligence.html for some examples of these arguments.
Also you should justify "this is clearly [1] not the case". You're probably taking the "necessarily" phrasing really literally, instead of as "with non-negligible risk", and thinking of corporations or evolution as "superhuman"? On the other hand, evolution and corporations respectively did and plan to produce smarter entities so that's not so clear cut.
1: https://medium.com/science-and-technology/how-to-spot-a-weak...