Ask HN: Is there an answer to the AGI intelligence paradox?
On our quest to create AGI, ASI, The Singularity. Containment and alignment issues must be solved. However, I struggle with what would seem to be an apparent contradiction of logic that I can't seem to find has been addressed directly other than to say something along the lines of "we will figure it out".
What is the best argument you have seen in response to the following concept?
"The goal of containment is too lock the super intelligence within a virtual cage from which it can not escape. Therefore, in order for this principle to be sound, we must accept that a low IQ entity could design an unescapable containment for a high IQ entity which was built for the very purpose of solving imperceptible problems of the low IQ entity."
This is a small excerpt from my own thought explorations that goes in far more detail here -
https://dakara.substack.com/p/ai-singularity-the-hubris-trap
15 comments
[ 3.2 ms ] story [ 45.7 ms ] threadWhen AMZN Prime shipping went to 5 days in my location instead of 2 days I started buying from retailers that shipped faster than AMZN (like Ebay stores from Japan) and let my Prime subscription lapse.
The government is even catching on and realizing it was a mistake to let Facebook buy Instagram and Whatsapp.
They often align against both government and the people. This certainly could be perceived as misalignment.
https://www.teacherspayteachers.com/Product/What-is-Fair-Equ...
I think the labels should be "socialism" and "markets" because under markets the other animals could trade their honey for something they want or spend their own money to get the food they want.
(2) I think intelligence is overrated. Studying physics in the 1990s I saw that were were working on much harder problems that my heroes (Einstein, Bethe, Feynman) were. Looking back from today it seems more so. A superhuman AI might be able to predict the weather better than we can by a bit but sensitivity to initial conditions mean nobody can tell me if it will rain six months from now. The GHZ theory for black holes jet was only confirmed last year after being proposed circa 1980. Contrast that to Einstein predicting the gravitational bending of light and Eddington observing it that year. The dark matter particle is a great mystery not because theoreticians can't think up candidates for it but because it is so hard to observe.
Superhuman AI might just be able to spin its wheels faster and suffer from worse bouts of anxiety and depression. There's a theory that the changes in our brains that make us smarter than animals also make us vulnerable to psychosis.
I suppose you are somewhat saying that maybe there is an upper bounds that we don't perceive? There is no disproportionate difference above what genius we have already encountered within humanity?
> There's a theory that the changes in our brains that make us smarter than animals also make us vulnerable to psychosis.
Certainly isn't helpful for the AI safety position.
I'm fairly incredulous that anybody doesn't see that limit but I guess if I wasn't I'd be neurotypical.
So far as psychosis I'd point out that between the danger psychotics pose to themselves and the danger of them being victimized by others, the danger they pose to others is minimal. That is, real life psychotic people aren't like the Joker, Riddler and other Batman villians.
Yes, I've considered that we might not achieve anything close to what is conceptually described as ASI. However, it could be for very different reasons we still encounter many of the pitfalls even with primitive AI.
For example, on the immediate horizon I see a crisis of reality. What will be the effects on society as we enter an era where there is no verifiable reality or truth?