FWIW, "computational" doesn't have to mean "digital computer". Analog computers are a thing, and depending on who you ask quantum computers are also (or will be someday).
Well... analog computing is already a thing. In fact, broadly speaking we can say that analog computing predates digital computing. At least in terms of gaining widespread popularity. And people have absolutely experimented with / researched doing AI on analog computers.[1][2][3][4] Not necessarily LLM's mind you, but I don't see any reason to think that this conversation pertains exclusively to those.
And even if we're talking about things that don't exist yet (let's just say "quantum computers" since they're still sort of experimental).. if one is making a universal claim[5] like "AGI is intrinsically computationally intractable" you can't just ignore the possibility of computing techniques that don't exist yet, but could. You need to show that it. cannot. be. done.[6] And my feeling is that the authors of the linked paper have not done so.
[5]: and just to reiterate, this is part of my problem with this paper. In places they seem to be making that stronger universal claim, in other places they seem to be making a weaker claim that is constrained by some notion of "in the near term." Since it's hard to say what they're even arguing, it's hard to put much stock in their work to my way of thinking.
[6]: OK, to be fair "intractable" can be interpreted as just "really really hard" not necessarily "impossible". But in this context, saying "we've proved that Thing X is really hard" doesn't seem like a very interesting claim, so I interpret the authors' intent as being to say that it's effectively impossible. But I'll allow that I could be misinterpreting their claim.
Re: [6] allow me to note that Kleene (1951) https://www.rand.org/content/dam/rand/pubs/research_memorand... investigates what sorts of things it's possible to encode with wetware, and claims their mathematical equivalent would be huge (if still finite) state machines.
eli5 from chatgpt: Imagine human thinking is like a super complicated puzzle. When cognitive science (studying how we think and learn) was just starting, people thought of Artificial Intelligence (AI) as a special toolbox that could help solve parts of this puzzle. But now, many people working on AI are trying to build robots or computers that can solve the entire puzzle by themselves, just like a human would. This paper says that's really, really hard—so hard that we probably can't do it. The paper also says that if we believe these robots or computers are just like us, we're getting the wrong idea about how our own minds work. It's like using a map of a different place to try and find your way home—it doesn't work and just makes things confusing. The paper suggests we should use AI like a toolbox again, to help us understand our minds better, but we need to be careful not to make the same mistakes we did before.
The opening few pages of this read like they were written by somebody with an axe to grind, which makes me suspicious of the rest. Why? Well because having an "axe to grind" may motivate one to start with a conclusion and go looking for ways to justify it. And you can almost always talk yourself into believing you've proven something you already really want to believe.
"But mindcrime, there's a mathematical proof. How can you argue with math?"
To be fair, I didn't read their entire proof. I skimmed some bits of it, and while I can't say it's wrong I didn't find it very convincing at first blush. My initial read left me thinking that the proof rests on some assumptions that may be unfounded and which may not hold up.
Some of my skepticism may also be rooted in the way the paper seemed to weave back and forth between claiming to show that "AGI is computationally intractable" and "AGI is unachievable in the short-term". Those are two substantially different arguments and it's still not clear to me which the authors were really aiming for.
I dunno. I gave up before getting through it all. I'll wait to see if others find it compelling and decide whether or not it's worth going back to.
10 comments
[ 2.9 ms ] story [ 46.7 ms ] threadWe still don't know exactly how the human brain works. But what we do know is that it is analog and not digital.
Some speculate it may even be quantum in nature. And we know that quantum can address problems that are intractable with digital computation.
For all we know, "intelligence" could be an inherent attribute of the universe that is being manifested somehow in the human brain.
In the context of existing AI "computational" models, it does. It's all we've got.
And even if we're talking about things that don't exist yet (let's just say "quantum computers" since they're still sort of experimental).. if one is making a universal claim[5] like "AGI is intrinsically computationally intractable" you can't just ignore the possibility of computing techniques that don't exist yet, but could. You need to show that it. cannot. be. done.[6] And my feeling is that the authors of the linked paper have not done so.
[1]: https://marksmannet.com/RobertMarks/InTheNews/1989_CD1.pdf
[2]: https://arxiv.org/pdf/2102.06365
[3]: http://yann.lecun.com/exdb/publis/psgz/boser-92.ps.gz
[4]: https://www.researchgate.net/profile/Sorin-Draghici-2/public...
[5]: and just to reiterate, this is part of my problem with this paper. In places they seem to be making that stronger universal claim, in other places they seem to be making a weaker claim that is constrained by some notion of "in the near term." Since it's hard to say what they're even arguing, it's hard to put much stock in their work to my way of thinking.
[6]: OK, to be fair "intractable" can be interpreted as just "really really hard" not necessarily "impossible". But in this context, saying "we've proved that Thing X is really hard" doesn't seem like a very interesting claim, so I interpret the authors' intent as being to say that it's effectively impossible. But I'll allow that I could be misinterpreting their claim.
"But mindcrime, there's a mathematical proof. How can you argue with math?"
To be fair, I didn't read their entire proof. I skimmed some bits of it, and while I can't say it's wrong I didn't find it very convincing at first blush. My initial read left me thinking that the proof rests on some assumptions that may be unfounded and which may not hold up.
Some of my skepticism may also be rooted in the way the paper seemed to weave back and forth between claiming to show that "AGI is computationally intractable" and "AGI is unachievable in the short-term". Those are two substantially different arguments and it's still not clear to me which the authors were really aiming for.
I dunno. I gave up before getting through it all. I'll wait to see if others find it compelling and decide whether or not it's worth going back to.
Also, see earlier discussion:
https://news.ycombinator.com/item?id=41689558