The proof seems sound, but the premises appear to me to be overly restrictive. In particular, seeing that a ML-based AI can write arbitrary code, there's nothing limiting the ability of these 2nd generation AIs from being AGI.
> Yet, as we formally prove herein, creating systems with human(-like or -level) cognition is intrinsically computationally intractable
This would involve proving that humans exceed the Turing computable, which would mean proving Church-Turing thesis is false.
Because if humans do not exceed the Turing computable, then every single human brain is existence-proof that AGI is intrinsically computationally tractable by demonstrating that sufficient calculations can be done in a small enough computational device, and that the creation of such a device is possible.
Their paper accepts as true that Turing-completeness is sufficient to "computationally capture human cognition".
If we postulate that this is true (and we have no evidence to suggest it is not), then if their "proof" shows that *their chosen mechanism can be proven to not allow for the creation of AGI, then all they have demonstrated is that their assumptions are wrong.
I'd like to "reclaim" both AI and machine learning as relatively emotionally neutral terms of art for useful software we have today or see a clearly articulated path towards.
Trying to get the most out of tools that sit somewhere between "the killer robots will eradicate humanity", " there goes my entire career", "fuck that guy with the skill I don't want to develop, let's take his career", and "I'm going to be so fucking rich if we can keep the wheels on this" is exhausting.
I don't think that's achievable with all the science fiction surrounding "AI" specifically. You wouldn't be "reclaiming" the term, you'd be conquering an established cultural principality of emotionally-resonant science fiction.
Which is, of course, the precise reason why stakeholders are so insistent on using "AI" and "LLM" interchangeably.
Personally I think the only reasonable way to get us out of that psycho-linguistic space is just say "LLMs" and "LLM agents" when that's what we mean (am I leaving out some constellation of SotA technology? no, right?)
> Yet, as we formally prove herein, creating systems with human(-like or -level) cognition is intrinsically computationally intractable.
Well, pregnant women create such systems routinely.
Due to the presence of the weasel word "factual" (it's not in the sentence I quoted, but is in the lead-up), no contradiction actually arises. It may well be intractable to create a perfectly factual human(-like or -level) AI -- but then, most of us would find much utility in a human(-like or -level) AI that is only factual most of the time -- IOW, a human(-like or -level) AI.
6 comments
[ 2.5 ms ] story [ 25.8 ms ] threadThis would involve proving that humans exceed the Turing computable, which would mean proving Church-Turing thesis is false.
Because if humans do not exceed the Turing computable, then every single human brain is existence-proof that AGI is intrinsically computationally tractable by demonstrating that sufficient calculations can be done in a small enough computational device, and that the creation of such a device is possible.
Their paper accepts as true that Turing-completeness is sufficient to "computationally capture human cognition".
If we postulate that this is true (and we have no evidence to suggest it is not), then if their "proof" shows that *their chosen mechanism can be proven to not allow for the creation of AGI, then all they have demonstrated is that their assumptions are wrong.
Trying to get the most out of tools that sit somewhere between "the killer robots will eradicate humanity", " there goes my entire career", "fuck that guy with the skill I don't want to develop, let's take his career", and "I'm going to be so fucking rich if we can keep the wheels on this" is exhausting.
And the cognitive science thing.
Which is, of course, the precise reason why stakeholders are so insistent on using "AI" and "LLM" interchangeably.
Personally I think the only reasonable way to get us out of that psycho-linguistic space is just say "LLMs" and "LLM agents" when that's what we mean (am I leaving out some constellation of SotA technology? no, right?)
Well, pregnant women create such systems routinely.
Due to the presence of the weasel word "factual" (it's not in the sentence I quoted, but is in the lead-up), no contradiction actually arises. It may well be intractable to create a perfectly factual human(-like or -level) AI -- but then, most of us would find much utility in a human(-like or -level) AI that is only factual most of the time -- IOW, a human(-like or -level) AI.