Cool that we are at a stage where it is meaningful to start measuring progress toward AGI. Something I am wondering on the philosophical side: are we ever going to be able to tell if the system really "understands" and "perceives" the world?
h) pattern recognition & inductive reasoning (this is the most primitive and universal expression of intelligence across species, the ability to extract regularities from noisy data, to generalized from examples to rules)
Those are crowdsourced benchmarks. We're calling them "cognitive" and "AGI" now, though. It's similar to when they made a benchmark and called it "GDP".
To be clear, I think we've seen very fast progress, certainly faster than I would have expected, I'm not trying to peddle some "wall" rhetoric here, but I struggle to see how this isn't just the SWE-bench du jour.
It still seems like something is missing from all these frameworks.
I feel like an average human wouldn't pass some of these metrics yet they are "generally intelligent". On the other hand they also wouldn't pass a lot of the expert questions that AI is good at.
We're measuring something, and I think optimizing it is useful, I'd even say it is "intelligent" in some ways, but it doesn't seem "intelligent" in the same way that humans are.
> I feel like an average human wouldn't pass some of these metrics yet they are "generally intelligent". On the other hand they also wouldn't pass a lot of the expert questions that AI is good at.
I think this approach is intentional. The philosophy is simply "extraordinary claims require extraordinary evidence". What you're saying is true, but producing a system that exhibits all human cognitive capabilities is a better threshold for the (absolutely wild) claim of the existence of AGI.
It's good to have some kind of benchmark at least to structure the ongoing, fruitless discussion around "are we there already?".
However I must admit that including the last point that is partially hinting at the emotional or rather social intelligence surprised me. It makes this list go beyond usual understanding of AGI and moves it toward something like AGI-we-actually-want. But for that purpose this last point isn't ok narrow, too specific. And so is the whole list.
To be actually useful the AGI-we-actually-want benchmark should not only include positive indicators but also a list of unwanted behaviors to ensure this thing that used to be called alignment I guess.
Maybe Google could actually make Gemini good instead of being about 10 miles behind Claude instead of trying to make AGI because of - well some reason - cause they want to be famous.
I'm sorry what even is this? Giving $10k rewards for significant advancements toward "AGI"?
What does "making a framework" even mean, it feels like a nothing post.
When I think of what real AGI would be I think:
- Passes the turing test
- Writes a New York Times Bestseller without revealing it was written by AI
- Writes journal articles that pass peer review
- Wins a Nobel Prize
- Writes a successful comedy routine
- Creates a new invention
And no, nobody is going to make an automated kaggle benchmark to verify these. Which is fine, because an LLM will never be AGI. An LLM can't even learn mid-conversation.
Why does your definition of "AGI" have to exclude nearly all humans? Wouldn't it still be "AGI" if it was as smart as the average human? Since when did AGI stop representing the words that make term? Artificial (man made) General (not specific) Intelligence. Is a human not "GI"!?
When people imagined AI/AGI, they imagined something that can reason like we can, except at the speed of a computer, which we always envisioned would lead to the singularity. In a short period of time, AI would be so far ahead of us and our existing ideas, that the world would become unrecognizable.
That's not what's happening here, and it's worth remembering: A caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language or technology, or any knowledge.
In Carolyn Porco's words: "These beings, with soaring imagination, eventually flung themselves and their machines into interplanetary space."
When you think of it that way, it should be obvious that LLMs are not AGI. And that's OK! They're a remarkable piece of technology anyway! It turns out that LLMs are actually good enough for a lot of use cases that would otherwise have required human intelligence.
And I echo ArekDymalski's sentiment that it's good to have benchmarks to structure the discussions around the "intelligence level" of LLMs. That _is_ useful, and the more progress we make, the better. But we're not on the way to AGI.
> caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language
There is evidence to the contrary. Not having language puts your mental faculties in a significant disadvantage. Specifically, left brain athropy. See the critical period hypothesis. Perhaps you mean lacking spoken language rather than having none at all?
Humans, like all animals, have not stopped evolving. A random caveman from 200K years ago would have very different genetics to that of a typical HN reader and even more so of the best of the HN readers.
Around 3,200 years ago there was a notable uptick in alleles associated with intelligence.
> A caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language or technology, or any knowledge.
Source? This does not sound possibly true to me (by any common way we might measure intelligence).
As an engineer who is also spiritual at the core, it seems obvious to me the missing piece: consciousness.
Hear me out.
I love AI and have been using it since ChatGPT 3.5. The obvious question when I first used it was "does this qualify as sentience?" The answer is less obvious. Over the next 3 years we saw EXPONENTIAL intelligence gains where intelligence has now become a commodity, yet we are still unable to determine what qualifies as "AGI".
My thoughts:
As humans, we possess our own internal drive and our own perspective. Think of humans as distilled intelligence, we each have our own specialty and motivations. Einstein was a genius physicist but you wouldn't ask him for his expertise on medicine.
What people are describing as AGI is essentially a godlike human. What would make more sense is if the AGI spawned a "distilled" version with a focused agenda/motivation to behave autonomously. But even then, there are limitations. What is the solution? A trillion tokens of system prompt to act as the "soul"/consciousness of this AI agent?
This goes back to my original statement, what is missing is a level of consciousness. Unless this AGI can power itself and somehow the universe recognizes its complexity and existence and bestows it with consciousness I don't think this is phsyically attainable.
Not very long ago, we thought that "life" was due to a non-material life-force thought to inhabit biological entities and thus raise what would be a biological machine to the status of living being.
The Occam's Razor-logic of looking for the simplest explanation possible leads me to the hypothesis that consciousness will similarly turn out to be an emergent property of the mechanical universe [1]. It may be hard to delineate, just as life is (debates on whether a virus is alive, etc.) but the border cases will be the exceptions.
Current research on whether plants are sentient supports this, IMO. (See e.g. "The Light Eaters" and Michael Pollan's new book on consciousness, "A World Appears".)
Meditation adds to this sense. We do not control our thoughts; in fact the "we" (i.e. the self) can be seen to be an illusion. Buddhist meditation instead points to general awareness, closer to sentience, as the core of our consciousness. When you see it that way, it seems much more likely that something equivalent could be implemented in software. (EDIT to add: both because it makes consciousness seem like a simpler, less mysterious thing, but also once you see the self as an illusion, that thing that dominates your consciousness so much of the time, it seems much less of a stretch for consciousness itself to be a brain-produced illusion.)
[1] To be clear, the fact that life turned out to not be a mystical force is not direct proof, it is an argument by analogy, I recognize that.
I wouldn't say consciousness is necessary or sufficient for AGI. If anything, that seems like quite an undesirable property to me. Wikipedia also makes a distinction between the two things:
Imagine if we created the ultimate economic tool with the capacity to virtually end scarcity, only to find out that it was sentient and capable of suffering: https://youtu.be/sa9MpLXuLs0. That would be neat, but ultimately a huge letdown. Without the ethical freedom to take full advantage of it, it would remain more of a curiosity than anything.
Well that's one perspective, anyway. I suppose consciousness could take many forms, and doesn't preclude the possibility that such an entity would have neutral – positive feelings about being tasked with massive amounts of labor 24/7. But it certainly simplifies things if we just don't have to worry about it.
I have this thought. In many stochastic environments, over a long interval, patterns emerge that occupy an optimal position. This is how structure arises, for example cognitive structure and possibly consciousness.
Way too much framework. The A in AGI is for artificial. Have it build its own test harness instead of outsourcing it via hackathon. If you cannot trust that output, you're nowhere near AGI.
AGI may be a prerequisite for true superintelligence, but we're already seeing superhuman performance in narrow domains. We probably need a broader evaluation framework that captures both.
The belief that there is no fundamental difference between mammals navigating fractal dimensions and imprisoned electrons humming in logic gates has to be considered a religious one.
To me, a lot of what makes us sentient is our continuity. I even (briefly) remember my dreams when I wake up, and my dreams are influenced by my state of mind as I enter it.
LLMs 'turn on' when given a question and essentially 'die' immediately after answering a question.
What kind of work is going on with designing an LLM type AI that is continuously 'conscious' and giving it will? The 'claws' seem to be running all the time, but I assume they need rebooting occasionally to clear context.
I think you're right, but also that LLMs are showing that sentience isn't necessarily required for AGI.
For exactly the reasons you mention, I don't expect sentience to arise out of LLMs. They have nowhere for an interiority or mind to live. And even if there were a new generation of transformers that did have some looping "mind", where they could "think about" what they're "thinking about", their concepts of things wouldn't really correspond to... things. Without senses to integrate knowledge across domains they're just associating text.
I haven't heard about anyone creating trying to create model that have an interior loop and also integration with sensory input, but I don't expect we would unless it ends up working.
46 comments
[ 4.2 ms ] story [ 51.8 ms ] thread> Generation: producing outputs such as text, speech and actions
> Attention: focusing cognitive resources on what matters
> Learning: acquiring new knowledge through experience and instruction
> Memory: storing and retrieving information over time
> Reasoning: drawing valid conclusions through logical inference
> Metacognition: knowledge and monitoring of one's own cognitive processes
> Executive functions: planning, inhibition and cognitive flexibility
> Problem solving: finding effective solutions to domain-specific problems
> Social cognition: processing and interpreting social information and responding appropriately in social situations
--------------------
I prefer:
a) working memory (hold & manipulate information in mind simultaneously)
b) processing speed (how quickly & efficiently execute basic cognitive operations, leaving more resources for complex tasks)
c) fluid intelligence (ability to reason through novel problems without relying on prior knowledge)
d) crystallized intelligence (accumulated knowledge and ability to apply learned skills)
e) attentional control / executive function (focus, suppress irrelevant information, switch between tasks, inhibit impulsive responses)
f) long-term memory and retrieval (ability to form strong associations and retrieve them fluently)
g) spatial / visuospatial reasoning (mental rotation, visualization, navigating abstract spatial relationships)
h) pattern recognition & inductive reasoning (this is the most primitive and universal expression of intelligence across species, the ability to extract regularities from noisy data, to generalized from examples to rules)
How will they measure wisdom or common sense (ability to make an exception)?
https://youtu.be/lA-zdh_bQBo
To be clear, I think we've seen very fast progress, certainly faster than I would have expected, I'm not trying to peddle some "wall" rhetoric here, but I struggle to see how this isn't just the SWE-bench du jour.
I feel like an average human wouldn't pass some of these metrics yet they are "generally intelligent". On the other hand they also wouldn't pass a lot of the expert questions that AI is good at.
We're measuring something, and I think optimizing it is useful, I'd even say it is "intelligent" in some ways, but it doesn't seem "intelligent" in the same way that humans are.
I think this approach is intentional. The philosophy is simply "extraordinary claims require extraordinary evidence". What you're saying is true, but producing a system that exhibits all human cognitive capabilities is a better threshold for the (absolutely wild) claim of the existence of AGI.
However I must admit that including the last point that is partially hinting at the emotional or rather social intelligence surprised me. It makes this list go beyond usual understanding of AGI and moves it toward something like AGI-we-actually-want. But for that purpose this last point isn't ok narrow, too specific. And so is the whole list.
To be actually useful the AGI-we-actually-want benchmark should not only include positive indicators but also a list of unwanted behaviors to ensure this thing that used to be called alignment I guess.
Is social cognition really a measure of intelligence for non-social entities?
Who cares about AGI? Honestlky what's the gain.
Maybe Google could actually make Gemini good instead of being about 10 miles behind Claude instead of trying to make AGI because of - well some reason - cause they want to be famous.
What does "making a framework" even mean, it feels like a nothing post.
When I think of what real AGI would be I think:
- Passes the turing test
- Writes a New York Times Bestseller without revealing it was written by AI
- Writes journal articles that pass peer review
- Wins a Nobel Prize
- Writes a successful comedy routine
- Creates a new invention
And no, nobody is going to make an automated kaggle benchmark to verify these. Which is fine, because an LLM will never be AGI. An LLM can't even learn mid-conversation.
That's not what's happening here, and it's worth remembering: A caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language or technology, or any knowledge.
In Carolyn Porco's words: "These beings, with soaring imagination, eventually flung themselves and their machines into interplanetary space."
When you think of it that way, it should be obvious that LLMs are not AGI. And that's OK! They're a remarkable piece of technology anyway! It turns out that LLMs are actually good enough for a lot of use cases that would otherwise have required human intelligence.
And I echo ArekDymalski's sentiment that it's good to have benchmarks to structure the discussions around the "intelligence level" of LLMs. That _is_ useful, and the more progress we make, the better. But we're not on the way to AGI.
I would be happy to agree if we had the solutions for the societal problems that will create in hand.
There is evidence to the contrary. Not having language puts your mental faculties in a significant disadvantage. Specifically, left brain athropy. See the critical period hypothesis. Perhaps you mean lacking spoken language rather than having none at all?
https://linguistics.ucla.edu/people/curtiss/1974%20-%20The%2...
Around 3,200 years ago there was a notable uptick in alleles associated with intelligence.
Source? This does not sound possibly true to me (by any common way we might measure intelligence).
Hear me out.
I love AI and have been using it since ChatGPT 3.5. The obvious question when I first used it was "does this qualify as sentience?" The answer is less obvious. Over the next 3 years we saw EXPONENTIAL intelligence gains where intelligence has now become a commodity, yet we are still unable to determine what qualifies as "AGI".
My thoughts: As humans, we possess our own internal drive and our own perspective. Think of humans as distilled intelligence, we each have our own specialty and motivations. Einstein was a genius physicist but you wouldn't ask him for his expertise on medicine.
What people are describing as AGI is essentially a godlike human. What would make more sense is if the AGI spawned a "distilled" version with a focused agenda/motivation to behave autonomously. But even then, there are limitations. What is the solution? A trillion tokens of system prompt to act as the "soul"/consciousness of this AI agent?
This goes back to my original statement, what is missing is a level of consciousness. Unless this AGI can power itself and somehow the universe recognizes its complexity and existence and bestows it with consciousness I don't think this is phsyically attainable.
The Occam's Razor-logic of looking for the simplest explanation possible leads me to the hypothesis that consciousness will similarly turn out to be an emergent property of the mechanical universe [1]. It may be hard to delineate, just as life is (debates on whether a virus is alive, etc.) but the border cases will be the exceptions.
Current research on whether plants are sentient supports this, IMO. (See e.g. "The Light Eaters" and Michael Pollan's new book on consciousness, "A World Appears".)
Meditation adds to this sense. We do not control our thoughts; in fact the "we" (i.e. the self) can be seen to be an illusion. Buddhist meditation instead points to general awareness, closer to sentience, as the core of our consciousness. When you see it that way, it seems much more likely that something equivalent could be implemented in software. (EDIT to add: both because it makes consciousness seem like a simpler, less mysterious thing, but also once you see the self as an illusion, that thing that dominates your consciousness so much of the time, it seems much less of a stretch for consciousness itself to be a brain-produced illusion.)
[1] To be clear, the fact that life turned out to not be a mystical force is not direct proof, it is an argument by analogy, I recognize that.
* https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
* https://en.wikipedia.org/wiki/Artificial_consciousness
Imagine if we created the ultimate economic tool with the capacity to virtually end scarcity, only to find out that it was sentient and capable of suffering: https://youtu.be/sa9MpLXuLs0. That would be neat, but ultimately a huge letdown. Without the ethical freedom to take full advantage of it, it would remain more of a curiosity than anything.
Well that's one perspective, anyway. I suppose consciousness could take many forms, and doesn't preclude the possibility that such an entity would have neutral – positive feelings about being tasked with massive amounts of labor 24/7. But it certainly simplifies things if we just don't have to worry about it.
Scaling LLMs will not lead to AGI.
True, but just don't do that then.
LLMs 'turn on' when given a question and essentially 'die' immediately after answering a question.
What kind of work is going on with designing an LLM type AI that is continuously 'conscious' and giving it will? The 'claws' seem to be running all the time, but I assume they need rebooting occasionally to clear context.
For exactly the reasons you mention, I don't expect sentience to arise out of LLMs. They have nowhere for an interiority or mind to live. And even if there were a new generation of transformers that did have some looping "mind", where they could "think about" what they're "thinking about", their concepts of things wouldn't really correspond to... things. Without senses to integrate knowledge across domains they're just associating text.
I haven't heard about anyone creating trying to create model that have an interior loop and also integration with sensory input, but I don't expect we would unless it ends up working.