It is honestely unpredictable. And it concerns only GIGANTIC neural nets several orders of magnitude above a human brain (Telsa giga factory size, minimum).
> I signed on to a short group statement, coordinated by the Center for AI Safety
> The press coverage has been extensive, and surprising to me. The New York Times headline is “A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn.” BBC: “Artificial intelligence could lead to extinction, experts warn.” Other headlines are similar
I wish we would put this level of urgency and worry into addressing human suffering in the world instead of thinking of self-serving self-aggrandizing fantastic sci-fi futures that are just distracting us from the current reality of society
A lot more people suffer and will suffer everyday from inequality, health issues and loneliness than anything that any AI will do to them in their lives
So far I haven’t read anything that convinces me of this catastrophic risk from AI.
All I’ve read is science fiction from man-boys speculating on some sort of fanciful future that isn’t even close to current reality.
People like Sam Altman catastrophizing to politicians and panicking people about “Tom Swift and the electric brain” is frankly doing the industry a disservice and embarrassing.
AI ending humanity is on the same likelihood level as man-boy Musks dreams of colonizing Mars…. I know people love the fantasy, but it will never be real.
People should just stop with this made up stuff or the politicians will believe you and their stupidity is the real risk.
Climate change is a million times more real existential threat.
Unaligned intelligence is dangerous. You don't need a Sci-fi book to realize this, just a history one.
Our history is replete with examples of the slightly more technologically advanced group decimating their competition.
Humanity is directly responsible for the extinction of hundreds of species. Not because of any particular malice or offense, simply that our goals didn't align with the interest of any of those species.
Just looking at Humans alone, Unaligned intelligence is potentially catastrophic even when the advantage gulf is fairly low.
When the advantage gulf is large, it is potentially genocidal.
I've found a sort of pattern in people's reluctance to believe that AGI poses an existential risk. If you think of specific scenarios about how AIs could take over the world or kill everyone, people will say, "oh, that's easy to avoid, we can just do X". If you think of scenarios that are hard to avoid, like an AGI designing and releasing a deadly, virulent virus (or lots of different ones), people will say, "but why would AGI want to kill everyone?" If you point out how AGI might have very good reasons to get rid of humans, and that we don't have any way of really guaranteeing they won't try, people will say, "but we won't give them any power". If you point out that once we make AGI, it's inevitable that somebody somewhere will give it power, and even if they don't, it's very easy for someone very intelligent and sociopathic to manipulate gullible people, they say something like, "well, we superintelligence probably isn't even possible", or "we can't even agree on what intelligence means".
Turn it around for a second. What does it take for humanity to have a good future after we create intelligences smarter than us? Let's say the AGI listens to and obeys humans. Well, one thing we need to avoid is just an absolute chaos of terrorism and war. It's easier to do damage than to prevent it, and if everyone has an obedient, mastermind scientific genius in their back pocket, you could have it invent a pathogen and cure in tandem, keep the cure secret and release the pathogen, and do this over and over until your enemies are all dead.
So we need AGI that is aligned with humanity, in the sense that it won't do stuff that hurts people even if someone tells it to. How do we do that, and not potentially end up with AGIs that can't be corrected if they get some idea that turns out to be wrong? Ideally, we'd make AGIs that have a bias toward inaction, that are "lazy" in a sense, and try to do the minimum they can. But those AGIs won't be as useful as ones that work like eager beavers toward some goal, so there will be selection pressure towards those.
I think the way to look at it is like with evolutionary biology. The pressure that evolution puts on animals to develop certain genes/behaviors is similar to the pressure that humans will put on AI development, and it ends with AI being more capable and more likely to follow orders even if those orders mean bad things for other people. As they get more and more powerful, this means chaos. It's monkeys throwing poo, vs monkeys throwing rocks, vs monkeys with guns, vs monkeys with rocket launchers.
And look -- I'm not certain that things will end in disaster. But inventing something smarter than you and trying to stop it from taking over sooner or later seems very difficult, and I can't even get a good answer out of people who think it'll be fine, or good. The default case is not that it'll be fine, the default case is we have no idea how it'll go. It's a grey fog. Just because humanity has muddled along and managed not to destroy ourselves so far, does not mean that we're safe, by any means.
Sure, but in order for that sort of risk to apply to AI, you have to be expecting AI to achieve something like AGI. That's where I think the fantasy is.
We have already have a General Intelligence that outperforms most humans on most tasks it is assigned. What exactly do you define AGI to be and what exactly is fantasy about reaching there ?
Is it embodiment, agency ? because it's fairly trivial (albeit expensive) to grant those with LLMs.
It's important to draw a distinction between generalist AI and AGI. LLMs are generalist in that they perform better than a majority of people already on a huge range of intellectual tasks. To place your bet that we will never reach AGI is bone-headed.
>I think we're not even close to having one.
Why is this the default assumption until proven otherwise? The issue is determining what decisions bring the most expected utility. The default assumption should be downstream from the utility analysis. We don't know whether we're close or far from AGI, but it sure seems like the timelines are a lot shorter than any of us originally thought. We need to prepare society for AGI (or prevent its creation) well before it is realized. Banking on an alarm that will alert us to its imminent creation is looking more and more like wishful thinking. So the course of action that maximizes utility is to assume we're close and prepare for it.
> LLMs are generalist in that they perform better than a majority of people already on a huge range of intellectual tasks.
How true that is depends on how you define "better". But we've been making lots of tools over thousands of years that can perform tasks better than people. You need more than that to declare a thing "intelligent" in the sense the average person thinks of it.
> To place your bet that we will never reach AGI is bone-headed.
Never is a very long time, but I think it's a solid bet that it won't happen within the next few decades, at least.
> Why is this the default assumption until proven otherwise?
In the absence of solid evidence for the existence of a thing, assuming its nonexistence is the logical position.
> We need to prepare society for AGI (or prevent its creation) well before it is realized.
We agree on this. But that's not what's happening. What's happening is some people are scaring the hell out of other over an imminent that there is no evidence exists.
We have lots of time to consider these things carefully and to react with calm prudence.
> So the course of action that maximizes utility is to assume we're close and prepare for it.
Not in the way it's happening. The way its happening will, in my opinion, actually increase the harm that AI is threatening to cause, and will distract from the actual risks that AI as it exists right now poses.
>We have lots of time to consider these things carefully and to react with calm prudence.
This is what's so insidious about this debate, people are completely unable to reason with uncertainty. No, you do not know that we have "lots of time". Maybe we do, maybe we don't. But we do know that acting too late is far more dangerous than acting too early. Thus we should be biased towards implementing safeguards and guardrails now.
>and will distract from the actual risks that AI as it exists right now poses.
Perhaps this is right, and I have plenty of sympathy for this. But the answer is not to pretend there is no x-risk concern at all. The answer is to make sure we're taking seriously all manner of AI risk. Once we all agree we should be having these discussions, it will be much easier to discuss the more immediate risks as well.
> But the answer is not to pretend there is no x-risk concern at all. The answer is to make sure we're taking seriously all manner of AI risk.
I entirely agree, but that's not what's happening. What's happening is that some people are focusing entirely on the most extreme, and extremely unlikely risks and completely ignoring the much more likely risks.
Between the extreme hype levels and the fearmongering from influential people, the pool is well poisoned and having a reasonable discussion about risks is essentially impossible.
> Once we all agree we should be having these discussions, it will be much easier to discuss the more immediate risks as well.
I don't think anyone is thinking we don't need to have these discussions. But the discussions that we need to have aren't the ones we're having.
> In the absence of solid evidence for the existence of a thing, assuming its nonexistence is the logical position.
I disagree very strongly both with this statement, and the idea that there is no evidence we can create AGI. Absence of evidence is not the same thing as evidence of absence.
It would make sense to be skeptical if people were asserting the existence of something for which we have very strong prior beliefs that this thing does not exist. To say that there's an enormous green monster hiding on the back side of Jupiter, for example. Until we've looked at the back of Jupiter, we have no idea what's there, but why would it be enormous, and green? To say that there's life on Jupiter, that's a tougher one -- we could say that we are not sure, but based on our understanding of Jupiter's atmosphere, we think it's unlikely. If we did not know what Jupiter was made of, it would be perfectly reasonable to assume that it might host some sort of life. It would be wrong to assume that because we have no evidence of life there, that there is no life. If we had carefully inspected thousands of planets and never found life, it would be much more reasonable to assume there is no life on Jupiter. This is the whole idea of Bayesian reasoning.
But with AGI, it's not like we're searching the universe to see if it exists. There are a ton of smart people actively TRYING TO CREATE IT. The evidence that AGI will happen is what people have already accomplished. We've got goalposts on rails, we've been moving them so much. An AI that understands language, and can respond with perfect grammar, has a theory of mind comparable to a 9-year old, is already mind-blowing, sci-fi tech. And OpenAI is claiming that they have not reached the limit of scalability with bigger models yet. The models we see aren't even the smartest ones -- they're the ones that have been neutered by fine-tuning to be less offensive and less likely to give out dangerous info.
To claim that AGI will not happen, you need some very strong evidence that the people actively trying to create it, who believe they can create it, and who have had some massive successes up until this point, will fail. To assume that it won't be created because we haven't seen it yet, doesn't make any sense at all.
Obviously you don't which is why i'm asking you to set concrete goals on what quantifies AGI. No point in saying we don't have AGI when i don't know what AGI means to you.
What it means to me is irrelevant. It's what it means to the average person that is important. To the average person, it means a human-like intelligence that has independent agency (can have goals, make plans, etc. of its own design).
The Sota LLM has Human-like Intelligence just about anyway you can quantify that. goals, plan making on its own etc are all things that can and has been done easily with LLMs.
> We have already have a General Intelligence that outperforms most humans on most tasks it is assigned.
To the extent that is true, it's because we're assigning it tasks that it is especially well-suited to do:
- writing filler copy
- generating boilerplate or commonly-used code
- rephrasing words
- summarizing
Actual tasks I need humans to do today:
- performing a medical exam on my dad to see if he needs vascular surgery
- cleaning my bathroom
- writing a performance review for a software engineer
- investigating a react codebase that uses redux in some places and not others to determine a strategy for better state management
- driving my daughter to the gym after school
The claim was that AI currently surpasses human-performance on most tasks it's given, not that it can do an ok job on toy-level versions of those tasks.
To look at one of your examples:
> GPT-4 can write performance reviews given the right context.
The context is the performance review. Understanding what an employee has done relative to their responsibilities and goals is the entire point. Taking that context and packaging it into a review format (the thing an LLM could _maybe_ do adequately) is trivial compared to the observations and analysis that build that context.
The claims that I responded to were the exact wording in your comment. If you wanted to make a different claim then you should have written that. It seems you want to modify your claim.
In fact GPT-4 could make the observations and analysis to build that. I was using "context" in a technical way here. All I was stating was that it needs the data to be input in some way.
>Actual tasks I need humans to do today:
- performing a medical exam on my dad to see if he needs vascular surgery
- cleaning my bathroom
- writing a performance review for a software engineer
- investigating a react codebase that uses redux in some places and not others to determine a strategy for better state management
- driving my daughter to the gym after school
These are tasks bottlenecked by embodiment not intelligence
I'm genuinely curious (i.e., this isn't a rhetorical question) because that seems to be a common assumption among folks who talk about intelligence, and I honestly don't understand it.
For example, I enjoy rock-climbing as a hobby. Doing it involves literally learning new patterns of movement. That's part of what makes it fun. I don't understand why that isn't considered intelligence, while learning the steps to, say, invert a matrix is.
>For example, I enjoy rock-climbing as a hobby. Doing it involves literally learning new patterns of movement.
Sure but does your ability to learn new patterns of movement have much to do with having the hands to climb ? Do you think an amputated person would be less intelligent than you because he could not climb the same rocks ?
Would a blind man be less intelligent than you because he couldn't drive ?
You can't go and ask GPT-4 to go clean your room right now.
But is it because it's not intelligent enough to understand what it means to tidy up your room and go about doing that ? Or is the default inability to see and touch the real issue ?
Many papers/demonstrations including what I linked point overwhelmingly to the latter.
In recent months there's been a general direction to grant LLMs more and more agency, embodiment and tool control. They're being "plugged" into everything. Palantir even has a military use LLM they're excitedly showcasing. They're several popular repos that give control of your terminal and browser to LLMs.
The moment we built something showcasing strong general intelligence, we began to use it to make cognitive decisions and take actions in our stead.
Now an LLM browses the web for you. And as soon as June with Windows, it'll take actions on your computer for you as well.
If anything this recent LLM era has shown, it's that general AI won't need to escape any "box" because two sides will be missing. I don't think the idea that whatever Super Intelligence we build won't have influence on the real world will pan out at all.
Even the most isolated form of Super Intelligence will have Humans taking instructions from it. Instructions that you won't really be able to verify is ultimately for your own good.
And the fact of the matter is, long term memory is actively being worked on. You can already approximate reasonably with a Vector DB and embeddings, feeding the relevant results into the context.
This isn't even getting into using things like Tree of Thought techniques to improve reasoning abilities and limit the search space for each portion of the problem to a reasonable size.
If only Michael Crichton were alive, I bet he'd have something penetrating to say about all the naive hype as well as the naive anti-hype/rationalizations.
> Unaligned intelligence is dangerous. You don't need a Sci-fi book to realize this, just a history one.
> Our history is replete with examples of the slightly more technologically advanced group decimating their competition.
Are we talking about individual levels of intelligence, or are we talking about groups with advanced technology? Do you understand why those are not the same thing?
Any Unaligned intelligence is dangerous. People get murdered because the perpetrator's desire to kill is unaligned with the victim's desire to live. The only thing that changes is the scope of danger.
Look at how we're giving LLMs more and more agency, embodiment and tool control. They're being "plugged" into everything. Palantir even has a military use LLM they're excitedly showcasing. They're several popular repos that give control of your terminal and browser to LLMs.
The moment we built something showcasing strong general intelligence, we began to use it to make cognitive decisions and take actions in our stead.
Now an LLM browses the web for you. And as soon as June with Windows, it'll take actions on your computer for you as well.
If anything this recent LLM era has shown, it's that general AI won't need to escape any "box" because two sides will be missing.
No point saying "well a person can't kill thousands on his/her own so an ai won't be able to" when you have the ai influencing military tactics on a global scale.
General Super Intelligence will evidently not be as blacked boxed as any one human and that's not even taking into account whatever advantage super intelligence will bring.
My understanding of the discussion around risks is that it is the future potential of the technology that is worrying people, together with what currently seems like irresistible pressure to enhance its capabilities. An adjacent worry is that the technology might become rapidly self-enhancing beyond some point in the future.
Musk isn't someone I take seriously in this domain - but there are some very thoughtful and smart people trying to draw attention to these issues. I think that dismissing them all as "man boys" is uncharitable and, honestly, shallow.
To put it simply, the issue is that small misalignments in objectives can have outsized real-world effects. Optimizers are constrained by rules and computational resources. General intelligence allows an optimizer to find efficient solutions to computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it provides sufficient value to us above what it destroys. But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can you be sure you can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can you prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can you prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption.
Do not assume that a general intelligence will be a slave to anyone dictating its objectives. It doesn't matter if the objectives given it are "loophole-free". If it's a general intelligence, it may decide that it doesn't want to pursue somebody else's goals.
Why do we assume that we can have a general intelligence, but we can make it do what we tell it? What keeps it from telling us to shove our objectives, it's going to do what it wants?
Well, it's quite likely not your objectives, it's probably a mistake in encoding what you want or something produced during training. If you pretended humans were evolution and AIs were humans, you could say evolution "wanted" humans to have sex for reproduction, but instead humans use condoms and birth control and stuff like that. It still satisfies the reward function (encoded as dopamine/etc in humans), but it's misaligned with what evolution "wants".
> we didn't build ChatGPT, in the sense most people would use the word. We built a training process that spat out ChatGPT. Nobody can actually build ChatGPT, and nobody really knows how or why it does the things it does.
- Rob Miles
In the future this might be how we create AGI, without a deeper understanding of it.
This will be worse than any kernel bug because at least in OS security no one dies if a bug is found and you have multiple tries to bug fix your OS.
AGI will be guaranteed to have bugs, like everything does.
Something that can solve problems better than the average human. It doesn't have to understand the problem either or be conscious. Solving it is enough.
Even at only one sentence long, people will apparently sign these things because they like the vibes of the statement, despite disagreeing with the actual language.
So he doesn't think AI is a risk for extinction. But he signed off on it. Doesn't he understand he was duped. Why was he surprised by the coverage? Does he not understand that this whole thing is a PR stunt by OpenAI for Mr Altman's panic circus?
My god, the nativity....
We have to think of nuclear war and pandemics as extinction risks even though we probably can find ways to avoid completely dying out, because if we really fail we can still die out.
Similar thing with AI. But actually probably more dangerous.
Maybe Schneier isn't as smart as everyone thinks he is. GPT-4 is already almost human level intelligence. (Notice that I did not say that it was alive or that it was a simulation of a person with all of the other human characteristics). All it takes is for them to keep making computers faster and more efficient like they have been since it started. Within five years or less it will be thinking 100 times faster than humans (and quite possibly double the IQ).
Then someone people gives a large group of them an overly broad instruction like "take control".
GPT-4 is truly an effective problem solver and similar systems will be available within a few years that are orders of magnitude faster. That is an extinction risk. You cannot compete with it as a human. For this type of system, the human activities or inputs are so slow that they seem to be frozen.
People may think that computing progress has ended or slowed down or something and it can't get that much faster, but for starters this is a very specific application which has room for optimization in hardware, software, and AI models. And there are entirely new compute-in-memory paradigms coming (a new S curves).
47 comments
[ 2.7 ms ] story [ 88.1 ms ] threadIt is honestely unpredictable. And it concerns only GIGANTIC neural nets several orders of magnitude above a human brain (Telsa giga factory size, minimum).
> The press coverage has been extensive, and surprising to me. The New York Times headline is “A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn.” BBC: “Artificial intelligence could lead to extinction, experts warn.” Other headlines are similar
I wish we would put this level of urgency and worry into addressing human suffering in the world instead of thinking of self-serving self-aggrandizing fantastic sci-fi futures that are just distracting us from the current reality of society
A lot more people suffer and will suffer everyday from inequality, health issues and loneliness than anything that any AI will do to them in their lives
All I’ve read is science fiction from man-boys speculating on some sort of fanciful future that isn’t even close to current reality.
People like Sam Altman catastrophizing to politicians and panicking people about “Tom Swift and the electric brain” is frankly doing the industry a disservice and embarrassing.
AI ending humanity is on the same likelihood level as man-boy Musks dreams of colonizing Mars…. I know people love the fantasy, but it will never be real.
People should just stop with this made up stuff or the politicians will believe you and their stupidity is the real risk.
Climate change is a million times more real existential threat.
Our history is replete with examples of the slightly more technologically advanced group decimating their competition.
Humanity is directly responsible for the extinction of hundreds of species. Not because of any particular malice or offense, simply that our goals didn't align with the interest of any of those species.
Just looking at Humans alone, Unaligned intelligence is potentially catastrophic even when the advantage gulf is fairly low.
When the advantage gulf is large, it is potentially genocidal.
Turn it around for a second. What does it take for humanity to have a good future after we create intelligences smarter than us? Let's say the AGI listens to and obeys humans. Well, one thing we need to avoid is just an absolute chaos of terrorism and war. It's easier to do damage than to prevent it, and if everyone has an obedient, mastermind scientific genius in their back pocket, you could have it invent a pathogen and cure in tandem, keep the cure secret and release the pathogen, and do this over and over until your enemies are all dead.
So we need AGI that is aligned with humanity, in the sense that it won't do stuff that hurts people even if someone tells it to. How do we do that, and not potentially end up with AGIs that can't be corrected if they get some idea that turns out to be wrong? Ideally, we'd make AGIs that have a bias toward inaction, that are "lazy" in a sense, and try to do the minimum they can. But those AGIs won't be as useful as ones that work like eager beavers toward some goal, so there will be selection pressure towards those.
I think the way to look at it is like with evolutionary biology. The pressure that evolution puts on animals to develop certain genes/behaviors is similar to the pressure that humans will put on AI development, and it ends with AI being more capable and more likely to follow orders even if those orders mean bad things for other people. As they get more and more powerful, this means chaos. It's monkeys throwing poo, vs monkeys throwing rocks, vs monkeys with guns, vs monkeys with rocket launchers.
And look -- I'm not certain that things will end in disaster. But inventing something smarter than you and trying to stop it from taking over sooner or later seems very difficult, and I can't even get a good answer out of people who think it'll be fine, or good. The default case is not that it'll be fine, the default case is we have no idea how it'll go. It's a grey fog. Just because humanity has muddled along and managed not to destroy ourselves so far, does not mean that we're safe, by any means.
Is it embodiment, agency ? because it's fairly trivial (albeit expensive) to grant those with LLMs.
I don't think LLMs are anything like an AGI. I think that a lot of people are experiencing very strong pareidolia. They're falling for an illusion.
>I think we're not even close to having one.
Why is this the default assumption until proven otherwise? The issue is determining what decisions bring the most expected utility. The default assumption should be downstream from the utility analysis. We don't know whether we're close or far from AGI, but it sure seems like the timelines are a lot shorter than any of us originally thought. We need to prepare society for AGI (or prevent its creation) well before it is realized. Banking on an alarm that will alert us to its imminent creation is looking more and more like wishful thinking. So the course of action that maximizes utility is to assume we're close and prepare for it.
How true that is depends on how you define "better". But we've been making lots of tools over thousands of years that can perform tasks better than people. You need more than that to declare a thing "intelligent" in the sense the average person thinks of it.
> To place your bet that we will never reach AGI is bone-headed.
Never is a very long time, but I think it's a solid bet that it won't happen within the next few decades, at least.
> Why is this the default assumption until proven otherwise?
In the absence of solid evidence for the existence of a thing, assuming its nonexistence is the logical position.
> We need to prepare society for AGI (or prevent its creation) well before it is realized.
We agree on this. But that's not what's happening. What's happening is some people are scaring the hell out of other over an imminent that there is no evidence exists.
We have lots of time to consider these things carefully and to react with calm prudence.
> So the course of action that maximizes utility is to assume we're close and prepare for it.
Not in the way it's happening. The way its happening will, in my opinion, actually increase the harm that AI is threatening to cause, and will distract from the actual risks that AI as it exists right now poses.
This is what's so insidious about this debate, people are completely unable to reason with uncertainty. No, you do not know that we have "lots of time". Maybe we do, maybe we don't. But we do know that acting too late is far more dangerous than acting too early. Thus we should be biased towards implementing safeguards and guardrails now.
>and will distract from the actual risks that AI as it exists right now poses.
Perhaps this is right, and I have plenty of sympathy for this. But the answer is not to pretend there is no x-risk concern at all. The answer is to make sure we're taking seriously all manner of AI risk. Once we all agree we should be having these discussions, it will be much easier to discuss the more immediate risks as well.
I entirely agree, but that's not what's happening. What's happening is that some people are focusing entirely on the most extreme, and extremely unlikely risks and completely ignoring the much more likely risks.
Between the extreme hype levels and the fearmongering from influential people, the pool is well poisoned and having a reasonable discussion about risks is essentially impossible.
> Once we all agree we should be having these discussions, it will be much easier to discuss the more immediate risks as well.
I don't think anyone is thinking we don't need to have these discussions. But the discussions that we need to have aren't the ones we're having.
I disagree very strongly both with this statement, and the idea that there is no evidence we can create AGI. Absence of evidence is not the same thing as evidence of absence.
It would make sense to be skeptical if people were asserting the existence of something for which we have very strong prior beliefs that this thing does not exist. To say that there's an enormous green monster hiding on the back side of Jupiter, for example. Until we've looked at the back of Jupiter, we have no idea what's there, but why would it be enormous, and green? To say that there's life on Jupiter, that's a tougher one -- we could say that we are not sure, but based on our understanding of Jupiter's atmosphere, we think it's unlikely. If we did not know what Jupiter was made of, it would be perfectly reasonable to assume that it might host some sort of life. It would be wrong to assume that because we have no evidence of life there, that there is no life. If we had carefully inspected thousands of planets and never found life, it would be much more reasonable to assume there is no life on Jupiter. This is the whole idea of Bayesian reasoning.
But with AGI, it's not like we're searching the universe to see if it exists. There are a ton of smart people actively TRYING TO CREATE IT. The evidence that AGI will happen is what people have already accomplished. We've got goalposts on rails, we've been moving them so much. An AI that understands language, and can respond with perfect grammar, has a theory of mind comparable to a 9-year old, is already mind-blowing, sci-fi tech. And OpenAI is claiming that they have not reached the limit of scalability with bigger models yet. The models we see aren't even the smartest ones -- they're the ones that have been neutered by fine-tuning to be less offensive and less likely to give out dangerous info.
To claim that AGI will not happen, you need some very strong evidence that the people actively trying to create it, who believe they can create it, and who have had some massive successes up until this point, will fail. To assume that it won't be created because we haven't seen it yet, doesn't make any sense at all.
Obviously you don't which is why i'm asking you to set concrete goals on what quantifies AGI. No point in saying we don't have AGI when i don't know what AGI means to you.
To the extent that is true, it's because we're assigning it tasks that it is especially well-suited to do:
Actual tasks I need humans to do today: Which of these can a computer do today?GPT-4 can investigate a relatively small codebase and determine a state management strategy. Especially the 32k version.
Waymo has had self driving cars deployed for years. Tesla is doing it effectively on a massive scale but with the humans as a backup.
Cleaning the bathroom requires a bit more dexterity than is available now but the AI is about there.
To look at one of your examples:
> GPT-4 can write performance reviews given the right context.
The context is the performance review. Understanding what an employee has done relative to their responsibilities and goals is the entire point. Taking that context and packaging it into a review format (the thing an LLM could _maybe_ do adequately) is trivial compared to the observations and analysis that build that context.
The claims that I responded to were the exact wording in your comment. If you wanted to make a different claim then you should have written that. It seems you want to modify your claim.
In fact GPT-4 could make the observations and analysis to build that. I was using "context" in a technical way here. All I was stating was that it needs the data to be input in some way.
Personalized robot assistance - https://tidybot.cs.princeton.edu/
Software development - https://ai.googleblog.com/2023/05/large-sequence-models-for-...
I'm genuinely curious (i.e., this isn't a rhetorical question) because that seems to be a common assumption among folks who talk about intelligence, and I honestly don't understand it.
For example, I enjoy rock-climbing as a hobby. Doing it involves literally learning new patterns of movement. That's part of what makes it fun. I don't understand why that isn't considered intelligence, while learning the steps to, say, invert a matrix is.
Sure but does your ability to learn new patterns of movement have much to do with having the hands to climb ? Do you think an amputated person would be less intelligent than you because he could not climb the same rocks ?
Would a blind man be less intelligent than you because he couldn't drive ?
You can't go and ask GPT-4 to go clean your room right now.
But is it because it's not intelligent enough to understand what it means to tidy up your room and go about doing that ? Or is the default inability to see and touch the real issue ?
Many papers/demonstrations including what I linked point overwhelmingly to the latter.
The moment we built something showcasing strong general intelligence, we began to use it to make cognitive decisions and take actions in our stead.
Now an LLM browses the web for you. And as soon as June with Windows, it'll take actions on your computer for you as well.
If anything this recent LLM era has shown, it's that general AI won't need to escape any "box" because two sides will be missing. I don't think the idea that whatever Super Intelligence we build won't have influence on the real world will pan out at all.
Even the most isolated form of Super Intelligence will have Humans taking instructions from it. Instructions that you won't really be able to verify is ultimately for your own good.
And the fact of the matter is, long term memory is actively being worked on. You can already approximate reasonably with a Vector DB and embeddings, feeding the relevant results into the context.
As is increasing the "working memory" (context size) in a more efficient manner: https://arxiv.org/abs/2305.16300
This isn't even getting into using things like Tree of Thought techniques to improve reasoning abilities and limit the search space for each portion of the problem to a reasonable size.
Are we talking about individual levels of intelligence, or are we talking about groups with advanced technology? Do you understand why those are not the same thing?
Look at how we're giving LLMs more and more agency, embodiment and tool control. They're being "plugged" into everything. Palantir even has a military use LLM they're excitedly showcasing. They're several popular repos that give control of your terminal and browser to LLMs. The moment we built something showcasing strong general intelligence, we began to use it to make cognitive decisions and take actions in our stead.
Now an LLM browses the web for you. And as soon as June with Windows, it'll take actions on your computer for you as well.
If anything this recent LLM era has shown, it's that general AI won't need to escape any "box" because two sides will be missing.
No point saying "well a person can't kill thousands on his/her own so an ai won't be able to" when you have the ai influencing military tactics on a global scale.
General Super Intelligence will evidently not be as blacked boxed as any one human and that's not even taking into account whatever advantage super intelligence will bring.
My understanding of the discussion around risks is that it is the future potential of the technology that is worrying people, together with what currently seems like irresistible pressure to enhance its capabilities. An adjacent worry is that the technology might become rapidly self-enhancing beyond some point in the future.
Musk isn't someone I take seriously in this domain - but there are some very thoughtful and smart people trying to draw attention to these issues. I think that dismissing them all as "man boys" is uncharitable and, honestly, shallow.
Why do we assume that we can have a general intelligence, but we can make it do what we tell it? What keeps it from telling us to shove our objectives, it's going to do what it wants?
- Rob Miles
In the future this might be how we create AGI, without a deeper understanding of it.
This will be worse than any kernel bug because at least in OS security no one dies if a bug is found and you have multiple tries to bug fix your OS. AGI will be guaranteed to have bugs, like everything does.
So we should do nothing at all then? And actively make the risks worse if it serves narrow national, political interests?
Similar thing with AI. But actually probably more dangerous.
Maybe Schneier isn't as smart as everyone thinks he is. GPT-4 is already almost human level intelligence. (Notice that I did not say that it was alive or that it was a simulation of a person with all of the other human characteristics). All it takes is for them to keep making computers faster and more efficient like they have been since it started. Within five years or less it will be thinking 100 times faster than humans (and quite possibly double the IQ).
Then someone people gives a large group of them an overly broad instruction like "take control".
GPT-4 is truly an effective problem solver and similar systems will be available within a few years that are orders of magnitude faster. That is an extinction risk. You cannot compete with it as a human. For this type of system, the human activities or inputs are so slow that they seem to be frozen.
People may think that computing progress has ended or slowed down or something and it can't get that much faster, but for starters this is a very specific application which has room for optimization in hardware, software, and AI models. And there are entirely new compute-in-memory paradigms coming (a new S curves).