Job loss? Tech already did that, the little old ladies that opened your envelope at sears and roebuck got replace by computers a long time ago...
Deepfakes? Are we talking the moon landing? Avengers movies?
AGI taking over the world? Does no one realize that without people going out every day and doing work the world ends in a week... Does no one remember when we had the spate of transformer shootings? A rifle or a bucket of water ends AGI threats.
Where is the threat from ML that doesn't already exist and is an ACTUAL threat to 8billion of us?
The risk is in losing control of the narrative. That's all the top brass actually cares about. As you mentioned we already have deepfakes, spam, etc. and the world hasn't imploded.
the section: "No, AI Safety isn't a fringe concern by sci-fi weebs"
> AI Safety / AI Risk used to be less mainstream, but now in 2024, the US & UK governments now have AI Safety-specific departments![10]10 This resulted from many of the top AI researchers raising alarm bells about it. These folks include:
> Geoffrey Hinton and Yoshua Bengio
> Stuart Russell and Peter Norvig
> Paul Christiano
> I'm aware "look at these experts" is an appeal to authority, but this is only to counter the idea of, "eh, only sci-fi weebs fear AI Risk". But in the end, appeal to authority/weebs isn't enough; you have to actually understand the dang thing. (Which you are doing, by reading this! So thank you.)
I think it is pretty telling that in response to a direct question about what we have to fear from AI, you link to an article that itself remains so obviously vague that they felt the need to address that they are just appealing to other authorities.
An actual plausible threat should be easy to describe. In terms of arguments, vagueness indicates weakness.
No, it’s pretty telling of the state of the intellectual discourse when we’re still getting “we could just turn it off” level responses, so appeals to knowledgeable scientific authorities is the most convincing for people who have strong opinions but haven’t done even 30 minutes of grappling with the case made by those they already dismissed. For the more well-read, there are plausible threads:
First, I want to address the precision of your response:
The content you link to is about AGI, which is a theoretical concept, and it is not at all clear that the path to AGI is incremental from our current approaches.
Regardless, it really feels like a pure marketing move to link concerns about AGI to our current tool chain of decision engines, content generators, and spell checkers.
Nothing that is publicly known about what OpenAI or its foes are currently working on suggests in any way that it should be linked to the emerging of AGI.
Now to the more general points made in the content you link to:
As the author states in the preamble, this is a "poorly organized list of individual rants".
And it seems that none of those make a good and plausible case for why even the theoretical idea(!) of AGI is particularly dangerous. Of course, with a good deal of good will, I can follow most of the ideas listed, but it feels forced and certainly requirement heavy.
Remember that the state they are thinking about is is in the future (probably far), and everything they write is hypothetical.
It's like if we learned to throw a stone for the first time, and suddenly the village is full of alarmist people discussing high-minded ideas about "what if everyone is suddenly thrown away, by which we mean displaced at high speed in a vaguely known direction - by a force we yet can not understand?
It's mostly not serious or related to the current technologies.
If someone is alarmist and wants to be taken serious they should be required to make an actual argument, themselves. .. without fleeing into references, vagueness or unnecessary complexity.
Where is it going to build them? Where are all the lights out factories? Where are all the robots to build the robots to build the robots? Your AGI has a massive bootstrapping problem.
Now it needs to build enough robots, that are reliable enough to maintain and run an entire supply chain. How does it get power, build more chips... The whole world is run by PEOPLE ...
There is the classic I, Pencil. The toaster project (https://www.thomasthwaites.com/the-toaster-project/ ) is the more modern take. It's people who watch the walking dead and dont know enough to ask "where the hell is the gas coming from" because that pipline is days long. It's the forgotten instances of "substation attack" never mind the fact that if linesmen stopped showing up to work the grid would fall apart in weeks.
To be candid that list is the hubris of people who do not have a clue of how the real world works outside their ivory tower.
The risk is AI laundering the rest of America’s middle class wealth to the 1% resulting in falling healthcare outcomes, education outcomes, and life expectancies, and much worse working conditions for all industries.
We’ve seen a huge shift of wealth in the last twenty years in no small part passing through and at the insistence of Venture Capital and Private Equity. AI will be used to again accelerate this with more layoffs more outsourcing and lower quality services captured.
GP's point (and I agree) is that an evil actor could, in principle, do those things already (and a much more intelligent one, while we're at it).
And yet, it doesn't happen. Why? Because reality is not a "winner takes all" game. Actions carry consequences and other factors limit the extent of damage any actor could do.
It is way more threatening to human life that there exists, today, a huge stockpile of nuclear weapons all around the world.
An evil actor could do those things, but they don’t because smart people work really hard to build systems that make doing those things more difficult and/or with dire consequences to the potential actors. Shrugging at the AI threat reminds me of “I felt better so I stopped taking the medication.”
Nah, you'll be surprised at how dumb things in the real world could be; like, the nuclear briefcase password is probably 0000-0000.
Punishment and retaliation is what deters most people from going rogue. Note that by retaliation I do not mean "fear of retaliation", I mean the actual consequences that come to you from FAFO in the real world.
Our parents and grandparents getting scammed by deepfakes seems like a very real threat. Not only do anecdotes abound (plus FBI's database of reports) but the elderly were already vulnerable to these scams since before the telephone was invented. LLMs and all the voice cloning tech really turns up scammers' psychological weapons to 11 and scale up processes that were previously very manual.
> AGI taking over the world? Does no one realize that without people going out every day and doing work the world ends in a week... Does no one remember when we had the spate of transformer shootings? A rifle or a bucket of water ends AGI threats
This isn't a real threat and I think most people are missing the more practical threat: someone - either a nation state or a small group of well resourced individuals - hitting an industrial singularity, allowing them to rapidly scale up self-improving robotics and automation to the point where their only limit is their ability to acquire raw materials. I think by definition, long before we hit an AI singularity with runaway super intelligence that can take over the world, we'll have a much dumber AI capable of maximizing killer paperclips for their human overlords to take over the world (that's my plan anyway :-)).
I don't think we're anywhere near the latter yet, but the language in the OpenAI blogpost announcing Sora points in that direction. The first step to an AI that can orchestrate a factory is learning the physical world through video. Once an AI model can follow instructions and directly control a robotic arm via camera, we'll be entering the next stage of the industrial revolution.
Skynet might actually be the more realistic scenario to consider, assuming we're specifically casting about for potentially far-fetched worst-case scenarios. Which would be a reasonable thing to do under the minimax principle.
So, some organization deliberately plugging the technology into something dangerous like a defense computer network that includes computers that can be used to control weapons systems. That would potentially be more worrisome because one would assume that the organization setting it up, being a military agency, would be building it to be more resistant to attack than your average municipal power grid.
Also, we don't necessarily need to posit that it deliberately tries to take over the world, Skynet-style. Just that it does something surprising and unintended. Which is not necessarily all that far-fetched? Despite the leaps and bounds we've made in raw performance at certain tasks, this whole branch of technology continues to be notoriously flaky.
Job loss for knowledge workers, especially affecting intro jobs (I'm thinking of frontend dev or paralegal work, which is how a lot of people get a foothold in these fields). What are reasonable degrees to take out a college loan for in 2024?
Militaries using AI checkpoints, guards, patrols, and loitering munitions (from a Humanitarian / rules of war perspective, not necessarily existential risk). Possibly to continue regimes where human soldiers would defect or differentiate civilians from armed opponents.
AI-reinforced media bubbles promoting alternative realities (most people blame this on social media, but from QAnonCasualties it seems like YouTube and Facebook recommendations hooked a lot of people before mainstream companies assessed whether they could be harmful). I think this is more core than "deepfakes" because people believe even fake Tweet screenshots if it backs the narrative.
>> What are reasonable degrees to take out a college loan for in 2024?
What degree do you take in college in 2004? How about 1960? Because realistically there are a lot of people who got degrees in what amounts to underwater basket weaving and are upset that there isnt a place for the in the work force already. I dont think foreknowlge would change peoples poor choices!
> Militaries using AI checkpoints, guards, patrols, and loitering munitions
> QAnonCasualties it seems like YouTube and Facebook
As an atheist, people stoped going to church, talking about Bigfoot and put their bullshit elsewhere... IN my book the new boss is the same as the old boss, just on a different pulpit.
This is like asking a dodo in 1598 to be able to explain what the threat of humans are. Their inability to do so isn't related to the probability that the threat is real.
Likewise, our inability to articulate the threat posed by AI isn't evidence that a threat does not exist. We simply don't know that threat is and even if we knew there was a threat, there's no guarantee that we would be able to comprehend it enough to describe it. Rather, our inability to comprehend super-intellegent threats make us more susceptible to identifying and stopping them.
Why don't we apply the same logic to the status quo? Because we are, for the most part, exposed to shocks and change within the regime of normal variance and have continued to exist which is evidence for continued existence. Planetary co-habitation with super-intelligence represents a many-times-beyond the standard variance of shocks to our natural order.
If any of the lifeforms we've extincted already were as intelligent as humans, would that have increased their chances of survival? Personally, I see that as an easy yes which implies that differences in intelligence are at least somewhat correlated with extinction outcomes. But perhaps others see it otherwise.
Anything that no matter what happens, could still maybe happen in the future but we don't know, is something that I treat the same way I treat claims that the Second Coming will happen someday.
Having a customer service issue (locked account, fraud on credit card, lost package, etc) and literally never being able to reach a human:
"Your problem has been found statistically unlikely and the projected calculations of the loss of your future business does not justify the risk to fix it. This is our final determination. There is no manager to speak to, and no higher tier to escalate to. Have a nice day."
It feels silly to focus on robots destroying humanity as the risk of AI, instead of the more obvious and imminent general enshittification of society and everything downstream of further wealth inequality.
But it also feels silly that if we did need protection from robots destroying humanity, that it should be performed by the companies creating the robots.
The term "AI" hypes the capabilities and dangers of LLMs by invoking lay people's thoughts about AGI and sentience. I think firms like OpenAI play up this idea to get people interested in the tech; and some firms seem to be working toward legislation to cement their position and limit competition (since just creating an LLM is not a sufficient moat).. and this AGI angle appears to play well in lobbying.
OpenAI--one of the most vocal about the danger--is particularly vulnerable to open competition since they lack the device control that Microsoft, Google, and Apple have. And while Microsoft has funded a lot of OpenAI, they actually have a parallel AI team. The moment they no longer need OpenAI, Microsoft can pull out of their deal. OpenAIs position is particularly precarious (similar to Anthropics position--another vocal AI-danger proponent).
Ilya says he thinks LLMs are capable of achieving AGI all by themselves, but there's no evidence of that and it's open for debate. LLMs simply generate the next token based on the scoring of previous tokens.. and then does that repeatedly. I'm very skeptical of the idea that the human mind is equivalent to an LLM.
Don't get me wrong.. this tech is all very impressive. But the idea that there's no limit to this tech is unfounded. And if people understood the way LLMs generate tokens, I doubt there would be as much discussion about the dangers of LLMs.
Gato and RoboCat[0] are examples of what transformer architectures with environment/action tokens can accomplish. These are pretrained with visual and experiential data, but ultimately are "simply generating the next token". This is why it's likely that transformers are capable of AGI, and the top models are no longer strictly LLMs since they are multimodal.
> Ilya says he thinks LLMs are capable of achieving AGI all by themselves, but there's no evidence of that and it's open for debate.
This debate is probably mostly just an argument over semantics. What exactly do we mean by "Large Language Model"? What exactly do we mean by "Artifical General Intelligence"?
On the strict end of things, if "LLM" the current state-of-the-art generative transformer architecture, and if we think that "general intelligence" requires something like sentience, or at least a certain capacity for flexible, iterative symbolic reasoning, then I think that most people would agree that LLMs clearly aren't any more capable of achieving artificial intelligence than a stock 1989 Buick LeSabre is of achieving sustained flight.
But the more flexible you're willing to be about how you define "large language model" and "artificial general intelligence", the more room there is for it to be possible. But also, the less practical value there is in making such a statement. Beyond some hand-waviness event horizon, you can probably just declare that any model capable of handling natural language inputs and outputs is a language model, take a nice slow toke, and begin pontificating about the Church-Turing thesis.
43 comments
[ 2.6 ms ] story [ 115 ms ] threadJob loss? Tech already did that, the little old ladies that opened your envelope at sears and roebuck got replace by computers a long time ago...
Deepfakes? Are we talking the moon landing? Avengers movies?
AGI taking over the world? Does no one realize that without people going out every day and doing work the world ends in a week... Does no one remember when we had the spate of transformer shootings? A rifle or a bucket of water ends AGI threats.
Where is the threat from ML that doesn't already exist and is an ACTUAL threat to 8billion of us?
the section: "No, AI Safety isn't a fringe concern by sci-fi weebs"
> AI Safety / AI Risk used to be less mainstream, but now in 2024, the US & UK governments now have AI Safety-specific departments![10]10 This resulted from many of the top AI researchers raising alarm bells about it. These folks include: > Geoffrey Hinton and Yoshua Bengio > Stuart Russell and Peter Norvig > Paul Christiano
> I'm aware "look at these experts" is an appeal to authority, but this is only to counter the idea of, "eh, only sci-fi weebs fear AI Risk". But in the end, appeal to authority/weebs isn't enough; you have to actually understand the dang thing. (Which you are doing, by reading this! So thank you.)
An actual plausible threat should be easy to describe. In terms of arguments, vagueness indicates weakness.
https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
The content you link to is about AGI, which is a theoretical concept, and it is not at all clear that the path to AGI is incremental from our current approaches.
Regardless, it really feels like a pure marketing move to link concerns about AGI to our current tool chain of decision engines, content generators, and spell checkers.
Nothing that is publicly known about what OpenAI or its foes are currently working on suggests in any way that it should be linked to the emerging of AGI.
Now to the more general points made in the content you link to:
As the author states in the preamble, this is a "poorly organized list of individual rants".
And it seems that none of those make a good and plausible case for why even the theoretical idea(!) of AGI is particularly dangerous. Of course, with a good deal of good will, I can follow most of the ideas listed, but it feels forced and certainly requirement heavy.
Remember that the state they are thinking about is is in the future (probably far), and everything they write is hypothetical.
It's like if we learned to throw a stone for the first time, and suddenly the village is full of alarmist people discussing high-minded ideas about "what if everyone is suddenly thrown away, by which we mean displaced at high speed in a vaguely known direction - by a force we yet can not understand?
It's mostly not serious or related to the current technologies.
If someone is alarmist and wants to be taken serious they should be required to make an actual argument, themselves. .. without fleeing into references, vagueness or unnecessary complexity.
Let's say it wants to kill us.
It makes plans to build robot overlords....
Where is it going to build them? Where are all the lights out factories? Where are all the robots to build the robots to build the robots? Your AGI has a massive bootstrapping problem.
Now it needs to build enough robots, that are reliable enough to maintain and run an entire supply chain. How does it get power, build more chips... The whole world is run by PEOPLE ...
There is the classic I, Pencil. The toaster project (https://www.thomasthwaites.com/the-toaster-project/ ) is the more modern take. It's people who watch the walking dead and dont know enough to ask "where the hell is the gas coming from" because that pipline is days long. It's the forgotten instances of "substation attack" never mind the fact that if linesmen stopped showing up to work the grid would fall apart in weeks.
To be candid that list is the hubris of people who do not have a clue of how the real world works outside their ivory tower.
We’ve seen a huge shift of wealth in the last twenty years in no small part passing through and at the insistence of Venture Capital and Private Equity. AI will be used to again accelerate this with more layoffs more outsourcing and lower quality services captured.
Imagine an AI agent which manipulates exposed SCADA systems. That could be incredibly destructive!
GP's point (and I agree) is that an evil actor could, in principle, do those things already (and a much more intelligent one, while we're at it).
And yet, it doesn't happen. Why? Because reality is not a "winner takes all" game. Actions carry consequences and other factors limit the extent of damage any actor could do.
It is way more threatening to human life that there exists, today, a huge stockpile of nuclear weapons all around the world.
Nah, you'll be surprised at how dumb things in the real world could be; like, the nuclear briefcase password is probably 0000-0000.
Punishment and retaliation is what deters most people from going rogue. Note that by retaliation I do not mean "fear of retaliation", I mean the actual consequences that come to you from FAFO in the real world.
“Can we just turn it off?"
"It has thought of that. It will not give you a sign that makes you want to turn it off before it is too late to do that."
https://youtu.be/_8q9bjNHeSo?si=a7PAHtiuDIAL2uQD&t=4817
If AI ever replaces white collar jobs then that will be at an unprecedented scale and with essentially nowhere to shift to.
But so far it doesn‘t seem to happen.
Our parents and grandparents getting scammed by deepfakes seems like a very real threat. Not only do anecdotes abound (plus FBI's database of reports) but the elderly were already vulnerable to these scams since before the telephone was invented. LLMs and all the voice cloning tech really turns up scammers' psychological weapons to 11 and scale up processes that were previously very manual.
> AGI taking over the world? Does no one realize that without people going out every day and doing work the world ends in a week... Does no one remember when we had the spate of transformer shootings? A rifle or a bucket of water ends AGI threats
This isn't a real threat and I think most people are missing the more practical threat: someone - either a nation state or a small group of well resourced individuals - hitting an industrial singularity, allowing them to rapidly scale up self-improving robotics and automation to the point where their only limit is their ability to acquire raw materials. I think by definition, long before we hit an AI singularity with runaway super intelligence that can take over the world, we'll have a much dumber AI capable of maximizing killer paperclips for their human overlords to take over the world (that's my plan anyway :-)).
I don't think we're anywhere near the latter yet, but the language in the OpenAI blogpost announcing Sora points in that direction. The first step to an AI that can orchestrate a factory is learning the physical world through video. Once an AI model can follow instructions and directly control a robotic arm via camera, we'll be entering the next stage of the industrial revolution.
So, some organization deliberately plugging the technology into something dangerous like a defense computer network that includes computers that can be used to control weapons systems. That would potentially be more worrisome because one would assume that the organization setting it up, being a military agency, would be building it to be more resistant to attack than your average municipal power grid.
Also, we don't necessarily need to posit that it deliberately tries to take over the world, Skynet-style. Just that it does something surprising and unintended. Which is not necessarily all that far-fetched? Despite the leaps and bounds we've made in raw performance at certain tasks, this whole branch of technology continues to be notoriously flaky.
Militaries using AI checkpoints, guards, patrols, and loitering munitions (from a Humanitarian / rules of war perspective, not necessarily existential risk). Possibly to continue regimes where human soldiers would defect or differentiate civilians from armed opponents.
AI-reinforced media bubbles promoting alternative realities (most people blame this on social media, but from QAnonCasualties it seems like YouTube and Facebook recommendations hooked a lot of people before mainstream companies assessed whether they could be harmful). I think this is more core than "deepfakes" because people believe even fake Tweet screenshots if it backs the narrative.
What degree do you take in college in 2004? How about 1960? Because realistically there are a lot of people who got degrees in what amounts to underwater basket weaving and are upset that there isnt a place for the in the work force already. I dont think foreknowlge would change peoples poor choices!
> Militaries using AI checkpoints, guards, patrols, and loitering munitions
This ship sailed: https://www.military.com/daily-news/2020/06/17/missile-full-...
> QAnonCasualties it seems like YouTube and Facebook
As an atheist, people stoped going to church, talking about Bigfoot and put their bullshit elsewhere... IN my book the new boss is the same as the old boss, just on a different pulpit.
On this site, of all sites, it's remarkable how many people overlook that scale and automation lead to important qualitative shifts.
This is like asking a dodo in 1598 to be able to explain what the threat of humans are. Their inability to do so isn't related to the probability that the threat is real.
Likewise, our inability to articulate the threat posed by AI isn't evidence that a threat does not exist. We simply don't know that threat is and even if we knew there was a threat, there's no guarantee that we would be able to comprehend it enough to describe it. Rather, our inability to comprehend super-intellegent threats make us more susceptible to identifying and stopping them.
Why don't we apply the same logic to the status quo? Because we are, for the most part, exposed to shocks and change within the regime of normal variance and have continued to exist which is evidence for continued existence. Planetary co-habitation with super-intelligence represents a many-times-beyond the standard variance of shocks to our natural order.
If any of the lifeforms we've extincted already were as intelligent as humans, would that have increased their chances of survival? Personally, I see that as an easy yes which implies that differences in intelligence are at least somewhat correlated with extinction outcomes. But perhaps others see it otherwise.
"Your problem has been found statistically unlikely and the projected calculations of the loss of your future business does not justify the risk to fix it. This is our final determination. There is no manager to speak to, and no higher tier to escalate to. Have a nice day."
This ship sailed.
- Gandalf the Grey
But it also feels silly that if we did need protection from robots destroying humanity, that it should be performed by the companies creating the robots.
Jan Leike Resigns from OpenAI - https://news.ycombinator.com/item?id=40363273 - May 2024 (391 comments)
Ilya Sutskever to leave OpenAI - https://news.ycombinator.com/item?id=40361128 - May 2024 (781 comments)
OpenAI--one of the most vocal about the danger--is particularly vulnerable to open competition since they lack the device control that Microsoft, Google, and Apple have. And while Microsoft has funded a lot of OpenAI, they actually have a parallel AI team. The moment they no longer need OpenAI, Microsoft can pull out of their deal. OpenAIs position is particularly precarious (similar to Anthropics position--another vocal AI-danger proponent).
Ilya says he thinks LLMs are capable of achieving AGI all by themselves, but there's no evidence of that and it's open for debate. LLMs simply generate the next token based on the scoring of previous tokens.. and then does that repeatedly. I'm very skeptical of the idea that the human mind is equivalent to an LLM.
Don't get me wrong.. this tech is all very impressive. But the idea that there's no limit to this tech is unfounded. And if people understood the way LLMs generate tokens, I doubt there would be as much discussion about the dangers of LLMs.
[0] https://deepmind.google/research/publications/35829/
This debate is probably mostly just an argument over semantics. What exactly do we mean by "Large Language Model"? What exactly do we mean by "Artifical General Intelligence"?
On the strict end of things, if "LLM" the current state-of-the-art generative transformer architecture, and if we think that "general intelligence" requires something like sentience, or at least a certain capacity for flexible, iterative symbolic reasoning, then I think that most people would agree that LLMs clearly aren't any more capable of achieving artificial intelligence than a stock 1989 Buick LeSabre is of achieving sustained flight.
But the more flexible you're willing to be about how you define "large language model" and "artificial general intelligence", the more room there is for it to be possible. But also, the less practical value there is in making such a statement. Beyond some hand-waviness event horizon, you can probably just declare that any model capable of handling natural language inputs and outputs is a language model, take a nice slow toke, and begin pontificating about the Church-Turing thesis.
If someone can hack your system, regardless of whether that someone is an LLM, I don't think it's accurate to call it "secure".