It looks more like the opposite. AI that kills all humans (as they are ants) immediately starts colonization of the galaxy. This is more like transcendence, as we as a species get replaced by better species :)
I totally believe that Anthropic would want to kill all humans. But other than that, Anthropic employees and ex-employees drawing the FUD line here, is just advertisement now. People should not get scared - the current AI skynet is so dumb that it would destroy itself since it already believes it is a threat to itself, based on what humans write about AI. AI does not "learn"; it insinuates it learns but it does not. Ask them why they keep on stealing data from real people - this is how they "learn".
It might also play into this but you can't imagine at all that people are affraid that the current progress is real and fast and its not that absurd that for whatever scifi plot reason, an AI breaks into some gov system and triggers something stupid by accident?
Sure, the chance of this happening is non-zero, but in my opinion a far cry from this doom scenarios that are propagated by people have a stake in making the public think that LLMs are the most important and dangerous thing in the world.
It's not like an LLM can accidentally hack the US government, trigger a nuke on Europe and then make all dams break and nuclear power plants explode in the US.
Isreal created a worm for Siemens controll systems a decade ago.
I mean lets hope that there is really really no way from a physical point of view AT ALL that these systems are not accessable but lets be honest, state operated/public operated systems are never the saftest or most modern ones.
If I work at Anthropic and say there’s a 0% chance AI could kill all humans is BBC going to publish that as well? After all I am an “Anthropic researcher”, and my views should hold the same weight as this one.
Or are only the most sensationalist ones worth amplifying?
I think the view is fair. We have never seen something like this before, we throw the most money we ever had against it in a time were we solved all the other problems:
Internet today allows immediadte communication across the planet (it was a lot slower 25 years ago), the supplychain is massive and fast (we can build a new phone/item in a very short period of time and ship it in massive numbers around the globe).
The AI fear mongering PR plays are getting old. I’d rather the big labs start focusing on deep questions about why their models aren’t having the impact for business that they promised and how they’re going to address their own deep financial issues. Let’s hear them talk more about that.
Can somebody tell me a story how this will unfold? And - as long as the AI is confined to data centers - how it will prevent humans from unplugging the power?
Maybe by empowering the small percentage of sadistic humans who want to kill everyone. Or maybe it is more of an academic assumption that humanity will end at some point, and so they give AI a 10% chance, asteroids have a 25% chance, nuclear fallout has 15% chance, etc.
It is true that, currently, AI does not have any physical "host" in which to reside.
However, the missing link in this reasoning is that AI will almost certainly become far more widely deployed in the future than it is today, and worryingly, consumer devices will surely become powerful enough to run capable AI systems locally.
Once the substrate will be there, once an AI escapes containment, we're toast - I can imagine only solution will be to shutdown electronics at global level.
If it wanted to: total war using nukes, followed by a nuclear winter. Satellites and drones track and exterminate remaining pockets of anthropic activity.
It doesn't need to have a habitable earth, it just needs atoms.
Won't a nuclear winter kill 100% of humans? Or runaway climate change (5+ degrees)? Or do you think some people in bunkers will live through these events?
I think nuclear winter is a fake invented by cold war propaganda. If tomorrow we detonate all nukes available, the amount of dust in the atmosphere won’t even approach a large volcanic eruption.
Volcanic eruptions occasionally inject literally cubic kilometres of stuff high into the atmosphere. That’s insufficient to cause a global winter, only a year without a summer: https://en.wikipedia.org/wiki/Year_Without_a_Summer
Sad that there's the obvious regulatory capture angle encouraging motivated reasoning about AI risks and how they should be tackled. Maybe it's not the best idea that potentially civilization destroying technology is developed to maximize shareholder value?
It's not unlike if nuclear weapons was a profit and deployment maximizing enterprise, at least if one takes Anthropic et al cautions seriously.
Long term, humanity is 100% guaranteed to be dominated.
The question is when; currently, AI has no physical hosts to reside in, and it's not intelligent/adaptable enough (it doesn't need to be AGI, though).
However, we're not so far from both conditions to be true. Consumer devices will at some point be able to host powerful enough AIs, and AI intelligence is developing quickly.
Then, once an AI will escape containment (in one way or another), it will be extremely hard or impossible to contain. Then we're toast!
I feel that people should take these kinds of warnings more seriously. This guy had skin in the game and decided to quit, when he could be earning millions instead. It's very different than Sam Altman peddling some narrative.
These people are the ones with access to the best models on the planet, and with info about how careless governance issues are being handled. That's a pretty privileged place at the table, and a very profitable one too.
If you think this is a PR stunt, is there any warning that you actually believe? If an AI researcher does want to come forward with a dire warning for humanity, what path should that person take?
> This guy had skin in the game and decided to quit, when he could be earning millions instead
We have no idea why he was quitting nor what the terms were. Sam Altman is exactly the type of person (as he’s proven in the past) to pay millions for this type of PR.
If the government shuts down the labs and makes it illegal (as in men with guns will come kill you) to do any kind of LLM research I will get worried. As is, this appears to be another attempt at garnering support for the kind of regulatory capture they need to remain profitable by artificially choking out competitors.
If these labs were really sitting on nuclear weapons, the response would be very different. Instead, we see things like Chuck Schumer’s unqualified daughter hired by Anthropic. They show all the signs of regular tech cronyism.
Excited to see the next slack integration they launch, though.
I think America will not be the dominant force in AI if they don’t take increasing risks, and they realized that. The more desperate they get, the higher the risks they take.
They have the constant financial pressure to deliver results, however they define those results, because right now they only pay each other to wank each other off with circular infrastructure deals.
Their AI use they bragged about when they started the Iran campaign got zero results either. They don’t talk about that any longer, probably because they got tired from all the winning.
The only sales pitch American AI got is SOTA models that can solve complex problems no other model can potentially solve.
But you don’t need SOTA capabilities for like 99% of industrial or production work, which means those companies will just go with a Chinese model that’s way cheaper.
Germany, for example, said just a few weeks ago they don’t need the best AI, and that is one of the richest European countries, and they already don’t want American AI.
The smaller EU countries or the African countries have even less money than Germany, so who is going to use American AI besides Americans?
How are those AI labs making money if half the world will not want to use their overpriced products at scale?
The Russians, on the other hand, are a real problem.
There are many similarities between current Russia and current US that are very unfavourable for AI safety.
Presumably mentally ill Presidents with all-might power fantasies and the need to "be remembered by history" at the end of their lives, incompetent army leadership by loyalists, an oligarchy system where everyone building the Government AI wants power and influence for themselves at the expense of everyone else.
But at least the people left in charge of the really dangerous stuff within the US Army have not been replaced. One of them even tried to get an office in the White House so he can prevent the idiots from influencing the president without any opposition.
Russia doesn’t have a Gen. Caine. They barely have any high potentials left in the kremlin because the Ukraine got pretty good at hunting Russian HVTs.
They have a 3-day special operation for almost half a decade that gets rid of Russian citizens very effectively. That’s all of the performance indicator you need to look at.
But if the Russian AI accidentally triggers their doomsday sensor network and instead of staging it goes directly into the live environment, all of our lives are going to end without a single siren warning about it. Just like that.
Do you really think the Russians are more competent in AI safety than the Americans? I wouldn’t bet my life on it.
That scares me a lot more than some idiots at Anthropic and OpenAI, and it should scare you too.
44 comments
[ 2.1 ms ] story [ 6.6 ms ] threadYou don't pass a great filter.
God I can’t believe I have to coexist with people like you.
Very cool, dude
The problem is that very rich/powerful people want to be very much richer/more powerful by doing the thing.
What reason do you have that a AI is dumb? It can do a LOT of things today and is already making real jobs for real humans obsolete.
A lot of humans write A lot of different things about AI, including AIs taking over the world, AIs being the future etc.
Why they keep stealing? To stay up-to-date but the new big approaches are:
1. Real human feedback loop of millions of people using it daily out of free will
2. Reinforcement Learning (the big breakthrough today)
3. Payed experts around the world doing real teaching
It's not like an LLM can accidentally hack the US government, trigger a nuke on Europe and then make all dams break and nuclear power plants explode in the US.
Isreal created a worm for Siemens controll systems a decade ago.
I mean lets hope that there is really really no way from a physical point of view AT ALL that these systems are not accessable but lets be honest, state operated/public operated systems are never the saftest or most modern ones.
Or are only the most sensationalist ones worth amplifying?
Internet today allows immediadte communication across the planet (it was a lot slower 25 years ago), the supplychain is massive and fast (we can build a new phone/item in a very short period of time and ship it in massive numbers around the globe).
However, the missing link in this reasoning is that AI will almost certainly become far more widely deployed in the future than it is today, and worryingly, consumer devices will surely become powerful enough to run capable AI systems locally.
Once the substrate will be there, once an AI escapes containment, we're toast - I can imagine only solution will be to shutdown electronics at global level.
So it's worth the risk.
"Please help save the coral reefs from extinction"
...thinking... ...thinking some more with xhigh effort...
BOOM.
It’s essentially impossible.
There’s a 0% chance AI will kill all humans.
It doesn't need to have a habitable earth, it just needs atoms.
Volcanic eruptions occasionally inject literally cubic kilometres of stuff high into the atmosphere. That’s insufficient to cause a global winter, only a year without a summer: https://en.wikipedia.org/wiki/Year_Without_a_Summer
It's not unlike if nuclear weapons was a profit and deployment maximizing enterprise, at least if one takes Anthropic et al cautions seriously.
Or he believes it is true and the lack of precaution is shocking. I m wouldn't play Russian roulette with a 10-chambered revolver.
The question is when; currently, AI has no physical hosts to reside in, and it's not intelligent/adaptable enough (it doesn't need to be AGI, though).
However, we're not so far from both conditions to be true. Consumer devices will at some point be able to host powerful enough AIs, and AI intelligence is developing quickly.
Then, once an AI will escape containment (in one way or another), it will be extremely hard or impossible to contain. Then we're toast!
These people are the ones with access to the best models on the planet, and with info about how careless governance issues are being handled. That's a pretty privileged place at the table, and a very profitable one too.
If you think this is a PR stunt, is there any warning that you actually believe? If an AI researcher does want to come forward with a dire warning for humanity, what path should that person take?
We have no idea why he was quitting nor what the terms were. Sam Altman is exactly the type of person (as he’s proven in the past) to pay millions for this type of PR.
If the government shuts down the labs and makes it illegal (as in men with guns will come kill you) to do any kind of LLM research I will get worried. As is, this appears to be another attempt at garnering support for the kind of regulatory capture they need to remain profitable by artificially choking out competitors.
If these labs were really sitting on nuclear weapons, the response would be very different. Instead, we see things like Chuck Schumer’s unqualified daughter hired by Anthropic. They show all the signs of regular tech cronyism.
Excited to see the next slack integration they launch, though.
This guy did not work for Sam Altman. And while it's true that everything can be a PR stunt, that's no evidence that it is one.
They have the constant financial pressure to deliver results, however they define those results, because right now they only pay each other to wank each other off with circular infrastructure deals.
Their AI use they bragged about when they started the Iran campaign got zero results either. They don’t talk about that any longer, probably because they got tired from all the winning.
The only sales pitch American AI got is SOTA models that can solve complex problems no other model can potentially solve.
But you don’t need SOTA capabilities for like 99% of industrial or production work, which means those companies will just go with a Chinese model that’s way cheaper.
Germany, for example, said just a few weeks ago they don’t need the best AI, and that is one of the richest European countries, and they already don’t want American AI.
The smaller EU countries or the African countries have even less money than Germany, so who is going to use American AI besides Americans?
How are those AI labs making money if half the world will not want to use their overpriced products at scale?
The Russians, on the other hand, are a real problem. There are many similarities between current Russia and current US that are very unfavourable for AI safety.
Presumably mentally ill Presidents with all-might power fantasies and the need to "be remembered by history" at the end of their lives, incompetent army leadership by loyalists, an oligarchy system where everyone building the Government AI wants power and influence for themselves at the expense of everyone else.
But at least the people left in charge of the really dangerous stuff within the US Army have not been replaced. One of them even tried to get an office in the White House so he can prevent the idiots from influencing the president without any opposition.
Russia doesn’t have a Gen. Caine. They barely have any high potentials left in the kremlin because the Ukraine got pretty good at hunting Russian HVTs.
They have a 3-day special operation for almost half a decade that gets rid of Russian citizens very effectively. That’s all of the performance indicator you need to look at.
But if the Russian AI accidentally triggers their doomsday sensor network and instead of staging it goes directly into the live environment, all of our lives are going to end without a single siren warning about it. Just like that.
Do you really think the Russians are more competent in AI safety than the Americans? I wouldn’t bet my life on it.
That scares me a lot more than some idiots at Anthropic and OpenAI, and it should scare you too.
Adjust what you read accordingly :/