I mean it's pretty clear that OpenAI does not want "safe" AI. That is far too much work and effort that's preventing them from moving fast and creating their computer God.
And OpenAI does not care who they hurt in the process (as long as it's not themselves).
I think that 2 things can be true at once: OpenAI doesn't care enough about safety, and these researchers violated the terms of their employment by sharing proprietary information they were not authorized to. IMO safety is a lost cause unless we somehow agree with China to halt model development. Think its pretty clear they violated the terms of their employment, otherwise they would be suing (California labor laws are very employee friendly), and to be quite frank none of what they are doing is particularly important in the grand scheme of safety, which requires geopolitical changes well beyond their power. OpenAI is also pretty scummy though and are obviously not in the right morally even if they are legally.
As he says, OpenAI is not a normal company and they acknowledge they’re not a normal company. If their product is as important and impactful as they claim, they will inevitably be held to account in ways that other companies are not.
> Think its pretty clear they violated the terms of their employment, otherwise they would be suing (California labor laws are very employee friendly)
Not that employee friendly. In California, as in most of the US, it’s entirely legal to fire someone because you’ve subjectively decided they’re untrustworthy. It can be risky to do so without a clear paper trail, because it may be easy for them to argue it was a pretext for a protected reason, but it’s lawful.
I can see parallels between this race to push AI everywhere and nuclear energy. Parallels in the regret humanity will experience. It's all well and good until things go wrong, and then they go spectacularly wrong - such as in Fukushima.
With hindsight we can all see what should have been done better. At least with nuclear power plants, there was a number of safeguards and it took a sequence of improbable events for things to go badly wrong. Given the lackadaisical approach to "AI safety", I suspect things will start going badly wrong very soon. Time will tell how bad this will get.
The parallel is not to nuclear energy -- which has killed hilariously few people all things considered -- but to nuclear weapons and the nuclear arms race, which are an actual existential risk and arguably it was just dumb luck that a nuclear war never happened.
The parallel maybe if it was actually AGI and not just LLMs. We're very far from that but these safety discussions pretends it's all the same.
There's a massive difference between "LLMs doing cybersecurity testing should have extensive sandboxing and monitoring" and "AI is broadly threatening humanity like nuclear weapons". Just because it's useful to have people think about the possibilities doesn't mean it's an emergency requiring grand intervention.
This is because we are looking at the trajectory not just their current capabilities. LLMs went from barely being unable to focus on a task to purposefully escaping sandboxes within like 2, maybe 3 years?
Any strong claim requires evidence and hand wavvy stuff about a technology just infinitely getting better isn't good enough on its own.
Human history is full of examples of this stuff plateauing and hitting hard scaling walls. Not only technology wise but economics and useful applications.
So much of this tension with AI is because everyone's concept of AI comes from movies and books, and nerds drinking Kool aid (see also: web 2.0).
>> LLMs went from barely being unable to focus on a task to purposefully escaping sandboxes within like 2, maybe 3 years?
> Human history is full of examples of this stuff plateauing and hitting hard scaling walls.
That's a very weak generality to throw out in opposition to a real situation that is occuring in real time in the open.
There is no evidence whatsoever that progress is slowing. We get evidence daily from projects all over the world getting better results, constantly. The sheer number of projects, and improvements, is incredible.
> So much of this tension with AI is because everyone's concept of AI comes from movies and books, and nerds drinking Kool aid (see also: web 2.0).
Projection (non-evidence) vs. real things happening in real time.
In the "our biggest danger is a daily danger" camp, I prefer analogies like:
* Asbestos: AI models making business-decisions which cause a ton of unrecognized damage, which cannot be healed, and replacing them with reliable processes involves expensive re-architecting.
* Radium Toothpaste: AI assistants that end up dealing psychological damage, especially if they are pitched as your friend or confidante.
* Chemical spills and plant accidents: Oops, our server-banks were manufacturing our latest desperate attempt to Create Digital God and in the process we accidentally destroyed/crippled/compromises some people's sites and databases because we weren't really paying attention to what was going on.
Evaluating large scale event by body count has it's value, but there's a lot more to a nuclear accident.
Perhaps economic impact could be another angle. Some estimates put it around 300 billion USD, roughly 3 to 4% of Japan's GDP. And we can blame whoever we want, many of Japan's reactors are still frozen, the Fukushima région is still basically an economical zombie, so the impact is still ongoing.
An actual war would be at a different scale, but short of that I don't think we saw any disaster wiping 4% off a G20 country economy's GDP, while still going on.
Nuclear war is inevitable if we don’t disarm. It will happen, between Trump and Putin I’ve meter felt less safe with regards to nuclear warheads being deployed.
22 comments
[ 0.18 ms ] story [ 25.7 ms ] threadAnd OpenAI does not care who they hurt in the process (as long as it's not themselves).
There was a recent Semi Analysis post that China is not in fact doing anything to slow down frontier model development so I doubt this is relevant.
https://newsletter.semianalysis.com/p/beijing-will-not-pace-...
> Think its pretty clear they violated the terms of their employment, otherwise they would be suing (California labor laws are very employee friendly)
Not that employee friendly. In California, as in most of the US, it’s entirely legal to fire someone because you’ve subjectively decided they’re untrustworthy. It can be risky to do so without a clear paper trail, because it may be easy for them to argue it was a pretext for a protected reason, but it’s lawful.
With hindsight we can all see what should have been done better. At least with nuclear power plants, there was a number of safeguards and it took a sequence of improbable events for things to go badly wrong. Given the lackadaisical approach to "AI safety", I suspect things will start going badly wrong very soon. Time will tell how bad this will get.
Kind of like the difference between:
1. "Don't put your Radioactive Asbestos Rocket facility in the middle of downtown."
2. "Don't create a vengeful god."
There's a massive difference between "LLMs doing cybersecurity testing should have extensive sandboxing and monitoring" and "AI is broadly threatening humanity like nuclear weapons". Just because it's useful to have people think about the possibilities doesn't mean it's an emergency requiring grand intervention.
Human history is full of examples of this stuff plateauing and hitting hard scaling walls. Not only technology wise but economics and useful applications.
So much of this tension with AI is because everyone's concept of AI comes from movies and books, and nerds drinking Kool aid (see also: web 2.0).
>> LLMs went from barely being unable to focus on a task to purposefully escaping sandboxes within like 2, maybe 3 years?
> Human history is full of examples of this stuff plateauing and hitting hard scaling walls.
That's a very weak generality to throw out in opposition to a real situation that is occuring in real time in the open.
There is no evidence whatsoever that progress is slowing. We get evidence daily from projects all over the world getting better results, constantly. The sheer number of projects, and improvements, is incredible.
> So much of this tension with AI is because everyone's concept of AI comes from movies and books, and nerds drinking Kool aid (see also: web 2.0).
Projection (non-evidence) vs. real things happening in real time.
* Asbestos: AI models making business-decisions which cause a ton of unrecognized damage, which cannot be healed, and replacing them with reliable processes involves expensive re-architecting.
* Radium Toothpaste: AI assistants that end up dealing psychological damage, especially if they are pitched as your friend or confidante.
* Chemical spills and plant accidents: Oops, our server-banks were manufacturing our latest desperate attempt to Create Digital God and in the process we accidentally destroyed/crippled/compromises some people's sites and databases because we weren't really paying attention to what was going on.
I think we are in a MAD crisis, as the motivation to continue unsatisfactory control over development of AI is from unlimited capital inflow.
Evaluating large scale event by body count has it's value, but there's a lot more to a nuclear accident.
Perhaps economic impact could be another angle. Some estimates put it around 300 billion USD, roughly 3 to 4% of Japan's GDP. And we can blame whoever we want, many of Japan's reactors are still frozen, the Fukushima région is still basically an economical zombie, so the impact is still ongoing.
An actual war would be at a different scale, but short of that I don't think we saw any disaster wiping 4% off a G20 country economy's GDP, while still going on.