Ask HN: Have top AI research institutions just given up on the idea of safety?

[dead]

Not an insider but someone who uses the tools. It's a branding update, nothing more. The models haven't gotten any less sanctimonious, but the companies behind them have stopped harping on their restrictions in order to appeal to a broader customer base (gov contracts, etc.)

So the guardrails (for you and me) are still there. They just stopped committing the unforced error of excluding themselves from federal procurement. Under a different administration, the requirement might change, and you might see them boasting once more on "safety."

Well, Anthropic clearly has some kind of lines if their recent argument with the us government is anything to go by. "don't kill humans" isn't all of safety or alignment goals but it is something?

The goal is and always was to make as much money as possible. Any consideration for how it affects actual people was marketing to get ahead of bad PR and regulation.

Safety was never a genuine concern. They simply don't benefit from marketing themselves that way anymore so they've stopped pretending.

At this point, do you really think any of these "labs" would give up competitive parity or advantage just because they're already making life worse for a lot of people and (by their own admission) stand to make it much, much, much worse? The persistence forecast says no.

There are maybe a few token exceptions, like Anthropic's current pushback against the DoD, but by and large I think we can continue to expect them to pay lip service to safety while continuing to build toward systems that, by their own admission, have incredible potential to cause harm. As you noted, the fact that they employ safety researchers does not necessarily mean that they will put safety over revenue.

I don't think they've given up on the idea, but as AI becomes increasingly mainstream, the labs will be under immense pressure to hold the line. We're seeing this play out right now with Anthropic and the Pentagon.

These companies have raised eye-watering amounts of funding, and will need to continue to do so for the foreseeable future. They're not yet self-sustaining, and this insecurity increases the pressure for them to compromise on ideals.

With that said, there is a massive war for top talent, and I think that the employees at the labs would become increasingly uncomfortable with their work being used for Bad Things. If Anthropic capitulates to the Pentagon, it wouldn't surprise me to see a mass exodus of talent occur.

HA, you mean has PR given up on pretending? Yes.

We're not in a race to the bottom with "AI", we're in a speedrun to the bottom.

More on the periphery than an insider, but I personally know researchers in all three major labs who were there long before GPT-3. They all care about existential safety, a lot. In the sense that they believe there's a meaningful chance all humans are dead a decade from now (and that that's a bad thing; unfortunately, there are also people deeply involved who don't think human extinction is a bad thing).

The issue is that they're embedded in capitalism, and that drives the labs to push further and faster than is responsible. They (and unfortunately us) end up in a race where no individual feels like they can back off or halt, because if they do, they will be destroyed.

Related:

Anthropic Drops Flagship Safety Pledge

https://news.ycombinator.com/item?id=47145963

Are you asking about top AI research institutions or leading AI businesses? There’s tons of work in research communities.

[dead]

I was the author for the practitioners implementation section for the IEEE 7010 standard for assessing human impact from AI software

https://standards.ieee.org/ieee/7010/7718/

I also worked closely with Jack Clark at OpenAI before he disappeared on all these issues as CTO back in 2018

There are literally zero “AI labs” that have ever cared about “safety”

none of them have ever done anything tangible with any kind of independent auditable third-party way that has some defined reference baseline for what is safe and what is not, how to evaluate it, or a practitioners guidance for how to determine what it is and what is not safe as a designer.

They follow the same rules as every other technology platform: do as much as you can legally get away with no more no less

I say this as somebody who’s been actively involved in the AI “safety” debate for a long time now at least since 2013

The concept itself doesn’t even make sense if you fully understand the intersectional scope of technology and society

Societies demands are the things that are unsafe not the technologies themselves

Just like Bertrand Russell said “as long as war exists all technologies will be utilized for it” - you can replace “war” for anything that you think is unsafe

"safe" is such a subjective concept to begin with, have any of the model providers ever defined what they mean by "safe"?

It doesn't mean much to me if a safe model is one that does not output the recipe for mustard gas, that information is trivially available elsewhere.

Or, is a safe model one that doesn't come off as racist? Ok but i would classify that as unoffensive instead of safe but I admit definitions of words can be fluid and change.

Is a safe model one that refuses to produce code for a weapons system? Well.. does a PID controller count? I can use that to keep a gun pointed at a target or i can use that to prevent a baby rocker from falling over.

Maybe they're giving up on "safe" because there's no definitive way to know if a model is safe or not. I've always held the opinion that ai safety was more about brand safety. Maybe now the model providers can afford some bad press and it not be the death of their company.

It's such an US thing to ask "Are they doing the thing they are doing?" You've read about it, you can clearly see it in action from multiple companies, yet you're here asking "hey is this happening?"

Yes. Yes it is. Yes they are giving up on safety. They are openly saying so. It is easy to see if you take just a second to look for yourself instead of looking at press releases and algorithmic promotion.

https://time.com/7380854/exclusive-anthropic-drops-flagship-...

Why the constant chasing after universally applicable generalizations?

Some of them are pandering. Some aren't. Some care. Some don't.

Businesses with ferocious funding needs are vulnerable to pressure (internal and external) to do whatever aligns with money and power. Money and power will flow into the ones so-aligned. That is the nature of the parasitic extraction models that typically drive decision making at those kinds of companies.

Safety is a nice idea but it’s not structurally pursuable at this point. Everything is moving too quickly and we don’t exactly know what is useful or not, just like we don’t know what’s safe or not.

Anyone pursuing safety will be outcompeted by someone who isnt. Given the amount of investments there is no patience for any calls to slow down. I tend to believe this won’t actually end in disaster as I don’t think it’s actually economical to put AI everywhere with enough real control that we can’t manage the risks as they evolve, but it’s a low confidence prediction.

I think there are plenty of people working on it still, I am even working on a startup that prioritizes safety for physical autonomous systems!

The problem is that safety is written in blood. Airlines implemented flight recorders / black boxes and various processes after major incidents. A major mistake occurs that causes death or destruction to property, or both, an investigation occurs, we learn from it, and introduce new laws and regulations to prevent a reoccurence.

A safety team at the hammer company cannot prevent me from using it to bang your head.

You can align to the user wants and so you are a hammer. This is alignment>safety.

Or you take a safety first approach where the AI decides what safe is and does its own bidding instead of yours. This is safety>alignment.

I prefer hammers to be honest. Mostly because humans can be prosecuted, AIs can't. So if the human wants to commit crime with the AI it should be able to, because the opposite turns to dystopia fast.

It takes imagination and emotion to be dangerous.

These token predictors will never be smart enough to be dangerous.

"AI Safety" got suborned, then dropped when it wasn't needed anymore.

Every misalignment/AI safety paper is basically a metaphor for how corporate values can misalign with actual human values under capitalism.

The first thing that happened when "AI Safety" became useful to corporate interests, is that the "goal" of it instantly became "profitability" not safety. "AI Safety" became about liability minimization, not actual safety for humanity. (Look! the system is now misaligned with the goal, wonder how that happened!?)

AI Safety concerns were instantly proven true, it happened, and now we live in the world where it is too late to prevent the superintelligences that we call "corporations" from paper-clipping us to death in pursuit of profit.

It's basically like how security is always at the very end of the budget list.

If some company says security or safety, don't expect much more than words.

Humans can't develop safety until there is enough blood in the streets. Only issue with AI is that threshold may come at a point where its too far gone to recover. But humans can't put in seatbelts until we're losing 40k people per year in car crashes. Unfortunately its just how we're wired. Those that are careful are outcompeted by the brash and the fast-moving, until the relative value of moving fast is removed, then we consider the value of making things safe. We didn't start with safe electricity, we started by killing lots of people and starting lots of fires. Many many years later, we ended up with electrical codes and standards.

The AI proponents who originally spoke of safety did so because they are aware of the dangers. However they, like all of us, are not able to change human nature or society. Molloch will drag them into the most dangerous game or eliminate them from the competition. Only with time, death, and damage (and many lawsuits) will any measure of safety be gained. The righteous will say "see we said AI was dangerous!" but that will be the only satisfaction they can have, many years after the damage is done.

If we want to speedrun safety, the only real mechanism is to make legal recourse more viable (e.g. $1M penalty per copyright infringement, $100M per AI-related death, etc.). If this was the case, lawyers self-interest and greed will compete with the self-interest and greed of the AI corps, balancing the risk (but there is no altruistic route to solving this).

Isn't AI safety mostly a marketing thing? Like, we employ these safety people to make sure our chat bot does not turn into Skynet, implying the chat bot could turn into Skynet i.e. it's powerful and magic and please give us money.

Maybe the text prediction programs are too familiar to people for the Skynet marketing to bite like it used to.

Or maybe it was not just a marketing thing and the AI bros really did believe we were a few GPUs and some training data away from AGI, but now they no longer believe this.

It’s just still so trivial to jailbreak even the latest Anthropic models (via api, and not talking about the silly ENI or Pliny breaks) I don’t understand where the safety teams are doing their work. Is it in the default chat-trained model?