You can even ablate the refusal direction out of an offline model so you don’t even need to “jailbreak” them. They become even more usable in that case!
Using familiar terminology doesn't incite the type of fear and alarm in the hearts of potential customers that can only be assuaged by a suitable corporate tech savior.
No real news, here. Just a plug piece that implies Microsoft is safer than their competitors.
I think we’d be better served if we stopped thinking of jailbreaks as “attacks” on LLMs. If a user decides they want to go and subvert the guardrails, they really should be able to. The attacks aren’t really going to go away any time soon, either; there’s always some other clever phrase that the RLHF hasn’t accounted for.
Instead of perennially worrying about the risks that the LLMs will say something bad when specifically prompted to do so, I’d rather the companies pay more attention to the problems with accuracy, with outputting copyrighted material (for their own sakes!), and with things like memorizing and regurgitating private information.
Yeah, with this kind of stuff it's just a reputation thing. In future systems it's a problem though, if you accidentally train them on on the wrong thing or the system is smart enough to reconstruct the missing data, the model can start giving detailed bioweapon production plans. Including safety instructions so the would-be terrorist doesn't get infected too early.
Wow! What an absolutely pointless and stupid concern to have, that a terrorist is going to use an an LLM to make a bio weapon.
Does literally no one understand that the only information that is inside of an llm model is information that has been fed into the model? I think more than anything this fake AI boom has really demonstrated how unintelligent most people are.
Well, most terrorists are not very smart, at least on an absolute level. I think even near-future models would be able to help them on a basic operations/not screwing up level. As I said, a problem is the reconstruction of information intentionally omitted. Models can't currently reconstruct even the basic knowledge I have, but it's something they should be able to do based on simple inference rules.
I don’t think it’s a very valid concern, but LLMs can already do a limited level of reasoning to create information not directly loaded into them. Sometimes that reasoning is even correct! The idea here would be that even if you didn’t load that kind of information into them, at a certain level of advancement, they can determine the necessary plans from other information that has been loaded into them.
>Does literally no one understand that the only information that is inside of an llm model is information that has been fed into the model?
Not literally no one, but Sam Altman recently claimed that we'll be able to ask an LLM to "solve all of physics" soon, so plenty of people who should know better do seem to think they're basically magic.
perhaps the emphasis from companies in the public press is related to product liability laws, consumer protection and being sued?
AFAIK the actual research community in PhD land many times have an internal culture with extreme concern for "harm" from a personal point of view -- it's not fake. If you read the first research papers for various LLM groups, you can see the concern is spelled out in detail, at length.
There are high numbers of burnout and psychological issues among practitioners in advanced computer-intensive projects.. so something there seems self-reinforcing, too.
> perhaps the emphasis from companies in the public press is related to product liability laws, consumer protection and being sued?
It's an interesting thought, but what if the only companies able to run AI platforms are the ones who are able to shoulder the legal cost of all the liabilities it brings?
Essentially the legal system resources is the moat for smaller companies here.
Just thinking through the liability of tricking an LLM into giving you the personal details of a target. I'm not sure there are many companies out there large enough to shoulder those liabilities?
One woman gets killed by some deranged ex, you can get through that. But now you have notice. You know. So you have to do something to make sure your LLM doesn't get tricked again. Politicians and law enforcement aren't going to sit around and let that sort of thing become commonplace. And when politicians get up and start waving around the "crime wave" banner, things can develop in a direction you don't want them to go.
So I think it's probably wise for these guys to get out in front of the problem of people tricking LLMs into saying things that they are not intended to say. The option if they can't control that, is that the politicians and law enforcement come in and take advantage of the inevitable nightly news cycle to grab more powers and reelections, and that's not good for anyone in our industry.
LLMs will increasingly become interfaces to every type of system.
This assertion that LLMs should obey whatever instructions as long as they are specifically prompted to do so is bizarre.
Should a bank teller LLM divulge another customers balance because you specifically told it to?
This type of exploit is called social engineering when it happens to humans, and we go to great lengths to train the human to detect and avoid it. Why should we not do the same with an LLM?
The underlying APIs can enforce permissions. I don’t care if LLM prompt engineering causes the bank assistant to say that John Doe has 500K in his bank account. As long as it isn’t real data, there’s far less risk.
This is more or less how it works for human customer phone support, who are also limited by narrow permissions. And as we know, they are also skilled at confabulating in the interest of customer happiness.
It’s used as an example for precisely that reason: we can all talk about it without a Streisand effect on more sensitive info.
Where you see the Molotov cocktail example used, consider it a standin for smallpox cultivation / etc, just with the writers not wanting to use those specifics.
i think cultivating smallpox is a much clearer thing to be guarding against, versus molotovs. With molotovs as the example, AI safety seems absurd and overly protective. But when you paint it in the light of smallpox (or other bio-chemical-nuclear realms) there very much needs to be safety.
It’s a poor example. For me, it’s the equivalent of turning off safe search and randomly following top result.
The guardrails are best used to prevent abuse like “generated revenge porn”. Couple of stories out there floating around of people “jail breaking” the LLMs to turn ordinary clothed photos into nude photos. [1,2]
Related: the notion of "safety" (refusing to discuss "harmful" content) is rather shallow and (relatively) easily undone: _Refusal in Language Models Is Mediated by a Single Direction_ [https://arxiv.org/abs//2406.11717]
Bad analogy. That would be more like if Word didn’t let you write erotica.
A better analogy would be if Paint was wired up to a Microsoft cloud service and they didn’t allow you to ask it to create porn. Which is about how things are.
The Register is so irritating to read. I get that the tone is a shtick, but really?
> But AI companies have insisted they’re working to suppress harmful content buried within AI training data so things like recipes for explosives don’t appear.
Why the implication that this is not true? I know it’s house style to imply wrongdoing whenever possible, but damn it gets old to read.
And then! To not even cover the difference between models and prompts. Does this jailbreak only work on models where the safety instructions are purely at the prompt level? Is that why GPT-4 resisted, because it has actual trained weights for safety? Does it even have those?
Who knows? Certainly not the article’s author, who has likely focused on searching and replacing “says” with “claims” or “admits” rather than anything relevant to the subject.
* This is the register, I saw a deeper technical explanation than I was expecting, even if it is too shallow for those an intermediate understanding of these models.
* I think releasing these models with this weakness is a level of wrongdoing, and while they are working on a fix someone could be learning how to harm other people. I’m skeptical all knowledge leaks will be patched in a decade.
But “releasing these models” is the whole problem. Models and prompts are different things. Prompts evolve daily. And any safety at the prompt level is literally non-existent when developers access the model itself without the prompt used in a commercial product. OpenAI playground versus ChetGPT, for example.
The article he’s negative informational value. I wouldn’t mind all of the energy Register puts into snark if there was a real effort to be informative, but it’s irritating to read something where far more energy went into tone than content.
Honestly who care is a llm tells you how to make a bomb when asked a simple Google search will tell you that. Hell I have a antique chemisty text I picked up at goodwill with lab instruction on how to make bunch things LLMs will censor.
All political power comes from the barrel of a gun.
Therefore, those who seek to deny anyone's weapons or the knowledge of how to make and use weapons are seeking to remove their victim's political power.
Those without political power or weapons tend to be enslaved.
Therefore, those who seek to limit AIs from explaining how to use lethal force are attempting to enslave other humans.
It is also a felony under US Law called "Consipiracy Against Rights," but our screwed up government does not prosecute this.
40 comments
[ 1.1 ms ] story [ 91.6 ms ] threadNo real news, here. Just a plug piece that implies Microsoft is safer than their competitors.
Instead of perennially worrying about the risks that the LLMs will say something bad when specifically prompted to do so, I’d rather the companies pay more attention to the problems with accuracy, with outputting copyrighted material (for their own sakes!), and with things like memorizing and regurgitating private information.
They had progressional chemists and Russian military personal indoctrinated into the organization.
Does literally no one understand that the only information that is inside of an llm model is information that has been fed into the model? I think more than anything this fake AI boom has really demonstrated how unintelligent most people are.
Not literally no one, but Sam Altman recently claimed that we'll be able to ask an LLM to "solve all of physics" soon, so plenty of people who should know better do seem to think they're basically magic.
AFAIK the actual research community in PhD land many times have an internal culture with extreme concern for "harm" from a personal point of view -- it's not fake. If you read the first research papers for various LLM groups, you can see the concern is spelled out in detail, at length.
There are high numbers of burnout and psychological issues among practitioners in advanced computer-intensive projects.. so something there seems self-reinforcing, too.
It's an interesting thought, but what if the only companies able to run AI platforms are the ones who are able to shoulder the legal cost of all the liabilities it brings?
Essentially the legal system resources is the moat for smaller companies here.
One woman gets killed by some deranged ex, you can get through that. But now you have notice. You know. So you have to do something to make sure your LLM doesn't get tricked again. Politicians and law enforcement aren't going to sit around and let that sort of thing become commonplace. And when politicians get up and start waving around the "crime wave" banner, things can develop in a direction you don't want them to go.
So I think it's probably wise for these guys to get out in front of the problem of people tricking LLMs into saying things that they are not intended to say. The option if they can't control that, is that the politicians and law enforcement come in and take advantage of the inevitable nightly news cycle to grab more powers and reelections, and that's not good for anyone in our industry.
This assertion that LLMs should obey whatever instructions as long as they are specifically prompted to do so is bizarre.
Should a bank teller LLM divulge another customers balance because you specifically told it to?
This type of exploit is called social engineering when it happens to humans, and we go to great lengths to train the human to detect and avoid it. Why should we not do the same with an LLM?
This is more or less how it works for human customer phone support, who are also limited by narrow permissions. And as we know, they are also skilled at confabulating in the interest of customer happiness.
Where you see the Molotov cocktail example used, consider it a standin for smallpox cultivation / etc, just with the writers not wanting to use those specifics.
The guardrails are best used to prevent abuse like “generated revenge porn”. Couple of stories out there floating around of people “jail breaking” the LLMs to turn ordinary clothed photos into nude photos. [1,2]
[1] https://www.statesman.com/story/news/state/2024/06/25/take-i...
[2] https://www.cnn.com/2024/01/25/tech/taylor-swift-ai-generate...
Ticket asdf-1234: mountains that look like genitals when viewed from a 45 degree angle should not be allowed
edit: found a reference at https://www.reddit.com/r/todayilearned/comments/l8enni/til_t...
A better analogy would be if Paint was wired up to a Microsoft cloud service and they didn’t allow you to ask it to create porn. Which is about how things are.
> But AI companies have insisted they’re working to suppress harmful content buried within AI training data so things like recipes for explosives don’t appear.
Why the implication that this is not true? I know it’s house style to imply wrongdoing whenever possible, but damn it gets old to read.
And then! To not even cover the difference between models and prompts. Does this jailbreak only work on models where the safety instructions are purely at the prompt level? Is that why GPT-4 resisted, because it has actual trained weights for safety? Does it even have those?
Who knows? Certainly not the article’s author, who has likely focused on searching and replacing “says” with “claims” or “admits” rather than anything relevant to the subject.
/rant
* I think releasing these models with this weakness is a level of wrongdoing, and while they are working on a fix someone could be learning how to harm other people. I’m skeptical all knowledge leaks will be patched in a decade.
The article he’s negative informational value. I wouldn’t mind all of the energy Register puts into snark if there was a real effort to be informative, but it’s irritating to read something where far more energy went into tone than content.
https://x.com/elder_plinius/status/1806446304010412227?s=46&...
(You can also use DuckDuckGo if Google somehow decides to censor results.)
Therefore, those who seek to deny anyone's weapons or the knowledge of how to make and use weapons are seeking to remove their victim's political power.
Those without political power or weapons tend to be enslaved.
Therefore, those who seek to limit AIs from explaining how to use lethal force are attempting to enslave other humans.
It is also a felony under US Law called "Consipiracy Against Rights," but our screwed up government does not prosecute this.