Author here. Quick version: “abliteration” (basically removing the direction in the model that causes it to refuse) is the go-to method people use to make open models uncensored.
Most people treat it like a clean surgical cut - it just kills the refusals and leaves everything else untouched. I tested that assumption on Gemma and Qwen with 21,600 pre-registered decisions under uncertainty, using identical frozen inputs for the base vs. abliterated versions.
Turns out it’s not surgical at all.
The abliterated models systematically become more optimistic, hedge less, show no improvement in actual task performance, and the same edit even moves their expressed confidence in opposite directions depending on the model family.
Preregistration, dataset, and analysis code are all public. Happy to answer any methodology questions or hear where you think this falls apart.
> Quick version: “abliteration” (basically removing the direction in the model that causes it to refuse) is the go-to method people use to make open models uncensored.
Tru-ish (lots of people distinguish between abliteration and uncensoring, though.)
> Most people treat it like a clean surgical cut - it just kills the refusals and leaves everything else untouched.
Basically no one does this, its widely recognized that this isn’t how it works and it has for quite some time been common for makers of anliterated model versions to publish metrics for how far a particular abliteration (1) removes refusals (typical before/after refusal rate on a standard test set), and (2) diverges to the output of the base model (KL divergence), and it is widely understood that there is generally, in practice, a tradeoff between these two metrics, where more refusal reduction tends to come at the expense of higher KL divergence.
That’s not saying that it isn’t interesting and new to characterize the kind of divergence that occurs with abliteration in different model families, but there is no reason for a late-night informercial level of misrepresentation of the existing understanding to come along with that.
Censorship has no place in policy of any frontier company hoping to be competitive. That onus should be on the user, the law, and society. Anything less is not a competitive product, as proving out here.
Treating society with gloves reduces progress and creativity alike.
6 comments
[ 2.3 ms ] story [ 28.6 ms ] threadMost people treat it like a clean surgical cut - it just kills the refusals and leaves everything else untouched. I tested that assumption on Gemma and Qwen with 21,600 pre-registered decisions under uncertainty, using identical frozen inputs for the base vs. abliterated versions.
Turns out it’s not surgical at all.
The abliterated models systematically become more optimistic, hedge less, show no improvement in actual task performance, and the same edit even moves their expressed confidence in opposite directions depending on the model family.
Preregistration, dataset, and analysis code are all public. Happy to answer any methodology questions or hear where you think this falls apart.
Tru-ish (lots of people distinguish between abliteration and uncensoring, though.)
> Most people treat it like a clean surgical cut - it just kills the refusals and leaves everything else untouched.
Basically no one does this, its widely recognized that this isn’t how it works and it has for quite some time been common for makers of anliterated model versions to publish metrics for how far a particular abliteration (1) removes refusals (typical before/after refusal rate on a standard test set), and (2) diverges to the output of the base model (KL divergence), and it is widely understood that there is generally, in practice, a tradeoff between these two metrics, where more refusal reduction tends to come at the expense of higher KL divergence.
That’s not saying that it isn’t interesting and new to characterize the kind of divergence that occurs with abliteration in different model families, but there is no reason for a late-night informercial level of misrepresentation of the existing understanding to come along with that.
Treating society with gloves reduces progress and creativity alike.