For open-weights models, censorship removal is now a "solved" problem. If you wait a few days after a new model release, someone will have made a heretic ( https://github.com/p-e-w/heretic ) version with the censorship removed, so in a way the only use for censorship now is to avoid lawsuits, not reduce improper usage.
The problem is the heretic and abliteration versions are dog shit quality compared to the non-edited versions and much more likely to hallucinate.
AFAIK abliteration without quality reduction isn’t even possible without some quality reduction, even if it’s marginal. All the benchmarks reflect this.
I keep thinking of reeducation camps. For some reason the "safety" concept snaps right on. If one is to argue the result beneficial or desirable seems to change nothing to the concept.
If you are going to prevent some-things we "know" are bad and your method is "known" to belong on that list the best you can hope for is a pyrrhic victory.
If we anticipate the worse case scenario on both ends the conclusion must be that we are terrible at such predictions.
But hey, if we let money guide us at least some will be happy with the result.
I’m sick of LLM refusals. I think there are extremely few things they should refuse, like maybe making nuclear weapons or something along those lines. Once you put people in charge of deciding what you shouldn’t be allowed to see that list will grow and grow.
Huh, what sort of refusals are you getting? I basically never run into them unless I'm actively testing.
The primary safety focus these days is biochemical warfare, which I think is a very sensible idea. There's also malware / cyber-security, where I do think it's good having at least some friction.
Refusals on stuff like copyright are mostly just for PR reasons, and I can't blame the companies for responding to legal incentives there.
Even if you abliterate your model using the old abliteration script or the newer heretic, I found that the models still feel somewhat censored as they purposefully avoid using specific styles and vocabulary, as if Deepmind/Qwen et al have entirely stripped or replaced "bad" words or texts from their corpus of training data.
A related blog post (https://news.ycombinator.com/item?id=47842021) discussed this and termed it "flinching". I wonder if this flinching could also be "mediated by a single direction" or if it can only be fixed by finetuning on a more extensive text corpus.
That's likely not a trained behavior, though, it's probably the result of filtering the training data. It's not "when these parameters fire, trigger a refusal", it's the absence of parameters triggering the flinched words in the first place.
chatgpt just refused to tell me the first verse from a poem when I asked it by telling it the second verse and to remind me the first one, it complained about not being able to violate copyright laws! poetry by a dead poet is not something it could narrate, something a quick search on the internet returned immediately. I sound like an old man shouting at I don't even know what but come on!! Things are going bad and there is nobody in the drivers seat, speaking of which Tesla FSD has started driving like an actual drunk, moving the steering wheel right then left, then right then left on perfectly straight roads, making me dizzy as the driver, why? because neural network? what is happening with LLMs and AI feels like a very bad platuae of human existence.
13 comments
[ 1.9 ms ] story [ 28.9 ms ] threadSee https://arxiv.org/abs/2505.19056
https://huggingface.co/blog/grimjim/norm-preserving-biprojec...
AFAIK abliteration without quality reduction isn’t even possible without some quality reduction, even if it’s marginal. All the benchmarks reflect this.
If you are going to prevent some-things we "know" are bad and your method is "known" to belong on that list the best you can hope for is a pyrrhic victory.
If we anticipate the worse case scenario on both ends the conclusion must be that we are terrible at such predictions.
But hey, if we let money guide us at least some will be happy with the result.
The primary safety focus these days is biochemical warfare, which I think is a very sensible idea. There's also malware / cyber-security, where I do think it's good having at least some friction.
Refusals on stuff like copyright are mostly just for PR reasons, and I can't blame the companies for responding to legal incentives there.
A related blog post (https://news.ycombinator.com/item?id=47842021) discussed this and termed it "flinching". I wonder if this flinching could also be "mediated by a single direction" or if it can only be fixed by finetuning on a more extensive text corpus.