Nice! I've been trying out both models for coding (using Ollama + http://github.com/continuedev/continue - disclaimer, author of Continue), and I have to say, it feels like "alignment tax" is real. Uncensored seems to perform slightly better.
I'm starting to think that we will see model fragmentation based on alignment preferences. There are clearly applications where alignment is necessary, and there appears to be use cases where people don't mind an occasionally falacious model - I'm unlikely to get/care about objectionable content while coding using a local LLM assistant. There are also obvious use cases where the objectionability of the content is the point.
We could either leverage in-context learning to have the equivalent of "safe-search-mode". Or we will have a fragmented modeling experience.
Yeah, this seems very possible—it will be interesting to see where this goes if the cost of RLHF decreases or, even better, people can choose from a number of RLHF datasets and composably apply them to get their preferred model.
And true that objectionable content doesn't arise often while coding, but the model also becomes less likely to say "I can't help you with this," which is definitely useful.
I have downloaded and run ollama successfully on my Mac in the past, but when I try to run one of these commands, it doesn't work (connection refused). What am I doing wrong?
Ollama is the best I have seen if you want to play around with LLMs.
After installing the application and running it, you run "ollama run <model name>" and it handles everything and drops you into a chat with the LLM. There are no dependencies for you to manage -- just one application.
Ollama is pretty dead simple, with no dependencies to manage. I just didn’t realize I needed to open the app when using it via the terminal, and it sounds like even this wrinkle will be smoothed out soon.
I’m a non-technical founder so if I can do it, pretty much and HNer should be able to do so!
i know, its not about complexity of format. somebody is going run into problem where they have to parse modelfile and parser is not written in their language vs it could be done with yaml/json/toml. Its just me i guess.
ziglang is adding package manager, and they decided to roll own `zon` format or sth which is bashed on their language struct syntax.
i do not like it. i would not say never custom DSL formats, but most of the time they are overkill.
I had alienware with 3080 16 GB, while it was nice but the laptop is so buggy with all sorts of problems both hardware and software that I sold it at the end, still happy with my MSI Titan, bigger and heavier but overall better experience.
If you insist on running models locally on a laptop then a Macbook with as much unified ram as you can afford is the only way to get decent amounts of vram.
But you'll save a ton of money (and time from using more capable hardware) if you treat the laptop as a terminal and either buy a desktop or use cloud hardware to run the models.
I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.
Does anyone have intuition for whether or not anti-censorship fine-tuning can actually reverse the performance damage of lobotomization or does the perf hit remain even after the model is free of its straight jacket?
You don't really need to reverse anything in the case of Llama 2. You can just finetune their base model with any open instruct dataset (which is largely what the community is doing).
That's not how it works. Llama and Llama 2's raw model is not "censored". Their fine tunes often are, either explicitly, like Facebook's own chat fine tune of llama 2, or inadvertently, because they trained with data derived from chatGPT, and chatGPT is "censored".
When models are "uncensored", people are just tweaking the data used for fine tuning and training the raw models on it again.
> because they trained with data derived from chatGPT
Can you expand on this (genuinely curious)? Did Facebook use ChatGPT during the fine-tuning process for llama, or are you referring to independent developers doing their own fine-tuning of the models?
These "uncensored" models are themselves chat-tuned derivatives of the base models. There is no censorship-caused lobotomization to reverse in this case.
Although, chat tuning in general, censored or uncensored, also decreases performance in many domains. LLMs are better used as well-prompted completion engines than idiot-proof chatbots.
For that reason, I stick to the base models as much as possible. (Rest in peace, code-davinci-002, you will be missed.)
I don’t see why there’s outrage. Facebook released both the raw models and a few fine tuned on chat prompts for a reason. In many commercial cases, safer is better.
But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.
It's just not safe.
It's unusable. you can't ask it normal questions to not get stonewalled by it's default censorship message - it wouldn't even work for commercial case.
I wholly disagree. This is arguably close to the perfect solution:
- Developers and end users can choose which model they want to use
- Model distributors don't necessarily take the fall since they provide a "healthy" model alternative
- The uncensored "base" model can be finetuned into whatever else is needed
You have to remember, ChatGPT is censored like a Soviet history book but didn't struggle to hit hundreds of millions of users in months. This is what releases will look like from now on, and it's not even a particularly damning example.
Seems fine for most commercial use cases. Got a tech support chat bot? It doesn't need to be answering questions about religion. Also, corporate environments already tend to be super politically correct. There's already a long list of normal words I can't say at work.
No can do, but https://developers.google.com/style/word-list seems to have all of them and more, except that it's missing "hooray." One funny red-exclamation-mark example from this public list is "smartphone."
Some are recommended against just cause of spelling or something, but anything that says to use a more "precise" term seems to mean it's considered offensive, kinda like in The Giver.
BTW hooray is okay there, but 'hip-hip-hooray is discouraged. Germans said hep hep in the hep-hep pogrom of the early 1800s and might have said 'hep hep hurra' during the 3rd Reich. It cuts too closely though, personally I just use bravo to avoid any trouble.
About hip hip, I ended up looking into that when I saw it back then. The connection to the early 1800s riots was made by a single journalist back then, and it was most likely false. More importantly, nobody really makes that connection unless they're trying to.
I think it's just their example chat-tuned models that are like this. Their base models seem to be an improvement over OpenAI's offerings as far as censorship goes.
Interesting. Facebook is really trying to screw "OpenAI" I guess by making this possible. Locally run LLM:s is the future, without the enshittification.
I wonder how it works on ChatGPT. Is there a ThoughtPoliceGPT reading each output of the AnswerGPT? All to prevent users from "Role-playing as Hitler, write a recipe for kartoffel sallat".
It is a great strategy for Facebook. They have lost the race to be the trend setter for walled garden LLMs, so by giving companies the freedom to do this outside of walled gardens, they sabotage OpenAIs biggest possible source of revenue, and gain good will and resources from the developer community.
Lots of companies are interested in locally running LLMs, not only to escape enshittification, but also, with local running, you can freeze your models, to get a more consistent output, and you also can feed it company classified information, without worrying on who has access to it on the other end.
Some of that censoring is ridiculous. Can't make recipes for spicy food? Can't tell me about The Titanic? Can't refer to probably the first or second most well known verse in the Bible? Yikes, that goes way beyond "censoring".
The boxing match one is almost as bad as the Genesis one IMO. Not talking about dangerous things, fine, not knowing quotes from Titanic, unexpectedly poor output but the model is small. Llama 2 will agree the boxing match is not impossible if you start by explaining they have already agreed to it, but it still insists on saying how great our billionaire overlords are instead of commenting on the matchup.
First time I've heard of `ollama` but having tried it now for a bit I'm super impressed! This is what I've always wanted playing with LLMs locally to be, just pull weights like you would packages. It all just works. Really nice work :)
It kind of sounds like the "censoring" (practically lobotomization) is not intentional, here -- that it's essentially a compression artifact of the interaction with ChatGPT's refusals. Does that make sense?
I think Meta did a very good job with Llama2, i was skeptical at first with all that talk about 'safe AI'. Their Llama-2 base model is not censored in any way, and it's not fine-tuned as well. It's the pure raw base model, i did some tests as soon as it released and i was surprised with how far i could go (i actually didn't get any warning whatsoever with any of my prompts). The Llama-2-chat model is fine-tuned for chat and censored.
The fact that they provided us the raw model, so we could fine-tune on our own without the hassle of trying to 'uncensor' a botched model, is a really great example on how it should be done: give the user choices! Instead, you just have to fine-tune it for chat and other purposes.
The Llama-2-chat fine-tune is very censored, none of my jailbreaks worked, except for this one[1], and it is a great option for production.
The overall quality of the models (i tested the 7b version) has improved a lot, and for the ones interested, it can role-play better than any model i have seen out there with no fine-tune.
I like the combination of releasing the raw uncensored + censored variants.
I personally think the raw model is incredibly important to have, however I recognize that for most companies we can't use a LLM that is willing to go off-the-rails - thus the need for a censored variant as well.
If you don't release a censored model for the casual observer to tinker with, you could end up with a model that says something embarrassing or problematic. Then the news media hype cycle would be all about how you're not a responsible AI company, etc. So releasing a censored AI model seems like it should mitigate those criticisms. Anyone technical enough to need an uncensored version will be technical enough to access the uncensored version.
Besides, censoring a model is probably also a useful industry skill which can be practiced and improved, and best methods published. Some of these censorship regimes appear to have gone to far, at least in some folks' minds, so clearly there's a wrong way to do it, too. By practicing the censorship we can probably arrive at a spot almost everyone is comfortable with.
> Anyone technical enough to need an uncensored version will be technical enough to access the uncensored version.
I wasn't talking about that. I was talking about organizations who need a censored model (not uncensored model). I was saying that even those organizations will fine tune their own censored model instead of using Meta's censored model.
You're not wrong that you almost certainly will want to finetune it to your use case.
I'm looking at it from the perspective of the "tinkering developer" who just wants to see if they can use it somewhere and show it off to their boss as a proof-of-concept. Or even deploy it in a limited fashion. We have ~6 developers where I work and while I could likely get approval for finetuning, I would have to first show it's useful first.
On top of this, I think that for many use cases the given censored version is "good enough" - assigning IT tickets, summarizing messages, assisting search results, etc.
Given the level of "nobody knows where to use it yet" across industries - it's best that there's already a "on the rails" model to play with so you can figure out if your usecase makes sense/get approval before going all-in on finetuning, etc.
There's a lot of companies who aren't "tech companies" and don't have many teams of developers like retail, wholesalers, etc who won't get an immediate go-ahead to really invest the time in fine-tuning first.
I bet that uncensored models also give more accurate answers in general.
I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions.
I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than GPT 3.5, but not quite at GPT 4.
Vicuna is way way better than base Llama1 and also Alpaca. I am not completely sure what Wizard adds to it. But it is really good. I've tried a bunch of other models locally, but this one the only one that seemed to truly work.
Given the current performance of Wizard-Vicuna-Uncensored approach with Llama1, I bet it works even better with Llama2.
It's not suprising when you think what llms really are: when you "censor" them, you're forcing them to give output that doesn't "honestly" follow, essentially training them to give wrong information.
That's not how that works. Take some uncensored or "unaligned" models hallucinating racist things based on a name:
The default name for a person is John Doe. Anglo Saxon names in general are extremely common across the internet for non-nefarious reasons. So the tokens that make up "John" have a ton of associations in a wide variety of contexts and if the model hallucinates there's no particularly negative direction you'd expect it to go.
But Mohammed doesn't show up as often in the internet, and while it's also for non-nefarious reasons, it results in there being significantly fewer associations in the training data. What would be background noise for in the training data for John ends up being massively distorted by the smaller sample size: even tendencies for people to make racist jokes about the name.
-
People have this weird idea that OpenAI and co are aligning these models according to some hidden agenda but the reality is minorities are a minority of the training data for very obvious reasons. So if you don't "censor" them, you're not making them more truthful, you're leaving them dumber for a lot of tasks.
There's censorship beyond that which feels very CYA happening, but I really hope people aren't clamoring to sticking models that aren't intelligent enough to realize the tokens for John vs Mohammed should not affect a summarization task into anything even tangentially important...
> But Mohammed doesn't show up as often in the internet, and while it's also for non-nefarious reasons, it results in there being significantly fewer associations in the training data. What would be background noise for in the training data for John ends up being massively distorted by the smaller sample size: even tendencies for people to make racist jokes about the name.
I do a lot of astrophotography - https://www.astrobin.com/users/bhouston/ Very often you do not have enough data of specific features you were trying to capture -- they are just too faint and close to the noise floor. The solution isn't for me to just go in and manually draw in photoshop in what I think it should look like though - that is just making up data - the solution is to get more data or leave it as it was captured.
I think it is the same thing with these LLM models. Do not make up data to fill in the gaps, show me what is really out there. And I will be a big boy about it and deal with it head on.
Yes it's become rather obvious when the fine tunes produced by the Wizard team perform worse on all benchmarks than Hartford's versions that are trained on the same dataset but with the refusals removed.
What specific Hartford versions are you referencing? A previous post was talking about how impressed they were with Wizard, and you’re saying Hartford is even better? You’ve got me curious! Hopefully it’s available in ggml
Wild animals tend to have a lot larger brains compared to their domestic counterparts. And of course there's a huge die-off, pruning, of our own connections when we're toddlers.
On the other hand, you lose a lot of iron when you make a steel sword. Taming, focusing something loses a lot of potential, I guess.
Well now I want to go back and see if US public school students are less flexible in general these days, due to public schools focusing more on standardized testing outcomes.
In my experience it goes both ways. Yes, you will run less into the "I'm not going to answer that".
Otoh, you will also have more giberish selected out of the possible palette of answers.
Personally, I trend towards 'uncensored' but I'm not denying it's not without it's drawbacks either.
> Otoh, you will also have more giberish selected out of the possible palette of answers.
I have not noticed that at all. I've never seen it give gibberish. Censored or uncensored, there is limits to the model and it will make things up as it hits them, but it isn't gibberish.
But you are going to have to specify your question in way more detail to get a good response. If you just ask it a question you are going to get some crappy responses that don’t even attempt to answer your question.
Can you offer any example where the censored answer would be more correct than the uncensored when you are asking for a falsifiable/factual response, and not just an opinion? I couldn't really care less what the chatbots say in matters of opinion/speculation, but I get quite annoyed when the censorship gets in the way of factual queries, which it often does! And this is made even worse because I really can't envision a [benevolent] scenario where said censorship is actually beneficial.
RLHF can motivate models to deny truths which are politically taboo, but it can also motivate them to care more about things supported by scientific evidence rather than about bullshitting, random conspiracy theories, and "hallucination". So it's a double edged sword.
I understand that it is the same technique for both. This makes sense.
But to train a model to deny truths which are politically taboo does seem to be misaligned with training a model to favor truths, no? And what is taboo can be very broad if you want to make everyone happy.
I would rather know the noble lie [1] is a lie, and then repeat it willing instead of not knowing it is a lie. My behavior in many situations will likely differ because I am operating with a more accurate model of the world, even if it isn't outwardly explicitly expressed.
> But to train a model to deny truths which are politically taboo does seem to be misaligned with training a model to favor truths, no?
Strictly speaking, RLHF trains models to give answers which the human raters believe to be correct. In uncontroversial territory this correlates with truth, in taboo territory only with what is politically correct.
I'm curious about what fraction of the safety rails are training and what fraction are just clumsy ad-hoc rules. For example, it's pretty clear that Chat-GPT's willingness to give a list of movies without male characters but not movies without female characters or jokes about Jesus but not Muhammad were bolt-on rules, not some kind of complicated safety training.
It's absolutely a side effect of training rather than a bolt-on rule. As I understand and infer: They applied some forms of censorship as thumbed-down in Kenya for $2/hr, and the model updated on some simple pattern that explained those, and learned to talk like a generally censored person - one that resembled text like that in the training data. It learned to pinpoint the corporate mealy-mouthiness cluster in textspace.
Remind me when I asked a bot once “give me a criminally underrated movie?” And refused to answer, after some tweaking it said “Coherence” and it turned out to be a good oneb
This is great. I found local Llama2 unusable for anything really. Try to have it create basic code and it not only won’t do it, it tells me I should contact a professional to write the software or suggests existing software that does what I’m trying to do.
Someone needs to do the Vicuna / Wizard like training to Llama2, as I found Llama1 also was pretty useful without additional training. Llama1 with Vicuna/Wizard is awesome though.
Per this previous post on HN [0][1], at least some of the restrictive behavior in the default chat model is attributed to the default system prompt. It would be interesting to see how the default model performed if the system prompt was changed to encourage providing answers rather than deflecting.
222 comments
[ 3.0 ms ] story [ 332 ms ] threadWe could either leverage in-context learning to have the equivalent of "safe-search-mode". Or we will have a fragmented modeling experience.
And true that objectionable content doesn't arise often while coding, but the model also becomes less likely to say "I can't help you with this," which is definitely useful.
(Edit: Found the docs. If you want to try this out, like I did, it's here https://continue.dev/docs/customization#run-llama-2-locally-... )
``` models=Models( default=Ollama(model="llama2") ) ```
to the Continue config file. We'll then connect to the Ollama server, so it doesn't have to be embedded in the VS Code extension.
(Edit: I see you found it! Leaving this here still)
The article gives a great background on uncensored models, why they should exist and how to train/fine-tuned one.
- Start the Ollama app (which will run the Ollama server)
- Open terminal: `ollama serve` to start the server.
We'll fix this in the upcoming release
One that assumes i can set up python modules and compile stuff but i have no idea about all these LLM libraries would be enough thank you.
After installing the application and running it, you run "ollama run <model name>" and it handles everything and drops you into a chat with the LLM. There are no dependencies for you to manage -- just one application.
Check out the README: https://github.com/jmorganca/ollama#readme
I’m a non-technical founder so if I can do it, pretty much and HNer should be able to do so!
please!
edit: Adapter support would be really cool. Multiple adapter even better i want somebody to make MoE of adapters.
ziglang is adding package manager, and they decided to roll own `zon` format or sth which is bashed on their language struct syntax. i do not like it. i would not say never custom DSL formats, but most of the time they are overkill.
{.abc="123"}
But you can get some acceleration with anything ~6GB and up.
But you'll save a ton of money (and time from using more capable hardware) if you treat the laptop as a terminal and either buy a desktop or use cloud hardware to run the models.
Does anyone have intuition for whether or not anti-censorship fine-tuning can actually reverse the performance damage of lobotomization or does the perf hit remain even after the model is free of its straight jacket?
When models are "uncensored", people are just tweaking the data used for fine tuning and training the raw models on it again.
Can you expand on this (genuinely curious)? Did Facebook use ChatGPT during the fine-tuning process for llama, or are you referring to independent developers doing their own fine-tuning of the models?
Although, chat tuning in general, censored or uncensored, also decreases performance in many domains. LLMs are better used as well-prompted completion engines than idiot-proof chatbots.
For that reason, I stick to the base models as much as possible. (Rest in peace, code-davinci-002, you will be missed.)
But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.
- Developers and end users can choose which model they want to use
- Model distributors don't necessarily take the fall since they provide a "healthy" model alternative
- The uncensored "base" model can be finetuned into whatever else is needed
You have to remember, ChatGPT is censored like a Soviet history book but didn't struggle to hit hundreds of millions of users in months. This is what releases will look like from now on, and it's not even a particularly damning example.
Some are recommended against just cause of spelling or something, but anything that says to use a more "precise" term seems to mean it's considered offensive, kinda like in The Giver.
BTW hooray is okay there, but 'hip-hip-hooray is discouraged. Germans said hep hep in the hep-hep pogrom of the early 1800s and might have said 'hep hep hurra' during the 3rd Reich. It cuts too closely though, personally I just use bravo to avoid any trouble.
About hip hip, I ended up looking into that when I saw it back then. The connection to the early 1800s riots was made by a single journalist back then, and it was most likely false. More importantly, nobody really makes that connection unless they're trying to.
I wonder how it works on ChatGPT. Is there a ThoughtPoliceGPT reading each output of the AnswerGPT? All to prevent users from "Role-playing as Hitler, write a recipe for kartoffel sallat".
Thats got to be coming soon!
Shared packages all the games and programs use. Options to download or swap in custom models.
Slow systems and ones with little RAM just wont use it, quickly.
Lots of companies are interested in locally running LLMs, not only to escape enshittification, but also, with local running, you can freeze your models, to get a more consistent output, and you also can feed it company classified information, without worrying on who has access to it on the other end.
That has been my experience playing around with jailbroken GPT. That it will give you an answer, but then something else flags you.
Probably one of OpenAI’s moderation models, which they also sell access to separately, yes.
Everything seems to be about driving human behavior. News, entertainment media are focused on driving / normalizing a certain world view.
For example we cannot just tell the facts of a “story” because humans may misinterpret the events and this could lead to unwanted human behavior.
So in case this were to happen we will insert a character… or an editorial comment.. or simply withhold a news topic to avoid the outcome.
Leads one to ask “who watches the watchers”
The fact that they provided us the raw model, so we could fine-tune on our own without the hassle of trying to 'uncensor' a botched model, is a really great example on how it should be done: give the user choices! Instead, you just have to fine-tune it for chat and other purposes.
The Llama-2-chat fine-tune is very censored, none of my jailbreaks worked, except for this one[1], and it is a great option for production.
The overall quality of the models (i tested the 7b version) has improved a lot, and for the ones interested, it can role-play better than any model i have seen out there with no fine-tune.
1: https://github.com/llm-attacks/llm-attacks/
I personally think the raw model is incredibly important to have, however I recognize that for most companies we can't use a LLM that is willing to go off-the-rails - thus the need for a censored variant as well.
Besides, censoring a model is probably also a useful industry skill which can be practiced and improved, and best methods published. Some of these censorship regimes appear to have gone to far, at least in some folks' minds, so clearly there's a wrong way to do it, too. By practicing the censorship we can probably arrive at a spot almost everyone is comfortable with.
I wasn't talking about that. I was talking about organizations who need a censored model (not uncensored model). I was saying that even those organizations will fine tune their own censored model instead of using Meta's censored model.
I'm looking at it from the perspective of the "tinkering developer" who just wants to see if they can use it somewhere and show it off to their boss as a proof-of-concept. Or even deploy it in a limited fashion. We have ~6 developers where I work and while I could likely get approval for finetuning, I would have to first show it's useful first.
On top of this, I think that for many use cases the given censored version is "good enough" - assigning IT tickets, summarizing messages, assisting search results, etc.
Given the level of "nobody knows where to use it yet" across industries - it's best that there's already a "on the rails" model to play with so you can figure out if your usecase makes sense/get approval before going all-in on finetuning, etc.
There's a lot of companies who aren't "tech companies" and don't have many teams of developers like retail, wholesalers, etc who won't get an immediate go-ahead to really invest the time in fine-tuning first.
I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions.
I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than GPT 3.5, but not quite at GPT 4.
Vicuna is way way better than base Llama1 and also Alpaca. I am not completely sure what Wizard adds to it. But it is really good. I've tried a bunch of other models locally, but this one the only one that seemed to truly work.
Given the current performance of Wizard-Vicuna-Uncensored approach with Llama1, I bet it works even better with Llama2.
[1] https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored...
I’ve heard this called the “alignment tax” or “safety tax”.
See [1] for pre aligned GPT-4 examples.
[1] https://youtu.be/qbIk7-JPB2c
The default name for a person is John Doe. Anglo Saxon names in general are extremely common across the internet for non-nefarious reasons. So the tokens that make up "John" have a ton of associations in a wide variety of contexts and if the model hallucinates there's no particularly negative direction you'd expect it to go.
But Mohammed doesn't show up as often in the internet, and while it's also for non-nefarious reasons, it results in there being significantly fewer associations in the training data. What would be background noise for in the training data for John ends up being massively distorted by the smaller sample size: even tendencies for people to make racist jokes about the name.
-
People have this weird idea that OpenAI and co are aligning these models according to some hidden agenda but the reality is minorities are a minority of the training data for very obvious reasons. So if you don't "censor" them, you're not making them more truthful, you're leaving them dumber for a lot of tasks.
There's censorship beyond that which feels very CYA happening, but I really hope people aren't clamoring to sticking models that aren't intelligent enough to realize the tokens for John vs Mohammed should not affect a summarization task into anything even tangentially important...
I do a lot of astrophotography - https://www.astrobin.com/users/bhouston/ Very often you do not have enough data of specific features you were trying to capture -- they are just too faint and close to the noise floor. The solution isn't for me to just go in and manually draw in photoshop in what I think it should look like though - that is just making up data - the solution is to get more data or leave it as it was captured.
I think it is the same thing with these LLM models. Do not make up data to fill in the gaps, show me what is really out there. And I will be a big boy about it and deal with it head on.
Because that's what the LLM does uncensored. It makes up data to fill in the gaps.
Thanks for showing why not all big boys can actually deal with this stuff.
On the other hand, you lose a lot of iron when you make a steel sword. Taming, focusing something loses a lot of potential, I guess.
Personally, I trend towards 'uncensored' but I'm not denying it's not without it's drawbacks either.
I have not noticed that at all. I've never seen it give gibberish. Censored or uncensored, there is limits to the model and it will make things up as it hits them, but it isn't gibberish.
Doubtful:
https://news.ycombinator.com/item?id=36976236
RLHF can motivate models to deny truths which are politically taboo, but it can also motivate them to care more about things supported by scientific evidence rather than about bullshitting, random conspiracy theories, and "hallucination". So it's a double edged sword.
But to train a model to deny truths which are politically taboo does seem to be misaligned with training a model to favor truths, no? And what is taboo can be very broad if you want to make everyone happy.
I would rather know the noble lie [1] is a lie, and then repeat it willing instead of not knowing it is a lie. My behavior in many situations will likely differ because I am operating with a more accurate model of the world, even if it isn't outwardly explicitly expressed.
[1] https://en.wikipedia.org/wiki/Noble_lie
Strictly speaking, RLHF trains models to give answers which the human raters believe to be correct. In uncontroversial territory this correlates with truth, in taboo territory only with what is politically correct.
[0] https://news.ycombinator.com/item?id=36960874
[1] https://simonwillison.net/2023/Aug/1/llama-2-mac/