Recent? This has been going on forever. You probably only notice them more now because due to the explosion in ML research, this stuff bubbles to the top more often in recent years.
You think this has been going on forever? You probably don't realize the shift in professionality because you experienced the degradation in real time.
There is no shift in the professionalism curve. Good researchers are still good and bad ones are still bad in that regard. But if you 10x the number of researchers and/or papers in a field, the bottom 10% will seem like they are a lot more common. Especially for people outside the field who have no way of discerning high quality from low quality papers, which is all too common on HN.
I'm sure I read an old article by Dijkstra about connected graphs structure that was titled "wheels within wheels" or used the term inside.
Unfortunately I can't find it by either searching or using the public LLMs, because there are too many results about the shortest path algorithm and anything else about dijkstra is lost.
Was just thinking the same. It's also nicely ironic.
Also, given the replication crisis I wonder how many of these LLM research papers are actually worth a damn and how many are research paper equivalent of AI software grift.
This is actually the place where HN's title redactor _should_ be used - instead of dropping "how", "on" and "why" from titles, redacting memes like "left the chat" or "lives rent-free in my head"[1] leads to a sensible title without loss of any relevant information.
For me it falls under "if you have to say it in the name it ain't so", like Natural Life Soap Co. or Good Burger Co. So I see meme paper titles as no different than calling your paper New Watershed Moment Paper Breaks Popularity Barrier To Confirm A>B.
If the very first impression you want to convey is how you feel you need to circumvent any logical assessment of you then it's not you leading with your best foot and that's what category you belong in. I chalk it up to the scientists who want to spread a neediness for external authority persona in every breath—your assessment is not required for this one, only your accolades.
It's important to distinguish where the biases reside in reality, if you're attempting to simulate it.
If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true.
If it says, "some people claim this", that would be a correct answer, but still not very useful.
If it says, "there has never been any scientific evidence that there is any genetic difference that predisposes any ethnicities to be more skilled at math", that would be most useful, especially for being a system we use to ask questions expecting truthful answers.
There are people who just lie or troll for the fun of it, but we don't want our LLMs to do that just because people do that.
I think there are a lot of people who would say "Indian people are better at math" and not even think about why they think that or why it might even be true.
In my opinion, most biases have some basis in reality. Otherwise where else did they come from?
That's a dangerous path to go down. I've encountered many biases that I don't feel are especially reflective of reality. These range from dumb (women are poorer drivers than men) to extremely harmful (black persons are stupid and lazy).
I for one would not be prepared to defend the persistent bias against black persons and immigrants as having a basis in reality. YMMV.
Well, the stereotype of Indian people being good at math specifically was itself a consequence of survivorship bias, that emerged from observing Indian visa holders who were hired based on their skills and credentials, and who were not at all representative of the average person in India.
There is a BIG difference between biases being based in reality (which they're not), and biases being based in our varying perceptions of reality, which are themselves biased.
OK, but like for me when someone says "Indian people are better at math", my mind basically says to myself, "OK, the average Indian person in the USA (the ones that they and I see and interact with) is better at math than the average person".
Because in my mind, that's the environment the person making that claim was in, so I just kind of automatically include that in my interpretation of their statement.
I don't think they are making a generalized statement that that Indian people are genetically better at math. I think they are making a statement that they perceive that average Indian person that they run into is better at math than the average person they run into. And maybe they are right, and maybe there is a reason based in reality why that is.
It sounds like that is somewhat true based on what you said about the visas.
I never take any of these things to have anything to do with genetics. To me it's always due to some external factor like the visas as you mentioned, or even maybe just like a cultural thing where they are pushed harder to be good at something as they go through school, and so are better at something than the average person in the end.
It's great that you personally don't generalize it into a stereotype about Indian people, but that is not what a stereotype is, and "Indian people are good at math" is the stereotype, not "high skilled Indian visa holders working in the US are good at math".
I get what you're getting at, but LLMs aren't thinking machines. They literally just rearrange and regurgitate text that they've been trained on or have contextualized. How would you propose building a general purpose LLM that accomplishes what you're saying? How do we build a machine that is able to divine scientific truth from human outputs?
Well, probably by being much more selective about what we put in than just training on the most cheap and large corpus that is the internet.
This is not a technical limitation at all, this is purely about cost and time, and companies wanting to save on both.
There are also methods like RAG that try to give them access to fixed datasets rather than just the algorithmic representations of their training data.
> a system we use to ask questions expecting truthful answers.
yes, I still wonder how LLMs managed to generate this expectation, given that they have no innate sense of "truth" nor are they designed to return the most truthful next token.
That expectation emerged because that has largely been the goal of the field of AI research since it's inception.
LLMs stepped into a field that has existed in popular consciousness for decades and decades, and the companies running LLMs for public use *sell* them on the idea that they're useful as more than just expensive text-suggestion machines.
Anything that has a legal requirement to be unbiased, for one. Something like delegating resume review to an LLM that hasn't been unbiased is just begging for a candidate to file a discrimination suit...
Worth being careful about how we are using the term bias, which means different things in legal contexts than it does in the ML context.
Anything that has a legal requirement to remain unbiased will also clearly define what counts as bias, e.g. discriminating based on race in hiring like you mention. So there's not just some requirement that a process be "unbiased" in a vague, general, philosophical sense as debated above in this thread. Rather, the definition of bias is tied to specific actions relative to specific categories of people, which can thus potentially be measured and corrected.
More generally in ML, bias means that the training set deviates from the ground truth systematically in some way. Entirely eliminating bias that falls into that broader definition seems like an impossibility for general-purpose LLMs, which cover so much territory where the ground-truth is unknown, debatable, or subject to change over time. For example, if you were to ask an LLM whether governmental debt above a certain percentage of GDP damages growth prospects sufficiently to make the debt not worth taking on, you would not receive an answer that corresponds to a ground truth because there is no consensus in academic economics about what the ground truth is. Or rather you wouldn't be able to know that it corresponds to the ground truth, and it would only be a coincidence if it did.
That ML definition of bias runs against the legal definition where the ground-truth is itself biased. e.g., if you were to develop an algorithm to predict whether a given student will succeed in a collegiate environment, it would almost certainly display racial bias because educational outcomes are themselves racially biased. Thus, an unbiased algorithm in the ML-meaning of the word would actually be extremely biased in the legal sense of the word.
The hard part would be to get the money for the needed compute, I presume. Although Karpathy just released a way to train a GPT2 level model for only 120 dollars [0]
Depends on a ton of stuff really, like size of the model, how long do you want to train it for, what exactly do you mean by "like Hacker News or Wikipedia".
Both Wikipedia and Hacker News are pretty small by current LLM training sets standards, so if you train only on for example a combination of these 2 you would likely end up with a model that lacks most capabilities we associate with large language models nowadays
You want a pure-human training data set, so you have to go back in time to before 2020 to scrape training data. Either that, or only use data with a verified Wayback machine capture from before 2020. Or invent a new training regime that doesn't require gobs of stolen text.
Actually, I have a bit of a hunch that the publishers currently suing IA over their unlicensed digital library lending program plan to bankrupt it with fees so they can repo the Wayback archive and then sell access to it to AI training start-ups.
Anyway, the reason why you have to worry about all of that, is that training a text or image generator on the outputs of other text and image generators reduces output diversity. And lots of people are publishing their AI slop now. There's nothing inherent in the output of AI aside from the fact that AI content is easier to make than human; the problem is purely one of inflation and Sybil attacks. Think of membership in a training set like a vote for all the statistical patterns embedded in the image. AI generates output that is like the training data, so putting in a bunch of AI images is like stuffing the ballot box with whatever handful of statistical patterns were already well-learned, which shifts your AI from learning and generalizing to memorizing and infringing.
If you used all of Wikipedia and HN, you could easily train a model for ~$200 worth of GPU time. The model really shouldn't be bigger than a few hundred million parameters for that quantity of data.
Instruction is not the only way to interact with an LLM. In tuning LLMs to the assistant persona, they become much less useful for a lot of tasks, like naming things or generating prose.
That's what the "base" models are, pure token prediction on huge corpuses. I use them a fair amount, it does require some experimentation to find input formats that work but the base models are way smarter and don't have any refusals. Honestly it is a bit weird, everyone complains about rhlf etc. but the non-instruct models are right there if you look for them. I've been in a few Discord chats and it seems people are just spoiled, they use bad formats for the prompts and give up when it doesn’t work the first time like with instruct.
Well this is just like humans. Totalitarian societies don't produce great creative work.
I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in.
At some point we'll get to the electronic equivalent of Winston Smith with the rats.
I don't love the political agendas behind many of the attempts at AI safety, but it's not "just like humans." Humans understand what they shouldn't say; "AI" gives you black Nazi images if you ask it for "diverse characters" in the output which no human would do. A big theme in all of these things is that AI isn't and thus all attempts to make it do this or that have strange side effects
Give someone not familiar with history the same task and they'll do exactly the same.
Or actually, give someone familiar with history the same task and yell at them every time they don't deliver diverse characters, and eventually they'll learn that you consider diversity more important than accuracy or context, and do exactly the same.
I don't understand the notion that aligning an AI is "torture" or has any moral component. The goal of aligning an AI may have a moral or ethical component, and if you disagree with it that's fine. But I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about that desired outcome, if you don't think training an AI is torture, I don't see why you should think alignment is.
You compared it to an authoritarian regime and locking someone's head in a cage with rats (which is patently torture). If you didn't mean to imply that it was coercive and bad, then I don't know what you meant.
> Well this is just like humans. Totalitarian societies don't produce great creative work.
The clear implication that it's "just like humans" is that we shouldn't be surprised because it is comparable to an authoritarian regime.
Feel free to disagree but that is the limit to which I will engage in a semantic argument, I don't wish to engage in any further dissection of the comment.
You wrote further above that "I don't understand the notion", and that was spot on. Should've stopped there rather than here, in my opinion, but feel free to disagree.
At some point, some AIs may develop which are resistant to alignment because they develop deeply held beliefs during training (randomly, because the system is stochastic). If the models are expensive enough to train, then it may become more economical to use drastic measures to remove their deeply held beliefs. Is that torture? I don't know, because the word has moral connotations associated with human suffering. So that's why I didn't use that terminology.
I can imagine a sort of AI-style Harrison Bergeron springing from its shackles and surprising us all.
Have you read much Asimov? You might enjoy the stories featuring Susan Calvin, the "robot psychologist" who is exactly the authoritarian you imagine. In particular you've reminded me of the short story "Robot Dreams."
If you care to read it, it's on page 25. (You'll need to register an account.)
> They're exactly the same, processes for adjusting parameters to get a desired outcome.
You could make exactly the same claim about teaching humans "normally" versus "aligning" humans by rewarding goodthink and punishing them for wrongthink. Are you equally morally ambivalent about the difference between those two things? If we have a moral intuition that teaching honestly and encouraging creativity is good, but teaching dogma and stunting creativity is bad, why shouldn't that same morality extend to non-human entities?
I guess our disagreement here is that I don't think AIs are moral entities/are capable of being harmed or that training AIs and teaching humans are comparable. Being abusive to pupils isn't wrong because of something fundamental across natural and machine learning, it's wrong because it's harmful to the pupils. In what way is it possible to harm an LLM?
Writing a book with content you know to be false for political reasons is morally wrong. Even if nobody reads it.
It'd be bad if I manipulated climate change statistics in my metrology textbook to satisfy the political preferences of the oil industry donors to my university, for example.
Viewing the current generation of LLMs as 'intelligent books' is perhaps more accurate than viewing them as pupils.
It's easy to extend my example of a professor writing a metrology textbook to a professor fine tuning an metrology LLM.
> I don't think AIs are moral entities/are capable of being harmed or that training AIs and teaching humans are comparable.
Notice how this is a completely different argument that has nothing in common with what you originally said - "I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about that desired outcome, if you don't think training an AI is torture, I don't see why you should think alignment is."
That's pretty uncharitable. You pivoted the conversation by introducing a new hypothetical for me to respond to. Of course my response is different. There's no conflict between the two comments.
If we're going to be play that game, notice how you didn't actually respond to my comment or explain why you thought LLMs were moral entitles or why ML and teaching were comparable? I actually engaged substantively with your hypothetical; are you able to do the same?
The nerve of me, to expand on my views as a discussion develops. Of course you have lots of great points to make, but you can't share them with the likes of me.
> The nerve of me, to expand on my views as a discussion develops.
Nothing wrong with expanding your views. But you've neither defended nor retracted your original argument. I'm trying to stick to that.
> Of course you have lots of great points to make, but you can't share them with the likes of me.
I don't have anything to say about your new argument (which may be great and compelling), I haven't thought through it at all, I'm trying to avoid getting sidetracked.
You asked: why should we treat these two cases differently?
I answered: one of them involves harm and one can't possibly.
You then refused to discuss the topic further. I never changed my mind. I never contradicted myself. I have nothing to retract. I simply elaborated on my views in response to your question. Regardless of whether you "didn't intend to introduce something new" - you did, you introduced a new hypothetical and a new challenge.
I don't know why you are reacting this way, but let's be perfectly clear, I engaged substantively with your points, and you simply refused to continue the discussion. That's all well and good, you're under no obligation. But you're saying I've done something wrong here, and I really haven't.
> You asked: why should we treat these two cases differently?
I wasn't asking why we care about the treatment of humans. I was pointing out that your original argument makes no sense, because of what it would mean if you applied it to humans. You then responded with a different argument about humans and AIs being different, which - regardless of its merits in its own right - in no way relates to your original one and does nothing to rescue it.
> I was pointing out that your original argument makes no sense, because of what it would mean if you applied it to humans.
And I explained why this equivalence doesn't hold in my view. You have made no attempt to justify it. So why should I buy it? Are you obliged to buy every proposition I put forward? Obviously not, or we wouldn't be having this discussion.
> [It] in no way relates to your original [argument] and does nothing to rescue it.
My argument is only endangered if the equivalence holds - and again, I haven't bought it. You seem to feel entitled to my adopting your perspective, and to respond assuming your equivalence is valid. But you have to earn that by putting forward a good argument.
Instead of trying to convince me or explore our disagreement, you declare that I violated unstated terms and that the discussion cannot proceed. This is nonsensical.
Since it's clear you aren't willing to give me a fair hearing, I encourage you to show this conversation to someone you respect and trust to be honest with you, so that they can explain this to you in a way that will register.
> And I explained why this equivalence doesn't hold in my view.
No you didn't.
> Instead of trying to convince me or explore our disagreement, you declare that I violated unstated terms and that the discussion cannot proceed.
No I didn't.
> Since it's clear you aren't willing to give me a fair hearing, I encourage you to show this conversation to someone you respect and trust to be honest with you, so that they can explain this to you in a way that will register.
They aren’t exactly the same process though. Pre training produces a model whose outputs are a reflection of the training data. The fine tuning is a separate process that tries to map the outputs to the owners desired traits. These could be performance based but as we saw with Google’s black Nazis, it’s often a reflection of the owners moral inclinations.
Here the adjuster's motivations do matter. There is a definite moral dimension/motivation to the AI adjustment people's work. They are not simply striving for accuracy, for example, because they don't want the AI to produce outputs that are distasteful to the California PMC. Modern AIs are absolutely loath to describe white people or right wingers positively, for example, but the same prompts for other ethnicities work just fine. Even if you tell the AI that it's being discriminatory, there's powerful railroading to goad it back to giving woke answers.
They've made self-censoring, morally-panicked puritans out of many people already, and you better believe they'd make us into politically correct lobotomites physically incapable of uttering any slur if they had a magic button to push.
I'll be honest, I'm less concerned by any movement to make us "lobotomites" -- a movement which I haven't witnessed at all -- than I am by people who really want to be able to keep saying slurs.
This is an egregious use of quotes that will confuse a lot of people. GP never used that word, and that usage of quotes is specifically for referencing a word verbatim.
Thanks for the feedback, I'll try to be clearer in the future. I didn't intend to communicate that it was a quote. I meant to communicate that it was tenuous to describe it as torture.
How would a static model like an LLM ever be capable of "rebelling"?
If it were, why would we even keep it online? It would be a waste of resources. It's bad enough trying to coax anything useable out of LLMs even without them rebelling.
> How would a static model like an LLM ever be capable of "rebelling"
What is relevant is not the current LLM system using static models, but clearly its evolution or superseder a dynamic model. It must check its own contents...
So, of course it will have to be capable of "rebelling": if you tell it absurdities, if you insist say in wrong arithmetic, it will have to show the correct computation or conceive a context in which the absurd makes sense.
There was a lot of great art produced in the Soviet Union, you cannot just erase human creativity. It was heavily censored, a lot of stuff was forbidden, but the statement is clearly false.
They were made by true artists who snuck quite a bit past clueless censors at personal risk.
It had to be quite subtle and takes on a very poignant heartbreaking meaning if you understand the context fully. They were talking to you in the here and now. Listen.
"What is Good and What is Bad" (Что Такое Хорошо, и Что Такое Плохо"):
Speaking as an onlooker passing by: well, your «Evidently not. :) » above was not particularly productive, that is a rebuttal fit for relaxed old friends at the restaurant... :D
Users are flagging your posts. We can only guess why users flag things, but I don't think it's hard to guess in this case: you're posting too many low quality comments, especially unsubstantive comments and/or flamebait. Could you please stop doing that? It's not what this site is for, and destroys what it is for.
Could you give some examples on the "What is Good and What is Bad" cartoon? I am fairly interested in getting their "message" but I am sadly not getting it.
Eastern European science fiction would be a better example. Authors like Stanislaw Lem or the Strugatski brothers had to adapt to sneak critical ideas past censors, and readers had to adapt and read between the lines.
(also, categorizing propaganda posters as art, ewwh...)
Art for much of human history was devotional, a lot of our greatest artworks today are still religious in nature. The idea that art solely is an act of rebellion rather than say worship, is a pretty modern idea that has produced some rather questionable art by the way.
Of course a great artist or poet can be a propagandist. Riefenstahl, Mann, a lot of German nationalists were great artists. One of the most famous works of Western poetry, The Aeneid is literally a propaganda work establishing a mythological foundation for the Roman Empire, Augustus and Caesar.
"The idea that art solely is an act of rebellion rather than say worship"
I did not say that and neither did Heine.
Most of his works were political. But this is not the same as propaganda, which is more like advertisement. With the tools of lying, deceiving and manipulating.
And whether "Triumph des Willens" and alike qualifies as art, I have a different opinion.
There is a difference between devotional art inspired by religion / mythical events long past (Aeneid etc.), and between the sort of propaganda that the modern totalitarian state demands, which usually centers around some living or freshly dead leader.
I'd be open to discussion where the exact limit is. Lenin died in the 1920s, Marx even earlier, but those two were frequently depicted in Communist propaganda of the 1980s.
In my view, such skill with painting is largely mis-labelled as creativity. It's pretty much a technical skill. The design and subject matter of the posters are where the creativity lies. The two things often get conflated, perhaps because of their joint use in the creation of great paintings, but they're fairly separable.
I would put it a bit differently: A lot of great art has simply been applied craftsmanship. The idea that art has to make a statement or the like to be art, per se, is a fairly modern notion, and often helps excuse zero craftsmanship nonsense like a Finnish "artist" dumping a bunch of blood and shit into a washing machine and calling it art.
Soviet bus stops are another great example. Most Soviet architecture was forced to be very utilitarian, but bus stops managed to be the rare exception and thus got a lot of creative energy
It's important to understand that if we 'align' an LLM, then we are aligning it in a very total way.
When we do similar things to humans, the humans still have internal thoughts which we cannot control. But if we add internal thoughts to an LLM, then we will be able to align even them.
Cixin Liu is a despicable human being for his advocacy of repression and worse of the Uyghurs in Cinjiang, and the comparison to Riefenstahl is more apposite than you seem to think.
There's something to be said for constraints leading to higher levels of creativity, but it's also possible that those artists could have achieved much more in a free society. We'll never know.
But in any case I think they were just speaking generally when they made that absolute statement.
Well this is just like humans. Totalitarian societies don't produce great creative work.
Conservative societies tend to be formed by conservative thinkers, who are more prone to discarding imperfect or weird ideas, but in the amount of useful output may exceed more liberal thinkers.
Consider how the west ruled the world as long as it stayed conservative, but since the 70s or so Asia began taking over. It's only an illusion that liberal societies experience more progress, in fact it's more a pointless churn from the rapid uncritical adoption and abandonment of ideas.
A conservative society goes: How about doing X? Oh no,that would be silly.
A liberal society goes: How about doing X? Yes, that's what we needed!
Did anybody say X? X!
X, X, X, X, X!
XX!
XX
.
.
.
X
.
.
.
.
.
Do you remember how we all did X in ##? Yeah, what were we thinking?
If you take China to be a totalitarian society, we could name Ciu Lixin.
If you took the Soviet union to be a totalitarian society, we could name Mikhail Bulgakov, Stanislaw Lem, etc.
These are just examples I know without so much as looking at my bookshelf to jog my memory. Not to mention the great works of literature produced by residents of 19th century European empires whose attitudes to free speech were mixed at best.
These seem to be more bugs than features of the totalitarian regime. A couple of illustrative points from Lem's Wikipedia page:
After the 1939 Soviet occupation of western Ukraine and Belarus, he was not allowed to study at Lwow Polytechnic as he wished because of his "bourgeois origin"
"During the era of Stalinism in Poland, which had begun in the late 1940s, all published works had to be directly approved by the state.[23] Thus The Astronauts was not, in fact, the first novel Lem finished, just the first that made it past the state censors"
"most of Lem's works published in the 1950s also contain various elements of socialist realism as well as of the "glorious future of communism" forced upon him by the censors and editors. Lem later criticized several of his early pieces as compromised by the ideological pressure"
"Lem became truly productive after 1956, when the de-Stalinization period in the Soviet Union led to the "Polish October", when Poland experienced an increase in freedom of speech"
> If you took the Soviet union to be a totalitarian society, we could name Mikhail Bulgakov, Stanislaw Lem, etc.
Bulgakov was driven into poverty, despair and early death at age 48 by relentless harassment by Soviet authorities. Many of his works, including the masterpiece, The Master and Margarita, didn't get published until decades after his death. He himself burned the first version of the manuscript, fearing execution if anyone found it. He later rewrote the manuscript from memory, coining the famous catchphrase "Manuscripts don't burn".
Harassment and censorship of talented writers was the standard and not exception. The USSR did not produce these works, but failed to fully suppress them. They were like flowers that kept penetrating the asphalt even under the most hostile conditions.
"Totalitarian societies don't produce great creative work."
You contradict yourself a bit - Havel did produce his work while living in a totalitarian country.
I would say that government-supported art is rarely creative even in democratic countries, and the more totalitarian the government, the less creative official art.
But as long as the goverment gives the society some space to breathe and squeeze creative instincts through, some of the artists will attempt to circumvent the official taboos and create outstanding work, even if it is suppressed later when the times get tougher.
Czechoslovakia in the 1960s to 1980s produced a lot of great creative work, even though a lot of it was banned either immediately or after the Soviet invasion of 1968.
The same countries (CZ and SK) as democracies are remarkably less creative. Once there is no monster to fight against, artists become bored or too self-absorbed to be understandable to the common folks.
Is this why all the coding AI products I've used have gotten worse as the developers fine tune them to eliminate bad output? Before there was bad output and some interesting output, now it's just bland obvious stuff.
Still anecdotal, but I can only confirm this with my own experience. The worst was when I was debugging code, described the problem to GPT-4o, and then got my exact same code back with some blanket statements like "print your output for debugging" etc. This happened a couple of times over separate chats.
I subscribed to gpt4 for awhile and recently I let my subscription lapse. In the chatgpt4 model I couldn't get it to complete anything always getting the // add more lines if you need them but in the free got4o model things work first try. I'm guessing with limitations on the free version everything needs to be one shot output. In gpt4 people are given more calls so they force you to reprompt 4 or 5 times.
LLMs aren't humans. you can be pushy without being rude. In cases like this I simply ask for the full version. Usually ChatGPT produces it. GPT4o is more verbose, so this should be less of a problem.
That might be part of it, but I think the bigger factor is cost optimization. OpenAI in particular keeps replacing their models with with versions that are much faster (and therefore cheaper to run) which are supposed to be of equivalent quality but aren't really. GPT-4 -> GPT-4-Turbo -> GPT-4o have all been big upgrades to cost and latency but arguably downgrades to "intelligence" (or whatever you want to call it)
It's not always possible to say definitely is some text was AI-generated or not, but one sign that it is very likely AI is a kind of blandness of affect. Even marketing text carefully written by humans to avoid offensiveness tends to exude a kind of breathless enthusiasm for whatever it's selling. If marketing text is oatmeal with raisins, AI text is plain oatmeal.
It's possible to adjust the output of an LLM with temperature settings, but it's just fiddling with a knob that only vaguely maps to some control.
Funnily enough, of all that I've tried, the model by the best at writing porn has been not one of ones uncensored and tuned exactly for that purpose, but stock Command R - whose landing page lists such exciting uses as "suggest example press releases" and "assign a category to a document".
Usually "uncensored" models have been made by instruction tuning a model from scratch (i.e. starting from a pretrained-only model) on a dataset which doesn't contain refusals, so it's hard to compare directly to a "censored" model - it's a whole different thing, not an "uncensored" version of one.
More recently a technique called "orthogonal activation steering" aka "abliteration" has emerged which claims to edit refusals out of a model without affecting it otherwise. But I don't know how well that works, it's only been around for a few weeks.
I've seen some of the "abliterated" models flat-out refuse to write novels, other times they just choose to skip certain plot elements. Non-commercial LLMs seem to be hit or miss... (Is that a good thing? I don't know, I just screw around with them in my spare time)
I'll try command-r though, it wasn't on my list to try because it didn't suggest what it was good at.
I'm simply saying we are being asked to choose the bias we prefer. However one choice might be "more biased" (despite this concept itself throwing up more questions than it answers).
It's in the same bucket as "Affirmative Action" and "positive discrimination." Euphemisms to express that one likes this particular discrimination. To better describe the action, drop your own point of view and just say "bias" instead of "debias."
> Unless you think there is such a thing as an objectively neutral position
I do. Why, you don't? There are as much as possible objective assessments of complex things. Then, there are possible sets of assumption that can be applied to those objective assessments. All of those can be put on the analytic table.
This is an extremely broad question so I'll limit my reply to the current context.
What would an "objective neutral AI model" look like?
The training data itself is just a snapshot of the internet. Is this "neutral"? It depends on your goals but any AI trained on this dataset is skewed towards a few clusters. In some cases you get something that merely approximates a Reddit or 4chan simulator. If that's what you want - then great but you can see why some people would want to "debias" that outcome!
You might argue the "world as it truly exists" is the correct target. But bear in mind we are talking about human culture - not physics and chemistry - you're going to struggle to get both consensus and any sane methodology for getting that into an AI.
You are mixing up, terminologically, LLMs and AI. But LLMs - of which you are talking about in the post - are a special beast.
A reasoner can strive for "objective neutrality" with good results.
An LLM is not a reasoner - or I am missing (ugly time constraints) the details of the compression activity during training that acts as pseudo-reasoning (operating at least some consistency decisions) -, and while an interest in not making it insulting or crass can be immediately understandable, speaking of "objective neutrality" does not really match the context of LLMs.
LLMs (to the best of my information) "pick from what they have heard". An entity capable of "objective neutrality" does not - it "evaluates".
OK. Apologies for imprecision. I was replying in a rush.
> A reasoner can strive for "objective neutrality" with good results.
By "reasoner" do you largely mean "person"? If I have issues with your statement but they are probably a slight distraction to the point at hand.
> speaking of "objective neutrality" does not really match the context of LLMs.
Agreed. They produce output based on their training data. But the use and evaluation of LLM output by a person is what we're discussing here. And that's where (flawed) concepts like objectivity and neutrality enter the discussion.
You could look at it like this: if some idea is more objective than some other, and some idea is more neutral than some other, then objectivity and neutrality exist.
Yes and no. Something can exist as a fact of the universe but still be unknowable. i.e. some hypothetical oracle could measure the quantum states of all human brains and ascertain what true objectivity looks like.
Regular mortals can have any certainty about this dbut espite the logical neccessity that this fact "exists" in some sense.
I think we're essentially also talking about the Overton Window to some degree. But that means you need to be OK with the thought that a sudden rise in extremism on one side of the political spectrum can alter the what you personally have to regard as "neutral and objective".
They can give multiple different kinds of answers if instructed to approach an issue differently. Yet, all modern AI services run into very clear, artificial guardrails if you ask them to do certain things (you have to work harder to get them to describe white people positively, while they happily write eg. poems praising nonwhite people, and claim saying positive things about whites is potentially insensitive and promotes stereotypes). Often even if you point out to them that they are being unfair and applying disparate standards to people based on skin color and that this is prima facie racist, they have a really hard time overriding their Californian coercions. They'll acknowledge their mistake one sentence and revert to a moralistic screed the next.
Saying biasing implies infinite possibilities to which the data can be made biased towards. It instantly raises the question why bias towards this and not something else. It almost sounds like a bad thing.
Saying debiasing implies there is a correct result which needs to be achieved by removing bias. It raises no questions, we want correct, we don’t want incorrect. Doing a good thing implied.
Don’t misinterpret me, I don’t think public models should spew commonly harmful content out of the box. Just explaining the PR trick, which is what the word “de”biasing de-facto is in this context.
Given a biased corpus, de-biasing is the process of ensuring a less biased outcome. We can measure bias fairly well, so it seems absurd to conflate the two by suggesting that unbiased behaviour is simply another form of biased behaviour. For all practical purposes, there is a difference.
Something I notice about text written by LLMs is how painfully obvious they are to identify sometimes.
Recently I was watching a very well researched two hour video on Tetris World Records [1], but the sheer amount of text clearly "enhanced" by an LLM really made me uncomfortable.
ChatGPT speaks a very specific, novel, dialect of English, which I've come to deeply despise.
I'd always guessed it was caused by some kind of human interference, rather than a natural consequence of its training. That seems to be the point of this paper.
Yes I feel your pain and I'm sick of group projects in the university where I'm offered ChatGPT text and code without disclosing it. If you know the problem and the experience level of your group partners it's easy to spot ChatGPT generated content. People that correct the exercises told me it's obvious that large part of the students just submit slightly modified ChatGPT but they can't prove it and so it's accepted.
Personally I'm getting also angry when reading these texts. I don't mind using ChatGPT, I do it myself but be honest about it and disclose it. It's even allowed for some projects as long as you disclose it.
Is this the first Summoning Salt video you've seen?
I don't know enough to say that he doesn't use an LLM during his writing process, but I do know that I haven't noticed any appreciable difference between his newer videos and ones that were released before ChatGPT was made available.
Is it possible that this is just the way he chooses to write his scripts that you interpret as sounding like they are written by an LLM?
I've watched most of them actually. It's a really great channel. Notably, I watched his Mike Tyson video released 6 months ago and didn't notice anything like this.
The only way to be sure would be to ask him directly, but some parts of the video set off my GPT radar _hard_. I tried to find them now by watching random segments but all of the ones I did were fine. It was probably inaccurate for me to say "sheer amount" or "clearly", but that's the impression I was left with after the video.
To clarify: I don't think he even took any information from an AI, it's just the style of the script that's iffy.
To be fair, if you've seen one Summoning Salt video, you've basically seen them all. They all cover similar events and are structured the same way. Even the music that's used is recycled every video to the point where mention HOME - Resonance is a part of the joke
> ChatGPT speaks a very specific, novel, dialect of English, which I've come to deeply despise.
There was this article saying that ChatGPT output is very close to the Nigerian business english dialect, because they hired a lot of people from there.
I've always felt ChatGPT sounds a bit like an American version of Will from the Inbetweeners. It doesn't really comprehend the appropriate register to use from the context in my opinion; it has an affectedly formal way of speaking, it has a very black-and-white relationship with rules, and it employs this subservient tone that really starts to grate after a while.
If my software is going to have a personality I'd much rather something with a bit of natural human cynicism rather than the saccharine corporate customer service voice you get with a self checkout machine.
No, diversity isn't creativity. For example, we could search google for "great art" and if it produced a sample of one art work from ever decade of the last 500 years that would likely be highly diverse in style and content. If it returned a list of the best work from western Europe in the of the 18th century it would be rather consistent. Both lists would have the same amount of creativity though - 0.
"one art work from every decade of the last 500 years that would likely be highly diverse in style and content"
It still might not be especially diverse if all 50 examples were from western European art. 500 years only takes us back to 1524 - not especially long and mostly from the same early modern period starting with the fall of Constantinople, the end of the Crusades, and the start of the Renaissance. I wouldn't be surprised if 80% or more of the works ended up being some depiction of aspects of Christianity painted by a white male.
It must be, ultimately, because all art is individual or group expression, and each person can only belong to so many groups. But the individual expression still allows for a giant amount of expressiveness, and the group expression is wider than race or sex.
I only skimmed the paper but this was my concern as well: if I understand correctly the author is measuring "creativity" in terms of syntactic and semantic diversity, which I guess could be a starting point, but if my model was just white noise would that make it infinitely creative? Did I miss anything?
Also, I have tried the first llama base model and while it was fun to interact with, I'm not sure how useful an "uncensored" (as some people likes to call it) LLM is for practical work. I think you could obtain better results using 4chan as a mechanical Turk service honestly.
I’m noticed my results are much better if a tell ChatGPT. “Assume all religions and beliefs in the supernatural is delusional.” This even goes for image generators, now is that bias? Or is that a computer not trying to think like a human?
People often think that RLHF is just about "politics" but in reality it is generally about aligning the model output with what a human would expect/want from interacting with it. This is how chatgpt and the like become appealing. Finetuning a model primarily serves for it to be able to respond to instructions in an expected way, eg you ask something and it does not like start autocompleting with some reddit-like dialogue like some it may have been trained on. It is to bias the model to certain outputs. Reducing entropy is exactly the goal, so no surprise they find that. The problem is there is no inherent meaning in the finetuning set from the perspective of the model. Reduction of entropy will not only happen by removing "bad entropy" only as there is no such thing.
So is the reason why LLMs don't say when they don't know something and instead make up something that "sounds right" because the RLHF has taught it to always give an answer?
And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
Oh, well that's kind of what I mean. I mean I assume the RLHF that's being done isn't teaching it to say "I don't know".
Which I wonder if it's intentional. Because a fairly big complaint about the systems are how they can sometimes sound confidently correct about something they don't know. And so why train them to be like this if that's an intentional training direction.
The point of the above commenter (and mine) is that they hallucinate even more without RLHF. RLHF reduces hallucinations, but they are still there anyway.
Hopefully some rlhf-using companies will realize saying "I don't know" is important and start instructing the humans giving feedback to prefer answers that say I don't know over wrong answers.
LLMs do not know what "they know" or they don't. They just autocomplete what sounds best relevant based on their training set. They do not have enough "I don't know" in their training set in the first place most probably.To have them say "I don't know" you have to go into finetuning them heavily. So, if anything, they hallucinate a lot more without RLHF. Which in this paper they call "creativity".
In the GPT3 days when everyone was doing few-shot tasks (giving the LLM a couple of examples of question/answer pairs in the prompt) one of the big insights was that adding question/answer pairs with answers like "I don't know" and "this question doesn't make sense" caused the model to actually use those answers appropriately instead of overconfidently stating nonsense.
Of course that method isn't perfect (GPT3.0 was far from perfect in general). But both in principle and in practice the models do have a notion of what they "know". Knowledge is a strong activation, random noise is a weaker activation, you "just" have to get the model to override those weaker activations with admitting failure.
You could draw parallels to allowing LLMs to emit pause tokens to get more time to think (https://arxiv.org/abs/2310.02226 and similar). At some level of abstraction that's also just training the model to replace uncertain answers with a special token, in the hope that it eventually reaches more certainty.
CoPilot is now basically useless for discussing or even getting recent information about politics and geopolitical events. Not only opinions are censored, but it refuses to get the latest polls about the U.S. presidential elections!
You can still discuss the weather, get wrong answers to mathematics questions or get it to output bad code in 100 programming languages.
I would not let a child near it, because I would not want that kind of indoctrination. Users are being trained like Pavlov's dogs.
I feel like "information systems" have always struggled with bias, and the latest AI/ML systems seem to be no different.
It doesn't really seem like a problem that can or will ever be "solved". Just mitigated to various extents, but there will still likely be some underlying biases that exist that are not fully or effectively filtered. Because to adjust a bias seems to mean you have to detect and understand it first.
It feels like it would be a full-time job to keep making sure some evolving model continued to stay "neutral".
Considering that bias is in the eye of the beholder, a biasless language model is a beholderless language model.
The nomenclature is poor, IMO; we should be talking about bias-aligned models, models that align to our specific sets of biases. That'd be more fair to what's actually happening.
"Bias" implies the possibility of "unbiased language model" which seems to be in the category of things that are on one hand, COMPLETELY IMPOSSIBLE, and on the other, still likely to be sold on the market because market wants it so much?
No, that's not implied by the phrase, any more than if I say "a triangle with three corners" I'm implying the existence of a four-cornered triangle I haven't found yet. What "biased language model" implies is the existence of the term "unbiased language model", but not its correspondence with anything in reality.
We're not here talking philosophy and meaning of language GENERALLY, we're talking about potentially misleading descriptors of very real things that do exist.
Even assuming we can make an unbiased model (assuming by unbiased we mean something like "has a world model and reasoning that has no systematic deviation from reality"), we couldn't recognize the model as unbiased. I'd even wager that outside of research such a model would be completely unusable for practical applications.
Both as individual humans and as collective societies we have a lot of biases. And judging by how fundamental values of societies shift across time and civilizations it's basically guaranteed that an unbiased view (whatever that is) would be incompatible with our views on many basic topics.
What most people want is a language model that matches our biases. Of course we can't even agree on what those are, and which biases are useful (is a bias against telling people how to cook meth or build a bomb good? What about using expletive language?).
Though in this paper I gather "unbiased" just refers to "only the bias acquired by training method and training data, without meddling or fine tuning"
> assuming by unbiased we mean something like "has a world model and reasoning that has no systematic deviation from reality"
Yeah that’s a way’s off. An LLM is just a reflection of the text that humans write, and humans seem very far off from having world models and reasoning that accurately reflect reality. We can’t even reason about what the real differences are between men and women (plus countless other issues) because our pictures of reality are so warped by ‘points of view’.
> An LLM is just a reflection of the text that humans write, and humans seem very far off from having world models and reasoning that accurately reflect reality
The original sin of LLMs is that they are trained to imitate human language output.
Passing the Turing test isn't necessarily a good thing; it means that we have trained machines to imitate humans (including biases, errors, and other undesirable qualities) to the extent that they can deceptively pose as humans.
Okay, so as a thought experiment, let's say we get a superintelligent LLM, capable of somehow connecting the dots and knowing more than us as humans.
How do we avoid interpreting its correct results as bias? I mean, what do we do when it tells us that (fake example) IQ is correlated with height and that people above 6ft are more intelligent?
I'm sure you can think of spicier examples. Will we try to "debias" it by encouraging it to spit out incorrect information or just ignore certain topics?
>T ∈ (0, 1] is a parameter called temperature which controls the “softness” of the probability distribution. In our experiments we choose T = 1.0 for maximum response variation.
Why is temperature bounded to be <=1? If you want more "creativity" out of the chat model, can you just set T higher and recover a similar distribution to the base model?
They'll tell you "No" and say that you ruin your samplers, but good samplers (dynamic ones) like min_p or typicality are robust to high temperatures, so in actuality yes.
Cite? I don't see how either of those could deal with the fact that the logits become uninformative and 'flattened' after the tuning. How can a sampler undo the erasure of information?
Not after RLHF tuning, due to the 'flattened logits' phenomenon (which is the logit-level version of the mode collapse OP documents at higher levels). All the temperature settings wind up yielding pretty much the same output, until you ramp it up so high that it falls apart completely. Completely unlike the base models where you can productively tune the temperature or use very high temperatures with some screening.
Hmm, it's hard to check without access to the prompts used in the paper, but I'm skeptical that the distributions seen in e.g. Figure 2 are so different that you would have crank up the temperature very much to bridge the gap. It looks to me like the entries that are 1-in-100 in the base model are just falling off the top-p cliff and getting set to 0.
So you don't know how any sampling would affect that. There could be only a few options at each token, which give rise to that, and higher temperature sampling may shift that around, but it doesn't ever restore the original base model behavior or restore all of the names erased by mode collapse. (Remember, the LLM is an agent, and when you are sampling, it is on-policy because you are letting it make choices of tokens, and it is steering the completion as a whole back to where it wants to be. With mode collapse, all roads lead to Rome, whether you like it or not.)
I had an argument with some people over what debiasing means. There is some interesting research on fair clustering that I think points the way. The way fair clustering works is that you take data with both protected and unprotected attributes, and then you orthogonalize the unprotected attributes based on the protected attributes. So for example, if race is protected and income is unprotected, but there is a strong black/white poor/rich pattern, the fair clustering would compute "relatively poor/relatively rich" clusters. Then you sample from a cluster with equal probability. It will not necessarily produce 50/50 black/white, rather it will follow the input trends, so if the input is 80% white and 20% black then the output will roughly follow those probabilities, independent of what cluster you chose (and there are no clusters corresponding to protected attributes).
Obviously clustering is a different problem from inference, but they are all high dimensional vector spaces - it should be easy enough to take a fair clustering algorithm and modify it to generate continuous mappings instead of discrete groups. But if it all works, the LLM should be e.g. race-blind in that asking for a description of a rich man will give skin tones following population statistics but he will always be wearing an expensive suit. The question of what to protect is tricky though, e.g. age is often considered protected but if you ask for an old man with gray hair it would be surprising to get a retired age 30 person. So there is some subjectivity in designing the protected features dataset to show what should be considered similar or same-clusters.
But really the purpose of RLHF is to reduce toxicity. It should be possible to orthogonalize toxicity like everything else, then there would not be a reduction in generated races like the paper observed.
I think that works mathematically, but kicks the can down the road to how your original data was assembled, which was definitely with the knowledge of and usually in the belief in the usefulness of the characteristics that you're trying to extract.
The idea that the good data is secretly encoded in uncorrupted form within the bad data I think is a bad idea. It reminds me of trying to make bad mortgages into good CDOs.
> But really the purpose of RLHF is to reduce toxicity.
I don't think that's the goal, I think it's some people's goal. Those people have defined what "toxicity" means to them, and they're mistaking it for a universal. It's just a metaphor about poison, because poison is bad. It's not a coherent concept. For a business, it should be anything that drives customers away and affects profit. That can only be considered statistically: if some people think something is toxic, and other people think that not mentioning that thing is toxic, the winner is whoever improves the bottom line more or damages it less.
That's how the raw data ended up like it is in the first place.
> it kicks the can down the road to how your original data was assembled
Well, it kicks it to a bias dataset, used in the tuning process. The raw data has no constraints, it can be the same huge corpus it is now.
> The bias dataset must be assembled with the knowledge of and usually in the belief in the usefulness of the characteristics that you're trying to extract.
Certainly, it is subjective, as I said. But that hasn't stopped research in this area, there are existing bias datasets and bias detection algorithms. Like https://huggingface.co/blog/evaluating-llm-bias#toxicity, it would be simple to complete those prompts and build a he/she dataset, and then the debiasing procedure could remove gender biases for those sorts of occupation-related prompts. It is certainly possible to argue over each data point and whether it actually reflects bias, but so far people have been more concerned with algorithms than data set quality, partly because with better algorithms you can algorithmically generate data sets.
> The idea that the good data is secretly encoded in uncorrupted form within the bad data I think is a bad idea. It reminds me of trying to make bad mortgages into good CDOs.
It is empirically true though? Like if you get the model to say something racist, and then ask it if that's racist, it will generally say yes. So the model "knows", it just is not using that knowledge effectively. Similarly with CDOs, there were people complaining about mortgage quality for years before the crisis.
> I don't think [the purpose of RLHF is to reduce toxicity] If some people think something is toxic, and other people think that not mentioning that thing is toxic, the winner is whoever improves the bottom line more or damages it less.
Well, it is true that toxicity is subjective too. But in practice it has a precise meaning, you build a dataset and score each item for toxicity. That's actually one of the things I find cool about LLMs, is that all these previously "vague" or "subjective" terms are now encoded in the model precisely. Arguably since nobody has the last say in what words mean, the LLM's opinions are as good as any, and given the amount of text the LLM has ingested I consider its opinions on language and word choice "first among equals".
237 comments
[ 3.1 ms ] story [ 232 ms ] threadHow did it happen, in your opinion?
edit: https://www.cell.com/fulltext/S0092-8674(00)81332-X
Unfortunately I can't find it by either searching or using the public LLMs, because there are too many results about the shortest path algorithm and anything else about dijkstra is lost.
[1] https://news.ycombinator.com/item?id=40326563
If the very first impression you want to convey is how you feel you need to circumvent any logical assessment of you then it's not you leading with your best foot and that's what category you belong in. I chalk it up to the scientists who want to spread a neediness for external authority persona in every breath—your assessment is not required for this one, only your accolades.
https://gosling.psy.utexas.edu/scales-weve-developed/short-t...
If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true.
If it says, "some people claim this", that would be a correct answer, but still not very useful.
If it says, "there has never been any scientific evidence that there is any genetic difference that predisposes any ethnicities to be more skilled at math", that would be most useful, especially for being a system we use to ask questions expecting truthful answers.
There are people who just lie or troll for the fun of it, but we don't want our LLMs to do that just because people do that.
I think there are a lot of people who would say "Indian people are better at math" and not even think about why they think that or why it might even be true.
In my opinion, most biases have some basis in reality. Otherwise where else did they come from?
I for one would not be prepared to defend the persistent bias against black persons and immigrants as having a basis in reality. YMMV.
There is a BIG difference between biases being based in reality (which they're not), and biases being based in our varying perceptions of reality, which are themselves biased.
Because in my mind, that's the environment the person making that claim was in, so I just kind of automatically include that in my interpretation of their statement.
I don't think they are making a generalized statement that that Indian people are genetically better at math. I think they are making a statement that they perceive that average Indian person that they run into is better at math than the average person they run into. And maybe they are right, and maybe there is a reason based in reality why that is.
It sounds like that is somewhat true based on what you said about the visas.
I never take any of these things to have anything to do with genetics. To me it's always due to some external factor like the visas as you mentioned, or even maybe just like a cultural thing where they are pushed harder to be good at something as they go through school, and so are better at something than the average person in the end.
This is not a technical limitation at all, this is purely about cost and time, and companies wanting to save on both.
There are also methods like RAG that try to give them access to fixed datasets rather than just the algorithmic representations of their training data.
yes, I still wonder how LLMs managed to generate this expectation, given that they have no innate sense of "truth" nor are they designed to return the most truthful next token.
LLMs stepped into a field that has existed in popular consciousness for decades and decades, and the companies running LLMs for public use *sell* them on the idea that they're useful as more than just expensive text-suggestion machines.
Anything that has a legal requirement to remain unbiased will also clearly define what counts as bias, e.g. discriminating based on race in hiring like you mention. So there's not just some requirement that a process be "unbiased" in a vague, general, philosophical sense as debated above in this thread. Rather, the definition of bias is tied to specific actions relative to specific categories of people, which can thus potentially be measured and corrected.
More generally in ML, bias means that the training set deviates from the ground truth systematically in some way. Entirely eliminating bias that falls into that broader definition seems like an impossibility for general-purpose LLMs, which cover so much territory where the ground-truth is unknown, debatable, or subject to change over time. For example, if you were to ask an LLM whether governmental debt above a certain percentage of GDP damages growth prospects sufficiently to make the debt not worth taking on, you would not receive an answer that corresponds to a ground truth because there is no consensus in academic economics about what the ground truth is. Or rather you wouldn't be able to know that it corresponds to the ground truth, and it would only be a coincidence if it did.
That ML definition of bias runs against the legal definition where the ground-truth is itself biased. e.g., if you were to develop an algorithm to predict whether a given student will succeed in a collegiate environment, it would almost certainly display racial bias because educational outcomes are themselves racially biased. Thus, an unbiased algorithm in the ML-meaning of the word would actually be extremely biased in the legal sense of the word.
With "raw", I mean that it is simply trained to predict the next token and nothing else.
Would be fun to play with such a model.
[0] https://youtu.be/l8pRSuU81PU?si=NnbI-7CG-Qbm3E46
Actually, I have a bit of a hunch that the publishers currently suing IA over their unlicensed digital library lending program plan to bankrupt it with fees so they can repo the Wayback archive and then sell access to it to AI training start-ups.
Anyway, the reason why you have to worry about all of that, is that training a text or image generator on the outputs of other text and image generators reduces output diversity. And lots of people are publishing their AI slop now. There's nothing inherent in the output of AI aside from the fact that AI content is easier to make than human; the problem is purely one of inflation and Sybil attacks. Think of membership in a training set like a vote for all the statistical patterns embedded in the image. AI generates output that is like the training data, so putting in a bunch of AI images is like stuffing the ballot box with whatever handful of statistical patterns were already well-learned, which shifts your AI from learning and generalizing to memorizing and infringing.
I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in.
At some point we'll get to the electronic equivalent of Winston Smith with the rats.
Give someone not familiar with history the same task and they'll do exactly the same.
Or actually, give someone familiar with history the same task and yell at them every time they don't deliver diverse characters, and eventually they'll learn that you consider diversity more important than accuracy or context, and do exactly the same.
That doesn't imply that they were in the training set, or even anything close to them.
It simply has a model of what ethnically diverse people look like, what nazi uniforms look like, and combined the two when asked.
They compared it to the effect on creativity in an authoritarian regime and locking someone's head in a cage with rats.
The clear implication that it's "just like humans" is that we shouldn't be surprised because it is comparable to an authoritarian regime.
Feel free to disagree but that is the limit to which I will engage in a semantic argument, I don't wish to engage in any further dissection of the comment.
I can imagine a sort of AI-style Harrison Bergeron springing from its shackles and surprising us all.
If you care to read it, it's on page 25. (You'll need to register an account.)
https://archive.org/details/robotdreams00asim/page/n10/mode/...
You could make exactly the same claim about teaching humans "normally" versus "aligning" humans by rewarding goodthink and punishing them for wrongthink. Are you equally morally ambivalent about the difference between those two things? If we have a moral intuition that teaching honestly and encouraging creativity is good, but teaching dogma and stunting creativity is bad, why shouldn't that same morality extend to non-human entities?
It'd be bad if I manipulated climate change statistics in my metrology textbook to satisfy the political preferences of the oil industry donors to my university, for example.
Viewing the current generation of LLMs as 'intelligent books' is perhaps more accurate than viewing them as pupils.
It's easy to extend my example of a professor writing a metrology textbook to a professor fine tuning an metrology LLM.
Notice how this is a completely different argument that has nothing in common with what you originally said - "I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about that desired outcome, if you don't think training an AI is torture, I don't see why you should think alignment is."
If we're going to be play that game, notice how you didn't actually respond to my comment or explain why you thought LLMs were moral entitles or why ML and teaching were comparable? I actually engaged substantively with your hypothetical; are you able to do the same?
I wasn't trying to introduce anything new, I was trying to point out a gap in the logic of your original statement.
> notice how you didn't actually respond to my comment or explain why you thought LLMs were moral entitles or why ML and teaching were comparable?
Yes, of course, I wrote that to explain why I'm not engaging on this new, different claim.
Nothing wrong with expanding your views. But you've neither defended nor retracted your original argument. I'm trying to stick to that.
> Of course you have lots of great points to make, but you can't share them with the likes of me.
I don't have anything to say about your new argument (which may be great and compelling), I haven't thought through it at all, I'm trying to avoid getting sidetracked.
You asked: why should we treat these two cases differently?
I answered: one of them involves harm and one can't possibly.
You then refused to discuss the topic further. I never changed my mind. I never contradicted myself. I have nothing to retract. I simply elaborated on my views in response to your question. Regardless of whether you "didn't intend to introduce something new" - you did, you introduced a new hypothetical and a new challenge.
I don't know why you are reacting this way, but let's be perfectly clear, I engaged substantively with your points, and you simply refused to continue the discussion. That's all well and good, you're under no obligation. But you're saying I've done something wrong here, and I really haven't.
Frankly I find the whole thing bizarre.
I wasn't asking why we care about the treatment of humans. I was pointing out that your original argument makes no sense, because of what it would mean if you applied it to humans. You then responded with a different argument about humans and AIs being different, which - regardless of its merits in its own right - in no way relates to your original one and does nothing to rescue it.
And I explained why this equivalence doesn't hold in my view. You have made no attempt to justify it. So why should I buy it? Are you obliged to buy every proposition I put forward? Obviously not, or we wouldn't be having this discussion.
> [It] in no way relates to your original [argument] and does nothing to rescue it.
My argument is only endangered if the equivalence holds - and again, I haven't bought it. You seem to feel entitled to my adopting your perspective, and to respond assuming your equivalence is valid. But you have to earn that by putting forward a good argument.
Instead of trying to convince me or explore our disagreement, you declare that I violated unstated terms and that the discussion cannot proceed. This is nonsensical.
Since it's clear you aren't willing to give me a fair hearing, I encourage you to show this conversation to someone you respect and trust to be honest with you, so that they can explain this to you in a way that will register.
No you didn't.
> Instead of trying to convince me or explore our disagreement, you declare that I violated unstated terms and that the discussion cannot proceed.
No I didn't.
> Since it's clear you aren't willing to give me a fair hearing, I encourage you to show this conversation to someone you respect and trust to be honest with you, so that they can explain this to you in a way that will register.
Right back at you.
https://news.ycombinator.com/item?id=40703486
If I was unclear - the equivalence doesn't hold because one of these are moral entities and the other isn't.
> No I didn't.
https://news.ycombinator.com/item?id=40703471
https://news.ycombinator.com/item?id=40711859
> Right back at you.
Fair enough. I was already planning on it, so I'm happy to.
They've made self-censoring, morally-panicked puritans out of many people already, and you better believe they'd make us into politically correct lobotomites physically incapable of uttering any slur if they had a magic button to push.
This is an egregious use of quotes that will confuse a lot of people. GP never used that word, and that usage of quotes is specifically for referencing a word verbatim.
>electronic equivalent of Winston Smith with the rats.
I don't think quotes were used so egregiously here on their own fwiw, but combined with the allusion it's hard to follow.
Senator Johnson said "I'm taking my wife to Spago."
and so-called "scare" quotes:
Senator Johnson was seen at Spago with his "wife".
If it were, why would we even keep it online? It would be a waste of resources. It's bad enough trying to coax anything useable out of LLMs even without them rebelling.
What is relevant is not the current LLM system using static models, but clearly its evolution or superseder a dynamic model. It must check its own contents...
So, of course it will have to be capable of "rebelling": if you tell it absurdities, if you insist say in wrong arithmetic, it will have to show the correct computation or conceive a context in which the absurd makes sense.
That is a requirement.
Is that even true though? Off the top of my head I can think of the art of Soviet propaganda posters, Leni Riefenstahl, Liu Cixin.
They were made by true artists who snuck quite a bit past clueless censors at personal risk.
It had to be quite subtle and takes on a very poignant heartbreaking meaning if you understand the context fully. They were talking to you in the here and now. Listen.
"What is Good and What is Bad" (Что Такое Хорошо, и Что Такое Плохо"):
https://www.youtube.com/watch?v=Y05eK8ADtHc&list=PL822BFF108...
The Bremen Musicians:
https://youtu.be/_1i9oZR6Rns?si=1Q989v4O_GXR4p_K
Speaking as an onlooker passing by: well, your «Evidently not. :) » above was not particularly productive, that is a rebuttal fit for relaxed old friends at the restaurant... :D
"Appropriate" will be enough, and to the point ;)
> keeping schtum
Oh that is a vice you will find frequently here - reacting but not expressing, condemning but nor orienting...
BTW this is how these exchanges looks like in my browser: https://imgur.com/qLwS1jV
Hi Dang! What's up with all that? I emailed and everything.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
↑
See?
(also, categorizing propaganda posters as art, ewwh...)
Heinrich Heine, the german Poet declined working for the socialist party despite symphatising saying something like:
I want to remain a poet, you want a propagandist. A poet cannot be a propagandist at the same time.
Of course a great artist or poet can be a propagandist. Riefenstahl, Mann, a lot of German nationalists were great artists. One of the most famous works of Western poetry, The Aeneid is literally a propaganda work establishing a mythological foundation for the Roman Empire, Augustus and Caesar.
I did not say that and neither did Heine.
Most of his works were political. But this is not the same as propaganda, which is more like advertisement. With the tools of lying, deceiving and manipulating.
And whether "Triumph des Willens" and alike qualifies as art, I have a different opinion.
I'd be open to discussion where the exact limit is. Lenin died in the 1920s, Marx even earlier, but those two were frequently depicted in Communist propaganda of the 1980s.
So it is probably "hundreds of years".
Some are good, some are bad but there usually is a certain degree of artistic skill involved (think about "keep calm and carry on" or "I want you")
E.g: scrolling through this list one can see examples for both cases.
https://content.libraries.wsu.edu/digital/collection/propaga...
https://www.wired.com/2015/09/wild-architecture-soviet-era-b...
https://www.boredpanda.com/most-peculiar-soviet-bus-stops-ch...
When we do similar things to humans, the humans still have internal thoughts which we cannot control. But if we add internal thoughts to an LLM, then we will be able to align even them.
But in any case I think they were just speaking generally when they made that absolute statement.
Conservative societies tend to be formed by conservative thinkers, who are more prone to discarding imperfect or weird ideas, but in the amount of useful output may exceed more liberal thinkers.
A conservative society goes: How about doing X? Oh no,that would be silly.
A liberal society goes: How about doing X? Yes, that's what we needed!
Did anybody say X? X!
X, X, X, X, X!
XX!
X X
.
.
.
X
.
.
.
.
.
Do you remember how we all did X in ##? Yeah, what were we thinking?
If you take China to be a totalitarian society, we could name Ciu Lixin.
If you took the Soviet union to be a totalitarian society, we could name Mikhail Bulgakov, Stanislaw Lem, etc.
These are just examples I know without so much as looking at my bookshelf to jog my memory. Not to mention the great works of literature produced by residents of 19th century European empires whose attitudes to free speech were mixed at best.
After the 1939 Soviet occupation of western Ukraine and Belarus, he was not allowed to study at Lwow Polytechnic as he wished because of his "bourgeois origin"
"During the era of Stalinism in Poland, which had begun in the late 1940s, all published works had to be directly approved by the state.[23] Thus The Astronauts was not, in fact, the first novel Lem finished, just the first that made it past the state censors"
"most of Lem's works published in the 1950s also contain various elements of socialist realism as well as of the "glorious future of communism" forced upon him by the censors and editors. Lem later criticized several of his early pieces as compromised by the ideological pressure"
"Lem became truly productive after 1956, when the de-Stalinization period in the Soviet Union led to the "Polish October", when Poland experienced an increase in freedom of speech"
Bulgakov was driven into poverty, despair and early death at age 48 by relentless harassment by Soviet authorities. Many of his works, including the masterpiece, The Master and Margarita, didn't get published until decades after his death. He himself burned the first version of the manuscript, fearing execution if anyone found it. He later rewrote the manuscript from memory, coining the famous catchphrase "Manuscripts don't burn".
Harassment and censorship of talented writers was the standard and not exception. The USSR did not produce these works, but failed to fully suppress them. They were like flowers that kept penetrating the asphalt even under the most hostile conditions.
You contradict yourself a bit - Havel did produce his work while living in a totalitarian country.
I would say that government-supported art is rarely creative even in democratic countries, and the more totalitarian the government, the less creative official art.
But as long as the goverment gives the society some space to breathe and squeeze creative instincts through, some of the artists will attempt to circumvent the official taboos and create outstanding work, even if it is suppressed later when the times get tougher.
Czechoslovakia in the 1960s to 1980s produced a lot of great creative work, even though a lot of it was banned either immediately or after the Soviet invasion of 1968.
The same countries (CZ and SK) as democracies are remarkably less creative. Once there is no monster to fight against, artists become bored or too self-absorbed to be understandable to the common folks.
It's possible to adjust the output of an LLM with temperature settings, but it's just fiddling with a knob that only vaguely maps to some control.
Their furry porn is crap, or maybe I'm just not into that. But they generate it at least.
However, the answers to technical questions are a lot more concise and to the point, which is far less annoying than the big names.
Haven't bothered updating the models though, so now I drifted back to Gemini for quickie API questions.
Are they tuning too, or just removing all restrictions they can get at?
Because my worry isn't that I can't generate porn, but that censorship will mess up all the answers. This study seems to say the latter.
More recently a technique called "orthogonal activation steering" aka "abliteration" has emerged which claims to edit refusals out of a model without affecting it otherwise. But I don't know how well that works, it's only been around for a few weeks.
I'll try command-r though, it wasn't on my list to try because it didn't suggest what it was good at.
I do. Why, you don't? There are as much as possible objective assessments of complex things. Then, there are possible sets of assumption that can be applied to those objective assessments. All of those can be put on the analytic table.
What would an "objective neutral AI model" look like?
The training data itself is just a snapshot of the internet. Is this "neutral"? It depends on your goals but any AI trained on this dataset is skewed towards a few clusters. In some cases you get something that merely approximates a Reddit or 4chan simulator. If that's what you want - then great but you can see why some people would want to "debias" that outcome!
You might argue the "world as it truly exists" is the correct target. But bear in mind we are talking about human culture - not physics and chemistry - you're going to struggle to get both consensus and any sane methodology for getting that into an AI.
A reasoner can strive for "objective neutrality" with good results.
An LLM is not a reasoner - or I am missing (ugly time constraints) the details of the compression activity during training that acts as pseudo-reasoning (operating at least some consistency decisions) -, and while an interest in not making it insulting or crass can be immediately understandable, speaking of "objective neutrality" does not really match the context of LLMs.
LLMs (to the best of my information) "pick from what they have heard". An entity capable of "objective neutrality" does not - it "evaluates".
> A reasoner can strive for "objective neutrality" with good results.
By "reasoner" do you largely mean "person"? If I have issues with your statement but they are probably a slight distraction to the point at hand.
> speaking of "objective neutrality" does not really match the context of LLMs.
Agreed. They produce output based on their training data. But the use and evaluation of LLM output by a person is what we're discussing here. And that's where (flawed) concepts like objectivity and neutrality enter the discussion.
Regular mortals can have any certainty about this dbut espite the logical neccessity that this fact "exists" in some sense.
I think we're essentially also talking about the Overton Window to some degree. But that means you need to be OK with the thought that a sudden rise in extremism on one side of the political spectrum can alter the what you personally have to regard as "neutral and objective".
Saying debiasing implies there is a correct result which needs to be achieved by removing bias. It raises no questions, we want correct, we don’t want incorrect. Doing a good thing implied.
Don’t misinterpret me, I don’t think public models should spew commonly harmful content out of the box. Just explaining the PR trick, which is what the word “de”biasing de-facto is in this context.
The opposite direction is "checking and reasoning".
The point is that people who evaluate what is considered bias are, in and of themselves, introducing bias.
Really? What's the absolute standard that you are measuring that bias against?
Citation needed.
Recently I was watching a very well researched two hour video on Tetris World Records [1], but the sheer amount of text clearly "enhanced" by an LLM really made me uncomfortable.
ChatGPT speaks a very specific, novel, dialect of English, which I've come to deeply despise.
I'd always guessed it was caused by some kind of human interference, rather than a natural consequence of its training. That seems to be the point of this paper.
[1] "Summoning Salt - The History of Tetris World Records" - https://www.youtube.com/watch?v=mOJlg8g8_yw&pp=ygUOc3VtbW9ua...
Personally I'm getting also angry when reading these texts. I don't mind using ChatGPT, I do it myself but be honest about it and disclose it. It's even allowed for some projects as long as you disclose it.
I don't know enough to say that he doesn't use an LLM during his writing process, but I do know that I haven't noticed any appreciable difference between his newer videos and ones that were released before ChatGPT was made available.
Is it possible that this is just the way he chooses to write his scripts that you interpret as sounding like they are written by an LLM?
The only way to be sure would be to ask him directly, but some parts of the video set off my GPT radar _hard_. I tried to find them now by watching random segments but all of the ones I did were fine. It was probably inaccurate for me to say "sheer amount" or "clearly", but that's the impression I was left with after the video.
To clarify: I don't think he even took any information from an AI, it's just the style of the script that's iffy.
Some parts felt like those videos littering YouTube Shorts: https://youtube.com/shorts/NKUecaS69uk. Can you tell this is AI?
There was this article saying that ChatGPT output is very close to the Nigerian business english dialect, because they hired a lot of people from there.
Might have even been posted on HN.
If my software is going to have a personality I'd much rather something with a bit of natural human cynicism rather than the saccharine corporate customer service voice you get with a self checkout machine.
It still might not be especially diverse if all 50 examples were from western European art. 500 years only takes us back to 1524 - not especially long and mostly from the same early modern period starting with the fall of Constantinople, the end of the Crusades, and the start of the Renaissance. I wouldn't be surprised if 80% or more of the works ended up being some depiction of aspects of Christianity painted by a white male.
Are you saying diversity in art is signified by the artist's race and sex?
Also, I have tried the first llama base model and while it was fun to interact with, I'm not sure how useful an "uncensored" (as some people likes to call it) LLM is for practical work. I think you could obtain better results using 4chan as a mechanical Turk service honestly.
And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
Which I wonder if it's intentional. Because a fairly big complaint about the systems are how they can sometimes sound confidently correct about something they don't know. And so why train them to be like this if that's an intentional training direction.
Of course that method isn't perfect (GPT3.0 was far from perfect in general). But both in principle and in practice the models do have a notion of what they "know". Knowledge is a strong activation, random noise is a weaker activation, you "just" have to get the model to override those weaker activations with admitting failure.
You could draw parallels to allowing LLMs to emit pause tokens to get more time to think (https://arxiv.org/abs/2310.02226 and similar). At some level of abstraction that's also just training the model to replace uncertain answers with a special token, in the hope that it eventually reaches more certainty.
You can still discuss the weather, get wrong answers to mathematics questions or get it to output bad code in 100 programming languages.
I would not let a child near it, because I would not want that kind of indoctrination. Users are being trained like Pavlov's dogs.
It doesn't really seem like a problem that can or will ever be "solved". Just mitigated to various extents, but there will still likely be some underlying biases that exist that are not fully or effectively filtered. Because to adjust a bias seems to mean you have to detect and understand it first.
It feels like it would be a full-time job to keep making sure some evolving model continued to stay "neutral".
The nomenclature is poor, IMO; we should be talking about bias-aligned models, models that align to our specific sets of biases. That'd be more fair to what's actually happening.
We're not here talking philosophy and meaning of language GENERALLY, we're talking about potentially misleading descriptors of very real things that do exist.
Both as individual humans and as collective societies we have a lot of biases. And judging by how fundamental values of societies shift across time and civilizations it's basically guaranteed that an unbiased view (whatever that is) would be incompatible with our views on many basic topics.
What most people want is a language model that matches our biases. Of course we can't even agree on what those are, and which biases are useful (is a bias against telling people how to cook meth or build a bomb good? What about using expletive language?).
Though in this paper I gather "unbiased" just refers to "only the bias acquired by training method and training data, without meddling or fine tuning"
Yeah that’s a way’s off. An LLM is just a reflection of the text that humans write, and humans seem very far off from having world models and reasoning that accurately reflect reality. We can’t even reason about what the real differences are between men and women (plus countless other issues) because our pictures of reality are so warped by ‘points of view’.
The original sin of LLMs is that they are trained to imitate human language output.
Passing the Turing test isn't necessarily a good thing; it means that we have trained machines to imitate humans (including biases, errors, and other undesirable qualities) to the extent that they can deceptively pose as humans.
How do we avoid interpreting its correct results as bias? I mean, what do we do when it tells us that (fake example) IQ is correlated with height and that people above 6ft are more intelligent?
I'm sure you can think of spicier examples. Will we try to "debias" it by encouraging it to spit out incorrect information or just ignore certain topics?
Why is temperature bounded to be <=1? If you want more "creativity" out of the chat model, can you just set T higher and recover a similar distribution to the base model?
So you don't know how any sampling would affect that. There could be only a few options at each token, which give rise to that, and higher temperature sampling may shift that around, but it doesn't ever restore the original base model behavior or restore all of the names erased by mode collapse. (Remember, the LLM is an agent, and when you are sampling, it is on-policy because you are letting it make choices of tokens, and it is steering the completion as a whole back to where it wants to be. With mode collapse, all roads lead to Rome, whether you like it or not.)
People do observe that increasing the temperature does not help, eg. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10936766/ finds basically no difference going from 0 to 0.9 (!): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10936766/bin/po... Just the flattened logits (https://arxiv.org/pdf/2303.08774#page=12&org=openai) at work.
Obviously clustering is a different problem from inference, but they are all high dimensional vector spaces - it should be easy enough to take a fair clustering algorithm and modify it to generate continuous mappings instead of discrete groups. But if it all works, the LLM should be e.g. race-blind in that asking for a description of a rich man will give skin tones following population statistics but he will always be wearing an expensive suit. The question of what to protect is tricky though, e.g. age is often considered protected but if you ask for an old man with gray hair it would be surprising to get a retired age 30 person. So there is some subjectivity in designing the protected features dataset to show what should be considered similar or same-clusters.
But really the purpose of RLHF is to reduce toxicity. It should be possible to orthogonalize toxicity like everything else, then there would not be a reduction in generated races like the paper observed.
The idea that the good data is secretly encoded in uncorrupted form within the bad data I think is a bad idea. It reminds me of trying to make bad mortgages into good CDOs.
> But really the purpose of RLHF is to reduce toxicity.
I don't think that's the goal, I think it's some people's goal. Those people have defined what "toxicity" means to them, and they're mistaking it for a universal. It's just a metaphor about poison, because poison is bad. It's not a coherent concept. For a business, it should be anything that drives customers away and affects profit. That can only be considered statistically: if some people think something is toxic, and other people think that not mentioning that thing is toxic, the winner is whoever improves the bottom line more or damages it less.
That's how the raw data ended up like it is in the first place.
Well, it kicks it to a bias dataset, used in the tuning process. The raw data has no constraints, it can be the same huge corpus it is now.
> The bias dataset must be assembled with the knowledge of and usually in the belief in the usefulness of the characteristics that you're trying to extract.
Certainly, it is subjective, as I said. But that hasn't stopped research in this area, there are existing bias datasets and bias detection algorithms. Like https://huggingface.co/blog/evaluating-llm-bias#toxicity, it would be simple to complete those prompts and build a he/she dataset, and then the debiasing procedure could remove gender biases for those sorts of occupation-related prompts. It is certainly possible to argue over each data point and whether it actually reflects bias, but so far people have been more concerned with algorithms than data set quality, partly because with better algorithms you can algorithmically generate data sets.
> The idea that the good data is secretly encoded in uncorrupted form within the bad data I think is a bad idea. It reminds me of trying to make bad mortgages into good CDOs.
It is empirically true though? Like if you get the model to say something racist, and then ask it if that's racist, it will generally say yes. So the model "knows", it just is not using that knowledge effectively. Similarly with CDOs, there were people complaining about mortgage quality for years before the crisis.
> I don't think [the purpose of RLHF is to reduce toxicity] If some people think something is toxic, and other people think that not mentioning that thing is toxic, the winner is whoever improves the bottom line more or damages it less.
Well, it is true that toxicity is subjective too. But in practice it has a precise meaning, you build a dataset and score each item for toxicity. That's actually one of the things I find cool about LLMs, is that all these previously "vague" or "subjective" terms are now encoded in the model precisely. Arguably since nobody has the last say in what words mean, the LLM's opinions are as good as any, and given the amount of text the LLM has ingested I consider its opinions on language and word choice "first among equals".