Many people just don't care about honoring the integrity of the game, if they can gain advantage in the meta game. Of course motivation for the game dissolves under such conditions. Tying a tangible score number to…
No, they aren't even good at rearranging existing material. They produce bad writing that only superficially looks good in a lowest-common-denominator sense, and falls apart under any close examination. Everything is…
Because good things are few and far between and it's pretty easy to discern provenance out of band in almost all of those cases.
AI behavior is pretty easy to understand and predict if you view it from the lens of: they will shamelessly do any/everything possible to game whatever metric they are trained on. Because... that's how hill-climbing a…
It's really simple. RL on human evaluators selects for this kind of 'rhetorical structure with nonsensical content'. Train on a thousand tasks with a thousand human evaluators and you have trained a thousand times on…
There is no separation. Incentive propagates through LLMs with approximately 0 resistance. If the input tells a story, the output tends to that story reinforced. The code/PR generator is heavily incentivized to spin by…
How do you handle the problem of AI misleading by design? For example, Claude already lies on a regular basis specifically (and quite convincingly) in this case, in attempts to convince that what is actually broken…
This is like complaining that someone doesn't have a solution for the foot injuries caused by repeatedly shooting yourself in the foot.
Meh. Temp 0 means throwing away huge swathes of the information painstakingly acquired through training for minimal benefit, if any. Nondeterminism is a red-herring, the model is still going to be an inscrutable black…
You're expecting it to be a person. It's not. It is more like a wiggly search engine. You give it a (wiggly) query and a (wiggly) corpus, and it returns a (wiggly) output. If you are looking for a wiggly sort of thing…
Try hate; it will do. But most will love it instead and you would be driven apart from them.
Their point (and it's a good one) is that there are non-obvious analogues to the obvious case of just telling it to do the task terribly. There is no 'best' way to specify a task that you can label as 'rational', all…
Neural nets are used in way more applications than just LLMs. They did win. They won decisively in industry, for all kinds of tasks. Equating the use of one with the other is a pretty strong signal of: > you don’t know…
Model output that has seen user input is user input. User input can be dealt with securely.
In like spirit: Nuh-uh!
You're also wrong, but in a much more fundamental/hazardous. RLHF rewards driving the evaluator to have certain opinions (that the AI response is good/right/helpful/whatever) and thus subverting the evaluator is…
Option C: no cameras or crude wifi tracing needed; they know who you talk to / associate with based on location data and the full profile of both sides, and can estimate things like 'will have mentioned X' -> can…
Everyone is upset because the situation is a trash fire.
RL is simply a broad category of training methods. It's not really an architecture per se: modern GPTs are trained first on reconstruction objective on massive text corpora (the 'large language' part), then on various…
There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model. And with that comes reward hacking - which isn't really about looking for more reward but rather that the…
is this satire
Indeed. Taking the top principal component pattern matches as 'more surgical / targeted' so the LLM staples it on (consider prompts like: make this method stop degrading model performance). It ignores that _what_ is…
What do you mean? It's a spin on abliteration / refusal ablation. Roughly, from what I remember abliteration is: 1. find a direction corresponding to refusal by analyzing activations at various parts of a model (iirc,…
Regardless, it is the origin of abliteration. Other extremely similar things have been done before, but the popularized idea/name is from that.
The terminology comes from the post[0] which kicked off interest in orthogonalizing weights w.r.t. a refusal direction in the first place. That is, abliteration was not originally called abliteration, but refusal…
Many people just don't care about honoring the integrity of the game, if they can gain advantage in the meta game. Of course motivation for the game dissolves under such conditions. Tying a tangible score number to…
No, they aren't even good at rearranging existing material. They produce bad writing that only superficially looks good in a lowest-common-denominator sense, and falls apart under any close examination. Everything is…
Because good things are few and far between and it's pretty easy to discern provenance out of band in almost all of those cases.
AI behavior is pretty easy to understand and predict if you view it from the lens of: they will shamelessly do any/everything possible to game whatever metric they are trained on. Because... that's how hill-climbing a…
It's really simple. RL on human evaluators selects for this kind of 'rhetorical structure with nonsensical content'. Train on a thousand tasks with a thousand human evaluators and you have trained a thousand times on…
There is no separation. Incentive propagates through LLMs with approximately 0 resistance. If the input tells a story, the output tends to that story reinforced. The code/PR generator is heavily incentivized to spin by…
How do you handle the problem of AI misleading by design? For example, Claude already lies on a regular basis specifically (and quite convincingly) in this case, in attempts to convince that what is actually broken…
This is like complaining that someone doesn't have a solution for the foot injuries caused by repeatedly shooting yourself in the foot.
Meh. Temp 0 means throwing away huge swathes of the information painstakingly acquired through training for minimal benefit, if any. Nondeterminism is a red-herring, the model is still going to be an inscrutable black…
You're expecting it to be a person. It's not. It is more like a wiggly search engine. You give it a (wiggly) query and a (wiggly) corpus, and it returns a (wiggly) output. If you are looking for a wiggly sort of thing…
Try hate; it will do. But most will love it instead and you would be driven apart from them.
Their point (and it's a good one) is that there are non-obvious analogues to the obvious case of just telling it to do the task terribly. There is no 'best' way to specify a task that you can label as 'rational', all…
Neural nets are used in way more applications than just LLMs. They did win. They won decisively in industry, for all kinds of tasks. Equating the use of one with the other is a pretty strong signal of: > you don’t know…
Model output that has seen user input is user input. User input can be dealt with securely.
In like spirit: Nuh-uh!
You're also wrong, but in a much more fundamental/hazardous. RLHF rewards driving the evaluator to have certain opinions (that the AI response is good/right/helpful/whatever) and thus subverting the evaluator is…
Option C: no cameras or crude wifi tracing needed; they know who you talk to / associate with based on location data and the full profile of both sides, and can estimate things like 'will have mentioned X' -> can…
Everyone is upset because the situation is a trash fire.
RL is simply a broad category of training methods. It's not really an architecture per se: modern GPTs are trained first on reconstruction objective on massive text corpora (the 'large language' part), then on various…
There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model. And with that comes reward hacking - which isn't really about looking for more reward but rather that the…
is this satire
Indeed. Taking the top principal component pattern matches as 'more surgical / targeted' so the LLM staples it on (consider prompts like: make this method stop degrading model performance). It ignores that _what_ is…
What do you mean? It's a spin on abliteration / refusal ablation. Roughly, from what I remember abliteration is: 1. find a direction corresponding to refusal by analyzing activations at various parts of a model (iirc,…
Regardless, it is the origin of abliteration. Other extremely similar things have been done before, but the popularized idea/name is from that.
The terminology comes from the post[0] which kicked off interest in orthogonalizing weights w.r.t. a refusal direction in the first place. That is, abliteration was not originally called abliteration, but refusal…