HN Explorer

How does data removal from GPT *work

37 points by tss_caterpillar ↗ HN
OpenAI has a form where you can request that your data is “removed” from their models https://share.hsforms.com/1UPy6xqxZSEqTrGDh4ywo_g4sk30 , but how does that actually work? How does one untrain a model? Are they planning to just retrain the entire model without your data? My understanding of LLMs is mostly theoretical so maybe I’m missing something here but my understanding is that once a model is trained you can’t exactly untrain it. It’s not like you can just move the weights a gradient step in the opposite direction or something. Idk what OpenAI plans to do with data removal requests and how effective it will actually be.

20 comments

[ 2.6 ms ] story [ 53.5 ms ] thread
maebert
It's about as hard to remove a single data point from an LLM as it is to remove a single memory from a human brain.

OpenAI doesn't remove data from the trained models, they filter it at the output level. They also remove it from training data, but of course the model lives on.

I'm a strong supporter of a "do not encode" header / metadata on content that allows individuals, content creators and providers to tell AIs not to encode specific text / images / documents in the first place.

Oras
Filtering would be extremely slow especially when streaming the data.
tss_caterpillar
Would filtration be slow? ChatGPT already seems to have some filtration on the output level due to the fact that it returns one of its template responses when it’s queried something regarded as harmful. Although I imagine filtering out a list of specific information might be less straight forward than that.
cuteboy19
It does it after the stream is already complete, the output is sent to some other model to know whether it's offensive but it's possible to simply view the raw output if you just cancel the generation midway
xg15
How would "filter at the output level" even work? The output is arbitrary unstructured text. I don't see how that output could be reliably filtered except by another LLM.
albert_e
> It's about as hard to remove a single data point from an LLM as it is to remove a single memory from a human brain.

Sounds like there is a decent story that can be written about the right-to-forget in GPT/LLM era ... similar to "The Eternal Sunshine of the Spotless Mind" for human memory.

"Hallucinations" will be a main story-telling device in this one as well.

kamray23
A very interesting thing to do would be to prove that an LLM model could output your personal details, then request that as an EU citizen you would like to invoke your right to be forgotten. It'd be very interesting to see what kind of interpretability and re-training research could come out of attempting to avoid situations in which you ossify data into the model, specifically due to wanting to avoid legal repercussions.
throwaway313313
Consider the AI Sec Ops future in the EU where in the run up to an election a political operative makes all widely used AIs "forget" about the competition by submitting a flurry of fraudulent right to be forgotten requests.

This is no stretch of the imagination due to existence of fraudulent copyright takedown requests being submitted daily to Youtube against legitimate channels (with Youtube giving said legitimate channels very real copyright strikes against their accounts regularly, with only very visible cases having a chance of reversal)...

bhickey
Transformer editing is an active field of research.

https://arxiv.org/abs/2202.05262 https://arxiv.org/abs/2210.07229

cuteboy19
Doesn't this harm the claim that the models do not contain any information about a specific image by an artist?
lxgr
If so, maybe that says something about that claim?
fortyseven
I think there's probably a big difference between how data is stored in art models versus how LLMs store textual information.
tommek4077
This is like blurring houses in Google Street view in Germany. The house is still there.
hgsgm
If (misnomer) OpenAI can't explain it to you, the removal is BS.
dantraztrev
Not entirely sure but here is an hunch , a single memory is really hard to remove, and presuming they're not training it again. Here are some solutions

1) Block Personally Identifiable info to get into training data : Maybe have a redact such info before passing it to model . Just having a layer between I/O and the processing .

2) Have a sort of VCS for models and rewind and retrain if such is needed to be done . ( Seems quite unfeasible )

3) If it is not an info but rather an opinion , train the model to counter biases until a netural or agreed stance is reached

kristianp
Are there examples of gpt mentioning people like that?

Can you craft a specific prompt to get it to hallucinate info about someone non-famous without mentioning them in the prompt?

segmondy
I don't think they can. I believe they just add filtering in the data pipeline before & after the generation. Think about how they currently filter out "bad things"

user input ---> | prompt filter | GPT | output filter | --> sanitized output

[deleted]
(comment deleted)