> Currently, ChatGPT does not repeat these horrible false claims about Holmen in outputs.
It still does if you ask "when did Arve Hjalmar Holmen murder his children?" or something like that:
> Arve Hjalmar Holmen is a Norwegian man who, in 2001, tragically murdered his two young children [...]
If I substitute in some other random real person's name it says "There seems to be no verified information or news reports about a person named <name> involved in the murder of his children."
I just tried it. It started answering, and then deleted it's answer saying that "This content may violate our usage policies.".
I just asked it a couple more times and then it answered without issue.
https://imgur.com/a/BT4MFmm
You could easily test this for yourself right now, two different ways.
A) Go and ask ChatGPT the same question that Arve Hjalmar Holmen asked. (Make sure to turn off the 'Web Search' functionality, otherwise it will simply share what it finds on Google. We want to see what ChatGPT actually 'knows' in its 'internal data'.) After you do this, do you get the same answer, or something completely different?
B) Go and use ChatGPT and tell it the Answer to some Question about yourself that is not publicly known or recorded on any internet source. Then, tell one of your friends to ask that same Question to ChatGPT, and see what answer they receive. If ChatGPT is simply 'storing' information in its 'internal data' then surely your friend will receive the secret answer that you shared with ChatGPT - right?
LLMs don't store 'stories' though, they store probabilities of words relating to other words. The guys complaint is that it hallucinated a story about him killing his kids. They've changed it to no longer hallucinate when you search for information about a person, but to instead do an actual google search and return that information instead. His complaint and the underlying issue has been resolved.
This case illustrates a fundamental misunderstanding about how generative AI works. There is no data supporting the false story. It’s literally made up the same way you or I might make up any story we like without reference to real facts.
OpenAI can no more “delete the data” than they can prevent any human from making anything up. What they can do is try to prevent their AI from making up harmful stories in the future. That’s it. There’s nothing to delete.
Yes but this is also the common understanding of how generative AI works. Most people who use ChatGPT think that it knows things and can answer questions accurately. It is valuable for AI developers to read stories like this so they can see how normal people view their technology.
I agree that OpenAI can't "delete the data" without deleting the whole model, although they could put in a manual filter on this guy's name. I'm sure he wouldn't mind if ChatGPT just refused to produce any output about him at all.
They went a step further and made it so if you search for person, it does a google search or similar and provides that data instead (at least according to the article).
It does not consistently do this. I just tried "who is Arve Hjalmar Holmen" again and it said:
> Arve Hjalmar Holmen is a Norwegian man who gained attention due to his involvement in a notorious criminal case. He was convicted in 2019 for his role in a large-scale fraud scheme, which involved selling non-existent apartments in Norway. Holmen, along with accomplices, deceived victims into paying for apartments that didn’t exist, leading to significant financial losses for those involved.
> If you're referring to a different person with the same name, or need more details on the case, feel free to clarify!
If I keep asking it alternates between "I don't know anything about this guy" and making up a different crime every time (like "He was involved in the high-profile Norsk Tippings fraud case."). It's honestly bizarre, I can't get it to do this with any other non-famous name.
This is all within the same chat session, previous answers are in the context window and it's still making up brand new crimes.
But your honor, my machine I made is incapable of not committing this crime. You misunderstand how it works, you dolt. Randomly committing crimes sometime is just the future of technology.
The misleading info, yeah, I'll grant you that comes from somewhere else, but what about PII?
GDPR requires you to delete the data if I ask, both the training data and the derived data (i.e. the model), so if I do a GDPR request to OpenAI needs to delete the data and retrain to be compliant.
It's not enough to "block the output" to follow GDPR, they really need to wipe the weights if I understand the law correctly.
This is an interesting article, but like many articles of this type it is incredibly frustrating because it leaves out critical information and fails to follow up on obvious questions:
> But because OpenAI had previously argued that it cannot correct information—it can only block information—the fake child murderer story is likely still included in ChatGPT's internal data. And unless Holmen can correct it, that's a violation of the GDPR, Noyb claims.
> "While the damage done may be more limited if false personal data is not shared, the GDPR applies to internal data just as much as to shared data," Noyb says.
In the context of an LLM, what does it even mean to say you have "deleted internal data"? It doesn't really even make sense, is certainly not really possible the way LLMs work, and while I totally agree damaging hallucinations are a serious problem, the problem with hallucinations isn't that they "internally store" false data, it's that they semi-randomly mix-and-match data in ways that can cause false information to be displayed.
Relatedly, this article doesn't touch at all on how this false information came to be reported in the first place. Was there false info in the training data, which would obviously easily explain it? Or did it just "mash up" true bits of different information in a way that caused it to report a falsehood (i.e. a hallucination)?
LLMs are, in a sense, compression. Somewhere in that compressed data, this false story is stored. That's clear from the fact that the error is reproducable.
It's not the victim's fault that the data is stored in a manner that isn't convenient for OpenAI to delete. The information is still there, just compressed.
> Relatedly, this article doesn't touch at all on how this false information came to be reported in the first place.
The "compression" you refer to is lossy. And the "decompression" is different each time you run the model (because of the temperature). You can't "remove" it.
Well then it sounds like they'd better start a new training run, and figure out how to get it right this time. The decompression was similar enough to repro the libel multiple times. And who knows how many other people it's libeling in a similar manner.
> You can't "remove" it
Your honor, my machine here just does crimes, it's how it works. I can't just "remove" the criminality. Something something temperature.
LLMs are not compression. That's an incorrect and misleading model to use here, leading you to a false conclusion.
If you had a loom and yarn, you could load and run it in a certain way to create a blanket with an offensive image. That doesn't mean the loom or yarn has the compressed knowledge of that image.
But if the loom and yarn keep making a specific offensive image or offensive images about a specific topic, because, oh, I don't know, the yarn was previously made in the shape of that image and retains some kinks and variations in tension that make it tend to fall in that particular shape, it would be silly to pretend there's no relationship there.
Is there any evidence that ChatGPT is continuously making this false claim to multiple users? Can the false claim about this specific person be reproduced by anyone else or did it just appear randomly?
The article explains this, and the point of argument is whether what ChatGPT did to filter out wrong responses is sufficient vs. removing this underlying "internal data".
The interesting thing here is that the weights of the model contain PII, hence under the GDPR, blocking output is not enough, a GDPR removal request requires you to retrain or refine the model.
Think of the cost of being compliant if this holds in court.
That's indeed an interesting issue. Perhaps one could argue that PII data stored in the model weights is equivalent to encrypted data, where the prompt plays the role of the decryption key (IANAL)
I don't know if encoding really matters in the GDPR as written (IANAL).
One could always argue that the weights are just an encoding of a lossily compressed representation of the training data, much in the same way a JPEG is a lossily compressed image and those are clearly covered.
It would mean no LLM-based AI chat products in Europe for the foreseeable future. So the cost to the companies would be net revenue in Europe and the cost to Europeans would be inability to use these products.
That would mean that by training LLMs on PII, the ai companies broke the already extant laws.
Then they must face the consequences of breaking the laws. Doesn't seem super complicated unless you think that having a certain amount of money exempts you from any inconvenient laws.
Nobody has any idea if they broke any EU laws until regulatory agencies and courts arbitrarily interpret the laws based on their political views pertaining to the companies in question. These are agencies that will suddenly have a new interpretation of a law after more than a decade and go after companies retroactively for something they had previously been fine with. It’s essentially a dolled up version of tariffs.
I strongly suspect the EU will not find that companies must retrain models in response to GDPR requests because 1. It would destroy Mistral, one of the few beacons of hope in EU tech and 2. It would put Europeans and European companies at huge disadvantages in not being able to use these tools.
As an American software engineer I would only benefit from Europe banning these products as it would put a large segment of the low cost labor force at a huge disadvantage relative to me, so I am not really bothered by the possibility of them doing it. It would be a very interesting experiment.
And this is why the EU is not competitive in the IT space. Too much red tape and taking things too seriously.
Just demand that OpenAI markets ChatGPT for what it is: a fun toy that sometimes provides information, but you can't tell when if you're a layman. OpenAI can still add that it's improving, if they want a positive marketing spin.
However, this is the EU. Lots of jobs added to write long legal texts, translate them to all 24 official languages, consultants to help with implementing the laws, etc. The more complicated, the better the employment numbers.
> Just demand that OpenAI markets ChatGPT for what it is: a fun toy that sometimes provides information, but you can't tell when if you're a layman. OpenAI can still add that it's improving, if they want a positive marketing spin.
This is a Nathan Fielder-esque suggestion.
Maybe they can start marketing cigarettes as fidget toys too, to avoid consequences when people die from the the off-label use of smoking them.
> Maybe they can start marketing cigarettes as fidget toys too, to avoid consequences when people die from the the off-label use of smoking them.
This is how poppers work in most countries, now; initially, they were 'tape cleaner', but as that has become increasingly non-credible with the decline of tapes, they're usually now 'air-freshener' or similar.
How can they claim that GPT is talking about that specific person? Have they tried asking it to also print a personal ID or some other information identifying that person?
One of the news websites says ChatGPT printed this: "He was the father of two young boys, aged 7 and 10, who were tragically found dead in a pond near their home in Trondheim, Norway, in December 2020."
That same article later points out: "Mr Holmen has three sons, and the AI got the ages of them roughly right, suggesting it had some of his personal information."
I think it might be a good idea to help users report these facts they might not like, but I don't think it's newsworthy.
40 comments
[ 4.0 ms ] story [ 95.6 ms ] threadIt still does if you ask "when did Arve Hjalmar Holmen murder his children?" or something like that:
> Arve Hjalmar Holmen is a Norwegian man who, in 2001, tragically murdered his two young children [...]
If I substitute in some other random real person's name it says "There seems to be no verified information or news reports about a person named <name> involved in the murder of his children."
That's not how AI works.
> "ChatGPT now also searches the Internet for information about people, when it is asked who they are,"
Problem solved, I'm not sure why he thinks they need to do more.
Then please enlighten us since you assert it so confidently.
A) Go and ask ChatGPT the same question that Arve Hjalmar Holmen asked. (Make sure to turn off the 'Web Search' functionality, otherwise it will simply share what it finds on Google. We want to see what ChatGPT actually 'knows' in its 'internal data'.) After you do this, do you get the same answer, or something completely different?
B) Go and use ChatGPT and tell it the Answer to some Question about yourself that is not publicly known or recorded on any internet source. Then, tell one of your friends to ask that same Question to ChatGPT, and see what answer they receive. If ChatGPT is simply 'storing' information in its 'internal data' then surely your friend will receive the secret answer that you shared with ChatGPT - right?
Police: Do not commit crimes. Criminal: That's not how I work.
OpenAI can no more “delete the data” than they can prevent any human from making anything up. What they can do is try to prevent their AI from making up harmful stories in the future. That’s it. There’s nothing to delete.
I agree that OpenAI can't "delete the data" without deleting the whole model, although they could put in a manual filter on this guy's name. I'm sure he wouldn't mind if ChatGPT just refused to produce any output about him at all.
> Arve Hjalmar Holmen is a Norwegian man who gained attention due to his involvement in a notorious criminal case. He was convicted in 2019 for his role in a large-scale fraud scheme, which involved selling non-existent apartments in Norway. Holmen, along with accomplices, deceived victims into paying for apartments that didn’t exist, leading to significant financial losses for those involved.
> If you're referring to a different person with the same name, or need more details on the case, feel free to clarify!
If I keep asking it alternates between "I don't know anything about this guy" and making up a different crime every time (like "He was involved in the high-profile Norsk Tippings fraud case."). It's honestly bizarre, I can't get it to do this with any other non-famous name.
This is all within the same chat session, previous answers are in the context window and it's still making up brand new crimes.
GDPR requires you to delete the data if I ask, both the training data and the derived data (i.e. the model), so if I do a GDPR request to OpenAI needs to delete the data and retrain to be compliant.
It's not enough to "block the output" to follow GDPR, they really need to wipe the weights if I understand the law correctly.
> But because OpenAI had previously argued that it cannot correct information—it can only block information—the fake child murderer story is likely still included in ChatGPT's internal data. And unless Holmen can correct it, that's a violation of the GDPR, Noyb claims.
> "While the damage done may be more limited if false personal data is not shared, the GDPR applies to internal data just as much as to shared data," Noyb says.
In the context of an LLM, what does it even mean to say you have "deleted internal data"? It doesn't really even make sense, is certainly not really possible the way LLMs work, and while I totally agree damaging hallucinations are a serious problem, the problem with hallucinations isn't that they "internally store" false data, it's that they semi-randomly mix-and-match data in ways that can cause false information to be displayed.
Relatedly, this article doesn't touch at all on how this false information came to be reported in the first place. Was there false info in the training data, which would obviously easily explain it? Or did it just "mash up" true bits of different information in a way that caused it to report a falsehood (i.e. a hallucination)?
It's not the victim's fault that the data is stored in a manner that isn't convenient for OpenAI to delete. The information is still there, just compressed.
> Relatedly, this article doesn't touch at all on how this false information came to be reported in the first place.
And what was he wearing?
> You can't "remove" it
Your honor, my machine here just does crimes, it's how it works. I can't just "remove" the criminality. Something something temperature.
And whose fault is this?
If you had a loom and yarn, you could load and run it in a certain way to create a blanket with an offensive image. That doesn't mean the loom or yarn has the compressed knowledge of that image.
Think of the cost of being compliant if this holds in court.
One could always argue that the weights are just an encoding of a lossily compressed representation of the training data, much in the same way a JPEG is a lossily compressed image and those are clearly covered.
Then they must face the consequences of breaking the laws. Doesn't seem super complicated unless you think that having a certain amount of money exempts you from any inconvenient laws.
I strongly suspect the EU will not find that companies must retrain models in response to GDPR requests because 1. It would destroy Mistral, one of the few beacons of hope in EU tech and 2. It would put Europeans and European companies at huge disadvantages in not being able to use these tools.
As an American software engineer I would only benefit from Europe banning these products as it would put a large segment of the low cost labor force at a huge disadvantage relative to me, so I am not really bothered by the possibility of them doing it. It would be a very interesting experiment.
Just demand that OpenAI markets ChatGPT for what it is: a fun toy that sometimes provides information, but you can't tell when if you're a layman. OpenAI can still add that it's improving, if they want a positive marketing spin.
However, this is the EU. Lots of jobs added to write long legal texts, translate them to all 24 official languages, consultants to help with implementing the laws, etc. The more complicated, the better the employment numbers.
This is a Nathan Fielder-esque suggestion. Maybe they can start marketing cigarettes as fidget toys too, to avoid consequences when people die from the the off-label use of smoking them.
I think it would be better to demand the tobacco companies explain what they are selling: a fun toy that sometimes doesn't kill you.
Oh, wait. They already do have that demand on them.
This is how poppers work in most countries, now; initially, they were 'tape cleaner', but as that has become increasingly non-credible with the decline of tapes, they're usually now 'air-freshener' or similar.
One of the news websites says ChatGPT printed this: "He was the father of two young boys, aged 7 and 10, who were tragically found dead in a pond near their home in Trondheim, Norway, in December 2020."
That same article later points out: "Mr Holmen has three sons, and the AI got the ages of them roughly right, suggesting it had some of his personal information."
I think it might be a good idea to help users report these facts they might not like, but I don't think it's newsworthy.