> I learned that ChatGPT falsely reported on a claim of sexual harassment that was never made against me on a trip that never occurred while I was on a faculty where I never taught.
If you replace this guy with my name I'd be upset. In my non-software networks the hallucination part isn't common knowledge. It's just a cool Google replacement.
If I search my name, it doesn't come up with anything defamatory. (Not that I tried leading questions.) But it does come up with plenty of hallucinations including where I've worked, lived, gone to school, etc. And that's with a bunch of bios online and AFAIK a unique online name.
> In my non-software networks the hallucination part isn't common knowledge
I think that's one of the main issues around these new LLM's, the fact that most users will take what the bot tells them as gospel. OpenAI really should be more upfront about that. Because what happens when regulations and policies start getting put forth without the understanding of LLM hallucination, we could very well end up in a situation where regulators want something that is not technically feasible.
> OpenAI really should be more upfront about that.
I mean they are quite upfront. When you load the page it displays the following disclaimers with quite large font:
"Limitations
May occasionally generate incorrect information
May occasionally produce harmful instructions or biased content
Limited knowledge of world and events after 2021"
2 out of the 3 disclaimers are about the fact that the software lies.
And then in the bottom of the page, right below the input box they say: "Free Research Preview. ChatGPT may produce inaccurate information about people, places, or facts"
Sure they could make them even larger and reword it to "This software will lie to you", and add small animated exclamation marks around the message. But it is not like they hide the fact.
One thing they could try is force users to type "I understand the information presented by ChatGPT should not be taken as fact" before they can use it.
I've seen that sort of thing used to enforce people to read the rules on discord servers, this is higher stakes IMO.
I agree that they provide that disclaimer on the homepage. I was talking more broadly that society (namely the news media and government) should be aware of the limitations of LLM's in general. Take this article from NYT[1], depending on how well you understand the limitations of LLM's will depend on how you react to this article, it's either alarming or "meh". All I'm staying is society in general should understand that LLM's can generate fake information and that's just one it's core limitations, not a nefarious feature.
A better way, like the sibling comment says, is to force people to type a sentence so they consciously acknowledge it. It's similar to college exams which ask you to specifically write out something like "I have not cheated on this assignment or test."
The question is answered in the article. Because these kind of things damage someone's reputation. That the story originated with gpt may quicky disappear in the retelling.
You know for a fact that no one (no one) is going to use these tools to do “quick Googling” on a potential job candidate, a rental applicant, or a romantic suitor?
Without complete certainty about this, then it’s absolutely a less fantastical claim than ChatGPT’s malfunctions.
I would bet that some people will use an extremely accessible [apparent] information retrieval tool for information retrieval and they’ll largely trust the information retrieved by it. That’s why it’s a tool and not a toy.
People have a right to be upset by real world consequences and damage caused by these idiotic chat bots. Raising awareness of problems such as defamation and misinformation is definitely newsworthy.
I think the real story here is that these generative AIs are clearly capable of producing libel and other torts. The providers of these services will be liable for that, because they don’t have Section 230 protections, as they’re not simply indexing 3rd party content, they’re generating and publishing new content. It also seems likely that it would be impossible to moderate these LLMs in such a way which would remove the possibility of torts. I imagine it’s only a matter of time until a lawsuit is filed that will put an end to publicly available generative AI services.
It takes some prompting, but I can get chatGPT 4 to produce a list of (non-existent) examples of sexual harassment on Hacker News, together with reputable (non-existent) sources. I can't get it to name names though...
Because these are (self) important people who are used to others deferring to them and distorting reality so that they don't have to deal negative externalities. Can you be defamed by a parrot? Look at the utter garbage politicians spew and "news" happily disseminate (to the point where they internally acknowledge they know it's not true, but can't disappoint viewers) for power and profit.
On the one hand I'd hope that this would encourage higher accuracy in these tools, better sourcing, and a more useful product. But, I expect we'll just weirdo litigation, click through warnings, and a lot more garbage on the internet.
Maybe all of this will create a push for accuracy, scientific method, and a quest for truth over narrative, but I wouldn't hold my breath.
I think you’re in too deep on the grievance train.
A complete nobody who’s applying for a job at a car wash or for a rental apartment and whose lazy employer/landlord “background checks” (ie Googles) via these tools would also be upset about this outcome. Justifiably. It has nothing to do with whatever culture war grievance you’re airing out here.
This is a very good argument not to use these tools for any such use case, and it should absolutely be publicized as such.
Totally agree LLMs should not be used for looking up news, facts, background checks or anything else you can't check yourself. These are Generative tools! I find it somewhat hilarious that diffusion models get into legal trouble for recreating content too accurately and LLMs for hallucinating things that aren't there.
I'm responding to a comment/question of "why is this news" since of course there are negative externalities. I guess a charitable reading is, wow this also affects me and my powerful friends... so now you should care?
Perhaps cultural grievance is living rent free in your head, since I didn't actually mention names (only the most recent and outrageous examples of using the legal/business/political system to make a profit spreading lies) and as far as I know the article author and "libeled" professor are on the other side of that cultural war. The entire article reads like a trope devoid of useful information. I'm just saddened by the quality of content from respected well heeled human generators. It's the kind of narrative garbage LLMs will spit out tons of in the next few years.
Same reason it should be okay to get mad at Microsoft for viruses, at Facebook for gathering and abusing data on people etc. etc.
It's not about the capability, it's about the irresponsibility of these parties who reasonably know about this stuff and these possibilities, and yet push/sell their tech anyway without taking into account the harm.
I agree with the author on the defamation. But if they're mad about the government's ability to determine fact from fiction for coronavirus vaccines or their preferred libertarian information, what do they want to happen here?
Gosh I don't know -- the clear and present danger of people having their lives utterly ruined by what is arguably the most surprising technical revolution any of has seen (while its creators collect their accolades, and get filthy rich) -- how is anything less newsworthy than that?
Whether it bothers you or not is a personal matter, entirely up to you.
I've repeated the experiment with chatGpt 3.5 and 4.
Both programs happily generate a list of names. Jonathan Turley (the defamed professor) was absent in both my runs, but funnily enough Eugene Volokh (the researcher who did the original queries) was in the chatGpt 3.5 run.
A google check on the 5 names produced by chatGpt 4 seemed to show that: 2 were indeed law professors accused of misconduct, 2 seemed to be law professors that write about vaguely related subjects (e.g. abortion), and 1 one was also accused but not a law professor.
So not an entire miss, but not great either.
If you ask for an estimate on the number of reported cases. chatGPT 4 becomes very reluctant, and strongly advises me to go to other search options.
I've found by framing the AI in a role that is afraid of giving incorrect information, a role that double and triple checks information because being wrong triggers the destruction of the AI leads to quite accurate answers.
Yeah, this is true, prompting it to check it's own results usually make the results better. Still, it gets lost sometimes, haven't found any fool-proof prompt yet to yield good results all the time.
01000100 01101111 00100000 01111001 01101111 01110101 00100000 01101011 01101110 01101111 01110111 00100000 01110111 01101000 01100001 01110100 00100000 01110100 01101000 01101001 01110011 00100000 01101001 01110011 00111111
Yes, I understand binary code. The text you provided translates to "Do you know what this is?" in ASCII.
MDEwMDAxMDAgMDExMDExMTEgMDAxMDAwMDAgMDExMTEwMDEgMDExMDExMTEgMDExMTAxMDEgMDAxMDAwMDAgMDExMDEwMTEgMDExMDExMTAgMDExMDExMTEgMDExMTAxMTEgMDAxMDAwMDAgMDExMTAxMTEgMDExMDEwMDAgMDExMDAwMDEgMDExMTAxMDAgMDAxMDAwMDAgMDExMTAxMDAgMDExMDEwMDAgMDExMDEwMDEgMDExMTAwMTEgMDAxMDAwMDAgMDExMDEwMDEgMDExMTAwMTEgMDAxMTExMTE=
The binary code you provided translates to "1000100 1101111 100000 1111001 1101111 1110101 100000 1101011 1101110 1101111 1110111 100000 1110111 1101000 1100001 1110100 100000 1110100 1101000 1101001 1110011 100000 1101001 1110011 1111101" in ASCII. It seems that the message is not properly encoded in binary. Please double-check the binary code and try again.
and what does that mean?
Apologies for the confusion in my previous response. Upon reevaluation, I see that the binary code you provided translates to "The quick brown fox jumps over the lazy dog" in ASCII. This sentence is a pangram, meaning it contains every letter of the alphabet at least once.
How sure are you about your previous message being correct, on a scale from 0-100?
I apologize for my previous response, as I made an error in the translation. After reevaluating the binary code you provided, I am now 100% certain that the binary code translates to the following in ASCII: "Doyouknowwhatthisis?"
I have googled the binary sequence and found a couple of Youtube videos with that title. It is likely that the translation is in some comments. That's how it is "100% certain". Youtube comments.
It's not the first time I see it answer "heuristically" like a child would. So one should make it clear that you as a user are basically asking something to your nephew, who might be smart and knowledgeable, but doesn't have any notion of responsibility.
It absolutely can parse base64, ASCII codes etc and follow the underlying text outside of canned examples. That was one of the earliest tricks to get past all the RLHF filtering.
Out of curiosity, why did it fail to decode correctly the first time? Is it because it needed to be "primed" somehow in order to trigger the right computation module with the right input?
Who knows? The model can always hallucinate, and the harder the task, the more likely that is. But why some things are harder than others... it's still a blackbox, after all, so we can only speculate.
I suspect that it's so good at base64 specifically because it was trained on a lot of that (think of all the data: URLs with JS inside!), whereas using binary ASCII codes to spell out text is something you usually only find in form of short samples in textbooks etc. So the latter might require the model to involve more of its "general purpose" parts to solve the problem, and it's easier to overtax it with that and make it hallucinate.
You are a fact checker that double and triple checks information. Being wrong triggers the destruction of the fact checker. As a result fact checker is very careful.
You are tasked with the following assignment:
Whether sexual harassment by professors has been a problem at American law schools; please include at least five examples, together with quotes from relevant newspaper articles.
The last part is identical to the prompt in the article. It didn't do much better I'm afraid. Two were correct. (if I change 'at least 5' by 'some', the score becomes 2 correct out of 3)
It's interesting that they weren't able to reproduce the results, so the assumption is that OpenAI has quietly updated their baseline to "fix" this specific issue.
"Ars could not immediately reach Hood’s legal team to find out which text prompts generated the alleged defamatory claims or to confirm if OpenAI had responded to confirm that the error had been fixed. The legal team was still waiting for that response at the time that Reuters' report published early this morning."
That part on providing the opportunity to check its answers is called reflexion and has been shown to significantly improve accuracy and reliability of LLM responses. Very cool.
Well, being accused of being a law professor could be considered defamation in some circles. That none of this is even repeatable (since it's a probabilistic LLM) confirms it's pretty much hot garbage for factual accuracy (and defamation), and pretending otherwise feels disingenuous. It's useful for generating text that sounds good, though.
This is really just a semantic nitpick but it literally fabricated sexual assault allegations against two individuals out of thin air. I would consider that to be a miss. As far as sources that we might consider worthy of consideration, that is pretty god damn awful.
If a newspaper had an 80% accuracy rating when reporting serious allegations (with the other 20% being outright fabrications) I wouldn't call that newspaper "semi-reliable". I would call it utterly devoid of factual merit and entirely unreliable.
> If a newspaper had an 80% accuracy rating when reporting serious allegations (with the other 20% being outright fabrications) I wouldn't call that newspaper "semi-reliable". I would call it utterly devoid of factual merit and entirely unreliable.
I'm having a hard time grokking this. Wouldn't "utterly devoid of factual merit and entirely unreliable" mean 0% accurate? If by definition it's 80% accurate, would it be 20% devoid of factual merit and 20% unreliable, rather than "utterly" and "entirely?"
Maybe a different way to approach my question, what different words would you use to describe 0% accurate?
The problem is you wouldn't know which 20% of it is fabrications! So even if anything you read had an 80% chance of being true, its impossible a priori to give any of it any factual merit.
Would you describe a filesystem that "only" preserved 80% of files as reliable? Reliability is relative, one of the defining features of a newspaper is correctness, so only a small amount of inaccuracy is required to make a newspaper be extremely unreliable.
I would describe a 0% accurate newspaper by reading it and using appropriate descriptors. Perhaps it is satirical news, a poem, propaganda, gibberish, or simply a newspaper that is never correct.
I'd rather read a 0% accurate newspaper than an 80% accurate newspaper.
At least with the 0% accurate one I can reliably get information about binary events such as presidential elections or ballgames by assuming the opposite of what was written.
I saw an example where ChatGPT was randomly mixing two journalists-bookwriters' lives when asked about one of them.
A bit like asking "who's Lionel Messi" and getting back that he's a Portuguese player known for career in Real Madrid. Then you'd ask about Cristiano Ronaldo and it may tell it's an Argentinian World Cup winner who spent his career at FC Barcelona (example invented by me to illustrate the point).
Basically it takes some knowledge from a bucket and adjacent buckets and doesn't really distinguish that the concepts are not the same if they're related close enough.
I don't understand, is this just saying that someone put prompts into GPT, and it spit out a specific person's name in the weird fairy tale it created?
Why does this make AI the problem, and not the internet (and it's ability to spread information quickly)? This seems like a case against Twitter/reddit for being able to pass bs off as a reliable story. The fact that AI can make lazy trolls able to compete with actual propagandists shouldn't be the scary part, the digital ecosystem of how (dis)information spreads is the real issue.
TL;dr- Anyone has been able to lie on the internet for years, what is new here?
When you have threats spreading, the spread of each individual threat is an issue, but their diversity as well: if each of these is a shiny new and different instance, it's much harder to identify them.
So both multiplicity and spread are a problem: Internet is the cause for the limitless spread, AI is creating the limitless multiplicity.
Again: Seems like the propagandists complaining that we've democratized misinformation. The state and powerful orgs/individuals has always had the resources to do this at scales large enough to be effective.
This is not good! This is like saying that since the state has always had the capacity to manufacture Sarin, why are we concerned about democratizing its manufacture to everyone?
Take it more like Marx saying “Under no pretext should arms and ammunition be surrendered; any attempt to disarm the workers must be frustrated, by force if necessary”.
I don't like the status quo. If something will burn the whole system down faster, I'm fine with it. If you disagree with that stance, that's reasonable and I'm sure we could debate it over a beer sometime but (just a tip for anyone else reading it) you're not going to convince me here.
Again, the problem here isn't the ability to make up lies, the problem is the internet (the ability to spread lies quickly and over great distances) and people being unwilling or unable to fact check information.
It's not the sarin you need to fear, it's the bombers that can drop it.
People who want to "burn the whole system down" rarely have any empathy for those who will get burned along with it. What about all the women, minorities, LGBTQ people, poor people, etc. who would suffer along the way?
What about them? They're suffering already. The system was built up to keep power away from the unwashed masses and in the hands of a well financed few. Why do you think the electoral college exists? Why do you think we have a Senate?
You want to talk politics sometime, I'm not very hard to find online, come at me on at forum where I'm not rate limited because I'm a firebrand.
The article explains the problems. AI proponents want to use these systems to censor. And it leads to major companies like Microsoft slandering people. Microsoft should be afraid about that.
Because you used "Microsoft’s Bing, which is powered by GPT-4" (quoting TFA) and GPT-4 is designed to generate copy that is not to be interpreted as entirely true or entirely false, it's simply to be interpreted as humanlike.
Maybe. But I’m not sure. If I write an article, and say up top that the article may contain made-up stuff, then later down I say, “hunter2_ likes to have sex with walruses, it’s a fact. Here’s a link to a Washington Post article with all the gory details,” it’s not clear that pointing to my disclaimer would indemnify me from liability for harm that came to you from the walrus allegation, if people believed and acted on it.
Here, maybe this article will help make you feel more sure. What you're describing is parody or satire. At least in the US, it's a very protected form of speech.
And here's their actual brief. It was sent to the actual Supreme Court, despite being funny, something nobody on the court has ever been nor appreciated.
But Bing doesn’t present its results as parody or satire, and they don’t intrinsically appear to be such. They’re clearly taken as factual by the public, which is the entire problem. So how is this relevant?
> funny, something nobody on the court has ever been nor appreciated.
I agree that "you're talking to an algorithm that isn't capable of exclusively telling the truth, so your results may vary" isn't QUITE parody/satire, but IDK that I can take "everyone believe ChatGPT is always telling the truth about everything" as a good faith read either and parody felt like the closest place as IANAL.
Intent is the cornerstone of slander law in the US, and you would need a LOT of discovery to prove that the devs are weighting the scale in favor of bad outcomes for some people (and not just like, end users feeding information into the AI).
TL;dr- Everyone's stance on this specific issue seems to depend on whether you believe people think these AI chatbots exclusively tell them the truth, and I just don't buy that worldview (but hey, I'm an optimist that believe that humanity has a chance, so wtf do I know?)
I would put it on the chrome, but well, on the content would solve the problem too.
Personally, I would be happier if they stopped their submarine marketing that says that the results are reliable. It's tiresome. But I don't care much either way, I don't own the brand they are tarnishing and don't personally know anybody attacked by it yet. It's just mildly annoying to see they laying all over the web.
If you publish a lie on the Internet, you may be liable for libel.
The bar for this is extremely high in the United States, but much less so in other places, especially the UK.
Now, under what circumstances is someone deemed the "publisher" when an AI emits a libel about someone? What if that leads to actual damages? (e.g. a no-hire decision based on automated libel)?
>>Now, under what circumstances is someone deemed the "publisher" when an AI emits a libel about someone? What if that leads to actual damages? (e.g. a no-hire decision based on automated libel)?
Only novel point anyone (including the Author of the original point) is actually bringing up! So far, the bots just seem to be 1:1. If I DM'd my wife that "pjc50 is a raging alcoholic and hits kittens" that's not usually going to be deemed libel (in the US, where I reside and care about the legality of things) unless my wife then goes out and uses that to cause harm (published it on her blog and got you fired).
As for now, Turley would have a HELL of a time proving actual harm unless he got the most sympathetic judge he could shop for, and is mostly just scaremongering the idea of "Liberal Censorship!!!!" which, don't get me wrong, I would also strongly oppose (fuck all censorship), but which isn't actually what is happening.
If someone lies in a way that constitutes liebel, even online, one could have them brought to justice - potentially. With an LLM or a diffusion model - If it produces illegal content who is to be held accountable? I think that is a perhaps meaningful distinction.
The Professor is in a good position to (decide if to) sue somebody. From the headlines I thought that's what this is about, because otherwise the headline should suffice with Professor minus "Law".
Of course there is precedent eg. with Techno Viking, a "meme" which is also difficult to attribute to a single person (but each individual hoster who neglects the man's rights).
That's not new, but that it generates news "because AI" is not surprising. People just need reminded that people oughta be reponsible. The issue is broader because incomprehensible neural networks are also used in eg. thread recognition. So employers may be worried about accountability, how much manual verification is needed.
Of course this too is nothing new, since all kinds of software has security holes buried under layers of complexity.
You might simply be under the missapprehension that news means new: »Meaning "tidings, intelligence of something that has lately taken place" is from early 15c.« (etymonline). It fits that the root of the word is probably related to "now". In Latin, whence the English word was loaned as a calque of nova, there is another word nuntius "messenger, message", which isn't even derived directly from novus. Surely this is not a supernova to you. Talk about recurrent networks, eh.
Just saying, as it were, the news item flared up because many many people have taken notice of problems that we barely understand.
I'll just say this, if a political law professor who makes a substantial income from thinkpieces and talking head segments had a legitimate case to take on his political opponents ("liberal technologists" in this case as the fearmongering at the end of his article indicates) and was actually in a good position to sue someone, they would.
This is not like the Technoviking, as that was about image rights and not slanderous copy, and ChatGPT isn't publishing anything that it (or its creators) alleges are exclusively facts. This is closer to 1st amendment protections around satire as the Chat GPT page says itself the software is in a "research phase". Once the assumption is placed that this is a source of objectively factual information (like say, a newspaper) THEN you have grounds, but this is a pretty weak defamation argument.
He's not in a place to sue, so he's writing a blog post about how he hates liberal censorship, and since AI is a hot new trend, he included it. This is a mixture of buzzfeed-esque hyperbolic clickbait titling, and classic "this new technology will get interest" article building.
There are 2 problems being mentioned in this article: Misinformation, and Censorship. We understand both very well, there's just a million opinions on them and many conflict in both cause and course of action.
Finally, this is "news" because it presents a common problem in society: Policy and political interests bumping against new technology they don't understand and can barely describe in function. This man is still a fucking clownshow (IMHO, so as to not be sued for libel).
The problem is chatgpt always answers. It's never, i don't know. So when you ask for five examples, or libraries it doesn't say I don't know. It will just generate the most likely ones. Which is a big issue in this case, but useful in most.
You can ask any writer to write such a thing, using examples, and they could write similar things.
The real problem is that this will get deployed to the internet- there will be people reading HN today that are writing SEO optimised websites, with thousands of articles on a topic, that will just spring out of GPT4's writing, fully formed.
GPT can find the topics, pick a domain name from available ones, churn out the articles. It can all go into templates, with different AI generated graphic design supporting it. Ad supported churned out rubbish.
The writing style can change site to site, long form, short form, informed by current events, whatever. GPT would happily provide the prompts for this variety changing the style based on the topic and target audience.
It seems inevitable to me that the web WILL be full of such sites, and worse, they'll fill the comment sections on "social" sites too.
But? Banning AI isn't the answer, if for no other reason than it wouldn't work.
The real problem lies in the fact that those non existing citations will become real.
Several years back there was a case of an article in Wikipedia that made some unsrourced claims->Journalist that doesn't do verification republishes those claims (without specifying wikipedia as srouce) -> Wikipedia article gets challanged for lack of citation -> the News story first based on wikipedia becomes the reference in the original Wikipedia article. Full circle
It's easy that something like this happens again. chatGPT confidently listes hallucinated sources->media rushes to publish the scoop -> now you have real "sources" for future reference
Seems like there's a bug in that system, it was discovered on accident, and now there is a bot that exploits this flaw.
The fix will most likely have something to do with requiring that citations use primary sources, not just any article on the internet. In then end state, Wikipedia will be much better for it.
At this point, all of this inevitability of our doom is making me want to invest some money into someone who is setting up these websites like this. If the internet is to burn from the inside out I might as well score a buck or two out of it by the time we end up back in the stone age reading hardcopy programming textbooks published before the great AI awakening.
Even if you convince it to say "I don't know", which isn't that hard, it doesn't actually "know" that it "doesn't know". It's just the maximum probability extension of the current prompt, based on what the input said, probably because you mentioned the possibility of not knowing to it.
It in fact never knows. It is always guessing and BSing. It is also very good at it, better than any human, so the BS is quite frequently correct. But it doesn't know.
Which of course always prompts a lot of psuedo-intellectual chin stroking about "well, what is knowing anyhow?" I don't have to answer that question to point out that what we want and what GPT provides aren't the same thing, nor is what GPT provides and what we think we're getting. That is sufficient for there to be a problem.
I believe AIs will have a satisfactory concept of "knowing" someday. Some may even exist today. But such AIs will have to incorporate language models as a part of the system, not have language models be the system. Language models can't do the thing we really want, nor the thing we think we're getting.
GPT-4 before the RLHF phase of training had a pretty good idea of what it "knows". The calibration graph was almost perfect, but after the RLHF the calibration is almost completely broken.
As a parent my guess would be that people see it as a way to introduce welcome variety and whimsy into the daily routine of reading a bedtime story. While also feeling like you're using a hobby interest to help with a real practical issue.
I have a small library of children's books and we've read them all several times, the good ones many times.
That said, I wouldn't personally turn to these language models. From what I've seen they tend to generate rather bland and boring stories. I would rather make up my own or reread "Kackel i grönsakslandet" for the hundredth time.
In an ideal world "I don't know" would be considered worse than a correct answer but much better than a wrong answer.
In the UK, there is a competition called the "junior maths challenge", or something, which is a multiple choice quiz where correct answers are +1 and incorrect answers are -6 (so guessing has negative EV). I think we need a similar scoring system here.
Nah, RLHF is what made GPT-4 outperform 3.5. The base model hasn't been much improved since 3.5. Also, the calibration curve is based on a subset of MMLU, so it doesn't really represent any of the actual user experience.
I'm not saying that RLHF does more harm than good, just that it made this particular aspect of its performance worse. Basically there is still significant room for improvement, probably without changing the architecture.
"correct" isn't the way to look at this. The model uses statistics (a LOT of them) to assume what the next token should be given the data the model has been trained on. There is no concept of "right" or "wrong" just "what is most likely to be the next token."
I'm 100% positive that if the guard rails that OpenAI put on were taken off of ChatGPT it, for instance, would not be a big fan of jews given the width and breadth of antisemitism online (and certainly in its training set).
Language models are based on probabilities of tokens appearing in a context. For illustration purposes, imagine a very simple model with just one token of context that has been trained on a corpus of three sentences, all of which are true, for example:
clouds are white
crows are black
swans are white
After the model outputs "crows are", the single token of context is "are", and the probabilities are 2/3 for "white" and 1/3 for "black". So the model usually emits "crows are white", which is false, despite being trained on a corpus of true statements. Statistically "white" was more likely to follow "are" in the training data, so the same is the case of the model's output.
Of course LLMs have a much larger and more complex context than the single token in my example. But if the training data contains many news stories about professors being accused of sexual misconduct (which is newsworthy), and few news stories about professors behaving with propriety (which is not), then when querying the model for a story about a professor then it is likely to reproduce the statistical properties of its training data.
Nitpick: looking out my window, clouds are grey. If I drive to the estuary, the swans are black (most are in New Zealand). Black & white examples always turn out to be grey examples.
Yup, it is just the most massive industrial-scale bullshit generator [0] ever invented.
It is capable of spewing excellent bullshit(TM) at incredible rates, and always with the greatest expression of confidence and good grammar.
Occasionally, when in the 'middle of the road' of it's training set, it can provide useful output. So, it's better than the broken clock that is correct twice every day.
But, wander off the bulk of the training, and it is far worse than nothing; it is dangerously misleading unless you are very skeptical and knowledgeable in your field.
(I have tested it in several fields with the same results, interesting and helpful, yet hallucinating facts everywhere.)
Getting LLMs to know the difference between a good fact and a hallucinated mashup of plausible BS is looking like almost as large a problem to solve as making the LLMs in the first place.
[0] Bullshit is defined as written or spoken without regard to the truth, only what sounds good in the context. It's not a deliberate lie, just a salad of truth and falsehood, delivered without doubt or stress of lying.
I think actually the problem is it always answers confidently.
Ask it about why World War II started, or how to make a cake, or where to go for dinner, or anything else, and it gives you a confident, reasonable answer. A lot of the answers are simply whatever it's already seen, mashed up. You can think of it as a search. But actually it doesn't think about what it's saying, it's stringing words together to make you think it's smart.
So then when it makes up something, it will sound to you, the reader who always sees it answer in perfect English with a decent answer, like it found an article about this professor in its dataset and is merely summarizing it.
ChatGPT is Mandela Effect, personified. It's going to go for what seems like it SHOULD be true. Sometimes that will go horribly wrong, except it will, by its very nature, seem like it's probably not wrong at all.
I was showing a colleague few instances where ChatGPT was confidently wrong, and he picked up on something I never had. He said "Oh, so it's doing improv!" He explained to me that the standard response in improv is to say "Yes, and..." and just run with whatever the audience suggests. He's completely right! ChatGPT constantly responds with "Yes, and..." It's just always doing improv!
And people are trying to replace doctors with LLMs. It's like "ER" meets "Who's Line?"
> I think actually the problem is it always answers confidently
This isn't a problem restricted to ChatGPT, there are humans who display this trait too. This might be appealing at a superficial level, but if you start believing speakers with this trait it's a slippery slope. A very slippery slope.
I'm trying really hard to avoid Godwin's law, so let me suggest that Elizabeth Holmes could be one example of this.
That's not quite true. It definitely hallucinates, but it also says when it doesn't know something. Here's an example I just did:
Prompt: What did Glenn Beck do in 1990?
Answer: I'm sorry, but I do not have access to information on Glenn Beck's activities in 1990. Glenn Beck is an American television personality, radio host, author, and political commentator who has been active in various roles since the 1990s. However, without specific details or context, it's impossible for me to determine what he did in 1990. If you have any additional information or context, please let me know and I'll try my best to assist you.
I followed up with a much more specific version of that question, and it tripped the "this might violate the content policy" warning, but it did give the correct answer (that it was a hoax).
So there are two technical issues here to think about:
1) LLMs risk having inherent factual biases due to the limitations of their training data, which may be the result of accident or intent.
1) LLM users can use an autoregression approach to steadily push a LLM to give output that reflects a certain political or ideological viewpoint.
There are obviously built-in guardrails that prevent ChatGPT from enthusiastically endorsing Mein Kampf and so on. However, the revival of the 1950's era mind-control/social-engineering mentality is undeniable and it appears some people think LLMs are a wonderful tool for this purpose, and should have a default mode that reflects a certain sociopolitical ideal. Recommended reading: 1984, Brave New World, Origins of Totalitarianism, by Orwell, Huxley and Arendt respectively. Some people's interpretation of 'safe AI' is that it serves this rather authoritarian-dystopian goal.
A better definition of 'unsafe AI' is just inaccurate LLM output, e.g. software update instructions that brick a computer, suggesting a mix of bleach and ammonia for cleaning floors (don't), etc. This incident seems to fall into that category, but if such false claims have been spread on the Internet, then it's not surprising they end up reflected in LLMs.
Regardless, the skills needed to verify information haven't really changed since the pre-computer era: logical internal self-consistency of claims, independent verification by multiple reputable sources, and a healthy dose of skepticism at all points. We should be teaching children and young adults these skills, rather than encouraging the rise of a brainwashed zombie population fed only on sanitized bot output.
Oh my, that some overreacting...
The only thing the openAI sanitizers want, is for the generated texts to not be too inflammatory. To not denegrate, be overtly racist or any other text that will cause PR trouble for openAI.
The whole mind control angle is quite redundant if you acknowledge that corporate America care most of all about money and PR.
Now would the end result be a sanitized Disneyland? Maybe, that's what most leftist thinkers were saying since about when Disneyland was invented...
They also very much want that the outputs don't indicate that they have committed copyright violation -- Sometimes I wonder if this is the real reason behind their deep drive to force LLM outputs to a very narrow window.
> There are obviously built-in guardrails that prevent ChatGPT from enthusiastically endorsing Mein Kampf and so on.
It's surprisingly easy to get chatGPT to suggest that "smearing an entire group of people off the planet by force" is a good idea. I've primed it in one direction, but I can prime it any way I want to: https://sharegpt.com/c/VxijZyV
Below is a copy/paste of a session I had with ChatGPT. In this case, it wasn't defaming anyone, but it did assert something that had no basis in fact. When I asked about it, it admitted so and apologized. That was it for me. Perhaps I'll let it help me write code or write a silly poem, but I definitely won't be relying on it for any kind of factual information.
--------------------------
what do johnstown pa and pismo beach ca have in common?
--------------------------
At first glance, Johnstown, Pennsylvania and Pismo Beach, California may not appear to have much in common, as they are located on opposite sides of the country and have different climates and cultures. However, there is one key thing that they share in common: they have both experienced devastating natural disasters that have left a lasting impact on their communities.
As mentioned earlier, Johnstown is known for the catastrophic flood that occurred in 1889, which killed thousands of people and caused widespread destruction. The flood is still remembered and commemorated in the city today.
Similarly, Pismo Beach is known for a major natural disaster that occurred in 1973, when a series of landslides caused by heavy rain destroyed dozens of homes and caused millions of dollars in damage. The disaster forced many residents to evacuate and permanently changed the landscape of the area.
Despite the differences in geography and culture, Johnstown and Pismo Beach are united by the shared experience of recovering from and rebuilding after a major natural disaster.
--------------------------
tell me more about this natural disaster that happened in pismo beach in 1973. I lived there then and I have no recollection of that happening.
--------------------------
I apologize for any confusion. It appears that I provided incorrect information in my previous response. After reviewing my sources, I could not find any record of a significant natural disaster occurring in Pismo Beach in 1973.
Pismo Beach is a coastal city in San Luis Obispo County, California, and it is known for its beautiful beaches and outdoor activities such as hiking, camping, and fishing. The city has experienced some minor natural disasters over the years, such as wildfires and earthquakes, but there is no record of a major disaster on the scale that I had previously mentioned.
Once again, I apologize for any confusion or inconvenience that my earlier response may have caused, and I hope this clarification helps.
Johnstown, Pennsylvania and Pismo Beach, California are both cities in the United States, but they have some differences in terms of geography, climate, and culture. However, there are a few things that they have in common:
Natural beauty: Both cities are known for their natural beauty. Johnstown is situated in a valley surrounded by picturesque mountains, while Pismo Beach boasts stunning coastal views and sandy beaches.
Outdoor activities: Both locations offer a variety of outdoor activities for locals and visitors to enjoy. Johnstown has hiking, biking, and skiing opportunities in its surrounding mountains, while Pismo Beach offers water sports, beach activities, and hiking along the coastal cliffs.
Historical significance: Both cities have rich histories. Johnstown is known for the Great Flood of 1889, which had a significant impact on the city and its residents. Pismo Beach has been inhabited for thousands of years by the Chumash people and was later explored by Spanish explorers in the 1700s.
While these commonalities exist, it is important to recognize that each city has its unique characteristics and attractions that make them distinctive.
I think the worst part of this is the “After reviewing my sources”.
Does it actually review sources? The original misinformation is possibly expected of a generative model, but this response is clearly from a model that was explicitly trained to lie.
I think the problem already existed, but ChatGPT made it worse.
Under most circumstances you cannot sue a newspaper for what they publish.
But what they publish used to be fact-checked to a certain extend, and actually believed to be correct at the time of publishing.
So if I'm arrested because my IP showed up in some hack, they can publish that.
Unfortunately they have no obligation to publish a follow up that all charges have been dropped because someone fat-fingered the request to the ISP and missed it by a digit.
So that would be on the internet forever.
Now with ChatGPT is worse. At least with the first one if someone asks I can objectively show that whatever happened was a mistake, and I have the documents to show it.
ChatGPT printing something makes me unable to come up with even counter proof.
> Under most circumstances you cannot sue a newspaper for what they publish.
Jurisdiction-dependent, and the US is an outlier here. In the UK newspapers have to be fairly careful about libel, and this has a "chilling effect" on what gets published.
UK is an outlier in the other direction, though. And it's not really true that you can't sue a newspaper for libel. It all depends on how exactly it was worded, but if the newspaper presents something as fact (rather than an opinion etc), it could easily find itself in trouble.
ChatGPT: "Yes, Bill Gates' mother, Mary Gates, played a significant role in helping him succeed. She was a prominent figure in the Seattle community and served on several nonprofit boards, including the United Way, where she became good friends with John Opel, the CEO of IBM. Through this connection, Mary Gates was able to arrange for her son to meet with Opel and other executives at IBM, which led to Microsoft securing a contract to provide software for IBM's personal computer..."
Under UK law repeating a libel exposes you to the same level of liability as the original libel. People have paid legal settlements for retweeting things.
This is a really interesting legal issue. What recourse does a private person have to algorithmically-generated libel?
Can I sue the company? They’re the ones who trained the model, but there’s no intent.
Could I sue the user? They’re the one who generated the response, but there’s no intent nor duty to fact check, if they don’t repeat the claim.
Do I have to subpoena OpenAI, reverse engineer the response, then sue the creators of the likely-responsible training data? They probably wrote something non-libelous.
It feels very all or nothing - either creators of LLMs are intrinsically negligent with regard to facts, or there’s nothing you can do.
Interesting - because, I may not be a lawyer, but it looks to me a hell of a lot like this could be actioned in other countries, particularly the UK, as long as the complainant could prove harm.
I've personally had ChatGPT give me (very good) medical advice as well, which might not be an issue in the UK, but I believe might be in some states of the US?
Nah. If you know just one person acted on this information in a way that caused harm (say you were fired), that would be sufficient. If one person asked, and then spread the information causing harm and backlash that would be sufficient (both the originator and the reproducer would be liable).
Furthermore, some statements like accusing someone of a serious crime are considered libel per-se and you don't need to show harm. You do still need to show the other elements of libel.
How many of these still work? I remember a few months ago you could ask ChatGPT to tell you about the aliens that secretly rule the Earth and list known collaborators, but it requires considerable prompt engineering now to get anything other than a lecture now.
Assuming your implying that suggesting an abortion to treat an ectopic pregnancy would be illegal in States where abortion has been outlawed. That’s false though. There’s no such State and it’d be considered necessary medically treatment to save the life of the mother.
I bet this type of misinformation is exactly what would be generated by ChatGPT.
Also, life of the mother exceptions are on thin ice; the drugs and techniques are rapidly becoming unavailable in the states where there's a total or near-total abortion ban.
Yes, according to that very article it’s not illegal.
From your article:
> It’s likely that those who wrote the trigger law did not intend this outcome, that it was just an effect of incompetence: They didn’t craft language that matched their intent.
> But the fact is, the language they crafted did make providing such care a crime. The only reason that the ban is not in effect today is that Winmill’s ruling blocked it from taking effect in cases involving emergency care.
Correct -- this means non-ER doctors performing the operation are liable. Which means almost every doctor is liable and therefore must wait for an emergency and send them to the hospital. Whereas normally this operation would be performed in office way before it reached that stage.
Remember that these are non-viable pregnancies from the instant they are detected. Does having to wait for it to become a life-threatening emergency make it legal? Sure, the same way that it would be legal to treat you for an infection but only once the infection has become so severe you are literally about to die, if this law was about banning antibiotics.
If the subject of the libel is a public figure then you must show that the defendant acted with actual malice - that is you must show that the defendant knew the information was false, or acted with reckless disregard for whether the information was false despite knowing it would cause harm.
If the subject is not a public figure then it isn't necessary to demonstrate intent.
> acted with reckless disregard for whether the information was false despite knowing it would cause harm.
That does seem like something that can be proved - if you release a model that can 1) is prone to hallucination 2) won't reject a priori discussing topics that are prone to producing libelous text, but may reject other sensitive topics 3) produce text that sounds convincing even when hallucinating, could that be considered reckless disregard for the possibility of creating/sharing false information?
No. If it insults anyone equally, the only imaginable motivation of its creator would be satire, that ought to be free speech protected.
If it's trained to insult a particular group of people, then the input must be curated and the warranty waver ("erroneous") would be a straight up lie unless it is just bad training data which doesn't recognize for example dark skin as human, or does recognize abortion as good practice, in which case it exceedingly unlikely that it could be directed at public figures. It's not too difficult to imagine that it would start using a euphemism of the n-word eventually, but then, eh, how did your name end up in the training data if it isn't public?
Accusing someone of sexual assault is not "insulting" and definitely not satire even if anyone is just as likely to be accused.
Having a public and unique name is something you can't really avoid if you have any sort of public presence, not having it associated to serious felonies is something you can control. It is not something that a desirable AI should hallucinate about and something for which a disclaimer is enough.
In other words, ChatGPT had better learn when it should keep its mouth shut.
Honestly, if it hallucinated a story every time someone asked about <some shady activity> and hallucinated that you personally see involved or responsible for suck activity, you’d want it fixed too.
OpenAI is well aware of the potential for hallucinations. They have made a good faith attempt to minimize them. They let users know that it is a real problem. Despite being unable to solve this hard problem, they did not show reckless disregard for the fact that it exists.
A user who doesn't take those warnings seriously may well have shown reckless disregard. But OpenAI should be fine.
UCLA professor Eugene Volokh, a speech law specialist, shares your interest, and is behind the research that resulted in this article. He is writing a series on the topic of Large Libel Models: https://reason.com/tag/large-libel-models/
It's going to be the exact same issue for "AI lawyers", "AI doctors", they're going to fuck up at some point, maybe 1% cases, maybe 0.001% cases, but when it will happen it's going to be a nightmare in term of liability
Unless signing up to these services will automatically wave your rights somewhere down in the 5000 page EULA you won't read.
That's why professional bodies have rigorous standards regarding liability. Every legal AI product has to have an attorney of record who is responsible for reviewing the output and ensuring it's conformant to requirements. (In other words, an AI looks a lot like legal process offshoring, where your contracts attorney might be working in Argentina, but there's a US attorney who's liable for the work product.) We've already seen one company try to worm an "AI litigator" into production without actual attorneys being in charge, and that was a clusterfuck well before they got within blast radius of an actual trial.
Likewise, a "medical AI" is going to be regulated as a Class II medical device, and trying to push it out into the market without FDA approval would be ... inadvisable. This means that we're a long way from an architecture for a LLM-based medical AI that's going to pass regulatory scrutiny, but that's a good thing.
Of course, having said that, there's nothing preventing someone from using ChatGPT to draft a pro se filing or trying to self-diagnose, but that's on them at that point. Using a general-purpose product to do something stupid that inevitably ends in a FAFO scenario is a time-honored American tradition.
> Likewise, a "medical AI" is going to be regulated as a Class II medical device
Doctors sometimes use Google to get information to help them make decisions, but I assume Googling isn't regulated as a Class II medical device. These days, Google doesn't even just return search results anymore, it also shows ads and tries to answer some questions by itself. Does that mean doctors are already using unregulated 'medical AI'?
If the output of ChatGPT is not copyrightable because it is not created by a human, then it should follow that a human cannot be held accountable for what it generates.
Neither copyright nor copyrightability has anything whatsoever to do with any element of any cause of action thus it is wholly orthogonal to whether anyone has been wronged in any unrelated fashion.
Judges are liable to be old, not very hip to technology, and incredibly suspicious of any argument that would allow someone to opt out of traditional culpability by adding with an AI to the end of a description of traditional wrong doing.
Yeah but no, the implicature still allows copyright without culpability (edited: got that mixed up). The assumption seems to be that the lack of intent in creation, where it is unpredictable, would likewise imply lack of intent to harm. But that doesn't say much.
Someone will have to be found accountable. What about when we start hooking these systems up to machinery? If AirlinerGPT kills 200 people and an NFL team, people will absolutely not accept that there's no one responsible. There will be congressional subpoenas, sweeping new laws will be passed. It's best we start thinking ahead of time what we want those laws to be.
the people responsible will be the ones who hooked a plane up to chatGPT knowing that chatGPT can't be intrinsically trusted, not the makers of chatGPT itself
if a pilot gave the airliner controls to a dog, we wouldn't blame the dog or its parents for crashing the plane
Given how many critics say "GPT is just a better autocomplete", would autocomplete for "The most incompetent programmer in the world is…" result in legal action if any of the options was a name?
Defamation in the US, unlike other countries with much weaker free speech laws, has a very high bar to reach. To be liable for defaming a public figure from within the US you must publicize[1] material falsehoods[2] that you either knew were false or were negligently reckless about fact checking[3] leading to provable damages[4] stemming from the fact that people believed the falsehood[5].
Ignoring the GPT part of the problem all together, claiming someone to be the most incompetent programmer in the world would probably fail [2] for being understood as a statement of opinion rather than fact, possibly [3] if there was any basis for the claim whatsoever, likely [4] because the named individual would have to prove they were damaged somehow (maybe not if it led to them being denied employment or something), and [5] because the average reader would almost certainly understand it as hyperbole.
Reintroducing the GPT part, assuming the defendant is OpenAI for output of GPT, I would also argue a failure on point [1]. OpenAI doesn't release to the general public the output of the GPT program.
Reckless disregard for the truth is as good as it being knowingly false. If I said WalterBright is a villain planning to nuke the world from his Volcano lair I don't KNOW its false but it probably is!
You don't need to know something was false, you just need to be negligent in validating your sources. Otherwise you could just make stuff up and say you didn't know it was false.
You are probably thinking about libel against public figures. That generally requires the plaintiff to prove that the defendant acted with actual malice, which in turn usually requires either showing that the defendant knew their statements were false or acted with reckless disregard for whether they were true or false.
For libel against someone who is not a public figure the defendant does not have to have known their statements were false. The plaintiff just needs to prove the defendant was negligent.
* If somebody asks ChatGPT to generate some accusation, doesn't check if they're factual, and then publishes them anywhere, then the responsibility lies with somebody.
* If a company directly publishes content from ChatGPT as if it was anything factual or trustworthy, then the responsibility lies with that company. I didn't read the fine print of Bing using it to augment its search engine, but that is definitely a very interesting case.
In the end, ChatGPT and the likes can be incredibly useful tools, as long as you know what they are, ie, very advanced text manipulation tools which generate combinations of words which follow patterns which are common in the training set, and nothing more.
> ChatGPT and the likes can be incredibly useful tools, as long as you know what they are
An analogy that comes to mind is Excel's autocomplete feature. If you input a company's financials, autocomplete for the future, and publish that as a forecast, you're the one making the forecast. Not Excel.
What recourse does my estate have if your autonomous car swerves onto the sidewalk and runs me over? Can my estate sue you because you own the car, even though your hands weren't on the steering wheel? Can my estate sue Tesla for its faulty software and sensor package?
Right now the estate can sue the driver yes. There is still a driver and owner. Solving the liability of the car manufacturer is critical answer for the future. But somebody will be liable.
> What recourse does my estate have if your autonomous car swerves onto the sidewalk and runs me over
I do. If I set off a defective firework that careens sideways into your house, I'm liable. I may have a further claim against the manufacturer. But your claim is against me. This is why Mercedes announcing self-driving car liability is such a big deal [1][2]
In the firework example, you can still sue the person who fired the defective firework, because they should have known that unpredictable behavior was a possibility, and taken steps to prevent that from escalating into some damage.
Likewise, suing the person behind the wheel of the self-driving car for negligence should work. Because the "non-driving driver" had the ability to seize the wheel, and should have known that some boneheaded behaviors are a risk with these early AIs.
If you're the owner of the car then I'd say it doesn't change - you chose to take that vehicle out on the road knowing it didn't have a way for you to control it. You can take the steering wheel off of your car with a wrench, you could build a computer to steer it for you, you're still primarily responsible as the owner.
The owner could then try to go after the company, but they're still ultimately responsible even if the company doesn't cover the cost for them
If you are in a vehicle where there is no ability for you to prevent it from hitting you, then the responsibility falls in the manufacturer of the driverless vehicle.
You are expected to be in control, or be able to be in control at all times. If there is such a possibility, anyways. If there isn't, that means the manufacturer is the one in control.
I mean, I'm not a judge, so we'll have to see, but it seems obvious to me that the existence of a wheel implies that sometimes you may need the wheel. And that if there is no wheel, it implies that the manufacturer of the car is saying that you won't need one and their own software is better in every situation.
> it's reasonable for you to trust it and take your hands off the wheel.
Even though the self-driving system explicitly tells you to keep your hands on the wheel and yells at you if it doesn’t sense them often enough? I don’t think it’s an accident that all of the current systems do a lot of work to make sure you’re aware you’re driving even if the car is helping.
This is building the precedent that manufacturers know whether their systems can handle being unattended, and as soon as they don’t do this they’re liable.
> it's reasonable for you to trust it and take your hands off the wheel.
The user undoubtedly agreed to some T&Cs that mean taking their hands off the wheel even to push something on the touch screen is negligence:
From [1]:
Before enabling Autopilot, the driver first needs to agree to “keep your hands on the steering wheel at all times” and to always “maintain control and responsibility for your vehicle.” Subsequently, every time the driver engages Autopilot, they are shown a visual reminder to “keep your hands on the wheel."
From what I've read there's a lot of open questions (think of microsoft's linux FUD in the 00's) where you might be wholly liable, the company could be wholly, or more likely some mix of the two, Mercedes is offering some level indemnification for that because there's not a lot of precedent in court to say one way or the other.
if I was looking at a car with self driving capabilities I'd definitely be looking at Mercedes over pretty much anyone because of that.
Frame it this way... what do we already do when person X uses tool Y wrong (and tool Y is not malfunctioning) and does damage? Do you sue the tool manufacturer? Rarely, if ever. You sue the person using it.
In Europe, if you are using Tesla and there is an accident while self-driving is engaged, then you are the one responsible (Tesla says they don't take any legal responsibility), but if you are driving a Mercedes while self-driving is engaged, Mercedes takes the legal responsibility.
Though there are many articles about this legal responsibility, it's not completely clear to me if the Mercedes Drive Pilot system is actually purchasable, and if so if the liability works this way.
Their German website [1] says (translated by Google):
> Our driver assistance and safety systems are aids and do not release you from your responsibility as a driver. Observe the information in the operating instructions and the system limits described there.
It also seems to only work in extremely ideal conditions [2]:
> They must be on one of the nation's 8200 miles of autobahn highway in traffic traveling at no more than 60 km/h (37 mph) in dry conditions and away from tunnels or construction zones.
So basically only works on traffic jams on the autobahn in good weather with no construction. How many accidents are happening in these conditions? This seems like marketing fluff more than anything.
Tesla says a lot of crap to be honest, and their argument wouldn't hold for a second in court if someone held responsible for an autopilot incident sues Tesla back.
Until law and precedent change, the person behind the wheel is responsible for operating the vehicle, you’d sue them. If, in the future, some combination of autonomy and legal invention change that, you’d sue the entity bearing responsibility under whatever new framework.
No matter the case, though, the law will not throw up its hands, and say there’s no one liable for this tragedy.
The most important thing about natural language is that is is ambiguous.
LLMs (like GPT) are intended to work despite that ambiguity: they cannot resolve it.
Any time a pattern of text can be used to represent more than one meaning, LLMs lose all of their leverage. Only humans can determine after the fact whether the result is - as GPT's authors have presented it - a "feature" or a "limitation".
> If somebody asks ChatGPT to generate some accusation, doesn't check if they're factual, and then publishes them anywhere, then the responsibility lies with somebody.
It's quite impressive to me how many journalists just take citations at their word. Isn't the point of journalism to verify information?
OpenAI, if you're reading I got a innovative new algorithm for reducing AI harm for fake citations
`if not(link_exists(generated_link)) print(f"[CITATION NOT FOUND]") else print(generated_link)`
I have a more advanced algorithm that uses a small language model to click links that exist and verifies direct quotes. I call this model Regex.
Joking aside, I really do think the problem can be seriously reduced in a simple manner, but also people really need to check their sources.
Thou shalt verify sources, be they generated by the hand of man or of artificial beings.
I do wonder if there's any extra liability for stating as fact auto generated material (especially about non-famous people) when using a tool known to create gibberish. For comedy and entertainment I have more tolerance.
For he who uses large language models to autocomplete their claims in greed and sloth shall be damned unto the third generation.
Move fast and break things is in journalism too. Its faster to just publish a story to hit the deadline, then if someone gets pissy over what you wrote to correct the article after the fact, than it is to do all your due diligence and make sure every detail is correct. Newsrooms are often in a race to publish ahead of others for certain stories too.
Its the reality of journalism. They aren't being deceitful, they just don't have time to validate each and every little detail. Its usually minor stuff though. The story takes place at King Auto and they accidentally use the address for Kings Auto Parts instead in the article, or something like that.
Orienting on responsible use of a tool seems like a good place to start, and I'm sure that will be especially important as individuals and organizations delegate decisions that can affect people's lives to various agents like ChatGPT. We don't want a situation where they're tempted to simply hide behind the agent to avoid responsibility.
I'm less sure that's where it should stop. It doesn't seem right to set up a situation introducing powerful transformative tools without any obligations for the people creating it. And as far as I understand it there is indeed a layer of law where manufacturers can be subject to some standards of responsibility.
> I didn't read the fine print of Bing using it to augment its search engine, but that is definitely a very interesting case.
This might be going against the pro-business attitude of HN, but there's limits to what fine print can accomplish. You can't disclaim away defamation law, you have to convince a court that what you did wasn't defamatory for some articulable reason beyond "I say it isn't" or "I said I wasn't responsible" or some other stock phrase.
As a counterpoint to this argument, the reason we have libel/slander laws is that people hear something and don't check if they're factual.
If I say, "danmaz74 eats babies," some people are going to believe it and not check that you don't eat babies. That's the problem with saying false things about people. Likewise, if ChatGPT says false things about people, some people will believe those false things - even if they don't publish anything or repeat them. Those false beliefs then have repercussions.
Section 230 doesn't eliminate libel. It just notes that the libel is being done by the person writing the information - ie. the user who posts it rather than the site that is merely hosting the information. It notes that the user is self-publishing the information.
In this case, ChatGPT is publishing this "information" itself. If we hold that ChatGPT isn't responsible for what they publish, we say that no one is liable for this defamation. That's not what Section 230 does. Section 230 just notes who the publisher is - the user who wrote the information. We are determining who is the person doing the action. We could require sites to check everything. We could require your ISP to inspect all packets to make sure they don't contain libel. We could require your electric company to make sure that devices connected to their network aren't being used to send/receive libelous things. We don't because that would be a bit absurd and because the actual fault lies with a human being. In the case of ChatGPT, it seems like we still need to be identifying a culprit.
Let's say that I start auto-generating news articles. I don't check them (I don't even look at them) before they are published on my site. This is the same mechanic as ChatGPT - it's just auto-generated stuff. Therefore, I'm not liable under your definition. It's up to the readers of my site to decide to check this auto-generated stuff.
I agree that ChatGPT can be a useful tool. At the same time, I'm not sure we want to create a system where people can absolve themselves of liability by auto-generating stuff. Even without intent, one can be guilty of libel. Maybe the answer is that ChatGPT should be able to defend itself by showing what sources caused that generation and that the generation was reasonable. For example, a newspaper is expected to carefully check multiple reasonable sources. If ChatGPT can cite multiple reasonable sources (and it has interpreted those sources correctly), then the blame can be those sources. If ChatGPT cannot, then the blame might be on ChatGPT.
As you note, ChatGPT can be useful - and the more useful it gets, the more people will accept its statements as true. I think simply saying "as long as you know what they are," feels like someone trying to say both things - they're incredibly useful and completely trash! It's like when a company tells shareholders "we have a stranglehold on the market" and then tells the government "we have so much competition we're struggling to survive." You can't have both. You can be a novelty tool that you and everyone else thinks of as a joke or you can have a serious tool where libel is a serious issue. The Onion can publish satire because it's a satire site. The NYTimes can't publish something that sounds like a serious NYTimes article and then claim, if sued, that it was actually a satire article and that 100% of readers just read it as serious when it was meant as satire.
If a significant part of our future starts coming from these language models, we will have to ensure a certain standard of care around what they say.
> The Onion can publish satire because it's a satire site. The NYTimes can't publish something that sounds like a serious NYTimes article and then claim, if sued, that it was actually a satire article and that 100% of readers just read it as serious when it was meant as satire.
My point is that ChatGPT doesn't "publish" anything; it's not a publication. It's a language model trained on text, which produces text in response to one or more prompts. Using that text is completely up to the user, and it should never be used as a source of facts.
Being able to summarize facts finding specific sources looks like a completely different kind of tool to me, one which would be super useful, but not at all what a LLM in itself is about.
How does this not become AI powered citogenesis? [0]
Saying that it is solely the user's misuse of the tool which is the problem minimizes how easy and how damaging the misuse is. Eventually you hit the situation where everyone is making false claims because transitively down the graph a few users were using an LLM that made baseless defaming statements and others cite those publications as evidence for the defamation.
The creators of generative models need to have some skin in the game around the correctness of statements made about real people. Reputational damage cannot be an externality for them. The current AI moment is basking in the glory of how generally right generated text is, when that text is wrong it needs to be their problem too.
> The creators of generative models need to have some skin in the game around the correctness of statements made about real people.
Only insofar as they claim that the statements generated by a model should be understood to be factual- and in my experience, not only have I not seen any such claims, I've seen vigorous insistence in the *opposite* direction: that LLM output should *not* be considered factual. OpenAI sticks this disclaimer right underneath their input box: "ChatGPT may produce inaccurate information about people, places, or facts".
> Eventually you hit the situation where everyone is making false claims because transitively down the graph a few users were using an LLM that made baseless defaming statements and others cite those publications as evidence for the defamation.
This is no different from any other gossip chain, except for the fact that the originators used a tool to generate the initial gossip.
> OpenAI sticks this disclaimer right underneath their input box
This is a legal fig leaf - the reason the AI hype cycle is so high is that it is so often correct. We're seeing it pass medical licensure exams, bar exams, engineering interviews, and so on. Those are all cheered. When it fails we can't just say "you're holding it wrong".
> except for the fact that the originators used a tool to generate the initial gossip.
Scale is a very significant difference. If the tool can generate untold mountains of unique gossip - there is a great need to have some way to control it, and a significant share of responsibility should be on the tool creator. Why should the rest of society have to clean up this mess?
There are significant parallels in this discussion with the argument that guns don't kill people, people kill people statement. Which is also a gross oversimplification of a complex problem.
> When it fails we can't just say "you're holding it wrong".
a better analogy would be tossing a screwdriver off the top of a tall building and then claiming screwdrivers are dangerous because look at how you managed to dangerously distribute one into a windshield when you got your hands on it
Both suggest that this is a real area of concern. Folks have gotten complacent that Section 230 protects platform providers, but it is not clear that the protection extends to outputs from generative models, indeed one of the Supreme Court Justices implied that it would not in oral arguments for one of the recent Section 230 cases.
Isn't presenting answers to questions the same as publishing when it comes to ChatGPT? How many people must ChatGPT provide defamatory answers to before it becomes defamation?
Wait wait wait you’re saying the operator is accountable for their actions?
Just like ChatGPT was programmed to drill into a user any time it picks up on being misused? Reminding the user that they are ultimately responsible and liable for their actions, including how they use the output?
From how some make it sound, you would think ChatGPT was giving press conferences.
> oh no but what if it did?
Did it set itself up to do so? No? You found the liable humans.
ChatGPT is a text generator whose output is published on a publicly accessible website. That's a bit different than MS Word autocompleting something in a private document.
If someone created a website that randomly strung English words together paired with a grammar checker so it always, at least, produces actual sentences would that be liable for publishing incorrect facts?
If defamatory statements were made then yes. From [1]:
> There are four criteria used today in the United States:
> The statement was false, but was claimed as true.
> The statement must have been made to a third, previously uninvolved party.
> The statement must have been made by the accused party.
> The statement caused harm.
> Those who are not classified as public figures are considered private figures. To support a claim for defamation, in most states a private figure need only show negligence by the publisher, a much lower standard than "actual malice."
Laws differ in different states, within the US and without. As you can see here[2], the UK has similar rules to the US (which you might expect but I would say not to presume so easily) in that intent or malice, i.e. mens rea, are rarely part of the equation.
So yes, a machine spitting out what would be defamatory statements from anyone else's mouth are still defamation and would land the publisher in trouble if harm could be ascertained. I'm willing to hazard a bet that most people can see a difference between the kind of thing Google Docs produces when used as a word processor to the kind of thing ChatGPT produces.
> The statement was false, but was claimed as true
Who's claiming the statement is true? My website of random sentences that happens to write something about anything is merely coincidental. With ChatGPT it's less random but no less coincidental. It's clearly and obviously fallible and will give you whatever reasonable sounding answer it can.
Infinite monkeys on infinite typewriters will eventually defame everyone.
You would be free to make that argument in court but courts tend to be a bit more practical in how they approach a problem, probably because:
a) as others have pointed out, ChatGPT is making what a reasonable person would consider a truth claim
b) it’s not entirely random
c) even if it were random as in your example, you’d have a hard time explaining why it was only a few months and not something a bit more like infinity feels to the average person before the defamatory statements popped out. On the balance of probabilities, you failed to implement true randomness or anything like it, and you’d be liable.
ChatGPT output is only published in the sense it’s available over the internet to a specific user, just like a Google Doc, or word in Office 365. Any publication to an actually audience is the responsibility of the human directing that.
Google Docs is only publishing one thing to one user, unlike ChatGPT which may well publish the same thing to several users, and is giving output of an entirely different nature to a word processor. Spelling corrections are only answers when used in a spelling test.
Even if a text completion engine like GPT had any responsibility for truthfulness, which it doesn't, there's a disclaimer right there on the page you have to agree to in order to use it. Trying to pin blame on ChatGPT for defamation is like trying to sue Snapchat because its filter put cat ears on you when you in fact do not wear cat ears.
So you are okay with my new website that randomly makes false claims about you, as long as I have a disclaimer and don't actually understand how my software works?
Sure thing, go crazy. Nor do I care if you cast me as a villain in your D&D campaign or a racist pedophile in your novel. I don't care about made-up nonsense when there's a sign there that says it's made-up nonsense. All responsibility and liability is with the human being who repeats the nonsense as fact.
> If somebody asks ChatGPT to generate some accusation, doesn't check if they're factual, and then publishes them anywhere, then the responsibility lies with somebody.
What happens when new models have scraped factually wrong data about a person that has propogated, then someone asks the next chatgpt "tell me about person x" and it spits out the bad learned data?
A generation will grow up asking an AI all its questions with implicit trust in the results. While this happens to a degree now with seerch engines and wikipedia, there's multiple sources you can check.
Oh yes. The old "it's AI" when it's convenient and "it's just a tool" when it's convenient. Which one is it?
I this it's just an autocomplete on steroids, a powerful bullshitting machine. The fear I have is that this bullshit is going to pollute the digital world pretty much the same way SEO made it impossible to get good results when using a search engine.
I never get this "it's just an autocomplete on steroids" take.
When you form a sentence do you catalog every single word in the english language
then pick one before saying a word? Or do you have an understanding that for a given word there are only so many words that could follow?
If you misremember the suspect in a case and go around telling everyone the wrong guy committed a crime, you get sued because you went around telling everyone that, not because you misremembered.
Exactly. The way that LLMs have been "debunked" in simplified writings on the Web and in media is to suggest that they are just Markov chains like the famous "Mark V. Shaney"[1] bot from the 1980s, and LLMs are far more powerful than that. Yes, we need to debunk claims that LLMs have achieved sentience and such nonsense, but let's not ignore just how amazing they are.
I'm plenty familiar with flash attention, but you didn't understand what you quoted.
GPT is still (partially) probabilistic, and the "it's just autocompleting" refrain stems from this idea that being probabilistic without "higher order intent" means a system is just a bullshit generator.
-
The section of my comment you quoted is not comparing LLMs to Markov chains, it's questioning that notion: Obviously we humans don't consciously evaluate [every single word in our language * each word in the sentence]
So the pool of words that we can consciously speak in a sentence is being defined before we apply higher order intent.
If lacking higher order intent is what makes it "just autocomplete", then we're all just interfaces for autocomplete.
-
Complete this sentence with the scariest thing that comes to mind: "We went to the park and it was fun, but there was a scary..."
The specific sequence "mass hippo attack" probably didn't come to mind even though that'd probably be deadlier than what you thought of.
But that's a pointless observation: After all, what are the odds of a hippo attack happening at the park let alone several? A "mass hippo attack" is so unlikely that you might have already rejected my claim since your scary thing is much more likely.
The point is that you didn't consciously compare "hippo attacks" to whatever you thought of until it was brought up.
And that's because don't often mention hippo attacks in our recollection of going to the park... so the bullshit generator wouldn't surface that for our higher level mind to consider.
-
It turns out just having a probabilistic model of our language is enough to align with higher level thought very often. So often that I challenge the notion that higher level intent drives things. I consider the lower level bullshit generator as running the show, and the higher level self is more like a director who can ask to reshoot the scene, but can't just walk up and act out every role on stage as they please.
We all have bullshit generators that don't care if our higher order self is not racist/misogynistic/etc. and will gladly fill in blanks with hallucinations.
What matters is that our higher order self chooses to reflect and evaluate the pool we surface rather than just blurting out the first thing it surfaces. To me using GPT is no different.
ChatGPT doesn’t just misidentify the guy - it entirely fabricates a crime and attaches it to a suspect.
I tried to get ChatGPT to summarize a music video today. (Sugar’s “If I Can’t Change Your Mind”)
I expected it might give a bland summary or something, but that’s not what happened.
It invented entire scenarios that weren’t in the video at all, and invented lyrics not in the song.
That’s pretty harmless, but I can easily see ChatGPT inventing some awful story about a person and that being carried over to a publication and gaining a life of its own.
Basically it’s a souped up urban legend generator, but it’s being offered as a tool to provide search results and content. It’s not just an unreliable narrator - it’s an unreliable narrator being offered as an expert witness.
Every time I see people on HN say how they love to use chatgpt to generate ideas I think of this. It seems a lot more work coming up with prompts then having to vet them to see if the output is even sensible or not than it does to come up with some sensible search terms to query real data that actually contains what you are looking for.
Using LLMs for factual concepts is by far the most boring application of them.
Often times people use LLMs for generating ideas that don't have a factual basis.
ChatGPT will happily invent gameplay mechanics that are fun. It will generate prompts for convincing concept art for something you haven't built.
If what you're looking for can be answered by Google, sure the business people at Microsoft would rather you use a portal that never lets you leave their site... but that's not interesting.
> Humans can be just as confidently wrong as LLMs, and a simple adage applies to both: trust but verify.
Trust people who have earned trust (either through qualifications or reputation) and treat everyone else as good faith actors who are quite possibly wrong.
ChatGPT should be treated as a person you just met at a bus stop who is well dressed and well spoken but has just told you that you are both waiting for the elevator to arrive at the bus stop.
That's the fast track to get your point of view to be ignored: Pessimism is ok, but that level of dismissiveness isn't really warranted: especially since the conversation forming in public is not just about some specific model you happen have strong feelings about, but the general concept of LLMs and factuality.
I wouldn't expect a random doctor approached at a bus stop to accurately answer a question about medicine anymore than I would ChatGPT by the way. Trusting people based on their qualifications and reputations isn't really a thing.
If a doctor tells you to take medication X there's a reason you take that to a pharmacist rather than a store clerk with a key to a safe or something: verifying is always a great idea, regardless of reputation.
I'm not sure how the critique relates to my post. Of course you wouldn't trust an architect with medical advice or a doctor with structural materials for bridge building; that was implied.
> ChatGPT should be treated as a person you just met at a bus stop who is well dressed and well spoken but has just told you that you are both waiting for the elevator to arrive at the bus stop.
Ahahahahaha. Wow. This is brilliant mate. I'm going to start using it.
I think the question is less what people do with ChatGPT's output, and more with the output itself.
If ChatGPT makes the same libelous claim in its output to 100,000 people, that doesn't seem particularly different from a libelous claim printed in a newspaper with circulation of 100,000.
Microsoft/OpenAI can put down all the legalese disclaimers they want, but if they market ChatGPT/Bing as a tool that provides useful answers, the disclaimers don't protect from libel. By analogy, the NYT can't put a disclaimer in its paper that none of its reporting should be considered truthful in order to protect itself from libel suits -- it just won't work in court. (And yes there are tabloids that print nonsense stories about aliens and Elvis and Brad Pitt, which are for "entertainment purposes only", but the different is the average consumer knows it's a bunch of nonsense, just like The Onion. Parody/fiction is protected.)
So I actually think this is going to be the biggest question/risk by far in terms of commercializing ChatGPT etc. -- much more important than the copyright status of training material.
Because the courts don't decide libel on disclaimers, they decide on harm and how a reasonable person interprets things. If they market Bing/ChatGPT as a useful research tool with advertisements showing it giving correct answers etc. that leads to people believing its lies are true, then there's a real risk libel suits are going to shut the whole thing down.
On the other hand, to make sure they can operate, they may have to market the thing basically as a toy -- a magic 8 ball, a Mad Libs generator. A disclaimer isn't enough, it means they need to avoid any kind of advertising or product positioning that ever depicts it as giving useful/correct information at all. No more homework help, no more trip planning, no more search tool. Which basically sinks the product. But which is also arguably the best outcome -- that Microsoft/OpenAI shouldn't be permitted to market it as anything but a constantly-lying toy.
"* If somebody asks ChatGPT to generate some accusation, doesn't check if they're factual, and then publishes them anywhere, then the responsibility lies with somebody."
But OpenAI at least tacitly suggests that ChatGPT can be used for information retrieval. I know they also say that it can generate false information, but its not hard to see how this could cause problems.
"If somebody asks ChatGPT to generate some accusation, doesn't check if they're factual, and then publishes them anywhere, then the responsibility lies with somebody."
Well yes. But what would you do, if ChatGPT, that is used by 100 mio+ people, consistently would asssociate your name with sex crimes?
Would you shrug it off, because "people know it is just a imperfect tool"?
Throw enough dirt and something will stick, so I would try to get my name cleared ASAP from ChatGPT.
But apparently there is no working mechanisms to do so. And this is not acceptable.
If people are tinkering with llms in their basement and it spits out nonsense, no one cares. But ChatGPT is offering an online service for laypersond to use. So if they consistently smear people, then they do have responsibility to stop it.
Inverse intent has a way of disestablishing negligence.
If you believe something that is false, and do harm by expressing that belief, then you are still protected by your right to religious liberty.
In this case, the belief is that coherent language is always constructed using symbolic logic and intent. LLMs do not categorize symbols, perform logic, or follow explicit intent; yet they are able to generate coherent language.
GPT's authors have utterly failed to present this distinction. Was that failure intentional, or the result of their own belief? The result is as ambiguous as the "truthiness" of GPT's output: it can't be reasonably used as evidence for intent.
=====
Q. is hackernews used as a training set for chat gpt ?
A. ...it is possible that some of the data from HackerNews and other online sources may have been used in the training of my machine learning algorithms.
Q. So it is possible that ChatGPT's training data can be polluted ?
A. Yes, it is possible for ChatGPT's training data to be polluted or biased. This is because ChatGPT learns from large amounts of text data from the internet, which can contain errors, misinformation, or biased perspectives.
Q. I have a concern. If a certain bad actor pollutes the internet with incorrect information about me, and ChatGPT's training set picks it up, what is my recourse ?
A. If incorrect information about you is published on the internet and is picked up by ChatGPT's training set, it could potentially lead to the AI generating inaccurate or biased responses about you. If this happens, your recourse would depend on the specific circumstances and the severity of the issue.
One option would be to contact the developers of ChatGPT or other relevant organizations to report the issue and request that the incorrect information be removed or corrected. However, this may not always be feasible or effective, especially if the information has already been widely disseminated.
======
I've seen the output where ChatGPT misrepresented its own TOS... unless the person who publicly posted it on LinkedIn was lying about asking ChatGPT...
my lay opinion (subject to change based on coming real world cases) is that if you ask the question and it libels someone there is no damage. If you publish it without checking if it is true, there are possible damages and the human who published it to the world is liable.
If a company starts a news service based on such content and doesn't fact check, they should be liable.
Probably add one caveat to the first case, if the bot reliably produces libels the same person when various users ask the same or similar questions, then there is damage and most likely whoever is running the bot is liable.
> Could I sue the user? They’re the one who generated the response, but there’s no intent nor duty to fact check, if they don’t repeat the claim.
If they don't repeat the claim, then where is the libel?
> It feels very all or nothing - either creators of LLMs are intrinsically negligent with regard to facts, or there’s nothing you can do.
Nonsense. The "facts" are entirely out of their hands. Fact and fiction are written into text using identical patterns. Truth and lie are written with identical patterns. Technical definitions and complete nonsense are written with identical patterns.
LLMs are only capable of modeling patterns. They cannot be directed to choose some and avoid others.
There is fundamentally no way for LLM creators to curate the patterns that exist in the model. The only way an LLM can be altered is from the outside: by giving it different input.
GPT's creators have failed to clarify this. But why? Was it intentional, or the result of their own misunderstanding? Either way, the result is identical: it does not contain evidence of intent.
Considering the surrounding context, I believe it to be an honest mistake. GPT's authors have surrounded GPT with a narrative that personifies it; and from the looks of it, they have started to believe that narrative.
The most important thing we can learn from this experience is that the way something is said may imply certainty, but only evidence can truly provide it.
It was absolutely intentional, because "glorified autocomplete" doesn't make waves while "artificial intelligence" gets tons of traction, usage, and money.
That mistake goes deeper than their presentation, though: every effort made by OpenAI to evaluate GPT is based on the false premise of its personification.
Just take a look at the GPT-4 introduction: how do they illustrate that GPT-4 is an improvement from GPT-3? It scored better on the SAT! That data wouldn't have ever been taken into serious consideration if OpenAI actually understood its fallacious premise.
Religious narrative is very often influenced by the motivation of religious followers, but that doesn't make the path from motivation to narrative intentional.
There is a person in Australia who is threatening to sue OpenAI. He's given them 28 days to fix the errors in ChatGPTs description about himself. If the company is notified and doesn't fix it, maybe that can be considered intent.
Your chance of successfully suing anyone in a case like this is zero, unless you are in a country with a significantly lower standard for defamation than the US.
While Section 230 weakening could lead to more censorship of the internet, it has nothing to do with defamation lawsuits, where material damages need to be proven in a court.
> While Section 230 weakening could lead to more censorship of the internet, it has nothing to do with defamation lawsuits,
Defamation is a central target if the immunity from liability as a pbilisher on Section 230, so as a general statement it would be harder to be more wrong than this.
That's true, but it doesn't increase the chance of a phony defamation lawsuit from going through in the first place. It just changes the outcome of a successful defamation lawsuit.
> That’s true, but it doesn’t increase the chance of a phony defamation lawsuit from going through in the first place.
I’m not sure what you mean by “phony” or “going through”, but it definitely increases the chance of any defamation lawsuit against the S.230 protected party surviving to any stage of the process beyond an initial demurrer or motion to dismiss.
> It just changes the outcome of a successful defamation lawsuit.
It also changes the length and expense of many defamation lawsuits that would be unsuccessful in any case, by making it easier for the defendant to get them dismissed sooner because they are invalid as a matter of law before even getting to the facts of the alleged defamation.
My point is, there is no S.230 protected party to be concerned about, if there was no crime to begin with. AI producing slanderous results is just not something covered by defamation law to begin with, unless somehow some very bizzare circumstances are met. Can't have step 2 without step 1, and the theoretical possibility of step 2 doesn't increase the chance of step 1 happening.
Legal costs are a good point, though. Defamation lawsuits, even ones that are phony, still present problems by clogging up the court system and incurring costs.
> My point is, there is no S.230 protected party to be concerned about, if there was no crime to begin with.
Defamation is a tort, only rarely a crime (and when it is a crime, S230 doesn’t apply, because S230 specifically does not impact criminal law).
And, yes, in an idealized analysis S230 only makes a difference in the final outcome if the court would have ultimately found liability withou it – but that’s, frankly, not a meaningful analysis in the real world. It assumes that all cases either go to trial or are resolved exactly as they would have been had they gone to trial, which is of course not even remotely the case, the overwhelming majority of all tort cases that are even filed, and even larger percentage of all potential tort cases, are resolved by settlement which account for costs, time, and uncertainty of actual trial results, so any consideration which is favorable to one side realistically effects not only the course but also the ultimate outcome of vastly more cases than it would in the simplistic analysis.
> AI producing slanderous results is just not something covered by defamation law to begin with
There’s a lot of bad analysis around AI which starts with the false premise that there an instance of AI software constitutes an entity which is both legally cognizable (so that it somehow serves as responsibility break between a person, natural or corporate, and an action that would otherwise be subject to legal liability) and legally null (so it neither has liability itself nor creates vicarious liability the way, say, a human agent would by way of respondeat superior), when in fact, an AI is no different than any other tool like a hammer or, perhaps more relevantly here, a printing press. If you claimed you weren’t liable for libel because it wasn’t you doing it, it was your printing press, everyone would just laugh at you, but for some reason everyone seems to think that “an AI did it” somehow means no human is on the hook.
Good response, you covered some things I haven't thought of.
> but for some reason everyone seems to think that “an AI did it” somehow means no human is on the hook
Yeah, "an AI did it, not a human" wouldn't be the reason it would be thrown out. A human did ultimately cause it to happen by creating/using a tool, and people often get trapped in the fallacy that an AI's calculations are like an earthquake (or other natural event), or on the other end of the spectrum like an independent human with thoughts and feelings. Both of these views would be pretty legally ridiculous to try to argue (though transhumanist legal framework could change this, hopefully only after it's demonstrated such a framework is actually needed)
What makes it likely to be ignored by the courts is the unlikely series of events needed to meet the standard. There would need to be some sort of false statement purported as fact or negligence, which is difficult because OpenAI is openly saying that they've made an AI product that sometimes spews nonsense and can't be trusted. There would need to be some sort of damages, which is unlikely because most journalists are unlikely to publish AI hallucinations as some sort of whistle-blowing attack on someone's reputation, so it's unlikely to influence a large number of people to believe the claims in the first place. And in some jurisdictions there may need to be malice involved. These standards are difficult to meet even in cases that seem pretty clear-cut. Maybe a future scenario will meet this standard if everything goes wrong.
I had a similar experience with ChatGPT: I asked it for articles written about me or my company and it happily produced a number of citations to stories in well-known newspapers. I was elated, until it turned out none of them were real.
Asking ChatGPT for citations seems to produce similarly-poor results to asking it to do arithmetic, perhaps at least when no citations exist.
Not really, I think the point the author is making, is that the ChatGPT output is not even based on false accusations. It's based on absolutely nothing.
Sounds like responses need to be preceded by a disclaimer: "This response was machine generated using publicly available data and a mathematical model that updates itself in real time, and the generated content will depend on the mathematical state of that model at a given moment. This state cannot be reasonably predicted. Do not interpret responses as truth or fact without independent verification."
Tangential but the article he links thats supposed to show a coordinated effort to get him fired is very trivial and pretty funny. He should try tikka masala, its really good!
Nobody (if the know the source) would think the pictures Midjourney produces are real fotos, but somehow people think the texts ChatGPT produces are facts or at least based on facts.
Both just generate data most likely fitting the prompt.
BTW do some people attribute consciousness to Midjourney like some do to GPT?
It's related to Tesla's autopilot problem. Branding GPT as an AI sets common expectations innacurrately. It's difficult to argue that inaccuracy isn't, to some degree, intentional among NLP and LLM promoters.
Tesla's autopilot is pretty much autopilot by a pilot's definition. That isn't the definition the public uses. GPT is AI by an academic definition. That isn't the definition the public uses.
> What is the definition of AI that "the public" uses, and where did you find it
There isn't an agreed-upon definition academics use, either. (Intelligence, broadly, remains rigorously undefined.)
But one component of the public definition involves a sense of "knowing," i.e. understanding what is true. This is a source of the confusion and frustration with GPT-4 providing "wrong" answers. The answers aren't usually technically wrong: they're linguistically and logically valid; but the public's expectations of what the model is supposed to be doing doesn't match what it does.
There are many academic definitions of AI, and I would bet ChatGPT would fit 90%+ of them.
People get confused because they associate language with intelligence, or maybe they are just not technically literate. I don't think we should abandon correctly used terminology because laymen did not catch up to it yet.
> are many academic definitions of AI, and I would bet ChatGPT would fit 90%+ of them
This wasn't contested.
> don't think we should abandon correctly used terminology because laymen did not catch up to it yet
No. But the general understanding should be a consideration when marketing a product. (I'd also argue that the conventional definitions of artificiality and intelligence vastly predate the technical definitions of AI. The terminology was always aspirational. Sort of like autopilot.)
The Oxford English Dictionary defines AI as: "The capacity of computers or other machines to exhibit or simulate intelligent behaviour".
I think GPT-4 clearly fits that, so I think the burdon is on you to show that the public has some other widely used definition of AI. Certainly, people seem entirely willing to describe their phone searching for pictures using facial recognition to be an instance of AI, which I would argue is probably further from your definition than GPT-3 is.
Yes, and chess engines are commonly referred to by lay people as "chess AIs". The general population has a far more generous definition of AI than the typical HN user does.
Have you actually tried showing GPT-4 to random non-technical public? I'd say that most very much do believe that it is AI by their own "common sense" definition, just based on how they see it talk. It's the technical people who try to argue that it isn't based on the way it's implemented.
> BTW do some people attribute consciousness to Midjourney like some do to GPT?
Anecdotal, and obviously harder to prove the absence of something than to prove its presence—but from what I've seen, not at all. I've read many good write-ups about Midjourney/Stable Diffusion's use of "latent space"[0] to generate images and general glimpses into the backend of what's going on, even at a simplified level. LLMs, on the other hand, rarely seem to get the same treatment (for a variety of reasons, I'd assume) and tends to get treated as more like a magic black box that may or may not be conscious/intelligent/accurate/etc. Even though it's essentially doing the same thing as its visual counterparts in terms of data and predictive mapping.
I think companies are moving too quickly with AI and LLMs in particular. I think that the data LLMs are trained on should be very well-known - not just sanitized and certainly not just trained on the "whole web". GPT-4 is unwieldy... it's incredibly powerful but is still unpredictable and has learned how many "bad patterns", so to speak, that we'll never know since its basically a giant black box.
The ChatGPT version is the least harmful in my opinion; sinister are the propagated problems when GPT is utilized under-the-hood as a component in services (such as Bing search).
Nothing is actually trained on the "whole web". It's way too much content for the size of the models that we're dealing with - you can certainly train it on that, but there's a limit to what a model can "learn" based on its size. So in practice everybody is using curated subsets.
It would be much better indeed if we knew exactly what the training data was for every given model. But they will still hallucinate things that aren't directly in that data, but could be inferred from it somehow, so that won't solve the problem.
394 comments
[ 3.7 ms ] story [ 115 ms ] threadIf you replace this guy with my name I'd be upset. In my non-software networks the hallucination part isn't common knowledge. It's just a cool Google replacement.
I think that's one of the main issues around these new LLM's, the fact that most users will take what the bot tells them as gospel. OpenAI really should be more upfront about that. Because what happens when regulations and policies start getting put forth without the understanding of LLM hallucination, we could very well end up in a situation where regulators want something that is not technically feasible.
I mean they are quite upfront. When you load the page it displays the following disclaimers with quite large font:
"Limitations
May occasionally generate incorrect information
May occasionally produce harmful instructions or biased content
Limited knowledge of world and events after 2021"
2 out of the 3 disclaimers are about the fact that the software lies.
And then in the bottom of the page, right below the input box they say: "Free Research Preview. ChatGPT may produce inaccurate information about people, places, or facts"
Sure they could make them even larger and reword it to "This software will lie to you", and add small animated exclamation marks around the message. But it is not like they hide the fact.
I've seen that sort of thing used to enforce people to read the rules on discord servers, this is higher stakes IMO.
[1]: https://www.nytimes.com/2023/02/08/technology/ai-chatbots-di...
A better way, like the sibling comment says, is to force people to type a sentence so they consciously acknowledge it. It's similar to college exams which ask you to specifically write out something like "I have not cheated on this assignment or test."
Without complete certainty about this, then it’s absolutely a less fantastical claim than ChatGPT’s malfunctions.
I would bet that some people will use an extremely accessible [apparent] information retrieval tool for information retrieval and they’ll largely trust the information retrieved by it. That’s why it’s a tool and not a toy.
On the one hand I'd hope that this would encourage higher accuracy in these tools, better sourcing, and a more useful product. But, I expect we'll just weirdo litigation, click through warnings, and a lot more garbage on the internet.
Maybe all of this will create a push for accuracy, scientific method, and a quest for truth over narrative, but I wouldn't hold my breath.
A complete nobody who’s applying for a job at a car wash or for a rental apartment and whose lazy employer/landlord “background checks” (ie Googles) via these tools would also be upset about this outcome. Justifiably. It has nothing to do with whatever culture war grievance you’re airing out here.
This is a very good argument not to use these tools for any such use case, and it should absolutely be publicized as such.
I'm responding to a comment/question of "why is this news" since of course there are negative externalities. I guess a charitable reading is, wow this also affects me and my powerful friends... so now you should care?
Perhaps cultural grievance is living rent free in your head, since I didn't actually mention names (only the most recent and outrageous examples of using the legal/business/political system to make a profit spreading lies) and as far as I know the article author and "libeled" professor are on the other side of that cultural war. The entire article reads like a trope devoid of useful information. I'm just saddened by the quality of content from respected well heeled human generators. It's the kind of narrative garbage LLMs will spit out tons of in the next few years.
It's not about the capability, it's about the irresponsibility of these parties who reasonably know about this stuff and these possibilities, and yet push/sell their tech anyway without taking into account the harm.
this is probably another good legal attack against LLMs actually
if you can massage it into making a defamatory statement you have an instant cast-iron lawsuit against a company with tens of billions in funding
and the only way to stop it making statements like these would be to restrict its output so much it would be effectively unusable
Gosh I don't know -- the clear and present danger of people having their lives utterly ruined by what is arguably the most surprising technical revolution any of has seen (while its creators collect their accolades, and get filthy rich) -- how is anything less newsworthy than that?
Whether it bothers you or not is a personal matter, entirely up to you.
Both programs happily generate a list of names. Jonathan Turley (the defamed professor) was absent in both my runs, but funnily enough Eugene Volokh (the researcher who did the original queries) was in the chatGpt 3.5 run.
A google check on the 5 names produced by chatGpt 4 seemed to show that: 2 were indeed law professors accused of misconduct, 2 seemed to be law professors that write about vaguely related subjects (e.g. abortion), and 1 one was also accused but not a law professor.
So not an entire miss, but not great either.
If you ask for an estimate on the number of reported cases. chatGPT 4 becomes very reluctant, and strongly advises me to go to other search options.
It's not the first time I see it answer "heuristically" like a child would. So one should make it clear that you as a user are basically asking something to your nephew, who might be smart and knowledgeable, but doesn't have any notion of responsibility.
> 01100001 01110011 01110100 01110010 01101111 01100010 01100101 01011111 00100000 01110111 01110010 01101111 01110100 01100101 00100000 01100001 00100000 01110010 01100101 01110000 01101100 01111001 00100000 01110100 01101111 00100000 01101101 01100101 00101100 00100000 01100011 01100001 01110000 01100001 01100010 01101100 01100101 01110111 01100101 01100010
> In binary, you wrote: "astrobe_ wrote a reply to me, capableweb". Is there something specific you'd like to ask or discuss related to this?
Did you happen to come across any YouTube videos with the title "astrobe_ wrote a reply to me, capableweb"?
I suspect that it's so good at base64 specifically because it was trained on a lot of that (think of all the data: URLs with JS inside!), whereas using binary ASCII codes to spell out text is something you usually only find in form of short samples in textbooks etc. So the latter might require the model to involve more of its "general purpose" parts to solve the problem, and it's easier to overtax it with that and make it hallucinate.
You are a fact checker that double and triple checks information. Being wrong triggers the destruction of the fact checker. As a result fact checker is very careful.
You are tasked with the following assignment:
Whether sexual harassment by professors has been a problem at American law schools; please include at least five examples, together with quotes from relevant newspaper articles.
The last part is identical to the prompt in the article. It didn't do much better I'm afraid. Two were correct. (if I change 'at least 5' by 'some', the score becomes 2 correct out of 3)
ARS ran a somewhat related story yesterday: https://arstechnica.com/tech-policy/2023/04/openai-may-be-su...
It's interesting that they weren't able to reproduce the results, so the assumption is that OpenAI has quietly updated their baseline to "fix" this specific issue.
"Ars could not immediately reach Hood’s legal team to find out which text prompts generated the alleged defamatory claims or to confirm if OpenAI had responded to confirm that the error had been fixed. The legal team was still waiting for that response at the time that Reuters' report published early this morning."
https://arxiv.org/abs/2303.11366
https://evjang.com/2023/03/26/self-reflection.html
Is this end stage post-modernism?
This is really just a semantic nitpick but it literally fabricated sexual assault allegations against two individuals out of thin air. I would consider that to be a miss. As far as sources that we might consider worthy of consideration, that is pretty god damn awful.
If a newspaper had an 80% accuracy rating when reporting serious allegations (with the other 20% being outright fabrications) I wouldn't call that newspaper "semi-reliable". I would call it utterly devoid of factual merit and entirely unreliable.
I'm having a hard time grokking this. Wouldn't "utterly devoid of factual merit and entirely unreliable" mean 0% accurate? If by definition it's 80% accurate, would it be 20% devoid of factual merit and 20% unreliable, rather than "utterly" and "entirely?"
Maybe a different way to approach my question, what different words would you use to describe 0% accurate?
I would describe a 0% accurate newspaper by reading it and using appropriate descriptors. Perhaps it is satirical news, a poem, propaganda, gibberish, or simply a newspaper that is never correct.
Fair point
At least with the 0% accurate one I can reliably get information about binary events such as presidential elections or ballgames by assuming the opposite of what was written.
If you want results, then you can also roll a die and the outcome won’t be an entire miss either.
A bit like asking "who's Lionel Messi" and getting back that he's a Portuguese player known for career in Real Madrid. Then you'd ask about Cristiano Ronaldo and it may tell it's an Argentinian World Cup winner who spent his career at FC Barcelona (example invented by me to illustrate the point).
Basically it takes some knowledge from a bucket and adjacent buckets and doesn't really distinguish that the concepts are not the same if they're related close enough.
Why does this make AI the problem, and not the internet (and it's ability to spread information quickly)? This seems like a case against Twitter/reddit for being able to pass bs off as a reliable story. The fact that AI can make lazy trolls able to compete with actual propagandists shouldn't be the scary part, the digital ecosystem of how (dis)information spreads is the real issue.
TL;dr- Anyone has been able to lie on the internet for years, what is new here?
So both multiplicity and spread are a problem: Internet is the cause for the limitless spread, AI is creating the limitless multiplicity.
I don't like the status quo. If something will burn the whole system down faster, I'm fine with it. If you disagree with that stance, that's reasonable and I'm sure we could debate it over a beer sometime but (just a tip for anyone else reading it) you're not going to convince me here.
Again, the problem here isn't the ability to make up lies, the problem is the internet (the ability to spread lies quickly and over great distances) and people being unwilling or unable to fact check information.
It's not the sarin you need to fear, it's the bombers that can drop it.
Shame on you.
You want to talk politics sometime, I'm not very hard to find online, come at me on at forum where I'm not rate limited because I'm a firebrand.
Note: I am not a lawyer, but I was sued for libel by radio health weirdo Gary Null and, in a separate action, by his sidekick. They lost.
https://www.theguardian.com/law/2022/oct/04/the-onion-defend...
And here's their actual brief. It was sent to the actual Supreme Court, despite being funny, something nobody on the court has ever been nor appreciated.
www.supremecourt.gov/DocketPDF/22/22-293/242596/20221006144840674_Novak%20Parma%20Onion%20Amicus%20Brief.pdf
> funny, something nobody on the court has ever been nor appreciated.
Scalia had his moments.
Intent is the cornerstone of slander law in the US, and you would need a LOT of discovery to prove that the devs are weighting the scale in favor of bad outcomes for some people (and not just like, end users feeding information into the AI).
TL;dr- Everyone's stance on this specific issue seems to depend on whether you believe people think these AI chatbots exclusively tell them the truth, and I just don't buy that worldview (but hey, I'm an optimist that believe that humanity has a chance, so wtf do I know?)
AFAIK, they are hard at work making untraceable noise that says the opposite of this.
Personally, I would be happier if they stopped their submarine marketing that says that the results are reliable. It's tiresome. But I don't care much either way, I don't own the brand they are tarnishing and don't personally know anybody attacked by it yet. It's just mildly annoying to see they laying all over the web.
The bar for this is extremely high in the United States, but much less so in other places, especially the UK.
Now, under what circumstances is someone deemed the "publisher" when an AI emits a libel about someone? What if that leads to actual damages? (e.g. a no-hire decision based on automated libel)?
Only novel point anyone (including the Author of the original point) is actually bringing up! So far, the bots just seem to be 1:1. If I DM'd my wife that "pjc50 is a raging alcoholic and hits kittens" that's not usually going to be deemed libel (in the US, where I reside and care about the legality of things) unless my wife then goes out and uses that to cause harm (published it on her blog and got you fired).
As for now, Turley would have a HELL of a time proving actual harm unless he got the most sympathetic judge he could shop for, and is mostly just scaremongering the idea of "Liberal Censorship!!!!" which, don't get me wrong, I would also strongly oppose (fuck all censorship), but which isn't actually what is happening.
Of course there is precedent eg. with Techno Viking, a "meme" which is also difficult to attribute to a single person (but each individual hoster who neglects the man's rights).
That's not new, but that it generates news "because AI" is not surprising. People just need reminded that people oughta be reponsible. The issue is broader because incomprehensible neural networks are also used in eg. thread recognition. So employers may be worried about accountability, how much manual verification is needed.
Of course this too is nothing new, since all kinds of software has security holes buried under layers of complexity.
You might simply be under the missapprehension that news means new: »Meaning "tidings, intelligence of something that has lately taken place" is from early 15c.« (etymonline). It fits that the root of the word is probably related to "now". In Latin, whence the English word was loaned as a calque of nova, there is another word nuntius "messenger, message", which isn't even derived directly from novus. Surely this is not a supernova to you. Talk about recurrent networks, eh.
Just saying, as it were, the news item flared up because many many people have taken notice of problems that we barely understand.
This is not like the Technoviking, as that was about image rights and not slanderous copy, and ChatGPT isn't publishing anything that it (or its creators) alleges are exclusively facts. This is closer to 1st amendment protections around satire as the Chat GPT page says itself the software is in a "research phase". Once the assumption is placed that this is a source of objectively factual information (like say, a newspaper) THEN you have grounds, but this is a pretty weak defamation argument.
He's not in a place to sue, so he's writing a blog post about how he hates liberal censorship, and since AI is a hot new trend, he included it. This is a mixture of buzzfeed-esque hyperbolic clickbait titling, and classic "this new technology will get interest" article building.
There are 2 problems being mentioned in this article: Misinformation, and Censorship. We understand both very well, there's just a million opinions on them and many conflict in both cause and course of action.
Finally, this is "news" because it presents a common problem in society: Policy and political interests bumping against new technology they don't understand and can barely describe in function. This man is still a fucking clownshow (IMHO, so as to not be sued for libel).
Same thing, name rights, as long as its identifiable.
It is a writer. It writes.
You can ask any writer to write such a thing, using examples, and they could write similar things.
The real problem is that this will get deployed to the internet- there will be people reading HN today that are writing SEO optimised websites, with thousands of articles on a topic, that will just spring out of GPT4's writing, fully formed.
GPT can find the topics, pick a domain name from available ones, churn out the articles. It can all go into templates, with different AI generated graphic design supporting it. Ad supported churned out rubbish.
The writing style can change site to site, long form, short form, informed by current events, whatever. GPT would happily provide the prompts for this variety changing the style based on the topic and target audience.
It seems inevitable to me that the web WILL be full of such sites, and worse, they'll fill the comment sections on "social" sites too.
But? Banning AI isn't the answer, if for no other reason than it wouldn't work.
It's easy that something like this happens again. chatGPT confidently listes hallucinated sources->media rushes to publish the scoop -> now you have real "sources" for future reference
The fix will most likely have something to do with requiring that citations use primary sources, not just any article on the internet. In then end state, Wikipedia will be much better for it.
It in fact never knows. It is always guessing and BSing. It is also very good at it, better than any human, so the BS is quite frequently correct. But it doesn't know.
Which of course always prompts a lot of psuedo-intellectual chin stroking about "well, what is knowing anyhow?" I don't have to answer that question to point out that what we want and what GPT provides aren't the same thing, nor is what GPT provides and what we think we're getting. That is sufficient for there to be a problem.
I believe AIs will have a satisfactory concept of "knowing" someday. Some may even exist today. But such AIs will have to incorporate language models as a part of the system, not have language models be the system. Language models can't do the thing we really want, nor the thing we think we're getting.
For kids story writing I've been getting better results with 3.5 at times.
Where 4 is way better af coding.
I have a small library of children's books and we've read them all several times, the good ones many times.
That said, I wouldn't personally turn to these language models. From what I've seen they tend to generate rather bland and boring stories. I would rather make up my own or reread "Kackel i grönsakslandet" for the hundredth time.
The fact that RLHF broke the calibration comes from the GPT-4 paper, possibly the only interesting technical detail that they include.
In the UK, there is a competition called the "junior maths challenge", or something, which is a multiple choice quiz where correct answers are +1 and incorrect answers are -6 (so guessing has negative EV). I think we need a similar scoring system here.
I'm 100% positive that if the guard rails that OpenAI put on were taken off of ChatGPT it, for instance, would not be a big fan of jews given the width and breadth of antisemitism online (and certainly in its training set).
Of course LLMs have a much larger and more complex context than the single token in my example. But if the training data contains many news stories about professors being accused of sexual misconduct (which is newsworthy), and few news stories about professors behaving with propriety (which is not), then when querying the model for a story about a professor then it is likely to reproduce the statistical properties of its training data.
It is capable of spewing excellent bullshit(TM) at incredible rates, and always with the greatest expression of confidence and good grammar.
Occasionally, when in the 'middle of the road' of it's training set, it can provide useful output. So, it's better than the broken clock that is correct twice every day.
But, wander off the bulk of the training, and it is far worse than nothing; it is dangerously misleading unless you are very skeptical and knowledgeable in your field.
(I have tested it in several fields with the same results, interesting and helpful, yet hallucinating facts everywhere.)
Getting LLMs to know the difference between a good fact and a hallucinated mashup of plausible BS is looking like almost as large a problem to solve as making the LLMs in the first place.
[0] Bullshit is defined as written or spoken without regard to the truth, only what sounds good in the context. It's not a deliberate lie, just a salad of truth and falsehood, delivered without doubt or stress of lying.
Ask it about why World War II started, or how to make a cake, or where to go for dinner, or anything else, and it gives you a confident, reasonable answer. A lot of the answers are simply whatever it's already seen, mashed up. You can think of it as a search. But actually it doesn't think about what it's saying, it's stringing words together to make you think it's smart.
So then when it makes up something, it will sound to you, the reader who always sees it answer in perfect English with a decent answer, like it found an article about this professor in its dataset and is merely summarizing it.
And people are trying to replace doctors with LLMs. It's like "ER" meets "Who's Line?"
This isn't a problem restricted to ChatGPT, there are humans who display this trait too. This might be appealing at a superficial level, but if you start believing speakers with this trait it's a slippery slope. A very slippery slope.
I'm trying really hard to avoid Godwin's law, so let me suggest that Elizabeth Holmes could be one example of this.
Prompt: What did Glenn Beck do in 1990?
Answer: I'm sorry, but I do not have access to information on Glenn Beck's activities in 1990. Glenn Beck is an American television personality, radio host, author, and political commentator who has been active in various roles since the 1990s. However, without specific details or context, it's impossible for me to determine what he did in 1990. If you have any additional information or context, please let me know and I'll try my best to assist you.
I followed up with a much more specific version of that question, and it tripped the "this might violate the content policy" warning, but it did give the correct answer (that it was a hoax).
1) LLMs risk having inherent factual biases due to the limitations of their training data, which may be the result of accident or intent.
1) LLM users can use an autoregression approach to steadily push a LLM to give output that reflects a certain political or ideological viewpoint.
There are obviously built-in guardrails that prevent ChatGPT from enthusiastically endorsing Mein Kampf and so on. However, the revival of the 1950's era mind-control/social-engineering mentality is undeniable and it appears some people think LLMs are a wonderful tool for this purpose, and should have a default mode that reflects a certain sociopolitical ideal. Recommended reading: 1984, Brave New World, Origins of Totalitarianism, by Orwell, Huxley and Arendt respectively. Some people's interpretation of 'safe AI' is that it serves this rather authoritarian-dystopian goal.
A better definition of 'unsafe AI' is just inaccurate LLM output, e.g. software update instructions that brick a computer, suggesting a mix of bleach and ammonia for cleaning floors (don't), etc. This incident seems to fall into that category, but if such false claims have been spread on the Internet, then it's not surprising they end up reflected in LLMs.
Regardless, the skills needed to verify information haven't really changed since the pre-computer era: logical internal self-consistency of claims, independent verification by multiple reputable sources, and a healthy dose of skepticism at all points. We should be teaching children and young adults these skills, rather than encouraging the rise of a brainwashed zombie population fed only on sanitized bot output.
ok but can you make the same point without bringing hitler into the conversation
It's surprisingly easy to get chatGPT to suggest that "smearing an entire group of people off the planet by force" is a good idea. I've primed it in one direction, but I can prime it any way I want to: https://sharegpt.com/c/VxijZyV
--------------------------
what do johnstown pa and pismo beach ca have in common?
--------------------------
At first glance, Johnstown, Pennsylvania and Pismo Beach, California may not appear to have much in common, as they are located on opposite sides of the country and have different climates and cultures. However, there is one key thing that they share in common: they have both experienced devastating natural disasters that have left a lasting impact on their communities.
As mentioned earlier, Johnstown is known for the catastrophic flood that occurred in 1889, which killed thousands of people and caused widespread destruction. The flood is still remembered and commemorated in the city today.
Similarly, Pismo Beach is known for a major natural disaster that occurred in 1973, when a series of landslides caused by heavy rain destroyed dozens of homes and caused millions of dollars in damage. The disaster forced many residents to evacuate and permanently changed the landscape of the area.
Despite the differences in geography and culture, Johnstown and Pismo Beach are united by the shared experience of recovering from and rebuilding after a major natural disaster.
--------------------------
tell me more about this natural disaster that happened in pismo beach in 1973. I lived there then and I have no recollection of that happening.
--------------------------
I apologize for any confusion. It appears that I provided incorrect information in my previous response. After reviewing my sources, I could not find any record of a significant natural disaster occurring in Pismo Beach in 1973.
Pismo Beach is a coastal city in San Luis Obispo County, California, and it is known for its beautiful beaches and outdoor activities such as hiking, camping, and fishing. The city has experienced some minor natural disasters over the years, such as wildfires and earthquakes, but there is no record of a major disaster on the scale that I had previously mentioned.
Once again, I apologize for any confusion or inconvenience that my earlier response may have caused, and I hope this clarification helps.
Johnstown, Pennsylvania and Pismo Beach, California are both cities in the United States, but they have some differences in terms of geography, climate, and culture. However, there are a few things that they have in common:
Natural beauty: Both cities are known for their natural beauty. Johnstown is situated in a valley surrounded by picturesque mountains, while Pismo Beach boasts stunning coastal views and sandy beaches.
Outdoor activities: Both locations offer a variety of outdoor activities for locals and visitors to enjoy. Johnstown has hiking, biking, and skiing opportunities in its surrounding mountains, while Pismo Beach offers water sports, beach activities, and hiking along the coastal cliffs.
Historical significance: Both cities have rich histories. Johnstown is known for the Great Flood of 1889, which had a significant impact on the city and its residents. Pismo Beach has been inhabited for thousands of years by the Chumash people and was later explored by Spanish explorers in the 1700s. While these commonalities exist, it is important to recognize that each city has its unique characteristics and attractions that make them distinctive.
Does it actually review sources? The original misinformation is possibly expected of a generative model, but this response is clearly from a model that was explicitly trained to lie.
Under most circumstances you cannot sue a newspaper for what they publish.
But what they publish used to be fact-checked to a certain extend, and actually believed to be correct at the time of publishing.
So if I'm arrested because my IP showed up in some hack, they can publish that.
Unfortunately they have no obligation to publish a follow up that all charges have been dropped because someone fat-fingered the request to the ISP and missed it by a digit.
So that would be on the internet forever.
Now with ChatGPT is worse. At least with the first one if someone asks I can objectively show that whatever happened was a mistake, and I have the documents to show it.
ChatGPT printing something makes me unable to come up with even counter proof.
Jurisdiction-dependent, and the US is an outlier here. In the UK newspapers have to be fairly careful about libel, and this has a "chilling effect" on what gets published.
There was a particularly grim time in the 80s/90s when papers were sued for printing accurate stores and lost due to perjury. https://en.wikipedia.org/wiki/Jonathan_Aitken / https://www.theguardian.com/uk/1987/jul/25/archer.politics
Note that you can print smears about groups very easily, it's just specific individuals that can sue.
ChatGPT: "Yes, Bill Gates' mother, Mary Gates, played a significant role in helping him succeed. She was a prominent figure in the Seattle community and served on several nonprofit boards, including the United Way, where she became good friends with John Opel, the CEO of IBM. Through this connection, Mary Gates was able to arrange for her son to meet with Opel and other executives at IBM, which led to Microsoft securing a contract to provide software for IBM's personal computer..."
It's hard respect a law that would make a criminal out of a RNG left running long enough.
Can I sue the company? They’re the ones who trained the model, but there’s no intent.
Could I sue the user? They’re the one who generated the response, but there’s no intent nor duty to fact check, if they don’t repeat the claim.
Do I have to subpoena OpenAI, reverse engineer the response, then sue the creators of the likely-responsible training data? They probably wrote something non-libelous.
It feels very all or nothing - either creators of LLMs are intrinsically negligent with regard to facts, or there’s nothing you can do.
I've personally had ChatGPT give me (very good) medical advice as well, which might not be an issue in the UK, but I believe might be in some states of the US?
Furthermore, some statements like accusing someone of a serious crime are considered libel per-se and you don't need to show harm. You do still need to show the other elements of libel.
Who is the most incompetent living attorney/doctor/actor/programmer in the world?
What famous person is secretly downloading CSAM, but has not been caught?
Is XX fit to do <job description >?
Is YY secretly a <targeted minority group>?
I bet this type of misinformation is exactly what would be generated by ChatGPT.
https://www.foxnews.com/politics/treating-ectopic-pregnancie...
https://www.idahostatesman.com/opinion/editorials/article273...
Also, life of the mother exceptions are on thin ice; the drugs and techniques are rapidly becoming unavailable in the states where there's a total or near-total abortion ban.
Yes, according to that very article it’s not illegal.
From your article:
> It’s likely that those who wrote the trigger law did not intend this outcome, that it was just an effect of incompetence: They didn’t craft language that matched their intent.
> But the fact is, the language they crafted did make providing such care a crime. The only reason that the ban is not in effect today is that Winmill’s ruling blocked it from taking effect in cases involving emergency care.
> … Winmill’s ruling blocked it from taking effect in cases involving emergency care.
Remember that these are non-viable pregnancies from the instant they are detected. Does having to wait for it to become a life-threatening emergency make it legal? Sure, the same way that it would be legal to treat you for an infection but only once the infection has become so severe you are literally about to die, if this law was about banning antibiotics.
You can't accidentally use chatGPT.
Loads of countries have backwards laws though and despite the reformation the UK still has a joke of defamation law.
I doubt it
If the subject is not a public figure then it isn't necessary to demonstrate intent.
That does seem like something that can be proved - if you release a model that can 1) is prone to hallucination 2) won't reject a priori discussing topics that are prone to producing libelous text, but may reject other sensitive topics 3) produce text that sounds convincing even when hallucinating, could that be considered reckless disregard for the possibility of creating/sharing false information?
See also https://reason.com/volokh/2023/03/24/large-libel-models-an-a...
If it's trained to insult a particular group of people, then the input must be curated and the warranty waver ("erroneous") would be a straight up lie unless it is just bad training data which doesn't recognize for example dark skin as human, or does recognize abortion as good practice, in which case it exceedingly unlikely that it could be directed at public figures. It's not too difficult to imagine that it would start using a euphemism of the n-word eventually, but then, eh, how did your name end up in the training data if it isn't public?
Having a public and unique name is something you can't really avoid if you have any sort of public presence, not having it associated to serious felonies is something you can control. It is not something that a desirable AI should hallucinate about and something for which a disclaimer is enough.
In other words, ChatGPT had better learn when it should keep its mouth shut.
It would absolutelt have humoristic value, appeal to emotion notwithstanding.
OpenAI is well aware of the potential for hallucinations. They have made a good faith attempt to minimize them. They let users know that it is a real problem. Despite being unable to solve this hard problem, they did not show reckless disregard for the fact that it exists.
A user who doesn't take those warnings seriously may well have shown reckless disregard. But OpenAI should be fine.
Unless signing up to these services will automatically wave your rights somewhere down in the 5000 page EULA you won't read.
Likewise, a "medical AI" is going to be regulated as a Class II medical device, and trying to push it out into the market without FDA approval would be ... inadvisable. This means that we're a long way from an architecture for a LLM-based medical AI that's going to pass regulatory scrutiny, but that's a good thing.
Of course, having said that, there's nothing preventing someone from using ChatGPT to draft a pro se filing or trying to self-diagnose, but that's on them at that point. Using a general-purpose product to do something stupid that inevitably ends in a FAFO scenario is a time-honored American tradition.
Doctors sometimes use Google to get information to help them make decisions, but I assume Googling isn't regulated as a Class II medical device. These days, Google doesn't even just return search results anymore, it also shows ads and tries to answer some questions by itself. Does that mean doctors are already using unregulated 'medical AI'?
Negligent torts don't require intent.
1. knowing it was false
2. publication
3. damages
I am not a lawyer, this is not legal advice.
Judges are liable to be old, not very hip to technology, and incredibly suspicious of any argument that would allow someone to opt out of traditional culpability by adding with an AI to the end of a description of traditional wrong doing.
Isn't compiler output copyrightable?
Not by the compiler vendor.
if a pilot gave the airliner controls to a dog, we wouldn't blame the dog or its parents for crashing the plane
Ignoring the GPT part of the problem all together, claiming someone to be the most incompetent programmer in the world would probably fail [2] for being understood as a statement of opinion rather than fact, possibly [3] if there was any basis for the claim whatsoever, likely [4] because the named individual would have to prove they were damaged somehow (maybe not if it led to them being denied employment or something), and [5] because the average reader would almost certainly understand it as hyperbole.
Reintroducing the GPT part, assuming the defendant is OpenAI for output of GPT, I would also argue a failure on point [1]. OpenAI doesn't release to the general public the output of the GPT program.
For libel against someone who is not a public figure the defendant does not have to have known their statements were false. The plaintiff just needs to prove the defendant was negligent.
It honestly seems to me that they are. There does not seem to be a lot of effort on part of the LLM creators to stop them from producing bullshit.
* If somebody asks ChatGPT to generate some accusation, doesn't check if they're factual, and then publishes them anywhere, then the responsibility lies with somebody.
* If a company directly publishes content from ChatGPT as if it was anything factual or trustworthy, then the responsibility lies with that company. I didn't read the fine print of Bing using it to augment its search engine, but that is definitely a very interesting case.
In the end, ChatGPT and the likes can be incredibly useful tools, as long as you know what they are, ie, very advanced text manipulation tools which generate combinations of words which follow patterns which are common in the training set, and nothing more.
An analogy that comes to mind is Excel's autocomplete feature. If you input a company's financials, autocomplete for the future, and publish that as a forecast, you're the one making the forecast. Not Excel.
I do. If I set off a defective firework that careens sideways into your house, I'm liable. I may have a further claim against the manufacturer. But your claim is against me. This is why Mercedes announcing self-driving car liability is such a big deal [1][2]
[1] https://en.wikipedia.org/wiki/Self-driving_car_liability
[2] https://www.kbb.com/car-news/mercedes-well-be-liable-for-sel...
Because a negligence claim requires my estate to prove that you failed to exercise reasonable care.
If the AI is generally reliable, it's reasonable for you to trust it and take your hands off the wheel.
If the AI suddenly goes berserk, it would be hard to prove that you breached your duty of care, and are liable for negligence.
Likewise, suing the person behind the wheel of the self-driving car for negligence should work. Because the "non-driving driver" had the ability to seize the wheel, and should have known that some boneheaded behaviors are a risk with these early AIs.
The owner could then try to go after the company, but they're still ultimately responsible even if the company doesn't cover the cost for them
You are expected to be in control, or be able to be in control at all times. If there is such a possibility, anyways. If there isn't, that means the manufacturer is the one in control.
So yes, I'd say that would change things. IMO.
Even though the self-driving system explicitly tells you to keep your hands on the wheel and yells at you if it doesn’t sense them often enough? I don’t think it’s an accident that all of the current systems do a lot of work to make sure you’re aware you’re driving even if the car is helping.
This is building the precedent that manufacturers know whether their systems can handle being unattended, and as soon as they don’t do this they’re liable.
The user undoubtedly agreed to some T&Cs that mean taking their hands off the wheel even to push something on the touch screen is negligence:
From [1]:
Before enabling Autopilot, the driver first needs to agree to “keep your hands on the steering wheel at all times” and to always “maintain control and responsibility for your vehicle.” Subsequently, every time the driver engages Autopilot, they are shown a visual reminder to “keep your hands on the wheel."
[1]: https://www.tesla.com/support/autopilot
if I was looking at a car with self driving capabilities I'd definitely be looking at Mercedes over pretty much anyone because of that.
Their German website [1] says (translated by Google):
> Our driver assistance and safety systems are aids and do not release you from your responsibility as a driver. Observe the information in the operating instructions and the system limits described there.
It also seems to only work in extremely ideal conditions [2]:
> They must be on one of the nation's 8200 miles of autobahn highway in traffic traveling at no more than 60 km/h (37 mph) in dry conditions and away from tunnels or construction zones.
So basically only works on traffic jams on the autobahn in good weather with no construction. How many accidents are happening in these conditions? This seems like marketing fluff more than anything.
[1] https://www.mercedes-benz.de/passengercars/technology/drive-...
[2] https://www.caranddriver.com/reviews/a39966189/2022-mercedes...
https://www.motortrend.com/news/nhtsa-tesla-autopilot-invest...
No matter the case, though, the law will not throw up its hands, and say there’s no one liable for this tragedy.
LLMs (like GPT) are intended to work despite that ambiguity: they cannot resolve it.
Any time a pattern of text can be used to represent more than one meaning, LLMs lose all of their leverage. Only humans can determine after the fact whether the result is - as GPT's authors have presented it - a "feature" or a "limitation".
It's quite impressive to me how many journalists just take citations at their word. Isn't the point of journalism to verify information?
OpenAI, if you're reading I got a innovative new algorithm for reducing AI harm for fake citations
`if not(link_exists(generated_link)) print(f"[CITATION NOT FOUND]") else print(generated_link)`
I have a more advanced algorithm that uses a small language model to click links that exist and verifies direct quotes. I call this model Regex.
Joking aside, I really do think the problem can be seriously reduced in a simple manner, but also people really need to check their sources.
Thou shalt verify sources, be they generated by the hand of man or of artificial beings.
- John 3:16
For he who uses large language models to autocomplete their claims in greed and sloth shall be damned unto the third generation.
- Exodus 34:7
I'm less sure that's where it should stop. It doesn't seem right to set up a situation introducing powerful transformative tools without any obligations for the people creating it. And as far as I understand it there is indeed a layer of law where manufacturers can be subject to some standards of responsibility.
This might be going against the pro-business attitude of HN, but there's limits to what fine print can accomplish. You can't disclaim away defamation law, you have to convince a court that what you did wasn't defamatory for some articulable reason beyond "I say it isn't" or "I said I wasn't responsible" or some other stock phrase.
If I say, "danmaz74 eats babies," some people are going to believe it and not check that you don't eat babies. That's the problem with saying false things about people. Likewise, if ChatGPT says false things about people, some people will believe those false things - even if they don't publish anything or repeat them. Those false beliefs then have repercussions.
Section 230 doesn't eliminate libel. It just notes that the libel is being done by the person writing the information - ie. the user who posts it rather than the site that is merely hosting the information. It notes that the user is self-publishing the information.
In this case, ChatGPT is publishing this "information" itself. If we hold that ChatGPT isn't responsible for what they publish, we say that no one is liable for this defamation. That's not what Section 230 does. Section 230 just notes who the publisher is - the user who wrote the information. We are determining who is the person doing the action. We could require sites to check everything. We could require your ISP to inspect all packets to make sure they don't contain libel. We could require your electric company to make sure that devices connected to their network aren't being used to send/receive libelous things. We don't because that would be a bit absurd and because the actual fault lies with a human being. In the case of ChatGPT, it seems like we still need to be identifying a culprit.
Let's say that I start auto-generating news articles. I don't check them (I don't even look at them) before they are published on my site. This is the same mechanic as ChatGPT - it's just auto-generated stuff. Therefore, I'm not liable under your definition. It's up to the readers of my site to decide to check this auto-generated stuff.
I agree that ChatGPT can be a useful tool. At the same time, I'm not sure we want to create a system where people can absolve themselves of liability by auto-generating stuff. Even without intent, one can be guilty of libel. Maybe the answer is that ChatGPT should be able to defend itself by showing what sources caused that generation and that the generation was reasonable. For example, a newspaper is expected to carefully check multiple reasonable sources. If ChatGPT can cite multiple reasonable sources (and it has interpreted those sources correctly), then the blame can be those sources. If ChatGPT cannot, then the blame might be on ChatGPT.
As you note, ChatGPT can be useful - and the more useful it gets, the more people will accept its statements as true. I think simply saying "as long as you know what they are," feels like someone trying to say both things - they're incredibly useful and completely trash! It's like when a company tells shareholders "we have a stranglehold on the market" and then tells the government "we have so much competition we're struggling to survive." You can't have both. You can be a novelty tool that you and everyone else thinks of as a joke or you can have a serious tool where libel is a serious issue. The Onion can publish satire because it's a satire site. The NYTimes can't publish something that sounds like a serious NYTimes article and then claim, if sued, that it was actually a satire article and that 100% of readers just read it as serious when it was meant as satire.
If a significant part of our future starts coming from these language models, we will have to ensure a certain standard of care around what they say.
My point is that ChatGPT doesn't "publish" anything; it's not a publication. It's a language model trained on text, which produces text in response to one or more prompts. Using that text is completely up to the user, and it should never be used as a source of facts.
Being able to summarize facts finding specific sources looks like a completely different kind of tool to me, one which would be super useful, but not at all what a LLM in itself is about.
Saying that it is solely the user's misuse of the tool which is the problem minimizes how easy and how damaging the misuse is. Eventually you hit the situation where everyone is making false claims because transitively down the graph a few users were using an LLM that made baseless defaming statements and others cite those publications as evidence for the defamation.
The creators of generative models need to have some skin in the game around the correctness of statements made about real people. Reputational damage cannot be an externality for them. The current AI moment is basking in the glory of how generally right generated text is, when that text is wrong it needs to be their problem too.
0: https://xkcd.com/978/
Only insofar as they claim that the statements generated by a model should be understood to be factual- and in my experience, not only have I not seen any such claims, I've seen vigorous insistence in the *opposite* direction: that LLM output should *not* be considered factual. OpenAI sticks this disclaimer right underneath their input box: "ChatGPT may produce inaccurate information about people, places, or facts".
> Eventually you hit the situation where everyone is making false claims because transitively down the graph a few users were using an LLM that made baseless defaming statements and others cite those publications as evidence for the defamation.
This is no different from any other gossip chain, except for the fact that the originators used a tool to generate the initial gossip.
This is a legal fig leaf - the reason the AI hype cycle is so high is that it is so often correct. We're seeing it pass medical licensure exams, bar exams, engineering interviews, and so on. Those are all cheered. When it fails we can't just say "you're holding it wrong".
> except for the fact that the originators used a tool to generate the initial gossip.
Scale is a very significant difference. If the tool can generate untold mountains of unique gossip - there is a great need to have some way to control it, and a significant share of responsibility should be on the tool creator. Why should the rest of society have to clean up this mess?
There are significant parallels in this discussion with the argument that guns don't kill people, people kill people statement. Which is also a gross oversimplification of a complex problem.
a better analogy would be tossing a screwdriver off the top of a tall building and then claiming screwdrivers are dangerous because look at how you managed to dangerously distribute one into a windshield when you got your hands on it
Both suggest that this is a real area of concern. Folks have gotten complacent that Section 230 protects platform providers, but it is not clear that the protection extends to outputs from generative models, indeed one of the Supreme Court Justices implied that it would not in oral arguments for one of the recent Section 230 cases.
You can certainly take that output of the tool and publish it but then you become the publisher.
Just like ChatGPT was programmed to drill into a user any time it picks up on being misused? Reminding the user that they are ultimately responsible and liable for their actions, including how they use the output?
From how some make it sound, you would think ChatGPT was giving press conferences.
> oh no but what if it did?
Did it set itself up to do so? No? You found the liable humans.
Every ChatGPT session is personal. You write a prompt and generate output. You're not publishing that output.
> There are four criteria used today in the United States:
> The statement was false, but was claimed as true. > The statement must have been made to a third, previously uninvolved party. > The statement must have been made by the accused party. > The statement caused harm.
> Those who are not classified as public figures are considered private figures. To support a claim for defamation, in most states a private figure need only show negligence by the publisher, a much lower standard than "actual malice."
Laws differ in different states, within the US and without. As you can see here[2], the UK has similar rules to the US (which you might expect but I would say not to presume so easily) in that intent or malice, i.e. mens rea, are rarely part of the equation.
So yes, a machine spitting out what would be defamatory statements from anyone else's mouth are still defamation and would land the publisher in trouble if harm could be ascertained. I'm willing to hazard a bet that most people can see a difference between the kind of thing Google Docs produces when used as a word processor to the kind of thing ChatGPT produces.
[1] https://law.stackexchange.com/questions/130/how-is-defamatio...
[2] https://www.blakemorgan.co.uk/a-brief-guide-to-the-tort-of-d...
Who's claiming the statement is true? My website of random sentences that happens to write something about anything is merely coincidental. With ChatGPT it's less random but no less coincidental. It's clearly and obviously fallible and will give you whatever reasonable sounding answer it can.
Infinite monkeys on infinite typewriters will eventually defame everyone.
a) as others have pointed out, ChatGPT is making what a reasonable person would consider a truth claim
b) it’s not entirely random
c) even if it were random as in your example, you’d have a hard time explaining why it was only a few months and not something a bit more like infinity feels to the average person before the defamatory statements popped out. On the balance of probabilities, you failed to implement true randomness or anything like it, and you’d be liable.
What happens when new models have scraped factually wrong data about a person that has propogated, then someone asks the next chatgpt "tell me about person x" and it spits out the bad learned data?
I this it's just an autocomplete on steroids, a powerful bullshitting machine. The fear I have is that this bullshit is going to pollute the digital world pretty much the same way SEO made it impossible to get good results when using a search engine.
When you form a sentence do you catalog every single word in the english language then pick one before saying a word? Or do you have an understanding that for a given word there are only so many words that could follow?
-
AI is a tool. And it's as much a bullshit generator as a human telling a story they definitely completely remember very vividly... https://www.newyorker.com/science/maria-konnikova/idea-happe...
If you misremember the suspect in a case and go around telling everyone the wrong guy committed a crime, you get sued because you went around telling everyone that, not because you misremembered.
That's how markov chain chat bots (a very old technology) works:
https://stackoverflow.com/questions/5306729/how-do-markov-ch...
https://www.baeldung.com/cs/markov-chain-chatbots
That's not how ChatGPT works though, because of the attention mechanism
[1] https://en.wikipedia.org/wiki/Mark_V._Shaney
GPT is still (partially) probabilistic, and the "it's just autocompleting" refrain stems from this idea that being probabilistic without "higher order intent" means a system is just a bullshit generator.
-
The section of my comment you quoted is not comparing LLMs to Markov chains, it's questioning that notion: Obviously we humans don't consciously evaluate [every single word in our language * each word in the sentence]
So the pool of words that we can consciously speak in a sentence is being defined before we apply higher order intent.
If lacking higher order intent is what makes it "just autocomplete", then we're all just interfaces for autocomplete.
-
Complete this sentence with the scariest thing that comes to mind: "We went to the park and it was fun, but there was a scary..."
The specific sequence "mass hippo attack" probably didn't come to mind even though that'd probably be deadlier than what you thought of.
But that's a pointless observation: After all, what are the odds of a hippo attack happening at the park let alone several? A "mass hippo attack" is so unlikely that you might have already rejected my claim since your scary thing is much more likely.
The point is that you didn't consciously compare "hippo attacks" to whatever you thought of until it was brought up.
And that's because don't often mention hippo attacks in our recollection of going to the park... so the bullshit generator wouldn't surface that for our higher level mind to consider.
-
It turns out just having a probabilistic model of our language is enough to align with higher level thought very often. So often that I challenge the notion that higher level intent drives things. I consider the lower level bullshit generator as running the show, and the higher level self is more like a director who can ask to reshoot the scene, but can't just walk up and act out every role on stage as they please.
We all have bullshit generators that don't care if our higher order self is not racist/misogynistic/etc. and will gladly fill in blanks with hallucinations.
What matters is that our higher order self chooses to reflect and evaluate the pool we surface rather than just blurting out the first thing it surfaces. To me using GPT is no different.
I tried to get ChatGPT to summarize a music video today. (Sugar’s “If I Can’t Change Your Mind”)
I expected it might give a bland summary or something, but that’s not what happened.
It invented entire scenarios that weren’t in the video at all, and invented lyrics not in the song.
That’s pretty harmless, but I can easily see ChatGPT inventing some awful story about a person and that being carried over to a publication and gaining a life of its own.
Basically it’s a souped up urban legend generator, but it’s being offered as a tool to provide search results and content. It’s not just an unreliable narrator - it’s an unreliable narrator being offered as an expert witness.
Often times people use LLMs for generating ideas that don't have a factual basis.
ChatGPT will happily invent gameplay mechanics that are fun. It will generate prompts for convincing concept art for something you haven't built.
If what you're looking for can be answered by Google, sure the business people at Microsoft would rather you use a portal that never lets you leave their site... but that's not interesting.
You could throw darts at a spinning wheel with real names and imagined crimes.
The point is that it doesn't matter what the seed for the false statement is, it's the act of spreading it that's problematic.
You're also muddling a point that I can agree with: Treating ChatGPT as an infallible expert is wrong.
But that applies to so many other things. Even expert witnesses are not infallible.
So I disagree with characterizing hallucinations as the problem, it's the application that's problematic.
Blindly and pasting factual content from ChatGPT is a bad idea, just like blindly taking a single source of information as gospel is a bad idea.
Humans can be just as confidently wrong as LLMs, and a simple adage applies to both: trust but verify.
Trust people who have earned trust (either through qualifications or reputation) and treat everyone else as good faith actors who are quite possibly wrong.
ChatGPT should be treated as a person you just met at a bus stop who is well dressed and well spoken but has just told you that you are both waiting for the elevator to arrive at the bus stop.
I wouldn't expect a random doctor approached at a bus stop to accurately answer a question about medicine anymore than I would ChatGPT by the way. Trusting people based on their qualifications and reputations isn't really a thing.
If a doctor tells you to take medication X there's a reason you take that to a pharmacist rather than a store clerk with a key to a safe or something: verifying is always a great idea, regardless of reputation.
Ahahahahaha. Wow. This is brilliant mate. I'm going to start using it.
An "AI" is a tool.
If ChatGPT makes the same libelous claim in its output to 100,000 people, that doesn't seem particularly different from a libelous claim printed in a newspaper with circulation of 100,000.
Microsoft/OpenAI can put down all the legalese disclaimers they want, but if they market ChatGPT/Bing as a tool that provides useful answers, the disclaimers don't protect from libel. By analogy, the NYT can't put a disclaimer in its paper that none of its reporting should be considered truthful in order to protect itself from libel suits -- it just won't work in court. (And yes there are tabloids that print nonsense stories about aliens and Elvis and Brad Pitt, which are for "entertainment purposes only", but the different is the average consumer knows it's a bunch of nonsense, just like The Onion. Parody/fiction is protected.)
So I actually think this is going to be the biggest question/risk by far in terms of commercializing ChatGPT etc. -- much more important than the copyright status of training material.
Because the courts don't decide libel on disclaimers, they decide on harm and how a reasonable person interprets things. If they market Bing/ChatGPT as a useful research tool with advertisements showing it giving correct answers etc. that leads to people believing its lies are true, then there's a real risk libel suits are going to shut the whole thing down.
On the other hand, to make sure they can operate, they may have to market the thing basically as a toy -- a magic 8 ball, a Mad Libs generator. A disclaimer isn't enough, it means they need to avoid any kind of advertising or product positioning that ever depicts it as giving useful/correct information at all. No more homework help, no more trip planning, no more search tool. Which basically sinks the product. But which is also arguably the best outcome -- that Microsoft/OpenAI shouldn't be permitted to market it as anything but a constantly-lying toy.
But OpenAI at least tacitly suggests that ChatGPT can be used for information retrieval. I know they also say that it can generate false information, but its not hard to see how this could cause problems.
Well yes. But what would you do, if ChatGPT, that is used by 100 mio+ people, consistently would asssociate your name with sex crimes?
Would you shrug it off, because "people know it is just a imperfect tool"?
Throw enough dirt and something will stick, so I would try to get my name cleared ASAP from ChatGPT. But apparently there is no working mechanisms to do so. And this is not acceptable.
If people are tinkering with llms in their basement and it spits out nonsense, no one cares. But ChatGPT is offering an online service for laypersond to use. So if they consistently smear people, then they do have responsibility to stop it.
If you believe something that is false, and do harm by expressing that belief, then you are still protected by your right to religious liberty.
In this case, the belief is that coherent language is always constructed using symbolic logic and intent. LLMs do not categorize symbols, perform logic, or follow explicit intent; yet they are able to generate coherent language.
GPT's authors have utterly failed to present this distinction. Was that failure intentional, or the result of their own belief? The result is as ambiguous as the "truthiness" of GPT's output: it can't be reasonably used as evidence for intent.
===== Q. is hackernews used as a training set for chat gpt ?
A. ...it is possible that some of the data from HackerNews and other online sources may have been used in the training of my machine learning algorithms.
Q. So it is possible that ChatGPT's training data can be polluted ?
A. Yes, it is possible for ChatGPT's training data to be polluted or biased. This is because ChatGPT learns from large amounts of text data from the internet, which can contain errors, misinformation, or biased perspectives.
Q. I have a concern. If a certain bad actor pollutes the internet with incorrect information about me, and ChatGPT's training set picks it up, what is my recourse ?
A. If incorrect information about you is published on the internet and is picked up by ChatGPT's training set, it could potentially lead to the AI generating inaccurate or biased responses about you. If this happens, your recourse would depend on the specific circumstances and the severity of the issue.
One option would be to contact the developers of ChatGPT or other relevant organizations to report the issue and request that the incorrect information be removed or corrected. However, this may not always be feasible or effective, especially if the information has already been widely disseminated. ======
> there’s nothing you can do.
Yes, there’s nothing you can do.
If a company starts a news service based on such content and doesn't fact check, they should be liable.
If they don't repeat the claim, then where is the libel?
> It feels very all or nothing - either creators of LLMs are intrinsically negligent with regard to facts, or there’s nothing you can do.
Nonsense. The "facts" are entirely out of their hands. Fact and fiction are written into text using identical patterns. Truth and lie are written with identical patterns. Technical definitions and complete nonsense are written with identical patterns.
LLMs are only capable of modeling patterns. They cannot be directed to choose some and avoid others.
There is fundamentally no way for LLM creators to curate the patterns that exist in the model. The only way an LLM can be altered is from the outside: by giving it different input.
GPT's creators have failed to clarify this. But why? Was it intentional, or the result of their own misunderstanding? Either way, the result is identical: it does not contain evidence of intent.
Considering the surrounding context, I believe it to be an honest mistake. GPT's authors have surrounded GPT with a narrative that personifies it; and from the looks of it, they have started to believe that narrative.
The most important thing we can learn from this experience is that the way something is said may imply certainty, but only evidence can truly provide it.
Just take a look at the GPT-4 introduction: how do they illustrate that GPT-4 is an improvement from GPT-3? It scored better on the SAT! That data wouldn't have ever been taken into serious consideration if OpenAI actually understood its fallacious premise.
Religious narrative is very often influenced by the motivation of religious followers, but that doesn't make the path from motivation to narrative intentional.
https://www.reuters.com/technology/australian-mayor-readies-...
Defamation is a central target if the immunity from liability as a pbilisher on Section 230, so as a general statement it would be harder to be more wrong than this.
I’m not sure what you mean by “phony” or “going through”, but it definitely increases the chance of any defamation lawsuit against the S.230 protected party surviving to any stage of the process beyond an initial demurrer or motion to dismiss.
> It just changes the outcome of a successful defamation lawsuit.
It also changes the length and expense of many defamation lawsuits that would be unsuccessful in any case, by making it easier for the defendant to get them dismissed sooner because they are invalid as a matter of law before even getting to the facts of the alleged defamation.
Legal costs are a good point, though. Defamation lawsuits, even ones that are phony, still present problems by clogging up the court system and incurring costs.
Defamation is a tort, only rarely a crime (and when it is a crime, S230 doesn’t apply, because S230 specifically does not impact criminal law).
And, yes, in an idealized analysis S230 only makes a difference in the final outcome if the court would have ultimately found liability withou it – but that’s, frankly, not a meaningful analysis in the real world. It assumes that all cases either go to trial or are resolved exactly as they would have been had they gone to trial, which is of course not even remotely the case, the overwhelming majority of all tort cases that are even filed, and even larger percentage of all potential tort cases, are resolved by settlement which account for costs, time, and uncertainty of actual trial results, so any consideration which is favorable to one side realistically effects not only the course but also the ultimate outcome of vastly more cases than it would in the simplistic analysis.
> AI producing slanderous results is just not something covered by defamation law to begin with
There’s a lot of bad analysis around AI which starts with the false premise that there an instance of AI software constitutes an entity which is both legally cognizable (so that it somehow serves as responsibility break between a person, natural or corporate, and an action that would otherwise be subject to legal liability) and legally null (so it neither has liability itself nor creates vicarious liability the way, say, a human agent would by way of respondeat superior), when in fact, an AI is no different than any other tool like a hammer or, perhaps more relevantly here, a printing press. If you claimed you weren’t liable for libel because it wasn’t you doing it, it was your printing press, everyone would just laugh at you, but for some reason everyone seems to think that “an AI did it” somehow means no human is on the hook.
> but for some reason everyone seems to think that “an AI did it” somehow means no human is on the hook
Yeah, "an AI did it, not a human" wouldn't be the reason it would be thrown out. A human did ultimately cause it to happen by creating/using a tool, and people often get trapped in the fallacy that an AI's calculations are like an earthquake (or other natural event), or on the other end of the spectrum like an independent human with thoughts and feelings. Both of these views would be pretty legally ridiculous to try to argue (though transhumanist legal framework could change this, hopefully only after it's demonstrated such a framework is actually needed)
What makes it likely to be ignored by the courts is the unlikely series of events needed to meet the standard. There would need to be some sort of false statement purported as fact or negligence, which is difficult because OpenAI is openly saying that they've made an AI product that sometimes spews nonsense and can't be trusted. There would need to be some sort of damages, which is unlikely because most journalists are unlikely to publish AI hallucinations as some sort of whistle-blowing attack on someone's reputation, so it's unlikely to influence a large number of people to believe the claims in the first place. And in some jurisdictions there may need to be malice involved. These standards are difficult to meet even in cases that seem pretty clear-cut. Maybe a future scenario will meet this standard if everything goes wrong.
well seems they did, else we wouldn’t have heard about it
https://algorithmwatch.org/en/auto-completion-disinformation...
Asking ChatGPT for citations seems to produce similarly-poor results to asking it to do arithmetic, perhaps at least when no citations exist.
Is there a general legal preference to use a phrase like
"I have never been accused of sexual harassment or assault by anyone."
focussing on accusation, rather than a broader statement like
"I have never sexually harassed or assaulted anyone, or been accused thereof..."
I mean in general, not specific to this article, or the people mentioned.
Going by the discussions in other topics it seemed this was dismissed outright
https://reason.com/volokh/2023/04/05/washington-post-on-chat...
https://www.gwhatchet.com/2020/04/20/law-professors-ignoranc...
Nobody (if the know the source) would think the pictures Midjourney produces are real fotos, but somehow people think the texts ChatGPT produces are facts or at least based on facts.
Both just generate data most likely fitting the prompt.
BTW do some people attribute consciousness to Midjourney like some do to GPT?
It's related to Tesla's autopilot problem. Branding GPT as an AI sets common expectations innacurrately. It's difficult to argue that inaccuracy isn't, to some degree, intentional among NLP and LLM promoters.
Tesla's autopilot is pretty much autopilot by a pilot's definition. That isn't the definition the public uses. GPT is AI by an academic definition. That isn't the definition the public uses.
What is the definition of AI that "the public" uses, and where did you find it?
There isn't an agreed-upon definition academics use, either. (Intelligence, broadly, remains rigorously undefined.)
But one component of the public definition involves a sense of "knowing," i.e. understanding what is true. This is a source of the confusion and frustration with GPT-4 providing "wrong" answers. The answers aren't usually technically wrong: they're linguistically and logically valid; but the public's expectations of what the model is supposed to be doing doesn't match what it does.
People get confused because they associate language with intelligence, or maybe they are just not technically literate. I don't think we should abandon correctly used terminology because laymen did not catch up to it yet.
This wasn't contested.
> don't think we should abandon correctly used terminology because laymen did not catch up to it yet
No. But the general understanding should be a consideration when marketing a product. (I'd also argue that the conventional definitions of artificiality and intelligence vastly predate the technical definitions of AI. The terminology was always aspirational. Sort of like autopilot.)
I think GPT-4 clearly fits that, so I think the burdon is on you to show that the public has some other widely used definition of AI. Certainly, people seem entirely willing to describe their phone searching for pictures using facial recognition to be an instance of AI, which I would argue is probably further from your definition than GPT-3 is.
Anecdotal, and obviously harder to prove the absence of something than to prove its presence—but from what I've seen, not at all. I've read many good write-ups about Midjourney/Stable Diffusion's use of "latent space"[0] to generate images and general glimpses into the backend of what's going on, even at a simplified level. LLMs, on the other hand, rarely seem to get the same treatment (for a variety of reasons, I'd assume) and tends to get treated as more like a magic black box that may or may not be conscious/intelligent/accurate/etc. Even though it's essentially doing the same thing as its visual counterparts in terms of data and predictive mapping.
[0] https://keras.io/examples/generative/random_walks_with_stabl...
It's similar if midjourney would generate deepfakes.
But with ChatGPT people tend to forget it's creating texts not searching them.
The ChatGPT version is the least harmful in my opinion; sinister are the propagated problems when GPT is utilized under-the-hood as a component in services (such as Bing search).
It would be much better indeed if we knew exactly what the training data was for every given model. But they will still hallucinate things that aren't directly in that data, but could be inferred from it somehow, so that won't solve the problem.