Responses from LLMs are not facts (stopcitingai.com)

A simple static webpage, inspired by motherfuckingwebsite.com, comicsanscriminal.com, etc.

ironically, would be more valuable if this cited each of its claims rather than just "further reading". "But Stopcitingai Said "

Bosses love it when you call them foolish.

Sweet of you to think LLM consumers read things.

We used to say "stop copying and pasting from stackoverflow without reading it first". Everything changed, yet everything is the same.

If your boss says this to you, quit.

LLMs are still better than the trash-filled waste bin Google Search has become.

> Responses from Large Language Models like ChatGPT, Claude, or Gemini are not facts. > They’re predicting what words are most likely to come next in a sequence.

I wish we'd move away from these reductive statements that sound like they mean something but are actually a non-sequitur. "Articles on Wikipedia are not facts. They're variations in magnetic flux on a platter transferred over the network".

Yeah, that doesn't make them not facts, though. The LLM should simply cite its sources, and so should Wikipedia, a human, or a dog, otherwise I'm not believing any of them. Especially the human.

Even in small companies, its important to discuss what the expectations around AI are. In the absence of any further requirements (i.e. assuming privacy is not a major issue, regulatory issues etc), it can be as simple as clearly saying: "You can use AI, but you are ultimately responsible for what you deliver. It is expected you verify the data, test the code, and otherwise validate the responses."

Something as simple as that gives an expectation, without being overbearing to start with.

This snarky site may make you feel smart but really there’s no reason to cite and trust anything, and AI isn’t much worse than alternatives. Even peer review isn’t the guarantee you think it is. AI is often right as well and we should keep that in mind.

I think my main response to these messages is: “If ChatGPT is more trustworthy than me, the expert you hired, what do you have me for?”

I can spend hours refuting and explaining why what ChatGPT told you doesn’t apply in our situation, or you can… just trust me?

> Imagine someone who has read thousands of books, but doesn’t remember where they read what.

That sound like me! Well I probably read only a hundred, but I also mostly forgot the sources. I can halucinate a source, like "there is (probably) a Schaum book about that".

Alternatively, give the same prompt to another model and get a completely different answer. Sometimes the opposite. Or give the same prompt to the same model after its latest fine tuning and get a completely different answer. Or warm up the model with leading prompts and get a different answer.

These things are just addictive toys, nothing more.

"Can you share the chat so we can look at it together?"

Asking for the receipts so you can figure out where they put their thumb on the scale is more illuminating.

Is this lmgtfy of the AI era?

No, don’t do this. It’s as bad as the “no hello” thing.

If it bothers you when people do the “chatgpt said” thing (and it should), put your concerns into your own words. Or at least respond with an article in the news that you can discuss with that person.

Responding with one of these sites is just as worthless and devoid of interpersonal investment as responding with AI. Don’t be that person.

This is so passive aggressive it’s fireable—or at the very least unpromotable.

LLM's follow the old adage of "Garbage In, Garbage Out". LLM's work great for things that are well documented and understood.

If you use LLM's to understand things that are poorly understood in general, you're going to get poor information because the source was poor. Garbage in, Garbage out.

They are also terrible at understanding context unless you specify everything quite explicitly. In the tech support world, we get people arguing about a recommended course of action because ChatGPT said it should be something else. And it should, in the context for which the answer was originally given. But in proprietary systems that are largely undocumented (publicly) they fall apart fast.

I remember the guy who created Comic Sans said, “If you love Comic Sans you don’t know anything about typography and should get a new hobby. If you hate Comic Sans you also don’t know anything about typography and should get a new hobby.”

I feel like this applies to AI as well.

Remind me this useful URL when boss says "we need carousel".

https://shouldiuseacarousel.com/

Tell me you know nothing about AI without telling me you know nothing about AI

I know its disturbing to many, but there is something nice about the post-truth moment: it feels like more people are actually questioning things more than when I grew up in the 90s/00s.

I think we need to shift towards a socionormative understanding of knowledge; as Rorty put it: "a fact is just something we can't be bothered to argue about". I agree with him that talking about truth isn't so useful for moving our culture forward.

We should be talking about how to negotiate the diverse vocabularies of discursive communities as they increasingly clash in our globalized culture. Dialectical exclusion is the cultural catastrophe of the day.

This feels like getting taught in school not to cite Wikipedia when the actual digital literacy challenge is deeper— learn where the info comes from and to critically think.