My understanding that all AI can, and will hallucinate. I get downvoted for saying this, but no one ever says I'm wrong or cites any source. Perhaps it's more nuanced than that? Please, enlighten me.
I think its solved, with the right setup and model. I couldn't tell u the last time my agent hallucinated. For me I consider it solved. However some dude feeding massive docs into gpt chat and long conversations, It is not solved in this context.
I'm hoping for more than anecdotes. I believe AI is not ready for what it's being used for, but all I get are downvotes and no citations. Maybe all of you are just scared to say it out loud?
My personal opinion: all output from LLMs is hallucinations. The idea that it’s “right” or “wrong”, when stating factual information is really in the eye of the beholder/user. What year did astronauts land on the moon? — is a question that may or may not have a factual answer depending on your own beliefs. Philosophically, you would need to also define what a fact is, or what “not hallucinating” is, to define what a hallucination is. My understanding is that this property of the LLM architecture is innate, and until we have “world model” LLMs, or some other model that reasons from first principles, instead of the current “guess the next word” model, this isn’t going away. Just don’t rely on the for facts.
AI will never be fully reliable, and people never will be. But AI is wrong in much less predictable and stable ways, and with much less accountability.
There are two common ways to spot hallucinations. Talk to it about a topic you know well, or implement what it suggests and get an error or failure.
For the first, we're mostly past the point of just "testing". So I don't see that too much anymore. Mostly I still see that though in bug reports generated by AI by someone else. There is usually some underlying bug being reported, but the AI explanation and "helpful suggestion" is typically inaccurate. Generally, suggested fixes are terrible. (They likely work, but fix a symptom not the cause.)
The second still happens, but with much less regularity for me though. It does make mistakes though.
In areas where I'm not as skilled it's very hard to spot errors. When researching general information I'm mostly accepting it on face value.
I find the bug-report thing really interesting. For lots of simple bugs it's great. For more complex things it seems to be very superficial- if a 0 causes an issue here, add a simple guard for 0. There's no depth of understanding why the value is 0 in the first place, when it should be set. If it can (incorrectly) be 0 here, where else might 0 be impacting the code?
This informs my opinion of vibe coded stuff - where there is no skilled human inspection. I expect that code to be of a poor underlying quality. Especially if it's AI changes to an existing human-coded app.
The problem hasn't been solved, and, by the very nature of what LLMs are, can't be. However, it has gotten a lot better, as models have got much larger and pretraining more sophisticated.
Part of the difficulty--not in solving, but in discussing--is in defining what a hallucination is. On the face of it, it seems straightforward: an obviously counterfactual claim or manifest error of reasoning. However, it's not always that simple. A lot of what people consider to be hallucinations are misattributions, specious diagnoses, strangely lopsided preoccupations, eccentric design choices, needlessly verbose or circuitous output or explanations, a kind of metaphysical conflation of the trivial with the significant, etc.
You have to be specific. You only have vague comments about your own thoughts. We had this discussion with Tomahawks in the ninetees, it is not an easy one. This is like answering how long is a string.
IMHO This feels like a troll, OP have several comments about evil Ukrainian AI drones. I feel it is hard to have a naunced discussion if you only diss one part.
Your question is asking for easy technical opinions about a subject where you seem to have no technical knowledge.
A couple of things to clarify:
I can understand how you see it that way because I wasn't clear enough.
I don't beleive I characterized what I'm asking in the way you are putting it.
I never said that Ukraine is evil. I competely support and stand with the Ukranian Government and it's people, and I think we, as a country, should be supporting them and their fight to not be conqoured.
Clearly, if I post on a technical forum and am asking a question it is because I don't know the answer. I know how I see it. What I don't know is how everyone else sees it, and that's my motivation. I'm wondering if I'm right or I'm wrong, and have faith that the denizens of the internet will let me know one way or another.
I am asking a legit question and not trying to troll. I am, admittedly, agaist the use of AI on the battlefield, but as it pertains to Ukraine they are defending their existence.
We're being forced to use some AI features in our IDE at work these days, and we see it make stuff up all the time. Very unpleasant to work with. It will tell us implementing something a certain way is impossible, when I've implemented it myself that way in the past. It will fabricate reasons why builds are failing. It will make assumptions about database schemas from thin air.
It's like a science fiction writer or an improv actor doing technobabble. If by chance it knows the actual answer it might use it, but even if not it still has to say _something_ that sounds plausible to a layperson. It'll never say it doesn't actually know, because their character it's acting as _should_ know.
We can check it by asking it about the topics we knew. If it says correct many times , then it has less chance of assuming that this model hallucinates. If it gives wrong answer many times we can take it as frequently hallucinating model.
From my experience it has been largely solved for one significant use case which is chatting to frontier models about the reality as described by public knowledge. 2 years ago models would rely on their training data, today they go out of their way trying to look it up on the Internet and verify thoroughly. I have not had a problem for a very long time.
When working off of limited, private/unverifiable context, LLMs still hallucinate, but again, much less than 2 years ago, and more within a "getting confused where a human would easily get confused" range, rather than "outrageously making things up" range.
It hasn't and likely never will because it is a structural part of how AI works.
You can see proof of this if you ask it obscure enough questions. That doesn't mean obscure scientific questions, I asked it questions regarding ship fits and modules in EVE Online, which is an extremely well-documented videogame. There are hundreds of online tools to help you for different things, mining yield calculators and more.
Well, ChatGPT just made up almost everything. Very confidently. It couldn't even get the damage types right, I was quite shocked.
Hallucination has only be solved for extremely narrow sets of problems, and only partially. Coding is one of those problems.
I don't have a problem with hallucinations, rather the confabulations.
We want AI to make predictions under uncertainty that could be wrong. What we don't need is getting know facts wrong.
One of the best responses I got from ChatGPT was when it said "I don't know", and on questioning it - it responded that it was an unsolved problem and it couldn't objectively take a side iun the argument.
Gemini is the only model that I still see regularly flat-out hallucinate, e.g. the other day it told me we were using a particular technology (Fivetran) on a project when we aren’t and I have literally no idea where it could have gotten that from. No mention of it at all, etc.
Claude will make mistakes but they’re largely “reasonable”. In some ways that’s worse as they’re more believable.
GPT I can’t comment on too much, except to say we run a chatbot using it at my work and when extracting data from the conversation (contact info and such) it will occasionally make up an email address that wasn’t entered into the chat. We have guardrails around it, so it’s not a big deal, but it does happen.
I'd agree with other posters - I'm seeing less hallucination; probably the model-makers are tuning to reduce that, because people make a big deal about it.
I see it as a small deal - it reminds us to check the responses against references.
LLMs are a statistical construction, and everyone accepts without complaint that statistical models have a predicted false-positive and false-negative rate.
I think of "hallucination" as a false positive; false negative is no answer when the model could have made a useful response.
I think there's some relation between our creativity and the LLM capability we dismissively call hallucination.
It's not so much that AI hallucinates, but the fact that we know it does, and it's being used to make independent decisions to use lethal force against a human on the battlefield today.
At least that is my understanding of one of the ways that Ukraine is asymmetrically winning against a foe in Russia with more resources, at least at the start.
It's how we are using it knowing this limitation it has and the decisions being made about when and where it's being used in this capacity are mostly unknown.
29 comments
[ 2.2 ms ] story [ 49.3 ms ] threadFor the first, we're mostly past the point of just "testing". So I don't see that too much anymore. Mostly I still see that though in bug reports generated by AI by someone else. There is usually some underlying bug being reported, but the AI explanation and "helpful suggestion" is typically inaccurate. Generally, suggested fixes are terrible. (They likely work, but fix a symptom not the cause.)
The second still happens, but with much less regularity for me though. It does make mistakes though.
In areas where I'm not as skilled it's very hard to spot errors. When researching general information I'm mostly accepting it on face value.
I find the bug-report thing really interesting. For lots of simple bugs it's great. For more complex things it seems to be very superficial- if a 0 causes an issue here, add a simple guard for 0. There's no depth of understanding why the value is 0 in the first place, when it should be set. If it can (incorrectly) be 0 here, where else might 0 be impacting the code?
This informs my opinion of vibe coded stuff - where there is no skilled human inspection. I expect that code to be of a poor underlying quality. Especially if it's AI changes to an existing human-coded app.
Part of the difficulty--not in solving, but in discussing--is in defining what a hallucination is. On the face of it, it seems straightforward: an obviously counterfactual claim or manifest error of reasoning. However, it's not always that simple. A lot of what people consider to be hallucinations are misattributions, specious diagnoses, strangely lopsided preoccupations, eccentric design choices, needlessly verbose or circuitous output or explanations, a kind of metaphysical conflation of the trivial with the significant, etc.
IMHO This feels like a troll, OP have several comments about evil Ukrainian AI drones. I feel it is hard to have a naunced discussion if you only diss one part.
Your question is asking for easy technical opinions about a subject where you seem to have no technical knowledge.
I don't beleive I characterized what I'm asking in the way you are putting it.
I never said that Ukraine is evil. I competely support and stand with the Ukranian Government and it's people, and I think we, as a country, should be supporting them and their fight to not be conqoured.
Clearly, if I post on a technical forum and am asking a question it is because I don't know the answer. I know how I see it. What I don't know is how everyone else sees it, and that's my motivation. I'm wondering if I'm right or I'm wrong, and have faith that the denizens of the internet will let me know one way or another.
I am asking a legit question and not trying to troll. I am, admittedly, agaist the use of AI on the battlefield, but as it pertains to Ukraine they are defending their existence.
I honestly think it will be impossible to block the use of AI in weapons because you can not know where to draw the line.
It's like a science fiction writer or an improv actor doing technobabble. If by chance it knows the actual answer it might use it, but even if not it still has to say _something_ that sounds plausible to a layperson. It'll never say it doesn't actually know, because their character it's acting as _should_ know.
From my experience it has been largely solved for one significant use case which is chatting to frontier models about the reality as described by public knowledge. 2 years ago models would rely on their training data, today they go out of their way trying to look it up on the Internet and verify thoroughly. I have not had a problem for a very long time.
When working off of limited, private/unverifiable context, LLMs still hallucinate, but again, much less than 2 years ago, and more within a "getting confused where a human would easily get confused" range, rather than "outrageously making things up" range.
You can see proof of this if you ask it obscure enough questions. That doesn't mean obscure scientific questions, I asked it questions regarding ship fits and modules in EVE Online, which is an extremely well-documented videogame. There are hundreds of online tools to help you for different things, mining yield calculators and more.
Well, ChatGPT just made up almost everything. Very confidently. It couldn't even get the damage types right, I was quite shocked.
Hallucination has only be solved for extremely narrow sets of problems, and only partially. Coding is one of those problems.
We want AI to make predictions under uncertainty that could be wrong. What we don't need is getting know facts wrong.
One of the best responses I got from ChatGPT was when it said "I don't know", and on questioning it - it responded that it was an unsolved problem and it couldn't objectively take a side iun the argument.
Claude will make mistakes but they’re largely “reasonable”. In some ways that’s worse as they’re more believable.
GPT I can’t comment on too much, except to say we run a chatbot using it at my work and when extracting data from the conversation (contact info and such) it will occasionally make up an email address that wasn’t entered into the chat. We have guardrails around it, so it’s not a big deal, but it does happen.
I see it as a small deal - it reminds us to check the responses against references. LLMs are a statistical construction, and everyone accepts without complaint that statistical models have a predicted false-positive and false-negative rate. I think of "hallucination" as a false positive; false negative is no answer when the model could have made a useful response. I think there's some relation between our creativity and the LLM capability we dismissively call hallucination.
At least that is my understanding of one of the ways that Ukraine is asymmetrically winning against a foe in Russia with more resources, at least at the start.
It's how we are using it knowing this limitation it has and the decisions being made about when and where it's being used in this capacity are mostly unknown.