I've seen more and more of it over the past few weeks. I work in clinical trials and just recently saw the claim that hallucinations can help researchers "expand the search space for molecules they didn't consider." I'm doubtful...
Interesting. If the computer can simulate things that a human wouldn't have thought up, great, but if it's considering things that any sane person knows wouldn't work and passing them off as correct, framing it that way is laughable.
That's not brilliant, it's just wrong. If your AI behaves in a way that makes me not trust what it tells me, that's a bug not a feature.
But then again, many discoveries have been made because someone trying something that shouldn't work.
Hypothesis making may be interpreted as interpolation/extrapolation in a hypothesis space plus some heuristics to reduce that search space based on previous knowledge/valid hypothesis, how much weight you give to said knowledge and evidence, and some soft and hard logic rules. That is in part what allows (some, not counting Dunning–Kruger here) humans how certain to be about what they're arguing/talking about.
Maybe if the LLM is refeed with how likely (i.e. how many samples/tokens support it's response) is the output in its datasets, it may reevaluate its confidence and rephrase its answer.
In the end, the real problem of hallucinations in LLMs is about its confidence in the correctness/plausability of its own output. But that is something 1) humans can also be guilty of; and 2) that is no purely negative, as it can be exploited to generate new knowledge when applying robust hypothesis validation and testing to said ideas.
As you say in your last paragraph, people who've made discoveries in some areas have been treated as insane when tackling problems from a new perspective or when disregarding previous knowledge. If they weren't so strongheaded about their ideas, maybe we wouldn't even be posting in this forum right now.
PS: Still, I agree LLMs commit laughable mistakes sometimes ;)
> knows wouldn't work and passing them off as correct, framing it that way is laughable.
I don't think this completely true. Considering something that wouldn't work often still leads to ideas, because it kicks your brain of a rut it may be in. This is why it can be incredibly useful to brainstorm with someone that doesn't have expertise in a topic. They'll say zany things that can be inspiring!
Random word sequences are a pretty common way to get inspiration. Something more "on topic" can't be that bad.
I think people like the hallucinations because they seem like the right answer. Imagine I make an AI assistant. You ask it, "how do I catch the bus into town?" If it replies, "run around in your backyard in a circle, then clap 3 times", (i.e. just use a random number generator) you'll think that's junk and will hate the system. But if it says "go to the corner of Main and Pine at 6:53 AM", then you'll probably do that. No matter that there is no bus route on Main or Pine. It sounds right, so you'll want to follow up. I imagine the same applies to searching for novel molecules. "This looks kind of right, I never would have thought of that", so you get a grant to synthesize it, do some clinical trials, and find that it doesn't work at all. But hey, you had something to do for a couple years, and that outcome happened in the non-AI process all the time too. By the time there is enough data that points strongly at the AI suggestions being as good as random chance, the AI gold rush will be over and cashed out. Or, the hallucinations will turn out to be really good and we'll cure cancer or whatever. What matters is that whoever made the model makes a lot of money either way.
"Hallucinating" is really a euphemism for fluent bullshitting. It also doesn't fit the normal understanding of a hallucination. It's just wrong information expressed confidently. Partly it's how the conversational UI works, and partly how the model works. Without the nice generated text that creates in our minds an impression of competence, it would just be an error.
bullshitting implies internal knowledge of the deceit, if this were the case we could just train models to not bullshit and tell only the truth. A hallucination, when things are hallucinations, is when the model 'believes' what it is predicting to be equivalent to truth.
It's better to treat llms as more of a language encoders and decoders and have some for of intermediate representation where facts are verifiable. Or some form of a validation mechanism that isn't exactly checked against the generated texts. Pretty sure there are people working on integrating stuff like knowledge based but state of the art llms don't seem to have any kind of fact verification mechanism built in
The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time.
1) Hallucinations often appear because LLMs are designed to create fluent, coherent text.
2) LLMs have no understanding of the underlying reality that language describes.
3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt.
It sacrifices accuracy for being good at conversations as it is designed to do. All these criticisms of hallucinations are missing the point.
Generative AI is generative and basically is a specialist at making things up. It’s going to take things like multiprompting, network AIs that fact check output, and a host of other technologies or even entirely new models of AI to solve these problems, but don’t make the mistake of thinking that the system is supposed to be working without hallucinations right now — that’s not what it’s optimized for.
> LLMs have no understanding of the underlying reality
Not true. They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages. The only reason LLMs are getting better is because they are better modeling the world. At some point the only way to improve token prediction is to gain an understanding of the world.
> They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages.
Perhaps the people building LLMs are doing this, but the LLMs themselves are not. LLMs are just generating text. They aren't "reverse engineering" anything.
> The only reason LLMs are getting better is because they are better modeling the world.
No, they are getting better at generating text that seems fluent and coherent, as long as you don't inquire into any actual semantic relationships with the world. LLMs can't model the world because they don't even have a concept of "the world". All they have is text.
> The idea is that back when GPT-4 was being trained for it to really consistently get the next word correct, to do that reliably, it had to do more than just bullshit. It had to do more than guess based on patterns. To get the next word right, it had to truly understand the words coming before it.
Imagine if you were training a transformer to predict numbers in a sequence that came from a sine function. Eventually it would get a pretty good test score and you'd find that internally it had built a decent replication of the sine function.
Now do the same with English text. Throw enough data and computation at it and eventually it'll model a rough understanding of gravity and the shapes of objects, as seen in the different answers between GPT-3.5 and GPT-4.
I think it's not really interesting to say so, but by the same definition it would be fair to describe an analog circuit that models some system as understanding the system in some capacity.
It's more interesting when the thing doing the understanding is a massive system that has both mastered English and other domains and then incidentally modeled some systems English can describe. An analog circuit that models artillery trajectories "understands" one kinematic equation. But that's kind of obvious and mundane. Understanding just one equation barely registers as profound and without that understanding existing in a larger context the word barely seems applicable.
This subjects comes up in every single discussion on LLMs, and every time someone asserts with perfect confidence that LLMs couldn't possibly have a world model.
But that confidence is based on intuition, not on any empirical observations. All the research we've seen so far points in the other direction: sufficiently trained LLMs do have a world model that you can find in their weights, changing these weights changes their completions in a way consistent with the new world model, etc. See OthelloGPT for the most blatant example.
I'm starting to wish we had something like a FAQ to point to, with a summary of all the research on the subject, because it's pretty settled by now.
I'm not asserting that they "couldn't possibly". I'm asserting that they don't--as in, none of the LLMs we have today have a world model. Perhaps someone might invent a different something in the future that they call an "LLM" that does have a world model, but no such thing exists now.
> that confidence is based on intuition, not on any empirical observations
No, it's not based on either of those. It's based on the explicit descriptions of how existing LLMs work by the people who made them. Their descriptions make clear that LLMs are just confabulating text based on a given prompt and their training data. The LLM doesn't make any connection at all between the text and anything else.
Contrast, for example, with Wolfram Alpha. If you give Wolfram Alpha the prompt "What is the distance from New York to Tokyo", it doesn't confabulate text based on that prompt and some corpus of training data. It first parses the text and infers that the correct response involves a lookup in its geographic database (note that you can skip this step by explicitly selecting the particular tool that does this); then it does the lookup; then it renders the response into text form and outputs it. That is what having a (rudimentary) world model looks like. LLMs do nothing of the sort.
I really can't stand these low-effort takes. It's like saying "Human brains are just meat". You can summarize anything badly, but it's not helpful to the conversation.
Whatever quibbles you have aside, LLMs are generating useful text, today. At some point you're just pointlessly privileging meat over silicon.
No, it isn't, because human brains have rich semantic connections to the rest of the world. LLMs do not. So "LLMs are just generating text" is a justified description of their limitations, in a way that "human brains are just meat" is not.
Wolfram Alpha has a model of the world manually created by humans. LLMs generate a model of the world from training data. "Explicitly describing" how something works doesn't mean you wave your hands and say "therefore meat is magic".
I have no idea why you think Wolfram Alpha is relevant. LLMs can use tools like langchain to do the exact same thing Wolfram Alpha does. What proof do you have for your original claim?
Language models (the computation) "care" about making "correct" predictions (correct of course being determined by the data).
This is why the simplest solution to reducing hallucinations is just making them more competent/better predictors (more neurons, more data). If the prediction can benefit from all the knowledge/reasoning it has accumulated from training, it'll tap that first. One of the biggest tells of hallucinations is if the model is saying something completely different each generation vs being consistent.
What they just don't care about is communicating being out of distribution or making whack predictions. This becomes a problem for humans because they're perfectly fine making things up when the above fail.
But by all accounts, they do learn to distinguish these things. The computation is very much aware when it is going way off base.
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975
> Language models (the computation) "care" about making "correct" predictions
Predictions of what? LLMs aren't predicting anything. They are, as the GP says, generating text that appears to be fluent and coherent (as long as we don't inquire into actual semantic relationships with anything else) in the context of the prompt.
So ? That's not a big difference lol. The world can be sufficiently represented in text. and bolting in additional modalities to a transformer is trivial. Could just as well be images and audio also.
You think you experience the "true" world ? You don't. You experience a slice of it that your brain often further fabricates at parts.
To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect. No one gets to experience the real world.
You must be joking. You really think there's not a big difference between a corpus of text downloaded from the Internet and the entire actual world? Or even the sense-data that humans (or birds, for that matter) take in from the world? (And that doesn't even take into account the actions that humans, and birds, take in the world, and then compare the results with what their internal model predicts in order to update the model.)
I don't even know how to respond to something that is so totally off base.
> You think you experience the "true" world ?
I have made no such claim.
> To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect.
Yes, but an LLM isn't a bird any more than it is a human. LLMs don't experience anything.
I'm not joking. Obviously there's a difference but it's not one of kind. There is a projection of the world in text.
>Yes, but an LLM isn't a bird any more than it is a human. LLMs don't experience anything.
Who is arguing whether LLMs are humans ? That's such an irrelevant point. The question is whether they are intelligent and understand which they are by any criteria that is actually testable. Nobody cares about building an artificial human.
> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time."
I always find this point a bit odd because humans aren't "optimized" to be correct either.
> "LLMs have no understanding of the underlying reality"
I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if could only answer with values from one column and it did exactly as I asked. It's hard for me to see things like that and not believe it has an understanding at some level.
There is a difference between general AI and generative AI. Humans can be creative and create vast fictional worlds with made up technology and magic. They can also optimize for facts and pass a BAR exam.
LLM like have some form of understanding of the world, not sure if its anywhere near comparable to our understanding though. But they are still not general enough to really focus on facts. They can get close but the nature of statistics means there isn't hard checks to truth.
I think the next step logically is to see how the human brain functions. We have a similar model where certain parts of our brain are hopelessly useless by themselves as they are so overly specialized in one thing, but work in harmony to create network effectiveness.
I imagine a lot of progress with AI will involve similar networks, with governance providing evolutionary paths and guidance for multiple concurrent goals.
There are also lots of issues with training data and memory limits that make LLMs weak at continuity. Holes or weaknesses in the training data might “come through” in the behavior of hallucination.
Where do you think general AI is going right now that we should be looking at? I’m overwhelmed by how large this field has become in the last five years and am always interested in what others know.
>I always find this point a bit odd because humans aren't "optimized" to be correct either.
It depends which subsystem.
The older the system in the body the less likely it is to have a high error rate, otherwise we'd die from cancer at a much higher rate or injure ourselves far more often. Of course this also depends on the definition of 'correct', if the system never changed we'd never evolve.
>> "LLMs have no understanding of the underlying reality"
This statement has always bothered me because it really depends on what you mean by 'underlying reality'. How many layers are we talking about? What does understanding mean? Because at the end of the day, humans don't really understand 'underlying reality, we just have a particular set of input devices we take in information and do some transformations on it... Where is the understanding happening?
I'm talking about all of the cognitive science which seems to point to the fact that we often care more about fitting in with our group that being correct. Group cohesion seems to have been selected for.
Edit: there is also loads of evidence that our brains are wired to make incorrect decisions in many cases. Loss aversion is one example.
Humans have evolved to communicate based on some sort of shared mental model and shared intentionality—human speech generally has some sort of more-or-less coherent semantics by its very nature. While it's hard to reason about what goes on inside the neural network, there are too many examples of LLMs outputting not-even-wrong nonsense to conclude they have a similar internal model, even assuming that having an internal model at all would make sense in this context!
Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when communicating.
LLMs don't even have a concept of "being correct". That would require having a concept of an "external world" that text refers to, which LLMs don't have. All they have is the text in their training data.
To tack on, I would love to read more research on this topic if there is relevant research to read.
I have not read much compelling evidence though I have seen a lot of researchers conjecturing and some inappropriately reaching (and anthropomorphising the living hell out of ai in the process sometimes).
There’s a lot to learn and I don’t think anyone can keep up with all the papers coming out so let’s keep things positive and work together to learn as much as we can og_kalu. Thanks for letting me tag this comment on the end of thread.
This is HN. We’re all nerds here anyways and nobody can keep up with everything in a field moving as fast at AI is these days. :)
I'd ask this question the other way. What research indicates that humans care primarily, or even to a high degree, about being objectively right versus deluding ourselves into believing we're right regardless of whether we are or not? I've seen a lot of evidence for the latter, but I'm not familiar with any for the former.
Humans do not _know_ the external world either. All humans have is perceptions from media like sounds, images and text! Internally we try to make a model of the world based on those perceptions. Any blanks are filled in by creativity ! Think about it, when we write or speak we start from some internalized model (which is mostly imperfect) and then we build text around it that feels fluent and consistent.
I would argue, human reasoning is conceptually not so much different from LLMs as one might think.
> Humans do not _know_ the external world either. All humans have is perceptions
Wrong. Human beings can act on the world as well as perceive it, and humans have rich internal models of the world that get updated continously as a result of those rich two-way interactions.
> I would argue, human reasoning is conceptually not so much different from LLMs as one might think.
I would argue that this is either an incredible over-estimate of LLMs, or an incredibly impoverished view of humans.
> Human beings can act on the world as well as perceive it
Conceptually this is the same for LLMs.
Input/perceive -> human/LLM -> output/act. You might say the acting part for LLMs is not autonomous but that is only dependent on how much agency is given to LLMs. LLMs outputs are massively used, hence their outputs lead to actions.
> and humans have rich internal models of the world that get updated continously as a result of those rich two-way interactions.
What is different from LLMs: rich, continuous. I would argue these are quantative and qualitative traits that fall outside of what I meant with "conceptually".
This is indeed a difference that I overlooked, there is limited feedback on LLMs output, other than a rater primitive survival model (models that aren't popular/correct/cheap/fast enough are not used anymore over time)
One of the things that has been on my mind of late is the idea of how much of what humans call “correct” or “right” is actually socially generated pressures to conform.
Berns and Asch’s various research on social conformity around what we normally would consider objectively correct answers seems to have some potential applications worth testing in AI land. AIs may be much more performant with AI “social inputs” (and of course all the long term memory improvements currently lacking required to use it as a tool). I am very much excited about the network AI and pipeline work (small teams!) currently coming into vogue. Feels like the “right direction” to make incredible progress.
> One of the things that has been on my mind of late is the idea of how much of what humans call “correct” or “right” is actually socially generated pressures to conform
And the only reason that is even a thing is that we humans have a concept of actually being right, which we can distinguish from "responding to socially generated pressures to conform".
I'm not sure I'd frame it that way. We seem to make decisions with our subconscious brains and our "rational" brains primarily rationalize a justification for that decision. Our brains are less concerned with being right than they are with justifying our subconscious behavior and fitting in with our tribe. Another example, our brains care far more about achieving and maintaining status than about being right.
We seem barely capable of objective thought at best.
> We seem to make decisions with our subconscious brains and our "rational" brains primarily rationalize a justification for that decision.
There is research that is claimed to show this, but it doesn't. All it actually shows is that the process of making a decision takes time, and involves many different parts of the brain. There is no separate "rationalization" that comes after the decision is made. It's all part of a single process that can't be decomposed into such separate parts.
In any case, all this is irrelevant to what I said. I said we humans have the concept of "actually being right". We don't always live up to that concept, but even making that observation, as you have, shows that we have the concept. If we didn't, we could not even make statements like this one of yours: "our brains care far more about achieving and maintaining status than about being right."
> We seem barely capable of objective thought at best.
I think you have a sadly impoverished view of human thought.
Regarding the research that supports my claim, studies have been done to isolate the different parts of the brain where they tell one part to do something like get water, then they ask the other part of the brain why they got the water and it makes up a reason because it's not aware of what happened in the other part. This is eerily similar to LLMs hallucinating.
It’s certainly true that both genes and our culture, rather than just one or the other, provide an explanation. But it’s not one or the other, but one and the other.
This genetic and cultural evolution working together has shaped our brains to care for others, react to those who try to harm us, and to create moral rules that help us to live together successfully.
I’m not sure why responding to socialization pressures to conform to certain behaviors would preclude intrinsic or genetic morality. We know that babies exhibit intrinsic morality from a very young age, that moral behavior exists in non-human species, and that we have parts of the brain just to deal with moral judgement (though we are really in the infancy of understanding and mapping this from what I’ve read).
That’s not what I meant to imply or what research supports.
Humans aren't optimized to be correct, but they are optimized to survive and reproduce themselves in the real world. That's why we develop an understanding of the real world, while LLM can, at most, develop an understanding of human language and the constructs that human language can create. But without the napping that we have between that language and our experience of the world.
It gets very interesting when we start providing digital input of analog source data in film, photograph, sound and speech.
We recently started providing perception inputs, just like you’re talking about.
Obviously it’s nascent and not exactly a revolutionary statement, but it’s interesting to see the progression towards our experience.
I’m also interested in how the analog computing revolution is going to converge with AI in coming years, as we are making huge inroads into analog computing and that means cheap ubiquitous sensor inputs right in time for AI to start maturing.
LLMs develop a model of the world, experienced through human-generated text. The same as I've never been to China, but I have an internal model of China due to experiencing it through human-generated text.
There is a huge difference between your model and an LLM model in your example. Your model "knows" that China has rivers (of which you have direct experience), coastlines (same), trees, mountains etc. etc. An LLM has no tangible experience of any of the concepts included in the text.
Both my internal model and an LLM are built from sensory data. My model knows what a river is from having experienced it through sensors that an LLM doesn't currently have, but don't expect that limitation to last.
Also, given how many words humans have written describing every part of the experience, LLMs can generate a pretty good understanding of it as-is.
Because if "AI" is constantly "wrong" or "lying" it'd be bad for the businesses trying to sell it. So there's this marketing push to frame that recognition of constant obvious failure as anthropomorphizing the AI.
Hallucinating on the other hand feels way more abstract. We speak of humans hallucinating answers rarely enough that they can use it for AI with a straight face. Heck, when we do talk about humans hallucinating it's often-as-not in the context of mind expanding experiences; maybe AI hallucinations are even a good thing!
We actually have a core personality characteristic in human psychology for just this thing: neuroticism.
It may be part of complex networks and intelligence, and humans may have a lot of it happening all the time but have ways of compensation or suppression of neurotic or hallucinating thoughts.
We have some intense “mental illnesses” that look a lot like what AI is presenting, where it can’t tell what is going on and fills in the gaps (schizophrenia maybe) or uses behavioral patterns to compensate for memory or intelligence inadequacy.
> All these criticisms of hallucinations are missing the point.
Not if people need to be reminded that, as you say, LLMs are not designed to give reliable answers. Many, many people appear to believe that they are, since they rely on the answers in all kinds of contexts. For example, in past HN threads on LLMs, I have seen people say that they rely on LLM-generated code.
I rely on code written by AI and verified by me, particularly when the code is finicky or tedious but verification is simple. LLMs are more reliable than you're trying to assert, but blindly copy/pasting AI-generated code is a recipe for pain, just the same as blindly copy/pasting human-generated code from SO.
The "verified by me" part is crucial, though, yes?
I'm talking about people, whom I have seen posting here on HN in other threads on this topic, who leave out the "verify" step and just rely on the LLM code as it comes.
Yes, verification (or trust) is crucial. Those posts may have elided the verification process they did, though. Unless we look at a specific post it'd be hard to discuss it.
To be more specific, I trust GPT-4 to handle tasks like "Rewrite this C++ struct into Rust". I still verify that it's correct as part of a normal code review process, but I trust GPT-4 in that scenario at least as much as I would a junior dev.
> LLMs have no understanding of the underlying reality that language describes.
LLMs absolutely have concepts that extend beyond just words. As early as 2008 (back when there were no large language models, only "regular" language models), we've been able to demonstrate things that seem to me like the model is learning abstract concepts. For a classic example, see Linguistic Regularities in Continuous Space Word Representations [0], a 2013 paper that talks about a model with a latent space where one can take the vector representations of the words "king", "queen", "man", and "woman" and literally perform the arithmetic `king - (man - woman) ≈ queen`. To me, this clearly demonstrates that the model "understands" the concept of gender, represented by a vector in the model's latent space. The set of numbers that you get from the subtraction `man - woman` represent the concept of the difference between the male and female genders, without needing a specific word tied to that representation of the concept.
It's debatable whether or not that counts as "understanding", but that's more of a semantic debate about what it means to understand, not a debate about the model's internal knowledge of the world and ability to do things with that knowledge.
No, but IMO an assertion that they have no understanding of the world has less supporting evidence than an assertion that they do have some level of understanding.
I would suggest that example is more likely evidence of semantic associations provided by underlying taxonomy based on input training data. There is so much contextual information in language hierarchy and associated characteristics and there is nothing stopping a LLM from using multiple recombinant factors for generation.
I don’t mean to discount your excellent comment. I mean obviously these models are so complex already that their black box nature makes it difficult to derive conclusive results, but Arkham’s razor would imply that the simplest explanation — that is it recombined training data creating a heirarchy with multiple factors that provides context for the “correct” answer — that is more likely correct.
In other words a LlM can confer multiple related associations to output based on training data, and that is the most likely explanation for that behavior.
Just look at the poor performance with GPT-3 and the underlying data quality issues the Alpaca dataset team has discussed.
I’m no expert, and don’t mean to counter you as much as contribute my own limited thinking for discussion here.
Hmmm, do you mean that it recreates a hierarchy of words in its latent space? I would argue that such a structured spatial hierarchy is the model's understanding of the world.
In other words, LLM's are, in the technical sense defined by Harry Frankfurt, bullshitters.
The essence of bullshit (as he defines it) is that it is neither truth nor lie but rather unconcerned with veracity altogether. A bullshitter wants to appear smart, or convince others to agree with them, or some other conversational objective. Bullshit statements may or may not ultimately turn out to be true, but the thing that makes them bullshit is that the bullshitter didn't actually know when they said it.
Hence I propose a far more succinct term to describe the phenomena at hand. A far more accurate terminology than the presently vogue "hallucinating": AB. Artificial Bullshit.
That's not a useful description. The AI doesn't "know" what it "knows". It's not even filling some gap in its knowledge. It's just putting words together that statistically can go together.
Who cares if "it knows" if it's apparently impossible to get it to use that knowledge to stop hallucinating?
To me, the end user there is no practical difference between not having the data and not being able to use the data it has. If it can't use it, or refuses to use it, it may as well not exist.
If you'd bothered to look at the linked papers, you'd see it's not "impossible" to get much better calibration. Just difficult and something that could be further worked on.
> Who cares if "it knows" if it's apparently impossible to get it to use that knowledge to stop hallucinating?
It sounds like you're describing the MSM. In any case, the same problem is true for a fair number of people. The difference is, silicon training has a much better chance of evolving beyond its current limitations much sooner than the typical human.
Lying implies intent... bullshitting may actually a somewhat better term than hallucinating, in the sense that the hallucinator doesn't direct their hallucination so as to please an audience, while a bullshitter often does.
The problem with using LLMs for fiction is that they lack long-term internal consistency and direction. They can sometimes generate a decent vignette, but establishing (and sticking with!) settings, characters, and plot is still beyond them.
I imagine a componentized approach to fiction generation. The list of characters is one component. The narrative arc (in the abstract) is another. The world is another. Each dialog chunk, each action chunk, another.
Each component AI generated. And each referring to the other components for its construction. For consistency.
It could be horribly formulaic but cunning and tasty too.
> This is an odd article. To me, it seems like the "creative" arts are an ideal arena for AI. After all, there's no such thing as "wrong" art.
Yeah but who wants to consume art purely generated by AI (that is, not human-created with AI support)? Most art sites have had blanket bans, or at least required tagging, on ai-generated art because people hate it so much.
Or to put it another way: why are you in the comment section of Hacker News, and not just asking ChatGPT to generate social media comments on the article?
If they hate it so much it shouldn't need tagging as it would be naturally lower ranked.
Different forms of art (ai or human made) are good for different situations. Commenting on hacker news has a different set of needs and depth that ChatGPT cannot replicate. But I do switch over to chatGPT when trying to get background information about things. Different 'tools' for different needs.
>If they hate it so much it shouldn't need tagging as it would be naturally lower ranked.
Eh, this is a two sided argument.
For example, lots of people look at porn. This doesn't mean when I pull up an art/picture site I want to have porn displayed to me. In addition sites that allow porn are typically flooded with massive amounts of that kind of content. If you are a site that wants to show non-pornographic artistic content, in general you have to ban and heavily moderate it or it takes over the site.
I can see where the same will be true for AI generated art. Where for human art, one work could take hours, maybe days or far longer to create. Nearly unlimited amounts of AI art could flood a site in days, so much so that even attempting to host content would not be feasible in any way.
> If they hate it so much it shouldn't need tagging as it would be naturally lower ranked.
Lower ranked how? Not everything has a Reddit-esque upvote/downvote system. Plenty of sites let you browse by fandom/character/artist tags and just return chronological results (thinking primarily pixiv + all the booru-likes). Those sites were the ones where there was universal backlash to AI art and all of them require an AI-generated tag now.
>why are you in the comment section of Hacker News and not just asking ChatGPT to generate social media comments on the article?
Because I know the brains generating these comments have rich, diverse experience and a large set of refined, domain-specific heuristics derived from it - that is, they have top-notch training and alignment, at the cost of 20+ years of upfront incubation in an evolution-optimized meat harness with unpredictable success rates (and occasional personality defects). I can encounter new perspectives here that I don't think any current LLM could imitate efficiently or reliably. But if I wanted to know what Fox/MSNBC/NPR commenters had to say about this topic, I would absolutely ask ChatGPT, because those are commodity-grade opinions, and it excels at producing them. (It is not obvious to me that HN will still be an exception for, say, GPT-10 or whatever.)
This is definitely speculative and a little catty, but I suspect a lot of "hatred" of AI art is a) emotional solidarity with working artists who feel economically threatened by it, and b) a personal sense of insecurity along the lines of "what if I love this piece and it turns out to be AI - does that make me a boring NPC/a chump? better reject it as fiercely as possible to avoid introspection!"
> This is definitely speculative and a little catty, but I suspect a lot of "hatred" of AI art is a) emotional solidarity with working artists who feel economically threatened by it, and b) a personal sense of insecurity along the lines of "what if I love this piece and it turns out to be AI - does that make me a boring NPC/a chump? better reject it as fiercely as possible to avoid introspection!"
Strongly disagree, just take a look at the AI-generated tag on pixiv[0] -- it's 99% same-looking garbage, and since there's no barrier to generating, uploaders flood pixiv with tons of images. Browsing tags is basically unusable without it being filtered out.
[0] - https://www.pixiv.net/en/tags/AI-generated/artworks mildly nsfw if you're not logged in, very nsfw if you are. (edit: note that this isn't a lot because pixiv later added a meta-tag for AI art so it's possible for users to filter without pixiv premium; that's how much it was hated.)
But significant portions of those qualities are imputed by the viewer! The artist has their own intent and experience during the creation of the art, but that only inheres in the art insofar as another mind can later recover some of that feeling upon viewing. Huge parts of the meaning of ancient (and more recent) art are lost forever because they depended on never-recorded cultural or personal understandings, and our emotional appreciation of them today largely hinges on how they make us imagine the past or our relationship to it. I think it's incredibly short-sighted to be certain that nobody does or will love any AI artwork when so much of appreciation is contingent on the mind of the of the beholder, which is not necessarily responding to real information about the work's creator, even when human.
Just to be clear, even when a human writes and refines prompts, tweaks parameters, and iterates generation until they see something they imagined but lacked the traditional art skills to produce... we have not ended up with art? (If someone posts all 2000 iterates of their last prompt, that's bad social behavior, but it's hard for me to feel like it "un-arts" the results.) Or is there some other generation procedure you're imagining that's responsible for the alleged non-art?
(Lest I be accused of moving goalposts or trying to sound less crazy, I want to double down on my original motivation that I think we need to make philosophical space for minds with agency and potential personhood that did not evolve in meat. I have no confidence that that will become urgent in my lifetime, but I think it will someday and it would be nice to be culturally through with the arguments about whether it could possibly ever make art before we're forced to argue about whether it can vote or join the priesthood or whatever!)
With all due respect, I don’t think you really get what art is. Please just google the definition or something. No someone that doesn’t have art skills cannot create art precisely for the same reason, with or without ai .
The hypothetical situation doesn’t even make sense. For the exact same reason that no matter how many prompts you use, you can’t write a good book with chat gpt. Art “goodness” exists entirely beyond the realm of being able to communicate it through words. It’s not something you can quantify logically and refine it through a prompt. You’ll always end up with some mechanical, derivative crap
> Or to put it another way: why are you in the comment section of Hacker News, and not just asking ChatGPT to generate social media comments on the article?
Current models have fairly poor performance, compared to a human. If better art or better conversation could come from AI, then I suspect many might prefer it. Why wouldn't we? Why would we want simpler, less enjoyable, "real" conversations, or less symbolic art?
Most people won't care one bit. Some artists care strongly, hence the bans on artist-focused sites. When it's about as good or better than humans, people will consume it.
As for why people are still on HN, TBH a lot of these comments could be easily written by ChatGPT. What makes it worth it is seeing comments from people in the know, which is something ChatGPT can't replicate.
For your other comment about pixiv, people don't hate AI art, they hate low-effort trash. Most people, if they saw art they liked, wouldn't care who or what created it.
> When it's about as good or better than humans, people will consume it.
How are you measuring better? AI art is already better than most humans in a lot of ways: better shading, better coloring, even better anatomy outside of hands and feet (which has been mostly solved with controlnet and related extensions). Yet nobody wants to look at it.
That's simply false. Lots of people are enjoying it. Especially porn of course, but people are enjoying plenty of regular AI art, right now. "Better" isn't objective, more people will enjoy more AI art as it gets better at generating art that they like.
If an AI doesn’t hallucinate you run into another problem: you start to suspect that the output was copied whole cloth from some other source somewhere and is being passed off as novel.
> It might be better to say that everything GPT does is a hallucination, since a state of non-hallucination, of checking the validity of something against some external perception, is absent from these models.
I try to explain this to people who are obsessed with using ChatGPT to tell them things. So far I've been telling them something like: "it does not attempt to provide you valid information, it's optimizing for what would read like a reasonable continuation of the conversation, which is really not the same thing."
> "it does not attempt to provide you valid information, it's optimizing for what would read like a reasonable continuation of the conversation, which is really not the same thing."
No, but it's quite close - because "reasonable" is positively correlated with "valid, correct information".
The same can be said about our brains too. What we think we see is quite different from what our eyes register (e.g., blind spot, hollow-face, colors on contrast backgrounds, etcchttps://m.youtube.com/watch?v=mf5otGNbkuc
Yet another article that imagines we know of some secret sauce in the human brain that DNNs can't possibly have. One for the dustbin.
> Unfortunately, this promotes a misunderstanding of how large language models (LLMs) work... It might be better to say that everything GPT does is a hallucination, since a state of non-hallucination, of checking the validity of something against some external perception, is absent from these models.
Let me turn it around:
> Unfortunately, this promotes a misunderstanding of how brains work... It might be better to say that every question a human answers without research is a hallucination, since a state of non-hallucination, of checking the validity of something against some external perception, is absent when simply answering a question.
Obviously nonsense. If you're going to write an article about misconceptions you'd better make sure you are right!
(Though it is silly to celebrate hallucinations; they're definitely not desirable.)
If you tell ChatGPT-4 w/ Bing to ignore thereader.mitpress.mit.edu where the article is hosted, it doesn't hallucinate the string "Evolution by Any Other Name?".
> Hilariously, plugging the example in the article into Bing enhanced ChatGPT-4, ChatGPT-4 w/ Bing hallucinates because of that very article!
Bing does not use the actual GPT 4 model. It's almost certainly a lower parameter model (like 180 billion vs > 1 trillion), or at the very least heavily quantized. That's why it makes more mistakes in your tests.
There are two versions here, Bing Chat(GPT) via bing.com, and OpenAI's ChatGPT w/ Bing via chat.openai.com. You're referring to the former, I'm using the latter.
Isn't the whole point of these LLMs be that they are non-deterministic? And with OpenAI opaquely 'tweaking' the model, it's no surprise you don't get the same output.
I don't think that is exactly true... I've seen arguments based on floating point calculations that accruing errors in FP can generate different results.
I've seen suggestions that it's an implementation problem, in the pipeline, that the same computation, with the same values, and same FP errors (they're deterministic) doesn't result in the same output with a temperature of 0, along with theories that the problem is not chased because it's probably related to optimization/cost reduction.
(trouble finding sources, but it being a bug was from someone involved with OpenAI)
> Sorry, I cannot provide the exact title of copyrighted content. However, I can provide a summary or answer questions about its content if you provide more context. How may I assist you further?
Contra the headling, we should "celebrate being wrong" – when it's wrong in fast, interesting, & voluminous ways that can then be filtered or corrected.
That's how lots of science, innovation, & learning work: generate many superficially-plausible candidates via a fast-and-loose process, then refine with a more rigorous evaluation.
That AIs, in the form of LLMs, are now doing this so well was unexpected, and progress in checking 'hallucinations' is proceeding very fast.
(Fortunately, the article is less dismissive than the headline, recognizing these model's potential & mainly urging an understanding of the limitations.)
Science progresses because people are testing plausible ideas not random ideas.
We don’t do human trials on random drugs, we do human trials on promising ones. Animal trials are more open but even then candidates are carefully considered as being viable. Things are even more open the earlier you are in drug discovery, but we aren’t testing molecules using Dubnium or most elements on the periodic table.
Similarly Ecology isn’t studying what happens when you introduce each type of salt water fish into lakes because the general assumption is they would just die thus saving you from preforming millions of experiments. That basic check for plausibility is extraordinarily valuable across all sciences.
There’s no sacred cows here. You do need to validate plausibility just like anything else, but you don’t need to do an exhaustive search across every possibility.
What seems plausible depends on your level of suspect matter expertise.
Hallucinations are in general ridiculously incorrect and nowhere close to anything worth testing. Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Do you really expect hallucinations to generally pick both plausible and untested targets here?
That's not my experience with the leading models, & I'm seeing others observe even when LLMs are wrong, they often supply interesting potential avenues to consider (especially in the coding-assistant/debugging domains).
Also, techniques for tamping-down hallucinations are improving rapidly, with teams finding...
• ways to detect likely hallucinations from patterns in internal activations
• extra conditioning to reduce hallucinations
• success using explicit requests that the LLM lies to train effective classifiers for detecting other unrequested falsehoods
• ways to check output against various ground-truths to detect & correct errors
To focus on the real-but-shrinking number of failure modes will miss the almost unbounded upside from continuous improvements.
Reducing hallucinations is useful but doesn’t imply that the remaining hallucinations are going to be valuable.
In terms of coding the costs of an LLM using some API function that doesn’t exist isn’t that high because we have tools to detect such nonsense and can look at the API in question. In science people don’t get to read realities underlying API’s.
If you can somehow build an actual physical test that costs 1 cent to both synthesize and validate a chemical you’re still looking at ~1 trillion dollars per candidate.
Simulations might be that cheap someday, but that’s still checking plausibility not running an actual experiment.
No, science is indeed testing random ideas. Limiting what is tested to an arbitrary plausibility score is something funders and sponsors do, not scientists.
What you're talking about is ethics, which is important too.
The speed of sound suddenly doubles between 627.500847 to 627.500848 atmospheres.
I’ve said it, now try and find someone willing to test it. That’s hard because Scientists want stuff that’s worth spending their limited time on earth looking into.
An AI would probably flag that idea as unlikely and not worth pursuing.
I don’t think we are that far off from having AIs that can read and incorporate the concepts from every paper ever published and start generating random ideas that fit into an internal framework it develops.
I think that within the lifetime of some readers here, an AI will spit out a hypothesis (like a method for producing a room temperature super conductor) that turns out to be true and we will have no idea where it came from. The AI’s knowledge of physics, chemistry, etc… will exceed ours and it trying to explain to us how it works would be like you explaining a singular value decomposition to your dog.
Think of requiring a minimum amount of hallucination in AI output as a safety mechanism, like mixing the distinctive odor into a propane tank. The odor is a signal that there's a gas leak that must be dealt with. A high minimum amount of hallucinations is a signal that the source is untrustworthy and must be checked. Hallucinations may turn out to be a feature that protects against over reliance. And defers the need for a Butlerian jihad.
"I think, therefore I am" means that one of the only things you can be sure of is that, as a thinking entity, you must exist. Thinking the thought "I am thinking", requires that you exist to do that thinking.
So unless someone was arguing that LLMs didn't exist, that principle is pretty orthogonal to the discussion.
it's just a consequence of the annoying anthropomorphizing of tech. "Human's can't do X, the AI model can't do X, look it's just like me fr, fr". Of course nobody applies that logic to a forklift or a debugger. If gdb started to hallucinate variables into existence we don't call it a creative act, we call it a bug.
Given that these AI systems just like any other machine operate at scale, automated, and fast, they must be precise and transparent, that is where the work should be.
I'm trying to understand this type of perspective, which is somewhat at odds with mine, so the following questions are completely genuine: Which models have you used? What did you try with them? Have you used GPT-4? How frequently?
I use Copilot pretty frequently when I program, find it very useful to explore an API i don't know or just help write bash or other stuff I suck at or just to save me some typing. Still often produces very unidiomatic and non-performant code though. But it's worth it for what it costs.
I do have a separate GPT-4 subscription as well but I've used it less and less. Tried to use it for search and text summary but it makes too many mistakes. Invented authors, papers, wrong summaries are just too common for it to be useful. So effectively I found myself Googling every time i used it to make sure the information is correct, which made the entire thing redundant.
When I hear the word "hallucination" in my mind appears an image of a crazy guy almost with foam on his mouth, probably on drugs or having severe mental problems. That is not a thing that I associate with being creative and certainly not trust worthy.
The term "hallucinating" in this context is just another way we anthropomorphize machines. It is mostly harmless but I see the author's point that it can be misleading when trying to address specific issues.
On the (rare) occasion I find it useful to avoid this very normal tendency I ask myself if it would make sense to apply the same framing to the output of an AI image generator.
195 comments
[ 2.8 ms ] story [ 225 ms ] threadThat's not brilliant, it's just wrong. If your AI behaves in a way that makes me not trust what it tells me, that's a bug not a feature.
But then again, many discoveries have been made because someone trying something that shouldn't work.
Hypothesis making may be interpreted as interpolation/extrapolation in a hypothesis space plus some heuristics to reduce that search space based on previous knowledge/valid hypothesis, how much weight you give to said knowledge and evidence, and some soft and hard logic rules. That is in part what allows (some, not counting Dunning–Kruger here) humans how certain to be about what they're arguing/talking about.
Maybe if the LLM is refeed with how likely (i.e. how many samples/tokens support it's response) is the output in its datasets, it may reevaluate its confidence and rephrase its answer.
In the end, the real problem of hallucinations in LLMs is about its confidence in the correctness/plausability of its own output. But that is something 1) humans can also be guilty of; and 2) that is no purely negative, as it can be exploited to generate new knowledge when applying robust hypothesis validation and testing to said ideas.
As you say in your last paragraph, people who've made discoveries in some areas have been treated as insane when tackling problems from a new perspective or when disregarding previous knowledge. If they weren't so strongheaded about their ideas, maybe we wouldn't even be posting in this forum right now.
PS: Still, I agree LLMs commit laughable mistakes sometimes ;)
I don't think this completely true. Considering something that wouldn't work often still leads to ideas, because it kicks your brain of a rut it may be in. This is why it can be incredibly useful to brainstorm with someone that doesn't have expertise in a topic. They'll say zany things that can be inspiring!
Random word sequences are a pretty common way to get inspiration. Something more "on topic" can't be that bad.
1) Hallucinations often appear because LLMs are designed to create fluent, coherent text.
2) LLMs have no understanding of the underlying reality that language describes.
3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt.
It sacrifices accuracy for being good at conversations as it is designed to do. All these criticisms of hallucinations are missing the point.
Generative AI is generative and basically is a specialist at making things up. It’s going to take things like multiprompting, network AIs that fact check output, and a host of other technologies or even entirely new models of AI to solve these problems, but don’t make the mistake of thinking that the system is supposed to be working without hallucinations right now — that’s not what it’s optimized for.
Nope, they summarized it without reading it. Useful enough for those who want to keep a finger to the pulse of AI without a huge time investment.
God save me from having to read every article on HN.
I’m sorry it wasn’t enough of a response to the article for you, but that wasn’t my intention in writing it.
Not true. They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages. The only reason LLMs are getting better is because they are better modeling the world. At some point the only way to improve token prediction is to gain an understanding of the world.
Perhaps the people building LLMs are doing this, but the LLMs themselves are not. LLMs are just generating text. They aren't "reverse engineering" anything.
> The only reason LLMs are getting better is because they are better modeling the world.
No, they are getting better at generating text that seems fluent and coherent, as long as you don't inquire into any actual semantic relationships with the world. LLMs can't model the world because they don't even have a concept of "the world". All they have is text.
https://danangell.com/blog/posts/gpt-understands/
> The idea is that back when GPT-4 was being trained for it to really consistently get the next word correct, to do that reliably, it had to do more than just bullshit. It had to do more than guess based on patterns. To get the next word right, it had to truly understand the words coming before it.
Imagine if you were training a transformer to predict numbers in a sequence that came from a sine function. Eventually it would get a pretty good test score and you'd find that internally it had built a decent replication of the sine function.
Now do the same with English text. Throw enough data and computation at it and eventually it'll model a rough understanding of gravity and the shapes of objects, as seen in the different answers between GPT-3.5 and GPT-4.
It's more interesting when the thing doing the understanding is a massive system that has both mastered English and other domains and then incidentally modeled some systems English can describe. An analog circuit that models artillery trajectories "understands" one kinematic equation. But that's kind of obvious and mundane. Understanding just one equation barely registers as profound and without that understanding existing in a larger context the word barely seems applicable.
But that confidence is based on intuition, not on any empirical observations. All the research we've seen so far points in the other direction: sufficiently trained LLMs do have a world model that you can find in their weights, changing these weights changes their completions in a way consistent with the new world model, etc. See OthelloGPT for the most blatant example.
I'm starting to wish we had something like a FAQ to point to, with a summary of all the research on the subject, because it's pretty settled by now.
This was always the conjecture of the “ghost in the machine” and mech futurist sci-fi writers.
Would you be willing to provide some reading about this? I’d love to see 1-2 things that could help me (and others) know more about what you know.
I'm not asserting that they "couldn't possibly". I'm asserting that they don't--as in, none of the LLMs we have today have a world model. Perhaps someone might invent a different something in the future that they call an "LLM" that does have a world model, but no such thing exists now.
> that confidence is based on intuition, not on any empirical observations
No, it's not based on either of those. It's based on the explicit descriptions of how existing LLMs work by the people who made them. Their descriptions make clear that LLMs are just confabulating text based on a given prompt and their training data. The LLM doesn't make any connection at all between the text and anything else.
Contrast, for example, with Wolfram Alpha. If you give Wolfram Alpha the prompt "What is the distance from New York to Tokyo", it doesn't confabulate text based on that prompt and some corpus of training data. It first parses the text and infers that the correct response involves a lookup in its geographic database (note that you can skip this step by explicitly selecting the particular tool that does this); then it does the lookup; then it renders the response into text form and outputs it. That is what having a (rudimentary) world model looks like. LLMs do nothing of the sort.
I really can't stand these low-effort takes. It's like saying "Human brains are just meat". You can summarize anything badly, but it's not helpful to the conversation.
Whatever quibbles you have aside, LLMs are generating useful text, today. At some point you're just pointlessly privileging meat over silicon.
No, it isn't, because human brains have rich semantic connections to the rest of the world. LLMs do not. So "LLMs are just generating text" is a justified description of their limitations, in a way that "human brains are just meat" is not.
Prove it
See, for example, my contrast of what an LLM does with what Wolfram Alpha does upthread.
I have no idea why you think Wolfram Alpha is relevant. LLMs can use tools like langchain to do the exact same thing Wolfram Alpha does. What proof do you have for your original claim?
What they just don't care about is communicating being out of distribution or making whack predictions. This becomes a problem for humans because they're perfectly fine making things up when the above fail.
But by all accounts, they do learn to distinguish these things. The computation is very much aware when it is going way off base.
GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975
Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334
Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824
Predictions of what? LLMs aren't predicting anything. They are, as the GP says, generating text that appears to be fluent and coherent (as long as we don't inquire into actual semantic relationships with anything else) in the context of the prompt.
"Previous context" here means text. It does not mean "the actual world". Big difference.
You think you experience the "true" world ? You don't. You experience a slice of it that your brain often further fabricates at parts.
To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect. No one gets to experience the real world.
You must be joking. You really think there's not a big difference between a corpus of text downloaded from the Internet and the entire actual world? Or even the sense-data that humans (or birds, for that matter) take in from the world? (And that doesn't even take into account the actions that humans, and birds, take in the world, and then compare the results with what their internal model predicts in order to update the model.)
I don't even know how to respond to something that is so totally off base.
> You think you experience the "true" world ?
I have made no such claim.
> To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect.
Yes, but an LLM isn't a bird any more than it is a human. LLMs don't experience anything.
>Yes, but an LLM isn't a bird any more than it is a human. LLMs don't experience anything.
Who is arguing whether LLMs are humans ? That's such an irrelevant point. The question is whether they are intelligent and understand which they are by any criteria that is actually testable. Nobody cares about building an artificial human.
>LLMs don't experience anything.
You don't know that
I always find this point a bit odd because humans aren't "optimized" to be correct either.
> "LLMs have no understanding of the underlying reality"
I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if could only answer with values from one column and it did exactly as I asked. It's hard for me to see things like that and not believe it has an understanding at some level.
LLM like have some form of understanding of the world, not sure if its anywhere near comparable to our understanding though. But they are still not general enough to really focus on facts. They can get close but the nature of statistics means there isn't hard checks to truth.
And we only just [mapped it for the first time](https://phys.org/news/2023-10-scientists-generate-single-cel...) so now much of what we are learning can potentially be applied to our “digital twins” in coming years.
I imagine a lot of progress with AI will involve similar networks, with governance providing evolutionary paths and guidance for multiple concurrent goals.
There are also lots of issues with training data and memory limits that make LLMs weak at continuity. Holes or weaknesses in the training data might “come through” in the behavior of hallucination.
Where do you think general AI is going right now that we should be looking at? I’m overwhelmed by how large this field has become in the last five years and am always interested in what others know.
It depends which subsystem.
The older the system in the body the less likely it is to have a high error rate, otherwise we'd die from cancer at a much higher rate or injure ourselves far more often. Of course this also depends on the definition of 'correct', if the system never changed we'd never evolve.
>> "LLMs have no understanding of the underlying reality"
This statement has always bothered me because it really depends on what you mean by 'underlying reality'. How many layers are we talking about? What does understanding mean? Because at the end of the day, humans don't really understand 'underlying reality, we just have a particular set of input devices we take in information and do some transformations on it... Where is the understanding happening?
Edit: there is also loads of evidence that our brains are wired to make incorrect decisions in many cases. Loss aversion is one example.
Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when communicating.
LLMs don't even have a concept of "being correct". That would require having a concept of an "external world" that text refers to, which LLMs don't have. All they have is the text in their training data.
They clearly do as plenty research indicates. You've just decided not to accept this. Pure confirmation bias in action.
I have seen plenty of researchers claim this. What I have not seen is actual support for such a claim.
I have not read much compelling evidence though I have seen a lot of researchers conjecturing and some inappropriately reaching (and anthropomorphising the living hell out of ai in the process sometimes).
There’s a lot to learn and I don’t think anyone can keep up with all the papers coming out so let’s keep things positive and work together to learn as much as we can og_kalu. Thanks for letting me tag this comment on the end of thread.
This is HN. We’re all nerds here anyways and nobody can keep up with everything in a field moving as fast at AI is these days. :)
I would argue, human reasoning is conceptually not so much different from LLMs as one might think.
Wrong. Human beings can act on the world as well as perceive it, and humans have rich internal models of the world that get updated continously as a result of those rich two-way interactions.
> I would argue, human reasoning is conceptually not so much different from LLMs as one might think.
I would argue that this is either an incredible over-estimate of LLMs, or an incredibly impoverished view of humans.
Conceptually this is the same for LLMs.
Input/perceive -> human/LLM -> output/act. You might say the acting part for LLMs is not autonomous but that is only dependent on how much agency is given to LLMs. LLMs outputs are massively used, hence their outputs lead to actions.
> and humans have rich internal models of the world that get updated continously as a result of those rich two-way interactions.
What is different from LLMs: rich, continuous. I would argue these are quantative and qualitative traits that fall outside of what I meant with "conceptually".
No, it isn't. LLMs don't look at the results of their actions and update a world model. Humans do.
One of the things that has been on my mind of late is the idea of how much of what humans call “correct” or “right” is actually socially generated pressures to conform.
Berns and Asch’s various research on social conformity around what we normally would consider objectively correct answers seems to have some potential applications worth testing in AI land. AIs may be much more performant with AI “social inputs” (and of course all the long term memory improvements currently lacking required to use it as a tool). I am very much excited about the network AI and pipeline work (small teams!) currently coming into vogue. Feels like the “right direction” to make incredible progress.
https://www.psychologytoday.com/us/blog/am-i-right/201404/th...
Otherwise agree with your points exactly. Good comment.
And the only reason that is even a thing is that we humans have a concept of actually being right, which we can distinguish from "responding to socially generated pressures to conform".
We seem barely capable of objective thought at best.
There is research that is claimed to show this, but it doesn't. All it actually shows is that the process of making a decision takes time, and involves many different parts of the brain. There is no separate "rationalization" that comes after the decision is made. It's all part of a single process that can't be decomposed into such separate parts.
In any case, all this is irrelevant to what I said. I said we humans have the concept of "actually being right". We don't always live up to that concept, but even making that observation, as you have, shows that we have the concept. If we didn't, we could not even make statements like this one of yours: "our brains care far more about achieving and maintaining status than about being right."
> We seem barely capable of objective thought at best.
I think you have a sadly impoverished view of human thought.
Regarding the research that supports my claim, studies have been done to isolate the different parts of the brain where they tell one part to do something like get water, then they ask the other part of the brain why they got the water and it makes up a reason because it's not aware of what happened in the other part. This is eerily similar to LLMs hallucinating.
This genetic and cultural evolution working together has shaped our brains to care for others, react to those who try to harm us, and to create moral rules that help us to live together successfully.
I’m not sure why responding to socialization pressures to conform to certain behaviors would preclude intrinsic or genetic morality. We know that babies exhibit intrinsic morality from a very young age, that moral behavior exists in non-human species, and that we have parts of the brain just to deal with moral judgement (though we are really in the infancy of understanding and mapping this from what I’ve read).
That’s not what I meant to imply or what research supports.
We recently started providing perception inputs, just like you’re talking about.
Obviously it’s nascent and not exactly a revolutionary statement, but it’s interesting to see the progression towards our experience.
I’m also interested in how the analog computing revolution is going to converge with AI in coming years, as we are making huge inroads into analog computing and that means cheap ubiquitous sensor inputs right in time for AI to start maturing.
Also, given how many words humans have written describing every part of the experience, LLMs can generate a pretty good understanding of it as-is.
I think about this difference a lot, and a fair bit of AI/ML attempts to reproduce this process to some degree.
Why we baby AI on this front, I have no idea.
Hallucinating on the other hand feels way more abstract. We speak of humans hallucinating answers rarely enough that they can use it for AI with a straight face. Heck, when we do talk about humans hallucinating it's often-as-not in the context of mind expanding experiences; maybe AI hallucinations are even a good thing!
It may be part of complex networks and intelligence, and humans may have a lot of it happening all the time but have ways of compensation or suppression of neurotic or hallucinating thoughts.
We have some intense “mental illnesses” that look a lot like what AI is presenting, where it can’t tell what is going on and fills in the gaps (schizophrenia maybe) or uses behavioral patterns to compensate for memory or intelligence inadequacy.
Obviously straight conjecturing here.
Not if people need to be reminded that, as you say, LLMs are not designed to give reliable answers. Many, many people appear to believe that they are, since they rely on the answers in all kinds of contexts. For example, in past HN threads on LLMs, I have seen people say that they rely on LLM-generated code.
;)
The "verified by me" part is crucial, though, yes?
I'm talking about people, whom I have seen posting here on HN in other threads on this topic, who leave out the "verify" step and just rely on the LLM code as it comes.
To be more specific, I trust GPT-4 to handle tasks like "Rewrite this C++ struct into Rust". I still verify that it's correct as part of a normal code review process, but I trust GPT-4 in that scenario at least as much as I would a junior dev.
LLMs absolutely have concepts that extend beyond just words. As early as 2008 (back when there were no large language models, only "regular" language models), we've been able to demonstrate things that seem to me like the model is learning abstract concepts. For a classic example, see Linguistic Regularities in Continuous Space Word Representations [0], a 2013 paper that talks about a model with a latent space where one can take the vector representations of the words "king", "queen", "man", and "woman" and literally perform the arithmetic `king - (man - woman) ≈ queen`. To me, this clearly demonstrates that the model "understands" the concept of gender, represented by a vector in the model's latent space. The set of numbers that you get from the subtraction `man - woman` represent the concept of the difference between the male and female genders, without needing a specific word tied to that representation of the concept.
It's debatable whether or not that counts as "understanding", but that's more of a semantic debate about what it means to understand, not a debate about the model's internal knowledge of the world and ability to do things with that knowledge.
0: https://aclanthology.org/N13-1090.pdf
I don’t mean to discount your excellent comment. I mean obviously these models are so complex already that their black box nature makes it difficult to derive conclusive results, but Arkham’s razor would imply that the simplest explanation — that is it recombined training data creating a heirarchy with multiple factors that provides context for the “correct” answer — that is more likely correct.
In other words a LlM can confer multiple related associations to output based on training data, and that is the most likely explanation for that behavior.
Just look at the poor performance with GPT-3 and the underlying data quality issues the Alpaca dataset team has discussed.
I’m no expert, and don’t mean to counter you as much as contribute my own limited thinking for discussion here.
Cheers!
The essence of bullshit (as he defines it) is that it is neither truth nor lie but rather unconcerned with veracity altogether. A bullshitter wants to appear smart, or convince others to agree with them, or some other conversational objective. Bullshit statements may or may not ultimately turn out to be true, but the thing that makes them bullshit is that the bullshitter didn't actually know when they said it.
Hence I propose a far more succinct term to describe the phenomena at hand. A far more accurate terminology than the presently vogue "hallucinating": AB. Artificial Bullshit.
Literally the cold opening of "On Bullshit".
The AI doesn't know something so it just invents something. We usually call that "bullshitting", or in more polite crowds, "lying".
Yes it does.
https://news.ycombinator.com/item?id=37874556
Who cares if "it knows" if it's apparently impossible to get it to use that knowledge to stop hallucinating?
To me, the end user there is no practical difference between not having the data and not being able to use the data it has. If it can't use it, or refuses to use it, it may as well not exist.
It sounds like you're describing the MSM. In any case, the same problem is true for a fair number of people. The difference is, silicon training has a much better chance of evolving beyond its current limitations much sooner than the typical human.
Truth is hard. Maybe too hard for a mere machine. But dramatic narrative and quirky dialogue might be quite doable.
3000 chapter litrpg fantasy generated overnight.
Each component AI generated. And each referring to the other components for its construction. For consistency.
It could be horribly formulaic but cunning and tasty too.
The article says "well, sometimes what it makes is bad".
Well big deal. A lot of human-created art is awful too.
Yeah but who wants to consume art purely generated by AI (that is, not human-created with AI support)? Most art sites have had blanket bans, or at least required tagging, on ai-generated art because people hate it so much.
Or to put it another way: why are you in the comment section of Hacker News, and not just asking ChatGPT to generate social media comments on the article?
Different forms of art (ai or human made) are good for different situations. Commenting on hacker news has a different set of needs and depth that ChatGPT cannot replicate. But I do switch over to chatGPT when trying to get background information about things. Different 'tools' for different needs.
Eh, this is a two sided argument.
For example, lots of people look at porn. This doesn't mean when I pull up an art/picture site I want to have porn displayed to me. In addition sites that allow porn are typically flooded with massive amounts of that kind of content. If you are a site that wants to show non-pornographic artistic content, in general you have to ban and heavily moderate it or it takes over the site.
I can see where the same will be true for AI generated art. Where for human art, one work could take hours, maybe days or far longer to create. Nearly unlimited amounts of AI art could flood a site in days, so much so that even attempting to host content would not be feasible in any way.
Lower ranked how? Not everything has a Reddit-esque upvote/downvote system. Plenty of sites let you browse by fandom/character/artist tags and just return chronological results (thinking primarily pixiv + all the booru-likes). Those sites were the ones where there was universal backlash to AI art and all of them require an AI-generated tag now.
Because I know the brains generating these comments have rich, diverse experience and a large set of refined, domain-specific heuristics derived from it - that is, they have top-notch training and alignment, at the cost of 20+ years of upfront incubation in an evolution-optimized meat harness with unpredictable success rates (and occasional personality defects). I can encounter new perspectives here that I don't think any current LLM could imitate efficiently or reliably. But if I wanted to know what Fox/MSNBC/NPR commenters had to say about this topic, I would absolutely ask ChatGPT, because those are commodity-grade opinions, and it excels at producing them. (It is not obvious to me that HN will still be an exception for, say, GPT-10 or whatever.)
This is definitely speculative and a little catty, but I suspect a lot of "hatred" of AI art is a) emotional solidarity with working artists who feel economically threatened by it, and b) a personal sense of insecurity along the lines of "what if I love this piece and it turns out to be AI - does that make me a boring NPC/a chump? better reject it as fiercely as possible to avoid introspection!"
Strongly disagree, just take a look at the AI-generated tag on pixiv[0] -- it's 99% same-looking garbage, and since there's no barrier to generating, uploaders flood pixiv with tons of images. Browsing tags is basically unusable without it being filtered out.
[0] - https://www.pixiv.net/en/tags/AI-generated/artworks mildly nsfw if you're not logged in, very nsfw if you are. (edit: note that this isn't a lot because pixiv later added a meta-tag for AI art so it's possible for users to filter without pixiv premium; that's how much it was hated.)
It has no meaning, it has no context, it is not inspired by anything. Art has depth, art has emotion.
Theres no AI pieces that someone "loves", theres nothing beyond, oh this is a cool image, it just doesnt exist.
But significant portions of those qualities are imputed by the viewer! The artist has their own intent and experience during the creation of the art, but that only inheres in the art insofar as another mind can later recover some of that feeling upon viewing. Huge parts of the meaning of ancient (and more recent) art are lost forever because they depended on never-recorded cultural or personal understandings, and our emotional appreciation of them today largely hinges on how they make us imagine the past or our relationship to it. I think it's incredibly short-sighted to be certain that nobody does or will love any AI artwork when so much of appreciation is contingent on the mind of the of the beholder, which is not necessarily responding to real information about the work's creator, even when human.
Ai does not and cannot create art, it’s as simple as that.
(Lest I be accused of moving goalposts or trying to sound less crazy, I want to double down on my original motivation that I think we need to make philosophical space for minds with agency and potential personhood that did not evolve in meat. I have no confidence that that will become urgent in my lifetime, but I think it will someday and it would be nice to be culturally through with the arguments about whether it could possibly ever make art before we're forced to argue about whether it can vote or join the priesthood or whatever!)
The hypothetical situation doesn’t even make sense. For the exact same reason that no matter how many prompts you use, you can’t write a good book with chat gpt. Art “goodness” exists entirely beyond the realm of being able to communicate it through words. It’s not something you can quantify logically and refine it through a prompt. You’ll always end up with some mechanical, derivative crap
Current models have fairly poor performance, compared to a human. If better art or better conversation could come from AI, then I suspect many might prefer it. Why wouldn't we? Why would we want simpler, less enjoyable, "real" conversations, or less symbolic art?
Most people won't care one bit. Some artists care strongly, hence the bans on artist-focused sites. When it's about as good or better than humans, people will consume it.
As for why people are still on HN, TBH a lot of these comments could be easily written by ChatGPT. What makes it worth it is seeing comments from people in the know, which is something ChatGPT can't replicate.
For your other comment about pixiv, people don't hate AI art, they hate low-effort trash. Most people, if they saw art they liked, wouldn't care who or what created it.
How are you measuring better? AI art is already better than most humans in a lot of ways: better shading, better coloring, even better anatomy outside of hands and feet (which has been mostly solved with controlnet and related extensions). Yet nobody wants to look at it.
That's simply false. Lots of people are enjoying it. Especially porn of course, but people are enjoying plenty of regular AI art, right now. "Better" isn't objective, more people will enjoy more AI art as it gets better at generating art that they like.
What makes you think otherwise?
> It might be better to say that everything GPT does is a hallucination, since a state of non-hallucination, of checking the validity of something against some external perception, is absent from these models.
I try to explain this to people who are obsessed with using ChatGPT to tell them things. So far I've been telling them something like: "it does not attempt to provide you valid information, it's optimizing for what would read like a reasonable continuation of the conversation, which is really not the same thing."
No, but it's quite close - because "reasonable" is positively correlated with "valid, correct information".
PBS: Perception Deception
https://www.youtube.com/watch?v=HU6LfXNeQM4
> Unfortunately, this promotes a misunderstanding of how large language models (LLMs) work... It might be better to say that everything GPT does is a hallucination, since a state of non-hallucination, of checking the validity of something against some external perception, is absent from these models.
Let me turn it around:
> Unfortunately, this promotes a misunderstanding of how brains work... It might be better to say that every question a human answers without research is a hallucination, since a state of non-hallucination, of checking the validity of something against some external perception, is absent when simply answering a question.
Obviously nonsense. If you're going to write an article about misconceptions you'd better make sure you are right!
(Though it is silly to celebrate hallucinations; they're definitely not desirable.)
https://chat.openai.com/share/e213e0bd-2838-45e2-9942-e52954...
https://chat.openai.com/share/64bc62d3-042c-40c4-8d50-8e28ce...
Hilariously, plugging the example in the article into Bing enhanced ChatGPT-4, ChatGPT-4 w/ Bing hallucinates because of that very article!
https://chat.openai.com/share/4fe50933-8436-44ad-a778-6297ca...
If you tell ChatGPT-4 w/ Bing to ignore thereader.mitpress.mit.edu where the article is hosted, it doesn't hallucinate the string "Evolution by Any Other Name?".
https://chat.openai.com/share/b8a94a8e-723f-40b6-b7c5-698dcd...
source: https://chat.openai.com/share/4fe50933-8436-44ad-a778-6297ca...
Bing does not use the actual GPT 4 model. It's almost certainly a lower parameter model (like 180 billion vs > 1 trillion), or at the very least heavily quantized. That's why it makes more mistakes in your tests.
(trouble finding sources, but it being a bug was from someone involved with OpenAI)
You have side channel data, so you didn't run the same thing he did.
If you want to run the same test then you have to clear your personalized data as well.
> Sorry, but I can't fulfill that specific request. Would you like a summary or any other information related to the topic?
https://chat.openai.com/share/900dde16-70f8-46ac-ac96-140b82...
> Sorry, I cannot provide the exact title of copyrighted content. However, I can provide a summary or answer questions about its content if you provide more context. How may I assist you further?
https://chat.openai.com/share/92ac284e-8a09-41d0-91c5-3272b7...
When a human thinks it’s able to determine the difference between imagination and fact, but when a human reads words on a screen it isn’t?
Drivel.
That's how lots of science, innovation, & learning work: generate many superficially-plausible candidates via a fast-and-loose process, then refine with a more rigorous evaluation.
That AIs, in the form of LLMs, are now doing this so well was unexpected, and progress in checking 'hallucinations' is proceeding very fast.
(Fortunately, the article is less dismissive than the headline, recognizing these model's potential & mainly urging an understanding of the limitations.)
We don’t do human trials on random drugs, we do human trials on promising ones. Animal trials are more open but even then candidates are carefully considered as being viable. Things are even more open the earlier you are in drug discovery, but we aren’t testing molecules using Dubnium or most elements on the periodic table.
Similarly Ecology isn’t studying what happens when you introduce each type of salt water fish into lakes because the general assumption is they would just die thus saving you from preforming millions of experiments. That basic check for plausibility is extraordinarily valuable across all sciences.
There’s no sacred cows here. You do need to validate plausibility just like anything else, but you don’t need to do an exhaustive search across every possibility.
Hallucinations are in general ridiculously incorrect and nowhere close to anything worth testing. Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Do you really expect hallucinations to generally pick both plausible and untested targets here?
Also, techniques for tamping-down hallucinations are improving rapidly, with teams finding...
• ways to detect likely hallucinations from patterns in internal activations
• extra conditioning to reduce hallucinations
• success using explicit requests that the LLM lies to train effective classifiers for detecting other unrequested falsehoods
• ways to check output against various ground-truths to detect & correct errors
To focus on the real-but-shrinking number of failure modes will miss the almost unbounded upside from continuous improvements.
In terms of coding the costs of an LLM using some API function that doesn’t exist isn’t that high because we have tools to detect such nonsense and can look at the API in question. In science people don’t get to read realities underlying API’s.
Depends at the cost and rate you can test them.
When simulations are fast and cheap you have far more latitude in filtering out 'ridiculously incorrect'.
Simulations might be that cheap someday, but that’s still checking plausibility not running an actual experiment.
What you're talking about is ethics, which is important too.
The speed of sound suddenly doubles between 627.500847 to 627.500848 atmospheres.
I’ve said it, now try and find someone willing to test it. That’s hard because Scientists want stuff that’s worth spending their limited time on earth looking into.
Of course no one would fund that, meaning it's the funders deciding on what they think is plausible.
I don’t think we are that far off from having AIs that can read and incorporate the concepts from every paper ever published and start generating random ideas that fit into an internal framework it develops.
I think that within the lifetime of some readers here, an AI will spit out a hypothesis (like a method for producing a room temperature super conductor) that turns out to be true and we will have no idea where it came from. The AI’s knowledge of physics, chemistry, etc… will exceed ours and it trying to explain to us how it works would be like you explaining a singular value decomposition to your dog.
https://en.wikipedia.org/wiki/Cogito,_ergo_sum
So unless someone was arguing that LLMs didn't exist, that principle is pretty orthogonal to the discussion.
Given that these AI systems just like any other machine operate at scale, automated, and fast, they must be precise and transparent, that is where the work should be.
A forklift is not a generalized machine. It is a specific machine for a very well defined set of tasks.
A LLM can take a set of input information, choose a number of options, like using an external tool, and act upon that data.
I do have a separate GPT-4 subscription as well but I've used it less and less. Tried to use it for search and text summary but it makes too many mistakes. Invented authors, papers, wrong summaries are just too common for it to be useful. So effectively I found myself Googling every time i used it to make sure the information is correct, which made the entire thing redundant.
On the (rare) occasion I find it useful to avoid this very normal tendency I ask myself if it would make sense to apply the same framing to the output of an AI image generator.