I assume the title of this blog is a typo. Anyway, I'm not sure how interesting a blog post using an out-dated version of the technology, and telling us something we've known from day one is.
it's going to be interesting to see if with all the sudden hype and income around llm people will manage to let them acquire a sense of "facts".
From the gpt4 paper it looks like they already managed to get some interesting improvements, but it's quite likely some fundamental mechanism remains to be added..
This whole category of blog post where people ask ChatGPT about themselves and then act surprised when it gives inaccurate responses is really uncompelling. The purpose of this technology is to grasp and synthesize information on general topics of use to everyone – science, engineering, literature, language/semantic competency, etc – not to have godlike knowledge of every person who ever existed. As a feature, the only person who would expect or want ChatGPT to dispense accurate information about some random blog poster is said blog poster.
The article makes valid points about the danger of trusting an LLM as gospel. However, I am concerned about the anthropomorphic language. The LLM is only “hallucinating” if we define “hallucinating” to mean generating text that is syntactically correct and has the correct semantic shape but the text as interpreted by the reader is taken to be factually incorrect. Fundamentally this is a reader discernment issue —- which is why, in my opinion, using anthropomorphic language to describe the interaction with LLMs or the interpretation of the generated text is problematic. The imprecise description muddies our understanding of the tool to the point that it implies agency — thereby removing agency in the form of discretion from the user.
I saw a similar report that it makes up academic papers, and so since then I have been fact checking when I use it to find research.. and every paper it has cited to me has existed.
Perhaps it is a question of how relevant the subject you are asking about is? For instance the example that made me dubious was about eating banana peels for medical purposes, quite esoteric and not able to be confirmed by any outside sources.
In this case, it seems GPT is familiar with the blog but has fuzzy details that are not corroborated by any other sources.
It is a sort of dunning Kruger effect where the AI does not know the limits of it's limits but acts confidently regardless.
Note: There was a typo in the title - Instead of ChatGPT, it was ChapGPT . I have corrected the title, but the URL remains the same because Cool URIs don't change
The Onion's recent "ChatGPT Starting To Think Journalist Could One Day Be Capable Of Independent Thought"[1] is a good reaction to these sorts of unsurprising "old news" articles repeating what other journalists have been reporting on repeatedly.
I'll concur with the piece, definitely. It seems to completely hallucinate when giving any reference link, or at least GPT-3 did. I don't know if GPT-4 continues the behavior. GPT-4 is underneath Bing Search, so if it does, that'd be bad news for MS.
17 comments
[ 2.3 ms ] story [ 52.2 ms ] threadIt’s unreliableness is exhausting since you are always doubting its output.
From the gpt4 paper it looks like they already managed to get some interesting improvements, but it's quite likely some fundamental mechanism remains to be added..
Perhaps it is a question of how relevant the subject you are asking about is? For instance the example that made me dubious was about eating banana peels for medical purposes, quite esoteric and not able to be confirmed by any outside sources.
In this case, it seems GPT is familiar with the blog but has fuzzy details that are not corroborated by any other sources.
It is a sort of dunning Kruger effect where the AI does not know the limits of it's limits but acts confidently regardless.
Note: There was a typo in the title - Instead of ChatGPT, it was ChapGPT . I have corrected the title, but the URL remains the same because Cool URIs don't change
[1] https://www.theonion.com/chatgpt-starting-to-think-journalis...
That's what these models do. That's how they work. The fact that you can sometimes (often?) end up with factual information is a happy byproduct.