It seems like just about any time an AI company releases such a video, they're bound to make the same mistake of not cross checking their own demo.
I don't get it. I mean, how hard can it be? These are billion dollar ventures. For god's sake, at least have an intern fact check your press releases for 15 minutes before publishing them.
On the other hand, these repeated basic mistakes certainly help in keeping expectations in check. But I doubt that's the goal of the demo...
I concur, but, also am thinking that it might be some sort of "purist" drive, in the "eat your own dogfood" [to the utomost] kind of way, by not having a human "in the loop"? (ie. releasing inmediately post-AI action -without- human checking?. Even, an "ideological" stance?
(Granted, your point about the desirability of basics, still holds ...)
well the original voice/tutor demo by "open" AI was heavily choreographed, wasn't it? it seems like it's just different people doing it, so they just have to learn from the same mistakes again.
> Lack of polish indicates core belief in the message/product.
It certainly can, but not always. In any case, this effect is why when I see a product that has a high degree of polish, I take it as a weak signal that the product is not great.
Part of the problem is over-personification of these models. "Hallucination" as an industry term implies that the model is a thinking entity with a perception that is faulty. Instead, it's more like a slow database that has failed to provide correct information.
If you'd like a LLM to provide a stream of characters that look like text, they're very good for that. They're not good for providing information, full stop.
I think hallucination is just a euphemism for bullshitting. people do it too, I've met plenty of people who can talk confidently about things they clearly don't understand.
The issue here is that someone who forgets the concert is already over vs a search engine tells me the wrong dates for a concert is that, I expect the computer to provide accurate information. This wasn’t an issue 10 years ago. Google a festival or event, get a website or article about it, they provide the dates.
This is like your local newspaper giving the wrong movie times or incorrect scores from the big game. People make mistakes but people’s whose job it is to provide information about something and who are unable to reliably provide that information tend to lose their jobs. Why should I “hire” OpenAI’s solution if it fails to be even as reliable as a human?
It's beautiful isn't it? We had cases before of say mortgage advisors refusing you service based on this "score" that they won't tell you how they compute, despite your good history of managing money competently and being able to afford it. Now you don't just have a computer that says no, you also have one that will randomly tell you things that aren't true.
It's just that with people you don't expect them to update every few months with a new and improved version of themselves that supposedly bullshits less. Instead at some point you just stop asking, or at least don't take them seriously any more.
I don't agree that the demo got this wrong. It's not a hallucination, it's pulling the date range from the festival's ticketing page (https://appsummer.org/tickets/) mentions July 29 – August 16 date range. Perhaps a superintelligence should be smart enough mention the other dates, and to understand that the "Closed" dates are fake and the festival's not really happening in that interval, but this seems like perfectly reasonable output for the moderately-intelligent search tool OpenAI is advertising.
The user asked for music festivals in August, and the response confidently listed a festival that was not in August, with dates that were not the correct festival dates.
It's an understandable mistake given the dates included on that festival's page, but it's still a mistake. And the mistake is presented as a confident answer, which is the issue with these kinds of tools including Google's search result summary answers.
Also, can ai's tell the difference between US and EU dates common practices (referring to the person that complained, true or not - and my dates are just examples, that Taylor Swift was playing 10/11, 11/11 and 12/11..."Why doesn't she just play three consecutive dates instead of travelling over three months?")
Even if the demo was 100% correct, I don't even understand why it would have been impressive. "Traditional" search engines will give you this answer instantly. Isn't the point to do this stuff better? To do things a normal search engine can't do?
Yesterday my coworker was talking about using Gemini and wanted to show me how neat it was. So he typed a questing into Google. The search results came back instantly with the correct information as the #1 link. 5 seconds later, the Gemini box displayed the correct information. What the hell is so impressive about that?
I’ve seen a lot of engineering jerking off lately with some other teams I’ve hung around and it’s like developers don’t understand that if the end result isn’t better it’s irrelevant to consumers.
But hey look at this cool technical thing we can do!
This is one of the points missing in most conversations. Each has strengths and weaknesses, LLMs are not going to full-on replace search, but can often be a better starting point. The main advantage to LLMs is that you can either (1) provide vague questions or (2) provide a whole bunch of context
This example shows, for a python error printed at the terminal, how you can get a really nice response by also sharing a code snippet and directory layout (extra context)
I know that I've had plenty of situations where I know kind of what I'm looking for but I can't describe it and run 10-20 searches trying to figure out what to type to get the result I'm looking for. I bet that most people have had that kind of experience but if not would find this kind issue to be relatable.
A demo showing an LLM helping to solve some contrived problem of this sort would resonate with people, and show a valid use case for LLMs in the search process. That's what would be impressive.
To add, sometimes I know what the issue is, but the results are to generic. This is where the extra context comes in. Using an LLM is like posting to and getting a reply on a forum in real-time, with both equally likely to give you a right/wrong answer to boot!
Of course it would get something wrong. Everything is put into a probabilistic space and then get pulled out. Basically asking to draw a deterministic results from a non-delta distribution. I wonder why they manage to make OCR working well in their model but then suck at pulling links and quotes.
Imagine actually thinking that you can search for something and you are always going to get a correct answer on the internet. At least with LLMs you can fine tune or at least pick different models you want to use and communicate at will with it. It's not 100% but the alternative is a crap ton of research and verification into topics I don't really have time for. Can't tell you how many times now it's been useful to use AI as a researcher aid in prototyping. It has vastly improved my iteration times, especially on things I normally would spend weeks on tutorials.
Normally when you search you can look at many different results. With "AI Search" you just have to trust the magic box, which you already know is wrong a lot of the time.
I still believe that the best approach is the contextual search embedded everywhere - I think Microsoft's approach (and now Apple's one) is a way forward.
33 comments
[ 3.2 ms ] story [ 86.4 ms ] threadI don't get it. I mean, how hard can it be? These are billion dollar ventures. For god's sake, at least have an intern fact check your press releases for 15 minutes before publishing them.
On the other hand, these repeated basic mistakes certainly help in keeping expectations in check. But I doubt that's the goal of the demo...
(Granted, your point about the desirability of basics, still holds ...)
It might as well be dogfooding, but to me that seems a bit too, hm, say, intellectually honest for what is basically a PR move in a hype market.
On one hand:
Marketing copy should be word-perfect, with no grammar errors.
On the other:
Video of baby's first steps should be raw + unedited.
The more you move towards a "real moment", the less sense it makes to polish it to a mirror sheen
It certainly can, but not always. In any case, this effect is why when I see a product that has a high degree of polish, I take it as a weak signal that the product is not great.
TBH, seeing those press releases lately, they seem to do it intentionally.
Treating your users as idiots has become the norm.
People need to learn to use LLMs for what they are - useful but fallible and prone to hallucinations.
Until we have a technical solution for that it’s the people that will need to adapt
If you'd like a LLM to provide a stream of characters that look like text, they're very good for that. They're not good for providing information, full stop.
This is like your local newspaper giving the wrong movie times or incorrect scores from the big game. People make mistakes but people’s whose job it is to provide information about something and who are unable to reliably provide that information tend to lose their jobs. Why should I “hire” OpenAI’s solution if it fails to be even as reliable as a human?
It's an understandable mistake given the dates included on that festival's page, but it's still a mistake. And the mistake is presented as a confident answer, which is the issue with these kinds of tools including Google's search result summary answers.
Yesterday my coworker was talking about using Gemini and wanted to show me how neat it was. So he typed a questing into Google. The search results came back instantly with the correct information as the #1 link. 5 seconds later, the Gemini box displayed the correct information. What the hell is so impressive about that?
But hey look at this cool technical thing we can do!
It’s so cringey.
This is one of the points missing in most conversations. Each has strengths and weaknesses, LLMs are not going to full-on replace search, but can often be a better starting point. The main advantage to LLMs is that you can either (1) provide vague questions or (2) provide a whole bunch of context
https://topicalsource.dev/chat/023e7e54-947b-490d-bcd8-89cc2...
This example shows, for a python error printed at the terminal, how you can get a really nice response by also sharing a code snippet and directory layout (extra context)
A demo showing an LLM helping to solve some contrived problem of this sort would resonate with people, and show a valid use case for LLMs in the search process. That's what would be impressive.