Ask HN: Are we sure LLMs are that useful in a web search application?

When you need an introduction to the subject area as much as you needed specific information from it; then an LLM explaining terms and offering options for further exploration can be a nice thing. (I assume they'll use it that way...)

When you're hunting for a particular fact, like "that bit of code i half remember seeing on a page 15 years ago", then I don't see anything for an LLM to add. Google had a pretty good index for that purpose about 15 years ago, but they've chosen to prioritize other goals since then. I dunno if anyone works "find things you're searching for" as a market now.

Which is an answer to your question: Does it matter if an LLM helps search the web? That's not what people are doing, that's not what these companies are selling.

Sometimes I don't really know what exactly to type into google because its only something rough in my head. It then takes like multiple try&research iterations on the topic I have in mind until being able to formulate the actual question, at least if I wasn't derailed in the process (monkey brain). A Chatbot is a godsend for me here and I will happily pay money for that alone.

Another point is that I am either creative or productive at a time, but never both... at least aware of which state I am in. ChatGPT has proven to take over the other part surprisingly good. ie:

- when I am in a productive mood and stumble upon a thinking problem, generative AI is like on-the-spot creativity for "good enough" solutions, like naming a programming thingy or write some filler text around a few keywords instead of me looking for words.

- when I am in a creative mindset, I increasingly feed some code snippets into the bot and ask some questions to "fill in the gaps", like writing a specific function using library X, then to write a documentation explaining how it works, then to also write some unittests, and sometimes I even derail a bit or let the bot explain parts that stand out in some way so I can maybe learn a trick.

... And i used ChatGPT already in kinda emergency situations, like when I know 5 minutes in advance that I have to speak in front of a crowd/in a meeting and it gave me extremely useful outlines to quickly adapt to even when in a panicked mind state - calming me down through a given structure that sounded okay-ish, and it doesn't matter if the response is right or wrong.

On average I think I would be far better served if search providers just fixed their operators.

A major thing I find lacking in search engines is a way to disambiguate between homographs. "AI" search engines are one way to implement that capability. However, I want a search engine that gives me websites as a result, and is not capable of lying to me. So far every single "AI" search I have tried has given me incorrect, invented answers. This is a big step in the wrong direction.

most answers are "it depends". so you might need to offer more information to get a better answer. so a conversation would be better.

Years ago there was an idea called "the semantic web": https://en.wikipedia.org/wiki/Semantic_Web

The basic idea was to have enough metadata about web sites so that you could get programs to do something approaching Prolog-style reasoning about the content and meaning of the web pages.

With more advanced LLMs it looks like a slightly different approach to achive something like the semantic web idea.

I think the idea is to constantly feed the model with updates from crawling the web and have the LLM "digest" the content, apply some filters to remove bad stuff, and then provide a meaningful result to whatever queries it might be asked.

It's all going to go horribly wrong because people, but, yes:

1) Integration with voice assistants. Links/sources are irrelevant.

2) Models tuned against a particular body of work don't care if links go stale, or websites get SlashdottedHackerNewshuggedDDOSed, or etc. Links/sources are irrelevant.

3) Inbound "service requests" processed by something that can better understand the question and the available answers/solutions. Links don't matter much.

4) When "Okay what are some good websites to read more about this?" can be answered, too, bang.

5) Ever asked somebody a question and just rolled with their answer instead of demanding citations? I mean, you're doing it here. So, again, yes.

Yes, but not in the form of chatbots.

Among other things, a LLM can be seen as a store which you query and get results from. A chatbot is cute because it formats output text to look like conversation, and the recent applications are nice because the query (now known as prompt) can be complicated and long, and can influence the format and length of the results.

But the cool stuff is being able to link the relatively small amount of text you input as a query, into many other chunks of texts that are semantically similar (waves hands around like helicopter blades). So, an LLM is a sort of "knowledge" store, that can be used for expanding queries, and search results, to make it more likely that a good result seems similar to the input query.

What do I mean by similar? well, the first iteration of this idea is vector similarity (e.g. https://github.com/facebookresearch/DPR). The second iteration is to store the results into the model itself, so that the search operation is performed by the model itself.

This second iteration will lead, IMHO, to a different sort of search engine. Not one over "all the pages" as, in theory at least, google and the like currently work. Instead, it will be restricted to the "well learnt pages", those which, because of volume of repetition, structure of text, or just availability to the training algorithm, get picked up and encoded into the weights.

To make an analogy, is like asking a human who are the Knights of the Round Table and getting back the usual "Percival, Lanceelot and Galahad", but just because the other thousand knights mentioned in some works are not popular enough for that given human to know them.

This is a different sort of search engine than we are used to, one which might be more useful for many (most?) applications. The biases and dangers of it are things we are only starting to imagine.

On Google when I search for "X that does not Y" it would often return "X that does Y" instead. Hard to force Google to respect the query intent. LLMs on the other hand process intent very well, but they hallucinate. So the obvious solution is a combination of them.

Ideally Google search would have a flag to "follow my intent to the letter" and return empty if nothing is found. When you are searching for a specific thing, a response with other things feels like bullshit, Google trying to milk more clicks wasting your time. I don't mean exact phrase search, I mean exact semantic search.

This is causing issues when searching for bug fixes by ignoring the exact version, when shopping it will ignore some of your filters and respond with bad product suggestions, and when searching something specific that looks like some other popular keyword, it will give you the popular one, as if if has an exclusion zone and you cannot search for anything else around it.

"Minimum weight of a scooter with back suspension" -> matches information about carrying capacity. Of course more people discuss about max passenger weight than minimum scooter weight, but I really don't care about the other one.

No, I was not convinced to start with and am getting less convinced all the time. To explain I'll go back to the middle ages. This was a time when the average person did not have access to books and the books which were in existence mostly were related to religion. That in itself was not that much of a problem since religion was the guiding line in most of people's lives but what was a problem was that those who wanted to know about what those books contained had to go through a middle man - an ecclesiastic - to get an interpretation. This gave those middle men enormous power which many of them wielded to their advantage, something which eventually led to a schism inside the ecclesiastical world when one of them nailed a pamphlet to a church door deriding the excesses of his world. When books became more generally available and with that literacy became a thing the world changed for what it seemed to be for good...

...until the rise of the new ecclesiastical class? These models are biased in several ways with the training set defining their world view and those who train them intervening in specific areas to bend the output to their will. They can be made to negate the old dictum of garbage in, garbage out to garbage in, gospel out - where the gospel follows whatever the (small-c) creator thinks the populace should know and (in extension) think.

In short I don't trust these models any further than I can throw them.

Find me all GPUs sorted by price desc, where the max depth is 12 inches that get great reviews. They should all be NVidia.

Good luck wasting hours of your time googling for that by hand.

I'd like to see these things (eventually) used for automated phone systems like CVS and my bank.

Mostly this is just to calm me down because ChatGPT gives me the illusion that I'm interacting with a human. The current voice systems are infuriatingly bad.

It would be nice if CVS's phone system would actually listen to me and modify its output accordingly. "I already gave you my birth date. And NO, I don't need a COVID booster."

edit: I'd like to meet the person who sold CVS its prescription web-site and its voice system. Simply to marvel at them and the swindle they pulled off, delivering absolute trash and probably walking away with a king's ransom.

I think the you're going to end up there either way eventually, as instant answers will start to pick up and cite more "LLMs make stuff up" generated content.

I think the summarisers are going to be valuable. As a Kagi subscriber I'm looking forward to them integrating their AI labs demo that was showcased recently. There's certainly potential for search to become an order of magnitude better over the next few years.

The issue I see with the chat approach is trust. I've seen so many examples of these models just making shit up now that I reckon regular use of them will eventually lead to mistrust between the human and the chat-bot. If you can't trust the answers and have to go and check yourself, it's dead as an idea IMHO.

Maybe not interfacing directly with a LLM ala ChatGPT but as an extension of vector search using word embeddings. It offers users a much more flexible interface to explore different combinations of dimensions vs a rigid facet UI interface. Well you can argue that the engineering has shifted to making the model, and that is true, but by keeping it fuzzy there is more room for creative prompt engineering. One challenge will be teaching the user what one doesn't know.

Depends how you use them.

Generating bullshit text off the cuff is not the only use of LLMs. LLMs can perform very well at classification, regression, ranking, coloring proper names red, and other tasks. You could, for instance, use LLMs to encode a query and documents and rank them with a siamese network, something not too different for how a conventional search engine works.

If there is one thing wrong with the current crop of LLMs it is that these can only attend over a limited number of tokens. BERT can attend over 512 tokens, ChatGPT over 4096, where a token is shorter than a word on average. It easy to process the headline of an HN submission with BERT, but I classify a few hundred abstracts of scientific papers a day. A long abstract is about 1800 words which is too much for longformer but would fit in ChatGPT if there aren't too many $10 words.

Unless you can recast a problem as "does this document have a short window in it with this attribute?" (maybe "did the patient die?" in a clinical case report or "who won the game?" in a sports article) there is no way to cut a document up into pieces and feed it into an LLM, then combine the output vectors in a way that doesn't break the "magic" behavior the LLM was trained to do.

You'd imagine ChatGPT would produce accurate results if you could tell it "Write an review of topic T with citations" but if you try that you'll find it will write citations that look for real but don't actually exist if you look them up. You'd imagine at minimum that such a system would have to read the papers that it cites, maybe being able to attend over all of them at the same time which would take an attention window 100-1000x larger.

That's by no means cheap and it might be Kryptonite for Google in that Google's model involves indexing a huge amount of low quality content and financing it by ads that are a penny a click. A business or individual might get a larger amount of value per query out of a much smaller task-oriented document set.

I see this technology becoming more of a 'content provider' rather than strictly an internet search engine. In the last few days, I have gone to Google to get answers to programming questions, do an image search for a quick pic for a presentation in addition to the usual type of internet queries. The results mostly led me to go on to another site to get the actual data, but more and more of it will be pulled into the initial page. Google has done this for years and this tech will just allow for more of it. Why visit a joke site for some quick one-liners, if the 'search engine' can just generate you 20 good-enough jokes specific to your needs?

I could imagine the interface being similar to what we have today, but with it being much better at taking in full descriptions of what you want. If you want pictures of teddy bears, it could provide search results and AI generated ones. If you want programming answers, it could link to StackOverflow or just give you an AI generated answer with an explanation. Perhaps I am looking for a lively bit of free music to add to an indie game - it could generate that too.

I feel that this will eventually end search as we know it, but it will hurt the sites that are behind the search results far more than it will hurt Google. Google (or Bing, ...) can become the one-stop-shop for so much more than it is today

I think there are plenty of people that remain skeptical of their utility for this application.

People who want to get rich will tell you it's the next greatest thing that will revolutionize the industry.

Personally, I've been annoyed at how confidently wrong ChatGPT can be. Even when you point out the error and ask it to correct the mistake it comes back with an even-more-wrong answer. And it frames it like the answer is completely, 100% correct and accurate. Because it's essentially really deep auto-complete, it's designed to generate text that sounds plausible. This isn't useful in a search context when you want to find sources and truth.

I think there are useful applications for this technology but I think we should leave that to the people who understand LLM's best and keep the charlatans out of it. LLM's are really interesting and have come a long way by leaps and bounds... but I don't see how replacing entire institutions and processes by something that is only well understood by a handful of people is a great idea. It's like watering plants with gatorade.

Google search has been powered by a large language model (BERT) for years so we've all been successfully using LLMs for search for a while now, just not in the way you're thinking.

Because the information it gives back has to be so sanitized I'm starting to feel that if search boxes start returning LLM results it will end up more like some safety team at Google/MS is having to almost write web queries by hand.

Doesn't feel as simple as the search world where you can just up rank some approved sources, unless there is somehow a way to just generate the language from approved sources and ideas.

I think it could be useful in the followup to search queries or being smarter about using synonyms and others ways to phrase things to find better results but it reminds me of when facebook announced bots in there messenger. Lots of people tried it and showed all all these features and then they didn't really amount to much.

Perhaps tech hype cycles are more about valuations and less about utility. Everyone jumps on the bandwagon with the latest buzzwords, companies buy in. Nobody wants to be left behind. New tooling is demanded. Employees pad their resumes.

There's more to generative models than just the above, but how much of this hype cycle is substantive for end users or developers? You always had a query. Now you'll get answers in a different way.

Skeptical overall.

yes

> Only truly practical use case I can think of is summarizing articles or writing them which makes more sense as a word processor or browser add-ons

That use case is the one that makes more sense. Given that ChatGPT frequently hallucinates the wrong answer and confidently tells you how it is correct with its inability to cite and transparently explain to you how it got to that answer tells you that it is results are untrustworthy and this AI bubble is again, pure hype created by VCs.

The only worthy AI hype that will change everything is open-source LLMs that are smaller models and are more transparent.

Anytime anything starts getting touted as a panacea I start getting suspicious. Maybe LLMs are going to have a large role in search in future, but at the moment there's so much propaganda flying around I just assume it's mostly lies.