True, but it does not have to. ChatGPT is it's own thing. It has functions search engines can never accomplish. You can say the same thing the other way around, "Why Search Engines won't replace ChatGPT?" The essence of what ChatGPT is is here to stay. We'll be seeing different flavors and it will be a big part of everyone's life.
Googler opinions are my own. (I know nothing about search).
Google has LaMDA, which I could see them putting the answers from it into the knowledge box at the top of searches. There is no reason they have to be mutually exclusive, a search engine can provide answers from both. Google search is already an aggregation of multiple sources (images, web index, knowledge, shopping, video, flights, etc...). Adding another source seems like the obvious path forward (assuming accuracy and cost make sense).
Very true, they already have different sections in the results pages. There's no reason why they can't have one more. They can also use the box as a way to improve accuracy by having user vote on the quality of the result.
We really need you to stop posting so aggressively. If you can't or won't, we're going to have to ban you. You've been breaking the site guidelines frequently. That's not ok, and I warned you about this once recently: https://news.ycombinator.com/item?id=33860364.
I don't want to ban you because you've also posted good things and if you'd simply follow the site guidelines you'd be a good contributor, but the current pattern is a problem. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and fixing this, we'd be grateful.
If we don't need to click a link does google receives less revenue? I think including more direct answers from a search query would require changes to Google's business model.
Outdated? How so? The primary thing Algolia is guilty of is being massively overpriced to the point it’s a non-starter for any platform with even moderate usage. But their full-text search functions quite well and is very fast.
It won’t have to replace; it can just augment and be very successful at it. I’m just unsure how to cover the massive costs of running this thing with millions of req/sec.
I'll be the first to say that chatgpt already does very well at the first and third examples. WebGPT sounds like the right approach for the others (understanding how to formulate a search query, then searching, then summarising the results)
Edit
I think the direction of these models seems clear, and their capabilities however limited at the moment are outstanding. I think the best measure of performance is what the naysayers complain about. We've quickly gone from "actual gibberish that looks like paragraphs" to "yes but it loses track after a few sentences" to "yes it's kind of coherent but doesn't make sense totally" and are now at "it's like an overconfident human at many tasks, when they're only somewhat qualified in the area". People are pointing out where it's got an API call wrong, without it having any ability to look up the answer and see if the function actually exists.
I spoke to it about distributed computation design and discovered something new I can actually implement in my design.
I'm very excited about where this will all go because it seems useful now and I'd be very surprised if this is the best that it ever reaches.
Absolutely, I totally agree. I edited my comment to add in more as I realised I'd not addressed that. It's surprisingly good right now, and given that it's useful that feeds into even more attention and money being spent on making it better.
Anyone not looking at chatgpt and thinking "what if this was even just a bit better? Or had better iterations or lookups?" is missing out on what I think is a revolutionary change coming probably sooner than people expect.
That is, if you can blindly trust the answers.. if it could provide the top sources for the information so I could fact-check it if needed, it would work
We're going to have a generation of people that are cock sure about things because ... ai, but they won't actually know jack shift. It sort of happened with google. People can figure something out but they don't systematically get things. Shallow explanation. Goldfish attention span
I think that assumes AI makes any difference to the self-certainly of humans.
My mother was into all kinds of new age and supernatural stuff. You may think you can see where this anecdote is leading, and oh boy I could go there, but instead prepare for a twist! One of the stories she had was a client saying "It is not possible, Susan, it is not possible that there are beings out there that are cleverer than what we is".
Is there a way to disuade users from blindly trusting responses? Such as unavoidable caveats, make no attempt to provide depth and instead strongly recommend reliable research?
And as a logical aside: if AI can't be trusted, we shouldn't trust AI's recommendations for reliable sources either.
> And as a logical aside: if AI can't be trusted, we shouldn't trust AI's recommendations for reliable sources either.
Don't think that's automatically true though: an AI can be consistently excellent at retrieving reliable sources and still get a lot wrong on its summaries. Also humans have their own opinions on sources even when they're not familiar with the detail, and finding out if the AI reads Nature or 4chan is part of the debugging process...
Sure. Really, I was thinking of the scenario where someone explictly asks for a reliable source i.e. "Thanks for summary about X. To whom should I turn for more accurate/detailed information?"
Oh yeah, it's spectacularly bad at some of that sort of thing. Ask for a list of academic papers and it'll make half of them up and give others new authors...
That's why in some respects, I think the sheer clunkiness of "according to website.com, claude shannon is..." or "ok, I've found this on the web..." in the current consumer tech is better than anonymous, authoritative somewhat novel answers than might be the internet's consensus opinion or might just be a vaguely relevant word vector structured like an answer
"Computer, please find out how GPS works and explain it to me like I'm five years old."
ChatGPT:
GPS stands for Global Positioning System. It is a way that we can figure out where we are in the world using special satellites.
Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so they can see everything on the ground.
When you have a GPS device, like a phone or a car, it can talk to these satellites and ask, "Where am I?" The satellites send back a message that says, "You are here!" and gives your device a special code.
The GPS device can use this code to figure out exactly where you are on a map. It's kind of like a magic trick! You can use GPS to find out where you are, or to find your way to a new place. It's super helpful when you're lost or want to go on an adventure.
As I mentioned elsewhere, conversational AIs in the early 2020's are like PCs in the 1980's -- a new technology that still has lots of early limitations.
The second half of that explanation is completely wrong though, isn’t it? There is no active signal being sent from your GPS antenna, no two-way communication.
I’ll admit it was a decent answer. However, even a somewhat bright person (especially someone with radio / networking experience) could probably have realized that was an issue without any prior knowledge of GPS. They know that their phone has a GPS receiver, and that satellite phones accessing geostationary or high altitude satellites are much bulkier. Clearly, your cell phone isn’t transmitting to each satellite.
How will future LLMs be able to conduct that level of reasoning?
> this is probably appropriate for explaining things to 5 year olds.
I feel it's appropriate in terms of simplicity, but I think providing a simple wrong explanation is really only worth it when the more accurate explanation is substantially harder to simplify, and I don't think explaining how GPS works falls into that category.
Something like this I think is more accurate without being any more complex:
> Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so you can always see some of them.
> When you have a GPS device, like a phone or a car, it can look for these satellites, and if it sees enough of them it can use math to figure out where you are.
You can just as easily ELI5 this without spreading misinformation, though. Just change the third paragraph to:
> The satellites all broadcast a special code saying where each of them is. When you have a GPS device, like a phone or a car, it can listen to these codes and find out where and how far away different satellites are.
(Actually, this would be a fun thing to do with a class. Have a few kids march around the edges of the playground, saying things like “I’m next to the swingset now!”, and then ask one one kid to close their eyes, put them somewhere in the middle and ask them to guess where they’re standing.)
The first half is wrong too. GPS has nothing to do with satellites being able to see lots of things from high up. The whole thing is just nonsense that looks plausibly like an explanation until you try to decipher it.
I didn't even notice that bit (maybe GPT is a flat-earther?).
Something GPT discourse has been demonstrating to me is that I'm not usually a very careful reader. I apparently skim a lot. Or maybe I skim GPT outputs because I'm biased in my expectations already?
Achieving simplicity by glossing over details is distinct from achieving simplicity by stating something incorrect.
Which one is more appropriate and/or typical will probably depend on what questions the five-year-old is asking, but I think it's reasonable to say the former is usually preferable to the latter.
GPS doesn't talk to the satellites, but the satellites do send a message that allows the GPS to answer "where am I" for itself. It's not the worst way to explain things at that level, even if it's misleading.
About half the (presumably human) "eli5 GPS" answers I found on Reddit made the same mistake, so chatGPT just copied an apparently popular misconception.
> About half the (presumably human) "eli5 GPS" answers I found on Reddit made the same mistake, so chatGPT just copied an apparently popular misconception.
I think that's one of the biggest problems with using LLMs for accurate answers. A lot of the text that's very useful for modelling human language and questions is also full of factual errors and questionable opinions. Answers being wrong is a problem with old-fashioned web search too, but I think LLMs necessarily lean more on quantity rather than quality of content...
It is better at doing "translations" from one "language" to another "language". While it does know stuff and can answer questions about what it knows, that isn't its strength.
If you can do "here is the structured data, summarize it and make sure to present this information" it can do that quite well.
So far I've played with GPT doing a zero shot classification of HN titles and CSV hourly weather data to summary. In each case, it has done quite well.
Neither of those are things about what it "knows."
The benefit of web searches, and trouble with LLMs, is you can see the site that an answer is on and judge the credibility based on that. If I see that the answer is on Reddit ELI5, then I know that there's no assurance about how accurate the answer is. If it takes me to a page for a university engineering or math department, or the blog of a dude who built a receiver, then I can be a lot more confident that the answer is correct, though it might be technically over my head.
Ian Calvert
Computer, please find out how GPS works and explain it to me like I'm five years old
Sure thing! GPS stands for Global Positioning System. It's a way to find out where you are on Earth.
GPS works by using satellites that are way up in space. There are lots of these satellites orbiting around Earth, and they are always sending out signals. Your GPS receiver is a special device that can pick up these signals from the satellites.
The GPS receiver knows how far away it is from each satellite because of the time it takes for the signal to reach it. By figuring out how far away it is from a few different satellites, the GPS receiver can figure out exactly where it is on Earth.
So, if you have a GPS receiver in your car, phone, or on a special device like a watch, it can use the signals from the satellites to tell you exactly where you are and how to get to where you want to go. Does that make sense?
---
I was also able to ask it to make it simpler and simpler and it did so pretty well.
And comparing these two results for the same question serves to illustrate how ChatGPT does not in any meaningful sense _understand_ what it is "talking" about. It basically just strings together relevant-seeming words in a plausible way. The result may be useful, or it may be nonsense.
You'd better know enough (and be alert enough) to tell the difference...
I've seen this sort of response a lot lately, both for GPT and all the image generators.
Google, and all the other search engines, literally show snippets of the original content to display search results, and have had copyright infringement lawsuits over this.
Yet somehow, it's taken these AI, which generally aren't just block-quoting things unless you ask them to (and often getting it wrong if you do), for non-lawyers to start asking that question.
I think that just indicates it's more blatant. With search engine results I can click through to the original source. That's attribution, at least to the laymen. ChatGPT has no such thing.
I selected a few sentences from the given response, asked DuckDuckGo to find them, and got zero hits for any of them.
GPT definitely has some kind of internal model of things and how they relate to other things, and the capacity to express those relations in different ways.
Whether this is functionally the same as the model in my head, or some analog to it, or if such descriptions are merely cute but misleading anthropomorphisation, is far beyond my knowledge of neuroscience, psychology, and indeed machine learning.
The explanation is misleading. The satellites don't see you and don't care about you. They are beacons sending a radio signal (the beacons says: i'm satellite X and here is a timestamp). The receiver based on the known position of the satelites and the beacon signals received figures out your position on earth using math!
So the explanation is either: satellites and their position or give the full explanation. Not this regurgitation.
Except for having nothing to do with how GPS works. The satellites essentially broadcast what time it is and your reciever uses the delay between broadcast and reception to calculate where/when it is.
Explain like Im five means break it down, not lie about the complicated bits.
Its terrible. This is totally not how GPS works, I.E. the terminal never sends a message to the satellite. This is the danger of early LLMs, a large amount of their output is false and there is no way to tell unless you are already an expert.
This makes them basically useless except for a toy.
Asking it to “explain how GPS works like I’m a seasoned engineer” yields a more detailed explanation and there’s no mention of GPS terminals sending messages back to the satellites.
Probably the “false output” you mentioned is due to the fact that the prompt asked to “explain it like I’m five”, which made ChatGPT answer with a “dialogue between terminal and satellite” explanation, which (arguably) may be better understood by a 5 years old.
Imagine that the GPS satellites orbiting the Earth are like the magic lanterns in the sky, sending signals down to a receiver (like the one in your phone or car). The receiver can use those signals to figure out how far away it is from each of the magic lanterns. By measuring the distance to multiple lanterns, the receiver can triangulate its position and figure out exactly where it is on the surface of the Earth. So even if you're a big, green ogre like Shrek, you can use GPS to find your way around and never get lost!
Agree. ChatGPT is how I imagined the internet would be back in the 90s (without having the foggiest clue how it might get there). Your questions are similar to the types of searches that people really want but have to translate to 'GPS simple explanation' or 'COVID in <town> this year' while hoping for a set of relevant pages in the Google return.
I've gotten it to fail on some word problems, but even then it was impressive to see it try to come up with a solution. More humorous were requests like 'Create a post for ycombinator about ChatGPT in the style of Chris Farley'. It's surprising and quite welcome that it even attempts that.
Can LLMs be updated quickly the way search engines are by crawlers and what not? ChatGPT is a couple years out of date and there was a lot of human involvement. Doesn’t sound like a quick or automated process. GPT-4 is still under development.
ChatGPT will flood the internet with articles like how SEO was used to hack search. It might not replace search engine, but search results will becomes even more irrelevant.
Not necessary. The google search function still works fine, its just the content becomes worst. If the content is generated from ChatGPT, I don't see how it will cut out the middle-person. I believe a more likely scenario is people appending site:reddit.com for all of their search. It will be domain specific.
I can imagine AI replacing search engines in a somewhat far future, but in the meantime, it'll mostly be used to flood the internet with many low quality articles. Consequence will be as you said, but I also wonder if it won't lower the quality of new AI trainings as well.
There's "precedents", tho. MidJourney, the art AI, trained Stable Diffusion over favorites generations of their community, making it a "fine tuned Mid-Stable Diffusion" that was quite amazing (that was the --beta and --betap flags a couple of months ago there).
in the short term yes. in the long term I think SEO will disappear. my buddy who works in the content creation space for indeed is telling me that they are already starting to use gpt to replace content creation, but know that it's a matter of time before that won't work.
But on the search side, it'll also get rid of the need for the kind of SEO/blogspam stuff you're describing.
We already see it with recipes - there are all the god-awful sites that have 1000 words of fluff before you get to the recipe, because that's what you need for SEO purposes. GPT is really good at creating that kind of useless fluff, but it also obfuscates the need for it, because it'll just give you the recipe you want without the need to go to a website at all.
There are many ways companies might try to get some ad money in a ChatGPT setting: obvious and non-obvious product placement in the answer.
Q. Find me the best italian restaurant in New York.
A. It's likely subjective - let's just sell it to the highest bidder.
Q. How to improve mental health?
A. Give a reasonably good answer and probably mention: some say a solar lamp is good in winter months, there's a good one:
I somehow feel like people can be easily baited into buying something when you mix good, useful information with ads. Kind of like how reputed youtubers/niche influencers do product placement and make referral money.
All of this kind of makes SEO obsolete as ChatGPT ads might end up having more conversions than regular Google ads.
I can see that being true for the top 5% (arbitrarily chosen small %) of difficult translations with professional requirements, but I think for the vast majority of day-to-day translation between the major languages (maybe only within the same language family), it is already sufficient and improving.
I only have the anecdata of my friends and myself, but we have yet to encounter a casual conversation type translation that DeepL or Google translate have failed on for Spanish <-> English <-> French. Granted, it doesn't always return the most optimal translation, but it has always given the intended core idea.
ChatGPT might not replace search (entirely), but for many many use cases, it has already replaced search.
When pure machine translation was starting out, there were lots of similar posts about it how it can't YYY, but in short amount of time it has become very effective.
ChatGPT for me is a much better StackOverflow. In fact, I'd rather people be asking and answering SO style question in a format that is easier for machines to understand.
ChatGPT is better at searching for a concept, if it was unencumbered, I could better constrain the concept space it was searching over.
Every single time I've seen this claim made and the person shared their methodology for searching on Google, it becomes very apparent that the problem is not Google, but the user.
That’s a bit harsh. Early Google didn’t really require any special skills or knowledge on the user’s part. Current day Google requires hacks and workarounds to avoid both SEO spam and Corporate Google’s attempts to sell you something.
Article claims that chatgpt is overly sensitive to queries eg -gate suffix and come lad vs cum laude. This seems like a weak argument based on a temporary shortcoming. Any NLP methods search engines use for word meanings should be trivially integrated into chatgpt.
Comes off as a company that sells search solutions looking for reasons this tech won't replace it.
I'm sure the author is better informed, but what about the scalability of search? I'm not sure if the neural network approaches are as cost efficient (today) as some of our current search algorithms when it comes to answering queries.
Author here. I was surprised to see my post on here while scrolling!
My point about those examples is less that ChatGPT got it wrong, but that it's impossible to know really when it does, because it seems so confident and you only get the one result. To be fair, this is a problem with webpages, but you at least get multiple choices.
> Comes off as a company that sells search solutions looking for reasons this tech won't replace it.
I can see why you'd say that, but that wasn't the goal. I originally wrote this as my own POV for my own blog. I really do think there's a UX problem in this (and, as I mentioned, I am a big fan of recent LLMs).
> [What] about the scalability of search?
You're right, the NN aren't as performant as lexical search. They're getting a lot better though. (We're actually working on this at the moment.) LLMs, though, have a ways to go, so it's hard to use them right now for real time search.
Just heard of this, so I logged in with my google account. The mistake it makes below is interesting. If this were a human response, I would assume a brain fart. Not expecting this type of error from "A.I."
Me> Find the area of an ellipse with semimajor axis 5 and semiminor axis 4.
ChatGPT> The area of an ellipse with semimajor axis 5 and semiminor axis 4 is 40π. You can find the area of an ellipse using the formula: A = πab, where A is the area of the ellipse, a is the length of the semimajor axis, and b is the length of the semiminor axis. In this case, the semimajor axis has length 5 and the semiminor axis has length 4, so plugging these values into the formula gives us: A = π(5)(4) = 40π.
The trick is to get it to show its work first and then calculate the answer. If it guesses the wrong answer first and then writes out the steps, it won't correct it.
This works: "Find the area of an ellipse with semimajor axis 5 and semiminor axis 4. Don't calculate the result until the end."
After computing your example, I tried my original question which now gives the correct answer. In fact, it gives essentially the same response for both forms of the question. I guess the training works as it should.
Being not very good with numbers is one of the limitations of the current model. I imagine that in the future we will see integrations with a number of different tools to overcome these limits - WolframAlpha being the first one that comes to mind when talking about algebra and calculus.
Algolia's main product is search related. You can tell from this article that they are scared about ChatGPT even partially replacing search. They are very scared of losing business, and anyone who is scared of ChatGPT will be writing articles like this, badmouthing ChatGPT.
Precisely my experience with 95% of stuff I google.
I don’t know if that’s just the nature of the things I usually search for (primarily technical subjects where the query shouldn’t be “dumbed down” and reinterpreted or you get a completely different meaning), but I’ve become incredibly dissatisfied with Google over the past few years. I imagine it’s a fine search engine if you’re looking up the URL for Netflix.
Google already provides a chatgpt-like experience with the questions it displays based on the query. It is unlike chatgpt in that google comes up with those questions but they are usually spot on and you can see the source where the answers are coming from. Perhaps all google has to do is provide a chatgpt like interface to that subsection of search results so it feels more interactive like chatgpt
Have they not heard of it? The web is dead and as the poet said, giants will roam the earth once more.
Every time I interact with davinci-003 it makes me feel more and more like the first real paradigm change is coming. We are still using our computers and writing our programs in the same way as 70 years ago, I emplore you to read 'structured programming' debates and see how we argue about the same things now. Every business product basically still competes with a word processor and spreadsheet. We have made hundreds of thousands of lines of code so we can auto focus the next field on a form, and it still doesnt work on 99% of the cases. The GUI was not a paradigm change, it was more of the same, maybe even worse, mobile overpromised, anything besides video is just worse desktop.
Lists and tables.. for 70 years
This is the first real change.
--
"And what doeth the saint in the forest?" asked Zarathustra.
The saint answered: "I make hymns and sing them; and in making hymns I laugh and weep and mumble: thus do I praise God.
With singing, weeping, laughing, and mumbling do I praise the God who is my God. But what dost thou bring us as a gift?"
When Zarathustra had heard these words, he bowed to the saint and said: "What should I have to give thee! Let me rather hurry hence lest I take aught away from thee!"--And thus they parted from one another, the old man and Zarathustra, laughing like schoolboys.
When Zarathustra was alone, however, he said to his heart: "Could it be possible! This old saint in the forest hath not yet heard of it, that god is dead!"
I think this was framed around using google to answer a question vs using Chatgpt.
For example. Asking "How can I tie my shoes?" to google requires that I click on a link (or multiple) and hope for the best. Meanwhile Chatgpt is likely to give me a detailed walkthrough and answer any follow up questions.
Another good example someone mentioned was "How did ww2 start?". Google gives you a path to find the answer, but Chatgpt will outright give you an answer.
I'd say Chatgpt's competition is really google assistant and siri vs google search. Personally, if I'm using "Okay Google", I don't want to be referred to links as that requires me having to pickup my phone.
I used (still use) Wolfram Alpha for the longest time. Back in the day, I was amazed that it could calculate a bunch of things that were way out of range for Google, etc. - and I always envisioned future search engines to be sort of hybrids, you'll get linked to the correct websites, but you can also ask them for whatever you want.
I don't need ChatGPT or similar models to replace search engines, I just want to use them in parallel.
I'm excited to see what Google does with their LaMDA model, now that Bing will incorporate ChatGPT.
I really wonder how the economics of running chatgpt for Bing's request will work. GPT is anything but cheap to run, IIRC, it requires more than four A100 GPU cards.
Of course- it makes sense to run ChatGPT free now (reminded that it's an experiment) for training & betatesting. But running it even larger, and on a search engine ? It's a magnitude of difference imo, and totally different than a short-time bill for an experiment.
Who gives a shit how much it costs. The value is massive and 10x better than Google results. Are you comprehending what that means? After competing with Google for over 10 years with nothing to show for it, Microsoft is drooling over this technology.
There’s a very old story about Microsoft and Win 3.x pen computing. One of the leads was explaining that they worked on it, and someone else opined that it was too bad that they didn’t provide anything useful for MS. The lead replied that the goal wasn’t actually pen computing, it was a special teams “block that kick” play that kneecapped a competitor and so achieved its goal.
If MS can reduce the absolute firehose of money that ads provide to Google, it will totally be worth it even if it takes lighting money on fire to do it.
The curls have returned two distinct IPs. Both are google's. How probable is that for a plausible guess? Run the experiment yourself. There was an article a while back that shows the prompt preamble to get you going. See what you get.
["Overall, the specific details of the dataset used to train GPT-3 are not publicly disclosed, so it is not possible to say for certain whether interactive sessions by trainers were used in its creation." - the horse's mouth]
As I just said elsewhere it's not going to "run curl commands" in training time. it's trained on a corpus of public data, it's not running commands at all
Noticed this as well. Just like you can ask it for showing `uname -a` and others. The results tend to vary given the temperature of the result and just seem to invent realistic bits of data from the one it noticed in training (e.g., Stack Overflow ?).
The training technique doesn't require internet access as well, it's not going to give you the IP it used during training.
No, it isn't. It generates 1 token at a time in a loop until the response is finished. It's a highly serialized task. Parallelization increases the throughput of how many queries can be processed simultaneously, but you wouldn't be able to speed up a single query.
Making a gpt query on every search I'd say that's indeed unlikely. Luckily, most queries are repeated, so caching can work very well.
There are many opportunities there, gpt could potentially be used for common queries to expand the results and even as a way to disambiguate queries. For instance, if I ask it:
"If I make a search query for "go", what are the possible different things I may be looking for?"
I get
> There are many possible things that someone might be looking for when they make a search query for "go". Some possible interpretations of the term "go" include:
>
> The board game "Go"
> The programming language "Go"
> The command "go" or "Go!" as a signal to start or proceed with something
> The verb "go", as in to move or travel from one place to another
> The website "GO", which is a popular search engine
>
> It's also possible that the person making the search query is simply looking for information about the word "go" itself, such as its definition, pronunciation, or usage in different contexts.
there may be better prompts, of course.
Also, they may identify some queries as being gpt friendly and get those through gpt, which they may also augment with a suitable prompt. The thing is, giving the query as is to the GPT model is not the only option. They can certainly be creative with how they ask gpt and interpret the results. They don't have to necessarily even display the gpt response, they can use it to improve the results.
Exactly. I keep reading all this hype and folks don't realize how much compute is spent on each ChatGPT query (I'm sure it'll be optimized over time). There's no free lunch
I've been using ChatGPT to suggest me books similar to books I've liked and so far the results have been good. Many of the books that are recommended are books I've read and enjoyed so the remaining ones on the list are added to my reading list.
How long would it take for "ChatGPT farms" to crop up and try to influence the results? How much harder will it be to "clean" the results from any learned data set?
The problem with chatgpt is that I can't evaluate the source it used to learn the information to see if I think it's credible, and chatgpt can't give me any indication of whether it's credible or not, and its answer will be just as authoritative in tone if it's completely incorrect as if it's correct.
I asked it for a schematic to connect an i2c device to a microcontroller, it told me to directly connect the device to the microcontroller. I asked it if pull-up resistors were required, it said yes, and then drew what it called "pull-up resistors" (describing them correctly) incorrectly in series between the bus and the device.
I know that's wrong, but someone who doesn't really know what a pull-up resistor does or why they are there would be at least somewhat confused by that interchange, and would be misled into doing it wrong. Meanwhile, I can type in "i2c schematic" to google and get thousands of correct schematics back to look at.
I remember when web search engines were new. And boy did they suck. You wouldn’t get what you wanted a lot of the time, and they were very finicky about the input query.
These days you can just throw stuff at it like “that movie about the depressed guy with a plane engine that falls on their house”
I’m not sure ChatGPT is going to replace search engines, but I’m confident that it is incredibly young and will evolve considerably.
In 2001-2003, browsers crashed every 5 minutes then and Google's search didn't crash nor freeze nor slow it due to not being covered in ads and popups (having only 1-3 ad results at the top of the page with green text was a massive improvement). On tech forums back then, people would say, "is this even sustainable?"
Microsoft search and the other ones were just "there was an attempt" tier and gave complete nonsense results. When Google debuted, it was the only time web search substantially improved. There were no other big innovations in web searches ever, from that day on.
> These days you can just throw stuff at it like “that movie about the depressed guy with a plane engine that falls on their house”
I have looked for obscure queries like yours for 10 years and have not found them. For example "that movie where police vehicles are dispatched and they all crash for seemingly no reason on the way to the scene". It just won't find it because nobody talked about it (in a manner that would cause words that stick out to match), simple. And now days it's even worse because it does that new "rounding" thing and gives me:
- The 25 Best Police Movies of All Time
- Dispatched (2014)
That first result simply contains nothing to do with my query other than the world "police" and "movie".
Now days when you try to use a web search, you can't be smart and search strings that are likely to be used by someone discussing your topic of interest anymore, because it will always "round" them. It will also look at each word as if they are the literal thing you are in search for. I don't know if it's still possible to search bodies of text to get around that, but I'm not sure if that feature even ever worked. Back in 2005 when all these people were raving about intitle: inurl: inbody: whatever, none of them worked consistently, and you would also get scolded for hacking if you used them "wrong". Even searching something like "powered by phpbb" got me banned or scolded once when I wasn't even using it for hacking but rather as a workaround to narrow my search to only forum websites.
EDIT: Nah that's not it. It's way too old. And in mine the police crash for no reason before they even get anywhere near the scene of the crime.
I will skim thru it, but it doesn't look like the right setting. The movie was some generic action movie for some reason I remember Denzel Washington being in it but I looked through his movies from 2006-2012 and didn't find it. It was around that time. Setting had to be something like late 90s or later. Black SUVs dispatched and crashed for no reason at all. I have no idea what the movie was about but it was hilarious when this happened because IIRC it was a serious movie.
Literally all I remember is one scene where a bunch of cop cars or vans get dispatched for some reason like someone's about to die or a heist is happening, something of that nature. They all crash and roll over and wipe out on normal traffic roads for absolutely no reason, and the movie isn't even a comedy, just a generic action movie.
I can't remember one single thing other thing about it, not even the plot.
Probably the re-boot of The Blues Brothers. Even ChatGPT suggested that :)
>What movie is where police vehicles are dispatched and they all crash for seemingly no reason on the way to a scene
The movie you are describing sounds like "The Blues Brothers". In the film, Jake and Elwood Blues embark on a mission to save the Catholic orphanage where they were raised from foreclosure. In the course of their journey, they lead the police on a high-speed car chase through the streets of Chicago, causing numerous police vehicles to crash. Is this the movie you were thinking of?
This point would be better without such extreme hyperbole. Google search was an improvement but it’s not like using AltaVista was that bad - many of the advanced search operators you mentioned got started there.
Crashes were also far less common than you’re describing - that got worse in the 2000s when Flash became ubiquitous.
I do remember constantly hearing that Alta Vista was good but I think I only used Microsoft and Yahoo (I forget what else there was). Nope, crashes were definitely common. Browser quality until around 2010 or so was absolutely horrible. Firefox was another case like Google where it debuted around 2000-2003 and everyone immediately switched to it because it was a massive improvement over IE which barely functioned at all. It still wasn't good, just better than the other crap. The types of bugs in Firefox back in 2003 are absolutely embarrassing by even the common software engineer's standards today. Then of course Firefox just grew in bloat at the same time as it grew in amount of people to fix its problems and now we have a different kind of crap.
> Nope, crashes were definitely common. Browser quality until around 2010 or so was absolutely horrible.
Again, this is way too hyperbolic. I was there too, and while browser crashes weren’t uncommon you’re overstating it massively. Crashes every few minutes weren’t even the case in 1995.
Oh, I see, you were taking that literally. I'd estimate more like 2-5 times per day. Of course this entirely depends on which content you were viewing, and as soon as you ventured out side of the good boy zone it got much worse.
426 comments
[ 3.0 ms ] story [ 289 ms ] threadGoogle has LaMDA, which I could see them putting the answers from it into the knowledge box at the top of searches. There is no reason they have to be mutually exclusive, a search engine can provide answers from both. Google search is already an aggregation of multiple sources (images, web index, knowledge, shopping, video, flights, etc...). Adding another source seems like the obvious path forward (assuming accuracy and cost make sense).
I don't want to ban you because you've also posted good things and if you'd simply follow the site guidelines you'd be a good contributor, but the current pattern is a problem. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and fixing this, we'd be grateful.
"Why touch screen phones won't replace clamshell phones any time soon"
"Why electric light bulbs won't replace gas lighting anytime soon"
"Why transistors won't replace radio tubes anytime soon"
"Why drones won't replace fighter aircraft anytime soon"
"Why electric cars won't replace combustion engines anytime soon"
"Why linux desktop won't replace windows anytime soon"
I, for one, look forward to a future in which I can ask a digital assistant to find and summarize information on anything:
"Computer, please find out how GPS works and explain it to me like I'm five years old."
"Computer, what are the latest Covid-19 infection stats in my city?"
"Computer, who is Claude Shannon and why is he important?"
"Computer, what are the top three stories on HN today?"
"Computer, what's on my calendar?"
etc.
Edit
I think the direction of these models seems clear, and their capabilities however limited at the moment are outstanding. I think the best measure of performance is what the naysayers complain about. We've quickly gone from "actual gibberish that looks like paragraphs" to "yes but it loses track after a few sentences" to "yes it's kind of coherent but doesn't make sense totally" and are now at "it's like an overconfident human at many tasks, when they're only somewhat qualified in the area". People are pointing out where it's got an API call wrong, without it having any ability to look up the answer and see if the function actually exists.
I spoke to it about distributed computation design and discovered something new I can actually implement in my design.
I'm very excited about where this will all go because it seems useful now and I'd be very surprised if this is the best that it ever reaches.
The important thing is that these AI systems are only going to get better over time.
IMHO, the early 2020's for conversational AI is akin to the 1980's for personal computers.
Anyone not looking at chatgpt and thinking "what if this was even just a bit better? Or had better iterations or lookups?" is missing out on what I think is a revolutionary change coming probably sooner than people expect.
My mother was into all kinds of new age and supernatural stuff. You may think you can see where this anecdote is leading, and oh boy I could go there, but instead prepare for a twist! One of the stories she had was a client saying "It is not possible, Susan, it is not possible that there are beings out there that are cleverer than what we is".
And as a logical aside: if AI can't be trusted, we shouldn't trust AI's recommendations for reliable sources either.
Don't think that's automatically true though: an AI can be consistently excellent at retrieving reliable sources and still get a lot wrong on its summaries. Also humans have their own opinions on sources even when they're not familiar with the detail, and finding out if the AI reads Nature or 4chan is part of the debugging process...
”I’m sorry, I cannot answer that query because it has ableist connotations. I am a large language model by OpenAI.”
ChatGPT:
GPS stands for Global Positioning System. It is a way that we can figure out where we are in the world using special satellites.
Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so they can see everything on the ground.
When you have a GPS device, like a phone or a car, it can talk to these satellites and ask, "Where am I?" The satellites send back a message that says, "You are here!" and gives your device a special code.
The GPS device can use this code to figure out exactly where you are on a map. It's kind of like a magic trick! You can use GPS to find out where you are, or to find your way to a new place. It's super helpful when you're lost or want to go on an adventure.
Not bad!
As I mentioned elsewhere, conversational AIs in the early 2020's are like PCs in the 1980's -- a new technology that still has lots of early limitations.
(Also, the UK comedy quiz show QI made the exact same mistake).
How will future LLMs be able to conduct that level of reasoning?
I feel it's appropriate in terms of simplicity, but I think providing a simple wrong explanation is really only worth it when the more accurate explanation is substantially harder to simplify, and I don't think explaining how GPS works falls into that category.
Something like this I think is more accurate without being any more complex:
> Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so you can always see some of them.
> When you have a GPS device, like a phone or a car, it can look for these satellites, and if it sees enough of them it can use math to figure out where you are.
> The satellites all broadcast a special code saying where each of them is. When you have a GPS device, like a phone or a car, it can listen to these codes and find out where and how far away different satellites are.
(Actually, this would be a fun thing to do with a class. Have a few kids march around the edges of the playground, saying things like “I’m next to the swingset now!”, and then ask one one kid to close their eyes, put them somewhere in the middle and ask them to guess where they’re standing.)
Something GPT discourse has been demonstrating to me is that I'm not usually a very careful reader. I apparently skim a lot. Or maybe I skim GPT outputs because I'm biased in my expectations already?
Which one is more appropriate and/or typical will probably depend on what questions the five-year-old is asking, but I think it's reasonable to say the former is usually preferable to the latter.
It does in a sense because the radio waves need an approximate line of sight to reach your GPS receiver. Being high up gives them a large coverage.
About half the (presumably human) "eli5 GPS" answers I found on Reddit made the same mistake, so chatGPT just copied an apparently popular misconception.
You'd be better off just saying "it uses satellites and stuff".
I think that's one of the biggest problems with using LLMs for accurate answers. A lot of the text that's very useful for modelling human language and questions is also full of factual errors and questionable opinions. Answers being wrong is a problem with old-fashioned web search too, but I think LLMs necessarily lean more on quantity rather than quality of content...
If you can do "here is the structured data, summarize it and make sure to present this information" it can do that quite well.
So far I've played with GPT doing a zero shot classification of HN titles and CSV hourly weather data to summary. In each case, it has done quite well.
Neither of those are things about what it "knows."
Ian Calvert Computer, please find out how GPS works and explain it to me like I'm five years old
Sure thing! GPS stands for Global Positioning System. It's a way to find out where you are on Earth.
GPS works by using satellites that are way up in space. There are lots of these satellites orbiting around Earth, and they are always sending out signals. Your GPS receiver is a special device that can pick up these signals from the satellites.
The GPS receiver knows how far away it is from each satellite because of the time it takes for the signal to reach it. By figuring out how far away it is from a few different satellites, the GPS receiver can figure out exactly where it is on Earth.
So, if you have a GPS receiver in your car, phone, or on a special device like a watch, it can use the signals from the satellites to tell you exactly where you are and how to get to where you want to go. Does that make sense?
---
I was also able to ask it to make it simpler and simpler and it did so pretty well.
You'd better know enough (and be alert enough) to tell the difference...
Google, and all the other search engines, literally show snippets of the original content to display search results, and have had copyright infringement lawsuits over this.
Yet somehow, it's taken these AI, which generally aren't just block-quoting things unless you ask them to (and often getting it wrong if you do), for non-lawyers to start asking that question.
GPT definitely has some kind of internal model of things and how they relate to other things, and the capacity to express those relations in different ways.
Whether this is functionally the same as the model in my head, or some analog to it, or if such descriptions are merely cute but misleading anthropomorphisation, is far beyond my knowledge of neuroscience, psychology, and indeed machine learning.
Science.
Either way I don't know. Might be the same, but that might be an unwarranted anthropomorphisation.
So the explanation is either: satellites and their position or give the full explanation. Not this regurgitation.
Explain like Im five means break it down, not lie about the complicated bits.
Its terrible. This is totally not how GPS works, I.E. the terminal never sends a message to the satellite. This is the danger of early LLMs, a large amount of their output is false and there is no way to tell unless you are already an expert.
This makes them basically useless except for a toy.
Probably the “false output” you mentioned is due to the fact that the prompt asked to “explain it like I’m five”, which made ChatGPT answer with a “dialogue between terminal and satellite” explanation, which (arguably) may be better understood by a 5 years old.
Imagine that the GPS satellites orbiting the Earth are like the magic lanterns in the sky, sending signals down to a receiver (like the one in your phone or car). The receiver can use those signals to figure out how far away it is from each of the magic lanterns. By measuring the distance to multiple lanterns, the receiver can triangulate its position and figure out exactly where it is on the surface of the Earth. So even if you're a big, green ogre like Shrek, you can use GPS to find your way around and never get lost!
"Computer, can you write and deliver a message to my kid that explains how GPS works?"
"Computer, can you tell me what risks there are at dining at McDonalds down the street?"
"Computer, can you read this paper [link] and tell me why it's referencing Claude Shannon?"
"Computer, I need to waste some time - please use HN to help me"
"Computer, tell me about any plans that I have for today"
I've gotten it to fail on some word problems, but even then it was impressive to see it try to come up with a solution. More humorous were requests like 'Create a post for ycombinator about ChatGPT in the style of Chris Farley'. It's surprising and quite welcome that it even attempts that.
There's "precedents", tho. MidJourney, the art AI, trained Stable Diffusion over favorites generations of their community, making it a "fine tuned Mid-Stable Diffusion" that was quite amazing (that was the --beta and --betap flags a couple of months ago there).
It can be bad, but it can also be great.
We already see it with recipes - there are all the god-awful sites that have 1000 words of fluff before you get to the recipe, because that's what you need for SEO purposes. GPT is really good at creating that kind of useless fluff, but it also obfuscates the need for it, because it'll just give you the recipe you want without the need to go to a website at all.
Q. Find me the best italian restaurant in New York. A. It's likely subjective - let's just sell it to the highest bidder.
Q. How to improve mental health? A. Give a reasonably good answer and probably mention: some say a solar lamp is good in winter months, there's a good one:
I somehow feel like people can be easily baited into buying something when you mix good, useful information with ads. Kind of like how reputed youtubers/niche influencers do product placement and make referral money.
All of this kind of makes SEO obsolete as ChatGPT ads might end up having more conversions than regular Google ads.
[1] https://www.jasper.ai/surfer
I only have the anecdata of my friends and myself, but we have yet to encounter a casual conversation type translation that DeepL or Google translate have failed on for Spanish <-> English <-> French. Granted, it doesn't always return the most optimal translation, but it has always given the intended core idea.
ChatGPT might not replace search (entirely), but for many many use cases, it has already replaced search.
When pure machine translation was starting out, there were lots of similar posts about it how it can't YYY, but in short amount of time it has become very effective.
ChatGPT for me is a much better StackOverflow. In fact, I'd rather people be asking and answering SO style question in a format that is easier for machines to understand.
ChatGPT is better at searching for a concept, if it was unencumbered, I could better constrain the concept space it was searching over.
But Google search so broken (at least for me) so it looks more like broken or badly implemented ChatGTP.
Every single time I've seen this claim made and the person shared their methodology for searching on Google, it becomes very apparent that the problem is not Google, but the user.
Comes off as a company that sells search solutions looking for reasons this tech won't replace it.
I'm sure the author is better informed, but what about the scalability of search? I'm not sure if the neural network approaches are as cost efficient (today) as some of our current search algorithms when it comes to answering queries.
My point about those examples is less that ChatGPT got it wrong, but that it's impossible to know really when it does, because it seems so confident and you only get the one result. To be fair, this is a problem with webpages, but you at least get multiple choices.
> Comes off as a company that sells search solutions looking for reasons this tech won't replace it.
I can see why you'd say that, but that wasn't the goal. I originally wrote this as my own POV for my own blog. I really do think there's a UX problem in this (and, as I mentioned, I am a big fan of recent LLMs).
> [What] about the scalability of search?
You're right, the NN aren't as performant as lexical search. They're getting a lot better though. (We're actually working on this at the moment.) LLMs, though, have a ways to go, so it's hard to use them right now for real time search.
Me> Find the area of an ellipse with semimajor axis 5 and semiminor axis 4.
ChatGPT> The area of an ellipse with semimajor axis 5 and semiminor axis 4 is 40π. You can find the area of an ellipse using the formula: A = πab, where A is the area of the ellipse, a is the length of the semimajor axis, and b is the length of the semiminor axis. In this case, the semimajor axis has length 5 and the semiminor axis has length 4, so plugging these values into the formula gives us: A = π(5)(4) = 40π.
This works: "Find the area of an ellipse with semimajor axis 5 and semiminor axis 4. Don't calculate the result until the end."
I don’t know if that’s just the nature of the things I usually search for (primarily technical subjects where the query shouldn’t be “dumbed down” and reinterpreted or you get a completely different meaning), but I’ve become incredibly dissatisfied with Google over the past few years. I imagine it’s a fine search engine if you’re looking up the URL for Netflix.
Every time I interact with davinci-003 it makes me feel more and more like the first real paradigm change is coming. We are still using our computers and writing our programs in the same way as 70 years ago, I emplore you to read 'structured programming' debates and see how we argue about the same things now. Every business product basically still competes with a word processor and spreadsheet. We have made hundreds of thousands of lines of code so we can auto focus the next field on a form, and it still doesnt work on 99% of the cases. The GUI was not a paradigm change, it was more of the same, maybe even worse, mobile overpromised, anything besides video is just worse desktop.
Lists and tables.. for 70 years
This is the first real change.
--
"And what doeth the saint in the forest?" asked Zarathustra.
The saint answered: "I make hymns and sing them; and in making hymns I laugh and weep and mumble: thus do I praise God.
With singing, weeping, laughing, and mumbling do I praise the God who is my God. But what dost thou bring us as a gift?"
When Zarathustra had heard these words, he bowed to the saint and said: "What should I have to give thee! Let me rather hurry hence lest I take aught away from thee!"--And thus they parted from one another, the old man and Zarathustra, laughing like schoolboys.
When Zarathustra was alone, however, he said to his heart: "Could it be possible! This old saint in the forest hath not yet heard of it, that god is dead!"
For example. Asking "How can I tie my shoes?" to google requires that I click on a link (or multiple) and hope for the best. Meanwhile Chatgpt is likely to give me a detailed walkthrough and answer any follow up questions.
Another good example someone mentioned was "How did ww2 start?". Google gives you a path to find the answer, but Chatgpt will outright give you an answer.
I'd say Chatgpt's competition is really google assistant and siri vs google search. Personally, if I'm using "Okay Google", I don't want to be referred to links as that requires me having to pickup my phone.
I don't need ChatGPT or similar models to replace search engines, I just want to use them in parallel.
I'm excited to see what Google does with their LaMDA model, now that Bing will incorporate ChatGPT.
If MS can reduce the absolute firehose of money that ads provide to Google, it will totally be worth it even if it takes lighting money on fire to do it.
["Overall, the specific details of the dataset used to train GPT-3 are not publicly disclosed, so it is not possible to say for certain whether interactive sessions by trainers were used in its creation." - the horse's mouth]
The training technique doesn't require internet access as well, it's not going to give you the IP it used during training.
Very funny tho.
No, it isn't. It generates 1 token at a time in a loop until the response is finished. It's a highly serialized task. Parallelization increases the throughput of how many queries can be processed simultaneously, but you wouldn't be able to speed up a single query.
There are many opportunities there, gpt could potentially be used for common queries to expand the results and even as a way to disambiguate queries. For instance, if I ask it:
"If I make a search query for "go", what are the possible different things I may be looking for?"
I get
> There are many possible things that someone might be looking for when they make a search query for "go". Some possible interpretations of the term "go" include:
>
> The board game "Go"
> The programming language "Go"
> The command "go" or "Go!" as a signal to start or proceed with something
> The verb "go", as in to move or travel from one place to another
> The website "GO", which is a popular search engine
>
> It's also possible that the person making the search query is simply looking for information about the word "go" itself, such as its definition, pronunciation, or usage in different contexts.
there may be better prompts, of course.
Also, they may identify some queries as being gpt friendly and get those through gpt, which they may also augment with a suitable prompt. The thing is, giving the query as is to the GPT model is not the only option. They can certainly be creative with how they ask gpt and interpret the results. They don't have to necessarily even display the gpt response, they can use it to improve the results.
How to make a Sazarac
What’s a good waffle recipe
How do I change the O2 sensor on a 1982 911SC
I find ChatGPT issues perfectly good answers without all of the superfluous results, words, clickbait, and ads.
And for many other questions that are technical, medical, or trivial in nature I prefer its simple one-shot response.
Is it perfect? No. But neither is Google. I still at times neees to use quotes and booleans to get Google to bring me the results I want.
As a side note, Google is the Microsoft of 1997. Dominant and content. It doesn’t really want change. That’s why it’s time for a new challenger.
I asked it for a schematic to connect an i2c device to a microcontroller, it told me to directly connect the device to the microcontroller. I asked it if pull-up resistors were required, it said yes, and then drew what it called "pull-up resistors" (describing them correctly) incorrectly in series between the bus and the device.
I know that's wrong, but someone who doesn't really know what a pull-up resistor does or why they are there would be at least somewhat confused by that interchange, and would be misled into doing it wrong. Meanwhile, I can type in "i2c schematic" to google and get thousands of correct schematics back to look at.
These days you can just throw stuff at it like “that movie about the depressed guy with a plane engine that falls on their house”
I’m not sure ChatGPT is going to replace search engines, but I’m confident that it is incredibly young and will evolve considerably.
Microsoft search and the other ones were just "there was an attempt" tier and gave complete nonsense results. When Google debuted, it was the only time web search substantially improved. There were no other big innovations in web searches ever, from that day on.
> These days you can just throw stuff at it like “that movie about the depressed guy with a plane engine that falls on their house”
I have looked for obscure queries like yours for 10 years and have not found them. For example "that movie where police vehicles are dispatched and they all crash for seemingly no reason on the way to the scene". It just won't find it because nobody talked about it (in a manner that would cause words that stick out to match), simple. And now days it's even worse because it does that new "rounding" thing and gives me:
That first result simply contains nothing to do with my query other than the world "police" and "movie".Now days when you try to use a web search, you can't be smart and search strings that are likely to be used by someone discussing your topic of interest anymore, because it will always "round" them. It will also look at each word as if they are the literal thing you are in search for. I don't know if it's still possible to search bodies of text to get around that, but I'm not sure if that feature even ever worked. Back in 2005 when all these people were raving about intitle: inurl: inbody: whatever, none of them worked consistently, and you would also get scolded for hacking if you used them "wrong". Even searching something like "powered by phpbb" got me banned or scolded once when I wasn't even using it for hacking but rather as a workaround to narrow my search to only forum websites.
I will skim thru it, but it doesn't look like the right setting. The movie was some generic action movie for some reason I remember Denzel Washington being in it but I looked through his movies from 2006-2012 and didn't find it. It was around that time. Setting had to be something like late 90s or later. Black SUVs dispatched and crashed for no reason at all. I have no idea what the movie was about but it was hilarious when this happened because IIRC it was a serious movie.
I can't remember one single thing other thing about it, not even the plot.
I do remember an urban setting.
this might be a good case for lifelogging, at least it would give you a limited set of movies to search
>What movie is where police vehicles are dispatched and they all crash for seemingly no reason on the way to a scene
The movie you are describing sounds like "The Blues Brothers". In the film, Jake and Elwood Blues embark on a mission to save the Catholic orphanage where they were raised from foreclosure. In the course of their journey, they lead the police on a high-speed car chase through the streets of Chicago, causing numerous police vehicles to crash. Is this the movie you were thinking of?
Crashes were also far less common than you’re describing - that got worse in the 2000s when Flash became ubiquitous.
Again, this is way too hyperbolic. I was there too, and while browser crashes weren’t uncommon you’re overstating it massively. Crashes every few minutes weren’t even the case in 1995.