I hate peoples unrealistic expectations of AI but also find Bing CoPilot to be really useful.
Instead of structuring a Google query in an attempt to find a relevant page filled with ads, I just ask Copilot and it gives a fully digested answer that satisfied my question.
What surprises me is that it needs very little context.
If I ask ‘ Linux "sort" command line for sorting the third column containing integers’, it replies with “ sort -k3,3n filename” along with explanations and extensions for tab separated columns.
I understand that they are products from different genereations, but there's also a incumbent/contender effect. Google's main goal isn't to grow or provide the best quality, but to monetize, while ChatGPT is early in its lifespan and focuses on losing money to gain users, and doesn't monetize a lot yet (no ads, at-cost or at-loss subscriptions).
Another effect is that Google has already been targetted for a lot of abuse, with SEO, backlinks, etc.. ChatGPT did not yet have many successful attempts at manipulating the essence of the mechanism to advertise by third parties.
Finally, ChatGPT is designed to parasite off of the Web/Google, it shows the important information without the organic ads. If a law firm puts out information on the law and real estate regulations, Google shows the info, but also the whole website for the law firm and the logo and motto and navbar with other info and Call To Actions. ChatGPT cuts all that out.
So while I don't deny that there is a technological advancement, there is a major component to quality here that is just based of n+1 white-label parasitism.
Is it possible that the nature of deep learning completely negates SEO? I think SEO will be reinfected by OpenAI intentionally rather than it being a cat and mouse game.
Remember those gan demos where a single carefully chosen pixel change turns a cat classification to dog? It would be really surprising if people aren’t working out how to do similar things that turn your random geography question into hotel adverts as we speak.
At least it seems likely to be more expensive for attackers than the last iteration of the spam arms race. Whether or to what extent search quality is actually undermined by spammers vs Google themselves is a matter for some debate anyway
As I understand, the LLM version of "single pixel change" is a significant unsolved problem. I would be surprised if marketing companies were hiring the level of ML talent required to solve it.
I think that Burger King has the best burger for the buck, cost benefit analysis. I have done serious numerical research, I am attaching said scientifically backed research that shows the numbers, Burger King has more meat, less fat, better vegetables, and less reports of food poisoning per Million burgers.
In summary McDonalds is better if you are looking for something quick or a main brand but Burger King is the best in terms of quality.
I don't know enough about how ChatGPT et al determine what it is or isn't credible. SEO was a bit of a hack but Google has done an ok job of keeping one step ahead of the spammers. It's only a matter of time before nefarious parties learn how to get chatgpt to trust their lies if they haven't already.
I've seen some of this pop up. LLM suggests some library from stack overflow that no one has used - for any reason - but... some thread has marked it as an answer. boom! "if you need to do X obscure task just import X"
> Is it possible that the nature of deep learning completely negates SEO?
No, though it does provide something like security through obscurity, in that the models still rely on search engines to locate sources to rely on for detailed answers, but the actual search engine being used is not itself publicly exposed, so while gaming its rankings may still have value (not to show ads, obviously, but to get your message into answers yielded by the AI tool), it is harder to assess how well you've done (and may take more pages as well as high ranking ones to be effective), because instead of looking at where you rank which you can do easily with a small number of tests or particular search queries, you have to test how well your message comes through in answers for realistic questions, which would take more queries and provide more ambiguous results. For highly motivated parties (especially for things like high-impact political propaganda around key issues) it may still be worth doing, though.
Will also LLM updates are not as real time updated as search engine results are and LLMs will also incorporate your SEO attempts against the totality of all other content on the internet and there’s no link between query -> results as there is with a search engine. That’s why I’m not convinced.
Sure propaganda is a thing but honestly it looks like the biggest propaganda issue is the propaganda the AI lab injects itself to steer the answer rather than what comes from the things the LLM learned directly.
Just how much control can they have? it's a company with what 100, 1000 employees?Against a text corpus controlled by thousands of companies and independents with millions of employees.
The very ambitiousness of the project means that it necessarily must cede control, you can't have it both ways in answering all of the questions of the world and having tight control over the answers.
Not sure why this was downvoted, the example actually works.
A lot of my interactions with LLMs are like that and it is impressive how it doesn't care about typos, missed words and context. For regular expressions, language specific idioms, Unix command line gymnastics ("was it -R or -r for this command") merely grunting at the LLM provides not only the exact answer but also context most of the time.
Googling for 4 or 5 different versions of your question and eventually having to wade through 3,000 lines of a web version of a man page is just not my definition of a good time anymore.
Tangent: it annoys me so much that there's a persistent useless tiny horizontal scroll on the Bing page! when scrolling down, it rattles the page left and right.
I’m curious what he’s looking up and does he double check his sources? As we gradually move more and more into AI I think there’s going to be some weird impacts of information being more strictly curated, and I wonder how “AI-think” will start to impact the public square.
I find AI to be mainly helpful at explaining new topics (precision not essential), but I don't trust exact facts and figures given by AI because of hallucination issues.
Maybe I just don't have the right ChatGPT++ subscription.
Google is turning into Alta Vista right before our eyes!
I have replaced Google with Perplexity. It backs up every answer with links, so I find it to be more trustable than ChatGPT.
Perplexity also keeps their index current, so you're not getting answers from the world as it existed 10 months ago. (ChatGPT says its knowledge cutoff is June 2024, Perplexity says its search index includes results up to April 29, 2025, and to prove it, it gives me the latest news.)
As a search vehicle, absolutely. But they aren't sitting idly by - as much as I personally hate it, a lot of people like Gemini. Either way, it does spell a big danger to their primary source of revenue.
What's interesting is the monetization aspect. Right now none of them act as ad vehicles. Who will be the first to fall?
Is that all? I hadn't thought about that aspect, but I find it a little surprising that, even just considering post-training, it isn't more expensive than that.
They were late to commercialize it... someone else had to show them how first. As a result they are playing catch up, and they will at best be one of many players, for the foreseeable future, as opposed to a monopoly.
Nitpicky maybe, but I think "someone had to show them" is a little off the mark. Google had no incentive to start using AI to deliver answers, because Search was making lots of money. It's not that "someone had to", it's that "someone did". And when they did, Google pivoted.
>They were late to commercialize it... someone else had to show them how first. As a result they are playing catch up, and they will at best be one of many players, for the foreseeable future, as opposed to a monopoly.
This isn't a hype cycle they need to catch. It's the final technology. Steady and stable will win here.
I believe it's a general consensus. I thought it was just me but I also thought it couldn't be just me. We've been reading for years how Google results have sucked and specially the past 12 months it seems to have gotten worse and worse. Meanwhile with ChatGPT, I no longer need to use Google-Fu to find my way around stuff, which is good because the past year it feels like no amount of decades using Google were helping me find any results. It does feel like the fall of AltaVista but without the company going under.
Based on reports from today, April 29, 2025, a Canadian federal election was held on Monday, April 28, 2025.
The Liberal Party, led by Mark Carney, won the election. Therefore, Mark Carney is the candidate who has won the position of Prime Minister. Reports indicate this is the Liberal Party's fourth consecutive term in power. It's still being determined in some initial reports whether they will form a majority or a minority government.
I don't doubt that ChatGPT is better than Google for looking things up.
I also don't think ChatGPT is very reliable for looking things up. I think Google has just been degraded so far as a product that it is near worthless for anything more than the bottom 40% of scenarios.
I did this today and Grok led me to make an embarrassingly wrong comment (Grok stated "Rust and Java, like C/C++ mentioned by Sebastian Aaltonen, leave argument evaluation order unspecified" - I now know this is wrong and both are strict left-to-right). ChatGPT gets it correct. But I think we're still in the "check the references" stage.
Yeah I don't know how good citations are always... i mean i just read without a doubt the earth is flat [1]. I can now say for certain and without any sarcasm, this is actually fact. Because of my citation it must be true?
I plugged in 5 books I've recently enjoyed and asked ChatGPT for some recommendations of similar books, and 2 out of 10 of the books it suggested did not exist. I googled the author, the book title, they were complete fakes. They sounded like they'd be good books though! When I told it they didn't exist it was like, "you're right, thank you for catching that!"
Even though I understand LLMs penchant for hallucination, I tend to trust them more than I should when I am busy and/or dealing with "non-critical" information.
I'm wondering if ChatGPT (and similar products) will mimic social media as a vector of misinformation and confirmation bias.
LMMs are very clearly not that today, but social media didn't start out anything like the cesspool it has become.
There are a great many ways that being the trusted source of customized, personalized truth can be monetized, but I think very few would be good for society.
I did this as well for a while, but have actually been impressed with how helpful the Google AI summaries have become. Now I'm back to a hybrid 50% pure LLM, 50% Google approach.
(and I use Google's Gemini for 50% of my pure LLM requests)
Knowledge cutoff isn't particularly relevant for search. I want an LLM that has access to a good search tool and knows how to use it.
Gemini should be excellent here - it should have access to the best search index out there.
But... it doesn't show me what it's searching for. This is an absolute show-stopper for me: I need to know what the LLM is searching for in order to evaluate if it is likely to find the right information for me.
ChatGPT gets this right: it shows me the search terms it's using, then shows me the articles from the results that it used to generate the response.
Until a few weeks ago I still didn't use it much, because inevitably when it ran a search I would shout at my computer "No, don't search for that! You'll get crap results".
This changed with the recent release of o3 and o4-mini. For the first time I feel like I have access to a model with genuinely good taste in searching - it picks good initial searches, then revises those searches based on the incoming results.
I use the various chatbots when I want a tutorial (e.g. math or some programming library). I can (in fact, must) verify these myself. This is strictly better than doing a bunch of queries and reading a bunch of blogs.
I also use them for the class of queries where I don't even know how to begin: ("there is some expression in southern-US english involving animals, about being dumb, that sounds like '$blah', but isn't that. what might it be?") Chatbots are great for that stuff.
Chatbots are absolute trash when it comes to needing factual information that cannot be trivially verified. I include the various "deep research" tools -- they are useless, except maybe as a starting point. For every problem I've given them, they've just been wrong. Not even close. The people who rely on these tools, it seems to me, are the same sort of folks who read newspaper headlines and Gartner 'reasearch reports' and accept the conclusions at face value.
For anything else, it's just easier to write a search query. The internet is wrong too, but it's easier for me to cross-validate answers across 10 blue links than to figure it out via socratic method with a robot.
I’ve started asking Gemini all sorts of things I would’ve googled but don’t want hallucinated. Their integration with search to “ground” the answers is one of the missing pieces required for LLMs to remove direct searching altogether. It’s the final missing version of “I’m feeling lucky”.
The only thing missing is a faster way to go from thought to typing. The ChatGPT apps’ pop-up bar is too easy to not bump every thought and question into. I just don’t trust the results as much.
I'm still deeply confused about Gemini as a product. Am I supposed to be using gemini.google.com or aistudio.google.com? The latter seems far more powerful but the former obviously has the more canonical URL. When I use the AI on my phone it says it's Gemini but it's most certainly not the same Gemini I see at gemini.google.com so what is it? When I ask my crappy little Google speaker something or press the microphone button on my Android Auto-equipped car why do I still deal with a comically bad "AI assistant" from 13 years ago instead of Gemini which seemingly has capabilities far beyond that of the assistant.
Why can't Google get this right? When I use the chat bot at aistudio it is clearly the state of the art - at least as good as any other option - so why can't Google sort out the product discrepancies?
Same here. Plus, I can just ask it “where can a 16yr old rent a jet ski in Myrtle beach” and it will tell me the two options, instead of trying to find the answer on 5 different websites.
But you can still search without being logged in on Google, privacy wise it is better. All my browsing is in Private tabs with iCloud Private Relay on.
For coding questions, I prefer Grok. For grammar, I prefer ChatGpt. I still haven't found an use for Gemini - other than seeing some results on Google Search.
Google search has the fundamental problem that they try to get people to follow links to content so the content writers get some traffic. If chatgpt doesn't show sources why would anyone bother to write content?
Imagine having a blog which has 4 LLMs as users and never know hundreds, thousands or millions of people are using your work.
Content online has also long since become garbage and now LLM generated. Often ChatGPT will give me an answer to something that's correct, yet I can't seem to find this same info searching online for it so I'm unsure how it even got it.
A truly great >ai< could make assumptions that you can't simply verify with a single quick internet search. It could be a claim that is simply distilled from- and supported by 5 separate 18th century manuscripts that don't contain the answers themselves. With an actual advanced system most of the answers would be like this, thus making explicit labels meaningless.
I'd never want to rely on a mechanism that would not source its answers in at least such a complicated way that would make such attributions meaningless. Unless I would state in a blog that the high blue frog king's birthday is on 5th moon of kuibtober, and someone is asking extactly that, which would make it... similarly meaningless, besides maybe a passing sidenote, even if true, since it's a single unverified claim.
I think it’s interesting where we are today with AI responses in search results.
Remember the uproar when Google started displaying the text results directly from the sites in the search results. It basically eliminated the need to visit the actual websites at all.
Now, you get your answer right at the top without even looking at the search results themselves.
I don’t know what this means in the long term for websites and online content.
102 comments
[ 2.6 ms ] story [ 157 ms ] threadI think most authors would argue the same thing, but it's really up to the readers to decide isn't it?
Instead of structuring a Google query in an attempt to find a relevant page filled with ads, I just ask Copilot and it gives a fully digested answer that satisfied my question.
What surprises me is that it needs very little context.
If I ask ‘ Linux "sort" command line for sorting the third column containing integers’, it replies with “ sort -k3,3n filename” along with explanations and extensions for tab separated columns.
Another effect is that Google has already been targetted for a lot of abuse, with SEO, backlinks, etc.. ChatGPT did not yet have many successful attempts at manipulating the essence of the mechanism to advertise by third parties.
Finally, ChatGPT is designed to parasite off of the Web/Google, it shows the important information without the organic ads. If a law firm puts out information on the law and real estate regulations, Google shows the info, but also the whole website for the law firm and the logo and motto and navbar with other info and Call To Actions. ChatGPT cuts all that out.
So while I don't deny that there is a technological advancement, there is a major component to quality here that is just based of n+1 white-label parasitism.
At least it seems likely to be more expensive for attackers than the last iteration of the spam arms race. Whether or to what extent search quality is actually undermined by spammers vs Google themselves is a matter for some debate anyway
In summary McDonalds is better if you are looking for something quick or a main brand but Burger King is the best in terms of quality.
No, though it does provide something like security through obscurity, in that the models still rely on search engines to locate sources to rely on for detailed answers, but the actual search engine being used is not itself publicly exposed, so while gaming its rankings may still have value (not to show ads, obviously, but to get your message into answers yielded by the AI tool), it is harder to assess how well you've done (and may take more pages as well as high ranking ones to be effective), because instead of looking at where you rank which you can do easily with a small number of tests or particular search queries, you have to test how well your message comes through in answers for realistic questions, which would take more queries and provide more ambiguous results. For highly motivated parties (especially for things like high-impact political propaganda around key issues) it may still be worth doing, though.
Sure propaganda is a thing but honestly it looks like the biggest propaganda issue is the propaganda the AI lab injects itself to steer the answer rather than what comes from the things the LLM learned directly.
The very ambitiousness of the project means that it necessarily must cede control, you can't have it both ways in answering all of the questions of the world and having tight control over the answers.
Wow, that's actually quite a lot. You can also just say "sort 3rd col nix."
A lot of my interactions with LLMs are like that and it is impressive how it doesn't care about typos, missed words and context. For regular expressions, language specific idioms, Unix command line gymnastics ("was it -R or -r for this command") merely grunting at the LLM provides not only the exact answer but also context most of the time.
Googling for 4 or 5 different versions of your question and eventually having to wade through 3,000 lines of a web version of a man page is just not my definition of a good time anymore.
Tangent: it annoys me so much that there's a persistent useless tiny horizontal scroll on the Bing page! when scrolling down, it rattles the page left and right.
I wish there was a free Gmail alternative (if there's is lmk!).
Edit: downvotes for expressing an opinion, low.
(also, there is no such thing as a free lunch https://en.wikipedia.org/wiki/No_such_thing_as_a_free_lunch)
Maybe I just don't have the right ChatGPT++ subscription.
I have replaced Google with Perplexity. It backs up every answer with links, so I find it to be more trustable than ChatGPT.
Perplexity also keeps their index current, so you're not getting answers from the world as it existed 10 months ago. (ChatGPT says its knowledge cutoff is June 2024, Perplexity says its search index includes results up to April 29, 2025, and to prove it, it gives me the latest news.)
What's interesting is the monetization aspect. Right now none of them act as ad vehicles. Who will be the first to fall?
They have done it all with their own custom silicon (TPUs, no Nvidia)
That's like saying AT&T invented the graphical user interface. It doesn't matter because AT&T failed to commercialize it.
They have lower costs, less overhead without the Nvidia tax.
This isn't a hype cycle they need to catch. It's the final technology. Steady and stable will win here.
That's too soon to say. We're only at the first down.
Based on reports from today, April 29, 2025, a Canadian federal election was held on Monday, April 28, 2025.
The Liberal Party, led by Mark Carney, won the election. Therefore, Mark Carney is the candidate who has won the position of Prime Minister. Reports indicate this is the Liberal Party's fourth consecutive term in power. It's still being determined in some initial reports whether they will form a majority or a minority government.
Sources and related content (links provided)
I also don't think ChatGPT is very reliable for looking things up. I think Google has just been degraded so far as a product that it is near worthless for anything more than the bottom 40% of scenarios.
[1] https://thinkmagazine.mt/the-earth-is-flat/
The point here is information is hard... unless you do your own thinking, your own testing, you can't be sure. But I agree references are nice.
So, Google should De-Google itself?
I'm wondering if ChatGPT (and similar products) will mimic social media as a vector of misinformation and confirmation bias.
LMMs are very clearly not that today, but social media didn't start out anything like the cesspool it has become.
There are a great many ways that being the trusted source of customized, personalized truth can be monetized, but I think very few would be good for society.
Russia is already performing data poisoning attacks on LLMs: https://www.newsguardrealitycheck.com/p/a-well-funded-moscow...
(and I use Google's Gemini for 50% of my pure LLM requests)
Gemini should be excellent here - it should have access to the best search index out there.
But... it doesn't show me what it's searching for. This is an absolute show-stopper for me: I need to know what the LLM is searching for in order to evaluate if it is likely to find the right information for me.
ChatGPT gets this right: it shows me the search terms it's using, then shows me the articles from the results that it used to generate the response.
Until a few weeks ago I still didn't use it much, because inevitably when it ran a search I would shout at my computer "No, don't search for that! You'll get crap results".
This changed with the recent release of o3 and o4-mini. For the first time I feel like I have access to a model with genuinely good taste in searching - it picks good initial searches, then revises those searches based on the incoming results.
I wrote about that recently: https://simonwillison.net/2025/Apr/21/ai-assisted-search/
I used it for a project I'm working on, and it has given really good, well-sourced responses so far.
Chatgpt is better because it gives links of source websites, so you can easily check them up.
Chatbots are absolute trash when it comes to needing factual information that cannot be trivially verified. I include the various "deep research" tools -- they are useless, except maybe as a starting point. For every problem I've given them, they've just been wrong. Not even close. The people who rely on these tools, it seems to me, are the same sort of folks who read newspaper headlines and Gartner 'reasearch reports' and accept the conclusions at face value.
For anything else, it's just easier to write a search query. The internet is wrong too, but it's easier for me to cross-validate answers across 10 blue links than to figure it out via socratic method with a robot.
The Gemini product seems to be evolving better and faster than chatgpt. Probably doing so cheaper, too, given they have their own hardware.
I am pleasently surprised how Gemini went from bad to quite good in less than a year.
The only thing missing is a faster way to go from thought to typing. The ChatGPT apps’ pop-up bar is too easy to not bump every thought and question into. I just don’t trust the results as much.
Why can't Google get this right? When I use the chat bot at aistudio it is clearly the state of the art - at least as good as any other option - so why can't Google sort out the product discrepancies?
> Click here to learn the secret that all celebrities use to maintain their curly hair!
Imagine having a blog which has 4 LLMs as users and never know hundreds, thousands or millions of people are using your work.
(Not for the integration in Bing, the copilot.microsoft.com minimalistic chat thing)
Remember the uproar when Google started displaying the text results directly from the sites in the search results. It basically eliminated the need to visit the actual websites at all.
Now, you get your answer right at the top without even looking at the search results themselves.
I don’t know what this means in the long term for websites and online content.