Ask HN: Has the quality of Google search declined for anyone else, or just me?

72 points by ornxka ↗ HN
Google search used to be one of the wonders of the modern digital world, but (and I admit this is subjective) it feels like the quality of results has decreased drastically over the years. I just can't find stuff as easily as I used to. It still comes out on top for e.g. querying for technical information but for other things (image search) I don't even bother with it anymore. It also feels like certain kinds of queries are subject to malicious SEO practices where it becomes impossible to find results that aren't from the same group of tabloids/media outlets.

What happened? Did they let their golden goose starve to death? Or is this all in my head and Google is actually fine for everyone else?

105 comments

[ 18.5 ms ] story [ 3396 ms ] thread
For one, like you said, there are so many spam websites out there that it's extremely difficult for Google (or any search engine for that matter) to gather genuine results. I also think that it's a matter of incentive. As long as traffic doesn't decline, does Google really have any incentive to fix or improve things?

Also, I presume the search algorithm is not how it used to be too, what with ML models being used left and right. I wonder if anyone really knows how we get the results anymore. There are a few initiatives like kagi but I know too few about them to say anything at this point. For now, I am quite content with appending "reddit" at the end of every non-technical query nowadays... at least until they start gaming this as well.

Ask a technical query and you will invariably get a bunch of sales offers for the product in question.
Pretty much the core of the issue. Shouldn't it be possible to, somewhat reliably, filter out all commercial content? That would probably increase the average result quality by a lot.
(comment deleted)
Google search has been in decline for at least the past decade, and been surpassed easily for at least the past 5 years. Try running a local instance of a meta-search engine like https://github.com/searx/searx - you'll never want to go back.
This greatly appeals to me, but I don't quite understand how these work. I looked into searx and friends, and it seems too good to be true. Like, if it's turning around and making requests to Google, don't they detect that as bot-like traffic? Seems like a good way to get your IP blocked or worse.
There is another option: Google offers a paid API for search.
Is it feasibly priced? Often enough APIs from big walled services don't seem to be any more than something to point towards when regulators raise concerns about market dominance.
Feasible for my personal use. 100 free daily quota. 1000 calls per 5$. I'm not affiliated with Google.
Why would it look like a bot? Does your normal search traffic look botlike? It's just a search multiplexer.
I don't think it has declined

It serve its users, you are probably in the niche category that's declining, or rather the casual group is growing in number and as a result your category is being diluted even more, tech stuff usually are old stuff, users prefer new and up to date topics

If you are looking for something specific, i suggest trying few tricks to get better specific results: https://betterprogramming.pub/11-tricks-to-master-the-art-of...

link

That’s exactly what gets worse every year, ime.

It's not, it's the same, they haven't removed what they have already indexed

But if you want to loose credibility yet again and loose your investor money on a new search, go ahead, you do you

I know that's the big money-maker for them, but it irritates me that almost the entire first page of results is adverts made to look like search results.
uBlock Origin element zapper.
It's great at showing explicit ads or "content marketing" ads-that-land-in-it-without-paying-Google-directly.

On one hand you have Google, trying to keep patching an algorithm while also maintaining an ad revenue stream.

On the other hand you have the whole of humanity's ingenuity trying to make money for themselves.

Why do we think one system is going to be able to defeat the whole universe of creative hacks to try to beat that system at any reasonable rate?

(It's a wonder that it's even still adequate... but as long as the main competitors are using the same machine-and-math-driven approach, just with fewer resources to throw at it, it's unlikely that there will be a ubiquitous replacement making a major improvement any time soon.)

It's absolutely declined, no question. Further, the advanced search tools have been all but eviscerated, so it's even more difficult to find what you want even if you know what you're looking for.
Impossible to use. I’ve started adding “ Reddit” to my searches but even now that’s not working as well as it did.
OTOH adding 'forum' (or sometimes 'discussion') has been quite effective recently, unlike a couple of years ago as I remember. I even get the occasional forum result without having to do so. Possibly intentional in response to feedback?
You have to use "site:reddit.com" which selects that specific domain, otherwise you get any page that contains the word reddit, and spammers are certainly catching on to that trick.
Thanks I will start using that. Appreciate the tip.
Reddit is so heavily censored now that the quality is bunk. Been declining consistently for the last 5-6 years.
Isn’t it mostly that the structure of content on the web has changed? It's mostly centralized social media now, so that's where the most relevant results are going to be. Most of the attention is there, anyway.
This topic is posted about once a month for the last 13 years: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
It's the new "this is the year of linux desktop".
Except it seems true
Oh, well I guess it's not just me then.
I read through the search you linked, and it was an interesting journey.

I'd say that I'm not sure the search you linked supports the idea that people have been asking about a decline in quality in Google for 13 years continuously. Most of the threads about that topic seem to start about 4 years ago and pick up in frequency until the now.

Though this could just be a result of the search engine's limited ability to index old threads or some other sampling bias (like there being more users on the site recently).

All of that said here's roughly what I saw in the threads you linked:

2009: People notice that advanced settings in search seem to not work as well as before.

2011: It seems like Google now will start to ignore some of the words in your query if they're uncommon.

2011: Google starts giving preferential treatment to its own products like Google Reviews, Shopping, and Youtube.

2012: Users note Google Plus's "+1"s seem to have an outsized influence on search results.

2013: Google Hummingbird launches - it now appears that your query always gets passed through an interpretation layer. Making precise queries becomes significantly more difficult. Note this was probably (?) to make voice search return better results since those queries were more conversational.

2015: The importance of site performance in general and mobile specifically seems to rankings seemingly increased significantly. Likely influenced by Android One project.

2016: Google increases the importance of following their "webmaster guidelines" and W3C validation.

2016: Amit Singhal who oversaw Google Search for 15 years leaves the company to join Uber.

2017: Google starts to try and identify and deprioritize "offensive", "upsetting", and "inappropriate" content, as crowdsourced from users and paid evaluators.

2018: Google launches an update aimed at websites with low "expertise, authority, or trustworthiness." They state that they punish sites which are "not in line with the general scientific consensus" or which have negative sentiment associated with them on review sites.

- It's about this point that threads about Google's decreasing search quality show a strong uptick.

2019: Google starts using BERT - a machine learning model for interpreting search queries.

2020: Google releases updates aimed at further reducing the rankings of "misinformation" or "biased" content.

- At this point there appears to be strong consensus, even outside of technical circles that Google's search results are deteriorating. To the point where there are memes and popular Youtube videos about it.

2021: Google releases MUM - an AI/ML model that aims to reduce the number of queries a user needs to make by including results for related queries that it thinks you'll make next.

2022 (May): An update to Google's search algorithm deprioritizes reference websites like dictionaries, lyric websites, and wikis while promoting video content.

2022 (June): After years denying that a pages 'freshness' affects its ranking, Google increases its importance in the algorithm dramatically - reportedly affecting the ranking of 35% of search results.

----

I'm not a search engine guy and don't have any particular knowledge about Google's search engine. But reading through the history of HN discussion about it does paint an interesting picture.

I'll leave it to others to speculate about any correlations between these changes and the end product.

I appreciate the analysis! This is definitely interesting to see laid out.

You're right, it was definitely hyperbolic to say it's been posted monthly on a continuous basis dating back that far. As you've pointed out, it would have been more accurate to say it was first posted 14 years ago and has been posted (close to) monthly for the last few years.

So it took 13 years and now google search sucks.
13 years ago this won't be so true. But this topic is generally regarded as true amongst even the non-technical crowd.

Google search has gotten that bad recently.

There’s been some debate about this here on HN and someone made a point that resonated with me:

The quality of _the web_ has been in decline lately.

With ML’s capacity to paraphrase original content and to generate plausible rubbish content from scratch, it’s very difficult for Google’s pagerank (or whatever they call their algo these days) to fight back.

That been said, there does seem to be a fair bit of scraping and paste going on. I’m surprised G is not looking at published dates and lowering the the scammers ranking.

Is it the quality of the web that has been declining, or is our ability to find quality information on the web (due to Google and friends) affecting our perception of the web?

I'm on the fence on this.

Contributors and commenters on HN manage to surface many interesting sites that would be difficult to find using a modern search engine. There is also a large number of old sites that have continued to exist, even if they are otherwise unmaintained, which are also difficult to find using a modern search engine. On the other hand, these sites may be less common than they used to be.

Likewise, scraped and pasted websites are nothing new. It has always been a relatively low effort way to post content. What seems to be new is how often nearly identical pages appear in the top search results. This could be because it is more common, but it may also be because the algorithms are favouring very particular types of content.

Also the quality of hyperlinks has declined precipitously. Google's big insight was that what people say about a page matters more than what the page says itself. Hyperlinks were fundamental to the (pre-2013) ranking algorithm, in ways that were far more fundamental than just pagerank. This worked really well when people authored HTML pages by hand (or in FrontPage or DreamWeaver) and would promiscuously link out whenever another site was relevant.

It works really poorly when all the links are paid for, or bargained for, or part of social media sites that are overrun with spammers and use rel=nofollow anyway, or are internal-only because every site wants to be its own walled garden. That's the web we've got now.

As well as the fact fewer ppl are maintaining their own websites or blog. The content is now in places that cant be crawled as easily like Instagram, Tiktok and Twitter as well as FB groups.
The loss of blogs is so huge in all sorts of ways.

Some random person's travel blog in 2008 would be full of genuinely useful information. That's gone now, replaced by regular people just posting pics to social media and then affiliate-link filled seo-optimized travel sites, half the time written by someone who hasn't even gone to the place but is just copying info and adding sponsored content.

The same is true for essentially everything- gardening, video games, books, bikes, bird watching, baking. The genuine amateur enthusiast content isn't published on the open web in accessible text. It's locked in walled gardens or just never created in the first place, with people choosing to post a couple pictures or videos rather than write a blog post about it.

I used to ridicule geocities and tripod, but I sure miss all those niche sites now.
Look at python documentation. There is a metric ton of spam sites that copy and paste python documentation with ads inserted. Google ranks these higher than the primary source.

I suspect google is boosting ad supported content over non ad supported content. Directly incentivizing paraphrased/copy pasta content.

I see this with questions and answers from reddit or other forums which get syndicated into various other 'developer' sites and get high rankings on Google.

Search engines should let us configure a whitelist of sites for certain categories/context of search.

Which can be so frustrating when a bad answer gets propagated this way.

I've had times where the same "bad" information (whether completely wrong, incomplete, misleading, not best-practice, confusing, whatever manner of "bad") showed up on multiple different sites all on the first page of Google, often clearly copied from one another or the same original quora/stackoverflow/whatever.

I think it's plausible that there are types of spam Google can't fight against, but this ought to be possible (which supports the "malice" theory).
I'm pretty sure that Google search page rank, ranks sites with Google ads HIGHER than the same site without any ads. Of course they would, it gives them better metrics to show their Google ads are effective and they can charge more.
It's not just you. Google is now what Altavista and the other search engines it replaced became. The difference is that those went to crap in only a couple years where Google held out for decades and then suddenly has spiraled to where it is now. And it's the same reasons those search engines in the 90s went down the tubes. Everyone who wasn't there will tell you Google's superior technology (PageRank) is why it won. But that's not really why. It's because AltaVista, Lycos, etc. got awful to use, which cause us to look for an alternative. There was this nice clean upstart that said "Don't be evil." is their motto.

It's time for the cycle to begin again.

With all the Googlers here and this question being asked again and again including by myself I wonder if this feedback has already reached the right folks at google but there just isn't much that could be done given the business incentives at play.
Honestly I don’t think anyone at Google or Facebook really cares that much. They are old companies, the big money has been made, and it’s pretty much impossible to launch a new successful product. Everything big has been acquisitions for a decade. If you climb up to executive level it’s just twiddling thumbs until the cash cow starts to die, then you ride off into the sunset because nothing is going to replace the cash cow.

And then the products - does anyone wake up every day feeling super excited to help push ads and shrill partisan hacks to boomers? It’s like to work there you just accept “this is how it is and I like money” and you punch a card and leave. It must be a miserable existence for anyone who truly ponders it.

I dont know about miserable. The money, benefits and work/life balance can be very good. And while the work is uninteresting, its probably a lot more tolerable than working elsewhere.
It is a philosophical thing for me personally. I did a stretch at a meaningless company with good pay and perks and I was so unbelievably bored. If you have mentally accepted your productive output is without personal meaning and you are only in it for a check, then I agree it’s a good gig. I just personally can’t do it. And I think the general no-fucks-given by the workforce is why a lot of stuff is just stagnating and declining.
Gotcha. I also share the same sentiment but my sentiment is even darker. I think most products dont need to exist - not just Google or BigTech. The same applies to project mgt tools, that new fintech product, or that new SaaS tool. Almost everything has been done. Given that i cant go back to school to learn to do genomics or rocket engineering, im just gonna resign myself to go thru the motions and get a paycheck, and thats it.
I am a perpetual optimist and I disagree that everything has been done. We have 500 kinds of deodorant because the market needs it - each one is a little different and resonates with a different consumer for some reason. Believing "we only need one thing" of anything is authoritarian.

My path out of the grind was entrepreneurialism. Much more rewarding personally and financially. Good luck to you

I don't know, I've been there and it's like I have an opportunity to learn and do something wildly different or new and that's something to look forward to even if the only person that will appreciate what I did is myself. Imagine working at google if you found a flaw in their page rank algorithm or monarch db implementation/design? That would feel gratifying. I personally don't care much if I contribute to a society that gives -1 shits about me or appreciated by coworkers that will throw me under the bus if it meant a brownie point for them (been there, done that).
The right folks is the one person with a high profile that made the decision to switch to AI. The rank and file on here don't have any influence.
I think a small variety of issues...

1. Search has to be hard because filtering spam is a cat and mouse game inherently.

2. Useful information has moved off of searchable sources. Instead of forums, we have discord, facebook groups, slack channels, twitter...

3. There is simply more shit, exponentially, than when I was young, and therefore there's more garbage lying around. The ratio of shit:not shit hasn't gotten better.

My vision for the future of search:

Text search on the web will slowly die. No one trusts the results of random text. Google is in the adversarial position of wanting to censor certain answers as well as present answers that maximize their own revenue.

People will search video based content instead, and use the fact that a human spoke the information, as well as comments/upvotes to vet it as trustworthy material (like on TikTok).

Google search as we know it will slowly die, and then will decline like Facebook. TikTok will steal search marketshare as their video clips span all of human life.

I don't understand this argument. "Votes" and Comments are almost completely gamed on every popular website already, I personally don't trust them in 95% of cases. Video content is usually very much a lowest common denominator type of information, and I don't understand why I should trust "a human said this" over "a human wrote this".
TikTok basically eviscerated facebook over night. Gen Z already makes run of the mill searches on that platform, and is increasingly doing so. Older generations cannot see the massive changes occuring. Facebook stock tanking is not random. Neither is Netflix. TikTok is completely underestimated. All of the new content is being created there. People are not creating text based content on facebook or on blogs. Reddit is the last bastion
Nightmare fuel, suicide trigger, or both?
(comment deleted)
> use the fact that a human spoke the information, as well as comments/upvotes to vet it as trustworthy material

Ha, is this a joke? You can pay people on Fiver to produce videos for you given any script at all

Viral videos dont lie. I'm dead serious. I think the end of Google is social media content in video form
I agree. When Alex Jones looks me in the eye and tells me the military is putting chemicals in the water to turn the frogs gay, I know he's telling me the truth.
Thanks for your insight. My experience is that not all people (re)search, for some people, social media is their only source of information online.
What can be done about it?
What would happen to Alphabet/Google if over 50% of people were blocking their ads?

Stop using Google products. Block their ads. Don't use their web browser.

I recommend Mozilla Firefox and uBlock Origin but any other web browser reduces their tracking and allows ad blocking. It looks like Google Chrome will prevent ad blockers soon.

In the last couple of months Google's results have improved for me, I'm no longer seeing stack overflow clones, there are fewer content farm results too.
Two things I love about DuckDuckGo:

1) I can search for a handful of specific terms I remember were on a webpage I didn't bookmark, and I can type them in and find what I'm looking for instead of them being "interpreted" by something trying to be too smart

2) Nobody bothers to send them the same DMCA takedowns that they send to Google, so I can find torrents and things I couldn't otherwise

But on most days, Google is easier to find useful stuff when I don't know specifically what I'm looking for or the exact terms to use.

In an ideal world, I'd have Google default to more of how DuckDuckGo seems to work, with a "fuzzy search" option you can use if needed

If you want high quality materials, you need a University library or a journal, whitepapers or an academic textbook. Or if you want practical information and experience, you need to be an employee or apprentice or business owner.

The web is the lowest common denominator. The high quality information is elsewhere and Google isn't allowed to reveal it due to rights holders, such as Google Scholar and Google Books.

I tend to spend my time on Infinity Family https://o2oo.li and https://halfbakery.com a community of inventors and creators. We have an ontology to solve the world's problems and work towards goals and projects as a community. We relax and share opinionated perspectives to problems in the world.

I agree - stopped using it more than a year go. certain topics it does well on but overall its become a terrible search engine.
No, Google search remains awesome for me. I’d switch if there was something that gave better results, but that doesn’t exist for me. I gave DDG a fair shake (about 3 months) and it was horrid.

I’m also not concerned with privacy/whatever so I won’t switch to something equally as good or worse to make a statement.

I suspect, but can’t prove that the entire internet is rotting. I get the feeling that organic user contributions are plummeting and being swamped by bots. HN is a small life raft. Many of the niche subreddits I’ve been following for years are seeing their post frequency drop, from dozens a days to a few a week. All the other forums are gone. Maybe there’s something going on in the apps, but they aren’t searchable. The internet is becoming a cold dead place. There’s not much for Google to find these days.

Case in point: I was just trying to find some information about TGV rail routes. Standard Google searched yielded nothing but spam. Appending “Reddit” game me posts more than 5 years old with out of date information.

I think something is off with the internet too.

It feels as if internet is sharded just like MMORPG servers. Nowadays there are more humans than ever with access to internet but organic creations are getting harder to find and everything is swamped with SEO spam.

I still can’t exactly point my finger on what’s wrong.

Personally, I think we are in some sort of technological uncanny valley in between pervasive and ubiquitous.
The impact of walled gardens and "use our app for a better experience" has accumulated.

The constant moving of communities is also a problem. We dropped irc, join us on slack. Crickets.. er we moved to discord. Each move must impact engagement negatively.

I feel like so much has moved to Discord. That’s great for immediacy and back-and-fourth interaction but it puts all of that information that used to be Google searchable into a black hole of Discord searches.
You might be interested in Linen (https://www.linen.dev/) which aims to make Slack and Discord communities Google-Searchable. I cannot attest its effectiveness, but I figure it seems relevant.
But it would've been the same thing with IRC. Worse even, since that has no history (by default).
Could you elaborate a bit more about everything moving to Discord? Is this a thing generally or just a side effects of being in a team that uses discord for work?
Many communities have moved to discord. Where once there might have been a web site on geocities and an irc channel or mail list, now there is just a discord server.

geocities and all those little sites are gone, and what replaced them are a sequence of less-googlable less-preservable things like myspace pages, facebook pages and groups, blogspot/blogger pages, and now discord servers and youtube channels.

There isn't a good platform for individuals and small communities that aren't profitable businesses to host their resources and documents and communications. There are only companies trying to capitalize on people's desire for it, providing something that only purports to provide that platform but really doesn't.

In some ways, there is nothing stopping people from making niche topic sites, but something seems to have changed that makes it just less likely. Maybe there used to be free tiers (was tripod and geocities free?) and now there is only cheap ($5 for a vps), or maybe it's that now you have to do more work yourself setting up a vps vs a premade site host, and the current equivalents of free and fully managed are discord & facebook?

Something is definitely different and somehow the aggregate result is a lot less of the old plethora of niche topic sites. Maybe just that everyone got jobs and has no time to make those sites any more, and now all we have are developers personal blog sites which are really just promo for themselves not really of any value to you or me.

Maybe also the passing of the innocent initial window of time where an individual or small group could run a phpBB on the public net without it getting hacked or overrun by spam or sued or in legal trouble for the member-written content etc, without an income stream to pay moderators and lawyers.

Probably it's a combination of those and even more other factors I'm not thinking of.

I have also noticed a kind of implosion happening - an emptying-out of real human-generated thoughts and content. I have hypotheses but they would be considered highly "ideological" in this forum.
>highly "ideological" in this forum.

now you got my attention?

Eh, I'm left wing, that's all, and let me say that I come here to get points of view that are not my own as a way of not existing purely in echo chambers. So I'm not saying this place is bad in any way, in fact it's one of the few places I still come to find links to read and so forth. I barely touch Reddit even, anymore, cause it seems like every subreddit has become captive of one viewpoint or another. Matters purely technical soothe my mind at this point, because there's really no way to cram politics into a manual on network programming, for instance. I was probably feeling gloomy when I posted that last comment.

That said, in the comments here I see a lot of people doing great systemic analysis of the problems of the current internet, which nonetheless (as a Marxist in the real sense of the word) fails to probe deep enough to identify the fact that the internet with these problems is the internet dominated by sites with a profit motive first and foremost.

I got here in the early 90s and hung out on irc and usenet, and I watched the internet become how it is. Way I see it, we had a set of community rules that worked fine as long as we were all people enjoying our spare time here. Once people came wanting to make money, things started going bad and they've only gotten worse and worse. We always had shitty trolls, but now you can actually get rich by being a shitty troll, and that is a much worse situation.

But I think change is coming nonetheless at this point - back in the 90s there was still a lot of middle class affluence here in North America, and GenX was either doing well, or else believing the hard work mythology, the "hustle culture" thing, such that along with the collapse of communism (which was never my jam, though I have tankies in my family) the 90s and aughts were basically Capitalism's victory laps.

But these days American streets are not safe, my own quiet Canadian prairie hometown has shantytowns that spring up all over the place and get cleared out by the cops only to reappear down the street. We call stabbings "The Winnipeg Handshake" now. I grew up there, it was not like this even twenty years ago, let alone when I was a kid and rode my bike all over town, even through the bad neighbourhoods, and rarely got hassled by anyone.

Basically they've got this unsustainable economic system that requires constant steroid shots to keep going on the one hand, which they are simultaneously claiming to be the strongest and best system we can have, and you know how it goes with steroids - at first you get big and strong, but pretty soon you're randomly breaking things in public fits of rage, and eventually your heart explodes and you die.

Capitalism is in its "public fits of rage" stage of its abuse, and the surveillance capitalism internet is reflecting the weakness in that their endlessly gameable algorithms have snuffed out the old internet and simultaneously been seized by bad actors, which is also a good description of politics around these parts (we have fascists up here too, it's not as bad as it is for Americans but we've got our Trump wannabe taking over the Conservative party as I type this).

My remedies would be socialist in nature, and seem like complete nonstarters to me in the current sociopolitical environment: The Internet was conceived by governments, military and universities as a public good. My first Internet access was through my local university, not as a student, just as a public yahoo using their (not actually) paid service - in theory I owe them $1/hr for thousands and thousands of hours, but they never attempted to collect a single one of them, from me or anyone else. The old internet was not about money, and money coming to the internet is what has caused the near-complete societal collapse we are currently facing in just a few years, because the internet is indeed an amazing way to connect lots of people together and democratize things.

So, kick the capitalists off the Internet...

“Profit” isn’t always measured in money. Influence and control are also profit, even if do not increase your bank account size or improve your geopolitical strategic position.

Someone posted a timeline above in years, amusingly it lines up quite well with this hypothesis: https://forum.agoraroad.com/index.php?threads/dead-internet-...

I dunno man. Like I said, back then, we were all just occupying our spare time on MUDs and whatnot. Pretty sure everyone I interacted with could pass a turing test, back then. Bots were things we used to maintain our irc channels and get ops from.

I miss my eggdrop. Schnauzer, I called it.

Rest In Power Schnauzer, guardian of #Spatch

I have no problem with any of what you said. For what that's worth.
Cheers. I actually think that we all agree a lot more than not on most things, but the algorithms make line go up when we're fighting. Line must go up.
Do we need some kind of decentralized chat app? This is what the dweb exists for, right?
(comment deleted)