There's been a lot of HN discussion [1][2][3] about Google Search recently, and whether it’s gone downhill.
I used to work on Search and Search Measurement at YouTube, Twitter, and Microsoft, so I thought it would be fun to move beyond anecdotes, grab some data, and do a quantitative analysis.
tl;dr I didn't have historical data, to see how Google Search has trended over time. But compared to Bing, Google still generally outperforms -- although some of its failures are pretty surprising!
If there are particular areas of Google Search that people are interested in digging into, give a shout -- I love running these kinds of search / human eval analyses.
I think the broader theme that these outrage bait posts miss is that the web itself is rotting / deteriorating. Most interesting content isn't on the web anymore and is definitely not in text.
Content that would be have been on an open web forum 10 years ago, are now hidden behind various data silos/walled gardens - whether it's Reddit, YouTube, TikTok, Instagram, Twitter or Discord. Each of these walled gardens have different levels of tolerance for the open web, from using it as an SEO channel (e.g. Reddit) to completely opaque to it (e.g Discord).
Some car and photography forums I was on ~10 years ago are all barren now as the old users have moved on and new users prefer to communicate in a Facebook group or something like that.
Interesting self-published blogs and recipes aren't on the web anymore, they're on YouTube channels or on someone's Instagram channel (or whatever its called).
There's no less interesting information on the internet than there was a few years ago - to the contrary there's much more. I agree that the ratio of good to trash content has plummeted but it's the job of Google and its competitors to find the good stuff. If they're failing at their core competency then it's worth discussing.
youtube now seems to provide me only ~10 relevant results and then random other videos i might like with no way to delve further into results based on my search term. this results in it being almost impossible for me to find certain things, especially videos without a lot of views, unless i know the exact title of the video ahead of time.
what are some of the decisions that go into this ? is it just cost savings ?
The experiment I want to see is one where (1) the same set of test queries is run on multiple search engines (2) for each query result, the result links from all engines are combined into a single document (without including information about the search engine from which they came) (3) the links are ranked blindly for fitness in relation to the search query (4) the fitness rank of the results is then reunited to the source search engines.
Good list. To add to it: I'd like to see the same queries run from a) multiple IP addresses (where you search from seems to affect results), and b) what the differences in results are for those with Google accounts and those doing history-/accountless searches.
I'm pretty sure the term of art for submitting a headline in the form of a question when the article does not and cannot answer the question is "clickbait".
It's definitely deteriorating, and the worst part is that it completely ignores quotes if it thinks you meant something else, and shows the results for what it thinks you want. Completely useless in a lot of cases
This has only happened to me twice so far, but it has given me the "Did you mean:" even when using quotes for searches that definitely have results (as I found them with duckduckgo).
I can't remember what they were now
I have an Intel Realsense camera, which sometimes reports the error "Failed to recconect" (there being a typo in the drivers) [1] - that's a pretty unique error, so in combination with the product name that should be a very easy keyword search, right?
But no, when I search for realsense "failed to recconect" Google returns pages that contain neither realsense nor recconect [2]. They offer me a supreme court opinion, a review of a car dealership, and a facebook church service.
Correcting the spelling of a query is one thing - but also completely ignoring other keywords? I can see why there are so many people posting about the poor quality of Google's search results.
One reason it might deteriorated is that goog is constantly battling ppl 'optimizing' their content for Google while competitors likely see less than 1/1000 of this.
That's a good point! A lot of online content is created for search clicks only, not quality. For example, I often search Google in lithuanian language and I keep getting whole lot of auto-translated blog/article farms as a result. Those translations almost always are shoddy, but these pages keep popping up in the first Google results because a) there is not a lot of native (language) content to push out these clickbaits and b) those clickbaits are on domains that are also, well, in the first results of other lesser languages... I think these auto-translated pages started to pop up two or three years ago...
That would make sense if so many easy to detect low hanging fruit like blog spam, scraped stack overflow pirates, and listicles didn't make it to the top. Those are easy for google to de-rank and yet they don't.
What's the source for it being easy? It seems easy, from the perspective of a human looking at results, but I'm not sure how much that's worth given the scale and complexity of the problem.
Captchas are way harder to solve than it is to detect these sorts of poor results. Google should have absolutely no problem building a classifier that could scale to solve the problem. But as you said, it's not worth it to them, but for the reasons of losing revenue rather than scale and complexity.
Just block the domain. At first, you can block manually, but we know Google doesn't like doing things that way. Fortunately, they have a lot of heuristics to find sites like that; usually the content is just copied from another source. And since they scrape the web all the time, they should know which content has appeared first where.
But the issue isn't that they can't; the issue is that they don't want that. Why the sites with copied content exists? To earn money through ads. What earns Google money? Ads!
Any simple heuristic has false positives, meaning they'll end up taking down legitimate sites that had repeated content for a good reason.
Say, for example two sites quoting text from the us constitution. The second one to be crawled would be considered to be spam copying the first one and removed from web results. Then you'll get comments on hacker news complaining that Google is censoring it for political reasons.
And any simple heuristic is quickly reverse engineered by SEOs, who will find a way to mask it as legitimate.
They could use the heuristics to build a list of domains to block and then have someone review it. After doing it for a long time, they could build a neural model on top of that, and automate it.
As I have said, the reason they don't do it is not because they don't have the skills and know-how.
There are billions of dollars on stake from both sides, search engines and spammers, in an endless arms race that has been going on for more than 20 years.
Trust me, it's beyond naive to say fighting webspam is a low hanging fruit problem.
Why should I trust you? I trust my own eyes. I regularly see spam sites that get to the top of results that are seen by many people for months. These could be filtered with a one line change.
One of the worst things I've noticed recently about Google Search is how it is very anti-startup because of the concept of the Google Sandbox, an essentially arbitrary length of time they put a huge negative penalty on your site to try to entice you to buy paid ads instead before your funding runs out waiting for organic traffic.
Perhaps that's my biased opinion on their motivations as I've recently launched https://grizzlybulls.com and yet even though Bing has tiny market share, I'm getting 10x more organic traffic from Bing rather than Google...
When I search for "databricks series b valuation" in Google (from Argentina, using Google.com in English) result #6 is:
"Python get value from database - Büro Jorge Schmidt", which judging by title and preview seems to be a Python + MySQL tutorial. It returns a 403 error and might be a hacked site, since the home page is for a graphic design studio in Munich.
Result #8 is something similar:
"Intellij flatten packages - Músicos de Viaje". This is definitely a hacked site (from Spain, apparently) that redirects me somewhere else.
Result #10:
"How to calculate tax percentage in sql query". Another hacked site, this time for an evangelical church from Brazil.
Now... how can Google think that any of these sites are relevant? Even if it doesn't realize the pages are hacked... even its crawler has been fed content that included the keywords... :
A - The sites themselves don't match the query at all.
B - No legit site about the subject would link to these sites.
C - The results themselves (title, url, preview), as Google shows them, have nothing to do with the search!
I just tried that search, all of the results look relevant and I definitely don't get any of the results you are getting.
I wonder if you have some malware that is hijacking the results? I once had some malware (chrome extension) that was corrupting my search results. It was surprisingly difficult to remove (given that it was a chrome extension...).
No, I get the same results on Safari on iOS (iPhone) so I have to eliminate the possibility of malware.
Google results are personalized, based on location, search history, etc. The fact that I'm in Argentina has been adding a lot of noise to results on searches where my location is not relevant at all.
In this case, I suspect that Google thinks these hacked sites with developer target content are relevant to me, because of my regular search history.
It sounds like their location based targeting sucks. Spotify has the same problem for me - it spams my playlists & recommendations with songs that I have never, ever shown the slightest interest in purely based on location.
This seems to be a common theme in the industry. Recommenders heavily overweight location - an incredibly general factor, even in the presence of troves of specific, individual level data. Goes to show how little basic reasoning really goes into how these systems work.
How about spamming me with artists I have taken the time to mark as 'Do not play'? The FIRST recommended album in Album Picks is such an artist. Good reminder that I should get rid of my premium subscription, thanks!
All online tools should have a “pretend I’m in Silicon Valley” toggle so you can get the same results the engineers get.
Localized search is useful at times (restaurants for example, I like getting the local McDonald’s or China Wok rather than the biggest one in New York) but it’s completely useless for many terms. But maybe not the ones Google makes the most money on.
Rule #1 of Google usage: turn off results based on search history. Think about it, how could that possibly improve your search results when you search for something you have never searched for before? I (sort of) remember the day that was announced, almost everyone turned it off immediately.
Almost nobody turned if off. And, to answer your question: If your history shows hundreds of queries for Java, PHP, and Ruby, odds are your query about Perl or Crystal or Go isn't about the species or stone or game.
Yes, I did. No trash.
Safari on Mac (or iOS), once I log in, spews the same garbage results.
Copied from another answer:
---
My feeling is that since that query is SO unusual for me, based on my search history (I can't even say what it means) it raises the "likelyhood" that the hacked spam sites with programming terms that also include those keywords are good results for me.
If I search for something more typical, like "Spider-Man No Way Home" or "Ruby rails tutorial" the results don't include hacked sites.
If I use an incognito window I don't get the crappy results either.
My feeling is that since that query is SO unusual for me, based on my search history (I can't even say what it means) it raises the "likelyhood" that the hacked spam sites with programming terms that also include those keywords are good results for me.
If I search for something more typical, like "Spider-Man No Way Home" or "Ruby rails tutorial" the results don't include hacked sites.
I see some people are having trouble replicating the results, and I wonder if you some how got thrown into a really bad user test group--if they do that.
Just tried that search in English. First result is "The data- and AI-focused company has secured a $1.6 billion round at a $38 billion". And if you click the search box you get the first search option as databricks valuation history where the first result is funding every round.
Google has been so much better for search for me than other search engines. Atleast for what I search for programming, news etc.
That being said, I have issued queries in the past that I have just found absolute walls of malicious results. I haven't really invested much attention span in entertaining this sort of thing so I just move on and modify my query, but I'll keep an eye out for it going forward.
- I checked the cached (by Google) copies of the hacked pages and they include mentions of a "Databricks SQL Connector". So if I search for "Databricks" Google thinks "it must be a programming thing".
- If I now search for "databricks series a valuation" I don't get the spam results, for some reason. I think that if I repeat the search Google produces the exact same results... but internally, since I first searched, it might have realized that those sites were not good.
I've been gone from G for 4 1/2 years now. When I was there, the weekly meetings often featured "search quality" measurements that were rigorous in their objectivity (I thought). They bent over backwards to be non-self-deluding.
I distinctly remember Udi Manber saying "if the web is slow, it's our fault" (actually, the speech was that everything is "our fault"), meaning, really, "take responsibility for problems and don't throw up your hands."
However, the natural tendency of any organization is to reward the suckups and promote mediocre people who just get along with everyone. It wouldn't surprise me if that's what's happened with Google, too.
It seems weird to me that Google would take "search quality" seriously for 15+ years (including nearly a decade as one of the biggest companies in the world) and then suddenly stop. Are you able to share any of those objective measures of quality? Because it seems to me that most of the discussions around the declining quality of Google search amount to anecdata backed up by reasoning that doesn't really make a lot of sense to me (e.g., "Google only cares about short-term revenue!")
To me, it seems crazy that a company would actively try to self-sabotage their bread-and-butter product in search of a bit of extra revenue when they're already making record profits. Not to mention that, as the author of the article points out, Google published a well-known quantitative paper on why that's a bad tradeoff.
It happens all the time though. Look at amazon, they flood their marketplace with cheap knockoff products that hurt the credibility of their ecommerce offerings, but the money generated is too much to shut off. So in the short run, amazon makes more money for a worse reputation, and in the long run ??? its unclear
I don't think that's analogous: you'd have to argue that Amazon spent a ton of time and effort on keeping cheap knockoff products off their marketplace and then did a sudden u-turn and decided not to do that anymore. Even then, I don't think the analogy holds, because the quality of Amazon's marketplace listings isn't anywhere near as synonymous with Amazon as the quality of Google's search results is with Google.
>Even then, I don't think the analogy holds, because the quality of Amazon's marketplace listings isn't anywhere near as synonymous with Amazon as the quality of Google's search results is with Google.
What? Why not, and anyway Amazon I'm giving money. I'd think that quality there would be more deeply linked than anywhere I'm just giving some time.
My dad frequently says that sooner or later the finance guys get in charge of the company and then they cut the fat and keep cutting into the product. Sooner or later they go too far. Seems fairly inevitable, seems to happen a lot. I assume that what happens is that the product matures, so the obvious thing to do is make things more financially efficient. Problem is, the company sells a product and the finance CEO's core competency is reducing costs, and reducing costs isn't something that customers actually buy.
The CEO and upper management might simply not care about the long-term viability of the company. Cashing out in the short-term sounds more like what these executives generally do.
Sundar Pichai is probably the weakest of the FAANG executives - if not one of the weakest Fortune 100 executives. Google's search result quality, reduction in overall prestige and lack of new major products are just a few indicators of that.
But speaking in general, there is limited correlation between executive compensation and company performance. You can DuckDuckGo it if you want to find the evidence for this statement.
For whatever it's worth: I had a talk with a WSJ reporter, who said he's met all the FAANG chief executives, and Sundar was the least impressive of them all.
Having everyone you've worked with love you and find you brilliant is not at all the same thing as being brilliant.
It's rational and commonplace behavior for a monopoly to prioritize profit maximization over product quality, there must be thousands of case studies over the years. If a firm has no viable competitors, why do they care whether their product is good or not?
The cases you mention involve maximizing profits by saving on costs. The theory above is that lower quality leads to an increase in revenue, and it is tenuous at best.
That happens when companies have monopolies. They can sacrifice quality for profits. Happens frequently. Obviously you can't sacrifice quality to zero, but you can a bit. It would make sense that they would keep pushing it down little by little (returning higher and higher profits) to find the turning point. I don't think they're yet in danger though because there isn't a real uptick in the competition.
> It seems weird to me that Google would take "search quality" seriously for 15+ years (including nearly a decade as one of the biggest companies in the world) and then suddenly stop.
Is anybody claiming that it’s sudden? Everybody I’ve seen complain about Google search results has seemed to think that it’s been getting slowly worse over a long period of time.
It seems Google started decaying when they gave up their social platform G+
Nowadays all those tweaks to the algorithms, specially search and gmail spam filters, are like a desperate attempt to make the ship move faster by removing parts of the hull while keeping the water out. It will just sink at some point.
I don’t think google search has gotten worse, I think the SEO abusers have just started winning. So much content on the web is automatically generated and made to look just like real hand crafted content. So much of it has titles and headings which have little relation to the search.
I was recently searching for a piece of ceramic cookware, specifically looking to avoid non stick coatings. And google search showed me lots of listings with my exact search terms, but when I click the page it shows their generic product range which has nothing to do with ceramic despite the title saying so.
>I don’t think google search has gotten worse, I think the SEO abusers have just started winning. So much content on the web is automatically generated and made to look just like real hand crafted content. So much of it has titles and headings which have little relation to the search.
right, I mean we have nearly daily posts here about almost human level quality texts you can generate with various machine learning solutions.
Add to that sites who serve users needs must focus on doing that, the sites that just game Google search rankings can focus on that.
I mean maybe the problem is that the SEO abusers are only 35% of the way towards being constructive, a la XKCD https://xkcd.com/810/ - so the thing that wasn't considered is as your spammers get closer and closer to being constructive the more stress is placed on the systems that have to deal with fighting non-constructive spam and the worse it is for the users.
But maybe at some point there will be a breakthrough with SEO abusers being able to be as constructive as the actually useful sites, at which point there will be a short time when the SEO abusers are the only sites returning results, causing all other sites to go out of business, causing the downfall of the SEO sites which rely on actually useful sites for a great deal of their stolen content / content seeding corpora.
Vast majority of these SEO abusers are monetized through google ads. For every advertisement dollar spent through google, google takes 30-50%. And 80% of Google's income is from internet ads.
I would be surprised if google's search quality would not deteriorate - they have very strong incentive for that.
I still get strange results from flaky websites that have strange TLDs and the content is literal garbage. Like just words mangled together. Ads on there are the worst and reach into malware territory. Results like these are on the first page. I have no explanation for it, but 100% of my searches for personal use are DDG or Bing now.
Not sudden for me.
It has been progressively worse over time, and unfortunately the competition is not much better.
Searching today is not as bad as it was during the Altavista/Netscape/56k modem era, but it is getting dangerously close, from my personal and anecdotal experience.
I mean, there are two ex-Googlers in this thread claiming firsthand that search quality was considered sacrosanct as of a few years ago, so it seems like it'd have to have been sudden if indeed it has happened at all.
I didn't notice a rapid decline until early 2019, when the search results stopped containing the keywords I used. Before then there was a blend of synonyms and actual keyword matches.
This was roughly when Google's new AI began "interpreting" the "meaning" of the specific jargon and product serial numbers I was inputting, and then decided what I actually needed was song lyrics.
Well, it's possible they attempted to keep search quality high, and they were unsuccessful but they thought they were successful.
After all, if me, my team and my boss are rewarded when an imperfect measure of search quality goes up and it's going up, why would I fight to switch to a different measure that wasn't rising?
For example, my first week at Google/YouTube, I was in a New Hire meeting with our VP. Someone asked about profitability, and he responded that Larry said we didn't have to worry about revenue yet, since the main goal was user growth/happiness, and revenue could come later. Which I thought was fascinating, considering how big YouTube already was at the time (in 2013)!
Though I think this changed a year later, and I find YouTube ads a poor experience compared to Instagram and TikTok -- which aren't merely "better than the rest", but stuff I actually enjoy watching.
in 10+ years of targeted advertising, i still have yet to see a single ad for something that is actually relevant to what i want to buy.
i'm actually baffled that these companies that have my entire purchasing history and metadata still cannot do a better job at making me spend money than absolute randos on the internet.
Indeed - it really surprised me that we didn't see big ad companies saying:. "we've crunched all the data about you, and here is a list of the products you'll be wanting to buy this week. Buy them all? [Yes] [No]".
The company should be smart enough to know I only buy certain brands of dog food. It should know I need a gift for my mum (and what she likes). It knows I need new trainers but will only buy the cheapest. Etc.
The opportunity for this has pretty much closed with the era of data privacy coming in, but I think it is both surprising and a shame that this didn't happen in the last decade.
This is a fundamental issue with the ad business - the ads are selected from those who bid, not from those that are relevant. For example - if I'm listening to a podcast with a captivating guest, the relevant links would point to their books or blogs. But since they didn't bid for ads, I'm shown some crypto drivel or date-Thai girls scams
I was in Ads (the first time) from 2008 - 2010. At that time, there was a "user-specific" (I forget the name) group that specialized in modifying ads based on the user themself. They were VERY very restricted by the lawyers in what personal data they could use. I imagine it's been considerably loosened by now, but I don't know that.
To someone else's point: yes, they are restricted by who bid for the keywords, but honestly, that's the whole magic of Google Ads. You have a "motivated buyer", someone who actually wants to send flowers or stay in Duluth. How do you know that? They searched for "flowers" or "duluth hotels", duh.
Is an advertiser willing to throw money at people who probably want to send flowers, based on their past behavior? Well, maybe, but not on the Search Results page. There are lots of other web venues where they can place creepy ads like that.
I believe that would have been during the time that google broke the ability to + words (without initially providing another verbatim alternative) in order to use + for google plus somehow.
I can't square that change with the claimed commitment to search engine quality.
just an anecdote, but I see quite a few people in my linked in network who are now at google within the last few years. People I wouldn't consider "google quality".
Not completely sure what this “phenomenon”. There’s a few things I can imagine you are insinuating but they all had simple explanations so I’m not sure.
Don’t disagree with the main theory that search quality is deteriorating. I have to use increasingly contrived queries to get anything but bullshit blog spam, and indexing seems really odd at times.
Nothing that can’t be easily explained, of course, anyone coming with a reasonable explanation is being downvoted by the “critical thinkers” of HN who can only instead provide low effort quips.
But if you just search for “person” or “couple” you get results showing mostly white people and couples. I don’t think what you’ve observed is saying what you think it is…
For the first two, I think that black people use the phrase “white people” more than white people use the phrase “black people”, so “white people” includes black people in the results due to using quotes associated with people in the search ranking. (Whether things people have said should factor into image search is another question.)
well for the first white people I can see that the first two images of black persons has the words white people in it, specifically "Opinion: white people know racism..." (haven't clicked to see rest) and "Why I'm no longer talking to White..." where the next word is people, it's a Guardian article so that explains high ranking I guess - what is your point?
Image search does not classify image contents. It uses the site text for ranking of the images on that site.
Do a google (not image) search for "white people" and you'll see that this phrase is mostly used in pages that are in fact about racism and therefore likely to contain images of black people.
It most certainly does, and has been for at least a year or two, or possibly even a bit longer (can't remember when I first noticed this behaviour).
You can e.g. do a search along the lines of site:<domain of online clothing store> <hair style/hair colour/…>, and at least for the most common and recognisable kinds of hair styles, it will actually return relatively reasonable image results, even though online shops most certainly don't have the habit of annotating the hair styles worn by their models on their product pages.
Along the same lines, Google is now also in the habit of OCRing any text content it can find in images and indexing that for search, too.
It's true that it'll still also take the text surrounding the image into account, but it's no longer true that image search is only based on that.
But slow/fast isn't quality. What were the rigorous measurements? Latency of results to click? User clicked above the fold?
I have wondered about this now for several years, because from my point of view Google search has steadily degraded since around 2010. The specific degradation isn't that it returns nothing, or irrelevant pages, but that it returns mostly _recent_ content, and returns very little _older_ content which is still assuredly on the web.
I can see how Google's approach will work for the average search query - after all, the average query is probably about something happening now - but that doesn't equate to _quality_.
> But slow/fast isn't quality. What were the rigorous measurements? Latency of results to click? User clicked above the fold?
Slow and fast is what we experience. We live in a phenomenological world, not a world of millisecond metrics. While it isn't rigorous, for humans, numbers are pointless. We can't experience them. What matters is whether it feels fast.
That isn’t quality. That’s like saying a crappy textbook full of errors that ships prime same day is better than a good textbook that takes a week to arrive.
I think just attributing this to the right people not being in charge over simplifies the problem.
Search is a complexity beast and simply continued to grow in complexity during the several years I worked directly on it. Folks were proud of the fact no one could even enumerate all the features in the system (attempts were made and abandoned).
The tools to change search safely werent keeping up with the complexity of the system. Understanding impact with evals and experiments became much harder. Gwsdiff and friends grew flakier. Debugging had so many different entry points depending on what you needed to do.
The search stack deserves some really deep cleanups and refactoring, the eval and devtools are similarly in need of a ton of love.
> the weekly meetings often featured "search quality" measurements that were rigorous in their objectivity (I thought).
I wish this part got discussed, but every time I've attempted it, the discussion has been shut down by "lol they're experts at search and you're not and you don't know what they know."
I wouldn't put it past Google to be blindsided thinking their own metrics are objective (perhaps they are objective measurements of something but not of what they actually want to measure). If anything, the battle with SEO just shows how hard it is to do something right and avoid getting gamed. If they can't rank SEO spam off the front page, why would I believe their measurements are any better than the rankings?
There's also always a small possibility that metrics are worse than wrong; they could actually say everything is fine, keep serving these long form SEO spam articles that people click and read for far too long before realizing it doesn't have have the answers they seek.
I believe the theory that Google is optimizing for ad revenue. Sites without ads get ranked lower. The biggest example I can think of is Wikipedia. When I search a proper noun with a Wikipedia article, I almost want to go look at that article. Recently, I feel like I really have to dig for it.
Anecdotal evidence - but I feel like this could be completely valid.
One of my biggest side projects for many years was a student tool centered around test scores. It was a niche use case with a huge amount of students using the tool on one day per year (1m+). There was exactly one competitor. I had a better domain but a much worse site in terms of design, speed etc. We were nearly the same in traffic, until I decided to monetize the site with a lot of Google ads. Immediately, Google shot my site up in the rankings, and actually seemed to penalize the competitive site. My traffic went up 10x and the competitive site remained flat. This happened for 3 years, then the niche use of both of our tools was “patched”.
Do you have an adblocker that's accidentally blocking the side-bar? I find it difficult to find a proper-noun search that doesn't include a side-bar link to Wikipedia.
This seems accurate to me too. And it also seems the most likely - a profit maximizing corporation doing profit maximizing things? Not really a huge surprise.
Yup. I've noticed recently that when I search for a public figure (whether it be a politican or businessperson or celebrity), their Wikipedia is now at the bottom of page 1, or not at all anymore. Never used to be the case 5 years ago.
Incidentally, I noticed very recently (in the last month or so) that searches in google maps (at least on mobile) are also starting to return completely irrelevant results. Something is seriously wrong at google.
The real question about G is this: who is really in charge there? Answer that, and the explanations for the whys of their behavior will fall into place. I'm not pretending to know the answer, but this question has helped me figure out what's going on with other companies and their strange directions taken.
This drives me nuts when I'm trying to navigate somewhere, no pun intended. For example, I'll tell it I want to go to a trailhead for a lake but it will switch it to the literal middle of the lake itself.
That also happened to me. I once want to drive to a port. And google decides to direct me to drive straight into the port. (The residents even make a sign that says 'If google direct you to here, you are on the wrong way', It seems it had gone wrong for many years and never got fixed.) Who on the earth think list port/lake/ocean into driving target is a good idea???
It started happening to me around November. I can understand if it shows me an establishment/location with a similar name but the results it gives have nothing in common with the location I'm searching for.
One theory is that Google has made a substantial change to a neural network based search, and they are still working out the kinks in getting it to work. How could it not be A/B tested such that we wouldn't notice the bad searches? The answer to that I am not sure. I read their research publications, and the NLP research coming out of Google is far beyond any other company. I can only imagine what they aren't publishing.
Honestly its pure speculation. But I could see scenarios where neural network driven search is great 90% of the time, and misses the last 10%, but in aggregate improves their revenue and product. Users on HN complain about the results, whereas normal searchers dont. The SEO spam works since it answers the question/fulfills whatever distance metric they are using for the search query.
Basic explanation of how one of these systems works:
(offline computation)
user query comes in -> algorithm runs across all the text on the page and computes the distance between the query and the meaning of the text (as represented by word embeddings called BERT).
Maybe the SEO spam has a good distance metric there, causing flaws, but the search works very well for a large number of users when trying to actually extract an answer to a search.
I work in NLP, but could easily be 100% wrong. References to my knowledge of their NLP is literally just reading all of googles most recent research
Interesting. I wonder how they might be approaching continuous learning.I.e. The UI has mechanisms for user feedback that might be useful in some fine tuning process
A/B testing with the decision criteria being search ad revenue will prefer lower quality search results ... since you have to search more (and see more ads) to get what you're looking for. :(
Google is not meaningfully ahead of other megacorps in NLP like facebook, and likely based on this thread and other starting to backslide.
Google may publish more than others, but only because if you write "we trained our models on 24 specialized TPUs for 1 month" than the reviewers instantly know you work for deep-mind despite "double blind anonymity" and thus you are much more likely to be accepted.
When I search “tim lee food blogger age” Google actually shows results with “age” striked out (so it shows top results as if age wasn’t part of the queried string).
Trying to think why/how it’d conclude that age wasn’t necessary for good results.
Censoring terms of the search vector to find relevant results is a legitimate approach that works well. In this example, the article author gives a Wikipedia page as the desired top result. That document does not include the term "age" anywhere. Therefore you can see how the term is not necessary to the search.
It works like trash when searching for obscure technical problems, e.g. debugging some little used tool in linux. It seems more often than not the most important word in the search is omitted, making the results completely irrelevant.
As a user, I can’t think of a time when censoring a term has given me relevant results. It just serves to frustrate me. If Google doesn’t want to return a blank page, they ought to have a small section at the top that says something like, “No results found for [search terms]” and then another section following that clearly states it is excluding a term.
FaceBook Marketplace and Craigslist do something similar when there are no local results. They show results from outside your area but clearly call it out.
While I agree this isn't technically "censorship" from one perspective, I do think it's important to consider this: by forcibly omitting search terms against your wishes, it is actively and deliberately preventing you from searching from what you wanted to search for. This is both frustrating for the user, and has a detrimental effect upon the quality of the search results.
When most people search, they're generally looking for information that matches a topic, not necessarily every single word in a query. You can imagine the number of misses that would happen if we didn't compensate for misspellings, for example. Or for synonyms or plural forms. But in no way are people prevented from searching for exactly the words indicated, if they feel that's best. Use Tools > All Results > Verbatim from the search toolbar. Or put quotes around the word or words you want exactly sought.
We wouldn't say the word is censored. It's just an indication we found a page tha that might be relevant even though it doesn't contain one or more of the search terms, and we want to help the user understand that. But yes overall, that's exactly why the approach can be useful, in case a page with the right information uses a slightly different related word. Also, that Wikipedia page is for a different Tim Lee -- a comedian, not the food vlogger.
It no longer does this because now your comment is the only result for the query. Before, it could not find a single result, said as much, and gave you the wikipedia page that included the age. What's wrong with that, again?
I realized after testing some more, that the query might not have had many relevant search results. It’s still a strange tactic. Age is contextually important in my query - Google didn’t recognize this
Query rewriting, a useless and use-hostile technique, strikes again! (No I am not talking about trying to fix spelling mistakes with "did you mean". I mean straight up removal or rewriting of correct English words)
I am embarrassed by the whole field of NLP for making such a big deal out of a task that is fundamentally bad.
Google optimizing for the "average user" comes at the expense of the whole world, because the "power users" who are optimized against literally build the internet for the normies. Cater to the normies for too long, and we see the status quo.
Force the normies to get better at writing their queries. NLP will unironically have a large number of people with the title "query engineer" soon anyway due to the rise of increasingly large foundation language models proliferating. Welcome to the brace new world of botched semantic search!
> Google optimizing for the "average user" comes at the expense of the whole world, because the "power users" who are optimized against literally build the internet for the normies. Cater to the normies for too long, and we see the status quo.
If I were feeling cynical, I would point out that catering to us was a great idea for early Google, but not so much now. The problem with power users is that the more control you give them, the more they learn about how your product works and what decisions you're making. Power users can pick out dark UI and other user-hostile patterns more easily.
Early Google needed the power users, because we were an essential component of building Google's early dominance: It was us gesturing the normies over and saying "as the computer person (TM) in your life, you should be using Google. They're cleaner and their results are better." We were needed to both scale their user base and plug gaps since at the time there weren't dozens of engineers working on issues and search + web indexing was still in its infancy.
Now, though? There's no benefit to Google in engaging with power users. They have the numbers of normies they need for profit, and all engaging power users would do now is lead to more conversations like this in the real world, which is not what they want.
"Oh, you're still using Google? They suck now. Use X, Y, and Z." = conversations Google doesn't want.
We cater to power users. If you absolutely, positively need a word or phrase to be on a page, use quotes. "age" or "tim lee" would tell us that exact word or that exact phrase has to be on the page. Which still doesn't help with this query, given that his age doesn't appear to be known, but you can do it for others.
Even quotes don't actually work. I've witnessed google query rewrite even with full quotes. I've also seen other posts on HN from others who've had this happen to them.
Quotes do work. They really, really do. Every time I look into a complaint about this, I find that the text really is on the page. It just might not be readily visible. Twitter link is in my bio here. If you ever get an example where you find quotes don't seem to be working, do pass it on.
We show this to indicate to you that we found a page that might be relevant but which doesn't contain that exact word. And why we would conclude that is explained better if you search for ["tim lee" food vlogger "age"].
That's telling us to find only pages that have "tim lee" on it in that order, as well as the word "age" and food vlogger are nice to have but not required. And it turns out -- there's not a lot of pages. Certainly not many pages, it seems, about the Tim Lee you're looking for with his age. And that's because not everything we want is actually out there. There might not be a page that has his age.
So back to why we dropped it. By showing you some pages that don't have the word age on it, we're able to show some other pages that are generally about him, which might get you closer to the answer.
BTW, there is a Wikipedia page I've seen suggested as having the right answer for his age. But that's for a different Tim Lee -- not the food vlogger, and it also only lists his birth year, so knowing the exact age is hard
I think there's more quality content on the web now than there's ever been.
It's seems like Google wants to prioritize the commercialized web, though, by boosting BuzzFeed-like online publications that post ad-laden articles full of referral links in their search results, along with social media. Even in the development space, Google will prioritize w3schools and content farms that constantly churn out how to guides. Blogs and forums seem to get deprioritized in search results these days.
I just tried to find this article on google by searching "Can cats eat blueberries bad search results" (Can Cats eat blueberries is very relevant to the article) and it showed up on the first page of results, albeit at the end.
I'm very confused by this article. They seem to just assume that breaking up Google would fix the problem, but they don't explain why. To me it seems that a fundamental problem of search on the internet is that it's adversarial, which means Google needs to make these ranking changes to try to make things better in the SEO war. How does breaking Google up fix this? Wouldn't all the new search engines just be worse at fighting SEO spam, leading to even more spam?
disclaimer: i work at Google, though not on search.
Hah, I love that DDG, Bing and Google all return some article about cats being able to eat blueberries, while Kagi delivers your article at the first position :D
I tried the query "databricks series b valuation" on Kagi (just a beta user there) and the results were:
1. Databricks Funding Rounds, Valuation and Investors (https://craft.co/databricks/funding-rounds) - not directly to the point but does include information about all rounds.
2. Databricks Raises $1B at $28B Valuation, Plans Massive - not answering the question at all.
3. *Databricks Closes $33M Series B Funding - FinSMEs* (https://www.finsmes.com/2014/07/databricks-closes-33m-series-b-funding.html) - Direct hit! didn't even have to click into the page.
In my mind this is yet another proof of Google's search quality decline. I remember being so excited when I saw the first structured search result but now I tend to use other engines first.
I think that Google is optimizing for the "average user" to the detriment of power users such as the HN crowd. Most people treat Google as an internet oracle and send queries like "how do I do X" while power users will search for keywords. One example of this optimization is the automatic answer boxes that show up for certain questions, which are wrong disturbingly often or don't include important details.
I miss Altavista. I could generally either find exactly what I was looking for inside of three iterations of refining the search, or find that it wasn't to be found.
I still just want a blazing fast full text search of the reachable WWW that understands regexes and a basic predicate calculus. Unfortunately the overhead and small potential user base means that under the current regime such a thing will never be made.
Speaking of, if any government actually wants competition, they don't need to break up Google, they just need to force them to offer full access to their cache and compute at some reasonable rate, much like how the ILECs were made to carry the CLECs' traffic.
I think this is overall a good thing. Power users have trained their behaviour to work in the way that simple systems can deal with. While average users ask the question exactly how they would ask another human. Google has now reached a level where it works best when you deal with it in a natural and human level.
There is nothing actually better about the way we originally used search engines, it was just required at the time.
No, I disagree. With keyword search, I am confident that eventually, with enough included and excluded terms, I will find what I’m looking for.
With natural language search, sometimes it works great, but it’s a crapshoot and when you don’t get the results you need you’re stuck.
Several times in recent memory Google has returned results so bad, I completely gave up searching. Most recently was when i was trying to look up a Windows 11 BSOD error code (where even pasting the error code verbatim only brought up pages of garbage sites with no useful technical information).
Google results for Windows error codes have been gamed to hell and back by the likes of Easus and Drivereasy, where their tools are somehow the answer to every single fault Windows could possibly manifest.
For some queries, being able to ask off the top of your head without thinking is good. Think 'what day of the week was June 14th, 2002?' or 'Who is the mayor of Los Angeles?' For quick questions with clear answers, the current system is a huge advantage over what we had before.
For other, more complicated queries, the act of composing your search and considering your keywords, etc. is a step in the process that helps a searcher mentally understand the results they're going to receive along with what they mean. Having to stop and consider makes you aware that you're working within a system and its constraints, which makes it more suitable for questions that are complex or not socially settled.
The average use absolutely does this. I see this with family members and friends. Most just type in full questions.
From my experience the best way to get good results is to start typing keywords for yourr question and then creating the query based on the autocomplete results. If I notice I don't get autocompletion for a certain query I'll restructure it until I do. This has proven very useful in providing good results.
For technical stuff, using the quotation marks is almost essential.
This. And let's not forget that is the "average user" that mostly naively clicks on ads, not the power user. And selling ads is still Google core business.
Looks like Google is slowly turning into a big nigerian scam.
The article mostly talks about IA (instant answers) which are notoriously hard. The recent advances in machine learning have made the technology more approachable, so startups like Kagi Search (disclaimer: founder) can also leverage latest advances in NLP and compete on this ground.
Both engines use the same article for source, but Google completely misses the context.
These examples show that a search startup has a chance to go neck-to-neck with Google and compete even in technology as sophisticated as instant answers. We invested considerable resources in the Kagi Search AI capabilities, discussed in some detail here https://kagi.ai/last-mile-for-web-search.html
What is mind boggling though from a product management perspective is that Google had nearly a decade head start and a cash purse of hundreds of billions of dollars to get this right.
To be fair, it is likely that the vast majority of queries are answered correctly, but only the outliers get the public attention. Also Kagi is not without its own share of silly mistakes too, but just being able to be considered in the same basket as Google is already a huge thing for us.
My favorite is, given that I have a baby and I am a trained scientist and I live in the US, I find myself converting milliliters to fluid ounces a lot.
Right now the Google Assistant will correctly transcribe the request to a Google search... Only for the search to interpret “ml” as “miles” and, faced with the discrepancy between the length and a volume, cube the miles. So I am expecting an amount that is like 1 oz because I am converting like 30 mL... and I instead get 4 quadrillion ounces (exact number is 1 mi³ = 140,942,994,870,857 + 1/7 oz because of course it's got that extra 7th in there what were you expecting from our ridiculous US system, haha).
> The article mostly talks about IA (instant answers)
Not really. TFA is discussing the search results. If google/bing want to put instant answers then that is what will get judged. If your ML/AI is not good enough yet to provide natural language answers, don't make it the most prominent part of the search results.
> No one at Google is responsible for these half baked and largely irrelevant widgets or wants to stake their career fixing them.
You're just ... wrong about this. There's an entire team of dozens of people (maybe hundreds now) focusing on this specific web answer feature. I personally worked on the team (not this feature, though).
I don't understand why people say things they know nothing about.
I think people say this because some problems stick out like a sore thumb and stay that way for months if not years. The line of inference is that there must be no incentive inside the company to fix them.
Isn't this just the software engineering equivalent of the fundamental attribution problem? My backlog is long because I have important stuff(TM) to address, whereas their backlog is sign of dysfunction.
I think it largely stems from this very popular comment from a couple of years ago: https://news.ycombinator.com/item?id=19553840 From an external point of view certainly seems to explain a lot of Google's issues with the stagnation and death of many of it's products.
The fact that there have been plenty of other comments from Googlers to back it up since shows that, in some parts of Google at least, there's a grain of truth there. It might not be the whole story, many teams might be proud of their part, and many Googlers may not be focusing on promition and shiny new things, but that doesn't really matter to Google's users. What we see has been plenty of Google products being killed off, or left to rot. Now even Search has people complaining about it. Startups are getting traction competing against it. A decade ago that would have been unthinkable.
If you're right and teams in Google do actually care about the older, less shiny things they build then Google has a significant brand and reputation problem. If you're wrong then Google has a massive engineering culture problem. Either way, Google has a problem.
> "At the bottom of every question page we provide a link to answers-support@google.com. We encourage you to use this link whenever you see questionable content posted to the site. In your email, please provide information about the question, its ID number, and the reason you find the content questionable."
And yet despite all of those people, the quality is bad and seems to get worse. A startup seems to be outcompeting them.
Sometimes industries or teams have effectively negative value, or their preconceived notions about how something works based on teachings from their field is wrong. This is the case for chiropractors (the whole field is useless / a net negative), vs back doctors.
We see this happening today with Google moving away from traditional and high quality techniques like direct keyword ranking, bm25, and pagerank and move towards lower quality methods based on hype such as BERT/other "semantic search" based on LMs, query rewriting, and using these dense vectors directly in pagerank (and this degrading it).
The amazing power of language models in certain domains (text generation) has unfortunately caused a proliferation of them in a place where they are still pretty bad (information retrieval and search).
Google is full of search chiropractors when they need search back doctors.
If you search for anything political on Google, you'll notice that the results are clearly slanted in one direction, towards the opinion of a handful of pre-approved news outlets. This leads me to seek alternatives whenever I need neutral sources, for instance Yahoo search.
I wonder if they're getting cash kickbacks from established corporate media outlets for pushing their material to the top of the search results. That would actually be less creepy than if its being done as some kind of information manipulation program.
It's high time Google and other search engines were forced to expose the inner workings of their ranking algorithms to the public, particularly now that they have near-monopoly power in the sector. People should also be able to adjust the dials on the algorithm themselves.
In Australia, Google is being blackmailed into boosting Fairfax, Newscorp and Seven West Media content higher up in their index. It's reached a point where most queries are useless if they contain a word or even a synonym for that word that has been recently used in a major media site owned by those three.
I use Google through a VPN to avoid it. That breaks maps integration.
I'm curious, why go through the hassle of a VPN rather than another search engine? Are the other results so poor or do the others do the same as Google?
Non-factual answers are especially interesting, because otherwise reasonable and intelligent people believe in them. Everyone would be better off studying those things and see how they've drawn those conclusions we don't subscribe to, so that we can figure out what we are wrong about.
It's absolutely catastrophic if you are not allowed to draw your own conclusions about things. This is your inalienable prerogative as an adult in a free country. Even at the risk of some people being wrong sometimes, you simply cannot have authorities distributing doctrine and call yourself a democracy.
As a corollary, search on Google News (as in, browsing to news.google.com and searching there, or !gn via DuckDuckGo) is really bad. The index seems to update really slowly, so breaking events are usually missing entirely, and the grouping of articles into single events is also quite broken.
Try "what countries are using ivermectin" in google.com and then try Yandex.com. For me, the third site on Google (the kitchen sisters) appears broken and the rest are all some variation of "why ivermectin is bad" articles. Yandex actually answers the question.
I find these responses fascinating as the "clearly slanted" results tend to change direction depending on the political affiliation of the person making the claim! Having said that, I'd love to be proved wrong if you have any evidence to show a particular bias one way or the other?
Search "mass formation psychosis" on Google and DuckDuckGo. This is a trending phrase popularized by a doctor that has been canceled and banned from the mainstream due to his criticism of the world's COVID response.
DDG shows the author's substack as #1 result and is neutral otherwise. The other doesn't even have it on the first SERP, and is overwhelmingly critical.
If you argue that covid response is not politics, I will disagrer strongly.
Search politically controversial topics incognito with a VPN on using google vs yandex vs duckduckgo? Ivermectin, January 6th arrests, BLM protest deaths, VAED, mRNA studies before 2020, Robert Malone, Geert Vanden Bossche. I mean there's an endless list of things you can experiment with.
My experience is that it's slanted whichever side of the easel you're one. The evidence is really clear when you measure results between various search providers, and especially when searching up contentious or controversial topics. So it doesn't really change direction so much as it confirms one particular set of beliefs depending on which search engine you're on. I think this is clearly in Google's disfavour, because people have started to notice and they're actively searching for alternatives to Google in order to avoid it.
I think it is related to the fight against "fake news", "hate speech", etc... People don't tolerate a truly neutral search engine, because it will reflect human nature and human nature is not always pretty. I remember the time when Google returned antisemitic websites when searching for "jew", they refused to do anything about it because "jew" is used mostly by antisemites and therefore, an antisemitic website is what people searching for that term most likely want, the search engine did its job. I don't think it will fly today.
So search engines now have to get the "truth", preferably the politically correct one, and since you can't rely on the crowd for that, you have to introduce bias, and "pre-approved news outlets" are the most obvious choice.
I share the view that Google SERPS have dropped in quality the last 5-7 years. Of great annoyance to me is the amateurish way that a search results page will find relevant Twitter results but then clicking on the results takes you to the root page of the Twitter user and not the result. Since many Twitterers are prolific posters, it can be very time-consuming or even impossible to find the result listed. Thankfully Inoreader takes me to the exact Twitter result.
Instant answers (IA) caused a shift in the way contents are written. Content optimized for IA tend to be repetitive and shallow. Viewing content written for IA is a frustrating experience and these tend to dominant the result page now.
No idea if you're right or not, but I think it's an interesting take. It might be because of SEO, it might be more information in walled gardens, it might be the death of the personal web page.
Both Google search and the internet are changing, as are our perceptions, so it's really hard to say why search quality is better or worse.
Whether search quality is deteriorating really depends what we expect from a search engine. Many of us on HN use search engine as a way to look for web pages with specific words. Search engines did start out performing word searches, but somewhere along the way, they started trying to answer questions. Many of the complaints such as Google dropping search terms are actually the result of the search engine trying to answer questions instead of performing keyword searches. Search engines these days are doing well for the easy questions, but of course they completely misunderstand questions with more nuances.
A bit off topic: Some of the search queries in the article used Google like a keyword matching tool, while other queries in the article were using Google to answer questions. That's because we only have one search box trying to do everything. Would we be better served if we have a checkbox to tell the search engine that we just want to perform word searches?
The reason you are seeing a lot of that sort of content is because Google is looking for that sort of content, in point because of Google's peculiarities, but also because of how refined the art of black hat SEO has become.
Meaningful websites still exist. The bulk of the content on the Internet is older than IA, and it's still out there (not that you can find it with Google).
Hey, free advertising for you and your search engine:
Marginalia search, by punishing ad- and tracking-heavy pages and by being strict in how it interprets my queries sometimes surfaces better results than Google and DDG.
In particular I have found great resources about Linux partitioning and git usage after giving up mainstream search engines and trying them in marginalia.
That says quite something about how badly broken the situation is given that marginalia is one person and a tower pc in a living room.
I keep getting reminded about a Linux quote on how they managed to go forward by studying the latest 20 years of OS research and throwing it all away :-)
432 comments
[ 5.0 ms ] story [ 335 ms ] threadI used to work on Search and Search Measurement at YouTube, Twitter, and Microsoft, so I thought it would be fun to move beyond anecdotes, grab some data, and do a quantitative analysis.
tl;dr I didn't have historical data, to see how Google Search has trended over time. But compared to Bing, Google still generally outperforms -- although some of its failures are pretty surprising!
If there are particular areas of Google Search that people are interested in digging into, give a shout -- I love running these kinds of search / human eval analyses.
[1] https://news.ycombinator.com/item?id=29772136 [2] https://news.ycombinator.com/item?id=29417061 [3] https://news.ycombinator.com/item?id=29392702
Content that would be have been on an open web forum 10 years ago, are now hidden behind various data silos/walled gardens - whether it's Reddit, YouTube, TikTok, Instagram, Twitter or Discord. Each of these walled gardens have different levels of tolerance for the open web, from using it as an SEO channel (e.g. Reddit) to completely opaque to it (e.g Discord).
Some car and photography forums I was on ~10 years ago are all barren now as the old users have moved on and new users prefer to communicate in a Facebook group or something like that.
Interesting self-published blogs and recipes aren't on the web anymore, they're on YouTube channels or on someone's Instagram channel (or whatever its called).
what are some of the decisions that go into this ? is it just cost savings ?
Getting this wrong is probably why people think we need more than definitive assertions from people operating from subjective impression.
Changing a letter in the query, it says, right at the top:
I rarely see "No results found".
That is a completely valid result in my opinion.
I often see pages that doesn't contain my search string.
Sometimes is is something trivial like
query: "something about x y and z"
result contains: "... something about x. Y and Z however..."
which is understandable.
Often though I have no clue how a result got there. Maybe it is linked to using a certain text (link bombing)? I don't know.
But no, when I search for realsense "failed to recconect" Google returns pages that contain neither realsense nor recconect [2]. They offer me a supreme court opinion, a review of a car dealership, and a facebook church service.
Correcting the spelling of a query is one thing - but also completely ignoring other keywords? I can see why there are so many people posting about the poor quality of Google's search results.
[1] https://github.com/IntelRealSense/librealsense/blob/5ff27fca... [2] https://imgur.com/a/okYV5V2
But the issue isn't that they can't; the issue is that they don't want that. Why the sites with copied content exists? To earn money through ads. What earns Google money? Ads!
And any simple heuristic is quickly reverse engineered by SEOs, who will find a way to mask it as legitimate.
tl;dr it's a hard problem.
As I have said, the reason they don't do it is not because they don't have the skills and know-how.
Perhaps that's my biased opinion on their motivations as I've recently launched https://grizzlybulls.com and yet even though Bing has tiny market share, I'm getting 10x more organic traffic from Bing rather than Google...
"Python get value from database - Büro Jorge Schmidt", which judging by title and preview seems to be a Python + MySQL tutorial. It returns a 403 error and might be a hacked site, since the home page is for a graphic design studio in Munich.
Result #8 is something similar:
"Intellij flatten packages - Músicos de Viaje". This is definitely a hacked site (from Spain, apparently) that redirects me somewhere else.
Result #10:
"How to calculate tax percentage in sql query". Another hacked site, this time for an evangelical church from Brazil.
Now... how can Google think that any of these sites are relevant? Even if it doesn't realize the pages are hacked... even its crawler has been fed content that included the keywords... :
A - The sites themselves don't match the query at all.
B - No legit site about the subject would link to these sites.
C - The results themselves (title, url, preview), as Google shows them, have nothing to do with the search!
I wonder if you have some malware that is hijacking the results? I once had some malware (chrome extension) that was corrupting my search results. It was surprisingly difficult to remove (given that it was a chrome extension...).
Google results are personalized, based on location, search history, etc. The fact that I'm in Argentina has been adding a lot of noise to results on searches where my location is not relevant at all.
In this case, I suspect that Google thinks these hacked sites with developer target content are relevant to me, because of my regular search history.
This seems to be a common theme in the industry. Recommenders heavily overweight location - an incredibly general factor, even in the presence of troves of specific, individual level data. Goes to show how little basic reasoning really goes into how these systems work.
Localized search is useful at times (restaurants for example, I like getting the local McDonald’s or China Wok rather than the biggest one in New York) but it’s completely useless for many terms. But maybe not the ones Google makes the most money on.
Copied from another answer:
---
My feeling is that since that query is SO unusual for me, based on my search history (I can't even say what it means) it raises the "likelyhood" that the hacked spam sites with programming terms that also include those keywords are good results for me.
If I search for something more typical, like "Spider-Man No Way Home" or "Ruby rails tutorial" the results don't include hacked sites.
---
P.S. Chrome, incognito window.
My feeling is that since that query is SO unusual for me, based on my search history (I can't even say what it means) it raises the "likelyhood" that the hacked spam sites with programming terms that also include those keywords are good results for me.
If I search for something more typical, like "Spider-Man No Way Home" or "Ruby rails tutorial" the results don't include hacked sites.
They're so obviously unrelated hacked sites that they shouldn't even be listed.
Do you geolocate in Argentina?
Google has been so much better for search for me than other search engines. Atleast for what I search for programming, news etc.
Also, results are personalized. Not everyone sees the same.
If I do an incognito search (not logged in) I get results without trash.
That being said, I have issued queries in the past that I have just found absolute walls of malicious results. I haven't really invested much attention span in entertaining this sort of thing so I just move on and modify my query, but I'll keep an eye out for it going forward.
- I checked the cached (by Google) copies of the hacked pages and they include mentions of a "Databricks SQL Connector". So if I search for "Databricks" Google thinks "it must be a programming thing".
- If I now search for "databricks series a valuation" I don't get the spam results, for some reason. I think that if I repeat the search Google produces the exact same results... but internally, since I first searched, it might have realized that those sites were not good.
I distinctly remember Udi Manber saying "if the web is slow, it's our fault" (actually, the speech was that everything is "our fault"), meaning, really, "take responsibility for problems and don't throw up your hands."
However, the natural tendency of any organization is to reward the suckups and promote mediocre people who just get along with everyone. It wouldn't surprise me if that's what's happened with Google, too.
What? Why not, and anyway Amazon I'm giving money. I'd think that quality there would be more deeply linked than anywhere I'm just giving some time.
It only sounds like what they do because you only ever read stuff written by people who believe that without any evidence.
Sundar Pichai has been with Google since 2004.
But speaking in general, there is limited correlation between executive compensation and company performance. You can DuckDuckGo it if you want to find the evidence for this statement.
Having everyone you've worked with love you and find you brilliant is not at all the same thing as being brilliant.
Google Search has 86% market share in the US.
GM tried in the 80s
Is anybody claiming that it’s sudden? Everybody I’ve seen complain about Google search results has seemed to think that it’s been getting slowly worse over a long period of time.
Nowadays all those tweaks to the algorithms, specially search and gmail spam filters, are like a desperate attempt to make the ship move faster by removing parts of the hull while keeping the water out. It will just sink at some point.
I was recently searching for a piece of ceramic cookware, specifically looking to avoid non stick coatings. And google search showed me lots of listings with my exact search terms, but when I click the page it shows their generic product range which has nothing to do with ceramic despite the title saying so.
right, I mean we have nearly daily posts here about almost human level quality texts you can generate with various machine learning solutions.
Add to that sites who serve users needs must focus on doing that, the sites that just game Google search rankings can focus on that.
But maybe at some point there will be a breakthrough with SEO abusers being able to be as constructive as the actually useful sites, at which point there will be a short time when the SEO abusers are the only sites returning results, causing all other sites to go out of business, causing the downfall of the SEO sites which rely on actually useful sites for a great deal of their stolen content / content seeding corpora.
Vast majority of these SEO abusers are monetized through google ads. For every advertisement dollar spent through google, google takes 30-50%. And 80% of Google's income is from internet ads.
I would be surprised if google's search quality would not deteriorate - they have very strong incentive for that.
Searching today is not as bad as it was during the Altavista/Netscape/56k modem era, but it is getting dangerously close, from my personal and anecdotal experience.
I mean, there are two ex-Googlers in this thread claiming firsthand that search quality was considered sacrosanct as of a few years ago, so it seems like it'd have to have been sudden if indeed it has happened at all.
This was roughly when Google's new AI began "interpreting" the "meaning" of the specific jargon and product serial numbers I was inputting, and then decided what I actually needed was song lyrics.
> decided what I actually needed was song lyrics
Or random gossip about media celebs. I'm searching for SCSI cable stuff and getting Kim Kardashian (whoever the #@$%@ that is).
The process might have started before, or indeed after I left. And I wasn't in Search, anyway.
After all, if me, my team and my boss are rewarded when an imperfect measure of search quality goes up and it's going up, why would I fight to switch to a different measure that wasn't rising?
For example, my first week at Google/YouTube, I was in a New Hire meeting with our VP. Someone asked about profitability, and he responded that Larry said we didn't have to worry about revenue yet, since the main goal was user growth/happiness, and revenue could come later. Which I thought was fascinating, considering how big YouTube already was at the time (in 2013)!
Though I think this changed a year later, and I find YouTube ads a poor experience compared to Instagram and TikTok -- which aren't merely "better than the rest", but stuff I actually enjoy watching.
The company should be smart enough to know I only buy certain brands of dog food. It should know I need a gift for my mum (and what she likes). It knows I need new trainers but will only buy the cheapest. Etc.
The opportunity for this has pretty much closed with the era of data privacy coming in, but I think it is both surprising and a shame that this didn't happen in the last decade.
To someone else's point: yes, they are restricted by who bid for the keywords, but honestly, that's the whole magic of Google Ads. You have a "motivated buyer", someone who actually wants to send flowers or stay in Duluth. How do you know that? They searched for "flowers" or "duluth hotels", duh.
Is an advertiser willing to throw money at people who probably want to send flowers, based on their past behavior? Well, maybe, but not on the Search Results page. There are lots of other web venues where they can place creepy ads like that.
I can't square that change with the claimed commitment to search engine quality.
Now repeat the search but with "black people" without quotes.
Try the same for "white couple" without quotes.
Try the same for "black couple" without quotes.
Do you have any insight to this phenomenon?
Don’t disagree with the main theory that search quality is deteriorating. I have to use increasingly contrived queries to get anything but bullshit blog spam, and indexing seems really odd at times.
white people -> images of mostly blacks
black people -> images of mostly blacks
white couple -> mostly white couples with a decent percentage of mixed ethnicity couples
black couple -> black couples
on edit: grammatical correction
You can e.g. do a search along the lines of site:<domain of online clothing store> <hair style/hair colour/…>, and at least for the most common and recognisable kinds of hair styles, it will actually return relatively reasonable image results, even though online shops most certainly don't have the habit of annotating the hair styles worn by their models on their product pages.
Along the same lines, Google is now also in the habit of OCRing any text content it can find in images and indexing that for search, too.
It's true that it'll still also take the text surrounding the image into account, but it's no longer true that image search is only based on that.
/s
I have wondered about this now for several years, because from my point of view Google search has steadily degraded since around 2010. The specific degradation isn't that it returns nothing, or irrelevant pages, but that it returns mostly _recent_ content, and returns very little _older_ content which is still assuredly on the web.
I can see how Google's approach will work for the average search query - after all, the average query is probably about something happening now - but that doesn't equate to _quality_.
Slow and fast is what we experience. We live in a phenomenological world, not a world of millisecond metrics. While it isn't rigorous, for humans, numbers are pointless. We can't experience them. What matters is whether it feels fast.
Search is a complexity beast and simply continued to grow in complexity during the several years I worked directly on it. Folks were proud of the fact no one could even enumerate all the features in the system (attempts were made and abandoned).
The tools to change search safely werent keeping up with the complexity of the system. Understanding impact with evals and experiments became much harder. Gwsdiff and friends grew flakier. Debugging had so many different entry points depending on what you needed to do.
The search stack deserves some really deep cleanups and refactoring, the eval and devtools are similarly in need of a ton of love.
I wish this part got discussed, but every time I've attempted it, the discussion has been shut down by "lol they're experts at search and you're not and you don't know what they know."
I wouldn't put it past Google to be blindsided thinking their own metrics are objective (perhaps they are objective measurements of something but not of what they actually want to measure). If anything, the battle with SEO just shows how hard it is to do something right and avoid getting gamed. If they can't rank SEO spam off the front page, why would I believe their measurements are any better than the rankings?
There's also always a small possibility that metrics are worse than wrong; they could actually say everything is fine, keep serving these long form SEO spam articles that people click and read for far too long before realizing it doesn't have have the answers they seek.
One of my biggest side projects for many years was a student tool centered around test scores. It was a niche use case with a huge amount of students using the tool on one day per year (1m+). There was exactly one competitor. I had a better domain but a much worse site in terms of design, speed etc. We were nearly the same in traffic, until I decided to monetize the site with a lot of Google ads. Immediately, Google shot my site up in the rankings, and actually seemed to penalize the competitive site. My traffic went up 10x and the competitive site remained flat. This happened for 3 years, then the niche use of both of our tools was “patched”.
https://en.wikipedia.org/w/index.php?search=%s&title=Special...
It's baffling.
Basic explanation of how one of these systems works:
(offline computation) user query comes in -> algorithm runs across all the text on the page and computes the distance between the query and the meaning of the text (as represented by word embeddings called BERT).
Maybe the SEO spam has a good distance metric there, causing flaws, but the search works very well for a large number of users when trying to actually extract an answer to a search.
I work in NLP, but could easily be 100% wrong. References to my knowledge of their NLP is literally just reading all of googles most recent research
But the quality issues I've seen (useless results unless you add "reddit") are a lot newer than that.
Google may publish more than others, but only because if you write "we trained our models on 24 specialized TPUs for 1 month" than the reviewers instantly know you work for deep-mind despite "double blind anonymity" and thus you are much more likely to be accepted.
Pedigree is not the same as quality.
Trying to think why/how it’d conclude that age wasn’t necessary for good results.
FaceBook Marketplace and Craigslist do something similar when there are no local results. They show results from outside your area but clearly call it out.
Also, at least for me, Google does exactly what you describe at the right approach.
No doubt that’s bad for some opaque internal metric though.
I am embarrassed by the whole field of NLP for making such a big deal out of a task that is fundamentally bad.
Google optimizing for the "average user" comes at the expense of the whole world, because the "power users" who are optimized against literally build the internet for the normies. Cater to the normies for too long, and we see the status quo.
Force the normies to get better at writing their queries. NLP will unironically have a large number of people with the title "query engineer" soon anyway due to the rise of increasingly large foundation language models proliferating. Welcome to the brace new world of botched semantic search!
If I were feeling cynical, I would point out that catering to us was a great idea for early Google, but not so much now. The problem with power users is that the more control you give them, the more they learn about how your product works and what decisions you're making. Power users can pick out dark UI and other user-hostile patterns more easily.
Early Google needed the power users, because we were an essential component of building Google's early dominance: It was us gesturing the normies over and saying "as the computer person (TM) in your life, you should be using Google. They're cleaner and their results are better." We were needed to both scale their user base and plug gaps since at the time there weren't dozens of engineers working on issues and search + web indexing was still in its infancy.
Now, though? There's no benefit to Google in engaging with power users. They have the numbers of normies they need for profit, and all engaging power users would do now is lead to more conversations like this in the real world, which is not what they want.
"Oh, you're still using Google? They suck now. Use X, Y, and Z." = conversations Google doesn't want.
That's telling us to find only pages that have "tim lee" on it in that order, as well as the word "age" and food vlogger are nice to have but not required. And it turns out -- there's not a lot of pages. Certainly not many pages, it seems, about the Tim Lee you're looking for with his age. And that's because not everything we want is actually out there. There might not be a page that has his age.
So back to why we dropped it. By showing you some pages that don't have the word age on it, we're able to show some other pages that are generally about him, which might get you closer to the answer.
BTW, there is a Wikipedia page I've seen suggested as having the right answer for his age. But that's for a different Tim Lee -- not the food vlogger, and it also only lists his birth year, so knowing the exact age is hard
It's seems like Google wants to prioritize the commercialized web, though, by boosting BuzzFeed-like online publications that post ad-laden articles full of referral links in their search results, along with social media. Even in the development space, Google will prioritize w3schools and content farms that constantly churn out how to guides. Blogs and forums seem to get deprioritized in search results these days.
I just tried to find this article on google by searching "Can cats eat blueberries bad search results" (Can Cats eat blueberries is very relevant to the article) and it showed up on the first page of results, albeit at the end.
On Bing I wasn't able to find it
disclaimer: i work at Google, though not on search.
I still just want a blazing fast full text search of the reachable WWW that understands regexes and a basic predicate calculus. Unfortunately the overhead and small potential user base means that under the current regime such a thing will never be made.
Speaking of, if any government actually wants competition, they don't need to break up Google, they just need to force them to offer full access to their cache and compute at some reasonable rate, much like how the ILECs were made to carry the CLECs' traffic.
There is nothing actually better about the way we originally used search engines, it was just required at the time.
With natural language search, sometimes it works great, but it’s a crapshoot and when you don’t get the results you need you’re stuck.
Several times in recent memory Google has returned results so bad, I completely gave up searching. Most recently was when i was trying to look up a Windows 11 BSOD error code (where even pasting the error code verbatim only brought up pages of garbage sites with no useful technical information).
P: "Pfft. So, what? You think the current state of search is good?! Having to type in keywords instead of just asking a question in normal language?"
Me: "... is this a trick question?"
For some queries, being able to ask off the top of your head without thinking is good. Think 'what day of the week was June 14th, 2002?' or 'Who is the mayor of Los Angeles?' For quick questions with clear answers, the current system is a huge advantage over what we had before.
For other, more complicated queries, the act of composing your search and considering your keywords, etc. is a step in the process that helps a searcher mentally understand the results they're going to receive along with what they mean. Having to stop and consider makes you aware that you're working within a system and its constraints, which makes it more suitable for questions that are complex or not socially settled.
Not all information is the same.
From my experience the best way to get good results is to start typing keywords for yourr question and then creating the query based on the autocomplete results. If I notice I don't get autocompletion for a certain query I'll restructure it until I do. This has proven very useful in providing good results.
For technical stuff, using the quotation marks is almost essential.
Looks like Google is slowly turning into a big nigerian scam.
To give just a few examples:
Query 1: how many stars in the usa flag
Google: https://cln.sh/63sVzh
Kagi: https://cln.sh/bFEHsD
Pretty surprising that Google would get something like this wrong.
Query 2: when did moon explode
Google https://cln.sh/fUhdJS
Kagi https://cln.sh/5wDvXG
Both engines feature the same article but for some reason Google decides this is not fiction, and gives a (wrong) answer.
Query 3: do most rabbits have short or long ears
Google: https://cln.sh/JuOeqq
Kagi: https://cln.sh/BkZi6O
Both engines use the same article for source, but Google completely misses the context.
These examples show that a search startup has a chance to go neck-to-neck with Google and compete even in technology as sophisticated as instant answers. We invested considerable resources in the Kagi Search AI capabilities, discussed in some detail here https://kagi.ai/last-mile-for-web-search.html
What is mind boggling though from a product management perspective is that Google had nearly a decade head start and a cash purse of hundreds of billions of dollars to get this right.
To be fair, it is likely that the vast majority of queries are answered correctly, but only the outliers get the public attention. Also Kagi is not without its own share of silly mistakes too, but just being able to be considered in the same basket as Google is already a huge thing for us.
Right now the Google Assistant will correctly transcribe the request to a Google search... Only for the search to interpret “ml” as “miles” and, faced with the discrepancy between the length and a volume, cube the miles. So I am expecting an amount that is like 1 oz because I am converting like 30 mL... and I instead get 4 quadrillion ounces (exact number is 1 mi³ = 140,942,994,870,857 + 1/7 oz because of course it's got that extra 7th in there what were you expecting from our ridiculous US system, haha).
Perhaps everyone who understood the codebase has left?
Clearly they should have put a chat app in it.
Sometimes even without the !wa, as duckduckgo itself often provides the result
Not really. TFA is discussing the search results. If google/bing want to put instant answers then that is what will get judged. If your ML/AI is not good enough yet to provide natural language answers, don't make it the most prominent part of the search results.
The third query is wrong for me too, though.
No one at Google is responsible for these half baked and largely irrelevant widgets or wants to stake their career fixing them.
You're just ... wrong about this. There's an entire team of dozens of people (maybe hundreds now) focusing on this specific web answer feature. I personally worked on the team (not this feature, though).
I don't understand why people say things they know nothing about.
The fact that there have been plenty of other comments from Googlers to back it up since shows that, in some parts of Google at least, there's a grain of truth there. It might not be the whole story, many teams might be proud of their part, and many Googlers may not be focusing on promition and shiny new things, but that doesn't really matter to Google's users. What we see has been plenty of Google products being killed off, or left to rot. Now even Search has people complaining about it. Startups are getting traction competing against it. A decade ago that would have been unthinkable.
If you're right and teams in Google do actually care about the older, less shiny things they build then Google has a significant brand and reputation problem. If you're wrong then Google has a massive engineering culture problem. Either way, Google has a problem.
Well, in all honesty I wonder how this exact same thing when reading the "answers".
I'm having visuals of someone repeatedly trying to throw something and missing the wall entirely.
Lets examine the original product implemented in wetware:
http://answers.google.com/answers/index.html
> "At the bottom of every question page we provide a link to answers-support@google.com. We encourage you to use this link whenever you see questionable content posted to the site. In your email, please provide information about the question, its ID number, and the reason you find the content questionable."
This one went through the wall!
You have no further questions.
Sometimes industries or teams have effectively negative value, or their preconceived notions about how something works based on teachings from their field is wrong. This is the case for chiropractors (the whole field is useless / a net negative), vs back doctors.
We see this happening today with Google moving away from traditional and high quality techniques like direct keyword ranking, bm25, and pagerank and move towards lower quality methods based on hype such as BERT/other "semantic search" based on LMs, query rewriting, and using these dense vectors directly in pagerank (and this degrading it).
The amazing power of language models in certain domains (text generation) has unfortunately caused a proliferation of them in a place where they are still pretty bad (information retrieval and search).
Google is full of search chiropractors when they need search back doctors.
I'll happily say bye bye to instant answers across all search engines just to get my working search engine back (preferably with a blacklist).
Same results as you on the third one though.
It's high time Google and other search engines were forced to expose the inner workings of their ranking algorithms to the public, particularly now that they have near-monopoly power in the sector. People should also be able to adjust the dials on the algorithm themselves.
I use Google through a VPN to avoid it. That breaks maps integration.
It's absolutely catastrophic if you are not allowed to draw your own conclusions about things. This is your inalienable prerogative as an adult in a free country. Even at the risk of some people being wrong sometimes, you simply cannot have authorities distributing doctrine and call yourself a democracy.
DDG shows the author's substack as #1 result and is neutral otherwise. The other doesn't even have it on the first SERP, and is overwhelmingly critical.
If you argue that covid response is not politics, I will disagrer strongly.
So search engines now have to get the "truth", preferably the politically correct one, and since you can't rely on the crowd for that, you have to introduce bias, and "pre-approved news outlets" are the most obvious choice.
Instant answers (IA) caused a shift in the way contents are written. Content optimized for IA tend to be repetitive and shallow. Viewing content written for IA is a frustrating experience and these tend to dominant the result page now.
Both Google search and the internet are changing, as are our perceptions, so it's really hard to say why search quality is better or worse.
A bit off topic: Some of the search queries in the article used Google like a keyword matching tool, while other queries in the article were using Google to answer questions. That's because we only have one search box trying to do everything. Would we be better served if we have a checkbox to tell the search engine that we just want to perform word searches?
Poor Jeeves. He was before his time.
Meaningful websites still exist. The bulk of the content on the Internet is older than IA, and it's still out there (not that you can find it with Google).
Marginalia search, by punishing ad- and tracking-heavy pages and by being strict in how it interprets my queries sometimes surfaces better results than Google and DDG.
In particular I have found great resources about Linux partitioning and git usage after giving up mainstream search engines and trying them in marginalia.
That says quite something about how badly broken the situation is given that marginalia is one person and a tower pc in a living room.
I keep getting reminded about a Linux quote on how they managed to go forward by studying the latest 20 years of OS research and throwing it all away :-)