I dislike the original study, since it repeatedly (including the title) makes claims about Google results, but in reality the researchers did not analyze Google results due to scraping Google being too hard. Instead they analyzed results scraped from a search engine, and assert with no proof that they're representative of Google results.
But this 404 article is even worse. The conclusion of the study wasn't tha the results have gotten worse. It was literally the opposite!
> In fact, the Google results seem to have improved to some extent since the start of our experiment in terms of the amount of affiliate spam.
But just like the researchers got more attention with the misleading title about what was studied, the journalists at 404 got more clicks by outright lying about the results.
>But this 404 article is even worse. The conclusion of the study wasn't tha the results have gotten worse. It was literally the opposite!
> In fact, the Google results seem to have improved to some extent since the start of our experiment in terms of the amount of affiliate spam.
> But just like the researchers got more attention with the misleading title about what was studied, the journalists at 404 got more clicks by outright lying about the results.
There are some questions if the scraping via Startpage is messing with the result. They are using the Google crawler, but their anonymisation might mess with the results.
I don't agree with you interpretation of the result though. if we take a look on a longer excerpt of the conclusion, they do mention multiple times that the quality is getting lower:
> Although we cannot predict the rank of individual pages, at the population level, we can conclude that higher-ranked pages are *on average more optimized, more monetized with affiliate marketing, and they show signs of lower text quality*
and even the part you quoted goes on to mention a downward trend:
>In fact, the Google results seem to have improved to some extent since the start of our experiment in terms of the amount of affiliate spam. Yet, we can still find several spam domains and also see an *overall downwards trend in text quality in all three search engines*, so there is still quite a lot of room for improvement.
> > Although we cannot predict the rank of individual pages, at the population level, we can conclude that higher-ranked pages are on average more optimized, more monetized with affiliate marketing, and they show signs of lower text quality
As far as I can tell, that's not a claim about variance over time, i.e. about results being worse now than in the past. It's a claim about how the current population of pages and their current ranks, i.e. a page ranking higher is likely to be more SEO-optimized than a page ranking lower. It makes no claim about whether that was the case in the past, and if it was, whether it was true to a larger or lesser extent.
> Yet, we can still find several spam domains
They would have been able to find several spam domains at the start of their study, five years ago, ten years ago, or fifteen years ago. This statement is just totally empty when talking about whether the results are getting worse over time or not.
> and also see an overall downwards trend in text quality in all three search engines, so there is still quite a lot of room for improvement.
Sure. Did you check on what their definition of "text quality" is? I tried to, but couldn't since the paper never actually states it. But the only thing they actually report temporal statistics for is the "type-token ratio". Sounds fancy! What it turns out to be is "the number of unique words on page / number of total words on page".
That doesn't seem like a very strong claim about actual quality, especially when the only statistics they report is the 95th percentile.
Even if there were some kind of AI search on the horizon that would completely dispense with the shitty, SEO-bolstered sites (the faux-web) Google, etc. would still find a way to post-process or bootstrap their ads into the results.
I wonder if it will be possible in the near future for a "collective" or "org" to hoist their own search engine paid for by donations. Something to complete a Wikipedia/Archive.org triumvirate and usher in Web 1.5.
Hopefully more like archive.org and less like Wikipedia or the search results will be as badly biased as most Wikipedia topics are which are even tangentially related to politically contentious subjects.
I do, occasionally and seem to notice the same pattern, yes. If ever there were to be created a truly politically neutral LLM it could be used as the back end for a browser extension or wikipedia proxy which would add bias markings to all wikipedia-hosted content. Creating a truly neutral LLM would be truly hard though given the biased nature of the training set.
Nor was I referring to the quality of the search functions on those sites but rather to the quality of the content they provide - the very content you just dubbed the good Web content we deserve. Wikipedia does have some good content but it is a steaming pile of ${excrement} when it comes to anything even remotely politically contentious as those articles tend to be hijacked by activist editors who allow nothing but their own narrative to remain. For such articles the talk and history pages are far more informative than the agitprop in the article.
archive.org does not editorialise content so they are the better example here. They may be selective in which content they archive but I have not yet noticed any specific bias there - which I consider to be a good thing.
Maybe doesn't matter. I have shifted quite a bit to Bard. And have been wondering whether I need Google Search anymore. Its taking me back to the days when Google first came out and suddenly noticing I was using Google search all the time.
I have switched to using Perplexity, as I found it to be more accurate than to Bard. The last time I used Bard, it hallucinated an entire nonvexistent module of a Java NLP library. I asked about: Pair-HMM Learning and Inference in Java in Mallet ... and it provided incorrect information about a module that does not actually exist.
It doesn't take a genius to see that if you build a product in which people can artificially boost their rank, you're gonna get a whole lot of shit. From the get-go SEO's concept has been to inflate the value of the content with _optimizations_. You can't be surprised when you design your product with the optimization being more effective than having legit content at its core.
Good point, but before we get into that, let me talk about my grandmother for a moment. She was a great cook, eager to learning new recipes, especially with potatoes. Potato is a great plant, rich in essential vitamins, easy to cook, very healthy...
For giggles I asked ChatGPT to write a typical SEO optimized recipe. It picked cookies and started with:
"Ah, the sweet aroma of freshly baked cookies! It always takes me back to my grandmother's cozy kitchen, where the warmth wasn't just from the oven, but also from the love and laughter we shared."
It's always impressive (almost scary) how well ChatGPT manages to work with these types of prompts.
The Register I can anticipate giving me a clickbait headline, but I'm a little sad that 404 media is choosing to go that way.
They are relatively new and have an opportunity to choose the tone they want to set, and there's already enough clickbait on the internet. Claiming Google search has gotten worse and then following up with the immediate next sentence indicating that the study showed that search is worse across the board is a bad look.
It will only get worse as they adopt ai. Quite often searching for one term yields results for a different term. Most probably hallucinations. Still better than bard or chatgpt.
I would argue that it's not only Google that got worse, the whole web got worse. I wish for a search engine that would exclude any website that is overleaded with ads. The side effect would be better quality content.
The main problem researchers found is that the content quality has deteriorated. There is more spam now. However, Google is doing a better job at filtering low quality content than anyone else.
I've been working on a side project to filter spammy content and get straight to the point answers: Https://AskPandi.com
26 comments
[ 6.7 ms ] story [ 63.2 ms ] threadI dislike the original study, since it repeatedly (including the title) makes claims about Google results, but in reality the researchers did not analyze Google results due to scraping Google being too hard. Instead they analyzed results scraped from a search engine, and assert with no proof that they're representative of Google results.
But this 404 article is even worse. The conclusion of the study wasn't tha the results have gotten worse. It was literally the opposite!
> In fact, the Google results seem to have improved to some extent since the start of our experiment in terms of the amount of affiliate spam.
But just like the researchers got more attention with the misleading title about what was studied, the journalists at 404 got more clicks by outright lying about the results.
> In fact, the Google results seem to have improved to some extent since the start of our experiment in terms of the amount of affiliate spam.
> But just like the researchers got more attention with the misleading title about what was studied, the journalists at 404 got more clicks by outright lying about the results.
There are some questions if the scraping via Startpage is messing with the result. They are using the Google crawler, but their anonymisation might mess with the results.
I don't agree with you interpretation of the result though. if we take a look on a longer excerpt of the conclusion, they do mention multiple times that the quality is getting lower:
> Although we cannot predict the rank of individual pages, at the population level, we can conclude that higher-ranked pages are *on average more optimized, more monetized with affiliate marketing, and they show signs of lower text quality*
and even the part you quoted goes on to mention a downward trend:
>In fact, the Google results seem to have improved to some extent since the start of our experiment in terms of the amount of affiliate spam. Yet, we can still find several spam domains and also see an *overall downwards trend in text quality in all three search engines*, so there is still quite a lot of room for improvement.
As far as I can tell, that's not a claim about variance over time, i.e. about results being worse now than in the past. It's a claim about how the current population of pages and their current ranks, i.e. a page ranking higher is likely to be more SEO-optimized than a page ranking lower. It makes no claim about whether that was the case in the past, and if it was, whether it was true to a larger or lesser extent.
> Yet, we can still find several spam domains
They would have been able to find several spam domains at the start of their study, five years ago, ten years ago, or fifteen years ago. This statement is just totally empty when talking about whether the results are getting worse over time or not.
> and also see an overall downwards trend in text quality in all three search engines, so there is still quite a lot of room for improvement.
Sure. Did you check on what their definition of "text quality" is? I tried to, but couldn't since the paper never actually states it. But the only thing they actually report temporal statistics for is the "type-token ratio". Sounds fancy! What it turns out to be is "the number of unique words on page / number of total words on page".
That doesn't seem like a very strong claim about actual quality, especially when the only statistics they report is the 95th percentile.
I wonder if it will be possible in the near future for a "collective" or "org" to hoist their own search engine paid for by donations. Something to complete a Wikipedia/Archive.org triumvirate and usher in Web 1.5.
archive.org does not editorialise content so they are the better example here. They may be selective in which content they archive but I have not yet noticed any specific bias there - which I consider to be a good thing.
Even recipes nowadays are SEO boosted
Good point, but before we get into that, let me talk about my grandmother for a moment. She was a great cook, eager to learning new recipes, especially with potatoes. Potato is a great plant, rich in essential vitamins, easy to cook, very healthy...
They sell food, so the recipes seem to be quite to the point.
"Ah, the sweet aroma of freshly baked cookies! It always takes me back to my grandmother's cozy kitchen, where the warmth wasn't just from the oven, but also from the love and laughter we shared."
It's always impressive (almost scary) how well ChatGPT manages to work with these types of prompts.
They are relatively new and have an opportunity to choose the tone they want to set, and there's already enough clickbait on the internet. Claiming Google search has gotten worse and then following up with the immediate next sentence indicating that the study showed that search is worse across the board is a bad look.
I've been working on a side project to filter spammy content and get straight to the point answers: Https://AskPandi.com