HN Explorer

Ask HN: What are these low quality “code snippet” sites?

594 points by endofreach ↗ HN
Whenever i am trying to google a code issue i have, there is countless low quality sites just showing SO threads with no added value whatsoever. It is so annoying it actually drives me mad.

Does anyone know what's up with that?

I am really disappointed because the guys creating these sites (i guess for some kind of monetization) must have some relation to coding. But i feel this is an attack against all of us. Every programmer should be grateful for the opportunity to find good quality content quickly. Now my search results are flooded with copy & paste from SO. They are killing that.

Am I the only one experiencing this or being that annoyed by it?

P.S: I don't name URLs because if you don't know what I am talking about already, you probably don't have that issue.

322 comments

[ 5.6 ms ] story [ 280 ms ] thread
brianwawok
It’s an easy way to make money. Scrape a popular site like Stack overflow or Wikipedia and add a bunch of advertisements.

One of the many ways that scum ruin the web.

didericis
Wonder if there’s any way to get advertisers to veto scraped content sites. Not really in their incentive to boycott eyeballs, but if advertisers were actively avoiding those sites, it’d dry up the incentive to make them.

This could backfire, but fining advertisers that show up on those sites might work. The difficulty would be all the claim verification and process of determining what exactly is a “scraped site”, and backfire scenario could be another hurdle for non established sites/more centralization of content. But if you targeted the advertisers rather than the sites themselves, the advertising networks would be incentivized to do that identification themselves.

penteract
Contacting the advertisers to tell them that being shown on those sites makes you want to avoid their products might work, although that would involve not blocking the ads.
slater
Spammers. They mirror SO stuff on their own sites and put google ads on them
ruffrey
I suspect they were always there, but google and ddg are getting gamed more now. The quality of results has dropped quite a bit in the past 4 or 5 years in this regard.
lloydjones
As others have said: SO content is ripped-off (poorly) and mirrored. The page games Google's algorithm and shows up as a 'legit' result.

Probably more complex than simple keyword stuffing, which isn't supposed to work these days..

marginalia_nu
I don't understand how these sites rank so highly though, they can't get much organic traffic at all. Most people who click on them will bounce immediately.
Nasrudith
Generally they show up as one of the few options if you look for very specific subqueries - that is when I tend to find them.
legrande
Just whitelist Stack Overflow in your head and avoid splogs / spamblogs?
grinchygrinch
Same thing with various GitHub issues lookalikes.
TYPE_FASTER
People are crawling content that is searched for frequently, then using SEO to rank higher in the results than the original content to make money from the ad revenue.

Code and recipes are two examples.

I'm also seeing politicians posting Tweets containing a link to their personal website, which has ads.

pupppet
Really ticks me off that Google allows itself to be so easily gamed, it's your core business for christ's sake.
tenebrisalietum
Desktop web search is probably less the core business these days than making search work well on phones/mobile devices with specific applications, such as Maps, etc.
jjallen
Why would they do that though? Why would they show you a low quality result if there was a better one available? I have pondered some of these sites as well. I think the answer is that they are actually helpful to some people.
nickthesick
I don't think they track 'quality' they just see people clicking and use that as a proxy for 'quality'. (Quoted because what does quality even mean?)
nitrogen
I seem to recall reading that they (used to?) track abandonment of the search results page as a successful answer to your question from the embedded page snippets or info boxes. So if you get a search result page that's complete garbage and just give up, they increase the ranking of those sites.
tenebrisalietum
Businesses are profit seeking entities run by human beings who tend to prefer conservation of energy. If a business goal can be met with X effort, X+1 effort is a waste and the resources can be directed elsewhere or kept as profit.
pixelgeek
No it isn't. Ads are their core business. Those crap sites almost all run ads from Google
dgb23
That's a shortsighted view because it would imply that the quality of search doesn't have any bearing on their success. I highly doubt any Googler would agree with that. Especially not the ones who have been working on search specifically.
_jal
Extrapolating from an employee's feelings to the behavior of a huge public company is a terrible way to model reality.
spiralx
This whole topic is extrapolating from people's feelings to the behaviour of a huge public company. I've not seen a single piece of actual data anywhere so far.
dgb23
That's not what I did. I was implying that those who have a stake in Google's success also have a stake in their search quality.
throw_m239339
> That's a shortsighted view because it would imply that the quality of search doesn't have any bearing on their success. I highly doubt any Googler would agree with that. Especially not the ones who have been working on search specifically.

Google is the goto search engine today, the competition is virtually non-existant, so they don't have to compete anymore. Hence the quality of their search engine going down hill, conflict of interests and what not. It's not shortsighted, it would take billions to compete with Google search and even Bing isn't even trying anymore.

When I look at a Google search page result today I see loads of ads, then their shopping stuff, image search stuff, a list of questions related or not to my search and their answers extracted from third party web pages, then only when I scroll 1 page I see actual search results. Google search wasn't like that 15 years ago.

Googlers are working so that Google makes more money, and Google is an ad company first and foremost.

dgb23
But that's pretty much how Google pushed out the then popular search engines: with search quality and a minimalist UI. It would be irrational for them to stray away too much from that quality focus.
dehrmann
I used to work for a company that gamed Google like this, but with product search results (that were, themselves, CPC ads). The company got bought for scrap because Google got good at downranking their pages in search results and finally taking shopping seriously.

A lot has changed since then, but I doubt that Google has decided these kinds of content farms are suddenly OK. It should also give pause to people asking for "open ranking algorithms." Without some amount of secrecy, content farms will pop up for literally every keyword they can sell ads against, and they'll know how to rank more often rather than just blindly guessing.

WhisperingShiba
It does give me pause, but do you really think there is no way to have both open algorithm and good results? What is users assessed sites as they used a search engine, and then this was broadcast back to the engine?

I have a feeling there is a way to introduce randomness to this search engine in a way that smooths out the game theoretic arms race of gaming an algorithm. This goes against our natural urge to have a fully deterministic algorithm which always brings the most pertinent sites right at the top. However, because humans are trying to bring order to things for their own benefit, throwing in chaos could prevent people from even trying to game the algorithm

motoxpro
Markets dont work when everyone knows what I am THINKING. i.e. If I know with 100% certainty you are going to buy something for 1$ I just bid 1.0001. Same with Google search. If I know with 100% certainty that you rank pages in a certain way, I can game them. Markets only work when all data is available but no all opinions.

Introducing random elements just makes it worse for everyone.

WhisperingShiba
So why was google more effective before? Is this is product of the financial struggle of our society, or do you believe it is an inevitable consequence of a dominant search engine?
motoxpro
It is a lot more effective for me. I usually find what I am looking for in the first few results, there are flights, calculators, wiki pages, etc. So I disagree it has gotten worse.

The task of search has gotten exponentially harder as well. There are billions more website pages, more spam, more sophisticated attacks, more gaming of the system, etc. They could have gone from a 80% relevant result to a 99% relevant result and you perceive at 10% drop in relevancy. It's a hard problem.

Nasrudith
Neural network weighting loops and adversarial inputs say yes, it is impossible.
UweSchmidt
We have to assume that Google devs see these crappy spammy code websites too on a daily basis, right? What's up with the incredibly strong programmer culture over there, gone already after 20 years? You'd think someone would take these weak search results personally...
Hnaomyiph
As someone who knows a handful of devs and hear their near daily grievances of google, it seems like the culture over there has died and it’s just a bog standard Big Corp(tm) now. By how they describe it, a lot of the people, (but not all of course), who deeply care about the ‘craft’ of development have retired or moved onto more interesting roles at newer and more exciting companies and were replaced by people whose main driving force is climbing the corporate ladder at a FAANG. As a result the perf cycles/promotions seem to be heavily based on high impact projects/launches, leaving the “little things” like bad search results or other maintenance work forever on the back burner, because working on those things won’t get you a promotion/more money.
tomcooks
Search hasn't been Google core business for decades, they're into selling advertisement spaces
endofreach
Well, as i wrote i understand that they try to monetize it.

But: why the sudden explosion? I feel there is more of these sites going live regularly. Many times they make up 80% of the first pages on search results, just repeating the SO threads listed before. So it‘s really getting difficult.

Something must be done…

coldpie
Everyone should install an ad blocker so these types of "businesses" become unprofitable. Installing an ad blocker is the ethical choice.
bellyfullofbac
Maybe someone figured out The Google Algorithm (TM/all hail the algorithm...) is allowing source code on dodgy sites, so they've created and marketed to spammers a script/bot to create these kinds of sites (or all the sites just belong to them) to farm that ad money.

A bit like how when "hoverboards" got popular, Chinese factories started churning them. And then when fidget spinners happened, same thing: https://www.buzzfeednews.com/article/josephbernstein/how-to-...

lozenge
Maybe these sites share some code? If you've already captured 2% of searches with one site, why not capture 1.5+1.5=3% of searches by running two sites.
vegancap
I really hate these. Especially when I'm trying to figure something out and I'm struggling to find answers, I end up haplessly finding the exact same wrong answer on three different sites.
brootstrap
I've 100% noticed it in the past months. Have a new employee that i'm trying to train up a bit with python and googling has gotten more annoying. fucking codegrepper.com & shit. FUG EM . hilarious when you find the exact same snippet over and over. Even more hilarious those asswipes are probably making $$ (potentially lots) with all the stupid clicks n that.
vegancap
Oh god yeah, I wish SO got better at blocking scrapers, I'm pretty sure services like CloudFlare have anti-scraping protection. They need killing off ASAP!
notreallyserio
I've found that Bing does a better job at detecting spam like this. Not perfect, just better.
viktorcode
The issue is actually pretty old. There was a time when Google introduced blacklisting of search results and revenue of those sites dived. Sadly, later Google rolled back the blacklist.
matt_heimer
Any idea why they rolled it back?
datenarsch
Because Google is not interested in serving the best possible search results but rather in serving those that will make them the most money.
russh
It was effective and those affected were able to get the feature axed.
dmortin
Search spam sites can be reported to google: https://developers.google.com/search/docs/advanced/guideline...
beefield
Quoting: "While Google does not use these reports to take direct action against violations"

Given e.g. pinterest in the google results, I find it difficult to imagine as sure way to waste your time uselessly than to report SEO spam sites to google. It is obvious they do not care the slightest bit.

reliable
Is it bad that I've actually found my answer on some of these sites haha. But yeah, they're pretty low quality in general.
didericis
Yes; if the content were original or the different context was increasing discoverability, that’d be ok, but the duplicates of content posted elsewhere almost always just obscures the original, better post with better context and real users to follow up with.

I’ve googled things and gotten an answer pop up from one of these types of sites too/not saying its something to beat yourself up about, but if you’re playing devil’s advocate and suggesting there’s discoverability value added outweighing the other value lost, I disagree.

hawthornio
I’ve had to switch to !py on ddg because the official Python docs never make the first page. It’s really frustrating. :/
MiddleEndian
uBlacklist is a great tool to block sites you don't want from showing up on your search results on google (and a couple others including bing and duckduckgo). It supports regular expressions as well, I use /pinterest\..*/ to block all pinterest-related content.

https://addons.mozilla.org/en-US/firefox/addon/ublacklist/

https://chrome.google.com/webstore/detail/ublacklist/pncfbmi...

shever73
That's a great tool, thanks for the recommendation. Pity it doesn't work with Ecosia's image search. I wrote my own extension to remove Pinterest from image results, and I've been thinking of extending it to block Instagram and Facebook too. I hate how these companies point their sewer pipe at search engines, so you get into a walled garden where you have to sign up to see the indexed content.
jetrink
I wish Google would add that feature natively. Google could even use it to help detect spam sites.
[dead] bserge
Oh dear God, no. Don't need Google to have even more control. Imagine blocking something and it shows up anyway because Google decided so.

Oh wait, you don't have to imagine, just use Gmail on Android. "Spam? That's not spam" - Google.

Or just Android with it's fucking "notifications" from all the garbage news sites. Tbf, my fault for using Google apps.

Honestly, blocking shit you don't want to see locally is the best solution. Decentralized, under your control, no need to trust BigCorp number 294.

enobrev
Google used to make it possible to filter domains from results. I was sad to see it disappear
leephillips
You can still do "-site:spam.com".
thesuitonym
This works, but bad actors like pinterest have found a way around that by having a domain on every TLD and ccTLD out there.
cuspycode
I found out by experiment that wildcards work, e.g. -site:pinterest.*
BizarroLand
There's also Personal Blocklist, which appends a small "block site.etc" under every google search result.

If there is a site that is clogging up your search results, just click it and it's gone.

https://addons.mozilla.org/en-US/firefox/addon/personal-bloc...

riezebos
To me, this comment made it sound like this "block site.etc" was a feature that Personal Blocklist has and uBlacklist doesn't, but if I open the ublacklist page the first screenshot of the extension shows "Block this site" next to each search result. Is the "block site.etc" in Personal Blocklist different from the one in uBlacklist?
BizarroLand
I haven't used uBlacklist, this was a solution I had independently stumbled into and I wanted to share it.

Based on the dialogue from the posters I was responding to, it seemed like uBlacklist would require some ongoing maintenance or memorization of input fields to work.

If it also offers a single-click permanent block of all sites in a domain from a Google search then that's just as cool, but that feature wasn't obvious from the conversation.

nerdponx
It'd be pretty neat if DDG let you configure this as a setting.

They don't have "accounts" as such, but they do provide users with a one-time password that lets you save and restore settings.

Alternatively, if they at least supported "do not include" filters in regular searches, you could build a client that stored those kinds of search defaults locally. The best you can do now is strip out those results on the client side, which isn't bad but seems like a hack.

Kiro
They can't do that since they are getting their results from Bing.
freediver
We built this into Kagi Search, natively, I invite you to try it out. You can both “mute” and “prefer” domains in your searches. Currently in beta!
mypastself
The biggest problem is that they waste your time even when the content is ostensibly helpful, since the search result is usually listed after the Stack Overflow page it crawled from anyway. Each click steals a few seconds of developers’ time, which adds up, given how frequently these types of results pop up on Google search lately. That makes them worse than useless, they actively subtract value.
nickthesick
Not only SO threads, I particularly hate the ones that mirror GitHub Issues. They don't even link back to the original thread, for Christ's sake!
yumraj
And as I had mentioned in another thread, DDG so far ignores them, at least in my 1 case, but Google has those GitHub issues mirror sites as the first results. So they are clearly gaming Google SEO.
znpy
I'd blame gooogle more than the website, since google search results are generally getting worse and less relevant
monkeybutton
Totally. This is a exactly what I was thinking about yesterday in the thread where someone was asking if Google results were getting worse.

There's no reason the official docs for Python should be lower in the results than a shitty docs clone / spam site when searching for a common package/function in the standard library.

kikokikokiko
"There's no reason the official docs for Python should be lower in the results than a shitty docs clone"

If Google still used/respected the original page rank algo, the official docs would never be ranked lower than a spammy clone site. I just wish they reverted back to the "power of the crowd" algo, let each node vote with their links and reputation.

Nowadays I almost stopped using Google completely. I first noticed it with torrent/streaming censorship ~5 years ago, then the political/ideological censorship started to feel unbearable. I just want my search engine to show me the most relevant results BASED ON THE QUERY I SUBMITTED. No moral judgement crap.

My search stack nowadays is a mix o Bing, ddg, Yandex and if everything else failed, Google. It's a sad reality.

coryrc
I'm getting lots of these in DDG too. Right now I'm doing searches about home renovation and I see lots of pages which are just a rip off of an online forum page (which I find elsewhere in my results, usually). Which, unlike SO, is not licensed for people to copy.
deadbunny
Sample of 1 as well but I do see them on DDG, not nearly as much as Google though.
ziml77
My experience has been finding more garbage on DDG, not less. I ended up getting too tired of those results being above the good ones that I switched back to Google.
yumraj
Try this, the exact query I was referring to. I leave it to the reader to decide which is better.. The top 2 Google results are scrapped clones (or whatever they should be called) of the 1st result from DDG (Github issues #162 for docker-calibre-web)

The closest Google comes in top 4 links is a link to the entire Github issues for docker-calibre-web

DDG: https://duckduckgo.com/?t=ffab&q=FakeUserAgentError+calibre-...

Google: https://www.google.com/search?hl=en&q=FakeUserAgentError%20c...

DDG Results:

1) https://github.com/linuxserver/docker-calibre-web/issues/162

2) https://www.nas-forum.com/forum/topic/67746-tuto-calibre-web...

3) https://pypi.org/project/fake-useragent/

4) https://github.com/linuxserver/docker-calibre-web/issues/

Google Results:

1) https://githubmemory.com/repo/linuxserver/docker-calibre-web...

2) https://issueexplorer.com/issue/linuxserver/docker-calibre-w...

3) https://github.com/linuxserver/docker-calibre-web/issues

4) https://github.com/janeczku/calibre-web/issues/1527

nobody9999
Interestingly, as of 02 December 2021 ~0300 US/Eastern, this thread is the third entry in your Google search. It doesn't appear at all on the first page of the DDG results.
yumraj
Wow, I checked and that is the funniest thing ever and totally unexpected!!

FWIW, I see it as 4th item in Google search results, but that is not important. It's just weird that that will show up at all.

Again, I'll say, that's a win for DDG results, at least IMveryHO.

tomtheelder
Wow total opposite experience for me. I find Google much better for casual searches, but DDG oceans better for anything technical.
johnchristopher
Just found out about github clones today. It threw me off.

At least for SO you can get back to SO to search but github issues search engine is something else.

bschwindHN
The answer is, if someone can make money by doing something shitty but not illegal, they will do it.

Almost everything on the web is some scheme to put ads in your face so someone can make some money.

[deleted]
(comment deleted)
alx__
Yeah there's a whole mini scam network that copies programming how-to videos on YouTube and puts new thumbnails on them. So you watch the ad and then get 30 seconds into the video before realizing it's garbage, they've made some a few stupid pennies :(
bigtex
If I recall the Stackoverflow dataset is open source or at least made available to download so I assume all these sites just download that information regularly.
ljm
My pet peeve is ApiDock, which has managed to SEO itself so high up the rankings when searching for anything connected to Ruby or Rails that it is actually quite difficult to get to the legitimate, official documentation.

What's worse is most of the results are outdated so you're looking at web-scraped API docs for Rails 3 or something.

Really frustrating.

pricci
And the site is so slooow for me.
terafo
Sites that do auto translate of original SO threads and pretend that it's their original content are the worst. Google sometimes prefers to give me that results instead of the actual SO thread because I'm not in English-speaking country. I have to waste some time to understand that it's just stolen SO thread. And it's not even that useful because some of them AUTO TRANSLATE CODE.
bserge
Lol. Google's absolute insistence that you must want local language results has been a pain in the ass for years.

It's impossible for them to understand. Clearly they have zero people working and living in countries other than their own. /s

prox
DDG allows you to set your own country or any country easily (without needing an account)
throwaway_dcnt
I spoke to a VP at google in 2006 in london and discussed using a combination of curation and entropy to flush out duplicates. He seemed pretty excited by the idea but I don't think anything materialized. Which is another way of saying this is not new - in those days these sites were copying newsgroups too.