Ask HN: Why does Google rank the real Python documentation below content farms?

Honestly, changing the way they do it is a fine idea. Massive documentation pages were how we did it in the olden days. And I think they're still good for when you want to read through and get an understanding of the whole library. But having that also broken up into a lot of small pages that link back to the main doc would be really helpful to people who are searching for specific solutions.

I think that could mostly be done in an automated fashion to start. Then into the master doc you can start adding hinting and extra material that gets rendered into the sub-pages. So it could be approached gradually.

I really hope they don't enshittify their documentation just to rank in Google.

I disagree - Google can and should modify their algorithms until the site that users actually want is first. Tracking which sites users click on for those searches, or maybe which sites users with those searches click on last should give a pretty strong signal that python.org should be ranked far above the spam farms no matter what the content percentage difference is. Content percentage is a clearly gameable metric that shouldn't outweigh other factors.

I'm honestly surprised Google employees haven't fixed this somehow, surely they have thousands of people running into this problem daily.

Somehow kagi.com manages to rank the top results better than Google:

- a stackoverflow answer that has a useful example: https://stackoverflow.com/a/18351977/

- the Python documentation: https://docs.python.org/3/library/stdtypes.html

- Pandas documentation (they also have an endswith): https://pandas.pydata.org/docs/reference/api/pandas.Series.s...

- a blog post on Python regular expressions (this looks useful but is a wrong result for this query): https://www.johndcook.com/blog/python_regex/

- something that could be a content farm with rephrased Python documentation: https://pythontic.com/concepts/string/introduction

The top three results are good, and I guess it's a matter of taste whether the succinct Stackoverflow answer is better than the verbose official documentation.

Aren't you describing the exact problem that Page Rank was supposed to fix?

We've come full circle.

> unless Python themselves fundamentally change how the documentation is structured, which I doubt they have plans to do.

Nor should they, in my opinion. Quality of documentation is more important than that.

>The bottom line being that there is nothing you can do

Google's ENTIRE JOB is to surface the CORRECT content for a query. If they are objectively not doing that, then they are in the wrong. Imagine credit card fraud just kept increasing, and instead of adapting and improving their systems, companies just threw up their hands and said "deal with it, not our problem, nothing we can do" as you had to generate a new credit card number every other day because even the most trivial card number enumeration attacks were not stopped.

The fact that google is completely unwilling to change how they operate to prevent even the simplest, intern driven SEO trash from overtaking literally authoritative sources should be exhibit A in a trial to break up Google, as evidence of how negative value they are to the consumer.

It's the rough equivalent of Fedex just throwing away a third of the packages they are supposed to deliver at random, and people seemingly just saying "eh, nothing can be done"

Kagi has a simple fix for this. It’s not a hard problem to solve, they just have no incentive to solve it.

Google shouldn’t whitelist them but use them as a test case for the ranking algorithm. If a content farm comes up ahead of the official documentation site then the algorithm is obviously broken.

I don't think word count is a new metric. It's just that documentation wasn't much of an ad revenue market in the past. Link juice also flows for backlinks. These sites definitely get heavily linked by blog spam.

SO isn't a canonical source, they're another content farm using nofollow links to the content they appropriate to maintain their search edge.

Its entire purpose is to inject itself as a search middleman. Even worse, I find myself landing there from google results to a question they had closed as not worthy of the site, but worthy enough still to keep up for the search juice.

> Google has no real algorithmic idea who is copying who.

This is an irrelevant distraction, because Google's literal army of nearly 200,000 full time employees, a large number of whom are Python developers, do know. Geeksforgeeks, w3schools, programiz, tutorialspoint, and python-reference.readthedocs, which all rank higher than the official documentation, are not flash-in-the-pan sites. They've been polluting the search results for literally years. Trivial manual processes at Google scale would be 1000% fine and effective.

Google could, if it gave half a shit, give every employee a Chrome browser extension that lets them manually vote on the reasonability of site rankings within known problem genres for their own searches.

Isn't boosting websites without ads against their interest?

Are you using an ad blocker?

DX: "Developer Experience"

I suspect that someone who needs to know that their developer experience is broken probably doesn't know what "DX" means.

They've added plenty of exceptions in the name of "cards". Wikipedia, movies, celebrities, ...

I think your TLDR is accurate, but I think the mechanism is likely even simpler than Google looking at the last page visit.

The docs are written by and for experienced programmers. They're very dense with information but light on examples and comments that explain things for dummies.

Its popularity means the vast majority of Python users are novices who would find the docs hard to read. Geeksforgeeks et al are providing content for them and so is actually a better resource to show the majority of searchers (especially given that it's a basic string formatting question, something very likely to be searched by for novices).