Ask HN: Why does Pinterest dominate Google text search results?

689 points by Winterflow3r ↗ HN
More and more often when I search (using text queries, not image search, which I know has been polluted by Pinterest for years), I get pages upon pages of Pinterest results, sometimes the same Pinterest page but from the different pinterest country domains like pinterest.fi for Finland and pinterest.se for Sweden. Does anyone know if Google gives Pinterest preferential treatement in SEO rankings?

Edit: A few comments were asking what my queries were to generate search results where Pinterest dominates, so clarifying that a bit. I run a site that has a colour search engine for lipsticks and since Google is one of the dominant ways in which people land on my site (searching for things like "nyx budapest lipstick dupes"), I was studying various makeup related queries to see which sites ranked highest .

Edit2: Edited the title for clarity - I mean text search, not image search

328 comments

[ 3.0 ms ] story [ 291 ms ] thread
“Deconstructing Pinterest’s reverse-image-search SEO growth hack”

https://www.rankscience.com/blog/pinterest-image-seo-growth-...

Ugh, this is bad. Makes you wonder, why they let this happen. Do people at Google own Pinterest stocks?
Yeah, Google has done so many things to thwart so many kindS of bad SEO (and sometimes even good SEO) that it's really hard to believe they've just innocently dropped the ball on this one. At a certain point you just have to conclude that they want the search results to look like this. "Oopsie, we just accidentally let one bad actor trample on this valuable 'real estate' for years. Silly us." Yeah right.
> Do people at Google own Pinterest stocks?

TIL Pinterest is a publicly traded company (NYSE:PINS).

I think search at Google isn't getting the best and brightest anymore. It used to be the only product at the company, now it's just the oldest.

I also think there's a disconnect between how new algorithms are evaluated and real world results. Reverse image search is total trash now. Mostly returns annotations "red, dress, girl" instead of the pages where I can find the image.

Unless it's a brand name product, then it's exact matches.

So you're getting less than excellence in engineering, plus influence from shopping and ad dollar focused PMs and not much care is actually given to the search experience.

And it shows

Oh wow :( I'm a bit scared by this, because of what it means for smaller competitors, who can't afford to pour investor dollars into hiring a whole team of software engineers just focused on SEO driven growth
There's no intentional or manual effort at Google to promote pinterest.

Pinterest shows up because they understand how the Google algorithm works and built their website to display all the signals that Google looks for in relevant image content.

They understand user intent and generate URLs that present content in a way that google expects to see.

Examples of how they do this from their engineering team:

https://medium.com/pinterest-engineering/demystifying-seo-wi...

More:

https://medium.com/pinterest-engineering/tagged/seo

Google should be asking from a product perspective if they are providing the search results the user is most likely seeking.

I seriously doubt that most pinterest results are what users are seeking. As such, Google probably should consider changes to their algorithm to compensate (cat and mouse) for pinterest's gaming. That is assuming how pinterest does it is not against any google rules... because if it were against any rules, they should be slapped down hard.

When my searches result in mostly pinterest results, I give up more quickly. That means I spend less time using google, and that in theory means less money for google. Unfortunately, I do see other more regular user types operate their computers, and they are less or un-descriminating about the domain name listed under the image. So they'll head directly into the pinterest waste pond anyway.

Just do -pinterest in your search to filter
Google account preferences used to have an option to exclude domains from results. About a decade (?) ago I added Burleson Consulting and expertsexchange to blocked domains and Google was instantly useful again. In 2014-ish Google moved this functionality to Chrome so I lost access to it. Not sure if it is still there.
Or just avoid using google. The Pinterest results in google image search queries was a major driver to me using bing/ duck duck go.
Linking directly to the image file (and in fact a cached version of the full image) is one of the best reasons to use DDG.

Reverse image search is also intentionally gimped, unlike non-US jurisdiction Yandex which is quite good.

It's certainly for legal reasons, not technical, but Image Search has gotten progressively worse every year. It used to be godly effective 10 years ago.

> Linking directly to the image file (and in fact a cached version of the full image) is one of the best reasons to use DDG.

That’s possible thanks to Microsoft’s Bing API. I’m surprised Getty (who’s afaik responsible for Google removing direct image links) only went after Google and not Microsoft.

Thank you, but still, the regular person is not finding what they seek, but what SEO experts want them to find.
Clearly we should not have no include -fakenews -porn, etc, otherwise I might as well use bing / Duckduckgo/ whatever
User: This service has a problem and it's not working well.

Programmer: There's nothing wrong with the service, you just have to use [magic incantation] to make it work right.

I've been hoping for 20 years that this kind of exchange would stop happening. Still hoping! :-)

Services and users have different goals. Why is the helpful technologist the problem?
Your second statement is enabling bad behavior, your first statement is insane.
No. They're exactly right. The root problem here is incentive misalignment. Both Google and Pinterest sacrifice value they provide to extract more revenue from their users.
Care to elaborate on the insane part? I feel sane.
Real answer?

Because helpful technologists are the interface between less knowledgeable users and services like Google. We’re the power users and “mavens” who are both highly exposed to the product and have some understanding of what’s going wrong and how it might be made better. We also spend a lot more time talking to that product’s engineering staff than the typical user (see e.g. the fact that we’re here on HN.)

When helpful technologists opt out of the problematic aspects of a product by uttering some magic incantation, we essentially remove ourselves from having to be annoyed by the product’s rough edges. This means we’re less likely to help improve the service for everyone, and we’re probably more likely to incorporate similar carelessness into the things we make ourselves.

How could a technologist, who's not in charge of the $BigTech be able to fix the problem? They can't. So, they offer the user a work-around. Then, the user never complains to $BigCo because of the Help and still nothing gets done. And later, we complain that we're not fixing $BigCo and we're all mad at "the help".

How, really, could you improve the Google service for everyone?

Acausal decision theory, given the assumption that you have the same decision-making algorithm as the technologists in charge of $BigTech. (The assumption's completely invalid, so it doesn't actually work.)
How many Google Search* engineers saw this article on HN? I guarantee the number is greater than zero.

* Or people who talk to Search engineers regularly, etc.

I see your point. Working around abusive behaviour, and accepting that as a solution, just enables further abuse. I don't think we can influence said engineers here on HN. I know for a fact that I personally wouldn't be influenced, and I also think that if the product managers will want something, then it will get done, no matter what people write here.
"Programmer" here is wrong. Because we are all user of the product "google search".

But yes I see your point. I call it the "permanent workaround"

And there's always this kind of response. They didn't say the service was fine, they just provided a simple workaround which doesn't require forcing a change to Google's algorithm. Hoping they don't help with this seems odd.

So, to counter, I'm glad that they did say this as it regularly annoys me too.

It's indeed a useful tip - and no offense to the GP - but this kind of reply always feels like the tech support equivalent of a mandatory arbitration clause: like an attempt to solve an issue individually with each person reporting it instead of giving a systemic solution that would benefit anyone. The blame is shifted from devs providing a good service to individual users not knowing the latest workaround.
(comment deleted)
Most probably the devs know how to do it but specs written by someone else told them to obfuscate useful functionality.
But no one said that there’s nothing wrong with the service. They merely offered a very simple workaround. I can’t imagine why you’d hope for that to stop happening.
(comment deleted)
Google: we need your personal information so we can improve your experience

User: -pinterest

Google: we need your personal information so we can improve your experience

User: -pinterest!!

et cetera, ad infinitum ...

I think your premise is wrong. The service isn't broken. I'd assume Google has enough people and data to know that it's doing what most people want. They were even kind enough to put in a workaround for the minority for whom it doesn't work well (which appears to be OP).
Most people want to have a full result page of nothing but login walls when searching for any kind of image?
Most people who click on Pinterest links are already logged into Pinterest...
There’s no way that’s true. I’d be surprised if even 1% of the Google users paid attention to the source site underneath the image.

And you’d think people with a Pinterest account would opt to search on their website/app rather than going through Google (thus making it even more unlikely that the Google user has a Pinterest account).

Pinterest has a huge user base. They don't need to look at the source if they have an account; the link will work fine.

And, no, most people don't bother picking a site before searching. Why would they artificially limit their results? People don't even visit amazon before searching for Amazon products.

Pinterest’s user-base is surely minuscule compared to the amount of users using Google Image Search.

I’m not saying that 100% of all users with a Pinterest account use their website/app, but many of them will of course opt to use Pinterest’s website/app for their image searches, which means an even smaller percentage of the Google Image Search users will have a Pinterest account.

Not just Image Search - Pinterest often dominates Text Search, and people here are "Oh well, they figured out how to deliver what Google wants!"
But not most people doing a Google search. Anyone who is not on Pinterest has its result page spammed with useless links.
For the amount of people hired you would assume they had the best people.

But their hiring process is broken. They randomly hire a % of people who pass an algorithm test. It's so random that 70% of their workforce wouldn't be hired again if they had to go through that process.

The brand is so strong that people think anyone who makes it into google must be the best of the best. It's more like a random person who has studied leetcode.

User: your thing doesn't work the specific way I want. I haven't read the manual and refuse to learn now to use its features. You should make it magically work the way I imagine.

Multiply by 100 users with different opinions, all entitled, on how thing should work.

Why? This is a quick fix. Maybe the problem will get fixed "officially" someday. But I have the problem now, I need a workaround now, not in 5 years.
It would be great if Google had a little "hide this domain for me permanently" next to every search result. Not only would it improve that user's own search results, it's a very strong signal that other users also might not like it, for de-ranking purposes.
They used to offer that functionality but removed it years ago.

I've been using this userscript since then. https://greasyfork.org/en/scripts/1682-google-hit-hider-by-d...

It might be called Google Hit Hider by Domain but the author has expanded it to cover most major search engines over the years.

I've used a similar script; the problem is, for certain searches, literally every result is a pinterest link, and you end up with a blank page.
I just wanted to mention that excluding strings has been broken for me off-and-on over the last month at least.

Not directly relevant to anything except how lame google has become.

Only a month? I don't recall the last time it actually worked. I always end up having to do -"string" and put it in quotation marks, kind of like telling Google "why, yes, this thing I put that minus sign in front of really is something I don't want to see in my results."
-site:pinterest.* (there's supposed to be an asterisk at the end, but I have to type something after the asterisk for it to show up)
Stop using Google search. DDG works just as well
Would that this were true, but it's not. Searching for specific technical issues especially shows DDG's weakness.
Outside of searching for specific technical issues, DDG is pretty good. I use DDG for most of my searches and this is the first I've heard of search results being dominated by Pinterest, so it's at least better on that front.
I use DDG search most of the time, but I find this to be untrue. DDG is pretty good, but struggles with context a lot more than Google does. For example, when searching for an error in a programming library, DDG is likely to return the library homepage, whereas google is more likely to return the specific github issue where the error is being discussed.
For text it depends [1] but for images I find DDG to range from slightly to significantly better than Google. Also, I have direct access to the image file URL without having to rely on browser extensions.

[1] I wonder if the meme "DDG is not that good" is so prevalent on Hacker News because it sucks for programming stuff.

Yes, Google image searches produce wild results. Far-right groups have had a field day with it.
Agreed. I switched to DDG a few weeks ago as a default. I appreciate the privacy, but for some categories (like tech questions) the results are just... not as accurate. For general queries they're comparable. For searching specific issues, Google is more likely to show relevant discussion threads of the exact issue.
This used to be true. Regularly I had to use the !g bang to find what I was looking for. Lately, I've realised that I hardly use it anymore and I do results are far worse on Google. Your mileage may vary of course but I don't bother with Google anymore.
Same. I went the DDG route a couple of years ago, and searching tech topics made me constantly do g!. And a few other things. Suddenly, Google's results go so much worse, that the duckduckgo offering just proved to be actually functional by comparison. And I think it actually has improved over the years. It's now my default browser search and I'm comfortable not having to worry about Google capturing every feeling or thought I have when I google something, and storing that for eternity.

Not to mention the awful vendor blogs and stores and secretive marketing clickbait masquerading as a neutral information totally dominating Google's page 1 and 2 of results.

I’ve been using DDG as my default for years and it really fails to provide me helpful results for programming and Linux questions. I know this because I frustratingly have to add !g to queries all the time lately and I’m questioning my choice to default to DDG.

That said, I don’t like being tracked. It’s not clear to me if the quality of results I am seeing are due to Google’s tracking or not. (I suppose I could research this but I’ve not done so.)

It has improved for me to the point I don't use !g anymore. Maybe subconciously I now do queries that fit DDG better. Anyway, I get the answer I want so can now live without Google.
That's problem, DDG works nearly exactly the same (except a bit worse imo). DDG channels a lot of Bing, which virtually clones Google which makes it not much of an alternative at all.
>Google should be asking from a product perspective if they are providing the search results the user is most likely seeking.

I am cynical. Maybe Google's own experiments show that users click more ads when they get frustrated of the Pinterest spam that is technically not spam enough to be removed from the index.

What the users are going to do? Use Bing?

Google's most optimised version is probably being an Ads search engine where you are presented only with the most lucrative ads that are relative to your query. Essentially Yellow pages.

It's certainly going in that direction. With reverse image search being replaced with "Search with Google Lense" on mobile, the most prominent results are now all Google shopping ads.
Bing is better than google often enough. I've stopped using Google.
>What the users are going to do? Use Bing?

duckduckgo isn't horrific. And certainly not for image searches, which is how Pinterest's pollution ends up being noticed.

I had an experience the other day where duckduckgo worked where google had failed (an obscure java default bug). It was pleasantly surprising.
If that's the idea, isn't it easier to just show random junk? Why pinterest?
(comment deleted)
With Google featuring progressively more ads and Google content at the top of the results page, it's not clear if it's still in Google's self interest to give users the best results anymore.
This is why there's an aggressive firewall between the ads and search organizations. The ranking team isn't even aware of revenue numbers by ACL.
I'm sure the search team can, erm,google for the ad revenue figures in the earnings release.

After reading the anti trust filing it’s hard to argue Google is anything else but evil and a lot of what it does is to hide its monopoly.

You think the search team runs an experiment once per 3 months to test how their algorithm ideas impact earnings?
The company isn't a charity. If a search change demonstrably has a negative effect on company earnings (even if users like it) it will almost certainly be reversed. I don't think they run an experiment once every three months, I think they are testing continuously, including A/B testing (some users get the change, others don't,).
> If a search change demonstrably has a negative effect on company earnings (even if users like it) it will almost certainly be reversed.

This is completely false. I've personally been involved in launching ranking changes that negatively impacted revenue substantially. I didn't find out until months later through the grapevine, and only in vague terms like "ads had to scramble." Versions of them remain in production today. Revenue projections were never considered as part of the launch decision, nor have they they been consulted as part of evaluating subsequent versions.

Perhaps this has gone too far, as others have noted, there are so many ads and Google generated content (esp. YouTube links) that the organic search results are practically on the second page.

If Googlers were allowed to consider revenue in the search results perhaps they wouldn't have been so easily down graded to a second class citizen.

You can find public examples where the opposite has been done.

https://en.wikipedia.org/wiki/Google_Panda

> In 2016, Matt Cutts, Google's head of webspam at the time of the Panda update, commented that "with Panda, Google took a big enough revenue hit via some partners that Google actually needed to disclose Panda as a material impact on an earnings call. But I believe it was the right decision to launch Panda, both for the long-term trust of our users and for a better ecosystem for publishers."

Disclosure I work at Google.

Apropos of anything else, Google is famous for a "monorepo". Any team can see the underlying architecture of things, regardless of any supposed "aggressive firewall" alluded to above.
A monorepo does not let you deduce the revenue consequences of experiments, but regardless, even if you could figure it out, it isn't part of the incentive structure.
I’m... skeptical about this firewall
I doubt the ad organization and ranking teams decide where and how many sponsored listings appear in search results, which I believe is GP's point: the page design/layout and the proportions of ads vs. (more or less) organic listings. Someone decides that searches have e.g. four ads at the top and two at the bottom.
If you believe this I would love to sell you an extremely expensive bridge.
Yeah, it’s honestly time for a new search experience
If ever.

Well to be fair, probably the engineers want to on some level, but there is a business mandate to not (and there always has been).

This is a little bit like: 'McDonald's should serve healthier food, like Salads'.

'What Google Wants' and 'What We Want' have reached a crude equilibrium and it's unlikely to change that much, even if there are obvious improvements to make.

Qualitative improvements beyond need generally require a very specific corporate and cultural focus. If Sergei or Larry 'came back' and mandated it, the system would move around it. Or Sundar could, maybe, pull that off, but it's not in his DNA really.

The lack of material innovation in search stopped over a decade ago.

Otherwise, we should be getting betas every few months, with all sorts of options, features, etc. Even simple things like 'forever blocking a site by default' would be great.

I think they can't do anything about it. Regulators and lawmakers in the US and EU would be very receptive to complaints from pinterest if google were to act somehow.
Do you have references for this? I'm genuinely curious if this kind of thing is actually being done. Google's altered it's algorithm to remove low quality sites in the past and if anti-trust worries are stopping that now, well, I can't even.
Yeah, Pinterest is almost always disappointing as a result: their account requirements are annoying and it tends to obfuscate the source of the image or information you’re looking for.
A few times I've landed on Pinterest and they lock user copied (contrary to copyright law in my country AFAICT) content behind a registration-wall. They never show enough utility for new to register, they just tease on Google and then don't show what they teased.

Google results would be better for me with all Pinterest results removed.

Wasn’t there an option at one time to tell Google to hide results from a certain domain? I feel like I haven’t seen that in years though
Just add `-site:example.org` to your query.
I have done this throughout COVID for the NYTimes' content, as it's behind a paywall.
(comment deleted)
Interestingly, if you use Brave, it lets you hop to the tor versions of sites, and since NYTimes has a tor version, if you visit that, it’s paywall free.
I use Brave. How do I hop to the Tor version of a site?
If you go to a site that advertises (via HTTP header) that it has a Tor version, you'll get an "Open in Tor" button at the end of the address bar.

Examples are nytimes.com and dw.com

It still works

-site:pinterest.*

Even with wildcard tld, otherwise it is still all Pinterest

But it is interesting that those kind of Google/search engine feature hardly ever get talked about these days whereas in the past they were considered crucial to get decent results.

I don't know what the pathway for this is but just -pinterest does it for me.
-pinterest removes pages that contain the term pinterest, so pages from a pinterest.* domain, plus articles about pinterest, etc.
Sounds pretty good to me. If I'm searching for a term, but don't want results from pintrest, then I'm probably not wanting to read an article about pintrest either.
"Here's a picture I took today. I also posted it on Instagram and Pintrest."
If you wanted to read an article about Pinterest, it would probably be hard to search for it in any other way.
Except that the word "pinterest", like pinterest.* results, appears in all the nooks and crannies of the rest of the Web. Excluding the domain like that succeeds in narrowing the scope of the search without excluding content unduly.
I use that a lot. Wouldn't Google notice if a lot of people were telling it they don't want to see results from Pinterest and make some adjustments to their SERPs?
That's a nice hack, thanks! I've just modified my Google search engine entry in Chrome to include this.
It was too effective.
You can hide sites with a flag in the query. No idea if it works on image search but I assume it does.

    -site:pinterest.com
I don't get why Google doesn't give an option to permanently remove or downgrade certain domains, straight from the results page. There are results I am never interested in. It could also work as a further signal for their ranking, if used with some caution.
Probably Google would do good to give up total control and obscurity and put people in the driving seat of search customization. Maybe that would even save them from the anti-trust push (one indexing and retrieval back end + multiple ranking, filtering and UI front ends).

But I'm wondering if this would not be better implemented at browser level. Can we have the white/black list run locally in the browser, re-ranking results on web pages and hiding what we don't want to see? Not just on Google, but also on the rest of the web, like ad blocking extensions.

There use to be an easy way to configure your account to always block certain domains, but that's gone now, so you have to use -site per-query. Maybe there are browser plugins that help? I miss the old functionality. I only had to block about a dozen sites to get great search results. Maybe it'd be less viable now; I'd probably have to block hundreds or thousands of sites, and they'd keep breeding new ones.
You used to be able to tweak part of the URL in the Firefox google search thing, and append queries I thought so as well.
Just make a search that excludes all the domains you despise. Don't include any search terms or just a dummy one.

Then save it as a bookmark.

e.g.

  "-site:foo.* -site:bar.* ..."

Use that bookmark as your search engine: click on it, then fill in search terms.
(comment deleted)
The irritating part is that Google used to penalize websites dramatically if the user couldn't get to the content but the search engine could.

I guess that interfered with Google extracting ad money.

I have an account there and I use it but the interface is atrocious and they seem to actively be making it worse.

Recently they removed the facility to, when saving a pin, to type the name of the board to find it, now having to scroll through an alphabetical listing.

Before that, they also redesigned the website such that a refresh of the page was required in order to save a pin to two different boards, an action they explicitly support.

The website is also a slow resource hog.

Of course, it comes with the tried and tested “corporate morality” where there are rules against various things that are phrased in terms of moral concerns that are really about keeping their app on the Apple store — this naturally includes vague standards on “nudity” that are not worded in a gendered way, but there is an unofficial code that everyone knows that it very much is.

The most annoying moral preaching I encountered was that I searched for “boy's love” once — the genre of male–male tragic romantic fiction, that got me a preachy message about pædophilia that made me suspect my i.p. address was probably passed on to some U.S.A. authorities. It's really quite a big genre and has nothing to do with pædophilia.

They only recently added a way to search one's own's pins, and it's very lacking and doesn't reliably even catch words in the descriptions thereof. Sometimes it does; sometimes it doesn't.

What can be said about it is that the algorithm for finding related things is generally quite good — the interesting thing is that it's actually better at finding the aforementioned boy's love fiction than most of the websites that actually host it.

The problem is that there is no real competitor.

> I have an account there and I use it but the interface is atrocious and they seem to actively be making it worse.

If the user is confused on how to do something then it keeps them on the page longer and pushes the metrics up

taps on head

The absolute worst part of pinterest is the the time it takes to respond to ctl-w. It's slower than just about every other site.
Yeah, the results should definitely contain a HEAVY waiting for accessibility by the general user and most people don't have a pinterest account nor do they want one.
Yes, Ointerest and Quora are types of site that Google should really punish hard. Account requirements is something that should move sites outside the few top pages...
Both in Google, and in DDG, sometimes when I am looking for an image (i.e. image of a "full body workout plan") the images I find tend to be 'hosted' in Pinterest, but in reality following them links take me to Pinterest, and then back to the original source (which is some random fitness site).

I find Pinterest useful as an image search engine.

Google has shown time and time again they do not worry about providing the search results the user is most likely seeking. They monitor ad revenue and adjust accordingly.
They are playing a cat and mouse game, only the mouse is quite determined as its bonuses depend on evading, and the cat is very fat and comfortable :).
> Google should be asking from a product perspective if they are providing the search results the user is most likely seeking.

I never want Pinterest. I don't have an account. I don't want one. I will never want one.

At least for image search, I use a Firefox extension called Unpinterested. I'm very happy with my search results now. Note: I had to toggle an option to make it work only for image search, because it borked other Google results.

I remember when Google added a functionality to block domains entirely from your results. They removed it soon after though; I'm guessing some big advertiser was blacklisted many times.

It is basically an arms race between search engines (specifically google) and SEO. Google does change their algorithms, but with enough effort (and money) SEO proffesionals can figure out how to tweak their sites to keep gaming the system. And the casualties of this war are the authentic sites that don't spend a ton of money on SEO.
"That means I spend less time using google, and that in theory means less money for google."

In practice, losing customers means less money for Google. Pinterest is a large Google customer. As individual users, we are very tiny bits of a very large target, but we are not customers.

I think Google collects clickthorughs on the search result page and thus knows how people react to its responses. It also knows when the user is quickly returning to search after checking out a link. Pinterest is probably appreciated by people, probably of a different demographic than you.
Here is some background on this:

Recently, it has been revealed that Pinterest reverse-image searches Google for uploaded images and adds the Google query result to the Pinterest image page for SEO: https://www.rankscience.com/blog/pinterest-image-seo-growth-...

This degrades quality of Google image search, for example this Reddit post on removing Pinterest got 77k upvotes:

Reddit r/LifeProTips to remove Pinterest from Google searches: https://www.reddit.com/r/LifeProTips/comments/hejbl0/lpt_whe...

Earlier Reddit petition to Google to remove Pinterest from search results (66k upvotes): https://www.reddit.com/r/google/comments/85atho/pinterest_ne...

Hacker News discussion of Pinterest's Google scraping: https://news.ycombinator.com/item?id=23701998

Earlier discussion on HackerNews (1.1k upvotes): https://news.ycombinator.com/item?id=16613996

"Unpinterested" Chrome extension to combat Pinterest spam (10k+ downloads): https://chrome.google.com/webstore/detail/unpinterested/gefa...

I agree, but if you take uninformed users (or users at random, that don't subscribe to a Google related forum) they would be happy with the Pinterest results because the results look nice at first sight and they don't have time to learn about the problem. Google using this signal would conclude that Pinterest is OK for the users.
> I seriously doubt that most pinterest results are what users are seeking.

Well, yeah, since even if you click through Pinterest won't actually show you the content.

Isn't that supposed to disqualify them from appearing in search results at all?

Instead of playing cat and mouse with Pinterest, what would be a much better idea is to let users themselves decide what's important to them. Let us vote search results up or down, and let us block sites we don't like yet keep showing up. Those two changes would do a lot to improve user experience.
Still odd that Google tolerates this. Isn't this exactly the kind of black-hat SEO that they are usually downranking or banning sites for?

Pinterest landing pages are intentionally unusable for anyone without an account. It's hard to make a good-faith argument that those pages are the "most relevant result" for someone making a simple image search.

Yep, exactly. I find that Pinterest is slowly making me less and less likely to use Google. Pinterest shows up as the dominating search result even or very specific long tail keywords where one could argue that a collection of images on a login-gated page is not helpful at all.
I don't use google for images anymore because of this.

Bing is terrible, but it's better than google.

Which means that google is worse than terrible.

Yandex is great for images these days.
> Isn't this exactly the kind of black-hat SEO that they are usually downranking or banning sites for?

How is it blackhat to do exactly what Google wants?

Using tricks at the detriment of the user to the benefit of the service is blackhat. Pinterest is doing these tricks to their benefit at the extreme detriment to everyone but themselves.
It's weird to blame Pinterest for Google's choice to promote the site. Sure Pinterest wants to rank highly, but they aren't tricking Google. Google knows how popular they choose to make Pinterest results, and how much most users like those results.
If I make an algorithm to rank sites, and you figure out the internals of that algo and manipulate your behavior to rank at the top, that's bad and not the same as being Google's choice, at least not directly.
My friend, Google tells you how this works. Pinterest doesn't have secret knowledge.
What they are doing to get into google search is not the problem, the problem is all their links are scams, clicking on pinterest google images link lands you on a login screen, or on a farm of similar/adjacent images. Nowhere do they have the content you were actually looking for.

TLDR: they are gaming google

It is there— it usually just takes pages of scrolling. I heard a 2nd-hand account which said their product team leadership knows this annoys the hell out of people, know it's an easy fix, but they do it anyway. If that's true, I'd guess they're ignoring Goodhart's Law with their engagement metrics and need to re-focus on their UX, though I don't have any first-hand insight into it.
Not if you don't have Pininterest account. then you can't scroll more than half a page.
One could say they are gaming it
The Wikipedia clones gamed it and they were deranked. Why does Pinterest get special treatment?
I'm guessing because Pinterest buys ads from Google? They're the customers from Google's PoV.
I always thought that SEO errors should boost results to punish content factories.

Grandmas cookies are going to be better than 24 years old copypaste cookies for big bad marketing company.

pinterest is clearly being spared seo penalty so one can only assume pro-pinterest bias at google
"There's no intentional or manual effort at Google to promote pinterest."

But there is an effort at Pinterest to promote itself via Google. Pinterest pays Google for online ad aservices. Pinterest is a Google customer. The user is not.

The bottom line at Google is not driven by showing users the "best" results for users. The bottom line is driven by customers such as Pinterest who pay Google to assist them with online advertising.

Page and Brin identified this conflict of interest in their 1998 paper introducing Google, which was originally intended to be an academic endeavor, not to become an online ad services company.^1 According to the authors, (a) the interests of users and advertisers are not one in the same, (b) there are inherent effects on search results if the search engine is funded by advertising, and (c) the behind-the-scenes operation ("technical details") of search engines funded by advertising become largely non-transparent to users/competitors (a "black art").

I reckon (c) is why we get questions like this one from the OP.

I have also seen these inexplicable and useless Pinterest links in Google searches.

1. https://research.google/pubs/pub334.pdf

Quotes:

"To make matters worse, some advertisers attempt to gain people's attention by taking measures meant to mislead automated search engines."

"Up until now most search engine development has gone on at companies with little publication of technical details. This causes search engine technology to remain largely a black art and to be advertising oriented (see Appendix A). With Google, we have a strong goal to push more development and understanding into the academic realm."

"Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users."

"For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."

"Furthermore, advertising income often provides an incentive to provide poor quality search results."

"But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm."

Had anyone ever presented evidence that advertisers manipulate organic results? Former employees blow all sorts of whistles. Why has this one never blown?
The term "black art" appears to be a synonym for "black magic".

Wikipedia (https://en.wikipedia.org/wiki/Black_magic) describes black magic as "the use of supernatural powers or magic for evil and selfish purposes."^1

1. J. Gordon Melton, ed. (2001). "Black Magic". Encyclopedia of Occultism & Parapsychology. Vol 1: A-L (Fifth ed.). Gale Research Inc. ISBN 0-8103-9488-X.

Are we using different googles?
I see Pinterest appearing disproportionately in image search.
I practically never see Pinterest in the results. I suppose it really depends on what you are searching for.
We might be doing very different queries :) If you search for makeup related content, Pinterest comes up a lot because people tend to pin and curate makeup related stuff there
If lots of people have spent time curating the Pinterest content, and searchers are likely to be looking for curated content, doesn't it make sense that Pinterest would rank highly in the search results?
Pinterest spends a large percentage of their budget on gaming the search page.

Though I’m sure you’ve noticed in recent years that Google has begun to replace many search results with answers directly from them instead of redirecting you to places like Pinterest. For the better I say. Companies like Pinterest are parasitic and degrade the overall ecosystem of search.

Yeah - I've just been reading some of the Pinterest engineering blogs linked above about their sophisticated SEO experimentation framework and ugh, I think it's time to load up my browser with extensions or move away from google completely.

Image search has been borked due to Pinterest results for a few years now, but now my text queries have also started being dominated more and more often by their search results. I'm bummed that Google doesn't count the different Pinterest country domains as duplicates and remove them. instead there is like pages upon pages of the same Pinterest result.

> Google has begun to replace many search results with answers directly from them

> Companies like Pinterest are parasitic

Surely you mean companies like Google are parasitic?

Pinterest doesn't provide value with their image search spam hence they are parasites indeed.
I see my sarcasm didn't go over well. Replacing search results with quips from websites is parasitic as fuck.
There’s a great Chrome extension that will remove those from search results.

  google.*##a[href*=".pinterest."]:upward:(1)

  google.*##.g:has(a[href*=".pinterest."])
Add to uBlock Origin. in the "My Filters" tab. This will completely block pinterest. No need to use more extensions that can access your cookies and history.
Another example about how google is broken. It has great difficulty distinguishing noise from signal.
It's also another example of how modern tech companies are broken (or lacking ethics).

Currently, many big tech companies do basically whatever they want. Facebook and other ad companies have been caught knowingly defrauding ad spend customers, Amazon has been knowingly permitting counterfeit sales for years, Uber has been ignoring regional laws globally, Airbnb was (and may still be) ignoring or hiding data that illustrates their connection with increased housing pressures, and on and on.

Doing what you can get away with (because the cost of the penalty is lower than the profit that results from the bad behavior) is not ok. It doesn't contribute to an improved civilization, it pushes civilization back down toward a zero-sum, selfish survivalist mentality. I know it sounds corny or hippie, but if people can try to better balance their own desires against the good of the community, the end result will surely be better.

The economy is meant to be built so that anything harmful to the rest of society is either unprofitable, or criminal (i.e. theft).

Obviously we have a new frontier of digital stuff, but the law and economic incentives are still ignorant of it.

There are plenty in power now who would disagree with your definition of economy.

Sometimes law and policy is ignorant of technical concerns, but quite often it is aware and simply has a more selfish agenda. Prime example of this is modern US copyright and patent laws and enforcement.

As long as elected officials have to raise funds to campaign and win, elected officials will be beholden to the people who financed them. If politicians didn't write special laws and loopholes for corporate special interests, they wouldn't last more than one term. They would be replaced by the next challenger who would take special interest money and then "pay it back" via special laws.

Is it really difficulty, or indifference? I think they could do better, but as a near-monopoly in that space they'd rather spend the money elsewhere.
Sounds like something Yahoo might have said when they declined to by Google. Maybe it’s a good time to start a search engine.

  google.*##.g:has(a[href*=".pinterest."])

  google.*##a[href*=".pinterest."]:nth-ancestor(1)
Add to uBlock Origin. in the "My Filters" tab. This will completely block pinterest
Unfortunately this did not work for me. I added these two, and then I did a search on images.google.com for "custom carved door frame".

5 of 7 images on the first row of results was some pinterest site.

Can you try this?

  google.*##.g:has(a[href*=".pinterest."])

  google.*##a[href*=".pinterest."]:nth-ancestor(1)
Thank you that works, eliminated Pinterest results.
Thank you so much! A very quick copy paste into uBlock and it works wonders for Google & Google Image searches.

I wish I had looked into this before! :-)

ffpip is the real Santa
Many thanks, this does work. I do see a momentary flash of pinterest results, but they quickly disappear (and subsequent matches are re-tiled up). Good enough!
Yeah. uBlock Origin is just hiding the results, like it hides blank spaces after it blocks ads. You might notice a momentary flash only for the first few times, or on slower machines.
Weird, I don't do anything at all to block pinterest, and I just did that same search on Google Image search, and pinterest results are 6 out of 20 on the first 3 rows, and 1 per row at most for all of the rows below that.
I miss the days when google had an option to block entire domains. So sad that they removed it.

    -indomain:pinterest
As a configuration, not per search.
I used to have a .txt file full of - options for google back in 2009 or so.

Then Google started ignoring more and more of what I searched for and they also started blocking the worst offenders so I forgot about it.

These days I mostly use DuckDuckGo for my programming searches.

Same here. Although do they have a way to block certain sites?
One big reason why Pinterest ranks so highly, and why you are seeing different tld's in the results, is because they have a variety of tld's that they use and those tld's all have separate crawl budgets from each other. So Google crawls Pinterest about an order of magnitude more than most other sites. You can read more about this here: https://medium.com/pinterest-engineering/how-switching-our-d...

Source: I run a think tank focused on Google’s web crawling advantage and have been studying stuff like this for a couple years now.

Does your think tank come up with any ideas on how to keep companies like Pinterest from peeing in the pool so to speak? They made it impossible to find DIY craft stuff that I used to search for, and I can’t imagine it’s the only site doing it.
We are pursuing ideas that we think will increase competition in the search engine market. Our main focus right now is making Google’s index of the web available for use by competitors, since no one other than Google is really allowed to crawl the web. If people were allowed to use the index to build what they want, then you or somebody else would be able to build a search engine that avoids the Pinterest cruft and would be more suited to your needs, instead of Google’s one size fits all approach. So, to answer your question directly, yes we do have some ideas about how to do this.
> making Google’s index of the web available for use by competitors

What incentive would Google have to continue populating that index?

Would I be breaking the law if I independently crawled and hosted an index without publishing an API for it?

> since no one other than Google is really allowed to crawl the web

Maybe this is the problem that needs solving.

> What incentive would Google have to continue populating that index?

Presumably they would still want to run google.com and make money off of it.

> Would I be breaking the law if I independently crawled and hosted an index without publishing an API for it?

No. You would not get the advantage that Google gets when it crawls the web and so would not have access to a large amount of data that nobody else has access to.

Updated based on edit of parent post:

> Maybe this is the problem that needs solving.

Why have websites waste the money to serve all those requests all over again? Why don't we have Google share the results and we can use that money to do more productive things than recreating that work? I don't think website operators would be happy if there were a hundred more crawlers out there crawling as much as Google does now.

(comment deleted)
Do any site operators actually block non-Google search engine crawlers because being listed DDG/Bing/etc isn't worth the extra cost of serving the crawler? It sound a bit ridiculous unless they actually don't want to be found. Maybe they only allow GoogleBot because that's all they thought of and the extra cost is in researching what all the other search engines call theirs.

Perhaps other search engines should spoof GoogleBot. Browsers have being doing that since forever spoofing Netscape (Mozilla), Safari, etc. for the same reason.

> Why don't we have Google share the results and we can use that money to do more productive things than recreating that work?

This sounds like a common fallacy of people criticizing the free market. Duplicated effort looks wasteful but turns out to be far more productive than the lack of incentive that comes with not being able to profit from your work/investment.

I would think the site owner’s cost of being indexed is the same for every search engine that indexes the site.

The benefit varies with the quality of the search engines, and that will vary between search engines, but it does get larger the more a search engine is used, so a cost/benefits analysis may show Google and a few other large ones are the only ones worth supporting.

> Do any site operators actually block non-Google search engine crawlers because being listed DDG/Bing/etc isn't worth the extra cost of serving the crawler?

Many website operators do actually block crawlers from non Google search engines and it's because the cost of being crawled isn't worth it to them. Here's a good quote from one such webmaster:

    As a webmaster I get a bit tired of constantly having to deal with the startup crawler du jour.

    From law firms looking for DMCA violations to verticals search engines, to image aggregators, to company intelligence resellers… It feels to me that everybody and their brother has gotten into spidering sites.

    With 10,000s of pages that have content that is only relevant to a targeted audience who is perfectly able to find us on the majors, I do not hesitate to block (and possibly ban) when I see an aggressive crawler that does not provide me or my customers with direct benefits.
Taken from http://www.skrenta.com/2008/04/cuill_is_banned_on_10000_site...

> Perhaps other search engines should spoof GoogleBot. Browsers have being doing that since forever spoofing Netscape (Mozilla), Safari, etc. for the same reason.

People have tried this and it doesn't work. Google provides ways to check to make sure traffic is coming from Google IP addresses and practitioners and academics study how to spot fake Googlebots. https://developers.google.com/search/docs/advanced/crawling/... https://blogs.akamai.com/2014/07/search-engine-impersonation... https://ieeexplore.ieee.org/document/8421894

> This sounds like a common fallacy of people criticizing the free market.

I am asserting that crawling the web is a natural monopoly. This means that the free market has failed and that it is not possible for the market to heal itself in this regard. There is significant evidence that this is the case and I imagine you'll be hearing more and more about it soon.

(comment deleted)
> What incentive would Google have to continue populating that index?

Not going to jail, presumably.

> since no one other than Google is really allowed to crawl the web.

??

There are two main reasons why I say nobody besides Google is really allowed to crawl the web.

The first is that Google gets much more access to pages on websites than everybody else. You can see this by examining the robots.txt files of various websites[0]. I've been doing this for several years now and Google has a consistent advantage across many thousands websites that I've looked at. This adds up to a significant advatnage and many search engine operators complain about how it hampers their ability to compete with Google[1].

The second is that Google gets to ignore crawl delay directive in robots.txt while other search engines don't[2]. Website operators cannot tell Google how fast they want their website crawled, they can only request that Google slow down. If another search engine tried to do what Google does, they would likely be blocked by many important websites.

If you would like to read more about this, please checkout https://knuckleheads.club/

[0] https://pdf.sciencedirectassets.com/robots.txt

[1] https://www.nytimes.com/2020/12/14/technology/how-google-dom...

[2] https://www.seroundtable.com/google-noindex-in-robots-txt-de...

So, uh, don't respect robots.txt in your search engine? It's not like there's a law that you have to, and that you can't pretend you are Googlebot. The only real obstacle I can imagine is that some firewalls might be configured to be more permissive with traffic originating from Google subnets.
You would be blocked fairly quickly by many website operators and no longer able to access those websites if you straight up ignored robots.txt files. You also might even end up being served cease and desists by some websites and sued if you continue to persist and try to find ways around it.
And what if you do respect it but follow Googlebot rules?
Applebot was able to get away with doing exactly this but I imagine that's because it's Apple and websites knew that Apple was about to send them enough traffic via Apple News to make it worth their while. I don't know if other search engine operators have tried this but I would imagine they would get caught by rate limiters set for non Google IP's and then they would be blocked.
Still, you keep saying all that as if most websites even notice that they're being crawled, and that their operators are very aware exactly when by whom they're crawled. Like as if the admin gets a notification every time a crawler comes by or something, with precise details about it. I don't think it's nearly as serious as you're trying to make it look.
I've been a part of a team that operated a large website and I've been paged before because of the issues that somebody was causing because it was being crawled too much. Many people in the web operations field have had the same experience. Generally speaking, the larger the website, the more sensitive they are about who is crawling and why.
To add another data point for you: I have had one of my websites brought down by Yandex bots before. There are also dozens of no-name bots (often SEO tools like ahrefs, semrush, etc.) that can sometimes cause troubles.

For me it was a problem of having lots of pages, and having a high cost per request (due to the type of website it was).

For other websites, it is not necessarily about the volume of traffic from bots, but the risk of web scrapers getting their proprietary data. They're fine with Google scraping their info because that's where their traffic comes from. They're not okay with some random bot scraping them because it could be taking their content and republishing it, or scraping user profile data, or using it for some nefarious/competitive purpose.

> the risk of web scrapers getting their proprietary data

That's some weird logic, to me at least. That data is literally given away to everyone but some people or organizations can't have it? If you want to control access to it, maybe at least require people to register before they can see it? Is it even proprietary if it's public with no access control whatsoever?

This for-profit internet is just really such a parallel universe to me.

> This for-profit internet is just really such a parallel universe to me.

I know I have been a contrary commentor in this thread, but I hear you with this. What a monster we have built, and what always gets me is how trivial everything is. So much capital is flowing through these ephemeral software systems that, if gone tomorrow, would be ultimately inconsequential to mankind.

I mean it's ridiculous to think about it, but there's this giant, many-billion-dollar online marketing industry that I essentially don't exist for. If it's gone tomorrow, I would indeed not notice, but it'd be the end of the world for some.

> and what always gets me is how trivial everything is

Whenever I read about corporations and how they work, I always inevitably ask myself the question "where the hell does enough work to keep this many people busy even come from". Everything is ridiculously overengineered to meet imaginary deadlines.

> That data is literally given away to everyone but some people or organizations can't have it?

It's often a question of quantity. LinkedIn probably doesn't care about you scraping a few profiles, but if you're harvesting every bit of their publicly-available data, then they get a little scared that you're building something that's going to compete with them.

Same with Instagram, or Facebook, for example. Though in this case it's probably more of a user-privacy issue - at least that's what they say.

It's not really weird logic to me - seems to make sense.

> If you want to control access to it, maybe at least require people to register

Most of the time they can't do this because they need the Google traffic. LinkedIn wants a result in the SERP for Bob Smith when you search for "Bob Smith" because that helps them get signups. Google won't list the page if that content is gated by a sign-in/register page.

> Like as if the admin gets a notification every time a crawler comes by or something, with precise details about it.

This is exactly how it works with user-agents and your naivety here is shining bright.

There are syndicated blacklists that get fed into automatic traffic filters. Not to mention a surprising amount of the web is fronted by Cloudflare and other CDNs, making that kind of traffic detection and blocking more effective and widespread than you might expect.
Google tells website operators how to verify google bots in a way that can't be spoofed.
It's a situation where the rules seem obvious but the practical realities of it mean Google has the advantage by being the incumbent. No one would dare block Google for a search traffic reliant business, but some upstart search engine will quickly end up on blacklists even with reasonably slow crawling.
(comment deleted)
> making Google’s index of the web available for use by competitors

That does seem like a good idea since it amounts essentially to a database of stuff that Google does not own (by construction) and is of public utility.

Exactly. Everybody always talks about turning tech companies into public utilities without really explaining what the utility would be. A public index of the web would be an amazing utility for many of the reasons in this thread and would spur on a ton of new innovation and businesses.

If you would like to read more about all this, please checkout https://knuckleheads.club

The pursuit of a public index is excellent.

Some feedback:

- A cost of membership runs contrary to establishing this group, especially at such a high recurring charge.

- I'm not sure what your software/AWS situation looks like, but 20 million robots.txt files acquired from Common Crawl is something I can analyze on my PC. It doesn't seem to presently justify such high costs.

- Prioritize building a mockup index with an intuitive frontend. This is essential for non-technical people to understand

- Exclusively talk with EU legislators (they are motivated, whereas nothing will happen in the US).

Thank you for the feedback!

I think the price for membership dues is reasonable and many people agree evidenced by them signing up. I think I might start a petition that is free to sign up on though, thank you for the inspiration!

It is possible to analyze those files on the pc, it just takes a much longer time. The analysis is an iterative process and so the faster the computers the faster the iterations and process go. I was analyzing them on my pc with python for the first year until it got too slow and my I am using an aws server with some rust and that is going much better. I also need to increase the number of files analyzed by about two orders of magnitude soon as well.

Great idea, very cool. That’s going on the todo list!

And I am going to be reaching out to and speaking with whoever is interested. One of the fun things about this is that it is an international dynamic, with some jurisdictions having abilities that others don’t. For example, the UK CMA has subpoena powers that the US Congress lacks and got a ton of information out of Google and Bing that shocked me. The US has the ability to get the CEO’s to show up to hearings while the UK does not in the same way. Why limit ourselves to one government when there are so many to mix and match from here?

What kind of research is it? Is it just funding you or members can participate? I do Bayesian and non Bayesian data analysis and modeling, would be potentially fun to poke at the data
Right now the research is two parts, analyzing websites robots.txt files for bias and then talking with search engine operators and website operators to get their stories that validate these ideas. Right now I am the main person working on this research but I would like to get the robots.txt parsing and analysis code open sourced soon to allow people to start digging in. Getting people access to the data is trickier, but it feels doable as well. You'd be welcome to join in!
> Our main focus right now is making Google’s index of the web available for use by competitors

How would that work? I'm pretty curious about this if there's anything out there to read.

EDIT: Ah I saw the link to Knuckleheads' Club :P

:) No worries. If you'd like to chat about it more, shoot me an email at zack@knuckleheads.club, I'm obviously very passionate about the subject and love talking about it. There's still a lot more to write and publish, I won't be done with this hobby horse of mine for a while yet.
This I did not know about and I'm surprised they don't get penalised by Google in some way at the display of search results stage. I mean sure, I suppose one can do the -pinterest trick, but it's still a bit weird that Google thinks search results that are pretty much the same site but from different tlds are not spammy. Do you think they will change this to prevent Pinterest from essentially spamming some search queries by using the tld hack?
Live by the SERP updates, die by the SERP updates. It's been at least two years now and Google seems fine with it (until maybe one day they aren't). I know it might annoy folks like you and I to get reposted pinterest images in our results but it might be that Google has done research and found that people like seeing pinterest in images and so that is why they are keeping them around. It's hard to say why they are there until Google says something about it, Google rankings are a black box at times.
I think others here have alluded this but the thing is that once, doing everything you need to do to get listed but actually being crap was enough to get you de-listed. The criteria wasn't "do what Google wants", the criteria was "be good and also help the algorithm determine that you're good".

Only running the right SEO wasn't sufficient then and I certain wish that criteria remained.

Who determines what is the best ranking? Who was defining what was "good"? There isn't some abstract, mathematically verifiable best ranking, a platonic ideal that all search engines should aspire to. Google has always made choices about how to rank that trade off the interests of various groups of users, from the very beginning. The complaints that you and I have about pinterest being ranked so highly come because Google has decided that our desires for how the results should look are not important enough compared to how other, perhaps larger, groups of users want them to look. Maybe Google deranks pinterest tomorrow, but they've made the choice to keep it as such for a couple years now. Whether it's because they think they have better ways to spend their time or because they like it the way it is, it's not really possible for us to know without somebody leaking that information. But, I imagine, there would be a group of people who would be as disappointed as we are elated if Google were to change this.
There isn't some abstract, mathematically verifiable best ranking, a platonic ideal that all search engines should aspire to.

Of course, "quality" applied is a broad term indeed. But there no being a single certain measure of quality doesn't mean there aren't some things that most people would call crap. Google employs a large number of search quality raters and these folks can likely distinguish the terrible from the OK.

Google is far from ideal and getting worse but you seem to be exposing a pure democracy of algorithm conformance, which is guaranteed to result in hot, steaming idiocy.

I think what I am trying to espouse is two things. The first is that there might be a large group of users unknown to you and I who enjoy looking at pinterest in the image results and that is who Google wants to cater to. And the second is not that Google is or should be a "pure democracy of algorithmic conformance" but that Google has, does and always will make choices about what is best and that the definition of best is up to their discretion. They are making choices about who to cater to with their results and justify those choices with their own logic that we will likely never know. One company having the power to be the only one making all those choices isn't anything I would call a democracy.
The first is that there might be a large group of users unknown to you and I who enjoy looking at pinterest in the image results and that is who Google wants to cater to.

The discussion is about Pininterest in text searches. My only objection to Pinterest in image searches is that it's paywalled/login-walled.

But larger issue is I'd acknowledge some search results just aren't what I in particular want but I'd claim other results are actually objectively low quality and you seem to want to push things to realm of pure subjectivity, any old crap is something someone might want and who I am to deny to them that pnis enlargement pill.

I would say that the results Google returns involve a number of factor/filters. A. What's considered mainstream, what appeals to many B. What's could be more or less objectively called quality. C. SEO, What results just slip through based on the page spending a lot of time and money appearing like A or B to the algorithm (but not being that).

And Google spends a huge amount of time and money trying to keep C from being the only thing BUT that's still not enough because there's a lot of time and money spent on the other end. At the time, this tug-of-war serves as a moat keeping competitors out. Any Google competitor would have to invest similar amounts of money.

Google has, does and always will make choices about what is best and that the definition of best is up to their discretion.*

I'd love to have a decent Google competitor. But I don't think you have won many friend here by implying that Google's discretion is just arbitrary (as a number of your posts here seem to imply to me). Google won, back when they had competition, by caring a lot about the, uh, quality of their search results. Now that they've won, they're slipping into other things and moreover, the SEO trash are nipping their heels.

Which is to say you won't build Google alternatives on "democracy" but on some concept of quality people will want.

Many people have been upvoting my comments here so the evidence I am seeing suggests that I haven't lost too many friends here with this line of thinking.

Continuing out the logic of your suggestion here, you would seem to imply that every other currently existing search engine besides Google also has bad quality as well in this regard. I say this because if other search engines had better quality that Google, they would be doing better than Google as per your suggestion. But, obviously Google is on top in a big way, so I have to ask, do you think the entire search engine industry, not just Google, has poor quality search results? What do you know that they don't?

Knucklehead: But, obviously Google is on top in a big way, so I have to ask, do you think the entire search engine industry, not just Google, has poor quality search results? What do you know that they don't?

Me earlier: larger issue is I'd acknowledge some search results just aren't what I in particular want but I'd claim other results are actually objectively low quality

It seems like you just a kind of "shtick". Anyone talking about quality, no how much nuance they add to it, gets thrown the same "how do you your ideas of quality are right". I already mentioned that quality is piece of Google formula and Google spend real money attaining that, employs thousands of people to rate search quality. Of course I'm not claiming to be a personal expert on quality. You can read further what I actual wrote above.

I don’t think making Google’s index public would be sufficient. Their real core asset is not the index but the historical engagement data they have of searchers’ interactions with the index (both on the results page and offsite analytics Google has access to, eg Google Analytics/display ad network).

Making that historical engagement data public is not feasible/realistic, IMO.

I think it is necessary though. Other search engine operators have cited this as one of the reasons their search engines fail or struggle to provide as good results. As far as the engagement data goes, it’s something that the Competition and Markets Authority in the UK has looked at as part of their investigation into google. See here for a very interesting analysis of Google’s and Bings click and query data: https://assets.publishing.service.gov.uk/media/5efb1db6e90e0...
Thanks--I learned some valuable vocabulary today! "Click-and-query data" was the phrase I did not have on hand.

I have never worked for a search engine company (or devoted much time to SEO), but I would suggest the click-and-query data advantage is invaluable to cementing Google's supremacy over any newcomers.

It was new to me as well, I think it was a phrase that was invented to describe the data during the CMA investigation but I am not sure.

I’ve heard search engine operators complain about the index consistently while they seem spilt about the click and query data. For example, DuckDuckGo and StartPage have made pretty good businesses out of pointedly not collecting click and query data. For my part, I think it’s the dual lock on distribution via exclusive agreements on being the default search engine and the advantages it has when web crawling that cements Google’s dominance over the search engine market.

Out of curiosity, what exactly is a "think tank"? Is it the sort of thing a person can make money by founding? What makes it distinct in type from other businesses or organizations?

I've only ever met one person that had anything to do with a think tank at a party about a decade ago, and I never got her contact details so I'm still very much in the dark.

Good question! A think tank is an organization that researches and sometimes advocates for the government to do certain things. They can be a non profit or an llc (knuckleheads’ club is an llc right now because it was the easiest to set up). In theory, you can make money from founding a think tank, but in practice it’s not a lucrative endeavour unless it becomes big enough that the president start earning steady six figures (like at the major ones in dc).
I do not know whether Google gives Pinterest preferential treatment. That said, I'd like to know: What are you searching? Results will be different depending on if you're searching "cute bedroom ideas" or "epistemology in pre-Socratic Greece."
Ha good question! I run a company that builds visual search engines and one of the sites I build and develop is a colour search engine for lipsticks. One of the main ways people discover it is through Google searches so I was studying which sites rank highest for queries like "nyx budapest lipstick dupes" etc. So yeah, for sure, queries for stuff that lends itself well to visual curation will be full of Pinterest results unlike stuff about Greek philosophy.
Give it a few days and this page will be the #1 hit for "pre-Socratic bedroom ideas"
(comment deleted)
"epistemology in pre-Socratic Greece." has a few results from Pinterest.
You can search with adding `-pinterest` to remove all that noise.
And the Pinterest pages have no content! They've beaten Google or Google doesn't care anymore, either way, plummeting result quality is a strong signal that Google Search is ripe for disruption. Ten years ago, Google would've punished all Pinterest properties for this.
How much of the bafflement stems from the fact that Pinterest's own audience is overwhelmingly women, while the general search audience is 50/50 and the HN audience is, judging by the character of the comments, 99% men?
I feel like pinterest is being a "placeholder"/"fallback"/"default" of the web. Each time I encounter quora or pinterest search results I am getting the feeling I was looking for something that does not have good online answer/content.
Did you look beyond the Quora / Pinterest / other SEO spam results? There's good content on the web for virtually any query, it's just often buried under mountains of SEO spam results.
I did. Usually I am not clicking the pinterest/quora links and I am left with nothing.
I solved this nonsense by installing a plugin in chrome that automatically adds -site:pinterest.* to all searches. Almost living in bliss ever since.
Care to share? A convenient, trustworthy plugin that lets you exclude sounds ideal. Anything for Firefox?
Unpinterested
This answer has never made sense to me. Pinterest breaks the 2 signals I would assume Google search to use:

1. Often, the image I'm looking for isn't the main image on the page, just one of the small thumbnails and very hard to find on the page

2. Almost every single time I click on a pinterest result, I end up going back because it doesn't help me, i assume others do the same.

Facebook users are linking a lot to Pinterest
Wondered the same about Amazon as well. They show up first for most products.

Are those organic results? Or do they have a kickass SEO team that games Google’s algorithm.

(comment deleted)
I can't think of the last time I did a search and got any pinterest pages in my results. It must be more a factor of what you search for.
Are you referring to web/text search, or image search?
Web/text, but on the rare times I do image search, Pinterest results don't "dominate" my results as OP suggested.
Well, Google and Bing do favor big or older/established websites. And Google's search is trying to be smart when it clearly isn't.

I'm getting sick of my searches being "corrected", sometimes they even ignore the quotes.

Could you please post some examples of search queries (text or image) that are dominated by Pinterest?

I share the sentiment in other comments that search engines are (deliberately?) allowing SEO abusers to degrade the user's experience. I noticed that for some queries (Like "How to brush teeth"), there are more ads on the first page than results! They're marked, but not in an immediately-visible way; I'd be most people integrate them with search results mentally.

I started throwing together a search engine about a week ago to address these concerns, and put it online yesterday. (https://www.pageref.org). I'm deliberately penalizing SEO abusers, and promoting websites that have high-quality content. Running custom searches on these sites in some cases based on keyword.

In a lot of cases, I'm throwing in search queries, and going through each result one at a time, and categorizing them; eg penalizing sites that use clickbait or scammy ads, that are low-quality but show up high in results etc, or probably aren't relevant, but are highly SEO optimized, like Pinterest.

>>>Could you please post some examples of search queries (text or image) that are dominated by Pinterest?

These are all image searches "sci-fi combat exoskeleton" -> 7 of 24 top images are Pinterest "zen minimalist clothing" -> 8 of 24 "grav tank concept art" -> 9 of 19

Like many other users in this topic, I run most of my G-Image searches with "-pinterest". I use DDG as my primary non-technical search engine, typically only switching to Google for images, trying to solve error messages, or researching frameworks/APIs/etc.

Related: in the last several years, there started appearing these StackOverflow mirrors. It's gotten to the point that you can't google a programming question without running into at least one. Some copy the questions and answers verbatim, some use machine translation to (crappily) translate them into my native language. Sometimes they even rank higher than the real StackOverflow.

I really wish there was a non-hacky way to ban sites from Google search results. I also feel like Google's ranking algorithm is utterly broken since it's amenable to this kind of exploitation.

They're pretty old. I remember when Google added an option to block domains (the option you want) around 2011 and they were the first thing I removed from my results, along with Wikipedia mirrors.

After the option to block domains was removed around 2014-2015, I rarely came across those anymore, but it seems they're back...

I've been seeing the same even in English language.
I find those are actually useful from the perspective of being mirrors of content that the real SO has deleted for whatever reason.
I've seen the same, but with GitHub scrapers. One is called something like HappyGit, another is called GitMemory. And a lot of times Google thinks GitMemory is the original, so the _actual_ original source (GitHub) never even appears. So then I have to go to GitHub and try to find the original myself using snippets of text. Really makes my blood boil.
I presumed it was just DuckDuckGo lacking algorithms to detect such 'spammers'.

But now I learned that Google is suffering the same poor detection. Another 'downside of using DDG debunked', I guess.

We are so biased towards google search being the perfect technology, that every flaw a competitor has is simply ignored on google.com
Add this to your browser's search engines:

  Stack Overflow
  so
  https://www.google.com/search?q=%s (site:stackoverflow.com OR site:superuser.com OR site:stackexchange.com)
Then just begin the search by typing: so[spacebar]<your search term>
Pinterest search results are so frustrating. There's a page that's close to what I want - but I can't get to the text ... I don't want a picture, I want to read a document. Seems like something is off, cuz it's crowding out sites with real content.
For me this is an instance of Betteridge's law of headlines: "Any headline that ends in a question mark can be answered by the word no." :) I very rarely see Pinterest in my search results.