212 comments

[ 2.7 ms ] story [ 257 ms ] thread
"Market can be irrational for longer than you can remain solvent"
yeah but then how do you get organic traffic?
(comment deleted)
On a related note, what does organic search traffic look like these days? I was once personally invested in running several small content sites and at the time organic traffic was pretty much entirely from Google. There were other inbound sources - Bing, Yahoo!, etc. - but they were so negligible that it would have been reasonable to ignore them completely for the purposes of optimization.

I've been using DDG as my primary search engine now for several years, and using Google no longer "feels like the Web" for me. I imagine I have one hell of a case of observation bias in that, but it does seem like alternatives have been gaining some traction in recent years.

I agree, I use Startpage.com

That said, looks like most of the US does not feel the same way about Google. I still see 90-99% of organic traffic coming from Google for my client sites.

It's still 90-95 percent Google, surprisingly. You'd think it would be less with mobile but no. We are seeing more duck duck go, but bing and yahoo have stayed steady for years. Social is a rise, and that's a lot of mobile. Also then just pickups (links from sites like this). But yeah, still very much google dominated.
Just to be clear, there is a mistake in this post. Adding a noindex tag will cause google to not display the page in their search results. They will still crawl ( they have to), in order to see that directive.

Also, keep in mind that google has recently changed the way they deal with robots.txt. So they only way to do what the author is suggesting is to add the noindex meta tag and let them crawl. Once they are removed from the index, block the google IPs.

How do you block their IP? Google has hundreds of numbers and AFAIK they don’t even publish all of them.
You can block Googlebot by User-Agent and IP addresses.
There are services that track google IPs, those doing black hat SEO and cloaking rely on them.

Google is going to crawl, you just can’t stop them from crawling even if you tell them not to in robots.txt. Best option at this point is to add a noindex tag on pages and let them crawl to see it.

Wouldn't that be considered illegal due to the same reasons as scraping?
I don't think scraping is illegal, unless you're hitting protected pages behind a login after agreeing to a Terms of Service, or hitting it so often you'd constitute a DOS attack.

An implied TOS notice on a separate page or footer is not considered enforceable last time I checked, likewise for robots.txt.

Lol, of course not - it would be (and will be!) illegal if you would be doing it, google can crawl and scrape whatever it wants.
As others mentioned, you can block their user-agent. That will stop most crawling. They have some bots that use browser agents to see if you serve different content to non-google bots.

To block those, you can feed their IP's to haproxy or iptables ipset.

    fetch_google()
    {
    for line in $(dig +short txt _cloud-netblocks.googleusercontent.com | tr " " "\n" | grep include | cut -f 2 -d :)
    do
 dig +short txt "${line}"
    done | tr " " "\n" | grep ip4 | cut -f 2 -d : | sort -n | uniq | xz -9ecv > _GOOGLE.netset.xz
    };
In HAProxy, you could include the decompressed version of that IP list with something like

    acl BOTS src -f bot.txt
Then either redirect, reject, silent-drop.
Any clue if Google crawls from IPv6 addresses?
They will crawl ipv4 and ipv6, whichever your DNS advertises for your sites.
I've found that google will still include the page in their search index if it's been discovered by means other than crawling the resource itself (e.g. it's linked from other pages). The search result will simply not include any content from the page, since they've presumably not crawled the page.
if noindex == true { content = "" }

Of course they still have it indexed in their own systems .... would you trust Google not to.

Should read: 'Dear Google, I'm giving you full access to my website but I'm asking you not to index it.'
Yeah, just reject any access from their subnets
"I don't like one of their products so I don't want anyone to find the stuff I write online"

Unless you are dependent on SEM, AMP doesn't really hurt you. This whole no-index thing is ridiculous and not at all possible for any sufficiently large website/company.

Only thing I can agree with is the alternative search engine point. We really need more competition in the field, so Google is forced to cater to it's user's wishes more instead of purely pushing their agenda on them.

Are they actually blocking Google somehow?

No robots.txt, no 403 when using a Googlebot user agent to fetch the site page...

Yes.

<meta name="googlebot" content="noindex">

Googlebot looks for this meta tag.

> As fewer results of such not-for-profit websites show up in Google Search, competition will be reintroduced to the search market.

I don't think it will. While I personally hate AMP, and disagree with many of Google's practices, this is not the solution. Google is everywhere.

It is the default search engine on almost every browser, as well as on most devices. How often do you hear someone say "look it up" as opposed to "Google it"? The latter is much more common in my experience. Google is synonymous with internet searches.

Outside of tech circles, no one is going to make an effort to even try other search engines. There's a reason Bing and Yahoo are treated as a joke.

Google is also the best at searches. While other search engines are slowly catching up, and Google's own results are becoming worse, it's still way better then most search engines - especially when it comes to specific and local/regional results.

Lastly, blocking Google means that you site also won't show up on Startpage.com results, because it sources results from Google. This is unfortunate, since Startpage is definitely one of the search engines that respect your privacy the most.

I'm not saying it proves anything, but Kleenex is synonymous with facial tissue and Velcro is synonymous with hook and loop fasteners. I "google" things on DuckDuckGo. :)
Yeah, I was just pointing out that to most people, ["google" == "web search"]
Yeah, I've been using DDG for years, but I just can't get "Google it" out of my dictionary. I try to replace it with "Duck it", which feels stupid as fuck to say, but "Google it" doesn't feel any better.
I don't think it is a problem. Xerox is a thing.
I say "look it up". Took a while, but I managed to do it...
Yeah- I feel that in this case the whimsical name "Duckduckgo" is a liability for them. It's just hard to turn it into a verb in an unambiguous way.
i use "search for it".

i've completely removed said word from my vocabulary. it can be done. break yo' mind free!

I'm not saying it proves anything, but Kleenex is synonymous with facial tissue and Velcro is synonymous with hook and loop fasteners. I "google" things on DuckDuckGo. :)

The people within one of my company's divisions refer to the internal knowledge base as "the google." IT defaults their browsers to Bing.

> especially when it comes to specific ... results

Specific results for niche search queries have been getting way worse over the last few years though. To the point where you have to actively try to trick the engine into giving you anything remotely meaningful.

Almost every time I try, I find pages and pages of generic, SEO optimized crap.

Google is losing value as a search engine. It's great for finding facts but its shit for finding fun. That's really my impression of the internet as a whole lately - there's nothing left of a noncommercial nature.
But at the same time I'm not convinced that the greater reason for this is with easier access to creating internet content we are not simply creating too much garbage to filter out all of it. It is always easier to break a system than build a bulletproof one right? I am under the impression that we are just returning to the mean of too much garbage content that is simply trying to sell something.
We can have a sea of garbage but if sites would restore the practice of linking to things they find interesting or relevant, we'd be able to navigate islands of interconnected non-crap once we find something decent. That's based on the notion that what I think is interesting is produced by people who have similar opinions of relevance.
Outside of tech circles, no one is going to make an effort to even try other search engines

People used to say this about Altavista and Dogpile and Lycos before Google. Change is possible, but it's slow and hard.

Google is also the best at searches.

I disagree with you on this one. It used to be the best tool for searching. Now it's hit-and-miss. I've found about 70% of the time DDG works, and I only use Google when DDG turns up nothing useful.

Along those lines, other search engines don't have to be better at search than Google. They just have to be "good enough" at search. Then the competing search engine has to being in some killer feature so that the average person loses allegiance to Google and makes the switch full-time.

That killer feature probably won't be privacy, though. That's too abstract a notion for the masses.

> Change is possible, but it's slow and hard.

It is also fast, but you need a product that is better, not just "not google".

I tried DuckDuckGo for a while, but they have way too many spam results on the first pages, when searching for things in my native language, e.g. tried searching for details about a new tax law a while ago and got a lot of either NSFW spam pages, or pages redirecting to malware. When searching in English it is not too bad, I don't know what is causing this, perhaps just lack of testing/interest for non-english users, or Bing/DDG doesn't run a malware scanner on its search results. Bing has the same problem. Run the same query on Google and there is no spam.

I could try to give DDG another try, but where can I report problems with spam/malware in its search results?

They need to start with a niche, like entertainment. Be the best search engine for gaming or something. That'll get eyeballs, then branch out. Would take some serious marketing, guerilla and otherwise, though.
although i agree,

> Bing and Yahoo are treated as a joke.

i use bing or others like yandex for image search. google is a front for stock sites and pinterest.

Hmm, maybe bing should focus on image search , use that as a trojan horse to take on google.

> Google needs to stick to being a solid search engine instead of trying to be gate keepers

While I agree with the sentiment, I feel like this ship sailed years ago - it's like saying "Amazon should stick to being an ecommerce platform".

What's more, Google was never in the business of "being a solid search engine". Google is and always has been in the business of selling access to users' eyeballs. AMP - as much as I hate it - is directly in line with their core business.

> What's more, Google was never in the business of "being a solid search engine". Google is and always has been in the business of selling access to users' eyeballs.

That's not how they became the darling of the tech scene though.

http://infolab.stanford.edu/~backrub/google.html

> Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users [..] we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers. Since it is very difficult even for experts to evaluate search engines, search engine bias is particularly insidious. [..] we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm.

When Google was first released it was revolutionary. I remember just being amazed at the quality of the search results compared to any other engines. And amazingly, the competitors still haven't caught up, for all of Google's issues.
So true, the difference was monumental. Pagerank was a huge breakthrough that seems so simple and so obvious in hindsight, it's clear why it changed everything.

The popularity of a website shouldn't be based on the number of times they can stuff keywords into a page, it should be based on the number of human beings who used the site and determined it was valuable enough to link to.

I don't know if pagerank is used anymore, but at the time, with the state of the web then, it was an incredible breakthrough.

Yep, prior to Google the dominant SEO strategy was to spam keywords into hidden HTML tags invisible to readers.
The ship have sailed is never an acceptable argument it’s just a lazy way to justify status quo.

As a French I’m pretty glad in retrospect that my ancestors under Napoleon reign and the Bourbon restoration didn’t go like “I really think democracy would have been a nice feature but I feel like this ship sailed”

Mmm yea but it would take similar turmoil to call that ship to port again.
It starts with one, Google abused their power too many times. Get a good domain name and direct traffic. That is all.
Can anyone remember if the old AOL landing page linked to AOL created content or content by 3rd parties?

I’ve made the argument before that Google is the new AOL homepage. It’s the gateway to the web for a huge majority as AOL was at one time. AMP is concentrating Google’s web just as AOL once did for first timers.

Google has much more utility but it’s still very similar. AOL had email, web, chat, news, shopping? (maybe late days of AOL), and let’s not forget search.

This site loads amazingly fast! Proves the point that you really don't need AMP.
You do if you want Google to send any traffic to it. Your only option is to bend the knee unless you're site is nothing more than a blog which you do for fun.
All that AMP does for speed is impose limits on CSS and JS size (plus givs access to Google CDNs). You can simply write small assets without the need for Google to specify this.
I've recently built an AMP version of a marketing website (since AdWords supports AMP cache, thought it would be useful for speed). The truth is, while AMP as a library does provide solid speed improvements on its own by implementing all the best practices, the real speed improvements is the cache with preloading - as well as the other stuff it does like image optimisation.
Google once was a force for good by indexing pages and rewarding good content with SEO.

I deeply hate AMP. It's a monopolistic move to bully companies into handing over Google their content.

> It's a monopolistic move to bully companies into handing over Google their content.

That's nonsensical. Google already has their content.

If anything, AMP is a democratizing technology. If you want your articles in Apple News, you have to integrate with Apple. Same with Facebook Instant Articles. With AMP, you don't have to integrate directly with anybody. Just publish AMP pages from using standard HTTP, and every link aggregator that implements an AMP cache (including Bing, Baidu, Yandex, Yahoo Japan, etc.) will be able to instantly load your pages for users. If I start my own link aggregator, I don't need to have the clout of a Facebook, Apple, or Google to get publishers to enable instant loading from my site — I can cache the AMP pages they have already published.

It's the same as publishing bus routes according to Google Maps's standard feed format (https://developers.google.com/transit/gtfs/) . I publish once in that format, and not only do my bus routes show up better in Google, but they also show up in all other mapping applications that ingest that format, without me having to do any integration with them.

AMP is a complicated answer to an easy one.

AMP was presented as an answer to webpages not being lightweight anymore. Instead of just pushing for more lightweight webpages, Google cane up with the monstrosity that is AMP.

> AMP was presented as an answer to webpages not being lightweight anymore.

Wrong. AMP is a solution to web pages not loading instantly, not a solution to web pages not being lightweight. If you want to prerender links above the fold without deanonymizing them to the publisher, you will come up with AMP pretty much exactly.

And worse, there’s already a lightweight alternative format for news stories called RSS.
Which takes away even more control from the publisher (no analytics, A/B testing support, monetization options, etc.) and doesn't support prerendering above the fold. I can't imagine the backlash RSS would get if Google came up with that.
Multiple blogs I read sell RSS sponsorships, and given that RSS feeds are, you know, ultimately web pages, analytics are relatively trivial.
> Multiple blogs I read sell RSS sponsorships

Sponsorship is a poor substitute for subscriptions and ads.

> RSS feeds are, you know, ultimately web pages, analytics are relatively trivial.

No, the aggregator that shows the RSS results has no standard way of reporting to the publisher who viewed which RSS entries.

The publisher can absolutely get by with measuring how many people viewed which RSS entries, which this can absolutely deliver. Combined with other web metrics of the sort that non-privacy-invading analytics systems like GoAccess and OWA can deliver, there's more than enough information to determine the size of the audience, where they're located, when the most popular browsing times are, what devices they're using, and which articles are more successful than others.

It seems to me that you're arguing that publishers -- or in practice, ad networks -- are entitled to track the browsing habits of specific individuals, even if (theoretically) anonymized. If so, I rather strongly disagree.

> The publisher can absolutely get by with measuring how many people viewed which RSS entries, which this can absolutely deliver.

Wrong. The RSS entries are cached by the aggregators, and the publisher never sees how many people have seen them.

> It seems to me that you're arguing that publishers -- or in practice, ad networks -- are entitled to track the browsing habits of specific individuals, even if (theoretically) anonymized.

No. I'm saying that AMP takes less control away from publishers than RSS does, and all the complaints are about how AMP takes control from publishers to ruin the web, yet nobody here has complained about how RSS ruined the web. In either case, the system controlling the cache tracks everything.

> The RSS entries are cached by the aggregators, and the publisher never sees how many people have seen them.

Aggregators like NewsBlur, Feedly, and Feedbin pass on subscriber counts. I am literally looking at my own FeedPress dashboard in another window right now. I promise I have the analytics I say I do, cross my heart.

As for AMP, I'm not sure I see how AMP takes less control away from publishers than RSS does. There's nothing that stops anyone from running an RSS reader which directly hits each web site they subscribe to. Aggregators may be more convenient but they're not mandatory, and this seems to me to be a pretty important architectural difference. Even if we grant that RSS is pretty difficult to monetize, AMP takes my content and cuts me out of any monetization possibility entirely -- and Google may punish me in search rankings if I don't let them do that. Surely you see why that's going to raise some hackles that RSS doesn't?

> Aggregators like NewsBlur, Feedly, and Feedbin pass on subscriber counts. I am literally looking at my own FeedPress dashboard in another window right now.

They do not pass them back in a standard way. If there is a new aggregator, the publisher has to integrate with them for analytics.

> As for AMP, I'm not sure I see how AMP takes less control away from publishers than RSS does.

I already told you in the very first comment you replied to.

RSS is strictly worse for publishers than AMP. Apple News is strictly worse for publishers than AMP. Apple News has some advantages (monetization, analytics) over RSS and some disadvantages (single company integration). Yet nobody on HN complains about Apple News or RSS because they don't understand the technology.

> has no standard way of reporting to the publisher who viewed which RSS entries.

That's a good thing.

That said, I do see an awful lot of RSS feeds that include tracking pixels, advertising JS, and all the other assorted nastiness, so I don't think RSS is precluding those.

If we end up missing out on anti-user options like the ones that you mentioned then RSS sounds great.
Tell that to the bozo who wrote the article, who finds AMP too restrictive already. RSS would make him apoplectic.

Also, there is nothing anti-user about prerendering less, saving battery and network bandwidth.

There’s a better lightweight alternative format for news stories called pure HTML documents (without JavaScript).

Google could be a force for good by encouraging pure documents and down-ranking JavaScript crap.

Instead they chose AMP for lock-in.

> Instead they chose AMP for lock-in.

And because it made sense for their business of selling ads

What a bullshit, PR-speak comment.

Let's compare this to digital piracy. Let's say I make a game and host it on my website. I am giving the game for free. Now maybe I have some ads in the game, maybe not. I want to get some exposure, so I ask a gaming site to link to my game (Google). At this point, everything is cool. I have a cool game, the gaming site has something to show people.

Well, the gaming site is big business and I'm a little fish. So in the interest in serving their customers they decide to host the game themselves (AMP). Hey, I'm hosting it online for free anyway, right? And because they love their customers sooo much, they did a cool thing where they dig through my game's code and strip out any ads I might have. It turns out their customers love this feature! Ads are annoying after all.

Of course, if I don't like this hosting feature I can opt out of it. But if I do, they will take my game and tuck it waaaay at the bottom of their list (or just remove it completely).

The reality is way worse than this, though. Because I no longer know when anyone downloads my game from the gaming site or plays my game. Should I make another game like the last one? How would I even know? The gaming site says not to worry, they have some convenient code I can add to my game to keep track of this for me.

Ok, you say - well if you don't like this wonderful service of having your game taken and stripped down and its usage hidden from you, just opt-out of the gaming site. But it's the ONLY gaming site 95%+ people ever visit. And although there are some minor benefits of stability and speed to users, users ALSO can't opt out of this - they can't even get to my site to download the game anymore even if they want to.

So what started as a partnership between content and distribution, where everyone is happy, has degraded into the distribution channel completely owning the audience and taking every bit of control from the creators of the content.

Which leads us to: > It's a monopolistic move to bully companies into handing over Google their content.

So please, tell me again how AMP is "democratizing technology".

> So please, tell me again how AMP is "democratizing technology".

Your whole rant ignored the fact that people other than Google get to use AMP pages. That's how.

> So what started as a partnership between content and distribution, where everyone is happy, has degraded into the distribution channel completely owning the audience and taking every bit of control from the creators of the content.

This was always the case. AMP doesn't change the fact that the search engine's ranking determines how well the publishers do.

> users ALSO can't opt out of this - they can't even get to my site to download the game anymore even if they want to.

That's a problem with the search engine's ranker, not with AMP. If a publisher publishes separate mobile and desktop pages but cripples the mobile page in a way that makes it useless, a good search engine should prefer to show the desktop page even to mobile users. The same should happen with AMP. If the search engine doesn't rank properly, users will choose a different search engine. For example, there is a broad range of queries where I prefer AMP results because they load instantly, and I want them to appear first. If there is some feature in a web page that I want that I know won't appear in an AMP page, I will skip the AMP results, and the search engine should update its ranking model accordingly.

But amp sites support analytics and ads. Literally every complaint you're making isn't based in reality.
I think the point is that it removes the locus on control about these things from the user to the dominate AMP provider.

I often find the issue with many things is a change in the locus of control. People feel less comfortable when it falls in another’s domain, and often these large players then take advantage of this.

The great thing about the old web which I think people miss is that it was (supposed to be at least, and initially was) decentralised. Every year the web becomes more centralised and in the control of large organisations.

There is no way for a user to disable it on their phone.

It causes news stories and blog spam to pop up before product pages or other non-"article" style pages.

It basically shoe horns the internet into 1 type of page because if you don't have "article" style pages you don't have amp and don't have the higher ranking.

Your getting downmodded because you completely missed the point here. People don't merely hate AMP. They also (and sometimes more stongly) hate what google is doing with AMP and what that's doing to the internet.

> It causes news stories and blog spam to pop up before product pages or other non-"article" style pages.

Your whole issue with AMP seems to be not understanding that Google ranking pages badly has nothing to do with with AMP. Google, Baidu, Bing, and other link aggregators that implement AMP caches are free to rank pages however they like and still load AMP pages instantly.

> Your getting downmodded because you completely missed the point here.

I'm getting downmodded because people don't understand technology. That doesn't bother me. I'll still correct them.

No, you still ignored the point.

Google's behavior is what everybody from the linked article to the commenters here hates.

> Google's behavior

And what behavior is that? Providing a publishing standard that anybody can (and do) take advantage of, including their competitors, instead of requiring direct integration like Apple News?

It's an unnecessary publishing standard, intended to replace the one that is already widely used: normal HTML.
Users want instant loading result pages. That makes it necessary.
(comment deleted)
I also hate AMP and its getting harder and harder to avoid. I created a hosts file that blocks as many AMP domains as I can find. Personally, I would rather not read an article then to visit an AMP site. If you are interested, here is the AMP hosts file: https://www.github.developerdan.com/hosts/lists/amp-hosts-ex...

The hosts file is easily added to something like PiHole, uBlock Origin, or Steven Black's hosts project. Many other tools also accept host formatted block lists.

I'll be the first to admit that the list is lacking, but I would love to accept pull requests if anyone else has an interest in contributing their list of known AMP domains

There's an easier way - don't implement AMP.
Yeah, I don't understand this post. There are billions of pages out there without AMP and Google indexes them just fine. If you don't wanna implement AMP, then just don't implement it. You don't have to completely block Google over it.
(comment deleted)
Exactly! I am so confused as well. It's like this person doesn't understand that AMP is optional. And even then there's ways to instruct Google NOT to point to your AMP page. (And no, you don't need AMP for SEO -- it's only a slight boost)
The problem is if your competitor has an AMP page google will rank them higher than you. That will cost your money.
AMP only puts you in the top stories carousel.

Speed is measured neutrally for organic search results.

THIS. I don't understand how so many people don't actually get how Google and AMP work. You don't need AMP for organic search results. It's used by Google, yes, but only for certain things AND there's ways to turn it off/point to non-AMP pages. We choose not to use it, other publishers do use it, but everyone looks at the pros and cons and then just decides. It's like any other business decision, but you have to understand how it actually works first and not just go "Oh, AMP's bad" or "Oh, without AMP my site is dead.) I don't believe either of those is true.
True, but skepticism is always warranted with any large company. I don't believe AMP is evil, nor are they trying to seize control of the open web, but if there's possibility then scrutiny is critical.

A free and open web is still in Google's best interests. AMP content is still standards-compliant, it just builds upon the standards already in place. Isn't this what the web was designed for - to be extensible?

You fail to understand selection pressure. AMP is necessary now for a competitive ranking. This is not a small matter. Moving up or down a single space in search engine results can be the difference between a successful startup and washing dishes for a living.

Very quickly, a strong selection pressure can establish a trait "at fixation", as biologists say. When this happens, AMP just is the web.

Is it possible to make an AMP page that just redirects to a non-AMP one?
That is one technique some people use.
I am currently struggling with Google and been even considering just straight up creating a Brazilian "Yandex/Baidu" clone or something somehow.

My current business is heavily Google reliant, and Google been proving to be a very poor business partner:

* Keeps punishing my site both in organic rank and AdWords rank (thus price) for not being mobile, despite my site being intended to be viewed by engineers with big screens.

* They remove frequently whatever features they had to avoid wasting money, the one that offended me the most is that I can't anymore advertise on mobile websites unless I also allow apps too, and apps are purely fraudulent clicks, babies don't buy industrial parts...

* Google refuses to take care of fraud, I've visited the top sources of clicks on my display campaigns and there was comments for example on store pages explicitly complaining it was pure fraud, and Google does nothing about it (example: one source of clicks was a flashlight app on Android that opens ads whenever the user turns the flashlight on, clicking the ad is only way to turn it off again)

* Google often randomly stop my ads and search because bogus reasons, requiring me to call them to fix it.

* They removed from AdWords interface the max click price when the campaign is in automatic mode, in the past this was very useful to find out if their AI was bugged (like one time it insisted on setting max cpm to 0.01 despite average cost until that day being 1.20...) or oberpricing me (charging ludicrous prices per click)

Their product (ad publishing, not search) is crap, but they have literal Monopoly, all other ads publishers I tried gave me almost zero clicks...

Only thing I don't tried yet is more traditional ads, but I don't have money for that (for example TV ads)

I honestly don't know why so many governments with strong antimonopoly laws tolerate Google shenanigans.

>I honestly don't know why so many governments with strong antimonopoly laws tolerate Google shenanigans.

To build an anti-monopoly case you would have to do more than show that Google was the only good ad vendor. Are they the only ones that can give you clicks because they are smarter and better at their jobs than every other ad vendor, or are they the only ones that can give you clicks because of underhanded market tactics? If I was the best shoemaker in town, to the point where buying shoes from anybody else was a waste of time, that wouldn't be illegal. You'd have to show I was doing more to stop competition.

Yes but there's a network effect here. If Google owns all the billboards in town, then you can't really find a better advertisement platform, no matter how skilled they are.
Sure, but Google doesn't own all the billboards in town. I regularly see pages with Taboola panels on the bottom.
Taboola is scum.
Why do you say that?
Never heard of them so I searched the web. Wikipedia says they also provide the Around the Web and Recommended For You boxes at the bottom of web pages. Haven’t read further, but I assume the reaction was prompted by them serving that lowest quality click bait made to look like page excerpts from the site you’re actually visiting.
It is often more than clickbait, it's entirely misleading to the point where if you click (for science) it's nothing like what is shown.

At least that was true when I checked it out years ago when Slashdot started running them. Which is also around the time I stopped visiting that site.

Anything related to advertising is
But Google isn't owning the billboards. The billboards elect to choose Google. Why is Google being punished? Shouldn't the billboard owners instead be fined?
Reading the FTC's definition of monopolization is strongly encouraged. It's more nuanced than is generally understood:

As a first step, courts ask if the firm has "monopoly power" in any market. This requires in-depth study of the products sold by the leading firm, and any alternative products consumers may turn to if the firm attempted to raise prices. Then courts ask if that leading position was gained or maintained through improper conduct—that is, something other than merely having a better product, superior management or historic accident.... Courts do not require a literal monopoly before applying rules for single firm conduct; that term is used as shorthand for a firm with significant and durable market power.... Exclusionary or predatory acts may include such things as exclusive supply or purchase agreements; tying; predatory pricing; or refusal to deal....

https://www.ftc.gov/tips-advice/competition-guidance/guide-a...

OP wrote:

> I am currently struggling with Google and been even considering just straight up creating a Brazilian "Yandex/Baidu" clone or something somehow.

While FTC's definition of monopolization is interesting, its of particular relevance in the USA. Different legal definitions will apply of Brazil (where OP appears to be from) or EU or The Netherlands (latter 2 where I'm from). Other countries where Portuguese is spoken could be of relevance, such as Portugal.

Understood.

Point being that even narrower US legal standards are much broader than frequently asserted online discussion states.

Yeah the monopoly case is difficult and probably a distraction. We need to give their competitors more attention - most of their competitors have given up through the years, especially in the ads space.
(comment deleted)
Add to that that their "upgrade" of their adwords/adsense interface is an unmitigated failure of constantly spinning progress bars and cards cards cards everywhere making it really infuriating to use.

And charges and payments have become suspiciously opaque. Things like CPM swings (for adsense) in the exact same audiences. As for adwords, luckily i can afford to forgo all mobile traffic.

I don't know what shit google is going through, but it seems to be pretty bad.

> Only thing I don't tried yet is more traditional ads, but I don't have money for that (for example TV ads)

I think we should try a webring-like structure , there is a lot of inventory that can be shared between small publishers (and they are underserved/plain ignored by google)

It only works right on chrome, and you have to sometimes pay attention to memory leaks on all their sites.

I built myself a workstation with 32GB of overclocked ram...

A few times I left it on overnight with a google site open (adwords or gmail), the machine became unresponsive, using all 32GB of RAM and then freezing after running out of swap... I only found out what was happening on a day I was particularly patient and let windows trash for 20 minutes to open the task manager and let me see the ridiculous ram usage coming from browsers (I use both chrome and firefox on that machine... I wanted to use only firefox but as I said earlier chrome is needed to avoid the "infinite spinning bars" bug on adswords, also some types of ad editors straight up shows blank on firefox).

I AM using chrome! Let me try again: Opening adsense: 6 seconds to load a useless dashboards with 3 extra boxes "we thought you might find this card interesting"

I want a sites report

Clicking on "reports"... after 5 seconds the left menu loads, while the report on the right still loads... I click "sites" on the menu... The report on the right loads, but it is the default report, not the sites report. I have to click the sites report again and wait.

This happens every day , every time. By the time i get there, i forgot what i was looking for. This is just a sad state

Oh, they say my website has mobile access issues? But it is a bootstrap website optimized for mobile. Clicking on fix issue .... checking live version ... your site is mobile ready... yet it still says i have to fix 3 errors. I pressed validate fix 4 days ago and it's still there. Hmmm i guess i don't get their level of AI.

And dont get me started on adwords campaigns. Can't set up a simple campaign without going through some extremely difficult to parse terminology. Wanted to limit my ads to desktop - turns out they removed that simple option - you have to do tricks with bidding. I just don't get why

> Oh, they say my website has mobile access issues? But it is a bootstrap website optimized for mobile. Clicking on fix issue .... checking live version ... your site is mobile ready..

Same here. That a company with that amount of brainpower fails to deliver the very basic tools to partake in their market really baffles me.

From talking to a few ex-googlers, a lot of issues come from heavy incentives (and upwards momentum in employment level/compensation) tied to shipping a product/tool, while very little is tied to the maintenance/improvement of something existing.

When combined with the ease of lateral movement (jumping from one team to another) means that a product gets shipped, then a lot of the team members jump onto the next shiny thing while that product now gets stagnant/buggy since it now has a vastly reduced team.

Ha, and now they need companies to invest in GCP.
We used to look at this sort of response time as a big negative in adsense front end eng land. Guess that's changed...
Have you tried FB ads?
Those would be pointless.

Most of my income comes from search, both organic and ads, my business sell industrial parts that are hard to find, people tend to contact us after they put on some search engine the code for the part (example: DIN 912, the code for the popular Allen screw), and find us there.

I doubt someone would look for that on Facebook.

There was a cool magazine here that was basically just ads, before internet it was just a bunch of ads printed in magazine form, intended for my target market, but even them switched to use google now (also they are the reason I got pissed with the mobile display campaign issues, I want to put my ads on that magazine, but I can't do so anymore without turning on ads on mobile apps, and those easily eat up all the budget with fraudulent clicks)

Those are for different kinds of products/audiences and often not interchangeable with keyword-based ads
It's as big of a UX nightmare as Google Ads.
Out of curiosity (and I really do mean, out of pure curiosity), why do you care about fraudulent clicks? Unless the fraudulent click volume is "lumpy" or unpredictable, the only thing that matters is cost per lead/cost per purchase.

Fraudulent clicks make clicks worth less, so presumably the price already takes this into consideration?

After I decied to just drop mobile entirely, where 99% of the clicks were fraudulent (and indeed cheap, something like 1 or 2 cents was common), the campaigns in question had huge drops in total cost per month, one of them was sadly, literally 100%, meaning every single click on some months was fraudulent (that campaign is still active, it gets like 2, 3 clicks per month)

Mind you, the campaigns with most fraudulent clicks, were the ones setup by google employees, they regularly insist in calling me to "help", explain new features and whatnot, every single time I setup a campaign following step by step their employee, the result was ridiculous amounts of fraud.

The campaigns that bring me most of my real world income are the ones where I ignore what google wants for most part and do it my way.

Only thing I don't tried yet is more traditional ads, but I don't have money for that (for example TV ads)

When people think "advertising" they often think TV first. But you might be surprised how reasonable some advertising rates can be for radio (especially classical radio sponsorships and news radio), regional and local newspapers, and certain magazines.

A web site I used to own advertised with AdSense for a year. The second year I put the money into local public radio sponsorships and a single magazine ad, and the quality of the responses was much better.

I replied this in other comment, but here even that got google enroached...

The magazine that is popular on my field now focuses mostly on online ads, and they use adsense, so to buyads on the magazine you need to use adwords platform to do so.

Youtube (direct to creator) and Podcasts are pretty good as well.

Some small targeted channels can give good return as good videos are used for reference and you'll get a burst of business at the beginning and some followup later.

We had huge success with metro advertisements (in the actual trains, on one line), and no success with radio, despite having massive coverage. Mileage definitely varies with your product.
I even heard an advertisement here in Sydney the other day for Mailchimp on a mainstream popular radio channel.

Made me think twice about discounting radio as a channel.

Maybe you can reach out to specific youtubers to have them do their own ads in their broadcasts?
> Google been proving to be a very poor business partner

The company is more parasite than a partner. It'll punish you severely without remorse for not following its rules. Good luck trying to open any kind of dialog.

been even considering just straight up creating a Brazilian "Yandex/Baidu" clone or something somehow.

That would be truly something. After all your market is what, 200mln people?

As much as I agree with you, I don't see this happening any time soon.

See, Baidu and Yandex already existed for years before reaching the size they're today. Brazil had it's internally developed search engines before (Cadê, TodoBR, Aonde). Some of them even ranked higher than Google in the early 2000s.

However, with time they got out of business or were bought by Yahoo or Google itself.

And when you think about it, those business (Yandex, Baidu) now offer more than just search - Yandex offers email, maps and more; Baidu has Cloud Storage, Browser and even hosts a Wikipedia-like website - or have some degree of government ties at their countries - Google being banned from China; Yandex reportedly handled information for FSB.

You need to have all those advantages to replicate the success case of Yandex and Baidu. And there's one of them - time on market - that you can't buy.

My current business is heavily Google reliant, and Google been proving to be a very poor business partner:

Google and you are no more partners than the fox and the lamb are partners. That’s your main problem. Do you know how many high profile companies have gone belly up because of a Google algorithm change?

Way ahead of you. I started blocking Googlebot from my websites a few years ago, and never looked back!
It's a matter of principle, AMP is a terrible thing for the web in general.
Yes, a lot of people agree on that. But you don't need to block Google from your site to stop AMP.The way this article is written it makes it seem like the only way to have AMP not affect your site is by blocking Google entirely, and that's simply not the case. I think that's why those of us who understand how AMP is used are commenting here -- and you can see that people are confused because other people are posting questions about blocking amp, when you don't have to block amp, you don't use it in the first place.

The author should clarify that they're taking extreme measures, but that this isn't required to not use AMP.

the article clearly states. The reason to block is to improve competition. The lack of competition is the sole reason why google can stongarm amp instead of an open standard
Google must offer an option to opt out from those kind of products and at the same time, offer the search result. I am interested in this, I don't want them to replace me for their own UIs.
AMP is optional. You don't have to implement it. Even if you do there's ways to instruct Google when to use it/when not to.
Yes, but there are other things that they already display in their search results without opting.
Good move, kudos to Bradley
Yeah alright I can get behind this. I don't want to support a company looking to monopolize the market. I will start looking into how I can go about cutting Google out of my life.
You can make an argument that Google is a threat, but it's certainly not obvious. It requires a complex analysis to explain why Google leaning on AMP is different than Oracle and IBM leaning on ANSI SQL.
You might be amazed how easy it is to rationalize such concerns away for $250k (or more) per year. The first sentence of the internal response of a Googler will be, "You don't understand."
. . . and the marketability and prestige of having Google on your resume.
ANY sort of relevant engineering experience is good enough to get a great job in this industry.

Going out of your way to work for a mega-corp like Google is unnecessary.

Personally I would rather consider this to be a smudge on a resume, because for me it means, that that person worked for something, that is known to not be a force of good any longer, probably for money or wannabe fame "Hey, look, I worked at Google! Look how good I am!". Maybe it will work with HR people, but personally in my eyes Google on the resume disqualifies you from being hired.

To work at Google has to be an ethical decision by now. If you are not at Google internally working against its evil tactics subverting it wherever you can and work against its taking over the web, then you are probably working for the evil machinery. For an entity spying on people wherever it gets the chance, seducing uneducated people into helping them to track as many people as they can online and shred their privacy, while trying to come up with "new standards" all the time, trying to make others jump on the bandwagon to unfree Internet.

So in general, a person with ethical concerns should avoid working for Google, no matter how good the pay.

I largely agree, so I'd like to ask: do you hire people?
If he’s going offline from search, he should create an rss or mail list. I didn’t see where to subscribe, so honestly I will probably forget his site exists in a week...
They're people who value the rewards reaped by working there more. The prestige from working at a business like this can make positive impacts far beyond the period of your employment there. Heck, just getting to in person interviews are statistically rare and an indication of exceptional ability.
When the morals of a company with a supposed high value of prestige becomes bankrupt (morally), when does the prestige of working there diminish? Ever?
Well, essentially, once Google either:

A) Runs out of funds

B) Ruins its prestige by accepting lower quality people

I personally wouldn't bet on B), however once ad money dries up, it's possible Google we know will just be a Yahoo of future era.

> Google is a threat

Probably not a threat yet, but it is rotting. In the internet age everything happened so fast and in most ppl's mind google is still the Do no Evil company. But , everything is faster in the internet age, and so is the rot.

> engineers still sign up i

A major reason: They have bought up every viable competitor. Their services (search, videos, adwords, adsense, amp, mail) are building a fort around their search income. Others, e.g. Social networking was not needed for their fort so it died.

I like this. It reminds me of the pizza shop that gave 50% off to 1-star Yelp reviewers[1] or the ice cream truck that charges "influencers" double[2]. It's a statement of "I don't like your system, so I'm going to opt out of the whole thing even if it potentially harms me."

[1] https://thehustle.co/botto-bistro-1-star-yelp/

[2] https://metro.co.uk/2019/07/03/ice-cream-seller-charges-doub...

It's not going to potentially harm him, not being listed by Google is arguably equivalent to ceasing to exist. It's a nice gesture, but I cannot imagine how this is good idea in practice, regardless of how you view Google ethically.
Yeah, no.

Ultra-Local business - eg. ice cream truck - really don't need a google presence in any way.

Yeah, no.

A programmer's blog is not an ultra-local business.

If your blog is mostly a portfolio, there’s no reason for it to be “discoverable”. You can just link potential employers directly to it. Arguably it’ll be a net benefit because it might cut down on recruiter spam.

Alternatively, maybe this problem can be solved by bringing back web rings :P