405 comments

[ 208 ms ] story [ 891 ms ] thread
I wish Google would penalize websites with pay walls
I was under the impression they would (hidden content/different content for Googlebot).
Most paywalls which offer the Googlebot the full site also offer the full site if you follow a Google search result to the article. So while you're correct that providing the Googlebot and people coming from Google Search different results is prohibited, most paywall sites just don't show the paywall for either.
Why does it matter where you come from? Content cloaking is content cloaking, referrers shouldn't matter.
Google cares that if you click a link in a Google search result that the content you see is the same as what they crawl, so paywalled sites give Google search traffic a pass because they don't want to be penalized by Google.

Of course, I agree with you in principle.

Some only give you 2 or 3 article views then cut you off. But, of course, they still appear in search results.
(comment deleted)
Pay walls will probably inevitably be the only way quality content gets generated, given the increasing prevalence of ad blockers. As the return on investment for online advertising wanes, there's a good chance you'll see more of this, not less.

Furthermore, most paywalled sites seem to deliver a non-paywall version to Google servers and when coming from Google search explicitly to avoid this happening.

Most of the best content on the Internet was created by volunteers, for free, in my experience. The rise of ad-funded content has done more harm than good, overall.
> Most of the best content on the Internet was created by volunteers, for free

What content is that? It costs money to host/serve that content at the very minimum.

If it's videos like youtube, etc.. .the content creators absolutely get paid. Most people's blogs have ads to help offset hosting costs.

There might be a few select examples - but in general, somebody's got to pay the bills.

How many HN posters do you think are paid to comment here?
> How many HN posters do you think are paid to comment here?

How many articles that HN posters discuss were written for free, have no ads on the site/blog, nor are behind a paywall, nor have some ulterior motive for the content (recognition, advertising/promotion, etc)?

I don't disagree, but a lot of content we enjoy costs more money to produce and host than volunteers can afford. Especially high bandwidth content like video.
> Pay walls will probably inevitably be the only way quality content gets generated,

Patronage model may be more important for most content types than traditional paywalls, though for some types of content (news, particularly) the interest bias the patronage model creates may be contrary to quality.

A paywall is a pop up ad, and worse than most, in that you can't dismiss it.

Google would argue that visitors coming via search aren't paywalled, but often that isn't true. Some give you 3 free views, for example, then cut you off.

To me, if you want a paywall, that's fine...But then your content shouldn't show up on search engines, or perhaps if it does, it should be clearly marked as such.

Why shouldn't people be able to get paid for their work? No one is forcing you to pay them; it's just the same as anything else has been: If you want the product of someone's labor, you pay them what they ask for.
They can get paid. It's just whether they should enjoy free traffic from a search engine when their content isn't free. At the very least, if there's a possibility I'm hitting a paywall, the search engine should put a [paid content] label on the search result.
Where did anyone say that they can't get paid for their work?
The guy who said that he wants paywalled sites punished for trying to get paid for their work.
Nobody said that. OP said that he wishes google would penalize sites with paywalls, but that has nothing to do with sites getting paid for their work.

There are many ways to get paid for your work without using a bait-and-switch method where you show google a different page than the one you are showing to your users.

I completely disagree. Google punishing sites for using paywalls will have the affect of making it harder for people to get paid for their work. And I don't believe that having a paywall is in any way a "bait-n-switch".
I hear you but then search results would be full of diluted blogspam, because paid content tends to be higher signal:noise.
Good. Those have been becoming increasingly prevalent to the mobile web's detriment.

I don't mind a few ads, but many of these interstitials are downright maliciously designed, making the entire page load consistent on hitting a tiny "x" target, presumably designed with the intention of facilitating accidental clicks on the ad.

This title confused me. I thought Google was penalizing mobile websites by injecting intrusive pop-up ads.
You mean intrusive like the AMP header on every !@$! mobile page now that's not only annoying but breaks the standard UX?
Seriously, f..k AMP. It is breaking the internet with the google lock-in URLs.
I can't get the real URL from them on mobile at all.
And worse, when you click the "Share" button to text the URL to a friend, it sends them the AMP URL (which isn't always easy to extract the real URL from).
It's in the actual URL, hidden after all the google crap
Google likes clean urls for everybody except themselves. The Google results tracked url are driving me mad.
On my iPhone 6s Safari AMP sites don't even work. I see the AMP header and everything below it is just blank. I always have to "Request Desktop Site" to even load the site.
Are you using an adblocker?
Do ad blockers break AMP pages? If so that is a pretty crafty way of killing them.

"Would you like to search and browse the mobile web through Google? Only if you disable your ad blocker."

I'd be shocked if they don't try to force AMP on the desktop as well next if this works out for them.

Well an "ad blocker" is almost always just a list that some guy or repo online decides to block.

So if someone decides AMP pages should be blocked, your ad blocker will start blocking them.

It's actually one of the reasons I stopped using them. The idea of some person somewhere deciding what I see and don't see didn't quite jive with me. And it was impacting me by blocking things like Google shopping which I was trying to use.

That's not a reason not to use an ad-blocker, it's a reason not to use a public blocklist. If you want more control, just make your own filter rules.
It was one of the reasons.

I also never really liked the idea of blocking ads, depriving content creators of money, and I don't want to always pay for everything with money.

That, combined with the lack of easy control, opinionated ad blocking utilities (stop blocking webRTC dammit. I know you think it's a problem, I don't), general breakage across the web, performance issues with most blockers, and more made me stop using them.

Not sure, but if something on the "web" is broken, my first go to is always the adblocker.
That might depend on the specific ad blocker and which domains it is blocking.
I have 1Blocker on an iPhone with the default lists turned on, and AMP pages load fine for me.
Yea, I'm using a content blocker: Firefox Focus

Too bad AMP won't work with it, but there's no way I'm disabling it as life before content blockers on iOS was hell. (Sites hijacking you to the app store, etc.)

I am using FireFox Focus as well and AMP works for me.
It's unreliable. Reddit AMP pages definitely aren't loading for me on iOS with focus
You can just go to the url and remove the google.com/amp/
The feature that finally drove me to DDG :)
Where do yousee AMP? I use firefox on android, is that why I don't know about it?
if you search for a topic in the news, you should see a news carousel at the top of mobile search results pages. these results have the word "AMP" alongside a lighting-icon.
What's about intrusive cookie warnings?

(They are apparently mandatory in the European Union and Google made them part of the Adwords rules some time ago.)

They are explicitly permitted. From Google's own statement on this (https://webmasters.googleblog.com/2016/08/helping-users-easi...):

"By contrast, here are some examples of techniques that, used responsibly, would not be affected by the new signal:

Interstitials that appear to be in response to a legal obligation, such as for cookie usage or for age verification."

Funny thing is, they are supposed to let you use the websites while refusing cookies, but some of them force you to accept the cookies or else all you'll see is the popup.
Why do cookie warnings have to be a popup? Not to mention that the vast majority of times these warnings come up cookies are in no way required for the site.
They don't. The websites can insert a small hyperlink anywhere on the home page linking to the cookie policy and that counts as notifying the user.
Notification with a possible opt-out is not always sufficient, some countries require consent / opt-in.
Funny, because Google Adsense offers "Page-level vignette ads" which are full page interstitial ads shown for mobile devices.

The penalty must not apply because:

> They're displayed when the user leaves a page, rather than when they arrive on one, so the user doesn’t have to wait for them to load

https://support.google.com/adsense/answer/6245304?hl=en

(comment deleted)
Actually, they supposedly are also only displayed when Google's algorithms believe the user experience isn't negatively affected by the pop up. Not sure how much they have behind the scenes on this, but I recall them showing up erratically when I tested them for my site.
I specifically asked Google rep on that point and he said penalty would still apply.

One G product can affect another it seems.

It is worth bearing in mind that not all sites are reliant on google traffic, so more cash from Adsense at a hit on ranking may mean more £ and zero traffic impact.

This is absolutely true: in the past when certain Google teams have asked to have their pages bumped up in ranking, they've been told to work on their SEO and/or pay for ads.
Oh good, Google abusing their power again.

I guess as long as it's for "good" reasons.

edit: would any downvoters care to explain how google being able to arbitrarily dictate web content is a power they should have?

Google's users don't like popup ads so this will likely make them happier with their search results and make them more loyal Google users.

So why is Google beholden to deliver traffic to websites that use obtrusive popup ads when Google's users don't like it? Is Google somehow morally obligated to ensure a website's revenue model isn't disturbed when trying to provide a better experience for their users?

>Is Google somehow morally obligated to ensure a website's revenue model...

This particular change doesn't irk me, but the one where they penalized for too many "above the fold" ads did bother me..because Google is the king of above-the-fold ads in search results. Similar for penalties for "duplicate content", which is exactly what their kg/widget program is doing...scraping other people's content.

So, as long they as practice as they preach, it seems reasonable.

  would any downvoters care to explain how google being able to
  arbitrarily dictate web content is a power they should have?
Not actually a downvoter, but I do disagree with the sentiment: Google are simply choosing how to rank web-pages, for the people who visit their own search engine.

If there weren't any monopoly issues in play, there would be no question of Google's behaviour at all.

As it is, there are potentially monopoly issues in play, but in this instance, Google are simply penalising sites that provide a poor UX, which is a reasonable position for a search engine to take: prioritise the sites which provide the best value to visitors.

As an end-user, I would like duck-duck-go, bing, yahoo, and every other search engine, to penalise sites that use popups in this way.

>edit: would any downvoters care to explain how google being able to arbitrarily dictate web content is a power they should have?

Because that's literally how search engines work. They look at hundreds of metrics to try to return the most useful result.

If you want a metric-free search engine, you might as well ctrl+f for a list of the world's domains, because anything else will have subjectivity involved.

Abusing their power? You mean like rewarding sites that present content people are searching for instead of plastering a full page ad in front of them? The shit people say to justify their anti-Google agenda never ceases to be ridiculous.
I'd really like to see them work on improving relevancy instead of swinging their corporate weight around at whatever "benevolent" end they decide is important this week. Considering how much time I have to spend nowadays tweaking queries and futzing with the search tool options to get relevant results, I'm starting to look at all of these moves much more cynically. Taking on anti-patterns is great, but not when your search experience is rapidly becoming one.
I don't find myself having to tweak queries to get what I want. Can you give an example or two of what you're talking about?
i think what your parent post refers to is, sometimes searching for <term1> <term2> <term3> <term4> google will instead just try <term1> <something that may be a synonym for term 2> and no <term3> or <term4>. then you have to go back and put <term3> <term4> in quotes to make sure they're actually searched on.
Yes, ignoring parts of the query has just absurdly diluted result quality. And if you want something like the term or the term in a phrase, you have to put it twice and quote both.
It has diluted the quality of some results. I find it particularly irksome when googling for error messages, or other precise technical matters.

For more open ended queries, of course, the fuzziness can be helpful. And I would assume that the typical Google search is going to benefit more from the latter approach.

I can only speak to my own experience, no idea where I lay on the spectrum of users. But I'm an artist who writes code as a hobby, so I'm not constantly searching for esoteric tech stuff or anything.
Search in verbatim mode. It probably can be made into a search shortcut.
The biggest problem I've had recently is that verbatim results can't be combined with a date range. Quoted text seems to be real hit or miss now, so if I'm looking for something narrowly defined and date limited, well...good luck!
Unless I'm missing it, there's no date range option at all on the mobile search page, which is even worse. I find the mobile search hard to use as a result, especially for tech-related subjects. I'll search for how to do something on Android, and the results will be a bunch of blog posts that give me information about how to find the option I want in the Android settings... of Android circa 2011.
The UI affordance is terrible, but it's there if you scroll over the search categories, under "Search Tools" at the end

(also, not sure if the GP is in some test or something, but date range and verbatim are still there for me on desktop and mobile)

(comment deleted)
If you try switching the results to verbatim and then change the date range, you'll see that the one setting overrides the other. You can only return date limited results OR verbatim results, not a set with both.
Oh, weird. I had no idea what you were talking about, but just tried in mobile Chrome on the off chance it rendered differently, and I do see it there. In mobile Firefox the toolbar has many fewer options, and Search Tools isn't one of them. Looking around a bit, it seems that's been the case for years now: http://www.ghacks.net/2015/11/22/google-search-results-on-fi...
I find that the search giant goes through cycles where I can find results that I'm looking for and not being able to find anything at all.

For example, let's say I've seen a page on the Internet one year ago, and I remember the title and some text. Sometimes, it isn't possible for Google to return that site with any search.

There is also the perpetual search problem for computer science related material, where a major set of the search term is single letters, unsearchable symbols, or very common words.

But supposedly, Google is always getting better, even if this is not a problem I've seen before. I guess the Internet is just degrading then?

I think there is a real market for a software developer focused search engine. None of the issues we have matter for a search a normal person does which is what Google of course optimizes for. A search like "what are the best food carts in Portland" is fundamentally different from looking for the text of an exception that was thrown by a particular library version. Our needs are to small for giant Google to care about. But it might be a big enough market to build a nice little company.
I would love it if DDG could make that their niche. Unfortunately, as it stands having no option to filter results by time makes it hard to meet all my needs as a developer.
Would you be willing to pay for it? If so, how much monthly?

I'm afraid a search engine so much specialized won't be able to live off ads.

I wouldn't pay a lot, but I would definitely pay.

like a buck or two a month billed anually I would buy without hesitation if I believed it had real value.

I usually search github directly now for these things, and it works decently well; either finding the code in the repo to which it belongs or better yet, an issue addressed by the developer.

As the sibling said I would probably also pay a small amount. However, I could also see that corporate customers might by this.
I keep seeing a bunch of the same search spam sites show up in the first page of results. Seems like something they should be able to fix. I actually started blocking a bunch of these search spam sites on my firewall so I don't accidentally waste my time looking at them.
It's not benevolent. They are herding webmasters to use google's ad platform.
Why only "mobile websites"? I hate em just as much when I'm viewing on the desktop.
Probably because it's a much bigger issue on mobile websites where your ability to click precisely on the 'x' is much lower than when your input device is a mouse or trackpad.

Edit: I am new to HN, why this was downvoted?

And when the loading speed is slow, which makes the layout move, so your attempt to click the [X] accidentally clicks the advert instead, generating income for the site.

(why are you being downvoted? No idea, you make a reasonable, articulate statement, with a credible explanation. Sometimes it just happens on HN. I like to assume people are accidentally clicking the [▼] character by mistake).

(comment deleted)
I can understand why the desktop gets a pass, but I really, really wish they would consider doing it for sites which pop up those damned "Give us your email!!" forms. Especially those sites which pop it up almost immediately when I've arrived at their site for the first time. I don't even know your site yet. Just followed this link from somewhere else. And you're immediately demanding my email for your pointless newsletter?

Maybe it's just me, but I think I hate those things more than anything else on the current web. Maybe even more than autoplay videos.

I agree, though I've unfortunately seen several tests that suggest that they work. The immediate "Give us your email" is much more effective than requests that wait for a certain number of pages, trigger after a certain scroll amount, or wait until you've been on the site for a significant amount of time.

Of the people who see the pop up, the conversion rate is highest for those who see it immediately. And there's not a significant effect on bounce rates.

I have no idea why! I would never simply give my email address to an immediate pop up modal.

They probably think they have to give their email to see the site.

We are power users, a lot of people have no idea what they're doing on the web.

Yes, I even think there's a large amount of users that, when shown a random popup "please enter your password to continue", would just enter the password they use for every service, making phishing quite trivial.
It's like web-designers ignored all the lessons of the pop-up blockers in the 2000s. "Hey, it didn't work then, but maybe if we embed it in the page people will use it!"
Relatively it seems mobile ads cover way more area than when on desktop.
If users hate these, shouldn't google's ability to rank the best pages already take care of this problem without special casing it?
Because fuck people who are trying to get paid for their work? I mean, we can't all work 100% for free all the time like you must do.

EDIT: I guess people who downvote also don't get paid for their work, and don't feel anyone else should?

You keep arguing that people "should be paid for their work", but I've yet to see someone say that they shouldn't be.
A client has just asked me to add a pop-up "whatch this vid, join our newsletter", when the user scrolls to about half way down the homepage for their SAAS startup. Further the pop up is not to reappear for 90 days.

They got the approach from attending an online marketing workshop that suggested this increases their list.

Felt like a bit of an anti-pattern to me.

Anyone have advice as to if this is effective or if it will be affected by today's announcement?

Do it and measure how many more (or less) registrations you get.
You will probably get more registrations at the expense of pissing everyone else off
Pop-ups like that have long been shown to improve conversion rates... at the increase of pissing off people who wouldn't have been a conversion anyway.
Do they increase conversion rates long term? A lot of those pissed off people might not be customers today but could be in future.
What was the last hover ad you saw? Closing it is so instinctual I doubt it even registered. Those people won't remember next time. Annoying people is an evolutionarily stable strategy.
Good point, I'm not sure. My instinct has now transformed to closing the tab instead of the popup though, the close button is in a predictable place.
After writing this today, I was reading an interesting (to me) article on building two sided marketplaces, a pop up came up around half way through, didn't feel unreasonable actually as I was feeling value from the contents of the article and it felt likely appropriate moment for them to make the ask.

Thanks for the discussion everyone.

Ref: https://www.sharetribe.com/academy/how-to-communicate-your-v...

This is just a personal anecdote but there's no way I'll ever click on or even read those things. I just frantically search for a close button or link as soon as one appears and if I can't immediately find one I'll sometimes just leave the whole site.
Does this actually helps signups? You could be the best product in the world and showing a popup like that would ensure that I never, ever signup.
I get all these demands too. This is how sites end up covered in distracting junk.
This will. The penalty is not specifically targeting ads. It's targeting sites that serve full-page interstitials to mobile users.
"Google Has Started Penalizing Mobile Websites with Intrusive Pop-Up Ads"

I totally read this as "Google Has Started Penalizing Mobile Websites… [by using the penalty of imposed] Intrusive Pop-Up Ads" instead of "…penalizing those websites that use Intrusive Pop-Up Ads"

Use uBlock Origin people! In Firefox on Android. Don't see any pop-up ads! I can't believe how much pain people subjective themselves to needlessly!
These popups are one of the biggest reasons I started using ublock origin!
I'd love to see uMatrix on FF Android. uBO has a habit of killing things needed for the content in my experience . The lack of a simple selective bypass makes it frustrating for the site that will not load correctly.
The advanced mode panel in uBO works on FF for Android, though it's not as painless as on desktop.
Wait you are telling me it is even worse? I see those popups all the time even with ublock
I once saw someone using their phone with no blocking of anything, it was truly horrifying!
I dont block anything on my phone ether. My policy is to bounce whatever annoys me ASAP. Yes i mostly read only hacker news on my phone
What's really stupid are sites from which you can buy things, but then pop up an ad for something else. Fandango, which sells movie tickets, does this. As you're trying to get to the "buy ticket" page, they shove movie trailers for other movies in your face.

I mentioned a site earlier today which sold plumbing supplies.[1] They pop up a "gimme your email" box which 1) cannot be dismissed, and 2) isn't even theirs, it's from "justuno.com", a spamming service.

These outfits have lost sight of what their web site is for. They're putting obstacles in front of a customer who's about to give them money. This is usually considered a big mistake in retail.

[1] https://www.tushy.me/

Pop-ups asking me to please take a survey about the user experience of the website, too. Well, I was trying to actually buy something on your website, before your goddamn survey pop-up got in the way.
I've had many sites pop this up within seconds of being on the site. Though the attention span of some might be short, I suspect the data collected by these surveys is highly unhelpful simply because you have not yet experienced the product.

Given they're all from the same company usually whose name escapes me right now, you'd think they'd be better at it?

ForeSee
Lots of unhappy customers in the survey. And we gave that to you for free- we could optimize your web page - for a fee. Even take the survey out- or to the exist- after we are done.

A business model worth of our admiration!

The worst part about their worthless surveys is that it's pretty much impossible to give feedback about the actual survey itself.

It's easy to see how this happens; a clueless exec at big-corp decides they need data, survey company gives them a sales pitch they can't refuse, exec never bothers to actually check out the product.

These worthless surveys are almost exclusively on the websites large enterprisey companies that got off the clue-train years ago.

One of my lobby conversations at a SEM conference dealt with those quick survey popups. The guy I was talking to swore that by putting one on their landing pages conversion increased by like 30%. His theory was that it helps to refocus users who are about to bounce. Give them a second to digest the page and right before they click the back button bring SOMETHING up to grab their attention. They'll just click the survey away and will come back to the page fresh again.

I've never tested it, so I have no idea if it works. It's an interesting hypothesis at least.

Should be easy enough to A/B test.
It could even be true, for his customers. For all you know, he sells inflatable sex dolls, and it's important to focus users to get that sale. The tactics that work for sites selling more mainstream products are probably different.
The guy I was talking to swore that by putting one on their landing pages conversion increased by like 30%.

Yeah, they all say that. The question is: if that's true, then why aren't more sites using "survey popups'?

I usually rate one star when an app I'm using tries to force me to review when I'm in the middle of trying to use it.
Generally that survey result get discarded; one big trick used by many apps is to have surveys that distract you away from leaving negative results in the appstore or other places where the survey result actually counts.
I'm not sure I understand, they proxy the reviews?
Seen it a few times, although not so much recently... An app would pop up a rating dialog, asking you to give it 1-5 stars. If you chose 5 stars, it would send you on to the Play Store to leave your rating - if you chose 1-4, it would instead give you an in-app feedback form. They couldn't force you to actually leave a 5 star rating, but it was pretty dodgy anyway.
Especially when they pop up as soon as you land on the site. That is just so so stupid.
"Pop-ups asking me to please take a survey about the user experience of the website, too."

Agreed - and those surveys are often powered by? ... Google!

This is indeed a little hypocritical of them.

We need to do something abut intrusive ads, but I'm not sure this is it.

I don't understand why sites don't make this a simple checkbox on the checkout page.
Also, e-commerce websites that require creation of an account to buy something. There were countless times that I had a credit card in hand, and I gave up when the stupid website required me to register a user account in order to pay and check out.

Extra negative points if your account creation process requires a phone number or e-mail address verification step.

Totally.

A good e-commerce site will give you the option of creating an account only at the end of checkout and the account is totally optional. I've seen it before but unfortunately it's not the majority.

Also also e-commerce sites which don't carry your basket over from non-logged-in to logged-in states (ie you start shopping, then remember to login before checking out.)
The main problem I have is that these popups rarely preload the content. This means that if you are visiting the page internationally, you could be staring at a greyed-out page for a good second before the "X" loads. CTRL+F4 is now a reflex to that backdrop. If your page is less important than an advert then I can't imagine me finding it all that important.
I blame this on A/B tests which ignore externalities. Sure, maybe the 3 metrics you were aiming for were improved, but your brand is fucked. I think brands like Sears/Kenmore/Craftsman were A/Bed to death. So many little 'optimizations' (read: cost cutting that looked-good-on-paper) that eventually destroyed what the brand was about in the first place.
Not to mention that, for abstract judgements like "product quality", these can also compound in non-linear fashion. Three A/B tests could show only minor effects on perception of product quality, but then if you put the original A next to the final B, you'll get much bigger differences.
In the case of Fandango showing an interstitial ad when attempting to check out, I can't imagine that wouldn't hurt conversion, and I can't imagine that conversion wouldn't be one the key metrics they would test with. I would guess it's more just a lack of care. Someone said "we need our website to make more money" and someone else said "okay, I hear that ads are for making more money."
It could be that the ads are making more money than they're losing from conversion decrease.
yes, they are already targeting someone who will buy stuff from them.
It's probable that there is no grand strategy, but it might make sense if their model finds you unlikely to convert and the conversion price is very low. It's not necessarily flat per movie, they may be losing money on that one as part of a package deal.
Well, presumably they're reselling these tickets on behalf of the theaters which get the bulk of the revenue. It's possible Fandango takes no margin on the tickets sales and just makes money on the ads.
That's a classic problem for understanding these things: If you don't know how the money flows, you have no idea how to second-guess what's best.
They take a fixed fee per ticket. It's a good model.

Like most ticketing operations, they've built a niche monopoly. If you want to buy tickets online, they are usually the only game in town. If the obnoxious ads annoy you, their attitude is basically "fuck you, drive to the mall and miss your movie".

This is called a "moat". Build it for your company, it's a good thing for an entrepreneur.

To be significantly successful, you need to stop thinking this way. Start thinking about what benefits you & your company.

It may seem like contempt for the consumer, and perhaps it is. But it's also possible the consumer is irrational and you can benefit in the short and long term.

I work in marketing, and have seen scant evidence for the value of a "brand". It's mostly branding people making an argument that comic sans devalues our brand, whereas the data clearly shows it doesn't.

The "long term harm" argument is indistinguishable from "i know better than you and I want it this way." It may be true, but it's unprovable.

This actually makes a lot of sense.
> I work in marketing, and have seen scant evidence for the value of a "brand"

To what do you attribute the success of, say Coca Cola, rather than generic cola? Price and a good distribution network? Surely brand must contribute a little.

I don't know much about how branding works at that scale. But I don't think it's very effective for small/medium companies, and can be quite a money sink.

In nearly every SMB branding exercise/project I've participated in, the need for a brand, and the benefits it provides, are entirely limited to the egos of the employees.

I have seen this "we can't measure branding but we can measure conversion rates so let's optimize optimize optimize" approach borne out to create some of the worst websites. High converting, surely, but spammy, ugly, constantly trying to jam you into the funnel, full of dark patterns etc. Usually utterly forgettable, but often memorable for how crappy the experience is.

In my experience companies that do this are thinking very short term: how many people can we get through our funnel this month and how much money can we make from them? Yes it's hard to measure 'long term harm', but to assume it doesn't exist because it can't be directly measured makes it easy to shoot yourself in the foot.

To me it is worth aspiring to be respected as a business. Obviously not to the point of letting your business fail because you obsessed over branding while nobody actually bought your product, but as a larger goal. A business that tries to squeeze customers for all it can and achieves grudging acceptance isn't something I aspire to be a part of.

The problem with a brand is that you don't know if it's the most effective use of the resources that go into it.

Coca Cola has nearly infinite resources, compared to the jobs I've worked (<$75mm annual revenue).

Even then, can they prove the value of their brand is an effective use of their marketing dollars? What I've read has a lot of correlations that could be explained by other causes.

For a good read on this, check out the Intel Inside section on this article: https://conversionxl.com/cro-vs-branding/

It's difficult to measure the value of a brand, and most companies don't bother. It's expensive, because you have to pay actual people to talk to other people in a systematic way, over a long period of time.

Brand awareness is the measurement of the degree to which potential customers know that your product or service is an option when they are ready to make a purchase. You can't measure it from your marketing interactions alone, because the data set of your interactions is biased toward potential customers who are already aware of you. You can only measure it by performing statistically valid sampling of potential customers. The business purpose of measuring awareness is to help you find customers who have never interacted with your brand.

Brand affinity is the measurement of the degree to which potential customers have an emotional attachment to your product or service. A good shorthand is the degree to which a potential customer thinks of themselves as a "________ person." Think of a dude in a pickup truck saying "I'm a Ford guy, they've never let me down," or a person who won't even look at a phone that is not an iPhone. Again--the only way to measure this is by performing statistically valid sampling of potential customers. Measuring your marketing interactions alone will give you an artificially high measure of your affinity.

Design standards are important to both awareness and affinity.

A standard look helps the customer make mental connections between all the various times they've been exposed to your brand--this raises awareness. Comic Sans is not necessarily bad for awareness as long as you use it consistently in an intentional way. Randomly varying your fonts is bad, though, because it makes you "look" like several different brands, which means you will need to deliver more interactions to raise awareness.

Comic sans also might harm certain affinities, for example along the lines of being "sophisticated" or "beautiful." If a customer wants to think of themselves as sophisticated, they will seek brands that also present themselves that way. You're not going to see Mercedes Benz, which depends heavily on affinities of sophistication and elegance, using Comic Sans. You might see Scion, which depends on affinities of quirkiness and irony, use it. But again, the point is to be intentional.

Companies that are good at branding tend to be good either because they measure it compulsively (example: Proctor and Gamble), or because they are good at making internal opinionated decisions that stick (example: Apple).

Either way, the day to day operation of a strong brand does indeed look like a few people within a company telling everyone else what they can and cannot do with design.

Thank you for taking the time to write this. Honestly, it makes a lot of sense.

I don't have any experience with billion dollar companies. Have you worked in the industry and seen how they measure their campaign efficacies in huge companies? I bet it's really different than what I've seen.

Based on what I've seen, it's a lot of story telling that "branding" works. The question is, does it work better than if the company had put those resources into other avenues? Improve the product. Decrease the cost to consumer. Better customer service.

Here's a good cro vs. branding article, check out the Intel Inside section. [0].

There's a lot of money being made by people working in branding. If their field actually wasn't very effective, I wonder if it would be possible for them to accept it. I generally wouldn't want to find out that my career has essentially been a waste, and I created no value. (insert sinclair quote here)

[0] https://conversionxl.com/cro-vs-branding/

I get the feeling that nobody (untapped-market alert) measures whether they're trying to optimize something users don't really care about.
The cynic in me:. It is a great natural rate limiter to keep from too many customers seeing sold out while progressing directly through the process.
Heh, measuring average and unbiased user opinions on the internet is probably the second most uncertain thing in this universe after actual uncertainty in quantum systems.

Both opt-in and opt-out ways of doing surveys on your website introduce bias. The opt-out (read: in your face pop-up) is more likely to make people angry which makes them more likely to leave negative reviews.

The opt-in version will only be found by people having a relatively extreme opinion about your site, good or bad.

The difference being in the quantity of the data. The first will probably be answered by more people, since the people who'd have completed the survey in the second case are probably more or less a subset of the people who would complete the survey in the first case. All of this makes it no surprise that most websites that want to have any kind of feedback from their users go for the opt-out strategy. More is better if less is not in any way more :)

(comment deleted)
I think this is related to my fundamental problem with A/B testing: when people A/B test individual variables they'll take a greedy optimization that gets stuck in local minima. The other problem is that you only understand the change in metrics you measure and things like "satisfaction" are often lost.
> greedy optimization that gets stuck in local minima

I think you wanted to say local maxima, but yes I agree.

(comment deleted)
It's customary to state the optimization problem such that you're minimizing a function - hence "local minima". If you want to maximize something, you simply negate the function.
Good point. I see this almost on a daily basis with clients. They miss the forrest for the trees. Its dumb to optimize for email capture when people end up hating your brand as a result of it. I remove all pop ups and simply add clear calls to action in different places in the page. They are static components that are part of the layout. The copy is then adjusted to drive invite people to subscribe. I get higher quality conversions. Its not the quantity but the quality of the leads. People forget that.
Sears was not killed by any tweaks or optimizations - it was a brand that was past it's time, culture changed, and it could not find a new way forward.
I think it didn't adapt to the times. It was the Amazon of a past generation, with its own very high quality version of AmazonBasics. They didn't make the transition to a new reality though.
Sears was the Amazon of its day (around the early 1900s) where you could order almost anything via mail (including housed!). I guess the felt that products via the mail was so 19th century.
(comment deleted)
Interesting how pop-up blockers made their way into browsers years ago, so now they no longer open a new window, but simply display the "pop-up" within the page dom.
> This is usually considered a big mistake in retail.

Is it? Grocery stores keep essentials far away from each other so you have to walk past stuff you don't want in order to get to what you want. Imagine a store where they had milk at the checkout counter! That's the retail equivalent of amazon.com ;)

Milk is kept at the back of the store because that's the easiest please to put it from a logistics stand point. The truck the milk comes in pulls up to the back of the store and can immediately unload it into the giant refrigerated rooms at the back of the store. The fact that it helps "build the basket" by being at the back is a happy coincidence for the store.
Every Target I've been to has the milk before just about every other grocery item. Strangely enough, the seasonal stuff is tucked away in a corner.

Then again, I guess if you're going to target for milk you're probably doing it wrong...

Your last sentence is more relevant than it seems on first glance. If the majority of Target shoppers are looking for the seasonal section and not milk, then it's actually pretty equivalent.
I always find this interesting, as Target is pretty smart about merchandising. My guess here is that they know that they usually (at least around my hometown) are not top of mind for grocery. Milk is probably a trigger.
Definitely down to personal preference, but the milk at our Target is generally in better condition, tastes better, and lasts longer than the milk from Safeway. And the shopping experience in general is much nicer than Safeway.
Oh totally. What's weird is their organic milks expiration dates are really far out compared to other brands. Kind of insane actually.
It's not weird, there's a good reason for it. As I mentioned in a previous comment, its because the milks with a longer shelf life are UHT pasteurized, which uses higher temperatures for a shorter period of time than traditional pasteurization. They are typically labeled "ultra pasteurized." This extends the shelf life of milk.

Additionally, any time you see milk that is shelf stable (doesn't require refrigeration until opened) its been UHT pasteurized and packed in special aseptic packaging.

And, yes, I've noticed that, for some reason, organic milk tends to be UHT pasteurized more often than non-organic milk.

Safeway is pretty much the bottom-of-the-barrel grocery store in my area, so while I'd agree with that comparison of Safeway to Target, it's less true of Target than just about any competing conventional grocery store.
Seasonal stuff needs to be in an area of the store they can close off for a few weeks everytime the season changes while they redecorate. During those times they want the seasonable stuff to be out of the way so the construction efforts do not get in the way of customers.

I don't know if this is their logic, but it makes sense.

The amount of consumer studies done on grocer layouts is enormous. The direction of foot traffic has been decided from rtl despite our common ltr reading format because it's more profitable to do so, the checkout line with small items marked higher than in other parts of the same store for the same reason, and so much more.

If your sentiment were true, most heavily weighted expendables would all be located in the back simply because of labor. Labor is cheap when you can increase revenue from every potential customer with basic layout adjustments.

(comment deleted)
> most heavily weighted expendables would all be located in the back simply because of labor

They are. At least in almost every grocery store I've been in. The milk coolers are almost invariably against the back wall, accessible from the stockroom (which itself has the loading dock), as is the butcher / meat department. Both of those departments are probably getting daily deliveries, and it's of stuff that needs to be kept cold, has a short shelf life, and in the case of milk specifically, it needs to be stocked from the back of the shelves so that the oldest items get bought first (i.e. it's FIFO not FILO). It's a real pain to do this if you can't get to the back of the cooler.

Bread, however, is generally in its own aisle because it's not stocked by the store. Vendors (the bakeries) send their own delivery/logistics people in to stock it, so the stores don't really care about how easy it is, and those vendors also pull expired or near-expired items back off.

Produce, despite being expendable and delicate, tends to be towards the front because it's very high-margin for the store and also requires a lot of continual employee attention (and it generally has dedicated employees who just do that). It's admittedly been a long time since I worked in a grocery store, but back in the day in the pecking order of grocery-store floor workers, the produce guys were sort of at the top of the heap. (Although maybe that was because they could hook you up with the "damaged" fruit...)

The store is inevitably surrounded by 4 walls. Often more than one of those walls leads towards the front as part of the "cold chain".

If it were more profitable for the store to stock milk in the front they would - I agree. But the reasoning isn't delivery layout when they manufacture their own layout.

Either way, I thought this was an interesting take on it http://www.npr.org/sections/money/2014/07/23/334076398/episo...

My supermarket has a stock of the higher priced name brand milk at the front of the store, near the checkouts. I assume that's to grab both people who forgot to get milk and don't want to lose their place in line, and people who are there only for milk and want to get in and out.

The less expensive store-brand-label milk is at the back of the store where the big walk-in coolers are.

Contrary to what you would think as a consumer, grocery chains almost always want you to buy their brand (the less expensive one) because the margins on store-brand items are much better. That's why the stores bother selling them in the first place (and why margin-hungry companies like CVS will have every popular placed side-by-side with a store brand)
Name brands pay for premium shelf/fridge space.
And that is probably the reason for the milk up front. The name-brand milk producer is paying big bucks for space in that cooler near the checkout, enough to offset the supermarket's probable loss in sales of their own store-brand milk in its normal location in back.
The cold chain thing is plausible, but I doubt it.

Supermarkets have varying strategies for stuff, including cold items. The supermarkets around me put ice cream in either a center aisle or the second to last aisle, in the front of the store. The "smarter" store from a data standpoint has pint ice cream in an end cap next to the florist to nab women. Milk is always back corner.

If they can figure out how to maintain the cold chain for ice cream, I'm sure milk would be easier to move around.

> pint ice cream in an end cap next to the florist to nab women

My wife never buys me flowers :-(

But I am a sucker for a pint of ice cream :-)

They almost certainly move vastly higher volumes of milk than ice cream, making logistics a bigger concern.

It not that it's impossible to maintain the cold chain with other placements, it's that there is a cost (including that imposed by floor traffic) in doing so. Putting it someplace where a large volume walk-in cooler can be directly loaded from a truck and the display stocked from that without employee floor traffic across the store reduces the costs (labor, losses, and otherwise) of selling milk.

Physical store layout has nothing to do with online shopping.
Planet Money #555: http://www.npr.org/sections/money/2014/07/23/334076398/episo...

Their conclusion: the jury is out, but "never breaking the cold chain" is a good contender.

Thank you for this! For the couple of industry insiders they interviewed it seemed like they catered towards a 'helpful to customers' attitude despite what they implement. Could be saving face, could just be easier.

Idk being told to try making it more accessable by a consultant who's job it is to maximize profits and the grocer does "what it wants anyway" doesn't sound right on an industry scale.

Working with a big food manufacturer some years ago, had a detailed discussion about their relationship with big super markets and how they had to do X offers per year. More interestingly, the end of aisles were known as sh*t bins. You could sell anything there. It just shifted.
I don't think that's really comparable. No one's going to arrive at a store and decide not to buy the milk that they came for because the walk to the back of the store is too much hassle.

More comparable would be like someone standing in front of the checkout and insisting you take a survey before you can pay. In that scenario I can very much imagine someone getting pissed off and leaving.

They just won't go to the grocery store in the first place. They will buy milk at the gas station instead, or at a drugstore, precisely because the supermarket is so much hassle.
In the UK there has been a move away from giant supermarkets over the last few years. The consumer electronics and other non-food items that pad out the giant supermarket and make it hard to get to the milk, people buy that stuff online now so it is not possible to 'pile high' in the big box supermarkets.

With this move has been a growth in convenience stores that are operated/franchised by the big supermarket brands. People go to these places for their milk. (And do a big online shop for the bulk of their groceries).

So people are just deciding not to go to the big box stores in the first place.

Anecdote, but this happened to me. Where I was living the largest 'Tesco Extra' imaginable opened up literally across the road. This was AMAZING. Well, not really, within a week we were back to the convenience store buying everything there, it really was too much hassle going to the giant Tesco. Didn't need the kilometre walk past the TV's, clothing and other stuff that was placed between the store entrance and the milk. The 24/7 novelty wore off too, didn't actually need to go shopping for that pint of milk at 2 a.m.

Another part of the problem was that the customer service in the local store was more friendly. They didn't have people to greet you or people giving you free samples or any coupons at the checkout or any loyalty cards. Just polite 'how's your day?' chat with the small team of staff - all known by name - sufficed.

With the big supermarket I don't think there was this inate customer service aspect, 'customer service' was contrived marketing, nobody cared about you. And so it is online. People that do marketing online care more about their Google analytics and loading up websites with spam-scripts than they do about 'reality'. They may have one result per quarter from that pop up, that extra sale is good, a result, success even. Not on their dashboard is how many people didn't buy because of the stupid 'enter email address now:' popup they had on there. Marketing people are not the sharpest knives in the drawer, far too often they lack the skills of empathy and cannot see things from the customer's perspective. Hence they have to use things like statistics in an attempt to make up for their lack of intuition and appreciation of customer service.

I've always taken for granted that I'm only ever about a 15-30 minute drive from a Walmart Supercenter (which has a grocery store in it) that's open 24 hours a day.
Milk? That's funny. Here in France, it's drinks and wine ;-) But most of the milk sold here is pasteurized, so not kept in fridge (only once opened).
Milk in the US is pasteurized as well, but to be shelf stable you need to use a different pasteurization processes called UHT (ultra high temp) and package it in special aseptic packaging. UHT milk is put in traditional containers and stored in the fridge as well.

You can buy self stable milk in the US but it's not very popular for some reason. Except for Yoo-hoo, that's a popular milk product that's usually sold in shelf stable "juice boxes"... and I think it tastes even better at room temperature.

When milk alternatives like soy milk, almound milk, coconut milk, etc., first came to market they were almost always sold shelf stable. I can only assume that's because refrigeration space in supermarkets is sparse and expensive (learned that from Shark Tank) and these were niche products at the time. Now that they are mainstream you can find them refrigerated right next to the dairy milk. I guess Americans prefer to buy milk that way.

Re: wine. Interestingly in the US, due to the fact liquor laws are regulated by the state, when you cross state lines your supermarket alcohol selection will vary drastically. Some states ban alcohol sales in supermarkets entirely. Some allow only beer and wine. Some allow only beer only. Some allow beer, wine, and liquor. I've never heard of a state that banned beer but allowed liquor but who knows, it's possible. It mind boggling just how many crazy (and different) regulations about silly stuff like this exists around alcohol in the US. Funny story, a friend of mine went to a supermarket outside of the state she lived in and was totally mystified she couldn't find Bacardi's. "I've never been to a grocery store that didn't have Bacardi's" She didn't know they weren't allowed to sell it there.

Personally, I'm in a "beer only" state and and wine coolers and malt liquor (colloquially known as "40s") are categorized as "beer."

To be self stable milk needs to be heated hot enough to change the flavor. Americans prefer the taste of milk that has not been heated as much.

I once has breakfast on a real dairy farm, they got their milk from the tank right after milking. After drinking milk that was in the cow just 3 hours before I cannot stand milk you can buy. I cannot say if this is because it was never pasteurized, or just that is was fresh, but the flavor was better.

Note, do not take the above as advocacy for unpasteurized milk. The risks are real enough.

There's no discernible taste difference (to me, at the very least) between traditionally pasteurized milk and UHT pasteurized milk. I seriously doubt anyone could tell the difference in a blind taste test. Additionally, plenty of UHT milk is sold in the refrigerator section, so it's not an aversion to UHT itself. Organic milk is more often UHT pasteurized than non-organic milk and people usually buy organic because they believe it to taste better.
I think there's a very noticeable difference – but you don't have to take my word for it, because lots of research has been done on this subject. For example, see Vazquez-Landaverde, P., Torres, J. & Qian, M. (2006) Quantification of trace volatile sulfur compounds in milk by solid-phase microextraction and gas chromatography-pulsed flame photometric detection, Journal of Dairy Science 89, 2919–2927 – or just search for 'uht milk taste' on Google Scholar.
It's a good thing that ublock origin can be installed on firefox mobile (android). Firefox on Android isn't nearly as fast in UI performance as chrome, but the ability to use standard firefox plugins makes up for it.
I didn't know that, thanks. I remember how much I love ublock origin when I start visiting websites in chrome on mobile. We get used with a clean browsing and feels like this is the norm until you have to do the same thing on mobile.
I use firefox on mobile for exactly this reason, which is funny, as I use chrome on the desktop.

It used to be that Google Now obeyed your browser choice when opening links. When it started forcing chrome, I had to actually uninstall chrome in order to make it use firefox to open the news article tiles it presents to me.

I like your points, but I like even more how you just hopped on and essentially shouted into the internet "hey everyone, I'm trying to buy a bidet."
If only we could be so candid, maybe our tushies would get the royal treatment.
I grew up with a bidet, I don't have one now that I moved out - it's my number one source of unhappiness.
How do they not realize? Don't they do A/B testing..?
I make a point of providing what could be considered pointed feedback on their spam-feeding mechanisms formatted as obviously fake-but-valid-looking email addresses. If shortsighted jerks want email addresses, I'll give them some.

What I actually want for this sort of problem is a browser extension that allows me to annotate domains in some way, and then treats links differently based on that. This is a half-baked idea; I have no time to make this, so I haven't been thinking about it much. (Please steal the idea!) But perhaps a way to quickly tag domain names - 'annoying popup', 'spammer', 'dead to me', etc., and then indicate the tags visually next to links/change the colors/something like that, so I can more easily avoid sites that annoy me.

That's exactly what personalized search results are, but it changes rankings passively and automatically for you.
You're assuming they make money on ticket sales. Maybe all that money goes to the theater? If so, the ads are completely rational.
Then it becomes rational for the user to buy tickets elsewhere.
I really don't see how. What are the alternatives you envision? Switching to a more expensive service that doesn't show ads but charges a markup? Physically waiting in line at the box office?

Sadly, if "give it away for free and make money from advertising" is the dominant business model in an industry, it's very hard to disrupt. That's precisely because there aren't really any sustainable incentives for users to switch to a different model.

How much do they make per add shown? Is it more than a price difference users would care about, say 50c?

If one experience is awful and one is pleasant but 50c more then most aren't going to notice the difference.

Not to mention the effects on the advertiser, being disrupted from what I'm doing to see your ad makes me not want to see your movie.

For theatre's over reliance on advertising has already turned a lot of people away. We've all made and heard complaints about 20 minutes of advertising before a movie starts. The only thing they've got going for them anymore is timed exclusivity.

All those points could just as easily be used to argue that paid services are a superior business model to ad-supported services in general. Unfortunately, history doesn't bear that out. The revealed preferences of actual customers is different from what you and I think they should want. Just look at an app store.

For what it's worth, I'm with you. I'd rather pay a small fee to avoid ads (and I do so whenever available.) But I've come to the sad conclusion that we are a small minority of consumers. Better to accept that than fall prey to the is-ought fallacy.

I bet they have low conversions anyway, because who in the hell wants to pay $2 fees for a $15 movie ticket?

They don't really save time, as you need to show up early for a seat in most places.

Movie theaters make no money on tickets---it's concessions where the money is. That's why it's $10 for a small bag of popcorn and a drink (which costs the theater pennies on the dollars).
They make little money on tickets the first week or two. After that the percentage changes.
They make plenty of money on ticket sales. Just not much profit on tickets. They still rely on getting the ticket sales money on order to buy the rights to show movies and keep the lights on. They can't just give that up.
(comment deleted)
Yeah, it completely boggles my mind that companies do this. I was selling my old laptop on eBay, and on the desktop and mobile version of the auction page, there were ads for the same model of laptop I was selling at a cheaper price on another site. I guess ad revenue was literally more important than making money off of users selling on their platform.
Some of these sites might make more money selling ads than selling the actual product depending on their conversion. Just because the site isn't designed for your goal (buying plumbing supplies), doesn't mean it's not making the owner a profit (which is probably their goal).
(comment deleted)
Turnabout is fair play for people like this.
This is why I have stopped reading articles on Washington Post, and other similar article sites. The standard they are creating for the web is not one I can support.
It's not really a mistake. It's kinda like how gas stations rarely give you a clear path to drinks or why you needs to wander through produce, cheese and meat (i.e. Margin) to buy milk or bread.

The plumbing guys are dumb, but you don't know the business... of they are a local outfit they have a someone captive audience who can be abused a bit.

I respond to undismissable email popups by entering abuse@theirdomain.com as an email address. I'm sure their network operations team doesn't want to miss out on the incredible rewards.
I would love a Chrome extension that does this.
Google searches where the first 5-8 results are google search ads? Happens on some obscure searches.
> These outfits have lost sight of what their web site is for.

Their web site is for them to make money and considering that these practices exist, I have a feeling that they make more money by ad-sales than by actual sales of their actual product.

Long-term this is obviously a bad strategy, but who knows whether they even are interested in a long-term strategy.

(comment deleted)
This is actually pretty impressive for a ad company to take steps that are overwhelmingly better from a UX perspective but also directly target online ad revenue models. #bold #impressive #hardProblems #thumbsUP
The article states"

"In short, if a web page puposely hides content behind an ad or forces interaction with an ad"

Does this mean that it doesn't include those nauseating "follow us/sign up for our newsletter" light boxes that plague the web now?

Also why would this only be for mobile? Are they any less of a scourge for desktops?

Look, popups are great for marketing, but this penalization isn't bad from a marketing and UX standpoint.

Popups, while effective, are being overused, which means they will become less and less effective.

On top of that, too many are poorly created and don't work on mobile, making it difficult to impossible to close out. This is unfortunate.

I've long held that we need a less intrusive "popup" that nudges instead of disrupts users. Basically a Hello Bar style that while catching my attention is something I can easily ignore.

I wonder if forbes will have to remove that annoying "Quote of the day page" or if this only applies to JavaScript popups.

I also wonder how google distinguishes between things like floating nav bars and elements that obscure content.

How about Google let me install extensions on Chrome in Android?
Then I can penalize websites for any reason, including tracking me.
I don't think that's possible, I'm not a programmer so I can't be sure but I read somewhere that the way Chrome is built it can't be done.
interesting...I'd like to know more about the technical reasons why not. I should note that Firefox on Android is able to run desktop plugins.
The only thing a web site should need to “measure” is how long its visitors stay. I know that the instant I see any pop-up garbage, I immediately leave: I don’t care what the pop-up might say, I don’t care where they might’ve placed a little "X" to dismiss their message, I simply go BACK and I DON’T return.

Do not allow yourself to be bullied. Yes, services have some nonzero value but your time also has tremendous value and you should not undersell it by putting up with stupid crap. Any site that shoves things in your face is being disrespectful, it is wasting your time, and it is costing you, which is not OK. Let those sites die out.

Most users hate these pop-ups and cheer this move from Google. But let me add a little context to why these ads are so prevalent and why some companies view this move as Google abusing their power.

If you visit to any "guide" website like TripAdviser, Yelp, etc, these days on a mobile browser, you'll notice that the sites often barely let you do anything without downloading the native app. They all but refuse to let you see content and throw up "Download our app!" pop-ups everywhere.

By traditional logic, that seems insane. Why are they putting so many roadblocks between the user and the content? Surely that must be driving away users, right?

The reason for this behavior is that Google is systematically destroying the SEO traffic of these sites by adding their own competitive features to search result pages that appear above organic results.

If you search for a restaurant / hotel / flight on your phone, Google will often show its own custom widgets above the organic search results. It's not unusual that zero organic search results are visible "above the fold". The more Google does this, the more the share of clicks goes to them instead of to organic search results in these types of searches.

That means that even if these guide companies have #1 search rankings for every possible search term, they are seeing their SEO traffic plummet every month because they can't compete with Google's "above #1 result" placement. So as a defensive move, some companies are basically giving up on SEO traffic in the long term and trying to forcefully convert many visitors as possible into users who visit directly via a native app (and thus bypass Google). They know that every web user who doesn't download the native app is ever less likely to ever find them again via a search result page.

So to these companies, they see this change from Google as another anti-competitive move because Google is taking away one of their last remaining lifelines for user acquisition.

Personally, I find those full-page ads super annoying and hate them too and think they should go away. But like anything complicated, this isn't a simple black and white move to benefit users. It's also a strategic move that helps Google and hurts some competitors.

They might as well be more open about this on the actual page.
Kinda makes you wonder if people will stop using Google search if it keeps them within the walled garden of other Google services. It wouldn't surprise me if Google is the next AOL in a decade or two, for reasons including this one.
If AMP is an indicator of their long term strategy AOL2.0 is definitely in the cards.
I think many people are living in a Facebook world. They are clearly AOL 2.0.

I always thought AMP was a way to get some visibility into Facebook sharing traffic.

What's the open-web alternative? I've switched my phone search to Bing because I'm so frustrated with AMP, but the search results are pretty poor to the extent that I manually search google a few times a week.
Where do they expect these visitors to come from, if not from Google searches? It doesn't add up to me. If people can only get to your site from Google by scrolling past Google's own widgets, why make your site worse and thus discouraging them from scrolling down next time?
Presumably they are hoping to build brand loyalty - they want you to open your Hotels.com (or whatever) app directly next time you need to book, instead of doing a Google search
Because the user didn't scroll, the site got lucky and Google didn't have a result for that particular query. It was a miracle, it won't happen again, there is no second chance, hit them with everything.
Brand recognition and loyalty.

People don't use bookmarks, especially on mobile phones. Not many people are going to bookmark hostelworld.com and when looking for a hostel, they're not going to click the bookmark and search on hostelworld. They're going to Google search for "hostels in XYZ", and hostelworld will be there, underneath Google's own recommendations.

With the app though, the icon is conveniently sitting there on the home screen. You're probably getting less new users by giving on on web, but the users that you do get using your app are going to be much more loyal.

Wow didn't know this. Is this also why many e-commerce sites beg for your email every time you visit?
That's more so they can bring you back even if you weren't actively looking for something ("wow, widget x is 50% off - I'll probably need it in the next few months, I should buy it now."). Also to keep their brand in the front of your mind the next time you do actively look for something.
For many the reason is Facebook who canibalized the brands natural reach without paying.

Everyone panicked and good old email is trustworthy. FB holds the contacts in their paid garden that they once so hard fought for.

Marketing teams are usually data driven rather than user experience driven.

How does one counter the argument that "anecdotal evidence suggests people who sign up with us are 2x more likely to convert, so let's force more sign ups to increase revenue".

Do these people ever consider that they might have their causation backwards?
Of course, and this does get pointed out to them.

But it's easier for them to extrapolate data and say, hey, we couldn't get emails of xxx thousand users, so we lost xxx amount, vs our argument that they probably lost a number of users who were annoyed by the popup.

Except, you can count conversions, but you can't "measure annoyance"

Of course, and this does get pointed out to them.

But it's easier for them to extrapolate data and say, hey, we couldn't get emails of xxx thousand users, so we lost xxx amount, vs our argument that they probably lost a number of users who were annoyed by the popup.

The fact is, that you can count conversions, but you can't "measure annoyance". And hence, the argument rests.

Isn't this more of a sign that restaurant/hotel businesses are becoming more SEO savvy, and that aggregators are becoming outdated?

Remember, the aggregators originally dethroned the travel agencies as the 'experts' in connecting people with what they want. Google search is in the business of providing information, meaning if they have the info available they'll provide an aggregated list of restaurants to someone who wants restaurants.

I doubt it.

Google has a long and storied history (since Dejanews) of fucking up user generated content on google properties.

If you're looking for restaurants and hotels, TripAdvisor and Yelp are exponentially better than anything Google is going to give you.

That's the kicker though: how bad is "good enough"? If I ask Google for a donut shop and can pick the one that has devil's food chocolate donuts, who cares if Yelp would've given me better reviews; the basic information is there in front of me, and I don't have to install an app or deal with popups.

I stopped using Yelp the minute they made their mobile site unusable in a pinch while pulled over on the side of the road. It doesn't matter how good the content is if I don't have the time to jump through their hoops.

80% of my searches for this sort of thing is when I'm traveling. If I'm looking for food/lodging, I want the best possible option.
It's not user generated content though.

That's the point - that these businesses are providing enough information to allow Google to aggregate them for a user with intent, rather than just pass the user off to another website aggregator.

  The reason for this behavior is that Google is systematically destroying the SEO traffic of these sites
Another reason is that with apps, they have the opportunity to harvest the phone owners private data directly (contacts, messages, seeing what competing apps are installed, etc.)

Most people don't pay attention to the overbroad spectrum of access many apps demand.

Specific example: Samsung changed the Gallery app (default on my GS7 to display photos from within the Camera app) to demand access to my Calendar. If I hit Deny, it aborts. My Calendar? Really?

How much of a problem is this on iOS?
Most apps will work fine if you decline their extra permission requests, though obviously certain features may not work without them. The rest of your app has to function normally, and you definitely can't crash or otherwise programmatically quit.

There are one or two apps I use that do require extra permissions to function – like the NFL Sunday Ticket app won't work unless I enable location services, because it needs to black-out local games due to broadcast rights.

Some things are simply not available on iOS. For example, your app can't get a list of other apps you have installed, or send/receive text messages.

Of course, every once in a while someone finds a way to abuse a different API maliciously. Apple had to limit the availability of the API used to determine if a URL scheme can be handled by the device because Twitter was spamming it to determine what apps you had installed (if your device can open a 'dominospizza://' link, they knew you had the Dominos app installed). So now you have to whitelist up to 50 URL schemes your app might try to launch.

On Android 5.x and earlier, you had to grant all possible requested permissions up-front during installation.
Check out verify.ly to get some surprising info on what iOS apps are doing behind the scenes. There are plenty of apps that have the ability read your full contact data or your calendar without ever asking you. Skitch, Amazon, and Skype do one or more of those. I have deleted them and gone back to the web app for Amazon. FaceTime replaced Skype, and I'm stuck with Markup in Photos as a poor replacement for Skitch.

Of course there are plenty of safe apps that do not abuse APIs - for example, the Deliveries app by JuneCloud.

" to harvest the phone owners private data directly"

And Google is the king of that.

While you make a good point - it's not like Google can take the high road on this.

Perhaps I should clarify that the "they" I referred to are the app publishers, not necessarily Google.
Here's the thing: I don't care. I don't care if Google's algorithm makes life difficult for SEOs. I don't care if it prioritizes Google's own properties unfairly. I don't care if it hurts somebody's conversion rate.

All I care about is the quality of my browsing experience, which takes a nosedive every time one of these fullscreen mobile ads shows up. As long as Google's algorithm ranks results roughly in the order of how useful and pleasant they are to me - even if that's just a side-effect of their own self-interested intentions - I'm OK. In general I'm not a big fan of egoist philosophy, but in this case, all I care about is me, me, me. Google should keep doing what they're doing.

It might be dictatorial, but the web has so few incentives to not load your website down with a ratchet of ever-increasing invasive crap, that any incentive pushing in the opposite direction is good, and Google is one of the only agents with the might to make that happen.

" I don't care if it hurts somebody's conversion rate. All I care about is the quality of my browsing experience"

In the short run, it won't matter, but in the long run, it may likely affect your browsing experience on the whole.

Google doesn't do anything well but search. So you are trading a slightly better search experience - which may cost you the death of several brands that provide much more than search outcomes.

Trip Advisor, Yelp etc. ostensibly provide a useful service, beyond whatever Google will do. So when those companies go, we lose those greater services.

The impact of Google's monopoly is something we cannot ignore.

It's like 80% of Russians still support Putin, because he 'seems strong' - but Russians also don't realize how much he has de-facto degraded their standard of living. Russians would be a lot wealthier if they integrated into global markets, stop invading neighbouring territories. There'd be massive growth and opportunity for Russians - at least in globalist terms.

> Google doesn't do anything well but search.

I beg to differ. Google also provides great products for - Maps, Email, Browser, Mobile OS, Video, Photo management. If you think neither of those are great and you can think of a better alternative, you have a lot of market share to capture.

Having said that, I concur with the rest of your sentiment and would like to see services like TripAdvisor, Kayak etc flourish.

Disclaimer: hold a lot of GOOG

Google Cloud is also pretty great, but suffered from AWS's First Mover advantage.
But it's very easy to switch to another search engine. I used DuckDuckGo for some days. It's literally 10 seconds to change it.
As an engineer, I have found no other search engine better for programming related issues than Google. I've tried Bing, and DDG (which is my default on my work browser), and neither have performed as well as Google.
> Google doesn't do anything well but search. I beg to differ too. It's increasingly difficult to use google search effectively - it has no concept of content vs sidebar/nav, so searches (particularly technical) often turn up a plethora of unrelated* top ten list blog posts.

They're still the best in the game - but that doesn't mean they're still doing it well. Just better than the competition.

* "term A" is in the body, and "term B" is in the navlinks but has no bearing on "term A". Google search sees that the body includes both terms and include it in the results.

I really wish Google offered some way to indicate that you wanted a strong spatial correlation between two or more words that wasn't just the "I need this phrase verbatim" of quotes and allowed them to still do the smart stuff with synonyms/different forms of the words.
> Here's the thing: I don't care.

You will care when nobody's in the business of providing hotel/restaurant/etc reviews...except Google.

If google does a good eough job, then may be i will continue not to care?
Google has tried repeatedly to challenge Yelp. Every attempt fails, but they insist on hurting Yelp and businesses like it that have provided value over many years. This goes back to Larry asking his teams to answer the users' query on the Google landing page. He wants the entire users' session to be in Google products. These companies are as much at fault for relying on Google so heavily. Google is as much as fault for taking on companies with established products and histories. Google should look to new markets instead of cannibalizing its search business for short term gain.
I think Google has just recently reached a "good enough" state for Yelp like use-cases, while Yelp has simultaneously declined, that I'm nowadays more likely to just use Google.
Yelp used to be good until, among other things, hyperemotional crybabies destroyed it.
I don't use yelp anymore, all the places around me google reviews seems to become more and more popular. Instantly I get nice photos, hours, directions, and just as quality of reviews. The best thing, sometimes a direct link to the menu. Show times are much better than going to fandango etc. I want to spend less time on my phone navigating sites and more time giving options to my wife or friend who's traveling with me
(comment deleted)
(comment deleted)
Counterpoint: this is (mostly) a zero-sum game. If sitewithpopups.com gets bumped down a slot, someone else gets bumped up, and the site getting bumped up is either a.) more relevant or b.) isn't being a jerk to its users. Google's widgets aren't going away; if you're going to be mad, be mad at the widgets, not the pushing-you-to-not-be-awful.

I think we can all agree that these types of overlays are terrible for users. And there are alternatives, like banners and static inline suggestions to download the app. I don't mind when sites put a "Read More" button that does little more than unhide some content, why not do the same thing to invite users to your app?

> and the site getting bumped up is either a.) more relevant or b.) isn't being a jerk to its users.

This is the key point of SEO nowadays. Google basically awards you if you: 1) provide good (better) content 2) have good UX

> be mad at the widgets, not the pushing-you-to-not-be-awful

I think the point was: be mad at both :)

Do you have any data to back any of these claims up? Even if your claim that Google is destroying SEO traffic to these sites is true (which I am not convinced it is) and my site is getting less traffic from Google, the last thing I want to do is make the experience shittier for the less traffic that I do get. Making my site less usable is pretty much the dumbest decision I could make at that point. I should be working on ways to better convert the traffic that I do get so they are more likely to use my product and maybe even download the app one day.

Who the hell goes on a shitty site with annoying ads and decides, hey, I want a dedicated spot on my home screen for this shitty service.

> But like anything complicated, this isn't a simple black and white move to benefit users.

It is to me. Any page that greets me with large pop-ups and "Download our app" messages can sink into obscurity, for all I care. I'm not going to download just any app and trying to force me by making your site deliberately shitty will only make me angry at you. I realy hope this move from google includes sides that slide in a ("get our app!")-banner after a few seconds, just right so I accidantely click the banner instead of the link I wanted to click.

This drives me absolutely crazy. If I'm forced towards downloading your app I'll simply go elsewhere. Its ridiculous to expect your users to download a 126MB app (Tripadvisor's current iOS app size) and grant it all sorts of permissions just so they can search for a hotel once a year.
If I see their "Download our app!" page, doesn't that mean that I either went to their site directly via URL, used a non-Google link, or skipped the Google widgets? How would they increase their revenue by annoying me? If I use a site I only go there via search once.
Kind of scary that Google is so much of a gatekeeper to the entire internet that they can essentially decide which websites get visited and coerce them into submission.