As someone who believes that ad tech is the primary force driving the ever growing consumer distrust of technology, I can’t help but feel a little giddy reading this story.
I have this suspicion that mobile ads have way weaker effect than claimed. No-one in adtech has any interest in finding out. Marketers don't want to lose budgets or importance either. So a lot of money is wasted on annoying people. How sad.
I think the problem is that the overwhelming majority of digital
ad formats just don’t work. Does anyone who works in digital ad tech genuinely believe that these formats work? I have been here for the last ten years and have come to believe they do not, certainly not comparable like traditional linear TV or print in their heydays.
There are exceptions. I think Google search ad works so well it’s eating the product. I think YouTube’s TrueView is great for modern brand advertisers (safety issues aside). But by and large, banner ads, forced video pre-rolls, etc don’t work. The negative message of the format offsets any message you try to convey.
Yes, fraud is a part of this. But I think a lot of this evidence looks the same as if none of their targeted, performance-based advertising was doing anything at all, beside really weak branding.
Genuinely interested to hear from people who disagree.
> overwhelming majority of digital ad formats just don’t work
Doesn't it? It may have miserable conversion rate, limited target audience but it apparently is commercially viable. People would hardly spend money on sending e-mail spam in 2020 if it didn't work.
I thought that at first. But like the original post points out, Uber was spending a lot of money, not just in absolute terms but in percentage terms, and it wasn't working for them.
There's an allure to making something measurable and setting goals against it. Often, those goals take on a life of their own. That's what I am suggesting happened here, I guess.
if you read the twitter thread we see that uber was spending 10x more money than they should have on ads simply because they weren't auditing where their money was going towards.
so there's evidence that people (incl uber, a market leader) do indeed spend money on ads even when it does nothing
Mobile games run on this. They have a target user acquisition cost and spend based on that. It seems to work for some very profitable companies. People shouldn’t try to over read into this. Businesses can’t run on irrationality for very long
Indeed. Mobile games. Perhaps the people who play mobile games probably would click an ad of another alike game or otherwise related content. They usually aren't mindful enough to prefer finding and choosing exactly what they need over just clicking an ad.
Ya for sure. I was a Googler so my examples were biased, these aren't the only ad formats that work but there are fewer than most people (even in ad tech) realize.
However I have not logged into FB in six months because the actual content in my newsfeed is so toxic. I wonder if this will cause things to change, although it seems unlikely.
Mobile devices tend not to have adblockers, so exposure might be a lot higher.
Also, the only ads I've clicked are mobile ads. I've even installed apps via ad before - I've never once purchased an item from an ad online (but I've also run an adblocker for a decade).
Ads certainly "work" by some metric, and some definition of "work", but I think the metric and definition is unclear.
I think CTD ads (click-to-download) are one of the exceptions. It's a great format for some cases, eg games. For game devs, trust that yours is addicting enough that you can safely show your users a competitor's game knowing they'll come back to you. For players, you were looking to play a game anyway and maybe you want to try something new. Click, download and play that instead.
The only ads I click on are mobile ads, and that's because many times the X to close them is too small. Other times the ad is jumping around my screen and it steals my fingerpress.
Instagram ads for millenials are like flies to honey in my anecdotal experience. So many of my peers buy their furniture, dress shoes, clothing, water bottles, watches, dog leashes, pretty much everything in their lives from seeing some like-aged person use that thing on an instagram ad (and the product usually being pretty cheap). It blows my mind how successful these dime a dozen drop shipping companies based on instagram are with millenials and gen z, and buying like this will only be more normalized in the future as more people know someone who got something nice from that instagram ad.
Yeah I've noticed I've even been drawn to something because of the mindless scroll to that's pretty neat, the button to buy it is right in front of me.
I suspect this really only works for more novelty things like clothing that are cheap enough to enough to impulse buy and of interest to image conscious people.
>It blows my mind how successful these dime a dozen drop shipping companies based on instagram are with millenials and gen z, and buying like this will only be more normalized in the future as more people know someone who got something nice from that instagram ad.
Especially with stay-at-home orders due to the pandemic, everyone (well, not everyone... but hyperbole) now has their own drop shipping brands operated out of their basements, all selling the same cheap products off of AliExpress. Will consumers catch on to what is going on? Do they even care?
Digital advertising is super effective for mobile ftp games. It is not a question if they work, it’s a question if the game can earn enough to justify the cost. Companies in that space have a super smart team of economists and data scientist constantly checking when a mobile app is worth advertising.
Someone in this thread said no one has proved to them that digital ads are ever worth it. My experience is that when they are you don’t want to advertise it to potential competitors or the people who carry your ads.
I've been playing a silly phone game for about a month now. Only last week I realized it has ads: I'd become so ad-blind that I completely ignored the banner ad they keep at the bottom of the window at all times. I only noticed it because it flickered when transitioning to another ad.
CEO: "I read those are hot. Here's a bucket of money! Do lots of something so we have lots of growth!"
[spending ensues]
As the Uber story shows, a lot of money gets spent without anybody making sure that it actually works. The actual goal is not to have an effect. It's to be seen as taking bold action. If the business gets better, bonuses and promotions ensue, no matter what the actual cause. Cargo cult management.
It sounds insane, but I'm sure it's rife. Years ago I was coaching at a Lean Startup weekend class. One team really killed it, rapidly testing product hypotheses through real-world testing. Their initial idea was quickly proven worthless, but they listened to users and came up with an idea that would sell.
I kept in touch with one of those team members. He went back to his prominent, well-funded startup, excited to improve his company. But nobody wanted improvement. Execs wanted to sit in a big room, think big thoughts, and produce specs. The engineers were to implement those specs. But nobody would measure the effects of anything. Proof that a grand poobah's idea didn't actually pan out was deeply unwelcome. It slowly drove my pal mad and he quit. The company bumped along for a few years to a modest acquisition, but it never really lived up to the hopes.
Engaging random marketing consultants is a good way to waste money on ads.
However, anyone engineering for advertising from the start of your distribution plans should be able to measure these things. It’s not hard to track which signups came from ad clicks versus organic traffic. The problem is that most companies start out focused on their apps and simply assume advertising is a totally independent function that happens elsewhere in the company. Invest some time into including advertising as part of your signup process and it’s really not that difficult to track LTV of customers who signed up via ads.
In fact, that’s how Uber confirmed this fraud. They noticed that some of these fake customers were clicking on the ads and getting signed up within seconds, which is something only bots can pull off. If the customers you’re getting from ad signups aren’t actually spending money in the app, you should know about this from basic analytics.
Even if you do this the ad-people will claim that showing people ads repeatedly will make them buy your product eventually. Maybe when they actually need such a product.
At this point you cannot connect the events anymore.
This effect is probably weak but not falsifiable.
Wow, that’s quite a revelation. Uber turns off 2/3 of ads and metrics are the same. Eventually, with optimizations, they got down to 10% of previous spend without loss of revenue. I wonder if the Uber story is unique or it cuts across all advertisers. How would this affect Ad Tech industry?
Wikipedia is constantly running ads. They're just "charity" donation ads so you don't think of them as such, but turn on the TV and you'll notice that charities advertise themselves to get donations all the time. It doesn't make it not advertising.
As for HN, how long have you been using it? Are you aware it's ad supported? Some stories are ads, usually for hiring.
You are allowed to think differently, but personally I'm fine with the banners on Wikipedia (except IIRC and AFAIK they pretended they were short on money while in reality they were expanding).
I'm also fine with YC running job ads on YC.
I'm however not fine with 100 - 1000 [1] ad and tracking networks pretending they have reason to log my browsing habits on a typical site.
[1]: based on copying the list from "cookie banners" to Libre Office Calc and checking how many lines I got.
The ad industry has always claimed that fraud doesn't matter because it is priced in. It's been over a decade since I worked in click fraud detection but I believe Google's numbers were around < 2-5% of clicks were fraud (someone here might have up to date numbers). So this is a huge revelation from my perspective.
Working in iPhone games, our biz dev people made us integrate with the shadiest of ad publishers. But we required attribution and could track how much the users spent, so could stop spending on them quickly, which was the normal result. Most of the companies were surprised we wanted attribution and had to build something custom, which to me means at the time nobody else was tracking it so lots of customers must have been getting fleeced. From the tweet, it’s crazy that they were spending 100m without tracking the quality of the users.
Also worked in ad tech. There were some types of customers that cared about performance, but the largest were typically agencies and all they cared about was spending the budget they were given. Everybody knows this is happening, you can differentiate yourself slightly by doing better detecting it. But at the end of the day people really care about customer acquisition cost which includes the fraud, so if you had 0% fraud everyone would just raise their prices or margins
> Also worked in ad tech. There were some types of customers that cared about performance, but the largest were typically agencies and all they cared about was spending the budget they were given.
Large companies call it "brand" spending. Which is to say they're spending to maintain brand awareness. Measuring ROI would call the practice into question.
> From the tweet, it’s crazy that they were spending 100m without tracking the quality of the users.
They were high quality users: Uber didn't pay for installs, but for installs + first ride. What's being alleged is that the ad networks were detecting a user installing the app, and fraudulently creating an ad display event to make it look like the install happened due to the ad.
Depending on the content of the ads, I could easily believe that anyone who is remotely the market for even the occasional Uber ride already has the app on their phone and, if they want a ride will either use Uber, whichever ride-share is cheapest, Lyft if it's available because they don't like Uber, or a taxi/limo in situations where that's more convenient.
I guess you can offer promotions but then you're giving money away.
I'm not surprised by this a all. I do PPC for a living and I find that a lot of ad spend is totally wasted on paying for clicks that you would have gotten anyway, especially on Google and on Mobile devices.
A lot of big brands bid on, and pay for, their brand terms because someone at some point told then that they should. So when a person uses search to find Uber, they get an ad first, then a regular listing second. And almost everyone clicks the first link. Take away the ad and the free click is first.
In fairness, I've worked at several places that engaged in this practice, and they're perfectly clear that they're paying for clicks that they'd likely get for free.
The theory is that if they don't do this, their competitors buy this ad space and steal some of their organic traffic, and/or that they're just bidding up the cost of that space to make their competitors less efficient.
Now, that theory may not be cost-efficient in practice. But it's not like the marketers didn't understand that someone searching for the name of our app were going to see it in the list anyway.
Would you be willing to make that decision knowing full well that it could reduce revenue that is 100% traceable to that decision? I mean, obviously there are ways to make this politically acceptable in your organization, but you better be completely confident that it won’t bite you in the ass.
You can just turn the ads off in one small geography and/or for a short time. Seems to me the potential benefits far outweigh the risks here. If your organization can't see that, then you have bigger problems.
If the company ranking results and selling these ad spaces are the same every decision will not be in case of advertisers/users. Google has long lost it‘s spirit and is only optimizing for money.
DHH of Basecamp had a long twitter thread about this. Their competitors bought “basecamp” as a keyword for google ads, effectively forcing basecamp to pay to stay #1 on searches for basecamp.
Google has/had no policies restricting this, unless of course, you guessed it, the keyword you are squatting is “google”.
This being allowed shows the sad state Google is in - you search for a brand, get ads for a competitor? Search engine, my ass. You now effectively need to pay Google to keep your product relevant through ads, even when the user is specifically looking for you.
Certain types of product have become directly linked to a product name, yes. Google is smart enough to know this, though. If I google for my non VC funded product name, which is unique, why should the first result be for a direct competitor with deep pockets in advertising?
This would be fine if Google embraced their dependance on ads, but Google still labels itself as a “search engine”.
Isn't this more of a protection for established players? ie if your product is a GitHub alternative, should you be allowed to buy ads targeting ppl who search for Github to build awareness?
If I search "Kleenex" on Google, I'm asking Google to find me Kleenex, not some ads. If Google is going to start listing things in highest-to-lowest bid, then it stops being a search engine and starts becoming an auction house of links.
I’m not sure if this is equivalent to fraud. But I’ve recently discovered that 99.9% of the website visitors to my new startup’s website are just random @gmail addresses from India and China - most of which are coming from Google ads. We don’t spend much, about $100/m mostly to keep our #1 ranking in Google (which I think helps), but it does seem fraudulent. If I turn the ads off, the traffic from those IPs drops off a cliff.
With Google ads you can select the countries where you want the ads to run in. If you go worldwide and chose to maximize clicks, Google Ads will mainly run them in India and other emerging countries because it's much cheaper to get clicks from there.
I've been suspicious of this in my own campaigns as well. Google Ads lets you drill down to specific cities where your ads were clicked. Very often, Ashburn Virginia shows up very high on the list. Ashburn is where AWS's us-east region lives, and presumably also where many VPNs operate out of.
I simply ended up excluding Ashburn from our campaigns for this reason.
Google Ads default settings for location targeting is based on "Presence or interest" in a location, not whether the user is actually in the targeted country. That means even if you have "United States" selected as your target, your ads will still show mostly to people outside the US who "show interest in" the US.
To fix it: Campaign Settings > Location > Location Options, and under "Target" select "Presence". Never use the default "Presence or interest" option, as it will result in exactly the scenario you're describing.
This is my #1 pet peeve with Google Ads. Using the default setting is an incredibly common (and expensive) mistake people make setting up new campaigns.
By default Google will show ads to people in or "intersted in" the selected location. You can change this to only show ads to people actually present in the specified location, from campaign level location targeting. I spend low 5 figures per month on ads and changing this parameter has significantly reduced poor quality clicks.
I also had a small Google ad spend ($350 over 3 weeks), that I setup to run exclusively in the USA, and had the same experience where IP addresses recorded by hotjar were all from India, all new signups from the ad spend were '.gmail' email addresses and although I had about 100 signups not a single one entered a credit card.
Yeah, same here. We have done some spam mitigation, but we still get so many bogus sign ups and form submissions. In this instance, who is committing the fraud? Is it Google to ensure you spend your entire budget? I don’t see anyone else benefiting from these bots and click farms. We’re so small and new that it can’t be a competitor.
Almost the same here. Had an Google Ads budget of $200 a month. All I got from this were dubious sign ups, which all together never looked around in the application and never came back. As my ROI turned to be negative, I turned off ads completely. My organic sign-ups even increased since that. And those people are staying!
I think Uber is to blame here, reading the story it is obvious that they bought ads from very shady sources/networks, probably they went for the cheapest CPC. You get what you pay for.
On the other hand, if you pay someone $20 to buy a brand new ApplePhone 12 XS Max, you should expect to not get the phone. The Twitter didn't mention anything about price; and it is possible that the fraudsters were actually charging a normal market rate; and nobody deserves to be defrauded for trying to get something for cheap; but if Uber was taking the cheapest option then it's hard to feel sympathetic towards them - they're a big company with lots of marketing people and they should know better.
That's what happens when your own job is to waste oxygen, pull numbers out of thin air and take credit for "conversions" that were organic in reality; in this case you have no incentive to optimize things because you'd inevitably show the company your own position and maybe entire department is irrelevant and a waste of money.
I really can't feel bad for someone who wants to pay 100M to annoy people and waste their time. The "fraudsters" here have done a service to society: Uber wasn't satisfied with the free, organic growth they were getting and wanted to be more greedy and pay 100M to litter the web with ads; the fraudsters took the 100M and cleaned up their littering by "consuming" these ads with fraudulent clicks while giving Uber the outcome they wanted, albeit one they would've gotten anyway had they not been greedy.
Tl;dr: money is at best a halfway decent accelerator. And as long as you don't massive positive retention just a sinkhole of fake growth. And if you have this massive retention there are better ways to invest your money.
There was a freakonomics podcast recently about advertising (online and traditional).
No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.
> No one is willing to run the experiments necessary
The people that would have the power to run this experiment have their entire careers depending on things staying as-is. Running the experiment carries a significant risk of exposing that the advertising operations they're responsible for provide much less ROI than they pretend it does.
The unwillingness of anyone to run such an experiment is already an answer. Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
> Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
Because everyone is already acting like they know it works, so the only way that experiment can change things is in a way that's bad for the person in question. In that situation, they should (from a local, selfish perspective) be resisting even if they're awfully sure it does work (and perhaps even if they're right!).
Given that, I don't think the behavior has already given us an answer.
Facebook's ad research team have run a lot of experiments.
The goal was to demonstrate an impact of in-store sales from internet advertising. They did the first studies in about 2008-10, and have continued running these studies ever since.
They even built a tool so that advertisers can run these studies, and get a sense for the incremental impact of their ad.
Google have a similar (less full-featured) system.
Really, it's the rest of the ecosystem that has much of the fraud, and I think that a lot of people in the industry are aware of this.
> The goal was to demonstrate an impact of in-store sales from internet advertising.
So the goal was to advertise advertising. A more scientific goal would be to see if there was an increase of in-store sales from internet advertising. Instead they were looking to design experiments that would show a positive effect, with the goal of giving advertisers a dashboard so they could run those experiments themselves.
To let those advertisers do the experiments themselves, and to relate the campaign to in store sales, would require Facebook and google to share a ton lot of internet-user tracking data
> To let those advertisers do the experiments themselves, and to relate the campaign to in store sales, would require Facebook and google to share a ton lot of internet-user tracking data
I think you are misinterpreting what "do the experiments themselves" implies.
Certainly there is a JOIN with Google's or FB's data, but it can still happen entirely in a hosted sandbox with no data sharing necessary.
Heh. Just today, FB served me an ad claiming to be Macy's and talking about their closeout specials, accompanied by dozens of mourning comments. It was all phony.
We've run Google's Local Campaigns, which are supposed to drive brick and mortar store visits, which they measure using GPS and "other signals". I checked with one of our most remote and least frequented stores, to see whether they could see any of the 100 daily visits that Google claimed to have generated, and they couldn't see a thing (avg daily visits were around 80 before during and after the campaign)
Not really, scientists for example also have their whole careers based on the truth of some theories that they use. However, they're willing to put them to proof in different ways. The reason they do so is that they have a high degree of confidence that these theories are true. This cannot be said of people doing advertisement.
I’m not exactly sure what you’re trying to say. Scientists’ whole careers are based on running hypotheses to prove or disprove their theories. That IS their career.
And disproving a theory with lots of research backing it would be great. Imagine if someone found a huge hole in general relativity. There would be a boom in the grant writing industry.
The reproduction crisis in sciences suggest the self- interest is pretty wide spread.
And the social stigma in science of being someone who tries to take down, discredit, or disprove your colleagues/superiors/ competitors theories is pretty substantial: IME there has historically almost been a taboo against attacking our speaking negatively about publications and your own sciences + faculties practices.
The saying of sciences advancing one funeral at a time doesn't exist because they're all such great skeptics and falsifiers, and current scientific practice is heavily biased towards positive findings and contains general publication biases.
indeed there's actually a LOT of common ground with advertising self-interest, since a lot of publication in science is effectively just advertising your brand...
If you really believe truth is relative, walk to the grocery store tomorrow without wearing clothes and convince yourself that it will be fine. Or run a lighter under your hand - maybe it won’t burn today? Of course you wouldn’t to those things though because, at the end of the day, we rely on truth for everything we do. We just don’t always know what it is. Relativism is not the same as “it’s hard to figure out because it’s so complex”
Are you implying that the probability of someone going to a store named and getting out of it without harm is zero? There’s plenty of photographic and video evidence of the contrary. Probably not the best example.
Same for a lighter, there are many factors involved, and playing with lighters that way was a party trick where I grew up.
We use truth as a concept every day as an approximation, but the universe is not bound to follow. Very unlikely things happen all the time.
Truth is an unhelpful concept anyway (in science at least). What’s more useful is the likelihood to get a given result from a given experiment and the ability to make predictions.
And yes, when we put things that way it brings a lot of possible issues with sampling, processing, and measuring (and who’s doing the measuring). Some of these things would be harder to control for in a study of the effectiveness of advertising.
This is a dangerous misconception to be spreading. It is not at all related to your other claims, namely:
> Sampling can skew results. Scientists are people too. We all have flaws.
These are of course correct statements. But they do not influence truth. They may well influence people's understanding of what the truth is, but that's different.
I work in the field. Incrementality testing is a big part of marketing measurement at any reputable agency. Any claims to the contrary are FUD.
That said, companies like P&G, Airbnb, and Uber, which are oft-cited as examples of digital not being worth it, fail to understand their own brand recognition and organic power, built through prior marketing efforts, as key to their current standing.
Sure, TODAY, it doesn’t have the impact they’d like it to have but the investments PRIOR were key to ensuring their success.
Brand is a lot more than marketing spend, and not all brands are equal. Google and Facebook have companies that depend primarily on performance marketing spend over a barrel.
I worked next desk to people running a small ad agency. Because we shared office (and I found them an intern), I got a very good look at how they're working. What I've seen can be summarized as: people who have zero clue or interest in statistics writing "reports", with "graphs" they don't even understand beyond "pointing up = good", proving positive ROI to customers - who also have zero clue or interest in understanding the numbers in the report, and not enough visibility into the whole funnel to independently check attribution. Both the agency and the clients were engaging in a shared and completely unjustified fiction of positive advertising return - and as long as both sides were happy, the money kept flowing.
I've been long since suspecting a lot of advertising on-line looks like that. Every now and then, I see evidence in favor. Like that good ol' Optimizely debacle, where it turned out Optimizely was structurally optimized to help people make invalid A/B tests, that erred on the side of concluding the interventions were working[0]. And sure, big brands with some superstar ad teams probably do this right. But I think there's enough slack in most businesses that advertising spend can get quite far detached from actual ROI without anyone noticing (and with plenty of people happily riding the gravy train).
Have you ever engaged with the large advertizing holding companies? They have teams of HIGHLY acomplished data scientists, boutique vendors for specific niches etc. Even if you don't pay for it there will be some people skilled at statistics looking at reports. If you pay for it, there will be data scientists doing analysis.
The large ad firms deliver a very good product but it usually isn't cheap.
I haven't, and I don't doubt that's true (at least in many cases; I've learned not to underestimate the chances of big companies ending up running scams either).
I've personally dealt mostly with the other end - the individuals and small agencies - and what I saw revealed total lack of necessary competences for the reports to be corresponding to reality. Perhaps it's understandable - after all, people who have the required knowledge likely end up working for big advertisers, or in entirely different fields. But small business owners don't pick these big advertising companies either.
It would be fascinating if true if the big "brand name" companies whose entire business is based on charging more for the same commodity product as a generic store brand, but using advertising to justify the higher price in the customers' mind, weren't buying ads from people doing the same thing to them...
The question isn't are the scientists good, the question is if the vendor is honest.
It seems like it would be important to measure this standing and turn down spend once it is reached. I can see how there's a lack of incentive to help large advertising spenders understand this.
> Incrementality testing is a big part of marketing measurement at any reputable agency
In a huge number of cases these tests are run by people that don't have the statistical background to property run and understand these tests, in the remaining cases there is almost never follow through to ensure that the results of the test have persisted after the experiment.
People will say "oh this test shows 10% improvement, and this one does too! and this one! and this one!" But then you should see a nearly 50% improvement just because of ad spend and you almost never do. Nobody ever check that all the numbers add up, they just want the numbers that someone reported to feel like they are sound enough to hold up to scrutiny but scrutiny is never applied.
A small tech team investigated fraud on our platform and developed a system that was pretty robust at detecting and potentially shutting it down. But literally nobody was interested - even the people advertising don't want to know.
The people spending money are typically networks, media buyers, ad agencies, etc, far removed from the actual brand.
There are so many parties who want a slice of the brand's cash that they are all long past caring about whether the ad is being viewed by a human or not.
I worked in adtech. I built a simple ML system that detected signals of fraud on our network. When I reported my findings execs told me the group that was facilitating the fraud was our only profitable division. Soon after, our company went bankrupt. My final paycheck was months late.
IIRC our system wasn't even fancy ML - just finding known user IDs with identical timestamps over many simultaneous clicks / impressions on really diverse publishers.
Uber was just incredibly incompetent to not audit their ad spend at all.
I worked at an ad company. It was an absolutely standard metric to eg geo-fence ads out of a state or two for 3 months to demonstrate the impact of ads. This isn't easily externally visible, but tests like this are standard practice.
Particularly in the app install space, which is sketchy as hell once you stop buying from the top handful of vendors, buyers should be auditing by a couple million in annual spend. To get to $150m without looking hard at big chunks of their spend is just plain arrogance and/or incompetence.
I partly work in this space, and I can confirm that geo-fencing over a time period is the probably the most transparent and convincing (to the client) way to do this. Essentially you mark out local regions (as small as zip codes or as large as states, or something else in this spectrum), and ensure that your advertisements only show within certain "fences". And then you compare old and new numbers within those fences.
The technical aspect aside, there are a lot of "soft" factors that help: regular communication of easily consumable numbers/graphs/metrics to the client, calling out inconsistencies, etc. In other words, this is far from a fire-and-forget exercise; this can be an intense monitoring and client-engagement exercise.
The regular client engagement sometimes helps you in pinpointing cause-effect in the observations too. For ex company X might decide to advertise less of product Y in a certain state Z because of new laws there. But the team you're interacting with might not be aware of this change, or might not be cognizant of its potential impact on an advt. campaign. Regular dialogue helps here since you might observe a change in sales trends, and bring it up in a meeting - and the client team might be able to then rationalize the change. This is healthy for both parties.
It’s like saying that no one who works on reliability of the systems is willing to run experiment to throw 100% errors for a month, because running experiment like that may show that reliability of the systems doesn’t matter.
And then using data from a one system that went down and nothing happened as a proof that systems reliability doesn’t matter at all, and it’s huge scam by engineers.
> Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
Because advertising in some form certainly works. If you can determine that approach "A" that everybody is doing is actually a waste of money but approach "B" is effective, then you can develop services around approach "B" and market them based on these findings.
Nobody is arguing that advertising doesn't work at all, but my argument (which I think a lot of other people agree with) is that the effectiveness of online advertising (and maybe advertising in general) is overblown.
> The people that would have the power to run this experiment have their entire careers depending on things staying as-is.
Any large company could invest in some experimentation, whether their marketing directors want it or not. It makes sense that at least a few do but just don’t publish the results
> It also happens to be the sole form of revenue all of the largest tech firms enjoy
I don't think this is true unless you qualify it a bit. Apple, Amazon, Microsoft, don't make most of their money on ads. Even Google has other revenue streams.
It's exactly my point. Outside of Apple, all the biggest tech firms are 100% reliant on a form of income that is, at best, partially fraudulent. More realistically, almost entirely so.
> The unwillingness of anyone to run such an experiment is already an answer. Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
Or they have run the experiment and the results haven't lines up with their own personal biases so they were discarded.
> No one is willing to run the experiments necessary.
Not without reason. Even without the conflict of interest that Nextgrid points out in a sibling post, there's still a significant financial barrier to attempting to measure this stuff. According to a former professor of mine who spent a large chunk of his career studying this stuff, the size of study you need to conduct in order to get any kind of statistical power at all on an ROI study is just absurd. See, for example, the treatment starting on page 15 of: http://www.davidreiley.com/papers/OnlineAdsOfflineSales.pdf
Gabriel Leydon of Machine Zone spoke to the general topic at Code/Media back in 2016. Basically discussing that they'd gone through the trouble of building internal expertise and tools for optimizing their ad spend to better ensure specific outcomes they desired/required in a way that would inevitably lead to more sophisticated ad buyers and putting a nail in the coffin of traditional media advertising.
Alternatively, they dumped hundreds of millions of dollars into ads based on a wildly unrealistic notion of a customer lifetime (much like the rest of Silicon Valley).
I didn't say they achieved their goal. Though I do think it was/is an admirable one. It would be nice if marketing budgets weren't a near limitless accountability-free money pit.
This made the rounds in adtech in 2016 and it's really nothing special. His entire speech comes down to a single quote: "media will be quantified".
Sure, everyone wants that. Precise attribution has been the holy grail for a long time and the struggles are far larger than just a few technology products. The new battleground is first-party data and clean rooms vs privacy regulations. And that's after dealing with all the politics and perverse incentives that happen in such a massive industry.
It's unclear how this experiment would be done. In the case of brand advertising, it's likely that brand awareness would decay over some period of time and in turn purchase behavior would change.
It's not currently possible to run an A/B experiment with a hold out group of potential customers across all channels, let alone for any longer duration experiment. So how can we separate cause and effect? (although pay per conversion channels do get gamed left and right)
Come up with some new product that requires some personal data for usage (eg. age, gender/sex, address). Start to advertise this in just one country to one demographics, and look how many out-of-target orders you get.
Maybe it's even enough if you simply just sell it via mail order, you can then look at the addresses.
There's probably a natural information spread in any market (word of mouth, trade magazines), and there's probably a physical dispersion of the target group of people too (people move, visitors/tourists saw the ad/product and order it at home), but it still should be a valuable to see how much effect just one campaign has.
Maybe one of the best products for this could be a car. They are pretty standard, really not much difference between them, they are in all price ranges, and regularly new models come out. Advertise one in a few major US cities but don't in others.
For most vendors, not possible to accurately target age/sex/address online. They will certainly sell you the option, but they won't tell you how accurate the data is - or they will provide it a high accuracy with extremely low coverage.
To see this in action - spend a day watching pitbull videos on youtube and see how many spanish language ads you'll get.
Traditional brand advertising testing (TV, newspaper, etc.) would be geography segmented as I understand it. So half the cities got the campaign and half didn't. You can mimic that with IP based geo-location although you'd get more leakage than pre-internet.
This is incorrect. Companies do the experiments. They just don’t publish the results. Why would they? The idea that Amazon doesn’t know the ROI on their ad/marketing spend is laughable.
Because 50% of advertising spend is wasted, but it's hard to figure out which 50%.
If you're doing an $XY,000 ad campaign, it's a waste of your time and money to figure out which 50% it is.
If you're doing a $XY,000,000 ad campaign, you employ people whose job it is to figure out which 50% it is. The thing is, you usually don't go ahead and pay them to blog about it.
For example we can probably agree that for completely new companies spending on ads makes sense. Or giving out free samples, etc.
Similarly for big companies doing media campaigns to keep the new ones at bay makes some sense.
Even if word of mouth is a thing, even if there are organic searches, and even if it seems like a race to the bottom if everyone just tries to outspend each other.
It'd be great to make experiments about how to sensible prevent/regulate this ad arms-race. But first better data privacy laws.
I don't understand this line of reasoning. P&G, Unilever, Cocacola, etc have never, not once in history, had a gung-ho C level exec who said "Screw it, I'm going to find out if our advertising works". And then either found it works and kept spending, or found out it doesn't work and saved literally billions of dollars.
There is so much money at stake that could be either saved or generated, its simply not possible that no one has looked at it. I used to help Pepsi/Fritolay set up tracking to tie advertising on youtube to in-store sales. They spent millions of dollars to measure their ads, Google had a clean-room data center specifically for pepsi/frito. The idea that no one actually checked if this system works is simply not possible.
For brands that large the majority of their spend isn't campaigns specific to new products, but overall brand management. The experiments would need to run for years or decades and if the prevailing belief that these are Red Queen's Races is correct, the cost if they're wrong could be the whole company.
Fortune 500 is a cesspool of insane inefficiency balanced by equally ~~insane rent seeking~~ insanely secure revenue. The sooner everyone understands this, the better.
The sad thing is it's not inefficiency relative comparable things. But anyone that has worked at these places or sold B2B to them just knows it on intuitive level that they are garbage, and need a heavy sedative to think there are no alternative.
> not a predictive theory nor a powerful explanatory theory
It's not. It's about letting go of some efficient market ideal and then finding new ideas.
We can look at Fortune 500 case-by-case to learn new things
> Cola cola
Sugar drug cartel. Despicable business with very stable revenue despite being a net drag on society. (At least "regular" drugs have a lot more upside!)
> Proctor and Gamble
Just as restaurants are reaching down market, and the inefficiency of everyone cooking and cleaning is starting to have market implications, we should see their reign finally dwindle. Wash-and-Fold should follow laundromats. The specialization means that stupid differentiation between products for uninformative consumers (c.f. https://en.wikipedia.org/wiki/Monopolistic_competition) should go away and restaurants and laundromats optimize.
Personal soaps and cosmetics (of course many soaps are cosmetics) however will stay as cultural reasons ensure people will continue to clean themselves and not contract that out for the foreseeable future.
Thanks for elaborating. I don't think I disagree with many or most of the points you've made.
It might be interesting to define a systematic and testable definition of organizational efficiency relative to an ideal market. Just like combustion engines can be compared to their theoretical maximum efficiency, it would be interesting to compare a particular organization against its maximum potential.
Here is another angle on the topic of relative efficiency. Consider a particular organization. Coca Cola will do as an example. Given its environment (financial incentives, regulations, cultural norms, etc), are we surprised by its behavior? This raises the question of how much an organization's particulars (e.g. leadership, history) affect its efficiency.
I suppose I take a systems view of it. Many deplorable businesses survive in niches. But can you "blame" them? It is a matter of perspective: organizations, like life, adapt to their environment and modify it to their liking. Such organizations can stretch the environment to or past its breaking point for a relative long period of time before having to deal with the downstream consequences.
> the inefficiency of everyone cooking and cleaning is starting to have market implications
Elaborate on your reasoning here? While I can maybe imagine a more centralized food preparation system being more efficient, the currently growing market solutions (DoorDash, etc.) are certainly not, and I can't think of a system in which transporting clothing somewhere for someone else to clean would be more efficient than an in-home washing machine. The wasted energy alone would be colossal.
Effectively every restaurant in the US has to pay either PepsiCo or Coca Cola. Similarly, if you want to buy a non-alcoholic beverage at the grocery store it's mostly down to those two (with Dr. Pepper having a much smaller but still significant stake). Any competitor that emerges just gets bought up by one of the three.
Apple's half of the phone Duopoly. Either you pay them or pay Google if you want a phone and want to buy software for it.
Yes. But in fairness I retracted rent sneaking as a combination of monopolization, bonafied rents, addiction, and just shear damn inertia contributes to the malaise.
Calling it all "rent seeking" is not a hill I want to die one.
Restaurants only pay because consumers demand it and would not visit the restaurant otherwise. That’s not rent seeking. That’s market forces. And what they pay correlates with how much their customers demand coke/Pepsi.
The rent seeking is buying up competitors until its a duopoly and then protecting that through shenanigans like exclusive agreements. It has nothing to do with consumer demand.
Buying up competitor does not immediately make it a monopoly or duopoly. Especially in industry where barrier of entry are low.
Exclusive / Special agreement with serving only one kind of Soda ( or no Pepsi where there is Coke ) are Anti-Competitive arrangement. Not Rent Seeking.
The exclusivity contracts are what some consider to be rent seeking. I don't know if this is the correct term, but I do believe they are anti-capitalist and shouldn't be allowed in a capitalist economy. If freedom of choice is so important, than large businesses shouldn't be able to use their ability to corner the market to force you into exclusivity agreements.
I'm not sure this is quite right. I would have thought that most restaurants very much WANT to sell sodas, as the margins are enormous--much, much higher than the margins on cooked food.
This page [0], for instance, says the cost of goods sold for the soda itself is a penny an ounce for the syrup and CO2. Iced tea is apparently the margin champion, as the same page indicates it can cost as little as a penny per glass.
Coca Cola sells water with flavouring and quite astonishing amounts of sugar. Apple sells nice devices made by what might as well be slave labour.
Both are propped up - actually maintained - by huge and vastly expensive brand management strategies.
The criticisms of individual campaigns here are missing the point. It's not about micro ad spend, but the perception of value and manipulation of behaviour created by the combined effect of multiple PR and advertising efforts - which include traditional print and TV/radio ad campaigns, guided advertorials disguised as news in the MSM, interviews with prominent figures, political campaigns of more or less obvious relevance to the core business activity, state, national, and international political lobbying, direct political campaign donations, advertorials masquerading as "freelance" journalism and blogging, managed astroturfing on social media, shareholder relationship management, product placements in movies and music promo videos, articles about commercial visual design elements in trade journals. And on and on.
That perception of value is - unsurprisingly - extremely valuable. And it's very much a US way of doing business. Instead of producing products that are inherently superior, produce something that is functional but glossily packaged, brand it as a premium lifestyle commodity, and charge exorbitant prices for it.,
The prices are traditionally far out of proportion to its actual utility. In fact real utility may well be negative - see also diabetes and any number of other health issues, depression associated with social media use, debt-driven spending on lifestyle products. Etc.
So the rent seeking comes from a kind of cultural squatting. There is value in dominating discourse in all of these different ways, because discourse and narrative define markets and ultimately control behaviour. And while this is happening other kinds of discourse - which may well have more real utility - are diminished at best and crowded out at worst.
So in this case it doesn't matter if Uber "wasted" their money. Uber have their own branding thing going, and explicit ad spend is a tiny part of that.
And even if all online ad spending ended tomorrow, a small number of corporations would have a difficult time for a while, but the marketing industry as a whole would inevitably interpret the change as minor damage and route around it.
Because the ideology of capitalism is "cooridation failure, whatever", It sure would be nice to find a way for one of those companies to freeload off the culture-shaping the others do, and bring the whole enterprise crashing down!
> Both are propped up - actually maintained - by huge and vastly expensive brand management strategies.
In the case of Coca Cola, I agree - in the case of Apple, not at all. "Keyboardgate" aside, their products are vastly superior to the competition in build quality and life time. A typical Windows laptop is unusable after two, three years (almost all are made from plastic which breaks or looks extremely ugly rather sooner than later), and has next to no resale value while in contrast even old cheesegrater Mac Pro's fetch many hundreds of dollars today. iPhones are supplied with firmware updates far longer than most Android phones outside the flagship Pixel line are. OS X is, while it has gotten locked down a fair bit over the last years, still way better than Windows as it doesn't do tracking bullshit and start menu ads and decently better than Linux because no matter which device, stuff usually Just Works (tm) without having to fiddle for hours to get something as basic as a Bluetooth headset working.
Apple built themselves a loyal following with pure quality superiority over the competition, unlike HP or Dell who are only surviving because of enterprises who follow the "no one got fired for buying IBM" line.
And that doesn't even touch that it's Apple who's at the forefront of innovation. iPods, iPhones, iPads, Apple Watch, AirPods, now the M1 CPU - these entire device classes were created by Apple. When was the last time you heard about something truly innovative from the Windows/MS/Google side of IT? Only thing I can remember is the Google Glass.
What Apple is doing is not a "brand management strategy" per se - it is delivering actual value and innovation instead of rent-seeking. And for what it's worth Tesla are doing the same thing.
Apple isn't repairable. Apple devices have high prices simply because of the brand ecosystem. Old Thinkpads use commodity parts - but they command good prices because they are repairable.
Marketing the new thing is what Apple does. It is good at it. That is not the entire spectrum of customer value.
Thinkpads are pretty much the worst form factor for a laptop that doesn't transition into "toy" space.
Source: forced to use one daily. If I could get a Mac laptop running Linux but with the input smarts of the Apple trackpad and keyboard, I would in a heartbeat.
That shows that 1) people are different or 2) you are a fan of Apple hardware. It doesn't show that Thinkpads are bad. IMO they are light-years ahead of anything Apple sells.
My trackpad is automatically muted/unresponsive while I'm typing. I suggest you figure out how to get palm rejection to activate/work on your setup, and to look if a different Thinkpad series might better fit your body (ergonomics wise).
>A typical Windows laptop is unusable after two, three years
This shows that you are just spreading FUD. A Windows machine is no slower after 10 years than it is the day you bought it. Sure you might have added so much software-crap that the software runs slow (which is easily fixed) but the hardware is exactly as fast after 10 years as Apple hardware is. Do you think Apple builds their hardware out of magic and unicorns?
> A typical Windows laptop is unusable after two, three years (almost all are made from plastic which breaks or looks extremely ugly rather sooner than later), and has next to no resale value while in contrast even old cheesegrater Mac Pro's fetch many hundreds of dollars today.
By "typical windows laptop" do you mean a cheap model where the macbook equivalent is buying nothing at all? Of course it's going to be lacking in that case.
Well-made windows laptops exist, and hold plenty of value. But you have to pay for that, no matter what brand you choose.
> tracking bullshit
Like checking signing certificates by pinging servers almost every time you run a program?
This analysis excludes glossy packaging and status signalling from the inherent superiority, when (in terms of overall market value) they're part of the superiority.
And in Coca-Cola's case (among others) the advertising is much of what creates the value. When you know it's a Coke, it tastes better. I've had a lifetime of absorbing ads that make me think of good things whenever I know I'm drinking a Coke.
The answer is not complicated, and the logic on this board is strained by people I think that don't work in marketing.
1) Advertising works.
2) Coke, a sugary drink, is largely kept alive by smart marketing, that involves mostly advertising in the end.
3) Much of advertising is quite wasteful, because it is a fuzzy instrument - but on the whole, good marketing works.
4) There is a lot of self-awareness in the industry on this, and no doubt, a lot of borderline fraudulent actions by participants willingly spending money they know doesn't work.
5) Ad networks will happily sell you ads that don't work, and look the other way when there are shenanigans.
The problem here in the thread, is that people are having a hard time grasping how all of these things can be true at the same time, but it's not that hard really.
That some ad spending turned out to be 'not useful' is 'not news' when everyone knows that easily 50% of ad spending is probably wasted, we usually just don't know which 50%.
>2) Coke, a sugary drink, is largely kept alive by smart marketing, that involves mostly advertising in the end.
Not sure I agree. I haven't seen any Coke Cola Ads for years, not online digital Ads and I rarely consume any traditional media. But I will still buy Coke Cola every once in a while.
They are also on all of the convenience store with decent positional shelf space or visual merchandising. Something I learned that no one on HN actually knows anything about during the discussions of App Store.
Edit: All of my friends can taste the difference between Coke and Pepsi. It is just different. So I was surprised to read comments and youtube video about "blind tasting" on the two. No, I didn't drink Coke because of Ads, I drink it because it taste better. (And some people prefer Pepsi )
Your personal situation doesn't represent the market, moreover, you spent 20 years watching coke ads growing up and having coke, it's impossible to tell how that affected you.
You saw 1000 'product placements' for coke without being aware of it in films other things. Taste is associated with familiarity and comfort.
> Taste is associated with familiarity and comfort.
I respectfully disagree, I used to work in F&B industry. And product marketing and placement has absolutely no affect on how one think something taste is good or bad. It does however remind anyone next time the walk into store they should buy coke, which does affect sales number. But not taste difference.
My 7 years old nephew hasn't watched a single Coca Cola coke in his life time, but he prefer coke over Pepsi. While my friends daughter ( and her mum ) prefer Pepsi.
"And product marketing and placement has absolutely no affect on how one think something taste is good or bad"
Yes it odes.
"My 7 years old nephew hasn't watched a single Coca Cola coke in his life time"
1) It's ridiculous to say a 7 year old 'has never seen an add for coke'.
How would you possibly know that? Unless they are living as the Amish, that kid as seen ads.
2) That someone will prefer one drink over another is not relevant to the argument.
Nobody is making the claim that 'ads make you do stuff'. No Coke ad is going to 'force Pepsi drinkers to like Coke'.
Ads are influential. They root the product in feelings, emotions, those have impact, which is why advertising is there.
3) Your nephew was probably drinking code from some age, maybe as a treat. It was Coke and not some Italian cola, because Coke has such a powerful market position.
In Italy, they drink Brio and it's quite good, you can't even get it here, if you could, people would drink it.
Coke spends billions on ads because they work not because they don't.
The trick isn't figuring out whether advertising in general influences people's behavior. The trick is in figuring out if any particular advertising campaign generated more profit than it cost to run.
Also, there's one alternative that's often forgotten in these discussions: Perhaps game theory is at play. It may be, for example, that, across entire industries, advertising costs more money than it's worth. But that everyone has to do it anyway, because anyone who chooses not to will start losing ground to everyone else. IOW, just like in the standard prisoner's dilemma, choosing to act is less about increasing your potential gains than it is about limiting your potential losses.
There is an interesting long-running natural experiment in the pharmaceutical industry that suggests, albeit inconclusively, that this is the case.
Distilling it down into "any particular advertising campaign" is pretty myopic in this world. It's about strategy as a whole over decades. Not every ad campaign is to drive immediate sales or signups (direct response). Branding and awareness campaigns can take years to run, and these are ALL diligently measured at every stage. Losses do occur do to negligence, malice, and poor execution all the time, but the brands should take some blame as well as they often feel compelled to spend. They have huge budgets that they NEED to spend regardless of they perform - sometimes due to accounting shenanigans, sometimes because they don't know any better, etc.
No matter what time scale you pick, there's still a point where ad spending costs more than you benefit. As time passes, the boost from any campaign decays. For any particular person you only need to show them so many ads to maintain awareness.
> these are ALL diligently measured at every stage
Mmm. Sometimes. Sure, you can obsessively measure audience and sales numbers, but if you don't have any controls then it becomes nearly impossible to figure out how much of that came from ads and how much came from everything else in the entire world.
it is true that that profit/loss point exists, but it may be impossible to measure. But it's very clear that 75+ years of Coca Cola marketing strategy has produced the situation where any human on earth immediately associates "Coca Cola" with soft drink.
If Coca Cola cut advertising 90% this afternoon, they likely wouldn't see the consequences until a new generation or two grew up.
I suspect that a similar dynamic can be in effect with many industry events. Everyone would collectively perhaps be better off with pulling out of or at least scaling back on big industry shows. But that doesn't mean it makes sense for just you to pull out. (And, certainly, your events team probably isn't going to push for scaling back.)
There's also a huge mutual back-scatching thing going on. I remember in a former life we wanted to pass on a big software vendor's user group show because, while we sort of needed their software for some important customers, we got very little traffic at this expensive event. Their CEO called our CEO and basically said to him "Be a pity if something happened to our partnership."
> IOW, just like in the standard prisoner's dilemma, choosing to act is less about increasing your potential gains than it is about limiting your potential losses.
You seem to be assuming that the potential gains are limited to current players of the game in this comparison.
It's possible that Coke, Pepsi etc. are all losing more money on ads than they're gaining in market share against their current competitors and in increased sales. But that doesn't begin to address the question of whether advertising is a net loss.
For one, the advertising also serves as a collective moat against new competitors, which could enter the market and eat up both Coke and Pepsi's market share. Maybe their biggest gain from it is that a bankrupt soda company from the 80s didn't replace them, and that a similar company today wouldn't have room to enter the market.
It also doesn't account for more complex effects like a net increase in sales for all companies in that market. E.g. maybe Rolex, TAG Heuer etc. all benefit in the long term from expensive watches being seen as fashion accessories.
If Rolex stopped their advertising spend one month and saw little immediate change, then TAG Heuer etc. might follow suit. But both might only experience the real loss years later as "luxury watches = must buy" faded from the psyche of their current and potential customers.
Part of the claim is that the people who are checking are ad execs who, if PepsiCo stopped buying ads, would shortly be out of a job (or have their budget and influence slashed). A counter to this might be that different advertising channels are likely not identically effective, and a TV ad exec has a big incentive to poke holes in non-TV ads.
See the Pepsi Refresh Project of 2010. Where Pepsi redirected a sizable amount of ad budget (including super bowl ads) to community projects. They abandonded this strategy after a while because they lost market share.
This is the biggest large scale test of advertising I'm aware of. But it probably doesn't apply to all products.
They only lost market share because everybody else was still using advertising. Therefore, this is not a test in favor of advertising (only in the prisoner's dilemma sense).
If advertising is an arms race with more total cost than total benefit, that would be an argument that advertising works. Because anyone, at any given time, will benefit from running advertising.
It also means advertising is a net loss to society, but that is separate from the question 'does advertising work'
How do you look into it though? I don't know, so i'm asking - but my immediate thought is that you treat it like science. You isolate an environment, advertise, and see if it has an affect. But the implication there is that if it doesn't have an affect, money is on the table.
If this is even remotely close to reality then it makes sense to me. Companies are more concerned with constant growth than strict efficiency, imo. They're throw as much money around as possible, and every cent lost or left on the table is panic inducing.
I also imagine different types or products and/or markets behave quite differently. Eg a new product might very well benefit from advertising - since no one can buy your product or visit your store if they don't know it exists.
My company provided sales data from stores to a specific team on google that compares it to users who’d been advertised to. This team didn’t want my company to influence results, and didn’t want YouTube to influence results.
More complicated since there are multiple device graphs/etc in play, but effectively tens/hundreds of millions of dollars at play and no one trusted anyone. The largest thing at risk was YouTube’s ad revenue.
> No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary.
Depends enormously what sort of business you're in. I used to work for a company where all of our sales came from ads, 100%. It was trivially true that if we stopped advertising we would have no sales. We were also committed to running experiments: we knew how well all of our many advertising channels performed, and we ran A/B tests for every change.
Great subject matter, but boy does reading the transcription remind me why I hate podcasts. Stop infantilizing the audience with these coo-coo-ing sounds bites and get to the damn point already!
These experiments are run all the time even before the internet. I remember reading about how for broad brand advertising they used to segment by city. Half the cities got an ad for Coca Cola and half didn't. Then you compare sales of Coca Cola. No one will publicly publish numbers because it's a mix of sensitive sales data and competitive advantage (ie: otherwise your competitors don't have to burn money running their own tests). Also, smaller shops turn their online advertising campaigns on and off all the time to test impact.
> No one can actually prove it has any ROI at all.
That's a philosophical question of whether you consider statistics to be "proof".
> No one is willing to run the experiments necessary.
You're nuts if you think this is true. I assure you that companies in traditional industries (i.e., without venture capital) can and do run these experiments.
The Uber story is about venture capital and its anti-market incentives, not about the ad industry.
Read the podcast transcript. An economist proposed turning off print ads in one market to find out if they mattered, and the head of marketing said he’d rather not know.
Only because the head of marketing realized it would expose his past misjudgments - not because he didn't realize it would actually be a good thing for the company. You'll see this type of behavior in every department - people trying to cover up their mistakes.
Wait, so their marketers are unwilling to do anything that would reveal ad ineffectiveness, and you consider that an argument in favor of the claim that ads are effective (the topic at this point in the thread)? Like, you take that as a signal that marketers would converge on effective ads?
My point is that we have to differentiate between bad employees and the overall effectiveness of ads.
The latter is all about incrementality. When you adjust for it, you get the true effectiveness of ads. In some instances, the incrementality will be significant, in others it won't. It looks like that in Uber's case, the incrementality was low. Btw, if you're not familiar with this term, please look it up - there's a ton of literature on it: https://www.adroll.com/blog/marketing-analytics/beginners-gu...
Which brings me to the point about bad employees - marketers who are not adjusting their performance measurement based on incrementality should be fired (and it certainly looks like this was not being done at Uber at that time).
I interviewed at Uber for a marketing position back when they were small, and decided not to take the job (huge mistake). At least back then, the people I met were super smart (a lot of them were ex Facebook, again back when those teams were super good). I never met Kevin Frisch and don't know if anything he said was taken out of context, but the first impression is not good. Also it's a bit strange for a well-respected Uber guy to go to Intuit - it's not a common career path.
"Chief marketing officer of Unilever" isn't the guy running experiments, he's the guy that hires ad agencies and auditors that in turn have teams of data scientists running various experiments for Unilever.
I guess you could propose a conspiracy theory that all these people are conspiring to put together fake reports and data sets, but that sounds extremely unrealistic - ad agencies are essentially money-counting businesses like banks, and they take data security very seriously. (Because usually they'd be audited at every step.)
In short, smart people have already thought about these issues, and there isn't any inherent pro-advertising bias in industry, rather the opposite.
"No one is willing to run the experiments necessary."
Tesla is one natural experiment about not spending money on advertising in the mass media compared to traditional car companies that spend HUGE amount of money advertising.
"Hyundai spent $4,006 per Genesis vehicle sold in 2018. Ford’s Lincoln brand came in second with $2,106 per vehicle sold. After Jaguar and Alfa Romeo, GM’s Cadillac brand came in fifth with $1,242 spent per vehicle sold. Tesla was the lowest at just $3 spent per vehicle sold."
Tesla's marketing spend is whatever it costs them in legal fees and fines to keep the mouthy celebrity CEO - and their high-profile campaigns in 2018 seemed pretty effective.
Telsa is in an enviable position of selling most of their cars before they're produced. In that position, you don't really need to advertise much. They also get a lot of PR to keep up brand awareness.
If Hyundai couldn't keep Genesis vehicles on the dealer lots, they'd advertise them less too. Having a dealer network means dealers that want manufacturer support in advertising to keep dealers happy, even if the new cars sell themselves, dealers need to get people in to sell used cars.
I suspect that if our browser isn't blocking ads then our brain is. It's complete conjecture, but I assume that adblockers eliminate this cognitive load explaining a portion of why they became successful before they were necessary for security/malvertising.
Beware extremes like "no one can actually prove". One difference between internet ads and their more ethereal tv and radio predecessors is that adviews, clicks and purchases can be tracked. Also, there are techniques that can make even TV, radio, print and even digital to physical world ad performance more visible: coupons, response codes and campaign-specific phone numbers and URLS. That Uber was buying hundreds of millions in ads and could not attribute performance (sales, for example) speaks more to poorly designed campaigns and potentially very bad actors in the supply chain.
This story was on the Freakonomics podcast too. It definitely seems like a bubble, but there's no external pressure to make the bubble pop. With the mortgage crisis, eventually, people can't make the mortgage payments and the bubble bursts. But with people happily spending money on ads that don't work, there's no external pressure to stop. Will this bubble ever burst?
I'm only about halfway through the first of the podcast episodes you linked (thanks!), but just thinking about this logically for a moment:
I would not be shocked to learn that advertising for a specific product or event is not particularly effective. However, I'm inclined to believe it has a huge effect on overall brand recognition.
Let's say you go to Amazon to buy a roll of toilet paper. How do you choose from the literally hundreds of options? You could spend a day of your life reading reviews, and trying to parse which ones are fake. Or you could buy the toilet paper from Scott because you recognize the brand.
As I see it, buying brand advertising is a lot like buying an expensive suit. It's not that the suit makes you more productive, but it is a sign of professionalism, and—frankly—of wealth. If a brand is advertising everywhere, you know they aren't a fly-by-night company, and their products likely meet some standards of quality.
There's also the bit about luxury (mostly) car commercials. Half their purpose is to remind you that you made a good decision buying your <brand> car, and maybe your next car should be the same brand.
I'm a fan of the podcast but one argument they cited seemed to have a pretty glaring error - they looked at the case where eBay was comparing incremental gain on search ads over no ads. It's methodologically hairy because eBay is a very major player with significant brand recognition.
"When Tadelis was working for eBay, the company was in the practice of buying brand-keyword ads. Which meant that if you did an online search for “eBay,” the top result — before all the organic-search results — was a paid ad for eBay."
This doesn't show that advertising doesn't work per se, it shows that eBay didn't hire a competent ad buyer. Whether or not you can prove the efficacy of advertising as a whole, this is not a valid approach.
I work in digital marketing and this episode made me want to tear my ears off.
First of all, they didn't differentiate between display or PPC advertising. PPC you only pay if the user actually engaged with the ad. Nearly all of the anecdotes they used where about online display advertising - a known crock.
And we absolutely run experiments all the time! In fact, we tie our ad spend directly to conversions. If anything, the market is too efficient - it's really hard to get more than what you are paying for.
Absolutely. You can give a campaign its own website or URL if you want to test something like the effect of a specific billboard or radio ad.
Or you can do the equivalent of an excess mortality study but for client acquisition or product sales. You take something like the month of January and you say "this is what we expect for web traffic given our standard traffic for that month, and extrapolate it with your average web activity growth". And so long as you have "clean" months to compare it to (with no ads running), you can get a pretty decent back of envelope about what your ad campaign did.
Like, there's so many different ways you can cut the apple, it's patently ridiculous they made those claims.
To give them a bit of credit, there's something to be said for there being a lot of bad ad spending out there. But like the stock market, there's a tacit understanding that it's more or less efficient.
I'm struggling to follow the actual narrative presented in the chopped up Twitter style. It sounds like there were was a third party ad network involved?
One thing we learned a long time ago is never to trust "engagement" data from platforms (be it Google, Facebook, LinkedIn, whatever). There are enough random user behaviors that you generate a ton of noise. You need to build larger, more robust attribution models that tie ad spending directly to revenue sources - new leads, new accounts, purchases, etc. Patting yourself on the back for how many shares or clicks you get is not great.
as a nonexpert its hard for me to say too but perhaps you will benefit from hearing directly from source - the performance marketing guy at Uber who found all this out: https://www.marketingtodaypodcast.com/194-historic-ad-fraud-... honestly i am not sure if he did a good thing by asking all these questions, or was partially to blame for gross incompetence managing a 150mm ad budget, or both. (the chronology of when exactly he took over matters, it seems he was only in the job about a year)
This is really, really not true. Advertising lift has been well studied, especially in metastudies spanning hundreds of digital campaigns across Facebook and Google. These are independent academic metastudies, with hundreds of millions of impression data samples.
Positive lift in the range of 0-20% is very common, and many statistical aspects of causal inference on ad impacts are well understood.
Negative lift and flat campaigns are real phenomena too, and it does deserve more widespread publicity that negative lift happens in an appreciable number of campaigns, but that doesn’t take away from the overwhelming evidence that digital advertising works and that the mechanics of positive lift are well studied.
Here are two of the foundational papers in this area:
Particularly Figure 1 (page 26) in the second link. That figure alone utterly refutes any nonsense claim that digital advertising doesn’t have provably positive ROI.
I did, and I think there's a lot of p-hacking going on. Leading with TV advertising is disingenuous at best.
The ebay example twists the concept of user acquisition (new customers) and purchases (new or old customers). It is a common tactic to buy advertisements defensively, for example, if you're a product manager, and have determined that some of your user base are more transactional rather than frequent.
Another pet peeve I have is how they conflate direct response advertising and marketing.
The ad buyers aren't smart enough to measure the actual effectiveness of their ads, and the ad sellers are not incentivized to teach them how to do it. This can go on for an arbitrarily long time.
IDK. It's a risk worth chewing on. eg If Google and Facebook tanked, bringing down the stock indexes and all the institutional investors, triggering a sell off... What would the knockon effects be? I don't know (no clue). The 2008-9 crisis largely happened when CDS stuff popped. Is there a similar house of cards on top of FAANG stocks or the digital ad market or...?
Ecommerce sites have very fine grained measurement of their advertising spend and know exactly the ROI (which is why they focus so much on retargeting)
This is completely false. Large advertising platforms have many A/B tests that show significant differences in consumer behavior between groups that receive different ad treatments.
Ads might be less efficient than some believe, but it's super easy to see that they "work", and advertising platforms do it constantly.
This is nonsense. I had a startup completely powered by Google Ads. Ads brought in nearly 100% of the traffic. When my billing information with Google got mixed up and the ads stopped, the traffic went to 0 immediately.
Though not an intentional test, we found out what happens when a movie gets a wide release but does not advertise. In 2008, the movie 'Delgo' was released on over 2000 screens with nearly no advertising. Because the production company could not find a distributor, they spent their ad money on renting the screens for a week, with the hope that some people would randomly see it and word of mouth would spread, leading to the theaters wanting to keep it for additional weeks. It became the lowest earning wide release movie up to that point in time. Each screening averaged two people per screening. More people saw Conan O'Brien making fun of the movie in his monologue than actually saw it in the theater.
The reviews for the movie were poor but it had lots of household names as its voice talent, including Anne Bancroft in her final film. Good or bad, advertising likely would have lead to more people seeing it than two per screening. Of course there's no way to know for sure how many more would have been enticed by the ads but we do know that going with zero advertising resulted in a huge disaster.
These are not fair comparisons. Eminem and the Avengers have already established identities. Music cannot even compare to movies due to the sheer quantity of modern musicians. There are plenty of fantastic (IMO) bands out there that release music without advertising and they get nowhere.
As for movies, Avengers got to where they are today through lots and lots of advertising.
> It became the lowest earning wide release movie up to that point in time. Each screening averaged two people per screening.
On the other hand it got displaced by Oogieloves which had $40 million in marketing costs. Critics found it mostly bad and the only award it won was for films produced in Brazil. Maybe there was a reason no one wanted to spend ad money on that movie?
Maybe not for companies of uber’s size/current reach, but small businesses definitely do benefit from ads. They see an immediate uptick in sales when they start advertising on various platforms.
Ads are somewhat like propaganda. There's no proving it will work or not in short terms, but nobody can deny that persistent propaganda has a long-term effect on the whole population. You have think about a kid/youth who listens to some propaganda for years when growing up. If say you show an ad for a new Apple product. A small part of it is to inform consumers about it, the longer larger part is to enforce the Apple brand which has been happening over many years. You can't say well it is not quite working so let's stop doing Ads.
Depending on what kind of advertising you are talking about. Direct response advertisers measure ads to the cents. And they know exactly what the ROI is and where the customer is coming from.
They mention the common retort to this which is very bizarre to me: "If online ads don't work, then huge companies wouldn't spend billions of dollars on them. Therefore they must work. You academics are just missing something."
Ok, how does those companies know it works? They don't have any real data to show that it does, just the fear that if they stop they'll lose a lot of business.
They do have data. It's pretty easy to see sales without ads vs sales with ads and then do further testing to narrow down results from there. This has been done for over a century since the first billboards were put up.
The amount of data generated by adtech today is staggering. The problem isn't data or advertising, it's the wrong people running the wrong campaigns for the wrong reasons.
>No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.
This may be true but it's a separate issue than the fraudulent clicks.
Brand advertisement (the kind you are talking about, as opposed to closed-loop direct advertising) is an investment in a brand, that doesn't get paid off in a day, or a week, or a month, or even a year. It adds fractions of a penny onto a customers value, every day from today into eternity. It's an investment in your mind, but the long time horizon makes it, as you point out, nigh impossible to measure ROI.
However, this is one of those cases where despite being immeasurable, it still works.
Imagine doing a study on low-fat or low-fat diets and trying to measure health outcomes like lifespan, or heart disease, or cancer after just one month. You can't do it. The best you can do is measure markers of these outcomes, like insulin resistance, blood triglycerides etc. Brand advertising is similar. You measure markers of long-term purchase intent. It's not perfect.
And just like actually doing a long-term study of diet for example is riddled with confounding variables and very hard (nigh impossible) to do well, so it is with advertising.
> No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary.
This is entirely false. I work in adtech, and almost all companies run experiments in order to optimize their ad spend (everything from Ad A vs Ad B, to Ad vs No Ad, to Channel A vs Channel B, and more).
This isn't to say that advertising always produces ROI. Quite often, experiments will be run that will show a certain strategy isn't performing well, and the company will adjust accordingly. It's incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.
That's great, thanks for your experience, but can't comment on the argument provided?
Isn't it possible that these experiments you talk about are fatally flawed? I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.
That’s kinda a weird take when there’s every incentive to be the the person/manager/director who saves tens to hundreds of millions of dollars in ad spend.
Sure marketing people want to protect their jobs but the odds of “advertising is entirely worthless” being this closely guarded secret kept by millions of people in the industry or being a collective delusion is pretty darn low. The reward for defectors is just too high.
IMO large social media platforms are actually much more capable of running these experiments than academics. (Disclaimer: I work on ads at one of these companies).
Platforms can accurately determine who engaged with an ad (basic logging on their sites), they have infrastructure to create statistically balanced ad experiments, and can also accurately determine whether a conversion happened (either through a conversion pixel or through data brokers).
Running these tests on behalf of advertising clients, or for internal research is fairly standard. If we couldn't prove statistically that our ads were working, I would have left a long time ago.
I think the difference from academic studies that fail to replicate is less about capability and more about the fact that experimentation in adtech is fundamentally straightforward and occurs under far more idealized conditions - those you mentioned, plus massive sample sizes and easy iteration - than most academic research.
Of course, this just reinforces your point that experimentation in adtech is largely not subject to the same issues that have fueled the replication crisis in academia.
I'm not sure what argument you're referring to? I quoted two claims from the parent comment and refuted those based off of industry experience.
> I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.
By "company" are you referring to individual advertisers or digital advertising platforms? Why do you doubt that either are incapable of running well-designed experiments? It's in both parties best interest to spend ad dollars as efficiently as possible.
While it's certainly possible that every one of the 100 million+ ad experiments ran over the past decade have been fatally flawed, it's highly unlikely. I guess the alternative scenario is that digital advertising is one giant hoax being kept secret by the millions of employees who work in the industry, but given how easily tech news leaks, this also seems extremely improbable.
Another option is that the experiment design was wrong for every single one of those experiments. Sounds unlikely right? Running the same broken experiment any number of times won't change that it's broken. So if these "100 million+" ad experiments are more or less copies of each other they could all be wrong. How likely is that? I don't know, but I do know that at a big Silicon Valley tech company that you've definitely heard of I was involved in an exercise where we were told to create an experiment. The type A take charge person in our group did what they do, which in this case was talking over and shutting down the Los Alamos trained particle physicist who was trying to explain to her why her "experiment" could never show anything.
> without any attempt to validate their investments.
I've also worked in adtech, for quite a long time on all sides, and my experience is that far more companies do research to justify their ad spend not validate it.
I have managed plenty of A/B tests in my day, each one claiming to show some improvement, 10%, 15% etc (some, as you said, showing no improvement). Even though these tests are often run "correctly", essentially nobody goes back and asks "wait, we had a 10% increase multiple times in the past year, but is our <metric> really showing the cumulative improvement we expected?"
The greatest trick in the industry is that because VC are pouring money into everything, everywhere, every metric appears to grow. At every startup, everywhere, pre-pandemic, numbers were going up because people were pouring money into the system.
I've worked at companies who I know for a fact their adtech product cannot and does not work, yet their business continues to explode because in recent months nearly all ad spend has been on digital ads.
I've talked to companies whose entire function is bidding optimization who literally do not understand how to optimize bidding given the information you have.
Absolutely people are running "experiments" but the function of the "experiments" is to justify that the ad team is worth having, and then that the VP of marketing is doing their job and then that the CEO has hired some super smart people, and then that the VCs might have really found a unicorn. Everyone sees what they want and no one really wants to ask the question "wait, does this really work? are these tests really able to capture the complexity of the environment?" And if you are one of those ornery people that insists on probing into the details and seeing if any of this is working, you will eventually get fired.
Ad tech is largely a scam, but a huge number of rich and smart people benefit from the illusion that it is not, so we continue to see experiments showing that everything works as expected.
I agree that there a lot of (smaller) players in the ad tech space who are scamming companies out of money. And especially if you've been in this industry for a while, this practice was much more prevalent a decade ago.
I don't think your view is representative of how most F500 companies, or many of the new DTC brands, invest on major ad tech platforms though (FB/G/Snap/etc.). Experienced marketers try to measure metrics as close to core business KPIs as possible, and comparing ads based off of simple A/B tests is increasingly becoming a technique of the past. Measuring the incrementality of ad campaigns through long-term holdout groups gives companies a much more accurate read of their investment, while bringing higher statistical rigor as well. So rather than looking for cumulative increases on <metric> after advertising for a year, you could instead point to group A (who saw no ads) and see that group B (who saw ads) drove a 1.5x higher <metric>.
I'm not sure how you can say advertising is a scam when there are clear examples of popular brands that most likely wouldn't exist today without digital ads? Take Allbirds as an example. There are dozens of consumer shoe brands that are already in physical stores and have higher brand recognition; how can you argue that advertising didn't help them cut through the noise and grow their bottom line?
> I work in adtech, and almost all companies run experiments in order to optimize their ad spend ... It's incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.
We have a very large and conspicuous example here that suggests that Uber hasn't been doing this.
Running experiments like you suggest is difficult and to do them right requires some fairly specialized knowledge to do correctly. I'd doubt more than a small percentage of companies have the expertise to effectively run any kind of advertising experiment that returns useful data. Most businesses lack that knowledge and rely on metrics provided by the people selling the advertising—who are likely to provide self-serving numbers.
While I'm sure some of the bigger F500 companies do a good job validating their ad spend, I most companies don't even know how. Certainly the majority of small businesses have no idea how effective their advertising spend is aside from very crude word-of-mouth feedback. I suspect this creeps way up into the Fortune 500 as well.
> We have a very large and conspicuous example here that suggests that Uber hasn't been doing this.
I'm not sure this article is the smoking gun that the author makes it out to be. By 2017 Uber had reached saturation in what could reasonably be considered their addressable market. From my perspective, turning off app install campaigns and not seeing any dip in acquisition validates this position.
Meanwhile, their other major app, UberEats, is a business that exists in a highly competitive market, and they continue to invest in app marketing in order to grow that division. If the takeaway from this article was that Uber discovered app marketing was useless, I doubt you'd see them make that same mistake again.
> While I'm sure some of the bigger F500 companies do a good job validating their ad spend, most companies don't even know how.
If anything, I think bigger companies are more prone to overspending/spending inefficiently. While they do experiment to try and optimize strategy, there is some business inertia that gives them the flexibly to move a little slower, since one poor marketing decision will not sink the business overnight. Conversely, small businesses don't have the resources to investment as much in marketing. This leads to poorly run ad campaigns, but rather than continuing to invest in a bad strategy, they typically just kill the effort altogether.
Most small businesses don't need complex experiments to understand how marketing effects their bottom line - poor ROI is a lot more apparent when you're low on funds. That's also why it's in an ad tech platform's best interest to make efficient advertising as accessible as possible.
Counter-example: I know of at least one consumer goods company that has studied the long-term effects of certain kinds of sponsorship deals on consumer behavior. The study had tracked people for at least 10 years at the time I learned of it. Of course, this company would never publish the results, because they provide a competitive advantage in structuring and bidding on sponsorship deals.
You are conflating value of the marginal ad with value of any ads at all, and also exaggerating what those articles say
One of them said tracking cookies only boosted conversion 4%, and another said P&G did better with some traditional media advertising than some digital advertising.
This is patently false. I have friends who have worked years developing solutions for doing ad effectiveness comparisons. They are A/B tests in most cases but some methodologies require a lot of sophistication because of problems tracking conversions.
Traditional ad platforms like TV, newspapers might not have done this but online ones surely do. Infact that is one point they consider an advantage as you can measure effectiveness unlike the traditional platforms.
This was a superficial analysis at best. Adtech produces petabytes of data every day proving that advertising works. There's a reason why two of the most valuable internet companies sell ads.
The problem is knowing exactly which formats and campaigns are working down to the dollar, but part of that is just the reality of fuzzy attribution and it's only getting harder as privacy regulations get stronger. However you can definitely tell the difference when turning everything off, and if you can't then you were advertising to the wrong people in the first place.
Uber's mistake isn't that advertising didn't work, but rather that they didn't vet their vendors or even bother doing any checking and optimization of their own.
The implication here is that money spent on ads is largely wasted.
Companies like Uber and Ebay have turned off all their adspend and saw little to no change in their acquisition metrics. You can argue that they were just doing it wrong. But the point is that, if even they are doing it wrong - and getting nothing in return for the millions they're spending on ads - then it's very likely most others are in the same situation.
You are right that this doesn't mean _all_ advertising is useless, there are absolutely profitable usecases. But the larger points still stands: most money being spend on advertising right now is likely not returning anything.
We now have some strong precedents being set. I believe this will cause more major companies to run the ultimate experiment: turn off all ads and see what happens. It's too early to tell, but it's not impossible we'll see adspend drop significantly across the industry once everyone finds out they're just burning money.
Why the ad spend isn't working is a critical detail. It's definitely not most money but yes there's a lost of waste because it's a 12 figure industry with lots of politics, perverse incentives, thousands of vendors, and a complex global supply chain.
Fraud is a special case because it's criminal activity and has nothing to do with advertising. It happens in every industry but it's especially easy with online technology spanning multiple countries and data that can be easily faked. Uber was exceedingly oblivious here but I wouldn't extrapolate advertising efficacy from these examples.
At least in e-commerce it’s very much possible. The company that I work for does such experiments regularly (there’s a dedicated Data Science team for measurement) and I’ve personally been involved in lift studies for Google Ads. They work, you just have to be careful with the ‘how much’ combined with ‘for what’.
Happy to chat with anyone who is interested in the topic (pfalke at pfalke dot com).
Haven’t had a chance to listen to the podcast, apologies if that made me miss the point of the parent post!
It's an arms race. In theory advertising should give you an edge over non-advertising competitors, but if everyone is doing it demand remains unchanged and also you're wasting money
Lewis and Rao published a meta-analysis of 25 large scale controlled advertising experiment [1]. Here's the abstract:
Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.
Yeah I think Google and other advertisers are great at predicting what people will buy and then shows them ads for it. That’s different than showing a person an ad and changing their behavior, but to advertisers they can’t know unless they experiment.
> Yeah I think Google and other advertisers are great at predicting what people will buy and then shows them ads for it.
Seems to be entirely subjective.
I bought a new blade for a bread cutting machine on amazon, the last one lasted ~25 years. My current recommendations: blades for bread cutting machines.
I also checked some Christmas decorations on Amazon, but didn't buy any. So next to the blades I get Christmas decorations recommended. January is a bit late for that, isn't it?
I also bought a single server rack on eBay. Now I regularly get spam emails from eBay reminding me of new offers for server racks. Even the company I work for doesn't have that many, so why?
In short I am not really convinced that these platforms are good at predicting what I want. Rather it looks as if they are good at showing me what I already have.
I would guess it's fairly hard to encode a lot of rules that are obvious to most people algorithmically. Amazon does encode a likelihood of repeat purchases to some degree; they offer a coffee subscription but not a weed whacker subscription.
But besides doing something like "purchases >$100 are not likely to be repeat purchases," I would think it would be difficult.
The usual reply to this is that Amazon shows you that because you might return it and then buy a new one, gift it to someone, etc.
They show it because it "works" for them, not because it's wrong and they just can't somehow make sure they don't show something.
Because even if 99% of people don't buy anything twice, 1% probably does for some reason, and that signal just trumps all the noise. (And that's why most of ad spending is bullshit, but since Amazon's cost [even with opportunity cost] is basically 0 on their own site, they'll continue to do this.)
The answer to your confusion is that un sophisticated remarketing (showing ads for searched or purchased items) has in nearly all cases a measurable improvement on ROI. Improvements to filter out useless ad views are much more complicated and expensive.
Advertisers actually don’t care about any individual case, but buy on aggregate results.
Oh I don't think anyone is confused about why this happens or why it's still worth it. The next step we need to take is to agree that it is unethical to steal many people's time for a little reward.
Agreed. Companies often don't understand the "directionality" of purchasing behavior.
For instance, I recently bought a new iPhone directly from Apple, and I bought a case for it on Amazon. Now two months later, Amazon is recommending that I buy an iPhone (...which I bought two months ago from Apple).
Buying a case an then waiting two months to buy the actual phone would be a very strange thing to do, but Amazon seems to think I should do this.
And Amazon is definitely not alone: Google has been troubled with "AI" running rampant in search results for years, seemingly with no or very little QA.
>In short I am not really convinced that these platforms are good at predicting what I want. Rather it looks as if they are good at showing me what I already have.
While that's definitely true (I get ads from Amazon for stuff I've already purchased all the time too), in that case Amazon is leveraging their own marketing channels to support their own business -- which has, effectively, zero marginal cost to them.
However, when advertisers buy space from Google or FB, that's not the case. There is a definite external cost to such ad buys.
As such, being able to identify the "value" of such expenditures is (or can/should be) important to the advertisers.
That said, much of advertising is based upon the idea of "top-of-mind awareness."[0]
The idea being that if you are considering a purchase, the brand that comes to mind without prompting when considering that purchase will be preferred over brands of which you aren't immediately aware.
Purchase decisions made on a whim or without any research are usually those which are low cost. Which is why companies like P&G, Coca Cola and like companies focus on "top-of-mind awareness."
If you're going to buy a washing machine or a riding lawn mower, you're much more likely to do research than if you're going to buy a soft drink or a bag of chips.
That said, the same "top-of-mind awareness" can be helpful even for big ticket items, as those brands may well be the first ones about which research may be done.
Identifying how such advertising may impact purchase behavior is a complex topic, and unless you can quantify specific ad-views to actual purchases, doing so is generally speculative and is the subject of a great deal of quantitative market research[1].
The probability that a random person would want to buy a blade for a bread cutting machine is nearly 0%. The probability that you will buy a blade for a bread cutting machine again is non-zero because you may be dissatisfied with your new one.
I think a lot of this was already obvious to slightly-savvy mobile app users. Look at pretty much any free-with-ads app; a majority of the ads are going to be ads for other free-with-ads apps, many of which you already have installed. The thing is, outside of the Internet most ad spend is for marketing ubiquity, not direct response. You don't really buy, say, TV advertising with the expectation of getting so many clicks or calls out of it. And, up until a few years ago, TV was the lion's share of ad spend. So who knows if we'll see an actual industry reckoning or not. The whole point of advertising is to waste money, after all, and plenty of brands were fine with getting nothing but exposure out of it.
More interesting to me is the fact that tech companies are finding it surprisingly difficult to control where their ad-spend goes. I suppose this is an inverse of the problems with supply chains, where Apple can take three years to get a connector vendor out of their supply chain even when they were using literal child/slave labor. It's a market for lemons; bad "money" (publishers, suppliers) drives out good. The question is: will questionable ad publishing actually harm corporate reputation to the point where big ad spenders go away, or will we just see periodic Adpocalpse-style waves of spending being decreased and then brought back?
The point of traditional advertising IS pretty much to waste money, similar to how the point of a peacock's tail feathers are to waste resources... it is a signal to potential partners (business or romantic) that they are so capable they can waste resources. It is a signal of bonafides.
It only says that the margin on the sold products are high enough so they can afford expensive marketing. Does anyone seriously believe that the massive marketing for a product like Coca Cola is a sign that it is a high quality product?
The problem is that it is no longer true with online ads. The barrier to entry in online ads is non-existent and thus seeing an ad for a product should not be considered a trust signal (if anything, it should be the opposite) unlike on TV or print where the price of such ads works as an effective filter.
I'm genuinely shocked at how many people think advertising doesn't work.
Just because it doesn't make a person consciously stand up and say, "I want a Snickers" does not mean the ad didn't work.
Like, honestly, are there people here suggesting viral ads don't work? Or don't understand the point of Coke or Mercedes Benz ads are for? Coke establishes themselves as the defacto soda. There is no other soda, or if they are, wish they were coke.
MB ads among other things reinforce to MB owners the wisdom and luxury of their brand. To own an MB is to be part of a club, one that is advertised across the media spectrum. If MB didn't advertise at all, they would either become another commodity brand, or have to be so luxury that only word of mouth is necessary (Bently, etc).
Now, maybe that's what cortesoft is saying "get you to waste money" but if it got you to waste money on their product, it worked.
Sorry, I think you misunderstood my point a bit... I think I am agreeing with you.
My point was that advertising's purpose (from a consumer point of view) is for the ADVERTISER to waste money, to signal that they have confidence that their product is good enough to recover the cost of wasting money on advertising.
I understand the point but the MB example could be off the mark. According to their site [1] they sold more than 2 M cars in 2019. Luxury yes, at least a bit, but a pretty large club. Maybe Ferrari and similar brands? But they do a different kind of advertising.
I don't think they meant that it's a private club.
Personally, I think it's less about luxury and more about value signalling. Having a Mercedes says that you're comfortable financially, and they're "dignified". I'm not saying it's true, but it's what their marketing generally implies. Luxury is generally just a dog whistle for value signalling merchandise.
Ferrari does different value signalling. They imply wealth because of the cars performance, and they're generally a much more obvious signal. They're usually a bold color like red in the ads, and the body styles are distinctive. You could miss a Mercedes in a group of sedans; you're not going to miss the Ferrari.
The private club brand, to me, is Rolls Royce. I would guess because I never see ads (and rarely see them in person), so my only references are pop culture where they're associated with the ultra wealthy.
I'm not entirely sure about that, at least not in all cases.
I was recently in the market for a new mattress, so went to a mattress store and tried some out. Even though Tempur Pedic is all the rage with their insane ad spending, I was not all that impressed, and found a considerably cheaper mattress that felt much higher quality.
Now, this may all be sales mumbo-jumbo, but the sales rep noted that a lot of people come there only interested in Tempur Pedic mattresses, which he regards as "literally a piece of foam, total crap mattresses." They sell at such a high price point because they advertise so much, while the mattress I found came from a no-name brand that doesn't advertise at all, and was therefore cheaper.
So I guess in some markets, advertising does work. I've never heard of someone telling me how much they loved a mattress unless it's a Sleep Number or a Tempur Pedic, and those advertise by far the most.
I guess that's the thing; for a lot of industries, people aren't generally going to make a purchase without doing some amount of research. Maybe Tempur-Pedic was on their mind when they entered the store, but once they're in the store they're generally going to end up trying out several mattresses. I /love/ my mattress (and have said as much to friends), but I honestly couldn't tell you the brand off the top of my head; I just went to a store and picked whatever felt the best.
One could argue that Tempur-Pedic has to sell at a higher price point to make up for the money they spend on advertising.
This was true in the past, but nowdays it’s much easier to try out a product and see reviews.
With Uber it would be so easy to penalize drivers who cancel because they don’t like where I’m going, but I’m still wasting 30 minutes quite often, because Uber spends that money on ads and UI rewrites instead of improving the core experience.
Playing devils advocate, couldn’t this just be reflective of the fact that Ubers brand reach us strong enough that there is very little advertising that gives them a marginal benefit?
Uber owns taxi services as a brand as much as Google does search.
I guess there are 3 reasons why reducing ad spend might not show a negative effect:
(1) Due to fraud, the money wasn't actually going to ads.
(2) You didn't need the ads. You were over some saturation threshold and the law of diminishing returns kicked in hard.
(3) The ads were valuable to you but not in an easily measured way. Maybe you could coast on brand awareness inertia for 6 months or a year before it affects sales.
This case seems to have decent evidence for at least part of it being #1, but not necessarily all of it.
I like that this story references the #deleteUber tag on Twitter because it helps organize the timeline of events going on at Uber.
There was another twitter thread on here recently about the chaotic and disastrous (though successful) Swift rewrite they did which happened during the same time frame. https://twitter.com/StanTwinB/status/1336890442768547845
During those time frames is also when all of the execs were quitting or being fired, along with their internal harassment problems.
It feels like Uber has really succeeded in spite of itself.
Maybe. A lot of people have been saying this for over 10 years. If it's a bubble that's real though, the effects are going to be pretty widespread. Not only do two of the biggest (and best-paying) tech employers get slammed, but so do all the inter-related companies. And, also BTW there will also be a lot fewer fat exits for startups if Google and Facebook acquisitions get turned off.
First sign to me that the dot bomb was imminent was a two inch article in the WSJ summarizing a finding that something like 2/3rds of startups had board members or upper managers that had been sanctioned or investigated previously for securities fraud.
It's telling that all the FAANG companies are basically in open violation of a number of laws.
You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another.
You don't have control over your SEO results as well - but you can also measure against SEO traffic with a high degree of certainty.
All big companies do this.
Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.
But that honestly doesn't seem that important compared to the other metrics - which most functioning large companies have decent data on.
You're also never going to know how many of your customers ate Green Eggs and Ham for breakfast. It's irrelevant. You have decent insight into your direct-online ROAS, and that's unique to direct online advertising, and it's important!!
Yes, you're never going to know if that million dollars you invested in online ads was the best use of that million dollars. But that's not much different than building a new factory, either.
Edit:
Specifically to Uber's case - they did NOT turn off 66% of ALL ads randomly to no adverse effect (implying that all ads are worthless).
They DISCOVERED that a certain type of ad (paying for installs on dubious ad networks) was mostly fraud. After turning off 100% of this type of ad - they found no adverse effect.
This is a fail on their analytics team. They should've been measuring this type of ad better - especially given how big a portion of the total spend it was - and had insight into something not being right. They should have been able to do this - and if they couldn't, because the network somehow didn't give them enough data to do it, they probably shouldn't have been spending this much money for exactly these reasons!
> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. But that honestly doesn't seem that important compared to the other metrics...
Wait, why wouldn't that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.
It's critically important for adtech companies' customers, since it's an essential part of determining return on investment.
But it's also critically important for both adtech companies and data scientists who work in the space to direct people's attention away from those sorts of metrics. You generally don't want to call the attention of the person who signs your paycheck toward the fact that it's all but impossible to really know for sure if your service has delivered them any net benefit.
This is creeping pretty close to the legal definition of fraud.
But advertising has had a fair amount of safe-harbor carve-outs for over a 100 years or so that are not available to other industries. So it is no surprise it continues today.
edit:
I hate to say it but the auto-downvoters here on HN are approaching reddit levels. Everything I said above is true.
This is the industry whose job is to manipulate and deceive people. It should surprise no one that, if they do that to others, they'll do that to themselves as well.
> "edit: I hate to say it but the auto-downvoters here on HN are approaching reddit levels. Everything I said above is true."
It's not that you're wrong, it's that you're boring.
"I knew this years ago, look how my cynicism is superior to your naievite" is a boring comment to read. You could explain why you think this is any closer to fraud compared to selling anything else and claiming it's the best in its class, or what safe-harbor carve outs by whom, or why they matter more than other effects, or what you think should be done to change them, or anything more substantial than "I'm not surprised". Oh aren't you? Great, cool story bro. Except it's not /even/ a cool story.
It's old man "get off my lawn". It's every tech forum's old-man status grabbing which is now approaching Reddit /r/SysAdmin levels on HN. The only question is whether aggressive downvoting can curb its growth here before it gets a choke-hold.
I will rephrase as "your comment is boring" rather than "you're boring" in future; my intent was to describe the comment and not to attack the author's person.
It's sad to see the downvote culture come here. Downvoting is meant for removing contributions that don't add much to a discussion, not for indicating whether someone agrees or disagrees with you. It seems to be a norm that is spreading though, sadly.
It would appear jodrellblank is perfectly demonstrating how to misuse the feature, by calling you boring. Not exactly the kind of attitude that we want to have around here.
The downvote button should be replaced with an "off topic" flag. If downvote is opposed with upvote and upvote is understood to mean "I agree" or "this is informative" then downvote would be expected to mean "I disagree" or "this is bunk". It's semantically misleading.
> The downvote button should be replaced with an "off topic" flag.
I hope not. Some of the most informative, useful comments I've seen on Hacker News have taken the form of wild tangents.
I do see where you're going, though, that if "I agree" is sufficient cause for an upvote, then it's only reasonable that a vote in the opposite direction should be allowed to carry the opposite semantics.
It's just that, from a purely social standpoint, that's not actually how human beings typically think, feel, and react. Perhaps it could be that simple on the Vulcan version of Hacker News.
Oh for sure—wild tangents are fine as long as they're unpredictable, which is what makes them interesting. The offtopicness we try to avoid are generic tangents, the kind where people get sucked into the same argument about some topic that they always get sucked into. Those are the black holes of internet forums—it's best to steer clear.
To be clear, "off topic" here's just an example. My main point was the semantic implication of a downvote - I've seen the argument of "downvotes are not meant for disagreement" expressed here previously.
> It would appear jodrellblank is perfectly demonstrating how to misuse the feature, by calling you boring.
I don't have an opinion on what "correct" downvoting culture would be, one way or the other, but - if you say "contributions that don't add much to a discussion" should be downvoted, then isn't a "boring" comment precisely one that _should_ be downvoted?
From somewhere in the warren of links to dang posts I traced out from there:
> I think people have the wrong idea about HN downvotes because they think Reddit rules apply to HN.
That one gets me, because I had previously assumed that downvoting for disagreement would be a standard Reddit thing, but that HN's community is supposed to strive toward a more "we're all mature adults here" kind of community standard.
That's not true. Downvoting for disagreement has been ok on HN from the beginning.
I realize people have strong feelings about it, but that's because of the psychological intensity of the mechanism. There's no way HN would survive without downvotes, despite the fact that not all downvotes are fair. It's a critical part of the immune system.
I have noticed the same thing over the last four or five months. I’ve been curious if a voting ring has sprung up on HN or maybe a COVID impacted demographic suddenly has a lot of time to patrol the site, or if it’s something else. It would be really interesting to have some anonymized voting data to analyze and see how downvotes map to accounts, geography, and economic trends over time. Though voting data must be relatively huge compared to content and I bet it gets discarded after items close to replies.
Please don't break the site guidelines by going on about downvotes. I know they suck, but bilious addenda about downvotes also suck, which is why users here are asked to restrain themselves from posting them.
Apart from taking threads badly off topic, such complaints usually end up as uncollected garbage, since often the most unfairly downvoted comments end up getting corrective upvotes from users.
At least with ecommerce one can track users that have seen and/or clicked on ads with what they have purchased and return on ad spend can be calculated.
But yes, the question still is: does it raise intent or did the user already want the thing, googled it, and clicked on the first result (which is your ad, above the organic results that also lead to your online store)?
The fundamental problem with this is that if I bought your product, it's because I was looking for your product after doing a decent amount of research which meant I was on a bunch of retargeting lists and you've wasted money on me. I have never to my recollection purchased anything because of an ad. If I have clicked through an ad, it was simply because it was the quickest way to get to the website/product.
There is a role for advertising because Google polluted their organic search results with Ads making it harder for you to find what you want and forcing you to compete for the top Ad spot. I realize that I can only speak for myself, but I have never seen any evidence that advertising works and I've worked in ad tech.
It's pretty clear to see direct causation when using ads to promote your product. Promote your product using ads for 1 month, then turn off ads and see how many sales you get. For a majority of e-commerce, you'll see a substantial drop.
You're also bidding for eyeballs. An established player might want to scoop up the most relevant keywords forcing competitors to at least price match, effectively starving them either of views or cash
The thing about the "advertising doesn't matter" argument is that there are situations where advertising absolutely matters - when a company is just starting out and there's no way they can get organic referrals, when the public is unaware of their existence, etc.
Moreover, most companies started small at some point so advertising and maintaining advertising made sense up to a point.
And if we look at those companies where maybe advertising actually doesn't help, companies that have reached a level of success where organic referrals and public awareness drive most of their business. And they're ongoing businesses that probably reached that level through advertising and have money now to use for that.
But not only are they already advertising but they don't if their word-of-mouth/public-knowledge presence will last indefinitely, they don't know if just word-of-mouth would let them control their image, would stand-up against future competitors ads and so-forth. So, even supposing you could show advertising didn't offer any immediate increase in customers, continuing to spend on it doesn't seem to me as irrational as it sounds.
This is the point that a lot of people are missing
Uber shows up in the news and in conversation every day. Most people have heard about it by now. Very few people are going to install it because they saw an ad.
Not the case for your neighborhood "Bob's Burgers" or "Uber Competitor" or some other new company. Or for specific marketing actions of known companies (promotions, new services, etc)
Furthermore, many of the people who say they hate advertising also hate PR and other sort of marketing campaigns. So presumably their theory is that people should build something and just hope people will discover them somehow.
It's a subset of the misunderstanding of "meritocracy" that defines it as "how well you can do at a narrowly-defined task" as opposed to the practical definition of "How many people know how well you do at a narrowly-defined task."
Our industry is shot-through with idealists who believe meritocracy should be the former, when in reality it's always been the latter (and always will be, unless someone can invent omniscient humans).
> Furthermore, many of the people who say they hate advertising also hate PR and other sort of marketing campaigns. So presumably their theory is that people should build something and just hope people will discover them somehow.
Could it be that people simply don't like being interrupted and having their time wasted, whether it's by ads, marketing or PR? Just because you have a business to grow shouldn't give you the right to waste my time.
That still doesn't address the GP point, advertising doesn't matter if you can get customers coming to you organically. Business size changes the odds of that, irrespective of that if the customers would have come to you anyway (and there are ways to know this!) then the ad spending was largely a waste.
You misunderstand what advertisers are trying to achieve. It's about Brand Awareness. They just want their brand out there and seen. Even click-throughs aren't as important as they once were. Coca-Cola still advertise when they probably don't need to.
I understand the distinction between advertising and building brand awareness. It doesn't sound like Uber was doing the latter. It also doesn't take away that if you're advertising to an audience who'd have come to you anyway, you're wasting your money.
Brand awareness must also be maintained, just like human relationships! How often do you tell a loved-one that you love them? Is once enough, because going by your stance, since you've told them already once before you shouldn't need to do it again!
I don't think there's a large "advertising doesn't matter" camp. For most the real question is around the efficacy of certain types of ad buys, ad targeting, SEO efficacy and the amount of fraud in the system.
The real problem is that for large companies with brand awareness they're starting to find a lot of their spend may not be generating revenue. Ebay in the Freakonomics podcast turned of about $100M in ad spend with no loss of revenue. But smaller, companies usually don't have the staff/knowledge necessary to know how and where to use their ad dollars.
>when a company is just starting out and there's no way they can get organic referrals, when the public is unaware of their existence, etc.
I don't think I ever saw a Whatsapp or Telegram advertisement in the first few years of their existence. There's a lot of software in particular that gets around by word of mouth alone.
Necessity for advertisement seems correlated to how unclear it is what advantage your product has for the consumer. If you make something that's like X but twice as fast or twice as cheap you don't need advertisement.
Ad spend by incumbents will also drive up the price of advertising in general since it's a limited resource, making an uphill battle for the smaller competitors where it matters. It's another moat.
Depends on the timeline. If for financial (or other) reasons, they need to "reach the audience ASAP", then sure, advertising is likely to be a good or even required move.
But if they want to exploit a more organic growth curve, and are in a financial position to do so, then advertising isn't so useful.
Ardour is a cross-platform DAW that has been around for 21 years. Every 2 minutes somewhere in the world, someone starts a version they got from us. We've never advertised.
Maybe the person is referencing how difficult it is to determine the value of the counterfactual? If it were possible to determine I imagine people would really want to know.
I think the implication here is "that honestly doesn't seem that important" because the expectation is that the number of people who would find you anyway would be a small number compared to a well-run ad campaign, at least over timeframes startups care about.
Having worked with a decent number of small teams without a lot of brand recognition, ads bring in a lot of customers: it was pretty obvious in our metrics when we ran ad campaigns vs when we didn't. Sure, eventually a few of those people may have found their way to us somehow anyway, but there was a step change in volume from ads. At least in my experience, the idea that ads have no significant impact is not really supported by the evidence. I assume that's what the OP meant by it not seeming important: they're working under the assumption that "do any ads do anything" is clear and you don't need to prove the worth there — instead you need to prove which ads work better than others, and cull the ones that don't perform well (which will also help save you from scammers).
One could argue that advertising for large brands with name recognition could be less valuable. I don't really have data or experience to know that one way or another, but in a competitive market that seems a little hard to believe. Tesla is an interesting counterexample, but I think it may be in part due to the EV market in the US not being particularly competitive; no one makes attractive, mid-priced cars with long range other than Tesla, let alone having an established charging network for road trips. And Elon's antics are, tbh, a form of marketing in and of themselves; "all press is good press."
Edit: I wrote the above paragraph badly. I can definitely believe ads are less valuable if you're a big brand vs a small one. But I'd be surprised if they're useless, at least in a competitive market. People don't have infinite money, so if they're looking to buy X and an ad for X from your competitor pops up, it seems reasonable to me that some percentage of the time your competitor would make the sale and thus you wouldn't, even if organically that sale might've gone to your company instead.
Also, I wouldn't argue that even small companies need ads — just that they do seem to be effective at increasing the volume of people interested in using your product at a given time. That may or may not be necessary/useful depending on your situation. I have been on teams where we intentionally turned off ad campaigns because we learned what we needed to learn from the cohort of new users, and now wanted to improve the product based on their feedback before spending money on more ads.
Yeah, I wrote that badly. It's easy for me to believe they're less valuable. I really meant "One could argue that they're useless, but that seems hard to believe in a competitive market."
There’s an interesting question there about when this form of advertising becomes obsolete.
Uber is in a very different position than many other companies. Anyone who browses the internet with regularity is already aware of their existence and probably just needs the right set of circumstances to come together to make Uber useful to them.
I suspect the results would be different if Uber were earlier in their adoption curve, but maybe that’s not true either. Maybe they’d be ignored for different reasons at that time.
Likewise, I probably haven’t seen an ad for coca cola or kleenex in a while. Once a brand is ubiquitous to the point where soda becomes coke and tissues become kleenex in the lexicon, it feels like ad spend is wasted.
They mostly do it from inertia, how well it works is a separate question. Historically you can find many examples of hugely successful advertising campaigns, it’s much harder to quantify the negative.
I think both Uber and CocaCola has a lot of work to do until I like any of them. I like CocaCola as a product though. Never tried Uber. For example I really like some Nestle products, but try to avoid them because of their bad company/brand reputation, even if their product is both cheaper and tastier I buy from a competitor instead. Whatever a company stands for and how they act will affect their brand.
There's a huge difference between a company that is widely known for being the app for ride hailing and a small company trying to get eyes on their new product.
Well, Uber is known to everybody by now. To the effect that "to Uber" is even kind of a verb/noun.
Also people either need a hired transportation or not. If they don't have a car or don't want to mess with the traffic and need to go somewhere, it's either Uber or Taxi usually.
It's not like Coca Cola, which is well known, but people could do without it (unless addicted), so needs to constantly nag people.
And it's not like some new product, which without advertising nobody would even know it existed.
In fact most of Uber's existance its operation has been 100% advertising (spending VS money to offer cheap rides and expand and gather "eyeballs" and "customers" without a profit). In my book, customer acquisition without profit is another name for advertising.
So it doesn't sound strange that it could do without advertising today.
But what if there were 3-4 strong players in the same, each eating in Uber's market share? You'll see how fast they'd found advertising indispensable again...
Literally lighting them on fire in a cold office to help provide some BTUs would’ve been a better use of that money, since at least that would offset spending somewhere else.
Except Uber, apparently? Or would the method you're talking about not have discovered that something fishy was going on? (I don't work with ads so I don't know the limitations of the type of experiment that you're talking about)
It would work for some of Uber's ads. From the article, it looks like they were frauded mostly by in-app ads. And they were paying for installs - not actual trips.
So, no, Uber's advertising here is a little different than (I think) the majority of companies. They are mostly paying for installs rather than sales / conversions. A lot of newer "app" companies could be in similar situations.
Though, honestly, this seems like a massive fail on their analytics team for not figuring this out earlier. They should have been able to see that all of these "installs" from certain advertisers were not leading to trips.
In fact, it says they turned off 66% of ads. They didn't randomly turn of 66% of ALL ads. They turned off this TYPE of ad, which they failed to earlier recognize was ineffective.
Step 1) assume your ads won't work.
Step 2) have enough analytics / logging in place to convince yourself the ads do work.
Step 3) if they don't work, turn them off.
Looks like they skipped step 2 - which honestly, is not uncommon for a fast growing business - even if they are huge and already make a lot of money.
What they found isn't even what people are discussing. They found that certain networks they were buying ads from were almost 100% fraud (which is pretty well known).
Instead, people here seem to be discussing that most online advertising is fraud, and/or that there's no way to prove it's effective. That is absurd.
Hang on though, in the linked thread they said that the fraud was incurred at the point of real users signing up for uber -- when they type uber into the app store search (e.g. an organic install), the ad network fraudulently takes credit for it at that point. So "enough analytics/logging" would not do the trick here - I think they would have noticed if a certain type of ad wasn't leading to trips, as you are pointing at.
A/B testing ads is a complex matter. you can A/B traffic and conversion easily, but a lot of established companies with fierce competition fight for mind share, not direct conversion; for that, you have both awareness effects (user won't forget about coca cola if they don't run ads for a month, and an ad that doesn't directly convert but increase awareness still has value) and coverage synergies (the number of repetitions in a day will increase coverage non linearly and the amount of channel repetitions will increase awareness more than a single channel view, even if it doesn't convert immediately)
> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.
This is typically an order of magnitude less than the attribution figures stated by digital marketing experts. A/B test is the right method to use, but the crucial thing is you need an earmarked population to see zero adds over your attribution window, since what you care about is impact on incremental sales, rather than incremental click likelihood.
There is a long econometrics literature on this and it is not a fussy technicality, the figures typically differ by 10x +.
It can happen that an almost certainly bogus scientific field persists for decades using state of the art research methods and no bad faith on the part of the researchers (any bias they introduce is probably not conscious). https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
A/B testing is... not a state of the art scientific research technique. Moreover the companies that provide the tools to do A/B testing are the same companies that sell you access to advertising space. I'm not saying that it's a common practice to defraud A/B tests, I'm saying that the fact that adtech companies have chosen to enable that research methodology out of the set of all methodologies they could offer suggests that we should expect, before seeing the results of any A/B trial, that the results will tend to favor the adtech narrative.
I don't think that people in the adtech space believe they're selling a bogus product, but I do think they wouldn't want to know if they were — they have a good thing going.
If you're employed in adtech you're mostly fine, skills transfer. If you're invested in adtech, do what you can to diversify away. Advertising is overvalued to some extent and that bubble will burst at some point or other. The question is just how much actual value advertising provides, how much will remain when the bubble bursts.
>but you can also measure against SEO traffic with a high degree of certainty.
None of these benchmarks distinguish between the selection effect (clicks, purchases and downloads that are happening anyway) and the advertising effect (clicks, purchases and downloads that would not have happened without ads).
You can fix this by dividing the target group into two random cohorts in advance: one group sees the ad, the other does not. Designing the experiment thus excludes the effects of selection.
When you do this experiment correctly, you find out that ads have low effect or are not cost effective (as eBay discovered).
Across this and the other[0] highly active digital ads post, there's 3 interesting forces at play:
a) technologists screaming "ads are literally the worst societal cost, like ever" who don't want to understand the industry
b) advertising folks taking up arms to defend their shamanistic, money-printing machines without statistics (Reason No. 7 will SHOCK you)
c) A tiny, tiny group of Mandalorian-like voices who have a necessary statistics AND industry understanding who are being drowned out
(this is in jest, but I double-dare you to say it isn't at least directionally accurate)
Other commenters have mentioned it throughout this whole comment thread as "incrementality tests". Amongst other approaches, this is the way.
Freakonomics severely lack the industry understanding. Listening to the podcast was like hearing how HTML is a programming language from the kid in week 2 at code camp. Then too the article, all the issues with Uber was just doing a bad job of managing their ad spend and they can fall into group B, noted above.
The baseline of this work is a control group who see no ads and then you build your tests from there that factor in channels, cohorting, and other components to get a statistically significant outcome. Yes, this will get more challenging with upcoming privacy changes (IDFA removal, et al). However, the last 10 years this wasn't a problem and I'm sure the corporations in the identity resolution business will hand-shake on a bunch of 2nd party data deals that just move the deals done in broad daylight around identity tracking and audience creation to the alley. Further, any advertising that is tied to an already known customer is able to be backed into at an audience level with login and cookie data. Even Pi-hole users may not be exempt here.
To finish with some constructive advice:
1) Advertising is not a synonym for marketing. We're only talking about advertising in both HN threads.
2) Every industry has high and low quality. Pareto's principle should be aggressively applied to where one spends their budget in the cesspool that is the Internet.
3) If you're ever seeking an agency to provide advertising services and they don't have a qualified data science or statistics leader (10+ years work experience, degree in Stats/Math/Econ, an MBA, or similar), run. Run from those shiny-shoes gurus. Channel your inner Usain Bolt and run.
You can definitely measure 'lift' - though it's far easier with online only sales.
For sure it's harder for say Gucci to measure incremental cross-platform lift but any good large brand spends big money trying to parse it out
One example way to get some value measurement would be Facebook's powerful tools. Beyond a basic 1:1 audience 50/50 exposure test, Facebook also lets you upload offline sales (or use their pretty effective pixel for real time online sales) and then feed into their system. Using the data your purchase data FB matches to ads delivered and you (with PII but also purchase value and frequency). Then the advertiser can define lookback/conversion definitions to get a measure of incremental lift. E.g. those who clicked ad bought __ right away, those who viewed ad within 1, 7 or 28 days bought __.
For what it's worth the adtech forums I participate on view Uber as incompetent idiots on this issue (which has been reported on like last year) - all they had to do was measure beyond the initial install for say did these installs actually pay to ride...
Install fraud from SSPs and resellers is huge, doing this basic measurement and quality control is the reason Facebook and Apple have huge and growing app install business. Though TBD on Apple fucking over Facebook and keeping the attribution for themselves alone with their new privacy features
I read a news article yesterday that had the same ad repeated roughly 20 times. Would the metrics show a separate impression for every one of those ads I scrolled past?
Also, despite the repetition, I don't remember what it was for. I think it had a photo of a woman outdoors.
930 comments
[ 3.0 ms ] story [ 306 ms ] threadThere are exceptions. I think Google search ad works so well it’s eating the product. I think YouTube’s TrueView is great for modern brand advertisers (safety issues aside). But by and large, banner ads, forced video pre-rolls, etc don’t work. The negative message of the format offsets any message you try to convey.
Yes, fraud is a part of this. But I think a lot of this evidence looks the same as if none of their targeted, performance-based advertising was doing anything at all, beside really weak branding.
Genuinely interested to hear from people who disagree.
Doesn't it? It may have miserable conversion rate, limited target audience but it apparently is commercially viable. People would hardly spend money on sending e-mail spam in 2020 if it didn't work.
There's an allure to making something measurable and setting goals against it. Often, those goals take on a life of their own. That's what I am suggesting happened here, I guess.
so there's evidence that people (incl uber, a market leader) do indeed spend money on ads even when it does nothing
Also, the only ads I've clicked are mobile ads. I've even installed apps via ad before - I've never once purchased an item from an ad online (but I've also run an adblocker for a decade).
Ads certainly "work" by some metric, and some definition of "work", but I think the metric and definition is unclear.
I suspect this really only works for more novelty things like clothing that are cheap enough to enough to impulse buy and of interest to image conscious people.
Especially with stay-at-home orders due to the pandemic, everyone (well, not everyone... but hyperbole) now has their own drop shipping brands operated out of their basements, all selling the same cheap products off of AliExpress. Will consumers catch on to what is going on? Do they even care?
Someone in this thread said no one has proved to them that digital ads are ever worth it. My experience is that when they are you don’t want to advertise it to potential competitors or the people who carry your ads.
CEO: "We need to do something about growth!"
Marketer: "Mobile ads are something!"
CEO: "I read those are hot. Here's a bucket of money! Do lots of something so we have lots of growth!"
[spending ensues]
As the Uber story shows, a lot of money gets spent without anybody making sure that it actually works. The actual goal is not to have an effect. It's to be seen as taking bold action. If the business gets better, bonuses and promotions ensue, no matter what the actual cause. Cargo cult management.
It sounds insane, but I'm sure it's rife. Years ago I was coaching at a Lean Startup weekend class. One team really killed it, rapidly testing product hypotheses through real-world testing. Their initial idea was quickly proven worthless, but they listened to users and came up with an idea that would sell.
I kept in touch with one of those team members. He went back to his prominent, well-funded startup, excited to improve his company. But nobody wanted improvement. Execs wanted to sit in a big room, think big thoughts, and produce specs. The engineers were to implement those specs. But nobody would measure the effects of anything. Proof that a grand poobah's idea didn't actually pan out was deeply unwelcome. It slowly drove my pal mad and he quit. The company bumped along for a few years to a modest acquisition, but it never really lived up to the hopes.
However, anyone engineering for advertising from the start of your distribution plans should be able to measure these things. It’s not hard to track which signups came from ad clicks versus organic traffic. The problem is that most companies start out focused on their apps and simply assume advertising is a totally independent function that happens elsewhere in the company. Invest some time into including advertising as part of your signup process and it’s really not that difficult to track LTV of customers who signed up via ads.
In fact, that’s how Uber confirmed this fraud. They noticed that some of these fake customers were clicking on the ads and getting signed up within seconds, which is something only bots can pull off. If the customers you’re getting from ad signups aren’t actually spending money in the app, you should know about this from basic analytics.
- programming blogs that doesn't have ads
- Wikipedia
- three or four online newspapers/websites that I pay for anyway.
- HN
- all the old stuff: web rings and enthusiast web sites
- I'd probably miss stackoverflow but I use that site less and less these days anyways.
Most of the rest can burn and we'd maybe be better off. At least search results would be cleaner :-)
As for HN, how long have you been using it? Are you aware it's ad supported? Some stories are ads, usually for hiring.
This is 100% wrong. HN is paid for by YCombinator. Yes, some of the posts are hiring posts, but no one is paying for those.
What pays for HN is the success of the startups that YC invested in.
> Wikipedia is constantly running ads.
This is wrong too. They only run ads in November. The rest of the time the site is 100% ad free.
You are allowed to think differently, but personally I'm fine with the banners on Wikipedia (except IIRC and AFAIK they pretended they were short on money while in reality they were expanding).
I'm also fine with YC running job ads on YC.
I'm however not fine with 100 - 1000 [1] ad and tracking networks pretending they have reason to log my browsing habits on a typical site.
[1]: based on copying the list from "cookie banners" to Libre Office Calc and checking how many lines I got.
We don’t need more internet content, we need a higher signal to noise ratio.
This version we're currently in, no it can die off for all I care.
Bring on the disruption!
Also worked in ad tech. There were some types of customers that cared about performance, but the largest were typically agencies and all they cared about was spending the budget they were given. Everybody knows this is happening, you can differentiate yourself slightly by doing better detecting it. But at the end of the day people really care about customer acquisition cost which includes the fraud, so if you had 0% fraud everyone would just raise their prices or margins
Large companies call it "brand" spending. Which is to say they're spending to maintain brand awareness. Measuring ROI would call the practice into question.
They were high quality users: Uber didn't pay for installs, but for installs + first ride. What's being alleged is that the ad networks were detecting a user installing the app, and fraudulently creating an ad display event to make it look like the install happened due to the ad.
I guess you can offer promotions but then you're giving money away.
A lot of big brands bid on, and pay for, their brand terms because someone at some point told then that they should. So when a person uses search to find Uber, they get an ad first, then a regular listing second. And almost everyone clicks the first link. Take away the ad and the free click is first.
The theory is that if they don't do this, their competitors buy this ad space and steal some of their organic traffic, and/or that they're just bidding up the cost of that space to make their competitors less efficient.
Now, that theory may not be cost-efficient in practice. But it's not like the marketers didn't understand that someone searching for the name of our app were going to see it in the list anyway.
One popular method is a geolocation test. Turn off the test campaign (usually remarketing) in certain geos and see what happens.
Google has/had no policies restricting this, unless of course, you guessed it, the keyword you are squatting is “google”.
This would be fine if Google embraced their dependance on ads, but Google still labels itself as a “search engine”.
Click farms click on everything in order to dodge fraud detection, as far as I know.
I simply ended up excluding Ashburn from our campaigns for this reason.
Google Ads default settings for location targeting is based on "Presence or interest" in a location, not whether the user is actually in the targeted country. That means even if you have "United States" selected as your target, your ads will still show mostly to people outside the US who "show interest in" the US.
To fix it: Campaign Settings > Location > Location Options, and under "Target" select "Presence". Never use the default "Presence or interest" option, as it will result in exactly the scenario you're describing.
This is my #1 pet peeve with Google Ads. Using the default setting is an incredibly common (and expensive) mistake people make setting up new campaigns.
This story goes to show that.
Tl;dr: money is at best a halfway decent accelerator. And as long as you don't massive positive retention just a sinkhole of fake growth. And if you have this massive retention there are better ways to invest your money.
No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.
https://freakonomics.com/podcast/advertising-part-1/
https://freakonomics.com/podcast/advertising-part-2/
The people that would have the power to run this experiment have their entire careers depending on things staying as-is. Running the experiment carries a significant risk of exposing that the advertising operations they're responsible for provide much less ROI than they pretend it does.
The unwillingness of anyone to run such an experiment is already an answer. Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
https://quoteinvestigator.com/2017/11/30/salary/
~Chomsky to Andrew Marr.
Because everyone is already acting like they know it works, so the only way that experiment can change things is in a way that's bad for the person in question. In that situation, they should (from a local, selfish perspective) be resisting even if they're awfully sure it does work (and perhaps even if they're right!).
Given that, I don't think the behavior has already given us an answer.
The goal was to demonstrate an impact of in-store sales from internet advertising. They did the first studies in about 2008-10, and have continued running these studies ever since.
They even built a tool so that advertisers can run these studies, and get a sense for the incremental impact of their ad.
Google have a similar (less full-featured) system.
Really, it's the rest of the ecosystem that has much of the fraud, and I think that a lot of people in the industry are aware of this.
So the goal was to advertise advertising. A more scientific goal would be to see if there was an increase of in-store sales from internet advertising. Instead they were looking to design experiments that would show a positive effect, with the goal of giving advertisers a dashboard so they could run those experiments themselves.
I think you are misinterpreting what "do the experiments themselves" implies.
Certainly there is a JOIN with Google's or FB's data, but it can still happen entirely in a hosted sandbox with no data sharing necessary.
We've run Google's Local Campaigns, which are supposed to drive brick and mortar store visits, which they measure using GPS and "other signals". I checked with one of our most remote and least frequented stores, to see whether they could see any of the 100 daily visits that Google claimed to have generated, and they couldn't see a thing (avg daily visits were around 80 before during and after the campaign)
And the social stigma in science of being someone who tries to take down, discredit, or disprove your colleagues/superiors/ competitors theories is pretty substantial: IME there has historically almost been a taboo against attacking our speaking negatively about publications and your own sciences + faculties practices.
The saying of sciences advancing one funeral at a time doesn't exist because they're all such great skeptics and falsifiers, and current scientific practice is heavily biased towards positive findings and contains general publication biases.
indeed there's actually a LOT of common ground with advertising self-interest, since a lot of publication in science is effectively just advertising your brand...
Same for a lighter, there are many factors involved, and playing with lighters that way was a party trick where I grew up.
We use truth as a concept every day as an approximation, but the universe is not bound to follow. Very unlikely things happen all the time.
And yes, when we put things that way it brings a lot of possible issues with sampling, processing, and measuring (and who’s doing the measuring). Some of these things would be harder to control for in a study of the effectiveness of advertising.
This is a dangerous misconception to be spreading. It is not at all related to your other claims, namely:
> Sampling can skew results. Scientists are people too. We all have flaws.
These are of course correct statements. But they do not influence truth. They may well influence people's understanding of what the truth is, but that's different.
That said, companies like P&G, Airbnb, and Uber, which are oft-cited as examples of digital not being worth it, fail to understand their own brand recognition and organic power, built through prior marketing efforts, as key to their current standing.
Sure, TODAY, it doesn’t have the impact they’d like it to have but the investments PRIOR were key to ensuring their success.
http://jacek.zlydach.pl/blog/2019-07-31-ads-as-cancer.html
I worked next desk to people running a small ad agency. Because we shared office (and I found them an intern), I got a very good look at how they're working. What I've seen can be summarized as: people who have zero clue or interest in statistics writing "reports", with "graphs" they don't even understand beyond "pointing up = good", proving positive ROI to customers - who also have zero clue or interest in understanding the numbers in the report, and not enough visibility into the whole funnel to independently check attribution. Both the agency and the clients were engaging in a shared and completely unjustified fiction of positive advertising return - and as long as both sides were happy, the money kept flowing.
I've been long since suspecting a lot of advertising on-line looks like that. Every now and then, I see evidence in favor. Like that good ol' Optimizely debacle, where it turned out Optimizely was structurally optimized to help people make invalid A/B tests, that erred on the side of concluding the interventions were working[0]. And sure, big brands with some superstar ad teams probably do this right. But I think there's enough slack in most businesses that advertising spend can get quite far detached from actual ROI without anyone noticing (and with plenty of people happily riding the gravy train).
--
[0] - https://news.ycombinator.com/item?id=10872359
The large ad firms deliver a very good product but it usually isn't cheap.
I've personally dealt mostly with the other end - the individuals and small agencies - and what I saw revealed total lack of necessary competences for the reports to be corresponding to reality. Perhaps it's understandable - after all, people who have the required knowledge likely end up working for big advertisers, or in entirely different fields. But small business owners don't pick these big advertising companies either.
The question isn't are the scientists good, the question is if the vendor is honest.
It seems like it would be important to measure this standing and turn down spend once it is reached. I can see how there's a lack of incentive to help large advertising spenders understand this.
In a huge number of cases these tests are run by people that don't have the statistical background to property run and understand these tests, in the remaining cases there is almost never follow through to ensure that the results of the test have persisted after the experiment.
People will say "oh this test shows 10% improvement, and this one does too! and this one! and this one!" But then you should see a nearly 50% improvement just because of ad spend and you almost never do. Nobody ever check that all the numbers add up, they just want the numbers that someone reported to feel like they are sound enough to hold up to scrutiny but scrutiny is never applied.
A small tech team investigated fraud on our platform and developed a system that was pretty robust at detecting and potentially shutting it down. But literally nobody was interested - even the people advertising don't want to know.
The people spending money are typically networks, media buyers, ad agencies, etc, far removed from the actual brand.
There are so many parties who want a slice of the brand's cash that they are all long past caring about whether the ad is being viewed by a human or not.
I assure you the big brands CFO's take a huge amount of interest in what is spent on advertising.
Other dept;s like IT have to justify every penny, but Sales and Marketing not sooo much
I've seen it not just in the phenomenon you describe, but also in layoffs and restructurings.
Engineering , R&D, Manufacturing, Tech, Testing : "cost centers". Ever tightening budgets.
MBA, Beancouting 101
IIRC our system wasn't even fancy ML - just finding known user IDs with identical timestamps over many simultaneous clicks / impressions on really diverse publishers.
I worked at an ad company. It was an absolutely standard metric to eg geo-fence ads out of a state or two for 3 months to demonstrate the impact of ads. This isn't easily externally visible, but tests like this are standard practice.
Particularly in the app install space, which is sketchy as hell once you stop buying from the top handful of vendors, buyers should be auditing by a couple million in annual spend. To get to $150m without looking hard at big chunks of their spend is just plain arrogance and/or incompetence.
The technical aspect aside, there are a lot of "soft" factors that help: regular communication of easily consumable numbers/graphs/metrics to the client, calling out inconsistencies, etc. In other words, this is far from a fire-and-forget exercise; this can be an intense monitoring and client-engagement exercise.
The regular client engagement sometimes helps you in pinpointing cause-effect in the observations too. For ex company X might decide to advertise less of product Y in a certain state Z because of new laws there. But the team you're interacting with might not be aware of this change, or might not be cognizant of its potential impact on an advt. campaign. Regular dialogue helps here since you might observe a change in sales trends, and bring it up in a meeting - and the client team might be able to then rationalize the change. This is healthy for both parties.
And then using data from a one system that went down and nothing happened as a proof that systems reliability doesn’t matter at all, and it’s huge scam by engineers.
Because advertising in some form certainly works. If you can determine that approach "A" that everybody is doing is actually a waste of money but approach "B" is effective, then you can develop services around approach "B" and market them based on these findings.
Any large company could invest in some experimentation, whether their marketing directors want it or not. It makes sense that at least a few do but just don’t publish the results
It also happens to be the sole form of revenue all of the largest tech firms enjoy.
No wonder there hasn't been a real audit.
Also, all those marketing execs buying all the ads would be out of a job too, so they aren't speak up either.
I don't think this is true unless you qualify it a bit. Apple, Amazon, Microsoft, don't make most of their money on ads. Even Google has other revenue streams.
How long do they survive in their current form relying solely on them?
Or they have run the experiment and the results haven't lines up with their own personal biases so they were discarded.
Half was true in his time (a century ago) but I think our numbers are way worse.
Not without reason. Even without the conflict of interest that Nextgrid points out in a sibling post, there's still a significant financial barrier to attempting to measure this stuff. According to a former professor of mine who spent a large chunk of his career studying this stuff, the size of study you need to conduct in order to get any kind of statistical power at all on an ROI study is just absurd. See, for example, the treatment starting on page 15 of: http://www.davidreiley.com/papers/OnlineAdsOfflineSales.pdf
https://www.youtube.com/watch?v=oXBqzpExvrk
Sure, everyone wants that. Precise attribution has been the holy grail for a long time and the struggles are far larger than just a few technology products. The new battleground is first-party data and clean rooms vs privacy regulations. And that's after dealing with all the politics and perverse incentives that happen in such a massive industry.
Or, no one is willing to share the results of the experiments.
It's not currently possible to run an A/B experiment with a hold out group of potential customers across all channels, let alone for any longer duration experiment. So how can we separate cause and effect? (although pay per conversion channels do get gamed left and right)
Maybe it's even enough if you simply just sell it via mail order, you can then look at the addresses.
There's probably a natural information spread in any market (word of mouth, trade magazines), and there's probably a physical dispersion of the target group of people too (people move, visitors/tourists saw the ad/product and order it at home), but it still should be a valuable to see how much effect just one campaign has.
Maybe one of the best products for this could be a car. They are pretty standard, really not much difference between them, they are in all price ranges, and regularly new models come out. Advertise one in a few major US cities but don't in others.
To see this in action - spend a day watching pitbull videos on youtube and see how many spanish language ads you'll get.
Plus, maybe it's possible to somehow offer a coupon for those who buy via the ad, and so on.
After all the ad/tracking industry probably have a thousand tricks to increase accuracy of this.
But, yeah, I'm not holding my breath for a conclusive answer on this.
I understand you no longer work there, but ads started in 2009 I believe, so you perhaps had some input on this?
If you're doing an $XY,000 ad campaign, it's a waste of your time and money to figure out which 50% it is.
If you're doing a $XY,000,000 ad campaign, you employ people whose job it is to figure out which 50% it is. The thing is, you usually don't go ahead and pay them to blog about it.
At worst, it’s a front for building “profiles” of everyone.
I can’t recall the last time I bought anything -because- of an ad.
If anything, ads have sometimes actually put me OFF from buying something!
For example we can probably agree that for completely new companies spending on ads makes sense. Or giving out free samples, etc.
Similarly for big companies doing media campaigns to keep the new ones at bay makes some sense.
Even if word of mouth is a thing, even if there are organic searches, and even if it seems like a race to the bottom if everyone just tries to outspend each other.
It'd be great to make experiments about how to sensible prevent/regulate this ad arms-race. But first better data privacy laws.
https://news.ycombinator.com/item?id=25624112
“URL shorteners set ad tracking cookies”
There is so much money at stake that could be either saved or generated, its simply not possible that no one has looked at it. I used to help Pepsi/Fritolay set up tracking to tie advertising on youtube to in-store sales. They spent millions of dollars to measure their ads, Google had a clean-room data center specifically for pepsi/frito. The idea that no one actually checked if this system works is simply not possible.
Second, inefficient relative to what?
Third, so far, what you've written here looks like a rant, not a predictive theory nor a powerful explanatory theory.
Would you like to quantify your claim? Or at least make it more precise? As it is, I don't think it advances your argument.
I'm interested in strong logic, data, explanations, and persuasion. I see none yet.
The sad thing is it's not inefficiency relative comparable things. But anyone that has worked at these places or sold B2B to them just knows it on intuitive level that they are garbage, and need a heavy sedative to think there are no alternative.
> not a predictive theory nor a powerful explanatory theory
It's not. It's about letting go of some efficient market ideal and then finding new ideas.
We can look at Fortune 500 case-by-case to learn new things
> Cola cola
Sugar drug cartel. Despicable business with very stable revenue despite being a net drag on society. (At least "regular" drugs have a lot more upside!)
> Proctor and Gamble
Just as restaurants are reaching down market, and the inefficiency of everyone cooking and cleaning is starting to have market implications, we should see their reign finally dwindle. Wash-and-Fold should follow laundromats. The specialization means that stupid differentiation between products for uninformative consumers (c.f. https://en.wikipedia.org/wiki/Monopolistic_competition) should go away and restaurants and laundromats optimize.
Personal soaps and cosmetics (of course many soaps are cosmetics) however will stay as cultural reasons ensure people will continue to clean themselves and not contract that out for the foreseeable future.
It might be interesting to define a systematic and testable definition of organizational efficiency relative to an ideal market. Just like combustion engines can be compared to their theoretical maximum efficiency, it would be interesting to compare a particular organization against its maximum potential.
Here is another angle on the topic of relative efficiency. Consider a particular organization. Coca Cola will do as an example. Given its environment (financial incentives, regulations, cultural norms, etc), are we surprised by its behavior? This raises the question of how much an organization's particulars (e.g. leadership, history) affect its efficiency.
I suppose I take a systems view of it. Many deplorable businesses survive in niches. But can you "blame" them? It is a matter of perspective: organizations, like life, adapt to their environment and modify it to their liking. Such organizations can stretch the environment to or past its breaking point for a relative long period of time before having to deal with the downstream consequences.
Elaborate on your reasoning here? While I can maybe imagine a more centralized food preparation system being more efficient, the currently growing market solutions (DoorDash, etc.) are certainly not, and I can't think of a system in which transporting clothing somewhere for someone else to clean would be more efficient than an in-home washing machine. The wasted energy alone would be colossal.
Apple's half of the phone Duopoly. Either you pay them or pay Google if you want a phone and want to buy software for it.
Calling it all "rent seeking" is not a hill I want to die one.
Exclusive / Special agreement with serving only one kind of Soda ( or no Pepsi where there is Coke ) are Anti-Competitive arrangement. Not Rent Seeking.
This page [0], for instance, says the cost of goods sold for the soda itself is a penny an ounce for the syrup and CO2. Iced tea is apparently the margin champion, as the same page indicates it can cost as little as a penny per glass.
[0] https://www.restaurantowner.com/public/Restaurant-Rules-of-T...
Both are propped up - actually maintained - by huge and vastly expensive brand management strategies.
The criticisms of individual campaigns here are missing the point. It's not about micro ad spend, but the perception of value and manipulation of behaviour created by the combined effect of multiple PR and advertising efforts - which include traditional print and TV/radio ad campaigns, guided advertorials disguised as news in the MSM, interviews with prominent figures, political campaigns of more or less obvious relevance to the core business activity, state, national, and international political lobbying, direct political campaign donations, advertorials masquerading as "freelance" journalism and blogging, managed astroturfing on social media, shareholder relationship management, product placements in movies and music promo videos, articles about commercial visual design elements in trade journals. And on and on.
That perception of value is - unsurprisingly - extremely valuable. And it's very much a US way of doing business. Instead of producing products that are inherently superior, produce something that is functional but glossily packaged, brand it as a premium lifestyle commodity, and charge exorbitant prices for it.,
The prices are traditionally far out of proportion to its actual utility. In fact real utility may well be negative - see also diabetes and any number of other health issues, depression associated with social media use, debt-driven spending on lifestyle products. Etc.
So the rent seeking comes from a kind of cultural squatting. There is value in dominating discourse in all of these different ways, because discourse and narrative define markets and ultimately control behaviour. And while this is happening other kinds of discourse - which may well have more real utility - are diminished at best and crowded out at worst.
So in this case it doesn't matter if Uber "wasted" their money. Uber have their own branding thing going, and explicit ad spend is a tiny part of that.
And even if all online ad spending ended tomorrow, a small number of corporations would have a difficult time for a while, but the marketing industry as a whole would inevitably interpret the change as minor damage and route around it.
In the case of Coca Cola, I agree - in the case of Apple, not at all. "Keyboardgate" aside, their products are vastly superior to the competition in build quality and life time. A typical Windows laptop is unusable after two, three years (almost all are made from plastic which breaks or looks extremely ugly rather sooner than later), and has next to no resale value while in contrast even old cheesegrater Mac Pro's fetch many hundreds of dollars today. iPhones are supplied with firmware updates far longer than most Android phones outside the flagship Pixel line are. OS X is, while it has gotten locked down a fair bit over the last years, still way better than Windows as it doesn't do tracking bullshit and start menu ads and decently better than Linux because no matter which device, stuff usually Just Works (tm) without having to fiddle for hours to get something as basic as a Bluetooth headset working.
Apple built themselves a loyal following with pure quality superiority over the competition, unlike HP or Dell who are only surviving because of enterprises who follow the "no one got fired for buying IBM" line.
And that doesn't even touch that it's Apple who's at the forefront of innovation. iPods, iPhones, iPads, Apple Watch, AirPods, now the M1 CPU - these entire device classes were created by Apple. When was the last time you heard about something truly innovative from the Windows/MS/Google side of IT? Only thing I can remember is the Google Glass.
What Apple is doing is not a "brand management strategy" per se - it is delivering actual value and innovation instead of rent-seeking. And for what it's worth Tesla are doing the same thing.
Marketing the new thing is what Apple does. It is good at it. That is not the entire spectrum of customer value.
Source: forced to use one daily. If I could get a Mac laptop running Linux but with the input smarts of the Apple trackpad and keyboard, I would in a heartbeat.
Source: Me.
The trackpad is superior to apples and it can come preloaded with linux
This shows that you are just spreading FUD. A Windows machine is no slower after 10 years than it is the day you bought it. Sure you might have added so much software-crap that the software runs slow (which is easily fixed) but the hardware is exactly as fast after 10 years as Apple hardware is. Do you think Apple builds their hardware out of magic and unicorns?
Capacitor, DRAM, Heat Sink, etc.... There are quality difference in hardware. Although the two / three years remarks were definitely exaggerated.
By "typical windows laptop" do you mean a cheap model where the macbook equivalent is buying nothing at all? Of course it's going to be lacking in that case.
Well-made windows laptops exist, and hold plenty of value. But you have to pay for that, no matter what brand you choose.
> tracking bullshit
Like checking signing certificates by pinging servers almost every time you run a program?
1) Advertising works.
2) Coke, a sugary drink, is largely kept alive by smart marketing, that involves mostly advertising in the end.
3) Much of advertising is quite wasteful, because it is a fuzzy instrument - but on the whole, good marketing works.
4) There is a lot of self-awareness in the industry on this, and no doubt, a lot of borderline fraudulent actions by participants willingly spending money they know doesn't work.
5) Ad networks will happily sell you ads that don't work, and look the other way when there are shenanigans.
The problem here in the thread, is that people are having a hard time grasping how all of these things can be true at the same time, but it's not that hard really.
That some ad spending turned out to be 'not useful' is 'not news' when everyone knows that easily 50% of ad spending is probably wasted, we usually just don't know which 50%.
>2) Coke, a sugary drink, is largely kept alive by smart marketing, that involves mostly advertising in the end.
Not sure I agree. I haven't seen any Coke Cola Ads for years, not online digital Ads and I rarely consume any traditional media. But I will still buy Coke Cola every once in a while.
They are also on all of the convenience store with decent positional shelf space or visual merchandising. Something I learned that no one on HN actually knows anything about during the discussions of App Store.
Edit: All of my friends can taste the difference between Coke and Pepsi. It is just different. So I was surprised to read comments and youtube video about "blind tasting" on the two. No, I didn't drink Coke because of Ads, I drink it because it taste better. (And some people prefer Pepsi )
You saw 1000 'product placements' for coke without being aware of it in films other things. Taste is associated with familiarity and comfort.
I respectfully disagree, I used to work in F&B industry. And product marketing and placement has absolutely no affect on how one think something taste is good or bad. It does however remind anyone next time the walk into store they should buy coke, which does affect sales number. But not taste difference.
My 7 years old nephew hasn't watched a single Coca Cola coke in his life time, but he prefer coke over Pepsi. While my friends daughter ( and her mum ) prefer Pepsi.
Yes it odes.
"My 7 years old nephew hasn't watched a single Coca Cola coke in his life time"
1) It's ridiculous to say a 7 year old 'has never seen an add for coke'.
How would you possibly know that? Unless they are living as the Amish, that kid as seen ads.
2) That someone will prefer one drink over another is not relevant to the argument.
Nobody is making the claim that 'ads make you do stuff'. No Coke ad is going to 'force Pepsi drinkers to like Coke'.
Ads are influential. They root the product in feelings, emotions, those have impact, which is why advertising is there.
3) Your nephew was probably drinking code from some age, maybe as a treat. It was Coke and not some Italian cola, because Coke has such a powerful market position.
In Italy, they drink Brio and it's quite good, you can't even get it here, if you could, people would drink it.
Coke spends billions on ads because they work not because they don't.
Also, there's one alternative that's often forgotten in these discussions: Perhaps game theory is at play. It may be, for example, that, across entire industries, advertising costs more money than it's worth. But that everyone has to do it anyway, because anyone who chooses not to will start losing ground to everyone else. IOW, just like in the standard prisoner's dilemma, choosing to act is less about increasing your potential gains than it is about limiting your potential losses.
There is an interesting long-running natural experiment in the pharmaceutical industry that suggests, albeit inconclusively, that this is the case.
> these are ALL diligently measured at every stage
Mmm. Sometimes. Sure, you can obsessively measure audience and sales numbers, but if you don't have any controls then it becomes nearly impossible to figure out how much of that came from ads and how much came from everything else in the entire world.
If Coca Cola cut advertising 90% this afternoon, they likely wouldn't see the consequences until a new generation or two grew up.
There's also a huge mutual back-scatching thing going on. I remember in a former life we wanted to pass on a big software vendor's user group show because, while we sort of needed their software for some important customers, we got very little traffic at this expensive event. Their CEO called our CEO and basically said to him "Be a pity if something happened to our partnership."
You seem to be assuming that the potential gains are limited to current players of the game in this comparison.
It's possible that Coke, Pepsi etc. are all losing more money on ads than they're gaining in market share against their current competitors and in increased sales. But that doesn't begin to address the question of whether advertising is a net loss.
For one, the advertising also serves as a collective moat against new competitors, which could enter the market and eat up both Coke and Pepsi's market share. Maybe their biggest gain from it is that a bankrupt soda company from the 80s didn't replace them, and that a similar company today wouldn't have room to enter the market.
It also doesn't account for more complex effects like a net increase in sales for all companies in that market. E.g. maybe Rolex, TAG Heuer etc. all benefit in the long term from expensive watches being seen as fashion accessories.
If Rolex stopped their advertising spend one month and saw little immediate change, then TAG Heuer etc. might follow suit. But both might only experience the real loss years later as "luxury watches = must buy" faded from the psyche of their current and potential customers.
This is the biggest large scale test of advertising I'm aware of. But it probably doesn't apply to all products.
It also means advertising is a net loss to society, but that is separate from the question 'does advertising work'
Not so clear, especially before internet search engines existed. As a consumer, I find it useful to know what's available for me to purchase.
If this is even remotely close to reality then it makes sense to me. Companies are more concerned with constant growth than strict efficiency, imo. They're throw as much money around as possible, and every cent lost or left on the table is panic inducing.
I also imagine different types or products and/or markets behave quite differently. Eg a new product might very well benefit from advertising - since no one can buy your product or visit your store if they don't know it exists.
Why?
More complicated since there are multiple device graphs/etc in play, but effectively tens/hundreds of millions of dollars at play and no one trusted anyone. The largest thing at risk was YouTube’s ad revenue.
Have definitely heard that one before. No disrespect.
Depends enormously what sort of business you're in. I used to work for a company where all of our sales came from ads, 100%. It was trivially true that if we stopped advertising we would have no sales. We were also committed to running experiments: we knew how well all of our many advertising channels performed, and we ran A/B tests for every change.
That's a philosophical question of whether you consider statistics to be "proof".
> No one is willing to run the experiments necessary.
You're nuts if you think this is true. I assure you that companies in traditional industries (i.e., without venture capital) can and do run these experiments.
The Uber story is about venture capital and its anti-market incentives, not about the ad industry.
The latter is all about incrementality. When you adjust for it, you get the true effectiveness of ads. In some instances, the incrementality will be significant, in others it won't. It looks like that in Uber's case, the incrementality was low. Btw, if you're not familiar with this term, please look it up - there's a ton of literature on it: https://www.adroll.com/blog/marketing-analytics/beginners-gu...
Which brings me to the point about bad employees - marketers who are not adjusting their performance measurement based on incrementality should be fired (and it certainly looks like this was not being done at Uber at that time).
I guess you could propose a conspiracy theory that all these people are conspiring to put together fake reports and data sets, but that sounds extremely unrealistic - ad agencies are essentially money-counting businesses like banks, and they take data security very seriously. (Because usually they'd be audited at every step.)
In short, smart people have already thought about these issues, and there isn't any inherent pro-advertising bias in industry, rather the opposite.
Tesla is one natural experiment about not spending money on advertising in the mass media compared to traditional car companies that spend HUGE amount of money advertising.
https://www.motorbiscuit.com/gm-spends-an-embarrassing-amoun...
"Hyundai spent $4,006 per Genesis vehicle sold in 2018. Ford’s Lincoln brand came in second with $2,106 per vehicle sold. After Jaguar and Alfa Romeo, GM’s Cadillac brand came in fifth with $1,242 spent per vehicle sold. Tesla was the lowest at just $3 spent per vehicle sold."
If Hyundai couldn't keep Genesis vehicles on the dealer lots, they'd advertise them less too. Having a dealer network means dealers that want manufacturer support in advertising to keep dealers happy, even if the new cars sell themselves, dealers need to get people in to sell used cars.
I suspect that if our browser isn't blocking ads then our brain is. It's complete conjecture, but I assume that adblockers eliminate this cognitive load explaining a portion of why they became successful before they were necessary for security/malvertising.
https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...
https://www.japantimes.co.jp/news/2017/09/19/business/corpor...
I would not be shocked to learn that advertising for a specific product or event is not particularly effective. However, I'm inclined to believe it has a huge effect on overall brand recognition.
Let's say you go to Amazon to buy a roll of toilet paper. How do you choose from the literally hundreds of options? You could spend a day of your life reading reviews, and trying to parse which ones are fake. Or you could buy the toilet paper from Scott because you recognize the brand.
As I see it, buying brand advertising is a lot like buying an expensive suit. It's not that the suit makes you more productive, but it is a sign of professionalism, and—frankly—of wealth. If a brand is advertising everywhere, you know they aren't a fly-by-night company, and their products likely meet some standards of quality.
https://news.ycombinator.com/item?id=21465873
"When Tadelis was working for eBay, the company was in the practice of buying brand-keyword ads. Which meant that if you did an online search for “eBay,” the top result — before all the organic-search results — was a paid ad for eBay."
This doesn't show that advertising doesn't work per se, it shows that eBay didn't hire a competent ad buyer. Whether or not you can prove the efficacy of advertising as a whole, this is not a valid approach.
First of all, they didn't differentiate between display or PPC advertising. PPC you only pay if the user actually engaged with the ad. Nearly all of the anecdotes they used where about online display advertising - a known crock.
And we absolutely run experiments all the time! In fact, we tie our ad spend directly to conversions. If anything, the market is too efficient - it's really hard to get more than what you are paying for.
Or you can do the equivalent of an excess mortality study but for client acquisition or product sales. You take something like the month of January and you say "this is what we expect for web traffic given our standard traffic for that month, and extrapolate it with your average web activity growth". And so long as you have "clean" months to compare it to (with no ads running), you can get a pretty decent back of envelope about what your ad campaign did.
Like, there's so many different ways you can cut the apple, it's patently ridiculous they made those claims.
To give them a bit of credit, there's something to be said for there being a lot of bad ad spending out there. But like the stock market, there's a tacit understanding that it's more or less efficient.
Example: TVguide.com/listings. On most devices, I get a nondismissable audio ad when I'm looking at telecast schedules. I kill the audio immediately.
I never experience the bulk of the ad, and even if I can identify the product being advertised, I'll form a strongly negative impression of them.
I don't count this as "engagement", but it's charged as that.
One thing we learned a long time ago is never to trust "engagement" data from platforms (be it Google, Facebook, LinkedIn, whatever). There are enough random user behaviors that you generate a ton of noise. You need to build larger, more robust attribution models that tie ad spending directly to revenue sources - new leads, new accounts, purchases, etc. Patting yourself on the back for how many shares or clicks you get is not great.
and of course for more detail the full text of the fraud suit is public https://www.adexchanger.com/wp-content/uploads/2019/06/Uberf...
Positive lift in the range of 0-20% is very common, and many statistical aspects of causal inference on ad impacts are well understood.
Negative lift and flat campaigns are real phenomena too, and it does deserve more widespread publicity that negative lift happens in an appreciable number of campaigns, but that doesn’t take away from the overwhelming evidence that digital advertising works and that the mechanics of positive lift are well studied.
Here are two of the foundational papers in this area:
- https://www.kellogg.northwestern.edu/faculty/gordon_b/files/...
- https://www8.gsb.columbia.edu/media/sites/media/files/Garret...
Particularly Figure 1 (page 26) in the second link. That figure alone utterly refutes any nonsense claim that digital advertising doesn’t have provably positive ROI.
They back up their claims with studies of their own as well as metanalysis.
The ebay example twists the concept of user acquisition (new customers) and purchases (new or old customers). It is a common tactic to buy advertisements defensively, for example, if you're a product manager, and have determined that some of your user base are more transactional rather than frequent.
Another pet peeve I have is how they conflate direct response advertising and marketing.
I definitely trust the academic studies over an entertainment podcast.
Ads might be less efficient than some believe, but it's super easy to see that they "work", and advertising platforms do it constantly.
But none of them test "no ads" vs "ads".
The reviews for the movie were poor but it had lots of household names as its voice talent, including Anne Bancroft in her final film. Good or bad, advertising likely would have lead to more people seeing it than two per screening. Of course there's no way to know for sure how many more would have been enticed by the ads but we do know that going with zero advertising resulted in a huge disaster.
Eminem released his last two albums without even announcing them. Word of mouth got them both to #1 on the charts.
If you tried that same movie experiment with something like Avengers, I bet the results would look a lot closer to Kamikaze than Delgo.
As for movies, Avengers got to where they are today through lots and lots of advertising.
On the other hand it got displaced by Oogieloves which had $40 million in marketing costs. Critics found it mostly bad and the only award it won was for films produced in Brazil. Maybe there was a reason no one wanted to spend ad money on that movie?
This is incorrect.
Ok, how does those companies know it works? They don't have any real data to show that it does, just the fear that if they stop they'll lose a lot of business.
The amount of data generated by adtech today is staggering. The problem isn't data or advertising, it's the wrong people running the wrong campaigns for the wrong reasons.
This may be true but it's a separate issue than the fraudulent clicks.
Brand advertisement (the kind you are talking about, as opposed to closed-loop direct advertising) is an investment in a brand, that doesn't get paid off in a day, or a week, or a month, or even a year. It adds fractions of a penny onto a customers value, every day from today into eternity. It's an investment in your mind, but the long time horizon makes it, as you point out, nigh impossible to measure ROI.
However, this is one of those cases where despite being immeasurable, it still works.
Imagine doing a study on low-fat or low-fat diets and trying to measure health outcomes like lifespan, or heart disease, or cancer after just one month. You can't do it. The best you can do is measure markers of these outcomes, like insulin resistance, blood triglycerides etc. Brand advertising is similar. You measure markers of long-term purchase intent. It's not perfect.
And just like actually doing a long-term study of diet for example is riddled with confounding variables and very hard (nigh impossible) to do well, so it is with advertising.
This is entirely false. I work in adtech, and almost all companies run experiments in order to optimize their ad spend (everything from Ad A vs Ad B, to Ad vs No Ad, to Channel A vs Channel B, and more).
This isn't to say that advertising always produces ROI. Quite often, experiments will be run that will show a certain strategy isn't performing well, and the company will adjust accordingly. It's incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.
Isn't it possible that these experiments you talk about are fatally flawed? I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.
Sure marketing people want to protect their jobs but the odds of “advertising is entirely worthless” being this closely guarded secret kept by millions of people in the industry or being a collective delusion is pretty darn low. The reward for defectors is just too high.
Platforms can accurately determine who engaged with an ad (basic logging on their sites), they have infrastructure to create statistically balanced ad experiments, and can also accurately determine whether a conversion happened (either through a conversion pixel or through data brokers).
Running these tests on behalf of advertising clients, or for internal research is fairly standard. If we couldn't prove statistically that our ads were working, I would have left a long time ago.
Of course, this just reinforces your point that experimentation in adtech is largely not subject to the same issues that have fueled the replication crisis in academia.
(I'm a data scientist but not in adtech.)
> I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.
By "company" are you referring to individual advertisers or digital advertising platforms? Why do you doubt that either are incapable of running well-designed experiments? It's in both parties best interest to spend ad dollars as efficiently as possible.
While it's certainly possible that every one of the 100 million+ ad experiments ran over the past decade have been fatally flawed, it's highly unlikely. I guess the alternative scenario is that digital advertising is one giant hoax being kept secret by the millions of employees who work in the industry, but given how easily tech news leaks, this also seems extremely improbable.
I've also worked in adtech, for quite a long time on all sides, and my experience is that far more companies do research to justify their ad spend not validate it.
I have managed plenty of A/B tests in my day, each one claiming to show some improvement, 10%, 15% etc (some, as you said, showing no improvement). Even though these tests are often run "correctly", essentially nobody goes back and asks "wait, we had a 10% increase multiple times in the past year, but is our <metric> really showing the cumulative improvement we expected?"
The greatest trick in the industry is that because VC are pouring money into everything, everywhere, every metric appears to grow. At every startup, everywhere, pre-pandemic, numbers were going up because people were pouring money into the system.
I've worked at companies who I know for a fact their adtech product cannot and does not work, yet their business continues to explode because in recent months nearly all ad spend has been on digital ads.
I've talked to companies whose entire function is bidding optimization who literally do not understand how to optimize bidding given the information you have.
Absolutely people are running "experiments" but the function of the "experiments" is to justify that the ad team is worth having, and then that the VP of marketing is doing their job and then that the CEO has hired some super smart people, and then that the VCs might have really found a unicorn. Everyone sees what they want and no one really wants to ask the question "wait, does this really work? are these tests really able to capture the complexity of the environment?" And if you are one of those ornery people that insists on probing into the details and seeing if any of this is working, you will eventually get fired.
Ad tech is largely a scam, but a huge number of rich and smart people benefit from the illusion that it is not, so we continue to see experiments showing that everything works as expected.
I don't think your view is representative of how most F500 companies, or many of the new DTC brands, invest on major ad tech platforms though (FB/G/Snap/etc.). Experienced marketers try to measure metrics as close to core business KPIs as possible, and comparing ads based off of simple A/B tests is increasingly becoming a technique of the past. Measuring the incrementality of ad campaigns through long-term holdout groups gives companies a much more accurate read of their investment, while bringing higher statistical rigor as well. So rather than looking for cumulative increases on <metric> after advertising for a year, you could instead point to group A (who saw no ads) and see that group B (who saw ads) drove a 1.5x higher <metric>.
I'm not sure how you can say advertising is a scam when there are clear examples of popular brands that most likely wouldn't exist today without digital ads? Take Allbirds as an example. There are dozens of consumer shoe brands that are already in physical stores and have higher brand recognition; how can you argue that advertising didn't help them cut through the noise and grow their bottom line?
We have a very large and conspicuous example here that suggests that Uber hasn't been doing this.
Running experiments like you suggest is difficult and to do them right requires some fairly specialized knowledge to do correctly. I'd doubt more than a small percentage of companies have the expertise to effectively run any kind of advertising experiment that returns useful data. Most businesses lack that knowledge and rely on metrics provided by the people selling the advertising—who are likely to provide self-serving numbers.
While I'm sure some of the bigger F500 companies do a good job validating their ad spend, I most companies don't even know how. Certainly the majority of small businesses have no idea how effective their advertising spend is aside from very crude word-of-mouth feedback. I suspect this creeps way up into the Fortune 500 as well.
I'm not sure this article is the smoking gun that the author makes it out to be. By 2017 Uber had reached saturation in what could reasonably be considered their addressable market. From my perspective, turning off app install campaigns and not seeing any dip in acquisition validates this position.
Meanwhile, their other major app, UberEats, is a business that exists in a highly competitive market, and they continue to invest in app marketing in order to grow that division. If the takeaway from this article was that Uber discovered app marketing was useless, I doubt you'd see them make that same mistake again.
> While I'm sure some of the bigger F500 companies do a good job validating their ad spend, most companies don't even know how.
If anything, I think bigger companies are more prone to overspending/spending inefficiently. While they do experiment to try and optimize strategy, there is some business inertia that gives them the flexibly to move a little slower, since one poor marketing decision will not sink the business overnight. Conversely, small businesses don't have the resources to investment as much in marketing. This leads to poorly run ad campaigns, but rather than continuing to invest in a bad strategy, they typically just kill the effort altogether.
Most small businesses don't need complex experiments to understand how marketing effects their bottom line - poor ROI is a lot more apparent when you're low on funds. That's also why it's in an ad tech platform's best interest to make efficient advertising as accessible as possible.
Counter-example: I know of at least one consumer goods company that has studied the long-term effects of certain kinds of sponsorship deals on consumer behavior. The study had tracked people for at least 10 years at the time I learned of it. Of course, this company would never publish the results, because they provide a competitive advantage in structuring and bidding on sponsorship deals.
One of them said tracking cookies only boosted conversion 4%, and another said P&G did better with some traditional media advertising than some digital advertising.
Because advertising is like military spending. It takes a lot of money to maintain the status quo.
Traditional ad platforms like TV, newspapers might not have done this but online ones surely do. Infact that is one point they consider an advantage as you can measure effectiveness unlike the traditional platforms.
The problem is knowing exactly which formats and campaigns are working down to the dollar, but part of that is just the reality of fuzzy attribution and it's only getting harder as privacy regulations get stronger. However you can definitely tell the difference when turning everything off, and if you can't then you were advertising to the wrong people in the first place.
Uber's mistake isn't that advertising didn't work, but rather that they didn't vet their vendors or even bother doing any checking and optimization of their own.
Companies like Uber and Ebay have turned off all their adspend and saw little to no change in their acquisition metrics. You can argue that they were just doing it wrong. But the point is that, if even they are doing it wrong - and getting nothing in return for the millions they're spending on ads - then it's very likely most others are in the same situation.
You are right that this doesn't mean _all_ advertising is useless, there are absolutely profitable usecases. But the larger points still stands: most money being spend on advertising right now is likely not returning anything.
We now have some strong precedents being set. I believe this will cause more major companies to run the ultimate experiment: turn off all ads and see what happens. It's too early to tell, but it's not impossible we'll see adspend drop significantly across the industry once everyone finds out they're just burning money.
Fraud is a special case because it's criminal activity and has nothing to do with advertising. It happens in every industry but it's especially easy with online technology spanning multiple countries and data that can be easily faked. Uber was exceedingly oblivious here but I wouldn't extrapolate advertising efficacy from these examples.
Happy to chat with anyone who is interested in the topic (pfalke at pfalke dot com).
Haven’t had a chance to listen to the podcast, apologies if that made me miss the point of the parent post!
Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.
[1] https://academic.oup.com/qje/article-abstract/130/4/1941/191...
But if you can't see any change in business from 2/3 of ad spend at all it must be fraudulent.
Seems to be entirely subjective.
I bought a new blade for a bread cutting machine on amazon, the last one lasted ~25 years. My current recommendations: blades for bread cutting machines.
I also checked some Christmas decorations on Amazon, but didn't buy any. So next to the blades I get Christmas decorations recommended. January is a bit late for that, isn't it?
I also bought a single server rack on eBay. Now I regularly get spam emails from eBay reminding me of new offers for server racks. Even the company I work for doesn't have that many, so why?
In short I am not really convinced that these platforms are good at predicting what I want. Rather it looks as if they are good at showing me what I already have.
But besides doing something like "purchases >$100 are not likely to be repeat purchases," I would think it would be difficult.
They show it because it "works" for them, not because it's wrong and they just can't somehow make sure they don't show something.
Because even if 99% of people don't buy anything twice, 1% probably does for some reason, and that signal just trumps all the noise. (And that's why most of ad spending is bullshit, but since Amazon's cost [even with opportunity cost] is basically 0 on their own site, they'll continue to do this.)
For instance, I recently bought a new iPhone directly from Apple, and I bought a case for it on Amazon. Now two months later, Amazon is recommending that I buy an iPhone (...which I bought two months ago from Apple).
Buying a case an then waiting two months to buy the actual phone would be a very strange thing to do, but Amazon seems to think I should do this.
And Amazon is definitely not alone: Google has been troubled with "AI" running rampant in search results for years, seemingly with no or very little QA.
While that's definitely true (I get ads from Amazon for stuff I've already purchased all the time too), in that case Amazon is leveraging their own marketing channels to support their own business -- which has, effectively, zero marginal cost to them.
However, when advertisers buy space from Google or FB, that's not the case. There is a definite external cost to such ad buys.
As such, being able to identify the "value" of such expenditures is (or can/should be) important to the advertisers.
That said, much of advertising is based upon the idea of "top-of-mind awareness."[0]
The idea being that if you are considering a purchase, the brand that comes to mind without prompting when considering that purchase will be preferred over brands of which you aren't immediately aware.
Purchase decisions made on a whim or without any research are usually those which are low cost. Which is why companies like P&G, Coca Cola and like companies focus on "top-of-mind awareness."
If you're going to buy a washing machine or a riding lawn mower, you're much more likely to do research than if you're going to buy a soft drink or a bag of chips.
That said, the same "top-of-mind awareness" can be helpful even for big ticket items, as those brands may well be the first ones about which research may be done.
Identifying how such advertising may impact purchase behavior is a complex topic, and unless you can quantify specific ad-views to actual purchases, doing so is generally speculative and is the subject of a great deal of quantitative market research[1].
[0] https://en.wikipedia.org/wiki/Top-of-mind_awareness
[1] https://en.wikipedia.org/wiki/Quantitative_marketing_researc...
I run an ad agency with relatively well known clients and I consider the word "programmatic" in a resume a negative correlation.
Doing it directly means placing ads yourself on the biggest platforms (Facebook, google).
Direct response just means "if you actually want results/ROI", as opposed to for nebulous brand goals.
More interesting to me is the fact that tech companies are finding it surprisingly difficult to control where their ad-spend goes. I suppose this is an inverse of the problems with supply chains, where Apple can take three years to get a connector vendor out of their supply chain even when they were using literal child/slave labor. It's a market for lemons; bad "money" (publishers, suppliers) drives out good. The question is: will questionable ad publishing actually harm corporate reputation to the point where big ad spenders go away, or will we just see periodic Adpocalpse-style waves of spending being decreased and then brought back?
There is a really good essay about this that I re-read quite often: https://zgp.org/targeted-advertising-considered-harmful/
Just because it doesn't make a person consciously stand up and say, "I want a Snickers" does not mean the ad didn't work.
Like, honestly, are there people here suggesting viral ads don't work? Or don't understand the point of Coke or Mercedes Benz ads are for? Coke establishes themselves as the defacto soda. There is no other soda, or if they are, wish they were coke.
MB ads among other things reinforce to MB owners the wisdom and luxury of their brand. To own an MB is to be part of a club, one that is advertised across the media spectrum. If MB didn't advertise at all, they would either become another commodity brand, or have to be so luxury that only word of mouth is necessary (Bently, etc).
Now, maybe that's what cortesoft is saying "get you to waste money" but if it got you to waste money on their product, it worked.
My point was that advertising's purpose (from a consumer point of view) is for the ADVERTISER to waste money, to signal that they have confidence that their product is good enough to recover the cost of wasting money on advertising.
[1] https://www.daimler.com/investors/key-figures/markets/
Personally, I think it's less about luxury and more about value signalling. Having a Mercedes says that you're comfortable financially, and they're "dignified". I'm not saying it's true, but it's what their marketing generally implies. Luxury is generally just a dog whistle for value signalling merchandise.
Ferrari does different value signalling. They imply wealth because of the cars performance, and they're generally a much more obvious signal. They're usually a bold color like red in the ads, and the body styles are distinctive. You could miss a Mercedes in a group of sedans; you're not going to miss the Ferrari.
The private club brand, to me, is Rolls Royce. I would guess because I never see ads (and rarely see them in person), so my only references are pop culture where they're associated with the ultra wealthy.
I was recently in the market for a new mattress, so went to a mattress store and tried some out. Even though Tempur Pedic is all the rage with their insane ad spending, I was not all that impressed, and found a considerably cheaper mattress that felt much higher quality.
Now, this may all be sales mumbo-jumbo, but the sales rep noted that a lot of people come there only interested in Tempur Pedic mattresses, which he regards as "literally a piece of foam, total crap mattresses." They sell at such a high price point because they advertise so much, while the mattress I found came from a no-name brand that doesn't advertise at all, and was therefore cheaper.
So I guess in some markets, advertising does work. I've never heard of someone telling me how much they loved a mattress unless it's a Sleep Number or a Tempur Pedic, and those advertise by far the most.
One could argue that Tempur-Pedic has to sell at a higher price point to make up for the money they spend on advertising.
With Uber it would be so easy to penalize drivers who cancel because they don’t like where I’m going, but I’m still wasting 30 minutes quite often, because Uber spends that money on ads and UI rewrites instead of improving the core experience.
Also this is the former CIA employee who started a techno religion. He works with Dr. Garry Nolan at Stanford.
https://youtu.be/rvovZBT54Oo
Uber owns taxi services as a brand as much as Google does search.
(1) Due to fraud, the money wasn't actually going to ads.
(2) You didn't need the ads. You were over some saturation threshold and the law of diminishing returns kicked in hard.
(3) The ads were valuable to you but not in an easily measured way. Maybe you could coast on brand awareness inertia for 6 months or a year before it affects sales.
This case seems to have decent evidence for at least part of it being #1, but not necessarily all of it.
There was another twitter thread on here recently about the chaotic and disastrous (though successful) Swift rewrite they did which happened during the same time frame. https://twitter.com/StanTwinB/status/1336890442768547845
During those time frames is also when all of the execs were quitting or being fired, along with their internal harassment problems.
It feels like Uber has really succeeded in spite of itself.
It's telling that all the FAANG companies are basically in open violation of a number of laws.
You don't have control over your SEO results as well - but you can also measure against SEO traffic with a high degree of certainty.
All big companies do this.
Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.
But that honestly doesn't seem that important compared to the other metrics - which most functioning large companies have decent data on.
You're also never going to know how many of your customers ate Green Eggs and Ham for breakfast. It's irrelevant. You have decent insight into your direct-online ROAS, and that's unique to direct online advertising, and it's important!!
Yes, you're never going to know if that million dollars you invested in online ads was the best use of that million dollars. But that's not much different than building a new factory, either.
Edit:
Specifically to Uber's case - they did NOT turn off 66% of ALL ads randomly to no adverse effect (implying that all ads are worthless).
They DISCOVERED that a certain type of ad (paying for installs on dubious ad networks) was mostly fraud. After turning off 100% of this type of ad - they found no adverse effect.
This is a fail on their analytics team. They should've been measuring this type of ad better - especially given how big a portion of the total spend it was - and had insight into something not being right. They should have been able to do this - and if they couldn't, because the network somehow didn't give them enough data to do it, they probably shouldn't have been spending this much money for exactly these reasons!
Wait, why wouldn't that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.
But it's also critically important for both adtech companies and data scientists who work in the space to direct people's attention away from those sorts of metrics. You generally don't want to call the attention of the person who signs your paycheck toward the fact that it's all but impossible to really know for sure if your service has delivered them any net benefit.
But advertising has had a fair amount of safe-harbor carve-outs for over a 100 years or so that are not available to other industries. So it is no surprise it continues today.
edit: I hate to say it but the auto-downvoters here on HN are approaching reddit levels. Everything I said above is true.
It's not that you're wrong, it's that you're boring.
"I knew this years ago, look how my cynicism is superior to your naievite" is a boring comment to read. You could explain why you think this is any closer to fraud compared to selling anything else and claiming it's the best in its class, or what safe-harbor carve outs by whom, or why they matter more than other effects, or what you think should be done to change them, or anything more substantial than "I'm not surprised". Oh aren't you? Great, cool story bro. Except it's not /even/ a cool story.
It's old man "get off my lawn". It's every tech forum's old-man status grabbing which is now approaching Reddit /r/SysAdmin levels on HN. The only question is whether aggressive downvoting can curb its growth here before it gets a choke-hold.
https://news.ycombinator.com/newsguidelines.html
It would appear jodrellblank is perfectly demonstrating how to misuse the feature, by calling you boring. Not exactly the kind of attitude that we want to have around here.
I hope not. Some of the most informative, useful comments I've seen on Hacker News have taken the form of wild tangents.
I do see where you're going, though, that if "I agree" is sufficient cause for an upvote, then it's only reasonable that a vote in the opposite direction should be allowed to carry the opposite semantics.
It's just that, from a purely social standpoint, that's not actually how human beings typically think, feel, and react. Perhaps it could be that simple on the Vulcan version of Hacker News.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
I don't have an opinion on what "correct" downvoting culture would be, one way or the other, but - if you say "contributions that don't add much to a discussion" should be downvoted, then isn't a "boring" comment precisely one that _should_ be downvoted?
Downvoting for disagreement has always been ok on HN.[1]
I don't really like it, but it has always been the case on HN. The question is if the community voting behavior has actually changed.
[1] https://news.ycombinator.com/item?id=17666145
Thanks.
> I think people have the wrong idea about HN downvotes because they think Reddit rules apply to HN.
That one gets me, because I had previously assumed that downvoting for disagreement would be a standard Reddit thing, but that HN's community is supposed to strive toward a more "we're all mature adults here" kind of community standard.
I realize people have strong feelings about it, but that's because of the psychological intensity of the mechanism. There's no way HN would survive without downvotes, despite the fact that not all downvotes are fair. It's a critical part of the immune system.
https://news.ycombinator.com/item?id=16131314
What safe-harbor carve-out existed 30 years ago for advertising that another industry might have wanted something similar on?
The "reasoning" allowing advertising/advertisers to lie is that it is assumed consumers do not believe it.
Therefore ad companies are not going to try to prove that their product doesn't work.
https://news.ycombinator.com/newsguidelines.html
Apart from taking threads badly off topic, such complaints usually end up as uncollected garbage, since often the most unfairly downvoted comments end up getting corrective upvotes from users.
https://hn.algolia.com/?query=corrective%20upvote&dateRange=...
But yes, the question still is: does it raise intent or did the user already want the thing, googled it, and clicked on the first result (which is your ad, above the organic results that also lead to your online store)?
There is a role for advertising because Google polluted their organic search results with Ads making it harder for you to find what you want and forcing you to compete for the top Ad spot. I realize that I can only speak for myself, but I have never seen any evidence that advertising works and I've worked in ad tech.
Moreover, most companies started small at some point so advertising and maintaining advertising made sense up to a point.
And if we look at those companies where maybe advertising actually doesn't help, companies that have reached a level of success where organic referrals and public awareness drive most of their business. And they're ongoing businesses that probably reached that level through advertising and have money now to use for that.
But not only are they already advertising but they don't if their word-of-mouth/public-knowledge presence will last indefinitely, they don't know if just word-of-mouth would let them control their image, would stand-up against future competitors ads and so-forth. So, even supposing you could show advertising didn't offer any immediate increase in customers, continuing to spend on it doesn't seem to me as irrational as it sounds.
Uber shows up in the news and in conversation every day. Most people have heard about it by now. Very few people are going to install it because they saw an ad.
Not the case for your neighborhood "Bob's Burgers" or "Uber Competitor" or some other new company. Or for specific marketing actions of known companies (promotions, new services, etc)
Our industry is shot-through with idealists who believe meritocracy should be the former, when in reality it's always been the latter (and always will be, unless someone can invent omniscient humans).
Could it be that people simply don't like being interrupted and having their time wasted, whether it's by ads, marketing or PR? Just because you have a business to grow shouldn't give you the right to waste my time.
The real problem is that for large companies with brand awareness they're starting to find a lot of their spend may not be generating revenue. Ebay in the Freakonomics podcast turned of about $100M in ad spend with no loss of revenue. But smaller, companies usually don't have the staff/knowledge necessary to know how and where to use their ad dollars.
I don't think I ever saw a Whatsapp or Telegram advertisement in the first few years of their existence. There's a lot of software in particular that gets around by word of mouth alone.
Necessity for advertisement seems correlated to how unclear it is what advantage your product has for the consumer. If you make something that's like X but twice as fast or twice as cheap you don't need advertisement.
But if they want to exploit a more organic growth curve, and are in a financial position to do so, then advertising isn't so useful.
Ardour is a cross-platform DAW that has been around for 21 years. Every 2 minutes somewhere in the world, someone starts a version they got from us. We've never advertised.
Having worked with a decent number of small teams without a lot of brand recognition, ads bring in a lot of customers: it was pretty obvious in our metrics when we ran ad campaigns vs when we didn't. Sure, eventually a few of those people may have found their way to us somehow anyway, but there was a step change in volume from ads. At least in my experience, the idea that ads have no significant impact is not really supported by the evidence. I assume that's what the OP meant by it not seeming important: they're working under the assumption that "do any ads do anything" is clear and you don't need to prove the worth there — instead you need to prove which ads work better than others, and cull the ones that don't perform well (which will also help save you from scammers).
One could argue that advertising for large brands with name recognition could be less valuable. I don't really have data or experience to know that one way or another, but in a competitive market that seems a little hard to believe. Tesla is an interesting counterexample, but I think it may be in part due to the EV market in the US not being particularly competitive; no one makes attractive, mid-priced cars with long range other than Tesla, let alone having an established charging network for road trips. And Elon's antics are, tbh, a form of marketing in and of themselves; "all press is good press."
Edit: I wrote the above paragraph badly. I can definitely believe ads are less valuable if you're a big brand vs a small one. But I'd be surprised if they're useless, at least in a competitive market. People don't have infinite money, so if they're looking to buy X and an ad for X from your competitor pops up, it seems reasonable to me that some percentage of the time your competitor would make the sale and thus you wouldn't, even if organically that sale might've gone to your company instead.
Also, I wouldn't argue that even small companies need ads — just that they do seem to be effective at increasing the volume of people interested in using your product at a given time. That may or may not be necessary/useful depending on your situation. I have been on teams where we intentionally turned off ad campaigns because we learned what we needed to learn from the cohort of new users, and now wanted to improve the product based on their feedback before spending money on more ads.
It is without a doubt, and this is what the Freakonomics advertising episodes are mostly about.
Uber turns off a shitload of ad spending, nothing bad happens to new user acquisitions.
I'd say there are about a million better uses of a million dollars than just pissing them away on a scam.
Uber is in a very different position than many other companies. Anyone who browses the internet with regularity is already aware of their existence and probably just needs the right set of circumstances to come together to make Uber useful to them.
I suspect the results would be different if Uber were earlier in their adoption curve, but maybe that’s not true either. Maybe they’d be ignored for different reasons at that time.
8 million views: https://www.youtube.com/watch?v=yg4Mq5EAEzw
So, less than a viral cat video?
What they should have done was a proper hyperlocal SEO campaign like Firestone Gieco, and Mc Donalds do.
Also people either need a hired transportation or not. If they don't have a car or don't want to mess with the traffic and need to go somewhere, it's either Uber or Taxi usually.
It's not like Coca Cola, which is well known, but people could do without it (unless addicted), so needs to constantly nag people.
And it's not like some new product, which without advertising nobody would even know it existed.
In fact most of Uber's existance its operation has been 100% advertising (spending VS money to offer cheap rides and expand and gather "eyeballs" and "customers" without a profit). In my book, customer acquisition without profit is another name for advertising.
So it doesn't sound strange that it could do without advertising today.
But what if there were 3-4 strong players in the same, each eating in Uber's market share? You'll see how fast they'd found advertising indispensable again...
Except Uber, apparently? Or would the method you're talking about not have discovered that something fishy was going on? (I don't work with ads so I don't know the limitations of the type of experiment that you're talking about)
So, no, Uber's advertising here is a little different than (I think) the majority of companies. They are mostly paying for installs rather than sales / conversions. A lot of newer "app" companies could be in similar situations.
Though, honestly, this seems like a massive fail on their analytics team for not figuring this out earlier. They should have been able to see that all of these "installs" from certain advertisers were not leading to trips.
In fact, it says they turned off 66% of ads. They didn't randomly turn of 66% of ALL ads. They turned off this TYPE of ad, which they failed to earlier recognize was ineffective.
Step 1) assume your ads won't work.
Step 2) have enough analytics / logging in place to convince yourself the ads do work.
Step 3) if they don't work, turn them off.
Looks like they skipped step 2 - which honestly, is not uncommon for a fast growing business - even if they are huge and already make a lot of money.
What they found isn't even what people are discussing. They found that certain networks they were buying ads from were almost 100% fraud (which is pretty well known).
Instead, people here seem to be discussing that most online advertising is fraud, and/or that there's no way to prove it's effective. That is absurd.
A/B testing ads is a complex matter. you can A/B traffic and conversion easily, but a lot of established companies with fierce competition fight for mind share, not direct conversion; for that, you have both awareness effects (user won't forget about coca cola if they don't run ads for a month, and an ad that doesn't directly convert but increase awareness still has value) and coverage synergies (the number of repetitions in a day will increase coverage non linearly and the amount of channel repetitions will increase awareness more than a single channel view, even if it doesn't convert immediately)
This is typically an order of magnitude less than the attribution figures stated by digital marketing experts. A/B test is the right method to use, but the crucial thing is you need an earmarked population to see zero adds over your attribution window, since what you care about is impact on incremental sales, rather than incremental click likelihood.
There is a long econometrics literature on this and it is not a fussy technicality, the figures typically differ by 10x +.
A/B testing is... not a state of the art scientific research technique. Moreover the companies that provide the tools to do A/B testing are the same companies that sell you access to advertising space. I'm not saying that it's a common practice to defraud A/B tests, I'm saying that the fact that adtech companies have chosen to enable that research methodology out of the set of all methodologies they could offer suggests that we should expect, before seeing the results of any A/B trial, that the results will tend to favor the adtech narrative.
I don't think that people in the adtech space believe they're selling a bogus product, but I do think they wouldn't want to know if they were — they have a good thing going.
If you're employed in adtech you're mostly fine, skills transfer. If you're invested in adtech, do what you can to diversify away. Advertising is overvalued to some extent and that bubble will burst at some point or other. The question is just how much actual value advertising provides, how much will remain when the bubble bursts.
None of these benchmarks distinguish between the selection effect (clicks, purchases and downloads that are happening anyway) and the advertising effect (clicks, purchases and downloads that would not have happened without ads).
You can fix this by dividing the target group into two random cohorts in advance: one group sees the ad, the other does not. Designing the experiment thus excludes the effects of selection.
When you do this experiment correctly, you find out that ads have low effect or are not cost effective (as eBay discovered).
Are they easy to categorize? Or is it a big secret what's dubious and what isn't?
There’s no revenue change even after testing (when you tack on costs of ad delivery).
a) technologists screaming "ads are literally the worst societal cost, like ever" who don't want to understand the industry
b) advertising folks taking up arms to defend their shamanistic, money-printing machines without statistics (Reason No. 7 will SHOCK you)
c) A tiny, tiny group of Mandalorian-like voices who have a necessary statistics AND industry understanding who are being drowned out
(this is in jest, but I double-dare you to say it isn't at least directionally accurate)
Other commenters have mentioned it throughout this whole comment thread as "incrementality tests". Amongst other approaches, this is the way.
Freakonomics severely lack the industry understanding. Listening to the podcast was like hearing how HTML is a programming language from the kid in week 2 at code camp. Then too the article, all the issues with Uber was just doing a bad job of managing their ad spend and they can fall into group B, noted above.
The baseline of this work is a control group who see no ads and then you build your tests from there that factor in channels, cohorting, and other components to get a statistically significant outcome. Yes, this will get more challenging with upcoming privacy changes (IDFA removal, et al). However, the last 10 years this wasn't a problem and I'm sure the corporations in the identity resolution business will hand-shake on a bunch of 2nd party data deals that just move the deals done in broad daylight around identity tracking and audience creation to the alley. Further, any advertising that is tied to an already known customer is able to be backed into at an audience level with login and cookie data. Even Pi-hole users may not be exempt here.
To finish with some constructive advice:
1) Advertising is not a synonym for marketing. We're only talking about advertising in both HN threads.
2) Every industry has high and low quality. Pareto's principle should be aggressively applied to where one spends their budget in the cesspool that is the Internet.
3) If you're ever seeking an agency to provide advertising services and they don't have a qualified data science or statistics leader (10+ years work experience, degree in Stats/Math/Econ, an MBA, or similar), run. Run from those shiny-shoes gurus. Channel your inner Usain Bolt and run.
0. https://news.ycombinator.com/item?id=25620707
For sure it's harder for say Gucci to measure incremental cross-platform lift but any good large brand spends big money trying to parse it out
One example way to get some value measurement would be Facebook's powerful tools. Beyond a basic 1:1 audience 50/50 exposure test, Facebook also lets you upload offline sales (or use their pretty effective pixel for real time online sales) and then feed into their system. Using the data your purchase data FB matches to ads delivered and you (with PII but also purchase value and frequency). Then the advertiser can define lookback/conversion definitions to get a measure of incremental lift. E.g. those who clicked ad bought __ right away, those who viewed ad within 1, 7 or 28 days bought __.
For what it's worth the adtech forums I participate on view Uber as incompetent idiots on this issue (which has been reported on like last year) - all they had to do was measure beyond the initial install for say did these installs actually pay to ride...
Install fraud from SSPs and resellers is huge, doing this basic measurement and quality control is the reason Facebook and Apple have huge and growing app install business. Though TBD on Apple fucking over Facebook and keeping the attribution for themselves alone with their new privacy features
Also, despite the repetition, I don't remember what it was for. I think it had a photo of a woman outdoors.
It mentions that other big companies, such as P&G and Chase, also noticed that cutting ad spending had no result on their business outcomes.
[1] https://www.marketingtodaypodcast.com/194-historic-ad-fraud-...