Remember back in the early 2000's before popup blockers were a thing? You'd hit a page and 100 popups fill the page. When you hit the X and it would spawn new popups filled with ads.
"We're really sorry that we didn't manage to get your ad about nursing homes in this months issue of AARP monthly, however we managed to get it into Rad Skateboarders Weekly so we will be sending you a invoice for it anyway."
I think much specifically it’s internal competition. My bet, I have no insider knowledge here, is that there are a bunch of individual product owners vying to look good compared to each other here. Nobody wants to have their weekly numbers look bad because they didn’t accelerate delivery while their peers do, so as a whole the company does something bad for everyone over a trifling amount of profit.
This doesn't seem _that_ strange. In online advertising, there are the concepts of pacing (to ensure the ad campaign delivers somewhat evenly over the lifespan - not because there's any evidence that it improves performance, but because the advertiser would feel miffed if you spent all the budget in the first day of their month long campaign), and frequency capping (attempting not to show the same ad to the same person too many times - clearer benefits to performance here).
Both of these will limit the rate at which an ad spends money. If it doesn't get shown for a long period of time, both of these factors get reduced, and the ad can get selected more aggressively, particularly if you are a higher bidder than the ads that would have got chosen in your absence.
pacing... not because there's any evidence that it improves
In the limited experience I have in this arena, it does improve performance. Hitting someone a second time about t+2weeks had the biggest bump, and then the next biggest another month later at t+6weeks.
I don't know if it's universally true, and the optimization may vary depending on the context and content, but pacing can definitely make a difference for some.
I don’t think they were talking about the same person seeing the ad, but if you show 1000 people the ad a day for 10 days or show 10,000 people the ad in one day.
Unless there is a guarantee there is 1,000 new people per day then it certainly does apply. I think the usual pattern for pacing is to cover multiple bases from repeat visits to trying to cast a wider net to catch people at the right time(s) both personally and in perhaps wider campaign with other sites/mediums.
That would be different, but I think still have a negative impact. You can't show 10,000 people the same number of ads in the same period of time as usual while also delivering ads faster. So it means you're showing people more ads than usual.
But attention is a limited resource. If you throw more ads at me, I'm not necessarily going to respond to more of them. The more ads there are, the more it becomes background noise.
Facebook can show 1 ad campaign per day to the user base instead of 30 ads getting 1/30th of the target market each day over a month.
To the individual user, they can’t tell anything different between these two types of campaigns. It just makes the advertiser feel better that their money is lasting longer.
You can't speed that up en masse without showing more adds to the people than you normally would. If facebook is taking the backlog of all the ads from all the advertisers and pushing that out faster to the same group of users over a short period of time, there is no avoiding the fact that users will see more adds per unit of time during this time period.
Sometimes pacing makes a huge difference, actually, and you can smurf your performance metrics to a better place by cutting out times, day ranges, demographics, and so on.
Yeah, this news belongs in the pile "Things advertisers will probably care about," but it doesn't have a universal moral / ethical category. It's more a side-effect of existing rules being applied to an unusual circumstance (I doubt the rules were written with an eye towards "What if Facebook is down a whole day?" because that isn't supposed to ever happen). Some advertisers won't like the outcome, some will be ambivalent to it.
DSPs are like dogs if you let them be: they eat all the budget, you want them to act like cats and stop eating when they're 'full'. If the outage causes budget to be spend in an accelerated way, this will probably mean less relevant ad placements.
I mean what is a shady about the dropshipping store trying to sell you a cheap imported product for a 400% markup or the guru with the rented Lamborghini who is trying to sell you on his course on how to 10000x your money by trading forex? /s
I know the example mentioned is facebook, but I feel like a similar development is going on with other ad networks as well. Even on YouTube around 40-50% of all ads I get are by some guy trying to sell me on his dropshipping, forex trading, crypto or tax evasion course.
Are we post honor? I have this magical machine that makes me, what ten million dollars an hour? After many years of constant service, one of my employees messes up and it goes down for a few hours. Rather then eating the loss, that isn't much of loss since my margins are as wide as the grand canyon, I crank the machine up so it runs faster and produces a inferior quality product.
Did Veruca Salt grow up and get a job at Facebook?
"That terminator is out there, it cant be bargained with, it cant be reasoned with, it doesn't feel pity or remorse or fear, and it absolutely will not stop... EVER, until you are dead!"
When you setup a facebook ad, you literally say to them: spend $100 per day.
Then Facebook has a 6 hour outage. Facebook can then show more ads to users per time on Facebook, to get to the $100 per day. Or it can only spend $75 that day.
I imagine some advertisers will prefer the first option, some the second. Facebook's machine learning algorithms will do something.
But I don't see the massively malicious intent here. Most likely this isn't even a human decision. The algorithms just know then have to spend $100 per day.
Fine, if there really are 50% less conversions, that's an issue, but frankly 6 hours won't really matter.
> Fine, if there really are 50% less conversions, that's an issue, but frankly 6 hours won't really matter.
You buried the lede with things we already know for some reason.
Two things:
- It's 6 hours and 60 million dollars to them, which is absolutely nothing, but they risked harming their customers (advertisers) over it. This was Facebook's screw-up, and now they're passing it on to their advertisers. When you view advertisers as captured, you worry less about their experience and attempt to extract as much value as possible.
- It's systemic. Someone in another thread mentioned there are good people at Facebook, and they can't reconcile the behavior exposed in internal documents with those good people. This is a great example. When your performance is tied to metrics like advertiser spend, even when you had the largest outage in your recent history, you'll immediately hop on afterwards to direct your team to try and extract the most money out of advertisers possible to make up for it. These individual incentives add up to form the predatory company that we see externally.
I think it's a bit too early to break out the outrage cannon. This is just an automated system. They're probably still putting out the fires and assessing damage. It will be some time before somebody can assess the data and figure out what "the right thing" for customers is.
If you were an ad customer during this period and you feel you weren't getting your expected value, ask for a credit?
If you weren't - why are you so upset? Are you just looking for some excuse to feel angry at Facebook?
> I think it's a bit too early to break out the outrage cannon. This is just an automated system.
Maybe? I'm not in the ads space. I don't know if there are automated systems built in with "you now only have 16 hours to display the ads, display them even when they're less relevant".
> If you were an ad customer during this period and you feel you weren't getting your expected value, ask for a credit?
Why do customers need to do this when Facebook messed up, and had to learn about this, and look into performance, from a source other than Facebook?
> If you weren't - why are you so upset? Are you just looking for some excuse to feel angry at Facebook?
If you say 60 million is absolutely nothing, then it is also clear that no ethics whatsoever will survive unsupervised in that kind of climate. No matter how good intentions initially were.
I think it's also 6 prime hours of the day too - if it was down between 12-6am I'm sure that wouldn't have been nearly a big of a problem (in the US at least)
Is it not possible the ads are being shown to more people than usual due to greater usage of Facebook properties immediately following it coming back online?
I can’t find anything that indicates individuals are seeing more ads than usual as a result of the outage, besides the anecdote of the one agency in the article (but maybe the lower conversion rates can be explained by some other factors)
Yes, but I wonder if this anonymous agency has clients all in the same geographic area or industry. The sample size is too small to draw definitive conclusions, because there might be confounding factors, or maybe the agency even made a mistake somewhere.
Of course, the type of data that would be really useful here (detailed ad performance metrics by the hour, across all of Facebook) is the type fb would never willingly publish.
This is too stupid to be true. For a 6-hour outage to significantly increase the ad frequency (and lower the conversion rate) you should pressure the system to recover the outage in a day or two; do they really need it? We're not even at the end of the month...
This sounds like straight-up fraud to me. They promised one thing, and now, after an outage that was their own fault, they are intentionally warping and devaluing the nature of the advertising product they sold to customers, so that instead of Facebook deservedly absorbing that loss, customers will absorb it, instead.
Not precisely fraud, but it is an example of the negatives of the product vendor (impressions, in this case) also owning the marketplace.
It would be rational for advertisers to pull down their spend to avoid getting hit with devalued impressions. Ironically, if enough folks did, that could cause a panic, which would make things worse for FB.
I am no fan at all of Facebook, and I would tend to regard the prospect of a dispute between it and its advertisers as a good thing, but here's a slightly-expanded version of the passage quoted in the article:
"We can also confirm that ads did not deliver during the time the systems were offline, and advertisers were not and will not be billed for ads during the outage. Ads have now resumed delivery and some advertisers may see accelerated delivery as our services recover from the outage. We encourage advertisers to review your bids and budgets to ensure they accurately reflect your marketing goals."
One way of looking at this is that Facebook is notifying its advertisers that the outage will affect how advertising is delivered in the following days, and if advertisers want something else to happen, they will have to make the changes themselves. Facebook could reasonably claim that it does not know how any given advertiser would prefer the situation to be handled.
Of course, if it is expensive or inordinately difficult to make such adjustments, then Facebook would come across as dissembling (and not for the first time.)
Reader beware. Don't take unsubstantiated claims (such as this article) as fact.
As someone who spends five figures daily on FB, I saw no such drop in performance -- nor have other advertisers I've been in touch with.
Performance on FB (especially at low volumes) is highly volatile on a day-over-day basis. So, at any time, there will always be a subset of advertisers who think "CPA's are skyrocketing". And, it's easy to draw the conclusion that the pattern is platform-wide.
The "accelerated delivery" that Facebook quoted in its response only applied to the remainder of the day that the outage took place -- not future days as the article suggests.
This. Facebook makes available a lot of configurable parameters around bidding and budgeting[1].
If you've set a campaign-level budget, Facebook's system paces delivery to spread that budget from the start date to the end date. It automatically adjusts pacing to fluctuations in delivery, this 6 hour outage is just a drastic version of that, where there was zero delivery across the board vs. more localized fluctuations in consumer behavior that usually impact pacing for individual advertisers.
So it has to accelerate delivery for the remainder of the campaign to compensate, and since it's doing that for all ads with this type of budgeting configuration it's resulting in an auction grab for available impressions and temporarily bumping up CPMs as a result.
Which isn't necessarily a positive for advertisers, but sounds like their system doing what it normally does, and just having some unanticipated second order impacts when faced with an extreme event (compensating for global lack of delivery for 25% of an entire day).
If your budgeting/bidding/optimization configuration doesn't require a specific spend within a specific timeframe, you're not going to see this sort of acceleration. But you will see higher CPMs due to the increased auction activity from others having their pacing accelerated. And they weren't collecting FB pixel data for 6 hours either, so conversion activity will also be underreported and show higher CPAs. But CPA-based optimization configurations are still in place and doing what they normally do, and frequency caps if you use them are still limiting how many times your ad will show to the same person. So even those seeing accelerated delivery and not wanting to could adjust their ads' settings to avoid it.
> The "accelerated delivery" that Facebook quoted in its response only applied to the remainder of the day that the outage took place -- not future days as the article suggests.
It does actually impact future days, just to a far lesser extent than day-of. Pacing jacked up the day of to attempt to compensate for expected pacing that day, but for those that didn't achieve it they'll continued seeing accelerated delivery beyond that to hit the spend target by end of campaign. Plus second order effects as their optimization see-saws around until things settle back down.
I think they’re just saying that just like you shouldn’t take one person’s opinion/experience (in the article) as gospel, you shouldn’t take another single person’s opinion/experience (in the comment) as gospel.
Facebook is the same company that has allowed a bunch of scam artists to advertise a $1500 ice machine for $30 (which is their S&H cost) while using another company's name in their ads. I can't remember the last time I got on facebook and saw a useful ad that wasn't either a scam or completely irrelevant.
77 comments
[ 5.8 ms ] story [ 134 ms ] threadYea, that's FBs new business model :D
I suspect that it has less to do with actual profits and more to do with some wanting to keep their quarterly ad impression numbers up
Both of these will limit the rate at which an ad spends money. If it doesn't get shown for a long period of time, both of these factors get reduced, and the ad can get selected more aggressively, particularly if you are a higher bidder than the ads that would have got chosen in your absence.
In the limited experience I have in this arena, it does improve performance. Hitting someone a second time about t+2weeks had the biggest bump, and then the next biggest another month later at t+6weeks.
I don't know if it's universally true, and the optimization may vary depending on the context and content, but pacing can definitely make a difference for some.
But attention is a limited resource. If you throw more ads at me, I'm not necessarily going to respond to more of them. The more ads there are, the more it becomes background noise.
To the individual user, they can’t tell anything different between these two types of campaigns. It just makes the advertiser feel better that their money is lasting longer.
It is legit hard to find ads which are not a scam in some way.
I know the example mentioned is facebook, but I feel like a similar development is going on with other ad networks as well. Even on YouTube around 40-50% of all ads I get are by some guy trying to sell me on his dropshipping, forex trading, crypto or tax evasion course.
Did Veruca Salt grow up and get a job at Facebook?
Then Facebook has a 6 hour outage. Facebook can then show more ads to users per time on Facebook, to get to the $100 per day. Or it can only spend $75 that day.
I imagine some advertisers will prefer the first option, some the second. Facebook's machine learning algorithms will do something.
But I don't see the massively malicious intent here. Most likely this isn't even a human decision. The algorithms just know then have to spend $100 per day.
Fine, if there really are 50% less conversions, that's an issue, but frankly 6 hours won't really matter.
You buried the lede with things we already know for some reason.
Two things:
- It's 6 hours and 60 million dollars to them, which is absolutely nothing, but they risked harming their customers (advertisers) over it. This was Facebook's screw-up, and now they're passing it on to their advertisers. When you view advertisers as captured, you worry less about their experience and attempt to extract as much value as possible.
- It's systemic. Someone in another thread mentioned there are good people at Facebook, and they can't reconcile the behavior exposed in internal documents with those good people. This is a great example. When your performance is tied to metrics like advertiser spend, even when you had the largest outage in your recent history, you'll immediately hop on afterwards to direct your team to try and extract the most money out of advertisers possible to make up for it. These individual incentives add up to form the predatory company that we see externally.
If you were an ad customer during this period and you feel you weren't getting your expected value, ask for a credit?
If you weren't - why are you so upset? Are you just looking for some excuse to feel angry at Facebook?
Maybe? I'm not in the ads space. I don't know if there are automated systems built in with "you now only have 16 hours to display the ads, display them even when they're less relevant".
> If you were an ad customer during this period and you feel you weren't getting your expected value, ask for a credit?
Why do customers need to do this when Facebook messed up, and had to learn about this, and look into performance, from a source other than Facebook?
> If you weren't - why are you so upset? Are you just looking for some excuse to feel angry at Facebook?
Y u mad bro? hasn't worked in quite a while.
Have you bought anything from anyone ever...?
That said, it seems wrong to use accelerated delivery for anybody using the Facebook pixel for conversion optimization.
I can’t find anything that indicates individuals are seeing more ads than usual as a result of the outage, besides the anecdote of the one agency in the article (but maybe the lower conversion rates can be explained by some other factors)
Of course, the type of data that would be really useful here (detailed ad performance metrics by the hour, across all of Facebook) is the type fb would never willingly publish.
But Facebook is actually trying to help its costumers recover the money lost during the outage.
US$ 60 million? Pff. Just ask every executive to skip one executive jet travel per week and they recover that in a month or less.
It would be rational for advertisers to pull down their spend to avoid getting hit with devalued impressions. Ironically, if enough folks did, that could cause a panic, which would make things worse for FB.
"We can also confirm that ads did not deliver during the time the systems were offline, and advertisers were not and will not be billed for ads during the outage. Ads have now resumed delivery and some advertisers may see accelerated delivery as our services recover from the outage. We encourage advertisers to review your bids and budgets to ensure they accurately reflect your marketing goals."
One way of looking at this is that Facebook is notifying its advertisers that the outage will affect how advertising is delivered in the following days, and if advertisers want something else to happen, they will have to make the changes themselves. Facebook could reasonably claim that it does not know how any given advertiser would prefer the situation to be handled.
Of course, if it is expensive or inordinately difficult to make such adjustments, then Facebook would come across as dissembling (and not for the first time.)
As someone who spends five figures daily on FB, I saw no such drop in performance -- nor have other advertisers I've been in touch with.
Performance on FB (especially at low volumes) is highly volatile on a day-over-day basis. So, at any time, there will always be a subset of advertisers who think "CPA's are skyrocketing". And, it's easy to draw the conclusion that the pattern is platform-wide.
The "accelerated delivery" that Facebook quoted in its response only applied to the remainder of the day that the outage took place -- not future days as the article suggests.
Do you actually need to spend that much..?
Delivery for the day dropped, but performance as measured through CPA (or ROAS) remained stable.
> Do you actually need to spend that much..?
Ultimately, we're trying to maximize profit. So, we advertise until the cost per incremental conversion is greater than a customer's ltv.
But how can your engagement from clients be "the same", despite half the availability?
Does this imply that all else perfect andavailable,every day, you could have a volatile swing of up to 50 percent engagement?
If so, that seems really really silly to drop 10k+ on daily, if on half of all days you don't NEED the platform for half the day...what am I missing?
Essentially, it's 'engagement per dollar', not 'total engagement'
CPA should be affected, seems like FB is giving an artificial measurement. Reminds me of the Big Short.
If you've set a campaign-level budget, Facebook's system paces delivery to spread that budget from the start date to the end date. It automatically adjusts pacing to fluctuations in delivery, this 6 hour outage is just a drastic version of that, where there was zero delivery across the board vs. more localized fluctuations in consumer behavior that usually impact pacing for individual advertisers.
So it has to accelerate delivery for the remainder of the campaign to compensate, and since it's doing that for all ads with this type of budgeting configuration it's resulting in an auction grab for available impressions and temporarily bumping up CPMs as a result.
Which isn't necessarily a positive for advertisers, but sounds like their system doing what it normally does, and just having some unanticipated second order impacts when faced with an extreme event (compensating for global lack of delivery for 25% of an entire day).
If your budgeting/bidding/optimization configuration doesn't require a specific spend within a specific timeframe, you're not going to see this sort of acceleration. But you will see higher CPMs due to the increased auction activity from others having their pacing accelerated. And they weren't collecting FB pixel data for 6 hours either, so conversion activity will also be underreported and show higher CPAs. But CPA-based optimization configurations are still in place and doing what they normally do, and frequency caps if you use them are still limiting how many times your ad will show to the same person. So even those seeing accelerated delivery and not wanting to could adjust their ads' settings to avoid it.
> The "accelerated delivery" that Facebook quoted in its response only applied to the remainder of the day that the outage took place -- not future days as the article suggests.
It does actually impact future days, just to a far lesser extent than day-of. Pacing jacked up the day of to attempt to compensate for expected pacing that day, but for those that didn't achieve it they'll continued seeing accelerated delivery beyond that to hit the spend target by end of campaign. Plus second order effects as their optimization see-saws around until things settle back down.
[1] https://www.facebook.com/business/help/1619591734742116