Ask HN: How does Facebook know what I buy on Amazon or vice versa?

178 points by kmonad ↗ HN
Following just happened: An hour before lunch I googled and visited websites that sell bicycles. I also visited Amazon during this research. I then bought a bike from one of the manufacturers' websites. A few hours later I browse facebook and see ads to this manufacturers' bikes in my newsfeed, via an Amazon sponsored ad.

I use one browser (Safari) for facebook exclusively, and browsed the bikes / Amazon / made the purchase on Chrome. I have different email addresses for facebook, amazon and, well, google.

130 comments

[ 3.0 ms ] story [ 105 ms ] thread
Some data matching isn't done with straight cookies. You visited from the same computer, same IP. It may be a guess done by comparing IP + installed fonts.

The other possibility is that your multiple email addresses have been matched as the same person. So even though you use different browsers and have different cookies, they're collapsed on Facebook's side.

These companies can also track you by IP address. I've had conversations with co-workers about them considering a specific large purchases, and then seen ads for what we talked about popping up. I'm assuming that if the CPM is low enough, many ad-retargeters will take the risk of targeting ads based on IP address alone.
Three words......Facebook Tracking Pixel

You go to Amazon, they tell Facebook what you are looking at, and then can dynamically create ads specifically for you.

Does Amazon really use the fb tracking pixel? I wouldn't think so
Op said he visited a website that sold bicycles. If this website has a "Like" button or such a tracking pixel or Facebook analytics (do they have that?), Facebook would then know that OP is interested in this bike.

An effective ad network would know that OP already bought such a bike, so it's actually useless to try to sell him bikes... maybe accessories.

Looks like they don't! Must be a different retargeting technique used by Amazon. FB Pixel could have also been on manufacturer's site.
Straightforward:

1) Amazon tracked your research and likes to use " ad retargeting" if they see you leave before checking out to remind you to come back and purchase.

3) FB offers advertisers a variety of retargeting means + insanely advanced cross-device tracking. Like a building full of PhD's advanced. Maintaining privacy by logging in from different devices / accounts / etc is a thing of the past if you EVER cross-pollute between browsers / devices / etc the signal is picked up and then compared with browsing behavior etc to get a strong profile. (1)

(1) https://www.facebook.com/business/a/performance-marketing-st...

But... what about step 2?

> Like a building full of PhD's advanced.

On a more serious note, it's unfortunate that so many smart people use their intelligence to enable this kind of gross technology. It's all about "solving puzzles" rather than making the world a better place. There's gotta be more productive ways to advance your career.

Whatever made you think that tech industry is a beacon symbolizing all the good things about humanity!
(comment deleted)
> It's all about "solving puzzles" rather than making the world a better place.

I'd figure it's more about getting paid. Making the world better usually doesn't make anyone money, or at least, costs too much before it starts making anyone money.

Making the world truly better requires us to give up a lot of what gets ingrained into us since childhood. We've made money an ends instead of a means. I'm not making a political statement – this is true of every human society, culture and country today. We should collectively rethink what it means to be a successful human and have a fulfilling life (without abandoning technology and progress.)

What's the end goal of civilization? Are we to just multiply and expand, ad infinitum?

>> It's all about "solving puzzles" rather than making the world a better place.

>I'd figure it's more about getting paid. Making the world better usually doesn't make anyone money, or at least, costs too much before it starts making anyone money.

This. I get paid (a lot) for handling marketing campaigns. I also started a health project this year (would love to work on it fulltime on a 10% payment vs what I make now), but that gets ignored.

The banality of evil. So many engineers just want to make the trains run on time with a better algorithm. Never mind that they cary oil or weapons or the victims of our next genocide. Engrossing technical problems are convenient blinders for inconvenient truths.
But maybe for all of us there is something about what the Germans did that pleases and excites us-something that opens the catacombs of the imagination. Maybe part of our dread and horror comes from a secret knowledge that under the right-or wrong-set of circumstances, we ourselves would be willing to build such places and staff them. Black serendipity. Maybe we know that under the right set of circumstances the things that live in the catacombs would be glad to crawl out And what do you think they would look like? Like mad fuehrers with forelocks and shoe-polish moustaches, heiling all over the place? Like red devils, or demons, or the dragon that floats on its stinking reptile wings?' 'I don't know,' Richler said. 'I think most of them would look like ordinary accountants,' Weiskopf said. 'Little mind-men with graphs and flow-charts and electronic calculators, all ready to start maximizing the kill ratios so that next time we could perhaps kill twenty or thirty millions instead of only seven or eight or twelve.

Excerpt From: King, Stephen. “Different Seasons.”

You can't tell in advance what trains, in general, will carry.

When writing ad targeting code, user surveillance, nuclear missile guidance, VW pollution test circumvention code... you know exactly what is going on.

Not necessarily. I don't know much details about other technologies you have mentioned, but user surveillance has many overlapping technologies with IT security: deep packet inspection, network flows monitoring, white/black listing etc. can be used for both good and evil and switched in-between quite easily. I'm very pro-privacy and sometimes get to work with such technologies, though I emphasize that what I do is strictly for security purpose only.
> deep packet inspection, network flows monitoring, white/black listing

This is usual network security.

At some point someone need to glue together existing devices and software into something that has an obvious use case with ethical implications e.g. the chinese "great firewall".

"Engrossing technical problems are convenient blinders for inconvenient truths."

Where does this take us though? If you build a bridge, how do you know what will go across it? Either we all stop or we accept that not all ethical questions can be resolved by individuals acting alone.

I suppose it depends on whether you think that Facebook is a worthwhile product that should be commercially viable, rather than closing. If you do think that, you are making the the world a better place for some values of better and world.
> it's unfortunate that so many smart people use their intelligence to enable this kind of gross technology

We live in a capitalist society.

It just so happens that this "gross technology" makes advertising insanely profitable for millions of companies (mine included), which makes FB billions, so they'll happily pay really smart people millions to build it. And those smart people will happily accept millions to solve those hard problems.

It's just business. If you can figure out how to better align individual and corporate incentives with making the world a better place, we're all ears.

It's unfortunate, not surprising.
>We live in a capitalist society.

Always a solid cop-out for allowing bad things to continue

I completely agree. I pity those PhD's using their Intellect for increasing Ad revenues to FB rather than creating/using algos to solve the pressing real-world problems - Diseases, Nutrition,Drought predictions,Curb terrorism etc. Are they even motivated by what they are doing..after all money is not everything (I agree FB pays a bomb).
I can understand the feeling, but it might seem wasted but Facebook and Google have actually made it easier for small business to do business by enabling them to get segmented customers more easily by breaking down barriers for entry. 20 years ago one could not as a small company get US or global reach without using a lot of money and it was not easy (possible) to segment either. This again have increased competition and enabled new products an lower prices. The technologies pioneered by Google and Facebook (BigData, NoSql, ML) have again improved other disciplines and fields.
While true, I feel like they shouldn't break down all the barriers. They were helping small business do business before re-targeting. Just giving us an affordable way to get in front of interested prospects was huge.
>> It's all about "solving puzzles" rather than making the world a better place.

How many people do go into computer engineering with an idea of making the world a better place?

Don't misunderstand me, I'm sure many computer engineers are nice, socially conscious people, but if your primary motivation in life is to make the world a better place through your work, there are many more direct ways to achieve this.

It's telling that the Gates Foundation, which has enormous resources, and which could build, should it so wish, the world's best-funded 'change the world with code' startup, instead chooses to allocate its resources on physical things such as malaria drugs.

I love picturing a room full of PhDs developing a crazy complicated algorithmic wonderland. Their celebrations at the complexity and genius of their work probably cut short, when users are targeted with ads telling them to buy something that they just bought.

When human male user "Joe" purchases a women's bath robe 1 week before Mother's day, does he really need ads for women's bath robes for the next month? Of course I know this is nothing to do with the fault of the room full of PhDs.

This has always bothered me too - guess they focused more on tracking users and building a strong graph rather than actually making good recommendations.

Perhaps the coming A.I.pocalypse™ will give us better predictive suggestion algorithms before turning on us... Or the sentient networks just start recommending really dangerous products to wipe ourselves out with! ;(

Chatting with friends, I've discovered FB are also really, really good at showing me adverts for what other people are buying my chums for Xmas, which is of limited use frankly.

I don't know how much does it have to do with advertisers choosing profiling criteria and profiling itself, but often it feels like ad targeting is actively trying to teach me to ignore the ads.

Not really technical friend gets a smarthpone. We check specs on gsmarena, then check few other phones, read a bit more. Lots of hits over short time. Ad targeting: you must be searching for a smartphone, here get some ads for smartphones! I check various pages over few weeks, compare phones, check availability. Ad targeting: very rare hits, must be accidental.

For the past few weeks I have been selecting a pair of products. Watching youtube, reading various forums, browsing manufacturer pages, browsing stores, comparing prices, usual stuff. A single ad for at least one of those two product categories? Nope, instead I see ads to subscribe to my current carrier.

but is he 0.5% more likely to need a bath robe than [random internet user]?
But sort of silly to be advertising you for something you've already bought? They think maybe you liked the bike so much you'll buy another, but only if you see an ad?
tracking you as you browse around is relatively easy, which is what I believe these particular scripts are doing.

To actually track checkouts/purchases correctly so that you don't advertise to those people requires a completely different setup.

I'm pretty sure I read here on Hacker News that one idea to advertise the exact same product that you just bought (I had that with a basketball I got for my daughter) is that they want you to feel good about the purchase. If you see it in ads after you bought it, you'll start thinking it's a popular product and will feel that your choice to buy is being validated.

And they of course get to repeat the name of the store for you so that you're more likely to think of them the next time you need a related product, even if it's not a basketball. You'll remember that you've seen the name of store X in multiple places, so they must be popular.

I have had similar experiences. Others have already mentioned some cross-browser fingerprintig techniques. One of the worst that many people don't know about is that browsers hand over your local IP. Check this proof of concept:

http://net.ipcalf.com/

The media device IDs the browsers provide look even worse:

https://jsfiddle.net/u4n4s296/

I am not sure if these are unique to the device type (for example a certain soundcard model) or to the device itself. If it's the latter, then that is an indestructible cross-browser cookie right there. EDIT: As per icebraining's comment, in Firefox they are not not cross-domain, not cross-browser and get randomized when you delete your cookies.

Whoa, for a moment I felt pretty sure you would be mistaken. Hadn't read about this particular vulnerability before.
WebRTC leaks have been known, and the WebRTC or Firefox people said they'd disable peer to peer data channels off if this technique became used for tracking. Because a permission prompt was too much. WebRTC folks couldn't even come up with a real use case for silent data channels.
Love the comment. "In Chrome and Firefox your IP should display automatically, by the power of WebRTCskull."
Haha, wow. What is wrong with people nowadays, that they think this is reasonable behavior...
Interestingly, this doesn't seem to work for me if I have uBlock Origin enabled. However, nothing is logged as blocked in uBO, so I can't tell what it's doing to break it.
Same with Privacy Badger. I think it's the "Prevent WebRTC from leaking local IP address" option in the General Settings, except that mine is unset, yet they say the "Default options" protect me so I'm not sure if it's partially set by default anyway.
Ah, it definitely is that option, in both uBO and Privacy Badger. The latter doesn't actually seem to stop the demo working, though. I vaguely remember reading something in uBO's documentation about only one extension (the first one to ask for it) getting access to some filtering feature, although I can't find the page now.
In Firefox, the media device IDs are different for each site (origin): It is un-guessable by other applications and unique to the origin of the calling application. It is reset when the user clears cookies (for Private Browsing, a different identifier is used that is not persisted across sessions).

https://developer.mozilla.org/en-US/docs/Web/API/MediaDevice...

It looks like chrome is doing something similar. Each profile has a different value and each site has a different value, but the value is stable across restarts.

Safari (tech preview or high sierra) seems to give a new value every time you reload the page.

Just tried this on iOS Safari (running iOS beta, haven’t checked earlier version). Those media ids change on every page load.
They use the Facebook Pixel https://www.facebook.com/business/a/facebook-pixel

They segment users that visited each product on Facebook with a custom audience and then create ads for similar products that they show you. This is all done programmatically.

How would that work across different browsers?
If you're logged into facebook on the browser the pixel fires on then that event is associated with your facebook account and is used in further targeting.
I am working at FB but not in ads or related team. But I had talked to a person who is directly working on it.

Amazon is different to other AD buyers. Amazon does not want FB to know what customers are doing on its own site, so there is no FB tracker on Amazon at all. However, Amazon can choose what ad to deliver to you on FB backed up by its own team.

OP says they have different email addresses for Amazon and FB. How would Amazon know which user to target ads on FB in this case?
Cookie is a thing.
It's not cookie, it happens across different browsers. It's IP based.
How could IP-based possibly work for people at places like college campuses?
They take the number of people from the same IP into account. IPs are broken down into public IPs vs private IPs based on traffic/timing of usage etc. There are research papers on this sort of feature contruction using only IPs. Cross device especially uses it extensively to be able to probabilistically ascertain if the person from your house who is checking their phone is the same person who checked smth on the Desktop computer last night based on your online timing, IPs, behaviour over the day. They can figure out, for instance, your office vs home browsing timing, interests etc with the same methods.
Interesting. Do you have the links of these papers so that I can read more?
It's probably not cookie-based, or not only cookie-based. Browser fingerprinting and user profiling have gotten very sophisticated, and don't think the big 4.5 aren't using these techniques extensively.
It's probably still cookie based.

If one logged into Amazon in both browsers then it is really just linking to the account (even after logging out).

OP also said they used two completely different browsers
From the same network ? If both browsers are from different computers (worse still from same computer) from same network then the externally visible ip address is the same isn't it ?
Amazon customers connect their Facebook friends to find people with wishlists. Someone who has the contact could have information that could link the two accounts, for instance if the OP has someone in their friends' wishlists and they have someone with OP's name in theirs. Or could be connected to a subsidiary like IMDb.
Interesting indeed. It is therefore more likely that I should have asked "How does Amazon know who I am on Facebook?". I suspected this could be the directionality (weak attempt 'vice versa'). Thanks for the special insight!
Another reason to install ublock and add the social tracking filters.
That would not help. Amazon knows what you looked at because you did it on their site.

It's not Facebook tracking you, it's Amazon.

It's also not Facebook giving you the ad, it's Amazon. So technically not a privacy violation.

If you stay logged in on FaceBook and browse & search other sites they know. Only login when you need it the service, then logout and delete cookies.
That doesn't make any difference. It tracks across browsers that are not connected in any way except IP.
This is all fairly straight forward however, what blows my mind is Facebook makes recommendations of friends that I've NEVER had any digital interaction with.

For some reason it picked up the kid who stocks the craft beer at my local family run grocer. Literally only talked to the kid face-to-face. No phone number, no texting, no contact entry (not that I share those with Facebook anyways).

That made the hair on the back of my neck stick up when that happened.

Maybe he searched and looked you up on Facebook and it's somehow enough signal to suggest him?
Wouldn't it violate their promise that users cannot know who looked up their profile?
There is some process that is running a spatial join. You were both logged into facebook at the same time, and were in close proximity.

I had a similar situtation where Facebook recommended that I friend a coworker that sat next to me.

Facebook claimed a year ago that they did not use location data. https://www.theguardian.com/technology/2016/jun/29/how-does-...
"location data" and "factors that are location dependent" are not the same thing. e.g. SSID + MAC address of a wifi access point that two people used at the same time.
Interesting.

The situation I described happened over a year ago, so I imagine they were using location data then.

I suspect FB uses location and/or WiFi data as one of entry points. Note that smartphones often scan WiFi networks even when WiFi is off.

Based on the high intersection of two sets of SSIDs names visible from two devices at the same time, you can decide two people were in one room together. If this happens regularly, you can be quite confident people "know" each other in some way, which can unveil many surprising "friends" recommendations.

Have you ever called the grocer's phone number? Maybe it's linked to his account.
It's by IP address.

I know this because you can browse for something on Amazon on one machine, then find ads for that item on an entirely different machine - but one that's using the same WiFi.

Good luck buying a surprise present for a Significant Other. If you try, they'll see ads for it on Facebook.

If you want to know what advertisers are retargeting you, you might want to check: http://whoisretargeting.me It can be enlightening. You can also opt out of a lot of it here: https://www.facebook.com/help/568137493302217
I don't seem to see too much specific to me, other than anything you can gleam from geolocation... Suspect NoScript/uBo/Self-destructing Cookies have mitigated a lot of it.
Does not work under Brave - which means I'm protected from retargeting, too?
Interesting. I whitelisted this site, and 2 of the 4 ads it displays on each refresh are from Facebook.

I've never had an FB account and they seem to know it.

Yet, FB doesn't know that you bought it. Just knows that you did the research. It would show you the ads regardless of if you bought it or not.
All of those actions (searching, viewing, purchasing) sent signals to various tracking companies that all exchange data either directly or indirectly through third parties. While Amazon may not directly work with Facebook and exchange tracking data, Facebook may work with another third party that works with Amazon.
DMPs.

Amazon buys (and sells) data to/from DMPs. That data can (and often does) include a hash of your credit cards, all the e-mail addresses you go by, etc. Amazon can basically buy programmable ad inventory that says "I want to show this ad for chainsaws to kmonad" and the DMP resolves who 'kmonad' is through a variety of methods.

Realistically, the opsec you would need to have to avoid this would be astronomically inconvenient. These DMPs work off statistics, so they don't need to know 100% that this browser session is probably kmonad, just 70%. Maybe you have the same IP, OS version, browser extensions, cookie sets...

This is most likely. To take it one level further, since you bought something, it can be DLX data (or Oracle now) or some other purchase-based data (from Visa or Nielsen Catalina). Facebook can ingest these as custom segments to target. For instance, I can buy a data segment of past purchasers of Giant bicycles for $1 CPM that will be layered on whatever partner is integrated. With every "match" there will be drop off as one ID system needs to be matched with a separate ID system (i.e. FB <-> DLX, or Liveramp <-> Mobile App)
Right; I think it would really creep people out to know what data is available to target for advertising purposes and how deeply it goes.

Basically, there are a bunch of methods for tying your session back to your identity, and most DMPs will run through a good dozen or two. Most will fall back on geolocation patterns (which are surprisingly accurate themselves) but it's actually very hard to totally anonymize your internet activity. We are creatures of habit, but our patterns betray us. :)

But even with all that data, they still show irrelevant apps? In this case the user already bought the item and gets shown ads for it afterwards. That's very likely ineffective marketing. Is data mining this complex really worth it if the result is that bad?
I would bet that Amazon does this intentionally. I don't know their rationale, but they're a data-driven company so they wouldn't keep doing it if it didn't produce desirable results.
FWIW, Home Depot does this too.
It's called the internet. That's what it does. Do you ask why bacon is delicious?
It's just Amazon cookies. In short, you didn't see the ad because of what Facebook know about you (though they'll know a great deal more than Amazon), just that the two cut a deal to allow Amazon to read Amazon cookies from Amazon ads on Facebook.
amazon serves their ads on facebook with scripts hosted on their domain, and the browser willinging gives amazon it's own cookie back, even if you're on fb and not amazon.
It's fascinating to see to what lengths advertisers go to identify users only to serve irrelevant ads. The bike ad after a purchase is completely useless and would've better been served to any other user.
> It's fascinating to see to what lengths advertisers go to identify users only to serve irrelevant ads. The bike ad after a purchase is completely useless and would've better been served to any other user.

On the contrary, I'd wager that a person who's purchased a bike is far more likely to buy another than a random other person. I'm not saying there's isn't someone more likely to buy one than the guy who just did. But he would be more likely than someone who has no recent bike related purchases or interest.

It could be as simple as, "Hey this is a nice bike I bought. I bet my wife/sister/brother/friend would like one too. Hey Bike Co is having a sale!".

Would the same thing have happened if you were using the Brave browser (supposed to protect your privacy) - https://brave.com/?
No. Using Brave they have no way to follow you (no cookies).