Ask HN: Ads triggered by WhatsApp “end to end encrypted” messages?
Hi all. We've been playing a silly little game with my wife lately: we send each other messages about some topic we never talk about and then wait for ads related to our conversation to start showing up in Instagram. As of the last month, they never fail to show up.
Please keep in mind that this is a conversation between two "personal" accounts, no business accounts involved. More so, we haven't accepted the new terms of use that "allowed" WhatsApp to access messages between personal accounts and business accounts.
Is WhatsApp scanning personal messages to target their ads as we are noticing? Weren't WhatsApp messages end to end encrypted? Is this a violation of their Terms of Use or am I missing something silly?
472 comments
[ 3.1 ms ] story [ 315 ms ] threadGoogle are definitely collecting data from gboard.
That may not be directly shared with meta but is likely to get indirectly shared through overlapping advertising identifiers. They won't be openly sharing your text, but they will be scanning to and flagging you as having interests in something in your text or something related to what you said, then sharing that with advertisers.
I’ve never been able to reproduce these experiments. But keep in mind that I’m european and my WhatsApp app is slightly different - it is a version from WhatsApp Ireland (instead of WhatsApp Inc) which shares less data with third parties, the privacy policy is also slightly different for the european union.
Edit. Another idea: try to reproduce while disabling predictive text on your keyboard.
Even Swiftkey keyboard (bought by microsoft) sends back telemetry to MSFT. Try a keyboard from Fdroid, but it may not be as feature rich.
e2e encryption doesn't forbid to read the messages as you type or read them or read a screenshot of the screen or whatever they can do inside an app :P
They were caught activating your camera by "error" a while ago https://www.macrumors.com/2019/11/12/facebook-bug-camera-bac...
As per the experiment you did...
We did the same experiment with a female friend a while ago. We started talking about her pregnancy (a topic we never touched, as she was single and of course not pregnant) in a group chat, specifically targeting her. Sure enough, after a couple of days her fb and instagram were full of strolley ads (but not ours) :)
yup, do you think all these img recognition stuff is made for fun? companies wants to use them to read our pics on your phones and profile us
There's no reason you'd have noticed an ad about the puzzle in the wash of content and other types of ads.
Then randomly select one of those three topics to discuss on WhatsApp.
Finally, keep an objective eye out for ads about all three topics. Ideally, log every single ad you see.
Meta has control over the app Sue uses. So they could send them to Meta unencrypted in addition to sending them to Joe in an encrypted fashion.
Or they just extract the relevant terms:
Sue->Joe: "Hello Joe, I'm so excited! We are going to have a baby! Let's call it Dingbert. You're not the father! Jim is. I hope you don't mind too much!".
Sue->Meta: "Sue will have a baby"
Insta->Sue: "Check out these cute baby clothes!"
The other day I noticed the yahoo mail app on iOS was reading my clipboard for no reason. I’m going to start blocking photos on most of my apps.
Under Privacy > Photos, you can set “Selected Photos” instead of “All Photos” on a per-app basis.
Then when you go to add a photo to the app, you first go through an iOS prompt to select the photos the app will have access to. Only then do you go through the app’s photo selection dialogue.
I have all my apps set this way (or “None”).
Not saying it doesn’t work like you say, just saying it doesn’t look like it does.
I assume Instagram and friends would do the same.
I often just take the photo via Telegram instead, which automatically adds it to your photo roll and gives Telegram access to it. It works relatively well.
Then on Instagram (for example) when you go to post, you’ll get a message like “you’ve only let Instagram have partial access to your photos - Manage”. Tapping Manage will let you select photos that Instagram can access.
I see you’ve never heard of Jane Manchun Wong...
You can reverse engineer those things and analyze your network traffic. You can’t have a client in a device controlled by the user, in this case an app, send anything to a server without anyone noticing it.
And frankly, they don’t even need it. Just with your contacts they can link you to your friends and common interests without even you having a facebook account, all you need is friends with a fb/ig account who have linked their accounts to their phones and use whatsapp.
The contacts are known to be sent to the server, they are known to be linked to facebook except in the european union where there is a different app from WhatsApp Ireland and a different privacy policy that specifically states (in the version outside of EU) that it shares your contacts with facebook and they are much more valuable and much less risky than reading your messages.
I frankly don't think people realize how much obfuscation of both app code and network traffic goes on under the hood. "analyzing network traffic" isn't a sustainable option when things are encrypted and behind dozens of layers of protobuf, websockets and other fancy protocols, and get updated and change around all the time. Far from everything is introspectable http, javascript and json these days, and that applies espeically to big apps like these. It's not hard to send privacy-sensitive data along with "legitimate" data like analytics at unexpected times and evade scrutiny.
Yes there's people that dedicate themselves to reverse engineering apps like this, but they're few and far between, and most of them focus on either the easy fish, or security vulns. Considering nobody's building public documentation on the protocols of these apps I'll have to assume it's hard enough and changes often enough to be worth the time of people without special monetary interests.
I agree with the rest of your assessment, there's way less "obviously malicious" ways to exfiltrate data about users than literally uploading users' pictures, since for example whatsapp stored unencrypted backups on google drive until very recently, among other things. I'm just trying to shed a light on the fact that apps like this have a lot of ways to accomplish this without raising too many eyebrows.
Turns out they were also building a database of everyone's face so they could build shadow profiles...
> her Instagram was showing ads for a store that was selling the same type of puzzle
How did she take the pic ?
I've always suspected them of recording conversations, also why I think Android has gradually tightened permissions and visibilty around speech to text/microphone/camera use.
Looking at this from a reality perspective is not very helpful.
Or they can say, technically it wasn't a message before it was sent. The dictionary definition[1] even mentions "send".
[1] https://www.oxfordlearnersdictionaries.com/definition/englis...
I think it does actually no one except them can read them. If someone else can, then by definition it's not end-to-end encryption.
From https://www.definitions.net/definition/End-To-End%20Encrypti...
> End-to-end encryption (E2EE) is a system of communication where only the communicating users can read the messages.
E2ee means only the messages themselves can't be intercepted and read. But if anyone can actually prove fb acting on message contents, I suspect the EU banhammer would be interested.
But if the message is copied, read, analyzed and sent further on behalf of a third party before encryption, then that puts that third party in the middle between the sender and the recipient. A man in the middle directly undermines e2ee: "no one else reads your message".
It doesn't matter if the third party made the messaging app or not. What matters is whether information in your messages is accessible to anyone besides you and the recipient.
This said, analyzing messages for the purpose of ad display is creepy, whatever the way it is done.
Notice that "ends" in "end-to-end" are users, not applications. When an application forwards things to an entity, then that entity becomes an "end" of the conversation. When it displays a message to the user, the way the user wants, then the user is the end. When it processes the message and delivers results to Facebook, the way Facebook wants it, then the application makes Facebook the "third end".
In such scenario, Facebook had intercepted the message, just chose to forward only some extracted information (which may or may not be enough to reconstruct the original). This does not match the definition of "end-to-end encryption".
That's not right. First, it's technically an impossible, since users can't do encryption themselves - it's the application that does it. That's where the e2ee boundary is.
Second, we've got e2ee communication between non-user entities as well. There's are servers using for example zerotier which communicate e2ee through other nodes. Third, applications can definitely send the data to other parties automatically. WhatsApp executing backups as configured does not make it not e2ee.
We know the app decrypts it to display it. But if the app decrypts it to send it to the parent company, then it is by definition not end to end encrypted anymore.
If the app decrypts it, analyzes it and sends information about the message to the parent company, then the same thing is happening. The parent company is reading the message, INSTEAD of E2E encrypting it. It doesn't matter whether that reading happens on device or on the company's servers. E2E means the company is not reading it.
It’s possible that this data harvesting ad company has redefined what E2E means (to them) to advance their business interests.
HTTPS is E2E between the client and the server.
No, that's precisely what End-to-End encryption means.
Signal might be the only app unable to read, but even that, I would not trust.
It's illustrated in their example below that they if you say you're having a baby, meta can send some type of distilled ad-keywords to its servers (eg `[mother, baby]` if it knows the user is a woman based on their name/profile, but probably more sophisticated than that). The message you sent is still technically end-to-end encrypted, though,
https://news.ycombinator.com/item?id=32951417
Facebook loves to use newspeak, wouldn't surprise me if they applied newspeak to what "end-to-end encryption" means.
The quandary of what one allows to run on those implants sounds like a chilling sci-fi novel (chilling not because "but FAANG could read your thoughts!" but because people would absolutely still get them installed).
https://en.wikipedia.org/wiki/End-to-end_encryption#Endpoint...
so what's the point? just inconvenience. better to use telegram at this point.
Not that I trust WhatsApp (I use Signal) but that's an odd comparison.
First of all, transport security (server-client encryption as you called it) like TLS is irrelevant for this discussion. All major platforms on the internet employ transport security these days, so this is a given.
The point I'm trying to make is that Telegram does not offer E2E encryption by default: (Non-"secret") Messages on Telegram pass through Telegram's servers unencrypted and are also stored there unencrypted, meaning that Telegram has access to all your messages. This is not speculation – Telegram openly admits to this in their FAQ:
https://telegram.org/faq?setln=ru#q-do-i-need-to-trust-teleg...
(See also the link contained therein.)
Meanwhile, the speculations in the present HN discussions aside, WhatsApp does provide E2E encryption, so – from this POV – is orders of magnitude more secure.
> We limit the information we share with Meta in important ways. For example, we will always protect your personal conversations with end-to-end encryption, so that neither WhatsApp nor Meta can see these private messages.
They are saying they dont store or forward your message text, not that your phone doesnt send them topics of interest
Assuming they're not blatantly violating the policy (which I think they've done before), it's pretty easy to weasel out of that statement by only sharing keywords from the conversation, or only sharing the info with advertisers (but not WhatsApp and Meta), or redefining what a "personal conversation" is, or carefully redefining what "end-to-end encryption" means, or ...
There's no transparency, a huge power imbalance, and terrific pressure on WhatsApp/Meta to monetize as much as possible.
I don't think there is any fault with the e2e encryption. Humans are very bad at seeing causality when there is none, or accidentally leaking their thoughts into the search box.
There could also be leaks with the clipboard, photo gallery or keyboard - all things that freemium apps love to scan in the background. The way real-time-bidding ad markets work, anyone that the data leaks to can influence ad ranking - doesn't have to be FB/Meta.
If you did a true blind study, I think you'd find no link.
For example, start with a list of 1000 products images and accompanying text. Select 2 at random each day. Flip a coin to decide which to send (keep the other as control). Cover the screen so the user can't see what they've sent/received. Then, a few days later ask the user to select which product they think they sent.
I'd bet that even after months of doing this, there will be no finding of a leak.
1. Nobody is reading your WA messages, the same topics can be learned from your browsing activity or other msgs, eg. by reading your sms texts.
2. Meta is reading your messages directly in-transit, server-side.
3. Meta is not reading your messages server-side, but the Meta apps extract keywords from your conversations and request relevant ads from the ad servers.
4. Another non-Meta app is doing the above.
5...
Tell that to Amazon who never fail to recommend me things I just bought.
The thesis is that recent vacuum cleaner purchasers are many times more likely (than the average person) to be looking to buy a vacuum cleaner.
Apparently about 20% of Amazon purchases are returned. And most returners are looking for a replacement. Some of the replacement product research is done before the return decision is made, so you get ads even if you have not initiated a return.
As much as Amazon doesn't want you to return your purchase, they really don't want you to buy the replacement somewhere else.
It would be interesting to measure how the ad ratio changes over time. Particularly when you exit the return window, but of course Amazon will know the return-likelihood curve with much greater precision.
I think that 1 is the most plausible, however the original post is about "topics they never talk about", so assuming that WhatsApp is the only channel and they don't leak data in other ways (and there are many other ways to leak data), then 1 becomes unlikely.
3 is the most compatible. All the targeting can be done locally, so no end-to-end unencrypted message leaves the app. The app then sends your topics of interests to Meta.
4 again assuming WhatsApp is the only channel, then there is probably some malware somewhere, and it is unlikely that Meta accepts illegally collected data (they can do it legally, better, and with less potential trouble). There are however a few legitimate apps that can do the above. I am thinking about things like predictive keyboards, accessibility apps (screen readers, ...), backup apps (end-to-end encryption is about transmission, not storage), and the OS itself. I don't think Meta controls any of these, and I don't think they would buy data from them (Google and Apple are competitors after all).
So I would go for an accidental leak (case 1). For example, for the experiment to be meaningful, you shouldn't tell anyone about the test topic before you receive the ads. Or with the WhatsApp app hinting Meta about your topics of interest.
I agree, I'm pretty sure 2. is not the case; I just listed it as a theoretical possibility. Despite all the bad press and problems, FB has very (very) high integrity and standards, at least the parts I saw.
WhatsApp's end-to-end encryption is used when you chat with another person using WhatsApp Messenger. End-to-end encryption ensures only you and the person you're communicating with can read or listen to what is sent, and nobody in between, not even WhatsApp. This is because with end-to-end encryption, your messages are secured with a lock, and only the recipient and you have the special key needed to unlock and read them. All of this happens automatically: no need to turn on any special settings to secure your messages.
https://faq.whatsapp.com/791574747982248
I guess the key lies in "what is sent" in the above statement. The casual reader might reasonably interpret as "no-one except the intended recipient can see _what I type_". But it doesn't say that. It only covers what gets _sent_. It doesn't say anything about what happens to the content outside specifically _sending_ it to the other party(ies).
The more obvious explanation is that before or after the e2ee (i.e within the app itself), an algorithm scans the content, categorizes it and sends this to Meta/Facebook.
In this scenario, *Nobody* has read the content other than the person you're communicating with.
This does not exclude an algorithm running running on the sender/recipient's App from scanning the content and sending suggestions to AD servers.
Basically, let the AI figure out what ads get clicked the most for a given string of encrypted 24h window of chat history. Eventually, the AI is going to hit on its “Rosetta Stone”, even without ever formally decrypting the text, much less any human reading it.
With millions of conversations happening on WhatsApp, why shouldn’t that be possible?
And it’s not even a breach, technically, because nothing ever got decrypted, and the similarity vector generated by the AI have, per se, nothing to do with the content of the conversation or the individual that sent them. Run the same training algorithm again and they’d look completely different! Hence they can’t possibly be “personal data” in the sense of the law.
If you can discern meaning from noise, then your theory would work. But discerning meaning from random noise is obviously impossible (i.e. what if there is no meaning?).
If you leak information than you say, then the encryption is worthless. Harmful, even, because you think you have protection when you do not.
[0] https://en.wikipedia.org/wiki/Targeted_advertising
How We Work With Other Meta Companies
As part of the Meta Companies, WhatsApp receives information from, and shares information (see here) with, the other Meta Companies. We may use the information we receive from them, and they may use the information we share with them, to help operate, provide, improve, understand, customize, support, and market our Services and their offerings, including the Meta Company Products. This includes:
...
- improving their services and your experiences using them, such as making suggestions for you (for example, of friends or group connections, or of interesting content), personalizing features and content, helping you complete purchases and transactions, and showing relevant offers and ads across the Meta Company Products; and
(All this is just a guess based on OP's report and the above quote.)
<<Your Messages.
We do not retain your messages in the ordinary course of providing our Services to you. Instead, your messages are stored on your device and not typically stored on our servers. Once your messages are delivered, they are deleted from our servers. The following scenarios describe circumstances where we may store your messages in the course of delivering them:
Undelivered Messages. If a message cannot be delivered immediately (for example, if the recipient is offline), we keep it in encrypted form on our servers for up to 30 days as we try to deliver it. If a message is still undelivered after 30 days, we delete it.
Media Forwarding. When a user forwards media within a message, we store that media temporarily in encrypted form on our servers to aid in more efficient delivery of additional forwards.
We offer end-to-end encryption for our Services. End-to-end encryption means that your messages are encrypted to protect against us and third parties from reading them. Learn more about end-to-end encryption and how businesses communicate with you on WhatsApp.>>
https://www.whatsapp.com/legal/privacy-policy/?lang=en
Assuming they are compliant with their privacy policy, they don’t even have the messages.
So, they say the protection is there once the encryption has been applied. They say nothing about what happens to the content before or after that on the end user's devices. That handling is however covered by other legitimate use clauses in the privacy statement. This covers keyword scanning for targetted ads (so a defence lawyer will say at some point.)
Tangental aside; it still confounds me where the business opportunity of WhatsApp resides for Meta if they "can't" get access to the data.
If this is just in text—and I'm definitely not defending Meta here—could it also be that the ads you see have got us so figured out already? The topic you choose to talk about may be influenced or seeded by your environment (online/offline), and one thing leads to the other almost deterministically.
Here's an experiment: try rolling a die a few times or using a random number generator to pick one word or more from a list like the EFF wordlists [0], and then talk about that exclusively.
[0]: https://www.eff.org/deeplinks/2016/07/new-wordlists-random-p...
I didn't agree to the recent WhatApp nor Facebook's TOS so no longer have their product on my devices. I suggest you do the same, or just sit back and enjoy the specialised, relevant, targeted ads, but think twice before each send.