Ask HN: How can this happen without the phone covertly listening?

39 points by MobileVet ↗ HN
Yesterday I had my 3rd 'tin foil hat' moment with my iPhone and although these were spread out over some time, I find it very hard to believe these were coincidences.

Long story shortish... yesterday we were sitting around the table enjoying our Easter lunch with extended family. Someone else was mentioning a new family at their church and I caught the first names in passing. At first it didn't click but then I said, 'oh are you talking about X & Y Smith? They used to go to abc church and lived in our neighborhood briefly.' Turns out they were.

Fast forward an hour or so and the kids were playing outside. I took a picture of a friend pushing the kids on the swing and went to send it to the friend. I pressed send, Messages and typed out the first 3 letters of her name. The Messages app showed 2 'autocompletions,' the friend I was intending and the woman I had mentioned an hour before!

- the 3 letters that I typed were NOT in the 'Y Smiths' name! - my phone was in my pocket when I uttered the name - I am pretty sure I only said their name once, but could have said it twice - I have not sent 'Y Smith' a text in at least 4 years - I have not called 'Y Smith' in at least 4 years - I was shocked I even had 'Y Smith' contact info in my address book - I have 'hey Siri' turned OFF

WTF?! How is this possible without the device selectively listening for random words and matching against a profile that includes my address book?

72 comments

[ 3.3 ms ] story [ 144 ms ] thread
If you hadn't had that conversation it would not have mattered that she showed up in autocomplete. You would've thought nothing of it. It's only because you had the conversation that the name had any significance. Probably hundreds of times it autocompleted names of people from 4 years ago but since there was no corresponding conversation, you didn't take note and just looked past it.
There are far, far too many "coincidences" from many people with this same or similar story.
Far too many based on what? Consider how many chances there are for stories like this: just about everyone owns a phone, and people share things all time.
Ads based on private conversations is my primary experience.

People talk about something, never look it up, never search for it, never try to buy it, but they see ads for it.

I guess the question then becomes "would that name have shown up if op hadn't said it?" And frankly it seems pretty naive to say that it's only an issue because op noticed it this time.
Isn't it possible that someone else messaged the Smith's about the conversation, and mentioned OP's name, and then some algorithm made the connection?
Actually, I would have been dumbfounded as to why it was an autocomplete solution. The letters were not an intersection for the second lady's name.
> it would not have mattered that she showed up in autocomplete

It certainly would, if you cannot figure out why a query returns an apparently fully unrelated record.

> Someone else was mentioning a new family at their church and I caught the first names in passing

Did they happen to add them to their contacts recently or send the X&Y Smiths a message about you?

are the letter close on a standard qwerty layout from the intended friend to the "eavesdropped" name?
There could also be things in the contact like address or other information that it was going off of.
Could it be that you had interacted with those contacts in close succession at the time?
Do they also have iPhones? It could be as simple as the Find My network identifying that you'd been near each other (still creepy).
I am beyond convinced that our phones are hot mics to the point that my wife and I make a conscious effort to move our phones to another room when we have conversations.

There have been far too many “anecdotal” examples of me mentioning some thing/product out loud, never searching for anything related to it and then seeing ads for it over the next few days.

I resisted the conspiracy theory type explanation but it just occurs too often to be some sort of Baader-Meinhof phenomenon.

> It happens far too often to be some sort of Baader-Meinhof phenomenon (or whatever).

Hmmm... is awareness of the Baader-Meinhof phenomena leading to an increase in its own phenomena because one actively tries to disprove it by deliberately noticing an even greater frequency due to their own awareness of it?

Or it’s not that simple and that explanation is only a component of reality, and not the complete reality

Lets stop giving these apps a pass. We already know some abuse privileged signing keys to access permissions without asking or a permission is used for other things like third party packages

I wasn't saying that the problem they were having was real or not, I don't have the knowledge for that, I was simply wondering if the vigilance from an awareness of and a desire to disprove the phenemona exacerbates it.
My first thought is: this is probably just coincidence. It was just a useless completion that normally would have gone totally unnoticed except for your recent conversation about this person.

It seems possible that a phone might be continually listening to background chatter and trying to extract salient/useful pieces of information to then feed into predictive assistants... but honestly that seems like a lot of computational work (especially on a device known for excellent use of its power budget), and a lot of development work. All for a very small incremental improvement in assistant UX that is probably not going to be noticed by most people.

Your phone might be listening in the background (e.g. Hey Siri needs this to work). Even with the option off (as you said), it might still be processing words on-device for better suggestions.

Another non-tinfoil explanation would be cross matching a lot of data from here and there (for example, if you had some event in calender with their names or their name was in some message/text even if you haven't called them for years) and might got its way in there too. There is probably much more going on that I don't even know, too.

The point is: while it could be phone listening, it can very well be a result of some huge data processing on your device without listening too.

edit: oh, having read the other comments, coincidence/selective bias is even more likely than anything I've written.

> Your phone might be listening in the background (e.g. Hey Siri needs this to work). Even with the option off (as you said), it might still be processing words on-device for better suggestions.

That's _barely_ any better. That's essentially the behavior that's being complained about but implemented a different way.

The easiest way to understand this isn't happening is just to hook your phone up to a proxy like Charles, you see all network requests. If speaking automatically triggers some request you'd see it immediately (and see the URL and payload)

There was a similar conspiracy when I was in the army, about phones being able to be eavesdropped/wiretapped even when turned off. In short - they can't, but people always believed there was some super secret way to do that

Technically, the scenario described could happen entirely offline: voice recognition, transcription, then match for autocomplete (not that I am claiming this is what happened).
Full-text voice recognition is CPU intensive though, so one can expect mobile that's doing that all the time to drain its battery pretty fast in a noisy environment which should be rather obvious.
> If speaking automatically triggers some request you'd see it immediately (and see the URL and payload)

That's one possible way to implement a feature like this. Ruling out this mechanism doesn't rule out any others.

It could happen offline, it could also happen async.
They could also queue the data then piggy back it on a “legit” network transaction and since it’s encrypted you’ll never be the wiser.
It is also possible that the device does not send the data instantly, but rather at a time intervall or bundled with other legit requests.
> phones being able to be eavesdropped/wiretapped even when turned off.

Nobody is auditing the baseband firmware blobs, and with non-removable batteries phones are never really turned off. I would be surprised if the NSA doesn't have a backdoor in it. Pure conjecture, of course, but the alphabet soup boys have the motivation, the ability, and the history, of getting their backdoors put into tech.

(comment deleted)
The EG.25 (Pinephone modem) baseband (not the Linux firmware (and of course, not the application processor OS)) blob source was up on Github for a while. I read through part of it. I think they managed to get it taken down with a DMCA request and I haven't seen it since.

I really wish I had cloned it locally, I don't know why I didn't.

Many apps are "pinning" TLS certificates, so you can't see their traffic w/o changing a hard-coded public key in the app itself.
So you had this person's info in your contacts, and when you started autocompleting a number with a picture of them while you were in the same (geo)location (no less surrounded by other people that know/have this info), the information was autocompleted...

and your conclusion is that this is audio data being misused!?

One time I was with some friends and we tested something like this.

We chose a random thing I've never purchased, looked at, or have any interest in: "lawnmowers".

We put my phone on the table and said lawnmowers a lot, waited a few minutes, then I went to a website I know that has a lot of ads (Notebook Check - not that it matters).

Then I scrolled down and waited for an ad to load and it was FOR A LAWNMOWER. I don't even have a lawn!

I have in mind a YouTube original tv series hosted by cold reading psychics ("Crossing Over With John Edward") except instead of guessing your grandma's maiden name they just talk about mesothelioma then have the audience open their phones and look for spooky targeted ads.
An ad and analytics package in any app with enterprise credentials could have this permission (or in an app you authorized to listen for a different reason)

And then the data brokers share this with all the other apps

Every app creator can honestly say “noo we dont listen wdym” to you, to Congress, to everyone

and the analytics package creator and data brokers are going to be laughing for decades

This was a few years, it does make me wonder if I had a malicious app on my phone.

Now on Android there is a green light when the mic is on so presumably that would catch this? Unless it is at the OS level

Additionally, I’ve seen ads served by being on someone else’s home wifi or the same residential wifi together. You get stuff they looked up (or said). I’ve seen it happen in proximity with shared contacts too. Meeting a friend on vacation and they start getting ads about where I live.
>How can this happen without the phone covertly listening?

Glad to see someone else asking themselves this question, I've found myself in exactly this situation a number of times in recent years.

I've had this experience several times as well. Last time was on a date. We were talking a little bit about board games and tarot cards, and some other third thing . I have never purchased any of the three things or even browsed them on Amazon. Never get ads/recs for those on Amazon, just books. Next day, however, my Amazon recs were all for board games, tarot cards, and whatever the third thing was.
Telemetry is the catch-all phrase used to describe data captured about a device. Telemetry servers combine this information to build profiles on users. Network models of relationships between devices and user behaviours are used in recommendation engines including autocomplete results.

If you are in the same room as a cohort of people, if one or more people had been searching or messaging the X & Y Smiths, then by geolocation since you all attend the same church, you're associated as potentially interested, especially considering you already had their details saved.

There are also fuzzy logic factors, like maybe those three letters weren't in their name but were in their phone number (each number corresponds to several alphabet characters), or might have been in a message you've long since deleted but is still in your autocomplete index which combined with the geolocation weighting could have caused it to pop up as an option.

In these instances it might appear your phone is listening, but you were in the same room as some people who were probably also interested in that family, had their data in your history, and being a new connection could have boosted its relevance too. (I just saw someone else also answered that you probably only noticed this event because of the conversation, or the Baader–Meinhof phenomenon, which is also plausible)

Not saying definitively that your phone was not covertly listening but our devices are capturing and correlating a massive amount of dimensions related to our behaviours at all times, so it's not beyond reason that enough of these factors lined up to cause the autocomplete engine to suggest it as a reasonable option.

I could definitely believe it was deep telemetry as the intended message recipient did have proximity to the unexpected 'match' earlier in the day. I am 99% sure they hadn't swapped contacts though, so it wouldn't be as easy as a matched contact connection.

Interesting idea about the mapping of numbers to letters, oh how I miss 800-call-sal advertisements.

I was reaching on the mapping of numbers to letters to be honest. If I were designing a system to track and corelate everything I'd take it into account since it's still in use, but yeah that was a definite stretch in this case.

But I am pretty sure this was just a proximity match. When you're dealing with the quantity of telemetry the big players deal with, you're talking billions of people in real-time all day long, then how do you figure out what's relevant and what isn't?

Physical proximity is important. You don't have to swap contacts with someone for your telemetry to connect you with them. I mean you were right next to someone who was right next to them, out of billions of people you're relevant, so no one needs to share contact details. The manufacturer of the phone knows your geolocation. Your telecom company knows your geolocation. If you have bluetooth or wifi switched on and you're already fingerprinted, then every chain store knows your geolocation. If you use a credit card or eftpos card anywhere, the products you purchase are combined into your profile, etc etc.

That and you already had them in your contact list (even though you were surprised they were, you're not saying they weren't, I have people in my contact list from 15 years ago I only spoke with one time...), they already know that you've bumped into this contact before in the past, and boost the recommendation because you shared the contact and the geospatial relevance in a short period of time.

Like I'm pretty cynical and suspicious at the best of times, but once I started to realise the above, all my "oh shit they're listening" moments kind of dissolved because I could trace all of them back to being in the same room as someone who had met a person, or had been actively searching a related topic in the past few days.

Yeah it's still spooky, it's the reason I run a pi-hole, and got myself off most social media.

Also I noticed this thread got flagged. Not sure exactly why but I think it's because this same subject has come up a few times. I do think people need to better understand how network analysis can reveal spooky shit about our behaviours, like our devices don't need to be literally listening to our words in order for corporations and governments to know exactly who we are or what we're about. There's tons of different signals we all send out each day that fingerprint exactly who we are, who we're related to, and what we care about, they don't need realtime voice processing.

Yea, I understood the number mapping was a stretch... just a fun memory.

Thanks for laying out a very plausible case for how this match could occur without actually listening to the ambient conversation.

I also run a pi-hole at home and almost never visit social media. Frankly I am surprised it took this long for people to understand how giving up their privacy had a cost, I was hoping the backlash against Facebook et al was going to start a decade ago.

Sometimes a friend comments on how ugly some car model is in its yellow variant. Then onwards you start realizing that the streets are full of that ugly yellow car !!! How come?

Right after bringing some trivial detail into consciousness, there is a period of time where we are more aware about this arbitrary piece of data. Our brain is a pattern-matching machine, so it does the rest.

Want to know if it was the iPhone spying on you? Follow the somewhat scientific (ish) method and take note of absolutely all autocomplete suggestions and make a conscious effort to realize how much relevance they had every time. That way you'll learn to differentiate "coincidence" from coincidence.

Ah, the Baader-Meinhof Phenomenon.
Does the Y Smith name match perhaps by letters from the number pad?
That people can be so oblivious to the ways in which targeted advertising actually works while simultaneously so vulnerable to conspiracy theories about "hot mics" has me worried.
Do all participants have iPhones?

Apple could have an almost complete social graph. If Y Smith is the common acquaintance between you and your friend, then it would be the best second suggestion if you want to share something in that part of your social graph.

Yes, there is some potential there... though I am pretty sure the other parties had not swapped contact info yet as they were new and only just met.
We need an led for microphone on.
Most likely it would be software based which means how could you trust it? Even if you mandated hardware based they can still conceivably access it after the led via a hardware side channel.
It's worse:

The Analytics tools know so much about you that they can actually predict what you're thinking much of the time. They know the new family went to your church and know you would likely talk about it.

I convinced my wife of the opposite by deciding to talk about a topic we never talk about in front of our phones. She was worried that she was seeing ads for things she'd talked about and I said that it was more than likely that she had searched for similar things and was getting ads based on what she likes. She wasn't convinced.

So, we chose X. Neither of us play or follow X in any way. It's not something we'd search for or anything related to it. But we experimented with talking about X and at least one famous Xer that we could name. With our phones on the table.

We never received any advertising about X.

I hear the same thing from some of my friends, what do they have in common? They don't block trackers while surfing. Haven't heard it from people who block those. If it were really voice recognition, it shouldn't have an effect on that. I think what most people don't want to admit is how predictable they are and how easy it is to make predictions based on lots of data.
Another possible explanation could be that your extended family searched for Y Smith after the event and facebook listed it seeing that you are friends with both.

Contrary to others’ experience, I couldn’t reproduce the “talk about X and wait for an ad on X” experiment, I believe it is mostly just the mentioned Frequency illusion, or perhaps some network level thing like A and B talk about something, B later searches for that and “rightfully” get ads on it, and due to A and B’s similar profile or something A also get recommended said product.

Happens all the time. The tiny digital versions of you residing on all the servers are a better version of you. So good, they know what you want before you do. Accept that you're being surveilled, accept that you can't do anything about it and don't listen to the people who claim it's a coincidence because they're still connected to the machine.