It is new. It is fancy. It is not clear where HE/DP is being used, it depends if the code is written using the Swift toolkit, but even that has paths for exfiltration if used incorrectly. They claim they are using DP in Photos as stated in the article here:
But the fact remains they are looking at your pictures. I do not trust them for one fleeting fart in the wind on this. Think about it for a hot second: HE/DP allows you to perform operations on the data without knowing the data, but what if someone goofs an operation and it ends up returning the actual data?
Sorry, not buying it. Crypto is hard to get right, and when it is monetized like this for "new features", it is wildly unnecessary and exposes users to more risk.
> what if someone goofs an operation and it ends up returning the actual data
That's kind of like saying "they can take a picture of your face and transmit it, but what if someone goofs an operation and sends your actual head instead".
Any encrypted data the server has cannot 'accidentally' be decrypted, and as someone else explained in this thread they only send encrypted vectors describing features of the image (building, tree, etc) and not the actual image. It's certainly not a fact that "they are looking at your pictures" [1].
[1] "they" being Apple; the other guys could have backdoored the whole OS and Apple employee's computers for all I know
> From my own perspective, computing privacy is simple: if something happens entirely on my computer, then it's private, whereas if my computer sends data to the manufacturer of the computer, then it's not private, or at least not entirely private. Thus, the only way to guarantee computing privacy is to not send data off the device.
I don’t even use iCloud Photos and this was on by default. Very bad move by Apple to ship my photos off my device, without my permission, in any shape or form, I don’t care.
It’s funny how the wordsmiths come out to defend Apple here.
Your “photos” aren’t shipped off-device without your knowledge, just an arbitrary blob of ”metadata” that you can’t audit describing everything about that photo. :)
It’s sort of like “I don’t want my WiFi router uploading my house online!” And someone replying “it’s not your physical house, just a map of the house and the real time location of everyone in it! The house never moves!”
So it sends your photos to be indexed on Apple servers. Turned on by default.
This is probably done to compete with Google Photos which has a great photo search by word feature.
With that said, Apple can use whatever privacy measures to protect user data. But at the end of the day, a subpoena can easily force them to hand over data.
The best privacy measure is to just not have the data. I guess indexing photos offline in phone is not very feasible yet.
it literally says it uses global search index on the label below the check mark. It seems more than likely that (now or at least in the long run) they will use user data to enhance this index.
It's funny how one can use "literally" while almost demonstrating illiteracy.
It's right there in the help text:
> Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest.
They are saying that they match your photo against their global index.
They are not saying that they will add your photo to their global index.
> This is probably done to compete with Google Photos which has a great photo search by word feature.
Apple Photos is already capable of searching your photos by word/content fully offline on-device.
Google Photos is unable to search your photos by word/content fully offline on-device. Hell, it can't even search by file name if you don't sync your photos to the cloud!
I don't think Apple has to worry about Google at all.
> I guess indexing photos offline in phone is not very feasible yet.
There has been a form of on device indexing since at least iOS 12. My understanding is it performs a basic indexing typically overnight when the phone is charging and allows one to perform searches like "dog" or "car" and pull up matching images.
The Internet Society makes the following recommendations based on the European Commission’s proposal:
That the European Committee introduce safeguards for end-to-end encryption.
That the European Committee prohibits the use of scanning technologies for general monitoring, including client-side scanning
"I don't understand most of the technical details of Apple's blog post"
I do:
- Client side vectorization: the photo is processed locally, preparing a non-reversible vector representation before sending (think semantic hash).
- Differential privacy: a decent amount of noise is added the the vector before sending it. Enough to make it impossible to reverse lookup the vector. The noise level here is ε = 0.8, which is quite good privacy.
- OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library).
- Homomorphic encryption: The lookup work is performed on server with encrypted data. Apple can't decrypt the vector contents, or response contents. Only the client can decrypt the result of the lookup.
This is what a good privacy story looks like. Multiple levels of privacy security, when any one of the latter 3 should be enough alone to protect privacy.
"It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here.
I'm not sure what the right call is here. But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true. There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk. But if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this.
I appreciate the explanation. However, I think you do not address the main problem, which is that my data is being sent off my device by default and without any (reasonable) notice. Many users may agree to such a feature (as you say, it may be very secure), but to assume that everyone ought to be opted in by default is the issue.
I’m not the person you asked, but I agree with them. To answer your question: No, I do not use iCloud to store my photos. Even if I did, consent to store data is not the same as consent to scan or run checks on it. For a company whose messaging is all about user consent and privacy, that matters.
This would be easily solvable: On first run show a window with:
> Hey, we have this new cool feature that does X and is totally private because of Y [link to Learn More]
> Do you want to turn it on? You can change your mind later in Settings
In response to your question in the parent comment, no, I do not use iCloud. And I do not sync any of the things you mentioned here. If someone already consented to using iCloud to store their photos then I would not consider the service mentioned this post to be such a big issue, because Apple would already have the data on their servers with the user's consent.
edit: I will just add, even if we accept the argument that it's extremely secure and impossible to leak information, then where do we draw the line between "extremely secure" and "somewhat secure" and "not secure at all"? Should we trust Apple to make this decision for us?
I do not have anything backed up on any cloud servers on any provider. If I had to buy a new phone I would start from a fresh installation and move all of my data locally. It's not that I'm a "luddite", I just couldn't keep track of all of the different ways each cloud provider was managing my data, so I disabled all of them.
If only Apple had a centralized backup service that could store everything automatically at a click of a button so you wouldn’t have to juggle multiple cloud providers…
Apple IS just another cloud provider / centralized backup service. It's not fundamentally different than others, and if you're not in select group of whatever the respectful term is for those who stay strictly inside apple ecosystem, you will have multiple clouds and multiple data sets and multiple backups that all interact with each other and your heterogeneous devices in unpredictable ways. Icloud will not help you with that any more than google cloud or Samsung cloud etc. They all want to own all of your stuff, neither is simply a hyper helpful neutral director.
The “fundamental difference” is that it’s better integrated with your device and can backup the internal state of your device and the apps.
Even if you use Microsoft Office or GSuite and save using the standard file picker, you can save to iCloud. iCloud has a native app for Windows and plug ins on Windows to sync browser bookmarks for Chrome, Edge and Firefox
And the alternative people are proposing are four or five self hosted solutions?
Again, I think there's an assumption of single device / ecosystem loyalty in your statement? I have an android phone and iOS phone and three android tablets and a bunch of laptops with various operating systems.
Iphone is "just another device". I don't feel Icloud is any better integrated with my Samsung note, than google is integrated with my iPhone - in fact, the opposite. Google, for example, CAN sync my photos across iphone and Android and windows devices. Whereas my wife knows the primeval scream from the home office every 6 months I try to claw photos out of apple's greedy selfish hands :-)
For people who JUST use iphone, sure, Icloud is the boss just like for people who JUST use e.g. Samsung galaxy the Samsung cloud is awesome. But that's not a high bar. I feel we are still lacking empathy here for people like original poster who may have more than one device in their lives.
And none of these can sync your bookmarks, iOS settings or store the internal state of apps on your iPhone.
And I wouldn’t have the same arguments if they weee using Google cloud. But they are concerned about “privacy” and trust Google?
But my argument is about people thinking that Apple or Google should care about the minuscule number of people who are hosting their own syncing services
By one lone outlier who decides for “security” that the don’t want to support the platforms backup solution. That app purchase didn’t have to do anything besides store information locally in their sandbox
> If someone already consented to using iCloud to store their photos then I would not consider the service mentioned this post to be such a big issue, because Apple would already have the data on their servers with the user's consent.
No, if you enable Advanced Data Protection for iCloud[1], the photos stored in Apple Photos are end to end encrypted.
None of that is relevant to my point. You seem to be trying to catch people in some kind of gotcha instead of engaging honestly with the problem at hand. But alright, I’ll bite.
Yes, I always start with clean installs, both on iOS and on macOS. Sometimes I even restart fresh on the same device, as I make sure my hardware lasts. I don’t sync bookmarks, I keep them in Pinboard and none of them has any private or remotely identifiable information anyway. I don’t care about saving browser history either, in fact I have it set to periodically auto-clear, which is a feature in Safari.
No I am trying to say with a connected device using online services, the service provider is going to have access to your data that you use to interact with them.
To a first approximation, everyone in 2024 expects their data and settings to be transferred across devices.
People aren’t working as if it is 2010 when you had to backup and restore devices via iTunes. If I’m out of town somewhere and my phone gets lost, damaged or stolen, I can buy another iPhone, log into my account and everything gets restored as it was.
Just as I expect my watch progress to work when I use Netflix between my phone, iPad, Roku devices etc.
And that should rightfully be your informed choice. Just like everyone else should have the right to know what data their devices are sending before it happens and be given the informed choice to refuse. People shouldn’t have to learn that from a random blog post shared on a random website.
How many times are you going to shift the goalposts? This is getting tiresome, so I’ll make it my last reply.
I don’t have Netflix but neither is that relevant to the point, you’re obviously and embarrassingly grasping at straws.
No one is arguing against continuous cloud backups, they’re arguing about sending data without consent. Which, by the way, is something Apple used to understand not to do.
Apple’s OS are already filled with Windows Vista style popups and permissions for inconsequential crap, people have been making fun of them for that for years.
If you are doing continuous cloud backups and using Apple services - you are already giving Apple your data and your solution is to add even more permissions? You are not going to both use any Apple service that requires an online component and keep Apple from having your data.
Isn’t it bad enough that I have a popup every time I copy and paste between apps?
> Isn’t it bad enough that I have a popup every time I copy and paste between apps?
For me, not really no. It reminds me I am copying information and not from some phishing app, I find it informative.
And I'm probably one of the few who actually click "Reject" to the cookie pop ups having to click no on 3742 legitimate consents.
The simple answer is everything should be opt-out. I'll opt-in if I require it because frankly, regardless to how Fort-Knox my data is $CORP still cannot be trusted.
Strictly Signal via self-hosted VPN for messages. My email web client provided by my email server (Zimbra) which are hosted on colocated servers. 3cx for calls via self-hosted PBX.
Video conferencing instead of FaceTime are made via self-hosted Jitsi and if I am to brag all running on FreeBSD.
Out of Apple or Google I trust neither however will align with Apple more than Google. It's as close as I can get from not having data collected from mongrels.
As a general principle, I think computers should execute commands that users issue, and then wait for the next command. That's it.
Computers should not be sneakily doing things in the background without my commanding them to do so. But if they insist that the only way they can work is by doing things in the background, then I expect the computer to at the very least obtain my consent before doing those things. And computers should definitely not be exfiltrating anything over to the network without my explicit command to do so. This shit world we are living in where your computer just does whatever the application developer wants it to do rather than what the user wants it to do has to come to an end!
Netflix being unable to know your watch history on their service is exactly the goal of homomorphic encryption. The technology to make that work at that scale does not exist, however for smaller bits of data, eg phone numbers, that's entirely possible!
With PIR, an Apple phone recieving a phone call queries Apple's database with that phone number, but because it's using homomorphic encryption, Apple doesn't know the number that called despite looking it up in their database to provide caller id info, so they can't tie your phone number and the callers phone number together.
If you care about your “privacy” and no external service providers having access to your data - that means you can’t use iCloud - at all, any messages service, any back up service, use Plex and your own hosted media, not use a search engine, etc.
No what we need is for people to realize that no multi trillion dollar company is going to make life harder for 99.999% of their users because of a few outliers
How exactly is a new feature that is not advertised harder for you, or for anyone for that matter?
I bet most of those made up numbers of yours will have no idea that the feature exists.
A simple screen like they usually do with "Whats new in iOS" could easily have let you enabled it on the get go, with the additional benefit that you would have been made aware of it existing.
This iOS 18.2 update had no such screen, I just updated.
I think it does address the main problem. What he is saying is that multiple layers of security is used to ensure (mathematically and theoretically proved) that there is no risk in sending the data, because it is encrypted and sent is such a way that apple or any third party will never be able to read/access it (again, based on theoretically provable math) . If there is no risk there is no harm, and then there is a different need for ‘by default’, opt in/out, notifications etc.
The problem with this feature is that we cannot verify that Apple’s implementation of the math is correct and without security flaws. Everyone knows there is security flaws in all software, and this implementation is not open (I.e. we cannot review the code, and even if we could review code we cannot verify that the provided code was the code used in the iOS build). So, we have to trust Apple did not make any mistakes in their implementation.
If you don’t trust Apple to do what they say they do, you should throw your phone in the bin because it has total control here and could still be sending your data even if you opt out.
As someone with a background in mathematics I appreciate your point about cryptography. That said, there is no guarantee that any particular implementation of a secure theoretical algorithm is actually secure.
Agreed, but surely you see a difference between an open source implementation that is out for audit by anyone, and a closed source implementation that is kept under lock & key? They could both be compromised intentionally or unintentionally, but IMHO one shows a lot more good faith than the other.
No. That’s your bias as a nerd. There are countless well-publicised examples of ‘many eyeballs’ not being remotely as effective as nerds make it out to be.
You say this as if being shamed into patching the occasional vuln is equivalent to security best practices.
Open code which can be independently audited is only a baseline for trustworthy code. A baseline none of those three meet. And one which by itself is insufficient to counter a reflections on trusting trust style attack. For that you need open code, diverse open build toolchains, and reproducible builds. None of which is being done by those three.
Are you getting your ideas about security from the marketing department?
Go ahead and put that cup of kool-aid down for a minute. There are so so many OSS packages out there that have never been audited? Why not? Because people have better things to do. How many packages have you audited? Personally, I don't have the skillz to do that. The people that do expect to be compensated for their efforts. That's why so many OSS packges have vulns that go unnoticed until after they are exploited, which is the same thing as closed source.
OSS is not the panacea that everyone touts it to be.
> There are so so many OSS packages out there that have never been audited? Why not? Because people have better things to do.
I'm not aware of any major open source projects that haven't experienced some level of auditing. Coverity alone scans everything you're likely to find in a distribution like Debian or Fedora: https://scan.coverity.com/o/oss_success_stories
> How many packages have you audited?
Several on which I depend. And I'm just one pair of eyeballs.
> Personally, I don't have the skillz to do that.
Then why are you commenting about it?
> OSS is not the panacea that everyone touts it to be.
I don't know who's touting it as a panacea, seems like a strawman you've erected. It's a necessary pre-requisite without which best practices aren't possible or verifiable.
> There is also no guarantee that Apple isn't lying about everything.
And at that point all the opt-in dialogs in the world don't matter and you should not be running iOS but building some custom Android ROM from scratch.
Except for the fact (?) that quantum computers will break this encryption so if you wanted to you could horde the data and just wait a few years and then decrypt?
>Let’s say you wanted to count how many of your online friends were dogs, while respecting the maxim that, on the Internet, nobody should know you’re a dog. To do this, you could ask each friend to answer the question “Are you a dog?” in the following way. Each friend should flip a coin in secret, and answer the question truthfully if the coin came up heads; but, if the coin came up tails, that friend should always say “Yes” regardless. Then you could get a good estimate of the true count from the greater-than-half fraction of your friends that answered “Yes”. However, you still wouldn’t know which of your friends was a dog: each answer “Yes” would most likely be due to that friend’s coin flip coming up tails.
Quantum computers don't and won't meaningfully exist for a while, and once they do exist, they still won't be able to crack it. Quantum computers aren't this magical "the end is nigh" gotcha to everything and unless you're that deep into the subject, the bigger question you've got to ask yourself is why is a magic future technology so important to you that you just had to post your comment?
Anyway, back to the subject at hand; here's Apple on that subject:
> We use BFV parameters that achieve post-quantum 128-bit security, meaning they provide strong security against both classical and potential future quantum attacks
Hypothetical scenario: Theo de Raadt and Bruce Schneier are hired to bring Apple products up to their security standards. They are given a public blog, and they are not required to sign an NDA. They fix every last vulnerability in the architecture. Vladimir Putin can buy MacBooks for himself and his generals in Moscow, enable Advanced Data Protection, and collaborate on war plans in total confidence.
I am pretty sure that if we had those people in charge of stuff like this there would be no bar above which "opt in by default" would happen, so I am unsure of your point?
Theo de Raadt is less competent than Apple's security team (and its external researchers). The main thing OpenBSD is known for among security people is adding random mitigations that don't do anything because they thought them up without talking to anyone in the industry.
I think I'm saying: you're not sending "your data" off device. You are sending a homomorphically encrypted locally differentially private vector (through an anonymous proxy). No consumer can really understand what that means, what the risks are, and how it would compare to the risk of sending someone like Facebook/Google raw data.
I'm asking: what does an opt in for that really look like? You're not going to be able to give the user enough info to make an educated decision. There's ton of risk of "privacy washing" ("we use DP" but at very poor epsilon, or "we use E2E encryption" with side channel data gathering).
There's no easy answer. "ask the user", when the question requires a phd level understanding of stats to evaluate the risk isn't a great answer. But I don't have another one.
In response your second question, opt in would look exactly like this: don't have the box checked by default, with an option to enable it: "use this to improve local search, we will create an encrypted index of your data to send securely to our servers, etc..." A PhD is not necessary to understand the distinction between storing data locally on a machine vs. on the internet.
Exactly. It's the height of arrogance to insist that normal users just can't understand such complex words and math, and therefore the company should not have to obtain consent from the user. As a normal lay user, I don't want anything to leave my device or computer without my consent. Period. That includes personal information, user data, metadata, private vectors, homomorphic this or locally differential that. I don't care how private Poindexter assures me it is. Ask. For. Consent.
Don't do things without my consent!!! How hard is it for Silicon Valley to understand this very simple concept?
> The average smartphone is probably doing a hundred things you didn’t knowingly consent to every second.
You've succinctly identified a (maybe the) huge problem in the computing world today. Computers should not do anything without the user's command/consent. This seems like a hopeless and unachievable ideal only because of how far we've already strayed from the light.
Even Linux, supposedly the last bastion of user control... it's a mess. Do a fresh install and type ps ax at a shell. You'll see dozens of processes in the background doing god knows what. I didn't consent to any of this! The distribution's maintainer simply decided on my behalf that I want the computer to be running all these processes. This is totally normalized!
I don't expect my computer to ask for consent again and again for every byte sent over the network, but I do expect it to obtain my consent before generally accessing the network and sending bytes over the network.
It’s amazing how hostile Silicon Valley (and HN commenters) are to the basic idea of consent. It’s as if simply asking the user for permission is a grave insult to these technologists. “I shouldn’t have to ask permission! It implies I’m doing something bad!” they might be thinking.
If the world was a nightclub, “Silicon Valley” would be a creepy guy who walks up to every woman and says “You’re now dating me. To stop, you need to opt out using a form that I will do my best to make sure you can’t read.”
You're inverting morality and infantilising the consumer. Apple is a corporation. Corporations don't owe you moral anything, except as required by law.
Choosing an Apple product is consent to trusting Apple. Continued use their products represents ongoing consent. This is an objective fact about all complex connected devices and it cannot possibly be otherwise.
Corporation are driven by people. They’re not a separate entity that decides to do things while their owners are sleeping. Every actions have someone that suggested it and someone that gave the green light.
Yes you did. You purchased a computer, put this software on it and executed it. If you didn't want it to do whatever it's doing you should have determined what it would do beforehand and chose not to do it.
Even assuming that running the software implies my consent (which I would dispute), how do I make the decision about whether I should execute the software if I don't know what it is doing?
This all-or-nothing approach is also problematic. I should not have to allow the developer free rein to do whatever he wants, as a condition of using the software. This is why operating systems are slowly building granular permissions and consent checks.
Installing and booting Linux absolutely implies consent to let it do what it does. It's open source, you can evaluate what it does before booting it. You know it's comprised of many processes, you know it has a networking stack, you connected it to a network. You can't then ask OMG why didn't it ask before sending something?
I agree that all-or-nothing is problematic but even with a flexible permission system the best you can hope for is for all the things apps do to be itemized and set to sane defaults. But even then sanity is subjective. For every person like you (and me fwiw) who values privacy there are 1000 people who will never find the settings, don't care about privacy, and will wonder why stuff isn't working.
Ultimately privacy is similar to security in that it comes down to trust. If you don't trust your OS you're screwed. Your choices are try to exert as much control over it as possible, or don't use it.
> I do expect it to obtain my consent before generally accessing the network and sending bytes over the network.
How would that make any difference in this case? Presumably, you'll have long-ago checked the "allow general access to the network" setting, so you've given consent to the "send my photo data" action. Heck, surely connecting to the internet in the first place is implicit consent that you want to send stuff over the network?
If I were actually given the choice, I would not check any checkbox allowing an application broad, unfettered access to the network. But, in most cases I'm not even given that choice!
Every TCP session leaks some PRNG state for the ISN. That might leak information about key material.
Every NTP session leaks time desync information, which reveals—on modern hardware—relativistic travel, including long airplane trips.
Every software update leaks a fortune about what you run and when you connect.
I don’t think it’s reasonable to ask that people consent to these; I don’t think they can. I absolutely agree that photo metadata is different and at a way higher level of the stack.
Even here with HN crowd: it's not an index, it's not stored on a server, and it's not typical send-securely encryption (not PK or symmetric "encrypted in transit", but homomorphic "encrypted processing"). Users will think that's all gibberish (ask a user if they want to send an index or vector representation? no clue).
Sure, you can ask users "do you want to use this". But why do we ask that? Historically it's user consent (knowingly opting in), and legal requirements around privacy. We don't have that pop up on any random new feature, it's gated to ones with some risk. There are questions to ask: does this technical method have any privacy risk? Can the user make informed consent? Again: I'm not pitching we ditch opt-in (I really don't have a fix in mind), but I feel like we're defaulting too quickly to "old tools for new problems". The old way is services=collection=consent. These are new privacy technologies which use a service, but the privacy is applied locally before leaving your device, and you don't need to trust the service (if you trust the DP/HE research).
End of the day: I'd really like to see more systems like this. I think there were technically flawed statements in the original blog article under discussion. I think new design methods might be needed when new technologies come into play. I don't have any magic answers.
The third choice, after opt-in and opt-out is to force the user to choose on upgrade before they can use their device again. "Can we use an encrypted, low-resolution copy of your photos that even we ourselves can't see?"
Okay except "encrypted, low-resolution copy of your photos" is an incredibly bad explanation of how this feature works. If nobody on HN so far has managed to find an explanation that is both accurate and understandable to the average consumer, any "hey can we do this" prompt for this feature is essentially useless anyways. And, IMO, unnecessary since it is theoretically 100% cryptographically secure.
I think it's sufficiently accurate, why don't you think it is? I don't think the vector vs low-res aspect is particularly material to understanding the key fact that "even we ourselves can't see?"
I don't care if all they collect is the bottom right pixel of the image and blur it up before sending it, the sending part is the problem. I don't want anything sent from MY device without my consent, whether it's plaintext or quantum proof.
You're presenting it as if you have to explain elliptic curve cryptography in order to toggle a "show password" dialogue but that's disingenuous framing, all you have to say is "Allow Apple to process your images", simple as that. Otherwise you can argue many things can't possibly be made into options. Should location data always be sent, because satellites are complicated and hard to explain? Should we let them choose whether they can turn wifi on or off, because you have to explain IEEE 802.11 to them?
> I don't want anything sent from MY device without my consent
Then don’t run someone else’s software on your device. It’s not your software, you are merely a licensee. Don’t delude yourself that you are morally entitled to absolute control over it.
The only way to have absolute control over software is with an RMS style obsession with Free software.
The “moral entitlement” has nothing to do with this. The software is legally required to abide by its license agreement (which, by the way, you are supposed to have read, understood, and accepted prior to using said software).
I honestly can’t tell if you’re being sarcastic. A license grants the end user permission to use the software. It is not a series of obligations for how the software operates. This would be excruciatingly obvious if you read any software license.
A license agreement is, well, an agreement between the manufacturer and the consumer which may include a requirement to acknowledge certain aspects of how the software operates (e.g. the user may be required to agree to “share” some data).
Some commercial software licenses may include various disclaimers which exist to ward away litigious assholes. They only serve to protect the vendor against legal complaints, and do not impart responsibilities upon the vendor. Such disclaimers are not necessary but corporate lawyers have a raison d'être, and at a certain scale assholes become inevitable.
They might not be legally entitled to it, but that's just because of our shitty "intellectual property" laws. Morally speaking, OP is absolutely entitled to have a device that they own not spying on them.
Regardless of one's opinion of intellectual property laws, nobody is morally entitled to demand that someone else build the exact oroduct they want. In fact it is immoral to demand that of other people — and you certainly wouldn’t like it if other people could demand that of you.
Want a phone that doesn’t spy on you? Make it yourself. If you can’t, find some like-minded people and incentivise them (with money or otherwise) to make it for you. If they can’t (or won’t) perhaps contemplate the possibility that large capitalist enterprises might be the only practical way to develop some products.
This has absolutely nothing to do with "might makes right". If a fast food store decides to offer a Vietnamese Peanut Burger and Sugar Cane Juice combo, nut allergy suffers are not "morally entitled" to a nut-free option and diabetics are not "morally entitled" to a sugar-free juice option. This applies whether the fast food store is a small family run business, or McDonalds.
To suggest that customers are "morally entitled" to a Samsung phone with zero tracking and zero telemetry is similarly absurd. If you don't like Samsung's product, don't buy it.
> If a fast food store decides to offer a Vietnamese Peanut Burger and Sugar Cane Juice combo, nut allergy suffers are not "morally entitled" to a nut-free option and diabetics are not "morally entitled" to a sugar-free juice option.
Why not? What gives McD the right to make such a decision unilaterally, other than might?
In fact, this is how disability legislation (for example) already tends to work. You don't get to tell disabled people to just go somewhere else, you have to make reasonable accomodations for them.
Apple's food scientists have verified the food safety of their new recipe, and they are sufficiently confident that nobody will suffer any allergic reaction. Nobody has disputed their assessment.
That doesn't stop consumers from engaging in Info Wars style paranoia, and grandstanding about the aforementioned paranoia.
You're seriously arguing that it's absurd for customers to have "absolute control" over all software?
No EU regulation could regulate away all "moral" concerns over software. More specifically, they EU could regulate, but the overwhelming majority of software companies would either strip significant features out for EU customers, or exit the market altogether.
There is significant middle ground between "do it without asking" and "ask about every single thing". A reasonable option would be "ask if the device can send anonymized data to Apple to enable such and such features". This setting can apply to this specific case, as well as other similar cases for other apps.
If you can't meaningfully explain what you're doing then you can't obtain informed consent. If you can't obtain informed consent then that's not a sign to go ahead anyway, it's a sign that you shouldn't do it.
I mostly agree. I'm just annoyed "this new privacy tech is too hard to explain" leads to "you shouldn't do it". This new privacy tech is a huge net positive for users.
Also: from other comments sounds like it might have been opt-in the whole time. Someone said a fresh install has it off.
> This new privacy tech is a huge net positive for users.
It's a positive compared to doing the same "feature" without the privacy tech. It's not necessarily a positive compared to not forcing the "feature" on the user at all.
The privacy tech isn't necessarily a positive as a whole if it leads companies to take more liberties in the name of "hey you don't need to be able to turn it off because we have this magical privacy tech (that nobody understands and may or may not actually work please don't look into it too hard)".
Notice is always good and Apple should implement notice.
However, "my data is being sent off my device" is incorrect, as GP explained. Metadata, derived from your data, with noise added to make it irreversible, is being sent off your device. It's the equivalent of sending an MD5 of your password somewhere; you may still object, but it is not factually correct to say your password was transmitted.
> It's the equivalent of sending an MD5 of your password somewhere; you may still object, but it is not factually correct to say your password was transmitted.
Hackers love to have MD5 checksums of passwords. They make it way easier to find the passwords in a brute force attack.
Nobody responding seriously to this because you seem to have missed the part where GP said "with noise added to make it irreversible" and the third sentence in that wikipedia article.
> However, "my data is being sent off my device" is incorrect, as GP explained. Metadata, derived from your data, with noise added to make it irreversible, is being sent off your device.
Sounds like my data is being sent off my device.
> It's the equivalent of sending an MD5 of your password somewhere
Well that's what you're told is happening. As it's all proprietary closed source software that you can't inspect or look at or verify in any manner, you have absolutely zero evidence whether that's what's actually happening or not.
If the information being sent from my advice cannot be derived from anything other than my own data then it is my data. I don't care what pretty dress you put on it.
When your phone sends out a ping to search for cellular towers, real estate brokers collect all that information to track everywhere you go and which stores you visit.
Owning a phone is a privacy failure by default in the United States.
You are being downvoted because you're so painfully correct. It's not an issue exclusive to the United States, but American intelligence leads the field far-and-away on both legal and extralegal surveillance. The compliance forced by US Government agencies certainly helps make data tracking inescapable for the average American.
Unfortunately, the knee-jerk reaction of many defense industry pundits (and VCs, for that matter) is that US intelligence is an unparalleled moral good, and the virtues of privacy aren't worth hamstringing our government's work. Many of these people will try to suppress comments like yours because it embarrasses Americans and American business by association. And I sympathize completely - I'm dumbfounded by the response from my government now that we know China is hacking our telecom records.
Do you consider your data to include non-reversible hashes of your data injected with random noise? I'm not sure I consider that my data. Its also not even really meta-data about my data.
I'm not sure I agree -- asking users about every single minor feature is (a) incredibly annoying, and (b) quickly causes request-blindness in even reasonably security-conscious users. So restraining the nagging for only risky or particularly invasive things makes sense to me.
Maybe they should lump its default state into something that already exists? E.g. assume that if you already have location access enabled for Photos (it does ask!), you've already indicated that you're okay with something about this identifying being sent to Apple whenever you take a picture.
My understanding is that Location Services will, among other things, send a hash of local WiFi network SSIDs and signal strengths to a database Apple maintains, and use that to triangulate a possible position for you. This seems loosely analogous to what's going on here with the compute-a-vector thing.
What's the Venn diagram of people who both (1) deliberately refrain from enabling iCloud Photos but nonetheless (2) want the Photos app to phone home to Apple in order to identify landmarks in locally stored photos?
It's probably a pretty large set of people, perhaps even the majority, since I'd suspect that most people don't pay for additional iCloud storage and can't fit their photo library into 5GB.
In fact, I'm willing to bet that if they'd added this feature and gated it behind iCloud Photos being enabled, we'd have different articles complaining about Apple making a cash grab by trying to get people to pay for premium storage. :P
> It's probably a pretty large set of people, perhaps even the majority
As the article notes, this new feature is so "popular" that neither Apple nor the Apple media have bothered to mention it. AFAICT it's not even in Apple's document listing all the new features of iOS 18: https://www.apple.com/ios/ios-18/pdf/iOS_18_All_New_Features...
True, but I don't see how that relates to anything? You asked for a hypothetical set of people who'd have iCloud Photos disabled but would accept metadata being sent to Apple for better search. I can't help you if you want to move the goalposts after I give you that.
Well, neither of us have any way of surveying the public about that, do we? My claim that those people would be okay with it has as much weight as yours that they wouldn't.
I can try to turn down my natural inclination towards cautious phrasing, if you'd like? Get us on the same level. :D
> It's probably a pretty large set of people, perhaps even the majority, since I'd suspect that most people don't pay for additional iCloud storage and can't fit their photo library into 5GB.
Large set? Yes. Majority? No. CIRP says 2/3 of US Apple users pay for iCloud storage[0]. It's this popular for the exact reason you mentioned. Almost no one can fit their photo library into 5GB so they opt in to the cheap 50GB for $0.99/month. 50GB is enough for a lot of people.
Huh, I'm honestly kind of surprised. Good to learn something!
Well, I'll take back what I said about the majority. I do still think that the remaining 1/3 of users who don't have enough storage to turn on iCloud Photos qualify as what lapcat was asking for, though.
Time Machine does not backup your desktop and other spots that might be essential in case of needing a backup. iCloud does.
I know users who would prefer not to trust Apple for anything, and only pay for and use iCloud to backup the Desktop [and similar locations]. If they were to hear that their opt-in for iCloud means that Apple starts copying random things, they would not be happy.
[OT, I use Arq. But admit that iCloud is simpler, and it is not apples to apples.]
IMO, the fact that Apple backs up your keychain to the Mothership; and that this is a "default" behavior that will re-enable itself when shut off, reflects an attitude that makes me very distrustful of Apple.
I'm sure you know that the point of that billboard is to state that your iPhone protects your privacy. That is generally true, Apple is by far the most privacy-focused major phone and software company. Advertising isn't literal, if we're going to be pedantic here the photons emitted by your iPhone's screen technically leave your iPhone and definitely contain private information.
If you one-way encrypt a value, and that value leaves the phone, with no way to recover the original value, then the original data never left the phone.
Especially for a company which heavily markets about how privacy-focused it is,
1)sending my personal data to them in any way is not a "feature." It's especially not a feature because what it sets out to do is rather unnecessary because every photo has geotagging, time-based grouping, and AI/ML/whatever on-device keyword assignments and OCR. I can open up my phone right now and search for every picture that has grass in it. I can search for "washington" and if I took a picture of a statue of george washington that shows the plaque, my iPhone already OCR'd that and will show the photo.
2)"minor" is not how I would ever describe sending data based off my photos to them, regardless of how much it's been stuffed through a mathematical meat grinder.
3)Apple is usually very upfront about this sort of thing, and also loves to mention the most minor, insignificant, who-gives-a-fuck feature addition in the changenotes for "point" system updates. We're talking things like "Numbers now supports setting font size in chart legends" (I'm making that up but you get the point.)
This was very clearly an "ask for forgiveness because the data we want is absolutely priceless and we'll get lots of it by the time people notice / word gets out." It's along the lines of Niantic using the massive trove of photos from the pokemon games to create 3d maps of everywhere.
I specifically use iOS because I value my privacy (and don't want my cell phone data plan, battery power, etc to be a data collection device for Google.) Sending data based off my photos is a hard, do-not-pass-go-fuck-off-and-die line in the sand for me.
It's especially shitty because they've gated a huge amount of their AI shit behind owning the current iPhone model....but apparently my several generation old iPhone is more than good enough to do some AI analysis on all my photos, to upload data for them?
> This was very clearly an "ask for forgiveness because the data we want is absolutely priceless and we'll get lots of it by the time people notice / word gets out.
It's very clearly not, since they've gone to huge lengths to make sure they can't actually see the data themselves see the grandparent post.
> It's especially shitty because they've gated a huge amount of their AI shit behind owning the current iPhone model....but apparently my several generation old iPhone is more than good enough to do some AI analysis on all my photos
Hear hear. As if they can do this but not Visual Intelligence, which is just sending a photo to their servers for analysis. Apple has always had artificial limitations but they've been getting more egregious of late.
I guess it depends on what you're calling "your data" -- without being able to reconstruct an image from a noised vector, can we say that that vector in any way represents "your data"? The way the process works, Apple makes their own data that leaves your device, but the photo never does.
It's the same as the CSAM initiative. It doesn't matter what they say they send, you cannot trust them to send what they say they send or trust them not to change it in the future.
Anything that leaves my devices should do so with my opt-IN permission.
I’m a cryptographer and I just learned about this feature today while I’m on a holiday vacation with my family. I would have loved the chance to read about the architecture, think hard about how much leakage there is in this scheme, but I only learned about it in time to see that it had already been activated on my device. Coincidentally on a vacation where I’ve just taken about 400 photos of recognizable locations.
This is not how you launch a privacy-preserving product if your intentions are good, this is how you slip something under the radar while everyone is distracted.
To play Apple's advocate, this system will probably never be perfect, and stand up to full scrutinity from everyone on the planet. And they also need the most people possible activated as it's an adverserial feature.
The choice probably looks to them like:
A - play the game, give everyone a heads up, respond to all feedback, and never ship the feature
B - YOLO it, weather the storm, have people forget about it after the holiday, and go on with their life.
Wether B works is up to debate, but that was probably their only chance to have it ship from their POV.
Yes. That also was a thoroughly botched version of A, but I think even a good version of A won't see them ship anything within this century.
IMO giving up on having it widely used and just ship it turned off would be the best choice. But it's so obvious, there must be other ceitical reasons (good or bad) that's not an option.
In engineering we distinguish the "how" of verification from the "why" of validation; it looks like much comments disagreement in this post is about the premise of whether ANY outgoing data counts as a privacy consent issue. It's not a technical issue, it's a premises disagreement issue and that can be hard to explain to the other side.
The premise of my disagreement is that privacy-preserving schemes should get some outside validation by experts before being turned on as a default. Those experts don’t have to be me, there are plenty of people I trust to check Apple’s work. But as far as I can tell, most of the expert community is learning about this the same way that everyone else is. I just think that’s a bad way to approach a deployment like this.
How would you explain client side vectorization, differential privacy and homomorphic encryption to a layman in a single privacy popup so that they can make an informed choice?
Or is it better to just trust that mathematics works and thus encryption is a viable way to preserve privacy and skip the dialog?
"Your data" is not actually being sent off your device, actually, it is being scrambled into completely unusable form for anyone except you.
This is a much greater level of security than what you would expect from a bank, for example, who needs to fully decrypt the data you send it. When using your banking apps over HTTPS (TLS), you are trusting the CA infrastructure, you are trusting all sorts of things. You have fewer points of failure when a key for homomorphic encryption resides only on your device.
The right call is to provide the feature and let users opt-in. Apple knows this is bad, they've directly witnessed the backlash to OCSP, lawful intercept and client-side-scanning. There is no world in which they did not realize the problem and decided to enable it by default anyways knowing full-well that users aren't comfortable with this.
People won't trust homomorphic encryption, entropy seeding or relaying when none of it is transparent and all of it is enabled in an OTA update.
> This is what a good privacy story looks like.
This is what a coverup looks like. Good privacy stories never force third-party services on a user, period. When you see that many puppets on stage in one security theater, it's only natural for things to feel off.
It's not that binary. Nobody is forcing anything, you can not buy a phone, you can not use the internet. Heck, you can even not install any updates!
What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. Plenty of people might just 'go with the flow', but putting what Apple did here in the same bucket as what for example Microsoft or Google does is a gross misrepresentation. Present it all as equals just kills the discussion, and doesn't inform anyone to a better degree.
When you want to take part in an interconnected network, you cannot do that on your own, and you will have to trust other parties to some degree. This includes things that might 'feel' like you can judge them (like your browser used to access HN right here), but you actually can't unless you understand the entire codebase of your OS and Browser, all the firmware on the I/O paths, and the silicon it all runs on. So you make a choice, which as you are reading this, is apparently that you trust this entire chain enough to take part in it.
It would be reasonable to make this optional (as in, opt-in), but the problem is that you end up asking a user for a ton of "do you want this" questions, almost every upgrade and install cycle, which is not what they want (we have had this since Mavericks and Vista, people were not happy). So if you can engineer a feature to be as privacy-centric yet automated as possible, it's a win for everyone.
> What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with.
People aren't making tradeoffs - that's the problem. Apple is making the tradeoffs for them, and then retroactively asking their users "is this okay?"
Users shouldn't need to buy a new phone to circumevent arbitrary restrictions on the hardware that is their legal property. If America had functional consumer protections, Apple would have been reprimanded harder than their smackdowns in the EU.
People make plenty of tradeoffs. Most people trade most of their attention/time for things that are not related to thinking about technical details, legal issues or privacy concerns. None of this exists in their minds. Maybe the fact that they implicitly made this tradeoff isn't even something they are aware of.
As for vectorised and noise-protected PCC, sure, they might have an opinion about that, but people rarely are informed enough to think about it, let alone gain the insight to make a judgment about it at all.
I don't want my photos to take part in any network; never asked for it, never expected it to happen. I never used iCloud or other commercial cloud providers. This is just forceful data extraction by Apple, absolutely egregious behavior.
The "network" mention was in reply to your comment about "participating in a network" which was never the case for one's personal photos (unless explicitly shared on a social network I guess).
I did read the article, yes :) Maybe our photos are not sent bit-by-bit but enough data from the photos is being sent to be able to infer a location (and possibly other details) so it is the same thing: my personal data is being sent to Apple's servers (directly or indirectly, partially or fully) without my explicit consent.
At least the last time they tried to scan everyone's photos (in the name of the children) they pinky promised they'd only do it before uploading to iCloud, now they're doing it for everyone's photos all the time - it's disgusting.
No, your photos aren't sent, also not 'pieces' of it. They are creating vector data which can be used to create searchable vectors which in turn can be used on-device to find visual matches for your search queries (which are local).
You can imagine it as hashes (created locally), some characters of that hash from some random positions being used to find out if those can be turned into a query (which is compute intensive so they use PCC for that). So there is no 'data' about what it is, where it is or who it is. There isn't even enough data to create a picture with.
Technically, everything could of course be changed, heck, Apple could probably hire someone with binoculars and spy on you 24/7. But this is not that. Just like baseband firmware is not that, and activation is not that, yet using them requires communication with Apple all the same.
My understanding as the blog laid it out was that the cloud service is doing the vector similarity search against a finite database of landmark feature vectors, but they are performing that mathematical function under homomorphic encryption such that the result of the vector comparison can only be read with a key that never left your device, so it's just adding a tag "Eiffel tower" that only you see, but the feature vector is sent off device, it's just never able to be read by another party.
Yep. It's essentially an implementation of remote attestation "the other way around". Normally the edge device is untrusted and needs to attest a variety of things before compute is done and the result is accepted, but PCC is the other way where the edge device holds the keys (technically octagon works that out, but it's backed by the on-device SEP).
So it does it multiple ways:
- Finite sets and added noise, doesn't hurt performance too much but does make it nearly impossible to ID/locate a photo
- Encryption at rest and in transit
- Transit over hops they don't own
- Homomorphic Encryption during remote compute
The data it finds was available in two ways: the "result" and the vector embedding. Not sure which one you end up consuming since it also has to work on older models that might not be able to load the embeddings and perform adequately, but it doesn't matter since the data itself will be unique so you can't do parallel reconstruction, but it is also uniquely meaningless to anyone without a key. They are essentially computing on needles that aren't in a haystack, but in a needle stack.
The primitives all this is built on have been around for quite a while, including their HBONE implementation, the cryptographically hidden data distribution and the SEP. So far, it has been the only one of its kind outside of disjointed options like buying and operating your own HSM, a large TOR network and a yet to-be-invented self-hosted PCC solution (AMD was supposed to release something but they failed at that, just not as bad as Intel messed up with SGX).
Technically, even with everything else removed, just some good TLS 1.2+ and homomorphic encryption would have been more than any other mass market manufacturer has ever done in an effective way. But by adding the additional factors such as degrees of separation so they couldn't get in themselves (without breaking it for everyone in the process) is what makes this so much more robust.
That is incorrect. If everything was local they wouldn't need HE and OHTTP and everything else.
I would be ok with this being a local feature, where I can download the signature database to my device and run the search locally (as you say), but as it stands some information about my photos (enough to detect places at least, possibly more in the future) is being sent out of my device. I want zero information about my photos to leave my device.
> Just like baseband firmware is not that, and activation is not that, yet using them requires communication with Apple all the same.
I mean, this is just wrong. Baseband firmware and carrier activation can be managed entirely independently of Apple, they just choose to manage it themselves. The number of places where Apple chooses to insert their own services as arbitrary middlemen has been a perennially worrying topic among Apple enthusiasts. It's not just disrespectful to people that pay a premium for fewer service advertisements, it's downright unsafe and does not reflect the sort of forward-thinking security that people in the industry respect.
There was a time when Apple focused on real and innovative product differentiation, but I'll be damned if you can give me a post-Wozniak example that isn't under antitrust scrutiny. Apple relies on marketing and branding to make people feel unsafe in a fundamentally insecure system - I don't respect that as a proponent of innovation and competitive digital markets.
Baseband firmware and OS activation have nothing to do with the carrier, just like it didn't on RIM devices back in the day (which is probably the only somewhat comparable version of this).
Perhaps you are thinking about subscription activation (be it GSM or CDMA) and parameters for cell networks (which can indeed be consumed by the baseband, which will be running firmware supplied by the manufacturer, sometimes re-packaged in system images as done in OEM feature phones and many android phones).
Either way, macOS devices do the same thing (activation) as do iPads without cell networking. Same goes for radio firmware loading and updates. You'll find most wintel laptops doing the same for things like WiFi (regardless of softmac/halfmac/hardmac chips).
That’s starting to veer into unreasonable levels of conspiracy theory. There’s nothing to “cover up”, the feature has an off switch right in the Settings and a public document explaining how it works. It should not be on by default but that’s not a reason to immediately assume bad faith. Even the author of the article is concerned more about bugs than intentions.
It is a coverup. Apple is overtly and completely aware of the optics surrounding photo scanning - they know that an opt-in scheme cannot work as they found out previously.
Since they cannot convince users to enable this feature in good-faith, they are resorting to subterfuge. We know that Apple is vehement about pushing client-side scanning on users that do not want it, I do not believe for a second that this was a mistake or unintended behavior. If this was a bug then it would have been hotfixed immediately to prevent the unintended behavior from reaching any more phones than it already had.
Exactly. It is absolutely bonkers that people are claiming that Apple is trying to cover up something for which they have a settings toggle and public documentation.
Yeah, we might quibble about what the default value of the toggle should be, but them adding settings for minor features like this is absolutely a good thing, and very much a sign that they're not trying to hide things.
If anything, the lesson Apple might take from this could be "adding the settings toggle was a bad idea, because nobody would have cared about this at all otherwise".
That is not the point at all and you either didn’t try to understand one iota of it or are outright arguing in bad faith.
I am not claiming for one moment that enabling this by default is OK. In fact, I have explicitly said it is not.
What I am saying is that it is ignorant to call this a cover up, because a cover up requires subterfuge. This feature has a freaking settings toggle and public documentation. Calling it a cover up is the type of uneducated rhetoric that makes these issues being brushed off by those in power as “it’s just a bunch of loonies conspiracy theorists complaining”.
Got it, so if Windows Defender (that is enabled by default on all Windows PCs) pushes an update that scans all your files on all connected drives and uploads hashes to the mothership, enables this by default and proceeds to execute the scan and upload immediately after update, but also includes a setting that lets you turn it off when you find out about its existence from some 3rd party article, that is all perfectly fine? (since there is no subterfuge)
> the feature has an off switch right in the Settings and a public document explaining how it works
Irrelevant.
This is Apple's proprietary software, running on Apple's computers, devices which use cryptography to prevent you from inspecting it or running software they don't control. Very few people have any idea how it actually works or what it actually does.
That there's some "public document" describing it is not evidence of anything.
> that’s not a reason to immediately assume bad faith
The mere existence of this setting is evidence of bad faith. The client side scanning nonsense proved controversial despite their use of children as political weapons. That they went ahead and did this despite the controversy removes any possible innocence. It tells you straight up that they cannot be trusted.
> Even the author of the article is concerned more about bugs than intentions.
Why wouldn't this mistake be addressed in a security hotfix then? Apple has to pick a lane - this is either intended behavior being enabled against user's wishes, or unintended behavior that compromises the security and privacy of iPhone owners.
If it's a useful, harmless feature, then let users opt-in. You'd think Apple would have learned their lesson after a decade of people bitching about U2 being secreted-in to their iTunes library, but apparently not.
Users are smart enough to decide for themselves. Let them.
All I've seen this year is people complaining about all the security dialogs Apple are throwing at them when they upgrade macOS ARE YOU SURE ARE YOU SURE ARE YOU SURE
Quantum makes the homomorphic stuff ineffective in the mid-term. All they have to do is hold on to the data and they can get the results of the lookup table computation, in maybe 10-25 years. Shouldn't be on by default.
What makes you think that this is the biggest problem if things like AES and RSA are suddenly breakable?
If someone wanted to get a hold of your cloud hosted data at that point, they would use their capacity to simply extract enough key material to impersonate a Secure Enclave. That that point, you "are" the device and as such you "are" the user. No need to make it more complicated than that.
In theory, Apple and other manufacturers would already use PQC to prevent such scenarios. Then again, QC has been "coming soon" for so long, it's doubtful that any information that is currently protected by encryption will still be valuable by the time it can be cracked. Most real-world process implementations don't rely on some "infinite insurance", but assume it will be breached at some point and just try to make it difficult or costly enough to run out the clock on confidentiality, which is all that really matters. Nothing that exists really needs to be confidential forever. Things either get lost/destroyed or become irrelevant.
> The author themselves looks to be an Apple security researcher
They’re not. Jeff Johnson develops apps (specifically Safari extensions) for Apple platforms and frequently blogs about their annoyances with Apple, but they’re not a security researcher.
This must be the first consumer or commercial product implementing homomorphic encryption is it not?
I would be surprised if doing noisy vector comparisons is actually the most effective way to tell if someone is in front of the Eiffel tower. A small large language model could caption it just as well on device, my spider sense tells me someone saw an opportunity to apply bleeding edge, very cool tech so that they can gain experience and do it bigger and better in the future, but they're fumbling their reputation by doing this kind of user data scanning.
> This must be the first consumer or commercial product implementing homomorphic encryption is it not?
Not really, it's been around for a bit now. From 2021:
> The other major reason we’re talking about HE and FL now is who is using them. According to a recent repository of PETs, there are 19 publicly announced pilots, products, and proofs of concept for homomorphic encryption and federated analytics (another term for federated learning) combined. That doesn’t seem like a lot … but the companies offering them include Apple,7 Google, Microsoft, Nvidia, IBM, and the National Health Service in the United Kingdom, and users and investors include DARPA, Intel, Oracle, Mastercard, and Scotiabank. Also, the industries involved in these early projects are among the largest. Use cases are led by health and social care and finance, with their use in digital and crime and justice also nontrivial (figure 1).
I do wonder why we don't hear about it more often though. "Homomorphic encryption" as a buzzword has a lot of headline potential, so I'm surprised companies don't brag about it more.
But what are the products from them that implement HE and that consumers are using? Microsoft, IBM, Intel, and Google have all released libraries for HE, and there's Duality SecurePlu, but as far as actual consumer products, Apple's caller ID phone number lookup and other features in iOS 18 is very possibly the first.
As far as why it's not more of a buzzword, it's far too in the weeds and ultimately consumers either trust you or they don't. And even if they don't trust you, many of them are still going to use Apple/Google/Facebook system anyway.
> - OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library).
I don't have a list on hand, but at least Cloudflare and Akamai are part of the network hops. Technically you only need 2 hops to make sure no origin or data extraction can be done.
a) Cloudflare doesn't know about you. It sees an IP address.
b) If we follow your tortured logic then every hop along the path from your phone to Apple will have one more data point on you. That's thousands of companies a day.
I'm just griping that cloudflare has many eyeballs and sees most of the traffic on the internet at this point. How many websites have I used with Cloudflare DDoS protection that checks if I'm a bot by fingerprinting my device? They know plenty about me.
I'm also griping that "the data is encrypted !" is not a good enough excuse seeing as how we've known for years that the metadata is a bigger pot of gold for intelligence agencies. That my mac address is taking a photo and hitting a particular cell tower is pretty detailed information, even without knowing the content of the photo.
Regarding HE: since the lookup is generated by the requestor, it can be used as an adversarial vector, which can result in exfiltration by nearest neighbor (closest point to vector) methods. In other words, you can change what you are searching for, and much like differential power analysis attacks on crypto, extract information.
You're presenting a false dichotomy between "perfect user understanding" and "no user choice." The issue isn't whether users can fully comprehend homomorphic encryption or differential privacy – it's about basic consent and transparency.
Consider these points:
1. Users don't need a PhD to understand "This feature will send data about your photos to Apple's servers to enable better search."
2. The complexity of the privacy protections doesn't justify removing user choice. By that logic, we should never ask users about any technical feature.
3. Many privacy-conscious users follow a simple principle: they want control over what leaves their device, regardless of how it's protected.
The "it's too complex to explain" argument could justify any privacy-invasive default. Would you apply the same logic to, say, enabling location services by default because explaining GPS technology is too complex?
The real solution is simple: explain the feature in plain language, highlight the benefits, outline the privacy protections, and let users make their own choice. Apple already does this for many other features. "Default off with opt-in" is a core principle of privacy-respecting design, regardless of how robust the underlying protections are.
I don't believe I said or implied that anywhere: 'You're presenting a false dichotomy between "perfect user understanding" and "no user choice."'? Happy to be corrected if wrong.
Closest I come to presenting an opinion on the right way UX was "I'm not sure what the right call is here.". The thing I disagreed with was a technical statement "the only way to guarantee computing privacy is to not send data off the device.".
Privacy respecting design and tech is a passion of mine. I'm pointing out "user choice" gets hard as the techniques used for privacy exceed the understanding of users. Users can intuitively understand "send my location to Google [once/always]" without understanding GPS satellites. User's can't understand the difference between "send my photo" and "send homomorphicly encrypted locally differentially private vector of e=0.8" and "send differentially private vector of e=50". Your prompt "send data about your photos..." would allow for much less private designs than this. If we want to move beyond "ask the user then do it", we need to get into the nitty gritty details here. I'd love to see more tech like this in consumer products, where it's private when used, even when opted-in.
The choice is between "use an online service" or "don't use an online service". That's simple enough for anyone to understand.
Apple can try to explain as best it can how user data is protected when they use the online service, and then the user makes a choice to either use the service or not.
In my case, I have don't even have a practical use for the new feature, so it's irrelevant how private the online service is. As it is, though, Apple silently forced me to use an online service that I never wanted.
I appreciate your passion for privacy-respecting technology and your clarification. You make good points about the nuances of privacy-preserving techniques. However, I think we can separate two distinct issues:
1. The technical excellence of Apple's privacy protections (which you've explained well and seem robust)
2. The ethical question of enabling data transmission by default
Even with best-in-class privacy protections, the principle of user agency matters. A simplified prompt like "This feature will analyze your photos locally and send secure, anonymized data to Apple's servers to enable better search" would give users the basic choice while being technically accurate. The technical sophistication of the privacy measures, while commendable, doesn't override the need for informed consent.
This is not a matter of respect, it is a matter of ethics. Otherwise you will just end up rationalizating technocratic, unethical technology. No amount of passion will justify that.
> "It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here
I don’t think that that’s what the author is saying at all, I think he’s saying that Apple should let the user decide for themself if they want to send all this shit to Apple, freedom for the individual. They’re not saying “I dunno”
So what? Why should the application talk over the Internet to begin with? And why isn't that functionality off by default under a settings option that clearly warns the user of the consequences? I think you're missing the forest for the trees here.
And the claims that this is good privacy/security are not at all obvious either. And who are those third-parties anyway? Did you verify each one of them?
> if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this.
Well I want them to fuck off.
Hidden in your commentary here is the fact that the vector representation of the image is _the contents of the image_. It very well may be that they cannot reverse the exact image. But it’s still a representation of the image that has to be good for something. Without being too familiar I would be willing to hazard a guess that this could include textual labels and classifications of what is in the image.
I don’t give a shit how good your internal controls are. I don’t have anything particularly interesting to hide. I still do not want you taking my pictures.
If you read the research you'd know that they don't have access to the vector either. They never decrypt the data. All operations on their server are done directly on the encrypted data. They get 0 information about your photos. They cannot even see which landmark your vector was closest to.
I don’t care! I do not want them helping themselves to representations of my data. I don’t care if it’s encrypted or run through a one way hash. I don’t care if they only interact with it via homomorphic methods. They can, again, fuck the hell off.
A private corporation has no business reading my files for its own benefit.
> A private corporation has no business reading my files for its own benefit.
that's what i'm saying!! because of all these special privacy measures it's genuinely impossible for them to get any benefit from this. it is only used for your benefit to improve photo search on your local device.
they don't see any of the data, they simply perform computations blindly over the encrypted data that are then returned to your device to then decrypt the results locally and use them to improve local photo indexing for you.
I’m deeply familiar with all of these techniques, the core issue here is informed consent which they have not obtained.
Furthermore, Apples privacy stance is generally a sham as their definition of “human rights” doesn’t extend to China. Which either means Apple doesn’t respect human rights, or they don’t view Chinese people as human.
That's not really fair; Apple's in a sticky wicket when it comes to the Chinese government, and they're not the only ones.
The Chinese government are debatably inhuman. They've literally censored the word "censorship." (Then they censored what people used euphemistically for censorship--"harmonious.") It's funny from the outside but also a miserable state of affairs in 2024.
It’s very fair, Apple has historically been very happy to be the sponsor of horrible human rights violations in their supply chain, only marginally paying attention to suicides in their factories when the PR got too bad.
Apples version of “human right” includes suicide nets as an alternative to treating people humanely. That’s why their stance is pure marketing - they have blood on their hands.
And guess what? You can’t use Google in China, and while Google isn’t by any means perfect, they aren’t Apple.
Oh come on, it’s not like Google is better. 1. Google isn’t available in China not because of some moral reason, and they were in fact available in the past and Google has attempted to go back to China as a search engine, etc. before. They aren’t available because China hacked into their systems and took code and got access to certain accounts, and at the time Google essentially decided it wasn’t worth offering services there. There’s no reason Google wouldn’t start working in China again on a moral level. 2. Google works directly with the US Department of Defense, so as much as Apple has blood on their hands, so does Google.
A) Google refused to censor in the mainland and that’s why they got blocked, B) Google employees had a walkout when they tried to go back to China. C) I’ve never seen an Apple employee protest anything.
Now, that happened under a different type of Google (I’m sure they’ll go back now that they torched the culture), but Apple employees who walk around SV like they are better than everybody else - which people on their privacy teams very much do - are pathetically deluded.
All the megacorps are evil, and while Apple likes to put on a holier-than-thou act, it’s just as much bullshit for them to claim they value human rights as it is for Google to say they value privacy.
Apple follows the law. First you need to get the Chinese government to respect those rights. The only other choice is to stop doing business entirely in the country.
A choice many companies have made. Apple is in China to make money, which is what a corporation is set up to do. My point is them claiming the moral high ground of a human rights defender is utterly laughable bullshit.
The best you can hope is integrity and security until your information reaches the destination but to assume that Apple or the U.S government cannot decipher the information you sent it or use it against you(i.e. set a person of interest as "landmark" and find out who's iPhone matches that "landmark) you must be foolish.
It's no longer a conspiracy. I think we are all over past that time(i.e with Snowden and Wikileaks). We live in a surveillance world and "They're guarding all the doors and holding all the keys".
>There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk.
It is simple for any user to evaluate risk the risk of their data being breached on 3rd party servers when their data isn't being sent off the device - there is none. It is only when corporations insist that they are going to send the data off your device whether you like it or not that evaluating risk becomes necessary.
Sure, but if we follow that line of thinking to its logical conclusion, we must move to a cabin in the woods, 100 miles from the nearest civilization, growing our own food and never connecting our computing devices to anything resembling a network.
No? You can have a photos app that doesn't phone home while not having to move to a cabin in the woods. See: every photos app that doesn't phone home, and I currently don't live in a cabin in the woods.
I've read the post you're responding to like 3 times, and after pondering it deeply, I'm pretty sure the conclusion of their line of thinking pretty definitively stops at "Apple should not be sending data off the device without the user requesting it." If you think otherwise, you should maybe provide more of an argument.
The line of thinking is right there: "not sending any info to anyone else anywhere at any time"
There are way more egregious privacy concerns than sending non-reversibly encrypted noisy photos to Apple. Why draw the line here and not the far worse things happening on your phone and computer right now?
Almost every single app today interacts with the network in some way.
You would be constantly annoying the user with prompt after prompt if you wanted to get consent for sending any relatively harmless data off the device.
FWIW, a "Network Access" app permission is one of the features that GrapheneOS provides. It is only setting offered to the user every single app install. It should be in base AOSP, and I have to wonder why it isn't already.
The initiative is for the user to command their computer to communicate or not with the information of their choosing.
"Computer, I command thee to send this and only this information over the channel of my choosing, using following encryption scheme, for here be my seal of approval for anyone who might want to verify, and here be the key"
I understand the enthusiasm but from the business perspective it does not matter. Many businesses would fail if they go too deep on this. Their only audience would be people who are experts in the area. Other people are confused and disappointed since things are not working as they expect.
On Apple's scale, most people care about the things they can do, not about how it happens. For that reason, default matters when the option is only about the internal process pipeline and privacy.
As a result, it is enough to showcase that in case some expert investigates the matter, they show that privacy is considered in a reasonable level.
Maybe some day in the future these things are common knowledge, but I fear that the knowledge gap just increases.
If you think that sending data to a remote server is equally private to not sending it, then you are the one who doesn't know what privacy means.
Of course it's fine to not desire privacy, or to desire a privacy level that is less than private. That's up to you. I liked the privacy of my old Canon digicam that had no internet. A photo app on a phone that sends stuff over the network might bring some useful functionality in return, but it can only be considered a regression in terms of privacy.
Thank you for this comment. I found the author's ignorance to be fairly discrediting, and was surprised to find so many follow up comments equally railing on Apple.
Between the quote you pointed out and:
"One thing I do know, however, is that Apple computers are constantly full of privacy and security vulnerabilities, as proved by Apple's own security release notes"
which just reeks of survivorship bias.
I think the final call of what is right here _shouldn't_ be informed by the linked article.
IMO, enabled by default without opt-in is absolutely the right call when judging between 1: Feature value 2: Security risk 3: Consent Fatigue.
If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.
If you disagree with my prior statement because of how Apple locks you into Photos, :shake_hands:.
If Enhanced Visual Search is still enabled by default in lockdown mode, then I think that's worth a conversation.
> I found the author's ignorance to be fairly discrediting
Why in the world am I supposed to be an expert on homomorphic encryption? How many people in the world are experts on homomorphic encryption?
> which just reeks of survivorship bias.
What does that even mean in this context?
> 1: Feature value
What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it.
> If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.
That's ridiculous. Apple itself has said, "Lockdown Mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature." https://support.apple.com/105120
Lockdown mode is basically for famous people and nobody else.
> Why in the world am I supposed to be an expert on homomorphic encryption? How many people in the world are experts on homomorphic encryption?
No one, at any point, implied you had to be an expert on homomorphic encryption. But if you're going to evaluate the security risk of a feature, and then end up on the front page of HN for said security risk, I think it's fair to criticize your lack of objectivity (or attempt at objectivity) by way of not even trying to understand the technical details of the blog.
I will say I think my word choice was unnecessarily harsh, I'm sorry. I think I meant more indifference/inattention.
> What does that even mean in this context?
Apple's list of Security releases is long and storied. By comparison, the Solana Saga Web3 phone's list of security releases is short and succinct. Therefore, the Solana Saga must be more secure and has better security than an Apple device!
> What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it.
The marketability of a feature is not necessarily correlated with its value. Some features are simply expected and would be silly to advertise, i.e. the ability to check email or text friends. Other features are difficult to evaluate efficacy, so you release and collect feedback instead of advertising and setting false expectations.
> Lockdown mode is basically for famous people and nobody else.
Similar to Feature value, that audience of that statement is your average person (read: does not read/post on hacker news). Based off the your pedigree, I feel as though you probably know better, and given your "no tolerance for risk" for such a feature, it's something worth at least considering, and definitely isn't ridiculous.
I think it's great you started this conversation. I disagree with your opinion, and that's okay!! But I don't think it's particularly beneficial to any discourse to
1. Imply that you are evaluating security risk
2. Be given a well written technical article so that you are able to make an informed decision (and then share that informed decision)
3. Ignore relevant information from said article, make an uninformed decision
4. Be surprised when someone says you made an uninformed decision
5. Imply the only way to make an informed decision would be to be an expert in the relevant fields from the technical article
Anyway - thanks for writing and replying. Creating and putting yourself out there is hard (as evidenced by my empty blog that I promised I'd add to for the past 2 years). And my criticism was too harsh.
> if you're going to evaluate the security risk of a feature
I wouldn't characterize that as the point of my blog post. It's primarily about user consent, or lack thereof.
> and then end up on the front page of HN for said security risk
I have no control over that. I didn't even submit the article to HN.
> Apple's list of Security releases is long and storied. By comparison, the Solana Saga Web3 phone's list of security releases is short and succinct. Therefore, the Solana Saga must be more secure and has better security than an Apple device!
This is a red herring. I wasn't comparing Apple security to any other company's security. I was merely pointing out the possibility of bugs and vulnerabilities in Apple's new feature.
> Other features are difficult to evaluate efficacy, so you release and collect feedback instead of advertising and setting false expectations.
Well, I've now given my feedback on the new feature.
> Similar to Feature value, that audience of that statement is your average person (read: does not read/post on hacker news). Based off the your pedigree, I feel as though you probably know better
I'm not sure I understand. Are you claiming that Apple, in its support document, is deliberately mischaracterizing Lockdown Mode?
> But I don't think it's particularly beneficial to any discourse to 1. Imply that you are evaluating security risk
As I've said above, I wasn't.
> 3. Ignore relevant information from said article
I didn't ignore the relevant information from said article. I read the article, but some of the technical details are beyond my current knowledge.
> make an uninformed decision
What uninformed decision are you talking about?
> 4. Be surprised when someone says you made an uninformed decision
I'm surprised because I have no idea what "uninformed decision" you mean.
> 5. Imply the only way to make an informed decision would be to be an expert in the relevant fields from the technical article
I didn't imply that at all. To the contrary, I insisted that the decision to enable the feature should be up to the user, not up to Apple.
I don't think you're trying to understand what I'm saying, e.g.
> 3. Ignore relevant information from said article
I didn't ignore the relevant information from said article. I read the article, but some of the technical details are beyond my current knowledge.
> make an uninformed decision
What uninformed decision are you talking about?
I don't think I need to specify that by uninformed decision I mean evaluating the security risk of the feature. I think I criticized too harshly, and you're (understandably) not engaging with me fairly in retaliation. If you actually want to engage with me and discuss this further, feel free to shoot me an email (in my about section). Otherwise, obligatory https://www.paulgraham.com/vb.html.
Enhanced Visual Search was enabled despite my default lockdown mode. I worry about enhanced visual search capabilities much less than several of the other risky features that lockdown mode disables, but was a bit surprised by the default opt-in in my lockdown mode phone.
This sounds exactly like that CSAM "feature" they wanted to add but created a huge outrage because of how incredibly invasive it was.
It sounds like it only needs a few extra lines of code to get exactly what they wanted before, they just packaged it differently and we all fell for it like frogs getting boiled in water.
The CSAM filtering was a best of class implementation. I'm pretty sure I'm one of maybe a dozen people who actually read the spec before throwing a hissy-fit about "muh privacy!"
The only actual "flaw" was that maybe a state-level actor could make it scan for bad stuff on your device.
BUT they can do it to your cloud data _today_. And if you disabled cloud uploads, the local scanning was disabled too.
how about they don't send anything about my photos to their servers and i get to keep my shit on my own device
i suppose we're past that to the point where techbros like you will defend personal data exfiltration because.. uhh idk? trillion dollar corporation knows best?
The right call is to never send any data from the device to anyone unless the user explicitly tells the device to do it.
The only thing the device should do is whatever its user tells it to do.
The user didn't tell it to do this. Apple did.
> But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true
Irrelevant. It was never about privacy to begin with. It was always about power, who owns the keys to the machine, who commands it.
Vectorization, differential privacy, relays, homomorphic encryption, none of it matters. What matters is the device is going behind the user's back, doing somebody else's bidding, protecting somebody else's interests. That they were careful about it offers little comfort to users who are now aware of the fact "their" devices are doing things they weren't supposed to be doing.
Complete nonsense. *All networked devices do things behind their users back* at this point, and have for years, and do not ask for consent for most of it. And users would REJECT granular opt-in as a terrible UX.
Let's look at the primary alternative, Android. It generally does not provide you this level of granular control on network access either without rooted hacks. Apps and the phone vendor can do whatever they want with far less user control unless you're a deep Android nerd and know how to install root-level restriction software.
Yes, apps going behind people's back and exfiltrating personal information has become normal. That's not an argument, it's merely a statement of fact. This shouldn't be happening at all. The fact it got to this point doesn't imply it shouldn't be stopped.
No one's advocating for granular opt in either. There are much better ways. We have to make it so that data is toxic to corporations. Turn data into expensive legal liabilities they don't want to deal with. These corporations should not even be thinking about it. They should be scrambling to forget all about us the second we are done with them, not covertly collecting all the data they possibly can for "legitimate" purposes. People should be able to use their computers without ever worrying that corporations are exploiting them in any way whatsoever.
The Android situation is just as bad, by the way. Rooting is completely irrelevant. You may think you can hack it but if you actually do it the phone fails remote attestation and the corporations discriminate against you based on that, usually by straight up denying you service. On a very small number of devices, Google's phones ironically, you can access those keys and even set your own. And it doesn't matter, because the corporations don't trust your keys, they only trust the keys of other corporations. They don't care to know your device is secure, they want to know it's fully owned by Google so that you can't do things the corporations don't like.
It's not something that can be solved with technology. Computer freedom needs to become law.
You aren't wrong, but... it's odd coming here to HN and seeing people talk about privacy like we aren't in the nth generation of people trading theirs away for a pittance. I think the market for the sort of privacy envisioned by some here is incredibly small, incredibly niche, and honestly one of the least likely to buy an iPhone in the first place.
Most people broadcast their lives on social media, happily opt in to all sorts of schemes that track them just for minor conveniences. For people like that, the idea that the privacy protection outlined by the OP isn't enough rings really hollow.
Or to put it bluntly, at some point this really stops feeling like a practical debate, and more of an ideological one.
I have an idea: send an encrypted, relayed, non-reversible, noised vector representation of your daily phone habits and interactions. That way you can be bucketed, completely anonymously of course, with other user cohorts for tracking, advertising, and other yet-to-be discovered purposes.
It's a great privacy story! Why would you have a problem with that?
What would be the value to the user in your scenario? In the photos app real scenario, it’s to enable a search feature that requires pairing photos with data not on the phone. (I understand you’re being sarcastic.)
That doesn’t make sense, and the other user is right that you can’t give up personal data with this scheme. Perhaps focus on the real privacy leaks from cell phones like tower connections and sign-ins to Instagram.
They don't "have your data," even at an aggregated and noised level, due to the homomorphic encryption part.
Restating the layers above, in reverse:
- They don't see either your data or the results of the query (it's fully encrypted even from them where they compute the query -- this is what homomorphic encryption means)
- Even if they broke the encryption and had your query data / the query result, they don't know who "you" are (the relay part)
- Even if they had your query hash and your identity, they couldn't reverse the hash to identify which specific photos you have in your library (the client-side vectorization + differential privacy part), though by the this point they could know what records in the places database were hits. So they could know that you took a photo of a landmark, but only if the encryption and relay were both broken.
I am bit bit confused: Data is being sent to Apple, in such a way that it can not be traced back to the user. Apple does some processing on it. Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are.....
There is a way to perform processing on encrypted data so the result is also encrypted and the person doing the processing never knows anything about the data that was processed on or the result (which can only be decrypted by the user with the original encryption keys)
> Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first having to decrypt it. The resulting computations are left in an encrypted form which, when decrypted, result in an output that is identical to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and outsourced to commercial cloud environments for processing, all while encrypted
And the way the data comes back to you is via the third-party relay which knows your IP but nothing else
Ok, that's the step that was missing. I couldn't figure out how there was a benefit to the users without data being fed back and data can't be fed back without knowing some ID.
So, while Apple doesn't know the ID of the person sending the data, they have a 'room number' that links back to an ID.
If Apple were to decide to scan photos for pictures of 'lines of white powder' they couldn't tell the police your name but they could say that the 3rd party knows who you are.
The nearest neighbour search is sharded, which apple's blog admits is a privacy issue, which is why they're running the DP and OHTTP parts.
If apple were to add additional clusters that match "sensitive" content and endeavour to put them in their own shards distinct from landmarks, they defeat the homomorphic encryption part while still technically doing it.
The DP part can be defeated with just statistics over time; someone with any volume of sensitive content will hit these sensitive clusters with a higher likelihood than someone generateing noise injected fake searches.
The OHTTP part can be defeated in several ways, the simplest of which is just having a clause in a non-public contract allowing apple to request logs for some purpose. They're paying them and they can make up the rules as they go.
I'm moving my family out of apple photos, self hosted options have come a long way. I landed on immich [0] and a caddy plugin that allows for PKI certificate for account access while still allowing public shared URLs [1]*
There's also LibrePhotos which is packed with features but doesn't have as much polish as immich. They do however have a list of python libraries that can be used for offline/local inference for things like having an image model create a description of a photo that can benefit the full text search. [2]
Sure, I use Ente and have iCloud Photos turned off, but TFA is referring to what happens in the Apple Photos app, which happens even if you have iCloud Photos turned off.
i like to self host but i’d never do it for something critical like photos. what if you’re hit by a bus next month? Then your family (probably not tech savvy) need to figure out how to migrate all their stuff to a commercial offering - while dealing with all the emotional stuff after death. it’s just added stress for your loved ones.
100% private is nice but not practical for something like photos in my opinion. unless you have someone in the family that can take over after you’re gone and is very technical.
> On macOS, I can usually prevent Apple software from phoning home by using Little Snitch. Unfortunately, Apple doesn't allow anything like Little Snitch on iOS.
On Android, NetGuard uses a "local VPN" to firewall outgoing traffic. Could the same be done on iOS, or does Apple network traffic bypass VPNs? Lockdown mentions ads, but not Apple servers, https://lockdownprivacy.com/.
Apple does publish IP ranges for different services, so it's theoretically possible to block 17.0.0.0/8 and then open up connections just for notifications and security updates, https://support.apple.com/en-us/101555
>On Android, NetGuard uses a "local VPN" to firewall outgoing traffic. Could the same be done on iOS, or does Apple network traffic bypass VPNs? Lockdown mentions ads, but not Apple servers, https://lockdownprivacy.com/.
NetGuard firewall doesn't run on iOS, so there's no point in comparing to Apple. For those on Android, NetGuard is open-source, https://github.com/M66B/NetGuard
An iOS "local VPN" could definitely block all traffic to Apple IP ranges. But it lacks the ability to associate traffic with the originating process/framework. Like if, for example, I wanted to only allow iMessage to talk to Apple but nothing else. This is what Little Snitch and other software gives you on macOS/Linux/etc.
But even blanket blocking of all Apple IP ranges probably wouldn't do anything here. As documented, your device sends noise injected image vectors to OHTTP relays and doesn't contact Apple directly. By definition those relays are operated by 3rd parties. So if you consider this type of data "phoning home" you'll need to find the IPs of all of OHTTP relays iOS uses. (or block the traffic that looks up the OHTTP relays).
Apple's "enterprise networking" guide lists 3rd-party CDNs as subdomains of apple.com, which usually resolve to akamai or cloudflare subdomains. This allows those dynamic IPs to be blocked via dnsmasq ipset rules. In theory, they could use similar subdomain resolution for the OHTTP relays.
Since iOS was derived from macOS, perhaps Apple could restore the link between network traffic and process.
It looks sensible assuming that Little Snitch has some high level manager agent inside Apple manipulating the company making these kind of sneaky attacks on customers' privacy that drives the sales of Little Snitch. On the end they will also make them to buy Liitle Snitch for lots of millions or billions for elimination so they can attack customers freely afterwards. Little Snitch hidden agents are smart!
I do not assume that Apple managers are that degenerate idiots pushing through trust eroding marginal idiocy like this.
Summary: "Homomorphic encryption (HE) is a cryptographic technique that enables computation on encrypted data without revealing the underlying unencrypted data to the operating process. It provides a means for clients to send encrypted data to a server, which operates on that encrypted data and returns a result that the client can decrypt. During the execution of the request, the server itself never decrypts the original data or even has access to the decryption key. Such an approach presents new opportunities for cloud services to operate while protecting the privacy and security of a user’s data, which is obviously highly attractive for many scenarios."
Another setting that surprised me with being turned on by default apparently on macOS 15 is System Settings - Spotlight - "Help Apple Improve Search":
"Help improve Search by allowing Apple to store your Safari, Siri, Spotlight, Lookup, and #images search queries. The information collected is stored in a way that does not identify you and is used to improve search results."
No, this is not on by default. After system install at first boot it asks if you want to help improve search and it describes how your data will be handled, anonymized, etc. If you clicked on yes it is on. There is a choice to opt out.
I was on by default for me, both after the macOS 14 -> 15 upgrade and after installing macOS 15 cleanly. I wonder if they ask for consent in some regions only.
To me it seems like a reasonable feature that was, for the most part, implemented with great consideration for user privacy, though maybe I’m too trusting of the description. I mostly think this article is rage-bait and one should be wary of ‘falling for it’ when it shows up on hacker news in much the same way that one should be wary when rage-bait articles show up in tabloids or on Facebook.
It seems likely to me that concerns like those of the article or some of the comments in this thread are irrelevant to Apple’s bottom line. A concern some customers may actually have is data usage, but I guess it’s likely that the feature is off if in low data mode.
I wonder if this particular sort of issue would be solved by some setting for ‘privacy defaults’ or something where journalists/activists/some corporate IT departments/people who write articles like the OP can choose something to cause OS updates to set settings to values that talk less on the network. Seems hard to make a UI that is understandable. There is already a ‘lockdown mode’ for iOS. I don’t know if it affects this setting.
Literally all Apple needed to do was not have it enabled by default. Sending stuff over the network without asking is why trust in Apple is reduced further and further.
The OP is evidence. My phone had it turned on which I think is evidence. Together this feels like reasonably strong evidence but maybe something even stronger is easy to find. Vaguely related: https://markxu.com/strong-evidence
I think both of you had it turned on from past OS installs - it's bundled under other search metadata, the config isn't new to 18. And "it's on and I don't remember agreeing" is absolutely not evidence. This is super common - nobody remembers what they enabled a year ago.
Think about it this way: for people who turn off any & everything that phones home (aka anyone who is frustrated by this new "feature"), the chances of them having turned on something like that upon install is almost nil.
Not enabling something by default is pretty close to not having it at all. Accessibility is a reasonable exception where it makes sense to have the features even though they are off by default.
I mostly think the reaction to this article is overblown because it appeals popular ideas here about big tech. I think one should be wary of Apple’s claims about privacy: the reason is competition with Google and so they want users to be distrustful of the kinds of features that Google are better at implementing (I don’t want to say Apple isn’t trying to do the right thing either – if you look at accessibility, the competition was very bad for a lot of things for a long time and Apple was good despite the lack of commercial pressure). But I think one should also be wary of articles that make you angry and tell you what you suspected all along. (eg see the commenter elsewhere who doesn’t care about the details and is just angry). It’s much easier to spot this kind of rage-bait piece when it is targeting ‘normal people’ rather than the in-group.
> But I think one should also be wary of articles that make you angry and tell you what you suspected all along. (eg see the commenter elsewhere who doesn’t care about the details and is just angry). It’s much easier to spot this kind of rage-bait piece when it is targeting ‘normal people’ rather than the in-group.
The article was published by an Apple developer and user, i.e., myself, on my personal blog, which is followed mainly by other Apple developers and users. My blog comprises my own personal observations, insights, and opinions. If you see any rage, it would be my own personal rage, and not "bait". Bait for what?
I’m not interested in telling you what to put on your blog. Do whatever you like.
The headline is defensible but, in my opinion, quite sensationalised. People are likely to interpret it as being for an article making much stronger claims than the article actually does. I think a lot of the interactions people had with this submission, especially early on, were because the headline made them mad, rather than because of its contents. I think if one is interacting with some submission here due to a maddening headline, one should be wary about such interactions being driven by emotion, leading to poor discussion that is not particularly anchored to the topic of the article, rather than being driven by curiosity.
> The headline is defensible but, in my opinion, quite sensationalised.
How would you write the headline?
There's always a criticism of headlines, but headlines are necessarily short. It's like critics want the entire article text to appear in the headline, which is impossible.
I don't know what defensible but sensationalized is supposed to mean.
> I think a lot of the interactions people had with this submission, especially early on, were because the headline made them mad, rather than because of its contents.
That's pure speculation on your part, because the headline is very short and vague. In any case, it's not my fault if people read the headline but not the article. I want people to read the article, not just the headline.
I would probably aim for a title like ‘what does the “Enhanced Visual Search” feature do in iOS 18 and MacOS 15?’ Or ‘how much data is sent to Apple for the new “Enhanced Visual Search” in Photos’.
I think the early comments on this submission were because the headline made them mad because they were incurious and not particularly related to the topic of the article making – most could be under any article about Apple Photos. One early comment was about switching to alternatives to Photos and another was, paraphrasing, ‘I’m so angry. I don’t care about homeomorphic encryption or differential privacy’. Others seemed to follow the theme of the latter. After a while a comment attempted an overview of some of the technical details, which I thought was better.
Perhaps a more precise complaint is that many of the early comments didn’t really depend on the contents of the article – they could go under many Apple submissions – and I think it’s more likely for comments like that to be written while incensed.
I don’t think you’re to blame for the comments people choose to leave.
> I would probably aim for a title like ‘what does the “Enhanced Visual Search” feature do in iOS 18 and MacOS 15?’ Or ‘how much data is sent to Apple for the new “Enhanced Visual Search” in Photos’.
In my opinion, these are actually misleading headlines that don't represent the content of the article. In fact, I've never used the Enhanced Visual Search feature, nor do I know exactly how much data is sent to Apple.
My article was never intended as a kind of feature introduction that you might see in the news media. The main point was always about user consent for uploading data to Apple. To be clear, my article is and has always been a complaint. Thus, I think the headline is accurate. I am complaining, and readers should know that.
> another was, paraphrasing, ‘I’m so angry. I don’t care about homeomorphic encryption or differential privacy’.
> many of the early comments didn’t really depend on the contents of the article
The headline of the article didn't mention homeomorphic encryption or differential privacy, so they must have read the contents.
In the comment you are replying to, I’m trying to explain some of the reasons for the world being the way it is. I’m not trying to convince you to be happy about it.
Apple already communicates home by default. They never even fixed the macOS app signature check that they said they would, and yet people still choose to use the OS.
(And to be clear I’m not even bothered by the signature check)
At a certain point you have to figure that they realize it doesn’t matter short of some government entity forcing them to stop. At the very least the protections they put in place (homomorphic encryption, etc) are more than I think most other companies would ever bother doing.
if anyone else had done this then yes probably it's reasonable feature done reasonably. The problem is Apple has spent tens if not hundreds of millions of dollars advertising that they don't do things like this. That stuff stays on your iPhone unlike that other OS run by yucky advertising company. Apple would never siphon your data, because they care and you aren't the product.
Shit like this, reasonable in isolation or not, undermines that story completely. If they are so willing to just outright lie on a massive billboard, what else will they do when profits demand it?
It’s a reasonable feature, but should nevertheless require opt-in by the user. The opt-ins could certainly be bundled at install/upgrade time to reduce annoyance.
One thing particularly not clear to me is weather ios scan all data in the phone and send it to be part of public index or not.
I see from how the feature works from the UI it seems it's not. If the feature activated by user action does this still constitute as phoning home?
At this point, Mac Mini M4's are cheap enough and capable enough to just purchase two: one for off-line use, another on-.
Perhaps this is marketing genius (from an AAPL-shareholder POV)?
----
I'm laughing at the insanity of all this interconnectivity, but an NDA prevents me from typing the greatest source of my ironic chuckles. Described in an obtuse way: a privacy-focused hardware product ships with an undisclosed phone-home feature, letting the feds see every time you use the product (to produce a controversial product, at home).
Kick in my fucking door / sue me: it'll just re-enforce that I'm correct about concessionary-allowances...
Computer are inexpensive enough to own both on- & off-line hardware.
----
Even privacy-focused hardware manufacturers will allow undisclosed usage tracking (in order to continue existing, themselves). In my example, the OEM delivers a physical product which allows users to make tangible objects, at home. Every time you launch the hardware's control software (to make more controversial objects), it phones home.
Wrong. Anything that makes any network request is by definition a privacy leak. The network itself is always listening, and the act of making any connection says that you are using the computer, and where, and when.
Particularly among "controversial" product manufacturers.
----
Entirely unrelated: I recently visited my favorite hometown (Austin) museum, UT's Harry Ransom Center. Their P.E.N. exhibit has an exposé on how various mid-20th Century authors were in fact funded by The Ford Foundation (a known CIA "investment vehicle"), including Aldous Huxley (author of Brave New World). One of the exhibits features multiple author signatories adamently denying this CIA-funding, over the 1950's decade... ultimately this conspiracy theory ended up being true/revealed. Adds an entirely new dimension to The Savage's [BNW's main character] decision to hang himself.
If every app and system is making network connections by default without the user's knowledge or intervention then it doesn't really prove that you are using the computer at all.
Completely missing the point, but you're right that I should have said “using or in possession” of the computer. The point is that they reveal your physical location to the network, so if they make requests while not directly in use then it's actually even worse.
Every other product sold by Apple (Mac notebooks, iPhones, watches, etc) are designed to be carried around with you, happily beaconing your presence to every network and to every other Apple device that will listen. I have seen this with my own eyes where my M3 MBP showed up on my WiFi AP status page as soon as I got home even though it was supposedly “““asleep””” in my bag.
What I mean is, an M3 MacBook has essentially the same processor as a cell phone. Of course it’s able to perform activities when the system is “asleep.” It even has specific online behavior labeled as a feature that can be turned on/off (Power Nap).
This is behavior that is desired by Apple’s users.
I sometimes wish that more Apple users who aren’t satisfied would stop complaining while continuing to throw money at Apple and buy themselves a Linux laptop. If you want a laptop with special features like a WiFi kill switch you should buy one of those, not buy a consumer appliance device that is marketed toward convenience-oriented home users.
> I sometimes wish that more Apple users who aren’t satisfied would stop complaining while continuing to throw money at Apple and buy themselves a Linux laptop.
My work machine — not my choice. Don't let reality get in the way of your smug assumptions about me though.
Now they probably don't care about you personally, but you'd be surprised how many people are connected indirectly to a person of interest. Google "Maher Arar" for the horrific consequences of false positives.
>you'd be surprised how many people are connected indirectly to a person of interest.
I get called "paranoid," often, but I'm being honest when responding to people requesting my phone number: "you don't want my number in your phone, because it'll associate you with people you probably don't want to be associated with" [see Five Eyes' 3-Degrees of Separation].
Holy crap! Enabled by default! Thank you for letting everyone know.
“Enhanced Visual Search in Photos allows you to search for photos using landmarks or points of interest. Your device privately matches places in your photos to a global index Apple maintains on our servers. We apply homomorphic encryption and differential privacy, and use an OHTTP relay that hides IP address. This prevents Apple learning about the information in your photos. You can turn off Enhanced Visual Search at any time on your iOS or iPadOS device by going to Settings > Apps > Photos. On Mac, open Photos and go to Settings > General.”
I think the user should be prompted to enable new functionality that sends data to the cloud. I think there should be a link to details about which data is being sent, how it is being transmitted, if it is stored, if it is provided to 3rd-parties, and what is being done with it.
Maybe this is exactly what the GDPR does. But it is what I would appreciate as a user.
I have seen how sending metrics and crash dumps from a mobile device can radically improve the user experience.
I have also seen enough of the Google bid traffic to know how much they know about us.
I want to enable sending metrics, but most of the time it's a radio button labeled "metrics" and I don't know what's going on.
Now "What happens on your iPhone stays on your iPhone" seems like it deserves the Lindt chocolates defense: "exaggerated advertising, blustering, and boasting upon which no reasonable buyer would rely."
> the only way to guarantee computing privacy is to not send data off the device.
> It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations. [...] By enabling the "feature" without asking, Apple disrespects users and their preferences. I never wanted my iPhone to phone home to Apple.
Regardless of how obfuscated or "secure" or otherwise "privacy-protecting" the feature is, the fact is that some information derived from one's personal content is transmitted, without prior consent. Even if the information is protected, all network queries are information. A timestamp that proves you took a certain action at a certain time (like taking a photo, assuming stuff is sent to this service immediately upon adding a new photo), from a certain location (by correlating your location information at that time), etc etc.. and that's just the tip of the iceberg. Transmitting information from a user's device without their explicit consent is a violation of their privacy.
How much size would it take to store a model of every known location in the world and common things?
For ex: I sent a friend a photo of my puppy in the bathtub and her Airpods (via iphone) announced "(name) sent you a photo of a dog in a bathtub". She thought it was really cool and so do I personally. That's a useful feature. IDK how much that requires going off-device though.
I’m not an expert, but I would say extremely small.
For comparison Hunyuan video encodes a shit-ton of videos and rudimentary real world physics understanding, at very high quality in only 13B parameters.
LLAMA 3.3 encodes a good chunk of all the knowledge available to humanity in only 70B parameters.
And this is only considering open source models, the closed source one may be even more efficient.
Maybe we have different understandings of what extremely small is (including that emphasis) but an LLM is not that by definition (the first L). I'm no expert either but the smaller value mentioned is 13e9. If these things are 8-bit integers, that's 13 GB data (more for a normal integer or a float). That's a significant percentage of long term storage on a phone (especially Apple models) let alone that it would fit in RAM on even most desktops which is afaik required for useful speeds. Taking this as upper bound and saying it must be extremely small to encode only landmarks, idk. I'd be impressed if it's down to a few dozen megabytes, but with potentially hundreds of such mildly useful neural nets, it adds up and isn't so small that you'd include it as a no-brainer either
There is a large number of people out there who receive hundreds of notifications (willingly but mostly unwillingly) daily from apps they have installed (not just messengers), and nearly all of them can't cope with the flood of the notifications. Most give up on tending to the notifications altogether, but some resort to the notification summaries which alleviate the cognitive overload («oh, it is a picture of a dog that I will check out when I get a chance to, and not a pic of significant other who got themselves into a car accident»).
The solution is, of course, to not allow all random apps to spam the user with the notifications, but that is not how laypeople use their phones.
Even the more legit apps (e.g. IG) dump complete garbage upon unsuspecting users throughout the day («we thought you would love this piece of trash because somebody paid us»).
Most people using the device will never go into the notification settings, if an app on install tells you they need notifications to tell you about your driver arriving…and then sends you marketing notifications on the same channel most people will just accept that.
My IG notifications are disabled in perpetuity precisely for that reason, even if it has come at a cost of not receiving IM arrival notifications from a couple of friends who message me exclusively via IG (another enigma – why, as both have my contact details on proper messaging platforms).
Frankly, hundreds of daily notifications is the reality we can't change, and the question is: why do people choose to succumb to the misery of it rather than availing themselves of the proper implements so readily at their disposal?
My armchair explanation is that people can’t cope with being bored. All those notifications are little bits of dopamine hits each time they arrive. It’s not even FOMO, just waiting for something that may be interesting.
With an assault of notifications while you are busy, being able to determine without lifting your phone whether the alert is something mundane versus what you've been waiting for, is useful. If I'm working outside, removing gloves to check the phone each time it vibrates becomes endlessly disruptive, but I want the phone with me in case family calls, someone's delivering something, there's an urgent work issue, etc.
> like taking a photo, assuming stuff is sent to this service immediately upon adding a new photo
So you jumped to a conclusion based on an incorrect premise. This is easy to see that this does not happen immediately after taking a photo. One the network traffic will show this (and it won’t), two homomorphic encryption is expensive so it cannot. Photos classically doesn’t sync on demand, as most iPhone users will know by way if it telling you this in the photos app when it does sync. Most expensive operations are queued up for when the device is plugged in (and on WiFi) because it’ll otherwise drain battery.
you're splitting a very fine hair while ignoring the larger privacy implication of the feature. So the timestamp might or might not be delayed a bit from being perfectly accurate? So what? It still is data approximating when the photo was taken, even if the resolution were as bad as "within a few days"
So Signal messages aren't secure because they're transmitted and so their "obfuscation" isn't enough to protect your data? Have you read what the author cited (and then admitted to not understanding) what Apple says they actually do to the data before transmission?
I could see an argument in the metadata (though there are multiple assumptions involved there, not least that they don't truly do OHTTP but instead conspire to learn at what timestamp a user took a picture), but if you already don't trust in what is essentially math, I'm not sure where the uncertainty and doubt ends
The obvious difference is that by sending your photos with Signal, you are doing it willingly. You let it encrypt and decrypt willingly. You decide who gets it.
Your ISP, a bunch of routers and switches and the servers run by Signal can also see your encrypted photo. You don’t really get to decide who sees the encrypted photo. You do get to decide which photo you encrypt and send though.
All of those are parts of the network infrastructure. They neither "see" the photo, edit it or need it. They don't even know if it's a photo.
Everybody knows that there is a network infrastructure where your content flows through. You willingly accept that as a connected device user because it is necessary to be connected.
What Apple did is not necessary and users don't know about it.
I agree that this isn’t necessary and that Apple should have asked for consent from the user.
I do want to point out that Apple doesn’t get to see your photo. Homomorphic encryption is really cool in that way. Apple isn’t able to decrypt the photo and the results they produce are also encrypted. That’s the beauty of homomorphic encryption. I can send you an encrypted spreadsheet and you can compute the sum of a column for me. I’m the only one who can see the actual sum since you can only see the encrypted sum.
The difference being that the signal message is sent with consent: You literally press a button to send it there is a clear causal relationship between clicking the button and the message being sent.
These issues are all addressed in the Apple blog post that talks about how this feature is implemented. Two steps are taken to deal with these risks:
1) iOS creates additional fake queries, and all queries pass through scheduler that ensures you can use time-of-lookup to either discriminate real queries from fake queries, or identify when a photo was taken.
2) All queries are performed anonymously, with the use of a third party relaying service. So there’s no way for Apple to tie a specific query back to a specific device, or even IP address.
Between those two mitigating features. Getting hold of an individuals personal data using this feature requires you to first compromise the targets phone, to disable the fake queries. Then compromise the relaying party to correlate queries back to a specific IP address.
If you can manage all that, then quite frankly you’re a fool for expending all that effort. When you could just use your iOS compromise to have the device send you its location data directly. No need to faff about waiting for your target to take photos, then track multiple landmark lookups, carefully collecting a few bits of additional data per query, until you finally have enough to identify the location of your target or targets.
Is there a way to verify the claims of obfuscation, security and privacy? Or is the only verifiable fact the sending of unknown data to apple by the photos app?
This whole thing is reminding me of the outrage over Apple and Google's privacy preserving 'Exposure Notification System' system from the Covid years. It defies intuition that they can alert you to exposure without also tracking you, but indeed that's what the technology lets you do.
Similarly here, it feels like the author is leaning into a knee jerk reaction about invasion of privacy without really trying to evaluate the effectiveness of the technologies here (client side vectorization, differential privacy, OHTTP relays, and homomorphic encryption).
Though I 100% agree Apple should ask the user for consent first for a feature like this.
The comment I was replying to was stating that source code was necessary to solve privacy, I just said you’d need to get down to silicon if you’re going that far. Don’t be rude. I’m unemployed right now.
If you have the capability to actually skillfully analyze this type of crypto, disassembling the binaries from your device (or at the very least, an ipsw for your device) should be trivial.
After all, you wouldn’t actually be trusting the source code given to you to match what’s running on your device, would you?
Reverse engineering is a separate skillet on its own, on top of the other ones you need to read the source code and good developers aren't necessarily good at that.
> After all, you wouldn’t actually be trusting the source code given to you to match what’s running on your device, would you?
That's why the best practice in the industry follows reproducible builds.
Of course it is not the whole technology stack, but it is something at least. If your evaluation leads to potential problems, you can create issues right there on the github project!
What “places” would Apple be able to match that aren’t either obvious or irrelevant? (e.g. “The Grand Canyon” is obvious. The interior of a home is irrelevant.) Maybe the interior of a restaurant?
I hate to jump to conspiracy theories, but the mechanism seems like a more effective version of their prior CSAM scanning scheme[1]. Best part? It doesn’t even require iCloud usage.
It's called "Enhanced Visual Search" and they described it like this:
> Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest.
I guess it's useful if you get very lost and don't recognize the pyramids right in front of you, and forgot you have a GPS in the phone.
Oh I dunno. Your comment made me chuckle and I’m largely on the fence on this one but it doesn’t seem too outrageous to say that maybe this is a slight invasion of people’s privacy without asking them to opt in.
So basically - You take a picture. Apple encrypts it and uploads it to their server. The server matches the (still encrypted) picture to a database and tells your device "this picture contains the Eiffel Tower". Later when you search for Eiffel Tower on your device the photo pops up.
Is the complexity and security risk really worth it for such a niche feature?
It's also funny that Apple is simultaneously saying "don't worry the photo is encrypted so we can't read it" and "we are extracting data from the encrypted photo to enhance your experience".
Not really. It's more like apple runs a local algorithm that takes your picture of the Eiffel tower, and outputs some text "Eiffel tower, person smiling", and then encrypts that text and sends it securely to apples servers to help you when you perform a search.
Locally, a small ML model identifies potential POIs in an image.
Another model turns these regions into a series of numbers (a vector) that represent the image. For instance, one number might correlate with how "skyscraper-like" the image is. (We don't actually know the definition of each dimension of the vector, but we can turn an image that we know is the eiffel tower into a vector, and measure how closely our reference image and our sample image are located)
The thing is, we aren't storing this database with the vectors of all known locations on our phone. We could send the vector we made on device off to Apple's servers. The vector is lossy, after all, so apple wouldn't have the image. If we did this, however, apple would know that we have an image of the eiffel tower.
So, this is the magic part. The device encrypts the vector using a private key known only to it, then sends this unreadable vector off to the server. Somehow, using Homomorphic Encryption and other processes I do not understand, mathematical operations like cosine similarity can be applied to this encrypted vector without reading the actual contents of the vector. Each one of these operations changes the value, which is still encrypted, but we do not know how the value changed.
I don't know if this is exactly what Apple does, I think they have more efficient ways, but theoretically what you could do is apply each row in your database to this encrypted value, in such a way that the encrypted value becomes the name of the POI of the best match, or otherwise junk is appended (completely changing the encrypted value) Again, the server has not read the encrypted value, it does not know which row won out. Only the client will know when it decrypts the new value.
Because it turns out that mathematicians and computer scientists have devised schemes that allow for certain computational operations to be performed on encrypted data without revealing the data itself. You can do a+b=c and it doesn’t reveal anything about what a and b are is the intuition here. This has been mostly confined to the realm of theory and mathematics until very recently but Apple has operationalized it for the first time.
The phone has intelligence to detect things that look like landmarks, and does cropping/normalization and converts to a mathematical form.
Apple has a database trained on multiple photos of each landmark (or part of a landmark), to give a likelihood of a match.
Homomorphic encryption means that the encrypted mathematical form of a potential landmark from the phone can be applied to the encrypted set of landmark data, to get an encrypted result set.
The phone can then decrypt this and see the result of the query. But anyone else sees this as noise being translated to new noise, including Apple's server.
The justification for this approach is storage - the data set of landmarks can only get larger as the data set gets more comprehensive. Imagine trying to match photos for inside castles, cathedrals and museums as examples.
I don't completely understand the maths of how this works, but no, they don't.
Here's a theoretical way I wrote in another comment:
> I think they have more efficient ways, but theoretically what you could do is apply each row in your database to this encrypted value, in such a way that the encrypted value becomes the name of the POI of the best match, or otherwise junk is appended (completely changing the encrypted value) Again, the server has not read the encrypted value, it does not know which row won out. Only the client will know when it decrypts the new value.
They do something like this, using homomorphic encryption. Whatever they do, there is no doubt they incur serious performance hits.
> They do something like this, using homomorphic encryption. Whatever they do, there is no doubt they incur serious performance hits.
Right, I've seen similar engineering efforts to target this sort of functionality fail because of the computational cost and resulting latency. I'm curious to read the paper for the tradeoffs they made toward practicality at Apple's scale of users.
They don’t send the photo. They send some encrypted metadata to which some noise is added. The metadata can be loosely understood as “I have this photo that looks sort of like this”. Then the server takes that encrypted data from the anonymized device and responds something like “that looks like the Eiffel Tower” and sends it back to the device. The actual photo never goes to the server.
With the added caveat that HE is magic sauce - so the server cannot see the metadata (cropped/normalized image data), and doesn't know how much it does or does not look like the Eiffel Tower.
It sounds like Apple implemented a feature and went out of the way to preserve privacy. For most users the correct choice here is to enable the feature by default.
1,038 comments
[ 3.4 ms ] story [ 312 ms ] threadIt is new. It is fancy. It is not clear where HE/DP is being used, it depends if the code is written using the Swift toolkit, but even that has paths for exfiltration if used incorrectly. They claim they are using DP in Photos as stated in the article here:
https://machinelearning.apple.com/research/scenes-differenti...
But the fact remains they are looking at your pictures. I do not trust them for one fleeting fart in the wind on this. Think about it for a hot second: HE/DP allows you to perform operations on the data without knowing the data, but what if someone goofs an operation and it ends up returning the actual data?
Sorry, not buying it. Crypto is hard to get right, and when it is monetized like this for "new features", it is wildly unnecessary and exposes users to more risk.
That's kind of like saying "they can take a picture of your face and transmit it, but what if someone goofs an operation and sends your actual head instead".
Any encrypted data the server has cannot 'accidentally' be decrypted, and as someone else explained in this thread they only send encrypted vectors describing features of the image (building, tree, etc) and not the actual image. It's certainly not a fact that "they are looking at your pictures" [1].
[1] "they" being Apple; the other guys could have backdoored the whole OS and Apple employee's computers for all I know
+1
Then why is it happening to me?
Your “photos” aren’t shipped off-device without your knowledge, just an arbitrary blob of ”metadata” that you can’t audit describing everything about that photo. :)
It’s sort of like “I don’t want my WiFi router uploading my house online!” And someone replying “it’s not your physical house, just a map of the house and the real time location of everyone in it! The house never moves!”
This is probably done to compete with Google Photos which has a great photo search by word feature.
With that said, Apple can use whatever privacy measures to protect user data. But at the end of the day, a subpoena can easily force them to hand over data.
The best privacy measure is to just not have the data. I guess indexing photos offline in phone is not very feasible yet.
It's right there in the help text:
> Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest.
They are saying that they match your photo against their global index.
They are not saying that they will add your photo to their global index.
Nobody said that except you.
Apple Photos is already capable of searching your photos by word/content fully offline on-device.
Google Photos is unable to search your photos by word/content fully offline on-device. Hell, it can't even search by file name if you don't sync your photos to the cloud!
I don't think Apple has to worry about Google at all.
There has been a form of on device indexing since at least iOS 12. My understanding is it performs a basic indexing typically overnight when the phone is charging and allows one to perform searches like "dog" or "car" and pull up matching images.
https://support.apple.com/guide/iphone/search-in-photos-iph3...
Hence why Apple started sending photo hashes to their servers for indexing by default.
There's only so much offline computing power.
https://www.internetsociety.org/resources/doc/2023/client-si...
I do:
- Client side vectorization: the photo is processed locally, preparing a non-reversible vector representation before sending (think semantic hash).
- Differential privacy: a decent amount of noise is added the the vector before sending it. Enough to make it impossible to reverse lookup the vector. The noise level here is ε = 0.8, which is quite good privacy.
- OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library).
- Homomorphic encryption: The lookup work is performed on server with encrypted data. Apple can't decrypt the vector contents, or response contents. Only the client can decrypt the result of the lookup.
This is what a good privacy story looks like. Multiple levels of privacy security, when any one of the latter 3 should be enough alone to protect privacy.
"It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here.
I'm not sure what the right call is here. But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true. There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk. But if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this.
This would be easily solvable: On first run show a window with:
> Hey, we have this new cool feature that does X and is totally private because of Y [link to Learn More]
> Do you want to turn it on? You can change your mind later in Settings
> [Yes] [No]
You don’t use iCloud for anything? When you change phones do you start fresh or use your computer for backups? Do sync bookmarks? Browsing history?
Do you use iMessage?
edit: I will just add, even if we accept the argument that it's extremely secure and impossible to leak information, then where do we draw the line between "extremely secure" and "somewhat secure" and "not secure at all"? Should we trust Apple to make this decision for us?
Apple IS just another cloud provider / centralized backup service. It's not fundamentally different than others, and if you're not in select group of whatever the respectful term is for those who stay strictly inside apple ecosystem, you will have multiple clouds and multiple data sets and multiple backups that all interact with each other and your heterogeneous devices in unpredictable ways. Icloud will not help you with that any more than google cloud or Samsung cloud etc. They all want to own all of your stuff, neither is simply a hyper helpful neutral director.
Even if you use Microsoft Office or GSuite and save using the standard file picker, you can save to iCloud. iCloud has a native app for Windows and plug ins on Windows to sync browser bookmarks for Chrome, Edge and Firefox
And the alternative people are proposing are four or five self hosted solutions?
Iphone is "just another device". I don't feel Icloud is any better integrated with my Samsung note, than google is integrated with my iPhone - in fact, the opposite. Google, for example, CAN sync my photos across iphone and Android and windows devices. Whereas my wife knows the primeval scream from the home office every 6 months I try to claw photos out of apple's greedy selfish hands :-)
For people who JUST use iphone, sure, Icloud is the boss just like for people who JUST use e.g. Samsung galaxy the Samsung cloud is awesome. But that's not a high bar. I feel we are still lacking empathy here for people like original poster who may have more than one device in their lives.
And I wouldn’t have the same arguments if they weee using Google cloud. But they are concerned about “privacy” and trust Google?
But my argument is about people thinking that Apple or Google should care about the minuscule number of people who are hosting their own syncing services
Or as the GP suggested, forego the cloud entirely. iCloud and Apple’s built in iOS backup is not a magic bullet unfortunately.
No, if you enable Advanced Data Protection for iCloud[1], the photos stored in Apple Photos are end to end encrypted.
[1] https://support.apple.com/en-us/108756
Yes, I always start with clean installs, both on iOS and on macOS. Sometimes I even restart fresh on the same device, as I make sure my hardware lasts. I don’t sync bookmarks, I keep them in Pinboard and none of them has any private or remotely identifiable information anyway. I don’t care about saving browser history either, in fact I have it set to periodically auto-clear, which is a feature in Safari.
To a first approximation, everyone in 2024 expects their data and settings to be transferred across devices.
People aren’t working as if it is 2010 when you had to backup and restore devices via iTunes. If I’m out of town somewhere and my phone gets lost, damaged or stolen, I can buy another iPhone, log into my account and everything gets restored as it was.
Just as I expect my watch progress to work when I use Netflix between my phone, iPad, Roku devices etc.
How many people are going to say in 2024 that they don’t want continuous cloud backup? You want Windows Vista style pop ups and permissions?
I don’t have Netflix but neither is that relevant to the point, you’re obviously and embarrassingly grasping at straws.
No one is arguing against continuous cloud backups, they’re arguing about sending data without consent. Which, by the way, is something Apple used to understand not to do.
https://www.youtube.com/watch?v=39iKLwlUqBo
Apple’s OS are already filled with Windows Vista style popups and permissions for inconsequential crap, people have been making fun of them for that for years.
Isn’t it bad enough that I have a popup every time I copy and paste between apps?
For me, not really no. It reminds me I am copying information and not from some phishing app, I find it informative.
And I'm probably one of the few who actually click "Reject" to the cookie pop ups having to click no on 3742 legitimate consents.
The simple answer is everything should be opt-out. I'll opt-in if I require it because frankly, regardless to how Fort-Knox my data is $CORP still cannot be trusted.
Video conferencing instead of FaceTime are made via self-hosted Jitsi and if I am to brag all running on FreeBSD.
Out of Apple or Google I trust neither however will align with Apple more than Google. It's as close as I can get from not having data collected from mongrels.
Computers should not be sneakily doing things in the background without my commanding them to do so. But if they insist that the only way they can work is by doing things in the background, then I expect the computer to at the very least obtain my consent before doing those things. And computers should definitely not be exfiltrating anything over to the network without my explicit command to do so. This shit world we are living in where your computer just does whatever the application developer wants it to do rather than what the user wants it to do has to come to an end!
With PIR, an Apple phone recieving a phone call queries Apple's database with that phone number, but because it's using homomorphic encryption, Apple doesn't know the number that called despite looking it up in their database to provide caller id info, so they can't tie your phone number and the callers phone number together.
https://machinelearning.apple.com/research/homomorphic-encry...
"Ah, I see you care about privacy, but you own a phone! How hypocritical of you!"
https://thenib.com/mister-gotcha/
I also don’t run a private chat server that people log into - I’m like most of the iPhone and Android using world
We need less sarcasm, not more.
I bet most of those made up numbers of yours will have no idea that the feature exists.
A simple screen like they usually do with "Whats new in iOS" could easily have let you enabled it on the get go, with the additional benefit that you would have been made aware of it existing.
This iOS 18.2 update had no such screen, I just updated.
You're not making any sense.
The question I asked was
> How exactly is a new feature that is not advertised harder for you, or for anyone for that matter?
The problem with this feature is that we cannot verify that Apple’s implementation of the math is correct and without security flaws. Everyone knows there is security flaws in all software, and this implementation is not open (I.e. we cannot review the code, and even if we could review code we cannot verify that the provided code was the code used in the iOS build). So, we have to trust Apple did not make any mistakes in their implementation.
They could just have the OS batch uploads until a later point e.g. when the phone checks for updates.
The point is that this is all about risk mitigation not elimination.
Other than their entire reputation
https://www.csoonline.com/article/571797/the-apache-log4j-vu...
What was the other package that had the mysterious .?
You say this as if being shamed into patching the occasional vuln is equivalent to security best practices.
Open code which can be independently audited is only a baseline for trustworthy code. A baseline none of those three meet. And one which by itself is insufficient to counter a reflections on trusting trust style attack. For that you need open code, diverse open build toolchains, and reproducible builds. None of which is being done by those three.
Are you getting your ideas about security from the marketing department?
1: https://arstechnica.com/security/2024/03/hackers-can-extract... 2: https://www.wired.com/story/google-android-pixel-showcase-vu... 3: https://blog.morphisec.com/5-ntlm-vulnerabilities-unpatched-...
OSS is not the panacea that everyone touts it to be.
I'm not aware of any major open source projects that haven't experienced some level of auditing. Coverity alone scans everything you're likely to find in a distribution like Debian or Fedora: https://scan.coverity.com/o/oss_success_stories
> How many packages have you audited?
Several on which I depend. And I'm just one pair of eyeballs.
> Personally, I don't have the skillz to do that.
Then why are you commenting about it?
> OSS is not the panacea that everyone touts it to be.
I don't know who's touting it as a panacea, seems like a strawman you've erected. It's a necessary pre-requisite without which best practices aren't possible or verifiable.
https://www.theverge.com/2021/4/30/22410164/linux-kernel-uni...
A non-advertized feature, which is not independently verified, which about image contents? I would be prefer independent verification of their claims.
Hey! That’s wrong.
But I promise I won’t do anything wrong with it.
Well ok then.
And at that point all the opt-in dialogs in the world don't matter and you should not be running iOS but building some custom Android ROM from scratch.
>Let’s say you wanted to count how many of your online friends were dogs, while respecting the maxim that, on the Internet, nobody should know you’re a dog. To do this, you could ask each friend to answer the question “Are you a dog?” in the following way. Each friend should flip a coin in secret, and answer the question truthfully if the coin came up heads; but, if the coin came up tails, that friend should always say “Yes” regardless. Then you could get a good estimate of the true count from the greater-than-half fraction of your friends that answered “Yes”. However, you still wouldn’t know which of your friends was a dog: each answer “Yes” would most likely be due to that friend’s coin flip coming up tails.
Quantum computers will make breaking RSA and Diff-Hellman public key encryption easier. They will not effect things like AES, nor things like hashing:
> Client side vectorization: the photo is processed locally, preparing a non-reversible vector representation before sending (think semantic hash).
And for RSA and DH, there are algorithms being deployed to deal with that:
* https://en.wikipedia.org/wiki/NIST_Post-Quantum_Cryptography...
Anyway, back to the subject at hand; here's Apple on that subject:
> We use BFV parameters that achieve post-quantum 128-bit security, meaning they provide strong security against both classical and potential future quantum attacks
https://machinelearning.apple.com/research/homomorphic-encry...
https://security.apple.com/blog/imessage-pq3/
Where are the boundaries in this scenario?
https://github.com/apple/swift-homomorphic-encryption
I'm asking: what does an opt in for that really look like? You're not going to be able to give the user enough info to make an educated decision. There's ton of risk of "privacy washing" ("we use DP" but at very poor epsilon, or "we use E2E encryption" with side channel data gathering).
There's no easy answer. "ask the user", when the question requires a phd level understanding of stats to evaluate the risk isn't a great answer. But I don't have another one.
https://www.youtube.com/watch?v=39iKLwlUqBo
Don't do things without my consent!!! How hard is it for Silicon Valley to understand this very simple concept?
Should Apple insist that every end user consents to the user agent string sent on every HTTP request?
You've succinctly identified a (maybe the) huge problem in the computing world today. Computers should not do anything without the user's command/consent. This seems like a hopeless and unachievable ideal only because of how far we've already strayed from the light.
Even Linux, supposedly the last bastion of user control... it's a mess. Do a fresh install and type ps ax at a shell. You'll see dozens of processes in the background doing god knows what. I didn't consent to any of this! The distribution's maintainer simply decided on my behalf that I want the computer to be running all these processes. This is totally normalized!
I don't expect my computer to ask for consent again and again for every byte sent over the network, but I do expect it to obtain my consent before generally accessing the network and sending bytes over the network.
To me, there's never been a case, except maybe in the first decade or so of the hobby/tinkering PC movement, where most users had this ability.
Should we just not use computers?
I don't think "should we just give up?" is a reasonable question to anything.
And getting downvoted for saying it, which is a fascinating incongruity.
If the world was a nightclub, “Silicon Valley” would be a creepy guy who walks up to every woman and says “You’re now dating me. To stop, you need to opt out using a form that I will do my best to make sure you can’t read.”
Choosing an Apple product is consent to trusting Apple. Continued use their products represents ongoing consent. This is an objective fact about all complex connected devices and it cannot possibly be otherwise.
Or signal of non-named stakeholders.
Yes you did. You purchased a computer, put this software on it and executed it. If you didn't want it to do whatever it's doing you should have determined what it would do beforehand and chose not to do it.
Even assuming that running the software implies my consent (which I would dispute), how do I make the decision about whether I should execute the software if I don't know what it is doing?
This all-or-nothing approach is also problematic. I should not have to allow the developer free rein to do whatever he wants, as a condition of using the software. This is why operating systems are slowly building granular permissions and consent checks.
I agree that all-or-nothing is problematic but even with a flexible permission system the best you can hope for is for all the things apps do to be itemized and set to sane defaults. But even then sanity is subjective. For every person like you (and me fwiw) who values privacy there are 1000 people who will never find the settings, don't care about privacy, and will wonder why stuff isn't working.
Ultimately privacy is similar to security in that it comes down to trust. If you don't trust your OS you're screwed. Your choices are try to exert as much control over it as possible, or don't use it.
How would that make any difference in this case? Presumably, you'll have long-ago checked the "allow general access to the network" setting, so you've given consent to the "send my photo data" action. Heck, surely connecting to the internet in the first place is implicit consent that you want to send stuff over the network?
Every NTP session leaks time desync information, which reveals—on modern hardware—relativistic travel, including long airplane trips.
Every software update leaks a fortune about what you run and when you connect.
I don’t think it’s reasonable to ask that people consent to these; I don’t think they can. I absolutely agree that photo metadata is different and at a way higher level of the stack.
Sure, you can ask users "do you want to use this". But why do we ask that? Historically it's user consent (knowingly opting in), and legal requirements around privacy. We don't have that pop up on any random new feature, it's gated to ones with some risk. There are questions to ask: does this technical method have any privacy risk? Can the user make informed consent? Again: I'm not pitching we ditch opt-in (I really don't have a fix in mind), but I feel like we're defaulting too quickly to "old tools for new problems". The old way is services=collection=consent. These are new privacy technologies which use a service, but the privacy is applied locally before leaving your device, and you don't need to trust the service (if you trust the DP/HE research).
End of the day: I'd really like to see more systems like this. I think there were technically flawed statements in the original blog article under discussion. I think new design methods might be needed when new technologies come into play. I don't have any magic answers.
Such as?
You're presenting it as if you have to explain elliptic curve cryptography in order to toggle a "show password" dialogue but that's disingenuous framing, all you have to say is "Allow Apple to process your images", simple as that. Otherwise you can argue many things can't possibly be made into options. Should location data always be sent, because satellites are complicated and hard to explain? Should we let them choose whether they can turn wifi on or off, because you have to explain IEEE 802.11 to them?
Then don’t run someone else’s software on your device. It’s not your software, you are merely a licensee. Don’t delude yourself that you are morally entitled to absolute control over it.
The only way to have absolute control over software is with an RMS style obsession with Free software.
Want a phone that doesn’t spy on you? Make it yourself. If you can’t, find some like-minded people and incentivise them (with money or otherwise) to make it for you. If they can’t (or won’t) perhaps contemplate the possibility that large capitalist enterprises might be the only practical way to develop some products.
To suggest that customers are "morally entitled" to a Samsung phone with zero tracking and zero telemetry is similarly absurd. If you don't like Samsung's product, don't buy it.
Why not? What gives McD the right to make such a decision unilaterally, other than might?
In fact, this is how disability legislation (for example) already tends to work. You don't get to tell disabled people to just go somewhere else, you have to make reasonable accomodations for them.
This cannot be a serious question.
Restaurant have a legal obligation to warn the customers. AKA "opt-in" which is NOT what Apple is doing. And it's the whole issue with their behavior.
That doesn't stop consumers from engaging in Info Wars style paranoia, and grandstanding about the aforementioned paranoia.
We can regulate these problems.
If the EU can regulate away the lightning connector they can regulate away this kind of stuff.
No EU regulation could regulate away all "moral" concerns over software. More specifically, they EU could regulate, but the overwhelming majority of software companies would either strip significant features out for EU customers, or exit the market altogether.
This isn't rocket surgery.
I mostly agree. I'm just annoyed "this new privacy tech is too hard to explain" leads to "you shouldn't do it". This new privacy tech is a huge net positive for users.
Also: from other comments sounds like it might have been opt-in the whole time. Someone said a fresh install has it off.
It's a positive compared to doing the same "feature" without the privacy tech. It's not necessarily a positive compared to not forcing the "feature" on the user at all.
The privacy tech isn't necessarily a positive as a whole if it leads companies to take more liberties in the name of "hey you don't need to be able to turn it off because we have this magical privacy tech (that nobody understands and may or may not actually work please don't look into it too hard)".
However, "my data is being sent off my device" is incorrect, as GP explained. Metadata, derived from your data, with noise added to make it irreversible, is being sent off your device. It's the equivalent of sending an MD5 of your password somewhere; you may still object, but it is not factually correct to say your password was transmitted.
Hackers love to have MD5 checksums of passwords. They make it way easier to find the passwords in a brute force attack.
https://en.wikipedia.org/wiki/Rainbow_table
> Hackers love to have MD5 checksums of passwords.
Hackers love not understanding analogies. :)
a) MD5 is reversible, it just cost GPU time to brute force
b) It is unproven that their implementation is irreversible
https://github.com/apple/swift-homomorphic-encryption
Sounds like my data is being sent off my device.
> It's the equivalent of sending an MD5 of your password somewhere
Sounds even worse lol
There is plenty of data on your device that isn’t “your data” simply due to existing on your device.
Owning a phone is a privacy failure by default in the United States.
Care to provide a pointer to what device they are using? I would absolutely get my real estate license for this.
Unfortunately, the knee-jerk reaction of many defense industry pundits (and VCs, for that matter) is that US intelligence is an unparalleled moral good, and the virtues of privacy aren't worth hamstringing our government's work. Many of these people will try to suppress comments like yours because it embarrasses Americans and American business by association. And I sympathize completely - I'm dumbfounded by the response from my government now that we know China is hacking our telecom records.
It's apparent it has been kept in place because of all of the value it provides to the 5 eyes.
Maybe they should lump its default state into something that already exists? E.g. assume that if you already have location access enabled for Photos (it does ask!), you've already indicated that you're okay with something about this identifying being sent to Apple whenever you take a picture.
My understanding is that Location Services will, among other things, send a hash of local WiFi network SSIDs and signal strengths to a database Apple maintains, and use that to triangulate a possible position for you. This seems loosely analogous to what's going on here with the compute-a-vector thing.
It could be tied to iCloud Photos, perhaps, because then you already know that your photos are getting uploaded to Apple.
(We could argue about it, but personally I think some kind of hash doesn't qualify.)
In fact, I'm willing to bet that if they'd added this feature and gated it behind iCloud Photos being enabled, we'd have different articles complaining about Apple making a cash grab by trying to get people to pay for premium storage. :P
As the article notes, this new feature is so "popular" that neither Apple nor the Apple media have bothered to mention it. AFAICT it's not even in Apple's document listing all the new features of iOS 18: https://www.apple.com/ios/ios-18/pdf/iOS_18_All_New_Features...
No, I didn't ask for a hypothetical set. I wanted the actual set of people.
I can try to turn down my natural inclination towards cautious phrasing, if you'd like? Get us on the same level. :D
Large set? Yes. Majority? No. CIRP says 2/3 of US Apple users pay for iCloud storage[0]. It's this popular for the exact reason you mentioned. Almost no one can fit their photo library into 5GB so they opt in to the cheap 50GB for $0.99/month. 50GB is enough for a lot of people.
[0] https://wccftech.com/paid-icloud-subscription-is-apples-most...
Well, I'll take back what I said about the majority. I do still think that the remaining 1/3 of users who don't have enough storage to turn on iCloud Photos qualify as what lapcat was asking for, though.
I know users who would prefer not to trust Apple for anything, and only pay for and use iCloud to backup the Desktop [and similar locations]. If they were to hear that their opt-in for iCloud means that Apple starts copying random things, they would not be happy.
[OT, I use Arq. But admit that iCloud is simpler, and it is not apples to apples.]
IMO, the fact that Apple backs up your keychain to the Mothership; and that this is a "default" behavior that will re-enable itself when shut off, reflects an attitude that makes me very distrustful of Apple.
Then why lie and mislead customers that your data stays local?
Apple patting itself on the back.
1)sending my personal data to them in any way is not a "feature." It's especially not a feature because what it sets out to do is rather unnecessary because every photo has geotagging, time-based grouping, and AI/ML/whatever on-device keyword assignments and OCR. I can open up my phone right now and search for every picture that has grass in it. I can search for "washington" and if I took a picture of a statue of george washington that shows the plaque, my iPhone already OCR'd that and will show the photo.
2)"minor" is not how I would ever describe sending data based off my photos to them, regardless of how much it's been stuffed through a mathematical meat grinder.
3)Apple is usually very upfront about this sort of thing, and also loves to mention the most minor, insignificant, who-gives-a-fuck feature addition in the changenotes for "point" system updates. We're talking things like "Numbers now supports setting font size in chart legends" (I'm making that up but you get the point.)
This was very clearly an "ask for forgiveness because the data we want is absolutely priceless and we'll get lots of it by the time people notice / word gets out." It's along the lines of Niantic using the massive trove of photos from the pokemon games to create 3d maps of everywhere.
I specifically use iOS because I value my privacy (and don't want my cell phone data plan, battery power, etc to be a data collection device for Google.) Sending data based off my photos is a hard, do-not-pass-go-fuck-off-and-die line in the sand for me.
It's especially shitty because they've gated a huge amount of their AI shit behind owning the current iPhone model....but apparently my several generation old iPhone is more than good enough to do some AI analysis on all my photos, to upload data for them?
Fuck everyone Apple who was involved in this.
It's very clearly not, since they've gone to huge lengths to make sure they can't actually see the data themselves see the grandparent post.
Hear hear. As if they can do this but not Visual Intelligence, which is just sending a photo to their servers for analysis. Apple has always had artificial limitations but they've been getting more egregious of late.
Then perhaps the system is of poor design and needs further work before being unleashed on users…
Anything that leaves my devices should do so with my opt-IN permission.
This is not how you launch a privacy-preserving product if your intentions are good, this is how you slip something under the radar while everyone is distracted.
The choice probably looks to them like:
Wether B works is up to debate, but that was probably their only chance to have it ship from their POV.IMO giving up on having it widely used and just ship it turned off would be the best choice. But it's so obvious, there must be other ceitical reasons (good or bad) that's not an option.
"we had to sneak it out because people wouldn't consent if we told them" isn't the best of arguments
Or is it better to just trust that mathematics works and thus encryption is a viable way to preserve privacy and skip the dialog?
This is a much greater level of security than what you would expect from a bank, for example, who needs to fully decrypt the data you send it. When using your banking apps over HTTPS (TLS), you are trusting the CA infrastructure, you are trusting all sorts of things. You have fewer points of failure when a key for homomorphic encryption resides only on your device.
"Opting-in by default" is therefore not unsafe.
People won't trust homomorphic encryption, entropy seeding or relaying when none of it is transparent and all of it is enabled in an OTA update.
> This is what a good privacy story looks like.
This is what a coverup looks like. Good privacy stories never force third-party services on a user, period. When you see that many puppets on stage in one security theater, it's only natural for things to feel off.
What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. Plenty of people might just 'go with the flow', but putting what Apple did here in the same bucket as what for example Microsoft or Google does is a gross misrepresentation. Present it all as equals just kills the discussion, and doesn't inform anyone to a better degree.
When you want to take part in an interconnected network, you cannot do that on your own, and you will have to trust other parties to some degree. This includes things that might 'feel' like you can judge them (like your browser used to access HN right here), but you actually can't unless you understand the entire codebase of your OS and Browser, all the firmware on the I/O paths, and the silicon it all runs on. So you make a choice, which as you are reading this, is apparently that you trust this entire chain enough to take part in it.
It would be reasonable to make this optional (as in, opt-in), but the problem is that you end up asking a user for a ton of "do you want this" questions, almost every upgrade and install cycle, which is not what they want (we have had this since Mavericks and Vista, people were not happy). So if you can engineer a feature to be as privacy-centric yet automated as possible, it's a win for everyone.
People aren't making tradeoffs - that's the problem. Apple is making the tradeoffs for them, and then retroactively asking their users "is this okay?"
Users shouldn't need to buy a new phone to circumevent arbitrary restrictions on the hardware that is their legal property. If America had functional consumer protections, Apple would have been reprimanded harder than their smackdowns in the EU.
As for vectorised and noise-protected PCC, sure, they might have an opinion about that, but people rarely are informed enough to think about it, let alone gain the insight to make a judgment about it at all.
I did read the article, yes :) Maybe our photos are not sent bit-by-bit but enough data from the photos is being sent to be able to infer a location (and possibly other details) so it is the same thing: my personal data is being sent to Apple's servers (directly or indirectly, partially or fully) without my explicit consent.
At least the last time they tried to scan everyone's photos (in the name of the children) they pinky promised they'd only do it before uploading to iCloud, now they're doing it for everyone's photos all the time - it's disgusting.
You can imagine it as hashes (created locally), some characters of that hash from some random positions being used to find out if those can be turned into a query (which is compute intensive so they use PCC for that). So there is no 'data' about what it is, where it is or who it is. There isn't even enough data to create a picture with.
Technically, everything could of course be changed, heck, Apple could probably hire someone with binoculars and spy on you 24/7. But this is not that. Just like baseband firmware is not that, and activation is not that, yet using them requires communication with Apple all the same.
So it does it multiple ways:
- Finite sets and added noise, doesn't hurt performance too much but does make it nearly impossible to ID/locate a photo
- Encryption at rest and in transit
- Transit over hops they don't own
- Homomorphic Encryption during remote compute
The data it finds was available in two ways: the "result" and the vector embedding. Not sure which one you end up consuming since it also has to work on older models that might not be able to load the embeddings and perform adequately, but it doesn't matter since the data itself will be unique so you can't do parallel reconstruction, but it is also uniquely meaningless to anyone without a key. They are essentially computing on needles that aren't in a haystack, but in a needle stack.
The primitives all this is built on have been around for quite a while, including their HBONE implementation, the cryptographically hidden data distribution and the SEP. So far, it has been the only one of its kind outside of disjointed options like buying and operating your own HSM, a large TOR network and a yet to-be-invented self-hosted PCC solution (AMD was supposed to release something but they failed at that, just not as bad as Intel messed up with SGX).
Technically, even with everything else removed, just some good TLS 1.2+ and homomorphic encryption would have been more than any other mass market manufacturer has ever done in an effective way. But by adding the additional factors such as degrees of separation so they couldn't get in themselves (without breaking it for everyone in the process) is what makes this so much more robust.
I would be ok with this being a local feature, where I can download the signature database to my device and run the search locally (as you say), but as it stands some information about my photos (enough to detect places at least, possibly more in the future) is being sent out of my device. I want zero information about my photos to leave my device.
I mean, this is just wrong. Baseband firmware and carrier activation can be managed entirely independently of Apple, they just choose to manage it themselves. The number of places where Apple chooses to insert their own services as arbitrary middlemen has been a perennially worrying topic among Apple enthusiasts. It's not just disrespectful to people that pay a premium for fewer service advertisements, it's downright unsafe and does not reflect the sort of forward-thinking security that people in the industry respect.
There was a time when Apple focused on real and innovative product differentiation, but I'll be damned if you can give me a post-Wozniak example that isn't under antitrust scrutiny. Apple relies on marketing and branding to make people feel unsafe in a fundamentally insecure system - I don't respect that as a proponent of innovation and competitive digital markets.
Perhaps you are thinking about subscription activation (be it GSM or CDMA) and parameters for cell networks (which can indeed be consumed by the baseband, which will be running firmware supplied by the manufacturer, sometimes re-packaged in system images as done in OEM feature phones and many android phones).
Either way, macOS devices do the same thing (activation) as do iPads without cell networking. Same goes for radio firmware loading and updates. You'll find most wintel laptops doing the same for things like WiFi (regardless of softmac/halfmac/hardmac chips).
That’s starting to veer into unreasonable levels of conspiracy theory. There’s nothing to “cover up”, the feature has an off switch right in the Settings and a public document explaining how it works. It should not be on by default but that’s not a reason to immediately assume bad faith. Even the author of the article is concerned more about bugs than intentions.
Since they cannot convince users to enable this feature in good-faith, they are resorting to subterfuge. We know that Apple is vehement about pushing client-side scanning on users that do not want it, I do not believe for a second that this was a mistake or unintended behavior. If this was a bug then it would have been hotfixed immediately to prevent the unintended behavior from reaching any more phones than it already had.
This is illogical. If Apple wanted to engage in subterfuge they would simply compromise the OS.
When a company controls the entire stack either you trust everything they do. Or nothing.
If anything, the lesson Apple might take from this could be "adding the settings toggle was a bad idea, because nobody would have cared about this at all otherwise".
They worked very hard on security these past few months, so it should be all good, right?
I am not claiming for one moment that enabling this by default is OK. In fact, I have explicitly said it is not.
What I am saying is that it is ignorant to call this a cover up, because a cover up requires subterfuge. This feature has a freaking settings toggle and public documentation. Calling it a cover up is the type of uneducated rhetoric that makes these issues being brushed off by those in power as “it’s just a bunch of loonies conspiracy theorists complaining”.
Clearly you have not. If you did, you wouldn’t continue to give an example which is not equivalent.
No, it would not be “perfectly fine”, I just said it wouldn’t. You can do something wrong without it being a cover up.
Irrelevant.
This is Apple's proprietary software, running on Apple's computers, devices which use cryptography to prevent you from inspecting it or running software they don't control. Very few people have any idea how it actually works or what it actually does.
That there's some "public document" describing it is not evidence of anything.
> that’s not a reason to immediately assume bad faith
The mere existence of this setting is evidence of bad faith. The client side scanning nonsense proved controversial despite their use of children as political weapons. That they went ahead and did this despite the controversy removes any possible innocence. It tells you straight up that they cannot be trusted.
> Even the author of the article is concerned more about bugs than intentions.
We'll draw our own conclusions.
This is a dumb take. They literally own the entire stack Photos runs on.
If they really wanted to do a coverup we would never know about it.
This is a useful, harmless feature that does not comprise security or privacy in any way.
Users are smart enough to decide for themselves. Let them.
If someone wanted to get a hold of your cloud hosted data at that point, they would use their capacity to simply extract enough key material to impersonate a Secure Enclave. That that point, you "are" the device and as such you "are" the user. No need to make it more complicated than that.
In theory, Apple and other manufacturers would already use PQC to prevent such scenarios. Then again, QC has been "coming soon" for so long, it's doubtful that any information that is currently protected by encryption will still be valuable by the time it can be cracked. Most real-world process implementations don't rely on some "infinite insurance", but assume it will be breached at some point and just try to make it difficult or costly enough to run out the clock on confidentiality, which is all that really matters. Nothing that exists really needs to be confidential forever. Things either get lost/destroyed or become irrelevant.
They’re not. Jeff Johnson develops apps (specifically Safari extensions) for Apple platforms and frequently blogs about their annoyances with Apple, but they’re not a security researcher.
I would be surprised if doing noisy vector comparisons is actually the most effective way to tell if someone is in front of the Eiffel tower. A small large language model could caption it just as well on device, my spider sense tells me someone saw an opportunity to apply bleeding edge, very cool tech so that they can gain experience and do it bigger and better in the future, but they're fumbling their reputation by doing this kind of user data scanning.
Not really, it's been around for a bit now. From 2021:
> The other major reason we’re talking about HE and FL now is who is using them. According to a recent repository of PETs, there are 19 publicly announced pilots, products, and proofs of concept for homomorphic encryption and federated analytics (another term for federated learning) combined. That doesn’t seem like a lot … but the companies offering them include Apple,7 Google, Microsoft, Nvidia, IBM, and the National Health Service in the United Kingdom, and users and investors include DARPA, Intel, Oracle, Mastercard, and Scotiabank. Also, the industries involved in these early projects are among the largest. Use cases are led by health and social care and finance, with their use in digital and crime and justice also nontrivial (figure 1).
https://www2.deloitte.com/us/en/insights/industry/technology...
I do wonder why we don't hear about it more often though. "Homomorphic encryption" as a buzzword has a lot of headline potential, so I'm surprised companies don't brag about it more.
As far as why it's not more of a buzzword, it's far too in the weeds and ultimately consumers either trust you or they don't. And even if they don't trust you, many of them are still going to use Apple/Google/Facebook system anyway.
homomorphobia ?
Which 3rd party is that?
b) If we follow your tortured logic then every hop along the path from your phone to Apple will have one more data point on you. That's thousands of companies a day.
I'm also griping that "the data is encrypted !" is not a good enough excuse seeing as how we've known for years that the metadata is a bigger pot of gold for intelligence agencies. That my mac address is taking a photo and hitting a particular cell tower is pretty detailed information, even without knowing the content of the photo.
Consider these points:
1. Users don't need a PhD to understand "This feature will send data about your photos to Apple's servers to enable better search."
2. The complexity of the privacy protections doesn't justify removing user choice. By that logic, we should never ask users about any technical feature.
3. Many privacy-conscious users follow a simple principle: they want control over what leaves their device, regardless of how it's protected.
The "it's too complex to explain" argument could justify any privacy-invasive default. Would you apply the same logic to, say, enabling location services by default because explaining GPS technology is too complex?
The real solution is simple: explain the feature in plain language, highlight the benefits, outline the privacy protections, and let users make their own choice. Apple already does this for many other features. "Default off with opt-in" is a core principle of privacy-respecting design, regardless of how robust the underlying protections are.
Closest I come to presenting an opinion on the right way UX was "I'm not sure what the right call is here.". The thing I disagreed with was a technical statement "the only way to guarantee computing privacy is to not send data off the device.".
Privacy respecting design and tech is a passion of mine. I'm pointing out "user choice" gets hard as the techniques used for privacy exceed the understanding of users. Users can intuitively understand "send my location to Google [once/always]" without understanding GPS satellites. User's can't understand the difference between "send my photo" and "send homomorphicly encrypted locally differentially private vector of e=0.8" and "send differentially private vector of e=50". Your prompt "send data about your photos..." would allow for much less private designs than this. If we want to move beyond "ask the user then do it", we need to get into the nitty gritty details here. I'd love to see more tech like this in consumer products, where it's private when used, even when opted-in.
Apple can try to explain as best it can how user data is protected when they use the online service, and then the user makes a choice to either use the service or not.
In my case, I have don't even have a practical use for the new feature, so it's irrelevant how private the online service is. As it is, though, Apple silently forced me to use an online service that I never wanted.
1. The technical excellence of Apple's privacy protections (which you've explained well and seem robust)
2. The ethical question of enabling data transmission by default
Even with best-in-class privacy protections, the principle of user agency matters. A simplified prompt like "This feature will analyze your photos locally and send secure, anonymized data to Apple's servers to enable better search" would give users the basic choice while being technically accurate. The technical sophistication of the privacy measures, while commendable, doesn't override the need for informed consent.
I don’t think that that’s what the author is saying at all, I think he’s saying that Apple should let the user decide for themself if they want to send all this shit to Apple, freedom for the individual. They’re not saying “I dunno”
And the claims that this is good privacy/security are not at all obvious either. And who are those third-parties anyway? Did you verify each one of them?
Well I want them to fuck off.
Hidden in your commentary here is the fact that the vector representation of the image is _the contents of the image_. It very well may be that they cannot reverse the exact image. But it’s still a representation of the image that has to be good for something. Without being too familiar I would be willing to hazard a guess that this could include textual labels and classifications of what is in the image.
I don’t give a shit how good your internal controls are. I don’t have anything particularly interesting to hide. I still do not want you taking my pictures.
A private corporation has no business reading my files for its own benefit.
that's what i'm saying!! because of all these special privacy measures it's genuinely impossible for them to get any benefit from this. it is only used for your benefit to improve photo search on your local device.
they don't see any of the data, they simply perform computations blindly over the encrypted data that are then returned to your device to then decrypt the results locally and use them to improve local photo indexing for you.
Furthermore, Apples privacy stance is generally a sham as their definition of “human rights” doesn’t extend to China. Which either means Apple doesn’t respect human rights, or they don’t view Chinese people as human.
The Chinese government are debatably inhuman. They've literally censored the word "censorship." (Then they censored what people used euphemistically for censorship--"harmonious.") It's funny from the outside but also a miserable state of affairs in 2024.
Apples version of “human right” includes suicide nets as an alternative to treating people humanely. That’s why their stance is pure marketing - they have blood on their hands.
And guess what? You can’t use Google in China, and while Google isn’t by any means perfect, they aren’t Apple.
Now, that happened under a different type of Google (I’m sure they’ll go back now that they torched the culture), but Apple employees who walk around SV like they are better than everybody else - which people on their privacy teams very much do - are pathetically deluded.
All the megacorps are evil, and while Apple likes to put on a holier-than-thou act, it’s just as much bullshit for them to claim they value human rights as it is for Google to say they value privacy.
They value money, that’s it.
It's no longer a conspiracy. I think we are all over past that time(i.e with Snowden and Wikileaks). We live in a surveillance world and "They're guarding all the doors and holding all the keys".
It is simple for any user to evaluate risk the risk of their data being breached on 3rd party servers when their data isn't being sent off the device - there is none. It is only when corporations insist that they are going to send the data off your device whether you like it or not that evaluating risk becomes necessary.
A good privacy story actually looks like not sending any info to anyone else anywhere at any time.
There are way more egregious privacy concerns than sending non-reversibly encrypted noisy photos to Apple. Why draw the line here and not the far worse things happening on your phone and computer right now?
Almost every single app today interacts with the network in some way.
You would be constantly annoying the user with prompt after prompt if you wanted to get consent for sending any relatively harmless data off the device.
My Calculator app does not need to call home.
A good portion of the apps I use fall into this category, and a straightforward mechanism to opt them out of access would be welcome.
They have option to disble if they care.
"Computer, I command thee to send this and only this information over the channel of my choosing, using following encryption scheme, for here be my seal of approval for anyone who might want to verify, and here be the key"
"Sicut Vult"
On Apple's scale, most people care about the things they can do, not about how it happens. For that reason, default matters when the option is only about the internal process pipeline and privacy.
As a result, it is enough to showcase that in case some expert investigates the matter, they show that privacy is considered in a reasonable level.
Maybe some day in the future these things are common knowledge, but I fear that the knowledge gap just increases.
Debian has popcon, which must be enabled explicitly and that is it. It sends the list of the installed packages.
Of course it's fine to not desire privacy, or to desire a privacy level that is less than private. That's up to you. I liked the privacy of my old Canon digicam that had no internet. A photo app on a phone that sends stuff over the network might bring some useful functionality in return, but it can only be considered a regression in terms of privacy.
What Apple has implemented is a LOT closer to "private" than "not private"
Between the quote you pointed out and:
"One thing I do know, however, is that Apple computers are constantly full of privacy and security vulnerabilities, as proved by Apple's own security release notes" which just reeks of survivorship bias.
I think the final call of what is right here _shouldn't_ be informed by the linked article.
IMO, enabled by default without opt-in is absolutely the right call when judging between 1: Feature value 2: Security risk 3: Consent Fatigue.
If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.
If you disagree with my prior statement because of how Apple locks you into Photos, :shake_hands:.
If Enhanced Visual Search is still enabled by default in lockdown mode, then I think that's worth a conversation.
Why in the world am I supposed to be an expert on homomorphic encryption? How many people in the world are experts on homomorphic encryption?
> which just reeks of survivorship bias.
What does that even mean in this context?
> 1: Feature value
What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it.
> If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.
That's ridiculous. Apple itself has said, "Lockdown Mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature." https://support.apple.com/105120
Lockdown mode is basically for famous people and nobody else.
No one, at any point, implied you had to be an expert on homomorphic encryption. But if you're going to evaluate the security risk of a feature, and then end up on the front page of HN for said security risk, I think it's fair to criticize your lack of objectivity (or attempt at objectivity) by way of not even trying to understand the technical details of the blog.
I will say I think my word choice was unnecessarily harsh, I'm sorry. I think I meant more indifference/inattention.
> What does that even mean in this context?
Apple's list of Security releases is long and storied. By comparison, the Solana Saga Web3 phone's list of security releases is short and succinct. Therefore, the Solana Saga must be more secure and has better security than an Apple device!
> What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it.
The marketability of a feature is not necessarily correlated with its value. Some features are simply expected and would be silly to advertise, i.e. the ability to check email or text friends. Other features are difficult to evaluate efficacy, so you release and collect feedback instead of advertising and setting false expectations.
> Lockdown mode is basically for famous people and nobody else.
Similar to Feature value, that audience of that statement is your average person (read: does not read/post on hacker news). Based off the your pedigree, I feel as though you probably know better, and given your "no tolerance for risk" for such a feature, it's something worth at least considering, and definitely isn't ridiculous.
I think it's great you started this conversation. I disagree with your opinion, and that's okay!! But I don't think it's particularly beneficial to any discourse to 1. Imply that you are evaluating security risk 2. Be given a well written technical article so that you are able to make an informed decision (and then share that informed decision) 3. Ignore relevant information from said article, make an uninformed decision 4. Be surprised when someone says you made an uninformed decision 5. Imply the only way to make an informed decision would be to be an expert in the relevant fields from the technical article
Anyway - thanks for writing and replying. Creating and putting yourself out there is hard (as evidenced by my empty blog that I promised I'd add to for the past 2 years). And my criticism was too harsh.
I wouldn't characterize that as the point of my blog post. It's primarily about user consent, or lack thereof.
> and then end up on the front page of HN for said security risk
I have no control over that. I didn't even submit the article to HN.
> Apple's list of Security releases is long and storied. By comparison, the Solana Saga Web3 phone's list of security releases is short and succinct. Therefore, the Solana Saga must be more secure and has better security than an Apple device!
This is a red herring. I wasn't comparing Apple security to any other company's security. I was merely pointing out the possibility of bugs and vulnerabilities in Apple's new feature.
> Other features are difficult to evaluate efficacy, so you release and collect feedback instead of advertising and setting false expectations.
Well, I've now given my feedback on the new feature.
> Similar to Feature value, that audience of that statement is your average person (read: does not read/post on hacker news). Based off the your pedigree, I feel as though you probably know better
I'm not sure I understand. Are you claiming that Apple, in its support document, is deliberately mischaracterizing Lockdown Mode?
> But I don't think it's particularly beneficial to any discourse to 1. Imply that you are evaluating security risk
As I've said above, I wasn't.
> 3. Ignore relevant information from said article
I didn't ignore the relevant information from said article. I read the article, but some of the technical details are beyond my current knowledge.
> make an uninformed decision
What uninformed decision are you talking about?
> 4. Be surprised when someone says you made an uninformed decision
I'm surprised because I have no idea what "uninformed decision" you mean.
> 5. Imply the only way to make an informed decision would be to be an expert in the relevant fields from the technical article
I didn't imply that at all. To the contrary, I insisted that the decision to enable the feature should be up to the user, not up to Apple.
I don't think I need to specify that by uninformed decision I mean evaluating the security risk of the feature. I think I criticized too harshly, and you're (understandably) not engaging with me fairly in retaliation. If you actually want to engage with me and discuss this further, feel free to shoot me an email (in my about section). Otherwise, obligatory https://www.paulgraham.com/vb.html.
I'm trying, but obviously I'm failing.
> I don't think I need to specify that by uninformed decision I mean evaluating the security risk of the feature.
For the third time, that wasn't what I was trying to do with the blog post.
> you're (understandably) not engaging with me fairly in retaliation
I don't think you're understanding me either. I'm not retaliating. I was trying to clarify.
A good privacy story starts with "Do you consent" and not transmitting a byte if you answer "no"
It sounds like it only needs a few extra lines of code to get exactly what they wanted before, they just packaged it differently and we all fell for it like frogs getting boiled in water.
https://en.m.wikipedia.org/wiki/Boiling_frog
The CSAM filtering was a best of class implementation. I'm pretty sure I'm one of maybe a dozen people who actually read the spec before throwing a hissy-fit about "muh privacy!"
The only actual "flaw" was that maybe a state-level actor could make it scan for bad stuff on your device.
BUT they can do it to your cloud data _today_. And if you disabled cloud uploads, the local scanning was disabled too.
No, they can't if you enable ADP[1].
[1] https://support.apple.com/en-us/108756
Not at all. A good privacy story is not sending this data anywhere.
how about they don't send anything about my photos to their servers and i get to keep my shit on my own device
i suppose we're past that to the point where techbros like you will defend personal data exfiltration because.. uhh idk? trillion dollar corporation knows best?
I am sure.
The right call is to never send any data from the device to anyone unless the user explicitly tells the device to do it.
The only thing the device should do is whatever its user tells it to do.
The user didn't tell it to do this. Apple did.
> But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true
Irrelevant. It was never about privacy to begin with. It was always about power, who owns the keys to the machine, who commands it.
Vectorization, differential privacy, relays, homomorphic encryption, none of it matters. What matters is the device is going behind the user's back, doing somebody else's bidding, protecting somebody else's interests. That they were careful about it offers little comfort to users who are now aware of the fact "their" devices are doing things they weren't supposed to be doing.
Let's look at the primary alternative, Android. It generally does not provide you this level of granular control on network access either without rooted hacks. Apps and the phone vendor can do whatever they want with far less user control unless you're a deep Android nerd and know how to install root-level restriction software.
No one's advocating for granular opt in either. There are much better ways. We have to make it so that data is toxic to corporations. Turn data into expensive legal liabilities they don't want to deal with. These corporations should not even be thinking about it. They should be scrambling to forget all about us the second we are done with them, not covertly collecting all the data they possibly can for "legitimate" purposes. People should be able to use their computers without ever worrying that corporations are exploiting them in any way whatsoever.
The Android situation is just as bad, by the way. Rooting is completely irrelevant. You may think you can hack it but if you actually do it the phone fails remote attestation and the corporations discriminate against you based on that, usually by straight up denying you service. On a very small number of devices, Google's phones ironically, you can access those keys and even set your own. And it doesn't matter, because the corporations don't trust your keys, they only trust the keys of other corporations. They don't care to know your device is secure, they want to know it's fully owned by Google so that you can't do things the corporations don't like.
It's not something that can be solved with technology. Computer freedom needs to become law.
What a good privacy story looks like is that my photos aren’t sent anywhere in any way shape or form without explicit opt in permission.
Most people broadcast their lives on social media, happily opt in to all sorts of schemes that track them just for minor conveniences. For people like that, the idea that the privacy protection outlined by the OP isn't enough rings really hollow.
Or to put it bluntly, at some point this really stops feeling like a practical debate, and more of an ideological one.
If only you and three other people have a unique file, Apple knows you are a group.
I have an idea: send an encrypted, relayed, non-reversible, noised vector representation of your daily phone habits and interactions. That way you can be bucketed, completely anonymously of course, with other user cohorts for tracking, advertising, and other yet-to-be discovered purposes.
It's a great privacy story! Why would you have a problem with that?
I don't know, I'm sure we'll figure something out once we have your data!
Restating the layers above, in reverse:
- They don't see either your data or the results of the query (it's fully encrypted even from them where they compute the query -- this is what homomorphic encryption means)
- Even if they broke the encryption and had your query data / the query result, they don't know who "you" are (the relay part)
- Even if they had your query hash and your identity, they couldn't reverse the hash to identify which specific photos you have in your library (the client-side vectorization + differential privacy part), though by the this point they could know what records in the places database were hits. So they could know that you took a photo of a landmark, but only if the encryption and relay were both broken.
https://en.wikipedia.org/wiki/Homomorphic_encryption
> Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first having to decrypt it. The resulting computations are left in an encrypted form which, when decrypted, result in an output that is identical to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and outsourced to commercial cloud environments for processing, all while encrypted
And the way the data comes back to you is via the third-party relay which knows your IP but nothing else
So, while Apple doesn't know the ID of the person sending the data, they have a 'room number' that links back to an ID.
If Apple were to decide to scan photos for pictures of 'lines of white powder' they couldn't tell the police your name but they could say that the 3rd party knows who you are.
> Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are.....
Yes, this is the whole point.
If apple were to add additional clusters that match "sensitive" content and endeavour to put them in their own shards distinct from landmarks, they defeat the homomorphic encryption part while still technically doing it.
The DP part can be defeated with just statistics over time; someone with any volume of sensitive content will hit these sensitive clusters with a higher likelihood than someone generateing noise injected fake searches.
The OHTTP part can be defeated in several ways, the simplest of which is just having a clause in a non-public contract allowing apple to request logs for some purpose. They're paying them and they can make up the rules as they go.
There's also LibrePhotos which is packed with features but doesn't have as much polish as immich. They do however have a list of python libraries that can be used for offline/local inference for things like having an image model create a description of a photo that can benefit the full text search. [2]
[0] https://immich.app/
[1] https://github.com/alangrainger/immich-public-proxy/blob/mai...
[2] https://docs.librephotos.com/docs/user-guide/offline-setup
* Haven't actually tried this plugin yet, my weekend project is setting up my caddy VPS to tunnel to the immich container running on my Synology NAS
100% private is nice but not practical for something like photos in my opinion. unless you have someone in the family that can take over after you’re gone and is very technical.
On Android, NetGuard uses a "local VPN" to firewall outgoing traffic. Could the same be done on iOS, or does Apple network traffic bypass VPNs? Lockdown mentions ads, but not Apple servers, https://lockdownprivacy.com/.
Apple does publish IP ranges for different services, so it's theoretically possible to block 17.0.0.0/8 and then open up connections just for notifications and security updates, https://support.apple.com/en-us/101555
Why is NetGuard more trustworthy than Apple?
On iOS, Lockdown firewall is open-source, https://github.com/confirmedcode/Lockdown-iOS
But even blanket blocking of all Apple IP ranges probably wouldn't do anything here. As documented, your device sends noise injected image vectors to OHTTP relays and doesn't contact Apple directly. By definition those relays are operated by 3rd parties. So if you consider this type of data "phoning home" you'll need to find the IPs of all of OHTTP relays iOS uses. (or block the traffic that looks up the OHTTP relays).
Since iOS was derived from macOS, perhaps Apple could restore the link between network traffic and process.
I do not assume that Apple managers are that degenerate idiots pushing through trust eroding marginal idiocy like this.
Summary: "Homomorphic encryption (HE) is a cryptographic technique that enables computation on encrypted data without revealing the underlying unencrypted data to the operating process. It provides a means for clients to send encrypted data to a server, which operates on that encrypted data and returns a result that the client can decrypt. During the execution of the request, the server itself never decrypts the original data or even has access to the decryption key. Such an approach presents new opportunities for cloud services to operate while protecting the privacy and security of a user’s data, which is obviously highly attractive for many scenarios."
Both the search and photos settings were on for me in both iOS and MacOs. Completely unacceptable Apple
It seems likely to me that concerns like those of the article or some of the comments in this thread are irrelevant to Apple’s bottom line. A concern some customers may actually have is data usage, but I guess it’s likely that the feature is off if in low data mode.
I wonder if this particular sort of issue would be solved by some setting for ‘privacy defaults’ or something where journalists/activists/some corporate IT departments/people who write articles like the OP can choose something to cause OS updates to set settings to values that talk less on the network. Seems hard to make a UI that is understandable. There is already a ‘lockdown mode’ for iOS. I don’t know if it affects this setting.
It's a brand new feature of iOS 18 and macOS 15. It did not exist in iOS 17 or macOS 14.
Moreover, my macOS 15 volume was a totally fresh, clean install on a test Mac.
I mostly think the reaction to this article is overblown because it appeals popular ideas here about big tech. I think one should be wary of Apple’s claims about privacy: the reason is competition with Google and so they want users to be distrustful of the kinds of features that Google are better at implementing (I don’t want to say Apple isn’t trying to do the right thing either – if you look at accessibility, the competition was very bad for a lot of things for a long time and Apple was good despite the lack of commercial pressure). But I think one should also be wary of articles that make you angry and tell you what you suspected all along. (eg see the commenter elsewhere who doesn’t care about the details and is just angry). It’s much easier to spot this kind of rage-bait piece when it is targeting ‘normal people’ rather than the in-group.
The article was published by an Apple developer and user, i.e., myself, on my personal blog, which is followed mainly by other Apple developers and users. My blog comprises my own personal observations, insights, and opinions. If you see any rage, it would be my own personal rage, and not "bait". Bait for what?
The headline is defensible but, in my opinion, quite sensationalised. People are likely to interpret it as being for an article making much stronger claims than the article actually does. I think a lot of the interactions people had with this submission, especially early on, were because the headline made them mad, rather than because of its contents. I think if one is interacting with some submission here due to a maddening headline, one should be wary about such interactions being driven by emotion, leading to poor discussion that is not particularly anchored to the topic of the article, rather than being driven by curiosity.
How would you write the headline?
There's always a criticism of headlines, but headlines are necessarily short. It's like critics want the entire article text to appear in the headline, which is impossible.
I don't know what defensible but sensationalized is supposed to mean.
> I think a lot of the interactions people had with this submission, especially early on, were because the headline made them mad, rather than because of its contents.
That's pure speculation on your part, because the headline is very short and vague. In any case, it's not my fault if people read the headline but not the article. I want people to read the article, not just the headline.
I think the early comments on this submission were because the headline made them mad because they were incurious and not particularly related to the topic of the article making – most could be under any article about Apple Photos. One early comment was about switching to alternatives to Photos and another was, paraphrasing, ‘I’m so angry. I don’t care about homeomorphic encryption or differential privacy’. Others seemed to follow the theme of the latter. After a while a comment attempted an overview of some of the technical details, which I thought was better.
Perhaps a more precise complaint is that many of the early comments didn’t really depend on the contents of the article – they could go under many Apple submissions – and I think it’s more likely for comments like that to be written while incensed.
I don’t think you’re to blame for the comments people choose to leave.
In my opinion, these are actually misleading headlines that don't represent the content of the article. In fact, I've never used the Enhanced Visual Search feature, nor do I know exactly how much data is sent to Apple.
My article was never intended as a kind of feature introduction that you might see in the news media. The main point was always about user consent for uploading data to Apple. To be clear, my article is and has always been a complaint. Thus, I think the headline is accurate. I am complaining, and readers should know that.
> another was, paraphrasing, ‘I’m so angry. I don’t care about homeomorphic encryption or differential privacy’.
> many of the early comments didn’t really depend on the contents of the article
The headline of the article didn't mention homeomorphic encryption or differential privacy, so they must have read the contents.
And I care "why" exactly? It was turned on by default on my phone without my consent, it's a privacy violation, nothing else matters in that case.
(And to be clear I’m not even bothered by the signature check)
At a certain point you have to figure that they realize it doesn’t matter short of some government entity forcing them to stop. At the very least the protections they put in place (homomorphic encryption, etc) are more than I think most other companies would ever bother doing.
Shit like this, reasonable in isolation or not, undermines that story completely. If they are so willing to just outright lie on a massive billboard, what else will they do when profits demand it?
Perhaps this is marketing genius (from an AAPL-shareholder POV)?
----
I'm laughing at the insanity of all this interconnectivity, but an NDA prevents me from typing the greatest source of my ironic chuckles. Described in an obtuse way: a privacy-focused hardware product ships with an undisclosed phone-home feature, letting the feds see every time you use the product (to produce a controversial product, at home).
Kick in my fucking door / sue me: it'll just re-enforce that I'm correct about concessionary-allowances...
----
Even privacy-focused hardware manufacturers will allow undisclosed usage tracking (in order to continue existing, themselves). In my example, the OEM delivers a physical product which allows users to make tangible objects, at home. Every time you launch the hardware's control software (to make more controversial objects), it phones home.
This does not happen.
You are saying that every network request is going to a target controlled by the US government.
https://en.wikipedia.org/wiki/Five_Eyes
https://en.wikipedia.org/wiki/Room_641A
Why yes, I did happen to work in US data centers throughout the 2010's... including during Snowden's [then adamently-denied] revelations.
"If you're taking flak, you're near the target."
Particularly among "controversial" product manufacturers.
----
Entirely unrelated: I recently visited my favorite hometown (Austin) museum, UT's Harry Ransom Center. Their P.E.N. exhibit has an exposé on how various mid-20th Century authors were in fact funded by The Ford Foundation (a known CIA "investment vehicle"), including Aldous Huxley (author of Brave New World). One of the exhibits features multiple author signatories adamently denying this CIA-funding, over the 1950's decade... ultimately this conspiracy theory ended up being true/revealed. Adds an entirely new dimension to The Savage's [BNW's main character] decision to hang himself.
For context, stationary desktop machines comprise single-digit percentages of Mac sales: https://appleinsider.com/articles/24/03/06/macbook-pro-and-m...
Every other product sold by Apple (Mac notebooks, iPhones, watches, etc) are designed to be carried around with you, happily beaconing your presence to every network and to every other Apple device that will listen. I have seen this with my own eyes where my M3 MBP showed up on my WiFi AP status page as soon as I got home even though it was supposedly “““asleep””” in my bag.
You can randomize your MAC address. Built in OS feature.
What I mean is, an M3 MacBook has essentially the same processor as a cell phone. Of course it’s able to perform activities when the system is “asleep.” It even has specific online behavior labeled as a feature that can be turned on/off (Power Nap).
https://support.apple.com/guide/mac-help/turn-power-nap-on-o...
This is behavior that is desired by Apple’s users.
I sometimes wish that more Apple users who aren’t satisfied would stop complaining while continuing to throw money at Apple and buy themselves a Linux laptop. If you want a laptop with special features like a WiFi kill switch you should buy one of those, not buy a consumer appliance device that is marketed toward convenience-oriented home users.
My work machine — not my choice. Don't let reality get in the way of your smug assumptions about me though.
Now they probably don't care about you personally, but you'd be surprised how many people are connected indirectly to a person of interest. Google "Maher Arar" for the horrific consequences of false positives.
I get called "paranoid," often, but I'm being honest when responding to people requesting my phone number: "you don't want my number in your phone, because it'll associate you with people you probably don't want to be associated with" [see Five Eyes' 3-Degrees of Separation].
“Enhanced Visual Search in Photos allows you to search for photos using landmarks or points of interest. Your device privately matches places in your photos to a global index Apple maintains on our servers. We apply homomorphic encryption and differential privacy, and use an OHTTP relay that hides IP address. This prevents Apple learning about the information in your photos. You can turn off Enhanced Visual Search at any time on your iOS or iPadOS device by going to Settings > Apps > Photos. On Mac, open Photos and go to Settings > General.”
Maybe this is exactly what the GDPR does. But it is what I would appreciate as a user.
I have seen how sending metrics and crash dumps from a mobile device can radically improve the user experience.
I have also seen enough of the Google bid traffic to know how much they know about us.
I want to enable sending metrics, but most of the time it's a radio button labeled "metrics" and I don't know what's going on.
> the only way to guarantee computing privacy is to not send data off the device.
> It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations. [...] By enabling the "feature" without asking, Apple disrespects users and their preferences. I never wanted my iPhone to phone home to Apple.
Regardless of how obfuscated or "secure" or otherwise "privacy-protecting" the feature is, the fact is that some information derived from one's personal content is transmitted, without prior consent. Even if the information is protected, all network queries are information. A timestamp that proves you took a certain action at a certain time (like taking a photo, assuming stuff is sent to this service immediately upon adding a new photo), from a certain location (by correlating your location information at that time), etc etc.. and that's just the tip of the iceberg. Transmitting information from a user's device without their explicit consent is a violation of their privacy.
For ex: I sent a friend a photo of my puppy in the bathtub and her Airpods (via iphone) announced "(name) sent you a photo of a dog in a bathtub". She thought it was really cool and so do I personally. That's a useful feature. IDK how much that requires going off-device though.
For comparison Hunyuan video encodes a shit-ton of videos and rudimentary real world physics understanding, at very high quality in only 13B parameters. LLAMA 3.3 encodes a good chunk of all the knowledge available to humanity in only 70B parameters. And this is only considering open source models, the closed source one may be even more efficient.
I’m really curious how this feature is considered useful. It’s cool, but can’t you just open the photo to view it?
There is a large number of people out there who receive hundreds of notifications (willingly but mostly unwillingly) daily from apps they have installed (not just messengers), and nearly all of them can't cope with the flood of the notifications. Most give up on tending to the notifications altogether, but some resort to the notification summaries which alleviate the cognitive overload («oh, it is a picture of a dog that I will check out when I get a chance to, and not a pic of significant other who got themselves into a car accident»).
The solution is, of course, to not allow all random apps to spam the user with the notifications, but that is not how laypeople use their phones.
Even the more legit apps (e.g. IG) dump complete garbage upon unsuspecting users throughout the day («we thought you would love this piece of trash because somebody paid us»).
You can disable IG trash notifications, for example.
Most people using the device will never go into the notification settings, if an app on install tells you they need notifications to tell you about your driver arriving…and then sends you marketing notifications on the same channel most people will just accept that.
Frankly, hundreds of daily notifications is the reality we can't change, and the question is: why do people choose to succumb to the misery of it rather than availing themselves of the proper implements so readily at their disposal?
So you jumped to a conclusion based on an incorrect premise. This is easy to see that this does not happen immediately after taking a photo. One the network traffic will show this (and it won’t), two homomorphic encryption is expensive so it cannot. Photos classically doesn’t sync on demand, as most iPhone users will know by way if it telling you this in the photos app when it does sync. Most expensive operations are queued up for when the device is plugged in (and on WiFi) because it’ll otherwise drain battery.
I could see an argument in the metadata (though there are multiple assumptions involved there, not least that they don't truly do OHTTP but instead conspire to learn at what timestamp a user took a picture), but if you already don't trust in what is essentially math, I'm not sure where the uncertainty and doubt ends
Here, Apple does that for you.
Everybody knows that there is a network infrastructure where your content flows through. You willingly accept that as a connected device user because it is necessary to be connected.
What Apple did is not necessary and users don't know about it.
I do want to point out that Apple doesn’t get to see your photo. Homomorphic encryption is really cool in that way. Apple isn’t able to decrypt the photo and the results they produce are also encrypted. That’s the beauty of homomorphic encryption. I can send you an encrypted spreadsheet and you can compute the sum of a column for me. I’m the only one who can see the actual sum since you can only see the encrypted sum.
1) iOS creates additional fake queries, and all queries pass through scheduler that ensures you can use time-of-lookup to either discriminate real queries from fake queries, or identify when a photo was taken.
2) All queries are performed anonymously, with the use of a third party relaying service. So there’s no way for Apple to tie a specific query back to a specific device, or even IP address.
Between those two mitigating features. Getting hold of an individuals personal data using this feature requires you to first compromise the targets phone, to disable the fake queries. Then compromise the relaying party to correlate queries back to a specific IP address.
If you can manage all that, then quite frankly you’re a fool for expending all that effort. When you could just use your iOS compromise to have the device send you its location data directly. No need to faff about waiting for your target to take photos, then track multiple landmark lookups, carefully collecting a few bits of additional data per query, until you finally have enough to identify the location of your target or targets.
The whole thing reminds me of XKCD 538.
https://machinelearning.apple.com/research/homomorphic-encry...
Similarly here, it feels like the author is leaning into a knee jerk reaction about invasion of privacy without really trying to evaluate the effectiveness of the technologies here (client side vectorization, differential privacy, OHTTP relays, and homomorphic encryption).
Though I 100% agree Apple should ask the user for consent first for a feature like this.
Someone reply with a link to the source code so I can see exactly what it is doing, without having to take an internet rando's word for it.
Better yet, let me compile it myself.
After all, you wouldn’t actually be trusting the source code given to you to match what’s running on your device, would you?
> After all, you wouldn’t actually be trusting the source code given to you to match what’s running on your device, would you?
That's why the best practice in the industry follows reproducible builds.
Of course it is not the whole technology stack, but it is something at least. If your evaluation leads to potential problems, you can create issues right there on the github project!
I hate to jump to conspiracy theories, but the mechanism seems like a more effective version of their prior CSAM scanning scheme[1]. Best part? It doesn’t even require iCloud usage.
1: https://www.macworld.com/article/1428633/csam-photo-scanning...
Let go of the freaking tether.
And let me choose my own photo editing service provider.
Um, because that is what their customers are asking for?
> Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest.
I guess it's useful if you get very lost and don't recognize the pyramids right in front of you, and forgot you have a GPS in the phone.
Is the complexity and security risk really worth it for such a niche feature?
It's also funny that Apple is simultaneously saying "don't worry the photo is encrypted so we can't read it" and "we are extracting data from the encrypted photo to enhance your experience".
How many landmarks are there to justify sending your data off? Can't the database be stored on the device?
The usual context is "oh I saw some dolphins forever ago. let me see if I can find the photos..."
But now you suddenly need the cloud to have it recognize the Eiffel tower.
Last night I was able to find the image using the single word "creme". Definitely saved the story.
Locally, a small ML model identifies potential POIs in an image.
Another model turns these regions into a series of numbers (a vector) that represent the image. For instance, one number might correlate with how "skyscraper-like" the image is. (We don't actually know the definition of each dimension of the vector, but we can turn an image that we know is the eiffel tower into a vector, and measure how closely our reference image and our sample image are located)
The thing is, we aren't storing this database with the vectors of all known locations on our phone. We could send the vector we made on device off to Apple's servers. The vector is lossy, after all, so apple wouldn't have the image. If we did this, however, apple would know that we have an image of the eiffel tower.
So, this is the magic part. The device encrypts the vector using a private key known only to it, then sends this unreadable vector off to the server. Somehow, using Homomorphic Encryption and other processes I do not understand, mathematical operations like cosine similarity can be applied to this encrypted vector without reading the actual contents of the vector. Each one of these operations changes the value, which is still encrypted, but we do not know how the value changed.
I don't know if this is exactly what Apple does, I think they have more efficient ways, but theoretically what you could do is apply each row in your database to this encrypted value, in such a way that the encrypted value becomes the name of the POI of the best match, or otherwise junk is appended (completely changing the encrypted value) Again, the server has not read the encrypted value, it does not know which row won out. Only the client will know when it decrypts the new value.
Apple has a database trained on multiple photos of each landmark (or part of a landmark), to give a likelihood of a match.
Homomorphic encryption means that the encrypted mathematical form of a potential landmark from the phone can be applied to the encrypted set of landmark data, to get an encrypted result set.
The phone can then decrypt this and see the result of the query. But anyone else sees this as noise being translated to new noise, including Apple's server.
The justification for this approach is storage - the data set of landmarks can only get larger as the data set gets more comprehensive. Imagine trying to match photos for inside castles, cathedrals and museums as examples.
seems to me at that point, the server knows what segment of the overall dataset is being returned.
Here's a theoretical way I wrote in another comment:
> I think they have more efficient ways, but theoretically what you could do is apply each row in your database to this encrypted value, in such a way that the encrypted value becomes the name of the POI of the best match, or otherwise junk is appended (completely changing the encrypted value) Again, the server has not read the encrypted value, it does not know which row won out. Only the client will know when it decrypts the new value.
They do something like this, using homomorphic encryption. Whatever they do, there is no doubt they incur serious performance hits.
You may also be interested: https://arxiv.org/abs/2406.06761
Right, I've seen similar engineering efforts to target this sort of functionality fail because of the computational cost and resulting latency. I'm curious to read the paper for the tradeoffs they made toward practicality at Apple's scale of users.