I know what it does and how it works. I just don't like being considered a potential criminal without any reason.
For starters they should exclude photos made on the phone's own camera. Because it's literally impossible for a just-taken photo to appear in this database since that only contains already known content found in the wild. And most people's photos would be original content. So it would alleviate a lot of concern while not harming Apple's goals.
If those goals are indeed what they say they are, of course.
I don't think this algorithm is meant to capture that anyway. It's relying on content staying digital. It can deal with cropping according to its developers but I doubt it will capture a photo of a photo if it really has a false positive chance of one in a trillion.
Also, this is not a viable distribution method anyway. Every photo introduces more noise. Like dubbing tapes back in the day but worse.
Perceptual hashes between a source image and its derivatives will be similar if they kind of look similar to one another. That's the point of perceptual hashing.
That's a distinction with no functional difference. Two images that were modified from a source image will sometimes kind of look like one another, and so will two images that coincidentally look like one another. The first two will have similar or the same hashes, as will the latter two. That's how you get false positives.
And it's incorrect. The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 etc image and then transformations are applied to them to exaggerate features.
If two images look kind of the same when shrunken down, they will have the same or similar hashes. If two images kind of look the same when shrunken down, then their parent images will also kind of look the same.
Please read the OPs of the two links I posted. They're both from people who work in this field. The latter link is from someone[2] who invented many perceptual hashing methods himself that are used widely across the industry. Both articles touch on this subject, and the first[1] one includes two photo examples. I have built products using these methods, and what is said by these two experts matches my experiences.
It’s not a distinction without a difference. ‘Kind of like’ is going to be read by most people as ‘easily fooled’. That is a reflection of the false positive rate. It all depends on the quality and tuning of the algorithm and how it is deployed. If you are going to imply false positives are common, then you need to back it up.
Nobody is saying false positives are impossible.
Apple is saying false positives are on the order of one in a trillion per user account per year. That doesn’t sound like something that matches images that are only ‘kind of similar’. Yes - cryptographic hashes have much lower false positives rates even than that, but that is a distinction without a difference since both make the risk negligible.
> The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 image and then
Which is it for Apple’s hashes?
There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction.
What can you tell us about the likelyhood of Apple’s hashes to create false positives?
> It’s not a distinction without a difference. ‘Kind of like’ is going to be read by most people as ‘easily fooled’.
I'm not really concerned with what you're afraid most people will think. Two images that kind of look like one another will have the same or similar hashes. There are literal examples of this in the links I posted above. And it's literally the point of perceptual hashing, to find images that look similar to a source image by comparing hash similarity.
> If you are going to imply false positives are common, then you need to back it up.
I just did with two links I posted above. Twice.
> Apple is saying false positives are on the order of one in a trillion per user account per year.
Sounds like a claim that wasn't replicated or independently verified. Of course Apple is going to say their system is nearly perfect, that's what all companies do. The onus is on Apple to prove that their marketing claims reflect reality.
> There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction.
The onus is on Apple to demonstrate that their methods are remarkably different from the rest of the science and industry.
This is like saying the normal principles of computing don't apply to new Apple products because they might have invented a new brand computing paradigm that isn't anything like any classical or quantum computer mentioned in scientific literature at all. Yeah, maybe they did, but it's unlikely and the onus is on Apple to prove it.
> I'm not really concerned with what you're afraid most people will think.
What matters is not what I think, but whether you care about making misleading comments.
> > If you are going to imply false positives are common, then you need to back it up.
> I just did with two links I posted above. Twice.
No, you posted some links that are not about Apple’s system, and you can’t explain how they apply presumably because you don’t understand what Apple is doing.
> Sounds like a claim that wasn't replicated or independently verified. Of course Apple is going to say their system is nearly perfect, that's what all companies do. The onus is on Apple to prove that their marketing claims reflect reality.
So this tells us you don’t know what algorithm Apple is using…
…And are accusing Apple of lying, when it is clear that you haven’t read about how they avoid false positives.
I think the onus is on you to prove your accusation.
> This is like saying the normal principles of computing don't apply to new Apple products because they might have invented a new brand computing paradigm that isn't anything like any classical or quantum computer
That just silly. It’s doesn’t take breaking the laws of quantum or classical computing to build a system with a low false positive rate.
One obvious way would be to leverage multiple images rather than just one. Increasing the sample size of a population sample generally reduces the false positive rate.
Have you considered that someone could build a system this way?
Very patronizing of you. You clearly never learned that both classical (like your "sample size" example) and advanced statistical models are susceptible to first-order evasion attacks (or “Adversarial Examples”) that fool models at run-time. But go trust Apple's 1/1.000.000.000.000 claims.
In particular, false positives with perceptual hashes are not because they are similar in semantic content but because they are similar in whatever features the neural network determined stay stable across transformations. Your fall colors landscape photo is just as liable to be a neural hash match as your college sweetheart’s nudes.
He works with software. He probably also knows that to filter out specific files in a filesystem, it requires “scanning” all files and checking the ones that really represent image and video blobs, presumably for CSMA post processing. For doing so, one really needs to scan everything. The way I see it, there is really no confusion.
Right now, Apple says they chose to configure it such that the feature runs only on the files that are uploaded to iCloud using iCloud Photos, and end users have no way to confirm whether this claim is actually true.
People who sincerely believe this is accidentally possible should be constantly freaked out that iCloud Backups is “accidentally” uploading all their photos to a warrant accessible database.
> Apple says they chose to configure it such that the feature runs only on the files that are uploaded to iCloud using iCloud Photos
This must be implemented in software. A bug in this software that causes the "only on files that are uploaded" return the wrong boolean value means it will scan local photos.
I never said random files on the filesystem. That's your strawman. In any case, it could well happen as well. Something is deciding what to scan, and that's software.
It's part of the process that uploads the photos to iCloud. If you think through the likely implementation of such a thing, it would be more than a simple config change to change this.
> end users have no way to confirm whether this claim is actually true
Which I find really puzzling because if you announce this fact to people who will basically have their lives ended if they get caught with CSAM material... then they will be the ones to avoid it. While the rest of the population is being scanned.
It will most likely scan files in the /iCloud directory inside iPhone/iPad. In there, the file scan process described above will be executed, regardless of what the user stores there.
I do not see how this wouldn’t be easily extended to all mountpoints on the device. And again, one needs to have faith and assume the /iCloud binary information on storage is really physically isolated from everything else. Sorry, it is very unlikely they aren’t really scanning, as I said, everything.
This is just plain bullshit. The process is well documented, and there is no general scan of files inside iCloud.
In fact there is no filesystem scan at all. There is only a check that takes place during the upload process to iCloud Photo Library, which is separate from iCloud Drive.
The only well documented process in tech is source-available. There is documents and speculation regarding this process but we don't actually know the details beyond what they claim and they're not saying too much.
The information about what is checked by this mechanism is well documented.
If you want to say that Apple could be lying or mistaken about what the code does, that is a different claim from whether they have documented what I said they documented.
There's no "most likely", it's already implemented. I just saw a USENIX talk from a developer and a cryptographer and they said it's a hook when iCloud Photos opens a file to upload it to cloud.
I don't like it. I wish it had never happened. Fuck the government. But you are wrong and blowing this out of proportion.
This is incorrect. They scan the photo as part of the iCloud upload process. If that process does not run for a given file or photo, this scanning does not run, according to the interview.
Are they wrong? It seems like most people don’t understand that Apple was already scanning iCloud Photos on their servers like Google scans Google Photos, they’re just going to be doing it client side now.
I’m not defending Apple, I wish they wouldn’t do this, but I see section 230 levels of lack of understanding out there.
Why move it on-device then? They've made no announcement or attempt to encrypt iCloud backups, so they are free to keep scanning on their servers. Moving it on-device has zero value-add for iPhone users, it only serves as a Trojan Horse for any future "scanning" projects that they may adopt, willingly or otherwise.
I wasn’t commenting on the why. I’m just saying that all the words in the phrase “Apple regrets confusion” are probably true: There is a lot of confusion, and Apple’s PR is probably really regretting it right now.
If people understood what was going on, would they be as upset? I don’t know. Apple doesn’t seem to think so.
They actually weren't. They wanted to start, and they thought this way of doing it was more privacy-friendly. Craig says all this, but apparently listening to what he says is forbidden.
I didn’t see that. Do you have a link? I only was able to find Jane Horvath (Apple Chief Privacy Officer) saying they were already scanning photos using PhotoDNA at CES 2020 [1].
This article just says "iCloud," not specifically Photos. My understanding is that they previously scanned iCloud Mail, but not iCloud Photos. I don't have a link handy unfortunately, and don't have time to dig it up again.
"Sow confusion, and reap inaction" - common military tactic. If what you're doing is too big to hide, you also cause visible activity in multiple places, so the enemy can't decide where to send reinforcements. The Normandy invasion had quite a bit of that. The enemy was confused for days about where the main attack was hitting, and didn't commit reserves until it was too late.
It's sadly typical of their apologies. The "I'm sorry you feel that way" apology. As a long time customer its beginning to annoy me. You fucked up. Say "we fucked up" and move on.
It reminds me of 2002 and the "patriot act" it had the perfect name at the perfect time so that you could just choose to be a "patriot" or a "pro terrorist"...nothing in between was acceptable in public.
Besides anger, I guess we are all learning very valuable marketing tactics. Just name it extremely so we cannot choose the other option because it makes us a bad person.
Right, the hubris rained there after Jobs. All they have to do is to explain and educate their customers that this is fine, because we are just stupid mass.
It’s also a bit of a laugh to suggest Jobs didn’t suffer from hubris. See: arguably the Mac itself, $10,000 NeXT Cube, the Power Mac cube…really anything with cubes.
I think the "confusion" was 100% intentional. That the two features (iMessage scanning & on-device spying pre-upload to iCloud) were intentionally released at the same time to make the whole thing harder to criticize in a soundbite.
Confusion is the best-case scenario for Apple because people will tune it out. If they had released just the on-device spying, public outcry and backlash would have been laser targeted on a single issue.
Fanatics also have a tendency to try to latch onto whatever details may offer a respite from the narrative. The core problem here is that Apple is effectively putting code designed to inform the government of criminal activity on the device. It’s a bad precedent.
Apple gave its legendary fan base a fair few facts to latch onto; the first being that it’s a measure against child abuse, which can be used to equate detractors to pedophile apologists or simply pedophiles (these days, more likely directly to the latter.) Thankfully this seems cliché enough to have not been a dominant take. Then there’s the fact that right now, it only runs in certain situations where the data would currently be unencrypted anyways. This is extremely interesting because if they start using E2EE for these things in the future, it will basically be uncharted territory, but what they’re doing now is only merely lining up the capability to do that and not actually doing that. Not to mention, these features have a tendency to expand in scope in the longer term. I wouldn’t call it a slippery slope, it’s more like an overton window of how much people are OK with a surveillance state. I’d say Americans on the whole are actually pretty strongly averse to this, despite everything, and it seems like this was too creepy for many people. Then there’s definitely the confusion; because of course, Apple isn’t doing anything wrong; everyone is just confusing what these features do and their long-term implications.
Here’s where I think it backfired: because it runs on the device, psychologically it feels like the phone is not trustworthy of you. And because of that, using anti-CSAM measures as a starting point was a Terrible misfire, because to users, it just feels like your phone is constantly assuming you could be a pedophile and need to be monitored. It feels much more impersonal when a cloud service does it off into the distance for all content.
In practice, the current short-term outcome doesn’t matter so much as the precedent of what can be done with features like this. And it feels like pure hypocrisy coming from a company whose CEO once claimed they couldn’t build surveillance features into their phones because of pressures for it to be abused. It was only around 5 years ago. Did something change?
I feel like to Apple it is really important that their employees and fans believe they are actually a principled company who makes tough decisions with disregard for “haters” and luddites. In reality, though, I think it’s only fair to recognize that this is just too idealistic. Between this, the situation with iCloud in China, and the juxtaposition of their fight with the U.S. government, one can only conclude that Apple is, after all, just another company, though one whose direction and public relations resonated with a lot of consumers.
A PR misfire from Apple of this size is rare, but I think what it means for Apple is big, as it shatters even some of the company’s most faithful. For Google, this kind of misfire would’ve just been another Tuesday. And I gotta say, between this and Safari, I’m definitely not planning on my next phone being from Cupertino.
> I’d say Americans on the whole are actually pretty strongly averse to this, despite everything, and it seems like this was too creepy for mant people.
You mean that country which gives a damn about privacy altogether because all those fancy corps are giving them toys to play? You know, those companies which feed on the worlds populations data as a business model. The country which has a camera on their front door which films their neighbourhood 24/7? The country which has listening devices all over their homes in useless gadgets?
You have to be joking or that scale you impose here is useless.
This whole thing will go by fast and there won't be much damage on the sales side. Apple is the luxus brand. People don't buy it for privacy. Most of the customers won't probably even understand the problem here.
The only thing we might be rid of are those songs of glory in technical spheres.
> Apple is the luxus brand. People don't buy it for privacy.
Privacy is the main selling point Apple is pushing in their current PR campaigns. They've been slowly building up a brand around privacy with new privacy features.
They've just sunk that entire brand/campaign. Instead of "iPhone, the phone that keeps all your data private", it's "iPhone, the phone that looks through your pictures and actively rats you out to police to ruin your life".
The reason they pushed privacy was because of the media attention that Androids bad privacy got. Please don't tell me you believe privacy was at the usual consumers mind when they bought their devices...this is ridiculous or you don't meet many normal users. It's marketing. They'll something new. You can fit everything in front of a white background...
>A majority (72%) of iPhone & iPad users are aware of new privacy changes in recent software updates. When asked how well they understand Apple’s new privacy policies, these were the responses: Extremely well (13%), Very well (29%), Moderately well (21%), Slightly well (9%), and Not well at all (28%). Two in three (65%) users are “extremely” or “very” concerned about their activities being tracked as they use certain websites and apps, while only 14% said they were not at all concerned.
>You told us: You really want an Apple-like anti-app tracking feature on Android... Over 30,000 people voted in favor of an App Tracking Transparency feature on Android.
People are becoming extremely conscious of online (and on-phone) privacy issues. Where have you been?
Your statement "don't tell me you believe privacy was at the usual consumers mind when they bought their devices... this is ridiculous or you don't meet many normal users." is itself, ridiculous.
You just quote some survey stats. They are bullshit. What people want and what they say they want are two entirely different things.
Small example: I run a small app. The number of GDPR-related requests is zero. The number of emails like "can my deleted account and deleted data still be recovered?" is like 1 per month or so.
WhatsApp is also a good counter example. People want privacy, but what they want more is utility and network effects. Most people I know didn’t abandon WhatsApp despite numerous privacy mishaps and I think the same will happen here with Apple. This will blow over – unfortunately.
People don't care about privacy, got it. No one is quitting Facebook, no one is de-Googling their lives, installing privacy extensions/apps and making purchases based on privacy issues.
The worlds largest online companies increasingly making privacy a priority in their marketing are all wrong. Got it.
>Most people I know didn’t abandon WhatsApp despite numerous privacy mishaps
I never said privacy was the top issue, trumping all else. You're absolutely correct that other issues like convenience and network effects are important.
Besides the obvious and continuos popularity of apps with bad reputation in the privacy department, those surveys have all been made after the campaign was launched. The only thing it does is to show that the advertising message has been delivered. Nothing else.
So, the marketing is working and people are increasingly aware of privacy issues? In that case, would you say this statement:
>don't tell me you believe privacy was at the usual consumers mind when they bought their devices... this is ridiculous or you don't meet many normal users.
Is true or false? Because you seem to be contradicting yourself. Are people aware of privacy or not? Is the marketing working or not?
I suspect you were trying to say that people weren't aware previously. Is that correct? Because I don't think anyone would disagree with that.
Maybe in the future people will realize that it's a mistake to market your company as being unequivocally on the side of privacy. Nothing was ever private at the point where cloud companies are forced to comply with the government to some extent or face extinction through legal pressure.
It isn't like the ultimate goal of protecting children isn't worth fighting for, and the ICMEC considers half the countries in the world having no laws against CSAM to be "simply unacceptable." But companies that insist that everything they host can remain private to everyone are lying to their users, and will have to align their marketing claims with the reality of the law, or this kind of backlash will result.
But in general, there are a lot of other descriptors besides "private" that are nothing more than baseless Corporate Memphis copy.
> The core problem here is that Apple is effectively putting code designed to inform the government of criminal activity on the device. It’s a bad precedent.
This is wildly disingenuous.
Apple is putting code on the device which generates a hash, compares hashes, and creates a token out of that comparison. That is 100% of what happens on the device.
Once the images and tokens are uploaded to iCloud photos, iCloud will alert if 30+ of those security tokens show a match, it will alert Apple's team, and they will get access to only those 30+ photos. They will manually review those photos, and if they then discover that you are indeed hoarding known child pornography then they report you to the authorities.
Thus, it would be more accurate to say that apple is putting on your device code which can detect known child pornographic images.
> And it feels like pure hypocrisy coming from a company whose CEO once claimed they couldn’t build surveillance features into their phones because of pressures for it to be abused.
This isn't a surveillance feature. If you don't like it, disable iCloud Photos. Yes, it could theoretically be abused if Apple went to the dark side, but we'll have to see what this 'auditability' that he was talking about is all about.
Honestly, with all of the hoops that Apple has jumped through to promote privacy, and to call out people who are violating privacy, it feels as though we should give Apple the benefit of the doubt at least until we have all the facts. At the moment, we have very few of the facts.
Describing the implementation details does nothing to change the reality that the device is acting as an informant against its owner. The number of hoops literally changes nothing. Adding an AI model versus using SHA sums changes nothing. Adding some convoluted cryptography system to implement some additional policy changes nothing. In trivial cases like anti-piracy measures or anti-cheat in games, we tolerate that the device will sometimes act against our best interest, but at least in this case, the stakes are low and the intentions are transparent.
We have every fact we need to know to know this shouldn’t be done, and I’m glad that privacy orgs like EFF have already spoken much to this effect.
> Yes, it could theoretically be abused if Apple went to the dark side, but we'll have to see what this 'auditability' that he was talking about is all about.
Or we can just short circuit the entire issue by deciding firmly we don't want this and punish Apple's behaviour accordingly. Which is what appears to be happening.
> it feels as though we should give Apple the benefit of the doubt
It really doesn't feel like this to me at all. Users have clearly stated: we don't want this. It's time for Apple to simply pull it all back and apologize.
They created a tool that, in principle, lets a government ask about certain hash matches that are on the iPhone but not necessarily on iCloud, correct?
There is no way to determine whether the hashes are about CP or about HK protests.
> Yes, it could theoretically be abused if Apple went to the dark side, but...
> ...we should give Apple the benefit of the doubt...
You have to take off your apple branded rose tinted glasses my friend.
Any company as big as apple needs to be scrutinized as harshly and critically as possible.
Their influence on the world is so big that a botched roll out of this sort of tech could be absolutely devastating for so many people, for so many reasons.
I don't care if it's hashed tokens or carrier pidgins. We should only allow companies to act in ways that improve our lives. Full stop.
> Thus, it would be more accurate to say that apple is putting on your device code which can detect known child pornographic images
> If you don't like it, disable iCloud Photos.
> Yes, it could theoretically be abused if Apple went to the dark side [...]
> [...] it feels as though we should give Apple the benefit of the doubt at least until we have all the facts.
No, nobody gets "the benefit of the doubt". The very use of that phrase admits that you are being put into a situation where you could be screwed in the future.
There is zero transparency or oversight into the code that does the scanning, the in-person review process, or the database of images being scanned for.
> Once the images and tokens are uploaded to iCloud photos, iCloud will alert if 30+ of those security tokens show a match, it will alert Apple's team, and they will get access to only those 30+ photos. They will manually review those photos, and if they then discover that you are indeed hoarding known child pornography then they report you to the authorities.
Apple: We have 29 matches on your device of "known CSAM"[0]. However, despite our confidence level being very high, we won't report it to the authorities because we value your privacy!
It's a feature that only applies to kids under 18 who're in a family group, whose parents turn it on. It warns the kid before letting them see an image which machine-learning thinks is nudity. If the kid is 12 or under, their parents can be notified if they choose to see it. It apparently does no reporting to anyone apart from that parental notification.
I shudder to think of the thousands of gay children this will out to their unaccepting parents, some resulting in physical, verbal or emotional abuse -- or worse.
The thousands of gay children under the age of 13, whose parents have opted into this program, who are sending or receiving pornographic images via iMessage and have chosen when prompted to notify their parents of that?
Hot take, but: they never should have released the news about the neural hash side of things with the iMessage child account scanning.
Regardless of how you feel about it, both issues were being completely mixed up by every single person I saw discussing this - even otherwise very technically competent people on this very site.
I've no doubt that it muddied the waters significantly when it comes to discussing this.
The other part was not comparing it to either existing server-side scanning or E2E. Maybe it's just optimistic but it seems like the reaction might have been different if it had been something like “we are currently scanning our servers. To make our service E2E with this narrow exception, we are moving that scan to the client.”
That’s a fair question. The idea that they aren’t scanning already comes from the fact that they make very few reports compared to Google or Facebook. Literally a few hundred vs 10s of millions.
If they were already scanning, you’d expect more reports since although there is no legal requirement to scan, there is a legal requirement to report detections.
How do we know that, though? They're secretive enough that it's hard to tell — they certainly aren't reporting high numbers to NCMEC's tipline, although it's possible that some of that might be the difference between human-reviewed and aggregated reports versus other companies having a fully-automated process making one report per image or something like that, but that doesn't necessarily mean that they aren't using other channels.
Which, again, really hits the need for disclosure — so much of the response to this announcement has been heavily shaped by both that secrecy and just springing it on the world without much prior public recognition of this issue.
At some level, if you're uploading files to their servers, you have to trust them. And to a lesser extent if you're using their proprietary software (although you can monitor network traffic and so on.)
> Which, again, really hits the need for disclosure
I think there was a misunderstanding: I wasn't saying that you don't have to trust them to use their cloud services but rather that it would be a surprise to me if they were _not_ already scanning iCloud Photos (i.e. is this a change from “scanned after upload” to “scanned before upload” or from “not scanned” to “scanned”?). I've always assumed that they do scan your hosted files, just like Dropbox, Google, etc. do.
In the interview, Craig discusses this. They did not previously scan iCloud photos because they consider it too invasive. They consider this less invasive, because Apple does not look at the content of your photo, except on the device. So the change was from "not scanned" to "scanned before upload on device".
I assumed the hair they're splitting is it's your own device that's doing the scanning, and not "Apple". Meaning, their iCloud servers won't scan your photos. They want people to read that as "oh, I thought my pictures were going to get scanned without my say so, I guess that was just wrong and this is a false alarm", but what is actually going to happen is that you are going to scan your pictures, then send Apple a hash.
To be clear, this is a distinction without a meaningful difference. Or, if there is a difference, it's that it's actually worse than the alternative (cf. the Stratechery article that's been making the rounds).
If that's right, then this isn't a lie, but it's incredibly mealy-mouthed, misleading, and disrespectful of their customers' intelligence.
“Scanning iPhones for images” can be understood as “without prompting, all images on your iPhone will be scanned and the information sent to Apple”. But from what they claim, photos are scanned as they are uploaded to iCloud Photo Library—only those photos and only at that time. The former is absolute, while the latter is contained to specific photographs on a specific trigger (which you can disable).
Perhaps the nuance is irrelevant or disingenuous, but there is a valid interpretation of his words where he isn’t “blatantly lying”.
> The system could only match "exact fingerprints" of specific known child sexual abuse images, he said.
It has to match the fingerprint exactly, but the fingerprints themselves are not exact, otherwise they would be useless.
And this is completely beside the point. People's concerns aren't mostly over false positives, they're over the possibility that this feature will be perverted by authoritarian governments. Way to miss the point.
> Mr Federighi said the "soundbyte" that spread after the announcement was that Apple was scanning iPhones for images.
> "That is not what is happening," he told the Wall Street Journal.
I'm not the parent but if the question is "who is meant by authoritarian regimes?" a great example on how to measure authoritarianism is the democracy index assembled by the Economist Intelligence Unit:
The distinction he's making (which I realize you will likely not find satisfactory) is that they aren't proactively scanning all of the photos on your device or in your photo library. The scan happens as part of the pipeline that uploads images to iCloud.
> Mr Federighi said the "soundbyte" that spread after the announcement was that Apple was scanning iPhones for images.
But that's exactly what's happening? Most people using an iPhone sync photos with iCloud (especially after they introduced the more cost-effective 2TB Apple One plan), images are scanned before they are uploaded to iCloud, ergo Apple will be scanning the iPhone for images.
There was no confusion, this is a turnkey surveillance system who's scope will expand to whatever those with power over apple decide is taboo. I think we all got the message loud and clear.
I think the problem Apple ran into was that there was no confusion at all. Apple announced they were going to scan users' devices after years of marketing themselves as a "privacy-focused" company. Shockingly, customers were pretty mad about the whole thing.
They are scanning images on iPhones and iPads prior to uploading those images to iCloud. If you're not uploading images to iCloud, your photos won't be scanned -- but if you are using iCloud, Apple will absolutely check images on your device.
From Apple's Child Safety page:
> Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child safety organizations. Apple further transforms this database into an unreadable set of hashes that is securely stored on users’ devices.
> Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection, which determines if there is a match without revealing the result. The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image.
> Apple will absolutely check images on your device.
Yes, they will check the images you have chosen to upload. No ‘scanning is involved’.
Claiming this is ‘scanning users devices’ is just dishonest - it’s obvious that it creates a false dichotomy impression of what they are actually doing.
Even if we accept that. It is a lie to say the device is being scanned. It is definitely not. Only the photos the user chooses to upload are checked. That is not the device.
It is part of the device, and this specific part is being scanned. Can I physically remove this “checking” part and end up with a working iDevice D that resembles D = {{Device \ {/iCloud/Photo Library}}?
The checking is being done on the device. Nobody disputes that. Indeed it is being marketed by Apple as a feature. Yes, this feature is part of the device.
If you say Apple is scanning the device, you are lying. They are not scanning the device. They are scanning photos chosen for upload.
Another commenter put it in better terms, so you may understand it:
Suppose we know there are people who smuggle drugs on airplanes on their person for the purpose of something terrible, like addicting children or poisoning people. If I run an airport I could say: to stop this, I'm going to subject everyone who flies out of my airport to a body-cavity search. Tim, and Craig, are you OK with this? If I can say, "Don't worry! We have created this great robots that ensure the body cavity searches are gentle and the minimum needed to check for illegal drugs," does it really change anything to make it more acceptable to you?
No falsehoods, it is the device, even though it is only a specific part of it. I know you got the point I tried (or rather, the other commenter) to make about the part of someone's body meaning "the whole" of a person. Same philosophical view can be applied to the device.
Anyway, someone in here can accept what the other can't, so let's leave at that and let history tells.
It’s a lie to say they are scanning the device, when you know that are only scanning files that are uploaded. We know now that you understand both this distinction and what Apple is actually doing, so it’s clear that you were lying.
Chrome safe browsing feature shows a warning if you browse to a URL which Google have flagged as hosting known malware. If you described this as "Google blocks your device from connecting to servers that don't have their approval" would the listener get an accurate understanding of what was and was not happening?
If Chrome scanned downloaded files for viruses and you described that to someone as "scans your computer for viruses" do you think the listener would come away with an accurate understanding of what was happening and accurate understanding of what they were and were not being protected from?
If BestBuy GeekSquad offered a service to "check your device for problems" and all they did was open your photo collection, would you walk away arguing that it proves the screen and mouse and CPU and disk must function and nobody could expect them to do any more than that, service provided, full marks, that's "the device" checked? Or would you hope that "checking the device for problems" might involve at least exercising all the major features like speakers, wifi, bluetooth, at least once, and preferably with stress and thermal tests?
When the TSA ask you to switch your device on to demonstrate that it's not a bomb, are you on the side of "if the screen lights up, the device is thoroughly and effectively tested and cannot contain anything else" or on the side of "an attacker could make up many ways to make the screen glow while hollowing out the insides, this does not really demonstrate that 'the device' is safe"?
They're not misunderstanding it; you're deliberately using an inaccurate description to mislead people while trying to hide behind "technically not lying", and they're calling you out on it.
Am I, though? Is Apple? Is parent's? It seems "their" (whoever you meant) interpretation of what does and doesn't constitute something is looser than my interpretation.
What you're doing is changing "Two Americans run their homes on solar panels" into "American homes run on solar panels" with the intent of fudging the quantity so that readers assume it means most or all of them, while being able to argue "they are American homes, plural, so it's correct".
"Device scans photos" and "Apple scans device" imply two very different things about how much is scanned, and you're using the latter because you know that if you describe it accurately readers won't be as panicked as you want them to be.
>Apple didn’t announce any timeframe about when will they implement child safety features in third-party apps. Apple said that they still have to complete testing of child features and ensure that the use of this feature in third-party apps will not bring any privacy harm
> EDIT: For Question below
https://technokilo.com/apple-child-safety-feature-third-part...
>Apple didn’t announce any timeframe about when will they implement child safety features in third-party apps. Apple said that they still have to complete testing of child features and ensure that the use of this feature in third-party apps will not bring any privacy harm
The Q&A mentioned has no date or time. No Apple Spokespeople are named. There are no actual quotes. No well known news outlets have mentioned this very consequential detail.
At what point would you consider it "scanning the device"? What if they start scanning messages? Browsing history? Downloads? Where do you draw the line?
People do understand the difference. It is only you who seem to be confused about the easily understand words that's people are using.
When people say that the device is being scanned for pictures, they know what that means. So it is fine for them to say that the device is being scanned for pictures.
This is about the 16th time I have seen language just like this used to explain away this concern. I don't know if you realize, but this wording makes it sound like you can select some photos and leave others local. I can find no indication anywhere, including on my phone, that iCloud Photos is anything other than an All Or Nothing singular toggle in iCloud settings. If you have instructions to the contrary, I will be happy to stand corrected.
Seriously, everybody is wording it like this. "Photos you choose..." and similar.
You spend almost too much time defending Apple[0]. If you have any association with them, you should probably disclose that first.
0. Check the comment history, every comment is defending Apple and this has been going on for many months (years?). In fact, I don't see any comment that is not defending Apple.
I know I am making very serious allegations, but please go through the comment history to form your own opinion.
I don't believe that the user is a bot though, most comments are 'well-reasoned'.
Wow you're right - over the past 9 days zepto's generated ~5 pages of comments defending Apple's scanning. Why would one dedicate so much time and effort to rise to the defense of a trillion dollar company?
First of all, I have no affiliation with Apple, not do I own any Apple stock.
Do you have any affiliations we should be aware of?
Secondly, I haven’t ‘defended Apple’ in any comments. Indeed there are comments in which I make a judgement about this topic where I say that what Apple is doing is distasteful and offensive.
Elsewhere I have pointed out that if Apple wants to scan people’s devices they have many other mechanisms at their fingertips than this narrowly tailored tool.
What exactly do you think I’m ‘defending Apple’ from? Quite a few of my comments are critical of false or immaculate characterizations of what Apple is actually doing.
If you consider that to be a defense of Apple, then I disagree.
For the most part there just seems to be a lot of confusion about what Apple is doing, and general frustration about the state of computing.
Do you really think of these as ‘attacks’ on Apple?
You're splitting hairs unnecessarily. Apple is scanning users' photos on their devices. To say that they are not "scanning devices" because they are (currently) only targeting photos and not every single other part of the phone is unhelpful at best, and detracts from the point that this is a massive violation of their users' privacy. The exact wording here really doesn't matter as much as you think it does.
The "confusion" is splitting hairs. Federighi is trying to draw an artificial distinction between client-side scanning and in-transit scanning where the code performing that in-transit scanning merely happens to be running... on the client.
I was definitely confused, for the record. I had the impression that Apple would scan all photos on device, but that is not true. I was also confused because several changes were announced at once, and the conversations sometimes blended them.
That’s what Federighi says in this interview. It’s only scanning photos that are about to be uploaded to iCloud. Otherwise photos aren’t scanned. Disable iCloud and there’s no scanning.
Still, I think Apple misjudged the whole cloud vs. device thing in this case. They’ve historically preached a lot about how everything should happen on the users device, not the cloud. I think that got myopic for them, and led them to this decision.
But in this case I think users would be much happier if Apple had just said “under pressure from law enforcement we are now scanning photos when they arrive at the iCloud data centers. If you don’t want scanning don’t use iCloud.” Because it’s not so much the scanning of uploaded photos that has people upset, it’s the fact that the scanning and phoning home is baked into the device itself.
Many people (like myself) are worried about the slippery slope where this is turned on for all photos, since why not? Not all abusers will upload their CSAM content to the cloud, why wouldn't Apple flip a flag in the future to scan everything, including photos and downloaded content? If they are serious about fighting CSAM and have this great privacy preserving platform, I don't see why they wouldn't do this?
It’s not a flag, the feature is part of the iCloud upload process. No upload, no scan. Of course a code change could do anything.
The point of the feature is to prevent people from using iCloud to distribute CSAM. If you’re recording it with your phone, it’s no different than using an slr camera. The cloud part is what they’re worried about.
Why flag the account and report it if the only goal is to politely prevent people from uploading? Like you say, a “code change can do anything” and we simply don’t know how the current feature is done or how it will evolve.
edit: like many comments here already say, reporting doesn’t sound terrible for CSAM, but nothing about the feature guarantees it wont be extended to other kind of content.
Completely agree with this. But apples perspective is that on-device scanning prevents them from looking at your photos in the cloud at all. This is actually more secure than other cloud providers that do all kinds of shit with your photos.
But if they find a few which have a high correlation, they will eagerly upload them to Apple’s police service and have them viewed by the entire floor.
Also, the matches are supposedly only to actual babyporn pictures. We have 0% way to verify that, as even employees of NEMSEC are not all allowed to view them. Such DBs are often full of unreviewed fluff, and why not unrelated photos entirely, cookware, computer cases, who knows, as long as “some degree of matching” with your photos allows Apple to send a .zip to the police.
> But apples perspective is that on-device scanning prevents them from looking at your photos in the cloud at all.
In terms of the privacy intrusion, what difference does it make whether the images are scanned on your device or on their servers? They're getting scanned just the same either way.
But if they did it on their servers, it would provide a technical impediment to expanding beyond the use of iCloud, and remove the need to trust Apple as much. That seems like it would be much better for users while still allowing the functionality they claim to be seeking.
> That’s what Federighi says in this interview. It’s only scanning photos that are about to be uploaded to iCloud. Otherwise photos aren’t scanned. Disable iCloud and there’s no scanning.
That's for now. The first update can change that and you will have no recourse.
I think right now it only scans pictures destined to be sent to iCloud. The problem is you won't hear from them once they start scanning everything. Besides, it's just not really provable, right? You still have to take their word for it.
They are gas lighting people who are upset about what is really happening, and what will happen in 5-10 years, and portraying them as confused and ignorant instead. It’s the standard “you’re holding it wrong” Apple play
Judging from the threads so far, HN believes Apple is scanning everything using a naive perceptual hash implementation it probably got off of Stack Overflow, with no oversight or sanity checks, and that even a loose match (which will be trivial) means SWAT teams automatically being sent to bust down your front door like the FBI meme, and that it's all just a pretext for CCP style authoritarian surveillance anyway, and we'll all be in dissident reeducation camps by the end of the year.
I suppose it's great if you're looking for entertainment value. For rational, informed discussion of the technology and its political and social ramifications, not so much. It's just the same refrain of "we never bother to actually RTFA but we imagine a boot stomping on a human face forever."
I've tagged HN users using a small browser plugin I wrote. It's amusing to see all the Apple users jump to the defense of Apple despite their continued shitty behavior.
It's also great to see the handful that have changed their minds.
The actual iPhone quote you're referring to was "Just avoid holding it in that way" not "you're holding it wrong" as it's often misquoted to.
Most of the commentary on this more recent issue is similarly misrepresented and inaccurate.
I think Apple's mistake here was a PR one, they shouldn't have announced this until they had e2ee ready. Then they could have announced that which would have gotten most of the (positive) press attention. Then they could have gone into details about how they were able to do it while still fighting CSAM.
Well there is a huge difference to AntennaGate. The two aren't really comparable. Not to mention Apple did in a way admit to the mistake and gave out a bumper within weeks of the complain.
Compared to their Keyboard which took nearly 3 years before they have a programme for free repair.
Not going to happen. Words consistently get watered down in meaning by slight misuses that compound over time. There's no stopping that. People don't like being told they're using a word wrong. (And if language is constantly evolving, is there even a way to use a word incorrectly?)
This is an instance of trying to make the public doubt their own reality. They're saying we are confused, or that there is confusion. They don't admit blame, so it's to assume we, the public, are the confused ones. And even if we don't immediately assume that we're the confused ones, we are still going to question ourselves. It implants that there is confusion somewhere, and that maybe we misunderstood them somehow. That maybe we missed something? It is gaslighting by definition and, in fact, a very good example of it.
There is no way Apple released their initial PR piece without thinking it through and deliberately fusing all those new features together as one big unassailable initiative. It was typical my way or the highway.
Which also make it funny now that they attempt to distinguish between them and run into same hole that they dug for other people.
This is inaccurate by definition, of course. Obviously. "My way or the highway" implies there is no alternative.
But in this case, of course, if you're an adult, the Messages part of this doesn't apply to you at all, and the photos part can be completely avoided by not using iCloud Photos.
No offense, but have you even owned an iPhone/iPad for a considerable length of time? The darn things include a maze of settings that are inter-dependent and unexpectedly lose/alter their values; perhaps not on regular basis, but always once in a while (e.g., with a new iOS version). If file scanning and reporting capability is present, code-wise, on your device, you can consider it active - sooner or later.
No offense, but are you mentally retarded? Can you not figure out how to click a slider? My 95 year old grandmother, who was schooled to the age of 14, has figured it out.
Sorry for commenting on a comment, but it was so hauntingly offensive that it wrapped around to the poetic.
It reminds me of the when BMW designed their motorcycles to be so ugly they'd find beauty of their own, K1200R for example [0]
Why thank you. There is a beauty of its own in that K1200R - except for the headlamp, that is offensive, no offense intended of course.
My knowledge in the space is limited to the GS range, having been privy to a few storys of romance between rider and bike while crossing continents. A beauty of its own.
Everyone who's working a non-volunteer position is doing it for the money. So, obviously, yes.
There's a facade that we really work for other reasons, and money is just an inconvenient byproduct. During a job interview, you may be asked "Why do you want to work for us?". And for some reason "So I can afford to buy food" is not a good answer.
There is something called "company culture". I changed jobs just because of that before. Instead of 200k, make 120 and be much more happier somewhere else. Mental health is more important than money after certain amount of it.
As someone who doesn’t work for Apple I wouldn’t work for Apple on principle even if they tripled my salary today. There are some lines that none of us should cross. There needs to be an ethical code for software engineering.
Quitting IS a form of protesting the administrative decisions. Joining a company that does respect privacy IS a form of exercising one's own values in one's employment.
So you end up selecting for a company full of spineless yes-people. The logic makes zero sense to me. Apple isn’t going away anytime soon and the best opportunities for change will come from consistent internal pressure.
I don't know what you and tharne are talking about here. There was definitely confusion. HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone. You are allowed to be against Apple's plan while still recognizing that many people didn't understand what exactly was part of that plan.
That is the where of the story. There was a inarguably confusion over the when, what, and how of the story.
It was not universally understood that this would only apply to photos sent to iCloud.
It was not universally understood that this was only looking for previously known CSAM.
It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
I understand if you consider the where more important than the others, but it is simply a fact that there was confusion on what exactly was happening here.
There's always going to be plenty of people commenting who didn't even bother to read the article at all. But by and large, from what I saw, people did understand the nuances of this and outlined how little stands in the way of expanding these policies' scope once the technology is in place
I think "universally understood" is doing a lot of work to portray a much higher degree of certainty about each of those statements than is justified.
A lot of the contention wasn't about the specifics of their plan, but rather how subtle changes could vastly expand the scope of their plan.
"this would only apply to photos sent to iCloud." for now, until scope creeps.
"this was only looking for previously known CSAM." for now, until scope creeps.
"using some sort of hash matching so photos you took yourself would not trigger the system." well this one is immediately concerning even within claimed scope because there ARE going to be false positives that apple records some database. Millions of iphone users are going to have a non-zero "possible childporn" score.
They are building an engine for iphone users to self-incriminate. If they rigidly hold the scope to only what they announced and never expand, it could be argued that this is a reasonable concession to fight CSAM. However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
>A lot of the contention wasn't about the specifics of their plan, but rather how subtle changes could vastly expand the scope of their plan.
The how helps show us how changing this system is not a subtle change. It isn't like they can flip a switch and suddenly they are identifying new suspected CSAM on people's phones. That would require a new system since the current one is only hash matching.
>However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
This is an arbitrary line that is being drawn. These are photos that are marked for sending to iCloud. Whether the scanning happens on the phone before they are sent or in the cloud after they sent is largely immaterial when it comes to the impact of the code. People are acting as if the line Apple drew was motivated by technology. That was never the deciding factor. Technology is the easy part here. That line was only a policy line and that policy has not changed. Only photos that are sent to iCloud are scanned. If you fear Apple changing that policy going forward, you should have always feared Apple changing that policy.
> It isn't like they can flip a switch and suddenly they are identifying new suspected CSAM on people's phones.
This is a strawman. Identifying *novel* CSAM is a very hard problem to do accurately - the reason they can't flip the switch is because they don't have the technical capability. All of the other things people are concerned about are things that apple does have the capability to do.
EDIT to reply since at max thread depth: *novel* image detection was never on the table, I think you missed that word
> This is an arbitrary line that is being drawn.
It seems the vast majority of people in this thread disagree that this line is arbitrary.
EDIT: whether the threshold is "verging on impossible." depends entirely on the effective false positive rate. If apple's claimed 1 in 1 trillion rate is true, it's probably not a concern. However, I find it unlikely that perceptual hashes on portions of images won't have higher false positive rates when subject matter is similar (non-CSAM legal adult porn, or images of children in swimsuits, etc). If that rises to 1 in 1 million for these types of images, that's hundreds of thousands of people being falsely accused.
>This is a strawman. Identifying novel CSAM is a very hard problem to do accurately - the reason they can't flip the switch is because they don't have the technical capability. All of the other things people are concerned about are things that apple does have the capability to do.
You just used this as an argument against this system. Why do you fear this if you don't think Apple can even accomplish this technologically?
>It seems the vast majority of people in this thread disagree that this line is arbitrary.
I would argue that people who believe that this decision crossed that line were being naïve to not have always known this was a possibility. I don't think this move brings us any closer to Apple scanning our devices for anti-government memes or whatever the fear is because what was stopping that was always more policy than technology.
Also to go back to your earlier comment, in the time since you posted it has now been revealed that this system needs to trigger 30 times before any action is taken. The odds of 30+ false positives is likely verging on impossible.
> It seems the vast majority of people in this thread disagree that this line is arbitrary.
Maybe. I wonder how many people are choosing not to respond because they’ll only be voted down by the people who feel very strongly about this. I’ve curtailed and hedged many my contributions to this topic on HN because of this.
This is a common issue when social media is used to debate matters which are highly emotionally asymmetric.
Most of the time I'm quite outspoken. But it's clear nobody on HN is willing to change their mind on this issue. I'm just not interested in the emotional investment only to deal with oblique inferences that I'm stupid or don't understand their arguments.
This in my mind is the mentality that contributes to a perpetual outrage culture. Standing up for your beliefs and trying to convince others by argument is exhausting and not always effective (see: Brandolini's law). My preference is to just leave my thoughts without the intention of trying to convince anyone, and to learn what can be learned. Sometimes it is necessary for us to disengage when constantly arguing isn't worth the effort.
> ... there ARE going to be false positives that apple records some database. Millions of iphone users are going to have a non-zero "possible childporn" score.
I don't think this is right. The apple pdf on the features says
> With the initial match threshold chosen as described above, iCloud Photos servers learn nothing about any of the user's photos unless that user's iCloud Photos account exceeded the match threshold.
This implies to me that they are using some sort of encryption to prevent iCloud from learning even how many matches there are until the threshold is met.
It was not universally "understood" because it was not universally agreed upon. To wit:
* It was not universally understood that this would only apply to photos sent to iCloud.
Since the scanning doesn't happen on iCloud, this distinction is irrelevant.
"We are going to intrusively scan the subset of your photos that you care enough to back up to the cloud that we've been pushing to you for years" is pretty clear.
* It was not universally understood that this was only looking for previously known CSAM.
It was only looking for whatever is in an opaque database which, according to a third party we don't have any contract with, contains CSAM.
* It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
Yeah right, I feel totally safe knowing that I won't be falsely reported to FBI by a "some sort of" hash matching.
> Since the scanning doesn't happen on iCloud, this distinction is irrelevant.
It’s really not irrelevant. A third party photo library that avoids using the PhotoKit library would not be touched by the CSAM detector. There are many of these on the App Store.
One step further and store the photos encrypted, with a custom renderer that decrypts the content on the heap, and that would take some tremendous performance-hitting detection abilities it’s extremely unlikely to ever happen.
"Don't support this company and it's entire product line because the software on their entire product line is going to call a SWAT team on your location if it thinks it matched something in a database of what some other guys say is kiddie porn, but nobody can see!"
I hope that's precise enough, and sure, everyone is welcome to make their own decisions regarding this.
> HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone.
They're confused about this. NeuralHash doesn't look for pictures of naked kids. It looks for pictures that are identical to the ones they've put in their signatures list.
The problem is that Apple claims that the signatures in their list are all pictures of sexually-abused kids, but we have no way of verifying that. Heck, they don't even have any way of verifying that. Everyone just has to take NCMEC's word for it.
You’re also wrong no? The perceptual hashing doesn’t match EXACTLY the same photos, it intentionally matches photos similar to the same photos, so a minor rotate by a few degrees or crop or whatever also matches
Perceptual hashing can have collisions, and they will be at a higher rate than completely "identical".
The public does not know what the false positive rate is for 'average iphone user pictures'. As engineers we can be certain the false positive rate is not zero. This means that some number of iphone users are going to have non zero "possible child pornographer" scores in the apple database.
The false positive rate is crucial to understanding how concerning this should be. If the average iphone user has 1000 photos, and the false positive rate is the claimed 1 in 1 trillion, there is a 1 in a billion chance that you'll be flagged as a potential child pornographer. (~1 in the world will be falsely accused). This seems reasonable enough with the apple-internal screening step.
If the chunking and perceptual hashing functionally ends up having a much higher false positive rate for images which have similarities to the dataset (parents' pictures of kids playing shirtless, legal adult porn, etc), the false positive rate could actually be more like 1 in 1 million or worse. In which case there are potentially hundreds of thousands of people who will be falsely accused by this system.
How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.
Even if we put aside collisions, new system features ‘Synthetic Match Vouchers’, which is seemingly adding noise into actual CP counter.
I yet to understand what happens to people who only have those synthetic positives? Regardless of what counter threshold is, can’t those people be hoovered up by a subpoena of counter >0 ?
> How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.
Even that doesn't come without issue. How long before '1' becomes the value, because, say for example the number is ten, there's also a horrendous PR spin of "Apple has a high degree of suspicion that you have CSAM on your device, but since there's only 8 images, they won't do anything about it" - "Apple allows up to X non-reported CSAM images on Apple devices" is hard to represent in any positive fashion.
True, but a small point of contention compared to the introduction of on-device scanning for illegal activities.
The policies of which could be changed at a moment's notice. Other details are relatively unimportant.
This is in essence the problem with the new on-device initiative from Apple (calling it a tool is rather misleading).
If allowed to go forward, it is only a matter of time before the capability is expanded.
So it's a big no to the scanning capability, you would think that Apple had gotten the message by now.
And the other initiative is also open for abuse, by allowing the device administrator to spy on the user. Admittedly not as bad as the on-device scanning.
I don't think that was confusion either, because there were discussion and articles on how a hash collision is possible in the scenario you mention due to the way perceptual hashes work.
They have said that the system won't be triggered on a single image. You would need to have multiple photos on your phone experience this hash collision which drops the odds of false positives considerably.
EDIT: It has now come out that you need to trigger the system 30 times before Apple acts on it. I can't imagine the odds for someone to have 30 hash collisions.
So they understand that their system is very susceptible to false positives, but they are saying their clients shouldn't worry because the black box hash gets compared with a black box inauditable threshold, both of which solely determined by Apple. I don't think the reaction was due to any confusion. People understood what Apple was trying to do and realized how much it sucked from a technical perspective.
Where did "very susceptible to false positives" come from? If the odds of a collision are one in a million that is troublesome if they only need one match. If they ignore anyone that has less than 3 matches, we don't really have to worry about false positives. People who have CSAM generally don't have only 1 or 2 images.
I assume because they know the system is so false-positive-prone that even 29 false matches is a regular occurrence not worthy of wasting an employee's time to look at.
Severe loss of marketshare? It wouldn't even register.
I don't like the feature. Putting this on the client device is dubious and should never have made it past the brainstorming stage.
Having said that, technology companies, big and small, are bound in the US to do this. By law. If anything Apple was by far the laggard of the bunch (with reporting counts magnitudes lower than peers, despite a larger customer base). As I said in another comment, no company can protect you from your government.
Much has been made about it being on device, which while a serious optics issue...the hot takes being given on here are manifestly absurd. Like, literally the company that holds all of your data, all of your passwords, all of your info and you need to invent slippery slopes to imagine up what they "might" do?
If they want to have their way with your data, they could have been doing it for decades.
They should never have announced two very different systems at the same time. Contrary to some of the insincere claims given in this very thread, there is massive disinformation and confusion about them. In the end I feel like 98% of the "the end is nigh!" comments are by long time Apple detractors who just see this glorious opening.
And while I still hope that Apple says "Mea culpa, we're just going to scan on the ingress to iCloud Photos", whatever they do in a month this is going to be completely forgotten.
This actually enables Apple to apply end-to-end encryption, while still complying with US CSAM laws. Users' content is scanned for CSAM, and is encrypted before leaving their phone. Whether one considers E2E encryption worth it if the content is still scanned before being encrypted is an exercise left to the reader.
This potentially means all of iCloud, not just photos, could start to use E2E encryption as well - which is fantastic.
Which law specifically says that user data has to be scanned, breaking encryption if necessary? Everything I've seen says that they have to report what they find, but are under no obligation to actively look for it.
>This potentially means all of iCloud, not just photos, could start to use E2E encryption as well - which is fantastic.
They have to scan things that are not photos. What if the bad guys just zip their photos and upload that?
There also needs to be a solution to CSAM uploaded before NCMEC had a chance to tag it, especially to cases where the bad guys uploaded their CSAM and deleted it from their iPhone. What happens then, the bad guys get E2E and nobody can find them? There has to be a technical solution in mind for this, and everything I can think of has implications (Let the iPhone store hashes of deleted images? Would it be enough to scan also during download?)
IMHO, any serious attempt to find CSAM using Apple's client-side approach requires more scanning, and Apple not being forward on that makes me trust them less. Also, the moment they expand the scanning, we should think carefully if there actually are any privacy benefits.
Putting it server side is categorically worse. Putting it in the client SDK for iCloud (architecturally speaking) rather than on cloud storage or in the OS is clearly the better correct technical choice, tying surveillance’s hands in a way server side or OS would not.
Most every client SDK routinely checks content before upload, it’s a best practice. Careful examination suggests this was engineered better than that practice.
(Note: even tech trade posts such as LWN, Stratechery, or Daring Fireball trying to write well about this need to sit down a minute and have how it actually works walked through for them, as do many in this community.)
FWIW, I agree with much of the rest of your post except the rationale for low reporting counts.
No, architecturally there's no reason to do it on the client side. You're uploading a photo to iCloud, encrypted with a key that Apple controls. The only reason to use even a little bit of battery power to do the check on the client, and to do this elaborate song-and-dance around only notifying Apple if there are a certain number of positive matches, is that it creates the (false) impression that Apple couldn't just check your photos on the server side if they wanted to.
It's basically engineering with the goal of creating a perception of privacy rather than actual privacy. I don't really care that much about Apple policing what you upload to iCloud, but this disingenuous architecture does annoy me a bit, and there's also the added insult of having a device in your pocket that by design works against its owner (reminiscent of "treacherous computing"). These problems would go away if they just did the check on the server side.
Apple gets hung up on weird stuff. Likely doing work phone side was an environmental argument. The amount of energy used to decrypt server side to create the hash was marginally larger than the amount used by the phone pre-encryption with cost of transmitting. (Bonus that the energy and CPU was paid for by the client). The result for either location is identical.
> No, architecturally there's no reason to do it on the client side.
The architectural reason is to do things the server couldn’t.
> You're uploading a photo to iCloud, encrypted with a key that Apple controls.
The architectural reason is so the server doesn’t have to be able to read the photo, and it need not be a key Apple controls.
Your first two sentences are the exact reason to do it on the client instead of on the server, such that it’s possible to have e2e encryption opaque to the server.
Why? Putting it server-side means that it's only possible to examine the images that they claim they are targeting -- those going to the cloud. That seems like a much better and more private way to do it, because it's putting the surveillance "in their house", so to speak, instead of mine.
If I were to design a spying device, I would definitively put it on both the client device and the server. That is because third party apps are not required to put their contents on Apple servers. By putting it on the client, you can potentially hoover up any app’s data. Just write a background service that scans the client’s file system and track all calls to encryption APIs.
The list of apps is such a treasure trove. Signal? Clubhouse? Telegram?
Tech folks drastically overestimate how much the average person cares about privacy.
Plenty of people believe that the Facebook and Instagram apps are recording audio 24/7 and target you ads based on the speech the apps hear. That doesn't stop people from using the apps.
A few years ago some of the most famous people in their world had their iClouds accounts hacked and had their naked photos leaked. That is a lot of people's worst fear. People literally commit suicide over this sort of thing. It didn't hurt the iPhone's market share.
IMO, people largely DO care. If you ask the average Joe - is online privacy important to you, the majority would obviously say yes. Why then don't they stop using FaceCrook?
1. The penalty of social ostracism due to the network effect. This is a severe punishment to most humans - particularly in today's socially disconnected world. Without these apps, a large # of people would not have any contact with much of their social circle - including family.
2. Learned helplessness. I think that many people have just given up. Even if they knew how to fight for their privacy, they see time and time again that money always wins.
A better way of saying it would be that they just don't care that much. Most people care in theory about taking care of the environment, but that doesn't mean that they're willing to tolerate even 1c/gal higher fuel taxes. Most people care in theory about taking care of the poor but balk at the personal economic consequences of doing so. People have different priorities and when one consistently loses out to the others that means that they largely DON'T care about it.
I’d argue that “privacy” is just a significant trigger argument for most people, but not felt as important per se outside of the context of being afraid of the social consequences of some behaviors being exposed.
Take all the paranoia and “muh privacy” around contact tracing or the European Green Pass: It’s as anonymous as it can be, yet millions of people argue against them across the political chasms that separate them. So yes: unless it becomes a divisive topic (which would be undesirable), this CSAM scanning will go away with the news cycle.
About your comment on celebrities’ “sex-tapes”. I guess it takes a certain kind of extroversion and (positive) narcissism to be one, and they’re expected to be gossiped about or to show their bodies in movies or photos so I don’t think those victims blinked that much - some like P. Hilton probably manufactured some porn to ride the wave. On the other hand, revenge porn did drive ordinary people to suicide because in this case the victims weren’t prepared or expected to let the public to see that.
> Tech folks drastically overestimate how much the average person cares about privacy.
This is old thinking. In the 90s and 00s it was miraculous to trade in a little privacy for some awesome free service on the internet. Google, Facebook and others became huge, and everyone has been influenced / manipulated by them and people are getting tired of having a family dinner conversation next to their smart speaker and seeing ads for six weeks for Depends diapers because someone told a bad joke at the dinner table.
> Severe loss of marketshare? It wouldn't even register.
Disagree.
Something this ludicrously intrusive, over-reaching, reckless and trust-destroying is a dealbreaker for exactly the demographic of technology enthusiasts who influence others purchasing. It may not be instantaneous, but it would certainly propagate.
It's anecdotal, but I know zero people who are NOT reconsidering even lifelong loyalty to Apple over this.
It's just too crazy big a wrong, it's like the company just had a nuke go off inside it and is trying to pretend nothing happened.
They may as well have announced a partnership with Trump to put MAGA engravings on all future products, and then in the ensuing furore say they "regret the confusion", whilst carrying on with it regardless.
Though I suppose in that scenario they'd at least be targeting a significant market.
The same can't be said for the size of "sign me up for software-automated police raids" market. A market whose naiveity-induced initial "size" would rapidly shrink after the first few innocents went down, as they absolutely would.
I mean depending on the targets chat software preferences, could a malicious actor potentially ruin an Apple iOS users life just by sending them an image?
There are many troubling scenarios one could imagine. In fact, there is nothing but troubling scenarios.
It's not a can of worms so much as a wormhole, blasting an endless stream of worms at the speed of light.
I'm left not just questioning Apple's leadership, but - honestly - their mental faculties. Really.
No innocent person wants to walk around with an automated snitch in their pocket - I mean, hacking? Bugs? Oversights? Just the overall preponderance of fear that every millisecond you walk around with this thing, it - and its parent corporation and all the depersonalised machinations that go along with it - could be busy organising a blithely mistaken police raid on your family?
Say, because someone you've never met, sent you a message on that new chat app you forgot you installed, that autosaves all media to your iCloud? Or because someone stole the spare phone you keep in a drawer at work, used it for God-knows what and you didn't even notice it was gone? Or, maybe your teenage son got sent something from his teenage girlfriend who unbeknownst to any of them had her phones images uploaded and subsequently catalogued and flagged? Or any number of other entirely plausible scenarios that provide the very reason we have law enforcement protocols and procedures for reporting crimes and that are complex, nuanced and have evolved over hundreds of years and mountains of cases into a massive structure that exists primarily to protect the innocent from exactly this kind of freaking crazy shit?
And Apple expects people to pay them to carry the weight of all that around with them?
I keep seeing this asserted, but there is never any legal citation. What exactly compels a software company to make their software product scan for CSAM?
I get that a service provider like cloud storage may need to scan what they themselves are storing to avoid possessing such material themselves. And iCloud could scan uploaded blobs all day to fulfill their legal department's recommendation.
But what exactly compels a software developer to include a content scanning function in code they distribute? And does this requirement also apply to the authors of rclone?
What distinction are you making with what I stated? What disagreement is there?
Apple only scans photos being uploaded to iCloud photos. Google scans. Facebook scans. Microsoft scans. Even tiny image hosting sites scan.
Apple decided to implement this functionality on device, but they could as easily (with much less fanfare and dissent) have placed it on the ingress to iCloud.
The distinction is between providing services and providing software.
When iCloud scans stored files like Google Drive, nobody complains. It's understandable that risk adverse legal departments have come to the conclusion that doing such things is necessary, to avoid a company being in possession of trivially-discoverable CSAM. And from an individual security perspective, you should consider everything you upload unencrypted to be the subject of similar analysis.
If iOS were to encrypt all files before uploading, making any iCloud scanning mostly pointless, Apple would still not be running afoul of any law. Apple is making general software for end users, and encrypting files to upload would be doing the basic user diligence I alluded to. If users end up using the software to do bad things, those specific users are liable and not Apple.
As far as I am aware, there is no legal requirement for a software developer to modify their software to perform scanning of content it will process while being run by other people. But this is what is implied when you say that Apple is forced to do this by law.
Furthermore if this development was driven by iCloud worrying about legal liability from the combined system, then iCloud should be spun out into a separate company that cannot affect the development of the iOS software.
US law makes them responsible for what they host, to a reasonable, discretionary degree. Literally *every* technology company that I know of scans for CSAM for this reason. Apple is only implementing this for photos that go to iCloud Photos for this reason.
They are surely not required to scan client devices, and that was a foolishly, ill-considered plan (that I still would wager they will abandon), however they absolutely must scan iCloud Photos unless they were technically incapable of doing this. US law doesn't say "you go to jail if you don't", it says "you face enormous liability if you don't".
I have dedicated and virtual servers at OVH and Hetzner and other companies. I'm quite confident they don't scan for anything on the hard drives. One of the selling points of Scaleway.com's C14 backup product (previous incarnation, don't know about current one) was that the storage was encrypted and that they would delete their copy of the decryption key if you checked a box. If you chose that option, you could not restore your backup without re-entering the key (block of hex digits that you had to copy on making the backup). It wouldn't surprise me if that resulted in the obvious sob stories about lost keys.
Those are not US companies but I've never heard of AWS, DO, etc. scanning people's storage either.
AWS offers encrypted backups, and afaik there's no scanning of anything you put in a bucket, but why trust them? Just encrypt your own backups before uploading them, and keep your own keys. In other words, your storage provider needn't and shouldn't be the purveyor of your backup software.
I do think it will result in a loss of market share if it comes to pass, if, for the simple reason that Apple will likely lose the Privacy Moat.
Upcoming contenders like Purism [1] and the Pine Phone [2] will start gaining a great deal more traction from this. Other SV firms will sense business opportunity..... If merely 5% of the TAM around mobile is willing to prioritize non-spying features that would be enough to stand up very healthy businesses.
It isn't like an iPhone is very customizable, repairable, or that usable with all the App restrictions Walled-Garden stuff.
I don't bet - I trade and invest both early stage, and on public markets.
Since you want to have a friendly competition around "put your money where your mouth is" will look into whether or not Purism is accepting investments and what the terms are. AAPL valuations are pretty lofty right now at ~$149 a share if you'd be interested in the reverse :)
Because stock price is based on investors reactions to quarterly reports. Any loss in market share, if seen, will come when people usually buy new devices every x years don't.
It's a hit to their brand from technically knowledgable people for sure though. When someone asks their tech friend if they should buy Apple, more people will likely say, "yabut," or "nah." We'll see if that makes a difference in a year or three.
To me, it just feels icky. I'm sick of all the spying. I've been in computers since the Commodore. The current computer world is shit because of spying. It killed any passion I had left. It seems like no last vestige of privacy remains.
The person above you gave an example of confusion. I'll admit that I was confused about how this new system would work as well. So there was confusion.
As for market share, this might barely register on people's radar outside the tech community beyond "Apple is trying to prevent child porn."
And who exactly will the market share go to? Do you thing the privacy-conscious folks will go to Google? Or a dumbphone? Or Purism maybe? I mean, no need to fret over my photos if I can't take photos, right?
There are legitimate privacy concerns, and Hacker News users are right to be upset. But that's not an excuse to pretend like this issue is broadly understood.
Or even without calling it 'AI', perceptual hashing like you typically see in applications like CSAM is pretty damn close to ML techniques. The normal thought process is "we don't want child predators to sneak through just by cropping, or sticking a water mark on the image, or some other way slightly modifying the image like the color balance. Can we come up with something that'll hash the same even with minor modifications to the image?". And you basically end up with intentionally overfit AI.
It is a very tenuous theory that this will only end at CSAM with alleged trillion to 1 odds. It is unclear how they justify stopping there - all laws are roughly equal in the eyes of government, why shouldn't an iPhone enforce them all? This incident shows that Apple is comfortable with the idea of using my phone to get me imprisoned:
1. I don't want to financially support anyone going through my private things in conjunction with what is basically the police, looking for reasons to imprison me. Reasonable suspicion first, thankyou very much.
2. I don't trust them to use this in a politically neutral way. This is too much power.
3. I don't trust them to manage the false positive rate. Everyone knows how reliable software engineers are when they claim a system does something. At heart they aren't 1 in a trillion people.
This is a great time to be outraged. I can't really do much about what they do on their servers, but I can certainly get antsy about what happens on my phone.
It's not like this is some new fancy technology no one could do before. There has never been any technical barriers to scanning your phone for insert-reason as part of the OS. The slippery slope argument could be made from the first smart phone. The government could always pressure any company to put a backdoor in any OS. Apple hasn't done this (to our knowledge) so far and this new CSAM scanning feature doesn't remove any non-existing barriers to that end.
They just gave a talk at the USENIX Security Symposium on how this works and the safeguards put in place. The scanning functionality is part of the icloud photo upload module. Security researchers can determine if they change the scanning algorithm in future updates. I don't know what else you can hope for.
Going back to the slipper slope argument, you could say nothing stops them (or Google on Android) from uploading your passcode and share it with X. It's all just software in the end that they write that powers the lock screen and security checks.
> Going back to the slipper slope argument, you could say nothing stops them (or Google on Android) from uploading your passcode and share it with X. It's all just software in the end that they write that powers the lock screen and security checks.
If they start claiming they do that then I'll get angry about that too. You'll notice that only the fringe is accusing Apple of being liars or acting in bad faith here. Apple are pretty up front about what they do.
The problem with this plan is they're claiming they are only going to do selective law enforcement (only 1 US law, only things that are strongly supported by consensus and even then only sometimes). That isn't a position with a reasonable foundation, they are going to change their mind. I want them to change it in the "we don't snoop on people's phones" direction rather than open season.
A possible workaround - but not remotely one I expect them to do - is for them to amend their own terms of use to forbid themselves from expanding this feature without explicit opt-in consent, forbid themselves from making any expanded version mandatory to continue using the device with their software, contractually agree to liquidated damages in case of breach equal to three times the greater of original phone MSRP or actual purchase price plus included taxes/fees and inflation and interest and attorney/court fees, and remove their right to amend this provision without that same non-mandatory opt-in consent.
Again, I don't expect them to do this. But they could add a useful level of trust if they were really motivated to do so.
Not true. If they had previously adopted my suggestion in the old ToS, that itself would either be a breach of that ToS (leading to liability for prohibitively large contractual damages) or would be an invalid change. I suggested that they explicitly remove their right to make this kind of change unilaterally. Terms of Service don't have to allow their authors an unfettered unilateral right of amendment.
And what’s the recourse if a company breaks their own ToS? Also no company will give up any rights as they have no real way in knowing if the gov will come force more images to be added. What then?
The main recourse in practice with the arrangement I described would be a class action lawsuit alleging on breach of contract. I'm quite sure courts would have no problem with the lawsuit, assuming there were no arbitration clause or class action waiver.
As for no company giving up rights, I did say I didn't expect Apple to actually do this, only that they could. Though, companies do actually make binding contractual promises all the time. It's just that they don't tend to do it with consumers or small businesses who have no negotiating leverage and/or no desire to insist on those promises.
Regarding your hypothetical, anything that's actually legally mandated by the government generally overrides contract law since the courts won't enforce illegal contracts and illegality is a defense to breach of contract. They can always do what the law actually requires.
But we're not talking about the government mandating things, since both the iOS 14 behavior and the iOS 15 behavior comply with the law.
> Regarding your hypothetical, anything that's actually legally mandated by the government generally overrides contract law
Which means this action is just a charade, as I pointed out. Because fundamentally it doesn't really matter. The gov is going to do what they do, and Apple is not going to create a situation in which an external group can determine how or if they can be sued in court based on how overreaching they feel that day.
Also go check how many arbitration clauses exist in your ToS, just as an exercise. I'm all for banning arbitration clauses and allowing consumers recourse in actual courts. I don't believe we should have companies sign their death sentence prior to doing business, this raises the barrier to entry and results in even less competitors who have to do the same.
I could hope for being able to trust that an intensely personal device such as a smartphone isn't something that I have to be constantly suspicious of.
Admittedly, that ship sailed a long time ago, but I could still hope.
That ship was never in port. Apple control the hardware and software and always have.
You being able to trust them is a personal choice. It seems people are happy to trust companies that don’t talk about what they do with their data rather than the ones that do. Which, while understandable seems counter intuitive.
I keep hearing this argument and I don't understand it. The entire world would know within about 30 minutes if Apple ever pushed some malware update that did an end run around security and started grabbing files off people's hard drives. I don't trust them, I know they don't do that - and we would know pretty quickly if they did. Now they're saying they will.
Could they write and deploy something overnight that hoovered up everyone's data? Maybe on iOS, less so on OS X, but now they're going to ship with that capability. I don't understand how some people can't see the difference between "they could always push some nasty update," versus literally shipping hardware and software with a backdoor.
People do know the difference. The point of disagreement is in what you said firtst. You say that you know they don't currently ship with a backdoor, while others say you cannot possibly know that.
> I keep hearing this argument and I don't understand it. The entire world would know within about 30 minutes if Apple ever pushed some malware update that did an end run around security and started grabbing files off people's hard drives. I don't trust them, I know they don't do that - and we would know pretty quickly if they did.
Do you know this for certain about any vendor? If a company the size of Apple were pushing the same update to everybody, then it would likely be known about by the world pretty quickly. But... if a targeted signed update is sent to a handful of selected devices, that's harder for the world to find out about. It's risky, but it's definitely technically possible.
> The entire world would know within about 30 minutes if Apple ever pushed some malware update that did an end run around security and started grabbing files off people's hard drives
But it would be very hard to spot new weights of the neural network and a new list of target hashes. These are pretty much guaranteed to change regularly as they retrain the embedding network and/or change the list of known target images. So it will be very hard if not impossible to see what they're searching for. That latest update could just add the ability to recognize CSAM pictures that had meme texts added to them. Or it could change the embeddings and target list to spot unlicensed posting of copyrighted still frames from movies. Or it could now retrieve any picture of people in police uniform. No way to know if you don't have a hunch and a targeted picture you want to test with.
I'd like to respond to all three responses: We have the ability to monitor upstream traffic. If Apple takes away that ability to see what's being pushed on the device, we can look for suspicious traffic on the router or network level. I personally don't let any of my devices communicate with known Apple, Facebook, Microsoft or Adobe IP addresses, so it'll be interesting if they suddenly start popping up new addresses in order to do so.
> I don't trust them to manage the false positive rate.
Let’s compare:
— Apple’s market cap is more than twice of Facebook’s.
— Apple’s user base is less than half of Facebook’s. Even fewer use iCloud Photos (and much fewer upload more than the free 5 GB, shared across photos and other content).
— Facebook is entirely made of UGC with no storage limit for a given user, and their moderators have to review every bit of content to ensure the gore is away from advertising targets’ eyeballs (and, indeed, report CSAM). Apple only needs to review images that trigger multiple hash matches occurring within the same account.
Considering the above, my intuition is that Apple’s NeuralHash would have to be completely broken for a company this size to not be able to afford enough moderators to manage the false positive rate. I sincerely doubt they are so inept, and I’d be willing to live with the potentiality of an Apple employee peeking at a photo of mine once a year (me using Apple tech already implies I trust them enough, as I’d never be able to personally verify every privacy claim they make).
What is worth screaming about, and what is eroding my trust, is the fact that since 2019 Apple’s ToS (quietly changed, presumably, to accommodate this feature currently in the news) give them carte blanche for pre-screening any content that they deem potentially illegal. Unless they tighten up that phrasing to limit it to CSAM only, they’re allowing it to be used as a political prosecution tool.
Edit: Factual error, iCloud Photos does have a free tier.
Your argument here seems to be that Apple is adopting privacy policies that can be favourably compared to Facebook, and have taken a step closer to how Facebook handles data. I don't disagree at all. That is the problem I see here.
Anyone trusting Facebook with a level of access to their life comparable to a mobile phone is foolish. There is no way I would pay money for Facebook to control my phone.
If it costs them too much money to have too many people personally looking over every single positive hit to make sure it’s not false, then they’re not going to do it even if they technically could.
Yes, and I tried to summarize why I think the cost of hiring enough people for this particular job is extremely unlikely to be more than a rounding error for Apple.
What are they achieving? Let's say there is a pedo who has cp on his iPhone. He is scared that apple might be after him but he also sees how this scanning only applies to cloud scanning so apple also gives this pedo option to not getting caught so what is the point ?
I live in a place where the government arrests you for having a VPN installed on the phone and labels you a terrorist outright. My phone is checking with physical frisking on the roadside and content critical of state gets automatic manhandling and trip to the police station where I am treated as a criminal.
How I see this as a problem not because I am not going to buy an iPhone in future, but because such ideology would be made normal. That is what is scaring me
The technical implementation is trying to hide the fact that the design mojo of this system is an actual Backdoor for governments of the world to oppress, censure and do whatever they like.
The technical implementation is optimized towards minimizing the cost for Apple. That's why they scan on the device. And this decision is made with knowledge that in the near future "scanning" will be not limited only to hashes. Scanning and processing will be required for $Some_Cool_Functionality to work.
Gmail is on the Google servers. Did you pay for Gmail?
Did Gmail has access to your computer by default?
So on..
What analogy are you making?
When Gmail advertised "What is on your Gmail, stays on your Gmail"? :)
No, no analogy. GMail scans every email and every attachment. Server side because - of course - it’s an online service. It’s main feature is analyzing and indexing so it’s a direct example of a killer feature based on content analysis. Even spam filters are.
It's also using the resources of your phone that you have paid for to scan and police your own content, much like AV software. I assume the hashes are either uploaded or stored on the phone.
There's plenty of confusion, but what's telling in this story is that the people who are the least confused are among the most opposed to Apple's plans.
>There is no way Apple released their initial PR piece without thinking it through and deliberately fusing all those new features together as one big unassailable initiative.
Something I bet wouldn't have happened when Katie Cotton was in charge. But yeah. Tim Cook thought he need new PR direction. And that is what we got. The new Apple PR machine since 2014.
Your rebuttal is, at best, a specific, straw man instance of "If you were doing nothing wrong then you have nothing to hide."
I needn't be holding child pornography to be concerned about a third party viewing my photos, writing, or other media on a device that is just mine and not published, public content.
CSAM is a hash database. The images are converted to a hash and then compared to the hashes of known pornography of children, not directly viewed.
The weirdly less discussed aspect of this is that anyone who is storing their images of any kind on someone else’s computer and network thinks that nothing could have been viewed before. If Apple or Google or Amazon want to scan the data you store with them they could be doing it, so if that was a concern for a person from the get go then they wouldn’t have been storing their data with third parties to begin with.
> The images are converted to a hash and then compared to the hashes of known pornography of children, not directly viewed.
I honestly don't understand why this is a relevant point. It's still surveillance.
> that anyone who is storing their images of any kind on someone else’s computer and network thinks that nothing could have been viewed before
I don't think that's the confusion. I think a huge part of the issue is that the surveillance is not taking place on someone else's computer, it's taking place on your smartphone. Yes, Apple says it only happens if you're uploading to the cloud -- but that's just Apple saying "trust us". If they did the scanning on their computers instead of yours, it wouldn't be necessary to trust them on this point.
That’s true of any software you’d use anywhere if you accept the updates of the creator. Linus Torvalds could accept an update tomorrow that surveils people’s data and YES people might notice but plenty of people just accept updates and move on (if you’ve done code reviews you know how arduous multi-hundred or thousand line contributions can be to review).
My point is we've already been taking the same risks and the only reason it’s something now is because it’s a transparent process. It’s always a “trust us” scenario unless a person routinely scans all software they is and all updates for malicious server calls or some other kind of recording of data and maybe opening of a back door.
It's not just this. This is a major push, certainly, but... as we come up on about a decade of smartphones being more than "that weird nerd phone one person I know has" it's worth stepping back and looking at the benefits and costs.
Where you put these will depend on your view on a lot of the issues, certainly.
But, in the past decade:
- Every interaction with your primary device is now, by default, an opportunity for aggressive data collection, often in ways even the people who write the software don't know (because they rely on tons of other libraries and toolkits that are doing this quietly under the hood).
- The default is now that you use a smartphone for everything, with the desktop experience limited or turned into a crappy version of the smartphone version (Image! Video! Scroll, scroll, scroll, never stopping, always seeing more ads! Text, who cares about that ancient stuff?)
- The default has gone from "If you're alone in a social space, you talk to other people" to "You stare at your phone." Certainly was a trend before, with the Walkman/iPod/etc, but it accelerated dramatically.
- Everything has been turned into either a subscription service, or a "Free-to-play" world in which the goal is addiction and microtransactions.
There are plenty of benefits of smartphones, but culturally we're exceedingly bad at looking at the opportunity costs of new technology, and they're increasingly becoming harder to ignore.
If you can honestly evaluate the device and decide it's a net positive, great. But I know an increasing number of people, myself included, who are evaluating them and saying, "You know, never mind. They're not worth the downsides."
Unfortunately, we’re so far down the path that I no longer have a choice.
I’m starting graduate school in the fall. A few weeks ago, I went in to pick up my new college ID card. The security guard would not let me into the building until I downloaded an app called “Everbridge” on my phone and used it to answer a series of health screening questions (ie, have you tested positive for COVID in the past 14 days).
The app was for iOS and Android. There was no web version. There was no option to fill out a paper form. I was not warned in advanced. But I guess it wasn’t a problem for anyone (including me), because who the heck doesn’t have a smartphone? It’s like having a wallet now—an expected requirement for modern life, even in situations when an analog solution could have worked just as well.
So what would they do if you emptied your pockets out and demonstrated that you did not have a smartphone? You pulled out the candy bar or the flip phone?
Again, I'm at a point where I can be a thorny pain in the ass about stuff like this, but you carrying a smartphone, even though you (presumably?) know it's evil means that people can do things like this - expect you to download some large blob of unknown code that you're going to run.
As long as they don't encounter people who literally can't comply, it's fine. It works for them.
I mean, I would have refused to download an unknown app I'd never heard of, but... if I pull out a clearly-not-a-smartphone, what are they going to make me do? Go down the street to Best Buy, buy a phone, and come back?
They wouldn’t have let me into the building. Yes, I assume they wouldn’t have retracted my acceptance and we would have made some arrangement, but I have better things to deal with in my life. I’m on a (Jailbroken) iPhone, so the app should at least be sandboxed—I’m not entirely sure what I would have done on Android.
I understand not wanting to deal with it, but that's quite literally how we got to this situation in the first place - everyone has a smartphone because everyone has a smartphone.
What if your phone was too old to run the app (which looks like a steaming pile, based on reviews)?
Unless there was something in the application documentation about "owning a modern smartphone and being willing to install random applications as required by the university," I would have plopped right down, pulled out a laptop, and started making phone calls to figure it out.
But, again, I'm at a point in my life where I can be a thorny pain in the ass about stuff like this without any real consequences.
For the past week (entirely related to this being a kicker of a motivation on top of a bunch of other simmering long term concerns over Apple and the tech industry in general), I've been carrying around a Nokia 8110 4G - also known, for very understandable and valid reasons, as "The Bananaphone." It's quite literally curved and bright yellow.
The world hasn't ended yet...
It's a bit less of a step for me than other people because I'm already pretty cell-phone hostile. My iPhone (I regret buying a 2020 SE to replace my 6S under the assumption that the 6S wouldn't get iOS 15, which it's getting... maybe...) was pretty well nerfed to start with - very few apps, literally the only apps on my homescreen were person to person or group chat apps (Signal, iMessage, Google Chat, and the Element Matrix client, plus phone, browser, and camera in the bottom). Everything else had to live in the app library thing, which increased friction to use it, and I really didn't have much on there.
But that has been shut down except for a few 10-15 minute windows the past week, and I've been trying, very hard, to work out the transition back to a "dumbphone" (or, as we used to call them, a cellphone).
The main pain point so far is that all my messaging apps used to come to a central point on my phone - so if someone wanted to contact me, it didn't matter what they used, it would ping me if I had my phone on me. Now, that's split (my wife is the main party impacted, I'm pretty high lag on other platforms anyway). If I'm out and about, I can get SMS, but not Matrix/Signal/Chat. If I'm in my office, I can get all of them, but would rather not have a long conversation over T9 - except some of them don't do a great job of notifying me, depending on what machines are running and muted at any given time. Etc. I'm still working this out, and some of it is simple enough - add audio notifications to my "Chat Pi" by wiring in a speaker instead of relying on my phone to chirp if I get a message in Chat or element. That my M1 Mac Mini is going out the door at some point gives me added motivation to solve this.
When out and about, I do at least have the option of tethering to the banana - so I could carry some other device that handles more than the phone does (which seriously isn't much). I'm debating between going back to a small tablet (2nd gen Nexus 7 would be a perfect form factor), or something like a YARH (http://yarh.io) of some variety - a little Pi based mobile computer thing that is exceedingly "We didn't invent smartphones"punk.
I'm at a point in my life (professionally, socially, culturally, etc) where I can happily do "You're weird... whatever..." sort of things with regards to technology, and I'm going to pull the thread until I either figure out alternatives, or determine that they simply don't exist and I can't live without them.
/e/os seems reasonably good for phones. It’s far from an iPhone or even stock new Android but not an order of magnitude worse and none of this file scanning.
Honestly what do you need in a smartphone? Good camera? IDK about you but all I use it for is texting, calling, taking pictures, and maybe checking Hacker News while I'm standing in line. 100% of phones above the $500 mark are going to fit 100% of peoples needs for people like me. Let's be honest, those needs are camera and battery life. What do you need that is the latest and greatest? I am willing to bet that this is fine for 90% of people, including us here.
And we're on Hacker News. People know about ROMs and how to use them. Get a Pixel and throw Lineage onto it. It'll be at minimum $100 cheaper and the specs are pretty damn close (minor trades in either direction).
Integration between Apple devices makes the experience greater than the sum of its parts. If someone switches from iPhone they'll lose the ability to use iMessage on their phone, to receive SMS messages on their Mac, to sync Notes, Calendar, Photos, etc. That's why the alternatives are an order of magnitude worse for me.
What? I can do all this without Apple. I mean I might have to browse photos.google.com instead of opening up my photos folder on my desktop but that's not meaningfully different. I have all these things between an Android phone and a linux computer. It may be in different locations than what Apple has them, but everything does sync if I want them to. I can even sync all these things without Google if I want to and have them go into the corresponding folders on my desktop. How is this an order of magnitude? The same services exist.
Have you used the Apple ecosystem? I haven't used others and I am telling you how it looks from my point of view. I will have to invest resources to discover the things you talk about. And there are lots of assumptions in the things you suggested, like that I am fine with using Google or Linux, or not using iMessage at all.
Yes, and I've used it. I'll admit, it is nice. Apple is great about things "just working," but there are also costs to that feature. I'm not sure why people think this is the only way to have said convenience. The major difference is just that Apple this convenience is by default. Honestly Google provides most of this by default too.
> I haven't used others and I am telling you how it looks from my point of view.
And I've used both, and telling you how it is from my point of view. Stop having strong opinions about things that you admittedly don't have experience with. It isn't a good look.
> I will have to invest resources to discover the things you talk about.
You had to invest resources to adapt Apple's ecosystem.
> And there are lots of assumptions in the things you suggested, like that I am fine with using Google or Linux,
Well Google, Linux, Windows, and third party ROMs like Lineage are essentially the only other options out there, so I'm not sure I made many assumptions. And of the most popular, for phones your options are most likely to be Google (i.e. Android) or iOS. The only other option is... third party ROMs. I guess for desktop you could in fact develop your own (besides Linux or Windows), but I think this would fall under a third party custom ROM (if phone) or Linux (if computer). Then again, you could be a BSD person or someone that insists it is GNU/Linux. Let's be real, that's just pedantic, you know what I mean. I'm not going to name every alternative (that's impossible), especially since I covered by far the most popular ones, which share >90% of user share. Why do we have to be so pedantic?
I don't know man, the more you're fighting against this the more it seems like you're just saying "I don't want to try something new." That's fine, but just be open about it. I get it, new things are scary, different, and require you to relearn some things. But that's a different argument. Just state that argument if that's what you're trying to say. Don't make me work for it. Just be honest.
I am sorry that carrying this discussion won't be productive for me --- and looks like for you as well. I told that there were assumptions, and now there are more of them including the ones about my personality. I hope to have a better discussion in future.
I am too. I haven't pulled the trigger yet, but I'm thinking about one of Google's Pixels. The reason for that specific product line is because they are well supported by a wide range of de-Googled Android variants. I'm leaning towards CalyxOS, which seems to have the best mix of privacy, security, and ability to use apps from the Play Store. But GrapheneOS looks tempting too.
I already own a Pinephone but it's not at a point where I'd want to use it as a daily driver. But they're only $150-$200, so worth taking a chance if you don't want an Android alternative. You may end up liking it. I do know people who are using it daily. It's just not for me. Not yet.
If you want to look into the Android alternatives further, this HN discussion about CalyxOS went into some great detail about that OS, and about other alternatives too.
On a side note: I went to Apple site trying to find that page for all those new features and I could not find one (at least by going to obvious places). The way I was able to get it to link in my posts is by googling it… this whole thing is not yet obvious to laypeople.
Agreed - even if Apple doesn’t back down, giving them hell would make other companies less likely to follow suit. This is a very important line in the sand that they have crossed
I didn't realize at first how much this was going to change my view of Apple. Used to be when I saw the "Verifying xyz..." popup on my laptop, I felt a little more secure. It popped up tonight, and I found myself wondering, "Am I in trouble?"
I guess what I'm saying is, at least for me, this backlash is blurring their entire privacy and security pitch. Apple built a walled garden, told me it was for my own protection, then come to find out the cameras are all pointing to the inside.
This is limited to users of iCloud photos. If you want to store your photos on Apple servers, shouldn’t they have the right to exclude CSAM content? Apple owns those servers and is legally liable. Why is this such a big issue?
Because this way they can encrypt things on your phone and claim that they can't see your photos once on the server (because whatever they had to do with those photos, was already done on your own phone).
(There are many ways this can be a slippery slope, but we don't have to pretend they could just so what ever body else is doing just as easily and they just want to do it on your phone because they are lazy or whatever. This is a solution to a legitimate problem and also it turns out that people are rightfully worried about what's next; those two facts can coexist)
The issue is that the scanning happens on your device just before upload. So now your own device is scanning for illegal activity _on_ your phone not the servers.
The second issue is that it will alert authorities.
In regards to CSAM content those issues may not sound terrible. But the second it is expanded to texts, things you say, websites you visit or apps you use it's a lot scarier. And what if instead of CSAM content it is extended to alert authorities for _any_ activity deemed undesirable by your government
Logically the next step is to scan for any copyrighted content and notify authorities that you're watching a movie without paying for it. After all, it's all about catching criminals, how could you possibly object.
That's a fairly large step, though. Apple cares first and foremost about their reputation. If this feature catches a real predator, it is 100% good PR. Every single false positive that makes it into the news is a huge loss, which strongly incentivizes them to avoid that. The last thing I expect them to do is expand the risk surface for something as trivial as copyright enforcement.
> Every single false positive that makes it into the news is a huge loss, which strongly incentivizes them to avoid that.
Just to be clear, "false positive" in this case means an innocent person is accused of trafficking in child sexual abuse material. It's likely they will be raided.
Sure, that's bad if you're Apple, but it's a lot worse if you're the alleged predator.
The thing (many) people are worried about is that, having instituted a backdoor on the user's device, Apple may lose the ability to control how it is used. In the past, their strongest defense when the FBI asked them to use a backdoor to decrypt a user's device password has been "we don't have a backdoor that allows us to decrypt a user's device password, so we couldn't do that for you even if we wanted to".
From now on, when asked to check whether a user's encrypted phone contains arbitrary content the FBI wants to know about, Apple can no longer say "we don't have a way to do that." Sooner or later, you can bet they will start doing it, whether they want to or not.
CASM is already a pretty huge step into legal gray areas. There is no guarantee by anyone that images in the database are illegal. If the database could be guaranteed to only include real photos directly showing sex with a minor I personally would be all for device side scanning even without upload. However who knows what is considered CASM. I would not be at all surprised if it included Guess ads and cartoon parody. Heck in Australia it is illegal to share pictures of adult topless women with small busts because people may pretend it is child porn.
If this feature leads to anyone losing their job due to incorrect criminal accusations it will not even make the papers because we expect the accused are guilty anyway. Apple won't shed a tear until there is a class action.
While apple owns the servers they shouldn't be legally lake. No more than a self storage facility is liable for the items individuals sure in their units.
FOSTA-SESTA is all about "knowingly" participating in "sex trafficking". a) I'm not sure that you can knowingly have an idea what's in encrypted blobs, so I don't think this would enable E2EE; they seem orthogonal. b) They're not policing a communication medium, but local files. Sex trafficking almost tautologically implies some kind of communication, so the target being local files instead of, say, local storage of apps like messenger and signal seems a bit off as an explanation.
> If you want to store your photos on Apple servers, shouldn’t they have the right to exclude CSAM content?
This seems worded to get a Yes answer. So, yes.
It's a big deal because it's unprecedented (to my knowledge) outside of the domain of malware*. Other cloud providers run checks of their own property, on their own property. This runs a check of your property, on your property. That's why people care now. The fact that this occurs because of an intention to upload to their server doesn't really change the problem, not unless you're only looking at this like an architectural diagram. Which I fear many people are.
A techie might look at this and see a simple architectural choice. Client-side code instead of server-side. Ok, neat. A more sophisticated techie might see a master plan to pave the way for E2EE. A net-win for privacy. Cool. But the problem doesn't go away. My phone, in my pocket, is now checking itself for evidence of a heinous crime.
*I hope the comparison isn't too extra. I was thinking, the idea of code running on my device, that I don't want to run, that can gather criminal evidence against me, and report it over the internet... yeah I can't get around it, that really reminds me of malware. Not from society's perspective. From society's perspective maybe it's verygoodware. But from the traditional user's perspective, code that runs on your device, that hurts you, is at least vigilante malware, even if you are terrible.
> My phone, in my pocket, is now checking itself for evidence of a heinous crime.
I see your point here - this is a slippery slope for Apple.
However I don’t see how anyone could achieve both purposes - no fingerprint reporting and prevention of CSAM storage on Apple servers.
Also, a practical thing to do is to just not store your photos on iCloud but use something else for sync and backup - there might be a startup opportunity here if enough people care.
I don't understand your argument. You say people care about this because it "runs a check of your property, on your property". Would iCloud servers running this check after the photos are uploaded to iCloud be any better?
> "Other cloud providers run checks of their own property, on their own property. This runs a check of your property, on your property. That's why people care now."
iTunes Match is an iCloud service which (if you buy it and opt-in) scans your local on-device music library for copyrighted songs, tells Apple you have them, and then they let you listen to high quality versions of those songs on all your devices. And it's not filename or id3 tag matching, it's doing a fuzzy match that can identify the same song in low quality rips and in different file formats. It would be concievable for them to scan for banned audio lectures, or scan your whole device outside the iTunes library, or change it to check for other copyrighted files e.g. movies. It could concievably be reporting you to the MPAA/RIAA if it finds certain songs along with your public IP address so they can check if that IP address has ever been logged as torrenting those songs. It could "in future" be changed to look for and report evidence of torrenting or movie copying. There's nothing technical or regulatory(?) stopping Apple from saying "people with CSAM on their computers can't use iTunes Match" and making it scan the computer as a condition of use, is there? There's nothing technical stopping a government from asking "can your iTunes Match scan engine report video files in the iTunes library which match popular Tiannamen Square video MD5 hashes?", is there?
I do get that these are not the same seriousness, iTunes Match isn't (so far as we know) scanning to report crime but in so far as "Unprecedented on-device scanning for known content using an opaque database and a closed-source fuzzy-matching engine, it would only take a small change to make it look for other things, governments will definitely pressure them to do that and since they willingly built this system they will definitely agree, and all you can do is trust them", are they not samey enough to be relevant?
> From society's perspective maybe it's verygoodware.
At least in the US, the historical distinction between verygood searches and mal searches is given in the 4th amendment: "no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized."
Of course that has been twisted and stretched before, and this is more of the same. But if literally everything is being scanned, "probable cause" is completely absent from the process. It is a fishing expedition of the exact type that the 4A was designed to prevent.
Because if the content is entirely encrypted(like apple says it is) they aren't legally liable and it's entirely voluntary that they do this.
Also, no one(well, most people) has any issue with photos being scanned in the iCloud. Photos in Google Photos have been scanned for years and no one cares. The problem is that apple said that photos are encrypted on your device and in the cloud, but now your phone will scan the pictures and if they fail some magical test that you can't inspect, your pictures will be sent unencrypted for verification without telling you. So you think you're sending pictures to secure storage, but nope, actually their algorithm decided that the picture is dodgy in some way so in fact it's sent for viewing by some unknown person. But hey don't worry, you can trust apple, they will definitely only verify it and do nothing else. Because a big American corporation is totally trustworthy.
I mean even the verification is problematic. At no point I want an "certified" Apple employee or another parent looking at naked pictures of my kids, for example.
But there's nothing in this proposed implementation that could ever possibly result in that, because random pics of your kids would not be in a database of known CSAM content. So your pics wouldn't match the hash values.
So you are saying the hash values can never result in a collision? That in fact there is literally zero chance that two different images could result in the same hash?
It is essentially true that a cryptographic hash can never collide. But what Apple claims to be using is a perceptual hash. Perceptual hashes can not just be scaled wider trivially, to reduce probability of collision, like cryptographic hashes can.
So in other words, there is a chance for collision. And when (not if) there's a false positive, eventually the system will transmit private information to some underpaid army of people who will be receiving either real child porn or pictures of someone's naked children for confirmation.
Ignoring the nasty aspects of doing that kind of work, I don't see any way to buttress the fact that Apple has taken another step on the universally one way street of ever increasing surveillance. And whataboutism in the form of "oh but other cloud providers are already doing this" is an extremely weak argument because I don't want to use other cloud providers. I liked Apple, because irrespective of their real motives (ie money), they seemed to be privacy focused. The backhanded way they tried to sell this "feature" shows that they are just as bad as the others.
I'm constantly surprised how even people on HN are confused about this - read the white paper apple published. It very explicitly says that they are using a perceptual(similarity based) hash, and we(well, not me specifically, researchers) have demonstrated that it's trivial to produce a picture that isn't even remotely similar in theme but still produces the same perceptual hash.
Apple's solution to this problem is that their employee will actually verify the picture before sending it to authorities. Which again, is one of the problems people have with this system.
If that hash stuff actually works (identifies CSAM with high probability), then that puts Apple in the position of knowingly possessing CSAM. The "One Bad Apple" article a couple days ago mentions that their staff was sometimes exposed to CSAM but that it was unknowing, because the prior probability of an image they looked at being CSAM was quite low. If the probability was high (like > 50%?), it wounded like they would have been obligated to send it to NCEMC without looking at it.
If they were scanning images that were uploaded to icloud on Apple's servers, no one would care. iCloud is not encrypted and Apple provides governments access to iClod data, everyone knows that, and other cloud providers already scan content for CSAM material. The difference is that Apple is doing this scanning on your phone/computer. Right now, they say that only images that uploaded to iCloud will be scanned, but what's to stop them from scanning other files too? There's been a lot of pushback because this is essentially a back door into the device that governments can abuse.
I should have said, "iCloud is not end-to-end encrypted". Apple has full access to everything you upload to iCloud, because they control the encryption keys, not you.
Apple can do anything they want on every iphone, always have, and always will. Whether this feature exists or not changes that in no way, their technical ability to snoop through everyone’s stuff is the same. So far they’ve shown restraint with that ability.
I think what people are getting riled up about is not the technical ability, it’s the lack of restraint, the willingness to search through everyone’s personal stuff on their phones. This is like the cops sending a drug-sniffing dog into everyone’s home once a day, with the excuse that it is privacy-preserving because no human enters the premises, and that only truly bad people will get caught. There is a difference between scanning in the cloud and scanning on device. One is looking through your stuff after you’ve stored it in a storage unit, and the other is looking through your stuff while it is still in your home. Apple’s excuse is that you were going to move it anyway, but somehow that doesn’t actually excuse things.
Personally I don't see on device scanning as significantly different than cloud scanning. I think the widespread acceptance of scanning personal data stored on the cloud is a serious mistake. Cloud storage services are acting as agents of the user and so should not be doing any scanning or interpreting of data not explicitly for providing the service to the end user. Scanning/interpreting should only happen when data is shared or disseminated, as that is a non-personal action.
If I own my data, someone processing this data on my behalf has no right or obligation to scan it for illegal content. The fact that this data sometimes sits on hard drives owned by another party just isn't a relevant factor. Presumably I still own my car when it sits in the garage at the shop. They have no right or obligation to rummage around looking for evidence of a crime. I don't see abstract data as any different.
What does the law say, though? Possession of CSAM by any organization or person other than NCMEC is flatly illegal. Even other branches of government, including law enforcement, may not have any in their possession. My question is -- does CSAM residing on Apple's servers, even when it is 'owned' by a customer, count as them possessing it? What about if it is encrypted?
Ben's Stratechery article explains the distinction:
> (f)Protection of Privacy.—Nothing in this section shall be construed to require a provider to—
(1) monitor any user, subscriber, or customer of that provider;
(2) monitor the content of any communication of any person described in paragraph (1); or
(3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b).
(Edit: it seems like the algorithm does not work like I thought it did, you can basically disregard this comment.
This has been mentioned on here before, but it's known CSAM possession that's illegal. Apple keeps your files encrypted until its algorithm thinks your encrypted file is too similar to CSAM, and then it decrypts it and sends it to Apple for review. There's a few things here.
- The algorithm is a black box, so nobody knows how many false positives it hits.
- Apple's willingness to decrypt files without the consent of the owner makes the encryption seem like a bit of a sham.
- I imagine many are skeptical of Apple's ability to judge CSAM accurately. If I take a photo of my kids in a bathtub, is that CSAM? What about teenagers in a relationship sharing nudes. The law is a blunt and cruel instrument, and we've gotten away without hurting too many innocent people so far because the process is run by humans, but computers are not known for being gracious.
That's fair. Apple did a shit job of explaining themselves, and it has been compounded by a lot of misinformation (deliberate or not) in response. I'm trying really, really hard to moderate my reaction to this whole mess until I feel like I actually understand what Apple intends to do. I don't make platform jumps lightly.
Should every minecraft server be checking if any arbitrary sequence of blocks on that server can encode to a binary representation of CSAM which when hashed matches something in the NCMEC database?
You could argue that a minecraft server is technically in possession of CSAM if that's the case, but you could spend an infinite amount of money looking at various possible sequences and are bound to find many more false positives than true positives.
Services should have a duty to report CSAM when they notice it, but the lengths they should go to search for CSAM should be limited by cost/benefit and privacy concerns.
My impression is that letting people upload CSAM to a cloud service has no positive benefit because of the supposed link between CSAM consumption and CSA, and it carries a very high risk of criminal liability, so there's no incentive for companies to completely ignore the files that users upload. Otherwise, people will eventually notice and the service will be denounced as "pedophile-friendly," and then the law will take notice and force them to give up the data.
This type of scenario is what happened with the messaging service Kik, which was reportedly used to distribute CSAM in private chats. Law enforcement agencies said the company wasn't providing timely responses and that children were being actively abused as a result. This is about as damaging of an accusation you can leverage against a company.
Laws against CSAM worldwide are not going away for good reasons, so there is always going to be a justifiable argument that storing certain classes of data is illegal. Hence, anyone wanting to run a cloud service that stores user data will have to obey by those laws, regardless of how proactive they are in scanning for the material. Absolute privacy in the cloud is impossible to achieve with those rules in place.
But at least you sent them away and can have reasonable expectation to be betrayed. But that your camera device also scans and report what it doesn't like.
According to Krawetz's article, Apple is not allowed to scan the files on their servers without a warrant. That's why they want to run the scan on your phone instead.
I’d expect a secure and privacy focused cloud data storage provider to not know what I’m storing _at all_.
Let’s not beat about the bush, if someone wants to store information in a form that can’t be decrypted by Apple, they can. This is a stupid dragnet policy that won’t catch anyone sophisticated.
Apple focused the last years pitching themselves as the tech giant who actually cares about privacy. They seemed to be consciously building this image.
To now implement scanning of private information and then try and sell this obvious 180degree slippery slope turnaround in the most weasel worded “but think of the children” trope is an insult to the customers’ intelligence.
I was a keen Apple consumer because I felt that even if their motivation was profit, this was a company who focused on privacy. It was a distinct selling point.
I certainly won’t be buying more Apple products.
For me, Apple lost the main reason to buy their stuff. If they are going to do the same thing everyone else is doing, I refuse to pay the premium they charge.
They knew they were being hypocritical, so they were reluctant to even divulge the fact that other cloud providers have already been doing it; they wanted to position themselves as the pioneer.
I can't imagine how they thought this would go well.
It's another example of Apple being stuck in an echo chamber and not being able to objectively assess how their actions will be perceived.
How many times have they made product and PR blunders like this?
It really seems like there are a lot of bad faith arguments in this thread. Of all places I expect Hacker News to understand that you can put custom open sourced operating systems on a phone (aka: flashing). If I can't talk nerdy here, then where can I?
What efforts? Pixels have always been the easiest to flash. Phones have always been hacked. Everything is always hacked. I don't see that changing anytime soon.
If Craig tells me I’m misunderstanding this I distrust them further because I completely understand the full arena of possibilities and not just the narrow intent.
Because of this interview I’ve added Craig to my mental list of executives who need to lose their job over this little stunt of theirs.
Apple has spent billions in engineering and marketing to establish themselves as the privacy leader, all wiped away by this idiotic system so full of holes you could serve it on crackers.
"On 5 August, the company revealed new image detection software that can alert Apple if known illegal images are uploaded to its iCloud storage."
People assumed this opens the door for Apple to alerted of any known file uploaded to its iCloud storage.
IOW, they assumed Apple can check what someone is uploading,1 despite alleged "end-to-end encryption" and a gazilion promises of "privacy".
No one except the people managing the "detection software" know what files the hashes represent.
Theres no way for the owner of an Apple computer to verify what files Apple is actually checking for.
Is this confusion. It sounds more like lack of trust.
1 Mind you, for a majority of computer owners the uploading is likely occuring by default, automatically, outside of the owner's awareness. As opposed to the owner consciously deciding to upload a particular file to a computer in an Apple datacenter. Tech cmpanies know that users rarely change defaults.
Vote with your money. If you own AAPL stock, sell it. And loudly refuse to buy Apple devices. That is the only language organizations like this will understand.
No, there's no confusion. I'm not happy to have my personal files scanned on my personal device that I paid for, simple as that. Apple aren't getting another penny from me.
But according to this article you can avoid this by not uploading photos to Apple’s service. Google is already doing this when you upload photos to Google’s service and Microsoft too.
The distinction is whether the matching happens on-device before upload or in the cloud after upload, it seems. If Apple already does on-device ML, it makes sense they would add more photo processing client-side to take advantage of encrypted or archival blob storage server-side.
Additionally, there’s still the option of using a third-party camera app, which wouldn’t upload photos by default at all.
Shouldn’t you also be unhappy that your personal files are scanned on your Dropbox that you pay for? Is that not the exact same betrayal of trust? I don’t get the whole “because I’m holding the computer it’s happening on there should be different rules” thing.
It's pretty simple to me. If I give something to someone else to take care of, like a stack of magazines, and make no effort to conceal the magazines when I give it to them - they're going to see what magazines I have. We understand this. If they're magazines about something illegal, then of course they might report you to the police. This is like storing your unencrypted files on Dropbox.
If you store the magazines in your basement, locked in a vault (password protected computer), then we have an expectation of privacy. To have the vault manufacturer sneak into your home, forcibly open the safe, and inspect your magazines is a very invasive thing. Especially because if this wasn't all over the news, no one would ever think such a thing is happening.
I am disappointed that none of their messaging at all attempts to explain how the feature won’t be further misused (by governments or others, quietly or loudly) in the future.
Apple said that they are intersecting multiples databases in different jurisdiction to avoid rogue hashes and that this will be available to audit.
They also said that because it's on device, security researchers will be able to check any change to the program. (probably via the Apple Security Research Device Program ?)
How can you figure out what a neural net is trained to find? Are they releasing the data set to verify? That would be the bad images we are told it is scanning for and that would be bad.. is there some 3rd party that can do the audit?
Also now that this is a thing how effective will it be at all? Or these sick people that dumb? After all this news? I do hope they are that dumb but who knows.
> How can you figure out what a neural net is trained to find
See, it's comments like this that clearly illustrate that there is confusion, and many people are still outraged over things they don't understand. A neural net is not scanning your phone for CP. You are conflating two things. Just watch the video that you're commenting on before commenting on it.
The neural net isn't exactly searching for CSAM itself. Its role is to extract perceptual features from the image, and it is applied to both the CSAM images and your iCloud images. If those were the same to start with, then the extracted features will be the same.
As for exactly how they'll do the auditing, I'm confused as well.
781 comments
[ 509 ms ] story [ 1554 ms ] threadhttps://www.wsj.com/video/series/joanna-stern-personal-techn...
For starters they should exclude photos made on the phone's own camera. Because it's literally impossible for a just-taken photo to appear in this database since that only contains already known content found in the wild. And most people's photos would be original content. So it would alleviate a lot of concern while not harming Apple's goals.
If those goals are indeed what they say they are, of course.
Also, this is not a viable distribution method anyway. Every photo introduces more noise. Like dubbing tapes back in the day but worse.
[1] https://news.ycombinator.com/item?id=28091750
[2] https://news.ycombinator.com/item?id=28110159
That isn’t the same as saying it will match things that look kind of the same.
And it's incorrect. The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 etc image and then transformations are applied to them to exaggerate features.
If two images look kind of the same when shrunken down, they will have the same or similar hashes. If two images kind of look the same when shrunken down, then their parent images will also kind of look the same.
Please read the OPs of the two links I posted. They're both from people who work in this field. The latter link is from someone[2] who invented many perceptual hashing methods himself that are used widely across the industry. Both articles touch on this subject, and the first[1] one includes two photo examples. I have built products using these methods, and what is said by these two experts matches my experiences.
[1] https://rentafounder.com/the-problem-with-perceptual-hashes/
[2] https://www.hackerfactor.com/blog/index.php?/archives/929-On...
Nobody is saying false positives are impossible.
Apple is saying false positives are on the order of one in a trillion per user account per year. That doesn’t sound like something that matches images that are only ‘kind of similar’. Yes - cryptographic hashes have much lower false positives rates even than that, but that is a distinction without a difference since both make the risk negligible.
> The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 image and then
Which is it for Apple’s hashes?
There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction.
What can you tell us about the likelyhood of Apple’s hashes to create false positives?
I'm not really concerned with what you're afraid most people will think. Two images that kind of look like one another will have the same or similar hashes. There are literal examples of this in the links I posted above. And it's literally the point of perceptual hashing, to find images that look similar to a source image by comparing hash similarity.
> If you are going to imply false positives are common, then you need to back it up.
I just did with two links I posted above. Twice.
> Apple is saying false positives are on the order of one in a trillion per user account per year.
Sounds like a claim that wasn't replicated or independently verified. Of course Apple is going to say their system is nearly perfect, that's what all companies do. The onus is on Apple to prove that their marketing claims reflect reality.
> There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction.
The onus is on Apple to demonstrate that their methods are remarkably different from the rest of the science and industry.
This is like saying the normal principles of computing don't apply to new Apple products because they might have invented a new brand computing paradigm that isn't anything like any classical or quantum computer mentioned in scientific literature at all. Yeah, maybe they did, but it's unlikely and the onus is on Apple to prove it.
What matters is not what I think, but whether you care about making misleading comments.
> > If you are going to imply false positives are common, then you need to back it up.
> I just did with two links I posted above. Twice.
No, you posted some links that are not about Apple’s system, and you can’t explain how they apply presumably because you don’t understand what Apple is doing.
> Sounds like a claim that wasn't replicated or independently verified. Of course Apple is going to say their system is nearly perfect, that's what all companies do. The onus is on Apple to prove that their marketing claims reflect reality.
So this tells us you don’t know what algorithm Apple is using…
…And are accusing Apple of lying, when it is clear that you haven’t read about how they avoid false positives.
I think the onus is on you to prove your accusation.
> This is like saying the normal principles of computing don't apply to new Apple products because they might have invented a new brand computing paradigm that isn't anything like any classical or quantum computer
That just silly. It’s doesn’t take breaking the laws of quantum or classical computing to build a system with a low false positive rate.
One obvious way would be to leverage multiple images rather than just one. Increasing the sample size of a population sample generally reduces the false positive rate.
Have you considered that someone could build a system this way?
But, setting that aside, can you explain how a first order evasion attack can be used against Apple’s mechanism?
They are a real kind of attack in the lab, but it’s not obvious how they could be used to exploit Apple’s CSAM detection.
If you have reason to think they are a real threat, I’m sure you can explain.
At face value, they indeed don’t. But the premise from Apple’s white paper that their mechanism can’t be tricked is not realistic.
> If you have reason to think they are a real threat, I’m sure you can explain.
Yes, I can.
So this wasn’t relevant.
> But the premise from Apple’s white paper that their mechanism can’t be tricked is not realistic.
They don’t say it can’t be tricked. You are misrepresenting them.
>> If you have reason to think they are a real threat, I’m sure you can explain.
> Yes, I can.
No you can’t, because they are not a real threat.
Well it appears the CSAM scanning algo doesn't have Dost[0] scanning built in so, many people will evade this 'utility' made by Apple
[0] https://en.wikipedia.org/wiki/Dost_test
Oh wait, Apple software don't have bugs though, right? /s
This must be implemented in software. A bug in this software that causes the "only on files that are uploaded" return the wrong boolean value means it will scan local photos.
I never said random files on the filesystem. That's your strawman. In any case, it could well happen as well. Something is deciding what to scan, and that's software.
> end users have no way to confirm whether this claim is actually true
This is true of all proprietary software.
Once they have the capability rolled out, it's just a one-line config change to enable it.
I do not see how this wouldn’t be easily extended to all mountpoints on the device. And again, one needs to have faith and assume the /iCloud binary information on storage is really physically isolated from everything else. Sorry, it is very unlikely they aren’t really scanning, as I said, everything.
Edit: clarity.
In fact there is no filesystem scan at all. There is only a check that takes place during the upload process to iCloud Photo Library, which is separate from iCloud Drive.
In any case the claim that the system will scan files other than photos chosen for upload is just a lie.
The only well documented process in tech is source-available. There is documents and speculation regarding this process but we don't actually know the details beyond what they claim and they're not saying too much.
If you want to say that Apple could be lying or mistaken about what the code does, that is a different claim from whether they have documented what I said they documented.
I don't like it. I wish it had never happened. Fuck the government. But you are wrong and blowing this out of proportion.
I’m not defending Apple, I wish they wouldn’t do this, but I see section 230 levels of lack of understanding out there.
If people understood what was going on, would they be as upset? I don’t know. Apple doesn’t seem to think so.
[1] https://www.engadget.com/2020-01-07-apple-facebook-ces-priva...
>but I see section 230 levels of lack of understanding out there.
Mirror mirror on the wall....
That would require them actually changing their plans though, and it doesn't seem like they're willing to do that (yet).
rained -> reigned
It’s also a bit of a laugh to suggest Jobs didn’t suffer from hubris. See: arguably the Mac itself, $10,000 NeXT Cube, the Power Mac cube…really anything with cubes.
Confusion is the best-case scenario for Apple because people will tune it out. If they had released just the on-device spying, public outcry and backlash would have been laser targeted on a single issue.
Apple gave its legendary fan base a fair few facts to latch onto; the first being that it’s a measure against child abuse, which can be used to equate detractors to pedophile apologists or simply pedophiles (these days, more likely directly to the latter.) Thankfully this seems cliché enough to have not been a dominant take. Then there’s the fact that right now, it only runs in certain situations where the data would currently be unencrypted anyways. This is extremely interesting because if they start using E2EE for these things in the future, it will basically be uncharted territory, but what they’re doing now is only merely lining up the capability to do that and not actually doing that. Not to mention, these features have a tendency to expand in scope in the longer term. I wouldn’t call it a slippery slope, it’s more like an overton window of how much people are OK with a surveillance state. I’d say Americans on the whole are actually pretty strongly averse to this, despite everything, and it seems like this was too creepy for many people. Then there’s definitely the confusion; because of course, Apple isn’t doing anything wrong; everyone is just confusing what these features do and their long-term implications.
Here’s where I think it backfired: because it runs on the device, psychologically it feels like the phone is not trustworthy of you. And because of that, using anti-CSAM measures as a starting point was a Terrible misfire, because to users, it just feels like your phone is constantly assuming you could be a pedophile and need to be monitored. It feels much more impersonal when a cloud service does it off into the distance for all content.
In practice, the current short-term outcome doesn’t matter so much as the precedent of what can be done with features like this. And it feels like pure hypocrisy coming from a company whose CEO once claimed they couldn’t build surveillance features into their phones because of pressures for it to be abused. It was only around 5 years ago. Did something change?
I feel like to Apple it is really important that their employees and fans believe they are actually a principled company who makes tough decisions with disregard for “haters” and luddites. In reality, though, I think it’s only fair to recognize that this is just too idealistic. Between this, the situation with iCloud in China, and the juxtaposition of their fight with the U.S. government, one can only conclude that Apple is, after all, just another company, though one whose direction and public relations resonated with a lot of consumers.
A PR misfire from Apple of this size is rare, but I think what it means for Apple is big, as it shatters even some of the company’s most faithful. For Google, this kind of misfire would’ve just been another Tuesday. And I gotta say, between this and Safari, I’m definitely not planning on my next phone being from Cupertino.
You mean that country which gives a damn about privacy altogether because all those fancy corps are giving them toys to play? You know, those companies which feed on the worlds populations data as a business model. The country which has a camera on their front door which films their neighbourhood 24/7? The country which has listening devices all over their homes in useless gadgets?
You have to be joking or that scale you impose here is useless.
This whole thing will go by fast and there won't be much damage on the sales side. Apple is the luxus brand. People don't buy it for privacy. Most of the customers won't probably even understand the problem here.
The only thing we might be rid of are those songs of glory in technical spheres.
How did that work out for you?
I never did that.
Americans were the topic here. See quote.
Privacy is the main selling point Apple is pushing in their current PR campaigns. They've been slowly building up a brand around privacy with new privacy features.
They've just sunk that entire brand/campaign. Instead of "iPhone, the phone that keeps all your data private", it's "iPhone, the phone that looks through your pictures and actively rats you out to police to ruin your life".
>A majority (72%) of iPhone & iPad users are aware of new privacy changes in recent software updates. When asked how well they understand Apple’s new privacy policies, these were the responses: Extremely well (13%), Very well (29%), Moderately well (21%), Slightly well (9%), and Not well at all (28%). Two in three (65%) users are “extremely” or “very” concerned about their activities being tracked as they use certain websites and apps, while only 14% said they were not at all concerned.
And from https://www.androidauthority.com/android-app-tracking-transp...
>You told us: You really want an Apple-like anti-app tracking feature on Android... Over 30,000 people voted in favor of an App Tracking Transparency feature on Android.
People are becoming extremely conscious of online (and on-phone) privacy issues. Where have you been?
Your statement "don't tell me you believe privacy was at the usual consumers mind when they bought their devices... this is ridiculous or you don't meet many normal users." is itself, ridiculous.
Small example: I run a small app. The number of GDPR-related requests is zero. The number of emails like "can my deleted account and deleted data still be recovered?" is like 1 per month or so.
WhatsApp is also a good counter example. People want privacy, but what they want more is utility and network effects. Most people I know didn’t abandon WhatsApp despite numerous privacy mishaps and I think the same will happen here with Apple. This will blow over – unfortunately.
The worlds largest online companies increasingly making privacy a priority in their marketing are all wrong. Got it.
>Most people I know didn’t abandon WhatsApp despite numerous privacy mishaps
I never said privacy was the top issue, trumping all else. You're absolutely correct that other issues like convenience and network effects are important.
>don't tell me you believe privacy was at the usual consumers mind when they bought their devices... this is ridiculous or you don't meet many normal users.
Is true or false? Because you seem to be contradicting yourself. Are people aware of privacy or not? Is the marketing working or not?
I suspect you were trying to say that people weren't aware previously. Is that correct? Because I don't think anyone would disagree with that.
It isn't like the ultimate goal of protecting children isn't worth fighting for, and the ICMEC considers half the countries in the world having no laws against CSAM to be "simply unacceptable." But companies that insist that everything they host can remain private to everyone are lying to their users, and will have to align their marketing claims with the reality of the law, or this kind of backlash will result.
But in general, there are a lot of other descriptors besides "private" that are nothing more than baseless Corporate Memphis copy.
This is wildly disingenuous.
Apple is putting code on the device which generates a hash, compares hashes, and creates a token out of that comparison. That is 100% of what happens on the device.
Once the images and tokens are uploaded to iCloud photos, iCloud will alert if 30+ of those security tokens show a match, it will alert Apple's team, and they will get access to only those 30+ photos. They will manually review those photos, and if they then discover that you are indeed hoarding known child pornography then they report you to the authorities.
Thus, it would be more accurate to say that apple is putting on your device code which can detect known child pornographic images.
> And it feels like pure hypocrisy coming from a company whose CEO once claimed they couldn’t build surveillance features into their phones because of pressures for it to be abused.
This isn't a surveillance feature. If you don't like it, disable iCloud Photos. Yes, it could theoretically be abused if Apple went to the dark side, but we'll have to see what this 'auditability' that he was talking about is all about.
Honestly, with all of the hoops that Apple has jumped through to promote privacy, and to call out people who are violating privacy, it feels as though we should give Apple the benefit of the doubt at least until we have all the facts. At the moment, we have very few of the facts.
We have every fact we need to know to know this shouldn’t be done, and I’m glad that privacy orgs like EFF have already spoken much to this effect.
Or we can just short circuit the entire issue by deciding firmly we don't want this and punish Apple's behaviour accordingly. Which is what appears to be happening.
> it feels as though we should give Apple the benefit of the doubt
It really doesn't feel like this to me at all. Users have clearly stated: we don't want this. It's time for Apple to simply pull it all back and apologize.
There is no way to determine whether the hashes are about CP or about HK protests.
> ...we should give Apple the benefit of the doubt...
You have to take off your apple branded rose tinted glasses my friend.
Any company as big as apple needs to be scrutinized as harshly and critically as possible.
Their influence on the world is so big that a botched roll out of this sort of tech could be absolutely devastating for so many people, for so many reasons.
I don't care if it's hashed tokens or carrier pidgins. We should only allow companies to act in ways that improve our lives. Full stop.
> Thus, it would be more accurate to say that apple is putting on your device code which can detect known child pornographic images
> If you don't like it, disable iCloud Photos.
> Yes, it could theoretically be abused if Apple went to the dark side [...]
> [...] it feels as though we should give Apple the benefit of the doubt at least until we have all the facts.
No, nobody gets "the benefit of the doubt". The very use of that phrase admits that you are being put into a situation where you could be screwed in the future.
There is zero transparency or oversight into the code that does the scanning, the in-person review process, or the database of images being scanned for.
Apple: We have 29 matches on your device of "known CSAM"[0]. However, despite our confidence level being very high, we won't report it to the authorities because we value your privacy!
[0] for varying definitions of 'known CSAM'.
Check the section "WHAT IS APPLE DOING WITH MESSAGES?" in this article: https://www.theverge.com/2021/8/10/22613225/apple-csam-scann...
This feature is a catastrophe.
I can’t imagine that’s a big issue…
Regardless of how you feel about it, both issues were being completely mixed up by every single person I saw discussing this - even otherwise very technically competent people on this very site.
I've no doubt that it muddied the waters significantly when it comes to discussing this.
https://www.forbes.com/sites/thomasbrewster/2020/02/11/how-a...
Do they mean they haven't been doing it for iCloud Photos, but were arbitrarily doing it for other parts of iCloud?
If they were already scanning, you’d expect more reports since although there is no legal requirement to scan, there is a legal requirement to report detections.
I read elsewhere they they scanned Mail but not Photos.
Which, again, really hits the need for disclosure — so much of the response to this announcement has been heavily shaped by both that secrecy and just springing it on the world without much prior public recognition of this issue.
At some level, if you're uploading files to their servers, you have to trust them. And to a lesser extent if you're using their proprietary software (although you can monitor network traffic and so on.)
> Which, again, really hits the need for disclosure
Isn't that what they did?
There was no reason for these two features to be bundled, other than try to dilute the nefarious one with the benign one.
> [Federighi] said it would do the image-matching on a user's iPhone or iPad (...)
be reconciled with this:
> Mr Federighi said the "soundbyte" that spread after the announcement was that Apple was scanning iPhones for images.
> "That is not what is happening," he told the Wall Street Journal.
without at least one of them be a blatant lie?
Is it the tense of "was" in "Apple was scanning (...)" as opposed to "will start to scan"?
To be clear, this is a distinction without a meaningful difference. Or, if there is a difference, it's that it's actually worse than the alternative (cf. the Stratechery article that's been making the rounds).
If that's right, then this isn't a lie, but it's incredibly mealy-mouthed, misleading, and disrespectful of their customers' intelligence.
Perhaps the nuance is irrelevant or disingenuous, but there is a valid interpretation of his words where he isn’t “blatantly lying”.
"Confusion" is what they're trying to sow now.
It has to match the fingerprint exactly, but the fingerprints themselves are not exact, otherwise they would be useless.
And this is completely beside the point. People's concerns aren't mostly over false positives, they're over the possibility that this feature will be perverted by authoritarian governments. Way to miss the point.
> Mr Federighi said the "soundbyte" that spread after the announcement was that Apple was scanning iPhones for images.
> "That is not what is happening," he told the Wall Street Journal.
That's... exactly what's happening.
https://en.wikipedia.org/wiki/Democracy_Index
> The index is based on 60 indicators grouped in five different categories, measuring pluralism, civil liberties and political culture.
The 2020 report classifies 57 countries as authoritarian regimes, 35 more as hybrids.
What's wrong with you? Why are you still with Apple? Are you paid by them?
But that's exactly what's happening? Most people using an iPhone sync photos with iCloud (especially after they introduced the more cost-effective 2TB Apple One plan), images are scanned before they are uploaded to iCloud, ergo Apple will be scanning the iPhone for images.
They are scanning images on iPhones and iPads prior to uploading those images to iCloud. If you're not uploading images to iCloud, your photos won't be scanned -- but if you are using iCloud, Apple will absolutely check images on your device.
From Apple's Child Safety page:
> Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child safety organizations. Apple further transforms this database into an unreadable set of hashes that is securely stored on users’ devices.
> Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection, which determines if there is a match without revealing the result. The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image.
Source: https://www.apple.com/child-safety/
Yes, they will check the images you have chosen to upload. No ‘scanning is involved’.
Claiming this is ‘scanning users devices’ is just dishonest - it’s obvious that it creates a false dichotomy impression of what they are actually doing.
Don’t do that.
Frame it the way you want. It is the device.
If you say Apple is scanning the device, you are lying. They are not scanning the device. They are scanning photos chosen for upload.
Suppose we know there are people who smuggle drugs on airplanes on their person for the purpose of something terrible, like addicting children or poisoning people. If I run an airport I could say: to stop this, I'm going to subject everyone who flies out of my airport to a body-cavity search. Tim, and Craig, are you OK with this? If I can say, "Don't worry! We have created this great robots that ensure the body cavity searches are gentle and the minimum needed to check for illegal drugs," does it really change anything to make it more acceptable to you?
I was just pointing out a falsehood you wrote about what is actually being done.
Anyway, someone in here can accept what the other can't, so let's leave at that and let history tells.
If Chrome scanned downloaded files for viruses and you described that to someone as "scans your computer for viruses" do you think the listener would come away with an accurate understanding of what was happening and accurate understanding of what they were and were not being protected from?
If BestBuy GeekSquad offered a service to "check your device for problems" and all they did was open your photo collection, would you walk away arguing that it proves the screen and mouse and CPU and disk must function and nobody could expect them to do any more than that, service provided, full marks, that's "the device" checked? Or would you hope that "checking the device for problems" might involve at least exercising all the major features like speakers, wifi, bluetooth, at least once, and preferably with stress and thermal tests?
When the TSA ask you to switch your device on to demonstrate that it's not a bomb, are you on the side of "if the screen lights up, the device is thoroughly and effectively tested and cannot contain anything else" or on the side of "an attacker could make up many ways to make the screen glow while hollowing out the insides, this does not really demonstrate that 'the device' is safe"?
"Device scans photos" and "Apple scans device" imply two very different things about how much is scanned, and you're using the latter because you know that if you describe it accurately readers won't be as panicked as you want them to be.
>They are scanning photos chosen for upload
That's pretty much scanning on the device.
That is different from scanning the device. Saying they are ‘scanning the device’ is a lie.
Yes, Apple could scan the device in future. It’s still a lie to say they are doing it now.
>Yes, Apple could scan the device in future. It’s still a lie to say they are doing it now.
Puh..i am relieved now...wait i don't even have a apple product.
EDIT: For Question below
https://technokilo.com/apple-child-safety-feature-third-part...
>Apple didn’t announce any timeframe about when will they implement child safety features in third-party apps. Apple said that they still have to complete testing of child features and ensure that the use of this feature in third-party apps will not bring any privacy harm
Do they? Where have they said that?
[1] https://www.apple.com/child-safety/
Were you aware of that when you posted it?
The Q&A mentioned has no date or time. No Apple Spokespeople are named. There are no actual quotes. No well known news outlets have mentioned this very consequential detail.
This has all the indicators of a fake.
I'd call that photo scanning... and they are scanning the photos on the device.
This isn’t some ambiguous case that needs to be addressed philosophically. They aren’t scanning anything other than the photos being uploaded.
When people say that the device is being scanned for pictures, they know what that means. So it is fine for them to say that the device is being scanned for pictures.
This is about the 16th time I have seen language just like this used to explain away this concern. I don't know if you realize, but this wording makes it sound like you can select some photos and leave others local. I can find no indication anywhere, including on my phone, that iCloud Photos is anything other than an All Or Nothing singular toggle in iCloud settings. If you have instructions to the contrary, I will be happy to stand corrected.
Seriously, everybody is wording it like this. "Photos you choose..." and similar.
0. Check the comment history, every comment is defending Apple and this has been going on for many months (years?). In fact, I don't see any comment that is not defending Apple. I know I am making very serious allegations, but please go through the comment history to form your own opinion.
I don't believe that the user is a bot though, most comments are 'well-reasoned'.
The amount of time he or she has invested in sticking up for Apple is astronomical.
Do you have any affiliations we should be aware of?
Secondly, I haven’t ‘defended Apple’ in any comments. Indeed there are comments in which I make a judgement about this topic where I say that what Apple is doing is distasteful and offensive.
Elsewhere I have pointed out that if Apple wants to scan people’s devices they have many other mechanisms at their fingertips than this narrowly tailored tool.
What exactly do you think I’m ‘defending Apple’ from? Quite a few of my comments are critical of false or immaculate characterizations of what Apple is actually doing.
If you consider that to be a defense of Apple, then I disagree.
For the most part there just seems to be a lot of confusion about what Apple is doing, and general frustration about the state of computing.
Do you really think of these as ‘attacks’ on Apple?
If you intentionally make someone think that anything other than the photos they are uploading are being scanned, then you are deceiving them.
Deceiving people to make your point doesn’t help anything. It just makes you a liar and the other person misinformed.
If this is a massive privacy violation in itself, then you shouldn’t need to exaggerate it.
Still, I think Apple misjudged the whole cloud vs. device thing in this case. They’ve historically preached a lot about how everything should happen on the users device, not the cloud. I think that got myopic for them, and led them to this decision.
But in this case I think users would be much happier if Apple had just said “under pressure from law enforcement we are now scanning photos when they arrive at the iCloud data centers. If you don’t want scanning don’t use iCloud.” Because it’s not so much the scanning of uploaded photos that has people upset, it’s the fact that the scanning and phoning home is baked into the device itself.
Many people (like myself) are worried about the slippery slope where this is turned on for all photos, since why not? Not all abusers will upload their CSAM content to the cloud, why wouldn't Apple flip a flag in the future to scan everything, including photos and downloaded content? If they are serious about fighting CSAM and have this great privacy preserving platform, I don't see why they wouldn't do this?
The point of the feature is to prevent people from using iCloud to distribute CSAM. If you’re recording it with your phone, it’s no different than using an slr camera. The cloud part is what they’re worried about.
edit: like many comments here already say, reporting doesn’t sound terrible for CSAM, but nothing about the feature guarantees it wont be extended to other kind of content.
Right, which is why it's so utterly baffling that they don't do this scanning server-side instead of client-side.
Also, the matches are supposedly only to actual babyporn pictures. We have 0% way to verify that, as even employees of NEMSEC are not all allowed to view them. Such DBs are often full of unreviewed fluff, and why not unrelated photos entirely, cookware, computer cases, who knows, as long as “some degree of matching” with your photos allows Apple to send a .zip to the police.
In terms of the privacy intrusion, what difference does it make whether the images are scanned on your device or on their servers? They're getting scanned just the same either way.
But if they did it on their servers, it would provide a technical impediment to expanding beyond the use of iCloud, and remove the need to trust Apple as much. That seems like it would be much better for users while still allowing the functionality they claim to be seeking.
That's for now. The first update can change that and you will have no recourse.
What makes you say this? They announced this change after all. Why wouldn’t they announce future changes?
And I don't think that would include photos saved in other apps.
Apple promising not to use the scanner is a weak promise they know they can’t keep (NSLs)
Five years from now it'll be: "you're wrong"
HN will as usual agree and take pride in being wrong.
I suppose it's great if you're looking for entertainment value. For rational, informed discussion of the technology and its political and social ramifications, not so much. It's just the same refrain of "we never bother to actually RTFA but we imagine a boot stomping on a human face forever."
It's also great to see the handful that have changed their minds.
Most of the commentary on this more recent issue is similarly misrepresented and inaccurate.
I think Apple's mistake here was a PR one, they shouldn't have announced this until they had e2ee ready. Then they could have announced that which would have gotten most of the (positive) press attention. Then they could have gone into details about how they were able to do it while still fighting CSAM.
It wasn’t “one of the most widely accepted and ergonomic grips people use”.
The entire thing was a non-scandal, it wasn’t a real issue. In a lot of ways this is similar.
Compared to their Keyboard which took nearly 3 years before they have a programme for free repair.
There is no way Apple released their initial PR piece without thinking it through and deliberately fusing all those new features together as one big unassailable initiative. It was typical my way or the highway.
Which also make it funny now that they attempt to distinguish between them and run into same hole that they dug for other people.
[1] https://www.apple.com/child-safety/
But in this case, of course, if you're an adult, the Messages part of this doesn't apply to you at all, and the photos part can be completely avoided by not using iCloud Photos.
[0] https://ibb.co/zZMtqQk
My knowledge in the space is limited to the GS range, having been privy to a few storys of romance between rider and bike while crossing continents. A beauty of its own.
Apple:
- Isn't going to be remote work friendly.
- Shut down internal polls on compensation.
- Bows to the FBI, CIA, FSB, CCP.
- Treats its customers as criminals.
- Treats its employees as criminals.
- (Spies on both!)
- Doesn't let customers repair their devices or use them as they'd like.
- Closes up (not opens up) the world of computing. Great synergy with the spy dragnet.
Take your time and talent elsewhere. This bloated whale is bad for the world. There are a lot of good jobs out there that pay well and help society.
There's a facade that we really work for other reasons, and money is just an inconvenient byproduct. During a job interview, you may be asked "Why do you want to work for us?". And for some reason "So I can afford to buy food" is not a good answer.
But usually not solely for the money. And usually there are lines people aren't willing to cross just for the paycheck.
As opposed to what??? Free apple stickers??
Of course we might not know of cases that got resolved by internal pressure, because they got resolved, however we do know this was not one of them.
I don't know what you and tharne are talking about here. There was definitely confusion. HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone. You are allowed to be against Apple's plan while still recognizing that many people didn't understand what exactly was part of that plan.
It was not universally understood that this would only apply to photos sent to iCloud.
It was not universally understood that this was only looking for previously known CSAM.
It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
I understand if you consider the where more important than the others, but it is simply a fact that there was confusion on what exactly was happening here.
This is ignorance in extreme.
To the extent that other parts of this story was explained to us by Apple, I did try to clarify some exaggeration in other thread.
A lot of the contention wasn't about the specifics of their plan, but rather how subtle changes could vastly expand the scope of their plan.
"this would only apply to photos sent to iCloud." for now, until scope creeps.
"this was only looking for previously known CSAM." for now, until scope creeps.
"using some sort of hash matching so photos you took yourself would not trigger the system." well this one is immediately concerning even within claimed scope because there ARE going to be false positives that apple records some database. Millions of iphone users are going to have a non-zero "possible childporn" score.
They are building an engine for iphone users to self-incriminate. If they rigidly hold the scope to only what they announced and never expand, it could be argued that this is a reasonable concession to fight CSAM. However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
The how helps show us how changing this system is not a subtle change. It isn't like they can flip a switch and suddenly they are identifying new suspected CSAM on people's phones. That would require a new system since the current one is only hash matching.
>However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
This is an arbitrary line that is being drawn. These are photos that are marked for sending to iCloud. Whether the scanning happens on the phone before they are sent or in the cloud after they sent is largely immaterial when it comes to the impact of the code. People are acting as if the line Apple drew was motivated by technology. That was never the deciding factor. Technology is the easy part here. That line was only a policy line and that policy has not changed. Only photos that are sent to iCloud are scanned. If you fear Apple changing that policy going forward, you should have always feared Apple changing that policy.
This is a strawman. Identifying *novel* CSAM is a very hard problem to do accurately - the reason they can't flip the switch is because they don't have the technical capability. All of the other things people are concerned about are things that apple does have the capability to do.
EDIT to reply since at max thread depth: *novel* image detection was never on the table, I think you missed that word
> This is an arbitrary line that is being drawn.
It seems the vast majority of people in this thread disagree that this line is arbitrary.
EDIT: whether the threshold is "verging on impossible." depends entirely on the effective false positive rate. If apple's claimed 1 in 1 trillion rate is true, it's probably not a concern. However, I find it unlikely that perceptual hashes on portions of images won't have higher false positive rates when subject matter is similar (non-CSAM legal adult porn, or images of children in swimsuits, etc). If that rises to 1 in 1 million for these types of images, that's hundreds of thousands of people being falsely accused.
You just used this as an argument against this system. Why do you fear this if you don't think Apple can even accomplish this technologically?
>It seems the vast majority of people in this thread disagree that this line is arbitrary.
I would argue that people who believe that this decision crossed that line were being naïve to not have always known this was a possibility. I don't think this move brings us any closer to Apple scanning our devices for anti-government memes or whatever the fear is because what was stopping that was always more policy than technology.
Also to go back to your earlier comment, in the time since you posted it has now been revealed that this system needs to trigger 30 times before any action is taken. The odds of 30+ false positives is likely verging on impossible.
Maybe. I wonder how many people are choosing not to respond because they’ll only be voted down by the people who feel very strongly about this. I’ve curtailed and hedged many my contributions to this topic on HN because of this.
This is a common issue when social media is used to debate matters which are highly emotionally asymmetric.
I'm a huge Apple fan and a Mac devotee of 30+ years. I love my iPhone. If this thing becomes real, I will go shopping for another phone.
This is absolute insanity. I can't believe they even thought about launching this.
I'm completely disgusted.
I don't think this is right. The apple pdf on the features says
> With the initial match threshold chosen as described above, iCloud Photos servers learn nothing about any of the user's photos unless that user's iCloud Photos account exceeded the match threshold.
This implies to me that they are using some sort of encryption to prevent iCloud from learning even how many matches there are until the threshold is met.
- Whose fault is it that those points were not clearly communicated?
- Who wrote the perceptual hash matching code?
- Who is allowed to audit the code, the review system, and the hash database?
- Who updates this code?
- Who decides if your phone OS is updated?
- Who decides the iCloud upload defaults?
- Who decides if you are reported?
- Who asked for this feature?
* It was not universally understood that this would only apply to photos sent to iCloud.
Since the scanning doesn't happen on iCloud, this distinction is irrelevant.
"We are going to intrusively scan the subset of your photos that you care enough to back up to the cloud that we've been pushing to you for years" is pretty clear.
* It was not universally understood that this was only looking for previously known CSAM.
It was only looking for whatever is in an opaque database which, according to a third party we don't have any contract with, contains CSAM.
* It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
Yeah right, I feel totally safe knowing that I won't be falsely reported to FBI by a "some sort of" hash matching.
Here's a hash function: f(x) = 0 for all x
It's "some sort of" hash, too.
It’s really not irrelevant. A third party photo library that avoids using the PhotoKit library would not be touched by the CSAM detector. There are many of these on the App Store.
One step further and store the photos encrypted, with a custom renderer that decrypts the content on the heap, and that would take some tremendous performance-hitting detection abilities it’s extremely unlikely to ever happen.
OK.
Misrepresenting each other’s arguments is not how you have a discussion.
I hope that's precise enough, and sure, everyone is welcome to make their own decisions regarding this.
> HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone.
They're confused about this. NeuralHash doesn't look for pictures of naked kids. It looks for pictures that are identical to the ones they've put in their signatures list.
The problem is that Apple claims that the signatures in their list are all pictures of sexually-abused kids, but we have no way of verifying that. Heck, they don't even have any way of verifying that. Everyone just has to take NCMEC's word for it.
The public does not know what the false positive rate is for 'average iphone user pictures'. As engineers we can be certain the false positive rate is not zero. This means that some number of iphone users are going to have non zero "possible child pornographer" scores in the apple database.
The false positive rate is crucial to understanding how concerning this should be. If the average iphone user has 1000 photos, and the false positive rate is the claimed 1 in 1 trillion, there is a 1 in a billion chance that you'll be flagged as a potential child pornographer. (~1 in the world will be falsely accused). This seems reasonable enough with the apple-internal screening step.
If the chunking and perceptual hashing functionally ends up having a much higher false positive rate for images which have similarities to the dataset (parents' pictures of kids playing shirtless, legal adult porn, etc), the false positive rate could actually be more like 1 in 1 million or worse. In which case there are potentially hundreds of thousands of people who will be falsely accused by this system.
How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.
I yet to understand what happens to people who only have those synthetic positives? Regardless of what counter threshold is, can’t those people be hoovered up by a subpoena of counter >0 ?
That's really really really user-hostile design.
Even that doesn't come without issue. How long before '1' becomes the value, because, say for example the number is ten, there's also a horrendous PR spin of "Apple has a high degree of suspicion that you have CSAM on your device, but since there's only 8 images, they won't do anything about it" - "Apple allows up to X non-reported CSAM images on Apple devices" is hard to represent in any positive fashion.
If allowed to go forward, it is only a matter of time before the capability is expanded.
So it's a big no to the scanning capability, you would think that Apple had gotten the message by now.
And the other initiative is also open for abuse, by allowing the device administrator to spy on the user. Admittedly not as bad as the on-device scanning.
EDIT: It has now come out that you need to trigger the system 30 times before Apple acts on it. I can't imagine the odds for someone to have 30 hash collisions.
But I haven't seen any positive discussions about it, which is odd.
I don't like the feature. Putting this on the client device is dubious and should never have made it past the brainstorming stage.
Having said that, technology companies, big and small, are bound in the US to do this. By law. If anything Apple was by far the laggard of the bunch (with reporting counts magnitudes lower than peers, despite a larger customer base). As I said in another comment, no company can protect you from your government.
Much has been made about it being on device, which while a serious optics issue...the hot takes being given on here are manifestly absurd. Like, literally the company that holds all of your data, all of your passwords, all of your info and you need to invent slippery slopes to imagine up what they "might" do?
If they want to have their way with your data, they could have been doing it for decades.
They should never have announced two very different systems at the same time. Contrary to some of the insincere claims given in this very thread, there is massive disinformation and confusion about them. In the end I feel like 98% of the "the end is nigh!" comments are by long time Apple detractors who just see this glorious opening.
And while I still hope that Apple says "Mea culpa, we're just going to scan on the ingress to iCloud Photos", whatever they do in a month this is going to be completely forgotten.
Other companies scan their servers instead. And what law banned E2E encryption?
This potentially means all of iCloud, not just photos, could start to use E2E encryption as well - which is fantastic.
What law do you think banned real E2E encryption?
And I said information. The hash matching is information.
[1] https://www.apple.com/child-safety/pdf/Security_Threat_Model...
They have to scan things that are not photos. What if the bad guys just zip their photos and upload that?
There also needs to be a solution to CSAM uploaded before NCMEC had a chance to tag it, especially to cases where the bad guys uploaded their CSAM and deleted it from their iPhone. What happens then, the bad guys get E2E and nobody can find them? There has to be a technical solution in mind for this, and everything I can think of has implications (Let the iPhone store hashes of deleted images? Would it be enough to scan also during download?)
IMHO, any serious attempt to find CSAM using Apple's client-side approach requires more scanning, and Apple not being forward on that makes me trust them less. Also, the moment they expand the scanning, we should think carefully if there actually are any privacy benefits.
Putting it server side is categorically worse. Putting it in the client SDK for iCloud (architecturally speaking) rather than on cloud storage or in the OS is clearly the better correct technical choice, tying surveillance’s hands in a way server side or OS would not.
Most every client SDK routinely checks content before upload, it’s a best practice. Careful examination suggests this was engineered better than that practice.
(Note: even tech trade posts such as LWN, Stratechery, or Daring Fireball trying to write well about this need to sit down a minute and have how it actually works walked through for them, as do many in this community.)
FWIW, I agree with much of the rest of your post except the rationale for low reporting counts.
It's basically engineering with the goal of creating a perception of privacy rather than actual privacy. I don't really care that much about Apple policing what you upload to iCloud, but this disingenuous architecture does annoy me a bit, and there's also the added insult of having a device in your pocket that by design works against its owner (reminiscent of "treacherous computing"). These problems would go away if they just did the check on the server side.
The architectural reason is to do things the server couldn’t.
> You're uploading a photo to iCloud, encrypted with a key that Apple controls.
The architectural reason is so the server doesn’t have to be able to read the photo, and it need not be a key Apple controls.
Your first two sentences are the exact reason to do it on the client instead of on the server, such that it’s possible to have e2e encryption opaque to the server.
Why? Putting it server-side means that it's only possible to examine the images that they claim they are targeting -- those going to the cloud. That seems like a much better and more private way to do it, because it's putting the surveillance "in their house", so to speak, instead of mine.
The list of apps is such a treasure trove. Signal? Clubhouse? Telegram?
https://developer.apple.com/documentation/security/complying...
I need to research what Signal does on iOS. My next canary is encryption of messaging apps other than iMessage.
Plenty of people believe that the Facebook and Instagram apps are recording audio 24/7 and target you ads based on the speech the apps hear. That doesn't stop people from using the apps.
A few years ago some of the most famous people in their world had their iClouds accounts hacked and had their naked photos leaked. That is a lot of people's worst fear. People literally commit suicide over this sort of thing. It didn't hurt the iPhone's market share.
People largely don't care.
1. The penalty of social ostracism due to the network effect. This is a severe punishment to most humans - particularly in today's socially disconnected world. Without these apps, a large # of people would not have any contact with much of their social circle - including family.
2. Learned helplessness. I think that many people have just given up. Even if they knew how to fight for their privacy, they see time and time again that money always wins.
Who knows how much it will matter in the end? But it is hard to argue it doesn't matter.
Take all the paranoia and “muh privacy” around contact tracing or the European Green Pass: It’s as anonymous as it can be, yet millions of people argue against them across the political chasms that separate them. So yes: unless it becomes a divisive topic (which would be undesirable), this CSAM scanning will go away with the news cycle.
About your comment on celebrities’ “sex-tapes”. I guess it takes a certain kind of extroversion and (positive) narcissism to be one, and they’re expected to be gossiped about or to show their bodies in movies or photos so I don’t think those victims blinked that much - some like P. Hilton probably manufactured some porn to ride the wave. On the other hand, revenge porn did drive ordinary people to suicide because in this case the victims weren’t prepared or expected to let the public to see that.
This is old thinking. In the 90s and 00s it was miraculous to trade in a little privacy for some awesome free service on the internet. Google, Facebook and others became huge, and everyone has been influenced / manipulated by them and people are getting tired of having a family dinner conversation next to their smart speaker and seeing ads for six weeks for Depends diapers because someone told a bad joke at the dinner table.
Disagree.
Something this ludicrously intrusive, over-reaching, reckless and trust-destroying is a dealbreaker for exactly the demographic of technology enthusiasts who influence others purchasing. It may not be instantaneous, but it would certainly propagate.
It's anecdotal, but I know zero people who are NOT reconsidering even lifelong loyalty to Apple over this.
It's just too crazy big a wrong, it's like the company just had a nuke go off inside it and is trying to pretend nothing happened.
They may as well have announced a partnership with Trump to put MAGA engravings on all future products, and then in the ensuing furore say they "regret the confusion", whilst carrying on with it regardless.
Though I suppose in that scenario they'd at least be targeting a significant market.
The same can't be said for the size of "sign me up for software-automated police raids" market. A market whose naiveity-induced initial "size" would rapidly shrink after the first few innocents went down, as they absolutely would.
I mean depending on the targets chat software preferences, could a malicious actor potentially ruin an Apple iOS users life just by sending them an image?
There are many troubling scenarios one could imagine. In fact, there is nothing but troubling scenarios.
It's not a can of worms so much as a wormhole, blasting an endless stream of worms at the speed of light.
I'm left not just questioning Apple's leadership, but - honestly - their mental faculties. Really.
No innocent person wants to walk around with an automated snitch in their pocket - I mean, hacking? Bugs? Oversights? Just the overall preponderance of fear that every millisecond you walk around with this thing, it - and its parent corporation and all the depersonalised machinations that go along with it - could be busy organising a blithely mistaken police raid on your family?
Say, because someone you've never met, sent you a message on that new chat app you forgot you installed, that autosaves all media to your iCloud? Or because someone stole the spare phone you keep in a drawer at work, used it for God-knows what and you didn't even notice it was gone? Or, maybe your teenage son got sent something from his teenage girlfriend who unbeknownst to any of them had her phones images uploaded and subsequently catalogued and flagged? Or any number of other entirely plausible scenarios that provide the very reason we have law enforcement protocols and procedures for reporting crimes and that are complex, nuanced and have evolved over hundreds of years and mountains of cases into a massive structure that exists primarily to protect the innocent from exactly this kind of freaking crazy shit?
And Apple expects people to pay them to carry the weight of all that around with them?
I keep seeing this asserted, but there is never any legal citation. What exactly compels a software company to make their software product scan for CSAM?
I get that a service provider like cloud storage may need to scan what they themselves are storing to avoid possessing such material themselves. And iCloud could scan uploaded blobs all day to fulfill their legal department's recommendation.
But what exactly compels a software developer to include a content scanning function in code they distribute? And does this requirement also apply to the authors of rclone?
Apple only scans photos being uploaded to iCloud photos. Google scans. Facebook scans. Microsoft scans. Even tiny image hosting sites scan.
Apple decided to implement this functionality on device, but they could as easily (with much less fanfare and dissent) have placed it on the ingress to iCloud.
When iCloud scans stored files like Google Drive, nobody complains. It's understandable that risk adverse legal departments have come to the conclusion that doing such things is necessary, to avoid a company being in possession of trivially-discoverable CSAM. And from an individual security perspective, you should consider everything you upload unencrypted to be the subject of similar analysis.
If iOS were to encrypt all files before uploading, making any iCloud scanning mostly pointless, Apple would still not be running afoul of any law. Apple is making general software for end users, and encrypting files to upload would be doing the basic user diligence I alluded to. If users end up using the software to do bad things, those specific users are liable and not Apple.
As far as I am aware, there is no legal requirement for a software developer to modify their software to perform scanning of content it will process while being run by other people. But this is what is implied when you say that Apple is forced to do this by law.
Furthermore if this development was driven by iCloud worrying about legal liability from the combined system, then iCloud should be spun out into a separate company that cannot affect the development of the iOS software.
Law requires reporting CSAM. They are not required to scan for it.
They are surely not required to scan client devices, and that was a foolishly, ill-considered plan (that I still would wager they will abandon), however they absolutely must scan iCloud Photos unless they were technically incapable of doing this. US law doesn't say "you go to jail if you don't", it says "you face enormous liability if you don't".
Those are not US companies but I've never heard of AWS, DO, etc. scanning people's storage either.
Pre-Snowden, that was insourced by the NSA.
Except with an iPhone, you don't have a choice.
I'm tossing the mac and iphone because my phone is mine.
Yep. They just altered that deal. (Darth Vader quote deleted)
Upcoming contenders like Purism [1] and the Pine Phone [2] will start gaining a great deal more traction from this. Other SV firms will sense business opportunity..... If merely 5% of the TAM around mobile is willing to prioritize non-spying features that would be enough to stand up very healthy businesses.
It isn't like an iPhone is very customizable, repairable, or that usable with all the App restrictions Walled-Garden stuff.
[1] https://puri.sm/products/librem-5/
[2] https://www.pine64.org/pinephone/
I’ll bet you $500 to the charity of your choice that this won’t come to be. Set the terms on how you want to measure the outcome.
Since you want to have a friendly competition around "put your money where your mouth is" will look into whether or not Purism is accepting investments and what the terms are. AAPL valuations are pretty lofty right now at ~$149 a share if you'd be interested in the reverse :)
AAPL closed 90 cents short of an all time high share price today. Why isn’t the market pricing in the loss of market share?
It's a hit to their brand from technically knowledgable people for sure though. When someone asks their tech friend if they should buy Apple, more people will likely say, "yabut," or "nah." We'll see if that makes a difference in a year or three.
To me, it just feels icky. I'm sick of all the spying. I've been in computers since the Commodore. The current computer world is shit because of spying. It killed any passion I had left. It seems like no last vestige of privacy remains.
As for market share, this might barely register on people's radar outside the tech community beyond "Apple is trying to prevent child porn."
There are legitimate privacy concerns, and Hacker News users are right to be upset. But that's not an excuse to pretend like this issue is broadly understood.
1. I don't want to financially support anyone going through my private things in conjunction with what is basically the police, looking for reasons to imprison me. Reasonable suspicion first, thankyou very much.
2. I don't trust them to use this in a politically neutral way. This is too much power.
3. I don't trust them to manage the false positive rate. Everyone knows how reliable software engineers are when they claim a system does something. At heart they aren't 1 in a trillion people.
This is a great time to be outraged. I can't really do much about what they do on their servers, but I can certainly get antsy about what happens on my phone.
They just gave a talk at the USENIX Security Symposium on how this works and the safeguards put in place. The scanning functionality is part of the icloud photo upload module. Security researchers can determine if they change the scanning algorithm in future updates. I don't know what else you can hope for.
Going back to the slipper slope argument, you could say nothing stops them (or Google on Android) from uploading your passcode and share it with X. It's all just software in the end that they write that powers the lock screen and security checks.
If they start claiming they do that then I'll get angry about that too. You'll notice that only the fringe is accusing Apple of being liars or acting in bad faith here. Apple are pretty up front about what they do.
The problem with this plan is they're claiming they are only going to do selective law enforcement (only 1 US law, only things that are strongly supported by consensus and even then only sometimes). That isn't a position with a reasonable foundation, they are going to change their mind. I want them to change it in the "we don't snoop on people's phones" direction rather than open season.
Again, I don't expect them to do this. But they could add a useful level of trust if they were really motivated to do so.
As for no company giving up rights, I did say I didn't expect Apple to actually do this, only that they could. Though, companies do actually make binding contractual promises all the time. It's just that they don't tend to do it with consumers or small businesses who have no negotiating leverage and/or no desire to insist on those promises.
Regarding your hypothetical, anything that's actually legally mandated by the government generally overrides contract law since the courts won't enforce illegal contracts and illegality is a defense to breach of contract. They can always do what the law actually requires.
But we're not talking about the government mandating things, since both the iOS 14 behavior and the iOS 15 behavior comply with the law.
Which means this action is just a charade, as I pointed out. Because fundamentally it doesn't really matter. The gov is going to do what they do, and Apple is not going to create a situation in which an external group can determine how or if they can be sued in court based on how overreaching they feel that day.
Also go check how many arbitration clauses exist in your ToS, just as an exercise. I'm all for banning arbitration clauses and allowing consumers recourse in actual courts. I don't believe we should have companies sign their death sentence prior to doing business, this raises the barrier to entry and results in even less competitors who have to do the same.
I could hope for being able to trust that an intensely personal device such as a smartphone isn't something that I have to be constantly suspicious of.
Admittedly, that ship sailed a long time ago, but I could still hope.
You being able to trust them is a personal choice. It seems people are happy to trust companies that don’t talk about what they do with their data rather than the ones that do. Which, while understandable seems counter intuitive.
You cannot be serious.
Could they write and deploy something overnight that hoovered up everyone's data? Maybe on iOS, less so on OS X, but now they're going to ship with that capability. I don't understand how some people can't see the difference between "they could always push some nasty update," versus literally shipping hardware and software with a backdoor.
Do you know this for certain about any vendor? If a company the size of Apple were pushing the same update to everybody, then it would likely be known about by the world pretty quickly. But... if a targeted signed update is sent to a handful of selected devices, that's harder for the world to find out about. It's risky, but it's definitely technically possible.
But it would be very hard to spot new weights of the neural network and a new list of target hashes. These are pretty much guaranteed to change regularly as they retrain the embedding network and/or change the list of known target images. So it will be very hard if not impossible to see what they're searching for. That latest update could just add the ability to recognize CSAM pictures that had meme texts added to them. Or it could change the embeddings and target list to spot unlicensed posting of copyrighted still frames from movies. Or it could now retrieve any picture of people in police uniform. No way to know if you don't have a hunch and a targeted picture you want to test with.
It's Sir <firstname> or Sir <firstname> <lastname>.
https://www.geni.com/projects/Naming-Conventions-for-Knights...
Let’s compare:
— Apple’s market cap is more than twice of Facebook’s.
— Apple’s user base is less than half of Facebook’s. Even fewer use iCloud Photos (and much fewer upload more than the free 5 GB, shared across photos and other content).
— Facebook is entirely made of UGC with no storage limit for a given user, and their moderators have to review every bit of content to ensure the gore is away from advertising targets’ eyeballs (and, indeed, report CSAM). Apple only needs to review images that trigger multiple hash matches occurring within the same account.
Considering the above, my intuition is that Apple’s NeuralHash would have to be completely broken for a company this size to not be able to afford enough moderators to manage the false positive rate. I sincerely doubt they are so inept, and I’d be willing to live with the potentiality of an Apple employee peeking at a photo of mine once a year (me using Apple tech already implies I trust them enough, as I’d never be able to personally verify every privacy claim they make).
What is worth screaming about, and what is eroding my trust, is the fact that since 2019 Apple’s ToS (quietly changed, presumably, to accommodate this feature currently in the news) give them carte blanche for pre-screening any content that they deem potentially illegal. Unless they tighten up that phrasing to limit it to CSAM only, they’re allowing it to be used as a political prosecution tool.
Edit: Factual error, iCloud Photos does have a free tier.
Anyone trusting Facebook with a level of access to their life comparable to a mobile phone is foolish. There is no way I would pay money for Facebook to control my phone.
If it costs them too much money to have too many people personally looking over every single positive hit to make sure it’s not false, then they’re not going to do it even if they technically could.
I live in a place where the government arrests you for having a VPN installed on the phone and labels you a terrorist outright. My phone is checking with physical frisking on the roadside and content critical of state gets automatic manhandling and trip to the police station where I am treated as a criminal.
How I see this as a problem not because I am not going to buy an iPhone in future, but because such ideology would be made normal. That is what is scaring me
The technical implementation is trying to hide the fact that the design mojo of this system is an actual Backdoor for governments of the world to oppress, censure and do whatever they like.
The technical implementation is optimized towards minimizing the cost for Apple. That's why they scan on the device. And this decision is made with knowledge that in the near future "scanning" will be not limited only to hashes. Scanning and processing will be required for $Some_Cool_Functionality to work.
The same slippery slope applies there.
Something I bet wouldn't have happened when Katie Cotton was in charge. But yeah. Tim Cook thought he need new PR direction. And that is what we got. The new Apple PR machine since 2014.
I googled that name and gawker article from 2014 showed up… and I’m speechless…
Industry is learning from omnibus bills
We drank the Apple Privacy Kool-aid, and now we are holding them to it.
This is totally a battle worth fighting!
Are you sure? My local Apple store is just as crowded as it was two weeks ago.
I think the pros list stays longer than the cons list.
I needn't be holding child pornography to be concerned about a third party viewing my photos, writing, or other media on a device that is just mine and not published, public content.
The weirdly less discussed aspect of this is that anyone who is storing their images of any kind on someone else’s computer and network thinks that nothing could have been viewed before. If Apple or Google or Amazon want to scan the data you store with them they could be doing it, so if that was a concern for a person from the get go then they wouldn’t have been storing their data with third parties to begin with.
I honestly don't understand why this is a relevant point. It's still surveillance.
> that anyone who is storing their images of any kind on someone else’s computer and network thinks that nothing could have been viewed before
I don't think that's the confusion. I think a huge part of the issue is that the surveillance is not taking place on someone else's computer, it's taking place on your smartphone. Yes, Apple says it only happens if you're uploading to the cloud -- but that's just Apple saying "trust us". If they did the scanning on their computers instead of yours, it wouldn't be necessary to trust them on this point.
My point is we've already been taking the same risks and the only reason it’s something now is because it’s a transparent process. It’s always a “trust us” scenario unless a person routinely scans all software they is and all updates for malicious server calls or some other kind of recording of data and maybe opening of a back door.
Where you put these will depend on your view on a lot of the issues, certainly.
But, in the past decade:
- Every interaction with your primary device is now, by default, an opportunity for aggressive data collection, often in ways even the people who write the software don't know (because they rely on tons of other libraries and toolkits that are doing this quietly under the hood).
- The default is now that you use a smartphone for everything, with the desktop experience limited or turned into a crappy version of the smartphone version (Image! Video! Scroll, scroll, scroll, never stopping, always seeing more ads! Text, who cares about that ancient stuff?)
- The default has gone from "If you're alone in a social space, you talk to other people" to "You stare at your phone." Certainly was a trend before, with the Walkman/iPod/etc, but it accelerated dramatically.
- Everything has been turned into either a subscription service, or a "Free-to-play" world in which the goal is addiction and microtransactions.
There are plenty of benefits of smartphones, but culturally we're exceedingly bad at looking at the opportunity costs of new technology, and they're increasingly becoming harder to ignore.
If you can honestly evaluate the device and decide it's a net positive, great. But I know an increasing number of people, myself included, who are evaluating them and saying, "You know, never mind. They're not worth the downsides."
I’m starting graduate school in the fall. A few weeks ago, I went in to pick up my new college ID card. The security guard would not let me into the building until I downloaded an app called “Everbridge” on my phone and used it to answer a series of health screening questions (ie, have you tested positive for COVID in the past 14 days).
The app was for iOS and Android. There was no web version. There was no option to fill out a paper form. I was not warned in advanced. But I guess it wasn’t a problem for anyone (including me), because who the heck doesn’t have a smartphone? It’s like having a wallet now—an expected requirement for modern life, even in situations when an analog solution could have worked just as well.
Again, I'm at a point where I can be a thorny pain in the ass about stuff like this, but you carrying a smartphone, even though you (presumably?) know it's evil means that people can do things like this - expect you to download some large blob of unknown code that you're going to run.
As long as they don't encounter people who literally can't comply, it's fine. It works for them.
I mean, I would have refused to download an unknown app I'd never heard of, but... if I pull out a clearly-not-a-smartphone, what are they going to make me do? Go down the street to Best Buy, buy a phone, and come back?
What if your phone was too old to run the app (which looks like a steaming pile, based on reviews)?
Unless there was something in the application documentation about "owning a modern smartphone and being willing to install random applications as required by the university," I would have plopped right down, pulled out a laptop, and started making phone calls to figure it out.
But, again, I'm at a point in my life where I can be a thorny pain in the ass about stuff like this without any real consequences.
For the past week (entirely related to this being a kicker of a motivation on top of a bunch of other simmering long term concerns over Apple and the tech industry in general), I've been carrying around a Nokia 8110 4G - also known, for very understandable and valid reasons, as "The Bananaphone." It's quite literally curved and bright yellow.
The world hasn't ended yet...
It's a bit less of a step for me than other people because I'm already pretty cell-phone hostile. My iPhone (I regret buying a 2020 SE to replace my 6S under the assumption that the 6S wouldn't get iOS 15, which it's getting... maybe...) was pretty well nerfed to start with - very few apps, literally the only apps on my homescreen were person to person or group chat apps (Signal, iMessage, Google Chat, and the Element Matrix client, plus phone, browser, and camera in the bottom). Everything else had to live in the app library thing, which increased friction to use it, and I really didn't have much on there.
But that has been shut down except for a few 10-15 minute windows the past week, and I've been trying, very hard, to work out the transition back to a "dumbphone" (or, as we used to call them, a cellphone).
The main pain point so far is that all my messaging apps used to come to a central point on my phone - so if someone wanted to contact me, it didn't matter what they used, it would ping me if I had my phone on me. Now, that's split (my wife is the main party impacted, I'm pretty high lag on other platforms anyway). If I'm out and about, I can get SMS, but not Matrix/Signal/Chat. If I'm in my office, I can get all of them, but would rather not have a long conversation over T9 - except some of them don't do a great job of notifying me, depending on what machines are running and muted at any given time. Etc. I'm still working this out, and some of it is simple enough - add audio notifications to my "Chat Pi" by wiring in a speaker instead of relying on my phone to chirp if I get a message in Chat or element. That my M1 Mac Mini is going out the door at some point gives me added motivation to solve this.
When out and about, I do at least have the option of tethering to the banana - so I could carry some other device that handles more than the phone does (which seriously isn't much). I'm debating between going back to a small tablet (2nd gen Nexus 7 would be a perfect form factor), or something like a YARH (http://yarh.io) of some variety - a little Pi based mobile computer thing that is exceedingly "We didn't invent smartphones"punk.
I'm at a point in my life (professionally, socially, culturally, etc) where I can happily do "You're weird... whatever..." sort of things with regards to technology, and I'm going to pull the thread until I either figure out alternatives, or determine that they simply don't exist and I can't live without them.
I thought you were going to say it's cellular, modular, interactivodular.
And we're on Hacker News. People know about ROMs and how to use them. Get a Pixel and throw Lineage onto it. It'll be at minimum $100 cheaper and the specs are pretty damn close (minor trades in either direction).
Yes, and I've used it. I'll admit, it is nice. Apple is great about things "just working," but there are also costs to that feature. I'm not sure why people think this is the only way to have said convenience. The major difference is just that Apple this convenience is by default. Honestly Google provides most of this by default too.
> I haven't used others and I am telling you how it looks from my point of view.
And I've used both, and telling you how it is from my point of view. Stop having strong opinions about things that you admittedly don't have experience with. It isn't a good look.
> I will have to invest resources to discover the things you talk about.
You had to invest resources to adapt Apple's ecosystem.
> And there are lots of assumptions in the things you suggested, like that I am fine with using Google or Linux,
Well Google, Linux, Windows, and third party ROMs like Lineage are essentially the only other options out there, so I'm not sure I made many assumptions. And of the most popular, for phones your options are most likely to be Google (i.e. Android) or iOS. The only other option is... third party ROMs. I guess for desktop you could in fact develop your own (besides Linux or Windows), but I think this would fall under a third party custom ROM (if phone) or Linux (if computer). Then again, you could be a BSD person or someone that insists it is GNU/Linux. Let's be real, that's just pedantic, you know what I mean. I'm not going to name every alternative (that's impossible), especially since I covered by far the most popular ones, which share >90% of user share. Why do we have to be so pedantic?
I don't know man, the more you're fighting against this the more it seems like you're just saying "I don't want to try something new." That's fine, but just be open about it. I get it, new things are scary, different, and require you to relearn some things. But that's a different argument. Just state that argument if that's what you're trying to say. Don't make me work for it. Just be honest.
I already own a Pinephone but it's not at a point where I'd want to use it as a daily driver. But they're only $150-$200, so worth taking a chance if you don't want an Android alternative. You may end up liking it. I do know people who are using it daily. It's just not for me. Not yet.
If you want to look into the Android alternatives further, this HN discussion about CalyxOS went into some great detail about that OS, and about other alternatives too.
https://news.ycombinator.com/item?id=28090024
On a side note: I went to Apple site trying to find that page for all those new features and I could not find one (at least by going to obvious places). The way I was able to get it to link in my posts is by googling it… this whole thing is not yet obvious to laypeople.
I guess what I'm saying is, at least for me, this backlash is blurring their entire privacy and security pitch. Apple built a walled garden, told me it was for my own protection, then come to find out the cameras are all pointing to the inside.
(There are many ways this can be a slippery slope, but we don't have to pretend they could just so what ever body else is doing just as easily and they just want to do it on your phone because they are lazy or whatever. This is a solution to a legitimate problem and also it turns out that people are rightfully worried about what's next; those two facts can coexist)
The second issue is that it will alert authorities.
In regards to CSAM content those issues may not sound terrible. But the second it is expanded to texts, things you say, websites you visit or apps you use it's a lot scarier. And what if instead of CSAM content it is extended to alert authorities for _any_ activity deemed undesirable by your government
Just to be clear, "false positive" in this case means an innocent person is accused of trafficking in child sexual abuse material. It's likely they will be raided.
Sure, that's bad if you're Apple, but it's a lot worse if you're the alleged predator.
Which could also be spun as "Apple allows X freebies of known/highly suspected CSAM on your device before they'll tell anybody".
The amount of PR failure that has gone into all this is huge and multi-level.
From now on, when asked to check whether a user's encrypted phone contains arbitrary content the FBI wants to know about, Apple can no longer say "we don't have a way to do that." Sooner or later, you can bet they will start doing it, whether they want to or not.
If this feature leads to anyone losing their job due to incorrect criminal accusations it will not even make the papers because we expect the accused are guilty anyway. Apple won't shed a tear until there is a class action.
This seems worded to get a Yes answer. So, yes.
It's a big deal because it's unprecedented (to my knowledge) outside of the domain of malware*. Other cloud providers run checks of their own property, on their own property. This runs a check of your property, on your property. That's why people care now. The fact that this occurs because of an intention to upload to their server doesn't really change the problem, not unless you're only looking at this like an architectural diagram. Which I fear many people are.
A techie might look at this and see a simple architectural choice. Client-side code instead of server-side. Ok, neat. A more sophisticated techie might see a master plan to pave the way for E2EE. A net-win for privacy. Cool. But the problem doesn't go away. My phone, in my pocket, is now checking itself for evidence of a heinous crime.
*I hope the comparison isn't too extra. I was thinking, the idea of code running on my device, that I don't want to run, that can gather criminal evidence against me, and report it over the internet... yeah I can't get around it, that really reminds me of malware. Not from society's perspective. From society's perspective maybe it's verygoodware. But from the traditional user's perspective, code that runs on your device, that hurts you, is at least vigilante malware, even if you are terrible.
I see your point here - this is a slippery slope for Apple. However I don’t see how anyone could achieve both purposes - no fingerprint reporting and prevention of CSAM storage on Apple servers.
Also, a practical thing to do is to just not store your photos on iCloud but use something else for sync and backup - there might be a startup opportunity here if enough people care.
iTunes Match is an iCloud service which (if you buy it and opt-in) scans your local on-device music library for copyrighted songs, tells Apple you have them, and then they let you listen to high quality versions of those songs on all your devices. And it's not filename or id3 tag matching, it's doing a fuzzy match that can identify the same song in low quality rips and in different file formats. It would be concievable for them to scan for banned audio lectures, or scan your whole device outside the iTunes library, or change it to check for other copyrighted files e.g. movies. It could concievably be reporting you to the MPAA/RIAA if it finds certain songs along with your public IP address so they can check if that IP address has ever been logged as torrenting those songs. It could "in future" be changed to look for and report evidence of torrenting or movie copying. There's nothing technical or regulatory(?) stopping Apple from saying "people with CSAM on their computers can't use iTunes Match" and making it scan the computer as a condition of use, is there? There's nothing technical stopping a government from asking "can your iTunes Match scan engine report video files in the iTunes library which match popular Tiannamen Square video MD5 hashes?", is there?
I do get that these are not the same seriousness, iTunes Match isn't (so far as we know) scanning to report crime but in so far as "Unprecedented on-device scanning for known content using an opaque database and a closed-source fuzzy-matching engine, it would only take a small change to make it look for other things, governments will definitely pressure them to do that and since they willingly built this system they will definitely agree, and all you can do is trust them", are they not samey enough to be relevant?
At least in the US, the historical distinction between verygood searches and mal searches is given in the 4th amendment: "no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized."
Of course that has been twisted and stretched before, and this is more of the same. But if literally everything is being scanned, "probable cause" is completely absent from the process. It is a fishing expedition of the exact type that the 4A was designed to prevent.
Also, no one(well, most people) has any issue with photos being scanned in the iCloud. Photos in Google Photos have been scanned for years and no one cares. The problem is that apple said that photos are encrypted on your device and in the cloud, but now your phone will scan the pictures and if they fail some magical test that you can't inspect, your pictures will be sent unencrypted for verification without telling you. So you think you're sending pictures to secure storage, but nope, actually their algorithm decided that the picture is dodgy in some way so in fact it's sent for viewing by some unknown person. But hey don't worry, you can trust apple, they will definitely only verify it and do nothing else. Because a big American corporation is totally trustworthy.
Because that doesn’t sound correct to me…
Ignoring the nasty aspects of doing that kind of work, I don't see any way to buttress the fact that Apple has taken another step on the universally one way street of ever increasing surveillance. And whataboutism in the form of "oh but other cloud providers are already doing this" is an extremely weak argument because I don't want to use other cloud providers. I liked Apple, because irrespective of their real motives (ie money), they seemed to be privacy focused. The backhanded way they tried to sell this "feature" shows that they are just as bad as the others.
Apple's solution to this problem is that their employee will actually verify the picture before sending it to authorities. Which again, is one of the problems people have with this system.
https://www.hackerfactor.com/blog/index.php?/archives/929-On...
I think what people are getting riled up about is not the technical ability, it’s the lack of restraint, the willingness to search through everyone’s personal stuff on their phones. This is like the cops sending a drug-sniffing dog into everyone’s home once a day, with the excuse that it is privacy-preserving because no human enters the premises, and that only truly bad people will get caught. There is a difference between scanning in the cloud and scanning on device. One is looking through your stuff after you’ve stored it in a storage unit, and the other is looking through your stuff while it is still in your home. Apple’s excuse is that you were going to move it anyway, but somehow that doesn’t actually excuse things.
If I own my data, someone processing this data on my behalf has no right or obligation to scan it for illegal content. The fact that this data sometimes sits on hard drives owned by another party just isn't a relevant factor. Presumably I still own my car when it sits in the garage at the shop. They have no right or obligation to rummage around looking for evidence of a crime. I don't see abstract data as any different.
> (f)Protection of Privacy.—Nothing in this section shall be construed to require a provider to—
(1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b).
https://stratechery.com/2021/apples-mistake/
Which is exactly why these policies are so dim witted.
Dragnet violation of everyone’s privacy while anyone even remotely sophisticated can easily evade it by just encrypting the data upfront.
This has been mentioned on here before, but it's known CSAM possession that's illegal. Apple keeps your files encrypted until its algorithm thinks your encrypted file is too similar to CSAM, and then it decrypts it and sends it to Apple for review. There's a few things here.
- The algorithm is a black box, so nobody knows how many false positives it hits.
- Apple's willingness to decrypt files without the consent of the owner makes the encryption seem like a bit of a sham.
- I imagine many are skeptical of Apple's ability to judge CSAM accurately. If I take a photo of my kids in a bathtub, is that CSAM? What about teenagers in a relationship sharing nudes. The law is a blunt and cruel instrument, and we've gotten away without hurting too many innocent people so far because the process is run by humans, but computers are not known for being gracious.
So we know for sure they're not just using PhotoDNA?
> If I take a photo of my kids in a bathtub, .....
Kinda the same question. If they're using PhotoDNA, then that's not really a risk, right? Isn't this technology well understood at this point?
- There's a system to catch CSAM that is either PhotoDNA or something that works similarly.
- There's a system to detect novel nudes, and notify parents if their children view them.
I think I got these two mixed together.
That's fair. Apple did a shit job of explaining themselves, and it has been compounded by a lot of misinformation (deliberate or not) in response. I'm trying really, really hard to moderate my reaction to this whole mess until I feel like I actually understand what Apple intends to do. I don't make platform jumps lightly.
You could argue that a minecraft server is technically in possession of CSAM if that's the case, but you could spend an infinite amount of money looking at various possible sequences and are bound to find many more false positives than true positives.
Services should have a duty to report CSAM when they notice it, but the lengths they should go to search for CSAM should be limited by cost/benefit and privacy concerns.
This type of scenario is what happened with the messaging service Kik, which was reportedly used to distribute CSAM in private chats. Law enforcement agencies said the company wasn't providing timely responses and that children were being actively abused as a result. This is about as damaging of an accusation you can leverage against a company.
Laws against CSAM worldwide are not going away for good reasons, so there is always going to be a justifiable argument that storing certain classes of data is illegal. Hence, anyone wanting to run a cloud service that stores user data will have to obey by those laws, regardless of how proactive they are in scanning for the material. Absolute privacy in the cloud is impossible to achieve with those rules in place.
Let’s not beat about the bush, if someone wants to store information in a form that can’t be decrypted by Apple, they can. This is a stupid dragnet policy that won’t catch anyone sophisticated.
Apple focused the last years pitching themselves as the tech giant who actually cares about privacy. They seemed to be consciously building this image.
To now implement scanning of private information and then try and sell this obvious 180degree slippery slope turnaround in the most weasel worded “but think of the children” trope is an insult to the customers’ intelligence.
I was a keen Apple consumer because I felt that even if their motivation was profit, this was a company who focused on privacy. It was a distinct selling point.
I certainly won’t be buying more Apple products.
For me, Apple lost the main reason to buy their stuff. If they are going to do the same thing everyone else is doing, I refuse to pay the premium they charge.
I can't imagine how they thought this would go well.
It's another example of Apple being stuck in an echo chamber and not being able to objectively assess how their actions will be perceived.
How many times have they made product and PR blunders like this?
Me (Last month): "Apple is taking privacy very seriously. I'm going to vote with my dollars and switch from Android."
Me (This month): "..."
Apple has spent billions in engineering and marketing to establish themselves as the privacy leader, all wiped away by this idiotic system so full of holes you could serve it on crackers.
People assumed this opens the door for Apple to alerted of any known file uploaded to its iCloud storage.
IOW, they assumed Apple can check what someone is uploading,1 despite alleged "end-to-end encryption" and a gazilion promises of "privacy".
No one except the people managing the "detection software" know what files the hashes represent.
Theres no way for the owner of an Apple computer to verify what files Apple is actually checking for.
Is this confusion. It sounds more like lack of trust.
1 Mind you, for a majority of computer owners the uploading is likely occuring by default, automatically, outside of the owner's awareness. As opposed to the owner consciously deciding to upload a particular file to a computer in an Apple datacenter. Tech cmpanies know that users rarely change defaults.
Remember how Apple had zero accountability:
https://www.newscientist.com/article/dn26133-jennifer-lawren...
https://arstechnica.com/information-technology/2014/09/what-...
Ricky Gervais' advice made sense. Wonder why he deleted it.
And that's the crux of the problem.
I sold every share the day they announced this.
Limiting the scanner to iCloud is a policy decision one NSL away from changing.
The distinction is whether the matching happens on-device before upload or in the cloud after upload, it seems. If Apple already does on-device ML, it makes sense they would add more photo processing client-side to take advantage of encrypted or archival blob storage server-side.
Additionally, there’s still the option of using a third-party camera app, which wouldn’t upload photos by default at all.
...for now.
If you store the magazines in your basement, locked in a vault (password protected computer), then we have an expectation of privacy. To have the vault manufacturer sneak into your home, forcibly open the safe, and inspect your magazines is a very invasive thing. Especially because if this wasn't all over the news, no one would ever think such a thing is happening.
https://www.wsj.com/video/series/joanna-stern-personal-techn...
They also said that because it's on device, security researchers will be able to check any change to the program. (probably via the Apple Security Research Device Program ?)
[1] https://developer.apple.com/programs/security-research-devic...
With new bits of infos too.
Also now that this is a thing how effective will it be at all? Or these sick people that dumb? After all this news? I do hope they are that dumb but who knows.
See, it's comments like this that clearly illustrate that there is confusion, and many people are still outraged over things they don't understand. A neural net is not scanning your phone for CP. You are conflating two things. Just watch the video that you're commenting on before commenting on it.
As for exactly how they'll do the auditing, I'm confused as well.