This article misses the point completely. It was never about what they were looking for, it was that they were looking at all. The quiet part is, once they get acceptance of the snooping using the bogeyman of the day, it will eventually encompass other behavior up to and including political dissent etc.
There is also the issue that closed door arbitration of heinous crimes are the perfect tool to put away dissidents. A very convenient excuse to avoid proper checks and balances. "No, you cannot verify, because you aren't allowed to look at the illegal material that proves their guilt"
Is this actually true? If I was arrested for an image on my phone, would no one in the judge/jury team be allowed to see the image that was flagged? I understand it’s allegedly contraband, but if I were a judge I don’t think I’d take an algorithm’s word for it.
> it’s allegedly contraband, but if I were a judge I don’t think I’d take an algorithm’s word for it.
The prosecution can hire more expert witnesses with excessive credentials than you can, and they will explain how there's a one in one trillion possibility that the system is wrong, and that the defendant is assuredly a monster.
Juries eat that up when it comes to bogus DNA, bite mark, fingerprint, or other forensic evidence claims. Most people think computers can't be wrong or biased, and people's perceptions of what can be deemed reasonable doubts or not seem to shift when computers are involved, or when smart, credentialed people tell them their reasonable doubts aren't reasonable at all because of that one in a trillion chance of the computers being wrong.
I agree. Further, is there any doubt that these “CSAM”² hash databases aren’t already shared between countries? Because if there was doubt, there shouldn’t be, because agencies do share across borders.
² - reason for the scare quotes is because I have first hand knowledge of non-CSAM content being in NCMEC’s database (most likely via data entry errors, but I can’t be for sure).
Ha! Come to think of it, I think that was one example. The main examples that came to memory (it’s been almost 8 years since I was involved) when discussing this last week were essentially extremely common photos (like the stock Windows XP background, among others).
Instead of hashes looking for sexual abuse images, hashes look at signs of white supremacist behavior. Instead of reporting such racism or behavior to the government, it is shared with all the major tech companies, including AirBnB and Paypal.
So, if Facebook deems you to be a political outcast, based on your conversations on their platform, you may find yourself without access to hotels or online payment platforms. The bogeyman of the day is the white supremacist, and coordinated snooping by big tech is the quiet part.
> Over the next few months, the group will add attacker manifestos - often shared by sympathizers after white supremacist violence - and other publications and links flagged by U.N. initiative Tech Against Terrorism. It will use lists from intelligence-sharing group Five Eyes, adding URLs and PDFs from more groups, including the Proud Boys, the Three Percenters and neo-Nazis.
> The firms, which include Twitter (TWTR.N) and Alphabet Inc's (GOOGL.O) YouTube, share "hashes," unique numerical representations of original pieces of content that have been removed from their services. Other platforms use these to identify the same content on their own sites in order to review or remove it.
As a company to become part of the Global Internet Forum to Counter Terrorism, you have to pledge your support to expanding from terrorism towards all online extremism. If you can't accept that, they will still send a speaker to educate your company.
And, just introducing this idea can make the concept real, even if Apple cancels this plan tomorrow. Country X could introduce a law saying “all companies that sell phones here must scan and report for these hashes” and based on recent trends, the companies will just roll over and say Uncle.
Yeah. "Dear debtor countries, the attached file contain hashes of new files deemed illegal for the harmonious well-being of peoples for this month, please enter them into your databases."
A FISA subpoena or an NSL can only compel Apple to give certain types of information that Apple already has. They can’t compel Apple to gather information they don’t already have.
Yep. FBI Lawyer, Kevin Clinesmith gets to lie to the FISA court, fabricate emails to get 2-hop warrants and then gets zero days in prison and as of this month is allowed to practice law again. It's an absolute joke.
A wiretap order is a different kind of order and goes through the district court, not FISA. The legal standard for it is also much higher than for other subpoenas by statute.
This is just business. Governments around the world are thinking of breaking up the app store, and Apple is saying they'll help governments surveil on their citizens inside their walled garden. Would be a shame if users can escape from the surveillance by choosing alternates outside of the walled garden, wouldn't it?
Expect an App Store rule soon - all photos apps must scan for CSAM.
In a year, expect Signal to be banned from all app stores, just like how it's already banned from some countries.
USA-only referred to which accounts would be scanned. Whereas it's theoretically a good thing to require the CSAM hashes to be verified by organisations in multiple countries (rather than a single US government-tied organisation that dismissed privacy concerns as "the screeching voices of the minority").
I'm hoping for something along the lines of "iOS will reject hashes that haven't been signed by independent organisations in US, Russia, China, and India" to make it very difficult to push through anything except actual CSAM. Won't be much of a guarantee if it's just Apple saying "we promise we're only using hashes that have been checked by Australia and the US".
My impression was that it's currently US only because that's the only country so far where they've arranged a process to report their users to the cops. No reason they wouldn't expand it as they make arrangements with other countries.
They've really bought into a ton of complexity at all levels by doing this instead of just scanning stuff on their own servers, which would at least have been a clear dividing line. Privacy is really all about control, and on-device scanning dangerously blurs that line.
It should be obvious by now that Apple has been coerced into building this scanning system, and the CSAM scanning is just the front for a much more broad "national security" backdoor. Apple's PR team has surely been shitting bricks for the entire week--this is the worst PR debacle they've had since the iPhone 4 Antennagate in 2010.
Completely agree. A friend of mine asked what was happening, and I just said "it's a big deal, biggest thing in a long time". Antennagate is a good comparison -- though this seems bigger. Impacts of Antennagate long term seemed minimal.
Sadly this will likely be little to no impact on Apple long term either. I too agree this is coming from the government. They have found their loophole of using "private industry" to do all the dirty stuff they cannot directly do.
Given that privacy has been what Apple was using as they key selling point in their ad campaign, and now they've completely subverted that and mainstream media is picking it up and people are noticing, I suspect this will impact them far more than they expected.
While Apple deserves all the fallout they're getting and more, I'm disappointed that the NCMEC that pushed for this isn't also receiving more scrutiny and criticism. Multiple people have now pointed out that their database contains false positives, which is absolutely terrifying. Completely legal, harmless, no-nudity-no-humans pictures can get your life ruined. The truth coming out later doesn't matter when your home gets raided and it slips out that you were caught sharing multiple images that matched hashes from the NCMEC child porn database.
This is way bigger than any RF engineering mishap.
I’m going to actively evangelize alternatives to Apple devices and that’s coming from someone who has been doing Apple evangelizing since OSX Panther days. Tens of thousands of dollars of bought devices and services not including all the people I convinced to make the switch over the years.
Potentially something like a national security letter and they can’t reveal the real reason behind it. I agree this feature is off-brand for Apple and came out of the blue.
This sounds like a conspiracy theory unless you provide more explanations with some evidence. For example, who are these handlers? What are their motives?
The simplest explanation is usually the correct one. The simplest explanation would be some director in Apple wants to get promoted to VP so he or she started this project, pitched it, and very few people opposed it because who wouldn't to prevent harm of children?
They're convinced that it's worth the benefit to abused children to let the tech community vent fears about the world at them for a while. I don't really blame them. My views don't align with the mob here, so it would be a waste of my time to discuss them while y'all are upset and angry about this; best just to hold my position, remain silent or reserved, and wait for the pitchfork mob to find another target or exhaust itself. I'm honestly surprised Apple even commented at all.
> I'm honestly surprised Apple even commented at all.
I think they commented because they expected a brief storm of outrage that quickly dies down, but instead it has become a growing wave that is spilling into mainstream media, with a mostly negative reaction, and threatening to completely destroy their "Privacy. That's iPhone" mantra that has been the core of their marketing campaign this year.
Because this is likely not about making money, it's about expanding surveillance. Once this becomes accepted they can ratchet it up bit-by-bit. Each the the outrage will be less and less.
For example imagine creating a new tax for 1% of purchases. People would be outraged. Now imagine raising sales tax instead by 1% (ex 7% to 8%.) Sure some people would be upset, but it would be a smaller number than the "new" tax. The reason is because people are already used to paying a sales tax. What is "normal" is more easily accepted.
If Apple can get past the initial outrage (like when sales tax was implemented in many countries in history) they can increase the surveillance once scanning local files becomes the "new normal".
I love how people will jump through hoops to land on China even when discussing something that was conceptualized and first (only, as of now) implemented in the US.
You can't have a massive new fake war - the War on Domestic Terorrism - without ideally having new ways to track and surveil anyone (everyone) that is going against the machine politically. They need new capabilities and they're absolutely going to put them into place, asap. Did you see their reaction to the populist Trump circus? The globalists viewed it as a mortal threat. Everything they built since WW2 was in jeopardy as far as they were concerned. Their ability to launch more forever wars, endless foreign adventurism, obsessive (psychotic) superpower global meddling, all of it under threat from rising populism (represented by both Trump and Sanders) and a growing rejection of the entirely failed decades-long approach by the globalists that has led to the US being buried under nearly $30 trillion in debt and numerous catastrophic wars. Just look at what these monsters are saying publicly on MSNBC, CNN, et al. They're telling everybody what they're planning to do, they're putting it right out there. Look at their framing: we're going to target the domestic population; they're de facto saying that. They couldn't be more open about the authoritarian nightmare they have in mind. They perceive their system as being under risk from populist revolt against them and their agenda, and they're launching a counter offensive (the empire is striking back), it's under way right now. Bet on it.
It's quite simple: the culture of bureaucrats that have dominated Washington DC and its policies in the post WW2 era, they acquired a position of power the likes of which has rarely been seen in all of human history. Now, ask yourself: do you really think they're ready to give that power up? They're ready to stop treating the rest of the planet like it's their toy, to do with as they please. Then ask yourself: what do you think they're prepared to do to stave off risks to that power? The risk that they might lose their precious. And there you go, you have your answer. They're willing to do many of the same things other authoritarians have been willing to do. The US isn't Soviet Russia, that's not the suggestion; it doesn't need to be to start putting political opponents (fringe rabble rousers; whether libertarian or socialist - see: Clinton vs Sanders; that's globalist establishment vs domestic socialist) into prison and surveiling everyone in a desperate attempt to retain their power. That kind of power is an intense drug, they're addicts of the worst kind, you can tell by how they behave in regards to the rest of the world, how they treat the rest of the world.
To ensure they keep their power, to keep their globalist forever war machine (and everything that goes with it), they need to put the boot on rising domestic risks to that power.
They put themselves into a position where backing out is also extremely costly.
a) it means they have to publicly admit that what they did was a bad idea, which will fuel another news cycle
b) backing out of it will get them criticized for "protecting pedophiles", maybe even the NCMEC (who privately praised them in the "screeching voices of the minority" memo) will now publicly criticize and shame them to get what it wants
c) now that they've put this idea on the table, there will be even more government pressure to mandate/implement it. Even if they back out of the implementation, just by bringing this proposal up, Apple may have just destroyed not just the privacy of their users, but of everyone.
The thing that strikes me as most suspicious is their insistence on keeping the client-side scanning feature while wrapping it in all sorts of supposed restrictions so it's essentially server-side scanning, just implemented on the client device for some reason.
If the problem is "We need to scan stuff on iCloud for CSAM" then "Let's build a client-side scanning agent and distribute it to hundreds of millions of devices and figure out a way to protect the hashes from the end user and then figure out a way to lock it down so it can't be used to scan other things even if we wanted to or somebody ordered us to" is a singularly (and suspiciously) inelegant solution to the problem.
Just scan the files on your own servers, like everybody else does. Anything else is rightly going to make people suspicious that you've got some ulterior motive.
Notice how they haven't done that though. It would be trivial to do. Just change all of the private keys to ones supplied by the clients. But they haven't done that. Why would you expect them to do that when they are taking steps away from individual privacy? Leaving a hypothetical option on the table that costs them nothing to do means they actively do not want to do it. So why would they do it?
It’s quite simple, they would like to store photos without being able to see them, only knowing they’re not CP. If they scan on server, they need to be able to see all photos, and eventually give them to any requesting authority.
So the reasonable approach is to make clients that only upload photos that are not CP. The problem is that the implementation of that feels creepier than they anticipated.
So they're still retaining the ability to look on users devices? I have nothing against them looking at the abuse material, I have a problem with them having power over my device, which could be piggy-backed on by state actors (or others) for their own purposes later.
Nobody is against them trying to prevent child sexual abuse, pretty sure we all agree that fighting that is important, but doing so by creating what is essentially a back door of sorts into my devices isn't the right approach to doing so.
It sounds to me that this is still allowing them that access, so this changes absolutely nothing.
How is it day 47 of HN arguing about this and you all still can’t get basic facts right, and oh boy, you showed me downvoting me twice in 60 seconds for pointing out a lack of fundamental comprehension of what you’re actually upset about
This is as unhelpful as saying "don't have CSAM on your phone." As has been reiterated many times, the problem is not the capability it is claimed to start off with, it is how the capability will be evolved, abused, and exploited as time goes on.
If they only intended to scan iCloud photos, then they would’ve done it on the server (since they have encryption keys) instead of installing spyware on everybody’s iPhone. If they had done that then there would’ve been little to no outrage since people wouldn’t need to worry as they could just turn off iCloud.
What do you think will happen when e.g. the Chinese government demand that they scan all photos for hashes provided by the government, upload them regardless of whether iCloud is enabled, and let a Chinese company handle the reviews? If they couldn’t stop the CCP from gaining full control of iCloud in China then they most definitely can’t prevent this either.. and other authoritarian governments will copy the CCP and make similar demands.
Makes zero sense. Installing spyware and compromising everybody’s iPhone does a million times more harm than whatever governments can achieve with complete access to people’s iCloud account. At least before we could protect our privacy by turning off all iCloud sync, but now your only option is to switch away from iOS.
Yeah, all they need to do is follow suit with what everybody else is doing and scan once images are uploaded onto iCloud.
Scan every single file. I don't care. Because once in iCloud, files are sitting on Apple's server and hard drives. I don't have much expectation that those files are 100% private.
And... Apple misses the point of the criticism completely.
This problem is the capability, not what it's used for. Any such capability will be abused by new use cases be it terrorism, drug trafficking, human trafficking or whatever. Plus there will inevitably be unauthorized access.
The only way to prevent all this is for the system not to exist.
I don't buy into theories that Apple is being pressured or coerced on any of this. I believe it's far more likely this is just tone-deaf but well-intentioned incompetence. It's classic "why won't anyone think of the children?" and we've seen it time and time again with encryption backdoors and similar.
The big question for me is how and why Tim Cook and Apple's board signed off on a plan to do huge damage to Apple's reputation and user trust. If they didn't know, it's a problem. If they knew and didn't realize the inevitable backlash, well that's a different problem.
> This problem is the capability, not what it's used for. Any such capability will be abused by new use cases be it terrorism, drug trafficking, human trafficking or whatever. Plus there will inevitably be unauthorized access.
Apple even says it themselves[1]:
> This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time.
I can't wait until /child-safety 404s or redirects to /safety and there's a wall of marketing blurb (possibly only in Chinese at first) that explains how 'national security' concerns are reported to the CCP.
This has totally pushed me over the edge, though I'll admit I was oblivious to begin with. My plan is to replace the MacBooks with a Thinkpad P15 gen 2 running Ubuntu and replace the iPhone with something running Ubuntu Touch (Volla Phone, Fairphone, OnePlus One). Screw not having control.
As another commentator stated, if they actually cared about this issue, they would have done what all the other cloud providers have been doing for many years, and what everyone opposed to this wants them to do - just scan the cloud. That's all they're effectively doing right now, only with the scanning happening on a person's private device.
I also note that /child-safety page is only(?) accessible from outside search/link. There is no corresponding press article. This thing is just floating in the air somehow…
Same here. Just that I'll get rid of MS and Google. That means CalyxOS on my Pixel 2 and some version of Linux on my private ThinkPad. Once I find time that is, which to be honest can take a while. I think the last time I used my private laptop was over a month ago. Phone is different, but again time constraints. It will happen so.
I'm in the same situation in terms of tech debt against my personal infrastructure. I have plans to migrate my personal (currently Ubuntu) server to use Proxmox on top of selling the current machines & buying new non-Apple ones for this. The evening and weekend time is tight.
There is another option - don’t put anything sensitive on your phone and basically treat it as an adversary already has root access. This pretty much what I do for a long time. All the sensitive information (e.g. banking) is on Linux desktop and there are no logins or apps on iPhone or work laptop (hn is not sensitive ;) ).
I started a thread on what specific actions we can take to avoid Apple and other big tech in future. While it did not gain much traction, I am hoping there are at least 1,000 people in the world who think like me and over time we will be able to influence more people.
Even if you trust Apple and NCMEC to not add out of scope hashes that some government or law enforcement or intelligence agency “asks” them to, does anybody who’s ever worked for Apple have any doubt at all they they’d use this to check employee’s personal devices for Apple IP? Especially if a big spectacular leak hits the media? Apple’s IP enforcement goons are legendary. And not in a good way in most people’s opinion. Particularly Gizmodos…
Moreover, things could become retroactively bad. If tomorrow multiple countries decide to change the minimum age of adult performers from 18 to 19 a bunch of pics legal today could become illegal tomorrow leading to wide dragnets enabling law enforcement to go through everything you have.
Unfortunately, I don't think this has been anything more than a small blip to Apple's reputation. If I were not on Hackernews (like 99+% of Apple users), I would have either not heard or given second thought to this move.
Bad press on tech sites is the worst thing for a tech company's reputation. They're the people other people go to when deciding what to buy.
When Uncle Bob asks the family computer engineer about the pros and cons of a given platform and hears that one of them scans your device and a false positive could get you arrested for child pornography, Uncle Bob may develop an aversion to that platform.
The only explanation that makes sense to me is that they are doing huge damage to their reputation and trust for bigger reasons. Perhaps there are international or national security issues they are told to comply with that are not quite in the public view yet.
That would arouse suspicion as soon as someone (perhaps someone inside the company) figured out what was going on. Here, a key person introduces the idea, says it's "for the children," and people are baffled but the masses will forget about it, even come to accept the logic, soon enough.
> It would be a perfectly logical order from the power structures Snowden exposed.
Snowden exposed that Apple was actively part[1] of the PRISM data collection and surveillance program targeting Americans.
Apple also voluntarily gives up customer's data when requested by the US government for about 120,000 individuals a year[2]. They also hand over the data for over 31,000 users/accounts in response to FISA and NSL requests[2] in a six month period.
> eta: It makes way more sense to me than Apple actually thinking this was a good idea.
Based on the company's own messaging, they simply seem proud of the project and find it ambitious[3]. They're also excited about the project's expansion and evolution[3].
They do seem proud, but one of these things makes less sense than the others, given Apple's prior reasoning.
We don't know what goes on in the secret courts these days but we have been assured watchdogs are in place and certain programs are no longer in use [1]. This is all speculation based on past behavior, but I assume new reasoning has been constructed that passes watchdog's interpretation of the law but requires new hoops for three-letter agencies to jump through to get the mountains of data they so yearn for.
maybe they want to stop giving up customer data and doing this on the phone is the only way to do e2ee backups - can't hand over what you can't decrypt.
Seems like a reach to me. The government doesn't just care about CSAM. They care about terrorism, human and drug trafficking, gangs, organized crime, fraud, etc.
That might make sense if CSAM detection is just the start, and they plan on detecting all those other things, as well.
They got what they wanted. It did sting a bit, but it was calculated.
They won't suffer any massive losses over that, they have the moral high ground among non tech people, who mistakenly believe that they're doing something to protect children.
> The only way to prevent all this is for the system not to exist.
I've been having trouble following this argument over the last week. Isn't it clear that the capability already exists? Whether or not Apple goes through with its CSAM plan, the capability is evidently there.
In other words, since Apple is a closed system, the capability was there before they announced the CSAM plans. Their announcement has changed nothing about capability other than reminding people that Apple has privileged access to the data on iPhones.
I guess the question is, if Apple does the CSAM program does that make them more likely to cave to government pressure to search for other things? And to do so without telling users?
There shouldn’t be a single line of code, or single binary blob, on my device that can compute these photo DNA hashes.
Apple could always go ahead and add that functionality in an update, but then there would be a big backlash and the opportunity to not update or switch providers.
Wouldn't users have the ability to not update or switch providers if Apple expands the scope of its search beyond CSAM?
I guess I don't see the huge difference between the surveillance code existing on the phone but not used for objectionable purposes versus the line of code sitting in a different branch in git and not deployed on actual phones (yet).
I'm completely against this move by the way -- I'm not trying to defend it. But I want to be able to argue effectively against it.
No, because they’re not scanning for CSAM, they’re scanning for hashes. That hash database needs to be updateable for it to be useful at all. A significant part of the policy protections are implemented off device, so those can’t be trusted.
This is like renting a house and the owner saying "we installed a camera in your house, but we'll only look at the video, if someone screams 'help'"
Then someone says "what if they change it so it sends the video if someone says 'do you want some pot?'"
And your argument would be "wouldn't you just move if they did that?" - this argument is pointless, because it's not the "help" part that bothersome, it's the always recording camera in a black box, for which you never know what's recording and what's it sending to apple. Let's say hypothetically they do this, and modify the database to trigger on the "tank man" photo, and give that photo such weight, that it triggers manual review by the CCP observer (remember, apple data for chinese users is hosted in china) - you'll never know the image got flagged, you'll never even know that it was sent anywhere, but your social score will go down and you might "vanish" during the night. Imagine some new wikileaks happening... 15 journalists around the world get the data, NSA/CIA/FBI has hacked one of them and found some photos of USA doing something bad.... add those photos to the database, have apple scan all the phones, you find the other 14 journalists, and possible even the leaker who took the original photo. If this is expended to macbooks (which it probably will), even worse, because people keep more sensitive stuff on ther computers than their phones.
And all this for what? Child molesters and rapists (the worst of the pedos) create new photos and videos, that are in none of the databases, so you never catch them. Their direct sponsors (paying customers) get the photos first, and again, before they're in the database. So all you've caught is someone browsing 4chan.
> And all this for what? Child molesters and rapists (the worst of the pedos) create new photos and videos, that are in none of the databases, so you never catch them. Their direct sponsors (paying customers) get the photos first, and again, before they're in the database.
I hadn’t considered that. This would seem to create a market incentive to abuse more children and create a greater variety of new abusive content. Has that been considered at all?
> Apple could always go ahead and add that functionality in an update, but then there would be a big backlash and the opportunity to not update or switch providers.
There is a qualitative difference between tweaking an existing code base (expanding the scope of this detector) and coercing a company to dedicate teams of tens to hundreds of engineers over years to create a brand new code base from scratch (writing this detector in the first place.)
It's a very ill-considered argument, because Apple obviously already has the capability of pushing literally any conceivable software update to iPhones. Apple (and thus iPhones) already have the capability to do anything you can conceive of, regardless of how terrible. For instance, iPhones have the capability of activating the cameras at all times, detecting when they capture compromising or embarrassing video, and uploading those videos to a public website along with your name and contact information. The only thing preventing Apple from doing that is their own policies.
If you think it's bad that Apple can push software updates to iPhones, then by all means make that argument. But "the only way to prevent bad changes to this system down the road is for this first version of the system to never exist" is a very poor argument, given that the only "system" that needs to exist is the ability to push software updates, and that system existed long ago.
That isn't the argument at all. What Apple can't have happen without major reputational damage is get caught doing it without telling its customers.
Thus this is seen as a marketing backdoor to the actual backdoor. This is seen as a ploy that later gives Apple the plausible deniability argument that the government is making them do it (other forms of content) even though they created the means (on device scanning) for the government to exploit.
Why limit it to content that will be uploaded to iCloud? Why limit iMessage scanning to kids? Why not also let the government know? Why limit it to child porn since the system works for any image? Why limit it to 30 hits?
Imagine the reputational damage to Apple if say in 2 years it was discovered that they had been scanning all photos on your device and that the matches had been going to the government and that they didn't tell you. Their privacy marketing angle would be completely toast. Imagine if they did this to phones outside the US.
Any sane marketing person would try to find a way to sugarcoat it first.
> But "the only way to prevent bad changes to this system down the road is for this first version of the system to never exist" is a very poor argument, given that the only "system" that needs to exist is the ability to push software updates, and that system existed long ago.
Apple relied on that argument 5 years ago successfully.[1]
Once a certain piece of technology exists, courts can order that it be used a certain way other than the way it was intended.
Neither courts nor legislative bodies are capable of explicitly requiring the invention of technology that doesn't yet exist, although in some cases they could make it an implicit requirement of a law.
At least in the US most lawmakers are varying shades of technologically illiterate and don't have a grasp of what is possible or not possible. If technology doesn't exist yet, you can say to them "this is impossible", and with enough of a bribe, they will believe it.
Apple by building this technology, has put it on the map and announced that this technology both possible and is ready.
I just have to wonder who green-lit this idea to begin with. Every single other company is riding a post-covid tidal wave of sales. as WFH becomes a reality for so many workers, apple products are positioned to really take center stage for a massive segment of consumers...
and then this. Apple intentionally injects uncertainty and controversy? what were they thinking?? sure its just phones right now but im sure mac users are wondering about workstations and laptops? the damage control being spun right now is absolutely overwhelming.
im also surprised to see no other players like MS or Google rushing to take advantage of the outrage. even players like Purism seem to be ignoring the event.
How much of an impact do you think this will have on the non tech savvy user though? Most people already don't care about ad tracking, location tracking, etc. I think your standard user will just shrug this off and go "eh, I have nothing to hide and it's good for the kids"
I don’t like that I have to resort to this, but I was going to mention Proles and point to Wikipedia, except that Wikipedia deleted article about that topic. Yet I can see article’s blurb in search engine’s cache! Everything is a meta nowadays…
Sooner or later the copyright enforcement companies will find a judge whose willing to rule that since Apple has the capability they have to use it even though this system is aimed at still images only, no sound or video. Then Apple will end up fighting it for years if they're lucky and it doesn't get a quick trip up to Supreme Court that's drastically Hollywood-friendly. The legal difference between mandating that a company create a brand new scanning capability and "just apply" one they already have is significant.
Apple doesn't have that capabilty though. There are countless ways to get copyrighted material on your phone that are perfectly legal. Apple would have to be able to decide if you're legally owning your files. With how the IP system works currently that's impossible. For CSAM this isn't a problem because we assume that there is no legitimate way to have CSAM files on your phone.
Or take a photo of CSAM on display on another device or printout using your iPhone camera, which is often accessible from the lock screen without a PIN.
>>I just have to wonder who green-lit this idea to begin with.
I suspect, with no inside knowledge, that there's someone high up in Apple who feels very, very passionately about child abuse/these sorts of images due to some personal connection, and they are championing the idea internally. And tbh not the worst bugbear to have, but obviously this isn't the way to address it.
As for why the big players aren't jumping on this - it's a tricky line to walk, you don't ever want to be seen as pro child abuse.
Having read the coverage and the documents that Apple has been putting out to manage the … what, crisis? Backlash? … I’ve been thinking about the design of the system and how I would want CSAM to be detected, if different than this.
Honestly, I’m pretty impressed by what this approach accomplishes. Your comment says this obviously isn’t the way to do it— what’s better than this?
In a legal context where detecting CSAM is a strong requirement, what’s preferable to this approach?
>>>In a legal context where detecting CSAM is a strong requirement, what’s preferable to this approach?
It's not a strong requirement though? Only if the company sees it (unencrypted) do they need to report it, detecting it on-device is wholly different. It would honestly be better if they adjusted their TOS and started scanning the files on upload (on server side). They have the encryption keys already.
Apple is planning to install on all (recent) iPhones software that will let them scan for any image similar (using perceptual hashing) to some set of images. Right now, that's CSAM, and only on upload to iCloud. But what do you want to bet China is salivating at the idea of including images of Tank Man and other "seditious" content? And maybe checking images anytime they're added to the phone, or sent to someone else, not just up to iCloud?
It completely ruins the idea that Apple has your privacy/security in mind.
Even if scanning for the content is not explicitly required by law, what happens when a pedophile ring hoarding thousands of images of CSAM on iCloud is busted, and the news gets out? The article at [1] makes it sound like Apple choosing not to scan any of its users' videos in iCloud was seen as evidence of Apple lagging behind the status quo of companies like Facebook that were proactively reporting CSAM.
According to that article, in the last year Apple only submitted 265 reports to the NCEMC while Facebook submitted 20 million. Would law enforcement believe they'd be missing out on catching abusers after seeing this disparity?
If a company is found to allow criminals to store CSAM on their servers for extended periods of time, the law is going to want to know why they let it pass, irrespective of the extent the company chose to scan for it. Apple probably doesn't want to deal with that fallout, so maybe they figured that being proactive about scanning for CSAM in a way that could enable the use of E2EE wouldn't hurt, and that pushing the privacy narrative would satisfy enough people - which it didn't.
>>>If a company is found to allow criminals to store CSAM on their servers
I'd bet dollars to doughnuts that AWS, GCP, Azure, et al. have terabytes of CSAM stored within their data centers - because users have uploaded encrypted files and the companies (rightly) don't have the keys. I'll also bet there are many WD hard drives full of CSAM, many Linux servers hosting it, many nginx or apache installs serving it up, etc. Should we mandate that all hard drives scan files as they're written to see if it's CSAM? Or maybe nginx should alert law enforcement anytime a CSAM image is served.
Apple should have actually let their users have E2EE or given up on that and just scanned stuff server-side.
> The only way to prevent all this is for the system not to exist.
I'll go one step further. Apple should have implemented a system that makes these kind of backdoors impossible. I don't know how or if such a system is possible, but given Apple's track record, they are in a position to attempt it.
I wonder if "we won't use it for anything else" is really any less credible than other vendors who simply aren't talking about any scanning they may or may not be doing.
We know TV makers scan what you watch, even from other inputs, for ad targeting. This sort of capability is already widespread and basically a commodity - who else is using it that hasn't told us? And can we do anything about it short of only using hardware and services where trusted parties have verified the source?
> The big question for me is how and why Tim Cook and Apple's board signed off on a plan to do huge damage to Apple's reputation and user trust. If they didn't know, it's a problem. If they knew and didn't realize the inevitable backlash, well that's a different problem.
Comments like this make me wonder how in touch HN is with the average person because I genuinely don't believe most people care or even know that this is happening. I may be wrong but Apple could've pretended this backlash doesn't exist and did nothing about it and they would lose nothing for it.
I agree regarding the average person but I’ve forwarded the EFF article to friends who are surgeons, electricians, firefighters, etc. Who wouldn’t know what HN is and the reaction has been a universal “wtf Apple is scanning all my photos?”. I guess the question is how big the angry minority is.
I think a lot of people blindly trust Apple as a reputable corporate entity and this definitely shakes that trust.
Perhaps the result of internal stakeholders trying to hold on to any vestige of a massive project they've put a lot of effort into. I doubt they want to see it scrapped, even after admitting to themselves it was a huge error.
Sooner, rather than later, I think adding this to all devices will be mandated, by law. Apple can intend to do only good things with this, but governments can bend and twist things very easily so that a beneficial idea becomes dystopian. Regardless of intention, Apple still has to answer to government, not the other way around.
> I don't buy into theories that Apple is being pressured or coerced on any of this.
Of course they were.
It’s well known that Apple chose not to introduce e2e encrypted iCloud backups back i 2017/18 or so, due to FBI complaints.[1]
This is clearly Apple’s play to be able to introduce that again and tell law enforcement “Look, we ‘thought of the children’, if you want further access to our customers data, you’re going to need to come up with a better justification than that.”
If Apple pull that off, adopting client side image scanning with this quite impressive privacy preserving system behind it, and then e2e encrypt everything they upload to iCloud, that’d arguably be a very big win for Apple customers privacy.
Whether that’s an acceptable trade off for having a device I purchased run code I didn’t ask for and don’t want to monitor where or not I’m a paedophile, possibly snitching on me for false positives or bad-faith additions of non CSAM hashes into the database, or not is a good question still.
Their promise is completely childishly unrealistic, the person who said it should feel ashamed. Does anybody believe they will refuse to use this in <country> if the government demands when it’s a few button clicks of work to do it? When market access is at risk, and with apple’s track record? Or that two countries can’t collaborate to save face while doing this? This promise is so unrealistic that it’s just a lie
This is the time to make a donation to a foss project related to the Linux desktop. Off the top of my head, there is Debian, Ubuntu, mint, gnome, plasma, the fsf, wine, or other projects that could use contributions of code or cash. It adds up and could help one of them hire an extra full timer if enough people set up recurring donations
I wouldn't say "this doesn't matter", I'd say "this makes it worse" because they're reaffirming that they want to push forward with this despite the criticism.
Of course it matters- if Apple does a 180º reversal I'll applaud and buy more Apple products. I want companies to be able to acknowledge mistakes unequivocally and fix them.
If Apple says "we heard the backlash, we're sorry, we'll never do it again", I'll be more of a supporter than I had been before.
This does not address the issue of perceptual hash collisions and false positives at all, nor does it address the issue of on device scanning nor their claim that, according to Apple[1]:
> This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time.
People don't want their property spying on them and reporting them to the police. They don't want people looking at their photos or thumbnails. It's patronizing, invasive and embarrassing to have your privacy violated like that.
They know they can essentially wait out the public outcry, plus they can even buffer things a little bit by calling their critics "confused" and introduce measures that don't address anything but sound reasonable to lay persons in an effort to distort the conversation.
The platform is not in danger. But if it were, then yes, I think they'd still risk it. It's either that or getting shut out of markets gradually by law enforcement and governments.
Desktop Linux in no way compares to iOS market share, and even if it did, it'd require a different type of attack. Apple is vulnerable precisely because it is the one single vendor in complete control of anything related to the iPhone.
Personally I do think it's likely we'll see a successful anti-Linux smear campaign and/or laws in the next few years, but I have to admit that's a pretty pessimistic take.
These stories have been top of HN for more than a week, indicating fascination. Yet non-technical mainstream articles make the bulk of it, and Apple's just released detailed (and convincing) threat report PDF is getting buried. An unfortunate recipe that'll solidify misunderstandings and keep the conversation much more emotional than it should be.
They can publish the dev design doc, the test plan, even the fricking source code, that doesn't change the basic fact which is local device scanning.
There is no misunderstanding of that fact by anyone with sufficient technical know-how. There is also no misunderstanding of what comes next once this Pandora box is opened.
We could probably increase the vaccination rate if we snooped on peoples' phones to find out if they are participating in vaccine-hesitant behavior, and then rate-limited traffic to websites and apps that peddle this dangerous misinformation.
How many people have to lose their lives in the service of some pedantic idea of "privacy"? It's a computer looking at it, it's not even a human person.
I think tech companies need a hippocratic oath similar to "first do no harm". Apple should not be engaging in misinformation trafficking, and should at the very least be working to minimize harm by preventing people form falling victim to dangerous, unsubstantiated, and un-fact-checked information. This is especially important when our elected officials use the considerable power that has been gifted to them by the people to put peoples' lives in danger by spreading dangerous misinformation.
What role did apple's inaction on this have in the pandemic? In the January 6th insurrectionist's attempt to overturn a validated, secure, and duly certified democratic election? What role did Apple's inaction play in the attempted kidnapping of the governor of Michigan?
By refusing to help, they are partially responsible for these things. It's time for us to demand that they do their fair share of helping. Inaction is itself an action.
> Building a version of iOS that bypasses security in this way would undeniably create a backdoor. And while the government may argue that its use would be limited to this case, there is no way to guarantee such control.
> For many years, we have used encryption to protect our customers’ personal data because we believe it’s the only way to keep their information safe. We have even put that data out of our own reach, because we believe the contents of your iPhone are none of our business.
Apple these days is the type of company to make a mistake, think they are never wrong, blame anyone but themselves and then change a little bit, only for it to go over like a dead balloon. It reminds me of the Apple Silicon dev kit. No refunds, ok refunds but you have to spend them in a certain time, ok fine you can use them for a longer period and for anything.
They are probably completely flabbergasted that people are upset about this so they make this change after a week, completely missing the point.
They've been like that since the early days of the company, it's core to their culture, which was built by Steve Jobs. Apple has always had an elitist snobbery at their center, such that if you don't get a thing then you're wrong. That's what Think Different is all about, that's a hyper arrogant proclamation that could only be spewed out of an elitist never-wrong orifice, it's borderline assholic in nature. Part of the cult of Apple has always been that attitude, the Apple fans eat it up, they love the idea that buying Apple products made them feel different (or it used to, before Apple became mainstream), made them feel separate from the masses. Steve Jobs points that out in the D5 interview in 2007 with Gates, how Apple needs the inferior taste of the masses to prime their superiority off of (which is another way of saying: they're wrong, we're right).
Go back to their response over the iPhone 4 reception problems ("you're holding it wrong"), same asshole culture then as now, and that culture has been in place for decades:
Eh, I would go even deeper: the original Macintosh keyboard didn't have arrow keys because you shouldn't need it according to z Jobs. People complained and Apple held their feet for a long time until giving up and adding the goddamn arrow keys. That's always been very emblematic of Apple for me.
(Hard to find a good citation link on a phone, bit the story's out there.)
I mean I don’t consider myself an elitist but as a software engineer who just moved from doing dev work on a Mac to a Windows laptop, I’m so frustrated by how bad the experience is compared to the Mac. I love the new job but even these $2000 Dell laptops feel junky in comparison to a similarly priced MacBook Pro. I’m constantly having problems with my audio devices (which is a necessity during the pandemic) the built in microphone is terrible, the webcam on the laptop is on the BOTTOM of the screen which just makes me look super fat because reasons I guess. It’s like how a fish doesn’t realize it’s in water until it flops up on the shore.
I have a 4" iPhone SE. Love it, but Apple has continued to push me away these last four years. I no longer use any other Apple products except for this 4-year-old phone. What alternatives are there for small phone fans? Sony used to make an Xperia Compact that was nice, but I think they discontinued it.
254 comments
[ 3.2 ms ] story [ 293 ms ] threadThe prosecution can hire more expert witnesses with excessive credentials than you can, and they will explain how there's a one in one trillion possibility that the system is wrong, and that the defendant is assuredly a monster.
Juries eat that up when it comes to bogus DNA, bite mark, fingerprint, or other forensic evidence claims. Most people think computers can't be wrong or biased, and people's perceptions of what can be deemed reasonable doubts or not seem to shift when computers are involved, or when smart, credentialed people tell them their reasonable doubts aren't reasonable at all because of that one in a trillion chance of the computers being wrong.
² - reason for the scare quotes is because I have first hand knowledge of non-CSAM content being in NCMEC’s database (most likely via data entry errors, but I can’t be for sure).
Ha! Come to think of it, I think that was one example. The main examples that came to memory (it’s been almost 8 years since I was involved) when discussing this last week were essentially extremely common photos (like the stock Windows XP background, among others).
Instead of hashes looking for sexual abuse images, hashes look at signs of white supremacist behavior. Instead of reporting such racism or behavior to the government, it is shared with all the major tech companies, including AirBnB and Paypal.
So, if Facebook deems you to be a political outcast, based on your conversations on their platform, you may find yourself without access to hotels or online payment platforms. The bogeyman of the day is the white supremacist, and coordinated snooping by big tech is the quiet part.
> Over the next few months, the group will add attacker manifestos - often shared by sympathizers after white supremacist violence - and other publications and links flagged by U.N. initiative Tech Against Terrorism. It will use lists from intelligence-sharing group Five Eyes, adding URLs and PDFs from more groups, including the Proud Boys, the Three Percenters and neo-Nazis.
> The firms, which include Twitter (TWTR.N) and Alphabet Inc's (GOOGL.O) YouTube, share "hashes," unique numerical representations of original pieces of content that have been removed from their services. Other platforms use these to identify the same content on their own sites in order to review or remove it.
As a company to become part of the Global Internet Forum to Counter Terrorism, you have to pledge your support to expanding from terrorism towards all online extremism. If you can't accept that, they will still send a speaker to educate your company.
https://en.m.wikipedia.org/wiki/Foreign_Intelligence_Surveil...
[1] https://en.wikipedia.org/wiki/PRISM_(surveillance_program)#/...
This sounds like one of those errors where they double down on it and call it a "solution".
Sounds like they're misunderstanding people's concerns pretty badly. Sorry if we miscommunicated. Kill this "feature" yesterday. Thanks Tim!
Expect an App Store rule soon - all photos apps must scan for CSAM.
In a year, expect Signal to be banned from all app stores, just like how it's already banned from some countries.
I'm hoping for something along the lines of "iOS will reject hashes that haven't been signed by independent organisations in US, Russia, China, and India" to make it very difficult to push through anything except actual CSAM. Won't be much of a guarantee if it's just Apple saying "we promise we're only using hashes that have been checked by Australia and the US".
They've really bought into a ton of complexity at all levels by doing this instead of just scanning stuff on their own servers, which would at least have been a clear dividing line. Privacy is really all about control, and on-device scanning dangerously blurs that line.
While Apple deserves all the fallout they're getting and more, I'm disappointed that the NCMEC that pushed for this isn't also receiving more scrutiny and criticism. Multiple people have now pointed out that their database contains false positives, which is absolutely terrifying. Completely legal, harmless, no-nudity-no-humans pictures can get your life ruined. The truth coming out later doesn't matter when your home gets raided and it slips out that you were caught sharing multiple images that matched hashes from the NCMEC child porn database.
I’m going to actively evangelize alternatives to Apple devices and that’s coming from someone who has been doing Apple evangelizing since OSX Panther days. Tens of thousands of dollars of bought devices and services not including all the people I convinced to make the switch over the years.
I’m sure I’m not the only one in this regard.
Lets push that idea so allies in the government can go looking and void it
The simplest explanation is usually the correct one. The simplest explanation would be some director in Apple wants to get promoted to VP so he or she started this project, pitched it, and very few people opposed it because who wouldn't to prevent harm of children?
I think they commented because they expected a brief storm of outrage that quickly dies down, but instead it has become a growing wave that is spilling into mainstream media, with a mostly negative reaction, and threatening to completely destroy their "Privacy. That's iPhone" mantra that has been the core of their marketing campaign this year.
For example imagine creating a new tax for 1% of purchases. People would be outraged. Now imagine raising sales tax instead by 1% (ex 7% to 8%.) Sure some people would be upset, but it would be a smaller number than the "new" tax. The reason is because people are already used to paying a sales tax. What is "normal" is more easily accepted.
If Apple can get past the initial outrage (like when sales tax was implemented in many countries in history) they can increase the surveillance once scanning local files becomes the "new normal".
It's quite simple: the culture of bureaucrats that have dominated Washington DC and its policies in the post WW2 era, they acquired a position of power the likes of which has rarely been seen in all of human history. Now, ask yourself: do you really think they're ready to give that power up? They're ready to stop treating the rest of the planet like it's their toy, to do with as they please. Then ask yourself: what do you think they're prepared to do to stave off risks to that power? The risk that they might lose their precious. And there you go, you have your answer. They're willing to do many of the same things other authoritarians have been willing to do. The US isn't Soviet Russia, that's not the suggestion; it doesn't need to be to start putting political opponents (fringe rabble rousers; whether libertarian or socialist - see: Clinton vs Sanders; that's globalist establishment vs domestic socialist) into prison and surveiling everyone in a desperate attempt to retain their power. That kind of power is an intense drug, they're addicts of the worst kind, you can tell by how they behave in regards to the rest of the world, how they treat the rest of the world.
To ensure they keep their power, to keep their globalist forever war machine (and everything that goes with it), they need to put the boot on rising domestic risks to that power.
a) it means they have to publicly admit that what they did was a bad idea, which will fuel another news cycle
b) backing out of it will get them criticized for "protecting pedophiles", maybe even the NCMEC (who privately praised them in the "screeching voices of the minority" memo) will now publicly criticize and shame them to get what it wants
c) now that they've put this idea on the table, there will be even more government pressure to mandate/implement it. Even if they back out of the implementation, just by bringing this proposal up, Apple may have just destroyed not just the privacy of their users, but of everyone.
If the problem is "We need to scan stuff on iCloud for CSAM" then "Let's build a client-side scanning agent and distribute it to hundreds of millions of devices and figure out a way to protect the hashes from the end user and then figure out a way to lock it down so it can't be used to scan other things even if we wanted to or somebody ordered us to" is a singularly (and suspiciously) inelegant solution to the problem.
Just scan the files on your own servers, like everybody else does. Anything else is rightly going to make people suspicious that you've got some ulterior motive.
Right now, they have to hand over anyone’s photo library if they are ordered to do so.
If they turn on E2E that will no longer be possible, and this will be strictly better and harder to abuse.
That doesn’t mean this mechanism isn’t offensive. It just is better from a privacy point of view for them to do this and enable E2E.
So the reasonable approach is to make clients that only upload photos that are not CP. The problem is that the implementation of that feels creepier than they anticipated.
Nobody is against them trying to prevent child sexual abuse, pretty sure we all agree that fighting that is important, but doing so by creating what is essentially a back door of sorts into my devices isn't the right approach to doing so.
It sounds to me that this is still allowing them that access, so this changes absolutely nothing.
How is it day 47 of HN arguing about this and you all still can’t get basic facts right, and oh boy, you showed me downvoting me twice in 60 seconds for pointing out a lack of fundamental comprehension of what you’re actually upset about
This community is obnoxious
What do you think will happen when e.g. the Chinese government demand that they scan all photos for hashes provided by the government, upload them regardless of whether iCloud is enabled, and let a Chinese company handle the reviews? If they couldn’t stop the CCP from gaining full control of iCloud in China then they most definitely can’t prevent this either.. and other authoritarian governments will copy the CCP and make similar demands.
Scan every single file. I don't care. Because once in iCloud, files are sitting on Apple's server and hard drives. I don't have much expectation that those files are 100% private.
They're completely missing the point.
This problem is the capability, not what it's used for. Any such capability will be abused by new use cases be it terrorism, drug trafficking, human trafficking or whatever. Plus there will inevitably be unauthorized access.
The only way to prevent all this is for the system not to exist.
I don't buy into theories that Apple is being pressured or coerced on any of this. I believe it's far more likely this is just tone-deaf but well-intentioned incompetence. It's classic "why won't anyone think of the children?" and we've seen it time and time again with encryption backdoors and similar.
The big question for me is how and why Tim Cook and Apple's board signed off on a plan to do huge damage to Apple's reputation and user trust. If they didn't know, it's a problem. If they knew and didn't realize the inevitable backlash, well that's a different problem.
Apple even says it themselves[1]:
> This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time.
[1] https://www.apple.com/child-safety/
This has totally pushed me over the edge, though I'll admit I was oblivious to begin with. My plan is to replace the MacBooks with a Thinkpad P15 gen 2 running Ubuntu and replace the iPhone with something running Ubuntu Touch (Volla Phone, Fairphone, OnePlus One). Screw not having control.
Or, perhaps they feel really strongly about child exploitation?
In literally 30s looking at your comments it shows that you're an HR and an amateur pilot.
Seeing the number of comments, with slightly more time, your place of residence and political leaning would probably become apparent.
Why am I paying 100s of $$$ to own a device that is adversarial?
https://news.ycombinator.com/item?id=28157281
we all know this is gonna be dmca 2.0
And you know who that gun is aimed at first?
Apple employees.
Even if you trust Apple and NCMEC to not add out of scope hashes that some government or law enforcement or intelligence agency “asks” them to, does anybody who’s ever worked for Apple have any doubt at all they they’d use this to check employee’s personal devices for Apple IP? Especially if a big spectacular leak hits the media? Apple’s IP enforcement goons are legendary. And not in a good way in most people’s opinion. Particularly Gizmodos…
Kind of surprising how tone deaf Apple's response has been despite this.
When Uncle Bob asks the family computer engineer about the pros and cons of a given platform and hears that one of them scans your device and a false positive could get you arrested for child pornography, Uncle Bob may develop an aversion to that platform.
eta: It makes way more sense to me than Apple actually thinking this was a good idea.
Snowden exposed that Apple was actively part[1] of the PRISM data collection and surveillance program targeting Americans.
Apple also voluntarily gives up customer's data when requested by the US government for about 120,000 individuals a year[2]. They also hand over the data for over 31,000 users/accounts in response to FISA and NSL requests[2] in a six month period.
> eta: It makes way more sense to me than Apple actually thinking this was a good idea.
Based on the company's own messaging, they simply seem proud of the project and find it ambitious[3]. They're also excited about the project's expansion and evolution[3].
[1] https://en.wikipedia.org/wiki/PRISM_(surveillance_program)#/...
[2] https://www.apple.com/legal/transparency/us.html
[3] https://www.apple.com/child-safety/
We don't know what goes on in the secret courts these days but we have been assured watchdogs are in place and certain programs are no longer in use [1]. This is all speculation based on past behavior, but I assume new reasoning has been constructed that passes watchdog's interpretation of the law but requires new hoops for three-letter agencies to jump through to get the mountains of data they so yearn for.
[1] https://reason.com/2015/11/29/the-nsa-will-stop-collecting-y...
Now it kind of sounds like I'm playing 5D chess, but since we don't know what takes place under secret Executive Orders...
That might make sense if CSAM detection is just the start, and they plan on detecting all those other things, as well.
They got what they wanted. It did sting a bit, but it was calculated.
They won't suffer any massive losses over that, they have the moral high ground among non tech people, who mistakenly believe that they're doing something to protect children.
> The only way to prevent all this is for the system not to exist.
I've been having trouble following this argument over the last week. Isn't it clear that the capability already exists? Whether or not Apple goes through with its CSAM plan, the capability is evidently there.
In other words, since Apple is a closed system, the capability was there before they announced the CSAM plans. Their announcement has changed nothing about capability other than reminding people that Apple has privileged access to the data on iPhones.
I guess the question is, if Apple does the CSAM program does that make them more likely to cave to government pressure to search for other things? And to do so without telling users?
Apple could always go ahead and add that functionality in an update, but then there would be a big backlash and the opportunity to not update or switch providers.
I guess I don't see the huge difference between the surveillance code existing on the phone but not used for objectionable purposes versus the line of code sitting in a different branch in git and not deployed on actual phones (yet).
I'm completely against this move by the way -- I'm not trying to defend it. But I want to be able to argue effectively against it.
Then someone says "what if they change it so it sends the video if someone says 'do you want some pot?'"
And your argument would be "wouldn't you just move if they did that?" - this argument is pointless, because it's not the "help" part that bothersome, it's the always recording camera in a black box, for which you never know what's recording and what's it sending to apple. Let's say hypothetically they do this, and modify the database to trigger on the "tank man" photo, and give that photo such weight, that it triggers manual review by the CCP observer (remember, apple data for chinese users is hosted in china) - you'll never know the image got flagged, you'll never even know that it was sent anywhere, but your social score will go down and you might "vanish" during the night. Imagine some new wikileaks happening... 15 journalists around the world get the data, NSA/CIA/FBI has hacked one of them and found some photos of USA doing something bad.... add those photos to the database, have apple scan all the phones, you find the other 14 journalists, and possible even the leaker who took the original photo. If this is expended to macbooks (which it probably will), even worse, because people keep more sensitive stuff on ther computers than their phones.
And all this for what? Child molesters and rapists (the worst of the pedos) create new photos and videos, that are in none of the databases, so you never catch them. Their direct sponsors (paying customers) get the photos first, and again, before they're in the database. So all you've caught is someone browsing 4chan.
I hadn’t considered that. This would seem to create a market incentive to abuse more children and create a greater variety of new abusive content. Has that been considered at all?
Isn't that exactly what is happening now?
If you think it's bad that Apple can push software updates to iPhones, then by all means make that argument. But "the only way to prevent bad changes to this system down the road is for this first version of the system to never exist" is a very poor argument, given that the only "system" that needs to exist is the ability to push software updates, and that system existed long ago.
Thus this is seen as a marketing backdoor to the actual backdoor. This is seen as a ploy that later gives Apple the plausible deniability argument that the government is making them do it (other forms of content) even though they created the means (on device scanning) for the government to exploit.
Why limit it to content that will be uploaded to iCloud? Why limit iMessage scanning to kids? Why not also let the government know? Why limit it to child porn since the system works for any image? Why limit it to 30 hits?
Any sane marketing person would try to find a way to sugarcoat it first.
Apple relied on that argument 5 years ago successfully.[1]
[1] https://en.wikipedia.org/wiki/FBI–Apple_encryption_dispute
Neither courts nor legislative bodies are capable of explicitly requiring the invention of technology that doesn't yet exist, although in some cases they could make it an implicit requirement of a law.
At least in the US most lawmakers are varying shades of technologically illiterate and don't have a grasp of what is possible or not possible. If technology doesn't exist yet, you can say to them "this is impossible", and with enough of a bribe, they will believe it.
Apple by building this technology, has put it on the map and announced that this technology both possible and is ready.
and then this. Apple intentionally injects uncertainty and controversy? what were they thinking?? sure its just phones right now but im sure mac users are wondering about workstations and laptops? the damage control being spun right now is absolutely overwhelming.
im also surprised to see no other players like MS or Google rushing to take advantage of the outrage. even players like Purism seem to be ignoring the event.
[1] https://en.m.wikipedia.org/wiki/Proles_(Nineteen_Eighty-Four...
WhatsApp has "Save pictures to camera roll" on by default. Someone can send you a bunch of CSAM via WhatsApp and now you are in trouble.
Purism addressed it yesterday[1]. Here's the HN thread[2].
[1] https://puri.sm/posts/internet-of-snitches/
[2] https://news.ycombinator.com/item?id=28158939
I suspect, with no inside knowledge, that there's someone high up in Apple who feels very, very passionately about child abuse/these sorts of images due to some personal connection, and they are championing the idea internally. And tbh not the worst bugbear to have, but obviously this isn't the way to address it.
As for why the big players aren't jumping on this - it's a tricky line to walk, you don't ever want to be seen as pro child abuse.
Honestly, I’m pretty impressed by what this approach accomplishes. Your comment says this obviously isn’t the way to do it— what’s better than this?
In a legal context where detecting CSAM is a strong requirement, what’s preferable to this approach?
It's not a strong requirement though? Only if the company sees it (unencrypted) do they need to report it, detecting it on-device is wholly different. It would honestly be better if they adjusted their TOS and started scanning the files on upload (on server side). They have the encryption keys already.
Apple is planning to install on all (recent) iPhones software that will let them scan for any image similar (using perceptual hashing) to some set of images. Right now, that's CSAM, and only on upload to iCloud. But what do you want to bet China is salivating at the idea of including images of Tank Man and other "seditious" content? And maybe checking images anytime they're added to the phone, or sent to someone else, not just up to iCloud?
It completely ruins the idea that Apple has your privacy/security in mind.
According to that article, in the last year Apple only submitted 265 reports to the NCEMC while Facebook submitted 20 million. Would law enforcement believe they'd be missing out on catching abusers after seeing this disparity?
If a company is found to allow criminals to store CSAM on their servers for extended periods of time, the law is going to want to know why they let it pass, irrespective of the extent the company chose to scan for it. Apple probably doesn't want to deal with that fallout, so maybe they figured that being proactive about scanning for CSAM in a way that could enable the use of E2EE wouldn't hurt, and that pushing the privacy narrative would satisfy enough people - which it didn't.
[1] https://www.nytimes.com/2021/08/05/technology/apple-iphones-...
I'd bet dollars to doughnuts that AWS, GCP, Azure, et al. have terabytes of CSAM stored within their data centers - because users have uploaded encrypted files and the companies (rightly) don't have the keys. I'll also bet there are many WD hard drives full of CSAM, many Linux servers hosting it, many nginx or apache installs serving it up, etc. Should we mandate that all hard drives scan files as they're written to see if it's CSAM? Or maybe nginx should alert law enforcement anytime a CSAM image is served.
Apple should have actually let their users have E2EE or given up on that and just scanned stuff server-side.
I'll go one step further. Apple should have implemented a system that makes these kind of backdoors impossible. I don't know how or if such a system is possible, but given Apple's track record, they are in a position to attempt it.
Signed updates (which they already have) that are not designed to spy on their customers and bog-standard E2E would achieve that.
But then they couldn't even scan the stuff on icloud.
"Do us a favor or the AG launches an investigation into your business practices."
We know TV makers scan what you watch, even from other inputs, for ad targeting. This sort of capability is already widespread and basically a commodity - who else is using it that hasn't told us? And can we do anything about it short of only using hardware and services where trusted parties have verified the source?
Comments like this make me wonder how in touch HN is with the average person because I genuinely don't believe most people care or even know that this is happening. I may be wrong but Apple could've pretended this backlash doesn't exist and did nothing about it and they would lose nothing for it.
I think a lot of people blindly trust Apple as a reputable corporate entity and this definitely shakes that trust.
Of course they were.
It’s well known that Apple chose not to introduce e2e encrypted iCloud backups back i 2017/18 or so, due to FBI complaints.[1]
This is clearly Apple’s play to be able to introduce that again and tell law enforcement “Look, we ‘thought of the children’, if you want further access to our customers data, you’re going to need to come up with a better justification than that.”
If Apple pull that off, adopting client side image scanning with this quite impressive privacy preserving system behind it, and then e2e encrypt everything they upload to iCloud, that’d arguably be a very big win for Apple customers privacy.
Whether that’s an acceptable trade off for having a device I purchased run code I didn’t ask for and don’t want to monitor where or not I’m a paedophile, possibly snitching on me for false positives or bad-faith additions of non CSAM hashes into the database, or not is a good question still.
1: https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...
"Make it so we are the only ones on that list, or we ban iphones in China".
This is the time to make a donation to a foss project related to the Linux desktop. Off the top of my head, there is Debian, Ubuntu, mint, gnome, plasma, the fsf, wine, or other projects that could use contributions of code or cash. It adds up and could help one of them hire an extra full timer if enough people set up recurring donations
Their point is that they don't care about that other point.
If Apple says "we heard the backlash, we're sorry, we'll never do it again", I'll be more of a supporter than I had been before.
Until they do, trust lost.
Soon your pictures will be subject to a 3-week review period to decide whether they 're fit to be scanned
> This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time.
People don't want their property spying on them and reporting them to the police. They don't want people looking at their photos or thumbnails. It's patronizing, invasive and embarrassing to have your privacy violated like that.
[1] https://www.apple.com/child-safety/
Security Threat Model Review of the Apple Child Safety Features [pdf] (apple.com) https://news.ycombinator.com/item?id=28173134
The platform is not in danger. But if it were, then yes, I think they'd still risk it. It's either that or getting shut out of markets gradually by law enforcement and governments.
How come that's not happening to desktop Linux then?
Personally I do think it's likely we'll see a successful anti-Linux smear campaign and/or laws in the next few years, but I have to admit that's a pretty pessimistic take.
Security Threat Model Review of the Apple Child Safety Features [pdf] (apple.com) https://news.ycombinator.com/item?id=28173134
The key issue here is the very concept of "Apple turns your iPhone into a snitch", and I haven't seen any misunderstanding around that.
There is no misunderstanding of that fact by anyone with sufficient technical know-how. There is also no misunderstanding of what comes next once this Pandora box is opened.
How many people have to lose their lives in the service of some pedantic idea of "privacy"? It's a computer looking at it, it's not even a human person.
I think tech companies need a hippocratic oath similar to "first do no harm". Apple should not be engaging in misinformation trafficking, and should at the very least be working to minimize harm by preventing people form falling victim to dangerous, unsubstantiated, and un-fact-checked information. This is especially important when our elected officials use the considerable power that has been gifted to them by the people to put peoples' lives in danger by spreading dangerous misinformation.
What role did apple's inaction on this have in the pandemic? In the January 6th insurrectionist's attempt to overturn a validated, secure, and duly certified democratic election? What role did Apple's inaction play in the attempted kidnapping of the governor of Michigan?
By refusing to help, they are partially responsible for these things. It's time for us to demand that they do their fair share of helping. Inaction is itself an action.
--
This is sarcasm, of course. For how long?
> Building a version of iOS that bypasses security in this way would undeniably create a backdoor. And while the government may argue that its use would be limited to this case, there is no way to guarantee such control.
How ironic of you, Apple
They are probably completely flabbergasted that people are upset about this so they make this change after a week, completely missing the point.
Go back to their response over the iPhone 4 reception problems ("you're holding it wrong"), same asshole culture then as now, and that culture has been in place for decades:
https://www.engadget.com/2010-06-24-apple-responds-over-ipho...
(Hard to find a good citation link on a phone, bit the story's out there.)