Apple did this in such a myopic, I want to say almost comically stupid manner. Craig's interview in and of itself was the biggest indictment of technology companies one could ever hope for. Dude literally stumbles as he says "multi-step algorithm" as if the world is being graced by his bountiful excellence.
Can't believe this dude is running software at Apple.
The headline for all of this should be "We know what it does and we don't want it"
And that interview is likely supposed to telegraph that they “don’t care”. It wasn’t a live one, was it? They could have improved parts of it if they wanted.
At some point, the back story behind this will leak out. That will be interesting. This has to be something Apple was asked/pressured/ordered to do. As a business activity it makes no sense.
Bear in mind that over the last week, people in Afghanistan have been frantically trying to erase any evidence of doing things the Taliban doesn't like, such as women playing sports.
I’m inclined to believe it was a quid-pro-quo (or a play at it). Apple inserts a little back door, in exchange the antitrust case against them will be eased.
Based on the letter by NCMEC describing opponents to the technology as “screeching voices of the minority”, I suspect this is some kind of variant of being guilted into doing something by either lobbying, marketing or internal proponents. This resulted in a tone deaf death march and now they’re trying to work out how to spin it.
This is a fairly typical corporate failure mode.
Sometimes the best thing is to do nothing as they have found out because now they’re stuck in a position where the two exits are either piss off everyone by writing off their privacy stance or piss off everyone else for canning child porn scanning.
I'm convinced none of these conspiracy theories hold up. This is a typical Apple thing. Thought long and hard about something in a mostly isolated setting and then missing some obvious things.
You might be right. On the other hand, all their past controversial ideas were reasonable from the financial point of view: introduce non-standard connectors, remove the minijack, make the hardware non-upgradeable and so on. All these things were bad for consumers, but good for Apple's bottom line. This one is bad for everybody. Apple's management is not stupid, they must have a good reason.
> This has to be something Apple was asked/pressured/ordered to do. As a business activity it makes no sense.
I firmly believe we'll never discover who applied that pressure, or how. And it's hard for me to imagine the US government doing that without some court battle with serious risk of leaks to journalists. I mean, it makes no financial sense, so Apple would fight that. The only thing I can imagine is pressure from a private interest whose market is big enough for Apple to care. And if it is that market, then I think we're in for a tough ride until we can build our own devices.
I think there's some strong hints in apple's implementation that they were under external pressure.
If you read the documentation on how it's implemented, there's some fairly advanced crypto - private set intersection, threshold secret sharing - that only makes sense if Apple took the line "we have to do this, but we're willing to do it in a really expensive way so that we ourselves have as little access as possible". They went to the effort of running the NeuralHash on the client device, as far as I understand.
The standard implementation on other cloud providers is that the provider has access to your data on the server if they really need it - much cheaper, and makes it technically possible in future to easily change the T&C to "we may use this for market research and to improve our products". I view Apple's client-side implementation as drawing a very big line in the sand saying we will not do that and we are willing to put our money where our mouth is by writing this crypto protocol.
But that certainly raises the question, why would Apple do anything like that at all, unless there's some (current or anticipated) external threat that forced them to?
As to whether we'll ever discover it - I wouldn't have predicted the Snowden revelations, maybe this will come out the next time something like that happens. If it does, I also predict we'll be shocked to discover just how big the market and distribution network for child abuse online is, and how many people are involved.
> I also predict we'll be shocked to discover just how big the market and distribution network for child abuse online is, and how many people are involved.
I don't think it is big compared to what authoritarian states would like to use this for. You're talking about, at the very least, putting everyone in non-democratic states under even more pressure to obey than they're already under. And in democratic states this invites bad actors both in and outside the government. There is no net-gain in justice here. It's wrong across the board.
Welcome back to HN after taking 5 months off, btw. You're a SSC fan?
Yes, I agree the state market is even bigger - China's potential market for surveillance alone is 1.3 billion and rising - but I've heard from colleagues in tech who seriously think child abuse is only a very small number of bad actors. My understanding is the "industry" is almost as out of control as the illegal drugs trade.
I am just going to say....That little blurb about SSC/ACX kinda caught my eye. I had/have never heard of this persons blog.
And now I am down a rabbit hole. Probably for a while. But thanks for that. Seems like the world needs more thinkers like this person, and everyone, including myself could learn a lot from them.
Telling an engineer about SSC for the first time could almost be considered a denial-of-service attack, or at least nerd-sniping :) Glad you're enjoying it!
I don't think any amount of child abuse can be considered "small" on its own given the lifetime impact it has. However, it is small compared to putting eyes inside the whole world's devices. Context is everything with this kind of comment. Without context it will sound uninformed and heartless.
> As to whether we'll ever discover it - I wouldn't have predicted the Snowden revelations
Snowden paid a very high price not many people can afford, especially when they have a family. So I highly doubt we could know it within a reasonable timeframe.
> And it's hard for me to imagine the US government doing that without some court battle with serious risk of leaks to journalists.
Anything going through a FISA court (Foreign Intelligence Surveillance Court) could come with a gag order, preventing anyone involved to talk about it.
While that is true, the security concerns are much wider than child abuse scanning. From the article:
> "Though these capabilities are intended to protect children and to reduce the spread of child sexual abuse material, we are concerned that they will be used to censor protected speech, threaten the privacy and security of people around the world, and have disastrous consequences for many children,"
Many people and organizations fear that Apple's plans are paving the way for governments to scan for more than just child abuse images.
(e.g. https://appleprivacyletter.com/)
In that light, FISA is relevant. My statement was to show that not all dealings of the US government are public, because not all courts and legal proceedings are available to the public. So "serious risk of leaks to journalists" won't likely happen in that case.
The scanning matches against CSAM hashes provided by NCMEC - the fear is that NCMEC will be pressured into including hashes of non-CSAM material that USGOV want to track as well (say MAGA images for the current administration.)
(One counterpoint to this is that PhotoDNA has been used for scanning cloud images using NCMEC-provided hashes for a decade now and this slippery slope doesn't seem to have happened yet.)
this is a very weak argument. The MAGA crowd has been posting images all over social media.
My understanding is that there is a threshold that needs to be met, and then an Apple employee with a very difficult job will need to confirm the image. So even if NCMEC get pressured, I don't see what the result can be.
That Apple employee (assuming the role doesn't get contracted out) has no incentive to protect you or me. But they'll absolutely err on the side of caution to protect their own livelihood.
There is also international perspective: once this mechanism is in place, forced by US, it will become normalized in several years. Then China will have much better leverage to implement their own scans. Once China does it, every powerful state does it. And after that every third world hole will scan for everything.
Because Apple does not currently have code for scanning local devices for illegal images, and would have to implement it, which is a much higher bar than having already implemented it and just needing to add a few hashes to a database.
This part is legit though: governments around the world could pressure Apple to add other forms of surveillance. Be it hashes of non-CSAM, or simply pressure NEMEC or other hash providers, or extending its capabilities to all messages or photos on device.
In my opinion, this is the biggest concern, not the technology. Before, Apple could simply refuse by saying we don't have the capabilities. But now, that excuse is gone. Apple's promise to human review content and only report CSAM is the weakest link.
If you read their Security Threat Model Review [1] they're only using the "intersection of hashes provided by at least two child safety organizations operating in separate sovereign jurisdictions".
So you'd have to pressure NEMEC and another org under a different government to both add the non-CSAM hash, plus Apple would need to be pressured to verify a non-CSAM derivative image, plus you'd need other hash matches on-device to exceed the threshold before they could even do the review in the first place (they can't even tell if there was a match unless the threshold is exceeded).
I get why people are concerned, but between this thread and the other thread yesterday it's clear that pretty much everyone discussing this has no idea how it works.
I think you're missing what concerns people; what guarantee do we have that Apple will, always and forever, only use the "intersection of hashes provided by at least two child safety organizations operating in separate sovereign jurisdictions"?
I've actually read your comment, along with the linked PDF, in full multiple times in an effort to make sure I wasn't missing a point you were trying to make. Before I continue, I want to make it clear that I fully understand that adding non-CSAM content into the existing NCMEC/CSAM/IWF environment would more than likely raise a lot of red flags. You and I have an understanding there.
As it happens, I still think you're missing the point that myself and others are trying to make. Perhaps the following is a better way of phrasing my inquiry:
What guarantees to we have that Apple will, always and forever, only use the NCMEC/CSAM/IWF system to scan images on phones? By that I mean, isn't it perfectly plausible that Apple can leave the NCMEC/CSAM/IWF system in place, as it is described in your post and the linked PDF, and at the same time partner with [random body X/Y/Z], who has their own database of [whatever] that they scan separately from the NCMEC/CSAM/IWF system? These two different scans wouldn't ever need to communicate with each other, and could run at the same time.
Well yeah it's plausible they would just turn the contents of everyone's entire iCloud backups over to the FBI, which they can totally do right now at any time since they already have the keys!
So what? What you're describing is not what this system does. You could argue against literally any possibility anyone could dream up, because they're the platform vendor and they can make any change they want. If that's a problem when you're evaluating your threat model then this new system is the least of your worries.
And as far as I can see this system is the least intrusive version of what any of the other cloud vendors are doing. Don't want it? Turn off iCloud Photos and sync to something else like your own Nextcloud instance.
>What you're describing is not what this system does.
I said that the hypothetical system could be set up, and function, in the same way as the NCMEC/CSAM/IWF system, just with different entities behind it. Despite chastising me for what you thought was a failure to comprehend your post, this is the third time you seem to have been unable to grasp that concept.
That's part of what people are concerned about - an identical system running in tandem to this, but with less savory characters behind it. Given how many comments on HN have addressed that this week, including all of mine right here, I am at a loss for how to make that any clearer to you.
>Sure but you are ignoring that such a scenario would need a different system to be deployed.
Right! I clearly stated multiple times that this would be a second system working in tandem, of course it would need to be deployed on its own. I am entirely unsure how you think I am ignoring that when it's the premise of my argument and of the concern of others.
Forgive me - I typically try to adhere to the ideal that posts on HN should be constructive and refrain from snark and sarcasm, but this particular comment chain has become so circular and repetitive that the only response I can think of at this point is, “No shit!”.
Edit: Removing some paragraphs, because if that's not perfectly clear to you by reading this entire comment thread, even when I basically said as much in the post you just responded to, then I really don't know what to tell you.
Right, but the point is that you are talking about this fictional system as though it is causally connected to what Apple is doing, when it really isn’t.
You being able to imagine a fictional evil parallel system isn’t a valid criticism of what Apple is actually doing.
>You being able to imagine a fictional evil parallel system isn’t a valid criticism of what Apple is actually doing.
Right! And where did I criticize them? All I did was ask, “What guarantee do we have that Apple will, always and forever, only scan for data that comes from NCMEC/CSAM/IWF?” and then gave an example of what such a scenario might look like when another user responded to me.
I expressed a concern (I even used that word, and not "critique" in my last post that you responded to) that this type of scanning could be expanded in the future, but I have not criticized them broadly nor have I criticized this specific program. How has that not been clear?
If you think I’ve done otherwise, I’d appreciate it if you could quote what you interpret as a critique, so that I may work to avoid such confusion in future conversations similar to this.
The implication of expressing a concern is that it is related to what is being done.
Otherwise why post it here? I doubt you’d claim that your ‘concern’ is irrelevant to this topic.
It’s an innuendo intended to imply a likelyhood of future wrongdoing.
It works like this: ‘What guarantee do we have that he won’t hit his wife after we leave?’
Obviously this is a way of suggesting that the man will hit his wife, phrased as a question to facilitate the same kind of denial that you are using here.
>The implication of expressing a concern is that it is related to what is being done.
… duh? I never said it wasn’t related, I said that it’s not a criticism of the current program.
>Otherwise why post it here? I doubt you’d claim that your ‘concern’ is irrelevant to this topic.
Correct!
>It’s an innuendo intended to imply a likelyhood of future wrongdoing.
No, it’s intended to state clearly - not imply - a concern around whether or not Apple will be pressured by other bodies to begin scanning on-device content for hashes pertaining to subjects beyond child abuse.
The question is simple - what guarantee do we have that Apple won’t expand what kind of content they scan for? What is so difficult to grasp about that, and why do you, it seems, feel that that is an entirely irrelevant question that isn’t worth discussing, answering or being concerned about?
>It works like this: ‘What guarantee do we have that he won’t hit his wife after we leave?’
>Obviously this is a way of suggesting that the man will hit his wife, phrased as a question to facilitate the same kind of denial that you are using here.
You leave and hope that he doesn’t, but would you never follow up with your friends to see how they’re doing after a domestic altercation, to make sure it hasn’t happened again and that they’re safe from harm? What is so wrong about being aware of possibilities and remaining vigilant?
Nobody said the man had done anything wrong, and yet you are assuming he did simply because someone asked a question and recommending that people act on this false impression.
Your scenario around leaving and whether or not the man will hit his wife was flawed from the start. Given the context of this broader conversation - Apple implementing a new photo-scanning system - one assumes that with the way you left the beginning of your scenario open to interpretation/assumption, that you are leaving after he's already hit his wife (in this case, Apple implementing the CSAM scanning) and hoping he doesn't do it again (Apple scanning for different content in the future). You would do well to better set up your hypothetical scenario next time, perhaps clarifying that you haven't actually witnessed any violence firsthand prior to your departure.
But again, since you continue to avoid what I'm asking:
>The question is simple - what guarantee do we have that Apple won’t expand what kind of content they scan for? What is so difficult to grasp about that, and why do you, it seems, feel that that is an entirely irrelevant question that isn’t worth discussing, answering or being concerned about?
and perhaps more curiously
>What is so wrong about being aware of possibilities and remaining vigilant?
> Your scenario around leaving and whether or not the man will hit his wife was flawed from the start
My scenario never said anything about ‘leaving’. You continue to confirm the point. You read that in to the scenario.
That’s the point. You completely failed to understand the scenario as written and made up your own story to suit your prejudices. The only violence was in your imagination.
What guarantee do we have that you aren’t doing the same thing with Apple?
>It works like this: ‘What guarantee do we have that he won’t hit his wife after we leave?’
At this point, after continuously misinterpreting my posts (almost intentionally, as though you're arguing in bad faith), you've effectively moved on to gaslighting, so I'm just going to continue to reiterate the same questions I have been posting, which you continue to avoid answering. Carry on as you have been if you want, but this is about all you'll get from me moving forward until you answer them directly.
- Apple has decided to scan for a specific kind of content on their phones, what guarantee do we have that they won't scan for other content in the future?
- For someone who commented, "stand up for civil liberties now", why are you so opposed to people being aware of possibilities and remaining vigilant?
We have the same guarantee about that today as we did a month ago before this was announced. Apple writes the software that runs their devices, so they’ll always have the option to ship something like that.
Apple could have quietly implemented CSAM scanning server-side, and left the door open to it being quietly exploited in who knows what way. But they didn’t: instead they put a whole bunch of infrastructure in place that all but guarantees that they’ll be immediately caught (and publicly excoriated) if they try to use this CSAM mechanism for anything other than CSAM. (See the PDF that GP linked for technical details on why.)
Of course, they could still do it with some other mechanism. But in that case none of these CSAM changes are at all relevant to the concern, as that risk is unchanged from a month ago.
Their privacy page[1] certainly suggests they have been since 2019 - "pre-screening or scanning uploaded content for potentially illegal content, including child sexual exploitation material".
Do you have some source to indicate definitively that they have not been scanning iCloud Photos for CSAM?
> So you'd have to pressure NEMEC and another org under a different government
To be fair, if the other organisation is IWF[1] under the UK government, I don't think there'd be much pressure needed to get them to comply - just offer to bung them and their mates a few million in contracts and you'd be golden.
It's a sensible plan, it just might not be as strong as it seems.
No, just pointing out that the first part isn't as strong as Apple seem to think it might be. The rest of it might well catch this, yes, (and that's good!) but it doesn't obviate that "only considering images from two or more providers" can be subverted, possibly easily, given the pressure USGOV can bring to bear on their partners.
Apple doesn't even have to know how their scanner will be used. It could simply be a requirement for selling phones in Country X, Apple must scan devices for hashes from DB-20210819-Illegal-Hashes-Country-X.zip. The hash file will be provided by Country X as will the messaging contacts for appropriate state security services.
> governments around the world could pressure Apple to add other forms of surveillance. Be it hashes of non-CSAM, or simply pressure NEMEC or other hash providers
could, yes, but given PhotoDNA has been using similar hashes from NCMEC for a decade and doesn't appear to have had similar surveillance pressure imposed, what makes the Apple scanning different to warrant slippery slope arguments like this?
My initial response was that maybe that's why Apples software development has gone down in quality so much in the recent years. But then again he's been leading it since 2009 so maybe not. But it's definitely around the same time that they started removing programmability from the core os.
I deeply hope that he gets pushed out because of this. The skeuomorphism guy was definitely horrible and obviously was the cause of many assaults upon the senses, but this CSAM spyware scandal was such an affront on all that democratic, enlightened society holds close to the chest.
There are so many more talented software development leaders out there (maybe scratch all the ones working on adtech), but he's clearly a cultural troglodyte and this is evidence that he is incapable of actually leading the future engineers and software workers of the world.
If I saw him in a social setting I would try to ask him how he lives with himself, actively enabling the potential future dystopia and being incapable of defending the actual impetus of his actions, much less the reaction to it. Truly unforgivable, especially in an age of increased social fragmentation.
> If I saw him in a social setting I would try to ask him how he lives with himself, actively enabling the potential future dystopia and being incapable of defending the actual impetus of his actions, much less the reaction to it. Truly unforgivable, especially in an age of increased social fragmentation.
He seems like a guy who genuinely believe that what he's doing is a net positive for society and that society should just leave things in his hands.
There are plenty of people like that at google last I remember. I used to go to the google dev day and the was no greater difference between really talented technical people presenting interesting things and engineering leadership believe that everyone should just have a dumb terminal in their hand and that google should do any processing and how that would be so much better for society.
> Craig's interview in and of itself was the biggest indictment of technology companies one could ever hope for. Dude literally stumbles as he says "multi-step algorithm"
I’m starting to wonder if this is Apple’s strategy all along. They’ve had pressure for years from law enforcement to abuse their monopoly power to invade people's privacy. So they implement it in a relatively naive approach and sell in a hamfisted way, so that everyone turns against them for a little while. Apple backs off, and every time after this they point law enforcement to this debacle. See, we can’t help you, our customers wouldn’t tolerate it!
Not sure if it’s true or not, but we shouldn’t assume companies always intend their public plans to succeed. Maybe they actually want a high profile retreat — it could certainly save them development time down the road by not implementing government misfeatures.
Still incompetent. This would have made sense under another administration. The only thing holding back the antitrust dogs is public opinion. An Apple antitrust case would be politically costly in a way a Facebook case wouldn't. This type of thing corrodes the core of the pro-Apple vocal minority. It's bad GR, if that was the plan.
The second the world realized those weird nerds no one paid any attention to were gold mines was the beginning of the end. Whatever values we held as hackers, engineers, scientists is now subservient to the monied powers who could never figure any of this out on their own but are exceptionally skilled at convincing or threatening everyone into dancing to the beat of their own self interest.
Stop feeding them and they will starve on their ignorance. I'd rather be poor in a better world than rich in a dystopia.
> I'd rather be poor in a better world than rich in a dystopia.
The problem is that we all end up being poor in a dystopia. The regulatory capture of industries is an ongoing thing. When 20 years ago you could start your own business in your garage, without substantial capital, today you are dependent on "Angel investors" and bank loans, which is an equivalent of having to be referred into a "club". If you can't find an investor, it will take you a lifetime to save money to start a business, if you are lucky. Take notice that all taxation is being shifted to workers to limit their chances of raising capital and creating competition.
The "screeching minority" memo gives some insight. Let me break it down line by line. This is just my personal opinion as an external observer; I have no actual insight into what was going on.
Team Apple,
This is from someone outside the Apple ecosystem that management brought in and gave a lot of credence to.
I wanted to share a note of encouragement to say that everyone at NCMEC is SO PROUD of each of you and the incredible decisions you have made in the name of prioritizing child protection.
The recipients of this memo didn't make any of the decisions and those were foisted on them by management who in turn were heavily influenced by this person/organization. As to why management was so heavily influenced, I have no idea. This is an attempt to get buy-in from Apple employees I suspect, or at least to ease their concerns. "Don't worry, you are doing the right thing."
It’s been invigorating for our entire team to see (and play a small role in) what you unveiled today.
The team they are referring to are the ones that pushed for the implementation for child protection. The recipients did all the work because management told them to.
I know it’s been a long day and that many of you probably haven’t slept in 24 hours. We know that the days to come will be filled with the screeching voices of the minority.
This is the big quote. The author knew what this would do, and knows the tarnish the Apple brand is about to go through, but is trying to convince the audience that it's only a small group of people who will reject this and it will pass over. For the author, the ends justify any means because they are probably personally emotionally traumatized by child porn (rightfully so). Once this happens though, any rationality about consequence of action goes out the window. This is an attempt to get everyone concerned on the same level of ends-justify-the-means with the author. I don't know if the author believes this is just a temporary outrage or not. I suspect not. It doesn't really matter though the author got what they wanted, Apple brand be damned.
Our voices will be louder.
This obviously isn't the case. Not sure if the author means their organization or their organization + Apple. Either way, the counter resistance to this has been pretty minimal. They left Apple hang out to dry it seems.
Our commitment to lift up kids who have lived through the most unimaginable abuse and victimizations will be stronger.
During these long days and sleepless nights, I hope you take solace in knowing that because of you many thousands of sexually exploited victimized children will be rescued, and will get a chance at healing and the childhood they deserve.
This is the emotional sell, or "for the children." It's also a difficult bridge to reason across; comparing the possession of pictures and the manufacture of pictures as the same thing. Obviously the hash compare only finds existing manufactured pictures, so I don't see how it will protect children from any abuse that hasn't already occurred. Not only is this an emotional sell, it's a lie.
Thank you for finding a path forward for child protection while preserving privacy.
The final emotional pitch directed at people at Apple typically concerned with privacy. They aren't really protecting children and they also aren't preserving privacy. It's a boondoggle.
In closing, "for the children," is almost always a trap. It was used when I was a kid to mass incarcerate Americans that had drug problems, many of them the children they were protecting. It's typically done in an any-means-necessary heavy handed manner, as seen here. I don't condone child pornography in any way. I have never seen child pornography and hope to never see it. At the same time, what they are doing is the road to hell paved with good intentions seemingly by people who are so traumatized (based on their work) that they c...
Yeah I still remember Scott Forstall being pushed out over the Apple Maps failures. And that is how Jony Ive came to take over both hardware and software design, leading to the end of skeumorphic design that Jobs loved so much - and made the iPhone so unique.
I heard he was pushed out not due to the failure but that he refused to improve and accept the mistakes that he made. It pissed off Tim Cook who was already mad that he apparently couldn't work with the other team members well.
Take all this with a grain of salt. I wonder if Jobs was just good at managing these brash personalities together.
Trust takes years to build and seconds to break. I could be grossly overestimating the impact that this has had to their brand. But in my eyes, I can't see a way that they revert the damage, even if they revert the decision.
Even as someone in tech, I am having a terrible time finding a viable replacement for my iPhone and MacBooks. People won’t inconvenience themselves too much. If there is no easy alternative, most people will choose not to think about it and carry on as before.
Hard disagree. This entire process reveals systemic rot at Apple.
The engineers who built, the project managers who coordinated, and the executives who signed off on this smoothbrain idea need to be drummed out without ceremony.
No severance, no golden parachutes, no cushy board jobs etc.
People need to understand they risk their livelihoods building and deploying a surveillance system like this. In a just world they would face criminal charges.
Yes, we do indeed disagree. This was a mistake. This was a bad idea. But nobody died, and it hasn't even been deployed yet. We should try to lower the stakes involved in admitting mistakes and correcting bad decisions. If making a mistake is a career death sentence, then people just dig in and become intransigent.
It's so bizarre how hard they're pushing this, even though they don't really gain much from it. This is especially weird because Apple has always pushed for user privacy. I just don't get it.
> As a cloud photo hosting provider (iCloud) they might legally be required to (IANAL)
This doesn't explain the insistence on scanning all images on device, whether they're hosted on iCloud or not.
The moment for leadership from Cook has come and almost passed. It's surprising watching Apple squander its brand at this moment. A good amount of its previous support with respect to federal antitrust has permanently dissipated.
They're only obligated to report if found, but they don't have to proactively scan (yet?). In the EU until recently they weren't even allowed to voluntarily scan.
This is the text of the current law [1]. Alerting authorities about CP is required if discovered. Actively searching for CP is not required. Look for the aptly named "protection of privacy" paragraph.
2. FBI/CIA, frustrated at previous failures, pressure senators/congresspeople to turn up the heat.
3. FBI/CIA then gives Apple the option to implement surveillance with the promise that they can make the antitrust troubles go away if Apple does what they want.
4. Apple does the thing. (We're here now.)
5. Antitrust case goes away.
The FSB, CCP, and other intelligence orgs might be trying the same strategy in their countries.
I expect this is how everything works in the big leagues.
I don’t think anyone within Apple is pushing for this, I assume the pressure is coming from somewhere external or they’re anticipating pressure because the feature doesn’t really make a lot of sense right now. We’ll probably get the actual truth years from now in some leak.
Apple will not drop their plans. Google and Facebook are already doing this. Be happy at least Apple’s implementation happens client side and therefore can preserve end to end encryption.
> Be happy at least Apple’s implementation happens client side and therefore can preserve end to end encryption potentially.
No, why would I be happy with this? I actually prefer the alternative: scan photos server side, fine. That's Apple's computer and if I send my photos there, then I can't justifiably argue against that. But my device shouldn't be part of their attempt to appease law enforcement by turning the world into a dragnet.
Not sure why you are getting voted down - you are 100% correct.
Especially in this post 9/11 world - look at what was supposed to be "temporary" to deal with the "terrorists". Ha! You have to be an utter simpleton if you don't think use of this mechanism will expand. It's not even a leap of logic - it's been going on for 20 years all around us!
Sure. And the police only have your house under constant surveillance in case you sell heroin out of the garage. Don't worry, comrade, there is no reason to worry if you have nothing to hide.
Nobody expects Google and Facebook to care about privacy. Apple was the one running TV ads about how they were the privacy company. If Apple was really planning E2EE they would have announced it at the same time, because it would have been the perfect time to do it. Disarm the critics of E2EE with CP scanning and disarm the critics of CP scanning with E2EE.
I agree. The claims that E2EE are right around the corner are getting really tiring. If it was coming, Apple should have announced it alongside this "feature" and not expected everyone to trust them.
No, iCloud Photos are not currently end-to-end encrypted. Apple lists all of the items in iCloud that are covered by E2EE in the "End-to-end encrypted data" section after the table, and photos aren't on the list:
What’s your point? Apple said RISC was better and faster than Intel, until the day they adopted Intel, and then they said the complete opposite. They acted like they invented the Omnibox for Safari and so forth. It’s for marketing.
What if this is four-dimensional chess by Apple: maybe they deliberately announced this back door, just to prove how politically poor the backdoor topic is with the public? Maybe Congress will think twice about trying to force this via legislation on anyone.
The real question is: how can they make sure people are not committing other crimes? They better scan all messages at all times. Are they meeting more people face to face than the government permits? The only way to be sure is to log all gps data and compare.
They better monitor every aspect of their customers lives to protect the children. Think of the children
No. This is more like Apple enabling telemetry on your iPhone, and if it looks like you're traveling on a road higher than its designated speed, you'll be auto-fined.
> It is also the only country from all of those I have been in almost 50 years, where great care is taken to never park in an illegal way.
This is only true for the masses. Rich people just leave their cars parked anywhere they want in Mayfair in London because they don't care about a 50 quid fine.
Everything you said is already being collected about you. Cellphone towers can log where you are. Cameras around your city can recognize you. All they have to do is link all the feeds into one database that can be searched. The only place you can escape the watchful eye is your end to end encrypted stuff… which ain’t your SMS. And even there, you are just taking their word for it since it’s a closed source product.
In fact, couldn't we go even further, taking inspiration from Apple?
Plumbing installed in people's homes should scan for illegally consumed narcotics or prescription medications and check them against pharmacy records. Your home's electrical wiring should make sure it's not being used to illegally grow drugs. Cellphone cameras should make sure they never record a naked body without a signed consent waiver. Guitar amplifiers should verify that you're not playing an owned piece of music without a license.
How could we ever survive in a world where we relied upon targeted investigations and probable cause? How could we ever live in a world where pipes just moved things from A to B without inspecting them.
1) Some dev team at Apple is writing the on device scanning code.
2) Some dev team at Apple is writing a "sendMessagePolice()" API.
3) Some QA team at Apple is testing both systems.
I would personally quit if asked to work on any of these teams.
If you were the QA team lead for this, how would you test it? I can't imagine anyone would do day-to-day testing with actual illegal images.
It would not surprise me if the dev and QA teams at Apple already have hash colliding images to run their tests?
This is all awful. Steve Jobs must be rolling over in his grave.
Understood that that is how you test in the early stages. But at some point you have to run tests against the actual deployed perceptual hash database.
No reason why you can't just generate random images and add them to the real database, indistinguishable from actual hashes. As long as the test images aren't public it's low risk, and even if they leak they could be removed from the database and new test images could be generated.
Apple's "CSAM detection" feature is a two-part on-device/server as I understand it:
1: Run on-device code to perform perceptual hash comparison of each photo against
an on-device encrypted database of known CSAM hashes.
2: On iCloud Photos servers, send out the relevant notifications when a user’s iCloud Photos
account exceeds a threshold of positive matches.
So as a high level testing strategy, I would want to:
- Verify on-device lookup of CSAM hashes. This could be tested by provisioning a test device with an on-device database containing CSAM hashes of images that aren't illegal. As a bystander, I think I'd be fairly confident with this approach because I'm guessing the on-device database that Apple ships could conceivably be changed over time to expand the definition of the images it will flag as CSAM.
- Do some exploratory testing to discover the threshold of how much image manipulation can be done on a flagged image before the perceptual hash comparison fails to return a match.
- Verify that the notification system notifies the correct parties once a user account exceeds the defined threshold of positive CSAM matches.
- Ensure the flagged account can still be investigated if user deletes the offending material from iCloud, or their account by the time a real person gets around to investigating.
- Ensure that the logging is informative and adequate (contains device name, timestamp, etc.).
- Test behaviour on same iCloud account logged in to multiple devices.
- Figure out any additional business logic - are positive matches a permanent count on the account or are they reset after a certain amount of time?
i imagine they are doing this to comply with some law that requires they scan images in iCloud, and they don't want to start storing those un-encrypted. Does anyone know if there is such a law?
This is the text of the current law in the U.S. [1]. Alerting authorities about CP is required if discovered. Actively searching for CP is not required. Look for the aptly named "protection of privacy" paragraph.
There might not be a law, but in practice, letting criminals host illegal data on your servers is unlikely to be a good look.
The intent of the laws making the possession CSAM illegal are ultimately to stop the spread of CSAM. If those laws fail in their stated purpose just because a company chooses not to be proactive in how it searches for the material, that would be missing the point. The media and governmental pressure would wipe any company foolish enough not to implement child safety measures out of existence, but no major tech company is that foolish (except Kik perhaps). And hypothetically, the government could just pass a law enforcing proactive scanning if there were to exist a company that chose not to proactively scan, which would mean that the government would fail to solve the issue of preventing the spread of CSAM, and thus CSA.
But in practice, such a law isn't needed. No company wants to be derided as a service that allows child abusers to get away with their crimes.
—Nothing in this section shall be construed to require a provider to
—(1)monitor any user, subscriber, or customer of that provider;
(2)monitor the content of any communication of any person described in paragraph (1); or
(3)affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b).
Hrm, that's interesting. I wonder if Apple iCloud has surfaced in enough cases as where criminal activity is actively taking place that it prompted Apple to do something about it.
Homosexuality is a crime in some countries. Would Apple allow country level filters that call the police in some countries but not others due to local criminal statutes? What about cannabis use?
What is the alternative? Not following local law? Apple is just a company, they cannot and do not make their own laws, this should be obvious and not a point of criticism. If you don’t like the laws as they are written in China, then you shouldn’t be lobbying Apple to ignore or break them, you should be lobbying politicians to change them.
I find this discussion interesting, because there is no way to prove that Apple does not already do this, since their operating system source code is completely opaque, and the lenghts they go to to prevent you from viewing or modifying it.
Imagine this: Apple's software is scanning for abuse images and finds a ton of false positives which are then manually inspected. Those false positives are... >>private photos of naked adult man and woman<<... and this is exactly what you want to hide from >>everybody<< on the planet... but no: they will be manually inspected... Good luck Apple fans! :-)
False positives will be very rare. We probably won't hear of any incidents except for activists trying to spoof the system.
CSAM is one of the most extreme crimes against a person and the number of people sharing and profiting from CSAM is growing. This is something every tech company will have to deal with.
In the US, if I understand correctly, those would be true positives, and everyone who created, received, or distributed them would be guilty of a crime. There’s no exception in the law for pictures you took of yourself.
If true, it’s incredible that the numbers are growing. With today’s big data and surveillance (i.e., privacy is dead), I would think it would be impossible for there to be any sizable groups profiting from CSAM.
Have any wireless providers verified that Firmware/OS/App updates can't be pushed via OTA to those devices? Asking because to my knowledge no such verification has been attempted. Background: I used to push firmware updates to phones in the 90's on a GSM network. The only reason we didn't do this globally was risk of customer support issues.
I understand what you are saying, but in my opinion that doesn't really answer the question. For a phone to attach to a network, there is some level of bidirectional trust and those kill-switches aren't going to just arbitrarily activate. Has anyone at a wireless provider tested pushing OTA updates to those phones? Has the vendor explicitly stated that the phones will explicitly block by default any attempts at OTA updates or that the phones explicitly do not have the capability?
I don't think any vendor confirmed that. But is this possible to update software on my device without permission by its OS via a well-defined external interface? These phones are desktop computers running GNU/Linux. If they can be updated this way, then my laptop can as well be updated by a WiFi card, can't it? (I don't think so)
I suppose it depends on what standards each of these Linux phone implementations have followed. Android also runs Linux, but they modified it to give the wireless providers and Google full control, hence people wanting to root their phones. Just running Linux is not enough to give me confidence that the phone is truly isolated from network management. So I suppose it comes down to what kernel modules, libraries and trust these phones are giving to the provider and what standards they are following. A good start would be independent kernel hackers digging into the hardware and OS to see what they can make the phone do and what they see the phone doing under normal operations when attaching to the network and how the phone responds to OTA update attempts and what the firmware on the phone is doing. If I control the firmware and you control the OS, I control what your OS can read and write to. So I suppose my unanswered questions are:
- Who writes and maintains the firmware the OS is running on.
- Who writes and maintains the modem firmware and who can update it.
- Who can update that firmware for the board the OS is running on. This could be a different answer than who initially creates it for the retail distribution of the phone.
- What level of trust has been inserted into the OS by kernel modules and who maintains those kernel modules.
- What control is given to the end user to see what those modules are doing and limit what they can do.
I suspect more questions could arise as kernel hackers audited the phone. The dilemma I see is that such kernel hackers won't be interested until those phones are wildly popular. The only other way to get their interest is with money.
I am glad that Apple has shown the true colors of trillion dollar corporation thinking.
I see opportunity for people to understand the biggest problem with a future in which humans are only a datapoint in framework designed to benefit few on expense of many.
Can you imagine this "narrative" applied everywhere? From the moment you open your eyes everything around you is extracting data, monitoring and policing you.
We have to redefine our relationship with software and hardware (electronics). The infrastructure (5g) and IOT advancements are here. Data is the fuel of the coming "consumer revolution".
Apple just positioned themselves perfectly as a Data Broker for high level customers - special groups, oppressive governments, you name it.
They will not back up from this. This is more profitable than selling shiny toys and "services".
This gives political elites a reason not to break Apple monopolistic business practices.
I will go even further: This gives Apple a favorable position to become a trusted partner of governments of the World.
Deconstructing this only from a technical point of view is non-efficient. Apple is making an entrance on political stage by embracing the role of World Digital Police.
Everyone is free to apply his own conclusions and analysis.
Mine are proportionally adequate. I have removed macOS from production workflow.
I will never buy another Apple product. And I am willing to throw away "conveniences" and polished UX for preserving my privacy.
This privacy, not Apples corporate redefinition:
The quality or condition of being secluded from the presence or view of others.
The state of being free from public attention or unsanctioned intrusion.
A state of being private, or in retirement from the company or from the knowledge or observation of others; seclusion.
There's been a lot of question about the motivation for this feature. Am I the only one that thinks this could be related to things like the EARN IT (or something similar).
Here's an article from a year ago on EARN IT:
> Theoretically, a system that uses client-side scanning could still send messages encrypted end to end, and so the Leahy amendment would not offer any protection, but many of the same confidentiality concerns with backdoored “e2ee” systems would continue to apply.
Sensitizing people to the idea that a benevolent dictator is going through content on their local device for their benefit.
It's all about conditioning. First it's to save the children. Then it will be about preventing the terrorists. Finally it will be about preventing "hate speech" - which these days hate speech is pretty much speech from anyone you disagree with :p
Like the 1984 novel, Minority Report wasn't an instruction manual!
I want to ignore most of this article to complain about an inaccuracy that keeps coming up.
> More broadly, they said the change will break end-to-end encryption for iMessage, which Apple has staunchly defended in other contexts.
But... it wouldn't. The iMessage feature doesn't expose the contents of your message to anyone else under any circumstance.
If you're a child under 13 and your parents have opted in to this feature, you get a choice of seeing naked-pictures sent to you and having your parents be notified that you chose to, and not seeing it with no notifications of anything. (But once you're 13+, no notifications would occur either way.)
There are potential issues with this, mostly relating to abusive families being controlling. They'd have to do weird things like forcing their teenaged children to keep making new under-13 accounts to actually take advantage of it like that, though. And none of these issues impact the e2e status of iMessage in any way.
Apple really screwed up PR by launching the iMessage feature alongside the scanning-and-reporting iCloud Photos feature. There's so much confusion out there about this.
(The breaking-e2e aspect does exist with the iCloud Photos scanning... not that it's currently e2e, of course.)
How does this break e2e for Messages? They're still only ever decrypted on your device, and their content is never shared with others. You have to radically redefine what "e2e encryption" means for this to qualify as breaking it.
There's no e2e encryption for Photos currently, so it can't be breaking it there. If you use iCloud Photos, your photos are stored encrypted, but Apple has the key and could be doing all the cloud-scanning it felt like.
A hypothetical future where Apple doesn't have the encryptions keys for iCloud Photo Library, but this on-device scanning for uploads to iCloud Photo Library is happening -- that sounds more secure than the status quo, in which totally arbitrary searches of your photos could be done on the server in response to warrants/NSLs.
I'm sure in such a future there'd be quite the argument happening about whether you could call it e2e.
What also sounds more secure than the status quo is not scanning my private photos anywhere – not on my device, and not in your cloud where I grant you custody of them.
Although this is true, none of the major cloud services seem inclined to actually do that. Google, Microsoft, Dropbox, Facebook, they're all scanning photos[0].
Let's not pretend that this sort of thing is happening in isolation. There really is a big legal movement towards requiring encryption backdoors, and programs like Apple's here are effectively part of a big negotiation with law enforcement -- note that the reason Apple's currently not e2e is apparently that the FBI pushed them on it[1]. "We'll give you what you claim is important, CSAM scans, and in return we'll lock everything else up securely". Is this great? No. Would an absolutist "law enforcement can have nothing" position backfire into stupid legal decisions? Plausibly.
Content is no longer private if there is anything inspecting it other than you.
I expect cloud providers to fiddle with my data - because they can and do. I don't expect vendors, especially one touting themselves as being more privacy focused than everyone else, to be sifting through my data on device.
Yes, today it's a carefully designed algorithm designed to be one way. The algorithm isn't the problem. The problem is the method used to deliver this algorithm can also be used to deliver *other* algorithms. And no, especially since 9/11 I have zero faith that this is as far as this mechanism will ever be used.
Okay, but your device literally has to decrypt the message in order to show it to you. It then does other things that are vaguely "content analysis" like generating link previews and marking up the text to make dates clickable.
The iMessage feature is "it looks like someone has sent you a picture of their dick, are you sure you want to see that?" I know adults who'd like to turn that feature on.
Could it hypothetically be extended in the future? Sure. But arguing against this feature as-is is about the same as arguing against the feature that lets you search for "dog" in your photos.
"It's a backdoor. We know we said backdoors are bad and that a backdoor can be used by both good guys and bad guys, but this backdoor is made with Apple-certified lumber."
212 comments
[ 2.7 ms ] story [ 284 ms ] threadCan't believe this dude is running software at Apple.
The headline for all of this should be "We know what it does and we don't want it"
And that interview is likely supposed to telegraph that they “don’t care”. It wasn’t a live one, was it? They could have improved parts of it if they wanted.
Everyone understands well enough how it works and understands how trivially it could be changed to search for other kinds of content.
At some point, the back story behind this will leak out. That will be interesting. This has to be something Apple was asked/pressured/ordered to do. As a business activity it makes no sense.
Bear in mind that over the last week, people in Afghanistan have been frantically trying to erase any evidence of doing things the Taliban doesn't like, such as women playing sports.
This is a fairly typical corporate failure mode.
Sometimes the best thing is to do nothing as they have found out because now they’re stuck in a position where the two exits are either piss off everyone by writing off their privacy stance or piss off everyone else for canning child porn scanning.
What a complete fuck up.
I firmly believe we'll never discover who applied that pressure, or how. And it's hard for me to imagine the US government doing that without some court battle with serious risk of leaks to journalists. I mean, it makes no financial sense, so Apple would fight that. The only thing I can imagine is pressure from a private interest whose market is big enough for Apple to care. And if it is that market, then I think we're in for a tough ride until we can build our own devices.
If you read the documentation on how it's implemented, there's some fairly advanced crypto - private set intersection, threshold secret sharing - that only makes sense if Apple took the line "we have to do this, but we're willing to do it in a really expensive way so that we ourselves have as little access as possible". They went to the effort of running the NeuralHash on the client device, as far as I understand.
The standard implementation on other cloud providers is that the provider has access to your data on the server if they really need it - much cheaper, and makes it technically possible in future to easily change the T&C to "we may use this for market research and to improve our products". I view Apple's client-side implementation as drawing a very big line in the sand saying we will not do that and we are willing to put our money where our mouth is by writing this crypto protocol.
But that certainly raises the question, why would Apple do anything like that at all, unless there's some (current or anticipated) external threat that forced them to?
As to whether we'll ever discover it - I wouldn't have predicted the Snowden revelations, maybe this will come out the next time something like that happens. If it does, I also predict we'll be shocked to discover just how big the market and distribution network for child abuse online is, and how many people are involved.
I don't think it is big compared to what authoritarian states would like to use this for. You're talking about, at the very least, putting everyone in non-democratic states under even more pressure to obey than they're already under. And in democratic states this invites bad actors both in and outside the government. There is no net-gain in justice here. It's wrong across the board.
Welcome back to HN after taking 5 months off, btw. You're a SSC fan?
Yes, I agree the state market is even bigger - China's potential market for surveillance alone is 1.3 billion and rising - but I've heard from colleagues in tech who seriously think child abuse is only a very small number of bad actors. My understanding is the "industry" is almost as out of control as the illegal drugs trade.
I am indeed a SSC/ACX fan (and subscriber).
And now I am down a rabbit hole. Probably for a while. But thanks for that. Seems like the world needs more thinkers like this person, and everyone, including myself could learn a lot from them.
Snowden paid a very high price not many people can afford, especially when they have a family. So I highly doubt we could know it within a reasonable timeframe.
Anything going through a FISA court (Foreign Intelligence Surveillance Court) could come with a gag order, preventing anyone involved to talk about it.
> "Though these capabilities are intended to protect children and to reduce the spread of child sexual abuse material, we are concerned that they will be used to censor protected speech, threaten the privacy and security of people around the world, and have disastrous consequences for many children,"
Many people and organizations fear that Apple's plans are paving the way for governments to scan for more than just child abuse images. (e.g. https://appleprivacyletter.com/)
In that light, FISA is relevant. My statement was to show that not all dealings of the US government are public, because not all courts and legal proceedings are available to the public. So "serious risk of leaks to journalists" won't likely happen in that case.
(One counterpoint to this is that PhotoDNA has been used for scanning cloud images using NCMEC-provided hashes for a decade now and this slippery slope doesn't seem to have happened yet.)
My understanding is that there is a threshold that needs to be met, and then an Apple employee with a very difficult job will need to confirm the image. So even if NCMEC get pressured, I don't see what the result can be.
Replace “FBI” and “Biden” with their equivalents in various other countries and yes, it’s absolutely plausible.
I think it's not really a problem of what is being looked for, but the mere fact that it will be done without a warrant on a personal device.
There are good reasons not to want this. Just not making everyone into a suspect would be a start.
But whoever wrote the letter is clueless.
In my opinion, this is the biggest concern, not the technology. Before, Apple could simply refuse by saying we don't have the capabilities. But now, that excuse is gone. Apple's promise to human review content and only report CSAM is the weakest link.
What do they have now that they didn’t have before?
So you'd have to pressure NEMEC and another org under a different government to both add the non-CSAM hash, plus Apple would need to be pressured to verify a non-CSAM derivative image, plus you'd need other hash matches on-device to exceed the threshold before they could even do the review in the first place (they can't even tell if there was a match unless the threshold is exceeded).
I get why people are concerned, but between this thread and the other thread yesterday it's clear that pretty much everyone discussing this has no idea how it works.
1: https://www.apple.com/child-safety/pdf/Security_Threat_Model...
As it happens, I still think you're missing the point that myself and others are trying to make. Perhaps the following is a better way of phrasing my inquiry:
What guarantees to we have that Apple will, always and forever, only use the NCMEC/CSAM/IWF system to scan images on phones? By that I mean, isn't it perfectly plausible that Apple can leave the NCMEC/CSAM/IWF system in place, as it is described in your post and the linked PDF, and at the same time partner with [random body X/Y/Z], who has their own database of [whatever] that they scan separately from the NCMEC/CSAM/IWF system? These two different scans wouldn't ever need to communicate with each other, and could run at the same time.
So what? What you're describing is not what this system does. You could argue against literally any possibility anyone could dream up, because they're the platform vendor and they can make any change they want. If that's a problem when you're evaluating your threat model then this new system is the least of your worries.
And as far as I can see this system is the least intrusive version of what any of the other cloud vendors are doing. Don't want it? Turn off iCloud Photos and sync to something else like your own Nextcloud instance.
I said that the hypothetical system could be set up, and function, in the same way as the NCMEC/CSAM/IWF system, just with different entities behind it. Despite chastising me for what you thought was a failure to comprehend your post, this is the third time you seem to have been unable to grasp that concept.
That's part of what people are concerned about - an identical system running in tandem to this, but with less savory characters behind it. Given how many comments on HN have addressed that this week, including all of mine right here, I am at a loss for how to make that any clearer to you.
Right!
>Yes, a lot of comments have addressed something that isn't happening.
No, and the confusion lies in the tense, "isn't" vs. "won't". A lot of comments have addressed a lack of trust that such a scenario won't happen.
Yes they could do that, but this doesn’t help them.
Right! I clearly stated multiple times that this would be a second system working in tandem, of course it would need to be deployed on its own. I am entirely unsure how you think I am ignoring that when it's the premise of my argument and of the concern of others.
Edit: Removing some paragraphs, because if that's not perfectly clear to you by reading this entire comment thread, even when I basically said as much in the post you just responded to, then I really don't know what to tell you.
You being able to imagine a fictional evil parallel system isn’t a valid criticism of what Apple is actually doing.
Right! And where did I criticize them? All I did was ask, “What guarantee do we have that Apple will, always and forever, only scan for data that comes from NCMEC/CSAM/IWF?” and then gave an example of what such a scenario might look like when another user responded to me.
I expressed a concern (I even used that word, and not "critique" in my last post that you responded to) that this type of scanning could be expanded in the future, but I have not criticized them broadly nor have I criticized this specific program. How has that not been clear?
If you think I’ve done otherwise, I’d appreciate it if you could quote what you interpret as a critique, so that I may work to avoid such confusion in future conversations similar to this.
Otherwise why post it here? I doubt you’d claim that your ‘concern’ is irrelevant to this topic.
It’s an innuendo intended to imply a likelyhood of future wrongdoing.
It works like this: ‘What guarantee do we have that he won’t hit his wife after we leave?’
Obviously this is a way of suggesting that the man will hit his wife, phrased as a question to facilitate the same kind of denial that you are using here.
… duh? I never said it wasn’t related, I said that it’s not a criticism of the current program.
>Otherwise why post it here? I doubt you’d claim that your ‘concern’ is irrelevant to this topic.
Correct!
>It’s an innuendo intended to imply a likelyhood of future wrongdoing.
No, it’s intended to state clearly - not imply - a concern around whether or not Apple will be pressured by other bodies to begin scanning on-device content for hashes pertaining to subjects beyond child abuse.
The question is simple - what guarantee do we have that Apple won’t expand what kind of content they scan for? What is so difficult to grasp about that, and why do you, it seems, feel that that is an entirely irrelevant question that isn’t worth discussing, answering or being concerned about?
>It works like this: ‘What guarantee do we have that he won’t hit his wife after we leave?’
>Obviously this is a way of suggesting that the man will hit his wife, phrased as a question to facilitate the same kind of denial that you are using here.
You leave and hope that he doesn’t, but would you never follow up with your friends to see how they’re doing after a domestic altercation, to make sure it hasn’t happened again and that they’re safe from harm? What is so wrong about being aware of possibilities and remaining vigilant?
You’re both proving the point.
Nobody said the man had done anything wrong, and yet you are assuming he did simply because someone asked a question and recommending that people act on this false impression.
That is exactly the goal of innuendo.
But again, since you continue to avoid what I'm asking:
>The question is simple - what guarantee do we have that Apple won’t expand what kind of content they scan for? What is so difficult to grasp about that, and why do you, it seems, feel that that is an entirely irrelevant question that isn’t worth discussing, answering or being concerned about?
and perhaps more curiously
>What is so wrong about being aware of possibilities and remaining vigilant?
My scenario never said anything about ‘leaving’. You continue to confirm the point. You read that in to the scenario.
That’s the point. You completely failed to understand the scenario as written and made up your own story to suit your prejudices. The only violence was in your imagination.
What guarantee do we have that you aren’t doing the same thing with Apple?
LOL, surely you jest:
>It works like this: ‘What guarantee do we have that he won’t hit his wife after we leave?’
At this point, after continuously misinterpreting my posts (almost intentionally, as though you're arguing in bad faith), you've effectively moved on to gaslighting, so I'm just going to continue to reiterate the same questions I have been posting, which you continue to avoid answering. Carry on as you have been if you want, but this is about all you'll get from me moving forward until you answer them directly.
- Apple has decided to scan for a specific kind of content on their phones, what guarantee do we have that they won't scan for other content in the future?
- For someone who commented, "stand up for civil liberties now", why are you so opposed to people being aware of possibilities and remaining vigilant?
Apple could have quietly implemented CSAM scanning server-side, and left the door open to it being quietly exploited in who knows what way. But they didn’t: instead they put a whole bunch of infrastructure in place that all but guarantees that they’ll be immediately caught (and publicly excoriated) if they try to use this CSAM mechanism for anything other than CSAM. (See the PDF that GP linked for technical details on why.)
Of course, they could still do it with some other mechanism. But in that case none of these CSAM changes are at all relevant to the concern, as that risk is unchanged from a month ago.
Aren't iCloud Photos already scanned for CSAM though?
Do you have some source to indicate definitively that they have not been scanning iCloud Photos for CSAM?
[1] https://web.archive.org/web/20190701000647/https://www.apple...
To be fair, if the other organisation is IWF[1] under the UK government, I don't think there'd be much pressure needed to get them to comply - just offer to bung them and their mates a few million in contracts and you'd be golden.
It's a sensible plan, it just might not be as strong as it seems.
[1] https://www.iwf.org.uk
could, yes, but given PhotoDNA has been using similar hashes from NCMEC for a decade and doesn't appear to have had similar surveillance pressure imposed, what makes the Apple scanning different to warrant slippery slope arguments like this?
Not with the hashes provided by NCMEC, though, right?
The same system is already in use for things other than NCMEC by other providers.
https://www.apple.com/leadership/craig-federighi/
There are so many more talented software development leaders out there (maybe scratch all the ones working on adtech), but he's clearly a cultural troglodyte and this is evidence that he is incapable of actually leading the future engineers and software workers of the world.
If I saw him in a social setting I would try to ask him how he lives with himself, actively enabling the potential future dystopia and being incapable of defending the actual impetus of his actions, much less the reaction to it. Truly unforgivable, especially in an age of increased social fragmentation.
He seems like a guy who genuinely believe that what he's doing is a net positive for society and that society should just leave things in his hands.
There are plenty of people like that at google last I remember. I used to go to the google dev day and the was no greater difference between really talented technical people presenting interesting things and engineering leadership believe that everyone should just have a dumb terminal in their hand and that google should do any processing and how that would be so much better for society.
Anybody a link?
Here you go
Not sure if it’s true or not, but we shouldn’t assume companies always intend their public plans to succeed. Maybe they actually want a high profile retreat — it could certainly save them development time down the road by not implementing government misfeatures.
Still incompetent. This would have made sense under another administration. The only thing holding back the antitrust dogs is public opinion. An Apple antitrust case would be politically costly in a way a Facebook case wouldn't. This type of thing corrodes the core of the pro-Apple vocal minority. It's bad GR, if that was the plan.
Neither do they about privacy.
Stop feeding them and they will starve on their ignorance. I'd rather be poor in a better world than rich in a dystopia.
The problem is that we all end up being poor in a dystopia. The regulatory capture of industries is an ongoing thing. When 20 years ago you could start your own business in your garage, without substantial capital, today you are dependent on "Angel investors" and bank loans, which is an equivalent of having to be referred into a "club". If you can't find an investor, it will take you a lifetime to save money to start a business, if you are lucky. Take notice that all taxation is being shifted to workers to limit their chances of raising capital and creating competition.
Team Apple,
This is from someone outside the Apple ecosystem that management brought in and gave a lot of credence to.
I wanted to share a note of encouragement to say that everyone at NCMEC is SO PROUD of each of you and the incredible decisions you have made in the name of prioritizing child protection.
The recipients of this memo didn't make any of the decisions and those were foisted on them by management who in turn were heavily influenced by this person/organization. As to why management was so heavily influenced, I have no idea. This is an attempt to get buy-in from Apple employees I suspect, or at least to ease their concerns. "Don't worry, you are doing the right thing."
It’s been invigorating for our entire team to see (and play a small role in) what you unveiled today.
The team they are referring to are the ones that pushed for the implementation for child protection. The recipients did all the work because management told them to.
I know it’s been a long day and that many of you probably haven’t slept in 24 hours. We know that the days to come will be filled with the screeching voices of the minority.
This is the big quote. The author knew what this would do, and knows the tarnish the Apple brand is about to go through, but is trying to convince the audience that it's only a small group of people who will reject this and it will pass over. For the author, the ends justify any means because they are probably personally emotionally traumatized by child porn (rightfully so). Once this happens though, any rationality about consequence of action goes out the window. This is an attempt to get everyone concerned on the same level of ends-justify-the-means with the author. I don't know if the author believes this is just a temporary outrage or not. I suspect not. It doesn't really matter though the author got what they wanted, Apple brand be damned.
Our voices will be louder.
This obviously isn't the case. Not sure if the author means their organization or their organization + Apple. Either way, the counter resistance to this has been pretty minimal. They left Apple hang out to dry it seems.
Our commitment to lift up kids who have lived through the most unimaginable abuse and victimizations will be stronger.
During these long days and sleepless nights, I hope you take solace in knowing that because of you many thousands of sexually exploited victimized children will be rescued, and will get a chance at healing and the childhood they deserve.
This is the emotional sell, or "for the children." It's also a difficult bridge to reason across; comparing the possession of pictures and the manufacture of pictures as the same thing. Obviously the hash compare only finds existing manufactured pictures, so I don't see how it will protect children from any abuse that hasn't already occurred. Not only is this an emotional sell, it's a lie.
Thank you for finding a path forward for child protection while preserving privacy.
The final emotional pitch directed at people at Apple typically concerned with privacy. They aren't really protecting children and they also aren't preserving privacy. It's a boondoggle.
In closing, "for the children," is almost always a trap. It was used when I was a kid to mass incarcerate Americans that had drug problems, many of them the children they were protecting. It's typically done in an any-means-necessary heavy handed manner, as seen here. I don't condone child pornography in any way. I have never seen child pornography and hope to never see it. At the same time, what they are doing is the road to hell paved with good intentions seemingly by people who are so traumatized (based on their work) that they c...
Rather than fighting small battles to defend privacy, just take a hyperbolic extreme and use the backlash to bolster their actual position.
(or I geuninely think they have lost their f'in minds)
Now Apple’s just a follower…
Jobs loved the end of skeumorphic design or Jobs loved skeumorphic design?
https://www.businessinsider.com/steve-jobss-signature-design...?
Take all this with a grain of salt. I wonder if Jobs was just good at managing these brash personalities together.
The engineers who built, the project managers who coordinated, and the executives who signed off on this smoothbrain idea need to be drummed out without ceremony.
No severance, no golden parachutes, no cushy board jobs etc.
People need to understand they risk their livelihoods building and deploying a surveillance system like this. In a just world they would face criminal charges.
Google [1], Dropbox [2], Microsoft [3], and many many other Photo Cloud storage providers have implemented similar child porn detection.
[1] https://www.columbiatribune.com/story/news/crime/2021/03/22/...
[2] https://gizmodo.com/dropbox-refuses-to-explain-its-mysteriou...
[3] https://www.local10.com/news/local/2020/02/21/microsoft-aler...
This doesn't explain the insistence on scanning all images on device, whether they're hosted on iCloud or not.
The moment for leadership from Cook has come and almost passed. It's surprising watching Apple squander its brand at this moment. A good amount of its previous support with respect to federal antitrust has permanently dissipated.
They're only obligated to report if found, but they don't have to proactively scan (yet?). In the EU until recently they weren't even allowed to voluntarily scan.
This is the text of the current law [1]. Alerting authorities about CP is required if discovered. Actively searching for CP is not required. Look for the aptly named "protection of privacy" paragraph.
[1] https://www.law.cornell.edu/uscode/text/18/2258A
2. FBI/CIA, frustrated at previous failures, pressure senators/congresspeople to turn up the heat.
3. FBI/CIA then gives Apple the option to implement surveillance with the promise that they can make the antitrust troubles go away if Apple does what they want.
4. Apple does the thing. (We're here now.)
5. Antitrust case goes away.
The FSB, CCP, and other intelligence orgs might be trying the same strategy in their countries.
I expect this is how everything works in the big leagues.
https://transparencyreport.google.com/child-sexual-abuse-mat...
https://www.facebook.com/safety/onlinechildprotection
https://www.microsoft.com/en-us/photodna
https://blog.flickr.net/en/2021/04/16/keeping-flickr-and-chi...
Most companies dealing with photos have had this in place for almost a decade now. The FUD and slippery slope arguments are getting old.
Just because the bar is so low does not mean we have to accept privacy abuse.
No, why would I be happy with this? I actually prefer the alternative: scan photos server side, fine. That's Apple's computer and if I send my photos there, then I can't justifiably argue against that. But my device shouldn't be part of their attempt to appease law enforcement by turning the world into a dragnet.
Especially in this post 9/11 world - look at what was supposed to be "temporary" to deal with the "terrorists". Ha! You have to be an utter simpleton if you don't think use of this mechanism will expand. It's not even a leap of logic - it's been going on for 20 years all around us!
I'm sure 3 weeks ago no one could have fathomed that Apple would make an announcement like this, yet here we are.
https://support.apple.com/en-us/HT202303
https://m.youtube.com/watch?v=Af0gtsjfy7E
What’s your point? Apple said RISC was better and faster than Intel, until the day they adopted Intel, and then they said the complete opposite. They acted like they invented the Omnibox for Safari and so forth. It’s for marketing.
(This comment is intended as a joke.)
Thinking two steps ahead can be done even when you make mistakes
They better monitor every aspect of their customers lives to protect the children. Think of the children
It's fair IMO to use fit for purpose technology to target specific crimes.
Though I agree this is bad technology, not fit for purpose, and they should yank it.
It is also the only country from all of those I have been in almost 50 years, where great care is taken to never park in an illegal way.
Sure you will see it, away from city centers where no cameras are around.
This is only true for the masses. Rich people just leave their cars parked anywhere they want in Mayfair in London because they don't care about a 50 quid fine.
Although, to their credit, apparently[1] they do pay their parking fines.
[1] https://www.independent.co.uk/news/uk/home-news/us-diplomats...
Everything you said is already being collected about you. Cellphone towers can log where you are. Cameras around your city can recognize you. All they have to do is link all the feeds into one database that can be searched. The only place you can escape the watchful eye is your end to end encrypted stuff… which ain’t your SMS. And even there, you are just taking their word for it since it’s a closed source product.
Plumbing installed in people's homes should scan for illegally consumed narcotics or prescription medications and check them against pharmacy records. Your home's electrical wiring should make sure it's not being used to illegally grow drugs. Cellphone cameras should make sure they never record a naked body without a signed consent waiver. Guitar amplifiers should verify that you're not playing an owned piece of music without a license.
How could we ever survive in a world where we relied upon targeted investigations and probable cause? How could we ever live in a world where pipes just moved things from A to B without inspecting them.
/s
I would personally quit if asked to work on any of these teams.
If you were the QA team lead for this, how would you test it? I can't imagine anyone would do day-to-day testing with actual illegal images.
It would not surprise me if the dev and QA teams at Apple already have hash colliding images to run their tests?
This is all awful. Steve Jobs must be rolling over in his grave.
You select any random image as your test image, marking that image as CP in the backend and then try to upload that image from your test phone.
They didn't even last 10 years since his death. :/
June, 2010 - Jobs: "Privacy means people know what they're signing up for."
https://youtu.be/39iKLwlUqBo?t=154
December, 2015 - Cook: "If you put a back door in, that back door is for everybody, for good guys and bad guys."
https://youtu.be/rQebmygKq7A?t=66
August, 2021
Interviewer: "Isn't this in a way a back door?"
Federighi: "I really don't understand that characterization."
https://youtu.be/OQUO1DSwYN0?t=426
1: Run on-device code to perform perceptual hash comparison of each photo against an on-device encrypted database of known CSAM hashes.
2: On iCloud Photos servers, send out the relevant notifications when a user’s iCloud Photos account exceeds a threshold of positive matches.
So as a high level testing strategy, I would want to:
- Verify on-device lookup of CSAM hashes. This could be tested by provisioning a test device with an on-device database containing CSAM hashes of images that aren't illegal. As a bystander, I think I'd be fairly confident with this approach because I'm guessing the on-device database that Apple ships could conceivably be changed over time to expand the definition of the images it will flag as CSAM.
- Do some exploratory testing to discover the threshold of how much image manipulation can be done on a flagged image before the perceptual hash comparison fails to return a match.
- Verify that the notification system notifies the correct parties once a user account exceeds the defined threshold of positive CSAM matches.
- Ensure the flagged account can still be investigated if user deletes the offending material from iCloud, or their account by the time a real person gets around to investigating.
- Ensure that the logging is informative and adequate (contains device name, timestamp, etc.).
- Test behaviour on same iCloud account logged in to multiple devices.
- Figure out any additional business logic - are positive matches a permanent count on the account or are they reset after a certain amount of time?
source: https://www.apple.com/child-safety/pdf/Security_Threat_Model...
[1] https://www.law.cornell.edu/uscode/text/18/2258A
The intent of the laws making the possession CSAM illegal are ultimately to stop the spread of CSAM. If those laws fail in their stated purpose just because a company chooses not to be proactive in how it searches for the material, that would be missing the point. The media and governmental pressure would wipe any company foolish enough not to implement child safety measures out of existence, but no major tech company is that foolish (except Kik perhaps). And hypothetically, the government could just pass a law enforcing proactive scanning if there were to exist a company that chose not to proactively scan, which would mean that the government would fail to solve the issue of preventing the spread of CSAM, and thus CSA.
But in practice, such a law isn't needed. No company wants to be derided as a service that allows child abusers to get away with their crimes.
—Nothing in this section shall be construed to require a provider to
—(1)monitor any user, subscriber, or customer of that provider;
(2)monitor the content of any communication of any person described in paragraph (1); or
(3)affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b).
Hrm, that's interesting. I wonder if Apple iCloud has surfaced in enough cases as where criminal activity is actively taking place that it prompted Apple to do something about it.
As could PhotoDNA, though, which everyone has been using for a decade to scan cloud photos - but it doesn't seem to have happened.
CSAM is one of the most extreme crimes against a person and the number of people sharing and profiting from CSAM is growing. This is something every tech company will have to deal with.
Really? Based on what? Multiple stories of hash collisions (false positives) are *already* popping up.
If only all the people who think this isn't a big deal were the first to get hit by false positives. Now that would be poetic justice :p
After advocating to all my friends that Apple is a beacon of how privacy should be done, I just can't understand how they've made such a hash of this.
Upd: I am not an expert, but maybe the schematics could help you here: https://source.puri.sm/Librem5/l5-schematic and https://wiki.pine64.org/index.php/PinePhone#PinePhone_board_....
- Who writes and maintains the firmware the OS is running on.
- Who writes and maintains the modem firmware and who can update it.
- Who can update that firmware for the board the OS is running on. This could be a different answer than who initially creates it for the retail distribution of the phone.
- What level of trust has been inserted into the OS by kernel modules and who maintains those kernel modules.
- What control is given to the end user to see what those modules are doing and limit what they can do.
I suspect more questions could arise as kernel hackers audited the phone. The dilemma I see is that such kernel hackers won't be interested until those phones are wildly popular. The only other way to get their interest is with money.
Both Librem 5 and Pinephone run FLOSS operating systems with software maintained by Purism and community. The latter smartphone has two firmware blobs in kernel for WiFi/Bluetooth AFAIK: https://www.pine64.org/2020/01/24/setting-the-record-straigh....
> Who writes and maintains the modem firmware and who can update it.
See my link above concerning the Pinephone modem. It's software can be updated by the OS. AFAIK it's the same for Librem 5.
> Who can update that firmware for the board the OS is running on.
See the first answer. Librem 5 is going to get FSF certification "Respects Your Freedom". Proprietary software only runs on the modem and WiFi card. See also: https://puri.sm/posts/librem5-solving-the-first-fsf-ryf-hurd....
> What level of trust has been inserted into the OS by kernel modules and who maintains those kernel modules.
I am not knowledgeable enough to answer that. Maybe the schematics linked above could help you.
> What control is given to the end user to see what those modules are doing and limit what they can do.
User has full control. This is a selling point of Purism the company: https://source.puri.sm/Librem5/community-wiki/-/wikis/Freque...
Can you imagine this "narrative" applied everywhere? From the moment you open your eyes everything around you is extracting data, monitoring and policing you.
We have to redefine our relationship with software and hardware (electronics). The infrastructure (5g) and IOT advancements are here. Data is the fuel of the coming "consumer revolution". Apple just positioned themselves perfectly as a Data Broker for high level customers - special groups, oppressive governments, you name it.
They will not back up from this. This is more profitable than selling shiny toys and "services". This gives political elites a reason not to break Apple monopolistic business practices.
I will go even further: This gives Apple a favorable position to become a trusted partner of governments of the World.
Deconstructing this only from a technical point of view is non-efficient. Apple is making an entrance on political stage by embracing the role of World Digital Police.
Everyone is free to apply his own conclusions and analysis. Mine are proportionally adequate. I have removed macOS from production workflow. I will never buy another Apple product. And I am willing to throw away "conveniences" and polished UX for preserving my privacy.
This privacy, not Apples corporate redefinition:
The quality or condition of being secluded from the presence or view of others. The state of being free from public attention or unsanctioned intrusion. A state of being private, or in retirement from the company or from the knowledge or observation of others; seclusion.
https://www.youtube.com/watch?v=z15JLtAuwVI
https://www.apple.com/privacy/
Here's an article from a year ago on EARN IT:
> Theoretically, a system that uses client-side scanning could still send messages encrypted end to end, and so the Leahy amendment would not offer any protection, but many of the same confidentiality concerns with backdoored “e2ee” systems would continue to apply.
Source: https://cdt.org/insights/the-new-earn-it-act-still-threatens...
It's all about conditioning. First it's to save the children. Then it will be about preventing the terrorists. Finally it will be about preventing "hate speech" - which these days hate speech is pretty much speech from anyone you disagree with :p
Like the 1984 novel, Minority Report wasn't an instruction manual!
> More broadly, they said the change will break end-to-end encryption for iMessage, which Apple has staunchly defended in other contexts.
But... it wouldn't. The iMessage feature doesn't expose the contents of your message to anyone else under any circumstance.
If you're a child under 13 and your parents have opted in to this feature, you get a choice of seeing naked-pictures sent to you and having your parents be notified that you chose to, and not seeing it with no notifications of anything. (But once you're 13+, no notifications would occur either way.)
There are potential issues with this, mostly relating to abusive families being controlling. They'd have to do weird things like forcing their teenaged children to keep making new under-13 accounts to actually take advantage of it like that, though. And none of these issues impact the e2e status of iMessage in any way.
Apple really screwed up PR by launching the iMessage feature alongside the scanning-and-reporting iCloud Photos feature. There's so much confusion out there about this.
(The breaking-e2e aspect does exist with the iCloud Photos scanning... not that it's currently e2e, of course.)
There's no e2e encryption for Photos currently, so it can't be breaking it there. If you use iCloud Photos, your photos are stored encrypted, but Apple has the key and could be doing all the cloud-scanning it felt like.
I'm sure in such a future there'd be quite the argument happening about whether you could call it e2e.
Let's not pretend that this sort of thing is happening in isolation. There really is a big legal movement towards requiring encryption backdoors, and programs like Apple's here are effectively part of a big negotiation with law enforcement -- note that the reason Apple's currently not e2e is apparently that the FBI pushed them on it[1]. "We'll give you what you claim is important, CSAM scans, and in return we'll lock everything else up securely". Is this great? No. Would an absolutist "law enforcement can have nothing" position backfire into stupid legal decisions? Plausibly.
[0]: https://en.wikipedia.org/wiki/PhotoDNA [1]: https://www.popularmechanics.com/technology/security/a306318...
Content is no longer private if there is anything inspecting it other than you.
I expect cloud providers to fiddle with my data - because they can and do. I don't expect vendors, especially one touting themselves as being more privacy focused than everyone else, to be sifting through my data on device.
Yes, today it's a carefully designed algorithm designed to be one way. The algorithm isn't the problem. The problem is the method used to deliver this algorithm can also be used to deliver *other* algorithms. And no, especially since 9/11 I have zero faith that this is as far as this mechanism will ever be used.
The iMessage feature is "it looks like someone has sent you a picture of their dick, are you sure you want to see that?" I know adults who'd like to turn that feature on.
Could it hypothetically be extended in the future? Sure. But arguing against this feature as-is is about the same as arguing against the feature that lets you search for "dog" in your photos.