As usual, they trot out "think of the children!" whenever they want to start doing something horrifically invasive or trivial to abuse. Just what we want: social standing determined by YouTube algorithm.
If it's private and/or sensitive, it probably shouldn't be stored with a cloud-synced service.
Recently I've seen a lot of "experts" trying to reassure the non-technically oriented about the fact that it's not true that "the cloud is not someone else's computer", that that's just a meme and it's not accurate. But they never thoroughly explain why that wouldn't be the case.
When you dig deeper, it turns out that more often than not those "experts" are people whose job is to sell cloud services, or who work for companies that sell cloud services.
"Chrome/ChromeOS/Android" doesn't work when Android is a completely separate piece of software from the other two, and only Android/ChromeOS have any sort of file syncing to Google servers thus meaning no PhotoDNA scans if you only use the Chrome browser.
Chrome for Windows already scans local files for "unwanted software" (using ESET’s technology), other kinds of hashes could be easily added there:
>As applied in Chrome Cleanup, ESET’s technology is used by Google to alert users about unwanted or potentially harmful software attempting to get on users’ devices through stealth, for example, by being bundled into the download of legitimate software or content. Google Chrome, using ESET’s security technology, then provides users with the option to remove the unwanted software. Chrome Cleanup operates in the background, without visibility or interruptions to the user. It deletes the unwanted software and notifies the user once the cleanup has been successfully completed.
What do you think the public reception would’ve been if this technology had been available last year, and multiple jurisdictions published hashes of images from the riots at the capitol?
Or maybe, more directly, what do you think Apple employees reviewing these hits would do if those types of images started showing up in the triage list.
It would achieve exactly what the system is designed to do, mine personal devices for photos that authorities are aware of. Just because a lot of people were sloppy doesn’t mean every person was.
However, my primary point is that there is no possibility of a technical control over the scope of the program. It’s a policy decision and is subject to explicit expansion or collusion/parallel construction.
You can quickly build relational graphs if you know which devices contain media that shares hashes. Sure, theoretical danger and not how this is implemented as hashes are just compared locally. This is different from accessing peoples contact lists they upload anyway because you want to group people in information bubbles that not necessarily correspond with real life contacts. This is very valuable info, isn't Google trying something similar?
> Apple does not report you to law enforcement, as this article claims. The article also ignores human review.
They do report you to a private organization who is primarily funded from the US Government and has zero interest in not passing along any information about potential child abusers.
If you believe Apple, there will be a human review element. What sort of people would go after this kind of job? I can't imagine. And this group will also have a huge incentive to err on the side of reporting any potential abuse content.
Edit: A legitimate, almost definitely "wants to the the right thing" private organization in the US. Have we heard about who gets notified in other jurisdictions?
> There are human reviewers at Google, Facebook, and every other organization that has to moderate content. Not a novel idea.
For unencrypted content uploaded to their servers. That makes it a "Them Problem". Apple's approach is to scan before it is uploaded (and like the article says, "for now" - this can change with only a configuration change for all we know). And this could be a per-jurisdiction option and again, no one would know.
You have a point that this is not completely novel problem but do we have actual data on the number of real vs. false positives on the "everyone but Apple" group? Matthew Green probably would know but he doesn't[0].
There is an website[0] set up already to show to show how easy it is to find collisions with a photo of a beagle.
The left side photos are all apparently kitten photos but what if it was a photo of the Nirvana baby and, as a human reviewer, you had no access to the right side photo because it was just a hash in an anonymous database?
The Nirvana baby, as with many non-sexual pictures of naked babies, isn't considered CSAM and thus wouldn't be flagged. Even if it was, NCMEC reviewers (and Apple reviewers) almost certainly view the original CSAM next to the new photo when they receive potential re-circulations of the photo, if their task is indeed to verify the authenticity of a match. If reviewers had to make a CSAM judgement call on every incoming image, it'd be an extremely brittle system.
No one would know? iOS is constantly analyzed by people around the world for security vulnerabilities, strange behavior, etc. Why would this be different?
My concern is that those mechanisms that regularly fail in other domains, as we have seen time and time again in recent years, would also fail in this case.
Are you able to explain why this specific case would be different?
As your link states, its “law enforcement powers” are basically that it’s allowed to handle CSAM. NCMEC isn’t going to arrest you and charge you with a crime.
This is a distinction without a difference, because the FBI, US Marshals, and others have agents who work full-time only with NCMEC, literally in the same office building.
> Representatives of multiple law enforcement agencies have offices in the NCMEC building, including the FBI, Department of Homeland Security, U.S. Marshals, U.S. Postal Inspection Service ...
There seems to be several misunderstandings in this article. For instance:
> Your phone will run your images through NeuralHash, and compare to the list of forbidden hash codes. If there’s a match, some data will be uploaded to Apple’s servers when you upload the image.
Your phone doesn’t know whether there is a match or not. There is no “if match then upload”. The safety voucher is attached to every iCloud upload, not just the matches.
This is why Apple can’t just flip a setting and scan all your images regardless of whether you add them to iCloud or not. If they wanted to run this against all your images, they’d have to either upload all your images to iCloud (easy to spot; difficult for them to do without consent) or change the way this mechanism works (not “just flipping a setting”; if they wanted to scan all your images, then why build it this way in the first place?).
Technically, but not substantively, true since they report to the NCMEC, a government-funded government-established “private nonprofit” that is delegated a whole bunch of jobs for government, including gathering info on CSAM and distributing it to law enforcement for DoJ and processing child return/access requests under the Hague Convention on the Civil Aspects of International Child Abduction for the Department of State.
It might be a good idea not to travel outside your home country with an iPhone or other Apple tech, because you don't necessarily know what kinds of photos or other digital files a country's government would object to. The picture of the man against the tanks is a good example.
Indeed, you don't necessarily know what photos are on your phone.
The less gadgets you bring across the border the better. In New Zealand for instance a customs official can now force you to unlock a device on your person for "suspicions" no matter how vague. The only way around it is to refuse to play and leave everything at home.
A few years ago a little girl went missing and unfortunately was later found dead in a nearby lake. It was determined that she had been abducted and killed. The local news station reported that there were approximately 15000 pedophiles in the surrounding region.
I found it incredible that there were so many deviants known by the police in my community, they eventually caught the killer. He turned out to be a neighbor and confessed.
I think the police are well aware of every download/upload of child porn or are atleast they are able to look it up when the need arises.
The time for concern of protecting ones privacy on any digital connected device has come and gone, best to assume anything You do on your device is readily available for the authorities(and others) to exploit.
*it was 21 years ago, and the only news report I found says 100 known but I am positive they said 15K.*
It is completely trivial to render your communications opaque to casual surveillance. If someone were truly committed to surveiling you it would be drastically harder. At the point where people are hacking your devices and watching you from across the street impossible.
About 1/2 of 1% (estimates vary) of people are sexually attracted to children and an unknown number commit acts against children. Statistically if your kid gets abused it will probably be by someone you know and shouldn't have trusted.
You should probably have a healthy mistrust regarding men alone with your kid and the government without being paranoid.
>>>It is completely trivial to render your communications opaque to casual surveillance...
Im not sure I understand what You mean by this, my thought or possible concern is that all web traffic, especially that which goes through known proxy addresses is being logged and flagged by local carriers and ISPs. Then in the event of a crime being commited in my area, my actions online including stuff from decades past will invite a knock on my door by the authorities or worse the media.
I must stress, Im not worried about anything sexual but have done a great deal of research on some other subjects that are not exactly above board.
I can appreciate the concern regarding apple scanning personal files and reporting matching files to the government, but feel the odds of a virus, vulnerability or exploit being used by a criminal ngo is just as big if not bigger risk and has been for a long time now.
Anyways, like I said...for a while now I have been of the mindset that anything I do on a connected device (even temporarily connected) leaves me exposed, now or in the future. Maybe cancel culture has encouraged my current way of thinking.
Pop!_OS (no Ubuntu telemetry and OpenSnitch) over MacBook Air is working flawlessly. Company MacBooks are with Arch/Manjaro. Will run Catalina for specific tasks without internet until it breaks completely.
The hostile design decisions of Apple are unbelievable.
After 20 days of public backslash Apple is preparing for massive iPhone 13 campaign and pushing this obviously flawed intrusion with "common good" goals, normalizing surveillance on a new level. And all of this after Pegasus/NSO.
But actually I am glad. No more shiny toys and marketing illusions for me.
Expect everybody to follow suit. Year or two down the road this type of scanning and control will be mandatory.
So the only way to respond is to invest in your privacy. Not in user-hostile, dark pattern designed products.
After three years of carrying my iPhone XS with no case with no problems, I dropped it several times in the last thirty days. It's about time for an upgrade.
Happen to have any suggestions for what a privacy-conscious consumer should look into?
Personally I have moved my habits in different direction.
If I need a photo - I grab my old Sony RX. If I know that I will shoot - my old Leica MP240 with Summilux and 24mpx is enough.
If I have to communicate with text - email or signal on computer. If I need in car navigation - Garmin.
Deleted Facebook/Instagram spyware long time ago. Flickr also.
Self-hosting is the way to go.
I am using my iPhone for calls and sms with clear knowledge that everything is recorded. Banking apps when I don't have access to computer.
I have iPad Pro 12"WIFI with faulty WIFI and no Apple care which I use only for drawing and digital painting.
People around me are switching to deGoogled smartphones with Lineage/Calyx.
We are infected by cancer. You can read the arguments on the article at the front page today about the abuse of juvenile delinquents by their wardens, and you can read the comments complaining that that the lack of sexual agency given to the prisoners was somehow itself a crime. I was at Facebook while their CSAM monitoring and warrant response group was defunded by a person heavily into "ageplay". Jeffrey Epstein was killed in prison while the cameras were turned off. These are not isolated, small incidents. There's a distinct undercurrent of pedophilia brewing in American society, and people are starting to notice. Tim Cook has to bend the knee, lest his own monstrosities are noticed.
> Expect everybody to follow suit. Year or two down the road this type of scanning and control will be mandatory.
Indeed, this article (despite many incorrect facts about the process) is right in that this is just a vocal minority 'screeching' to get the message out there to anyone willing to listen. The absolute majority of the population, or at least adult population, will always choose the path of least resistance - they just wants to live, work, avoid hardship, and partake in dopamine-inducing activity every day. There's a reason Google and Amazon are trillion-dollar enterprises and Hulu still has an ad-supported version that 70% of people choose[0] - people don't value privacy at all when dealing with these faceless companies. Society has always shifted the level of privacy they accept (think taxes, audits, search & seizure with probably cause), and this will just be another shift that people accept so that they continue on with their lives while not sacrificing any of their valuable time.
The hulu number was quite surprising to me, given that the ad supported tier is $6/mo vs $12/mo for ad free, and if someone watches enough the extra $6/mo would seem worth it.
It might have a lot to do with their $1.99 for one year promotion they do on Black Fridays though, which would change things considerably ($2 vs $144 per year), especially for households without enough disposable income. Easy enough to switch to a different person every year and keep it going for a while, and also at the time of that article it had been less than 3 years since the free tier with ads was removed.
I don't think privacy has much to do with it though. People that don't have any idea about these topics just wouldn't care, and at least in my case I care but I'm under no illusion that paying for ad-free changes anything other than the convenience of not having ads. iirc hulu's ads cannot be skipped anyway, so they pretty much can collect the same exact information from both tiers.
Ad-supported is a distinct concern from "level of privacy", though. Google (et al) have done a lot to muddy the waters and keep telling us that ad-support "requires" panopticon levels of tracking and no privacy, but centuries of advertising history before that existed with little to no individual tracking and/or where is was maybe physically impossible (in magazines, newspapers, broadcast TV).
Personally, I have no problem with ad-supported and generic/no-tracking. Ads themselves are not the privacy invasion, and have a reasonable history of helping to pay for things, and it is possible to rebuild ad companies to respect privacy (again). (It might take burning a few big players to the ground first, unfortunately.)
> The hostile design decisions of Apple are unbelievable.
Hostile for sure, unbelievable, not really.
This type of "we know what's best for you" attitude is deeply ingrained in the DNA of Apple as a company.
The fact that they did get it right many time and that their batting average is above other tech companies is no excuse for the fact that the attitude is deeply flawed.
Agree. As an 20+ years Apple user, I have suffered some big annoyances.
You are holding it wrong.
Here we go, we are removing "escape" but you have a touch-bar with customizable functionality, sorry this is the best laptop keyboard design - you will adapt to it.
We are removing ports - but you will have Dongles - lot of dongles and this MagSafe shit that you love so much - no more for you.
Innovation my TrashCan - not upgradable peace of trash, after 6 years - here we go MacPro - only for the Apple "elite hustler" club members, as a bonus 1000 dollars monitor stand.
We don't like MacOS - to much control for the user is not good for Apple Profits - we like iOS and we will make everything like it. You will adapt.
Now we are innovating on a geo-political stage, protecting our brand with the all time classic "Neural Cash" and bending a knee or two in the name of Apple dominance and antitrust avoidance. You will adapt, because you are trapped and we know you love us.
It is an interesting counter-point to Apple's previously selling itself as the only company to respect your privacy.
This kinda flies in the face of that except for the fact it's wrapped in, as you said, "common good" goals.
I've never been Apple, but I'm currently attempting to migrate to a LineageOS daily-driver mobile, although I still require some amount (nano) of Google to get by. Ironically, I needed a Windows machine to do this, whilst I'm already almost exclusively Linux (Ubuntu and Pop!_OS).
A company that does not listen to its customers. Apple is different from the "tech" companies it seeks to compete with because its primary customers are not advertisers, they are Apple computer purchasers. It makes some sense that "tech" companies ignore feedback from users; users are not their customers and user privacy is antithetical to their core business of surveillance. However when Apple ignores its customers (users) it makes little sense.
Maybe only a minority of Apple customers are aware of and understand the scanning issue and are giving feedback to Apple via the web. But here is a thought experiment: Does anyone think that if the majority of Apple customers were briefed on the issue and then asked if they agree/disgree with the minority of customers would they disagree with this informed minority.
"Tech" companies should know whats best for advertisers. They can be expected to ignore user feedback complaining about surveillance. OTOH, Apple should know whats best for users. Its users are telling the company and the company is irgnoring the feedback.
The core problem here is reporting to prevent CSAM (hopefully, it is uncontroversial that doing so is desirable) but rather that this makes on-device snitching on any topic a matter of -policy- rather than -technology-. It is much harder for a government to say, you must implement this large and complex feature, you must keep it secret, etc - especially a foreign government - than it is to say, here are some new hashes, please plug them in to your database that four people have access to. Or to say, please scan all images instead of just those uploaded. Even if Apple wants to right by its users, it is making it much easier for Apple to be compelled to do wrong.
“We promise” is easily forgotten. “We can’t” requires dev teams to change.
I agree about how the technology can be applied to other instances, but I feel that people who are criticizing Apple are crucially placing all the blame on Apple but not on the government that pressured them into doing this in the first place. This CSAM issue is proof that if even the most valuable company in the world can be pressured into compliance, then any company in the world could be, calling into question whether it is realistic in the first place to expect companies to act ethically while also creating highly profitable products.
For all the things that have been said about the issue, I don’t remember a single comment that has been directed at the US government, or China itself, or any discussions about how nation-states should be run and genuine introspections on what could prevent surveillance states from existing in the first place.
This is because people, for some reason, see Apple's 3-trillion-dollar valuation as being some purely self-made fortune to the point that Apple could rival these governments with their technology stance, when in reality Apple's success has only been enabled by the governments that house and profit from Apple's dominance - and now they're coming for their share of surveillance instead of dollars.
America is kneeling down to the same playbook that authoritarian regimes use, just a more vanilla version of it. Ready to be exploited one day. I am happy that the tech community on HN and elsewhere are speaking up about this. I was expecting it to be just another "think of the children" theatre ready to be swept under the rug. Major newspapers are talking about it. If anything else, next time they say the word "Privacy" at the Apple WWDC, we should boo that non-sense and let them pay the price in PR-damage. Apple spent last 3-4 years building their image as privacy-pro alternative to Google/FB/Microsoft in various product/service categories. They compltely fucked up. Privacy is about trust and it takes a single event to forever destroy it.
If you have any proof that'd be nice. Just because there's a microphone in the room, such as via a flip phone or even rotary phone, doesn't mean it's actively recording audio to be shipped off for later blackmail/extortion/embarrassment.
Echo devices' microphones are turned on all the time, unlike a flip/rotary phone, whose mics are physically disconnected when the handset is in the cradle.
And? Just like Alexa, on a flip phone the software is the only thing preventing the microphone (which is never physically disconnected) from sending audio to some other device/person.
The argument those points are supporting, as the article explains, this is an ineffective way to fight CP, but it's being done anyway, and can be used for nefarious purposes.
Could a government not have a different level of interest in law enforcement over CP vs, say, political material that undermines government authority?
Indeed; I'm not sure this premise holds. I personally don't trust Apple's promises on privacy given its actions in China shows that it values market access more than its privacy values. But I find this argument weak.
> That’s so that criminals can’t just disguise the image with a trivial change.
It's utterly trivial to modify the image to avoid matching. Most minor modifications will do so.
The purpose of the hash is so that idiotic criminals that aren't trying to avoid getting caught don't accidentally avoid getting caught just by resizing or recompressing the images.
The flawed 'hashing' function also creates plausible denyability: If its somehow proved, perhaps via a whistleblower, that Apple was in fact matching the historic tank man photo, Apple or the government could easily hand over a piece of child porn with the same neuralhash due to the extraordinary vulnerability of the function.
Don't know why you are downvoted because I think this is still true for the most part that you should be able to trivially create another hash for the same image with unintrusive operations. You might need to put some efforts in it and reverse engineer the hashing algorithm a bit and see what causes rolling a new hash value.
A lot of research went into classifying images, mostly from the side that wanted visible or invisible watermarks in pictures. It is a cold war against image cropping, compression and modification that tends to not favor the classification side.
But here you only want to have a different hash, the operations for that are numerous and probably indeed often trivial.
I don't think you could just have a completely different picture create a collision though. Maybe they can because it is their hash-function, but that would be quite unusual and I doubt this is the case.
Where I post good looking examples of standard test images altered fairly subtly to give the specific hashes.
The apple neuralhash is broken as a 'hash function'.
It's much much easier to modify images to just have a different hash. A simple blemish on the image-- or, with whiteboxing using the hash function, no visually noticeable change is required at all.
That is seriously the hash function they used? I assumed a competent hash function, hell, any common hash function is better even if they fail at resize/compression. This one would too.
Apple 'neuralhash' usually survives resize (the first step it does is resize the image to 360x360) or minor compression lossy changes.
But the price of that is that it's not secure against generating arbitrary preimages based on fairly arbitrarily targets, with results that look nothing like the original image you're matching (if one even exists).
They could do some downsample, transform, quantize, sha256 approach that would still survive some tiny amount of differences. ... but apple favors easily constructed false positives over false negatives, even though its trivial for anyone trying to avoid their function to get a false negative. Their scheme only protects against accidental false negatives. It's just a symptom of systematically favoring factors other than the privacy and security of the actual owner and user of the computer.
Wow, that is seriously a naive approach. Honestly, I would have expected much more here. That makes the libraries of hashes for content almost useless. Hope they extracted features that could be used in a more sophisticated approach.
"If you’re right wing, look at how privately taken video and photos have been used to track down the January 6th Capitol rioters"
How is that supposed to appeal to right wingers? He thinks all right wingers want those rioters to escape justice? Most republicans oppose them according to several polls. The author may be a victim of political divisiveness due to paying too much attention to social media and news.
In fact, Democrat protesters occupied the Capitol building in 2018. I'm not aware of a manhunt for those protesters.
> Most republicans oppose them according to several polls.
I suggest the poll be re-run after you tell people that the MSM narrative was a lie - protesters killed nobody, and police opened the doors. Nobody arrested had a gun.
AOC has told 3 different stories, yet she was a mile away at the time.
In fact, there was no "insurrection", confirmed by the FBI last week. But there was a Marxist-style narrative - they're very clever with wording, aren't they?
Why didn't Pelosi add more security for 100,000 protesters? That's the only fact we really need to know, isn't it?
It does show an ignorance of how the right wing thinks. However, it is directionally accurate - being an effective right wing leader is likely to get someone's phone targeted.
Not a partisan thing; effective left wing leaders also have their phones targeted. The swamp doesn't like change.
I thought with enough matches Apple would decrypt your images, at which point they would be able to verify if they were CSAM (and notify authorities) or realise they weren't and that a government had tried to trick them (and do nothing or go public with this information). Is this not how it works?
That is how they've described the system. After 30 or so matches, they'll be able to decrypt a version (presumably scaled down, the description is vague) of the images and manually review them. With the present system targeting CSAM imagery, unless the images you have match their expectation of matching images, they wouldn't pass it on to law enforcement.
This is where the slippery slope argument comes in. Apple would have to actively cooperate with governments to both detect and report other kinds of images. So if Tiananmen Square images were showing up in the review pile, Apple would have to know to pass those on to the Chinese government and be willing to.
The only other way they'd be passed on is if Apple drops the manual review portion.
Not quite. Any manual review system is going to have infrastructure to pick reviewers, route the data around, etc. it would not be hard, and could be done under the radar, to intercept data for specific targets in a way that wouldn’t arouse suspicion, and it would be awfully hard to prove it had happened too.
So you're agreeing with me. Apple would have to actively cooperate by providing reviewers who would go along with the kind of media besides CP that nations may want to detect, or Apple would have to actively cooperate by simply passing matches on and bypassing the review process. But Apple would have to be the ones doing it either way.
You know what I hate? Titles that say specific words that are never repeated in the article.
A title that has so little to do with the contents of article, that it might as well be a random reference to some throw-away line in some comment somebody uttered sometime, which might as well be left alone so a proper title could be selected.
The "screeching voice of the minority" is that which Apple is catering to in it's image scanning, and since the image scanning is what the article is about, then I don't have a problem with it.
Normally I would agree, but I'm fine with the subtle(?) implication of this one.
That's not what the title is about. The title is establishing the author and the article as part of the "screeching voice of the minority", a reference to a press release that dismissed concerns about the system as, essentially, being unimportant and which used that phrase to dismiss, out of hand, opponents of the scanning system.
After the Pegasus news that showed that a relatively small private company in a very small country could have complete remote access to your phone and sell it on the market, do you really think that a powerful government like China or the US can't already do it??? How could you be this naive? Especially after the Snowden revelations.
And about Apple's having a change of hearth in the future and using something like this against you. Have you stopped to think for one second and consider that they have complete control over the updates that your phone receives??
They can quietly do what ever they want without you ever knowing and you can't do sh*t about it. So drop the concerned citizen act. We all abdicate privacy a long time ago when we sold our souls for cool free services like google and sexy gadgets.
At the end of the day the "slippery slope" argument boils down to one thing: do you trust your institutions? Any law or service can be abused, and the only thing preventing that is the integrity of the people on your society and the institutions both public and private that they work for. The exact same law that is used to fight corruption in a great country like Norway or Denmark can be abused to persecute political opponents in a shitty country like North Korea or China.
If the answer is no, I don't trust my institutions and my people, then you have a far more severe and fundamental problem than an specific law or service. In that scenario, focusing in an specific issue is like trying to cover the sun with your hands.
These whataboutisms are a distraction and in no way support the idea that people shouldn't be worried about Apple implementing a new client-side system that's ripe for abuse.
If things are already bad, we should be working towards making them better instead of rolling over and apathetically allowing them to get worse.
If we don't speak up when they push the boundaries they will just continue pushing. The only thing stopping Apple from doing all of those bad things today is that they are afraid of bad PR. You should be happy that others speak up now rather than you waiting until what they do becomes bad enough that you feel the need to speak up against it.
I think the disagreement is whether this actually is pushing a boundary. To me, this is roughly identical to the ways that services have been checking for illegal material for years and years. All of the threats are the same.
There's nothing childish or hysteric about caring about principles.
> How could you be this naive? Especially after the Snowden revelations.
You make a good argument, but you're wrong to dismiss the slippery-slope counter-argument. This move by Apple is normalizing your own devices betraying you. Like if door locks all had a master key the police could use to open them. It doesn't matter that they're easy to lockpick, what matters is the principle, because that is what stops you from taking the first step down a very dark road. And every step along that road will be portrayed as the "first".
As for "do you really think that a powerful government like China or the US can't already do it???" - this argument can be used to dismiss any effort at preserving privacy.
Complete hogwash. Not everyone sold their privacy. There were and are loud and clear voices that warn against closed ecosystems. Any company is open to blackmail or pressure.
Also I don't trust institutions. This is axiomatic and why separation of power is essential. So
> the only thing preventing that is the integrity of the people on your society and the institutions both public and private that they work for
is plainly wrong in my opinion. If you let people abuse this, they will. Also I don't think that can meet with reality. Police departments alone have quotas they must match, so they have to find criminals... It doesn't take a genius to figure out the repercussions. But it is still practiced in many countries.
Separation of powers is plainly ignored for everything under the guise of safety and a disease infecting politics the last few decades. That is due to incompetent leadership. Another reason why you should not trust "institutions". Instead, you check them and burn their asses if they fail as often they do today. The reaction that trust is falling in press and public officials is completely rational on the other hand.
I still have the impression that Apple is trying to do the privacy-supporting best option in their toolkit. The FBI has asked them multiple times to provide a generic back door and Apple has refused to give them general access and asks for specific warrants. So the FBI comes to them with a lot of very specific warrants about child pornography and they build a system specific to that use case and only that use case.
There's definitely still a "slippery slope" argument here that in handling this specific use case they are going to feel more pressure to handle other specific use cases of government actors, but they are also setting the precedence that they still will not support a generic backdoor. That seems to me, at face value, a greater privacy win than a loss.
98 comments
[ 1.1 ms ] story [ 181 ms ] threadSame with Microsoft. And Dropbox. And…
Recently I've seen a lot of "experts" trying to reassure the non-technically oriented about the fact that it's not true that "the cloud is not someone else's computer", that that's just a meme and it's not accurate. But they never thoroughly explain why that wouldn't be the case.
When you dig deeper, it turns out that more often than not those "experts" are people whose job is to sell cloud services, or who work for companies that sell cloud services.
>As applied in Chrome Cleanup, ESET’s technology is used by Google to alert users about unwanted or potentially harmful software attempting to get on users’ devices through stealth, for example, by being bundled into the download of legitimate software or content. Google Chrome, using ESET’s security technology, then provides users with the option to remove the unwanted software. Chrome Cleanup operates in the background, without visibility or interruptions to the user. It deletes the unwanted software and notifies the user once the cleanup has been successfully completed.
https://www.eset.com/int/about/newsroom/press-releases/compa...
The article compares scanning to facial/object/scene recognition, which it can’t do.
The article talks about making it easier to find who took a photo, which is not the problem the Jan 6 sleuths were trying to solve.
The article worries about China, but ignores that matchable photos must appear in multiple jurisdictions’ databases.
The article also ignores the several points of third-party auditing and several layers of human reviews.
Or maybe, more directly, what do you think Apple employees reviewing these hits would do if those types of images started showing up in the triage list.
The photographers posted them publicly on Facebook and Twitter. That’s how we have them in the first place.
The problem wasn’t knowing who took the photos, it was knowing who was in the photos. And that needs facial detection, which this system can’t do.
However, my primary point is that there is no possibility of a technical control over the scope of the program. It’s a policy decision and is subject to explicit expansion or collusion/parallel construction.
They do report you to a private organization who is primarily funded from the US Government and has zero interest in not passing along any information about potential child abusers.
If you believe Apple, there will be a human review element. What sort of people would go after this kind of job? I can't imagine. And this group will also have a huge incentive to err on the side of reporting any potential abuse content.
Edit: A legitimate, almost definitely "wants to the the right thing" private organization in the US. Have we heard about who gets notified in other jurisdictions?
And why would the NCMEC give law enforcement info about people they know are innocent?
For unencrypted content uploaded to their servers. That makes it a "Them Problem". Apple's approach is to scan before it is uploaded (and like the article says, "for now" - this can change with only a configuration change for all we know). And this could be a per-jurisdiction option and again, no one would know.
You have a point that this is not completely novel problem but do we have actual data on the number of real vs. false positives on the "everyone but Apple" group? Matthew Green probably would know but he doesn't[0].
[0] https://twitter.com/matthew_d_green/status/14283829528580833...
The left side photos are all apparently kitten photos but what if it was a photo of the Nirvana baby and, as a human reviewer, you had no access to the right side photo because it was just a hash in an anonymous database?
[0] https://thishashcollisionisnotporn.com/
After all, Youtube copyright strikes have never been abused thanks to the fact that there are human reviewers.
Also, no app has ever been incorrectly removed from the Apple App Store, for the same reason that there are human reviewers.
This is thanks to the fact that human reviewers never make mistakes, unlike the rest of humans.
What could go wrong if we adopt the same system for something that could potentially result in a criminal investigation?
Are you able to explain why this specific case would be different?
1. https://cases.justia.com/federal/appellate-courts/ca10/14-32...
> Representatives of multiple law enforcement agencies have offices in the NCMEC building, including the FBI, Department of Homeland Security, U.S. Marshals, U.S. Postal Inspection Service ...
> Your phone will run your images through NeuralHash, and compare to the list of forbidden hash codes. If there’s a match, some data will be uploaded to Apple’s servers when you upload the image.
Your phone doesn’t know whether there is a match or not. There is no “if match then upload”. The safety voucher is attached to every iCloud upload, not just the matches.
This is why Apple can’t just flip a setting and scan all your images regardless of whether you add them to iCloud or not. If they wanted to run this against all your images, they’d have to either upload all your images to iCloud (easy to spot; difficult for them to do without consent) or change the way this mechanism works (not “just flipping a setting”; if they wanted to scan all your images, then why build it this way in the first place?).
Technically, but not substantively, true since they report to the NCMEC, a government-funded government-established “private nonprofit” that is delegated a whole bunch of jobs for government, including gathering info on CSAM and distributing it to law enforcement for DoJ and processing child return/access requests under the Hague Convention on the Civil Aspects of International Child Abduction for the Department of State.
Indeed, you don't necessarily know what photos are on your phone.
That said, leaving your devices at home is a sensible idea.
I found it incredible that there were so many deviants known by the police in my community, they eventually caught the killer. He turned out to be a neighbor and confessed.
I think the police are well aware of every download/upload of child porn or are atleast they are able to look it up when the need arises.
The time for concern of protecting ones privacy on any digital connected device has come and gone, best to assume anything You do on your device is readily available for the authorities(and others) to exploit.
*it was 21 years ago, and the only news report I found says 100 known but I am positive they said 15K.*
https://www.cbc.ca/news/canada/missing-girl-s-body-found-in-...
About 1/2 of 1% (estimates vary) of people are sexually attracted to children and an unknown number commit acts against children. Statistically if your kid gets abused it will probably be by someone you know and shouldn't have trusted.
You should probably have a healthy mistrust regarding men alone with your kid and the government without being paranoid.
Im not sure I understand what You mean by this, my thought or possible concern is that all web traffic, especially that which goes through known proxy addresses is being logged and flagged by local carriers and ISPs. Then in the event of a crime being commited in my area, my actions online including stuff from decades past will invite a knock on my door by the authorities or worse the media.
I must stress, Im not worried about anything sexual but have done a great deal of research on some other subjects that are not exactly above board.
I can appreciate the concern regarding apple scanning personal files and reporting matching files to the government, but feel the odds of a virus, vulnerability or exploit being used by a criminal ngo is just as big if not bigger risk and has been for a long time now.
Anyways, like I said...for a while now I have been of the mindset that anything I do on a connected device (even temporarily connected) leaves me exposed, now or in the future. Maybe cancel culture has encouraged my current way of thinking.
Pop!_OS (no Ubuntu telemetry and OpenSnitch) over MacBook Air is working flawlessly. Company MacBooks are with Arch/Manjaro. Will run Catalina for specific tasks without internet until it breaks completely.
The hostile design decisions of Apple are unbelievable. After 20 days of public backslash Apple is preparing for massive iPhone 13 campaign and pushing this obviously flawed intrusion with "common good" goals, normalizing surveillance on a new level. And all of this after Pegasus/NSO.
But actually I am glad. No more shiny toys and marketing illusions for me. Expect everybody to follow suit. Year or two down the road this type of scanning and control will be mandatory.
So the only way to respond is to invest in your privacy. Not in user-hostile, dark pattern designed products.
Happen to have any suggestions for what a privacy-conscious consumer should look into?
If I need a photo - I grab my old Sony RX. If I know that I will shoot - my old Leica MP240 with Summilux and 24mpx is enough. If I have to communicate with text - email or signal on computer. If I need in car navigation - Garmin.
Deleted Facebook/Instagram spyware long time ago. Flickr also. Self-hosting is the way to go.
I am using my iPhone for calls and sms with clear knowledge that everything is recorded. Banking apps when I don't have access to computer.
I have iPad Pro 12"WIFI with faulty WIFI and no Apple care which I use only for drawing and digital painting.
People around me are switching to deGoogled smartphones with Lineage/Calyx.
Indeed, this article (despite many incorrect facts about the process) is right in that this is just a vocal minority 'screeching' to get the message out there to anyone willing to listen. The absolute majority of the population, or at least adult population, will always choose the path of least resistance - they just wants to live, work, avoid hardship, and partake in dopamine-inducing activity every day. There's a reason Google and Amazon are trillion-dollar enterprises and Hulu still has an ad-supported version that 70% of people choose[0] - people don't value privacy at all when dealing with these faceless companies. Society has always shifted the level of privacy they accept (think taxes, audits, search & seizure with probably cause), and this will just be another shift that people accept so that they continue on with their lives while not sacrificing any of their valuable time.
0: https://variety.com/2019/digital/news/hulu-ad-supported-subs...
Personally, I have no problem with ad-supported and generic/no-tracking. Ads themselves are not the privacy invasion, and have a reasonable history of helping to pay for things, and it is possible to rebuild ad companies to respect privacy (again). (It might take burning a few big players to the ground first, unfortunately.)
Hostile for sure, unbelievable, not really.
This type of "we know what's best for you" attitude is deeply ingrained in the DNA of Apple as a company.
The fact that they did get it right many time and that their batting average is above other tech companies is no excuse for the fact that the attitude is deeply flawed.
This was bound to happen.
You are holding it wrong. Here we go, we are removing "escape" but you have a touch-bar with customizable functionality, sorry this is the best laptop keyboard design - you will adapt to it.
We are removing ports - but you will have Dongles - lot of dongles and this MagSafe shit that you love so much - no more for you.
Innovation my TrashCan - not upgradable peace of trash, after 6 years - here we go MacPro - only for the Apple "elite hustler" club members, as a bonus 1000 dollars monitor stand.
We don't like MacOS - to much control for the user is not good for Apple Profits - we like iOS and we will make everything like it. You will adapt.
Now we are innovating on a geo-political stage, protecting our brand with the all time classic "Neural Cash" and bending a knee or two in the name of Apple dominance and antitrust avoidance. You will adapt, because you are trapped and we know you love us.
Privacy, my ass. :)
This kinda flies in the face of that except for the fact it's wrapped in, as you said, "common good" goals.
I've never been Apple, but I'm currently attempting to migrate to a LineageOS daily-driver mobile, although I still require some amount (nano) of Google to get by. Ironically, I needed a Windows machine to do this, whilst I'm already almost exclusively Linux (Ubuntu and Pop!_OS).
Maybe only a minority of Apple customers are aware of and understand the scanning issue and are giving feedback to Apple via the web. But here is a thought experiment: Does anyone think that if the majority of Apple customers were briefed on the issue and then asked if they agree/disgree with the minority of customers would they disagree with this informed minority.
"Tech" companies should know whats best for advertisers. They can be expected to ignore user feedback complaining about surveillance. OTOH, Apple should know whats best for users. Its users are telling the company and the company is irgnoring the feedback.
“We promise” is easily forgotten. “We can’t” requires dev teams to change.
For all the things that have been said about the issue, I don’t remember a single comment that has been directed at the US government, or China itself, or any discussions about how nation-states should be run and genuine introspections on what could prevent surveillance states from existing in the first place.
- government may use this in future to tag people for wrongthink imagery, but also
- CP is rampant on FB and FB reports it all but nothing happens so what difference does it make
Could a government not have a different level of interest in law enforcement over CP vs, say, political material that undermines government authority?
Indeed; I'm not sure this premise holds. I personally don't trust Apple's promises on privacy given its actions in China shows that it values market access more than its privacy values. But I find this argument weak.
It's utterly trivial to modify the image to avoid matching. Most minor modifications will do so.
The purpose of the hash is so that idiotic criminals that aren't trying to avoid getting caught don't accidentally avoid getting caught just by resizing or recompressing the images.
The flawed 'hashing' function also creates plausible denyability: If its somehow proved, perhaps via a whistleblower, that Apple was in fact matching the historic tank man photo, Apple or the government could easily hand over a piece of child porn with the same neuralhash due to the extraordinary vulnerability of the function.
A lot of research went into classifying images, mostly from the side that wanted visible or invisible watermarks in pictures. It is a cold war against image cropping, compression and modification that tends to not favor the classification side.
But here you only want to have a different hash, the operations for that are numerous and probably indeed often trivial.
I don't think you could just have a completely different picture create a collision though. Maybe they can because it is their hash-function, but that would be quite unusual and I doubt this is the case.
Allow me to introduce you to my posts on github: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...
Where I post good looking examples of standard test images altered fairly subtly to give the specific hashes.
The apple neuralhash is broken as a 'hash function'.
It's much much easier to modify images to just have a different hash. A simple blemish on the image-- or, with whiteboxing using the hash function, no visually noticeable change is required at all.
But the price of that is that it's not secure against generating arbitrary preimages based on fairly arbitrarily targets, with results that look nothing like the original image you're matching (if one even exists).
They could do some downsample, transform, quantize, sha256 approach that would still survive some tiny amount of differences. ... but apple favors easily constructed false positives over false negatives, even though its trivial for anyone trying to avoid their function to get a false negative. Their scheme only protects against accidental false negatives. It's just a symptom of systematically favoring factors other than the privacy and security of the actual owner and user of the computer.
How is that supposed to appeal to right wingers? He thinks all right wingers want those rioters to escape justice? Most republicans oppose them according to several polls. The author may be a victim of political divisiveness due to paying too much attention to social media and news.
In fact, Democrat protesters occupied the Capitol building in 2018. I'm not aware of a manhunt for those protesters.
> Most republicans oppose them according to several polls.
I suggest the poll be re-run after you tell people that the MSM narrative was a lie - protesters killed nobody, and police opened the doors. Nobody arrested had a gun.
AOC has told 3 different stories, yet she was a mile away at the time.
In fact, there was no "insurrection", confirmed by the FBI last week. But there was a Marxist-style narrative - they're very clever with wording, aren't they?
Why didn't Pelosi add more security for 100,000 protesters? That's the only fact we really need to know, isn't it?
Not a partisan thing; effective left wing leaders also have their phones targeted. The swamp doesn't like change.
This is where the slippery slope argument comes in. Apple would have to actively cooperate with governments to both detect and report other kinds of images. So if Tiananmen Square images were showing up in the review pile, Apple would have to know to pass those on to the Chinese government and be willing to.
The only other way they'd be passed on is if Apple drops the manual review portion.
Killer argument. Why not focus here instead?
A title that has so little to do with the contents of article, that it might as well be a random reference to some throw-away line in some comment somebody uttered sometime, which might as well be left alone so a proper title could be selected.
Normally I would agree, but I'm fine with the subtle(?) implication of this one.
Didn't know about the memo until reading other comments.
> We know that the days to come will be filled with the screeching voices of the minority.
https://9to5mac.com/2021/08/06/apple-internal-memo-icloud-ph...
from a letter from NCMEC that Apple circulated in an internal memo.
Who of the two gets to decide how much expository context is right?
After the Pegasus news that showed that a relatively small private company in a very small country could have complete remote access to your phone and sell it on the market, do you really think that a powerful government like China or the US can't already do it??? How could you be this naive? Especially after the Snowden revelations.
And about Apple's having a change of hearth in the future and using something like this against you. Have you stopped to think for one second and consider that they have complete control over the updates that your phone receives?? They can quietly do what ever they want without you ever knowing and you can't do sh*t about it. So drop the concerned citizen act. We all abdicate privacy a long time ago when we sold our souls for cool free services like google and sexy gadgets.
At the end of the day the "slippery slope" argument boils down to one thing: do you trust your institutions? Any law or service can be abused, and the only thing preventing that is the integrity of the people on your society and the institutions both public and private that they work for. The exact same law that is used to fight corruption in a great country like Norway or Denmark can be abused to persecute political opponents in a shitty country like North Korea or China.
If the answer is no, I don't trust my institutions and my people, then you have a far more severe and fundamental problem than an specific law or service. In that scenario, focusing in an specific issue is like trying to cover the sun with your hands.
These whataboutisms are a distraction and in no way support the idea that people shouldn't be worried about Apple implementing a new client-side system that's ripe for abuse.
If things are already bad, we should be working towards making them better instead of rolling over and apathetically allowing them to get worse.
> How could you be this naive? Especially after the Snowden revelations.
You make a good argument, but you're wrong to dismiss the slippery-slope counter-argument. This move by Apple is normalizing your own devices betraying you. Like if door locks all had a master key the police could use to open them. It doesn't matter that they're easy to lockpick, what matters is the principle, because that is what stops you from taking the first step down a very dark road. And every step along that road will be portrayed as the "first".
As for "do you really think that a powerful government like China or the US can't already do it???" - this argument can be used to dismiss any effort at preserving privacy.
Also I don't trust institutions. This is axiomatic and why separation of power is essential. So
> the only thing preventing that is the integrity of the people on your society and the institutions both public and private that they work for
is plainly wrong in my opinion. If you let people abuse this, they will. Also I don't think that can meet with reality. Police departments alone have quotas they must match, so they have to find criminals... It doesn't take a genius to figure out the repercussions. But it is still practiced in many countries.
Separation of powers is plainly ignored for everything under the guise of safety and a disease infecting politics the last few decades. That is due to incompetent leadership. Another reason why you should not trust "institutions". Instead, you check them and burn their asses if they fail as often they do today. The reaction that trust is falling in press and public officials is completely rational on the other hand.
There's definitely still a "slippery slope" argument here that in handling this specific use case they are going to feel more pressure to handle other specific use cases of government actors, but they are also setting the precedence that they still will not support a generic backdoor. That seems to me, at face value, a greater privacy win than a loss.