This entire platform is the first time I've strategically considered realigning the majority of my use to Apple.
Airtag anonymity was pretty cool, technically speaking, but a peripheral use case for me.
To me, PCC is a well-reasoned, surprisingly customer-centric response to the fact that due to (processing, storage, battery) limitations not all useful models can be run on-device.
And they tried to build a privacy architecture before widely deploying it, instead of post-hoc bolting it on.
>> 4. Non-targetability. An attacker should not be able to attempt to compromise personal data that belongs to specific, targeted Private Cloud Compute users without attempting a broad compromise of the entire PCC system. This must hold true even for exceptionally sophisticated attackers who can attempt physical attacks on PCC nodes in the supply chain or attempt to obtain malicious access to PCC data centers.
Oof. That's a pretty damn specific (literally) attacker, and it's impressive that made it into their threat model.
And neat use of onion-style encryption to expose the bare minimum necessary for routing, before the request reaches its target node. Also [0]
>> For example, the [PCC node OS] doesn’t even include a general-purpose logging mechanism. Instead, only pre-specified, structured, and audited logs and metrics can leave the node, and multiple independent layers of review help prevent user data from accidentally being exposed through these mechanisms.
My condolences to Apple SREs, between this and the other privacy guarantees.
>> Our commitment to verifiable transparency includes: (1) Publishing the measurements of all code running on PCC in an append-only and cryptographically tamper-proof transparency log. (2) Making the log and associated binary software images publicly available for inspection and validation by privacy and security experts. (3) Publishing and maintaining an official set of tools for researchers analyzing PCC node software. (4) Rewarding important research findings through the Apple Security Bounty program.
So binary-only for majority, except the following:
>> While we’re publishing the binary images of every production PCC build, to further aid research we will periodically also publish a subset of the security-critical PCC source code.
>> In a first for any Apple platform, PCC images will include the sepOS firmware and the iBoot bootloader in plaintext, making it easier than ever for researchers to study these critical components.
Another good step in this direction would be publishing a list of all on-device Apple software (including Spotlight models for image analysis) and details of any information that is sent to Apple, along with opt-out instructions via device Settings or Apple Configurator MDM profiles.
Apple does publish a list of network ports and servers, so that network traffic can be permitted for specific services. The list is complicated by 3rd-party CDNs, but can be made to work with dnsmasq and ipset, "Use Apple products on enterprise networks", https://support.apple.com/en-us/101555
> Oof. That's a pretty damn specific (literally) attacker, and it's impressive that made it into their threat model.
How so ?
There are any number of state and state sponsored attackers who it should apply it including china, North Korea , Russia , Israel as nation states and their various affiliates like NSO group .
Even if NSA its related entities are going to be notably absent. If your threat model includes unfriendly nation state actors then the security depends on security at NSA and less on Apple, they have all your data anyway.
If nation state actors are interested in you, no smartphone that is not fully open source on both hardware and OS side that has been independently verified by multiple reviewers is worth it, i.e. no phone in the market today, everything else is tradeoff for convenience for risk, the degree of each is quite subjective to each individual.
For the rest of us, the threat model is advertisers, identity thieves, scammers and spammers and now AI companies using it for training.
Apple will protect against other advertisers insofar to grow their own ad platform , they already sell searches to Google for $20B/year and there is no knowing the details of the OpenAI deal on what kind of data will be shared.
A lot of this sounds like Apple has been 10-20 years behind the state of the art and now wants to tell you that they partially caught up. Verifiable hardware roots of trust and end-to-end software supply chain integrity are things that have existed for a while. The interesting part doesn't come until the end where they promise to publish system images for inspection.
Most of the stuff in the blog post reads like common security precautions: don't run as root, stateless immutable nodes, use secure boot, etc. All wrapped up in some Apple marketing pizzazz.
> common security precautions ... marketing pizzazz.
If it were this common, Meta, Google, and others would have announced or launched something similar for its consumer apps/services; I can't seem to recall anything of note.
Apple's system goes further by having incoming requests choose and verify a server and then encrypt itself using the public key of the node to prevent MITM attacks.
And a one-time credential to prevent replay attacks.
As well as minor things like obfuscating IP addresses, metadata etc.
Apples system is also the entire pipeline. Borg SREs can still change behavior here. It’s a lot better than what most places have but does not go far enough.
This is not true at all. Apple is the first to roll out end to end remote attestation of an enclave that includes an ML accelerator in the root of trust, with public verifiability of the entire stack. They are way ahead.
Perhaps one of these days we'll get a 'jeffbee that realizes that Google is not actually ahead of everyone in everything all the time. But not today, I guess.
> In a first for any Apple platform, PCC images will include the sepOS firmware and the iBoot bootloader in plaintext, making it easier than ever for researchers to study these critical components.
Yes!
> Software will be published within 90 days of inclusion in the log, or after relevant software updates are available, whichever is sooner.
I think this theoretically leaves a 90-day maximum gap between publishing vulnerable software and potential-for-discovery. I sincerely hope that the actual availability of images is closer to instant than the maximum, though.
Well, a 89-day "update-and-revert" schedule will take care of those pesky auditors asking too many questions about NSA's backdoor or CCP's backdoor and all that.
No, because the log of what source was used will still show the backdoored version, and you can't unpublish the information that it was used. Reverting doesn't solve the problem that people will be able to say "this software was attested 90 days ago and it hasn't been released".
If you're trying to do a quiet backdoor and you have the power to compel Apple to assist, the route to take is to simply misuse the keys that are supposed to only go into hardware for attestation, and instead simply use them to forge messages attesting to be running software on hardware that you aren't.
Or just find a bug in the software stack that gives you RCE and use it
> simply use them to forge messages attesting to be running software on hardware that you aren't
Well, your messages have to be congruent with the expected messages from the real hardware, and your fake hardware has to register with the real load balancers to receive user requests.
> RCE
That’s probably the best attack vector, and presumably why Apple is only making binary executables available. Not that that stops RCE.
But even then you can’t pick and choose the users whose data you compromise. It’s still a sev0 problem, but less exploitable for the goals of nation states so less likely to be heavily invested in for exploiting.
> Well, your messages have to be congruent with the expected messages from the real hardware,
Yes, which is why you need the keys that are used to make real hardware. Provided you have those very secret and well protected keys (you are Apple being compelled by the government) that's not an issue.
> and your fake hardware has to register with the real load balancers to receive user requests.
Absolutely, but we're apple in this scenario so that's "easy".
I think I misunderstood your point -- I took it to mean someone impersonating a server, but you're saying it's Apple. So the part you're attacking (as Apple) is:
> The process involves multiple Apple teams that cross-check data from independent sources, and the process is further monitored by a third-party observer not affiliated with Apple. At the end, a certificate is issued for keys rooted in the Secure Enclave UID for each PCC node.
So, in your scenario, the in-house certificate issuer is compelled to provide certificates for unverified hardware, which will then be loaded with a parallel software stack that is malicious but reports the attestation ID of a verified stack.
So far, so good. Seems like a lot of people involved, but probably still just tens of people, so maybe possible.
Are you envisioning this being done on every server, so there are no real ones in use? Or a subset? Just for sampling, or also with a way to circumvent user diffusion so you can target specific users?
It's an interesting thought exercise but the complexity of getting anything of real value from this without leaks or errors that expose the program seems pretty small.
Well, the broader context of the proposal is as an alternative to the original comment in this HN thread
> Well, a 89-day "update-and-revert" schedule will take care of those pesky auditors asking too many questions about NSA's backdoor or CCP's backdoor and all that.
As a backdoor I am taking it to mean they can compel assistance from inside of Apple, it's not a hack where they have to break in and hide it from everyone (though certainly they would want to keep it to as few people as possible).
At least in the NSAs case I think it would be reasonable to imagine that they are limited to compromising a subset of the users data. Specific users they've gotten court orders against or something... so yes a subset of nodes and also circumventing user diffusion (which sounds like traffic analysis right up the NSAs alley, or a court order to whatever third party Apple has providing the service).
> so yes a subset of nodes and also circumventing user diffusion (which sounds like traffic analysis right up the NSAs alley, or a court order to whatever third party Apple has providing the service).
How does traffic analysis help? The client picks the server to send the query to, and encrypts with that particular server's private key. I guess maybe your have the load balancer identify the target and only provide compromised servers to it? But then every single load balancer has to have the list of targeted individuals and compromised servers, which seems problematic for secrecy at scale.
The load balancer is blind to which client sent a request via ohttp. You need to do something to bypass that (traffic analysis or ordering the ohttp provider to help).
> But then every single load balancer has to have the list of targeted individuals and compromised servers, which seems problematic for secrecy at scale.
It really doesn't. This seems well within the realms of what you could achieve with a court order without it becoming public.
Sounds good. Still won’t send anything sensitive there but I appreciate the effort and direction, especially when current industry trend seems to be fuck you were rewriting our TOS to take your data.
I really want to see this OS, and have cautious optimism that this could be the first time we'll see a big tech company actually provide an auditable security guarantee!
I think depending on how this plays out, Apple might manage to earn some of the trust its users have in it, which would be pretty cool! But even cooler will be if we get full chain-of-custody audits, which I think will have to entail opening up some other bits of their stack
In particular, the cloud OS being open-source, if they make good on that commitment, will be incredibly valuable. My main concern right now is that if virtualization is employed in their actual deployment, there could be a backdoor that passes keys from secure enclaves in still-proprietary parts of the OSes running on user devices to a hypervisor we didn't audit that can access the containers. Surely people with more security expertise than me will have even better questions.
Maybe Apple will be responsive to feedback from researchers and this could lead to more of this toolchain being auditable. But even if we can't verify that their sanctioned use case is secure, the cloud OS could be a great step forward in secure inference and secure clouds, which people could independently host or build an independent derivative of
The worst case is still that they just don't actually do it, but it seems reasonably likely they'll follow through on at least that, and then the worst case becomes "Super informative open-source codebase for secure computing at scale just dropped" which is a great thing no matter how the other stuff goes
> even if we can't verify that their sanctioned use case is secure, the cloud OS could be a great step forward in secure inference and secure clouds, which people could independently host or build an independent derivative of
> My main concern right now is that if virtualization is employed in their actual deployment, there could be a backdoor that passes keys from secure enclaves in still-proprietary parts of the OSes running on user devices to a hypervisor we didn't audit that can access the containers.
We’ll release a PCC Virtual Research Environment: a set of tools and images that simulate a PCC node on a Mac with Apple silicon, and that can boot a version of PCC software minimally modified for successful virtualization.
This seems to imply that PCC nodes are bare-metal.
Could a PCC node be simulated on iPad Pro with M4 Apple Silicon?
Yes, most technology is built on other technology ;)
> This seems to imply that normal PCC nodes are bare-metal.
I realize that, but there's plausible deniability in it, especially since the modification could also hide the mechanism I've described in some other virtualization context that uses the unmodified image, without the statement being untrue
> As best I can tell, Apple does not have explicit plans to announce when your data is going off-device for to Private Compute. You won't opt into this, you won't necessarily even be told it's happening. It will just happen. Magically.
Presumably it will be possible to opt out of AI features entirely, i.e. both on-device and off-device?
Why would a device vendor not have an option for on-device AI only? iOS 17 AI features can be used today without iCloud.
Hopefully Apple uses a unique domain (e.g. *.pcc.apple.com) that can be filtered at the network level.
You would have to activate a clearly LLM-powered software feature and have internet access. I don't know if settings will appear to disable this, but you could imagine it would be the case. This isn't just siphoning off all your data at random.
Would Spotlight be considered a "clearly LLM-powered software feature"? Will there be an option for "non-AI Spotlight"? Disabling dozens of software features, or identifying all apps which might use LLM services, is a daunting proposition. It would be good to have a PCC kill switch, which makes opt-in usage meaningful, rather than forced.
Privacy "consent" is fundamentally broken. We've moved from "we're doing whatever the fuck we want" to "we're doing whatever the fuck we want, but on paper it's whatever the fuck you expressly asked for, whether you wanted to or not."
If you have no threat model and want to opt out of random features just because... you probably shouldn't use Apple products at all. Or Google or Microsoft.
For years, Apple has a documented set of security policies to disable off-device processing (e.g iCloud, Siri), via MDM / Apple Configurator. Apple also published details needed for enterprise network filtering to limit Apple telemetry, if all you want from Apple servers are software security updates and notifications.
With a hardened configuration, Apple has world-class device security. In time, remote PCC may prove as robust against real-world threats. Until then, it would be good to retain on-device security policy and choice for remote computation.
Apple does not publish details to limit telemetry. Nowhere in MDM or in their docs do they tell you that you can safely block xp.apple.com (telemetry) but not gs.apple.com (boot ticket signing server for updates).
I think the main reason might be the on-device AI is fairly limited features wise. For Apple to actually offer something useful they would need to switch between device/server constantly and they don't want to limit the product by allowing users to disable going to a server.
With OpenAI calls is different because the privacy point is stronger
> Ok there are probably half a dozen more technical details in the blog post. It’s a very thoughtful design. Indeed, if you gave an excellent team a huge pile of money and told them to build the best “private” cloud in the world, it would probably look like this.
and
> And of course, keep in mind that super-spies aren’t your biggest adversary. For many people your biggest adversary is the company who sold you your device/software. This PCC system represents a real commitment by Apple not to “peek” at your data. That’s a big deal.
I'd prefer things stay on the device but at least this is a big commitment in the right direction - or in the wrong direction but done better than their competitors, I'm not sure which.
Is there more to that thread? I can't read it if it exists, not sure if that is what the parent is talking about? But i don't have a Twitter account anymore, so maybe it's locked?
I don't know what you're seeing. It's a very long thread. Exceptionally good take on the whole thing. Apple has gone way out of their way to try and sell this thing. Above and beyond compared to how I imagine Microsoft or Google would have tackled this.
the most successful sheeple operation is the one the sheeple and the entire world is completely oblivious of it.
jokes aside, this is no different from people selling bunker beds, gold, ammunition, crypto, vpns. It is specifically for the set of gullible people who think they and their data is so important. Reality (except for 10,000 people or so) is, most lives and their 'precious' data is worthless. (I'm not talking about SSN, Bank Accounts -- those are well protected by tech cos HN seem to hate on)
That's likely due to tracking prevention or protection by your browser because X really, really wants to track you. If you disable the tracking protection and related settings, you may be able to see the single tweet.
weird, i get a bunch of music and programming stuff on my Threads feed. it's not very deep, but what's on the surface is quite nice and not a bunch of almost-porn. Twitters become half porn though
He's not wrong that, given that you want to do this, this is the best way. The alternative would be to not do it at all (though an opt-out would have been good).
Can some ELI5 how remote attestation is supposed to work? It feels like asking a remote endpoint “are you who you say you are”. What’s stopping remote endpoint always responding “yes”
They rely on Trusted Execution Environments and the fact that hash functions are one-way functions.
Verifier -> requests a Prover to attest its software state
Prover -> goes into RoT, verifies authenticity of Verifier (and request), computes hash of attested memory region, sends hash digest
Verifier -> receives digest and compares to known hash
> What’s stopping remote endpoint always responding “yes”
The attestation code is inside of a RoT, so a bad actor shouldn't be able to call this code, only callable by receiving a request from a Verifier
My understanding is that it's similar to TLS authentication.
The remote endpoint has special hardware which keeps secret signing keys (similar to a TLS server's signing keys). The hardware refuses to reveal the private keys, but will sign certain payloads under certain conditions. In addition, Intel or AMD or whoever also has super duper mega secret master keys (similar to a CA's signing keys), which they use to sign the device's signing keys. The certificate signing the device keys is also stored on the device.
So, each time the endpoint is asked to attest its software, it says yes and signs its response with its keys, and it also sends a certificate showing its keys are signed by the master key. That way, the client knows the special hardware really said yes and that Intel or AMD or whoever said that particular special hardware is legit.
Who is this for? Dont get me wrong I think it's a great effort. This is some A+ nerd stuff right here. It's speaking my languge.
But Im just going to figure out how to turn off "calls home". Cause I dont want it doing this at all.
Is this speaking to me so I tell others "apple is the most secure option"? I don't want to tell others "linux" because I don't want to do tech support for that.
At this point I feel like an old man shouting "Dam you keep your hands off my data".
Apple needs to differentiate itself, and they have chosen privacy as a way to do that, which I'm all for. The headlines around Microsoft's AI efforts have largely been a nightmare, with a ton of bad press. If the press around Apple's AI is all about how over the top they went with security and privacy, that will likely make people feel a little better about using it.
I'm not a big user of OpenAI's stuff, but if I was going to use any of it, I'd rather use it through Apple's anonymizing layer than going directly to OpenAI.
I actually thought one notable thing in the presentation was that they spent all this time talking about their new private cloud compute architecture.
And then showed that they have a prompt asking if you're ok sending the data to OpenAI. Presumably because despite OpenAI promising not to use your data (a promise apple relayed) OpenAI didn't buy into this new architecture.
Thank you for mentioning this. I thought I was going crazy, because I heard this too, but kept seeing comment after comment on other sites asking if a person could choose not to use OpenAI, or that it was happening magically in the background. The way I heard it, the user was in control.
I think this goes back to what Steve said in 2010.
And yes, while the data might not be linked to the user and striped of sensitive data, I could see people not wanting something very personal things to go to OpenAI, even if there should be no link. For example, I wouldn’t want any of my pictures going to OpenAI unless I specifically say it is OK for a given image.
I was under the impression that the OpenAI integrations were more about content generation and correction than the Apple Intelligence-driven personal stuff.
OpenAI provides the chatbot interface we all know.
The PCC cloud serves all of the other integrated AI features like notification prioritization, summarization, semantic search, etc. At least when those can’t be run on device.
I don’t care if the government has access to the data. I just don’t want “bad actors” (scammers, foreign governments, ad-tech companies, insurance companies, etc) to have access to my private data. But i also want the power of LLM’s. Does that sound so far fetched?
I’m a realist. I already EXPECT the US govt has all my data. I don’t like the status quos, but it is what is is.
It's for shareholders. Microsoft and Nvidia have a bigger market cap than Apple now, thanks to the AI investor boom. Apple need to show they can be all about AI, too. But Apple have the institutional culture to maintain privacy.
This is exactly what I was thinking during the entire keynote. It was blatantly the WWSC (worldwide shareholder conference) and hackernews commenters are eating it up.
Don't get me wrong, I've always appreciated apples on device ml/AI features, those have always been powerful, interesting and private but these announcements feel very rushed, it's literally a few weeks after Microsoft's announcements.
They've basically done almost exactly what Microsoft announced with a better UX and a pinky promise about privacy. How are they going to pay for all of that compute? Is this going to be adjusted into the price of the iPhones and MacBook? and then a subscription layer is going to be added to continue paying for it? I don't feel comfortable with the fact that my phone is basically extending it's hardware to the cloud. No matter how "private" it is it's just discomforting to know that apple will be doing inference on things seemingly randomly to "extend" compute capabilities.
Also what on earth is apple high on, integrating a third party API into the OS, how does that even make sense. Google was always a separate app, or a setting in safari, you didn't have Google integrated at an OS level heck you don't have that on Android. It feels very discomforting to know that today my phone could phone home to somewhere other than iCloud.
NSA already has all our data and if they don't, they have direct contacts at Meta and Alphabet to get it same-day delivery.
I'm trusting Apple more in this case, they have an incentive to keep things private and according to experts they're doing everything they can to do so.
"Indeed, if you gave an excellent team a huge pile of money and told them to build the best “private” cloud in the world, it would probably look like this." - Matthew D. Green
The NSA partners directly with telecoms companies, especially AT&T. Its easier when companies like Meta and Facebook will play along, but that's not the only way they get access to a bunch of our data.
It’s for their competitors who have pushed a narrative that Apple was caught “flat-footed”. Well there is actually a line of LLM and cloud infrastructure at iPhone scale. This is not merely 2 years of work. They push the privacy because it’s expected of them.
Gemini could claim privacy but I think people would assume that if true, it would make it less effective.
What I'm most curious about here is if a state actor comes to Apple with a subpoena and compels them to release information on an individual, what would Apple be able to release?
... I suppose this is ultimately a question that will be tested sooner or later in the US.
Unless their statements regarding the design of these systems are blatantly false, or they are forced to add data collectors on purpose to target individuals, the answer is close to nothing.
You can opt into full E2E encryption [1] which makes it nothing, presumably at the cost of some convenience features.
I mean it was famously tested in 2015 after the San Bernardino attack. Apple didn’t back down [1] and later sued the company who sold the zero-day to the govt to unlock the phone [2].
That's an ordinary subpoena and that data is not being specially collected and not e2ee encrypted. Has nothing to do with the guarantee in this article.
Probably everything uploaded after the intercept is in place if you can convince a court to compel it.
One option is to release a malicious software update, sign it, publish the signature on the public chain, and then simply not release the binaries until after whatever associated gag orders there are (if any) expire. Apple gave themselves a 90 day timeline for this before they'd even be in violation of their promises.
Another option is to use the cryptographic keys used to make the hardware that attests to the software running on it, to simply falsely attest to what software is running. Unless Apple's somehow moved those keys outside of the courts jurisdiction (which means outside of Apple's control in the case of most courts) that should be within the courts power. If they can still create new hardware, it seems likely whoever is making that hardware must still have access to the keys...
Both of these attacks are outside the "threat model" proposed, because they are broad compromises against the entire PCC infrastructure. The fact that they are possible and within the legal systems power... well... why are we advertising this as secure again?
The main value of this whole architecture in my mind isn't actually security though, it's that it's Apple implicitly making the promise that they won't under any circumstance use the data, or let anyone else use the data, for business purposes (not even for running the service itself).
> One option is to release a malicious software update, sign it, publish the signature on the public chain,
In this option it would be Apple releasing a malicious software update?
> If they can still create new hardware, it seems likely whoever is making that hardware must still have access to the keys...
This option reads like the keys are stored in apple-keys.txt
> Both of these attacks are outside the "threat model" proposed, because they are broad compromises against the entire PCC infrastructure
They mentioned that the in-depth write up will be shared later, might they still address this concern in writing? Your wording makes you sound so certain, but this is just a broad overview. How are you so sure?
> In this option it would be Apple releasing a malicious software update?
Yes, compelled by something like the all writs act (if the US is the one doing the compelling).
> This option reads like the keys are stored in apple-keys.txt
They probably are. That file might live on a CD drive in a safe that requires two people to open it, but ultimately it's a short chunk of binary data that exists somewhere (until it is destroyed)...
> might they still address this concern in writing?
Can I say beyond all doubt that this won't happen? Of course not.
On the first approach I'm quite confident though, because it's both the type of attack they discuss in their initial press release, and pretty fundamental to and explicitly allowed by their model of updating the software.
On the second approach I'm reasonably confident. Like the first issue it's the type of issue that they were discussing in their initial press release. Unlike the first issue it's not something that is explicitly allowed in the model. If Apple can find a way to make the attestation keys irretrievable while still allowing themselves to manufacture hardware I believe they'd do it - I just don't see a method and think it would have warranted a mention if they had one. I tried to insert a level of uncertainty in my original writing on this one because I could be missing a way to solve it.
Ultimately I'd rather over-correct now then have people start thinking this is going to be more secure than it is and then have some fraction of them miss the extremely-likely follow up of "and we could be compelled to work around our security".
I'm well aware of these. They don't solve the problem at hand. You need a way to put keys into new hardware. Thus you need a way to get keys out of wherever you've stored your cryptographic material. Thus it can't be on a HSM (or it can be if it's a master key signing child keys, but in that case the attack only needs a signed child key).
“A randomly generated UID is fused into the SoC at manufacturing time. Starting with A9 SoCs, the UID is generated by the Secure Enclave TRNG during manufacturing and written to the fuses using a software process that runs entirely in the Secure Enclave. This process protects the UID from being visible outside the device during manufacturing and therefore isn’t available for access or storage by Apple or any of its suppliers.“
Sure, and even Apple can't imitate a different server that they made.
They're making new servers though. Take the keys that are used to vouch for the UIDs in actual secure enclaves, and use them to vouch for the UID in your evil simulated "secure" enclave. Your simulated secure enclave doesn't present as any particular real secure enclave, it just presents as a newly made secure enclave that Apple has vouched for as being a secure enclave.
I'm very curious as well because my very limited understanding tells me the answer is nothing. The relay hides your identity. Your phone checks the attestations so it won't send your data to servers not running the published software which ensures encryption keys are ephemeral. Once your session is done, the keys are deleted.
Law enforcement would need to seize the right server among millions while it's processing your request and perform an attack on it to get the keys before they're gone.
My next question is what happens if/when the attestation keys are stolen.
Outside of all the security aspects which look to be handled quite well on the surface I do enjoy that the client mainframe architecture is still a staple of computing.
Following the loss of Apple, easily its biggest client, Imagination was bought out by a Chinese-based investment group. Apple subsequently released its first in-house designed mobile GPU as part of the A11 Bionic SoC that powered the iPhone X.. The new “multi-year license agreement” gives Apple official access to much wider range of Imagination’s mobile GPU IP as well as its AI technologies. The A11 Bionic also included the first neural processing engine in an iPhone
Apple doesn't want to pay the Jensen Leather Jacket Fee and has $200 billion in cash it is sitting on to make it happen. If anyone can create an Nvidia substitute for AI chips its Apple and their cash hoard combined with their world-class design team and exclusive access to all of TSMC 3nm and next year 2nm production they could possibly want.
> The Apple Security Bounty will reward research findings in the entire Private Cloud Compute software stack — with especially significant payouts for any issues that undermine our privacy claims.
> The Secure Enclave randomizes the data volume’s encryption keys on every reboot and does not persist these random keys, ensuring that data written to the data volume cannot be retained across reboot. In other words, there is an enforceable guarantee that the data volume is cryptographically erased every time the PCC node’s Secure Enclave Processor reboots.
Intel and AMD server processors can use DRTM late launch for fast attested restart, https://www.semanticscholar.org/paper/An-Execution-Infrastru.... If future Apple Silicon processors can support late launch, then PCC nodes can reduce intermingling of data from multiple customer transactions.
> The server can't afford
What reboot frequency is affordable for PCC nodes?
Well the options from China's perspective is: Come to the table and meet some/all of our demands or stop doing business here.
Since Apple devices are now on the Chinese Governments poopy list, I assume Apple is only meeting some, not all of China's demands. I assume if Apple did everything the Chinese govt wanted, they wouldn't be on the poopy list. Personally I see being on the Chinese govt poopy list as an endorsement that it's probably a net positive for privacy and security compared to those not on the list. :)
Around WhatsApp, it's probably part of the whole compromise mess above. WhatsApp now does E2E and that's something China is not a fan of, so it's probably China's doing that it's not in the app store in China any more. Apple is just following the laws China forces them to follow.
It should be noted I've never been to China(yet) and have zero 1st hand knowledge.
It's a silly oversimplification that nothing in China is ever allowed to have privacy ever. China has privacy/data protection laws just like other countries do. Even an authoritarian government doesn't want other random private actors getting to see everything.
I agree, but I was talking specifically about the govt.
The govt basically requires total access doesn't it? I mean every govt basically wants it, and the US has tried many times, but so far hasn't quite gotten complete access everywhere.
I was sceptical of the announcement, but this actually sounds really well thought out.
One key part though will be the remote attestation that the servers are actually running what they say they're running. Without any access to the servers, how do we do that? Am I correctly expecting that that part remains a "trust me bro" situation?
>While we’re publishing the binary images of every production PCC build, to further aid research we will periodically also publish a subset of the security-critical PCC source code.
I expect that they'll publish the attestation source code.
But, basically what will happen is the Verifier will request a certain memory region to be attested, then that region will be hashed and the digest will be sent back to the Verifier. If the memory is different from what is expected, the hash digest will NOT match.
It is not possible for this to be fully private in the United States because the government not only can force Apple to open up the kimono, it can also forbid it to talk about it. There’s not really anything Apple can do to work around this “limitation”. Thank your “representative” for extending the PATRIOT Act when you get a chance.
It makes zero sense for a company of this size. I bet they are served with gag orders like daily, so the warrant canary is going to expire the moment it is published.
There's a difference between guaranteed privacy and certifiable privacy. Yes, the government can request one's data. However, Apple's system would reveal those intrusions to the public, even if Apple themselves couldn't say it.
I mean, the software running on the client (phone/mac/ipad) is closed-source and, if we assume Apple is compromised, can be made to circumvent all of these fancy protections at the push of a button.
If pressured by the government, Apple can simply change the client software to loosen the attestation requirements for private compute. And that would be the most inconspicuous choice.
Or target a device by IMEI or iCloud to be candidate to receive a software update, and push an update that sends data to "dev-llm-assistant.ai.apple.com".
"oh it's our dev version ? what's the problem ? we need data access for troubleshooting"
Private Cloud Compute servers have no persistent storage so there would be nothing to see upon opening the kimono. You'd need some sort of government requested live wire tap thing to harvest the data out of the incoming requests, which might be a different situation. I'm, of course, just some dude on the internet, thinking up a counter-point to this concern, who knows if I am even remotely in the right ballpark.
Speaking of tech, has anyone ever independently audited Apple's encrypted infrastructure a-la what they're promising for Private Compute? I'm unconvinced that the government couldn't crack that if they wanted.
> Private Cloud Compute servers have no persistent storage so there would be nothing to see upon opening the kimono
It doesn't actually say there is no persistent storage, it says that the compute node will not store it for longer than the request. There's nothing to stop the data coming from a datastore outside of the "PCC" in another part of apple's infrastructure.
What Apple can do (and appears to be doing throughout its products) is not have the data requested. Or not have it in cleartext. NSLs can't request data that doesn't exist anymore.
But the setup is that Apple doesn't know which cleartexts currently being processed are associated with which user meaning that even live surveillance can't work without surveilling everybody, making any such program very quickly discoverable. Read the section about non-targetability in the link.
Apple deserves credit for correctly analyzing their threat model and designing their system accordingly.
I’m willing to concede that Apple’s system is the best designed of the bunch. I’m not willing to call it “private”, however, if it processes unencrypted inputs in the jurisdiction of a nation state with pervasive government surveillance.
I agree from a technical perspective actually. What this system does to counteract that is use a social strategy. The government couldn't break "this specific system" without a lot of people knowing, and that makes it very likely that they would be discovered. And not in a vague way but in a very specific one.
Would this work in some authoritarian countries? No and Apple doesn't care. Would it work in a western one? Maybe.
That's not what they meant or what the phrase means. It's mildly racist and misogynist, and the kinds of discomfort it elicits are not the kind that will make people trust you, the user of the phrase.
It seems to me that this security architecture is a direct response to the hostile regulatory environment Apple finds themselves in wrt USA PATRIOT and the CCP et al.
ZKML is actually not horrible, probably only 100-1000x overhead atm. Unfortunately it doesn’t solve the problem, you would need FHE which has much higher overhead
Here's the answer to the "what's taking Apple so long to get on the LLM train?" folks. Per usual, they lag a bit and then do it better than anyone else.
392 comments
[ 3.7 ms ] story [ 332 ms ] threadAirtag anonymity was pretty cool, technically speaking, but a peripheral use case for me.
To me, PCC is a well-reasoned, surprisingly customer-centric response to the fact that due to (processing, storage, battery) limitations not all useful models can be run on-device.
And they tried to build a privacy architecture before widely deploying it, instead of post-hoc bolting it on.
>> 4. Non-targetability. An attacker should not be able to attempt to compromise personal data that belongs to specific, targeted Private Cloud Compute users without attempting a broad compromise of the entire PCC system. This must hold true even for exceptionally sophisticated attackers who can attempt physical attacks on PCC nodes in the supply chain or attempt to obtain malicious access to PCC data centers.
Oof. That's a pretty damn specific (literally) attacker, and it's impressive that made it into their threat model.
And neat use of onion-style encryption to expose the bare minimum necessary for routing, before the request reaches its target node. Also [0]
>> For example, the [PCC node OS] doesn’t even include a general-purpose logging mechanism. Instead, only pre-specified, structured, and audited logs and metrics can leave the node, and multiple independent layers of review help prevent user data from accidentally being exposed through these mechanisms.
My condolences to Apple SREs, between this and the other privacy guarantees.
>> Our commitment to verifiable transparency includes: (1) Publishing the measurements of all code running on PCC in an append-only and cryptographically tamper-proof transparency log. (2) Making the log and associated binary software images publicly available for inspection and validation by privacy and security experts. (3) Publishing and maintaining an official set of tools for researchers analyzing PCC node software. (4) Rewarding important research findings through the Apple Security Bounty program.
So binary-only for majority, except the following:
>> While we’re publishing the binary images of every production PCC build, to further aid research we will periodically also publish a subset of the security-critical PCC source code.
>> In a first for any Apple platform, PCC images will include the sepOS firmware and the iBoot bootloader in plaintext, making it easier than ever for researchers to study these critical components.
[0] Oblivious HTTP, https://www.rfc-editor.org/rfc/rfc9458
Another good step in this direction would be publishing a list of all on-device Apple software (including Spotlight models for image analysis) and details of any information that is sent to Apple, along with opt-out instructions via device Settings or Apple Configurator MDM profiles.
Apple does publish a list of network ports and servers, so that network traffic can be permitted for specific services. The list is complicated by 3rd-party CDNs, but can be made to work with dnsmasq and ipset, "Use Apple products on enterprise networks", https://support.apple.com/en-us/101555
How so ? There are any number of state and state sponsored attackers who it should apply it including china, North Korea , Russia , Israel as nation states and their various affiliates like NSO group .
Even if NSA its related entities are going to be notably absent. If your threat model includes unfriendly nation state actors then the security depends on security at NSA and less on Apple, they have all your data anyway.
If nation state actors are interested in you, no smartphone that is not fully open source on both hardware and OS side that has been independently verified by multiple reviewers is worth it, i.e. no phone in the market today, everything else is tradeoff for convenience for risk, the degree of each is quite subjective to each individual.
For the rest of us, the threat model is advertisers, identity thieves, scammers and spammers and now AI companies using it for training.
Apple will protect against other advertisers insofar to grow their own ad platform , they already sell searches to Google for $20B/year and there is no knowing the details of the OpenAI deal on what kind of data will be shared.
Your comment makes me curious on how guarantee-to-guarantee looks (and associated architectures).
If it were this common, Meta, Google, and others would have announced or launched something similar for its consumer apps/services; I can't seem to recall anything of note.
And a one-time credential to prevent replay attacks.
As well as minor things like obfuscating IP addresses, metadata etc.
https://cloud.google.com/blog/products/identity-security/exp...
Yes!
> Software will be published within 90 days of inclusion in the log, or after relevant software updates are available, whichever is sooner.
I think this theoretically leaves a 90-day maximum gap between publishing vulnerable software and potential-for-discovery. I sincerely hope that the actual availability of images is closer to instant than the maximum, though.
If you're trying to do a quiet backdoor and you have the power to compel Apple to assist, the route to take is to simply misuse the keys that are supposed to only go into hardware for attestation, and instead simply use them to forge messages attesting to be running software on hardware that you aren't.
Or just find a bug in the software stack that gives you RCE and use it
Well, your messages have to be congruent with the expected messages from the real hardware, and your fake hardware has to register with the real load balancers to receive user requests.
> RCE
That’s probably the best attack vector, and presumably why Apple is only making binary executables available. Not that that stops RCE.
But even then you can’t pick and choose the users whose data you compromise. It’s still a sev0 problem, but less exploitable for the goals of nation states so less likely to be heavily invested in for exploiting.
Yes, which is why you need the keys that are used to make real hardware. Provided you have those very secret and well protected keys (you are Apple being compelled by the government) that's not an issue.
> and your fake hardware has to register with the real load balancers to receive user requests.
Absolutely, but we're apple in this scenario so that's "easy".
> The process involves multiple Apple teams that cross-check data from independent sources, and the process is further monitored by a third-party observer not affiliated with Apple. At the end, a certificate is issued for keys rooted in the Secure Enclave UID for each PCC node.
So, in your scenario, the in-house certificate issuer is compelled to provide certificates for unverified hardware, which will then be loaded with a parallel software stack that is malicious but reports the attestation ID of a verified stack.
So far, so good. Seems like a lot of people involved, but probably still just tens of people, so maybe possible.
Are you envisioning this being done on every server, so there are no real ones in use? Or a subset? Just for sampling, or also with a way to circumvent user diffusion so you can target specific users?
It's an interesting thought exercise but the complexity of getting anything of real value from this without leaks or errors that expose the program seems pretty small.
> Well, a 89-day "update-and-revert" schedule will take care of those pesky auditors asking too many questions about NSA's backdoor or CCP's backdoor and all that.
As a backdoor I am taking it to mean they can compel assistance from inside of Apple, it's not a hack where they have to break in and hide it from everyone (though certainly they would want to keep it to as few people as possible).
At least in the NSAs case I think it would be reasonable to imagine that they are limited to compromising a subset of the users data. Specific users they've gotten court orders against or something... so yes a subset of nodes and also circumventing user diffusion (which sounds like traffic analysis right up the NSAs alley, or a court order to whatever third party Apple has providing the service).
How does traffic analysis help? The client picks the server to send the query to, and encrypts with that particular server's private key. I guess maybe your have the load balancer identify the target and only provide compromised servers to it? But then every single load balancer has to have the list of targeted individuals and compromised servers, which seems problematic for secrecy at scale.
> But then every single load balancer has to have the list of targeted individuals and compromised servers, which seems problematic for secrecy at scale.
It really doesn't. This seems well within the realms of what you could achieve with a court order without it becoming public.
The transparency & architecture together are intended to be more than enough to publicly detect any major retooling of the system.
I think depending on how this plays out, Apple might manage to earn some of the trust its users have in it, which would be pretty cool! But even cooler will be if we get full chain-of-custody audits, which I think will have to entail opening up some other bits of their stack
In particular, the cloud OS being open-source, if they make good on that commitment, will be incredibly valuable. My main concern right now is that if virtualization is employed in their actual deployment, there could be a backdoor that passes keys from secure enclaves in still-proprietary parts of the OSes running on user devices to a hypervisor we didn't audit that can access the containers. Surely people with more security expertise than me will have even better questions.
Maybe Apple will be responsive to feedback from researchers and this could lead to more of this toolchain being auditable. But even if we can't verify that their sanctioned use case is secure, the cloud OS could be a great step forward in secure inference and secure clouds, which people could independently host or build an independent derivative of
The worst case is still that they just don't actually do it, but it seems reasonably likely they'll follow through on at least that, and then the worst case becomes "Super informative open-source codebase for secure computing at scale just dropped" which is a great thing no matter how the other stuff goes
Yes, the tech industry loves to copy Apple :)
Asahi Linux has a good overview of on-device boot chain security, https://github.com/AsahiLinux/docs/wiki/Apple-Platform-Secur...
> My main concern right now is that if virtualization is employed in their actual deployment, there could be a backdoor that passes keys from secure enclaves in still-proprietary parts of the OSes running on user devices to a hypervisor we didn't audit that can access the containers.
This seems to imply that PCC nodes are bare-metal.Could a PCC node be simulated on iPad Pro with M4 Apple Silicon?
Yes, most technology is built on other technology ;)
> This seems to imply that normal PCC nodes are bare-metal.
I realize that, but there's plausible deniability in it, especially since the modification could also hide the mechanism I've described in some other virtualization context that uses the unmodified image, without the statement being untrue
Eh, it goes both ways. Even Apple devices got widgets eventually :)
AWS Nitro Enclaves [0] come close but of course what Apple has done is productize private compute for its 1b+ macOS & iOS customers!
[0] https://docs.aws.amazon.com/enclaves/latest/user/nitro-encla...
https://aws.amazon.com/bottlerocket
(I wonder if Matt realizes nobody can read his tweets without a X account? Use BlueSky or Masto man)
Edit: here's his thread combined https://threadreaderapp.com/thread/1800291897245835616.html?...
> As best I can tell, Apple does not have explicit plans to announce when your data is going off-device for to Private Compute. You won't opt into this, you won't necessarily even be told it's happening. It will just happen. Magically.
Presumably it will be possible to opt out of AI features entirely, i.e. both on-device and off-device?
Why would a device vendor not have an option for on-device AI only? iOS 17 AI features can be used today without iCloud.
Hopefully Apple uses a unique domain (e.g. *.pcc.apple.com) that can be filtered at the network level.
With a hardened configuration, Apple has world-class device security. In time, remote PCC may prove as robust against real-world threats. Until then, it would be good to retain on-device security policy and choice for remote computation.
Is there a good non-Apple reference for the functions performed by their servers?
This is actually what you have to do now if you don’t want Siri and Mail to leak your address book to Apple.
By disabling Siri and iCloud, or other policies?
With OpenAI calls is different because the privacy point is stronger
> Ok there are probably half a dozen more technical details in the blog post. It’s a very thoughtful design. Indeed, if you gave an excellent team a huge pile of money and told them to build the best “private” cloud in the world, it would probably look like this.
and
> And of course, keep in mind that super-spies aren’t your biggest adversary. For many people your biggest adversary is the company who sold you your device/software. This PCC system represents a real commitment by Apple not to “peek” at your data. That’s a big deal.
I'd prefer things stay on the device but at least this is a big commitment in the right direction - or in the wrong direction but done better than their competitors, I'm not sure which.
You build the machine god all day and somehow find in yourself respect for what Jack let Twitter become?
If you dream of Napoleon, an elephant tooting a horn is a signal to sell.
Create irrational fear about piracy, push privacy focused products and profits as the sheeple promptly fall for this
jokes aside, this is no different from people selling bunker beds, gold, ammunition, crypto, vpns. It is specifically for the set of gullible people who think they and their data is so important. Reality (except for 10,000 people or so) is, most lives and their 'precious' data is worthless. (I'm not talking about SSN, Bank Accounts -- those are well protected by tech cos HN seem to hate on)
Impossible to see any content.
https://ioc.exchange/@matthew_d_green - And he's there BTW :)
False; works fine for me logged out or incognito..
Nitter still works [1]. Also Threadreader (as can be seen linked in Green's tweet).
[0] https://tweetdelete.net/resources/view-twitter-without-accou... [1] https://nitter.poast.org/matthew_d_green
Probably the mainstream Twitter alternative at this point?
a) Top 10 App Store charts in every country.
b) Heavily promoted through Facebook and Instagram.
c) DAUs are higher than X.
But there is far more.
Kara Swisher is on Threads, for instance.
https://nitter.poast.org/matthew_d_green/status/180029189724...
He actually has an active Mastodon account, but this particular story is not on there (yet): https://ioc.exchange/@matthew_d_green
It requires a small, trusted remote observer hardware component, e.g. TCG TPM/DICE, Apple Secure Enclave, Google OpenTitan, Microsoft Pluton.
2021 literature review, https://arxiv.org/abs/2105.02466
2022 HN thread on remote attestation, https://news.ycombinator.com/item?id=32282305
Verifier -> requests a Prover to attest its software state
Prover -> goes into RoT, verifies authenticity of Verifier (and request), computes hash of attested memory region, sends hash digest
Verifier -> receives digest and compares to known hash
> What’s stopping remote endpoint always responding “yes” The attestation code is inside of a RoT, so a bad actor shouldn't be able to call this code, only callable by receiving a request from a Verifier
The remote endpoint has special hardware which keeps secret signing keys (similar to a TLS server's signing keys). The hardware refuses to reveal the private keys, but will sign certain payloads under certain conditions. In addition, Intel or AMD or whoever also has super duper mega secret master keys (similar to a CA's signing keys), which they use to sign the device's signing keys. The certificate signing the device keys is also stored on the device.
So, each time the endpoint is asked to attest its software, it says yes and signs its response with its keys, and it also sends a certificate showing its keys are signed by the master key. That way, the client knows the special hardware really said yes and that Intel or AMD or whoever said that particular special hardware is legit.
The absolute worst acronym for anything even remotely related to personal privacy.
Who is this for? Dont get me wrong I think it's a great effort. This is some A+ nerd stuff right here. It's speaking my languge.
But Im just going to figure out how to turn off "calls home". Cause I dont want it doing this at all.
Is this speaking to me so I tell others "apple is the most secure option"? I don't want to tell others "linux" because I don't want to do tech support for that.
At this point I feel like an old man shouting "Dam you keep your hands off my data".
I'm not a big user of OpenAI's stuff, but if I was going to use any of it, I'd rather use it through Apple's anonymizing layer than going directly to OpenAI.
And then showed that they have a prompt asking if you're ok sending the data to OpenAI. Presumably because despite OpenAI promising not to use your data (a promise apple relayed) OpenAI didn't buy into this new architecture.
I think this goes back to what Steve said in 2010.
https://youtube.com/watch?v=Ij-jlF98SzA
And yes, while the data might not be linked to the user and striped of sensitive data, I could see people not wanting something very personal things to go to OpenAI, even if there should be no link. For example, I wouldn’t want any of my pictures going to OpenAI unless I specifically say it is OK for a given image.
OpenAI provides the chatbot interface we all know.
The PCC cloud serves all of the other integrated AI features like notification prioritization, summarization, semantic search, etc. At least when those can’t be run on device.
I don’t care if the government has access to the data. I just don’t want “bad actors” (scammers, foreign governments, ad-tech companies, insurance companies, etc) to have access to my private data. But i also want the power of LLM’s. Does that sound so far fetched?
I’m a realist. I already EXPECT the US govt has all my data. I don’t like the status quos, but it is what is is.
Don't get me wrong, I've always appreciated apples on device ml/AI features, those have always been powerful, interesting and private but these announcements feel very rushed, it's literally a few weeks after Microsoft's announcements.
They've basically done almost exactly what Microsoft announced with a better UX and a pinky promise about privacy. How are they going to pay for all of that compute? Is this going to be adjusted into the price of the iPhones and MacBook? and then a subscription layer is going to be added to continue paying for it? I don't feel comfortable with the fact that my phone is basically extending it's hardware to the cloud. No matter how "private" it is it's just discomforting to know that apple will be doing inference on things seemingly randomly to "extend" compute capabilities.
Also what on earth is apple high on, integrating a third party API into the OS, how does that even make sense. Google was always a separate app, or a setting in safari, you didn't have Google integrated at an OS level heck you don't have that on Android. It feels very discomforting to know that today my phone could phone home to somewhere other than iCloud.
Hardware sales. Only the latest pro/max models will run these models, everyone else is going to have to upgrade.
I'm trusting Apple more in this case, they have an incentive to keep things private and according to experts they're doing everything they can to do so.
"Indeed, if you gave an excellent team a huge pile of money and told them to build the best “private” cloud in the world, it would probably look like this." - Matthew D. Green
Gemini could claim privacy but I think people would assume that if true, it would make it less effective.
... I suppose this is ultimately a question that will be tested sooner or later in the US.
You can opt into full E2E encryption [1] which makes it nothing, presumably at the cost of some convenience features.
[1] https://support.apple.com/en-us/108756
[1] https://en.m.wikipedia.org/wiki/Apple%E2%80%93FBI_encryption...
[2] https://www.washingtonpost.com/technology/2021/04/14/azimuth...
Apple shills are the worst.
One option is to release a malicious software update, sign it, publish the signature on the public chain, and then simply not release the binaries until after whatever associated gag orders there are (if any) expire. Apple gave themselves a 90 day timeline for this before they'd even be in violation of their promises.
Another option is to use the cryptographic keys used to make the hardware that attests to the software running on it, to simply falsely attest to what software is running. Unless Apple's somehow moved those keys outside of the courts jurisdiction (which means outside of Apple's control in the case of most courts) that should be within the courts power. If they can still create new hardware, it seems likely whoever is making that hardware must still have access to the keys...
Both of these attacks are outside the "threat model" proposed, because they are broad compromises against the entire PCC infrastructure. The fact that they are possible and within the legal systems power... well... why are we advertising this as secure again?
The main value of this whole architecture in my mind isn't actually security though, it's that it's Apple implicitly making the promise that they won't under any circumstance use the data, or let anyone else use the data, for business purposes (not even for running the service itself).
In this option it would be Apple releasing a malicious software update?
> If they can still create new hardware, it seems likely whoever is making that hardware must still have access to the keys...
This option reads like the keys are stored in apple-keys.txt
> Both of these attacks are outside the "threat model" proposed, because they are broad compromises against the entire PCC infrastructure
They mentioned that the in-depth write up will be shared later, might they still address this concern in writing? Your wording makes you sound so certain, but this is just a broad overview. How are you so sure?
Yes, compelled by something like the all writs act (if the US is the one doing the compelling).
> This option reads like the keys are stored in apple-keys.txt
They probably are. That file might live on a CD drive in a safe that requires two people to open it, but ultimately it's a short chunk of binary data that exists somewhere (until it is destroyed)...
> might they still address this concern in writing?
Can I say beyond all doubt that this won't happen? Of course not.
On the first approach I'm quite confident though, because it's both the type of attack they discuss in their initial press release, and pretty fundamental to and explicitly allowed by their model of updating the software.
On the second approach I'm reasonably confident. Like the first issue it's the type of issue that they were discussing in their initial press release. Unlike the first issue it's not something that is explicitly allowed in the model. If Apple can find a way to make the attestation keys irretrievable while still allowing themselves to manufacture hardware I believe they'd do it - I just don't see a method and think it would have warranted a mention if they had one. I tried to insert a level of uncertainty in my original writing on this one because I could be missing a way to solve it.
Ultimately I'd rather over-correct now then have people start thinking this is going to be more secure than it is and then have some fraction of them miss the extremely-likely follow up of "and we could be compelled to work around our security".
https://en.m.wikipedia.org/wiki/Hardware_security_module
“A randomly generated UID is fused into the SoC at manufacturing time. Starting with A9 SoCs, the UID is generated by the Secure Enclave TRNG during manufacturing and written to the fuses using a software process that runs entirely in the Secure Enclave. This process protects the UID from being visible outside the device during manufacturing and therefore isn’t available for access or storage by Apple or any of its suppliers.“
They're making new servers though. Take the keys that are used to vouch for the UIDs in actual secure enclaves, and use them to vouch for the UID in your evil simulated "secure" enclave. Your simulated secure enclave doesn't present as any particular real secure enclave, it just presents as a newly made secure enclave that Apple has vouched for as being a secure enclave.
Law enforcement would need to seize the right server among millions while it's processing your request and perform an attack on it to get the keys before they're gone.
My next question is what happens if/when the attestation keys are stolen.
Interesting to see Swift on Server here!
https://www.swift.org/documentation/server/
https://www.notebookcheck.net/Apple-and-Imagination-strike-G...
https://9to5mac.com/2020/01/01/apple-imagination-agreement/And Apple clearly has made some custom server hardware and slapped a ton of them on a board just to do the PCC stuff.
Let the games begin!
> The server can't afford
What reboot frequency is affordable for PCC nodes?
Since Apple devices are now on the Chinese Governments poopy list, I assume Apple is only meeting some, not all of China's demands. I assume if Apple did everything the Chinese govt wanted, they wouldn't be on the poopy list. Personally I see being on the Chinese govt poopy list as an endorsement that it's probably a net positive for privacy and security compared to those not on the list. :)
Around WhatsApp, it's probably part of the whole compromise mess above. WhatsApp now does E2E and that's something China is not a fan of, so it's probably China's doing that it's not in the app store in China any more. Apple is just following the laws China forces them to follow.
It should be noted I've never been to China(yet) and have zero 1st hand knowledge.
The govt basically requires total access doesn't it? I mean every govt basically wants it, and the US has tried many times, but so far hasn't quite gotten complete access everywhere.
One key part though will be the remote attestation that the servers are actually running what they say they're running. Without any access to the servers, how do we do that? Am I correctly expecting that that part remains a "trust me bro" situation?
>While we’re publishing the binary images of every production PCC build, to further aid research we will periodically also publish a subset of the security-critical PCC source code.
I expect that they'll publish the attestation source code.
But, basically what will happen is the Verifier will request a certain memory region to be attested, then that region will be hashed and the digest will be sent back to the Verifier. If the memory is different from what is expected, the hash digest will NOT match.
70,000+ Apple user accounts are surveilled in this manner every year.
If pressured by the government, Apple can simply change the client software to loosen the attestation requirements for private compute. And that would be the most inconspicuous choice.
"oh it's our dev version ? what's the problem ? we need data access for troubleshooting"
That's why you use them just as dumb pipes forwarding encrypted data traffic from one place to another.
No SMS, no phone calls if you can avoid it.
How often do PCC servers reboot and wipe the temporary encryption key?
It doesn't actually say there is no persistent storage, it says that the compute node will not store it for longer than the request. There's nothing to stop the data coming from a datastore outside of the "PCC" in another part of apple's infrastructure.
Apple deserves credit for correctly analyzing their threat model and designing their system accordingly.
Would this work in some authoritarian countries? No and Apple doesn't care. Would it work in a western one? Maybe.
Also, got any links for interesting ZKML papers/projects?