> its purpose is to request a region of memory that is mapped only for the calling process and inaccessible to anybody else, including the kernel.
Easy fix: at the next context switch, the kernel overwrites the code of the process with its own code that copies the contents of the "secret" memory to somewhere the kernel can read it, and then executes its own code.
I guess I don't understand the threat model in which having data protected from the kernel is helpful.
If the idea is that the kernel is malicious, then it's not going to work in the first place (e.g. the kernel could just fail to honour the MAP_EXCLUSIVE flag in the first place). If the idea is that the kernel is initially trustworthy but gets compromised some time after the memory is made "exclusive", then it can execute its own arbitrary code in that process's context and get the contents that way.
The threat model is basically another process compromising the kernel. Think security systems in shared environments.
It's supposed to be hardware based. If process A requests memory with MAP_EXCLUSIVE, and it verifies that it got it that way, the hardware is supposed to make sure the memory is only available to that particular process. No amount of context switching is allowing the kernel to read it, nor could should it be able to modify the running code in a process if it's properly hardened (e.g. put the code to run in the protected area). It pose some interesting challenges, but it seems quite possible
This seems weird - the hardware has no notion of processes, that's a construct implemented by the kernel. It can enforce a separation of privileges between kernel and non-kernel, but that's about it.
Hardware gets a flag that says "am I in privileged mode now?" and it does not (and fundamentally can not) know to whether the currently running code belongs to process A or something else, or to whom a particular memory region belongs - as far as hardware is concerned, there's no "whom" at all, that's an abstract construct arbitrarily introduced and maintained by the OS.
There's no reason hardware can't be given a notion of processes, and used to set boundaries between each other and the kernel. It's not an untested idea.
While the current crop of mainstream CPUs does not support anything like it, the idea is old. Basically the CPU need to have hardware support for a PCB and be solely responsible for context switching, and the already complex MMU needs to be context aware.
I'm pretty sure we've seen specialized CPUs in the past that did exactly that.
I think the threat model is a buggy kernel - or a kernel insuffiently hardened against buggy processors with speculative execution side channels - with a malicious process.
E.g. if syscall S reads kernel memory without properly checking pointers/offsets (maybe GCC optimized them out), then process A can possibly use that to steal process B's secrets. Or maybe the kernel does check the pointers/offsets, but isn't hardened against yet another speculative execution side channel attack your processor is vulnerable to, so leaks some information via timing information.
MAP_EXCLUSIVE breaks that exploit, and to unbreak it you need both syscall S and another buggy syscall (or maybe another row hammer attack) - to write kernel memory - and to successfully leverage that into tricking the kernel into remapping process B's pages.
No panacea, but I could see it rasing the bar in exploit difficulty as part of a defense in depth strategy.
It does seem weird to protect user space memory from the kernel but not the memory of kernel subsystems from other kernel subsystems.
But at that point we're at a microkernel design that needs context switches between kernel processes... Good, I'd welcome that. This feature just implements a level of security that the whole kernel needs to catch up to before it makes sense.
I think you are getting at the same question Alan Cox had. As quoted in the article:
IMHO the question is what is the actual semantic here. What are you asking for ? Does it mean "at any cost", what does it guarantee (100% or statistically), what level of guarantee is acceptable, what level is -EOPNOTSUPP or similar ?
I think different semantics as Cox mentions would correspond to different threat models. With the right hardware support, it could mitigate against a kernel that falls under an attacker's complete control after the memory has been allocated. With less support it could mitigate against an attacker who can exploit read accesses at kernel privilege. But the details seem a bit open still...
I am thinking windows, so perhaps things are a bit different in linux, but most of your drivers run in kernel mode, that's a lot of code contributed by a lot of different companies that is fully trusted with all you can eat access.
But it still will do if the nvidia graphics driver can override the code of your process, even if the nvidia graphics driver can't directly read the protected area of memory.
It could also just access the physical memory via the direct map (the 1:1 mapping of physical to kernel virtual address space). In fact, this is kind of easier now, since you're guaranteed to find all the interesting stuff in physical memory since MAP_EXCLUSIVE memory cannot be paged out.
Ugh, I missed that. But there is still nothing preventing a malicious (nvidia) driver from re-adding the pages back. In fact, since they are missing from the direct map, it points anything malicious right at the interesting physical addresses..
I can't figure out why there doesn't seem to be a commercial market for something like CHERI ( https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/ ) ... even if it was considerably slower than top end CPUs there are plenty of applications that don't care (and find chips like arm a8 or atom fast enough) but where high security is paramount.
Maybe Intel still remembers the pain of iAXP 432 but that doesn't explain the rest of the industry.
It'd be interesting to have the kernel transparently encrypt the secret page whenever the process is put to sleep or waiting for IO, so that the contents are only readable when the process is active. Could be a nice, if not perfect, layer of protection against having the secrets read from other processes. IIRC there's a way to store crypto keys in CPU registers, so that they won't be available in a RAM dump.
That's OK if your threat model trusts the kernel and only wants to protect against speculative execution attacks within the application itself. While the application is running, they keys will only ever be accessed from registers -- context switches are transparent to the application.
SEV is an extension to the AMD-V architecture which supports running encrypted virtual machine (VMs) under the control of KVM. Encrypted VMs have their pages (code and data) secured such that only the guest itself has access to the unencrypted version. Each encrypted VM is associated with a unique encryption key; if its data is accessed to a different entity using a different key the encrypted guests data will be incorrectly decrypted, leading to unintelligible data.
IBM z-mainframes have complete hw assisted total encryption in z14 and later. Keys are not visible to hypervisor, OS or application (code and data) and it's authenticated. Databases, datasets and network traffic can be transparently encrypted using hardware without altering applications.
This is a SW-only solution, and therefore is probably also buggy, but can be patched and only accessed via ring < 0. Only via yet unknown kernel bugs (patchable) or the mythical ME or ASP/PSP backdoors.
How about where you need some data that is available to all processes running under the same UID, but not to other processes? How do people handle that nowadays, especially on a generic server with no special cryptographic or security hardware?
To make it more interesting, assume that (1) the processes do not have a common ancestor running as that UID, and (2) if all processes with that UID exit, the secret should still be available if any new processes are started with that UID, and (3) the secret must be forgotten when the machine shuts down or loses power.
Assume that the secret should be safe from attackers who can run arbitrary code as other non-root users, who can force the machine to reboot, and who can steal the machine itself. (Stealing the machine causes it to lose power--no need to worry about a George Costanza/"Frogger" scenario).
For example, imagine a service implemented as a CGI that needs the keys for an encrypted database it uses, or needs the password for a service that it needs to access, using Apache using suEXEC to run this service's CGI under a dedicated UID.
It would seem that you need some kind of persistent storage. The first thing that comes to mind is a file. It can be owned by the UID of the CGI, and mode 0400. This doesn't meet the requirement of the secret being forgotten on shut down, though.
Would good old fashioned System V shared memory work for this? It has ownership and access control similar to files, so a shared memory segment owned by UID and mode 0600 should only be accessible to processes with the right UID (and root). It goes away when the machine shuts down. Linux's System V shared memory implementation adds a flag you can set via shmctl to keep it from swapping it.
It sounds like you want a file on a ram disk. Linux creates a ram disk by default in /dev/shm. You could create a file or directory there only accessible by a specific UID.
Since MAP_EXCLUSIVE pages are implicitly MAP_LOCKED & MAP_PRIVATE, how much are they expected to be used in practice on a given system?
If it's an edge use case, couldn't they just use huge pages for such mappings to avoid the whole page-splitting performance problem in the kernel's linear mapping? Or only resort to splitting after some amount of huge pages have been wasted on exclusive mappings?
41 comments
[ 3.1 ms ] story [ 105 ms ] threadEasy fix: at the next context switch, the kernel overwrites the code of the process with its own code that copies the contents of the "secret" memory to somewhere the kernel can read it, and then executes its own code.
I guess I don't understand the threat model in which having data protected from the kernel is helpful.
If the idea is that the kernel is malicious, then it's not going to work in the first place (e.g. the kernel could just fail to honour the MAP_EXCLUSIVE flag in the first place). If the idea is that the kernel is initially trustworthy but gets compromised some time after the memory is made "exclusive", then it can execute its own arbitrary code in that process's context and get the contents that way.
It's supposed to be hardware based. If process A requests memory with MAP_EXCLUSIVE, and it verifies that it got it that way, the hardware is supposed to make sure the memory is only available to that particular process. No amount of context switching is allowing the kernel to read it, nor could should it be able to modify the running code in a process if it's properly hardened (e.g. put the code to run in the protected area). It pose some interesting challenges, but it seems quite possible
Hardware gets a flag that says "am I in privileged mode now?" and it does not (and fundamentally can not) know to whether the currently running code belongs to process A or something else, or to whom a particular memory region belongs - as far as hardware is concerned, there's no "whom" at all, that's an abstract construct arbitrarily introduced and maintained by the OS.
There's no reason hardware can't be given a notion of processes, and used to set boundaries between each other and the kernel. It's not an untested idea.
I'm pretty sure we've seen specialized CPUs in the past that did exactly that.
E.g. if syscall S reads kernel memory without properly checking pointers/offsets (maybe GCC optimized them out), then process A can possibly use that to steal process B's secrets. Or maybe the kernel does check the pointers/offsets, but isn't hardened against yet another speculative execution side channel attack your processor is vulnerable to, so leaks some information via timing information.
MAP_EXCLUSIVE breaks that exploit, and to unbreak it you need both syscall S and another buggy syscall (or maybe another row hammer attack) - to write kernel memory - and to successfully leverage that into tricking the kernel into remapping process B's pages.
No panacea, but I could see it rasing the bar in exploit difficulty as part of a defense in depth strategy.
But at that point we're at a microkernel design that needs context switches between kernel processes... Good, I'd welcome that. This feature just implements a level of security that the whole kernel needs to catch up to before it makes sense.
IMHO the question is what is the actual semantic here. What are you asking for ? Does it mean "at any cost", what does it guarantee (100% or statistically), what level of guarantee is acceptable, what level is -EOPNOTSUPP or similar ?
I think different semantics as Cox mentions would correspond to different threat models. With the right hardware support, it could mitigate against a kernel that falls under an attacker's complete control after the memory has been allocated. With less support it could mitigate against an attacker who can exploit read accesses at kernel privilege. But the details seem a bit open still...
An exploit in your closed-source nvidia graphics driver should not get access to your private keys.
After all, SPARC ADI is used in production, and ARM as well (partially with iOS, enforced in future Android devices).
Maybe Intel still remembers the pain of iAXP 432 but that doesn't explain the rest of the industry.
There's also Dover Microsystems, which is trying to commercialize similar-in-spirit processor extensions originally developed at Draper Labs.
https://news.ycombinator.com/item?id=11022275
SEV is an extension to the AMD-V architecture which supports running encrypted virtual machine (VMs) under the control of KVM. Encrypted VMs have their pages (code and data) secured such that only the guest itself has access to the unencrypted version. Each encrypted VM is associated with a unique encryption key; if its data is accessed to a different entity using a different key the encrypted guests data will be incorrectly decrypted, leading to unintelligible data.
https://github.com/vschiavoni/sgx-papers
This is a SW-only solution, and therefore is probably also buggy, but can be patched and only accessed via ring < 0. Only via yet unknown kernel bugs (patchable) or the mythical ME or ASP/PSP backdoors.
https://semiaccurate.com/2017/06/22/amds-epyc-major-advance-...
To make it more interesting, assume that (1) the processes do not have a common ancestor running as that UID, and (2) if all processes with that UID exit, the secret should still be available if any new processes are started with that UID, and (3) the secret must be forgotten when the machine shuts down or loses power.
Assume that the secret should be safe from attackers who can run arbitrary code as other non-root users, who can force the machine to reboot, and who can steal the machine itself. (Stealing the machine causes it to lose power--no need to worry about a George Costanza/"Frogger" scenario).
For example, imagine a service implemented as a CGI that needs the keys for an encrypted database it uses, or needs the password for a service that it needs to access, using Apache using suEXEC to run this service's CGI under a dedicated UID.
It would seem that you need some kind of persistent storage. The first thing that comes to mind is a file. It can be owned by the UID of the CGI, and mode 0400. This doesn't meet the requirement of the secret being forgotten on shut down, though.
Would good old fashioned System V shared memory work for this? It has ownership and access control similar to files, so a shared memory segment owned by UID and mode 0600 should only be accessible to processes with the right UID (and root). It goes away when the machine shuts down. Linux's System V shared memory implementation adds a flag you can set via shmctl to keep it from swapping it.
Security can't be a best-effort platform-dependent QoI issue.
What could work would be several tiers with certain minimum guarantees that will either be fulfilled 100% or EOPNOTSUPP.
If a developer doesn't know what level of security they need or want, the feature is not for them.
If it's an edge use case, couldn't they just use huge pages for such mappings to avoid the whole page-splitting performance problem in the kernel's linear mapping? Or only resort to splitting after some amount of huge pages have been wasted on exclusive mappings?