I skimmed the paper, but the premise makes sense in some scenarios. I don't think unikernels have quite taken off as they were hyped a few years ago, but in general, micro VMs are quite more usable nowadays.
I recently gave Firecracker a try, via the friendlier interface of Ignite[1], and was pleasantly surprised how quick it was to start a VM from a Docker image, and how little resources it used. The UX is pretty much on par with Docker, yet gives much more flexibility, control, and of course, security.
I wish there was a competitor to QubesOS that uses KVM, and is a bit friendlier and more polished. Is there such a distro?
What do you trust more: kernel-mode virtio drivers or the limited subset of kernel-mode /dev drivers and user-mode APIs exposed to containers? It's not at all clear to me that a VM is safer than a (well-designed) containerization system, such as freebsd jails.
Than what? In most of my containers I only expose extremely basic devices like `/dev/random`, which are extremely unlikely to have vulnerabilities. Virtio drivers are very complicated and very privileged.
it's not a matter of what you expose. VMs are simply more isolated than containers.
VM isolation happens in hardware -- the processor, and is supported by the processor; container isolation happens in the OS kernel, and the hardware can not offer any features to containers that it can offer to virtual machines.
> VM isolation happens in hardware -- the processor, and is supported by the processor
This is also meaningless, or at least doesn't mean what you think it does. Container isolation also happens "in hardware", in the sense that both are facilitated by e.g. features in the MMU. VMs may or may not take advantage of some additional hardware features (e.g. hardware-assisted HBA virtualization), but most of these do not necessarily provide any security benefit, and in many cases expose additional attack surface area that would not be accessible from inside a container.
The only difference from a security standpoint is which APIs are exposed to processes running inside the guest environment. Both VMs (with practical peripherals) and containers have access to scary stuff running in kernel mode. It's mostly a disjoint set of stuff, and it's not clear to me which set is actually preferable from a security standpoint.
I don't know what you're so angry about, but it's dripping off of the page. no one is angry here but you. why?
CPUs do not have hardware containerization support. they DO have hardware virtualization support.
containers know they are containers, necessarily. virtual machines do not know they are virtual machines unless the platform they see as physical hardware tells them they are a virtual machine, which is not a requirement for virtualization.
CPUs which support virtualization extensions isolate virtual machines from each other at a hardware level. the CPU, at a hardware level, knows about each virtual machine, where its memory is, and what each virtual machine is doing.
for containers, that's all the kernel, and only the kernel. the containers talk through the host kernel to the outside world, and they all share the same OS kernel. that is arguably the opposite of isolation.
a virtual machine host (in reality a sibling virtual machine with special privileges) can run operating system A, and there can be x number of sibling VMs, all running different operating systems; Linux, MacOS, Windows, FreeBSD, BeOS, whatever. they are separated all the way down to the CPU and RAM.
the managing virtual machine cannot see inside its sibling virtual machines normally, and would require escalation to CPU ring -2 (or is it -3?) to become a superuser in the eyes of the hypervisor.
I don't know how to be more clear. if you are still in disagreement, maybe you are unaware of type-1 vs. type-2 hypervisors, and/or you're talking about one while I am talking about another.
others have given up on this conversation and I now join them.
Why do you think I'm angry? I'm just pointing out that you are saying things that don't carry semantic content without prevaricating about it. Apologies if that comes off as hostile - I don't really have time to write this in a gentler way.
> CPUs do not have hardware containerization support. they DO have hardware virtualization support.
I already explained why this is wrong with specific examples.
> containers know they are containers, necessarily
User-mode processes need not know they are containerized, and this has nothing to do with security.
> virtual machines do not know they are virtual machines unless...
This is in practice not true, even with fully simulated hardware. Look up "VM detection".
> the CPU, at a hardware level, knows about each virtual machine, where its memory is, and what each virtual machine is doing
That is not how virtualization works at all. Once again, not trying to be hostile, just don't have time to say this gently. This is the kind of thing people say when they haven't actually written code against virtualization extensions. The CPU does not "know" anything about what VMs are running - that is not an abstraction that is reified at the hardware level.
> and they all share the same OS kernel
With access to kernel resources gated via a specific set of APIs - same as with a VM. Again, the question is just which set of APIs you trust.
> they are separated all the way down to the CPU and RAM.
Another meaningless cliche that suffices for a basic mental model of VMs, but insufficient for security reasoning
> the managing virtual machine cannot see inside its sibling virtual machines normally, and would require escalation
Precisely the same statement applies for containers.
> I don't know how to be more clear.
There is no need - it is already perfectly clear that you are simply using an insufficiently precise approximative model of how VMs work.
This is why it's not an obvious win for containers over VMs. Existing low-assurance operating systems aren't good enough to trust the basic set of usermode syscalls to be totally privesc-free. But I guarantee you that the set of syscalls permitted to a jailed process on e.g. freebsd has been pretty thoroughly worked over. If a PE exists, it's going to be expensive. It's possibly a smaller effective attack surface than virtio networking, display, disk, etc.
13 comments
[ 2.8 ms ] story [ 41.3 ms ] threadI recently gave Firecracker a try, via the friendlier interface of Ignite[1], and was pleasantly surprised how quick it was to start a VM from a Docker image, and how little resources it used. The UX is pretty much on par with Docker, yet gives much more flexibility, control, and of course, security.
I wish there was a competitor to QubesOS that uses KVM, and is a bit friendlier and more polished. Is there such a distro?
[1]: https://github.com/weaveworks/ignite
VM isolation happens in hardware -- the processor, and is supported by the processor; container isolation happens in the OS kernel, and the hardware can not offer any features to containers that it can offer to virtual machines.
This doesn't mean anything - it is just a cliché.
> VM isolation happens in hardware -- the processor, and is supported by the processor
This is also meaningless, or at least doesn't mean what you think it does. Container isolation also happens "in hardware", in the sense that both are facilitated by e.g. features in the MMU. VMs may or may not take advantage of some additional hardware features (e.g. hardware-assisted HBA virtualization), but most of these do not necessarily provide any security benefit, and in many cases expose additional attack surface area that would not be accessible from inside a container.
The only difference from a security standpoint is which APIs are exposed to processes running inside the guest environment. Both VMs (with practical peripherals) and containers have access to scary stuff running in kernel mode. It's mostly a disjoint set of stuff, and it's not clear to me which set is actually preferable from a security standpoint.
CPUs do not have hardware containerization support. they DO have hardware virtualization support.
containers know they are containers, necessarily. virtual machines do not know they are virtual machines unless the platform they see as physical hardware tells them they are a virtual machine, which is not a requirement for virtualization.
CPUs which support virtualization extensions isolate virtual machines from each other at a hardware level. the CPU, at a hardware level, knows about each virtual machine, where its memory is, and what each virtual machine is doing.
for containers, that's all the kernel, and only the kernel. the containers talk through the host kernel to the outside world, and they all share the same OS kernel. that is arguably the opposite of isolation.
a virtual machine host (in reality a sibling virtual machine with special privileges) can run operating system A, and there can be x number of sibling VMs, all running different operating systems; Linux, MacOS, Windows, FreeBSD, BeOS, whatever. they are separated all the way down to the CPU and RAM.
the managing virtual machine cannot see inside its sibling virtual machines normally, and would require escalation to CPU ring -2 (or is it -3?) to become a superuser in the eyes of the hypervisor.
I don't know how to be more clear. if you are still in disagreement, maybe you are unaware of type-1 vs. type-2 hypervisors, and/or you're talking about one while I am talking about another.
others have given up on this conversation and I now join them.
> CPUs do not have hardware containerization support. they DO have hardware virtualization support.
I already explained why this is wrong with specific examples.
> containers know they are containers, necessarily
User-mode processes need not know they are containerized, and this has nothing to do with security.
> virtual machines do not know they are virtual machines unless...
This is in practice not true, even with fully simulated hardware. Look up "VM detection".
> the CPU, at a hardware level, knows about each virtual machine, where its memory is, and what each virtual machine is doing
That is not how virtualization works at all. Once again, not trying to be hostile, just don't have time to say this gently. This is the kind of thing people say when they haven't actually written code against virtualization extensions. The CPU does not "know" anything about what VMs are running - that is not an abstraction that is reified at the hardware level.
> and they all share the same OS kernel
With access to kernel resources gated via a specific set of APIs - same as with a VM. Again, the question is just which set of APIs you trust.
> they are separated all the way down to the CPU and RAM.
Another meaningless cliche that suffices for a basic mental model of VMs, but insufficient for security reasoning
> the managing virtual machine cannot see inside its sibling virtual machines normally, and would require escalation
Precisely the same statement applies for containers.
> I don't know how to be more clear.
There is no need - it is already perfectly clear that you are simply using an insufficiently precise approximative model of how VMs work.