I can see why he would make that argument. When you don't have any process isolation, a software fault means your entire stack is untrustworthy. The network driver, fs driver might be corrupted, so nothing you write to disk or send over the network can be trusted.
You also have to recreate your entire userspace and debugging tools to work in this environment, and testing or even running or debugging your software is also a headache.
I've been using unikraft (https://unikraft.org/) unikernels for a while and the startup times are quite impressive (easily sub-second for our Rust application).
Projects like this and Docker make me seriously wonder where software engineering is going. Don't get me wrong, I don't mean to criticize Docker or Toro in parcicular. It's the increasing dependency on such approaches that bothers me.
Docker was conceived to solve the problem of things "working on my machine", and not anywhere else. This was generally caused by the differences in the configuration and versions of dependencies. Its approach was simple: bundle both of these together with the application in unified images, and deploy these images as atomic units.
Somewhere along the lines however, the problem has mutated into "works on my container host". How is that possible? Turns out that with larger modular applications, the configuration and dependencies naturally demand separation. This results in them moving up a layer, in this case creating a network of inter-dependent containers that you now have to put together for the whole thing to start... and we're back to square one, with way more bloat in between.
Now hardware virtualization. I like how AArch64 generalizes this: there are 4 levels of privilege baked into the architecture. Each has control over the lower and can call up the one immediately above to request a service. Simple. Let's narrow our focus to the lowest three: EL0 (classically the user space), EL1 (the kernel), EL2 (the hypervisor). EL0, in most operating systems, isn't capable of doing much on its own; its sole purpose is to do raw computation and request I/O from EL1. EL1, on the other hand, has the powers to directly talk to the hardware.
Everyone is happy, until the complexity of EL1 grows out of control and becomes a huge attack surface, difficult to secure and easy to exploit from EL0. Not good. The naive solution? Go a level above, and create a layer that will constrain EL1, or actually, run multiple, per-application EL1s, and punch some holes through for them to still be able to do the job—create a hypervisor. But then, as those vaguely defined "holes", also called system calls and hyper calls, grow, won't so the attack surface?
Or in other words, with the user space shifting to EL1, will our hypervisor become the operating system, just like docker-compose became a dynamic linker?
> Now hardware virtualization. I like how AArch64 generalizes this: there are 4 levels of privilege baked into the architecture. Each has control over the lower and can call up the one immediately above to request a service. Simple. Let's narrow our focus to the lowest three: EL0 (classically the user space), EL1 (the kernel), EL2 (the hypervisor). EL0, in most operating systems, isn't capable of doing much on its own; its sole purpose is to do raw computation and request I/O from EL1. EL1, on the other hand, has the powers to directly talk to the hardware.
> Everyone is happy, until the complexity of EL1 grows out of control and becomes a huge attack surface, difficult to secure and easy to exploit from EL0. Not good. The naive solution? Go a level above, and create a layer that will constrain EL1, or actually, run multiple, per-application EL1s, and punch some holes through for them to still be able to do the job—create a hypervisor. But then, as those vaguely defined "holes", also called system calls and hyper calls, grow, won't so the attack surface?
(Disclaimer: this is all very far from my area of expertise, so some of the below may be wrong/misleading)
Nobody can agree whether microkernels or monolithic kernel are "better" in general, but most people seem to agree that microkernels are better for security [0], with seL4 [1] being a fairly strong example. But microkernels are quite a bit slower, so in the past when computers were slower, microkernels were noticeably slower, and security was less of a concern than it is now, so essentially every mainstream operating system in the 90s used some sort of monolithic kernel. These days, people might prefer different security–performance tradeoffs, but we're still using kernels designed in the 90s, so it isn't easy to change this any more.
Moving things to the hypervisor level lets us gain most of the security benefits of microkernels while maintaining near-perfect compatibility with the classic Linux/NT kernels. And the combination of faster computers (performance overheads therefore being less of an issue), more academic research, and high-quality practical implementations [2] means that I don't expect the current microkernel-style hypervisors to gain much new attack surface.
This idea isn't without precedent either—Multics (from the early 70s) was partially designed around security, and used a similar design with hardware-enforced hierarchical security levels [3]. Classic x86 also supports 4 different "protection rings" [4], and virtualisation plus nested virtualisation adds 2 more, but nothing ever used rings 1 and 2, so adding virtualisation just brings us back to the same number of effective rings as the original design.
I don't want the observability of my applications to be bound by themselves, it's kind of a real pain. I'm all for microvm images without excess dependencies, but coupling the kernel and diagnostic tools to rapidly developing application code can be a real nightmare as soon as the sun stops shining.
I use LXD + LXC, wondering if this is worth trying or if the overhead of accessing (network, etc) would be too much to deal with/care about.
Also always a little wary of projects that have bad typos or grammar problems in a README - in particular on one of the headings (thought it's possible these are on purpose?). But that's just me :\",
22 comments
[ 3.4 ms ] story [ 59.6 ms ] threadfile sharing is complex too it seems
would be good to see a benchmark or something showing where it shines
Neat.
so, generally at the edge (gateways, shims, protocol boundaries)
[0]: https://www.tritondatacenter.com/blog/unikernels-are-unfit-f...
You also have to recreate your entire userspace and debugging tools to work in this environment, and testing or even running or debugging your software is also a headache.
Docker was conceived to solve the problem of things "working on my machine", and not anywhere else. This was generally caused by the differences in the configuration and versions of dependencies. Its approach was simple: bundle both of these together with the application in unified images, and deploy these images as atomic units.
Somewhere along the lines however, the problem has mutated into "works on my container host". How is that possible? Turns out that with larger modular applications, the configuration and dependencies naturally demand separation. This results in them moving up a layer, in this case creating a network of inter-dependent containers that you now have to put together for the whole thing to start... and we're back to square one, with way more bloat in between.
Now hardware virtualization. I like how AArch64 generalizes this: there are 4 levels of privilege baked into the architecture. Each has control over the lower and can call up the one immediately above to request a service. Simple. Let's narrow our focus to the lowest three: EL0 (classically the user space), EL1 (the kernel), EL2 (the hypervisor). EL0, in most operating systems, isn't capable of doing much on its own; its sole purpose is to do raw computation and request I/O from EL1. EL1, on the other hand, has the powers to directly talk to the hardware.
Everyone is happy, until the complexity of EL1 grows out of control and becomes a huge attack surface, difficult to secure and easy to exploit from EL0. Not good. The naive solution? Go a level above, and create a layer that will constrain EL1, or actually, run multiple, per-application EL1s, and punch some holes through for them to still be able to do the job—create a hypervisor. But then, as those vaguely defined "holes", also called system calls and hyper calls, grow, won't so the attack surface?
Or in other words, with the user space shifting to EL1, will our hypervisor become the operating system, just like docker-compose became a dynamic linker?
> Everyone is happy, until the complexity of EL1 grows out of control and becomes a huge attack surface, difficult to secure and easy to exploit from EL0. Not good. The naive solution? Go a level above, and create a layer that will constrain EL1, or actually, run multiple, per-application EL1s, and punch some holes through for them to still be able to do the job—create a hypervisor. But then, as those vaguely defined "holes", also called system calls and hyper calls, grow, won't so the attack surface?
(Disclaimer: this is all very far from my area of expertise, so some of the below may be wrong/misleading)
Nobody can agree whether microkernels or monolithic kernel are "better" in general, but most people seem to agree that microkernels are better for security [0], with seL4 [1] being a fairly strong example. But microkernels are quite a bit slower, so in the past when computers were slower, microkernels were noticeably slower, and security was less of a concern than it is now, so essentially every mainstream operating system in the 90s used some sort of monolithic kernel. These days, people might prefer different security–performance tradeoffs, but we're still using kernels designed in the 90s, so it isn't easy to change this any more.
Moving things to the hypervisor level lets us gain most of the security benefits of microkernels while maintaining near-perfect compatibility with the classic Linux/NT kernels. And the combination of faster computers (performance overheads therefore being less of an issue), more academic research, and high-quality practical implementations [2] means that I don't expect the current microkernel-style hypervisors to gain much new attack surface.
This idea isn't without precedent either—Multics (from the early 70s) was partially designed around security, and used a similar design with hardware-enforced hierarchical security levels [3]. Classic x86 also supports 4 different "protection rings" [4], and virtualisation plus nested virtualisation adds 2 more, but nothing ever used rings 1 and 2, so adding virtualisation just brings us back to the same number of effective rings as the original design.
[0]: https://en.wikipedia.org/wiki/Microkernel#Security
[1]: https://sel4.systems/
[2]: https://xenproject.org/projects/hypervisor/
[3]: https://en.wikipedia.org/wiki/Multics#Project_history
[4]: https://en.wikipedia.org/wiki/Protection_ring
Also always a little wary of projects that have bad typos or grammar problems in a README - in particular on one of the headings (thought it's possible these are on purpose?). But that's just me :\",
What use cases would Toro fit? pros and cons ?