One of the coolest things about gVisor to me is that it's the ultimate implication of core computer engineering concepts like "the OS is just software" and "network traffic is just bytes". It's one thing to learn these ideas in theory, but it's another altogether to be able to play with an entire network stack in userspace and inject arbitrary behavior in the OSI stack. It's also been cool to see what companies like Fly.io and Tailscale can do with complete flexibility in the network, enabled by tools like gVisor.
I love the concept of gVisor; it's surprising to me that it hasn't seemingly gotten more real world traction— even GHA is booting you a fresh machine for every build when probably 80%+ of them could run just fine in a gVisor sandbox.
I'd be curious to hear from someone at Google if gVisor gets a ton of internal use there, or it really was built mainly for GCP/GKE
I have wondered for a long time why we don’t see more networking in userspace for high security applications that don’t require high performance. I guess the answer is just that Linux has enough features now to hook into the kernel with userspace code that it usually isn’t necessary to move the whole IP and TCP stacks out.
gVisor also has a complete userspace networking stack that you can pull in, which makes it a lot easier to do some neat things like run an HTTP server responding to packets intercepted via eBPF and sent to an AF_XDP socket, which would otherwise be a pain.
We've run gVisor for over 2 years at Modal, and it's been a huge unlock for us. We get a secure sandbox with GPU support that can run on VMs. Just recently it allowed us to checkpoint/restore containers AND its GPUs[1].
gVisor's achilles heel is it's missing or inaccurate syscalls, but the gVisor team is first class in responding to Github issues so it's really quite manageable in practice if you know how to debug and hack on a userspace kernel.
I've been working on KVMServer [2] recently which uses TinyKVM to run existing Linux server applications by intercepting epoll calls. While there is a small overhead to crossing the KVM boundary to handle sys calls we get the ability to quickly reset the state of the guest. This means we can provide per-request isolation with an order of magnitude less overhead than alternative approaches like forking a process or even spinning up a v8 isolate.
13 comments
[ 2.7 ms ] story [ 25.6 ms ] thread[0] - https://unikraft.org
I'd be curious to hear from someone at Google if gVisor gets a ton of internal use there, or it really was built mainly for GCP/GKE
gVisor's achilles heel is it's missing or inaccurate syscalls, but the gVisor team is first class in responding to Github issues so it's really quite manageable in practice if you know how to debug and hack on a userspace kernel.
1. https://news.ycombinator.com/item?id=44747116
I am way out of my depth here, but can anyone make a comparison with the "micro virtual machines" concept?
EDIT: It seems that gVisor has a KVM mode too. https://gvisor.dev/docs/architecture_guide/platforms/#kvm
I've been working on KVMServer [2] recently which uses TinyKVM to run existing Linux server applications by intercepting epoll calls. While there is a small overhead to crossing the KVM boundary to handle sys calls we get the ability to quickly reset the state of the guest. This means we can provide per-request isolation with an order of magnitude less overhead than alternative approaches like forking a process or even spinning up a v8 isolate.
[1] Previous discussion: https://news.ycombinator.com/item?id=43358980
[2] https://github.com/libriscv/kvmserver