The Great Virtualization-Container-Sandbox Race

A comparison of similar sandboxing tech implemented by cloud giants that all go beyond docker and are meant to provide the speed of traditional containers while getting rid of the latency full fledged virtualization in multi tenancy environments. Hopefully we can evaluate these options and pick one we can ship with Whonix/KS. All these options take security to the next level beyond MAC framewrks and approahces like Firejail. Any of these are an improvement becuase they can reduce the attack surface than that of a full linux kernel running inside a VM as is currently the case, These projects have had some time to mature and be adopted in realworld usecases that I would be confident using them now. This post is a brain dump and needs extensive processing of the latest info to make a good decision. My preliminary understanding may be mistaken :

gVisor is a kernel emulation layer. it implements most of the Linux syscall in userspace, thereby preventing kernel exploitation (an obvious target for attackers in a rootful container). runq run a container image as a VM using qemu. Instead of using the namespacing API of the host kernel, it spawn a qemu VM image with a given kernel and run the application directly on that VM. Firecracker is VMM using KVM to manage microVMs, i.e. VMs that only virtualize the minimum set of devices to run common apps. Kata Containers is like runq but uses Firecracker to run containers in a (micro)VM. –
Margaret Bloom
Commented Dec 12, 2024 at 16:32
1
Firecracker was invented by Amazon to speed up its Lambda service. A full blown VM is not necessary to run a JS function, instead they load a small kernel with a minimal environment to just read from a storage device and network. On top of this, Kata Containers build an OCI implementation on the lines of runq. They are all different products and you must decide what’s best for your self. Sometime it’s a matter of taste. –
Margaret Bloom
Commented Dec 12, 2024 at 16:35
IIUC one difference between Kata/Firecracker vs. runq (and runCVM) is that the latter execute the container workload directly inside the VM, whereas the former start another runc container inside the VM. This additional layer might increase security (slightly) but make it harder to e.g. nest containers / spawn child containers. –
balu
Commented Aug 6 at 18:26


Gvisor

Gvisor implements a small custom kernel simulating a subset of the syscall interface. It provides a secrutiy level between LXC and full VMs. Kata and Firecracker provide full VM level security but require nested VTx to work. Firecracker is for headless application deployments.

Dangerzone’s Gvisor security implementation and testing details:

Intro on what is and isn’t compared to Apparmor or Firejail:

Writeup describes circumstances under which Gvisor can be bypassed

Kata vs Gvisor

1 Like

Also worth mentioning here: Cloud Hypervisor.

From my understanding, this lets you get some of the advantages of Xen (very minimal emulated hardware, mostly paravirtualized) while still using KVM. This is what Spectrum OS is using as their hypervisor. It’s written in Rust.

Personally I trust full virtualization quite a bit more than containerization or kernel emulation, though I’m happy to be proven wrong there.

2 Likes

Same, the mentioned projects are degrading to security (specially for App-per-VM concept).