Protected KVM on Arm64: A Technical Deep Dive - Quentin Perret, Google

The architecture pattern is similar to Bromium/HP AX + Type 2 μXen on x86, [LINK] (2018), which ships on HP business PCs. Bare metal runs a tiny L0 hypervisor making use of hardware support for nested virtualization. In turn, the L0 can run an L1 hypervisor, e.g. KVM or "host" OS, or minimal L1 VMs that are peers to the L1 "host"-guest of L0. Google pKVM-for-Arm tech talk (2022), hopefully x86 will follow, [VIDEO]

A modern virtualization architecture can be found in the OSS pKVM L0 nested hypervisor for Android Virtualization Framework, which has some architectural overlap with HP/Bromium AX L0 + [Hyper-V | KVM | Xen] L1 + uXen L2 micro-VMs with copy-on-write memory. A Bromium demo circa 2014 was a web browser where every tab was an isolated VM, and every HTTP request was an isolated VM. Hundreds of VMs could be launched in a couple of hundred milliseconds. Firecracker has some overlap. > Lastly, this approach is almost certainly more expensive. Our instances sit idle for the most part and we pay EC2 a pretty penny for the privilege. With many near-idle server VMs running identical code for each customer, there may be an opportunity to use copy-on-memory-write VMs with fast restore of unique memory state, using the techniques employed in live migration. Xen/uXen/AX: [LINK] pKVM: [VIDEO]

> Separation from the hypervisor kernel? Not really. Separation from (type 1) hypervisor (or rather distrust of the host [0]) requires hardware support; ex: ARM CCA / AMD SEV-SNP / Intel TDX. For separation from the supervisor, Android developed a peculiar approach in "pKVM" for ARM where the host (supervisor) is partitioned away from the guest [1]. Both those "separations" is not something Toro provides on its own; the Toro unikernel would totally be under the control of the host, from what I can tell. That said, what Toro (or any unikernel, really) does is reduce the attack surface area, as the (guest) supervisor is pruned to run just one particular application (more code to partition things up will eliminate a class of attacks but may result in new attack vectors [2]). [0] ex: [LINK] [1] Protected KVM on Arm64: A Technical Deep Dive - Quentin Perret, Google [VIDEO] (2023) [2] Mitigations are attack surface, too [LINK] (2020)