rust-vmm: The Ecosystem Beneath Firecracker
Firecracker did not invent its own way to open /dev/kvm, its own guest-memory
abstraction, its own ELF kernel loader, or its own epoll event loop. It shares
them. Underneath the vmm crate sits a layer of small, single-purpose, heavily
audited Rust crates maintained by a cross-company community — rust-vmm — and
several VMMs build on the same foundation: Firecracker, Cloud Hypervisor,
and (historically) the lineage back to Google's crosvm that Firecracker was
forked from. When Firecracker calls Kvm::new(), parses a vmlinux, or reads a
descriptor chain, it is running rust-vmm code that Cloud Hypervisor also runs.
This is the Firecracker analogue of "Apache Lucene beneath OpenSearch": the
layer the rest of the curriculum stands on. The
deep dives told you that the vCPU run loop calls
KVM_RUN, that guest memory is an mmap registered with KVM, that the VMM
thread runs an EventManager. This section is the layer underneath those
sentences — the actual crates, their key types and traits, the versions
Firecracker pins, and how the vmm crate wires each one in. If you want to be a
core Firecracker contributor rather than someone who edits the device layer
from the outside, you have to be fluent in rust-vmm — because a real fraction of
"Firecracker bugs" are rust-vmm behavior, and a real fraction of good
contributions belong upstream, not in the Firecracker tree.
Note: This curriculum will not hand you a tour of pre-read code. It names a crate and a type, then gives you the
cargo/rg/findcommand that finds the real thing on your checkout, plus thedocs.rspage for the crate's own docs. Every version number here is flagged "(verify on your branch)" — the single source of truth is yourCargo.toml/Cargo.lock, not this page.
What rust-vmm is and why it exists
rust-vmm is a GitHub organization (github.com/rust-vmm) of independent
crates, each in its own repository, jointly maintained by engineers from AWS,
Intel, Red Hat, Alibaba, Google, Cloud Base and others. It is not a VMM. It
ships no binary and boots no guest. It is a collection of the building
blocks every Linux/KVM VMM needs, factored so that the security-critical,
fiddly, easy-to-get-wrong low-level code is written once, audited once, fuzzed
once, and shared instead of re-implemented (subtly differently, and subtly
buggily) in every VMM.
The argument is the same one that makes Firecracker itself worth building: the
VMM is privileged host code on the attack surface, and the lowest layers — FFI
to the KVM uapi, raw ioctl plumbing, guest-memory bounds checks, virtqueue
parsing of attacker-controlled rings — are exactly where a memory-safety or
bounds bug becomes a guest-to-host escape. Concentrating that code in a few
shared crates means more eyes, more fuzzing, and one fix that propagates to
every consumer.
flowchart TD
subgraph VMMs["VMMs (consumers)"]
FC["Firecracker (AWS)"]
CH["Cloud Hypervisor (Intel/community)"]
OT["dragonball, crosvm-lineage, others"]
end
subgraph RV["rust-vmm crates (shared building blocks)"]
KI["kvm-ioctls / kvm-bindings"]
VM["vm-memory"]
LL["linux-loader"]
VQ["virtio-queue"]
VS["vm-superio"]
EM["event-manager"]
SU["vmm-sys-util"]
FD["vm-fdt"]
end
FC --> KI & VM & LL & VS & EM & SU & FD
CH --> KI & VM & LL & VQ & VS & EM & SU & FD
OT --> KI & VM
KI --> KVM["Linux KVM (/dev/kvm)"]
The relationship is genuinely bidirectional, and this is the part that
matters most for a contributor: Firecracker is not only a consumer of rust-vmm.
Several rust-vmm crates were born inside Firecracker and donated upstream so
the wider community could share them — seccompiler, event-manager, and
vm-superio all started life as Firecracker code. You will sometimes find that
the "right place" for your change is the upstream crate, and that landing it
there benefits Cloud Hypervisor too. The full philosophy and governance live in
what-is-rust-vmm.md.
The crates Firecracker uses
Firecracker pins specific rust-vmm crate versions in its workspace. All versions are version-sensitive — grep your own checkout, never trust a number printed in a book:
# The single source of truth for which rust-vmm crates and versions FC uses:
rg -n "kvm-ioctls|kvm-bindings|vm-memory|linux-loader|virtio-queue|vm-superio|event-manager|vmm-sys-util|vm-fdt" \
Cargo.toml src/*/Cargo.toml
# Resolved (exact) versions live in the lockfile:
rg -n "^name = \"(kvm-ioctls|vm-memory|linux-loader|vm-superio|event-manager|vmm-sys-util|vm-fdt|virtio-queue|kvm-bindings)\"" -A1 Cargo.lock
| Crate | Role in Firecracker | Covered in |
|---|---|---|
kvm-bindings | bindgen FFI to the KVM uapi (kvm_run, kvm_regs, kvm_userspace_memory_region, …). The raw structs. | kvm-ioctls-and-kvm-bindings.md |
kvm-ioctls | Safe RAII wrappers over the KVM ioctls: Kvm/VmFd/VcpuFd/DeviceFd, the VcpuExit enum. | kvm-ioctls-and-kvm-bindings.md |
vm-memory | Guest-memory model: GuestMemoryMmap, GuestAddress, the GuestMemory/Bytes/ByteValued traits. Every safe read/write of guest RAM. | vm-memory.md |
linux-loader | Kernel loading: the KernelLoader trait, Elf/BzImage loaders, bootparam, the boot configurators, Cmdline. | linux-loader.md |
vm-superio | Legacy device models: the Serial 16550 UART, I8042Device, Rtc (PL031). Extracted from Firecracker. | vm-superio.md |
event-manager | The epoll abstraction: EventManager, EventSubscriber/MutEventSubscriber, EventOps. The VMM thread's loop. Donated from Firecracker. | event-manager.md |
vmm-sys-util | Low-level glue: EventFd, the ioctl_with_* macros, epoll wrappers, FamStructWrapper, errno::Error, TempFile. | vmm-sys-util.md |
vm-fdt | aarch64 device tree: FdtWriter, used to build the FDT/DTB the ARM kernel reads at boot (no zero page on ARM). | linux-loader.md (aarch64 boot) |
virtio-queue | The split/packed virtqueue ring logic (Queue, DescriptorChain). FC keeps its own in-tree virtio; this is the shared reference model. | virtio-queue.md |
Note: Firecracker maintains its own virtio device code in-tree (
src/vmm/src/devices/virtio/) rather than consumingvirtio-queue/virtio-devicewholesale. We still covervirtio-queuebecause the concepts — descriptor table, available ring, used ring,pop_descriptor_chain,add_used— are identical, and reading the shared crate is the cleanest way to understand the in-tree code. Check what your branch actually depends on with thergabove.
The crates Firecracker donated
| Crate | Origin | Why it left the tree |
|---|---|---|
seccompiler | Firecracker's in-tree seccomp compiler | Other VMMs need JSON→BPF seccomp too; shared and audited once. (FC still keeps an in-tree seccompiler crate that mirrors it — verify which is built.) |
event-manager | Firecracker's EventManager | Every VMM with a device epoll loop needs the same subscriber abstraction. |
vm-superio | Firecracker's serial/i8042 device models | The 16550 UART and i8042 are generic; Cloud Hypervisor uses the same Serial. |
This donor relationship is the single most useful framing for a contributor. When you touch the serial console, the seccomp compiler, or the event loop, ask: does this fix belong here, or upstream? The answer is covered crate-by-crate and summarized in what-is-rust-vmm.md and Lab R4: Contribute to rust-vmm.
How to read this section
Read the chapters roughly in dependency order. Each is a self-contained crate
chapter following the same shape: what problem it solves → its key
types/traits → how Firecracker uses it (with an rg into the FC tree and a
pointer to docs.rs) → a small runnable code example.
| # | Chapter | One line |
|---|---|---|
| 1 | What rust-vmm Is | The project, its governance, rust-vmm-ci, how a crate gets added, why it matters to you |
| 2 | kvm-ioctls and kvm-bindings | Safe RAII KVM wrappers + the raw uapi FFI; create a VM and a vCPU |
| 3 | vm-memory | GuestMemoryMmap, GuestAddress, safe typed reads/writes of guest RAM |
| 4 | linux-loader | Load a vmlinux, write the zero page + e820, build the kernel cmdline |
| 5 | virtio-queue | The shared split/packed virtqueue model; descriptor chains, the used ring |
| 6 | vm-superio | The 16550 serial console, i8042, the PL031 RTC; the Trigger trait |
| 7 | event-manager | The epoll loop; EventSubscriber, EventOps, EventSet |
| 8 | seccompiler | JSON seccomp filters → BPF; compile_from_json, apply_filter |
| 9 | vmm-sys-util | EventFd, the ioctl_with_* macros, FamStructWrapper, the low-level glue |
The four hands-on labs
| Lab | What you build |
|---|---|
| Lab R1: Build a KVM VM | A microVM skeleton with kvm-ioctls + vm-memory from scratch |
| Lab R2: Load a Kernel | Use linux-loader to load a vmlinux and write the zero page |
| Lab R3: Drive a Virtqueue | Stand up a virtio-queue Queue and walk a descriptor chain |
| Lab R4: Contribute to rust-vmm | The real upstream PR workflow on a rust-vmm repo |
Tip: The fastest way to read any of these crates is
cargo doc. From your Firecracker checkout:cargo doc -p kvm-ioctls -p vm-memory -p linux-loader -p vm-superio \ -p event-manager -p vmm-sys-util --no-deps --openThat builds the docs for the exact pinned versions on your branch — strictly better than reading
docs.rs/latest, which may be ahead of what FC pins.
rust-vmm-ci and governance, in one paragraph
rust-vmm keeps the bar high with rust-vmm-ci — a shared CI harness
(a git submodule each crate pulls in) that enforces the same cargo build,
clippy -D warnings, cargo fmt, coverage, and (where applicable) fuzzing
across every crate, so a contribution to any crate is held to one consistent
standard. Each repo has its own maintainers, plus a small set of org-wide
gatekeepers, and a community repo that holds the meeting notes, the
process for proposing a new crate, and the membership rules. There is no CLA;
contribution is by GitHub PR with the usual sign-off discipline. None of that is
hypothetical for a Firecracker contributor — the moment your fix is "really a
vm-superio fix," you are a rust-vmm contributor, held to rust-vmm-ci. The full
story is what-is-rust-vmm.md.
Next: What rust-vmm Is — the project, its governance, and why a Firecracker change sometimes belongs upstream.