Feature Masterclasses: Deep Intensives

The levels are the spine of this curriculum and the deep dives are the muscle — but a deep dive covers a subsystem (the vCPU run loop, guest memory, the virtio transport) at the depth needed to reason about and debug it. A masterclass does something narrower and harder: it takes one feature area, nails it down to the data structures, the exact ioctls, the byte-level layout, and the precise Firecracker types that implement it, and then makes you build and measure across three or four full labs. Where a deep dive shows you the path, a masterclass hands you a stopwatch and a counter and makes you run it until you can predict the numbers.

The bar is concrete: a contributor who finishes a masterclass can open a real issue in that area and have an informed opinion about the fix — not "the boot is slow," but "the kernel is an uncompressed vmlinux with three PT_LOAD segments, e_entry lands at HIMEM_START, and the zero page's e820 map has a gap I can see in the guest's /sys/firmware/memmap; here's the test that proves where the regression is."

Note: "Masterclass" is a teaching format, not a code module. Everything you learn maps to real firecracker-microvm/firecracker code you can check out, rg, build with tools/devtool, and run. This curriculum will not hold your hand: it points you at the right parts of the tree, gives you the right questions, and makes you run everything you read. Where it names a struct, it gives you the rg/find that locates it on your branch — because line numbers are stale the day they're written.


What makes a masterclass different from a deep dive

DimensionDeep diveMasterclass
ScopeA whole subsystem (the run loop, guest memory, the device manager)One feature area, end to end
DepthEnough to reason about and debug the subsystemData structure + ioctl + the constants + byte-level layout + the exact types
CodeIllustrative excerpts, rg targetsYou read the real types and write a standalone program
MeasurementConceptual trade-offsYou count exits / time boots / dump bytes and find the crossover yourself
OutputUnderstandingUnderstanding plus a reproducible experiment and an informed opinion

Each masterclass has the same shape: a topic index.md (the feature from first principles — the problem, the data structures, the algorithm, the Firecracker types, real Rust excerpts, diagrams, a worked numeric example, a trade-offs table, a "common bugs" table, and a validation self-check) plus three or four labs built on the standard Lab template (Background → Why This Matters for Contributors → Prerequisites → numbered Steps with real runnable code → Deliverables → Troubleshooting → Expected Output → Stretch Goals → Validation).

The labs in each masterclass follow a deliberate arc: a build lab (write a minimal standalone version of the mechanism so you own the contract), an instrument lab (turn the feature on and measure it — count exits, dump the zero page, trace a virtqueue), and a deepen lab (a second architecture, a compatibility edge, a security implication). Once you've done one masterclass you know the rhythm of all eight.

Note: Don't skip the build lab because it "looks like toy code." Writing the mechanism standalone — a multi-vCPU KVM VMM, a UFFD page-fault handler, a tiny virtio device — is what converts "I read about it" into "I could fix it." The real Firecracker code is the same contract with production concerns, security, and a threading model bolted on.


The eight masterclasses

#MasterclassFocusExtends (deep dives)When to take itLink
1KVM & vCPUsDrive /dev/kvm directly: multi-region memory, long mode, multi-vCPU threads, the full VcpuExit taxonomy, CPUID/MSRskvm-fundamentals, vcpu-run-loop-and-vm-exits, cpu-templates-and-cpuidDuring/after Level 4Start →
2The Boot ProcessHow an uncompressed vmlinux ELF is parsed, placed, and entered; the zero page / e820; the aarch64 FDTthe-boot-sequence, guest-memory-management, acpi-and-mptableDuring/after Level 6Start →
3Virtio DevicesThe split virtqueue, the MMIO transport, block/net/vsock end to end, and building a device from scratchvirtio-transport-mmio, virtqueues, virtio-block, virtio-net-and-tapDuring/after Level 7Start →
4SnapshottingThe two-file snapshot, the Persist trait, full vs diff, COW restore, and a UFFD page-fault handler you writesnapshotting, guest-memory-managementLevel 9Start →
5SecurityThe jailer barrier, seccomp-BPF filters per thread category, and a hands-on threat-model auditthe-jailer, seccomp-filteringLevel 9Start →
6NetworkingTAP devices and host bridges, the token-bucket rate limiter, and the MMDS metadata servicevirtio-net-and-tap, rate-limiting-token-bucket, mmds-metadata-serviceAfter Level 7Start →
7Debugging & ProfilingGDB against a live guest, the logging/metrics systems, and reproduce-and-bisect disciplinelogging-and-metrics, signals-shutdown-and-resetAfter Level 5Start →
8Performance & DensityBoot-time decomposition, oversubscription and balloon/KSM, and the Sync-vs-io_uring block engines benchmarkedboot-time-optimization, oversubscription-and-density, io-enginesAfter Level 9Start →

Note: Each masterclass index.md lists its own prerequisites. The "when to take it" column maps each one to the level whose material it deepens — take a masterclass once you've finished the corresponding level's labs, not before. KVM & vCPUs deepens Level 4; the Boot Process deepens Level 6; Virtio Devices deepens Level 7; Snapshotting and Security deepen Level 9.


Reading order

The masterclasses are independent — take them in the order that matches what you're working on. But they layer naturally, and if you intend to do all eight, this dependency graph builds each on the last:

flowchart TD
    K["1 · KVM &amp; vCPUs<br/>(the layer beneath everything)"] --> B["2 · The Boot Process<br/>(get a real kernel running on those vCPUs)"]
    B --> V["3 · Virtio Devices<br/>(give the booted guest I/O)"]
    V --> N["6 · Networking<br/>(the net device, rate limiting, MMDS)"]
    K --> S["4 · Snapshotting<br/>(capture and restore the vCPU + device + memory state)"]
    V --> S
    K --> SEC["5 · Security<br/>(the jailer + seccomp around it all)"]
    V --> DBG["7 · Debugging &amp; Profiling<br/>(GDB, metrics, bisect — on a working system)"]
    DBG --> PERF["8 · Performance &amp; Density<br/>(now make it fast and dense)"]
    S --> PERF
  • Start with KVM & vCPUs. Everything else runs on the vCPU threads and the KVM_RUN loop. Once you've built a multi-vCPU VMM by hand and counted every exit class of a real boot, the rest of Firecracker is recognizable.
  • Then the Boot Process. Now that you can run code on vCPUs, learn how a real Linux kernel gets loaded onto them — the ELF parse, the zero page, the FDT.
  • Virtio Devices gives the booted guest its I/O: the virtqueue contract, the MMIO transport, and a device you build yourself.
  • Networking sits on top of virtio (the net device is a virtio device) and adds the host-side plumbing and rate limiting.
  • Snapshotting captures the combined vCPU + device + memory state you built up in the first four — it can't be understood without them.
  • Security wraps the whole thing in the jailer and seccomp; Debugging & Profiling and Performance & Density are the operational capstones you take once you have a working system to debug and tune.

Note: If you're here for a specific contribution, skip the order and go straight to the masterclass that owns your feature. Backfill only the prerequisites you're missing — each index.md lists them.


Prerequisites for the whole set

Before starting any masterclass you should be through the early levels and comfortable with the build/test flow:

  • You can build Firecracker with tools/devtool build and boot a microVM by hand (Lab 1.1, Lab 1.3).
  • You wrote the ~70-line KVM VMM in Lab 1.4 — the KVM & vCPUs masterclass extends it directly.
  • You can read unfamiliar Rust fluently and write a small standalone Cargo program (the labs hand you the exact Cargo.toml and cargo run).
  • You have a Linux host with a working, accessible /dev/kvm (ls -l /dev/kvm).

Each masterclass narrows this to a feature-specific prerequisite list. None of them assume you've done the others first except where a lab explicitly links back.


How masterclasses feed the capstone

The capstone is one full real contribution cycle — issue → reproduce → execution-path analysis → root cause → fix → test → PR. A masterclass is the training that earns you the right to attempt the kind of issue that touches that feature. The mapping is direct: the issue roadmap stages 6 (vcpu/KVM), 7 (virtio), 8 (snapshot compat), 10 (performance), and 11 (security/seccomp) each have a masterclass that teaches the area cold before you go off-road on a real bug.

Warning: Don't attempt an issue in an area you can't yet measure. The masterclasses are where you build the reflexes — counting exits, dumping the zero page, driving a virtqueue, writing a UFFD handler — on a known-good path, so that when a real issue is weird you have a baseline to diff against.


A note on rg over line numbers

Throughout the masterclasses you'll see rg/find commands instead of "open file X line 412." That's deliberate and it is the most important rule in this book. Firecracker moves fast — there was a large refactor that merged most crates into vmm; line numbers and even file paths drift. An rg for a struct name, an ioctl, or a layout constant finds the real site in your checkout, whatever branch you're on. When a masterclass says "HIMEM_START is 0x100000 — verify on your branch," it means the number is real but you confirm it against the source tree, not against this page. That habit — trust the source, not the doc — is the single most valuable thing the masterclasses teach.


Begin with the KVM & vCPUs Intensive, or jump to the masterclass that owns the feature you're working on.