Milestones & Checkpoints

These are the gates. The 16-week plan paces you toward them, but the calendar does not decide whether you advance — these milestones do. Each one (M1–M9) aligns to a level and certifies a specific competence. You pass a milestone when you can answer its self-checks without notes and could plausibly score the full 20 points on its rubric. If you cannot, you are not ready for the next level, regardless of what week it is.

This curriculum will not hold your hand here either. The self-check questions have real answers in the source; the rubric describes what demonstrated competence looks like, not what reading looks like. Be honest. The point of a gate is to fail you when you are not ready, so the next level does not collapse under you.

How to use this page. After finishing a level, sit down without the book open and answer that milestone's questions out loud or in writing. Then score yourself against the 20-point rubric (four dimensions × 5 points). A passing score is ≥16/20 with no dimension below 3. Anything less: re-do the weakest labs and the deep dives they reference, then re-test.

flowchart LR
    M1[M1: Build & Boot] --> M2[M2: Contributor Workflow]
    M2 --> M3[M3: Architecture & Threads]
    M3 --> M4[M4: KVM & the Run Loop]
    M4 --> M5[M5: Testing & Debugging]
    M5 --> M6[M6: Boot & Guest Memory]
    M6 --> M7[M7: Virtio Device Model]
    M7 --> M8[M8: Real Contribution]
    M8 --> M9[M9: Advanced Maintainer]
    M9 --> CAP([Capstone])
MilestoneLevelCertifies
M1Level 1Build, test, and boot Firecracker; KVM from scratch.
M2Level 2The GitHub/DCO contribution workflow; a clean first PR.
M3Level 3The threading model and API→VMM action channel.
M4Level 4KVM ioctls, the vCPU run loop, and VM exits.
M5Level 5The pytest framework, unit tests, debugging.
M6Level 6Kernel loading, the boot protocol, guest memory layout.
M7Level 7The virtio device model: virtqueues, MMIO, block/net.
M8Level 8A real issue reproduced, root-caused, fixed, reviewed.
M9Level 9Security model, snapshot compat, performance reasoning.

M1 — Build, Test & Boot (the Floor)

Aligns to: Level 1. Certifies that you can build Firecracker with tools/devtool, run its test suites, boot a microVM from your own binary, and that you understand KVM well enough to have written a tiny VMM by hand.

Skills certified:

  • Build (tools/devtool build [--release] [-l musl|gnu]) and locate the binary under build/cargo_target/.
  • Run unit tests (cargo test) and the pytest integration suite (tools/devtool test); pass checkstyle/checkbuild.
  • Boot a microVM via the curl --unix-socket PUT sequence and read its state with GET /.
  • Open /dev/kvm, create a VM and vCPU, map guest memory, run KVM_RUN, and handle an I/O exit — from scratch.

Self-check (no notes):

  1. Where does tools/devtool build --release put the firecracker binary, and what find command locates it without you guessing the path? What is the default C library?
  2. Why is the integration suite run with tools/devtool test (pytest) and not cargo test? What does each cover?
  3. Name the three KVM file-descriptor levels and the ioctl that creates each.
  4. In your hand-written VMM, after KVM_RUN returns, what field do you read to decide what happened, and what does a KVM_EXIT_IO exit mean?
  5. Walk through the minimal curl sequence to boot a microVM — which call loads the kernel, which supplies the rootfs, which starts it?
  6. What are the default vcpu_count and mem_size_mib if you never call PUT /machine-config?

20-point rubric:

Dimension0–2 (not yet)3 (adequate)4–5 (strong)
Build & test fluencyBuild fails or needs helpBuilds and runs tests; finds the binaryBuilds release+debug, runs unit+pytest, checkstyle green, all from memory
Boot a microVMCannot reliably bootBoots with the reference commandsBoots, reads state via GET /, explains each PUT and the defaults
KVM from scratchVMM incompleteVMM maps memory and runs KVM_RUNVMM handles an IO/MMIO exit and you can explain every ioctl used
Anti-staleness habitRelies on remembered pathsUses rg/find when promptedReflexively locates code with rg/find; verifies version-sensitive facts

M2 — The Contributor Workflow (Your First PR)

Aligns to: Level 2. Certifies that you can navigate the codebase and produce a clean, mergeable pull request that respects Firecracker's process.

Skills certified:

  • Fork-and-pull flow against main; branch hygiene; one logical change per commit.
  • DCO sign-off on every commit (git commit -s; --amend -s; git rebase --signoff).
  • A CHANGELOG.md entry and a ≤72-char commit title; new functionality carries an integration test.
  • Reading a PR critically and reviewing someone else's.

Self-check (no notes):

  1. What exactly does git commit -s add, and why does the email have to match your GitHub account?
  2. How many maintainer approvals does a PR need to merge, and who merges it?
  3. What three things does CONTRIBUTING.md expect of each commit (besides the sign-off)?
  4. Where do security vulnerabilities get reported, and why never as a public issue?
  5. Which tools/devtool commands do you run before pushing, and what does clippy-as-errors mean for you?
  6. You forgot -s on three commits already pushed to your branch. What is the exact recovery?

20-point rubric:

Dimension0–234–5
PR mechanicsMissing sign-off / CHANGELOG / failing CISigned, CHANGELOG present, CI greenClean history, atomic commits, ≤72-char titles, CI green first try
Codebase navigationGets lostFinds the right crate/module with helpNavigates vmm/firecracker/jailer fluently with rg/editor
Process understandingUnsure of the rulesStates DCO, ≥2 approvals, CHANGELOG, test ruleExplains why each rule exists (security, bisectability, trust)
Review skillCannot critique a PRSpots obvious issuesGives a substantive, kind, code-grounded review of a real PR

M3 — Architecture & the Threading Model

Aligns to: Level 3. Certifies that you can trace a control-plane request from the API socket to the VMM thread and explain the three-thread architecture.

Skills certified:

  • The three thread classes: API thread, VMM thread (EventManager epoll loop), one thread per vCPU.
  • The API→VMM path: ParsedRequest → VmmAction over an mpsc channel + an eventfd wake → PrebootApiController/RuntimeApiController → VmmData/VmmActionError reply.
  • Where this wiring lives (rpc_interface.rs, api_server_adapter::run_with_api()), and that the API server is in the firecracker binary, not vmm.

Self-check (no notes):

  1. Name the three thread classes and the single responsibility of each. Which is absent under --no-api?
  2. Trace PUT /machine-config from the socket to where the config lands. What channel and what synchronization primitive carry the action across threads?
  3. What is the difference between PrebootApiController and RuntimeApiController, and what event switches between them?
  4. The API server lives in which crate/binary — vmm or firecracker? Why does that boundary exist?
  5. What does the VMM thread's EventManager loop actually wait on, and what wakes it?
  6. How does a vCPU thread communicate with the VMM thread (e.g. for pause/resume)?

20-point rubric:

Dimension0–234–5
Thread modelConfuses the threadsNames all three and their jobsExplains the epoll loop, eventfd wakes, and the fast-path split precisely
Action channel traceCannot trace a requestTraces socket → VmmAction → handlerTraces both pre-boot and runtime paths, naming each type and the reply
Crate boundariesUnsure where code livesKnows API server is in firecrackerExplains the firecracker/vmm split and locates the channel wiring with rg
DiagrammingNo coherent pictureSketches the pathDraws the full three-thread + channel diagram from memory

M4 — KVM, vCPUs & the Run Loop

Aligns to: Level 4. Certifies that you can read Firecracker's vCPU run loop without a guide and reason about VM exits and CPUID/MSR setup.

Skills certified:

  • The KVM_RUN loop and the struct kvm_run shared page; locating it with rg 'fn run|KVM_RUN|VcpuExit' src/vmm/src/vstate/vcpu/.
  • The VM-exit taxonomy: KVM_EXIT_IO (PIO) vs KVM_EXIT_MMIO vs HLT/SHUTDOWN/FAIL_ENTRY, and how Firecracker dispatches each.
  • The rust-vmm kvm-ioctls interface (Kvm/VmFd/VcpuFd/VcpuExit) and CPUID/MSR setup (KVM_GET_SUPPORTED_CPUID/KVM_SET_CPUID2, CPU templates).
  • KVM_IRQFD (eventfd→IRQ injection) and KVM_IOEVENTFD (guest write→eventfd; the virtio fast path).

Self-check (no notes):

  1. What does a single iteration of the run loop do, from KVM_RUN to the next KVM_RUN?
  2. Distinguish KVM_EXIT_IO from KVM_EXIT_MMIO: which devices cause each, and where is the data?
  3. What is the role of KVM_IOEVENTFD, and why does it mean a virtio kick need not exit the vCPU thread?
  4. Why does Firecracker call KVM_SET_CPUID2, and what problem do CPU templates solve across heterogeneous hosts?
  5. What happens on KVM_EXIT_FAIL_ENTRY, and what kind of bug usually causes it?
  6. How is an interrupt delivered to the guest — what ioctl, and what host primitive triggers it?

20-point rubric:

Dimension0–234–5
Run loop comprehensionCannot follow itReads it with helpReads vstate/vcpu/ unaided; explains each exit branch
VM-exit taxonomyConfuses IO/MMIOKnows the main exitsMaps every common exit to a device/cause and the data location
CPUID/MSR & templatesUnawareKnows CPUID is setExplains normalization across hosts and the template mechanism
eventfd plumbingUnawareKnows IRQFD/IOEVENTFD existExplains the fast path: kick→IOEVENTFD→VMM loop, IRQ via IRQFD

M5 — Testing & Debugging

Aligns to: Level 5. Certifies that you can use Firecracker's test machinery to prove a change and to chase a bug, including flaky-test diagnosis.

Skills certified:

  • The pytest integration framework in tests/, run via tools/devtool test [-- <pytest args>].
  • Writing a unit test (cargo test) and a Rust-side assertion; not lowering coverage.
  • Writing an integration test that boots a microVM and asserts behavior.
  • Diagnosing a flaky test (ordering, timing, resource leakage) and debugging a running microVM.

Self-check (no notes):

  1. Where do integration tests live, what drives them, and how do you run a single one?
  2. When does a change require an integration test versus a unit test? (Recall the CONTRIBUTING rule.)
  3. Name two common causes of flaky integration tests in a VMM test harness and how you'd confirm each.
  4. How do you observe what a microVM is doing at runtime — what signals, logs, or metrics are available?
  5. What does tools/devtool checkbuild --all protect against that a single cargo build does not?
  6. You have a test that passes locally but fails in CI. What is your first diagnostic move?

20-point rubric:

Dimension0–234–5
Test authoringCannot add a testWrites a unit or integration testWrites both; chooses the right kind; runs targeted via tools/devtool test
Debugging a microVMNo methodReads logsUses logs/metrics/serial + a debugger to localize a fault
Flaky-test diagnosisReruns and hopesIdentifies a likely causeReproduces deterministically and explains the race/leak
CI/coverage disciplineIgnores CIKeeps CI greenReasons about coverage, checkbuild --all, and CI/local divergence

M6 — The Boot Process & Guest Memory

Aligns to: Level 6. Certifies that you can trace how a kernel is loaded and started, and reason about the guest-physical memory layout.

Skills certified:

  • Kernel loading: uncompressed vmlinux ELF via linux-loader (PT_LOAD → copy → e_entry).
  • The x86_64 boot protocol: boot_params/zero page, the e820 map, cmdline and initrd pointers; layout constants in arch/x86_64/layout.rs (verify on your branch).
  • Initial vCPU register state for 64-bit long mode (rip=e_entry, rsi=ZERO_PAGE_START, paging on).
  • aarch64 differences: no zero page, an FDT/DTB passed in x0, the arm64 Image.
  • Guest memory: GuestMemoryMmap/GuestAddress, the MMIO gap below 4 GiB, high RAM above.

Self-check (no notes):

  1. Why is there no BIOS, and what two jobs (firmware's and the bootloader's) does the VMM do instead?
  2. What is the zero page, and name three things it carries to the kernel.
  3. Why does the e820 map matter — what breaks if it is wrong?
  4. What initial register values put the guest into 64-bit long mode at the kernel entry?
  5. How does aarch64 boot differ from x86_64 at the "hand the kernel its environment" step?
  6. Why is there an MMIO gap below 4 GiB, and where does RAM above it go?

20-point rubric:

Dimension0–234–5
Kernel load pathCannot trace itTraces ELF load with helpTraces linux-loader PT_LOAD → entry → first instruction unaided
Boot protocolVague on zero pageKnows zero page + e820Explains boot_params fields, cmdline, initrd, and failure modes
Initial CPU stateUnsureKnows long mode is set upStates the exact register setup and why each is needed
Memory layoutConfuses address spacesKnows guest-phys vs hostExplains the full layout, MMIO gap, high RAM, and aarch64 FDT

M7 — The Virtio Device Model

Aligns to: Level 7. Certifies that you can trace a virtio I/O end to end and reason about the virtqueue/MMIO machinery well enough to extend it.

Skills certified:

  • Split virtqueues: descriptor table (addr/len/flags/next, flags NEXT/WRITE/INDIRECT), available ring, used ring; the kick/interrupt cycle.
  • The virtio-MMIO transport: the register map (MagicValue, QueueSel, QueueNotify, InterruptStatus, Status), feature negotiation, the status state machine (ACKNOWLEDGE→DRIVER→FEATURES_OK→DRIVER_OK).
  • virtio-block (one request queue, host pread/pwrite, I/O engine) and virtio-net (RX/TX, host TAP); where they live (src/vmm/src/devices/virtio/{block,net}/).
  • How a device is placed on the bus (MMIODeviceManager) and how the fast path uses KVM_IOEVENTFD.

Self-check (no notes):

  1. Trace a guest disk read from the driver placing a descriptor to the guest seeing the result — every ring, the kick, the host call, and the interrupt.
  2. What do the descriptor flags NEXT, WRITE, and INDIRECT each mean?
  3. What is the status handshake, and what goes wrong if a device claims a feature the driver did not acknowledge?
  4. How does the guest learn where a virtio-MMIO device's registers and IRQ are on x86 vs aarch64?
  5. Why is the "kick" not necessarily a vCPU exit, and how is the completion interrupt delivered?
  6. Where is virtio-block's request handling, and what host syscall actually reads the data? Find it with rg.

20-point rubric:

Dimension0–234–5
Virtqueue mechanicsConfused by the ringsKnows avail/used/desc rolesTraces a descriptor chain and the kick/interrupt cycle precisely
MMIO transportUnaware of the register mapKnows key registersExplains negotiation, the status FSM, and device discovery on both arches
A specific deviceCannot locate itFinds block/net with rgTraces block or net I/O to the host syscall and back to the used ring
ExtensibilityNo idea how to add a deviceKnows the device trait shapeCould scaffold a new virtio device on the MMIO bus

M8 — Real Issue Contribution

Aligns to: Level 8. Certifies that you can take a real, open issue from reproduction through root cause to a reviewed PR.

Skills certified:

  • Selecting an appropriately scoped real issue and posting a minimal, reliable reproduction.
  • Execution-path analysis: tracing the bug through the actual code, not guessing.
  • A correct fix with an integration/unit test that fails before and passes after.
  • A PR description tying fix to root cause; responding to ≥2 maintainers across review rounds.

Self-check (no notes):

  1. What makes a reproduction "good" enough that a maintainer can confirm it?
  2. How do you distinguish a symptom from a root cause, and how do you prove the root cause?
  3. Why must your test fail before your fix and pass after, and where does it go?
  4. What belongs in a PR description so a reviewer can evaluate it quickly?
  5. A maintainer pushes back on your approach citing the minimal-device-model philosophy. How do you respond?
  6. How do you keep your branch clean across multiple review rounds (rebase, re-sign, squash)?

20-point rubric:

Dimension0–234–5
ReproductionVague/flakyReproduces reliablyMinimal, deterministic repro that maintainers confirm
Root-cause analysisGuessesFinds the areaProves the root cause via execution-path analysis
Fix & testNo test / wrong fixFix + a testMinimal correct fix; test fails-before/passes-after; no coverage loss
Review collaborationDefensive/silentResponds to feedbackEngages every comment, iterates cleanly, helps review others

M9 — Advanced Maintainer

Aligns to: Level 9. Certifies maintainer-grade reasoning about security, snapshot compatibility, and performance.

Skills certified:

  • The defense-in-depth model: KVM boundary + jailer (chroot/namespaces/cgroups/priv-drop) + seccomp-BPF (per-category vmm/api/vcpu filters) + Rust; the explicit threat model (guest+guest-kernel untrusted).
  • Snapshot/restore internals and backward compatibility: the Persist trait, full vs diff, track_dirty_pages, UFFD; why a snapshot taken on version N must load on N+1 within policy.
  • Performance reasoning: boot time, oversubscription/density, I/O engines, hugepages; diagnosing a regression with measurement and bisection.

Self-check (no notes):

  1. Name the four defense-in-depth layers and what each stops. Which one does the jailer not do (hint: seccomp)?
  2. How are seccomp filters structured (per-category, default action + rules), and what does --no-seccomp mean for production?
  3. What is the threat model in one sentence, and why is the device model deliberately minimal?
  4. What makes a change a snapshot-compatibility break, and how would you reason about whether it is acceptable?
  5. How does diff snapshotting work, and what must be enabled for it?
  6. You see a boot-time regression after a change. What is your measurement-and-bisection plan?

20-point rubric:

Dimension0–234–5
Security modelVagueNames the layersExplains each layer, audits a device's surface, reasons about the threat model
Snapshot compatibilityUnaware of the riskKnows full vs diffReasons correctly about cross-version compat and the Persist contract
Performance reasoningAnecdotalMeasures somethingDiagnoses a regression with rigorous measurement + bisection
Maintainer judgmentDefers entirelyHas opinionsWeighs surface area, compat, and perf like a maintainer would

Passing the Whole Set: Capstone Readiness

You are ready for the Capstone when all nine milestones are passed (≥16/20 each, no dimension below 3) — and especially when M4, M7, and M8 are strong, because the capstone leans hardest on reading the run loop, tracing a device, and shepherding a real PR. The capstone is scored against its own evaluation rubric, which is these milestones applied to one continuous, real contribution.

If you are short on any milestone, the fix is always the same: re-do the weakest labs and read the deep dives they reference, then re-test. Do not paper over a gate — a contributor who skipped M4 will stall in the capstone the moment a bug lives in the vCPU run loop.


Where to Go Next

If you have not started yet, begin with Level 1 and return here after each level to test the matching milestone. Keep the 16-Week Plan open alongside this page — the plan tells you when to test; this page tells you whether you passed. The internals behind every self-check live in the Deep Dives.