Lab 3: A Threat-Model Audit

Prerequisite reading: Security — Intensive (the two boundaries and the four-layer model), and Labs 1 and 2 — you must have felt the jailer and seccomp work and break before the audit will land.

Background

The first two labs were mechanical: you ran the jailer, you read the seccomp filters, you broke them and watched the failures. This lab is the synthesis, and it is a review-it exercise — the kind of thinking a maintainer does when a security report lands or a PR proposes adding a device. You will enumerate the entire attack surface (every emulated device, the API socket, both trust boundaries), walk the production hardening guide line by line on a real or simulated host, reason through a real recently-fixed vulnerability (CVE-2026-5747 in the virtio-PCI transport) to see how each defense layer would and would not stop it, and produce a reusable audit checklist you could hand a security reviewer.

The skill this lab builds is dual: you must think like the attacker (where is the host code that parses my bytes? which syscall do I need? what gets me across a boundary?) and like the defender (what stops that? what stops it if the first thing fails? what did we forget?). Security at Firecracker's level is not a feature you add; it is an adversarial argument you can defend.

Why this matters for contributors

• The minimal-device-model philosophy is the recurring reason proposals get rejected. "QEMU has it" is not an argument; "this adds N lines of attacker-reachable parser to the host" is the counter-argument, and you must be able to make it precisely. See the minimal-device-model essay.
• Real security work on Firecracker means reading a device's data path and asking "is every guest-supplied length and address bounds-checked before use?" — the exact question CVE-2026-5747 turned on. This lab teaches you to ask it systematically.
• Maintainers reason about disclosure, embargo, and the CHANGELOG every time a CVE is fixed. Knowing how a real fix flowed (private report → patched releases → public advisory) is part of operating at that level. See the security/seccomp issue stage.

Prerequisites

• Completed Labs 1 and 2.
• The Firecracker repo checked out (you will rg the device code and the docs).
• Optional: a host you can run the production-hardening checks against (a bare-metal or nested-KVM box you own). Where you lack a real host, simulate the checks and note what you would verify.

# Confirm you can read the docs and the device tree you'll audit.
ls docs/prod-host-setup.md SECURITY.md
find src/vmm/src/devices -maxdepth 3 -name "*.rs" | head
rg -n "CHANGELOG" CHANGELOG.md >/dev/null && echo "CHANGELOG present"

Step-by-step tasks

Step 1 — Enumerate the emulated devices (count the parsers)

Every emulated device is host code that processes attacker-controlled bytes. Auditing the attack surface starts by counting them. List the devices Firecracker actually implements and, for each, name the guest-controlled input it parses — that input is the attack surface.

# The device tree. Each subdir is a parser the guest can drive.
find src/vmm/src/devices/virtio -maxdepth 1 -type d
rg -n "TYPE_NET|TYPE_BLOCK|TYPE_VSOCK|TYPE_BALLOON|TYPE_RNG|device_type" src/vmm/src/devices/virtio/
# Legacy (non-virtio) devices: serial UART + partial i8042.
rg -n "Serial|i8042|com_evt|uart|Bus" src/vmm/src/devices/legacy/ 2>/dev/null \
  || rg -n "Serial|i8042" src/vmm/src/devices/

Note: This list is the entire device attack surface, and it is short on purpose. Every entry is a parser inside the host process that a hostile guest drives directly. The argument for keeping the list short is exactly this table — fewer rows means fewer places a bug can live. When you audit a PR that adds a device, you are adding a row, and you must justify the new attack surface against the minimal-device-model philosophy.

• For each device, identify in the source where guest-supplied addr/len from a descriptor is turned into a host memory access, and confirm it goes through bounds-checked vm-memory accessors, not raw pointer math. That bounds check is the front-line defense on boundary A.

# The pattern to look for in every device: guest-controlled length/address → checked access.
rg -n "checked_|GuestMemory|read_slice|write_slice|read_obj|write_obj|desc\.addr|desc\.len" \
  src/vmm/src/devices/virtio/block/ src/vmm/src/devices/virtio/net/

Step 2 — Audit the API socket and the two boundaries

The API socket is the control-plane attack surface, distinct from the device data plane. Map it, and then write down both trust boundaries explicitly.

# The API server lives in the firecracker binary, NOT vmm.
find src/firecracker/src/api_server -name "*.rs" | head
rg -n "UnixListener|api_sock|bind|ParsedRequest|VmmAction" src/firecracker/src/

# Who is allowed to talk to the socket? It's a filesystem object — permissions are the gate.
rg -n "permission|chmod|0o7|mode|umask" src/firecracker/src/ src/jailer/src/

Warning: The API socket is not a guest-facing surface — the guest cannot reach it (it is a host Unix socket, and under the jailer it lives inside the chroot). The threat here is a confused or over-privileged orchestrator, not the guest. Conflating the two is a classic audit error: a guest attack comes through boundary A (a device), never through the API socket.

• State, in your notes, which boundary each of the following crosses: a malformed virtio descriptor (A), an attempt by a compromised VMM to execve (B, stopped by seccomp), and a misconfigured orchestrator deleting a drive (C).

Step 3 — Walk docs/prod-host-setup.md hardening line by line

The defenses in Labs 1–2 live inside the process. This step is the host-level hardening that the process sandbox assumes. Read the production guide and check each item on your host (or note what you'd check).

sed -n '1,200p' docs/prod-host-setup.md

# A subset you can actually run on a disposable host:
cat /sys/devices/system/cpu/smt/control 2>/dev/null
cat /sys/kernel/mm/ksm/run 2>/dev/null
swapon --show
# The MMDS egress block (example, nftables/iptables — adapt to your firewall):
sudo iptables -C FORWARD -d 169.254.169.254 -j DROP 2>/dev/null \
  && echo "MMDS egress block present" || echo "MISSING: block guest egress to 169.254.169.254"

Note: Most of these defend against side channels and physical attacks that the KVM boundary does not stop — SMT siblings, page dedup, Rowhammer. This is the layer people forget: hardware virtualization isolates architectural state, but micro-architectural and physical leakage need host-level mitigation. A threat model that stops at "we use KVM" is incomplete; this table is why.

• Produce a pass/fail line for each row against your host. For every fail, write the one-line remediation. This is the core of your deliverable checklist.

Step 4 — Reason through a real device vulnerability: CVE-2026-5747

Now make the audit concrete on a real bug. CVE-2026-5747 was a vulnerability in Firecracker's virtio-PCI transport (the --enable-pci path), fixed in 1.14.4 and 1.15.1 (verify the exact versions and details against the advisory and CHANGELOG on your branch). Walk it as both attacker and defender.

# The PCI transport is the newer, opt-in path (virtio-mmio is still the default).
rg -n "enable-pci|enable_pci|VirtioPci|pci|Pci" src/vmm/src/ src/firecracker/src/ | head
# Find the CHANGELOG entry and any security note.
rg -n "CVE-2026-5747|pci|PCI|security" CHANGELOG.md | head
sed -n '1,60p' SECURITY.md

As the attacker. The PCI transport, like all virtio transports, takes guest-supplied configuration (BAR accesses, capability reads, queue setup) and turns it into host-side state changes. A transport-layer bug — say, an out-of-bounds access or an integer overflow in handling a guest-controlled offset/length during config or queue setup — is reachable purely from inside the guest, over boundary A, with no host cooperation. That is the worst class: the guest drives it directly. Walk the path:

guest writes crafted value to a virtio-PCI config/BAR region
        │  (KVM_EXIT_MMIO / PIO → VMM transport handler)
        ▼  transport handler uses a guest-controlled offset/length
        ▼  IF unchecked: out-of-bounds read/write in the VMM's address space
        ▼  potential: info leak, or corruption → VMM code execution

As the defender. Now apply the four layers and ask what each one does given a successful boundary-A exploit:

The lesson is the whole intensive in one row: the minimal device model meant most deployments were never exposed (PCI is opt-in), and the jailer + seccomp meant that even a fully-exploited VMM could not reach another tenant. Defense in depth is not theater — it is the difference between "a config-parser bug" and "a host compromise." This is exactly why "do almost nothing" is a security argument, not minimalism for its own sake.

As the maintainer. Note how the fix flowed, because you may one day be on this side:

# How the project handles this: private report, patched releases, public advisory + CHANGELOG.
sed -n '1,80p' SECURITY.md   # "report privately to AWS Security, never a public issue"
rg -n "1.14.4|1.15.1|Security\|Fixed" CHANGELOG.md | head

• Reports go privately to AWS Security (SECURITY.md), never as public GitHub issues.

• The fix ships in patch releases on the supported minor lines (here 1.14.4 and 1.15.1), so operators on either line can upgrade without a major bump.

• The CHANGELOG and a public advisory document it after fixes are available — coordinated disclosure.

• Write a short attacker→defender narrative for CVE-2026-5747 in your own words: the trigger (boundary A, PCI config/queue setup), why Rust didn't catch it, why most deployments were unaffected (opt-in PCI), and what would have contained a full exploit (jailer + seccomp).

Step 5 — Reason about a hypothetical device bug, systematically

Generalize the CVE into a repeatable audit method. Pick any device from Step 1 (try the net device, whose data path you know from virtio-net deep dive) and run this five-question audit:

1. What guest-controlled input does it parse? (descriptor addr/len, virtio_net_hdr fields, frame bytes.)
2. Where does that input become a host action? (a read/write on the TAP fd; a guest-memory access; the MMDS detour.)
3. Is every length/offset bounds-checked before use? (rg for the vm-memory accessors; flag any raw arithmetic on guest-supplied values.)
4. If that check were missing, what could the guest do? (OOB read = info leak; OOB write = corruption → possible VMM code execution.)
5. Given a successful exploit, what contains it? (seccomp denies execve/ptrace/arbitrary opens; the jailer denies filesystem/process reach; the cgroup denies resource exhaustion.)

# Run the audit on the net TX path: find where guest-controlled length is used.
rg -n "fn process_tx|read_from_desc|frame_buf|write_to_tap|num_buffers|hdr_len" \
  src/vmm/src/devices/virtio/net/
# And the per-frame error handling — a malformed guest frame must NEVER take down the VMM thread.
rg -n "error|drop|continue|return|Result" src/vmm/src/devices/virtio/net/device.rs | head -30

• Apply the five questions to two devices and record the answers. Note any place where a guest-supplied value is used without an obvious bounds check — that is exactly what a real audit flags for closer reading (it may still be safe; your job is to prove it, not assume it).

Step 6 — Produce the audit checklist (the deliverable artifact)

Assemble everything into a single, reusable checklist a security reviewer could run. Organize it by the four layers plus host hardening. This is the lab's keep-forever output.

FIRECRACKER DEPLOYMENT — SECURITY AUDIT CHECKLIST
─────────────────────────────────────────────────
[ Boundary A — guest ↔ VMM ]
  [ ] virtio-mmio is the transport (PCI not enabled unless required & patched ≥1.14.4/1.15.1)
  [ ] device list is minimal — only devices in use are configured
  [ ] all guest-supplied addr/len go through bounds-checked vm-memory accessors
  [ ] per-frame / per-request errors are isolated (one bad descriptor never kills the VMM thread)

[ Boundary B — VMM ↔ host ]
  [ ] seccomp ENABLED (no --no-seccomp anywhere in production config)
  [ ] filters are the shipped/baked-in ones, or a reviewed custom filter compiled for this arch
  [ ] jailer in use: chroot + per-instance uid/gid + dedicated netns + cgroup limits
  [ ] only /dev/kvm and /dev/net/tun present in the jail; both chowned to the target uid/gid
  [ ] privileges dropped (gid before uid); VMM runs unprivileged

[ Boundary C — orchestrator ↔ API socket ]
  [ ] API socket lives inside the jail; filesystem permissions restrict who can open it
  [ ] orchestrator least-privileged; no broad host access leaks through staging

[ Host hardening (docs/prod-host-setup.md) ]
  [ ] SMT/hyper-threading OFF
  [ ] KSM OFF
  [ ] swap OFF
  [ ] guest egress to 169.254.169.254 blocked at the host firewall
  [ ] ECC + TRR RAM (Rowhammer)
  [ ] CPU microcode / kernel up to date for side-channel mitigations

[ Process / patching ]
  [ ] running a supported Firecracker version with current security patches
  [ ] a plan to consume patch releases (per SECURITY.md) and watch advisories

• Fill in every box with PASS / FAIL / N-A for your host (real or simulated), and a one-line remediation for each FAIL. Cross-reference each item back to the lab/deep-dive that explains it.

Deliverables

• The device-attack-surface table from Step 1, each device annotated with its guest-controlled input and a pointer to where that input becomes a host access.
• The three-boundary table from Step 2, with one concrete example crossing assigned to each.
• A pass/fail audit of every docs/prod-host-setup.md hardening item against your host, with remediations for failures.
• The CVE-2026-5747 attacker→defender narrative from Step 4, including the per-layer "stops trigger? / contains blast radius?" reasoning.
• The five-question audit applied to two devices (Step 5).
• The completed security audit checklist from Step 6, with every box marked and remediations noted.

Troubleshooting

You can't find a virtio-PCI directory in the source

PCI is the newer, opt-in transport and may sit under a feature flag; rg -n "enable.pci|VirtioPci|pci" src/ and check whether it is gated behind --enable-pci. If your branch predates it, audit the default virtio-mmio transport instead and note that the CVE-2026-5747 class did not apply to it (verify on your branch).

The hardening files (smt/control, ksm/run) don't exist on your host

Kernel build or virtualization layer may not expose them. Note "cannot verify on this host" and record what you would check on a production bare-metal host. The point is the method, not the specific sysfs path.

CHANGELOG/advisory details for the CVE differ from this lab

Treat the advisory and CHANGELOG.md on your branch as ground truth and this lab as a signpost — exact fixed versions and wording are version-sensitive (verify on your branch). The reasoning method (trigger on boundary A, contained by layers 3–4) is the durable part.

Expected output

• A complete enumeration of the device attack surface — a short list, by design — with each device's guest-controlled input named.
• A per-host pass/fail against the production hardening guide.
• A coherent attacker-and-defender account of CVE-2026-5747 showing that the minimal device model kept most deployments unexposed and the jailer + seccomp would have contained any full exploit.
• A reusable audit checklist you could hand a reviewer.

Stretch goals

1. Audit a snapshot/restore deployment. Snapshots add attack surface: a malicious or corrupted snapshot file is parsed by the VMM on load. Read the snapshotting deep dive and add snapshot-file integrity to your checklist (who can write the snapshot? is it validated on load?).
2. Write the rejection comment. Imagine a PR proposing a new emulated device (say, a virtio-gpu). Using the minimal-device-model argument and your Step 1 table, write the review comment that asks the right questions: what attack surface does it add, can it be opt-in, what is the bounds-checking story? See pr-quality.
3. Map every defense to the CVE that motivated it. For SMT-off, KSM-off, and ECC RAM, find the class of public hardware vulnerability (L1TF/MDS, memory-dedup side channels, Rowhammer) each mitigates, and add a one-line citation to your checklist. Defenders justify every control.
4. Trace the MMDS attack surface end to end. dumbo parses guest-supplied packets (MMDS deep dive). Run the five-question audit on the dumbo TCP/IP parser and connect it to the 169.254.169.254 egress-block hardening.

Validation / self-check

Answer without notes; these gate completion and the intensive.

• Enumerate Firecracker's emulated devices and, for each, name the guest-controlled input that is its attack surface. Why is the list deliberately short?
• Name the two trust boundaries, the untrusted side of each, and the front-line defense plus backstop for each. Why is the API socket not a guest-facing surface?
• List the docs/prod-host-setup.md hardening items and state, for SMT-off, KSM-off, and ECC RAM, the specific class of attack each blocks — and why the KVM boundary alone does not.
• For CVE-2026-5747: where was the trigger (which boundary, which code), why did Rust not catch it, why were most deployments unaffected, and what would have contained a full exploit?
• Walk the five-question device audit for one device and identify the bounds-check that is the front-line defense on boundary A.
• How does a security vulnerability flow through the project — report, fix, disclosure — and where do you send a report?
• Defense in depth: explain, using one concrete exploit, why each of the four layers matters and what is lost if any single one is removed.

Next: you have completed the Security intensive. Continue with the Networking intensive — TAP devices, host packet plumbing, rate limiting, and MMDS — or return to the masterclass index to choose your next intensive. To turn this audit skill into contributions, see the security/seccomp issue stage.