The Maintainer Mindset: Compatibility & Risk

This chapter inverts the table. You've spent the curriculum learning to write changes; this chapter is about the questions a Firecracker maintainer asks while deciding whether to accept yours — and why those questions are weighted so differently from a normal application codebase. The thesis is blunt: a Firecracker maintainer's default posture toward any change is suspicion, not enthusiasm, because the project is security-critical multi-tenant infrastructure, every change is a potential attack-surface expansion or compatibility break, and the cost of a bad merge is measured in CVEs and in production breakage across Lambda, Fargate, and every downstream that pins a Firecracker version. Understanding that posture is what lets you write the rare PR that earns a yes.

If you internalize one thing: at Firecracker, "the patch works" is the start of the review, not the end. A working patch that adds surface, weakens a guarantee, or constrains a future release can be — and routinely is — declined. This chapter is the taxonomy of what the maintainer is protecting.

Note: This is the receiving end of compatibility.md and the cultural sibling of the minimal-device-model philosophy. Read those for the contributor's discipline and the security argument; read this for how that discipline is enforced in review.

The Five Things a Maintainer Is Protecting

Almost every "why did this clean PR get pushback?" reduces to one of these. Map your change against them before you open the PR, and pre-empt the objection in your description.

The rest of this chapter takes each in turn.

1. The API-Change Runbook

The REST API is a published contract. Orchestrators (firecracker-containerd, the Go SDK, Kata, every operator's homegrown controller) are written against the OpenAPI spec, and a broken contract is a silent production outage downstream. So API changes follow an unwritten-but-rigid runbook:

cd ~/fc-src
# The API contract itself — every change to behavior must be reflected here:
sed -n '1,40p' src/firecracker/swagger/firecracker.yaml
rg -n "version:|paths:|/machine-config|/snapshot|/drives" src/firecracker/swagger/firecracker.yaml | head
# How additive, versioned changes were worded in the past — copy the pattern:
rg -n -i "added|new field|deprecat|renamed|breaking" CHANGELOG.md | head -30

The rules the maintainer applies:

• Additive, not breaking. A new optional field or a new endpoint is the easy case. Changing the meaning of an existing field, making an optional field required, or removing a field is a breaking change that, by SemVer, waits for a major version — and Firecracker majors are rare and deliberate.
• The swagger and the CHANGELOG move with the code. An API change with no firecracker.yaml update and no CHANGELOG entry is incomplete; the contract has to be machine-readable and announced.
• Renames are removals. Recall from the fact sheet that enable_diff_snapshots was renamed to track_dirty_pages and the standalone mem_file_path-on-load was deprecated — those weren't free. A rename means supporting both the old and new name through a deprecation window, then removing the old one only at a version boundary, with CHANGELOG Deprecated then Removed entries.
• Pre-boot vs runtime matters. Many endpoints are valid only before InstanceStart (PrebootApiController) vs after (RuntimeApiController). A change that lets something be set at the wrong phase is a correctness and compatibility question, not a convenience.

Tip: Before proposing an API change, read how the request flows from the socket to a VmmAction in the API server and action channel deep dive. A maintainer can tell instantly whether you understand that path; a PR that mis-models pre-boot vs runtime betrays that you don't.

2 & 3. Snapshot and Version Compatibility — The Hardest Gate

This is the gate that declines the most well-meaning PRs, and the one that surprises engineers most. A Firecracker snapshot captures the full microVM state — KVM state plus every device's state via the Persist trait — to a file, and that file must be restorable by a later Firecracker version. Snapshot/restore across versions is a load-bearing production feature (it's how AWS does fast scaling and live upgrades), so the on-disk snapshot format is a near-permanent contract.

cd ~/fc-src
# Every device implements Persist; its persisted state IS the compatibility contract:
rg -n "impl Persist|trait Persist|struct .*State" src/vmm/src/devices/ | head -30
# The snapshot/versioning machinery — the SnapshotHdr, version constants, the format:
rg -n "version|VERSION|magic|MAGIC|SnapshotHdr|Versionize" src/vmm/src/snapshot/ src/vmm/src/persist.rs | head
sed -n '1,80p' docs/snapshotting/snapshot-support.md 2>/dev/null

What the maintainer scrutinizes in any diff that touches a device's persisted state:

• Does it change a ...State struct that is serialized into a snapshot? Adding, reordering, or removing a field changes the format. A snapshot taken by version N must still load on version N+1, and (within policy) the reverse may matter too. This is why a seemingly innocent "add a field to the block device state" is a compatibility event, not a refactor.
• Is the change versioned correctly? Firecracker's snapshot machinery carries explicit version information so old snapshots can be migrated. A new field has to be introduced in a way the version logic understands, with a sane default for older snapshots that lack it.
• Was it tested across versions? The integration suite includes cross-version snapshot/restore tests. A snapshot-touching PR with no cross-version test is incomplete on its face. See the snapshotting deep dive and snapshotting at scale.

Warning: "It serializes fine on my branch" proves nothing about compatibility. The question is whether a snapshot from the released version restores on yours and vice versa within the support window. If you can't answer that with a test, the maintainer can't accept the change. This is the single most common reason a competent contributor's first device-state PR is sent back.

The Minimal-Device-Model Gate

Layered on top of the attack-surface concern is an explicit cultural gate: does this even belong in Firecracker? The default answer to "should we add this device / this knob / this format" is no, and the change must earn its way in. This is not the maintainer being obstructive — it is the minimal-device-model philosophy being applied, the thing that keeps the attack surface small enough for AWS to run thousands of untrusting tenants per host.

cd ~/fc-src
# The whole device catalog — note how short it is. Adding to it is a big deal.
ls src/vmm/src/devices/virtio/
ls src/vmm/src/devices/legacy/
# The scope statement that backs the maintainer's "no":
sed -n '1,60p' CHARTER.md

The questions in this gate:

• Is this surface attacker-reachable? Anything that parses guest-controlled data (a virtqueue, a config field, an MMDS request) is host code an exploit can target. New such code faces the highest bar and the most careful review.
• Could this live behind a feature flag / be off by default? The PCI transport, for example, exists behind --enable-pci — opt-in, so it doesn't enlarge the default attack surface. (It's also the cautionary tale: the PCI transport is where the CVE fixed in 1.14.4 / 1.15.1 lived — see release-process.md. New surface is new risk, demonstrably.)
• Is "QEMU/Cloud Hypervisor has it" the only argument? That argument carries no weight. Those projects have different threat models and scopes. Firecracker's value is in not having most of what they have.

4. Performance Gates

Firecracker's product is numbers: boot in under 125 ms, under 5 MiB overhead per microVM, oversubscription past 20x, thousands per host. A change that erodes any of those erodes the reason the project exists, so performance-sensitive changes carry a numbers-or-it-didn't-happen burden.

The maintainer expects a perf-relevant PR to come with measurements, not assurances. "It shouldn't affect performance" is not data. Run the relevant tests, paste the before/after, and if there is a regression, justify it against the benefit. A latency regression with no number attached is treated as a regression that the author didn't measure.

5. "Does This Belong Upstream in rust-vmm?"

A recurring redirect — and a sign of a maintainer thinking about the ecosystem, not just the repo — is "this should go to rust-vmm, not into Firecracker's tree." rust-vmm is the set of community crates (kvm-ioctls, vm-memory, linux-loader, vm-superio, event-manager, …) shared between Firecracker, Cloud Hypervisor, and others. Firecracker has donated code upstream (seccompiler, event-manager, vm-superio all originated in or were contributed by Firecracker). So when your change is a piece of generic VMM machinery rather than something Firecracker-specific, the maintainer may well say: this belongs in the shared crate, where Cloud Hypervisor benefits too and the logic has one home.

cd ~/fc-src
# What FC pulls from rust-vmm vs maintains in-tree — the boundary the maintainer polices:
rg -n "kvm-ioctls|kvm-bindings|vm-memory|linux-loader|vm-superio|event-manager|vmm-sys-util|vm-fdt" Cargo.toml

For you this is good news disguised as a redirect: a contribution to a rust-vmm crate is higher leverage (multiple VMMs benefit), and rust-vmm has its own, more conventional community governance — a path to influence that single-vendor Firecracker doesn't offer (project-governance.md, the rust-vmm section). When a maintainer points you upstream, take it as a route to broader impact, not a brush-off. See lab-r4.

Why Some Good PRs Are Still Declined

This is the chapter's hardest lesson, and the one that separates a contributor who grows from one who burns out. A PR can be correct, well-tested, cleanly formatted, signed off — and still be declined. Not because the maintainer is wrong or political, but because:

• It expands the attack surface more than its benefit justifies (Gate 1 / the minimal-device-model gate).
• It constrains a future release — pins a format, an API, or a behavior the maintainers want to keep free (Gates 2/3).
• It belongs downstream or upstream, not in the VMM core (Gate 5).
• It doesn't fit the scope in CHARTER.md, however nice it is in the abstract.
• The timing is wrong — it's risky during a release-stabilization window (release-process.md).

The professional response is not to argue harder or take it personally. It is to (a) make sure you proposed it as an issue first so you didn't build it on spec (communication-channels.md), (b) understand the recorded reasoning, and (c) either reshape the change to fit or accept the decision and move to one that does. Maintainers remember the contributor who took a clean "no" gracefully far more fondly than the one who litigated it — and that memory is exactly the trust that, over time, earns you de facto ownership of an area (project-governance.md, code-style-trust.md).

How to Pre-Empt the Maintainer's Objections

Turn the five gates into a pre-flight you run on your own PR:

• Attack surface: Does this add a device, a syscall, or host-side parsing of guest data? If yes, justify it explicitly and consider an opt-in flag.
• API: Is every change additive and versioned? Is firecracker.yaml updated? Any rename handled as deprecate-then-remove at a version boundary?
• Snapshot: Does this touch a persisted ...State struct? If yes, is it versioned, defaulted for old snapshots, and covered by a cross-version restore test?
• Performance: Is there a plausible boot-time/memory/IO impact? If yes, measure it and paste numbers.
• Scope: Does this belong in the VMM core, or in rust-vmm / downstream? Does it fit CHARTER.md?

A PR description that walks the maintainer through these five points, honestly, is a PR that says I think like you do. That is what gets the benefit of the doubt.

Prove You Understand This

1. Name the five things a Firecracker maintainer is protecting, and give one change that threatens each.
2. Why is "the patch works" the start of the review, not the end? Give two reasons a correct PR is declined.
3. Walk the API-change runbook: what makes a change additive vs breaking, and what does a rename actually cost?
4. Why is snapshot compatibility the hardest gate? What must a PR that adds a field to a device's persisted state include before a maintainer can accept it?
5. A maintainer says "this belongs in rust-vmm." Why might that be better for your impact and your path to influence?
6. Reproduce the five-point pre-flight you'd run on your own PR to pre-empt these objections.

Next: The Release Process and Policy — once the two maintainers approve and merge, how does your change become something a user actually runs?