Step 7: Validation

Your fix compiles, clippy is quiet, and your new tests are green. That is the bar for writing a fix; it is not the bar for shipping one into privileged host code that runs untrusted guests for AWS Lambda and Fargate. Validation is the step where you stop trusting your own change and try to break it: re-run the original reproduction end to end, hunt for the regressions your fix might have introduced, and self-review against the checklist a Firecracker maintainer will apply before they spend their two approvals on you.

The mindset shift: in Steps 5 and 6 you were proving the fix works. Here you are proving the fix did not break anything else — and proving it with evidence, not optimism. "It works on my machine" is the floor of this project, not the goal.

Goal

Write capstone-work/validation.md: the original repro re-run and now passing, the full gate suite green (checkstyle, checkbuild --all, the relevant test subsets on both architectures where reachable), and an explicit regression pass across the four Firecracker risk axes — performance, snapshot compatibility, seccomp/attack surface, and API/boot behavior — each either checked with a command or explicitly ruled out in writing.

Re-run the Original Reproduction, Unchanged

Go back to capstone-work/repro.md from Step 2 and run the exact sequence that demonstrated the bug — same curl/config-file/pytest commands, same artifacts, same version of everything except your fix. The point is to close the loop with the same instrument you opened it with.

# The same repro from Step 2, now against your patched build.
tools/devtool build
./build/cargo_target/x86_64-unknown-linux-musl/debug/firecracker --api-sock /tmp/fc.sock &
# ... the identical Step-2 curl sequence / config file ...
# Observe: the symptom is gone, and the behavior is the documented-correct one.

Then do the harder confirmation: build main without your change and re-run the same repro to confirm it still fails there. A fix that "passes" against a tree that no longer reproduces the bug for an unrelated reason (a different artifact, a masked error) is not validated. You want the clean contrast: fails on main, passes with your branch, same inputs.

Tip: If your Step 2 repro was a config-file boot or a curl sequence rather than a pytest, that is fine for validation — but the permanent guard is the Step 6 test. Validation re-runs the human repro; the test file is what protects the fix forever. Keep both.

The Four Regression Axes

Every Firecracker change is judged against four risk axes. For each, either run a check or write one sentence ruling it out — silence on any of them is what a reviewer flags. These are the same dimensions you considered in Step 4; now you measure them.

Performance regressions

If the fix lives anywhere near the fast path, measure. Firecracker's tests/integration_tests/performance/ suite and the project's A/B testing (tests/framework/ab_test.py; CI runs A/B comparisons) exist precisely for this. At minimum, sanity-check boot time and the throughput of any device you touched:

# List the perf tests and run the one for the subsystem you touched.
ls tests/integration_tests/performance/
tools/devtool test -- integration_tests/performance/ -k boottime   # or block/net/...

A correctness fix that only fires in the error case should show no measurable delta on the common path. If your diff added a per-descriptor allocation, a lock, or a check inside the run loop's hot loop, the number will tell you — and you go back to Step 5 to move the cost off the common path. State the result either way: "no statistically significant boot-time delta across N runs" is a sentence a reviewer wants to see.

Snapshot compatibility regressions

Snapshots are a documented compatibility contract. If you touched device state, serialization, or anything under persist.rs / a device's Persist impl, prove the round trip both directions where it makes sense:

# Create on one build, restore on the other, resume, assert the guest survives.
tools/devtool test -- integration_tests/functional/ -k snapshot

If you did not touch serialized state, say so explicitly: "No Persist state or snapshot layout changed; no version bump or compat test required." That single sentence saves a reviewer a careful read of persist.rs. See compatibility and snapshotting.

Seccomp and attack-surface regressions

This is the axis new contributors most often forget, and the one maintainers care about most, because Firecracker's whole value is a small attack surface. Ask: did your fix cause firecracker to call any syscall it didn't before? Add an emulated device? Widen a guest-reachable code path?

# Did the seccomp filters change? They should NOT, unless that IS the fix.
git diff origin/main -- resources/seccomp/
# Re-run the seccomp/jailer security tests.
tools/devtool test -- integration_tests/security/ -k "seccomp or jail"

A correctness fix should shrink or hold the attack surface, never grow it silently. If your change genuinely needs a new syscall (rare), that is a deliberate, documented seccomp-filter change with its own review weight, and the filter test must pin it. If it does not, confirm the filters are byte-identical to main. For a guest-data-plane fix, re-confirm that every malformed input is rejected, not just the reported one — Step 6's adversarial cases are part of this axis. See the threat model and seccomp filtering.

API and boot behavior regressions

Run the broad functional smoke for the surface you touched and diff the behavior of the cases you did not intend to change:

tools/devtool test -- integration_tests/functional/test_api.py
# And a plain boot, to confirm the unaffected boot path is untouched:
#   the same Step-2 boot WITHOUT the trigger should behave exactly as on main.

If the swagger spec (src/firecracker/swagger/firecracker.yaml) describes the endpoint you changed, confirm your behavior matches the spec — or, if you had to change the spec, that is a contract change you call out loudly in the PR.

The Maintainer Self-Review Checklist

Before a maintainer reads your diff, read it as one. Open git diff origin/main and git diff origin/main --stat and walk this checklist. Each "no" is a comment you can pre-empt now instead of receiving in Step 9.

git diff origin/main --stat   # only the files the root cause predicted?
git diff origin/main          # every line justifiable in one sentence?
git log origin/main..HEAD --format='%H %s%n%b' | rg -i "signed-off-by"  # every commit signed?

Warning: The single most common reason a clean fix gets a "please revise" is scope creep that snuck in during Steps 5–6 — a reformatted block, a renamed local, a "while I was here" tidy in a test file. Run git diff origin/main one more time with fresh eyes and rip out anything you cannot justify in one sentence. A 6-line diff that reads in 30 seconds clears two approvals faster than a 60-line diff with 6 lines of fix.

When Validation Fails (the loop back)

Validation is a gate, and gates exist to send you back. The capstone flow's dotted arrows are real:

• A perf regression → back to Step 5: move the cost off the hot path.
• A broken snapshot round-trip → back to Step 5 with a version story, and Step 6 for the compat test.
• A seccomp diff you didn't intend → you added a syscall somewhere; find it and remove it, or justify and pin it.
• The test passes on main too → back to Step 6: the test isn't testing your fix.
• A second, different failure appears → back to Step 3: your localization missed something.

Do not paper over a failed axis to keep moving. A regression you ship is worse than the bug you fixed, because it lands in privileged host code with your name on the sign-off.

Deliverable for Step 7

• capstone-work/validation.md: the original repro re-run, failing on main and passing on your branch with identical inputs.
• Each of the four regression axes (perf, snapshot, seccomp/surface, API/boot) either checked with a recorded command+result, or explicitly ruled out in one sentence.
• tools/devtool checkstyle and checkbuild --all green on the branch; the relevant test subsets green; both architectures considered.
• The 10-point maintainer self-review walked, with any "no" resolved.
• git diff origin/main --stat shows only the predicted files; every line justifiable in one sentence.

Rubric Hooks

Validation feeds Fix quality (no regressions, attack-surface and compatibility discipline) and Tests (the re-run proves red-before/green-after end to end). It is also the evidence base for the write-up's Performance/Behavior Impact and Final Design sections in Step 10. A validation doc that names all four axes with commands scores high; "I re-ran it and it worked" scores low. See the evaluation rubric.

Validation / Self-check

Before advancing to Step 8:

1. The original Step 2 repro fails on main and passes on your branch, same inputs — you ran both.
2. You checked or explicitly ruled out a regression on each of the four axes: performance, snapshot compat, seccomp/attack surface, API/boot behavior.
3. The seccomp filters under resources/seccomp/ are byte-identical to main (or the change is deliberate and pinned by a test).
4. You measured performance if the fix is anywhere near a hot path, and recorded the result.
5. You walked the 10-point maintainer checklist and resolved every "no."
6. git diff origin/main --stat shows only the files your root cause predicted, and checkstyle + checkbuild --all are green.

Then go to Step 8: PR Preparation.