Project 7: A Seccomp Filter Audit & Diff Tool

Firecracker's seccomp filter is the last line of defense in the threat model. After the KVM boundary, after the jailer's chroot/namespaces/privilege-drop, a per-thread seccomp-BPF filter restricts the host syscalls each Firecracker thread is allowed to make at all. If a guest somehow corrupts the VMM, the seccomp filter is what stops the compromised process from execve-ing a shell, opening arbitrary files, or making the syscalls an exploit chain needs. The filter is an allowlist: every syscall Firecracker genuinely makes must be on it (or the process dies on a disallowed syscall), and every syscall on it that Firecracker doesn't actually need is unnecessary attack surface.

The allowlist is maintained by hand, in JSON, per thread category (vmm, api, vcpu), and compiled to BPF at build time. Hand-maintained allowlists drift: a refactor adds a syscall the filter doesn't cover (a latent crash, or a too-loose filter that was widened to avoid one), or a syscall stays on the list long after the code that needed it was deleted. This project asks you to build the tool that catches that drift: a seccomp audit tool that observes the syscalls Firecracker actually makes (via strace/ptrace or the audit subsystem), diffs them against the JSON allowlist, and reports both the gap (syscalls used but not allowed — latent bugs) and the slack (syscalls allowed but never used — removable attack surface), with a concrete tightening proposal.

Note: Read the seccomp filtering deep dive in full and do the security masterclass, especially Lab 2: seccomp filters and Lab 3: threat-model audit, plus Level 9 Lab 1: seccomp & jailer. This brief assumes you understand seccomp-BPF, the per-thread-category filter model (vmm/api/vcpu), the SyscallRule shape (a syscall plus optional args with operators like eq/ge/masked_eq), how seccompiler compiles JSON → BPF and bakes it into the binary, and the --seccomp-filter/--no-seccomp overrides. If those are fuzzy, do the lab first — this brief will not re-derive seccomp.

Problem & motivation

The seccomp allowlist is a hand-maintained security artifact, and hand-maintained security artifacts decay in two directions, both bad:

• Too tight is a crash. If Firecracker's code makes a syscall the filter doesn't allow, the process is killed (or the call returns EPERM, depending on the default_action) the moment that path executes — often only under a specific, rare condition (an error path, a particular device op, a snapshot edge). These are latent, hard-to-find bugs: the filter is "correct" until the day that code path runs in production. A tool that proves every syscall the code can make is covered would catch them before release.
• Too loose is attack surface. Every syscall on the allowlist that Firecracker no longer needs is a syscall an attacker who compromised the VMM can use. Allowlists accrete: a syscall added for a feature that was later removed, an argument constraint loosened during debugging and never re-tightened, a whole syscall allowed when only one of its argument shapes is needed. Each is removable attack surface — and the threat model's entire argument is that attack surface should be minimal.

Firecracker takes this seriously — the filters live in resources/seccomp/<arch>.json, are documented in docs/seccomp.md, and the project even ships filter tooling — but the verification that the allowlist matches reality is exactly the kind of thing that benefits from a dedicated tool. The motivation is the sharpest in the portfolio: this is the security boundary, the numbers are syscalls, and a tightening proposal backed by evidence is a direct, auditable improvement to the threat model.

What you'll build

A seccomp audit tool that produces an evidence-based diff between the syscalls Firecracker actually makes and the syscalls its allowlist permits:

1. An observer that records the real syscall usage of each Firecracker thread category across a representative workload — boot, device I/O, snapshot/restore, API calls, error paths — using strace -f/ptrace or the kernel audit subsystem (auditd/SECCOMP_RET_LOG).
2. A parser for the JSON allowlist (resources/seccomp/<arch>.json) that understands the per-category structure and the SyscallRule arguments.
3. A differ that reports, per thread category:
   • Gap — syscalls observed but not allowed (latent crash risk).
   • Slack — syscalls allowed but never observed (removable surface).
   • Argument slack — syscalls allowed with looser argument constraints than the observed usage requires (tightenable).
4. A tightening proposal — a concrete diff to the JSON (remove unused syscalls, narrow argument constraints), with the evidence for each change and the caveat that "never observed" is not "never used" (you must reason about coverage).

The deliverable is the tool plus a written audit with a defensible tightening proposal — not a blind allowlist edit. The reasoning is the contribution.

Prerequisites

• Level 9, especially Lab 1: seccomp & jailer.
• The seccomp filtering deep dive and the jailer deep dive.
• The security masterclass, all three labs.
• The rust-vmm seccompiler chapter (FC donated seccompiler upstream — the compiler that turns the JSON into BPF).
• The VMM threading model deep dive — the filter is per thread category, so you must know which threads make which syscalls.
• Working knowledge of strace/ptrace and Linux syscall numbers per architecture (they differ between x86_64 and aarch64 — this matters).

Phased plan

Phase 0 — Build, read the allowlist, and run with logging (1–2 days)

Find the filters, read them by category, and observe Firecracker's real syscalls the easy way first.

tools/devtool build --release
# The filters and the compiler:
ls resources/seccomp/ && cat resources/seccomp/$(uname -m)*.json | head -60
rg -n "vmm|api|vcpu|default_action|filter|syscall|SyscallRule|args|comment" resources/seccomp/
rg -n "fn|SeccompFilter|apply|install_filter|compile" src/seccompiler/src/ src/vmm/src/seccomp.rs
# Where each thread installs its category filter (which thread = which category):
rg -n "seccomp|install|apply_filter|ThreadCategory|vmm|api|vcpu" src/vmm/src/ src/firecracker/src/

Observe the real syscalls. Two complementary methods — use both:

# Method A: strace every thread across a full lifecycle (boot → I/O → snapshot):
sudo strace -ff -e trace=all -o fc-trace ./firecracker --api-sock /tmp/fc.sock --no-seccomp &
#  ... drive a full workload over the API, then:
#  collect distinct syscalls per traced thread file (fc-trace.<tid>)
sort -u <(grep -hoE '^[a-z_0-9]+\(' fc-trace.* | tr -d '(')

# Method B: run WITH seccomp in LOG mode (SECCOMP_RET_LOG) to see what would be blocked:
#  build a log-mode filter (default_action = log) and watch the kernel audit log:
sudo dmesg -w | rg -i "seccomp|audit"      # disallowed syscalls show up here

Anti-staleness: the seccomp JSON path (resources/seccomp/), the per-arch filenames, the SyscallRule schema, the thread-category names, and where each thread installs its filter all move between releases. Confirm on your branch with rg/find, read docs/seccomp.md, and check git log --oneline -- resources/seccomp/ src/seccompiler/ and gh issue list --repo firecracker-microvm/firecracker --search "seccomp" for recent changes and live work. The syscall numbers also differ per architecture — never hard-code them.

Warning: Running --no-seccomp disables the security boundary — only ever in a throwaway audit environment, never anything resembling production. The point of running without (or in log mode) is to observe the syscalls so you can tighten the real filter, not to ship without one.

Produce capstone-work/seccomp-map.md: the per-category allowlist (which syscalls, with which argument constraints), how each thread installs its filter, and your two observation methods with their coverage limits noted.

Phase 1 — Build the observer and the allowlist parser

Make syscall observation reproducible and per-category. The hard part is attributing each observed syscall to the right thread category (vmm vs api vs vcpu), because the filter is per-category and a syscall allowed for vmm but used by vcpu is still a gap.

  Firecracker process
   ├─ API thread   ── category: api   ─┐
   ├─ VMM thread   ── category: vmm   ─┤── strace -ff gives one file per TID;
   └─ vCPU thread  ── category: vcpu  ─┘   map TID → category → observed syscall set

Parse the JSON allowlist into the same per-category structure, including argument rules, so the two are directly comparable. Write tests for the parser against the real JSON.

Milestone 1: for a given workload, your tool emits, per category, the set of syscalls observed and the set allowed — the two inputs to the diff.

Phase 2 — The differ: gap, slack, and argument slack

Now compute the three diffs that matter, per category:

# The shape of the tool's output:
./seccomp-audit --filter resources/seccomp/x86_64.json --traces fc-trace.* \
  --report capstone-work/seccomp-audit.md
# → per category: GAP (fix the filter or you'll crash), SLACK (candidate to remove),
#                 ARG-SLACK (candidate to narrow), with the evidence for each.

Milestone 2: a per-category report. A non-empty gap for any category is a real finding — either a latent bug (the filter is too tight for a path your workload hit) or a coverage gap in your workload. Investigate every gap; do not assume.

Phase 3 — Coverage and the tightening proposal (the judgement)

This is the phase that separates a tool from a security tool. "Never observed" does not mean "never used" — it means your workload didn't exercise the path. A tightening proposal that removes a syscall the rare error path needs reintroduces exactly the kind of latent crash this project exists to prevent. So:

• Characterize coverage. Enumerate the paths your workload exercised (boot, block I/O, net I/O, vsock, balloon, snapshot create, snapshot restore, API calls, error paths) and state what you did not exercise. Slack found under thin coverage is a candidate, not a conclusion.
• Cross-reference the code. For each slack syscall, rg the codebase for where it's called (directly or via a libc/crate wrapper) before proposing removal. If the code can still reach it, it's not slack — your coverage missed it.
• Propose narrowing, not just removal. Argument-slack tightening (e.g. an mmap allowed with any flags when only one flag shape is used, an ioctl allowed for any request when only specific KVM ioctls are used) is often a safer, higher-value tightening than removing a whole syscall.

Milestone 3: a tightening proposal — a concrete JSON diff — where every change carries (a) the evidence it's unused, (b) a code search confirming the code can't reach it, and (c) an explicit statement of the coverage that backs the claim.

Phase 4 — Validate the tightening

A tightening you can't validate is a guess. Apply the proposed filter and prove Firecracker still works across the full workload — including the paths you used to argue coverage.

# Build with (or run --seccomp-filter pointing at) the tightened filter:
./firecracker --api-sock /tmp/fc.sock --seccomp-filter ./tightened.bpf &
#  ... run the ENTIRE workload: boot, every device, snapshot create+restore, error paths ...
#  any path that now hits a removed syscall will be killed/EPERM — that's the test.
sudo dmesg | rg -i seccomp       # must be clean; a kill means you cut too much

Then run the integration suite under the tightened filter (rg -n "seccomp|no_seccomp|def test_" tests/integration_tests/ | head) — the existing seccomp tests are your template and your safety net.

Key code areas

The seccompiler crate is the same one FC donated to rust-vmm; reuse its JSON parsing rather than reinventing it where you can (cargo doc -p seccompiler if it's the external dep on your branch — verify).

Design considerations & trade-offs

• Observation is necessarily incomplete. Dynamic tracing only sees the paths you run. The whole project hinges on being honest about coverage: state what you exercised, treat slack as a candidate, and confirm with a code search before proposing removal. Over-claiming here is a security regression, not a style nit.
• Per-category, not per-process. A syscall allowed for vmm but used by a vcpu thread is a gap for vcpu. Attributing syscalls to the right thread category is the core technical challenge; get it wrong and your diff is meaningless.
• Per-architecture. Syscall numbers and even which syscalls exist differ between x86_64 and aarch64, and the filters are per-arch. Your tool must be arch-aware and your proposal must hold on both (or be scoped to one and say so).
• Gap > slack in priority. A gap is a latent crash that ships; slack is surface that might be exploitable only after a separate VMM compromise. Both matter, but a gap is a bug to fix now; slack is a hardening to propose carefully.
• Argument tightening is the high-value, low-risk win. Removing a whole syscall is risky (coverage). Narrowing an argument constraint to what's actually used (e.g. restricting ioctl to the specific KVM request numbers FC issues) tightens surface with far less risk of breaking a rare path — and is often where the real, mergeable hardening is.
• The default_action matters. Whether a disallowed syscall kills the thread (trap/kill) or returns EPERM (errno) changes the failure mode and how you detect a gap. Read the current default_action per category; don't assume.

How to test & validate

• Tool correctness: unit-test the allowlist parser against the real JSON and the differ against synthetic observed/allowed sets with known gap/slack/arg-slack.
• Gap validation: any gap your tool reports must be reproduced — run the path with the real (un-loosened) filter and show the kill/EPERM, or show it was a coverage artifact.
• Tightening validation: apply the tightened filter and run the entire workload plus the pytest seccomp/security tests; dmesg must be clean. A single seccomp kill means the tightening cut a syscall a real path needs — back it out and refine.
• Cross-arch: if you claim a tightening, validate it on both x86_64 and aarch64, or scope the claim to the arch you tested and say so explicitly.
• The tool is a deliverable — it runs with one command against a stated workload and emits the per-category report reproducibly.

Stretch goals

• Coverage-driven workload: drive the observation from the integration test suite itself (tools/devtool test) so the observed set reflects the project's own notion of "exercised behavior" — dramatically stronger coverage than a hand-run workload.
• A CI-shaped check: package the gap detector as a check that, run under log-mode seccomp during the test suite, fails if any thread hits a disallowed syscall — catching too-tight filters before release. This is genuinely the form the maintainers want.
• Argument-level tightening PR: turn one well-evidenced argument-slack finding (e.g. narrowing an ioctl or mmap rule) into a real, scoped, tested Firecracker PR.
• Diff across releases: run the audit on two FC versions and report how the real syscall footprint changed — surfacing both new gaps and new slack introduced by a release.

What a strong deliverable looks like

A strong deliverable is a seccomp audit tool that observes real per-category syscall usage across a stated workload, diffs it against the JSON allowlist into gap / slack / argument-slack, and produces an evidence-backed tightening proposal — every proposed change justified by observation and a code search and an explicit coverage statement, then validated by running the tightened filter across the full workload with a clean dmesg.

The upstreaming path is strong because this is security tooling the project values:

1. Two mergeable shapes. A gap finding (the filter is too tight for a real path) is a bug fix — clean and mergeable. An argument-tightening (narrowing a rule to what's used) is a hardening — mergeable with evidence. Find the live state: gh issue list --repo firecracker-microvm/firecracker --search "seccomp OR syscall filter" and read docs/seccomp.md.
2. Evidence is the contribution. A blind allowlist edit gets rejected; an edit backed by observation, a code search, a coverage statement, and a validation run is exactly what a security-conscious maintainer can approve. The reasoning is the deliverable.
3. The tool may be wanted too. A CI-shaped log-mode gap detector — something that catches a too-tight filter during the existing test run — is the kind of tooling that protects the threat model continuously.

Even if no filter change lands upstream, the tool plus the audit write-up is a portfolio-grade artifact: it demonstrates you can reason rigorously about the project's last line of defense. A finished version at 90+ on the rubric is maintainer-grade security work.

Next: Project 8 — an MMDS feature extension, the last brief in the portfolio, or back to the portfolio overview to pick your one or two.