Stage 5 — Device Configuration Issues

What class of issue this is

Stage 5 takes the validation skill from Stage 4 and points it at the device configuration layer — the seam where an API config struct becomes a real emulated device. Each device has a config type, a builder that constructs the device from it, and (for net/block) a rate limiter; MMDS has its own config and contents. The bugs here are richer than Stage 4's generic bounds checks because the config feeds device behaviour: a wrong rate-limiter param actually thro ttles I/O incorrectly, a mishandled cache_type changes the block device's flush semantics, a bad vsock CID or MMDS network config makes a feature silently not work.

Concretely, a Stage 5 PR is one of:

• A block-device config issue: is_read_only / cache_type (Unsafe/Writeback) handling, io_engine (Sync vs Async/io_uring) selection, path_on_host validation, PATCH-at-runtime semantics.
• A net-device config issue: host_dev_name (TAP) handling, guest_mac, the RX/TX rate limiters.
• A vsock config issue: guest_cid, uds_path handling.
• A balloon config issue: amount_mib, deflate_on_oom, stats_polling_interval_s, and the statistics endpoint.
• A rate-limiter config issue: the bandwidth/ops token buckets (size, one_time_burst, refill_time), shared by net and block.
• An MMDS config issue: the version (V1 deprecated / V2 token-session), the network stack binding, the IPv4 address / allowed interfaces.

Why it's at this difficulty

You are now inside the device model. The config validation is Level 3 work, but understanding what a mis-set field does requires Level 7 (the virtio device model) for the I/O path and the rate limiter, and the MMDS deep dive for the metadata stack. That is why this stage straddles Levels 3 and 7: you can fix the config-validation half with Level 3, but to fix what the device does with the value you need the device internals. Pairs with the virtio transport and rate-limiting deep dives.

What you must already understand

• Where the device configs and builders live (paths drift after the crate merge — rg/ls):

ls src/vmm/src/vmm_config/          # boot_source, drive, net, vsock, balloon, mmds, entropy, ...
ls src/vmm/src/devices/virtio/      # block, net, vsock, balloon, rng, ...
rg -n "struct .*DeviceConfig|fn build|fn from_config|set_block_device|set_net_device" \
  src/vmm/src/vmm_config/ src/vmm/src/device_manager/ | head

• The rate limiter as a token bucket — two buckets (bandwidth in bytes/s, ops in operations/s):

rg -n "struct TokenBucket|struct RateLimiter|one_time_burst|refill_time|fn consume" \
  src/vmm/src/rate_limiter/ | head

• MMDS = an IMDS-like metadata service backed by dumbo, a tiny in-VMM TCP/IP stack:

rg -n "MmdsVersion|V1|V2|struct Mmds|MmdsConfig|allow_mmds_requests|ipv4" src/vmm/src/mmds/ | head

• The PATCH semantics: drive path and rate limiters are runtime-PATCHable; most else is pre-boot.

Representative tasks

How to approach one — worked example: a rate-limiter config bug

Illustrative of the pattern. The rg finds the real token-bucket constructor; do not trust paths.

Symptom: an issue reports that a drive configured with a bandwidth rate limiter of size: 1048576, refill_time: 0 is accepted, and then the token bucket divides by refill_time and either panics or effectively disables throttling. A refill_time of 0 is meaningless — the bucket can never refill on a sane schedule.

Step 1 — reproduce and locate

sudo ./firecracker --api-sock /tmp/fc.sock &
# (configure boot-source + a rootfs drive first, then PATCH a bad rate limiter)
curl -s -X PATCH --unix-socket /tmp/fc.sock \
  --data '{"drive_id":"rootfs","rate_limiter":{"bandwidth":{"size":1048576,"refill_time":0}}}' \
  http://localhost/drives/rootfs
# BUG: accepted; the token bucket has refill_time == 0.

rg -n "fn new|refill_time|size|TokenBucket" src/vmm/src/rate_limiter/mod.rs | head
git log --oneline -n 5 -- src/vmm/src/rate_limiter/

Step 2 — comment the plan, then add the check where the bucket is built

Open the discussion first (this stage touches device behaviour). Then validate at construction so the invariant holds everywhere the bucket is built, and return a clear error:

--- a/src/vmm/src/rate_limiter/mod.rs
+++ b/src/vmm/src/rate_limiter/mod.rs
@@  impl TokenBucket {
     pub fn new(size: u64, one_time_burst: u64, refill_time: u64) -> Option<Self> {
-        // (previously: assumed refill_time > 0)
+        // A zero size or zero refill time makes the bucket degenerate; reject it so the
+        // caller surfaces a clear config error instead of mis-throttling or dividing by zero.
+        if size == 0 || refill_time == 0 {
+            return None;
+        }

…and make the config layer turn that None into a good rejection error (Stage 3 skill):

@@  // in the drive/net rate-limiter config building:
-        let bucket = TokenBucket::new(size, burst, refill_time).unwrap();
+        let bucket = TokenBucket::new(size, burst, refill_time)
+            .ok_or(RateLimiterConfigError::InvalidTokenBucket { size, refill_time })?;

Step 3 — test the device behaviour, not just the parse

Unit-test the bucket invariant and an integration test that PATCHes the bad config and asserts the 400:

#![allow(unused)]
fn main() {
#[test]
fn test_token_bucket_rejects_zero_refill() {
    assert!(TokenBucket::new(1024, 0, 0).is_none());
    assert!(TokenBucket::new(0, 0, 100).is_none());
    assert!(TokenBucket::new(1024, 0, 100).is_some());
}
}

cargo test -p vmm rate_limiter
rg -n "def test_.*rate_limit|rate_limiter" tests/integration_tests/ | head
tools/devtool test -- -k rate_limit

For a bandwidth-behaviour bug (rather than a config-shape bug), the integration test should also assert the observed throughput is throttled — Firecracker's performance/functional tests measure this; copy the existing pattern.

A second pattern — MMDS config

Illustrative.

Symptom: PUT /mmds/config accepts an MMDS version string the code does not actually handle, or binds MMDS to a network interface that was never configured, so guest metadata requests silently fail.

rg -n "MmdsVersion|from_str|network_interfaces|ipv4_addr" src/vmm/src/vmm_config/mmds.rs src/vmm/src/mmds/ | head

The fix validates the version against the supported set and that each named interface exists in VmResources, returning a clear error otherwise. Because V1 is deprecated (V2 is the token/session, IMDSv2-like model), check on your branch whether new V1 configs should warn or be rejected — that is a maintainer call worth raising on the issue.

What a good PR looks like

• The fix is at the construction/config seam, so the invariant holds no matter which caller (API, config file, snapshot restore) builds the device.
• A clear rejection error (Stage 3) naming the device and field, pinned by a test.
• The behaviour is tested, not just the parse. For a rate-limiter or I/O-engine change, an integration test asserts the device behaves correctly (throttles, flushes, falls back), not just that a bad config is rejected.
• You did not expand the device's surface. Adding a new config knob or a new device feature is a much higher bar (minimal-device-model philosophy) — a config fix tightens or corrects existing behaviour. If you think a knob is missing, raise it as a discussion, not a surprise PR.
• Pre-boot vs runtime PATCH semantics respected: a runtime PATCH (drive path, rate limiter) is handled by the runtime controller; pre-boot-only fields rejected after boot.
• CHANGELOG entry; gates green; integration test present.

Graduation criteria — ready for Stage 6 when

• You have one merged device-config PR that corrects how a block/net/vsock/balloon/rate-limiter/ MMDS config is validated or applied, with an integration test that exercises the device behaviour.
• You can name, for one device, its config type, its builder, and where the config value changes what the device does at runtime — with an rg for each.
• You can explain the token-bucket rate limiter (two buckets, size/one_time_burst/refill_time) and why a degenerate config must be rejected at construction.
• You can tell a config fix (tightening existing behaviour) from a feature (new surface), and you route the latter through a discussion.

You have worked the device configuration layer. The next two stages go beneath it: Stage 6 into the vCPU/KVM layer every device sits on, and Stage 7 into the virtio device internals themselves.

Next: Stage 6 — vCPU and KVM Issues.