Lab 2: Rate limiting

Background

A microVM that can saturate its NIC and disk at full host speed is a weapon against its neighbors. On a host packing thousands of mutually-untrusting microVMs, one tenant that floods the network or hammers the disk steals bandwidth and IOPS from everyone else — the noisy-neighbor problem. Firecracker's answer is a rate limiter: a pair of token buckets wired onto the virtio block and net device fast paths that bound both operations per second and bytes per second, independently, per device. It does this without ever blocking the VMM thread — an exhausted bucket pauses virtqueue processing and arms a timer fd, then resumes when tokens refill.

This lab makes the limiter concrete and measurable. You will configure token buckets on a network interface and on a drive, saturate them with iperf3 and fio, and watch the measured throughput land at the configured cap. You will change the limits at runtime with PATCH — no reboot — exactly as an orchestrator re-throttling a misbehaving tenant would. And you will read the rate_limiter implementation so the numbers you measured are backed by the math in the source. The motivation is multi-tenant fairness; the mechanism is a token bucket; this lab connects the two.

Why This Matters for Contributors

Rate limiting sits directly on the device fast path, so a bug here is a performance bug or a fairness bug for every microVM. Reviewers read consume() line by line because a subtle error in how last_update advances silently changes the effective rate — the device throttles to the wrong number and nobody notices until a tenant complains. A contributor who has measured a configured cap, seen it hold, then read the lazy-refill math and predicted the measured number is exactly who the maintainers trust with this code. This lab is also the fairness half of the oversubscription story: density only works if no single tenant can starve the host, and the rate limiter is one of the mechanisms that guarantees it.

Prerequisites

You completed Lab 1: a microVM with working host networking (a TAP with internet access, or at least host reachability), so iperf3 between guest and host runs.
A guest rootfs with iperf3 and fio installed, and iperf3 on the host.
Read The Rate Limiter and Token Bucket — you need the two-bucket model and the size/refill_time/one_time_burst parameters fresh.

# Host and guest both have iperf3; guest has fio.
iperf3 --version
# Inside the guest (later): iperf3 --version; fio --version
rg -n "struct TokenBucket|struct RateLimiter|fn consume" src/vmm/src/rate_limiter/   # the code exists

Step-by-Step Tasks

Step 1 — Establish an unlimited baseline

You cannot see a cap without knowing the uncapped number. Boot a microVM with no rate limiter on the net interface, run iperf3 host↔guest, and record the line-rate throughput. This is your control.

API=/tmp/fc.sock
rm -f "$API"; sudo ./firecracker --api-sock "$API" &

curl -X PUT --unix-socket "$API" --data \
 '{"kernel_image_path":"./vmlinux-6.1.x","boot_args":"console=ttyS0 reboot=k panic=1 pci=off"}' \
 http://localhost/boot-source
curl -X PUT --unix-socket "$API" --data \
 '{"drive_id":"rootfs","path_on_host":"./ubuntu-24.04.ext4","is_root_device":true,"is_read_only":false}' \
 http://localhost/drives/rootfs
# NO rate_limiter on this interface — the baseline.
curl -X PUT --unix-socket "$API" --data \
 '{"iface_id":"net1","guest_mac":"06:00:AC:10:00:02","host_dev_name":"tap0"}' \
 http://localhost/network-interfaces/net1
curl -X PUT --unix-socket "$API" --data '{"vcpu_count":2,"mem_size_mib":1024}' \
 http://localhost/machine-config
curl -X PUT --unix-socket "$API" --data '{"action_type":"InstanceStart"}' http://localhost/actions

Configure the guest's network (from Lab 1), then measure:

# Host: start an iperf3 server on the host side of the TAP.
iperf3 -s -B 172.16.0.1 &
# --- INSIDE the guest --- send to the host; record the uncapped Mbits/sec.
iperf3 -c 172.16.0.1 -t 10
# e.g. ~10-30 Gbit/s over the TAP — note YOUR number; it's the ceiling.

Step 2 — Cap the network interface with a bandwidth bucket

Now reboot with an rx_rate_limiter/tx_rate_limiter on the interface. The JSON shape is the bucket parameters: size is the bucket capacity (and the burst), refill_time is the milliseconds to refill size tokens, so the rate is size / refill_time. For bytes, size is in bytes; for ops, size counts operations.

# tx_rate_limiter: cap the GUEST→HOST direction at ~50 MiB/s = 52428800 bytes / 1000 ms.
curl -X PUT --unix-socket "$API" --data '{
  "iface_id":"net1","guest_mac":"06:00:AC:10:00:02","host_dev_name":"tap0",
  "tx_rate_limiter": { "bandwidth": { "size": 52428800, "refill_time": 1000 } }
}' http://localhost/network-interfaces/net1

JSON	Meaning	Resulting rate
`bandwidth.size = 52428800`	50 MiB of byte-tokens, the burst	—
`bandwidth.refill_time = 1000`	refill those 50 MiB over 1000 ms	50 MiB/s ≈ 419 Mbit/s

Re-measure with iperf3. The throughput should now plateau near 419 Mbit/s regardless of how much the guest tries to push — the bandwidth bucket drains, processing pauses, the timer refills it, and the sustained rate equals size/refill_time.

# --- INSIDE the guest --- expect ~50 MiB/s now, not the Step-1 ceiling.
iperf3 -c 172.16.0.1 -t 15

Tip: iperf3 shows a bursty start then a flat plateau. The initial burst is the bucket draining its full size at once (that's what size is — the instantaneous burst); the plateau is the refill rate. If you add "one_time_burst", the very first second can exceed the plateau by that many extra tokens — a one-shot allowance, spent once, never refilled. Watch for exactly that shape.

Step 3 — Add the ops bucket and see which bucket binds

A limiter has two buckets — ops and bandwidth — and an operation must satisfy both. A workload of many tiny packets is bounded by the ops bucket; a workload of few large transfers by the bandwidth bucket. Add a tight ops cap and use small-packet traffic to make the ops bucket the binding constraint.

curl -X PUT --unix-socket "$API" --data '{
  "iface_id":"net1","guest_mac":"06:00:AC:10:00:02","host_dev_name":"tap0",
  "tx_rate_limiter": {
    "ops":       { "size": 2000,     "refill_time": 1000 },
    "bandwidth": { "size": 52428800, "refill_time": 1000 }
  }
}' http://localhost/network-interfaces/net1

# --- INSIDE the guest --- many small UDP datagrams stress ops, not bytes:
iperf3 -c 172.16.0.1 -u -l 64 -b 0 -t 10    # 64-byte datagrams, unlimited offered load
# Now the cap is ~2000 packets/s (ops), reached long before 50 MiB/s of bytes.

This is the whole reason for two buckets: you can independently cap "how many I/Os per second" and "how many bytes per second," and a tenant cannot evade an IOPS cap by sending huge packets nor a bandwidth cap by sending many tiny ones.

Step 4 — Rate-limit a drive and measure with fio

Block devices carry the same two-bucket limiter (in the drive's rate_limiter). Attach a second non-root drive with a cap and benchmark it with fio inside the guest.

# Pre-boot: a second drive backed by a host file, capped at ~1000 IOPS and ~10 MiB/s.
truncate -s 1G /tmp/scratch.ext4 && mkfs.ext4 -q /tmp/scratch.ext4
curl -X PUT --unix-socket "$API" --data '{
  "drive_id":"scratch","path_on_host":"/tmp/scratch.ext4","is_root_device":false,"is_read_only":false,
  "rate_limiter": {
    "ops":       { "size": 1000,     "refill_time": 1000 },
    "bandwidth": { "size": 10485760, "refill_time": 1000 }
  }
}' http://localhost/drives/scratch

# --- INSIDE the guest --- /dev/vdb is the scratch drive (vda is rootfs).
# 4k random reads stress IOPS -> bounded by the ops bucket (~1000 IOPS):
fio --name=iops --filename=/dev/vdb --rw=randread --bs=4k --iodepth=32 \
    --runtime=20 --time_based --direct=1
# Large sequential reads stress bytes -> bounded by the bandwidth bucket (~10 MiB/s):
fio --name=bw --filename=/dev/vdb --rw=read --bs=1M --iodepth=8 \
    --runtime=20 --time_based --direct=1

The first fio should report IOPS plateauing near 1000; the second, bandwidth near 10 MiB/s. Same device, two different binding buckets, chosen by the workload — exactly mirroring the net case.

Step 5 — PATCH the limits at runtime

The orchestrator's real use case: re-throttle a running microVM without rebooting. PATCH the same endpoints with a new rate_limiter/rx_rate_limiter/tx_rate_limiter. The change takes effect on the live limiter immediately.

# Loosen the drive cap to 5000 IOPS / 50 MiB/s while fio is running in the guest.
curl -X PATCH --unix-socket "$API" --data '{
  "drive_id":"scratch",
  "rate_limiter": {
    "ops":       { "size": 5000,     "refill_time": 1000 },
    "bandwidth": { "size": 52428800, "refill_time": 1000 }
  }
}' http://localhost/drives/scratch

# Tighten the net cap to 10 MiB/s live:
curl -X PATCH --unix-socket "$API" --data '{
  "iface_id":"net1",
  "tx_rate_limiter": { "bandwidth": { "size": 10485760, "refill_time": 1000 } }
}' http://localhost/network-interfaces/net1

Watch the running fio/iperf3 numbers step to the new cap within a second. The fact that PATCH mutates the live limiter (not a copy) is itself a thing to verify in the code:

rg -n "PATCH|update_rate_limiter|UpdateNetworkInterface|UpdateBlockDevice|RateLimiterUpdate" \
  src/vmm/src/rpc_interface.rs src/vmm/src/devices/virtio/

Step 6 — Read the implementation; make the math match

Now connect numbers to code. Read consume() and the lazy refill, and confirm your measured plateau equals size/refill_time.

# 1. The token bucket and its lazy refill — the heart of the math.
rg -n "struct TokenBucket|fn consume|fn auto_replenish|last_update|refill_time|one_time_burst|budget" \
  src/vmm/src/rate_limiter/

# 2. The RateLimiter wrapping two buckets + a TimerFd.
rg -n "struct RateLimiter|TokenType::Ops|TokenType::Bytes|TimerFd" src/vmm/src/rate_limiter/

# 3. The device side: where block/net call consume on the hot path, and the blocked state.
rg -n "consume|is_blocked|rate_limited|process_rate_limiter|RATE_LIMITER" \
  src/vmm/src/devices/virtio/block/ src/vmm/src/devices/virtio/net/

# 4. The timer-fd registration in activate — what lets a throttled device RESUME.
rg -n "fn activate|TimerFd|register" src/vmm/src/devices/virtio/

# 5. The unit tests pin the exact math — read them to confirm your model.
rg -n "#\[test\]|fn test_.*bucket|consume|replenish" src/vmm/src/rate_limiter/

The lazy-refill formula you should find (read the real consume, do not trust this sketch): on each consume, the bucket credits elapsed_time * size / refill_time new tokens (capped at size), then checks whether the request fits. There is no background refill thread — that is why an idle limiter costs nothing and a denied request arms a timer to fire exactly when the deficit is covered. Trace that timer fd from activate (registration) to the process_rate_limiter-style handler (resume) and you have the complete non-blocking pause/resume cycle that the deep dive describes.

sequenceDiagram
    participant D as device (process_queue)
    participant RL as RateLimiter (ops + bw buckets)
    participant T as TimerFd
    participant EM as VMM thread (EventManager)
    D->>RL: consume(ops=1, bytes=N)
    alt tokens in BOTH buckets
        RL-->>D: Ok — do the I/O, add_used, IRQ
    else either bucket empty
        RL-->>D: Blocked(deficit)
        D->>T: arm timer for the deficit
        D->>EM: return — thread free for other devices
        T-->>EM: timer fires (eventfd readable)
        EM->>D: handler → replenish, resume process_queue
    end

Implementation Requirements / Deliverables

An uncapped iperf3 baseline recorded (the ceiling).
A net tx_rate_limiter bandwidth cap configured; iperf3 shows the plateau near size/refill_time.
An ops cap added; small-packet traffic shows the ops bucket binding before the bandwidth one.
A drive rate_limiter configured; fio shows IOPS bounded by the ops bucket and throughput bounded by the bandwidth bucket on the same device.
A runtime PATCH that steps a live cap up and down, observed in the running benchmark.
A written derivation: your measured plateau computed from size/refill_time, matching the code's consume math, citing the file you read it in.

Troubleshooting

Measured rate is far above the configured cap

Either the limiter isn't on the path you're testing, or you're reading size as the rate. Check that the PUT/PATCH actually applied (re-GET the interface/drive), that you capped the direction you're measuring (tx_ is guest→host), and that you computed size/refill_time — {"size":52428800, "refill_time":1000} is 50 MiB/s, not 52 MB total.

Device throttles, then never resumes (traffic stops dead)

The classic timer-fd-registration bug. The bucket emptied, processing paused, but the timer fd was never registered with the EventManager in activate, so the "tokens refilled" wake-up never fires. rg -n "fn activate|TimerFd|register" src/vmm/src/devices/virtio/ and confirm the limiter's timer is registered. This is the first place to look for "throttles correctly but hangs."

`fio` numbers don't budge with the cap

fio on a buffered file inside the guest can be served from the guest's page cache, bypassing the block device entirely. Use --direct=1 (as above) so I/O actually hits /dev/vdb and the limiter. Also confirm you targeted the capped drive (/dev/vdb), not the uncapped rootfs (/dev/vda).

`iperf3` UDP shows huge loss instead of a clean cap

The ops/bandwidth bucket pauses processing, it does not drop — but UDP at the guest end can still overrun its own socket buffers and report loss before the limiter even sees the packets. That's a guest artifact; use TCP (iperf3 -c ... -t 15) for a clean bandwidth-cap reading, and reserve UDP small packets for demonstrating which bucket binds, not for precise numbers.

One_time_burst seems to apply every second

one_time_burst is a one-shot reserve, spent once at startup, not folded into the refillable pool. If your burst appears to repeat, you're seeing the normal per-second burst of spending the full size each refill window — that's the bucket capacity, not the one-time burst. Re-read how the bucket constructor tracks the two separately.

Expected Output

# Net, uncapped baseline (Step 1):
[  5]   0.00-10.00  sec  18.6 GBytes  16.0 Gbits/sec   sender

# Net, capped at ~50 MiB/s = ~419 Mbit/s (Step 2):
[  5]   0.00-15.00  sec   750 MBytes   419 Mbits/sec   sender   <-- plateau at size/refill_time

# Drive, fio IOPS bounded by ops bucket (~1000 IOPS) (Step 4):
read: IOPS=1002, BW=3.9MiB/s (4.1MB/s)

# Drive, fio bandwidth bounded by bandwidth bucket (~10 MiB/s) (Step 4):
read: IOPS=10, BW=10.0MiB/s (10.5MB/s)

Stretch Goals

Predict, then measure. Pick size/refill_time for a 7 MiB/s cap, compute the expected iperf3 Mbit/s, then measure and explain any gap (TCP overhead, the virtio_net_hdr, ACK traffic).
Burst characterization. Add a large one_time_burst and capture the shape of iperf3 -i 1 per-second output — the first interval should exceed the plateau by exactly the burst, then never again.
Two tenants, one host. Boot two microVMs on two TAPs, cap one and not the other, run iperf3 from both simultaneously, and show the capped tenant cannot starve the uncapped one of host bandwidth — the fairness argument, demonstrated.
Read the math, find the trap. In consume, find exactly how last_update advances. Construct (on paper) a sequence of consume calls where advancing it incorrectly would over- or under-credit tokens, and confirm the real code does not have that bug. This is what a reviewer does.
A regression test idea. Look at the rate-limiter unit tests and sketch one that would catch a future change accidentally changing the effective rate by 10%.

Validation / Self-check

Answer without notes; these gate completion:

Write the formula for the steady-state rate in terms of size and refill_time. What does size alone control? What does one_time_burst add, and how often?
Why are there two buckets per limiter? Give a network workload bounded by each, and explain why an operation must satisfy both.
Walk the full pause→resume cycle for an exhausted bucket: what stops, what is armed, which thread is not blocked, and what wakes the device.
Why does the limiter refill lazily on consume instead of running a background thread? What does that buy you, and what is the single bug class a reviewer watches for in that math?
You PATCH a new cap on a running microVM and nothing changes. Name two distinct causes and the command that distinguishes them.
A PR proposes blocking the VMM thread until tokens refill "to simplify the code." Give the correctness and isolation argument for rejecting it.

Next: Lab 3: MMDS — the one piece of guest traffic that never reaches the TAP at all: configure the metadata service, fetch it through the IMDSv2 token flow, and trace how the net device detours 169.254.169.254 into the in-VMM dumbo stack.

Open-Source Engineer & Contributor