The microVM Metadata Service (MMDS)

MMDS is a small metadata service that lives inside the VMM and answers HTTP requests from the guest at the link-local address 169.254.169.254 — exactly like the AWS EC2 Instance Metadata Service (IMDS) that cloud users already know. The orchestrator pushes a JSON document into the VMM over the Firecracker REST API; the guest reads it back over its network interface as if it were talking to a real metadata endpoint. No host networking is involved, no real server is listening: Firecracker fabricates the HTTP responses internally using a tiny in-VMM TCP/IP stack.

This chapter covers what MMDS is and where it sits, the V1 (deprecated) vs V2 (token/session) data plane, how it's configured (PUT /mmds for contents, PUT /mmds/config for settings), the dumbo TCP/IP stack that answers from 169.254.169.254, the net-device interception hook that diverts those packets, and why MMDS is deliberately not part of a snapshot.

Note: MMDS is the one place where Firecracker originates network traffic to the guest without a host backend. It is an intentional, audited exception to "the VMM does almost nothing." Because it parses guest-supplied packets, dumbo is part of the attack surface — which is exactly why it is minimal, and why production hosts are told to block guest egress to 169.254.169.254 from leaking out (docs/prod-host-setup.md).

Where MMDS lives

rg -n "mod mmds|struct Mmds|MmdsVersion|MMDS_IPV4_ADDR|169.254.169.254|fn get|fn put" src/vmm/src/mmds/
find src/vmm/src/mmds -name "*.rs"
find src/vmm/src/dumbo -name "*.rs"

The metadata store and HTTP logic are in src/vmm/src/mmds/; the network stack that carries the responses is in src/vmm/src/dumbo/. Both are now modules inside the vmm crate (historically they were separate crates — rg/find to confirm on your branch). The store is a JSON document; the MMDS module exposes get/put operations on it and renders HTTP responses for guest requests.

┌──────────────────────────────────────────────────────────┐
│ guest                                                     │
│   curl http://169.254.169.254/latest/meta-data/...        │
└───────────────┬──────────────────────────────────────────┘
                │ packets to 169.254.169.254 on the guest's NIC
                ▼
┌──────────────────────────────────────────────────────────┐
│ virtio-net device (TX queue)                              │
│   hook: is dst == MMDS address?  ── yes ─► divert         │
└───────────────┬──────────────────────────────────────────┘
                │ diverted frames
                ▼
┌──────────────────────────────────────────────────────────┐
│ dumbo: tiny in-VMM TCP/IP  →  MMDS HTTP handler           │
│   builds the HTTP response from the JSON store            │
└───────────────┬──────────────────────────────────────────┘
                │ synthesized reply frames
                ▼  injected back into the guest's RX queue

The host network (the TAP device) never sees these packets — they are intercepted before they leave the VMM.

V1 vs V2

rg -n "MmdsVersion|V1|V2|token|session|X-metadata-token|imds_compat|fn generate_token" src/vmm/src/mmds/
sed -n '1,80p' docs/mmds/mmds-user-guide.md

V2 mirrors AWS IMDSv2: the guest first does a PUT to obtain a short-lived session token, then must present that token on every metadata GET. This defends against confused-deputy attacks (e.g. an SSRF in a guest workload tricking the workload into fetching credentials), the same reason AWS pushed IMDSv2. You select the version with PUT /mmds/config.

Configuring MMDS

rg -n "mmds-config|MmdsConfig|version|network_interfaces|ipv4_address|imds_compat|put_config" src/vmm/src/

There are two distinct configuration endpoints:

A typical setup over the API socket:

API=/tmp/fc.sock
# 1. Decide which interface(s) can reach MMDS and which version.
curl -X PUT --unix-socket $API --data \
 '{"version":"V2","network_interfaces":["net1"],"ipv4_address":"169.254.169.254"}' \
 http://localhost/mmds/config
# 2. Push the metadata document the guest will see.
curl -X PUT --unix-socket $API --data \
 '{"latest":{"meta-data":{"instance-id":"i-abc","local-hostname":"fc"}}}' \
 http://localhost/mmds

/mmds/config is a pre-boot configuration (set it before InstanceStart); /mmds contents can be updated while the VM runs, which is how an orchestrator pushes fresh metadata to a live guest.

dumbo: the tiny in-VMM TCP/IP stack

rg -n "struct .*Tcp|Connection|fn rx|fn tx|parse|Ipv4|EthernetFrame|checksum|arp|window" src/vmm/src/dumbo/

dumbo is a deliberately minimal TCP/IP implementation — just enough to terminate the handful of short HTTP request/response exchanges MMDS needs and nothing more. It parses Ethernet frames, IPv4 packets, and a constrained subset of TCP (enough for a single short connection per request), handles ARP for the MMDS address, and hands the HTTP bytes to the MMDS handler. It is not a general network stack: it does not route, does not do the host network, and exists only to answer 169.254.169.254. That minimalism is a security property — less parser, less attack surface.

flowchart TD
    Frame["guest TX frame to 169.254.169.254"] --> Eth["dumbo: parse Ethernet"]
    Eth --> Arp{"ARP?"}
    Arp -->|yes| ArpReply["synthesize ARP reply"]
    Arp -->|no| Ip["parse IPv4"]
    Ip --> Tcp["minimal TCP state machine"]
    Tcp --> Http["MMDS HTTP handler"]
    Http --> Store["read JSON store / check V2 token"]
    Store --> Resp["build HTTP response"]
    Resp --> Inject["inject reply frame into guest RX queue"]

The net-device interception hook

rg -n "mmds|ns\.|MmdsNetworkStack|detour|intercept|is_mmds|process_tx|process_rx" src/vmm/src/devices/virtio/net/

MMDS does not get its own virtio device. Instead, the virtio-net device's TX path is hooked: when the guest transmits a frame, the net device checks whether the destination is the MMDS address (and whether this interface is allowed to reach MMDS per /mmds/config). If so, the frame is diverted to dumbo/MMDS instead of being written to the host TAP, and any reply is injected back into the net device's RX queue. Find this in the net device's TX processing (rg -n "mmds" src/vmm/src/devices/virtio/net/). This is why MMDS is reachable only over a virtio-net interface that was explicitly granted access — the hook lives in that specific device's data path. The net device itself is covered in virtio-net-and-tap.md.

Why MMDS is not snapshotted

rg -n "Persist|mmds|snapshot" src/vmm/src/mmds/ || echo "MMDS state not part of the Persist graph — confirm on your branch"

The MMDS contents are configuration the orchestrator owns, not intrinsic VM state. On restore, the orchestrator re-pushes the metadata document appropriate to the new instance (a clone restored on a different host should generally get different metadata — a different instance id, hostname, etc.). Baking the old metadata into the snapshot would be wrong. So MMDS contents are deliberately outside the Persist graph: a restored microVM starts with an empty (or freshly-pushed) MMDS. This is a clean illustration of the line between "VM state" (snapshotted) and "orchestrator policy" (re-applied on restore).

Reading exercise

# 1. The MMDS module and the metadata store.
find src/vmm/src/mmds -name "*.rs"
rg -n "struct Mmds|MmdsVersion|fn get|fn put|169.254.169.254" src/vmm/src/mmds/

# 2. V1 vs V2 token handling.
rg -n "V1|V2|token|session|X-metadata-token|generate_token" src/vmm/src/mmds/

# 3. The dumbo stack.
find src/vmm/src/dumbo -name "*.rs"
rg -n "Tcp|Ipv4|EthernetFrame|arp|checksum" src/vmm/src/dumbo/

# 4. The interception hook in the net device.
rg -n "mmds|is_mmds|MmdsNetworkStack" src/vmm/src/devices/virtio/net/

# 5. Configure and read MMDS by hand following the API blocks above (use a guest with curl).

# 6. The user guide and config docs.
ls docs/mmds/

Answer:

1. Where does MMDS run, what address does the guest use, and what's listening there?
2. Contrast V1 and V2. What attack does the V2 token model defend against, and what real-world IMDS feature is it modeled on?
3. What is the difference between PUT /mmds and PUT /mmds/config? Which can you call while the VM is running?
4. What is dumbo and why is it deliberately minimal? Name three protocol layers it must handle.
5. Explain the net-device interception hook: how does a frame to 169.254.169.254 reach MMDS instead of the host TAP?
6. Why is MMDS content not part of a snapshot, and what is the orchestrator expected to do on restore?

Common bugs and symptoms

Validation: prove you understand this

1. Draw the path of a guest request to 169.254.169.254 from the guest NIC to the synthesized reply, naming the net hook, dumbo, and the MMDS store.
2. Explain V1 vs V2 and exactly what extra step a V2 guest must perform.
3. Distinguish /mmds from /mmds/config and say which is pre-boot vs runtime.
4. Explain why dumbo is minimal and why that minimalism is a security property, not a limitation.
5. Describe the net-device interception hook and why MMDS is only reachable over an explicitly allowed interface.
6. Justify the decision to exclude MMDS contents from snapshots, using the "VM state vs orchestrator policy" distinction.

Next: the-event-manager.md — the epoll loop on the VMM thread that drives the net device (and therefore the MMDS hook), every other device, and the API eventfd.