Golden and Differential Tests

Two techniques that scale further than hand-written unit tests: golden tests compare your terminal against its own recorded past, and differential tests compare it against another implementation.

Together they cover the space unit tests cannot — the interactions between features that only real programs produce.

Golden Tests

Feed recorded terminal byte streams into the terminal core and compare the resulting screen state against expected snapshots.

The layout

tests/golden/
├── cases/
│   ├── ls-color.cast              recorded PTY output (asciinema v2)
│   ├── ls-color.snapshot          the expected screen, checked in
│   ├── ls-color.meta.json         { rows, cols, description, source }
│   ├── vim-startup.cast           + .snapshot + .meta.json
│   ├── vim-edit-quit.cast
│   ├── top-two-frames.cast
│   ├── less-scroll.cast
│   ├── htop-mouse-drag.cast
│   ├── git-log-graph.cast
│   ├── python-repl.cast
│   ├── cjk-mixed.cast
│   ├── emoji-zwj.cast
│   ├── man-page.cast
│   ├── ncurses-dialog.cast
│   ├── progress-bar.cast          \r-based, no newlines
│   ├── nested-tmux.cast
│   └── urandom-4k.cast            adversarial: random bytes
└── golden_test.rs

The runner

#![allow(unused)]
fn main() {
#[test]
fn golden_snapshots_match() {
    let mut failures = Vec::new();
    for case in golden_cases() {
        let rec = Recording::parse(&fs::read_to_string(&case.cast).unwrap()).unwrap();
        let mut term = Terminal::new(rec.height as usize, rec.width as usize, cfg());
        rec.replay_output_fast(&mut |b| term.advance(b));
        let got = term.snapshot_text();
        let expected = fs::read_to_string(&case.snapshot).unwrap_or_default();
        if got != expected {
            if std::env::var("UPDATE_GOLDEN").is_ok() {
                fs::write(&case.snapshot, &got).unwrap();
            } else {
                failures.push(format!("--- {} ---\n{}", case.name, unified_diff(&expected, &got)));
            }
        }
    }
    assert!(failures.is_empty(), "{} golden mismatches:\n{}",
            failures.len(), failures.join("\n"));
}

#[test]
fn bytewise_replay_gives_identical_screens() {
    // The parser-correctness property, applied at the screen level over the whole
    // corpus. If chunking changes the screen, the parser is not a real state
    // machine — and the bug will appear in production over SSH, intermittently.
    for case in golden_cases() {
        let rec = Recording::parse(&fs::read_to_string(&case.cast).unwrap()).unwrap();
        let mut fast = Terminal::new(rec.height as usize, rec.width as usize, cfg());
        rec.replay_output_fast(&mut |b| fast.advance(b));
        let mut slow = Terminal::new(rec.height as usize, rec.width as usize, cfg());
        rec.replay_output_bytewise(&mut |b| slow.advance(b));
        assert_eq!(fast.snapshot_debug(), slow.snapshot_debug(),
                   "chunking changed the result in {}", case.name);
    }
}
}

UPDATE_GOLDEN — the discipline

UPDATE_GOLDEN=1 cargo test golden
git diff tests/golden/cases/       # ← READ THIS. Every line.

The environment variable makes accepting an intentional change one command. The discipline is reading the diff. A golden test that is regenerated without inspection is worse than no test — it converts a regression into a committed expectation, and now the wrong behavior is protected.

The rule: an updated golden file requires a commit message explaining why the output changed.

What makes a good golden case

Recording a good case:

# Fixed size, fixed environment, deterministic program.
mini-term run --rows 24 --cols 80 --record tests/golden/cases/vim-startup.cast \
  --timeout 3000 -- vim -u NONE -c 'q' /dev/null

cat > tests/golden/cases/vim-startup.meta.json <<'EOF'
{ "rows": 24, "cols": 80,
  "description": "vim starting up and quitting immediately: exercises the alternate screen, scroll regions, and the status line",
  "command": "vim -u NONE -c q /dev/null",
  "recorded": "2026-01-15" }
EOF

UPDATE_GOLDEN=1 cargo test golden
git add tests/golden/cases/vim-startup.*

Tip: -u NONE for vim, --norc --noprofile for bash, -XF for less. Every interactive program has a "no config" flag, and using it is the difference between a reproducible recording and a recording of your dotfiles.

Snapshot formats: which to golden

Use text for most cases and debug for the handful where color and attributes are the point (ls-color, git-log-graph).

Differential Tests

At an advanced stage, compare selected behavior against an established terminal implementation or parser.

Golden tests tell you you changed. Differential tests tell you you are different from everyone else — which is a different and often more useful signal.

Level 1: Against a reference parser (vte)

[dev-dependencies]
vte = "0.13"     # Alacritty's parser. DEV-dependency only — never a real one.

#![allow(unused)]
fn main() {
/// Feed the same bytes to both parsers and compare the ACTION streams.
/// This isolates parsing from screen semantics: a difference here is
/// unambiguously a parser bug, not a disagreement about what a sequence means.
#[test]
fn parser_agrees_with_vte_on_the_corpus() {
    let mut differences = Vec::new();
    for case in golden_cases() {
        let bytes = recording_bytes(&case);
        let ours = collect_our_actions(&bytes);
        let theirs = collect_vte_actions(&bytes);
        if ours != theirs {
            differences.push(format!("{}: first divergence at {}",
                                     case.name, first_difference(&ours, &theirs)));
        }
    }
    // Some differences are DELIBERATE — 8-bit C1 handling, OSC bounds. Assert
    // against a known-and-justified list rather than demanding zero.
    let unexpected: Vec<_> = differences.iter()
        .filter(|d| !KNOWN_DIVERGENCES.iter().any(|k| d.contains(k)))
        .collect();
    assert!(unexpected.is_empty(), "{unexpected:#?}");
}

/// Every deliberate divergence, with its reason. This list IS the documentation
/// of where you chose to differ, and it should be short.
const KNOWN_DIVERGENCES: &[&str] = &[
    // We deliberately ignore 8-bit C1 controls: in UTF-8 mode 0x80-0x9F are
    // continuation bytes, and honoring them corrupts non-ASCII text.
    "c1-controls",
    // We cap OSC payloads at 4 KB; vte's limit differs.
    "osc-overlong",
];
}

Level 2: Against a real terminal, via CSI 6n

Compare observable behavior by asking the terminal where its cursor is:

#!/usr/bin/env bash
# probe.sh — ask the terminal where the cursor ends up after a sequence.
probe() {
  printf '\033[H'          # home
  printf '%b' "$1"
  printf '\033[6n'
  IFS='[;' read -rsd R -p '' _ row col
  printf '%-40s → row %s col %s\n' "$(printf %q "$1")" "$row" "$col"
}

probe 'hello'
probe '\033[10C'                        # CUF beyond content
probe '\033[999C'                       # CUF clamping
probe "$(printf 'x%.0s' {1..80})"       # exactly cols: PENDING WRAP
probe "$(printf 'x%.0s' {1..81})"       # one past
probe '日本語'                           # wide characters
probe 'e\xcc\x81'                        # combining mark
probe '\t\t\t'                          # tab stops
probe '\033[5;5H\033[2J'                # ED 2 does not move the cursor
probe '\033[?7l'"$(printf 'x%.0s' {1..100})"   # DECAWM off

Run it in xterm, your terminal, tmux, and inside ssh. Any row where they disagree is a finding.

./probe.sh > /tmp/xterm.txt          # in xterm
./probe.sh > /tmp/mine.txt           # in your terminal
diff /tmp/xterm.txt /tmp/mine.txt

Why CSI 6n is the right probe: it is the only way to ask a terminal a question and get a machine-readable answer about its internal state. It turns "does my terminal behave like xterm?" from an eyeball comparison into a diff.

Level 3: Screen-state comparison

The most thorough and the most work: run the same input through your terminal and a reference, and compare full screens.

# Your terminal:
mini-term replay --format json case.cast > /tmp/mine.json

# A reference, via tmux's capture-pane (which dumps its internal screen state):
tmux new-session -d -x 80 -y 24 'cat case.raw'
sleep 1
tmux capture-pane -p -e > /tmp/tmux.txt      # -e keeps escape sequences
tmux kill-session

# Compare, normalizing formats.
mini-term diff --format text /tmp/mine.json /tmp/tmux.txt

Expect differences. tmux has its own opinions, its own width table, and its own bugs. The value is not "zero differences" — it is a documented list of every difference and why.

Fuzzing

cargo fuzz run parse -- -max_total_time=3600
cargo fuzz run terminal -- -max_total_time=3600

#![allow(unused)]
fn main() {
// fuzz/fuzz_targets/terminal.rs
#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: &[u8]| {
    let mut t = terminal_core::Terminal::new(24, 80, Default::default());

    // 1. Never panic, never hang, on any input.
    t.advance(data);

    // 2. Structural invariants survive arbitrary input.
    assert!(t.cursor().row < 24 && t.cursor().col < 80);
    for r in 0..24 { assert_eq!(t.screen().line(r).len(), 80); }
    assert!(t.screen().scrollback().len() <= t.config().scrollback_limit);

    // 3. Memory is bounded: replies and clipboard requests cannot grow without limit.
    assert!(t.take_replies().len() < 1_000_000);

    // 4. THE CHUNKING PROPERTY, fuzzed. This finds real bugs that a fixed corpus
    //    misses, because the fuzzer explores split points you would not choose.
    let mut t2 = terminal_core::Terminal::new(24, 80, Default::default());
    for b in data { t2.advance(std::slice::from_ref(b)); }
    assert_eq!(t.snapshot_debug(), t2.snapshot_debug());
});
}

That fourth assertion is the highest-value line in the fuzz target. It turns the fuzzer loose on exactly the property that breaks in production over SSH.

Commit the corpus. A fuzz corpus is a regression suite that took CPU-hours to build; regenerating it from scratch is wasteful, and the crashes it encodes are the interesting ones.

Building the Corpus

A checklist of what a good corpus covers:

Fifteen to twenty cases covering these is enough to catch most regressions.

Experiment

CLAIM. Differential testing finds bugs that neither unit tests nor golden tests would.

METHOD.

# 1. Run the probe script in three terminals and diff.
./probe.sh > /tmp/xterm.txt        # xterm
./probe.sh > /tmp/ghostty.txt      # a modern terminal
./probe.sh > /tmp/mine.txt         # yours
diff3 /tmp/xterm.txt /tmp/ghostty.txt /tmp/mine.txt

# 2. Run the vte differential over your corpus.
cargo test --test differential -- --nocapture

# 3. For each difference, classify:
#      BUG        — you are wrong; fix it and add a unit test
#      DELIBERATE — you chose differently; add it to KNOWN_DIVERGENCES with a reason
#      REFERENCE  — the reference is wrong (rare, but it happens)

PREDICTION. Before running: how many rows of the probe will differ? Which category do you expect the most differences in?

RESULT. Record the table. Every "BUG" row becomes a unit test; every "DELIBERATE" row becomes a documented divergence. That table is a genuinely useful artifact — publish it.

Test

#![allow(unused)]
fn main() {
#[test]
fn every_golden_case_has_a_snapshot_and_metadata() {
    // Prevents the slow rot of "someone added a .cast and forgot the rest".
    for entry in fs::read_dir("tests/golden/cases").unwrap() {
        let p = entry.unwrap().path();
        if p.extension().and_then(|e| e.to_str()) != Some("cast") { continue; }
        assert!(p.with_extension("snapshot").exists(), "{p:?} has no snapshot");
        assert!(p.with_extension("meta.json").exists(), "{p:?} has no metadata");
    }
}

#[test]
fn golden_snapshots_have_no_trailing_whitespace() {
    // Trailing whitespace makes diffs unreadable and hides real changes.
    for case in golden_cases() {
        let s = fs::read_to_string(&case.snapshot).unwrap();
        for (i, line) in s.lines().enumerate() {
            assert_eq!(line.trim_end(), line, "{}:{}", case.name, i + 1);
        }
    }
}

#[test]
fn adversarial_cases_do_not_panic() {
    for case in golden_cases().filter(|c| c.name.contains("urandom")
                                       || c.name.contains("adversarial")) {
        let mut t = Terminal::new(24, 80, cfg());
        t.advance(&recording_bytes(&case));
        assert!(t.cursor().row < 24);
    }
}

#[test]
fn known_divergences_are_documented_and_few() {
    // A growing divergence list is a smell: either you are drifting from the
    // ecosystem, or you are using it to suppress real bugs.
    assert!(KNOWN_DIVERGENCES.len() <= 5,
            "{} divergences — review them", KNOWN_DIVERGENCES.len());
}
}

Challenge Extensions

1. A bisect helper: given a failing golden case, binary-search the byte stream for the first byte at which your snapshot diverges from the expected one. About 40 lines, and it turns a two-hour debugging session into a two-minute one.
2. Automated corpus recording: a script that re-records every case from its meta.json command, so the corpus can be refreshed when a tool's version changes.
3. Screenshot-based differential against a real terminal using xdotool/screencapture and perceptual image diffing. Fragile, but it catches rendering bugs nothing else does.
4. A differential harness against alacritty_terminal as well as vte, comparing screen state rather than just actions.
5. Publish your probe table for five terminals. This is a genuine contribution to the ecosystem — nobody maintains a current one.
6. Fuzz with a structured generator that emits valid-ish escape sequences rather than random bytes; it reaches deeper into the state machine.

Validation / Self-check

1. What does a golden test catch that a unit test does not?
2. What is the UPDATE_GOLDEN discipline, and what goes wrong without it?
3. Why does every golden case need a meta.json?
4. What does the bytewise-replay test catch, and where does that bug appear in production?
5. Why compare action streams with vte rather than screen state?
6. Why is CSI 6n the right probe for behavioral differential testing?
7. Name the three classifications for a differential finding, and what you do with each.
8. Why is a growing KNOWN_DIVERGENCES list a smell?
9. What are the four assertions in the terminal fuzz target, and which is most valuable?
10. Name eight categories a golden corpus should cover.

Next: Interactive Compatibility.