Lab 25: The CLI and REPL (Milestone 14)
Background
You will finish src/bin/ember.rs: seven subcommands, the global flags, and a REPL that cannot be
crashed by its user.
The subcommands already exist in pieces — you built each one as its representation appeared. This lab makes them a coherent tool and adds the interactive front end.
Why This Lab Matters
- The REPL is the most hostile input source you have: a human, typing, interactively, with no obligation to type anything valid.
.interpmakes ADR-003 tangible. Typing an expression and watching two backends agree is a better demonstration than any explanation.- Exit codes are an interface. A CI pipeline running policies needs "the policy is wrong" and "the runtime is broken" to be different.
Prerequisites
- Lab 24 complete: the renderer works.
- The CLI and the REPL read.
Predict First
- Type
function f()at the REPL and press enter. What should happen? - Type
1 + 1. What should print, and what does the implementation have to try first? - Type
while true do end. What should happen, and how long should it take? - Ctrl-C during that loop. Ctrl-C at an empty prompt. Ctrl-D. Three different behaviors — which?
- A registered host function panics inside a REPL evaluation. Is the session over?
ember run script.emberwhere the script hits its instruction budget. What is$??
Step 1: The Subcommand Table
#![allow(unused)] fn main() { enum Cmd { Run { path: Option<PathBuf>, eval: Option<String>, interp: bool }, Repl, Tokens { src: Source }, Ast { src: Source }, Disassemble { src: Source }, Trace { src: Source }, Compile { src: Source, out: PathBuf }, } }
Every subcommand accepts a path, -e EXPR, or - for stdin. That uniformity costs one helper and
makes the tool scriptable:
$ echo 'return 1 + 1' | ember run -
$ ember tokens -e 'local x = 1'
$ ember ast script.ember
Step 2: The Flags
#![allow(unused)] fn main() { struct GlobalFlags { no_color: bool, // also honors NO_COLOR stats: bool, max_instructions: Option<u64>, max_memory: Option<usize>, max_depth: Option<usize>, gc_stress: bool, trace: TraceFlags, // gc, tables, upvalues, modules, host, ic } }
Every trace flag writes to stderr. Add the test:
#![allow(unused)] fn main() { #[test] fn no_trace_flag_writes_to_stdout() { for flag in ["--trace-gc", "--trace-tables", "--trace-upvalues", "--trace-modules", "--trace-host"] { let out = run_cli(&[flag, "-e", "return 1"]); assert_eq!(out.stdout.trim(), "1", "{flag} polluted stdout"); } } }
That test exists because a --trace-gc that wrote to stdout once broke an entire golden suite, and
the failure looked like "the GC is producing wrong values".
Step 3: The REPL Loop
#![allow(unused)] fn main() { fn repl(mut engine: Engine) -> ExitCode { let mut editor = LineEditor::new(); // history, arrow keys — a small dependency or hand-rolled let mut pending = String::new(); let mut chunk_no = 0; loop { let prompt = if pending.is_empty() { "> " } else { ">> " }; let line = match editor.readline(prompt) { Ok(l) => l, // Ctrl-C: cancel the LINE (and any pending multi-line input), keep the session. Err(Interrupted) => { pending.clear(); continue } Err(Eof) => break, // Ctrl-D: exit Err(e) => { eprintln!("input error: {e}"); break } }; if let Some(cmd) = line.strip_prefix('.') { dot_command(&mut engine, cmd); continue } pending.push_str(&line); pending.push('\n'); chunk_no += 1; // Limits RESET per line: an accidental `while true do end` returns the // prompt, not the process. State (globals, functions) persists. engine.limits().reset_per_line(); match engine.eval_repl(&pending, &format!("<repl:{chunk_no}>")) { Ok(values) => { print_values(&values); pending.clear() } // INCOMPLETE input continues rather than erroring. One predicate, // a large experience difference. Err(e) if e.is_incomplete() => continue, Err(e) => { eprint!("{}", render(&e, engine.sources())); pending.clear() } } } ExitCode::SUCCESS } }
Five properties, each a test:
#![allow(unused)] fn main() { #[test] fn incomplete_input_continues() { /* "function f()" → ">> " */ } #[test] fn a_bare_expression_prints_its_value() { /* "1 + 1" → "2" */ } #[test] fn errors_do_not_end_the_session() { /* syntax, runtime, limit, host panic */ } #[test] fn each_line_is_its_own_source_file() { /* tracebacks name <repl:3> */ } #[test] fn limits_reset_per_line_but_state_persists() { } }
Step 4: Incomplete-Input Detection
One predicate on the error:
#![allow(unused)] fn main() { impl EmberError { /// True when the parse failed because input RAN OUT, rather than because a /// token was wrong. `function f()` is incomplete; `function f(] ` is wrong. pub fn is_incomplete(&self) -> bool { self.kind == ErrorKind::Parse && self.at_eof } } }
The at_eof flag is set by expect when the token it found was Eof. Two lines, and it is the
difference between a REPL people use and one they tolerate.
Note: Lua's own REPL does exactly this, and its trick is worth knowing: it retries the line prefixed with
returnto make bare expressions print, and treats a specific<eof>-mentioning error as "keep reading". Ember's version is cleaner because the flag is structured rather than matched from message text — which is the same "match onkind, not on the message" rule from the diagnostics chapter.
Step 5: Bare Expressions
#![allow(unused)] fn main() { fn eval_repl(&mut self, src: &str, name: &str) -> Result<Vec<Value>> { // Parse first, classify second — the same move as `expr_stat` in Lab 5. // Try it as an EXPRESSION so `1 + 1` prints 2; fall back to a statement. match self.compile_expression(src, name) { Ok(chunk) => self.run_chunk(chunk), Err(e) if e.is_incomplete() => Err(e), Err(_) => { let chunk = self.compile_chunk(src, name)?; self.run_chunk(chunk) } } } }
Order matters: try expression first, because f() is both a valid expression and a valid
statement, and as an expression it prints its result.
Step 6: Dot Commands
.help .exit
.tokens EXPR .ast EXPR .disasm EXPR .trace EXPR
.stats .gc .heap .globals
.interp EXPR — evaluate under BOTH backends and report agreement
.limits [k=v ...] — inspect or set budgets for this session
.interp is the one to build carefully:
#![allow(unused)] fn main() { fn dot_interp(engine: &mut Engine, expr: &str) { let vm = engine.eval_with(Backend::Vm, expr); let interp = engine.eval_with(Backend::Tree, expr); match (&vm, &interp) { (Ok(a), Ok(b)) if a == b => println!("{a} (both backends agree)"), (Ok(a), Ok(b)) => println!("DIVERGENCE\n vm: {a}\n interp: {b}"), (Err(a), Err(b)) if a.kind == b.kind => println!("both failed: {}", a.kind), (a, b) => println!("DIVERGENCE\n vm: {a:?}\n interp: {b:?}"), } } }
Step 7: Exit Codes
#![allow(unused)] fn main() { match result { Ok(_) => ExitCode::SUCCESS, // 0 Err(e) if e.kind == ErrorKind::Internal => ExitCode::from(70), // a bug in EMBER Err(_) => ExitCode::FAILURE, // 1: the script is wrong } // 2 for usage errors, emitted by the argument parser. }
70 versus 1 is operationally load-bearing. A pipeline running policies wants "the policy is wrong" (1, tell the policy author) and "the runtime is broken" (70, page the service owner) to be different signals.
The Trace
$ ember repl
Ember 0.1.0 — .help for commands, Ctrl-D to exit
> 1 + 1
2
> local x = 10
> x * 2
20
> function counter()
>> local n = 0
>> return function() n = n + 1 return n end
>> end
> local c = counter()
> c() c() c()
3
> .disasm c()
== <repl:8> ==
0000 1 GET_GLOBAL 0 ; c
0002 | CALL 0 2
0004 | RETURN 2
> .interp 2 ^ 3 ^ 2
512.0 (both backends agree)
> while true do end
error: instruction budget exhausted
> x
10
> nosuchthing.field
<repl:11>:1:1: error: attempt to index a nil value (global 'nosuchthing')
help: did you mean `nothing`?
> .stats
instructions: 1,204 / 10,000,000 allocations: 41 live: 28 gc runs: 0
> ^D
Read what survived. The infinite loop hit its budget and returned the prompt; x was still 10
afterwards. The error did not end the session. The closure created across four lines works, which
means each line's chunk shares the same globals and the same heap. And .interp confirmed the two
backends agree on right-associativity, interactively.
Now the hostile-input check, which is the actual deliverable:
$ printf '%s\n' \
'local' \
'((((((((((' \
'"unterminated' \
'return \xff\xfe\x00' \
'while true do end' \
'local function f() return f() end f()' \
'("x"):rep(1e12)' \
| ember repl 2>&1 | tail -3
error: memory budget exhausted
>
$ echo $?
0
Exit code 0: every one of those was handled, and the session ended normally at EOF. If any of them ends the session or aborts, the REPL is not done.
Expected Output
$ ember run --max-instructions 1000 -e 'while true do end'; echo $?
error: instruction budget exhausted
1
$ ember run -e 'return 1'; echo $?
1
0
$ ember badcommand; echo $?
error: unknown subcommand 'badcommand'
2
$ cargo test --test cli
test no_trace_flag_writes_to_stdout ... ok
test incomplete_input_continues ... ok
test errors_do_not_end_the_session ... ok
test each_repl_line_is_its_own_source_file ... ok
test exit_codes ... ok
Debugging Steps
function f() reports a syntax error instead of prompting
is_incomplete is missing, or expect does not set at_eof.
1 + 1 prints nothing
eval_repl tries the statement path first. Try expression first.
A traceback in the REPL says line 1 for everything
Every line is being registered under the same SourceId. Each needs its own <repl:N>.
Ctrl-C exits the session
The Interrupted arm is falling through to break. Ctrl-C cancels the line; Ctrl-D exits.
The REPL leaks memory over a long session
The SourceMap grows by one SourceFile per line, forever. Bound it and degrade gracefully to
"location unavailable" for evicted chunks — or accept it and document the bound.
A budget-exhausted line leaves the session unusable
The budget was not reset, or the VM's stack was not unwound after the error. Both must be true for the next line to work.
--trace-gc broke the golden tests
Trace output on stdout. The Step 2 test exists for this.
Experiment
CLAIM. The REPL cannot be crashed by any input.
METHOD. A fuzz harness that drives the REPL as if typing: feed it lines from your fuzz corpus, plus deliberately incomplete input, plus control characters, plus 1 MB single lines. Assert the process exits 0 at EOF.
cargo fuzz run repl -- -max_total_time=600
PREDICTION. What is the first thing that crashes it? (Most people's answer is the multi-line buffer growing without bound.)
RESULT. Record it. Then add a pending size cap and re-run. A REPL fuzz target is the cheapest
high-yield fuzzing in the project, because the REPL exercises the lexer, parser, compiler, VM, and
renderer in one loop with a stateful engine.
Test
#![allow(unused)] fn main() { #[test] fn errors_do_not_end_the_session() { let out = drive_repl(&["local", "1 +", "\"unterminated", "nosuch.field", "while true do end", "1 + 1"]); assert!(out.stdout.contains('2'), "the session died before the last line"); assert_eq!(out.exit_code, 0); } #[test] fn state_persists_but_limits_reset() { let out = drive_repl(&["x = 10", "while true do end", "return x"]); assert!(out.stdout.contains("10"), "state did not survive a budget error"); } #[test] fn each_repl_line_is_its_own_source_file() { let out = drive_repl(&["function f() error('x') end", "f()"]); assert!(out.stderr.contains("<repl:1>"), "traceback lost the defining chunk"); assert!(out.stderr.contains("<repl:2>"), "traceback lost the calling chunk"); } #[test] fn a_host_panic_does_not_end_the_session() { let out = drive_repl_with_panicking_host(&["boom()", "1 + 1"]); assert!(out.stdout.contains('2')); } #[test] fn exit_codes() { assert_eq!(run_cli(&["run", "-e", "return 1"]).exit_code, 0); assert_eq!(run_cli(&["run", "-e", "return nil + 1"]).exit_code, 1); assert_eq!(run_cli(&["run", "--max-instructions", "10", "-e", "while true do end"]).exit_code, 1); assert_eq!(run_cli(&["nonsense"]).exit_code, 2); } #[test] fn every_subcommand_accepts_a_path_an_expr_and_stdin() { for cmd in ["run", "tokens", "ast", "disassemble", "trace"] { assert!(run_cli(&[cmd, "-e", "return 1"]).exit_code < 2, "{cmd} -e"); assert!(run_cli_stdin(&[cmd, "-"], "return 1").exit_code < 2, "{cmd} -"); } } }
Challenge Extensions
- Completion. Tab-complete globals, table fields after a
., and dot commands. The globals table is enumerable, so this is cheaper than it sounds. .edit— open$EDITORwith the pending buffer, for multi-line functions.- A pager for long output.
.disasmon a large chunk is hundreds of lines. - Session save/restore.
.save session.emberwriting every successfully-evaluated line, so a REPL exploration becomes a script. - An LSP server.
ember lspspeaking enough of the protocol for diagnostics and hover. The diagnostics are already structured; this is mostly plumbing, and it is the natural home for Lab 24's machine-applicable fixes.
Deliverables
-
Seven subcommands, each accepting a path,
-e, or stdin. - All global flags; every trace flag writes to stderr, with the test.
- A REPL with history, multi-line continuation, bare-expression printing, and per-line source files.
-
is_incompleteas a structured predicate, not a message match. - Limits reset per line; state persists; both tested.
- No input ends the session: syntax error, runtime error, limit, host panic, control characters, 1 MB line.
-
The
pendingbuffer is bounded. -
Dot commands, including
.interpreporting agreement or divergence. - Exit codes 0/1/2/70, tested.
- A REPL fuzz target, run 10 minutes clean, corpus committed.
Validation / Self-check
- Which representation does each subcommand print, and which lab built it?
- How does the REPL distinguish incomplete input from a syntax error? Why is a structured predicate better than matching the message?
- Why does
eval_repltry the expression path first? - Why is each REPL line its own
SourceFile, and what must be bounded as a result? - Give the three keyboard behaviors (Ctrl-C at a prompt, Ctrl-C mid-input, Ctrl-D).
- Why do limits reset per line while state persists?
- What are the four exit codes and which distinction matters operationally?
- Why is a REPL fuzz target unusually high-yield?
Next: Lab 26 — The Test Matrix.