Lab 23: Limits and Sandboxing (Milestone 14)

Background

You will wire every limit end to end, write docs/sandboxing.md, and produce fifteen tests — one per threat in the table, each asserting ErrorKind::Limit rather than "an error happened".

Nothing in this lab is new machinery. The instruction budget has been in the fetch position since Lab 11; the memory budget has been in Heap::alloc since Lab 15. What this lab adds is completeness, observability, and a document that says what is and is not defended.

Why This Lab Matters

A control without a test is a hope. Fifteen threats, fifteen tests.
The non-goals list is the most credible part of the security document, because it is the part that only gets written if the analysis actually happened.
Fresh globals per evaluation is the single cheapest isolation primitive Ember has, and it closes the cross-request state leak that every long-lived embedded runtime eventually hits.

Prerequisites

Labs 19–22 complete.
The Threat Model read.
The capability-surface golden file committed and read.

Predict First

Which of the fifteen threats is not currently covered by anything you have built?
while true do pcall(function() while true do end end) end — does it terminate?
A script sets a global. The next evaluation on the same engine reads it. Should it see it?
A host function sleeps for a minute. Which budget stops it?
--max-instructions 0 — does the script run zero instructions, or infinitely many?
A 10 MB source file with 100,000 nested parentheses. Which two limits does it hit, and in which order?

Step 1: The `Limits` Struct, Complete

#![allow(unused)]
fn main() {
#[derive(Clone, Debug)]
pub struct Limits {
    pub instructions: u64,      // Lab 11: the fetch position
    pub memory:       usize,    // Lab 15: Heap::alloc, and every allocating stdlib function
    pub call_depth:   usize,    // Lab 7/11: counted across NATIVE boundaries too
    pub expr_depth:   usize,    // Lab 2: the parser; MULTIPLIES with call_depth
    pub max_results:  usize,    // Lab 17
    pub max_args:     usize,    // Lab 17
    pub source_bytes: usize,    // NEW: checked before lexing
    pub modules:      usize,    // Lab 22: distinct modules per engine
    pub meta_chain:   usize,    // Lab 18: __index chain depth
}

impl Default for Limits {
    fn default() -> Self {
        // SAFE DEFAULTS ARE STRICT. A generous default is a control nobody
        // enabled. Hosts raise these deliberately, and `stats()` tells them
        // when they need to.
        Limits { instructions: 10_000_000, memory: 16 << 20, call_depth: 200,
                 expr_depth: 200, max_results: 1_000, max_args: 250,
                 source_bytes: 1 << 20, modules: 64, meta_chain: 100 }
    }
}
}

expr_depth × call_depth is the one that multiplies. 200 × 200 nested eval frames is the worst case for the tree walker; verify against your measured bytes-per-frame number from Lab 7. If the product exceeds a quarter of a 2 MiB thread stack, lower the limits — do not raise the threshold.

Step 2: Fresh Globals Per Evaluation

#![allow(unused)]
fn main() {
impl Engine {
    /// Evaluate with a FRESH globals table. The library and host functions are
    /// re-linked (cheap: they are the same handles); anything the script wrote
    /// last time is gone.
    pub fn evaluate<A: ToValues, R: FromValues>(&mut self, entry: &str, args: A) -> Result<R> {
        let saved = self.vm.globals;
        self.vm.globals = self.fresh_globals()?;      // pre-built prototype, cloned
        let r = self.call(entry, args);
        self.vm.globals = saved;
        r
    }
}
}

This is the cheapest isolation primitive in the whole section. Without it:

-- request 1
function score(a) cache = cache or {} ; cache[a.id] = a ; return 1 end
-- request 2 reads request 1's articles out of `cache`

That is a cross-request data leak written by a well-meaning script author trying to memoize. It is not an attack; it is the normal failure mode, and a fresh globals table removes it entirely.

Note: evaluate is distinct from execute. execute loads a policy (defining functions, which live in the persistent globals); evaluate runs one against fresh mutable state. Two methods because two lifetimes — and the capstone uses exactly this split.

Step 3: The Fifteen Tests

One per row of the threat table. Each asserts ErrorKind::Limit (or the specific control's kind) and names the threat:

#![allow(unused)]
fn main() {
macro_rules! threat {
    ($name:ident, $src:expr, $kind:expr) => {
        #[test] fn $name() {
            let e = run_with_limits($src, &Limits::strict()).unwrap_err();
            assert_eq!(e.kind, $kind, "threat not controlled: {}", stringify!($name));
        }
    };
}

threat!(t01_infinite_loop,        "while true do end",                              ErrorKind::Limit);
threat!(t02_infinite_recursion,   "local function f() return f() end return f()",   ErrorKind::Limit);
threat!(t02b_mutual_recursion,    "local a,b
                                   function a() return b() end
                                   function b() return a() end return a()",         ErrorKind::Limit);
threat!(t03_table_bomb,           "local t={} for i=1,1e9 do t[i]=i end",           ErrorKind::Limit);
threat!(t04_string_bomb,          "return ('x'):rep(1e9)",                          ErrorKind::Limit);
threat!(t05_result_amplification, "local t={} for i=1,1e6 do t[i]=i end
                                   return table.unpack(t)",                          ErrorKind::Limit);
threat!(t06_nesting_bomb,         &("return ".to_owned() + &"(".repeat(100_000)),   ErrorKind::Parse);
threat!(t14_pcall_cannot_swallow, "while true do pcall(function() while true do end end) end",
                                                                                     ErrorKind::Limit);
// … t07 error amplification, t08 capability escape, t09 state leak, t10 hash DoS,
//    t11 nondeterminism, t12 no panics, t13 malformed bytecode, t15 module flood
}

Every test names its threat number, so a failure points at a row in the document rather than at a line of code.

Step 4: Source Size and Compile Budget

Two threats are pre-execution and easy to forget:

#![allow(unused)]
fn main() {
pub fn execute(&mut self, src: &str) -> Result<()> {
    if src.len() > self.limits.source_bytes {
        return Err(limit(format!("source exceeds {} bytes", self.limits.source_bytes)));
    }
    // Compilation is real work and a script that requires generated module
    // names in a loop is a CPU vector. Charge it.
    let before = self.vm.budget.remaining;
    let proto = self.compile(src)?;
    self.vm.budget.charge_compile(before)?;
    // ...
}
}

Step 5: Observability

A limit that fires with no warning is an outage; a limit that reports usage is a tuning knob.

#![allow(unused)]
fn main() {
pub struct Stats {
    pub instructions: u64,      pub instruction_budget: u64,
    pub peak_memory: usize,     pub memory_budget: usize,
    pub max_call_depth: usize,  pub call_depth_budget: usize,
    pub allocations: u64,       pub live_objects: usize,
    pub gc_runs: u32,           pub gc_pause_total: Duration,
    pub calls: u64,             pub native_calls: u64,
}

impl Stats {
    /// The number a host should alert on: "this policy is at 82% of budget".
    pub fn instruction_utilization(&self) -> f64 { /* ... */ }
}
}

Emit these per evaluation. The capstone puts them in a tracing span, and the operational rule is: alert at 80% utilization, not at 100%, because 100% is already a failed request.

Step 6: Write `docs/sandboxing.md`

The seven-section shape from the threat-model chapter. Two rules for writing it:

Every control row links to its test. A document whose claims are executable is a different kind of document.
The non-goals section is not shorter than the controls section. If it is, you have not finished the analysis.

Then the README paragraph — the production profile, verbatim, above the fold. A security property three clicks deep is a property nobody evaluated.

The Trace

$ ember run --max-instructions 100000 --stats -e '
local t = {}
for i = 1, 1000 do t[i] = i * 2 end
local s = 0
for i = 1, #t do s = s + t[i] end
return s'
1001000
--- stats ---
instructions:  14,027 / 100,000   (14%)
peak memory:   41 KB / 16 MB      (0%)
call depth:    1 / 200
allocations:   3      gc runs: 0

Then the same program with a budget that is too small, which is what a host sees when a policy grows:

$ ember run --max-instructions 10000 --stats -e '<the same program>'
error: instruction budget exhausted

stack traceback:
  in main chunk   <argv>:4

--- stats ---
instructions:  10,000 / 10,000    (100%)   ← the number that should have alerted at 8,000
$ echo $?
1

And the fifteen threats, run as a suite:

$ cargo test --test sandbox
running 17 tests
test t01_infinite_loop ... ok
test t02_infinite_recursion ... ok
test t02b_mutual_recursion ... ok
test t03_table_bomb ... ok
test t04_string_bomb ... ok
test t05_result_amplification ... ok
test t06_nesting_bomb ... ok
test t07_error_message_truncation ... ok
test t08_capability_surface_matches_golden ... ok
test t09_globals_are_fresh_per_evaluation ... ok
test t10_hash_seed_is_per_engine ... ok
test t11_output_is_seed_independent ... ok
test t12_no_panic_on_fuzz_corpus ... ok
test t13_malformed_bytecode_is_rejected ... ok
test t14_pcall_cannot_swallow_limits ... ok
test t15_module_flood_is_bounded ... ok
test all_threats_in_the_document_have_a_test ... ok

That last test is the one to write carefully:

#![allow(unused)]
fn main() {
#[test]
fn all_threats_in_the_document_have_a_test() {
    // docs/sandboxing.md's control table has a `tNN` id per row. Every id must
    // correspond to a test in this file. Adding a threat to the document
    // without a test FAILS THE BUILD — which is the only way a threat model
    // stays true.
    let documented: HashSet<String> = threat_ids_in_sandboxing_doc();
    let tested: HashSet<String> = threat_ids_in_this_file();
    assert_eq!(documented, tested);
}
}

Expected Output

$ ember run --max-instructions 0 -e 'return 1'
error: instruction budget exhausted
$ echo $?
1

$ ember run -e 'g = 5' && ember run -e 'return tostring(g)'
nil                                     # separate processes, obviously

$ cargo run --example fresh_globals      # same ENGINE, two evaluations
evaluation 1: set g = 5
evaluation 2: g is nil                  ← the isolation primitive, visible

Debugging Steps

A budget-exhausted script keeps running

pcall is catching Limit. Threat 14.

`--max-instructions 0` runs forever

The check is remaining -= 1 before the zero test, so 0 wraps to u64::MAX. Saturating arithmetic and a test at exactly zero.

The memory limit fires during a legitimate workload

You wired the GC threshold to the memory limit. They are different numbers with different meanings — the Lab 15 warning.

Deep recursion aborts instead of erroring

call_depth is not counted on one path — usually the native or metamethod route. Every entry into a call must pass the same check.

The nesting bomb aborts in `Drop` rather than erroring in the parser

The parse-depth limit is too high for the recursive-Drop hazard. One guard, two hazards — the AST chapter.

An error message contains 3 MB of script data

Threat 7. Truncate script-controlled data in every message, and add the test.

Two engines produce different output for the same script

The hash seed leaked into observable behavior. Iteration order must not depend on it — ADR-008.

Experiment

CLAIM. The instruction budget's cost when enabled is small, and its cost when disabled is zero — so there is no performance reason to ship without it.

METHOD. Benchmark fib_25 and loop_10m in three configurations: budget disabled at compile time (a feature flag), budget enabled with u64::MAX remaining, and budget enabled with a realistic limit.

PREDICTION. What is the overhead of the enabled-but-unlimited case? (It is one predictable branch per instruction, which a modern predictor learns immediately.)

RESULT. Record it in docs/learning/13-sandboxing.md. If the overhead is in the noise — it will be — then delete the compile-time disable flag. A safety control with no measurable cost should not have an off switch, and having measured it is what lets you say so.

Test

#![allow(unused)]
fn main() {
#[test]
fn globals_are_fresh_per_evaluation() {
    let mut e = Engine::new();
    e.execute("function leak() cache = (cache or 0) + 1 return cache end").unwrap();
    assert_eq!(e.evaluate::<_, i64>("leak", ()).unwrap(), 1);
    assert_eq!(e.evaluate::<_, i64>("leak", ()).unwrap(), 1);   // NOT 2
}

#[test]
fn functions_defined_by_execute_survive_evaluate() {
    // The other half of the split: a policy LOADS once and RUNS many times.
    let mut e = Engine::new();
    e.execute("function score(x) return x * 2 end").unwrap();
    for _ in 0..3 { assert_eq!(e.evaluate::<_, i64>("score", (21,)).unwrap(), 42); }
}

#[test]
fn budget_of_zero_permits_nothing() {
    assert_eq!(run_with_budget("return 1", 0).unwrap_err().kind, ErrorKind::Limit);
}

#[test]
fn error_messages_truncate_script_controlled_data() {
    let e = run("local s = ('x'):rep(100000) error(s)").unwrap_err();
    assert!(e.message.len() < 1024, "message was {} bytes", e.message.len());
    assert!(e.message.contains("…"), "truncation must be visible");
}

#[test]
fn call_depth_is_counted_across_native_boundaries() {
    let mut e = Engine::new();
    e.register_function("bounce", |ctx, args| ctx.call(args[0], &[args[0]])).unwrap();
    e.execute("function f(g) return bounce(g) end").unwrap();
    assert_eq!(e.evaluate::<_, i64>("f", ()).unwrap_err().kind, ErrorKind::Limit);
}

#[test]
fn output_is_independent_of_the_hash_seed() {
    for case in corpus() {
        let a = Engine::with_seed(1).run(&case.src);
        let b = Engine::with_seed(0xdeadbeef).run(&case.src);
        assert_eq!(a, b, "{} depends on the hash seed", case.name);
    }
}

#[test]
fn stats_report_utilization_not_just_violations() {
    let mut e = Engine::builder().limits(Limits { instructions: 100_000, ..Default::default() })
                                 .build();
    e.execute("local s=0 for i=1,1000 do s=s+i end").unwrap();
    let s = e.stats();
    assert!(s.instruction_utilization() > 0.0 && s.instruction_utilization() < 1.0);
}

#[test]
fn all_threats_in_the_document_have_a_test() { /* Step 6 */ }
}

Challenge Extensions

A watchdog backstop. A supervisor thread that sets an atomic "abort" flag checked in the fetch position, as a secondary control for host functions the budget cannot see. Then document precisely what it does and does not guarantee about where execution stops.
Per-capability budgets. Charge require more than ADD. Is a weighted budget more useful than a uniform one, or just harder to explain to a host?
A memory high-water alarm. Call a host callback when the heap crosses 80% so a service can shed load before the limit fires.
Landlock/seccomp escalation. Wrap the whole engine in a restricted subprocess and show that even a hypothetical runtime exploit cannot open a file. This is the escalation path from ADR-013, made real.
A fuzz target per threat. Generate programs targeting each row and assert the control fires. The most valuable fuzzing in the project, because it targets a specification.

Deliverables

Limits with all nine fields, strict defaults, and the expr_depth × call_depth product verified against the measured stack-frame size.
Engine::evaluate with fresh globals per evaluation, distinct from execute.
Fifteen threat tests, each named for its row and asserting the right ErrorKind.
The all_threats_in_the_document_have_a_test meta-test.
Source-size and compile-cost limits.
Error messages truncate script-controlled data, with a test.
Call depth counted across native and metamethod boundaries.
Stats reporting utilization against each budget; per-evaluation emission.
docs/sandboxing.md with all seven sections, every control linked to its test, and a non-goals section no shorter than the controls section.
The production-profile paragraph in the README, above the fold.
The budget-overhead experiment recorded, and the disable flag deleted if the result says so.
docs/adr/ADR-013-production-profile.md written.
Milestone 14, half one, complete.

Validation / Self-check

Which two limits multiply, and how did you verify the product is safe?
Why is evaluate distinct from execute? What lives in each lifetime?
Give the cross-request leak that fresh globals prevents, and say why it is a normal failure rather than an attack.
Why must safe defaults be strict rather than generous?
Which threat is not controlled by anything you built, and where is that written down?
Why does the meta-test comparing the document to the test file matter more than any individual threat test?
What should a host alert on, and why not at 100%?
What did the budget-overhead experiment show, and what did you delete because of it?

Next: Section 6 — Production.

Open-Source Engineer & Contributor