Lab 19: The Engine API (Milestone 13)

Background

You will build src/engine.rs and src/marshal.rs: the seven-method public API, Ctx, ToValue/FromValue, rooting, and the re-entrancy machinery that lets a registered Rust function call back into a script.

This lab contains the hardest Rust in the curriculum, concentrated in about forty lines.

Why This Lab Matters

The public API is a forever-promise. Everything you make pub here is something you support.
Re-entrancy is where the borrow checker stops being an inconvenience and starts being right. The error it gives you describes a real use-after-free that C embeddings have.
This is the first time ember is a crate rather than a program. cargo doc output becomes a deliverable.

Prerequisites

Section 4 complete; differential tests green; corpus green under --gc-stress.
The Host Boundary and Value Marshaling read.

Predict First

Write the naive call_native that borrows the heap and then calls the function. What does the compiler say? Is it right?
A host function calls a script function that calls the same host function. What must be true for that to work?
let t = ctx.new_table()?; let u = ctx.new_table()?; — is t still valid? Why?
What should engine.call::<f64>("score", args) do if score returns a string?
Should Engine be Send? Sync? What does each answer cost?
What happens if a registered Rust function panics?

Step 1: The Public Surface, and Nothing Else

#![allow(unused)]
fn main() {
// src/lib.rs — the entire public API. Everything else is pub(crate).
pub use engine::{Engine, EngineBuilder, Ctx, RootGuard};
pub use value::Value;
pub use error::{EmberError, ErrorKind, Result};
pub use marshal::{ToValue, FromValue, ToValues, FromValues};
pub use limits::{Limits, Capabilities};
pub use stats::Stats;
}

Notice what is not exported: Vm, Heap, GcRef, Chunk, Op, Table, Proto. A host that cannot name a handle cannot hold a stale one, and a host that cannot name Op is not depending on your instruction set.

# The check that keeps it honest, run in CI:
cargo public-api --diff-git-checkouts main HEAD

Add #![deny(missing_docs)] to lib.rs now. Every pub item needs a doc comment with an example, and cargo test --doc runs them.

Step 2: `Ctx` and Re-entrancy

Write call_native from the concept chapter. Then prove the re-entrancy works, because "it compiles" is not evidence:

#![allow(unused)]
fn main() {
#[test]
fn rust_can_call_script_can_call_rust_can_call_script() {
    let mut e = Engine::new();
    e.register_function("twice", |ctx, args| {
        let f = args[0];
        let n: i64 = FromValue::from_value(args[1], ctx)?;
        // Re-enter the VM, twice, from inside a native call.
        let a = ctx.call(f, &[Value::Integer(n)])?;
        let b = ctx.call(f, &[a[0]])?;
        Ok(b)
    }).unwrap();
    e.execute("function double(x) return x * 2 end
               function apply() return twice(double, 5) end").unwrap();
    assert_eq!(e.call::<_, i64>("apply", ()).unwrap(), 20);
}
}

Warning: The naive version — borrowing the heap to get the function, then calling it — will not compile, and you should write it once and read the error before writing the working version. cannot borrow *self as mutable because it is also borrowed as immutable is the compiler telling you that the callee could free the callee. That is the whole lesson, and skipping the error skips it.

Step 3: Rooting

#![allow(unused)]
fn main() {
#[must_use = "a RootGuard that is immediately dropped roots nothing"]
pub fn root(&mut self, v: Value) -> RootGuard<'_>;
}

The temp-root stack is root set 9 (after the module cache, which arrives in Lab 22 — for now it is 8). Add it to enumerate_roots, bump the count constant, and re-run the corpus under --gc-stress.

Then write the test that fails without it:

#![allow(unused)]
fn main() {
#[test]
fn host_locals_need_rooting_across_allocations() {
    with_gc_stress(|| {
        let mut e = Engine::new();
        e.register_function("build", |ctx, _| {
            let a = ctx.new_table()?;
            let _root = ctx.root(a);        // ← delete this line and the test fails
            let b = ctx.new_table()?;       // ← may collect
            ctx.table_set(b, Value::Integer(1), a)?;
            Ok(vec![b])
        }).unwrap();
        e.execute("local t = build() assert(t[1] ~= nil)").unwrap();
    });
}
}

Step 4: Marshaling

Write ToValue/FromValue for the primitives, String, Vec<u8>, Option<T>, Vec<T>, HashMap<String, T>, and tuples up to 8; plus ToValues/FromValues for argument and result lists.

Then the errors, which are the part hosts actually see:

#![allow(unused)]
fn main() {
// "bad argument #2 to 'score' (integer expected, got string)"
pub fn arg<T: FromValue>(ctx: &Ctx, args: &[Value], i: usize, fname: &str) -> Result<T>;
}

Convert every argument before running any host logic. Structural prevention of half-applied side effects, and it is one line of ordering:

#![allow(unused)]
fn main() {
e.register_function("record", |ctx, args| {
    let name: String = arg(ctx, args, 0, "record")?;    // ← all conversions FIRST
    let value: f64   = arg(ctx, args, 1, "record")?;
    metrics.record(&name, value);                        // ← then the effect
    Ok(vec![])
})?;
}

Step 5: `execute`, `call`, and the Panic Boundary

#![allow(unused)]
fn main() {
pub fn register_function<F>(&mut self, name: &str, f: F) -> Result<()>
where F: Fn(&mut Ctx<'_>, &[Value]) -> Result<Vec<Value>> + 'static {
    let wrapped = move |ctx: &mut Ctx<'_>, args: &[Value]| {
        // A HOST bug must not become a RUNTIME bug. A panic unwinding through
        // the VM's frames leaves invariants half-updated; catch it at the
        // boundary, convert it, and poison the engine.
        match std::panic::catch_unwind(AssertUnwindSafe(|| f(ctx, args))) {
            Ok(r) => r,
            Err(_) => Err(ctx.poison("host function panicked")),
        }
    };
    // ...
}
}

Poisoning matters. After a panic the VM's stack depth and frame stack may be inconsistent. A poisoned Engine refuses further execute/call with a clear message rather than running on a suspect VM. Test it.

Step 6: `!Send + !Sync`, Declared

#![allow(unused)]
fn main() {
pub struct Engine {
    vm: Vm,
    // Makes the negative impls explicit and stops an accidental auto-impl if
    // every field later happens to be Send. See ADR-011.
    _not_send_sync: PhantomData<*const ()>,
}
}

#![allow(unused)]
fn main() {
#[test]
fn engine_is_neither_send_nor_sync() {
    // If this test ever fails, someone changed a threading guarantee by
    // accident. Guarantees should change on purpose.
    static_assertions::assert_not_impl_any!(Engine: Send, Sync);
}
}

Write ADR-011 now, including the honest cost: a tokio task cannot hold an Engine across an .await. Put that sentence in the README too.

The Trace

// examples/embedding.rs
use ember::{Engine, Value, Limits, Capabilities};

fn main() -> ember::Result<()> {
    let mut engine = Engine::builder()
        .limits(Limits { instructions: 1_000_000, memory: 4 << 20, depth: 64, ..Default::default() })
        .capabilities(Capabilities::SAFE)
        .build();

    engine.register_function("log", |ctx, args| {
        let msg: String = ember::arg(ctx, args, 0, "log")?;
        println!("[script] {msg}");
        Ok(vec![])
    })?;

    engine.set_global("threshold", 0.75_f64)?;
    engine.execute(r#"
        function score(base, boost)
            log("scoring " .. base)
            local s = base * boost
            if s > threshold then s = s * 1.1 end
            return s
        end
    "#)?;

    let s: f64 = engine.call("score", (0.5_f64, 1.8_f64))?;
    println!("score = {s}");
    println!("{:#?}", engine.stats());
    Ok(())
}

$ cargo run --example embedding
[script] scoring 0.5
score = 0.9900000000000001
Stats {
    instructions: 41,
    allocations: 3,
    live_objects: 12,
    gc_runs: 0,
    gc_pause_total: 0ns,
    calls: 2,
    max_stack_depth: 6,
    max_call_depth: 2,
}

Read the trace of one value crossing the boundary twice:

   Rust f64 0.5
     │  ToValue::to_value        → Value::Float(0.5)          (no allocation)
     ▼
   engine.call("score", …)
     │  get_global("score")      → Value::Closure
     │  push callee, push args   → the callee's slots, no copy
     ▼
   VM: score runs
     │  "scoring " .. base       → CONCAT allocates an EmberStr
     │  log(msg)                 → CALL → call_native
     │      Rc clone, borrow ends, Ctx created
     │      FromValue::<String>  → validates UTF-8, may FAIL (returns Result)
     │      println!             → host side effect
     │      Ok(vec![])           → adjusted to `want`
     ▼
   RETURN 1 → Value::Float(0.99…)
     │  FromValues::<f64>        → checks the tag, no conversion needed
     ▼
   Rust f64 0.99…

Two things to notice. The f64 crossed twice with no allocation and no conversion — the tag was already right. The String crossing in was fallible, and that fallibility is the boundary doing its job: a script that passed a non-UTF-8 byte string gets a Marshal error naming the argument position, not a panic inside println!.

Expected Output

$ cargo test --doc
$ cargo test --test engine
test reentrancy ... ok
test host_locals_need_rooting_across_allocations ... ok
test host_panic_poisons_the_engine ... ok
test engine_is_neither_send_nor_sync ... ok
test marshal_errors_name_the_argument_position ... ok

$ cargo doc --no-deps
$ cargo public-api | head -20

PREDICTION. What is the per-call overhead in (a)? At what collection size does marshaling exceed the script's own execution time?

RESULT. Record it in docs/learning/12-embedding.md. It is the number that decides the capstone's candidate-passing design, and the userdata chapter's O(fields exposed) vs O(fields read) claim becomes a measurement rather than an assertion.

Test

#![allow(unused)]
fn main() {
#[test]
fn the_documented_api_example_compiles_and_runs() {
    // The README's example, as a test. Documentation that is not executed rots.
    include!("../examples/embedding.rs");
}

#[test]
fn marshal_errors_name_the_argument_position_and_both_types() {
    let mut e = Engine::new();
    e.register_function("f", |ctx, args| {
        let _: i64 = ember::arg(ctx, args, 1, "f")?;
        Ok(vec![])
    }).unwrap();
    let err = e.execute("f(1, 'oops')").unwrap_err();
    assert_eq!(err.kind, ErrorKind::Marshal);
    assert!(err.message.contains("#2"),      "{}", err.message);
    assert!(err.message.contains("integer"), "{}", err.message);
    assert!(err.message.contains("string"),  "{}", err.message);
}

#[test]
fn float_to_integer_conversion_follows_the_language() {
    assert_eq!(call_returning::<i64>("return 3.0").unwrap(), 3);
    assert_eq!(call_returning::<i64>("return 3.5").unwrap_err().kind, ErrorKind::Marshal);
}

#[test]
fn host_panic_poisons_the_engine() {
    let mut e = Engine::new();
    e.register_function("boom", |_, _| panic!("host bug")).unwrap();
    assert_eq!(e.execute("boom()").unwrap_err().kind, ErrorKind::Host);
    // Subsequent calls must refuse rather than run on a suspect VM.
    assert!(e.execute("return 1").is_err());
}

#[test]
fn no_public_type_exposes_a_gcref() {
    // A structural check: the public API must not let a host name a handle.
    let api = std::fs::read_to_string("target/public-api.txt").unwrap();
    assert!(!api.contains("GcRef"), "GcRef leaked into the public API");
    assert!(!api.contains("Heap"),  "Heap leaked into the public API");
}

#[test]
fn arguments_are_all_converted_before_any_side_effect() {
    let calls = Rc::new(Cell::new(0));
    let c = calls.clone();
    let mut e = Engine::new();
    e.register_function("record", move |ctx, args| {
        let _n: String = ember::arg(ctx, args, 0, "record")?;
        let _v: f64    = ember::arg(ctx, args, 1, "record")?;
        c.set(c.get() + 1);                       // the side effect
        Ok(vec![])
    }).unwrap();
    let _ = e.execute("record('ok', 'not-a-number')");
    assert_eq!(calls.get(), 0, "the side effect ran despite a conversion failure");
}
}

Challenge Extensions

serde support. Behind a feature: impl<T: Serialize> ToValue for T. Then measure the compile-time cost of the feature and decide whether it earns default status. (It should not.)
A Function handle. engine.get_function("score") returning an owned, rooted handle a host can call repeatedly without a global lookup. Measure the saving on the capstone's inner loop.
Engine::stash/unstash. A persistent registry for host-held values, traced by the collector, with unstash actually removing. Then write the test that proves it does not leak.
A send feature. Swap Rc→Arc, add + Send bounds, measure the atomic-refcount cost on fib_25. Report the number and decide. This is ADR-011's evidence.
cargo semver-checks in CI. Now that there is a public API, breaking it should be a build failure rather than a discovery.

Deliverables

The seven-method Engine API plus a builder; #![deny(missing_docs)]; every doc example runs under cargo test --doc.
Vm, Heap, GcRef, Chunk, Op, Table, Proto are not public; a test asserts it.
Re-entrancy works: Rust → script → Rust → script, with a test.
The naive borrow-conflicting version was written once and its error recorded in the journal.
Ctx with call, new_table, root, error; root is #[must_use].
The temp-root stack is a GC root set; the count constant bumped; corpus green under --gc-stress.
ToValue/FromValue for primitives, String, Vec<u8>, Option, Vec, HashMap, tuples.
arg::<T> errors name the function, position, and both types.
All arguments convert before any side effect, with a test.
Host panics become ErrorKind::Host and poison the engine.
Engine is !Send + !Sync, asserted; ADR-011 written including the async cost.
examples/embedding.rs runs and is itself a test.
The marshaling-cost experiment recorded.

Validation / Self-check

Write the naive call_native. What does the compiler say, and what real bug is it describing?
Give the four steps of the working version, in order, and say what each one is for.
Why is root #[must_use]? Write the mistake it catches.
Why must every argument convert before any side effect? Give the failing scenario.
Why does register_function use catch_unwind, and why poison the engine afterwards?
Why is Engine neither Send nor Sync? What is the honest cost, and where is it documented?
Why does the public API expose no GcRef? What bug class does that eliminate?
At what collection size did marshaling start to dominate, and what does that imply for the capstone?

Next: Lab 20 — Host Objects.

Open-Source Engineer & Contributor

Lab 19: The Engine API (Milestone 13)

Background

Why This Lab Matters

Prerequisites

Predict First

Step 1: The Public Surface, and Nothing Else

Step 2: `Ctx` and Re-entrancy

Step 3: Rooting

Step 4: Marshaling

Step 5: `execute`, `call`, and the Panic Boundary

Step 6: `!Send + !Sync`, Declared

The Trace

Expected Output

Debugging Steps

`cannot borrow self as mutable` in `call_native`

Re-entrancy compiles but the inner call sees a corrupted stack

`--gc-stress` fails in a host function

A host function panic takes down the test binary

`engine.call::<f64>` succeeds when the script returned a string

`Engine` became `Send`

Experiment

Test

Challenge Extensions

Deliverables

Validation / Self-check