Section 5: The Host Boundary

The language is done. Now make it a component.

This section turns ember from a program that runs scripts into a crate a Rust service embeds: an Engine with a small public API, a way to move values across the boundary in both directions, host objects exposed to scripts, a capability-gated standard library, host-controlled modules, and a sandbox with a written threat model.

It covers Milestones M12 and M13, and it contains the hardest Rust in the curriculum.

What You Build

The Layer You Are Building

   ┌──────────────────────────────────────────────────────────────────────┐
   │  YOUR RUST APPLICATION                                               │
   │    owns: Article, User, the request deadline, the metrics registry   │
   └───────────────────────────────┬──────────────────────────────────────┘
                                   │
   ┌───────────────────────────────▼──────────────────────────────────────┐
   │  ENGINE                                            engine.rs         │
   │    execute(src)          compile + run a chunk                       │
   │    call("score", args)   invoke a script function from Rust          │
   │    set_global/get_global                                             │
   │    register_function     Rust code the SCRIPT may call               │
   │    limits() / stats() / collect()                                    │
   │                                                                      │
   │  ┌─────────────────┐   ┌──────────────────┐   ┌───────────────────┐  │
   │  │ MARSHALING      │   │ CAPABILITIES     │   │ BUDGETS           │  │
   │  │ Rust ⟷ Value    │   │ what the script  │   │ instructions      │  │
   │  │ fallible, typed │   │ MAY do — granted │   │ memory · depth    │  │
   │  │ never panics    │   │ never assumed    │   │ results · sources │  │
   │  └─────────────────┘   └──────────────────┘   └───────────────────┘  │
   └───────────────────────────────┬──────────────────────────────────────┘
                                   │
 ══════════════════════════════════▼══════════════ THE OWNERSHIP BOUNDARY ═
   ┌──────────────────────────────────────────────────────────────────────┐
   │  VM + HEAP  (Sections 3–4, unchanged)                                │
   │    scripts never hold a raw pointer to host memory                   │
   │    the host never holds an unrooted handle to a collectable object   │
   └──────────────────────────────────────────────────────────────────────┘

The Concepts, and Where Each Is Treated

The Labs

The Target API

Everything in this section exists to make this compile, run, and be defensible:

#![allow(unused)]
fn main() {
use ember::{Engine, Value, Limits};

let mut engine = Engine::builder()
    .limits(Limits { instructions: 10_000_000, memory: 8 << 20, depth: 200, ..Default::default() })
    .capabilities(Capabilities::SAFE)          // no fs, no net, no process, no env, no clock
    .build();

engine.register_function("log", |_ctx, args| {
    tracing::info!(?args, "policy log");
    Ok(vec![Value::Nil])
})?;

engine.execute(POLICY_SOURCE)?;

let score: f64 = engine.call("score", (&user, &article))?;
let stats = engine.stats();
}

Read it against the introduction's promise. Nothing in it is new machinery — it is Sections 1–4 with a door.

The Central Rust Problem: Re-entrancy

State it now, because it will consume an afternoon otherwise.

A registered Rust function is called by the VM, and it may call back into the VM. So the VM holds &mut self when it invokes the callback, and the callback needs &mut self to do anything useful. That does not typecheck, and the borrow checker is right: the callback could drop the very closure it is executing.

#![allow(unused)]
fn main() {
// The shape that does NOT work.
let f = self.heap.native(handle)?;      // borrows self.heap
(f.func)(self, args)?;                  // ERROR: self is already borrowed
}

The fix is small and the reasoning is the lesson:

#![allow(unused)]
fn main() {
// Native functions are stored as Rc<dyn Fn>. CLONE the Rc (a refcount bump),
// which ends the borrow of the heap, and only then call. The callback receives
// a Ctx that owns the &mut Vm for the duration.
let f: Rc<NativeFn> = self.heap.native(handle)?.func.clone();
drop_borrow();
let results = (f)(&mut Ctx::new(self), args)?;
}

Three rules follow, and Lab 19 enforces all three:

1. No borrow of the heap or the VM may be held across a call into host code. Copy out what you need first; Value is Copy, which is why this works at all.
2. A host callback sees a consistent VM. ip synced, stack balanced, no half-updated table. Every native call site is a safe point, exactly as every metamethod dispatch site is.
3. Anything the host holds across an allocation must be rooted. Lua's C API is stack-based for precisely this reason; Ember's Ctx gives you a scoped root guard.

Deliverables

• The target API above compiles and runs, with #[doc] examples exercised by cargo test --doc.
• A registered Rust function can call back into Ember, and back out again, with no unsafe and no RefCell.
• ToValue/FromValue for the primitives, Option<T>, Vec<T>, HashMap<String, T>, and tuples; a failed conversion is a typed error naming both types and the argument position.
• A Rust struct is exposed as userdata with field access from script, traced by the collector, and safe under --gc-stress.
• print, type, tostring, tonumber, assert, error, pcall, select, ipairs, pairs, rawget/rawset/rawequal/rawlen, setmetatable/getmetatable, and math.*, string.*, table.*.
• No filesystem, network, process, environment, or clock access is reachable by default, and a test asserts it by enumerating every registered global.
• require works through a host-supplied Resolver, with a cache, a cycle error, and no ambient filesystem.
• Every limit is enforced and returns ErrorKind::Limit: instructions, memory, call depth, result count, source size.
• docs/sandboxing.md states assets, adversaries, controls, and — explicitly — non-goals.
• docs/adr/ADR-010-host-controlled-modules.md, ADR-011-send-sync.md, and ADR-013-production-profile.md written.
• Differential tests still green (the tree walker gets the same Engine).

Common Mistakes in This Section

How to Verify Success

# 1. The embedding example runs.
cargo run --example recommendation_policy

# 2. Re-entrancy: Rust → script → Rust → script.
cargo test --test engine reentrancy

# 3. The default capability set is actually empty of dangerous things.
cargo test --test sandbox default_globals_contain_no_ambient_authority

# 4. Every limit fires and is distinguishable from a script bug.
for t in instructions memory depth results source_size; do
  cargo test --test sandbox "limit_$t"
done

# 5. No panics from any script input.
cargo fuzz run run -- -max_total_time=600

# 6. The docs compile and their examples run.
cargo test --doc
cargo doc --no-deps --open

Section Profile: What a Section 5 Graduate Can Do

• Design an embedding API for a runtime, and say why each function is on it.
• Explain the ownership boundary: what the host may hold, what the script may hold, and for how long.
• Solve the re-entrancy problem in Rust without unsafe, RefCell, or a redesign.
• Marshal values between a host language and a dynamic runtime, with typed errors and no panics.
• Expose a host object safely, including its interaction with the collector.
• Design a capability set, and defend every capability that is not in it.
• Write a threat model with explicit non-goals, and explain what a sandbox cannot promise.

Next: The Host Boundary.