The Capstone: A Recommendation Policy Engine

Build a real thing with the runtime you built.

Rust owns the data, the budgets, the metrics, and the clock. Ember owns the business rules. The result is a service component that reloads its ranking policy without a deploy, evaluates it deterministically under a budget, and tells you when a policy is about to become too expensive.

This is where the runtime stops being an exercise.

The Shape

   ┌──────────────────────────────────────────────────────────────────────────┐
   │  RUST OWNS                                                               │
   │    candidate retrieval        user + article data      metrics           │
   │    the request deadline       execution limits         policy loading    │
   │    logging                    A/B assignment           the RNG seed      │
   └───────────────────────────────┬──────────────────────────────────────────┘
                                   │  candidates as ONE userdata (Lab 20)
                                   │  user as userdata
                                   ▼
   ┌──────────────────────────────────────────────────────────────────────────┐
   │  EMBER OWNS                                                              │
   │    the scoring function       boosts and penalties                       │
   │    business rules             experiment branches                        │
   └───────────────────────────────┬──────────────────────────────────────────┘
                                   │  a score per candidate
                                   ▼
   ┌──────────────────────────────────────────────────────────────────────────┐
   │  RUST OWNS                                                               │
   │    sorting     truncation     the response      the stats emission       │
   └──────────────────────────────────────────────────────────────────────────┘

Note what Ember does not own: the sort. The policy produces a score per candidate; Rust sorts. That keeps the hot loop out of the script, makes the result deterministic without trusting a script comparator, and sidesteps the hostile-comparator hazard entirely. Scoring is policy; sorting is mechanism.

The Policy

-- policies/v3.ember
local BOOST_FRESH   = 1.4
local BOOST_TOPIC   = 0.2
local PENALTY_SEEN  = 0.5

function score(user, article)
  local s = article.semantic_score

  if article.age_hours < 6 then
    s = s * BOOST_FRESH
  end

  if user.preferred_topics[article.topic] then
    s = s + BOOST_TOPIC
  end

  if user.seen[article.id] then
    s = s * PENALTY_SEEN
  end

  return s
end

Eleven lines. Now ask what it would take to change BOOST_FRESH from 1.4 to 1.5:

And now ask what it would take to add "boost articles whose topic matches the user's second preference, but only on weekends, and only for users in the experiment arm." TOML cannot express it. Rust needs a deploy. That is the case for embedding a language, and the honest chapter is about the cases where it is not.

The Requirements

The Host

#![allow(unused)]
fn main() {
pub struct PolicyEngine {
    engine: Engine,
    version: PolicyVersion,
}

impl PolicyEngine {
    /// Load and VALIDATE. A policy that compiles but returns nil never serves traffic.
    pub fn load(src: &str, version: PolicyVersion, canary: &CanaryInput) -> Result<Self> {
        let mut engine = Engine::builder()
            .limits(Limits { instructions: 5_000_000, memory: 8 << 20,
                             call_depth: 64, ..Default::default() })
            .capabilities(Capabilities::SAFE)          // no clock, no fs, no net, no RNG
            .build();
        engine.execute(src)?;                          // compile errors surface HERE
        let mut pe = PolicyEngine { engine, version };
        pe.smoke_test(canary)?;                        // ← run it before promoting
        Ok(pe)
    }

    pub fn rank(&mut self, user: &User, candidates: Vec<Article>, top_n: usize)
        -> Result<(Vec<Ranked>, Stats)>
    {
        let user_ud  = self.engine.create_userdata(UserRef::from(user))?;
        let cands_ud = self.engine.create_userdata(Candidates { articles: candidates })?;
        self.engine.set_global("user", user_ud)?;
        self.engine.set_global("candidates", cands_ud)?;

        // `evaluate`, not `execute`: FRESH globals, so a policy that writes a
        // global cannot leak state into the next request.
        let scores: Vec<f64> = self.engine.evaluate("score_all", ())?;

        let mut ranked = zip_scores(scores);
        // RUST sorts. Deterministic, and no script comparator is trusted.
        ranked.sort_by(|a, b| b.score.total_cmp(&a.score).then(a.id.cmp(&b.id)));
        ranked.truncate(top_n);
        Ok((ranked, self.engine.stats()))
    }
}
}

.then(a.id.cmp(&b.id)) is the tie-break, and it is not optional: two candidates with equal scores must order identically on every replica. Floating-point scores tie more often than you expect.

Live Reload

#![allow(unused)]
fn main() {
// The pattern from Lab 22: swap WHOLE ENGINES, and validate before promoting.
async fn reload_loop(current: Arc<ArcSwap<PolicyEngine>>, source: PolicySource) {
    loop {
        let (src, version) = source.next_version().await;
        match PolicyEngine::load(&src, version, &CANARY) {
            Ok(next) => {
                current.store(Arc::new(next));
                tracing::info!(%version, "policy promoted");
            }
            Err(e) => {
                // The OLD policy keeps serving. A bad push is a metric, not an outage.
                tracing::error!(%version, error = %render(&e), "policy rejected");
                metrics::counter!("policy_reload_rejected").increment(1);
            }
        }
    }
}
}

Three properties, and each one is why this pattern rather than module-level reload:

1. A bad policy never serves traffic. It fails at load, which includes a smoke evaluation.
2. In-flight requests finish on the old engine. ArcSwap is atomic; nothing is torn down under a running evaluation.
3. Fresh everything. New globals, new heap, new intern table. No state survives a reload, which removes the entire class of "the new policy inherited the old one's cache".

A/B Comparison

Requirement 6, and it is where determinism pays:

#![allow(unused)]
fn main() {
pub fn compare(a: &mut PolicyEngine, b: &mut PolicyEngine,
               user: &User, candidates: &[Article]) -> Comparison {
    let (ra, sa) = a.rank(user, candidates.to_vec(), 100).unwrap();
    let (rb, sb) = b.rank(user, candidates.to_vec(), 100).unwrap();
    Comparison {
        rank_correlation: kendall_tau(&ra, &rb),
        moved_into_top_10: set_difference(&ra[..10], &rb[..10]),
        // Deterministic counters: a COST comparison with no timing noise.
        instruction_delta: sb.instructions as i64 - sa.instructions as i64,
        allocation_delta:  sb.allocations  as i64 - sa.allocations  as i64,
    }
}
}

instruction_delta is the number a reviewer actually wants: "v3 costs 12% more instructions per request than v2" is a fact, computed offline, with no benchmark and no noise. That is deterministic instrumentation turning into a product feature.

Deliverables

• examples/recommendation_policy.rs — the full host, runnable.
• Article, User, and Candidates as userdata; the candidate list is one object.
• PolicyEngine::load with validation and a canary smoke test.
• Live reload by whole-engine swap; a rejected policy never serves traffic; tested.
• Deterministic ranking, including a tie-break, verified across 20 runs and two engines.
• Per-evaluation stats emitted; a utilization alert threshold documented.
• compare() producing rank correlation and an instruction delta.
• Errors that name the policy version, the source line, and — where possible — the candidate id.
• A benchmark: 10,000 candidates, a realistic policy, with the p99 recorded.
• A load test that exceeds the instruction budget, showing ErrorKind::Limit and the metric.
• The mandatory end-to-end trace.
• The honest chapter, written in your own words.

Common Mistakes

Validation / Self-check

1. Why does Rust sort rather than the policy? Give two reasons.
2. Why is the candidate list one userdata rather than 10,000? What did Lab 20 measure?
3. What does the canary smoke test catch that compilation does not?
4. Why swap whole engines rather than reload a module? Give three properties.
5. Why is evaluate used rather than execute?
6. Why does the sort need a tie-break?
7. What makes instruction_delta more useful to a reviewer than a wall-clock comparison?
8. What must be true for two replicas to return identical rankings? List every requirement.

Next: The End-to-End Trace — the mandatory final walkthrough.