Aura/docs/plans/0092-run-from-blueprint.md

Run a harness from a serialized blueprint — Implementation Plan

Parent spec: docs/specs/0092-run-from-blueprint.md (#165)

For agentic workers: REQUIRED SUB-SKILL: use the implement skill to run this plan. Steps use - [ ] checkboxes for tracking.

Goal: aura run <blueprint.json> [--params <json>] loads a serialized signal blueprint, wraps it in the Stage-1-R run scaffolding via a shared seam extracted from run_stage1_r, runs it, and emits a RunReport whose manifest carries a topology_hash — bit-identical to the Rust-builder path (C1).

Architecture: A serialized blueprint is a signal (sinks/broker/data out of the round-trippable set, C24), so aura run wraps the loaded signal in the existing scaffolding. The wrap is shared: stage1_r_graph is restructured into stage1_signal() + wrap_stage1r(signal,…), and a shared run_signal_stage1r runs either the Rust-built or the loaded signal — so "bit-identical to the Rust path" is the same seam over the same signal.

Tech Stack: aura-cli (the seam + CLI + the sha2 hash, research-side), aura-engine (RunManifest.topology_hash field only — no sha2 in the frozen engine, invariant 8).

Task order / focus: Tasks 1–2 are the risky/mechanical infrastructure (the behaviour-preserving restructure + the workspace-wide field add); run them and verify behaviour-preservation BEFORE Tasks 3–5 (seam + CLI + demo/tests).

---

Task 1: Restructure stage1_r_graph into stage1_signal() + wrap_stage1r() (behaviour-preserving)

Files:

• Modify: crates/aura-cli/src/main.rs:2630-2790 (stage1_r_graph)

This is a behaviour-preserving restructure, gated by the existing tests staying green — NOT a behaviour change. The signal nodes (fast, slow, spread, exposure) move into a standalone Composite with a price input-role and a bias output; everything else (broker / sinks / exec / cost / vol_proxy) stays in wrap_stage1r, which nests the signal and wires price + bias across the boundary.

• Step 1: Add stage1_signal — extract the signal leg (current main.rs:2644-2655 + the price feed into fast/slow from :2703-2715 + the spread/exposure wiring) into:

/// The Stage-1 signal leg as a standalone, serializable Composite: SMA-cross →
/// Bias. Exposes a `price` input-role and a single `bias` output — the boundary
/// shape a serialized blueprint round-trips (the `blueprint_serde_e2e.rs`
/// template). The two SMA knobs are bound when `Some` (byte-identical to the old
/// build-time bind) and left open as `fast.length` / `slow.length` when `None`.
fn stage1_signal(fast_len: Option<i64>, slow_len: Option<i64>) -> Composite {
    let mut g = GraphBuilder::new("stage1_signal");
    let mut fast_b = Sma::builder().named("fast");
    if let Some(l) = fast_len {
        fast_b = fast_b.bind("length", Scalar::i64(l));
    }
    let fast = g.add(fast_b);
    let mut slow_b = Sma::builder().named("slow");
    if let Some(l) = slow_len {
        slow_b = slow_b.bind("length", Scalar::i64(l));
    }
    let slow = g.add(slow_b);
    let spread = g.add(Sub::builder());
    let exposure = g.add(Bias::builder().named("bias").bind("scale", Scalar::f64(0.5)));
    let price = g.source_role("price", ScalarKind::F64);
    g.feed(price, vec![fast.input("series"), slow.input("series")]);
    g.connect(fast.output("value"), spread.input("lhs"));
    g.connect(slow.output("value"), spread.input("rhs"));
    g.connect(spread.output("value"), exposure.input("signal"));
    g.expose(exposure.output("bias"), "bias");
    g.build().expect("stage1 signal wiring resolves")
}

NOTE: confirm the exact expose method name/signature against GraphBuilder (the recon names g.expose(handle, "name"); the blueprint_serde_e2e.rs signal() fixture at :49-70 is the authoritative template — match its role + single-output-expose shape exactly).

• Step 2: Rename stage1_r_graph → wrap_stage1r taking a signal — the body keeps everything EXCEPT the signal nodes; it nests the passed signal and rewires the boundary. The signature drops fast_len/slow_len:

#[allow(clippy::type_complexity, clippy::too_many_arguments)]
fn wrap_stage1r(
    signal: Composite,
    tx_eq: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
    tx_ex: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
    tx_r: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
    tx_req: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
    stop_open: bool,
    reduce: bool,
    cost: Option<(CostConfig, mpsc::Sender<(Timestamp, Vec<Scalar>)>, mpsc::Sender<(Timestamp, Vec<Scalar>)>)>,
) -> Composite {
    let mut g = GraphBuilder::new("stage1_r");
    let sig = g.add(BlueprintNode::Composite(signal));   // nest the signal
    // ... broker / eq / ex / exec / rrec / vol_proxy / cost — VERBATIM from the
    //     current stage1_r_graph body (main.rs:2656-2788) ...
}

The boundary rewiring (replacing the old internal fast/slow/exposure references):

• the price feed (:2703-2713) feeds the signal's role instead of fast/slow: price_targets becomes vec![sig.input("price"), broker.input("price"), exec.input("price")] (+ the vol_proxy vhi/vlo pushes, unchanged);

• drop the now-internal fast/slow/spread connects (:2714-2716 — they live inside stage1_signal now);

• every exposure.output("bias") (:2717-2719) becomes sig.output("bias"): broker.input("exposure"), ex.input("col[0]"), exec.input("bias").

• everything from :2720 down (broker.equity, the PM record loop, the !reduce r_equity + cost block) is VERBATIM.

• Step 3: Reinstate stage1_r_graph as a thin wrapper so all callers and the cost/sweep paths are untouched:

#[allow(clippy::type_complexity, clippy::too_many_arguments)]
fn stage1_r_graph(
    tx_eq: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
    tx_ex: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
    tx_r: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
    tx_req: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
    fast_len: Option<i64>,
    slow_len: Option<i64>,
    stop_open: bool,
    reduce: bool,
    cost: Option<(CostConfig, mpsc::Sender<(Timestamp, Vec<Scalar>)>, mpsc::Sender<(Timestamp, Vec<Scalar>)>)>,
) -> Composite {
    wrap_stage1r(stage1_signal(fast_len, slow_len), tx_eq, tx_ex, tx_r, tx_req, stop_open, reduce, cost)
}

(BlueprintNode must be imported in main.rs; the recon notes Composite, GraphBuilder are already imported.)

• Step 4: Verify behaviour-preservation (THE GATE).

Run: cargo test -p aura-cli --test cli_run Expected: PASS — every stage1-r / sweep cli test unchanged.

Run: cargo test -p aura-cli run_stage1_r_synthetic_folds_an_r_block Expected: PASS — the structural stage1-r metrics test unchanged.

Run: cargo test -p aura-engine --test blueprint_serde_e2e Expected: PASS — the bit-identical round-trip template green.

Run: cargo test --workspace Expected: PASS — no behaviour change anywhere. If any stage1-r metric or trace test changes, the restructure altered the graph: STOP, this is not behaviour-preserving (route to debug).

---

Task 2: Add RunManifest.topology_hash (Tier-1 optional) + thread None into every literal

Files:

• Modify: crates/aura-engine/src/report.rs:53 (add the field)

• Modify: 24 RunManifest { … } literal sites across the workspace (add topology_hash: None)

• Step 1: Add the field after instrument (report.rs:53):

    /// SHA256 (hex) of the canonical `blueprint_to_json` of the run's signal
    /// topology — the #158 reproducibility anchor. `None` for a run not built
    /// from a serialized blueprint and for every pre-0092 line. One-directional
    /// serde widening (Tier-1, #156), identical idiom to `selection` / `instrument`.
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub topology_hash: Option<String>,

• Step 2: Build the workspace to enumerate the broken literal sites

Run: cargo build --workspace --all-targets 2>&1 | grep "missing field" Expected: ~24 missing field topology_hash in initializer of RunManifest errors. This compiler enumeration is authoritative (the recon's 24-site list is advisory).

• Step 3: Add topology_hash: None to every flagged literal. The recon-enumerated sites (confirm against the build): main.rs:172 (sim_optimal_manifest), blueprint.rs:997, mc.rs:239,264, report.rs:273,290,299,316,446,550,580,602, sweep.rs:664, walkforward.rs:288, tests/random_sweep_e2e.rs:112, the four aura-ingest examples/tests, aura-registry compat.rs:71 / lib.rs:546,886 / trace_store.rs:250. Each adds topology_hash: None alongside the existing selection/instrument defaults.

• Step 4: Verify the field is Tier-1 (byte-identical when None)

Run: cargo test -p aura-engine --lib report Expected: PASS — to_json_renders_the_canonical_form, runmanifest_without_instrument_field_deserialises_to_none, runreport_serde_round_trips all green: a None topology_hash omits its key, so every existing JSON pin is byte-unchanged (#156 Tier-1).

Run: cargo build --workspace --all-targets Expected: 0 errors.

---

Task 3: The shared run seam run_signal_stage1r + the sha256_hex helper + sha2

Files:

• Modify: crates/aura-cli/Cargo.toml (add sha2)

• Modify: crates/aura-cli/src/main.rs (imports + the helper + the seam; rewire run_stage1_r)

• Step 1: Add sha2 to crates/aura-cli/Cargo.toml [dependencies] (per-case C16 review: a vetted RustCrypto crate, SHA256 user-settled; lives in the research-side CLI, not the frozen engine):

sha2 = "0.10"

• Step 2: The hash helper + imports. Add to main.rs use aura_engine::{… , blueprint_from_json, blueprint_to_json, BlueprintNode}; and use aura_std::{… , std_vocabulary};, then:

/// SHA256 (hex) of the canonical (#164, no-trailing-newline) serialization of a
/// signal blueprint — the run's `topology_hash` (#158). Research-side (aura-cli),
/// off the frozen engine (invariant 8).
fn topology_hash(signal: &Composite) -> String {
    use sha2::{Digest, Sha256};
    let canonical = blueprint_to_json(signal).expect("a buildable signal serializes");
    let digest = Sha256::digest(canonical.as_bytes());
    digest.iter().map(|b| format!("{b:02x}")).collect()
}

• Step 3: The shared run seam. The basic (no-cost) run path that both the bit-identical test and aura run use. manifest.params derives from the loaded signal's open-param names (param_space) zipped with the bound values:

/// Run a signal blueprint through the Stage-1-R scaffolding: hash the signal,
/// wrap it (broker + equity/exposure/R sinks), compile with `params`, bootstrap,
/// run over `data`, and build the RunReport (manifest carries topology_hash).
/// The single construction+run path shared by `aura run` and its bit-identical
/// test.
fn run_signal_stage1r(signal: Composite, params: &[Scalar], data: RunData, seed: u64) -> RunReport {
    let topo = topology_hash(&signal);                          // before signal is consumed
    let names: Vec<String> = signal.param_space().iter().map(|p| p.name.clone()).collect(); // confirm ParamSpec shape
    let (tx_eq, rx_eq) = mpsc::channel();
    let (tx_ex, rx_ex) = mpsc::channel();
    let (tx_r, rx_r) = mpsc::channel();
    let (tx_req, rx_req) = mpsc::channel();
    let wrapped = wrap_stage1r(signal, tx_eq, tx_ex, tx_r, tx_req, false, false, None);
    let flat = wrapped.compile_with_params(params).expect("signal binds + wraps to a valid harness");
    let mut h = Harness::bootstrap(flat).expect("valid stage1-r harness");
    let (sources, window, pip_size): (Vec<Box<dyn aura_engine::Source>>, _, f64) = match &data {
        RunData::Synthetic => {
            let sources: Vec<Box<dyn aura_engine::Source>> = vec![Box::new(VecSource::new(stage1_r_prices()))];
            let window = window_of(&sources).expect("non-empty synthetic stream");
            (sources, window, SYNTHETIC_PIP_SIZE)
        }
        RunData::Real { symbol, from, to } => {
            let (source, window, pip_size) = open_real_source(symbol, *from, *to);
            (vec![source], window, pip_size)
        }
    };
    h.run(sources);
    let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
    let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
    let r_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_r.try_iter().collect();
    let _req_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_req.try_iter().collect();
    let named_params: Vec<(String, Scalar)> = names.into_iter().zip(params.iter().copied()).collect();
    let mut manifest = sim_optimal_manifest(named_params, window, seed, pip_size);
    manifest.broker = stage1_r_broker_label(pip_size);
    manifest.topology_hash = Some(topo);
    let mut metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0));
    metrics.r = Some(summarize_r(&r_rows, &[]));
    RunReport { manifest, metrics }
}

NOTE (confirm at implement time, all recon-flagged): Composite::param_space() return type + the field exposing the param name (ParamSpec.name or similar) — match the existing compile_with_params arity-check source (blueprint.rs:258); Scalar is Copy (the .copied()); summarize / summarize_r / f64_field / sim_optimal_manifest / stage1_r_broker_label signatures as in run_stage1_r.

• Step 4: Give run_stage1_r its topology_hash via the shared signal. In run_stage1_r (main.rs:3035 + :3076), the manifest must gain the hash of its signal (behaviour-preserving on metrics/traces; the manifest GAINS the key by design, Q4). Minimal edit — after manifest.broker = … (:3076):

    manifest.topology_hash = Some(topology_hash(&stage1_signal(Some(2), Some(4))));

• Step 5: Verify

Run: cargo test -p aura-cli Expected: PASS — run_stage1_r_synthetic_folds_an_r_block green (metrics unchanged); the manifest now carries topology_hash (no exact-JSON pin on the stage1-r manifest breaks, recon-verified).

---

Task 4: The aura run <blueprint.json> CLI arm

Files:

• Modify: crates/aura-cli/src/main.rs:3117-3310 (HarnessKind, parse_run_args, run_dispatch, the ["run", …] arm)

• Step 1: Add the blueprint arm BEFORE the harness-kind parse. In the run dispatch, if the first positional argument is an existing .json file, take the blueprint path; otherwise the existing HarnessKind path is untouched (#159 stays). Parse new flags --params <json>, --seed <n>, --pip-size <n> (the --real/--from/--to data flags already exist on the run path). --params parses a JSON array of typed Scalar cells ([{"I64":2},{"F64":0.5}], the #155 representation):

// in the run-subcommand dispatch, before parse_run_args(rest):
if let Some(path) = rest.first().filter(|a| a.ends_with(".json") && std::path::Path::new(a).is_file()) {
    let doc = std::fs::read_to_string(path).unwrap_or_else(|e| { eprintln!("aura: {path}: {e}"); std::process::exit(2); });
    let signal = blueprint_from_json(&doc, &|t| std_vocabulary(t)).unwrap_or_else(|e| { eprintln!("aura: {path}: {e:?}"); std::process::exit(2); });
    let params = parse_param_cells(/* --params arg or [] */);
    let data = /* --real <SYM> [--from --to] → RunData::Real, else RunData::Synthetic */;
    let seed = /* --seed or 0 */;
    let report = run_signal_stage1r(signal, &params, data, seed);
    println!("{}", report.to_json());
    return;
}

Thread the data/seed parsing through the existing parse_run_args helpers where they already exist (--real/--from/--to); add a small parse_param_cells(&str) -> Vec<Scalar> parsing the JSON cell array via serde_json into Vec<Scalar> (Scalar already derives Deserialize — confirm).

• Step 2: Verify it builds + the existing run dispatch is intact

Run: cargo build -p aura-cli Expected: 0 errors.

Run: cargo test -p aura-cli parse_run_args Expected: PASS — the existing harness-kind arg parsing unchanged.

---

Task 5: Demo blueprint + the bit-identical / E2E tests

Files:

• Create: crates/aura-cli/tests/fixtures/stage1_signal.json — the demo signal blueprint

• Test: crates/aura-cli/src/main.rs (unit: bit-identical + topology_hash determinism)

• Test: crates/aura-cli/tests/cli_run.rs (E2E: aura run <demo> + unknown-type)

• Step 1: Generate the demo signal blueprint from the canonical signal so it is guaranteed loadable:

Run: cargo run -p aura-cli -- run --emit-signal > /dev/null is NOT available; instead generate it in a one-off test or via a tiny helper that writes blueprint_to_json(&stage1_signal(Some(2), Some(4))) to the fixture path. Acceptable: add a #[test] fn write_demo_signal_fixture() (ignored by default, #[ignore]) that emits it, OR commit the JSON produced by blueprint_to_json(&stage1_signal(Some(2),Some(4))). The fixture content is the format_version:1 signal blueprint (4 nodes: fast/slow SMA, Sub, Bias; one price role; one bias output).

• Step 2: The bit-identical test (unit, in main.rs test module):

#[test]
fn loaded_signal_runs_bit_identical_to_rust_built() {
    let rust = stage1_signal(Some(2), Some(4));
    let json = blueprint_to_json(&rust).expect("serializes");
    let loaded = blueprint_from_json(&json, &|t| std_vocabulary(t)).expect("loads");
    let report_a = run_signal_stage1r(stage1_signal(Some(2), Some(4)), &[], RunData::Synthetic, 0);
    let report_b = run_signal_stage1r(loaded, &[], RunData::Synthetic, 0);
    assert_eq!(report_a.to_json(), report_b.to_json(), "loaded run is bit-identical incl. topology_hash");
    assert!(report_a.manifest.topology_hash.as_ref().expect("hash present").len() == 64, "64-hex sha256");
}

• Step 3: topology_hash determinism test:

#[test]
fn topology_hash_is_stable_and_distinguishes() {
    let h1 = topology_hash(&stage1_signal(Some(2), Some(4)));
    let h2 = topology_hash(&stage1_signal(Some(2), Some(4)));
    assert_eq!(h1, h2, "same signal -> same hash");
    assert_ne!(h1, topology_hash(&stage1_signal(Some(3), Some(4))), "different topology -> different hash");
}

• Step 4: E2E in crates/aura-cli/tests/cli_run.rs (the binary-spawn pattern already in that file): aura run tests/fixtures/stage1_signal.json exits 0 and prints a RunReport JSON containing "topology_hash":" + 64 hex chars; an unknown-type blueprint (a fixture referencing "type":"Nope") exits non-zero with an UnknownNodeType-derived message.

• Step 5: Full verification

Run: cargo test -p aura-cli loaded_signal_runs_bit_identical_to_rust_built Expected: PASS.

Run: cargo test --workspace Expected: PASS — full suite green.

Run: cargo clippy --workspace --all-targets -- -D warnings Expected: clean.