Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

plan: 0092 run-from-blueprint

Browse Source 
Five-task decomposition of #165 (run a serialized signal blueprint):
1. behaviour-preserving restructure of stage1_r_graph into stage1_signal() +
   wrap_stage1r() (the serializable signal boundary + the shared wrapping),
   gated by the stage1-r + bit-identical-e2e tests staying green;
2. RunManifest.topology_hash (Tier-1 optional) + thread None into the ~24
   workspace literal sites (one compile-gate);
3. the shared run_signal_stage1r seam + the sha256_hex helper (sha2 in aura-cli,
   off the frozen engine, invariant 8) + run_stage1_r gains its topology_hash;
4. the `aura run <blueprint.json>` CLI arm (.json discriminator, --params cells),
   leaving the existing harness-kind dispatch intact (#159 stays);
5. demo signal blueprint + the bit-identical / determinism / E2E tests.

Tasks 1-2 (restructure + field-add) run and verify first. Derived grounding
decisions (the restructure target, param_space-derived manifest.params, the Q4
behaviour-preserving semantics) recorded on #165.

refs #165
This commit is contained in:
Brummel
2026-06-30 18:27:43 +02:00
parent 29a519dbf6
commit e25ca15be4
1 changed files with 405 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/0092-run-from-blueprint.md
+405
View File
@@ -0,0 +1,405 @@
# Run a harness from a serialized blueprint — Implementation Plan
> **Parent spec:** `docs/specs/0092-run-from-blueprint.md` (#165)
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
> this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** `aura run <blueprint.json> [--params <json>]` loads a serialized signal
blueprint, wraps it in the Stage-1-R run scaffolding via a shared seam extracted
from `run_stage1_r`, runs it, and emits a `RunReport` whose manifest carries a
`topology_hash` — bit-identical to the Rust-builder path (C1).
**Architecture:** A serialized blueprint is a *signal* (sinks/broker/data out of
the round-trippable set, C24), so `aura run` wraps the loaded signal in the
existing scaffolding. The wrap is shared: `stage1_r_graph` is restructured into
`stage1_signal()` + `wrap_stage1r(signal,…)`, and a shared `run_signal_stage1r`
runs either the Rust-built or the loaded signal — so "bit-identical to the Rust
path" is the same seam over the same signal.
**Tech Stack:** `aura-cli` (the seam + CLI + the `sha2` hash, research-side),
`aura-engine` (`RunManifest.topology_hash` field only — no `sha2` in the frozen
engine, invariant 8).
**Task order / focus:** Tasks 1–2 are the risky/mechanical infrastructure (the
behaviour-preserving restructure + the workspace-wide field add); run them and
verify behaviour-preservation BEFORE Tasks 3–5 (seam + CLI + demo/tests).
---
### Task 1: Restructure `stage1_r_graph` into `stage1_signal()` + `wrap_stage1r()` (behaviour-preserving)
**Files:**
- Modify: `crates/aura-cli/src/main.rs:2630-2790` (`stage1_r_graph`)
This is a behaviour-preserving restructure, gated by the existing tests staying
green — NOT a behaviour change. The signal nodes (`fast`, `slow`, `spread`,
`exposure`) move into a standalone `Composite` with a `price` input-role and a
`bias` output; everything else (broker / sinks / `exec` / cost / `vol_proxy`)
stays in `wrap_stage1r`, which nests the signal and wires price + bias across the
boundary.
- [ ] **Step 1: Add `stage1_signal`** — extract the signal leg (current
`main.rs:2644-2655` + the price feed into `fast`/`slow` from `:2703-2715` + the
spread/exposure wiring) into:
```rust
/// The Stage-1 signal leg as a standalone, serializable Composite: SMA-cross →
/// Bias. Exposes a `price` input-role and a single `bias` output — the boundary
/// shape a serialized blueprint round-trips (the `blueprint_serde_e2e.rs`
/// template). The two SMA knobs are bound when `Some` (byte-identical to the old
/// build-time bind) and left open as `fast.length` / `slow.length` when `None`.
fn stage1_signal(fast_len: Option<i64>, slow_len: Option<i64>) -> Composite {
let mut g = GraphBuilder::new("stage1_signal");
let mut fast_b = Sma::builder().named("fast");
if let Some(l) = fast_len {
fast_b = fast_b.bind("length", Scalar::i64(l));
}
let fast = g.add(fast_b);
let mut slow_b = Sma::builder().named("slow");
if let Some(l) = slow_len {
slow_b = slow_b.bind("length", Scalar::i64(l));
}
let slow = g.add(slow_b);
let spread = g.add(Sub::builder());
let exposure = g.add(Bias::builder().named("bias").bind("scale", Scalar::f64(0.5)));
let price = g.source_role("price", ScalarKind::F64);
g.feed(price, vec![fast.input("series"), slow.input("series")]);
g.connect(fast.output("value"), spread.input("lhs"));
g.connect(slow.output("value"), spread.input("rhs"));
g.connect(spread.output("value"), exposure.input("signal"));
g.expose(exposure.output("bias"), "bias");
g.build().expect("stage1 signal wiring resolves")
}
```
NOTE: confirm the exact `expose` method name/signature against `GraphBuilder`
(the recon names `g.expose(handle, "name")`; the `blueprint_serde_e2e.rs`
`signal()` fixture at `:49-70` is the authoritative template — match its
role + single-output-expose shape exactly).
- [ ] **Step 2: Rename `stage1_r_graph` → `wrap_stage1r` taking a `signal`** —
the body keeps everything EXCEPT the signal nodes; it nests the passed `signal`
and rewires the boundary. The signature drops `fast_len`/`slow_len`:
```rust
#[allow(clippy::type_complexity, clippy::too_many_arguments)]
fn wrap_stage1r(
signal: Composite,
tx_eq: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
tx_ex: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
tx_r: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
tx_req: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
stop_open: bool,
reduce: bool,
cost: Option<(CostConfig, mpsc::Sender<(Timestamp, Vec<Scalar>)>, mpsc::Sender<(Timestamp, Vec<Scalar>)>)>,
) -> Composite {
let mut g = GraphBuilder::new("stage1_r");
let sig = g.add(BlueprintNode::Composite(signal)); // nest the signal
// ... broker / eq / ex / exec / rrec / vol_proxy / cost — VERBATIM from the
// current stage1_r_graph body (main.rs:2656-2788) ...
}
```
The boundary rewiring (replacing the old internal `fast`/`slow`/`exposure`
references):
- the price feed (`:2703-2713`) feeds the signal's role instead of `fast`/`slow`:
`price_targets` becomes `vec![sig.input("price"), broker.input("price"), exec.input("price")]`
(+ the `vol_proxy` `vhi`/`vlo` pushes, unchanged);
- drop the now-internal `fast`/`slow`/`spread` `connect`s (`:2714-2716` — they
live inside `stage1_signal` now);
- every `exposure.output("bias")` (`:2717-2719`) becomes `sig.output("bias")`:
`broker.input("exposure")`, `ex.input("col[0]")`, `exec.input("bias")`.
- everything from `:2720` down (broker.equity, the PM record loop, the
`!reduce` r_equity + cost block) is VERBATIM.
- [ ] **Step 3: Reinstate `stage1_r_graph` as a thin wrapper** so all callers and
the cost/sweep paths are untouched:
```rust
#[allow(clippy::type_complexity, clippy::too_many_arguments)]
fn stage1_r_graph(
tx_eq: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
tx_ex: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
tx_r: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
tx_req: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
fast_len: Option<i64>,
slow_len: Option<i64>,
stop_open: bool,
reduce: bool,
cost: Option<(CostConfig, mpsc::Sender<(Timestamp, Vec<Scalar>)>, mpsc::Sender<(Timestamp, Vec<Scalar>)>)>,
) -> Composite {
wrap_stage1r(stage1_signal(fast_len, slow_len), tx_eq, tx_ex, tx_r, tx_req, stop_open, reduce, cost)
}
```
(`BlueprintNode` must be imported in `main.rs`; the recon notes `Composite`,
`GraphBuilder` are already imported.)
- [ ] **Step 4: Verify behaviour-preservation (THE GATE).**
Run: `cargo test -p aura-cli --test cli_run`
Expected: PASS — every stage1-r / sweep cli test unchanged.
Run: `cargo test -p aura-cli run_stage1_r_synthetic_folds_an_r_block`
Expected: PASS — the structural stage1-r metrics test unchanged.
Run: `cargo test -p aura-engine --test blueprint_serde_e2e`
Expected: PASS — the bit-identical round-trip template green.
Run: `cargo test --workspace`
Expected: PASS — no behaviour change anywhere. If any stage1-r metric or trace
test changes, the restructure altered the graph: STOP, this is not
behaviour-preserving (route to `debug`).
---
### Task 2: Add `RunManifest.topology_hash` (Tier-1 optional) + thread `None` into every literal
**Files:**
- Modify: `crates/aura-engine/src/report.rs:53` (add the field)
- Modify: 24 `RunManifest { … }` literal sites across the workspace (add `topology_hash: None`)
- [ ] **Step 1: Add the field** after `instrument` (`report.rs:53`):
```rust
/// SHA256 (hex) of the canonical `blueprint_to_json` of the run's signal
/// topology — the #158 reproducibility anchor. `None` for a run not built
/// from a serialized blueprint and for every pre-0092 line. One-directional
/// serde widening (Tier-1, #156), identical idiom to `selection` / `instrument`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub topology_hash: Option<String>,
```
- [ ] **Step 2: Build the workspace to enumerate the broken literal sites**
Run: `cargo build --workspace --all-targets 2>&1 | grep "missing field"`
Expected: ~24 `missing field topology_hash in initializer of RunManifest` errors.
This compiler enumeration is authoritative (the recon's 24-site list is advisory).
- [ ] **Step 3: Add `topology_hash: None` to every flagged literal.** The
recon-enumerated sites (confirm against the build): `main.rs:172`
(`sim_optimal_manifest`), `blueprint.rs:997`, `mc.rs:239,264`,
`report.rs:273,290,299,316,446,550,580,602`, `sweep.rs:664`,
`walkforward.rs:288`, `tests/random_sweep_e2e.rs:112`, the four `aura-ingest`
examples/tests, `aura-registry` `compat.rs:71` / `lib.rs:546,886` /
`trace_store.rs:250`. Each adds `topology_hash: None` alongside the existing
`selection`/`instrument` defaults.
- [ ] **Step 4: Verify the field is Tier-1 (byte-identical when None)**
Run: `cargo test -p aura-engine --lib report`
Expected: PASS — `to_json_renders_the_canonical_form`,
`runmanifest_without_instrument_field_deserialises_to_none`,
`runreport_serde_round_trips` all green: a `None` `topology_hash` omits its key,
so every existing JSON pin is byte-unchanged (#156 Tier-1).
Run: `cargo build --workspace --all-targets`
Expected: 0 errors.
---
### Task 3: The shared run seam `run_signal_stage1r` + the `sha256_hex` helper + `sha2`
**Files:**
- Modify: `crates/aura-cli/Cargo.toml` (add `sha2`)
- Modify: `crates/aura-cli/src/main.rs` (imports + the helper + the seam; rewire `run_stage1_r`)
- [ ] **Step 1: Add `sha2`** to `crates/aura-cli/Cargo.toml` `[dependencies]`
(per-case C16 review: a vetted RustCrypto crate, SHA256 user-settled; lives in
the research-side CLI, not the frozen engine):
```toml
sha2 = "0.10"
```
- [ ] **Step 2: The hash helper + imports.** Add to `main.rs`
`use aura_engine::{… , blueprint_from_json, blueprint_to_json, BlueprintNode};`
and `use aura_std::{… , std_vocabulary};`, then:
```rust
/// SHA256 (hex) of the canonical (#164, no-trailing-newline) serialization of a
/// signal blueprint — the run's `topology_hash` (#158). Research-side (aura-cli),
/// off the frozen engine (invariant 8).
fn topology_hash(signal: &Composite) -> String {
use sha2::{Digest, Sha256};
let canonical = blueprint_to_json(signal).expect("a buildable signal serializes");
let digest = Sha256::digest(canonical.as_bytes());
digest.iter().map(|b| format!("{b:02x}")).collect()
}
```
- [ ] **Step 3: The shared run seam.** The basic (no-cost) run path that both the
bit-identical test and `aura run` use. `manifest.params` derives from the loaded
signal's open-param names (`param_space`) zipped with the bound values:
```rust
/// Run a signal blueprint through the Stage-1-R scaffolding: hash the signal,
/// wrap it (broker + equity/exposure/R sinks), compile with `params`, bootstrap,
/// run over `data`, and build the RunReport (manifest carries topology_hash).
/// The single construction+run path shared by `aura run` and its bit-identical
/// test.
fn run_signal_stage1r(signal: Composite, params: &[Scalar], data: RunData, seed: u64) -> RunReport {
let topo = topology_hash(&signal); // before signal is consumed
let names: Vec<String> = signal.param_space().iter().map(|p| p.name.clone()).collect(); // confirm ParamSpec shape
let (tx_eq, rx_eq) = mpsc::channel();
let (tx_ex, rx_ex) = mpsc::channel();
let (tx_r, rx_r) = mpsc::channel();
let (tx_req, rx_req) = mpsc::channel();
let wrapped = wrap_stage1r(signal, tx_eq, tx_ex, tx_r, tx_req, false, false, None);
let flat = wrapped.compile_with_params(params).expect("signal binds + wraps to a valid harness");
let mut h = Harness::bootstrap(flat).expect("valid stage1-r harness");
let (sources, window, pip_size): (Vec<Box<dyn aura_engine::Source>>, _, f64) = match &data {
RunData::Synthetic => {
let sources: Vec<Box<dyn aura_engine::Source>> = vec![Box::new(VecSource::new(stage1_r_prices()))];
let window = window_of(&sources).expect("non-empty synthetic stream");
(sources, window, SYNTHETIC_PIP_SIZE)
}
RunData::Real { symbol, from, to } => {
let (source, window, pip_size) = open_real_source(symbol, *from, *to);
(vec![source], window, pip_size)
}
};
h.run(sources);
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
let r_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_r.try_iter().collect();
let _req_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_req.try_iter().collect();
let named_params: Vec<(String, Scalar)> = names.into_iter().zip(params.iter().copied()).collect();
let mut manifest = sim_optimal_manifest(named_params, window, seed, pip_size);
manifest.broker = stage1_r_broker_label(pip_size);
manifest.topology_hash = Some(topo);
let mut metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0));
metrics.r = Some(summarize_r(&r_rows, &[]));
RunReport { manifest, metrics }
}
```
NOTE (confirm at implement time, all recon-flagged): `Composite::param_space()`
return type + the field exposing the param name (`ParamSpec.name` or similar) —
match the existing `compile_with_params` arity-check source (`blueprint.rs:258`);
`Scalar` is `Copy` (the `.copied()`); `summarize` / `summarize_r` /
`f64_field` / `sim_optimal_manifest` / `stage1_r_broker_label` signatures as in
`run_stage1_r`.
- [ ] **Step 4: Give `run_stage1_r` its `topology_hash` via the shared signal.**
In `run_stage1_r` (`main.rs:3035` + `:3076`), the manifest must gain the hash of
its signal (behaviour-preserving on metrics/traces; the manifest GAINS the key by
design, Q4). Minimal edit — after `manifest.broker = …` (`:3076`):
```rust
manifest.topology_hash = Some(topology_hash(&stage1_signal(Some(2), Some(4))));
```
- [ ] **Step 5: Verify**
Run: `cargo test -p aura-cli`
Expected: PASS — `run_stage1_r_synthetic_folds_an_r_block` green (metrics
unchanged); the manifest now carries `topology_hash` (no exact-JSON pin on the
stage1-r manifest breaks, recon-verified).
---
### Task 4: The `aura run <blueprint.json>` CLI arm
**Files:**
- Modify: `crates/aura-cli/src/main.rs:3117-3310` (`HarnessKind`, `parse_run_args`, `run_dispatch`, the `["run", …]` arm)
- [ ] **Step 1: Add the blueprint arm BEFORE the harness-kind parse.** In the
`run` dispatch, if the first positional argument is an existing `.json` file,
take the blueprint path; otherwise the existing `HarnessKind` path is untouched
(#159 stays). Parse new flags `--params <json>`, `--seed <n>`, `--pip-size <n>`
(the `--real`/`--from`/`--to` data flags already exist on the run path). `--params`
parses a JSON array of typed `Scalar` cells (`[{"I64":2},{"F64":0.5}]`, the #155
representation):
```rust
// in the run-subcommand dispatch, before parse_run_args(rest):
if let Some(path) = rest.first().filter(|a| a.ends_with(".json") && std::path::Path::new(a).is_file()) {
let doc = std::fs::read_to_string(path).unwrap_or_else(|e| { eprintln!("aura: {path}: {e}"); std::process::exit(2); });
let signal = blueprint_from_json(&doc, &|t| std_vocabulary(t)).unwrap_or_else(|e| { eprintln!("aura: {path}: {e:?}"); std::process::exit(2); });
let params = parse_param_cells(/* --params arg or [] */);
let data = /* --real <SYM> [--from --to] → RunData::Real, else RunData::Synthetic */;
let seed = /* --seed or 0 */;
let report = run_signal_stage1r(signal, &params, data, seed);
println!("{}", report.to_json());
return;
}
```
Thread the data/seed parsing through the existing `parse_run_args` helpers where
they already exist (`--real`/`--from`/`--to`); add a small `parse_param_cells(&str) ->
Vec<Scalar>` parsing the JSON cell array via `serde_json` into `Vec<Scalar>`
(Scalar already derives Deserialize — confirm).
- [ ] **Step 2: Verify it builds + the existing run dispatch is intact**
Run: `cargo build -p aura-cli`
Expected: 0 errors.
Run: `cargo test -p aura-cli parse_run_args`
Expected: PASS — the existing harness-kind arg parsing unchanged.
---
### Task 5: Demo blueprint + the bit-identical / E2E tests
**Files:**
- Create: `crates/aura-cli/tests/fixtures/stage1_signal.json` — the demo signal blueprint
- Test: `crates/aura-cli/src/main.rs` (unit: bit-identical + topology_hash determinism)
- Test: `crates/aura-cli/tests/cli_run.rs` (E2E: `aura run <demo>` + unknown-type)
- [ ] **Step 1: Generate the demo signal blueprint** from the canonical signal so
it is guaranteed loadable:
Run: `cargo run -p aura-cli -- run --emit-signal > /dev/null` is NOT available;
instead generate it in a one-off test or via a tiny helper that writes
`blueprint_to_json(&stage1_signal(Some(2), Some(4)))` to the fixture path. Acceptable:
add a `#[test] fn write_demo_signal_fixture()` (ignored by default, `#[ignore]`) that
emits it, OR commit the JSON produced by `blueprint_to_json(&stage1_signal(Some(2),Some(4)))`.
The fixture content is the `format_version:1` signal blueprint (4 nodes: fast/slow
SMA, Sub, Bias; one `price` role; one `bias` output).
- [ ] **Step 2: The bit-identical test** (unit, in `main.rs` test module):
```rust
#[test]
fn loaded_signal_runs_bit_identical_to_rust_built() {
let rust = stage1_signal(Some(2), Some(4));
let json = blueprint_to_json(&rust).expect("serializes");
let loaded = blueprint_from_json(&json, &|t| std_vocabulary(t)).expect("loads");
let report_a = run_signal_stage1r(stage1_signal(Some(2), Some(4)), &[], RunData::Synthetic, 0);
let report_b = run_signal_stage1r(loaded, &[], RunData::Synthetic, 0);
assert_eq!(report_a.to_json(), report_b.to_json(), "loaded run is bit-identical incl. topology_hash");
assert!(report_a.manifest.topology_hash.as_ref().expect("hash present").len() == 64, "64-hex sha256");
}
```
- [ ] **Step 3: topology_hash determinism test:**
```rust
#[test]
fn topology_hash_is_stable_and_distinguishes() {
let h1 = topology_hash(&stage1_signal(Some(2), Some(4)));
let h2 = topology_hash(&stage1_signal(Some(2), Some(4)));
assert_eq!(h1, h2, "same signal -> same hash");
assert_ne!(h1, topology_hash(&stage1_signal(Some(3), Some(4))), "different topology -> different hash");
}
```
- [ ] **Step 4: E2E** in `crates/aura-cli/tests/cli_run.rs` (the binary-spawn
pattern already in that file): `aura run tests/fixtures/stage1_signal.json` exits
0 and prints a `RunReport` JSON containing `"topology_hash":"` + 64 hex chars; an
unknown-type blueprint (a fixture referencing `"type":"Nope"`) exits non-zero with
an `UnknownNodeType`-derived message.
- [ ] **Step 5: Full verification**
Run: `cargo test -p aura-cli loaded_signal_runs_bit_identical_to_rust_built`
Expected: PASS.
Run: `cargo test --workspace`
Expected: PASS — full suite green.
Run: `cargo clippy --workspace --all-targets -- -D warnings`
Expected: clean.