diff --git a/docs/plans/0058-family-trace-persistence.md b/docs/plans/0058-family-trace-persistence.md new file mode 100644 index 0000000..d71ddbd --- /dev/null +++ b/docs/plans/0058-family-trace-persistence.md @@ -0,0 +1,558 @@ +# Family-member trace persistence — Implementation Plan + +> **Parent spec:** `docs/specs/0058-family-trace-persistence.md` +> +> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run +> this plan. Steps use `- [ ]` checkboxes for tracking. + +**Goal:** Persist each family member's equity/exposure streams to disk (opt-in +`--trace` on `aura sweep|mc|walkforward`), so any single member is chartable +today and the C21 family-comparison view has per-member data. + +**Architecture:** Persist *inside* each per-member closure (aura-cli), reusing +`persist_traces` verbatim — the engine and the family types are untouched. A new +`--trace ` opt-in gates persistence (without it, byte-unchanged). Each +member is a standalone run-dir at `runs/traces///`; keys are +content-derived and deterministic (MC `seed{N}`, sweep `f{fast}s{slow}`, +walk-forward `oos{ns}`) because the engine HOFs are `Fn + Sync` (members run in +parallel — a runtime counter would be non-deterministic, a C1 break). + +**Tech Stack:** `crates/aura-cli/src/main.rs` (the only production file) + +`crates/aura-cli/tests/cli_run.rs` (integration tests, spawn-with-temp-cwd). + +--- + +**Files this plan creates or modifies:** + +- Modify: `crates/aura-cli/src/main.rs` — arg dispatch + `USAGE` (`--trace` + forms); `run_sweep`/`run_mc`/`run_walkforward` gain `persist: bool`; + `sweep_family`/`mc_family`/`walkforward_family`/`run_oos` gain + `trace: Option<&str>` and persist per member when set; a new + `sweep_member_key` helper; `#[cfg(test)]` caller fixups. +- Test: `crates/aura-cli/tests/cli_run.rs` — three new spawn tests (one per + family kind) asserting per-member dirs round-trip as columnar SoA and that a + plain (no-`--trace`) run writes no trace tree. + +Each task is self-contained: it changes one family's builder signature **and +every caller of that builder** (dispatch arm, `run_*`, the `#[cfg(test)]` report +helper, and the in-`main.rs` integration test at `:1144-1156`), so the workspace +compiles green at the end of every task. The three families are independent +(distinct callers), so Task order is MC → sweep → walk-forward (MC is the spec's +worked template). + +--- + +### Task 1: Monte-Carlo member-trace persistence (the template) + +**Files:** +- Modify: `crates/aura-cli/src/main.rs` — `persist_traces` neighbourhood (new helper not needed for MC), `run_mc` (`:721`), `mc_family` (`:678-702`), dispatch arms (`:957-958`), `USAGE` (`:929`), `mc_report` caller (`:747`), integration test (`:1148`) +- Test: `crates/aura-cli/tests/cli_run.rs` + +- [ ] **Step 1: Write the failing test** + +Append to `crates/aura-cli/tests/cli_run.rs` (uses the existing `BIN`, +`temp_cwd`, `json_array_body`): + +```rust +/// Property: `aura mc --trace ` persists one standalone, round-tripping +/// run-dir per draw seed (the built-in MC family draws seeds 1,2,3), and a plain +/// `aura mc` (no --trace) writes no trace tree (opt-in; byte-unchanged path). +#[test] +fn mc_trace_persists_a_member_dir_per_seed() { + let cwd = temp_cwd("mc-trace"); + + // opt-in OFF: plain `aura mc` persists no per-member trace dirs. + let plain = Command::new(BIN).arg("mc").current_dir(&cwd).output().expect("spawn aura mc"); + assert!(plain.status.success(), "plain mc exit: {:?}", plain.status); + assert!(!cwd.join("runs/traces").exists(), "plain mc must write no trace files"); + + // opt-in ON: one standalone run-dir per draw seed. + let traced = Command::new(BIN) + .args(["mc", "--trace", "mc1"]) + .current_dir(&cwd) + .output() + .expect("spawn aura mc --trace"); + assert!(traced.status.success(), "traced mc exit: {:?}", traced.status); + for seed in [1, 2, 3] { + let dir = cwd.join(format!("runs/traces/mc1/seed{seed}")); + assert!(dir.join("index.json").exists(), "seed{seed} index.json missing"); + assert!(dir.join("equity.json").exists(), "seed{seed} equity tap missing"); + assert!(dir.join("exposure.json").exists(), "seed{seed} exposure tap missing"); + // the persisted equity tap is columnar SoA (C7): kind tag + ts/column parity. + let equity = std::fs::read_to_string(dir.join("equity.json")).expect("read equity.json"); + assert!(equity.contains("\"kinds\":[\"F64\"]"), "seed{seed} F64 tag: {equity}"); + let ts_len = json_array_body(&equity, "\"ts\":[").split(',').count(); + let col_len = json_array_body(&equity, "\"columns\":[[").split(',').count(); + assert_eq!(col_len, ts_len, "seed{seed} SoA parity: ts={ts_len} col={col_len}; {equity}"); + } + + let _ = std::fs::remove_dir_all(&cwd); +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test -p aura-cli --test cli_run mc_trace_persists_a_member_dir_per_seed` +Expected: FAIL — `aura mc --trace mc1` is not a recognised arg form yet, so the +binary prints usage to stderr and exits 2; `traced.status.success()` is false +(the first assert in the opt-in-ON block fires). + +- [ ] **Step 3: Write minimal implementation** + +(a) Dispatch arms — `main.rs:957-958` (add the `persist` argument + the new +`--trace` arm): + +```rust +// before +["mc"] => run_mc("mc"), +["mc", "--name", n] => run_mc(n), +// after +["mc"] => run_mc("mc", false), +["mc", "--name", n] => run_mc(n, false), +["mc", "--trace", n] => run_mc(n, true), +``` + +(b) `USAGE` — `main.rs:929`, the `aura mc` alternative: + +``` +// before: ... | aura mc [--name ] | ... +// after: ... | aura mc [--name |--trace ] | ... +``` + +(c) `run_mc` — `main.rs:721` (signature + the `mc_family` call): + +```rust +// before +fn run_mc(name: &str) { + let reg = default_registry(); + let family = mc_family(); +// after +fn run_mc(name: &str, persist: bool) { + let reg = default_registry(); + let family = mc_family(persist.then_some(name)); +``` + +(d) `mc_family` — `main.rs:678-702` (thread `trace`, collect rows once, build the +manifest before folding, persist the `seed{seed}` member when `trace` is set): + +```rust +fn mc_family(trace: Option<&str>) -> McFamily { + let base_point: Vec = Vec::new(); + monte_carlo(&base_point, &[1, 2, 3], |seed, _base| { + let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE); + let spec = SyntheticSpec { start: 1.0, len: 32, step: 1 }; + let sources: Vec> = vec![Box::new(spec.source(seed))]; + let window = window_of(&sources).expect("non-empty synthetic stream"); + h.run(sources); + let eq_rows = rx_eq.try_iter().collect::>(); + let ex_rows = rx_ex.try_iter().collect::>(); + let manifest = sim_optimal_manifest( + vec![ + ("sma_fast".to_string(), Scalar::i64(2)), + ("sma_slow".to_string(), Scalar::i64(4)), + ("exposure_scale".to_string(), Scalar::f64(0.5)), + ], + window, + seed, + SYNTHETIC_PIP_SIZE, + ); + if let Some(name) = trace { + persist_traces(&format!("{name}/seed{seed}"), &manifest, &eq_rows, &ex_rows); + } + let equity = f64_field(&eq_rows, 0); + let exposure = f64_field(&ex_rows, 0); + RunReport { manifest, metrics: summarize(&equity, &exposure) } + }) +} +``` + +(e) `#[cfg(test)]` callers of `mc_family()` — pass `None` (no behaviour change): + +```rust +// main.rs:747 (in `fn mc_report`): mc_family() -> mc_family(None) +// main.rs:1148 (integration test): mc_family() -> mc_family(None) +``` + +- [ ] **Step 4: Run test to verify it passes + workspace gates** + +Run: `cargo test -p aura-cli --test cli_run mc_trace_persists_a_member_dir_per_seed` +Expected: PASS. + +Run: `cargo build --workspace` +Expected: 0 errors (every `mc_family` caller threaded). + +Run: `cargo test -p aura-cli --bin aura mc_report` +Expected: PASS — `mc_report_is_deterministic_and_one_line_per_seed` unchanged +(it calls `mc_family(None)`, byte-identical metrics). + +--- + +### Task 2: Sweep member-trace persistence + +**Files:** +- Modify: `crates/aura-cli/src/main.rs` — new `sweep_member_key` helper (after `persist_traces`, `:175`), `run_sweep` (`:500`), `sweep_family` (`:448-476`), dispatch arms (`:953-954`), `USAGE` (`:929`), `sweep_report` caller (`:483`), integration test (`:1145`) +- Test: `crates/aura-cli/tests/cli_run.rs` + +- [ ] **Step 1: Write the failing test** + +Append to `crates/aura-cli/tests/cli_run.rs`: + +```rust +/// Property: `aura sweep --trace ` persists one standalone, round-tripping +/// run-dir per grid point. The built-in grid is fast∈{2,3} × slow∈{4,5} = 4 +/// points, content-keyed f2s4/f2s5/f3s4/f3s5; a plain `aura sweep` writes nothing. +#[test] +fn sweep_trace_persists_a_member_dir_per_grid_point() { + let cwd = temp_cwd("sweep-trace"); + + let plain = Command::new(BIN).arg("sweep").current_dir(&cwd).output().expect("spawn aura sweep"); + assert!(plain.status.success(), "plain sweep exit: {:?}", plain.status); + assert!(!cwd.join("runs/traces").exists(), "plain sweep must write no trace files"); + + let traced = Command::new(BIN) + .args(["sweep", "--trace", "swp"]) + .current_dir(&cwd) + .output() + .expect("spawn aura sweep --trace"); + assert!(traced.status.success(), "traced sweep exit: {:?}", traced.status); + for key in ["f2s4", "f2s5", "f3s4", "f3s5"] { + let dir = cwd.join(format!("runs/traces/swp/{key}")); + assert!(dir.join("index.json").exists(), "{key} index.json missing"); + assert!(dir.join("equity.json").exists(), "{key} equity tap missing"); + let equity = std::fs::read_to_string(dir.join("equity.json")).expect("read equity.json"); + assert!(equity.contains("\"kinds\":[\"F64\"]"), "{key} F64 tag: {equity}"); + let ts_len = json_array_body(&equity, "\"ts\":[").split(',').count(); + let col_len = json_array_body(&equity, "\"columns\":[[").split(',').count(); + assert_eq!(col_len, ts_len, "{key} SoA parity: ts={ts_len} col={col_len}; {equity}"); + } + + let _ = std::fs::remove_dir_all(&cwd); +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test -p aura-cli --test cli_run sweep_trace_persists_a_member_dir_per_grid_point` +Expected: FAIL — `aura sweep --trace swp` is not a recognised arg form yet +(usage error, exit 2); `traced.status.success()` is false. + +- [ ] **Step 3: Write minimal implementation** + +(a) New helper — add after `persist_traces` (`main.rs:175`). Derives the +deterministic, collision-free key from the two grid axes that vary in the +built-in sweep (the singleton axes do not affect distinctness; the spec leaves +the exact rendering to the plan): + +```rust +/// The content-derived member key for a swept grid point: the two varying axes +/// of the built-in grid (`signals.trend.fast.length`, `signals.trend.slow.length`). +/// Deterministic + collision-free over that grid (distinct points differ in ≥1 +/// of these). `named` is the `zip_params(&space, point)` the manifest already builds. +fn sweep_member_key(named: &[(String, Scalar)]) -> String { + let val = |name: &str| { + named.iter().find(|(n, _)| n.as_str() == name).map(|(_, v)| v.as_i64()).unwrap_or(0) + }; + format!("f{}s{}", val("signals.trend.fast.length"), val("signals.trend.slow.length")) +} +``` + +(b) Dispatch arms — `main.rs:953-954`: + +```rust +// before +["sweep"] => run_sweep("sweep"), +["sweep", "--name", n] => run_sweep(n), +// after +["sweep"] => run_sweep("sweep", false), +["sweep", "--name", n] => run_sweep(n, false), +["sweep", "--trace", n] => run_sweep(n, true), +``` + +(c) `USAGE` — `main.rs:929`, the `aura sweep` alternative: + +``` +// before: ... | aura sweep [--name ] | ... +// after: ... | aura sweep [--name |--trace ] | ... +``` + +(d) `run_sweep` — `main.rs:500` (signature + the `sweep_family` call): + +```rust +// before +fn run_sweep(name: &str) { + let reg = default_registry(); + let family = sweep_family(); +// after +fn run_sweep(name: &str, persist: bool) { + let reg = default_registry(); + let family = sweep_family(persist.then_some(name)); +``` + +(e) `sweep_family` — `main.rs:448-476` (thread `trace`; collect rows once; build +the named params + manifest before folding; persist the `f{fast}s{slow}` member): + +```rust +fn sweep_family(trace: Option<&str>) -> SweepFamily { + let bp = sample_blueprint_with_sinks().0; + let space = bp.param_space(); + bp.axis("signals.trend.fast.length", [2, 3]) + .axis("signals.trend.slow.length", [4, 5]) + .axis("signals.momentum.fast.length", [2]) + .axis("signals.momentum.slow.length", [4]) + .axis("signals.momentum.signal.length", [3]) + .axis("signals.blend.weights[0]", [1.0]) + .axis("signals.blend.weights[1]", [1.0]) + .axis("exposure.scale", [0.5]) + .sweep(|point| { + let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks(); + let mut h = bp + .bootstrap_with_cells(point) + .expect("grid points are kind-checked against param_space"); + let sources: Vec> = + vec![Box::new(VecSource::new(showcase_prices()))]; + let window = window_of(&sources).expect("non-empty showcase stream"); + h.run(sources); + let eq_rows = rx_eq.try_iter().collect::>(); + let ex_rows = rx_ex.try_iter().collect::>(); + let named = zip_params(&space, point); + let key = sweep_member_key(&named); + let manifest = sim_optimal_manifest(named, window, 0, SYNTHETIC_PIP_SIZE); + if let Some(name) = trace { + persist_traces(&format!("{name}/{key}"), &manifest, &eq_rows, &ex_rows); + } + let equity = f64_field(&eq_rows, 0); + let exposure = f64_field(&ex_rows, 0); + RunReport { manifest, metrics: summarize(&equity, &exposure) } + }) + .expect("the built-in named grid matches the sample param-space") +} +``` + +(f) `#[cfg(test)]` callers of `sweep_family()` — pass `None`: + +```rust +// main.rs:483 (in `fn sweep_report`): sweep_family() -> sweep_family(None) +// main.rs:1145 (integration test): sweep_family() -> sweep_family(None) +``` + +- [ ] **Step 4: Run test to verify it passes + workspace gates** + +Run: `cargo test -p aura-cli --test cli_run sweep_trace_persists_a_member_dir_per_grid_point` +Expected: PASS. + +Run: `cargo build --workspace` +Expected: 0 errors (every `sweep_family` caller threaded). + +Run: `cargo test -p aura-cli --bin aura sweep_report` +Expected: PASS — `sweep_report_is_deterministic` and +`sweep_report_renders_four_points_in_odometer_order` unchanged +(`sweep_family(None)`, byte-identical). + +--- + +### Task 3: Walk-forward OOS member-trace persistence + +**Files:** +- Modify: `crates/aura-cli/src/main.rs` — `run_walkforward` (`:522`), `walkforward_family` (`:544-563`), `run_oos` (`:599-616`), dispatch arms (`:955-956`), `USAGE` (`:929`), `walkforward_report` caller (`:662`), integration test (`:1154`) +- Test: `crates/aura-cli/tests/cli_run.rs` + +Note: only the **OOS** member (`run_oos`) is persisted. The in-sample sub-sweeps +inside `sweep_over` (`:567`) are **not** persisted and `sweep_over` is left +untouched (the spec excludes them). + +- [ ] **Step 1: Write the failing test** + +Append to `crates/aura-cli/tests/cli_run.rs` (the OOS key is `oos` with a +deterministic but synthetic timestamp, so the test asserts the member **count** +and that each entry is `oos`-keyed and round-trips, rather than pinning the ns): + +```rust +/// Property: `aura walkforward --trace ` persists one standalone, +/// round-tripping run-dir per OOS window (the built-in roll = 3 windows), each +/// keyed `oos`; a plain `aura walkforward` writes nothing. +#[test] +fn walkforward_trace_persists_a_member_dir_per_oos_window() { + let cwd = temp_cwd("wf-trace"); + + let plain = Command::new(BIN).arg("walkforward").current_dir(&cwd).output().expect("spawn aura walkforward"); + assert!(plain.status.success(), "plain walkforward exit: {:?}", plain.status); + assert!(!cwd.join("runs/traces").exists(), "plain walkforward must write no trace files"); + + let traced = Command::new(BIN) + .args(["walkforward", "--trace", "wf1"]) + .current_dir(&cwd) + .output() + .expect("spawn aura walkforward --trace"); + assert!(traced.status.success(), "traced walkforward exit: {:?}", traced.status); + + let base = cwd.join("runs/traces/wf1"); + let mut members: Vec = std::fs::read_dir(&base) + .expect("read wf1 trace dir") + .map(|e| e.expect("dir entry").file_name().to_string_lossy().into_owned()) + .collect(); + members.sort(); + assert_eq!(members.len(), 3, "built-in roll = 3 OOS windows -> 3 member dirs; got {members:?}"); + for key in &members { + assert!(key.starts_with("oos"), "member key must be oos: {key}"); + let dir = base.join(key); + assert!(dir.join("index.json").exists(), "{key} index.json missing"); + let equity = std::fs::read_to_string(dir.join("equity.json")).expect("read equity.json"); + assert!(equity.contains("\"kinds\":[\"F64\"]"), "{key} F64 tag: {equity}"); + let ts_len = json_array_body(&equity, "\"ts\":[").split(',').count(); + let col_len = json_array_body(&equity, "\"columns\":[[").split(',').count(); + assert_eq!(col_len, ts_len, "{key} SoA parity: ts={ts_len} col={col_len}; {equity}"); + } + + let _ = std::fs::remove_dir_all(&cwd); +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test -p aura-cli --test cli_run walkforward_trace_persists_a_member_dir_per_oos_window` +Expected: FAIL — `aura walkforward --trace wf1` is not a recognised arg form yet +(usage error, exit 2); `traced.status.success()` is false. + +- [ ] **Step 3: Write minimal implementation** + +(a) Dispatch arms — `main.rs:955-956`: + +```rust +// before +["walkforward"] => run_walkforward("walkforward"), +["walkforward", "--name", n] => run_walkforward(n), +// after +["walkforward"] => run_walkforward("walkforward", false), +["walkforward", "--name", n] => run_walkforward(n, false), +["walkforward", "--trace", n] => run_walkforward(n, true), +``` + +(b) `USAGE` — `main.rs:929`, the `aura walkforward` alternative: + +``` +// before: ... | aura walkforward [--name ] | ... +// after: ... | aura walkforward [--name |--trace ] | ... +``` + +(c) `run_walkforward` — `main.rs:522` (signature + the `walkforward_family` call): + +```rust +// before +fn run_walkforward(name: &str) { + let reg = default_registry(); + let result = walkforward_family(); +// after +fn run_walkforward(name: &str, persist: bool) { + let reg = default_registry(); + let result = walkforward_family(persist.then_some(name)); +``` + +(d) `walkforward_family` — `main.rs:544-563` (thread `trace` into the `run_oos` +call; `sweep_over` stays unchanged): + +```rust +// before +fn walkforward_family() -> WalkForwardResult { + ... + walk_forward(roller, space, |w: WindowBounds| { + let is_family = sweep_over(w.is.0, w.is.1); + let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric"); + let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1); +// after +fn walkforward_family(trace: Option<&str>) -> WalkForwardResult { + ... + walk_forward(roller, space, |w: WindowBounds| { + let is_family = sweep_over(w.is.0, w.is.1); + let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric"); + let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1, trace); +``` + +(e) `run_oos` — `main.rs:599-616` (signature gains `trace`; collect rows once; +persist the `oos` member keyed by the OOS start `from.0`): + +```rust +// before +fn run_oos(params: &[Cell], from: Timestamp, to: Timestamp) -> (Vec<(Timestamp, f64)>, RunReport) { + let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks(); + let space = bp.param_space(); + let mut h = bp + .bootstrap_with_cells(params) + .expect("chosen params pre-validated by the in-sample GridSpace::new"); + let sources: Vec> = + vec![Box::new(walkforward_window_source(from, to))]; + let window = window_of(&sources).expect("non-empty out-of-sample window"); + h.run(sources); + let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); + let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); + let report = RunReport { + manifest: sim_optimal_manifest(zip_params(&space, params), window, 0, SYNTHETIC_PIP_SIZE), + metrics: summarize(&equity, &exposure), + }; + (equity, report) +} +// after +fn run_oos(params: &[Cell], from: Timestamp, to: Timestamp, trace: Option<&str>) -> (Vec<(Timestamp, f64)>, RunReport) { + let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks(); + let space = bp.param_space(); + let mut h = bp + .bootstrap_with_cells(params) + .expect("chosen params pre-validated by the in-sample GridSpace::new"); + let sources: Vec> = + vec![Box::new(walkforward_window_source(from, to))]; + let window = window_of(&sources).expect("non-empty out-of-sample window"); + h.run(sources); + let eq_rows = rx_eq.try_iter().collect::>(); + let ex_rows = rx_ex.try_iter().collect::>(); + let manifest = sim_optimal_manifest(zip_params(&space, params), window, 0, SYNTHETIC_PIP_SIZE); + if let Some(name) = trace { + persist_traces(&format!("{name}/oos{}", from.0), &manifest, &eq_rows, &ex_rows); + } + let equity = f64_field(&eq_rows, 0); + let exposure = f64_field(&ex_rows, 0); + let report = RunReport { manifest, metrics: summarize(&equity, &exposure) }; + (equity, report) +} +``` + +(f) `#[cfg(test)]` callers of `walkforward_family()` — pass `None`: + +```rust +// main.rs:662 (in `fn walkforward_report`): walkforward_family() -> walkforward_family(None) +// main.rs:1154 (integration test): walkforward_family() -> walkforward_family(None) +``` + +- [ ] **Step 4: Run test to verify it passes + workspace gates** + +Run: `cargo test -p aura-cli --test cli_run walkforward_trace_persists_a_member_dir_per_oos_window` +Expected: PASS. + +Run: `cargo build --workspace` +Expected: 0 errors (every `walkforward_family` / `run_oos` caller threaded). + +Run: `cargo test -p aura-cli --bin aura walkforward_report` +Expected: PASS — `walkforward_report_is_deterministic` and +`walkforward_report_has_one_oos_line_per_window_plus_summary` unchanged. + +--- + +### Task 4: Full-suite + lint gate + +**Files:** none (verification only). + +- [ ] **Step 1: Whole workspace test suite** + +Run: `cargo test --workspace` +Expected: PASS — the three new `cli_run` tests plus every pre-existing test +(family determinism, single-run trace, chart-viewer guards) green. No existing +stdout-pinning test shifts (the no-`--trace` path is byte-unchanged). + +- [ ] **Step 2: Lint gate** + +Run: `cargo clippy --workspace --all-targets -- -D warnings` +Expected: exit 0, no warnings. + +- [ ] **Step 3: Manual smoke (optional, observable payoff)** + +Run: `cd "$(mktemp -d)" && aura sweep --trace swp >/dev/null && aura chart swp/f2s4 | head -c 64` +Expected: the first bytes of a self-contained uPlot HTML page — a single sweep +member charts with the existing viewer, no new view-side code.