From 1d6fafbc395e77e91466f578091006e337577530 Mon Sep 17 00:00:00 2001 From: Brummel Date: Sat, 20 Jun 2026 20:01:13 +0200 Subject: [PATCH] =?UTF-8?q?audit(family-traces):=20cycle=20close=200058=20?= =?UTF-8?q?=E2=80=94=20drift-clean,=20retire=20spec/plan?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cycle-close tidy for family-member trace persistence (spec/plan 0058, reference issue #104, implementation d3cb5f8). Architect drift review (range 06fdafa..d3cb5f8): drift_found, low-grade only — no contract violated. C1 (determinism), C7 (SoA), C8 (sinks), C18/C21 (registry/families) all upheld; the opt-in --trace path is genuinely byte-neutral. Two items, both resolved here: - [high] Ledger gap: the C22/#101 disk-trace note recorded only the single-run runs/traces// layout. This cycle establishes a new, load-bearing on-disk convention (per-member nested standalone run-dirs runs/traces///) that the deferred family-comparison view will build on. FIXED: amended the C22/#101 realization note in docs/design/INDEX.md to record the layout, the content-derived deterministic member keys, and the parallel-safe lock-free write. - [medium] sweep_member_key hard-codes two grid axes; collision-free only over today's built-in grid, a latent foot-gun if the grid varies an unlisted axis. Not a current defect (one grid, covered, fail-loud on a missing listed axis). CARRY-ON: filed #105 (idea) for generalised key derivation when the grid generalises. Regression: the project declares no dedicated regression command; the gate is the architect review plus the full suite, verified green at d3cb5f8 (cargo test --workspace, clippy --all-targets -D warnings). Spec docs/specs/0058 and plan docs/plans/0058 retired (git rm) per the ephemeral active-cycle artefact lifecycle; durable rationale lifted to the ledger amendment above. refs #104 --- docs/design/INDEX.md | 13 + docs/plans/0058-family-trace-persistence.md | 558 -------------------- docs/specs/0058-family-trace-persistence.md | 268 ---------- 3 files changed, 13 insertions(+), 826 deletions(-) delete mode 100644 docs/plans/0058-family-trace-persistence.md delete mode 100644 docs/specs/0058-family-trace-persistence.md diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index 2b4fb71..5949257 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -1025,6 +1025,19 @@ is persisted on disk in `index.json` but deliberately not surfaced in this naive viewer — a ratified scope call, follow-up filed). Families-comparison meta-views, a local server, and replay-clock controls remain open (see "Open architectural threads"). +**Amendment (family-member traces, #104, d3cb5f8).** The disk-trace layout extends +to family runs: `aura sweep|mc|walkforward --trace ` persists *each member* as +a nested standalone run-dir `runs/traces///`, reusing +`persist_traces`/`TraceStore` verbatim (engine untouched — persistence is a +side-effect inside each per-member closure). The `member_key` is content-derived and +deterministic — sweep `f{fast}s{slow}`, MC `seed{N}`, walk-forward `oos{ns}` — never +a runtime ordinal, because members run in parallel under the engine's `Fn + Sync` +HOFs, where a counter would be schedule-dependent (C1); concurrent `TraceStore::write` +targets disjoint member dirs, so it is lock-free. Opt-in: without `--trace`, +stdout/registry are byte-unchanged. Because `TraceStore` resolves `/` +as a subpath, `aura chart /` charts any single member with no +view-side change — the write-side precondition for the family-comparison **view** +(overlay / small-multiples across members), which itself remains open. ### C23 — Graph compilation and behaviour-preserving optimisation **Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic, diff --git a/docs/plans/0058-family-trace-persistence.md b/docs/plans/0058-family-trace-persistence.md deleted file mode 100644 index d71ddbd..0000000 --- a/docs/plans/0058-family-trace-persistence.md +++ /dev/null @@ -1,558 +0,0 @@ -# Family-member trace persistence — Implementation Plan - -> **Parent spec:** `docs/specs/0058-family-trace-persistence.md` -> -> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run -> this plan. Steps use `- [ ]` checkboxes for tracking. - -**Goal:** Persist each family member's equity/exposure streams to disk (opt-in -`--trace` on `aura sweep|mc|walkforward`), so any single member is chartable -today and the C21 family-comparison view has per-member data. - -**Architecture:** Persist *inside* each per-member closure (aura-cli), reusing -`persist_traces` verbatim — the engine and the family types are untouched. A new -`--trace ` opt-in gates persistence (without it, byte-unchanged). Each -member is a standalone run-dir at `runs/traces///`; keys are -content-derived and deterministic (MC `seed{N}`, sweep `f{fast}s{slow}`, -walk-forward `oos{ns}`) because the engine HOFs are `Fn + Sync` (members run in -parallel — a runtime counter would be non-deterministic, a C1 break). - -**Tech Stack:** `crates/aura-cli/src/main.rs` (the only production file) + -`crates/aura-cli/tests/cli_run.rs` (integration tests, spawn-with-temp-cwd). - ---- - -**Files this plan creates or modifies:** - -- Modify: `crates/aura-cli/src/main.rs` — arg dispatch + `USAGE` (`--trace` - forms); `run_sweep`/`run_mc`/`run_walkforward` gain `persist: bool`; - `sweep_family`/`mc_family`/`walkforward_family`/`run_oos` gain - `trace: Option<&str>` and persist per member when set; a new - `sweep_member_key` helper; `#[cfg(test)]` caller fixups. -- Test: `crates/aura-cli/tests/cli_run.rs` — three new spawn tests (one per - family kind) asserting per-member dirs round-trip as columnar SoA and that a - plain (no-`--trace`) run writes no trace tree. - -Each task is self-contained: it changes one family's builder signature **and -every caller of that builder** (dispatch arm, `run_*`, the `#[cfg(test)]` report -helper, and the in-`main.rs` integration test at `:1144-1156`), so the workspace -compiles green at the end of every task. The three families are independent -(distinct callers), so Task order is MC → sweep → walk-forward (MC is the spec's -worked template). - ---- - -### Task 1: Monte-Carlo member-trace persistence (the template) - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` — `persist_traces` neighbourhood (new helper not needed for MC), `run_mc` (`:721`), `mc_family` (`:678-702`), dispatch arms (`:957-958`), `USAGE` (`:929`), `mc_report` caller (`:747`), integration test (`:1148`) -- Test: `crates/aura-cli/tests/cli_run.rs` - -- [ ] **Step 1: Write the failing test** - -Append to `crates/aura-cli/tests/cli_run.rs` (uses the existing `BIN`, -`temp_cwd`, `json_array_body`): - -```rust -/// Property: `aura mc --trace ` persists one standalone, round-tripping -/// run-dir per draw seed (the built-in MC family draws seeds 1,2,3), and a plain -/// `aura mc` (no --trace) writes no trace tree (opt-in; byte-unchanged path). -#[test] -fn mc_trace_persists_a_member_dir_per_seed() { - let cwd = temp_cwd("mc-trace"); - - // opt-in OFF: plain `aura mc` persists no per-member trace dirs. - let plain = Command::new(BIN).arg("mc").current_dir(&cwd).output().expect("spawn aura mc"); - assert!(plain.status.success(), "plain mc exit: {:?}", plain.status); - assert!(!cwd.join("runs/traces").exists(), "plain mc must write no trace files"); - - // opt-in ON: one standalone run-dir per draw seed. - let traced = Command::new(BIN) - .args(["mc", "--trace", "mc1"]) - .current_dir(&cwd) - .output() - .expect("spawn aura mc --trace"); - assert!(traced.status.success(), "traced mc exit: {:?}", traced.status); - for seed in [1, 2, 3] { - let dir = cwd.join(format!("runs/traces/mc1/seed{seed}")); - assert!(dir.join("index.json").exists(), "seed{seed} index.json missing"); - assert!(dir.join("equity.json").exists(), "seed{seed} equity tap missing"); - assert!(dir.join("exposure.json").exists(), "seed{seed} exposure tap missing"); - // the persisted equity tap is columnar SoA (C7): kind tag + ts/column parity. - let equity = std::fs::read_to_string(dir.join("equity.json")).expect("read equity.json"); - assert!(equity.contains("\"kinds\":[\"F64\"]"), "seed{seed} F64 tag: {equity}"); - let ts_len = json_array_body(&equity, "\"ts\":[").split(',').count(); - let col_len = json_array_body(&equity, "\"columns\":[[").split(',').count(); - assert_eq!(col_len, ts_len, "seed{seed} SoA parity: ts={ts_len} col={col_len}; {equity}"); - } - - let _ = std::fs::remove_dir_all(&cwd); -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cargo test -p aura-cli --test cli_run mc_trace_persists_a_member_dir_per_seed` -Expected: FAIL — `aura mc --trace mc1` is not a recognised arg form yet, so the -binary prints usage to stderr and exits 2; `traced.status.success()` is false -(the first assert in the opt-in-ON block fires). - -- [ ] **Step 3: Write minimal implementation** - -(a) Dispatch arms — `main.rs:957-958` (add the `persist` argument + the new -`--trace` arm): - -```rust -// before -["mc"] => run_mc("mc"), -["mc", "--name", n] => run_mc(n), -// after -["mc"] => run_mc("mc", false), -["mc", "--name", n] => run_mc(n, false), -["mc", "--trace", n] => run_mc(n, true), -``` - -(b) `USAGE` — `main.rs:929`, the `aura mc` alternative: - -``` -// before: ... | aura mc [--name ] | ... -// after: ... | aura mc [--name |--trace ] | ... -``` - -(c) `run_mc` — `main.rs:721` (signature + the `mc_family` call): - -```rust -// before -fn run_mc(name: &str) { - let reg = default_registry(); - let family = mc_family(); -// after -fn run_mc(name: &str, persist: bool) { - let reg = default_registry(); - let family = mc_family(persist.then_some(name)); -``` - -(d) `mc_family` — `main.rs:678-702` (thread `trace`, collect rows once, build the -manifest before folding, persist the `seed{seed}` member when `trace` is set): - -```rust -fn mc_family(trace: Option<&str>) -> McFamily { - let base_point: Vec = Vec::new(); - monte_carlo(&base_point, &[1, 2, 3], |seed, _base| { - let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE); - let spec = SyntheticSpec { start: 1.0, len: 32, step: 1 }; - let sources: Vec> = vec![Box::new(spec.source(seed))]; - let window = window_of(&sources).expect("non-empty synthetic stream"); - h.run(sources); - let eq_rows = rx_eq.try_iter().collect::>(); - let ex_rows = rx_ex.try_iter().collect::>(); - let manifest = sim_optimal_manifest( - vec![ - ("sma_fast".to_string(), Scalar::i64(2)), - ("sma_slow".to_string(), Scalar::i64(4)), - ("exposure_scale".to_string(), Scalar::f64(0.5)), - ], - window, - seed, - SYNTHETIC_PIP_SIZE, - ); - if let Some(name) = trace { - persist_traces(&format!("{name}/seed{seed}"), &manifest, &eq_rows, &ex_rows); - } - let equity = f64_field(&eq_rows, 0); - let exposure = f64_field(&ex_rows, 0); - RunReport { manifest, metrics: summarize(&equity, &exposure) } - }) -} -``` - -(e) `#[cfg(test)]` callers of `mc_family()` — pass `None` (no behaviour change): - -```rust -// main.rs:747 (in `fn mc_report`): mc_family() -> mc_family(None) -// main.rs:1148 (integration test): mc_family() -> mc_family(None) -``` - -- [ ] **Step 4: Run test to verify it passes + workspace gates** - -Run: `cargo test -p aura-cli --test cli_run mc_trace_persists_a_member_dir_per_seed` -Expected: PASS. - -Run: `cargo build --workspace` -Expected: 0 errors (every `mc_family` caller threaded). - -Run: `cargo test -p aura-cli --bin aura mc_report` -Expected: PASS — `mc_report_is_deterministic_and_one_line_per_seed` unchanged -(it calls `mc_family(None)`, byte-identical metrics). - ---- - -### Task 2: Sweep member-trace persistence - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` — new `sweep_member_key` helper (after `persist_traces`, `:175`), `run_sweep` (`:500`), `sweep_family` (`:448-476`), dispatch arms (`:953-954`), `USAGE` (`:929`), `sweep_report` caller (`:483`), integration test (`:1145`) -- Test: `crates/aura-cli/tests/cli_run.rs` - -- [ ] **Step 1: Write the failing test** - -Append to `crates/aura-cli/tests/cli_run.rs`: - -```rust -/// Property: `aura sweep --trace ` persists one standalone, round-tripping -/// run-dir per grid point. The built-in grid is fast∈{2,3} × slow∈{4,5} = 4 -/// points, content-keyed f2s4/f2s5/f3s4/f3s5; a plain `aura sweep` writes nothing. -#[test] -fn sweep_trace_persists_a_member_dir_per_grid_point() { - let cwd = temp_cwd("sweep-trace"); - - let plain = Command::new(BIN).arg("sweep").current_dir(&cwd).output().expect("spawn aura sweep"); - assert!(plain.status.success(), "plain sweep exit: {:?}", plain.status); - assert!(!cwd.join("runs/traces").exists(), "plain sweep must write no trace files"); - - let traced = Command::new(BIN) - .args(["sweep", "--trace", "swp"]) - .current_dir(&cwd) - .output() - .expect("spawn aura sweep --trace"); - assert!(traced.status.success(), "traced sweep exit: {:?}", traced.status); - for key in ["f2s4", "f2s5", "f3s4", "f3s5"] { - let dir = cwd.join(format!("runs/traces/swp/{key}")); - assert!(dir.join("index.json").exists(), "{key} index.json missing"); - assert!(dir.join("equity.json").exists(), "{key} equity tap missing"); - let equity = std::fs::read_to_string(dir.join("equity.json")).expect("read equity.json"); - assert!(equity.contains("\"kinds\":[\"F64\"]"), "{key} F64 tag: {equity}"); - let ts_len = json_array_body(&equity, "\"ts\":[").split(',').count(); - let col_len = json_array_body(&equity, "\"columns\":[[").split(',').count(); - assert_eq!(col_len, ts_len, "{key} SoA parity: ts={ts_len} col={col_len}; {equity}"); - } - - let _ = std::fs::remove_dir_all(&cwd); -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cargo test -p aura-cli --test cli_run sweep_trace_persists_a_member_dir_per_grid_point` -Expected: FAIL — `aura sweep --trace swp` is not a recognised arg form yet -(usage error, exit 2); `traced.status.success()` is false. - -- [ ] **Step 3: Write minimal implementation** - -(a) New helper — add after `persist_traces` (`main.rs:175`). Derives the -deterministic, collision-free key from the two grid axes that vary in the -built-in sweep (the singleton axes do not affect distinctness; the spec leaves -the exact rendering to the plan): - -```rust -/// The content-derived member key for a swept grid point: the two varying axes -/// of the built-in grid (`signals.trend.fast.length`, `signals.trend.slow.length`). -/// Deterministic + collision-free over that grid (distinct points differ in ≥1 -/// of these). `named` is the `zip_params(&space, point)` the manifest already builds. -fn sweep_member_key(named: &[(String, Scalar)]) -> String { - let val = |name: &str| { - named.iter().find(|(n, _)| n.as_str() == name).map(|(_, v)| v.as_i64()).unwrap_or(0) - }; - format!("f{}s{}", val("signals.trend.fast.length"), val("signals.trend.slow.length")) -} -``` - -(b) Dispatch arms — `main.rs:953-954`: - -```rust -// before -["sweep"] => run_sweep("sweep"), -["sweep", "--name", n] => run_sweep(n), -// after -["sweep"] => run_sweep("sweep", false), -["sweep", "--name", n] => run_sweep(n, false), -["sweep", "--trace", n] => run_sweep(n, true), -``` - -(c) `USAGE` — `main.rs:929`, the `aura sweep` alternative: - -``` -// before: ... | aura sweep [--name ] | ... -// after: ... | aura sweep [--name |--trace ] | ... -``` - -(d) `run_sweep` — `main.rs:500` (signature + the `sweep_family` call): - -```rust -// before -fn run_sweep(name: &str) { - let reg = default_registry(); - let family = sweep_family(); -// after -fn run_sweep(name: &str, persist: bool) { - let reg = default_registry(); - let family = sweep_family(persist.then_some(name)); -``` - -(e) `sweep_family` — `main.rs:448-476` (thread `trace`; collect rows once; build -the named params + manifest before folding; persist the `f{fast}s{slow}` member): - -```rust -fn sweep_family(trace: Option<&str>) -> SweepFamily { - let bp = sample_blueprint_with_sinks().0; - let space = bp.param_space(); - bp.axis("signals.trend.fast.length", [2, 3]) - .axis("signals.trend.slow.length", [4, 5]) - .axis("signals.momentum.fast.length", [2]) - .axis("signals.momentum.slow.length", [4]) - .axis("signals.momentum.signal.length", [3]) - .axis("signals.blend.weights[0]", [1.0]) - .axis("signals.blend.weights[1]", [1.0]) - .axis("exposure.scale", [0.5]) - .sweep(|point| { - let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks(); - let mut h = bp - .bootstrap_with_cells(point) - .expect("grid points are kind-checked against param_space"); - let sources: Vec> = - vec![Box::new(VecSource::new(showcase_prices()))]; - let window = window_of(&sources).expect("non-empty showcase stream"); - h.run(sources); - let eq_rows = rx_eq.try_iter().collect::>(); - let ex_rows = rx_ex.try_iter().collect::>(); - let named = zip_params(&space, point); - let key = sweep_member_key(&named); - let manifest = sim_optimal_manifest(named, window, 0, SYNTHETIC_PIP_SIZE); - if let Some(name) = trace { - persist_traces(&format!("{name}/{key}"), &manifest, &eq_rows, &ex_rows); - } - let equity = f64_field(&eq_rows, 0); - let exposure = f64_field(&ex_rows, 0); - RunReport { manifest, metrics: summarize(&equity, &exposure) } - }) - .expect("the built-in named grid matches the sample param-space") -} -``` - -(f) `#[cfg(test)]` callers of `sweep_family()` — pass `None`: - -```rust -// main.rs:483 (in `fn sweep_report`): sweep_family() -> sweep_family(None) -// main.rs:1145 (integration test): sweep_family() -> sweep_family(None) -``` - -- [ ] **Step 4: Run test to verify it passes + workspace gates** - -Run: `cargo test -p aura-cli --test cli_run sweep_trace_persists_a_member_dir_per_grid_point` -Expected: PASS. - -Run: `cargo build --workspace` -Expected: 0 errors (every `sweep_family` caller threaded). - -Run: `cargo test -p aura-cli --bin aura sweep_report` -Expected: PASS — `sweep_report_is_deterministic` and -`sweep_report_renders_four_points_in_odometer_order` unchanged -(`sweep_family(None)`, byte-identical). - ---- - -### Task 3: Walk-forward OOS member-trace persistence - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` — `run_walkforward` (`:522`), `walkforward_family` (`:544-563`), `run_oos` (`:599-616`), dispatch arms (`:955-956`), `USAGE` (`:929`), `walkforward_report` caller (`:662`), integration test (`:1154`) -- Test: `crates/aura-cli/tests/cli_run.rs` - -Note: only the **OOS** member (`run_oos`) is persisted. The in-sample sub-sweeps -inside `sweep_over` (`:567`) are **not** persisted and `sweep_over` is left -untouched (the spec excludes them). - -- [ ] **Step 1: Write the failing test** - -Append to `crates/aura-cli/tests/cli_run.rs` (the OOS key is `oos` with a -deterministic but synthetic timestamp, so the test asserts the member **count** -and that each entry is `oos`-keyed and round-trips, rather than pinning the ns): - -```rust -/// Property: `aura walkforward --trace ` persists one standalone, -/// round-tripping run-dir per OOS window (the built-in roll = 3 windows), each -/// keyed `oos`; a plain `aura walkforward` writes nothing. -#[test] -fn walkforward_trace_persists_a_member_dir_per_oos_window() { - let cwd = temp_cwd("wf-trace"); - - let plain = Command::new(BIN).arg("walkforward").current_dir(&cwd).output().expect("spawn aura walkforward"); - assert!(plain.status.success(), "plain walkforward exit: {:?}", plain.status); - assert!(!cwd.join("runs/traces").exists(), "plain walkforward must write no trace files"); - - let traced = Command::new(BIN) - .args(["walkforward", "--trace", "wf1"]) - .current_dir(&cwd) - .output() - .expect("spawn aura walkforward --trace"); - assert!(traced.status.success(), "traced walkforward exit: {:?}", traced.status); - - let base = cwd.join("runs/traces/wf1"); - let mut members: Vec = std::fs::read_dir(&base) - .expect("read wf1 trace dir") - .map(|e| e.expect("dir entry").file_name().to_string_lossy().into_owned()) - .collect(); - members.sort(); - assert_eq!(members.len(), 3, "built-in roll = 3 OOS windows -> 3 member dirs; got {members:?}"); - for key in &members { - assert!(key.starts_with("oos"), "member key must be oos: {key}"); - let dir = base.join(key); - assert!(dir.join("index.json").exists(), "{key} index.json missing"); - let equity = std::fs::read_to_string(dir.join("equity.json")).expect("read equity.json"); - assert!(equity.contains("\"kinds\":[\"F64\"]"), "{key} F64 tag: {equity}"); - let ts_len = json_array_body(&equity, "\"ts\":[").split(',').count(); - let col_len = json_array_body(&equity, "\"columns\":[[").split(',').count(); - assert_eq!(col_len, ts_len, "{key} SoA parity: ts={ts_len} col={col_len}; {equity}"); - } - - let _ = std::fs::remove_dir_all(&cwd); -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cargo test -p aura-cli --test cli_run walkforward_trace_persists_a_member_dir_per_oos_window` -Expected: FAIL — `aura walkforward --trace wf1` is not a recognised arg form yet -(usage error, exit 2); `traced.status.success()` is false. - -- [ ] **Step 3: Write minimal implementation** - -(a) Dispatch arms — `main.rs:955-956`: - -```rust -// before -["walkforward"] => run_walkforward("walkforward"), -["walkforward", "--name", n] => run_walkforward(n), -// after -["walkforward"] => run_walkforward("walkforward", false), -["walkforward", "--name", n] => run_walkforward(n, false), -["walkforward", "--trace", n] => run_walkforward(n, true), -``` - -(b) `USAGE` — `main.rs:929`, the `aura walkforward` alternative: - -``` -// before: ... | aura walkforward [--name ] | ... -// after: ... | aura walkforward [--name |--trace ] | ... -``` - -(c) `run_walkforward` — `main.rs:522` (signature + the `walkforward_family` call): - -```rust -// before -fn run_walkforward(name: &str) { - let reg = default_registry(); - let result = walkforward_family(); -// after -fn run_walkforward(name: &str, persist: bool) { - let reg = default_registry(); - let result = walkforward_family(persist.then_some(name)); -``` - -(d) `walkforward_family` — `main.rs:544-563` (thread `trace` into the `run_oos` -call; `sweep_over` stays unchanged): - -```rust -// before -fn walkforward_family() -> WalkForwardResult { - ... - walk_forward(roller, space, |w: WindowBounds| { - let is_family = sweep_over(w.is.0, w.is.1); - let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric"); - let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1); -// after -fn walkforward_family(trace: Option<&str>) -> WalkForwardResult { - ... - walk_forward(roller, space, |w: WindowBounds| { - let is_family = sweep_over(w.is.0, w.is.1); - let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric"); - let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1, trace); -``` - -(e) `run_oos` — `main.rs:599-616` (signature gains `trace`; collect rows once; -persist the `oos` member keyed by the OOS start `from.0`): - -```rust -// before -fn run_oos(params: &[Cell], from: Timestamp, to: Timestamp) -> (Vec<(Timestamp, f64)>, RunReport) { - let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks(); - let space = bp.param_space(); - let mut h = bp - .bootstrap_with_cells(params) - .expect("chosen params pre-validated by the in-sample GridSpace::new"); - let sources: Vec> = - vec![Box::new(walkforward_window_source(from, to))]; - let window = window_of(&sources).expect("non-empty out-of-sample window"); - h.run(sources); - let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); - let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); - let report = RunReport { - manifest: sim_optimal_manifest(zip_params(&space, params), window, 0, SYNTHETIC_PIP_SIZE), - metrics: summarize(&equity, &exposure), - }; - (equity, report) -} -// after -fn run_oos(params: &[Cell], from: Timestamp, to: Timestamp, trace: Option<&str>) -> (Vec<(Timestamp, f64)>, RunReport) { - let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks(); - let space = bp.param_space(); - let mut h = bp - .bootstrap_with_cells(params) - .expect("chosen params pre-validated by the in-sample GridSpace::new"); - let sources: Vec> = - vec![Box::new(walkforward_window_source(from, to))]; - let window = window_of(&sources).expect("non-empty out-of-sample window"); - h.run(sources); - let eq_rows = rx_eq.try_iter().collect::>(); - let ex_rows = rx_ex.try_iter().collect::>(); - let manifest = sim_optimal_manifest(zip_params(&space, params), window, 0, SYNTHETIC_PIP_SIZE); - if let Some(name) = trace { - persist_traces(&format!("{name}/oos{}", from.0), &manifest, &eq_rows, &ex_rows); - } - let equity = f64_field(&eq_rows, 0); - let exposure = f64_field(&ex_rows, 0); - let report = RunReport { manifest, metrics: summarize(&equity, &exposure) }; - (equity, report) -} -``` - -(f) `#[cfg(test)]` callers of `walkforward_family()` — pass `None`: - -```rust -// main.rs:662 (in `fn walkforward_report`): walkforward_family() -> walkforward_family(None) -// main.rs:1154 (integration test): walkforward_family() -> walkforward_family(None) -``` - -- [ ] **Step 4: Run test to verify it passes + workspace gates** - -Run: `cargo test -p aura-cli --test cli_run walkforward_trace_persists_a_member_dir_per_oos_window` -Expected: PASS. - -Run: `cargo build --workspace` -Expected: 0 errors (every `walkforward_family` / `run_oos` caller threaded). - -Run: `cargo test -p aura-cli --bin aura walkforward_report` -Expected: PASS — `walkforward_report_is_deterministic` and -`walkforward_report_has_one_oos_line_per_window_plus_summary` unchanged. - ---- - -### Task 4: Full-suite + lint gate - -**Files:** none (verification only). - -- [ ] **Step 1: Whole workspace test suite** - -Run: `cargo test --workspace` -Expected: PASS — the three new `cli_run` tests plus every pre-existing test -(family determinism, single-run trace, chart-viewer guards) green. No existing -stdout-pinning test shifts (the no-`--trace` path is byte-unchanged). - -- [ ] **Step 2: Lint gate** - -Run: `cargo clippy --workspace --all-targets -- -D warnings` -Expected: exit 0, no warnings. - -- [ ] **Step 3: Manual smoke (optional, observable payoff)** - -Run: `cd "$(mktemp -d)" && aura sweep --trace swp >/dev/null && aura chart swp/f2s4 | head -c 64` -Expected: the first bytes of a self-contained uPlot HTML page — a single sweep -member charts with the existing viewer, no new view-side code. diff --git a/docs/specs/0058-family-trace-persistence.md b/docs/specs/0058-family-trace-persistence.md deleted file mode 100644 index 0170248..0000000 --- a/docs/specs/0058-family-trace-persistence.md +++ /dev/null @@ -1,268 +0,0 @@ -# Family-member trace persistence — Design Spec - -**Date:** 2026-06-20 -**Status:** Draft — awaiting user spec review -**Authors:** orchestrator + Claude - -Reference issue: Brummel/Aura #104. Fork decisions (A–D) and the member-key -correction are recorded on that issue's thread (the run's decision log). - -## Goal - -Make each family member's recorded streams land on disk, so the C21 -family-comparison view has per-member data to chart and any single member is -chartable *today* with the existing single-run viewer. - -Single backtest runs already do this: `aura run --trace ` drains the -equity + exposure recorders and persists them via `persist_traces` -(`crates/aura-cli/src/main.rs:161`) → `TraceStore` -(`crates/aura-registry/src/trace_store.rs:40`), under `runs/traces//`. -Family runs (`sweep` / `mc` / `walkforward`) do not: each member *does* run a -full harness with both recorders and *is* drained inside its per-member closure, -but the raw rows are immediately folded to `metrics: summarize(...)` and -discarded. The streams the comparison view needs exist transiently and are -thrown away. - -This cycle is the **write side only**. The family-comparison *view* (how to -render N members — overlay / small-multiples / rank-top-N), decimation / LOD, and -a binary trace container are explicitly out of scope (each its own later cycle). - -## Architecture - -Four derived decisions (recorded on #104): - -- **A — persist in the closure; engine untouched.** The three per-member - closures live in aura-cli and already hold the drained rows. When persistence - is enabled they call the existing `persist_traces` directly, before folding to - metrics. The engine's `sweep` / `monte_carlo` / `walk_forward` and the - `SweepFamily` / `McFamily` / `WalkForwardResult` types are **not** changed. -- **B — opt-in.** A new `--trace` opt-in on `aura sweep|mc|walkforward` gates - persistence. Without it, behaviour is byte-unchanged (no member dirs written — - the cardinality cost of N members × full resolution stays user-chosen). -- **C — content-derived deterministic member keys.** The engine higher-order fns - are `F: Fn(...) + Sync` (`sweep.rs:314`, `mc.rs:124`, `walkforward.rs:187`): - members run in parallel across threads (C1 parallelism *across* sims). A - runtime ordinal counter would be non-deterministic (thread-race → C1 - violation). Keys are therefore derived from what each closure receives - deterministically (see the member-key table). -- **D — standalone run-dir per member.** Each member is persisted under - `runs/traces///` as a complete run-dir (its own `index.json` - + `equity.json` + `exposure.json`), by calling `persist_traces` with a - slash-containing name. `TraceStore::write` resolves that to nested directories; - the existing single-run viewer charts any one member as-is, with no view-side - code. - -`--trace ` sets both the family registry name and the trace prefix to -`` (a member registers under the family, and its streams land under -`runs/traces//`). This avoids a redundant second name while keeping -`--trace ` taking an argument, exactly as the single-run form does. - -## Concrete code shapes - -### User-facing surface (the acceptance evidence) - -```console -# opt in: register the family under "swp" AND persist each member's streams -$ aura sweep --trace swp -{"family_id":"swp-1","report":{...}} # stdout unchanged from `aura sweep --name swp` -... one line per grid point ... - -$ tree runs/traces/swp -runs/traces/swp -├── f2s4/ # one standalone run-dir per grid point (content-keyed) -│ ├── index.json -│ ├── equity.json -│ └── exposure.json -├── f2s5/ ├── … -├── f3s4/ ├── … -└── f3s5/ └── … - -# any single member charts with the existing viewer, no new code: -$ aura chart swp/f2s4 > member.html - -$ aura mc --trace mc1 # one dir per draw seed: runs/traces/mc1/seed{1,2,3}/ -$ aura walkforward --trace wf1 # one dir per OOS window: runs/traces/wf1/oos{from-ns}/ -``` - -Without `--trace`, every byte of stdout and the run registry is identical to -today, and no `runs/traces//` tree is created. - -### Member-key derivation (per family kind, all deterministic + collision-free) - -| Family | Closure receives | Member key | -|--------------|-------------------------|-----------------------------------------| -| Monte-Carlo | `seed: u64` | `seed{seed}` (e.g. `seed1`) | -| sweep | `point: &[Cell]` | a stable join of the grid point's param values (e.g. `f{fast}s{slow}`); deterministic + unique across the grid | -| walk-forward | `w: WindowBounds` | `oos{w.oos.0}` (OOS window start ns; unique per window) | - -The exact rendering of the sweep key is the planner's to fix; the contract is -**deterministic, collision-free, and stable across runs** (distinct grid points -differ in ≥1 swept axis). The closure already builds the named params via -`zip_params(&space, point)` for the manifest, so the key is derivable locally -with no engine change. - -### Arg dispatch (before → after, `main.rs:953-958`) - -```rust -// before -["sweep"] => run_sweep("sweep"), -["sweep", "--name", n] => run_sweep(n), -["walkforward"] => run_walkforward("walkforward"), -["walkforward","--name",n]=> run_walkforward(n), -["mc"] => run_mc("mc"), -["mc", "--name", n] => run_mc(n), - -// after — add the --trace opt-in (persist = true); run signatures gain `persist: bool` -["sweep"] => run_sweep("sweep", false), -["sweep", "--name", n] => run_sweep(n, false), -["sweep", "--trace", n] => run_sweep(n, true), -["walkforward"] => run_walkforward("walkforward", false), -["walkforward","--name",n]=> run_walkforward(n, false), -["walkforward","--trace",n]=> run_walkforward(n, true), -["mc"] => run_mc("mc", false), -["mc", "--name", n] => run_mc(n, false), -["mc", "--trace", n] => run_mc(n, true), -``` - -`USAGE` (`main.rs:929`) gains the `[--trace ]` alternative for the three -family subcommands. - -### Closure persistence (before → after, Monte-Carlo as the template — `mc_family`:678) - -```rust -// before: rows collected inline into f64_field, then folded; raw rows lost. -fn mc_family() -> McFamily { - monte_carlo(&base_point, &[1, 2, 3], |seed, _base| { - let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE); - /* … build sources, run … */ h.run(sources); - let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); - let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); - RunReport { manifest: sim_optimal_manifest(/* … */), metrics: summarize(&equity, &exposure) } - }) -} - -// after: collect rows ONCE, build the manifest first, persist when enabled, -// then fold. `trace: Option<&str>` threads the opt-in prefix into the closure -// (a shared &str — Fn + Sync hold). Engine call site unchanged. -fn mc_family(trace: Option<&str>) -> McFamily { - monte_carlo(&base_point, &[1, 2, 3], |seed, _base| { - let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE); - /* … build sources, run … */ h.run(sources); - let eq_rows = rx_eq.try_iter().collect::>(); - let ex_rows = rx_ex.try_iter().collect::>(); - let manifest = sim_optimal_manifest(/* … */); - if let Some(name) = trace { - persist_traces(&format!("{name}/seed{seed}"), &manifest, &eq_rows, &ex_rows); - } - let equity = f64_field(&eq_rows, 0); - let exposure = f64_field(&ex_rows, 0); - RunReport { manifest, metrics: summarize(&equity, &exposure) } - }) -} -``` - -`sweep_family`:448 and `run_oos`:599 (the walk-forward OOS member) take the same -shape: thread `trace: Option<&str>`, collect rows once, build the manifest, -persist under `/` when `trace` is `Some`, then fold. The -sweep IS sweeps inside `sweep_over` (`walkforward_family`) are **not** persisted -— only the OOS member is the walk-forward family's recorded run (it is what -`append_family` stores). - -## Components - -- **`crates/aura-cli/src/main.rs`** — the only file changed: - - arg dispatch + `USAGE` (the `--trace` forms); - - `run_sweep` / `run_mc` / `run_walkforward` gain a `persist: bool` and pass - `Some(name)` / `None` down to their family builders; - - `sweep_family` / `mc_family` / `walkforward_family` (and `run_oos`) gain a - `trace: Option<&str>` and call `persist_traces` per member when set; - - the `#[cfg(test)]` helpers (`sweep_report`, `mc_report`, - `walkforward_report`) call the builders with `None` (no behaviour change). -- **`persist_traces` (`main.rs:161`)** — reused verbatim. It already accepts an - arbitrary `name`; a slash-containing name is a nested member dir. -- **`TraceStore` (`trace_store.rs`)** — reused verbatim. `write` joins the name - under `traces/` and `create_dir_all`s it, so `"swp/f2s4"` yields - `traces/swp/f2s4/`. No registry/store change. -- **Engine (`sweep` / `mc` / `walkforward`)** — untouched. - -## Data flow - -``` -per family member (parallel, disjoint — C1): - build harness → run → drain eq_rows, ex_rows (already happens) - │ - ├─ if --trace: persist_traces("/", manifest, eq_rows, ex_rows) - │ → TraceStore::write → runs/traces///{index,equity,exposure}.json - │ - └─ fold: f64_field → summarize → RunReport (already happens) - → append_family(...) (registry metrics — unchanged) -``` - -The persist step is a pure side-effect on the disjoint per-member path; it does -not feed back into the fold or the registry. Determinism (C1) of *what each -member computes* is untouched; only a write is added. - -## Error handling - -`persist_traces` already maps an I/O failure to `eprintln! + std::process::exit(2)` -(`main.rs:171-174`), the CLI's usage-error convention. Reused as-is: a member -trace that cannot be written aborts the process with exit 2, at parity with the -single-run path. (A worker-thread `exit` under the parallel families is abrupt -but acceptable — it matches existing fatal-I/O semantics; converting to a -propagated `Result` is a deliberate non-goal here.) With no `--trace`, no I/O is -attempted and the path is unchanged. - -## Testing strategy - -New integration coverage (aura-cli), one property per test, observable behaviour -only: - -1. **sweep persists one round-tripping run-dir per grid point.** - `run_sweep("t", true)` (or the dispatched form) then `TraceStore::open(tmp) - .read("t/")` for each member returns equity + exposure taps with the - expected SoA shape; the dir count equals the grid size. -2. **mc persists one dir per seed; wf one per OOS window.** Same round-trip, - keyed by `seed{N}` / `oos{from}`. -3. **member keys are deterministic + distinct.** Two `persist=true` runs produce - identical member-dir names and byte-identical tap files (C1); keys across one - family are pairwise distinct. -4. **`persist=false` writes nothing and changes no stdout.** No - `runs/traces//` tree is created, and the family's stdout / registry - output is byte-identical to the pre-change behaviour (guards the - byte-unchanged claim). - -Tests isolate the trace root to a temp dir (as the existing trace tests do) so -they neither read nor write the repo's gitignored `runs/`. The existing family -determinism tests (`walkforward_report_is_deterministic`, the sweep/mc report -tests) continue to pass unchanged (they call the builders with `None`). - -## Acceptance criteria - -Applying aura's feature-acceptance criterion: - -- **The audience naturally reaches for it.** A trader running a sweep / MC / - walk-forward wants to *see* individual members' equity curves, not only the - aggregate metric line — the family-comparison is the C21 differentiator. This - cycle puts the per-member streams on disk, the precondition for that view; a - single member is already chartable via `aura chart /`. -- **It measurably adds capability without redundancy.** Streams that were - produced-then-discarded are now persisted, reusing the single-run - `persist_traces` / `TraceStore` path verbatim — no second persistence - mechanism. -- **It reintroduces no failure class the core constraints guard against.** - Determinism (C1) is preserved by content-derived keys (no runtime-order - dependence) and by persistence being a disjoint per-member side-effect; SoA - (C7) and the sink/recorder contract (C8) are upheld (the same `ColumnarTrace` - taps as single runs); the engine is untouched; without `--trace` behaviour is - byte-unchanged. - -Concretely, the cycle is accepted when: - -- `aura sweep --trace s` / `aura mc --trace m` / `aura walkforward --trace w` - each write one standalone, round-tripping run-dir per member under - `runs/traces///`; -- member keys are deterministic, stable across runs, and pairwise distinct; -- the same commands without `--trace` create no trace dirs and emit - byte-identical stdout / registry output; -- `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D - warnings` are green.