From 04f13af88ac4ff2b0c195f796411c14a8de0810b Mon Sep 17 00:00:00 2001 From: Brummel Date: Sat, 20 Jun 2026 18:40:41 +0200 Subject: [PATCH] spec: 0058 family-trace-persistence (boss-signed) Persist each family member's equity/exposure streams to disk (opt-in --trace on sweep|mc|walkforward), mirroring the single-run trace path, so the C21 family-comparison view has per-member data and any single member is chartable today. Write side only; the comparison view, decimation, and a binary trace container are deferred. Autonomously signed under /boss on the Step-5 grounding-check PASS. refs #104 --- docs/specs/0058-family-trace-persistence.md | 268 ++++++++++++++++++++ 1 file changed, 268 insertions(+) create mode 100644 docs/specs/0058-family-trace-persistence.md diff --git a/docs/specs/0058-family-trace-persistence.md b/docs/specs/0058-family-trace-persistence.md new file mode 100644 index 0000000..0170248 --- /dev/null +++ b/docs/specs/0058-family-trace-persistence.md @@ -0,0 +1,268 @@ +# Family-member trace persistence — Design Spec + +**Date:** 2026-06-20 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +Reference issue: Brummel/Aura #104. Fork decisions (A–D) and the member-key +correction are recorded on that issue's thread (the run's decision log). + +## Goal + +Make each family member's recorded streams land on disk, so the C21 +family-comparison view has per-member data to chart and any single member is +chartable *today* with the existing single-run viewer. + +Single backtest runs already do this: `aura run --trace ` drains the +equity + exposure recorders and persists them via `persist_traces` +(`crates/aura-cli/src/main.rs:161`) → `TraceStore` +(`crates/aura-registry/src/trace_store.rs:40`), under `runs/traces//`. +Family runs (`sweep` / `mc` / `walkforward`) do not: each member *does* run a +full harness with both recorders and *is* drained inside its per-member closure, +but the raw rows are immediately folded to `metrics: summarize(...)` and +discarded. The streams the comparison view needs exist transiently and are +thrown away. + +This cycle is the **write side only**. The family-comparison *view* (how to +render N members — overlay / small-multiples / rank-top-N), decimation / LOD, and +a binary trace container are explicitly out of scope (each its own later cycle). + +## Architecture + +Four derived decisions (recorded on #104): + +- **A — persist in the closure; engine untouched.** The three per-member + closures live in aura-cli and already hold the drained rows. When persistence + is enabled they call the existing `persist_traces` directly, before folding to + metrics. The engine's `sweep` / `monte_carlo` / `walk_forward` and the + `SweepFamily` / `McFamily` / `WalkForwardResult` types are **not** changed. +- **B — opt-in.** A new `--trace` opt-in on `aura sweep|mc|walkforward` gates + persistence. Without it, behaviour is byte-unchanged (no member dirs written — + the cardinality cost of N members × full resolution stays user-chosen). +- **C — content-derived deterministic member keys.** The engine higher-order fns + are `F: Fn(...) + Sync` (`sweep.rs:314`, `mc.rs:124`, `walkforward.rs:187`): + members run in parallel across threads (C1 parallelism *across* sims). A + runtime ordinal counter would be non-deterministic (thread-race → C1 + violation). Keys are therefore derived from what each closure receives + deterministically (see the member-key table). +- **D — standalone run-dir per member.** Each member is persisted under + `runs/traces///` as a complete run-dir (its own `index.json` + + `equity.json` + `exposure.json`), by calling `persist_traces` with a + slash-containing name. `TraceStore::write` resolves that to nested directories; + the existing single-run viewer charts any one member as-is, with no view-side + code. + +`--trace ` sets both the family registry name and the trace prefix to +`` (a member registers under the family, and its streams land under +`runs/traces//`). This avoids a redundant second name while keeping +`--trace ` taking an argument, exactly as the single-run form does. + +## Concrete code shapes + +### User-facing surface (the acceptance evidence) + +```console +# opt in: register the family under "swp" AND persist each member's streams +$ aura sweep --trace swp +{"family_id":"swp-1","report":{...}} # stdout unchanged from `aura sweep --name swp` +... one line per grid point ... + +$ tree runs/traces/swp +runs/traces/swp +├── f2s4/ # one standalone run-dir per grid point (content-keyed) +│ ├── index.json +│ ├── equity.json +│ └── exposure.json +├── f2s5/ ├── … +├── f3s4/ ├── … +└── f3s5/ └── … + +# any single member charts with the existing viewer, no new code: +$ aura chart swp/f2s4 > member.html + +$ aura mc --trace mc1 # one dir per draw seed: runs/traces/mc1/seed{1,2,3}/ +$ aura walkforward --trace wf1 # one dir per OOS window: runs/traces/wf1/oos{from-ns}/ +``` + +Without `--trace`, every byte of stdout and the run registry is identical to +today, and no `runs/traces//` tree is created. + +### Member-key derivation (per family kind, all deterministic + collision-free) + +| Family | Closure receives | Member key | +|--------------|-------------------------|-----------------------------------------| +| Monte-Carlo | `seed: u64` | `seed{seed}` (e.g. `seed1`) | +| sweep | `point: &[Cell]` | a stable join of the grid point's param values (e.g. `f{fast}s{slow}`); deterministic + unique across the grid | +| walk-forward | `w: WindowBounds` | `oos{w.oos.0}` (OOS window start ns; unique per window) | + +The exact rendering of the sweep key is the planner's to fix; the contract is +**deterministic, collision-free, and stable across runs** (distinct grid points +differ in ≥1 swept axis). The closure already builds the named params via +`zip_params(&space, point)` for the manifest, so the key is derivable locally +with no engine change. + +### Arg dispatch (before → after, `main.rs:953-958`) + +```rust +// before +["sweep"] => run_sweep("sweep"), +["sweep", "--name", n] => run_sweep(n), +["walkforward"] => run_walkforward("walkforward"), +["walkforward","--name",n]=> run_walkforward(n), +["mc"] => run_mc("mc"), +["mc", "--name", n] => run_mc(n), + +// after — add the --trace opt-in (persist = true); run signatures gain `persist: bool` +["sweep"] => run_sweep("sweep", false), +["sweep", "--name", n] => run_sweep(n, false), +["sweep", "--trace", n] => run_sweep(n, true), +["walkforward"] => run_walkforward("walkforward", false), +["walkforward","--name",n]=> run_walkforward(n, false), +["walkforward","--trace",n]=> run_walkforward(n, true), +["mc"] => run_mc("mc", false), +["mc", "--name", n] => run_mc(n, false), +["mc", "--trace", n] => run_mc(n, true), +``` + +`USAGE` (`main.rs:929`) gains the `[--trace ]` alternative for the three +family subcommands. + +### Closure persistence (before → after, Monte-Carlo as the template — `mc_family`:678) + +```rust +// before: rows collected inline into f64_field, then folded; raw rows lost. +fn mc_family() -> McFamily { + monte_carlo(&base_point, &[1, 2, 3], |seed, _base| { + let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE); + /* … build sources, run … */ h.run(sources); + let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); + let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); + RunReport { manifest: sim_optimal_manifest(/* … */), metrics: summarize(&equity, &exposure) } + }) +} + +// after: collect rows ONCE, build the manifest first, persist when enabled, +// then fold. `trace: Option<&str>` threads the opt-in prefix into the closure +// (a shared &str — Fn + Sync hold). Engine call site unchanged. +fn mc_family(trace: Option<&str>) -> McFamily { + monte_carlo(&base_point, &[1, 2, 3], |seed, _base| { + let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE); + /* … build sources, run … */ h.run(sources); + let eq_rows = rx_eq.try_iter().collect::>(); + let ex_rows = rx_ex.try_iter().collect::>(); + let manifest = sim_optimal_manifest(/* … */); + if let Some(name) = trace { + persist_traces(&format!("{name}/seed{seed}"), &manifest, &eq_rows, &ex_rows); + } + let equity = f64_field(&eq_rows, 0); + let exposure = f64_field(&ex_rows, 0); + RunReport { manifest, metrics: summarize(&equity, &exposure) } + }) +} +``` + +`sweep_family`:448 and `run_oos`:599 (the walk-forward OOS member) take the same +shape: thread `trace: Option<&str>`, collect rows once, build the manifest, +persist under `/` when `trace` is `Some`, then fold. The +sweep IS sweeps inside `sweep_over` (`walkforward_family`) are **not** persisted +— only the OOS member is the walk-forward family's recorded run (it is what +`append_family` stores). + +## Components + +- **`crates/aura-cli/src/main.rs`** — the only file changed: + - arg dispatch + `USAGE` (the `--trace` forms); + - `run_sweep` / `run_mc` / `run_walkforward` gain a `persist: bool` and pass + `Some(name)` / `None` down to their family builders; + - `sweep_family` / `mc_family` / `walkforward_family` (and `run_oos`) gain a + `trace: Option<&str>` and call `persist_traces` per member when set; + - the `#[cfg(test)]` helpers (`sweep_report`, `mc_report`, + `walkforward_report`) call the builders with `None` (no behaviour change). +- **`persist_traces` (`main.rs:161`)** — reused verbatim. It already accepts an + arbitrary `name`; a slash-containing name is a nested member dir. +- **`TraceStore` (`trace_store.rs`)** — reused verbatim. `write` joins the name + under `traces/` and `create_dir_all`s it, so `"swp/f2s4"` yields + `traces/swp/f2s4/`. No registry/store change. +- **Engine (`sweep` / `mc` / `walkforward`)** — untouched. + +## Data flow + +``` +per family member (parallel, disjoint — C1): + build harness → run → drain eq_rows, ex_rows (already happens) + │ + ├─ if --trace: persist_traces("/", manifest, eq_rows, ex_rows) + │ → TraceStore::write → runs/traces///{index,equity,exposure}.json + │ + └─ fold: f64_field → summarize → RunReport (already happens) + → append_family(...) (registry metrics — unchanged) +``` + +The persist step is a pure side-effect on the disjoint per-member path; it does +not feed back into the fold or the registry. Determinism (C1) of *what each +member computes* is untouched; only a write is added. + +## Error handling + +`persist_traces` already maps an I/O failure to `eprintln! + std::process::exit(2)` +(`main.rs:171-174`), the CLI's usage-error convention. Reused as-is: a member +trace that cannot be written aborts the process with exit 2, at parity with the +single-run path. (A worker-thread `exit` under the parallel families is abrupt +but acceptable — it matches existing fatal-I/O semantics; converting to a +propagated `Result` is a deliberate non-goal here.) With no `--trace`, no I/O is +attempted and the path is unchanged. + +## Testing strategy + +New integration coverage (aura-cli), one property per test, observable behaviour +only: + +1. **sweep persists one round-tripping run-dir per grid point.** + `run_sweep("t", true)` (or the dispatched form) then `TraceStore::open(tmp) + .read("t/")` for each member returns equity + exposure taps with the + expected SoA shape; the dir count equals the grid size. +2. **mc persists one dir per seed; wf one per OOS window.** Same round-trip, + keyed by `seed{N}` / `oos{from}`. +3. **member keys are deterministic + distinct.** Two `persist=true` runs produce + identical member-dir names and byte-identical tap files (C1); keys across one + family are pairwise distinct. +4. **`persist=false` writes nothing and changes no stdout.** No + `runs/traces//` tree is created, and the family's stdout / registry + output is byte-identical to the pre-change behaviour (guards the + byte-unchanged claim). + +Tests isolate the trace root to a temp dir (as the existing trace tests do) so +they neither read nor write the repo's gitignored `runs/`. The existing family +determinism tests (`walkforward_report_is_deterministic`, the sweep/mc report +tests) continue to pass unchanged (they call the builders with `None`). + +## Acceptance criteria + +Applying aura's feature-acceptance criterion: + +- **The audience naturally reaches for it.** A trader running a sweep / MC / + walk-forward wants to *see* individual members' equity curves, not only the + aggregate metric line — the family-comparison is the C21 differentiator. This + cycle puts the per-member streams on disk, the precondition for that view; a + single member is already chartable via `aura chart /`. +- **It measurably adds capability without redundancy.** Streams that were + produced-then-discarded are now persisted, reusing the single-run + `persist_traces` / `TraceStore` path verbatim — no second persistence + mechanism. +- **It reintroduces no failure class the core constraints guard against.** + Determinism (C1) is preserved by content-derived keys (no runtime-order + dependence) and by persistence being a disjoint per-member side-effect; SoA + (C7) and the sink/recorder contract (C8) are upheld (the same `ColumnarTrace` + taps as single runs); the engine is untouched; without `--trace` behaviour is + byte-unchanged. + +Concretely, the cycle is accepted when: + +- `aura sweep --trace s` / `aura mc --trace m` / `aura walkforward --trace w` + each write one standalone, round-tripping run-dir per member under + `runs/traces///`; +- member keys are deterministic, stable across runs, and pairwise distinct; +- the same commands without `--trace` create no trace dirs and emit + byte-identical stdout / registry output; +- `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D + warnings` are green.