diff --git a/docs/specs/0048-walk-forward-param-plane.md b/docs/specs/0048-walk-forward-param-plane.md new file mode 100644 index 0000000..52b078e --- /dev/null +++ b/docs/specs/0048-walk-forward-param-plane.md @@ -0,0 +1,280 @@ +# Walk-forward param plane: space on the family, tag-free `chosen_params`, schema-checked `param_stability` — Design Spec + +**Date:** 2026-06-16 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +> Seeding issue: `Brummel/Aura#75`. Overturns the "Fork A" out-of-scope note from +> cycle 0047 (`refactor(aura-core): Scalar as a native tagged enum …`, 86746e3). + +## Goal + +Make the walk-forward param plane C7-clean and symmetric with the sweep param plane: +the param **kinds** live once on the family (`space: Vec`), the per-window +chosen points are tag-free (`Vec`), and `param_stability` does its one numeric +coercion against the **schema** (once per slot) instead of against the **value** (once +per window-value, guarded by a per-value `unreachable!`). + +This is a single, indivisible, behaviour-preserving (C1) cut. It removes a redundancy +(the param kind, constant per slot, was carried N×M times at the value) and a latent +unsafety (the per-value `unreachable!` re-asserting an invariant the schema already +fixes). + +### Why now / why this is not still Fork A + +Cycle 0047 kept `WindowRun.chosen_params` as `Vec`, justified by "Scalar is now +serializable, so a `WindowRun` is a self-describing record." That defense does not hold: +`WalkForwardResult` and `WindowRun` carry **no serde derives** — they are never +serialized. Only the embedded `oos_report` (a `RunReport`, already typed via the 0047 +`RunManifest.params` lift) is. So `chosen_params` is a **pure in-memory substrate** for +`param_stability`, and the substantive case (C7 purity, `SweepFamily` symmetry, per-value +`unreachable!` elimination) wins over a moot serialization defense — the same +argument-form that retired the `{kind, cell}` Scalar struct in 0047: "it works" does not +beat "it fits structurally." + +## Architecture + +`SweepFamily` already shows the C7-clean shape in the same crate (`sweep.rs`): the family +owns `space: Vec`; each point's coordinate is a tag-free `Vec`; the named +view is recombined on demand via `zip_params(&space, point)`. Walk-forward is the lone +asymmetry — it dissolves the point to `Scalar` early and discards the space. This cycle +makes walk-forward share that **idiom** (`space` + tag-free `Cell` points + `zip_params`), +while the two stay **distinct types**: Sweep's axis *is* the param-space (`points[i].params` +is an enumerated **input** coordinate); walk-forward's axis is **time** (C12 axis 3), +`chosen_params` is the inner optimizer's **output**, and the walk-forward family +additionally carries `bounds`, per-window `oos_equity`, and `stitched_oos_equity` — none of +which Sweep has. No unified container, no shared trait: the shared substance is the +convention plus the free function `zip_params`, not the struct. + +Three coupled parts, indivisible (β1's `Vec` forces [A]'s space-threading, which is +what lets [C]/[D] read kinds from the schema): + +- **β1 — data shape.** `WalkForwardResult` gains `space: Vec`; + `WindowRun.chosen_params: Vec` → `Vec`. +- **[A] — the `walk_forward` seam.** The `space` is threaded as an explicit by-value + parameter, *beside* the run-window closure; the `Fn` generic is untouched. +- **[C]/[D] — schema-checked reduction.** `param_stability` builds a per-slot coercer from + `result.space` *before* a type-blind reduction loop; `scalar_as_f64` and its per-value + `unreachable!` are deleted. + +## Concrete code shapes + +### What the cycle delivers (the consuming code, after) + +`param_stability` is the one real consumer of `chosen_params`. Its public signature is +**unchanged** (the space is on the result now); its body becomes schema-checked and +type-blind: + +```rust +// crates/aura-engine/src/walkforward.rs — param_stability, AFTER +pub fn param_stability(result: &WalkForwardResult) -> Vec { + // [C] ONE schema pass, before any reduction: resolve a coercer per slot. + // Kind is touched here only — nslots-many times, never per window-value. + let coerce: Vec f64> = result + .space + .iter() + .map(|ps| match ps.kind { + ScalarKind::I64 => (|c: Cell| c.i64() as f64) as fn(Cell) -> f64, + ScalarKind::F64 => |c: Cell| c.f64(), + // [D] Bool -> 0/1 keeps the coercer TOTAL over the knob kinds + // (mean = fraction "on"). Timestamp is never a knob (C20) -> a + // single schema-level unreachable, not a per-value one. + ScalarKind::Bool => |c: Cell| c.bool() as i64 as f64, + ScalarKind::Timestamp => unreachable!("timestamp is a structural axis (C20), never a param knob"), + }) + .collect(); + + // The reduction loop is now type-blind: no kind, no match, no per-value unreachable. + coerce + .iter() + .enumerate() + .map(|(slot, f)| { + let vals: Vec = + result.windows.iter().map(|w| f(w.run.chosen_params[slot])).collect(); + MetricStats::from_values(&vals) + }) + .collect() +} +``` + +### North-star slice — the one production caller, after + +`walkforward_family` (in `aura-cli`) is the real walk-forward author surface. With β1 + +[A] its closure gets *simpler* — it no longer reconstructs `Scalar`s, and the space is +computed once: + +```rust +// crates/aura-cli/src/main.rs — walkforward_family, AFTER +fn walkforward_family() -> WalkForwardResult { + // ... sources, span, roller ... + let space = sample_blueprint_with_sinks().0.param_space(); // ONCE; same blueprint every window + walk_forward(roller, space, |w: WindowBounds| { + let is_family = sweep_over(w.is.0, w.is.1); + let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric"); + let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1); + WindowRun { + chosen_params: best.params, // already Vec; the from_cell+zip map is gone + oos_equity, + oos_report, + } + }) +} +``` + +The typed-but-tag-free shape is what a downstream walk-forward consumer reads: the named +view is `zip_params(&result.space, &result.windows[i].run.chosen_params)` — exactly the +`SweepFamily::named_params` idiom. (An inherent `WalkForwardResult::named_params` mirroring +it is a natural follow-on, **out of scope** for this cycle.) + +### Before → after, per load-bearing change + +**1. The two structs (β1).** + +```rust +// walkforward.rs — BEFORE +pub struct WindowRun { + pub chosen_params: Vec, + pub oos_equity: Vec<(Timestamp, f64)>, + pub oos_report: RunReport, +} +pub struct WalkForwardResult { + pub windows: Vec, + pub stitched_oos_equity: Vec<(Timestamp, f64)>, +} + +// AFTER +pub struct WindowRun { + pub chosen_params: Vec, // tag-free coordinate (kind lives on the family) + pub oos_equity: Vec<(Timestamp, f64)>, + pub oos_report: RunReport, +} +pub struct WalkForwardResult { + pub space: Vec, // the kinds, once (mirrors SweepFamily.space) + pub windows: Vec, + pub stitched_oos_equity: Vec<(Timestamp, f64)>, +} +``` + +`WindowOutcome` (`{ bounds, run }`) is unchanged. The derives on both structs +(`Clone, Debug, PartialEq`) are unchanged — no serde is added (the structs are not +serialized; `ParamSpec` already derives `PartialEq`/`Eq`). + +**2. The `walk_forward` seam ([A]).** + +```rust +// walkforward.rs — BEFORE +pub fn walk_forward(roller: WindowRoller, run_window: F) -> WalkForwardResult +where F: Fn(WindowBounds) -> WindowRun + Sync { /* … */ } + +fn walk_forward_with_threads(roller: WindowRoller, nthreads: usize, run_window: F) -> WalkForwardResult +where F: Fn(WindowBounds) -> WindowRun + Sync { /* … */ } + +// AFTER — space is a sibling parameter; the Fn bound is IDENTICAL +pub fn walk_forward(roller: WindowRoller, space: Vec, run_window: F) -> WalkForwardResult +where F: Fn(WindowBounds) -> WindowRun + Sync { /* … */ } + +fn walk_forward_with_threads(roller: WindowRoller, space: Vec, nthreads: usize, run_window: F) -> WalkForwardResult +where F: Fn(WindowBounds) -> WindowRun + Sync { + // … collect bounds, run_indexed (UNCHANGED — space is never captured by the Sync closure) … + debug_assert!( + runs.iter().all(|r| r.chosen_params.len() == space.len()), + "every window's chosen point must match the param-space arity (same blueprint)" + ); + WalkForwardResult { space, windows, stitched_oos_equity } // space moved in at the end +} +``` + +The space is schema metadata, passed by value and moved onto the result; it is never +captured by the `Sync` run-window closure, so `run_indexed` (the parallelism core, +`sweep.rs`) needs **zero** change. The `debug_assert` guards the one new caller contract: +the externally-supplied `space` must match the closure-produced points' arity — guaranteed +because every window runs the same blueprint, so its param-space (arity + kinds) is +identical (the bootstrap invariant the current code already relies on when it reads +`nslots` from the first window). + +**3. `scalar_as_f64` is deleted.** The per-value coercion + `unreachable!` +(`walkforward.rs` ~`:254`) is removed entirely; the schema-pass coercer in `param_stability` +replaces it. + +## Components + +- **`crates/aura-engine/src/walkforward.rs`** — the bulk. Struct defs (`WindowRun`, + `WalkForwardResult`); `walk_forward` / `walk_forward_with_threads` signatures + bodies + (+ the arity `debug_assert`); `param_stability` rewrite (schema-pass + type-blind loop); + delete `scalar_as_f64`. Imports gain `ParamSpec`; the `chosen_params` path no longer + needs `Scalar`. The in-file test fixtures migrate (see Testing strategy). +- **`crates/aura-cli/src/main.rs`** — `walkforward_family`: compute `space` once at the top, + pass it to `walk_forward`, simplify the closure to `chosen_params: best.params`. The other + `WalkForwardResult` readers in this file (`result.windows` → `oos_report`, + `result.stitched_oos_equity` → total pips) are **unaffected** — they never touch + `chosen_params` or `space`. +- **`crates/aura-registry/src/lineage.rs`** — reads `result.windows[..].run.oos_report` + only; **unaffected** (verified: no `chosen_params` / `space` reader). Listed so the + planner confirms it during implement. + +## Data flow + +- **Kinds (schema):** caller `bp.param_space()` → `walk_forward(space, …)` → moved onto + `WalkForwardResult.space`. Computed once; identical for every window (same blueprint). +- **Points (values):** run-window closure → `best.params` (`Vec`, the sweep winner) + → `WindowRun.chosen_params`. Tag-free; never carries a kind. +- **Reduction:** `param_stability` reads kinds from `result.space` (once per slot → coercer + table) and values from `result.windows[*].run.chosen_params[slot]` (per window) → + `MetricStats::from_values`. +- **Named view (existing idiom, on demand):** `zip_params(&result.space, &chosen_params)` + recombines slot names with typed values — the same mechanism `SweepFamily::named_params` + uses. + +## Error handling + +- **Arity coupling:** a `debug_assert` in `walk_forward_with_threads` checks every window's + `chosen_params.len() == space.len()`. No release-mode runtime error variant is added — a + mismatch is a wiring bug (caller passed the wrong blueprint's space), surfaced like the + engine's other "checked at wiring" violations. +- **Non-numeric slot:** `Bool` is handled (→ 0/1), not an error. `Timestamp` in a param + slot is structurally impossible (C20: a timestamp is a structural axis, never a numeric + knob), so the schema pass carries a single `unreachable!` on that arm — reached once per + slot at most, never per window-value. This replaces the old per-value `unreachable!`. +- No new `CompileError` / runtime-error variants; no change to any error type. + +## Testing strategy + +Behaviour-preserving (C1): the migration re-spells fixtures, not values. + +- **Existing tests stay green, adapted to the new shapes:** + - `walk_forward_runs_one_window_per_split_in_roll_order` and + `walk_forward_is_deterministic_across_thread_counts` — thread the new `space` argument + into the `walk_forward` / `walk_forward_with_threads` calls; the run-window fixture's + `WindowRun` now yields `chosen_params: Vec` (`Scalar::i64/f64(x)` → + `Cell::from_i64/from_f64(x)`). Window bounds / determinism assertions are unchanged. + - `param_stability_reduces_chosen_params_per_slot` — the `WindowRun` literals use `Cell` + points and the `WalkForwardResult` literal gains a two-slot `space` fixture + (`ParamSpec { name, kind: I64 }`, `ParamSpec { name, kind: F64 }`). The asserted + `MetricStats` are **identical** (the `I64→f64` / `F64` coercions are value-identical to + the deleted `scalar_as_f64`). +- **One genuinely new behaviour — the `Bool → 0/1` arm.** No built-in grid produces a + `Bool` param, so add a focused `param_stability` test with a `Bool` slot in `space` and + `Cell::from_bool` points across windows, asserting `mean` = the fraction of `true` + windows. This keeps the new coercer arm non-dead and tested. +- **The four project gates:** `cargo build --workspace --all-targets`; + `cargo test --workspace`; `cargo clippy --workspace --all-targets -- -D warnings`; + `cargo doc --workspace --no-deps` — all clean. + +## Acceptance criteria + +1. `WindowRun.chosen_params` is `Vec`; `WalkForwardResult` has `space: Vec`; + neither struct gains a serde derive. +2. `walk_forward` and `walk_forward_with_threads` take `space: Vec` by value; the + `Fn(WindowBounds) -> WindowRun + Sync` bound is unchanged; `run_indexed` is unchanged. +3. `walkforward_family` computes the space once and passes `chosen_params: best.params` + (no `from_cell` reconstruction). +4. `param_stability` reads `result.space`, builds a per-slot coercer before a type-blind + reduction loop, and contains no per-value `unreachable!`; `scalar_as_f64` is deleted. + Its public signature is unchanged. +5. The arity `debug_assert` is present in `walk_forward_with_threads`. +6. The three existing walk-forward tests are green with **identical** `MetricStats`; the new + `Bool`-slot `param_stability` test is green. +7. All four gates clean. +8. Spot-check greps: `scalar_as_f64` absent from `crates/`; no `unreachable!` in a per-value + path in `walkforward.rs`; `WindowRun.chosen_params: Vec`; `WalkForwardResult` has + `space: Vec`.