Aura/docs/specs/0048-walk-forward-param-plane.md

Walk-forward param plane: space on the family, tag-free chosen_params, schema-checked param_stability — Design Spec

Date: 2026-06-16 Status: Draft — awaiting user spec review Authors: orchestrator + Claude

Seeding issue: Brummel/Aura#75. Overturns the "Fork A" out-of-scope note from cycle 0047 (refactor(aura-core): Scalar as a native tagged enum …, 86746e3).

Goal

Make the walk-forward param plane C7-clean and symmetric with the sweep param plane: the param kinds live once on the family (space: Vec<ParamSpec>), the per-window chosen points are tag-free (Vec<Cell>), and param_stability does its one numeric coercion against the schema (once per slot) instead of against the value (once per window-value, guarded by a per-value unreachable!).

This is a single, indivisible, behaviour-preserving (C1) cut. It removes a redundancy (the param kind, constant per slot, was carried N×M times at the value) and a latent unsafety (the per-value unreachable! re-asserting an invariant the schema already fixes).

Why now / why this is not still Fork A

Cycle 0047 kept WindowRun.chosen_params as Vec<Scalar>, justified by "Scalar is now serializable, so a WindowRun is a self-describing record." That defense does not hold: WalkForwardResult and WindowRun carry no serde derives — they are never serialized. Only the embedded oos_report (a RunReport, already typed via the 0047 RunManifest.params lift) is. So chosen_params is a pure in-memory substrate for param_stability, and the substantive case (C7 purity, SweepFamily symmetry, per-value unreachable! elimination) wins over a moot serialization defense — the same argument-form that retired the {kind, cell} Scalar struct in 0047: "it works" does not beat "it fits structurally."

Architecture

SweepFamily already shows the C7-clean shape in the same crate (sweep.rs): the family owns space: Vec<ParamSpec>; each point's coordinate is a tag-free Vec<Cell>; the named view is recombined on demand via zip_params(&space, point). Walk-forward is the lone asymmetry — it dissolves the point to Scalar early and discards the space. This cycle makes walk-forward share that idiom (space + tag-free Cell points + zip_params), while the two stay distinct types: Sweep's axis is the param-space (points[i].params is an enumerated input coordinate); walk-forward's axis is time (C12 axis 3), chosen_params is the inner optimizer's output, and the walk-forward family additionally carries bounds, per-window oos_equity, and stitched_oos_equity — none of which Sweep has. No unified container, no shared trait: the shared substance is the convention plus the free function zip_params, not the struct.

Three coupled parts, indivisible (β1's Vec<Cell> forces [A]'s space-threading, which is what lets [C]/[D] read kinds from the schema):

• β1 — data shape. WalkForwardResult gains space: Vec<ParamSpec>; WindowRun.chosen_params: Vec<Scalar> → Vec<Cell>.
• [A] — the walk_forward seam. The space is threaded as an explicit by-value parameter, beside the run-window closure; the Fn generic is untouched.
• [C]/[D] — schema-checked reduction. param_stability builds a per-slot coercer from result.space before a type-blind reduction loop; scalar_as_f64 and its per-value unreachable! are deleted.

Concrete code shapes

What the cycle delivers (the consuming code, after)

param_stability is the one real consumer of chosen_params. Its public signature is unchanged (the space is on the result now); its body becomes schema-checked and type-blind:

// crates/aura-engine/src/walkforward.rs — param_stability, AFTER
pub fn param_stability(result: &WalkForwardResult) -> Vec<MetricStats> {
    // [C] ONE schema pass, before any reduction: resolve a coercer per slot.
    //     Kind is touched here only — nslots-many times, never per window-value.
    let coerce: Vec<fn(Cell) -> f64> = result
        .space
        .iter()
        .map(|ps| match ps.kind {
            ScalarKind::I64 => (|c: Cell| c.i64() as f64) as fn(Cell) -> f64,
            ScalarKind::F64 => |c: Cell| c.f64(),
            // [D] Bool -> 0/1 keeps the coercer TOTAL over the knob kinds
            //     (mean = fraction "on"). Timestamp is never a knob (C20) -> a
            //     single schema-level unreachable, not a per-value one.
            ScalarKind::Bool => |c: Cell| c.bool() as i64 as f64,
            ScalarKind::Timestamp => unreachable!("timestamp is a structural axis (C20), never a param knob"),
        })
        .collect();

    // The reduction loop is now type-blind: no kind, no match, no per-value unreachable.
    coerce
        .iter()
        .enumerate()
        .map(|(slot, f)| {
            let vals: Vec<f64> =
                result.windows.iter().map(|w| f(w.run.chosen_params[slot])).collect();
            MetricStats::from_values(&vals)
        })
        .collect()
}

North-star slice — the one production caller, after

walkforward_family (in aura-cli) is the real walk-forward author surface. With β1 + [A] its closure gets simpler — it no longer reconstructs Scalars, and the space is computed once:

// crates/aura-cli/src/main.rs — walkforward_family, AFTER
fn walkforward_family() -> WalkForwardResult {
    // ... sources, span, roller ...
    let space = sample_blueprint_with_sinks().0.param_space(); // ONCE; same blueprint every window
    walk_forward(roller, space, |w: WindowBounds| {
        let is_family = sweep_over(w.is.0, w.is.1);
        let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric");
        let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1);
        WindowRun {
            chosen_params: best.params, // already Vec<Cell>; the from_cell+zip map is gone
            oos_equity,
            oos_report,
        }
    })
}

The typed-but-tag-free shape is what a downstream walk-forward consumer reads: the named view is zip_params(&result.space, &result.windows[i].run.chosen_params) — exactly the SweepFamily::named_params idiom. (An inherent WalkForwardResult::named_params mirroring it is a natural follow-on, out of scope for this cycle.)

Before → after, per load-bearing change

1. The two structs (β1).

// walkforward.rs — BEFORE
pub struct WindowRun {
    pub chosen_params: Vec<Scalar>,
    pub oos_equity: Vec<(Timestamp, f64)>,
    pub oos_report: RunReport,
}
pub struct WalkForwardResult {
    pub windows: Vec<WindowOutcome>,
    pub stitched_oos_equity: Vec<(Timestamp, f64)>,
}

// AFTER
pub struct WindowRun {
    pub chosen_params: Vec<Cell>,          // tag-free coordinate (kind lives on the family)
    pub oos_equity: Vec<(Timestamp, f64)>,
    pub oos_report: RunReport,
}
pub struct WalkForwardResult {
    pub space: Vec<ParamSpec>,             // the kinds, once (mirrors SweepFamily.space)
    pub windows: Vec<WindowOutcome>,
    pub stitched_oos_equity: Vec<(Timestamp, f64)>,
}

WindowOutcome ({ bounds, run }) is unchanged. The derives on both structs (Clone, Debug, PartialEq) are unchanged — no serde is added (the structs are not serialized; ParamSpec already derives PartialEq/Eq).

2. The walk_forward seam ([A]).

// walkforward.rs — BEFORE
pub fn walk_forward<F>(roller: WindowRoller, run_window: F) -> WalkForwardResult
where F: Fn(WindowBounds) -> WindowRun + Sync { /* … */ }

fn walk_forward_with_threads<F>(roller: WindowRoller, nthreads: usize, run_window: F) -> WalkForwardResult
where F: Fn(WindowBounds) -> WindowRun + Sync { /* … */ }

// AFTER — space is a sibling parameter; the Fn bound is IDENTICAL
pub fn walk_forward<F>(roller: WindowRoller, space: Vec<ParamSpec>, run_window: F) -> WalkForwardResult
where F: Fn(WindowBounds) -> WindowRun + Sync { /* … */ }

fn walk_forward_with_threads<F>(roller: WindowRoller, space: Vec<ParamSpec>, nthreads: usize, run_window: F) -> WalkForwardResult
where F: Fn(WindowBounds) -> WindowRun + Sync {
    // … collect bounds, run_indexed (UNCHANGED — space is never captured by the Sync closure) …
    debug_assert!(
        runs.iter().all(|r| r.chosen_params.len() == space.len()),
        "every window's chosen point must match the param-space arity (same blueprint)"
    );
    WalkForwardResult { space, windows, stitched_oos_equity } // space moved in at the end
}

The space is schema metadata, passed by value and moved onto the result; it is never captured by the Sync run-window closure, so run_indexed (the parallelism core, sweep.rs) needs zero change. The debug_assert guards the one new caller contract: the externally-supplied space must match the closure-produced points' arity — guaranteed because every window runs the same blueprint, so its param-space (arity + kinds) is identical (the bootstrap invariant the current code already relies on when it reads nslots from the first window).

3. scalar_as_f64 is deleted. The per-value coercion + unreachable! (walkforward.rs ~:254) is removed entirely; the schema-pass coercer in param_stability replaces it.

Components

• crates/aura-engine/src/walkforward.rs — the bulk. Struct defs (WindowRun, WalkForwardResult); walk_forward / walk_forward_with_threads signatures + bodies (+ the arity debug_assert); param_stability rewrite (schema-pass + type-blind loop); delete scalar_as_f64. Imports gain ParamSpec; the chosen_params path no longer needs Scalar. The in-file test fixtures migrate (see Testing strategy).
• crates/aura-cli/src/main.rs — walkforward_family: compute space once at the top, pass it to walk_forward, simplify the closure to chosen_params: best.params. The other WalkForwardResult readers in this file (result.windows → oos_report, result.stitched_oos_equity → total pips) are unaffected — they never touch chosen_params or space.
• crates/aura-registry/src/lineage.rs — reads result.windows[..].run.oos_report only; unaffected (verified: no chosen_params / space reader). Listed so the planner confirms it during implement.

Data flow

• Kinds (schema): caller bp.param_space() → walk_forward(space, …) → moved onto WalkForwardResult.space. Computed once; identical for every window (same blueprint).
• Points (values): run-window closure → best.params (Vec<Cell>, the sweep winner) → WindowRun.chosen_params. Tag-free; never carries a kind.
• Reduction: param_stability reads kinds from result.space (once per slot → coercer table) and values from result.windows[*].run.chosen_params[slot] (per window) → MetricStats::from_values.
• Named view (existing idiom, on demand): zip_params(&result.space, &chosen_params) recombines slot names with typed values — the same mechanism SweepFamily::named_params uses.

Error handling

• Arity coupling: a debug_assert in walk_forward_with_threads checks every window's chosen_params.len() == space.len(). No release-mode runtime error variant is added — a mismatch is a wiring bug (caller passed the wrong blueprint's space), surfaced like the engine's other "checked at wiring" violations.
• Non-numeric slot: Bool is handled (→ 0/1), not an error. Timestamp in a param slot is structurally impossible (C20: a timestamp is a structural axis, never a numeric knob), so the schema pass carries a single unreachable! on that arm — reached once per slot at most, never per window-value. This replaces the old per-value unreachable!.
• No new CompileError / runtime-error variants; no change to any error type.

Testing strategy

Behaviour-preserving (C1): the migration re-spells fixtures, not values.

• Existing tests stay green, adapted to the new shapes:
  • walk_forward_runs_one_window_per_split_in_roll_order and walk_forward_is_deterministic_across_thread_counts — thread the new space argument into the walk_forward / walk_forward_with_threads calls; the run-window fixture's WindowRun now yields chosen_params: Vec<Cell> (Scalar::i64/f64(x) → Cell::from_i64/from_f64(x)). Window bounds / determinism assertions are unchanged.
  • param_stability_reduces_chosen_params_per_slot — the WindowRun literals use Cell points and the WalkForwardResult literal gains a two-slot space fixture (ParamSpec { name, kind: I64 }, ParamSpec { name, kind: F64 }). The asserted MetricStats are identical (the I64→f64 / F64 coercions are value-identical to the deleted scalar_as_f64).
• One genuinely new behaviour — the Bool → 0/1 arm. No built-in grid produces a Bool param, so add a focused param_stability test with a Bool slot in space and Cell::from_bool points across windows, asserting mean = the fraction of true windows. This keeps the new coercer arm non-dead and tested.
• The four project gates: cargo build --workspace --all-targets; cargo test --workspace; cargo clippy --workspace --all-targets -- -D warnings; cargo doc --workspace --no-deps — all clean.

Acceptance criteria

1. WindowRun.chosen_params is Vec<Cell>; WalkForwardResult has space: Vec<ParamSpec>; neither struct gains a serde derive.
2. walk_forward and walk_forward_with_threads take space: Vec<ParamSpec> by value; the Fn(WindowBounds) -> WindowRun + Sync bound is unchanged; run_indexed is unchanged.
3. walkforward_family computes the space once and passes chosen_params: best.params (no from_cell reconstruction).
4. param_stability reads result.space, builds a per-slot coercer before a type-blind reduction loop, and contains no per-value unreachable!; scalar_as_f64 is deleted. Its public signature is unchanged.
5. The arity debug_assert is present in walk_forward_with_threads.
6. The three existing walk-forward tests are green with identical MetricStats; the new Bool-slot param_stability test is green.
7. All four gates clean.
8. Spot-check greps: scalar_as_f64 absent from crates/; no unreachable! in a per-value path in walkforward.rs; WindowRun.chosen_params: Vec<Cell>; WalkForwardResult has space: Vec<ParamSpec>.