# Walk-forward orchestration family (C12 axis 3) — Design Spec

**Date:** 2026-06-15
**Status:** Approved — user sign-off (R2, 2026-06-15)
**Authors:** orchestrator + Claude

## Goal

Deliver C12's third orchestration axis: **walk-forward**. Unlike the grid sweep
(axis 1, varies a param) and Monte-Carlo (axis 4, varies the seed), walk-forward
varies the **data window** — it rolls (in-sample, out-of-sample) splits over a
time span, runs a disjoint harness per split (optionally optimizing params on the
in-sample slice, reusing axis 2), and stitches the out-of-sample pip-equity
segments into one continuous out-of-sample curve. The result carries the
per-window **chosen params** as the stability substrate; a per-parameter
stability summary is an **on-demand reduction** over them (a provided helper),
**not** a stored field — see Architecture.

The axis is realized **eager-agnostically** (the #71 firewall): the public
surface carries **time-bounds, never tick data**. A 20-year history is the
~190 GB eager object; the walk-forward surface must not be able to encode a
materialized history. The window roller is a pure iterator of `WindowBounds`;
per-window data comes from a bounds-keyed producer closure. The C2 no-look-ahead
guarantee is then a pure bounds invariant (`oos.0 > is.1`), checkable with zero
ticks materialized.

## Architecture

A new `walkforward` module in `aura-engine`, alongside `sweep` and `mc`. It owns
three things the other two axes' modules already model: **enumeration** (the
`WindowRoller`, analog to `GridSpace`), **execution** (the `walk_forward`
function, reusing the shared `run_indexed` disjoint-parallel core — C1), and
**collection** (`WalkForwardResult`, analog to `SweepFamily`/`McFamily`).

The orchestrator owns rolling + disjoint execution + stitching. It
delegates the per-window run — build/bootstrap/run, and the *optional* in-sample
param optimization — to a caller-supplied closure, exactly as `sweep` and `mc`
delegate the per-unit run to `run_one`. This is forced, not chosen:

- **Crate separation (C9) + dependency direction.** `aura-engine` is the engine;
  `aura-registry` depends on `aura-engine` and owns `optimize` (axis 2). The
  walk-forward orchestrator lives in `aura-engine` (it reuses the `pub(crate)`
  `run_indexed`), so it **cannot** call `optimize` directly without inverting the
  dependency. The in-sample selection is therefore a closure concern: the caller
  (the CLI / a World, both of which depend on both crates) supplies `optimize` as
  the in-sample step.
- **C12 forbids baking search policy into the primitive.** "Optimization" is a
  pluggable policy atop the atomic unit, not part of the orchestration primitive.
  A closure-supplied in-sample step is the contract C12 already mandates.

The in-sample/out-of-sample data both flow through a bounds-keyed producer
`Fn(Timestamp, Timestamp) -> impl Source` living **inside** the caller's closure
— which maps onto the existing `aura-ingest` method
`DataServer::stream_m1_windowed(symbol, from: Option<i64>, to: Option<i64>)`
(`aura-ingest/src/lib.rs`; per-bound `Option` windows) for real-data production.
No materialized stream `Vec` crosses the `walk_forward` API boundary (#71). The
engine code this iteration adds never calls that method — it is the production
mapping the closure body fulfils; the built-in CLI demo uses an **in-memory
windowed source** instead (mirroring `run_sweep`'s `showcase_prices`), so the
iteration adds no new `aura-ingest` dependency.

**Param-stability is on-demand, not a stored field — and this is the load-bearing
shape decision (resolved with the user, see the #69 reconciliation comment).**
The result stores only the raw per-window outcomes + the stitched curve, exactly
as a `SweepFamily` stores only raw points and `optimize`/`rank_by` are computed on
demand (never cached). The raw per-window `chosen_params` *are* the stability
substrate; a stored summary would be a pure reduction over them — recomputable,
hence redundant — and would force one statistic to be canonical when "stability"
admits several (the MC `MetricStats` distribution, a moment std/CV, IQR, a
distinct-value count). So the result caches none of them. A convenience reduction
ships as the **public on-demand fn `param_stability(&WalkForwardResult) ->
Vec<MetricStats>`**, which reuses the Monte-Carlo family's `MetricStats` (mean +
type-7 quantiles) per param slot; the caller is free to compute any other measure
from the same `windows[*].run.chosen_params`. The `MetricStats` reduction is
extracted from `McAggregate::from_draws` into a shared
`MetricStats::from_values(&[f64])` (a behaviour-preserving refactor; the seven
existing `mc` tests are the green guard) so both the MC aggregate and this helper
share one reduction.

## Concrete code shapes

### Worked author example (the user-facing program — acceptance evidence)

A World author validates a strategy out-of-sample by composing the roller, the
axis-2 optimizer, and a per-window run inside one `walk_forward` call:

```rust
use aura_engine::{param_stability, walk_forward, RollMode, WindowBounds, WindowRoller, WindowRun, Timestamp};
use aura_registry::optimize;

// A bounds-keyed producer: for real data, maps onto the method
// DataServer::stream_m1_windowed(symbol, Option<i64>, Option<i64>). The SURFACE is
// bounds; a first cut MAY collect (from..to) internally (#71 firewall: the *type*
// cannot encode a materialized history, the impl may be eager for now).
let make_source = |from: Timestamp, to: Timestamp| -> MyWindowedSource { /* ... */ };

// 1 year in-sample, 3 months out-of-sample, stepping 3 months, rolling
// (fixed-length IS). Anchored (growing IS) is RollMode::Anchored.
let roller = WindowRoller::new(span, IS_LEN, OOS_LEN, STEP, RollMode::Rolling)?;

let result = walk_forward(roller, |w: WindowBounds| -> WindowRun {
    // in-sample: sweep a grid, pick the best params (axis 2 INSIDE axis 3)
    let is_family = grid.sweep(|pt| run_point(pt, make_source(w.is.0, w.is.1)));
    let best = optimize(&is_family, "total_pips").expect("known metric");
    // out-of-sample: run the chosen params on the held-out window, record equity
    let (oos_equity, oos_report) = run_and_record(&best.params, make_source(w.oos.0, w.oos.1));
    WindowRun { chosen_params: best.params, oos_equity, oos_report }
});
// Runs are disjoint across windows (C1); no look-ahead by construction
// (C2: every w.oos.0 > w.is.1, guaranteed by the roller).

result.stitched_oos_equity;          // one continuous OOS pip curve across all windows
result.windows[0].run.chosen_params; // raw per-window chosen params (the stability substrate)
param_stability(&result)[0];         // on-demand: MetricStats of param 0 across windows
```

CLI surface (mirrors `aura sweep`):

```
$ aura walkforward
{"manifest":{...,"window":[<oos0_from>,<oos0_to>],...},"metrics":{...}}   # window 0 OOS record
{"manifest":{...,"window":[<oos1_from>,<oos1_to>],...},"metrics":{...}}   # window 1 OOS record
...
{"walkforward":{"windows":N,"stitched_total_pips":X.0,"param_stability":[{"mean":..,"p5":..,...}]}}
```

Each per-window OOS `RunReport` is persisted to the run registry (C18) and printed
as one JSON line; a final summary line carries the window count, the stitched
total, and the per-param stability.

### Implementation shapes (secondary)

New, in `crates/aura-engine/src/walkforward.rs`:

```rust
/// One rolling split's time bounds: an in-sample window and the out-of-sample
/// window that follows it. Carries ONLY bounds — never tick data (#71). C2
/// no-look-ahead is a pure invariant on these: oos.0 > is.1. Bounds are
/// inclusive (from, to) epoch-unit timestamps, matching RunManifest.window.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct WindowBounds {
    pub is: (Timestamp, Timestamp),
    pub oos: (Timestamp, Timestamp),
}

/// Rolling (fixed-length IS, start advances by `step`) vs anchored (IS always
/// starts at the span origin and grows; OOS positions identical to rolling).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum RollMode { Rolling, Anchored }

/// A pure iterator of WindowBounds over a [origin, end] span — never holds or
/// indexes tick data (#71). Window k (k = 0, 1, ...):
///   oos_start_k = origin + is_len + k*step
///   is  = (origin + k*step (Rolling) | origin (Anchored),  oos_start_k - 1)
///   oos = (oos_start_k,  oos_start_k + oos_len - 1)
/// Emitted while oos.1 <= end. Every split satisfies oos.0 == is.1 + 1 > is.1 (C2).
pub struct WindowRoller { /* span, is_len, oos_len, step, mode, k */ }

impl WindowRoller {
    /// Validate config: is_len/oos_len/step must be > 0; the span must fit at
    /// least window 0 (origin + is_len + oos_len - 1 <= end). Else WalkForwardError.
    pub fn new(
        span: (Timestamp, Timestamp),
        is_len: i64,
        oos_len: i64,
        step: i64,
        mode: RollMode,
    ) -> Result<WindowRoller, WalkForwardError>;
}

impl Iterator for WindowRoller {
    type Item = WindowBounds;
    fn next(&mut self) -> Option<WindowBounds>;
}

/// A structural fault configuring a walk-forward — the typed gate before any run,
/// analog to SweepError.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum WalkForwardError {
    /// is_len / oos_len / step was <= 0.
    NonPositiveLength { field: &'static str, value: i64 },
    /// The span is too short to fit even window 0.
    SpanTooShort { span: (Timestamp, Timestamp), need: i64 },
}

/// What a per-window run yields: the params chosen on the in-sample window, the
/// recorded OOS pip-equity segment (for stitching), and the OOS RunReport (the
/// per-window C18 record). Self-describing, analog to SweepPoint/McDraw.
#[derive(Clone, Debug, PartialEq)]
pub struct WindowRun {
    pub chosen_params: Vec<Scalar>,
    pub oos_equity: Vec<(Timestamp, f64)>,
    pub oos_report: RunReport,
}

/// One completed window: its bounds + the run the closure produced, in roll order.
#[derive(Clone, Debug, PartialEq)]
pub struct WindowOutcome {
    pub bounds: WindowBounds,
    pub run: WindowRun,
}

/// The result family of a walk-forward. Analog to SweepFamily — it stores ONLY
/// the raw per-window outcomes plus the stitched curve; any summary over the
/// windows (param stability, etc.) is an on-demand reduction, NOT a stored field
/// (just as SweepFamily stores raw points and optimize/rank_by are computed on
/// demand). Non-redundant by construction.
#[derive(Clone, Debug, PartialEq)]
pub struct WalkForwardResult {
    /// Per-window outcomes in roll order (window 0 first), independent of thread
    /// completion (C1). Each carries its bounds + the chosen params (the stability
    /// substrate) + the OOS equity segment + the OOS RunReport.
    pub windows: Vec<WindowOutcome>,
    /// The stitched continuous OOS pip-equity curve: each window's OOS segment
    /// offset by the running sum of all prior segments' final cumulative values,
    /// so equity carries forward across the IS/OOS boundaries (continuous, not a
    /// per-window sawtooth). An empty OOS segment contributes 0.0 to the running
    /// offset (mirroring summarize's unwrap_or(0.0) for an empty curve) and adds
    /// no points, so an equity-less window leaves the curve unbroken. For the
    /// canonical step == oos_len tiling the segment timestamps are also contiguous.
    pub stitched_oos_equity: Vec<(Timestamp, f64)>,
}

/// On-demand param-stability summary: one MetricStats per param slot, over the
/// chosen values (Scalar coerced to f64) across all windows — reuses the MC
/// family's MetricStats + type-7 quantile via MetricStats::from_values. A pure
/// reduction over result.windows[*].run.chosen_params, NOT stored on the result
/// (sweep-precedent: a summary is computed on demand, not cached), so it stays
/// non-redundant and the caller can compute any other measure (std, IQR,
/// distinct-count) from the same raw substrate. One entry per param slot; empty
/// if the strategy has no params. Slot count is the first window's arity (all
/// windows of one blueprint share the param-space arity, C11).
pub fn param_stability(result: &WalkForwardResult) -> Vec<MetricStats>;

/// Roll the windows, run `run_window` on each disjointly in parallel (C1, via the
/// shared run_indexed core), then stitch the OOS equity into one continuous curve.
/// The varying dimension is the DATA WINDOW (C12 axis 3). Eager-agnostic (#71):
/// run_window receives only WindowBounds; its data comes from a bounds-keyed
/// producer in the closure body, never a materialized stream Vec in this API.
/// Precondition: the roller yields >= 1 window (WindowRoller::new rejects an empty
/// roll). Param stability is a separate on-demand reduction — param_stability(&result)
/// — never computed here.
pub fn walk_forward<F>(roller: WindowRoller, run_window: F) -> WalkForwardResult
where
    F: Fn(WindowBounds) -> WindowRun + Sync;

// Module-private thread-count-explicit core (walk_forward_with_threads), as in
// sweep/mc, so tests pin determinism at 1 and at N threads (C1).
```

Refactor, in `crates/aura-engine/src/mc.rs` (behaviour-preserving):

```rust
impl MetricStats {
    /// Mean + the fixed type-7 quantile set over a value set. Sorts a copy;
    /// values must be finite and non-empty. The shared reduction behind both the
    /// MC aggregate (a metric across draws) and the walk-forward param-stability
    /// summary (a param across windows).
    pub fn from_values(values: &[f64]) -> MetricStats {
        let mut xs = values.to_vec();
        xs.sort_by(|a, b| a.partial_cmp(b).expect("values are finite"));
        MetricStats {
            mean: xs.iter().sum::<f64>() / xs.len() as f64,
            p5: quantile(&xs, 0.05),
            p25: quantile(&xs, 0.25),
            p50: quantile(&xs, 0.50),
            p75: quantile(&xs, 0.75),
            p95: quantile(&xs, 0.95),
        }
    }
}
// McAggregate::from_draws' per-metric `pick` collects the metric's values, then
// calls MetricStats::from_values(&xs) — same numbers, one reduction. quantile
// stays module-private (from_values is in mc.rs).

// MetricStats also gains serde::{Serialize, Deserialize} (consistent with
// RunMetrics/RunManifest/RunReport) so the CLI can render the stability summary
// (C14) and a later registry-lineage cycle (#70) can persist it.
```

Exports, in `crates/aura-engine/src/lib.rs`:

```rust
mod walkforward;
pub use walkforward::{
    param_stability, walk_forward, RollMode, WalkForwardError, WalkForwardResult,
    WindowBounds, WindowOutcome, WindowRoller, WindowRun,
};
```

CLI, in `crates/aura-cli/src/main.rs`:

```rust
// dispatch (the strict full-vector match):
["walkforward"] => run_walkforward(),
// USAGE gains "| aura walkforward".

/// `aura walkforward`: run a built-in rolling walk-forward over the sample
/// blueprint + a synthetic windowed source; per window, sweep the built-in grid
/// on the in-sample slice, optimize by total_pips (axis 2 inside axis 3, where
/// aura-cli bridges engine + registry), run the chosen params out-of-sample,
/// persist the OOS RunReport (C18), print it, and finally print the stitched
/// summary. Deterministic (C1).
fn run_walkforward() { /* mirrors run_sweep */ }
```

## Components

- **`WindowRoller`** (new, `walkforward.rs`) — bounds-only iterator; the C12
  axis-3 enumeration. Validated at construction (`WalkForwardError`). Pure: no
  tick data, no I/O.
- **`walk_forward` / `walk_forward_with_threads`** (new, `walkforward.rs`) —
  collects the roller's bounds (tiny: 4 timestamps each), runs `run_window` over
  them via `run_indexed` (C1 disjoint parallel, roll-order output), then stitches
  and summarizes.
- **`stitch` helper** (new, private, `walkforward.rs`) — pure reduction folding
  the per-window OOS segments into `stitched_oos_equity` (running cumulative
  offset; an empty segment adds +0.0 and no points).
- **`param_stability`** (new, **public on-demand fn**, `walkforward.rs`) — pure
  reduction over `result.windows[*].run.chosen_params`; coerces `Scalar` to `f64`
  (mirroring the CLI's existing `scalar_as_param_f64`) and calls
  `MetricStats::from_values` per slot. NOT stored on the result.
- **`MetricStats::from_values`** (extracted, `mc.rs`) — the shared sort + mean +
  type-7-quantile reduction; `McAggregate::from_draws` is rewired to it.
- **`run_walkforward`** (new, `main.rs`) — the CLI surface; the place the
  engine→registry layering is bridged (it calls `aura_registry::optimize`).

## Data flow

1. `WindowRoller::new` validates config → an iterator of `WindowBounds`.
2. `walk_forward` collects the bounds (firewall-safe: bounds only), then
   `run_indexed(n, nthreads, |i| run_window(bounds[i]))` runs each window
   disjointly, results sorted back to roll order (C1).
3. Inside `run_window` (caller's): `make_source(is.0, is.1)` → in-sample run(s) →
   `optimize` picks params → `make_source(oos.0, oos.1)` → OOS run → records the
   OOS equity stream + builds the OOS `RunReport`.
4. `walk_forward` assembles `WindowOutcome`s and stitches the OOS segments into
   `stitched_oos_equity` (running cumulative offset; empty segment → +0.0). Param
   stability is **not** computed here — it is the on-demand `param_stability(&result)`
   reduction a caller invokes when wanted.
5. CLI: each OOS `RunReport` → registry append (C18) + stdout line; the final
   summary line calls `param_stability(&result)` for its per-param block.

## Error handling

- **Config faults** are typed and caught before any run: `WalkForwardError`
  (`NonPositiveLength`, `SpanTooShort`) from `WindowRoller::new` — the same
  "typed gate before execution" shape as `SweepError`. No look-ahead is not an
  error path: it is structurally impossible (the roller only ever emits
  `oos.0 = is.1 + 1`).
- **`walk_forward` returns the family directly** (no `Result`), matching
  `sweep`/`mc`: the roller is pre-validated (non-empty by construction) and the
  per-window closure owns its own fallibility. A `debug_assert!(!bounds.is_empty())`
  guards the precondition, mirroring `monte_carlo`.
- **CLI registry I/O** failure → `eprintln!` + `exit(2)`, identical to `run_sweep`.

## Testing strategy

RED-first. New tests pin each load-bearing property; the `mc` refactor is guarded
by its seven existing tests staying green.

`walkforward.rs`:

1. `roller_rolling_emits_consecutive_oos_windows` — known span/lengths: window
   count and `is`/`oos` bounds match the rolling formula; OOS windows tile
   contiguously when `step == oos_len`.
2. `roller_anchored_fixes_is_start_grows_is_end` — Anchored: every `is.0 == origin`,
   `is.1` grows by `step`; OOS positions identical to Rolling.
3. `roller_every_split_has_no_lookahead` — C2 bounds invariant: for every emitted
   `WindowBounds`, `oos.0 > is.1` (zero ticks materialized).
4. `roller_rejects_nonpositive_lengths_and_short_span` — `WalkForwardError` on
   `is_len<=0`, `oos_len<=0`, `step<=0`, and a span too short for window 0.
5. `walk_forward_runs_one_window_per_split_in_roll_order` — N bounds → N outcomes,
   `windows[k].bounds` in roll order (window 0 first).
6. `walk_forward_is_deterministic_across_thread_counts` — `walk_forward_with_threads`
   at 1 and at 8 are identical, and equal the public `walk_forward` (C1: order is
   roll order, not completion order).
7. `stitched_curve_carries_equity_forward` — known per-window OOS segments stitch
   with a running offset: segs `[(t0,2),(t1,5)]` then `[(t2,1),(t3,3)]` →
   `[(t0,2),(t1,5),(t2,6),(t3,8)]` (continuous; no reset to 1 at t2).
7b. `stitched_curve_passes_through_empty_segment` — an empty OOS segment adds 0.0
   to the running offset and no points: segs `[(t0,2),(t1,5)]`, `[]`, `[(t2,1)]`
   → `[(t0,2),(t1,5),(t2,6)]` (offset after the empty window stays 5, curve
   unbroken).
8. `param_stability_reduces_chosen_params_per_slot` — the **on-demand**
   `param_stability(&result)` returns one `MetricStats` per slot over the coerced
   chosen values across windows (computed from `result.windows`, not a stored
   field); known fixture (e.g. slot 0 values `[2,2,3,3]` → `mean 2.5`, `p50 2.5`).

`mc.rs`:

9. `metric_stats_from_values_matches_known_fixture` — `MetricStats::from_values(&[0,1,2,3,4])`
   → `mean 2.0`, `p50 2.0`, `p5 ≈ 0.2`, `p95 ≈ 3.8` (the same numbers the existing
   aggregate test pins, now directly on the extracted fn). Existing 7 tests stay green.

`main.rs`:

10. `walkforward_report_is_deterministic` — the built-in WFO render is byte-identical
    across two calls (C1).
11. `walkforward_report_has_one_oos_line_per_window_plus_summary` — output is
    `N` per-window OOS `RunReport` JSON lines followed by one `{"walkforward":...}`
    summary line, for the built-in `N`-window roll.

Gates: `cargo test --workspace`, `cargo clippy --workspace --all-targets -D warnings`,
`cargo doc -p aura-engine --no-deps`.

## Acceptance criteria

Applying aura's feature-acceptance criterion (CLAUDE.md), prospectively:

- **The audience naturally reaches for it.** Walk-forward is a named C12 axis and
  the canonical out-of-sample robustness validation; the worked example shows a
  World author composing roller + `optimize` + run in one call. ✓
- **Measurably improves correctness / removes redundancy.** It adds the only
  axis that varies the data window (no current way to do rolling IS/OOS), reuses
  `run_indexed`, `optimize`, and `MetricStats` rather than duplicating, and the
  shared `MetricStats::from_values` extraction removes the duplicated reduction.
  The result stores only raw per-window outcomes + the stitched curve — every
  summary (param stability, std, IQR) is an on-demand reduction over the retained
  substrate (sweep-precedent), so nothing recomputable is cached. ✓
- **Reintroduces no eliminated failure class.** C1 (disjoint parallel via
  `run_indexed`, roll-order output, 1-vs-N determinism pinned); C2 (no look-ahead
  is structural — the roller only emits `oos.0 > is.1`, checkable with zero
  ticks); #71 firewall (the API carries bounds + a closure, never a materialized
  stream `Vec`); C9 (engine stays free of a registry dependency — `optimize` is
  closure-supplied). ✓

Concretely, the issue's acceptance: **a known multi-window split produces
deterministic per-window results (tests 5, 6, 10) and a correctly-stitched OOS
curve (test 7), with no look-ahead across the IS/OOS boundary (test 3, C2).**
