Walk-forward orchestrator: rolling in-sample/out-of-sample windows to a WalkForwardResult (C12 axis 3) #69

New Issue

2026-06-14T23:08:29+02:00

Brummel commented

2026-06-14 23:08:29 +02:00

C12 axis 3: walk-forward — the most structurally novel of the family axes, because it varies the data window, not a param or a seed. A window roller produces rolling (in-sample, out-of-sample) splits over the data window (C12); each split bootstraps a harness on its in-sample slice (optionally optimizing params there, reusing axis 2), then evaluates out-of-sample; the OOS pip-equity segments stitch into a continuous out-of-sample curve, plus a param-stability summary across windows.

Build

A window roller (anchored / rolling; configurable in-sample / out-of-sample lengths + step).
Per-window bootstrap (each window is a disjoint C1 unit).
WalkForwardResult: stitched OOS equity + per-window chosen params + a stability metric.
CLI surface (aura walkforward).

Depends on

Optimize/argmax (axis 2) if the in-sample step selects params. Independent of MC.

Acceptance

A known multi-window split produces deterministic per-window results and a correctly-stitched OOS curve, with no look-ahead across the IS/OOS boundary (C2).

Contracts

C12 axis 3, C21, C2 (the IS/OOS split is the structural anti-look-ahead boundary).

C12 axis 3: walk-forward — the most structurally novel of the family axes, because it varies the *data window*, not a param or a seed. A window roller produces rolling (in-sample, out-of-sample) splits over the data window (C12); each split bootstraps a harness on its in-sample slice (optionally optimizing params there, reusing axis 2), then evaluates out-of-sample; the OOS pip-equity segments stitch into a continuous out-of-sample curve, plus a param-stability summary across windows. ## Build - A window roller (anchored / rolling; configurable in-sample / out-of-sample lengths + step). - Per-window bootstrap (each window is a disjoint C1 unit). - `WalkForwardResult`: stitched OOS equity + per-window chosen params + a stability metric. - CLI surface (`aura walkforward`). ## Depends on - Optimize/argmax (axis 2) *if* the in-sample step selects params. Independent of MC. ## Acceptance A known multi-window split produces deterministic per-window results and a correctly-stitched OOS curve, with no look-ahead across the IS/OOS boundary (C2). ## Contracts C12 axis 3, C21, C2 (the IS/OOS split is the structural anti-look-ahead boundary).

Brummel added this to the The World, part II — orchestration families milestone 2026-06-14 23:08:29 +02:00

Brummel added the feature label 2026-06-14 23:08:29 +02:00

Brummel referenced this issue

2026-06-15 09:26:04 +02:00

engine+ingest: realize the Source trait as the run() ingestion seam, co-designed with a real DataServer-backed source (Ledger 0004/0011) #71

Brummel commented

2026-06-15 09:27:04 +02:00

Design constraint (World-II eager-agnostic firewall — PRIMARY — refs #71)

Walk-forward is the only milestone axis that varies the data window, at the worst scale (a 20y history is the ~190 GB eager object). Its public surface must carry TIME-BOUNDS, not data slices:

The window roller is a pure iterator of WindowBounds { is: (Timestamp, Timestamp), oos: (Timestamp, Timestamp) } — it never holds or indexes tick data, so its type cannot encode a materialized history. NO WindowRoller::new(history: Vec<...>) + &history[lo..hi] slicing.
Per-window data comes from a re-runnable producer keyed by bounds: make_source: Fn(Timestamp, Timestamp) -> impl Source — which maps 1:1 onto DataServer::stream_*_windowed(symbol, from, to). A first impl MAY (from..to).collect() internally.
The no-look-ahead acceptance (C2) is then a pure bounds invariant (oos.0 > is.1), checkable with zero ticks materialized.

requires_lazy_at_scale = true. Depends on #71.

## Design constraint (World-II eager-agnostic firewall — PRIMARY — refs #71) Walk-forward is the only milestone axis that varies the data window, at the worst scale (a 20y history is the ~190 GB eager object). Its public surface must carry TIME-BOUNDS, not data slices: - The window roller is a pure iterator of `WindowBounds { is: (Timestamp, Timestamp), oos: (Timestamp, Timestamp) }` — it never holds or indexes tick data, so its type cannot encode a materialized history. NO `WindowRoller::new(history: Vec<...>)` + `&history[lo..hi]` slicing. - Per-window data comes from a re-runnable producer keyed by bounds: `make_source: Fn(Timestamp, Timestamp) -> impl Source` — which maps 1:1 onto `DataServer::stream_*_windowed(symbol, from, to)`. A first impl MAY `(from..to).collect()` internally. - The no-look-ahead acceptance (C2) is then a pure bounds invariant (`oos.0 > is.1`), checkable with zero ticks materialized. requires_lazy_at_scale = true. Depends on #71.

Brummel commented

2026-06-15 14:54:24 +02:00

Design reconciliation (specify, in-context entry)

Spec: 0044-walk-forward-family. The fork below is named only loosely in this
issue ("a param-stability summary across windows" / "a stability metric"); the
shape of that summary was resolved in the in-context design discussion.

Fork: param-stability summary — stored field vs on-demand reduction, and which
statistic. → On-demand reduction, not a stored field. WalkForwardResult
stores only the raw per-window outcomes (each carrying its chosen_params) plus
the stitched OOS curve. param_stability(&WalkForwardResult) -> Vec<MetricStats>
is a public on-demand reduction over the retained per-window chosen params,
reusing the Monte-Carlo family's MetricStats (mean + type-7 quantiles) per
slot.
Rationale (substantive, not effort): a stored summary is a pure reduction over
the raw windows — recomputable, hence redundant — and caching one statistic
forces it canonical when "stability" admits several (a quantile distribution, a
moment std/CV, IQR, a distinct-value count). This mirrors SweepFamily, which
stores raw points and computes optimize/rank_by on demand (never cached),
rather than McFamily, whose stored aggregate is the axis's headline. The raw
chosen_params stay the substrate for any measure the caller wants.
Provenance: user decision "R2" on 2026-06-15 in the /boss specify session. The
user drove the reduction explicitly — first "Kann B [std/CV] nicht aus A
[MetricStats] abgeleitet werden?", then "Genügt A, oder ist es sogar selbst
redundant?", then chose R2 = raw-only result + on-demand param_stability
helper.

## Design reconciliation (specify, in-context entry) Spec: `0044-walk-forward-family`. The fork below is named only loosely in this issue ("a param-stability summary across windows" / "a stability metric"); the *shape* of that summary was resolved in the in-context design discussion. - **Fork: param-stability summary — stored field vs on-demand reduction, and which statistic.** → **On-demand reduction, not a stored field.** `WalkForwardResult` stores only the raw per-window outcomes (each carrying its `chosen_params`) plus the stitched OOS curve. `param_stability(&WalkForwardResult) -> Vec<MetricStats>` is a **public on-demand** reduction over the retained per-window chosen params, reusing the Monte-Carlo family's `MetricStats` (mean + type-7 quantiles) per slot. Rationale (substantive, not effort): a stored summary is a pure reduction over the raw windows — recomputable, hence redundant — and caching one statistic forces it canonical when "stability" admits several (a quantile distribution, a moment std/CV, IQR, a distinct-value count). This mirrors `SweepFamily`, which stores raw points and computes `optimize`/`rank_by` on demand (never cached), rather than `McFamily`, whose stored aggregate *is* the axis's headline. The raw `chosen_params` stay the substrate for any measure the caller wants. Provenance: user decision "R2" on 2026-06-15 in the `/boss` specify session. The user drove the reduction explicitly — first "Kann B [std/CV] nicht aus A [MetricStats] abgeleitet werden?", then "Genügt A, oder ist es sogar selbst redundant?", then chose R2 = raw-only result + on-demand `param_stability` helper.

Brummel referenced this issue from a commit

2026-06-15 15:28:02 +02:00

spec: 0044 walk-forward family (C12 axis 3)

Brummel referenced this issue from a commit

2026-06-15 15:28:02 +02:00

plan: 0044 walk-forward family (C12 axis 3)

Brummel referenced this issue from a commit

2026-06-15 15:28:02 +02:00

feat(aura-engine): walk-forward family — WindowRoller + walk_forward (C12 axis 3)

Brummel referenced this issue from a commit

2026-06-15 15:28:02 +02:00

audit: cycle 0044 — drift-clean (walk-forward family)