Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

audit(0097): cycle close — IS-refit walk-forward over a loaded blueprint; ledger refreshed (C24/C18); rm ephemeral spec/plan

Browse Source 
cycle 0097 tidy (clean). Architect drift review over 485f3c8..HEAD: no code
drift, no debt. Regression: no dedicated script (architect is the sole gate);
its "What holds" is non-empty (design commitments confirmed against the diff):
- Inv. 8 / 9: the cycle touches only crates/aura-cli (main.rs + cli_run.rs +
  the ephemeral spec/plan); walk_forward/WindowRoller/FamilyKind::WalkForward/
  append_family/put_blueprint are read-only reuse, no engine/registry change.
- Inv. 1 / 2: blueprint_walkforward_family reuses the generic walk_forward roller
  (windows disjoint-parallel, across-sim); each IS-sweep/OOS-run sees only its
  window's windowed_sources; IS precedes OOS per window — no look-ahead.
- Reproduce lockstep: the reproduce_family_in WalkForward arm rebuilds each OOS
  slice from stored.manifest.window (before the _ => run_sources fallthrough that
  would rebuild the wrong full-window source); winner params via the shared
  manifest->cells recovery; pinned by the reproduces_bit_identically E2E.
- Structural mirror + panic-safety: one-substitution-deep twin of the hard-wired
  Stage1R arm (select_winner + walk_forward reused, metric sqn_normalized);
  run_blueprint_walkforward follows the content-addressed sibling pattern (no
  ensure_name_free); blueprint_sweep_over returns Result<_,BindError>, a bad axis
  is in-closure exit 2 (never a panic); run_oos_blueprint derives pip internally
  (7 args, no #[allow]).

Ledger: refreshed the C24 realization block (docs/design/INDEX.md) — replaced the
now-false "Walk-forward is deferred (#173, dep #169)" sentence with the "IS-refit
walk-forward shipped (cycle 0097)" note; and extended the C18 reproduction note —
reproduce now spans the third family kind (WalkForward), so all three FamilyKind
variants persist AND reproduce through the one shared topology_hash+put_blueprint
hook.

Milestone: the World/C21 blueprint-data family surface (run/sweep/mc/walkforward/
list-axes/reproduce) is coherent and complete per the architect review — no
structural gap. This audit proves drift-clean; a green end-to-end MILESTONE
FIELDTEST remains the deliberate gate to formally close the World/C21 milestone.
Ephemeral spec/plan git-rm'd per the lifecycle convention.

refs #173
This commit is contained in:
Brummel
2026-07-01 16:44:54 +02:00
parent ff20f74e8f
commit 4710fccf32
3 changed files with 17 additions and 890 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/INDEX.md
+17 -4
View File
@@ -1316,7 +1316,11 @@ synthetic (deterministic) data; recorded-dataset reproduction rides the DataServ
(#124). **Cycle 0095 (#170):** `aura reproduce` now also spans `FamilyKind::MonteCarlo` —
branching on the family kind, it reconstructs each member's seed-driven synthetic walk from
the recorded `manifest.seed` (the `Sweep` arm unchanged), so an `aura mc` family re-derives
bit-identically through the same `run_blueprint_member` path.
bit-identically through the same `run_blueprint_member` path. **Cycle 0097 (#173):** reproduce
spans the third variant, `FamilyKind::WalkForward` — the same branch rebuilds each OOS member's
windowed slice from `manifest.window` (winner params via the shared `manifest→cells` recovery),
so an `aura walkforward` family re-derives bit-identically too. All three family kinds now
persist *and* reproduce through the one shared `topology_hash`+`put_blueprint` hook.
### C19 — Bootstrap: blueprint → instance (recursive)
**Guarantee.** Construction is a distinct phase, recursive at every level. Each
@@ -1897,9 +1901,18 @@ draws disjoint-parallel via the engine `monte_carlo` seam (invariant 1) — and
`aura reproduce`s bit-identically (C1). MC binds no axis, so it needs a **closed** blueprint
(the sweep's open/closed distinction inverted); an open one returns a named `Err` (rendered
exit-2 at the `run_blueprint_mc` boundary, the sweep sibling's contract — no hidden exit in the
pure builder). **Walk-forward is deferred (#173, dep #169):** a loaded blueprint declares no
axes to re-fit per window, so it could only be a non-IS-refit windowed eval — which would
collide with the hard-wired `walkforward` verb's IS-optimizing guarantee. The synthetic-walk DGP
pure builder). **IS-refit walk-forward shipped (cycle 0097, #173).** `aura walkforward
<blueprint.json> --axis <name>=<csv> [--select argmax|plateau:mean|plateau:worst]` re-optimizes
the loaded blueprint's params over the user `--axis` grid (the #169 prefixed names) on each
24/12/12 IS window, selects the winner by `sqn_normalized` (the hard-wired arm's metric +
`select_winner` reused verbatim), runs it out-of-sample, and aggregates to a
`FamilyKind::WalkForward` family — persisted + content-addressed + `aura reproduce` bit-identical
(the read-side WalkForward branch rebuilds each OOS window from `manifest.window`). One
substitution deep from the hard-wired stage1-r WF arm (the loaded blueprint via
`blueprint_axis_probe` replaces the built-in strategy); a bad `--axis` is a clean in-closure
exit 2, never a panic. This is Arm A (the settled direction): the loaded IS-optimizing form
honestly carries the `walkforward` name. Reduce-mode members are R-measured (`oos_r` the
meaningful summary; stitched pip-equity empty, C10). The synthetic-walk DGP
is the machinery, not trader-grade MC statistics; a real-data block-bootstrap — and the
`synthetic_walk_sources` `len:60`↔warm-up coupling it retires (a deep-lookback closed blueprint
warms poorly → silent-vacuous draws today) — ride #172. **Axis-name discovery shipped (cycle
docs/plans/0097-is-refit-walk-forward-over-a-loaded-blueprint.md
-572
View File
@@ -1,572 +0,0 @@
# IS-refit walk-forward over a loaded blueprint — Implementation Plan
> **Parent spec:** `docs/specs/0097-is-refit-walk-forward-over-a-loaded-blueprint.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to
> run this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** Add `aura walkforward <blueprint.json> --axis <name>=<csv> [--axis …]` —
an in-sample-refit walk-forward over a loaded blueprint (re-optimize per IS
window, run the winner OOS), aggregated to a persisted, content-addressed,
reproduce-aware `FamilyKind::WalkForward` family.
**Architecture:** Mirror `walkforward_family`'s `Stage1R` arm one substitution
deep — reuse the generic engine `walk_forward` driver + `select_winner` +
the 24/12/12 synthetic schedule; the per-window IS-sweep and OOS-run source the
LOADED blueprint (via cycle-0096's `blueprint_axis_probe`) over the user `--axis`
grid, windowed by `data.windowed_sources`. Persist via `put_blueprint` +
`append_family` (like the loaded sweep/mc), and add the `reproduce_family_in`
`WalkForward` branch so the family reproduces bit-identically. All changes in
`crates/aura-cli`; engine/registry are read-only callees (invariants 1/2/8/9).
**Tech Stack:** `crates/aura-cli/src/main.rs` (six new fns + two modified sites),
`crates/aura-cli/tests/cli_run.rs` (E2Es). Read-only reuse: `walk_forward` /
`WindowRun` / `WalkForwardResult` (aura-engine), `FamilyKind::WalkForward` /
`walkforward_member_reports` / `put_blueprint` / `append_family` (aura-registry).
---
**Files this plan creates or modifies:**
- Modify: `crates/aura-cli/src/main.rs` — new `blueprint_sweep_over`,
`run_oos_blueprint`, `blueprint_walkforward_family`, `WfBlueprintArgs`,
`parse_walkforward_blueprint_args`, `run_blueprint_walkforward`; modified
`["walkforward", ..]` dispatch (~4175) + `reproduce_family_in` match (~2566).
- Test: `crates/aura-cli/src/main.rs` (`#[cfg(test)] mod tests` ~4238) — unit
`blueprint_walkforward_family` + unit `parse_walkforward_blueprint_args`.
- Test: `crates/aura-cli/tests/cli_run.rs` — 4 new E2Es beside the blueprint-family
block (~3269-3700).
Line numbers are current-tree recon anchors; match on surrounding code, not the
number. **Gate note:** Tasks 1-3 use `cargo build` + scoped `cargo test` (early
tasks add not-yet-wired fns, which are a build *warning*, not an error); the full
`cargo clippy -- -D warnings` + `cargo test --workspace` gate runs in Task 4,
after every fn is wired.
---
## Task 1: The family builder + its two windowed helpers
**Files:**
- Modify: `crates/aura-cli/src/main.rs` (new fns near `blueprint_sweep_family` ~3197 and `stage1_r_sweep_over` ~1329)
- Test: `crates/aura-cli/src/main.rs` (`mod tests`)
- [ ] **Step 1: Write the failing unit test**
Add to the `#[cfg(test)] mod tests` block (near `blueprint_axis_probe_lists_prefixed_open_knobs`):
```rust
#[test]
fn blueprint_walkforward_family_refits_each_window() {
// The open fixture's two SMA lengths are re-fit per IS window over a 2x2 grid.
let doc = include_str!("../tests/fixtures/stage1_signal_open.json");
let axes = vec![
("stage1_signal.fast.length".to_string(), vec![Scalar::i64(2), Scalar::i64(3)]),
("stage1_signal.slow.length".to_string(), vec![Scalar::i64(4), Scalar::i64(6)]),
];
let result = blueprint_walkforward_family(doc, &axes, &DataSource::Synthetic, Selection::Argmax);
// 24/12/12 over the 60-bar synthetic span -> 3 rolling windows.
assert_eq!(result.windows.len(), 3, "three rolling IS/OOS windows");
for w in &result.windows {
assert_eq!(w.run.chosen_params.len(), 2, "both axes re-fit each window");
assert!(w.run.oos_report.metrics.r.is_some(), "OOS record is R-metrics");
}
// reduce-mode retains no raw pip curve -> the stitched pip-equity is empty.
assert!(result.stitched_oos_equity.is_empty(), "no raw pip curve in reduce-mode");
}
```
- [ ] **Step 2: Run the test to verify it fails (RED — unresolved name)**
Run: `cargo test -p aura-cli --bin aura blueprint_walkforward_family`
Expected: FAIL to compile — `cannot find function 'blueprint_walkforward_family'`.
- [ ] **Step 3: Add `blueprint_sweep_over` (windowed IS-sweep, with lattice)**
Insert near `blueprint_sweep_family` (~3232). Twin of `stage1_r_sweep_over`
(uses `sweep_with_lattice`) crossed with `blueprint_sweep_family`'s loaded-member
seed-axes pattern, windowed by `[from,to]`. Returns `Result<_, BindError>` (user
axes are fallible — an unknown/kind-mismatched `--axis` returns `BindError`, not a
panic; the sweep terminal name/kind-checks):
```rust
/// Sweep the LOADED blueprint over the user `--axis` grid on an in-sample window
/// `[from,to]` — the windowed, lattice-carrying twin of `stage1_r_sweep_over` and
/// `blueprint_sweep_family`. `sweep_with_lattice` gives the grid lattice `--select
/// plateau` needs. An unknown/kind-mismatched axis surfaces as `BindError` at the
/// sweep terminal (no panic, no hidden exit) for the caller to render.
fn blueprint_sweep_over(
doc: &str, axes: &[(String, Vec<Scalar>)], from: Timestamp, to: Timestamp, data: &DataSource,
) -> Result<(SweepFamily, Vec<usize>), BindError> {
let reload = |d: &str| {
blueprint_from_json(d, &|t| std_vocabulary(t))
.expect("doc parse-validated at the dispatch boundary; reload is infallible")
};
let pip = data.pip_size();
let topo = topology_hash(&reload(doc));
let probe = blueprint_axis_probe(doc);
let space = probe.param_space();
let mut iter = axes.iter();
let (first_name, first_vals) = iter.next().expect("a blueprint walk-forward declares >= 1 axis");
let mut binder = probe.axis(first_name, first_vals.clone());
for (n, vals) in iter {
binder = binder.axis(n, vals.clone());
}
binder.sweep_with_lattice(|point| {
let sources = data.windowed_sources(from, to);
let window = window_of(&sources).expect("non-empty in-sample window");
run_blueprint_member(reload(doc), point, &space, sources, window, 0, pip, &topo)
})
}
```
- [ ] **Step 4: Add `run_oos_blueprint` (windowed OOS-run)**
Insert next to `blueprint_sweep_over`. Loaded-member analog of `run_oos_r`;
reduce-mode retains no raw pip curve, so the pip segment is empty:
```rust
/// Run the winner params over an out-of-sample window `[from,to]` on the loaded
/// blueprint — the loaded-member analog of `run_oos_r`. The reduce-mode member
/// (`run_blueprint_member`) retains R-metrics, not a raw pip curve, so the stitching
/// segment is empty (an empty segment leaves the stitched curve unbroken).
fn run_oos_blueprint(
doc: &str, params: &[Cell], space: &[ParamSpec], from: Timestamp, to: Timestamp,
pip: f64, topo: &str, data: &DataSource,
) -> (Vec<(Timestamp, f64)>, RunReport) {
let reload = blueprint_from_json(doc, &|t| std_vocabulary(t))
.expect("doc parse-validated at the dispatch boundary; reload is infallible");
let sources = data.windowed_sources(from, to);
let window = window_of(&sources).expect("non-empty out-of-sample window");
let report = run_blueprint_member(reload, params, space, sources, window, 0, pip, topo);
(Vec::new(), report)
}
```
- [ ] **Step 5: Add `blueprint_walkforward_family` (the per-window re-fit driver)**
Insert next to the two helpers. Structural twin of `walkforward_family`'s
`Stage1R` arm — reuses the generic `walk_forward` driver; the per-window closure
re-fits the loaded blueprint. In-closure errors `exit(2)` exactly as the Stage1R
arm does for `select_winner` (clean process exit, never a panic):
```rust
/// The loaded-blueprint IS-refit walk-forward: per IS window, re-optimize the
/// blueprint over the user `--axis` grid, select by `sqn_normalized`, run the
/// winner OOS. Structural twin of `walkforward_family`'s `Stage1R` arm — the
/// generic `walk_forward` driver + `select_winner` reused; only the per-window
/// sweep/OOS source the loaded blueprint. In-closure errors (a bad `--axis`)
/// `exit(2)` with the sweep terminal's message, as the hard-wired arm does.
fn blueprint_walkforward_family(
doc: &str, axes: &[(String, Vec<Scalar>)], data: &DataSource, select: Selection,
) -> WalkForwardResult {
let span = data.wf_full_span();
let (is_len, oos_len, step) = data.wf_window_sizes();
let roller = match WindowRoller::new(span, is_len, oos_len, step, RollMode::Rolling) {
Ok(r) => r,
Err(e) => {
eprintln!("aura: walk-forward window too short for one IS+OOS span: {e:?}");
std::process::exit(2);
}
};
let space = blueprint_axis_probe(doc).param_space();
let pip = data.pip_size();
let topo = topology_hash(&blueprint_from_json(doc, &|t| std_vocabulary(t))
.expect("doc parse-validated at the dispatch boundary; reload is infallible"));
walk_forward(roller, space.clone(), |w: WindowBounds| {
let (is_family, lattice) = blueprint_sweep_over(doc, axes, w.is.0, w.is.1, data)
.unwrap_or_else(|e| { eprintln!("aura: {e}"); std::process::exit(2); });
let (best, selection) = match select_winner(&is_family, "sqn_normalized", select, Some(&lattice)) {
Ok(v) => v,
Err(msg) => { eprintln!("aura: {msg}"); std::process::exit(2); }
};
let (oos_equity, mut oos_report) =
run_oos_blueprint(doc, &best.params, &space, w.oos.0, w.oos.1, pip, &topo, data);
oos_report.manifest.selection = Some(selection);
WindowRun { chosen_params: best.params, oos_equity, oos_report }
})
}
```
- [ ] **Step 6: Verify the unit test passes + the workspace builds**
Run: `cargo test -p aura-cli --bin aura blueprint_walkforward_family`
Expected: PASS (1 test — 3 windows, both axes re-fit, R-metrics present, empty curve).
Run: `cargo build --workspace`
Expected: `Finished` — 0 errors (unused-fn *warnings* for the not-yet-wired fns
are expected until Task 2 wires them; warnings are not errors).
---
## Task 2: The parse grammar, IO wrapper, and dispatch (the persisting verb)
**Files:**
- Modify: `crates/aura-cli/src/main.rs` (new `WfBlueprintArgs` + `parse_walkforward_blueprint_args` near `parse_sweep_blueprint_args` ~3939; new `run_blueprint_walkforward` near `run_blueprint_sweep` ~3301; dispatch ~4175)
- Test: `crates/aura-cli/src/main.rs` (`mod tests`)
- [ ] **Step 1: Write the failing unit test**
Add beside `parse_sweep_blueprint_args_grammar`:
```rust
#[test]
fn parse_walkforward_blueprint_args_grammar() {
let a = parse_walkforward_blueprint_args(&[
"--axis", "stage1_signal.fast.length=2,3", "--select", "plateau:mean", "--name", "wf",
]).expect("valid tail parses");
assert_eq!(a.axes.len(), 1);
assert_eq!(a.name, "wf");
assert!(matches!(a.select, Selection::Plateau(_)));
// a walk-forward with nothing to re-fit is rejected:
assert!(parse_walkforward_blueprint_args(&[]).is_err());
assert!(parse_walkforward_blueprint_args(&["--name", "wf"]).is_err());
// an unknown --select value is rejected:
assert!(parse_walkforward_blueprint_args(&["--axis", "a=1,2", "--select", "bogus"]).is_err());
// default name + default argmax:
let b = parse_walkforward_blueprint_args(&["--axis", "a=1,2"]).expect("minimal tail parses");
assert_eq!(b.name, "walkforward");
assert!(matches!(b.select, Selection::Argmax));
}
```
- [ ] **Step 2: Run the test to verify it fails (RED)**
Run: `cargo test -p aura-cli --bin aura parse_walkforward_blueprint_args`
Expected: FAIL to compile — `cannot find function 'parse_walkforward_blueprint_args'`.
- [ ] **Step 3: Add `WfBlueprintArgs` + `parse_walkforward_blueprint_args`**
Insert near `parse_sweep_blueprint_args` (~3987). Mirrors the sweep `--axis`
grammar + the `--select` parse from `parse_walkforward_args` (`parse_select`
~1614); requires ≥1 `--axis`. This is a content-addressed family verb (always
persists, like `aura mc`), so it takes `--name` (default "walkforward") but no
`--trace`/persist toggle:
```rust
struct WfBlueprintArgs {
axes: Vec<(String, Vec<Scalar>)>,
name: String,
select: Selection,
}
/// Parse the `aura walkforward <blueprint.json> …` tail: repeatable `--axis
/// <name>=<csv>` (the per-window re-fit grid, >= 1 required), `--select
/// <argmax|plateau:mean|plateau:worst>` (default argmax), `--name <fam>` (default
/// "walkforward"). Reduce-mode + content-addressed, so no `--trace`.
fn parse_walkforward_blueprint_args(rest: &[&str]) -> Result<WfBlueprintArgs, String> {
let usage = || "walkforward <blueprint.json> --axis <name>=<csv> [--axis …] [--select <argmax|plateau:mean|plateau:worst>] [--name <n>]".to_string();
let mut axes: Vec<(String, Vec<Scalar>)> = Vec::new();
let mut name = "walkforward".to_string();
let mut select = Selection::Argmax;
let mut tail = rest;
while let Some((flag, t)) = tail.split_first() {
let (value, t) = t.split_first().ok_or_else(usage)?;
match *flag {
"--axis" => {
let (n, csv) = value.split_once('=').ok_or_else(usage)?;
if n.is_empty() || axes.iter().any(|(a, _)| a == n) { return Err(usage()); }
let vals = parse_scalar_csv(csv).ok_or_else(usage)?;
axes.push((n.to_string(), vals));
}
"--select" => select = parse_select(value).ok_or_else(usage)?,
"--name" => name = (*value).to_string(),
_ => return Err(usage()),
}
tail = t;
}
if axes.is_empty() {
return Err("walkforward <blueprint.json> requires >= 1 --axis to re-fit per window".to_string());
}
Ok(WfBlueprintArgs { axes, name, select })
}
```
- [ ] **Step 4: Add `run_blueprint_walkforward` (IO wrapper: persist + print)**
Insert near `run_blueprint_sweep` (~3340). Mirrors `run_blueprint_sweep`'s persist
seam (`put_blueprint` + `append_family`) with the WF family/print:
```rust
/// `aura walkforward <blueprint.json> --axis …`: build the loaded-blueprint
/// IS-refit walk-forward, store the canonical blueprint ONCE keyed by the shared
/// `topology_hash` (the C18 hook, so `aura reproduce` re-derives it), record it as
/// a `FamilyKind::WalkForward` family, and print each OOS member line + the summary.
/// Mirrors `run_blueprint_sweep` (content-addressed family verb; no ensure_name_free).
fn run_blueprint_walkforward(doc: &str, axes: &[(String, Vec<Scalar>)], name: &str, data: DataSource, select: Selection) {
let result = blueprint_walkforward_family(doc, axes, &data, select);
let reg = default_registry();
let topo = result.windows[0]
.run
.oos_report
.manifest
.topology_hash
.clone()
.expect("a blueprint walk-forward stamps every member's topology_hash");
let canonical = blueprint_to_json(
&blueprint_from_json(doc, &|t| std_vocabulary(t)).expect("doc parse-validated at the dispatch boundary"),
)
.expect("a loaded blueprint re-serializes");
reg.put_blueprint(&topo, &canonical).unwrap_or_else(|e| { eprintln!("aura: {e}"); std::process::exit(2); });
let id = match reg.append_family(name, FamilyKind::WalkForward, &walkforward_member_reports(&result)) {
Ok(id) => id,
Err(e) => { eprintln!("aura: {e}"); std::process::exit(2); }
};
for w in &result.windows {
println!("{}", family_member_line(&id, &w.run.oos_report));
}
println!("{}", walkforward_summary_json(&result));
}
```
- [ ] **Step 5: Add the `.json` discriminator to the `["walkforward", ..]` dispatch**
Modify the dispatch arm (~4175). Add the `.json`-first branch BEFORE the existing
`match parse_walkforward_args(rest)`, mirroring the `["sweep", ..]` arm (~4135-4164):
```rust
["walkforward", rest @ ..] => {
if let Some(path) =
rest.first().filter(|a| a.ends_with(".json") && std::path::Path::new(a).is_file())
{
let doc = std::fs::read_to_string(path).unwrap_or_else(|e| {
eprintln!("aura: {path}: {e}");
std::process::exit(2);
});
if let Err(e) = blueprint_from_json(&doc, &|t| std_vocabulary(t)) {
eprintln!("aura: {path}: {e:?}");
std::process::exit(2);
}
let WfBlueprintArgs { axes, name, select } =
parse_walkforward_blueprint_args(&rest[1..]).unwrap_or_else(|msg| {
eprintln!("aura: {msg}");
std::process::exit(2);
});
run_blueprint_walkforward(&doc, &axes, &name, DataSource::Synthetic, select);
return;
}
match parse_walkforward_args(rest) {
Ok((strategy, name, persist, choice, grid, select)) => {
run_walkforward(strategy, &name, persist, DataSource::from_choice(choice), &grid, select)
}
Err(msg) => {
eprintln!("aura: {msg}");
std::process::exit(2);
}
}
}
```
(Confirm the existing hard-wired `parse_walkforward_args` arm is preserved verbatim
inside the `else`; match its exact current tuple/body when transcribing.)
- [ ] **Step 6: Verify the unit test passes + the workspace builds**
Run: `cargo test -p aura-cli --bin aura parse_walkforward_blueprint_args`
Expected: PASS.
Run: `cargo build --workspace`
Expected: `Finished` — 0 errors (the verb now wires blueprint_walkforward_family →
no more unused-fn warnings for it or the helpers).
---
## Task 3: The reproduce WalkForward branch
**Files:**
- Modify: `crates/aura-cli/src/main.rs` (`reproduce_family_in` match ~2566)
- Test: `crates/aura-cli/tests/cli_run.rs`
- [ ] **Step 1: Write the failing E2E**
Add to `crates/aura-cli/tests/cli_run.rs` (beside the mc-reproduce template
`aura_mc_over_a_blueprint_reproduces_bit_identically` ~3495):
```rust
/// E2E (#173): an IS-refit walk-forward over a loaded blueprint persists a
/// content-addressed WalkForward family that `aura reproduce`s bit-identically —
/// each OOS member's windowed slice + winner params are recovered from its manifest.
#[test]
fn aura_walkforward_over_a_blueprint_reproduces_bit_identically() {
let cwd = temp_cwd("blueprint-walkforward-reproduce");
let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR"));
let run = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3",
"--axis", "stage1_signal.slow.length=4,6", "--name", "wfr"])
.current_dir(&cwd)
.output()
.expect("spawn aura walkforward");
assert_eq!(run.status.code(), Some(0), "stderr: {}", String::from_utf8_lossy(&run.stderr));
let repro = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["reproduce", "wfr-0"])
.current_dir(&cwd)
.output()
.expect("spawn aura reproduce");
assert_eq!(repro.status.code(), Some(0));
let out = String::from_utf8(repro.stdout).expect("utf-8");
assert!(out.contains("reproduced 3/3 members bit-identically"), "stdout: {out}");
let _ = std::fs::remove_dir_all(&cwd);
}
```
- [ ] **Step 2: Run to verify it fails (RED — reproduce reconstructs the wrong source)**
Run: `cargo test -p aura-cli --test cli_run aura_walkforward_over_a_blueprint_reproduces`
Expected: FAIL — reproduce falls through the `_` arm (full-window `run_sources`),
so the re-run does not match the OOS-windowed original (not bit-identical), or a
non-zero exit.
- [ ] **Step 3: Add the `WalkForward` arm to `reproduce_family_in`**
In the `match family.kind` block (~2566-2573), add a `FamilyKind::WalkForward` arm
before the `_ =>` fallthrough. It supplies only the (sources, window) pair — the
winner params are recovered by the SAME shared `point_from_params(&space,
&stored.manifest.params)` line the Sweep/MC arms already use:
```rust
FamilyKind::WalkForward => {
// each member is one OOS window: rebuild its windowed slice from the
// stored window bounds; the winner params come from the shared
// manifest->cells recovery below (as a Sweep member's do).
let (from, to) = stored.manifest.window;
let s = data.windowed_sources(from, to);
let w = window_of(&s).expect("non-empty OOS window");
(s, w)
}
```
(Insert it as a new match arm; keep the existing `FamilyKind::MonteCarlo` arm and
the `_ =>` fallthrough. Match the exact binding names the surrounding code uses —
transcribe the current arm shape at ~2566 before editing.)
- [ ] **Step 4: Verify the E2E passes**
Run: `cargo test -p aura-cli --test cli_run aura_walkforward_over_a_blueprint_reproduces`
Expected: PASS — "reproduced 3/3 members bit-identically", exit 0.
---
## Task 4: E2E coverage (persist shape, no-axis, malformed) + the full gate
**Files:**
- Test: `crates/aura-cli/tests/cli_run.rs`
- [ ] **Step 1: Write the three E2Es**
Add to `crates/aura-cli/tests/cli_run.rs` (beside the sweep-persist template
`aura_sweep_loads_a_blueprint_and_persists_a_sweep_family` ~3269):
```rust
/// E2E (#173): the loaded-blueprint walk-forward runs + persists a WalkForward
/// family (3 OOS members) with a {"walkforward":…} summary carrying oos_r, and
/// stores exactly one content-addressed blueprint.
#[test]
fn aura_walkforward_over_a_blueprint_persists_a_family() {
let cwd = temp_cwd("blueprint-walkforward-persist");
let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR"));
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3",
"--axis", "stage1_signal.slow.length=4,6", "--name", "wfx"])
.current_dir(&cwd)
.output()
.expect("spawn aura walkforward");
assert_eq!(out.status.code(), Some(0), "stderr: {}", String::from_utf8_lossy(&out.stderr));
let stdout = String::from_utf8(out.stdout).expect("utf-8");
assert_eq!(stdout.matches("wfx-").count(), 3, "3 OOS member lines: {stdout}");
assert!(stdout.contains("\"walkforward\""), "summary line: {stdout}");
assert!(stdout.contains("\"oos_r\""), "R-measured summary: {stdout}");
let families = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["runs", "families"]).current_dir(&cwd).output().expect("spawn families");
let fo = String::from_utf8(families.stdout).expect("utf-8");
assert!(fo.contains("\"kind\":\"WalkForward\""), "families: {fo}");
let n = std::fs::read_dir(cwd.join("runs").join("blueprints")).map(|d| d.count()).unwrap_or(0);
assert_eq!(n, 1, "exactly one content-addressed blueprint stored");
let _ = std::fs::remove_dir_all(&cwd);
}
/// E2E (#173): an OPEN blueprint with no --axis (nothing to re-fit) is a usage
/// error (exit 2), naming that >= 1 --axis is required.
#[test]
fn aura_walkforward_over_a_blueprint_rejects_no_axis() {
let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR"));
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["walkforward", &bp])
.output()
.expect("spawn aura walkforward (no axis)");
assert_eq!(out.status.code(), Some(2));
let stderr = String::from_utf8(out.stderr).expect("utf-8");
assert!(stderr.contains("requires >= 1 --axis"), "stderr: {stderr}");
}
/// E2E (#173, safety): a malformed blueprint is rejected at the dispatch boundary
/// (exit 2, UnknownNodeType) rather than panicking.
#[test]
fn aura_walkforward_over_a_blueprint_rejects_malformed() {
let bp = format!("{}/tests/fixtures/unknown_node.json", env!("CARGO_MANIFEST_DIR"));
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3"])
.output()
.expect("spawn aura walkforward (malformed)");
assert_eq!(out.status.code(), Some(2), "malformed must be rejected, not panic");
let stderr = String::from_utf8(out.stderr).expect("utf-8");
assert!(stderr.contains("UnknownNodeType"), "stderr: {stderr}");
assert!(!stderr.contains("panicked"), "must reject cleanly: {stderr}");
}
```
- [ ] **Step 2: Run the three new E2Es**
Run: `cargo test -p aura-cli --test cli_run aura_walkforward_over_a_blueprint`
Expected: PASS — 4 tests (the three above + the reproduce test from Task 3, which
shares the `aura_walkforward_over_a_blueprint` prefix).
- [ ] **Step 3: Full-suite + lint gate**
Run: `cargo test --workspace`
Expected: PASS — no regressions (esp. the hard-wired walkforward tests
`walkforward_plateau_select_stamps_plateau_provenance`,
`walkforward_select_argmax_is_byte_identical_to_default`, and the sweep/mc/reproduce
E2Es — the dispatch + reproduce edits preserved them).
Run: `cargo clippy --workspace --all-targets -- -D warnings`
Expected: `Finished` — 0 warnings (every new fn is now wired; no dead code).
---
## Self-review (planner Step 5)
1. **Spec coverage:** §Concrete shapes (blueprint_sweep_over / run_oos_blueprint /
blueprint_walkforward_family / WfBlueprintArgs+parse / dispatch / reproduce) →
Tasks 1-3; §Testing (unit family, unit parse, 4 E2Es) → Tasks 1/2/3/4;
§Error handling (no-axis, malformed, bad-axis in-closure exit) → Task 2 parse +
Task 4 E2Es + Task 1 blueprint_sweep_over Result; §Persist + reproduce → Tasks
2+3 (land together, one commit); §Acceptance 1-7 → the E2Es + the Task-4 gate. ✓
2. **Placeholder scan:** no TBD/TODO/"similar to"/"add appropriate". The
transcribe-the-current-arm notes (dispatch else-branch, reproduce arm) name the
exact site + say match it verbatim — not deferred decisions. ✓
3. **Type/name consistency:** blueprint_sweep_over / run_oos_blueprint /
blueprint_walkforward_family / WfBlueprintArgs / parse_walkforward_blueprint_args /
run_blueprint_walkforward / FamilyKind::WalkForward / walkforward_member_reports /
Selection / select_winner / walk_forward / WindowRun / WindowBounds consistent
across tasks and match the recon signatures. `blueprint_walkforward_family`
returns `WalkForwardResult` (not Result) in Task 1, Task 2's wrapper, and the
unit test — consistent. ✓
4. **Step granularity:** each step is one fn/edit or one command. ✓
5. **No commit steps.** ✓
6. **Pin/replacement contiguity:** E2E pins — `"reproduced 3/3 members
bit-identically"` (the reproduce path's literal, reused from the mc-reproduce
template), `"kind":"WalkForward"` (append_family serialization), `"walkforward"`
/ `"oos_r"` (walkforward_summary_json keys), `"requires >= 1 --axis"` (the
parse Err literal in Task 2 Step 3), `UnknownNodeType` (engine error) — each
appears contiguously in the corresponding impl/literal. ✓
7. **Compile-gate vs. deferred-caller:** the new `WfBlueprintArgs` struct + its
destructure in the dispatch are BOTH in Task 2 (Steps 3+5) before Task 2's
build gate (Step 6). No caller deferred. Task 1's fns are unused until Task 2 —
handled by using `cargo build` (warning-tolerant) not clippy in Tasks 1-3, with
the `-D warnings` clippy gate in Task 4 after wiring. ✓
8. **Verification filters resolve:** `blueprint_walkforward_family` (Task 1 new),
`parse_walkforward_blueprint_args` (Task 2 new),
`aura_walkforward_over_a_blueprint_reproduces` (Task 3 new),
`aura_walkforward_over_a_blueprint` (Task 4 — matches the 4 WF E2Es by shared
prefix), `walkforward_plateau_select_stamps_plateau_provenance` (existing, Task
4 regression) — every filter matches ≥1 named test in the post-task tree. ✓
docs/specs/0097-is-refit-walk-forward-over-a-loaded-blueprint.md
-314
View File
@@ -1,314 +0,0 @@
# In-sample-refit walk-forward over a loaded blueprint — Design Spec
**Date:** 2026-07-01
**Status:** Draft — awaiting user spec review
**Authors:** orchestrator + Claude
## Goal
Ship the World/C21 milestone's last piece: a **true in-sample-refit walk-forward
over a loaded blueprint**. `aura walkforward <blueprint.json> --axis <name>=<csv>
[--axis …]` rolls IS/OOS windows over a synthetic span; on each IS window it
**re-optimizes** the blueprint's params over the user's `--axis` grid, runs the
winner **out-of-sample**, and aggregates the per-window OOS reports into a
`FamilyKind::WalkForward` family that is content-addressed and `aura reproduce`s
bit-identically. This is Arm A (the user's decision, #173) — the honest OOS
validation the milestone exists to enable, and the form that legitimately carries
the "walk-forward" name (matching the hard-wired verb's IS-optimizing guarantee).
## Architecture
The hard-wired `aura walkforward --strategy stage1-r` ALREADY does IS-refit
(`walkforward_family`'s `Stage1R` arm, `crates/aura-cli/src/main.rs`): for each
window it `stage1_r_sweep_over(is) → select_winner("sqn_normalized") →
run_oos_r(winner, oos)`. The loaded-blueprint form is the **same shape with one
substitution** — the per-window IS-sweep and OOS-run source the **loaded
blueprint** (wrapped by `wrap_stage1r` via cycle-0096's `blueprint_axis_probe`)
instead of the built-in strategy. Everything else is reused verbatim: the generic
engine driver `walk_forward(roller, space, run_window)` (strategy-agnostic,
`crates/aura-engine/src/walkforward.rs`), `select_winner` (Argmax-deflated /
Plateau), the `24/12/12`-over-60-bar synthetic schedule (`wf_full_span` /
`wf_window_sizes`), and `FamilyKind::WalkForward` + `walkforward_member_reports` +
`walkforward_summary_json`.
Reused windowing seam: `data.windowed_sources(from, to)` (the same seam
`stage1_r_sweep_over` and `run_oos_r` use). No look-ahead: IS strictly precedes
OOS in every window (`WindowRoller`), and the loaded-member path (`run_blueprint_member`)
already only sees its window's stream (invariants 1/2).
Persistence + reproduce are shipped **together** (they are coupled — a persisted
family with no store hook reports members as "blueprint missing", the #170
hazard). The family is stored once by the shared `topology_hash`
(`put_blueprint`, the C18 hook the sweep/mc use) + `append_family`; `aura
reproduce` gains a `WalkForward` branch that reconstructs each OOS member's
windowed source (from its manifest window) and re-runs it with the member's
stored winner params — mirroring the `MonteCarlo` reproduce branch from cycle
0095. One signal topology → one shared `topology_hash` across all windows; each
member manifest carries `(topology_hash, window, chosen params, selection)`.
Scope: `crates/aura-cli` only (+ read-only reuse of the engine/registry WF
machinery). No engine/registry change — invariants 8/9 hold.
## Concrete code shapes
### Worked user program (the acceptance evidence)
A researcher has an OPEN strategy blueprint and wants an honest walk-forward that
re-fits its two SMA lengths each window (the axis names come from `--list-axes`,
cycle 0096):
```console
$ aura sweep crates/aura-cli/tests/fixtures/stage1_signal_open.json --list-axes
stage1_signal.fast.length:I64
stage1_signal.slow.length:I64
$ aura walkforward crates/aura-cli/tests/fixtures/stage1_signal_open.json \
--axis stage1_signal.fast.length=2,3 --axis stage1_signal.slow.length=4,6 --name wf1
walkforward-0 <oos window 0 metric line>
walkforward-1 <oos window 1 metric line>
walkforward-2 <oos window 2 metric line>
{"walkforward":{"windows":3,"stitched_total_pips":<f>,"param_stability":<…>,"oos_r":{…}}}
$ # content-addressed: the family reproduces bit-identically
$ aura reproduce walkforward-0
reproduced 3/3 members bit-identically
```
Per window the IS grid (2×2 here) is re-optimized by `sqn_normalized`; the winner
runs OOS; the three OOS reports are the family members (line shape identical to
the hard-wired `aura walkforward`, which reuses `family_member_line`). Exact
metric numbers are run-determined; the *shape* — three re-fit windows, OOS
members, a `{"walkforward":…}` summary, and a bit-identical reproduce — is the
criterion.
An OPEN blueprint with **no** `--axis` (nothing to re-fit) is a usage error:
```console
$ aura walkforward crates/aura-cli/tests/fixtures/stage1_signal_open.json
aura: walkforward <blueprint.json> requires >= 1 --axis to re-fit per window
$ echo $?
2
```
### Implementation shape (secondary)
**The family builder** — the loaded-blueprint analog of `walkforward_family`'s
`Stage1R` arm; reuses the generic `walk_forward` driver, substituting the loaded
blueprint into the per-window closure:
```rust
fn blueprint_walkforward_family(
doc: &str, axes: &[(String, Vec<Scalar>)], data: &DataSource, select: Selection,
) -> WalkForwardResult {
let span = data.wf_full_span();
let (is_len, oos_len, step) = data.wf_window_sizes();
let roller = WindowRoller::new(span, is_len, oos_len, step, RollMode::Rolling)
.unwrap_or_else(|e| { eprintln!("aura: walk-forward window too short: {e:?}"); std::process::exit(2); });
let space = blueprint_axis_probe(doc).param_space(); // the wrapped namespace (cycle 0096)
let pip = data.pip_size();
let topo = topology_hash(&blueprint_from_json(doc, &|t| std_vocabulary(t))
.expect("doc parse-validated at the dispatch boundary"));
walk_forward(roller, space.clone(), |w: WindowBounds| {
// IS: re-fit the loaded blueprint over the user grid, windowed to [is]
let (is_family, lattice) = blueprint_sweep_over(doc, axes, w.is.0, w.is.1, data);
let (best, selection) = select_winner(&is_family, "sqn_normalized", select, lattice.as_deref())
.unwrap_or_else(|msg| { eprintln!("aura: {msg}"); std::process::exit(2); });
// OOS: run the winner over [oos] via the loaded-member path
let (oos_equity, mut oos_report) =
run_oos_blueprint(doc, &best.params, &space, w.oos.0, w.oos.1, pip, &topo, data);
oos_report.manifest.selection = Some(selection);
WindowRun { chosen_params: best.params, oos_equity, oos_report }
})
}
```
**The windowed IS-sweep** — `blueprint_sweep_over`: the structural twin of
`stage1_r_sweep_over` (uses `sweep_with_lattice` so `--select plateau` works) and
of `blueprint_sweep_family` (loaded blueprint, `--axis` grid), but windowed to
`[from,to]` via `data.windowed_sources`:
```rust
fn blueprint_sweep_over(doc: &str, axes: &[(String, Vec<Scalar>)],
from: Timestamp, to: Timestamp, data: &DataSource)
-> (SweepFamily, Option<Vec<usize>>) {
let pip = data.pip_size();
let topo = topology_hash(&reload(doc));
let probe = blueprint_axis_probe(doc);
let space = probe.param_space();
// seed the user axes verbatim (same as blueprint_sweep_family), then sweep WITH lattice
let mut binder = /* probe.axis(first) then SweepBinder::axis(rest) */;
binder.sweep_with_lattice(|point| {
let sources = data.windowed_sources(from, to);
let window = window_of(&sources).expect("non-empty in-sample window");
run_blueprint_member(reload(doc), point, &space, sources, window, 0, pip, &topo)
}).map(|(fam, lat)| (fam, Some(lat)))
.expect("the user axes are name/kind-checked at the sweep terminal")
}
```
**The windowed OOS-run** — `run_oos_blueprint`: the loaded-member analog of
`run_oos_r`, running the winner params over the OOS window. The loaded-member
path (`run_blueprint_member`) is **reduce-mode**: it folds to R-metrics
(`summarize_r`) and retains the per-trade R series, but NOT the raw per-cycle pip
curve the hard-wired non-reduce `run_oos_r` returns. So `oos_equity` is **empty**
— the loaded walk-forward is measured in **R** (the C10-reworked yardstick), and
the meaningful per-window record is the OOS `RunReport`'s R-metrics:
```rust
fn run_oos_blueprint(doc: &str, params: &[Cell], space: &[ParamSpec],
from: Timestamp, to: Timestamp, pip: f64, topo: &str, data: &DataSource)
-> (Vec<(Timestamp, f64)>, RunReport) {
let sources = data.windowed_sources(from, to);
let window = window_of(&sources).expect("non-empty out-of-sample window");
let report = run_blueprint_member(reload(doc), params, space, sources, window, 0, pip, topo);
// reduce-mode: no raw pip curve retained -> empty stitching segment. The OOS
// record is R-metrics (report.metrics.r); an empty segment leaves the stitched
// curve unbroken (stitch treats it as a 0-offset, no-point contribution).
(Vec::new(), report)
}
```
Consequence for the summary: `walkforward_summary_json`'s `stitched_total_pips`
is `0.0` (no retained pip curve — the loaded path is R-measured, not pip-measured),
and its `oos_r` block (pooled per-trade Rs across the OOS windows) is the
meaningful line. This matches the R-first world of the C10 rework; a raw pip-equity
curve would need a non-reduce OOS run (`--trace`-style), out of scope here.
**The IO wrapper + dispatch** — `run_blueprint_walkforward` mirrors
`run_walkforward` (persist via `put_blueprint` + `append_family`, print members +
summary); the `["walkforward", rest @ ..]` dispatch gains the `.json`
discriminator BEFORE `parse_walkforward_args`, mirroring the sweep/mc arms:
```rust
["walkforward", rest @ ..] => {
if let Some(path) = rest.first().filter(|a| a.ends_with(".json") && Path::new(a).is_file()) {
let doc = std::fs::read_to_string(path).unwrap_or_else(|e| { eprintln!("aura: {path}: {e}"); exit(2) });
if let Err(e) = blueprint_from_json(&doc, &|t| std_vocabulary(t)) {
eprintln!("aura: {path}: {e:?}"); exit(2);
}
let WfBlueprintArgs { axes, name, persist, select } =
parse_walkforward_blueprint_args(&rest[1..]).unwrap_or_else(|m| { eprintln!("aura: {m}"); exit(2) });
run_blueprint_walkforward(&doc, &axes, &name, persist, DataSource::Synthetic, select);
return;
}
match parse_walkforward_args(rest) { /* unchanged hard-wired path */ }
}
```
**The reproduce branch** — `reproduce_family_in` already re-runs each member via
`run_blueprint_member` with a per-member `(sources, window, params)` triple chosen
by `family.kind` (cycle 0094 recovers the member's params from its manifest
against the reloaded space; cycle 0095 added the `MonteCarlo` arm that rebuilds
the seed-driven walk). The new `WalkForward` arm keeps the **same per-member param
recovery** (the winner params are stored in each OOS member's manifest, exactly as
a sweep member's are) and differs only in the **source**: the OOS window's slice,
rebuilt from the member manifest's stored window bounds:
```rust
match family.kind {
FamilyKind::MonteCarlo => { /* cycle 0095: manifest.seed -> synthetic walk */ }
FamilyKind::WalkForward => {
// the OOS slice from the member manifest's window; params via the SAME
// manifest->cells recovery the Sweep arm already uses (winner params).
let (from, to) = stored.manifest.window;
let sources = data.windowed_sources(from, to);
(sources, window_of(&sources)/* the member window */)
}
_ => (data.run_sources(), data.full_window()), // Sweep: the showcase series
}
```
The member's params are recovered by the existing shared path (not re-invented
here); only the `(sources, window)` pair is realization-specific — `WalkForward`
supplies the windowed OOS slice, exactly as `MonteCarlo` supplies the seeded walk.
## Components
- `blueprint_walkforward_family(doc, axes, data, select) -> WalkForwardResult`
(new) — the per-window re-fit driver over the loaded blueprint.
- `blueprint_sweep_over(doc, axes, from, to, data)` (new) — windowed IS-sweep with
lattice (twin of `stage1_r_sweep_over` + `blueprint_sweep_family`).
- `run_oos_blueprint(doc, params, space, from, to, pip, topo, data)` (new) —
windowed OOS run over the loaded-member path.
- `run_blueprint_walkforward(doc, axes, name, persist, data, select)` (new) — IO
wrapper: `put_blueprint` + `append_family(FamilyKind::WalkForward)` + print
members + `walkforward_summary_json`.
- `WfBlueprintArgs` + `parse_walkforward_blueprint_args` (new) — `--axis`
(≥1 required) + `--select` + `--name|--trace`; rejects a no-axis invocation.
- `["walkforward", ..]` dispatch (modified) — `.json` discriminator.
- `reproduce_family_in` (modified) — `WalkForward` realization branch.
## Data flow
`aura walkforward <bp.json> --axis …` → boundary read + `blueprint_from_json`
validate → `parse_walkforward_blueprint_args` → `blueprint_walkforward_family`:
`WindowRoller` rolls 3 windows → per window `blueprint_sweep_over(is)` re-fits the
grid (each member a windowed loaded-blueprint run) → `select_winner` picks by
`sqn_normalized` → `run_oos_blueprint(oos)` runs the winner → `WindowRun` →
`WalkForwardResult` → `put_blueprint(topo)` + `append_family(WalkForward)` → print
per-window OOS members + summary. `aura reproduce <id>` → reload blueprint by
`topology_hash` → per member reconstruct `windowed_sources(manifest.window)` +
`manifest.params` → `run_blueprint_member` → assert bit-identical.
## Error handling
- **OPEN blueprint with no `--axis`** → usage error (exit 2): a walk-forward with
nothing to re-fit is meaningless; mirror the loaded-blueprint sweep's "≥1 axis"
contract. Message names that ≥1 `--axis` is required.
- **Unknown / kind-mismatched `--axis` name** → the sweep terminal's
`UnknownKnob`/kind error, rendered `aura: …` + exit 2 (reused from the
loaded-blueprint sweep; the axis names come from `--list-axes`).
- **Malformed / unreadable blueprint** → rejected at the dispatch boundary
(`blueprint_from_json`, exit 2, `UnknownNodeType`), before any run — pinned by
an E2E (as cycle 0096 did), so the internal `.expect` reloads are unreachable.
- **`--select plateau` without a lattice** → `select_winner` already returns the
named refusal; the loaded IS-sweep uses `sweep_with_lattice`, so a grid sweep
always has a lattice (the refusal is unreachable here but preserved).
- **Window too short for one IS+OOS span** → `WindowRoller::new` error, exit 2
(reused).
- No panics on any user-reachable input; `walk_forward` runs windows disjointly
in parallel, deterministic in roll order (C1).
## Testing strategy
- **Unit — `blueprint_walkforward_family`** (`cargo test -p aura-cli --bin aura
blueprint_walkforward_family`): over `stage1_signal_open.json` with a 2×2
`--axis` grid → a `WalkForwardResult` with 3 windows, each `chosen_params` of
arity 2 (both axes re-fit), and each window's OOS `RunReport` carrying R-metrics
(`metrics.r`, incl. `sqn_normalized`). The stitched OOS pip-equity curve is
**empty** (reduce-mode retains no raw pip curve — the empty segments leave the
curve unbroken), so the per-window R-metrics, not a pip curve, are the OOS
record. Pins the per-window re-fit shape.
- **Unit — `parse_walkforward_blueprint_args`** (`cargo test -p aura-cli --bin
aura parse_walkforward_blueprint_args`): `--axis a=1,2 --select plateau:mean`
parses; a no-`--axis` tail → `Err`; `--select bogus` → `Err`.
- **E2E — the walk-forward runs + persists** (`cargo test -p aura-cli --test
cli_run <filter>`): `aura walkforward <open fixture> --axis
stage1_signal.fast.length=2,3 --axis stage1_signal.slow.length=4,6 --name wfx`
→ exit 0, 3 `walkforward-*` OOS member lines, a `{"walkforward":…}` summary,
and exactly one `runs/blueprints/<hash>.json`.
- **E2E — reproduce bit-identical** (`--test cli_run`): after the above, `aura
reproduce wfx-0` → "reproduced 3/3 members bit-identically", exit 0.
- **E2E — no-axis rejection** (`--test cli_run`): `aura walkforward <open fixture>`
(no `--axis`) → exit 2 + "requires >= 1 --axis" on stderr.
- **E2E — malformed-blueprint safety** (`--test cli_run`): `aura walkforward
crates/aura-cli/tests/fixtures/unknown_node.json --axis a=1,2` → exit 2 +
`UnknownNodeType` on stderr + no `panicked`.
## Acceptance criteria
1. `aura walkforward <blueprint.json> --axis <name>=<csv> [--axis …]` re-optimizes
the blueprint's params on each IS window and runs the winner OOS, aggregating
to a `FamilyKind::WalkForward` family (per-window OOS member lines + a
`{"walkforward":…}` summary), exit 0.
2. The family is content-addressed (one `runs/blueprints/<topology_hash>.json`)
and `aura reproduce`s every OOS member bit-identically.
3. An OPEN blueprint with no `--axis` is a usage error (exit 2), as is an unknown
axis name; a malformed blueprint is rejected at the boundary (exit 2, no panic).
4. `--select argmax` (default) and `--select plateau:mean|plateau:worst` both work
(the IS-sweep carries a lattice).
5. No look-ahead: IS strictly precedes OOS per window; runs are deterministic
(C1), parallelism across windows only.
6. No `aura-engine` / `aura-registry` change; invariants 1, 2, 8, 9 preserved.
7. `cargo build --workspace`, `cargo test --workspace`, `cargo clippy --workspace
--all-targets -- -D warnings` all green.