diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index ce2bbb0..46de599 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -1316,7 +1316,11 @@ synthetic (deterministic) data; recorded-dataset reproduction rides the DataServ (#124). **Cycle 0095 (#170):** `aura reproduce` now also spans `FamilyKind::MonteCarlo` — branching on the family kind, it reconstructs each member's seed-driven synthetic walk from the recorded `manifest.seed` (the `Sweep` arm unchanged), so an `aura mc` family re-derives -bit-identically through the same `run_blueprint_member` path. +bit-identically through the same `run_blueprint_member` path. **Cycle 0097 (#173):** reproduce +spans the third variant, `FamilyKind::WalkForward` — the same branch rebuilds each OOS member's +windowed slice from `manifest.window` (winner params via the shared `manifest→cells` recovery), +so an `aura walkforward` family re-derives bit-identically too. All three family kinds now +persist *and* reproduce through the one shared `topology_hash`+`put_blueprint` hook. ### C19 — Bootstrap: blueprint → instance (recursive) **Guarantee.** Construction is a distinct phase, recursive at every level. Each @@ -1897,9 +1901,18 @@ draws disjoint-parallel via the engine `monte_carlo` seam (invariant 1) — and `aura reproduce`s bit-identically (C1). MC binds no axis, so it needs a **closed** blueprint (the sweep's open/closed distinction inverted); an open one returns a named `Err` (rendered exit-2 at the `run_blueprint_mc` boundary, the sweep sibling's contract — no hidden exit in the -pure builder). **Walk-forward is deferred (#173, dep #169):** a loaded blueprint declares no -axes to re-fit per window, so it could only be a non-IS-refit windowed eval — which would -collide with the hard-wired `walkforward` verb's IS-optimizing guarantee. The synthetic-walk DGP +pure builder). **IS-refit walk-forward shipped (cycle 0097, #173).** `aura walkforward + --axis = [--select argmax|plateau:mean|plateau:worst]` re-optimizes +the loaded blueprint's params over the user `--axis` grid (the #169 prefixed names) on each +24/12/12 IS window, selects the winner by `sqn_normalized` (the hard-wired arm's metric + +`select_winner` reused verbatim), runs it out-of-sample, and aggregates to a +`FamilyKind::WalkForward` family — persisted + content-addressed + `aura reproduce` bit-identical +(the read-side WalkForward branch rebuilds each OOS window from `manifest.window`). One +substitution deep from the hard-wired stage1-r WF arm (the loaded blueprint via +`blueprint_axis_probe` replaces the built-in strategy); a bad `--axis` is a clean in-closure +exit 2, never a panic. This is Arm A (the settled direction): the loaded IS-optimizing form +honestly carries the `walkforward` name. Reduce-mode members are R-measured (`oos_r` the +meaningful summary; stitched pip-equity empty, C10). The synthetic-walk DGP is the machinery, not trader-grade MC statistics; a real-data block-bootstrap — and the `synthetic_walk_sources` `len:60`↔warm-up coupling it retires (a deep-lookback closed blueprint warms poorly → silent-vacuous draws today) — ride #172. **Axis-name discovery shipped (cycle diff --git a/docs/plans/0097-is-refit-walk-forward-over-a-loaded-blueprint.md b/docs/plans/0097-is-refit-walk-forward-over-a-loaded-blueprint.md deleted file mode 100644 index 78805ec..0000000 --- a/docs/plans/0097-is-refit-walk-forward-over-a-loaded-blueprint.md +++ /dev/null @@ -1,572 +0,0 @@ -# IS-refit walk-forward over a loaded blueprint — Implementation Plan - -> **Parent spec:** `docs/specs/0097-is-refit-walk-forward-over-a-loaded-blueprint.md` -> -> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to -> run this plan. Steps use `- [ ]` checkboxes for tracking. - -**Goal:** Add `aura walkforward --axis = [--axis …]` — -an in-sample-refit walk-forward over a loaded blueprint (re-optimize per IS -window, run the winner OOS), aggregated to a persisted, content-addressed, -reproduce-aware `FamilyKind::WalkForward` family. - -**Architecture:** Mirror `walkforward_family`'s `Stage1R` arm one substitution -deep — reuse the generic engine `walk_forward` driver + `select_winner` + -the 24/12/12 synthetic schedule; the per-window IS-sweep and OOS-run source the -LOADED blueprint (via cycle-0096's `blueprint_axis_probe`) over the user `--axis` -grid, windowed by `data.windowed_sources`. Persist via `put_blueprint` + -`append_family` (like the loaded sweep/mc), and add the `reproduce_family_in` -`WalkForward` branch so the family reproduces bit-identically. All changes in -`crates/aura-cli`; engine/registry are read-only callees (invariants 1/2/8/9). - -**Tech Stack:** `crates/aura-cli/src/main.rs` (six new fns + two modified sites), -`crates/aura-cli/tests/cli_run.rs` (E2Es). Read-only reuse: `walk_forward` / -`WindowRun` / `WalkForwardResult` (aura-engine), `FamilyKind::WalkForward` / -`walkforward_member_reports` / `put_blueprint` / `append_family` (aura-registry). - ---- - -**Files this plan creates or modifies:** - -- Modify: `crates/aura-cli/src/main.rs` — new `blueprint_sweep_over`, - `run_oos_blueprint`, `blueprint_walkforward_family`, `WfBlueprintArgs`, - `parse_walkforward_blueprint_args`, `run_blueprint_walkforward`; modified - `["walkforward", ..]` dispatch (~4175) + `reproduce_family_in` match (~2566). -- Test: `crates/aura-cli/src/main.rs` (`#[cfg(test)] mod tests` ~4238) — unit - `blueprint_walkforward_family` + unit `parse_walkforward_blueprint_args`. -- Test: `crates/aura-cli/tests/cli_run.rs` — 4 new E2Es beside the blueprint-family - block (~3269-3700). - -Line numbers are current-tree recon anchors; match on surrounding code, not the -number. **Gate note:** Tasks 1-3 use `cargo build` + scoped `cargo test` (early -tasks add not-yet-wired fns, which are a build *warning*, not an error); the full -`cargo clippy -- -D warnings` + `cargo test --workspace` gate runs in Task 4, -after every fn is wired. - ---- - -## Task 1: The family builder + its two windowed helpers - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` (new fns near `blueprint_sweep_family` ~3197 and `stage1_r_sweep_over` ~1329) -- Test: `crates/aura-cli/src/main.rs` (`mod tests`) - -- [ ] **Step 1: Write the failing unit test** - -Add to the `#[cfg(test)] mod tests` block (near `blueprint_axis_probe_lists_prefixed_open_knobs`): - -```rust -#[test] -fn blueprint_walkforward_family_refits_each_window() { - // The open fixture's two SMA lengths are re-fit per IS window over a 2x2 grid. - let doc = include_str!("../tests/fixtures/stage1_signal_open.json"); - let axes = vec![ - ("stage1_signal.fast.length".to_string(), vec![Scalar::i64(2), Scalar::i64(3)]), - ("stage1_signal.slow.length".to_string(), vec![Scalar::i64(4), Scalar::i64(6)]), - ]; - let result = blueprint_walkforward_family(doc, &axes, &DataSource::Synthetic, Selection::Argmax); - // 24/12/12 over the 60-bar synthetic span -> 3 rolling windows. - assert_eq!(result.windows.len(), 3, "three rolling IS/OOS windows"); - for w in &result.windows { - assert_eq!(w.run.chosen_params.len(), 2, "both axes re-fit each window"); - assert!(w.run.oos_report.metrics.r.is_some(), "OOS record is R-metrics"); - } - // reduce-mode retains no raw pip curve -> the stitched pip-equity is empty. - assert!(result.stitched_oos_equity.is_empty(), "no raw pip curve in reduce-mode"); -} -``` - -- [ ] **Step 2: Run the test to verify it fails (RED — unresolved name)** - -Run: `cargo test -p aura-cli --bin aura blueprint_walkforward_family` -Expected: FAIL to compile — `cannot find function 'blueprint_walkforward_family'`. - -- [ ] **Step 3: Add `blueprint_sweep_over` (windowed IS-sweep, with lattice)** - -Insert near `blueprint_sweep_family` (~3232). Twin of `stage1_r_sweep_over` -(uses `sweep_with_lattice`) crossed with `blueprint_sweep_family`'s loaded-member -seed-axes pattern, windowed by `[from,to]`. Returns `Result<_, BindError>` (user -axes are fallible — an unknown/kind-mismatched `--axis` returns `BindError`, not a -panic; the sweep terminal name/kind-checks): - -```rust -/// Sweep the LOADED blueprint over the user `--axis` grid on an in-sample window -/// `[from,to]` — the windowed, lattice-carrying twin of `stage1_r_sweep_over` and -/// `blueprint_sweep_family`. `sweep_with_lattice` gives the grid lattice `--select -/// plateau` needs. An unknown/kind-mismatched axis surfaces as `BindError` at the -/// sweep terminal (no panic, no hidden exit) for the caller to render. -fn blueprint_sweep_over( - doc: &str, axes: &[(String, Vec)], from: Timestamp, to: Timestamp, data: &DataSource, -) -> Result<(SweepFamily, Vec), BindError> { - let reload = |d: &str| { - blueprint_from_json(d, &|t| std_vocabulary(t)) - .expect("doc parse-validated at the dispatch boundary; reload is infallible") - }; - let pip = data.pip_size(); - let topo = topology_hash(&reload(doc)); - let probe = blueprint_axis_probe(doc); - let space = probe.param_space(); - let mut iter = axes.iter(); - let (first_name, first_vals) = iter.next().expect("a blueprint walk-forward declares >= 1 axis"); - let mut binder = probe.axis(first_name, first_vals.clone()); - for (n, vals) in iter { - binder = binder.axis(n, vals.clone()); - } - binder.sweep_with_lattice(|point| { - let sources = data.windowed_sources(from, to); - let window = window_of(&sources).expect("non-empty in-sample window"); - run_blueprint_member(reload(doc), point, &space, sources, window, 0, pip, &topo) - }) -} -``` - -- [ ] **Step 4: Add `run_oos_blueprint` (windowed OOS-run)** - -Insert next to `blueprint_sweep_over`. Loaded-member analog of `run_oos_r`; -reduce-mode retains no raw pip curve, so the pip segment is empty: - -```rust -/// Run the winner params over an out-of-sample window `[from,to]` on the loaded -/// blueprint — the loaded-member analog of `run_oos_r`. The reduce-mode member -/// (`run_blueprint_member`) retains R-metrics, not a raw pip curve, so the stitching -/// segment is empty (an empty segment leaves the stitched curve unbroken). -fn run_oos_blueprint( - doc: &str, params: &[Cell], space: &[ParamSpec], from: Timestamp, to: Timestamp, - pip: f64, topo: &str, data: &DataSource, -) -> (Vec<(Timestamp, f64)>, RunReport) { - let reload = blueprint_from_json(doc, &|t| std_vocabulary(t)) - .expect("doc parse-validated at the dispatch boundary; reload is infallible"); - let sources = data.windowed_sources(from, to); - let window = window_of(&sources).expect("non-empty out-of-sample window"); - let report = run_blueprint_member(reload, params, space, sources, window, 0, pip, topo); - (Vec::new(), report) -} -``` - -- [ ] **Step 5: Add `blueprint_walkforward_family` (the per-window re-fit driver)** - -Insert next to the two helpers. Structural twin of `walkforward_family`'s -`Stage1R` arm — reuses the generic `walk_forward` driver; the per-window closure -re-fits the loaded blueprint. In-closure errors `exit(2)` exactly as the Stage1R -arm does for `select_winner` (clean process exit, never a panic): - -```rust -/// The loaded-blueprint IS-refit walk-forward: per IS window, re-optimize the -/// blueprint over the user `--axis` grid, select by `sqn_normalized`, run the -/// winner OOS. Structural twin of `walkforward_family`'s `Stage1R` arm — the -/// generic `walk_forward` driver + `select_winner` reused; only the per-window -/// sweep/OOS source the loaded blueprint. In-closure errors (a bad `--axis`) -/// `exit(2)` with the sweep terminal's message, as the hard-wired arm does. -fn blueprint_walkforward_family( - doc: &str, axes: &[(String, Vec)], data: &DataSource, select: Selection, -) -> WalkForwardResult { - let span = data.wf_full_span(); - let (is_len, oos_len, step) = data.wf_window_sizes(); - let roller = match WindowRoller::new(span, is_len, oos_len, step, RollMode::Rolling) { - Ok(r) => r, - Err(e) => { - eprintln!("aura: walk-forward window too short for one IS+OOS span: {e:?}"); - std::process::exit(2); - } - }; - let space = blueprint_axis_probe(doc).param_space(); - let pip = data.pip_size(); - let topo = topology_hash(&blueprint_from_json(doc, &|t| std_vocabulary(t)) - .expect("doc parse-validated at the dispatch boundary; reload is infallible")); - walk_forward(roller, space.clone(), |w: WindowBounds| { - let (is_family, lattice) = blueprint_sweep_over(doc, axes, w.is.0, w.is.1, data) - .unwrap_or_else(|e| { eprintln!("aura: {e}"); std::process::exit(2); }); - let (best, selection) = match select_winner(&is_family, "sqn_normalized", select, Some(&lattice)) { - Ok(v) => v, - Err(msg) => { eprintln!("aura: {msg}"); std::process::exit(2); } - }; - let (oos_equity, mut oos_report) = - run_oos_blueprint(doc, &best.params, &space, w.oos.0, w.oos.1, pip, &topo, data); - oos_report.manifest.selection = Some(selection); - WindowRun { chosen_params: best.params, oos_equity, oos_report } - }) -} -``` - -- [ ] **Step 6: Verify the unit test passes + the workspace builds** - -Run: `cargo test -p aura-cli --bin aura blueprint_walkforward_family` -Expected: PASS (1 test — 3 windows, both axes re-fit, R-metrics present, empty curve). - -Run: `cargo build --workspace` -Expected: `Finished` — 0 errors (unused-fn *warnings* for the not-yet-wired fns -are expected until Task 2 wires them; warnings are not errors). - ---- - -## Task 2: The parse grammar, IO wrapper, and dispatch (the persisting verb) - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` (new `WfBlueprintArgs` + `parse_walkforward_blueprint_args` near `parse_sweep_blueprint_args` ~3939; new `run_blueprint_walkforward` near `run_blueprint_sweep` ~3301; dispatch ~4175) -- Test: `crates/aura-cli/src/main.rs` (`mod tests`) - -- [ ] **Step 1: Write the failing unit test** - -Add beside `parse_sweep_blueprint_args_grammar`: - -```rust -#[test] -fn parse_walkforward_blueprint_args_grammar() { - let a = parse_walkforward_blueprint_args(&[ - "--axis", "stage1_signal.fast.length=2,3", "--select", "plateau:mean", "--name", "wf", - ]).expect("valid tail parses"); - assert_eq!(a.axes.len(), 1); - assert_eq!(a.name, "wf"); - assert!(matches!(a.select, Selection::Plateau(_))); - // a walk-forward with nothing to re-fit is rejected: - assert!(parse_walkforward_blueprint_args(&[]).is_err()); - assert!(parse_walkforward_blueprint_args(&["--name", "wf"]).is_err()); - // an unknown --select value is rejected: - assert!(parse_walkforward_blueprint_args(&["--axis", "a=1,2", "--select", "bogus"]).is_err()); - // default name + default argmax: - let b = parse_walkforward_blueprint_args(&["--axis", "a=1,2"]).expect("minimal tail parses"); - assert_eq!(b.name, "walkforward"); - assert!(matches!(b.select, Selection::Argmax)); -} -``` - -- [ ] **Step 2: Run the test to verify it fails (RED)** - -Run: `cargo test -p aura-cli --bin aura parse_walkforward_blueprint_args` -Expected: FAIL to compile — `cannot find function 'parse_walkforward_blueprint_args'`. - -- [ ] **Step 3: Add `WfBlueprintArgs` + `parse_walkforward_blueprint_args`** - -Insert near `parse_sweep_blueprint_args` (~3987). Mirrors the sweep `--axis` -grammar + the `--select` parse from `parse_walkforward_args` (`parse_select` -~1614); requires ≥1 `--axis`. This is a content-addressed family verb (always -persists, like `aura mc`), so it takes `--name` (default "walkforward") but no -`--trace`/persist toggle: - -```rust -struct WfBlueprintArgs { - axes: Vec<(String, Vec)>, - name: String, - select: Selection, -} - -/// Parse the `aura walkforward …` tail: repeatable `--axis -/// =` (the per-window re-fit grid, >= 1 required), `--select -/// ` (default argmax), `--name ` (default -/// "walkforward"). Reduce-mode + content-addressed, so no `--trace`. -fn parse_walkforward_blueprint_args(rest: &[&str]) -> Result { - let usage = || "walkforward --axis = [--axis …] [--select ] [--name ]".to_string(); - let mut axes: Vec<(String, Vec)> = Vec::new(); - let mut name = "walkforward".to_string(); - let mut select = Selection::Argmax; - let mut tail = rest; - while let Some((flag, t)) = tail.split_first() { - let (value, t) = t.split_first().ok_or_else(usage)?; - match *flag { - "--axis" => { - let (n, csv) = value.split_once('=').ok_or_else(usage)?; - if n.is_empty() || axes.iter().any(|(a, _)| a == n) { return Err(usage()); } - let vals = parse_scalar_csv(csv).ok_or_else(usage)?; - axes.push((n.to_string(), vals)); - } - "--select" => select = parse_select(value).ok_or_else(usage)?, - "--name" => name = (*value).to_string(), - _ => return Err(usage()), - } - tail = t; - } - if axes.is_empty() { - return Err("walkforward requires >= 1 --axis to re-fit per window".to_string()); - } - Ok(WfBlueprintArgs { axes, name, select }) -} -``` - -- [ ] **Step 4: Add `run_blueprint_walkforward` (IO wrapper: persist + print)** - -Insert near `run_blueprint_sweep` (~3340). Mirrors `run_blueprint_sweep`'s persist -seam (`put_blueprint` + `append_family`) with the WF family/print: - -```rust -/// `aura walkforward --axis …`: build the loaded-blueprint -/// IS-refit walk-forward, store the canonical blueprint ONCE keyed by the shared -/// `topology_hash` (the C18 hook, so `aura reproduce` re-derives it), record it as -/// a `FamilyKind::WalkForward` family, and print each OOS member line + the summary. -/// Mirrors `run_blueprint_sweep` (content-addressed family verb; no ensure_name_free). -fn run_blueprint_walkforward(doc: &str, axes: &[(String, Vec)], name: &str, data: DataSource, select: Selection) { - let result = blueprint_walkforward_family(doc, axes, &data, select); - let reg = default_registry(); - let topo = result.windows[0] - .run - .oos_report - .manifest - .topology_hash - .clone() - .expect("a blueprint walk-forward stamps every member's topology_hash"); - let canonical = blueprint_to_json( - &blueprint_from_json(doc, &|t| std_vocabulary(t)).expect("doc parse-validated at the dispatch boundary"), - ) - .expect("a loaded blueprint re-serializes"); - reg.put_blueprint(&topo, &canonical).unwrap_or_else(|e| { eprintln!("aura: {e}"); std::process::exit(2); }); - let id = match reg.append_family(name, FamilyKind::WalkForward, &walkforward_member_reports(&result)) { - Ok(id) => id, - Err(e) => { eprintln!("aura: {e}"); std::process::exit(2); } - }; - for w in &result.windows { - println!("{}", family_member_line(&id, &w.run.oos_report)); - } - println!("{}", walkforward_summary_json(&result)); -} -``` - -- [ ] **Step 5: Add the `.json` discriminator to the `["walkforward", ..]` dispatch** - -Modify the dispatch arm (~4175). Add the `.json`-first branch BEFORE the existing -`match parse_walkforward_args(rest)`, mirroring the `["sweep", ..]` arm (~4135-4164): - -```rust - ["walkforward", rest @ ..] => { - if let Some(path) = - rest.first().filter(|a| a.ends_with(".json") && std::path::Path::new(a).is_file()) - { - let doc = std::fs::read_to_string(path).unwrap_or_else(|e| { - eprintln!("aura: {path}: {e}"); - std::process::exit(2); - }); - if let Err(e) = blueprint_from_json(&doc, &|t| std_vocabulary(t)) { - eprintln!("aura: {path}: {e:?}"); - std::process::exit(2); - } - let WfBlueprintArgs { axes, name, select } = - parse_walkforward_blueprint_args(&rest[1..]).unwrap_or_else(|msg| { - eprintln!("aura: {msg}"); - std::process::exit(2); - }); - run_blueprint_walkforward(&doc, &axes, &name, DataSource::Synthetic, select); - return; - } - match parse_walkforward_args(rest) { - Ok((strategy, name, persist, choice, grid, select)) => { - run_walkforward(strategy, &name, persist, DataSource::from_choice(choice), &grid, select) - } - Err(msg) => { - eprintln!("aura: {msg}"); - std::process::exit(2); - } - } - } -``` - -(Confirm the existing hard-wired `parse_walkforward_args` arm is preserved verbatim -inside the `else`; match its exact current tuple/body when transcribing.) - -- [ ] **Step 6: Verify the unit test passes + the workspace builds** - -Run: `cargo test -p aura-cli --bin aura parse_walkforward_blueprint_args` -Expected: PASS. - -Run: `cargo build --workspace` -Expected: `Finished` — 0 errors (the verb now wires blueprint_walkforward_family → -no more unused-fn warnings for it or the helpers). - ---- - -## Task 3: The reproduce WalkForward branch - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` (`reproduce_family_in` match ~2566) -- Test: `crates/aura-cli/tests/cli_run.rs` - -- [ ] **Step 1: Write the failing E2E** - -Add to `crates/aura-cli/tests/cli_run.rs` (beside the mc-reproduce template -`aura_mc_over_a_blueprint_reproduces_bit_identically` ~3495): - -```rust -/// E2E (#173): an IS-refit walk-forward over a loaded blueprint persists a -/// content-addressed WalkForward family that `aura reproduce`s bit-identically — -/// each OOS member's windowed slice + winner params are recovered from its manifest. -#[test] -fn aura_walkforward_over_a_blueprint_reproduces_bit_identically() { - let cwd = temp_cwd("blueprint-walkforward-reproduce"); - let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR")); - let run = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3", - "--axis", "stage1_signal.slow.length=4,6", "--name", "wfr"]) - .current_dir(&cwd) - .output() - .expect("spawn aura walkforward"); - assert_eq!(run.status.code(), Some(0), "stderr: {}", String::from_utf8_lossy(&run.stderr)); - let repro = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args(["reproduce", "wfr-0"]) - .current_dir(&cwd) - .output() - .expect("spawn aura reproduce"); - assert_eq!(repro.status.code(), Some(0)); - let out = String::from_utf8(repro.stdout).expect("utf-8"); - assert!(out.contains("reproduced 3/3 members bit-identically"), "stdout: {out}"); - let _ = std::fs::remove_dir_all(&cwd); -} -``` - -- [ ] **Step 2: Run to verify it fails (RED — reproduce reconstructs the wrong source)** - -Run: `cargo test -p aura-cli --test cli_run aura_walkforward_over_a_blueprint_reproduces` -Expected: FAIL — reproduce falls through the `_` arm (full-window `run_sources`), -so the re-run does not match the OOS-windowed original (not bit-identical), or a -non-zero exit. - -- [ ] **Step 3: Add the `WalkForward` arm to `reproduce_family_in`** - -In the `match family.kind` block (~2566-2573), add a `FamilyKind::WalkForward` arm -before the `_ =>` fallthrough. It supplies only the (sources, window) pair — the -winner params are recovered by the SAME shared `point_from_params(&space, -&stored.manifest.params)` line the Sweep/MC arms already use: - -```rust - FamilyKind::WalkForward => { - // each member is one OOS window: rebuild its windowed slice from the - // stored window bounds; the winner params come from the shared - // manifest->cells recovery below (as a Sweep member's do). - let (from, to) = stored.manifest.window; - let s = data.windowed_sources(from, to); - let w = window_of(&s).expect("non-empty OOS window"); - (s, w) - } -``` - -(Insert it as a new match arm; keep the existing `FamilyKind::MonteCarlo` arm and -the `_ =>` fallthrough. Match the exact binding names the surrounding code uses — -transcribe the current arm shape at ~2566 before editing.) - -- [ ] **Step 4: Verify the E2E passes** - -Run: `cargo test -p aura-cli --test cli_run aura_walkforward_over_a_blueprint_reproduces` -Expected: PASS — "reproduced 3/3 members bit-identically", exit 0. - ---- - -## Task 4: E2E coverage (persist shape, no-axis, malformed) + the full gate - -**Files:** -- Test: `crates/aura-cli/tests/cli_run.rs` - -- [ ] **Step 1: Write the three E2Es** - -Add to `crates/aura-cli/tests/cli_run.rs` (beside the sweep-persist template -`aura_sweep_loads_a_blueprint_and_persists_a_sweep_family` ~3269): - -```rust -/// E2E (#173): the loaded-blueprint walk-forward runs + persists a WalkForward -/// family (3 OOS members) with a {"walkforward":…} summary carrying oos_r, and -/// stores exactly one content-addressed blueprint. -#[test] -fn aura_walkforward_over_a_blueprint_persists_a_family() { - let cwd = temp_cwd("blueprint-walkforward-persist"); - let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR")); - let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3", - "--axis", "stage1_signal.slow.length=4,6", "--name", "wfx"]) - .current_dir(&cwd) - .output() - .expect("spawn aura walkforward"); - assert_eq!(out.status.code(), Some(0), "stderr: {}", String::from_utf8_lossy(&out.stderr)); - let stdout = String::from_utf8(out.stdout).expect("utf-8"); - assert_eq!(stdout.matches("wfx-").count(), 3, "3 OOS member lines: {stdout}"); - assert!(stdout.contains("\"walkforward\""), "summary line: {stdout}"); - assert!(stdout.contains("\"oos_r\""), "R-measured summary: {stdout}"); - let families = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args(["runs", "families"]).current_dir(&cwd).output().expect("spawn families"); - let fo = String::from_utf8(families.stdout).expect("utf-8"); - assert!(fo.contains("\"kind\":\"WalkForward\""), "families: {fo}"); - let n = std::fs::read_dir(cwd.join("runs").join("blueprints")).map(|d| d.count()).unwrap_or(0); - assert_eq!(n, 1, "exactly one content-addressed blueprint stored"); - let _ = std::fs::remove_dir_all(&cwd); -} - -/// E2E (#173): an OPEN blueprint with no --axis (nothing to re-fit) is a usage -/// error (exit 2), naming that >= 1 --axis is required. -#[test] -fn aura_walkforward_over_a_blueprint_rejects_no_axis() { - let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR")); - let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args(["walkforward", &bp]) - .output() - .expect("spawn aura walkforward (no axis)"); - assert_eq!(out.status.code(), Some(2)); - let stderr = String::from_utf8(out.stderr).expect("utf-8"); - assert!(stderr.contains("requires >= 1 --axis"), "stderr: {stderr}"); -} - -/// E2E (#173, safety): a malformed blueprint is rejected at the dispatch boundary -/// (exit 2, UnknownNodeType) rather than panicking. -#[test] -fn aura_walkforward_over_a_blueprint_rejects_malformed() { - let bp = format!("{}/tests/fixtures/unknown_node.json", env!("CARGO_MANIFEST_DIR")); - let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3"]) - .output() - .expect("spawn aura walkforward (malformed)"); - assert_eq!(out.status.code(), Some(2), "malformed must be rejected, not panic"); - let stderr = String::from_utf8(out.stderr).expect("utf-8"); - assert!(stderr.contains("UnknownNodeType"), "stderr: {stderr}"); - assert!(!stderr.contains("panicked"), "must reject cleanly: {stderr}"); -} -``` - -- [ ] **Step 2: Run the three new E2Es** - -Run: `cargo test -p aura-cli --test cli_run aura_walkforward_over_a_blueprint` -Expected: PASS — 4 tests (the three above + the reproduce test from Task 3, which -shares the `aura_walkforward_over_a_blueprint` prefix). - -- [ ] **Step 3: Full-suite + lint gate** - -Run: `cargo test --workspace` -Expected: PASS — no regressions (esp. the hard-wired walkforward tests -`walkforward_plateau_select_stamps_plateau_provenance`, -`walkforward_select_argmax_is_byte_identical_to_default`, and the sweep/mc/reproduce -E2Es — the dispatch + reproduce edits preserved them). - -Run: `cargo clippy --workspace --all-targets -- -D warnings` -Expected: `Finished` — 0 warnings (every new fn is now wired; no dead code). - ---- - -## Self-review (planner Step 5) - -1. **Spec coverage:** §Concrete shapes (blueprint_sweep_over / run_oos_blueprint / - blueprint_walkforward_family / WfBlueprintArgs+parse / dispatch / reproduce) → - Tasks 1-3; §Testing (unit family, unit parse, 4 E2Es) → Tasks 1/2/3/4; - §Error handling (no-axis, malformed, bad-axis in-closure exit) → Task 2 parse + - Task 4 E2Es + Task 1 blueprint_sweep_over Result; §Persist + reproduce → Tasks - 2+3 (land together, one commit); §Acceptance 1-7 → the E2Es + the Task-4 gate. ✓ -2. **Placeholder scan:** no TBD/TODO/"similar to"/"add appropriate". The - transcribe-the-current-arm notes (dispatch else-branch, reproduce arm) name the - exact site + say match it verbatim — not deferred decisions. ✓ -3. **Type/name consistency:** blueprint_sweep_over / run_oos_blueprint / - blueprint_walkforward_family / WfBlueprintArgs / parse_walkforward_blueprint_args / - run_blueprint_walkforward / FamilyKind::WalkForward / walkforward_member_reports / - Selection / select_winner / walk_forward / WindowRun / WindowBounds consistent - across tasks and match the recon signatures. `blueprint_walkforward_family` - returns `WalkForwardResult` (not Result) in Task 1, Task 2's wrapper, and the - unit test — consistent. ✓ -4. **Step granularity:** each step is one fn/edit or one command. ✓ -5. **No commit steps.** ✓ -6. **Pin/replacement contiguity:** E2E pins — `"reproduced 3/3 members - bit-identically"` (the reproduce path's literal, reused from the mc-reproduce - template), `"kind":"WalkForward"` (append_family serialization), `"walkforward"` - / `"oos_r"` (walkforward_summary_json keys), `"requires >= 1 --axis"` (the - parse Err literal in Task 2 Step 3), `UnknownNodeType` (engine error) — each - appears contiguously in the corresponding impl/literal. ✓ -7. **Compile-gate vs. deferred-caller:** the new `WfBlueprintArgs` struct + its - destructure in the dispatch are BOTH in Task 2 (Steps 3+5) before Task 2's - build gate (Step 6). No caller deferred. Task 1's fns are unused until Task 2 — - handled by using `cargo build` (warning-tolerant) not clippy in Tasks 1-3, with - the `-D warnings` clippy gate in Task 4 after wiring. ✓ -8. **Verification filters resolve:** `blueprint_walkforward_family` (Task 1 new), - `parse_walkforward_blueprint_args` (Task 2 new), - `aura_walkforward_over_a_blueprint_reproduces` (Task 3 new), - `aura_walkforward_over_a_blueprint` (Task 4 — matches the 4 WF E2Es by shared - prefix), `walkforward_plateau_select_stamps_plateau_provenance` (existing, Task - 4 regression) — every filter matches ≥1 named test in the post-task tree. ✓ diff --git a/docs/specs/0097-is-refit-walk-forward-over-a-loaded-blueprint.md b/docs/specs/0097-is-refit-walk-forward-over-a-loaded-blueprint.md deleted file mode 100644 index 7570f61..0000000 --- a/docs/specs/0097-is-refit-walk-forward-over-a-loaded-blueprint.md +++ /dev/null @@ -1,314 +0,0 @@ -# In-sample-refit walk-forward over a loaded blueprint — Design Spec - -**Date:** 2026-07-01 -**Status:** Draft — awaiting user spec review -**Authors:** orchestrator + Claude - -## Goal - -Ship the World/C21 milestone's last piece: a **true in-sample-refit walk-forward -over a loaded blueprint**. `aura walkforward --axis = -[--axis …]` rolls IS/OOS windows over a synthetic span; on each IS window it -**re-optimizes** the blueprint's params over the user's `--axis` grid, runs the -winner **out-of-sample**, and aggregates the per-window OOS reports into a -`FamilyKind::WalkForward` family that is content-addressed and `aura reproduce`s -bit-identically. This is Arm A (the user's decision, #173) — the honest OOS -validation the milestone exists to enable, and the form that legitimately carries -the "walk-forward" name (matching the hard-wired verb's IS-optimizing guarantee). - -## Architecture - -The hard-wired `aura walkforward --strategy stage1-r` ALREADY does IS-refit -(`walkforward_family`'s `Stage1R` arm, `crates/aura-cli/src/main.rs`): for each -window it `stage1_r_sweep_over(is) → select_winner("sqn_normalized") → -run_oos_r(winner, oos)`. The loaded-blueprint form is the **same shape with one -substitution** — the per-window IS-sweep and OOS-run source the **loaded -blueprint** (wrapped by `wrap_stage1r` via cycle-0096's `blueprint_axis_probe`) -instead of the built-in strategy. Everything else is reused verbatim: the generic -engine driver `walk_forward(roller, space, run_window)` (strategy-agnostic, -`crates/aura-engine/src/walkforward.rs`), `select_winner` (Argmax-deflated / -Plateau), the `24/12/12`-over-60-bar synthetic schedule (`wf_full_span` / -`wf_window_sizes`), and `FamilyKind::WalkForward` + `walkforward_member_reports` + -`walkforward_summary_json`. - -Reused windowing seam: `data.windowed_sources(from, to)` (the same seam -`stage1_r_sweep_over` and `run_oos_r` use). No look-ahead: IS strictly precedes -OOS in every window (`WindowRoller`), and the loaded-member path (`run_blueprint_member`) -already only sees its window's stream (invariants 1/2). - -Persistence + reproduce are shipped **together** (they are coupled — a persisted -family with no store hook reports members as "blueprint missing", the #170 -hazard). The family is stored once by the shared `topology_hash` -(`put_blueprint`, the C18 hook the sweep/mc use) + `append_family`; `aura -reproduce` gains a `WalkForward` branch that reconstructs each OOS member's -windowed source (from its manifest window) and re-runs it with the member's -stored winner params — mirroring the `MonteCarlo` reproduce branch from cycle -0095. One signal topology → one shared `topology_hash` across all windows; each -member manifest carries `(topology_hash, window, chosen params, selection)`. - -Scope: `crates/aura-cli` only (+ read-only reuse of the engine/registry WF -machinery). No engine/registry change — invariants 8/9 hold. - -## Concrete code shapes - -### Worked user program (the acceptance evidence) - -A researcher has an OPEN strategy blueprint and wants an honest walk-forward that -re-fits its two SMA lengths each window (the axis names come from `--list-axes`, -cycle 0096): - -```console -$ aura sweep crates/aura-cli/tests/fixtures/stage1_signal_open.json --list-axes -stage1_signal.fast.length:I64 -stage1_signal.slow.length:I64 - -$ aura walkforward crates/aura-cli/tests/fixtures/stage1_signal_open.json \ - --axis stage1_signal.fast.length=2,3 --axis stage1_signal.slow.length=4,6 --name wf1 -walkforward-0 -walkforward-1 -walkforward-2 -{"walkforward":{"windows":3,"stitched_total_pips":,"param_stability":<…>,"oos_r":{…}}} - -$ # content-addressed: the family reproduces bit-identically -$ aura reproduce walkforward-0 -reproduced 3/3 members bit-identically -``` - -Per window the IS grid (2×2 here) is re-optimized by `sqn_normalized`; the winner -runs OOS; the three OOS reports are the family members (line shape identical to -the hard-wired `aura walkforward`, which reuses `family_member_line`). Exact -metric numbers are run-determined; the *shape* — three re-fit windows, OOS -members, a `{"walkforward":…}` summary, and a bit-identical reproduce — is the -criterion. - -An OPEN blueprint with **no** `--axis` (nothing to re-fit) is a usage error: - -```console -$ aura walkforward crates/aura-cli/tests/fixtures/stage1_signal_open.json -aura: walkforward requires >= 1 --axis to re-fit per window -$ echo $? -2 -``` - -### Implementation shape (secondary) - -**The family builder** — the loaded-blueprint analog of `walkforward_family`'s -`Stage1R` arm; reuses the generic `walk_forward` driver, substituting the loaded -blueprint into the per-window closure: - -```rust -fn blueprint_walkforward_family( - doc: &str, axes: &[(String, Vec)], data: &DataSource, select: Selection, -) -> WalkForwardResult { - let span = data.wf_full_span(); - let (is_len, oos_len, step) = data.wf_window_sizes(); - let roller = WindowRoller::new(span, is_len, oos_len, step, RollMode::Rolling) - .unwrap_or_else(|e| { eprintln!("aura: walk-forward window too short: {e:?}"); std::process::exit(2); }); - let space = blueprint_axis_probe(doc).param_space(); // the wrapped namespace (cycle 0096) - let pip = data.pip_size(); - let topo = topology_hash(&blueprint_from_json(doc, &|t| std_vocabulary(t)) - .expect("doc parse-validated at the dispatch boundary")); - walk_forward(roller, space.clone(), |w: WindowBounds| { - // IS: re-fit the loaded blueprint over the user grid, windowed to [is] - let (is_family, lattice) = blueprint_sweep_over(doc, axes, w.is.0, w.is.1, data); - let (best, selection) = select_winner(&is_family, "sqn_normalized", select, lattice.as_deref()) - .unwrap_or_else(|msg| { eprintln!("aura: {msg}"); std::process::exit(2); }); - // OOS: run the winner over [oos] via the loaded-member path - let (oos_equity, mut oos_report) = - run_oos_blueprint(doc, &best.params, &space, w.oos.0, w.oos.1, pip, &topo, data); - oos_report.manifest.selection = Some(selection); - WindowRun { chosen_params: best.params, oos_equity, oos_report } - }) -} -``` - -**The windowed IS-sweep** — `blueprint_sweep_over`: the structural twin of -`stage1_r_sweep_over` (uses `sweep_with_lattice` so `--select plateau` works) and -of `blueprint_sweep_family` (loaded blueprint, `--axis` grid), but windowed to -`[from,to]` via `data.windowed_sources`: - -```rust -fn blueprint_sweep_over(doc: &str, axes: &[(String, Vec)], - from: Timestamp, to: Timestamp, data: &DataSource) - -> (SweepFamily, Option>) { - let pip = data.pip_size(); - let topo = topology_hash(&reload(doc)); - let probe = blueprint_axis_probe(doc); - let space = probe.param_space(); - // seed the user axes verbatim (same as blueprint_sweep_family), then sweep WITH lattice - let mut binder = /* probe.axis(first) then SweepBinder::axis(rest) */; - binder.sweep_with_lattice(|point| { - let sources = data.windowed_sources(from, to); - let window = window_of(&sources).expect("non-empty in-sample window"); - run_blueprint_member(reload(doc), point, &space, sources, window, 0, pip, &topo) - }).map(|(fam, lat)| (fam, Some(lat))) - .expect("the user axes are name/kind-checked at the sweep terminal") -} -``` - -**The windowed OOS-run** — `run_oos_blueprint`: the loaded-member analog of -`run_oos_r`, running the winner params over the OOS window. The loaded-member -path (`run_blueprint_member`) is **reduce-mode**: it folds to R-metrics -(`summarize_r`) and retains the per-trade R series, but NOT the raw per-cycle pip -curve the hard-wired non-reduce `run_oos_r` returns. So `oos_equity` is **empty** -— the loaded walk-forward is measured in **R** (the C10-reworked yardstick), and -the meaningful per-window record is the OOS `RunReport`'s R-metrics: - -```rust -fn run_oos_blueprint(doc: &str, params: &[Cell], space: &[ParamSpec], - from: Timestamp, to: Timestamp, pip: f64, topo: &str, data: &DataSource) - -> (Vec<(Timestamp, f64)>, RunReport) { - let sources = data.windowed_sources(from, to); - let window = window_of(&sources).expect("non-empty out-of-sample window"); - let report = run_blueprint_member(reload(doc), params, space, sources, window, 0, pip, topo); - // reduce-mode: no raw pip curve retained -> empty stitching segment. The OOS - // record is R-metrics (report.metrics.r); an empty segment leaves the stitched - // curve unbroken (stitch treats it as a 0-offset, no-point contribution). - (Vec::new(), report) -} -``` - -Consequence for the summary: `walkforward_summary_json`'s `stitched_total_pips` -is `0.0` (no retained pip curve — the loaded path is R-measured, not pip-measured), -and its `oos_r` block (pooled per-trade Rs across the OOS windows) is the -meaningful line. This matches the R-first world of the C10 rework; a raw pip-equity -curve would need a non-reduce OOS run (`--trace`-style), out of scope here. - -**The IO wrapper + dispatch** — `run_blueprint_walkforward` mirrors -`run_walkforward` (persist via `put_blueprint` + `append_family`, print members + -summary); the `["walkforward", rest @ ..]` dispatch gains the `.json` -discriminator BEFORE `parse_walkforward_args`, mirroring the sweep/mc arms: - -```rust -["walkforward", rest @ ..] => { - if let Some(path) = rest.first().filter(|a| a.ends_with(".json") && Path::new(a).is_file()) { - let doc = std::fs::read_to_string(path).unwrap_or_else(|e| { eprintln!("aura: {path}: {e}"); exit(2) }); - if let Err(e) = blueprint_from_json(&doc, &|t| std_vocabulary(t)) { - eprintln!("aura: {path}: {e:?}"); exit(2); - } - let WfBlueprintArgs { axes, name, persist, select } = - parse_walkforward_blueprint_args(&rest[1..]).unwrap_or_else(|m| { eprintln!("aura: {m}"); exit(2) }); - run_blueprint_walkforward(&doc, &axes, &name, persist, DataSource::Synthetic, select); - return; - } - match parse_walkforward_args(rest) { /* unchanged hard-wired path */ } -} -``` - -**The reproduce branch** — `reproduce_family_in` already re-runs each member via -`run_blueprint_member` with a per-member `(sources, window, params)` triple chosen -by `family.kind` (cycle 0094 recovers the member's params from its manifest -against the reloaded space; cycle 0095 added the `MonteCarlo` arm that rebuilds -the seed-driven walk). The new `WalkForward` arm keeps the **same per-member param -recovery** (the winner params are stored in each OOS member's manifest, exactly as -a sweep member's are) and differs only in the **source**: the OOS window's slice, -rebuilt from the member manifest's stored window bounds: - -```rust -match family.kind { - FamilyKind::MonteCarlo => { /* cycle 0095: manifest.seed -> synthetic walk */ } - FamilyKind::WalkForward => { - // the OOS slice from the member manifest's window; params via the SAME - // manifest->cells recovery the Sweep arm already uses (winner params). - let (from, to) = stored.manifest.window; - let sources = data.windowed_sources(from, to); - (sources, window_of(&sources)/* the member window */) - } - _ => (data.run_sources(), data.full_window()), // Sweep: the showcase series -} -``` - -The member's params are recovered by the existing shared path (not re-invented -here); only the `(sources, window)` pair is realization-specific — `WalkForward` -supplies the windowed OOS slice, exactly as `MonteCarlo` supplies the seeded walk. - -## Components - -- `blueprint_walkforward_family(doc, axes, data, select) -> WalkForwardResult` - (new) — the per-window re-fit driver over the loaded blueprint. -- `blueprint_sweep_over(doc, axes, from, to, data)` (new) — windowed IS-sweep with - lattice (twin of `stage1_r_sweep_over` + `blueprint_sweep_family`). -- `run_oos_blueprint(doc, params, space, from, to, pip, topo, data)` (new) — - windowed OOS run over the loaded-member path. -- `run_blueprint_walkforward(doc, axes, name, persist, data, select)` (new) — IO - wrapper: `put_blueprint` + `append_family(FamilyKind::WalkForward)` + print - members + `walkforward_summary_json`. -- `WfBlueprintArgs` + `parse_walkforward_blueprint_args` (new) — `--axis` - (≥1 required) + `--select` + `--name|--trace`; rejects a no-axis invocation. -- `["walkforward", ..]` dispatch (modified) — `.json` discriminator. -- `reproduce_family_in` (modified) — `WalkForward` realization branch. - -## Data flow - -`aura walkforward --axis …` → boundary read + `blueprint_from_json` -validate → `parse_walkforward_blueprint_args` → `blueprint_walkforward_family`: -`WindowRoller` rolls 3 windows → per window `blueprint_sweep_over(is)` re-fits the -grid (each member a windowed loaded-blueprint run) → `select_winner` picks by -`sqn_normalized` → `run_oos_blueprint(oos)` runs the winner → `WindowRun` → -`WalkForwardResult` → `put_blueprint(topo)` + `append_family(WalkForward)` → print -per-window OOS members + summary. `aura reproduce ` → reload blueprint by -`topology_hash` → per member reconstruct `windowed_sources(manifest.window)` + -`manifest.params` → `run_blueprint_member` → assert bit-identical. - -## Error handling - -- **OPEN blueprint with no `--axis`** → usage error (exit 2): a walk-forward with - nothing to re-fit is meaningless; mirror the loaded-blueprint sweep's "≥1 axis" - contract. Message names that ≥1 `--axis` is required. -- **Unknown / kind-mismatched `--axis` name** → the sweep terminal's - `UnknownKnob`/kind error, rendered `aura: …` + exit 2 (reused from the - loaded-blueprint sweep; the axis names come from `--list-axes`). -- **Malformed / unreadable blueprint** → rejected at the dispatch boundary - (`blueprint_from_json`, exit 2, `UnknownNodeType`), before any run — pinned by - an E2E (as cycle 0096 did), so the internal `.expect` reloads are unreachable. -- **`--select plateau` without a lattice** → `select_winner` already returns the - named refusal; the loaded IS-sweep uses `sweep_with_lattice`, so a grid sweep - always has a lattice (the refusal is unreachable here but preserved). -- **Window too short for one IS+OOS span** → `WindowRoller::new` error, exit 2 - (reused). -- No panics on any user-reachable input; `walk_forward` runs windows disjointly - in parallel, deterministic in roll order (C1). - -## Testing strategy - -- **Unit — `blueprint_walkforward_family`** (`cargo test -p aura-cli --bin aura - blueprint_walkforward_family`): over `stage1_signal_open.json` with a 2×2 - `--axis` grid → a `WalkForwardResult` with 3 windows, each `chosen_params` of - arity 2 (both axes re-fit), and each window's OOS `RunReport` carrying R-metrics - (`metrics.r`, incl. `sqn_normalized`). The stitched OOS pip-equity curve is - **empty** (reduce-mode retains no raw pip curve — the empty segments leave the - curve unbroken), so the per-window R-metrics, not a pip curve, are the OOS - record. Pins the per-window re-fit shape. -- **Unit — `parse_walkforward_blueprint_args`** (`cargo test -p aura-cli --bin - aura parse_walkforward_blueprint_args`): `--axis a=1,2 --select plateau:mean` - parses; a no-`--axis` tail → `Err`; `--select bogus` → `Err`. -- **E2E — the walk-forward runs + persists** (`cargo test -p aura-cli --test - cli_run `): `aura walkforward --axis - stage1_signal.fast.length=2,3 --axis stage1_signal.slow.length=4,6 --name wfx` - → exit 0, 3 `walkforward-*` OOS member lines, a `{"walkforward":…}` summary, - and exactly one `runs/blueprints/.json`. -- **E2E — reproduce bit-identical** (`--test cli_run`): after the above, `aura - reproduce wfx-0` → "reproduced 3/3 members bit-identically", exit 0. -- **E2E — no-axis rejection** (`--test cli_run`): `aura walkforward ` - (no `--axis`) → exit 2 + "requires >= 1 --axis" on stderr. -- **E2E — malformed-blueprint safety** (`--test cli_run`): `aura walkforward - crates/aura-cli/tests/fixtures/unknown_node.json --axis a=1,2` → exit 2 + - `UnknownNodeType` on stderr + no `panicked`. - -## Acceptance criteria - -1. `aura walkforward --axis = [--axis …]` re-optimizes - the blueprint's params on each IS window and runs the winner OOS, aggregating - to a `FamilyKind::WalkForward` family (per-window OOS member lines + a - `{"walkforward":…}` summary), exit 0. -2. The family is content-addressed (one `runs/blueprints/.json`) - and `aura reproduce`s every OOS member bit-identically. -3. An OPEN blueprint with no `--axis` is a usage error (exit 2), as is an unknown - axis name; a malformed blueprint is rejected at the boundary (exit 2, no panic). -4. `--select argmax` (default) and `--select plateau:mean|plateau:worst` both work - (the IS-sweep carries a lattice). -5. No look-ahead: IS strictly precedes OOS per window; runs are deterministic - (C1), parallelism across windows only. -6. No `aura-engine` / `aura-registry` change; invariants 1, 2, 8, 9 preserved. -7. `cargo build --workspace`, `cargo test --workspace`, `cargo clippy --workspace - --all-targets -- -D warnings` all green.