feat(0097): IS-refit walk-forward over a loaded blueprint — aura walkforward <bp.json> --axis

Ship the World/C21 milestone's last piece (Arm A, #173): a true in-sample-refit walk-forward over a loaded blueprint. `aura walkforward <blueprint.json> --axis <name>=<csv> [--axis …] [--select argmax|plateau:mean|plateau:worst]` rolls 24/12/12 IS/OOS windows over the synthetic span; per IS window it re-optimizes the blueprint's params over the user's --axis grid (the #169 prefixed names), selects the winner by sqn_normalized, runs it out-of-sample, and aggregates the per-window OOS reports into a FamilyKind::WalkForward family — content-addressed (put_blueprint) and aura-reproduce bit-identical. Structure mirrors the hard-wired stage1-r walk-forward arm one substitution deep: the generic engine walk_forward driver + select_winner + the synthetic schedule are reused; only the per-window IS-sweep (blueprint_sweep_over) and OOS-run (run_oos_blueprint) source the loaded blueprint via cycle-0096's blueprint_axis_probe. reproduce_family_in gains a WalkForward arm reconstructing each OOS window's windowed slice from the member manifest's window bounds (winner params via the shared manifest->cells recovery, as the MonteCarlo/Sweep arms do). Panic-safety: blueprint_sweep_over returns Result<_, BindError>; a bad/unknown --axis is a clean in-closure exit 2 (mirroring the hard-wired arm's select_winner handling), never a panic — pinned by the rejects_an_unknown_axis E2E. A no-axis invocation (nothing to re-fit) is a usage error; a malformed blueprint is rejected at the dispatch boundary (exit 2, no panic). Reduce-mode members carry R-metrics (no raw pip curve) → empty stitched equity; oos_r is the meaningful summary (C10 R-first yardstick). Held quality finding adjudicated: run_oos_blueprint's plan-prescribed redundant `pip: f64` (== data.pip_size()) removed — derived internally to match its siblings run_oos_r / blueprint_sweep_over, which also drops the too_many_arguments band-aid #[allow] (8->7 args). Two plan transcription typos the loop caught and fixed verified correct: BindError renders via Debug ({e:?}), parse_select via map_err (Result, not Option). Verified myself: cargo build --workspace, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace all green; 2 new unit tests + 5 new cli_run E2Es (persist shape, reproduce bit-identical, no-axis, malformed-safety, unknown-axis) pass; no regression in the hard-wired walkforward / sweep / mc / reproduce suites. Engine/registry untouched (invariants 1/2/8/9). closes #173
2026-07-01 16:38:58 +02:00
parent c8c34c72e7
commit ff20f74e8f
2 changed files with 335 additions and 7 deletions
@@ -3533,6 +3533,111 @@ fn aura_mc_over_a_blueprint_reproduces_bit_identically() {
    let _ = std::fs::remove_dir_all(&cwd);
 }

+/// E2E (#173): an IS-refit walk-forward over a loaded blueprint persists a
+/// content-addressed WalkForward family that `aura reproduce`s bit-identically —
+/// each OOS member's windowed slice + winner params are recovered from its manifest.
+#[test]
+fn aura_walkforward_over_a_blueprint_reproduces_bit_identically() {
+    let cwd = temp_cwd("blueprint-walkforward-reproduce");
+    let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR"));
+    let run = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
+        .args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3",
+               "--axis", "stage1_signal.slow.length=4,6", "--name", "wfr"])
+        .current_dir(&cwd)
+        .output()
+        .expect("spawn aura walkforward");
+    assert_eq!(run.status.code(), Some(0), "stderr: {}", String::from_utf8_lossy(&run.stderr));
+    let repro = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
+        .args(["reproduce", "wfr-0"])
+        .current_dir(&cwd)
+        .output()
+        .expect("spawn aura reproduce");
+    assert_eq!(repro.status.code(), Some(0));
+    let out = String::from_utf8(repro.stdout).expect("utf-8");
+    assert!(out.contains("reproduced 3/3 members bit-identically"), "stdout: {out}");
+    let _ = std::fs::remove_dir_all(&cwd);
+}
+
+/// E2E (#173): the loaded-blueprint walk-forward runs + persists a WalkForward
+/// family (3 OOS members) with a {"walkforward":…} summary carrying oos_r, and
+/// stores exactly one content-addressed blueprint.
+#[test]
+fn aura_walkforward_over_a_blueprint_persists_a_family() {
+    let cwd = temp_cwd("blueprint-walkforward-persist");
+    let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR"));
+    let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
+        .args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3",
+               "--axis", "stage1_signal.slow.length=4,6", "--name", "wfx"])
+        .current_dir(&cwd)
+        .output()
+        .expect("spawn aura walkforward");
+    assert_eq!(out.status.code(), Some(0), "stderr: {}", String::from_utf8_lossy(&out.stderr));
+    let stdout = String::from_utf8(out.stdout).expect("utf-8");
+    assert_eq!(stdout.matches("wfx-").count(), 3, "3 OOS member lines: {stdout}");
+    assert!(stdout.contains("\"walkforward\""), "summary line: {stdout}");
+    assert!(stdout.contains("\"oos_r\""), "R-measured summary: {stdout}");
+    let families = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
+        .args(["runs", "families"]).current_dir(&cwd).output().expect("spawn families");
+    let fo = String::from_utf8(families.stdout).expect("utf-8");
+    assert!(fo.contains("\"kind\":\"WalkForward\""), "families: {fo}");
+    let n = std::fs::read_dir(cwd.join("runs").join("blueprints")).map(|d| d.count()).unwrap_or(0);
+    assert_eq!(n, 1, "exactly one content-addressed blueprint stored");
+    let _ = std::fs::remove_dir_all(&cwd);
+}
+
+/// E2E (#173): an OPEN blueprint with no --axis (nothing to re-fit) is a usage
+/// error (exit 2), naming that >= 1 --axis is required.
+#[test]
+fn aura_walkforward_over_a_blueprint_rejects_no_axis() {
+    let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR"));
+    let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
+        .args(["walkforward", &bp])
+        .output()
+        .expect("spawn aura walkforward (no axis)");
+    assert_eq!(out.status.code(), Some(2));
+    let stderr = String::from_utf8(out.stderr).expect("utf-8");
+    assert!(stderr.contains("requires >= 1 --axis"), "stderr: {stderr}");
+}
+
+/// E2E (#173, safety): a malformed blueprint is rejected at the dispatch boundary
+/// (exit 2, UnknownNodeType) rather than panicking.
+#[test]
+fn aura_walkforward_over_a_blueprint_rejects_malformed() {
+    let bp = format!("{}/tests/fixtures/unknown_node.json", env!("CARGO_MANIFEST_DIR"));
+    let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
+        .args(["walkforward", &bp, "--axis", "stage1_signal.fast.length=2,3"])
+        .output()
+        .expect("spawn aura walkforward (malformed)");
+    assert_eq!(out.status.code(), Some(2), "malformed must be rejected, not panic");
+    let stderr = String::from_utf8(out.stderr).expect("utf-8");
+    assert!(stderr.contains("UnknownNodeType"), "stderr: {stderr}");
+    assert!(!stderr.contains("panicked"), "must reject cleanly: {stderr}");
+}
+
+/// E2E (#173, panic-safety): a `--axis` naming a knob that is not in the loaded
+/// blueprint's param_space fails clean (named BindError, exit 2, no panic). This
+/// pins the in-closure fallible-bind path — `blueprint_sweep_over` returns a
+/// `BindError` from inside the `walk_forward` window closure — which is a DISTINCT
+/// branch from the dispatch-boundary blueprint-parse rejection (the malformed test):
+/// the blueprint here is well-formed and passes dispatch, then the per-window
+/// re-fit's bind fails. A regression to `.expect()` on that bind would panic here
+/// while leaving the malformed test green.
+#[test]
+fn aura_walkforward_over_a_blueprint_rejects_an_unknown_axis() {
+    let bp = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR"));
+    let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
+        .args(["walkforward", &bp, "--axis", "nope=1,2"])
+        .output()
+        .expect("spawn aura walkforward (unknown axis)");
+    assert_eq!(out.status.code(), Some(2), "an unknown axis must fail clean, not panic");
+    let stderr = String::from_utf8(out.stderr).expect("utf-8");
+    assert!(
+        stderr.contains("Knob") && stderr.contains("nope"),
+        "names the unresolved axis via BindError: {stderr}"
+    );
+    assert!(!stderr.contains("panicked"), "must reject cleanly, never panic: {stderr}");
+}
+
 /// E2E (acc 2): `aura mc <open blueprint.json>` is rejected with a named error + exit 2
 /// (NOT a panic / exit 101) — MC binds no axis, so a free knob has no binder.
 #[test]