feat(0075): walk-forward strategy-selectable + R-reporting (iter 1)
Make `aura walkforward --strategy stage1-r [--real <SYM>]` roll IS->OOS windows, sweep the stage1-r grid in-sample, pick the winner by an R metric (sqn_normalized), run it out-of-sample, and report per-window + pooled OOS R-metrics. The bare SMA walkforward path is byte-identical (its dispatch arm is verbatim today's body; the pooled `oos_r` block is emitted only when a window carries an `r` block). Engine: - RMetrics gains an in-memory `trade_rs: Vec<f64>` (realised R per closed trade), excluded from serde (`#[serde(skip)]`) and from a hand-written PartialEq, so the C18 wire shape and every existing equality assertion / round-trip stay unchanged. summarize_r now retains the per-trade R vector it used to drop. - New `r_metrics_from_rs(&[f64])` reduces a flat (pooled across-window) R series to RMetrics. Its R-distribution arithmetic is copied verbatim from summarize_r (the byte-pinned floats must not be algebraically refactored); the two copies are guarded in lockstep by a cross-reducer equality test. net_expectancy_r = expectancy_r (exact at the Stage-1 cost=0 invariant); conviction_terciles_r = [0,0,0] (per-trade conviction is not pooled). CLI: - walkforward gains --strategy + the four stage1-r grid flags (reusing parse_csv_list / Stage1RGrid); walkforward_family is strategy-dispatched. - Windowed stage1-r helpers: stage1_r_sweep_over (reduce-mode folded IS sweep, O(trades)/member) and run_oos_r (non-reduce OOS run, for the stitched pip-equity curve), plus stage1_r_space. Frictionless Stage-1 R (costs are Stage-2). Verified: cargo build clean, full `cargo test --workspace` green (SMA walkforward, synthetic mc, stage1_r_single_run_output_golden, and the C18 round-trips all preserved), clippy -D warnings clean. Monte-Carlo R-bootstrap is iter 2. refs #139
This commit is contained in:
@@ -2310,3 +2310,100 @@ fn sweep_strategy_stage1_meanrev_grids_one_member_per_window() {
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&cwd);
|
||||
}
|
||||
|
||||
/// Property: `aura walkforward --strategy stage1-r` on synthetic data reports R
|
||||
/// quality per window AND pooled across windows — every per-window member line
|
||||
/// carries a `metrics.r` block (the windowed reduce-mode IS sweep folds the dense
|
||||
/// R-record), and the summary line carries a pooled `oos_r` block (the across-window
|
||||
/// reduction of the OOS per-trade R series). Deterministic: a second run is
|
||||
/// byte-identical (C1).
|
||||
#[test]
|
||||
fn walkforward_strategy_stage1_r_reports_oos_r() {
|
||||
let run = || {
|
||||
let cwd = temp_cwd("wf-stage1r");
|
||||
let out = Command::new(BIN)
|
||||
.args(["walkforward", "--strategy", "stage1-r"])
|
||||
.current_dir(&cwd)
|
||||
.output()
|
||||
.expect("spawn aura walkforward --strategy stage1-r");
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"walkforward --strategy stage1-r exit: {:?}; stderr: {}",
|
||||
out.status,
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
);
|
||||
let stdout = String::from_utf8(out.stdout).expect("utf-8");
|
||||
let _ = std::fs::remove_dir_all(&cwd);
|
||||
stdout
|
||||
};
|
||||
let out = run();
|
||||
let lines: Vec<&str> = out.trim().lines().collect();
|
||||
let summary: serde_json::Value = serde_json::from_str(lines.last().unwrap()).unwrap();
|
||||
assert!(summary["walkforward"]["oos_r"].is_object(), "summary carries oos_r");
|
||||
assert!(summary["walkforward"]["oos_r"]["n_trades"].is_number());
|
||||
// each per-window member line carries metrics.r
|
||||
for l in &lines[..lines.len() - 1] {
|
||||
let v: serde_json::Value = serde_json::from_str(l).unwrap();
|
||||
assert!(v["report"]["metrics"]["r"].is_object(), "per-window r block present");
|
||||
}
|
||||
let out2 = run();
|
||||
assert_eq!(out, out2, "stage1-r walkforward is deterministic");
|
||||
}
|
||||
|
||||
/// Property: the bare `aura walkforward` (default SMA-cross) summary is preserved
|
||||
/// byte-for-byte by the new `--strategy`/grid plumbing — it still carries exactly
|
||||
/// `windows`, `stitched_total_pips`, `param_stability` and NOT the stage1-r-only
|
||||
/// `oos_r` block. The spec promised the bare path's golden is unchanged; the
|
||||
/// signature widening defaults to SmaCross and the summary gates `oos_r` on
|
||||
/// `metrics.r.is_some()`, so a regression that leaked R-reporting (or any other
|
||||
/// key) into the SMA summary would be a silent contract break. This pins the
|
||||
/// negative: the SMA arm produces no `metrics.r`, hence no `oos_r`.
|
||||
#[test]
|
||||
fn walkforward_bare_sma_summary_has_no_oos_r() {
|
||||
let cwd = temp_cwd("wf-bare-sma");
|
||||
let out = Command::new(BIN)
|
||||
.arg("walkforward")
|
||||
.current_dir(&cwd)
|
||||
.output()
|
||||
.expect("spawn aura walkforward");
|
||||
assert!(out.status.success(), "bare walkforward exit: {:?}", out.status);
|
||||
let stdout = String::from_utf8(out.stdout).expect("utf-8");
|
||||
let lines: Vec<&str> = stdout.trim().lines().collect();
|
||||
let summary: serde_json::Value = serde_json::from_str(lines.last().unwrap()).unwrap();
|
||||
let wf = &summary["walkforward"];
|
||||
assert!(wf["oos_r"].is_null(), "bare SMA summary must NOT carry oos_r: {wf}");
|
||||
assert!(wf["windows"].is_number(), "bare SMA summary keeps windows: {wf}");
|
||||
assert!(wf["stitched_total_pips"].is_number(), "bare SMA summary keeps stitched_total_pips: {wf}");
|
||||
assert!(wf["param_stability"].is_array(), "bare SMA summary keeps param_stability: {wf}");
|
||||
// no per-window member line carries a metrics.r block (R-reporting is stage1-r-only)
|
||||
for l in &lines[..lines.len() - 1] {
|
||||
let v: serde_json::Value = serde_json::from_str(l).unwrap();
|
||||
assert!(v["report"]["metrics"]["r"].is_null(), "bare SMA per-window line must NOT carry metrics.r: {l}");
|
||||
}
|
||||
let _ = std::fs::remove_dir_all(&cwd);
|
||||
}
|
||||
|
||||
/// Property: a strategy that parses but has no walk-forward form yet (e.g.
|
||||
/// `stage1-breakout`) is rejected with exit 2 and a stderr message that names the
|
||||
/// OFFENDING strategy by its CLI token — not silently run as SMA, not crashed.
|
||||
/// The dispatch's catch-all arm routes through `Strategy::cli_token()`, so the
|
||||
/// diagnostic must echo the token the user typed (`stage1-breakout`), proving the
|
||||
/// inverse map is wired. Distinct from a bad `--strategy bogus` (a usage parse
|
||||
/// error): here the token is a valid strategy with no walk-forward arm.
|
||||
#[test]
|
||||
fn walkforward_unsupported_strategy_exits_2_naming_the_token() {
|
||||
let cwd = temp_cwd("wf-unsupported");
|
||||
let out = Command::new(BIN)
|
||||
.args(["walkforward", "--strategy", "stage1-breakout"])
|
||||
.current_dir(&cwd)
|
||||
.output()
|
||||
.expect("spawn aura walkforward --strategy stage1-breakout");
|
||||
assert_eq!(out.status.code(), Some(2), "unsupported walkforward strategy must exit 2");
|
||||
let stderr = String::from_utf8(out.stderr).expect("utf-8 stderr");
|
||||
assert!(
|
||||
stderr.contains("stage1-breakout"),
|
||||
"stderr must name the offending strategy token: {stderr:?}"
|
||||
);
|
||||
assert!(out.stdout.is_empty(), "no family summary on the rejected path: {:?}", out.stdout);
|
||||
let _ = std::fs::remove_dir_all(&cwd);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user