Iteration 1 of cycle 0075: RMetrics.trade_rs (in-memory, serde+eq-excluded) + summarize_r retains it; r_metrics_from_rs pooled reducer; walkforward --strategy + grid flags; strategy-dispatched walkforward_family (SMA arm verbatim); windowed stage1-r helpers (stage1_r_sweep_over reduce-mode IS sweep, run_oos_r non-reduce OOS run, stage1_r_space); pooled oos_r summary. Monte-Carlo bootstrap is iteration 2. refs #139
34 KiB
Walk-forward strategy-selectable + R-reporting (iteration 1) — Implementation Plan
Parent spec:
docs/specs/0075-walkforward-mc-r-validation.mdFor agentic workers: REQUIRED SUB-SKILL: use the
implementskill to run this plan. Steps use- [ ]checkboxes for tracking.
Goal: Make aura walkforward --strategy stage1-r --real <SYM> roll IS→OOS
windows, sweep the stage1-r grid in-sample, pick the best by an R metric, run the
chosen params OOS, and report per-window + across-window-pooled OOS R-metrics —
while the bare SMA walkforward path stays byte-identical.
Architecture: Two additive library changes in aura-engine (RMetrics carries
an in-memory trade_rs vector excluded from serde + equality; a new
r_metrics_from_rs pooled reducer), then aura-cli gains --strategy + grid
flags on walkforward, a strategy-dispatched walkforward_family (SMA arm
verbatim), windowed stage1-r helpers (stage1_r_sweep_over reduce-mode IS sweep,
run_oos_r non-reduce OOS run, stage1_r_space), and a pooled oos_r summary
block. The Monte-Carlo bootstrap half is iteration 2 (separate plan).
Tech Stack: Rust workspace — aura-engine (report.rs), aura-registry
(lib.rs test helper), aura-cli (main.rs). Test command:
cargo test --workspace (one positional filter per invocation).
Files this plan creates or modifies:
- Modify:
crates/aura-engine/src/report.rs:44-62—RMetricsgains#[serde(skip)] trade_rs: Vec<f64>; dropPartialEqfrom the derive, hand-write it (ignoringtrade_rs). - Modify:
crates/aura-engine/src/report.rs:89-219—summarize_rsetstrade_rsin both return arms; add newpub fn r_metrics_from_rs. - Modify:
crates/aura-registry/src/lib.rs:255—report_with_rtest helperRMetrics { .. }literal gainstrade_rs: Vec::new(). - Modify:
crates/aura-cli/src/main.rs:1526-1546—parse_walkforward_args3-tuple → 5-tuple (--strategy+ four grid flags). - Modify:
crates/aura-cli/src/main.rs:1615-1666—run_walkforward+walkforward_familygainstrategy/grid; strategy dispatch. - Modify:
crates/aura-cli/src/main.rs:1734-1744—walkforward_summary_jsongains theoos_rpooled block. - Modify:
crates/aura-cli/src/main.rs:2591-2592— USAGE walkforward fragment. - Modify:
crates/aura-cli/src/main.rs:2637-2645— walkforward dispatch arm destructures the 5-tuple. - Create (new fns in
crates/aura-cli/src/main.rs):stage1_r_sweep_over,run_oos_r,stage1_r_space. - Test:
crates/aura-engine/src/report.rs(#cfg test) —summarize_rpopulatestrade_rs;RMetricsround-trip equal despite skippedtrade_rs;r_metrics_from_rsgoldens. - Test:
crates/aura-cli/src/main.rs(#cfg test) —parse_walkforward_args5-tuple +--strategy/grid; update the two existing assertions at 3653-3656. - Test:
crates/aura-cli/tests/cli_run.rs—aura walkforward --strategy stage1-remits per-windowrblocks + anoos_rsummary (synthetic).
Task 1: RMetrics.trade_rs — in-memory per-trade R series
Files:
-
Modify:
crates/aura-engine/src/report.rs:44-62, 89-135, 206-219 -
Modify:
crates/aura-registry/src/lib.rs:255 -
Test:
crates/aura-engine/src/report.rs(#cfg test) -
Step 1: Write the failing tests — append to the
#[cfg(test)] mod testsincrates/aura-engine/src/report.rs:
#[test]
fn summarize_r_populates_trade_rs_in_trade_order() {
// two closed trades (R = +2.0, then -1.0) over a minimal PositionManagement
// record; trade_rs must carry [2.0, -1.0] in trade order.
let rec = pm_record_two_closed_trades(); // helper below
let m = summarize_r(&rec, 0.0);
assert_eq!(m.trade_rs, vec![2.0, -1.0]);
assert_eq!(m.n_trades, 2);
}
#[test]
fn summarize_r_empty_record_has_empty_trade_rs() {
let m = summarize_r(&[], 0.0);
assert!(m.trade_rs.is_empty());
assert_eq!(m.n_trades, 0);
}
#[test]
fn rmetrics_partial_eq_ignores_trade_rs() {
// two RMetrics equal in every metric but differing in trade_rs compare EQUAL
// (trade_rs is an in-memory conduit, excluded from equality) — this is what
// keeps serialize->deserialize round-trips equal (trade_rs is serde-skipped,
// so it deserializes empty).
let a = summarize_r(&pm_record_two_closed_trades(), 0.0);
let mut b = a.clone();
b.trade_rs = Vec::new();
assert_eq!(a, b);
}
/// A minimal dense PositionManagement record with two closed trades at R = +2, -1.
/// Columns per `r_col` (CLOSED=0, REALIZED_R=1, DIRECTION=4, ENTRY_PRICE=6,
/// STOP_PRICE=7, CONVICTION_AT_ENTRY=9, SIZE=10, OPEN=11, UNREALIZED_R=12); width 13.
fn pm_record_two_closed_trades() -> Vec<(Timestamp, Vec<Scalar>)> {
let row = |closed: bool, r: f64, open: bool| {
let mut c = vec![Scalar::f64(0.0); 13];
c[0] = Scalar::bool(closed);
c[1] = Scalar::f64(r);
c[6] = Scalar::f64(1.0); // entry
c[7] = Scalar::f64(0.5); // stop -> latched 0.5
c[9] = Scalar::f64(0.3); // conviction
c[11] = Scalar::bool(open);
c
};
vec![
(Timestamp(1), row(true, 2.0, false)),
(Timestamp(2), row(true, -1.0, false)),
]
}
- Step 2: Run to verify it fails
Run: cargo test --workspace summarize_r_populates_trade_rs_in_trade_order
Expected: FAIL — compile error no field trade_rson typeRMetrics`` (and the
two siblings fail to compile likewise).
- Step 3: Add the field + hand-written
PartialEq— incrates/aura-engine/src/report.rs, change theRMetricsderive (line 44) to dropPartialEq, add the field afterconviction_terciles_r(line 61), and add theimpl PartialEqdirectly after the struct:
// line 44: drop PartialEq from the derive
#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)]
pub struct RMetrics {
// ... all existing fields unchanged ...
pub conviction_terciles_r: [f64; 3],
/// Realised R per closed trade, in trade order — an in-memory conduit for the
/// OOS R-series bootstrap (#139). Excluded from serde (`skip`) so the C18 wire
/// shape is unchanged, and from `PartialEq` (below) so every existing
/// `RMetrics`/`RunReport` equality assertion and round-trip stays green.
#[serde(skip)]
pub trade_rs: Vec<f64>,
}
impl PartialEq for RMetrics {
fn eq(&self, o: &Self) -> bool {
self.expectancy_r == o.expectancy_r
&& self.n_trades == o.n_trades
&& self.win_rate == o.win_rate
&& self.avg_win_r == o.avg_win_r
&& self.avg_loss_r == o.avg_loss_r
&& self.profit_factor == o.profit_factor
&& self.max_r_drawdown == o.max_r_drawdown
&& self.n_open_at_end == o.n_open_at_end
&& self.sqn == o.sqn
&& self.sqn_normalized == o.sqn_normalized
&& self.net_expectancy_r == o.net_expectancy_r
&& self.conviction_terciles_r == o.conviction_terciles_r
// trade_rs deliberately excluded
}
}
- Step 4: Populate
trade_rsin bothsummarize_rarms — incrates/aura-engine/src/report.rs: the empty-input arm (line 122-135 literal) addstrade_rs: Vec::new(),, and the main return literal (line 206-219) addstrade_rs: rs,(thersvector built at line 137 is the per-trade realised R in trade order; it is last-read by the sqn block before the return, so the move is valid):
// empty-input arm (line 122) — add as the last field:
return RMetrics {
// ... existing zero fields ...
conviction_terciles_r: [0.0; 3],
trade_rs: Vec::new(),
};
// main return (line 206) — add as the last field:
RMetrics {
// ... existing fields ...
conviction_terciles_r,
trade_rs: rs,
}
- Step 5: Thread the other two
RMetrics { .. }literal sites (the only remaining struct literals workspace-wide — both test helpers):
// crates/aura-engine/src/report.rs — the runmetrics_with_r_block_round_trips test
// literal (~line 1135): add `trade_rs: Vec::new(),` as the last field.
// crates/aura-registry/src/lib.rs:255 — the report_with_r helper literal: add
// `trade_rs: Vec::new(),` as the last field.
- Step 6: Build gate (all literal sites threaded)
Run: cargo build --workspace
Expected: 0 errors (every RMetrics { .. } literal now carries trade_rs).
- Step 7: Run the new + back-compat tests
Run: cargo test --workspace summarize_r_populates_trade_rs_in_trade_order
Expected: PASS
Run: cargo test --workspace rmetrics_partial_eq_ignores_trade_rs
Expected: PASS
Run: cargo test --workspace runmetrics_with_r_block_round_trips
Expected: PASS (the serde-skip + hand-written eq keep the round-trip equal)
Task 2: r_metrics_from_rs — pooled reducer over a flat R series
Files:
-
Modify:
crates/aura-engine/src/report.rs(newpub fnaftersummarize_r) -
Test:
crates/aura-engine/src/report.rs(#cfg test) -
Step 1: Write the failing test — append to the report.rs test module:
#[test]
fn r_metrics_from_rs_folds_a_flat_series() {
// pooled across-window R series [2.0, -1.0, 1.0]: expectancy = 2/3, 2 wins of 3,
// profit_factor = (2+1)/1 = 3. At cost 0 (frictionless Stage-1) net == gross.
// conviction terciles are not pooled -> [0,0,0]; n_open_at_end is not a pooled
// concept -> 0.
let m = r_metrics_from_rs(&[2.0, -1.0, 1.0]);
assert_eq!(m.n_trades, 3);
assert!((m.expectancy_r - 2.0 / 3.0).abs() < 1e-12);
assert!((m.win_rate - 2.0 / 3.0).abs() < 1e-12);
assert!((m.profit_factor - 3.0).abs() < 1e-12);
assert_eq!(m.net_expectancy_r, m.expectancy_r);
assert_eq!(m.conviction_terciles_r, [0.0; 3]);
assert_eq!(m.n_open_at_end, 0);
}
#[test]
fn r_metrics_from_rs_empty_is_all_zero() {
let m = r_metrics_from_rs(&[]);
assert_eq!(m.n_trades, 0);
assert_eq!(m.expectancy_r, 0.0);
assert_eq!(m.sqn, 0.0);
}
- Step 2: Run to verify it fails
Run: cargo test --workspace r_metrics_from_rs_folds_a_flat_series
Expected: FAIL — cannot find function r_metrics_from_rs in this scope.
- Step 3: Add the reducer — in
crates/aura-engine/src/report.rs, directly aftersummarize_r(line 220). The R-distribution arithmetic is copied verbatim fromsummarize_r(lines 137-177) — the byte-pinned float expressions must not be algebraically refactored (a shared factoring would shift the golden by ~1 ULP);summarize_ritself is left untouched:
/// Reduce a flat per-trade R series (e.g. the pooled across-window OOS series of a
/// walk-forward) into `RMetrics`. The R-distribution fields use the same arithmetic
/// as [`summarize_r`] (copied verbatim — the pinned floats are not refactored). The
/// fields a flat R series cannot carry are set honestly: `n_open_at_end = 0`,
/// `net_expectancy_r = expectancy_r` (exact under the Stage-1 cost = 0 invariant),
/// `conviction_terciles_r = [0,0,0]` (per-trade conviction is not pooled). Empty
/// input -> a well-defined all-zero `RMetrics`.
pub fn r_metrics_from_rs(rs: &[f64]) -> RMetrics {
let n = rs.len() as u64;
if n == 0 {
return RMetrics {
expectancy_r: 0.0, n_trades: 0, win_rate: 0.0, avg_win_r: 0.0,
avg_loss_r: 0.0, profit_factor: 0.0, max_r_drawdown: 0.0,
n_open_at_end: 0, sqn: 0.0, sqn_normalized: 0.0, net_expectancy_r: 0.0,
conviction_terciles_r: [0.0; 3], trade_rs: Vec::new(),
};
}
let sum: f64 = rs.iter().sum();
let mean = sum / n as f64;
let wins: Vec<f64> = rs.iter().copied().filter(|&r| r > 0.0).collect();
let losses: Vec<f64> = rs.iter().copied().filter(|&r| r <= 0.0).collect();
let sum_win: f64 = wins.iter().sum();
let sum_loss: f64 = losses.iter().sum();
let avg = |v: &[f64]| if v.is_empty() { 0.0 } else { v.iter().sum::<f64>() / v.len() as f64 };
let mut peak = f64::NEG_INFINITY;
let mut cum = 0.0;
let mut max_dd = 0.0_f64;
for &r in rs {
cum += r;
if cum > peak { peak = cum; }
let dd = peak - cum;
if dd > max_dd { max_dd = dd; }
}
let (sqn, sqn_normalized) = if n < 2 {
(0.0, 0.0)
} else {
let var = rs.iter().map(|&r| (r - mean).powi(2)).sum::<f64>() / (n as f64 - 1.0);
let sd = var.sqrt();
if sd > 0.0 {
((n as f64).sqrt() * mean / sd, (n.min(SQN_CAP) as f64).sqrt() * mean / sd)
} else {
(0.0, 0.0)
}
};
RMetrics {
expectancy_r: mean,
n_trades: n,
win_rate: wins.len() as f64 / n as f64,
avg_win_r: avg(&wins),
avg_loss_r: avg(&losses),
profit_factor: if sum_loss < 0.0 { sum_win / (-sum_loss) } else { 0.0 },
max_r_drawdown: max_dd,
n_open_at_end: 0,
sqn,
sqn_normalized,
net_expectancy_r: mean, // cost = 0 -> net == gross (Stage-1 frictionless)
conviction_terciles_r: [0.0; 3],
trade_rs: Vec::new(),
}
}
- Step 4: Run to verify it passes
Run: cargo test --workspace r_metrics_from_rs_folds_a_flat_series
Expected: PASS
Run: cargo test --workspace r_metrics_from_rs_empty_is_all_zero
Expected: PASS
Task 3: walkforward signature plumbing — --strategy + grid flags
This task threads the strategy/grid through parse_walkforward_args →
run_walkforward → walkforward_family (SMA arm verbatim; stage1-r arm a
graceful error stub filled in Task 4) and every call site, so the workspace
compiles and all SMA goldens stay green.
Files:
-
Modify:
crates/aura-cli/src/main.rs:1526-1546, 1607-1666, 2591-2592, 2637-2645 -
Test:
crates/aura-cli/src/main.rs(#cfg test, lines 3651-3660 updated + a new case) -
Step 1: Write/adjust the failing parse test — in the report-less
crates/aura-cli/src/main.rstest module, replace the body ofparse_walkforward_args_defaults_and_accepts_real(3652-3660) to expect the 5-tuple, and add a stage1-r case:
#[test]
fn parse_walkforward_args_defaults_and_accepts_real() {
assert_eq!(
parse_walkforward_args(&[]),
Ok((Strategy::SmaCross, "walkforward".to_string(), false, DataChoice::Synthetic, Stage1RGrid::default()))
);
assert_eq!(
parse_walkforward_args(&["--real", "EURUSD", "--trace", "w"]),
Ok((Strategy::SmaCross, "w".to_string(), true,
DataChoice::Real { symbol: "EURUSD".to_string(), from_ms: None, to_ms: None },
Stage1RGrid::default()))
);
assert!(parse_walkforward_args(&["--name", "a", "--trace", "b"]).is_err());
assert!(parse_walkforward_args(&["--real"]).is_err());
}
#[test]
fn parse_walkforward_args_accepts_strategy_and_grid_flags() {
let r = parse_walkforward_args(&["--strategy", "stage1-r", "--fast", "5,10", "--stop-k", "2.0,3.0"]);
let (strategy, _, _, _, grid) = r.expect("valid stage1-r walkforward args");
assert_eq!(strategy, Strategy::Stage1R);
assert_eq!(grid.fast, vec![5, 10]);
assert_eq!(grid.stop_k, vec![2.0, 3.0]);
assert!(parse_walkforward_args(&["--strategy", "bogus"]).is_err());
}
- Step 2: Run to verify it fails
Run: cargo test --workspace parse_walkforward_args_accepts_strategy_and_grid_flags
Expected: FAIL — compile error (the 5-tuple / Strategy return type does not exist
yet; existing 3-tuple assertions also fail to compile).
- Step 3: Extend
parse_walkforward_args— replace lines 1523-1546 with the 5-tuple grammar mirroringparse_sweep_args(1480-1521):
/// Parse the `walkforward` tail:
/// `[--strategy <sma|momentum|stage1-r|stage1-breakout|stage1-meanrev>] [--real <SYMBOL> [--from <ms>] [--to <ms>]] [--name <n> | --trace <n>] [--fast <csv>] [--slow <csv>] [--stop-length <csv>] [--stop-k <csv>] [--channel <csv>] [--window <csv>] [--band-k <csv>]`.
/// Defaults: SMA-cross, synthetic, name "walkforward", no persist. `--name`/`--trace`
/// mutually exclusive; `--from`/`--to` require `--real`. Pure (no I/O / exit).
fn parse_walkforward_args(
rest: &[&str],
) -> Result<(Strategy, String, bool, DataChoice, Stage1RGrid), String> {
let usage = || "walkforward [--strategy <sma|momentum|stage1-r|stage1-breakout|stage1-meanrev>] [--real <SYMBOL> [--from <ms>] [--to <ms>]] [--name <n> | --trace <n>] [--fast <csv>] [--slow <csv>] [--stop-length <csv>] [--stop-k <csv>] [--channel <csv>] [--window <csv>] [--band-k <csv>]".to_string();
let mut strategy = Strategy::SmaCross;
let mut name: Option<(String, bool)> = None;
let mut real = RealWindowGrammar::default();
let mut grid = Stage1RGrid::default();
let mut tail = rest;
while let Some((flag, t)) = tail.split_first() {
let (value, t) = t.split_first().ok_or_else(usage)?;
if real.accept(flag, value, &usage)? {
tail = t;
continue;
}
match *flag {
"--strategy" => {
strategy = match *value {
"sma" => Strategy::SmaCross,
"momentum" => Strategy::Momentum,
"stage1-r" => Strategy::Stage1R,
"stage1-breakout" => Strategy::Stage1Breakout,
"stage1-meanrev" => Strategy::Stage1MeanRev,
_ => return Err(usage()),
};
}
"--name" if name.is_none() => name = Some(((*value).to_string(), false)),
"--trace" if name.is_none() => name = Some(((*value).to_string(), true)),
"--fast" => grid.fast = parse_csv_list(value).map_err(|()| usage())?,
"--slow" => grid.slow = parse_csv_list(value).map_err(|()| usage())?,
"--stop-length" => grid.stop_length = parse_csv_list(value).map_err(|()| usage())?,
"--stop-k" => grid.stop_k = parse_csv_list(value).map_err(|()| usage())?,
"--channel" => grid.channel = parse_csv_list(value).map_err(|()| usage())?,
"--window" => grid.window = parse_csv_list(value).map_err(|()| usage())?,
"--band-k" => grid.band_k = parse_csv_list(value).map_err(|()| usage())?,
_ => return Err(usage()),
}
tail = t;
}
let (name, persist) = name.unwrap_or_else(|| ("walkforward".to_string(), false));
Ok((strategy, name, persist, real.finish(&usage)?, grid))
}
- Step 4: Thread
run_walkforward+walkforward_family— changerun_walkforward(1615) andwalkforward_family(1643) signatures and add the dispatch (SMA arm is today's body verbatim; stage1-r arm errors for now):
fn run_walkforward(strategy: Strategy, name: &str, persist: bool, data: DataSource, grid: &Stage1RGrid) {
if persist
&& let Err(e) = TraceStore::open("runs").ensure_name_free(name, WriteKind::Family)
{
eprintln!("aura: {e}");
std::process::exit(2);
}
let reg = default_registry();
let result = walkforward_family(strategy, persist.then_some(name), &data, grid);
// ... rest of run_walkforward body unchanged (append_family / print loop / summary) ...
}
fn walkforward_family(
strategy: Strategy, trace: Option<&str>, data: &DataSource, grid: &Stage1RGrid,
) -> WalkForwardResult {
let span = data.wf_full_span();
let (is_len, oos_len, step) = data.wf_window_sizes();
let roller = match WindowRoller::new(span, is_len, oos_len, step, RollMode::Rolling) {
Ok(r) => r,
Err(e) => {
eprintln!("aura: walk-forward window too short for one IS+OOS span: {e:?}");
std::process::exit(2);
}
};
match strategy {
Strategy::SmaCross => {
let space = sample_blueprint_with_sinks(data.pip_size()).0.param_space();
walk_forward(roller, space, |w: WindowBounds| {
let is_family = sweep_over(w.is.0, w.is.1, data);
let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric");
let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1, trace, data);
WindowRun { chosen_params: best.params, oos_equity, oos_report }
})
}
Strategy::Stage1R => {
// filled in Task 4 (stage1_r_sweep_over / run_oos_r / stage1_r_space).
eprintln!("aura: walkforward --strategy stage1-r not yet wired");
std::process::exit(2);
}
_ => {
eprintln!("aura: walkforward has no form for the selected strategy");
std::process::exit(2);
}
}
}
- Step 5: Update the dispatch arm + USAGE
// dispatch arm (main.rs:2637)
["walkforward", rest @ ..] => match parse_walkforward_args(rest) {
Ok((strategy, name, persist, choice, grid)) => {
run_walkforward(strategy, &name, persist, DataSource::from_choice(choice), &grid)
}
Err(msg) => {
eprintln!("aura: {msg}");
std::process::exit(2);
}
},
// USAGE (main.rs:2592) — replace the walkforward fragment with:
// aura walkforward [--strategy <sma|momentum|stage1-r|stage1-breakout|stage1-meanrev>] [--real <SYMBOL> [--from <ms>] [--to <ms>]] [--name <n>|--trace <n>] [--fast <csv>] [--slow <csv>] [--stop-length <csv>] [--stop-k <csv>]
- Step 6: Thread the remaining
walkforward_familytest callers (1775, 3151) so the workspace compiles — both pass the new args:
// main.rs:1775 (walkforward_report helper) and :3151 (member-reports test):
// walkforward_family(Strategy::SmaCross, None, &DataSource::Synthetic, &Stage1RGrid::default())
- Step 7: Build gate
Run: cargo build --workspace
Expected: 0 errors (every parse_walkforward_args / run_walkforward /
walkforward_family caller threaded).
- Step 8: Run parse tests + the SMA walkforward goldens
Run: cargo test --workspace parse_walkforward_args_accepts_strategy_and_grid_flags
Expected: PASS
Run: cargo test --workspace parse_walkforward_args_defaults_and_accepts_real
Expected: PASS
Run: cargo test --workspace walkforward_report_has_one_oos_line_per_window_plus_summary
Expected: PASS (the SMA arm is byte-verbatim)
Task 4: stage1-r walk-forward arm — windowed helpers + dispatch + oos_r
Files:
-
Modify:
crates/aura-cli/src/main.rs— addstage1_r_space,stage1_r_sweep_over,run_oos_r; fill theStrategy::Stage1Rarm inwalkforward_family; extendwalkforward_summary_json. -
Test:
crates/aura-cli/tests/cli_run.rs— synthetic stage1-r walkforward E2E. -
Step 1: Write the failing E2E test — append to
crates/aura-cli/tests/cli_run.rs(model on the existing walkforward + stage1-r shell tests; runs the built binary on synthetic data):
#[test]
fn walkforward_strategy_stage1_r_reports_oos_r() {
// `aura walkforward --strategy stage1-r` on synthetic data: every per-window
// line carries an `r` block, and the summary line carries a pooled `oos_r`
// block. Deterministic: a second run is byte-identical.
let out = run_aura(&["walkforward", "--strategy", "stage1-r"]);
let lines: Vec<&str> = out.trim().lines().collect();
let summary: serde_json::Value = serde_json::from_str(lines.last().unwrap()).unwrap();
assert!(summary["walkforward"]["oos_r"].is_object(), "summary carries oos_r");
assert!(summary["walkforward"]["oos_r"]["n_trades"].is_number());
// each per-window member line carries metrics.r
for l in &lines[..lines.len() - 1] {
let v: serde_json::Value = serde_json::from_str(l).unwrap();
assert!(v["report"]["metrics"]["r"].is_object(), "per-window r block present");
}
let out2 = run_aura(&["walkforward", "--strategy", "stage1-r"]);
assert_eq!(out, out2, "stage1-r walkforward is deterministic");
}
(If run_aura is not the existing harness name in cli_run.rs, use that file's
established binary-invocation helper — the same one
walkforward_real_persists_one_oos_member_per_window uses.)
- Step 2: Run to verify it fails
Run: cargo test --workspace walkforward_strategy_stage1_r_reports_oos_r
Expected: FAIL — the process exits 2 ("not yet wired") so the output has no lines /
the assertion on oos_r fails.
- Step 3: Add
stage1_r_space— afterstage1_r_sweep_family(main.rs:1275):
/// The param-space of the OPEN stage1-r blueprint (all four knobs free) — the kinds
/// the per-window `WindowRun::chosen_params` are read against (C7). Mirrors the
/// throwaway-build param_space resolution inside `stage1_r_sweep_family`.
fn stage1_r_space() -> Vec<ParamSpec> {
let (tx_eq, _) = mpsc::channel();
let (tx_ex, _) = mpsc::channel();
let (tx_r, _) = mpsc::channel();
let (tx_req, _) = mpsc::channel();
stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None, true, true).param_space()
}
- Step 4: Add
stage1_r_sweep_over— the windowed, reduce-mode IS sweep. It isstage1_r_sweep_family(main.rs:1180-1275) with two changes: it always folds (reduce = true, never persists), and it sourcesdata.windowed_sources(from, to)instead ofdata.run_sources():
/// Windowed reduce-mode stage1-r sweep over `[from, to]` — the in-sample leg of the
/// stage1-r walk-forward. Identical grid/axis/fold logic to `stage1_r_sweep_family`,
/// but windowed (`windowed_sources`) and always folded (O(trades)/member): each
/// member's RunReport carries `metrics.r = Some(summarize_r(..))`, so the family is
/// rankable by an R metric.
fn stage1_r_sweep_over(from: Timestamp, to: Timestamp, data: &DataSource, grid: &Stage1RGrid) -> SweepFamily {
let pip = data.pip_size();
let (tx_eq, _) = mpsc::channel();
let (tx_ex, _) = mpsc::channel();
let (tx_r, _) = mpsc::channel();
let (tx_req, _) = mpsc::channel();
let bp = stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None, true, true);
let space = bp.param_space();
let stop_length_axis = space.iter().map(|p| p.name.clone())
.find(|n| n.ends_with(STOP_LENGTH_SUFFIX)).expect("open stage1-r vol-stop exposes a stop_length axis");
let stop_k_axis = space.iter().map(|p| p.name.clone())
.find(|n| n.ends_with(STOP_K_SUFFIX)).expect("open stage1-r vol-stop exposes a stop_k axis");
let binder = bp
.axis("fast.length", grid.fast.clone())
.axis("slow.length", grid.slow.clone())
.axis(&stop_length_axis, grid.stop_length.clone())
.axis(&stop_k_axis, grid.stop_k.clone());
let varying: HashSet<String> = binder.varying_axes().into_iter()
.map(|n| stage1_r_friendly_name(&n)).collect();
binder
.sweep(|point| {
let (tx_eq, rx_eq) = mpsc::channel();
let (tx_ex, rx_ex) = mpsc::channel();
let (tx_r, rx_r) = mpsc::channel();
let (tx_req, _rx_req) = mpsc::channel();
let mut h = stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None, true, true)
.bootstrap_with_cells(point)
.expect("stage1-r grid points are kind-checked against param_space");
let sources = data.windowed_sources(from, to);
let window = window_of(&sources).expect("non-empty in-sample window");
h.run(sources);
let mut named: Vec<(String, Scalar)> = zip_params(&space, point).into_iter()
.map(|(n, v)| (stage1_r_friendly_name(&n), v)).collect();
named.push(("bias_scale".to_string(), Scalar::f64(0.5)));
let key = member_key(&named, &varying);
let _ = &key; // member key not persisted here (in-memory IS family)
let mut manifest = sim_optimal_manifest(named, window, 0, pip);
manifest.broker = stage1_r_broker_label(pip);
let r_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_r.try_iter().collect();
let (total_pips, max_drawdown) = rx_eq.try_iter().next()
.map(|(_, row)| (row[0].as_f64(), row[1].as_f64())).unwrap_or((0.0, 0.0));
let bias_sign_flips = rx_ex.try_iter().next().map(|(_, row)| row[2].as_i64() as u64).unwrap_or(0);
let mut m = RunMetrics { total_pips, max_drawdown, bias_sign_flips, r: None };
m.r = Some(summarize_r(&r_rows, 0.0));
RunReport { manifest, metrics: m }
})
.expect("the stage1-r named grid matches the stage1-r param-space")
}
- Step 5: Add
run_oos_r— the windowed, non-reduce OOS run (the pip-equity curve is needed forstitch; foldssummarize+summarize_r):
/// Run the chosen stage1-r params over an OOS window; return the recorded pip-equity
/// segment (for stitching) and the OOS RunReport whose `metrics.r` carries both the
/// R metrics and the per-trade `trade_rs`. Non-reduce (raw recorders): one window's
/// curve is bounded, and `stitch` needs the full pip-equity series.
fn run_oos_r(
params: &[Cell], from: Timestamp, to: Timestamp, trace: Option<&str>, data: &DataSource,
) -> (Vec<(Timestamp, f64)>, RunReport) {
let pip = data.pip_size();
let (tx_eq, rx_eq) = mpsc::channel();
let (tx_ex, rx_ex) = mpsc::channel();
let (tx_r, rx_r) = mpsc::channel();
let (tx_req, rx_req) = mpsc::channel();
let bp = stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None, true, false);
let space = bp.param_space();
let mut h = bp.bootstrap_with_cells(params)
.expect("chosen params pre-validated by the in-sample GridSpace::new");
let sources = data.windowed_sources(from, to);
let window = window_of(&sources).expect("non-empty out-of-sample window");
h.run(sources);
let eq_rows = rx_eq.try_iter().collect::<Vec<_>>();
let ex_rows = rx_ex.try_iter().collect::<Vec<_>>();
let r_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_r.try_iter().collect();
let req_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_req.try_iter().collect();
let mut named: Vec<(String, Scalar)> = zip_params(&space, params).into_iter()
.map(|(n, v)| (stage1_r_friendly_name(&n), v)).collect();
named.push(("bias_scale".to_string(), Scalar::f64(0.5)));
let mut manifest = sim_optimal_manifest(named, window, 0, pip);
manifest.broker = stage1_r_broker_label(pip);
if let Some(name) = trace {
persist_traces_r(&format!("{name}/oos{}", from.0), &manifest, &eq_rows, &ex_rows, &req_rows);
}
let equity = f64_field(&eq_rows, 0);
let exposure = f64_field(&ex_rows, 0);
let mut metrics = summarize(&equity, &exposure);
metrics.r = Some(summarize_r(&r_rows, 0.0));
(equity, RunReport { manifest, metrics })
}
- Step 6: Fill the
Strategy::Stage1Rarm inwalkforward_family(replace the Task-3 error stub):
Strategy::Stage1R => {
let space = stage1_r_space();
walk_forward(roller, space, |w: WindowBounds| {
let is_family = stage1_r_sweep_over(w.is.0, w.is.1, data, grid);
let best = optimize(&is_family, "sqn_normalized").expect("sqn_normalized is a known metric");
let (oos_equity, oos_report) = run_oos_r(&best.params, w.oos.0, w.oos.1, trace, data);
WindowRun { chosen_params: best.params, oos_equity, oos_report }
})
}
- Step 7: Extend
walkforward_summary_jsonto add the pooledoos_rblock (only when at least one window carries anrblock — the SMA path's windows haver: None, so its summary line stays byte-identical):
fn walkforward_summary_json(result: &WalkForwardResult) -> String {
let total = result.stitched_oos_equity.last().map(|&(_, v)| v).unwrap_or(0.0);
// pool the per-window OOS per-trade R series in roll order, then reduce.
let pooled_rs: Vec<f64> = result.windows.iter()
.flat_map(|w| w.run.oos_report.metrics.r.as_ref().map(|r| r.trade_rs.clone()).unwrap_or_default())
.collect();
let mut obj = serde_json::json!({
"windows": result.windows.len(),
"stitched_total_pips": total,
"param_stability": param_stability(result),
});
if result.windows.iter().any(|w| w.run.oos_report.metrics.r.is_some()) {
// RMetrics serializes its scalar fields (trade_rs is serde-skipped, so the
// oos_r block is the clean R-metric summary of the pooled series).
obj["oos_r"] = serde_json::to_value(r_metrics_from_rs(&pooled_rs))
.expect("RMetrics serializes");
}
serde_json::json!({ "walkforward": obj }).to_string()
}
(Import r_metrics_from_rs from aura_engine alongside the existing summarize_r
/ summarize imports at the top of main.rs.)
- Step 8: Build + run the E2E test + the determinism/golden guards
Run: cargo build --workspace
Expected: 0 errors
Run: cargo test --workspace walkforward_strategy_stage1_r_reports_oos_r
Expected: PASS
Run: cargo test --workspace walkforward_report_has_one_oos_line_per_window_plus_summary
Expected: PASS (SMA summary unchanged — no oos_r key, windows have r: None)
- Step 9: Full-suite regression gate
Run: cargo test --workspace
Expected: PASS — all pre-existing goldens (SMA walkforward, synthetic mc, stage1-r
single-run stage1_r_single_run_output_golden, C18 round-trips) green; the new
tests green.
Run: cargo clippy --workspace --all-targets -- -D warnings
Expected: 0 warnings.
Self-review
- Spec coverage: Changes 1 (RMetrics.trade_rs / summarize_r — Task 1), the pooled reducer (Task 2), Change 3 walkforward dispatch (Tasks 3+4), Change 5 USAGE (Task 3). Monte-Carlo (spec Changes 2+4) is iteration 2 — out of this plan's scope by design. ✓
- Placeholder scan: no TBD/TODO; the Task-3 stage1-r error stub is an explicit, compiling intermediate filled in Task 4 (named, not a placeholder). ✓
- Type/name consistency:
RMetrics.trade_rs,r_metrics_from_rs,stage1_r_sweep_over,run_oos_r,stage1_r_space,Strategy::Stage1R,Stage1RGridused identically across tasks. ✓ - Step granularity: each step is one edit or one command. ✓
- No commit steps. ✓
- Compile-gate ordering (rule 7): Task 1 threads all 4
RMetrics { .. }literal sites before its build gate; Task 3 threads allparse_walkforward_args/walkforward_family/run_walkforwardcallers (dispatch arm + 2 test helpers + parse-test assertions) before its build gate, with the stage1-r arm a compiling error-stub so the gate is satisfiable. ✓ - Filter strings (rule 8): every
cargo testfilter names a real test — pre-existing (runmetrics_with_r_block_round_trips,walkforward_report_has_one_oos_line_per_window_plus_summary,parse_walkforward_args_defaults_and_accepts_real,stage1_r_single_run_output_golden) or introduced in the same task; the final gate is the unfilteredcargo test --workspace. ✓