plan: 0075 walk-forward strategy-selectable + R-reporting (iteration 1)

Iteration 1 of cycle 0075: RMetrics.trade_rs (in-memory, serde+eq-excluded)
+ summarize_r retains it; r_metrics_from_rs pooled reducer; walkforward
--strategy + grid flags; strategy-dispatched walkforward_family (SMA arm
verbatim); windowed stage1-r helpers (stage1_r_sweep_over reduce-mode IS
sweep, run_oos_r non-reduce OOS run, stage1_r_space); pooled oos_r summary.
Monte-Carlo bootstrap is iteration 2.

refs #139
This commit is contained in:
2026-06-26 10:26:39 +02:00
parent 3a1de05ef3
commit 2593af86f8
@@ -0,0 +1,768 @@
# Walk-forward strategy-selectable + R-reporting (iteration 1) — Implementation Plan
> **Parent spec:** `docs/specs/0075-walkforward-mc-r-validation.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
> this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** Make `aura walkforward --strategy stage1-r --real <SYM>` roll IS→OOS
windows, sweep the stage1-r grid in-sample, pick the best by an R metric, run the
chosen params OOS, and report per-window + across-window-pooled OOS R-metrics —
while the bare SMA `walkforward` path stays byte-identical.
**Architecture:** Two additive library changes in `aura-engine` (`RMetrics` carries
an in-memory `trade_rs` vector excluded from serde + equality; a new
`r_metrics_from_rs` pooled reducer), then `aura-cli` gains `--strategy` + grid
flags on `walkforward`, a strategy-dispatched `walkforward_family` (SMA arm
verbatim), windowed stage1-r helpers (`stage1_r_sweep_over` reduce-mode IS sweep,
`run_oos_r` non-reduce OOS run, `stage1_r_space`), and a pooled `oos_r` summary
block. The Monte-Carlo bootstrap half is iteration 2 (separate plan).
**Tech Stack:** Rust workspace — `aura-engine` (`report.rs`), `aura-registry`
(`lib.rs` test helper), `aura-cli` (`main.rs`). Test command:
`cargo test --workspace` (one positional filter per invocation).
---
**Files this plan creates or modifies:**
- Modify: `crates/aura-engine/src/report.rs:44-62``RMetrics` gains
`#[serde(skip)] trade_rs: Vec<f64>`; drop `PartialEq` from the derive, hand-write
it (ignoring `trade_rs`).
- Modify: `crates/aura-engine/src/report.rs:89-219``summarize_r` sets `trade_rs`
in both return arms; add new `pub fn r_metrics_from_rs`.
- Modify: `crates/aura-registry/src/lib.rs:255``report_with_r` test helper
`RMetrics { .. }` literal gains `trade_rs: Vec::new()`.
- Modify: `crates/aura-cli/src/main.rs:1526-1546``parse_walkforward_args` 3-tuple
→ 5-tuple (`--strategy` + four grid flags).
- Modify: `crates/aura-cli/src/main.rs:1615-1666``run_walkforward` +
`walkforward_family` gain `strategy`/`grid`; strategy dispatch.
- Modify: `crates/aura-cli/src/main.rs:1734-1744``walkforward_summary_json` gains
the `oos_r` pooled block.
- Modify: `crates/aura-cli/src/main.rs:2591-2592` — USAGE walkforward fragment.
- Modify: `crates/aura-cli/src/main.rs:2637-2645` — walkforward dispatch arm
destructures the 5-tuple.
- Create (new fns in `crates/aura-cli/src/main.rs`): `stage1_r_sweep_over`,
`run_oos_r`, `stage1_r_space`.
- Test: `crates/aura-engine/src/report.rs` (#cfg test) — `summarize_r` populates
`trade_rs`; `RMetrics` round-trip equal despite skipped `trade_rs`;
`r_metrics_from_rs` goldens.
- Test: `crates/aura-cli/src/main.rs` (#cfg test) — `parse_walkforward_args`
5-tuple + `--strategy`/grid; update the two existing assertions at 3653-3656.
- Test: `crates/aura-cli/tests/cli_run.rs``aura walkforward --strategy stage1-r`
emits per-window `r` blocks + an `oos_r` summary (synthetic).
---
### Task 1: `RMetrics.trade_rs` — in-memory per-trade R series
**Files:**
- Modify: `crates/aura-engine/src/report.rs:44-62, 89-135, 206-219`
- Modify: `crates/aura-registry/src/lib.rs:255`
- Test: `crates/aura-engine/src/report.rs` (#cfg test)
- [ ] **Step 1: Write the failing tests** — append to the `#[cfg(test)] mod tests`
in `crates/aura-engine/src/report.rs`:
```rust
#[test]
fn summarize_r_populates_trade_rs_in_trade_order() {
// two closed trades (R = +2.0, then -1.0) over a minimal PositionManagement
// record; trade_rs must carry [2.0, -1.0] in trade order.
let rec = pm_record_two_closed_trades(); // helper below
let m = summarize_r(&rec, 0.0);
assert_eq!(m.trade_rs, vec![2.0, -1.0]);
assert_eq!(m.n_trades, 2);
}
#[test]
fn summarize_r_empty_record_has_empty_trade_rs() {
let m = summarize_r(&[], 0.0);
assert!(m.trade_rs.is_empty());
assert_eq!(m.n_trades, 0);
}
#[test]
fn rmetrics_partial_eq_ignores_trade_rs() {
// two RMetrics equal in every metric but differing in trade_rs compare EQUAL
// (trade_rs is an in-memory conduit, excluded from equality) — this is what
// keeps serialize->deserialize round-trips equal (trade_rs is serde-skipped,
// so it deserializes empty).
let a = summarize_r(&pm_record_two_closed_trades(), 0.0);
let mut b = a.clone();
b.trade_rs = Vec::new();
assert_eq!(a, b);
}
/// A minimal dense PositionManagement record with two closed trades at R = +2, -1.
/// Columns per `r_col` (CLOSED=0, REALIZED_R=1, DIRECTION=4, ENTRY_PRICE=6,
/// STOP_PRICE=7, CONVICTION_AT_ENTRY=9, SIZE=10, OPEN=11, UNREALIZED_R=12); width 13.
fn pm_record_two_closed_trades() -> Vec<(Timestamp, Vec<Scalar>)> {
let row = |closed: bool, r: f64, open: bool| {
let mut c = vec![Scalar::f64(0.0); 13];
c[0] = Scalar::bool(closed);
c[1] = Scalar::f64(r);
c[6] = Scalar::f64(1.0); // entry
c[7] = Scalar::f64(0.5); // stop -> latched 0.5
c[9] = Scalar::f64(0.3); // conviction
c[11] = Scalar::bool(open);
c
};
vec![
(Timestamp(1), row(true, 2.0, false)),
(Timestamp(2), row(true, -1.0, false)),
]
}
```
- [ ] **Step 2: Run to verify it fails**
Run: `cargo test --workspace summarize_r_populates_trade_rs_in_trade_order`
Expected: FAIL — compile error `no field `trade_rs` on type `RMetrics`` (and the
two siblings fail to compile likewise).
- [ ] **Step 3: Add the field + hand-written `PartialEq`** — in
`crates/aura-engine/src/report.rs`, change the `RMetrics` derive (line 44) to drop
`PartialEq`, add the field after `conviction_terciles_r` (line 61), and add the
`impl PartialEq` directly after the struct:
```rust
// line 44: drop PartialEq from the derive
#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)]
pub struct RMetrics {
// ... all existing fields unchanged ...
pub conviction_terciles_r: [f64; 3],
/// Realised R per closed trade, in trade order — an in-memory conduit for the
/// OOS R-series bootstrap (#139). Excluded from serde (`skip`) so the C18 wire
/// shape is unchanged, and from `PartialEq` (below) so every existing
/// `RMetrics`/`RunReport` equality assertion and round-trip stays green.
#[serde(skip)]
pub trade_rs: Vec<f64>,
}
impl PartialEq for RMetrics {
fn eq(&self, o: &Self) -> bool {
self.expectancy_r == o.expectancy_r
&& self.n_trades == o.n_trades
&& self.win_rate == o.win_rate
&& self.avg_win_r == o.avg_win_r
&& self.avg_loss_r == o.avg_loss_r
&& self.profit_factor == o.profit_factor
&& self.max_r_drawdown == o.max_r_drawdown
&& self.n_open_at_end == o.n_open_at_end
&& self.sqn == o.sqn
&& self.sqn_normalized == o.sqn_normalized
&& self.net_expectancy_r == o.net_expectancy_r
&& self.conviction_terciles_r == o.conviction_terciles_r
// trade_rs deliberately excluded
}
}
```
- [ ] **Step 4: Populate `trade_rs` in both `summarize_r` arms** — in
`crates/aura-engine/src/report.rs`: the empty-input arm (line 122-135 literal) adds
`trade_rs: Vec::new(),`, and the main return literal (line 206-219) adds
`trade_rs: rs,` (the `rs` vector built at line 137 is the per-trade realised R in
trade order; it is last-read by the sqn block before the return, so the move is
valid):
```rust
// empty-input arm (line 122) — add as the last field:
return RMetrics {
// ... existing zero fields ...
conviction_terciles_r: [0.0; 3],
trade_rs: Vec::new(),
};
// main return (line 206) — add as the last field:
RMetrics {
// ... existing fields ...
conviction_terciles_r,
trade_rs: rs,
}
```
- [ ] **Step 5: Thread the other two `RMetrics { .. }` literal sites** (the only
remaining struct literals workspace-wide — both test helpers):
```rust
// crates/aura-engine/src/report.rs — the runmetrics_with_r_block_round_trips test
// literal (~line 1135): add `trade_rs: Vec::new(),` as the last field.
// crates/aura-registry/src/lib.rs:255 — the report_with_r helper literal: add
// `trade_rs: Vec::new(),` as the last field.
```
- [ ] **Step 6: Build gate (all literal sites threaded)**
Run: `cargo build --workspace`
Expected: 0 errors (every `RMetrics { .. }` literal now carries `trade_rs`).
- [ ] **Step 7: Run the new + back-compat tests**
Run: `cargo test --workspace summarize_r_populates_trade_rs_in_trade_order`
Expected: PASS
Run: `cargo test --workspace rmetrics_partial_eq_ignores_trade_rs`
Expected: PASS
Run: `cargo test --workspace runmetrics_with_r_block_round_trips`
Expected: PASS (the serde-skip + hand-written eq keep the round-trip equal)
### Task 2: `r_metrics_from_rs` — pooled reducer over a flat R series
**Files:**
- Modify: `crates/aura-engine/src/report.rs` (new `pub fn` after `summarize_r`)
- Test: `crates/aura-engine/src/report.rs` (#cfg test)
- [ ] **Step 1: Write the failing test** — append to the report.rs test module:
```rust
#[test]
fn r_metrics_from_rs_folds_a_flat_series() {
// pooled across-window R series [2.0, -1.0, 1.0]: expectancy = 2/3, 2 wins of 3,
// profit_factor = (2+1)/1 = 3. At cost 0 (frictionless Stage-1) net == gross.
// conviction terciles are not pooled -> [0,0,0]; n_open_at_end is not a pooled
// concept -> 0.
let m = r_metrics_from_rs(&[2.0, -1.0, 1.0]);
assert_eq!(m.n_trades, 3);
assert!((m.expectancy_r - 2.0 / 3.0).abs() < 1e-12);
assert!((m.win_rate - 2.0 / 3.0).abs() < 1e-12);
assert!((m.profit_factor - 3.0).abs() < 1e-12);
assert_eq!(m.net_expectancy_r, m.expectancy_r);
assert_eq!(m.conviction_terciles_r, [0.0; 3]);
assert_eq!(m.n_open_at_end, 0);
}
#[test]
fn r_metrics_from_rs_empty_is_all_zero() {
let m = r_metrics_from_rs(&[]);
assert_eq!(m.n_trades, 0);
assert_eq!(m.expectancy_r, 0.0);
assert_eq!(m.sqn, 0.0);
}
```
- [ ] **Step 2: Run to verify it fails**
Run: `cargo test --workspace r_metrics_from_rs_folds_a_flat_series`
Expected: FAIL — `cannot find function `r_metrics_from_rs` in this scope`.
- [ ] **Step 3: Add the reducer** — in `crates/aura-engine/src/report.rs`, directly
after `summarize_r` (line 220). The R-distribution arithmetic is copied
**verbatim** from `summarize_r` (lines 137-177) — the byte-pinned float
expressions must not be algebraically refactored (a shared factoring would shift
the golden by ~1 ULP); `summarize_r` itself is left untouched:
```rust
/// Reduce a flat per-trade R series (e.g. the pooled across-window OOS series of a
/// walk-forward) into `RMetrics`. The R-distribution fields use the same arithmetic
/// as [`summarize_r`] (copied verbatim — the pinned floats are not refactored). The
/// fields a flat R series cannot carry are set honestly: `n_open_at_end = 0`,
/// `net_expectancy_r = expectancy_r` (exact under the Stage-1 cost = 0 invariant),
/// `conviction_terciles_r = [0,0,0]` (per-trade conviction is not pooled). Empty
/// input -> a well-defined all-zero `RMetrics`.
pub fn r_metrics_from_rs(rs: &[f64]) -> RMetrics {
let n = rs.len() as u64;
if n == 0 {
return RMetrics {
expectancy_r: 0.0, n_trades: 0, win_rate: 0.0, avg_win_r: 0.0,
avg_loss_r: 0.0, profit_factor: 0.0, max_r_drawdown: 0.0,
n_open_at_end: 0, sqn: 0.0, sqn_normalized: 0.0, net_expectancy_r: 0.0,
conviction_terciles_r: [0.0; 3], trade_rs: Vec::new(),
};
}
let sum: f64 = rs.iter().sum();
let mean = sum / n as f64;
let wins: Vec<f64> = rs.iter().copied().filter(|&r| r > 0.0).collect();
let losses: Vec<f64> = rs.iter().copied().filter(|&r| r <= 0.0).collect();
let sum_win: f64 = wins.iter().sum();
let sum_loss: f64 = losses.iter().sum();
let avg = |v: &[f64]| if v.is_empty() { 0.0 } else { v.iter().sum::<f64>() / v.len() as f64 };
let mut peak = f64::NEG_INFINITY;
let mut cum = 0.0;
let mut max_dd = 0.0_f64;
for &r in rs {
cum += r;
if cum > peak { peak = cum; }
let dd = peak - cum;
if dd > max_dd { max_dd = dd; }
}
let (sqn, sqn_normalized) = if n < 2 {
(0.0, 0.0)
} else {
let var = rs.iter().map(|&r| (r - mean).powi(2)).sum::<f64>() / (n as f64 - 1.0);
let sd = var.sqrt();
if sd > 0.0 {
((n as f64).sqrt() * mean / sd, (n.min(SQN_CAP) as f64).sqrt() * mean / sd)
} else {
(0.0, 0.0)
}
};
RMetrics {
expectancy_r: mean,
n_trades: n,
win_rate: wins.len() as f64 / n as f64,
avg_win_r: avg(&wins),
avg_loss_r: avg(&losses),
profit_factor: if sum_loss < 0.0 { sum_win / (-sum_loss) } else { 0.0 },
max_r_drawdown: max_dd,
n_open_at_end: 0,
sqn,
sqn_normalized,
net_expectancy_r: mean, // cost = 0 -> net == gross (Stage-1 frictionless)
conviction_terciles_r: [0.0; 3],
trade_rs: Vec::new(),
}
}
```
- [ ] **Step 4: Run to verify it passes**
Run: `cargo test --workspace r_metrics_from_rs_folds_a_flat_series`
Expected: PASS
Run: `cargo test --workspace r_metrics_from_rs_empty_is_all_zero`
Expected: PASS
### Task 3: `walkforward` signature plumbing — `--strategy` + grid flags
This task threads the strategy/grid through `parse_walkforward_args` →
`run_walkforward` → `walkforward_family` (SMA arm **verbatim**; stage1-r arm a
graceful error stub filled in Task 4) and every call site, so the workspace
compiles and all SMA goldens stay green.
**Files:**
- Modify: `crates/aura-cli/src/main.rs:1526-1546, 1607-1666, 2591-2592, 2637-2645`
- Test: `crates/aura-cli/src/main.rs` (#cfg test, lines 3651-3660 updated + a new
case)
- [ ] **Step 1: Write/adjust the failing parse test** — in the report-less
`crates/aura-cli/src/main.rs` test module, replace the body of
`parse_walkforward_args_defaults_and_accepts_real` (3652-3660) to expect the
5-tuple, and add a stage1-r case:
```rust
#[test]
fn parse_walkforward_args_defaults_and_accepts_real() {
assert_eq!(
parse_walkforward_args(&[]),
Ok((Strategy::SmaCross, "walkforward".to_string(), false, DataChoice::Synthetic, Stage1RGrid::default()))
);
assert_eq!(
parse_walkforward_args(&["--real", "EURUSD", "--trace", "w"]),
Ok((Strategy::SmaCross, "w".to_string(), true,
DataChoice::Real { symbol: "EURUSD".to_string(), from_ms: None, to_ms: None },
Stage1RGrid::default()))
);
assert!(parse_walkforward_args(&["--name", "a", "--trace", "b"]).is_err());
assert!(parse_walkforward_args(&["--real"]).is_err());
}
#[test]
fn parse_walkforward_args_accepts_strategy_and_grid_flags() {
let r = parse_walkforward_args(&["--strategy", "stage1-r", "--fast", "5,10", "--stop-k", "2.0,3.0"]);
let (strategy, _, _, _, grid) = r.expect("valid stage1-r walkforward args");
assert_eq!(strategy, Strategy::Stage1R);
assert_eq!(grid.fast, vec![5, 10]);
assert_eq!(grid.stop_k, vec![2.0, 3.0]);
assert!(parse_walkforward_args(&["--strategy", "bogus"]).is_err());
}
```
- [ ] **Step 2: Run to verify it fails**
Run: `cargo test --workspace parse_walkforward_args_accepts_strategy_and_grid_flags`
Expected: FAIL — compile error (the 5-tuple / `Strategy` return type does not exist
yet; existing 3-tuple assertions also fail to compile).
- [ ] **Step 3: Extend `parse_walkforward_args`** — replace lines 1523-1546 with the
5-tuple grammar mirroring `parse_sweep_args` (1480-1521):
```rust
/// Parse the `walkforward` tail:
/// `[--strategy <sma|momentum|stage1-r|stage1-breakout|stage1-meanrev>] [--real <SYMBOL> [--from <ms>] [--to <ms>]] [--name <n> | --trace <n>] [--fast <csv>] [--slow <csv>] [--stop-length <csv>] [--stop-k <csv>] [--channel <csv>] [--window <csv>] [--band-k <csv>]`.
/// Defaults: SMA-cross, synthetic, name "walkforward", no persist. `--name`/`--trace`
/// mutually exclusive; `--from`/`--to` require `--real`. Pure (no I/O / exit).
fn parse_walkforward_args(
rest: &[&str],
) -> Result<(Strategy, String, bool, DataChoice, Stage1RGrid), String> {
let usage = || "walkforward [--strategy <sma|momentum|stage1-r|stage1-breakout|stage1-meanrev>] [--real <SYMBOL> [--from <ms>] [--to <ms>]] [--name <n> | --trace <n>] [--fast <csv>] [--slow <csv>] [--stop-length <csv>] [--stop-k <csv>] [--channel <csv>] [--window <csv>] [--band-k <csv>]".to_string();
let mut strategy = Strategy::SmaCross;
let mut name: Option<(String, bool)> = None;
let mut real = RealWindowGrammar::default();
let mut grid = Stage1RGrid::default();
let mut tail = rest;
while let Some((flag, t)) = tail.split_first() {
let (value, t) = t.split_first().ok_or_else(usage)?;
if real.accept(flag, value, &usage)? {
tail = t;
continue;
}
match *flag {
"--strategy" => {
strategy = match *value {
"sma" => Strategy::SmaCross,
"momentum" => Strategy::Momentum,
"stage1-r" => Strategy::Stage1R,
"stage1-breakout" => Strategy::Stage1Breakout,
"stage1-meanrev" => Strategy::Stage1MeanRev,
_ => return Err(usage()),
};
}
"--name" if name.is_none() => name = Some(((*value).to_string(), false)),
"--trace" if name.is_none() => name = Some(((*value).to_string(), true)),
"--fast" => grid.fast = parse_csv_list(value).map_err(|()| usage())?,
"--slow" => grid.slow = parse_csv_list(value).map_err(|()| usage())?,
"--stop-length" => grid.stop_length = parse_csv_list(value).map_err(|()| usage())?,
"--stop-k" => grid.stop_k = parse_csv_list(value).map_err(|()| usage())?,
"--channel" => grid.channel = parse_csv_list(value).map_err(|()| usage())?,
"--window" => grid.window = parse_csv_list(value).map_err(|()| usage())?,
"--band-k" => grid.band_k = parse_csv_list(value).map_err(|()| usage())?,
_ => return Err(usage()),
}
tail = t;
}
let (name, persist) = name.unwrap_or_else(|| ("walkforward".to_string(), false));
Ok((strategy, name, persist, real.finish(&usage)?, grid))
}
```
- [ ] **Step 4: Thread `run_walkforward` + `walkforward_family`** — change
`run_walkforward` (1615) and `walkforward_family` (1643) signatures and add the
dispatch (SMA arm is today's body verbatim; stage1-r arm errors for now):
```rust
fn run_walkforward(strategy: Strategy, name: &str, persist: bool, data: DataSource, grid: &Stage1RGrid) {
if persist
&& let Err(e) = TraceStore::open("runs").ensure_name_free(name, WriteKind::Family)
{
eprintln!("aura: {e}");
std::process::exit(2);
}
let reg = default_registry();
let result = walkforward_family(strategy, persist.then_some(name), &data, grid);
// ... rest of run_walkforward body unchanged (append_family / print loop / summary) ...
}
fn walkforward_family(
strategy: Strategy, trace: Option<&str>, data: &DataSource, grid: &Stage1RGrid,
) -> WalkForwardResult {
let span = data.wf_full_span();
let (is_len, oos_len, step) = data.wf_window_sizes();
let roller = match WindowRoller::new(span, is_len, oos_len, step, RollMode::Rolling) {
Ok(r) => r,
Err(e) => {
eprintln!("aura: walk-forward window too short for one IS+OOS span: {e:?}");
std::process::exit(2);
}
};
match strategy {
Strategy::SmaCross => {
let space = sample_blueprint_with_sinks(data.pip_size()).0.param_space();
walk_forward(roller, space, |w: WindowBounds| {
let is_family = sweep_over(w.is.0, w.is.1, data);
let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric");
let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1, trace, data);
WindowRun { chosen_params: best.params, oos_equity, oos_report }
})
}
Strategy::Stage1R => {
// filled in Task 4 (stage1_r_sweep_over / run_oos_r / stage1_r_space).
eprintln!("aura: walkforward --strategy stage1-r not yet wired");
std::process::exit(2);
}
_ => {
eprintln!("aura: walkforward has no form for the selected strategy");
std::process::exit(2);
}
}
}
```
- [ ] **Step 5: Update the dispatch arm + USAGE**
```rust
// dispatch arm (main.rs:2637)
["walkforward", rest @ ..] => match parse_walkforward_args(rest) {
Ok((strategy, name, persist, choice, grid)) => {
run_walkforward(strategy, &name, persist, DataSource::from_choice(choice), &grid)
}
Err(msg) => {
eprintln!("aura: {msg}");
std::process::exit(2);
}
},
// USAGE (main.rs:2592) — replace the walkforward fragment with:
// aura walkforward [--strategy <sma|momentum|stage1-r|stage1-breakout|stage1-meanrev>] [--real <SYMBOL> [--from <ms>] [--to <ms>]] [--name <n>|--trace <n>] [--fast <csv>] [--slow <csv>] [--stop-length <csv>] [--stop-k <csv>]
```
- [ ] **Step 6: Thread the remaining `walkforward_family` test callers** (1775,
3151) so the workspace compiles — both pass the new args:
```rust
// main.rs:1775 (walkforward_report helper) and :3151 (member-reports test):
// walkforward_family(Strategy::SmaCross, None, &DataSource::Synthetic, &Stage1RGrid::default())
```
- [ ] **Step 7: Build gate**
Run: `cargo build --workspace`
Expected: 0 errors (every `parse_walkforward_args` / `run_walkforward` /
`walkforward_family` caller threaded).
- [ ] **Step 8: Run parse tests + the SMA walkforward goldens**
Run: `cargo test --workspace parse_walkforward_args_accepts_strategy_and_grid_flags`
Expected: PASS
Run: `cargo test --workspace parse_walkforward_args_defaults_and_accepts_real`
Expected: PASS
Run: `cargo test --workspace walkforward_report_has_one_oos_line_per_window_plus_summary`
Expected: PASS (the SMA arm is byte-verbatim)
### Task 4: stage1-r walk-forward arm — windowed helpers + dispatch + `oos_r`
**Files:**
- Modify: `crates/aura-cli/src/main.rs` — add `stage1_r_space`,
`stage1_r_sweep_over`, `run_oos_r`; fill the `Strategy::Stage1R` arm in
`walkforward_family`; extend `walkforward_summary_json`.
- Test: `crates/aura-cli/tests/cli_run.rs` — synthetic stage1-r walkforward E2E.
- [ ] **Step 1: Write the failing E2E test** — append to
`crates/aura-cli/tests/cli_run.rs` (model on the existing walkforward + stage1-r
shell tests; runs the built binary on synthetic data):
```rust
#[test]
fn walkforward_strategy_stage1_r_reports_oos_r() {
// `aura walkforward --strategy stage1-r` on synthetic data: every per-window
// line carries an `r` block, and the summary line carries a pooled `oos_r`
// block. Deterministic: a second run is byte-identical.
let out = run_aura(&["walkforward", "--strategy", "stage1-r"]);
let lines: Vec<&str> = out.trim().lines().collect();
let summary: serde_json::Value = serde_json::from_str(lines.last().unwrap()).unwrap();
assert!(summary["walkforward"]["oos_r"].is_object(), "summary carries oos_r");
assert!(summary["walkforward"]["oos_r"]["n_trades"].is_number());
// each per-window member line carries metrics.r
for l in &lines[..lines.len() - 1] {
let v: serde_json::Value = serde_json::from_str(l).unwrap();
assert!(v["report"]["metrics"]["r"].is_object(), "per-window r block present");
}
let out2 = run_aura(&["walkforward", "--strategy", "stage1-r"]);
assert_eq!(out, out2, "stage1-r walkforward is deterministic");
}
```
(If `run_aura` is not the existing harness name in `cli_run.rs`, use that file's
established binary-invocation helper — the same one
`walkforward_real_persists_one_oos_member_per_window` uses.)
- [ ] **Step 2: Run to verify it fails**
Run: `cargo test --workspace walkforward_strategy_stage1_r_reports_oos_r`
Expected: FAIL — the process exits 2 ("not yet wired") so the output has no lines /
the assertion on `oos_r` fails.
- [ ] **Step 3: Add `stage1_r_space`** — after `stage1_r_sweep_family` (main.rs:1275):
```rust
/// The param-space of the OPEN stage1-r blueprint (all four knobs free) — the kinds
/// the per-window `WindowRun::chosen_params` are read against (C7). Mirrors the
/// throwaway-build param_space resolution inside `stage1_r_sweep_family`.
fn stage1_r_space() -> Vec<ParamSpec> {
let (tx_eq, _) = mpsc::channel();
let (tx_ex, _) = mpsc::channel();
let (tx_r, _) = mpsc::channel();
let (tx_req, _) = mpsc::channel();
stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None, true, true).param_space()
}
```
- [ ] **Step 4: Add `stage1_r_sweep_over`** — the windowed, reduce-mode IS sweep. It
is `stage1_r_sweep_family` (main.rs:1180-1275) with two changes: it always folds
(reduce = true, never persists), and it sources `data.windowed_sources(from, to)`
instead of `data.run_sources()`:
```rust
/// Windowed reduce-mode stage1-r sweep over `[from, to]` — the in-sample leg of the
/// stage1-r walk-forward. Identical grid/axis/fold logic to `stage1_r_sweep_family`,
/// but windowed (`windowed_sources`) and always folded (O(trades)/member): each
/// member's RunReport carries `metrics.r = Some(summarize_r(..))`, so the family is
/// rankable by an R metric.
fn stage1_r_sweep_over(from: Timestamp, to: Timestamp, data: &DataSource, grid: &Stage1RGrid) -> SweepFamily {
let pip = data.pip_size();
let (tx_eq, _) = mpsc::channel();
let (tx_ex, _) = mpsc::channel();
let (tx_r, _) = mpsc::channel();
let (tx_req, _) = mpsc::channel();
let bp = stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None, true, true);
let space = bp.param_space();
let stop_length_axis = space.iter().map(|p| p.name.clone())
.find(|n| n.ends_with(STOP_LENGTH_SUFFIX)).expect("open stage1-r vol-stop exposes a stop_length axis");
let stop_k_axis = space.iter().map(|p| p.name.clone())
.find(|n| n.ends_with(STOP_K_SUFFIX)).expect("open stage1-r vol-stop exposes a stop_k axis");
let binder = bp
.axis("fast.length", grid.fast.clone())
.axis("slow.length", grid.slow.clone())
.axis(&stop_length_axis, grid.stop_length.clone())
.axis(&stop_k_axis, grid.stop_k.clone());
let varying: HashSet<String> = binder.varying_axes().into_iter()
.map(|n| stage1_r_friendly_name(&n)).collect();
binder
.sweep(|point| {
let (tx_eq, rx_eq) = mpsc::channel();
let (tx_ex, rx_ex) = mpsc::channel();
let (tx_r, rx_r) = mpsc::channel();
let (tx_req, _rx_req) = mpsc::channel();
let mut h = stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None, true, true)
.bootstrap_with_cells(point)
.expect("stage1-r grid points are kind-checked against param_space");
let sources = data.windowed_sources(from, to);
let window = window_of(&sources).expect("non-empty in-sample window");
h.run(sources);
let mut named: Vec<(String, Scalar)> = zip_params(&space, point).into_iter()
.map(|(n, v)| (stage1_r_friendly_name(&n), v)).collect();
named.push(("bias_scale".to_string(), Scalar::f64(0.5)));
let key = member_key(&named, &varying);
let _ = &key; // member key not persisted here (in-memory IS family)
let mut manifest = sim_optimal_manifest(named, window, 0, pip);
manifest.broker = stage1_r_broker_label(pip);
let r_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_r.try_iter().collect();
let (total_pips, max_drawdown) = rx_eq.try_iter().next()
.map(|(_, row)| (row[0].as_f64(), row[1].as_f64())).unwrap_or((0.0, 0.0));
let bias_sign_flips = rx_ex.try_iter().next().map(|(_, row)| row[2].as_i64() as u64).unwrap_or(0);
let mut m = RunMetrics { total_pips, max_drawdown, bias_sign_flips, r: None };
m.r = Some(summarize_r(&r_rows, 0.0));
RunReport { manifest, metrics: m }
})
.expect("the stage1-r named grid matches the stage1-r param-space")
}
```
- [ ] **Step 5: Add `run_oos_r`** — the windowed, non-reduce OOS run (the pip-equity
curve is needed for `stitch`; folds `summarize` + `summarize_r`):
```rust
/// Run the chosen stage1-r params over an OOS window; return the recorded pip-equity
/// segment (for stitching) and the OOS RunReport whose `metrics.r` carries both the
/// R metrics and the per-trade `trade_rs`. Non-reduce (raw recorders): one window's
/// curve is bounded, and `stitch` needs the full pip-equity series.
fn run_oos_r(
params: &[Cell], from: Timestamp, to: Timestamp, trace: Option<&str>, data: &DataSource,
) -> (Vec<(Timestamp, f64)>, RunReport) {
let pip = data.pip_size();
let (tx_eq, rx_eq) = mpsc::channel();
let (tx_ex, rx_ex) = mpsc::channel();
let (tx_r, rx_r) = mpsc::channel();
let (tx_req, rx_req) = mpsc::channel();
let bp = stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None, true, false);
let space = bp.param_space();
let mut h = bp.bootstrap_with_cells(params)
.expect("chosen params pre-validated by the in-sample GridSpace::new");
let sources = data.windowed_sources(from, to);
let window = window_of(&sources).expect("non-empty out-of-sample window");
h.run(sources);
let eq_rows = rx_eq.try_iter().collect::<Vec<_>>();
let ex_rows = rx_ex.try_iter().collect::<Vec<_>>();
let r_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_r.try_iter().collect();
let req_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_req.try_iter().collect();
let mut named: Vec<(String, Scalar)> = zip_params(&space, params).into_iter()
.map(|(n, v)| (stage1_r_friendly_name(&n), v)).collect();
named.push(("bias_scale".to_string(), Scalar::f64(0.5)));
let mut manifest = sim_optimal_manifest(named, window, 0, pip);
manifest.broker = stage1_r_broker_label(pip);
if let Some(name) = trace {
persist_traces_r(&format!("{name}/oos{}", from.0), &manifest, &eq_rows, &ex_rows, &req_rows);
}
let equity = f64_field(&eq_rows, 0);
let exposure = f64_field(&ex_rows, 0);
let mut metrics = summarize(&equity, &exposure);
metrics.r = Some(summarize_r(&r_rows, 0.0));
(equity, RunReport { manifest, metrics })
}
```
- [ ] **Step 6: Fill the `Strategy::Stage1R` arm** in `walkforward_family` (replace
the Task-3 error stub):
```rust
Strategy::Stage1R => {
let space = stage1_r_space();
walk_forward(roller, space, |w: WindowBounds| {
let is_family = stage1_r_sweep_over(w.is.0, w.is.1, data, grid);
let best = optimize(&is_family, "sqn_normalized").expect("sqn_normalized is a known metric");
let (oos_equity, oos_report) = run_oos_r(&best.params, w.oos.0, w.oos.1, trace, data);
WindowRun { chosen_params: best.params, oos_equity, oos_report }
})
}
```
- [ ] **Step 7: Extend `walkforward_summary_json`** to add the pooled `oos_r` block
(only when at least one window carries an `r` block — the SMA path's windows have
`r: None`, so its summary line stays byte-identical):
```rust
fn walkforward_summary_json(result: &WalkForwardResult) -> String {
let total = result.stitched_oos_equity.last().map(|&(_, v)| v).unwrap_or(0.0);
// pool the per-window OOS per-trade R series in roll order, then reduce.
let pooled_rs: Vec<f64> = result.windows.iter()
.flat_map(|w| w.run.oos_report.metrics.r.as_ref().map(|r| r.trade_rs.clone()).unwrap_or_default())
.collect();
let mut obj = serde_json::json!({
"windows": result.windows.len(),
"stitched_total_pips": total,
"param_stability": param_stability(result),
});
if result.windows.iter().any(|w| w.run.oos_report.metrics.r.is_some()) {
// RMetrics serializes its scalar fields (trade_rs is serde-skipped, so the
// oos_r block is the clean R-metric summary of the pooled series).
obj["oos_r"] = serde_json::to_value(r_metrics_from_rs(&pooled_rs))
.expect("RMetrics serializes");
}
serde_json::json!({ "walkforward": obj }).to_string()
}
```
(Import `r_metrics_from_rs` from `aura_engine` alongside the existing `summarize_r`
/ `summarize` imports at the top of `main.rs`.)
- [ ] **Step 8: Build + run the E2E test + the determinism/golden guards**
Run: `cargo build --workspace`
Expected: 0 errors
Run: `cargo test --workspace walkforward_strategy_stage1_r_reports_oos_r`
Expected: PASS
Run: `cargo test --workspace walkforward_report_has_one_oos_line_per_window_plus_summary`
Expected: PASS (SMA summary unchanged — no `oos_r` key, windows have `r: None`)
- [ ] **Step 9: Full-suite regression gate**
Run: `cargo test --workspace`
Expected: PASS — all pre-existing goldens (SMA walkforward, synthetic mc, stage1-r
single-run `stage1_r_single_run_output_golden`, C18 round-trips) green; the new
tests green.
Run: `cargo clippy --workspace --all-targets -- -D warnings`
Expected: 0 warnings.
---
## Self-review
- **Spec coverage:** Changes 1 (RMetrics.trade_rs / summarize_r — Task 1), the
pooled reducer (Task 2), Change 3 walkforward dispatch (Tasks 3+4), Change 5 USAGE
(Task 3). Monte-Carlo (spec Changes 2+4) is iteration 2 — out of this plan's scope
by design. ✓
- **Placeholder scan:** no TBD/TODO; the Task-3 stage1-r error stub is an explicit,
compiling intermediate filled in Task 4 (named, not a placeholder). ✓
- **Type/name consistency:** `RMetrics.trade_rs`, `r_metrics_from_rs`,
`stage1_r_sweep_over`, `run_oos_r`, `stage1_r_space`, `Strategy::Stage1R`,
`Stage1RGrid` used identically across tasks. ✓
- **Step granularity:** each step is one edit or one command. ✓
- **No commit steps.** ✓
- **Compile-gate ordering (rule 7):** Task 1 threads all 4 `RMetrics { .. }` literal
sites before its build gate; Task 3 threads all `parse_walkforward_args` /
`walkforward_family` / `run_walkforward` callers (dispatch arm + 2 test helpers +
parse-test assertions) before its build gate, with the stage1-r arm a compiling
error-stub so the gate is satisfiable. ✓
- **Filter strings (rule 8):** every `cargo test` filter names a real test —
pre-existing (`runmetrics_with_r_block_round_trips`,
`walkforward_report_has_one_oos_line_per_window_plus_summary`,
`parse_walkforward_args_defaults_and_accepts_real`,
`stage1_r_single_run_output_golden`) or introduced in the same task; the final
gate is the unfiltered `cargo test --workspace`. ✓