diff --git a/docs/plans/walkforward-dissolution.md b/docs/plans/walkforward-dissolution.md new file mode 100644 index 0000000..0e1e5a9 --- /dev/null +++ b/docs/plans/walkforward-dissolution.md @@ -0,0 +1,808 @@ +# Walkforward Dissolution — Implementation Plan + +> **Parent spec:** `docs/specs/walkforward-dissolution.md` +> +> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run +> this plan. Steps use `- [ ]` checkboxes for tracking. + +**Goal:** Route `aura walkforward --strategy r-sma --real` through the one campaign +executor as thin sugar (a generated, content-addressed campaign document), +reproducing the `{"walkforward":{…}}` summary line byte-for-byte, while retaining +the inline `run_walkforward` for the fenced synthetic / non-r-sma branches. + +**Architecture:** Pure sugar — NO executor change. `dispatch_walkforward`'s +built-in arm splits on `--strategy r-sma && --real`; that branch synthesizes a bare +`sma_signal` blueprint (exactly as generalize), builds a `[std::sweep(argmax), +std::walk_forward]` campaign (multi-value fast/slow axes, one Vol risk regime +carrying the stop), runs it via `run_campaign_returning`, and reprints the summary +from the recorded `WalkForward` `StageFamily.reports`. The summary reducer +reconstructs `stitched_total_pips = Σ per-window total_pips`, pooled `oos_r` from +the reports' in-memory `trade_rs`, and `param_stability` via `r_sma_space` + +`MetricStats::from_values` — all byte-verified against the committed anchor. + +**Tech Stack:** `crates/aura-cli/src/verb_sugar.rs` (translator + sugar runner), +`crates/aura-cli/src/main.rs` (args parser, reducer, dispatch split), +`crates/aura-cli/tests/cli_run.rs` (front-end + dissolution-proof tests). The +committed anchor `walkforward_real_e2e_pins_the_exact_current_grade` is the gate. + +**Files this plan creates or modifies:** + +- Modify: `crates/aura-cli/src/verb_sugar.rs` — `GeneratedWalkforward`, + `translate_walkforward`, `register_generated_wf`, `run_walkforward_sugar`; add + `WfMode` to the `aura_research` import. +- Modify: `crates/aura-cli/src/main.rs` — `walkforward_summary_json_from_reports` + reducer, `walkforward_args_from`, `wf_ms_sizes`, `select_is_plateau`, the + `dispatch_walkforward` built-in-arm split (retain the existing body). +- Test: `crates/aura-cli/src/verb_sugar.rs` (mod tests) — translator determinism, + pipeline shape, regime, diverging-stop content ids, registry resolution. +- Test: `crates/aura-cli/src/main.rs` (mod tests) — reducer characterization. +- Test: `crates/aura-cli/tests/cli_run.rs` — front-end refusals + dissolution-proof + e2e; the committed anchor stays green. + +--- + +## Task 1: `translate_walkforward` + `register_generated_wf` (verb_sugar.rs) + +**Files:** +- Modify: `crates/aura-cli/src/verb_sugar.rs:13-17` (import), after `:352` + (`run_generalize_sugar` end) +- Test: `crates/aura-cli/src/verb_sugar.rs` (mod tests, after `:528`) + +- [ ] **Step 1: Add `WfMode` to the `aura_research` import** + +In the `use aura_research::{…}` block (lines 13-17), add `WfMode` (alphabetically +near `Window`). The others (`StageBlock`, `SelectRule`, `RiskRegime`, +`SweepSelection`, `Axis`, `CampaignDoc`, `ProcessDoc`, `DataSection`, `Window`, +`StrategyEntry`, `DocRef`, `ProcessRef`, `Presentation`, `DocKind`, +`FORMAT_VERSION`, `content_id_of`, `process_to_json`) are already present. + +Replace `RiskRegime, SelectRule, StageBlock,` (line 15) with +`RiskRegime, SelectRule, StageBlock, WfMode,` — verify `WfMode` is a real +re-export first: + +Run: `cargo build -p aura-cli 2>&1 | tail -3` +Expected: clean (or an unused-import warning on `WfMode`, resolved by Step 3). + +- [ ] **Step 2: Add `GeneratedWalkforward` + `translate_walkforward` + `register_generated_wf`** + +Append after `run_generalize_sugar` (line 352), mirroring +`GeneratedGeneralize`/`translate_generalize`/`register_generated_g`: + +```rust +/// The two generated documents of one dissolved `walkforward` invocation. +#[derive(Debug)] +pub(crate) struct GeneratedWalkforward { + pub process: ProcessDoc, + pub campaign: CampaignDoc, +} + +/// Translate one `aura walkforward --strategy r-sma --real` invocation into its two +/// generated documents: a `[std::sweep(argmax), std::walk_forward]` process (the +/// sweep only enumerates the IS-refit survivor grid for the wf stage; its recorded +/// argmax selection is not part of the summary) and a campaign running the fast/slow +/// grid over one instrument under a single risk regime that carries the stop. The +/// roller sizes (`in_sample_ms`/`out_of_sample_ms`/`step_ms`) come from the caller +/// (ms). `blueprint_canonical` is the bare `sma_signal` blueprint already stored by +/// topology hash; its content id is the strategy ref. +#[allow(clippy::too_many_arguments)] +pub(crate) fn translate_walkforward( + fast: &[i64], + slow: &[i64], + stop_length: i64, + stop_k: f64, + metric: &str, + in_sample_ms: u64, + out_of_sample_ms: u64, + step_ms: u64, + name: &str, + symbol: &str, + from_ms: i64, + to_ms: i64, + blueprint_canonical: &str, +) -> Result { + let process = ProcessDoc { + format_version: FORMAT_VERSION, + kind: DocKind::Process, + name: "walkforward".to_string(), + description: None, + pipeline: vec![ + StageBlock::Sweep { + selection: Some(SweepSelection { + metric: metric.to_string(), + select: SelectRule::Argmax, + deflate: false, + }), + }, + StageBlock::WalkForward { + in_sample_ms, + out_of_sample_ms, + step_ms, + mode: WfMode::Rolling, + metric: metric.to_string(), + select: SelectRule::Argmax, + }, + ], + }; + let mut doc_axes: BTreeMap = BTreeMap::new(); + doc_axes.insert( + "fast.length".to_string(), + axis_from_values("fast.length", &fast.iter().map(|&v| Scalar::i64(v)).collect::>())?, + ); + doc_axes.insert( + "slow.length".to_string(), + axis_from_values("slow.length", &slow.iter().map(|&v| Scalar::i64(v)).collect::>())?, + ); + let campaign = CampaignDoc { + format_version: FORMAT_VERSION, + kind: DocKind::Campaign, + name: name.to_string(), + description: None, + data: DataSection { + instruments: vec![symbol.to_string()], + windows: vec![Window { from_ms, to_ms }], + }, + strategies: vec![StrategyEntry { + r#ref: DocRef::ContentId(content_id_of(blueprint_canonical)), + axes: doc_axes, + }], + process: ProcessRef { r#ref: DocRef::ContentId(content_id_of(&process_to_json(&process))) }, + risk: vec![RiskRegime::Vol { length: stop_length, k: stop_k }], + seed: 0, + presentation: Presentation { + persist_taps: vec![], + emit: vec!["family_table".to_string()], + }, + }; + Ok(GeneratedWalkforward { process, campaign }) +} + +/// Auto-register both generated walkforward documents (content-addressed, +/// idempotent). Returns `(process_id, campaign_id)`. +pub(crate) fn register_generated_wf( + reg: &aura_registry::Registry, + generated: &GeneratedWalkforward, +) -> Result<(String, String), String> { + let process_id = + reg.put_process(&process_to_json(&generated.process)).map_err(|e| e.to_string())?; + let campaign_id = + reg.put_campaign(&campaign_to_json(&generated.campaign)).map_err(|e| e.to_string())?; + Ok((process_id, campaign_id)) +} +``` + +Note: `campaign_to_json` is already imported (line 14, used by +`register_generated_g`). + +- [ ] **Step 3: Build to confirm it compiles (dead-code allowed until Task 3 wires it)** + +`translate_walkforward`/`register_generated_wf`/`GeneratedWalkforward` have no +caller yet. To keep `clippy -D warnings` green this task, add +`#[allow(dead_code)]` directly above `pub(crate) struct GeneratedWalkforward`, +above `pub(crate) fn translate_walkforward`, and above +`pub(crate) fn register_generated_wf`. Task 3 removes all three once the dispatch +wires them (mirrors the generalize cycle's dead-code bridge). + +Run: `cargo build -p aura-cli 2>&1 | tail -3` +Expected: clean build (no errors). + +- [ ] **Step 4: Add translator unit tests** + +Append inside the `mod tests` block (after line 528, before the closing `}` at +529), mirroring `translate_generalize_is_deterministic_and_carries_the_regime` +and `translate_generalize_diverging_regimes_do_not_collide_content_ids`: + +```rust + #[test] + fn translate_walkforward_is_deterministic_and_carries_the_regime() { + let a = translate_walkforward( + &[3, 5], &[12, 20], 14, 2.0, "sqn_normalized", + 7_776_000_000, 2_592_000_000, 2_592_000_000, + "walkforward", "GER40", 100, 200, "{\"bp\":1}", + ) + .unwrap(); + let b = translate_walkforward( + &[3, 5], &[12, 20], 14, 2.0, "sqn_normalized", + 7_776_000_000, 2_592_000_000, 2_592_000_000, + "walkforward", "GER40", 100, 200, "{\"bp\":1}", + ) + .unwrap(); + assert_eq!( + content_id_of(&campaign_to_json(&a.campaign)), + content_id_of(&campaign_to_json(&b.campaign)), + "identical invocations must generate identical campaign ids" + ); + // pipeline: sweep(argmax) then walk_forward(argmax, rolling) + assert_eq!( + a.process.pipeline, + vec![ + StageBlock::Sweep { + selection: Some(SweepSelection { + metric: "sqn_normalized".to_string(), + select: SelectRule::Argmax, + deflate: false, + }) + }, + StageBlock::WalkForward { + in_sample_ms: 7_776_000_000, + out_of_sample_ms: 2_592_000_000, + step_ms: 2_592_000_000, + mode: WfMode::Rolling, + metric: "sqn_normalized".to_string(), + select: SelectRule::Argmax, + }, + ] + ); + assert_eq!(a.campaign.risk, vec![RiskRegime::Vol { length: 14, k: 2.0 }]); + assert_eq!(a.campaign.data.instruments, vec!["GER40".to_string()]); + assert_eq!(a.campaign.strategies.len(), 1); + // multi-value IS-refit grid axes + assert_eq!( + a.campaign.strategies[0].axes["fast.length"].values, + vec![Scalar::i64(3), Scalar::i64(5)] + ); + assert_eq!( + a.campaign.strategies[0].axes["slow.length"].values, + vec![Scalar::i64(12), Scalar::i64(20)] + ); + assert_eq!(a.process.name, "walkforward"); + assert!(aura_research::validate_campaign(&a.campaign).is_empty()); + assert!(aura_research::validate_process(&a.process).is_empty()); + assert!(aura_campaign::preflight(&a.process, &a.campaign).is_ok()); + } + + #[test] + fn translate_walkforward_diverging_regimes_do_not_collide_content_ids() { + let base = translate_walkforward( + &[3, 5], &[12, 20], 14, 2.0, "sqn_normalized", + 7_776_000_000, 2_592_000_000, 2_592_000_000, + "walkforward", "GER40", 100, 200, "{\"bp\":1}", + ) + .unwrap(); + let different_k = translate_walkforward( + &[3, 5], &[12, 20], 14, 3.0, "sqn_normalized", + 7_776_000_000, 2_592_000_000, 2_592_000_000, + "walkforward", "GER40", 100, 200, "{\"bp\":1}", + ) + .unwrap(); + assert_ne!( + content_id_of(&campaign_to_json(&base.campaign)), + content_id_of(&campaign_to_json(&different_k.campaign)), + "a different stop_k must not collide onto the same campaign id" + ); + } + + #[test] + fn register_generated_wf_stores_documents_the_campaign_can_actually_resolve() { + let dir = std::env::temp_dir().join(format!( + "aura-verb-sugar-walkforward-registry-test-{}", + std::process::id() + )); + std::fs::create_dir_all(&dir).unwrap(); + let reg = aura_registry::Registry::open(dir.join("runs.jsonl")); + let generated = translate_walkforward( + &[3, 5], &[12, 20], 14, 2.0, "sqn_normalized", + 7_776_000_000, 2_592_000_000, 2_592_000_000, + "walkforward", "GER40", 100, 200, "{\"bp\":1}", + ) + .unwrap(); + let (process_id, campaign_id) = register_generated_wf(®, &generated).unwrap(); + let stored_campaign = reg.get_campaign(&campaign_id).unwrap().expect("campaign stored"); + assert_eq!(stored_campaign, campaign_to_json(&generated.campaign)); + let DocRef::ContentId(ref_id) = &generated.campaign.process.r#ref else { + panic!("walkforward's process ref is a content id"); + }; + assert_eq!(ref_id, &process_id, "the stored process id must be the campaign's own ref"); + let stored_process = reg.get_process(ref_id).unwrap().expect("process stored"); + assert_eq!(stored_process, process_to_json(&generated.process)); + let _ = std::fs::remove_dir_all(&dir); + } +``` + +- [ ] **Step 5: Run the translator tests** + +Run: `cargo test -p aura-cli --bin aura translate_walkforward 2>&1 | tail -8` +Expected: PASS (`translate_walkforward_is_deterministic_and_carries_the_regime`, +`translate_walkforward_diverging_regimes_do_not_collide_content_ids` run). + +Run: `cargo test -p aura-cli --bin aura register_generated_wf 2>&1 | tail -6` +Expected: PASS (`register_generated_wf_stores_documents_the_campaign_can_actually_resolve`). + +--- + +## Task 2: `walkforward_summary_json_from_reports` reducer (main.rs) + +**Files:** +- Modify: `crates/aura-cli/src/main.rs` — new fn after `walkforward_summary_json` + (`:1877`) +- Test: `crates/aura-cli/src/main.rs` (mod tests) + +- [ ] **Step 1: Write the pure-sugar reducer** + +Add after `walkforward_summary_json` (ends `:1877`). It reconstructs the exact +summary from the recorded per-window OOS reports, reusing the same reductions the +inline path uses (`MetricStats::from_values` via `r_sma_space`, `r_metrics_from_rs`), +so its output is byte-identical to `walkforward_summary_json`: + +```rust +/// The walk-forward summary line reconstructed from the recorded per-window OOS +/// reports (the campaign path's `WalkForward` `StageFamily.reports`) rather than a +/// live `WalkForwardResult`. Byte-identical to `walkforward_summary_json`: +/// `stitched_total_pips` = the per-window `total_pips` summed left-to-right in roll +/// order (the engine `stitch` folds each segment's final cumulative value, and a +/// window's OOS segment ends at its `total_pips`); `param_stability` reduces each +/// r-sma IS-refit axis over the per-window chosen params (read by exact name from +/// each report's prefix-stripped `manifest.params`) through the same +/// `MetricStats::from_values`; the `oos_r` block pools the per-window `trade_rs` +/// through `r_metrics_from_rs`. Canonical JSON (C14). +/// +/// The axis list + order mirror what `r_sma_space` yields (fast, slow, stop_length, +/// stop_k) — the same order + values the inline `param_stability` reduces. A wrong +/// order or axis would fail the committed exact-grade anchor loudly (it pins +/// `param_stability[0].mean` = the fast-MA refit mean), never silently. +fn walkforward_summary_json_from_reports(reports: &[RunReport]) -> String { + let total: f64 = reports.iter().map(|r| r.metrics.total_pips).sum(); + // (name, is the axis an f64 knob?) — i64 axes coerce value-as-f64, the f64 axis + // directly, exactly as `param_stability`'s per-`ScalarKind` coerce does. + const AXES: [&str; 4] = ["fast.length", "slow.length", "stop_length", "stop_k"]; + let param_stability: Vec = AXES + .iter() + .map(|axis| { + let vals: Vec = reports + .iter() + .map(|r| { + let (_, v) = r + .manifest + .params + .iter() + .find(|(name, _)| name == axis) + .expect("each r-sma walk-forward window records its chosen axis"); + match v { + Scalar::I64(i) => *i as f64, + Scalar::F64(f) => *f, + Scalar::Bool(b) => *b as i64 as f64, + Scalar::Timestamp(_) => { + unreachable!("a timestamp is a structural axis, never a param knob") + } + } + }) + .collect(); + MetricStats::from_values(&vals) + }) + .collect(); + let pooled_rs: Vec = reports + .iter() + .flat_map(|r| r.metrics.r.as_ref().map(|m| m.trade_rs.clone()).unwrap_or_default()) + .collect(); + let mut obj = serde_json::json!({ + "windows": reports.len(), + "stitched_total_pips": total, + "param_stability": param_stability, + }); + if reports.iter().any(|r| r.metrics.r.is_some()) { + obj["oos_r"] = serde_json::to_value(r_metrics_from_rs(&pooled_rs)) + .expect("RMetrics serializes"); + } + serde_json::json!({ "walkforward": obj }).to_string() +} +``` + +Confirm `MetricStats` is in scope (imported from `aura_engine`; `param_stability` +already returns `Vec`, so the type is reachable — if the bare name is +not imported, qualify as `aura_engine::MetricStats`). `r_metrics_from_rs`, +`r_sma_space`, `Scalar`, `RunReport` are already in scope. + +- [ ] **Step 2: Build** + +Run: `cargo build -p aura-cli 2>&1 | tail -3` +Expected: clean (a dead-code warning on `walkforward_summary_json_from_reports` is +acceptable this task — Task 3 wires it; add `#[allow(dead_code)]` above the fn to +keep clippy green, removed in Task 3). + +- [ ] **Step 3: Write a characterization test for the reducer** + +Add to the `main.rs` `mod tests` block a hand-built two-window fixture and assert +the exact reconstructed JSON fields. Build the reports with `total_pips` and +`r.trade_rs` set, and `manifest.params` carrying the four r-sma slots (stripped +names) so `param_stability` reconstructs: + +```rust + #[test] + fn walkforward_summary_from_reports_reconstructs_the_summary() { + // Two windows, chosen params (3,12,14,2.0) then (5,20,14,2.0). + let mk = |fast: i64, slow: i64, pips: f64, rs: Vec| -> RunReport { + let mut rep = RunReport { + manifest: RunManifest { + commit: "t".into(), + params: vec![ + ("fast.length".into(), Scalar::I64(fast)), + ("slow.length".into(), Scalar::I64(slow)), + ("stop_length".into(), Scalar::I64(14)), + ("stop_k".into(), Scalar::F64(2.0)), + ("bias_scale".into(), Scalar::F64(0.5)), + ], + window: (Timestamp(0), Timestamp(0)), + seed: 0, + broker: "t".into(), + selection: None, + instrument: None, + topology_hash: None, + project: None, + }, + metrics: summarize(&[], &[]), + }; + rep.metrics.total_pips = pips; + rep.metrics.r = Some(r_metrics_from_rs(&rs)); + rep + }; + let reports = vec![mk(3, 12, 10.0, vec![1.0, -0.5]), mk(5, 20, -4.0, vec![0.25])]; + let line = walkforward_summary_json_from_reports(&reports); + let v: serde_json::Value = serde_json::from_str(&line).unwrap(); + let wf = &v["walkforward"]; + assert_eq!(wf["windows"].as_u64(), Some(2)); + assert_eq!(wf["stitched_total_pips"].as_f64(), Some(6.0)); // 10.0 + -4.0 + let ps = wf["param_stability"].as_array().unwrap(); + assert_eq!(ps.len(), 4, "four r-sma slots"); + assert_eq!(ps[0]["mean"].as_f64(), Some(4.0)); // fast: (3+5)/2 + assert_eq!(ps[1]["mean"].as_f64(), Some(16.0)); // slow: (12+20)/2 + assert!(wf["oos_r"].is_object(), "pooled oos_r present when any window has r"); + assert_eq!(wf["oos_r"]["n_trades"].as_u64(), Some(3)); // 2 + 1 + } +``` + +Confirm `RunManifest`, `summarize`, `Timestamp` are importable in the test module +(the `walkforward.rs` engine tests use exactly this `RunManifest{…}` shape — mirror +their field set; if a field name differs, read `aura-engine/src/report.rs` and +match it). + +- [ ] **Step 4: Run the reducer test** + +Run: `cargo test -p aura-cli --bin aura walkforward_summary_from_reports 2>&1 | tail -6` +Expected: PASS. + +--- + +## Task 3: `walkforward_args_from` + `wf_ms_sizes` + dispatch split (main.rs, verb_sugar.rs) + +**Files:** +- Modify: `crates/aura-cli/src/main.rs` — `walkforward_args_from`, `wf_ms_sizes`, + `select_is_plateau`, `dispatch_walkforward:4681-4714`; remove the Task-2 + `#[allow(dead_code)]` on the reducer. +- Modify: `crates/aura-cli/src/verb_sugar.rs` — `run_walkforward_sugar`; remove the + Task-1 `#[allow(dead_code)]` bridges. +- Test: `crates/aura-cli/tests/cli_run.rs` — front-end refusals. + +- [ ] **Step 1: Add `run_walkforward_sugar` to verb_sugar.rs (and drop the Task-1 dead-code allows)** + +Append after `register_generated_wf`, and delete the three `#[allow(dead_code)]` +lines added in Task 1: + +```rust +/// Run one dissolved `walkforward` invocation end-to-end: register the generated +/// documents, run through the one campaign path, then reprint the per-window member +/// lines + the summary line from the recorded `WalkForward` family. The stdout +/// summary line is byte-identical to the inline path (the committed exact-grade +/// anchor is the gate). +#[allow(clippy::too_many_arguments)] +pub(crate) fn run_walkforward_sugar( + fast: &[i64], + slow: &[i64], + stop_length: i64, + stop_k: f64, + metric: &str, + in_sample_ms: u64, + out_of_sample_ms: u64, + step_ms: u64, + name: &str, + symbol: &str, + from_ms: i64, + to_ms: i64, + blueprint_canonical: &str, + env: &crate::project::Env, +) -> Result<(), String> { + let generated = translate_walkforward( + fast, slow, stop_length, stop_k, metric, in_sample_ms, out_of_sample_ms, step_ms, + name, symbol, from_ms, to_ms, blueprint_canonical, + )?; + let probe_params: Vec<(String, Scalar)> = vec![ + ("fast.length".to_string(), Scalar::i64(fast[0])), + ("slow.length".to_string(), Scalar::i64(slow[0])), + ]; + validate_before_register(&generated.process, &generated.campaign, blueprint_canonical, &probe_params, env)?; + let reg = env.registry(); + let (_process_id, campaign_id) = register_generated_wf(®, &generated)?; + let run = crate::campaign_run::run_campaign_returning(&campaign_id, env)?; + + // The single cell's walk_forward StageFamily. + let wf = run + .outcome + .cells + .iter() + .flat_map(|c| c.families.iter()) + .find(|f| f.block == "std::walk_forward") + .ok_or("walkforward produced no walk_forward family")?; + + for report in &wf.reports { + println!("{}", crate::family_member_line(&wf.family_id, report)); + } + println!("{}", crate::walkforward_summary_json_from_reports(&wf.reports)); + Ok(()) +} +``` + +Also delete the `#[allow(dead_code)]` line above +`walkforward_summary_json_from_reports` in main.rs (it now has a caller). + +Ordering note: after this step `run_walkforward_sugar` itself has no caller until +Step 3 wires the dispatch split, and Steps 2-3 add the helpers it and the dispatch +need. The workspace build + clippy gate is Step 4 (and Task 4), AFTER the wiring — +do not run `clippy -D warnings` between Steps 1 and 3, where `run_walkforward_sugar` +is transiently unused. + +- [ ] **Step 2: Add `wf_ms_sizes` + `select_is_plateau` + `walkforward_args_from` to main.rs** + +Add near `generalize_args_from` (`:4144-4183`). `wf_ms_sizes` converts the ns +roller constants to ms; `select_is_plateau` detects the plateau modes via the +existing `parse_select`; `walkforward_args_from` mirrors `generalize_args_from` +but for a single instrument, multi-value fast/slow, single-value stop: + +```rust +/// The real walk-forward roller sizes in ms (the campaign wf stage works in ms: +/// span `cell.window_ms`, sizes the `StageBlock` `_ms` fields). `WF_REAL_*_NS` are +/// nanoseconds; divide by 1e6. The ms sizes over the doc window reproduce the same +/// calendar windows the inline ns roller produces (anchor-gated). +fn wf_ms_sizes() -> (u64, u64, u64) { + ( + (WF_REAL_IS_NS / 1_000_000) as u64, + (WF_REAL_OOS_NS / 1_000_000) as u64, + (WF_REAL_STEP_NS / 1_000_000) as u64, + ) +} + +/// True iff `--select` names a plateau mode (refused on the dissolved path, Fork B, +/// until #215 ships the wf-stage plateau relaxation). Reuses `parse_select` so the +/// vocabulary stays single-sourced. +fn select_is_plateau(select: Option<&str>) -> bool { + matches!(select.map(parse_select), Some(Ok(Selection::Plateau(_)))) +} + +/// Convert `WalkforwardCmd` into the resolved argument shape the walkforward sugar +/// consumes (the dissolved `--strategy r-sma --real` branch). Single instrument, +/// multi-value fast/slow (the IS-refit grid), single-value stop (Fork A: the stop is +/// a risk regime, not a swept axis); all four knobs required. `parse_csv_list` +/// (`:1586`) is generic over `T: FromStr`, so the same helper parses the i64 grids +/// and the f64 stop; the local `knobs` closure (captureless → `Copy`, mirroring +/// `generalize_args_from`'s idiom) is reused across the four `ok_or_else` calls. +#[allow(clippy::type_complexity)] +fn walkforward_args_from( + a: &WalkforwardCmd, +) -> Result<(String, String, Vec, Vec, i64, f64, Option, Option), String> { + let symbol = match a.real.as_deref() { + None | Some("") => return Err("walkforward --strategy r-sma dissolves only over --real ".to_string()), + Some(s) => s.to_string(), + }; + let knobs = || "walkforward requires --fast --slow --stop-length --stop-k (fast/slow may be CSV grids; the stop is single-value)".to_string(); + let fast: Vec = parse_csv_list(a.fast.as_deref().ok_or_else(knobs)?).map_err(|()| knobs())?; + let slow: Vec = parse_csv_list(a.slow.as_deref().ok_or_else(knobs)?).map_err(|()| knobs())?; + let stop_length: Vec = parse_csv_list(a.stop_length.as_deref().ok_or_else(knobs)?).map_err(|()| knobs())?; + let stop_k: Vec = parse_csv_list(a.stop_k.as_deref().ok_or_else(knobs)?).map_err(|()| knobs())?; + if stop_length.len() != 1 || stop_k.len() != 1 { + return Err("walkforward: the stop is a single risk regime; --stop-length and --stop-k take one value each".to_string()); + } + let name = a.name.clone().or_else(|| a.trace.clone()).unwrap_or_else(|| "walkforward".to_string()); + Ok((name, symbol, fast, slow, stop_length[0], stop_k[0], a.from, a.to)) +} +``` + +- [ ] **Step 3: Split the `dispatch_walkforward` built-in arm** + +In `dispatch_walkforward`'s `None` arm (`:4681-4714`), insert the dissolution +split at the TOP of the arm body (after the existing `if a.blueprint.is_some() || +!a.axis.is_empty()` guard, before `strategy_from`). Keep the entire existing body +as the fenced `else`: + +```rust + None => { + let usage = || "Usage: aura walkforward [--strategy ] [--real [--from ] [--to ]] [--name | --trace ] [--fast ] [--slow ] [--stop-length ] [--stop-k ] [--select ]".to_string(); + if a.blueprint.is_some() || !a.axis.is_empty() { + eprintln!("aura: {}", usage()); + std::process::exit(2); + } + // Dissolution split (#210, Reading A): only the real-archive r-sma + // execution routes through the campaign path. Everything else — synthetic + // (any strategy) and non-r-sma real — stays on the inline handler below, + // fenced until #159. + if a.strategy.as_deref() == Some("r-sma") && a.real.is_some() { + if select_is_plateau(a.select.as_deref()) { + eprintln!("aura: --select plateau is not yet available on the campaign path; see #215"); + std::process::exit(2); + } + let (name, symbol, fast, slow, stop_length, stop_k, from, to) = + walkforward_args_from(&a).unwrap_or_else(|m| { + eprintln!("aura: {m}"); + std::process::exit(2); + }); + let signal = sma_signal(None, None); + let canonical = blueprint_to_json(&signal).expect("a bare sma_signal serializes"); + let reg = env.registry(); + let topo = topology_hash(&signal); + reg.put_blueprint(&topo, &canonical).unwrap_or_else(|e| { + eprintln!("aura: {e}"); + std::process::exit(1); + }); + let (from_ms, to_ms) = match (from, to) { + (Some(f), Some(t)) => (f, t), + _ => { + let source = DataSource::from_choice( + DataChoice::Real { symbol: symbol.clone(), from_ms: from, to_ms: to }, + env, + ); + let (from_ts, to_ts) = source.full_window(env); + ( + aura_ingest::epoch_ns_to_unix_ms(from_ts), + aura_ingest::epoch_ns_to_unix_ms(to_ts), + ) + } + }; + let (is_ms, oos_ms, step_ms) = wf_ms_sizes(); + verb_sugar::run_walkforward_sugar( + &fast, &slow, stop_length, stop_k, "sqn_normalized", + is_ms, oos_ms, step_ms, &name, &symbol, from_ms, to_ms, &canonical, env, + ) + .unwrap_or_else(|m| { + eprintln!("aura: {m}"); + std::process::exit(1); + }); + return; + } + // …EXISTING fenced body, unchanged: strategy_from / RGrid parse / + // parse_select / name_persist / data_choice_from / run_walkforward(...)… +``` + +Leave everything from `let strategy = strategy_from(...)` (`:4687`) to the end of +the arm exactly as it is (the fenced path). The window-resolution match mirrors +`dispatch_generalize:4375-4388` verbatim (confirm `DataChoice::Real`, +`DataSource::from_choice`, `source.full_window`, `aura_ingest::epoch_ns_to_unix_ms` +are the same symbols dispatch_generalize uses). + +- [ ] **Step 4: Build the workspace** + +Run: `cargo build --workspace 2>&1 | tail -5` +Expected: clean build (0 errors). No dead-code warnings (all bridges removed, all +new fns wired). + +- [ ] **Step 5: The committed anchor must stay green (the acceptance gate)** + +Run: `cargo test -p aura-cli --test cli_run walkforward_real_e2e_pins_the_exact_current_grade 2>&1 | tail -8` +Expected: PASS — the dissolved `--strategy r-sma --real` path reproduces +windows=9, stitched_total_pips=-10398606.666650848, oos_r, param_stability means +byte-for-byte. (Skips clean only if the GER40 archive is absent; on this box it +runs.) + +- [ ] **Step 6: Add front-end refusal tests** + +Append to `crates/aura-cli/tests/cli_run.rs`: + +```rust +/// Fork A: the dissolved walkforward binds the stop as a single risk regime, so a +/// multi-value --stop-length/--stop-k is refused (exit 2) before any run. +#[test] +fn walkforward_dissolved_refuses_a_multi_value_stop() { + let cwd = temp_cwd("walkforward-multi-stop"); + let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) + .args([ + "walkforward", "--strategy", "r-sma", "--real", "GER40", + "--fast", "3", "--slow", "12", "--stop-length", "14,20", "--stop-k", "2.0", + ]) + .current_dir(&cwd) + .output() + .expect("spawn aura"); + assert_eq!(out.status.code(), Some(2), "multi-value stop is a usage refusal"); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!(stderr.contains("single risk regime"), "stop-regime refusal: {stderr}"); +} + +/// Fork B: --select plateau is refused on the dissolved path (exit 2) with a +/// pointer to #215 until the wf-stage plateau relaxation ships. +#[test] +fn walkforward_dissolved_refuses_select_plateau() { + let cwd = temp_cwd("walkforward-plateau"); + let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) + .args([ + "walkforward", "--strategy", "r-sma", "--real", "GER40", + "--fast", "3", "--slow", "12", "--stop-length", "14", "--stop-k", "2.0", + "--select", "plateau:worst", + ]) + .current_dir(&cwd) + .output() + .expect("spawn aura"); + assert_eq!(out.status.code(), Some(2), "plateau is refused on the dissolved path"); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!(stderr.contains("#215"), "forward pointer to the plateau follow-up: {stderr}"); +} +``` + +- [ ] **Step 7: Run the refusal tests** + +Run: `cargo test -p aura-cli --test cli_run walkforward_dissolved_refuses_a_multi_value_stop 2>&1 | tail -6` +Expected: PASS. + +Run: `cargo test -p aura-cli --test cli_run walkforward_dissolved_refuses_select_plateau 2>&1 | tail -6` +Expected: PASS. + +--- + +## Task 4: dissolution-proof e2e (cli_run.rs) + +**Files:** +- Test: `crates/aura-cli/tests/cli_run.rs` + +- [ ] **Step 1: Add the dissolution-proof e2e** + +Mirror `generalize_dissolves_through_the_campaign_path` (`:3706`): a successful +real run auto-registers exactly one process + one campaign + one campaign-run +record, and the walk_forward family is present. + +```rust +/// Property: `aura walkforward --strategy r-sma --real` is now thin sugar over the +/// one campaign path — a successful run durably auto-registers exactly one generated +/// process document, one generated campaign document (carrying the stop as a single +/// risk regime), and one campaign-run record, and persists a WalkForward family. +/// The observable proof the r-sma-real path runs through the executor. Gated on the +/// GER40 archive; skips cleanly on a data refusal. +#[test] +fn walkforward_dissolves_through_the_campaign_path() { + const FROM_MS: &str = "1735689600000"; + const TO_MS: &str = "1767225599000"; + let cwd = temp_cwd("walkforward-dissolves"); + let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) + .args([ + "walkforward", "--strategy", "r-sma", "--real", "GER40", + "--fast", "3,5", "--slow", "12,20", "--stop-length", "14", "--stop-k", "2.0", + "--from", FROM_MS, "--to", TO_MS, + ]) + .current_dir(&cwd) + .output() + .expect("spawn aura"); + if out.status.code() == Some(1) { + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + stderr.contains("no local data") || stderr.contains("no recorded geometry"), + "exit 1 must be a data refusal, got: {stderr}" + ); + eprintln!("skip: no local GER40 data"); + return; + } + assert_eq!(out.status.code(), Some(0), "exit: {:?}", out.status); + let count = |sub: &str| { + std::fs::read_dir(cwd.join("runs").join(sub)).map(|d| d.count()).unwrap_or(0) + }; + assert_eq!(count("processes"), 1, "one generated process document registered"); + assert_eq!(count("campaigns"), 1, "one generated campaign document registered"); + let runs_log = std::fs::read_to_string(cwd.join("runs").join("campaign_runs.jsonl")) + .expect("campaign_runs.jsonl written"); + assert_eq!(runs_log.lines().count(), 1, "exactly one campaign-run record"); + let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout"); + assert!(stdout.contains("\"walkforward\":"), "summary line present: {stdout}"); + assert!(stdout.contains("\"windows\":9"), "nine windows: {stdout}"); +} +``` + +Confirm the `runs/processes` + `runs/campaigns` + `runs/campaign_runs.jsonl` +layout matches what `generalize_dissolves_through_the_campaign_path` asserts (read +`:3706-3730` and mirror the exact directory/file names it uses). + +- [ ] **Step 2: Run the e2e** + +Run: `cargo test -p aura-cli --test cli_run walkforward_dissolves_through_the_campaign_path 2>&1 | tail -8` +Expected: PASS (or clean skip on a data refusal). + +- [ ] **Step 3: Full suite + lint gate** + +Run: `cargo test --workspace 2>&1 | tail -15` +Expected: all green (no regressions; the fenced synthetic/inline walkforward tests +`walkforward_bare_sma_summary_has_no_oos_r` / `walkforward_strategy_r_sma_reports_oos_r` +still pass unchanged). + +Run: `cargo clippy --workspace --all-targets -- -D warnings 2>&1 | tail -8` +Expected: clean (no dead-code, no too_many_arguments — the `#[allow]`s cover the +new translator/runner). diff --git a/docs/specs/walkforward-dissolution.md b/docs/specs/walkforward-dissolution.md new file mode 100644 index 0000000..7b68001 --- /dev/null +++ b/docs/specs/walkforward-dissolution.md @@ -0,0 +1,352 @@ +# Walkforward dissolution — Design Spec + +**Date:** 2026-07-06 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +> Milestone #210 ("Verb dissolution — the campaign path as canonical +> orchestration"), third dissolution (after sweep and generalize). Reference +> issue: Gitea Brummel/Aura #210. Scope reading: decision 2, Reading A (the verb +> surfaces stay fenced until #159; a real-archive execution of a built-in +> strategy routes through the campaign path now). Derived forks A/B and the +> pure-sugar finding are recorded as #210 comments (2026-07-06). + +## Goal + +Dissolve `aura walkforward`'s inline built-in `--strategy r-sma --real` execution +into thin sugar over the one campaign executor (`aura_campaign::execute`), +exactly as sweep and generalize already were. The `dispatch_walkforward` built-in +arm gains an r-sma-real branch that runs through a generated, auto-registered, +content-addressed campaign document, reproducing the `{"walkforward":{…}}` +summary line byte-for-byte. The committed anchor +`walkforward_real_e2e_pins_the_exact_current_grade` is the acceptance gate. + +`run_walkforward` is NOT deleted (unlike generalize, which had no synthetic +path): the built-in arm splits, and everything the dissolution predicate does not +match stays on the inline handler. + +Out of scope (stay verb-wired, fenced until #159 — Reading A): the blueprint-file +branch (`run_blueprint_walkforward`, hardcoded `DataSource::Synthetic`), the +synthetic built-in path (any strategy, no `--real`), and the non-r-sma real path +(`--strategy sma --real` — a different blueprint the r-sma sugar does not +synthesize). The `--select plateau:*` sub-path is carved out to a follow-up (Fork +B): the dissolved dispatch refuses it (exit 2) with a forward pointer until the +decision-4 wf-stage plateau relaxation (#215) ships. + +## Architecture + +The dissolution reuses the generalize seam verbatim; the only new insight is that +the walk-forward verb summary reconstructs from the campaign's recorded outcome +as **pure sugar — no executor change**. + +Grounding (verified, recorded on #210): + +- The campaign `std::walk_forward` stage (`crates/aura-campaign/src/exec.rs`, + `run_walk_forward_stage`) already runs the per-window IS-sweep → `optimize_deflated` + winner → OOS run, and appends the per-window OOS reports as a + `FamilyKind::WalkForward` family. `CellOutcome.families` surfaces that + `StageFamily { block: "std::walk_forward", family_id, reports }` in-memory to + the caller, with each report's `metrics.r.trade_rs` intact (it is `#[serde(skip)]` + on the wire but present in-memory). +- `stitched_total_pips` = Σ (per-window `metrics.total_pips`), summed + left-to-right in window (roll) order. The engine `stitch` offsets each segment + by the running sum of prior segments' final cumulative values, so + `stitched_oos_equity.last() = Σ_i seg_i.last`, and `seg_i.last == total_pips_i`. + **Verified byte-exact** on the anchor: `-10398606.666650848`. The discarded + `oos_equity: vec![]` (exec.rs:794) is therefore NOT needed. +- pooled `oos_r` = `r_metrics_from_rs`(pool of the per-window `trade_rs`, roll + order) — the same reduction the inline `pooled_oos_trade_rs` performs. No + `std::monte_carlo` stage is appended. (Σ per-window `n_trades` = 20681 verified + == the pooled `n_trades`.) +- `param_stability` is the on-demand reduction over the per-window chosen params; + reconstructable from the family reports' recorded manifest params (roll order). + +The canonical process shape for a pure walk-forward is `[std::sweep(argmax), +std::walk_forward]`: the wf stage's per-window IS-refit needs a survivor +population, and the campaign feeds it from a preceding sweep (a selection-free +sweep is permitted only as the terminal stage). The preceding sweep's whole +family flows on as survivors (`select` names a recorded selection, never a +filter — exec.rs:319), so the wf stage receives the full fast×slow grid and +refits it per window. That preceding full-window sweep records an extra +`Sweep` family (a campaign-path audit-trail addition, accepted exactly as +generalize's family-set change was — the anchor pins only the summary line). + +Fork A (stop mapping): fast/slow are the IS-refit grid (multi-value campaign +strategy axes); the stop is a single risk regime `RiskRegime::Vol{stop_length, +stop_k}` (the member runner maps `cell.regime → StopRule::Vol`). `--stop-length` +/ `--stop-k` are single-value; a multi-value stop is refused. + +Load-bearing implementation detail (anchor-gated, not an assumption about current +behaviour): the inline roller works in ns (span from `probe_window`, sizes +`WF_REAL_*_NS`); the campaign wf stage works in ms (span `cell.window_ms` from the +doc's `from_ms`/`to_ms`, sizes the `StageBlock` `_ms` fields). Since ns = ms × +1e6 linearly, the ms roller sizes (`WF_REAL_*_NS / 1_000_000`) over the doc window +must reproduce the identical 9 calendar windows — the anchor's `windows` count and +exact floats are what verify this end-to-end; any span/roller drift fails there +loudly. + +## Concrete code shapes + +### The user-facing program (unchanged surface, dissolved execution) + +```console +$ aura walkforward --strategy r-sma --real GER40 \ + --from 1735689600000 --to 1767225599000 \ + --fast 3,5 --slow 12,20 --stop-length 14 --stop-k 2.0 +{"family_id":"…","report":{…}} # per-window OOS member line (×9) +… +{"walkforward":{"oos_r":{…},"param_stability":[…], + "stitched_total_pips":-10398606.666650848,"windows":9}} +``` + +The summary line is byte-identical to today's inline output (the acceptance +gate). The per-window member lines carry the campaign-generated `WalkForward` +family id rather than the inline `walkforward-N` handle (a family-handle change, +accepted like generalize's — the anchor pins the summary line, not the member +handles). + +This example IS the acceptance-criterion evidence (aura CLAUDE.md domain +invariant 12: the World runs *any* harness through the one path; invariant 10: +closed-vocabulary campaign data, no verb-special logic). The audience — a +researcher validating a candidate out-of-sample — reaches for the identical +command; nothing about the surface changes, while the execution becomes a durable, +diffable, reproducible campaign document (C18). It reintroduces no look-ahead or +determinism failure: the campaign wf stage is the same deterministic engine +`walk_forward` (C1/C2), and the dissolution adds no new streamed non-scalar. + +`--select plateau:mean|plateau:worst` is refused on the dissolved path: + +```console +$ aura walkforward --strategy r-sma --real GER40 --select plateau:worst … +aura: --select plateau is not yet available on the campaign path; see #215 +$ echo $? +2 +``` + +### Sugar translator — mirror of `translate_generalize` (verb_sugar.rs) + +The new `translate_walkforward` differs from `translate_generalize` in three +ways: the process is `[Sweep(argmax), WalkForward]` (not `[Sweep(argmax), +Generalize]`); the campaign carries **one instrument** and **multi-value** +fast/slow axes (the IS-refit grid, not a single candidate); the wf stage's roller +sizes are computed from the data kind. + +```rust +// crates/aura-cli/src/verb_sugar.rs — new, after translate_generalize +pub(crate) struct GeneratedWalkforward { pub process: ProcessDoc, pub campaign: CampaignDoc } + +#[allow(clippy::too_many_arguments)] // #214: bundle deferred until all verbs dissolve +pub(crate) fn translate_walkforward( + fast: &[i64], slow: &[i64], // the IS-refit grid axes (multi-value) + stop_length: i64, stop_k: f64, // single-value → one risk regime + metric: &str, // "sqn_normalized" (the RSma wf objective) + in_sample_ms: u64, out_of_sample_ms: u64, step_ms: u64, // roller sizes, ms + name: &str, symbol: &str, from_ms: i64, to_ms: i64, + blueprint_canonical: &str, +) -> Result { + let process = ProcessDoc { + format_version: FORMAT_VERSION, kind: DocKind::Process, + name: "walkforward".to_string(), description: None, + pipeline: vec![ + // the preceding sweep only enumerates the survivor grid for the wf + // stage; its recorded argmax selection is not part of the summary + StageBlock::Sweep { selection: Some(SweepSelection { + metric: metric.to_string(), select: SelectRule::Argmax, deflate: false }) }, + StageBlock::WalkForward { + in_sample_ms, out_of_sample_ms, step_ms, + mode: WfMode::Rolling, metric: metric.to_string(), select: SelectRule::Argmax }, + ], + }; + let mut doc_axes: BTreeMap = BTreeMap::new(); + doc_axes.insert("fast.length".to_string(), + axis_from_values("fast.length", &fast.iter().map(|&v| Scalar::i64(v)).collect::>())?); + doc_axes.insert("slow.length".to_string(), + axis_from_values("slow.length", &slow.iter().map(|&v| Scalar::i64(v)).collect::>())?); + let campaign = CampaignDoc { + format_version: FORMAT_VERSION, kind: DocKind::Campaign, + name: name.to_string(), description: None, + data: DataSection { instruments: vec![symbol.to_string()], + windows: vec![Window { from_ms, to_ms }] }, + strategies: vec![StrategyEntry { + r#ref: DocRef::ContentId(content_id_of(blueprint_canonical)), axes: doc_axes }], + process: ProcessRef { r#ref: DocRef::ContentId(content_id_of(&process_to_json(&process))) }, + risk: vec![RiskRegime::Vol { length: stop_length, k: stop_k }], + seed: 0, + presentation: Presentation { persist_taps: vec![], emit: vec!["family_table".to_string()] }, + }; + Ok(GeneratedWalkforward { process, campaign }) +} +``` + +### Sugar runner — reprint the summary from the recorded WalkForward family + +```rust +// crates/aura-cli/src/verb_sugar.rs — new, mirror of run_generalize_sugar +#[allow(clippy::too_many_arguments)] +pub(crate) fn run_walkforward_sugar(/* same args as translate_walkforward + env */) + -> Result<(), String> { + let generated = translate_walkforward(/* … */)?; + let probe_params = vec![ + ("fast.length".to_string(), Scalar::i64(fast[0])), + ("slow.length".to_string(), Scalar::i64(slow[0])), + ]; + validate_before_register(&generated.process, &generated.campaign, blueprint_canonical, &probe_params, env)?; + let reg = env.registry(); + let (_p, campaign_id) = register_generated_wf(®, &generated)?; // put_process + put_campaign + let run = crate::campaign_run::run_campaign_returning(&campaign_id, env)?; + + // The single cell's walk_forward StageFamily (block == "std::walk_forward"). + let wf = run.outcome.cells.iter() + .flat_map(|c| c.families.iter()) + .find(|f| f.block == "std::walk_forward") + .ok_or("walkforward produced no walk_forward family")?; + + // Per-window member lines (campaign family id), then the summary line — + // both reconstructed from wf.reports in roll order. + for report in &wf.reports { println!("{}", crate::family_member_line(&wf.family_id, report)); } + println!("{}", crate::walkforward_summary_json_from_reports(&wf.reports)); // new pure-sugar reducer + Ok(()) +} +``` + +The new `walkforward_summary_json_from_reports` is the pure-sugar reducer over +the recorded per-window reports (verified byte-exact against the inline +`walkforward_summary_json`): `windows = reports.len()`; `stitched_total_pips = +reports.iter().map(|r| r.metrics.total_pips).sum()` (left-to-right, roll order — +IEEE-754 order preserved); the `oos_r` block from +`r_metrics_from_rs`(pooled `trade_rs`); `param_stability` from the per-window +recorded params. It reuses the existing `pooled_oos_trade_rs`/`param_stability` +reductions where their inputs are reconstructable from the reports. + +### Dispatch rewrite — split the built-in arm; `run_walkforward` STAYS for the fenced paths + +Unlike generalize (which requires `--real` and has no synthetic path), +`dispatch_walkforward`'s `None` (built-in) arm today routes BOTH synthetic +(`aura walkforward` without `--real`) and real through `run_walkforward`. Under +decision 2 / Reading A only the **real-archive r-sma execution** dissolves; the +synthetic path (any strategy) and the non-r-sma real path (`--strategy sma +--real` — a different blueprint the r-sma sugar does not synthesize) stay +verb-wired, fenced until #159. So the arm **splits** on the dissolution +predicate and `run_walkforward` is **retained** for the fenced branch — this +cycle deletes NO inline code; it adds an r-sma-real branch alongside the existing +body. + +```rust +// crates/aura-cli/src/main.rs — dispatch_walkforward, the None (built-in) arm +None => { + // …existing usage / blueprint-flag guards, unchanged… + let dissolves = a.strategy.as_deref() == Some("r-sma") && a.real.is_some(); + if !dissolves { + // Fenced (Reading A): synthetic (any strategy) + non-r-sma real stay on + // the inline handler — the EXISTING arm body, byte-for-byte unchanged: + // strategy_from / RGrid parse / parse_select / name_persist / + // data_choice_from → run_walkforward(strategy, &name, persist, + // DataSource::from_choice(data, env), &grid, select, env); + return; /* existing body, verbatim */ + } + // r-sma --real: the dissolved sugar path. + if is_plateau(a.select.as_deref()) { // Fork B: refuse plateau on the dissolved path + eprintln!("aura: --select plateau is not yet available on the campaign path; see #215"); + std::process::exit(2); + } + let (name, symbol, fast, slow, stop_length, stop_k, from, to) = + walkforward_args_from(&a) /* unwrap_or_else: eprintln + exit 2, the generalize idiom */; + // Synthesize the bare open sma_signal blueprint (identical to generalize). + let signal = sma_signal(None, None); + let canonical = blueprint_to_json(&signal).expect("a bare sma_signal serializes"); + let reg = env.registry(); + reg.put_blueprint(&topology_hash(&signal), &canonical) /* exit 1 on err */; + // Window: explicit --from/--to when both present, else the symbol's full + // archive window (as generalize; no-window grade drifts with the archive). + let (from_ms, to_ms) = resolve_window(&symbol, from, to, env); + let (is_ms, oos_ms, step_ms) = wf_ms_sizes(); // WF_REAL_*_NS / 1_000_000 + verb_sugar::run_walkforward_sugar(&fast, &slow, stop_length, stop_k, + "sqn_normalized", is_ms, oos_ms, step_ms, &name, &symbol, from_ms, to_ms, &canonical, env) + /* unwrap_or_else: eprintln + exit 1 */; +} +``` + +`run_walkforward` (main.rs:1665) and every helper it uses — `walkforward_family`, +`walkforward_summary_json`, `pooled_oos_trade_rs`, `param_stability`, +`select_winner`, `run_oos_r`, `family_member_line` — are **retained**: recon +confirmed each keeps a live non-test caller (the synthetic path, the blueprint +walkforward path, and the `mc` R-bootstrap path). Nothing is deleted; the new +sugar functions are additive. + +## Components + +- **`crates/aura-cli/src/verb_sugar.rs`** — `translate_walkforward`, + `GeneratedWalkforward`, `register_generated_wf`, `run_walkforward_sugar`, + `walkforward_summary_json_from_reports` (or reuse the inline reducer if it can + read `&[RunReport]`). Reuses `validate_before_register`, `axis_from_values`. +- **`crates/aura-cli/src/main.rs`** — `walkforward_args_from` (mirror of + `generalize_args_from`: single instrument, multi-value fast/slow, single-value + stop with a multi-value refusal, all-four-knobs-required), a `--select plateau` + refusal, the `dispatch_walkforward` built-in-arm split (add the r-sma-real sugar + branch; **retain** the existing body + `run_walkforward` for the fenced else), + the ns→ms roller-size helper. No inline handler or helper is removed. +- **No executor change** (`aura-campaign`, `aura-engine`, `aura-research` + untouched) for the argmax path. + +## Data flow + +argv → `walkforward_args_from` (name, symbol, fast[], slow[], stop, window) → +`dispatch_walkforward` synthesizes + stores the `sma_signal` blueprint, resolves +the window, computes ms roller sizes → `run_walkforward_sugar` → +`translate_walkforward` builds `[Sweep(argmax), WalkForward]` + a campaign +(fast/slow axes, one instrument, one Vol regime) → `validate_before_register` → +`register_generated_wf` → `run_campaign_returning` runs the one executor (sweep +enumerates survivors → wf refits per window → appends the WalkForward family) → +the sugar reads the WalkForward `StageFamily` from `CellOutcome.families`, +reprints the member lines + the summary reducer. + +## Error handling + +- Non-`r-sma` strategy, missing knobs, empty/absent `--real`: reuse the inline + message strings (byte-identical front-end refusals, exit 2), as generalize did. +- Multi-value `--stop-length`/`--stop-k`: exit 2 with "the stop is a single risk + regime; --stop-length/--stop-k take one value each" (Fork A). +- `--select plateau:*`: exit 2 with the forward-pointer message (Fork B). +- Referential faults (unbound axis, unresolvable ref): `validate_before_register` + refuses BEFORE any store write (no store litter), reusing the shared seam. +- Data refusal (no local archive): the campaign member runner's existing + `no_real_data` path (exit 1), same as generalize. + +## Testing strategy + +- **The committed anchor** `walkforward_real_e2e_pins_the_exact_current_grade` + (96dc783) is the equivalence gate: it must stay green through the dissolution + (windows=9, stitched_total_pips, oos_r, param_stability means all exact). +- **`translate_walkforward` unit tests** (mirror the generalize translator + tests): deterministic content ids for identical invocations; the pipeline is + `[Sweep(argmax), WalkForward]` with the right roller sizes + metric; the risk + regime carries the stop; diverging stops do not collide content ids; + `register_generated_wf` stores documents the campaign can resolve. +- **Dissolution-proof e2e** (mirror `generalize_dissolves_through_the_campaign_path`): + a successful real run auto-registers exactly one process + one campaign doc + + one campaign-run record, and the walk_forward family is present. +- **Front-end refusal tests**: multi-value stop (exit 2), `--select plateau` + (exit 2), non-r-sma (exit 2) — each byte-identical or forward-pointing. +- **`walkforward_summary_json_from_reports` unit test**: on a small fixture of + per-window reports, its output equals the inline `walkforward_summary_json` on + the same windows (the pure-sugar equivalence, in isolation). + +## Acceptance criteria + +1. `aura walkforward --strategy r-sma --real --from --to --fast + --slow --stop-length --stop-k ` runs through the campaign path; + the anchor stays green (summary line byte-identical). +2. `--strategy r-sma --real` no longer flows through `run_walkforward`; it runs + the sugar/campaign path. `run_walkforward` is **retained** (not deleted) and + still reached for the fenced branches. +3. The fenced branches are behaviour-unchanged (decision 2 / Reading A): the + blueprint-file path, the synthetic built-in path (any strategy), and the + non-r-sma real path (`--strategy sma --real`) still run inline through + `run_walkforward` exactly as before; every shared helper (`walkforward_family`, + `walkforward_summary_json`, `select_winner`, …) is retained. +4. `--select plateau:*` is explicitly refused (exit 2) with a forward pointer; + the decision-4 wf-stage plateau relaxation is filed as a follow-up issue. +5. Multi-value `--stop-length`/`--stop-k` is refused (Fork A). +6. Full workspace suite green; `clippy --all-targets -D warnings` clean. +7. No executor change (`aura-campaign`/`aura-engine`/`aura-research` diff-free for + the argmax path).