diff --git a/docs/plans/walkforward-dissolution.md b/docs/plans/walkforward-dissolution.md deleted file mode 100644 index 0e1e5a9..0000000 --- a/docs/plans/walkforward-dissolution.md +++ /dev/null @@ -1,808 +0,0 @@ -# Walkforward Dissolution — Implementation Plan - -> **Parent spec:** `docs/specs/walkforward-dissolution.md` -> -> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run -> this plan. Steps use `- [ ]` checkboxes for tracking. - -**Goal:** Route `aura walkforward --strategy r-sma --real` through the one campaign -executor as thin sugar (a generated, content-addressed campaign document), -reproducing the `{"walkforward":{…}}` summary line byte-for-byte, while retaining -the inline `run_walkforward` for the fenced synthetic / non-r-sma branches. - -**Architecture:** Pure sugar — NO executor change. `dispatch_walkforward`'s -built-in arm splits on `--strategy r-sma && --real`; that branch synthesizes a bare -`sma_signal` blueprint (exactly as generalize), builds a `[std::sweep(argmax), -std::walk_forward]` campaign (multi-value fast/slow axes, one Vol risk regime -carrying the stop), runs it via `run_campaign_returning`, and reprints the summary -from the recorded `WalkForward` `StageFamily.reports`. The summary reducer -reconstructs `stitched_total_pips = Σ per-window total_pips`, pooled `oos_r` from -the reports' in-memory `trade_rs`, and `param_stability` via `r_sma_space` + -`MetricStats::from_values` — all byte-verified against the committed anchor. - -**Tech Stack:** `crates/aura-cli/src/verb_sugar.rs` (translator + sugar runner), -`crates/aura-cli/src/main.rs` (args parser, reducer, dispatch split), -`crates/aura-cli/tests/cli_run.rs` (front-end + dissolution-proof tests). The -committed anchor `walkforward_real_e2e_pins_the_exact_current_grade` is the gate. - -**Files this plan creates or modifies:** - -- Modify: `crates/aura-cli/src/verb_sugar.rs` — `GeneratedWalkforward`, - `translate_walkforward`, `register_generated_wf`, `run_walkforward_sugar`; add - `WfMode` to the `aura_research` import. -- Modify: `crates/aura-cli/src/main.rs` — `walkforward_summary_json_from_reports` - reducer, `walkforward_args_from`, `wf_ms_sizes`, `select_is_plateau`, the - `dispatch_walkforward` built-in-arm split (retain the existing body). -- Test: `crates/aura-cli/src/verb_sugar.rs` (mod tests) — translator determinism, - pipeline shape, regime, diverging-stop content ids, registry resolution. -- Test: `crates/aura-cli/src/main.rs` (mod tests) — reducer characterization. -- Test: `crates/aura-cli/tests/cli_run.rs` — front-end refusals + dissolution-proof - e2e; the committed anchor stays green. - ---- - -## Task 1: `translate_walkforward` + `register_generated_wf` (verb_sugar.rs) - -**Files:** -- Modify: `crates/aura-cli/src/verb_sugar.rs:13-17` (import), after `:352` - (`run_generalize_sugar` end) -- Test: `crates/aura-cli/src/verb_sugar.rs` (mod tests, after `:528`) - -- [ ] **Step 1: Add `WfMode` to the `aura_research` import** - -In the `use aura_research::{…}` block (lines 13-17), add `WfMode` (alphabetically -near `Window`). The others (`StageBlock`, `SelectRule`, `RiskRegime`, -`SweepSelection`, `Axis`, `CampaignDoc`, `ProcessDoc`, `DataSection`, `Window`, -`StrategyEntry`, `DocRef`, `ProcessRef`, `Presentation`, `DocKind`, -`FORMAT_VERSION`, `content_id_of`, `process_to_json`) are already present. - -Replace `RiskRegime, SelectRule, StageBlock,` (line 15) with -`RiskRegime, SelectRule, StageBlock, WfMode,` — verify `WfMode` is a real -re-export first: - -Run: `cargo build -p aura-cli 2>&1 | tail -3` -Expected: clean (or an unused-import warning on `WfMode`, resolved by Step 3). - -- [ ] **Step 2: Add `GeneratedWalkforward` + `translate_walkforward` + `register_generated_wf`** - -Append after `run_generalize_sugar` (line 352), mirroring -`GeneratedGeneralize`/`translate_generalize`/`register_generated_g`: - -```rust -/// The two generated documents of one dissolved `walkforward` invocation. -#[derive(Debug)] -pub(crate) struct GeneratedWalkforward { - pub process: ProcessDoc, - pub campaign: CampaignDoc, -} - -/// Translate one `aura walkforward --strategy r-sma --real` invocation into its two -/// generated documents: a `[std::sweep(argmax), std::walk_forward]` process (the -/// sweep only enumerates the IS-refit survivor grid for the wf stage; its recorded -/// argmax selection is not part of the summary) and a campaign running the fast/slow -/// grid over one instrument under a single risk regime that carries the stop. The -/// roller sizes (`in_sample_ms`/`out_of_sample_ms`/`step_ms`) come from the caller -/// (ms). `blueprint_canonical` is the bare `sma_signal` blueprint already stored by -/// topology hash; its content id is the strategy ref. -#[allow(clippy::too_many_arguments)] -pub(crate) fn translate_walkforward( - fast: &[i64], - slow: &[i64], - stop_length: i64, - stop_k: f64, - metric: &str, - in_sample_ms: u64, - out_of_sample_ms: u64, - step_ms: u64, - name: &str, - symbol: &str, - from_ms: i64, - to_ms: i64, - blueprint_canonical: &str, -) -> Result { - let process = ProcessDoc { - format_version: FORMAT_VERSION, - kind: DocKind::Process, - name: "walkforward".to_string(), - description: None, - pipeline: vec![ - StageBlock::Sweep { - selection: Some(SweepSelection { - metric: metric.to_string(), - select: SelectRule::Argmax, - deflate: false, - }), - }, - StageBlock::WalkForward { - in_sample_ms, - out_of_sample_ms, - step_ms, - mode: WfMode::Rolling, - metric: metric.to_string(), - select: SelectRule::Argmax, - }, - ], - }; - let mut doc_axes: BTreeMap = BTreeMap::new(); - doc_axes.insert( - "fast.length".to_string(), - axis_from_values("fast.length", &fast.iter().map(|&v| Scalar::i64(v)).collect::>())?, - ); - doc_axes.insert( - "slow.length".to_string(), - axis_from_values("slow.length", &slow.iter().map(|&v| Scalar::i64(v)).collect::>())?, - ); - let campaign = CampaignDoc { - format_version: FORMAT_VERSION, - kind: DocKind::Campaign, - name: name.to_string(), - description: None, - data: DataSection { - instruments: vec![symbol.to_string()], - windows: vec![Window { from_ms, to_ms }], - }, - strategies: vec![StrategyEntry { - r#ref: DocRef::ContentId(content_id_of(blueprint_canonical)), - axes: doc_axes, - }], - process: ProcessRef { r#ref: DocRef::ContentId(content_id_of(&process_to_json(&process))) }, - risk: vec![RiskRegime::Vol { length: stop_length, k: stop_k }], - seed: 0, - presentation: Presentation { - persist_taps: vec![], - emit: vec!["family_table".to_string()], - }, - }; - Ok(GeneratedWalkforward { process, campaign }) -} - -/// Auto-register both generated walkforward documents (content-addressed, -/// idempotent). Returns `(process_id, campaign_id)`. -pub(crate) fn register_generated_wf( - reg: &aura_registry::Registry, - generated: &GeneratedWalkforward, -) -> Result<(String, String), String> { - let process_id = - reg.put_process(&process_to_json(&generated.process)).map_err(|e| e.to_string())?; - let campaign_id = - reg.put_campaign(&campaign_to_json(&generated.campaign)).map_err(|e| e.to_string())?; - Ok((process_id, campaign_id)) -} -``` - -Note: `campaign_to_json` is already imported (line 14, used by -`register_generated_g`). - -- [ ] **Step 3: Build to confirm it compiles (dead-code allowed until Task 3 wires it)** - -`translate_walkforward`/`register_generated_wf`/`GeneratedWalkforward` have no -caller yet. To keep `clippy -D warnings` green this task, add -`#[allow(dead_code)]` directly above `pub(crate) struct GeneratedWalkforward`, -above `pub(crate) fn translate_walkforward`, and above -`pub(crate) fn register_generated_wf`. Task 3 removes all three once the dispatch -wires them (mirrors the generalize cycle's dead-code bridge). - -Run: `cargo build -p aura-cli 2>&1 | tail -3` -Expected: clean build (no errors). - -- [ ] **Step 4: Add translator unit tests** - -Append inside the `mod tests` block (after line 528, before the closing `}` at -529), mirroring `translate_generalize_is_deterministic_and_carries_the_regime` -and `translate_generalize_diverging_regimes_do_not_collide_content_ids`: - -```rust - #[test] - fn translate_walkforward_is_deterministic_and_carries_the_regime() { - let a = translate_walkforward( - &[3, 5], &[12, 20], 14, 2.0, "sqn_normalized", - 7_776_000_000, 2_592_000_000, 2_592_000_000, - "walkforward", "GER40", 100, 200, "{\"bp\":1}", - ) - .unwrap(); - let b = translate_walkforward( - &[3, 5], &[12, 20], 14, 2.0, "sqn_normalized", - 7_776_000_000, 2_592_000_000, 2_592_000_000, - "walkforward", "GER40", 100, 200, "{\"bp\":1}", - ) - .unwrap(); - assert_eq!( - content_id_of(&campaign_to_json(&a.campaign)), - content_id_of(&campaign_to_json(&b.campaign)), - "identical invocations must generate identical campaign ids" - ); - // pipeline: sweep(argmax) then walk_forward(argmax, rolling) - assert_eq!( - a.process.pipeline, - vec![ - StageBlock::Sweep { - selection: Some(SweepSelection { - metric: "sqn_normalized".to_string(), - select: SelectRule::Argmax, - deflate: false, - }) - }, - StageBlock::WalkForward { - in_sample_ms: 7_776_000_000, - out_of_sample_ms: 2_592_000_000, - step_ms: 2_592_000_000, - mode: WfMode::Rolling, - metric: "sqn_normalized".to_string(), - select: SelectRule::Argmax, - }, - ] - ); - assert_eq!(a.campaign.risk, vec![RiskRegime::Vol { length: 14, k: 2.0 }]); - assert_eq!(a.campaign.data.instruments, vec!["GER40".to_string()]); - assert_eq!(a.campaign.strategies.len(), 1); - // multi-value IS-refit grid axes - assert_eq!( - a.campaign.strategies[0].axes["fast.length"].values, - vec![Scalar::i64(3), Scalar::i64(5)] - ); - assert_eq!( - a.campaign.strategies[0].axes["slow.length"].values, - vec![Scalar::i64(12), Scalar::i64(20)] - ); - assert_eq!(a.process.name, "walkforward"); - assert!(aura_research::validate_campaign(&a.campaign).is_empty()); - assert!(aura_research::validate_process(&a.process).is_empty()); - assert!(aura_campaign::preflight(&a.process, &a.campaign).is_ok()); - } - - #[test] - fn translate_walkforward_diverging_regimes_do_not_collide_content_ids() { - let base = translate_walkforward( - &[3, 5], &[12, 20], 14, 2.0, "sqn_normalized", - 7_776_000_000, 2_592_000_000, 2_592_000_000, - "walkforward", "GER40", 100, 200, "{\"bp\":1}", - ) - .unwrap(); - let different_k = translate_walkforward( - &[3, 5], &[12, 20], 14, 3.0, "sqn_normalized", - 7_776_000_000, 2_592_000_000, 2_592_000_000, - "walkforward", "GER40", 100, 200, "{\"bp\":1}", - ) - .unwrap(); - assert_ne!( - content_id_of(&campaign_to_json(&base.campaign)), - content_id_of(&campaign_to_json(&different_k.campaign)), - "a different stop_k must not collide onto the same campaign id" - ); - } - - #[test] - fn register_generated_wf_stores_documents_the_campaign_can_actually_resolve() { - let dir = std::env::temp_dir().join(format!( - "aura-verb-sugar-walkforward-registry-test-{}", - std::process::id() - )); - std::fs::create_dir_all(&dir).unwrap(); - let reg = aura_registry::Registry::open(dir.join("runs.jsonl")); - let generated = translate_walkforward( - &[3, 5], &[12, 20], 14, 2.0, "sqn_normalized", - 7_776_000_000, 2_592_000_000, 2_592_000_000, - "walkforward", "GER40", 100, 200, "{\"bp\":1}", - ) - .unwrap(); - let (process_id, campaign_id) = register_generated_wf(®, &generated).unwrap(); - let stored_campaign = reg.get_campaign(&campaign_id).unwrap().expect("campaign stored"); - assert_eq!(stored_campaign, campaign_to_json(&generated.campaign)); - let DocRef::ContentId(ref_id) = &generated.campaign.process.r#ref else { - panic!("walkforward's process ref is a content id"); - }; - assert_eq!(ref_id, &process_id, "the stored process id must be the campaign's own ref"); - let stored_process = reg.get_process(ref_id).unwrap().expect("process stored"); - assert_eq!(stored_process, process_to_json(&generated.process)); - let _ = std::fs::remove_dir_all(&dir); - } -``` - -- [ ] **Step 5: Run the translator tests** - -Run: `cargo test -p aura-cli --bin aura translate_walkforward 2>&1 | tail -8` -Expected: PASS (`translate_walkforward_is_deterministic_and_carries_the_regime`, -`translate_walkforward_diverging_regimes_do_not_collide_content_ids` run). - -Run: `cargo test -p aura-cli --bin aura register_generated_wf 2>&1 | tail -6` -Expected: PASS (`register_generated_wf_stores_documents_the_campaign_can_actually_resolve`). - ---- - -## Task 2: `walkforward_summary_json_from_reports` reducer (main.rs) - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` — new fn after `walkforward_summary_json` - (`:1877`) -- Test: `crates/aura-cli/src/main.rs` (mod tests) - -- [ ] **Step 1: Write the pure-sugar reducer** - -Add after `walkforward_summary_json` (ends `:1877`). It reconstructs the exact -summary from the recorded per-window OOS reports, reusing the same reductions the -inline path uses (`MetricStats::from_values` via `r_sma_space`, `r_metrics_from_rs`), -so its output is byte-identical to `walkforward_summary_json`: - -```rust -/// The walk-forward summary line reconstructed from the recorded per-window OOS -/// reports (the campaign path's `WalkForward` `StageFamily.reports`) rather than a -/// live `WalkForwardResult`. Byte-identical to `walkforward_summary_json`: -/// `stitched_total_pips` = the per-window `total_pips` summed left-to-right in roll -/// order (the engine `stitch` folds each segment's final cumulative value, and a -/// window's OOS segment ends at its `total_pips`); `param_stability` reduces each -/// r-sma IS-refit axis over the per-window chosen params (read by exact name from -/// each report's prefix-stripped `manifest.params`) through the same -/// `MetricStats::from_values`; the `oos_r` block pools the per-window `trade_rs` -/// through `r_metrics_from_rs`. Canonical JSON (C14). -/// -/// The axis list + order mirror what `r_sma_space` yields (fast, slow, stop_length, -/// stop_k) — the same order + values the inline `param_stability` reduces. A wrong -/// order or axis would fail the committed exact-grade anchor loudly (it pins -/// `param_stability[0].mean` = the fast-MA refit mean), never silently. -fn walkforward_summary_json_from_reports(reports: &[RunReport]) -> String { - let total: f64 = reports.iter().map(|r| r.metrics.total_pips).sum(); - // (name, is the axis an f64 knob?) — i64 axes coerce value-as-f64, the f64 axis - // directly, exactly as `param_stability`'s per-`ScalarKind` coerce does. - const AXES: [&str; 4] = ["fast.length", "slow.length", "stop_length", "stop_k"]; - let param_stability: Vec = AXES - .iter() - .map(|axis| { - let vals: Vec = reports - .iter() - .map(|r| { - let (_, v) = r - .manifest - .params - .iter() - .find(|(name, _)| name == axis) - .expect("each r-sma walk-forward window records its chosen axis"); - match v { - Scalar::I64(i) => *i as f64, - Scalar::F64(f) => *f, - Scalar::Bool(b) => *b as i64 as f64, - Scalar::Timestamp(_) => { - unreachable!("a timestamp is a structural axis, never a param knob") - } - } - }) - .collect(); - MetricStats::from_values(&vals) - }) - .collect(); - let pooled_rs: Vec = reports - .iter() - .flat_map(|r| r.metrics.r.as_ref().map(|m| m.trade_rs.clone()).unwrap_or_default()) - .collect(); - let mut obj = serde_json::json!({ - "windows": reports.len(), - "stitched_total_pips": total, - "param_stability": param_stability, - }); - if reports.iter().any(|r| r.metrics.r.is_some()) { - obj["oos_r"] = serde_json::to_value(r_metrics_from_rs(&pooled_rs)) - .expect("RMetrics serializes"); - } - serde_json::json!({ "walkforward": obj }).to_string() -} -``` - -Confirm `MetricStats` is in scope (imported from `aura_engine`; `param_stability` -already returns `Vec`, so the type is reachable — if the bare name is -not imported, qualify as `aura_engine::MetricStats`). `r_metrics_from_rs`, -`r_sma_space`, `Scalar`, `RunReport` are already in scope. - -- [ ] **Step 2: Build** - -Run: `cargo build -p aura-cli 2>&1 | tail -3` -Expected: clean (a dead-code warning on `walkforward_summary_json_from_reports` is -acceptable this task — Task 3 wires it; add `#[allow(dead_code)]` above the fn to -keep clippy green, removed in Task 3). - -- [ ] **Step 3: Write a characterization test for the reducer** - -Add to the `main.rs` `mod tests` block a hand-built two-window fixture and assert -the exact reconstructed JSON fields. Build the reports with `total_pips` and -`r.trade_rs` set, and `manifest.params` carrying the four r-sma slots (stripped -names) so `param_stability` reconstructs: - -```rust - #[test] - fn walkforward_summary_from_reports_reconstructs_the_summary() { - // Two windows, chosen params (3,12,14,2.0) then (5,20,14,2.0). - let mk = |fast: i64, slow: i64, pips: f64, rs: Vec| -> RunReport { - let mut rep = RunReport { - manifest: RunManifest { - commit: "t".into(), - params: vec![ - ("fast.length".into(), Scalar::I64(fast)), - ("slow.length".into(), Scalar::I64(slow)), - ("stop_length".into(), Scalar::I64(14)), - ("stop_k".into(), Scalar::F64(2.0)), - ("bias_scale".into(), Scalar::F64(0.5)), - ], - window: (Timestamp(0), Timestamp(0)), - seed: 0, - broker: "t".into(), - selection: None, - instrument: None, - topology_hash: None, - project: None, - }, - metrics: summarize(&[], &[]), - }; - rep.metrics.total_pips = pips; - rep.metrics.r = Some(r_metrics_from_rs(&rs)); - rep - }; - let reports = vec![mk(3, 12, 10.0, vec![1.0, -0.5]), mk(5, 20, -4.0, vec![0.25])]; - let line = walkforward_summary_json_from_reports(&reports); - let v: serde_json::Value = serde_json::from_str(&line).unwrap(); - let wf = &v["walkforward"]; - assert_eq!(wf["windows"].as_u64(), Some(2)); - assert_eq!(wf["stitched_total_pips"].as_f64(), Some(6.0)); // 10.0 + -4.0 - let ps = wf["param_stability"].as_array().unwrap(); - assert_eq!(ps.len(), 4, "four r-sma slots"); - assert_eq!(ps[0]["mean"].as_f64(), Some(4.0)); // fast: (3+5)/2 - assert_eq!(ps[1]["mean"].as_f64(), Some(16.0)); // slow: (12+20)/2 - assert!(wf["oos_r"].is_object(), "pooled oos_r present when any window has r"); - assert_eq!(wf["oos_r"]["n_trades"].as_u64(), Some(3)); // 2 + 1 - } -``` - -Confirm `RunManifest`, `summarize`, `Timestamp` are importable in the test module -(the `walkforward.rs` engine tests use exactly this `RunManifest{…}` shape — mirror -their field set; if a field name differs, read `aura-engine/src/report.rs` and -match it). - -- [ ] **Step 4: Run the reducer test** - -Run: `cargo test -p aura-cli --bin aura walkforward_summary_from_reports 2>&1 | tail -6` -Expected: PASS. - ---- - -## Task 3: `walkforward_args_from` + `wf_ms_sizes` + dispatch split (main.rs, verb_sugar.rs) - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` — `walkforward_args_from`, `wf_ms_sizes`, - `select_is_plateau`, `dispatch_walkforward:4681-4714`; remove the Task-2 - `#[allow(dead_code)]` on the reducer. -- Modify: `crates/aura-cli/src/verb_sugar.rs` — `run_walkforward_sugar`; remove the - Task-1 `#[allow(dead_code)]` bridges. -- Test: `crates/aura-cli/tests/cli_run.rs` — front-end refusals. - -- [ ] **Step 1: Add `run_walkforward_sugar` to verb_sugar.rs (and drop the Task-1 dead-code allows)** - -Append after `register_generated_wf`, and delete the three `#[allow(dead_code)]` -lines added in Task 1: - -```rust -/// Run one dissolved `walkforward` invocation end-to-end: register the generated -/// documents, run through the one campaign path, then reprint the per-window member -/// lines + the summary line from the recorded `WalkForward` family. The stdout -/// summary line is byte-identical to the inline path (the committed exact-grade -/// anchor is the gate). -#[allow(clippy::too_many_arguments)] -pub(crate) fn run_walkforward_sugar( - fast: &[i64], - slow: &[i64], - stop_length: i64, - stop_k: f64, - metric: &str, - in_sample_ms: u64, - out_of_sample_ms: u64, - step_ms: u64, - name: &str, - symbol: &str, - from_ms: i64, - to_ms: i64, - blueprint_canonical: &str, - env: &crate::project::Env, -) -> Result<(), String> { - let generated = translate_walkforward( - fast, slow, stop_length, stop_k, metric, in_sample_ms, out_of_sample_ms, step_ms, - name, symbol, from_ms, to_ms, blueprint_canonical, - )?; - let probe_params: Vec<(String, Scalar)> = vec![ - ("fast.length".to_string(), Scalar::i64(fast[0])), - ("slow.length".to_string(), Scalar::i64(slow[0])), - ]; - validate_before_register(&generated.process, &generated.campaign, blueprint_canonical, &probe_params, env)?; - let reg = env.registry(); - let (_process_id, campaign_id) = register_generated_wf(®, &generated)?; - let run = crate::campaign_run::run_campaign_returning(&campaign_id, env)?; - - // The single cell's walk_forward StageFamily. - let wf = run - .outcome - .cells - .iter() - .flat_map(|c| c.families.iter()) - .find(|f| f.block == "std::walk_forward") - .ok_or("walkforward produced no walk_forward family")?; - - for report in &wf.reports { - println!("{}", crate::family_member_line(&wf.family_id, report)); - } - println!("{}", crate::walkforward_summary_json_from_reports(&wf.reports)); - Ok(()) -} -``` - -Also delete the `#[allow(dead_code)]` line above -`walkforward_summary_json_from_reports` in main.rs (it now has a caller). - -Ordering note: after this step `run_walkforward_sugar` itself has no caller until -Step 3 wires the dispatch split, and Steps 2-3 add the helpers it and the dispatch -need. The workspace build + clippy gate is Step 4 (and Task 4), AFTER the wiring — -do not run `clippy -D warnings` between Steps 1 and 3, where `run_walkforward_sugar` -is transiently unused. - -- [ ] **Step 2: Add `wf_ms_sizes` + `select_is_plateau` + `walkforward_args_from` to main.rs** - -Add near `generalize_args_from` (`:4144-4183`). `wf_ms_sizes` converts the ns -roller constants to ms; `select_is_plateau` detects the plateau modes via the -existing `parse_select`; `walkforward_args_from` mirrors `generalize_args_from` -but for a single instrument, multi-value fast/slow, single-value stop: - -```rust -/// The real walk-forward roller sizes in ms (the campaign wf stage works in ms: -/// span `cell.window_ms`, sizes the `StageBlock` `_ms` fields). `WF_REAL_*_NS` are -/// nanoseconds; divide by 1e6. The ms sizes over the doc window reproduce the same -/// calendar windows the inline ns roller produces (anchor-gated). -fn wf_ms_sizes() -> (u64, u64, u64) { - ( - (WF_REAL_IS_NS / 1_000_000) as u64, - (WF_REAL_OOS_NS / 1_000_000) as u64, - (WF_REAL_STEP_NS / 1_000_000) as u64, - ) -} - -/// True iff `--select` names a plateau mode (refused on the dissolved path, Fork B, -/// until #215 ships the wf-stage plateau relaxation). Reuses `parse_select` so the -/// vocabulary stays single-sourced. -fn select_is_plateau(select: Option<&str>) -> bool { - matches!(select.map(parse_select), Some(Ok(Selection::Plateau(_)))) -} - -/// Convert `WalkforwardCmd` into the resolved argument shape the walkforward sugar -/// consumes (the dissolved `--strategy r-sma --real` branch). Single instrument, -/// multi-value fast/slow (the IS-refit grid), single-value stop (Fork A: the stop is -/// a risk regime, not a swept axis); all four knobs required. `parse_csv_list` -/// (`:1586`) is generic over `T: FromStr`, so the same helper parses the i64 grids -/// and the f64 stop; the local `knobs` closure (captureless → `Copy`, mirroring -/// `generalize_args_from`'s idiom) is reused across the four `ok_or_else` calls. -#[allow(clippy::type_complexity)] -fn walkforward_args_from( - a: &WalkforwardCmd, -) -> Result<(String, String, Vec, Vec, i64, f64, Option, Option), String> { - let symbol = match a.real.as_deref() { - None | Some("") => return Err("walkforward --strategy r-sma dissolves only over --real ".to_string()), - Some(s) => s.to_string(), - }; - let knobs = || "walkforward requires --fast --slow --stop-length --stop-k (fast/slow may be CSV grids; the stop is single-value)".to_string(); - let fast: Vec = parse_csv_list(a.fast.as_deref().ok_or_else(knobs)?).map_err(|()| knobs())?; - let slow: Vec = parse_csv_list(a.slow.as_deref().ok_or_else(knobs)?).map_err(|()| knobs())?; - let stop_length: Vec = parse_csv_list(a.stop_length.as_deref().ok_or_else(knobs)?).map_err(|()| knobs())?; - let stop_k: Vec = parse_csv_list(a.stop_k.as_deref().ok_or_else(knobs)?).map_err(|()| knobs())?; - if stop_length.len() != 1 || stop_k.len() != 1 { - return Err("walkforward: the stop is a single risk regime; --stop-length and --stop-k take one value each".to_string()); - } - let name = a.name.clone().or_else(|| a.trace.clone()).unwrap_or_else(|| "walkforward".to_string()); - Ok((name, symbol, fast, slow, stop_length[0], stop_k[0], a.from, a.to)) -} -``` - -- [ ] **Step 3: Split the `dispatch_walkforward` built-in arm** - -In `dispatch_walkforward`'s `None` arm (`:4681-4714`), insert the dissolution -split at the TOP of the arm body (after the existing `if a.blueprint.is_some() || -!a.axis.is_empty()` guard, before `strategy_from`). Keep the entire existing body -as the fenced `else`: - -```rust - None => { - let usage = || "Usage: aura walkforward [--strategy ] [--real [--from ] [--to ]] [--name | --trace ] [--fast ] [--slow ] [--stop-length ] [--stop-k ] [--select ]".to_string(); - if a.blueprint.is_some() || !a.axis.is_empty() { - eprintln!("aura: {}", usage()); - std::process::exit(2); - } - // Dissolution split (#210, Reading A): only the real-archive r-sma - // execution routes through the campaign path. Everything else — synthetic - // (any strategy) and non-r-sma real — stays on the inline handler below, - // fenced until #159. - if a.strategy.as_deref() == Some("r-sma") && a.real.is_some() { - if select_is_plateau(a.select.as_deref()) { - eprintln!("aura: --select plateau is not yet available on the campaign path; see #215"); - std::process::exit(2); - } - let (name, symbol, fast, slow, stop_length, stop_k, from, to) = - walkforward_args_from(&a).unwrap_or_else(|m| { - eprintln!("aura: {m}"); - std::process::exit(2); - }); - let signal = sma_signal(None, None); - let canonical = blueprint_to_json(&signal).expect("a bare sma_signal serializes"); - let reg = env.registry(); - let topo = topology_hash(&signal); - reg.put_blueprint(&topo, &canonical).unwrap_or_else(|e| { - eprintln!("aura: {e}"); - std::process::exit(1); - }); - let (from_ms, to_ms) = match (from, to) { - (Some(f), Some(t)) => (f, t), - _ => { - let source = DataSource::from_choice( - DataChoice::Real { symbol: symbol.clone(), from_ms: from, to_ms: to }, - env, - ); - let (from_ts, to_ts) = source.full_window(env); - ( - aura_ingest::epoch_ns_to_unix_ms(from_ts), - aura_ingest::epoch_ns_to_unix_ms(to_ts), - ) - } - }; - let (is_ms, oos_ms, step_ms) = wf_ms_sizes(); - verb_sugar::run_walkforward_sugar( - &fast, &slow, stop_length, stop_k, "sqn_normalized", - is_ms, oos_ms, step_ms, &name, &symbol, from_ms, to_ms, &canonical, env, - ) - .unwrap_or_else(|m| { - eprintln!("aura: {m}"); - std::process::exit(1); - }); - return; - } - // …EXISTING fenced body, unchanged: strategy_from / RGrid parse / - // parse_select / name_persist / data_choice_from / run_walkforward(...)… -``` - -Leave everything from `let strategy = strategy_from(...)` (`:4687`) to the end of -the arm exactly as it is (the fenced path). The window-resolution match mirrors -`dispatch_generalize:4375-4388` verbatim (confirm `DataChoice::Real`, -`DataSource::from_choice`, `source.full_window`, `aura_ingest::epoch_ns_to_unix_ms` -are the same symbols dispatch_generalize uses). - -- [ ] **Step 4: Build the workspace** - -Run: `cargo build --workspace 2>&1 | tail -5` -Expected: clean build (0 errors). No dead-code warnings (all bridges removed, all -new fns wired). - -- [ ] **Step 5: The committed anchor must stay green (the acceptance gate)** - -Run: `cargo test -p aura-cli --test cli_run walkforward_real_e2e_pins_the_exact_current_grade 2>&1 | tail -8` -Expected: PASS — the dissolved `--strategy r-sma --real` path reproduces -windows=9, stitched_total_pips=-10398606.666650848, oos_r, param_stability means -byte-for-byte. (Skips clean only if the GER40 archive is absent; on this box it -runs.) - -- [ ] **Step 6: Add front-end refusal tests** - -Append to `crates/aura-cli/tests/cli_run.rs`: - -```rust -/// Fork A: the dissolved walkforward binds the stop as a single risk regime, so a -/// multi-value --stop-length/--stop-k is refused (exit 2) before any run. -#[test] -fn walkforward_dissolved_refuses_a_multi_value_stop() { - let cwd = temp_cwd("walkforward-multi-stop"); - let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args([ - "walkforward", "--strategy", "r-sma", "--real", "GER40", - "--fast", "3", "--slow", "12", "--stop-length", "14,20", "--stop-k", "2.0", - ]) - .current_dir(&cwd) - .output() - .expect("spawn aura"); - assert_eq!(out.status.code(), Some(2), "multi-value stop is a usage refusal"); - let stderr = String::from_utf8_lossy(&out.stderr); - assert!(stderr.contains("single risk regime"), "stop-regime refusal: {stderr}"); -} - -/// Fork B: --select plateau is refused on the dissolved path (exit 2) with a -/// pointer to #215 until the wf-stage plateau relaxation ships. -#[test] -fn walkforward_dissolved_refuses_select_plateau() { - let cwd = temp_cwd("walkforward-plateau"); - let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args([ - "walkforward", "--strategy", "r-sma", "--real", "GER40", - "--fast", "3", "--slow", "12", "--stop-length", "14", "--stop-k", "2.0", - "--select", "plateau:worst", - ]) - .current_dir(&cwd) - .output() - .expect("spawn aura"); - assert_eq!(out.status.code(), Some(2), "plateau is refused on the dissolved path"); - let stderr = String::from_utf8_lossy(&out.stderr); - assert!(stderr.contains("#215"), "forward pointer to the plateau follow-up: {stderr}"); -} -``` - -- [ ] **Step 7: Run the refusal tests** - -Run: `cargo test -p aura-cli --test cli_run walkforward_dissolved_refuses_a_multi_value_stop 2>&1 | tail -6` -Expected: PASS. - -Run: `cargo test -p aura-cli --test cli_run walkforward_dissolved_refuses_select_plateau 2>&1 | tail -6` -Expected: PASS. - ---- - -## Task 4: dissolution-proof e2e (cli_run.rs) - -**Files:** -- Test: `crates/aura-cli/tests/cli_run.rs` - -- [ ] **Step 1: Add the dissolution-proof e2e** - -Mirror `generalize_dissolves_through_the_campaign_path` (`:3706`): a successful -real run auto-registers exactly one process + one campaign + one campaign-run -record, and the walk_forward family is present. - -```rust -/// Property: `aura walkforward --strategy r-sma --real` is now thin sugar over the -/// one campaign path — a successful run durably auto-registers exactly one generated -/// process document, one generated campaign document (carrying the stop as a single -/// risk regime), and one campaign-run record, and persists a WalkForward family. -/// The observable proof the r-sma-real path runs through the executor. Gated on the -/// GER40 archive; skips cleanly on a data refusal. -#[test] -fn walkforward_dissolves_through_the_campaign_path() { - const FROM_MS: &str = "1735689600000"; - const TO_MS: &str = "1767225599000"; - let cwd = temp_cwd("walkforward-dissolves"); - let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) - .args([ - "walkforward", "--strategy", "r-sma", "--real", "GER40", - "--fast", "3,5", "--slow", "12,20", "--stop-length", "14", "--stop-k", "2.0", - "--from", FROM_MS, "--to", TO_MS, - ]) - .current_dir(&cwd) - .output() - .expect("spawn aura"); - if out.status.code() == Some(1) { - let stderr = String::from_utf8_lossy(&out.stderr); - assert!( - stderr.contains("no local data") || stderr.contains("no recorded geometry"), - "exit 1 must be a data refusal, got: {stderr}" - ); - eprintln!("skip: no local GER40 data"); - return; - } - assert_eq!(out.status.code(), Some(0), "exit: {:?}", out.status); - let count = |sub: &str| { - std::fs::read_dir(cwd.join("runs").join(sub)).map(|d| d.count()).unwrap_or(0) - }; - assert_eq!(count("processes"), 1, "one generated process document registered"); - assert_eq!(count("campaigns"), 1, "one generated campaign document registered"); - let runs_log = std::fs::read_to_string(cwd.join("runs").join("campaign_runs.jsonl")) - .expect("campaign_runs.jsonl written"); - assert_eq!(runs_log.lines().count(), 1, "exactly one campaign-run record"); - let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout"); - assert!(stdout.contains("\"walkforward\":"), "summary line present: {stdout}"); - assert!(stdout.contains("\"windows\":9"), "nine windows: {stdout}"); -} -``` - -Confirm the `runs/processes` + `runs/campaigns` + `runs/campaign_runs.jsonl` -layout matches what `generalize_dissolves_through_the_campaign_path` asserts (read -`:3706-3730` and mirror the exact directory/file names it uses). - -- [ ] **Step 2: Run the e2e** - -Run: `cargo test -p aura-cli --test cli_run walkforward_dissolves_through_the_campaign_path 2>&1 | tail -8` -Expected: PASS (or clean skip on a data refusal). - -- [ ] **Step 3: Full suite + lint gate** - -Run: `cargo test --workspace 2>&1 | tail -15` -Expected: all green (no regressions; the fenced synthetic/inline walkforward tests -`walkforward_bare_sma_summary_has_no_oos_r` / `walkforward_strategy_r_sma_reports_oos_r` -still pass unchanged). - -Run: `cargo clippy --workspace --all-targets -- -D warnings 2>&1 | tail -8` -Expected: clean (no dead-code, no too_many_arguments — the `#[allow]`s cover the -new translator/runner). diff --git a/docs/specs/walkforward-dissolution.md b/docs/specs/walkforward-dissolution.md deleted file mode 100644 index 7b68001..0000000 --- a/docs/specs/walkforward-dissolution.md +++ /dev/null @@ -1,352 +0,0 @@ -# Walkforward dissolution — Design Spec - -**Date:** 2026-07-06 -**Status:** Draft — awaiting user spec review -**Authors:** orchestrator + Claude - -> Milestone #210 ("Verb dissolution — the campaign path as canonical -> orchestration"), third dissolution (after sweep and generalize). Reference -> issue: Gitea Brummel/Aura #210. Scope reading: decision 2, Reading A (the verb -> surfaces stay fenced until #159; a real-archive execution of a built-in -> strategy routes through the campaign path now). Derived forks A/B and the -> pure-sugar finding are recorded as #210 comments (2026-07-06). - -## Goal - -Dissolve `aura walkforward`'s inline built-in `--strategy r-sma --real` execution -into thin sugar over the one campaign executor (`aura_campaign::execute`), -exactly as sweep and generalize already were. The `dispatch_walkforward` built-in -arm gains an r-sma-real branch that runs through a generated, auto-registered, -content-addressed campaign document, reproducing the `{"walkforward":{…}}` -summary line byte-for-byte. The committed anchor -`walkforward_real_e2e_pins_the_exact_current_grade` is the acceptance gate. - -`run_walkforward` is NOT deleted (unlike generalize, which had no synthetic -path): the built-in arm splits, and everything the dissolution predicate does not -match stays on the inline handler. - -Out of scope (stay verb-wired, fenced until #159 — Reading A): the blueprint-file -branch (`run_blueprint_walkforward`, hardcoded `DataSource::Synthetic`), the -synthetic built-in path (any strategy, no `--real`), and the non-r-sma real path -(`--strategy sma --real` — a different blueprint the r-sma sugar does not -synthesize). The `--select plateau:*` sub-path is carved out to a follow-up (Fork -B): the dissolved dispatch refuses it (exit 2) with a forward pointer until the -decision-4 wf-stage plateau relaxation (#215) ships. - -## Architecture - -The dissolution reuses the generalize seam verbatim; the only new insight is that -the walk-forward verb summary reconstructs from the campaign's recorded outcome -as **pure sugar — no executor change**. - -Grounding (verified, recorded on #210): - -- The campaign `std::walk_forward` stage (`crates/aura-campaign/src/exec.rs`, - `run_walk_forward_stage`) already runs the per-window IS-sweep → `optimize_deflated` - winner → OOS run, and appends the per-window OOS reports as a - `FamilyKind::WalkForward` family. `CellOutcome.families` surfaces that - `StageFamily { block: "std::walk_forward", family_id, reports }` in-memory to - the caller, with each report's `metrics.r.trade_rs` intact (it is `#[serde(skip)]` - on the wire but present in-memory). -- `stitched_total_pips` = Σ (per-window `metrics.total_pips`), summed - left-to-right in window (roll) order. The engine `stitch` offsets each segment - by the running sum of prior segments' final cumulative values, so - `stitched_oos_equity.last() = Σ_i seg_i.last`, and `seg_i.last == total_pips_i`. - **Verified byte-exact** on the anchor: `-10398606.666650848`. The discarded - `oos_equity: vec![]` (exec.rs:794) is therefore NOT needed. -- pooled `oos_r` = `r_metrics_from_rs`(pool of the per-window `trade_rs`, roll - order) — the same reduction the inline `pooled_oos_trade_rs` performs. No - `std::monte_carlo` stage is appended. (Σ per-window `n_trades` = 20681 verified - == the pooled `n_trades`.) -- `param_stability` is the on-demand reduction over the per-window chosen params; - reconstructable from the family reports' recorded manifest params (roll order). - -The canonical process shape for a pure walk-forward is `[std::sweep(argmax), -std::walk_forward]`: the wf stage's per-window IS-refit needs a survivor -population, and the campaign feeds it from a preceding sweep (a selection-free -sweep is permitted only as the terminal stage). The preceding sweep's whole -family flows on as survivors (`select` names a recorded selection, never a -filter — exec.rs:319), so the wf stage receives the full fast×slow grid and -refits it per window. That preceding full-window sweep records an extra -`Sweep` family (a campaign-path audit-trail addition, accepted exactly as -generalize's family-set change was — the anchor pins only the summary line). - -Fork A (stop mapping): fast/slow are the IS-refit grid (multi-value campaign -strategy axes); the stop is a single risk regime `RiskRegime::Vol{stop_length, -stop_k}` (the member runner maps `cell.regime → StopRule::Vol`). `--stop-length` -/ `--stop-k` are single-value; a multi-value stop is refused. - -Load-bearing implementation detail (anchor-gated, not an assumption about current -behaviour): the inline roller works in ns (span from `probe_window`, sizes -`WF_REAL_*_NS`); the campaign wf stage works in ms (span `cell.window_ms` from the -doc's `from_ms`/`to_ms`, sizes the `StageBlock` `_ms` fields). Since ns = ms × -1e6 linearly, the ms roller sizes (`WF_REAL_*_NS / 1_000_000`) over the doc window -must reproduce the identical 9 calendar windows — the anchor's `windows` count and -exact floats are what verify this end-to-end; any span/roller drift fails there -loudly. - -## Concrete code shapes - -### The user-facing program (unchanged surface, dissolved execution) - -```console -$ aura walkforward --strategy r-sma --real GER40 \ - --from 1735689600000 --to 1767225599000 \ - --fast 3,5 --slow 12,20 --stop-length 14 --stop-k 2.0 -{"family_id":"…","report":{…}} # per-window OOS member line (×9) -… -{"walkforward":{"oos_r":{…},"param_stability":[…], - "stitched_total_pips":-10398606.666650848,"windows":9}} -``` - -The summary line is byte-identical to today's inline output (the acceptance -gate). The per-window member lines carry the campaign-generated `WalkForward` -family id rather than the inline `walkforward-N` handle (a family-handle change, -accepted like generalize's — the anchor pins the summary line, not the member -handles). - -This example IS the acceptance-criterion evidence (aura CLAUDE.md domain -invariant 12: the World runs *any* harness through the one path; invariant 10: -closed-vocabulary campaign data, no verb-special logic). The audience — a -researcher validating a candidate out-of-sample — reaches for the identical -command; nothing about the surface changes, while the execution becomes a durable, -diffable, reproducible campaign document (C18). It reintroduces no look-ahead or -determinism failure: the campaign wf stage is the same deterministic engine -`walk_forward` (C1/C2), and the dissolution adds no new streamed non-scalar. - -`--select plateau:mean|plateau:worst` is refused on the dissolved path: - -```console -$ aura walkforward --strategy r-sma --real GER40 --select plateau:worst … -aura: --select plateau is not yet available on the campaign path; see #215 -$ echo $? -2 -``` - -### Sugar translator — mirror of `translate_generalize` (verb_sugar.rs) - -The new `translate_walkforward` differs from `translate_generalize` in three -ways: the process is `[Sweep(argmax), WalkForward]` (not `[Sweep(argmax), -Generalize]`); the campaign carries **one instrument** and **multi-value** -fast/slow axes (the IS-refit grid, not a single candidate); the wf stage's roller -sizes are computed from the data kind. - -```rust -// crates/aura-cli/src/verb_sugar.rs — new, after translate_generalize -pub(crate) struct GeneratedWalkforward { pub process: ProcessDoc, pub campaign: CampaignDoc } - -#[allow(clippy::too_many_arguments)] // #214: bundle deferred until all verbs dissolve -pub(crate) fn translate_walkforward( - fast: &[i64], slow: &[i64], // the IS-refit grid axes (multi-value) - stop_length: i64, stop_k: f64, // single-value → one risk regime - metric: &str, // "sqn_normalized" (the RSma wf objective) - in_sample_ms: u64, out_of_sample_ms: u64, step_ms: u64, // roller sizes, ms - name: &str, symbol: &str, from_ms: i64, to_ms: i64, - blueprint_canonical: &str, -) -> Result { - let process = ProcessDoc { - format_version: FORMAT_VERSION, kind: DocKind::Process, - name: "walkforward".to_string(), description: None, - pipeline: vec![ - // the preceding sweep only enumerates the survivor grid for the wf - // stage; its recorded argmax selection is not part of the summary - StageBlock::Sweep { selection: Some(SweepSelection { - metric: metric.to_string(), select: SelectRule::Argmax, deflate: false }) }, - StageBlock::WalkForward { - in_sample_ms, out_of_sample_ms, step_ms, - mode: WfMode::Rolling, metric: metric.to_string(), select: SelectRule::Argmax }, - ], - }; - let mut doc_axes: BTreeMap = BTreeMap::new(); - doc_axes.insert("fast.length".to_string(), - axis_from_values("fast.length", &fast.iter().map(|&v| Scalar::i64(v)).collect::>())?); - doc_axes.insert("slow.length".to_string(), - axis_from_values("slow.length", &slow.iter().map(|&v| Scalar::i64(v)).collect::>())?); - let campaign = CampaignDoc { - format_version: FORMAT_VERSION, kind: DocKind::Campaign, - name: name.to_string(), description: None, - data: DataSection { instruments: vec![symbol.to_string()], - windows: vec![Window { from_ms, to_ms }] }, - strategies: vec![StrategyEntry { - r#ref: DocRef::ContentId(content_id_of(blueprint_canonical)), axes: doc_axes }], - process: ProcessRef { r#ref: DocRef::ContentId(content_id_of(&process_to_json(&process))) }, - risk: vec![RiskRegime::Vol { length: stop_length, k: stop_k }], - seed: 0, - presentation: Presentation { persist_taps: vec![], emit: vec!["family_table".to_string()] }, - }; - Ok(GeneratedWalkforward { process, campaign }) -} -``` - -### Sugar runner — reprint the summary from the recorded WalkForward family - -```rust -// crates/aura-cli/src/verb_sugar.rs — new, mirror of run_generalize_sugar -#[allow(clippy::too_many_arguments)] -pub(crate) fn run_walkforward_sugar(/* same args as translate_walkforward + env */) - -> Result<(), String> { - let generated = translate_walkforward(/* … */)?; - let probe_params = vec![ - ("fast.length".to_string(), Scalar::i64(fast[0])), - ("slow.length".to_string(), Scalar::i64(slow[0])), - ]; - validate_before_register(&generated.process, &generated.campaign, blueprint_canonical, &probe_params, env)?; - let reg = env.registry(); - let (_p, campaign_id) = register_generated_wf(®, &generated)?; // put_process + put_campaign - let run = crate::campaign_run::run_campaign_returning(&campaign_id, env)?; - - // The single cell's walk_forward StageFamily (block == "std::walk_forward"). - let wf = run.outcome.cells.iter() - .flat_map(|c| c.families.iter()) - .find(|f| f.block == "std::walk_forward") - .ok_or("walkforward produced no walk_forward family")?; - - // Per-window member lines (campaign family id), then the summary line — - // both reconstructed from wf.reports in roll order. - for report in &wf.reports { println!("{}", crate::family_member_line(&wf.family_id, report)); } - println!("{}", crate::walkforward_summary_json_from_reports(&wf.reports)); // new pure-sugar reducer - Ok(()) -} -``` - -The new `walkforward_summary_json_from_reports` is the pure-sugar reducer over -the recorded per-window reports (verified byte-exact against the inline -`walkforward_summary_json`): `windows = reports.len()`; `stitched_total_pips = -reports.iter().map(|r| r.metrics.total_pips).sum()` (left-to-right, roll order — -IEEE-754 order preserved); the `oos_r` block from -`r_metrics_from_rs`(pooled `trade_rs`); `param_stability` from the per-window -recorded params. It reuses the existing `pooled_oos_trade_rs`/`param_stability` -reductions where their inputs are reconstructable from the reports. - -### Dispatch rewrite — split the built-in arm; `run_walkforward` STAYS for the fenced paths - -Unlike generalize (which requires `--real` and has no synthetic path), -`dispatch_walkforward`'s `None` (built-in) arm today routes BOTH synthetic -(`aura walkforward` without `--real`) and real through `run_walkforward`. Under -decision 2 / Reading A only the **real-archive r-sma execution** dissolves; the -synthetic path (any strategy) and the non-r-sma real path (`--strategy sma ---real` — a different blueprint the r-sma sugar does not synthesize) stay -verb-wired, fenced until #159. So the arm **splits** on the dissolution -predicate and `run_walkforward` is **retained** for the fenced branch — this -cycle deletes NO inline code; it adds an r-sma-real branch alongside the existing -body. - -```rust -// crates/aura-cli/src/main.rs — dispatch_walkforward, the None (built-in) arm -None => { - // …existing usage / blueprint-flag guards, unchanged… - let dissolves = a.strategy.as_deref() == Some("r-sma") && a.real.is_some(); - if !dissolves { - // Fenced (Reading A): synthetic (any strategy) + non-r-sma real stay on - // the inline handler — the EXISTING arm body, byte-for-byte unchanged: - // strategy_from / RGrid parse / parse_select / name_persist / - // data_choice_from → run_walkforward(strategy, &name, persist, - // DataSource::from_choice(data, env), &grid, select, env); - return; /* existing body, verbatim */ - } - // r-sma --real: the dissolved sugar path. - if is_plateau(a.select.as_deref()) { // Fork B: refuse plateau on the dissolved path - eprintln!("aura: --select plateau is not yet available on the campaign path; see #215"); - std::process::exit(2); - } - let (name, symbol, fast, slow, stop_length, stop_k, from, to) = - walkforward_args_from(&a) /* unwrap_or_else: eprintln + exit 2, the generalize idiom */; - // Synthesize the bare open sma_signal blueprint (identical to generalize). - let signal = sma_signal(None, None); - let canonical = blueprint_to_json(&signal).expect("a bare sma_signal serializes"); - let reg = env.registry(); - reg.put_blueprint(&topology_hash(&signal), &canonical) /* exit 1 on err */; - // Window: explicit --from/--to when both present, else the symbol's full - // archive window (as generalize; no-window grade drifts with the archive). - let (from_ms, to_ms) = resolve_window(&symbol, from, to, env); - let (is_ms, oos_ms, step_ms) = wf_ms_sizes(); // WF_REAL_*_NS / 1_000_000 - verb_sugar::run_walkforward_sugar(&fast, &slow, stop_length, stop_k, - "sqn_normalized", is_ms, oos_ms, step_ms, &name, &symbol, from_ms, to_ms, &canonical, env) - /* unwrap_or_else: eprintln + exit 1 */; -} -``` - -`run_walkforward` (main.rs:1665) and every helper it uses — `walkforward_family`, -`walkforward_summary_json`, `pooled_oos_trade_rs`, `param_stability`, -`select_winner`, `run_oos_r`, `family_member_line` — are **retained**: recon -confirmed each keeps a live non-test caller (the synthetic path, the blueprint -walkforward path, and the `mc` R-bootstrap path). Nothing is deleted; the new -sugar functions are additive. - -## Components - -- **`crates/aura-cli/src/verb_sugar.rs`** — `translate_walkforward`, - `GeneratedWalkforward`, `register_generated_wf`, `run_walkforward_sugar`, - `walkforward_summary_json_from_reports` (or reuse the inline reducer if it can - read `&[RunReport]`). Reuses `validate_before_register`, `axis_from_values`. -- **`crates/aura-cli/src/main.rs`** — `walkforward_args_from` (mirror of - `generalize_args_from`: single instrument, multi-value fast/slow, single-value - stop with a multi-value refusal, all-four-knobs-required), a `--select plateau` - refusal, the `dispatch_walkforward` built-in-arm split (add the r-sma-real sugar - branch; **retain** the existing body + `run_walkforward` for the fenced else), - the ns→ms roller-size helper. No inline handler or helper is removed. -- **No executor change** (`aura-campaign`, `aura-engine`, `aura-research` - untouched) for the argmax path. - -## Data flow - -argv → `walkforward_args_from` (name, symbol, fast[], slow[], stop, window) → -`dispatch_walkforward` synthesizes + stores the `sma_signal` blueprint, resolves -the window, computes ms roller sizes → `run_walkforward_sugar` → -`translate_walkforward` builds `[Sweep(argmax), WalkForward]` + a campaign -(fast/slow axes, one instrument, one Vol regime) → `validate_before_register` → -`register_generated_wf` → `run_campaign_returning` runs the one executor (sweep -enumerates survivors → wf refits per window → appends the WalkForward family) → -the sugar reads the WalkForward `StageFamily` from `CellOutcome.families`, -reprints the member lines + the summary reducer. - -## Error handling - -- Non-`r-sma` strategy, missing knobs, empty/absent `--real`: reuse the inline - message strings (byte-identical front-end refusals, exit 2), as generalize did. -- Multi-value `--stop-length`/`--stop-k`: exit 2 with "the stop is a single risk - regime; --stop-length/--stop-k take one value each" (Fork A). -- `--select plateau:*`: exit 2 with the forward-pointer message (Fork B). -- Referential faults (unbound axis, unresolvable ref): `validate_before_register` - refuses BEFORE any store write (no store litter), reusing the shared seam. -- Data refusal (no local archive): the campaign member runner's existing - `no_real_data` path (exit 1), same as generalize. - -## Testing strategy - -- **The committed anchor** `walkforward_real_e2e_pins_the_exact_current_grade` - (96dc783) is the equivalence gate: it must stay green through the dissolution - (windows=9, stitched_total_pips, oos_r, param_stability means all exact). -- **`translate_walkforward` unit tests** (mirror the generalize translator - tests): deterministic content ids for identical invocations; the pipeline is - `[Sweep(argmax), WalkForward]` with the right roller sizes + metric; the risk - regime carries the stop; diverging stops do not collide content ids; - `register_generated_wf` stores documents the campaign can resolve. -- **Dissolution-proof e2e** (mirror `generalize_dissolves_through_the_campaign_path`): - a successful real run auto-registers exactly one process + one campaign doc + - one campaign-run record, and the walk_forward family is present. -- **Front-end refusal tests**: multi-value stop (exit 2), `--select plateau` - (exit 2), non-r-sma (exit 2) — each byte-identical or forward-pointing. -- **`walkforward_summary_json_from_reports` unit test**: on a small fixture of - per-window reports, its output equals the inline `walkforward_summary_json` on - the same windows (the pure-sugar equivalence, in isolation). - -## Acceptance criteria - -1. `aura walkforward --strategy r-sma --real --from --to --fast - --slow --stop-length --stop-k ` runs through the campaign path; - the anchor stays green (summary line byte-identical). -2. `--strategy r-sma --real` no longer flows through `run_walkforward`; it runs - the sugar/campaign path. `run_walkforward` is **retained** (not deleted) and - still reached for the fenced branches. -3. The fenced branches are behaviour-unchanged (decision 2 / Reading A): the - blueprint-file path, the synthetic built-in path (any strategy), and the - non-r-sma real path (`--strategy sma --real`) still run inline through - `run_walkforward` exactly as before; every shared helper (`walkforward_family`, - `walkforward_summary_json`, `select_winner`, …) is retained. -4. `--select plateau:*` is explicitly refused (exit 2) with a forward pointer; - the decision-4 wf-stage plateau relaxation is filed as a follow-up issue. -5. Multi-value `--stop-length`/`--stop-k` is refused (Fork A). -6. Full workspace suite green; `clippy --all-targets -D warnings` clean. -7. No executor change (`aura-campaign`/`aura-engine`/`aura-research` diff-free for - the argmax path).