Aura/docs/specs/walkforward-dissolution.md

# Walkforward dissolution — Design Spec

**Date:** 2026-07-06
**Status:** Draft — awaiting user spec review
**Authors:** orchestrator + Claude

> Milestone #210 ("Verb dissolution — the campaign path as canonical
> orchestration"), third dissolution (after sweep and generalize). Reference
> issue: Gitea Brummel/Aura #210. Scope reading: decision 2, Reading A (the verb
> surfaces stay fenced until #159; a real-archive execution of a built-in
> strategy routes through the campaign path now). Derived forks A/B and the
> pure-sugar finding are recorded as #210 comments (2026-07-06).

## Goal

Dissolve `aura walkforward`'s inline built-in `--strategy r-sma --real` execution
into thin sugar over the one campaign executor (`aura_campaign::execute`),
exactly as sweep and generalize already were. The `dispatch_walkforward` built-in
arm gains an r-sma-real branch that runs through a generated, auto-registered,
content-addressed campaign document, reproducing the `{"walkforward":{…}}`
summary line byte-for-byte. The committed anchor
`walkforward_real_e2e_pins_the_exact_current_grade` is the acceptance gate.

`run_walkforward` is NOT deleted (unlike generalize, which had no synthetic
path): the built-in arm splits, and everything the dissolution predicate does not
match stays on the inline handler.

Out of scope (stay verb-wired, fenced until #159 — Reading A): the blueprint-file
branch (`run_blueprint_walkforward`, hardcoded `DataSource::Synthetic`), the
synthetic built-in path (any strategy, no `--real`), and the non-r-sma real path
(`--strategy sma --real` — a different blueprint the r-sma sugar does not
synthesize). The `--select plateau:*` sub-path is carved out to a follow-up (Fork
B): the dissolved dispatch refuses it (exit 2) with a forward pointer until the
decision-4 wf-stage plateau relaxation (#215) ships.

## Architecture

The dissolution reuses the generalize seam verbatim; the only new insight is that
the walk-forward verb summary reconstructs from the campaign's recorded outcome
as **pure sugar — no executor change**.

Grounding (verified, recorded on #210):

- The campaign `std::walk_forward` stage (`crates/aura-campaign/src/exec.rs`,
  `run_walk_forward_stage`) already runs the per-window IS-sweep → `optimize_deflated`
  winner → OOS run, and appends the per-window OOS reports as a
  `FamilyKind::WalkForward` family. `CellOutcome.families` surfaces that
  `StageFamily { block: "std::walk_forward", family_id, reports }` in-memory to
  the caller, with each report's `metrics.r.trade_rs` intact (it is `#[serde(skip)]`
  on the wire but present in-memory).
- `stitched_total_pips` = Σ (per-window `metrics.total_pips`), summed
  left-to-right in window (roll) order. The engine `stitch` offsets each segment
  by the running sum of prior segments' final cumulative values, so
  `stitched_oos_equity.last() = Σ_i seg_i.last`, and `seg_i.last == total_pips_i`.
  **Verified byte-exact** on the anchor: `-10398606.666650848`. The discarded
  `oos_equity: vec![]` (exec.rs:794) is therefore NOT needed.
- pooled `oos_r` = `r_metrics_from_rs`(pool of the per-window `trade_rs`, roll
  order) — the same reduction the inline `pooled_oos_trade_rs` performs. No
  `std::monte_carlo` stage is appended. (Σ per-window `n_trades` = 20681 verified
  == the pooled `n_trades`.)
- `param_stability` is the on-demand reduction over the per-window chosen params;
  reconstructable from the family reports' recorded manifest params (roll order).

The canonical process shape for a pure walk-forward is `[std::sweep(argmax),
std::walk_forward]`: the wf stage's per-window IS-refit needs a survivor
population, and the campaign feeds it from a preceding sweep (a selection-free
sweep is permitted only as the terminal stage). The preceding sweep's whole
family flows on as survivors (`select` names a recorded selection, never a
filter — exec.rs:319), so the wf stage receives the full fast×slow grid and
refits it per window. That preceding full-window sweep records an extra
`Sweep` family (a campaign-path audit-trail addition, accepted exactly as
generalize's family-set change was — the anchor pins only the summary line).

Fork A (stop mapping): fast/slow are the IS-refit grid (multi-value campaign
strategy axes); the stop is a single risk regime `RiskRegime::Vol{stop_length,
stop_k}` (the member runner maps `cell.regime → StopRule::Vol`). `--stop-length`
/ `--stop-k` are single-value; a multi-value stop is refused.

Load-bearing implementation detail (anchor-gated, not an assumption about current
behaviour): the inline roller works in ns (span from `probe_window`, sizes
`WF_REAL_*_NS`); the campaign wf stage works in ms (span `cell.window_ms` from the
doc's `from_ms`/`to_ms`, sizes the `StageBlock` `_ms` fields). Since ns = ms ×
1e6 linearly, the ms roller sizes (`WF_REAL_*_NS / 1_000_000`) over the doc window
must reproduce the identical 9 calendar windows — the anchor's `windows` count and
exact floats are what verify this end-to-end; any span/roller drift fails there
loudly.

## Concrete code shapes

### The user-facing program (unchanged surface, dissolved execution)

```console
$ aura walkforward --strategy r-sma --real GER40 \
    --from 1735689600000 --to 1767225599000 \
    --fast 3,5 --slow 12,20 --stop-length 14 --stop-k 2.0
{"family_id":"…","report":{…}}          # per-window OOS member line (×9)
…
{"walkforward":{"oos_r":{…},"param_stability":[…],
  "stitched_total_pips":-10398606.666650848,"windows":9}}
```

The summary line is byte-identical to today's inline output (the acceptance
gate). The per-window member lines carry the campaign-generated `WalkForward`
family id rather than the inline `walkforward-N` handle (a family-handle change,
accepted like generalize's — the anchor pins the summary line, not the member
handles).

This example IS the acceptance-criterion evidence (aura CLAUDE.md domain
invariant 12: the World runs *any* harness through the one path; invariant 10:
closed-vocabulary campaign data, no verb-special logic). The audience — a
researcher validating a candidate out-of-sample — reaches for the identical
command; nothing about the surface changes, while the execution becomes a durable,
diffable, reproducible campaign document (C18). It reintroduces no look-ahead or
determinism failure: the campaign wf stage is the same deterministic engine
`walk_forward` (C1/C2), and the dissolution adds no new streamed non-scalar.

`--select plateau:mean|plateau:worst` is refused on the dissolved path:

```console
$ aura walkforward --strategy r-sma --real GER40 --select plateau:worst …
aura: --select plateau is not yet available on the campaign path; see #215
$ echo $?
2
```

### Sugar translator — mirror of `translate_generalize` (verb_sugar.rs)

The new `translate_walkforward` differs from `translate_generalize` in three
ways: the process is `[Sweep(argmax), WalkForward]` (not `[Sweep(argmax),
Generalize]`); the campaign carries **one instrument** and **multi-value**
fast/slow axes (the IS-refit grid, not a single candidate); the wf stage's roller
sizes are computed from the data kind.

```rust
// crates/aura-cli/src/verb_sugar.rs  — new, after translate_generalize
pub(crate) struct GeneratedWalkforward { pub process: ProcessDoc, pub campaign: CampaignDoc }

#[allow(clippy::too_many_arguments)]           // #214: bundle deferred until all verbs dissolve
pub(crate) fn translate_walkforward(
    fast: &[i64], slow: &[i64],                // the IS-refit grid axes (multi-value)
    stop_length: i64, stop_k: f64,             // single-value → one risk regime
    metric: &str,                              // "sqn_normalized" (the RSma wf objective)
    in_sample_ms: u64, out_of_sample_ms: u64, step_ms: u64,   // roller sizes, ms
    name: &str, symbol: &str, from_ms: i64, to_ms: i64,
    blueprint_canonical: &str,
) -> Result<GeneratedWalkforward, String> {
    let process = ProcessDoc {
        format_version: FORMAT_VERSION, kind: DocKind::Process,
        name: "walkforward".to_string(), description: None,
        pipeline: vec![
            // the preceding sweep only enumerates the survivor grid for the wf
            // stage; its recorded argmax selection is not part of the summary
            StageBlock::Sweep { selection: Some(SweepSelection {
                metric: metric.to_string(), select: SelectRule::Argmax, deflate: false }) },
            StageBlock::WalkForward {
                in_sample_ms, out_of_sample_ms, step_ms,
                mode: WfMode::Rolling, metric: metric.to_string(), select: SelectRule::Argmax },
        ],
    };
    let mut doc_axes: BTreeMap<String, Axis> = BTreeMap::new();
    doc_axes.insert("fast.length".to_string(),
        axis_from_values("fast.length", &fast.iter().map(|&v| Scalar::i64(v)).collect::<Vec<_>>())?);
    doc_axes.insert("slow.length".to_string(),
        axis_from_values("slow.length", &slow.iter().map(|&v| Scalar::i64(v)).collect::<Vec<_>>())?);
    let campaign = CampaignDoc {
        format_version: FORMAT_VERSION, kind: DocKind::Campaign,
        name: name.to_string(), description: None,
        data: DataSection { instruments: vec![symbol.to_string()],
                            windows: vec![Window { from_ms, to_ms }] },
        strategies: vec![StrategyEntry {
            r#ref: DocRef::ContentId(content_id_of(blueprint_canonical)), axes: doc_axes }],
        process: ProcessRef { r#ref: DocRef::ContentId(content_id_of(&process_to_json(&process))) },
        risk: vec![RiskRegime::Vol { length: stop_length, k: stop_k }],
        seed: 0,
        presentation: Presentation { persist_taps: vec![], emit: vec!["family_table".to_string()] },
    };
    Ok(GeneratedWalkforward { process, campaign })
}
```

### Sugar runner — reprint the summary from the recorded WalkForward family

```rust
// crates/aura-cli/src/verb_sugar.rs  — new, mirror of run_generalize_sugar
#[allow(clippy::too_many_arguments)]
pub(crate) fn run_walkforward_sugar(/* same args as translate_walkforward + env */)
  -> Result<(), String> {
    let generated = translate_walkforward(/* … */)?;
    let probe_params = vec![
        ("fast.length".to_string(), Scalar::i64(fast[0])),
        ("slow.length".to_string(), Scalar::i64(slow[0])),
    ];
    validate_before_register(&generated.process, &generated.campaign, blueprint_canonical, &probe_params, env)?;
    let reg = env.registry();
    let (_p, campaign_id) = register_generated_wf(&reg, &generated)?;   // put_process + put_campaign
    let run = crate::campaign_run::run_campaign_returning(&campaign_id, env)?;

    // The single cell's walk_forward StageFamily (block == "std::walk_forward").
    let wf = run.outcome.cells.iter()
        .flat_map(|c| c.families.iter())
        .find(|f| f.block == "std::walk_forward")
        .ok_or("walkforward produced no walk_forward family")?;

    // Per-window member lines (campaign family id), then the summary line —
    // both reconstructed from wf.reports in roll order.
    for report in &wf.reports { println!("{}", crate::family_member_line(&wf.family_id, report)); }
    println!("{}", crate::walkforward_summary_json_from_reports(&wf.reports));   // new pure-sugar reducer
    Ok(())
}
```

The new `walkforward_summary_json_from_reports` is the pure-sugar reducer over
the recorded per-window reports (verified byte-exact against the inline
`walkforward_summary_json`): `windows = reports.len()`; `stitched_total_pips =
reports.iter().map(|r| r.metrics.total_pips).sum()` (left-to-right, roll order —
IEEE-754 order preserved); the `oos_r` block from
`r_metrics_from_rs`(pooled `trade_rs`); `param_stability` from the per-window
recorded params. It reuses the existing `pooled_oos_trade_rs`/`param_stability`
reductions where their inputs are reconstructable from the reports.

### Dispatch rewrite — split the built-in arm; `run_walkforward` STAYS for the fenced paths

Unlike generalize (which requires `--real` and has no synthetic path),
`dispatch_walkforward`'s `None` (built-in) arm today routes BOTH synthetic
(`aura walkforward` without `--real`) and real through `run_walkforward`. Under
decision 2 / Reading A only the **real-archive r-sma execution** dissolves; the
synthetic path (any strategy) and the non-r-sma real path (`--strategy sma
--real` — a different blueprint the r-sma sugar does not synthesize) stay
verb-wired, fenced until #159. So the arm **splits** on the dissolution
predicate and `run_walkforward` is **retained** for the fenced branch — this
cycle deletes NO inline code; it adds an r-sma-real branch alongside the existing
body.

```rust
// crates/aura-cli/src/main.rs  — dispatch_walkforward, the None (built-in) arm
None => {
    // …existing usage / blueprint-flag guards, unchanged…
    let dissolves = a.strategy.as_deref() == Some("r-sma") && a.real.is_some();
    if !dissolves {
        // Fenced (Reading A): synthetic (any strategy) + non-r-sma real stay on
        // the inline handler — the EXISTING arm body, byte-for-byte unchanged:
        //   strategy_from / RGrid parse / parse_select / name_persist /
        //   data_choice_from → run_walkforward(strategy, &name, persist,
        //   DataSource::from_choice(data, env), &grid, select, env);
        return; /* existing body, verbatim */
    }
    // r-sma --real: the dissolved sugar path.
    if is_plateau(a.select.as_deref()) {           // Fork B: refuse plateau on the dissolved path
        eprintln!("aura: --select plateau is not yet available on the campaign path; see #215");
        std::process::exit(2);
    }
    let (name, symbol, fast, slow, stop_length, stop_k, from, to) =
        walkforward_args_from(&a) /* unwrap_or_else: eprintln + exit 2, the generalize idiom */;
    // Synthesize the bare open sma_signal blueprint (identical to generalize).
    let signal = sma_signal(None, None);
    let canonical = blueprint_to_json(&signal).expect("a bare sma_signal serializes");
    let reg = env.registry();
    reg.put_blueprint(&topology_hash(&signal), &canonical) /* exit 1 on err */;
    // Window: explicit --from/--to when both present, else the symbol's full
    // archive window (as generalize; no-window grade drifts with the archive).
    let (from_ms, to_ms) = resolve_window(&symbol, from, to, env);
    let (is_ms, oos_ms, step_ms) = wf_ms_sizes();   // WF_REAL_*_NS / 1_000_000
    verb_sugar::run_walkforward_sugar(&fast, &slow, stop_length, stop_k,
        "sqn_normalized", is_ms, oos_ms, step_ms, &name, &symbol, from_ms, to_ms, &canonical, env)
        /* unwrap_or_else: eprintln + exit 1 */;
}
```

`run_walkforward` (main.rs:1665) and every helper it uses — `walkforward_family`,
`walkforward_summary_json`, `pooled_oos_trade_rs`, `param_stability`,
`select_winner`, `run_oos_r`, `family_member_line` — are **retained**: recon
confirmed each keeps a live non-test caller (the synthetic path, the blueprint
walkforward path, and the `mc` R-bootstrap path). Nothing is deleted; the new
sugar functions are additive.

## Components

- **`crates/aura-cli/src/verb_sugar.rs`** — `translate_walkforward`,
  `GeneratedWalkforward`, `register_generated_wf`, `run_walkforward_sugar`,
  `walkforward_summary_json_from_reports` (or reuse the inline reducer if it can
  read `&[RunReport]`). Reuses `validate_before_register`, `axis_from_values`.
- **`crates/aura-cli/src/main.rs`** — `walkforward_args_from` (mirror of
  `generalize_args_from`: single instrument, multi-value fast/slow, single-value
  stop with a multi-value refusal, all-four-knobs-required), a `--select plateau`
  refusal, the `dispatch_walkforward` built-in-arm split (add the r-sma-real sugar
  branch; **retain** the existing body + `run_walkforward` for the fenced else),
  the ns→ms roller-size helper. No inline handler or helper is removed.
- **No executor change** (`aura-campaign`, `aura-engine`, `aura-research`
  untouched) for the argmax path.

## Data flow

argv → `walkforward_args_from` (name, symbol, fast[], slow[], stop, window) →
`dispatch_walkforward` synthesizes + stores the `sma_signal` blueprint, resolves
the window, computes ms roller sizes → `run_walkforward_sugar` →
`translate_walkforward` builds `[Sweep(argmax), WalkForward]` + a campaign
(fast/slow axes, one instrument, one Vol regime) → `validate_before_register` →
`register_generated_wf` → `run_campaign_returning` runs the one executor (sweep
enumerates survivors → wf refits per window → appends the WalkForward family) →
the sugar reads the WalkForward `StageFamily` from `CellOutcome.families`,
reprints the member lines + the summary reducer.

## Error handling

- Non-`r-sma` strategy, missing knobs, empty/absent `--real`: reuse the inline
  message strings (byte-identical front-end refusals, exit 2), as generalize did.
- Multi-value `--stop-length`/`--stop-k`: exit 2 with "the stop is a single risk
  regime; --stop-length/--stop-k take one value each" (Fork A).
- `--select plateau:*`: exit 2 with the forward-pointer message (Fork B).
- Referential faults (unbound axis, unresolvable ref): `validate_before_register`
  refuses BEFORE any store write (no store litter), reusing the shared seam.
- Data refusal (no local archive): the campaign member runner's existing
  `no_real_data` path (exit 1), same as generalize.

## Testing strategy

- **The committed anchor** `walkforward_real_e2e_pins_the_exact_current_grade`
  (96dc783) is the equivalence gate: it must stay green through the dissolution
  (windows=9, stitched_total_pips, oos_r, param_stability means all exact).
- **`translate_walkforward` unit tests** (mirror the generalize translator
  tests): deterministic content ids for identical invocations; the pipeline is
  `[Sweep(argmax), WalkForward]` with the right roller sizes + metric; the risk
  regime carries the stop; diverging stops do not collide content ids;
  `register_generated_wf` stores documents the campaign can resolve.
- **Dissolution-proof e2e** (mirror `generalize_dissolves_through_the_campaign_path`):
  a successful real run auto-registers exactly one process + one campaign doc +
  one campaign-run record, and the walk_forward family is present.
- **Front-end refusal tests**: multi-value stop (exit 2), `--select plateau`
  (exit 2), non-r-sma (exit 2) — each byte-identical or forward-pointing.
- **`walkforward_summary_json_from_reports` unit test**: on a small fixture of
  per-window reports, its output equals the inline `walkforward_summary_json` on
  the same windows (the pure-sugar equivalence, in isolation).

## Acceptance criteria

1. `aura walkforward --strategy r-sma --real <SYM> --from --to --fast <csv>
   --slow <csv> --stop-length <v> --stop-k <v>` runs through the campaign path;
   the anchor stays green (summary line byte-identical).
2. `--strategy r-sma --real` no longer flows through `run_walkforward`; it runs
   the sugar/campaign path. `run_walkforward` is **retained** (not deleted) and
   still reached for the fenced branches.
3. The fenced branches are behaviour-unchanged (decision 2 / Reading A): the
   blueprint-file path, the synthetic built-in path (any strategy), and the
   non-r-sma real path (`--strategy sma --real`) still run inline through
   `run_walkforward` exactly as before; every shared helper (`walkforward_family`,
   `walkforward_summary_json`, `select_winner`, …) is retained.
4. `--select plateau:*` is explicitly refused (exit 2) with a forward pointer;
   the decision-4 wf-stage plateau relaxation is filed as a follow-up issue.
5. Multi-value `--stop-length`/`--stop-k` is refused (Fork A).
6. Full workspace suite green; `clippy --all-targets -D warnings` clean.
7. No executor change (`aura-campaign`/`aura-engine`/`aura-research` diff-free for
   the argmax path).