From cffc297e9a2e3fac16be41a03efd966f84586010 Mon Sep 17 00:00:00 2001 From: Brummel Date: Fri, 26 Jun 2026 18:33:24 +0200 Subject: [PATCH] plan: 0078 cross-instrument generalization Three tasks for #146: (1) RunManifest.instrument serde-widened field + compat-mirror lockstep + thread all in-workspace literals + back-compat tests; (2) aura-registry FamilyKind::CrossInstrument + the R-only generalization reduction (worst-case floor + sign-agreement + breakdown) + check_r_metric + two RegistryError variants; (3) aura-cli generalize subcommand (single-cell stage1-r candidate x instrument list), data-free arity/metric refusals, a gated-skip cross-instrument E2E. fieldtests/ confirmed outside the workspace compile set; FamilyKind has no exhaustive match (serde-derive); only RegistryError's Display match is a gate. refs #146 --- .../0078-cross-instrument-generalization.md | 789 ++++++++++++++++++ 1 file changed, 789 insertions(+) create mode 100644 docs/plans/0078-cross-instrument-generalization.md diff --git a/docs/plans/0078-cross-instrument-generalization.md b/docs/plans/0078-cross-instrument-generalization.md new file mode 100644 index 0000000..cdcb049 --- /dev/null +++ b/docs/plans/0078-cross-instrument-generalization.md @@ -0,0 +1,789 @@ +# Cross-instrument generalization — Implementation Plan + +> **Parent spec:** `docs/specs/0078-cross-instrument-generalization.md` +> +> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run +> this plan. Steps use `- [ ]` checkboxes for tracking. + +**Goal:** Add an `aura generalize` subcommand that grades how consistently a +single candidate holds across a set of instruments (worst-case R floor + +sign-agreement + per-instrument breakdown), as the last anti-false-discovery +piece of the Inferential-validation milestone (#146). + +**Architecture:** A first-class `RunManifest.instrument` lineage field +(serde-widened, C14/C18) + a new `FamilyKind::CrossInstrument` (C12) + an R-only +`generalization` reduction in aura-registry beside `optimize_deflated`/ +`optimize_plateau` (C9) + a CLI `generalize` subcommand that runs a single-cell +stage1-r candidate across an instrument list, stamps each member's instrument, +reduces, prints, and persists a CrossInstrument family. Additive — existing +sweep/walkforward/mc/standalone paths stay byte-identical (C23). + +**Tech Stack:** aura-engine (`RunManifest`), aura-registry (`FamilyKind`, +`RegistryError`, `generalization`/`Generalization`/`check_r_metric`), aura-cli +(`generalize` subcommand). + +--- + +**Files this plan creates or modifies:** + +- Modify: `crates/aura-engine/src/report.rs:463-482` — add `RunManifest.instrument` field + back-compat serde tests +- Modify: `crates/aura-registry/src/compat.rs:26-80` — the `RunManifestRead` mirror + destructure + literal (lockstep with the engine field) +- Modify: `crates/aura-registry/src/lineage.rs:22-27` — `FamilyKind::CrossInstrument` variant +- Modify: `crates/aura-registry/src/lib.rs:429-455` — `Generalization` struct + `generalization` fn + `check_r_metric` fn + two `RegistryError` variants + Display arms + unit tests +- Modify: `crates/aura-cli/src/main.rs` — imports (`:26`), `sim_optimal_manifest` literal (`:170`), `parse_generalize_args` + `run_generalize` + `generalize_json` (new fns near the sweep helpers), dispatch arm (`:2993`), `USAGE` (`:2956`), parser unit tests +- Modify (thread `instrument: None`): every in-workspace `RunManifest { .. }` literal — enumerated set in Task 1, compiler-driven by `cargo check --workspace --all-targets` +- Test: `crates/aura-cli/tests/cli_run.rs` — gated-skip success E2E + two data-free must-fail E2Es + +**Out of scope (named in the spec):** reading-B cross-instrument *selection*; +per-instrument walk-forward; a persisted family-aggregate record; cross-symbol +R-pooling (#139 stays deferred); pip-metric generalization. `fieldtests/` is NOT +a workspace member (verified: workspace members are the 7 `crates/` only), so its +16 `RunManifest` literals are out of the compile-gate. + +--- + +## Task 1: `RunManifest.instrument` field + compat mirror + thread all literals + back-compat tests + +**Files:** +- Modify: `crates/aura-engine/src/report.rs:463-482` (struct) + the `#[cfg(test)] mod tests` (new serde tests) +- Modify: `crates/aura-registry/src/compat.rs:26-80` (mirror struct + destructure + literal — lockstep) +- Modify: every in-workspace `RunManifest { .. }` literal (enumerated below) + +> **RED-shape note for the implementer:** a serde-widening struct field is a +> *compile-gated additive* change — the back-compat test that asserts +> `m.instrument == None` / round-trips `Some` cannot exist until the field does, +> so the field and its pin land in one diff (the #144 `selection`-field +> precedent). The "RED" is the field's compile-absence, not a separate failing +> commit. Do not block as spec-ambiguous: add the field, thread the literals, +> add the tests, verify green. + +- [ ] **Step 1: Add the `instrument` field to `RunManifest`** + +In `crates/aura-engine/src/report.rs`, after the `selection` field (`:480-481`), +add: + +```rust + #[serde(default, skip_serializing_if = "Option::is_none")] + pub selection: Option, + /// The instrument this run evaluated, when it is a real-data run; `None` for a + /// synthetic run and for every pre-0078 line. The "instrument set as lineage" + /// (C18) for a cross-instrument family is each member's stamped symbol. Same + /// one-directional serde widening as `selection` (C14/C18). + #[serde(default, skip_serializing_if = "Option::is_none")] + pub instrument: Option, +} +``` + +- [ ] **Step 2: Thread the field through the compat mirror (lockstep)** + +In `crates/aura-registry/src/compat.rs`, the read-side mirror `RunManifestRead` +(`:27-35`), its destructure (`:67`), and the `From` literal +(`:69-76`) must all gain the field, or the registry will not compile. + +Mirror struct — after `selection` (`:33-34`): +```rust + #[serde(default)] + selection: Option, + #[serde(default)] + instrument: Option, +} +``` +Destructure (`:67`): +```rust + let RunManifestRead { commit, params, window, seed, broker, selection, instrument } = r.manifest; +``` +Literal (`:69-76`) — add `instrument,` after `selection,`: +```rust + manifest: RunManifest { + commit, + params: params.into_iter().map(|(name, v)| (name, v.0)).collect(), + window, + seed, + broker, + selection, + instrument, + }, +``` + +- [ ] **Step 3: Thread `instrument: None` into the production CLI literal** + +In `crates/aura-cli/src/main.rs`, `sim_optimal_manifest` (`:170-177`) — add +`instrument: None,` after `selection: None,`: +```rust + broker: format!("sim-optimal(pip_size={pip_size})"), + selection: None, + instrument: None, + } +``` + +- [ ] **Step 4: Thread `instrument: None` into every remaining in-workspace literal** + +A struct-field addition compile-breaks every `RunManifest { .. }` literal until +threaded. There is **no `Default` impl / builder** for `RunManifest` (verified), +so each literal must add the field explicitly. Add `instrument: None,` to each of +the literals below (the value is always `None` here — the only `Some(...)` stamp +is in Task 3's `run_generalize`). Line numbers are pre-edit anchors and will +drift as Step 1 shifts `report.rs`; the **authoritative enumerator is the +compiler** — run `cargo check --workspace --all-targets` and add the field to any +site it flags as missing `instrument`. + +Known in-workspace set (test-`src`, integration tests, examples): +- `crates/aura-engine/src/blueprint.rs:942` (test) +- `crates/aura-engine/src/mc.rs:239`, `:263` (test) +- `crates/aura-engine/src/report.rs:968`, `:1344`, `:1373`, `:1394` (test — the + pre-existing test literals; the new tests in Step 5 already include the field) +- `crates/aura-engine/src/sweep.rs:664` (test) +- `crates/aura-engine/src/walkforward.rs:288` (test) +- `crates/aura-engine/tests/random_sweep_e2e.rs:112` (integration test) +- `crates/aura-registry/src/lib.rs:474` (the `report_with` fixture — threading it + here covers every registry test that builds a report via `report_with`/ + `report_with_r`), `:813` (test) +- `crates/aura-registry/src/trace_store.rs:250` (test) +- `crates/aura-ingest/tests/ger40_breakout_world.rs:65`, `real_bars.rs:82`, + `streaming_seam.rs:82` (integration tests) +- `crates/aura-ingest/examples/ger40_breakout_sweep.rs:63`, + `ger40_breakout_walkforward.rs:73`, `shared/breakout_real.rs:390` (examples) + +Plus the two pre-existing `report.rs` test literals at the top of the test module +(`runmanifest_without_selection_serialises_without_the_key` `:771`, +`family_selection_round_trips_on_the_manifest` `:787`) — add `instrument: None,` +after their `selection:` field. + +- [ ] **Step 5: Add the back-compat serde tests** + +In `crates/aura-engine/src/report.rs`, inside `#[cfg(test)] mod tests` (beside the +`runmanifest_without_selection_*` tests near `:762`), add: + +```rust + #[test] + fn runmanifest_without_instrument_field_deserialises_to_none() { + // A pre-0078 line carries no `instrument` key; serde `default` -> None. + let json = r#"{"commit":"c","params":[],"window":[0,0],"seed":0,"broker":"b"}"#; + let m: RunManifest = serde_json::from_str(json).expect("legacy manifest loads"); + assert!(m.instrument.is_none()); + } + + #[test] + fn runmanifest_instrument_round_trips_and_omits_when_none() { + let with = RunManifest { + commit: "c".into(), params: vec![], window: (Timestamp(0), Timestamp(0)), + seed: 0, broker: "b".into(), selection: None, instrument: Some("GER40".into()), + }; + let json = serde_json::to_string(&with).unwrap(); + assert!(json.contains("\"instrument\":\"GER40\""), "json: {json}"); + let back: RunManifest = serde_json::from_str(&json).unwrap(); + assert_eq!(back.instrument, Some("GER40".to_string())); + + let without = RunManifest { + commit: "c".into(), params: vec![], window: (Timestamp(0), Timestamp(0)), + seed: 0, broker: "b".into(), selection: None, instrument: None, + }; + // skip_serializing_if omits the key when None -> existing manifest bytes unchanged (C23). + assert!(!serde_json::to_string(&without).unwrap().contains("instrument")); + } +``` + +- [ ] **Step 6: Verify the new tests pass and the workspace compiles** + +Run: `cargo test -p aura-engine runmanifest_` +Expected: PASS — `runmanifest_without_instrument_field_deserialises_to_none`, +`runmanifest_instrument_round_trips_and_omits_when_none`, and the pre-existing +`runmanifest_without_selection_*` all green (4+ tests run, 0 failed). + +- [ ] **Step 7: Verify every literal is threaded (workspace gate)** + +Run: `cargo check --workspace --all-targets` +Expected: clean (0 errors). Any "missing field `instrument`" error names a literal +Step 4 missed — add `instrument: None,` there and re-run. + +- [ ] **Step 8: Verify the whole workspace suite stays green** + +Run: `cargo test --workspace` +Expected: PASS (0 failed) — the threaded literals compile and every existing test, +including the registry compat-load tests and the #144/#145 serde tests, stays +green. + +--- + +## Task 2: aura-registry — `FamilyKind::CrossInstrument`, the `generalization` reduction, `check_r_metric`, error variants + +**Files:** +- Modify: `crates/aura-registry/src/lineage.rs:22-27` (`FamilyKind`) +- Modify: `crates/aura-registry/src/lib.rs:429-455` (struct + fns + error variants + Display) + `#[cfg(test)] mod tests` (unit tests) + +- [ ] **Step 1: Add the `CrossInstrument` family kind** + +In `crates/aura-registry/src/lineage.rs` (`:22-27`): +```rust +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub enum FamilyKind { + Sweep, + MonteCarlo, + WalkForward, + CrossInstrument, +} +``` +(No exhaustive `match` over `FamilyKind` exists workspace-wide — it serializes via +the derive — so this breaks no call site; the `runs families` display picks up +`"CrossInstrument"` automatically.) + +- [ ] **Step 2: Add the two `RegistryError` variants + their Display arms** + +In `crates/aura-registry/src/lib.rs`, the enum (`:432-440`): +```rust +#[derive(Debug)] +pub enum RegistryError { + /// An I/O error reading or writing the JSONL file. + Io(std::io::Error), + /// A stored line did not parse as a `RunReport` (1-based line number). + Parse { line: usize, source: serde_json::Error }, + /// `rank_by` was given a metric name it does not know. + UnknownMetric(String), + /// A known metric that is not R-based — cross-instrument generalization is + /// R-only (R is the only account/instrument-agnostic unit, C10). + NonRMetric(String), + /// A generalization was asked for with fewer than two instruments. + TooFewInstruments(usize), +} +``` +The Display match (`:444-453`) — add two arms before the closing brace: +```rust + RegistryError::UnknownMetric(m) => write!( + f, + "unknown metric '{m}' (known: total_pips, max_drawdown, bias_sign_flips, sqn, sqn_normalized, expectancy_r, net_expectancy_r)" + ), + RegistryError::NonRMetric(m) => write!( + f, + "metric '{m}' is not comparable across instruments; cross-instrument scoring is R-only (sqn, sqn_normalized, expectancy_r, net_expectancy_r)" + ), + RegistryError::TooFewInstruments(n) => write!( + f, + "a generalization needs >= 2 instruments, got {n}" + ), + } + } +} +``` + +- [ ] **Step 3: Write the failing unit tests for `generalization` + `check_r_metric`** + +In `crates/aura-registry/src/lib.rs`, inside `#[cfg(test)] mod tests` (the module +that already defines `report_with`/`report_with_r`), add. These reuse +`report_with_r(sqn, expectancy_r, net_expectancy_r)` (sets `metrics.r = Some` with +the given `expectancy_r`); the default metric is `expectancy_r`. + +```rust + #[test] + fn generalization_worst_case_is_the_floor_not_the_mean() { + let a = report_with_r(0.0, 0.40, 0.0); // +0.40 + let b = report_with_r(0.0, -0.20, 0.0); // -0.20 (the floor) + let c = report_with_r(0.0, 0.10, 0.0); // +0.10 + let pairs = vec![("A".to_string(), &a), ("B".to_string(), &b), ("C".to_string(), &c)]; + let g = generalization(&pairs, "expectancy_r").expect("R metric"); + assert_eq!(g.n_instruments, 3); + assert!((g.worst_case - (-0.20)).abs() < 1e-12, "worst_case must be the min, got {}", g.worst_case); + assert_eq!(g.sign_agreement, 2, "two instruments are net-positive"); + assert_eq!(g.per_instrument, vec![ + ("A".to_string(), 0.40), ("B".to_string(), -0.20), ("C".to_string(), 0.10), + ]); + } + + #[test] + fn generalization_is_not_lifted_by_a_strong_instrument() { + let a = report_with_r(0.0, 0.40, 0.0); + let b = report_with_r(0.0, -0.20, 0.0); + let strong = report_with_r(0.0, 99.0, 0.0); + let pairs = vec![("A".to_string(), &a), ("B".to_string(), &b), ("S".to_string(), &strong)]; + let g = generalization(&pairs, "expectancy_r").expect("R metric"); + assert!((g.worst_case - (-0.20)).abs() < 1e-12, "a strong instrument must not raise the floor"); + } + + #[test] + fn generalization_refuses_a_non_r_metric() { + let a = report_with_r(0.0, 0.4, 0.0); + let b = report_with_r(0.0, 0.2, 0.0); + let pairs = vec![("A".to_string(), &a), ("B".to_string(), &b)]; + match generalization(&pairs, "total_pips") { + Err(RegistryError::NonRMetric(m)) => assert_eq!(m, "total_pips"), + other => panic!("expected NonRMetric, got {other:?}"), + } + } + + #[test] + fn generalization_refuses_fewer_than_two_instruments() { + let a = report_with_r(0.0, 0.4, 0.0); + let pairs = vec![("A".to_string(), &a)]; + match generalization(&pairs, "expectancy_r") { + Err(RegistryError::TooFewInstruments(n)) => assert_eq!(n, 1), + other => panic!("expected TooFewInstruments(1), got {other:?}"), + } + } + + #[test] + fn generalization_is_deterministic() { + let a = report_with_r(0.0, 0.40, 0.0); + let b = report_with_r(0.0, -0.20, 0.0); + let pairs = vec![("A".to_string(), &a), ("B".to_string(), &b)]; + let g1 = generalization(&pairs, "expectancy_r").expect("R metric"); + let g2 = generalization(&pairs, "expectancy_r").expect("R metric"); + assert_eq!(g1, g2, "a pure fold must be deterministic (C1)"); + } + + #[test] + fn check_r_metric_accepts_r_and_refuses_pip() { + assert!(check_r_metric("expectancy_r").is_ok()); + assert!(check_r_metric("sqn_normalized").is_ok()); + match check_r_metric("total_pips") { + Err(RegistryError::NonRMetric(m)) => assert_eq!(m, "total_pips"), + other => panic!("expected NonRMetric, got {other:?}"), + } + match check_r_metric("nope") { + Err(RegistryError::UnknownMetric(m)) => assert_eq!(m, "nope"), + other => panic!("expected UnknownMetric, got {other:?}"), + } + } +``` + +- [ ] **Step 4: Run the new tests to verify they fail** + +Run: `cargo test -p aura-registry generalization` +Expected: FAIL to compile — `cannot find function `generalization` / `check_r_metric`` +and `cannot find type `Generalization`` (the symbols do not exist yet). This is the +RED state (compile-absence). + +- [ ] **Step 5: Implement `Generalization` + `generalization` + `check_r_metric`** + +In `crates/aura-registry/src/lib.rs`, after `optimize_deflated` ends (`:429`), add: + +```rust +/// The cross-instrument generalization aggregate (#146): a brought candidate's +/// per-instrument R-metric, reduced to a worst-case floor + a sign-agreement count +/// + the full breakdown. R-only (C10): `total_pips` is not comparable across +/// instruments. Pure, deterministic (C1) — a fold over `per_instrument` in input +/// order. `worst_case` is `min_i metric_i`; since every R-metric is +/// higher-is-better, the min is unconditionally the conservative floor (no +/// direction branch, unlike `PlateauMode::Worst`). +#[derive(Clone, Debug, PartialEq)] +pub struct Generalization { + pub selection_metric: String, + pub n_instruments: usize, + pub worst_case: f64, + pub sign_agreement: usize, + pub per_instrument: Vec<(String, f64)>, +} + +/// Validate that `metric` is a known, R-based ranking key — the data-free pre-check +/// the CLI runs before evaluating any instrument. The registry owns metric truth +/// (C9), so the CLI never duplicates the R-set. +pub fn check_r_metric(metric: &str) -> Result<(), RegistryError> { + let m = resolve_metric(metric)?; + if is_r_metric(m) { + Ok(()) + } else { + Err(RegistryError::NonRMetric(metric.to_string())) + } +} + +/// Grade how consistently one candidate holds across instruments: read the chosen +/// R-metric from each per-instrument report and reduce to the worst-case floor + +/// the sign-agreement count + the per-instrument breakdown. R-only and `>= 2` +/// instruments (the metric check precedes the arity check so a bad metric is +/// reported even with an empty slice). Additive — reads, never re-ranks (C23). +pub fn generalization( + per_instrument: &[(String, &RunReport)], + metric: &str, +) -> Result { + let m = resolve_metric(metric)?; + if !is_r_metric(m) { + return Err(RegistryError::NonRMetric(metric.to_string())); + } + if per_instrument.len() < 2 { + return Err(RegistryError::TooFewInstruments(per_instrument.len())); + } + let vals: Vec<(String, f64)> = per_instrument + .iter() + .map(|(label, rep)| (label.clone(), metric_value(rep, m))) + .collect(); + let worst_case = vals.iter().map(|(_, v)| *v).fold(f64::INFINITY, f64::min); + let sign_agreement = vals.iter().filter(|(_, v)| *v > 0.0).count(); + Ok(Generalization { + selection_metric: metric.to_string(), + n_instruments: vals.len(), + worst_case, + sign_agreement, + per_instrument: vals, + }) +} +``` + +- [ ] **Step 6: Run the new tests to verify they pass** + +Run: `cargo test -p aura-registry generalization` +Expected: PASS — the five `generalization_*` tests green. + +Run: `cargo test -p aura-registry check_r_metric` +Expected: PASS — `check_r_metric_accepts_r_and_refuses_pip` green. + +- [ ] **Step 7: Verify the registry suite stays green (preservation gate)** + +Run: `cargo test -p aura-registry` +Expected: PASS (0 failed) — the #144/#145 selector tests and the compat-load tests +all stay green. + +--- + +## Task 3: aura-cli — the `generalize` subcommand + +**Files:** +- Modify: `crates/aura-cli/src/main.rs` — imports (`:26-30`), new fns, dispatch arm (`:2993`), `USAGE` (`:2956`), parser unit tests +- Test: `crates/aura-cli/tests/cli_run.rs` — E2Es + +- [ ] **Step 1: Import the new registry items** + +In `crates/aura-cli/src/main.rs`, extend the `use aura_registry::{ ... }` block +(`:26-30`) to add `generalization, Generalization, check_r_metric`: +```rust +use aura_registry::{ + check_r_metric, generalization, group_families, mc_member_reports, optimize_deflated, + optimize_plateau, rank_by, sweep_member_reports, walkforward_member_reports, FamilyKind, + FamilyMember, Generalization, NameKind, PlateauMode, Registry, RunTraces, TraceStore, WriteKind, +}; +``` + +- [ ] **Step 2: Write the failing parser unit tests** + +In `crates/aura-cli/src/main.rs`, inside `#[cfg(test)] mod tests` (where +`parse_sweep_args` tests live), add: + +```rust + #[test] + fn parse_generalize_requires_a_single_value_candidate() { + // a candidate is one cell: a multi-value grid flag is refused + assert!(parse_generalize_args(&[ + "--real", "GER40,USDJPY", "--fast", "2,3", "--slow", "12", + "--stop-length", "14", "--stop-k", "2.0", + ]).is_err()); + } + + #[test] + fn parse_generalize_requires_all_four_knobs() { + // --slow absent -> refuse (no defaulting to the multi-value sweep grid) + assert!(parse_generalize_args(&[ + "--real", "GER40,USDJPY", "--fast", "3", + "--stop-length", "14", "--stop-k", "2.0", + ]).is_err()); + } + + #[test] + fn parse_generalize_refuses_fewer_than_two_instruments() { + assert!(parse_generalize_args(&[ + "--real", "GER40", "--fast", "3", "--slow", "12", + "--stop-length", "14", "--stop-k", "2.0", + ]).is_err()); + } + + #[test] + fn parse_generalize_refuses_a_duplicate_instrument() { + assert!(parse_generalize_args(&[ + "--real", "GER40,GER40", "--fast", "3", "--slow", "12", + "--stop-length", "14", "--stop-k", "2.0", + ]).is_err()); + } + + #[test] + fn parse_generalize_refuses_a_non_stage1r_strategy() { + assert!(parse_generalize_args(&[ + "--strategy", "sma", "--real", "GER40,USDJPY", "--fast", "3", "--slow", "12", + "--stop-length", "14", "--stop-k", "2.0", + ]).is_err()); + } + + #[test] + fn parse_generalize_defaults_name_and_metric() { + let (name, symbols, grid, metric, _from, _to) = parse_generalize_args(&[ + "--real", "GER40,USDJPY", "--fast", "3", "--slow", "12", + "--stop-length", "14", "--stop-k", "2.0", + ]).expect("valid generalize args"); + assert_eq!(name, "generalize"); + assert_eq!(metric, "expectancy_r"); + assert_eq!(symbols, vec!["GER40".to_string(), "USDJPY".to_string()]); + assert_eq!(grid.fast, vec![3]); + assert_eq!(grid.slow, vec![12]); + assert_eq!(grid.stop_length, vec![14]); + assert_eq!(grid.stop_k, vec![2.0]); + } +``` + +- [ ] **Step 3: Run the parser tests to verify they fail** + +Run: `cargo test -p aura-cli parse_generalize` +Expected: FAIL to compile — `cannot find function `parse_generalize_args`` +(does not exist yet). RED state. + +- [ ] **Step 4: Implement `parse_generalize_args`** + +In `crates/aura-cli/src/main.rs`, near `parse_sweep_args` (`:1622`), add. It mirrors +`parse_sweep_args` but: defaults strategy to stage1-r and refuses any other; +`--real` is a comma-list of `>= 2` distinct non-empty symbols; the four grid knobs +are each required and single-valued; `--metric` defaults to `expectancy_r`; +`--name` defaults to `generalize`. + +```rust +/// Parse the `generalize` tail: +/// `[--strategy stage1-r] --real --fast --slow --stop-length +/// --stop-k [--from ] [--to ] [--metric ] [--name ]`. +/// The candidate is a single cell: each grid knob takes exactly one value and all +/// four are required. `--real` is a comma list of >= 2 distinct non-empty symbols. +/// Pure (no I/O / exit) so the grammar is unit-testable. +fn parse_generalize_args( + rest: &[&str], +) -> Result<(String, Vec, Stage1RGrid, String, Option, Option), String> { + let usage = || "generalize [--strategy stage1-r] --real --fast --slow --stop-length --stop-k [--from ] [--to ] [--metric ] [--name ]".to_string(); + let mut symbols: Option> = None; + let mut from_ms: Option = None; + let mut to_ms: Option = None; + let mut fast: Option = None; + let mut slow: Option = None; + let mut stop_length: Option = None; + let mut stop_k: Option = None; + let mut metric = "expectancy_r".to_string(); + let mut name = "generalize".to_string(); + // a single grid knob, required and single-valued (csv with exactly one item) + let one = |value: &str, usage: &dyn Fn() -> String| -> Result, String> { + let items: Vec<&str> = value.split(',').collect(); + if items.len() != 1 || items[0].is_empty() { return Err(usage()); } + Ok(items) + }; + let mut tail = rest; + while let Some((flag, t)) = tail.split_first() { + let (value, t) = t.split_first().ok_or_else(usage)?; + match *flag { + "--strategy" => { if *value != "stage1-r" { return Err(usage()); } } + "--real" => { + let parts: Vec = value.split(',').map(|s| s.to_string()).collect(); + if parts.iter().any(|s| s.is_empty()) { return Err(usage()); } + symbols = Some(parts); + } + "--from" => from_ms = Some(value.parse().map_err(|_| usage())?), + "--to" => to_ms = Some(value.parse().map_err(|_| usage())?), + "--fast" => { one(value, &usage)?; fast = Some(value.parse().map_err(|_| usage())?); } + "--slow" => { one(value, &usage)?; slow = Some(value.parse().map_err(|_| usage())?); } + "--stop-length" => { one(value, &usage)?; stop_length = Some(value.parse().map_err(|_| usage())?); } + "--stop-k" => { one(value, &usage)?; stop_k = Some(value.parse().map_err(|_| usage())?); } + "--metric" => metric = (*value).to_string(), + "--name" => name = (*value).to_string(), + _ => return Err(usage()), + } + tail = t; + } + let symbols = symbols.ok_or_else(usage)?; + if symbols.len() < 2 { return Err(usage()); } + // distinct: a duplicate would double-count one instrument in the floor / sign count + let mut seen = std::collections::HashSet::new(); + if !symbols.iter().all(|s| seen.insert(s.clone())) { return Err(usage()); } + let grid = Stage1RGrid { + fast: vec![fast.ok_or_else(usage)?], + slow: vec![slow.ok_or_else(usage)?], + stop_length: vec![stop_length.ok_or_else(usage)?], + stop_k: vec![stop_k.ok_or_else(usage)?], + ..Stage1RGrid::default() + }; + Ok((name, symbols, grid, metric, from_ms, to_ms)) +} +``` + +- [ ] **Step 5: Run the parser tests to verify they pass** + +Run: `cargo test -p aura-cli parse_generalize` +Expected: PASS — all six `parse_generalize_*` tests green. + +- [ ] **Step 6: Implement `generalize_json` and `run_generalize`** + +In `crates/aura-cli/src/main.rs`, near `walkforward_summary_json` (`:1969`), add the +JSON builder (mirrors the `{"walkforward": obj}` shape; renders `selection_metric` +as `"metric"`): + +```rust +fn generalize_json(agg: &Generalization) -> String { + let per: Vec = agg + .per_instrument + .iter() + .map(|(sym, v)| serde_json::json!([sym, v])) + .collect(); + let obj = serde_json::json!({ + "metric": agg.selection_metric, + "n_instruments": agg.n_instruments, + "worst_case": agg.worst_case, + "sign_agreement": agg.sign_agreement, + "per_instrument": per, + }); + serde_json::json!({ "generalize": obj }).to_string() +} +``` + +And the run handler (near `run_sweep`, `:1750`). It pre-checks the metric +(data-free refusal), runs the single-cell candidate per instrument, stamps the +instrument, reduces, prints, and persists a `CrossInstrument` family: + +```rust +fn run_generalize( + name: &str, + symbols: &[String], + grid: &Stage1RGrid, + metric: &str, + from_ms: Option, + to_ms: Option, +) { + // data-free metric pre-check: refuse a non-R / unknown metric before any run. + if let Err(e) = check_r_metric(metric) { + eprintln!("aura: {e}"); + std::process::exit(2); + } + let mut members: Vec = Vec::new(); + for symbol in symbols { + let choice = DataChoice::Real { symbol: symbol.clone(), from_ms, to_ms }; + let data = DataSource::from_choice(choice); // per-instrument pip_or_refuse + has_symbol + let family = stage1_r_sweep_family(None, &data, grid); // single-cell grid -> 1 member + let mut report = family.points[0].report.clone(); + report.manifest.instrument = Some(symbol.clone()); + members.push(report); + } + let pairs: Vec<(String, &RunReport)> = symbols.iter().cloned().zip(members.iter()).collect(); + let agg = match generalization(&pairs, metric) { + Ok(a) => a, + Err(e) => { eprintln!("aura: {e}"); std::process::exit(2); } + }; + println!("{}", generalize_json(&agg)); + let reg = default_registry(); + match reg.append_family(name, FamilyKind::CrossInstrument, &members) { + Ok(id) => println!("{{\"family_id\":\"{id}\"}}"), + Err(e) => { eprintln!("aura: failed to persist family: {e}"); std::process::exit(1); } + } +} +``` + +- [ ] **Step 7: Wire the dispatch arm and extend USAGE** + +In `crates/aura-cli/src/main.rs`, add a `["generalize", rest @ ..]` arm beside +`["sweep", ..]` (`:2993`): +```rust + ["generalize", rest @ ..] => match parse_generalize_args(rest) { + Ok((name, symbols, grid, metric, from_ms, to_ms)) => { + run_generalize(&name, &symbols, &grid, &metric, from_ms, to_ms) + } + Err(msg) => { + eprintln!("aura: {msg}"); + std::process::exit(2); + } + }, +``` +Append the form to the `USAGE` const (`:2956`), before the closing `aura runs ...`: +``` + | aura generalize [--strategy stage1-r] --real --fast --slow --stop-length --stop-k [--from ] [--to ] [--metric ] [--name ] +``` + +- [ ] **Step 8: Write the E2E tests** + +In `crates/aura-cli/tests/cli_run.rs`, add (model the gated-skip on +`run_real_sidecar_index_pip_reaches_the_emitted_manifest`, `:250`): + +```rust +/// Property: `aura generalize` grades a single stage1-r candidate across two +/// instruments — its stdout carries the per-instrument breakdown + the worst-case +/// floor + the sign-agreement, then the persisted CrossInstrument family handle. +/// Gated on local GER40/USDJPY data (the shared Sept-2024 window), skips cleanly +/// when the archive is absent. +#[test] +fn generalize_grades_a_candidate_across_two_instruments() { + const FROM_MS: &str = "1725148800000"; + const TO_MS: &str = "1727740799999"; + let cwd = temp_cwd("generalize-two"); + let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) + .args([ + "generalize", "--strategy", "stage1-r", "--real", "GER40,USDJPY", + "--fast", "3", "--slow", "12", "--stop-length", "14", "--stop-k", "2.0", + "--from", FROM_MS, "--to", TO_MS, + ]) + .current_dir(&cwd) + .output() + .expect("spawn aura"); + if out.status.code() == Some(2) { + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + stderr.contains("no local data") || stderr.contains("no recorded geometry"), + "exit 2 must be a data refusal, got: {stderr}" + ); + eprintln!("skip: no local GER40/USDJPY data"); + return; + } + assert_eq!(out.status.code(), Some(0), "exit: {:?}", out.status); + let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout"); + assert!(stdout.contains("\"generalize\":"), "aggregate object: {stdout}"); + assert!(stdout.contains("\"n_instruments\":2"), "two instruments: {stdout}"); + assert!(stdout.contains("\"worst_case\":"), "worst_case present: {stdout}"); + assert!(stdout.contains("\"sign_agreement\":"), "sign_agreement present: {stdout}"); + assert!(stdout.contains("GER40") && stdout.contains("USDJPY"), "per-instrument breakdown: {stdout}"); + assert!(stdout.contains("\"family_id\":\"generalize-0\""), "family handle: {stdout}"); +} + +/// Property: a single-instrument generalize is refused at parse time (exit 2), +/// before any data access — so it asserts the arity refusal on any machine. +#[test] +fn generalize_refuses_a_single_instrument() { + let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) + .args([ + "generalize", "--strategy", "stage1-r", "--real", "GER40", + "--fast", "3", "--slow", "12", "--stop-length", "14", "--stop-k", "2.0", + ]) + .output() + .expect("spawn aura"); + assert_eq!(out.status.code(), Some(2), "single instrument must exit 2"); +} + +/// Property: a non-R metric is refused before any instrument runs (the data-free +/// `check_r_metric` pre-check), so this asserts the R-only refusal on any machine. +#[test] +fn generalize_refuses_a_non_r_metric() { + let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura")) + .args([ + "generalize", "--strategy", "stage1-r", "--real", "GER40,USDJPY", + "--fast", "3", "--slow", "12", "--stop-length", "14", "--stop-k", "2.0", + "--metric", "total_pips", + ]) + .output() + .expect("spawn aura"); + assert_eq!(out.status.code(), Some(2), "a non-R metric must exit 2"); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!(stderr.contains("R-only"), "must name the R-only refusal, got: {stderr}"); +} +``` + +> `temp_cwd(name) -> PathBuf` (cli_run.rs:13) is the confirmed registry-isolation +> helper (a fresh per-test cwd, so the family-store run index starts at 0 → +> `generalize-0`); `env!("CARGO_BIN_EXE_aura")` is the binary handle (the same the +> real-data E2Es at cli_run.rs:256 use). Both already exist in the file. + +- [ ] **Step 9: Run the E2E tests** + +Run: `cargo test -p aura-cli generalize` +Expected: PASS — `generalize_refuses_a_single_instrument` and +`generalize_refuses_a_non_r_metric` assert exit 2 (data-free); +`generalize_grades_a_candidate_across_two_instruments` passes on a machine with +GER40/USDJPY data or skips cleanly otherwise; plus the six `parse_generalize_*` +unit tests green. + +- [ ] **Step 10: Verify the CLI suite stays green (C23 preservation)** + +Run: `cargo test -p aura-cli` +Expected: PASS (0 failed) — every existing sweep/walkforward/mc/run/`runs` test +stays green; the existing manifest goldens are byte-identical (instrument is +omitted when None — `skip_serializing_if`), so no sweep/walkforward/standalone +output changed. + +- [ ] **Step 11: Final workspace gate** + +Run: `cargo test --workspace` +Expected: PASS (0 failed). + +Run: `cargo clippy --workspace --all-targets -- -D warnings` +Expected: clean (0 warnings).