From e5c87a51732899127f1bbaaa1b779c73b1cb2f91 Mon Sep 17 00:00:00 2001 From: Brummel Date: Wed, 24 Jun 2026 17:16:37 +0200 Subject: [PATCH] spec: 0067 r-sweep-followups (boss-signed) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two settled, additive follow-ups to the cycle-0065/0066 R-sweep surface, named together by the user under /boss. Signed on the grounding-check PASS (9 load-bearing current-behaviour assumptions, each ratified by a named green test). Part A (refs #130) — n-normalized SQN (SQN100): a new opt-in ranking metric sqn_normalized = (mean_R/stdev_R)·√(min(n,100)), turnover-robust across sweep members with different trade counts. Additive RMetrics field (serde(default) for C18 back-compat) + a new Metric::SqnNormalized rank key mirroring Sqn. Raw sqn and the default ranker are unchanged; the 0066 single-run golden is re-baselined (the r: block gains one field — intended, not drift). Part B (refs #135) — wire --trace for the stage1-r sweep: stage1_r_sweep_family honours --trace by persisting each member's equity/exposure/r_equity under runs/traces/// via the existing persist_traces_r, mirroring momentum_sweep_family; the interim refusal guard in run_sweep is dropped. Makes --trace symmetric across all three sweep strategies. Derived fork decisions logged on #130 and #135 (reconciliation comments). --- docs/specs/0067-r-sweep-followups.md | 284 +++++++++++++++++++++++++++ 1 file changed, 284 insertions(+) create mode 100644 docs/specs/0067-r-sweep-followups.md diff --git a/docs/specs/0067-r-sweep-followups.md b/docs/specs/0067-r-sweep-followups.md new file mode 100644 index 0000000..b83f939 --- /dev/null +++ b/docs/specs/0067-r-sweep-followups.md @@ -0,0 +1,284 @@ +# R-sweep follow-ups (SQN100 + stage1-r --trace) — Design Spec + +**Date:** 2026-06-24 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +## Goal + +Two settled, additive follow-ups to the cycle-0065/0066 R-sweep surface, named +together by the user. Neither changes existing behaviour by default; both extend +an already-shipped surface. + +- **Part A (#130) — n-normalized SQN (SQN100).** The raw `sqn` (Van Tharp + `√n · mean(R) / stdev(R)`) scales with the raw trade count, so a member that + merely trades more earns a higher SQN partly for turnover. Add an + n-normalized variant `sqn_normalized = (mean/stdev) · √(min(n, 100))` as a new + **opt-in** ranking objective, comparable across sweep members with different + trade counts. +- **Part B (#135) — wire `--trace` for the stage1-r sweep.** Cycle 0066 added + `aura sweep --strategy stage1-r` but refused `--trace` for it (explicit + `exit 2`), because a stage1-r member has more taps than the sma/momentum + members. Persist each member's `equity` / `exposure` / `r_equity` under + `runs/traces///` by reusing the existing `persist_traces_r`, + making `--trace` symmetric across all three sweep strategies and letting a + swept member be charted. + +The two parts are independent in code (Part A: `aura-engine` + `aura-registry`; +Part B: `aura-cli`) and share only the R-sweep theme. + +## Architecture + +**Part A** threads a new scalar through the existing R-metric reduction and the +existing rank vocabulary, touching three layers, all additively: + +1. `RMetrics` gains one `f64` field, `sqn_normalized`, carrying `#[serde(default)]` + so a pre-0067 `r:` block (serialized without the field) still deserializes to + `0.0` (C18 back-compat; the registry reads `runs.jsonl`). +2. `summarize_r` computes it next to `sqn`, under the same dispersion guard, with + a named module constant `SQN_CAP = 100`. +3. `aura-registry` learns a new `Metric::SqnNormalized` variant — one enum arm, + one name-resolution arm, one comparator arm — mirroring the existing `Sqn` + key exactly (higher-is-better, `total_cmp`, `r: None → NEG_INFINITY`). The + `optimize`/`rank_by`/`runs family rank` paths pick it up for free, since they + all route through `metric_cmp`. + +The raw `sqn` is untouched and keeps the meaning of the metric name `"sqn"`. The +default ranker is unchanged; `sqn_normalized` is selected explicitly. This +preserves the 0066 byte-compat surface except for the single intended additive +change: every serialized `r:` block now carries one extra field, so the 0066 +single-run golden is re-baselined (see Testing strategy). + +**Part B** makes `stage1_r_sweep_family` honour its `trace` argument exactly as +`momentum_sweep_family` already honours its own, and removes the interim refusal +guard. The stage1-r member persists three taps (`equity`, `exposure`, +`r_equity`) via the existing `persist_traces_r` — the same persister the +single-run path (`run_stage1_r`) already uses — keyed by the generic +`member_key` over the varying axes (`fast.length`, `slow.length`). The fixed +R-defining params (`stop_length` / `stop_k` / `bias_scale`) remain recorded in +the manifest but are excluded from `member_key` (they do not vary), exactly as +the 0066 manifest-completeness fix arranged. + +## Concrete code shapes + +### User-facing surface (the empirical evidence for the acceptance criterion) + +**Part A — rank a stage1-r family by the turnover-robust objective:** + +```console +$ aura sweep --strategy stage1-r --name fam1 +... (writes a rankable family to runs/runs.jsonl) +$ aura runs family rank sqn_normalized +... members printed best-first by SQN100 (turnover-robust); ties stable +``` + +`sqn_normalized` joins `sqn` / `expectancy_r` / `net_expectancy_r` as a metric +name the family ranker accepts. An unknown name still errors, now listing +`sqn_normalized` among the known metrics: + +```console +$ aura runs family rank sharpe +aura: unknown metric 'sharpe' (known: total_pips, max_drawdown, bias_sign_flips, sqn, sqn_normalized, expectancy_r, net_expectancy_r) +``` + +**Part B — trace, then chart, a swept stage1-r member:** + +```console +$ aura sweep --strategy stage1-r --trace fam2 +... (writes per-member trace dirs, one per grid point) +$ aura chart fam2/fast.length-2_slow.length-6 --tap r_equity +... renders the member's R-equity curve +``` + +Previously this refused with `aura: --trace is not yet supported for --strategy +stage1-r ...` and `exit 2`. After this cycle it writes +`runs/traces/fam2//` with the `equity` / `exposure` / `r_equity` +taps, symmetric with `aura sweep --strategy momentum --trace`. + +### Before → after implementation shapes (secondary) + +**`RMetrics` (crates/aura-engine/src/report.rs) — add one field:** + +```rust +// before +pub struct RMetrics { + // ... existing fields ... + pub sqn: f64, // √n · mean_R / sample-stdev_R; n<2 or zero-variance -> 0.0 + pub net_expectancy_r: f64, + pub conviction_terciles_r: [f64; 3], +} + +// after +pub struct RMetrics { + // ... existing fields ... + pub sqn: f64, // √n · mean_R / sample-stdev_R; n<2 or zero-variance -> 0.0 + /// (mean_R / sample-stdev_R) · √(min(n, SQN_CAP)): turnover-robust SQN + /// (Van Tharp's capped "SQN score", CAP=100). n<2 or zero-variance -> 0.0. + #[serde(default)] + pub sqn_normalized: f64, + pub net_expectancy_r: f64, + pub conviction_terciles_r: [f64; 3], +} +``` + +**`summarize_r` (same file) — compute it under the same guard as `sqn`:** + +```rust +const SQN_CAP: u64 = 100; // Van Tharp's conventional trade-count cap for the SQN score + +// after the existing `let sqn = if n < 2 { 0.0 } else { ... sd ... };` block: +let sqn_normalized = if n < 2 { + 0.0 +} else { + // recompute (or reuse) the sample stdev; 0.0 when variance is zero + let var = rs.iter().map(|&r| (r - mean).powi(2)).sum::() / (n as f64 - 1.0); + let sd = var.sqrt(); + if sd > 0.0 { (n.min(SQN_CAP) as f64).sqrt() * mean / sd } else { 0.0 } +}; +``` + +The empty-ledger early return adds `sqn_normalized: 0.0`. (The planner decides +whether to factor the shared `sd` computation; the behaviour is fixed here.) + +**`aura-registry` (crates/aura-registry/src/lib.rs) — mirror the `Sqn` key:** + +```rust +// enum Metric { ... Sqn, [+ SqnNormalized,] ExpectancyR, NetExpectancyR } + +// name resolution: +"sqn" => Metric::Sqn, +"sqn_normalized" => Metric::SqnNormalized, // new + +// comparator (higher-is-better, total_cmp, r: None -> NEG_INFINITY): +Metric::Sqn => r_get(b, |r| r.sqn).total_cmp(&r_get(a, |r| r.sqn)), +Metric::SqnNormalized => + r_get(b, |r| r.sqn_normalized).total_cmp(&r_get(a, |r| r.sqn_normalized)), // new + +// UnknownMetric Display known-list gains "sqn_normalized"; +// the metric_cmp doc-comment "the three R keys" becomes "the four R keys". +``` + +**`stage1_r_sweep_family` (crates/aura-cli/src/main.rs) — honour `trace`, +mirroring `momentum_sweep_family`:** + +```rust +// before: fn stage1_r_sweep_family(_trace: Option<&str>, data: &DataSource) -> SweepFamily +// - sweep closure drops tx_req into `_rx_req`; no member_key; no persist. + +// after: fn stage1_r_sweep_family(trace: Option<&str>, data: &DataSource) -> SweepFamily +let binder = bp.axis("fast.length", [2, 3]).axis("slow.length", [6, 12]); +let varying: HashSet = binder.varying_axes().into_iter().collect(); +binder.sweep(|point| { + // ... bootstrap + run ... + let req_rows: Vec<(Timestamp, Vec)> = rx_req.try_iter().collect(); // now drained + // ... named (incl. the fixed R-params) + manifest as today ... + let key = member_key(&named, &varying); + if let Some(name) = trace { + persist_traces_r(&format!("{name}/{key}"), &manifest, &eq_rows, &ex_rows, &req_rows); + } + // ... metrics + RunReport as today ... +}) +``` + +**`run_sweep` (same file) — drop the refusal guard:** + +```rust +// before: +// if persist && strategy == Strategy::Stage1R { +// eprintln!("aura: --trace is not yet supported for --strategy stage1-r ..."); +// std::process::exit(2); +// } +// after: (guard removed; the existing ensure_name_free check and the +// `Strategy::Stage1R => stage1_r_sweep_family(persist.then_some(name), &data)` +// dispatch now carry the persist flag through unchanged.) +``` + +## Components + +- **`aura-engine::report` (`RMetrics`, `summarize_r`).** Part A's field + + computation + the `SQN_CAP` constant. Pure (C1); no I/O. +- **`aura-registry` (`Metric`, `metric_cmp`, `RegistryError::UnknownMetric`).** + Part A's new rank key. The single source of "best" for `rank_by` / `optimize` / + the CLI `runs family rank`. +- **`aura-cli` (`stage1_r_sweep_family`, `run_sweep`, `persist_traces_r`).** + Part B's trace wiring. `persist_traces_r` is reused unchanged; only its caller + set grows by one (the swept member, joining the single run). + +## Data flow + +- **Part A:** dense `PositionManagement` record → `summarize_r` → + `RMetrics { ..., sqn_normalized }` → `RunReport.metrics.r` → + `runs.jsonl` line → `metric_cmp("sqn_normalized")` → best-first order. +- **Part B:** swept stage1-r member run → `(eq_rows, ex_rows, req_rows)` → + `persist_traces_r("/", manifest, eq, ex, req)` → + `runs/traces///{equity,exposure,r_equity}` → `aura chart`. + +## Error handling + +- **Part A:** no new error paths. `sqn_normalized` is total over its inputs + (degenerate cases → `0.0`, like `sqn`). A `r: None` member ranks last under + `sqn_normalized` via the existing `NEG_INFINITY` injection. An unknown metric + name is the existing `RegistryError::UnknownMetric` (its message grows by one + entry). +- **Part B:** the removed guard's job (no silent no-op) is now done by actually + persisting. `persist_traces_r` keeps its existing failure path (`eprintln!` + + `exit 2` on a `TraceStore::write` error). The `ensure_name_free` collision + check in `run_sweep` already covered all strategies and now also guards the + stage1-r `--trace` name, exactly as for sma/momentum. + +## Testing strategy + +RED-first for the new behaviour. + +**Part A (`crates/aura-engine/src/report.rs` unit tests):** +- `sqn_normalized` caps on a hand-built ledger: a ledger of `n > 100` trades with + known `mean`/`sd` yields `√100 · mean/sd` (= `10 · mean/sd`), strictly below the + raw `sqn` (`√n · mean/sd`) for the same ledger. +- `sqn_normalized` equals `sqn` for `n ≤ 100` (the cap is inactive). +- empty ledger → `sqn_normalized == 0.0`. + +**Part A (`crates/aura-registry/src/lib.rs` unit tests):** +- `rank_by("sqn_normalized")` orders members best-first (descending). +- the `UnknownMetric` message lists `sqn_normalized` (extend the existing + `unknown_metric_message_lists_r_metrics` assertion). + +**Part A (re-baseline):** the 0066 golden +`crates/aura-cli/tests/cli_run.rs::stage1_r_single_run_output_golden` pins the +`metrics` JSON tail; it gains `sqn_normalized` in the `r:` block and is re-pinned. +This is an intended additive output change, not drift. + +**Part B (`crates/aura-cli/tests/cli_run.rs`):** +- invert the existing `sweep_strategy_stage1_r_trace_is_refused_name_works` into a + test asserting `aura sweep --strategy stage1-r --trace ` exits `0` and writes + per-member dirs `runs/traces///` each containing the `r_equity` + tap (mirror the momentum `*_trace_persists_*` pattern named in the issue). +- the `--name` path for stage1-r still records the family (unchanged). + +Build `cargo build --workspace`; test `cargo test --workspace` (one positional +filter substring per invocation); lint `cargo clippy --workspace --all-targets -- +-D warnings`. + +## Acceptance criteria + +Applying the project's feature-acceptance criterion (a feature ships when the +intended audience naturally reaches for it, it measurably improves the surface, +and it reintroduces no failure class a core invariant forbids): + +- **Audience reach (shown above):** a researcher comparing sweep members with + different trade counts reaches for a turnover-robust objective — + `aura runs family rank sqn_normalized` is exactly that, and the + refused-then-charted `--trace` flow restores the obvious per-member inspection + for the third sweep strategy. +- **Measurable improvement:** `sqn_normalized` removes the turnover bias from the + single-number objective; `--trace` removes an asymmetry (two of three sweep + strategies could persist members, one could not). +- **No reintroduced failure class:** both changes are additive and total. Part A + preserves C18 back-compat via `serde(default)` and leaves the default ranker / + raw `sqn` byte-unchanged (only the intended new field appears, re-pinned in the + golden). Part B reuses the audited `persist_traces_r` and the generic + `member_key`, introduces no new I/O path, and keeps determinism (C1). + +**Out of scope:** a variable/parameterized cap (fixed `SQN_CAP = 100` by the +recorded decision); changing the default ranking metric; persisting the dense +R-record itself as a trace tap (only the three curves, as the single run does); +the n-normalization of any non-R metric.