plan: 0077 plateau over peak selection

Executable plan for cycle 0077 (#145), decomposed into 5 tasks with the
compile-gates sequenced:

1. Reshape FamilySelection/SelectionMode across 3 crates (mechanical,
   byte-preserving): deflation fields -> Option, add PlateauMean/PlateauWorst
   + neighbourhood_score/n_neighbours; thread optimize_deflated + all tests +
   the runs_family display (argmax bytes unchanged).
2. GridSpace::axis_lens() + SweepBinder::sweep_with_lattice (sweep delegates,
   every existing .sweep() caller byte-unchanged).
3. optimize_plateau + PlateauMode + closed_neighbourhood (mixed-radix grid
   neighbourhood, argmax of the smoothed surface) + a higher_is_better refactor
   so direction lives in one place.
4. CLI --select parse + Selection threading + sweep-entry Option<lattice>
   tuples + select_winner (the RandomSpace-refuse guard) + walkforward_family
   dispatch; all signature-change callers threaded before the build gate.
5. runs-family plateau E2E golden.

Two plan-time forks were derived and recorded on #145 (comment 1974): the
lattice surfaces at the engine terminal (only the post-resolve_axes GridSpace
holds param_space order), and the RandomSpace-refuse is a structural guard
unit-tested at select_winner (walkforward has no --random producer to E2E).

refs #145
This commit is contained in:
2026-06-26 15:54:57 +02:00
parent 45b8e2b28b
commit fd221c81ec
@@ -0,0 +1,974 @@
# Prefer a robust parameter plateau over the in-sample peak — Implementation Plan
> **Parent spec:** `docs/specs/0077-plateau-over-peak-selection.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to
> run this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** Add an opt-in `optimize_plateau` selection objective (argmax over the
grid-neighbourhood-smoothed metric) reachable via a new `--select` flag, recorded
on the `RunManifest.selection` carrier #144 introduced; default `argmax` stays
byte-identical (C23).
**Architecture:** A new family selector lives beside `optimize_deflated` in
`aura-registry` (C9), reading each member's metric via the shared `metric_value`
and smoothing it over a mixed-radix grid neighbourhood built from `axis_lens`
(the `GridSpace` odometer radixes). The lattice is surfaced from the engine via a
new `SweepBinder::sweep_with_lattice` terminal (`sweep` delegates to it,
byte-unchanged for all existing callers). The shared `FamilySelection` carrier is
reshaped — orthogonal rule (`SelectionMode`) × annotation (deflation fields →
`Option`, plus plateau fields) — and the CLI threads a `Selection` from
`--select` into `walkforward_family`.
**Tech Stack:** `aura-engine` (`report.rs` carrier, `sweep.rs` accessor,
`blueprint.rs` terminal), `aura-registry` (`lib.rs` selector + index math),
`aura-cli` (`main.rs` flag/dispatch/display, `tests/cli_run.rs` E2E).
---
## Design decisions resolved at plan time (recorded on #145, comment 1974)
1. **Lattice seam → engine `SweepBinder::sweep_with_lattice`.** Only the engine's
post-`resolve_axes` `GridSpace` holds the lattice in `param_space()`/odometer
order, which `closed_neighbourhood`'s mixed-radix arithmetic requires; a sibling
terminal (`sweep` delegates) contains the blast radius — every existing
`.sweep()` caller stays byte-identical.
2. **RandomSpace-refuse → structural guard, no new feature.** `walkforward` has no
`--random` flag (verified: `--random`/`RandomSpace`/`RandomBinder` appear
nowhere in `aura-cli`); wiring a random walk-forward producer is unspecified,
out-of-scope. So `sweep_over`/`stage1_r_sweep_over` return
`(SweepFamily, Option<Vec<usize>>)` (`Some` today, grid-only), and a
`select_winner` helper returns the refuse on `Plateau + None`-lattice; the
caller prints the spec's message and exits 2. **Unit-tested** at the helper
level (the `Err` path), not via a `walkforward --random` E2E.
3. **`FamilySelection.seed``Option<u64>`** (spec line 135; the recon carrier's
`Option<usize>` was a transcription slip).
## Files this plan creates or modifies
- Modify: `crates/aura-engine/src/report.rs:41-66` — reshape `SelectionMode` +
`FamilySelection`; `:762-775` round-trip test literal; new plateau-shape test.
- Modify: `crates/aura-engine/src/sweep.rs` — new `GridSpace::axis_lens()` + test.
- Modify: `crates/aura-engine/src/blueprint.rs:399-411` — new
`SweepBinder::sweep_with_lattice`; `sweep` delegates; new test.
- Modify: `crates/aura-registry/src/lib.rs:308-312``optimize_deflated`
construction to `Option` deflation fields; `:163-173` `metric_cmp` refactor to
`higher_is_better`; `:744-794` test assertions → `.unwrap()`/`Some`; add
`PlateauMode`, `higher_is_better`, `closed_neighbourhood`, `optimize_plateau` +
tests.
- Modify: `crates/aura-cli/src/main.rs` — import `SelectionMode` (engine),
`optimize_plateau`/`PlateauMode` (registry); `Selection` enum; `parse_select` +
`--select` in `parse_walkforward_args`; `select_winner`; `sweep_over` /
`stage1_r_sweep_over` tuple returns; `walkforward_family` `+select` dispatch;
`run_walkforward` `+select`; the dispatch call site; `runs_family` mode-aware
display; 4 test callers of `walkforward_family`; unit tests.
- Test: `crates/aura-cli/tests/cli_run.rs``walkforward_plateau_select_stamps_plateau_provenance`.
- Unchanged (preservation gate): `crates/aura-registry/src/compat.rs:34` embeds
`FamilySelection` by value — the reshape flows through with no edit; the
legacy-load test `load_lifts_a_selection_block_through_the_compat_mirror`
(`lib.rs:520`) and `runs_family_rank_shows_deflated_line` (`cli_run.rs:2358`)
must stay green.
---
## Task 1: Reshape `FamilySelection` / `SelectionMode` (mechanical, byte-preserving across 3 crates)
**Files:**
- Modify: `crates/aura-engine/src/report.rs:41-66`, `:762-775`
- Modify: `crates/aura-registry/src/lib.rs:308-312`, `:744-794`
- Modify: `crates/aura-cli/src/main.rs:19-25` (import), `:2215-2223` (display)
This is the cross-crate compile-gate. The reshape breaks every construction and
field-read site; all are threaded here. No new runtime behaviour — the argmax
path stays byte-identical; the one new test pins the new fields' serde contract.
- [ ] **Step 1: Write the failing plateau-shape serde test (RED)**
In `crates/aura-engine/src/report.rs`, in the `#[cfg(test)] mod tests` block,
beside `family_selection_round_trips_on_the_manifest` (after line 775), add:
```rust
#[test]
fn family_selection_plateau_shape_round_trips() {
let sel = FamilySelection {
selection_metric: "sqn_normalized".into(), n_trials: 16, raw_winner_metric: 1.42,
mode: SelectionMode::PlateauMean,
deflated_score: None, overfit_probability: None,
n_resamples: None, block_len: None, seed: None,
neighbourhood_score: Some(1.27), n_neighbours: Some(5),
};
let json = serde_json::to_string(&sel).unwrap();
// plateau annotation present; deflation fields omitted (skip_serializing_if)
assert!(json.contains("\"neighbourhood_score\":1.27"), "json: {json}");
assert!(json.contains("\"n_neighbours\":5"), "json: {json}");
assert!(!json.contains("deflated_score"), "deflation fields omitted under plateau: {json}");
assert!(!json.contains("n_resamples"), "json: {json}");
let back: FamilySelection = serde_json::from_str(&json).unwrap();
assert_eq!(back, sel);
}
```
- [ ] **Step 2: Run the new test to verify it fails to compile**
Run: `cargo test -p aura-engine family_selection_plateau`
Expected: FAIL — compile error (`SelectionMode::PlateauMean` / the `neighbourhood_score`/`n_neighbours` fields do not exist yet).
- [ ] **Step 3: Reshape `SelectionMode` and `FamilySelection`**
In `crates/aura-engine/src/report.rs`, replace the `SelectionMode` enum + doc
(lines 41-48) and the `FamilySelection` struct + doc (lines 50-66) with:
```rust
/// Which selection objective produced the record (additive provenance, C23).
/// `Argmax` is the bare-best pick (cycle 0076), deflated for the number of trials.
/// `PlateauMean` / `PlateauWorst` (cycle 0077) argmax the neighbourhood-smoothed
/// surface instead, so peak-vs-plateau runs stay distinguishable on this field.
#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
pub enum SelectionMode {
Argmax,
PlateauMean,
PlateauWorst,
}
/// Selection-provenance for a sweep winner. The selection RULE (`mode`) and its
/// ANNOTATION are orthogonal: `Argmax` carries the trials-deflation annotation
/// (`deflated_score` / `overfit_probability` / `n_resamples` / `block_len` /
/// `seed`); `Plateau*` carries the smoothing annotation (`neighbourhood_score` /
/// `n_neighbours`). Each annotation is `Option`, present iff its rule produced the
/// record; all are additive — recorded, never re-ranking (C23). A legacy line
/// (pre-0077) carries the deflation scalars as bare values that deserialize to
/// `Some` (serde default), so it still loads (C14/C18).
#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
pub struct FamilySelection {
pub selection_metric: String,
pub n_trials: usize,
pub raw_winner_metric: f64,
pub mode: SelectionMode,
// deflation annotation (present iff mode == Argmax with a deflation run)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub deflated_score: Option<f64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub overfit_probability: Option<f64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub n_resamples: Option<usize>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub block_len: Option<usize>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub seed: Option<u64>,
// plateau annotation (present iff mode is Plateau*)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub neighbourhood_score: Option<f64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub n_neighbours: Option<usize>,
}
```
- [ ] **Step 4: Update the round-trip test literal**
In `crates/aura-engine/src/report.rs`, in `family_selection_round_trips_on_the_manifest`,
replace the `let sel = FamilySelection { … };` literal (lines 764-768) with:
```rust
let sel = FamilySelection {
selection_metric: "sqn_normalized".into(), n_trials: 4, raw_winner_metric: 1.83,
mode: SelectionMode::Argmax,
deflated_score: Some(0.21), overfit_probability: Some(0.06),
n_resamples: Some(1000), block_len: Some(5), seed: Some(42),
neighbourhood_score: None, n_neighbours: None,
};
```
- [ ] **Step 5: Run the engine tests (GREEN, engine internally consistent)**
Run: `cargo test -p aura-engine family_selection`
Expected: PASS — both `family_selection_round_trips_on_the_manifest` and
`family_selection_plateau_shape_round_trips` green. (aura-engine does not depend
on aura-registry, so its build is consistent even though the registry is not yet
updated.)
- [ ] **Step 6: Update `optimize_deflated`'s construction to the Option fields**
In `crates/aura-registry/src/lib.rs`, replace the returned `FamilySelection`
literal (lines 308-312) with:
```rust
Ok((winner.clone(), FamilySelection {
selection_metric: metric.to_string(), n_trials: k, raw_winner_metric: raw,
mode: SelectionMode::Argmax,
deflated_score: Some(deflated_score), overfit_probability,
n_resamples: Some(n_resamples), block_len: Some(block_len), seed: Some(seed),
neighbourhood_score: None, n_neighbours: None,
}))
```
- [ ] **Step 7: Update the `optimize_deflated_*` test assertions to `.unwrap()` / `Some`**
In `crates/aura-registry/src/lib.rs`, apply these exact line edits:
- Line 749: ` assert!(sel.deflated_score <= sel.raw_winner_metric); // dispersion floor ≤ raw`
` assert!(sel.deflated_score.unwrap() <= sel.raw_winner_metric); // dispersion floor ≤ raw`
- Line 762: ` assert_eq!(sel.n_resamples, 0);`
` assert_eq!(sel.n_resamples, Some(0));`
- Line 763: ` assert_eq!(sel.deflated_score, sel.raw_winner_metric, "p95 of an empty null is 0");`
` assert_eq!(sel.deflated_score.unwrap(), sel.raw_winner_metric, "p95 of an empty null is 0");`
- Line 782: ` assert!(sel.deflated_score.is_finite(), "deflated_score must be finite, got {}", sel.deflated_score);`
` assert!(sel.deflated_score.unwrap().is_finite(), "deflated_score must be finite, got {:?}", sel.deflated_score);`
- Line 783: ` assert_eq!(sel.deflated_score, sel.raw_winner_metric);`
` assert_eq!(sel.deflated_score.unwrap(), sel.raw_winner_metric);`
- Line 794: ` assert!(sel.deflated_score > 0.0);`
` assert!(sel.deflated_score.unwrap() > 0.0);`
(The compat test `load_lifts_a_selection_block_through_the_compat_mirror` at line
520-527 is **unchanged** — its legacy JSON line carries the deflation scalars as
bare floats that deserialize to `Some` under the new `Option` + `#[serde(default)]`,
and its assertions read only `n_trials`/`mode`. It is the C14/C18 legacy-load gate.)
- [ ] **Step 8: Run the registry tests (GREEN)**
Run: `cargo test -p aura-registry optimize_deflated`
Expected: PASS — all nine `optimize_deflated_*` tests green against the reshaped fields.
Run: `cargo test -p aura-registry load_lifts`
Expected: PASS — the legacy `selection` block still lifts through the compat mirror.
- [ ] **Step 9: Import `SelectionMode` and rewrite the `runs_family` display (mode-aware, argmax byte-preserved)**
In `crates/aura-cli/src/main.rs`, add `SelectionMode` to the `aura_engine` import
list (lines 19-24) — e.g. extend the `RunManifest, RunMetrics,` line to
`RunManifest, RunMetrics, SelectionMode,` (any slot in the brace list is fine).
Then replace the selection-display block in `runs_family` (lines 2215-2223):
```rust
for report in &ordered {
println!("{}", report.to_json());
if let Some(sel) = &report.manifest.selection {
match sel.mode {
SelectionMode::Argmax => match sel.overfit_probability {
Some(p) => println!(" deflated={:.4} P(overfit)={:.4}", sel.deflated_score.unwrap(), p),
None => println!(" deflated={:.4}", sel.deflated_score.unwrap()),
},
SelectionMode::PlateauMean | SelectionMode::PlateauWorst => {
let label = if matches!(sel.mode, SelectionMode::PlateauMean) { "mean" } else { "worst" };
if let (Some(score), Some(n)) = (sel.neighbourhood_score, sel.n_neighbours) {
println!(" plateau({label})={score:.4} over {n} cells");
}
}
}
}
}
```
(Under `Argmax`, `deflated_score` is always `Some` by construction, so `.unwrap()`
preserves the exact pre-0077 bytes — the `deflated=…`/`P(overfit)=…` golden stays
green. The `Plateau*` branch is exercised by the Task 5 E2E.)
- [ ] **Step 10: Workspace build + full per-crate regression gate**
Run: `cargo build --workspace`
Expected: clean (0 errors) — all three crates consistent.
Run: `cargo test -p aura-engine`
Expected: PASS (no regressions).
Run: `cargo test -p aura-registry`
Expected: PASS (no regressions).
---
## Task 2: `GridSpace::axis_lens()` + `SweepBinder::sweep_with_lattice` (engine, additive)
**Files:**
- Modify: `crates/aura-engine/src/sweep.rs` (accessor + test)
- Modify: `crates/aura-engine/src/blueprint.rs:399-411` (terminal + delegate + test)
- [ ] **Step 1: Write the failing `axis_lens` golden (RED)**
In `crates/aura-engine/src/sweep.rs`, in the `#[cfg(test)] mod tests` block (which
already imports `aura_core::{Cell, ParamSpec, Scalar, ScalarKind, Timestamp}`),
add:
```rust
#[test]
fn grid_axis_lens_are_the_per_axis_radixes() {
let space = vec![
ParamSpec { name: "a".into(), kind: ScalarKind::I64 },
ParamSpec { name: "b".into(), kind: ScalarKind::I64 },
];
let grid = GridSpace::new(&space, vec![
vec![Scalar::i64(10), Scalar::i64(20)], // axis 0: 2 values
vec![Scalar::i64(1), Scalar::i64(2), Scalar::i64(3)], // axis 1: 3 values
]).expect("2x3 grid");
assert_eq!(grid.axis_lens(), vec![2, 3]);
assert_eq!(grid.axis_lens().iter().product::<usize>(), grid.len());
}
```
- [ ] **Step 2: Run to verify it fails to compile**
Run: `cargo test -p aura-engine grid_axis_lens`
Expected: FAIL — `no method named axis_lens found for struct GridSpace`.
- [ ] **Step 3: Add the `axis_lens` accessor**
In `crates/aura-engine/src/sweep.rs`, in the `impl GridSpace` block, immediately
after `is_empty` (after line 65), add:
```rust
/// Per-axis cardinalities in `param_space()` order (the odometer radixes,
/// last-axis-fastest). `∏ axis_lens() == len()`. The lattice shape a plateau
/// neighbourhood walks (cycle 0077).
pub fn axis_lens(&self) -> Vec<usize> {
self.axes.iter().map(Vec::len).collect()
}
```
- [ ] **Step 4: Run the accessor test (GREEN)**
Run: `cargo test -p aura-engine grid_axis_lens`
Expected: PASS.
- [ ] **Step 5: Write the failing `sweep_with_lattice` test (RED)**
In `crates/aura-engine/src/blueprint.rs`, in the `#[cfg(test)] mod tests` block
(which has `composite_sma_cross_harness()` and `run_point`), add:
```rust
#[test]
fn sweep_with_lattice_surfaces_grid_radixes_in_param_space_order() {
let bp = composite_sma_cross_harness().0;
let (fam, lattice) = bp
.axis("sma_cross.fast.length", vec![Scalar::i64(2), Scalar::i64(3)]) // 2 values
.axis("sma_cross.slow.length", vec![Scalar::i64(4), Scalar::i64(5)]) // 2 values
.axis("bias.scale", vec![Scalar::f64(0.5)]) // 1 value
.sweep_with_lattice(run_point)
.expect("named binding resolves and runs");
assert_eq!(lattice, vec![2, 2, 1], "radixes in param_space slot order");
assert_eq!(lattice.iter().product::<usize>(), fam.points.len()); // 4
}
```
- [ ] **Step 6: Run to verify it fails to compile**
Run: `cargo test -p aura-engine sweep_with_lattice`
Expected: FAIL — `no method named sweep_with_lattice found`.
- [ ] **Step 7: Add `sweep_with_lattice`; refactor `sweep` to delegate**
In `crates/aura-engine/src/blueprint.rs`, replace the `SweepBinder::sweep` method
(lines 399-411) with:
```rust
/// Resolve the named axes against `param_space()` into a positional grid and
/// run the disjoint sweep. `sweep` is [`SweepBinder::sweep_with_lattice`] with
/// the lattice dropped, so every existing caller is byte-unchanged.
pub fn sweep<F>(self, run_one: F) -> Result<SweepFamily, BindError>
where
F: Fn(&[Cell]) -> RunReport + Sync,
{
self.sweep_with_lattice(run_one).map(|(family, _lattice)| family)
}
/// As [`SweepBinder::sweep`], plus the grid's per-axis radixes (`axis_lens`,
/// in `param_space()` / odometer order). The lattice is what a plateau
/// neighbourhood walk needs (cycle 0077); only the engine's post-`resolve_axes`
/// grid holds it in the correct order.
pub fn sweep_with_lattice<F>(self, run_one: F) -> Result<(SweepFamily, Vec<usize>), BindError>
where
F: Fn(&[Cell]) -> RunReport + Sync,
{
let space = self.bp.param_space();
check_param_namespace_injective(&space).map_err(BindError::Compile)?;
let ordered = resolve_axes(&space, &self.axes)?;
let grid = GridSpace::new(&space, ordered)
.expect("named layer pre-validates arity/kind/non-empty");
let lattice = grid.axis_lens();
Ok((sweep(&grid, run_one), lattice))
}
```
- [ ] **Step 8: Run the new test + the existing sweep tests (GREEN, delegation preserved)**
Run: `cargo test -p aura-engine sweep_with_lattice`
Expected: PASS.
Run: `cargo test -p aura-engine`
Expected: PASS — every existing `.sweep()` test (named-axes parity, grid goldens)
stays green; `sweep` delegating to `sweep_with_lattice` is behaviour-preserving.
---
## Task 3: `optimize_plateau` + `PlateauMode` + `closed_neighbourhood` (registry, additive)
**Files:**
- Modify: `crates/aura-registry/src/lib.rs:163-173` (`metric_cmp``higher_is_better`),
plus new `PlateauMode`, `higher_is_better`, `closed_neighbourhood`,
`optimize_plateau` and their tests.
- [ ] **Step 1: Write the failing plateau tests (RED)**
In `crates/aura-registry/src/lib.rs`, in the `#[cfg(test)] mod tests` block (which
has the `member(total_pips, trade_rs)` helper), add:
```rust
/// A 1-D grid (7 cells, odometer order) with two end spikes and a broad middle
/// plateau in `total_pips`. Bare argmax takes an end spike; the plateau-mean
/// argmax takes the interior plateau centre.
fn fixture_grid_spike_vs_plateau() -> SweepFamily {
let pips = [80.0, 5.0, 48.0, 50.0, 49.0, 5.0, 80.0];
SweepFamily { space: vec![], points: pips.iter().map(|&p| member(p, vec![1.0])).collect() }
}
#[test]
fn closed_neighbourhood_2x3_golden() {
// 2x3 lattice, odometer (last axis fastest):
// (0,0)=0 (0,1)=1 (0,2)=2
// (1,0)=3 (1,1)=4 (1,2)=5
let sorted = |i: usize| { let mut v = closed_neighbourhood(i, &[2, 3]); v.sort_unstable(); v };
assert_eq!(sorted(0), vec![0, 1, 3]); // corner (0,0): self + 2
assert_eq!(sorted(4), vec![1, 3, 4, 5]); // interior (1,1): self + 3
assert_eq!(sorted(2), vec![1, 2, 5]); // corner (0,2): self + 2
}
#[test]
fn plateau_picks_the_plateau_not_the_spike() {
let fam = fixture_grid_spike_vs_plateau();
let spike = optimize(&fam, "total_pips").unwrap();
let (centre, sel) = optimize_plateau(&fam, &[7], "total_pips", PlateauMode::Mean).unwrap();
assert_ne!(centre.report.metrics.total_pips, spike.report.metrics.total_pips);
assert_eq!(centre.report.metrics.total_pips, 50.0, "plateau centre is the middle cell");
assert_eq!(sel.mode, SelectionMode::PlateauMean);
assert_eq!(sel.raw_winner_metric, 50.0);
assert!(sel.neighbourhood_score.unwrap() < spike.report.metrics.total_pips,
"smoothed score is below the spike's raw peak");
assert_eq!(sel.n_neighbours, Some(3), "interior cell has self + 2 neighbours");
assert!(sel.deflated_score.is_none() && sel.n_resamples.is_none(),
"deflation fields omitted under plateau");
}
#[test]
fn plateau_worst_is_more_conservative_than_mean() {
let fam = fixture_grid_spike_vs_plateau();
let (_, mean_sel) = optimize_plateau(&fam, &[7], "total_pips", PlateauMode::Mean).unwrap();
let (_, worst_sel) = optimize_plateau(&fam, &[7], "total_pips", PlateauMode::Worst).unwrap();
// Worst scores each cell by its weakest neighbour, never above the mean.
assert!(worst_sel.neighbourhood_score.unwrap() <= mean_sel.neighbourhood_score.unwrap());
assert_eq!(worst_sel.mode, SelectionMode::PlateauWorst);
assert!(worst_sel.n_neighbours.unwrap() >= 2, "winner is an interior cell");
}
#[test]
fn plateau_single_member_equals_optimize() {
let fam = SweepFamily { space: vec![], points: vec![member(42.0, vec![1.0])] };
let plain = optimize(&fam, "total_pips").unwrap();
let (winner, sel) = optimize_plateau(&fam, &[1], "total_pips", PlateauMode::Mean).unwrap();
assert_eq!(winner.report.metrics.total_pips, plain.report.metrics.total_pips);
assert_eq!(sel.n_neighbours, Some(1), "a 1-cell grid's closed neighbourhood is itself");
assert_eq!(sel.neighbourhood_score, Some(42.0));
assert_eq!(sel.raw_winner_metric, 42.0);
}
#[test]
fn optimize_plateau_is_deterministic() {
let fam = fixture_grid_spike_vs_plateau();
let a = optimize_plateau(&fam, &[7], "total_pips", PlateauMode::Worst).unwrap().1;
let b = optimize_plateau(&fam, &[7], "total_pips", PlateauMode::Worst).unwrap().1;
assert_eq!(a, b);
}
```
- [ ] **Step 2: Run to verify they fail to compile**
Run: `cargo test -p aura-registry closed_neighbourhood`
Expected: FAIL — `closed_neighbourhood` / `optimize_plateau` / `PlateauMode` do not exist.
- [ ] **Step 3: Add `higher_is_better` and refactor `metric_cmp` to use it**
In `crates/aura-registry/src/lib.rs`, immediately after `is_r_metric` (after line
207), add:
```rust
/// The metric's optimisation direction: `true` if a larger value is better
/// (`total_pips`, the four R keys), `false` for the lower-is-better keys
/// (`max_drawdown`, `bias_sign_flips`). The single direction source — `metric_cmp`
/// (best-first ordering) and `optimize_plateau` (smoothed argmax + worst-neighbour)
/// both read it, so the per-metric sense lives in exactly one place.
fn higher_is_better(m: Metric) -> bool {
!matches!(m, Metric::MaxDrawdown | Metric::BiasSignFlips)
}
```
Then replace the `metric_cmp` body (lines 163-173) with:
```rust
fn metric_cmp(metric: &str) -> Result<impl Fn(&RunReport, &RunReport) -> Ordering, RegistryError> {
let m = resolve_metric(metric)?;
let hib = higher_is_better(m);
Ok(move |a: &RunReport, b: &RunReport| {
let (va, vb) = (metric_value(a, m), metric_value(b, m));
// higher-is-better ranks the larger value first; lower-is-better the smaller.
if hib { vb.total_cmp(&va) } else { va.total_cmp(&vb) }
})
}
```
(Behaviour-identical: `rank_by` / `optimize` and their existing tests are the gate.)
- [ ] **Step 4: Add `closed_neighbourhood`**
In `crates/aura-registry/src/lib.rs`, after `higher_is_better`, add:
```rust
/// The closed grid neighbourhood of flat index `i` over a mixed-radix lattice
/// (`axis_lens`, last axis fastest — the `GridSpace` odometer convention): `{i}`
/// plus each in-range ±1-per-axis cell. Pure index math, deterministic. The order
/// is `i` first then per-axis neighbours; callers use the result only as a set and
/// for its length, so the order is not load-bearing.
fn closed_neighbourhood(i: usize, axis_lens: &[usize]) -> Vec<usize> {
// decompose i to per-axis coords (last axis fastest)
let mut coords = vec![0usize; axis_lens.len()];
let mut rem = i;
for k in (0..axis_lens.len()).rev() {
coords[k] = rem % axis_lens[k];
rem /= axis_lens[k];
}
// recompose a coord vector to its flat index (Horner, last axis fastest)
let recompose = |c: &[usize]| c.iter().zip(axis_lens).fold(0usize, |flat, (&ck, &len)| flat * len + ck);
let mut out = vec![i];
for k in 0..axis_lens.len() {
for delta in [-1isize, 1] {
let ck = coords[k] as isize + delta;
if ck < 0 || ck as usize >= axis_lens[k] { continue; }
let mut nb = coords.clone();
nb[k] = ck as usize;
out.push(recompose(&nb));
}
}
out
}
```
- [ ] **Step 5: Add `PlateauMode` and `optimize_plateau`**
In `crates/aura-registry/src/lib.rs`, immediately before `optimize_deflated` (before
line 258, before its doc comment), add:
```rust
/// Plateau-selection aggregate: how a member's grid-neighbourhood metric is
/// reduced to one smoothed score before the argmax. `Mean` averages the closed
/// neighbourhood; `Worst` takes the most-pessimistic neighbour by the metric's
/// direction (biasing toward interior cells, which keep more neighbours).
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum PlateauMode {
Mean,
Worst,
}
/// `optimize`'s argmax over the NEIGHBOURHOOD-SMOOTHED surface, plus its plateau
/// provenance. Each member scores as the mean (or worst-case) of its closed grid
/// neighbourhood's `metric_value`; the winner is the best smoothed score by the
/// metric's own direction (earliest-odometer tie, as `optimize`). `axis_lens` are
/// the grid radixes (`param_space()` order, last axis fastest). Pure,
/// deterministic (C1). The returned `SweepPoint` is the winning grid CELL;
/// `raw_winner_metric` is that cell's own metric, `neighbourhood_score` the
/// smoothed value the argmax maximised.
pub fn optimize_plateau(
family: &SweepFamily, axis_lens: &[usize], metric: &str, mode: PlateauMode,
) -> Result<(SweepPoint, FamilySelection), RegistryError> {
let m = resolve_metric(metric)?;
let n = family.points.len();
debug_assert_eq!(
axis_lens.iter().product::<usize>(), n,
"axis_lens product must equal the family size (a valid grid lattice)",
);
let hib = higher_is_better(m);
// smoothed score + closed-neighbourhood size per member, in odometer order
let scored: Vec<(usize, f64, usize)> = (0..n)
.map(|i| {
let nbrs = closed_neighbourhood(i, axis_lens);
let vals: Vec<f64> = nbrs.iter().map(|&j| metric_value(&family.points[j].report, m)).collect();
let score = match mode {
PlateauMode::Mean => vals.iter().sum::<f64>() / vals.len() as f64,
PlateauMode::Worst => {
if hib {
vals.iter().copied().fold(f64::INFINITY, f64::min)
} else {
vals.iter().copied().fold(f64::NEG_INFINITY, f64::max)
}
}
};
(i, score, nbrs.len())
})
.collect();
// argmax the smoothed surface by direction; earliest-odometer tie (only a
// strictly-better later cell displaces the incumbent, as `optimize`).
let &(wi, wscore, wn) = scored
.iter()
.reduce(|best, cur| {
let better = if hib { cur.1 > best.1 } else { cur.1 < best.1 };
if better { cur } else { best }
})
.expect("a SweepFamily is non-empty by construction (EmptyAxis is rejected upstream)");
let winner = family.points[wi].clone();
let raw = metric_value(&winner.report, m);
Ok((
winner,
FamilySelection {
selection_metric: metric.to_string(),
n_trials: n,
raw_winner_metric: raw,
mode: match mode {
PlateauMode::Mean => SelectionMode::PlateauMean,
PlateauMode::Worst => SelectionMode::PlateauWorst,
},
deflated_score: None,
overfit_probability: None,
n_resamples: None,
block_len: None,
seed: None,
neighbourhood_score: Some(wscore),
n_neighbours: Some(wn),
},
))
}
```
- [ ] **Step 6: Run the plateau tests + the existing rank/optimize gate (GREEN)**
Run: `cargo test -p aura-registry plateau`
Expected: PASS — `plateau_picks_the_plateau_not_the_spike`,
`plateau_worst_is_more_conservative_than_mean`, `plateau_single_member_equals_optimize`,
`optimize_plateau_is_deterministic` all green.
Run: `cargo test -p aura-registry closed_neighbourhood`
Expected: PASS.
Run: `cargo test -p aura-registry`
Expected: PASS — `rank_by` / `optimize` / `optimize_deflated` stay green after the
`metric_cmp``higher_is_better` refactor.
---
## Task 4: CLI `--select` parse + threading + sweep-entry tuples + `select_winner` (compile-gate)
**Files:**
- Modify: `crates/aura-cli/src/main.rs``aura_registry` import; `Selection` enum;
`parse_select`; `parse_walkforward_args` (`+--select`, 6-tuple); the dispatch
call site (`:2935`); `run_walkforward` (`+select`); `walkforward_family`
(`+select` dispatch via `select_winner`); `sweep_over` / `stage1_r_sweep_over`
(tuple returns); `select_winner`; the 4 test callers of `walkforward_family`;
unit tests.
This is the CLI compile-gate: the signature changes break their callers, all
threaded here. Default `Argmax` keeps the existing walk-forward output
byte-identical (C23).
- [ ] **Step 1: Write the failing `select_winner` refuse + parse tests (RED)**
In `crates/aura-cli/src/main.rs`, in the `#[cfg(test)] mod tests` block at line
2965, add:
```rust
#[test]
fn select_winner_refuses_plateau_without_a_lattice() {
// A plateau request with no lattice (a random sweep would yield None) is
// refused, never silently argmaxed. The refuse short-circuits before the
// family is read, so an empty family is fine here.
let fam = SweepFamily { space: vec![], points: vec![] };
let err = select_winner(&fam, "total_pips", Selection::Plateau(PlateauMode::Mean), None)
.unwrap_err();
assert!(err.contains("requires a grid sweep"), "refuse message: {err}");
}
#[test]
fn parse_walkforward_select_flag() {
let argmax = parse_walkforward_args(&["--strategy", "stage1-r"]).unwrap();
assert!(matches!(argmax.5, Selection::Argmax), "default is argmax");
let mean = parse_walkforward_args(&["--select", "plateau:mean"]).unwrap();
assert!(matches!(mean.5, Selection::Plateau(PlateauMode::Mean)));
let worst = parse_walkforward_args(&["--select", "plateau:worst"]).unwrap();
assert!(matches!(worst.5, Selection::Plateau(PlateauMode::Worst)));
assert!(parse_walkforward_args(&["--select", "bogus"]).is_err(),
"unknown --select token is a usage error");
}
```
- [ ] **Step 2: Run to verify they fail to compile**
Run: `cargo test -p aura-cli select_winner_refuses`
Expected: FAIL — `Selection` / `PlateauMode` / `select_winner` not in scope; the
`parse_walkforward_args` tuple has no `.5`.
- [ ] **Step 3: Imports + add the `Selection` enum and `parse_select`**
In `crates/aura-cli/src/main.rs`, extend the `aura_registry` import (lines 26-30) —
add `optimize_plateau, PlateauMode,` to the brace list (alongside `optimize_deflated`).
Also add `FamilySelection` to the `aura_engine` import list (lines 19-24; it is not
currently imported, and `select_winner` in Step 4 names it). (`SelectionMode` was
already added to that list in Task 1 Step 9.)
Then, immediately after the `Strategy` enum (after line 1551's enum block; place it
near the other CLI enums), add:
```rust
/// In-sample winner-selection objective for walk-forward (cycle 0077). `Argmax` is
/// the bare-best pick deflated for trials (#144, the default); `Plateau` argmaxes
/// the neighbourhood-smoothed surface instead (opt-in via `--select`).
#[derive(Clone, Copy)]
enum Selection {
Argmax,
Plateau(PlateauMode),
}
/// Parse a `--select` token: `argmax` | `plateau:mean` | `plateau:worst`. Unknown
/// tokens are a usage error (the caller maps `Err(())` to exit 2).
fn parse_select(s: &str) -> Result<Selection, ()> {
match s {
"argmax" => Ok(Selection::Argmax),
"plateau:mean" => Ok(Selection::Plateau(PlateauMode::Mean)),
"plateau:worst" => Ok(Selection::Plateau(PlateauMode::Worst)),
_ => Err(()),
}
}
```
- [ ] **Step 4: Add the `select_winner` dispatch helper**
In `crates/aura-cli/src/main.rs`, immediately before `walkforward_family` (before
line 1786), add:
```rust
/// Resolve the in-sample winner under the chosen selection objective. `Argmax`
/// defers to the trials-deflation pick (#144). `Plateau` argmaxes the smoothed grid
/// surface — it needs the grid lattice, so a sweep with no lattice (a future random
/// walk-forward producer) is refused rather than silently argmaxed. The metric is
/// always known at the call sites, so a metric error is unreachable (`expect`); the
/// only fallible outcome is the plateau-without-lattice refusal, returned as
/// `Err(message)` for the caller to print and exit 2.
fn select_winner(
family: &SweepFamily, metric: &str, select: Selection, lattice: Option<&[usize]>,
) -> Result<(SweepPoint, FamilySelection), String> {
match select {
Selection::Argmax => Ok(optimize_deflated(
family, metric, DEFLATION_N_RESAMPLES, DEFLATION_BLOCK_LEN, DEFLATION_SEED,
).expect("walk-forward metrics are known")),
Selection::Plateau(mode) => match lattice {
Some(lens) => Ok(optimize_plateau(family, lens, metric, mode)
.expect("walk-forward metrics are known")),
None => Err(
"--select plateau requires a grid sweep; a random sweep has no parameter lattice"
.to_string(),
),
},
}
}
```
(`select_winner` names `FamilySelection`, which is **not** currently imported in
`main.rs` — Step 3 above adds it to the `aura_engine` import list.)
- [ ] **Step 5: Switch `sweep_over` / `stage1_r_sweep_over` to return `(SweepFamily, Option<Vec<usize>>)`**
In `crates/aura-cli/src/main.rs`:
`stage1_r_sweep_over` (line 1301): change the signature return type
`-> SweepFamily` to `-> (SweepFamily, Option<Vec<usize>>)`; change its terminal
`.sweep(|point| {` (line 1318) to `.sweep_with_lattice(|point| {`, and insert a
`.map(...)` line before the existing trailing `.expect(...)` (line 1342, currently
` .expect("the stage1-r named grid matches the stage1-r param-space")`). The
closure's closing ` })` and the existing `.expect(...)` line both stay; the
new `.map` goes between them, so the function tail reads:
```rust
})
.map(|(fam, lat)| (fam, Some(lat)))
.expect("the stage1-r named grid matches the stage1-r param-space")
```
`sweep_over` (line 1841): change the signature return type `-> SweepFamily` to
`-> (SweepFamily, Option<Vec<usize>>)`; change its terminal `.sweep(|point| {`
(line 1854) to `.sweep_with_lattice(|point| {`, and insert the `.map(...)` line
before the existing trailing
` .expect("the built-in named grid matches the sample param-space")` (line
1869), so the function tail reads:
```rust
})
.map(|(fam, lat)| (fam, Some(lat)))
.expect("the built-in named grid matches the sample param-space")
```
- [ ] **Step 6: Thread `Selection` through `walkforward_family` (signature + both arms)**
In `crates/aura-cli/src/main.rs`, change `walkforward_family`'s signature (line
1786-1788) to add the `select` parameter:
```rust
fn walkforward_family(
strategy: Strategy, trace: Option<&str>, data: &DataSource, grid: &Stage1RGrid,
select: Selection,
) -> WalkForwardResult {
```
In the `Strategy::SmaCross` arm, replace the sweep + optimize lines (1802-1807):
```rust
let (is_family, lattice) = sweep_over(w.is.0, w.is.1, data);
let (best, selection) = match select_winner(&is_family, "total_pips", select, lattice.as_deref()) {
Ok(v) => v,
Err(msg) => { eprintln!("aura: {msg}"); std::process::exit(2); }
};
let (oos_equity, mut oos_report) = run_oos(&best.params, w.oos.0, w.oos.1, trace, data);
oos_report.manifest.selection = Some(selection);
```
In the `Strategy::Stage1R` arm, replace the sweep + optimize lines (1820-1825):
```rust
let (is_family, lattice) = stage1_r_sweep_over(w.is.0, w.is.1, data, grid);
let (best, selection) = match select_winner(&is_family, "sqn_normalized", select, lattice.as_deref()) {
Ok(v) => v,
Err(msg) => { eprintln!("aura: {msg}"); std::process::exit(2); }
};
let (oos_equity, mut oos_report) = run_oos_r(&best.params, w.oos.0, w.oos.1, trace, data);
oos_report.manifest.selection = Some(selection);
```
- [ ] **Step 7: Thread the new `select` argument through every `walkforward_family` caller**
In `crates/aura-cli/src/main.rs`, update the production caller and the four test
callers to pass a `Selection`:
- `run_walkforward` (line 1755): add `select: Selection` to its signature:
`fn run_walkforward(strategy: Strategy, name: &str, persist: bool, data: DataSource, grid: &Stage1RGrid, select: Selection) {`
and update its `walkforward_family(...)` call (line 1763) to
`let result = walkforward_family(strategy, persist.then_some(name), &data, grid, select);`
- Line 1963: `walkforward_family(Strategy::SmaCross, None, &DataSource::Synthetic, &Stage1RGrid::default(), Selection::Argmax)`
- Line 2131: `walkforward_family(Strategy::Stage1R, None, data, grid, Selection::Argmax)`
- Line 3456: `walkforward_family(Strategy::SmaCross, None, &DataSource::Synthetic, &Stage1RGrid::default(), Selection::Argmax)`
- Line 4131: `walkforward_family(Strategy::Stage1R, None, &DataSource::Synthetic, &Stage1RGrid::default(), Selection::Argmax)`
- [ ] **Step 8: Add `--select` to `parse_walkforward_args` and thread it to the dispatch**
In `crates/aura-cli/src/main.rs`, in `parse_walkforward_args` (line 1648):
Change the return type to add `Selection`:
`) -> Result<(Strategy, String, bool, DataChoice, Stage1RGrid, Selection), String> {`
Update the usage string to mention `--select` (append `[--select <argmax|plateau:mean|plateau:worst>]`
to both the `///` doc usage line and the `usage` closure string).
Add `let mut select = Selection::Argmax;` beside the other `let mut` locals.
Add a match arm for the flag (beside `--stop-k`):
`"--select" => select = parse_select(value).map_err(|()| usage())?,`
Change the final return to:
`Ok((strategy, name, persist, real.finish(&usage)?, grid, select))`
In the command dispatch (line 2935-2939), update the destructure + call:
```rust
["walkforward", rest @ ..] => match parse_walkforward_args(rest) {
Ok((strategy, name, persist, choice, grid, select)) => {
run_walkforward(strategy, &name, persist, DataSource::from_choice(choice), &grid, select)
}
```
- [ ] **Step 9: Compile-gate — workspace + tests build clean**
Run: `cargo build -p aura-cli --tests`
Expected: clean (0 errors) — every caller of the changed signatures threaded.
- [ ] **Step 10: Run the new unit tests (GREEN)**
Run: `cargo test -p aura-cli select_winner_refuses`
Expected: PASS.
Run: `cargo test -p aura-cli parse_walkforward_select`
Expected: PASS.
- [ ] **Step 11: C23 — existing walk-forward goldens stay byte-identical**
Run: `cargo test -p aura-cli walkforward_strategy_stage1_r`
Expected: PASS — argmax path unchanged.
Run: `cargo test -p aura-cli runs_family_rank_shows_deflated`
Expected: PASS — the deflated display line is byte-identical.
Run: `cargo test -p aura-cli walkforward_bare_sma`
Expected: PASS — the SMA summary golden is unchanged.
---
## Task 5: `runs family rank` plateau E2E golden
**Files:**
- Test: `crates/aura-cli/tests/cli_run.rs``walkforward_plateau_select_stamps_plateau_provenance`
Tasks 1-4 deliver the behaviour; this pins it end-to-end (C18: the plateau
provenance reaches disk and the display renders it).
- [ ] **Step 1: Write the plateau E2E (RED until Tasks 1-4 land; GREEN after)**
In `crates/aura-cli/tests/cli_run.rs`, beside `runs_family_rank_shows_deflated_line`
(after line 2388), add:
```rust
/// Property: a stage1-r walk-forward run with `--select plateau:mean` selects the
/// neighbourhood-smoothed winner and stamps each OOS manifest with the plateau
/// provenance (`mode = PlateauMean`, `neighbourhood_score`, `n_neighbours`); `runs
/// family … rank` renders the `plateau(mean)=… over N cells` line, and the
/// deflation fields are omitted (orthogonal annotation). End-to-end witness that
/// the opt-in selection rule reaches disk (C18) and the display path renders it.
#[test]
fn walkforward_plateau_select_stamps_plateau_provenance() {
let cwd = temp_cwd("runs-plateau");
let wf = Command::new(BIN)
.args([
"walkforward", "--strategy", "stage1-r", "--select", "plateau:mean",
"--fast", "50,100", "--slow", "200,400",
"--stop-length", "14,21", "--stop-k", "2.0,3.0",
"--name", "wf-plat",
])
.current_dir(&cwd)
.output()
.expect("spawn walkforward --select plateau:mean");
assert!(
wf.status.success(),
"walkforward exit: {:?}; stderr: {}",
wf.status,
String::from_utf8_lossy(&wf.stderr)
);
let rank = Command::new(BIN)
.args(["runs", "family", "wf-plat-0", "rank", "sqn_normalized"])
.current_dir(&cwd)
.output()
.expect("spawn runs family wf-plat-0 rank sqn_normalized");
assert!(rank.status.success(), "rank exit: {:?}", rank.status);
let rank_out = String::from_utf8(rank.stdout).expect("utf-8");
let _ = std::fs::remove_dir_all(&cwd);
assert!(rank_out.contains("plateau(mean)="), "rank output missing plateau(mean)=: {rank_out:?}");
assert!(rank_out.contains(" cells"), "rank output missing 'over N cells': {rank_out:?}");
assert!(rank_out.contains("\"mode\":\"PlateauMean\""), "manifest carries PlateauMean: {rank_out:?}");
assert!(!rank_out.contains("\"deflated_score\""), "plateau run omits deflated_score: {rank_out:?}");
}
```
- [ ] **Step 2: Run the plateau E2E (GREEN)**
Run: `cargo test -p aura-cli walkforward_plateau`
Expected: PASS.
- [ ] **Step 3: Full CLI suite + workspace + clippy (no regressions)**
Run: `cargo test -p aura-cli`
Expected: PASS — every existing golden green; the plateau E2E green.
Run: `cargo test --workspace`
Expected: PASS.
Run: `cargo clippy --workspace --all-targets -- -D warnings`
Expected: clean (no warnings).