diff --git a/docs/specs/0046-sweep-named-binding.md b/docs/specs/0046-sweep-named-binding.md new file mode 100644 index 0000000..ada56f6 --- /dev/null +++ b/docs/specs/0046-sweep-named-binding.md @@ -0,0 +1,320 @@ +# Sweep named-binding — Design Spec + +**Date:** 2026-06-15 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +## Goal + +A sweep/walk-forward consumer that wants readable knob names — in a +`RunManifest`, or when reading back a returned family — currently re-derives +the name↔value pairing by hand from `param_space()`, because the engine hands +the per-point closure a bare positional `&[Scalar]` and `SweepPoint` carries +only positional `Vec`. This is open-coded in three near-identical sites +in the CLI (`sweep_family`, `sweep_over`, `run_oos`), each re-reading +`param_space()` and zipping it back onto the positional point. + +This cycle removes that redundancy with a single derived projection — a free +function `zip_params(space, point)` — and makes a returned `SweepFamily` +self-describing (it carries its `param_space()` once, exposing +`named_params(i)`). The named view is the prerequisite for #52 (random +param-sweep), whose `RandomSpace` must report readable per-point params over the +same execution layer. + +The fix is **behaviour-preserving** and **does not touch the run-closure +signature**: `F: Fn(&[Scalar]) -> RunReport + Sync` stays byte-for-byte, so +#52's `RandomSpace` plugs into the same `sweep()` layer with zero signature +reconciliation (#52's #71-firewall constraint, 2026-06-15 comment). + +## Architecture + +A derived named view over the **unchanged positional** sweep machinery. The +positional core keeps its shape — `GridSpace.axes`, `GridSpace::points() -> +Vec>`, the closure `Fn(&[Scalar])`, and `SweepPoint.params: +Vec` are all untouched in kind. Three small additions thread the names, +which already exist in `param_space()`, through to the two consumers that need +them: + +1. **`GridSpace` retains the `&[ParamSpec]` it already receives** in `new()` + (today it validates against them, then discards them). No `new()` signature + change. +2. **`SweepFamily` carries that `Vec` once** (stamped where the + family is assembled), so a returned family is self-describing. +3. **A free function `zip_params(space, point)`** is the one shared projection + that pairs names with a positional point — used in-closure to build the + manifest (consumer i) and by `SweepFamily::named_params` to read a returned + family by name (consumer ii). + +The name is a **derived projection**, not new identity and not new per-point +state: a param's identity remains its positional slot (C23/C8); `ParamSpec.name` +is the non-load-bearing debug symbol it always was. `SweepPoint` stays +positional. + +## Concrete code shapes + +### Worked consumer code (the acceptance evidence) + +**Consumer (i) — in the per-point closure** (the friction the issue names). The +three CLI sites collapse from a 5-line hand-zip to one function call. Today: + +```rust +// crates/aura-cli/src/main.rs ~386 (also ~503, ~530) — BEFORE +.sweep(|point: &[Scalar]| { + let mut h = bp.bootstrap_with_params(point.to_vec()).expect("kind-checked"); + // ...run, drain sinks... + let params = space + .iter() + .zip(point) + .map(|(ps, v)| (ps.name.clone(), scalar_as_param_f64(v))) // hand re-zip + lossy coerce + .collect(); + RunReport { manifest: sim_optimal_manifest(params, window, 0), metrics } +}) +``` + +After — the name pairing is one call; the lossy f64 collapse moves into the +manifest constructor (it owns its own lossiness): + +```rust +// crates/aura-cli/src/main.rs ~386 — AFTER +.sweep(|point: &[Scalar]| { // signature UNCHANGED: Fn(&[Scalar]) + let mut h = bp.bootstrap_with_params(point.to_vec()).expect("kind-checked"); + // ...run, drain sinks... + RunReport { manifest: sim_optimal_manifest(zip_params(&space, point), window, 0), metrics } +}) +``` + +**Consumer (ii) — reading a returned family by name** (post-hoc; the readable +half that #52 needs). No blueprint in hand, no re-zip: + +```rust +// a downstream reader of a returned SweepFamily +let fam: SweepFamily = bp.axis("sma_cross.fast", [2, 3]).axis("exposure.scale", [0.5]).sweep(run)?; +for (i, pt) in fam.points.iter().enumerate() { + let named: Vec<(String, Scalar)> = fam.named_params(i); // [("sma_cross.fast", I64(2)), ("exposure.scale", F64(0.5))] + println!("{named:?} -> {}", pt.report.metrics.total_pips); +} +``` + +### Implementation shapes (before → after) + +**New free function** — the one shared projection (aura-core, alongside +`ParamSpec` in `node.rs`): + +```rust +// crates/aura-core/src/node.rs (or a small params module) — NEW +/// Pair each param-space name with the co-indexed positional value. +/// The inverse of the positional binding (C23): names are a derived view, +/// not identity. `space` and `point` are co-indexed in param_space() slot order. +pub fn zip_params(space: &[ParamSpec], point: &[Scalar]) -> Vec<(String, Scalar)> { + space.iter().zip(point).map(|(ps, v)| (ps.name.clone(), *v)).collect() +} +``` + +**`GridSpace` retains its specs** (sweep.rs): + +```rust +// crates/aura-engine/src/sweep.rs ~14 — BEFORE +#[derive(Debug)] +pub struct GridSpace { axes: Vec> } + +// AFTER +#[derive(Debug)] +pub struct GridSpace { space: Vec, axes: Vec> } + +impl GridSpace { + // new(space, axes) signature UNCHANGED; it now stores `space: space.to_vec()` + // after the existing validation, instead of discarding it. + pub fn param_specs(&self) -> &[ParamSpec] { &self.space } +} +``` + +**`SweepFamily` carries the space + exposes the named view** (sweep.rs): + +```rust +// crates/aura-engine/src/sweep.rs ~106 — BEFORE +#[derive(Clone, Debug, PartialEq)] +pub struct SweepFamily { pub points: Vec } + +// AFTER +#[derive(Clone, Debug, PartialEq)] +pub struct SweepFamily { pub space: Vec, pub points: Vec } + +impl SweepFamily { + /// The i-th point's params paired with their names (derived; reuses zip_params). + pub fn named_params(&self, i: usize) -> Vec<(String, Scalar)> { + zip_params(&self.space, &self.points[i].params) + } +} +// SweepPoint is UNCHANGED: pub struct SweepPoint { pub params: Vec, pub report: RunReport } +``` + +The single stamp point in the hot path — `sweep_with_threads` already has the +`&GridSpace`: + +```rust +// crates/aura-engine/src/sweep.rs ~176 — the family assembly gains the space +SweepFamily { + space: space.param_specs().to_vec(), + points: points.into_iter().zip(reports).map(|(params, report)| SweepPoint { params, report }).collect(), +} +``` + +**`sim_optimal_manifest` owns the lossy f64 collapse** (CLI): + +```rust +// crates/aura-cli/src/main.rs ~130 — BEFORE +fn sim_optimal_manifest(params: Vec<(String, f64)>, window: (Timestamp, Timestamp), seed: u64) -> RunManifest { + RunManifest { commit: commit_hash(), params, window, seed, broker: "sim-optimal".into() } +} + +// AFTER — takes typed Scalar pairs, collapses to f64 internally (one place); +// preserves the existing scalar_as_param_f64 contract verbatim (I64/F64 -> f64, +// panic on Bool/Ts). The stored field stays Vec<(String, f64)> (no serde change). +fn sim_optimal_manifest(params: Vec<(String, Scalar)>, window: (Timestamp, Timestamp), seed: u64) -> RunManifest { + let params = params.into_iter().map(|(n, s)| (n, scalar_as_param_f64(&s))).collect(); + RunManifest { commit: commit_hash(), params, window, seed, broker: "sim-optimal".into() } +} +``` + +All **five** hand-listing callers adapt their literals to `Scalar` (every +caller that passes a literal `Vec<(String, f64)>` today — not only the two the +sweep path uses): + +```rust +// crates/aura-cli/src/main.rs — the five literal callers, BEFORE -> AFTER +// run_sample (~161), run_sample_real / `aura run --real` (~224), +// mc_family (~613), run_sample_seeded (test helper, ~970): +vec![("sma_fast".into(), 2.0), ("sma_slow".into(), 4.0), ("exposure_scale".into(), 0.5)] +// becomes +vec![("sma_fast".into(), Scalar::F64(2.0)), ("sma_slow".into(), Scalar::F64(4.0)), ("exposure_scale".into(), Scalar::F64(0.5))] + +// run_macd / `aura run --macd` (~869) — a 4-tuple incl. ema_signal: +vec![("ema_fast".into(), 2.0), ("ema_slow".into(), 4.0), ("ema_signal".into(), 3.0), ("exposure_scale".into(), 0.5)] +// becomes +vec![("ema_fast".into(), Scalar::F64(2.0)), ("ema_slow".into(), Scalar::F64(4.0)), ("ema_signal".into(), Scalar::F64(3.0)), ("exposure_scale".into(), Scalar::F64(0.5))] +``` + +## Components + +- **`aura-core::zip_params`** — free function, the single name↔value projection. + No lifetime, no struct. Reusable by mc / walk_forward / registry consumers. + Re-exported from `aura-core`'s lib root and (transitively, as needed) reachable + by the CLI. +- **`GridSpace` (aura-engine, sweep.rs)** — gains a private `space: + Vec` field, retained from the `&[ParamSpec]` already passed to + `new()`, plus a `param_specs(&self) -> &[ParamSpec]` accessor. `new()` keeps + its signature; `points()` and the odometer are untouched. +- **`SweepFamily` (aura-engine, sweep.rs)** — gains a public `space: + Vec` field, stamped once in `sweep_with_threads`, plus + `named_params(&self, i) -> Vec<(String, Scalar)>`. `SweepPoint` is unchanged. +- **`sim_optimal_manifest` (aura-cli)** — input parameter type changes from + `Vec<(String, f64)>` to `Vec<(String, Scalar)>`; the lossy f64 collapse moves + inside. It has **eight** call sites (main.rs:161, 224, 392, 509, 536, 613, + 869, 970), all of which the signature change touches — every one must be + migrated for the tree to compile: + - **three sweep closures** now pass `zip_params(...)`: `sweep_family` (~392), + `sweep_over` (~509), `run_oos` (~536, zips `&space` against its `params` + argument); + - **five hand-listing callers** now pass `Scalar` literals: `run_sample` + (~161), `run_sample_real` / `aura run --real` (~224), `mc_family` (~613), + `run_macd` / `aura run --macd` (~869, a 4-tuple incl. `ema_signal`), + `run_sample_seeded` (test helper, ~970). + +## Data flow + +1. `Composite::param_space() -> Vec` (names, slot order) — unchanged + source of names. +2. `SweepBinder::sweep` resolves named axes → positional grid → + `GridSpace::new(&space, axes)`, which now **retains** `space`. +3. `sweep_with_threads`: `points = grid.points()` (positional); the closure runs + over `&points[i]` (bare `&[Scalar]`, unchanged); the family is assembled as + `SweepFamily { space: grid.param_specs().to_vec(), points }`. The low-level + `sweep(&GridSpace, ..)` path gets the names too, with no `sweep()` signature + change. +4. **Consumer (i)** — in-closure: `sim_optimal_manifest(zip_params(&space, + point), ..)` at the three CLI sites; `space` is the `param_space()` already + captured before each closure. +5. **Consumer (ii)** — post-hoc: `family.named_params(i)` reads names off the + carried `space`. + +The positional binding path (`bootstrap_with_params(point.to_vec())`, C23 +by-index) is entirely untouched. + +## Error handling + +- `zip_params` pairs co-indexed slices; equal slot order and length is the + invariant (both originate from the same family / same `param_space()`). `zip` + truncates to the shorter on a (contract-violating) mismatch; no panic. +- `SweepFamily::named_params(i)` indexes `self.points[i]` — an out-of-range `i` + panics like any slice index (caller's contract), consistent with existing + `points` access. +- The f64 collapse in `sim_optimal_manifest` preserves the existing + `scalar_as_param_f64` contract verbatim: `I64`/`F64 -> f64`, `unreachable!` + on `Bool`/`Ts`. Changing this (a typed `Vec<(String, Scalar)>` manifest field) + is the deferred "typed param-space" item, explicitly out of scope here. + +## Testing strategy + +Engine + core unit tests: + +- `zip_params_pairs_names_with_values_in_slot_order` — names and values are + paired in `param_space()` slot order for a multi-axis space. +- `zip_params_matches_hand_zip` — behaviour-preservation pin: `zip_params(space, + point)` (after coercion) equals the old + `space.iter().zip(point).map(...).collect()` output for a known point. +- `sweep_family_carries_param_space` — a returned `SweepFamily.space` equals the + blueprint's `param_space()`. +- `family_named_params_round_trips` — `fam.named_params(i)` pairs the right names + with the right slot values for each member. + +Existing tests that must stay green (regression guard, enumeration untouched): +the determinism suite — `points_enumerate_in_odometer_order`, +`family_is_deterministic_across_thread_counts`, `sweep_equals_n_independent_runs`. + +**Cross-crate blast radius — the new `SweepFamily.space` field reaches beyond +aura-engine.** The planner threads the new `SweepFamily.space` / `GridSpace.space` +fields through every site that constructs those structs by literal, including +**`crates/aura-registry/src/lib.rs` (~278, the `optimize` test +`optimize_picks_the_max_metric_point_ties_to_earliest`)**, which builds a +`SweepFamily { points: vec![..] }` literal and would otherwise fail to compile. +`optimize`/`rank_by` read only `.points[].report`, so the test's `space` can be +an empty or representative `Vec` — the planner picks. **aura-registry +is therefore a touched crate this iteration**, not only a `zip_params` consumer. + +E2E (the worked consumer is the acceptance evidence): `aura sweep` output is +**byte-identical** before and after (the manifest's `params` are unchanged), and +`aura runs family` / walk-forward outputs are unchanged — pinning that the +consumer migration is behaviour-preserving. + +No serde impact: `SweepFamily` and `ParamSpec` are not serde-derived; the +lineage extractors read only `.report`; `Scalar` stays serde-free and +`RunManifest.params` stays `Vec<(String, f64)>`. + +## Acceptance criteria + +Judged against aura's domain invariants and the worked consumer code above: + +1. **Removes redundancy (measurable).** The three identical hand-zip sites in + `aura-cli` collapse to one `zip_params(&space, point)` call each; the + name↔value pairing logic lives once, tested. A consumer reading a returned + family by name calls `named_params(i)` instead of re-fetching `param_space()` + and re-zipping. +2. **Audience naturally reaches for it.** A sweep/walk-forward author building a + manifest, and #52's `RandomSpace` author reporting readable per-point params, + both reach for these surfaces rather than re-deriving the pairing. +3. **Reintroduces no failure class.** The run-closure signature is unchanged + (firewall held; #52 plugs in with zero reconciliation); names stay a derived + view (C23 — slot is identity); enumeration/determinism is untouched (C1); + no serde/lineage change. +4. **Behaviour-preserving.** `aura sweep` / walk-forward output is byte-identical + before and after; the existing determinism suite stays green. + +## Non-goals + +- **Typed `RunManifest.params`** (`Vec<(String, Scalar)>`) + `Scalar` serde — + the deferred "typed param-space" cycle. `zip_params`' typed output sets it up + but this cycle does not ship it; the manifest field stays f64. +- **#52's `RandomSpace`** (random param-sweep) — a separate issue; this is its + prerequisite. +- **Any change to the run-closure signature** — firewall-forbidden. +- **Cross-family comparison** — out of scope.