diff --git a/docs/specs/0028-param-sweep-grid.md b/docs/specs/0028-param-sweep-grid.md new file mode 100644 index 0000000..4803261 --- /dev/null +++ b/docs/specs/0028-param-sweep-grid.md @@ -0,0 +1,484 @@ +# Param-sweep: a grid family of disjoint instances from one blueprint — Design Spec + +**Date:** 2026-06-10 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +## Goal + +Turn one value-empty blueprint into a **family** of frozen instances — one per +point in a declared grid — run them **disjointly in parallel**, and collect each +point's metrics into an ordered family. This is cycle C (#32) of the milestone +"The World — parameter-space & sweep": the first orchestration axis (C12.1), the +first "N instead of 1". The walking skeleton and the construction layer each +produce a single instance; the World begins here. + +#30 landed the declarable param-space (`Composite::param_space() -> Vec`, +flat, path-qualified, kind-typed). #31 landed the binding primitive +(`Composite::bootstrap_with_params(Vec) -> Harness`, one point → one +instance). This cycle is the axis *over* that primitive: enumerate a grid of +points across the param-space, bootstrap and run each disjointly, and fold the +results into a family that #33 (the run registry) will later index. + +Scope is the **sweep axis only**, and within it **grid enumeration only**: a +cartesian product over per-slot discrete value-lists. Random enumeration +(per-slot ranges + sample-count + seed + RNG), optimize (argmax, C12.2), +walk-forward (C12.3), and Monte-Carlo (C12.4) are deferred to later cycles. + +## Architecture + +A new module `crates/aura-engine/src/sweep.rs` carries three parts: + +1. **Enumeration — `GridSpace`.** A validated cartesian grid over a blueprint's + param-space: one discrete value-list per param slot, kind-checked against the + `&[ParamSpec]` the blueprint reports. `points()` materializes the cartesian + product in a deterministic odometer order (last axis varies fastest). + +2. **Execution — `sweep()`.** A closure-driven primitive. It owns enumeration + + **disjoint parallel execution** + family collection; the **author** owns the + per-point closure `run_one: Fn(&[Scalar]) -> RunMetrics`. Each point's + `run_one` builds a fresh blueprint, bootstraps it under the point vector, + runs it, drains its sinks, and summarizes — entirely within one thread, so it + shares nothing mutable (C1). Execution is `std::thread::scope` over + `available_parallelism()` workers pulling point indices from a shared atomic + cursor (work-stealing load-balances uneven per-point cost). **No external + dependency** — `rayon` would violate C16 (the engine workspace is + zero-external-dependency by commitment; only `aura-ingest` links an external + tree). `std::thread::scope` is std and sufficient for a flat, + embarrassingly-parallel map. + +3. **Collection — `SweepFamily`.** An ordered `Vec`, each a + `{ params, metrics }` pair. The family order is the enumeration order, + **independent of thread completion order**: workers tag each result with its + point index, and the family is assembled by sorting on that index after the + scope joins. Only the cursor is shared (atomic); the results side is + lock-free. + +### Why closure-driven (the engine ↔ author boundary) + +Metrics extraction is **harness-specific glue**: a run's metrics are reduced by +draining its recording sinks, and the engine cannot generically know which +`Recorder` is equity vs exposure — the sink channels live *outside* the graph, +held by the author (the run loop is oblivious to the side effect, by C8/C18). +So "run one point → `RunMetrics`" stays author-side, and the sweep's engine-level +job is purely enumeration + disjoint execution + collection. + +A fresh per-point build also resolves two structural facts from #31 cleanly: +`bootstrap_with_params(self, …)` **consumes** the blueprint (N instances need N +fresh builds), and a `Recorder` bakes an `mpsc::Sender` into the blueprint (a +fresh build gives each point its **own** drainable channels). Both fall out of +`run_one` constructing its blueprint per call. + +### Determinism and disjointness (C1) + +Each point's run is a pure function of its param vector and the shared read-only +data window (`Arc<[…]>` / `&[…]`, `Sync`). No two points share mutable state, so +they are lock-free disjoint — and the *family* is bit-identical regardless of how +many threads ran it or in what order they finished, because the assembled order +is the enumeration order, not the completion order. + +## Concrete code shapes + +### The user-facing program (the acceptance evidence) + +The author authors the topology once (a value-empty blueprint factory), declares +a grid over its param-space, and sweeps — getting one `RunMetrics` per grid point. +The engine's own SMA-cross sample is the worked example: + +```rust +use aura_engine::{GridSpace, Scalar, sweep, summarize, f64_field}; + +// 1. The value-empty blueprint factory (built fresh per point — fresh channels). +// `sma_cross_sample()` returns the blueprint plus a way to drain its two sinks. +let param_space = sma_cross_sample().0.param_space(); +// param_space == ["sma_cross.fast": I64, "sma_cross.slow": I64, "scale": F64] + +// 2. Declare a grid: one discrete value-list per param slot, in param_space order. +let grid = GridSpace::new(¶m_space, vec![ + vec![Scalar::I64(2), Scalar::I64(3)], // fast ∈ {2, 3} + vec![Scalar::I64(4), Scalar::I64(5)], // slow ∈ {4, 5} + vec![Scalar::F64(0.5)], // scale ∈ {0.5} +])?; // 2 × 2 × 1 = 4 points +assert_eq!(grid.len(), 4); + +let prices = synthetic_prices(); // shared read-only data window + +// 3. Sweep: each point builds fresh, bootstraps, runs, drains, summarizes. +let family = sweep(&grid, |point| { + let (bp, rx_eq, rx_ex) = sma_cross_sample(); // fresh blueprint + channels + let mut h = bp.bootstrap_with_params(point.to_vec()) + .expect("grid points are kind-checked against param_space"); + h.run(vec![prices.clone()]); + let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); + let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); + summarize(&equity, &exposure) +}); + +// 4. The family is ordered (odometer: last axis fastest), self-describing. +assert_eq!(family.points.len(), 4); +assert_eq!(family.points[0].params, vec![Scalar::I64(2), Scalar::I64(4), Scalar::F64(0.5)]); +assert_eq!(family.points[3].params, vec![Scalar::I64(3), Scalar::I64(5), Scalar::F64(0.5)]); +for pt in &family.points { + assert!(pt.metrics.total_pips.is_finite()); // every point produced metrics +} +``` + +The CLI demonstrator (iteration 2) makes the same payoff visible without writing +Rust — `aura sweep` runs the built-in sample over a small built-in grid and +prints the family as one JSON line per point: + +```text +$ aura sweep +{"params":{"sma_cross.fast":2,"sma_cross.slow":4,"scale":0.5},"metrics":{"total_pips":…,"max_drawdown":…,"exposure_sign_flips":…}} +{"params":{"sma_cross.fast":2,"sma_cross.slow":5,"scale":0.5},"metrics":{…}} +{"params":{"sma_cross.fast":3,"sma_cross.slow":4,"scale":0.5},"metrics":{…}} +{"params":{"sma_cross.fast":3,"sma_cross.slow":5,"scale":0.5},"metrics":{…}} +``` + +### Implementation shapes (secondary — the new module) + +The four public types + the free function, in `aura-engine/src/sweep.rs`: + +```rust +use aura_core::{ParamSpec, Scalar, ScalarKind}; +use crate::RunMetrics; +use std::sync::atomic::{AtomicUsize, Ordering}; + +/// A validated cartesian grid over a blueprint's param-space: one discrete +/// value-list per param slot, in `param_space()` order. Enumerates a family of +/// points (C12.1 grid axis). +pub struct GridSpace { + axes: Vec>, +} + +impl GridSpace { + /// Validate `axes` against `space` (the blueprint's `param_space()`): + /// one axis per slot (`Arity`), every value the slot's declared kind + /// (`KindMismatch`), no empty axis (`EmptyAxis`, which would yield zero + /// points). On success the grid enumerates `∏ |axis_i|` points. + pub fn new(space: &[ParamSpec], axes: Vec>) -> Result { + if axes.len() != space.len() { + return Err(SweepError::Arity { expected: space.len(), got: axes.len() }); + } + for (slot, (axis, ps)) in axes.iter().zip(space).enumerate() { + if axis.is_empty() { + return Err(SweepError::EmptyAxis { slot }); + } + for (value_index, v) in axis.iter().enumerate() { + if v.kind() != ps.kind { + return Err(SweepError::KindMismatch { + slot, value_index, expected: ps.kind, got: v.kind(), + }); + } + } + } + Ok(Self { axes }) + } + + /// The number of grid points (`∏ |axis_i|`), always `>= 1`. + pub fn len(&self) -> usize { + self.axes.iter().map(|a| a.len()).product() + } + + /// Always `false` — a valid `GridSpace` rejects empty axes, so it has at + /// least one point. Present to satisfy clippy's `len_without_is_empty`. + pub fn is_empty(&self) -> bool { + false + } + + /// The cartesian product, in odometer order: the **last** axis varies + /// fastest. Deterministic — the same grid yields the same point sequence. + pub fn points(&self) -> Vec> { + let mut out = Vec::with_capacity(self.len()); + let mut idx = vec![0usize; self.axes.len()]; + loop { + out.push(self.axes.iter().zip(&idx).map(|(a, &i)| a[i]).collect()); + // odometer increment from the last axis + let mut k = self.axes.len(); + loop { + if k == 0 { + return out; + } + k -= 1; + idx[k] += 1; + if idx[k] < self.axes[k].len() { + break; + } + idx[k] = 0; + } + } + } +} + +/// One enumerated point and the metrics its run produced. Self-describing: the +/// `params` vector is the point's coordinate in `param_space()` order. +#[derive(Clone, Debug, PartialEq)] +pub struct SweepPoint { + pub params: Vec, + pub metrics: RunMetrics, +} + +/// The ordered result family of a sweep — one `SweepPoint` per grid point, in +/// enumeration (odometer) order, independent of thread completion order. +#[derive(Clone, Debug, PartialEq)] +pub struct SweepFamily { + pub points: Vec, +} + +/// Run `run_one` over every grid point, disjointly in parallel (C1), and collect +/// the family in enumeration order. `run_one` builds + bootstraps + runs + +/// summarizes one point; it shares nothing mutable, so it is `Sync` and the runs +/// are lock-free. Workers pull point indices from a shared atomic cursor +/// (work-stealing); each tags its result with the point index, and the family is +/// assembled by sorting on that index after the scope joins — so the order is the +/// enumeration order, not the completion order. +pub fn sweep(space: &GridSpace, run_one: F) -> SweepFamily +where + F: Fn(&[Scalar]) -> RunMetrics + Sync, +{ + let points = space.points(); + let cursor = AtomicUsize::new(0); + let nthreads = std::thread::available_parallelism() + .map(|n| n.get()) + .unwrap_or(1) + .min(points.len().max(1)); + + let mut results: Vec<(usize, RunMetrics)> = std::thread::scope(|scope| { + let handles: Vec<_> = (0..nthreads) + .map(|_| { + scope.spawn(|| { + let mut local: Vec<(usize, RunMetrics)> = Vec::new(); + loop { + let i = cursor.fetch_add(1, Ordering::Relaxed); + if i >= points.len() { + break; + } + local.push((i, run_one(&points[i]))); + } + local + }) + }) + .collect(); + handles.into_iter().flat_map(|h| h.join().unwrap()).collect() + }); + + results.sort_by_key(|&(i, _)| i); + SweepFamily { + points: results + .into_iter() + .map(|(i, metrics)| SweepPoint { params: points[i].clone(), metrics }) + .collect(), + } +} + +/// A structural fault constructing a `GridSpace` — the typed gate before any run. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum SweepError { + /// The number of axes does not equal the param-space slot count. + Arity { expected: usize, got: usize }, + /// A grid value's kind does not match its slot's declared kind. + KindMismatch { slot: usize, value_index: usize, expected: ScalarKind, got: ScalarKind }, + /// A slot was given no values (would yield zero points). + EmptyAxis { slot: usize }, +} +``` + +The `lib.rs` re-export (one import surface, beside the existing engine exports): + +```rust +// crates/aura-engine/src/lib.rs (new) +pub mod sweep; +pub use sweep::{sweep, GridSpace, SweepError, SweepFamily, SweepPoint}; +``` + +### Implementation shapes (secondary — the CLI demonstrator, iteration 2) + +`aura sweep` reuses the existing sample wiring, lifted so the receivers reach the +caller. Today `build_sample()` drops its `Recorder` receivers (`crates/aura-cli/ +src/main.rs:161`); the demonstrator factors a `sample_blueprint_with_sinks() -> +(Composite, Receiver, Receiver)` that returns them, so `run_one` can drain per +point. The subcommand declares a small built-in grid, sweeps, and prints one JSON +line per point: + +```rust +// crates/aura-cli/src/main.rs (new arm, beside `run` and `graph`) +fn cmd_sweep() { + let space = sample_blueprint_with_sinks().0.param_space(); + let grid = GridSpace::new(&space, vec![ + vec![Scalar::I64(2), Scalar::I64(3)], + vec![Scalar::I64(4), Scalar::I64(5)], + vec![Scalar::F64(0.5)], + ]) + .expect("the built-in grid matches the sample param-space"); + let prices = synthetic_prices(); + let family = sweep(&grid, |point| { + let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks(); + let mut h = bp.bootstrap_with_params(point.to_vec()).expect("kind-checked point"); + h.run(vec![prices.clone()]); + let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); + let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); + summarize(&equity, &exposure) + }); + for (pt, ps) in family.points.iter().zip(std::iter::repeat(&space)) { + println!("{}", sweep_point_to_json(pt, ps)); // hand-rolled JSON (C14) + } +} +``` + +`sweep_point_to_json` renders `{"params":{name:value,…},"metrics":{…}}` with the +param names from `param_space()` zipped onto the point vector, reusing the +canonical `f64`/`i64` token rules already established for `RunReport::to_json` +(`crates/aura-engine/src/report.rs:80`). The exact byte format is the planner's +to pin against a golden test. + +## Components + +| Crate / file | Change | +|---|---| +| `crates/aura-engine/src/sweep.rs` (new) | `GridSpace` (`new` / `len` / `is_empty` / `points`), `SweepPoint`, `SweepFamily`, `SweepError`, and the free `sweep()` with the `std::thread::scope` execution. The only new module this cycle. | +| `crates/aura-engine/src/lib.rs` | `pub mod sweep;` + re-export `sweep`, `GridSpace`, `SweepError`, `SweepFamily`, `SweepPoint`. | +| `crates/aura-cli/src/main.rs` (iteration 2) | Factor `sample_blueprint_with_sinks() -> (Composite, Receiver, Receiver)` (returns the receivers `build_sample` drops); add the `sweep` subcommand arg parsing + `cmd_sweep()` + `sweep_point_to_json`. | + +No change to `Node`, the run loop, `bootstrap_with_params`, `param_space`, or +`RunMetrics`/`summarize` — the sweep is a new layer *over* the unchanged #30/#31 +primitives. + +## Data flow + +``` +author: build_blueprint() factory + per-slot value-lists + │ GridSpace::new(bp.param_space(), axes) + │ ├─ axes.len() == space.len() else SweepError::Arity + │ ├─ each axis non-empty else SweepError::EmptyAxis { slot } + │ └─ each value.kind() == slot.kind else SweepError::KindMismatch { slot, value_index, .. } + ▼ space.points() -> cartesian product, odometer order (last axis fastest), deterministic + ▼ sweep(&space, run_one): + cursor = AtomicUsize(0) + std::thread::scope over min(available_parallelism(), npoints) workers: + worker loop: i = cursor.fetch_add(1); if i >= npoints break + metrics = run_one(&points[i]) ── disjoint, lock-free (C1) + └ build fresh bp (fresh channels) + -> bootstrap_with_params(points[i]) (#31, kind-checked -> .expect honest) + -> run(prices) (#3 shared read-only window) + -> drain sinks -> summarize (#9 RunMetrics) + local.push((i, metrics)) + join all locals -> sort_by_key(i) ── order = enumeration order, not completion + ▼ SweepFamily { points: [SweepPoint { params: points[i], metrics }, … ] } +C1: same grid -> same family, bit-identical, thread-count-independent. +``` + +## Error handling + +`GridSpace::new` is the **typed gate** — it rejects a malformed grid before any +run, mirroring #31's `CompileError::{ParamKindMismatch, ParamArity}` discipline +at the enumeration layer: + +- `SweepError::Arity { expected, got }` — the axis count does not equal + `param_space().len()`. +- `SweepError::KindMismatch { slot, value_index, expected, got }` — a grid + value's `ScalarKind` does not match the slot's declared kind. `slot` is the + flat param-space index; `value_index` is the position within that axis. +- `SweepError::EmptyAxis { slot }` — a slot was given no values (an empty axis + collapses the cartesian product to zero points; an author mistake, caught + rather than silently producing an empty family). + +`run_one` returns `RunMetrics` directly (no `Result`). The grid's points are +kind-checked at `GridSpace::new`, and `bootstrap_with_params`'s arity matches +`param_space()` by construction, so the `bootstrap_with_params(…).expect(…)` +inside the closure is the honest gate — exactly how `run_sample` treats it today +(`crates/aura-cli/src/main.rs`). The **value domain** (e.g. `length >= 1`) stays +the node constructor's own `assert` (`Sma::new`); the grid author is responsible +for in-domain values this iteration — consistent with #31's deliberate cut, where +the search-range / valid domain belongs to the run (C20), and full domain +validation is a later cycle's concern. The sweep does **not** catch a panicking +point this iteration (a panic in `run_one` propagates through `scope`); introducing +search-range validation or panic recovery here would be scope creep. + +## Testing strategy + +All in `crates/aura-engine/src/sweep.rs` `#[cfg(test)]`, except the CLI golden +(iteration 2). + +**Enumeration (`GridSpace`).** +- `points()` over `[[2,3],[4,5]]` is exactly `[[2,4],[2,5],[3,4],[3,5]]` — odometer + order, last axis fastest. +- `len()` equals `∏ |axis_i|` (e.g. `2 × 2 × 1 == 4`); a single-axis and a + three-axis grid both enumerate correctly. + +**Faults (`GridSpace::new`).** +- Wrong axis count → `SweepError::Arity { expected, got }`. +- A value of the wrong kind (e.g. `F64` where the slot is `I64`) → + `SweepError::KindMismatch { slot, value_index, .. }`. +- An empty axis → `SweepError::EmptyAxis { slot }`. + +**Sweep == N independent runs (central, C1).** Build a value-empty SMA-cross +two-sink blueprint fixture (the existing `report.rs` / `blueprint.rs` two-sink +pattern, value-empty). For a grid over the SMA lengths, each family point's +`metrics` equals `run_one` called **directly** on that same point vector — the +sweep adds enumeration + execution, never a metrics change. + +**Determinism under concurrency (C1).** The same grid swept twice yields an +identical `SweepFamily` (`PartialEq`). The family is identical whether the sweep +ran on one worker or many — pinned by comparing a sweep forced to a single point +of parallelism against the default multi-worker run (e.g. a grid large enough to +spread across workers), asserting equal families. Order is the enumeration order +regardless of scheduling. + +**Distinct points produce distinct, populated metrics.** Over `fast ∈ {2,3}`, +`slow ∈ {4,5}`, `scale ∈ {0.5}` (4 points), every point has a finite +`total_pips`, and points whose lengths differ produce differing metrics — the +family is not a constant. + +**CLI demonstrator (iteration 2).** A golden test on `aura sweep` stdout: four +JSON lines, one per grid point, in odometer order, each +`{"params":{…},"metrics":{…}}` with names from `param_space()`. Asserted as the +deterministic contract (the JSON model, like `RunReport::to_json`). + +## Acceptance criteria + +The feature passes aura's acceptance criterion (CLAUDE.md): + +- **The intended author reaches for exactly this.** The World / a sweep is the + milestone's intended author, and the worked program above is precisely "one + blueprint, a grid of points, a family of metrics" — the C12.1 grid axis. It is + the first "N instead of 1", which #30/#31 set up but could not express. +- **It measurably enables the next cycle.** #33 (the run registry) indexes and + compares a *sweep family*; without `sweep()` producing a `SweepFamily`, #33 has + nothing to index. This cycle is its precondition. +- **It reintroduces no forbidden failure class.** Determinism (C1) is preserved — + the same grid yields a bit-identical family, thread-count-independent (disjoint, + lock-free; the assembled order is the enumeration order). No look-ahead is + introduced (each point is an ordinary `bootstrap_with_params` + `run`). + Topology is param-invariant (C19 — the grid varies values, never structure). + Zero-external-dependency (C16) holds — execution is `std::thread::scope`, not + `rayon`. + +Concretely, the cycle is accepted when: + +1. `GridSpace::new` validates arity / kind / non-empty and `points()` enumerates + the cartesian product in deterministic odometer order. +2. `sweep()` runs a value-empty SMA-cross blueprint over a grid and returns a + `SweepFamily` whose per-point metrics each equal the single-run metrics for + that point. +3. The family is bit-identical across repeated runs and across thread counts + (determinism under concurrency, C1). +4. `aura sweep` prints the family as one JSON line per point in odometer order + (iteration 2). +5. `cargo build/test --workspace` is green and `cargo clippy --workspace + --all-targets -- -D warnings` is clean. + +## Out of scope (deferred) + +- **Random enumeration** (per-slot ranges + sample-count + seed + hand-rolled + deterministic RNG, C12.1's other half) — a follow-up issue filed at cycle + close. +- **Per-point `RunManifest` assembly** (commit / window / seed / broker per + point) and indexing the family — #33 (the run registry). This cycle's family + carries `params + metrics` only; the manifest constants are sweep-level, not + per-point. +- **Value-domain / search-range validation** beyond the node constructor's own + `assert` — C20 (the range belongs to the run); a later cycle. +- **Optimize (argmax), walk-forward, Monte-Carlo** — C12.2 / C12.3 / C12.4, later + milestones. +- Any change to `Node::eval`, the run loop, `bootstrap_with_params`, or + `param_space`.