spec: 0059 generic portable sweep key + bool-param demo strategy (boss-signed)

Generalises the family-member trace key (#105 foot-gun): derive it from the axes that actually vary (not two hardcoded axis names), and render it as a filesystem-conformant directory name on every major OS — charset [A-Za-z0-9._-], case-less values (no case-insensitive-FS collision), length-bounded with an FNV-1a fallback for the unbounded-grid edge. Adds the first bool *param* node (LongOnly) and a momentum demo strategy (Ema -> Sub -> Exposure -> LongOnly -> SimBroker) swept via a new `aura sweep --strategy <sma|momentum>` selector — the empirical proof the generic key works for a strategy whose params (incl. a bool) differ entirely from the SMA-cross demo. Boss-signed on the Step-5 grounding-check PASS (an independent fresh-context agent): all 9 load-bearing current-behaviour assumptions ratified by green tests; the end-to-end bool-param sweep path is correctly scoped as new behaviour proven by new tests, not assumed. Workspace type-checks clean. refs #105
2026-06-21 10:41:31 +02:00
parent 1d6fafbc39
commit a8ba019d4d
1 changed files with 431 additions and 0 deletions
@@ -0,0 +1,431 @@
+# Generic, filesystem-portable sweep member-key + a bool-param demo strategy — Design Spec
+
+**Date:** 2026-06-21
+**Status:** Draft — awaiting user spec review
+**Authors:** orchestrator + Claude
+
+> Reference issue: #105 (Brummel/Aura). Design decision log: the latest comment
+> on #105 (forks: key-from-varying-axes, portable charset, demo strategy shape).
+
+## Goal
+
+Two coupled deliverables, in this order:
+
+1. **Generalise the family-member trace key** so it is collision-free for *any*
+   sweep grid, decoupled from the two hardcoded axis names — and so the on-disk
+   key is a **filesystem-conformant directory name on every major OS** (Linux,
+   Windows, macOS), including case-insensitive filesystems.
+2. **A new demo strategy** with three params of three kinds — one of them a
+   `bool` — swept generically. It is the empirical proof that (1) is generic: the
+   same `aura sweep` machinery drives a strategy whose params have nothing in
+   common with the SMA-cross demo, and the bool axis shows up, conformant, in the
+   member directory name.
+
+The foot-gun being closed (#105): `sweep_member_key` renders `f{fast}s{slow}`
+from two hardcoded axes. It is collision-free only over *today's* grid; a grid
+that varies any other axis silently maps distinct points to the same directory
+and overwrites traces. The fix derives the key from the axes that *actually
+vary*, generically.
+
+## Architecture
+
+Three changes, smallest-blast-radius first:
+
+- **Engine (additive, one pure read accessor).** `SweepBinder` already
+  accumulates its declared axes as `Vec<(String, Vec<Scalar>)>`
+  (`blueprint.rs:379-382`). Add `SweepBinder::varying_axes(&self) -> Vec<String>`
+  returning the names of axes with **> 1 value** — the axes that distinguish one
+  grid point from another. This is the binder's own knowledge (it holds the
+  value-lists); exposing it is a benign, behaviour-preserving addition. Nothing
+  else in the engine changes — the `sweep` HOF signature, `GridSpace`,
+  `SweepFamily`, and the `Fn(&[Cell]) -> RunReport + Sync` closure contract are
+  untouched (consistent with the #104 design: persistence stays in the CLI
+  closure, the engine carries no trace state).
+
+- **CLI (the portable key + the seam).** A new shared, pure key renderer
+  replaces `sweep_member_key`. The per-member closure captures the binder's
+  `varying_axes()` set and renders the member key from the *varying* axes of
+  `zip_params(&space, point)`, in param-space slot order, through a portable
+  sanitiser. Filesystem-portability (charset, case, length) lives here — it is a
+  presentation concern of the on-disk trace store, not the engine's.
+
+- **CLI + aura-std (the demo strategy).** A new `LongOnly` node in `aura-std`
+  carries the first `bool` *param*; a new `momentum` strategy composite in the
+  CLI wires `Ema → Sub → Exposure → LongOnly → SimBroker`; and `aura sweep` gains
+  a `--strategy <sma|momentum>` selector so the new strategy is swept and traced
+  the same way the built-in SMA sweep is.
+
+### The portable key — the rules that make it conformant
+
+A member key is **one path component** under `runs/traces/<name>/<key>/`. It must
+be valid on Linux, Windows, and macOS, and must not silently collide on
+case-insensitive filesystems (NTFS / APFS default). The rules:
+
+- **Charset:** the rendered key contains only `[A-Za-z0-9._-]`. Every other byte
+  (the Windows-forbidden `<>:"/\|?*`, shell/cloud-sync-hostile `=,` `[]`, spaces,
+  control chars) is mapped to `_` by a single `sanitize_component` pass. This set
+  is also URL-path-safe (member keys appear as URL segments when served) and
+  survives cloud-sync clients (OneDrive/Dropbox) that reject more than the bare
+  filesystem does.
+- **Case-less values → no case-insensitive-FS collision.** Within one family the
+  axis *names* are fixed (they are the grid's axes); two members differ only in
+  axis *values*. Values are rendered case-lessly — `i64`/`timestamp` as decimal
+  digits, `bool` as `true`/`false`, `f64` via Rust's `f64 Display` (decimal, with
+  **no** scientific notation) — so two distinct members can never differ only by
+  letter case. Therefore they never collapse to one directory on a
+  case-insensitive filesystem.
+- **Collision-free by construction.** Distinct grid points differ in ≥1 varying
+  axis value at the same slot. Each value renders injectively per kind (Rust's
+  `f64 Display` is the shortest round-tripping decimal; integers/bools are
+  trivially injective), and the token structure (same names, same order, same
+  separators) is identical across a family's members — so distinct points yield
+  distinct keys.
+- **Bounded length.** The key is capped at a fixed limit (`MAX_KEY = 200` bytes,
+  comfortably under the 255-byte POSIX `NAME_MAX` and inside Windows' 260-char
+  `MAX_PATH` once the `runs/traces/<name>/` prefix and `/<tap>.json` suffix are
+  added). A key within the cap is used verbatim. A key that would exceed it
+  degrades to a conformant, collision-resistant fallback: `h-<16-hex>` where the
+  hex is FNV-1a-64 of the full untruncated readable key (a fixed, version-stable,
+  non-cryptographic byte hash — chosen so the on-disk name never drifts across
+  Rust releases the way `DefaultHasher` may). The built-in demo grids stay far
+  under the cap, so the fallback never fires for them; it exists so the
+  *generic* promise holds for an arbitrarily large grid.
+
+### Token grammar
+
+For each varying axis, in param-space slot order, a token
+`sanitize_component(name)` + `-` + `sanitize_component(render_value(value))`;
+tokens joined by `_`. Example (the new strategy's grid):
+`ema.length-5_exposure.scale-0.5_longonly.enabled-true`.
+
+## Concrete code shapes
+
+### (1) The user-facing program — the new strategy, authored and swept
+
+The new demo strategy a user authors (CLI-local, mirroring the existing
+`sma_cross` / `macd` sample builders), and the command that sweeps + traces it:
+
+```rust
+/// EMA-distance momentum: the signal is how far price sits above/below its own
+/// EMA (a mean-reversion/trend score), sized by Exposure, then passed through a
+/// long-only gate. Three swept knobs of three kinds: `ema.length` (i64),
+/// `exposure.scale` (f64), `longonly.enabled` (bool). Nothing in common with the
+/// SMA-cross demo — the generic-sweep proof.
+fn momentum_blueprint_with_sinks() -> (Composite, Receiver<…>, Receiver<…>) {
+    let (tx_eq, rx_eq) = mpsc::channel();
+    let (tx_ex, rx_ex) = mpsc::channel();
+    let mut g = GraphBuilder::new("momentum");
+    let ema   = g.add(Ema::builder());        // default path segment: ema.length
+    let dist  = g.add(Sub::builder());        // price - ema  (momentum score)
+    let expo  = g.add(Exposure::builder());   // exposure.scale
+    let gate  = g.add(LongOnly::builder());    // longonly.enabled  (the bool param)
+    let broker = g.add(SimBroker::builder(SYNTHETIC_PIP_SIZE));
+    let eq = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_eq));
+    let ex = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_ex));
+    let price = g.source_role("price", ScalarKind::F64);
+    g.feed(price, [ema.input("series"), dist.input("lhs"), broker.input("price")]);
+    g.connect(ema.output("value"),    dist.input("rhs"));      // momentum = price - ema
+    g.connect(dist.output("value"),   expo.input("signal"));
+    g.connect(expo.output("exposure"), gate.input("exposure"));
+    g.connect(gate.output("exposure"), broker.input("exposure")); // long-only exposure -> broker
+    g.connect(broker.output("equity"), eq.input("col[0]"));
+    g.connect(gate.output("exposure"), ex.input("col[0]"));       // record the FINAL (gated) exposure
+    (g.build().expect("momentum wiring resolves"), rx_eq, rx_ex)
+}
+```
+
+Param-space (lowering order): `[ema.length: I64, exposure.scale: F64,
+longonly.enabled: Bool]` — the path segments are the nodes' default names
+(lowercased builder labels), exactly as `exposure.scale` already arises in the
+SMA sample.
+
+The invocation and its observable result:
+
+```console
+$ aura sweep --strategy momentum --trace mom
+{"family_id":"mom-0","report":{…}}      # one line per grid point (8 points)
+… 8 lines …
+$ ls runs/traces/mom/
+ema.length-5_exposure.scale-0.5_longonly.enabled-true
+ema.length-5_exposure.scale-0.5_longonly.enabled-false
+ema.length-5_exposure.scale-1_longonly.enabled-true
+…
+ema.length-10_exposure.scale-1_longonly.enabled-false
+$ aura chart 'mom/ema.length-5_exposure.scale-0.5_longonly.enabled-true'
+<!doctype html> … uPlot page …
+```
+
+The grid: `ema.length ∈ {5,10} × exposure.scale ∈ {0.5,1.0} × longonly.enabled ∈
+{true,false}` = 8 members, **all three axes varying** — so all three appear in
+every member key, the bool among them. This is the criterion's evidence: a user
+authoring a genuinely different strategy reaches for exactly the same `aura
+sweep`, and gets correct, portable, self-describing member directories for free.
+
+### (2) The portable key renderer (replaces `sweep_member_key`)
+
+Before — `main.rs:186-196`, two hardcoded axes, panics on drift:
+
+```rust
+fn sweep_member_key(named: &[(String, Scalar)]) -> String {
+    let val = |name: &str| named.iter().find(|(n,_)| n == name)
+        .unwrap_or_else(|| panic!("… missing the key axis '{name}'")).1.as_i64();
+    format!("f{}s{}", val("signals.trend.fast.length"), val("signals.trend.slow.length"))
+}
+```
+
+After — generic over the *varying* axes, portable, collision-free:
+
+```rust
+const MAX_KEY: usize = 200;
+
+/// Map any byte outside the portable directory-name charset `[A-Za-z0-9._-]` to
+/// `_`. The single source of filesystem-portability for an on-disk path component.
+fn sanitize_component(s: &str) -> String {
+    s.chars().map(|c| if c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-') { c } else { '_' }).collect()
+}
+
+/// Render a scalar value case-lessly: integers/timestamps as decimal digits,
+/// bool as `true`/`false`, f64 via Rust's decimal `Display` (no sci-notation).
+fn render_value(v: &Scalar) -> String {
+    match v {
+        Scalar::I64(n) => n.to_string(),
+        Scalar::F64(x) => x.to_string(),          // Rust f64 Display: decimal, shortest round-trip
+        Scalar::Bool(b) => b.to_string(),         // "true" / "false"
+        Scalar::Timestamp(t) => t.to_string(),    // epoch i64 digits, never a colon-bearing date
+    }
+}
+
+/// The portable, collision-free member key for a grid point: a `name-value` token
+/// per *varying* axis (slot order), joined by `_`. `varying` is the binder's
+/// `varying_axes()` set; pinned singletons carry no information and are omitted.
+/// Over the cap, degrade to a conformant FNV-1a fallback so the key stays one
+/// valid path component for ANY grid.
+fn member_key(named: &[(String, Scalar)], varying: &HashSet<String>) -> String {
+    let key: String = named.iter()
+        .filter(|(n, _)| varying.contains(n))
+        .map(|(n, v)| format!("{}-{}", sanitize_component(n), sanitize_component(&render_value(v))))
+        .collect::<Vec<_>>()
+        .join("_");
+    let key = if key.is_empty() { "m".to_string() } else { key };   // 1-point grid: no varying axis
+    if key.len() <= MAX_KEY { key } else { format!("h-{:016x}", fnv1a64(key.as_bytes())) }
+}
+```
+
+Worked examples (the unit-test corpus):
+
+| varying named pairs | key |
+|---|---|
+| `[(ema.length, I64 5), (exposure.scale, F64 0.5), (longonly.enabled, Bool true)]` | `ema.length-5_exposure.scale-0.5_longonly.enabled-true` |
+| `[(longonly.enabled, Bool false)]` | `longonly.enabled-false` |
+| `[(exposure.scale, F64 -0.5)]` | `exposure.scale--0.5` (leading `-` is a value sign; `-` is in the portable set) |
+| `[(weird key!, F64 1.0)]` (sanitised name) | `weird_key_-1` |
+| `[(t, Timestamp 1725148800000)]` | `t-1725148800000` (epoch digits, no `:`/`T`/`Z`) |
+
+`fnv1a64` is a fixed FNV-1a-64 over the bytes (offset `0xcbf29ce484222325`, prime
+`0x100000001b3`) — a ~5-line, version-stable, non-security disambiguator for the
+rare over-cap key. (Per the project dependency policy this is the per-case
+judgement: a tiny fixed algorithm whose output must be stable *on disk* across
+toolchain versions, where a hand-written FNV is the simplest vetted-equivalent
+and `std`'s `DefaultHasher` is explicitly unstable across releases.)
+
+### (3) The `LongOnly` node (aura-std) — the first bool param
+
+New file `crates/aura-std/src/longonly.rs`, mirroring `exposure.rs` exactly; the
+bool *param* drives `eval`:
+
+```rust
+/// Long-only exposure gate. The bool param `enabled`: true clamps short
+/// (negative) exposure to 0 (long-only); false passes exposure through unchanged.
+/// One f64 input, one f64 output. Emits None until its input is present (C8).
+pub struct LongOnly { enabled: bool, out: [Cell; 1] }
+
+impl LongOnly {
+    pub fn new(enabled: bool) -> Self { Self { enabled, out: [Cell::from_f64(0.0)] } }
+
+    pub fn builder() -> PrimitiveBuilder {
+        PrimitiveBuilder::new(
+            "LongOnly",
+            NodeSchema {
+                inputs: vec![PortSpec { kind: ScalarKind::F64, firing: Firing::Any, name: "exposure".into() }],
+                output: vec![FieldSpec { name: "exposure".into(), kind: ScalarKind::F64 }],
+                params: vec![ParamSpec { name: "enabled".into(), kind: ScalarKind::Bool }],
+            },
+            |p| Box::new(LongOnly::new(p[0].bool())),   // first use of Cell::bool() in a builder
+        )
+    }
+}
+
+impl Node for LongOnly {
+    fn lookbacks(&self) -> Vec<usize> { vec![1] }
+    fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Cell]> {
+        let w = ctx.f64_in(0);
+        if w.is_empty() { return None; }
+        self.out[0] = Cell::from_f64(if self.enabled { w[0].max(0.0) } else { w[0] });
+        Some(&self.out)
+    }
+    fn label(&self) -> String { format!("LongOnly({})", self.enabled) }
+}
+```
+
+Exported from `aura-std/src/lib.rs` (`mod longonly; pub use longonly::LongOnly;`).
+
+The end-to-end bool path the cycle *proves* (it is not assumed): the type
+machinery is already in place and ratified — `ScalarKind::Bool`, `Scalar::Bool` /
+`as_bool`, `Cell::from_bool` / `Cell::bool`, and `GridSpace::new`'s per-value
+kind-check that already classifies `Bool` (see the green
+`random_space_new_non_numeric_slot`, which shows `Bool` is a known, handled kind
+on the *random* side). No node has yet *used* a bool param, so this is the first;
+its node test and the momentum bool-axis sweep test are what ratify the
+authoring + enumeration + bootstrap path for a bool param.
+
+### (4) The `--strategy` selector (a pure parse fn, mirroring `parse_real_args`)
+
+Before — three fixed arms (`main.rs:993-995`):
+
+```rust
+["sweep"] => run_sweep("sweep", false),
+["sweep", "--name", n] => run_sweep(n, false),
+["sweep", "--trace", n] => run_sweep(n, true),
+```
+
+After — one arm + a pure, unit-testable grammar; `run_sweep` gains a strategy:
+
+```rust
+#[derive(Clone, Copy, PartialEq, Debug)]
+enum Strategy { SmaCross, Momentum }
+
+/// `sweep [--strategy <sma|momentum>] [--name <n> | --trace <n>]`. Defaults:
+/// SMA-cross, name "sweep", no persist (today's behaviour preserved). `--name`
+/// and `--trace` are mutually exclusive. Pure (no I/O / exit) so it is testable.
+fn parse_sweep_args(rest: &[&str]) -> Result<(Strategy, String, bool), String> { … }
+
+// dispatch:
+["sweep", rest @ ..] => match parse_sweep_args(rest) {
+    Ok((strategy, name, persist)) => run_sweep(strategy, &name, persist),
+    Err(msg) => { eprintln!("aura: {msg}"); std::process::exit(2); }
+},
+
+fn run_sweep(strategy: Strategy, name: &str, persist: bool) {
+    let family = match strategy {
+        Strategy::SmaCross => sweep_family(persist.then_some(name)),
+        Strategy::Momentum => momentum_sweep_family(persist.then_some(name)),
+    };
+    // … unchanged: append_family + print each point's record line …
+}
+```
+
+`momentum_sweep_family` mirrors `sweep_family`: declare the 3-axis grid via the
+`.axis()` chain, capture `binder.varying_axes()`, and in the closure render the
+key with `member_key(&zip_params(&space, point), &varying)`. `sweep_family`
+(SMA) is refactored the same way — it drops the hardcoded `sweep_member_key` and
+uses the generic `member_key` with its own `varying_axes()` (so its member dirs
+change from `f2s4` to `signals.trend.fast.length-2_signals.trend.slow.length-4`).
+
+### Seam: `SweepBinder::varying_axes` (engine, additive)
+
+```rust
+impl SweepBinder {
+    /// The names of the axes that vary (more than one value) — the axes that
+    /// distinguish one grid point from another. Declaration order; the caller
+    /// derives membership, key token order comes from param-space slot order.
+    pub fn varying_axes(&self) -> Vec<String> {
+        self.axes.iter().filter(|(_, v)| v.len() > 1).map(|(n, _)| n.clone()).collect()
+    }
+}
+```
+
+The CLI captures it *before* the consuming `.sweep()`:
+`let binder = bp.axis(…)….axis(…); let varying: HashSet<String> =
+binder.varying_axes().into_iter().collect(); binder.sweep(move |point| { … })`.
+
+## Components
+
+- **`aura-engine` `SweepBinder::varying_axes`** — new pure read accessor
+  (`blueprint.rs`). Additive; no existing signature changes.
+- **`aura-std` `LongOnly`** — new node + module + export. First bool-param node.
+- **`aura-cli`** —
+  - `sanitize_component`, `render_value`, `member_key`, `fnv1a64`, `MAX_KEY`
+    (replace `sweep_member_key`);
+  - `momentum_blueprint_with_sinks`, `momentum_sweep_family`, a momentum point
+    grid + (re-used) synthetic stream;
+  - `Strategy` enum, `parse_sweep_args`, `run_sweep(strategy, name, persist)`;
+  - `sweep_family` / `sweep_over` refactored to the generic key via
+    `varying_axes()`;
+  - `USAGE` updated to show `aura sweep [--strategy <sma|momentum>] [--name <n>|--trace <n>]`.
+
+## Data flow
+
+Unchanged through the sweep engine (enumerate → disjoint parallel runs →
+`SweepFamily` in odometer order). The only new flow is in the per-member closure:
+`point: &[Cell]` → `zip_params(&space, point)` → filter by the captured
+`varying` set → `member_key` → portable dir component → `persist_traces(
+"{name}/{key}", …)` → `runs/traces/<name>/<key>/<tap>.json`. The MC (`seed{seed}`)
+and walk-forward (`oos{from}`) keys are already charset-conformant; they are left
+as-is this cycle (single-axis, trivially portable). `sanitize_component` is the
+one shared portability gate the sweep key flows through.
+
+## Error handling
+
+- A 1-point grid (no varying axis) → `member_key` returns `"m"` (a fixed valid
+  component) rather than the empty string. Specified, tested.
+- An over-`MAX_KEY` key → FNV-1a fallback (`h-<hex>`), conformant and
+  collision-resistant. Specified, tested with a synthetic many-axis case.
+- `parse_sweep_args` rejects an unknown flag, a flag missing its value, an
+  unknown strategy token, or both `--name` and `--trace` — `Err(usage)`, surfaced
+  as stderr + `exit(2)` (the CLI's convention), never a panic.
+- The old fail-loud-on-missing-axis panic is **removed** with `sweep_member_key`:
+  it guarded a hardcoded-axis assumption that no longer exists. Its two unit tests
+  are removed with it.
+
+## Testing strategy
+
+- **`member_key` unit tests (aura-cli `--bin aura`)** — the worked-examples table:
+  charset conformance (`key.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.'|'_'|'-'))`),
+  the bool case, a negative-float case, a sanitised-name case, the empty→`"m"`
+  case, and an over-cap case (key length ≤ `MAX_KEY`, still conformant, two
+  distinct over-cap inputs → distinct keys).
+- **Collision-freedom over a different-axis grid (the #105 regression)** — a
+  grid that varies an axis set *other than* the old `trend.fast`/`trend.slow`
+  (e.g. the momentum grid, which varies `exposure.scale` and a bool) yields N
+  distinct member keys for N points. The old `f…s…` key would have collapsed
+  them; the new key does not.
+- **Determinism** — two runs of the momentum sweep produce byte-identical member
+  keys (the family + its keys are a pure function of the build, C1).
+- **The bool axis in the key** — the momentum sweep's member dirs include both
+  `…_longonly.enabled-true` and `…_longonly.enabled-false`.
+- **`SweepBinder::varying_axes` (aura-engine)** — a grid with mixed singleton and
+  multi-value axes returns exactly the multi-value names.
+- **`LongOnly` node (aura-std)** — `enabled=true` clamps a negative input to 0
+  and passes a positive through; `enabled=false` passes both through; `None`
+  until input present. Plus `enabled` is read from the bool param
+  (`LongOnly::builder()` schema param is `enabled: Bool`).
+- **Integration (aura-cli `--test cli_run`, spawn-with-temp-cwd)** — `aura sweep
+  --strategy momentum --trace mom` persists 8 member dirs whose names match the
+  portable charset and include the bool; `aura chart 'mom/<key>'` emits a uPlot
+  page for one. The existing SMA `--trace` integration test is **updated** from
+  the `f2s4`-style assertion to the new portable key form.
+- **Backward-compat** — the existing `["sweep"]` / `--name` / `--trace` forms
+  still work through `parse_sweep_args` (default SMA), so the existing sweep
+  integration + determinism tests stay green (with the key-string assertions
+  updated to the new form).
+
+## Acceptance criteria
+
+The project's feature-acceptance criterion (audience reaches for it; improves
+correctness; reintroduces no failure the core constraints forbid):
+
+- **Audience reaches for it.** A researcher authoring a new strategy with new
+  params (incl. a bool) runs the *same* `aura sweep` and gets correct,
+  self-describing, portable per-member traces — shown by the worked momentum
+  example. No per-strategy key code.
+- **Improves correctness.** Closes #105: the key is collision-free for any grid,
+  not just the built-in two-axis one. Removes the latent silent-overwrite.
+- **Reintroduces no forbidden failure.** Determinism (C1) preserved — keys are a
+  pure function of the grid; parallelism across members unchanged. No look-ahead,
+  no engine merge, no per-eval emission added (C2/C3/C8). The engine change is a
+  pure read accessor; trace state stays out of the engine (C-consistency with
+  #104).
+- **Concrete, conformant, generic.** Member keys match `^[A-Za-z0-9._-]+$`, are
+  case-insensitive-FS safe, length-bounded, and carry the bool axis — proven by
+  the test corpus above over a grid that varies a different axis set than the old
+  hardcoded two.