Aura/docs/specs/0059-generic-portable-sweep-key.md

Generic, filesystem-portable sweep member-key + a bool-param demo strategy — Design Spec

Date: 2026-06-21 Status: Draft — awaiting user spec review Authors: orchestrator + Claude

Reference issue: #105 (Brummel/Aura). Design decision log: the latest comment on #105 (forks: key-from-varying-axes, portable charset, demo strategy shape).

Goal

Two coupled deliverables, in this order:

1. Generalise the family-member trace key so it is collision-free for any sweep grid, decoupled from the two hardcoded axis names — and so the on-disk key is a filesystem-conformant directory name on every major OS (Linux, Windows, macOS), including case-insensitive filesystems.
2. A new demo strategy with three params of three kinds — one of them a bool — swept generically. It is the empirical proof that (1) is generic: the same aura sweep machinery drives a strategy whose params have nothing in common with the SMA-cross demo, and the bool axis shows up, conformant, in the member directory name.

The foot-gun being closed (#105): sweep_member_key renders f{fast}s{slow} from two hardcoded axes. It is collision-free only over today's grid; a grid that varies any other axis silently maps distinct points to the same directory and overwrites traces. The fix derives the key from the axes that actually vary, generically.

Architecture

Three changes, smallest-blast-radius first:

• Engine (additive, one pure read accessor). SweepBinder already accumulates its declared axes as Vec<(String, Vec<Scalar>)> (blueprint.rs:379-382). Add SweepBinder::varying_axes(&self) -> Vec<String> returning the names of axes with > 1 value — the axes that distinguish one grid point from another. This is the binder's own knowledge (it holds the value-lists); exposing it is a benign, behaviour-preserving addition. Nothing else in the engine changes — the sweep HOF signature, GridSpace, SweepFamily, and the Fn(&[Cell]) -> RunReport + Sync closure contract are untouched (consistent with the #104 design: persistence stays in the CLI closure, the engine carries no trace state).

• CLI (the portable key + the seam). A new shared, pure key renderer replaces sweep_member_key. The per-member closure captures the binder's varying_axes() set and renders the member key from the varying axes of zip_params(&space, point), in param-space slot order, through a portable sanitiser. Filesystem-portability (charset, case, length) lives here — it is a presentation concern of the on-disk trace store, not the engine's.

• CLI + aura-std (the demo strategy). A new LongOnly node in aura-std carries the first bool param; a new momentum strategy composite in the CLI wires Ema → Sub → Exposure → LongOnly → SimBroker; and aura sweep gains a --strategy <sma|momentum> selector so the new strategy is swept and traced the same way the built-in SMA sweep is.

The portable key — the rules that make it conformant

A member key is one path component under runs/traces/<name>/<key>/. It must be valid on Linux, Windows, and macOS, and must not silently collide on case-insensitive filesystems (NTFS / APFS default). The rules:

• Charset: the rendered key contains only [A-Za-z0-9._-]. Every other byte (the Windows-forbidden <>:"/\|?*, shell/cloud-sync-hostile =, [], spaces, control chars) is mapped to _ by a single sanitize_component pass. This set is also URL-path-safe (member keys appear as URL segments when served) and survives cloud-sync clients (OneDrive/Dropbox) that reject more than the bare filesystem does.
• Case-less values → no case-insensitive-FS collision. Within one family the axis names are fixed (they are the grid's axes); two members differ only in axis values. Values are rendered case-lessly — i64/timestamp as decimal digits, bool as true/false, f64 via Rust's f64 Display (decimal, with no scientific notation) — so two distinct members can never differ only by letter case. Therefore they never collapse to one directory on a case-insensitive filesystem.
• Collision-free by construction. Distinct grid points differ in ≥1 varying axis value at the same slot. Each value renders injectively per kind (Rust's f64 Display is the shortest round-tripping decimal; integers/bools are trivially injective), and the token structure (same names, same order, same separators) is identical across a family's members — so distinct points yield distinct keys.
• Bounded length. The key is capped at a fixed limit (MAX_KEY = 200 bytes, comfortably under the 255-byte POSIX NAME_MAX and inside Windows' 260-char MAX_PATH once the runs/traces/<name>/ prefix and /<tap>.json suffix are added). A key within the cap is used verbatim. A key that would exceed it degrades to a conformant, collision-resistant fallback: h-<16-hex> where the hex is FNV-1a-64 of the full untruncated readable key (a fixed, version-stable, non-cryptographic byte hash — chosen so the on-disk name never drifts across Rust releases the way DefaultHasher may). The built-in demo grids stay far under the cap, so the fallback never fires for them; it exists so the generic promise holds for an arbitrarily large grid.

Token grammar

For each varying axis, in param-space slot order, a token sanitize_component(name) + - + sanitize_component(render_value(value)); tokens joined by _. Example (the new strategy's grid): ema.length-5_exposure.scale-0.5_longonly.enabled-true.

Concrete code shapes

(1) The user-facing program — the new strategy, authored and swept

The new demo strategy a user authors (CLI-local, mirroring the existing sma_cross / macd sample builders), and the command that sweeps + traces it:

/// EMA-distance momentum: the signal is how far price sits above/below its own
/// EMA (a mean-reversion/trend score), sized by Exposure, then passed through a
/// long-only gate. Three swept knobs of three kinds: `ema.length` (i64),
/// `exposure.scale` (f64), `longonly.enabled` (bool). Nothing in common with the
/// SMA-cross demo — the generic-sweep proof.
fn momentum_blueprint_with_sinks() -> (Composite, Receiver<…>, Receiver<…>) {
    let (tx_eq, rx_eq) = mpsc::channel();
    let (tx_ex, rx_ex) = mpsc::channel();
    let mut g = GraphBuilder::new("momentum");
    let ema   = g.add(Ema::builder());        // default path segment: ema.length
    let dist  = g.add(Sub::builder());        // price - ema  (momentum score)
    let expo  = g.add(Exposure::builder());   // exposure.scale
    let gate  = g.add(LongOnly::builder());    // longonly.enabled  (the bool param)
    let broker = g.add(SimBroker::builder(SYNTHETIC_PIP_SIZE));
    let eq = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_eq));
    let ex = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_ex));
    let price = g.source_role("price", ScalarKind::F64);
    g.feed(price, [ema.input("series"), dist.input("lhs"), broker.input("price")]);
    g.connect(ema.output("value"),    dist.input("rhs"));      // momentum = price - ema
    g.connect(dist.output("value"),   expo.input("signal"));
    g.connect(expo.output("exposure"), gate.input("exposure"));
    g.connect(gate.output("exposure"), broker.input("exposure")); // long-only exposure -> broker
    g.connect(broker.output("equity"), eq.input("col[0]"));
    g.connect(gate.output("exposure"), ex.input("col[0]"));       // record the FINAL (gated) exposure
    (g.build().expect("momentum wiring resolves"), rx_eq, rx_ex)
}

Param-space (lowering order): [ema.length: I64, exposure.scale: F64, longonly.enabled: Bool] — the path segments are the nodes' default names (lowercased builder labels), exactly as exposure.scale already arises in the SMA sample.

The invocation and its observable result:

$ aura sweep --strategy momentum --trace mom
{"family_id":"mom-0","report":{…}}      # one line per grid point (8 points)
… 8 lines …
$ ls runs/traces/mom/
ema.length-5_exposure.scale-0.5_longonly.enabled-true
ema.length-5_exposure.scale-0.5_longonly.enabled-false
ema.length-5_exposure.scale-1_longonly.enabled-true
…
ema.length-10_exposure.scale-1_longonly.enabled-false
$ aura chart 'mom/ema.length-5_exposure.scale-0.5_longonly.enabled-true'
<!doctype html> … uPlot page …

The grid: ema.length ∈ {5,10} × exposure.scale ∈ {0.5,1.0} × longonly.enabled ∈ {true,false} = 8 members, all three axes varying — so all three appear in every member key, the bool among them. This is the criterion's evidence: a user authoring a genuinely different strategy reaches for exactly the same aura sweep, and gets correct, portable, self-describing member directories for free.

(2) The portable key renderer (replaces sweep_member_key)

Before — main.rs:186-196, two hardcoded axes, panics on drift:

fn sweep_member_key(named: &[(String, Scalar)]) -> String {
    let val = |name: &str| named.iter().find(|(n,_)| n == name)
        .unwrap_or_else(|| panic!("… missing the key axis '{name}'")).1.as_i64();
    format!("f{}s{}", val("signals.trend.fast.length"), val("signals.trend.slow.length"))
}

After — generic over the varying axes, portable, collision-free:

const MAX_KEY: usize = 200;

/// Map any byte outside the portable directory-name charset `[A-Za-z0-9._-]` to
/// `_`. The single source of filesystem-portability for an on-disk path component.
fn sanitize_component(s: &str) -> String {
    s.chars().map(|c| if c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-') { c } else { '_' }).collect()
}

/// Render a scalar value case-lessly: integers/timestamps as decimal digits,
/// bool as `true`/`false`, f64 via Rust's decimal `Display` (no sci-notation).
fn render_value(v: &Scalar) -> String {
    match v {
        Scalar::I64(n) => n.to_string(),
        Scalar::F64(x) => x.to_string(),          // Rust f64 Display: decimal, shortest round-trip
        Scalar::Bool(b) => b.to_string(),         // "true" / "false"
        Scalar::Timestamp(t) => t.to_string(),    // epoch i64 digits, never a colon-bearing date
    }
}

/// The portable, collision-free member key for a grid point: a `name-value` token
/// per *varying* axis (slot order), joined by `_`. `varying` is the binder's
/// `varying_axes()` set; pinned singletons carry no information and are omitted.
/// Over the cap, degrade to a conformant FNV-1a fallback so the key stays one
/// valid path component for ANY grid.
fn member_key(named: &[(String, Scalar)], varying: &HashSet<String>) -> String {
    let key: String = named.iter()
        .filter(|(n, _)| varying.contains(n))
        .map(|(n, v)| format!("{}-{}", sanitize_component(n), sanitize_component(&render_value(v))))
        .collect::<Vec<_>>()
        .join("_");
    let key = if key.is_empty() { "m".to_string() } else { key };   // 1-point grid: no varying axis
    if key.len() <= MAX_KEY { key } else { format!("h-{:016x}", fnv1a64(key.as_bytes())) }
}

Worked examples (the unit-test corpus):

varying named pairs	key
[(ema.length, I64 5), (exposure.scale, F64 0.5), (longonly.enabled, Bool true)]	ema.length-5_exposure.scale-0.5_longonly.enabled-true
[(longonly.enabled, Bool false)]	longonly.enabled-false
[(exposure.scale, F64 -0.5)]	exposure.scale--0.5 (leading - is a value sign; - is in the portable set)
[(weird key!, F64 1.0)] (sanitised name)	weird_key_-1
[(t, Timestamp 1725148800000)]	t-1725148800000 (epoch digits, no :/T/Z)

fnv1a64 is a fixed FNV-1a-64 over the bytes (offset 0xcbf29ce484222325, prime 0x100000001b3) — a ~5-line, version-stable, non-security disambiguator for the rare over-cap key. (Per the project dependency policy this is the per-case judgement: a tiny fixed algorithm whose output must be stable on disk across toolchain versions, where a hand-written FNV is the simplest vetted-equivalent and std's DefaultHasher is explicitly unstable across releases.)

(3) The LongOnly node (aura-std) — the first bool param

New file crates/aura-std/src/longonly.rs, mirroring exposure.rs exactly; the bool param drives eval:

/// Long-only exposure gate. The bool param `enabled`: true clamps short
/// (negative) exposure to 0 (long-only); false passes exposure through unchanged.
/// One f64 input, one f64 output. Emits None until its input is present (C8).
pub struct LongOnly { enabled: bool, out: [Cell; 1] }

impl LongOnly {
    pub fn new(enabled: bool) -> Self { Self { enabled, out: [Cell::from_f64(0.0)] } }

    pub fn builder() -> PrimitiveBuilder {
        PrimitiveBuilder::new(
            "LongOnly",
            NodeSchema {
                inputs: vec![PortSpec { kind: ScalarKind::F64, firing: Firing::Any, name: "exposure".into() }],
                output: vec![FieldSpec { name: "exposure".into(), kind: ScalarKind::F64 }],
                params: vec![ParamSpec { name: "enabled".into(), kind: ScalarKind::Bool }],
            },
            |p| Box::new(LongOnly::new(p[0].bool())),   // first use of Cell::bool() in a builder
        )
    }
}

impl Node for LongOnly {
    fn lookbacks(&self) -> Vec<usize> { vec![1] }
    fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Cell]> {
        let w = ctx.f64_in(0);
        if w.is_empty() { return None; }
        self.out[0] = Cell::from_f64(if self.enabled { w[0].max(0.0) } else { w[0] });
        Some(&self.out)
    }
    fn label(&self) -> String { format!("LongOnly({})", self.enabled) }
}

Exported from aura-std/src/lib.rs (mod longonly; pub use longonly::LongOnly;).

The end-to-end bool path the cycle proves (it is not assumed): the type machinery is already in place and ratified — ScalarKind::Bool, Scalar::Bool / as_bool, Cell::from_bool / Cell::bool, and GridSpace::new's per-value kind-check that already classifies Bool (see the green random_space_new_non_numeric_slot, which shows Bool is a known, handled kind on the random side). No node has yet used a bool param, so this is the first; its node test and the momentum bool-axis sweep test are what ratify the authoring + enumeration + bootstrap path for a bool param.

(4) The --strategy selector (a pure parse fn, mirroring parse_real_args)

Before — three fixed arms (main.rs:993-995):

["sweep"] => run_sweep("sweep", false),
["sweep", "--name", n] => run_sweep(n, false),
["sweep", "--trace", n] => run_sweep(n, true),

After — one arm + a pure, unit-testable grammar; run_sweep gains a strategy:

#[derive(Clone, Copy, PartialEq, Debug)]
enum Strategy { SmaCross, Momentum }

/// `sweep [--strategy <sma|momentum>] [--name <n> | --trace <n>]`. Defaults:
/// SMA-cross, name "sweep", no persist (today's behaviour preserved). `--name`
/// and `--trace` are mutually exclusive. Pure (no I/O / exit) so it is testable.
fn parse_sweep_args(rest: &[&str]) -> Result<(Strategy, String, bool), String> { … }

// dispatch:
["sweep", rest @ ..] => match parse_sweep_args(rest) {
    Ok((strategy, name, persist)) => run_sweep(strategy, &name, persist),
    Err(msg) => { eprintln!("aura: {msg}"); std::process::exit(2); }
},

fn run_sweep(strategy: Strategy, name: &str, persist: bool) {
    let family = match strategy {
        Strategy::SmaCross => sweep_family(persist.then_some(name)),
        Strategy::Momentum => momentum_sweep_family(persist.then_some(name)),
    };
    // … unchanged: append_family + print each point's record line …
}

momentum_sweep_family mirrors sweep_family: declare the 3-axis grid via the .axis() chain, capture binder.varying_axes(), and in the closure render the key with member_key(&zip_params(&space, point), &varying). sweep_family (SMA) is refactored the same way — it drops the hardcoded sweep_member_key and uses the generic member_key with its own varying_axes() (so its member dirs change from f2s4 to signals.trend.fast.length-2_signals.trend.slow.length-4).

Seam: SweepBinder::varying_axes (engine, additive)

impl SweepBinder {
    /// The names of the axes that vary (more than one value) — the axes that
    /// distinguish one grid point from another. Declaration order; the caller
    /// derives membership, key token order comes from param-space slot order.
    pub fn varying_axes(&self) -> Vec<String> {
        self.axes.iter().filter(|(_, v)| v.len() > 1).map(|(n, _)| n.clone()).collect()
    }
}

The CLI captures it before the consuming .sweep(): let binder = bp.axis(…)….axis(…); let varying: HashSet<String> = binder.varying_axes().into_iter().collect(); binder.sweep(move |point| { … }).

Components

• aura-engine SweepBinder::varying_axes — new pure read accessor (blueprint.rs). Additive; no existing signature changes.
• aura-std LongOnly — new node + module + export. First bool-param node.
• aura-cli —
  • sanitize_component, render_value, member_key, fnv1a64, MAX_KEY (replace sweep_member_key);
  • momentum_blueprint_with_sinks, momentum_sweep_family, a momentum point grid + (re-used) synthetic stream;
  • Strategy enum, parse_sweep_args, run_sweep(strategy, name, persist);
  • sweep_family / sweep_over refactored to the generic key via varying_axes();
  • USAGE updated to show aura sweep [--strategy <sma|momentum>] [--name <n>|--trace <n>].

Data flow

Unchanged through the sweep engine (enumerate → disjoint parallel runs → SweepFamily in odometer order). The only new flow is in the per-member closure: point: &[Cell] → zip_params(&space, point) → filter by the captured varying set → member_key → portable dir component → persist_traces( "{name}/{key}", …) → runs/traces/<name>/<key>/<tap>.json. The MC (seed{seed}) and walk-forward (oos{from}) keys are already charset-conformant; they are left as-is this cycle (single-axis, trivially portable). sanitize_component is the one shared portability gate the sweep key flows through.

Error handling

• A 1-point grid (no varying axis) → member_key returns "m" (a fixed valid component) rather than the empty string. Specified, tested.
• An over-MAX_KEY key → FNV-1a fallback (h-<hex>), conformant and collision-resistant. Specified, tested with a synthetic many-axis case.
• parse_sweep_args rejects an unknown flag, a flag missing its value, an unknown strategy token, or both --name and --trace — Err(usage), surfaced as stderr + exit(2) (the CLI's convention), never a panic.
• The old fail-loud-on-missing-axis panic is removed with sweep_member_key: it guarded a hardcoded-axis assumption that no longer exists. Its two unit tests are removed with it.

Testing strategy

• member_key unit tests (aura-cli --bin aura) — the worked-examples table: charset conformance (key.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.'|'_'|'-'))), the bool case, a negative-float case, a sanitised-name case, the empty→"m" case, and an over-cap case (key length ≤ MAX_KEY, still conformant, two distinct over-cap inputs → distinct keys).
• Collision-freedom over a different-axis grid (the #105 regression) — a grid that varies an axis set other than the old trend.fast/trend.slow (e.g. the momentum grid, which varies exposure.scale and a bool) yields N distinct member keys for N points. The old f…s… key would have collapsed them; the new key does not.
• Determinism — two runs of the momentum sweep produce byte-identical member keys (the family + its keys are a pure function of the build, C1).
• The bool axis in the key — the momentum sweep's member dirs include both …_longonly.enabled-true and …_longonly.enabled-false.
• SweepBinder::varying_axes (aura-engine) — a grid with mixed singleton and multi-value axes returns exactly the multi-value names.
• LongOnly node (aura-std) — enabled=true clamps a negative input to 0 and passes a positive through; enabled=false passes both through; None until input present. Plus enabled is read from the bool param (LongOnly::builder() schema param is enabled: Bool).
• Integration (aura-cli --test cli_run, spawn-with-temp-cwd) — aura sweep --strategy momentum --trace mom persists 8 member dirs whose names match the portable charset and include the bool; aura chart 'mom/<key>' emits a uPlot page for one. The existing SMA --trace integration test is updated from the f2s4-style assertion to the new portable key form.
• Backward-compat — the existing ["sweep"] / --name / --trace forms still work through parse_sweep_args (default SMA), so the existing sweep integration + determinism tests stay green (with the key-string assertions updated to the new form).

Acceptance criteria

The project's feature-acceptance criterion (audience reaches for it; improves correctness; reintroduces no failure the core constraints forbid):

• Audience reaches for it. A researcher authoring a new strategy with new params (incl. a bool) runs the same aura sweep and gets correct, self-describing, portable per-member traces — shown by the worked momentum example. No per-strategy key code.
• Improves correctness. Closes #105: the key is collision-free for any grid, not just the built-in two-axis one. Removes the latent silent-overwrite.
• Reintroduces no forbidden failure. Determinism (C1) preserved — keys are a pure function of the grid; parallelism across members unchanged. No look-ahead, no engine merge, no per-eval emission added (C2/C3/C8). The engine change is a pure read accessor; trace state stays out of the engine (C-consistency with #104).
• Concrete, conformant, generic. Member keys match ^[A-Za-z0-9._-]+$, are case-insensitive-FS safe, length-bounded, and carry the bool axis — proven by the test corpus above over a grid that varies a different axis set than the old hardcoded two.