Aura/docs/specs/0029-run-registry-sweep-family.md

Run Registry — index & compare a sweep family — Design Spec

Date: 2026-06-10 Status: Draft — awaiting user spec review Authors: orchestrator + Claude

Goal

Cycle D of the milestone The World — parameter-space & sweep (#33). A sweep (cycle C) already produces a family of disjoint runs. This cycle makes that family — and runs across separate invocations — persistent, listable, and comparable over time: the substance of C18's run registry, which has no home in git or Gitea.

First cut (this cycle): persist + list + rank. Each run's (manifest, metrics) is appended as one record to a runs/ store; a read surface lists all stored records and ranks them by a named metric. Determinism (C1/C12) makes the record a faithful, reproducible descriptor of its run.

Deferred to later iterations / follow-up issues (named here so the cut is honest): promotion/status labels, the lineage graph (composite←signals, run←inputs), run-diff, on-demand re-derivation of full results, a run_id content digest, and Aura.toml runs-dir wiring.

This cycle also makes a contract-level change: it amends C16 (see Architecture → C16 amendment).

Architecture

C16 amendment (load-bearing, user-ratified)

C16 today carries a blanket clause: the engine workspace is "zero-external-dependency by commitment", with aura-ingest as the sole "external-dependency firewall". That blanket is struck and replaced by a considered, per-case dependency policy:

Dependencies are admitted by deliberate, per-case review — what a crate pulls in is weighed against what it buys — with particular scrutiny for anything that enters the frozen deploy artifact (C13: this bot = this commit). Well-established standard crates (serde, rayon, …) pass that review and are used wherever they do the job, including in the bot. There is no blanket zero-dependency commitment and no blanket admission; hand-rolling what a vetted standard crate already does is the anti-pattern, not the dependency.

What of C16 is unchanged: the engine/project split and the three-tier node reuse (aura-std / shared node crates / project-local nodes/). aura-ingest remains the data-source ingestion edge — it is simply no longer framed as the only place an external crate may appear.

C14's "hand-rolled JSON" rationale falls with the zero-dep clause. This cycle keeps the RunReport::to_json writer (it now drives all stdout); serde is added alongside it for the registry's storage/parse layer, and the stdout→serde unification (one shape for stored and printed records) is a follow-up. Two stdout consequences:

• aura run stdout is byte-stable (still run_sample().to_json()).
• aura sweep stdout changes shape: the per-point hand-rolled sweep_point_to_json ({params, metrics}) is retired in favour of each point's RunReport::to_json ({manifest, metrics}) — a consistency win (aura run, aura sweep, aura runs now all print the same RunReport shape) and the natural consequence of SweepPoint carrying a full report. The cycle-C params-only golden is updated accordingly.

Per-case justification for the dependencies this cycle admits (the policy requires one, sharper where it reaches the frozen artifact):

• serde + serde_json (derive feature). Buys: a typed read-path for the registry record (parse a stored line back into a RunReport) and idempotent serialization — the exact capability ranking/comparison needs, and the thing C14's writer-only JSON could never give. Pulls in: serde, serde_derive, serde_json, itoa, ryu, memchr — a tiny, ubiquitous, deterministic, heavily-audited closure. Frozen-artifact reach: the derives land on RunReport/RunManifest/RunMetrics (in aura-engine) and Timestamp (in aura-core), both linked by the frozen bot. Justified: serde is the single most-vetted crate in the ecosystem and its output is deterministic (C1-safe).

Where the registry lives

A new crate aura-registry (recommended over an aura-cli module): C18 calls the run registry a first-class concern. A dedicated crate keeps aura-engine pure compute (no filesystem I/O in the run-loop crate) and is the natural home as the registry grows (lineage, promotion/status, run-diff). It depends on aura-engine for RunReport and on serde_json for storage/parse.

The family becomes self-describing

SweepPoint today carries only RunMetrics. To index a family the registry needs each point's full (manifest, metrics). So SweepPoint carries a full RunReport — closing the manifest-per-point gap cycle C explicitly deferred to #33. The sweep's per-point closure returns RunReport instead of RunMetrics; the engine builds nothing new (the manifest is caller-supplied, as report.rs already documents).

Concrete code shapes

The user-facing program (acceptance-criterion evidence)

$ aura sweep
{"manifest":{"commit":"unknown","params":{"sma_cross.fast":2,"sma_cross.slow":4,"scale":0.5},"window":[1,6],"seed":0,"broker":"sim-optimal(pip_size=0.0001)"},"metrics":{"total_pips":-0.13,"max_drawdown":0.17,"exposure_sign_flips":1}}
… three more points …
# each point now prints its full RunReport (manifest+metrics, like `aura run`)
# — cycle C printed {params,metrics}; AND each point is appended to runs/runs.jsonl

$ aura runs list
# every record ever stored, across invocations, one per line (to_json shape)
{"manifest":{…fast=2,slow=4…},"metrics":{"total_pips":-0.13,…}}
…

$ aura runs rank total_pips
# the substance of #33 — the family compared, BEST first (total_pips: higher-is-better → desc)
{"manifest":{…fast=3,slow=4…},"metrics":{"total_pips":-0.09,…}}
{"manifest":{…fast=2,slow=5…},"metrics":{"total_pips":-0.112,…}}
…

$ aura runs rank max_drawdown
# best-first is per-metric: max_drawdown is lower-is-better → ascending (shallowest drawdown first)
{"manifest":{…fast=3,slow=4…},"metrics":{…"max_drawdown":0.09,…}}
…

$ aura runs rank bogus_metric
aura: unknown metric 'bogus_metric' (known: total_pips, max_drawdown, exposure_sign_flips)
# exit code 2 — strict, like every other aura arg error

Implementation shapes (secondary)

serde derives (before → after). In crates/aura-core/src/scalar.rs:

// before
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Timestamp(pub i64);
// after — newtype struct serializes transparently as its inner i64,
// matching to_json's [from,to] window shape
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, PartialOrd, Ord, Hash, serde::Serialize, serde::Deserialize)]
pub struct Timestamp(pub i64);

In crates/aura-engine/src/report.rs, the same two derives are added to RunMetrics, RunManifest, and RunReport (no field changes). Scalar is not derived — the registry serializes RunReport, never a raw Scalar.

SweepPoint carries a report (before → after). In crates/aura-engine/src/sweep.rs:

// before
pub struct SweepPoint { pub params: Vec<Scalar>, pub metrics: RunMetrics }
pub fn sweep<F>(space: &GridSpace, run_one: F) -> SweepFamily
where F: Fn(&[Scalar]) -> RunMetrics + Sync { … }

// after — the point keeps its typed enumeration coordinate AND the full result
pub struct SweepPoint { pub params: Vec<Scalar>, pub report: RunReport }
pub fn sweep<F>(space: &GridSpace, run_one: F) -> SweepFamily
where F: Fn(&[Scalar]) -> RunReport + Sync { … }

The thread-fan-out internals are untouched except the collected element type ((usize, RunMetrics) → (usize, RunReport)). SweepFamily is unchanged in shape ({ points: Vec<SweepPoint> }).

The registry (new crates/aura-registry/src/lib.rs).

use std::path::{Path, PathBuf};
use aura_engine::RunReport;

/// An append-only run registry over a JSONL file: one serde_json line per
/// RunReport. The stored shape is serde's; display is the caller's concern
/// (the CLI prints via RunReport::to_json for stdout consistency).
pub struct Registry { path: PathBuf }

impl Registry {
    /// Bind to a JSONL path. No I/O; the file is created lazily on first append.
    pub fn open(path: impl AsRef<Path>) -> Registry { Registry { path: path.as_ref().to_path_buf() } }

    /// Append one record as a single JSON line. Creates the file (and parent
    /// dir) if absent.
    pub fn append(&self, report: &RunReport) -> std::io::Result<()> { /* serde_json::to_string + writeln in append mode */ }

    /// Parse every non-empty line back into a typed RunReport, in file order.
    /// A missing file is an empty registry (Ok(vec![])), not an error.
    pub fn load(&self) -> Result<Vec<RunReport>, RegistryError> { /* read; serde_json::from_str per line */ }
}

/// Sort reports **best-first** by a named metric. "Best" is per-metric, fixed by
/// each metric's meaning (spec 0009): `total_pips` higher-is-better → descending;
/// `max_drawdown` (a positive magnitude) and `exposure_sign_flips` (a churn
/// count) lower-is-better → ascending. So `aura runs rank <metric>` always puts
/// the genuinely best run first, whichever metric. Ties keep file (insertion)
/// order — a stable sort over `load()`'s file-order vector. `f64` keys use
/// `partial_cmp` (total, since metrics are finite by construction). Unknown
/// metric is an error the CLI maps to exit 2.
pub fn rank_by(reports: Vec<RunReport>, metric: &str) -> Result<Vec<RunReport>, RegistryError> { … }

#[derive(Debug)]
pub enum RegistryError {
    Io(std::io::Error),
    Parse { line: usize, source: serde_json::Error },
    UnknownMetric(String),
}

CLI dispatch (before → after) in crates/aura-cli/src/main.rs:

// before
["sweep"] => print!("{}", sweep_report()),
// after — sweep_report now also persists each point; new read subcommands
["sweep"]            => print!("{}", sweep_report()),               // persists as a side effect
["runs", "list"]     => print!("{}", runs_list(&default_registry())),
["runs", "rank", m]  => match runs_rank(&default_registry(), m) {
    Ok(s)  => print!("{s}"),
    Err(e) => { eprintln!("aura: {e}"); std::process::exit(2); }
},

The per-point sweep closure returns a RunReport. The manifest's params: Vec<(String, f64)> is built by zipping the param_space() names onto the point's Vec<Scalar> values, coercing each Scalar to f64 — the manifest's params are f64-typed, but a grid point carries Scalar::I64 (SMA lengths) and Scalar::F64 (the scale):

fn scalar_as_param_f64(s: &Scalar) -> f64 {
    match s {
        Scalar::I64(n) => *n as f64,   // fast=2 -> 2.0
        Scalar::F64(f) => *f,          // scale=0.5 -> 0.5
        // numeric tuning params only; Bool/Ts never appear in a tuning grid
        other => unreachable!("non-numeric sweep param: {other:?}"),
    }
}

The fixed manifest facts — commit/window/seed/broker — are exactly those run_sample already builds. sweep_report (after iteration 3) appends each point.report to default_registry() and prints it via RunReport::to_json (the hand-rolled sweep_point_to_json is retired — the report renders itself). default_registry() binds runs/runs.jsonl under cwd.

Components

Component	Crate / file	Change
C16 amendment	docs/design/INDEX.md	strike blanket zero-dep; record per-case policy
workspace deps	Cargo.toml	add [workspace.dependencies] with serde (derive) + serde_json
Timestamp serde	crates/aura-core/src/scalar.rs	add Serialize/Deserialize derives; serde dep
report serde	crates/aura-engine/src/report.rs	derives on RunMetrics/RunManifest/RunReport; serde dep
SweepPoint	crates/aura-engine/src/sweep.rs	metrics: RunMetrics → report: RunReport; closure -> RunReport
aura-registry	crates/aura-registry/ (new)	Registry::{open, append, load} + rank_by + RegistryError
CLI subcommands	crates/aura-cli/src/main.rs	sweep persists; runs list; runs rank <metric>; USAGE

Data flow

aura sweep
  → GridSpace enumerates N points
  → sweep(): per point, fresh blueprint → bootstrap_with_params → run
             → drain sinks → summarize → RunReport{manifest, metrics}
  → SweepFamily{ points: [SweepPoint{params, report}, …] }
  → for each point: Registry::open("runs/runs.jsonl").append(&point.report)   [side effect]
  → stdout: each point.report.to_json()                                       [full RunReport shape; cycle-C params-only golden updated]

aura runs rank total_pips
  → Registry::open("runs/runs.jsonl").load() -> Vec<RunReport>   (serde_json::from_str per line)
  → rank_by(reports, "total_pips")  -> sorted desc
  → stdout: each report.to_json()

Error handling

• Append: creates the file and its parent dir on first write; an I/O failure surfaces as RegistryError::Io → the CLI prints aura: <err> and exits 2.
• Missing registry on read: load() returns Ok(vec![]) — an empty registry is not an error; aura runs list on a fresh tree prints nothing and exits 0.
• Malformed stored line: load() returns RegistryError::Parse { line, … } naming the 1-based line — a corrupt record is loud, never silently skipped.
• Unknown metric: aura runs rank bogus → RegistryError::UnknownMetric → aura: unknown metric 'bogus' (known: …) and exit 2.
• Non-finite f64: RunMetrics/manifest params are finite by construction (report.rs); serde_json would error on NaN/∞, which cannot occur here.
• Strict argv: aura runs, aura runs rank (no metric), aura sweep extra, etc. are usage errors → aura: <USAGE> and exit 2, consistent with the existing whole-vector match.

Testing strategy

• aura-core: a round-trip unit test — Timestamp serde survives to_string→from_str.
• aura-engine: RunReport serde round-trip (a built report → JSON → back, equal). sweep.rs tests updated for point.report (the closure now returns RunReport); the existing determinism/odometer/distinct-metrics properties are preserved, reading point.report.metrics.
• aura-registry: append then load round-trips N reports in order; load on a missing path is Ok(empty); a hand-written corrupt line makes load return Parse{line}; rank_by orders by each known metric and errors on an unknown one. Tests use a temp-dir path (no shared cwd state).
• aura-cli: process-level goldens with cwd set to a temp dir — aura sweep writes 4 lines to runs/runs.jsonl and prints 4 odometer lines, each now a full RunReport ({"manifest":…,"metrics":…}); the cycle-C params-only aura sweep golden is updated to the RunReport shape. A second aura sweep then aura runs list shows 8 records; aura runs rank total_pips prints them highest-first; aura runs rank bogus exits 2.

Acceptance criteria

Against aura's feature-acceptance (CLAUDE.md): the audience naturally reaches for it, it measurably improves the loop, and it reintroduces no failure the core constraints forbid.

1. Audience reaches for it. Comparing experiments over time is the heart of the research loop (C18). aura runs rank total_pips is the first time a researcher can ask "which point in my sweep — or across my sweeps — was best?" without re-running anything. The worked program above is that ask.
2. Measurable improvement / removed redundancy. Closes the manifest-per-point gap cycle C deferred (the family becomes self-describing) and the typed read-path gap (#17 — serde Deserialize is the typed handle). Retires the hand-rolled sweep_point_to_json.
3. No reintroduced failure class. Determinism (C1) holds — the registry stores a reproducible manifest; serde_json output is deterministic. The sweep stays disjoint-parallel. The new dependency is admitted under the amended C16's per-case review, justified above, not by a blanket waiver.

Recommended iteration cut (first planner handoff = iteration 1):

• Iteration 1 — foundation. C16 amendment in the ledger; [workspace.dependencies] serde/serde_json; serde derives on the report types + Timestamp; round-trip tests. No behaviour change to the CLI. Workspace green.
• Iteration 2 — engine + registry + CLI closure rethread (lands together). SweepPoint carries RunReport (closure bound -> RunReport). Changing that bound breaks its only non-test caller — the CLI's sweep_report closure (main.rs:280-289) — and the sweep.rs test callers; per the planner's caller-threading rule, all of them are rethreaded in this same iteration so cargo test --workspace stays green: the CLI closure now returns RunReport and prints via RunReport::to_json (retiring sweep_point_to_json), and the cycle-C aura sweep golden is updated to the RunReport shape here. New aura-registry crate (open/append/load/rank_by) with unit tests. No persistence side-effect yet, no new subcommands. Workspace green.
• Iteration 3 — CLI persistence + read surface. aura sweep appends each point's report to runs/runs.jsonl; aura runs list / aura runs rank <metric>; USAGE; process goldens (persist + list + rank + bogus-metric exit 2). Workspace green.

Each iteration leaves cargo build/test --workspace green: the signature-change iteration (2) threads every caller it breaks, never deferring one past its own build gate.

Out of scope (follow-up issues / later iterations): promotion/status, lineage graph, run-diff, on-demand re-derivation, run_id digest, Aura.toml runs-dir wiring, and the aura run/aura sweep stdout→serde unification.