plan: 0029 run-registry CLI persistence + read surface (iteration 3)

Final iteration of cycle D: aura sweep persists each point's RunReport to runs/runs.jsonl; aura runs list prints every stored record; aura runs rank <metric> prints them best-first. sweep_report splits into a production sweep_family() + a #[cfg(test)] renderer; process goldens run in a temp cwd so persistence never dirties the repo; /runs/ is git-ignored. refs #33
2026-06-10 20:33:57 +02:00
parent fe11cfea8a
commit 2811c060dc
1 changed files with 433 additions and 0 deletions
@@ -0,0 +1,433 @@
+# Run Registry — Iteration 3 (CLI persistence + read surface) — Implementation Plan
+
+> **Parent spec:** `docs/specs/0029-run-registry-sweep-family.md`
+>
+> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
+> this plan. Steps use `- [ ]` checkboxes for tracking.
+
+**Goal:** Wire `aura-registry` into the CLI: `aura sweep` persists each point's
+`RunReport` to `runs/runs.jsonl`; `aura runs list` prints every stored record;
+`aura runs rank <metric>` prints them best-first; with process-level goldens.
+
+**Architecture:** `sweep_report` is split — a production `sweep_family() ->
+SweepFamily` (the pure run) and a `#[cfg(test)] sweep_report() -> String`
+renderer kept for the in-bin tests. The `["sweep"]` arm becomes `run_sweep`,
+which persists each point via `default_registry()` and prints it. Two new arms,
+`["runs","list"]` and `["runs","rank", metric]`, read the registry back (typed,
+via serde) and print via `RunReport::to_json`. Process tests run the binary in a
+temp cwd so persistence never dirties the repo; `/runs/` is git-ignored as a
+backstop.
+
+**Tech Stack:** `aura-cli`, `aura-registry` (Registry, rank_by).
+
+---
+
+**Files this plan creates or modifies:**
+
+- Modify: `.gitignore` — ignore `/runs/`
+- Modify: `crates/aura-cli/Cargo.toml:12-15` — add the `aura-registry` dependency
+- Modify: `crates/aura-cli/src/main.rs:12-15` — import `SweepFamily` + `aura_registry::{rank_by, Registry}`
+- Modify: `crates/aura-cli/src/main.rs:225-271` — split `sweep_report` into `sweep_family` + `#[cfg(test)] sweep_report`; add `default_registry`/`run_sweep`/`runs_list`/`runs_rank`/`load_runs_or_exit`
+- Modify: `crates/aura-cli/src/main.rs:429` — extend `USAGE`
+- Modify: `crates/aura-cli/src/main.rs:440` — replace the `["sweep"]` arm + add `runs` arms
+- Test: `crates/aura-cli/tests/cli_run.rs` — temp-cwd helper, update the sweep golden, add the runs-flow / empty / strictness tests
+
+---
+
+### Task 1: Prep — git-ignore `/runs/` and add the registry dependency
+
+**Files:**
+- Modify: `.gitignore`
+- Modify: `crates/aura-cli/Cargo.toml:12-15`
+
+- [ ] **Step 1: Ignore the default registry path**
+
+In `.gitignore`, append:
+
+```
+# `aura sweep` persists run records to ./runs/runs.jsonl by default; that store
+# is local run telemetry, not a repo artifact.
+/runs/
+```
+
+- [ ] **Step 2: Add the aura-registry dependency**
+
+In `crates/aura-cli/Cargo.toml`, replace the `[dependencies]` block (lines
+12-15):
+
+```toml
+[dependencies]
+aura-core = { path = "../aura-core" }
+aura-engine = { path = "../aura-engine" }
+aura-std = { path = "../aura-std" }
+```
+
+with:
+
+```toml
+[dependencies]
+aura-core = { path = "../aura-core" }
+aura-engine = { path = "../aura-engine" }
+aura-registry = { path = "../aura-registry" }
+aura-std = { path = "../aura-std" }
+```
+
+- [ ] **Step 3: Verify the workspace still builds**
+
+Run: `cargo build --workspace`
+Expected: builds green, exit 0 (the new dependency is not yet used — that is
+fine; `unused_crate_dependencies` is allow-by-default).
+
+---
+
+### Task 2: CLI wiring — persist on sweep, list + rank subcommands (RED-first)
+
+**Files:**
+- Modify: `crates/aura-cli/tests/cli_run.rs`
+- Modify: `crates/aura-cli/src/main.rs`
+
+- [ ] **Step 1: Write the failing process tests**
+
+In `crates/aura-cli/tests/cli_run.rs`, add the temp-cwd helper just after the
+`BIN` constant (line 8):
+
+```rust
+/// A fresh, unique working directory for a process test that persists run
+/// records (so `aura sweep`'s `./runs/runs.jsonl` never dirties the repo).
+/// Unique per test + per process; no external tempfile dependency.
+fn temp_cwd(name: &str) -> std::path::PathBuf {
+    let dir = std::env::temp_dir().join(format!("aura-cli-{}-{}", std::process::id(), name));
+    let _ = std::fs::remove_dir_all(&dir);
+    std::fs::create_dir_all(&dir).expect("create temp cwd");
+    dir
+}
+```
+
+Then replace `sweep_prints_four_json_lines_and_exits_zero` (the body up to and
+including its closing `}`) — it must now run in a temp cwd, since `aura sweep`
+persists:
+
+```rust
+#[test]
+fn sweep_prints_four_json_lines_and_exits_zero() {
+    let cwd = temp_cwd("sweep4");
+    let out = Command::new(BIN).arg("sweep").current_dir(&cwd).output().expect("spawn aura sweep");
+    assert!(out.status.success(), "exit status: {:?}", out.status);
+
+    let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout");
+    assert_eq!(stdout.lines().count(), 4, "stdout was: {stdout:?}");
+    let first = stdout.lines().next().unwrap();
+    // each line is a full RunReport; commit is the real git HEAD (volatile), so
+    // pin the first odometer point's manifest params, not the commit value.
+    assert!(first.starts_with("{\"manifest\":{\"commit\":\""), "got: {stdout}");
+    assert!(
+        first.contains("\"params\":{\"sma_cross.fast\":2,\"sma_cross.slow\":4,\"scale\":0.5}"),
+        "got: {stdout}"
+    );
+    let _ = std::fs::remove_dir_all(&cwd);
+}
+```
+
+Then add the new tests at the end of the file:
+
+```rust
+#[test]
+fn runs_list_and_rank_across_invocations() {
+    let cwd = temp_cwd("runs-flow");
+    // two sweeps accumulate 8 records over time
+    for _ in 0..2 {
+        let out = Command::new(BIN).arg("sweep").current_dir(&cwd).output().expect("spawn sweep");
+        assert!(out.status.success(), "sweep exit: {:?}", out.status);
+    }
+
+    // `runs list` shows all 8 stored records
+    let list = Command::new(BIN).args(["runs", "list"]).current_dir(&cwd).output().expect("spawn list");
+    assert!(list.status.success(), "list exit: {:?}", list.status);
+    let list_out = String::from_utf8(list.stdout).expect("utf-8");
+    assert_eq!(list_out.lines().count(), 8, "list: {list_out:?}");
+    assert!(list_out.lines().all(|l| l.starts_with("{\"manifest\":{")), "list shape: {list_out:?}");
+
+    // `runs rank total_pips`: 8 lines, highest total_pips first
+    let rank = Command::new(BIN).args(["runs", "rank", "total_pips"]).current_dir(&cwd).output().expect("spawn rank");
+    assert!(rank.status.success(), "rank exit: {:?}", rank.status);
+    let rank_out = String::from_utf8(rank.stdout).expect("utf-8");
+    assert_eq!(rank_out.lines().count(), 8, "rank: {rank_out:?}");
+    let pips: Vec<f64> = rank_out
+        .lines()
+        .map(|l| {
+            let key = "\"total_pips\":";
+            let start = l.find(key).expect("line has total_pips") + key.len();
+            let tail = &l[start..];
+            let end = tail.find([',', '}']).expect("token end");
+            tail[..end].parse().expect("total_pips is an f64")
+        })
+        .collect();
+    assert!(pips.windows(2).all(|w| w[0] >= w[1]), "rank not descending: {pips:?}");
+
+    // unknown metric is a usage error
+    let bogus = Command::new(BIN).args(["runs", "rank", "bogus"]).current_dir(&cwd).output().expect("spawn bogus");
+    assert_eq!(bogus.status.code(), Some(2), "bogus exit: {:?}", bogus.status);
+
+    let _ = std::fs::remove_dir_all(&cwd);
+}
+
+#[test]
+fn runs_list_on_empty_registry_is_zero_lines_exit_zero() {
+    let cwd = temp_cwd("runs-empty");
+    let out = Command::new(BIN).args(["runs", "list"]).current_dir(&cwd).output().expect("spawn list");
+    assert!(out.status.success(), "list exit: {:?}", out.status);
+    assert_eq!(out.stdout.len(), 0, "expected no output, got: {:?}", out.stdout);
+    let _ = std::fs::remove_dir_all(&cwd);
+}
+
+#[test]
+fn runs_bare_subcommand_is_strict_and_exits_two() {
+    // `aura runs` with no/unknown sub-subcommand is a usage error (#16 strict).
+    let out = Command::new(BIN).arg("runs").output().expect("spawn aura runs");
+    assert_eq!(out.status.code(), Some(2), "exit status: {:?}", out.status);
+}
+```
+
+- [ ] **Step 2: Run the new tests to verify they fail**
+
+Run: `cargo test -p aura-cli --test cli_run runs_`
+Expected: FAIL — `runs_list_and_rank_across_invocations` and
+`runs_list_on_empty_registry_is_zero_lines_exit_zero` fail because the current
+binary has no `runs` arm, so `aura runs list` falls through to the usage-error
+path and exits 2 (the success/line-count assertions fail).
+(`runs_bare_subcommand_is_strict_and_exits_two` already passes — `aura runs` is
+already a usage error today.)
+
+- [ ] **Step 3: Add the imports**
+
+In `crates/aura-cli/src/main.rs`, replace the import block (lines 12-15):
+
+```rust
+use aura_engine::{
+    f64_field, summarize, sweep, BlueprintNode, Composite, Edge, FlatGraph, GridSpace, Harness,
+    OutField, ParamAlias, Role, RunManifest, RunReport, SourceSpec, Target,
+};
+use aura_std::{Ema, Exposure, Recorder, SimBroker, Sma, Sub};
+```
+
+with (add `SweepFamily` to the engine list; add the registry import):
+
+```rust
+use aura_engine::{
+    f64_field, summarize, sweep, BlueprintNode, Composite, Edge, FlatGraph, GridSpace, Harness,
+    OutField, ParamAlias, Role, RunManifest, RunReport, SourceSpec, SweepFamily, Target,
+};
+use aura_registry::{rank_by, Registry};
+use aura_std::{Ema, Exposure, Recorder, SimBroker, Sma, Sub};
+```
+
+- [ ] **Step 4: Split `sweep_report` into `sweep_family` + a test renderer**
+
+In `crates/aura-cli/src/main.rs`, replace the whole `sweep_report` function
+(from `fn sweep_report() -> String {` at line 225 through its closing `}` at
+line 271 — keep the doc comment at line 224 above it):
+
+```rust
+fn sweep_report() -> String {
+    let space = sample_blueprint_with_sinks().0.param_space();
+    let grid = GridSpace::new(
+        &space,
+        vec![
+            vec![Scalar::I64(2), Scalar::I64(3)], // fast  ∈ {2, 3}
+            vec![Scalar::I64(4), Scalar::I64(5)], // slow  ∈ {4, 5}
+            vec![Scalar::F64(0.5)],               // scale ∈ {0.5}
+        ],
+    )
+    .expect("the built-in grid matches the sample param-space");
+    let family = sweep(&grid, |point| {
+        let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks();
+        let mut h = bp
+            .bootstrap_with_params(point.to_vec())
+            .expect("grid points are kind-checked against param_space");
+        let prices = synthetic_prices();
+        let window = (
+            prices.first().expect("non-empty stream").0,
+            prices.last().expect("non-empty stream").0,
+        );
+        h.run(vec![prices]);
+        let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
+        let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
+        let params = space
+            .iter()
+            .zip(point)
+            .map(|(ps, v)| (ps.name.clone(), scalar_as_param_f64(v)))
+            .collect();
+        RunReport {
+            manifest: RunManifest {
+                commit: option_env!("AURA_COMMIT").unwrap_or("unknown").to_string(),
+                params,
+                window,
+                seed: 0,
+                broker: "sim-optimal(pip_size=0.0001)".to_string(),
+            },
+            metrics: summarize(&equity, &exposure),
+        }
+    });
+    let mut out = String::new();
+    for pt in &family.points {
+        out.push_str(&pt.report.to_json());
+        out.push('\n');
+    }
+    out
+}
+```
+
+with (the run becomes the production `sweep_family`; the renderer stays as a
+`#[cfg(test)]` helper so it is not dead code in a normal build):
+
+```rust
+fn sweep_family() -> SweepFamily {
+    let space = sample_blueprint_with_sinks().0.param_space();
+    let grid = GridSpace::new(
+        &space,
+        vec![
+            vec![Scalar::I64(2), Scalar::I64(3)], // fast  ∈ {2, 3}
+            vec![Scalar::I64(4), Scalar::I64(5)], // slow  ∈ {4, 5}
+            vec![Scalar::F64(0.5)],               // scale ∈ {0.5}
+        ],
+    )
+    .expect("the built-in grid matches the sample param-space");
+    sweep(&grid, |point| {
+        let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks();
+        let mut h = bp
+            .bootstrap_with_params(point.to_vec())
+            .expect("grid points are kind-checked against param_space");
+        let prices = synthetic_prices();
+        let window = (
+            prices.first().expect("non-empty stream").0,
+            prices.last().expect("non-empty stream").0,
+        );
+        h.run(vec![prices]);
+        let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
+        let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
+        let params = space
+            .iter()
+            .zip(point)
+            .map(|(ps, v)| (ps.name.clone(), scalar_as_param_f64(v)))
+            .collect();
+        RunReport {
+            manifest: RunManifest {
+                commit: option_env!("AURA_COMMIT").unwrap_or("unknown").to_string(),
+                params,
+                window,
+                seed: 0,
+                broker: "sim-optimal(pip_size=0.0001)".to_string(),
+            },
+            metrics: summarize(&equity, &exposure),
+        }
+    })
+}
+
+/// Render a sweep family as one `RunReport` JSON line per point. Test helper:
+/// production (`run_sweep`) renders *and* persists per point.
+#[cfg(test)]
+fn sweep_report() -> String {
+    let mut out = String::new();
+    for pt in &sweep_family().points {
+        out.push_str(&pt.report.to_json());
+        out.push('\n');
+    }
+    out
+}
+
+/// The default run registry: an append-only JSONL store under the current
+/// working directory. (A project-configured runs-dir via `Aura.toml` is a later
+/// refinement.)
+fn default_registry() -> Registry {
+    Registry::open("runs/runs.jsonl")
+}
+
+/// `aura sweep`: run the built-in sweep, persist each point's `RunReport` to the
+/// registry (the run record, queryable over time — C18), and print each as one
+/// JSON line.
+fn run_sweep() {
+    let reg = default_registry();
+    for pt in &sweep_family().points {
+        if let Err(e) = reg.append(&pt.report) {
+            eprintln!("aura: {e}");
+            std::process::exit(2);
+        }
+        println!("{}", pt.report.to_json());
+    }
+}
+
+/// `aura runs list`: print every stored run record, in store (over-time) order.
+fn runs_list() {
+    for report in &load_runs_or_exit() {
+        println!("{}", report.to_json());
+    }
+}
+
+/// `aura runs rank <metric>`: print the stored runs best-first by `metric`.
+fn runs_rank(metric: &str) {
+    match rank_by(load_runs_or_exit(), metric) {
+        Ok(ranked) => {
+            for report in &ranked {
+                println!("{}", report.to_json());
+            }
+        }
+        Err(e) => {
+            eprintln!("aura: {e}");
+            std::process::exit(2);
+        }
+    }
+}
+
+/// Load the registry or exit 2 with the error. A missing registry loads empty.
+fn load_runs_or_exit() -> Vec<RunReport> {
+    default_registry().load().unwrap_or_else(|e| {
+        eprintln!("aura: {e}");
+        std::process::exit(2);
+    })
+}
+```
+
+- [ ] **Step 5: Extend USAGE and the dispatch arms**
+
+In `crates/aura-cli/src/main.rs`, replace the `USAGE` constant (line 429):
+
+```rust
+const USAGE: &str = "usage: aura run [--macd] | aura graph | aura sweep";
+```
+
+with:
+
+```rust
+const USAGE: &str =
+    "usage: aura run [--macd] | aura graph | aura sweep | aura runs list | aura runs rank <metric>";
+```
+
+Then replace the `["sweep"]` dispatch arm (line 440):
+
+```rust
+        ["sweep"] => print!("{}", sweep_report()),
+```
+
+with:
+
+```rust
+        ["sweep"] => run_sweep(),
+        ["runs", "list"] => runs_list(),
+        ["runs", "rank", metric] => runs_rank(metric),
+```
+
+- [ ] **Step 6: Run the new tests to verify they pass**
+
+Run: `cargo test -p aura-cli --test cli_run runs_`
+Expected: PASS — `runs_list_and_rank_across_invocations`,
+`runs_list_on_empty_registry_is_zero_lines_exit_zero`, and
+`runs_bare_subcommand_is_strict_and_exits_two` all green.
+
+- [ ] **Step 7: Full workspace gate**
+
+Run: `cargo test --workspace`
+Expected: PASS — all crates green, including the updated `aura sweep` golden and
+the in-bin `sweep_report_*` tests (still calling the `#[cfg(test)] sweep_report`).
+
+Run: `cargo clippy --workspace --all-targets -- -D warnings`
+Expected: no warnings (no dead `sweep_report`; no unused imports).