# Run Registry — Iteration 2 (engine + registry + CLI rethread) — Implementation Plan > **Parent spec:** `docs/specs/0029-run-registry-sweep-family.md` > > **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run > this plan. Steps use `- [ ]` checkboxes for tracking. **Goal:** Make the sweep family self-describing — `SweepPoint` carries a full `RunReport` — rethreading every caller in lockstep so the workspace returns green, and add the new `aura-registry` crate (`open`/`append`/`load`/`rank_by`), built and unit-tested but not yet CLI-wired. **Architecture:** `SweepPoint.metrics: RunMetrics` becomes `SweepPoint.report: RunReport`; the `sweep` closure bound changes to `Fn(&[Scalar]) -> RunReport`. Task 1 threads all engine-internal + engine-test callers (gate scoped to `-p aura-engine`; the workspace is intentionally red until Task 2). Task 2 rewrites the CLI's `sweep_report` closure to build a per-point `RunReport`, retires the hand-rolled `sweep_point_to_json`/`json_string`/`scalar_token`, drops the now-orphaned `ParamSpec`/`SweepPoint` imports, and updates both `aura sweep` goldens to the RunReport shape (pinning the per-point manifest **params**, not the volatile git-HEAD `commit`). Task 3 adds the `aura-registry` crate. **Tech Stack:** `aura-engine` (sweep, report types), `aura-cli`, new `aura-registry` (serde_json read-path). --- **Files this plan creates or modifies:** - Modify: `crates/aura-engine/src/sweep.rs:8,94-100,113,116,131,139,143,162` — `SweepPoint` field + sweep bounds + internals + doc - Modify: `crates/aura-engine/src/sweep.rs:170-173,330-343,374-375,394,396` — engine test module (use, `run_point`, the three tests) - Modify: `crates/aura-cli/src/main.rs:11,14` — drop orphaned `ParamSpec`/`SweepPoint` imports - Modify: `crates/aura-cli/src/main.rs:209-261` — retire `sweep_point_to_json`/`json_string`/`scalar_token`; add `scalar_as_param_f64` - Modify: `crates/aura-cli/src/main.rs:263-296` — rewrite `sweep_report` closure to build per-point `RunReport`, print via `to_json` - Modify: `crates/aura-cli/src/main.rs:489-537` — remove one in-bin test, update the odometer golden to RunReport shape - Modify: `crates/aura-cli/tests/cli_run.rs:191-207` — update the process golden to RunReport shape - Create: `crates/aura-registry/Cargo.toml` — new member crate - Create: `crates/aura-registry/src/lib.rs` — `Registry`/`rank_by`/`RegistryError` + tests - Modify: `Cargo.toml` (root) — add `"crates/aura-registry"` to `members` --- ### Task 1: Engine — `SweepPoint` carries `RunReport` (+ all engine callers/tests) **Files:** - Modify: `crates/aura-engine/src/sweep.rs` > Gate is scoped to `-p aura-engine`. The downstream `aura-cli` crate will not > compile until Task 2 rethreads its closure — that is expected; do NOT run > `cargo test --workspace` at this task's gate. - [ ] **Step 1: Production import + struct field** In `crates/aura-engine/src/sweep.rs`, replace line 8: ```rust use crate::RunMetrics; ``` with: ```rust use crate::RunReport; ``` Then replace the `SweepPoint` doc + struct (lines 94-100): ```rust /// One enumerated point and the metrics its run produced. Self-describing: the /// `params` vector is the point's coordinate in `param_space()` order. #[derive(Clone, Debug, PartialEq)] pub struct SweepPoint { pub params: Vec, pub metrics: RunMetrics, } ``` with: ```rust /// One enumerated point and the full `RunReport` its run produced. /// Self-describing: `params` is the point's coordinate in `param_space()` order, /// and `report` carries the run's `(manifest, metrics)` — the unit the run /// registry indexes (C18). #[derive(Clone, Debug, PartialEq)] pub struct SweepPoint { pub params: Vec, pub report: RunReport, } ``` - [ ] **Step 2: Closure bounds + internal tuple types + family assembly** In the same file, make these five replacements: - line 113 (the `sweep` doc tail) — replace: ```rust /// no external dependency (C16). ``` with: ```rust /// via `std::thread::scope`. ``` - line 116 (`sweep` where-bound) — replace `F: Fn(&[Scalar]) -> RunMetrics + Sync,` with `F: Fn(&[Scalar]) -> RunReport + Sync,` - line 131 (`sweep_with_threads` where-bound) — replace `F: Fn(&[Scalar]) -> RunMetrics + Sync,` with `F: Fn(&[Scalar]) -> RunReport + Sync,` - line 139 — replace `let mut results: Vec<(usize, RunMetrics)> = std::thread::scope(|scope| {` with `let mut results: Vec<(usize, RunReport)> = std::thread::scope(|scope| {` - line 143 — replace `let mut local: Vec<(usize, RunMetrics)> = Vec::new();` with `let mut local: Vec<(usize, RunReport)> = Vec::new();` - line 162 — replace: ```rust .map(|(i, metrics)| SweepPoint { params: points[i].clone(), metrics }) ``` with: ```rust .map(|(i, report)| SweepPoint { params: points[i].clone(), report }) ``` - [ ] **Step 3: Engine test module — use block + `run_point` returns `RunReport`** In the `#[cfg(test)] mod tests` block, replace the `use crate::{…}` (lines 171-173): ```rust use crate::{ f64_field, summarize, BlueprintNode, Composite, Edge, OutField, ParamAlias, Role, RunMetrics, Target, }; ``` with (drop `RunMetrics`, add `RunManifest`; `RunReport` is in scope via `use super::*`): ```rust use crate::{ f64_field, summarize, BlueprintNode, Composite, Edge, OutField, ParamAlias, Role, RunManifest, Target, }; ``` Then replace `run_point` (lines 330-342): ```rust /// Build + bootstrap + run + drain + summarize one grid point. A free `fn` /// (Copy + Sync) so it serves as the `sweep` closure AND a direct reference /// for the "sweep == N independent runs" comparison. fn run_point(point: &[Scalar]) -> RunMetrics { let (bp, rx_eq, rx_ex) = composite_sma_cross_harness(); let mut h = bp .bootstrap_with_params(point.to_vec()) .expect("grid points are kind-checked against param_space"); h.run(vec![synthetic_prices()]); let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); summarize(&equity, &exposure) } ``` with (returns a `RunReport`; a minimal fixed manifest — the tests read only `.report.metrics`, and `run_point` serving as the closure means `pt.report == run_point(&pt.params)` holds by determinism): ```rust /// Build + bootstrap + run + drain + summarize one grid point into a /// `RunReport`. A free `fn` (Copy + Sync) so it serves as the `sweep` closure /// AND a direct reference for the "sweep == N independent runs" comparison. /// The manifest is a minimal fixed fixture — the metrics are the run's, and /// determinism makes `run_point` reproduce a point's report exactly. fn run_point(point: &[Scalar]) -> RunReport { let (bp, rx_eq, rx_ex) = composite_sma_cross_harness(); let mut h = bp .bootstrap_with_params(point.to_vec()) .expect("grid points are kind-checked against param_space"); h.run(vec![synthetic_prices()]); let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); RunReport { manifest: RunManifest { commit: "test".to_string(), params: Vec::new(), window: (Timestamp(0), Timestamp(0)), seed: 0, broker: "test".to_string(), }, metrics: summarize(&equity, &exposure), } } ``` - [ ] **Step 4: Engine tests — read `.report.metrics`** In `sweep_equals_n_independent_runs`, replace the loop tail (lines 373-376): ```rust for pt in &family.points { assert_eq!(pt.metrics, run_point(&pt.params)); assert!(pt.metrics.total_pips.is_finite()); } ``` with: ```rust for pt in &family.points { assert_eq!(pt.report, run_point(&pt.params)); assert!(pt.report.metrics.total_pips.is_finite()); } ``` In `distinct_points_produce_distinct_metrics`, replace lines 394 and 396: - line 394: `let first = family.points[0].metrics.total_pips;` → `let first = family.points[0].report.metrics.total_pips;` - line 396: `family.points.iter().any(|p| p.metrics.total_pips != first),` → `family.points.iter().any(|p| p.report.metrics.total_pips != first),` (`family_is_deterministic_across_thread_counts` needs no body change — it passes `run_point` as the closure and compares whole families.) - [ ] **Step 5: Engine gate (scoped)** Run: `cargo test -p aura-engine` Expected: PASS — all engine tests green, including the three sweep tests now reading `.report`. (The workspace as a whole does not build yet — `aura-cli` is rethreaded in Task 2.) --- ### Task 2: CLI — rethread the `sweep_report` closure to `RunReport`; retire the hand-rolled per-point JSON; update goldens **Files:** - Modify: `crates/aura-cli/src/main.rs` - Modify: `crates/aura-cli/tests/cli_run.rs` - [ ] **Step 1: Drop the orphaned imports** In `crates/aura-cli/src/main.rs`, replace line 11: ```rust use aura_core::{Firing, ParamSpec, Scalar, ScalarKind, Timestamp}; ``` with (drop `ParamSpec` — used only by the retired `sweep_point_to_json` + its removed test): ```rust use aura_core::{Firing, Scalar, ScalarKind, Timestamp}; ``` Then in the `use aura_engine::{…}` block, replace line 14: ```rust OutField, ParamAlias, Role, RunManifest, RunReport, SourceSpec, SweepPoint, Target, ``` with (drop `SweepPoint` — used only by the retired fn + removed test; `RunManifest`/`RunReport` stay, now also used by the new closure): ```rust OutField, ParamAlias, Role, RunManifest, RunReport, SourceSpec, Target, ``` - [ ] **Step 2: Retire the hand-rolled per-point JSON helpers, add the coercion** In `crates/aura-cli/src/main.rs`, delete the three functions `sweep_point_to_json` (lines 209-232), `json_string` (lines 234-249), and `scalar_token` (lines 251-261) in full, **replacing** that whole 209-261 span with the single coercion helper: ```rust /// Coerce a sweep point's `Scalar` value to the manifest's `f64` param type. /// A tuning grid carries only numeric params (`I64` lengths, `F64` scales). fn scalar_as_param_f64(s: &Scalar) -> f64 { match s { Scalar::I64(n) => *n as f64, Scalar::F64(f) => *f, other => unreachable!("non-numeric sweep param: {other:?}"), } } ``` - [ ] **Step 3: Rewrite `sweep_report` to build a per-point `RunReport`** In `crates/aura-cli/src/main.rs`, replace the `sweep_report` body's closure + print loop (the `let family = sweep(&grid, |point| { … });` at lines 280-289 and the `for pt in &family.points { … }` loop at lines 290-294) with: ```rust let family = sweep(&grid, |point| { let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks(); let mut h = bp .bootstrap_with_params(point.to_vec()) .expect("grid points are kind-checked against param_space"); let prices = synthetic_prices(); let window = ( prices.first().expect("non-empty stream").0, prices.last().expect("non-empty stream").0, ); h.run(vec![prices]); let equity = f64_field(&rx_eq.try_iter().collect::>(), 0); let exposure = f64_field(&rx_ex.try_iter().collect::>(), 0); let params = space .iter() .zip(point) .map(|(ps, v)| (ps.name.clone(), scalar_as_param_f64(v))) .collect(); RunReport { manifest: RunManifest { commit: option_env!("AURA_COMMIT").unwrap_or("unknown").to_string(), params, window, seed: 0, broker: "sim-optimal(pip_size=0.0001)".to_string(), }, metrics: summarize(&equity, &exposure), } }); let mut out = String::new(); for pt in &family.points { out.push_str(&pt.report.to_json()); out.push('\n'); } out ``` (The `sweep_report` doc comment at lines 263-268 may keep its wording; the closure now folds each point into a full `RunReport` rather than bare metrics — optional one-word tidy, not required.) - [ ] **Step 4: Remove the retired in-bin test, update the odometer golden** In `crates/aura-cli/src/main.rs`, delete the test `sweep_point_to_json_renders_canonical_form` in full (lines 491-509). Then replace `sweep_report_renders_four_points_in_odometer_order` (lines 511-537) with (pins the per-point manifest **params** in odometer order; the `commit` is the real git HEAD, volatile, so it is not pinned): ```rust #[test] fn sweep_report_renders_four_points_in_odometer_order() { let out = sweep_report(); let lines: Vec<&str> = out.lines().collect(); assert_eq!(lines.len(), 4, "one JSON line per grid point; got: {out:?}"); // each line is a full RunReport; the commit is the real git HEAD // (volatile), so pin the per-point manifest params (odometer order, last // axis fastest) + the metric keys, not the commit value. for line in &lines { assert!(line.starts_with(r#"{"manifest":{"commit":""#), "not a RunReport: {line}"); } assert!(lines[0].contains(r#""params":{"sma_cross.fast":2,"sma_cross.slow":4,"scale":0.5}"#), "line0: {}", lines[0]); assert!(lines[1].contains(r#""params":{"sma_cross.fast":2,"sma_cross.slow":5,"scale":0.5}"#), "line1: {}", lines[1]); assert!(lines[2].contains(r#""params":{"sma_cross.fast":3,"sma_cross.slow":4,"scale":0.5}"#), "line2: {}", lines[2]); assert!(lines[3].contains(r#""params":{"sma_cross.fast":3,"sma_cross.slow":5,"scale":0.5}"#), "line3: {}", lines[3]); for line in &lines { assert!(line.contains(r#""total_pips":"#), "missing total_pips: {line}"); assert!(line.contains(r#""max_drawdown":"#), "missing max_drawdown: {line}"); assert!(line.contains(r#""exposure_sign_flips":"#), "missing flips: {line}"); assert!(line.ends_with('}'), "line not closed: {line}"); } } ``` (`sweep_report_is_deterministic` at lines 539-543 is unchanged — still valid.) - [ ] **Step 5: Update the process golden** In `crates/aura-cli/tests/cli_run.rs`, replace the body of `sweep_prints_four_json_lines_and_exits_zero` after the `assert_eq!(stdout.lines().count(), 4, …)` line (lines 199-206): ```rust assert_eq!(stdout.lines().count(), 4, "stdout was: {stdout:?}"); // the first line is the first odometer point (fast=2, slow=4), params pinned. assert!( stdout.lines().next().unwrap().starts_with( "{\"params\":{\"sma_cross.fast\":2,\"sma_cross.slow\":4,\"scale\":0.5},\"metrics\":{" ), "got: {stdout}" ); ``` with (each line is now a full RunReport leading with the volatile `commit`; pin the manifest params, not the commit): ```rust assert_eq!(stdout.lines().count(), 4, "stdout was: {stdout:?}"); let first = stdout.lines().next().unwrap(); // each line is a full RunReport; commit is the real git HEAD (volatile), so // pin the first odometer point's manifest params, not the commit value. assert!(first.starts_with("{\"manifest\":{\"commit\":\""), "got: {stdout}"); assert!( first.contains("\"params\":{\"sma_cross.fast\":2,\"sma_cross.slow\":4,\"scale\":0.5}"), "got: {stdout}" ); ``` (`sweep_with_trailing_token_is_strict_and_exits_two` is unchanged.) - [ ] **Step 6: Workspace gate** Run: `cargo test --workspace` Expected: PASS — `aura-cli` compiles again; all tests green including the two updated sweep goldens; `run`/`graph` goldens unchanged. Run: `cargo clippy --workspace --all-targets -- -D warnings` Expected: no warnings (no orphaned `ParamSpec`/`SweepPoint` imports remain). --- ### Task 3: New `aura-registry` crate **Files:** - Create: `crates/aura-registry/Cargo.toml` - Create: `crates/aura-registry/src/lib.rs` - Modify: `Cargo.toml` (root) - [ ] **Step 1: Add the crate to the workspace members** In the root `Cargo.toml`, replace the `members` list (lines 11-17): ```toml members = [ "crates/aura-core", "crates/aura-std", "crates/aura-engine", "crates/aura-cli", "crates/aura-ingest", ] ``` with: ```toml members = [ "crates/aura-core", "crates/aura-std", "crates/aura-engine", "crates/aura-cli", "crates/aura-ingest", "crates/aura-registry", ] ``` - [ ] **Step 2: Create the crate manifest** Create `crates/aura-registry/Cargo.toml`: ```toml [package] name = "aura-registry" edition.workspace = true version.workspace = true license.workspace = true publish.workspace = true [dependencies] # the run registry's typed read-path: serde_json parses stored RunReport records # back (admitted under the amended C16 per-case policy, INDEX.md). RunReport # derives serde in aura-engine. aura-engine = { path = "../aura-engine" } serde_json = { workspace = true } [dev-dependencies] # fixtures construct RunReport/RunManifest literals, whose window needs Timestamp aura-core = { path = "../aura-core" } ``` - [ ] **Step 3: Create the registry implementation** Create `crates/aura-registry/src/lib.rs`: ```rust //! The run registry (C18): an append-only store of run records — one //! `(manifest, metrics)` `RunReport` per line in a JSONL file — with a typed //! read-path (serde) for listing and ranking runs across invocations ("compare //! experiments over time", which has no home in git or Gitea). Storage is //! serde_json; display is the caller's concern (the CLI prints via //! `RunReport::to_json`). use std::fmt; use std::fs; use std::io::Write; use std::path::{Path, PathBuf}; use aura_engine::RunReport; /// An append-only run registry over a JSONL file: one serde_json line per /// `RunReport`. pub struct Registry { path: PathBuf, } impl Registry { /// Bind to a JSONL path. No I/O — the file is created lazily on first append. pub fn open(path: impl AsRef) -> Registry { Registry { path: path.as_ref().to_path_buf() } } /// Append one record as a single JSON line, creating the file (and its parent /// directory) if absent. pub fn append(&self, report: &RunReport) -> Result<(), RegistryError> { if let Some(parent) = self.path.parent().filter(|p| !p.as_os_str().is_empty()) { fs::create_dir_all(parent)?; } // a RunReport is finite by construction (its f64 fields are finite), so // serialization is infallible here. let line = serde_json::to_string(report).expect("a finite RunReport serializes"); let mut file = fs::OpenOptions::new().create(true).append(true).open(&self.path)?; writeln!(file, "{line}")?; Ok(()) } /// Parse every non-empty line back into a typed `RunReport`, in file order. A /// missing file is an empty registry (`Ok(vec![])`), not an error. pub fn load(&self) -> Result, RegistryError> { let text = match fs::read_to_string(&self.path) { Ok(t) => t, Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()), Err(e) => return Err(RegistryError::Io(e)), }; let mut reports = Vec::new(); for (i, raw) in text.lines().enumerate() { if raw.trim().is_empty() { continue; } let report = serde_json::from_str(raw).map_err(|source| RegistryError::Parse { line: i + 1, source })?; reports.push(report); } Ok(reports) } } /// Sort `reports` **best-first** by a named metric. "Best" is per-metric, fixed /// by each metric's meaning: `total_pips` higher-is-better (descending); /// `max_drawdown` and `exposure_sign_flips` lower-is-better (ascending). A stable /// sort keeps file (insertion) order among ties; `f64` keys use `partial_cmp` /// (total, since metrics are finite by construction). An unknown metric name is a /// `RegistryError::UnknownMetric`. pub fn rank_by(mut reports: Vec, metric: &str) -> Result, RegistryError> { match metric { "total_pips" => { reports.sort_by(|a, b| b.metrics.total_pips.partial_cmp(&a.metrics.total_pips).unwrap()) } "max_drawdown" => { reports.sort_by(|a, b| a.metrics.max_drawdown.partial_cmp(&b.metrics.max_drawdown).unwrap()) } "exposure_sign_flips" => { reports.sort_by(|a, b| a.metrics.exposure_sign_flips.cmp(&b.metrics.exposure_sign_flips)) } other => return Err(RegistryError::UnknownMetric(other.to_string())), } Ok(reports) } /// What can go wrong reading or ranking the registry. #[derive(Debug)] pub enum RegistryError { /// An I/O error reading or writing the JSONL file. Io(std::io::Error), /// A stored line did not parse as a `RunReport` (1-based line number). Parse { line: usize, source: serde_json::Error }, /// `rank_by` was given a metric name it does not know. UnknownMetric(String), } impl fmt::Display for RegistryError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { RegistryError::Io(e) => write!(f, "registry i/o: {e}"), RegistryError::Parse { line, source } => { write!(f, "registry parse error at line {line}: {source}") } RegistryError::UnknownMetric(m) => write!( f, "unknown metric '{m}' (known: total_pips, max_drawdown, exposure_sign_flips)" ), } } } impl std::error::Error for RegistryError {} impl From for RegistryError { fn from(e: std::io::Error) -> Self { RegistryError::Io(e) } } #[cfg(test)] mod tests { use super::*; use aura_core::Timestamp; use aura_engine::{RunManifest, RunMetrics, RunReport}; fn report_with(total_pips: f64, max_drawdown: f64, flips: u64) -> RunReport { RunReport { manifest: RunManifest { commit: "c".to_string(), params: vec![("p".to_string(), 1.0)], window: (Timestamp(1), Timestamp(2)), seed: 0, broker: "b".to_string(), }, metrics: RunMetrics { total_pips, max_drawdown, exposure_sign_flips: flips }, } } fn temp_path(name: &str) -> PathBuf { // unique per test + per process; no external tempfile dependency std::env::temp_dir().join(format!("aura-registry-{}-{}.jsonl", std::process::id(), name)) } #[test] fn append_then_load_round_trips_in_order() { let path = temp_path("roundtrip"); let _ = fs::remove_file(&path); let reg = Registry::open(&path); let a = report_with(1.0, 0.5, 1); let b = report_with(2.0, 0.3, 2); reg.append(&a).expect("append a"); reg.append(&b).expect("append b"); assert_eq!(reg.load().expect("load"), vec![a, b]); let _ = fs::remove_file(&path); } #[test] fn load_missing_file_is_empty() { let path = temp_path("missing"); let _ = fs::remove_file(&path); let reg = Registry::open(&path); assert_eq!(reg.load().expect("load missing"), Vec::::new()); } #[test] fn corrupt_line_is_a_parse_error_with_line_number() { let path = temp_path("corrupt"); let _ = fs::remove_file(&path); let reg = Registry::open(&path); reg.append(&report_with(1.0, 0.5, 1)).expect("append"); let mut f = fs::OpenOptions::new().append(true).open(&path).expect("open"); writeln!(f, "not json").expect("write corrupt line"); drop(f); match reg.load() { Err(RegistryError::Parse { line, .. }) => assert_eq!(line, 2), other => panic!("expected Parse at line 2, got {other:?}"), } let _ = fs::remove_file(&path); } #[test] fn rank_by_orders_best_first_per_metric() { let reports = vec![ report_with(1.0, 0.9, 5), report_with(3.0, 0.1, 1), report_with(2.0, 0.5, 3), ]; let by_pips = rank_by(reports.clone(), "total_pips").expect("rank pips"); assert_eq!(by_pips[0].metrics.total_pips, 3.0); // higher-is-better -> desc let by_dd = rank_by(reports.clone(), "max_drawdown").expect("rank dd"); assert_eq!(by_dd[0].metrics.max_drawdown, 0.1); // lower-is-better -> asc let by_flips = rank_by(reports.clone(), "exposure_sign_flips").expect("rank flips"); assert_eq!(by_flips[0].metrics.exposure_sign_flips, 1); // lower-is-better -> asc } #[test] fn rank_by_unknown_metric_is_an_error() { let reports = vec![report_with(1.0, 0.5, 1)]; match rank_by(reports, "sharpe") { Err(RegistryError::UnknownMetric(m)) => assert_eq!(m, "sharpe"), other => panic!("expected UnknownMetric, got {other:?}"), } } } ``` - [ ] **Step 4: Crate gate** Run: `cargo test -p aura-registry` Expected: PASS — 5 tests (`append_then_load_round_trips_in_order`, `load_missing_file_is_empty`, `corrupt_line_is_a_parse_error_with_line_number`, `rank_by_orders_best_first_per_metric`, `rank_by_unknown_metric_is_an_error`). - [ ] **Step 5: Full workspace gate** Run: `cargo test --workspace` Expected: PASS — all crates including the new `aura-registry`. Run: `cargo clippy --workspace --all-targets -- -D warnings` Expected: no warnings, exit 0.