plan: 0029 run-registry engine + registry (iteration 2)
Iteration 2 of cycle D: SweepPoint carries a full RunReport (closure bound -> RunReport), all engine + CLI callers rethreaded in lockstep so the workspace returns green; the hand-rolled sweep_point_to_json retired (each point prints via RunReport::to_json); new aura-registry crate (open/append/load/rank_by + RegistryError), built and unit-tested but not yet CLI-wired. The aura sweep goldens pin the per-point manifest params, not the volatile git-HEAD commit. refs #33
This commit is contained in:
@@ -0,0 +1,686 @@
|
||||
# Run Registry — Iteration 2 (engine + registry + CLI rethread) — Implementation Plan
|
||||
|
||||
> **Parent spec:** `docs/specs/0029-run-registry-sweep-family.md`
|
||||
>
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
|
||||
> this plan. Steps use `- [ ]` checkboxes for tracking.
|
||||
|
||||
**Goal:** Make the sweep family self-describing — `SweepPoint` carries a full
|
||||
`RunReport` — rethreading every caller in lockstep so the workspace returns
|
||||
green, and add the new `aura-registry` crate (`open`/`append`/`load`/`rank_by`),
|
||||
built and unit-tested but not yet CLI-wired.
|
||||
|
||||
**Architecture:** `SweepPoint.metrics: RunMetrics` becomes `SweepPoint.report:
|
||||
RunReport`; the `sweep` closure bound changes to `Fn(&[Scalar]) -> RunReport`.
|
||||
Task 1 threads all engine-internal + engine-test callers (gate scoped to
|
||||
`-p aura-engine`; the workspace is intentionally red until Task 2). Task 2
|
||||
rewrites the CLI's `sweep_report` closure to build a per-point `RunReport`,
|
||||
retires the hand-rolled `sweep_point_to_json`/`json_string`/`scalar_token`, drops
|
||||
the now-orphaned `ParamSpec`/`SweepPoint` imports, and updates both `aura sweep`
|
||||
goldens to the RunReport shape (pinning the per-point manifest **params**, not
|
||||
the volatile git-HEAD `commit`). Task 3 adds the `aura-registry` crate.
|
||||
|
||||
**Tech Stack:** `aura-engine` (sweep, report types), `aura-cli`, new
|
||||
`aura-registry` (serde_json read-path).
|
||||
|
||||
---
|
||||
|
||||
**Files this plan creates or modifies:**
|
||||
|
||||
- Modify: `crates/aura-engine/src/sweep.rs:8,94-100,113,116,131,139,143,162` — `SweepPoint` field + sweep bounds + internals + doc
|
||||
- Modify: `crates/aura-engine/src/sweep.rs:170-173,330-343,374-375,394,396` — engine test module (use, `run_point`, the three tests)
|
||||
- Modify: `crates/aura-cli/src/main.rs:11,14` — drop orphaned `ParamSpec`/`SweepPoint` imports
|
||||
- Modify: `crates/aura-cli/src/main.rs:209-261` — retire `sweep_point_to_json`/`json_string`/`scalar_token`; add `scalar_as_param_f64`
|
||||
- Modify: `crates/aura-cli/src/main.rs:263-296` — rewrite `sweep_report` closure to build per-point `RunReport`, print via `to_json`
|
||||
- Modify: `crates/aura-cli/src/main.rs:489-537` — remove one in-bin test, update the odometer golden to RunReport shape
|
||||
- Modify: `crates/aura-cli/tests/cli_run.rs:191-207` — update the process golden to RunReport shape
|
||||
- Create: `crates/aura-registry/Cargo.toml` — new member crate
|
||||
- Create: `crates/aura-registry/src/lib.rs` — `Registry`/`rank_by`/`RegistryError` + tests
|
||||
- Modify: `Cargo.toml` (root) — add `"crates/aura-registry"` to `members`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Engine — `SweepPoint` carries `RunReport` (+ all engine callers/tests)
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-engine/src/sweep.rs`
|
||||
|
||||
> Gate is scoped to `-p aura-engine`. The downstream `aura-cli` crate will not
|
||||
> compile until Task 2 rethreads its closure — that is expected; do NOT run
|
||||
> `cargo test --workspace` at this task's gate.
|
||||
|
||||
- [ ] **Step 1: Production import + struct field**
|
||||
|
||||
In `crates/aura-engine/src/sweep.rs`, replace line 8:
|
||||
|
||||
```rust
|
||||
use crate::RunMetrics;
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
use crate::RunReport;
|
||||
```
|
||||
|
||||
Then replace the `SweepPoint` doc + struct (lines 94-100):
|
||||
|
||||
```rust
|
||||
/// One enumerated point and the metrics its run produced. Self-describing: the
|
||||
/// `params` vector is the point's coordinate in `param_space()` order.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct SweepPoint {
|
||||
pub params: Vec<Scalar>,
|
||||
pub metrics: RunMetrics,
|
||||
}
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
/// One enumerated point and the full `RunReport` its run produced.
|
||||
/// Self-describing: `params` is the point's coordinate in `param_space()` order,
|
||||
/// and `report` carries the run's `(manifest, metrics)` — the unit the run
|
||||
/// registry indexes (C18).
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct SweepPoint {
|
||||
pub params: Vec<Scalar>,
|
||||
pub report: RunReport,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Closure bounds + internal tuple types + family assembly**
|
||||
|
||||
In the same file, make these five replacements:
|
||||
|
||||
- line 113 (the `sweep` doc tail) — replace:
|
||||
```rust
|
||||
/// no external dependency (C16).
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
/// via `std::thread::scope`.
|
||||
```
|
||||
|
||||
- line 116 (`sweep` where-bound) — replace `F: Fn(&[Scalar]) -> RunMetrics + Sync,` with `F: Fn(&[Scalar]) -> RunReport + Sync,`
|
||||
- line 131 (`sweep_with_threads` where-bound) — replace `F: Fn(&[Scalar]) -> RunMetrics + Sync,` with `F: Fn(&[Scalar]) -> RunReport + Sync,`
|
||||
- line 139 — replace `let mut results: Vec<(usize, RunMetrics)> = std::thread::scope(|scope| {` with `let mut results: Vec<(usize, RunReport)> = std::thread::scope(|scope| {`
|
||||
- line 143 — replace `let mut local: Vec<(usize, RunMetrics)> = Vec::new();` with `let mut local: Vec<(usize, RunReport)> = Vec::new();`
|
||||
- line 162 — replace:
|
||||
```rust
|
||||
.map(|(i, metrics)| SweepPoint { params: points[i].clone(), metrics })
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
.map(|(i, report)| SweepPoint { params: points[i].clone(), report })
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Engine test module — use block + `run_point` returns `RunReport`**
|
||||
|
||||
In the `#[cfg(test)] mod tests` block, replace the `use crate::{…}` (lines 171-173):
|
||||
|
||||
```rust
|
||||
use crate::{
|
||||
f64_field, summarize, BlueprintNode, Composite, Edge, OutField, ParamAlias, Role,
|
||||
RunMetrics, Target,
|
||||
};
|
||||
```
|
||||
|
||||
with (drop `RunMetrics`, add `RunManifest`; `RunReport` is in scope via `use super::*`):
|
||||
|
||||
```rust
|
||||
use crate::{
|
||||
f64_field, summarize, BlueprintNode, Composite, Edge, OutField, ParamAlias, Role,
|
||||
RunManifest, Target,
|
||||
};
|
||||
```
|
||||
|
||||
Then replace `run_point` (lines 330-342):
|
||||
|
||||
```rust
|
||||
/// Build + bootstrap + run + drain + summarize one grid point. A free `fn`
|
||||
/// (Copy + Sync) so it serves as the `sweep` closure AND a direct reference
|
||||
/// for the "sweep == N independent runs" comparison.
|
||||
fn run_point(point: &[Scalar]) -> RunMetrics {
|
||||
let (bp, rx_eq, rx_ex) = composite_sma_cross_harness();
|
||||
let mut h = bp
|
||||
.bootstrap_with_params(point.to_vec())
|
||||
.expect("grid points are kind-checked against param_space");
|
||||
h.run(vec![synthetic_prices()]);
|
||||
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
|
||||
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
|
||||
summarize(&equity, &exposure)
|
||||
}
|
||||
```
|
||||
|
||||
with (returns a `RunReport`; a minimal fixed manifest — the tests read only
|
||||
`.report.metrics`, and `run_point` serving as the closure means
|
||||
`pt.report == run_point(&pt.params)` holds by determinism):
|
||||
|
||||
```rust
|
||||
/// Build + bootstrap + run + drain + summarize one grid point into a
|
||||
/// `RunReport`. A free `fn` (Copy + Sync) so it serves as the `sweep` closure
|
||||
/// AND a direct reference for the "sweep == N independent runs" comparison.
|
||||
/// The manifest is a minimal fixed fixture — the metrics are the run's, and
|
||||
/// determinism makes `run_point` reproduce a point's report exactly.
|
||||
fn run_point(point: &[Scalar]) -> RunReport {
|
||||
let (bp, rx_eq, rx_ex) = composite_sma_cross_harness();
|
||||
let mut h = bp
|
||||
.bootstrap_with_params(point.to_vec())
|
||||
.expect("grid points are kind-checked against param_space");
|
||||
h.run(vec![synthetic_prices()]);
|
||||
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
|
||||
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
|
||||
RunReport {
|
||||
manifest: RunManifest {
|
||||
commit: "test".to_string(),
|
||||
params: Vec::new(),
|
||||
window: (Timestamp(0), Timestamp(0)),
|
||||
seed: 0,
|
||||
broker: "test".to_string(),
|
||||
},
|
||||
metrics: summarize(&equity, &exposure),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Engine tests — read `.report.metrics`**
|
||||
|
||||
In `sweep_equals_n_independent_runs`, replace the loop tail (lines 373-376):
|
||||
|
||||
```rust
|
||||
for pt in &family.points {
|
||||
assert_eq!(pt.metrics, run_point(&pt.params));
|
||||
assert!(pt.metrics.total_pips.is_finite());
|
||||
}
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
for pt in &family.points {
|
||||
assert_eq!(pt.report, run_point(&pt.params));
|
||||
assert!(pt.report.metrics.total_pips.is_finite());
|
||||
}
|
||||
```
|
||||
|
||||
In `distinct_points_produce_distinct_metrics`, replace lines 394 and 396:
|
||||
|
||||
- line 394: `let first = family.points[0].metrics.total_pips;` → `let first = family.points[0].report.metrics.total_pips;`
|
||||
- line 396: `family.points.iter().any(|p| p.metrics.total_pips != first),` → `family.points.iter().any(|p| p.report.metrics.total_pips != first),`
|
||||
|
||||
(`family_is_deterministic_across_thread_counts` needs no body change — it passes
|
||||
`run_point` as the closure and compares whole families.)
|
||||
|
||||
- [ ] **Step 5: Engine gate (scoped)**
|
||||
|
||||
Run: `cargo test -p aura-engine`
|
||||
Expected: PASS — all engine tests green, including the three sweep tests now
|
||||
reading `.report`. (The workspace as a whole does not build yet — `aura-cli` is
|
||||
rethreaded in Task 2.)
|
||||
|
||||
---
|
||||
|
||||
### Task 2: CLI — rethread the `sweep_report` closure to `RunReport`; retire the hand-rolled per-point JSON; update goldens
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-cli/src/main.rs`
|
||||
- Modify: `crates/aura-cli/tests/cli_run.rs`
|
||||
|
||||
- [ ] **Step 1: Drop the orphaned imports**
|
||||
|
||||
In `crates/aura-cli/src/main.rs`, replace line 11:
|
||||
|
||||
```rust
|
||||
use aura_core::{Firing, ParamSpec, Scalar, ScalarKind, Timestamp};
|
||||
```
|
||||
|
||||
with (drop `ParamSpec` — used only by the retired `sweep_point_to_json` + its
|
||||
removed test):
|
||||
|
||||
```rust
|
||||
use aura_core::{Firing, Scalar, ScalarKind, Timestamp};
|
||||
```
|
||||
|
||||
Then in the `use aura_engine::{…}` block, replace line 14:
|
||||
|
||||
```rust
|
||||
OutField, ParamAlias, Role, RunManifest, RunReport, SourceSpec, SweepPoint, Target,
|
||||
```
|
||||
|
||||
with (drop `SweepPoint` — used only by the retired fn + removed test;
|
||||
`RunManifest`/`RunReport` stay, now also used by the new closure):
|
||||
|
||||
```rust
|
||||
OutField, ParamAlias, Role, RunManifest, RunReport, SourceSpec, Target,
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Retire the hand-rolled per-point JSON helpers, add the coercion**
|
||||
|
||||
In `crates/aura-cli/src/main.rs`, delete the three functions
|
||||
`sweep_point_to_json` (lines 209-232), `json_string` (lines 234-249), and
|
||||
`scalar_token` (lines 251-261) in full, **replacing** that whole 209-261 span
|
||||
with the single coercion helper:
|
||||
|
||||
```rust
|
||||
/// Coerce a sweep point's `Scalar` value to the manifest's `f64` param type.
|
||||
/// A tuning grid carries only numeric params (`I64` lengths, `F64` scales).
|
||||
fn scalar_as_param_f64(s: &Scalar) -> f64 {
|
||||
match s {
|
||||
Scalar::I64(n) => *n as f64,
|
||||
Scalar::F64(f) => *f,
|
||||
other => unreachable!("non-numeric sweep param: {other:?}"),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Rewrite `sweep_report` to build a per-point `RunReport`**
|
||||
|
||||
In `crates/aura-cli/src/main.rs`, replace the `sweep_report` body's closure +
|
||||
print loop (the `let family = sweep(&grid, |point| { … });` at lines 280-289 and
|
||||
the `for pt in &family.points { … }` loop at lines 290-294) with:
|
||||
|
||||
```rust
|
||||
let family = sweep(&grid, |point| {
|
||||
let (bp, rx_eq, rx_ex) = sample_blueprint_with_sinks();
|
||||
let mut h = bp
|
||||
.bootstrap_with_params(point.to_vec())
|
||||
.expect("grid points are kind-checked against param_space");
|
||||
let prices = synthetic_prices();
|
||||
let window = (
|
||||
prices.first().expect("non-empty stream").0,
|
||||
prices.last().expect("non-empty stream").0,
|
||||
);
|
||||
h.run(vec![prices]);
|
||||
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
|
||||
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
|
||||
let params = space
|
||||
.iter()
|
||||
.zip(point)
|
||||
.map(|(ps, v)| (ps.name.clone(), scalar_as_param_f64(v)))
|
||||
.collect();
|
||||
RunReport {
|
||||
manifest: RunManifest {
|
||||
commit: option_env!("AURA_COMMIT").unwrap_or("unknown").to_string(),
|
||||
params,
|
||||
window,
|
||||
seed: 0,
|
||||
broker: "sim-optimal(pip_size=0.0001)".to_string(),
|
||||
},
|
||||
metrics: summarize(&equity, &exposure),
|
||||
}
|
||||
});
|
||||
let mut out = String::new();
|
||||
for pt in &family.points {
|
||||
out.push_str(&pt.report.to_json());
|
||||
out.push('\n');
|
||||
}
|
||||
out
|
||||
```
|
||||
|
||||
(The `sweep_report` doc comment at lines 263-268 may keep its wording; the
|
||||
closure now folds each point into a full `RunReport` rather than bare metrics —
|
||||
optional one-word tidy, not required.)
|
||||
|
||||
- [ ] **Step 4: Remove the retired in-bin test, update the odometer golden**
|
||||
|
||||
In `crates/aura-cli/src/main.rs`, delete the test
|
||||
`sweep_point_to_json_renders_canonical_form` in full (lines 491-509).
|
||||
|
||||
Then replace `sweep_report_renders_four_points_in_odometer_order` (lines
|
||||
511-537) with (pins the per-point manifest **params** in odometer order; the
|
||||
`commit` is the real git HEAD, volatile, so it is not pinned):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn sweep_report_renders_four_points_in_odometer_order() {
|
||||
let out = sweep_report();
|
||||
let lines: Vec<&str> = out.lines().collect();
|
||||
assert_eq!(lines.len(), 4, "one JSON line per grid point; got: {out:?}");
|
||||
// each line is a full RunReport; the commit is the real git HEAD
|
||||
// (volatile), so pin the per-point manifest params (odometer order, last
|
||||
// axis fastest) + the metric keys, not the commit value.
|
||||
for line in &lines {
|
||||
assert!(line.starts_with(r#"{"manifest":{"commit":""#), "not a RunReport: {line}");
|
||||
}
|
||||
assert!(lines[0].contains(r#""params":{"sma_cross.fast":2,"sma_cross.slow":4,"scale":0.5}"#), "line0: {}", lines[0]);
|
||||
assert!(lines[1].contains(r#""params":{"sma_cross.fast":2,"sma_cross.slow":5,"scale":0.5}"#), "line1: {}", lines[1]);
|
||||
assert!(lines[2].contains(r#""params":{"sma_cross.fast":3,"sma_cross.slow":4,"scale":0.5}"#), "line2: {}", lines[2]);
|
||||
assert!(lines[3].contains(r#""params":{"sma_cross.fast":3,"sma_cross.slow":5,"scale":0.5}"#), "line3: {}", lines[3]);
|
||||
for line in &lines {
|
||||
assert!(line.contains(r#""total_pips":"#), "missing total_pips: {line}");
|
||||
assert!(line.contains(r#""max_drawdown":"#), "missing max_drawdown: {line}");
|
||||
assert!(line.contains(r#""exposure_sign_flips":"#), "missing flips: {line}");
|
||||
assert!(line.ends_with('}'), "line not closed: {line}");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`sweep_report_is_deterministic` at lines 539-543 is unchanged — still valid.)
|
||||
|
||||
- [ ] **Step 5: Update the process golden**
|
||||
|
||||
In `crates/aura-cli/tests/cli_run.rs`, replace the body of
|
||||
`sweep_prints_four_json_lines_and_exits_zero` after the
|
||||
`assert_eq!(stdout.lines().count(), 4, …)` line (lines 199-206):
|
||||
|
||||
```rust
|
||||
assert_eq!(stdout.lines().count(), 4, "stdout was: {stdout:?}");
|
||||
// the first line is the first odometer point (fast=2, slow=4), params pinned.
|
||||
assert!(
|
||||
stdout.lines().next().unwrap().starts_with(
|
||||
"{\"params\":{\"sma_cross.fast\":2,\"sma_cross.slow\":4,\"scale\":0.5},\"metrics\":{"
|
||||
),
|
||||
"got: {stdout}"
|
||||
);
|
||||
```
|
||||
|
||||
with (each line is now a full RunReport leading with the volatile `commit`; pin
|
||||
the manifest params, not the commit):
|
||||
|
||||
```rust
|
||||
assert_eq!(stdout.lines().count(), 4, "stdout was: {stdout:?}");
|
||||
let first = stdout.lines().next().unwrap();
|
||||
// each line is a full RunReport; commit is the real git HEAD (volatile), so
|
||||
// pin the first odometer point's manifest params, not the commit value.
|
||||
assert!(first.starts_with("{\"manifest\":{\"commit\":\""), "got: {stdout}");
|
||||
assert!(
|
||||
first.contains("\"params\":{\"sma_cross.fast\":2,\"sma_cross.slow\":4,\"scale\":0.5}"),
|
||||
"got: {stdout}"
|
||||
);
|
||||
```
|
||||
|
||||
(`sweep_with_trailing_token_is_strict_and_exits_two` is unchanged.)
|
||||
|
||||
- [ ] **Step 6: Workspace gate**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
Expected: PASS — `aura-cli` compiles again; all tests green including the two
|
||||
updated sweep goldens; `run`/`graph` goldens unchanged.
|
||||
|
||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: no warnings (no orphaned `ParamSpec`/`SweepPoint` imports remain).
|
||||
|
||||
---
|
||||
|
||||
### Task 3: New `aura-registry` crate
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/aura-registry/Cargo.toml`
|
||||
- Create: `crates/aura-registry/src/lib.rs`
|
||||
- Modify: `Cargo.toml` (root)
|
||||
|
||||
- [ ] **Step 1: Add the crate to the workspace members**
|
||||
|
||||
In the root `Cargo.toml`, replace the `members` list (lines 11-17):
|
||||
|
||||
```toml
|
||||
members = [
|
||||
"crates/aura-core",
|
||||
"crates/aura-std",
|
||||
"crates/aura-engine",
|
||||
"crates/aura-cli",
|
||||
"crates/aura-ingest",
|
||||
]
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```toml
|
||||
members = [
|
||||
"crates/aura-core",
|
||||
"crates/aura-std",
|
||||
"crates/aura-engine",
|
||||
"crates/aura-cli",
|
||||
"crates/aura-ingest",
|
||||
"crates/aura-registry",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create the crate manifest**
|
||||
|
||||
Create `crates/aura-registry/Cargo.toml`:
|
||||
|
||||
```toml
|
||||
[package]
|
||||
name = "aura-registry"
|
||||
edition.workspace = true
|
||||
version.workspace = true
|
||||
license.workspace = true
|
||||
publish.workspace = true
|
||||
|
||||
[dependencies]
|
||||
# the run registry's typed read-path: serde_json parses stored RunReport records
|
||||
# back (admitted under the amended C16 per-case policy, INDEX.md). RunReport
|
||||
# derives serde in aura-engine.
|
||||
aura-engine = { path = "../aura-engine" }
|
||||
serde_json = { workspace = true }
|
||||
|
||||
[dev-dependencies]
|
||||
# fixtures construct RunReport/RunManifest literals, whose window needs Timestamp
|
||||
aura-core = { path = "../aura-core" }
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Create the registry implementation**
|
||||
|
||||
Create `crates/aura-registry/src/lib.rs`:
|
||||
|
||||
```rust
|
||||
//! The run registry (C18): an append-only store of run records — one
|
||||
//! `(manifest, metrics)` `RunReport` per line in a JSONL file — with a typed
|
||||
//! read-path (serde) for listing and ranking runs across invocations ("compare
|
||||
//! experiments over time", which has no home in git or Gitea). Storage is
|
||||
//! serde_json; display is the caller's concern (the CLI prints via
|
||||
//! `RunReport::to_json`).
|
||||
|
||||
use std::fmt;
|
||||
use std::fs;
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use aura_engine::RunReport;
|
||||
|
||||
/// An append-only run registry over a JSONL file: one serde_json line per
|
||||
/// `RunReport`.
|
||||
pub struct Registry {
|
||||
path: PathBuf,
|
||||
}
|
||||
|
||||
impl Registry {
|
||||
/// Bind to a JSONL path. No I/O — the file is created lazily on first append.
|
||||
pub fn open(path: impl AsRef<Path>) -> Registry {
|
||||
Registry { path: path.as_ref().to_path_buf() }
|
||||
}
|
||||
|
||||
/// Append one record as a single JSON line, creating the file (and its parent
|
||||
/// directory) if absent.
|
||||
pub fn append(&self, report: &RunReport) -> Result<(), RegistryError> {
|
||||
if let Some(parent) = self.path.parent().filter(|p| !p.as_os_str().is_empty()) {
|
||||
fs::create_dir_all(parent)?;
|
||||
}
|
||||
// a RunReport is finite by construction (its f64 fields are finite), so
|
||||
// serialization is infallible here.
|
||||
let line = serde_json::to_string(report).expect("a finite RunReport serializes");
|
||||
let mut file = fs::OpenOptions::new().create(true).append(true).open(&self.path)?;
|
||||
writeln!(file, "{line}")?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Parse every non-empty line back into a typed `RunReport`, in file order. A
|
||||
/// missing file is an empty registry (`Ok(vec![])`), not an error.
|
||||
pub fn load(&self) -> Result<Vec<RunReport>, RegistryError> {
|
||||
let text = match fs::read_to_string(&self.path) {
|
||||
Ok(t) => t,
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()),
|
||||
Err(e) => return Err(RegistryError::Io(e)),
|
||||
};
|
||||
let mut reports = Vec::new();
|
||||
for (i, raw) in text.lines().enumerate() {
|
||||
if raw.trim().is_empty() {
|
||||
continue;
|
||||
}
|
||||
let report =
|
||||
serde_json::from_str(raw).map_err(|source| RegistryError::Parse { line: i + 1, source })?;
|
||||
reports.push(report);
|
||||
}
|
||||
Ok(reports)
|
||||
}
|
||||
}
|
||||
|
||||
/// Sort `reports` **best-first** by a named metric. "Best" is per-metric, fixed
|
||||
/// by each metric's meaning: `total_pips` higher-is-better (descending);
|
||||
/// `max_drawdown` and `exposure_sign_flips` lower-is-better (ascending). A stable
|
||||
/// sort keeps file (insertion) order among ties; `f64` keys use `partial_cmp`
|
||||
/// (total, since metrics are finite by construction). An unknown metric name is a
|
||||
/// `RegistryError::UnknownMetric`.
|
||||
pub fn rank_by(mut reports: Vec<RunReport>, metric: &str) -> Result<Vec<RunReport>, RegistryError> {
|
||||
match metric {
|
||||
"total_pips" => {
|
||||
reports.sort_by(|a, b| b.metrics.total_pips.partial_cmp(&a.metrics.total_pips).unwrap())
|
||||
}
|
||||
"max_drawdown" => {
|
||||
reports.sort_by(|a, b| a.metrics.max_drawdown.partial_cmp(&b.metrics.max_drawdown).unwrap())
|
||||
}
|
||||
"exposure_sign_flips" => {
|
||||
reports.sort_by(|a, b| a.metrics.exposure_sign_flips.cmp(&b.metrics.exposure_sign_flips))
|
||||
}
|
||||
other => return Err(RegistryError::UnknownMetric(other.to_string())),
|
||||
}
|
||||
Ok(reports)
|
||||
}
|
||||
|
||||
/// What can go wrong reading or ranking the registry.
|
||||
#[derive(Debug)]
|
||||
pub enum RegistryError {
|
||||
/// An I/O error reading or writing the JSONL file.
|
||||
Io(std::io::Error),
|
||||
/// A stored line did not parse as a `RunReport` (1-based line number).
|
||||
Parse { line: usize, source: serde_json::Error },
|
||||
/// `rank_by` was given a metric name it does not know.
|
||||
UnknownMetric(String),
|
||||
}
|
||||
|
||||
impl fmt::Display for RegistryError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
RegistryError::Io(e) => write!(f, "registry i/o: {e}"),
|
||||
RegistryError::Parse { line, source } => {
|
||||
write!(f, "registry parse error at line {line}: {source}")
|
||||
}
|
||||
RegistryError::UnknownMetric(m) => write!(
|
||||
f,
|
||||
"unknown metric '{m}' (known: total_pips, max_drawdown, exposure_sign_flips)"
|
||||
),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for RegistryError {}
|
||||
|
||||
impl From<std::io::Error> for RegistryError {
|
||||
fn from(e: std::io::Error) -> Self {
|
||||
RegistryError::Io(e)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use aura_core::Timestamp;
|
||||
use aura_engine::{RunManifest, RunMetrics, RunReport};
|
||||
|
||||
fn report_with(total_pips: f64, max_drawdown: f64, flips: u64) -> RunReport {
|
||||
RunReport {
|
||||
manifest: RunManifest {
|
||||
commit: "c".to_string(),
|
||||
params: vec![("p".to_string(), 1.0)],
|
||||
window: (Timestamp(1), Timestamp(2)),
|
||||
seed: 0,
|
||||
broker: "b".to_string(),
|
||||
},
|
||||
metrics: RunMetrics { total_pips, max_drawdown, exposure_sign_flips: flips },
|
||||
}
|
||||
}
|
||||
|
||||
fn temp_path(name: &str) -> PathBuf {
|
||||
// unique per test + per process; no external tempfile dependency
|
||||
std::env::temp_dir().join(format!("aura-registry-{}-{}.jsonl", std::process::id(), name))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn append_then_load_round_trips_in_order() {
|
||||
let path = temp_path("roundtrip");
|
||||
let _ = fs::remove_file(&path);
|
||||
let reg = Registry::open(&path);
|
||||
let a = report_with(1.0, 0.5, 1);
|
||||
let b = report_with(2.0, 0.3, 2);
|
||||
reg.append(&a).expect("append a");
|
||||
reg.append(&b).expect("append b");
|
||||
assert_eq!(reg.load().expect("load"), vec![a, b]);
|
||||
let _ = fs::remove_file(&path);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn load_missing_file_is_empty() {
|
||||
let path = temp_path("missing");
|
||||
let _ = fs::remove_file(&path);
|
||||
let reg = Registry::open(&path);
|
||||
assert_eq!(reg.load().expect("load missing"), Vec::<RunReport>::new());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn corrupt_line_is_a_parse_error_with_line_number() {
|
||||
let path = temp_path("corrupt");
|
||||
let _ = fs::remove_file(&path);
|
||||
let reg = Registry::open(&path);
|
||||
reg.append(&report_with(1.0, 0.5, 1)).expect("append");
|
||||
let mut f = fs::OpenOptions::new().append(true).open(&path).expect("open");
|
||||
writeln!(f, "not json").expect("write corrupt line");
|
||||
drop(f);
|
||||
match reg.load() {
|
||||
Err(RegistryError::Parse { line, .. }) => assert_eq!(line, 2),
|
||||
other => panic!("expected Parse at line 2, got {other:?}"),
|
||||
}
|
||||
let _ = fs::remove_file(&path);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rank_by_orders_best_first_per_metric() {
|
||||
let reports = vec![
|
||||
report_with(1.0, 0.9, 5),
|
||||
report_with(3.0, 0.1, 1),
|
||||
report_with(2.0, 0.5, 3),
|
||||
];
|
||||
let by_pips = rank_by(reports.clone(), "total_pips").expect("rank pips");
|
||||
assert_eq!(by_pips[0].metrics.total_pips, 3.0); // higher-is-better -> desc
|
||||
let by_dd = rank_by(reports.clone(), "max_drawdown").expect("rank dd");
|
||||
assert_eq!(by_dd[0].metrics.max_drawdown, 0.1); // lower-is-better -> asc
|
||||
let by_flips = rank_by(reports.clone(), "exposure_sign_flips").expect("rank flips");
|
||||
assert_eq!(by_flips[0].metrics.exposure_sign_flips, 1); // lower-is-better -> asc
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rank_by_unknown_metric_is_an_error() {
|
||||
let reports = vec![report_with(1.0, 0.5, 1)];
|
||||
match rank_by(reports, "sharpe") {
|
||||
Err(RegistryError::UnknownMetric(m)) => assert_eq!(m, "sharpe"),
|
||||
other => panic!("expected UnknownMetric, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Crate gate**
|
||||
|
||||
Run: `cargo test -p aura-registry`
|
||||
Expected: PASS — 5 tests (`append_then_load_round_trips_in_order`,
|
||||
`load_missing_file_is_empty`, `corrupt_line_is_a_parse_error_with_line_number`,
|
||||
`rank_by_orders_best_first_per_metric`, `rank_by_unknown_metric_is_an_error`).
|
||||
|
||||
- [ ] **Step 5: Full workspace gate**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
Expected: PASS — all crates including the new `aura-registry`.
|
||||
|
||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: no warnings, exit 0.
|
||||
Reference in New Issue
Block a user