From 53eeba5e16a8f0bfb36a13c27471a3e76b31f742 Mon Sep 17 00:00:00 2001 From: Brummel Date: Sun, 21 Jun 2026 18:58:32 +0200 Subject: [PATCH] =?UTF-8?q?audit(families-view):=20cycle=20close=200061=20?= =?UTF-8?q?=E2=80=94=20drift-clean,=20ledger=20records=20the=20comparison?= =?UTF-8?q?=20view?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Architect drift review over 99fd32b..HEAD (the housekeeping commits #99/#98 plus the 0061 cycle): code drift-clean. Engine untouched (aura-engine + aura-core diff empty — C9/C14); C21 realized faithfully (the comparison view consumes a set of runs, a family is the first producer, no orchestration-axis rebuild); C18/C10 refuse-don't-guess honoured (every new error path exits 2, never panics; all six tracing entry points guarded before persist, so name resolution is total). The only finding was a ledger-realization gap, lifted in this commit. Ledger (docs/design/INDEX.md): added a C22 amendment recording the families-comparison view as realized (#107) — aura chart overlays one tap across a family's members on a shared y-scale, union-ts spine giving one mechanism / three correct readings, the write-guard totalising name resolution; reconciled the two stale "families-comparison ... remain open" lines (the #104 note and the open-threads list) and named the deferred follow-ons (#102 run-context header, #47 multi-column tap selection, the orchestration-composability rebuild). Regression: cargo test --workspace green (no failures), clippy --workspace --all-targets -D warnings exit 0, cargo build --workspace clean — verified by the orchestrator before close. No baseline to update (aura's regression gate is the gated determinism suite, which is part of the workspace tests). Retire the cycle's ephemeral spec + plan (0061). refs #107 --- docs/design/INDEX.md | 40 +- docs/plans/0061-families-comparison-view.md | 665 -------------------- docs/specs/0061-families-comparison-view.md | 250 -------- 3 files changed, 36 insertions(+), 919 deletions(-) delete mode 100644 docs/plans/0061-families-comparison-view.md delete mode 100644 docs/specs/0061-families-comparison-view.md diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index cac092c..14f1854 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -1040,7 +1040,8 @@ targets disjoint member dirs, so it is lock-free. Opt-in: without `--trace`, stdout/registry are byte-unchanged. Because `TraceStore` resolves `/` as a subpath, `aura chart /` charts any single member with no view-side change — the write-side precondition for the family-comparison **view** -(overlay / small-multiples across members), which itself remains open. +(overlay / small-multiples across members) — now realized (#107; see the +families-comparison amendment below). **Amendment (real-data family source, #106, 8e5d14b).** The family runs gain an **opt-in real-data source axis**: `aura sweep|walkforward --real [--from @@ -1067,6 +1068,34 @@ sizes — is a *demo-strategy calibration* patch; the real answer is project-aut strategies (C9: a project crate owns its own grid), deferred to the project-env work. +**Amendment (families-comparison view, #107, 4c64feb).** The family-comparison +**view** the #104 amendment left open is now realized. `aura chart ` +classifies the name on disk (`TraceStore::name_kind`: a top-level `index.json` is a +single run; else member subdirs with `index.json` are a family; else not-found) and, +for a family, overlays **one tap** (default `equity`, `--tap` to pick) of **every +member** in one self-contained uPlot page: `build_comparison_chart_data` emits one +labelled series per member, all on a **single shared y-scale** — members measure one +identical quantity, so a shared scale is what makes them comparable, unlike the +single-run overlay of *different* taps (per-series scales). The series align on the +same union-ts spine via `join_on_ts`, so **one mechanism yields three correct +readings**: sweep/MC members share the data window (a true overlay), walk-forward +members are disjoint OOS windows (null-complementary → the stitched curve). The view +consumes a **set of runs** (`&[FamilyMember]` from `TraceStore::read_family`, sorted +by key for determinism, C1); a family is its *first producer* — the deliberate first +step toward the programmable analysis meta-level (C21) **without** rebuilding the +orchestration axes (that is its own later cycle). Name resolution is made a **total +function** by the write-guard `TraceStore::ensure_name_free`, called once per tracing +command (`run`/`run --macd`/`run --real` → Run; `sweep`/`mc`/`walkforward` → Family) +to refuse cross-kind name reuse — so the one ambiguous on-disk state (a name used by +both a run and a family) is unreachable. Every error path exits 2, never panics +(C18/C10 refuse-don't-guess). Engine untouched (C9/C14): the whole change is +`aura-registry` (the resolver + classifier + guard) + `aura-cli` (the builder + +`emit_chart` name-kind branch + `--tap`); the single-run page is byte-unchanged when +no `--tap` is given. **Still deferred:** the run-context header on the served page +(#102); multi-column tap selection (`build_comparison_chart_data` projects column 0 — +the comparison taps in scope are single-column f64; #47); and the +orchestration-composability rebuild (sweep/MC/WFO as composable meta-level tools). + ### C23 — Graph compilation and behaviour-preserving optimisation **Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic, named **blueprint** (the authoring source — nodes, composites, strategy, harness; @@ -1133,9 +1162,12 @@ load-bearing in the flat graph. #101; not egui). Settled for the first cut: raw per-tap trace persistence to disk (columnar/SoA form, C7) + serve-time `join_on_ts` alignment + a static self-contained HTML chart page (uPlot, vendored like the `render_html` - Graphviz-WASM blob), feeds overlaid or timestamp-aligned in panels. Still open: - families-comparison meta-views (walk-forward, sweep surfaces, multi-strategy / - instrument comparison), a local server, and the run/replay clock controls. + Graphviz-WASM blob), feeds overlaid or timestamp-aligned in panels. The + families-comparison **view** — `aura chart ` overlaying one tap across a + family's members on a shared y-scale — shipped (#107, see the C22 amendment). + Still open: richer comparison meta-views (sweep-surface heatmaps, cross-family / + multi-strategy comparison, the run-context header #102, multi-column tap selection + #47), a local server, and the run/replay clock controls. - **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies atop the atomic sim unit (C12), not yet designed. - **`aura new` scaffolder, the experiment-builder API, and `Aura.toml`'s diff --git a/docs/plans/0061-families-comparison-view.md b/docs/plans/0061-families-comparison-view.md deleted file mode 100644 index c7c8d94..0000000 --- a/docs/plans/0061-families-comparison-view.md +++ /dev/null @@ -1,665 +0,0 @@ -# Families-comparison view — Implementation Plan - -> **Parent spec:** `docs/specs/0061-families-comparison-view.md` -> -> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run -> this plan. Steps use `- [ ]` checkboxes for tracking. - -**Goal:** `aura chart ` overlays one tap (default `equity`) of all members -of a sweep / MC / walk-forward family in one self-contained uPlot page, with a -write-guard making name resolution a total function. - -**Architecture:** Pure read-side, three layers. `aura-registry` gains name -classification (`name_kind`), the set-of-runs resolver (`read_family`), and the -write-guard (`ensure_name_free` + a `NameTaken` error). `aura-cli` gains the -comparison `ChartData` builder (one shared-y-scale series per member, union-ts -spine via the existing `join_on_ts`), a `parse_chart_args` helper, the `--tap` -flag, the `emit_chart` name-kind branch, and the write-guard calls. `aura-engine` -is untouched. - -**Tech Stack:** Rust; `aura-registry/src/trace_store.rs`, `aura-registry/src/lib.rs`, -`aura-cli/src/main.rs`, `aura-cli/tests/cli_run.rs`. Build `cargo build --workspace`; -test `cargo test -p ` / `cargo test --workspace`; lint -`cargo clippy --workspace --all-targets -- -D warnings`. - ---- - -**Files this plan creates or modifies:** - -- Modify: `crates/aura-registry/src/trace_store.rs` — add `NameKind`, - `FamilyMember`, `WriteKind`, `name_kind`, `read_family`, `ensure_name_free`, - `TraceStoreError::NameTaken` + its Display arm; new unit tests in `mod tests`. -- Modify: `crates/aura-registry/src/lib.rs:30-31` — re-export the three new types. -- Modify: `crates/aura-cli/src/main.rs` — import the new types; add - `build_comparison_chart_data`, `filter_to_tap`, `parse_chart_args`; rewrite the - `emit_chart` body + the chart dispatch arm; update `USAGE`; add `ensure_name_free` - calls in the six tracing fns; new unit tests in `mod tests`. -- Test: `crates/aura-cli/tests/cli_run.rs` — family-chart, unknown-exit-2, - collision, and single-run-regression integration tests. - ---- - -## Task 1: Registry — name classification, family resolver, write-guard - -**Files:** -- Modify: `crates/aura-registry/src/trace_store.rs:14-131` -- Modify: `crates/aura-registry/src/lib.rs:30-31` - -- [ ] **Step 1: Write the failing unit tests** - -Append these three tests inside the existing `#[cfg(test)] mod tests` block in -`crates/aura-registry/src/trace_store.rs` (after `corrupt_tap_file_is_a_parse_error_naming_the_file`, before the closing `}` at :210). They use the existing `temp_traces_root` / `sample_manifest` / `sample_taps` helpers and `super::*`. - -```rust - #[test] - fn name_kind_classifies_run_family_and_absent() { - let root = temp_traces_root("namekind"); - let store = TraceStore::open(&root); - store.write("solo", &sample_manifest(), &sample_taps()).expect("write solo"); - store.write("fam/m1", &sample_manifest(), &sample_taps()).expect("write m1"); - assert_eq!(store.name_kind("solo"), NameKind::Run); - assert_eq!(store.name_kind("fam"), NameKind::Family); - assert_eq!(store.name_kind("ghost"), NameKind::NotFound); - let _ = fs::remove_dir_all(&root); - } - - #[test] - fn read_family_returns_members_sorted_ignoring_strays() { - let root = temp_traces_root("readfamily"); - let store = TraceStore::open(&root); - // members written out of order; read_family must sort by key. - store.write("fam/b", &sample_manifest(), &sample_taps()).expect("write b"); - store.write("fam/a", &sample_manifest(), &sample_taps()).expect("write a"); - // a stray file directly under the family dir is ignored (no index.json there). - fs::write(root.join("traces/fam/stray.txt"), "x").expect("stray"); - let members = store.read_family("fam").expect("read_family"); - let keys: Vec<&str> = members.iter().map(|m| m.key.as_str()).collect(); - assert_eq!(keys, vec!["a", "b"]); - assert_eq!(members[0].traces.taps.len(), 2); - // an absent family reads as empty (treat-as-absent). - assert!(store.read_family("ghost").expect("ghost").is_empty()); - let _ = fs::remove_dir_all(&root); - } - - #[test] - fn ensure_name_free_refuses_cross_kind_reuse() { - let root = temp_traces_root("guard"); - let store = TraceStore::open(&root); - store.write("asrun", &sample_manifest(), &sample_taps()).expect("write run"); - store.write("asfam/m1", &sample_manifest(), &sample_taps()).expect("write fam"); - assert!(matches!( - store.ensure_name_free("asrun", WriteKind::Family), - Err(TraceStoreError::NameTaken { .. }) - )); - assert!(matches!( - store.ensure_name_free("asfam", WriteKind::Run), - Err(TraceStoreError::NameTaken { .. }) - )); - // same-kind re-run and a fresh name are fine. - assert!(store.ensure_name_free("asrun", WriteKind::Run).is_ok()); - assert!(store.ensure_name_free("asfam", WriteKind::Family).is_ok()); - assert!(store.ensure_name_free("fresh", WriteKind::Run).is_ok()); - let _ = fs::remove_dir_all(&root); - } -``` - -- [ ] **Step 2: Run the tests to verify they fail (do not compile)** - -Run: `cargo test -p aura-registry` -Expected: FAIL — compile errors `cannot find type NameKind`, `cannot find type WriteKind`, `no method named name_kind/read_family/ensure_name_free`, `no variant NameTaken`. - -- [ ] **Step 3: Add the new public types** - -In `crates/aura-registry/src/trace_store.rs`, immediately after the `RunTraces` -struct (after :25), insert: - -```rust -/// The on-disk shape of a trace name: a single run, a family of members, or absent. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum NameKind { - /// `traces//index.json` is present — a single recorded run. - Run, - /// No top-level `index.json`, but ≥1 immediate subdir has one — a family. - Family, - /// Neither — no recorded run or family of this name. - NotFound, -} - -/// One member of a family, read back: its key (the member-dir name) + its traces. -#[derive(Debug)] -pub struct FamilyMember { - pub key: String, - pub traces: RunTraces, -} - -/// Which kind of write a name is about to receive — the write-guard's intent. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum WriteKind { - Run, - Family, -} -``` - -- [ ] **Step 4: Add the three methods to `impl TraceStore`** - -In the same file, inside `impl TraceStore`, after `read` (after :97), insert: - -```rust - /// Classify a name by its on-disk shape (the read-side of total name - /// resolution). Top-level `index.json` -> `Run`; else ≥1 immediate subdir with - /// `index.json` -> `Family`; else `NotFound`. - pub fn name_kind(&self, name: &str) -> NameKind { - let run_dir = self.dir.join(name); - if run_dir.join("index.json").is_file() { - return NameKind::Run; - } - if let Ok(entries) = fs::read_dir(&run_dir) { - for entry in entries.flatten() { - if entry.path().join("index.json").is_file() { - return NameKind::Family; - } - } - } - NameKind::NotFound - } - - /// Read every member of a family: each immediate subdir of `traces//` - /// that carries an `index.json`, read via `read("/")`, collected and - /// **sorted by key** (deterministic, C1). An absent/empty family reads as an - /// empty Vec (treat-as-absent); a malformed member propagates its `Parse` error. - pub fn read_family(&self, name: &str) -> Result, TraceStoreError> { - let run_dir = self.dir.join(name); - let entries = match fs::read_dir(&run_dir) { - Ok(e) => e, - Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()), - Err(e) => return Err(TraceStoreError::Io(e)), - }; - let mut members = Vec::new(); - for entry in entries { - let entry = entry?; - if !entry.path().join("index.json").is_file() { - continue; // stray file or non-member subdir - } - let key = match entry.file_name().into_string() { - Ok(k) => k, - Err(_) => continue, // non-UTF8 dir name: skip - }; - let traces = self.read(&format!("{name}/{key}"))?; - members.push(FamilyMember { key, traces }); - } - members.sort_by(|a, b| a.key.cmp(&b.key)); - Ok(members) - } - - /// The write-guard: refuse cross-kind reuse of a trace name, so the only - /// ambiguous on-disk state (a name used by both a run and a family) is - /// unreachable. Same-kind (overwrite) and a fresh name are `Ok`. Called once - /// per command before any write. - pub fn ensure_name_free(&self, name: &str, intent: WriteKind) -> Result<(), TraceStoreError> { - match (intent, self.name_kind(name)) { - (WriteKind::Run, NameKind::Family) => { - Err(TraceStoreError::NameTaken { name: name.to_string(), existing: NameKind::Family }) - } - (WriteKind::Family, NameKind::Run) => { - Err(TraceStoreError::NameTaken { name: name.to_string(), existing: NameKind::Run }) - } - _ => Ok(()), - } - } -``` - -- [ ] **Step 5: Add the `NameTaken` variant + its Display arm** - -In the `TraceStoreError` enum (:101-109), add a variant before the closing `}`: - -```rust - /// A trace name already used by the other kind (run vs family) — refuse reuse. - NameTaken { name: String, existing: NameKind }, -``` - -In `impl fmt::Display for TraceStoreError` (:111-123), add an arm in the `match`: - -```rust - TraceStoreError::NameTaken { name, existing } => { - let kind = match existing { - NameKind::Run => "run", - NameKind::Family => "family", - NameKind::NotFound => "name", - }; - write!(f, "'{name}' already used as a {kind}; pick another --trace name") - } -``` - -- [ ] **Step 6: Re-export the new types** - -In `crates/aura-registry/src/lib.rs:30-31`, extend the `pub use trace_store::{...}` -line to include the three new types. Add `FamilyMember, NameKind, WriteKind` to the -brace list (alongside the existing `RunTraces, TraceStore, TraceStoreError`). - -- [ ] **Step 7: Run the tests to verify they pass** - -Run: `cargo test -p aura-registry` -Expected: PASS — `test result: ok.` with the three new tests plus the existing -`trace_store` tests all passing. - ---- - -## Task 2: CLI — comparison view (builder, parse, dispatch, emit_chart) - -**Files:** -- Modify: `crates/aura-cli/src/main.rs:24-27` (import), `:250-272` (builder), - `:277-290` (emit_chart), `:1396-1397` (USAGE), `:1418-1419` (dispatch), - `:1453-2120` (unit tests) -- Test: `crates/aura-cli/tests/cli_run.rs` - -- [ ] **Step 1: Write the failing builder unit tests** - -Append inside `crates/aura-cli/src/main.rs`'s `#[cfg(test)] mod tests` (after the -`use super::*;` region, beside the other unit tests). They construct -`FamilyMember`s directly (its fields are `pub`): - -```rust - fn cmp_member(key: &str, ts: &[i64], vals: &[f64]) -> aura_registry::FamilyMember { - let rows: Vec<(Timestamp, Vec)> = - ts.iter().zip(vals).map(|(&t, &v)| (Timestamp(t), vec![Scalar::f64(v)])).collect(); - let tap = ColumnarTrace::from_rows("equity", &[ScalarKind::F64], &rows); - aura_registry::FamilyMember { - key: key.to_string(), - traces: aura_registry::RunTraces { - manifest: sim_optimal_manifest(vec![], (Timestamp(0), Timestamp(0)), 0, 1.0), - taps: vec![tap], - }, - } - } - - #[test] - fn comparison_overlays_one_shared_scale_series_per_member() { - let members = vec![ - cmp_member("a", &[1, 2, 3], &[10.0, 11.0, 12.0]), - cmp_member("b", &[1, 2, 3], &[20.0, 21.0, 22.0]), - ]; - let data = build_comparison_chart_data(&members, "equity").expect("builds"); - assert_eq!(data.xs, vec![1, 2, 3]); - assert_eq!(data.series.len(), 2); - assert_eq!(data.series[0].name, "a"); - assert_eq!(data.series[1].name, "b"); - // ONE shared y-scale across members (same quantity). - assert_eq!(data.series[0].y_scale_id, data.series[1].y_scale_id); - // shared ts -> dense, no nulls. - assert!(data.series[0].points.iter().all(Option::is_some)); - } - - #[test] - fn comparison_disjoint_members_are_null_complementary() { - let members = vec![ - cmp_member("oos1", &[1, 2], &[10.0, 11.0]), - cmp_member("oos2", &[3, 4], &[20.0, 21.0]), - ]; - let data = build_comparison_chart_data(&members, "equity").expect("builds"); - assert_eq!(data.xs, vec![1, 2, 3, 4]); - assert_eq!(data.series[0].points, vec![Some(10.0), Some(11.0), None, None]); - assert_eq!(data.series[1].points, vec![None, None, Some(20.0), Some(21.0)]); - } - - #[test] - fn comparison_errors_when_no_member_has_the_tap() { - let members = vec![cmp_member("a", &[1], &[1.0])]; - assert!(build_comparison_chart_data(&members, "nosuch").is_err()); - } -``` - -- [ ] **Step 2: Run the unit tests to verify they fail (do not compile)** - -Run: `cargo test -p aura-cli` -Expected: FAIL — compile error `cannot find function build_comparison_chart_data` -and `FamilyMember`/`RunTraces` unresolved in `aura_registry`. - -- [ ] **Step 3: Import the new registry types** - -In `crates/aura-cli/src/main.rs`, the `use aura_registry::{...}` block (:24-27), -add `FamilyMember` and `NameKind` to the brace list. (`WriteKind` is added in -Task 3.) - -- [ ] **Step 4: Add `build_comparison_chart_data` and `filter_to_tap`** - -In `crates/aura-cli/src/main.rs`, immediately after `build_chart_data` (after :272), -insert: - -```rust -/// Build the comparison `ChartData` for a family: one `Series` per member (the -/// chosen `tap`'s column), labelled by `member.key`, ALL sharing ONE `y_scale_id` -/// (the members measure one identical quantity, so a shared scale is what makes -/// them comparable — unlike the single-run overlay, whose series are different -/// taps). Aligned on the union-ts spine via the same `join_on_ts` build_chart_data -/// uses. `Err` if NO member carries `tap` (refuse-don't-guess). -fn build_comparison_chart_data( - members: &[FamilyMember], - tap: &str, -) -> Result { - let mut member_rows: Vec<(String, Vec<(Timestamp, Vec)>)> = Vec::new(); - for m in members { - if let Some(t) = m.traces.taps.iter().find(|t| t.tap == tap) { - member_rows.push((m.key.clone(), t.to_rows())); - } - } - if member_rows.is_empty() { - return Err(format!("no family member has a tap named '{tap}'")); - } - - let mut xs: Vec = - member_rows.iter().flat_map(|(_, r)| r.iter().map(|(t, _)| t.0)).collect(); - xs.sort_unstable(); - xs.dedup(); - - let spine: Vec<(Timestamp, Vec)> = - xs.iter().map(|&t| (Timestamp(t), Vec::new())).collect(); - let sides: Vec<&[(Timestamp, Vec)]> = - member_rows.iter().map(|(_, r)| r.as_slice()).collect(); - let joined: Vec = join_on_ts(&spine, &sides); - - // One shared y-scale across all member series (same quantity). - let y_scale_id = format!("y_cmp_{tap}"); - let mut series: Vec = Vec::new(); - for (i, (key, _)) in member_rows.iter().enumerate() { - let points: Vec> = - joined.iter().map(|r| r.sides[i].as_ref().map(|row| row[0].as_f64())).collect(); - series.push(Series { name: key.clone(), y_scale_id: y_scale_id.clone(), points }); - } - Ok(ChartData { xs, series }) -} - -/// Restrict a single-run `ChartData` to the one series named `tap`. `Err` if the -/// run has no such tap (refuse-don't-guess). Used by the `--tap` flag on the -/// single-run chart path; without `--tap` the single-run page is unchanged. -fn filter_to_tap(data: ChartData, tap: &str) -> Result { - let series: Vec = data.series.into_iter().filter(|s| s.name == tap).collect(); - if series.is_empty() { - return Err(format!("run has no tap named '{tap}'")); - } - Ok(ChartData { xs: data.xs, series }) -} -``` - -- [ ] **Step 5: Add `parse_chart_args` and rewrite `emit_chart`** - -In `crates/aura-cli/src/main.rs`, add the parser beside the other arg parsers -(e.g. after `parse_real_args`, ~:395): - -```rust -/// Parse `aura chart` args: ` [--tap ] [--panels]` in any order. -fn parse_chart_args(rest: &[&str]) -> Result<(String, Option, ChartMode), String> { - let mut name: Option = None; - let mut tap: Option = None; - let mut mode = ChartMode::Overlay; - let mut i = 0; - while i < rest.len() { - match rest[i] { - "--panels" => { - mode = ChartMode::Panels; - i += 1; - } - "--tap" => { - let t = rest.get(i + 1).ok_or("--tap needs a value")?; - tap = Some((*t).to_string()); - i += 2; - } - other if !other.starts_with("--") && name.is_none() => { - name = Some(other.to_string()); - i += 1; - } - other => return Err(format!("unexpected chart argument '{other}'")), - } - } - let name = name.ok_or("chart needs a ")?; - Ok((name, tap, mode)) -} -``` - -Replace the entire body of `emit_chart` (:277-290) with this signature + body: - -```rust -/// `aura chart [--tap ] [--panels]`: classify the name and render. A -/// single run charts all its taps (or the one `--tap` selects); a family overlays -/// one tap (default `equity`) across its members; an unknown name is a usage error -/// (stderr + exit 2), never a panic. -fn emit_chart(name: &str, tap: Option<&str>, mode: ChartMode) { - let store = TraceStore::open("runs"); - match store.name_kind(name) { - NameKind::Run => { - let traces = match store.read(name) { - Ok(t) => t, - Err(e) => { - eprintln!("aura: {e}"); - std::process::exit(2); - } - }; - let mut data = build_chart_data(traces); - if let Some(t) = tap { - data = match filter_to_tap(data, t) { - Ok(d) => d, - Err(e) => { - eprintln!("aura: {e}"); - std::process::exit(2); - } - }; - } - print!("{}", render::render_chart_html(&data, mode)); - } - NameKind::Family => { - let members = match store.read_family(name) { - Ok(m) => m, - Err(e) => { - eprintln!("aura: {e}"); - std::process::exit(2); - } - }; - let data = match build_comparison_chart_data(&members, tap.unwrap_or("equity")) { - Ok(d) => d, - Err(e) => { - eprintln!("aura: {e}"); - std::process::exit(2); - } - }; - print!("{}", render::render_chart_html(&data, mode)); - } - NameKind::NotFound => { - eprintln!( - "aura: no recorded run or family '{name}' under runs/traces \ - (run `aura run --trace {name}` or `aura sweep --trace {name}` first)" - ); - std::process::exit(2); - } - } -} -``` - -- [ ] **Step 6: Rewire the chart dispatch + USAGE** - -In `main()`, replace the two chart arms (:1418-1419): - -```rust - ["chart", name] => emit_chart(name, ChartMode::Overlay), - ["chart", name, "--panels"] => emit_chart(name, ChartMode::Panels), -``` - -with one rest-match arm: - -```rust - ["chart", rest @ ..] => match parse_chart_args(rest) { - Ok((name, tap, mode)) => emit_chart(&name, tap.as_deref(), mode), - Err(msg) => { - eprintln!("aura: {msg}"); - std::process::exit(2); - } - }, -``` - -In the `USAGE` const (:1397), change the token `aura chart [--panels]` to -`aura chart [--tap ] [--panels]`. - -- [ ] **Step 7: Run unit tests + lint to verify green** - -Run: `cargo test -p aura-cli` -Expected: PASS — the three new builder tests plus all existing `aura-cli` tests -pass (the existing `chart_*` integration tests stay green: single-run path -unchanged when no `--tap`). - -Run: `cargo clippy -p aura-cli --all-targets -- -D warnings` -Expected: exit 0 (no dead-code: every new fn is wired through `emit_chart`). - -- [ ] **Step 8: Write the family-chart + regression integration tests** - -Append to `crates/aura-cli/tests/cli_run.rs` (using the existing `BIN` / `temp_cwd`): - -```rust -/// Property: `aura chart ` overlays one series per family member, labelled -/// by member key. The built-in sweep grid is fast∈{2,3} × slow∈{4,5} = 4 members; -/// each member key appears as a series name in the injected AURA_TRACES. -#[test] -fn chart_of_a_family_overlays_one_series_per_member() { - let cwd = temp_cwd("chart-family"); - let swept = Command::new(BIN) - .args(["sweep", "--trace", "fam"]) - .current_dir(&cwd) - .output() - .expect("spawn sweep --trace"); - assert!(swept.status.success(), "sweep --trace exit: {:?}", swept.status); - - let chart = Command::new(BIN).args(["chart", "fam"]).current_dir(&cwd).output().expect("spawn chart fam"); - assert!(chart.status.success(), "chart family exit: {:?}", chart.status); - let html = String::from_utf8(chart.stdout).expect("utf-8 html"); - assert!(html.contains("window.AURA_TRACES = {"), "AURA_TRACES not injected"); - for key in [ - "signals.trend.fast.length-2_signals.trend.slow.length-4", - "signals.trend.fast.length-2_signals.trend.slow.length-5", - "signals.trend.fast.length-3_signals.trend.slow.length-4", - "signals.trend.fast.length-3_signals.trend.slow.length-5", - ] { - assert!(html.contains(key), "member series '{key}' missing from the family chart"); - } - let _ = std::fs::remove_dir_all(&cwd); -} - -/// Property: the single-run chart path is unchanged — no `--tap` shows ALL taps -/// (equity + exposure); `--tap equity` filters to one. A regression net for the -/// "single-run byte-unchanged when no --tap" acceptance bullet. -#[test] -fn chart_single_run_tap_filter_keeps_only_the_named_tap() { - let cwd = temp_cwd("chart-tap"); - let traced = Command::new(BIN).args(["run", "--trace", "solo"]).current_dir(&cwd).output().expect("spawn run --trace"); - assert!(traced.status.success(), "run --trace exit: {:?}", traced.status); - - // no --tap: both taps present (unchanged behaviour). - let all = Command::new(BIN).args(["chart", "solo"]).current_dir(&cwd).output().expect("spawn chart"); - assert!(all.status.success()); - let all_html = String::from_utf8(all.stdout).expect("utf-8"); - assert!(all_html.contains("\"equity\""), "equity series missing"); - assert!(all_html.contains("\"exposure\""), "exposure series missing"); - - // --tap equity: only equity remains. - let one = Command::new(BIN).args(["chart", "solo", "--tap", "equity"]).current_dir(&cwd).output().expect("spawn chart --tap"); - assert!(one.status.success(), "chart --tap exit: {:?}", one.status); - let one_html = String::from_utf8(one.stdout).expect("utf-8"); - assert!(one_html.contains("\"equity\""), "equity series missing under --tap"); - assert!(!one_html.contains("\"exposure\""), "--tap equity must drop exposure"); - - let _ = std::fs::remove_dir_all(&cwd); -} -``` - -- [ ] **Step 9: Run the integration tests to verify they pass** - -Run: `cargo test -p aura-cli --test cli_run` -Expected: PASS — the two new tests plus all existing `cli_run` tests pass. - ---- - -## Task 3: CLI — write-guard calls + collision tests - -**Files:** -- Modify: `crates/aura-cli/src/main.rs:24-27` (import `WriteKind`); the six tracing - fns (`run_sample`, `run_sample_real`, `run_macd`, `run_sweep`, `run_walkforward`, - `run_mc`) -- Test: `crates/aura-cli/tests/cli_run.rs` - -- [ ] **Step 1: Write the failing collision integration tests** - -Append to `crates/aura-cli/tests/cli_run.rs`: - -```rust -/// Property: a trace name cannot be reused across kinds — the write-guard refuses -/// it (exit 2), keeping `aura chart ` resolution a total function. -#[test] -fn trace_name_collision_across_kinds_is_refused() { - let cwd = temp_cwd("collision"); - - // run then sweep on the same name -> the sweep is refused. - let run = Command::new(BIN).args(["run", "--trace", "t"]).current_dir(&cwd).output().expect("spawn run --trace"); - assert!(run.status.success(), "run --trace exit: {:?}", run.status); - let sweep = Command::new(BIN).args(["sweep", "--trace", "t"]).current_dir(&cwd).output().expect("spawn sweep --trace"); - assert_eq!(sweep.status.code(), Some(2), "sweep onto a run name must exit 2"); - - // reverse: sweep then run on a fresh name -> the run is refused. - let sweep2 = Command::new(BIN).args(["sweep", "--trace", "u"]).current_dir(&cwd).output().expect("spawn sweep --trace u"); - assert!(sweep2.status.success(), "sweep --trace u exit: {:?}", sweep2.status); - let run2 = Command::new(BIN).args(["run", "--trace", "u"]).current_dir(&cwd).output().expect("spawn run --trace u"); - assert_eq!(run2.status.code(), Some(2), "run onto a family name must exit 2"); - - let _ = std::fs::remove_dir_all(&cwd); -} -``` - -- [ ] **Step 2: Run to verify it fails** - -Run: `cargo test -p aura-cli --test cli_run trace_name_collision_across_kinds_is_refused` -Expected: FAIL — without the guard, the second write succeeds, so the status code -is `Some(0)`, not `Some(2)`. - -- [ ] **Step 3: Import `WriteKind`** - -In `crates/aura-cli/src/main.rs`, add `WriteKind` to the `use aura_registry::{...}` -brace list (:24-27). - -- [ ] **Step 4: Guard the single-run tracing paths** - -In each of `run_sample`, `run_sample_real`, `run_macd` (all take `trace: -Option<&str>`), add at the very top of the function body, before any run work: - -```rust - if let Some(n) = trace { - if let Err(e) = TraceStore::open("runs").ensure_name_free(n, WriteKind::Run) { - eprintln!("aura: {e}"); - std::process::exit(2); - } - } -``` - -- [ ] **Step 5: Guard the family tracing paths** - -In each of `run_sweep`, `run_walkforward`, `run_mc`, add at the very top of the -function body (they take `name: &str` + `persist: bool`), before the family build: - -```rust - if persist { - if let Err(e) = TraceStore::open("runs").ensure_name_free(name, WriteKind::Family) { - eprintln!("aura: {e}"); - std::process::exit(2); - } - } -``` - -- [ ] **Step 6: Run the collision test to verify it passes** - -Run: `cargo test -p aura-cli --test cli_run trace_name_collision_across_kinds_is_refused` -Expected: PASS. - -- [ ] **Step 7: Full workspace gate** - -Run: `cargo test --workspace` -Expected: PASS — no failures across all crates. - -Run: `cargo clippy --workspace --all-targets -- -D warnings` -Expected: exit 0. - -Run: `cargo build --workspace` -Expected: clean build. diff --git a/docs/specs/0061-families-comparison-view.md b/docs/specs/0061-families-comparison-view.md deleted file mode 100644 index 6b44a36..0000000 --- a/docs/specs/0061-families-comparison-view.md +++ /dev/null @@ -1,250 +0,0 @@ -# Families-comparison view — Design Spec - -**Date:** 2026-06-21 -**Status:** Draft — awaiting user spec review -**Authors:** orchestrator + Claude -**Reference issue:** #107 (Brummel/Aura) - -## Goal - -Give `aura chart ` a **cross-member comparison view**: overlay one tap -(default `equity`) of **all members of a family** in a single self-contained -uPlot page, consistent across sweep / Monte-Carlo / walk-forward families. Today -`aura chart ` charts a single run, and `aura chart /` a single -member; the family-as-a-whole comparison that the single-run trace charts (#101) -deferred is what this cycle adds. - -The view is **decoupled from family production**: it consumes a *set of runs to -compare*; a family is the first producer of such a set. This is the first concrete -step toward the programmable analysis meta-level (C21) — the orchestration axes -becoming composable tools that feed a general comparable-run-set — **without** -rebuilding those axes this cycle. - -## Architecture - -A pure read-side addition, the same three-layer split as the single-run chart -(#101 / cycle 0056): - -- **`aura-engine`** — **untouched** (headless serializer, C14). `join_on_ts` and - `ColumnarTrace` already perform the timestamp alignment the comparison needs. -- **`aura-registry`** — owner of the trace store + families lineage (C18). Gains - the *set-of-runs resolver* (`read_family`), the *name classifier* (`name_kind`), - and the *write-guard* (`ensure_name_free`). This is the C21-pointing seam: a - comparison set is `Vec`; a family is its first producer, and future - meta-level producers dock at the same shape. -- **`aura-cli`** — gains the comparison `ChartData` builder, the `emit_chart` - branch on name kind, and `--tap` parsing; **reuses `render_chart_html` - unchanged** (overlay default, `--panels` small-multiples). Calls the write-guard - once per tracing command. - -The whole change lives in `aura-registry` + `aura-cli` (C9 — exactly like #101). - -## Concrete code shapes - -### User-facing program (the acceptance-criterion evidence) - -```text -# A researcher sweeps a strategy, tracing every member (existing): -$ aura sweep --strategy sma --trace grid -{"family_id":"grid-1","report":{"manifest":{"commit":...}}} # one line per member -... - -# NEW — chart the whole family in one comparison view: -$ aura chart grid > grid.html -# grid.html: a self-contained uPlot page; one equity curve PER MEMBER, all on -# ONE shared y-scale (pips), each labelled by its member_key (cycle 0059's -# portable key, e.g. "longonly.enabled-false"), aligned on the union-ts spine. - -$ aura chart grid --tap exposure # overlay a different tap across members -$ aura chart grid --panels # small-multiples: one mini-chart per member -$ aura chart grid/ # UNCHANGED: a single member (subpath → single-run path) -$ aura chart somerun # UNCHANGED: a single run, all its taps overlaid - -# Refuse-don't-guess (all exit 2, never panic): -$ aura chart nope # no recorded run/family 'nope' -$ aura chart grid --tap nosuch # family members have no tap 'nosuch' -$ aura run --trace grid # 'grid' is already a family — refuse to reuse the name -``` - -The default — no `--tap` — differs by path and is deliberately so: a single run -shows **all** its taps overlaid (unchanged); a family overlays **one** tap -(`equity`), because all-taps × all-members would be unreadable. `--tap ` -restricts to one tap on **both** paths. - -### Implementation shape (secondary) - -**`aura-registry/src/trace_store.rs`** — new public surface beside `read`/`write`: - -```rust -// NEW: the on-disk shape of a name. -pub enum NameKind { Run, Family, NotFound } - -// NEW: one member of a family, read back. -pub struct FamilyMember { pub key: String, pub traces: RunTraces } - -// NEW: which kind a write intends, for the guard. -pub enum WriteKind { Run, Family } - -impl TraceStore { - // top-level index.json present -> Run - // else ≥1 immediate subdir has index.json -> Family - // else -> NotFound - pub fn name_kind(&self, name: &str) -> NameKind { ... } - - // Enumerate immediate subdirs of traces// that contain index.json, - // read each via the existing read("/"), collect FamilyMember - // { key = subdir name, traces }, SORTED BY key (deterministic, C1). - // A member subdir whose index.json exists but a tap file is malformed - // propagates TraceStoreError::Parse{file,..}. - pub fn read_family(&self, name: &str) -> Result, TraceStoreError> { ... } - - // The write-guard. Refuse cross-kind name reuse: - // WriteKind::Run && name_kind == Family -> Err(NameTaken{..}) - // WriteKind::Family && name_kind == Run -> Err(NameTaken{..}) - // Same-kind (or NotFound) -> Ok(()). Called ONCE per command, before any write. - pub fn ensure_name_free(&self, name: &str, intent: WriteKind) -> Result<(), TraceStoreError> { ... } -} - -// NEW error variant; Display: "'' already used as a run" / "... as a family". -// TraceStoreError::NameTaken { name: String, existing: NameKind } -``` - -**`aura-cli/src/main.rs`** — branch the chart verb; guard the tracing commands: - -```rust -// before: -["chart", name] => emit_chart(name, ChartMode::Overlay), -["chart", name, "--panels"] => emit_chart(name, ChartMode::Panels), -// after: a parse_chart_args helper (idiom of parse_sweep_args/parse_real_args), -// accepting ` [--tap ] [--panels]` in any order. -["chart", rest @ ..] => match parse_chart_args(rest) { - Ok((name, tap, mode)) => emit_chart(&name, tap.as_deref(), mode), - Err(msg) => { eprintln!("aura: {msg}"); std::process::exit(2); } -}, - -// emit_chart gains the tap selector and branches on name kind: -fn emit_chart(name: &str, tap: Option<&str>, mode: ChartMode) { - let store = TraceStore::open("runs"); - match store.name_kind(name) { - NameKind::Run => { - let data = build_chart_data(store.read(name)?); // existing path - let data = match tap { Some(t) => filter_to_tap(data, t)?, None => data }; - print!("{}", render::render_chart_html(&data, mode)); - } - NameKind::Family => { - let members = store.read_family(name)?; // propagate Parse → exit 2 - let data = build_comparison_chart_data(&members, tap.unwrap_or("equity"))?; - print!("{}", render::render_chart_html(&data, mode)); - } - NameKind::NotFound => { eprintln!("aura: no recorded run/family '{name}' ..."); exit(2); } - } -} - -// NEW: one Series per member (the chosen tap), labelled by member.key, all sharing -// ONE y-scale (same quantity), aligned on the union-ts spine via the existing -// join_on_ts — the mirror of build_chart_data with members-as-sides instead of -// taps-as-sides. Err if NO member carries `tap` (refuse-don't-guess). -fn build_comparison_chart_data(members: &[FamilyMember], tap: &str) -> Result { ... } -``` - -The write-guard call sites (once per command, when `--trace` is set): the single-run -trace path (`run`/`run --macd`/`run --real`) calls `ensure_name_free(name, -WriteKind::Run)`; the family paths (`run_sweep`/`run_mc`/`run_walkforward`) call -`ensure_name_free(name, WriteKind::Family)` **before** the member loop. On `Err`, -print and `exit(2)`. - -## Components - -1. **`TraceStore::name_kind`** — classifier; the read-side half of the total - name-resolution function. -2. **`TraceStore::read_family`** — the set-of-runs resolver; members sorted by key. -3. **`TraceStore::ensure_name_free`** — the write-guard; makes the only ambiguous - on-disk state (a name used by both a run and a family) unreachable. -4. **`build_comparison_chart_data`** (cli) — one shared-y-scale series per member. -5. **`emit_chart` branch + `parse_chart_args` + `--tap`** (cli) — the CLI surface. - -## Data flow - -```text -aura chart grid [--tap equity] [--panels] - → name_kind("grid") = Family - → read_family("grid") = [FamilyMember{key:"…-2_…-4", traces}, …] (sorted) - → build_comparison_chart_data(members, "equity"): - xs = sorted union of every member's equity-tap timestamps - per member: ONE Series over xs (null where absent, via join_on_ts), - name = member.key, y_scale_id = one shared id - → render_chart_html(&ChartData, mode) → self-contained uPlot page → stdout -``` - -**Cross-kind consistency from one mechanism.** Sweep/MC members share the data -window → the union spine is their shared axis → a true overlay. Walk-forward -members are *disjoint* OOS windows → each null outside its segment → the same union -spine renders them as the *stitched* curve. The ordinal-x default (cycle 0057) -carries both (index over the union spine). No per-family-kind special-casing — one -builder, three correct readings. - -## Error handling - -Refuse-don't-guess throughout (C10/C18); every failure is `exit 2`, never a panic: - -- `aura chart ` → `name_kind` = `NotFound` → exit 2. -- A `traces//` directory with no top-level `index.json` and no member subdir - carrying `index.json` → `NotFound` → exit 2 (empty/ill-formed family treated as - absent — the same treat-as-absent discipline as `read`'s `NotFound`). -- A family member whose `index.json` is present but a tap file is malformed → - `TraceStoreError::Parse{file,..}` propagated → exit 2, naming the file. -- `--tap ` where **no** member carries tap `t` → exit 2, naming the tap (members - of one family share a harness, so a tap is present in all or none). -- Write-guard cross-kind collision → `TraceStoreError::NameTaken` → exit 2, - `"'' already used as a run/family; pick another --trace name"`. - -## Testing strategy - -**`aura-registry` unit (`trace_store.rs`):** -- `read_family` over a constructed family of 2–3 member subdirs → members **sorted - by key**; a stray non-`index.json` file/dir is ignored; a dir with no member - `index.json` → empty; round-trips with `write`. -- `name_kind` → `Run` for a single-run dir, `Family` for a member-subdir dir, - `NotFound` for an absent / empty name. -- `ensure_name_free` → `Err(NameTaken)` for both collision directions (Run-over- - Family and Family-over-Run); `Ok` for same-kind and for a fresh name. - -**`aura-cli` unit (`main.rs`):** -- `build_comparison_chart_data` over 2 members with **shared** timestamps → 2 - series, labelled by key, all sharing one `y_scale_id`, aligned (no nulls). -- over 2 members with **disjoint** timestamps (walk-forward-like) → 2 series, - null-complementary on the union spine (the stitched shape). -- `--tap` naming an absent tap → `Err`. - -**`aura-cli` integration (`tests/cli_run.rs`):** -- `aura sweep --trace t` then `aura chart t` → a page whose injected - `window.AURA_TRACES` carries **N member series** with the member-key labels. -- `aura chart unknown` → exit 2. -- collision: `aura run --trace t` then `aura sweep --trace t` → exit 2 (and the - reverse order). -- regression: `aura chart ` (no `--tap`) → unchanged all-taps page. - -**Post-cycle:** a `tester` E2E that a real `aura chart ` page renders. - -## Acceptance criteria - -- `aura chart ` emits a self-contained uPlot page overlaying one tap per - member across **all three** family kinds, members sharing one y-scale, labelled - by member key — driven by one builder, no per-kind branching. -- Name resolution is a **total function**: every name → exactly one of - {single-run, family, not-found}; the cross-kind collision state is unreachable - (write-guard). -- Every error path is `exit 2`, never a panic. -- The single-run chart path is **byte-unchanged** when no `--tap` is given - (existing `cli_run.rs` chart tests stay green); `aura-engine` is untouched. -- Determinism (C1): `read_family` order is by key; a given family renders - identically across runs. - -## Out of scope (unblocked follow-ons, not this cycle) - -- The run-context header on the chart page (#102). -- The multi-output viewer case (#47). -- The **orchestration-composability rebuild** — sweep/MC/WFO as composable tools in - a programmable analysis meta-level (the larger C21 cycle). -- Stale-member cleanup on family re-run; level-of-detail / sampling for very large - families (the #101 scale-path non-goals carry over).