diff --git a/docs/plans/0055-trace-charts-persist.md b/docs/plans/0055-trace-charts-persist.md new file mode 100644 index 0000000..a40fcf1 --- /dev/null +++ b/docs/plans/0055-trace-charts-persist.md @@ -0,0 +1,769 @@ +# Trace charts — iteration 1 (persist) — Implementation Plan + +> **Parent spec:** `docs/specs/0056-trace-charts-persist-and-serve.md` +> +> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run +> this plan. Steps use `- [ ]` checkboxes for tracking. + +**Goal:** Persist a run's drained recorder taps to disk as columnar (SoA) JSON +under `runs/traces//`, reachable via `aura run [--macd] [--trace ]` and +`aura run --real ... [--trace ]`, leaving the default (no-`--trace`) run +byte-for-byte unchanged. (The serve/chart half is iteration 2.) + +**Architecture:** Three layers, each landing beside its existing siblings. +`ColumnarTrace` (a pure transpose of `&[(Timestamp, Vec)]` ↔ columnar f64, +kind-tagged) goes in `aura-engine/report.rs` next to `join_on_ts`/`summarize`. A +`TraceStore` (write/read a per-run directory of `.json` + `index.json`) goes in +a new `aura-registry/trace_store.rs`, mirroring the `families.jsonl` sibling-store +discipline. A CLI-local `persist_traces` helper threads into all three run forms via +a new `trace: Option<&str>` parameter. The engine gains no I/O (it only encodes); the +registry crate does the file I/O; the CLI wires it. + +**Tech Stack:** Rust; `serde`/`serde_json` (already deps of both crates); the +existing `Scalar`/`ScalarKind`/`Timestamp` types; `aura-engine`'s `RunManifest`; the +`std::process::Command`-driven `cli_run.rs` e2e harness. + +--- + +**Files this plan creates or modifies:** + +- Modify: `crates/aura-engine/src/report.rs:187` — add `ColumnarTrace` (struct + + `from_rows`/`to_rows` + two private coercion helpers) after `join_on_ts`, before + `#[cfg(test)]` at :189; add three unit tests inside the existing test module. +- Modify: `crates/aura-engine/src/lib.rs:64` — add `ColumnarTrace` to the + `pub use report::{…}` re-export. +- Create: `crates/aura-registry/src/trace_store.rs` — `TraceStore` + `RunTraces` + + `TraceStoreError` + three temp-dir unit tests. +- Modify: `crates/aura-registry/src/lib.rs:24` — add `mod trace_store;` + + `pub use trace_store::{RunTraces, TraceStore, TraceStoreError};`. +- Modify: `crates/aura-cli/src/main.rs` — imports (:17–26), `persist_traces` helper + (new, before `run_sample` at :158), `run_sample` (:158), `run_sample_real` (:193), + `parse_real_args` (:264), `run_macd` (:823), `USAGE` (:858), the argv arms + (:867–870), and the nine in-file test callers (:1099–1100, :1180–1181, :1215, + :1220, :1230–1239, :1248–1249). +- Test: `crates/aura-cli/tests/cli_run.rs` — one new e2e: `aura run --trace demo` + writes `index.json` + per-tap files; plain `aura run` writes none. + +--- + +### Task 1: `ColumnarTrace` (aura-engine) + +**Files:** +- Modify: `crates/aura-engine/src/report.rs` (add type + helpers before :189; tests in the existing `mod tests`) +- Modify: `crates/aura-engine/src/lib.rs:64` + +- [ ] **Step 1: Write the failing tests** + +Append these three tests inside the existing `#[cfg(test)] mod tests { … }` in +`crates/aura-engine/src/report.rs` (e.g. directly after the `join_on_ts_*` tests, +before the closing `}` of the module). They use `Scalar`/`ScalarKind`/`Timestamp`, +already imported by the test module via `use super::*;`: + +```rust + #[test] + fn columnar_trace_round_trips_f64_rows() { + let rows = vec![ + (Timestamp(2), vec![Scalar::f64(10.0)]), + (Timestamp(3), vec![Scalar::f64(20.0)]), + (Timestamp(4), vec![Scalar::f64(30.0)]), + ]; + let ct = ColumnarTrace::from_rows("equity", &[ScalarKind::F64], &rows); + assert_eq!(ct.tap, "equity"); + assert_eq!(ct.kinds, vec!["F64".to_string()]); + assert_eq!(ct.ts, vec![2, 3, 4]); + assert_eq!(ct.columns, vec![vec![10.0, 20.0, 30.0]]); + // to_rows is the inverse for f64 taps + assert_eq!(ct.to_rows(), rows); + } + + #[test] + fn columnar_trace_empty_rows_yields_empty_columns_sized_to_kinds() { + let ct = ColumnarTrace::from_rows("exposure", &[ScalarKind::F64], &[]); + assert_eq!(ct.ts, Vec::::new()); + assert_eq!(ct.columns, vec![Vec::::new()]); + assert!(ct.to_rows().is_empty()); + } + + #[test] + fn columnar_trace_coerces_non_f64_kinds_to_f64() { + let rows = vec![ + (Timestamp(1), vec![Scalar::i64(7), Scalar::bool(true)]), + (Timestamp(2), vec![Scalar::i64(9), Scalar::bool(false)]), + ]; + let ct = ColumnarTrace::from_rows("mix", &[ScalarKind::I64, ScalarKind::Bool], &rows); + assert_eq!(ct.kinds, vec!["I64".to_string(), "Bool".to_string()]); + assert_eq!(ct.columns, vec![vec![7.0, 9.0], vec![1.0, 0.0]]); + } +``` + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: `cargo test -p aura-engine columnar_trace` +Expected: FAIL — compile error `cannot find type/function ColumnarTrace in this scope` (the type does not exist yet). + +- [ ] **Step 3: Write the implementation** + +Insert this immediately after `join_on_ts` ends (`report.rs:187`) and before +`#[cfg(test)]` (`report.rs:189`). `Scalar`/`ScalarKind`/`Timestamp` are already +imported at `report.rs:11`: + +```rust +/// One drained recorder tap in columnar (SoA) form: parallel arrays, one per +/// recorded column, plus the shared recorded-timestamp axis. Chart-ready (a column +/// is a series of numbers) and kind-tagged (C7) — values are coerced to f64 for +/// plotting; the base type survives in `kinds`. Pure: identical rows always encode +/// to the same value (C1). +#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)] +pub struct ColumnarTrace { + pub tap: String, + pub kinds: Vec, + pub ts: Vec, + pub columns: Vec>, +} + +impl ColumnarTrace { + /// Transpose a drained tap's `(ts, row)` pairs into columns. An empty `rows` + /// yields empty `ts` and one empty column per kind. Each cell is coerced to f64: + /// f64 as-is, i64 as f64, bool 1.0/0.0, timestamp epoch as f64. + pub fn from_rows(tap: &str, kinds: &[ScalarKind], rows: &[(Timestamp, Vec)]) -> Self { + let ts: Vec = rows.iter().map(|(t, _)| t.0).collect(); + let mut columns: Vec> = vec![Vec::with_capacity(rows.len()); kinds.len()]; + for (_, row) in rows { + for (c, scalar) in row.iter().enumerate() { + columns[c].push(scalar_to_f64(*scalar)); + } + } + ColumnarTrace { + tap: tap.to_string(), + kinds: kinds.iter().map(kind_tag).collect(), + ts, + columns, + } + } + + /// Inverse for the serve/align path: rebuild `(ts, row)` pairs with each cell as + /// `Scalar::f64(columns[c][r])` — uniformly f64 (the on-disk store is f64 + /// columns; the base type survives only in `kinds`). A true value round-trip for + /// f64 taps; keeps the serve-side `as_f64` projection total. + pub fn to_rows(&self) -> Vec<(Timestamp, Vec)> { + self.ts + .iter() + .enumerate() + .map(|(r, &t)| { + let row = self.columns.iter().map(|col| Scalar::f64(col[r])).collect(); + (Timestamp(t), row) + }) + .collect() + } +} + +/// The four-kind tag string for a column (the on-disk `kinds` form; `ScalarKind` +/// derives no serde, so tags are plain strings). +fn kind_tag(kind: &ScalarKind) -> String { + match kind { + ScalarKind::F64 => "F64", + ScalarKind::I64 => "I64", + ScalarKind::Bool => "Bool", + ScalarKind::Timestamp => "Timestamp", + } + .to_string() +} + +/// Coerce a recorded scalar to the f64 a chart plots: f64 as-is, i64 as f64, +/// bool 1.0/0.0, timestamp epoch as f64. +fn scalar_to_f64(s: Scalar) -> f64 { + match s.kind() { + ScalarKind::F64 => s.as_f64(), + ScalarKind::I64 => s.as_i64() as f64, + ScalarKind::Bool => { + if s.as_bool() { + 1.0 + } else { + 0.0 + } + } + ScalarKind::Timestamp => s.as_ts().0 as f64, + } +} +``` + +- [ ] **Step 4: Re-export `ColumnarTrace`** + +In `crates/aura-engine/src/lib.rs:64`, change: + +```rust +pub use report::{f64_field, join_on_ts, summarize, JoinedRow, RunManifest, RunMetrics, RunReport}; +``` + +to: + +```rust +pub use report::{ + f64_field, join_on_ts, summarize, ColumnarTrace, JoinedRow, RunManifest, RunMetrics, RunReport, +}; +``` + +- [ ] **Step 5: Run the tests to verify they pass** + +Run: `cargo test -p aura-engine columnar_trace` +Expected: PASS — `columnar_trace_round_trips_f64_rows`, `columnar_trace_empty_rows_yields_empty_columns_sized_to_kinds`, `columnar_trace_coerces_non_f64_kinds_to_f64` all green (3 passed). + +--- + +### Task 2: `TraceStore` (aura-registry) + +**Files:** +- Create: `crates/aura-registry/src/trace_store.rs` +- Modify: `crates/aura-registry/src/lib.rs:24` + +- [ ] **Step 1: Register the new module** + +In `crates/aura-registry/src/lib.rs`, directly after `mod lineage;` (`lib.rs:24`) and +its `pub use lineage::{…}` block (`lib.rs:25–28`), add: + +```rust +mod trace_store; +pub use trace_store::{RunTraces, TraceStore, TraceStoreError}; +``` + +- [ ] **Step 2: Write the new file with its failing tests first** + +Create `crates/aura-registry/src/trace_store.rs` with ONLY the test module + the +`use` header below (the impl is added in Step 4, so the tests fail to compile = RED): + +```rust +//! The per-run trace store: a directory of columnar tap files under +//! `/traces//`, beside the run registry's `runs.jsonl`. One +//! subdirectory per run name; one `.json` (a `ColumnarTrace`) per tap; plus +//! `index.json` carrying the run's `RunManifest` and the tap order. Mirrors the +//! registry's directory-co-located sibling-store discipline; a missing run reads as +//! `NotFound` (treat-as-absent), never a panic. + +use std::fmt; +use std::fs; +use std::path::{Path, PathBuf}; + +use aura_engine::{ColumnarTrace, RunManifest}; +use serde::{Deserialize, Serialize}; + +#[cfg(test)] +mod tests { + use super::*; + use aura_core::{Scalar, ScalarKind, Timestamp}; + + fn temp_traces_root(name: &str) -> PathBuf { + let dir = + std::env::temp_dir().join(format!("aura-tracestore-{}-{}", std::process::id(), name)); + let _ = fs::remove_dir_all(&dir); + fs::create_dir_all(&dir).expect("create temp traces root"); + dir + } + + fn sample_manifest() -> RunManifest { + RunManifest { + commit: "c".to_string(), + params: vec![("p".to_string(), Scalar::f64(1.0))], + window: (Timestamp(1), Timestamp(5)), + seed: 0, + broker: "sim-optimal(pip_size=0.0001)".to_string(), + } + } + + fn sample_taps() -> Vec { + vec![ + ColumnarTrace::from_rows( + "equity", + &[ScalarKind::F64], + &[(Timestamp(1), vec![Scalar::f64(0.0)]), (Timestamp(2), vec![Scalar::f64(0.5)])], + ), + ColumnarTrace::from_rows( + "exposure", + &[ScalarKind::F64], + &[(Timestamp(1), vec![Scalar::f64(1.0)]), (Timestamp(2), vec![Scalar::f64(-1.0)])], + ), + ] + } + + #[test] + fn write_then_read_round_trips() { + let root = temp_traces_root("roundtrip"); + let store = TraceStore::open(&root); + let manifest = sample_manifest(); + let taps = sample_taps(); + store.write("demo", &manifest, &taps).expect("write"); + let back = store.read("demo").expect("read"); + assert_eq!(back.manifest, manifest); + assert_eq!(back.taps, taps); + assert!(root.join("traces/demo/index.json").exists()); + assert!(root.join("traces/demo/equity.json").exists()); + assert!(root.join("traces/demo/exposure.json").exists()); + let _ = fs::remove_dir_all(&root); + } + + #[test] + fn read_missing_run_is_not_found() { + let root = temp_traces_root("missing"); + let store = TraceStore::open(&root); + match store.read("nope") { + Err(TraceStoreError::NotFound(name)) => assert_eq!(name, "nope"), + other => panic!("expected NotFound, got {other:?}"), + } + let _ = fs::remove_dir_all(&root); + } + + #[test] + fn corrupt_tap_file_is_a_parse_error_naming_the_file() { + let root = temp_traces_root("corrupt"); + let store = TraceStore::open(&root); + store.write("demo", &sample_manifest(), &sample_taps()).expect("write"); + fs::write(root.join("traces/demo/equity.json"), "not json").expect("clobber"); + match store.read("demo") { + Err(TraceStoreError::Parse { file, .. }) => assert!(file.ends_with("equity.json")), + other => panic!("expected Parse, got {other:?}"), + } + let _ = fs::remove_dir_all(&root); + } +} +``` + +- [ ] **Step 3: Run the tests to verify they fail** + +Run: `cargo test -p aura-registry trace_store` +Expected: FAIL — compile error `cannot find type TraceStore / TraceStoreError / RunTraces` (impl not written yet). + +- [ ] **Step 4: Write the implementation** + +In `crates/aura-registry/src/trace_store.rs`, insert this BETWEEN the `use` header +and the `#[cfg(test)] mod tests` block: + +```rust +/// A per-run trace directory rooted at `/traces`. One subdir per run name. +pub struct TraceStore { + dir: PathBuf, +} + +/// A run's read-back traces: its manifest plus its taps in index (write) order. +pub struct RunTraces { + pub manifest: RunManifest, + pub taps: Vec, +} + +/// `index.json`: the run manifest + the tap names in write order (= read order). +#[derive(Serialize, Deserialize)] +struct Index { + manifest: RunManifest, + taps: Vec, +} + +impl TraceStore { + /// Bind to `/traces`. No I/O — directories are created on write. + pub fn open(runs_dir: impl AsRef) -> TraceStore { + TraceStore { dir: runs_dir.as_ref().join("traces") } + } + + /// Write one run's taps + index under `traces//`, creating directories as + /// needed. Each tap lands in `.json`; the manifest + tap order in + /// `index.json`. + pub fn write( + &self, + name: &str, + manifest: &RunManifest, + taps: &[ColumnarTrace], + ) -> Result<(), TraceStoreError> { + let run_dir = self.dir.join(name); + fs::create_dir_all(&run_dir)?; + for tap in taps { + let path = run_dir.join(format!("{}.json", tap.tap)); + let body = serde_json::to_string(tap).map_err(|source| TraceStoreError::Parse { + file: path.display().to_string(), + source, + })?; + fs::write(&path, body)?; + } + let index = Index { + manifest: manifest.clone(), + taps: taps.iter().map(|t| t.tap.clone()).collect(), + }; + let index_path = run_dir.join("index.json"); + let body = serde_json::to_string(&index).map_err(|source| TraceStoreError::Parse { + file: index_path.display().to_string(), + source, + })?; + fs::write(&index_path, body)?; + Ok(()) + } + + /// Read a run's index + taps back, in index order. A missing run directory (no + /// `index.json`) is `NotFound`, not a panic; a malformed file is `Parse{file,..}`. + pub fn read(&self, name: &str) -> Result { + let run_dir = self.dir.join(name); + let index_path = run_dir.join("index.json"); + let index_text = match fs::read_to_string(&index_path) { + Ok(t) => t, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + return Err(TraceStoreError::NotFound(name.to_string())) + } + Err(e) => return Err(TraceStoreError::Io(e)), + }; + let index: Index = serde_json::from_str(&index_text).map_err(|source| { + TraceStoreError::Parse { file: index_path.display().to_string(), source } + })?; + let mut taps = Vec::with_capacity(index.taps.len()); + for tap_name in &index.taps { + let path = run_dir.join(format!("{tap_name}.json")); + let text = fs::read_to_string(&path)?; + let tap: ColumnarTrace = serde_json::from_str(&text).map_err(|source| { + TraceStoreError::Parse { file: path.display().to_string(), source } + })?; + taps.push(tap); + } + Ok(RunTraces { manifest: index.manifest, taps }) + } +} + +/// What can go wrong reading or writing the trace store. +#[derive(Debug)] +pub enum TraceStoreError { + /// An I/O error reading or writing a trace file. + Io(std::io::Error), + /// A trace/index file did not parse (the offending file named). + Parse { file: String, source: serde_json::Error }, + /// No recorded run of this name (no `index.json` under `traces//`). + NotFound(String), +} + +impl fmt::Display for TraceStoreError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + TraceStoreError::Io(e) => write!(f, "trace store i/o: {e}"), + TraceStoreError::Parse { file, source } => { + write!(f, "trace store parse error in {file}: {source}") + } + TraceStoreError::NotFound(name) => { + write!(f, "no recorded run '{name}' under runs/traces") + } + } + } +} + +impl std::error::Error for TraceStoreError {} + +impl From for TraceStoreError { + fn from(e: std::io::Error) -> Self { + TraceStoreError::Io(e) + } +} +``` + +- [ ] **Step 5: Run the tests to verify they pass** + +Run: `cargo test -p aura-registry trace_store` +Expected: PASS — `write_then_read_round_trips`, `read_missing_run_is_not_found`, +`corrupt_tap_file_is_a_parse_error_naming_the_file` all green (3 passed). + +--- + +### Task 3: CLI persist wiring (aura-cli) + +This task is **one compile unit**: the `trace: Option<&str>` parameter on three run +fns and the `parse_real_args` 3→4-tuple widen break every caller (including nine +in-file test callers), so all caller-threading lands here before the build gate. The +RED test (Step 1) is an e2e that compiles against the *binary*, not the changed +signatures, so it fails cleanly before any code change. + +**Files:** +- Test: `crates/aura-cli/tests/cli_run.rs` (new e2e) +- Modify: `crates/aura-cli/src/main.rs` (imports :17–26, helper, :158, :193, :264, :823, USAGE :858, argv :867–870, test callers :1099–1100/:1180–1181/:1215/:1220/:1230–1239/:1248–1249) + +- [ ] **Step 1: Write the failing e2e test** + +Append to `crates/aura-cli/tests/cli_run.rs` (it already has `BIN` + `temp_cwd`): + +```rust +#[test] +fn run_trace_persists_taps_and_plain_run_writes_no_traces() { + let cwd = temp_cwd("run-trace"); + + // plain `aura run` (no --trace) persists nothing to disk. + let plain = Command::new(BIN).arg("run").current_dir(&cwd).output().expect("spawn aura run"); + assert!(plain.status.success(), "plain run exit: {:?}", plain.status); + assert!(!cwd.join("runs/traces").exists(), "plain run must write no trace files"); + + // `aura run --trace demo` persists the two taps + index, and still prints the report. + let traced = Command::new(BIN) + .args(["run", "--trace", "demo"]) + .current_dir(&cwd) + .output() + .expect("spawn aura run --trace"); + assert!(traced.status.success(), "traced run exit: {:?}", traced.status); + let stdout = String::from_utf8(traced.stdout).expect("utf-8 stdout"); + assert!(stdout.trim_end().starts_with("{\"manifest\":{"), "report still printed: {stdout:?}"); + assert!(cwd.join("runs/traces/demo/index.json").exists(), "index.json written"); + assert!(cwd.join("runs/traces/demo/equity.json").exists(), "equity tap written"); + assert!(cwd.join("runs/traces/demo/exposure.json").exists(), "exposure tap written"); + + let _ = std::fs::remove_dir_all(&cwd); +} +``` + +- [ ] **Step 2: Run the e2e to verify it fails** + +Run: `cargo test -p aura-cli --test cli_run run_trace_persists_taps_and_plain_run_writes_no_traces` +Expected: FAIL — `traced run exit` assertion fails: `["run","--trace","demo"]` has no argv arm yet, so it falls to the usage-error path (exit 2), and no trace files are written. (The tree still compiles — no source change yet.) + +- [ ] **Step 3: Extend the imports** + +In `crates/aura-cli/src/main.rs`, add `ColumnarTrace` to the `aura_engine::{…}` +import (`main.rs:18`) and `TraceStore` to the `aura_registry::{…}` import +(`main.rs:25`): + +```rust +use aura_engine::{ + f64_field, monte_carlo, param_stability, summarize, walk_forward, window_of, ColumnarTrace, + Composite, Edge, FlatGraph, GraphBuilder, Harness, McAggregate, McFamily, RollMode, + RunManifest, RunReport, SourceSpec, SweepFamily, SyntheticSpec, Target, VecSource, + WalkForwardResult, WindowBounds, WindowRoller, WindowRun, +}; +use aura_registry::{ + group_families, mc_member_reports, optimize, rank_by, sweep_member_reports, + walkforward_member_reports, FamilyKind, Registry, TraceStore, +}; +``` + +(Only `TraceStore` is needed this iteration; `persist_traces` Display-formats the +error inline, so `TraceStoreError` is not named here — importing it would be an +unused import under `-D warnings`. Iteration 2's `emit_chart` adds it.) + +- [ ] **Step 4: Add the `persist_traces` helper** + +Insert directly before `fn run_sample` (`main.rs:158`): + +```rust +/// Persist a run's drained taps to the on-disk trace store under `runs/traces//` +/// (beside the run registry's `runs/`). Shared by every run form — all drain the same +/// two f64 taps (equity off the broker, exposure off the strategy) and carry a +/// `RunManifest`. Pure wiring over `ColumnarTrace` + `TraceStore`; the engine is +/// untouched. An I/O failure is a usage-level error (stderr + exit 2), per the spec. +fn persist_traces( + name: &str, + manifest: &RunManifest, + eq_rows: &[(Timestamp, Vec)], + ex_rows: &[(Timestamp, Vec)], +) { + let taps = vec![ + ColumnarTrace::from_rows("equity", &[ScalarKind::F64], eq_rows), + ColumnarTrace::from_rows("exposure", &[ScalarKind::F64], ex_rows), + ]; + if let Err(e) = TraceStore::open("runs").write(name, manifest, &taps) { + eprintln!("aura: trace persist failed: {e}"); + std::process::exit(2); + } +} +``` + +- [ ] **Step 5: Thread `trace` through `run_sample`** + +Replace `fn run_sample` (`main.rs:158–184`) with (signature gains `trace`, manifest +hoisted before the persist call, fold inlined after): + +```rust +fn run_sample(trace: Option<&str>) -> RunReport { + let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE); + let sources: Vec> = + vec![Box::new(VecSource::new(synthetic_prices()))]; + let window = window_of(&sources).expect("non-empty synthetic stream"); + h.run(sources); + + let eq_rows: Vec<(Timestamp, Vec)> = rx_eq.try_iter().collect(); + let ex_rows: Vec<(Timestamp, Vec)> = rx_ex.try_iter().collect(); + let manifest = sim_optimal_manifest( + vec![ + ("sma_fast".to_string(), Scalar::i64(2)), + ("sma_slow".to_string(), Scalar::i64(4)), + ("exposure_scale".to_string(), Scalar::f64(0.5)), + ], + window, + 0, + SYNTHETIC_PIP_SIZE, + ); + if let Some(name) = trace { + persist_traces(name, &manifest, &eq_rows, &ex_rows); + } + let metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0)); + RunReport { manifest, metrics } +} +``` + +- [ ] **Step 6: Thread `trace` through `run_sample_real`** + +In `fn run_sample_real` (`main.rs:193–257`): change the signature to +`fn run_sample_real(symbol: &str, from_ms: Option, to_ms: Option, trace: Option<&str>) -> RunReport`, +then replace the drain-to-RunReport tail (`main.rs:238–256`) with: + +```rust + let eq_rows: Vec<(Timestamp, Vec)> = rx_eq.try_iter().collect(); + let ex_rows: Vec<(Timestamp, Vec)> = rx_ex.try_iter().collect(); + let manifest = sim_optimal_manifest( + vec![ + ("sma_fast".to_string(), Scalar::i64(2)), + ("sma_slow".to_string(), Scalar::i64(4)), + ("exposure_scale".to_string(), Scalar::f64(0.5)), + ], + window, + 0, + spec.pip_size, + ); + if let Some(name) = trace { + persist_traces(name, &manifest, &eq_rows, &ex_rows); + } + let metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0)); + RunReport { manifest, metrics } +} +``` + +- [ ] **Step 7: Widen `parse_real_args` to a 4-tuple with a `--trace` tail flag** + +Replace `fn parse_real_args` (`main.rs:264–283`) with: + +```rust +fn parse_real_args( + rest: &[&str], +) -> Result<(String, Option, Option, Option), String> { + let usage = || "run --real [--from ] [--to ] [--trace ]".to_string(); + let (symbol, mut tail) = match rest.split_first() { + Some((sym, t)) if !sym.is_empty() => (*sym, t), + _ => return Err(usage()), + }; + let mut from: Option = None; + let mut to: Option = None; + let mut trace: Option = None; + while let Some((flag, t)) = tail.split_first() { + let (value, t) = t.split_first().ok_or_else(usage)?; + match *flag { + "--from" if from.is_none() => from = Some(value.parse().map_err(|_| usage())?), + "--to" if to.is_none() => to = Some(value.parse().map_err(|_| usage())?), + "--trace" if trace.is_none() => trace = Some((*value).to_string()), + _ => return Err(usage()), + } + tail = t; + } + Ok((symbol.to_string(), from, to, trace)) +} +``` + +- [ ] **Step 8: Thread `trace` through `run_macd`** + +In `fn run_macd` (`main.rs:823–856`): change the signature to +`fn run_macd(trace: Option<&str>) -> RunReport`, then replace the drain-to-RunReport +tail (`main.rs:836–855`) with: + +```rust + let eq_rows: Vec<(Timestamp, Vec)> = rx_eq.try_iter().collect(); + let ex_rows: Vec<(Timestamp, Vec)> = rx_ex.try_iter().collect(); + let manifest = sim_optimal_manifest( + vec![ + ("ema_fast".to_string(), Scalar::i64(2)), + ("ema_slow".to_string(), Scalar::i64(4)), + ("ema_signal".to_string(), Scalar::i64(3)), + ("exposure_scale".to_string(), Scalar::f64(0.5)), + ], + window, + 0, + SYNTHETIC_PIP_SIZE, + ); + if let Some(name) = trace { + persist_traces(name, &manifest, &eq_rows, &ex_rows); + } + let metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0)); + RunReport { manifest, metrics } +} +``` + +- [ ] **Step 9: Update the argv arms** + +Replace the three run arms (`main.rs:867–875` — `["run"]`, `["run","--macd"]`, the +`["run","--real", rest @ ..]` block) with: + +```rust + ["run"] => println!("{}", run_sample(None).to_json()), + ["run", "--macd"] => println!("{}", run_macd(None).to_json()), + ["run", "--trace", name] => println!("{}", run_sample(Some(name)).to_json()), + ["run", "--macd", "--trace", name] => println!("{}", run_macd(Some(name)).to_json()), + ["run", "--real", rest @ ..] => match parse_real_args(rest) { + Ok((sym, from, to, trace)) => { + println!("{}", run_sample_real(&sym, from, to, trace.as_deref()).to_json()) + } + Err(msg) => { + eprintln!("aura: {msg}"); + std::process::exit(2); + } + }, +``` + +- [ ] **Step 10: Update `USAGE`** + +Replace the `USAGE` const (`main.rs:858–859`) with (one line; adds `[--trace ]` +on both run forms — the `chart` verb is added in iteration 2): + +```rust +const USAGE: &str = + "usage: aura run [--macd] [--trace ] | aura run --real [--from ] [--to ] [--trace ] | aura graph | aura sweep [--name ] | aura mc [--name ] | aura walkforward [--name ] | aura runs families | aura runs family [rank ]"; +``` + +- [ ] **Step 11: Migrate the nine in-file test callers** + +These break on the signature/tuple changes; update each in place: + +- `main.rs:1099–1100`: `run_macd()` → `run_macd(None)` (both lines). +- `main.rs:1248–1249`: `run_sample()` → `run_sample(None)` (both lines). +- `main.rs:1180`, `1181`, `1215`, `1220`: append `, None` as the 4th arg to each + `run_sample_real(SYMBOL_OR_"GER40", Some(..), Some(..))` call, e.g. + `run_sample_real(SYMBOL, Some(GER40_SEP2024_FROM_MS), Some(GER40_SEP2024_TO_MS), None)`. +- `parse_real_args` test (`main.rs:1229–1244`): replace the body with the 4-tuple + shape + a new `--trace` case: + +```rust + fn parse_real_args_accepts_symbol_and_optional_window() { + assert_eq!(parse_real_args(&["EURUSD"]), Ok(("EURUSD".to_string(), None, None, None))); + assert_eq!( + parse_real_args(&["EURUSD", "--from", "100", "--to", "200"]), + Ok(("EURUSD".to_string(), Some(100), Some(200), None)) + ); + // flags in any order + assert_eq!( + parse_real_args(&["EURUSD", "--to", "200", "--from", "100"]), + Ok(("EURUSD".to_string(), Some(100), Some(200), None)) + ); + // the --trace tail flag carries a string name, parsed alongside the window + assert_eq!( + parse_real_args(&["EURUSD", "--from", "100", "--trace", "demo"]), + Ok(("EURUSD".to_string(), Some(100), None, Some("demo".to_string()))) + ); + // a flag without its value, an empty symbol, and a non-numeric ms all reject + assert!(parse_real_args(&["EURUSD", "--from"]).is_err()); + assert!(parse_real_args(&[]).is_err()); + assert!(parse_real_args(&["EURUSD", "--from", "notanumber"]).is_err()); + } +``` + +- [ ] **Step 12: Build gate — workspace compiles, full aura-cli suite green** + +Run: `cargo build --workspace` +Expected: PASS — 0 errors (all callers threaded; no unresolved signature/tuple breaks). + +Run: `cargo test -p aura-cli` +Expected: PASS — the new e2e `run_trace_persists_taps_and_plain_run_writes_no_traces` +passes, the updated `parse_real_args_accepts_symbol_and_optional_window` passes, and +the byte-for-byte default-run pins (`run_prints_json_and_exits_zero`, +`run_with_trailing_token_is_strict_and_exits_two`) stay green (default run unchanged). + +- [ ] **Step 13: Workspace regression gate** + +Run: `cargo test --workspace` +Expected: PASS — Task 1 (`columnar_trace_*`) + Task 2 (`trace_store::tests::*`) + the +CLI suite all green; no regression elsewhere. + +Run: `cargo clippy --workspace --all-targets -- -D warnings` +Expected: PASS — no warnings. (Watch for: an unused import if `TraceStore`/`ColumnarTrace` +is imported but a step is skipped; a `clippy::needless_range_loop` on `from_rows`'s +index loop — if flagged, the `.enumerate()` form already shown avoids it.)