audit(trace-charts): cycle close — C22 realization note, retire ephemera
Cycle-close audit for the trace-charts visual face (#101, shipped across8bb5256persist +15ee6d5serve). Architect drift review (13941a2..HEAD) + the regression gate (cargo build/test/clippy --workspace, all exit 0, orchestrator-verified). What holds (carry-on): C14 clean — aura-engine/aura-core carry zero UI/HTML/uPlot knowledge; ColumnarTrace does struct->encoding only, file I/O in aura-registry, rendering in aura-cli. C1/C3 clean — persistence and serve-time alignment are post-run pure reductions (join_on_ts at serve time only, never in-graph). C7 clean — on-disk f64 columns + kinds-tagged base types, the documented coercion honest, no 5th scalar. Resolutions: - [fix] Added the C22 realization note now that the web-from-disk seam shipped (the amendment3b56efbhad deferred it to "when the seam ships"). - [ratify] The served page injects the chart SERIES but drops the run manifest (no run-context header) — a deliberate, defensible naive-viewer scope call (the manifest is on disk in index.json; a header matters most for the deferred families/comparison work, C21). The deviation from spec 0056 §1/§4 landed unflagged in 15ee6d5; it is now flagged here and tracked as a follow-up. refs #102 - [tidy] Retired this cycle's ephemera (spec 0056, plans 0055/0056) and the stale debris from the prior Runway close (spec 0055-fieldtest) per the ephemeral-artifact convention; the durable record is the ledger + git history. Cycle drift-clean (modulo the ratified, tracked manifest-header gap). refs #101
This commit is contained in:
@@ -1009,6 +1009,22 @@ run records *many* streams (one per recording node) instead of exactly one row.
|
|||||||
Recorded streams are sparse and timestamped (a record per fired cycle, tagged
|
Recorded streams are sparse and timestamped (a record per fired cycle, tagged
|
||||||
`ctx.now()`), matching a trace of timestamped events (C18). No new contract; the
|
`ctx.now()`), matching a trace of timestamped events (C18). No new contract; the
|
||||||
`Harness` API change (observe removed, `run -> ()`) is recorded here.
|
`Harness` API change (observe removed, `run -> ()`) is recorded here.
|
||||||
|
**Realization (the web-from-disk visual face, #101).** The C14/C22 web seam shipped
|
||||||
|
(amendment 3b56efb): a recording run persists each drained tap as a columnar (SoA,
|
||||||
|
C7) `ColumnarTrace` to `runs/traces/<name>/` (`aura-registry::TraceStore`, beside the
|
||||||
|
run registry's `runs.jsonl`), reachable via `aura run [--macd] [--real <SYM> …]
|
||||||
|
--trace <name>`; `aura chart <name> [--panels]` reads them back, aligns all taps on a
|
||||||
|
synthetic-union timestamp spine via the post-run `join_on_ts` (C3 — not in-graph;
|
||||||
|
C1 — pure, no live external call), and emits a static self-contained uPlot page
|
||||||
|
(vendored like `render_html`'s Graphviz-WASM). The engine stays headless (C14):
|
||||||
|
encoding lives in `aura-engine` (`ColumnarTrace`, struct→JSON only), file I/O in
|
||||||
|
`aura-registry`, rendering in `aura-cli` (`render_chart_html` + `chart-viewer.js`).
|
||||||
|
First cut only — overlay (per-series y-scale) / timestamp-aligned panels; the served
|
||||||
|
page injects the chart *series* but **not yet a run-context header** (the run manifest
|
||||||
|
is persisted on disk in `index.json` but deliberately not surfaced in this naive
|
||||||
|
viewer — a ratified scope call, follow-up filed). Families-comparison meta-views, a
|
||||||
|
local server, and replay-clock controls remain open (see "Open architectural
|
||||||
|
threads").
|
||||||
|
|
||||||
### C23 — Graph compilation and behaviour-preserving optimisation
|
### C23 — Graph compilation and behaviour-preserving optimisation
|
||||||
**Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic,
|
**Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic,
|
||||||
|
|||||||
@@ -1,769 +0,0 @@
|
|||||||
# Trace charts — iteration 1 (persist) — Implementation Plan
|
|
||||||
|
|
||||||
> **Parent spec:** `docs/specs/0056-trace-charts-persist-and-serve.md`
|
|
||||||
>
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
|
|
||||||
> this plan. Steps use `- [ ]` checkboxes for tracking.
|
|
||||||
|
|
||||||
**Goal:** Persist a run's drained recorder taps to disk as columnar (SoA) JSON
|
|
||||||
under `runs/traces/<name>/`, reachable via `aura run [--macd] [--trace <name>]` and
|
|
||||||
`aura run --real <SYM> ... [--trace <name>]`, leaving the default (no-`--trace`) run
|
|
||||||
byte-for-byte unchanged. (The serve/chart half is iteration 2.)
|
|
||||||
|
|
||||||
**Architecture:** Three layers, each landing beside its existing siblings.
|
|
||||||
`ColumnarTrace` (a pure transpose of `&[(Timestamp, Vec<Scalar>)]` ↔ columnar f64,
|
|
||||||
kind-tagged) goes in `aura-engine/report.rs` next to `join_on_ts`/`summarize`. A
|
|
||||||
`TraceStore` (write/read a per-run directory of `<tap>.json` + `index.json`) goes in
|
|
||||||
a new `aura-registry/trace_store.rs`, mirroring the `families.jsonl` sibling-store
|
|
||||||
discipline. A CLI-local `persist_traces` helper threads into all three run forms via
|
|
||||||
a new `trace: Option<&str>` parameter. The engine gains no I/O (it only encodes); the
|
|
||||||
registry crate does the file I/O; the CLI wires it.
|
|
||||||
|
|
||||||
**Tech Stack:** Rust; `serde`/`serde_json` (already deps of both crates); the
|
|
||||||
existing `Scalar`/`ScalarKind`/`Timestamp` types; `aura-engine`'s `RunManifest`; the
|
|
||||||
`std::process::Command`-driven `cli_run.rs` e2e harness.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Files this plan creates or modifies:**
|
|
||||||
|
|
||||||
- Modify: `crates/aura-engine/src/report.rs:187` — add `ColumnarTrace` (struct +
|
|
||||||
`from_rows`/`to_rows` + two private coercion helpers) after `join_on_ts`, before
|
|
||||||
`#[cfg(test)]` at :189; add three unit tests inside the existing test module.
|
|
||||||
- Modify: `crates/aura-engine/src/lib.rs:64` — add `ColumnarTrace` to the
|
|
||||||
`pub use report::{…}` re-export.
|
|
||||||
- Create: `crates/aura-registry/src/trace_store.rs` — `TraceStore` + `RunTraces` +
|
|
||||||
`TraceStoreError` + three temp-dir unit tests.
|
|
||||||
- Modify: `crates/aura-registry/src/lib.rs:24` — add `mod trace_store;` +
|
|
||||||
`pub use trace_store::{RunTraces, TraceStore, TraceStoreError};`.
|
|
||||||
- Modify: `crates/aura-cli/src/main.rs` — imports (:17–26), `persist_traces` helper
|
|
||||||
(new, before `run_sample` at :158), `run_sample` (:158), `run_sample_real` (:193),
|
|
||||||
`parse_real_args` (:264), `run_macd` (:823), `USAGE` (:858), the argv arms
|
|
||||||
(:867–870), and the nine in-file test callers (:1099–1100, :1180–1181, :1215,
|
|
||||||
:1220, :1230–1239, :1248–1249).
|
|
||||||
- Test: `crates/aura-cli/tests/cli_run.rs` — one new e2e: `aura run --trace demo`
|
|
||||||
writes `index.json` + per-tap files; plain `aura run` writes none.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 1: `ColumnarTrace` (aura-engine)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `crates/aura-engine/src/report.rs` (add type + helpers before :189; tests in the existing `mod tests`)
|
|
||||||
- Modify: `crates/aura-engine/src/lib.rs:64`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing tests**
|
|
||||||
|
|
||||||
Append these three tests inside the existing `#[cfg(test)] mod tests { … }` in
|
|
||||||
`crates/aura-engine/src/report.rs` (e.g. directly after the `join_on_ts_*` tests,
|
|
||||||
before the closing `}` of the module). They use `Scalar`/`ScalarKind`/`Timestamp`,
|
|
||||||
already imported by the test module via `use super::*;`:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
#[test]
|
|
||||||
fn columnar_trace_round_trips_f64_rows() {
|
|
||||||
let rows = vec![
|
|
||||||
(Timestamp(2), vec![Scalar::f64(10.0)]),
|
|
||||||
(Timestamp(3), vec![Scalar::f64(20.0)]),
|
|
||||||
(Timestamp(4), vec![Scalar::f64(30.0)]),
|
|
||||||
];
|
|
||||||
let ct = ColumnarTrace::from_rows("equity", &[ScalarKind::F64], &rows);
|
|
||||||
assert_eq!(ct.tap, "equity");
|
|
||||||
assert_eq!(ct.kinds, vec!["F64".to_string()]);
|
|
||||||
assert_eq!(ct.ts, vec![2, 3, 4]);
|
|
||||||
assert_eq!(ct.columns, vec![vec![10.0, 20.0, 30.0]]);
|
|
||||||
// to_rows is the inverse for f64 taps
|
|
||||||
assert_eq!(ct.to_rows(), rows);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn columnar_trace_empty_rows_yields_empty_columns_sized_to_kinds() {
|
|
||||||
let ct = ColumnarTrace::from_rows("exposure", &[ScalarKind::F64], &[]);
|
|
||||||
assert_eq!(ct.ts, Vec::<i64>::new());
|
|
||||||
assert_eq!(ct.columns, vec![Vec::<f64>::new()]);
|
|
||||||
assert!(ct.to_rows().is_empty());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn columnar_trace_coerces_non_f64_kinds_to_f64() {
|
|
||||||
let rows = vec![
|
|
||||||
(Timestamp(1), vec![Scalar::i64(7), Scalar::bool(true)]),
|
|
||||||
(Timestamp(2), vec![Scalar::i64(9), Scalar::bool(false)]),
|
|
||||||
];
|
|
||||||
let ct = ColumnarTrace::from_rows("mix", &[ScalarKind::I64, ScalarKind::Bool], &rows);
|
|
||||||
assert_eq!(ct.kinds, vec!["I64".to_string(), "Bool".to_string()]);
|
|
||||||
assert_eq!(ct.columns, vec![vec![7.0, 9.0], vec![1.0, 0.0]]);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run the tests to verify they fail**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-engine columnar_trace`
|
|
||||||
Expected: FAIL — compile error `cannot find type/function ColumnarTrace in this scope` (the type does not exist yet).
|
|
||||||
|
|
||||||
- [ ] **Step 3: Write the implementation**
|
|
||||||
|
|
||||||
Insert this immediately after `join_on_ts` ends (`report.rs:187`) and before
|
|
||||||
`#[cfg(test)]` (`report.rs:189`). `Scalar`/`ScalarKind`/`Timestamp` are already
|
|
||||||
imported at `report.rs:11`:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// One drained recorder tap in columnar (SoA) form: parallel arrays, one per
|
|
||||||
/// recorded column, plus the shared recorded-timestamp axis. Chart-ready (a column
|
|
||||||
/// is a series of numbers) and kind-tagged (C7) — values are coerced to f64 for
|
|
||||||
/// plotting; the base type survives in `kinds`. Pure: identical rows always encode
|
|
||||||
/// to the same value (C1).
|
|
||||||
#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
|
|
||||||
pub struct ColumnarTrace {
|
|
||||||
pub tap: String,
|
|
||||||
pub kinds: Vec<String>,
|
|
||||||
pub ts: Vec<i64>,
|
|
||||||
pub columns: Vec<Vec<f64>>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ColumnarTrace {
|
|
||||||
/// Transpose a drained tap's `(ts, row)` pairs into columns. An empty `rows`
|
|
||||||
/// yields empty `ts` and one empty column per kind. Each cell is coerced to f64:
|
|
||||||
/// f64 as-is, i64 as f64, bool 1.0/0.0, timestamp epoch as f64.
|
|
||||||
pub fn from_rows(tap: &str, kinds: &[ScalarKind], rows: &[(Timestamp, Vec<Scalar>)]) -> Self {
|
|
||||||
let ts: Vec<i64> = rows.iter().map(|(t, _)| t.0).collect();
|
|
||||||
let mut columns: Vec<Vec<f64>> = vec![Vec::with_capacity(rows.len()); kinds.len()];
|
|
||||||
for (_, row) in rows {
|
|
||||||
for (c, scalar) in row.iter().enumerate() {
|
|
||||||
columns[c].push(scalar_to_f64(*scalar));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
ColumnarTrace {
|
|
||||||
tap: tap.to_string(),
|
|
||||||
kinds: kinds.iter().map(kind_tag).collect(),
|
|
||||||
ts,
|
|
||||||
columns,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Inverse for the serve/align path: rebuild `(ts, row)` pairs with each cell as
|
|
||||||
/// `Scalar::f64(columns[c][r])` — uniformly f64 (the on-disk store is f64
|
|
||||||
/// columns; the base type survives only in `kinds`). A true value round-trip for
|
|
||||||
/// f64 taps; keeps the serve-side `as_f64` projection total.
|
|
||||||
pub fn to_rows(&self) -> Vec<(Timestamp, Vec<Scalar>)> {
|
|
||||||
self.ts
|
|
||||||
.iter()
|
|
||||||
.enumerate()
|
|
||||||
.map(|(r, &t)| {
|
|
||||||
let row = self.columns.iter().map(|col| Scalar::f64(col[r])).collect();
|
|
||||||
(Timestamp(t), row)
|
|
||||||
})
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The four-kind tag string for a column (the on-disk `kinds` form; `ScalarKind`
|
|
||||||
/// derives no serde, so tags are plain strings).
|
|
||||||
fn kind_tag(kind: &ScalarKind) -> String {
|
|
||||||
match kind {
|
|
||||||
ScalarKind::F64 => "F64",
|
|
||||||
ScalarKind::I64 => "I64",
|
|
||||||
ScalarKind::Bool => "Bool",
|
|
||||||
ScalarKind::Timestamp => "Timestamp",
|
|
||||||
}
|
|
||||||
.to_string()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Coerce a recorded scalar to the f64 a chart plots: f64 as-is, i64 as f64,
|
|
||||||
/// bool 1.0/0.0, timestamp epoch as f64.
|
|
||||||
fn scalar_to_f64(s: Scalar) -> f64 {
|
|
||||||
match s.kind() {
|
|
||||||
ScalarKind::F64 => s.as_f64(),
|
|
||||||
ScalarKind::I64 => s.as_i64() as f64,
|
|
||||||
ScalarKind::Bool => {
|
|
||||||
if s.as_bool() {
|
|
||||||
1.0
|
|
||||||
} else {
|
|
||||||
0.0
|
|
||||||
}
|
|
||||||
}
|
|
||||||
ScalarKind::Timestamp => s.as_ts().0 as f64,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Re-export `ColumnarTrace`**
|
|
||||||
|
|
||||||
In `crates/aura-engine/src/lib.rs:64`, change:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub use report::{f64_field, join_on_ts, summarize, JoinedRow, RunManifest, RunMetrics, RunReport};
|
|
||||||
```
|
|
||||||
|
|
||||||
to:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub use report::{
|
|
||||||
f64_field, join_on_ts, summarize, ColumnarTrace, JoinedRow, RunManifest, RunMetrics, RunReport,
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Run the tests to verify they pass**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-engine columnar_trace`
|
|
||||||
Expected: PASS — `columnar_trace_round_trips_f64_rows`, `columnar_trace_empty_rows_yields_empty_columns_sized_to_kinds`, `columnar_trace_coerces_non_f64_kinds_to_f64` all green (3 passed).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 2: `TraceStore` (aura-registry)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `crates/aura-registry/src/trace_store.rs`
|
|
||||||
- Modify: `crates/aura-registry/src/lib.rs:24`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Register the new module**
|
|
||||||
|
|
||||||
In `crates/aura-registry/src/lib.rs`, directly after `mod lineage;` (`lib.rs:24`) and
|
|
||||||
its `pub use lineage::{…}` block (`lib.rs:25–28`), add:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
mod trace_store;
|
|
||||||
pub use trace_store::{RunTraces, TraceStore, TraceStoreError};
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Write the new file with its failing tests first**
|
|
||||||
|
|
||||||
Create `crates/aura-registry/src/trace_store.rs` with ONLY the test module + the
|
|
||||||
`use` header below (the impl is added in Step 4, so the tests fail to compile = RED):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
//! The per-run trace store: a directory of columnar tap files under
|
|
||||||
//! `<runs_dir>/traces/<name>/`, beside the run registry's `runs.jsonl`. One
|
|
||||||
//! subdirectory per run name; one `<tap>.json` (a `ColumnarTrace`) per tap; plus
|
|
||||||
//! `index.json` carrying the run's `RunManifest` and the tap order. Mirrors the
|
|
||||||
//! registry's directory-co-located sibling-store discipline; a missing run reads as
|
|
||||||
//! `NotFound` (treat-as-absent), never a panic.
|
|
||||||
|
|
||||||
use std::fmt;
|
|
||||||
use std::fs;
|
|
||||||
use std::path::{Path, PathBuf};
|
|
||||||
|
|
||||||
use aura_engine::{ColumnarTrace, RunManifest};
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use aura_core::{Scalar, ScalarKind, Timestamp};
|
|
||||||
|
|
||||||
fn temp_traces_root(name: &str) -> PathBuf {
|
|
||||||
let dir =
|
|
||||||
std::env::temp_dir().join(format!("aura-tracestore-{}-{}", std::process::id(), name));
|
|
||||||
let _ = fs::remove_dir_all(&dir);
|
|
||||||
fs::create_dir_all(&dir).expect("create temp traces root");
|
|
||||||
dir
|
|
||||||
}
|
|
||||||
|
|
||||||
fn sample_manifest() -> RunManifest {
|
|
||||||
RunManifest {
|
|
||||||
commit: "c".to_string(),
|
|
||||||
params: vec![("p".to_string(), Scalar::f64(1.0))],
|
|
||||||
window: (Timestamp(1), Timestamp(5)),
|
|
||||||
seed: 0,
|
|
||||||
broker: "sim-optimal(pip_size=0.0001)".to_string(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn sample_taps() -> Vec<ColumnarTrace> {
|
|
||||||
vec![
|
|
||||||
ColumnarTrace::from_rows(
|
|
||||||
"equity",
|
|
||||||
&[ScalarKind::F64],
|
|
||||||
&[(Timestamp(1), vec![Scalar::f64(0.0)]), (Timestamp(2), vec![Scalar::f64(0.5)])],
|
|
||||||
),
|
|
||||||
ColumnarTrace::from_rows(
|
|
||||||
"exposure",
|
|
||||||
&[ScalarKind::F64],
|
|
||||||
&[(Timestamp(1), vec![Scalar::f64(1.0)]), (Timestamp(2), vec![Scalar::f64(-1.0)])],
|
|
||||||
),
|
|
||||||
]
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn write_then_read_round_trips() {
|
|
||||||
let root = temp_traces_root("roundtrip");
|
|
||||||
let store = TraceStore::open(&root);
|
|
||||||
let manifest = sample_manifest();
|
|
||||||
let taps = sample_taps();
|
|
||||||
store.write("demo", &manifest, &taps).expect("write");
|
|
||||||
let back = store.read("demo").expect("read");
|
|
||||||
assert_eq!(back.manifest, manifest);
|
|
||||||
assert_eq!(back.taps, taps);
|
|
||||||
assert!(root.join("traces/demo/index.json").exists());
|
|
||||||
assert!(root.join("traces/demo/equity.json").exists());
|
|
||||||
assert!(root.join("traces/demo/exposure.json").exists());
|
|
||||||
let _ = fs::remove_dir_all(&root);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn read_missing_run_is_not_found() {
|
|
||||||
let root = temp_traces_root("missing");
|
|
||||||
let store = TraceStore::open(&root);
|
|
||||||
match store.read("nope") {
|
|
||||||
Err(TraceStoreError::NotFound(name)) => assert_eq!(name, "nope"),
|
|
||||||
other => panic!("expected NotFound, got {other:?}"),
|
|
||||||
}
|
|
||||||
let _ = fs::remove_dir_all(&root);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn corrupt_tap_file_is_a_parse_error_naming_the_file() {
|
|
||||||
let root = temp_traces_root("corrupt");
|
|
||||||
let store = TraceStore::open(&root);
|
|
||||||
store.write("demo", &sample_manifest(), &sample_taps()).expect("write");
|
|
||||||
fs::write(root.join("traces/demo/equity.json"), "not json").expect("clobber");
|
|
||||||
match store.read("demo") {
|
|
||||||
Err(TraceStoreError::Parse { file, .. }) => assert!(file.ends_with("equity.json")),
|
|
||||||
other => panic!("expected Parse, got {other:?}"),
|
|
||||||
}
|
|
||||||
let _ = fs::remove_dir_all(&root);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Run the tests to verify they fail**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-registry trace_store`
|
|
||||||
Expected: FAIL — compile error `cannot find type TraceStore / TraceStoreError / RunTraces` (impl not written yet).
|
|
||||||
|
|
||||||
- [ ] **Step 4: Write the implementation**
|
|
||||||
|
|
||||||
In `crates/aura-registry/src/trace_store.rs`, insert this BETWEEN the `use` header
|
|
||||||
and the `#[cfg(test)] mod tests` block:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// A per-run trace directory rooted at `<runs_dir>/traces`. One subdir per run name.
|
|
||||||
pub struct TraceStore {
|
|
||||||
dir: PathBuf,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A run's read-back traces: its manifest plus its taps in index (write) order.
|
|
||||||
pub struct RunTraces {
|
|
||||||
pub manifest: RunManifest,
|
|
||||||
pub taps: Vec<ColumnarTrace>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `index.json`: the run manifest + the tap names in write order (= read order).
|
|
||||||
#[derive(Serialize, Deserialize)]
|
|
||||||
struct Index {
|
|
||||||
manifest: RunManifest,
|
|
||||||
taps: Vec<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl TraceStore {
|
|
||||||
/// Bind to `<runs_dir>/traces`. No I/O — directories are created on write.
|
|
||||||
pub fn open(runs_dir: impl AsRef<Path>) -> TraceStore {
|
|
||||||
TraceStore { dir: runs_dir.as_ref().join("traces") }
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Write one run's taps + index under `traces/<name>/`, creating directories as
|
|
||||||
/// needed. Each tap lands in `<tap>.json`; the manifest + tap order in
|
|
||||||
/// `index.json`.
|
|
||||||
pub fn write(
|
|
||||||
&self,
|
|
||||||
name: &str,
|
|
||||||
manifest: &RunManifest,
|
|
||||||
taps: &[ColumnarTrace],
|
|
||||||
) -> Result<(), TraceStoreError> {
|
|
||||||
let run_dir = self.dir.join(name);
|
|
||||||
fs::create_dir_all(&run_dir)?;
|
|
||||||
for tap in taps {
|
|
||||||
let path = run_dir.join(format!("{}.json", tap.tap));
|
|
||||||
let body = serde_json::to_string(tap).map_err(|source| TraceStoreError::Parse {
|
|
||||||
file: path.display().to_string(),
|
|
||||||
source,
|
|
||||||
})?;
|
|
||||||
fs::write(&path, body)?;
|
|
||||||
}
|
|
||||||
let index = Index {
|
|
||||||
manifest: manifest.clone(),
|
|
||||||
taps: taps.iter().map(|t| t.tap.clone()).collect(),
|
|
||||||
};
|
|
||||||
let index_path = run_dir.join("index.json");
|
|
||||||
let body = serde_json::to_string(&index).map_err(|source| TraceStoreError::Parse {
|
|
||||||
file: index_path.display().to_string(),
|
|
||||||
source,
|
|
||||||
})?;
|
|
||||||
fs::write(&index_path, body)?;
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Read a run's index + taps back, in index order. A missing run directory (no
|
|
||||||
/// `index.json`) is `NotFound`, not a panic; a malformed file is `Parse{file,..}`.
|
|
||||||
pub fn read(&self, name: &str) -> Result<RunTraces, TraceStoreError> {
|
|
||||||
let run_dir = self.dir.join(name);
|
|
||||||
let index_path = run_dir.join("index.json");
|
|
||||||
let index_text = match fs::read_to_string(&index_path) {
|
|
||||||
Ok(t) => t,
|
|
||||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
|
|
||||||
return Err(TraceStoreError::NotFound(name.to_string()))
|
|
||||||
}
|
|
||||||
Err(e) => return Err(TraceStoreError::Io(e)),
|
|
||||||
};
|
|
||||||
let index: Index = serde_json::from_str(&index_text).map_err(|source| {
|
|
||||||
TraceStoreError::Parse { file: index_path.display().to_string(), source }
|
|
||||||
})?;
|
|
||||||
let mut taps = Vec::with_capacity(index.taps.len());
|
|
||||||
for tap_name in &index.taps {
|
|
||||||
let path = run_dir.join(format!("{tap_name}.json"));
|
|
||||||
let text = fs::read_to_string(&path)?;
|
|
||||||
let tap: ColumnarTrace = serde_json::from_str(&text).map_err(|source| {
|
|
||||||
TraceStoreError::Parse { file: path.display().to_string(), source }
|
|
||||||
})?;
|
|
||||||
taps.push(tap);
|
|
||||||
}
|
|
||||||
Ok(RunTraces { manifest: index.manifest, taps })
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// What can go wrong reading or writing the trace store.
|
|
||||||
#[derive(Debug)]
|
|
||||||
pub enum TraceStoreError {
|
|
||||||
/// An I/O error reading or writing a trace file.
|
|
||||||
Io(std::io::Error),
|
|
||||||
/// A trace/index file did not parse (the offending file named).
|
|
||||||
Parse { file: String, source: serde_json::Error },
|
|
||||||
/// No recorded run of this name (no `index.json` under `traces/<name>/`).
|
|
||||||
NotFound(String),
|
|
||||||
}
|
|
||||||
|
|
||||||
impl fmt::Display for TraceStoreError {
|
|
||||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
|
||||||
match self {
|
|
||||||
TraceStoreError::Io(e) => write!(f, "trace store i/o: {e}"),
|
|
||||||
TraceStoreError::Parse { file, source } => {
|
|
||||||
write!(f, "trace store parse error in {file}: {source}")
|
|
||||||
}
|
|
||||||
TraceStoreError::NotFound(name) => {
|
|
||||||
write!(f, "no recorded run '{name}' under runs/traces")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl std::error::Error for TraceStoreError {}
|
|
||||||
|
|
||||||
impl From<std::io::Error> for TraceStoreError {
|
|
||||||
fn from(e: std::io::Error) -> Self {
|
|
||||||
TraceStoreError::Io(e)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Run the tests to verify they pass**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-registry trace_store`
|
|
||||||
Expected: PASS — `write_then_read_round_trips`, `read_missing_run_is_not_found`,
|
|
||||||
`corrupt_tap_file_is_a_parse_error_naming_the_file` all green (3 passed).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 3: CLI persist wiring (aura-cli)
|
|
||||||
|
|
||||||
This task is **one compile unit**: the `trace: Option<&str>` parameter on three run
|
|
||||||
fns and the `parse_real_args` 3→4-tuple widen break every caller (including nine
|
|
||||||
in-file test callers), so all caller-threading lands here before the build gate. The
|
|
||||||
RED test (Step 1) is an e2e that compiles against the *binary*, not the changed
|
|
||||||
signatures, so it fails cleanly before any code change.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Test: `crates/aura-cli/tests/cli_run.rs` (new e2e)
|
|
||||||
- Modify: `crates/aura-cli/src/main.rs` (imports :17–26, helper, :158, :193, :264, :823, USAGE :858, argv :867–870, test callers :1099–1100/:1180–1181/:1215/:1220/:1230–1239/:1248–1249)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing e2e test**
|
|
||||||
|
|
||||||
Append to `crates/aura-cli/tests/cli_run.rs` (it already has `BIN` + `temp_cwd`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
#[test]
|
|
||||||
fn run_trace_persists_taps_and_plain_run_writes_no_traces() {
|
|
||||||
let cwd = temp_cwd("run-trace");
|
|
||||||
|
|
||||||
// plain `aura run` (no --trace) persists nothing to disk.
|
|
||||||
let plain = Command::new(BIN).arg("run").current_dir(&cwd).output().expect("spawn aura run");
|
|
||||||
assert!(plain.status.success(), "plain run exit: {:?}", plain.status);
|
|
||||||
assert!(!cwd.join("runs/traces").exists(), "plain run must write no trace files");
|
|
||||||
|
|
||||||
// `aura run --trace demo` persists the two taps + index, and still prints the report.
|
|
||||||
let traced = Command::new(BIN)
|
|
||||||
.args(["run", "--trace", "demo"])
|
|
||||||
.current_dir(&cwd)
|
|
||||||
.output()
|
|
||||||
.expect("spawn aura run --trace");
|
|
||||||
assert!(traced.status.success(), "traced run exit: {:?}", traced.status);
|
|
||||||
let stdout = String::from_utf8(traced.stdout).expect("utf-8 stdout");
|
|
||||||
assert!(stdout.trim_end().starts_with("{\"manifest\":{"), "report still printed: {stdout:?}");
|
|
||||||
assert!(cwd.join("runs/traces/demo/index.json").exists(), "index.json written");
|
|
||||||
assert!(cwd.join("runs/traces/demo/equity.json").exists(), "equity tap written");
|
|
||||||
assert!(cwd.join("runs/traces/demo/exposure.json").exists(), "exposure tap written");
|
|
||||||
|
|
||||||
let _ = std::fs::remove_dir_all(&cwd);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run the e2e to verify it fails**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --test cli_run run_trace_persists_taps_and_plain_run_writes_no_traces`
|
|
||||||
Expected: FAIL — `traced run exit` assertion fails: `["run","--trace","demo"]` has no argv arm yet, so it falls to the usage-error path (exit 2), and no trace files are written. (The tree still compiles — no source change yet.)
|
|
||||||
|
|
||||||
- [ ] **Step 3: Extend the imports**
|
|
||||||
|
|
||||||
In `crates/aura-cli/src/main.rs`, add `ColumnarTrace` to the `aura_engine::{…}`
|
|
||||||
import (`main.rs:18`) and `TraceStore` to the `aura_registry::{…}` import
|
|
||||||
(`main.rs:25`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use aura_engine::{
|
|
||||||
f64_field, monte_carlo, param_stability, summarize, walk_forward, window_of, ColumnarTrace,
|
|
||||||
Composite, Edge, FlatGraph, GraphBuilder, Harness, McAggregate, McFamily, RollMode,
|
|
||||||
RunManifest, RunReport, SourceSpec, SweepFamily, SyntheticSpec, Target, VecSource,
|
|
||||||
WalkForwardResult, WindowBounds, WindowRoller, WindowRun,
|
|
||||||
};
|
|
||||||
use aura_registry::{
|
|
||||||
group_families, mc_member_reports, optimize, rank_by, sweep_member_reports,
|
|
||||||
walkforward_member_reports, FamilyKind, Registry, TraceStore,
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
(Only `TraceStore` is needed this iteration; `persist_traces` Display-formats the
|
|
||||||
error inline, so `TraceStoreError` is not named here — importing it would be an
|
|
||||||
unused import under `-D warnings`. Iteration 2's `emit_chart` adds it.)
|
|
||||||
|
|
||||||
- [ ] **Step 4: Add the `persist_traces` helper**
|
|
||||||
|
|
||||||
Insert directly before `fn run_sample` (`main.rs:158`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// Persist a run's drained taps to the on-disk trace store under `runs/traces/<name>/`
|
|
||||||
/// (beside the run registry's `runs/`). Shared by every run form — all drain the same
|
|
||||||
/// two f64 taps (equity off the broker, exposure off the strategy) and carry a
|
|
||||||
/// `RunManifest`. Pure wiring over `ColumnarTrace` + `TraceStore`; the engine is
|
|
||||||
/// untouched. An I/O failure is a usage-level error (stderr + exit 2), per the spec.
|
|
||||||
fn persist_traces(
|
|
||||||
name: &str,
|
|
||||||
manifest: &RunManifest,
|
|
||||||
eq_rows: &[(Timestamp, Vec<Scalar>)],
|
|
||||||
ex_rows: &[(Timestamp, Vec<Scalar>)],
|
|
||||||
) {
|
|
||||||
let taps = vec![
|
|
||||||
ColumnarTrace::from_rows("equity", &[ScalarKind::F64], eq_rows),
|
|
||||||
ColumnarTrace::from_rows("exposure", &[ScalarKind::F64], ex_rows),
|
|
||||||
];
|
|
||||||
if let Err(e) = TraceStore::open("runs").write(name, manifest, &taps) {
|
|
||||||
eprintln!("aura: trace persist failed: {e}");
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Thread `trace` through `run_sample`**
|
|
||||||
|
|
||||||
Replace `fn run_sample` (`main.rs:158–184`) with (signature gains `trace`, manifest
|
|
||||||
hoisted before the persist call, fold inlined after):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
fn run_sample(trace: Option<&str>) -> RunReport {
|
|
||||||
let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE);
|
|
||||||
let sources: Vec<Box<dyn aura_engine::Source>> =
|
|
||||||
vec![Box::new(VecSource::new(synthetic_prices()))];
|
|
||||||
let window = window_of(&sources).expect("non-empty synthetic stream");
|
|
||||||
h.run(sources);
|
|
||||||
|
|
||||||
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
|
|
||||||
let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
|
|
||||||
let manifest = sim_optimal_manifest(
|
|
||||||
vec![
|
|
||||||
("sma_fast".to_string(), Scalar::i64(2)),
|
|
||||||
("sma_slow".to_string(), Scalar::i64(4)),
|
|
||||||
("exposure_scale".to_string(), Scalar::f64(0.5)),
|
|
||||||
],
|
|
||||||
window,
|
|
||||||
0,
|
|
||||||
SYNTHETIC_PIP_SIZE,
|
|
||||||
);
|
|
||||||
if let Some(name) = trace {
|
|
||||||
persist_traces(name, &manifest, &eq_rows, &ex_rows);
|
|
||||||
}
|
|
||||||
let metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0));
|
|
||||||
RunReport { manifest, metrics }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 6: Thread `trace` through `run_sample_real`**
|
|
||||||
|
|
||||||
In `fn run_sample_real` (`main.rs:193–257`): change the signature to
|
|
||||||
`fn run_sample_real(symbol: &str, from_ms: Option<i64>, to_ms: Option<i64>, trace: Option<&str>) -> RunReport`,
|
|
||||||
then replace the drain-to-RunReport tail (`main.rs:238–256`) with:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
|
|
||||||
let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
|
|
||||||
let manifest = sim_optimal_manifest(
|
|
||||||
vec![
|
|
||||||
("sma_fast".to_string(), Scalar::i64(2)),
|
|
||||||
("sma_slow".to_string(), Scalar::i64(4)),
|
|
||||||
("exposure_scale".to_string(), Scalar::f64(0.5)),
|
|
||||||
],
|
|
||||||
window,
|
|
||||||
0,
|
|
||||||
spec.pip_size,
|
|
||||||
);
|
|
||||||
if let Some(name) = trace {
|
|
||||||
persist_traces(name, &manifest, &eq_rows, &ex_rows);
|
|
||||||
}
|
|
||||||
let metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0));
|
|
||||||
RunReport { manifest, metrics }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 7: Widen `parse_real_args` to a 4-tuple with a `--trace` tail flag**
|
|
||||||
|
|
||||||
Replace `fn parse_real_args` (`main.rs:264–283`) with:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
fn parse_real_args(
|
|
||||||
rest: &[&str],
|
|
||||||
) -> Result<(String, Option<i64>, Option<i64>, Option<String>), String> {
|
|
||||||
let usage = || "run --real <SYMBOL> [--from <ms>] [--to <ms>] [--trace <name>]".to_string();
|
|
||||||
let (symbol, mut tail) = match rest.split_first() {
|
|
||||||
Some((sym, t)) if !sym.is_empty() => (*sym, t),
|
|
||||||
_ => return Err(usage()),
|
|
||||||
};
|
|
||||||
let mut from: Option<i64> = None;
|
|
||||||
let mut to: Option<i64> = None;
|
|
||||||
let mut trace: Option<String> = None;
|
|
||||||
while let Some((flag, t)) = tail.split_first() {
|
|
||||||
let (value, t) = t.split_first().ok_or_else(usage)?;
|
|
||||||
match *flag {
|
|
||||||
"--from" if from.is_none() => from = Some(value.parse().map_err(|_| usage())?),
|
|
||||||
"--to" if to.is_none() => to = Some(value.parse().map_err(|_| usage())?),
|
|
||||||
"--trace" if trace.is_none() => trace = Some((*value).to_string()),
|
|
||||||
_ => return Err(usage()),
|
|
||||||
}
|
|
||||||
tail = t;
|
|
||||||
}
|
|
||||||
Ok((symbol.to_string(), from, to, trace))
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 8: Thread `trace` through `run_macd`**
|
|
||||||
|
|
||||||
In `fn run_macd` (`main.rs:823–856`): change the signature to
|
|
||||||
`fn run_macd(trace: Option<&str>) -> RunReport`, then replace the drain-to-RunReport
|
|
||||||
tail (`main.rs:836–855`) with:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
|
|
||||||
let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
|
|
||||||
let manifest = sim_optimal_manifest(
|
|
||||||
vec![
|
|
||||||
("ema_fast".to_string(), Scalar::i64(2)),
|
|
||||||
("ema_slow".to_string(), Scalar::i64(4)),
|
|
||||||
("ema_signal".to_string(), Scalar::i64(3)),
|
|
||||||
("exposure_scale".to_string(), Scalar::f64(0.5)),
|
|
||||||
],
|
|
||||||
window,
|
|
||||||
0,
|
|
||||||
SYNTHETIC_PIP_SIZE,
|
|
||||||
);
|
|
||||||
if let Some(name) = trace {
|
|
||||||
persist_traces(name, &manifest, &eq_rows, &ex_rows);
|
|
||||||
}
|
|
||||||
let metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0));
|
|
||||||
RunReport { manifest, metrics }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 9: Update the argv arms**
|
|
||||||
|
|
||||||
Replace the three run arms (`main.rs:867–875` — `["run"]`, `["run","--macd"]`, the
|
|
||||||
`["run","--real", rest @ ..]` block) with:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
["run"] => println!("{}", run_sample(None).to_json()),
|
|
||||||
["run", "--macd"] => println!("{}", run_macd(None).to_json()),
|
|
||||||
["run", "--trace", name] => println!("{}", run_sample(Some(name)).to_json()),
|
|
||||||
["run", "--macd", "--trace", name] => println!("{}", run_macd(Some(name)).to_json()),
|
|
||||||
["run", "--real", rest @ ..] => match parse_real_args(rest) {
|
|
||||||
Ok((sym, from, to, trace)) => {
|
|
||||||
println!("{}", run_sample_real(&sym, from, to, trace.as_deref()).to_json())
|
|
||||||
}
|
|
||||||
Err(msg) => {
|
|
||||||
eprintln!("aura: {msg}");
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
},
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 10: Update `USAGE`**
|
|
||||||
|
|
||||||
Replace the `USAGE` const (`main.rs:858–859`) with (one line; adds `[--trace <name>]`
|
|
||||||
on both run forms — the `chart` verb is added in iteration 2):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
const USAGE: &str =
|
|
||||||
"usage: aura run [--macd] [--trace <name>] | aura run --real <SYMBOL> [--from <ms>] [--to <ms>] [--trace <name>] | aura graph | aura sweep [--name <n>] | aura mc [--name <n>] | aura walkforward [--name <n>] | aura runs families | aura runs family <id> [rank <metric>]";
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 11: Migrate the nine in-file test callers**
|
|
||||||
|
|
||||||
These break on the signature/tuple changes; update each in place:
|
|
||||||
|
|
||||||
- `main.rs:1099–1100`: `run_macd()` → `run_macd(None)` (both lines).
|
|
||||||
- `main.rs:1248–1249`: `run_sample()` → `run_sample(None)` (both lines).
|
|
||||||
- `main.rs:1180`, `1181`, `1215`, `1220`: append `, None` as the 4th arg to each
|
|
||||||
`run_sample_real(SYMBOL_OR_"GER40", Some(..), Some(..))` call, e.g.
|
|
||||||
`run_sample_real(SYMBOL, Some(GER40_SEP2024_FROM_MS), Some(GER40_SEP2024_TO_MS), None)`.
|
|
||||||
- `parse_real_args` test (`main.rs:1229–1244`): replace the body with the 4-tuple
|
|
||||||
shape + a new `--trace` case:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
fn parse_real_args_accepts_symbol_and_optional_window() {
|
|
||||||
assert_eq!(parse_real_args(&["EURUSD"]), Ok(("EURUSD".to_string(), None, None, None)));
|
|
||||||
assert_eq!(
|
|
||||||
parse_real_args(&["EURUSD", "--from", "100", "--to", "200"]),
|
|
||||||
Ok(("EURUSD".to_string(), Some(100), Some(200), None))
|
|
||||||
);
|
|
||||||
// flags in any order
|
|
||||||
assert_eq!(
|
|
||||||
parse_real_args(&["EURUSD", "--to", "200", "--from", "100"]),
|
|
||||||
Ok(("EURUSD".to_string(), Some(100), Some(200), None))
|
|
||||||
);
|
|
||||||
// the --trace tail flag carries a string name, parsed alongside the window
|
|
||||||
assert_eq!(
|
|
||||||
parse_real_args(&["EURUSD", "--from", "100", "--trace", "demo"]),
|
|
||||||
Ok(("EURUSD".to_string(), Some(100), None, Some("demo".to_string())))
|
|
||||||
);
|
|
||||||
// a flag without its value, an empty symbol, and a non-numeric ms all reject
|
|
||||||
assert!(parse_real_args(&["EURUSD", "--from"]).is_err());
|
|
||||||
assert!(parse_real_args(&[]).is_err());
|
|
||||||
assert!(parse_real_args(&["EURUSD", "--from", "notanumber"]).is_err());
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 12: Build gate — workspace compiles, full aura-cli suite green**
|
|
||||||
|
|
||||||
Run: `cargo build --workspace`
|
|
||||||
Expected: PASS — 0 errors (all callers threaded; no unresolved signature/tuple breaks).
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli`
|
|
||||||
Expected: PASS — the new e2e `run_trace_persists_taps_and_plain_run_writes_no_traces`
|
|
||||||
passes, the updated `parse_real_args_accepts_symbol_and_optional_window` passes, and
|
|
||||||
the byte-for-byte default-run pins (`run_prints_json_and_exits_zero`,
|
|
||||||
`run_with_trailing_token_is_strict_and_exits_two`) stay green (default run unchanged).
|
|
||||||
|
|
||||||
- [ ] **Step 13: Workspace regression gate**
|
|
||||||
|
|
||||||
Run: `cargo test --workspace`
|
|
||||||
Expected: PASS — Task 1 (`columnar_trace_*`) + Task 2 (`trace_store::tests::*`) + the
|
|
||||||
CLI suite all green; no regression elsewhere.
|
|
||||||
|
|
||||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
|
||||||
Expected: PASS — no warnings. (Watch for: an unused import if `TraceStore`/`ColumnarTrace`
|
|
||||||
is imported but a step is skipped; a `clippy::needless_range_loop` on `from_rows`'s
|
|
||||||
index loop — if flagged, the `.enumerate()` form already shown avoids it.)
|
|
||||||
@@ -1,583 +0,0 @@
|
|||||||
# Trace charts — iteration 2 (serve) — Implementation Plan
|
|
||||||
|
|
||||||
> **Parent spec:** `docs/specs/0056-trace-charts-persist-and-serve.md`
|
|
||||||
>
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
|
|
||||||
> this plan. Steps use `- [ ]` checkboxes for tracking.
|
|
||||||
|
|
||||||
**Goal:** Read a persisted run's traces (`runs/traces/<name>/`) and serve them as a
|
|
||||||
self-contained uPlot HTML chart via `aura chart <name> [--panels]` — feeds overlaid
|
|
||||||
on a shared timestamp axis (default) or in timestamp-aligned stacked panels.
|
|
||||||
|
|
||||||
**Architecture:** Three layers on top of iteration 1's on-disk artifact. A
|
|
||||||
first-party `chart-viewer.js` exposes a pure `buildCharts(traces, mode)` (the
|
|
||||||
`AURA_TRACES` → uPlot-config transform, no DOM) plus a browser-only `mount()`.
|
|
||||||
`render_chart_html` (aura-cli/render.rs) assembles a self-contained page mirroring
|
|
||||||
`render_html`: vendored uPlot (already checked in, commit d534f10) + `chart-viewer.js`
|
|
||||||
inlined via `include_str!`, the data injected as `window.AURA_TRACES`. `emit_chart`
|
|
||||||
(aura-cli/main.rs) reads the `TraceStore`, builds `ChartData` by the spec-§6 3-step
|
|
||||||
union-spine `join_on_ts` alignment, and prints the page. The engine is untouched (C14).
|
|
||||||
|
|
||||||
**Tech Stack:** Rust (`aura-cli`); the iteration-1 building blocks
|
|
||||||
`aura_engine::{ColumnarTrace, join_on_ts, JoinedRow}` + `aura_registry::{TraceStore,
|
|
||||||
RunTraces, TraceStoreError}`; vendored uPlot 1.6.32; `serde_json` (already an
|
|
||||||
aura-cli dep); a `node` `.mjs` DOM guard mirroring `viewer_dot.rs`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Files this plan creates or modifies:**
|
|
||||||
|
|
||||||
- Create: `crates/aura-cli/assets/chart-viewer.js` — first-party `AURA_TRACES` →
|
|
||||||
uPlot bootstrap (pure `buildCharts` + browser `mount`).
|
|
||||||
- Create: `crates/aura-cli/tests/chart_viewer.rs` — `.rs` node-driver (shells `node`).
|
|
||||||
- Create: `crates/aura-cli/tests/chart_viewer_build.mjs` — the DOM-less guard over
|
|
||||||
`buildCharts`.
|
|
||||||
- Create: `crates/aura-cli/tests/fixtures/sample-traces.json` — an `AURA_TRACES`
|
|
||||||
fixture for the guard.
|
|
||||||
- Modify: `crates/aura-cli/src/render.rs` — add `ChartMode`/`ChartData`/`Series` +
|
|
||||||
`render_chart_html` + a self-contained unit test (mirror `render.rs:92`).
|
|
||||||
- Modify: `crates/aura-cli/src/main.rs` — imports (`:17–26`), `emit_chart` + the
|
|
||||||
`build_chart_data` alignment helper, the `["chart", …]` argv arms (`:885–914`,
|
|
||||||
before `["graph"]` at `:899`), `USAGE` (`:877–878`).
|
|
||||||
- Test: `crates/aura-cli/tests/cli_run.rs` — a `chart` e2e sibling (HTML emitted;
|
|
||||||
missing run exits 2).
|
|
||||||
|
|
||||||
uPlot is already vendored (commit d534f10: `assets/uPlot.iife.min.js` +
|
|
||||||
`uPlot.min.css` + the `refresh-assets.sh` line) — no fetch in this plan.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 1: `chart-viewer.js` + its DOM guard
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `crates/aura-cli/assets/chart-viewer.js`
|
|
||||||
- Create: `crates/aura-cli/tests/chart_viewer_build.mjs`
|
|
||||||
- Create: `crates/aura-cli/tests/chart_viewer.rs`
|
|
||||||
- Create: `crates/aura-cli/tests/fixtures/sample-traces.json`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the fixture**
|
|
||||||
|
|
||||||
Create `crates/aura-cli/tests/fixtures/sample-traces.json` (an `AURA_TRACES` payload
|
|
||||||
with two co-cadenced single-column series, matching what `render_chart_html` injects):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"mode": "overlay",
|
|
||||||
"manifest": { "commit": "test", "params": [], "window": [1, 3], "seed": 0, "broker": "sim-optimal(pip_size=0.0001)" },
|
|
||||||
"taps": ["equity", "exposure"],
|
|
||||||
"xs": [1, 2, 3],
|
|
||||||
"series": [
|
|
||||||
{ "name": "equity", "y_scale_id": "y_0", "points": [0.0, 0.001, 0.004] },
|
|
||||||
{ "name": "exposure", "y_scale_id": "y_1", "points": [1.0, 1.0, -1.0] }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Write the failing guard (.mjs + .rs driver)**
|
|
||||||
|
|
||||||
Create `crates/aura-cli/tests/chart_viewer_build.mjs`:
|
|
||||||
|
|
||||||
```js
|
|
||||||
// Headless guard for the trace-chart viewer (chart-viewer.js). Property protected:
|
|
||||||
// buildCharts maps an AURA_TRACES payload to uPlot configs WITHOUT a DOM — overlay
|
|
||||||
// is one chart with every series on its own y-scale over the shared xs; panels is
|
|
||||||
// one chart per series, each on the SAME shared xs (timestamp-aligned). It loads the
|
|
||||||
// real viewer module (a fix/regression there is observed here) and drives the
|
|
||||||
// exported pure `buildCharts`; uPlot itself (which needs `document`) is never
|
|
||||||
// instantiated — mount() is browser-only and not called here.
|
|
||||||
|
|
||||||
import { createRequire } from "node:module";
|
|
||||||
import { readFileSync } from "node:fs";
|
|
||||||
import { fileURLToPath } from "node:url";
|
|
||||||
import { dirname, join } from "node:path";
|
|
||||||
|
|
||||||
const require = createRequire(import.meta.url);
|
|
||||||
const here = dirname(fileURLToPath(import.meta.url));
|
|
||||||
const traces = JSON.parse(readFileSync(join(here, "fixtures", "sample-traces.json"), "utf8"));
|
|
||||||
|
|
||||||
const fail = (msg) => { console.error("FAIL: " + msg); process.exit(1); };
|
|
||||||
const eq = (a, b, msg) => { if (a !== b) fail(`${msg} (got ${a}, want ${b})`); };
|
|
||||||
const sameArr = (a, b, msg) => {
|
|
||||||
if (!Array.isArray(a) || !Array.isArray(b) || a.length !== b.length || a.some((v, i) => v !== b[i])) fail(msg);
|
|
||||||
};
|
|
||||||
|
|
||||||
// No `document` global -> chart-viewer.js's browser mount is skipped; require gives
|
|
||||||
// us only the pure export (mirrors viewer_dot_ids.mjs stubbing window before require).
|
|
||||||
global.window = { AURA_TRACES: traces };
|
|
||||||
const { buildCharts } = require(join(here, "..", "assets", "chart-viewer.js"));
|
|
||||||
|
|
||||||
const n = traces.series.length;
|
|
||||||
|
|
||||||
// overlay: ONE chart, all series superimposed on the shared xs, each on its own scale.
|
|
||||||
const overlay = buildCharts(traces, "overlay");
|
|
||||||
eq(overlay.length, 1, "overlay is one chart");
|
|
||||||
eq(overlay[0].data.length, 1 + n, "overlay data = xs + N series");
|
|
||||||
eq(overlay[0].opts.series.length, 1 + n, "overlay uPlot series = x + N");
|
|
||||||
sameArr(overlay[0].data[0], traces.xs, "overlay x-axis is the shared xs");
|
|
||||||
const scaleIds = overlay[0].opts.series.slice(1).map((s) => s.scale);
|
|
||||||
eq(new Set(scaleIds).size, n, "each overlay series has a distinct y-scale");
|
|
||||||
|
|
||||||
// panels: ONE chart per series, all timestamp-aligned on the SAME shared xs.
|
|
||||||
const panels = buildCharts(traces, "panels");
|
|
||||||
eq(panels.length, n, "panels = one chart per series");
|
|
||||||
for (const p of panels) {
|
|
||||||
eq(p.data.length, 2, "each panel = xs + its one series");
|
|
||||||
sameArr(p.data[0], traces.xs, "panel shares the same xs (timestamp-aligned)");
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log(`OK — overlay 1 chart / ${n} series, panels ${n} charts, shared xs.`);
|
|
||||||
process.exit(0);
|
|
||||||
```
|
|
||||||
|
|
||||||
Create `crates/aura-cli/tests/chart_viewer.rs` (mirrors `viewer_dot.rs`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
//! Integration guard for the `aura chart` viewer's data transform.
|
|
||||||
//!
|
|
||||||
//! `chart-viewer.js` builds the uPlot configs in JavaScript, so the property
|
|
||||||
//! "buildCharts maps AURA_TRACES to the right per-mode uPlot config shape" cannot
|
|
||||||
//! be checked from Rust. This shells out to `node`, running
|
|
||||||
//! `tests/chart_viewer_build.mjs`, which loads the real viewer module and asserts
|
|
||||||
//! the overlay/panels config shape over a fixture — without instantiating uPlot
|
|
||||||
//! (which needs a DOM). `node` is REQUIRED: if absent this test FAILS.
|
|
||||||
|
|
||||||
use std::path::PathBuf;
|
|
||||||
use std::process::Command;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn chart_viewer_builds_overlay_and_panels_configs() {
|
|
||||||
let script = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
|
||||||
.join("tests")
|
|
||||||
.join("chart_viewer_build.mjs");
|
|
||||||
assert!(script.exists(), "guard script missing at {}", script.display());
|
|
||||||
|
|
||||||
let out = match Command::new("node").arg(&script).output() {
|
|
||||||
Ok(out) => out,
|
|
||||||
Err(e) => panic!(
|
|
||||||
"node is required for the chart-viewer guard but could not be run \
|
|
||||||
({e}); install Node.js or ensure `node` is on PATH"
|
|
||||||
),
|
|
||||||
};
|
|
||||||
|
|
||||||
let stdout = String::from_utf8_lossy(&out.stdout);
|
|
||||||
let stderr = String::from_utf8_lossy(&out.stderr);
|
|
||||||
assert!(
|
|
||||||
out.status.success(),
|
|
||||||
"chart-viewer guard failed (exit {:?}).\n--- node stdout ---\n{stdout}\n--- node stderr ---\n{stderr}",
|
|
||||||
out.status.code()
|
|
||||||
);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Run the guard to verify it fails**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --test chart_viewer chart_viewer_builds_overlay_and_panels_configs`
|
|
||||||
Expected: FAIL — node cannot `require` `assets/chart-viewer.js` (it does not exist yet), so the guard exits non-zero and the `.rs` assert reports the node stderr.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Write `chart-viewer.js`**
|
|
||||||
|
|
||||||
Create `crates/aura-cli/assets/chart-viewer.js`:
|
|
||||||
|
|
||||||
```js
|
|
||||||
// The `aura chart` viewer: turns the injected window.AURA_TRACES (xs + per-series
|
|
||||||
// Option<f64> points, already timestamp-union-aligned by the CLI) into uPlot charts.
|
|
||||||
//
|
|
||||||
// buildCharts is PURE (no DOM, no uPlot) and is the unit the headless guard drives.
|
|
||||||
// mount() is browser-only — it instantiates the vendored global `uPlot` per config.
|
|
||||||
(function () {
|
|
||||||
var PALETTE = ["#89b4fa", "#f38ba8", "#a6e3a1", "#f9e2af", "#cba6f7", "#94e2d5"];
|
|
||||||
|
|
||||||
// {xs, series:[{name, y_scale_id, points}]} + mode -> array of {opts, data} configs.
|
|
||||||
// overlay -> one chart, all series, each on its own auto-ranged y-scale (shared xs).
|
|
||||||
// panels -> one chart per series, each on the SAME shared xs (timestamp-aligned).
|
|
||||||
function buildCharts(traces, mode) {
|
|
||||||
var xs = traces.xs;
|
|
||||||
var series = traces.series;
|
|
||||||
if (mode === "panels") {
|
|
||||||
return series.map(function (s, i) {
|
|
||||||
return {
|
|
||||||
opts: {
|
|
||||||
title: s.name,
|
|
||||||
scales: { x: { time: false }, [s.y_scale_id]: {} },
|
|
||||||
series: [{}, { label: s.name, scale: s.y_scale_id, stroke: PALETTE[i % PALETTE.length] }],
|
|
||||||
axes: [{}, { scale: s.y_scale_id }],
|
|
||||||
},
|
|
||||||
data: [xs, s.points],
|
|
||||||
};
|
|
||||||
});
|
|
||||||
}
|
|
||||||
// overlay (default)
|
|
||||||
var scales = { x: { time: false } };
|
|
||||||
var uSeries = [{}];
|
|
||||||
var axes = [{}];
|
|
||||||
series.forEach(function (s, i) {
|
|
||||||
scales[s.y_scale_id] = {};
|
|
||||||
uSeries.push({ label: s.name, scale: s.y_scale_id, stroke: PALETTE[i % PALETTE.length] });
|
|
||||||
// the first two series carry a drawn axis (left, then right); the rest keep
|
|
||||||
// their own auto-scale, read via the legend.
|
|
||||||
if (i < 2) axes.push({ scale: s.y_scale_id, side: i === 0 ? 3 : 1 });
|
|
||||||
});
|
|
||||||
return [{ opts: { scales: scales, series: uSeries, axes: axes }, data: [xs].concat(series.map(function (s) { return s.points; })) }];
|
|
||||||
}
|
|
||||||
|
|
||||||
// Browser-only: instantiate the vendored global uPlot per config into #stage.
|
|
||||||
function mount() {
|
|
||||||
var traces = window.AURA_TRACES;
|
|
||||||
var configs = buildCharts(traces, traces.mode);
|
|
||||||
var stage = document.getElementById("stage");
|
|
||||||
var w = stage.clientWidth || 900;
|
|
||||||
var h = configs.length > 1 ? Math.max(160, Math.floor(440 / configs.length)) : 460;
|
|
||||||
configs.forEach(function (cfg) {
|
|
||||||
var opts = Object.assign({ width: w, height: h }, cfg.opts);
|
|
||||||
// eslint-disable-next-line no-undef
|
|
||||||
new uPlot(opts, cfg.data, stage);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
if (typeof window !== "undefined" && typeof document !== "undefined" && window.AURA_TRACES) {
|
|
||||||
if (document.readyState === "loading") document.addEventListener("DOMContentLoaded", mount);
|
|
||||||
else mount();
|
|
||||||
}
|
|
||||||
if (typeof module !== "undefined" && module.exports) module.exports = { buildCharts: buildCharts };
|
|
||||||
})();
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Run the guard to verify it passes**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --test chart_viewer chart_viewer_builds_overlay_and_panels_configs`
|
|
||||||
Expected: PASS — `OK — overlay 1 chart / 2 series, panels 2 charts, shared xs.` (1 passed).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 2: `render_chart_html` + chart types (render.rs)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `crates/aura-cli/src/render.rs` (types + fn after `render_html`; test in `mod tests`)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing self-contained test**
|
|
||||||
|
|
||||||
Append inside the existing `#[cfg(test)] mod tests { … }` in `crates/aura-cli/src/render.rs`:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
fn sample_chart_data() -> ChartData {
|
|
||||||
ChartData {
|
|
||||||
manifest: aura_engine::RunManifest {
|
|
||||||
commit: "test".into(),
|
|
||||||
params: vec![],
|
|
||||||
window: (aura_engine::Timestamp(1), aura_engine::Timestamp(3)),
|
|
||||||
seed: 0,
|
|
||||||
broker: "sim-optimal(pip_size=0.0001)".into(),
|
|
||||||
},
|
|
||||||
taps: vec!["equity".into(), "exposure".into()],
|
|
||||||
xs: vec![1, 2, 3],
|
|
||||||
series: vec![
|
|
||||||
Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(0.0), Some(0.001), Some(0.004)] },
|
|
||||||
Series { name: "exposure".into(), y_scale_id: "y_1".into(), points: vec![Some(1.0), Some(1.0), Some(-1.0)] },
|
|
||||||
],
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn render_chart_html_is_self_contained_and_injects_traces() {
|
|
||||||
let html = render_chart_html(&sample_chart_data(), ChartMode::Overlay);
|
|
||||||
assert!(html.starts_with("<!doctype html>"), "not an HTML doc");
|
|
||||||
assert!(html.trim_end().ends_with("</html>"), "HTML doc not closed");
|
|
||||||
// self-contained: no remote asset
|
|
||||||
assert!(!html.contains("<script src"), "assets must be inlined");
|
|
||||||
// data injected as the viewer's source, with the mode reflected
|
|
||||||
assert!(html.contains("window.AURA_TRACES = {"), "AURA_TRACES not injected");
|
|
||||||
assert!(html.contains("\"mode\":\"overlay\""), "overlay mode not reflected");
|
|
||||||
assert!(html.contains("\"equity\""), "series name missing from injected data");
|
|
||||||
// the vendored uPlot bundle + first-party viewer are inlined
|
|
||||||
assert!(html.contains("uPlot=function"), "vendored uPlot bundle missing");
|
|
||||||
assert!(html.contains("buildCharts"), "chart-viewer.js not inlined");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn render_chart_html_reflects_panels_mode() {
|
|
||||||
let html = render_chart_html(&sample_chart_data(), ChartMode::Panels);
|
|
||||||
assert!(html.contains("\"mode\":\"panels\""), "panels mode not reflected");
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run the test to verify it fails**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --lib render_chart_html`
|
|
||||||
Expected: FAIL — compile error `cannot find type ChartData / Series / ChartMode / function render_chart_html` (not written yet).
|
|
||||||
|
|
||||||
- [ ] **Step 3: Write the types + `render_chart_html`**
|
|
||||||
|
|
||||||
In `crates/aura-cli/src/render.rs`, change the import line (`render.rs:9`) from
|
|
||||||
`use aura_engine::Composite;` to:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use aura_engine::{Composite, RunManifest};
|
|
||||||
```
|
|
||||||
|
|
||||||
Add the three vendored/first-party asset consts beside the existing ones
|
|
||||||
(`render.rs:55–57`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
const UPLOT_JS: &str = include_str!("../assets/uPlot.iife.min.js");
|
|
||||||
const UPLOT_CSS: &str = include_str!("../assets/uPlot.min.css");
|
|
||||||
const CHART_VIEWER_JS: &str = include_str!("../assets/chart-viewer.js");
|
|
||||||
```
|
|
||||||
|
|
||||||
Add this AFTER `render_html` ends (`render.rs:80`) and before `#[cfg(test)]`
|
|
||||||
(`render.rs:82`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// Which way the chart draws the (timestamp-aligned) series.
|
|
||||||
#[derive(Clone, Copy)]
|
|
||||||
pub enum ChartMode {
|
|
||||||
/// One plot, all series superimposed, each on its own y-scale.
|
|
||||||
Overlay,
|
|
||||||
/// One stacked plot per series, all sharing the timestamp x-axis.
|
|
||||||
Panels,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// One chart series: a name, its own y-scale id (so disparate magnitudes stay
|
|
||||||
/// legible), and one `Option<f64>` per shared-x slot (`null` = a gap where the tap
|
|
||||||
/// did not fire at that timestamp).
|
|
||||||
#[derive(serde::Serialize)]
|
|
||||||
pub struct Series {
|
|
||||||
pub name: String,
|
|
||||||
pub y_scale_id: String,
|
|
||||||
pub points: Vec<Option<f64>>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The serve-ready chart payload: the shared timestamp axis, the per-(tap,column)
|
|
||||||
/// series, and the run's manifest + tap list (for the page header). Injected verbatim
|
|
||||||
/// as `window.AURA_TRACES` (plus the `mode`); `chart-viewer.js` renders it.
|
|
||||||
pub struct ChartData {
|
|
||||||
pub manifest: RunManifest,
|
|
||||||
pub taps: Vec<String>,
|
|
||||||
pub xs: Vec<i64>,
|
|
||||||
pub series: Vec<Series>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Assemble the self-contained uPlot chart page for one run's aligned traces.
|
|
||||||
///
|
|
||||||
/// Order is load-bearing (mirrors `render_html`): the uPlot global and CSS, then the
|
|
||||||
/// injected `window.AURA_TRACES`, must be defined before `chart-viewer.js` runs.
|
|
||||||
pub fn render_chart_html(data: &ChartData, mode: ChartMode) -> String {
|
|
||||||
let mode_str = match mode {
|
|
||||||
ChartMode::Overlay => "overlay",
|
|
||||||
ChartMode::Panels => "panels",
|
|
||||||
};
|
|
||||||
let traces = serde_json::json!({
|
|
||||||
"mode": mode_str,
|
|
||||||
"manifest": data.manifest,
|
|
||||||
"taps": data.taps,
|
|
||||||
"xs": data.xs,
|
|
||||||
"series": data.series,
|
|
||||||
})
|
|
||||||
.to_string();
|
|
||||||
|
|
||||||
let mut html = String::with_capacity(
|
|
||||||
SHELL_HEAD.len() + UPLOT_JS.len() + UPLOT_CSS.len() + CHART_VIEWER_JS.len() + traces.len() + 256,
|
|
||||||
);
|
|
||||||
html.push_str(SHELL_HEAD);
|
|
||||||
html.push_str("<style>");
|
|
||||||
html.push_str(UPLOT_CSS);
|
|
||||||
html.push_str("</style>\n<script>");
|
|
||||||
html.push_str(UPLOT_JS);
|
|
||||||
html.push_str("</script>\n<script>window.AURA_TRACES = ");
|
|
||||||
html.push_str(&traces);
|
|
||||||
html.push_str(";</script>\n<script>");
|
|
||||||
html.push_str(CHART_VIEWER_JS);
|
|
||||||
html.push_str("</script>\n</body>\n</html>\n");
|
|
||||||
html
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
(`serde::Serialize` on `Series` lets `serde_json::json!` embed it; `RunManifest`
|
|
||||||
already derives `Serialize`. `serde_json` is already an aura-cli dependency — used by
|
|
||||||
`main.rs`'s `runs_family` JSON.)
|
|
||||||
|
|
||||||
- [ ] **Step 4: Run the test to verify it passes**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --lib render_chart_html`
|
|
||||||
Expected: PASS — `render_chart_html_is_self_contained_and_injects_traces` +
|
|
||||||
`render_chart_html_reflects_panels_mode` green (2 passed).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 3: `emit_chart` + serve-time alignment + the `chart` verb (main.rs)
|
|
||||||
|
|
||||||
This task is one compile unit (new `emit_chart`/`build_chart_data` + two argv arms +
|
|
||||||
imports + USAGE); it changes no existing signature, so there is no caller blast
|
|
||||||
radius. The RED e2e (Step 1) compiles against the binary and fails before the arms
|
|
||||||
exist.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Test: `crates/aura-cli/tests/cli_run.rs` (chart e2e sibling)
|
|
||||||
- Modify: `crates/aura-cli/src/main.rs` (imports `:17–26`, USAGE `:877–878`, argv
|
|
||||||
`:885–914`, new fns)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing e2e**
|
|
||||||
|
|
||||||
Append to `crates/aura-cli/tests/cli_run.rs`:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
#[test]
|
|
||||||
fn chart_emits_self_contained_html_for_a_persisted_run() {
|
|
||||||
let cwd = temp_cwd("chart-serve");
|
|
||||||
|
|
||||||
// persist a run first (iteration 1), then chart it.
|
|
||||||
let traced = Command::new(BIN).args(["run", "--trace", "demo"]).current_dir(&cwd).output().expect("spawn run --trace");
|
|
||||||
assert!(traced.status.success(), "run --trace exit: {:?}", traced.status);
|
|
||||||
|
|
||||||
let chart = Command::new(BIN).args(["chart", "demo"]).current_dir(&cwd).output().expect("spawn chart");
|
|
||||||
assert!(chart.status.success(), "chart exit: {:?}", chart.status);
|
|
||||||
let html = String::from_utf8(chart.stdout).expect("utf-8 html");
|
|
||||||
assert!(html.starts_with("<!doctype html>"), "not an HTML doc: {:?}", &html[..html.len().min(40)]);
|
|
||||||
assert!(html.contains("window.AURA_TRACES = {"), "AURA_TRACES not injected");
|
|
||||||
assert!(html.contains("\"equity\""), "equity series missing from the chart");
|
|
||||||
assert!(html.contains("uPlot=function"), "vendored uPlot not inlined");
|
|
||||||
|
|
||||||
// a missing run is a usage error (stderr + exit 2), not a panic.
|
|
||||||
let missing = Command::new(BIN).args(["chart", "nope"]).current_dir(&cwd).output().expect("spawn chart nope");
|
|
||||||
assert_eq!(missing.status.code(), Some(2), "missing run must exit 2");
|
|
||||||
assert!(!missing.status.success());
|
|
||||||
|
|
||||||
let _ = std::fs::remove_dir_all(&cwd);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run the e2e to verify it fails**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --test cli_run chart_emits_self_contained_html_for_a_persisted_run`
|
|
||||||
Expected: FAIL — `["chart","demo"]` has no argv arm yet, so it hits the usage-error path (exit 2) and prints no HTML. (Tree compiles — only a new test added.)
|
|
||||||
|
|
||||||
- [ ] **Step 3: Extend the imports**
|
|
||||||
|
|
||||||
In `crates/aura-cli/src/main.rs`, add `join_on_ts, JoinedRow` to the
|
|
||||||
`aura_engine::{…}` import (`main.rs:17–22`) and `RunTraces, TraceStoreError` to the
|
|
||||||
`aura_registry::{…}` import (`main.rs:23–26`). Also bring in the new render types:
|
|
||||||
the file already has `mod render;`, so reference them as `render::{ChartData, ChartMode, Series}`
|
|
||||||
at the call sites (no `use` needed) — OR add `use render::{ChartData, ChartMode, Series};`
|
|
||||||
after `mod render;` (`main.rs:14`). Use the explicit `use`:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
mod render;
|
|
||||||
use render::{ChartData, ChartMode, Series};
|
|
||||||
```
|
|
||||||
|
|
||||||
and the two engine/registry import lists become (only the added names shown in context):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use aura_engine::{
|
|
||||||
f64_field, join_on_ts, monte_carlo, param_stability, summarize, walk_forward, window_of,
|
|
||||||
ColumnarTrace, Composite, Edge, FlatGraph, GraphBuilder, Harness, JoinedRow, McAggregate,
|
|
||||||
McFamily, RollMode, RunManifest, RunReport, SourceSpec, SweepFamily, SyntheticSpec, Target,
|
|
||||||
VecSource, WalkForwardResult, WindowBounds, WindowRoller, WindowRun,
|
|
||||||
};
|
|
||||||
use aura_registry::{
|
|
||||||
group_families, mc_member_reports, optimize, rank_by, sweep_member_reports,
|
|
||||||
walkforward_member_reports, FamilyKind, Registry, RunTraces, TraceStore, TraceStoreError,
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Add the alignment helper + `emit_chart`**
|
|
||||||
|
|
||||||
Insert after `persist_traces` (it ends near `main.rs:174`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// Build the serve-ready `ChartData` from a run's read-back traces by the spec-§6
|
|
||||||
/// 3-step union-spine alignment — no tap privileged, no point dropped:
|
|
||||||
/// (1) xs = the sorted, deduped union of every tap's timestamps;
|
|
||||||
/// (2) synthesize an empty-payload spine over xs and pass ALL taps (via
|
|
||||||
/// `ColumnarTrace::to_rows`, which yields uniformly-f64 rows) as symmetric sides
|
|
||||||
/// of `join_on_ts`, so no side row is dropped and none occupies the privileged
|
|
||||||
/// `JoinedRow.spine`;
|
|
||||||
/// (3) flatten each (tap, column) to a `Series` of `Option<f64>` over xs.
|
|
||||||
fn build_chart_data(traces: RunTraces) -> ChartData {
|
|
||||||
let mut xs: Vec<i64> = traces.taps.iter().flat_map(|t| t.ts.iter().copied()).collect();
|
|
||||||
xs.sort_unstable();
|
|
||||||
xs.dedup();
|
|
||||||
|
|
||||||
let spine: Vec<(Timestamp, Vec<Scalar>)> = xs.iter().map(|&t| (Timestamp(t), Vec::new())).collect();
|
|
||||||
let tap_rows: Vec<Vec<(Timestamp, Vec<Scalar>)>> = traces.taps.iter().map(|t| t.to_rows()).collect();
|
|
||||||
let sides: Vec<&[(Timestamp, Vec<Scalar>)]> = tap_rows.iter().map(|r| r.as_slice()).collect();
|
|
||||||
let joined: Vec<JoinedRow> = join_on_ts(&spine, &sides);
|
|
||||||
|
|
||||||
let mut series: Vec<Series> = Vec::new();
|
|
||||||
for (i, tap) in traces.taps.iter().enumerate() {
|
|
||||||
for c in 0..tap.columns.len() {
|
|
||||||
let name = if tap.columns.len() == 1 { tap.tap.clone() } else { format!("{}[{c}]", tap.tap) };
|
|
||||||
let y_scale_id = format!("y_{}", series.len());
|
|
||||||
let points: Vec<Option<f64>> =
|
|
||||||
joined.iter().map(|r| r.sides[i].as_ref().map(|row| row[c].as_f64())).collect();
|
|
||||||
series.push(Series { name, y_scale_id, points });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
ChartData {
|
|
||||||
manifest: traces.manifest,
|
|
||||||
taps: traces.taps.iter().map(|t| t.tap.clone()).collect(),
|
|
||||||
xs,
|
|
||||||
series,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `aura chart <name> [--panels]`: read the persisted run's traces, align them, and
|
|
||||||
/// print the self-contained uPlot page. A missing run is a usage error (stderr +
|
|
||||||
/// exit 2), per the CLI's convention — never a panic.
|
|
||||||
fn emit_chart(name: &str, mode: ChartMode) {
|
|
||||||
let traces = match TraceStore::open("runs").read(name) {
|
|
||||||
Ok(t) => t,
|
|
||||||
Err(TraceStoreError::NotFound(n)) => {
|
|
||||||
eprintln!("aura: no recorded run '{n}' under runs/traces (run `aura run --trace {n}` first)");
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
eprintln!("aura: {e}");
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
print!("{}", render::render_chart_html(&build_chart_data(traces), mode));
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Add the argv arms**
|
|
||||||
|
|
||||||
In `main()`'s argv match, insert the two chart arms directly before the `["graph"]`
|
|
||||||
arm (`main.rs:899`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
["chart", name] => emit_chart(name, ChartMode::Overlay),
|
|
||||||
["chart", name, "--panels"] => emit_chart(name, ChartMode::Panels),
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 6: Update `USAGE`**
|
|
||||||
|
|
||||||
In the `USAGE` const (`main.rs:877–878`), add ` | aura chart <name> [--panels]` after
|
|
||||||
the `aura graph` clause so every dispatched verb is advertised:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
const USAGE: &str =
|
|
||||||
"usage: aura run [--macd] [--trace <name>] | aura run --real <SYMBOL> [--from <ms>] [--to <ms>] [--trace <name>] | aura chart <name> [--panels] | aura graph | aura sweep [--name <n>] | aura mc [--name <n>] | aura walkforward [--name <n>] | aura runs families | aura runs family <id> [rank <metric>]";
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 7: Run the e2e to verify it passes**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --test cli_run chart_emits_self_contained_html_for_a_persisted_run`
|
|
||||||
Expected: PASS — the persisted run charts to self-contained HTML carrying the equity
|
|
||||||
series + inlined uPlot; `aura chart nope` exits 2 (1 passed).
|
|
||||||
|
|
||||||
- [ ] **Step 8: Build + lint + workspace gate**
|
|
||||||
|
|
||||||
Run: `cargo build --workspace`
|
|
||||||
Expected: PASS — 0 errors.
|
|
||||||
|
|
||||||
Run: `cargo test --workspace`
|
|
||||||
Expected: PASS — Task 1 (`chart_viewer`), Task 2 (`render_chart_html_*`), Task 3
|
|
||||||
(`chart_emits_*`) green, and no regression (the iteration-1 trace tests + the
|
|
||||||
byte-for-byte default-run pins stay green).
|
|
||||||
|
|
||||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
|
||||||
Expected: PASS — no warnings. (Watch for: a `clippy::needless_range_loop` on the
|
|
||||||
`for c in 0..tap.columns.len()` loop — if flagged, it reads `tap.columns[c]` only via
|
|
||||||
the parallel `joined[..].sides[i]` index, so rewrite as `.iter().enumerate()` over
|
|
||||||
`tap.columns` keeping `c`; an unused import if a name is mis-listed.)
|
|
||||||
@@ -1,173 +0,0 @@
|
|||||||
# Fieldtest — milestone Runway (real-data ergonomics & honesty hardening) — 2026-06-18
|
|
||||||
|
|
||||||
**Status:** Draft — awaiting orchestrator triage
|
|
||||||
**Author:** fieldtester (dispatched by fieldtest skill, milestone-close gate)
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
Milestone-close fieldtest for **Runway — real-data ergonomics & honesty
|
|
||||||
hardening** (commit range `aedaa5d..HEAD`, HEAD `0d185fb`). The milestone's
|
|
||||||
promise: *a downstream consumer can run an honest real-data backtest end to
|
|
||||||
end* — open the real GER40 M1 archive via the canonical opener from
|
|
||||||
`aura-ingest` alone, run a strategy against real bars, get a PnL scaled by the
|
|
||||||
**correct per-instrument pip** (an index must not be mis-scaled as forex), be
|
|
||||||
**refused with a clear diagnostic** when the instrument has no vetted pip spec,
|
|
||||||
join multi-tap sink traces honestly on timestamp, and find the run-registry
|
|
||||||
surface coherent (the dead-end `runs list`/`rank` retired; `runs families`/
|
|
||||||
`family` live; the two-store Registry discoverable from the public docs).
|
|
||||||
|
|
||||||
Scenarios were derived **top-down from the promise**, not as a union of the
|
|
||||||
per-cycle axes. Public interface only: design ledger (`docs/design/INDEX.md`),
|
|
||||||
`docs/glossary.md`, `docs/project-layout.md`, `cargo doc` rustdoc, `git log`,
|
|
||||||
`CLAUDE.md`. No `crates/*/src` (incl. the engine's own
|
|
||||||
`crates/aura-ingest/examples/`, which sits under the forbidden code root) was
|
|
||||||
read. Built and ran HEAD from the working tree throughout: workspace built with
|
|
||||||
`cargo build --workspace`; every fixture/CLI run via `cargo run` so HEAD source
|
|
||||||
is always what executes. The two Rust fixtures path-dep the engine crates as a
|
|
||||||
real C16 project would; the CLI scenarios invoke the `aura` binary directly.
|
|
||||||
|
|
||||||
## Examples
|
|
||||||
|
|
||||||
### `fieldtests/milestone-runway/out_rw_1_pip_honesty.txt` — per-instrument pip honesty (#22)
|
|
||||||
- CLI invocations of `aura run --real` on a vetted index and on un-vetted
|
|
||||||
instruments.
|
|
||||||
- Fits the scope: this is the headline honesty fix — "symbol data carries no
|
|
||||||
pip metadata, so a real run guessed". A consumer must get the right pip on a
|
|
||||||
vetted symbol and a refusal on an un-vetted one.
|
|
||||||
- Outcome: ran. `GER40` → exit 0, manifest broker `sim-optimal(pip_size=1)`
|
|
||||||
(correct index point, not the FX `0.0001`), `total_pips=977.38`. `AAPL.US`
|
|
||||||
(un-specced) and `FOOBAR` (un-specced + no archive) → **exit 2**, stderr
|
|
||||||
diagnostic, **no stdout leak**. The refusal fires at the pip lookup *before*
|
|
||||||
data access (FOOBAR, which has no archive, gets the pip-refusal diagnostic,
|
|
||||||
not a data-not-found one). Matched expected.
|
|
||||||
|
|
||||||
### `fieldtests/milestone-runway/rw_2_open_ohlc_real.rs` — real GER40 M1 ingestion via the canonical opener (#80/#81/#92)
|
|
||||||
- A consumer opens the real GER40 archive with `aura_ingest::{default_data_server,
|
|
||||||
DEFAULT_DATA_PATH, open_ohlc}`, wires the four OHLC sources into a 4-source-role
|
|
||||||
harness (SMA-cross-on-close → Exposure → SimBroker(pip 1.0)), and runs to
|
|
||||||
completion over real bars.
|
|
||||||
- Fits the scope: it exercises the "build a real-data source from `aura-ingest`
|
|
||||||
alone" promise (#81), the canonical 4-source merge order (#92), and the
|
|
||||||
timestamp-window open (#80) in one realistic task.
|
|
||||||
- Outcome: built (after one fix — see the `Cell` constructor friction) and ran.
|
|
||||||
`open_ohlc` returned exactly 4 sources O/H/L/C, each reporting `bounds()`
|
|
||||||
without materializing; 53173 real bars streamed on all four taps; OHLC
|
|
||||||
ordering (`low<=open<=high`) held on **53173/53173** bars; close-driven equity
|
|
||||||
produced (`+314.46` pips at pip 1.0). The crate has **no `data-server`
|
|
||||||
dependency** — the #81 crutch the prior milestone-18 fixture needed is gone.
|
|
||||||
Matched expected.
|
|
||||||
|
|
||||||
### `fieldtests/milestone-runway/rw_3_multitap_ts_join.rs` — multi-tap trace honesty (#93)
|
|
||||||
- A consumer records two taps (exposure + cumulative equity) off the same real
|
|
||||||
GER40 run — the taps fire at **different cadences** — and fuses them with
|
|
||||||
`aura_engine::join_on_ts` / `JoinedRow`.
|
|
||||||
- Fits the scope: directly the #93 promise — "join multi-tap sink traces
|
|
||||||
honestly on timestamp, not by index."
|
|
||||||
- Outcome: built and ran. Exposure fired 27725×, equity 27744× (differ by 19);
|
|
||||||
`join_on_ts` produced exactly one `JoinedRow` per spine entry (27744); a
|
|
||||||
by-timestamp cross-check of every `Some`/`None` side against the exposure
|
|
||||||
stream found **0 mismatches**; the first rows are the warm-up bars where
|
|
||||||
exposure had not yet fired (`None`) — precisely the cardinality-misalignment a
|
|
||||||
positional zip would corrupt. Matched expected.
|
|
||||||
|
|
||||||
### `fieldtests/milestone-runway/out_rw_4_runs_coherence.txt` — run-registry coherence (#73/#82)
|
|
||||||
- CLI invocations confirming `runs list`/`rank` are retired and
|
|
||||||
`runs families`/`runs family … rank` work, plus a rustdoc check of the
|
|
||||||
two-store `Registry::open`.
|
|
||||||
- Fits the scope: the registry-honesty half of the promise.
|
|
||||||
- Outcome: ran. `runs list` and `runs rank` → **exit 2** with the usage line
|
|
||||||
(clean diagnostic; both absent from the usage string). `runs families` →
|
|
||||||
exit 0 listing the family; `runs family <id> rank total_pips` → exit 0 with
|
|
||||||
ranked members. The `Registry::open` rustdoc documents the two
|
|
||||||
directory-co-located stores and the per-directory-isolation consequence (#82).
|
|
||||||
Matched expected — with one edge observation (bogus family id, below).
|
|
||||||
|
|
||||||
## Findings
|
|
||||||
|
|
||||||
### [working] Per-instrument pip honesty is correct and refuses cleanly
|
|
||||||
- Example: `out_rw_1_pip_honesty.txt`.
|
|
||||||
- Vetted `GER40` emits `sim-optimal(pip_size=1)` in the manifest (index point,
|
|
||||||
not the FX `0.0001`); un-vetted `AAPL.US`/`FOOBAR` exit 2 with a stderr
|
|
||||||
diagnostic that names the symbol and the fix ("add it to the instrument
|
|
||||||
table") and **no stdout leak**. The refusal gates at the lookup before any
|
|
||||||
data access. This is the milestone's headline honesty win, working end to end.
|
|
||||||
- Recommended action: carry on.
|
|
||||||
|
|
||||||
### [working] A real-data source builds from `aura-ingest` alone — the `data-server` crutch is gone
|
|
||||||
- Example: `rw_2_open_ohlc_real.rs` + its Cargo.toml.
|
|
||||||
- `default_data_server()`, `DEFAULT_DATA_PATH`, `DataServer` (re-export), and
|
|
||||||
`open_ohlc` are all reachable from `aura-ingest`; the fixture crate carries
|
|
||||||
**no** `data-server` dependency. The prior milestone-18 fixture had to pull
|
|
||||||
`data-server` as a git dep because `M1FieldSource::open` wanted
|
|
||||||
`&Arc<DataServer>` and aura-ingest re-exported neither it nor the path. #81
|
|
||||||
closes that. `open_ohlc` vouches for the C4 O/H/L/C order so the consumer
|
|
||||||
never spells it out (#92); 53173 real bars streamed to completion.
|
|
||||||
- Recommended action: carry on.
|
|
||||||
|
|
||||||
### [working] Multi-tap traces fuse honestly on timestamp
|
|
||||||
- Example: `rw_3_multitap_ts_join.rs`.
|
|
||||||
- Two real-data taps at genuinely different cadences (Δ19 rows) join to exactly
|
|
||||||
one `JoinedRow` per spine entry, every `Some`/`None` side aligning to the
|
|
||||||
source stream **by timestamp** (0 by-ts mismatches over 27744 rows). The
|
|
||||||
Option-per-side semantics surfaced the warm-up `None` rows exactly where a
|
|
||||||
zip-by-index would have silently shifted every later row. The `Recorder`
|
|
||||||
rustdoc states the cadence rule ("join on the recorded timestamp") at the
|
|
||||||
point of use, so a consumer is steered to `join_on_ts` rather than a zip.
|
|
||||||
- Recommended action: carry on.
|
|
||||||
|
|
||||||
### [working] Run-registry surface is coherent; two-store behaviour is documented
|
|
||||||
- Example: `out_rw_4_runs_coherence.txt`.
|
|
||||||
- `runs list`/`rank` are gone from the usage string and exit 2; `runs families`/
|
|
||||||
`runs family … rank` work and emit JSON. `Registry::open`'s rustdoc documents
|
|
||||||
the two directory-co-located stores (`runs.jsonl` + the fixed-name
|
|
||||||
`families.jsonl` sibling) and the per-directory (not per-filename) isolation
|
|
||||||
consequence at the bind point (#82) — exactly the "discoverable from the
|
|
||||||
public docs" half of the promise.
|
|
||||||
- Recommended action: carry on.
|
|
||||||
|
|
||||||
### [friction] Hand-authoring a param-cell vector needs `Cell::from_i64`, not the obvious `Cell::I64`
|
|
||||||
- Example: `rw_2_open_ohlc_real.rs` (first compile attempt).
|
|
||||||
- A consumer bootstrapping a harness with literal params naturally reaches for
|
|
||||||
`Cell::I64(3)` / `Cell::F64(1.0)` (mirroring `Scalar::I64` / `Scalar::F64`,
|
|
||||||
which ARE tuple-style and used right beside it for grid values). `Cell` instead
|
|
||||||
exposes only `from_i64` / `from_f64` / `from_bool` / `from_ts` constructors, so
|
|
||||||
the first build failed with `no associated item 'I64' found for struct Cell`.
|
|
||||||
The compiler's suggestion was excellent (it listed `from_i64` etc.), so the fix
|
|
||||||
was a 5-second one — but the asymmetry with `Scalar`'s tuple variants is a real
|
|
||||||
papercut for hand-authored params (the prior fieldtests sidestepped it by only
|
|
||||||
ever feeding `Cell`s that came pre-built out of a `GridSpace`). Friction, not a
|
|
||||||
bug: the task completed and the diagnostic was clean.
|
|
||||||
- Recommended action: plan a tidy iteration — either add `Cell::I64`/`F64`/etc.
|
|
||||||
tuple-style constructors (or `From<i64>`/`From<f64>` impls) to match `Scalar`,
|
|
||||||
or document the `from_*` constructors on `Cell`'s rustdoc as the param-authoring
|
|
||||||
entry point. (Low priority; ergonomic only.)
|
|
||||||
|
|
||||||
### [spec_gap] `runs family <bogus-id>` exits 0 with empty output, not a "no such family" diagnostic
|
|
||||||
- Example: `out_rw_4_runs_coherence.txt`, final block.
|
|
||||||
- `aura runs family nope-99` (a family id that does not exist) prints nothing and
|
|
||||||
exits **0**. The retired commands exit 2 with a usage line, and a real consumer
|
|
||||||
who typos a family id would reasonably expect a non-zero exit or a "no such
|
|
||||||
family `nope-99`" message rather than silent success that looks
|
|
||||||
indistinguishable from "a real but empty family". The top-level usage string,
|
|
||||||
the ledger (C18 realization note), and the glossary do not constrain what an
|
|
||||||
unknown-but-well-formed family id should do, so I cannot tell whether exit 0 +
|
|
||||||
empty is the intended contract (treat-as-empty, mirroring how a missing store
|
|
||||||
is `Ok(vec![])`) or an oversight. Two readings are equally plausible:
|
|
||||||
(a) **treat-as-empty** — consistent with `Registry::load` treating a missing
|
|
||||||
file as an empty registry, so an unknown id is "a family with zero members";
|
|
||||||
(b) **not-found is an error** — consistent with the retired commands and with
|
|
||||||
typo-safety. I picked no behaviour to assert in a fixture; I recorded the
|
|
||||||
observation.
|
|
||||||
- Recommended action: ratify the intended behaviour into the C18 realization
|
|
||||||
note (and the CLI usage), or tighten it to a non-zero exit + diagnostic if (b).
|
|
||||||
|
|
||||||
## Recommendation summary
|
|
||||||
|
|
||||||
| Finding | Class | Action |
|
|
||||||
|---|---|---|
|
|
||||||
| Per-instrument pip honesty (vetted pip + refusal) | working | carry-on |
|
|
||||||
| Real-data source builds from `aura-ingest` alone (`open_ohlc`, re-exports) | working | carry-on |
|
|
||||||
| Multi-tap traces fuse on timestamp (`join_on_ts`) | working | carry-on |
|
|
||||||
| Run-registry coherence + two-store rustdoc | working | carry-on |
|
|
||||||
| `Cell` lacks tuple-style constructors (asymmetry with `Scalar`) | friction | plan (tidy iteration) |
|
|
||||||
| `runs family <bogus-id>` → exit 0, empty | spec_gap | ratify / tighten the design ledger |
|
|
||||||
@@ -1,432 +0,0 @@
|
|||||||
# Trace charts — persist recorder taps, serve as web charts — Design Spec
|
|
||||||
|
|
||||||
**Date:** 2026-06-18
|
|
||||||
**Status:** Draft — awaiting spec review (autonomous build; specify auto-sign gate)
|
|
||||||
**Authors:** orchestrator + Claude
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Make a backtest's recorded streams **visible**. Today a run's recorder taps
|
|
||||||
(`equity`, `exposure`, or any wired column) are drained in-memory, folded to three
|
|
||||||
summary numbers (`total_pips` / `max_drawdown` / `exposure_sign_flips`), and then
|
|
||||||
discarded — nothing persists the time series, and the only visual surface
|
|
||||||
(`aura graph`) shows graph *structure*, not run *data*.
|
|
||||||
|
|
||||||
This cycle closes the gap end-to-end: **tap data from a graph → persist it to disk
|
|
||||||
→ serve it as a chart in the browser.** It is the first cut of aura's visual face
|
|
||||||
under the web-from-disk seam ratified this session into C14/C22 (commit `3b56efb`,
|
|
||||||
issue #101): the engine writes recorded traces to disk; a static self-contained
|
|
||||||
HTML page (the `render_html` precedent) charts them; multiple feeds are overlaid on
|
|
||||||
a shared timestamp axis, or aligned in stacked panels.
|
|
||||||
|
|
||||||
In scope (first cut):
|
|
||||||
|
|
||||||
1. A **columnar trace encoding** (`ColumnarTrace`) — the SoA form of one drained
|
|
||||||
tap, on disk as JSON (C7: one column = one series).
|
|
||||||
2. A **disk trace store** under `runs/traces/<name>/`, beside the run registry
|
|
||||||
(C18, `runs/`): write a run's taps, read them back.
|
|
||||||
3. `aura run --trace <name>` — persist the sample harness's taps (additive; the
|
|
||||||
default `aura run` is byte-for-byte unchanged).
|
|
||||||
4. `aura chart <name> [--panels]` — read a persisted run's taps and emit a
|
|
||||||
self-contained HTML chart page (uPlot, vendored), feeds **overlaid** (default,
|
|
||||||
shared timestamp x via the existing `join_on_ts`) or in **stacked panels**.
|
|
||||||
|
|
||||||
Out of scope (explicit follow-on, named so the boundary is auditable): families /
|
|
||||||
comparison across multiple runs (C21), a local HTTP server, level-of-detail /
|
|
||||||
"mipmapping" downsampling, an Arrow/Parquet columnar store, and live streaming
|
|
||||||
during a run.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
Four layers, each landing where its existing siblings already live — no new crate,
|
|
||||||
no engine UI knowledge (C14 preserved):
|
|
||||||
|
|
||||||
| Layer | Crate / file | Sibling precedent it joins |
|
|
||||||
|-------|--------------|----------------------------|
|
|
||||||
| Columnar encoding (pure) | `aura-engine` / `report.rs` | `summarize`, `f64_field`, `join_on_ts` — pure post-run reductions over recorded streams |
|
|
||||||
| Disk trace store (I/O) | `aura-registry` / new `trace_store.rs` | `Registry` owns `runs/` + directory-co-located sibling stores (`families.jsonl`) |
|
|
||||||
| Chart-page assembly (string) | `aura-cli` / `render.rs` | `render_html` — self-contained HTML, vendored JS assets |
|
|
||||||
| CLI verbs + serve-time align | `aura-cli` / `main.rs` | the exhaustive argv match; `run_sample` drain→fold |
|
|
||||||
|
|
||||||
The crate-homes are dictated by the existing structure, not chosen: a columnar
|
|
||||||
encoding is the same family as `join_on_ts` (a pure reduction over
|
|
||||||
`&[(Timestamp, Vec<Scalar>)]`), the on-disk store is the same family as the
|
|
||||||
registry's directory-co-located stores, and the HTML assembly is the same family as
|
|
||||||
`render_html`. Engine purity holds: `aura-engine` does the *encoding* (struct →
|
|
||||||
JSON), never file I/O; the registry crate does the I/O; the CLI does stdout +
|
|
||||||
browser hand-off. No pixel/UI knowledge enters the engine (C14).
|
|
||||||
|
|
||||||
Determinism (C1) and causality (C2/C3) are untouched: persistence is a **post-run**
|
|
||||||
reduction over already-recorded sink output, exactly like `summarize`. `join_on_ts`
|
|
||||||
stays a post-run reduction (C3: not an in-graph join), now invoked at *serve* time
|
|
||||||
rather than only in example code.
|
|
||||||
|
|
||||||
## Concrete code shapes
|
|
||||||
|
|
||||||
### 1. The user-facing flow (the criterion's empirical evidence)
|
|
||||||
|
|
||||||
A researcher who just ran a backtest wants to *see* the equity curve and the
|
|
||||||
exposure, not read three numbers. The reach is two commands:
|
|
||||||
|
|
||||||
```console
|
|
||||||
# persist the run's taps under runs/traces/demo/ (default run output unchanged)
|
|
||||||
$ aura run --trace demo
|
|
||||||
{"manifest":{...},"metrics":{"total_pips":...,"max_drawdown":...,"exposure_sign_flips":...}}
|
|
||||||
|
|
||||||
$ ls runs/traces/demo/
|
|
||||||
index.json equity.json exposure.json
|
|
||||||
|
|
||||||
# serve them as a self-contained chart page (overlay: both feeds, shared ts axis)
|
|
||||||
$ aura chart demo > demo.html && xdg-open demo.html
|
|
||||||
|
|
||||||
# or timestamp-aligned stacked panels (one per feed, shared time axis)
|
|
||||||
$ aura chart demo --panels > demo.html
|
|
||||||
```
|
|
||||||
|
|
||||||
`equity.json` — the columnar (SoA) form of one tap, chart-ready (one column = one
|
|
||||||
series), kind-tagged for honesty (C7):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"tap": "equity",
|
|
||||||
"kinds": ["F64"],
|
|
||||||
"ts": [1, 2, 3, 4, 5, 6, 7],
|
|
||||||
"columns": [[0.0, 0.0, 0.0010, 0.0040, 0.0040, 0.0010, -0.0010]]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
`index.json` — the tap list (deterministic read order) + the run manifest, so the
|
|
||||||
page header can show run context (commit / params / window / broker), mirroring
|
|
||||||
`aura graph`'s header:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{ "manifest": { "commit": "...", "params": [...], "window": [1,7], "seed": 0,
|
|
||||||
"broker": "sim-optimal(pip_size=0.0001)" },
|
|
||||||
"taps": ["equity", "exposure"] }
|
|
||||||
```
|
|
||||||
|
|
||||||
The emitted page is self-contained (no remote fetch), injecting the aligned data as
|
|
||||||
`window.AURA_TRACES` and inlining vendored uPlot + a `chart-viewer.js` — exactly the
|
|
||||||
`render_html` pattern.
|
|
||||||
|
|
||||||
### 2. `ColumnarTrace` (aura-engine / report.rs) — NEW
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// One drained recorder tap in columnar (SoA) form: parallel arrays, one per
|
|
||||||
/// recorded column, plus the shared recorded-timestamp axis. Chart-ready (a column
|
|
||||||
/// is a series of numbers) and kind-tagged (C7: the true base type of each column
|
|
||||||
/// survives even though values are coerced to f64 for plotting). Pure: the same
|
|
||||||
/// rows always encode to the same struct (C1).
|
|
||||||
#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
|
|
||||||
pub struct ColumnarTrace {
|
|
||||||
pub tap: String,
|
|
||||||
pub kinds: Vec<String>, // "F64" | "I64" | "Bool" | "Timestamp" (no serde dep on ScalarKind)
|
|
||||||
pub ts: Vec<i64>, // recorded timestamps (Timestamp.0)
|
|
||||||
pub columns: Vec<Vec<f64>>, // columns[c][r] — coerced: f64 as-is, i64 as f64, bool 1/0, ts as f64
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ColumnarTrace {
|
|
||||||
/// Transpose a drained tap (`(ts, row)` pairs) into columns. Empty rows -> empty
|
|
||||||
/// ts + empty columns sized to `kinds`.
|
|
||||||
pub fn from_rows(tap: &str, kinds: &[ScalarKind], rows: &[(Timestamp, Vec<Scalar>)]) -> Self { /* ... */ }
|
|
||||||
|
|
||||||
/// Inverse for the serve/align path: rebuild `(ts, row)` pairs with each cell as
|
|
||||||
/// `Scalar::f64(columns[c][r])`. The on-disk store is already f64 columns, so the
|
|
||||||
/// rebuilt rows are **uniformly f64** (the original base type survives only in
|
|
||||||
/// `kinds` as display metadata) — which keeps the serve-side `as_f64` projection
|
|
||||||
/// (§6 step 3) total, never the F64-only panic path. A true value round-trip for
|
|
||||||
/// the shipped F64 taps; for other kinds the value round-trips through the f64
|
|
||||||
/// store with `kinds` carrying the base type.
|
|
||||||
pub fn to_rows(&self) -> Vec<(Timestamp, Vec<Scalar>)> { /* ... */ }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. `TraceStore` (aura-registry / trace_store.rs) — NEW
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// A per-run directory of columnar tap files under `<runs_dir>/traces/<name>/`,
|
|
||||||
/// beside the run registry's `runs.jsonl`. Directory-keyed (like the family
|
|
||||||
/// sibling store): one subdir per run name, one `<tap>.json` per tap, plus
|
|
||||||
/// `index.json` (tap order + manifest).
|
|
||||||
pub struct TraceStore { dir: PathBuf } // dir = <runs_dir>/traces
|
|
||||||
|
|
||||||
impl TraceStore {
|
|
||||||
pub fn open(runs_dir: impl AsRef<Path>) -> TraceStore { /* runs_dir/traces */ }
|
|
||||||
|
|
||||||
/// Write one run's taps + index under traces/<name>/, creating dirs as needed.
|
|
||||||
pub fn write(&self, name: &str, manifest: &RunManifest, taps: &[ColumnarTrace]) -> Result<(), TraceStoreError> { /* ... */ }
|
|
||||||
|
|
||||||
/// Read a run's index + taps back, in index order. A missing run dir is
|
|
||||||
/// `NotFound` (treat-as-absent, C18-consistent), not a panic.
|
|
||||||
pub fn read(&self, name: &str) -> Result<RunTraces, TraceStoreError> { /* ... */ }
|
|
||||||
}
|
|
||||||
|
|
||||||
pub struct RunTraces { pub manifest: RunManifest, pub taps: Vec<ColumnarTrace> }
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Chart-page assembly (aura-cli / render.rs) — NEW, mirrors `render_html`
|
|
||||||
|
|
||||||
```rust
|
|
||||||
const UPLOT_JS: &str = include_str!("../assets/uPlot.iife.min.js");
|
|
||||||
const UPLOT_CSS: &str = include_str!("../assets/uPlot.min.css");
|
|
||||||
const CHART_VIEWER_JS: &str = include_str!("../assets/chart-viewer.js"); // NEW asset
|
|
||||||
|
|
||||||
pub enum ChartMode { Overlay, Panels }
|
|
||||||
|
|
||||||
/// Self-contained chart page for one run's taps. `data` (a `ChartData`, §6) carries
|
|
||||||
/// the taps aligned on the shared union-spine x — one `Series` per (tap, column),
|
|
||||||
/// each with its own y-scale id so disparate magnitudes stay legible; `mode` selects
|
|
||||||
/// overlay vs panels over the *same* data. The page fetches nothing at view time.
|
|
||||||
pub fn render_chart_html(data: &ChartData, mode: ChartMode) -> String {
|
|
||||||
// SHELL_HEAD-style dark shell + run-context header
|
|
||||||
// + <style>UPLOT_CSS</style> + <script>UPLOT_JS</script>
|
|
||||||
// + window.AURA_TRACES = {mode, manifest, taps, xs, series:[{name,y_scale_id,points}]}
|
|
||||||
// + <script>CHART_VIEWER_JS</script> // overlay: one uPlot, per-series y-scale;
|
|
||||||
// // panels: one uPlot per series, shared xs
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. CLI wiring (aura-cli / main.rs) — before → after
|
|
||||||
|
|
||||||
`--trace <name>` is an **orthogonal modifier** that composes with **every** run form
|
|
||||||
— plain `run`, `run --macd`, and `run --real <SYM>`. All three already drain the
|
|
||||||
identical two taps (`rx_eq` / `rx_ex`) and build a `RunManifest` (`run_sample`:159,
|
|
||||||
`run_macd`:836, `run_sample_real`:238), so one shared helper serves all three; the
|
|
||||||
persist step is inserted between drain and fold (additive; runs only when the trace
|
|
||||||
name is `Some`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// before:
|
|
||||||
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
|
|
||||||
let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
|
|
||||||
let equity = f64_field(&eq_rows, 0);
|
|
||||||
let exposure = f64_field(&ex_rows, 0);
|
|
||||||
let metrics = summarize(&equity, &exposure);
|
|
||||||
// ... RunReport
|
|
||||||
|
|
||||||
// shared helper (aura-cli), called by run_sample / run_macd / run_sample_real:
|
|
||||||
fn persist_traces(name: &str, manifest: &RunManifest,
|
|
||||||
eq_rows: &[(Timestamp, Vec<Scalar>)],
|
|
||||||
ex_rows: &[(Timestamp, Vec<Scalar>)]) -> Result<(), TraceStoreError> {
|
|
||||||
let taps = vec![
|
|
||||||
ColumnarTrace::from_rows("equity", &[ScalarKind::F64], eq_rows),
|
|
||||||
ColumnarTrace::from_rows("exposure", &[ScalarKind::F64], ex_rows),
|
|
||||||
];
|
|
||||||
TraceStore::open("runs").write(name, manifest, &taps)
|
|
||||||
}
|
|
||||||
|
|
||||||
// after, in each run path (manifest hoisted before the fold; trace: Option<&str>):
|
|
||||||
let eq_rows = rx_eq.try_iter().collect::<Vec<_>>();
|
|
||||||
let ex_rows = rx_ex.try_iter().collect::<Vec<_>>();
|
|
||||||
if let Some(name) = trace { persist_traces(name, &manifest, &eq_rows, &ex_rows)?; }
|
|
||||||
let metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0));
|
|
||||||
// ... RunReport unchanged; when trace is None, stdout + behaviour are byte-for-byte as before
|
|
||||||
```
|
|
||||||
|
|
||||||
`main()` argv match. `--trace` composes with **every** run form; `run_sample` /
|
|
||||||
`run_macd` / `run_sample_real` each gain a `trace: Option<&str>` parameter — the
|
|
||||||
existing no-trace arms pass `None`, so their stdout stays byte-for-byte unchanged.
|
|
||||||
Every combination the new `USAGE` advertises has an explicit arm (no advertised form
|
|
||||||
falls through to the usage error):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// before:
|
|
||||||
["run"] => println!("{}", run_sample().to_json()),
|
|
||||||
["run", "--macd"] => println!("{}", run_macd().to_json()),
|
|
||||||
["run", "--real", rest @ ..] => match parse_real_args(rest) { Ok((sym, from, to)) => ... },
|
|
||||||
|
|
||||||
// after — existing arms pass None (unchanged stdout); new arms thread the trace name:
|
|
||||||
["run"] => println!("{}", run_sample(None).to_json()),
|
|
||||||
["run", "--macd"] => println!("{}", run_macd(None).to_json()),
|
|
||||||
["run", "--trace", name] => println!("{}", run_sample(Some(name)).to_json()),
|
|
||||||
["run", "--macd", "--trace", name] => println!("{}", run_macd(Some(name)).to_json()),
|
|
||||||
// parse_real_args -> 4-tuple (symbol, from, to, trace: Option<String>); the existing
|
|
||||||
// ["run","--real", rest @ ..] arm threads it: run_sample_real(&sym, from, to, trace.as_deref())
|
|
||||||
["chart", name] => emit_chart(name, ChartMode::Overlay),
|
|
||||||
["chart", name, "--panels"] => emit_chart(name, ChartMode::Panels),
|
|
||||||
```
|
|
||||||
|
|
||||||
`parse_real_args` gains one tail flag — `--trace <name>` — alongside the existing
|
|
||||||
`--from`/`--to` pairs (same once-only, value-required grammar), returning a fourth
|
|
||||||
`Option<String>` element; `run_sample_real` takes `trace: Option<&str>` and persists
|
|
||||||
the same two taps when it is `Some`. `emit_chart` reads the store and aligns
|
|
||||||
**all** taps as symmetric sides of a synthetic empty-payload union spine via
|
|
||||||
`join_on_ts` (the exact three-step mechanism in §6; no tap privileged, no points
|
|
||||||
dropped), producing one `ChartData` that feeds **both** modes (overlay superimposes
|
|
||||||
the series, panels stack one plot per series on that shared x), then prints the page; a
|
|
||||||
missing run name is a usage error (stderr + `exit 2`), consistent with the rest of the
|
|
||||||
CLI. `USAGE` becomes
|
|
||||||
`aura run [--macd] [--trace <name>] | aura run --real <SYMBOL> [--from <ms>] [--to <ms>] [--trace <name>] | aura chart <name> [--panels] | …`
|
|
||||||
— every advertised `[--macd] [--trace]` combination has a matching arm above, so the
|
|
||||||
usage string and the dispatch cannot disagree.
|
|
||||||
|
|
||||||
### 6. Overlay & panel chart semantics (shared timestamp x; the y-axis choice)
|
|
||||||
|
|
||||||
**Both modes are timestamp-aligned on one shared x-axis** — the **sorted union of all
|
|
||||||
taps' recorded timestamps**, aligned via `join_on_ts` with the union as the spine
|
|
||||||
(each series null-filled where its tap did not fire — uPlot renders the gap; a *union*
|
|
||||||
spine drops no tap's points). This is exactly the user's "mehrere Feeds in einem
|
|
||||||
Chart, **oder aligned nach Timestamps in Panels**": the shared timestamp x is what
|
|
||||||
makes a vertical time-cursor line up across feeds in **both** modes (issue #101,
|
|
||||||
Fork A — timestamp-aligned panels).
|
|
||||||
|
|
||||||
The two modes differ only in **render** and in the **y-axis**, which is load-bearing
|
|
||||||
whenever feeds differ in magnitude (pip-equity ~±0.004 vs exposure ±1.0 — a single
|
|
||||||
shared y-scale would flatten the equity curve to a line):
|
|
||||||
|
|
||||||
- **Overlay** (default) — **one** plot, all series superimposed, **each on its own
|
|
||||||
auto-ranged y-scale** (uPlot's multi-scale: a named scale per series, independently
|
|
||||||
min/max-ranged). The first two series carry a drawn y-axis (left, right); any
|
|
||||||
further series keep their own auto-scale, read via the legend/tooltip. Suits a few
|
|
||||||
feeds you want superimposed in one frame.
|
|
||||||
- **Panels** (`--panels`) — **one stacked uPlot per series**, each on its own y-scale,
|
|
||||||
**all sharing the common union-spine timestamp x-axis** so a vertical time-cursor
|
|
||||||
lines up across the stacked panels (the "aligned nach Timestamps in Panels" the user
|
|
||||||
asked for). The clean choice when feeds are numerous or wildly different in
|
|
||||||
magnitude.
|
|
||||||
|
|
||||||
**Serve-time alignment (both modes), the exact mechanism — no privileged tap.**
|
|
||||||
`join_on_ts(spine, sides)` keys `sides` off a caller-supplied `spine` of
|
|
||||||
`(Timestamp, Vec<Scalar>)` rows and drops any side row whose ts is absent from the
|
|
||||||
spine (report.rs:498); it does **not** itself compute a union. `emit_chart` therefore
|
|
||||||
builds the alignment in three concrete steps, putting **no tap in the privileged
|
|
||||||
`JoinedRow.spine` slot**:
|
|
||||||
|
|
||||||
1. **Union x.** Concatenate every tap's `ts` array, sort, dedup → `xs: Vec<i64>` —
|
|
||||||
the shared x-axis.
|
|
||||||
2. **Symmetric join.** Synthesize a spine of *empty-payload* rows
|
|
||||||
`xs.iter().map(|&t| (Timestamp(t), Vec::new()))`, and pass **all** taps (each
|
|
||||||
rebuilt via `ColumnarTrace::to_rows`) as the `sides` of
|
|
||||||
`join_on_ts(&spine, &all_tap_rows)`. Because the spine covers every tap's
|
|
||||||
timestamps, no side row is dropped; the synthetic spine's own empty payload is
|
|
||||||
never read. Every tap is a symmetric side — none occupies the never-gap-filled
|
|
||||||
`JoinedRow.spine`.
|
|
||||||
3. **Flatten to series.** Each tap contributes **one chart series per recorded
|
|
||||||
column** (the shipped `equity`/`exposure` taps are single-column → one series
|
|
||||||
each). Tap *i*'s column *c* maps to
|
|
||||||
`joined.iter().map(|r| r.sides[i].as_ref().map(|row| row[c].as_f64()))` — the
|
|
||||||
value where tap *i* fired at that x, `None` (a uPlot gap) otherwise. The result is
|
|
||||||
`ChartData { xs: Vec<i64>, series: Vec<Series>, manifest, taps }`, where a
|
|
||||||
`Series = { name: String, y_scale_id: String, points: Vec<Option<f64>> }` (one
|
|
||||||
per (tap, column); the `y_scale_id` is what gives overlay its per-series y-scale).
|
|
||||||
Both modes consume the *same* `ChartData`: overlay draws all `series` on one plot
|
|
||||||
over `xs`; panels draw one stacked uPlot per `series`, all over the same `xs`.
|
|
||||||
|
|
||||||
For the first-cut sample harness the two taps (`equity`, `exposure`) are
|
|
||||||
**co-cadenced** (both fire every cycle), so the union spine is exactly their shared
|
|
||||||
timestamps; the per-series y-scales are what keep the ±0.004 equity legible beside the
|
|
||||||
±1.0 exposure. The **timestamp-aligned shared x (both modes)** is the source-mandated
|
|
||||||
contract (issue #101, Fork A); the **per-series y-scale** and the **union (vs.
|
|
||||||
one-tap) spine** are documented naive-viewer defaults settled under the
|
|
||||||
autonomous-build mandate and recorded on issue #101 — not user-picked design forks,
|
|
||||||
but documented orchestrator decisions within that mandate.
|
|
||||||
|
|
||||||
## Components
|
|
||||||
|
|
||||||
- **`ColumnarTrace`** (aura-engine): transpose/round-trip of one tap. Pure.
|
|
||||||
- **`TraceStore` + `RunTraces` + `TraceStoreError`** (aura-registry): the on-disk
|
|
||||||
per-run trace directory; write + read; missing = `NotFound`.
|
|
||||||
- **`render_chart_html` + `ChartMode` + `ChartData`** (aura-cli): page assembly.
|
|
||||||
- **Vendored assets** (aura-cli/assets): `uPlot.iife.min.js`, `uPlot.min.css`
|
|
||||||
(third-party, pinned + recorded in `refresh-assets.sh` like the existing
|
|
||||||
Graphviz-WASM/pan-zoom blobs), and `chart-viewer.js` (first-party, the
|
|
||||||
`AURA_TRACES` → uPlot bootstrap).
|
|
||||||
- **CLI** (aura-cli/main.rs): the shared `persist_traces` helper threaded into
|
|
||||||
`run_sample` / `run_macd` / `run_sample_real` (each `trace: Option<&str>`),
|
|
||||||
`emit_chart`, the new argv arms, and the serve-time union-spine `join_on_ts`
|
|
||||||
alignment shared by both chart modes.
|
|
||||||
|
|
||||||
## Data flow
|
|
||||||
|
|
||||||
```
|
|
||||||
aura run --trace demo
|
|
||||||
harness.run() -> recorder taps drained -> Vec<(Timestamp, Vec<Scalar>)> per tap
|
|
||||||
| |
|
|
||||||
| ColumnarTrace::from_rows (transpose, coerce, kind-tag)
|
|
||||||
| v
|
|
||||||
| TraceStore::write -> runs/traces/demo/{index,equity,exposure}.json
|
|
||||||
v
|
|
||||||
summarize -> RunReport -> stdout (UNCHANGED)
|
|
||||||
|
|
||||||
aura chart demo [--panels]
|
|
||||||
TraceStore::read(demo) -> RunTraces { manifest, taps: [ColumnarTrace] }
|
|
||||||
| |
|
|
||||||
| synthetic empty union-spine + ALL taps as sides -> join_on_ts -> ChartData
|
|
||||||
| { xs, series:[{name,y_scale_id,points: Option<f64> per x}] } (null = gap, no drop)
|
|
||||||
| overlay: ONE plot, all series, per-series y-scale
|
|
||||||
| panels: ONE stacked plot per series, all on the shared xs (timestamp-aligned)
|
|
||||||
v
|
|
||||||
render_chart_html(ChartData, mode) -> self-contained HTML -> stdout -> browser (uPlot)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Error handling
|
|
||||||
|
|
||||||
- **Unknown run name** (`aura chart <name>` with no `runs/traces/<name>/`):
|
|
||||||
`TraceStore::read` returns `NotFound`; the CLI prints `aura: no recorded run
|
|
||||||
'<name>' under runs/traces` to stderr and `exit 2` — usage-error parity with the
|
|
||||||
rest of the CLI, never a panic. (Consistent with C18's treat-missing-as-absent.)
|
|
||||||
- **Corrupt / unreadable trace file**: `TraceStoreError::{Io, Parse{file,source}}`
|
|
||||||
surfaced like `RegistryError` (file named, propagated), `exit 2`.
|
|
||||||
- **`--trace` write failure** (I/O): propagated from the run path as a usage-level
|
|
||||||
error (stderr + `exit 2`); the run still computed its metrics, so the message
|
|
||||||
names the persistence step, not the run.
|
|
||||||
- **Empty taps** (a run that recorded nothing): `from_rows` yields empty arrays;
|
|
||||||
`render_chart_html` emits a page with an empty chart and a "no recorded samples"
|
|
||||||
note — no panic, no division-by-zero.
|
|
||||||
- **Bool/ts coercion**: documented and total (bool→1.0/0.0, ts→f64); `kinds`
|
|
||||||
preserves the real base type so the store stays honest (C7).
|
|
||||||
|
|
||||||
## Testing strategy
|
|
||||||
|
|
||||||
- **`ColumnarTrace` round-trip** (aura-engine unit): `from_rows` then `to_rows`
|
|
||||||
reproduces the row set; columnar shape asserted (ts length, column count =
|
|
||||||
kinds, per-kind coercion); empty-rows case.
|
|
||||||
- **`TraceStore` write→read** (aura-registry unit, temp dir): a written run reads
|
|
||||||
back equal (manifest + taps in index order); missing run = `NotFound`; a
|
|
||||||
corrupt tap file = `Parse{file,..}` with the file named. Temp-dir isolation
|
|
||||||
mirrors the registry's `temp_family_dir` pattern.
|
|
||||||
- **`render_chart_html` self-contained** (aura-cli unit, mirrors
|
|
||||||
`render_html_is_self_contained_and_embeds_the_model`): starts `<!doctype html>`,
|
|
||||||
closes `</html>`, no remote `<script src`, injects `window.AURA_TRACES =`, the
|
|
||||||
vendored uPlot global is present, overlay vs panels reflected in the injected
|
|
||||||
`mode`.
|
|
||||||
- **CLI e2e** (sibling of `tests/cli_run.rs`): `aura run --trace <tmp>` creates
|
|
||||||
`index.json` + per-tap files; a plain `aura run` writes nothing (default
|
|
||||||
unchanged); `aura chart <tmp>` emits HTML containing the equity series; `aura
|
|
||||||
chart <missing>` exits 2 with the diagnostic.
|
|
||||||
- **JS DOM behaviour** (node `.mjs`, mirroring `viewer_*.mjs`): load the emitted
|
|
||||||
page, assert uPlot built N series (overlay) / N panels (`--panels`) from
|
|
||||||
`AURA_TRACES`. Protects the one property a Rust test cannot: the page actually
|
|
||||||
renders the feeds.
|
|
||||||
|
|
||||||
Suggested iteration split for the planner (each independently shippable):
|
|
||||||
**iteration 1 (persist)** = `ColumnarTrace` + `TraceStore` + `aura run --trace`,
|
|
||||||
ending with traces on disk + round-trip green; **iteration 2 (serve)** = vendor
|
|
||||||
uPlot + `chart-viewer.js` + `render_chart_html` + `aura chart` + serve-time
|
|
||||||
alignment, ending with a chart in the browser.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
1. `aura run --trace demo` (and `aura run --real <SYM> --trace demo`) writes
|
|
||||||
`runs/traces/demo/{index,equity,exposure}.json` in the columnar shape above; a
|
|
||||||
plain `aura run` / `aura run --real …` *without* `--trace` is byte-for-byte
|
|
||||||
unchanged on stdout and writes no trace files.
|
|
||||||
2. Each `<tap>.json` is the columnar SoA form — `ts` + one array per recorded
|
|
||||||
column, kind-tagged — and `ColumnarTrace` round-trips (`from_rows`→`to_rows`)
|
|
||||||
the recorded row set.
|
|
||||||
3. `aura chart demo` emits a **self-contained** HTML page (no remote fetch) that
|
|
||||||
charts the persisted feeds **overlaid** on a shared timestamp x-axis (aligned via
|
|
||||||
the union-spine `join_on_ts`, no tap's points dropped) with **each series on its
|
|
||||||
own auto-ranged y-scale** (§6) so the ±0.004 equity stays legible beside the ±1.0
|
|
||||||
exposure; `aura chart demo --panels` emits one stacked panel per series, **all
|
|
||||||
timestamp-aligned on the shared union-spine x** (a time-cursor lines up across the
|
|
||||||
stacked panels — the user's "aligned nach Timestamps in Panels"), each on its own
|
|
||||||
y-scale.
|
|
||||||
4. `aura chart <unknown>` exits 2 with a stderr diagnostic naming the missing run;
|
|
||||||
no panic on missing / empty / corrupt traces.
|
|
||||||
5. The engine gains no UI/pixel knowledge (C14): `aura-engine` only encodes
|
|
||||||
(struct→JSON), the registry crate does the I/O, the CLI does the browser
|
|
||||||
hand-off. Determinism (C1) and the no-in-graph-join rule (C3) are preserved —
|
|
||||||
persistence and alignment are post-run reductions.
|
|
||||||
6. `cargo build --workspace`, `cargo test --workspace`, and
|
|
||||||
`cargo clippy --workspace --all-targets -- -D warnings` are green.
|
|
||||||
Reference in New Issue
Block a user