Cycle-close tidy for visual-world-cut-2 (decimation #108 + run-context header #102).
Architect drift review (53eeba5..HEAD): the only drift was a STALE LEDGER — the C22
section still listed #102 (run-context header) and #108 (decimation) as deferred/open
after they shipped in 476342d. Resolved in this commit:
- the #101 first-cut note no longer claims "not yet a run-context header";
- a new "served-page hardening — cut 2" amendment records #102 (ChartMeta +
buildHeader, family window = span across members) and #108 (the pure decimate
min-max transform) as landed, both CLI-side (C14);
- the "Open architectural threads" list drops the now-landed run-context header.
Substantive gates (self-verified): C14 honoured — all decimation/header/pixel logic
is in aura-cli (render.rs/main.rs/chart-viewer.js); the only engine touch in range is
the one-line #100 cell.rs doc. C1 preserved — decimate is pure, deterministic
bucketing over the sorted-deduped spine, no-op under budget, monotonic spine (pinned
by a unit test). Regression: cargo test --workspace = 446 passed / 0 failed; cargo
clippy --workspace --all-targets -D warnings clean. No separate baseline script.
Residual debt filed as #110 (idea): the decimation budget is a fixed const (no
--width flag), and the min-then-max sub-pixel ordering is a real ts-displacement in
the non-default continuous x-mode. Both are local aura-cli refinements, neither a bug.
Ephemeral spec/plan removed at cycle close (docs/specs/0062, docs/plans/0062).
Cycle 0062 tidy: drift-clean after this ledger amendment.
This commit is contained in:
+31
-11
@@ -1020,11 +1020,12 @@ C1 — pure, no live external call), and emits a static self-contained uPlot pag
|
||||
encoding lives in `aura-engine` (`ColumnarTrace`, struct→JSON only), file I/O in
|
||||
`aura-registry`, rendering in `aura-cli` (`render_chart_html` + `chart-viewer.js`).
|
||||
First cut only — overlay (per-series y-scale) / timestamp-aligned panels; the served
|
||||
page injects the chart *series* but **not yet a run-context header** (the run manifest
|
||||
is persisted on disk in `index.json` but deliberately not surfaced in this naive
|
||||
viewer — a ratified scope call, follow-up filed). Families-comparison meta-views, a
|
||||
local server, and replay-clock controls remain open (see "Open architectural
|
||||
threads").
|
||||
page injected the chart *series* but no run-context header (the run manifest is
|
||||
persisted on disk in `index.json`, deliberately not surfaced in that first cut — a
|
||||
ratified scope call). The header (#102) and serve-time decimation (#108) landed in
|
||||
cut 2 (see the served-page-hardening amendment below); the families-comparison view
|
||||
landed too (#107). A local server and replay-clock controls remain open (see "Open
|
||||
architectural threads").
|
||||
**Amendment (family-member traces, #104, d3cb5f8).** The disk-trace layout extends
|
||||
to family runs: `aura sweep|mc|walkforward --trace <name>` persists *each member* as
|
||||
a nested standalone run-dir `runs/traces/<name>/<member_key>/`, reusing
|
||||
@@ -1091,10 +1092,29 @@ both a run and a family) is unreachable. Every error path exits 2, never panics
|
||||
(C18/C10 refuse-don't-guess). Engine untouched (C9/C14): the whole change is
|
||||
`aura-registry` (the resolver + classifier + guard) + `aura-cli` (the builder +
|
||||
`emit_chart` name-kind branch + `--tap`); the single-run page is byte-unchanged when
|
||||
no `--tap` is given. **Still deferred:** the run-context header on the served page
|
||||
(#102); multi-column tap selection (`build_comparison_chart_data` projects column 0 —
|
||||
the comparison taps in scope are single-column f64; #47); and the
|
||||
orchestration-composability rebuild (sweep/MC/WFO as composable meta-level tools).
|
||||
no `--tap` is given. **Still deferred:** multi-column tap selection
|
||||
(`build_comparison_chart_data` projects column 0 — the comparison taps in scope are
|
||||
single-column f64; #47); and the orchestration-composability rebuild (sweep/MC/WFO as
|
||||
composable meta-level tools).
|
||||
|
||||
**Amendment (served-page hardening — cut 2, #102 + #108, 476342d).** The two deferred
|
||||
follow-ons the #107 amendment named are realized, both confined to `aura-cli` (engine
|
||||
+ registry untouched, C14). **Run-context header (#102):** a `ChartMeta`
|
||||
(kind/name/commit/window/broker/seed/taps, + member count for a family, + bound params
|
||||
for a single run) is built from the `RunManifest` in
|
||||
`build_chart_data`/`build_comparison_chart_data`, injected into
|
||||
`window.AURA_TRACES.meta`, and rendered by a pure `buildHeader` in `chart-viewer.js`
|
||||
(headless `.mjs`-guarded, like `buildCharts`). The family window is the **span** across
|
||||
members `(min from, max to)` — the only reading that labels a disjoint walk-forward
|
||||
family's true OOS coverage (it collapses to the shared window for sweep/MC).
|
||||
**Decimation (#108):** a pure `decimate(ChartData, buckets) -> ChartData` min-max
|
||||
transform thins the served page to ≤ ~2·`CHART_DECIMATE_BUCKETS` spine slots
|
||||
(per-bucket min+max, nulls preserved), applied in `emit_chart` on both paths — a
|
||||
multi-year M1 family now renders a few-thousand-point page instead of 100s of MB; full
|
||||
recorded data stays on disk (view-only), and the walk-forward null-fill page collapses
|
||||
for free. Deterministic (C1): pure bucketing over the sorted-deduped spine, no-op under
|
||||
budget. Deferred refinements (#110): a `--width` budget flag (the budget is a fixed
|
||||
const) and true intra-bucket min/max ordering for the non-default continuous x-mode.
|
||||
|
||||
### C23 — Graph compilation and behaviour-preserving optimisation
|
||||
**Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic,
|
||||
@@ -1166,8 +1186,8 @@ load-bearing in the flat graph.
|
||||
families-comparison **view** — `aura chart <family>` overlaying one tap across a
|
||||
family's members on a shared y-scale — shipped (#107, see the C22 amendment).
|
||||
Still open: richer comparison meta-views (sweep-surface heatmaps, cross-family /
|
||||
multi-strategy comparison, the run-context header #102, multi-column tap selection
|
||||
#47), a local server, and the run/replay clock controls.
|
||||
multi-strategy comparison, multi-column tap selection #47), a local server, and the
|
||||
run/replay clock controls.
|
||||
- **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies
|
||||
atop the atomic sim unit (C12), not yet designed.
|
||||
- **`aura new` scaffolder, the experiment-builder API, and `Aura.toml`'s
|
||||
|
||||
@@ -1,753 +0,0 @@
|
||||
# Visual World cut 2 — decimation + run-context header — Implementation Plan
|
||||
|
||||
> **Parent spec:** `docs/specs/0062-visual-world-cut-2.md`
|
||||
>
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
|
||||
> this plan. Steps use `- [ ]` checkboxes for tracking.
|
||||
|
||||
**Goal:** Make the families-comparison chart page openable on real multi-year data
|
||||
(serve-time min-max decimation, #108) and legible (a run-context header threaded
|
||||
from the manifest, #102), with the engine + registry untouched (C14).
|
||||
|
||||
**Architecture:** Three tasks on the aura-cli render path. Task 1 is the
|
||||
compile-atomic #102 plumbing: a `ChartMeta` struct + a `ChartData.meta` field, built
|
||||
from `RunManifest` in the two builders (which gain a `name` param), every call site
|
||||
threaded, and the manifest re-injected into `window.AURA_TRACES`. Task 2 is the
|
||||
#102 presentation: a `#ctx` header slot + a pure `buildHeader` in `chart-viewer.js`
|
||||
with a headless `.mjs` guard. Task 3 is #108: a pure `decimate(ChartData, buckets)`
|
||||
transform applied in `emit_chart` on both paths. Task 4 is the workspace gate.
|
||||
|
||||
**Tech Stack:** Rust (`crates/aura-cli` — `render.rs`, `main.rs`), JavaScript
|
||||
(`crates/aura-cli/assets/chart-viewer.js`), `node`-driven headless guards
|
||||
(`crates/aura-cli/tests/*.mjs` shelled by paired `*.rs`).
|
||||
|
||||
---
|
||||
|
||||
**Files this plan creates or modifies:**
|
||||
|
||||
- Modify: `crates/aura-cli/src/render.rs:128-143` — add `ChartMeta`; add `meta` field to `ChartData`; reword the `ChartData` doc.
|
||||
- Modify: `crates/aura-cli/src/render.rs:154-159` — inject `"meta"` into `render_chart_html`'s `window.AURA_TRACES`.
|
||||
- Modify: `crates/aura-cli/src/render.rs:65-72` — add the `#ctx` slot to `CHART_HEAD`.
|
||||
- Modify: `crates/aura-cli/src/render.rs:215-262` — update `sample_chart_data()`; flip the negative render test to `render_chart_html_injects_run_context`.
|
||||
- Modify: `crates/aura-cli/src/main.rs:15` — import `ChartMeta`.
|
||||
- Modify: `crates/aura-cli/src/main.rs:243-387` — `build_chart_data`/`build_comparison_chart_data` gain `name` + build `meta`; `filter_to_tap` narrows `meta.taps`; `emit_chart` threads `name` and (Task 3) applies `decimate`; add `scalar_display`, `decimate`, `CHART_DECIMATE_BUCKETS`.
|
||||
- Modify: `crates/aura-cli/src/main.rs:1642,1659,1668` — thread `name` into the 3 `build_comparison_chart_data` unit-test calls; add `decimate` unit tests.
|
||||
- Modify: `crates/aura-cli/assets/chart-viewer.js:90-151` — add pure `buildHeader`; fill `#ctx` in `mount`; export `buildHeader`.
|
||||
- Create: `crates/aura-cli/tests/chart_viewer_header.mjs` — headless guard driving `buildHeader`.
|
||||
- Create: `crates/aura-cli/tests/chart_viewer_header.rs` — `node`-shelling runner mirroring `chart_viewer.rs`.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: #102 — `ChartMeta` plumbing (compile-atomic)
|
||||
|
||||
Adds the run-context type and threads it from the manifest into the served page.
|
||||
The field add + the two signature changes break the whole crate until every call
|
||||
site is threaded, so all threading lands here and the build gate (Step 7) is the
|
||||
first compile.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-cli/src/render.rs`
|
||||
- Modify: `crates/aura-cli/src/main.rs`
|
||||
|
||||
- [ ] **Step 1: Add `ChartMeta` + the `ChartData.meta` field (render.rs)**
|
||||
|
||||
Replace the `ChartData` doc + struct at `crates/aura-cli/src/render.rs:135-143`:
|
||||
|
||||
```rust
|
||||
/// Run-context for the page header (#102): serialized into `window.AURA_TRACES.meta`
|
||||
/// and rendered client-side by `chart-viewer.js`'s pure `buildHeader`. A single run
|
||||
/// carries its manifest fields + bound params; a family carries the shared context
|
||||
/// (commit/window/broker) + member count — per-member identity is the series label
|
||||
/// (`member.key`), not repeated here.
|
||||
#[derive(serde::Serialize, Default)]
|
||||
pub struct ChartMeta {
|
||||
/// "run" | "family".
|
||||
pub kind: String,
|
||||
/// The chart name (run name or family id).
|
||||
pub name: String,
|
||||
/// Git commit of the run's frozen artifact (C18); `buildHeader` shows the short form.
|
||||
pub commit: String,
|
||||
/// Inclusive data window `(from, to)` epoch-ns.
|
||||
pub window: (i64, i64),
|
||||
/// Broker profile label, e.g. "sim-optimal(pip_size=1)".
|
||||
pub broker: String,
|
||||
/// RNG seed (0 for a seed-free synthetic run).
|
||||
pub seed: u64,
|
||||
/// The charted taps (single run: all charted taps or the `--tap` pick; family:
|
||||
/// the one compared tap).
|
||||
pub taps: Vec<String>,
|
||||
/// Family member count; `None` for a single run.
|
||||
pub members: Option<usize>,
|
||||
/// Bound params (single run only; empty for a family, where params are the
|
||||
/// per-member series labels). Stringified `name -> display` for the header.
|
||||
pub params: Vec<(String, String)>,
|
||||
}
|
||||
|
||||
/// The serve-ready chart payload: the shared timestamp axis, the per-(tap,column)
|
||||
/// series, and the run-context `meta`. These are exactly the fields `chart-viewer.js`
|
||||
/// reads; injected verbatim as `window.AURA_TRACES` (plus the `mode`). `meta` carries
|
||||
/// the manifest context for the header (#102); the full manifest stays on disk.
|
||||
pub struct ChartData {
|
||||
pub xs: Vec<i64>,
|
||||
pub series: Vec<Series>,
|
||||
pub meta: ChartMeta,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add `scalar_display` + build `meta` in `build_chart_data` (main.rs)**
|
||||
|
||||
Change the import at `crates/aura-cli/src/main.rs:15`:
|
||||
|
||||
```rust
|
||||
use render::{ChartData, ChartMeta, ChartMode, Series};
|
||||
```
|
||||
|
||||
Add this helper immediately above `build_chart_data` (before the doc at `main.rs:243`):
|
||||
|
||||
```rust
|
||||
/// Render a bound param's `Scalar` to a compact string for the chart header (#102).
|
||||
/// Local to the CLI header path: `Scalar` has no `Display` impl (C7 keeps it
|
||||
/// tag-only on the param plane), and the on-disk serde-tagged form (`{"I64":10}`)
|
||||
/// is too noisy for a one-line header.
|
||||
fn scalar_display(s: &Scalar) -> String {
|
||||
match s {
|
||||
Scalar::I64(v) => v.to_string(),
|
||||
Scalar::F64(v) => format!("{v}"),
|
||||
Scalar::Bool(v) => v.to_string(),
|
||||
Scalar::Timestamp(t) => t.0.to_string(),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Change `build_chart_data`'s signature at `main.rs:251` and its return at `main.rs:272`.
|
||||
Signature line:
|
||||
|
||||
```rust
|
||||
fn build_chart_data(name: &str, traces: RunTraces) -> ChartData {
|
||||
```
|
||||
|
||||
Return (replace `ChartData { xs, series }` at `main.rs:272`):
|
||||
|
||||
```rust
|
||||
let m = &traces.manifest;
|
||||
let meta = ChartMeta {
|
||||
kind: "run".to_string(),
|
||||
name: name.to_string(),
|
||||
commit: m.commit.clone(),
|
||||
window: (m.window.0.0, m.window.1.0),
|
||||
broker: m.broker.clone(),
|
||||
seed: m.seed,
|
||||
taps: traces.taps.iter().map(|t| t.tap.clone()).collect(),
|
||||
members: None,
|
||||
params: m.params.iter().map(|(k, v)| (k.clone(), scalar_display(v))).collect(),
|
||||
};
|
||||
ChartData { xs, series, meta }
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add `name` + family `meta` in `build_comparison_chart_data` (main.rs)**
|
||||
|
||||
Change the signature at `main.rs:285`:
|
||||
|
||||
```rust
|
||||
fn build_comparison_chart_data(
|
||||
name: &str,
|
||||
members: &[FamilyMember],
|
||||
tap: &str,
|
||||
) -> Result<ChartData, String> {
|
||||
```
|
||||
|
||||
Replace the return `Ok(ChartData { xs, series })` at `main.rs:321`:
|
||||
|
||||
```rust
|
||||
// member_rows is non-empty here (checked above) => members is non-empty, so
|
||||
// members[0] is safe; commit/window/broker are shared across a family.
|
||||
let m = &members[0].traces.manifest;
|
||||
let meta = ChartMeta {
|
||||
kind: "family".to_string(),
|
||||
name: name.to_string(),
|
||||
commit: m.commit.clone(),
|
||||
window: (m.window.0.0, m.window.1.0),
|
||||
broker: m.broker.clone(),
|
||||
seed: m.seed,
|
||||
taps: vec![tap.to_string()],
|
||||
members: Some(members.len()),
|
||||
params: Vec::new(),
|
||||
};
|
||||
Ok(ChartData { xs, series, meta })
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Narrow `meta.taps` in `filter_to_tap` (main.rs)**
|
||||
|
||||
Replace the body of `filter_to_tap` (`main.rs:327-333`):
|
||||
|
||||
```rust
|
||||
fn filter_to_tap(data: ChartData, tap: &str) -> Result<ChartData, String> {
|
||||
let series: Vec<Series> = data.series.into_iter().filter(|s| s.name == tap).collect();
|
||||
if series.is_empty() {
|
||||
return Err(format!("run has no tap named '{tap}'"));
|
||||
}
|
||||
let mut meta = data.meta;
|
||||
meta.taps = vec![tap.to_string()];
|
||||
Ok(ChartData { xs: data.xs, series, meta })
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Thread `name` into the `emit_chart` call sites (main.rs)**
|
||||
|
||||
In the Run arm, change `main.rs:350`:
|
||||
|
||||
```rust
|
||||
let mut data = build_chart_data(name, traces);
|
||||
```
|
||||
|
||||
In the Family arm, change `main.rs:370`:
|
||||
|
||||
```rust
|
||||
let data = match build_comparison_chart_data(name, &members, tap.unwrap_or("equity")) {
|
||||
```
|
||||
|
||||
(The `decimate` calls are added in Task 3; do not add them here.)
|
||||
|
||||
- [ ] **Step 6: Thread `name` into the 3 comparison unit tests + the render test helper**
|
||||
|
||||
In `main.rs`, the three `build_comparison_chart_data(&members, …)` calls become
|
||||
name-first. At `main.rs:1642`:
|
||||
|
||||
```rust
|
||||
let data = build_comparison_chart_data("fam", &members, "equity").expect("builds");
|
||||
```
|
||||
|
||||
At `main.rs:1659`:
|
||||
|
||||
```rust
|
||||
let data = build_comparison_chart_data("fam", &members, "equity").expect("builds");
|
||||
```
|
||||
|
||||
At `main.rs:1668`:
|
||||
|
||||
```rust
|
||||
assert!(build_comparison_chart_data("fam", &members, "nosuch").is_err());
|
||||
```
|
||||
|
||||
In `crates/aura-cli/src/render.rs`, replace `sample_chart_data()` at `render.rs:215-223`:
|
||||
|
||||
```rust
|
||||
fn sample_chart_data() -> ChartData {
|
||||
ChartData {
|
||||
xs: vec![1, 2, 3],
|
||||
series: vec![
|
||||
Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(0.0), Some(0.001), Some(0.004)] },
|
||||
Series { name: "exposure".into(), y_scale_id: "y_1".into(), points: vec![Some(1.0), Some(1.0), Some(-1.0)] },
|
||||
],
|
||||
meta: ChartMeta {
|
||||
kind: "run".into(),
|
||||
name: "sample".into(),
|
||||
commit: "abc1234def".into(),
|
||||
window: (1, 3),
|
||||
broker: "sim-optimal(pip_size=1)".into(),
|
||||
seed: 0,
|
||||
taps: vec!["equity".into(), "exposure".into()],
|
||||
members: None,
|
||||
params: vec![],
|
||||
},
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Build gate — the whole crate compiles again**
|
||||
|
||||
Run: `cargo build -p aura-cli --tests`
|
||||
Expected: `Finished` with 0 errors (all call sites threaded; `render_chart_html`
|
||||
does not yet inject `meta`, which is intentional for the Step 9 RED).
|
||||
|
||||
- [ ] **Step 8: Verify the OLD negative test still passes (meta not yet injected)**
|
||||
|
||||
Run: `cargo test -p aura-cli --lib render_chart_html_injects_only_what_the_viewer_reads`
|
||||
Expected: PASS (the field exists but is not injected into the page yet, so the
|
||||
`!contains("manifest")` / `!contains("taps")` assertions still hold).
|
||||
|
||||
- [ ] **Step 9: Replace the negative render test with the positive one (RED)**
|
||||
|
||||
Replace the test + its doc at `crates/aura-cli/src/render.rs:255-262`:
|
||||
|
||||
```rust
|
||||
/// `window.AURA_TRACES.meta` now carries the run-context the header reads
|
||||
/// (#102): the run's commit / window / broker / taps, threaded from the
|
||||
/// manifest (this flipped the earlier "manifest is NOT carried" pin).
|
||||
#[test]
|
||||
fn render_chart_html_injects_run_context() {
|
||||
let html = render_chart_html(&sample_chart_data(), ChartMode::Overlay);
|
||||
assert!(html.contains("\"meta\""), "run-context meta not injected");
|
||||
assert!(html.contains("abc1234def"), "manifest commit not injected");
|
||||
assert!(html.contains("sim-optimal(pip_size=1)"), "manifest broker not injected");
|
||||
assert!(html.contains("\"kind\":\"run\""), "meta kind not injected");
|
||||
}
|
||||
```
|
||||
|
||||
Run: `cargo test -p aura-cli --lib render_chart_html_injects_run_context`
|
||||
Expected: FAIL — `run-context meta not injected` (render_chart_html still injects
|
||||
only `{mode, xs, series}`).
|
||||
|
||||
- [ ] **Step 10: Inject `meta` into the page (GREEN)**
|
||||
|
||||
Replace the `serde_json::json!` block in `render_chart_html` at `render.rs:154-159`:
|
||||
|
||||
```rust
|
||||
let traces = serde_json::json!({
|
||||
"mode": mode_str,
|
||||
"xs": data.xs,
|
||||
"series": data.series,
|
||||
"meta": data.meta,
|
||||
})
|
||||
.to_string();
|
||||
```
|
||||
|
||||
Run: `cargo test -p aura-cli --lib render_chart_html_injects_run_context`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 11: Whole-crate test gate**
|
||||
|
||||
Run: `cargo test -p aura-cli`
|
||||
Expected: PASS — all lib + integration tests green (the family/single chart E2E
|
||||
tests in `cli_run.rs` tolerate the additive `"meta"` key).
|
||||
|
||||
---
|
||||
|
||||
## Task 2: #102 — run-context header (CHART_HEAD slot + `buildHeader` + guard)
|
||||
|
||||
Renders the injected `meta` as a header. The headless guard is written first (RED:
|
||||
`buildHeader` is not yet exported), then the function + DOM wiring (GREEN).
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/aura-cli/tests/chart_viewer_header.mjs`
|
||||
- Create: `crates/aura-cli/tests/chart_viewer_header.rs`
|
||||
- Modify: `crates/aura-cli/assets/chart-viewer.js`
|
||||
- Modify: `crates/aura-cli/src/render.rs:65-72`
|
||||
|
||||
- [ ] **Step 1: Write the headless guard (.mjs) driving `buildHeader`**
|
||||
|
||||
Create `crates/aura-cli/tests/chart_viewer_header.mjs`:
|
||||
|
||||
```js
|
||||
// Headless guard for the run-context header builder (chart-viewer.js buildHeader).
|
||||
// Property protected: buildHeader maps an AURA_TRACES.meta object to header chips
|
||||
// WITHOUT a DOM — a run shows name/tap/broker/window/commit chips; a family adds a
|
||||
// members chip; absent meta yields []. Loads the real viewer module and drives the
|
||||
// exported pure buildHeader; mount() (browser-only) is never called.
|
||||
import { createRequire } from "node:module";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
|
||||
const fail = (msg) => { console.error("FAIL: " + msg); process.exit(1); };
|
||||
const has = (chips, label, needle, msg) => {
|
||||
const c = chips.find((c) => c.label === label);
|
||||
if (!c) fail(msg + " (no '" + label + "' chip)");
|
||||
if (needle != null && String(c.value).indexOf(needle) < 0) fail(msg + " (got " + c.value + ")");
|
||||
};
|
||||
|
||||
// No `document` global -> chart-viewer.js's browser mount is skipped; require gives
|
||||
// only the pure export (mirrors chart_viewer_build.mjs stubbing window before require).
|
||||
global.window = {};
|
||||
const { buildHeader } = require(join(here, "..", "assets", "chart-viewer.js"));
|
||||
|
||||
// run meta -> name / tap / broker / window / commit chips; no members chip
|
||||
const run = {
|
||||
kind: "run", name: "eur-demo", commit: "abc1234def567",
|
||||
window: [1704067200000000000, 1735689600000000000], broker: "sim-optimal(pip_size=0.0001)",
|
||||
seed: 0, taps: ["equity", "exposure"], members: null, params: [["len", "20"]],
|
||||
};
|
||||
const rc = buildHeader(run);
|
||||
has(rc, "run", "eur-demo", "run name chip");
|
||||
has(rc, "tap", "equity", "tap chip");
|
||||
has(rc, "broker", "sim-optimal", "broker chip");
|
||||
has(rc, "window", "2024", "window chip (formatted day)");
|
||||
has(rc, "commit", "abc1234", "commit chip (shortened)");
|
||||
if (rc.find((c) => c.label === "members")) fail("a single run must not carry a members chip");
|
||||
|
||||
// family meta -> adds a members chip, name label is 'family'
|
||||
const fam = {
|
||||
kind: "family", name: "ger40-5y", commit: "deadbeef", window: [0, 0],
|
||||
broker: "sim-optimal(pip_size=1)", seed: 0, taps: ["equity"], members: 4, params: [],
|
||||
};
|
||||
const fc = buildHeader(fam);
|
||||
has(fc, "family", "ger40-5y", "family name chip");
|
||||
has(fc, "members", "4", "family members chip");
|
||||
|
||||
// absent meta -> []
|
||||
if (buildHeader(undefined).length !== 0) fail("absent meta must yield no chips");
|
||||
|
||||
console.log("OK — buildHeader run/family/empty chips.");
|
||||
process.exit(0);
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write the `node`-shelling runner (.rs)**
|
||||
|
||||
Create `crates/aura-cli/tests/chart_viewer_header.rs`:
|
||||
|
||||
```rust
|
||||
//! Integration guard for the `aura chart` run-context header builder.
|
||||
//!
|
||||
//! `chart-viewer.js`'s `buildHeader` maps `AURA_TRACES.meta` to header chips in
|
||||
//! JavaScript, so the property "buildHeader yields the right run/family chips"
|
||||
//! cannot be checked from Rust. This shells out to `node`, running
|
||||
//! `tests/chart_viewer_header.mjs`, which loads the real viewer module and drives
|
||||
//! the exported pure `buildHeader`. `node` is REQUIRED: if absent this test FAILS.
|
||||
|
||||
use std::path::PathBuf;
|
||||
use std::process::Command;
|
||||
|
||||
#[test]
|
||||
fn chart_viewer_builds_run_context_header() {
|
||||
let script = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||
.join("tests")
|
||||
.join("chart_viewer_header.mjs");
|
||||
assert!(script.exists(), "guard script missing at {}", script.display());
|
||||
|
||||
let out = match Command::new("node").arg(&script).output() {
|
||||
Ok(out) => out,
|
||||
Err(e) => panic!(
|
||||
"node is required for the chart-viewer header guard but could not be run \
|
||||
({e}); install Node.js or ensure `node` is on PATH"
|
||||
),
|
||||
};
|
||||
|
||||
let stdout = String::from_utf8_lossy(&out.stdout);
|
||||
let stderr = String::from_utf8_lossy(&out.stderr);
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"chart-viewer header guard failed (exit {:?}).\n--- node stdout ---\n{stdout}\n--- node stderr ---\n{stderr}",
|
||||
out.status.code()
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the guard to verify it fails (RED)**
|
||||
|
||||
Run: `cargo test -p aura-cli --test chart_viewer_header`
|
||||
Expected: FAIL — node throws because `buildHeader` is not exported by
|
||||
`chart-viewer.js` yet (`buildHeader is not a function` / destructure yields
|
||||
`undefined`).
|
||||
|
||||
- [ ] **Step 4: Add the pure `buildHeader` + export it (chart-viewer.js)**
|
||||
|
||||
Insert `buildHeader` into the IIFE immediately after `keyXRange` ends
|
||||
(`crates/aura-cli/assets/chart-viewer.js:88`, the line with the closing `}` of
|
||||
`keyXRange`), before the `mount` doc comment at line 90:
|
||||
|
||||
```js
|
||||
// {kind,name,commit,window:[from,to],broker,seed,taps,members,params} -> array of
|
||||
// {label,value} chips. PURE (no DOM), the unit the headless guard drives; mount()
|
||||
// renders the chips into the header's #ctx slot. Absent/empty meta -> [].
|
||||
function buildHeader(meta) {
|
||||
if (!meta) return [];
|
||||
function fmtDay(ns) { return new Date(ns / 1e6).toISOString().slice(0, 10); }
|
||||
var chips = [];
|
||||
chips.push({ label: meta.kind === "family" ? "family" : "run", value: meta.name });
|
||||
if (meta.members != null) chips.push({ label: "members", value: String(meta.members) });
|
||||
if (meta.taps && meta.taps.length) chips.push({ label: "tap", value: meta.taps.join(", ") });
|
||||
if (meta.broker) chips.push({ label: "broker", value: meta.broker });
|
||||
if (meta.window) chips.push({ label: "window", value: fmtDay(meta.window[0]) + "→" + fmtDay(meta.window[1]) });
|
||||
if (meta.seed) chips.push({ label: "seed", value: String(meta.seed) });
|
||||
if (meta.commit) chips.push({ label: "commit", value: String(meta.commit).slice(0, 7) });
|
||||
if (meta.params && meta.params.length)
|
||||
chips.push({ label: "params", value: meta.params.map(function (p) { return p[0] + "=" + p[1]; }).join(" ") });
|
||||
return chips;
|
||||
}
|
||||
```
|
||||
|
||||
Change the `module.exports` line at `chart-viewer.js:151`:
|
||||
|
||||
```js
|
||||
if (typeof module !== "undefined" && module.exports) module.exports = { buildCharts: buildCharts, keyXRange: keyXRange, buildHeader: buildHeader };
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Render the header in `mount()` (chart-viewer.js)**
|
||||
|
||||
Immediately after `var toggle = document.getElementById("xmode-toggle");`
|
||||
(`chart-viewer.js:97`), insert:
|
||||
|
||||
```js
|
||||
var ctx = document.getElementById("ctx");
|
||||
if (ctx) {
|
||||
ctx.innerHTML = "";
|
||||
buildHeader(traces.meta).forEach(function (c) {
|
||||
var span = document.createElement("span");
|
||||
var b = document.createElement("b");
|
||||
b.textContent = c.label + " ";
|
||||
span.appendChild(b);
|
||||
span.appendChild(document.createTextNode(c.value));
|
||||
span.style.marginRight = "12px";
|
||||
ctx.appendChild(span);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Add the `#ctx` slot to `CHART_HEAD` (render.rs)**
|
||||
|
||||
Replace `CHART_HEAD` at `crates/aura-cli/src/render.rs:65-72`:
|
||||
|
||||
```rust
|
||||
const CHART_HEAD: &str = r#"<header>
|
||||
<b>aura chart</b>
|
||||
<span class="sub">timestamp-aligned trace series</span>
|
||||
<span id="ctx" class="sub"></span>
|
||||
<button id="xmode-toggle" type="button">x: collapsed (gaps off)</button>
|
||||
</header>
|
||||
<div id="stage"></div>
|
||||
<pre id="err"></pre>
|
||||
"#;
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Run the guard to verify it passes (GREEN)**
|
||||
|
||||
Run: `cargo test -p aura-cli --test chart_viewer_header`
|
||||
Expected: PASS — `OK — buildHeader run/family/empty chips.`
|
||||
|
||||
- [ ] **Step 8: Verify the existing chart-label test still passes**
|
||||
|
||||
Run: `cargo test -p aura-cli --lib render_chart_html_wears_its_own_chart_label`
|
||||
Expected: PASS (the `#ctx` span is additive; the `aura chart` label + the
|
||||
`xmode-toggle` control are untouched).
|
||||
|
||||
---
|
||||
|
||||
## Task 3: #108 — serve-time min-max decimation
|
||||
|
||||
A pure `decimate(ChartData, buckets)` transform, applied in `emit_chart` on both
|
||||
paths. Written stub-first so the unit tests have a symbol to compile against, then
|
||||
RED on the bounds test, then GREEN.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-cli/src/main.rs`
|
||||
|
||||
- [ ] **Step 1: Add the const + an identity stub (main.rs)**
|
||||
|
||||
Add immediately above `scalar_display` (which sits above `build_chart_data` at
|
||||
`main.rs:243`):
|
||||
|
||||
```rust
|
||||
/// Default decimation budget: target horizontal buckets. ~2000 buckets ⇒ ≤ ~4000
|
||||
/// spine slots (min+max per bucket) — a few-thousand-point page regardless of the
|
||||
/// underlying multi-year M1 point count.
|
||||
const CHART_DECIMATE_BUCKETS: usize = 2000;
|
||||
|
||||
/// Serve-time min-max decimation on the aligned `ChartData` (#108). STUB — replaced
|
||||
/// in Step 4.
|
||||
fn decimate(data: ChartData, _buckets: usize) -> ChartData {
|
||||
data
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write the `decimate` unit tests (main.rs)**
|
||||
|
||||
Add to the `#[cfg(test)] mod tests` block in `main.rs` (next to the comparison
|
||||
tests, after `comparison_errors_when_no_member_has_the_tap` at `main.rs:1669`):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn decimate_bounds_the_spine_to_twice_the_bucket_count() {
|
||||
let n = 10_000usize;
|
||||
let xs: Vec<i64> = (0..n as i64).collect();
|
||||
let points: Vec<Option<f64>> = (0..n).map(|i| Some(i as f64)).collect();
|
||||
let data = ChartData {
|
||||
xs,
|
||||
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
|
||||
meta: ChartMeta::default(),
|
||||
};
|
||||
let out = decimate(data, 2000);
|
||||
assert!(out.xs.len() <= 4000, "spine not bounded: {}", out.xs.len());
|
||||
assert_eq!(out.xs.len(), out.series[0].points.len(), "xs and points must stay aligned");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn decimate_preserves_per_bucket_min_and_max() {
|
||||
// 10 points, 2 buckets -> bucket 0 = idx 0..5 (a spike), bucket 1 = idx 5..10 (a trough).
|
||||
let xs: Vec<i64> = (0..10).collect();
|
||||
let mut pv = vec![1.0_f64; 10];
|
||||
pv[3] = 999.0;
|
||||
pv[7] = -50.0;
|
||||
let points: Vec<Option<f64>> = pv.into_iter().map(Some).collect();
|
||||
let data = ChartData {
|
||||
xs,
|
||||
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
|
||||
meta: ChartMeta::default(),
|
||||
};
|
||||
let out = decimate(data, 2);
|
||||
let got = out.series[0].points.clone();
|
||||
assert!(got.contains(&Some(999.0)), "bucket max (spike) dropped: {got:?}");
|
||||
assert!(got.contains(&Some(-50.0)), "bucket min (trough) dropped: {got:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn decimate_keeps_an_all_null_bucket_null() {
|
||||
let xs: Vec<i64> = (0..10).collect();
|
||||
let mut points: Vec<Option<f64>> = (0..5).map(|i| Some(i as f64)).collect();
|
||||
points.extend(std::iter::repeat(None).take(5));
|
||||
let data = ChartData {
|
||||
xs,
|
||||
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
|
||||
meta: ChartMeta::default(),
|
||||
};
|
||||
let out = decimate(data, 2);
|
||||
assert_eq!(*out.series[0].points.last().unwrap(), None, "all-null bucket must stay null");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn decimate_is_a_noop_within_budget() {
|
||||
let data = ChartData {
|
||||
xs: vec![1, 2, 3],
|
||||
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(1.0), Some(2.0), Some(3.0)] }],
|
||||
meta: ChartMeta::default(),
|
||||
};
|
||||
let out = decimate(data, 2000);
|
||||
assert_eq!(out.xs, vec![1, 2, 3], "within-budget data must pass through unchanged");
|
||||
assert_eq!(out.series[0].points, vec![Some(1.0), Some(2.0), Some(3.0)]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn decimate_passes_meta_through_and_keeps_xs_monotonic() {
|
||||
let n = 10_000usize;
|
||||
let xs: Vec<i64> = (0..n as i64).collect();
|
||||
let points: Vec<Option<f64>> = (0..n).map(|i| Some(i as f64)).collect();
|
||||
let mut meta = ChartMeta::default();
|
||||
meta.name = "keep-me".into();
|
||||
let data = ChartData { xs, series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }], meta };
|
||||
let out = decimate(data, 2000);
|
||||
assert_eq!(out.meta.name, "keep-me", "meta must pass through decimation");
|
||||
assert!(out.xs.windows(2).all(|w| w[0] < w[1]), "decimated spine must stay strictly increasing");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run to verify the bounds test fails (RED)**
|
||||
|
||||
Run: `cargo test -p aura-cli decimate`
|
||||
Expected: FAIL — `decimate_bounds_the_spine_to_twice_the_bucket_count` (`spine not
|
||||
bounded: 10000`); the identity stub does not thin. (`decimate_is_a_noop_within_budget`
|
||||
and `decimate_passes_meta_through_*` may pass against the stub; the bounds /
|
||||
min-max / null tests fail.)
|
||||
|
||||
- [ ] **Step 4: Implement `decimate` (GREEN)**
|
||||
|
||||
Replace the stub `decimate` body:
|
||||
|
||||
```rust
|
||||
/// Serve-time min-max decimation on the aligned `ChartData` (#108). Partition the
|
||||
/// shared `xs` into at most `buckets` contiguous index ranges; per non-empty bucket
|
||||
/// emit the bucket's first (and, if it spans >1 index, last) timestamp as shared
|
||||
/// spine slots, and for each series the min then max of its non-null values in that
|
||||
/// range (sub-pixel ordering — the drawn vertical extent of the column is preserved).
|
||||
/// An all-null bucket emits null. `meta` passes through unchanged. Deterministic
|
||||
/// (C1). Full data stays on disk; only the served page is thinned. No-op when
|
||||
/// `xs.len() <= 2 * buckets`.
|
||||
fn decimate(data: ChartData, buckets: usize) -> ChartData {
|
||||
let buckets = buckets.max(1);
|
||||
let n = data.xs.len();
|
||||
if n <= 2 * buckets {
|
||||
return data;
|
||||
}
|
||||
let ChartData { xs, series, meta } = data;
|
||||
|
||||
// Bucket index bounds (lo, hi_exclusive, two_slots) + the decimated shared spine.
|
||||
// xs is sorted+deduped (strictly increasing) -> boundary timestamps are strictly
|
||||
// increasing across and within buckets, so the spine stays monotonic for uPlot.
|
||||
let mut bounds: Vec<(usize, usize, bool)> = Vec::with_capacity(buckets);
|
||||
let mut out_xs: Vec<i64> = Vec::with_capacity(2 * buckets);
|
||||
for b in 0..buckets {
|
||||
let lo = b * n / buckets;
|
||||
let hi = (b + 1) * n / buckets;
|
||||
if lo >= hi {
|
||||
continue;
|
||||
}
|
||||
let two = hi - lo > 1;
|
||||
out_xs.push(xs[lo]);
|
||||
if two {
|
||||
out_xs.push(xs[hi - 1]);
|
||||
}
|
||||
bounds.push((lo, hi, two));
|
||||
}
|
||||
|
||||
let out_series: Vec<Series> = series
|
||||
.into_iter()
|
||||
.map(|s| {
|
||||
let mut points: Vec<Option<f64>> = Vec::with_capacity(out_xs.len());
|
||||
for &(lo, hi, two) in &bounds {
|
||||
let mut mn = f64::INFINITY;
|
||||
let mut mx = f64::NEG_INFINITY;
|
||||
let mut any = false;
|
||||
for p in &s.points[lo..hi] {
|
||||
if let Some(v) = p {
|
||||
any = true;
|
||||
if *v < mn {
|
||||
mn = *v;
|
||||
}
|
||||
if *v > mx {
|
||||
mx = *v;
|
||||
}
|
||||
}
|
||||
}
|
||||
if any {
|
||||
points.push(Some(mn));
|
||||
if two {
|
||||
points.push(Some(mx));
|
||||
}
|
||||
} else {
|
||||
points.push(None);
|
||||
if two {
|
||||
points.push(None);
|
||||
}
|
||||
}
|
||||
}
|
||||
Series { name: s.name, y_scale_id: s.y_scale_id, points }
|
||||
})
|
||||
.collect();
|
||||
|
||||
ChartData { xs: out_xs, series: out_series, meta }
|
||||
}
|
||||
```
|
||||
|
||||
Run: `cargo test -p aura-cli decimate`
|
||||
Expected: PASS — all five `decimate_*` tests green.
|
||||
|
||||
- [ ] **Step 5: Apply `decimate` in `emit_chart` (both arms)**
|
||||
|
||||
In the Run arm, insert before the `print!` (after the `--tap` filter block,
|
||||
`main.rs:359`):
|
||||
|
||||
```rust
|
||||
let data = decimate(data, CHART_DECIMATE_BUCKETS);
|
||||
print!("{}", render::render_chart_html(&data, mode));
|
||||
```
|
||||
|
||||
In the Family arm, insert before the `print!` (`main.rs:377`), turning the existing
|
||||
`let data = match … {}` binding into a decimated one:
|
||||
|
||||
```rust
|
||||
let data = decimate(data, CHART_DECIMATE_BUCKETS);
|
||||
print!("{}", render::render_chart_html(&data, mode));
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Whole-crate test gate**
|
||||
|
||||
Run: `cargo test -p aura-cli`
|
||||
Expected: PASS — all tests green. The `cli_run.rs` chart E2E fixtures are small
|
||||
(`xs.len() <= 4000`), so `decimate` is a no-op and the existing page assertions
|
||||
still hold; the new `decimate_*` and `chart_viewer_header` tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Workspace verification
|
||||
|
||||
**Files:** none (verification only).
|
||||
|
||||
- [ ] **Step 1: Full workspace test**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
Expected: PASS — no regression in any crate (engine + registry untouched, C14).
|
||||
|
||||
- [ ] **Step 2: Lint gate**
|
||||
|
||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: `Finished` with 0 warnings.
|
||||
@@ -1,336 +0,0 @@
|
||||
# Visual World cut 2 — served-page hardening (decimation + run-context header) — Design Spec
|
||||
|
||||
**Date:** 2026-06-22
|
||||
**Status:** Draft — awaiting user spec review
|
||||
**Authors:** orchestrator + Claude
|
||||
|
||||
Bundles two tracker issues into one cut on the chart served page:
|
||||
- **#108** — serve-time point decimation / LOD (multi-year real-data pages reach
|
||||
100s of MB and are browser-unopenable).
|
||||
- **#102** — surface a run-context header (manifest / window / broker / taps) in
|
||||
the served page.
|
||||
|
||||
Both touch one code locus — `ChartData` / `render_chart_html` in
|
||||
`crates/aura-cli/src/render.rs`, `build_chart_data` / `build_comparison_chart_data`
|
||||
/ `emit_chart` in `crates/aura-cli/src/main.rs`, and `chart-viewer.js` — and one
|
||||
purpose under C22/C14: turn the #107 families-comparison first cut into a page a
|
||||
(usually remote) researcher can actually open and read on REAL multi-year families.
|
||||
|
||||
The derived fork decisions for this cut are recorded on the reference issue
|
||||
(Brummel/Aura #108, "Design reconciliation (specify)") and are not re-litigated
|
||||
here.
|
||||
|
||||
## Goal
|
||||
|
||||
1. **Decimation (#108).** A charted page's size must stop growing with the
|
||||
underlying point count. A 5y M1 family (1.6M points × 4 members = 148 MB today)
|
||||
must render a few-thousand-point page with its visual shape (min/max envelope)
|
||||
preserved. Full recorded data stays on disk; only the *served page* is thinned.
|
||||
2. **Run-context header (#102).** The served page must say *what it is* — name,
|
||||
broker, data window, commit, charted taps (and member count for a family) —
|
||||
instead of an anonymous set of curves. The manifest is already persisted and
|
||||
read back; today it is deliberately dropped from the page. Re-include it and
|
||||
render it client-side.
|
||||
|
||||
Non-goals: an Arrow/Parquet streaming/range-query store (the deferred "scale
|
||||
path"); a `--width`/`--max-points` CLI knob (a fixed viewport-scale default this
|
||||
cut, a clean future add); the page chrome fix (the chart page already wears its
|
||||
own `aura chart` title/header since the label split — only the run-context is
|
||||
missing). Engine + registry are untouched (C14).
|
||||
|
||||
## Architecture
|
||||
|
||||
Both halves live entirely in the CLI render path (C14: no UI / pixel knowledge in
|
||||
the engine). The pipeline in `emit_chart` becomes:
|
||||
|
||||
```
|
||||
read traces ─► build ChartData (xs + series + meta) ─► [--tap filter] ─► decimate ─► render_chart_html
|
||||
│ │
|
||||
meta from RunManifest pure ChartData->ChartData transform
|
||||
(#102, both paths) (#108, both paths), meta passed through
|
||||
```
|
||||
|
||||
- **#108** is a pure, deterministic (C1) transform `decimate(ChartData, buckets)
|
||||
-> ChartData` applied to the *already-aligned* `ChartData` (shared `xs` +
|
||||
per-series `Option<f64>`) right before rendering, on both the single-run and the
|
||||
family path. Operating after the union-join means a null-only bucket stays null,
|
||||
so the walk-forward null-fill blow-up collapses on the page for free (output is
|
||||
bounded to N × O(buckets) for sweep / MC / WFO alike). Transient generation
|
||||
memory (the full join before decimation) is unchanged and out of scope — the
|
||||
runs are "fine" per #108; only the page is the cliff.
|
||||
- **#102** adds a `meta: ChartMeta` field to `ChartData`, populated from the
|
||||
`RunManifest` (single run) or the shared family manifest (family), serialized
|
||||
into `window.AURA_TRACES.meta`, and rendered by a new pure `buildHeader(meta)`
|
||||
in `chart-viewer.js` — mirroring the pure `buildCharts` + headless `.mjs` guard
|
||||
architecture.
|
||||
|
||||
## Concrete code shapes
|
||||
|
||||
### The user-facing program (the acceptance criterion's evidence)
|
||||
|
||||
```console
|
||||
# A researcher charts a 5-year GER40 sweep family (4 members, ~1.6M M1 pts each),
|
||||
# on a remote session (charts served over python3 -m http.server):
|
||||
$ aura sweep --real GER40 …window… --trace ger40-5y # produces a 4-member family
|
||||
$ aura chart ger40-5y > page.html
|
||||
|
||||
# BEFORE this cut: page.html ≈ 148 MB — the browser stalls / cannot open it.
|
||||
# AFTER: page.html ≈ a few MB; opens instantly; the min/max envelope of
|
||||
# every member curve is preserved (no spike dropped).
|
||||
|
||||
# AND the page is no longer an anonymous set of curves — it carries a header:
|
||||
# aura chart family: ger40-5y · members 4 · tap equity ·
|
||||
# broker sim-optimal(pip_size=1) · window 2019-01-01→2023-12-31 · commit 4c64feb
|
||||
```
|
||||
|
||||
A single run reads the same way, with its bound params instead of a member count:
|
||||
|
||||
```console
|
||||
$ aura run --real EURUSD --trace eur-demo && aura chart eur-demo > page.html
|
||||
# aura chart run: eur-demo · tap equity, exposure ·
|
||||
# broker sim-optimal(pip_size=0.0001) · window 2024-01-01→2024-12-31 · commit 4c64feb
|
||||
```
|
||||
|
||||
### #102 — `ChartMeta` and `ChartData` (render.rs)
|
||||
|
||||
```rust
|
||||
// NEW: the run-context for the page header (#102). Serialized into
|
||||
// window.AURA_TRACES.meta and rendered client-side by chart-viewer.js's pure
|
||||
// buildHeader. Single run: the run's manifest fields + bound params. Family: the
|
||||
// shared context (commit/window/broker) + member count; per-member identity is the
|
||||
// series label (member.key), not repeated here.
|
||||
#[derive(serde::Serialize, Default)]
|
||||
pub struct ChartMeta {
|
||||
pub kind: String, // "run" | "family"
|
||||
pub name: String, // run name or family id
|
||||
pub commit: String, // git commit (C18); buildHeader shows the short form
|
||||
pub window: (i64, i64), // inclusive (from, to) epoch-ns
|
||||
pub broker: String, // e.g. "sim-optimal(pip_size=1)"
|
||||
pub seed: u64, // 0 for a seed-free synthetic run
|
||||
pub taps: Vec<String>, // the charted taps (single: all/--tap; family: the one compared)
|
||||
pub members: Option<usize>, // family member count; None for a single run
|
||||
pub params: Vec<(String, String)>, // single run: bound params name->display; empty for a family
|
||||
}
|
||||
|
||||
// BEFORE
|
||||
pub struct ChartData {
|
||||
pub xs: Vec<i64>,
|
||||
pub series: Vec<Series>,
|
||||
}
|
||||
// AFTER
|
||||
pub struct ChartData {
|
||||
pub xs: Vec<i64>,
|
||||
pub series: Vec<Series>,
|
||||
pub meta: ChartMeta,
|
||||
}
|
||||
```
|
||||
|
||||
`render_chart_html` injects it (the only change to the function body):
|
||||
|
||||
```rust
|
||||
// BEFORE
|
||||
let traces = serde_json::json!({ "mode": mode_str, "xs": data.xs, "series": data.series }).to_string();
|
||||
// AFTER
|
||||
let traces = serde_json::json!({ "mode": mode_str, "xs": data.xs, "series": data.series, "meta": data.meta }).to_string();
|
||||
```
|
||||
|
||||
The `ChartData` doc comment (render.rs:135-139, "The run's manifest stays on disk
|
||||
… it is not carried into the served page") is reworded: the manifest context is
|
||||
now carried as `meta` and read by `buildHeader`.
|
||||
|
||||
### #108 — `decimate` (main.rs, beside `build_chart_data`)
|
||||
|
||||
```rust
|
||||
/// Default decimation budget: target horizontal buckets. ~2000 buckets ⇒ ≤ ~4000
|
||||
/// spine slots (min+max per bucket) — a few-thousand-point page regardless of the
|
||||
/// underlying multi-year M1 point count.
|
||||
const CHART_DECIMATE_BUCKETS: usize = 2000;
|
||||
|
||||
/// Serve-time min-max decimation on the aligned `ChartData` (#108). Partition the
|
||||
/// shared `xs` into at most `buckets` contiguous index ranges; per bucket emit the
|
||||
/// bucket's first and last timestamp as two shared spine slots, and for each series
|
||||
/// the min and max of its non-null values in that range (min at the first slot, max
|
||||
/// at the second — sub-pixel ordering, so the drawn vertical extent of the column is
|
||||
/// preserved). A bucket spanning one timestamp collapses to a single slot; an
|
||||
/// all-null bucket emits null. `meta` passes through unchanged. Deterministic (C1):
|
||||
/// same input ⇒ same output. Full data stays on disk; only the served page is
|
||||
/// thinned. No-op (returns `data` unchanged) when `xs.len() <= 2 * buckets`.
|
||||
fn decimate(data: ChartData, buckets: usize) -> ChartData {
|
||||
// xs is sorted + deduped (strictly increasing) ⇒ bucket boundary timestamps are
|
||||
// strictly increasing across buckets, so the decimated spine stays monotonic
|
||||
// (uPlot requires it). Per series, fold each bucket's non-null Option<f64> to
|
||||
// (min, max); place min then max at the bucket's two boundary slots.
|
||||
// … exact bytes are the planner's.
|
||||
}
|
||||
```
|
||||
|
||||
Applied in `emit_chart` on both arms (the join is untouched):
|
||||
|
||||
```rust
|
||||
// Run arm (BEFORE): let mut data = build_chart_data(traces);
|
||||
// if let Some(t) = tap { data = filter_to_tap(data, t)?; }
|
||||
// print!("{}", render::render_chart_html(&data, mode));
|
||||
// Run arm (AFTER): let mut data = build_chart_data(name, traces); // name -> meta
|
||||
// if let Some(t) = tap { data = filter_to_tap(data, t)?; } // narrows meta.taps too
|
||||
// let data = decimate(data, CHART_DECIMATE_BUCKETS);
|
||||
// print!("{}", render::render_chart_html(&data, mode));
|
||||
|
||||
// Family arm (AFTER): let data = build_comparison_chart_data(name, &members, tap.unwrap_or("equity"))?;
|
||||
// let data = decimate(data, CHART_DECIMATE_BUCKETS);
|
||||
// print!("{}", render::render_chart_html(&data, mode));
|
||||
```
|
||||
|
||||
### #102 — meta construction (main.rs)
|
||||
|
||||
`build_chart_data` gains the run `name` and builds `meta` from `traces.manifest`
|
||||
(it currently drops the manifest); `filter_to_tap` narrows `meta.taps` to the
|
||||
picked tap; `build_comparison_chart_data` gains `name` and builds `meta` from the
|
||||
shared manifest of `members[0]` plus the member count and the compared tap.
|
||||
|
||||
```rust
|
||||
// build_chart_data (single run): manifest -> ChartMeta
|
||||
let m = &traces.manifest;
|
||||
let meta = ChartMeta {
|
||||
kind: "run".into(),
|
||||
name: name.to_string(),
|
||||
commit: m.commit.clone(),
|
||||
window: (m.window.0.0, m.window.1.0),
|
||||
broker: m.broker.clone(),
|
||||
seed: m.seed,
|
||||
taps: traces.taps.iter().map(|t| t.tap.clone()).collect(),
|
||||
members: None,
|
||||
params: m.params.iter().map(|(k, v)| (k.clone(), scalar_display(v))).collect(),
|
||||
};
|
||||
// … xs/series built as today; ChartData { xs, series, meta }
|
||||
|
||||
// build_comparison_chart_data (family): shared manifest -> ChartMeta
|
||||
let m = &members[0].traces.manifest;
|
||||
let meta = ChartMeta {
|
||||
kind: "family".into(),
|
||||
name: name.to_string(),
|
||||
commit: m.commit.clone(),
|
||||
window: (m.window.0.0, m.window.1.0),
|
||||
broker: m.broker.clone(),
|
||||
seed: m.seed,
|
||||
taps: vec![tap.to_string()],
|
||||
members: Some(members.len()),
|
||||
params: Vec::new(), // per-member params are the series labels
|
||||
};
|
||||
```
|
||||
|
||||
`scalar_display` renders a `Scalar` to a compact string for the header (an
|
||||
existing display path may be reused; the planner picks it).
|
||||
|
||||
### #102 — `buildHeader` (chart-viewer.js, pure, headless-guarded)
|
||||
|
||||
```js
|
||||
// {kind,name,commit,window:[from,to],broker,seed,taps,members,params} -> array of
|
||||
// {label, value} chips. PURE (no DOM), the unit the headless guard drives; mount()
|
||||
// renders the chips into the header's #ctx slot. Empty meta -> [].
|
||||
function buildHeader(meta) {
|
||||
if (!meta) return [];
|
||||
var fmtDay = function (ns) { return new Date(ns / 1e6).toISOString().slice(0, 10); };
|
||||
var chips = [];
|
||||
chips.push({ label: meta.kind === "family" ? "family" : "run", value: meta.name });
|
||||
if (meta.members != null) chips.push({ label: "members", value: String(meta.members) });
|
||||
if (meta.taps && meta.taps.length) chips.push({ label: "tap", value: meta.taps.join(", ") });
|
||||
if (meta.broker) chips.push({ label: "broker", value: meta.broker });
|
||||
if (meta.window) chips.push({ label: "window", value: fmtDay(meta.window[0]) + "→" + fmtDay(meta.window[1]) });
|
||||
if (meta.seed) chips.push({ label: "seed", value: String(meta.seed) });
|
||||
if (meta.commit) chips.push({ label: "commit", value: meta.commit.slice(0, 7) });
|
||||
if (meta.params && meta.params.length)
|
||||
chips.push({ label: "params", value: meta.params.map(function (p) { return p[0] + "=" + p[1]; }).join(" ") });
|
||||
return chips;
|
||||
}
|
||||
// exported beside buildCharts/keyXRange; mount() reads window.AURA_TRACES.meta and
|
||||
// fills <span id="ctx"> with the chips.
|
||||
```
|
||||
|
||||
`CHART_HEAD` (render.rs:65-72) gains a `<span id="ctx" class="sub"></span>` slot
|
||||
between the `aura chart` label and the `#xmode-toggle` button (the toggle keeps
|
||||
`margin-left:auto`, so it stays right).
|
||||
|
||||
## Components
|
||||
|
||||
| Component | File | Change |
|
||||
|---|---|---|
|
||||
| `ChartMeta` | `crates/aura-cli/src/render.rs` | NEW serializable run-context struct |
|
||||
| `ChartData` | `crates/aura-cli/src/render.rs` | + `meta: ChartMeta` field; doc reworded |
|
||||
| `render_chart_html` | `crates/aura-cli/src/render.rs` | inject `"meta"` into `window.AURA_TRACES` |
|
||||
| `CHART_HEAD` | `crates/aura-cli/src/render.rs` | + `#ctx` header slot |
|
||||
| `decimate` + `CHART_DECIMATE_BUCKETS` | `crates/aura-cli/src/main.rs` | NEW pure transform + default |
|
||||
| `build_chart_data` | `crates/aura-cli/src/main.rs` | + `name` param; build `meta` from manifest |
|
||||
| `filter_to_tap` | `crates/aura-cli/src/main.rs` | narrow `meta.taps` to the picked tap |
|
||||
| `build_comparison_chart_data` | `crates/aura-cli/src/main.rs` | + `name` param; build family `meta` |
|
||||
| `emit_chart` | `crates/aura-cli/src/main.rs` | apply `decimate` on both arms |
|
||||
| `buildHeader` + `mount` | `crates/aura-cli/assets/chart-viewer.js` | NEW pure header builder; render into `#ctx` |
|
||||
|
||||
## Data flow
|
||||
|
||||
1. `emit_chart(name, tap, mode)` classifies the name (`name_kind`) — unchanged.
|
||||
2. **Run:** `read(name)` → `build_chart_data(name, traces)` builds `xs`/`series`
|
||||
(union-join, unchanged) and `meta` from `traces.manifest`; optional
|
||||
`filter_to_tap` narrows series + `meta.taps`.
|
||||
**Family:** `read_family(name)` → `build_comparison_chart_data(name, members,
|
||||
tap)` builds one series per member on a shared y-scale (unchanged) and `meta`
|
||||
from the shared manifest + member count.
|
||||
3. `decimate(data, CHART_DECIMATE_BUCKETS)` thins `xs`+`series` (meta passes
|
||||
through). No-op when already within budget (every existing small fixture).
|
||||
4. `render_chart_html` injects `{mode, xs, series, meta}` as `window.AURA_TRACES`.
|
||||
5. In the browser, `mount()` calls `buildCharts` (unchanged) and `buildHeader`,
|
||||
filling `#ctx` with the run-context chips.
|
||||
|
||||
## Error handling
|
||||
|
||||
- `decimate` is total: any `ChartData` in, a valid `ChartData` out; empty `xs` →
|
||||
unchanged; `buckets == 0` is never passed (the const is non-zero), but a
|
||||
defensive `buckets.max(1)` keeps it total.
|
||||
- No new exit paths. The existing refuse-don't-guess errors (`NotFound`, no-such-tap
|
||||
→ stderr + exit 2) are unchanged.
|
||||
- `buildHeader(undefined)` → `[]` (a page built by an older CLI without `meta`
|
||||
still renders its charts; the header is simply empty).
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **`decimate` unit tests** (`crates/aura-cli`): (a) **bounds** — a 10 000-point
|
||||
`ChartData` decimated to 2000 buckets yields `xs.len() <= 4000`; (b) **min/max
|
||||
preserved** — a series with a single tall spike inside a bucket keeps that spike
|
||||
value in the output (max survives); (c) **nulls preserved** — an all-null bucket
|
||||
emits null; (d) **no-op under budget** — `xs.len() <= 2*buckets` returns the data
|
||||
unchanged; (e) **meta passthrough** — `meta` is identical in/out; (f)
|
||||
**monotonic** — output `xs` strictly increasing.
|
||||
- **`render.rs` render tests:** **flip** `render_chart_html_injects_only_what_the_viewer_reads`
|
||||
(asserts `!contains("manifest")` / `!contains("taps")`) → a positive
|
||||
`render_chart_html_injects_run_context` asserting the page carries `"meta"`, the
|
||||
commit, the broker, and `"kind":"run"`. Update `sample_chart_data()` to construct
|
||||
a `ChartMeta`. Keep `…is_self_contained_and_injects_traces`,
|
||||
`…wears_its_own_chart_label`, `…reflects_panels_mode`.
|
||||
- **`buildHeader` headless guard:** new `tests/chart_viewer_header.mjs` (loads the
|
||||
real `chart-viewer.js`, drives `buildHeader`) + `tests/chart_viewer_header.rs`
|
||||
(shells out to `node`, mirroring `chart_viewer.rs`). Asserts: a run meta yields
|
||||
chips for name/tap/broker/window/commit; a family meta adds a `members` chip;
|
||||
`buildHeader(undefined)` → `[]`.
|
||||
- **Existing E2E** (`tests/cli_run.rs` family + single chart tests) stays green:
|
||||
the fixtures are small, so `decimate` is a no-op, and `meta` is additive. If any
|
||||
asserts exact injected bytes, extend it to tolerate the new `meta` key.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
Against aura's feature-acceptance criterion (`CLAUDE.md`):
|
||||
|
||||
- **The audience naturally reaches for it.** A researcher charting a multi-year
|
||||
real-data family today gets a browser-unopenable 148 MB page; after this cut the
|
||||
page opens and tells them which run/window/broker they are looking at — the exact
|
||||
friction surfaced when probing the #107 view on real data.
|
||||
- **Measurably improves correctness / removes a failure.** The page size decouples
|
||||
from the point count (≤ ~4000 spine slots/series vs millions); the WFO null-fill
|
||||
page collapses; the dropped run-context is restored. The min/max envelope is
|
||||
preserved (no spike lost).
|
||||
- **Reintroduces no failure class.** Engine + registry untouched (C14: the view
|
||||
layer owns pixels, not the engine). `decimate` is a pure deterministic transform
|
||||
(C1). Full recorded data stays on disk unchanged — decimation is view-only, never
|
||||
a data mutation. No look-ahead / merge / streaming invariant is in scope (view
|
||||
layer).
|
||||
- **Concrete evidence:** the worked `aura chart ger40-5y` program above (page-size
|
||||
before/after + the header line) is the empirical check.
|
||||
```
|
||||
Reference in New Issue
Block a user