Brummel/Aura
1
0
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(chart): decimate served pages + run-context header (visual World cut 2)

Browse Source 
Hardens the families-comparison chart page (#107) so it is openable and legible on
real multi-year data. Two halves, both confined to the CLI render path (engine +
registry untouched, C14):

#108 — serve-time min-max decimation. A pure `decimate(ChartData, buckets)`
transform partitions the shared union-ts spine into <=~2000 buckets and emits each
series' per-bucket min+max (nulls preserved), bounding the served page to a few
thousand points regardless of the underlying M1 point count. Applied in emit_chart
on both the single-run and family paths; a no-op under budget (every existing
fixture). Full recorded data stays on disk; only the page is thinned. For a
walk-forward family the null-fill blow-up collapses for free (an all-null bucket
stays null). Deterministic (C1).

#102 — a run-context header. A `ChartMeta` (name/commit/window/broker/seed/taps,
+ member count for a family, + bound params for a single run) is built from the
RunManifest in build_chart_data / build_comparison_chart_data (which gain a `name`
arg), injected into window.AURA_TRACES.meta, and rendered by a new pure buildHeader
in chart-viewer.js (headless .mjs-guarded). The family window is the SPAN across
members (min from, max to), not the first member's window — the only reading that
labels a disjoint walk-forward family's true OOS coverage (it collapses to the
shared window for sweep/MC). Reuses the existing render_value for param display
(no new Scalar Display, keeping C7's tag-only param plane). The earlier
"manifest is NOT carried into the page" render pin is flipped to a positive one.

Verified: cargo test --workspace = 446 passed / 0 failed; cargo clippy --workspace
--all-targets -D warnings clean. New coverage: 5 decimate unit tests, single-run +
family meta wiring (incl. the span-window invariant), a buildHeader headless guard,
and two end-to-end cli_run tests pinning the served-page meta at the binary boundary.

closes #108
closes #102
This commit is contained in:
Brummel
2026-06-22 12:37:09 +02:00
parent 12bfa50a4f
commit 476342d7b1
6 changed files with 531 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/aura-cli/src/main.rs
+271 -11
View File
@@ -12,7 +12,7 @@
//! first real-data backtest from the CLI.
mod render;
use render::{ChartData, ChartMode, Series};
use render::{ChartData, ChartMeta, ChartMode, Series};
use aura_core::{zip_params, Cell, Firing, Scalar, ScalarKind, Timestamp};
use aura_engine::{
@@ -240,6 +240,82 @@ fn member_key(named: &[(String, Scalar)], varying: &HashSet<String>) -> String {
if key.len() <= MAX_KEY { key } else { format!("h-{:016x}", fnv1a64(key.as_bytes())) }
}
/// Default decimation budget: target horizontal buckets. ~2000 buckets ⇒ ≤ ~4000
/// spine slots (min+max per bucket) — a few-thousand-point page regardless of the
/// underlying multi-year M1 point count.
const CHART_DECIMATE_BUCKETS: usize = 2000;
/// Serve-time min-max decimation on the aligned `ChartData` (#108). Partition the
/// shared `xs` into at most `buckets` contiguous index ranges; per non-empty bucket
/// emit the bucket's first (and, if it spans >1 index, last) timestamp as shared
/// spine slots, and for each series the min then max of its non-null values in that
/// range (sub-pixel ordering — the drawn vertical extent of the column is preserved).
/// An all-null bucket emits null. `meta` passes through unchanged. Deterministic
/// (C1). Full data stays on disk; only the served page is thinned. No-op when
/// `xs.len() <= 2 * buckets`.
fn decimate(data: ChartData, buckets: usize) -> ChartData {
let buckets = buckets.max(1);
let n = data.xs.len();
if n <= 2 * buckets {
return data;
}
let ChartData { xs, series, meta } = data;
// Bucket index bounds (lo, hi_exclusive, two_slots) + the decimated shared spine.
// xs is sorted+deduped (strictly increasing) -> boundary timestamps are strictly
// increasing across and within buckets, so the spine stays monotonic for uPlot.
let mut bounds: Vec<(usize, usize, bool)> = Vec::with_capacity(buckets);
let mut out_xs: Vec<i64> = Vec::with_capacity(2 * buckets);
for b in 0..buckets {
let lo = b * n / buckets;
let hi = (b + 1) * n / buckets;
if lo >= hi {
continue;
}
let two = hi - lo > 1;
out_xs.push(xs[lo]);
if two {
out_xs.push(xs[hi - 1]);
}
bounds.push((lo, hi, two));
}
let out_series: Vec<Series> = series
.into_iter()
.map(|s| {
let mut points: Vec<Option<f64>> = Vec::with_capacity(out_xs.len());
for &(lo, hi, two) in &bounds {
let mut mn = f64::INFINITY;
let mut mx = f64::NEG_INFINITY;
let mut any = false;
for v in s.points[lo..hi].iter().flatten() {
any = true;
if *v < mn {
mn = *v;
}
if *v > mx {
mx = *v;
}
}
if any {
points.push(Some(mn));
if two {
points.push(Some(mx));
}
} else {
points.push(None);
if two {
points.push(None);
}
}
}
Series { name: s.name, y_scale_id: s.y_scale_id, points }
})
.collect();
ChartData { xs: out_xs, series: out_series, meta }
}
/// Build the serve-ready `ChartData` from a run's read-back traces by the spec-§6
/// 3-step union-spine alignment — no tap privileged, no point dropped:
/// (1) xs = the sorted, deduped union of every tap's timestamps;
@@ -248,7 +324,7 @@ fn member_key(named: &[(String, Scalar)], varying: &HashSet<String>) -> String {
/// of `join_on_ts`, so no side row is dropped and none occupies the privileged
/// `JoinedRow.spine`;
/// (3) flatten each (tap, column) to a `Series` of `Option<f64>` over xs.
fn build_chart_data(traces: RunTraces) -> ChartData {
fn build_chart_data(name: &str, traces: RunTraces) -> ChartData {
let mut xs: Vec<i64> = traces.taps.iter().flat_map(|t| t.ts.iter().copied()).collect();
xs.sort_unstable();
xs.dedup();
@@ -269,7 +345,19 @@ fn build_chart_data(traces: RunTraces) -> ChartData {
}
}
ChartData { xs, series }
let m = &traces.manifest;
let meta = ChartMeta {
kind: "run".to_string(),
name: name.to_string(),
commit: m.commit.clone(),
window: (m.window.0.0, m.window.1.0),
broker: m.broker.clone(),
seed: m.seed,
taps: traces.taps.iter().map(|t| t.tap.clone()).collect(),
members: None,
params: m.params.iter().map(|(k, v)| (k.clone(), render_value(v))).collect(),
};
ChartData { xs, series, meta }
}
/// One member's contribution to the comparison build: its key (the future series
@@ -283,6 +371,7 @@ type MemberRows = (String, Vec<(Timestamp, Vec<Scalar>)>);
/// taps). Aligned on the union-ts spine via the same `join_on_ts` build_chart_data
/// uses. `Err` if NO member carries `tap` (refuse-don't-guess).
fn build_comparison_chart_data(
name: &str,
members: &[FamilyMember],
tap: &str,
) -> Result<ChartData, String> {
@@ -318,7 +407,30 @@ fn build_comparison_chart_data(
joined.iter().map(|r| r.sides[i].as_ref().map(|row| row[0].as_f64())).collect();
series.push(Series { name: key.clone(), y_scale_id: y_scale_id.clone(), points });
}
Ok(ChartData { xs, series })
// member_rows is non-empty here (checked above) => members is non-empty, so
// members[0] is safe. commit/broker ARE shared across a family (one frozen
// artifact, one broker profile), but the window is NOT: a walk-forward family's
// members are disjoint OOS windows (commit 4c64feb), so the family window is the
// SPAN across all members — (min from, max to). For sweep/MC, whose members
// share one window, the span collapses to that shared window, so this is the one
// correct reading for all three kinds.
let m = &members[0].traces.manifest;
let window = (
members.iter().map(|fm| fm.traces.manifest.window.0.0).min().unwrap(),
members.iter().map(|fm| fm.traces.manifest.window.1.0).max().unwrap(),
);
let meta = ChartMeta {
kind: "family".to_string(),
name: name.to_string(),
commit: m.commit.clone(),
window,
broker: m.broker.clone(),
seed: m.seed,
taps: vec![tap.to_string()],
members: Some(members.len()),
params: Vec::new(),
};
Ok(ChartData { xs, series, meta })
}
/// Restrict a single-run `ChartData` to the one series named `tap`. `Err` if the
@@ -329,7 +441,9 @@ fn filter_to_tap(data: ChartData, tap: &str) -> Result<ChartData, String> {
if series.is_empty() {
return Err(format!("run has no tap named '{tap}'"));
}
Ok(ChartData { xs: data.xs, series })
let mut meta = data.meta;
meta.taps = vec![tap.to_string()];
Ok(ChartData { xs: data.xs, series, meta })
}
/// `aura chart <name> [--tap <t>] [--panels]`: classify the name and render. A
@@ -347,7 +461,7 @@ fn emit_chart(name: &str, tap: Option<&str>, mode: ChartMode) {
std::process::exit(2);
}
};
let mut data = build_chart_data(traces);
let mut data = build_chart_data(name, traces);
if let Some(t) = tap {
data = match filter_to_tap(data, t) {
Ok(d) => d,
@@ -357,6 +471,7 @@ fn emit_chart(name: &str, tap: Option<&str>, mode: ChartMode) {
}
};
}
let data = decimate(data, CHART_DECIMATE_BUCKETS);
print!("{}", render::render_chart_html(&data, mode));
}
NameKind::Family => {
@@ -367,13 +482,14 @@ fn emit_chart(name: &str, tap: Option<&str>, mode: ChartMode) {
std::process::exit(2);
}
};
let data = match build_comparison_chart_data(&members, tap.unwrap_or("equity")) {
let data = match build_comparison_chart_data(name, &members, tap.unwrap_or("equity")) {
Ok(d) => d,
Err(e) => {
eprintln!("aura: {e}");
std::process::exit(2);
}
};
let data = decimate(data, CHART_DECIMATE_BUCKETS);
print!("{}", render::render_chart_html(&data, mode));
}
NameKind::NotFound => {
@@ -1621,13 +1737,25 @@ mod tests {
use super::*;
fn cmp_member(key: &str, ts: &[i64], vals: &[f64]) -> FamilyMember {
cmp_member_win(key, ts, vals, (0, 0))
}
/// Like [`cmp_member`] but with an explicit manifest `window` so a test can
/// model walk-forward members (disjoint per-member OOS windows) and assert the
/// family window spans them.
fn cmp_member_win(key: &str, ts: &[i64], vals: &[f64], window: (i64, i64)) -> FamilyMember {
let rows: Vec<(Timestamp, Vec<Scalar>)> =
ts.iter().zip(vals).map(|(&t, &v)| (Timestamp(t), vec![Scalar::f64(v)])).collect();
let tap = ColumnarTrace::from_rows("equity", &[ScalarKind::F64], &rows);
FamilyMember {
key: key.to_string(),
traces: RunTraces {
manifest: sim_optimal_manifest(vec![], (Timestamp(0), Timestamp(0)), 0, 1.0),
manifest: sim_optimal_manifest(
vec![],
(Timestamp(window.0), Timestamp(window.1)),
0,
1.0,
),
taps: vec![tap],
},
}
@@ -1639,7 +1767,7 @@ mod tests {
cmp_member("a", &[1, 2, 3], &[10.0, 11.0, 12.0]),
cmp_member("b", &[1, 2, 3], &[20.0, 21.0, 22.0]),
];
let data = build_comparison_chart_data(&members, "equity").expect("builds");
let data = build_comparison_chart_data("fam", &members, "equity").expect("builds");
assert_eq!(data.xs, vec![1, 2, 3]);
assert_eq!(data.series.len(), 2);
assert_eq!(data.series[0].name, "a");
@@ -1648,6 +1776,13 @@ mod tests {
assert_eq!(data.series[0].y_scale_id, data.series[1].y_scale_id);
// shared ts -> dense, no nulls.
assert!(data.series[0].points.iter().all(Option::is_some));
// #102 meta wiring: a family carries kind/name/member-count + the one
// compared tap, and never the per-member params (those are the labels).
assert_eq!(data.meta.kind, "family");
assert_eq!(data.meta.name, "fam");
assert_eq!(data.meta.members, Some(2));
assert_eq!(data.meta.taps, vec!["equity".to_string()]);
assert!(data.meta.params.is_empty(), "family meta must not repeat per-member params");
}
#[test]
@@ -1656,16 +1791,141 @@ mod tests {
cmp_member("oos1", &[1, 2], &[10.0, 11.0]),
cmp_member("oos2", &[3, 4], &[20.0, 21.0]),
];
let data = build_comparison_chart_data(&members, "equity").expect("builds");
let data = build_comparison_chart_data("fam", &members, "equity").expect("builds");
assert_eq!(data.xs, vec![1, 2, 3, 4]);
assert_eq!(data.series[0].points, vec![Some(10.0), Some(11.0), None, None]);
assert_eq!(data.series[1].points, vec![None, None, Some(20.0), Some(21.0)]);
}
/// #102 family-window semantics: the header's `window` for a family is the
/// SPAN across all members — `(min member.from, max member.to)` — not the first
/// member's window. The distinction is load-bearing for a walk-forward family,
/// whose members are DISJOINT OOS windows (commit 4c64feb): labelling such a
/// family with `members[0]`'s window mislabels the family's true coverage. The
/// span reading is correct for all three kinds (sweep/MC members share a window,
/// so their span collapses to that shared window).
#[test]
fn comparison_window_spans_disjoint_walk_forward_members() {
let members = vec![
cmp_member_win("oos1", &[10, 20], &[1.0, 2.0], (10, 20)),
cmp_member_win("oos2", &[30, 40], &[3.0, 4.0], (30, 40)),
cmp_member_win("oos3", &[50, 60], &[5.0, 6.0], (50, 60)),
];
let data = build_comparison_chart_data("wf", &members, "equity").expect("builds");
// SPAN of all OOS windows (10..60), NOT members[0]'s window (10..20).
assert_eq!(data.meta.window, (10, 60));
}
#[test]
fn comparison_errors_when_no_member_has_the_tap() {
let members = vec![cmp_member("a", &[1], &[1.0])];
assert!(build_comparison_chart_data(&members, "nosuch").is_err());
assert!(build_comparison_chart_data("fam", &members, "nosuch").is_err());
}
#[test]
fn decimate_bounds_the_spine_to_twice_the_bucket_count() {
let n = 10_000usize;
let xs: Vec<i64> = (0..n as i64).collect();
let points: Vec<Option<f64>> = (0..n).map(|i| Some(i as f64)).collect();
let data = ChartData {
xs,
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2000);
assert!(out.xs.len() <= 4000, "spine not bounded: {}", out.xs.len());
assert_eq!(out.xs.len(), out.series[0].points.len(), "xs and points must stay aligned");
}
#[test]
fn decimate_preserves_per_bucket_min_and_max() {
// 10 points, 2 buckets -> bucket 0 = idx 0..5 (a spike), bucket 1 = idx 5..10 (a trough).
let xs: Vec<i64> = (0..10).collect();
let mut pv = vec![1.0_f64; 10];
pv[3] = 999.0;
pv[7] = -50.0;
let points: Vec<Option<f64>> = pv.into_iter().map(Some).collect();
let data = ChartData {
xs,
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2);
let got = out.series[0].points.clone();
assert!(got.contains(&Some(999.0)), "bucket max (spike) dropped: {got:?}");
assert!(got.contains(&Some(-50.0)), "bucket min (trough) dropped: {got:?}");
}
#[test]
fn decimate_keeps_an_all_null_bucket_null() {
let xs: Vec<i64> = (0..10).collect();
let mut points: Vec<Option<f64>> = (0..5).map(|i| Some(i as f64)).collect();
points.extend(std::iter::repeat_n(None, 5));
let data = ChartData {
xs,
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2);
assert_eq!(*out.series[0].points.last().unwrap(), None, "all-null bucket must stay null");
}
#[test]
fn decimate_is_a_noop_within_budget() {
let data = ChartData {
xs: vec![1, 2, 3],
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(1.0), Some(2.0), Some(3.0)] }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2000);
assert_eq!(out.xs, vec![1, 2, 3], "within-budget data must pass through unchanged");
assert_eq!(out.series[0].points, vec![Some(1.0), Some(2.0), Some(3.0)]);
}
#[test]
fn decimate_passes_meta_through_and_keeps_xs_monotonic() {
let n = 10_000usize;
let xs: Vec<i64> = (0..n as i64).collect();
let points: Vec<Option<f64>> = (0..n).map(|i| Some(i as f64)).collect();
let meta = ChartMeta { name: "keep-me".into(), ..Default::default() };
let data = ChartData { xs, series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }], meta };
let out = decimate(data, 2000);
assert_eq!(out.meta.name, "keep-me", "meta must pass through decimation");
assert!(out.xs.windows(2).all(|w| w[0] < w[1]), "decimated spine must stay strictly increasing");
}
/// #102 single-run meta wiring: `build_chart_data` maps the `RunManifest` into
/// `ChartData.meta` — kind "run", the name arg, the manifest window/broker, the
/// charted taps, and the bound params stringified (each typed `Scalar` rendered
/// via `render_value`, preserving its lexical form: `i64` decimal, `f64`
/// shortest round-trip). A single run carries no member count.
#[test]
fn build_chart_data_threads_run_manifest_into_meta() {
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> =
[1i64, 2, 3].iter().map(|&t| (Timestamp(t), vec![Scalar::f64(t as f64)])).collect();
let traces = RunTraces {
manifest: sim_optimal_manifest(
vec![("len".into(), Scalar::i64(10)), ("scale".into(), Scalar::f64(0.5))],
(Timestamp(1), Timestamp(3)),
7,
1.0,
),
taps: vec![ColumnarTrace::from_rows("equity", &[ScalarKind::F64], &eq_rows)],
};
let data = build_chart_data("demo", traces);
let meta = &data.meta;
assert_eq!(meta.kind, "run");
assert_eq!(meta.name, "demo");
assert_eq!(meta.window, (1, 3));
assert_eq!(meta.broker, "sim-optimal(pip_size=1)");
assert_eq!(meta.seed, 7);
assert_eq!(meta.taps, vec!["equity".to_string()]);
assert_eq!(meta.members, None);
// params stringified via render_value: typed Scalars keep their lexical form.
assert_eq!(
meta.params,
vec![("len".to_string(), "10".to_string()), ("scale".to_string(), "0.5".to_string())]
);
}
/// #99: a sweep/walk-forward family-member stdout line embeds the `RunReport` in
crates/aura-cli/src/render.rs
+61 -10
View File
@@ -65,6 +65,7 @@ const GRAPH_HEAD: &str = r#"<header>
const CHART_HEAD: &str = r#"<header>
<b>aura chart</b>
<span class="sub">timestamp-aligned trace series</span>
<span id="ctx" class="sub"></span>
<button id="xmode-toggle" type="button">x: collapsed (gaps off)</button>
</header>
<div id="stage"></div>
@@ -132,14 +133,49 @@ pub struct Series {
pub points: Vec<Option<f64>>,
}
/// The serve-ready chart payload: the shared timestamp axis and the per-(tap,column)
/// series. These are exactly the fields `chart-viewer.js` reads; injected verbatim
/// as `window.AURA_TRACES` (plus the `mode`). The run's manifest stays on disk in the
/// trace store's `index.json`; it is not carried into the served page, which renders
/// only the series, so nothing here is dead injected payload.
/// Run-context for the page header (#102): serialized into `window.AURA_TRACES.meta`
/// and rendered client-side by `chart-viewer.js`'s pure `buildHeader`. A single run
/// carries its manifest fields + bound params; a family carries its shared identity
/// (commit/broker — one frozen artifact, one broker profile) plus the SPANNING
/// window across its members (see the `window` field) + member count — per-member
/// identity is the series label (`member.key`), not repeated here.
#[derive(Default, serde::Serialize)]
pub struct ChartMeta {
/// "run" | "family".
pub kind: String,
/// The chart name (run name or family id).
pub name: String,
/// Git commit of the run's frozen artifact (C18); `buildHeader` shows the short form.
pub commit: String,
/// Inclusive data window `(from, to)` epoch-ns. For a single run, the run's
/// manifest window. For a family, the SPAN across all members —
/// `(min member.from, max member.to)` — NOT the first member's window: a
/// walk-forward family's members are disjoint OOS windows, so only the span
/// labels the family's true coverage (and for sweep/MC, whose members share a
/// window, the span collapses to it).
pub window: (i64, i64),
/// Broker profile label, e.g. "sim-optimal(pip_size=1)".
pub broker: String,
/// RNG seed (0 for a seed-free synthetic run).
pub seed: u64,
/// The charted taps (single run: all charted taps or the `--tap` pick; family:
/// the one compared tap).
pub taps: Vec<String>,
/// Family member count; `None` for a single run.
pub members: Option<usize>,
/// Bound params (single run only; empty for a family, where params are the
/// per-member series labels). Stringified `name -> display` for the header.
pub params: Vec<(String, String)>,
}
/// The serve-ready chart payload: the shared timestamp axis, the per-(tap,column)
/// series, and the run-context `meta`. These are exactly the fields `chart-viewer.js`
/// reads; injected verbatim as `window.AURA_TRACES` (plus the `mode`). `meta` carries
/// the manifest context for the header (#102); the full manifest stays on disk.
pub struct ChartData {
pub xs: Vec<i64>,
pub series: Vec<Series>,
pub meta: ChartMeta,
}
/// Assemble the self-contained uPlot chart page for one run's aligned traces.
@@ -155,6 +191,7 @@ pub fn render_chart_html(data: &ChartData, mode: ChartMode) -> String {
"mode": mode_str,
"xs": data.xs,
"series": data.series,
"meta": data.meta,
})
.to_string();
@@ -219,6 +256,17 @@ mod tests {
Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(0.0), Some(0.001), Some(0.004)] },
Series { name: "exposure".into(), y_scale_id: "y_1".into(), points: vec![Some(1.0), Some(1.0), Some(-1.0)] },
],
meta: ChartMeta {
kind: "run".into(),
name: "sample".into(),
commit: "abc1234def".into(),
window: (1, 3),
broker: "sim-optimal(pip_size=1)".into(),
seed: 0,
taps: vec!["equity".into(), "exposure".into()],
members: None,
params: vec![],
},
}
}
@@ -252,13 +300,16 @@ mod tests {
assert!(!html.contains("hover · [+] expand · body drill"), "graph breadcrumb leaked onto the chart page");
}
/// `window.AURA_TRACES` carries only the fields `chart-viewer.js` reads
/// (`mode`, `xs`, `series`) — no `manifest`/`taps` injected-but-unread payload.
/// `window.AURA_TRACES.meta` now carries the run-context the header reads
/// (#102): the run's commit / window / broker / taps, threaded from the
/// manifest (this flipped the earlier "manifest is NOT carried" pin).
#[test]
fn render_chart_html_injects_only_what_the_viewer_reads() {
fn render_chart_html_injects_run_context() {
let html = render_chart_html(&sample_chart_data(), ChartMode::Overlay);
assert!(!html.contains("\"manifest\""), "unread manifest carried into the served chart");
assert!(!html.contains("\"taps\""), "unread taps list carried into the served chart");
assert!(html.contains("\"meta\""), "run-context meta not injected");
assert!(html.contains("abc1234def"), "manifest commit not injected");
assert!(html.contains("sim-optimal(pip_size=1)"), "manifest broker not injected");
assert!(html.contains("\"kind\":\"run\""), "meta kind not injected");
}
#[test]