fix(chart): mean-decimate the bounded exposure series instead of a min/max band

The serve-time decimation (#108) reduced every series by per-bucket min/max. That is
the right envelope for a cumulative equity curve but wrong for the bounded exposure
stream (C10, f64 ∈ [-1,+1]): over a multi-year window each ~hundreds-of-bars bucket
straddles sign flips, so min/max is ±1 in nearly every bucket and the exposure
collapses to a solid -1..+1 band that reads as per-point oscillation — even for a
calm strategy (a 50/200 member flips only 0.81% of bars yet still bands out).

Make decimation tap-aware (RED-first, #111): a new `ReduceKind {MinMax, Mean}` on
each Series, set by the chart builders via `reduce_for_tap` (exposure -> Mean, else
MinMax). `decimate` honours it — a Mean series emits the per-bucket mean (its
net/duty-cycle level) in both spine slots, a MinMax series keeps min/max. The shared
spine, the equity rendering, and the page payload are unchanged (reduce is
server-side only, `#[serde(skip)]`). Verified on real GER40 1y M1: the served
exposure series goes from a ±1 band to a smooth net-level line in [-0.38, +0.34].

Rendering the min/max envelope honestly (range bars / OHLC, vs today's polyline) is
the deferred other half — filed as #112.

Verified: cargo test --workspace = 447 passed / 0 failed (incl. the RED-then-GREEN
decimate_mean_reduces_a_bipolar_series_to_its_bucket_level); clippy -D warnings clean.

closes #111
refs #112
This commit is contained in:
2026-06-22 14:34:21 +02:00
parent e01bdc3d5e
commit 2957561c30
2 changed files with 117 additions and 41 deletions
+91 -37
View File
@@ -12,7 +12,7 @@
//! first real-data backtest from the CLI.
mod render;
use render::{ChartData, ChartMeta, ChartMode, Series};
use render::{ChartData, ChartMeta, ChartMode, ReduceKind, Series};
use aura_core::{zip_params, Cell, Firing, Scalar, ScalarKind, Timestamp};
use aura_engine::{
@@ -245,14 +245,29 @@ fn member_key(named: &[(String, Scalar)], varying: &HashSet<String>) -> String {
/// underlying multi-year M1 point count.
const CHART_DECIMATE_BUCKETS: usize = 2000;
/// Serve-time min-max decimation on the aligned `ChartData` (#108). Partition the
/// shared `xs` into at most `buckets` contiguous index ranges; per non-empty bucket
/// emit the bucket's first (and, if it spans >1 index, last) timestamp as shared
/// spine slots, and for each series the min then max of its non-null values in that
/// range (sub-pixel ordering — the drawn vertical extent of the column is preserved).
/// An all-null bucket emits null. `meta` passes through unchanged. Deterministic
/// (C1). Full data stays on disk; only the served page is thinned. No-op when
/// `xs.len() <= 2 * buckets`.
/// Per-tap decimation kind (#111): the bounded exposure stream (C10, f64 ∈ [-1,+1])
/// reduces by per-bucket mean, so its net/duty-cycle level survives decimation
/// instead of collapsing to a -1..+1 band (every bucket of a multi-year exposure
/// straddles many sign flips, so min/max would be ±1 everywhere). An unbounded
/// cumulative curve (equity) keeps the min/max envelope so drawdowns survive. Keyed
/// on the tap name — `exposure` is the only bounded level tap today.
fn reduce_for_tap(tap: &str) -> ReduceKind {
if tap == "exposure" {
ReduceKind::Mean
} else {
ReduceKind::MinMax
}
}
/// Serve-time decimation on the aligned `ChartData` (#108). Partition the shared
/// `xs` into at most `buckets` contiguous index ranges; per non-empty bucket emit the
/// bucket's first (and, if it spans >1 index, last) timestamp as shared spine slots,
/// and reduce each series per its [`ReduceKind`] (#111): a `MinMax` series emits min
/// then max (the envelope — equity drawdowns survive), a `Mean` series emits the
/// per-bucket mean in both slots (the net level — a bounded exposure shows its
/// duty-cycle instead of a -1..+1 band). An all-null bucket emits null. `meta` passes
/// through unchanged. Deterministic (C1). Full data stays on disk; only the served
/// page is thinned. No-op when `xs.len() <= 2 * buckets`.
fn decimate(data: ChartData, buckets: usize) -> ChartData {
let buckets = buckets.max(1);
let n = data.xs.len();
@@ -285,31 +300,43 @@ fn decimate(data: ChartData, buckets: usize) -> ChartData {
.map(|s| {
let mut points: Vec<Option<f64>> = Vec::with_capacity(out_xs.len());
for &(lo, hi, two) in &bounds {
let mut mn = f64::INFINITY;
let mut mx = f64::NEG_INFINITY;
let mut any = false;
for v in s.points[lo..hi].iter().flatten() {
any = true;
if *v < mn {
mn = *v;
let (first, second) = match s.reduce {
ReduceKind::MinMax => {
// envelope: min at the first slot, max at the second.
let mut mn = f64::INFINITY;
let mut mx = f64::NEG_INFINITY;
let mut any = false;
for v in s.points[lo..hi].iter().flatten() {
any = true;
if *v < mn {
mn = *v;
}
if *v > mx {
mx = *v;
}
}
if any { (Some(mn), Some(mx)) } else { (None, None) }
}
if *v > mx {
mx = *v;
}
}
if any {
points.push(Some(mn));
if two {
points.push(Some(mx));
}
} else {
points.push(None);
if two {
points.push(None);
ReduceKind::Mean => {
// net level: the per-bucket mean written to both slots (a flat
// step), so a bounded high-flip series shows its duty-cycle
// instead of a -1..+1 band (#111).
let mut sum = 0.0;
let mut cnt = 0u32;
for v in s.points[lo..hi].iter().flatten() {
sum += *v;
cnt += 1;
}
let m = if cnt > 0 { Some(sum / cnt as f64) } else { None };
(m, m)
}
};
points.push(first);
if two {
points.push(second);
}
}
Series { name: s.name, y_scale_id: s.y_scale_id, points }
Series { name: s.name, y_scale_id: s.y_scale_id, points, reduce: s.reduce }
})
.collect();
@@ -341,7 +368,7 @@ fn build_chart_data(name: &str, traces: RunTraces) -> ChartData {
let y_scale_id = format!("y_{}", series.len());
let points: Vec<Option<f64>> =
joined.iter().map(|r| r.sides[i].as_ref().map(|row| row[c].as_f64())).collect();
series.push(Series { name, y_scale_id, points });
series.push(Series { name, y_scale_id, points, reduce: reduce_for_tap(&tap.tap) });
}
}
@@ -405,7 +432,7 @@ fn build_comparison_chart_data(
// A future multi-column tap selection would need a column index here.
let points: Vec<Option<f64>> =
joined.iter().map(|r| r.sides[i].as_ref().map(|row| row[0].as_f64())).collect();
series.push(Series { name: key.clone(), y_scale_id: y_scale_id.clone(), points });
series.push(Series { name: key.clone(), y_scale_id: y_scale_id.clone(), points, reduce: reduce_for_tap(tap) });
}
// member_rows is non-empty here (checked above) => members is non-empty, so
// members[0] is safe. commit/broker ARE shared across a family (one frozen
@@ -1829,7 +1856,7 @@ mod tests {
let points: Vec<Option<f64>> = (0..n).map(|i| Some(i as f64)).collect();
let data = ChartData {
xs,
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points, reduce: ReduceKind::MinMax }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2000);
@@ -1847,7 +1874,7 @@ mod tests {
let points: Vec<Option<f64>> = pv.into_iter().map(Some).collect();
let data = ChartData {
xs,
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points, reduce: ReduceKind::MinMax }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2);
@@ -1863,7 +1890,7 @@ mod tests {
points.extend(std::iter::repeat_n(None, 5));
let data = ChartData {
xs,
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }],
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points, reduce: ReduceKind::MinMax }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2);
@@ -1874,7 +1901,7 @@ mod tests {
fn decimate_is_a_noop_within_budget() {
let data = ChartData {
xs: vec![1, 2, 3],
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(1.0), Some(2.0), Some(3.0)] }],
series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(1.0), Some(2.0), Some(3.0)], reduce: ReduceKind::MinMax }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2000);
@@ -1888,12 +1915,39 @@ mod tests {
let xs: Vec<i64> = (0..n as i64).collect();
let points: Vec<Option<f64>> = (0..n).map(|i| Some(i as f64)).collect();
let meta = ChartMeta { name: "keep-me".into(), ..Default::default() };
let data = ChartData { xs, series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points }], meta };
let data = ChartData { xs, series: vec![Series { name: "equity".into(), y_scale_id: "y_0".into(), points, reduce: ReduceKind::MinMax }], meta };
let out = decimate(data, 2000);
assert_eq!(out.meta.name, "keep-me", "meta must pass through decimation");
assert!(out.xs.windows(2).all(|w| w[0] < w[1]), "decimated spine must stay strictly increasing");
}
/// #111: a bounded *level* series with `reduce = Mean` decimates to each bucket's
/// MEAN, not its min/max envelope — so a high-flip bipolar exposure shows its
/// net/duty-cycle level instead of collapsing to a -1..+1 band. RED under the
/// shipped min/max-only decimation (any bucket holding a +1 emits +1); GREEN once
/// `decimate` honours `ReduceKind::Mean`.
#[test]
fn decimate_mean_reduces_a_bipolar_series_to_its_bucket_level() {
// 10 points, 2 buckets. Bucket 0 (idx 0..5) = [+1,+1,-1,+1,+1] -> mean +0.6;
// bucket 1 (idx 5..10) = all -1 -> mean -1.0.
let xs: Vec<i64> = (0..10).collect();
let pv = vec![1.0, 1.0, -1.0, 1.0, 1.0, -1.0, -1.0, -1.0, -1.0, -1.0];
let points: Vec<Option<f64>> = pv.into_iter().map(Some).collect();
let data = ChartData {
xs,
series: vec![Series { name: "exposure".into(), y_scale_id: "y_0".into(), points, reduce: ReduceKind::Mean }],
meta: ChartMeta::default(),
};
let out = decimate(data, 2);
let got = out.series[0].points.clone();
// No -1..+1 envelope: bucket 0 is its mean (+0.6), not a min/max pair.
assert!(!got.contains(&Some(1.0)), "mean reduce must not emit a +1 envelope point: {got:?}");
assert!(got.contains(&Some(0.6)), "bucket-0 duty-cycle mean (+0.6) missing: {got:?}");
// bucket 0 spans two slots, both = the mean (a flat step, not a -1->+1 ramp).
assert_eq!(got[0], Some(0.6), "first slot must be the bucket mean");
assert_eq!(got[1], Some(0.6), "second slot must also be the bucket mean");
}
/// #102 single-run meta wiring: `build_chart_data` maps the `RunManifest` into
/// `ChartData.meta` — kind "run", the name arg, the manifest window/broker, the
/// charted taps, and the bound params stringified (each typed `Scalar` rendered
+26 -4
View File
@@ -123,14 +123,36 @@ pub enum ChartMode {
Panels,
}
/// How `decimate` reduces a series within each bucket (#111). An *envelope* series
/// (a cumulative equity curve) keeps its min+max so drawdown spikes survive; a
/// *level* series (the bounded exposure, f64 ∈ [-1,+1]) takes the per-bucket mean,
/// so a high-flip signal shows its net/duty-cycle level instead of collapsing to a
/// -1..+1 band (every bucket of a multi-year exposure straddles many flips, so
/// min/max would be ±1 everywhere). Server-side only — never serialized into the
/// page (the viewer reads decimated points, not how they were reduced). Rendering
/// min/max as range bars / OHLC is a separate follow-up; this only fixes the
/// reduction.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum ReduceKind {
/// Per-bucket min and max — the envelope (equity, unbounded cumulative).
#[default]
MinMax,
/// Per-bucket mean — the net level (exposure, bounded ∈ [-1,+1]).
Mean,
}
/// One chart series: a name, its own y-scale id (so disparate magnitudes stay
/// legible), and one `Option<f64>` per shared-x slot (`null` = a gap where the tap
/// did not fire at that timestamp).
/// legible), one `Option<f64>` per shared-x slot (`null` = a gap where the tap did
/// not fire at that timestamp), and how `decimate` reduces it per bucket.
#[derive(serde::Serialize)]
pub struct Series {
pub name: String,
pub y_scale_id: String,
pub points: Vec<Option<f64>>,
/// Per-bucket reduction for `decimate` (#111); server-side only, not part of the
/// served payload.
#[serde(skip)]
pub reduce: ReduceKind,
}
/// Run-context for the page header (#102): serialized into `window.AURA_TRACES.meta`
@@ -253,8 +275,8 @@ mod tests {
ChartData {
xs: vec![1, 2, 3],
series: vec![
Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(0.0), Some(0.001), Some(0.004)] },
Series { name: "exposure".into(), y_scale_id: "y_1".into(), points: vec![Some(1.0), Some(1.0), Some(-1.0)] },
Series { name: "equity".into(), y_scale_id: "y_0".into(), points: vec![Some(0.0), Some(0.001), Some(0.004)], reduce: ReduceKind::MinMax },
Series { name: "exposure".into(), y_scale_id: "y_1".into(), points: vec![Some(1.0), Some(1.0), Some(-1.0)], reduce: ReduceKind::Mean },
],
meta: ChartMeta {
kind: "run".into(),