Brummel/Aura
1
0
Fork 0
Code Issues 32 Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(aura-engine): walk-forward param plane — space on family, tag-free chosen_params, schema-checked param_stability

Browse Source 
Make the walk-forward param plane C7-clean and symmetric with the sweep param plane. WalkForwardResult gains `space: Vec<ParamSpec>` (the kinds, once, mirroring SweepFamily.space) and WindowRun.chosen_params changes Vec<Scalar> -> Vec<Cell> (the tag-free coordinate the inner sweep already produces). param_stability now resolves its numeric coercion once per slot against the schema (result.space) into a Vec<fn(Cell)->f64>, then runs a type-blind reduction loop — replacing the old per-window-value scalar_as_f64 with its per-value unreachable!. scalar_as_f64 is deleted.

This overturns cycle 0047's "Fork A" (chosen_params stays Vec<Scalar>), which was justified by Scalar's new serializability. That defense was moot: WalkForwardResult and WindowRun carry no serde derives — they are never serialized; only the embedded oos_report (a RunReport, typed via the 0047 lift) is. So chosen_params is a pure in-memory substrate for param_stability, and Vec<Scalar> made it C7-impure (carrying the kind N×M times at the value, where the kind is per-slot constant). The substantive case — kind at the schema, SweepFamily symmetry, per-value unreachable! eliminated — wins, the same argument-form that retired the {kind,cell} Scalar struct in 0047.

The seam ([A]): walk_forward / walk_forward_with_threads take `space: Vec<ParamSpec>` by value, a sibling of the run-window closure; the Fn(WindowBounds)->WindowRun+Sync bound is unchanged and run_indexed is untouched (space is never captured by the Sync closure, moved onto the result at the end). A debug_assert in walk_forward_with_threads guards the arity coupling (every window's chosen_params.len() == space.len(), guaranteed by the same blueprint). walkforward_family computes the space once before the roll and passes the tag-free winner (chosen_params: best.params) directly, dropping the from_cell+zip reconstruction.

[D]: a Bool param slot coerces to 0/1 (mean = fraction "on"), keeping the coercer total over the knob kinds; a Timestamp slot is structurally impossible (C20) so the schema pass carries one unreachable! (once per slot, never per value). param_stability keeps an `if result.windows.is_empty() { return Vec::new(); }` guard to preserve its documented empty-on-no-windows contract. WalkForwardResult and SweepFamily share the (space + tag-free Cell points + zip_params) idiom but stay distinct types — different axis (Sweep enumerates input coordinates; WF rolls time and chosen_params is the optimizer's output), and WF additionally carries bounds + oos_equity + stitched_oos_equity.

Behaviour-preserving (C1): the three migrated walk-forward tests stay green with identical MetricStats (param_stability_reduces_chosen_params_per_slot still pins mean 2.5 / p50 2.5 / mean 1.5); the i64->f64 and f64 coercions are value-identical to the deleted scalar_as_f64. The one genuinely new behaviour (Bool 0/1) is unexercised by built-in grids and pinned by the new param_stability_reduces_bool_slot_to_fraction_true (mean 0.75, 3-of-4 true).

Gates: build --workspace --all-targets, test --workspace, clippy --workspace --all-targets -D warnings, doc --workspace --no-deps — all clean.

closes #75
This commit is contained in:
Brummel
2026-06-16 19:01:22 +02:00
parent 6ce00ee053
commit 9febdb9623
2 changed files with 97 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/aura-cli/src/main.rs
+5 -9
View File
@@ -455,19 +455,15 @@ fn walkforward_family() -> WalkForwardResult {
let span = window_of(&sources).expect("non-empty synthetic stream");
let roller = WindowRoller::new(span, 24, 12, 12, RollMode::Rolling)
.expect("built-in walk-forward config fits the 60-bar synthetic span");
walk_forward(roller, |w: WindowBounds| {
let space = sample_blueprint_with_sinks().0.param_space();
walk_forward(roller, space, |w: WindowBounds| {
let is_family = sweep_over(w.is.0, w.is.1);
let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric");
let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1);
WindowRun {
// best.params is the enumerated cell winner; reconstruct the
// self-describing report record (Option A) via the in-sample space.
chosen_params: best
.params
.iter()
.zip(&is_family.space)
.map(|(c, ps)| Scalar::from_cell(ps.kind, *c))
.collect(),
// The tag-free sweep winner is the chosen point; its kinds live on
// WalkForwardResult.space (computed once above from the same blueprint).
chosen_params: best.params,
oos_equity,
oos_report,
}
crates/aura-engine/src/walkforward.rs
+92 -48
View File
@@ -12,7 +12,8 @@
//! reduction ([`param_stability`]), not a stored field — the result is non-redundant.
use crate::sweep::run_indexed;
use crate::{MetricStats, RunReport, Scalar, Timestamp};
use crate::{MetricStats, ParamSpec, RunReport, ScalarKind, Timestamp};
use aura_core::Cell;
/// One rolling split's time bounds: an in-sample window and the out-of-sample
/// window that follows it. Carries ONLY bounds — never tick data (#71). C2
@@ -122,11 +123,11 @@ impl Iterator for WindowRoller {
/// [`SweepPoint`](crate::SweepPoint)/[`McDraw`](crate::McDraw).
#[derive(Clone, Debug, PartialEq)]
pub struct WindowRun {
/// The chosen params, kept as self-describing [`Scalar`]s — the record carrier
/// (serializable). `param_stability` reduces them to `f64` only at the
/// statistic boundary (a distribution over windows is intrinsically
/// real-valued); the typed value is preserved up to that one reduction.
pub chosen_params: Vec<Scalar>,
/// The chosen params as the tag-free coordinate [`Cell`]s the inner sweep
/// produced (C7: the kind lives once on [`WalkForwardResult::space`], not at the
/// value). The named/typed view is `zip_params(&result.space, &chosen_params)` —
/// the same idiom as [`SweepFamily::named_params`](crate::SweepFamily).
pub chosen_params: Vec<Cell>,
pub oos_equity: Vec<(Timestamp, f64)>,
pub oos_report: RunReport,
}
@@ -146,6 +147,11 @@ pub struct WindowOutcome {
/// construction.
#[derive(Clone, Debug, PartialEq)]
pub struct WalkForwardResult {
/// The param-space schema (kinds + slot names), once for the whole family —
/// the kinds the per-window [`WindowRun::chosen_params`] cells are read against
/// (C7). Identical for every window (same blueprint). Mirrors
/// [`SweepFamily::space`](crate::SweepFamily); the two stay distinct types.
pub space: Vec<ParamSpec>,
/// Per-window outcomes in roll order (window 0 first), independent of thread
/// completion (C1).
pub windows: Vec<WindowOutcome>,
@@ -167,12 +173,12 @@ pub struct WalkForwardResult {
/// stream `Vec` in this API. Precondition: the roller yields `>= 1` window
/// (`WindowRoller::new` rejects an empty roll). Param stability is a separate
/// on-demand reduction ([`param_stability`]), never computed here.
pub fn walk_forward<F>(roller: WindowRoller, run_window: F) -> WalkForwardResult
pub fn walk_forward<F>(roller: WindowRoller, space: Vec<ParamSpec>, run_window: F) -> WalkForwardResult
where
F: Fn(WindowBounds) -> WindowRun + Sync,
{
let nthreads = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(1);
walk_forward_with_threads(roller, nthreads, run_window)
walk_forward_with_threads(roller, space, nthreads, run_window)
}
/// The thread-count-explicit core of [`walk_forward`]. Module-private: the public
@@ -183,6 +189,7 @@ where
/// onto their bounds in roll order, and stitches the OOS segments.
fn walk_forward_with_threads<F>(
roller: WindowRoller,
space: Vec<ParamSpec>,
nthreads: usize,
run_window: F,
) -> WalkForwardResult
@@ -198,7 +205,11 @@ where
.map(|(bounds, run)| WindowOutcome { bounds, run })
.collect();
let stitched_oos_equity = stitch(&windows);
WalkForwardResult { windows, stitched_oos_equity }
debug_assert!(
windows.iter().all(|w| w.run.chosen_params.len() == space.len()),
"every window's chosen point must match the param-space arity (same blueprint)"
);
WalkForwardResult { space, windows, stitched_oos_equity }
}
/// Fold the per-window OOS equity segments into one continuous curve: each
@@ -221,49 +232,47 @@ fn stitch(windows: &[WindowOutcome]) -> Vec<(Timestamp, f64)> {
}
/// On-demand param-stability summary: one [`MetricStats`](crate::MetricStats) per
/// param slot, over the chosen values (`Scalar` coerced to `f64`) across all
/// windows — reuses [`MetricStats::from_values`](crate::MetricStats::from_values).
/// A pure reduction over `result.windows[*].run.chosen_params`, NOT stored on the
/// result (sweep-precedent: a summary is computed on demand, not cached), so it
/// stays non-redundant and the caller can compute any other measure (std, IQR,
/// distinct-count) from the same raw substrate. One entry per param slot, in slot
/// order; empty if there are no windows or the strategy has no params. The slot
/// count is taken from the first window (all windows of one blueprint share the
/// param-space arity, C11).
/// param slot, over the chosen values across all windows — reuses
/// [`MetricStats::from_values`](crate::MetricStats::from_values). A pure reduction
/// over `result.windows[*].run.chosen_params`, NOT stored on the result
/// (sweep-precedent: a summary is computed on demand, not cached). One entry per
/// param slot, in slot order; empty if there are no windows or the strategy has no
/// params. The numeric coercion is resolved once per slot against the schema
/// (`result.space`, C7) — never per window-value: an `i64`/`f64` slot maps to its
/// value, a `bool` slot to `0.0`/`1.0` (so `mean` is the fraction of windows that
/// chose `true`). A `timestamp` slot is structurally impossible (C20: a timestamp
/// is a structural axis, never a numeric knob).
pub fn param_stability(result: &WalkForwardResult) -> Vec<MetricStats> {
let Some(first) = result.windows.first() else {
if result.windows.is_empty() {
return Vec::new();
};
let nslots = first.run.chosen_params.len();
(0..nslots)
.map(|slot| {
let vals: Vec<f64> = result
.windows
.iter()
.map(|w| scalar_as_f64(w.run.chosen_params[slot]))
.collect();
}
let coerce: Vec<fn(Cell) -> f64> = result
.space
.iter()
.map(|ps| match ps.kind {
ScalarKind::I64 => (|c: Cell| c.i64() as f64) as fn(Cell) -> f64,
ScalarKind::F64 => |c: Cell| c.f64(),
ScalarKind::Bool => |c: Cell| c.bool() as i64 as f64,
ScalarKind::Timestamp => {
unreachable!("timestamp is a structural axis (C20), never a param knob")
}
})
.collect();
coerce
.iter()
.enumerate()
.map(|(slot, f)| {
let vals: Vec<f64> =
result.windows.iter().map(|w| f(w.run.chosen_params[slot])).collect();
MetricStats::from_values(&vals)
})
.collect()
}
/// Coerce a numeric `Scalar` to `f64` for stability statistics — the one place a
/// distribution over windows demands a real-valued reduction. Params are
/// `i64`/`f64` typed values; a non-numeric scalar in a param slot is a wiring
/// bug, surfaced like the engine's other "checked at wiring" violations.
fn scalar_as_f64(s: Scalar) -> f64 {
match s.kind() {
aura_core::ScalarKind::I64 => s.as_i64() as f64,
aura_core::ScalarKind::F64 => s.as_f64(),
other => unreachable!("non-numeric param scalar: {other:?}"),
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{summarize, RunManifest};
use aura_core::Scalar;
fn dummy_report() -> RunReport {
RunReport {
@@ -278,9 +287,18 @@ mod tests {
}
}
/// The 2-slot param-space the run-window fixtures choose into: an i64 length
/// and an f64 scale. Matches `run_window_fixture` / `mk` arity (debug_assert).
fn fixture_space() -> Vec<ParamSpec> {
vec![
ParamSpec { name: "len".to_string(), kind: ScalarKind::I64 },
ParamSpec { name: "scale".to_string(), kind: ScalarKind::F64 },
]
}
/// A WindowOutcome with explicit OOS equity + chosen params (zero bounds — the
/// stitch/stability reductions ignore bounds).
fn outcome(oos_equity: Vec<(Timestamp, f64)>, chosen: Vec<Scalar>) -> WindowOutcome {
fn outcome(oos_equity: Vec<(Timestamp, f64)>, chosen: Vec<Cell>) -> WindowOutcome {
WindowOutcome {
bounds: WindowBounds {
is: (Timestamp(0), Timestamp(0)),
@@ -296,7 +314,7 @@ mod tests {
fn run_window_fixture(w: WindowBounds) -> WindowRun {
let Timestamp(k) = w.oos.0;
WindowRun {
chosen_params: vec![Scalar::i64(k % 3), Scalar::f64(k as f64 * 0.5)],
chosen_params: vec![Cell::from_i64(k % 3), Cell::from_f64(k as f64 * 0.5)],
oos_equity: vec![(w.oos.0, k as f64 * 0.1), (w.oos.1, k as f64 * 0.1 + 1.0)],
oos_report: dummy_report(),
}
@@ -312,7 +330,7 @@ mod tests {
WindowRoller::new((Timestamp(0), Timestamp(24)), 10, 5, 5, RollMode::Rolling)
.expect("valid")
.collect();
let result = walk_forward(roller, run_window_fixture);
let result = walk_forward(roller, fixture_space(), run_window_fixture);
assert_eq!(result.windows.len(), expected.len());
assert_eq!(
result.windows.iter().map(|w| w.bounds).collect::<Vec<_>>(),
@@ -327,10 +345,10 @@ mod tests {
WindowRoller::new((Timestamp(0), Timestamp(100)), 20, 5, 5, RollMode::Rolling)
.expect("valid")
};
let one = walk_forward_with_threads(cfg(), 1, run_window_fixture);
let many = walk_forward_with_threads(cfg(), 8, run_window_fixture);
let one = walk_forward_with_threads(cfg(), fixture_space(), 1, run_window_fixture);
let many = walk_forward_with_threads(cfg(), fixture_space(), 8, run_window_fixture);
assert_eq!(one, many);
assert_eq!(one, walk_forward(cfg(), run_window_fixture));
assert_eq!(one, walk_forward(cfg(), fixture_space(), run_window_fixture));
}
#[test]
@@ -377,12 +395,13 @@ mod tests {
oos: (Timestamp(0), Timestamp(0)),
},
run: WindowRun {
chosen_params: vec![Scalar::i64(a), Scalar::f64(b)],
chosen_params: vec![Cell::from_i64(a), Cell::from_f64(b)],
oos_equity: vec![],
oos_report: dummy_report(),
},
};
let result = WalkForwardResult {
space: fixture_space(),
windows: vec![mk(2, 1.0), mk(2, 1.0), mk(3, 2.0), mk(3, 2.0)],
stitched_oos_equity: vec![],
};
@@ -393,6 +412,31 @@ mod tests {
assert_eq!(stab[1].mean, 1.5);
}
#[test]
fn param_stability_reduces_bool_slot_to_fraction_true() {
// spec §Testing: a Bool param slot coerces 0/1, so mean = fraction of
// windows that chose `true`. 3 of 4 true -> 0.75.
let mk = |flag: bool| WindowOutcome {
bounds: WindowBounds {
is: (Timestamp(0), Timestamp(0)),
oos: (Timestamp(0), Timestamp(0)),
},
run: WindowRun {
chosen_params: vec![Cell::from_bool(flag)],
oos_equity: vec![],
oos_report: dummy_report(),
},
};
let result = WalkForwardResult {
space: vec![ParamSpec { name: "flag".to_string(), kind: ScalarKind::Bool }],
windows: vec![mk(true), mk(false), mk(true), mk(true)],
stitched_oos_equity: vec![],
};
let stab = param_stability(&result);
assert_eq!(stab.len(), 1);
assert_eq!(stab[0].mean, 0.75);
}
#[test]
fn roller_rolling_emits_consecutive_oos_windows() {
// spec §Testing 1: rolling, origin=0, is_len=10, oos_len=5, step=5, end=24.