Files
Aura/crates/aura-engine/tests/random_sweep_e2e.rs
T
Brummel 2c43296c2c feat(stage1-r): bias rename + stop-rule + position-management + summarize_r (iter 1)
Iteration 1 of the Stage-1 "R-based signal quality" cycle (spec 0065): a
strategy's signal quality, measured in R on its unsized bias stream, feed-forward.
New discrete-trade machinery, NOT a SimBroker extension.

What lands:
- exposure -> bias (refs #126): the Exposure node/type/file/output-field renamed to
  Bias (computation unchanged: clamp(signal/scale,-1,+1); the SEMANTICS change — the
  output is the unsized strategy bias, sizing leaves the strategy). Scoped to the
  semantic core; behaviour-preserving cosmetics are deferred (see below).
- stop-rule nodes (refs #119): VolStop = k*EMA(|price - prev_price|) (one fused node;
  the volatility that defines 1R, close-to-close — true-range ATR deferred, needs
  OHLC) + FixedStop (constant, the test fixture / fixed-vs-vol structural-axis sibling).
- PositionManagement (refs #127): the stateful heart. Latches the entry-cycle stop
  distance as the FROZEN R-denominator (never re-read), marks/exits no-look-ahead like
  SimBroker (a position earns from the next cycle; an exit realises against the cycle's
  close), and emits ONE dense per-cycle R-record (C8). Stop-outs are NOT capped at -1R
  (a gap through the stop realises R < -1 — the honest loss tail); R is computed
  size-invariantly (size = (exit-entry)*dir / latched_dist cancels). The trade ledger
  is the rows where closed_this_cycle; the R-equity is cum_realized + unrealized; a
  position open at window end is the last row (open=true) — explicit, never silent MtM.
- summarize_r + RMetrics (refs #129): the post-run fold (NOT an in-graph node —
  recorders drain post-run). E[R], win-rate, profit-factor, avg win/loss R, by-trade
  max R-drawdown, n_open_at_end (window-end force-close). SQN / conviction terciles /
  net-of-cost / RunMetrics.r are iteration 2.

Design adversarially hardened before implementation (12-juror refute panel; decisions
on #117). Ratified implementer deviations from the plan snippet, verified by hand:
- col-4 `direction` tracks the OPEN position at cycle end (window-end synthesis; names
  the reopened leg on a reversal), falling back to the closed trade's dir only when
  flat. summarize_r does not read col 4; the R-metrics are unaffected.
- .named("exposure") kept on the 4 previously-unnamed Bias nodes so the auto-derived
  param-path stays exposure.scale (no manifest/dir-name drift) — the unnamed nodes
  would otherwise flip to bias.scale.
- intra-doc links [`Exposure`] -> [`Bias`] in sim_broker/latch + the sample-model test
  pin (cosmetic, behaviour-neutral; broken intra-doc links fail cargo doc).
- the E2E drives a full bootstrapped Harness (stronger than the plan's direct-node
  drive — exercises the real cross-crate producer->consumer seam).

Deferred (behaviour-preserving, separate cosmetic pass — NOT this iteration):
RunMetrics.exposure_sign_flips / Metric::ExposureSignFlips, SimBroker/LongOnly
"exposure" input ports, the "exposure" tap/trace names, and the .named("exposure")
instance labels. Iteration 2: the Sizer seam, the RiskExecutor composite (+ the Veto
documented-seam), summarize_r enrichment, RunMetrics.r, and the CLI/recording surface.

Verification (orchestrator-run, not agent-claimed):
- cargo build --workspace: clean.
- cargo test --workspace: all green, 0 failed (incl. the new bias/stop_rule/
  position_management unit tests, the summarize_r arithmetic tests, and the
  stage1_r_e2e capstone + layout guard).
- cargo clippy --workspace --all-targets -- -D warnings: clean (exit 0).
- Keystone RED tests pass: no_lookahead_bias_exit_realises_the_held_move,
  stop_out_is_not_capped_at_minus_one_r, no_gap_stop_is_exactly_minus_one_r,
  reversal_closes_one_leg_and_reopens, open_at_window_end_is_carried_on_the_last_row.

refs #117 #119 #126 #127 #129
2026-06-23 19:48:58 +02:00

244 lines
10 KiB
Rust

//! End-to-end coverage for the random param-sweep axis (C12.1):
//! `RandomSpace` + `ParamRange` driven through the **public** `sweep` surface a
//! downstream researcher actually writes (the worked author example).
//!
//! The in-module unit tests in `sweep.rs` reach into crate internals
//! (`bootstrap_with_cells`, `sweep_with_threads`, `SplitMix64`); these tests use
//! only the exported API, so they pin the properties a real consumer observes:
//! a `SweepFamily` of `RunReport`s, the typed `SweepError` gate, and the
//! `named_params` view. The blueprint is reconstructed here (the crate-private
//! `test_fixtures` harness is unreachable from an integration test) through the
//! public `Composite` builder + `aura-std` nodes, so the test exercises the same
//! published surface the worked example does.
use std::sync::mpsc;
use aura_core::{Cell, Firing, ScalarKind, Timestamp};
use aura_engine::{
f64_field, summarize, sweep, BlueprintNode, Composite, Edge, OutField, ParamRange, ParamSpec,
RandomSpace, Role, RunManifest, RunReport, Scalar, Space, SweepError, Target, VecSource,
};
use aura_std::{Bias, Recorder, SimBroker, Sma, Sub};
/// Seven synthetic F64 ticks (mirrors the crate-private `synthetic_prices`):
/// short enough that small SMA windows warm up, so every run yields finite,
/// non-degenerate metrics. Deterministic input fixture.
fn synthetic_prices() -> Vec<(Timestamp, Scalar)> {
[
(1_i64, 1.0000_f64),
(2, 1.0010),
(3, 1.0030),
(4, 1.0060),
(5, 1.0040),
(6, 1.0010),
(7, 0.9990),
]
.iter()
.map(|&(t, p)| (Timestamp(t), Scalar::f64(p)))
.collect()
}
/// The SMA-cross signal-quality harness built through the PUBLIC builder API:
/// `[fast: I64, slow: I64, scale: F64]` param-space, ending in an equity sink and
/// an exposure sink. Returns the blueprint plus its two recording receivers (a
/// fresh channel pair per build, so each swept point runs disjointly — C1).
#[allow(clippy::type_complexity)]
fn sma_cross_harness() -> (
Composite,
mpsc::Receiver<(Timestamp, Vec<Scalar>)>,
mpsc::Receiver<(Timestamp, Vec<Scalar>)>,
) {
let (tx_eq, rx_eq) = mpsc::channel();
let (tx_ex, rx_ex) = mpsc::channel();
let sma_cross = Composite::new(
"sma_cross",
vec![
Sma::builder().named("fast").into(),
Sma::builder().named("slow").into(),
Sub::builder().into(),
],
vec![
Edge { from: 0, to: 2, slot: 0, from_field: 0 },
Edge { from: 1, to: 2, slot: 1, from_field: 0 },
],
vec![Role {
name: "price".into(),
targets: vec![Target { node: 0, slot: 0 }, Target { node: 1, slot: 0 }],
source: None,
}],
vec![OutField { node: 2, field: 0, name: "out".into() }],
);
let bp = Composite::new(
"root",
vec![
BlueprintNode::Composite(sma_cross),
Bias::builder().into(),
SimBroker::builder(0.0001).into(),
Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_eq).into(),
Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_ex).into(),
],
vec![
Edge { from: 0, to: 1, slot: 0, from_field: 0 }, // composite out -> Bias
Edge { from: 1, to: 2, slot: 0, from_field: 0 }, // exposure -> broker slot 0
Edge { from: 2, to: 3, slot: 0, from_field: 0 }, // equity -> sink
Edge { from: 1, to: 4, slot: 0, from_field: 0 }, // exposure -> sink
],
vec![Role {
name: "src".into(),
targets: vec![
Target { node: 0, slot: 0 }, // price -> sma_cross role 0
Target { node: 2, slot: 1 }, // price -> SimBroker price slot
],
source: Some(ScalarKind::F64),
}],
vec![], // root ends in sinks
);
(bp, rx_eq, rx_ex)
}
/// Build + bootstrap + run + summarize one swept point into a `RunReport`, using
/// only the public surface. A fresh harness (fresh sink channels) per point keeps
/// the runs disjoint (C1). The manifest is a fixed minimal fixture — only the
/// metrics carry the run, so determinism makes this reproduce a point exactly.
fn run_point(point: &[Cell]) -> RunReport {
let (bp, rx_eq, rx_ex) = sma_cross_harness();
let mut h = bp
.bootstrap_with_cells(point)
.expect("RandomSpace-drawn points are pre-validated against the param-space");
h.run(vec![Box::new(VecSource::new(synthetic_prices()))]);
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
RunReport {
manifest: RunManifest {
commit: "random-sweep-e2e".to_string(),
params: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "test".to_string(),
},
metrics: summarize(&equity, &exposure),
}
}
/// A `RandomSpace` over the harness's `[fast, slow, scale]` param-space: integer
/// windows drawn from the 7-tick fixture's proven domain (so SMAs warm up) and a
/// continuous scale. The integer ranges are exact single points so the family
/// stays small but every kind/arm is exercised.
fn sma_cross_random(count: usize, seed: u64) -> RandomSpace {
let space = sma_cross_harness().0.param_space();
RandomSpace::new(
&space,
vec![
ParamRange::i64(2, 3), // fast in [2, 3]
ParamRange::i64(4, 5), // slow in [4, 5]
ParamRange::f64(0.25, 1.5), // scale in [0.25, 1.5)
],
count,
seed,
)
.expect("ranges match the sample param-space kinds")
}
/// Property: a `RandomSpace` sweep is fully seed-determined end-to-end — the same
/// `(ranges, count, seed)` driven through the public `sweep` produces a
/// bit-identical `SweepFamily` of `RunReport`s, run after run (C1). The whole
/// pipeline (seeded draw -> bootstrap -> run -> summarize) reproduces, observed at
/// the published JSON boundary, not the internal `points()`.
#[test]
fn random_sweep_is_reproducible_at_the_report_boundary() {
let render = |seed: u64| -> Vec<String> {
sweep(&sma_cross_random(8, seed), run_point)
.points
.iter()
.map(|p| p.report.to_json())
.collect()
};
let a = render(0xC0FFEE);
let b = render(0xC0FFEE);
assert_eq!(a.len(), 8, "count points were swept");
assert_eq!(a, b, "same (ranges, count, seed) => bit-identical family (C1)");
}
/// Property: a different seed produces a different family — the sweep genuinely
/// samples the seed, it does not collapse to a constant set of points. Observed
/// via the public `named_params` coordinate view, never an internal field.
#[test]
fn random_sweep_seed_changes_the_family() {
let coords = |seed: u64| -> Vec<Vec<Scalar>> {
let family = sweep(&sma_cross_random(8, seed), run_point);
(0..family.points.len())
.map(|i| family.named_params(i).into_iter().map(|(_, v)| v).collect())
.collect()
};
assert_ne!(
coords(1),
coords(2),
"different seeds => different swept coordinate sets",
);
}
/// Property: every coordinate the sweep actually ran on lies inside its declared
/// `ParamRange` — the I64 slots inclusive `[lo, hi]`, the F64 slot half-open
/// `[lo, hi)`. A regression that let a draw escape its range would silently run
/// the strategy out of its declared domain; this pins the bound at the observable
/// `named_params` view of the family that was run.
#[test]
fn swept_points_stay_inside_their_declared_ranges() {
let family = sweep(&sma_cross_random(200, 0xABCDEF), run_point);
assert_eq!(family.points.len(), 200);
for i in 0..family.points.len() {
let named = family.named_params(i);
let fast = named[0].1.as_i64();
let slow = named[1].1.as_i64();
let scale = named[2].1.as_f64();
assert!((2..=3).contains(&fast), "fast in [2,3] inclusive, got {fast}");
assert!((4..=5).contains(&slow), "slow in [4,5] inclusive, got {slow}");
assert!((0.25..1.5).contains(&scale), "scale in [0.25,1.5), got {scale}");
// and the run that consumed this in-range point produced a finite metric
assert!(family.points[i].report.metrics.total_pips.is_finite());
}
}
/// Property: the typed validation gate rejects a non-numeric param slot BEFORE
/// any run. A `Bool` slot cannot carry a continuous range (it is degenerate), so
/// `RandomSpace::new` returns the public `SweepError::NonNumericRange` value — an
/// observable typed error at the published API, not a panic and not a swept run.
#[test]
fn bool_slot_is_rejected_as_non_numeric_before_any_run() {
let space = vec![ParamSpec { name: "flag".into(), kind: ScalarKind::Bool }];
let err = RandomSpace::new(&space, vec![ParamRange::i64(0, 1)], 10, 0)
.expect_err("a Bool slot is not range-sampleable");
assert_eq!(err, SweepError::NonNumericRange { slot: 0, kind: ScalarKind::Bool });
}
/// Property: a `count == 0` `RandomSpace` is a valid, explicit empty family (not
/// the "accidental collapse" an empty grid axis would be) — `sweep` over it
/// returns an empty `SweepFamily` while still carrying the param-space schema, so
/// a downstream `named_params` consumer sees a well-formed empty result.
#[test]
fn zero_count_sweep_is_a_well_formed_empty_family() {
let space = sma_cross_harness().0.param_space();
let rs = sma_cross_random(0, 0);
assert!(rs.is_empty(), "count == 0 is the explicit empty space");
let family = sweep(&rs, run_point);
assert!(family.points.is_empty(), "no points swept");
assert_eq!(family.space, space, "the empty family still carries the schema");
}
/// Property: `GridSpace` and `RandomSpace` are interchangeable through the `Space`
/// trait `sweep` is generic over — the same generic helper drives either
/// enumeration. This is the trait abstraction the cut introduced, observed via the
/// public `Space::param_specs`, kept from regressing back to a `GridSpace`-only
/// `sweep` signature.
#[test]
fn random_space_is_driven_through_the_space_trait() {
fn schema_len<S: Space>(s: &S) -> usize {
s.param_specs().len()
}
let rs = sma_cross_random(4, 7);
assert_eq!(schema_len(&rs), 3, "the [fast, slow, scale] schema reaches the trait surface");
// and the generic `sweep` accepts it by value of the same bound
let family = sweep(&rs, run_point);
assert_eq!(family.points.len(), 4);
}