bb0b0aeac2
The OOS bootstrap conduit RMetrics.trade_rs becomes net_trade_rs and carries the COST-NETTED per-trade R (r − cost_in_r, trade order). Every conduit consumer is thereby net-when-costed: the monte-carlo pooled-OOS and per-survivor bootstraps, the walk-forward oos_r pooling, and the deflation null-max — which previously compared a net observed statistic against a gross-resampled null whenever a costed campaign selected on net_expectancy_r. An uncosted run is bit-identical (empty cost stream ⇒ cost 0.0 per trade), so every existing golden pin stays green. Design: one conduit, no knob — an explicit net: knob would let a costed campaign silently produce a gross headline again (the exact misreading of the issue's evidence). Per-member RMetrics scalar fields stay gross; net_expectancy_r keeps its meaning. Wire shape unchanged (serde(skip)). The net series is materialized as a separate expression in summarize_r; the pinned-float expressions (net_sum, the SQN pair, the lockstep r_metrics_from_rs copies) keep their tokens verbatim. Fork decisions and rationales: issue #259 comments. New coverage, both hostless over the synthetic SYMA archive: a costed campaign twin shifts the pooled-OOS bootstrap (sweep→gate→wf→mc) and every per-survivor bootstrap (sweep→mc) below its gross sibling, with the trade population unchanged; a unit test pins the conduit as the cost-netted series with the empty-stream degeneracy. Existing conduit tests renamed with the field. Verified: full workspace suite green (71 result blocks), clippy clean, zero bare trade_rs tokens remain, ledger conduit prose updated in place. closes #259
143 lines
5.6 KiB
Rust
143 lines
5.6 KiB
Rust
//! End-to-end coverage for aura-campaign's public seam (cycle 0107 tasks
|
|
//! 4-5): `MemberRunner` is documented as the ONE binding point every
|
|
//! consumer implements ("every consumer ... binds its own runner while the
|
|
//! execution semantics live here once"), so these tests drive it as an
|
|
//! external consumer would — through `&dyn MemberRunner`, never an internal
|
|
//! function — and check the seam's plumbing plus the cross-roster invariant
|
|
//! `preflight` silently depends on. The in-crate unit tests in `lib.rs` pin
|
|
//! `member_metric`'s full 14-name mapping and `preflight`'s (crate-private)
|
|
//! shape rules against hand-built fixtures; these tests pin what a real,
|
|
//! external, dynamically-dispatched consumer sees.
|
|
|
|
use std::collections::BTreeMap;
|
|
|
|
use aura_campaign::{member_metric, CellSpec, MemberFault, MemberRunner, PER_MEMBER_METRICS, RANKABLE_METRICS};
|
|
use aura_engine::{summarize, RMetrics, RunManifest, RunMetrics, RunReport, Scalar, Timestamp};
|
|
|
|
fn cell() -> CellSpec {
|
|
CellSpec {
|
|
strategy_ordinal: 0,
|
|
strategy_id: "0".repeat(64),
|
|
blueprint_json: "{}".to_string(),
|
|
axes: BTreeMap::new(),
|
|
instrument: "EURUSD".to_string(),
|
|
window_ms: (0, 20),
|
|
regime: None,
|
|
regime_ordinal: 0,
|
|
}
|
|
}
|
|
|
|
/// A minimal external `MemberRunner`: ignores its params and produces
|
|
/// metrics via the real `aura_engine::summarize` reduction (not a hand-typed
|
|
/// `RunMetrics` literal), echoing the `window_ms` it was called with into
|
|
/// the manifest — exactly what a real consumer's harness-binding code does.
|
|
struct FixedRunner;
|
|
|
|
impl MemberRunner for FixedRunner {
|
|
fn run_member(
|
|
&self,
|
|
_cell: &CellSpec,
|
|
_params: &[(String, Scalar)],
|
|
window_ms: (i64, i64),
|
|
) -> Result<RunReport, MemberFault> {
|
|
let equity = vec![(Timestamp(0), 0.0), (Timestamp(1), 3.0), (Timestamp(2), 1.0)];
|
|
let metrics = summarize(&equity, &[]);
|
|
Ok(RunReport {
|
|
manifest: RunManifest {
|
|
commit: "e2e".to_string(),
|
|
params: vec![],
|
|
defaults: vec![],
|
|
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
|
|
seed: 0,
|
|
broker: "test".to_string(),
|
|
selection: None,
|
|
instrument: None,
|
|
topology_hash: None,
|
|
project: None,
|
|
},
|
|
metrics,
|
|
})
|
|
}
|
|
}
|
|
|
|
/// Property: `MemberRunner` is object-safe and dispatches through
|
|
/// `&dyn MemberRunner` (the shape a heterogeneous, per-consumer roster of
|
|
/// runners requires); the exact sub-window given at the call site — not
|
|
/// `cell.window_ms`, a walk-forward stage passes a narrower sub-range —
|
|
/// reaches the runner unmodified; and the real `aura_engine::summarize()`
|
|
/// output the runner produced is what `member_metric` reads back through the
|
|
/// crate's public per-name resolver.
|
|
#[test]
|
|
fn member_runner_seam_dispatches_through_dyn_trait_object_and_carries_the_window() {
|
|
let runner = FixedRunner;
|
|
let dyn_runner: &dyn MemberRunner = &runner;
|
|
let report = dyn_runner
|
|
.run_member(&cell(), &[("len".to_string(), Scalar::i64(3))], (10, 20))
|
|
.expect("member run succeeds");
|
|
|
|
// the call-site sub-window (10, 20), not cell().window_ms == (0, 20),
|
|
// reached the runner unmodified
|
|
assert_eq!(report.manifest.window, (Timestamp(10), Timestamp(20)));
|
|
|
|
// the real summarize() reduction over [0.0, 3.0, 1.0] is what
|
|
// member_metric reads back: last == 1.0, peak-to-trough == 2.0
|
|
assert_eq!(member_metric(&report, "total_pips"), Some(1.0));
|
|
assert_eq!(member_metric(&report, "max_drawdown"), Some(2.0));
|
|
}
|
|
|
|
/// Property: every metric name in `RANKABLE_METRICS` (the roster `preflight`
|
|
/// permits a `std::sweep`/`std::walk_forward` stage to SELECT on) is also a
|
|
/// member of `PER_MEMBER_METRICS` and resolves to `Some` via `member_metric`.
|
|
/// The two rosters are independently hand-copied (documented drift risk,
|
|
/// #190); were they to diverge, `preflight` would accept a selection metric
|
|
/// that `member_metric` silently reads back as `None` for every member — a
|
|
/// sweep/walk_forward stage that always empties, not a compile error or an
|
|
/// early refusal.
|
|
#[test]
|
|
fn rankable_metrics_are_all_per_member_resolvable() {
|
|
let report = RunReport {
|
|
manifest: RunManifest {
|
|
commit: "e2e".to_string(),
|
|
params: vec![],
|
|
defaults: vec![],
|
|
window: (Timestamp(0), Timestamp(1)),
|
|
seed: 0,
|
|
broker: "test".to_string(),
|
|
selection: None,
|
|
instrument: None,
|
|
topology_hash: None,
|
|
project: None,
|
|
},
|
|
metrics: RunMetrics {
|
|
total_pips: 1.0,
|
|
max_drawdown: 2.0,
|
|
bias_sign_flips: 3,
|
|
r: Some(RMetrics {
|
|
expectancy_r: 4.0,
|
|
n_trades: 5,
|
|
win_rate: 6.0,
|
|
avg_win_r: 7.0,
|
|
avg_loss_r: 8.0,
|
|
profit_factor: 9.0,
|
|
max_r_drawdown: 10.0,
|
|
n_open_at_end: 11,
|
|
sqn_normalized: 12.0,
|
|
sqn: 13.0,
|
|
net_expectancy_r: 14.0,
|
|
conviction_terciles_r: [0.0; 3],
|
|
net_trade_rs: Vec::new(),
|
|
}),
|
|
},
|
|
};
|
|
for &metric in RANKABLE_METRICS {
|
|
assert!(
|
|
PER_MEMBER_METRICS.contains(&metric),
|
|
"{metric} is rankable but missing from PER_MEMBER_METRICS"
|
|
);
|
|
assert!(
|
|
member_metric(&report, metric).is_some(),
|
|
"{metric} is rankable but member_metric returned None"
|
|
);
|
|
}
|
|
}
|