d3b1a1aead
closes #272 A member fault (no-data, bind, run, or a caught panic) is now a recorded per-cell outcome instead of aborting the whole campaign and discarding every already-computed cell. The incident that motivated this (a 22-instrument campaign lost ~36 healthy cells ~6.7 min in because Copper had an archive gap) now completes: the healthy cells persist, the gap cell is recorded as failed, and the run exits 3. Direction (owner decision 2026-07-14): run to completion and report compromised results; no coverage preflight, no window synthesis. Containment granularity: - The CELL for a sweep-stage member fault (a grid hole structurally compromises winner selection, so the whole cell fails). - The FOLD for a walk_forward member fault (independent time windows): the surviving folds pool into the family, failed folds are recorded as StageRealization.window_faults, and the summary names the ratio. - aura-registry: additive CellFault / CellFaultKind (closed: no_data|bind|run|panic|window) / WindowFault / CellCoverage, plus fault/coverage fields on CellRealization and window_faults on StageRealization — all serde-default-skipped, so pre-#272 campaign_runs lines parse and round-trip byte-identical. - aura-campaign: run_cell returns a fault-annotated CellRealization instead of Err (execute's accumulate-then-append-once tail is unchanged and now persists every healthy cell + the one run record); a `contain` split keeps ExecFault::Registry and doc-shape preflight faults global while Member/Window become per-cell/per-fold. Member panics are caught with catch_unwind(AssertUnwindSafe) at all three member-run sites (sweep IS/OOS) and recorded as MemberFault::Panic — a member panic no longer aborts the process. The wf stage partitions Registry faults (global) from Member/Window (per-fold) and filters faulted-fold placeholders (the "faulted-member-placeholder" broker sentinel) out of the persisted family. - aura-cli: exec_fault_prose gains the Panic arm; CliMemberRunner::window_coverage derives effective bounds + interior gap months from the #264 archive primitives; present_campaign prints per-cell failure notes + a completion summary and threads the failed-cell count; a run with >=1 failed cell exits 3 ("completed with failed cells") uniformly across `aura campaign run` and the dissolved sweep/walkforward/mc/generalize verbs (exit_on_campaign_result). Usage stays 2, refused-before-running stays 1, clean stays 0. Tests: the global-abort pins flip to containment (execute + the two wf fault tests → fold-containment + all-folds-fail-the-cell); new panic-containment tests on both the sweep path (PanicRunner) and the wf path (this commit adds the wf mirror the loop left uncovered); a new gapped-archive e2e (one covered cell + one gap cell → exit 3); the ~14 CLI exit-1 pins move to the exit-3 register; a pre-#272-line byte-identical round-trip guard. Suite: cargo test --workspace green (1309 tests, 0 failed); clippy clean. Decision log: #272 comments (fork rationale, the fold Registry/Member split, the placeholder sentinel, uniform exit-3). Follow-up (minor, not blocking): the plan under-scoped Task 1 to aura-registry though the additive fields also touch aura-campaign's exec.rs literals — the loop absorbed it mechanically; a future plan for a cross-crate additive-field change should scope every crate's construction sites in the first task.
1570 lines
63 KiB
Rust
1570 lines
63 KiB
Rust
//! aura-campaign — the campaign-execution library (#198, cycle 0107).
|
|
//!
|
|
//! Campaign *semantics* as a reusable leaf crate: cell enumeration over a
|
|
//! campaign document's (strategy, instrument, window) matrix, preflight of
|
|
//! the v1 executable pipeline shape, per-member gate evaluation, winner
|
|
//! selection, and realization assembly over the registry's family machinery.
|
|
//! Harness construction and data binding enter exclusively through the
|
|
//! one-method [`MemberRunner`] seam, so every consumer (the CLI today; the
|
|
//! playground and tests tomorrow) binds its own runner while the execution
|
|
//! semantics live here once — this crate is NOT the World (C12/C21): it
|
|
//! realizes one campaign document; it owns no topology, no data sources,
|
|
//! and no UI.
|
|
|
|
mod exec;
|
|
pub use exec::{
|
|
execute, member_fault_prose, CampaignOutcome, CellOutcome, StageFamily, StageSelectionOut,
|
|
};
|
|
|
|
use std::collections::BTreeMap;
|
|
|
|
use aura_core::Scalar;
|
|
use aura_engine::RunReport;
|
|
use aura_registry::{check_r_metric, RegistryError};
|
|
use aura_research::{Axis, CampaignDoc, Cmp, ProcessDoc, RiskRegime, SelectRule, StageBlock};
|
|
|
|
/// One structural cell of the campaign matrix: (strategy, instrument,
|
|
/// window) — #198 decision 7.
|
|
pub struct CellSpec {
|
|
pub strategy_ordinal: usize,
|
|
/// Resolved blueprint content id (== the topology hash).
|
|
pub strategy_id: String,
|
|
/// Canonical blueprint bytes from the store.
|
|
pub blueprint_json: String,
|
|
/// The campaign's tuning axes (raw `param_space()` names).
|
|
pub axes: BTreeMap<String, Axis>,
|
|
pub instrument: String,
|
|
/// Inclusive epoch-ms bounds.
|
|
pub window_ms: (i64, i64),
|
|
/// The cell's risk regime. `None` = the member runner's baked default (the
|
|
/// absent/empty-`risk` case); `Some` = an explicit document regime.
|
|
pub regime: Option<RiskRegime>,
|
|
/// The regime's ordinal in the resolved regime list (0 for the default) —
|
|
/// keys the generalize unit and discriminates family names.
|
|
pub regime_ordinal: usize,
|
|
}
|
|
|
|
/// The harness/data binding seam — the ONLY thing a consumer implements.
|
|
/// Params arrive as (raw axis name, value) pairs; the implementation binds
|
|
/// them to its harness convention and runs the member over `cell.instrument`
|
|
/// restricted to `window_ms` (inclusive epoch-ms, a sub-range of
|
|
/// `cell.window_ms` — a walk-forward stage passes sub-windows).
|
|
pub trait MemberRunner: Sync {
|
|
fn run_member(
|
|
&self,
|
|
cell: &CellSpec,
|
|
params: &[(String, Scalar)],
|
|
window_ms: (i64, i64),
|
|
) -> Result<RunReport, MemberFault>;
|
|
|
|
/// The archive coverage of this cell's (instrument, window), when the
|
|
/// runner can know it and it deviates from the requested window (#272).
|
|
/// One call per cell — coverage is a property of the cell, not a member.
|
|
fn window_coverage(&self, _cell: &CellSpec) -> Option<aura_registry::CellCoverage> {
|
|
None
|
|
}
|
|
}
|
|
|
|
/// Display-free member faults (the consumer phrases them — the RefFault
|
|
/// pattern).
|
|
#[derive(Clone, Debug, PartialEq)]
|
|
pub enum MemberFault {
|
|
NoData { instrument: String, window_ms: (i64, i64) },
|
|
Bind(String),
|
|
Run(String),
|
|
/// A panic that unwound out of `MemberRunner::run_member`, caught at the
|
|
/// member boundary (#272) — the payload's best-effort message.
|
|
Panic(String),
|
|
}
|
|
|
|
/// Preflight + runtime refusals. Display-free and by-identifier: the
|
|
/// consumer phrases them (the DocFault/RefFault pattern).
|
|
#[derive(Debug)]
|
|
pub enum ExecFault {
|
|
/// The pipeline is not `std::sweep (std::gate)* (std::walk_forward)?
|
|
/// (std::monte_carlo)? (std::generalize)?`.
|
|
PipelineShape { detail: String },
|
|
/// A sweep/walk_forward selection metric outside the registry's rankable
|
|
/// roster.
|
|
UnrankableMetric { stage: usize, metric: String },
|
|
/// A gate predicate metric that is not a per-member scalar (e.g. an
|
|
/// annotation name such as `deflated_score`).
|
|
GateMetricNotPerMember { stage: usize, metric: String },
|
|
/// `plateau:*` selection in walk_forward (a gated survivor subset has no
|
|
/// grid lattice to smooth over).
|
|
PlateauInWalkForward { stage: usize },
|
|
/// sweep `deflate: true` with a non-argmax select rule.
|
|
DeflatePlateauConflict { stage: usize },
|
|
/// A selection-free sweep (no metric/select group) anywhere but the
|
|
/// pipeline's terminal stage: gate/walk_forward/monte_carlo run fine off
|
|
/// `survivors` (a selection-free sweep still produces the whole family),
|
|
/// but no winner/nominee is recorded — a stage that needs the cell's
|
|
/// final candidate (`generalize`, campaign-scope) would have nothing to
|
|
/// grade.
|
|
SelectionFreeSweepNotTerminal { stage: usize },
|
|
/// generalize needs >= 2 instruments in the campaign (static).
|
|
GeneralizeNeedsInstruments { available: usize },
|
|
/// generalize's metric must be an R metric (static — the intrinsic tier
|
|
/// accepts any vocabulary name; the shipped generalization() would only
|
|
/// refuse at runtime).
|
|
GeneralizeNonRMetric { metric: String },
|
|
/// monte_carlo with resamples == 0 or block_len == 0 (static, instead of
|
|
/// the engine's defined all-zero degenerate).
|
|
ZeroBootstrapParam { stage: usize, field: &'static str },
|
|
/// `WindowRoller` construction refusals at runtime.
|
|
Window { stage: usize, detail: String },
|
|
Member(MemberFault),
|
|
Registry(RegistryError),
|
|
}
|
|
|
|
/// The 14 per-member scalars a gate predicate may reference: the 3
|
|
/// `RunMetrics` scalars plus the 11 `RMetrics` scalars. The three
|
|
/// selection-annotation names (`deflated_score` / `overfit_probability` /
|
|
/// `neighbourhood_score`) are deliberately NOT here — they describe a
|
|
/// selection, not a member. Hand-copied roster (the third metric-roster site
|
|
/// beside aura-research's 17-name vocabulary and aura-registry's rankable
|
|
/// set, #190); drift fails safe: an unknown name is a preflight refusal,
|
|
/// never a wrong number.
|
|
pub const PER_MEMBER_METRICS: &[&str] = &[
|
|
"total_pips", "max_drawdown", "bias_sign_flips",
|
|
"expectancy_r", "n_trades", "win_rate", "avg_win_r", "avg_loss_r",
|
|
"profit_factor", "max_r_drawdown", "n_open_at_end", "sqn_normalized",
|
|
"sqn", "net_expectancy_r",
|
|
];
|
|
|
|
/// The registry's rankable roster (`resolve_metric`'s name set) — the metrics
|
|
/// a sweep/walk_forward stage may select on. Hand-copied (#190); drift fails
|
|
/// safe (a name the registry would refuse is refused here first, before any
|
|
/// member runs).
|
|
pub const RANKABLE_METRICS: &[&str] = &[
|
|
"total_pips", "max_drawdown", "bias_sign_flips",
|
|
"sqn", "sqn_normalized", "expectancy_r", "net_expectancy_r",
|
|
];
|
|
|
|
/// Deflation resample count — re-exported from `aura-registry`'s shared
|
|
/// definition (single-sourced, #199).
|
|
pub use aura_registry::DEFLATION_N_RESAMPLES;
|
|
/// Deflation moving-block length — re-exported from `aura-registry`'s shared
|
|
/// definition (single-sourced, #199).
|
|
pub use aura_registry::DEFLATION_BLOCK_LEN;
|
|
|
|
/// Resolve one of the 14 [`PER_MEMBER_METRICS`] against a member's report.
|
|
/// An R-metric name against `metrics.r == None` reads `None` (conservative
|
|
/// and deterministic: a gate predicate over `None` fails the member).
|
|
/// Annotation names and unknown names read `None`.
|
|
pub fn member_metric(report: &RunReport, name: &str) -> Option<f64> {
|
|
let m = &report.metrics;
|
|
match name {
|
|
"total_pips" => Some(m.total_pips),
|
|
"max_drawdown" => Some(m.max_drawdown),
|
|
"bias_sign_flips" => Some(m.bias_sign_flips as f64),
|
|
_ => {
|
|
let r = m.r.as_ref()?;
|
|
match name {
|
|
"expectancy_r" => Some(r.expectancy_r),
|
|
"n_trades" => Some(r.n_trades as f64),
|
|
"win_rate" => Some(r.win_rate),
|
|
"avg_win_r" => Some(r.avg_win_r),
|
|
"avg_loss_r" => Some(r.avg_loss_r),
|
|
"profit_factor" => Some(r.profit_factor),
|
|
"max_r_drawdown" => Some(r.max_r_drawdown),
|
|
"n_open_at_end" => Some(r.n_open_at_end as f64),
|
|
"sqn_normalized" => Some(r.sqn_normalized),
|
|
"sqn" => Some(r.sqn),
|
|
"net_expectancy_r" => Some(r.net_expectancy_r),
|
|
_ => None,
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Whether `value <cmp> threshold` holds — the gate's comparator arm.
|
|
fn predicate_holds(cmp: &Cmp, value: f64, threshold: f64) -> bool {
|
|
match cmp {
|
|
Cmp::Gt => value > threshold,
|
|
Cmp::Ge => value >= threshold,
|
|
Cmp::Lt => value < threshold,
|
|
Cmp::Le => value <= threshold,
|
|
}
|
|
}
|
|
|
|
/// Statically refuse everything refusable before any member runs (the F7
|
|
/// lesson applied forward): the v2 executable pipeline shape is exactly
|
|
/// `std::sweep (std::gate)* (std::walk_forward)? (std::monte_carlo)?
|
|
/// (std::generalize)?` — each suffix stage at most once, order fixed,
|
|
/// `std::generalize` strictly last; every sweep/walk_forward selection
|
|
/// metric is in [`RANKABLE_METRICS`]; every gate predicate metric is in
|
|
/// [`PER_MEMBER_METRICS`]; walk_forward must not select `plateau:*` (a gated
|
|
/// survivor subset has no grid lattice); sweep `deflate: true` composes only
|
|
/// with `argmax`; walk_forward lengths must fit `i64` (the roller's
|
|
/// Timestamp unit); monte_carlo `resamples` and `block_len` must be > 0
|
|
/// (static, instead of the engine's defined all-zero degenerate); generalize
|
|
/// needs >= 2 campaign instruments and an R selection metric (the registry's
|
|
/// `check_r_metric`). The campaign parameter carries the campaign-level
|
|
/// static checks (generalize's instrument arity).
|
|
pub fn preflight(process: &ProcessDoc, campaign: &CampaignDoc) -> Result<(), ExecFault> {
|
|
// Position of a stage in the fixed v2 shape; gates (rank 1) may repeat,
|
|
// every other rank appears at most once and ranks never decrease.
|
|
fn shape_rank(stage: &StageBlock) -> usize {
|
|
match stage {
|
|
StageBlock::Sweep { .. } => 0,
|
|
StageBlock::Grid => 0,
|
|
StageBlock::Gate { .. } => 1,
|
|
StageBlock::WalkForward { .. } => 2,
|
|
StageBlock::MonteCarlo { .. } => 3,
|
|
StageBlock::Generalize { .. } => 4,
|
|
}
|
|
}
|
|
fn block_id(stage: &StageBlock) -> &'static str {
|
|
match stage {
|
|
StageBlock::Sweep { .. } => "std::sweep",
|
|
StageBlock::Grid => "std::grid",
|
|
StageBlock::Gate { .. } => "std::gate",
|
|
StageBlock::WalkForward { .. } => "std::walk_forward",
|
|
StageBlock::MonteCarlo { .. } => "std::monte_carlo",
|
|
StageBlock::Generalize { .. } => "std::generalize",
|
|
}
|
|
}
|
|
// shape: `std::sweep (std::gate)* (std::walk_forward)? (std::monte_carlo)?
|
|
// (std::generalize)?` — a monotone rank walk over adjacent pairs captures
|
|
// every violation: a rank drop is an out-of-order stage (an annotator
|
|
// before a population stage, anything after std::generalize), an adjacent
|
|
// equal rank above the gate tier is a duplicate suffix stage.
|
|
if !matches!(
|
|
process.pipeline.first(),
|
|
Some(StageBlock::Sweep { .. } | StageBlock::Grid)
|
|
) {
|
|
return Err(ExecFault::PipelineShape {
|
|
detail: "the first stage must be std::sweep or std::grid".to_string(),
|
|
});
|
|
}
|
|
// #256 fork B: an enumerate-only first stage yields points, not executed
|
|
// members — every stage except std::walk_forward consumes member reports,
|
|
// so std::grid must be immediately followed by std::walk_forward (in
|
|
// particular it is never terminal).
|
|
if matches!(process.pipeline.first(), Some(StageBlock::Grid))
|
|
&& !matches!(process.pipeline.get(1), Some(StageBlock::WalkForward { .. }))
|
|
{
|
|
return Err(ExecFault::PipelineShape {
|
|
detail: "a std::grid first stage must be immediately followed by \
|
|
std::walk_forward (every other stage consumes executed \
|
|
member reports)"
|
|
.to_string(),
|
|
});
|
|
}
|
|
let mut prev = &process.pipeline[0];
|
|
for (i, stage) in process.pipeline.iter().enumerate().skip(1) {
|
|
if matches!(stage, StageBlock::Sweep { .. } | StageBlock::Grid) {
|
|
return Err(ExecFault::PipelineShape {
|
|
detail: format!("stage {i}: only the first stage may be {}", block_id(stage)),
|
|
});
|
|
}
|
|
let (rank, prev_rank) = (shape_rank(stage), shape_rank(prev));
|
|
if rank < prev_rank {
|
|
return Err(ExecFault::PipelineShape {
|
|
detail: format!(
|
|
"stage {i}: {} cannot follow {}",
|
|
block_id(stage),
|
|
block_id(prev)
|
|
),
|
|
});
|
|
}
|
|
if rank == prev_rank && rank > 1 {
|
|
return Err(ExecFault::PipelineShape {
|
|
detail: format!("stage {i}: {} may appear at most once", block_id(stage)),
|
|
});
|
|
}
|
|
prev = stage;
|
|
}
|
|
// per-stage slot rules (shape already established above).
|
|
for (i, stage) in process.pipeline.iter().enumerate() {
|
|
match stage {
|
|
StageBlock::Grid => {}
|
|
StageBlock::Sweep { selection } => match selection {
|
|
Some(sel) => {
|
|
if !RANKABLE_METRICS.contains(&sel.metric.as_str()) {
|
|
return Err(ExecFault::UnrankableMetric {
|
|
stage: i,
|
|
metric: sel.metric.clone(),
|
|
});
|
|
}
|
|
if sel.deflate && sel.select != SelectRule::Argmax {
|
|
return Err(ExecFault::DeflatePlateauConflict { stage: i });
|
|
}
|
|
}
|
|
None => {
|
|
if i + 1 != process.pipeline.len() {
|
|
return Err(ExecFault::SelectionFreeSweepNotTerminal { stage: i });
|
|
}
|
|
}
|
|
},
|
|
StageBlock::Gate { all } => {
|
|
for p in all {
|
|
if !PER_MEMBER_METRICS.contains(&p.metric.as_str()) {
|
|
return Err(ExecFault::GateMetricNotPerMember {
|
|
stage: i,
|
|
metric: p.metric.clone(),
|
|
});
|
|
}
|
|
}
|
|
}
|
|
StageBlock::WalkForward {
|
|
in_sample_ms,
|
|
out_of_sample_ms,
|
|
step_ms,
|
|
mode: _,
|
|
metric,
|
|
select,
|
|
} => {
|
|
if !RANKABLE_METRICS.contains(&metric.as_str()) {
|
|
return Err(ExecFault::UnrankableMetric { stage: i, metric: metric.clone() });
|
|
}
|
|
if matches!(select, SelectRule::PlateauMean | SelectRule::PlateauWorst)
|
|
&& process.pipeline[..i].iter().any(|s| matches!(s, StageBlock::Gate { .. }))
|
|
{
|
|
return Err(ExecFault::PlateauInWalkForward { stage: i });
|
|
}
|
|
for (field, len) in [
|
|
("in_sample_ms", *in_sample_ms),
|
|
("out_of_sample_ms", *out_of_sample_ms),
|
|
("step_ms", *step_ms),
|
|
] {
|
|
if i64::try_from(len).is_err() {
|
|
return Err(ExecFault::PipelineShape {
|
|
detail: format!("stage {i}: walk_forward {field} does not fit i64"),
|
|
});
|
|
}
|
|
}
|
|
}
|
|
StageBlock::MonteCarlo { resamples, block_len } => {
|
|
if *resamples == 0 {
|
|
return Err(ExecFault::ZeroBootstrapParam { stage: i, field: "resamples" });
|
|
}
|
|
if *block_len == 0 {
|
|
return Err(ExecFault::ZeroBootstrapParam { stage: i, field: "block_len" });
|
|
}
|
|
}
|
|
StageBlock::Generalize { metric } => {
|
|
let available = campaign.data.instruments.len();
|
|
if available < 2 {
|
|
return Err(ExecFault::GeneralizeNeedsInstruments { available });
|
|
}
|
|
if check_r_metric(metric).is_err() {
|
|
return Err(ExecFault::GeneralizeNonRMetric { metric: metric.clone() });
|
|
}
|
|
}
|
|
}
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use aura_engine::{RMetrics, RunManifest, RunMetrics, Timestamp};
|
|
use aura_research::SweepSelection;
|
|
|
|
/// A report with every per-member scalar planted to a distinct value
|
|
/// (1..=14 in `PER_MEMBER_METRICS` order), r block present.
|
|
fn report_with_r() -> RunReport {
|
|
RunReport {
|
|
manifest: RunManifest {
|
|
commit: "test".to_string(),
|
|
params: vec![],
|
|
defaults: vec![],
|
|
window: (Timestamp(0), Timestamp(1)),
|
|
seed: 0,
|
|
broker: "sim".to_string(),
|
|
selection: None,
|
|
instrument: None,
|
|
topology_hash: None,
|
|
project: None,
|
|
},
|
|
metrics: RunMetrics {
|
|
total_pips: 1.0,
|
|
max_drawdown: 2.0,
|
|
bias_sign_flips: 3,
|
|
r: Some(RMetrics {
|
|
expectancy_r: 4.0,
|
|
n_trades: 5,
|
|
win_rate: 6.0,
|
|
avg_win_r: 7.0,
|
|
avg_loss_r: 8.0,
|
|
profit_factor: 9.0,
|
|
max_r_drawdown: 10.0,
|
|
n_open_at_end: 11,
|
|
sqn_normalized: 12.0,
|
|
sqn: 13.0,
|
|
net_expectancy_r: 14.0,
|
|
conviction_terciles_r: [0.0; 3],
|
|
net_trade_rs: Vec::new(),
|
|
}),
|
|
},
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn member_metric_resolves_all_fourteen_names() {
|
|
let rep = report_with_r();
|
|
let expected: &[(&str, f64)] = &[
|
|
("total_pips", 1.0),
|
|
("max_drawdown", 2.0),
|
|
("bias_sign_flips", 3.0),
|
|
("expectancy_r", 4.0),
|
|
("n_trades", 5.0),
|
|
("win_rate", 6.0),
|
|
("avg_win_r", 7.0),
|
|
("avg_loss_r", 8.0),
|
|
("profit_factor", 9.0),
|
|
("max_r_drawdown", 10.0),
|
|
("n_open_at_end", 11.0),
|
|
("sqn_normalized", 12.0),
|
|
("sqn", 13.0),
|
|
("net_expectancy_r", 14.0),
|
|
];
|
|
assert_eq!(expected.len(), 14);
|
|
assert_eq!(PER_MEMBER_METRICS.len(), 14);
|
|
for (name, want) in expected {
|
|
assert!(
|
|
PER_MEMBER_METRICS.contains(name),
|
|
"{name} missing from PER_MEMBER_METRICS"
|
|
);
|
|
assert_eq!(member_metric(&rep, name), Some(*want), "metric {name}");
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn member_metric_r_names_none_without_r_block() {
|
|
let mut rep = report_with_r();
|
|
rep.metrics.r = None;
|
|
// the three run-level scalars still resolve...
|
|
assert_eq!(member_metric(&rep, "total_pips"), Some(1.0));
|
|
assert_eq!(member_metric(&rep, "max_drawdown"), Some(2.0));
|
|
assert_eq!(member_metric(&rep, "bias_sign_flips"), Some(3.0));
|
|
// ...and every R name reads None (a gate predicate over it fails).
|
|
for name in [
|
|
"expectancy_r", "n_trades", "win_rate", "avg_win_r", "avg_loss_r",
|
|
"profit_factor", "max_r_drawdown", "n_open_at_end", "sqn_normalized",
|
|
"sqn", "net_expectancy_r",
|
|
] {
|
|
assert_eq!(member_metric(&rep, name), None, "R metric {name} without r block");
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn member_metric_refuses_annotation_and_unknown_names() {
|
|
let rep = report_with_r();
|
|
for name in [
|
|
"deflated_score", "overfit_probability", "neighbourhood_score", "no_such_metric",
|
|
] {
|
|
assert_eq!(member_metric(&rep, name), None, "{name} must not resolve");
|
|
assert!(
|
|
!PER_MEMBER_METRICS.contains(&name),
|
|
"{name} must not be in PER_MEMBER_METRICS"
|
|
);
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn predicate_holds_covers_all_four_cmps() {
|
|
assert!(predicate_holds(&Cmp::Gt, 1.0, 0.0));
|
|
assert!(!predicate_holds(&Cmp::Gt, 0.0, 0.0));
|
|
assert!(predicate_holds(&Cmp::Ge, 0.0, 0.0));
|
|
assert!(!predicate_holds(&Cmp::Ge, -0.1, 0.0));
|
|
assert!(predicate_holds(&Cmp::Lt, -1.0, 0.0));
|
|
assert!(!predicate_holds(&Cmp::Lt, 0.0, 0.0));
|
|
assert!(predicate_holds(&Cmp::Le, 0.0, 0.0));
|
|
assert!(!predicate_holds(&Cmp::Le, 0.1, 0.0));
|
|
}
|
|
|
|
// -----------------------------------------------------------------
|
|
// preflight
|
|
// -----------------------------------------------------------------
|
|
|
|
use aura_core::ScalarKind;
|
|
use aura_research::{
|
|
DataSection, DocKind, DocRef, Predicate, Presentation, ProcessRef, StrategyEntry, WfMode,
|
|
Window,
|
|
};
|
|
|
|
fn process_of(pipeline: Vec<StageBlock>) -> ProcessDoc {
|
|
ProcessDoc {
|
|
format_version: 1,
|
|
kind: DocKind::Process,
|
|
name: "p".to_string(),
|
|
description: None,
|
|
pipeline,
|
|
}
|
|
}
|
|
|
|
/// A minimal intrinsically-valid campaign; preflight's campaign parameter
|
|
/// carries no v1 rules, so one fixture serves every test.
|
|
fn campaign() -> CampaignDoc {
|
|
CampaignDoc {
|
|
format_version: 1,
|
|
kind: DocKind::Campaign,
|
|
name: "c".to_string(),
|
|
description: None,
|
|
data: DataSection {
|
|
instruments: vec!["EURUSD".to_string()],
|
|
windows: vec![Window { from_ms: 0, to_ms: 10_000 }],
|
|
bindings: BTreeMap::new(),
|
|
},
|
|
risk: vec![],
|
|
cost: vec![],
|
|
strategies: vec![StrategyEntry {
|
|
r#ref: DocRef::ContentId("0".repeat(64)),
|
|
axes: BTreeMap::from([(
|
|
"len".to_string(),
|
|
Axis {
|
|
kind: ScalarKind::I64,
|
|
values: vec![Scalar::i64(2), Scalar::i64(3)],
|
|
},
|
|
)]),
|
|
}],
|
|
process: ProcessRef { r#ref: DocRef::ContentId("1".repeat(64)) },
|
|
seed: 7,
|
|
presentation: Presentation { persist_taps: vec![], emit: vec![] },
|
|
}
|
|
}
|
|
|
|
fn sweep_stage(metric: &str, select: SelectRule, deflate: bool) -> StageBlock {
|
|
StageBlock::Sweep {
|
|
selection: Some(SweepSelection { metric: metric.to_string(), select, deflate }),
|
|
}
|
|
}
|
|
|
|
fn gate_stage(metric: &str) -> StageBlock {
|
|
StageBlock::Gate {
|
|
all: vec![Predicate { metric: metric.to_string(), cmp: Cmp::Gt, value: 0.0 }],
|
|
}
|
|
}
|
|
|
|
fn wf_stage(metric: &str, select: SelectRule) -> StageBlock {
|
|
StageBlock::WalkForward {
|
|
in_sample_ms: 4000,
|
|
out_of_sample_ms: 1000,
|
|
step_ms: 1000,
|
|
mode: WfMode::Rolling,
|
|
metric: metric.to_string(),
|
|
select,
|
|
}
|
|
}
|
|
|
|
fn mc_stage(resamples: u32, block_len: u32) -> StageBlock {
|
|
StageBlock::MonteCarlo { resamples, block_len }
|
|
}
|
|
|
|
fn generalize_stage(metric: &str) -> StageBlock {
|
|
StageBlock::Generalize { metric: metric.to_string() }
|
|
}
|
|
|
|
/// [`campaign`] with a second instrument — generalize's static arity
|
|
/// guard needs >= 2 to pass.
|
|
fn campaign_two_instruments() -> CampaignDoc {
|
|
let mut c = campaign();
|
|
c.data.instruments.push("GER40".to_string());
|
|
c
|
|
}
|
|
|
|
#[test]
|
|
fn preflight_accepts_the_v2_shapes() {
|
|
let c = campaign();
|
|
let c2 = campaign_two_instruments();
|
|
// the full v2 shape: sweep (gate)* (walk_forward)? (monte_carlo)?
|
|
// (generalize)?
|
|
let full = process_of(vec![
|
|
sweep_stage("sqn_normalized", SelectRule::Argmax, true),
|
|
gate_stage("net_expectancy_r"),
|
|
wf_stage("sqn_normalized", SelectRule::Argmax),
|
|
mc_stage(1000, 5),
|
|
generalize_stage("net_expectancy_r"),
|
|
]);
|
|
assert!(preflight(&full, &c2).is_ok());
|
|
// degenerate accepted shapes: bare sweep (plateau select without
|
|
// deflate is legal on a sweep); sweep + gates without walk_forward.
|
|
let bare = process_of(vec![sweep_stage("total_pips", SelectRule::PlateauMean, false)]);
|
|
assert!(preflight(&bare, &c).is_ok());
|
|
let gated = process_of(vec![
|
|
sweep_stage("expectancy_r", SelectRule::Argmax, false),
|
|
gate_stage("n_trades"),
|
|
gate_stage("win_rate"),
|
|
]);
|
|
assert!(preflight(&gated, &c).is_ok());
|
|
// suffix stages compose independently: mc without wf (single
|
|
// instrument fine), generalize without mc (needs 2 instruments),
|
|
// wf + mc without generalize.
|
|
let mc_only = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
mc_stage(1000, 5),
|
|
]);
|
|
assert!(preflight(&mc_only, &c).is_ok());
|
|
let gen_only = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
generalize_stage("expectancy_r"),
|
|
]);
|
|
assert!(preflight(&gen_only, &c2).is_ok());
|
|
let wf_mc = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
wf_stage("sqn", SelectRule::Argmax),
|
|
mc_stage(100, 3),
|
|
]);
|
|
assert!(preflight(&wf_mc, &c).is_ok());
|
|
}
|
|
|
|
/// v2 shape: an annotator may not precede a population stage, and
|
|
/// std::generalize is strictly last. `[sweep, mc, walk_forward]` is
|
|
/// intrinsically valid at the document tier yet refused here.
|
|
#[test]
|
|
fn preflight_refuses_annotator_order_violations() {
|
|
let c2 = campaign_two_instruments();
|
|
let cases: Vec<(Vec<StageBlock>, &str)> = vec![
|
|
(
|
|
vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
mc_stage(100, 5),
|
|
wf_stage("sqn", SelectRule::Argmax),
|
|
],
|
|
"stage 2: std::walk_forward cannot follow std::monte_carlo",
|
|
),
|
|
(
|
|
vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
mc_stage(100, 5),
|
|
gate_stage("expectancy_r"),
|
|
],
|
|
"stage 2: std::gate cannot follow std::monte_carlo",
|
|
),
|
|
(
|
|
vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
generalize_stage("expectancy_r"),
|
|
mc_stage(100, 5),
|
|
],
|
|
"stage 2: std::monte_carlo cannot follow std::generalize",
|
|
),
|
|
(
|
|
vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
wf_stage("sqn", SelectRule::Argmax),
|
|
gate_stage("expectancy_r"),
|
|
],
|
|
"stage 2: std::gate cannot follow std::walk_forward",
|
|
),
|
|
];
|
|
for (pipeline, want) in cases {
|
|
match preflight(&process_of(pipeline), &c2) {
|
|
Err(ExecFault::PipelineShape { detail }) => assert_eq!(detail, want),
|
|
other => panic!("expected PipelineShape({want}), got {other:?}"),
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Each v2 suffix stage appears at most once.
|
|
#[test]
|
|
fn preflight_refuses_duplicate_suffix_stages() {
|
|
let c2 = campaign_two_instruments();
|
|
let cases: Vec<(Vec<StageBlock>, &str)> = vec![
|
|
(
|
|
vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
wf_stage("sqn", SelectRule::Argmax),
|
|
wf_stage("sqn", SelectRule::Argmax),
|
|
],
|
|
"stage 2: std::walk_forward may appear at most once",
|
|
),
|
|
(
|
|
vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
mc_stage(100, 5),
|
|
mc_stage(100, 5),
|
|
],
|
|
"stage 2: std::monte_carlo may appear at most once",
|
|
),
|
|
(
|
|
vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
generalize_stage("expectancy_r"),
|
|
generalize_stage("expectancy_r"),
|
|
],
|
|
"stage 2: std::generalize may appear at most once",
|
|
),
|
|
];
|
|
for (pipeline, want) in cases {
|
|
match preflight(&process_of(pipeline), &c2) {
|
|
Err(ExecFault::PipelineShape { detail }) => assert_eq!(detail, want),
|
|
other => panic!("expected PipelineShape({want}), got {other:?}"),
|
|
}
|
|
}
|
|
}
|
|
|
|
/// generalize's static arity guard: the campaign must carry >= 2
|
|
/// instruments (the shipped generalization() would only refuse at
|
|
/// runtime, after members ran).
|
|
#[test]
|
|
fn preflight_refuses_single_instrument_generalize() {
|
|
let c = campaign(); // one instrument
|
|
let p = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
generalize_stage("expectancy_r"),
|
|
]);
|
|
assert!(matches!(
|
|
preflight(&p, &c),
|
|
Err(ExecFault::GeneralizeNeedsInstruments { available: 1 })
|
|
));
|
|
}
|
|
|
|
/// generalize's metric guard: check_r_metric refuses pip metrics AND
|
|
/// unknown names — both surface as GeneralizeNonRMetric.
|
|
#[test]
|
|
fn preflight_refuses_non_r_generalize_metric() {
|
|
let c2 = campaign_two_instruments();
|
|
for bad in ["total_pips", "no_such_metric"] {
|
|
let p = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
generalize_stage(bad),
|
|
]);
|
|
assert!(matches!(
|
|
preflight(&p, &c2),
|
|
Err(ExecFault::GeneralizeNonRMetric { metric }) if metric == bad
|
|
));
|
|
}
|
|
}
|
|
|
|
/// monte_carlo zero params are static refusals (instead of the engine's
|
|
/// defined all-zero degenerate at runtime), naming the offending field.
|
|
#[test]
|
|
fn preflight_refuses_zero_bootstrap_params() {
|
|
let c = campaign();
|
|
let zero_resamples = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
mc_stage(0, 5),
|
|
]);
|
|
assert!(matches!(
|
|
preflight(&zero_resamples, &c),
|
|
Err(ExecFault::ZeroBootstrapParam { stage: 1, field: "resamples" })
|
|
));
|
|
let zero_block = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
mc_stage(100, 0),
|
|
]);
|
|
assert!(matches!(
|
|
preflight(&zero_block, &c),
|
|
Err(ExecFault::ZeroBootstrapParam { stage: 1, field: "block_len" })
|
|
));
|
|
}
|
|
|
|
/// A selection-free sweep is permitted as the terminal (and here, only)
|
|
/// stage of a process: the family itself is the result, no winner needed.
|
|
#[test]
|
|
fn preflight_accepts_a_terminal_selection_free_sweep() {
|
|
let c = campaign();
|
|
let process = process_of(vec![StageBlock::Sweep { selection: None }]);
|
|
assert!(preflight(&process, &c).is_ok());
|
|
}
|
|
|
|
/// #256 fork B: the enumerate-only leading stage is accepted exactly in
|
|
/// the dissolved walkforward/mc shapes — first, immediately before
|
|
/// std::walk_forward.
|
|
#[test]
|
|
fn preflight_accepts_a_grid_first_stage_followed_by_walk_forward() {
|
|
let c = campaign();
|
|
let wf_shape =
|
|
process_of(vec![StageBlock::Grid, wf_stage("sqn_normalized", SelectRule::Argmax)]);
|
|
assert!(preflight(&wf_shape, &c).is_ok());
|
|
let mc_shape = process_of(vec![
|
|
StageBlock::Grid,
|
|
wf_stage("sqn_normalized", SelectRule::Argmax),
|
|
mc_stage(1000, 5),
|
|
]);
|
|
assert!(preflight(&mc_shape, &c).is_ok());
|
|
}
|
|
|
|
/// #256 fork B: every other placement is refused — grid terminal, grid
|
|
/// before a report-consuming stage (gate / monte_carlo / generalize),
|
|
/// grid anywhere but first. Every refusal is a PipelineShape prose fault.
|
|
#[test]
|
|
fn preflight_refuses_grid_placement_violations() {
|
|
let c = campaign();
|
|
let c2 = campaign_two_instruments();
|
|
let adjacency = "immediately followed by std::walk_forward";
|
|
let cases: Vec<(Vec<StageBlock>, &str, &CampaignDoc)> = vec![
|
|
(vec![StageBlock::Grid], adjacency, &c),
|
|
(
|
|
vec![
|
|
StageBlock::Grid,
|
|
gate_stage("net_expectancy_r"),
|
|
wf_stage("sqn_normalized", SelectRule::Argmax),
|
|
],
|
|
adjacency,
|
|
&c,
|
|
),
|
|
(vec![StageBlock::Grid, mc_stage(1000, 5)], adjacency, &c),
|
|
(
|
|
vec![StageBlock::Grid, generalize_stage("net_expectancy_r")],
|
|
adjacency,
|
|
&c2,
|
|
),
|
|
(
|
|
vec![
|
|
sweep_stage("sqn_normalized", SelectRule::Argmax, false),
|
|
StageBlock::Grid,
|
|
],
|
|
"only the first stage may be std::grid",
|
|
&c,
|
|
),
|
|
];
|
|
for (pipeline, needle, camp) in cases {
|
|
let err =
|
|
preflight(&process_of(pipeline), camp).expect_err("placement violation");
|
|
let ExecFault::PipelineShape { detail } = &err else {
|
|
panic!("expected PipelineShape, got {err:?}");
|
|
};
|
|
assert!(detail.contains(needle), "detail {detail:?} misses {needle:?}");
|
|
}
|
|
}
|
|
|
|
/// A selection-free sweep followed by any other stage is refused: a
|
|
/// downstream stage would have no selection/nominee to consume.
|
|
#[test]
|
|
fn preflight_refuses_a_non_terminal_selection_free_sweep() {
|
|
let c = campaign();
|
|
let process = process_of(vec![
|
|
StageBlock::Sweep { selection: None },
|
|
gate_stage("sqn"),
|
|
]);
|
|
match preflight(&process, &c) {
|
|
Err(ExecFault::SelectionFreeSweepNotTerminal { stage: 0 }) => {}
|
|
other => panic!("expected SelectionFreeSweepNotTerminal, got {other:?}"),
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn preflight_refuses_non_sweep_first_and_double_sweep() {
|
|
let c = campaign();
|
|
let gate_first = process_of(vec![gate_stage("expectancy_r")]);
|
|
assert!(matches!(preflight(&gate_first, &c), Err(ExecFault::PipelineShape { .. })));
|
|
let empty = process_of(vec![]);
|
|
assert!(matches!(preflight(&empty, &c), Err(ExecFault::PipelineShape { .. })));
|
|
let double = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
]);
|
|
assert!(matches!(preflight(&double, &c), Err(ExecFault::PipelineShape { .. })));
|
|
// a gate after walk_forward is still a shape refusal (v2 relaxes
|
|
// wf-final only for the annotator suffix: mc/generalize may follow).
|
|
let wf_mid = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
wf_stage("sqn", SelectRule::Argmax),
|
|
gate_stage("expectancy_r"),
|
|
]);
|
|
assert!(matches!(preflight(&wf_mid, &c), Err(ExecFault::PipelineShape { .. })));
|
|
}
|
|
|
|
#[test]
|
|
fn preflight_refuses_unrankable_select_metric() {
|
|
let c = campaign();
|
|
// win_rate is a per-member scalar but NOT in the registry's rankable roster.
|
|
let sweep_bad = process_of(vec![sweep_stage("win_rate", SelectRule::Argmax, false)]);
|
|
assert!(matches!(
|
|
preflight(&sweep_bad, &c),
|
|
Err(ExecFault::UnrankableMetric { stage: 0, metric }) if metric == "win_rate"
|
|
));
|
|
let wf_bad = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
wf_stage("profit_factor", SelectRule::Argmax),
|
|
]);
|
|
assert!(matches!(
|
|
preflight(&wf_bad, &c),
|
|
Err(ExecFault::UnrankableMetric { stage: 1, metric }) if metric == "profit_factor"
|
|
));
|
|
}
|
|
|
|
#[test]
|
|
fn preflight_refuses_annotation_gate_metric() {
|
|
let c = campaign();
|
|
let p = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
gate_stage("deflated_score"),
|
|
]);
|
|
assert!(matches!(
|
|
preflight(&p, &c),
|
|
Err(ExecFault::GateMetricNotPerMember { stage: 1, metric }) if metric == "deflated_score"
|
|
));
|
|
}
|
|
|
|
/// Plateau select in a walk_forward is only refused when a gate
|
|
/// precedes it: a gate-free sweep -> wf keeps the survivors as the full
|
|
/// grid (the parameter lattice is intact, so plateau is meaningful);
|
|
/// a gate stage in between filters survivors below the full grid,
|
|
/// breaking the lattice, so plateau is refused there.
|
|
#[test]
|
|
fn preflight_permits_plateau_in_gate_free_walk_forward_refuses_after_a_gate() {
|
|
let c = campaign();
|
|
for select in [SelectRule::PlateauMean, SelectRule::PlateauWorst] {
|
|
// gate-free: sweep -> wf(plateau) — survivors are the full grid,
|
|
// the lattice is intact, so plateau is permitted.
|
|
let ok = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
wf_stage("sqn", select),
|
|
]);
|
|
assert!(preflight(&ok, &c).is_ok(), "gate-free plateau wf must pass preflight");
|
|
|
|
// gate-preceded: sweep -> gate -> wf(plateau) — the gate breaks
|
|
// the lattice, so plateau is refused at stage 2.
|
|
let gated = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
gate_stage("sqn"),
|
|
wf_stage("sqn", select),
|
|
]);
|
|
assert!(matches!(
|
|
preflight(&gated, &c),
|
|
Err(ExecFault::PlateauInWalkForward { stage: 2 })
|
|
));
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn preflight_refuses_wf_length_exceeding_i64() {
|
|
let c = campaign();
|
|
let p = process_of(vec![
|
|
sweep_stage("sqn", SelectRule::Argmax, false),
|
|
StageBlock::WalkForward {
|
|
in_sample_ms: u64::MAX,
|
|
out_of_sample_ms: 1000,
|
|
step_ms: 1000,
|
|
mode: WfMode::Rolling,
|
|
metric: "sqn".to_string(),
|
|
select: SelectRule::Argmax,
|
|
},
|
|
]);
|
|
assert!(matches!(
|
|
preflight(&p, &c),
|
|
Err(ExecFault::PipelineShape { detail })
|
|
if detail.contains("in_sample_ms does not fit i64")
|
|
));
|
|
}
|
|
|
|
#[test]
|
|
fn preflight_refuses_deflate_with_plateau() {
|
|
let c = campaign();
|
|
for select in [SelectRule::PlateauMean, SelectRule::PlateauWorst] {
|
|
let p = process_of(vec![sweep_stage("sqn", select, true)]);
|
|
assert!(matches!(
|
|
preflight(&p, &c),
|
|
Err(ExecFault::DeflatePlateauConflict { stage: 0 })
|
|
));
|
|
}
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod wf_tests {
|
|
use std::collections::BTreeMap;
|
|
use std::sync::Mutex;
|
|
|
|
use aura_analysis::SelectionMode;
|
|
use aura_core::{Scalar, ScalarKind};
|
|
use aura_engine::{RollMode, RunManifest, RunMetrics, RunReport, Timestamp, WindowRoller};
|
|
use aura_registry::{FamilyKind, Registry};
|
|
use aura_research::{
|
|
Axis, CampaignDoc, Cmp, DataSection, DocKind, DocRef, Predicate, Presentation,
|
|
ProcessDoc, ProcessRef, SelectRule, StageBlock, StrategyEntry, SweepSelection, WfMode,
|
|
Window,
|
|
};
|
|
|
|
use super::{execute, CampaignOutcome, CellSpec, ExecFault, MemberFault, MemberRunner};
|
|
|
|
/// A per-test registry in a fresh temp dir (a Registry's family store
|
|
/// isolates per DIRECTORY, not per filename — see Registry::open).
|
|
fn wf_registry(name: &str) -> Registry {
|
|
// fixed, tag-keyed, pid-free name under the build-tree tmp anchor (#258): the
|
|
// pre-create wipe below then genuinely reclaims the previous run.
|
|
let dir = std::path::Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../target/tmp"))
|
|
.join(format!("aura-campaign-wf-{name}"));
|
|
let _ = std::fs::remove_dir_all(&dir);
|
|
std::fs::create_dir_all(&dir).expect("temp dir");
|
|
Registry::open(dir.join("runs.jsonl"))
|
|
}
|
|
|
|
/// One strategy x one instrument x one window, over a single I64 axis
|
|
/// "len" with values [1, 2, 3, 4] (the fake runner scores total_pips ==
|
|
/// the len value, so ranking and gating are fully determined).
|
|
fn wf_campaign(window: (i64, i64)) -> CampaignDoc {
|
|
CampaignDoc {
|
|
format_version: 1,
|
|
kind: DocKind::Campaign,
|
|
name: "wf-test".to_string(),
|
|
description: None,
|
|
data: DataSection {
|
|
instruments: vec!["SYNTH".to_string()],
|
|
windows: vec![Window { from_ms: window.0, to_ms: window.1 }],
|
|
bindings: BTreeMap::new(),
|
|
},
|
|
risk: vec![],
|
|
cost: vec![],
|
|
strategies: vec![StrategyEntry {
|
|
r#ref: DocRef::ContentId("c".repeat(64)),
|
|
axes: BTreeMap::from([(
|
|
"len".to_string(),
|
|
Axis {
|
|
kind: ScalarKind::I64,
|
|
values: vec![
|
|
Scalar::i64(1),
|
|
Scalar::i64(2),
|
|
Scalar::i64(3),
|
|
Scalar::i64(4),
|
|
],
|
|
},
|
|
)]),
|
|
}],
|
|
process: ProcessRef { r#ref: DocRef::ContentId("d".repeat(64)) },
|
|
seed: 7,
|
|
presentation: Presentation { persist_taps: vec![], emit: vec![] },
|
|
}
|
|
}
|
|
|
|
/// sweep -> (optional total_pips-gt gate) -> walk_forward, both ranked
|
|
/// stages on total_pips / argmax, rolling mode.
|
|
fn wf_process(gate_gt: Option<f64>, is_ms: u64, oos_ms: u64, step_ms: u64) -> ProcessDoc {
|
|
let mut pipeline = vec![StageBlock::Sweep {
|
|
selection: Some(SweepSelection {
|
|
metric: "total_pips".to_string(),
|
|
select: SelectRule::Argmax,
|
|
deflate: false,
|
|
}),
|
|
}];
|
|
if let Some(threshold) = gate_gt {
|
|
pipeline.push(StageBlock::Gate {
|
|
all: vec![Predicate {
|
|
metric: "total_pips".to_string(),
|
|
cmp: Cmp::Gt,
|
|
value: threshold,
|
|
}],
|
|
});
|
|
}
|
|
pipeline.push(StageBlock::WalkForward {
|
|
in_sample_ms: is_ms,
|
|
out_of_sample_ms: oos_ms,
|
|
step_ms,
|
|
mode: WfMode::Rolling,
|
|
metric: "total_pips".to_string(),
|
|
select: SelectRule::Argmax,
|
|
});
|
|
ProcessDoc {
|
|
format_version: 1,
|
|
kind: DocKind::Process,
|
|
name: "wf-proc".to_string(),
|
|
description: None,
|
|
pipeline,
|
|
}
|
|
}
|
|
|
|
/// One (window bounds, params) -> fault mapping; a type alias rather than
|
|
/// a struct-level `#[allow]` since `WfFakeRunner::faulty`'s own parameter
|
|
/// re-triggers the type-complexity lint independently of the field.
|
|
type WfFault = ((i64, i64), Vec<(String, Scalar)>, MemberFault);
|
|
|
|
/// Deterministic fake member runner keyed on (window bounds, params):
|
|
/// total_pips == the summed numeric param values (here: the single "len"
|
|
/// value), the report's manifest.window echoes the window it was run over,
|
|
/// and every call is logged for the IS-population assertions. `faults`
|
|
/// (empty for every plain `wf_*` fixture) lets the fault-attribution
|
|
/// tests below make one specific (window, params) call fail instead of
|
|
/// planting a report — the only way to reach the walk-forward stage's
|
|
/// IS-sweep / OOS-run / per-window `min_by_key` capture paths, which a
|
|
/// runner that always succeeds cannot exercise.
|
|
#[allow(clippy::type_complexity)]
|
|
struct WfFakeRunner {
|
|
log: Mutex<Vec<((i64, i64), Vec<(String, Scalar)>)>>,
|
|
faults: Vec<WfFault>,
|
|
}
|
|
|
|
impl WfFakeRunner {
|
|
fn new() -> Self {
|
|
WfFakeRunner { log: Mutex::new(Vec::new()), faults: Vec::new() }
|
|
}
|
|
|
|
fn faulty(faults: Vec<WfFault>) -> Self {
|
|
WfFakeRunner { log: Mutex::new(Vec::new()), faults }
|
|
}
|
|
}
|
|
|
|
impl MemberRunner for WfFakeRunner {
|
|
fn run_member(
|
|
&self,
|
|
_cell: &CellSpec,
|
|
params: &[(String, Scalar)],
|
|
window_ms: (i64, i64),
|
|
) -> Result<RunReport, MemberFault> {
|
|
self.log.lock().unwrap().push((window_ms, params.to_vec()));
|
|
if let Some((_, _, fault)) =
|
|
self.faults.iter().find(|(w, p, _)| *w == window_ms && p == params)
|
|
{
|
|
return Err(fault.clone());
|
|
}
|
|
let total: f64 = params
|
|
.iter()
|
|
.map(|(_, s)| match s {
|
|
Scalar::I64(v) => *v as f64,
|
|
Scalar::F64(v) => *v,
|
|
_ => 0.0,
|
|
})
|
|
.sum();
|
|
Ok(RunReport {
|
|
manifest: RunManifest {
|
|
commit: "wf-fake".to_string(),
|
|
params: params.to_vec(),
|
|
defaults: Vec::new(),
|
|
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
|
|
seed: 0,
|
|
broker: "fake".to_string(),
|
|
selection: None,
|
|
instrument: None,
|
|
topology_hash: None,
|
|
project: None,
|
|
},
|
|
metrics: RunMetrics {
|
|
total_pips: total,
|
|
max_drawdown: 0.0,
|
|
bias_sign_flips: 0,
|
|
r: None,
|
|
},
|
|
})
|
|
}
|
|
}
|
|
|
|
fn run_wf(
|
|
campaign: &CampaignDoc,
|
|
process: &ProcessDoc,
|
|
runner: &WfFakeRunner,
|
|
registry: &Registry,
|
|
) -> Result<CampaignOutcome, ExecFault> {
|
|
let strategies = vec![("s".repeat(64), "{}".to_string())];
|
|
let campaign_id = "e".repeat(64);
|
|
execute(&campaign_id, campaign, process, &strategies, runner, registry)
|
|
}
|
|
|
|
#[test]
|
|
fn wf_rolls_the_declared_window_in_ms() {
|
|
let registry = wf_registry("rolls");
|
|
let campaign = wf_campaign((0, 99));
|
|
let process = wf_process(None, 40, 20, 20);
|
|
let runner = WfFakeRunner::new();
|
|
let outcome = run_wf(&campaign, &process, &runner, ®istry).expect("wf executes");
|
|
|
|
// The oracle is the roller's own math over the same config.
|
|
let expected: Vec<_> =
|
|
WindowRoller::new((Timestamp(0), Timestamp(99)), 40, 20, 20, RollMode::Rolling)
|
|
.expect("valid roll")
|
|
.collect();
|
|
assert_eq!(expected.len(), 3, "fixture sanity: 3 windows over 0..=99");
|
|
|
|
let fam = outcome.cells[0]
|
|
.families
|
|
.iter()
|
|
.find(|f| f.block == "std::walk_forward")
|
|
.expect("wf family present");
|
|
assert_eq!(fam.reports.len(), expected.len());
|
|
for (report, bounds) in fam.reports.iter().zip(&expected) {
|
|
assert_eq!(report.manifest.window, (bounds.oos.0, bounds.oos.1));
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn wf_searches_only_the_survivor_points() {
|
|
let registry = wf_registry("survivors");
|
|
let campaign = wf_campaign((0, 99));
|
|
// total_pips == len; gate total_pips > 2.5 keeps len 3 and len 4 of [1,2,3,4]
|
|
let process = wf_process(Some(2.5), 40, 20, 20);
|
|
let runner = WfFakeRunner::new();
|
|
run_wf(&campaign, &process, &runner, ®istry).expect("wf executes");
|
|
|
|
let is_bounds: Vec<(i64, i64)> =
|
|
WindowRoller::new((Timestamp(0), Timestamp(99)), 40, 20, 20, RollMode::Rolling)
|
|
.expect("valid roll")
|
|
.map(|w| (w.is.0 .0, w.is.1 .0))
|
|
.collect();
|
|
let log = runner.log.lock().unwrap();
|
|
let mut is_total = 0;
|
|
let mut seen3 = 0;
|
|
let mut seen4 = 0;
|
|
for (window, params) in log.iter() {
|
|
if !is_bounds.contains(window) {
|
|
continue;
|
|
}
|
|
is_total += 1;
|
|
match params[0].1 {
|
|
Scalar::I64(3) => seen3 += 1,
|
|
Scalar::I64(4) => seen4 += 1,
|
|
other => panic!("gated-out point ran in an IS window: {other:?}"),
|
|
}
|
|
}
|
|
// 3 windows x exactly the 2 surviving points
|
|
assert_eq!(is_total, 6);
|
|
assert_eq!(seen3, 3);
|
|
assert_eq!(seen4, 3);
|
|
}
|
|
|
|
#[test]
|
|
fn wf_stamps_selection_on_oos_members() {
|
|
let registry = wf_registry("selection");
|
|
let campaign = wf_campaign((0, 99));
|
|
let process = wf_process(None, 40, 20, 20);
|
|
let runner = WfFakeRunner::new();
|
|
let outcome = run_wf(&campaign, &process, &runner, ®istry).expect("wf executes");
|
|
|
|
let fam = outcome.cells[0]
|
|
.families
|
|
.iter()
|
|
.find(|f| f.block == "std::walk_forward")
|
|
.expect("wf family present");
|
|
assert!(!fam.reports.is_empty());
|
|
for report in &fam.reports {
|
|
let sel = report
|
|
.manifest
|
|
.selection
|
|
.as_ref()
|
|
.expect("every OOS member carries its IS selection");
|
|
assert_eq!(sel.selection_metric, "total_pips");
|
|
assert_eq!(sel.seed, Some(7), "deflation is seeded from campaign.seed");
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn wf_roller_refusal_maps_to_window_fault() {
|
|
let registry = wf_registry("window-fault");
|
|
// 0..=49 cannot fit is 40 + oos 20 (window 0 needs 59) -> the roller
|
|
// refuses at runtime (zero lengths are already doc-tier faults).
|
|
let campaign = wf_campaign((0, 49));
|
|
let process = wf_process(None, 40, 20, 20);
|
|
let runner = WfFakeRunner::new();
|
|
let result = run_wf(&campaign, &process, &runner, ®istry);
|
|
let Err(err) = result else { panic!("span too short for one window must refuse") };
|
|
match err {
|
|
ExecFault::Window { stage, detail } => {
|
|
assert_eq!(stage, 1, "walk_forward is pipeline stage 1 here");
|
|
assert!(
|
|
detail.contains("in_sample_ms + out_of_sample_ms = 60 ms")
|
|
&& detail.contains("(50 ms)"),
|
|
"detail phrases the refusal in doc-unit ms prose: {detail}"
|
|
);
|
|
assert!(
|
|
!detail.contains("SpanTooShort") && !detail.contains("Timestamp("),
|
|
"no Debug form leaks through the detail seam: {detail}"
|
|
);
|
|
}
|
|
other => panic!("expected ExecFault::Window, got {other:?}"),
|
|
}
|
|
}
|
|
|
|
/// #272: a member fault raised INSIDE one window's in-sample sweep is
|
|
/// contained as that fold's `WindowFault` — the surviving windows still
|
|
/// run, pool into the walk_forward family, and the campaign run record
|
|
/// persists. Faulting w1's IS run out of 3 windows (w0, w1, w2) leaves
|
|
/// exactly 2 surviving OOS reports and one recorded window fault at
|
|
/// ordinal 1.
|
|
#[test]
|
|
fn wf_one_faulted_fold_is_contained_survivors_pool() {
|
|
let registry = wf_registry("is-fault");
|
|
let campaign = wf_campaign((0, 99));
|
|
let process = wf_process(None, 40, 20, 20);
|
|
// IS bounds: w0 (0,39), w1 (20,59), w2 (40,79) — fault only w1's IS
|
|
// run; w0 and w2 survive and pool.
|
|
let runner = WfFakeRunner::faulty(vec![(
|
|
(20, 59),
|
|
vec![("len".to_string(), Scalar::i64(4))],
|
|
MemberFault::Run("is-w1".to_string()),
|
|
)]);
|
|
let outcome = run_wf(&campaign, &process, &runner, ®istry)
|
|
.expect("a faulted fold is contained, not a stage abort");
|
|
|
|
let fam = outcome.cells[0]
|
|
.families
|
|
.iter()
|
|
.find(|f| f.block == "std::walk_forward")
|
|
.expect("wf family present despite the faulted fold");
|
|
assert_eq!(fam.reports.len(), 2, "only the 2 surviving windows pool");
|
|
|
|
let cell = &outcome.record.cells[0];
|
|
let wf_stage = cell
|
|
.stages
|
|
.iter()
|
|
.find(|s| s.block == "std::walk_forward")
|
|
.expect("wf stage realized");
|
|
assert_eq!(wf_stage.window_faults.len(), 1);
|
|
assert_eq!(wf_stage.window_faults[0].window_ordinal, 1);
|
|
assert_eq!(wf_stage.window_faults[0].kind, aura_registry::CellFaultKind::Run);
|
|
assert!(cell.fault.is_none(), "a partial-fold stage does not fail the cell");
|
|
|
|
assert_eq!(registry.load_campaign_runs().expect("load campaign runs").len(), 1);
|
|
let members = registry.load_family_members().expect("load members");
|
|
assert!(members.iter().any(|m| m.kind == FamilyKind::WalkForward));
|
|
}
|
|
|
|
/// #272: every fold faulting (both the IS sweep AND the OOS run, across
|
|
/// all 3 windows) leaves zero surviving OOS reports — the whole cell
|
|
/// fails at the wf stage, with no WalkForward family persisted, while the
|
|
/// campaign run record still records the failed cell.
|
|
#[test]
|
|
fn wf_all_folds_faulted_fails_the_cell() {
|
|
let registry = wf_registry("all-folds-fault");
|
|
let campaign = wf_campaign((0, 99));
|
|
let process = wf_process(None, 40, 20, 20);
|
|
// IS bounds: w0 (0,39), w1 (20,59), w2 (40,79) — fault every window's
|
|
// IS run so no window reaches its OOS call.
|
|
let runner = WfFakeRunner::faulty(vec![
|
|
(
|
|
(0, 39),
|
|
vec![("len".to_string(), Scalar::i64(4))],
|
|
MemberFault::Run("is-w0".to_string()),
|
|
),
|
|
(
|
|
(20, 59),
|
|
vec![("len".to_string(), Scalar::i64(4))],
|
|
MemberFault::Run("is-w1".to_string()),
|
|
),
|
|
(
|
|
(40, 79),
|
|
vec![("len".to_string(), Scalar::i64(4))],
|
|
MemberFault::Run("is-w2".to_string()),
|
|
),
|
|
]);
|
|
let outcome = run_wf(&campaign, &process, &runner, ®istry)
|
|
.expect("an all-folds-faulted stage fails the cell, not the process");
|
|
|
|
assert!(
|
|
outcome.cells[0].families.iter().all(|f| f.block != "std::walk_forward"),
|
|
"no WalkForward family is produced when every fold fails"
|
|
);
|
|
let cell = &outcome.record.cells[0];
|
|
let fault = cell.fault.as_ref().expect("the cell fails at the wf stage");
|
|
assert_eq!(fault.stage, 1, "walk_forward is pipeline stage 1 here");
|
|
assert!(fault.detail.contains("all 3 walk_forward folds failed"));
|
|
|
|
assert_eq!(registry.load_campaign_runs().expect("load campaign runs").len(), 1);
|
|
let members = registry.load_family_members().expect("load members");
|
|
assert!(members.iter().all(|m| m.kind != FamilyKind::WalkForward));
|
|
}
|
|
|
|
/// #272: a member PANIC inside a window's in-sample sweep is caught at the
|
|
/// wf member boundary (`catch_unwind`) and contained as that fold's
|
|
/// `WindowFault` with kind `panic` — the process does not abort, the
|
|
/// surviving folds pool, exactly as an `Err` fault is contained. Guards the
|
|
/// wf-path panic arm (the sweep-path arm is covered by `PanicRunner` in
|
|
/// tests/execute.rs; this is its walk_forward mirror).
|
|
#[test]
|
|
fn wf_member_panic_in_a_fold_is_contained() {
|
|
// Panics on w1's IS bounds (20,59); every other call returns a report.
|
|
struct WfPanicRunner;
|
|
impl MemberRunner for WfPanicRunner {
|
|
fn run_member(
|
|
&self,
|
|
_cell: &CellSpec,
|
|
params: &[(String, Scalar)],
|
|
window_ms: (i64, i64),
|
|
) -> Result<RunReport, MemberFault> {
|
|
assert_ne!(window_ms, (20, 59), "planted panic on w1's in-sample run");
|
|
let total: f64 = params
|
|
.iter()
|
|
.map(|(_, s)| match s {
|
|
Scalar::I64(v) => *v as f64,
|
|
Scalar::F64(v) => *v,
|
|
_ => 0.0,
|
|
})
|
|
.sum();
|
|
Ok(RunReport {
|
|
manifest: RunManifest {
|
|
commit: "wf-panic-fake".to_string(),
|
|
params: params.to_vec(),
|
|
defaults: Vec::new(),
|
|
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
|
|
seed: 0,
|
|
broker: "fake".to_string(),
|
|
selection: None,
|
|
instrument: None,
|
|
topology_hash: None,
|
|
project: None,
|
|
},
|
|
metrics: RunMetrics {
|
|
total_pips: total,
|
|
max_drawdown: 0.0,
|
|
bias_sign_flips: 0,
|
|
r: None,
|
|
},
|
|
})
|
|
}
|
|
}
|
|
let registry = wf_registry("is-panic");
|
|
let campaign = wf_campaign((0, 99));
|
|
let process = wf_process(None, 40, 20, 20);
|
|
let strategies = vec![("s".repeat(64), "{}".to_string())];
|
|
let outcome = execute(&"e".repeat(64), &campaign, &process, &strategies, &WfPanicRunner, ®istry)
|
|
.expect("a member panic in a fold is contained, not a process abort");
|
|
|
|
let cell = &outcome.record.cells[0];
|
|
let wf_stage = cell
|
|
.stages
|
|
.iter()
|
|
.find(|s| s.block == "std::walk_forward")
|
|
.expect("wf stage realized despite the panicked fold");
|
|
assert_eq!(wf_stage.window_faults.len(), 1);
|
|
assert_eq!(wf_stage.window_faults[0].window_ordinal, 1);
|
|
assert_eq!(wf_stage.window_faults[0].kind, aura_registry::CellFaultKind::Panic);
|
|
assert!(cell.fault.is_none(), "a partial-fold stage does not fail the cell");
|
|
assert_eq!(registry.load_campaign_runs().expect("load campaign runs").len(), 1);
|
|
}
|
|
|
|
/// select_sweep_winner's Plateau arms dispatch through the public
|
|
/// `execute()` path (only Argmax is exercised by the other wf_* tests):
|
|
/// each `SelectRule` variant lands on its matching `SelectionMode`.
|
|
#[test]
|
|
fn wf_sweep_stage_dispatches_plateau_select_modes() {
|
|
for (i, (select, expected_mode)) in [
|
|
(SelectRule::PlateauMean, SelectionMode::PlateauMean),
|
|
(SelectRule::PlateauWorst, SelectionMode::PlateauWorst),
|
|
]
|
|
.into_iter()
|
|
.enumerate()
|
|
{
|
|
let registry = wf_registry(&format!("plateau-{i}"));
|
|
let campaign = wf_campaign((0, 99));
|
|
let process = ProcessDoc {
|
|
format_version: 1,
|
|
kind: DocKind::Process,
|
|
name: "wf-plateau-proc".to_string(),
|
|
description: None,
|
|
pipeline: vec![StageBlock::Sweep {
|
|
selection: Some(SweepSelection {
|
|
metric: "total_pips".to_string(),
|
|
select,
|
|
deflate: false,
|
|
}),
|
|
}],
|
|
};
|
|
let runner = WfFakeRunner::new();
|
|
let outcome =
|
|
run_wf(&campaign, &process, &runner, ®istry).expect("plateau sweep executes");
|
|
let sel = &outcome.cells[0].selections[0];
|
|
assert_eq!(
|
|
sel.selection.mode, expected_mode,
|
|
"SelectRule::{select:?} must dispatch to SelectionMode::{expected_mode:?}"
|
|
);
|
|
}
|
|
}
|
|
|
|
/// The walk_forward stage's IS-winner selection dispatches through
|
|
/// `select_sweep_winner` too (exec.rs `run_walk_forward_stage`), not just
|
|
/// the sweep stage: a gate-free `sweep -> wf(plateau)` is
|
|
/// preflight-permitted (the lattice stays intact with no gate), and this
|
|
/// pins that the runtime actually SELECTS via plateau rather than
|
|
/// silently falling back to the deflated argmax the stage used
|
|
/// unconditionally before this dispatch existed.
|
|
#[test]
|
|
fn wf_walk_forward_stage_dispatches_plateau_select_modes() {
|
|
for (i, (select, expected_mode)) in [
|
|
(SelectRule::PlateauMean, SelectionMode::PlateauMean),
|
|
(SelectRule::PlateauWorst, SelectionMode::PlateauWorst),
|
|
]
|
|
.into_iter()
|
|
.enumerate()
|
|
{
|
|
let registry = wf_registry(&format!("wf-plateau-{i}"));
|
|
let campaign = wf_campaign((0, 99));
|
|
let process = ProcessDoc {
|
|
format_version: 1,
|
|
kind: DocKind::Process,
|
|
name: "wf-plateau-proc".to_string(),
|
|
description: None,
|
|
pipeline: vec![
|
|
StageBlock::Sweep {
|
|
selection: Some(SweepSelection {
|
|
metric: "total_pips".to_string(),
|
|
select: SelectRule::Argmax,
|
|
deflate: false,
|
|
}),
|
|
},
|
|
StageBlock::WalkForward {
|
|
in_sample_ms: 40,
|
|
out_of_sample_ms: 20,
|
|
step_ms: 20,
|
|
mode: WfMode::Rolling,
|
|
metric: "total_pips".to_string(),
|
|
select,
|
|
},
|
|
],
|
|
};
|
|
let runner = WfFakeRunner::new();
|
|
let outcome = run_wf(&campaign, &process, &runner, ®istry)
|
|
.expect("gate-free plateau walk_forward executes");
|
|
let fam = outcome.cells[0]
|
|
.families
|
|
.iter()
|
|
.find(|f| f.block == "std::walk_forward")
|
|
.expect("wf family present");
|
|
assert!(!fam.reports.is_empty());
|
|
for report in &fam.reports {
|
|
let sel = report
|
|
.manifest
|
|
.selection
|
|
.as_ref()
|
|
.expect("every OOS member carries its IS selection");
|
|
assert_eq!(
|
|
sel.mode, expected_mode,
|
|
"SelectRule::{select:?} on a walk_forward stage must dispatch to SelectionMode::{expected_mode:?}"
|
|
);
|
|
}
|
|
}
|
|
}
|
|
|
|
/// `WfMode::Anchored` maps to the engine's `RollMode::Anchored` (only
|
|
/// Rolling is exercised by the other wf_* tests) — the oracle is the same
|
|
/// roller construction the stage itself uses, just with `RollMode::Anchored`.
|
|
#[test]
|
|
fn wf_anchored_mode_maps_to_engine_rollmode_anchored() {
|
|
let registry = wf_registry("anchored");
|
|
let campaign = wf_campaign((0, 99));
|
|
let process = ProcessDoc {
|
|
format_version: 1,
|
|
kind: DocKind::Process,
|
|
name: "wf-anchored-proc".to_string(),
|
|
description: None,
|
|
pipeline: vec![
|
|
StageBlock::Sweep {
|
|
selection: Some(SweepSelection {
|
|
metric: "total_pips".to_string(),
|
|
select: SelectRule::Argmax,
|
|
deflate: false,
|
|
}),
|
|
},
|
|
StageBlock::WalkForward {
|
|
in_sample_ms: 40,
|
|
out_of_sample_ms: 20,
|
|
step_ms: 20,
|
|
mode: WfMode::Anchored,
|
|
metric: "total_pips".to_string(),
|
|
select: SelectRule::Argmax,
|
|
},
|
|
],
|
|
};
|
|
let runner = WfFakeRunner::new();
|
|
let outcome = run_wf(&campaign, &process, &runner, ®istry).expect("anchored wf executes");
|
|
|
|
let expected: Vec<_> =
|
|
WindowRoller::new((Timestamp(0), Timestamp(99)), 40, 20, 20, RollMode::Anchored)
|
|
.expect("valid roll")
|
|
.collect();
|
|
assert!(!expected.is_empty(), "fixture sanity: at least one anchored window");
|
|
|
|
let fam = outcome.cells[0]
|
|
.families
|
|
.iter()
|
|
.find(|f| f.block == "std::walk_forward")
|
|
.expect("wf family present");
|
|
assert_eq!(fam.reports.len(), expected.len());
|
|
for (report, bounds) in fam.reports.iter().zip(&expected) {
|
|
assert_eq!(report.manifest.window, (bounds.oos.0, bounds.oos.1));
|
|
}
|
|
}
|
|
}
|