Files
Aura/crates/aura-campaign/src/lib.rs
T
claude d3b1a1aead feat(campaign,registry,cli): per-cell fault isolation — a failed cell is recorded, never a global abort
closes #272

A member fault (no-data, bind, run, or a caught panic) is now a recorded
per-cell outcome instead of aborting the whole campaign and discarding every
already-computed cell. The incident that motivated this (a 22-instrument
campaign lost ~36 healthy cells ~6.7 min in because Copper had an archive gap)
now completes: the healthy cells persist, the gap cell is recorded as failed,
and the run exits 3.

Direction (owner decision 2026-07-14): run to completion and report
compromised results; no coverage preflight, no window synthesis.

Containment granularity:
- The CELL for a sweep-stage member fault (a grid hole structurally
  compromises winner selection, so the whole cell fails).
- The FOLD for a walk_forward member fault (independent time windows): the
  surviving folds pool into the family, failed folds are recorded as
  StageRealization.window_faults, and the summary names the ratio.

- aura-registry: additive CellFault / CellFaultKind (closed:
  no_data|bind|run|panic|window) / WindowFault / CellCoverage, plus
  fault/coverage fields on CellRealization and window_faults on
  StageRealization — all serde-default-skipped, so pre-#272 campaign_runs
  lines parse and round-trip byte-identical.
- aura-campaign: run_cell returns a fault-annotated CellRealization instead of
  Err (execute's accumulate-then-append-once tail is unchanged and now
  persists every healthy cell + the one run record); a `contain` split keeps
  ExecFault::Registry and doc-shape preflight faults global while Member/Window
  become per-cell/per-fold. Member panics are caught with
  catch_unwind(AssertUnwindSafe) at all three member-run sites (sweep IS/OOS)
  and recorded as MemberFault::Panic — a member panic no longer aborts the
  process. The wf stage partitions Registry faults (global) from Member/Window
  (per-fold) and filters faulted-fold placeholders (the
  "faulted-member-placeholder" broker sentinel) out of the persisted family.
- aura-cli: exec_fault_prose gains the Panic arm; CliMemberRunner::window_coverage
  derives effective bounds + interior gap months from the #264 archive
  primitives; present_campaign prints per-cell failure notes + a completion
  summary and threads the failed-cell count; a run with >=1 failed cell exits 3
  ("completed with failed cells") uniformly across `aura campaign run` and the
  dissolved sweep/walkforward/mc/generalize verbs (exit_on_campaign_result).
  Usage stays 2, refused-before-running stays 1, clean stays 0.

Tests: the global-abort pins flip to containment (execute + the two wf fault
tests → fold-containment + all-folds-fail-the-cell); new panic-containment
tests on both the sweep path (PanicRunner) and the wf path (this commit adds
the wf mirror the loop left uncovered); a new gapped-archive e2e (one covered
cell + one gap cell → exit 3); the ~14 CLI exit-1 pins move to the exit-3
register; a pre-#272-line byte-identical round-trip guard.

Suite: cargo test --workspace green (1309 tests, 0 failed); clippy clean.
Decision log: #272 comments (fork rationale, the fold Registry/Member split,
the placeholder sentinel, uniform exit-3).

Follow-up (minor, not blocking): the plan under-scoped Task 1 to aura-registry
though the additive fields also touch aura-campaign's exec.rs literals — the
loop absorbed it mechanically; a future plan for a cross-crate additive-field
change should scope every crate's construction sites in the first task.
2026-07-14 16:51:31 +02:00

1570 lines
63 KiB
Rust

//! aura-campaign — the campaign-execution library (#198, cycle 0107).
//!
//! Campaign *semantics* as a reusable leaf crate: cell enumeration over a
//! campaign document's (strategy, instrument, window) matrix, preflight of
//! the v1 executable pipeline shape, per-member gate evaluation, winner
//! selection, and realization assembly over the registry's family machinery.
//! Harness construction and data binding enter exclusively through the
//! one-method [`MemberRunner`] seam, so every consumer (the CLI today; the
//! playground and tests tomorrow) binds its own runner while the execution
//! semantics live here once — this crate is NOT the World (C12/C21): it
//! realizes one campaign document; it owns no topology, no data sources,
//! and no UI.
mod exec;
pub use exec::{
execute, member_fault_prose, CampaignOutcome, CellOutcome, StageFamily, StageSelectionOut,
};
use std::collections::BTreeMap;
use aura_core::Scalar;
use aura_engine::RunReport;
use aura_registry::{check_r_metric, RegistryError};
use aura_research::{Axis, CampaignDoc, Cmp, ProcessDoc, RiskRegime, SelectRule, StageBlock};
/// One structural cell of the campaign matrix: (strategy, instrument,
/// window) — #198 decision 7.
pub struct CellSpec {
pub strategy_ordinal: usize,
/// Resolved blueprint content id (== the topology hash).
pub strategy_id: String,
/// Canonical blueprint bytes from the store.
pub blueprint_json: String,
/// The campaign's tuning axes (raw `param_space()` names).
pub axes: BTreeMap<String, Axis>,
pub instrument: String,
/// Inclusive epoch-ms bounds.
pub window_ms: (i64, i64),
/// The cell's risk regime. `None` = the member runner's baked default (the
/// absent/empty-`risk` case); `Some` = an explicit document regime.
pub regime: Option<RiskRegime>,
/// The regime's ordinal in the resolved regime list (0 for the default) —
/// keys the generalize unit and discriminates family names.
pub regime_ordinal: usize,
}
/// The harness/data binding seam — the ONLY thing a consumer implements.
/// Params arrive as (raw axis name, value) pairs; the implementation binds
/// them to its harness convention and runs the member over `cell.instrument`
/// restricted to `window_ms` (inclusive epoch-ms, a sub-range of
/// `cell.window_ms` — a walk-forward stage passes sub-windows).
pub trait MemberRunner: Sync {
fn run_member(
&self,
cell: &CellSpec,
params: &[(String, Scalar)],
window_ms: (i64, i64),
) -> Result<RunReport, MemberFault>;
/// The archive coverage of this cell's (instrument, window), when the
/// runner can know it and it deviates from the requested window (#272).
/// One call per cell — coverage is a property of the cell, not a member.
fn window_coverage(&self, _cell: &CellSpec) -> Option<aura_registry::CellCoverage> {
None
}
}
/// Display-free member faults (the consumer phrases them — the RefFault
/// pattern).
#[derive(Clone, Debug, PartialEq)]
pub enum MemberFault {
NoData { instrument: String, window_ms: (i64, i64) },
Bind(String),
Run(String),
/// A panic that unwound out of `MemberRunner::run_member`, caught at the
/// member boundary (#272) — the payload's best-effort message.
Panic(String),
}
/// Preflight + runtime refusals. Display-free and by-identifier: the
/// consumer phrases them (the DocFault/RefFault pattern).
#[derive(Debug)]
pub enum ExecFault {
/// The pipeline is not `std::sweep (std::gate)* (std::walk_forward)?
/// (std::monte_carlo)? (std::generalize)?`.
PipelineShape { detail: String },
/// A sweep/walk_forward selection metric outside the registry's rankable
/// roster.
UnrankableMetric { stage: usize, metric: String },
/// A gate predicate metric that is not a per-member scalar (e.g. an
/// annotation name such as `deflated_score`).
GateMetricNotPerMember { stage: usize, metric: String },
/// `plateau:*` selection in walk_forward (a gated survivor subset has no
/// grid lattice to smooth over).
PlateauInWalkForward { stage: usize },
/// sweep `deflate: true` with a non-argmax select rule.
DeflatePlateauConflict { stage: usize },
/// A selection-free sweep (no metric/select group) anywhere but the
/// pipeline's terminal stage: gate/walk_forward/monte_carlo run fine off
/// `survivors` (a selection-free sweep still produces the whole family),
/// but no winner/nominee is recorded — a stage that needs the cell's
/// final candidate (`generalize`, campaign-scope) would have nothing to
/// grade.
SelectionFreeSweepNotTerminal { stage: usize },
/// generalize needs >= 2 instruments in the campaign (static).
GeneralizeNeedsInstruments { available: usize },
/// generalize's metric must be an R metric (static — the intrinsic tier
/// accepts any vocabulary name; the shipped generalization() would only
/// refuse at runtime).
GeneralizeNonRMetric { metric: String },
/// monte_carlo with resamples == 0 or block_len == 0 (static, instead of
/// the engine's defined all-zero degenerate).
ZeroBootstrapParam { stage: usize, field: &'static str },
/// `WindowRoller` construction refusals at runtime.
Window { stage: usize, detail: String },
Member(MemberFault),
Registry(RegistryError),
}
/// The 14 per-member scalars a gate predicate may reference: the 3
/// `RunMetrics` scalars plus the 11 `RMetrics` scalars. The three
/// selection-annotation names (`deflated_score` / `overfit_probability` /
/// `neighbourhood_score`) are deliberately NOT here — they describe a
/// selection, not a member. Hand-copied roster (the third metric-roster site
/// beside aura-research's 17-name vocabulary and aura-registry's rankable
/// set, #190); drift fails safe: an unknown name is a preflight refusal,
/// never a wrong number.
pub const PER_MEMBER_METRICS: &[&str] = &[
"total_pips", "max_drawdown", "bias_sign_flips",
"expectancy_r", "n_trades", "win_rate", "avg_win_r", "avg_loss_r",
"profit_factor", "max_r_drawdown", "n_open_at_end", "sqn_normalized",
"sqn", "net_expectancy_r",
];
/// The registry's rankable roster (`resolve_metric`'s name set) — the metrics
/// a sweep/walk_forward stage may select on. Hand-copied (#190); drift fails
/// safe (a name the registry would refuse is refused here first, before any
/// member runs).
pub const RANKABLE_METRICS: &[&str] = &[
"total_pips", "max_drawdown", "bias_sign_flips",
"sqn", "sqn_normalized", "expectancy_r", "net_expectancy_r",
];
/// Deflation resample count — re-exported from `aura-registry`'s shared
/// definition (single-sourced, #199).
pub use aura_registry::DEFLATION_N_RESAMPLES;
/// Deflation moving-block length — re-exported from `aura-registry`'s shared
/// definition (single-sourced, #199).
pub use aura_registry::DEFLATION_BLOCK_LEN;
/// Resolve one of the 14 [`PER_MEMBER_METRICS`] against a member's report.
/// An R-metric name against `metrics.r == None` reads `None` (conservative
/// and deterministic: a gate predicate over `None` fails the member).
/// Annotation names and unknown names read `None`.
pub fn member_metric(report: &RunReport, name: &str) -> Option<f64> {
let m = &report.metrics;
match name {
"total_pips" => Some(m.total_pips),
"max_drawdown" => Some(m.max_drawdown),
"bias_sign_flips" => Some(m.bias_sign_flips as f64),
_ => {
let r = m.r.as_ref()?;
match name {
"expectancy_r" => Some(r.expectancy_r),
"n_trades" => Some(r.n_trades as f64),
"win_rate" => Some(r.win_rate),
"avg_win_r" => Some(r.avg_win_r),
"avg_loss_r" => Some(r.avg_loss_r),
"profit_factor" => Some(r.profit_factor),
"max_r_drawdown" => Some(r.max_r_drawdown),
"n_open_at_end" => Some(r.n_open_at_end as f64),
"sqn_normalized" => Some(r.sqn_normalized),
"sqn" => Some(r.sqn),
"net_expectancy_r" => Some(r.net_expectancy_r),
_ => None,
}
}
}
}
/// Whether `value <cmp> threshold` holds — the gate's comparator arm.
fn predicate_holds(cmp: &Cmp, value: f64, threshold: f64) -> bool {
match cmp {
Cmp::Gt => value > threshold,
Cmp::Ge => value >= threshold,
Cmp::Lt => value < threshold,
Cmp::Le => value <= threshold,
}
}
/// Statically refuse everything refusable before any member runs (the F7
/// lesson applied forward): the v2 executable pipeline shape is exactly
/// `std::sweep (std::gate)* (std::walk_forward)? (std::monte_carlo)?
/// (std::generalize)?` — each suffix stage at most once, order fixed,
/// `std::generalize` strictly last; every sweep/walk_forward selection
/// metric is in [`RANKABLE_METRICS`]; every gate predicate metric is in
/// [`PER_MEMBER_METRICS`]; walk_forward must not select `plateau:*` (a gated
/// survivor subset has no grid lattice); sweep `deflate: true` composes only
/// with `argmax`; walk_forward lengths must fit `i64` (the roller's
/// Timestamp unit); monte_carlo `resamples` and `block_len` must be > 0
/// (static, instead of the engine's defined all-zero degenerate); generalize
/// needs >= 2 campaign instruments and an R selection metric (the registry's
/// `check_r_metric`). The campaign parameter carries the campaign-level
/// static checks (generalize's instrument arity).
pub fn preflight(process: &ProcessDoc, campaign: &CampaignDoc) -> Result<(), ExecFault> {
// Position of a stage in the fixed v2 shape; gates (rank 1) may repeat,
// every other rank appears at most once and ranks never decrease.
fn shape_rank(stage: &StageBlock) -> usize {
match stage {
StageBlock::Sweep { .. } => 0,
StageBlock::Grid => 0,
StageBlock::Gate { .. } => 1,
StageBlock::WalkForward { .. } => 2,
StageBlock::MonteCarlo { .. } => 3,
StageBlock::Generalize { .. } => 4,
}
}
fn block_id(stage: &StageBlock) -> &'static str {
match stage {
StageBlock::Sweep { .. } => "std::sweep",
StageBlock::Grid => "std::grid",
StageBlock::Gate { .. } => "std::gate",
StageBlock::WalkForward { .. } => "std::walk_forward",
StageBlock::MonteCarlo { .. } => "std::monte_carlo",
StageBlock::Generalize { .. } => "std::generalize",
}
}
// shape: `std::sweep (std::gate)* (std::walk_forward)? (std::monte_carlo)?
// (std::generalize)?` — a monotone rank walk over adjacent pairs captures
// every violation: a rank drop is an out-of-order stage (an annotator
// before a population stage, anything after std::generalize), an adjacent
// equal rank above the gate tier is a duplicate suffix stage.
if !matches!(
process.pipeline.first(),
Some(StageBlock::Sweep { .. } | StageBlock::Grid)
) {
return Err(ExecFault::PipelineShape {
detail: "the first stage must be std::sweep or std::grid".to_string(),
});
}
// #256 fork B: an enumerate-only first stage yields points, not executed
// members — every stage except std::walk_forward consumes member reports,
// so std::grid must be immediately followed by std::walk_forward (in
// particular it is never terminal).
if matches!(process.pipeline.first(), Some(StageBlock::Grid))
&& !matches!(process.pipeline.get(1), Some(StageBlock::WalkForward { .. }))
{
return Err(ExecFault::PipelineShape {
detail: "a std::grid first stage must be immediately followed by \
std::walk_forward (every other stage consumes executed \
member reports)"
.to_string(),
});
}
let mut prev = &process.pipeline[0];
for (i, stage) in process.pipeline.iter().enumerate().skip(1) {
if matches!(stage, StageBlock::Sweep { .. } | StageBlock::Grid) {
return Err(ExecFault::PipelineShape {
detail: format!("stage {i}: only the first stage may be {}", block_id(stage)),
});
}
let (rank, prev_rank) = (shape_rank(stage), shape_rank(prev));
if rank < prev_rank {
return Err(ExecFault::PipelineShape {
detail: format!(
"stage {i}: {} cannot follow {}",
block_id(stage),
block_id(prev)
),
});
}
if rank == prev_rank && rank > 1 {
return Err(ExecFault::PipelineShape {
detail: format!("stage {i}: {} may appear at most once", block_id(stage)),
});
}
prev = stage;
}
// per-stage slot rules (shape already established above).
for (i, stage) in process.pipeline.iter().enumerate() {
match stage {
StageBlock::Grid => {}
StageBlock::Sweep { selection } => match selection {
Some(sel) => {
if !RANKABLE_METRICS.contains(&sel.metric.as_str()) {
return Err(ExecFault::UnrankableMetric {
stage: i,
metric: sel.metric.clone(),
});
}
if sel.deflate && sel.select != SelectRule::Argmax {
return Err(ExecFault::DeflatePlateauConflict { stage: i });
}
}
None => {
if i + 1 != process.pipeline.len() {
return Err(ExecFault::SelectionFreeSweepNotTerminal { stage: i });
}
}
},
StageBlock::Gate { all } => {
for p in all {
if !PER_MEMBER_METRICS.contains(&p.metric.as_str()) {
return Err(ExecFault::GateMetricNotPerMember {
stage: i,
metric: p.metric.clone(),
});
}
}
}
StageBlock::WalkForward {
in_sample_ms,
out_of_sample_ms,
step_ms,
mode: _,
metric,
select,
} => {
if !RANKABLE_METRICS.contains(&metric.as_str()) {
return Err(ExecFault::UnrankableMetric { stage: i, metric: metric.clone() });
}
if matches!(select, SelectRule::PlateauMean | SelectRule::PlateauWorst)
&& process.pipeline[..i].iter().any(|s| matches!(s, StageBlock::Gate { .. }))
{
return Err(ExecFault::PlateauInWalkForward { stage: i });
}
for (field, len) in [
("in_sample_ms", *in_sample_ms),
("out_of_sample_ms", *out_of_sample_ms),
("step_ms", *step_ms),
] {
if i64::try_from(len).is_err() {
return Err(ExecFault::PipelineShape {
detail: format!("stage {i}: walk_forward {field} does not fit i64"),
});
}
}
}
StageBlock::MonteCarlo { resamples, block_len } => {
if *resamples == 0 {
return Err(ExecFault::ZeroBootstrapParam { stage: i, field: "resamples" });
}
if *block_len == 0 {
return Err(ExecFault::ZeroBootstrapParam { stage: i, field: "block_len" });
}
}
StageBlock::Generalize { metric } => {
let available = campaign.data.instruments.len();
if available < 2 {
return Err(ExecFault::GeneralizeNeedsInstruments { available });
}
if check_r_metric(metric).is_err() {
return Err(ExecFault::GeneralizeNonRMetric { metric: metric.clone() });
}
}
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use aura_engine::{RMetrics, RunManifest, RunMetrics, Timestamp};
use aura_research::SweepSelection;
/// A report with every per-member scalar planted to a distinct value
/// (1..=14 in `PER_MEMBER_METRICS` order), r block present.
fn report_with_r() -> RunReport {
RunReport {
manifest: RunManifest {
commit: "test".to_string(),
params: vec![],
defaults: vec![],
window: (Timestamp(0), Timestamp(1)),
seed: 0,
broker: "sim".to_string(),
selection: None,
instrument: None,
topology_hash: None,
project: None,
},
metrics: RunMetrics {
total_pips: 1.0,
max_drawdown: 2.0,
bias_sign_flips: 3,
r: Some(RMetrics {
expectancy_r: 4.0,
n_trades: 5,
win_rate: 6.0,
avg_win_r: 7.0,
avg_loss_r: 8.0,
profit_factor: 9.0,
max_r_drawdown: 10.0,
n_open_at_end: 11,
sqn_normalized: 12.0,
sqn: 13.0,
net_expectancy_r: 14.0,
conviction_terciles_r: [0.0; 3],
net_trade_rs: Vec::new(),
}),
},
}
}
#[test]
fn member_metric_resolves_all_fourteen_names() {
let rep = report_with_r();
let expected: &[(&str, f64)] = &[
("total_pips", 1.0),
("max_drawdown", 2.0),
("bias_sign_flips", 3.0),
("expectancy_r", 4.0),
("n_trades", 5.0),
("win_rate", 6.0),
("avg_win_r", 7.0),
("avg_loss_r", 8.0),
("profit_factor", 9.0),
("max_r_drawdown", 10.0),
("n_open_at_end", 11.0),
("sqn_normalized", 12.0),
("sqn", 13.0),
("net_expectancy_r", 14.0),
];
assert_eq!(expected.len(), 14);
assert_eq!(PER_MEMBER_METRICS.len(), 14);
for (name, want) in expected {
assert!(
PER_MEMBER_METRICS.contains(name),
"{name} missing from PER_MEMBER_METRICS"
);
assert_eq!(member_metric(&rep, name), Some(*want), "metric {name}");
}
}
#[test]
fn member_metric_r_names_none_without_r_block() {
let mut rep = report_with_r();
rep.metrics.r = None;
// the three run-level scalars still resolve...
assert_eq!(member_metric(&rep, "total_pips"), Some(1.0));
assert_eq!(member_metric(&rep, "max_drawdown"), Some(2.0));
assert_eq!(member_metric(&rep, "bias_sign_flips"), Some(3.0));
// ...and every R name reads None (a gate predicate over it fails).
for name in [
"expectancy_r", "n_trades", "win_rate", "avg_win_r", "avg_loss_r",
"profit_factor", "max_r_drawdown", "n_open_at_end", "sqn_normalized",
"sqn", "net_expectancy_r",
] {
assert_eq!(member_metric(&rep, name), None, "R metric {name} without r block");
}
}
#[test]
fn member_metric_refuses_annotation_and_unknown_names() {
let rep = report_with_r();
for name in [
"deflated_score", "overfit_probability", "neighbourhood_score", "no_such_metric",
] {
assert_eq!(member_metric(&rep, name), None, "{name} must not resolve");
assert!(
!PER_MEMBER_METRICS.contains(&name),
"{name} must not be in PER_MEMBER_METRICS"
);
}
}
#[test]
fn predicate_holds_covers_all_four_cmps() {
assert!(predicate_holds(&Cmp::Gt, 1.0, 0.0));
assert!(!predicate_holds(&Cmp::Gt, 0.0, 0.0));
assert!(predicate_holds(&Cmp::Ge, 0.0, 0.0));
assert!(!predicate_holds(&Cmp::Ge, -0.1, 0.0));
assert!(predicate_holds(&Cmp::Lt, -1.0, 0.0));
assert!(!predicate_holds(&Cmp::Lt, 0.0, 0.0));
assert!(predicate_holds(&Cmp::Le, 0.0, 0.0));
assert!(!predicate_holds(&Cmp::Le, 0.1, 0.0));
}
// -----------------------------------------------------------------
// preflight
// -----------------------------------------------------------------
use aura_core::ScalarKind;
use aura_research::{
DataSection, DocKind, DocRef, Predicate, Presentation, ProcessRef, StrategyEntry, WfMode,
Window,
};
fn process_of(pipeline: Vec<StageBlock>) -> ProcessDoc {
ProcessDoc {
format_version: 1,
kind: DocKind::Process,
name: "p".to_string(),
description: None,
pipeline,
}
}
/// A minimal intrinsically-valid campaign; preflight's campaign parameter
/// carries no v1 rules, so one fixture serves every test.
fn campaign() -> CampaignDoc {
CampaignDoc {
format_version: 1,
kind: DocKind::Campaign,
name: "c".to_string(),
description: None,
data: DataSection {
instruments: vec!["EURUSD".to_string()],
windows: vec![Window { from_ms: 0, to_ms: 10_000 }],
bindings: BTreeMap::new(),
},
risk: vec![],
cost: vec![],
strategies: vec![StrategyEntry {
r#ref: DocRef::ContentId("0".repeat(64)),
axes: BTreeMap::from([(
"len".to_string(),
Axis {
kind: ScalarKind::I64,
values: vec![Scalar::i64(2), Scalar::i64(3)],
},
)]),
}],
process: ProcessRef { r#ref: DocRef::ContentId("1".repeat(64)) },
seed: 7,
presentation: Presentation { persist_taps: vec![], emit: vec![] },
}
}
fn sweep_stage(metric: &str, select: SelectRule, deflate: bool) -> StageBlock {
StageBlock::Sweep {
selection: Some(SweepSelection { metric: metric.to_string(), select, deflate }),
}
}
fn gate_stage(metric: &str) -> StageBlock {
StageBlock::Gate {
all: vec![Predicate { metric: metric.to_string(), cmp: Cmp::Gt, value: 0.0 }],
}
}
fn wf_stage(metric: &str, select: SelectRule) -> StageBlock {
StageBlock::WalkForward {
in_sample_ms: 4000,
out_of_sample_ms: 1000,
step_ms: 1000,
mode: WfMode::Rolling,
metric: metric.to_string(),
select,
}
}
fn mc_stage(resamples: u32, block_len: u32) -> StageBlock {
StageBlock::MonteCarlo { resamples, block_len }
}
fn generalize_stage(metric: &str) -> StageBlock {
StageBlock::Generalize { metric: metric.to_string() }
}
/// [`campaign`] with a second instrument — generalize's static arity
/// guard needs >= 2 to pass.
fn campaign_two_instruments() -> CampaignDoc {
let mut c = campaign();
c.data.instruments.push("GER40".to_string());
c
}
#[test]
fn preflight_accepts_the_v2_shapes() {
let c = campaign();
let c2 = campaign_two_instruments();
// the full v2 shape: sweep (gate)* (walk_forward)? (monte_carlo)?
// (generalize)?
let full = process_of(vec![
sweep_stage("sqn_normalized", SelectRule::Argmax, true),
gate_stage("net_expectancy_r"),
wf_stage("sqn_normalized", SelectRule::Argmax),
mc_stage(1000, 5),
generalize_stage("net_expectancy_r"),
]);
assert!(preflight(&full, &c2).is_ok());
// degenerate accepted shapes: bare sweep (plateau select without
// deflate is legal on a sweep); sweep + gates without walk_forward.
let bare = process_of(vec![sweep_stage("total_pips", SelectRule::PlateauMean, false)]);
assert!(preflight(&bare, &c).is_ok());
let gated = process_of(vec![
sweep_stage("expectancy_r", SelectRule::Argmax, false),
gate_stage("n_trades"),
gate_stage("win_rate"),
]);
assert!(preflight(&gated, &c).is_ok());
// suffix stages compose independently: mc without wf (single
// instrument fine), generalize without mc (needs 2 instruments),
// wf + mc without generalize.
let mc_only = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
mc_stage(1000, 5),
]);
assert!(preflight(&mc_only, &c).is_ok());
let gen_only = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
generalize_stage("expectancy_r"),
]);
assert!(preflight(&gen_only, &c2).is_ok());
let wf_mc = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
wf_stage("sqn", SelectRule::Argmax),
mc_stage(100, 3),
]);
assert!(preflight(&wf_mc, &c).is_ok());
}
/// v2 shape: an annotator may not precede a population stage, and
/// std::generalize is strictly last. `[sweep, mc, walk_forward]` is
/// intrinsically valid at the document tier yet refused here.
#[test]
fn preflight_refuses_annotator_order_violations() {
let c2 = campaign_two_instruments();
let cases: Vec<(Vec<StageBlock>, &str)> = vec![
(
vec![
sweep_stage("sqn", SelectRule::Argmax, false),
mc_stage(100, 5),
wf_stage("sqn", SelectRule::Argmax),
],
"stage 2: std::walk_forward cannot follow std::monte_carlo",
),
(
vec![
sweep_stage("sqn", SelectRule::Argmax, false),
mc_stage(100, 5),
gate_stage("expectancy_r"),
],
"stage 2: std::gate cannot follow std::monte_carlo",
),
(
vec![
sweep_stage("sqn", SelectRule::Argmax, false),
generalize_stage("expectancy_r"),
mc_stage(100, 5),
],
"stage 2: std::monte_carlo cannot follow std::generalize",
),
(
vec![
sweep_stage("sqn", SelectRule::Argmax, false),
wf_stage("sqn", SelectRule::Argmax),
gate_stage("expectancy_r"),
],
"stage 2: std::gate cannot follow std::walk_forward",
),
];
for (pipeline, want) in cases {
match preflight(&process_of(pipeline), &c2) {
Err(ExecFault::PipelineShape { detail }) => assert_eq!(detail, want),
other => panic!("expected PipelineShape({want}), got {other:?}"),
}
}
}
/// Each v2 suffix stage appears at most once.
#[test]
fn preflight_refuses_duplicate_suffix_stages() {
let c2 = campaign_two_instruments();
let cases: Vec<(Vec<StageBlock>, &str)> = vec![
(
vec![
sweep_stage("sqn", SelectRule::Argmax, false),
wf_stage("sqn", SelectRule::Argmax),
wf_stage("sqn", SelectRule::Argmax),
],
"stage 2: std::walk_forward may appear at most once",
),
(
vec![
sweep_stage("sqn", SelectRule::Argmax, false),
mc_stage(100, 5),
mc_stage(100, 5),
],
"stage 2: std::monte_carlo may appear at most once",
),
(
vec![
sweep_stage("sqn", SelectRule::Argmax, false),
generalize_stage("expectancy_r"),
generalize_stage("expectancy_r"),
],
"stage 2: std::generalize may appear at most once",
),
];
for (pipeline, want) in cases {
match preflight(&process_of(pipeline), &c2) {
Err(ExecFault::PipelineShape { detail }) => assert_eq!(detail, want),
other => panic!("expected PipelineShape({want}), got {other:?}"),
}
}
}
/// generalize's static arity guard: the campaign must carry >= 2
/// instruments (the shipped generalization() would only refuse at
/// runtime, after members ran).
#[test]
fn preflight_refuses_single_instrument_generalize() {
let c = campaign(); // one instrument
let p = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
generalize_stage("expectancy_r"),
]);
assert!(matches!(
preflight(&p, &c),
Err(ExecFault::GeneralizeNeedsInstruments { available: 1 })
));
}
/// generalize's metric guard: check_r_metric refuses pip metrics AND
/// unknown names — both surface as GeneralizeNonRMetric.
#[test]
fn preflight_refuses_non_r_generalize_metric() {
let c2 = campaign_two_instruments();
for bad in ["total_pips", "no_such_metric"] {
let p = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
generalize_stage(bad),
]);
assert!(matches!(
preflight(&p, &c2),
Err(ExecFault::GeneralizeNonRMetric { metric }) if metric == bad
));
}
}
/// monte_carlo zero params are static refusals (instead of the engine's
/// defined all-zero degenerate at runtime), naming the offending field.
#[test]
fn preflight_refuses_zero_bootstrap_params() {
let c = campaign();
let zero_resamples = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
mc_stage(0, 5),
]);
assert!(matches!(
preflight(&zero_resamples, &c),
Err(ExecFault::ZeroBootstrapParam { stage: 1, field: "resamples" })
));
let zero_block = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
mc_stage(100, 0),
]);
assert!(matches!(
preflight(&zero_block, &c),
Err(ExecFault::ZeroBootstrapParam { stage: 1, field: "block_len" })
));
}
/// A selection-free sweep is permitted as the terminal (and here, only)
/// stage of a process: the family itself is the result, no winner needed.
#[test]
fn preflight_accepts_a_terminal_selection_free_sweep() {
let c = campaign();
let process = process_of(vec![StageBlock::Sweep { selection: None }]);
assert!(preflight(&process, &c).is_ok());
}
/// #256 fork B: the enumerate-only leading stage is accepted exactly in
/// the dissolved walkforward/mc shapes — first, immediately before
/// std::walk_forward.
#[test]
fn preflight_accepts_a_grid_first_stage_followed_by_walk_forward() {
let c = campaign();
let wf_shape =
process_of(vec![StageBlock::Grid, wf_stage("sqn_normalized", SelectRule::Argmax)]);
assert!(preflight(&wf_shape, &c).is_ok());
let mc_shape = process_of(vec![
StageBlock::Grid,
wf_stage("sqn_normalized", SelectRule::Argmax),
mc_stage(1000, 5),
]);
assert!(preflight(&mc_shape, &c).is_ok());
}
/// #256 fork B: every other placement is refused — grid terminal, grid
/// before a report-consuming stage (gate / monte_carlo / generalize),
/// grid anywhere but first. Every refusal is a PipelineShape prose fault.
#[test]
fn preflight_refuses_grid_placement_violations() {
let c = campaign();
let c2 = campaign_two_instruments();
let adjacency = "immediately followed by std::walk_forward";
let cases: Vec<(Vec<StageBlock>, &str, &CampaignDoc)> = vec![
(vec![StageBlock::Grid], adjacency, &c),
(
vec![
StageBlock::Grid,
gate_stage("net_expectancy_r"),
wf_stage("sqn_normalized", SelectRule::Argmax),
],
adjacency,
&c,
),
(vec![StageBlock::Grid, mc_stage(1000, 5)], adjacency, &c),
(
vec![StageBlock::Grid, generalize_stage("net_expectancy_r")],
adjacency,
&c2,
),
(
vec![
sweep_stage("sqn_normalized", SelectRule::Argmax, false),
StageBlock::Grid,
],
"only the first stage may be std::grid",
&c,
),
];
for (pipeline, needle, camp) in cases {
let err =
preflight(&process_of(pipeline), camp).expect_err("placement violation");
let ExecFault::PipelineShape { detail } = &err else {
panic!("expected PipelineShape, got {err:?}");
};
assert!(detail.contains(needle), "detail {detail:?} misses {needle:?}");
}
}
/// A selection-free sweep followed by any other stage is refused: a
/// downstream stage would have no selection/nominee to consume.
#[test]
fn preflight_refuses_a_non_terminal_selection_free_sweep() {
let c = campaign();
let process = process_of(vec![
StageBlock::Sweep { selection: None },
gate_stage("sqn"),
]);
match preflight(&process, &c) {
Err(ExecFault::SelectionFreeSweepNotTerminal { stage: 0 }) => {}
other => panic!("expected SelectionFreeSweepNotTerminal, got {other:?}"),
}
}
#[test]
fn preflight_refuses_non_sweep_first_and_double_sweep() {
let c = campaign();
let gate_first = process_of(vec![gate_stage("expectancy_r")]);
assert!(matches!(preflight(&gate_first, &c), Err(ExecFault::PipelineShape { .. })));
let empty = process_of(vec![]);
assert!(matches!(preflight(&empty, &c), Err(ExecFault::PipelineShape { .. })));
let double = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
sweep_stage("sqn", SelectRule::Argmax, false),
]);
assert!(matches!(preflight(&double, &c), Err(ExecFault::PipelineShape { .. })));
// a gate after walk_forward is still a shape refusal (v2 relaxes
// wf-final only for the annotator suffix: mc/generalize may follow).
let wf_mid = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
wf_stage("sqn", SelectRule::Argmax),
gate_stage("expectancy_r"),
]);
assert!(matches!(preflight(&wf_mid, &c), Err(ExecFault::PipelineShape { .. })));
}
#[test]
fn preflight_refuses_unrankable_select_metric() {
let c = campaign();
// win_rate is a per-member scalar but NOT in the registry's rankable roster.
let sweep_bad = process_of(vec![sweep_stage("win_rate", SelectRule::Argmax, false)]);
assert!(matches!(
preflight(&sweep_bad, &c),
Err(ExecFault::UnrankableMetric { stage: 0, metric }) if metric == "win_rate"
));
let wf_bad = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
wf_stage("profit_factor", SelectRule::Argmax),
]);
assert!(matches!(
preflight(&wf_bad, &c),
Err(ExecFault::UnrankableMetric { stage: 1, metric }) if metric == "profit_factor"
));
}
#[test]
fn preflight_refuses_annotation_gate_metric() {
let c = campaign();
let p = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
gate_stage("deflated_score"),
]);
assert!(matches!(
preflight(&p, &c),
Err(ExecFault::GateMetricNotPerMember { stage: 1, metric }) if metric == "deflated_score"
));
}
/// Plateau select in a walk_forward is only refused when a gate
/// precedes it: a gate-free sweep -> wf keeps the survivors as the full
/// grid (the parameter lattice is intact, so plateau is meaningful);
/// a gate stage in between filters survivors below the full grid,
/// breaking the lattice, so plateau is refused there.
#[test]
fn preflight_permits_plateau_in_gate_free_walk_forward_refuses_after_a_gate() {
let c = campaign();
for select in [SelectRule::PlateauMean, SelectRule::PlateauWorst] {
// gate-free: sweep -> wf(plateau) — survivors are the full grid,
// the lattice is intact, so plateau is permitted.
let ok = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
wf_stage("sqn", select),
]);
assert!(preflight(&ok, &c).is_ok(), "gate-free plateau wf must pass preflight");
// gate-preceded: sweep -> gate -> wf(plateau) — the gate breaks
// the lattice, so plateau is refused at stage 2.
let gated = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
gate_stage("sqn"),
wf_stage("sqn", select),
]);
assert!(matches!(
preflight(&gated, &c),
Err(ExecFault::PlateauInWalkForward { stage: 2 })
));
}
}
#[test]
fn preflight_refuses_wf_length_exceeding_i64() {
let c = campaign();
let p = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
StageBlock::WalkForward {
in_sample_ms: u64::MAX,
out_of_sample_ms: 1000,
step_ms: 1000,
mode: WfMode::Rolling,
metric: "sqn".to_string(),
select: SelectRule::Argmax,
},
]);
assert!(matches!(
preflight(&p, &c),
Err(ExecFault::PipelineShape { detail })
if detail.contains("in_sample_ms does not fit i64")
));
}
#[test]
fn preflight_refuses_deflate_with_plateau() {
let c = campaign();
for select in [SelectRule::PlateauMean, SelectRule::PlateauWorst] {
let p = process_of(vec![sweep_stage("sqn", select, true)]);
assert!(matches!(
preflight(&p, &c),
Err(ExecFault::DeflatePlateauConflict { stage: 0 })
));
}
}
}
#[cfg(test)]
mod wf_tests {
use std::collections::BTreeMap;
use std::sync::Mutex;
use aura_analysis::SelectionMode;
use aura_core::{Scalar, ScalarKind};
use aura_engine::{RollMode, RunManifest, RunMetrics, RunReport, Timestamp, WindowRoller};
use aura_registry::{FamilyKind, Registry};
use aura_research::{
Axis, CampaignDoc, Cmp, DataSection, DocKind, DocRef, Predicate, Presentation,
ProcessDoc, ProcessRef, SelectRule, StageBlock, StrategyEntry, SweepSelection, WfMode,
Window,
};
use super::{execute, CampaignOutcome, CellSpec, ExecFault, MemberFault, MemberRunner};
/// A per-test registry in a fresh temp dir (a Registry's family store
/// isolates per DIRECTORY, not per filename — see Registry::open).
fn wf_registry(name: &str) -> Registry {
// fixed, tag-keyed, pid-free name under the build-tree tmp anchor (#258): the
// pre-create wipe below then genuinely reclaims the previous run.
let dir = std::path::Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../target/tmp"))
.join(format!("aura-campaign-wf-{name}"));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).expect("temp dir");
Registry::open(dir.join("runs.jsonl"))
}
/// One strategy x one instrument x one window, over a single I64 axis
/// "len" with values [1, 2, 3, 4] (the fake runner scores total_pips ==
/// the len value, so ranking and gating are fully determined).
fn wf_campaign(window: (i64, i64)) -> CampaignDoc {
CampaignDoc {
format_version: 1,
kind: DocKind::Campaign,
name: "wf-test".to_string(),
description: None,
data: DataSection {
instruments: vec!["SYNTH".to_string()],
windows: vec![Window { from_ms: window.0, to_ms: window.1 }],
bindings: BTreeMap::new(),
},
risk: vec![],
cost: vec![],
strategies: vec![StrategyEntry {
r#ref: DocRef::ContentId("c".repeat(64)),
axes: BTreeMap::from([(
"len".to_string(),
Axis {
kind: ScalarKind::I64,
values: vec![
Scalar::i64(1),
Scalar::i64(2),
Scalar::i64(3),
Scalar::i64(4),
],
},
)]),
}],
process: ProcessRef { r#ref: DocRef::ContentId("d".repeat(64)) },
seed: 7,
presentation: Presentation { persist_taps: vec![], emit: vec![] },
}
}
/// sweep -> (optional total_pips-gt gate) -> walk_forward, both ranked
/// stages on total_pips / argmax, rolling mode.
fn wf_process(gate_gt: Option<f64>, is_ms: u64, oos_ms: u64, step_ms: u64) -> ProcessDoc {
let mut pipeline = vec![StageBlock::Sweep {
selection: Some(SweepSelection {
metric: "total_pips".to_string(),
select: SelectRule::Argmax,
deflate: false,
}),
}];
if let Some(threshold) = gate_gt {
pipeline.push(StageBlock::Gate {
all: vec![Predicate {
metric: "total_pips".to_string(),
cmp: Cmp::Gt,
value: threshold,
}],
});
}
pipeline.push(StageBlock::WalkForward {
in_sample_ms: is_ms,
out_of_sample_ms: oos_ms,
step_ms,
mode: WfMode::Rolling,
metric: "total_pips".to_string(),
select: SelectRule::Argmax,
});
ProcessDoc {
format_version: 1,
kind: DocKind::Process,
name: "wf-proc".to_string(),
description: None,
pipeline,
}
}
/// One (window bounds, params) -> fault mapping; a type alias rather than
/// a struct-level `#[allow]` since `WfFakeRunner::faulty`'s own parameter
/// re-triggers the type-complexity lint independently of the field.
type WfFault = ((i64, i64), Vec<(String, Scalar)>, MemberFault);
/// Deterministic fake member runner keyed on (window bounds, params):
/// total_pips == the summed numeric param values (here: the single "len"
/// value), the report's manifest.window echoes the window it was run over,
/// and every call is logged for the IS-population assertions. `faults`
/// (empty for every plain `wf_*` fixture) lets the fault-attribution
/// tests below make one specific (window, params) call fail instead of
/// planting a report — the only way to reach the walk-forward stage's
/// IS-sweep / OOS-run / per-window `min_by_key` capture paths, which a
/// runner that always succeeds cannot exercise.
#[allow(clippy::type_complexity)]
struct WfFakeRunner {
log: Mutex<Vec<((i64, i64), Vec<(String, Scalar)>)>>,
faults: Vec<WfFault>,
}
impl WfFakeRunner {
fn new() -> Self {
WfFakeRunner { log: Mutex::new(Vec::new()), faults: Vec::new() }
}
fn faulty(faults: Vec<WfFault>) -> Self {
WfFakeRunner { log: Mutex::new(Vec::new()), faults }
}
}
impl MemberRunner for WfFakeRunner {
fn run_member(
&self,
_cell: &CellSpec,
params: &[(String, Scalar)],
window_ms: (i64, i64),
) -> Result<RunReport, MemberFault> {
self.log.lock().unwrap().push((window_ms, params.to_vec()));
if let Some((_, _, fault)) =
self.faults.iter().find(|(w, p, _)| *w == window_ms && p == params)
{
return Err(fault.clone());
}
let total: f64 = params
.iter()
.map(|(_, s)| match s {
Scalar::I64(v) => *v as f64,
Scalar::F64(v) => *v,
_ => 0.0,
})
.sum();
Ok(RunReport {
manifest: RunManifest {
commit: "wf-fake".to_string(),
params: params.to_vec(),
defaults: Vec::new(),
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
seed: 0,
broker: "fake".to_string(),
selection: None,
instrument: None,
topology_hash: None,
project: None,
},
metrics: RunMetrics {
total_pips: total,
max_drawdown: 0.0,
bias_sign_flips: 0,
r: None,
},
})
}
}
fn run_wf(
campaign: &CampaignDoc,
process: &ProcessDoc,
runner: &WfFakeRunner,
registry: &Registry,
) -> Result<CampaignOutcome, ExecFault> {
let strategies = vec![("s".repeat(64), "{}".to_string())];
let campaign_id = "e".repeat(64);
execute(&campaign_id, campaign, process, &strategies, runner, registry)
}
#[test]
fn wf_rolls_the_declared_window_in_ms() {
let registry = wf_registry("rolls");
let campaign = wf_campaign((0, 99));
let process = wf_process(None, 40, 20, 20);
let runner = WfFakeRunner::new();
let outcome = run_wf(&campaign, &process, &runner, &registry).expect("wf executes");
// The oracle is the roller's own math over the same config.
let expected: Vec<_> =
WindowRoller::new((Timestamp(0), Timestamp(99)), 40, 20, 20, RollMode::Rolling)
.expect("valid roll")
.collect();
assert_eq!(expected.len(), 3, "fixture sanity: 3 windows over 0..=99");
let fam = outcome.cells[0]
.families
.iter()
.find(|f| f.block == "std::walk_forward")
.expect("wf family present");
assert_eq!(fam.reports.len(), expected.len());
for (report, bounds) in fam.reports.iter().zip(&expected) {
assert_eq!(report.manifest.window, (bounds.oos.0, bounds.oos.1));
}
}
#[test]
fn wf_searches_only_the_survivor_points() {
let registry = wf_registry("survivors");
let campaign = wf_campaign((0, 99));
// total_pips == len; gate total_pips > 2.5 keeps len 3 and len 4 of [1,2,3,4]
let process = wf_process(Some(2.5), 40, 20, 20);
let runner = WfFakeRunner::new();
run_wf(&campaign, &process, &runner, &registry).expect("wf executes");
let is_bounds: Vec<(i64, i64)> =
WindowRoller::new((Timestamp(0), Timestamp(99)), 40, 20, 20, RollMode::Rolling)
.expect("valid roll")
.map(|w| (w.is.0 .0, w.is.1 .0))
.collect();
let log = runner.log.lock().unwrap();
let mut is_total = 0;
let mut seen3 = 0;
let mut seen4 = 0;
for (window, params) in log.iter() {
if !is_bounds.contains(window) {
continue;
}
is_total += 1;
match params[0].1 {
Scalar::I64(3) => seen3 += 1,
Scalar::I64(4) => seen4 += 1,
other => panic!("gated-out point ran in an IS window: {other:?}"),
}
}
// 3 windows x exactly the 2 surviving points
assert_eq!(is_total, 6);
assert_eq!(seen3, 3);
assert_eq!(seen4, 3);
}
#[test]
fn wf_stamps_selection_on_oos_members() {
let registry = wf_registry("selection");
let campaign = wf_campaign((0, 99));
let process = wf_process(None, 40, 20, 20);
let runner = WfFakeRunner::new();
let outcome = run_wf(&campaign, &process, &runner, &registry).expect("wf executes");
let fam = outcome.cells[0]
.families
.iter()
.find(|f| f.block == "std::walk_forward")
.expect("wf family present");
assert!(!fam.reports.is_empty());
for report in &fam.reports {
let sel = report
.manifest
.selection
.as_ref()
.expect("every OOS member carries its IS selection");
assert_eq!(sel.selection_metric, "total_pips");
assert_eq!(sel.seed, Some(7), "deflation is seeded from campaign.seed");
}
}
#[test]
fn wf_roller_refusal_maps_to_window_fault() {
let registry = wf_registry("window-fault");
// 0..=49 cannot fit is 40 + oos 20 (window 0 needs 59) -> the roller
// refuses at runtime (zero lengths are already doc-tier faults).
let campaign = wf_campaign((0, 49));
let process = wf_process(None, 40, 20, 20);
let runner = WfFakeRunner::new();
let result = run_wf(&campaign, &process, &runner, &registry);
let Err(err) = result else { panic!("span too short for one window must refuse") };
match err {
ExecFault::Window { stage, detail } => {
assert_eq!(stage, 1, "walk_forward is pipeline stage 1 here");
assert!(
detail.contains("in_sample_ms + out_of_sample_ms = 60 ms")
&& detail.contains("(50 ms)"),
"detail phrases the refusal in doc-unit ms prose: {detail}"
);
assert!(
!detail.contains("SpanTooShort") && !detail.contains("Timestamp("),
"no Debug form leaks through the detail seam: {detail}"
);
}
other => panic!("expected ExecFault::Window, got {other:?}"),
}
}
/// #272: a member fault raised INSIDE one window's in-sample sweep is
/// contained as that fold's `WindowFault` — the surviving windows still
/// run, pool into the walk_forward family, and the campaign run record
/// persists. Faulting w1's IS run out of 3 windows (w0, w1, w2) leaves
/// exactly 2 surviving OOS reports and one recorded window fault at
/// ordinal 1.
#[test]
fn wf_one_faulted_fold_is_contained_survivors_pool() {
let registry = wf_registry("is-fault");
let campaign = wf_campaign((0, 99));
let process = wf_process(None, 40, 20, 20);
// IS bounds: w0 (0,39), w1 (20,59), w2 (40,79) — fault only w1's IS
// run; w0 and w2 survive and pool.
let runner = WfFakeRunner::faulty(vec![(
(20, 59),
vec![("len".to_string(), Scalar::i64(4))],
MemberFault::Run("is-w1".to_string()),
)]);
let outcome = run_wf(&campaign, &process, &runner, &registry)
.expect("a faulted fold is contained, not a stage abort");
let fam = outcome.cells[0]
.families
.iter()
.find(|f| f.block == "std::walk_forward")
.expect("wf family present despite the faulted fold");
assert_eq!(fam.reports.len(), 2, "only the 2 surviving windows pool");
let cell = &outcome.record.cells[0];
let wf_stage = cell
.stages
.iter()
.find(|s| s.block == "std::walk_forward")
.expect("wf stage realized");
assert_eq!(wf_stage.window_faults.len(), 1);
assert_eq!(wf_stage.window_faults[0].window_ordinal, 1);
assert_eq!(wf_stage.window_faults[0].kind, aura_registry::CellFaultKind::Run);
assert!(cell.fault.is_none(), "a partial-fold stage does not fail the cell");
assert_eq!(registry.load_campaign_runs().expect("load campaign runs").len(), 1);
let members = registry.load_family_members().expect("load members");
assert!(members.iter().any(|m| m.kind == FamilyKind::WalkForward));
}
/// #272: every fold faulting (both the IS sweep AND the OOS run, across
/// all 3 windows) leaves zero surviving OOS reports — the whole cell
/// fails at the wf stage, with no WalkForward family persisted, while the
/// campaign run record still records the failed cell.
#[test]
fn wf_all_folds_faulted_fails_the_cell() {
let registry = wf_registry("all-folds-fault");
let campaign = wf_campaign((0, 99));
let process = wf_process(None, 40, 20, 20);
// IS bounds: w0 (0,39), w1 (20,59), w2 (40,79) — fault every window's
// IS run so no window reaches its OOS call.
let runner = WfFakeRunner::faulty(vec![
(
(0, 39),
vec![("len".to_string(), Scalar::i64(4))],
MemberFault::Run("is-w0".to_string()),
),
(
(20, 59),
vec![("len".to_string(), Scalar::i64(4))],
MemberFault::Run("is-w1".to_string()),
),
(
(40, 79),
vec![("len".to_string(), Scalar::i64(4))],
MemberFault::Run("is-w2".to_string()),
),
]);
let outcome = run_wf(&campaign, &process, &runner, &registry)
.expect("an all-folds-faulted stage fails the cell, not the process");
assert!(
outcome.cells[0].families.iter().all(|f| f.block != "std::walk_forward"),
"no WalkForward family is produced when every fold fails"
);
let cell = &outcome.record.cells[0];
let fault = cell.fault.as_ref().expect("the cell fails at the wf stage");
assert_eq!(fault.stage, 1, "walk_forward is pipeline stage 1 here");
assert!(fault.detail.contains("all 3 walk_forward folds failed"));
assert_eq!(registry.load_campaign_runs().expect("load campaign runs").len(), 1);
let members = registry.load_family_members().expect("load members");
assert!(members.iter().all(|m| m.kind != FamilyKind::WalkForward));
}
/// #272: a member PANIC inside a window's in-sample sweep is caught at the
/// wf member boundary (`catch_unwind`) and contained as that fold's
/// `WindowFault` with kind `panic` — the process does not abort, the
/// surviving folds pool, exactly as an `Err` fault is contained. Guards the
/// wf-path panic arm (the sweep-path arm is covered by `PanicRunner` in
/// tests/execute.rs; this is its walk_forward mirror).
#[test]
fn wf_member_panic_in_a_fold_is_contained() {
// Panics on w1's IS bounds (20,59); every other call returns a report.
struct WfPanicRunner;
impl MemberRunner for WfPanicRunner {
fn run_member(
&self,
_cell: &CellSpec,
params: &[(String, Scalar)],
window_ms: (i64, i64),
) -> Result<RunReport, MemberFault> {
assert_ne!(window_ms, (20, 59), "planted panic on w1's in-sample run");
let total: f64 = params
.iter()
.map(|(_, s)| match s {
Scalar::I64(v) => *v as f64,
Scalar::F64(v) => *v,
_ => 0.0,
})
.sum();
Ok(RunReport {
manifest: RunManifest {
commit: "wf-panic-fake".to_string(),
params: params.to_vec(),
defaults: Vec::new(),
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
seed: 0,
broker: "fake".to_string(),
selection: None,
instrument: None,
topology_hash: None,
project: None,
},
metrics: RunMetrics {
total_pips: total,
max_drawdown: 0.0,
bias_sign_flips: 0,
r: None,
},
})
}
}
let registry = wf_registry("is-panic");
let campaign = wf_campaign((0, 99));
let process = wf_process(None, 40, 20, 20);
let strategies = vec![("s".repeat(64), "{}".to_string())];
let outcome = execute(&"e".repeat(64), &campaign, &process, &strategies, &WfPanicRunner, &registry)
.expect("a member panic in a fold is contained, not a process abort");
let cell = &outcome.record.cells[0];
let wf_stage = cell
.stages
.iter()
.find(|s| s.block == "std::walk_forward")
.expect("wf stage realized despite the panicked fold");
assert_eq!(wf_stage.window_faults.len(), 1);
assert_eq!(wf_stage.window_faults[0].window_ordinal, 1);
assert_eq!(wf_stage.window_faults[0].kind, aura_registry::CellFaultKind::Panic);
assert!(cell.fault.is_none(), "a partial-fold stage does not fail the cell");
assert_eq!(registry.load_campaign_runs().expect("load campaign runs").len(), 1);
}
/// select_sweep_winner's Plateau arms dispatch through the public
/// `execute()` path (only Argmax is exercised by the other wf_* tests):
/// each `SelectRule` variant lands on its matching `SelectionMode`.
#[test]
fn wf_sweep_stage_dispatches_plateau_select_modes() {
for (i, (select, expected_mode)) in [
(SelectRule::PlateauMean, SelectionMode::PlateauMean),
(SelectRule::PlateauWorst, SelectionMode::PlateauWorst),
]
.into_iter()
.enumerate()
{
let registry = wf_registry(&format!("plateau-{i}"));
let campaign = wf_campaign((0, 99));
let process = ProcessDoc {
format_version: 1,
kind: DocKind::Process,
name: "wf-plateau-proc".to_string(),
description: None,
pipeline: vec![StageBlock::Sweep {
selection: Some(SweepSelection {
metric: "total_pips".to_string(),
select,
deflate: false,
}),
}],
};
let runner = WfFakeRunner::new();
let outcome =
run_wf(&campaign, &process, &runner, &registry).expect("plateau sweep executes");
let sel = &outcome.cells[0].selections[0];
assert_eq!(
sel.selection.mode, expected_mode,
"SelectRule::{select:?} must dispatch to SelectionMode::{expected_mode:?}"
);
}
}
/// The walk_forward stage's IS-winner selection dispatches through
/// `select_sweep_winner` too (exec.rs `run_walk_forward_stage`), not just
/// the sweep stage: a gate-free `sweep -> wf(plateau)` is
/// preflight-permitted (the lattice stays intact with no gate), and this
/// pins that the runtime actually SELECTS via plateau rather than
/// silently falling back to the deflated argmax the stage used
/// unconditionally before this dispatch existed.
#[test]
fn wf_walk_forward_stage_dispatches_plateau_select_modes() {
for (i, (select, expected_mode)) in [
(SelectRule::PlateauMean, SelectionMode::PlateauMean),
(SelectRule::PlateauWorst, SelectionMode::PlateauWorst),
]
.into_iter()
.enumerate()
{
let registry = wf_registry(&format!("wf-plateau-{i}"));
let campaign = wf_campaign((0, 99));
let process = ProcessDoc {
format_version: 1,
kind: DocKind::Process,
name: "wf-plateau-proc".to_string(),
description: None,
pipeline: vec![
StageBlock::Sweep {
selection: Some(SweepSelection {
metric: "total_pips".to_string(),
select: SelectRule::Argmax,
deflate: false,
}),
},
StageBlock::WalkForward {
in_sample_ms: 40,
out_of_sample_ms: 20,
step_ms: 20,
mode: WfMode::Rolling,
metric: "total_pips".to_string(),
select,
},
],
};
let runner = WfFakeRunner::new();
let outcome = run_wf(&campaign, &process, &runner, &registry)
.expect("gate-free plateau walk_forward executes");
let fam = outcome.cells[0]
.families
.iter()
.find(|f| f.block == "std::walk_forward")
.expect("wf family present");
assert!(!fam.reports.is_empty());
for report in &fam.reports {
let sel = report
.manifest
.selection
.as_ref()
.expect("every OOS member carries its IS selection");
assert_eq!(
sel.mode, expected_mode,
"SelectRule::{select:?} on a walk_forward stage must dispatch to SelectionMode::{expected_mode:?}"
);
}
}
}
/// `WfMode::Anchored` maps to the engine's `RollMode::Anchored` (only
/// Rolling is exercised by the other wf_* tests) — the oracle is the same
/// roller construction the stage itself uses, just with `RollMode::Anchored`.
#[test]
fn wf_anchored_mode_maps_to_engine_rollmode_anchored() {
let registry = wf_registry("anchored");
let campaign = wf_campaign((0, 99));
let process = ProcessDoc {
format_version: 1,
kind: DocKind::Process,
name: "wf-anchored-proc".to_string(),
description: None,
pipeline: vec![
StageBlock::Sweep {
selection: Some(SweepSelection {
metric: "total_pips".to_string(),
select: SelectRule::Argmax,
deflate: false,
}),
},
StageBlock::WalkForward {
in_sample_ms: 40,
out_of_sample_ms: 20,
step_ms: 20,
mode: WfMode::Anchored,
metric: "total_pips".to_string(),
select: SelectRule::Argmax,
},
],
};
let runner = WfFakeRunner::new();
let outcome = run_wf(&campaign, &process, &runner, &registry).expect("anchored wf executes");
let expected: Vec<_> =
WindowRoller::new((Timestamp(0), Timestamp(99)), 40, 20, 20, RollMode::Anchored)
.expect("valid roll")
.collect();
assert!(!expected.is_empty(), "fixture sanity: at least one anchored window");
let fam = outcome.cells[0]
.families
.iter()
.find(|f| f.block == "std::walk_forward")
.expect("wf family present");
assert_eq!(fam.reports.len(), expected.len());
for (report, bounds) in fam.reports.iter().zip(&expected) {
assert_eq!(report.manifest.window, (bounds.oos.0, bounds.oos.1));
}
}
}