Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(campaign,cli): permit --select plateau in the walk_forward stage (#215)

Browse Source 
The campaign walk_forward stage picked its per-window in-sample winner by
optimize_deflated (argmax) unconditionally, dropping the stage's `select`
field, and both the preflight and the `walkforward` verb's `--real` campaign
sub-branch refused plateau:* with a forward-pointer here (the #210 decision-4
deferral; the synthetic non-`--real` sub-branch has accepted plateau since
#173). Honour plateau selection when no gate precedes the wf stage.

- Preflight: the plateau-in-wf refusal is narrowed to gate-preceded stages.
  With no preceding gate the survivors are the full sweep grid, so the
  parameter lattice (grid.axis_lens) is exact and optimize_plateau is sound; a
  gate filters survivors below the grid, breaking the lattice, so plateau after
  any gate stays refused. The refusal is static — document validity must be
  data-independent (a gate's filtering depends on the data) — and keeps the
  existing "a gated survivor subset has no parameter lattice" message, which now
  describes exactly the only case it fires.
- Executor: run_walk_forward_stage threads the per-cell grid.axis_lens and picks
  each window's in-sample winner via select_sweep_winner — the same dispatch the
  sweep stage uses. (Argmax, deflate=true) is byte-identical to the old
  optimize_deflated call, so the argmax grade and content-id pins are unchanged;
  the plateau arms read axis_lens and ignore deflate.
- CLI: the walkforward verb threads --select through translate_walkforward /
  run_walkforward_sugar (a walkforward-local param, not the shared four-verb
  SugarInvocation), mapping the parsed Selection to the research SelectRule via
  select_rule_of; the refusal and its select_is_plateau helper are deleted.
  Scope is walkforward-only — the mc verb's wf-stage select is untouched.

The plateau:worst grade anchor (stitched_total_pips = -9683776.67) is distinct
from the argmax anchor (-10398606.67), proving the plateau path genuinely
changes winner selection rather than falling back to argmax. A gate-free plateau
process validates clean while a gate-preceded one is refused (a new
campaign-validate e2e). Argmax defaults stay byte-identical.

closes #215
This commit is contained in:
Brummel
2026-07-10 02:59:33 +02:00
parent 35b996e3bf
commit 94cdbf90cc
6 changed files with 335 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/aura-campaign/src/exec.rs
+36 -29
View File
@@ -376,6 +376,7 @@ fn run_cell(
stage_ordinal,
stage,
&grid.specs,
&grid.axis_lens,
&survivor_points,
&family_name,
runner,
@@ -565,13 +566,17 @@ fn placeholder_report(params: &[(String, Scalar)], window_ms: (i64, i64)) -> Run
}
}
/// The sweep stage's winner + selection provenance. deflate=true composes only
/// with argmax (preflighted: `DeflatePlateauConflict`), seeding the deflation
/// nulls from the CAMPAIGN seed (C1: a campaign's realization is a pure
/// function of the document). Bare argmax has no annotation to compute, so its
/// `FamilySelection` is synthesized annotation-free; `raw_winner_metric`
/// mirrors the registry's ranking read (an R metric against `r: None` ranks
/// `NEG_INFINITY`).
/// The shared in-sample-winner dispatch: called both for the sweep stage's
/// own family and for each walk-forward window's in-sample family. deflate=
/// true composes only with argmax (preflighted: `DeflatePlateauConflict`),
/// seeding the deflation nulls from the CAMPAIGN seed (C1: a campaign's
/// realization is a pure function of the document). Bare argmax has no
/// annotation to compute, so its `FamilySelection` is synthesized annotation-
/// free; `raw_winner_metric` mirrors the registry's ranking read (an R metric
/// against `r: None` ranks `NEG_INFINITY`). The plateau arms (`PlateauMean`/
/// `PlateauWorst`) ignore `deflate` outright (the `_` guard): they never
/// deflate regardless of the caller's flag, so the walk-forward caller's
/// hardcoded `deflate=true` only ever takes effect on the `Argmax` arm.
fn select_sweep_winner(
family: &SweepFamily,
axis_lens: &[usize],
@@ -650,12 +655,16 @@ fn placeholder_window_run(specs: &[ParamSpec]) -> WindowRun {
/// Execute one `std::walk_forward` stage over the cell's surviving points:
/// roll (IS, OOS) window splits over the cell window — entirely in ms
/// (`Timestamp` is unit-agnostic `i64`; the driver owns ms->ns at its data
/// seam) — then per window sweep the survivor points over the IS bounds, pick
/// the winner (argmax with trials-deflation provenance, seeded from the
/// campaign seed — the shipped select_winner convention; plateau in
/// walk_forward is preflight-refused), run the winner over the OOS bounds
/// with the selection stamped on its fresh report, and append the per-window
/// OOS reports as a `FamilyKind::WalkForward` family.
/// seam) — then per window sweep the survivor points over the IS bounds and
/// pick the winner via `select_sweep_winner` (the same dispatch the sweep
/// stage uses), seeded from the campaign seed: `Argmax` keeps the shipped
/// trials-deflation provenance, and `PlateauMean`/`PlateauWorst` read
/// `axis_lens` to score the closed neighbourhood — sound only because
/// preflight refuses plateau whenever a gate precedes this stage (a gate
/// filters survivors below the full grid, breaking the lattice `axis_lens`
/// assumes intact). Then run the winner over the OOS bounds with the
/// selection stamped on its fresh report, and append the per-window OOS
/// reports as a `FamilyKind::WalkForward` family.
#[allow(clippy::too_many_arguments)]
fn run_walk_forward_stage(
seed: u64,
@@ -663,12 +672,13 @@ fn run_walk_forward_stage(
stage: usize,
block: &StageBlock,
specs: &[ParamSpec],
axis_lens: &[usize],
survivors: &[Vec<Scalar>],
family_name: &str,
runner: &dyn MemberRunner,
registry: &Registry,
) -> Result<StageFamily, ExecFault> {
let StageBlock::WalkForward { in_sample_ms, out_of_sample_ms, step_ms, mode, metric, .. } =
let StageBlock::WalkForward { in_sample_ms, out_of_sample_ms, step_ms, mode, metric, select } =
block
else {
return Err(ExecFault::PipelineShape {
@@ -763,21 +773,18 @@ fn run_walk_forward_stage(
faults.lock().unwrap().push((widx, ExecFault::Member(fault)));
return placeholder_window_run(specs);
}
// IS winner: argmax with trials-deflation provenance, seeded from the
// campaign seed (the shipped select_winner convention).
let (winner, selection) = match optimize_deflated(
&is_family,
metric,
DEFLATION_N_RESAMPLES,
DEFLATION_BLOCK_LEN,
seed,
) {
Ok(picked) => picked,
Err(e) => {
faults.lock().unwrap().push((widx, ExecFault::Registry(e)));
return placeholder_window_run(specs);
}
};
// IS winner: the same select_sweep_winner dispatch the sweep stage
// uses, seeded from the campaign seed. deflate=true keeps Argmax's
// shipped trials-deflation provenance identical to before this
// dispatch existed; the Plateau arms ignore the deflate flag.
let (winner, selection) =
match select_sweep_winner(&is_family, axis_lens, metric, *select, true, seed) {
Ok(picked) => picked,
Err(e) => {
faults.lock().unwrap().push((widx, e));
return placeholder_window_run(specs);
}
};
// OOS: run the winner over the out-of-sample bounds; the selection is
// stamped on the fresh OOS report (a new record, never a mutation of
// a stored one).
crates/aura-campaign/src/lib.rs
+87 -5
View File
@@ -287,7 +287,9 @@ pub fn preflight(process: &ProcessDoc, campaign: &CampaignDoc) -> Result<(), Exe
if !RANKABLE_METRICS.contains(&metric.as_str()) {
return Err(ExecFault::UnrankableMetric { stage: i, metric: metric.clone() });
}
if matches!(select, SelectRule::PlateauMean | SelectRule::PlateauWorst) {
if matches!(select, SelectRule::PlateauMean | SelectRule::PlateauWorst)
&& process.pipeline[..i].iter().any(|s| matches!(s, StageBlock::Gate { .. }))
{
return Err(ExecFault::PlateauInWalkForward { stage: i });
}
for (field, len) in [
@@ -795,17 +797,33 @@ mod tests {
));
}
/// Plateau select in a walk_forward is only refused when a gate
/// precedes it: a gate-free sweep -> wf keeps the survivors as the full
/// grid (the parameter lattice is intact, so plateau is meaningful);
/// a gate stage in between filters survivors below the full grid,
/// breaking the lattice, so plateau is refused there.
#[test]
fn preflight_refuses_plateau_in_walk_forward() {
fn preflight_permits_plateau_in_gate_free_walk_forward_refuses_after_a_gate() {
let c = campaign();
for select in [SelectRule::PlateauMean, SelectRule::PlateauWorst] {
let p = process_of(vec![
// gate-free: sweep -> wf(plateau) — survivors are the full grid,
// the lattice is intact, so plateau is permitted.
let ok = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
wf_stage("sqn", select),
]);
assert!(preflight(&ok, &c).is_ok(), "gate-free plateau wf must pass preflight");
// gate-preceded: sweep -> gate -> wf(plateau) — the gate breaks
// the lattice, so plateau is refused at stage 2.
let gated = process_of(vec![
sweep_stage("sqn", SelectRule::Argmax, false),
gate_stage("sqn"),
wf_stage("sqn", select),
]);
assert!(matches!(
preflight(&p, &c),
Err(ExecFault::PlateauInWalkForward { stage: 1 })
preflight(&gated, &c),
Err(ExecFault::PlateauInWalkForward { stage: 2 })
));
}
}
@@ -1248,6 +1266,70 @@ mod wf_tests {
}
}
/// The walk_forward stage's IS-winner selection dispatches through
/// `select_sweep_winner` too (exec.rs `run_walk_forward_stage`), not just
/// the sweep stage: a gate-free `sweep -> wf(plateau)` is
/// preflight-permitted (the lattice stays intact with no gate), and this
/// pins that the runtime actually SELECTS via plateau rather than
/// silently falling back to the deflated argmax the stage used
/// unconditionally before this dispatch existed.
#[test]
fn wf_walk_forward_stage_dispatches_plateau_select_modes() {
for (i, (select, expected_mode)) in [
(SelectRule::PlateauMean, SelectionMode::PlateauMean),
(SelectRule::PlateauWorst, SelectionMode::PlateauWorst),
]
.into_iter()
.enumerate()
{
let registry = wf_registry(&format!("wf-plateau-{i}"));
let campaign = wf_campaign((0, 99));
let process = ProcessDoc {
format_version: 1,
kind: DocKind::Process,
name: "wf-plateau-proc".to_string(),
description: None,
pipeline: vec![
StageBlock::Sweep {
selection: Some(SweepSelection {
metric: "total_pips".to_string(),
select: SelectRule::Argmax,
deflate: false,
}),
},
StageBlock::WalkForward {
in_sample_ms: 40,
out_of_sample_ms: 20,
step_ms: 20,
mode: WfMode::Rolling,
metric: "total_pips".to_string(),
select,
},
],
};
let runner = WfFakeRunner::new();
let outcome = run_wf(&campaign, &process, &runner, &registry)
.expect("gate-free plateau walk_forward executes");
let fam = outcome.cells[0]
.families
.iter()
.find(|f| f.block == "std::walk_forward")
.expect("wf family present");
assert!(!fam.reports.is_empty());
for report in &fam.reports {
let sel = report
.manifest
.selection
.as_ref()
.expect("every OOS member carries its IS selection");
assert_eq!(
sel.mode, expected_mode,
"SelectRule::{select:?} on a walk_forward stage must dispatch to SelectionMode::{expected_mode:?}"
);
}
}
}
/// `WfMode::Anchored` maps to the engine's `RollMode::Anchored` (only
/// Rolling is exercised by the other wf_* tests) — the oracle is the same
/// roller construction the stage itself uses, just with `RollMode::Anchored`.
crates/aura-cli/src/main.rs
+21 -17
View File
@@ -686,6 +686,16 @@ fn parse_select(s: &str) -> Result<Selection, ()> {
}
}
/// Map the CLI `--select` value to the research selection rule the campaign
/// document carries.
fn select_rule_of(sel: Selection) -> aura_research::SelectRule {
match sel {
Selection::Argmax => aura_research::SelectRule::Argmax,
Selection::Plateau(PlateauMode::Mean) => aura_research::SelectRule::PlateauMean,
Selection::Plateau(PlateauMode::Worst) => aura_research::SelectRule::PlateauWorst,
}
}
/// Parse a comma-separated list of `T` (each item parsed via `FromStr`), rejecting
/// any item that fails to parse — the shared validator for the r-sma grid flags
/// (#137). The caller folds the `Err` into the subcommand `usage()` so a malformed
@@ -2300,13 +2310,6 @@ fn wf_ms_sizes() -> (u64, u64, u64) {
)
}
/// True iff `--select` names a plateau mode (refused on the dissolved path, Fork B,
/// until #215 ships the wf-stage plateau relaxation). Reuses `parse_select` so the
/// vocabulary stays single-sourced.
fn select_is_plateau(select: Option<&str>) -> bool {
matches!(select.map(parse_select), Some(Ok(Selection::Plateau(_))))
}
/// Resolve a single optional stop knob (`--stop-length`/`--stop-k`): an absent flag
/// defaults to `default`; a present one parses via [`parse_csv_list`] and refuses
/// unless it holds exactly one value (the stop is a risk regime, not a swept axis).
@@ -2938,16 +2941,16 @@ fn dispatch_walkforward(a: WalkforwardCmd, env: &project::Env) {
// The campaign path (#220): the real-archive execution routes
// through the one campaign executor, over the user's own
// blueprint and axes (formerly the welded r-sma branch).
if let Some(s) = a.select.as_deref()
&& parse_select(s).is_err()
{
eprintln!("aura: {}", usage());
std::process::exit(2);
}
if select_is_plateau(a.select.as_deref()) {
eprintln!("aura: --select plateau is not yet available on the campaign path; see #215");
std::process::exit(2);
}
let select = match a.select.as_deref() {
Some(s) => match parse_select(s) {
Ok(sel) => select_rule_of(sel),
Err(()) => {
eprintln!("aura: {}", usage());
std::process::exit(2);
}
},
None => aura_research::SelectRule::Argmax,
};
let (name, symbol, stop_length, stop_k, from, to) =
walkforward_args_from(&a).unwrap_or_else(|m| {
eprintln!("aura: {m}");
@@ -2988,6 +2991,7 @@ fn dispatch_walkforward(a: WalkforwardCmd, env: &project::Env) {
out_of_sample_ms: oos_ms,
step_ms,
},
select,
env,
)
.unwrap_or_else(|m| {
crates/aura-cli/src/verb_sugar.rs
+10 -7
View File
@@ -375,6 +375,7 @@ pub(crate) fn translate_walkforward(
inv: &SugarInvocation,
metric: &str,
w: WfWindows,
select: SelectRule,
) -> Result<GeneratedWalkforward, String> {
let process = ProcessDoc {
format_version: FORMAT_VERSION,
@@ -395,7 +396,7 @@ pub(crate) fn translate_walkforward(
step_ms: w.step_ms,
mode: WfMode::Rolling,
metric: metric.to_string(),
select: SelectRule::Argmax,
select,
},
],
};
@@ -445,9 +446,10 @@ pub(crate) fn run_walkforward_sugar(
inv: &SugarInvocation,
metric: &str,
w: WfWindows,
select: SelectRule,
env: &crate::project::Env,
) -> Result<(), String> {
let generated = translate_walkforward(inv, metric, w)?;
let generated = translate_walkforward(inv, metric, w, select)?;
let probe_params = probe_params_from(inv.axes);
validate_before_register(&generated.process, &generated.campaign, inv.blueprint_canonical, &probe_params, env)?;
let reg = env.registry();
@@ -817,8 +819,8 @@ mod tests {
fn translate_walkforward_is_deterministic_and_carries_the_regime() {
let ax = wf_axes();
let i = inv(&ax, "walkforward", &["GER40"], Some(VolStop { length: 14, k: 2.0 }), "{\"bp\":1}");
let a = translate_walkforward(&i, "sqn_normalized", WF_W).unwrap();
let b = translate_walkforward(&i, "sqn_normalized", WF_W).unwrap();
let a = translate_walkforward(&i, "sqn_normalized", WF_W, SelectRule::Argmax).unwrap();
let b = translate_walkforward(&i, "sqn_normalized", WF_W, SelectRule::Argmax).unwrap();
assert_eq!(
content_id_of(&campaign_to_json(&a.campaign)),
content_id_of(&campaign_to_json(&b.campaign)),
@@ -866,9 +868,9 @@ mod tests {
#[test]
fn translate_walkforward_diverging_regimes_do_not_collide_content_ids() {
let ax = wf_axes();
let base = translate_walkforward(&inv(&ax, "walkforward", &["GER40"], Some(VolStop { length: 14, k: 2.0 }), "{\"bp\":1}"), "sqn_normalized", WF_W).unwrap();
let different_k = translate_walkforward(&inv(&ax, "walkforward", &["GER40"], Some(VolStop { length: 14, k: 3.0 }), "{\"bp\":1}"), "sqn_normalized", WF_W).unwrap();
let different_length = translate_walkforward(&inv(&ax, "walkforward", &["GER40"], Some(VolStop { length: 20, k: 2.0 }), "{\"bp\":1}"), "sqn_normalized", WF_W).unwrap();
let base = translate_walkforward(&inv(&ax, "walkforward", &["GER40"], Some(VolStop { length: 14, k: 2.0 }), "{\"bp\":1}"), "sqn_normalized", WF_W, SelectRule::Argmax).unwrap();
let different_k = translate_walkforward(&inv(&ax, "walkforward", &["GER40"], Some(VolStop { length: 14, k: 3.0 }), "{\"bp\":1}"), "sqn_normalized", WF_W, SelectRule::Argmax).unwrap();
let different_length = translate_walkforward(&inv(&ax, "walkforward", &["GER40"], Some(VolStop { length: 20, k: 2.0 }), "{\"bp\":1}"), "sqn_normalized", WF_W, SelectRule::Argmax).unwrap();
let base_id = content_id_of(&campaign_to_json(&base.campaign));
let k_id = content_id_of(&campaign_to_json(&different_k.campaign));
let length_id = content_id_of(&campaign_to_json(&different_length.campaign));
@@ -893,6 +895,7 @@ mod tests {
&inv(&ax, "walkforward", &["GER40"], Some(VolStop { length: 14, k: 2.0 }), "{\"bp\":1}"),
"sqn_normalized",
WF_W,
SelectRule::Argmax,
)
.unwrap();
let (process_id, campaign_id) = register_generated_wf(&reg, &generated).unwrap();
crates/aura-cli/tests/cli_run.rs
+82 -9
View File
@@ -1801,11 +1801,13 @@ fn walkforward_dissolved_refuses_a_multi_value_stop() {
assert!(stderr.contains("single risk regime"), "stop-regime refusal: {stderr}");
}
/// Fork B: --select plateau is refused on the dissolved path (exit 2) with a
/// pointer to #215 until the wf-stage plateau relaxation ships.
/// #215: --select plateau is no longer refused on the dissolved path — it now
/// threads through to the wf stage exactly like the built-in synthetic path.
/// Gated on the shared GER40 archive; skips cleanly on a data refusal (exit 1),
/// same tolerance as the sibling stop-knob-defaulting test.
#[test]
fn walkforward_dissolved_refuses_select_plateau() {
let cwd = temp_cwd("walkforward-plateau");
fn walkforward_dissolved_accepts_select_plateau() {
let (cwd, _g) = fresh_project();
let fixture = format!("{}/examples/r_sma_open.json", env!("CARGO_MANIFEST_DIR"));
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args([
@@ -1814,12 +1816,83 @@ fn walkforward_dissolved_refuses_select_plateau() {
"--stop-length", "14", "--stop-k", "2.0",
"--select", "plateau:worst",
])
.current_dir(&cwd)
.current_dir(cwd)
.output()
.expect("spawn aura");
assert_eq!(out.status.code(), Some(2), "plateau is refused on the dissolved path");
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(stderr.contains("#215"), "forward pointer to the plateau follow-up: {stderr}");
if out.status.code() == Some(1) {
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(
stderr.contains("no local data") || stderr.contains("no recorded geometry"),
"exit 1 must be a data refusal, got: {stderr}"
);
eprintln!("skip: no local GER40 data");
return;
}
assert_eq!(
out.status.code(),
Some(0),
"--select plateau:worst must no longer be refused on the dissolved path: {}",
String::from_utf8_lossy(&out.stderr)
);
}
/// Characterization pin (#215): the same identical invocation as
/// `walkforward_real_e2e_pins_the_exact_current_grade` (fixture, real
/// instrument, axes, stop, window), but selecting `plateau:worst` instead of
/// the default argmax. Plateau scores the closed neighbourhood over the full
/// IS grid rather than picking the single best point, so its winner (and
/// therefore its OOS stitch) legitimately differs from the argmax anchor;
/// this pins that divergence exactly, so a regression that silently
/// collapsed plateau back to argmax (or vice versa) inside
/// `run_walk_forward_stage`'s `select_sweep_winner` dispatch goes red here
/// even though the argmax anchor stays green. Gated on the shared GER40
/// archive; skips cleanly on a data refusal.
#[test]
fn walkforward_real_e2e_pins_the_exact_current_plateau_grade() {
const FROM_MS: &str = "1735689600000";
const TO_MS: &str = "1767225599000";
let (cwd, _g) = fresh_project();
let fixture = format!("{}/examples/r_sma_open.json", env!("CARGO_MANIFEST_DIR"));
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args([
"walkforward", &fixture, "--real", "GER40",
"--axis", "sma_signal.fast.length=3,5", "--axis", "sma_signal.slow.length=12,20",
"--stop-length", "14", "--stop-k", "2.0",
"--from", FROM_MS, "--to", TO_MS,
"--select", "plateau:worst",
])
.current_dir(cwd)
.output()
.expect("spawn aura");
if out.status.code() == Some(1) {
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(
stderr.contains("no local data") || stderr.contains("no recorded geometry"),
"exit 1 must be a data refusal, got: {stderr}"
);
eprintln!("skip: no local GER40 data");
return;
}
assert_eq!(out.status.code(), Some(0), "exit: {:?}", out.status);
let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout");
let grade_line = stdout
.lines()
.find(|l| l.starts_with("{\"walkforward\":"))
.unwrap_or_else(|| panic!("the walkforward summary line: {stdout}"));
let v: serde_json::Value = serde_json::from_str(grade_line).expect("summary line parses as JSON");
let wf = &v["walkforward"];
assert_eq!(wf["windows"].as_u64(), Some(9), "window count: {grade_line}");
// EXACT floats -- the byte-identity anchor a plateau-vs-argmax regression must preserve.
assert_eq!(wf["stitched_total_pips"].as_f64(), Some(-9683776.66663749), "stitched OOS pips: {grade_line}");
let r = &wf["oos_r"];
assert_eq!(r["n_trades"].as_u64(), Some(20667), "pooled OOS trade count: {grade_line}");
assert_eq!(r["expectancy_r"].as_f64(), Some(0.007635120926372627), "pooled OOS expectancy R: {grade_line}");
// param_stability varies across windows -> the per-window plateau refit picks
// different winners than the argmax anchor; pinning the means locks the
// plateau winner-selection path.
let ps = wf["param_stability"].as_array().expect("param_stability is an array");
assert_eq!(ps[0]["mean"].as_f64(), Some(4.777777777777778), "fast-MA refit mean: {grade_line}");
assert_eq!(ps[1]["mean"].as_f64(), Some(16.444444444444443), "slow-MA refit mean: {grade_line}");
}
/// Property (#217): `walkforward_args_from` no longer requires both stop knobs
@@ -1916,7 +1989,7 @@ fn walkforward_dissolved_refuses_an_unknown_axis_name() {
/// Front-end parity: an unknown `--select` token is refused (exit 2) on the
/// dissolved path exactly as it is on the inline path — it must not be silently
/// swallowed by `select_is_plateau`'s plateau-only check and fall through to argmax.
/// swallowed by `select_rule_of`'s conversion and fall through to argmax.
#[test]
fn walkforward_dissolved_refuses_an_unknown_select_token() {
let cwd = temp_cwd("walkforward-bad-select");
crates/aura-cli/tests/research_docs.rs
+99
View File
@@ -1582,6 +1582,105 @@ fn campaign_validate_runs_the_executor_preflight_as_a_third_tier() {
assert!(!bad_out.contains("PipelineShape"), "Debug leak: {bad_out}");
}
/// #215: `walk_forward` selecting a plateau mode with NO gate ahead of it —
/// the sweep survivors are the full parameter grid, so the plateau
/// neighbourhood scoring `axis_lens` assumes is intact.
const WF_PLATEAU_GATE_FREE_PROCESS_DOC: &str = r#"{
"format_version": 1,
"kind": "process",
"name": "wf-plateau-gate-free",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn_normalized", "select": "argmax" },
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000,
"step_ms": 604800000, "mode": "rolling", "metric": "net_expectancy_r", "select": "plateau:worst" }
]
}"#;
/// #215: the same `walk_forward` plateau select, but a `std::gate` stage now
/// sits between the sweep and the walk-forward — the gate can filter
/// survivors below the full grid, breaking the lattice the plateau
/// neighbourhood assumes, so the executor preflight must still refuse it.
const WF_PLATEAU_GATED_PROCESS_DOC: &str = r#"{
"format_version": 1,
"kind": "process",
"name": "wf-plateau-gated",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn_normalized", "select": "argmax" },
{ "block": "std::gate", "all": [ { "metric": "n_trades", "cmp": "ge", "value": 0.0 } ] },
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000,
"step_ms": 604800000, "mode": "rolling", "metric": "net_expectancy_r", "select": "plateau:worst" }
]
}"#;
/// Property (#215): the executor preflight's plateau-in-walk_forward refusal
/// is narrowed to gate-preceded stages, not a blanket ban on plateau select
/// in walk_forward — pinned through the full document surface (`campaign
/// validate`'s third, executor tier — data-free, before any member runs),
/// mirroring the pattern `campaign_validate_runs_the_executor_preflight_as_a_
/// third_tier` uses for the sibling shape guard. A regression that reverted
/// to the blanket refusal would flip the gate-free face to exit 1; a
/// regression that dropped the refusal entirely would flip the gated face to
/// exit 0 — either direction goes red here.
#[test]
fn campaign_validate_narrows_plateau_in_wf_refusal_to_gate_preceded_stages() {
let _fixture = project_lock();
let dir = built_project();
let runs_dir = dir.join("runs");
std::fs::remove_dir_all(&runs_dir).ok();
let _cleanup = ScratchGuard(vec![
ScratchPath::Dir(runs_dir.clone()),
ScratchPath::File(dir.join("wf-plateau-free.process.json")),
ScratchPath::File(dir.join("wf-plateau-free.campaign.json")),
ScratchPath::File(dir.join("wf-plateau-gated.process.json")),
ScratchPath::File(dir.join("wf-plateau-gated.campaign.json")),
]);
let bp_id = seed_blueprint(dir, "campaign-validate-wf-plateau-seed");
// Face 1 — gate-free: sweep -> walk_forward(plateau:worst). The lattice
// is intact, so the executor tier accepts it (exit 0).
let free_id =
register_process_doc(dir, "wf-plateau-free.process.json", WF_PLATEAU_GATE_FREE_PROCESS_DOC);
write_doc(
dir,
"wf-plateau-free.campaign.json",
&campaign_doc_json(&bp_id, &free_id, (1, 2), "", ""),
);
let (free_out, free_code) =
run_code_in(dir, &["campaign", "validate", "wf-plateau-free.campaign.json"]);
assert_eq!(free_code, Some(0), "gate-free plateau walk_forward validates clean: {free_out}");
assert!(
free_out.contains(
"campaign document valid (executable): pipeline shape and static guards pass"
),
"stdout/stderr: {free_out}"
);
// Face 2 — gate-preceded: sweep -> gate -> walk_forward(plateau:worst).
// The gate can break the lattice, so the executor tier still refuses it
// (exit 1), naming the offending stage.
let gated_id =
register_process_doc(dir, "wf-plateau-gated.process.json", WF_PLATEAU_GATED_PROCESS_DOC);
write_doc(
dir,
"wf-plateau-gated.campaign.json",
&campaign_doc_json(&bp_id, &gated_id, (1, 2), "", ""),
);
let (gated_out, gated_code) =
run_code_in(dir, &["campaign", "validate", "wf-plateau-gated.campaign.json"]);
assert_eq!(gated_code, Some(1), "gate-preceded plateau walk_forward still refuses: {gated_out}");
assert!(
gated_out.contains("campaign is not executable:"),
"the executor-tier fault header: {gated_out}"
);
assert!(
gated_out.contains(
"process stage 2: walk_forward cannot use a plateau select (a gated survivor subset has no parameter lattice)"
),
"the exec_fault_prose naming the gate-broken lattice: {gated_out}"
);
assert!(!gated_out.contains("PlateauInWalkForward"), "Debug leak: {gated_out}");
}
/// A v2 process doc with a static `std::monte_carlo resamples: 0` defect.
const ZERO_RESAMPLES_MC_PROCESS_DOC: &str = r#"{
"format_version": 1,