feat(campaign,cli): permit --select plateau in the walk_forward stage (#215)
The campaign walk_forward stage picked its per-window in-sample winner by optimize_deflated (argmax) unconditionally, dropping the stage's `select` field, and both the preflight and the `walkforward` verb's `--real` campaign sub-branch refused plateau:* with a forward-pointer here (the #210 decision-4 deferral; the synthetic non-`--real` sub-branch has accepted plateau since #173). Honour plateau selection when no gate precedes the wf stage. - Preflight: the plateau-in-wf refusal is narrowed to gate-preceded stages. With no preceding gate the survivors are the full sweep grid, so the parameter lattice (grid.axis_lens) is exact and optimize_plateau is sound; a gate filters survivors below the grid, breaking the lattice, so plateau after any gate stays refused. The refusal is static — document validity must be data-independent (a gate's filtering depends on the data) — and keeps the existing "a gated survivor subset has no parameter lattice" message, which now describes exactly the only case it fires. - Executor: run_walk_forward_stage threads the per-cell grid.axis_lens and picks each window's in-sample winner via select_sweep_winner — the same dispatch the sweep stage uses. (Argmax, deflate=true) is byte-identical to the old optimize_deflated call, so the argmax grade and content-id pins are unchanged; the plateau arms read axis_lens and ignore deflate. - CLI: the walkforward verb threads --select through translate_walkforward / run_walkforward_sugar (a walkforward-local param, not the shared four-verb SugarInvocation), mapping the parsed Selection to the research SelectRule via select_rule_of; the refusal and its select_is_plateau helper are deleted. Scope is walkforward-only — the mc verb's wf-stage select is untouched. The plateau:worst grade anchor (stitched_total_pips = -9683776.67) is distinct from the argmax anchor (-10398606.67), proving the plateau path genuinely changes winner selection rather than falling back to argmax. A gate-free plateau process validates clean while a gate-preceded one is refused (a new campaign-validate e2e). Argmax defaults stay byte-identical. closes #215
This commit is contained in:
@@ -1582,6 +1582,105 @@ fn campaign_validate_runs_the_executor_preflight_as_a_third_tier() {
|
||||
assert!(!bad_out.contains("PipelineShape"), "Debug leak: {bad_out}");
|
||||
}
|
||||
|
||||
/// #215: `walk_forward` selecting a plateau mode with NO gate ahead of it —
|
||||
/// the sweep survivors are the full parameter grid, so the plateau
|
||||
/// neighbourhood scoring `axis_lens` assumes is intact.
|
||||
const WF_PLATEAU_GATE_FREE_PROCESS_DOC: &str = r#"{
|
||||
"format_version": 1,
|
||||
"kind": "process",
|
||||
"name": "wf-plateau-gate-free",
|
||||
"pipeline": [
|
||||
{ "block": "std::sweep", "metric": "sqn_normalized", "select": "argmax" },
|
||||
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000,
|
||||
"step_ms": 604800000, "mode": "rolling", "metric": "net_expectancy_r", "select": "plateau:worst" }
|
||||
]
|
||||
}"#;
|
||||
|
||||
/// #215: the same `walk_forward` plateau select, but a `std::gate` stage now
|
||||
/// sits between the sweep and the walk-forward — the gate can filter
|
||||
/// survivors below the full grid, breaking the lattice the plateau
|
||||
/// neighbourhood assumes, so the executor preflight must still refuse it.
|
||||
const WF_PLATEAU_GATED_PROCESS_DOC: &str = r#"{
|
||||
"format_version": 1,
|
||||
"kind": "process",
|
||||
"name": "wf-plateau-gated",
|
||||
"pipeline": [
|
||||
{ "block": "std::sweep", "metric": "sqn_normalized", "select": "argmax" },
|
||||
{ "block": "std::gate", "all": [ { "metric": "n_trades", "cmp": "ge", "value": 0.0 } ] },
|
||||
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000,
|
||||
"step_ms": 604800000, "mode": "rolling", "metric": "net_expectancy_r", "select": "plateau:worst" }
|
||||
]
|
||||
}"#;
|
||||
|
||||
/// Property (#215): the executor preflight's plateau-in-walk_forward refusal
|
||||
/// is narrowed to gate-preceded stages, not a blanket ban on plateau select
|
||||
/// in walk_forward — pinned through the full document surface (`campaign
|
||||
/// validate`'s third, executor tier — data-free, before any member runs),
|
||||
/// mirroring the pattern `campaign_validate_runs_the_executor_preflight_as_a_
|
||||
/// third_tier` uses for the sibling shape guard. A regression that reverted
|
||||
/// to the blanket refusal would flip the gate-free face to exit 1; a
|
||||
/// regression that dropped the refusal entirely would flip the gated face to
|
||||
/// exit 0 — either direction goes red here.
|
||||
#[test]
|
||||
fn campaign_validate_narrows_plateau_in_wf_refusal_to_gate_preceded_stages() {
|
||||
let _fixture = project_lock();
|
||||
let dir = built_project();
|
||||
let runs_dir = dir.join("runs");
|
||||
std::fs::remove_dir_all(&runs_dir).ok();
|
||||
let _cleanup = ScratchGuard(vec![
|
||||
ScratchPath::Dir(runs_dir.clone()),
|
||||
ScratchPath::File(dir.join("wf-plateau-free.process.json")),
|
||||
ScratchPath::File(dir.join("wf-plateau-free.campaign.json")),
|
||||
ScratchPath::File(dir.join("wf-plateau-gated.process.json")),
|
||||
ScratchPath::File(dir.join("wf-plateau-gated.campaign.json")),
|
||||
]);
|
||||
let bp_id = seed_blueprint(dir, "campaign-validate-wf-plateau-seed");
|
||||
|
||||
// Face 1 — gate-free: sweep -> walk_forward(plateau:worst). The lattice
|
||||
// is intact, so the executor tier accepts it (exit 0).
|
||||
let free_id =
|
||||
register_process_doc(dir, "wf-plateau-free.process.json", WF_PLATEAU_GATE_FREE_PROCESS_DOC);
|
||||
write_doc(
|
||||
dir,
|
||||
"wf-plateau-free.campaign.json",
|
||||
&campaign_doc_json(&bp_id, &free_id, (1, 2), "", ""),
|
||||
);
|
||||
let (free_out, free_code) =
|
||||
run_code_in(dir, &["campaign", "validate", "wf-plateau-free.campaign.json"]);
|
||||
assert_eq!(free_code, Some(0), "gate-free plateau walk_forward validates clean: {free_out}");
|
||||
assert!(
|
||||
free_out.contains(
|
||||
"campaign document valid (executable): pipeline shape and static guards pass"
|
||||
),
|
||||
"stdout/stderr: {free_out}"
|
||||
);
|
||||
|
||||
// Face 2 — gate-preceded: sweep -> gate -> walk_forward(plateau:worst).
|
||||
// The gate can break the lattice, so the executor tier still refuses it
|
||||
// (exit 1), naming the offending stage.
|
||||
let gated_id =
|
||||
register_process_doc(dir, "wf-plateau-gated.process.json", WF_PLATEAU_GATED_PROCESS_DOC);
|
||||
write_doc(
|
||||
dir,
|
||||
"wf-plateau-gated.campaign.json",
|
||||
&campaign_doc_json(&bp_id, &gated_id, (1, 2), "", ""),
|
||||
);
|
||||
let (gated_out, gated_code) =
|
||||
run_code_in(dir, &["campaign", "validate", "wf-plateau-gated.campaign.json"]);
|
||||
assert_eq!(gated_code, Some(1), "gate-preceded plateau walk_forward still refuses: {gated_out}");
|
||||
assert!(
|
||||
gated_out.contains("campaign is not executable:"),
|
||||
"the executor-tier fault header: {gated_out}"
|
||||
);
|
||||
assert!(
|
||||
gated_out.contains(
|
||||
"process stage 2: walk_forward cannot use a plateau select (a gated survivor subset has no parameter lattice)"
|
||||
),
|
||||
"the exec_fault_prose naming the gate-broken lattice: {gated_out}"
|
||||
);
|
||||
assert!(!gated_out.contains("PlateauInWalkForward"), "Debug leak: {gated_out}");
|
||||
}
|
||||
|
||||
/// A v2 process doc with a static `std::monte_carlo resamples: 0` defect.
|
||||
const ZERO_RESAMPLES_MC_PROCESS_DOC: &str = r#"{
|
||||
"format_version": 1,
|
||||
|
||||
Reference in New Issue
Block a user