fix(cli): reject a silent-vacuous blueprint Monte-Carlo instead of returning it
`aura mc <bp.json>` builds a per-seed synthetic price walk with a fixed 60-bar SyntheticSpec. A deeper-lookback loaded blueprint warms poorly over that walk, so every seed can collapse to a bit-identical draw and the family was returned as an Ok result — a single point masquerading as a distribution, a wrong result with no error. Guard it: with >= 2 seeds, if every draw's metrics equal the first draw's, refuse with a named error (C10 refuse-don't-guess), rather than auto-sizing the walk (which would have to guess a warm-up depth the harness does not surface). Compares `report.metrics`, not the whole RunReport — the manifest seed differs per draw by construction, so a whole-report compare could never detect the collapse. RED-first; the existing `blueprint_mc_family_seeds_differ` pins that a legitimate (differing-draw) MC is not falsely rejected. Interim guard only — the real-data moving-block-bootstrap headline stays gated on the DataServer seam. refs #172
This commit is contained in:
@@ -3078,11 +3078,33 @@ fn blueprint_mc_family(doc: &str, n_seeds: u64, data: &DataSource) -> Result<McF
|
||||
// re-runs the shared reduce-mode member path over its own seeded synthetic walk.
|
||||
let seeds: Vec<u64> = (1..=n_seeds).collect();
|
||||
let base_point: Vec<Scalar> = Vec::new();
|
||||
Ok(monte_carlo(&base_point, &seeds, |seed, _base| {
|
||||
let family = monte_carlo(&base_point, &seeds, |seed, _base| {
|
||||
let sources = synthetic_walk_sources(seed);
|
||||
let window = window_of(&sources).expect("non-empty synthetic walk");
|
||||
run_blueprint_member(reload(doc), &[], &space, sources, window, seed, pip, &topo)
|
||||
}))
|
||||
});
|
||||
// Silent-vacuous MC guard (refuse-don't-guess, C10): with >= 2 seeds, if every draw's
|
||||
// metrics are bit-identical to the first, no seed reached a distinguishable realization —
|
||||
// the strategy never warmed over the fixed synthetic walk (e.g. a lookback as deep as the
|
||||
// walk is long), so the "distribution" is a single point masquerading as a family: a wrong
|
||||
// result with no error. Compare `metrics`, not the whole `RunReport` — the manifest's
|
||||
// `seed` differs per draw by construction, so a whole-report compare could never detect the
|
||||
// collapse; the metrics are the realization the seed is meant to move. A single-draw MC
|
||||
// (n == 1) is trivially "all identical" and is NOT this cross-seed condition, so it passes.
|
||||
if family.draws.len() >= 2
|
||||
&& family
|
||||
.draws
|
||||
.iter()
|
||||
.all(|d| d.report.metrics == family.draws[0].report.metrics)
|
||||
{
|
||||
return Err(
|
||||
"mc is vacuous: every seed produced an identical result — the strategy never warmed \
|
||||
over the synthetic walk, so no seed reached a distinguishable realization; use a \
|
||||
shallower-lookback blueprint or a longer walk"
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
Ok(family)
|
||||
}
|
||||
|
||||
/// `aura sweep <blueprint.json> --axis <name>=<csv> …`: sweep a loaded signal over its
|
||||
@@ -5486,6 +5508,24 @@ mod tests {
|
||||
assert!(m[0] != m[1] || m[1] != m[2], "seeds must yield differing realizations");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn blueprint_mc_family_rejects_vacuous_deep_lookback() {
|
||||
// Property: the mc family builder REFUSES a silent-vacuous Monte-Carlo — one where
|
||||
// every per-seed draw collapses to a bit-identical realization — by RETURNING a named
|
||||
// error rather than an `Ok` family that looks like a real (but indistinguishable)
|
||||
// distribution. A CLOSED deep-lookback signal whose slow SMA length (60) equals the
|
||||
// fixed 60-bar synthetic walk never warms, so every seed yields zero trades and thus
|
||||
// identical metrics; that is a wrong result with no error (C10 refuse-don't-guess).
|
||||
let deep = stage1_signal(Some(2), Some(60)); // slow len == walk len -> never warms
|
||||
let doc = blueprint_to_json(&deep).expect("serializes");
|
||||
let err = blueprint_mc_family(&doc, 3, &DataSource::Synthetic)
|
||||
.expect_err("a vacuous (all-identical) Monte-Carlo is rejected, not returned");
|
||||
assert!(
|
||||
err.contains("vacuous") || err.contains("identical"),
|
||||
"names the vacuous/degenerate condition: {err}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn blueprint_mc_family_rejects_an_open_blueprint() {
|
||||
// Property: the mc family builder REFUSES an open blueprint (free knobs) by RETURNING
|
||||
|
||||
Reference in New Issue
Block a user