feat(0076): deflate the sweep winner's metric for the number of trials

Adds optimize_deflated beside optimize in aura-registry: the in-sample
walk-forward selection now records, on each OOS winner's manifest, how much
the winner's metric is inflated by the size of the search it won — a deflated
score and (R arm) an overfit probability — without changing which member wins
(additive, C23; a regression test pins optimize_deflated's winner == optimize's).

- R arm (sqn_normalized/expectancy_r/...): a centred moving-block reality-check.
  Each member's per-trade R series is mean-subtracted to impose the no-edge null,
  resampled via the r_bootstrap kernel (extracted to a shared resample_block,
  byte-identical), and the best-of-K null-max gives
  overfit_probability = (count(null >= raw)+1)/(n+1) and
  deflated_score = raw - p95(null). Deterministic given the seed (C1).
- total_pips arm: a closed-form expected-max-of-K dispersion floor
  (inv_norm_cdf + expected_max_of_normals), no probability.

The record (FamilySelection on RunManifest) rides the proven
serde(default, skip_serializing_if) widening + a compat.rs mirror, so legacy
runs.jsonl/families.jsonl lines load unchanged and unstamped sweep/mc/standalone
manifests stay byte-identical (C14/C18). The SelectionMode enum reserves a
Plateau slot for #145. CLI: walk-forward stamps both arms; runs family <id> rank
surfaces a human-readable deflated/overfit line.

Quality review (re-dispatched after the implement-loop's quality gate exhausted)
caught a real defect: on the R arm with positive resamples but no member carrying
trade_rs (a family loaded from disk — trade_rs is serde-skipped), the null was
all-NEG_INFINITY, turning deflated_score into +inf. Fixed by collapsing "no usable
null" (zero resamples OR no trade_rs) into one degenerate floor (deflated == raw,
overfit == 1.0); pinned by optimize_deflated_no_trade_rs_floors_instead_of_infinity.

Verified: cargo test --workspace and cargo clippy --workspace --all-targets
-D warnings both green. Frictionless Stage-1 R; n_eff advisory and reload-time
recompute deferred (spec 0076 Out of scope).

closes #144
This commit is contained in:
2026-06-26 14:53:32 +02:00
parent 40e3f609c2
commit a2959050a4
19 changed files with 565 additions and 64 deletions
+25 -5
View File
@@ -24,7 +24,7 @@ use aura_engine::{
WalkForwardResult, WindowBounds, WindowRoller, WindowRun,
};
use aura_registry::{
group_families, mc_member_reports, optimize, rank_by, sweep_member_reports,
group_families, mc_member_reports, optimize_deflated, rank_by, sweep_member_reports,
walkforward_member_reports, FamilyKind, FamilyMember, NameKind, Registry, RunTraces, TraceStore,
WriteKind,
};
@@ -52,6 +52,13 @@ const WF_REAL_IS_NS: i64 = 90 * WF_DAY_NS;
const WF_REAL_OOS_NS: i64 = 30 * WF_DAY_NS;
const WF_REAL_STEP_NS: i64 = 30 * WF_DAY_NS;
/// Trials-deflation resampling budget for walk-forward winner selection. Recorded
/// on each winner's manifest (so `overfit_probability` is reproducible by re-run);
/// CLI flags for these are a deferred refinement.
const DEFLATION_N_RESAMPLES: usize = 1000;
const DEFLATION_BLOCK_LEN: usize = 5;
const DEFLATION_SEED: u64 = 0xDEF1_A7ED;
/// The built-in synthetic price stream: rises through t=4 then reverses, so the
/// demo trace carries one exposure sign flip and a real drawdown (C22 populated
/// trace). Deterministic and fixed (C1).
@@ -166,6 +173,7 @@ fn sim_optimal_manifest(
window,
seed,
broker: format!("sim-optimal(pip_size={pip_size})"),
selection: None,
}
}
@@ -1792,8 +1800,11 @@ fn walkforward_family(
let space = sample_blueprint_with_sinks(data.pip_size()).0.param_space();
walk_forward(roller, space, |w: WindowBounds| {
let is_family = sweep_over(w.is.0, w.is.1, data);
let best = optimize(&is_family, "total_pips").expect("total_pips is a known metric");
let (oos_equity, oos_report) = run_oos(&best.params, w.oos.0, w.oos.1, trace, data);
let (best, selection) = optimize_deflated(
&is_family, "total_pips", DEFLATION_N_RESAMPLES, DEFLATION_BLOCK_LEN, DEFLATION_SEED,
).expect("total_pips is a known metric");
let (oos_equity, mut oos_report) = run_oos(&best.params, w.oos.0, w.oos.1, trace, data);
oos_report.manifest.selection = Some(selection);
WindowRun {
// The tag-free sweep winner is the chosen point; its kinds live on
// WalkForwardResult.space (computed once above from the same blueprint).
@@ -1807,8 +1818,11 @@ fn walkforward_family(
let space = stage1_r_space();
walk_forward(roller, space, |w: WindowBounds| {
let is_family = stage1_r_sweep_over(w.is.0, w.is.1, data, grid);
let best = optimize(&is_family, "sqn_normalized").expect("sqn_normalized is a known metric");
let (oos_equity, oos_report) = run_oos_r(&best.params, w.oos.0, w.oos.1, trace, data);
let (best, selection) = optimize_deflated(
&is_family, "sqn_normalized", DEFLATION_N_RESAMPLES, DEFLATION_BLOCK_LEN, DEFLATION_SEED,
).expect("sqn_normalized is a known metric");
let (oos_equity, mut oos_report) = run_oos_r(&best.params, w.oos.0, w.oos.1, trace, data);
oos_report.manifest.selection = Some(selection);
WindowRun { chosen_params: best.params, oos_equity, oos_report }
})
}
@@ -2200,6 +2214,12 @@ fn runs_family(id: &str, rank: Option<&str>) {
};
for report in &ordered {
println!("{}", report.to_json());
if let Some(sel) = &report.manifest.selection {
match sel.overfit_probability {
Some(p) => println!(" deflated={:.4} P(overfit)={:.4}", sel.deflated_score, p),
None => println!(" deflated={:.4}", sel.deflated_score),
}
}
}
}