audit: cycle metric-vocabulary tidy — ledger records the retired #147 deferral
Cycle-close audit (architect: drift_found, design core clean). What holds, architect-confirmed: C18 bit-identity for RunMetrics (prose derived byte-identically, rng order preserved, registry suite unchanged); the C10 wall (monomorphic R-gates, r_based in the R vocabulary, cross-vocabulary refusal tested both ways — no leak); C28 direction (trait in aura-analysis, vocabulary supplied from the outer rungs, zero Cargo edge changes); C1 determinism pinned through the generic path. Drift items, all resolved as fixes in this commit: - design ledger: the C28 #147 disposition now records item 2 SHIPPED (the A1 cut) with A2 still deliberately deferred; the #136 one-implementor clause carries a supersession note (the IC is the second implementor). - seven stale doc comments describing the pre-#147 or mid-cycle state (analysis trait + estimator docs, registry check_r_metric C9 claim, engine re-export note, campaign PER_MEMBER_METRICS roster note, member-seam guard comment, research vocabulary note) updated to the shipped state. - R_BASED_METRICS is now oracle-pinned against RunMetricKey::r_based() in the vocabulary test (it feeds the NonRMetric refusal prose; a divergence would have misreported the R-gate silently). Noted, not amended (history stays): commit 6744f67's body says 'three syntax-only edits' where the test module actually took five. No regression scripts are configured (the bench is report-only); the architect review is the gate. Verification: extended vocabulary test, campaign suites, analysis/research suites green; clippy --workspace -D warnings clean. refs #147
This commit is contained in:
@@ -236,9 +236,9 @@ pub fn permute<T>(xs: &mut [T], rng: &mut SplitMix64) {
|
||||
|
||||
/// Per-metric vocabulary a report payload supplies to the registry's
|
||||
/// ranking/deflation machinery (#147). Narrow by design (#136): exactly the
|
||||
/// surface the backtest metrics (the sole implementor so far) and the
|
||||
/// measurement IC (deflatable today via its own permutation-null path, not
|
||||
/// yet through this trait) are expected to share. The registry owns the
|
||||
/// surface the two production implementors — the backtest metrics
|
||||
/// (`RunMetrics`, centred moving-block null) and the measurement IC
|
||||
/// (within-run permutation null) — demonstrably share. The registry owns the
|
||||
/// family fold — best-of-K, p95, Laplace, dispersion floor — the payload
|
||||
/// owns what each metric IS: its name, direction, value read, and what one
|
||||
/// draw of its null statistic means.
|
||||
@@ -266,9 +266,8 @@ pub trait MetricVocabulary: Sized {
|
||||
|
||||
/// One-sided Laplace-smoothed tail estimate `(ge + 1) / (n + 1)`: the
|
||||
/// probability mass at-or-above an observed statistic given that `ge` of `n`
|
||||
/// null draws reached it. Intended as the single source of the formula the
|
||||
/// registry's R deflation arm and the CLI's IC reduction each compute inline
|
||||
/// today; wiring those call sites to this function is deferred follow-up.
|
||||
/// null draws reached it. The single source of the formula: the registry's
|
||||
/// R deflation arm and the CLI's IC reduction both call it.
|
||||
pub fn one_sided_p_laplace(ge: usize, n: usize) -> f64 {
|
||||
(ge + 1) as f64 / (n + 1) as f64
|
||||
}
|
||||
|
||||
@@ -1389,6 +1389,22 @@ mod tests {
|
||||
assert_eq!(<RunMetrics as MetricVocabulary>::known(), RANKABLE_METRICS);
|
||||
assert!(RunMetricKey::Sqn.r_based());
|
||||
assert!(!RunMetricKey::TotalPips.r_based());
|
||||
// Oracle-pin R_BASED_METRICS against the classification itself: the
|
||||
// const feeds the NonRMetric refusal prose, so a divergence would
|
||||
// misreport the R-gate silently rather than fail a lookup.
|
||||
let derived: Vec<&str> = RANKABLE_METRICS
|
||||
.iter()
|
||||
.copied()
|
||||
.filter(|n| {
|
||||
<RunMetrics as MetricVocabulary>::resolve(n)
|
||||
.expect("roster names resolve")
|
||||
.r_based()
|
||||
})
|
||||
.collect();
|
||||
assert_eq!(
|
||||
derived, R_BASED_METRICS,
|
||||
"R_BASED_METRICS must be exactly the r_based slice of the roster"
|
||||
);
|
||||
}
|
||||
|
||||
/// Property: an R-based key on a member with no `r` block ranks worst for
|
||||
|
||||
@@ -133,10 +133,11 @@ pub enum ExecFault {
|
||||
/// `RunMetrics` scalars plus the 11 `RMetrics` scalars. The three
|
||||
/// selection-annotation names (`deflated_score` / `overfit_probability` /
|
||||
/// `neighbourhood_score`) are deliberately NOT here — they describe a
|
||||
/// selection, not a member. Hand-copied roster (the third metric-roster site
|
||||
/// beside aura-research's 17-name vocabulary and aura-registry's rankable
|
||||
/// set, #190); drift fails safe: an unknown name is a preflight refusal,
|
||||
/// never a wrong number.
|
||||
/// selection, not a member. Hand-copied roster beside aura-research's 17-name
|
||||
/// vocabulary (#190); the rankable set is single-sourced from the backtest
|
||||
/// vocabulary since #147, and this roster's nesting is oracle-pinned by the
|
||||
/// extended #190 guard. Drift fails safe: an unknown name is a preflight
|
||||
/// refusal, never a wrong number.
|
||||
pub const PER_MEMBER_METRICS: &[&str] = &[
|
||||
"total_pips", "max_drawdown", "bias_sign_flips",
|
||||
"expectancy_r", "n_trades", "win_rate", "avg_win_r", "avg_loss_r",
|
||||
|
||||
@@ -89,8 +89,9 @@ fn member_runner_seam_dispatches_through_dyn_trait_object_and_carries_the_window
|
||||
/// Property: every metric name in `RANKABLE_METRICS` (the roster `preflight`
|
||||
/// permits a `std::sweep`/`std::walk_forward` stage to SELECT on) is also a
|
||||
/// member of `PER_MEMBER_METRICS` and resolves to `Some` via `member_metric`.
|
||||
/// The two rosters are independently hand-copied (documented drift risk,
|
||||
/// #190); were they to diverge, `preflight` would accept a selection metric
|
||||
/// Since #147 the rankable roster is a single-source re-export and the
|
||||
/// nesting is guard-pinned; were the rosters still to diverge, `preflight`
|
||||
/// would accept a selection metric
|
||||
/// that `member_metric` silently reads back as `None` for every member — a
|
||||
/// sweep/walk_forward stage that always empties, not a compile error or an
|
||||
/// early refusal.
|
||||
|
||||
@@ -76,9 +76,9 @@ pub use sweep::{
|
||||
run_indexed, sweep, GridSpace, ListSpace, ParamRange, RandomSpace, Space, SweepError,
|
||||
SweepFamily, SweepPoint,
|
||||
};
|
||||
// `resample_block`/`MetricStats` are foundation-grade (aura-analysis, #291);
|
||||
// re-exported here (unchanged exported name set) so existing
|
||||
// `aura_engine::{resample_block, MetricStats}` consumers stay source-compatible.
|
||||
// Foundation-grade aura-analysis items, re-exported so ladder rungs without a
|
||||
// direct aura-analysis edge (e.g. aura-backtest) reach them through the engine
|
||||
// (#291; the metric-vocabulary seam rides the same path, #147).
|
||||
pub use aura_analysis::{one_sided_p_laplace, resample_block, MetricStats, MetricVocabulary};
|
||||
pub use walkforward::{
|
||||
param_stability, walk_forward, RollMode, WalkForwardError, WalkForwardResult,
|
||||
|
||||
@@ -793,8 +793,9 @@ pub struct Generalization {
|
||||
}
|
||||
|
||||
/// Validate that `metric` is a known, R-based ranking key — the data-free pre-check
|
||||
/// the CLI runs before evaluating any instrument. The registry owns metric truth
|
||||
/// (C9), so the CLI never duplicates the R-set.
|
||||
/// the CLI runs before evaluating any instrument. The R classification lives in the
|
||||
/// backtest vocabulary (`RunMetricKey::r_based`, C10); this gate is the registry's
|
||||
/// monomorphic wall around it, so the CLI never duplicates the R-set.
|
||||
pub fn check_r_metric(metric: &str) -> Result<(), RegistryError> {
|
||||
let key = <RunMetrics as MetricVocabulary>::resolve(metric).ok_or_else(|| {
|
||||
RegistryError::UnknownMetric {
|
||||
|
||||
@@ -736,8 +736,9 @@ pub fn content_id_of(canonical: &str) -> String {
|
||||
/// (`aura-campaign/tests/metric_vocabulary_e2e.rs`, #190) also enumerates the
|
||||
/// shipped scalar fields via serde and goes red on any desync, so a stale
|
||||
/// hand-edit here is test-caught rather than silent. Keep this list in sync by
|
||||
/// hand when the upstream field sets change (single-sourcing it away waits on
|
||||
/// #147).
|
||||
/// hand when the upstream field sets change (this list stays a hand-copy by
|
||||
/// the crate's ratified core-only isolation; #147 single-sourced the rankable
|
||||
/// roster elsewhere and the extended #190 guard pins this list's nesting).
|
||||
pub fn metric_vocabulary() -> &'static [&'static str] {
|
||||
&[
|
||||
"expectancy_r",
|
||||
|
||||
Reference in New Issue
Block a user