Cut the engine's backtest-metrics edge via RunReport<M> (C28 phase 2) #292

Closed
opened 2026-07-20 12:24:19 +02:00 by claude · 1 comment
Collaborator

Phase 2 of the Stratification milestone (contract C28, Status block in docs/design/INDEX.md). User-ratified 2026-07-20 together with phase 5; ratification and the split disposition of the coupled deferral are recorded on #147. This realizes item 1 of #147 (RunReport<M> genericity); item 2 (the registry deflation vocabulary) stays deferred until a second metric consumer exists (#290).

Situation (post-#291)

aura-engine still names backtest-metric types in production:

  • crates/aura-engine/Cargo.toml carries aura-backtest in [dependencies] -- a commented TRANSIENT widening introduced by #291 so the report.rs re-export stays name-compatible.
  • RunReport.metrics is concretely RunMetrics; summarize (pip reduction, report.rs) returns RunMetrics.
  • mc.rs carries R/bias-typed fields inside otherwise generic structs (McAggregate.total_pips/max_drawdown/bias_sign_flips, RBootstrap.e_r) beside a genuinely generic statistics kernel (MetricStats, quantile, resample_block).

Goal

The engine's production surface names no backtest-metric type: RunReport generic over its metric payload; the R-typed reductions and aggregations live in the backtest layer; the generic statistics kernel lives at its C28 foundation home; consumers instantiate the trading metric at the process column / CLI edge. Behaviour-preserving (C1); runs.jsonl byte-identity (C18) is the hard gate. Exact genericity shape, item routing, and consumer rewiring are derived against C28 during spec production and recorded here as decision-log comments before the spec is written.

refs #147

Phase 2 of the Stratification milestone (contract C28, Status block in `docs/design/INDEX.md`). User-ratified 2026-07-20 together with phase 5; ratification and the split disposition of the coupled deferral are recorded on #147. This realizes **item 1** of #147 (`RunReport<M>` genericity); item 2 (the registry deflation vocabulary) stays deferred until a second metric consumer exists (#290). ## Situation (post-#291) `aura-engine` still names backtest-metric types in production: - `crates/aura-engine/Cargo.toml` carries `aura-backtest` in `[dependencies]` -- a commented TRANSIENT widening introduced by #291 so the `report.rs` re-export stays name-compatible. - `RunReport.metrics` is concretely `RunMetrics`; `summarize` (pip reduction, `report.rs`) returns `RunMetrics`. - `mc.rs` carries R/bias-typed fields inside otherwise generic structs (`McAggregate.total_pips/max_drawdown/bias_sign_flips`, `RBootstrap.e_r`) beside a genuinely generic statistics kernel (`MetricStats`, `quantile`, `resample_block`). ## Goal The engine's production surface names no backtest-metric type: `RunReport` generic over its metric payload; the R-typed reductions and aggregations live in the backtest layer; the generic statistics kernel lives at its C28 foundation home; consumers instantiate the trading metric at the process column / CLI edge. Behaviour-preserving (C1); `runs.jsonl` byte-identity (C18) is the hard gate. Exact genericity shape, item routing, and consumer rewiring are derived against C28 during spec production and recorded here as decision-log comments before the spec is written. refs #147
claude self-assigned this 2026-07-20 12:24:19 +02:00
Author
Collaborator

Decision log (spec production) -- genericity shape and item routing

Recon of the post-#291 tree (worktree HEAD 94aaa4c) settles the load-bearing forks. Derived decisions with rationale, for audit and veto.

  • D1 -- RunReport<M> in the engine, no default type parameter; the concrete alias lives in aura-backtest (pub type RunReport = aura_engine::RunReport<RunMetrics>;, plus aliases for the generic family types where consumers name them). Rationale: a default M = RunMetrics in the engine would itself name the backtest type and not cut the edge; "run report with trading metrics" is backtest vocabulary, so the alias is layer-honest. RunManifest stays concrete and engine-owned (its selection: Option<FamilySelection> embeds the foundation-grade analysis type -- the #291 D3 payoff).
  • D2 -- item routing. To aura-analysis (foundation statistics): MetricStats (+from_values), quantile, resample_block, and the SplitMix64 RNG they consume (deterministic pure math; the engine re-imports them, a legal inner->foundation edge). To aura-backtest: summarize (constructs RunMetrics), McDraw/McFamily/McAggregate(+from_draws)/assemble_mc/monte_carlo, RBootstrap/r_bootstrap. Rationale: McAggregate::from_draws reads RunMetrics fields BY NAME (total_pips/max_drawdown/bias_sign_flips) -- generifying it is the phase-6 metric-vocabulary abstraction, explicitly deferred; wholesale relocation is the honest cut. monte_carlo composes the engine's generic parallel kernel run_indexed, which stays in the engine (C1 machinery). Generic in the engine: SweepPoint<M>/SweepFamily<M>/sweep/assemble_sweep, WindowRun<M>/WindowOutcome<M>/WalkForwardResult<M>/walk_forward, the SweepBinder/RandomBinder bounds, param_stability.
  • D3 -- dependency inversion, one commit. aura-engine drops aura-backtest from [dependencies] (the #291 TRANSIENT widening, back to [dev-dependencies] for its tests); aura-backtest gains aura-engine in [dependencies]. End state is cycle-free for lib targets -- the cycle closes only through the engine's dev-dep edge, the same pattern the workspace already tolerates via aura-vocabulary. aura-registry gains aura-backtest ([dependencies]); aura-campaign promotes its dev-dep to [dependencies] (production r_bootstrap/RunMetrics uses in exec.rs); aura-cli already depends on it.
  • D4 -- the engine re-export of backtest names is REMOVED (this IS the cut): consumers import trading-concrete names (RunReport alias, RunMetrics, summarize, summarize_r, monte_carlo, McFamily, McAggregate, RBootstrap, r_bootstrap, position events) from aura_backtest, metric-agnostic machinery (RunManifest, MeasurementReport, spaces, sweep, walk_forward, trace utils) from aura_engine. The analysis re-export (FamilySelection etc.) stays -- foundation-grade, legal.
  • D5 -- the C28 structural test extends to the full ladder: new/widened rows aura-analysis ⊆ {} (no aura-* deps), aura-engine ⊆ {aura-core, aura-analysis}, aura-backtest ⊆ {aura-core, aura-engine}. The process column (registry/campaign/cli) stays out of the table -- its trading-awareness is C28-legal and its edge policy is #147-item-2 territory.
  • D6 -- out of scope: the RunManifest.broker "measurement" sentinel honesty fix (stays deferred with #147 item 2); the registry Metric/deflation vocabulary (phase 6, #290 seeder). The ledger's phase-2 label ("cut the engine -> analysis edge"), stale since #291 re-identified the violation as the backtest re-export, is corrected by this cycle's Status-block update.
  • D7 -- C18 gates: runs.jsonl/families.jsonl byte-identity (all moved types verbatim; RunReport<M> keeps field order manifest,metrics; serde derive auto-bounds cover M = RunMetrics); the CLI pre-serialized-splice contract (main.rs family_member_line/mc_member_line) unchanged. All existing byte-pin tests stay green: the engine to_json/round-trip pins (instantiated via dev-dep RunMetrics), the registry legacy-line tests, the moved metrics legacy tests, the drift guard.
## Decision log (spec production) -- genericity shape and item routing Recon of the post-#291 tree (worktree HEAD 94aaa4c) settles the load-bearing forks. Derived decisions with rationale, for audit and veto. - **D1 -- `RunReport<M>` in the engine, no default type parameter; the concrete alias lives in `aura-backtest`** (`pub type RunReport = aura_engine::RunReport<RunMetrics>;`, plus aliases for the generic family types where consumers name them). Rationale: a default `M = RunMetrics` in the engine would itself name the backtest type and not cut the edge; "run report with trading metrics" is backtest vocabulary, so the alias is layer-honest. `RunManifest` stays concrete and engine-owned (its `selection: Option<FamilySelection>` embeds the foundation-grade analysis type -- the #291 D3 payoff). - **D2 -- item routing.** To `aura-analysis` (foundation statistics): `MetricStats` (+`from_values`), `quantile`, `resample_block`, and the `SplitMix64` RNG they consume (deterministic pure math; the engine re-imports them, a legal inner->foundation edge). To `aura-backtest`: `summarize` (constructs `RunMetrics`), `McDraw`/`McFamily`/`McAggregate`(+`from_draws`)/`assemble_mc`/`monte_carlo`, `RBootstrap`/`r_bootstrap`. Rationale: `McAggregate::from_draws` reads `RunMetrics` fields BY NAME (`total_pips`/`max_drawdown`/`bias_sign_flips`) -- generifying it is the phase-6 metric-vocabulary abstraction, explicitly deferred; wholesale relocation is the honest cut. `monte_carlo` composes the engine's generic parallel kernel `run_indexed`, which stays in the engine (C1 machinery). Generic in the engine: `SweepPoint<M>`/`SweepFamily<M>`/`sweep`/`assemble_sweep`, `WindowRun<M>`/`WindowOutcome<M>`/`WalkForwardResult<M>`/`walk_forward`, the `SweepBinder`/`RandomBinder` bounds, `param_stability`. - **D3 -- dependency inversion, one commit.** `aura-engine` drops `aura-backtest` from `[dependencies]` (the #291 TRANSIENT widening, back to `[dev-dependencies]` for its tests); `aura-backtest` gains `aura-engine` in `[dependencies]`. End state is cycle-free for lib targets -- the cycle closes only through the engine's dev-dep edge, the same pattern the workspace already tolerates via `aura-vocabulary`. `aura-registry` gains `aura-backtest` (`[dependencies]`); `aura-campaign` promotes its dev-dep to `[dependencies]` (production `r_bootstrap`/`RunMetrics` uses in `exec.rs`); `aura-cli` already depends on it. - **D4 -- the engine re-export of backtest names is REMOVED** (this IS the cut): consumers import trading-concrete names (`RunReport` alias, `RunMetrics`, `summarize`, `summarize_r`, `monte_carlo`, `McFamily`, `McAggregate`, `RBootstrap`, `r_bootstrap`, position events) from `aura_backtest`, metric-agnostic machinery (`RunManifest`, `MeasurementReport`, spaces, `sweep`, `walk_forward`, trace utils) from `aura_engine`. The analysis re-export (`FamilySelection` etc.) stays -- foundation-grade, legal. - **D5 -- the C28 structural test extends to the full ladder:** new/widened rows `aura-analysis ⊆ {}` (no aura-* deps), `aura-engine ⊆ {aura-core, aura-analysis}`, `aura-backtest ⊆ {aura-core, aura-engine}`. The process column (registry/campaign/cli) stays out of the table -- its trading-awareness is C28-legal and its edge policy is #147-item-2 territory. - **D6 -- out of scope:** the `RunManifest.broker` "measurement" sentinel honesty fix (stays deferred with #147 item 2); the registry `Metric`/deflation vocabulary (phase 6, #290 seeder). The ledger's phase-2 label ("cut the engine -> analysis edge"), stale since #291 re-identified the violation as the backtest re-export, is corrected by this cycle's Status-block update. - **D7 -- C18 gates:** `runs.jsonl`/`families.jsonl` byte-identity (all moved types verbatim; `RunReport<M>` keeps field order `manifest`,`metrics`; serde derive auto-bounds cover `M = RunMetrics`); the CLI pre-serialized-splice contract (`main.rs` `family_member_line`/`mc_member_line`) unchanged. All existing byte-pin tests stay green: the engine `to_json`/round-trip pins (instantiated via dev-dep `RunMetrics`), the registry legacy-line tests, the moved metrics legacy tests, the drift guard.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#292