Files

T

claude a56ab7859d refactor: cut the engine's backtest-metrics edge via RunReport<M>

C28 phase 2 (Stratification); realizes item 1 of the deferred #147. The
engine's production surface no longer names a backtest-metric type:

- RunReport becomes generic over its metric payload M; sweep/mc/walkforward/
  blueprint thread the parameter (SweepPoint<M>, SweepFamily<M>, WindowRun<M>,
  WalkForwardResult<M>). RunManifest stays concrete and engine-owned (its
  selection: Option<FamilySelection> embeds the foundation-grade analysis type).
- summarize and the MC assembly (McDraw/McFamily/McAggregate/RBootstrap/
  r_bootstrap/monte_carlo) move to aura-backtest - McAggregate::from_draws reads
  RunMetrics fields by name, so generifying it is the phase-6 metric-vocabulary
  abstraction (#147 item 2), still deferred; wholesale relocation is the honest
  cut. The concrete instantiation lives in aura-backtest as
  `type RunReport = aura_engine::RunReport<RunMetrics>` + sibling aliases.
- the statistics kernel (MetricStats/quantile/resample_block/SplitMix64) moves
  to the aura-analysis foundation; the engine re-imports it (inner->foundation,
  legal) and re-exports it so existing consumers stay source-compatible.

Dependency inversion in one commit: aura-engine drops aura-backtest from
[dependencies] (back to dev-deps for its SimBroker/RunMetrics test fixtures);
aura-backtest gains aura-engine. Cycle-free for lib targets - the cycle closes
only through the engine's dev-dep edge, the pattern aura-vocabulary already
uses. aura-backtest reaches the kernel transitively through the engine
re-export, so no aura-backtest -> aura-analysis edge exists (the C28 ladder
permits backtest -> {core, engine} only). run_indexed / SplitMix64::next_f64
widened pub(crate) -> pub for cross-crate use.

Consumers (registry/campaign/cli/composites/ingest/bench) rewired by import
path only, no call-site logic changed. The c28_layering structural test extends
to the full ladder: aura-analysis (no aura-* deps), aura-engine ⊆ {core,
analysis}, aura-backtest ⊆ {core, engine}.

Behaviour-preserving: 1448/0 tests, clippy -D warnings clean, serde shapes
byte-identical (C18 - RunReport<M> keeps field order manifest,metrics; the
CLI pre-serialized-splice contract unchanged), moved code traceable via git
rename detection. Cycle-introduced broken intra-doc links fixed.

closes #292

2026-07-20 13:25:07 +02:00

baselines

fix(bench): make the cli_fixed_cost fingerprint invariant across build commits

2026-07-17 18:16:56 +02:00

src

refactor: cut the engine's backtest-metrics edge via RunReport<M>

2026-07-20 13:25:07 +02:00

tests

feat(bench): CLI driver, fixed-cost surface, committed baselines + docs

2026-07-17 18:10:00 +02:00

Cargo.toml

refactor: cut the engine's backtest-metrics edge via RunReport<M>

2026-07-20 13:25:07 +02:00

README.md

feat(bench): CLI driver, fixed-cost surface, committed baselines + docs

2026-07-17 18:10:00 +02:00

README.md

aura-bench

Wall-clock benchmarks with committed baselines (issue #251, phase 1). Catches hot-path regressions the E2E suite structurally cannot see: every surface pins its inputs and asserts a determinism fingerprint, so a fast-but-wrong binary fails the bench instead of pinning a bogus baseline.

Running

cargo run --release -p aura-bench -- run              # measure + compare, report-only
cargo run --release -p aura-bench -- run --surface engine_throughput
cargo run --release -p aura-bench -- pin              # rewrite committed baselines

Debug builds refuse to measure (exit 2) — wall-clock numbers from a debug build measure the profile, not the code.

Surfaces

surface	workload	metrics
`engine_throughput`	10M synthetic bars through an SMA-cross harness (library)	`wall_s`, `bars_per_s`
`ingest_throughput`	24 synthetic archive months x 20 fresh drains via the ingest window API (library)	`wall_s`, `bars_per_s`
`campaign_sweep`	8-cell sweep-only campaign, release `aura` binary on a scratch project	`wall_s`, `cpu_percent`, `peak_rss_mb`
`campaign_heavy`	same cells; sweep → gate → walk-forward → 1000-resample bootstrap	`wall_s`, `cpu_percent`, `peak_rss_mb`
`cli_fixed_cost`	spawn floor (`aura --help`) + minimal data-only project run	`help_ms`, `run_ms`

Discipline

Quiet box: the driver warns at 1-min loadavg > 2.0. Close heavy jobs first.
1 discarded warmup + 3 measured reps (median). Reps must agree on the fingerprint — cross-rep drift is reported as nondeterminism, not averaged.
Baselines are per-host context (host block inside each JSON): comparison on a different host prints an informational banner and stays report-only.
Timing drift NEVER fails the run (report-only phase, #251 cadence decision); a NOTICE line flags drift >= 10%. A fingerprint mismatch exits 1.
Re-pin (pin) deliberately, after an accepted behaviour or performance change; the baseline diff is reviewed like any other commit.

Deferred (follow-up on the tracker)

Instruction-count backend (iai-callgrind) once valgrind is installed on the measuring host — slots in as a sibling backend without reshaping baselines.
Native-crate project-load sub-cost (dylib SHA-256) — needs a build step in the scratch project.
Thresholded failure, once recorded variance data justifies thresholds.