World/C21: domain-agnostic orchestration substrate — RunReport<M> + abstract the registry metric/deflation vocabulary #147
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Captured from the #136 thread (the engine/domain SoC extraction, cycles 0079/0080). #136 got the trading-analysis leaf out of
aura-engine(RunMetrics, RMetrics, summarize_r, derive_position_events, the position-event types, the deflation hurdle math, and now FamilySelection/SelectionMode all live in the newaura-analysiscrate). Two things were deliberately left, and are the substance of this issue:RunReport/RunManifest/summarizestay trace-coupled in the engine. MakingRunReportgeneric over a metric typeM(and re-boundingsweep/mc/walkforward) so the engine's run-record layer is domain-agnostic too.aura-registrystays trading-aware by design. ItsMetric/metric_cmpvocabulary and — load-bearing —optimize_deflated's statistics branch irreducibly on metric identity (R-bootstrap vs.total_pipsdispersion floor). Making the orchestration/World layer a reusable domain-agnostic substrate would mean parameterisingoptimize/rank_by/deflateover a domain-supplied score + per-metric null-model, with the trading metric semantics + their statistics injected fromaura-analysis.Why deferred (user-ratified 2026-06-27): forcing this now would be a one-implementor abstraction (only the trading metrics implement it), the speculative-generality smell. It only pays off if/when the World layer is genuinely wanted as a reusable cross-domain orchestration substrate — i.e. as part of the World / C21 milestone, not bolted onto a behaviour-preserving SoC refactor.
This is an
idea, no commitment — the durable home for the decision so it is not lost. See ledger C16 cycle-0079 / 0080 realization notes for the rationale.Reconnoitred under a C21-follow run; disposition: keep deferred — precondition unmet (2026-07-01)
This item was pulled into a C21-follow work queue and reconnoitred against the
current tree before any edit. The fork it forces is a single one, and it is a
project-direction bet, not a code-derivable fact:
Fork. Un-defer now and make
RunReportgeneric over a metric type M(re-bounding sweep/mc/walkforward) + parameterize
optimize/rank_by/deflateover a domain-supplied score and per-metric null-model injected from
aura-analysis— vs. keep the deferral until the World is genuinely wanted as areusable cross-domain substrate.
Findings from the tree (grounding the recommendation):
crates/finds zero secondconsumers: no
trait Metric/Score, noNullModel, noRunReport<...>, nonon-trading M — every
RunReportcarries the tradingRunMetrics. Building theabstraction now is the speculative-generality this issue itself names.
RunReportand~72
RunManifestreferences acrossaura-engine,aura-registry,aura-analysis,aura-cli, plus ingest tests/examples.RunReport<M = RunMetrics>default-param edit is mechanically possible and C1-neutral, but it leaves the
hard half untouched:
optimize_deflated(aura-registry/src/lib.rs:415)branches irreducibly on metric identity (R-bootstrap for R metrics vs a
closed-form dispersion floor for
total_pips), so genericity there needs a realper-metric null-model injection, not a type parameter.
Recommendation: keep deferred. The deferral's precondition — a genuinely
wanted cross-domain substrate, ideally a real second domain/consumer — is
objectively unmet in the tree today. Opening this under a behaviour-preserving
pass would ship the one-implementor abstraction the original deferral (ledger
docs/design/INDEX.md:1176-1178, user-ratified 2026-06-27) was written to avoid.Why this needs a human call rather than an autonomous decision: whether to
build the generic substrate in anticipation of a second domain is a strategic
bet about what aura is to become — a preference no source in the tree settles.
The disposition above is a recommendation; un-defer or keep-deferred is the call
to make here.
Deferral ratified by the user (2026-07-02)
Following the disposition in the prior comment (keep deferred — the "reusable
cross-domain substrate" precondition is objectively unmet in the tree: zero second
consumers, ~196
RunReport+ ~72RunManifestuse sites, and the hard halfoptimize_deflatedneeds a real per-metric null-model injection, not a typeparameter), the user's call: "deferred lassen" ("leave it deferred"),
2026-07-02.
RunReport<M>genericity and the domain-agnostic orchestration substrate staydeferred. This issue remains open as the durable home for the decision; revisit
if/when a genuine second domain or a concrete cross-domain need appears.
Deferral split -- partially lifted, user-ratified (2026-07-20)
C28 (the internal-stratification contract, ratified 2026-07-19, commit
1c49d5d) moves the ground under this issue's twice-ratified deferral (2026-06-27 in the body, re-ratified in the 2026-07-02 comment on this issue). The deferral rested on one argument --RunReport<M>plus the registry deflation genericity would be a one-implementor abstraction until a second metric consumer exists. Under C28 that argument now cuts the issue in two:RunReport<M>, re-bounding sweep/mc/walkforward) is no longer speculative. Theaura-engine -> aura-analysisdependency (crates/aura-engine/src/report.rs:20re-exports the R-metrics, position-event and selection types;mc.rscarries R/bias-typed fields inside otherwise domain-free structs) is a ratified layer violation -- the innermost rung importing an outer one. Cutting it is contract obedience; no second domain is required.MeasurementReport,crates/aura-engine/src/report.rs:119).User ratification (2026-07-20): Stratification phases 5 (#291 -- split
aura-analysisinto generic statistics and backtest metrics) and 2 (cut theengine -> analysisedge viaRunReport<M>; behaviour-preserving,runs.jsonlbyte-identical per C18) are un-deferred, in that order -- after the split the engine may legally keep the domain-free statistics dependency, so the edge cut removes only the backtest-metrics half. Item 2 of this issue (C28 phase 6) stays deferred until measurement supplies its first deflatable metric -- a tap-to-scalar reduction carrying its own null model; the concrete candidate shape is recorded in #290.Item 1 shipped; item 2 stays deferred (2026-07-20)
The two-item split recorded in the 2026-07-20 disposition comment above has landed its first half on the run branch
worktree-issue-291-analysis-split(awaiting the user-reserved merge tomain).Item 1 --
RunReport<M>metric genericity -- DONE (#292, commita56ab78).RunReportis now generic over its metric payload;sweep/mc/walkforward/blueprintthread the parameter; the pip/R reductions and the MC assembly (summarize,McAggregate/RBootstrap/monte_carlo) moved toaura-backtest, the statistics kernel (MetricStats/quantile/resample_block/SplitMix64) to theaura-analysisfoundation.aura-engine's[dependencies]are nowaura-core+aura-analysisonly -- theaura-engine -> aura-backtestproduction edge is cut, and the C28 ladder direction (c28_layeringstructural test) is enforced for the engine/backtest/analysis rungs. Behaviour-preserving: 1448/0 tests,runs.jsonlbyte-identical (C18). (Prerequisite phase 5, #291, splitaura-analysisfirst -- commit 94aaa4c.)Item 2 -- registry
Metric/metric_cmp/deflation-vocabulary abstraction -- STAYS DEFERRED. C28 gave it no new motive: the registry sits in the process column, whose trading-awareness the contract permits, andoptimize_deflatedstill branches irreducibly on metric identity (R block-bootstrap vstotal_pipsdispersion floor) with exactly one real metric consumer. A measurement run reports tap names, not metrics, so there is no second implementor yet. This half lifts when measurement supplies its first deflatable metric -- a tap-to-scalar reduction carrying its own null model -- tracked in #290. The original one-implementor rationale (user-ratified 2026-06-27) holds until then.This issue stays open as the durable home for the item-2 decision.
Item 2 scoped against the current tree; the direction is an architectural fork for the user (2026-07-20)
With #290 shipped (the permutation-null IC reduction, on branch
worktree-issue-290-measure-metric), item 2 deferral precondition — a genuine second null-model computation — is now met. Scoping item 2 proper against the tree ata9d36dd, however, shows it is not the focused registry-branch refactor its wording alone suggests. Findings (verified):M = RunMetrics, not generic:optimize_deflated/rank_by/metric_cmp/metric_value/null_best_of_k/member_sdall take the concrete aura-backtest aliases and branch on theMetricenum /is_r_metric(13 metric-identity sites in crates/aura-registry/src/lib.rs). #292 generified the engine layer; the registry layer was untouched.null_best_of_kreachesrep.metrics.r.net_trade_rs(a#[serde(skip)]in-memory field onRMetrics). There is no generic per-member "null-input" slot onSweepPoint<M>/RunReport<M>for a non-R null (IC needs its aligned(signal, forward_return)pairs) to ride.SweepFamilytoday:run_measurementproducesMeasurementReport { manifest, taps }— nometrics: Mfield, structurally incompatible withsweep()boundF: Fn(...) -> RunReport<M>— and no measurement-sweep-assembly function exists (the sweep path is wired only to the broker/R evaluation). #290aura measure icruns on one already-persisted run traces, entirely outside the family/registry machinery.aura-registry::Metric; aura-campaignPER_MEMBER_METRICS/RANKABLE_METRICS/member_metric; aura-researchmetric_vocabulary/is_known_metric;RegistryError::NonRMetrichardcoded prose) — all four are in scope for "parameterise over a domain-supplied score".The fork (unresolved, user-reserved). Making IC flow through cross-trials deflation is not "abstract optimize_deflated branch"; the load-bearing choice is HOW measurement metrics enter the deflation machinery:
This is the same "strategic bet about what aura World/metric substrate is to become" the 2026-07-01 and 2026-07-02 comments on this issue twice escalated to the user. The minimal registry-branch-only refactor, done alone, re-creates the one-implementor abstraction the deferral was written to avoid — only R flows through families, so an abstracted branch would still have a single real family-user. Recommendation (derived lean, not a decision): (C) is the smallest demand-driven step that unifies the two null-model computations now that both exist; (A) is the fullest vision but milestone-scale and should be its own milestone. The direction is the user call. The duplicate-timestamp join hardening (#294) rides with whichever measurement-metric path is chosen.
Direction decision — #147 item 2 scope (ratified)
The owner ratified the direction on 2026-07-20, choosing among the three options recorded earlier in this thread (measurement runs as first-class sweep-family citizens / parallel measurement-side deflation path / shared null-model vocabulary).
Chosen: the "A1" cut — make the registry's deflation dispatch generic over a per-metric null model supplied from the outer rungs, WITHOUT making measurement runs sweep-family citizens. Scope:
RunMetrics(aura-backtest) supplies the best-of-K moving-block null; the Information Coefficient supplies the within-run permutation null. Narrow means: abstract only what the two live implementors demonstrably share; no speculative surface for hypothetical third metrics (per the #136 one-implementor rule).Metric/resolve_metric/metric_cmp/is_r_metric/optimize_deflated/check_r_metric/generalizationand neighbours) become generic over that vocabulary.Metricenum, campaignPER_MEMBER_METRICS/RANKABLE_METRICS/member_metric, researchmetric_vocabulary,NonRMetricerror prose) are retired by sourcing from the shared vocabulary.r_basedas the enforcement point — a meaningless mixed ranking must not pass silently.Explicitly deferred ("A2"): collapsing
MeasurementReportintoRunReport<M>, re-typing the campaign execution engine generic-over-M, and a measurement sweep verb — gated until a concrete family/campaign demand exists.Basis: owner decision, 2026-07-20, following an adversarial code-grounded comparison of the three recorded options (one advocate plus one cross-examination per option, then a comparative synthesis). The deciding findings: the deflation-dispatch genericity is exactly what the ledger's fired #290 trigger licenses (C28 disposition), while family citizenship overshoots it; the two live null models are structurally different statistical objects (family-level best-of-K moving-block bootstrap vs within-run permutation), so the vocabulary must dispatch per metric and never share one formula; and the vocabulary is supplied up from the outer rungs and injected into the registry — never sited in the domain-free core crate.
Status: design settled — ready for spec production.
Design reconciliation (spec production, A1 cut)
Spec: metric-vocabulary. The ratified A1 direction (recorded in this thread as issues/147#issuecomment — "Direction decision — #147 item 2 scope (ratified)", 2026-07-20) leaves seven design forks open at spec level. All seven are derivable from the code, the ledger, and the ratified constraints; each is decided here with its rationale. Derived orchestrator decisions unless marked otherwise.
Fork: trait home → the vocabulary trait lives in
aura-analysis, re-exported throughaura-engine(which already re-exportsresample_block/MetricStatsfrom analysis, so the path has precedent). Basis: derived — the trait must be visible to aura-registry (consumer) and implementable by aura-backtest (RunMetrics); aura-backtest deliberately has no direct aura-analysis dependency and thec28_layeringtest pins its edge set to[aura-core, aura-engine], so the impl reaches the trait through the engine re-export and NO Cargo edge changes anywhere. aura-analysis is the post-run statistics/selection kernel that already hosts the null primitives andFamilySelection.Fork: trait shape (narrow) → associated
Key: Copy;resolve(name) -> Option<Key>;known() -> &'static [&'static str](the roster, single source for error prose);higher_is_better(Key) -> bool;value(&report_payload, Key) -> f64(preserving today's missing-block-reads-NEG_INFINITY rank semantics);has_resampling_null(Key) -> bool(arm selector: resampled null vs dispersion floor);null_draw(&payload, Key, block_len, &mut SplitMix64) -> Option<f64>(ONE draw of this member's null statistic;None= no usable null input). The family best-of-K fold, the p95 subtraction, the Laplace estimator, and the dispersion-floor arm stay generic registry code. Basis: derived — this is exactly the intersection the two live implementors demonstrably share (the R null = centred moving-block resample per member folded best-of-K; the IC null = within-run permutation); anything more (null-parameter objects, cross-member hooks, posterior interfaces) would speculate past N=2, which the ratified "deliberately narrow" constraint and the #136 rationale forbid.Fork: C10 guard mechanics → structural-first. (1) Generic paths (
rank_by/optimize/optimize_plateau/optimize_deflated) resolve metric names strictly within the family's own vocabulary — a cross-vocabulary name is refused asUnknownMetric, whose known-list derives fromknown()so the refusal names the right vocabulary. (2) The R-only gates (check_r_metric,generalization) stay MONOMORPHIC over the R vocabulary — a measurement payload cannot even reach them by type, which is stronger than any flag check. (3) Ther_basedclassification survives as the R vocabulary's own knowledge (in aura-backtest), the enforcement point forNonRMetricrefusals within the R set. Basis: user-ratified "explicit C10 guard, r_based as enforcement point" (2026-07-20), realized structurally; a mixed family is unconstructible (SweepFamily<M>is homogeneous), so the guard's live surface is name resolution + the monomorphic gates.Fork: where the R vocabulary lives → the registry's private 7-variant
Metricenum and its R plumbing (resolve_metric,is_r_metric,higher_is_better, the per-member value/null-input readers) migrate toaura-backtestasRunMetrics's vocabulary impl (theexposure_sign_flipsback-compat alias preserved). Basis: derived — the ledger's stated direction is a vocabulary "supplied by measurement and backtest instead of baked in R-only" (C28); the R vocabulary is trading knowledge and belongs beside the types whose fields it reads.Fork: hand-copy disposition (scope refinement vs the ratification's letter) → the RANKABLE set's three hand-copies (registry resolve list, registry error prose, campaign
RANKABLE_METRICS) collapse onto the vocabulary impl — genuinely single-sourced. The research 17-namemetric_vocabulary()and campaign 14-namePER_MEMBER_METRICSare NOT source-collapsed: aura-research's core-only isolation is itself user-ratified (the #190 "accepted residual"), and those two rosters are different sets serving different surfaces (doc vocabulary incl. selection annotations; gate-predicate scalars). Instead the #190 cross-crate guard test extends to pin every roster against the vocabulary impl, so any drift fails a test loudly. This refines the ratified comment's "four hand-copies retired": two retired by single-sourcing, two retained by ratified isolation but newly oracle-pinned. Basis: derived — resolving the conflict between two user ratifications (retire the copies, 2026-07-20 vs. research isolation, #190) toward the structural one, with the guard closing the drift risk that motivated retirement.Fork: the standalone
aura measure icpath → reroutes onto the shared building blocks: the IC payload type implements the vocabulary trait in aura-cli, and the standalone reduction computes its permutation p via the samenull_draw+ single-sourced Laplace estimator the family path uses. One observable change: null permutations become independent draws (fresh clone per draw) instead of chained shuffles, so emitted overfit probabilities may differ in the last digits from the just-shipped values; determinism (same seed → same output) is preserved and the C18 measure-ic goldens assert self-consistency, not specific values. Basis: derived — without this the IC trait impl would be test-only decoration; with it the second implementor is production-real, which is what the two-implementor abstraction rests on.Fork: signature and provenance stability →
optimize_deflatedkeeps its exact signature (family, metric, n_resamples, block_len, seed) andFamilySelection's serde shape is untouched; a permutation-null family records the passedblock_leneven though permutation does not use it (documented as inert for that vocabulary). The R deflation output must be BIT-IDENTICAL for unchanged inputs (same rng consumption order, centring recomputed per draw yields the same values). Basis: derived — C18 byte-identity is the highest-consequence regression surface; keeping the signature and provenance shape frozen confines the whole change to dispatch internals.Status: forks resolved — spec production proceeding under these decisions.
Spec auto-sign (no human in the loop)
The metric-vocabulary spec — the A1 cut of item 2, scoped by the two decision comments of 2026-07-20 in this thread — was auto-signed on an independent grounding-check PASS. Eleven load-bearing assumptions about current behaviour were each ratified by a currently-green test: the registry's private vocabulary and resolve/refusal prose; the null-model rng schedule and centring (bit-identity surface); the duplicated Laplace estimator; the dispersion-floor arm; the R-only gates; the engine re-export seam and the dependency-edge/no-Cargo-change claims; the campaign/research rosters with their serde-oracle guard; and the measure-ic E2E surface (asserts shape/bounds/self-determinism, not absolute p values — so the permutation-draw-scheme change is unpinned). The signature is the checker's PASS, not orchestrator confidence.
Two non-blocking findings carried into planning: (1) the engine already re-exports SplitMix64, so only the two new names (the vocabulary trait and the Laplace estimator) need adding to the re-export seam; (2) the registry metric/deflation test inventory is ~31 tests, not the 26 the spec's prose counts — the acceptance criterion binds the full suite either way.
Status: spec signed — ready for implementation planning.
Cycle shipped (A1 cut) — awaiting the run-close merge
The ratified A1 cut of item 2 is implemented, audited, and sitting on branch
worktree-issue-147-metric-vocab(three commits on top ofa9d36dd):6744f67— the substrate. The deliberately narrowMetricVocabularytrait (resolve/roster/direction/value/one null draw) plus the single-sourcedone_sided_p_laplaceestimator in aura-analysis, re-exported through aura-engine (no Cargo.toml edge changes; the c28_layering table untouched). The R vocabulary —RunMetricKey(with theexposure_sign_flipsback-compat alias),r_based, theRANKABLE_METRICS/R_BASED_METRICSrosters, the centred moving-blocknull_draw,member_metric_from_rs— migrated into aura-backtest. The registry's rank/optimize/deflate machinery is generic over the trait (optimize/optimize_plateau/optimize_deflatedadditionallyM: Clone); refusal prose derives from the carried roster;check_r_metric/generalizationstay monomorphic on the R vocabulary — the C10 wall by type. Bit-identical for R inputs: the registry suite (75 tests) passes unchanged apart from five syntax-only edits for theUnknownMetricvariant reshape.0651e16— the consumers. aura-campaign'sRANKABLE_METRICSis now a re-export of the vocabulary roster, and the #190 guard test extends to pin the roster nesting (rankable == vocabulary roster ⊆ per-member ⊆ research vocabulary) and the legacy alias;IcMetricsin aura-cli is the second production implementor (within-run permutation null), and the standaloneaura measure icreduction now computes its p via the shared building blocks (independent permutation draws — statistically equivalent, deterministic per seed, last-digit p differences vs #290 possible). Acceptance proven at the registry boundary: an engineeredSweepFamily<IcMetrics>deflates throughoptimize_deflatedto a LOW overfit probability and a noise family to a HIGH one; same seed reproduces the identicalFamilySelection; cross-vocabulary names are refused with the family's own roster in the prose. A new CLI E2E exercises the legacy alias through the realaura runs family … rankboundary.2cf4574— the cycle audit. Architect-confirmed holds: C18 bit-identity, the C10 wall (no leak found), the C28 supplied-from-outer-rungs direction, C1 determinism through the generic path. All drift items fixed in the audit commit: the ledger's C28 disposition now records item 2 SHIPPED (this cut) with A2 explicitly still deferred; the #136 one-implementor clause carries its supersession note (the IC is the second implementor); seven stale doc comments updated;R_BASED_METRICSoracle-pinned againstRunMetricKey::r_based().Process notes for the record: one plan defect surfaced mid-cycle (missing
M: Clonebounds and an unused-import contradiction with the clippy gate) — diagnosed by the loop as spec-ambiguous, fixed at the plan level, and the working-tree implementation verified independently before commit; one compile-forced deviation (IcMetricsadditionally derivesDebug). Cross-cycle verification: full workspace suite green, clippy-D warningsclean, C18 goldens unchanged. No regression scripts are configured (the bench is report-only), so the architect review is the drift gate; a downstream-consumer fieldtest was judged not applicable — the user-facing CLI surface is unchanged and the new surface is in-workspace API, exercised by the acceptance tests.Deferred, deliberately ("A2"): measurement runs as sweep-family citizens (report unification, campaign engine generic-over-M, a measurement sweep verb) — gated until a concrete family/campaign demand exists.
The
closesmarker on0651e16fires when the branch is merged and pushed (a user-reserved act) — this issue correctly stays open until then.Status: cycle complete — branch awaiting review and the ratified merge.