Extract the trading analysis layer (report.rs) out of aura-engine; keep the engine domain-agnostic #136
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Motivation
aura-engineis named and documented as "the headless, UI-agnostic reactive SoAengine", and its hot path genuinely is domain-free: per C7 it streams only the
four type-erased scalars and never names
exposure/R/equity, and C14 keeps itUI-agnostic. But the crate also ships a trading-domain analysis layer that has
nothing to do with the dataflow engine, and the orchestration types are nailed to
it. aura is "a game engine for traders" — trading is the game, so agnosticism
is a means (a fast, deterministic, composable substrate), not an end — but SoC
is still worth keeping where it costs little: the engine should be agnostic as
far as possible, and the analysis layer is separable.
The drift is also self-reinforcing, and that is the trigger for filing this now.
Cycle 0068 (#115) placed
derive_position_eventsinaura-enginewith therationale "sibling of
summarize_r" / "besidesummarize" — i.e. "the tradingcode is already there, so more of it belongs there too." That is not a design
rationale; it is the existing wart justifying its own growth (cf. CLAUDE.md
"Design rationale ≠ implementation effort" — consistency-with-an-existing-placement
is the same category of non-reason). Each domain addition makes the next one look
natural. Left alone,
aura-enginekeeps accreting the trading stdlib.Validated: what is domain vs generic in
crates/aura-engine/src/report.rsTrading-domain (should leave the engine):
RunMetrics(report.rs:17) —total_pips,max_drawdown,bias_sign_flips,r: Option<RMetrics>RMetrics(report.rs:45) —expectancy_r,sqn,sqn_normalized,win_rate,profit_factor,max_r_drawdown,net_expectancy_r,conviction_terciles_r, …summarize(report.rs:369) — pip-equity + bias reductionsummarize_r(report.rs:89) — R-metric reductionPositionAction(report.rs:280) +PositionEvent(report.rs:314)derive_position_events(report.rs:231)Domain-free (stays — generic SoA / trace tooling):
ColumnarTrace(report.rs:487),join_on_ts(report.rs:462),JoinedRow(report.rs:443),
f64_field(report.rs:425) — generic columnar/trace utilitiesRunManifest(report.rs:328) — generic run identity (commit, params, window,seed); carries a free-text label that today happens to describe a broker
(judgement call at plan time whether the label stays free-form here)
RunReport(report.rs:348) — the C18(manifest, metrics)pair; generic onceparameterised over the metric type (today concrete on
RunMetrics)Validated: the coupling (why this is non-trivial, not a file move)
The concrete
RunMetricsis woven into the generic orchestration and theregistry, so extraction means parameterising the engine over a metric type:
crates/aura-engine/src/{sweep.rs (18), mc.rs (21), walkforward.rs (24)}— theSweepFamily/McFamily/WalkForwardResultfamilies and their optimiseclosures carry
RunMetrics.crates/aura-registry/src/{lib.rs (36), compat.rs (10), lineage.rs (6)}— theC18 run store loads/stores
RunReport/RunMetricsand owns themetric_cmp/rank vocabulary (itself domain:
sqn,expectancy_r,total_pips, …).crates/aura-cli/src/main.rs (66)— the largest consumer; foldssummarize/summarize_r, buildsRunReport, ranks families.Proposed direction (settle in a spec, not here)
aura-analysis(name TBD):RunMetrics,RMetrics,summarize,summarize_r,PositionEvent,PositionAction,derive_position_events.Depends on
aura-core(it reads type-erasedScalarrecords positionally, asit already does — so it does not link
aura-std, same discipline as today).aura-enginekeeps the generic trace utilities + a metric-genericRunReport<M>and orchestration (sweep/mc/walkforward) over a boundedgeneric
M(the bound covers serde + whatever the rank/compare surface needs).aura-registryeither parameterises overMtoo, or the domain rankvocabulary (
metric_cmp) moves toaura-analysis; decide at spec time.aura-cli+ the test crates instantiateM= the trading metric.Invariant fit / hard constraints
runs.jsonlshape muststay byte-identical (existing records still deserialise). A generic
Mmustserialise to the same JSON — the
serde(default)/skip_serializing_ifdiscipline already in
RunMetrics/RMetricshas to survive the move.touch it. Behaviour-preserving (C1): a pure refactor + re-parameterisation,
no metric value changes, all existing goldens/serde tests stay green.
sweep/mc/walkforwardand theregistry — this is the bulk of the work and likely spans 2-3 cycles (consider
promoting to a milestone with its own spec).
Out of scope
Acceptance
aura-engine.aura-engine's[dependencies]and public surface name no trading concept(the orchestration +
RunReportare generic over the metric type).aura-enginestill builds with onlyaura-core+serdedeps; the generictrace utilities (
ColumnarTrace/join_on_ts/f64_field) remain.runs.jsonlback-compat: existing records still deserialise, on-disk JSONbyte-unchanged (C18).
cargo build/test --workspace+clippy -D warningsclean; no metric valuedrift (goldens unchanged).
refs #115 (the placement whose "sibling of summarize_r" rationale is the creep this addresses)
/bossrun — decision log (cycle 1 of this extraction)Provenance. Entered
/bosson this issue by explicit user instruction("#136 — autonomously"). This run is anchored to session anchor
3e8fd73(last user-ratified
mainHEAD). A pre-iteration survey (architect driftreview + readiness recon of the open candidates + adversarial challenge)
recommended sequencing this SoC extraction before the only open milestone
(Stage-2 realistic broker), on substance: the broker milestone authors
currency-metrics + position-event code whose natural home is exactly
report.rs, and #121 already mandates the#[serde(default)]back-compatdiscipline this boundary establishes — so erecting the seam first means
Stage-2 lands in the right crate by construction rather than deepening a wart
a later sweep must unpick field-by-field.
Entry path:
compiler-driven(not the heavyspecify → planner → implementloop). The cycle-1 cut is a behaviour-preserving definition relocation the
type checker enumerates; the existing C18 on-disk byte-identity goldens
(
runs.jsonlshape, the full-object asserts incli_run.rs, the legacy-lineloads in
aura-registry) are the safety net. Verdict = clean workspace buildAND suite green unchanged.
Cycle-1 scope (derived, fork-free). Stand up a new crate
aura-analysis(dep:
aura-coreonly) and relocate the pure trading-domain leaf out ofcrates/aura-engine/src/report.rs:aura-analysis:RunMetrics,RMetrics(+ its hand-writtenPartialEq),PositionAction(+From/TryFrom<i64>),PositionEvent,summarize_r,r_metrics_from_rs,derive_position_events, and thedeflation stats helpers
inv_norm_cdf/expected_max_of_normals(+ theprivate
r_colindices andSQN_CAPthey pull). Verified: none of thesereference the engine-local trace utils in their bodies, so
aura-analysisneeds only
aura-core— noanalysis → engineedge, no dependency cycle.aura-engine: the generic trace plumbing (ColumnarTrace,join_on_ts,JoinedRow,f64_field,kind_tag,scalar_to_f64),RunManifest,RunReport, andsummarize(it bridges recorded tracecolumns into
RunMetrics, so it is trace-coupled — kept at the boundarythis cycle).
report.rspub uses themoved symbols back into its own namespace, so
aura-engine'slib.rsre-export block and every
crate::report::*/aura_engine::*consumer(
aura-registry,aura-cli,aura-composites,aura-ingest, and thein-crate tests) compile unchanged. The only edited files are the new crate,
the workspace
Cargo.tomlmember list,aura-engine'sCargo.toml(newdep), and
report.rsitself. The moved functions' unit tests stay inreport.rsthis cycle (exercising the re-exported symbols) to keep the testsurface byte-unchanged; relocating them is a later cosmetic pass.
Deferred to later cycles (the genuine design forks — NOT this cycle):
making
RunReportgeneric over a metric typeMand re-boundingsweep.rs/mc.rs/walkforward.rs; relocating theaura-registryMetric/metric_cmp/ rank vocabulary (the "registry-over-M vs.metric-moves" fork the issue itself leaves open); and relocating the
mis-placed orchestration types
FamilySelection/SelectionModeout ofreport.rs(a dependency-direction tangle — registry already depends on theengine — best decided with the registry-vocabulary fork). Those carry real
choices and route through
specifywhen reached.Adjacent. The ledger's C10 cycle-0065 note is stale on where composites
live (says
aura-engine; they moved toaura-composites,aura-stdis now adev-dep) — sync at this cycle's audit.
/bossdecision (cycle 1) —r_colprivate-module coupling forces test relocationThe first extraction attempt surfaced a hole the cycle-1 plan missed: report.rs's
#[cfg(test)]helpers (r_row/r_row_full/pm_row, 16 references) reach theprivate
mod r_colcolumn indices directly — not through the public movedfunctions. A
pub usebridge cannot re-export a private module, so the original"keep all tests in report.rs, reach the moved symbols via the re-export" framing is
unworkable once
mod r_colmoves.Decision (derived, not a brainstorm fork): tests move with their subject. The
moved functions' unit tests AND their private helpers (
r_row/r_row_full/pm_row) ANDmod r_colrelocate intoaura-analysisalongside the code theyexercise; report.rs keeps only the tests for the symbols that stay (
summarize,RunReport,RunManifest,FamilySelection/SelectionMode, the trace utils, theend-to-end harness test). This is a faithful relocation — assertions byte-identical,
just hosted in the crate where the subject now lives — and is strictly better than the
alternatives: making
r_colpublic would leak an internal column-index contract(deliberately private per its own doc); duplicating the constants would split a
single-source-of-truth.
The earlier "zero test-file churn" framing is dropped — it was an accommodation for the
compiler-driven-editworkflow's strict verify gate (which mis-fired and was abandoned;see Skills #11), not a project constraint. The C18 on-disk byte-identity goldens are
unaffected: they live in
aura-cli'scli_run.rs, theaura-registrylegacy-linetests, and the engine integration tests under
tests/— never in report.rs's unitmodule — and the re-exported types serialise byte-identically, so those goldens stay
green.
Partition (verified against the current test module):
aura-analysis:inv_norm_cdf/expected_max_of_normalstests; theposition_event*serde +PositionActiontests; thesummarize_r_*tests; thederive_*tests; helpersr_row/r_row_full/pm_row; the privatemod r_col.runmanifest_*,family_selection_*,report_is_deterministic_end_to_end(+
build_two_sink_harness/run_once), thesummarize_*(total-pips / drawdown /sign-flips) tests,
to_json_renders_the_canonical_form, helpersamples.Cycle 1 landed (commits
94c88ebextraction +a8f67c7audit). The pure analysis leaf (RunMetrics, RMetrics, summarize_r, r_metrics_from_rs, derive_position_events, PositionEvent/PositionAction, the deflation hurdle math) now lives in a new aura-analysis crate (aura-core + serde only; serde_json is a test-only dev-dep). aura-engine pub-re-exports for source compatibility, all four downstream crates byte-unchanged, full suite 665/0 incl. the C18 goldens, clippy clean. Ledger synced (a C16 cycle-0079 realization note + the stale C10 crate-topology spots). The key sequencing win is captured: the broker milestone's currency-metric additions (#121) now extend RunMetrics in aura-analysis by construction.Remaining #136 tail (a new, fork-bearing cycle — deferred, not started): relocate the aura-registry Metric/metric_cmp/rank vocabulary; make RunReport generic over the metric type M and re-bound sweep/mc/walkforward; relocate the mis-placed FamilySelection/SelectionMode. This carries the open registry-over-M fork the issue body flagged: does aura-registry parameterise over a Metric trait (generic), or just depend on aura-analysis for the concrete Metric vocabulary (simpler)? Decide at spec time.
/bossdecision (cycle 2) — registry stays domain-aware; #136 remainder is the FamilySelection/SelectionMode relocation onlyThe fork — and who decided it. The deferred #136 tail was framed as "make
RunReport generic over a metric type M and relocate the aura-registry
Metric/metric_cmp vocabulary". Inspecting the code showed the registry's
domain coupling is not in the vocabulary but in the algorithms:
metric_value(lib.rs:137) reaches intoRunMetrics/RMetricsfields, andoptimize_deflated(lib.rs:386) branches onis_r_metric— R metrics get amoving-block bootstrap,
total_pipsa closed-form dispersion floor. So "movethe vocabulary to aura-analysis, registry depends on it" would not decouple
the registry; it would stay thoroughly trading-aware.
This reframed the question to an architecture-intent fork the sources do not
settle: should the orchestration/World layer (aura-registry) be a reusable,
domain-agnostic substrate (symmetric with the engine), or is it legitimately
trading-aware? Even full genericity would not erase the domain knowledge — the
R-vs-pips deflation statistics are irreducibly domain and would have to be
injected via a
traitwith exactly one implementor (the trading metrics) — thespeculative-generality smell.
Decision (user-ratified in the session discussion, 2026-06-27 — provenance, not
a derived fork): the registry is legitimately trading-aware; the
Metric/metric_cmp/optimize_deflatedvocabulary and statistics stay inaura-registry (it is the trading selection layer, not the domain-free engine).
Genericity is not forced now (a one-implementor abstraction). "The World as a
domain-agnostic orchestration substrate" is deferred to the World / C21
milestone, where it would actually pay off — not bolted onto #136.
This cycle's scope (the genuine #136 remainder, small & behaviour-preserving):
relocate the orchestration-provenance types
FamilySelection/SelectionModeout of the engine's
report.rsintoaura-analysis(the only non-cyclic homethat gets them out of the engine —
RunManifest.selectionneeds the type and theengine already depends on aura-analysis; the registry/CLI reach them unchanged via
aura-engine's re-export). This finishes getting the trading types out of the
engine.
RunManifest/RunReport/summarizestay in the engine this cycle(trace-coupled; their genericization is the deferred World-layer work, not #136).
Entry path: compiler-driven (behaviour-preserving relocation), executed via a
plain implementer rather than the compiler-driven-edit workflow (which no-ops on
crate-relocation — Skills #11). Verdict: clean build + suite green unchanged
(the family_selection_* tests + the C18 goldens are the oracle).
Closing — engine-side extraction complete (cycles 0079 + 0080)
The trading-analysis layer is out of
aura-engine. Across two behaviour-preserving cycles (full suite 665/0 throughout, incl. the C18 goldens; clippy clean;aura-engine/src/lib.rsand all downstream crates byte-unchanged via re-export):94c88eb+ audita8f67c7): newaura-analysiscrate (depsaura-core+ serde only;serde_jsontest-only). Moved:RunMetrics,RMetrics,summarize_r,r_metrics_from_rs,derive_position_events,PositionEvent/PositionAction, and the deflation hurdle mathinv_norm_cdf/expected_max_of_normals.04c581a+ audit226bf57): moved the last trading types still defined inreport— the selection-provenanceFamilySelection/SelectionMode.Adjudicated (not done — decided), so this issue closes:
aura-registryMetric/metric_cmp/ deflation vocabulary stays in the registry by design — it is the trading selection layer, not the domain-free engine, andoptimize_deflated's statistics branch irreducibly on metric identity (R-bootstrap vs.total_pipsdispersion floor). Forcing genericity would be a one-implementor abstraction. (User-ratified, 2026-06-27.)RunReport/RunManifest/summarizestay trace-coupled in the engine.Making the orchestration/World layer a reusable domain-agnostic substrate (
RunReport<M>+ the registry vocabulary behind an injected score/null-model abstraction) is reframed to the World / C21 layer, where it would actually pay off — tracked as #147, not #136.Ledger: C16 cycle-0079 / 0080 realization notes carry the rationale and the current crate topology.