f26b33f3453e2117c38432a65d3ac15f3c20fbcd
497 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brummel
|
f26b33f345 |
spec: 0086 cost-flag CLI ergonomics (boss-signed)
Settled design from #153 (fieldtest friction of milestone #148): make the `aura run` cost-flag surface honest — a units/constraints note in the usage, a named non-negativity diagnostic, and a refuse-don't-guess rejection of cost flags on a non-R harness (the spec_gap; decided reject/exit-2, recorded on #153). CLI-ergonomics + one C10 ledger note only; no cost-model/R-math change. Boss-signed: grounding-check PASS (independent fresh-context agent; 6/6 load-bearing assumptions ratified by currently-green tests). refs #153 |
||
|
Brummel
|
dcf58c5d26 |
docs(project-layout): retire exposure/realistic-broker framing → bias/R/cost-model
The "day in the life" worked example was stale on the #117 (exposure→bias) and #116 (realistic broker retired → cost-model graph in R) reframes. Rewrite steps 3/4/7 to the live model: unsized bias stream → in-R risk-executor + R-evaluator (E[R]/SQN) → optional cost-model graph (net R = gross R − cost-in-R); money/real-broker pushed to the live deploy edge; the matrix axis swapped from the retired {sim-optimal, pepperstone} broker pair to a {fixed-stop, vol-stop} risk-executor axis. closes #151 |
||
|
Brummel
|
d8946b1d7b |
docs(ledger): correct stale realization notes — C12 sharing, aura-std, C15 SessionNode
Three drift items surfaced by the 2026-06-29 design-goal audit, all recording shipped reality into the ledger (no contract change): - C12: cross-sim `Arc<[T]>` sharing is no longer "still open" — the sweep/MC/walk-forward families build members over one shared `Arc<DataServer>`/`FileCache`, so a window is parsed once and shared zero-copy across the disjoint sims (aura-ingest/src/lib.rs:316). - open-threads: `aura-std` is no longer "doc-only" — ~30 node modules. - C15: record that `Session` deliberately ships only `bars_since_open` (the node's own contract), the other two streams deferred (refs #154). |
||
|
Brummel
|
8d04a845ee |
docs(glossary): align terminology with industry standard
Reconcile 68 terms against algotrading frameworks (LEAN, NautilusTrader, backtrader, VectorBT, Zipline), quant-validation literature (Van Tharp, Bailey & Lopez de Prado, White), and stream/dataflow vocabulary. Conservative pass: extend Avoid lists with industry synonyms/homonyms and fold short disambiguation notes into definitions; no canonical headings renamed. Touched 18 entries. Avoid additions: overfit probability (PBO/CSCV), signal (entry/exit signal, trade signal), position table (transactions, trade blotter), conviction-based risk allocation (risk budgeting/parity/ contribution), run registry (model registry), session node (session window), sign-agreement (market breadth). Disambiguation notes: broker vs fill/account simulator, edge (wiring/trading/deploy senses), cycle (clock step vs DAG loop), manifest vs Cargo.toml config, experiment vs MLflow container, run-count (version counter), bias vs statistical bias, bootstrap vs statistical bootstrap, deflated score anchored on the Deflated Sharpe Ratio, Monte-Carlo (input not trade sequence), World vs Unreal UWorld. Deferred (out of scope this pass): heading renames, a new `record` entry + composite sense-split, the playground egui->web wording, the SoA Avoid line. |
||
|
Brummel
|
dff99c93e4 |
fieldtest(milestone): cost-model graph (#148) — capability delivered green
Milestone-close gate for "Cost-model graph (in R)" (#148). The fieldtester exercised the milestone capability end-to-end from the public CLI only (no implementation source), on the stage1-r harness, against ledger C10 + the glossary. Roll-up: friction_found — 0 capability bugs; the milestone promise is empirically delivered: - bare gross-R run is clean (net == gross, no cost forced); - per-trade cost: gross pinned, net drops strictly/monotonically, net_r_equity tap persists; - the capstone per-held-cycle carry BLEEDS the net-R curve continuously over a hold (the gross-net gap grows 0.367 -> 0.734 -> 1.101 -> 1.468 across one hold, vs the per-trade run's flat 0.367) — the holding-cost shape change, observable end-to-end; - composition is exact-additive (the composed drag equals the sum of the single-cost drags to the last ULP) and `aura chart` renders the real net_r_equity series. Evidence under fieldtests/milestone-cost-model-graph/ (run_scenarios.sh + captured_outputs.txt + traces/ + the charted bleed + FINDINGS.md). Non-blocking findings filed forward (none gates the close): - cost-flag CLI ergonomics — units hint, negative-rate diagnostic, non-R-harness no-op + a C10 spec_gap -> #153; - docs/project-layout.md describes the retired `aura backtest --broker` path and omits the cost flags -> folded into #151. refs #148 |
||
|
Brummel
|
84ec2ab5d1 |
audit(0085): cycle close — drift-clean; C10 cycle-0085 realization note
Cycle 5 of milestone #148 (per-cycle-held accrual + CarryCost). Architect drift review over 39f9387..f5e00a9: status clean, no code drift. What holds (architect, evidence-of-review): - C11 byte-identity: the AtClose eval arm is the pre-cycle-5 tokens verbatim (if closed { per } / if open { per }, the per + cum lines unchanged), so the cycle-0083/0084 net_expectancy_r goldens (-614.3134020253314, -615.0304388396047) stay byte-identical and the AtClose unit regressions are untouched. - Accrual sound, no double-count: aura-analysis (summarize_r) is genuinely unchanged. During a hold the carry lives only in open_cost_in_r (cum=0); at close it moves to cum while open_cost->0 (open/closed mutually exclusive per cycle), so net_r_equity subtracts cum+open singly. summarize_r reads the close-row dumped total + the window-end accrued-so-far — one charge per trade, acc reset proven. The 3-field cost record's semantics generalize cleanly to accrual. - C9/C16/C23: the diff touches only aura-std + aura-cli; aura-engine stays domain-free. CarryCost is a CostNode factor, CostRunner a plain Node, composed via cost_graph/CostSum. label() carries its param. All three run_stage1_r call sites threaded; the negative-rate guard is symmetric with the sibling cost flags. Resolved this commit (the cycle-close ledger sync): - C10 realization note (cycle 0085) added under docs/design/INDEX.md — C10's "per-cycle-held factors accrue over the hold" clause is now first realized (the prior 0084 note covered the cost-graph composite-builder). Regression gate: cargo test --workspace (0 failures), cargo build --workspace, cargo clippy --workspace --all-targets -- -D warnings clean — the architect is the primary gate (no dedicated regression script). Ephemera (spec 0085 + plan 0085) git rm'd. Milestone "Cost-model graph (in R)" remains OPEN. refs #148 |
||
|
Brummel
|
f5e00a9c72 |
feat(0085): per-cycle-held accrual — CarryCost + ChargeMode (approach B)
Cycle 5 of milestone #148 (cost-model graph in R): the per-cycle-held accrual mechanism — C10's "per-cycle-held factors accrue over the hold". A carry/funding cost is charged for every cycle a position is held, so the net_r_equity equity curve bleeds continuously over the hold (approach B, "ja, mach B") rather than stepping at close. Mechanism (logged on #148): ChargeMode { AtClose, PerHeldCycle } is a property of the cost factor, read by the one shared CostRunner (no second runner type). The AtClose arm is the pre-cycle-5 eval tokens VERBATIM (IEEE-754 byte-identity). The PerHeldCycle arm accrues `per` into `acc` each held cycle, dumps the accrued total into `cum` at close (resetting acc), and marks the open position via a growing open_cost_in_r. The bleed lives in open_cost_in_r, which the net_r_equity tap already subtracts — so summarize_r and the CLI net_eq wiring are UNCHANGED, and the cycle-0083/0084 net_expectancy_r goldens (-614.3134020253314 flat, -615.0304388396047 composed) + the C18 no-cost golden stay byte-identical. CarryCost is a ConstantCost twin differing only in charge_mode() -> PerHeldCycle (a labelled stress parameter, the flat base of the accrual family). Wired through the existing cost_graph/CostSum aggregation; a per-trade ConstantCost and a per-held-cycle CarryCost compose in one run (the costs sum per-field — the composed golden is exactly the sum of the two single-cost charges). New run-path --carry-per-cycle flag (sweep/walkforward/mc pass None, as the existing cost flags do). Tests: 2 RED-first unit B-proofs (open_cost grows over the hold, dumps the total at close, acc resets between trades — the discriminator a scalar net_expectancy_r cannot see); the CarryCost twin's 5 unit tests; 2 captured net_expectancy_r goldens (-768.2095026600887 carry, -1383.7939051991182 composed); a net_r_equity-bleeds-over- the-hold integration test (the terminal open hold's r_equity-net_r_equity gap strictly grows); plus negative-rate-refused-exit-2, monotone-in-rate, and zero-rate-exactly-free guards. Verified: cargo test --workspace (0 failures), cargo build --workspace, cargo clippy --workspace --all-targets -- -D warnings clean. summarize_r / aura-analysis untouched. Deferred to later #148 cycles (logged there): notional-based carry, the calendar-aware overnight swap, sweep-path cost. refs #148 |
||
|
Brummel
|
2a8ee86489 |
plan: 0085 per-cycle-held accrual + CarryCost
Parent spec docs/specs/0085 (
|
||
|
Brummel
|
19ba1ec978 |
spec: 0085 per-cycle-held accrual + CarryCost (boss-signed)
Cycle 5 of milestone #148 (cost-model graph in R): the per-cycle-held accrual mechanism — C10's "per-cycle-held factors accrue over the hold" — exercised by a constant per-held-cycle CarryCost node and a --carry-per-cycle CLI flag. Approach B (user-decided, "ja, mach B"): the headline net_r_equity curve bleeds continuously over each hold, not stepping at close. Mechanism (derived, logged on #148): the bleed lives in open_cost_in_r (grown each held cycle = accrued-so-far mark-to-market); cost_in_r dumps the accrued total into cum at close. The net_r_equity tap already subtracts both cum_cost_in_r and open_cost_in_r, so the curve bleeds with NO change to summarize_r or the CLI net_eq wiring — and the AtClose runner branch is kept verbatim, so every cycle-0083/0084 net_expectancy_r golden stays byte-identical. ChargeMode is a property of the factor read by the one shared CostRunner (no second runner type); CarryCost is a ConstantCost twin differing only in charge_mode() -> PerHeldCycle. Auto-signed under /boss on a grounding-check PASS (all 6 current-behaviour assumptions ratified against named green tests). Reference issue #148 carries the A/B fork (user decision), the minimal-scope decision (derived), and the mechanism refinement (derived). refs #148 |
||
|
Brummel
|
39f9387c57 |
docs(glossary): add 'tap' — named recorded sink stream
Record-reality add: 'tap' is in consistent use across the code (ColumnarTrace.tap, the --tap flag, the equity/exposure/net_r_equity/ r_equity taps) and the design ledger, but had no glossary entry. The entry also draws the line the 'tap/sink' shorthand blurs: a sink is the recording-role node, a tap is the named stream it records. |
||
|
Brummel
|
face1879a6 |
audit(0084): cycle close — drift-clean (code); C10 cycle-0084 note + doc fixes
Cycle 4 of milestone #148 (cost-graph composite-builder). Architect drift review over fc52b4f..d5c44dd: drift_found, every item dispositioned. What holds (architect, evidence-of-review): C16/C9 — cost_graph lives in aura-composites, the factors in aura-std, aura-engine stays domain-free; the composite is ordinary downstream nodes. C11 behaviour-preservation is structural, not just asserted — the composite inlines to the same flat fan-in as the deleted CLI block (identical node set/order, cost[k].field slot map, geometry fan, net_eq terms/signs, cost_rec cols), so the cycle-0083 net_expectancy_r goldens stay byte-identical. C23 — role/port names non-load-bearing; 4 fixtures pin the role-set + output triple. Resolved this commit: - [high] ledger C10: the cycle-0083 note's "Still deferred (decision E)" is removed (E shipped this cycle) and a cycle-0084 realization note added. - [medium] cost_graph .leak() provenance (aura-composites/src/lib.rs): the "risk_executor precedent" was imprecise — that .leak() lives in risk_executor's TEST (one-shot, leak-safe), not its production builder. Doc corrected: the leak is fine for one-shot run-path construction but must be interned (the COL_PORTS production pattern) before cost reaches the sweep path. - [low] cli_run.rs golden docstring cited the deleted `slot * COST_WIDTH + f` CLI wiring; updated to the cost_graph composite's per-node cost[k].<port> wiring. Deferred (tracked, not pending) -> #152: - [medium] the cost[k].<port> index-namespacing restated across CostSum / cost_graph / the CLI (a build-validated lockstep — a divergent name fails loudly at g.build(), NOT the silent positional kind 0083 collapsed), AND the per-build .leak() that becomes an allocation regression on the per-member sweep path. Both are harmless on the current run-only cost path; the proper fix is one sweep-safe single-source cost-port-name contract (interned), best built with the deferred sweep-cost cycle. Regression gate: cargo test --workspace (0 failures), clippy --workspace --all-targets -- -D warnings clean, cargo doc clean — the architect is the primary gate (no dedicated regression script). Ephemera (spec 0084 + plan 0084) git rm'd. Milestone "Cost-model graph (in R)" remains OPEN. refs #148 |
||
|
Brummel
|
d5c44dd1ea |
feat(0084): cost-graph composite-builder
Cycle 4 of milestone #148 (Cost-model graph in R), decision E. New `cost_graph(Vec<PrimitiveBuilder>) -> Composite` in aura-composites: it fans the 4 PM-geometry inputs to N cost nodes, surfaces each node's extra inputs (discovered via schema introspection past GEOMETRY_WIDTH) as `cost[k].<port>` roles, sums them through CostSum, and exposes the 3-field aggregate. Replaces the CLI's manual slot-indexed cost-wiring + the hardcoded MAX_RUN_COST_NODES=2 cap with one principled composite of arbitrary arity. GEOMETRY_WIDTH re-exported from aura-std (first cross-crate consumer). Runtime port names .leak()'d (the risk_executor precedent), a bounded one-time cost at blueprint construction. Behaviour-preserving at the value level: the composite inlines at bootstrap (C11) to the same flat fan-in, so the two cycle-3 net_expectancy_r goldens (-614.3134020253314 flat, -615.0304388396047 composed), the C18 no-cost golden, and the full suite stay green verbatim. Honours C9 (a composite of cost nodes is ordinary downstream nodes), C16 (wiring lives in aura-composites, not aura-engine), C23 (label drift under cost_graph nesting is permitted; no with-cost graph label/shape golden exists). Four aura-composites unit tests pin the exposed contract: the two planned (n=2 heterogeneous role set; n=1 no-extra shape) plus two the implementer added that strengthen the headline — arbitrary-arity per-node namespacing (n=3, two VolSlippageCost -> distinct cost[1]/cost[2].volatility, the property that retires the 2-node cap) and geometry fan-once (n=2 extra-free -> only the 4 geometry roles). Accepted nit (held, non-gating): the CLI reconstructs the `cost[k].volatility` role name to feed the vol node's extra input — a string coupling to the composite's public role-naming convention. Kept: role-based composite wiring addresses roles by name (as the CLI already does for "closed"/"bias"/"price"); the convention is tested + build-validated; a decoupling accessor would widen this cycle's scope. Verified: cargo test --workspace (0 failures), clippy --workspace --all-targets -D warnings clean, cargo doc -p aura-composites clean. refs #148 |
||
|
Brummel
|
550d5ce947 |
plan: 0084 cost-graph composite-builder
Two tasks: (1) the cost_graph(Vec<PrimitiveBuilder>) -> Composite builder in aura-composites + the GEOMETRY_WIDTH re-export from aura-std + two unit tests pinning the exposed input-role set and the 3-field output; (2) the aura-cli stage1_r_graph rewire onto cost_graph, deleting MAX_RUN_COST_NODES and COST_SUM_PORTS. Runtime port names .leak()'d (the risk_executor precedent). Behaviour-preserving: the cycle-3 net_expectancy_r goldens are the regression net. refs #148 |
||
|
Brummel
|
9fb3d8b8ee |
spec: 0084 cost-graph composite-builder (boss-signed)
Cycle 4 of milestone #148 (Cost-model graph in R), decision E — the cost-graph composite-builder, recorded across cycles 2-3 as the natural next. Adds a `cost_graph(Vec<PrimitiveBuilder>) -> Composite` builder in aura-composites that fans the 4 PM-geometry inputs to N cost nodes, surfaces each node's extra inputs as `cost[k].<port>` roles (schema introspection past GEOMETRY_WIDTH), sums them via CostSum, and exposes the 3-field aggregate. Replaces the CLI's manual slot-indexed cost-wiring + the hardcoded MAX_RUN_COST_NODES=2 cap with one principled composite of arbitrary arity. Re-exports GEOMETRY_WIDTH from aura-std (first cross-crate consumer). Behaviour-preserving at the value level: the composite inlines at bootstrap (C11) to the same flat computation, so the two cycle-3 net_expectancy_r goldens, the C18 no-cost golden, and the full suite stay green verbatim. Routed spec-driven (specify direct-entry): settled in #148's scope + the mature aura-composites idiom + the CostNode contract. All four forks (composite API shape, extra-input namespacing, GEOMETRY_WIDTH visibility, CLI consumption) are derived decisions logged on #148; none hangs on pure user preference. Auto-signed under /boss on the Step-5 grounding-check PASS (every load-bearing current-behaviour assumption tied to a named green test or current source; no with-cost graph label/shape pin contradicts the C23-permitted label drift). refs #148 |
||
|
Brummel
|
fc52b4fced |
audit(0083): cycle close — drift-clean (code); C10 cycle-0083 note + lib.rs doc
Architect drift review over 6c7f5dd..HEAD. No regression scripts in this project → the architect is the sole gate; verdict: shipped code byte-clean. What holds (verified against the diff): - C9 — a CostRunner<F> is a plain downstream Node; the two cost nodes are thin factors wired through the same cost_node_builder. No runtime sub-object. - C18 / byte-identity — the numerator/latched token form is preserved verbatim; the two new CLI goldens pin exact net_expectancy_r and pass; the no-cost golden is untouched. - C10 co-temporality + C23 + invariant-4 — the gate-on-PM-geometry-only rule (cold factor input → 0 cost, row still emits) is now centralized once in the runner; name() dropped cleanly (label() remains the non-load-bearing symbol); the 3-field triple is a single source (COST_FIELD_NAMES) read by both producers + CostSum + the CLI — the cycle-0082 by-convention lockstep is structurally gone. Resolution: - [fix] docs/design/INDEX.md C10 — added the cycle-0083 realization note (each prior cost cycle got one; the deferred general CostNode trait is now shipped and the lockstep eliminated). Ledger deferred-work tracking re-synced. - [fix] crates/aura-std/src/lib.rs:5 — module doc now names the CostNode/CostRunner authoring surface. - [carry-on, low debt] vol_slippage_cost.rs still declares its extra-input port twice (the cost_node_builder arg and the trait extra_inputs()); a clean unifier needs design thought (builder() is static, no factor instance) and is not worth forcing at cycle close. Documented here; agreement rests on the helper + a test + the harness debug-assert. Cycle 0083 ephemera removed (docs/specs/0083 + docs/plans/0083). Verified: cargo build/test --workspace clean (0 failures), clippy -D warnings clean, cargo doc -p aura-std clean (CostNode/CostRunner intra-doc links resolve). refs #148 |
||
|
Brummel
|
6b53c239dd |
feat(0083): CostNode trait + shared cost-record contract
Lift the duplicated cost-node skeleton — shared verbatim by ConstantCost and VolSlippageCost and locked by convention against CostSum — into one abstraction: - A new aura-std/src/cost.rs owns the cost-record contract (COST_WIDTH, COST_FIELD_NAMES, GEOMETRY_WIDTH), the CostNode factor trait (one hook: cost_numerator, in price units), the generic CostRunner<F> adapter that holds the shared state (cum, out) and the co-temporality skeleton (geometry gating, numerator/latched R-normalization, closed/open charge, 3-field emit), and a cost_node_builder schema assembler. - ConstantCost and VolSlippageCost become thin CostNode factors; their new() returns CostRunner<Self>, so every existing call site and unit test binds transparently and stays green verbatim. - CostSum and main.rs drop their local triple consts for the shared COST_FIELD_NAMES — the cycle-2 audit-flagged by-convention 3-field lockstep is now a structural single-source contract (the four copies of the triple collapse to one). main.rs also reads COST_WIDTH in place of the literal slot stride. Behaviour-preserving: the builders emit unchanged schemas (same port names), so the wiring, the net_r_equity seam, and summarize_r are untouched; the numerator/latched token form is preserved verbatim (IEEE-754 byte-identity). The existing suite is the regression net — all green: the per-node unit tests pass verbatim (0.5/1.5, 0.375/1.375), the composition E2E exact, the C18 no-cost golden byte-identical. Two new CLI characterization goldens pin the exact net_expectancy_r of the flat and composed cost paths (the prior tests only asserted net < gross, which a numerator drift would pass silently). New cost.rs runner/contract tests + the HalfSpreadCost author doctest cover the new surface. Deviation from the plan: CostNode::name() was dropped (the plan over-specified it) — it is dead surface, nothing consumes it (CostRunner forwards label(); cost_node_builder takes the name as an explicit arg). Removed from the trait, both impls, the doctest, and the test stubs; build + suite + clippy green. Verified: cargo build --workspace --all-targets clean; cargo test --workspace 0 failures; cargo clippy --workspace --all-targets -- -D warnings clean; doctest green. refs #148 |
||
|
Brummel
|
c787488e64 |
plan: 0083 cost-node trait + shared cost-record contract
6 tasks: (1) new cost.rs — CostNode trait + CostRunner<F> + cost_node_builder + shared COST_FIELD_NAMES/COST_WIDTH/GEOMETRY_WIDTH + runner tests + author doctest; (2-3) migrate ConstantCost/VolSlippageCost to thin factors; (4) CostSum reads the shared contract; (5) main.rs reads aura_std::COST_FIELD_NAMES; (6) workspace gate. Behaviour-preserving — the existing suite is the regression net. refs #148 |
||
|
Brummel
|
16f4cbb548 |
spec: 0083 cost-node trait + shared cost-record contract (boss-signed)
Cycle 3 of the cost-model-graph milestone: lift the duplicated cost-node skeleton (shared verbatim by ConstantCost/VolSlippageCost, locked by convention against CostSum) into a CostNode factor trait + a generic CostRunner<F> adapter, with one source of truth for the 3-field cost record. Behaviour-preserving: same schemas, same wiring, byte-identical output; the existing suite is the regression net. Cost-graph composite-builder deferred (decision E). Grounding-check PASS (7/7 assumption groups against named green tests). Fork decisions logged on the reference issue. refs #148 |
||
|
Brummel
|
6c7f5dd147 |
audit(0082): cycle close — drift-clean; C10 cycle-0082 note + cost-flag validation symmetry
Cycle 0082 (cost-graph composition, cycle 2) closes drift-clean against C10. The
architect review confirms the cycle is fundamentally sound:
What holds: VolSlippageCost + CostSum are ordinary aura-std downstream nodes
(C9/C8), cost stays out of aura-engine; the vol proxy is a trailing realized range
(C2, no look-ahead); feed-forward, no equity feedback (C1); the co-temporality
contract (gate cost nodes on PM geometry only, 0-cost on a cold state input) is
consistent with C1/C2/C9 and the cycle-1 positional join, and charging 0 during
warm-up is honest (no estimate yet, not a fabricated guess). The no-cost C18 golden
is byte-identical.
Drift resolved at close:
- [medium -> fixed] docs/design/INDEX.md (C10): added the cycle-0082 realization
note. The load-bearing co-temporality contract (generalizes to all future cost
factors) lived only in code comments + spec 0082; the ledger is now its durable
home once the spec is removed.
- [low -> fixed] crates/aura-cli/src/main.rs: --cost-per-trade now rejects negatives
at parse, symmetric with --slip-vol-mult (it previously deferred to the node
assert's panic). Run-path tests + the no-cost golden stay green.
Carry-on (documented deferrals, not this cycle's debt to fix):
- The 3-field cost triple {cost_in_r, cum_cost_in_r, open_cost_in_r} is restated
by-convention across ConstantCost, VolSlippageCost, and CostSum (compiler-unlinked
lockstep) — to be unified by the general CostNode trait, now justified by two
concrete nodes (#148).
- SLIP_VOL_LENGTH=5 is sized to warm within the synthetic smoke fixture; documented
in the const doc (on real M1 data any reasonable window warms).
- A future per-cycle-held accrual (carry/funding) accrues over holding duration,
which will force a summarize_r fold change and stress the positional seam — flagged
for that deferred cycle.
Regression gate green and unchanged: full workspace suite 0-fail, clippy --workspace
--all-targets -D warnings clean. No baseline moved -> no ratify.
Spec + plan ephemera removed (git rm docs/specs/0082, docs/plans/0082).
refs #148
|
||
|
Brummel
|
7f3756a395 |
feat(0082): cost-graph composition — VolSlippageCost + CostSum net-R aggregate
Cycle 2 of the "Cost-model graph (in R)" milestone (#148): the milestone's real architectural claim — the cost graph composes. Two cost nodes now sum into one net-R curve while summarize_r and the net_r_equity tap stay structurally unchanged. - VolSlippageCost (aura-std): a second, STATE-DEPENDENT cost node; per-trade charge = slip_vol_mult * volatility / |entry-stop|, in R. The vol is an independent short-horizon realized range (SLIP_VOL_LENGTH=5, distinct from the stop's EWMA-3) — scaling slippage by the stop's own vol would collapse cost-in-R to a constant (indistinguishable from ConstantCost). - CostSum (aura-std): the cost-graph OUTPUT node — sums N cost nodes' 3-field cost-in-R records per-field into one aggregate. summarize_r and net_r_equity read the aggregate (one home for cost; n=1 is the identity, so the cost path is uniform). A future CostNode trait (deferred) will unify the cost-triple the two producer nodes currently restate by-convention. - Run path: --cost-per-trade and --slip-vol-mult combine, their costs summing into the net-R curve; a hoisted vol proxy (RollingMax-RollingMin) keeps the single feed. Run-path-scoped; sweep/walkforward/mc pass None. Co-temporality contract (the load-bearing design decision; corrected from the signed spec after the implement-loop correctly BLOCKED Task 3 on it). summarize_r positional-joins cost[i] <-> record[i], so the cost stream must be co-temporal 1:1 with the PM record. A cost node is therefore gated ONLY by the PM geometry (closed/open/entry/stop); a not-yet-warm state input (the vol proxy warms later than PM) contributes 0 cost that cycle rather than withholding and desyncing the stream. This makes co-temporality structural + warmup-independent, preserves the C18 golden, and generalizes to any future cost factor. The rejected alternative (a key-join in summarize_r) would have moved the golden and pushed cost-graph logic into the post-run fold. Recorded on #148. Tests: VolSlippageCost + CostSum unit sets (incl. the co-temporal-zero-during- warmup case); the CLI composition run (both flags -> net_both < net_flat, net_r_equity persisted); node-level EXACT additive composition (net_both == net_flat + net_vol - gross over the real nodes), the aggregate net_r_equity == post-run net total, and CostSum(1) identity. Full workspace suite green; clippy -D warnings clean; the no-cost C18 golden byte-identical (the regression floor). Deferred (later cycles of this milestone): the general CostNode trait + cost-graph composite-builder (now justified by two concrete nodes), the conviction-weighting R-aggregation axis, and cost on the reduce-mode sweep path. Spec + plan amended to the corrected contract; both are cycle ephemera (git rm at cycle close). refs #148 |
||
|
Brummel
|
31ef46314a |
plan: 0082 vol-slippage cost + cost-graph composition
Task-by-task plan for spec 0082: the VolSlippageCost node (Task 1) and the CostSum aggregator (Task 2) in aura-std; the run-path wiring as one compile-gate task (Task 3) — stage1_r_graph cost-param widening + vol-proxy hoist + the CostSum-aggregated cost block + the --slip-vol-mult thread-through across all call sites, with a CLI composition test and the no-cost golden floor; and the node-level exact-sum composition tests (Task 4). RED-first per task. The &'static str CostSum port names are interned (COST_SUM_PORTS, mirroring COL_PORTS); summarize_r is unchanged (it folds the CostSum aggregate). refs #148 |
||
|
Brummel
|
372eae9cac |
spec: 0082 vol-slippage cost + cost-graph composition (boss-signed)
Cycle 2 of the "Cost-model graph (in R)" milestone (#148): the milestone's real architectural claim — the cost graph composes. Two pieces: - VolSlippageCost (aura-std): a second, state-dependent cost node whose per-trade charge scales with a measured volatility input (k * vol / |entry - stop|), distinct from ConstantCost's flat charge. The vol is an independent short-horizon realized range, deliberately NOT the stop's own vol — scaling slippage by the vol the stop already normalizes would collapse cost-in-R to a constant (indistinguishable from ConstantCost). - CostSum (aura-std): the cost-graph output node — sums N cost nodes' 3-field cost-in-R records per-field into one aggregate record. So summarize_r and the net_r_equity LinComb(4) read the AGGREGATE and stay structurally unchanged (one home for cost; n=1 is the identity, keeping the cost path uniform). Run-path wiring: --cost-per-trade and --slip-vol-mult combine, their costs summing into the net-R curve; a hoisted vol proxy keeps the single-feed pattern. A no-cost run stays byte-identical (C18 golden floor). Scope is the run path; sweep/walkforward/mc still pass None (sweep-path cost remains deferred), and the general CostNode trait stays deferred (designed against two shipped nodes later). Auto-signed under /boss on the grounding-check PASS: all 8 load-bearing existing-behaviour assumptions tie to named green tests. Fork decisions recorded on #148. refs #148 |
||
|
Brummel
|
e82725f2d7 |
audit(0081): cycle close — drift-clean (code); ledger C10 realization note + stale-broker doc sync (refs #148)
Cycle 0081 (cost-model graph, cycle 1) closes drift-clean against C10: the architect review confirmed ConstantCost + the net-R seam realize the contract at its floor (C9 cost node; net_r_equity a C8/C18 sink; R-pure; one home for cost, subsuming the retired scalar round_trip_cost; pure feed-forward C1), and the no-cost baseline is byte-identical (C18 golden held). Regression gate green and unchanged: full workspace suite 0-fail, clippy --workspace --all-targets -D warnings clean. No baseline moved -> no ratify. Ledger sync (docs/design/INDEX.md): added the C10 cycle-0081 realization note (ConstantCost node + net_r_equity seam shipped; scalar round_trip_cost retired; run-path-scoped; later-cycle deferrals named). Drift swept (cycle-adjacent + cheap one-liners): - crates/aura-analysis/src/lib.rs net_expectancy_r doc: the removed in-fold mechanic (mean(R - round_trip_cost/latched_dist)) -> the cost-stream fold. - crates/aura-std/src/lib.rs module doc: "realistic broker profiles" (retired by #116) -> the legacy SimBroker pip yardstick + cost-model nodes. Backlogged (pre-existing, broader, predates this cycle): the docs/project-layout.md backtest worked example is stale on #117 (exposure->bias) and #116 (realistic broker); a proper rewrite is filed as #151, not folded here. Architect-accepted: the #[allow(dead_code)] on r_col::ENTRY_PRICE/STOP_PRICE (their production reader legitimately removed when summarize_r stopped recovering latched_dist; still the layout contract + exercised by test fixtures). Spec + plan ephemera removed (git rm docs/specs/0081, docs/plans/0081). refs #148 |
||
|
Brummel
|
3fe4684190 |
feat(0081): cost-model graph cycle 1 — ConstantCost node + net-R seam
Lands the first cycle of the "Cost-model graph (in R)" milestone: cost is
now a composable in-graph node, not a post-run scalar (C10).
- ConstantCost (aura-std): a flat round-trip cost per trade, emitted in R
as a 3-field record {cost_in_r, cum_cost_in_r, open_cost_in_r} isomorphic
to PositionManagement's R-triple. R-pure: cost_in_r = cost_per_trade /
|entry-stop|; notional cancels, no equity held.
- summarize_r folds the cost stream into net_expectancy_r, replacing its
scalar round_trip_cost (one home for cost, no double-count). Positional
co-temporal join; an empty stream is the gross-R baseline.
- net_r_equity tap (stage1_r_graph): LinComb(4) over [cum_realized_r,
unrealized_r, -cum_cost_in_r, -open_cost_in_r] -> Recorder, a sibling of
r_equity; --cost-per-trade on the run path wires the node + tap + fold.
- A no-cost run is byte-identical: no net_r_equity tap, net == gross, the
C18 golden unchanged.
Tests: exact-subsumption (the node-stream net == mean(r_i - cost_i) over
ALL trades incl. the window-end open one), the in-graph net_r_equity tap +
net < gross E2E, the no-cost golden held. Full suite green; clippy clean.
Implement-time decisions ratified (recorded on #148): the cost stream is
recorded 3-wide (the node's full output; summarize_r reads col 0/2, col 1
feeds the in-graph net curve) — resolving a 2-wide/3-wide inconsistency in
the plan's Step 3; Trade drops its `latched` field (cost now arrives in R),
so r_col ENTRY_PRICE/STOP_PRICE become #[allow(dead_code)] layout contract.
Scoped to the run path (non-reduce); sweep/walkforward/mc pass None — cost
on the reduce-mode sweep path and OOS-pooling cost are a deferred follow-up.
Known pre-existing concern (not this cycle): aura-std lib.rs module doc
still names "realistic broker profiles", retired by the C10 rework.
refs #148
|
||
|
Brummel
|
725545cbde |
plan: 0081 cost-model constant-cost
Task-by-task plan for spec 0081: the ConstantCost node (aura-std), the summarize_r scalar->cost-stream change (the workspace-wide call-site migration as one compile-gate task), and the run-path wiring (net_r_equity tap + --cost-per-trade) — RED-first per task. Scoped to the run path (non-reduce); sweep/walkforward/mc pass None this cycle. refs #148 |
||
|
Brummel
|
2e63ed9e26 |
spec: 0081 cost-model constant-cost (boss-signed)
Cycle 1 of the "Cost-model graph (in R)" milestone (#148): a single ConstantCost node (flat round-trip cost per trade) plus the net-R seam — an in-graph cost_in_r stream, a net_r_equity sink (sibling of r_equity), and summarize_r folding that one stream into net_expectancy_r (replacing the scalar round_trip_cost). A run with no cost stays a byte-identical gross-R baseline. Defers the general CostNode trait, the cost-graph composite-builder, data-grounded factors, and the conviction axis. Auto-signed under /boss on the grounding-check PASS: all load-bearing existing-behaviour assumptions are ratified by green tests. Fork decisions recorded on #148. refs #148 |
||
|
Brummel
|
a517899297 |
docs: sync ledger cross-ref + glossary to the C10 rework (refs #116)
Follows the C10 rework (
|
||
|
Brummel
|
29cdc8c218 |
design(C10): retire the realistic broker — cost is a cost-model graph in R (gross→net), money to the live deploy edge (refs #116)
Reworks the C10 contract per a ratified in-context design discussion (#116). The historical "realistic broker" with authored frictions is retired as horseshoe-throwing: real friction (slippage, swaps, spread) is not historically knowable; the only reliable friction is measured forward against a real broker. In its place: - Cost is a composable C9 graph of cost nodes, in R, that approximates (never claims) realism — generalizing/subsuming the scalar round_trip_cost. Output is a net-R equity curve (gross R = signal, net R = after cost). - Research is pure feed-forward: the Sizer and compounding leave the loop; the z^-1 fill-edge register and the flat-1R-vs-compounding structural axis are gone. Compounding/Kelly/drawdown are post-hoc transforms at the deploy/account layer. - Executor = stop-rule + position-management in R. size/volume is a deploy concept; the position-event table is demoted to a deploy/reconciliation artifact. - Conviction-based risk allocation survives as an R-aggregation axis (flat vs |bias|-weighted), distinct from the removed money-Sizer. - Money, a real broker, and cTrader Open API are a separate, later live/deploy edge (C11 record-then-replay; C13 frozen deploy) — the only ground truth. Stage-1 R realizations preserved as history; the #117 reframe's R spine survives, its Stage-2/currency/register/compounding portions superseded. |
||
|
Brummel
|
226bf57130 |
audit(0080): cycle close — drift-clean; ledger sync (selection types relocated, registry-vocab decision recorded) (refs #136)
Cycle 0080 was a 44-line behaviour-preserving re-export relocation along the exact engine↔domain seam the 0079 architect review just blessed, so the drift review is carried from 0079 rather than re-dispatched; the only drift introduced is the ledger note, synced here. Regression gate green and unchanged: full suite 665/0 (incl. the C18 goldens), clippy --workspace --all-targets -D warnings clean. Ledger sync (docs/design/INDEX.md): added a cycle-0080 realization note — FamilySelection/SelectionMode are now in aura-analysis (engine-side extraction complete); recorded the user-ratified decision that the aura-registry Metric/metric_cmp/deflation vocabulary stays in the registry by design (it is the trading selection layer, its deflation statistics branch irreducibly on metric identity, and genericity would be a one-implementor abstraction); and that making RunReport generic / a domain-agnostic orchestration substrate is deferred to the World/C21 layer, explicitly out of #136. No baseline moved (the suite is the regression gate; green-unchanged), so no ratify statement. No spec/plan ephemera to drop. |
||
|
Brummel
|
04c581a441 |
refactor(0080): relocate FamilySelection/SelectionMode to aura-analysis (refs #136)
The last trading-domain types still defined in aura-engine's report module — the selection-provenance types FamilySelection and SelectionMode — move to aura-analysis, finishing the #136 goal of getting the trading types out of the engine. aura-analysis is the only non-cyclic home that removes them from the engine: RunManifest.selection: Option<FamilySelection> needs the type and the engine already depends on aura-analysis; aura-registry and aura-cli reach them unchanged via aura-engine's re-export (report.rs pub-re-exports them, lib.rs untouched). Their tests stay in report.rs (one round-trips through RunManifest, which stays in the engine) and reach the types via the re-export. Behaviour-preserving (C1): full suite 665/0 unchanged incl. the C18 goldens, clippy clean, only report.rs and aura-analysis/src/lib.rs change. Staying in the engine (trace-coupled, deferred to the World/C21 layer, not #136): RunManifest, RunReport, summarize. The aura-registry Metric/metric_cmp vocabulary stays in the registry by design — the registry is the trading selection layer, not the domain-free engine (the deflation statistics branch on metric identity, irreducibly domain). See the #136 thread for the rationale. |
||
|
Brummel
|
a8f67c7c80 |
audit(0079): cycle close — drift-clean; ledger crate-topology sync + serde_json dev-dep tightening (refs #136)
Architect drift review (3e8fd73..HEAD): the extraction is faithful and behaviour-preserving — every moved symbol's definition left report.rs, 35 unit tests relocated with their subject, aura-engine/src/lib.rs byte-unchanged, the four downstream crates byte-unchanged. No code defect. Regression gate green: cargo build/test --workspace = 665 passed / 0 failed (incl. the C18 cli_run goldens and the registry legacy-line loads); clippy --workspace --all-targets -D warnings clean. Ledger sync (docs/design/INDEX.md) — pre-existing crate-topology drift the extraction made visible: - C10 notes (cycles 0063/0068) placed PositionAction / PositionEvent / derive_position_events "in aura-engine beside RunMetrics / summarize_r"; cycle 0079 moved all of them to aura-analysis — corrected, with a forward pointer to the new C16 note. - C10 cycle-0065 note claimed "RiskExecutor ships as a public aura-engine composite-builder" and "Composites live in aura-engine ... aura-engine -> aura-std a normal dependency"; both stale — composites live in aura-composites and aura-std is now an aura-engine [dev-dependencies] only. Corrected. - Added a C16 realization note recording the aura-analysis extraction as an architecture fact: the engine<->domain seam, the current five non-node workspace crates, and the deferred #136 generic-over-M tail. serde_json moved from [dependencies] to [dev-dependencies] in aura-analysis (architect debt item): the production types derive serde, but JSON rendering lives in aura-engine's RunReport::to_json, so serde_json is test-only here and must not enter the frozen artifact's library link (C16 frozen-artifact scrutiny). Build + the 35 crate tests stay green after the move. No baseline moved (the suite is the regression gate; it stayed green), so no ratify statement. No spec/plan ephemera to drop (compiler-driven / agent-driven cycle — no docs/specs or docs/plans files were created). |
||
|
Brummel
|
94c88eb7e1 |
refactor(0079): extract the trading-analysis leaf into aura-analysis (refs #136)
aura-engine is documented (C16) and named as the reusable, domain-agnostic reactive SoA substrate, yet `report.rs` concentrated the trading-domain analysis layer inside it. This cycle relocates the pure-domain leaf into a new `aura-analysis` crate (deps: aura-core + serde only), erecting the engine/domain seam before the Stage-2 broker milestone deposits more currency/position-event code into the same module. Behaviour-preserving (C1): the full workspace suite stays green unchanged (665 passed), including the C18 on-disk byte-identity goldens (cli_run, the registry legacy-line loads). Moved to aura-analysis (bodies verbatim — no float expression re-associated): RunMetrics, RMetrics (+ its hand-written PartialEq), the private r_col column indices + SQN_CAP, summarize_r, r_metrics_from_rs, derive_position_events, PositionAction (+ From/TryFrom<i64>), PositionEvent, and the deflation stats helpers inv_norm_cdf / expected_max_of_normals. Their unit tests and private helpers (r_row / r_row_full / pm_row) relocate with them — tests live with their subject (the r_col helpers reach the private module directly, which a re-export cannot bridge). Stays in aura-engine for this cycle: the generic trace plumbing (ColumnarTrace, join_on_ts, JoinedRow, f64_field), RunManifest, RunReport, and summarize (it bridges recorded trace columns into RunMetrics, so it is trace-coupled). The mis-placed orchestration types FamilySelection / SelectionMode also stay (their relocation is entangled with the registry-vocabulary fork, deferred). report.rs pub-re-exports the moved symbols, so aura-engine's lib.rs re-export block and every `crate::report::X` / `aura_engine::X` consumer (aura-registry, aura-cli, aura-composites, aura-ingest) compile byte-unchanged; only the new crate, the workspace member list, aura-engine's Cargo.toml dep, and report.rs itself change. Deferred to later cycles (genuine design forks, route through specify when reached): making RunReport generic over a metric type M and re-bounding sweep/mc/walkforward; relocating the aura-registry Metric / metric_cmp / rank vocabulary; relocating FamilySelection / SelectionMode. Fork decisions logged on #136. |
||
|
Brummel
|
3e8fd73f68 |
tidy: clear the milestone fieldtest findings (refusal reasons, --help, ledger + glossary)
Resolves the four non-bug findings from the inferential-validation milestone
fieldtest (the bug — the broken-pipe panic — is fixed in
|
||
|
Brummel
|
1088320571 |
fix: terminate cleanly on a broken stdout pipe across all family-emitting commands
GREEN for the broken-pipe RED test (
|
||
|
Brummel
|
c750b9fbc3 |
test: RED — family-emitting CLI commands must survive an early-closing stdout reader
Captures the pre-existing broken-pipe panic surfaced by the milestone fieldtest ([bug] finding): every multi-line subcommand (sweep / mc / walkforward / runs family / generalize) panics "failed printing to stdout: Broken pipe (os error 32)" and exits 101 when the stdout reader closes early (| head, | less, a closed UI pane), instead of terminating cleanly. Root cause: Rust's runtime resets SIGPIPE to SIG_IGN at startup, so a write to a closed pipe returns EPIPE and println! panics rather than the process dying quietly. crates/aura-cli/tests/cli_broken_pipe.rs spawns `aura sweep`, closes the stdout read end mid-stream, and asserts no panic in stderr + exit != 101. Verified RED (reproduces 6/6). The GREEN fix follows in the next commit. refs #146 |
||
|
Brummel
|
ab04ed01c3 |
fieldtest: milestone inferential-validation — 4 scenarios, 6 findings (1 bug, 2 friction, 2 spec_gap, 1 working)
Milestone-close gate for "Inferential validation (defend against false discovery at sweep scale)". Downstream-consumer exercise from the public interface only (CLI --help, ledger, glossary, examples — never the source). All four end-to-end scenarios PASS — the milestone delivers its promise: trials-deflation (#144), plateau-over-peak (#145), and cross-instrument generalization (#146) all reach the output and read sensibly, and the composed "is this edge real or overfit?" verdict is honest (scenario 4: a GER40 in-sample winner sqn 0.994 does NOT generalize — worst_case -0.149 on EURUSD, sign_agreement 2/3 → instrument-specific / likely overfit). Milestone status roll-up: bugs_found — NOT green, so the milestone is NOT yet closeable (the green roll-up is the functional leg of the close gate). Findings: - [bug] a family-emitting command piped into an early-closing reader (`aura sweep|walkforward|mc|runs family … | head`) panics "Broken pipe (os error 32)" on stdout (exit 101 under pipefail) instead of exiting cleanly on EPIPE. Pre-existing (not a #146 regression — reproduced on `aura sweep | head -1`); surfaced by the milestone commands. -> debug (RED-first). - [friction] two generalize refusals (single --real symbol; multi-value candidate flag) dump bare usage with no reason. -> tidy iteration. - [friction] the --select plateau objective is absent from `aura --help`. -> tidy. - [spec_gap] sweep vs generalize recompute the same member's sqn with a stable 1-ULP difference; the ledger's C1 / "recomputable" is silent on cross-command metric agreement. -> ratify (ledger note). - [spec_gap] the milestone vocabulary (deflated_score / overfit_probability / worst_case / sign_agreement) is absent from docs/glossary.md. -> docwriter. Artefacts: docs/specs/fieldtest-milestone-inferential-validation.md + fieldtests/milestone-inferential-validation/ (README + 4 scenario scripts). The existing fieldtests/ tree was untouched. refs #146 |
||
|
Brummel
|
15f5c0bc99 |
audit(0078): cycle close — drift addressed; ledger inferential-thread synced (milestone complete) + C12/C18 realization; drop ephemera
Architect drift review (d50a482..feat): drift_found — documentation only, NO contract violation. C1/C9/C10/C23 verified preserved against the diff: `generalization` is a pure deterministic fold (registry-side, C9; R-only, C10); the sole `instrument = Some` production site is `run_generalize`, every other manifest constructor threads `None` + `skip_serializing_if`, so sweep/mc/walkforward/standalone manifest bytes are byte-identical (C23); the compat read-mirror was threaded in lockstep. Drift fixed in this commit (ledger sync, docs/design/INDEX.md): - The inferential-honesty thread read "now two of three pieces built" / "#146 remains the last open piece" — false now that #146 landed. Updated to record the inferential half as structurally complete (all three built, cycle 0078) + the #146 realization, including the aggregator/validator-vs-selector distinction (#146 is a recomputable aggregate, not a FamilySelection annotation). - C18 realization block: added the cycle-0078 note — the comparison axis (C12) realized as FamilyKind::CrossInstrument + the first-class RunManifest.instrument lineage field. Regression: no regression scripts configured (project facts name only Build/Test/Lint/Doc); architect is the gate. cargo test --workspace green (0 failed) and cargo clippy --workspace --all-targets -D warnings clean, orchestrator-verified. Ephemera removed per the project lifecycle convention: docs/specs/0078-* and docs/plans/0078-* (git rm). Durable rationale lifted to the ledger above. refs #146 |
||
|
Brummel
|
0a06afed29 |
feat(0078): cross-instrument generalization as a validation step
Last piece of the Inferential-validation milestone (#146): a new `aura generalize` subcommand grades how consistently a single candidate holds across a set of instruments — the across-instrument axis of the same anti-false-discovery concern as trials-deflation (#144, within one family) and plateau-over-peak (#145, within one instrument's surface). Aggregator/validator, not a third selector (user-ratified): it runs a brought candidate (a single-cell stage1-r grid — --fast/--slow/--stop- length/--stop-k pinned to single values) across an instrument list (--real SYM1,SYM2,...), collects per-instrument R-metrics, and reduces them to a worst-case floor (min over instruments) + a sign-agreement count + the per-instrument breakdown. R-only (C10 — R is the only account/instrument-agnostic unit); the worst-case min is the cross- instrument sibling of PlateauMode::Worst and satisfies the non- domination constraint by construction (a strong instrument cannot lift a min). Rejected: a cross-instrument t-stat mean/std*sqrt(M), which reintroduces the pooled mean the milestone warns against. - RunManifest gains a first-class `instrument` lineage field (serde- widened, C14/C18 — legacy lines load as None, existing manifest bytes unchanged); the compat read-mirror threads it in lockstep. - FamilyKind::CrossInstrument (C12 comparison axis); the M per-instrument runs persist as a CrossInstrument family, each member self-identifying via its stamped instrument. The aggregate is recomputable from the members (McAggregate precedent), printed live. - The generalization reduction lives registry-side beside optimize_ deflated/optimize_plateau (C9), reusing resolve_metric/metric_value/ is_r_metric; check_r_metric is the data-free pre-check the CLI runs before evaluating any instrument. - Additive: existing sweep/walkforward/mc/standalone paths stay byte- identical (C23 — instrument omitted when None via skip_serializing_if). Verified by the orchestrator: cargo test --workspace green (0 failed — the new generalization_*, check_r_metric, parse_generalize_*, and the gated cross-instrument E2E all pass; the E2E ran the real GER40/USDJPY path on this host, exit 0); cargo clippy --workspace --all-targets -D warnings clean. closes #146 |
||
|
Brummel
|
cffc297e9a |
plan: 0078 cross-instrument generalization
Three tasks for #146: (1) RunManifest.instrument serde-widened field + compat-mirror lockstep + thread all in-workspace literals + back-compat tests; (2) aura-registry FamilyKind::CrossInstrument + the R-only generalization reduction (worst-case floor + sign-agreement + breakdown) + check_r_metric + two RegistryError variants; (3) aura-cli generalize subcommand (single-cell stage1-r candidate x instrument list), data-free arity/metric refusals, a gated-skip cross-instrument E2E. fieldtests/ confirmed outside the workspace compile set; FamilyKind has no exhaustive match (serde-derive); only RegistryError's Display match is a gate. refs #146 |
||
|
Brummel
|
428d092298 |
spec: 0078 cross-instrument generalization (boss-signed)
Last piece of the Inferential-validation milestone (#146): grade how consistently a brought candidate holds across a set of instruments. Aggregator/validator, not a third selector (user-ratified "mach es so", decision + derivations minuted on #146): a new `aura generalize` subcommand runs a single candidate (single-cell stage1-r grid) across an instrument list, collects per-instrument R-metrics, and reduces them to a worst-case floor (min over instruments) + a sign-agreement count + the per-instrument breakdown. R-only (C10 — R is the only cross-instrument-comparable unit); the worst-case min is the cross-instrument sibling of PlateauMode::Worst and satisfies the non-domination constraint by construction. New FamilyKind::CrossInstrument (C12) + a first-class RunManifest.instrument lineage field (serde-widened, C14/C18); the reduction lives registry-side beside optimize_deflated/optimize_plateau (C9). Additive — existing sweep/walkforward/mc paths stay byte-identical (C23). grounding-check PASS: all 9 current-behaviour assumptions ratified by green tests. refs #146 |
||
|
Brummel
|
d50a4828ad |
audit(0077): cycle close — drift addressed; legacy-load fixture + ledger sync; drop ephemera
Architect drift review over 9c6b9c7..cb32658 returned drift_found (3 items); all resolved here. What holds (architect): C23 default-argmax byte-identical, C9 separation (selector in aura-registry, walk_forward selection-agnostic), C1/C2 (plateau pure/deterministic/in-sample), one higher_is_better direction source. Resolutions: - [medium] Backfilled the pre-0077 FamilySelection legacy-load fixture (report.rs family_selection_loads_a_pre_0077_legacy_line). The feat commit and the struct doc-comment claimed "a legacy line still loads (C14/C18)" as landed, but no test pinned the field-level mapping: bare deflation scalars -> Some(...), absent plateau keys -> None. families.jsonl/runs.jsonl are append-only, so a winner stamped pre-0077 must read back; now pinned. - [low] Synced the ledger's inferential-validation open thread (docs/design/INDEX.md): #145 plateau-over-peak landed (cycle 0077), selection is now a pluggable objective (bare argmax -> trials-deflated -> plateau) on the manifest carrier; only #146 (cross-instrument generalization) remains of the milestone's three pieces. - [medium-high] Removed the cycle ephemera (git rm docs/specs/0077-*, docs/plans/0077-*) per the cycle-close convention. Regression: full workspace suite green and clippy --all-targets -D warnings clean (verified at feat-review time). No baseline scripts configured; the test + clippy gates are the regression gate. Cycle 0077 tidy (clean) — drift-clean, not a milestone close (the milestone fieldtest is a separate deliberate gate). |
||
|
Brummel
|
cb32658fd1 |
feat(0077): prefer a robust parameter plateau over the in-sample peak
Add an opt-in plateau selection objective for walk-forward, recorded on the same RunManifest.selection carrier #144 introduced. Default argmax is byte-identical (C23); plateau is reached only via a new --select flag. What landed: - FamilySelection / SelectionMode reshape (report.rs): the selection RULE (mode) and its ANNOTATION are orthogonal — deflation fields become Option (present iff Argmax), plus PlateauMean/PlateauWorst variants and neighbourhood_score/n_neighbours (present iff Plateau*). Legacy lines still load: the deflation scalars deserialize from bare values via serde default (C14/C18). compat.rs embeds FamilySelection by value — reshape flows through untouched. - GridSpace::axis_lens() + SweepBinder::sweep_with_lattice (engine): the grid radixes in param_space()/odometer order. sweep() delegates to sweep_with_lattice with the lattice dropped, so every existing .sweep() caller is byte-unchanged. - optimize_plateau + PlateauMode + closed_neighbourhood (aura-registry, C9): each member scores as the mean/worst of its closed mixed-radix grid neighbourhood's metric_value; the winner is the smoothed argmax by the metric's direction (earliest-odometer tie, as optimize). Pure, no RNG (C1); in-sample only (C2). A higher_is_better helper now sources the per-metric direction once, shared by metric_cmp and optimize_plateau. - CLI --select <argmax|plateau:mean|plateau:worst> (default argmax; unknown token exits 2). walkforward_family dispatches via a select_winner helper; sweep_over/stage1_r_sweep_over carry the lattice as Option<Vec<usize>>. runs-family display gains a plateau(<mode>)=<score> over <n> cells line. RandomSpace-refuse: walkforward has no --random producer, so the plateau-without-lattice guard is structural — select_winner returns the refuse on a None lattice (the caller prints the message and exits 2), unit-tested at the helper, not via a --random E2E (decided + recorded on #145 comment 1974, with the lattice-seam fork). Tests: plateau-picks-plateau-not-spike, mean-vs-worst, lower-is-better direction-flip, closed-neighbourhood golden, single-member degeneracy, C1 determinism; CLI select-parse, select_winner refuse; E2E argmax-byte-identical (C23), plateau:mean and plateau:worst provenance. Full workspace suite green; clippy --all-targets -D warnings clean. closes #145 |
||
|
Brummel
|
fd221c81ec |
plan: 0077 plateau over peak selection
Executable plan for cycle 0077 (#145), decomposed into 5 tasks with the compile-gates sequenced: 1. Reshape FamilySelection/SelectionMode across 3 crates (mechanical, byte-preserving): deflation fields -> Option, add PlateauMean/PlateauWorst + neighbourhood_score/n_neighbours; thread optimize_deflated + all tests + the runs_family display (argmax bytes unchanged). 2. GridSpace::axis_lens() + SweepBinder::sweep_with_lattice (sweep delegates, every existing .sweep() caller byte-unchanged). 3. optimize_plateau + PlateauMode + closed_neighbourhood (mixed-radix grid neighbourhood, argmax of the smoothed surface) + a higher_is_better refactor so direction lives in one place. 4. CLI --select parse + Selection threading + sweep-entry Option<lattice> tuples + select_winner (the RandomSpace-refuse guard) + walkforward_family dispatch; all signature-change callers threaded before the build gate. 5. runs-family plateau E2E golden. Two plan-time forks were derived and recorded on #145 (comment 1974): the lattice surfaces at the engine terminal (only the post-resolve_axes GridSpace holds param_space order), and the RandomSpace-refuse is a structural guard unit-tested at select_winner (walkforward has no --random producer to E2E). refs #145 |
||
|
Brummel
|
45b8e2b28b |
spec: 0077 plateau over peak selection (boss-signed)
Second cycle of the inferential-validation milestone. Adds an opt-in plateau selection objective: optimize_plateau beside optimize/optimize_deflated scores each grid member by the mean/worst of its grid-neighbourhood metric and argmaxes the smoothed surface (mixed-radix neighbours over GridSpace axis_lens, passed as an argument not a SweepFamily field). Strictly opt-in via --select <argmax|plateau:mean|plateau:worst>, default argmax (C23 byte-preserving). Reconciles the shared RunManifest.selection carrier with #144's landed reality: the selection RULE (argmax vs plateau) and the deflation ANNOTATION are orthogonal, so FamilySelection is extended — deflation fields become Option, SelectionMode gains PlateauMean/PlateauWorst (the reserved slot), and neighbourhood_score/n_neighbours are added. A small wire change to the one-cycle-old type; legacy lines still load. RandomSpace plateau refused (exit 2 — no lattice); kNN deferred. Grounding-check PASS (12 assumptions ratified). refs #145 |
||
|
Brummel
|
9c6b9c7ea9 |
audit(0076): cycle close — drift addressed; ledger inferential-thread synced; drop ephemera
Architect drift review (c192dfd..a295905): drift_found, medium/low only — no
ledger or hot-path contract crossed. C23-additive, C1-deterministic, C14/C18
back-compat all verified green (full workspace suite + clippy). No regression
scripts configured (test+clippy are the gate).
Resolution:
- [medium] three spec-promised optimize_deflated tests were missing (the
centring control, the all-noise high-p companion, the C2 IS-only read) — the
centring was unprotected ("removing it would pass every test"). Plus an
unguarded dispersion-floor sign assumption. FIXED in 0076.tidy (
|
||
|
Brummel
|
ecc9541a56 |
0076.tidy: backfill spec-promised deflation tests + guard the dispersion-floor sign
Audit (architect, cycle 0076) found three tests the spec's testing strategy promised but the implement-loop did not land, plus an unguarded sign assumption: - optimize_deflated_centres_the_null_so_a_uniform_edge_is_not_called_overfit: the load-bearing protection for the mean-subtraction. A family where every member shares the same +R edge reads as NOT overfit only because the null is centred; an uncentred null would give ≈0.5. Removing the centring now flips a test (it previously passed every test — the architect's headline finding). - optimize_deflated_all_noise_family_has_high_overfit_probability: the high-p companion to the existing clear-edge low-p test. - optimize_deflated_provenance_reflects_only_the_passed_family: C2 at the registry boundary (n_trials == family size, raw == the family's own argmax), pinning that the record encodes only in-sample information. - A debug_assert in the dispersion-floor arm: it subtracts inflation, valid only for higher-is-better metrics; lower-is-better (max_drawdown/bias_sign_flips) would deflate wrong-signed. Unreached from the two shipped call sites, but optimize_deflated is public — guard the assumption rather than leave it implicit. cargo test -p aura-registry (33) + clippy green. refs #144 |
||
|
Brummel
|
a2959050a4 |
feat(0076): deflate the sweep winner's metric for the number of trials
Adds optimize_deflated beside optimize in aura-registry: the in-sample walk-forward selection now records, on each OOS winner's manifest, how much the winner's metric is inflated by the size of the search it won — a deflated score and (R arm) an overfit probability — without changing which member wins (additive, C23; a regression test pins optimize_deflated's winner == optimize's). - R arm (sqn_normalized/expectancy_r/...): a centred moving-block reality-check. Each member's per-trade R series is mean-subtracted to impose the no-edge null, resampled via the r_bootstrap kernel (extracted to a shared resample_block, byte-identical), and the best-of-K null-max gives overfit_probability = (count(null >= raw)+1)/(n+1) and deflated_score = raw - p95(null). Deterministic given the seed (C1). - total_pips arm: a closed-form expected-max-of-K dispersion floor (inv_norm_cdf + expected_max_of_normals), no probability. The record (FamilySelection on RunManifest) rides the proven serde(default, skip_serializing_if) widening + a compat.rs mirror, so legacy runs.jsonl/families.jsonl lines load unchanged and unstamped sweep/mc/standalone manifests stay byte-identical (C14/C18). The SelectionMode enum reserves a Plateau slot for #145. CLI: walk-forward stamps both arms; runs family <id> rank surfaces a human-readable deflated/overfit line. Quality review (re-dispatched after the implement-loop's quality gate exhausted) caught a real defect: on the R arm with positive resamples but no member carrying trade_rs (a family loaded from disk — trade_rs is serde-skipped), the null was all-NEG_INFINITY, turning deflated_score into +inf. Fixed by collapsing "no usable null" (zero resamples OR no trade_rs) into one degenerate floor (deflated == raw, overfit == 1.0); pinned by optimize_deflated_no_trade_rs_floors_instead_of_infinity. Verified: cargo test --workspace and cargo clippy --workspace --all-targets -D warnings both green. Frictionless Stage-1 R; n_eff advisory and reload-time recompute deferred (spec 0076 Out of scope). closes #144 |
||
|
Brummel
|
40e3f609c2 |
plan: 0076 deflate sweep winner for trials
Six bite-sized tasks: aura-engine record types + RunManifest.selection field (partial-build gate) -> downstream compile completion incl. the compat mirror (workspace gate) -> resample_block extraction + SplitMix64 pub -> inv_norm_cdf / expected_max_of_normals -> optimize_deflated + the metric_cmp value-read refactor -> CLI wiring + display. RED where behaviour is pinnable (C23 winner-unchanged tripwire, C1 seed reproducibility, the centred reality-check statistic, the inv_norm_cdf golden); back-compat goldens preserved. refs #144 |
||
|
Brummel
|
a1908bd15e |
spec: 0076 deflate sweep winner for trials (boss-signed)
First cycle of the inferential-validation milestone. Adds optimize_deflated beside optimize: an additive selection-provenance record (deflated score + overfit probability) stamped on the OOS winner's manifest, never re-ranking (C23). R arm = centred moving-block reality-check reusing the r_bootstrap kernel; total_pips arm = expected-max-of-K dispersion floor. Shared RunManifest.selection carrier with a SelectionMode::Plateau slot reserved for #145. refs #144 |
||
|
Brummel
|
c192dfd344 |
audit(0075): cycle close — drift-clean; doc-link + trade_rs tripwire tidy; drop ephemera
Architect drift review (c99e7b8..2e77ab1) against the design ledger +
CLAUDE.md invariants: drift-clean, debt only — no contract or spec
violation. Verified intact: C18 wire shape (trade_rs is #[serde(skip)] +
excluded from the hand-written PartialEq; runs.jsonl/families.jsonl bytes
and digests unmoved; mc_r_bootstrap is a printed line, never a record),
C1 determinism (seeded r_bootstrap + disjoint walkforward windows, both
pinned by byte-identical-rerun e2e tests), C7 frictionless Stage-1
(round_trip_cost stays 0.0), C8/C12 streaming reduction (IS sweep folds
O(trades)/member, OOS run non-reduce). The verbatim summarize_r /
r_metrics_from_rs float duplication is guarded by the cross-reducer
equality test — sound.
Tidy applied this close:
- doc-link drift: the new E[R] notation in r_bootstrap / RBootstrap /
run_mc_r_bootstrap doc-comments was parsed by rustdoc as an intra-doc
link `[R]` (4 warnings). Wrapped in backticks; doc build now clean.
- coverage micro-gap [low, architect]: added a tripwire asserting a
populated trade_rs is absent from the serialized JSON (the serde(skip)
C18-preservation property had no direct test — only Vec::new() was
serialized before).
Carry-on (benign debt, not fixed): the mc usage literal carries a `usage:`
prefix the sibling sweep/walkforward usages omit — required because the
new `["mc", rest @ ..]` arm now handles the `--real` rejection and the
preserved golden asserts stderr.contains("usage"); churning the golden to
drop the prefix is not worth it.
Removed the cycle's ephemeral spec + plans (docs/specs/0075,
docs/plans/0075, docs/plans/0076) per the lifecycle rule.
Verified: cargo test --workspace 610 passed / 0 failed, clippy -D warnings
clean, cargo doc --no-deps clean.
closes #139
|