ffd18b98ea3d35a5fb930302e37c983166d9f256
184 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brummel
|
eb8653488f |
feat(0070): wire folding sink reductions into the stage1-r sweep (closes #138)
Completes the M3 fix for BLOCKER #138. The no-trace stage1-r sweep now folds its reductions online: `stage1_r_graph(reduce=true)` wires `SeriesReducer` (equity, exposure -> one summary row each) and `GatedRecorder` (gated on `closed_this_cycle` -> only the trade rows + the final row), and the folded consumer rebuilds `RunMetrics` from those compact outputs. Per-member memory drops from O(cycles) (~2 GiB over ~5.5M one-minute bars) to O(trades), so full-history multi-member sweeps no longer exhaust RAM+swap. The `r_equity` tap is omitted when reducing (it is consumed only by the `--trace` persistence path). `reduce = trace.is_none()`: the `--trace` path and the single `run_stage1_r` keep the raw recorders + `summarize`/`summarize_r` over full rows — the byte-for-byte reference. A new CLI-seam test (`sweep_strategy_stage1_r_folded_no_trace_metrics_equal_raw_trace_metrics`) proves the folded no-trace sweep reports byte-identical per-member metrics to the raw `--trace` path. Verified: cargo test --workspace green (42 suites); cargo clippy --workspace --all-targets -- -D warnings clean; stage1-r goldens byte-identical. closes #138 |
||
|
Brummel
|
cfe7bad0ff |
feat(0070): streaming sink reductions — finalize hook + folding sinks (#138)
M3 machinery for BLOCKER #138: sink reductions can now fold online instead of buffering every per-cycle row to an unbounded channel. - aura-core: additive `Node::finalize` end-of-stream hook (default no-op) + `SeriesFold` online accumulator (last / max_drawdown / sign_flips). - aura-engine: `Harness::run` calls `finalize` once per node in topo order after the source loop; `summarize` refactored to drive `SeriesFold` (byte-identical), the now-dead `sign0` removed. - aura-std: `GatedRecorder` (emits only gated rows + the final row on finalize) and `SeriesReducer` (folds one f64 series, emits one summary row on finalize), both emitting the `Recorder` channel type — no aura-engine dependency. - Integration tests: folded reductions are byte-for-byte equal to the raw-recorder path; the R-reduction's peak memory is O(trades), independent of cycle count. Ratified two off-plan fixes the implementer made in the tests (both plan defects, not code defects): the GatedRecorder correctly flushes the final non-gated row on finalize, so the retained count is K_TRADES + 1; and the heterogeneous PM record needs kind-correct zero scalars (a blanket f64 zero is rejected by the kind-typed AnyColumn::push). Wiring these reducers into the stage1-r sweep (the actual O(cycles)->O(trades) win) follows next. refs #138 |
||
|
Brummel
|
54a5d68f51 |
feat(stage1-r-sweep): grid signal+stop timescales via CSV flags
Enabling change for the strategy-research run: `aura sweep --strategy
stage1-r` gains four optional comma-separated grid flags `--fast`,
`--slow`, `--stop-length`, `--stop-k` that become the sweep's grid axes
over the four stage1_r_graph knobs (sma fast/slow length, vol-stop
length, vol-stop k). The family is their cartesian product. This lets a
slow time-series-momentum edge be screened across timeframes — Phase 0
showed fast (2/4-bar) signals are chop-dominated noise on M1, so any
momentum edge lives at slower timescales, and the stop timescale must be
tunable alongside the signal.
Mechanism:
- aura-composites: factor vol_stop and risk_executor into a single
shared topology body each (vol_stop_inner / risk_executor_inner), with
the vol-stop knobs either BOUND (the original constant-bind form,
byte-identical) or left OPEN and named `stop_length` / `stop_k` so
their slots surface in param_space as sweepable axes. New
risk_executor_vol_open(risk_budget) is the gridding sibling of
risk_executor(StopRule::Vol{..}, ..). One body, two arms → topology
cannot drift; a member at any (length, k) matches the bound form
byte-for-byte.
- aura-cli: parse_csv_list (generic FromStr, rejects non-numeric/empty),
pure unit-testable parse_sweep_args, all four knobs gridded via
.axis(); absent flags fall back to today's exact defaults (fast {2,3},
slow {6,12}, stop_length {3}, stop_k {2.0}).
Default behaviour byte-identical: all existing goldens/sweep tests stay
green (541 passed, 0 failed; clippy -D warnings clean). Frictionless
Stage-1 R unchanged.
refs #137
|
||
|
Brummel
|
e94b79eb98 |
test(stage1-r-sweep): RED — grid all four knobs from CSV flags
Executable-spec for the strategy-research enabling change: expose the
stage1-r sweep's hardcoded timescales as CLI grids so a slow
time-series-momentum edge can be screened across timeframes.
- sweep_strategy_stage1_r_grids_all_four_knobs_from_csv_flags (RED):
`aura sweep --strategy stage1-r --fast 240 --slow 960
--stop-length 240 --stop-k 2.0` must yield a 1-member family whose
member params are fast=240, slow=960, stop_length=240, stop_k=2.0.
Fails today: the four flags are rejected (exit 2), gridding absent.
- sweep_strategy_stage1_r_no_flags_keeps_the_default_grid (guard,
green today): a no-flags sweep still yields the {2,3}×{6,12} 4-member
family with the stop pinned at 3 / 2.0, so existing goldens stay
byte-green once the feature lands.
GREEN side follows via implement mini-mode.
refs #137
|
||
|
Brummel
|
7aaa3e5d5c |
feat(0068): position-event derive — book first-difference table (#115)
Add `derive_position_events(record, instrument_id) -> Vec<PositionEvent>` to aura-engine (report.rs), the broker-independent Stage-2 audit layer of C10: the first difference of the executed book derived from a PositionManagement dense record. A pure post-run reduction, sibling of summarize_r — same positional, type-erased Scalar read of the 14-column record, no in-graph node (the hot path stays domain-free, C14). A reversal emits Close then the opposite open at one event_ts (close first); a window-end open position emits its open with no synthetic Close (the table records actual executed events — unlike summarize_r, which force-closes for the R metric). instrument_id is a caller-supplied scalar (aura-engine depends only on aura-core; it cannot import InstrumentSpec). Two r_col indices added (DIRECTION=4, SIZE=10), the lockstep guard in stage1_r_e2e.rs extended to name-pin `direction`, and an agreement E2E folds both summarize_r and derive_position_events over one recorded ledger (one Close per closed round-trip; every open closed-or-open-at-end; referential integrity). Scope: the derive only. Fixed-fractional / currency / equity-feedback sizing is #116 (it owns the equity->Sizer z^-1 register per C10) — deferring it here avoids a circular dependency. Supersedes the rolled-back 0064 exposure-integral derive (abandoned per #117): this derives from the executed book, never exposure deltas. Verified by the orchestrator: cargo build + clippy --all-targets -D warnings clean; full workspace suite 0 failed; aura-engine lib 214 pass (incl. the 6 derive_* unit tests); stage1_r_e2e 11 pass (incl. the new E2E + extended guard). Fork decisions recorded on #115. closes #115 |
||
|
Brummel
|
89d967a940 |
feat(0067): SQN100 metric + stage1-r sweep --trace
Two additive follow-ups to the cycle-0065/0066 R-sweep surface. Part A (#130) — n-normalized SQN (SQN100): RMetrics gains sqn_normalized = (mean_R/stdev_R)·√(min(n,100)), a turnover-robust ranking objective comparable across sweep members with different trade counts. The raw sqn is kept byte-identical (computed sqrt(n)*mean/sd, NOT via a factored shared ratio, so the existing metric does not drift even by 1 ULP). The field carries #[serde(default)] for C18 back-compat (a pre-0067 r: block deserialises with sqn_normalized = 0.0). The registry learns Metric::SqnNormalized (opt-in; the default ranker and raw sqn are unchanged), so `runs family rank sqn_normalized` works. The 0066 single-run golden is re-baselined (the r: block gains the field; for n=3 ≤ 100 the cap is inactive so sqn_normalized == sqn). Part B (#135) — wire --trace for the stage1-r sweep: stage1_r_sweep_family now honours --trace, persisting each member's equity/exposure/r_equity under runs/traces/<n>/<member_key>/ via the existing persist_traces_r (mirroring momentum_sweep_family); the interim run_sweep refusal guard is dropped. --trace is now symmetric across all three sweep strategies and a swept member is chartable (chart <n>/<member> --tap r_equity). Tests: SQN100 cap / below-cap / empty + serde back-compat (engine); rank-by sqn_normalized + unknown-metric message (registry); an E2E guard that both SQN values fold from recorded records and agree below the cap (the cross-crate column seam); inverted stage1-r --trace persistence + a CLI rank-by- sqn_normalized integration. Full workspace suite green (527 passed, 0 failed); clippy --all-targets -D warnings clean. Derived fork decisions logged on #130 and #135 (cap = fixed const 100; opt-in metric, default ranker unchanged; serde(default); Part B mirrors momentum_sweep_family + reuses persist_traces_r). closes #130 closes #135 |
||
|
Brummel
|
499121e5c8 |
fix(stage1-r): record fixed R-defining params in swept member manifests
Cycle-0066 close audit found a C18/C10 gap: a stage1-r sweep member's manifest recorded only the floated knobs (fast.length, slow.length) via zip_params, omitting the fixed R-defining params (stop_length, stop_k, bias_scale) that the single run records. A family member must be reproducible from its own manifest (C18), and the constant stop -- which defines 1R -- must be visible so cross-member SQN comparability (C10) is auditable from the family record, not just the swept fast/slow knobs. The sweep run_one now appends the three fixed params, matching the single run; the integration test pins their presence per member. Verified: full workspace suite 521 passed / 0 failed; clippy -D warnings clean. refs #133 |
||
|
Brummel
|
b8ed6f4858 |
feat(stage1-r): R sweep families + rank-by-SQN
Realizes #133 (cycle 0066): SQN becomes the single-number objective for ranking a Stage-1 sweep family by R signal quality. aura-registry — metric_cmp learns three higher-is-better R metrics (sqn, expectancy_r, net_expectancy_r) that reach into RunReport.metrics.r; a member with r:None sorts to the worst position (treated as -inf via total_cmp), so a pip-only family ranked by an R metric degrades to ordinal order without error. The UnknownMetric "known:" list is extended. No persistence change — the family store already carries the r block. aura-cli — `aura sweep --strategy stage1-r` sweeps the stage1-r harness over a fast×slow SIGNAL grid (4 members), each folding the dense R-record via summarize_r so its RunReport carries r:Some(..). The grid varies only the signal: the stop and sizing are held fixed because risk_budget is R-invariant (#128) and bias.scale is sign-only under flat-1R (both degenerate axes for the ranked metric), and because the stop defines the R unit — varying it would break cross-member SQN comparability (the motivation for the deferred SQN100, #130). The stage1-r topology is extracted into a shared stage1_r_graph(.., fast_len, slow_len) helper: the single run binds Some(2)/Some(4) (output byte-unchanged), only the sweep floats the knobs. Per-member --trace tracing is deferred (a follow-up): rather than accept the flag and silently write nothing (a quality-review catch), `aura sweep --strategy stage1-r --trace` is refused explicitly (exit 2); --name records the rankable family. Verification — a golden characterization test (stage1_r_single_run_output_golden, added against HEAD before the refactor) verifies the helper extraction is byte-preserving; full workspace suite 521 passed / 0 failed; clippy --all-targets -D warnings clean. The grounding-check that signed the spec first BLOCKed a false "already byte-pinned" claim, which is why the golden is added rather than assumed. All load-bearing forks (signal-only grid, None-sorts-worst, --trace refusal, helper extraction, golden shape) are recorded on the #133 decision log. closes #133 refs #117 #130 |
||
|
Brummel
|
3d3baaeb24 |
refactor(stage1-r): dedicated broker label + intern col-port names (no per-build leak)
Two cosmetic smells the cycle-0065 close audit flagged (#132), neither a correctness issue: 1. Broker label. run_stage1_r reused sim_optimal_manifest, recording broker: "sim-optimal(pip_size=...)" for a harness that also runs a RiskExecutor branch. It now overrides the label to "sim-optimal+risk-executor(pip_size=...)" so the dual-tap harness reads honestly. A new assertion in run_stage1_r_synthetic_folds_an_r_block pins it. 2. col[i] leak. stage1_r_blueprint did format!("col[{i}]").leak() per PositionManagement field (14 strings) to satisfy the &'static str port-name API -- harmless for a CLI one-shot, but a leak-per-build smell once the harness is reused in a sweep / Monte-Carlo loop (the #133 direction). Replaced with a process-wide LazyLock<Vec<String>> (COL_PORTS), interned once: any number of builds now reuse the same 14 strings. (#132 deferred this to #133, but the intern is robust to any reuse pattern, so it lands here cleanly.) Behaviour-preserving (the port names and the recorded R-record are unchanged); the broker label is the only observable change, and it is pinned. Verified: clippy --all-targets -D warnings clean; full suite 514 passed / 0 failed; live `aura run --harness stage1-r` reports "broker":"sim-optimal+risk-executor(pip_size=0.0001)". closes #132 |
||
|
Brummel
|
68605b7d88 |
refactor(stage1-r): rename the strategy-output param namespace exposure -> bias
Complete the deferred rename tail from cycle 0065's #126 (which renamed only the typed metric, keeping the param namespace uniformly "exposure" so the single-run rename would not smuggle in a ~30-site sweep-pinned change). Now renamed together as one consistent unit, all driven by the Bias node's instance name: - Bias::builder().named("exposure") -> .named("bias") (the harness builders + the engine fixtures in test_fixtures.rs / blueprint.rs); - the derived knob path exposure.scale -> bias.scale (Composite::param_space derives the knob from the node name) -- the sweep .axis/.range bindings and .with() points (blueprint.rs, main.rs), the walkforward ParamSpecs (walkforward.rs), the member_key dir names, and the JSON byte-pins in cli_run.rs / report.rs / main.rs tests; - the exposure_scale manifest param label -> bias_scale (the single-run manifest builders + the streaming_seam / real_bars fixtures). The rename is surgical: the three targets (named("exposure"), exposure.scale, exposure_scale) are syntactically distinct from the deliberately-kept "exposure" forms, which are untouched -- SimBroker's pre-reframe exposure input slot + prev_exposure + the exposure-integral; the LongOnly / gate / latch ports named "exposure"; and the on-disk "exposure" tap label (the recorded bias series that feeds `chart --tap exposure`, decoupled from the node name via ColumnarTrace::from_rows("exposure", ...)). No on-disk back-compat break: the knob path is a runtime param address, and recorded params in old runs.jsonl / families.jsonl are inert data strings (a pre-existing recorded "exposure.scale" still loads, it is just data). INDEX.md cycle-0065 realization note amended: the param namespace is recorded as the now-completed tail; the SimBroker slot + the on-disk tap label remain the deliberate permanent keeps (#117), and exposure_sign_flips stays a serde alias. Verified: cargo build --workspace --all-targets clean; clippy --all-targets -D warnings clean; full suite 514 passed / 0 failed (source + byte-pins renamed together); live `aura run` emits "bias_scale", `aura sweep` emits "bias.scale". closes #134 refs #126 #117 |
||
|
Brummel
|
5c6de45b6e |
fix(cli): list valid taps on bad --tap, make --help uniform across subcommands
Two CLI-UX defects the Stage-1 R fieldtest surfaced (#131): 1. Tap discovery. `aura chart <name> --tap <bad>` rejected an unknown tap but the error omitted the valid names. filter_to_tap now enumerates the run's available taps in the message ("... (available: equity, exposure)"), so the user can correct a typo instead of guessing -- refuse, but help. 2. Uniform --help. `aura chart --help` errored with "unexpected chart argument '--help'" (exit 2) and `aura run --help` leaked usage to stderr via the error path, while bare `aura --help` printed to stdout (exit 0). main() now intercepts `--help`/`-h` anywhere in the invocation, prints usage to stdout, and exits 0 -- uniform for every subcommand. The now-dead bare-flag match arm is removed (the early intercept covers it). RED-first: the existing chart_tap_nonexistent test gains an available-taps assertion (was: message had no names), and a new per_subcommand_help test pins `aura <sub> --help` -> stdout usage + exit 0 for run/chart/sweep/walkforward/ mc/graph/runs (was: chart --help exited 2). Unknown-subcommand stays exit 2. Verified: clippy --all-targets -D warnings clean; full suite 514 passed / 0 failed (one new test); live `aura chart --help` prints usage to stdout, exit 0. closes #131 |
||
|
Brummel
|
0288878ad2 |
refactor(cli): remove the redundant --macd flag, superseded by --harness macd
The --macd flag was a back-compat alias for --harness macd, added in cycle 0065 iter-3 while the uniform `--harness <name>` selector was introduced. With that selector in place the alias is pure redundancy, so drop it: --harness <name> is now the sole way to pick a built-in harness. Removed from parse_run_args: the macd_flag accumulator, its match arm, and the (harness, macd_flag) conflict resolution -- harness selection collapses to `harness.unwrap_or(HarnessKind::Sma)`. Both usage strings drop `[--macd]`; the parse_run_args doc and the parse_real_args note no longer cite the flag. The MACD harness itself is untouched: run_macd, the macd() composite, and `--harness macd` all stay. Tests updated -- the alias test becomes a positive `--harness macd -> Macd` test plus an assertion that `--macd` is now rejected as an unknown token; the "repeated --macd" edge is dropped (the flag is gone, and repeated-flag strictness is still covered by --harness/--trace). INDEX.md: the two realization notes that documented `--macd` as a live run form updated (`aura run [--real ...]`; `run --harness macd`). The historical graph-viewer retirement note and the frozen fieldtest capture are left as the record of their time. Verified: clippy --all-targets -D warnings clean; full suite 513 passed / 0 failed (assertions rewritten in place, no test count change); live smoke -- `aura run --harness macd` emits a RunReport, `aura run --macd` exits with the updated usage. |
||
|
Brummel
|
1a6eafa056 |
refactor(composites): extract Stage-1 composite-builders into their own crate
Dissolve the aura-engine -> aura-std dependency by moving the vol_stop /
risk_executor composite-builders out of aura-engine into a new aura-composites
crate. This restores the engine's domain-free layering, which the
composites-in-engine placement (cycle 0065) had inverted.
Why: the engine routes type-erased Scalar records and never names a concrete
node -- summarize_r reads the PositionManagement record positionally, by column
index, without linking aura-std (which is exactly why aura-std was a dev-only
dependency before 0065). Parking the node-naming composite-builders inside
aura-engine forced the engine's production code to reference the node library
for the first time (aura-engine -> aura-std), an upside-down layering: the
engine is the foundation the node library builds upon, not the reverse.
The clean home is a dedicated crate that depends on BOTH the engine's
GraphBuilder and the std nodes, leaving each lower crate untouched:
aura-composites -> { aura-engine, aura-std } -> aura-core (acyclic)
aura-engine -> aura-core only (domain-free again)
aura-std -> aura-core only (lean node library)
aura-std reverts to a [dev-dependency] of aura-engine (the engine's own E2E
tests -- stage1_r_e2e, random_sweep_e2e, ger40_breakout -- still drive real
std nodes through a bootstrapped Harness; that test-only edge creates no cycle).
Moved with history (git mv):
- crates/aura-engine/src/composites.rs -> crates/aura-composites/src/lib.rs
- crates/aura-engine/tests/risk_executor.rs -> crates/aura-composites/tests/
- crates/aura-engine/tests/vol_stop_composite.rs -> crates/aura-composites/tests/
Consumers rewired: aura-cli imports risk_executor / StopRule from aura_composites;
the moved tests import the builders from the crate under test. The intra-doc link
to RollMode and the module rationale doc are updated for the new home.
Behaviour-preserving: workspace build clean; clippy --all-targets -D warnings
clean; full suite 513 passed / 0 failed, identical to baseline (the two moved
test files run unchanged under aura-composites); cargo doc --no-deps clean (no
broken intra-doc links).
|
||
|
Brummel
|
c6c1d3bb10 |
feat(stage1-r): rename the exposure_sign_flips metric to bias_sign_flips (#126)
Complete the typed-field half of the exposure->bias rename - the one the cycle-0065
close audit flagged as aging worst (a serde-stable on-disk key).
- RunMetrics.exposure_sign_flips -> bias_sign_flips with
#[serde(alias = "exposure_sign_flips")], so legacy runs.jsonl still deserialises
(pinned by the legacy-read tests, which keep the old key); new output serialises
bias_sign_flips (the byte-pin tests updated to the new key).
- The registry rank metric: Metric::ExposureSignFlips -> BiasSignFlips, the rank
string accepts BOTH "bias_sign_flips" and "exposure_sign_flips" (CLI back-compat),
the error/listing updated; the rank tests exercise both names.
- The MC aggregate field renamed too (McAggregate is hand-rendered to JSON, not
serde-derived - no alias needed).
Scope held to the typed metric. The strategy-output PARAM NAMESPACE - the
.named("exposure") Bias instance, its exposure.scale knob path (Composite::param_space
derives the knob from the node name), and the exposure_scale manifest label - is a
coherent ~30-site, sweep-pinned surface (the implement loop correctly flagged renaming
the knob as an unscoped expansion touching walkforward + the cli_run byte-pins). Kept
uniformly as "exposure" and deferred to a follow-on so the cluster renames as one
consistent unit. SimBroker's pre-reframe exposure concept and the on-disk "exposure"
tap label stay by the #117 fork decision.
cargo test --workspace green; clippy --workspace --all-targets -D warnings clean; a
live `aura run --harness stage1-r` emits "bias_sign_flips".
closes #126
refs #117
|
||
|
Brummel
|
4e6b1d9b53 |
audit(stage1-r): cycle-0065 close — ledger records the Stage-1 R signal-quality layer
Architect drift review of cycle 0065 (Stage-1 R: the exposure->bias rename, the
stop-rule + vol_stop, PositionManagement's dense R-record, the Sizer, summarize_r,
the public RiskExecutor, and the CLI surface): code is drift-clean - C10 fidelity
holds (bias->stop->Sizer->PositionManagement, flat-1R feed-forward, R stop-defined
and size-invariant, guarded by the cross-crate layout lockstep test), C9/C23
acyclic, the --harness selector is a compile-time match (no runtime registry/DSL,
C17). The milestone fieldtest is green after the tiny-pip tolerance fix (
|
||
|
Brummel
|
2ee0d74a8c |
fix(stage1-r): scale-robust tolerance for the R-record redundancy guard
The PositionManagement redundancy `debug_assert` (position_management.rs:229) re-derives realized_r from the dense-record columns as dir*(exit-entry)/|entry-stop| and compared it to the stored value with an ABSOLUTE < 1e-9 tolerance. At tiny-pip price scale (EURUSD/GBPUSD entry ~1.1, vol-stop distance ~1e-6) the denominator reconstruction `entry - stop` is a catastrophic-cancellation subtraction that loses ~6 significant digits; with a deep gap-through-stop realising a large |R|, the recomputed R drifts from the (correct) stored realized_r by more than 1e-9 - so a debug build PANICS (exit 101) while a release build (assert compiled out) returns the correct R. GER40 (large pip) never trips it. Found by the Stage-1 R milestone fieldtest: `aura run --harness stage1-r --real EURUSD` crashed the headline "score real FX in R" use case on the default (debug) build, and the debug/release disagreement made the R numbers' validity unknowable from the public surface. Root cause (via debug): the stored realized_r uses the cancellation-free FROZEN latched_dist and is exact - only the guard's tolerance was wrong for small price scales. Fix: a scale-robust relative tolerance `< 1e-9 * realized.abs().max(1.0)` (floored at the historical 1e-9 for |R| <= 1, so no change for normal-scale runs). RED-first: the hermetic synthetic-tiny-pip test `tiny_pip_gap_stop_does_not_trip_the_redundancy_guard` reproduces the panic and is pinned here. Verified end-to-end: `aura run --harness stage1-r --real EURUSD` now returns the R block (exit 0) on a debug build. cargo test --workspace green; clippy --workspace --all-targets -D warnings clean. refs #117 |
||
|
Brummel
|
a6fc48daa7 |
feat(stage1-r): operate the R layer from the CLI (iter 3)
`aura run --harness stage1-r [--real <SYM>] [--trace <n>]` now runs a Stage-1 R
backtest and prints its signal-quality metrics (the R block) in the RunReport
JSON - one shell call, the research loop's primary verb, ergonomic for Claude
to drive.
What shipped:
- vol_stop + risk_executor promoted from test fixtures to public aura-engine
composite-builders, with a StopRule{Fixed,Vol} axis (C11). aura-std becomes a
normal dependency of aura-engine (the composites are the engine's convenience
layer over the standard nodes - the sibling tier of summarize_r; the graph
stays acyclic: aura-engine -> aura-std -> aura-core). Both fixtures call the
public fns; a Vol-backed RED test pins the new match arm.
- stage1_r_blueprint fans one bias into BOTH a SimBroker (pip) and the
RiskExecutor (R), so one report carries both yardsticks honestly. run_stage1_r
folds summarize (pip) + summarize_r (R, round_trip_cost 0.0 - Stage-1 is
frictionless signal quality) and sets RunMetrics.r = Some(..). An r_equity tap
(cum_realized_r + unrealized_r via LinComb) persists through a separate 3-tap
persist_traces_r and renders via the existing `aura chart --tap r_equity`.
- `aura run --harness <sma|macd|stage1-r>`: a parse_run_args tokenizer replaces
the five `run` literal-slice arms so --harness/--real/--from/--to/--trace
compose in any order; --macd kept as a back-compat alias; --harness sma the
default. A compile-time match over Rust-authored built-ins - not a runtime node
registry, not a DSL (C9/C17): the CLI runs an authored harness, it does not
wire one.
Back-compat: --harness sma/macd leave RunMetrics.r = None, so skip_serializing_if
keeps pip-only and legacy runs.jsonl JSON byte-unchanged; the plain-run tap list
stays ["equity","exposure"].
Refactor beyond the plan (verified behavior-equivalent): the old standalone
parse_real_args was orphaned by the new tokenizer, so it and its now-redundant
unit test were removed and run's --real parsing inlined into parse_run_args. The
--real strictness (empty-symbol, non-i64 ms, repeated-flag, window-requires-real
-> exit 2) is preserved and now pinned by
parse_run_args_rejects_malformed_real_and_repeated_flags (added to restore the
deleted coverage). All five run_real integration tests stay green.
Verification: cargo build/test --workspace green (every crate, 0 failed); clippy
--workspace --all-targets -D warnings clean. The full diff was adversarially
reviewed (run-arg refactor equivalence against the removed function, harness
wiring, C1/C2/C9/C17, the serde back-compat pin) - verdict SOUND; the one flagged
coverage gap is closed here.
Follow-on (noted, not done): the stage1-r manifest reuses the
"sim-optimal(pip_size=...)" broker label; a dedicated "risk-executor" label would
read more honestly for a dual-tap harness that runs a RiskExecutor branch.
closes #129
refs #117 #128
|
||
|
Brummel
|
92591e5dc0 |
refactor(stage1-r): name the conviction column conviction_at_entry
Rename the dense PositionManagement record's column 9 from `bias_at_entry_abs` to `conviction_at_entry` (FIELD_NAMES, the `r_col`/test index constants, the cross-crate layout guard, and the doc comments). Why: the column holds |bias| at entry, which C10 defines AS conviction (bias = direction + conviction magnitude; ledger INDEX.md ~458). The record already names the other half of that decomposition by its domain concept (`direction` = sign, not `bias_sign`), so naming the magnitude half mechanically (`bias_at_entry_abs`) was an inconsistency; `conviction_at_entry` speaks the ledger's ubiquitous language and parallels `direction`, while `_at_entry` preserves the frozen-snapshot precision. Considered and rejected: collapsing `(direction, bias_at_entry_abs)` into one signed `bias_at_entry` and dropping `direction` as redundant — it is not redundant. On a reversal row col 4 (`direction`) tracks the reopened leg while col 9 tracks the closed trade's entry conviction, so the two describe different trades and cannot be losslessly merged; they are also distinct semantic axes (current-position direction vs per-trade entry conviction). Scope: domain name at the data boundary, mechanical name internally — the internal `bias.abs()` variables (`Open.bias_abs`, the fold's `Trade.bias_abs`) keep their computation-faithful names, exactly the raw-internal / domain-external split the record uses elsewhere. Behaviour-preserving: cargo build --workspace clean; cargo test --workspace 500 passed, 0 failed (unchanged); clippy --all-targets -D warnings clean. The layout guard (`r_col_indices_match_producer_field_layout`) confirms the renamed string. refs #117 |
||
|
Brummel
|
b4e84335c4 |
feat(stage1-r): Sizer + RiskExecutor + R-metric enrichment (iter 2)
Iteration 2 of cycle 0065 (Stage-1 R signal quality) — the node + metric layer. Builds on iter-1's PositionManagement dense R-record + core summarize_r. What ships: - PositionManagement gains a 4th `size` input (slot 3 -> col 10), fed by the Sizer. R is computed size-INVARIANTLY (pure stop-distance ratio), so size never touches realized_r — pinned by a RED test (scaling size leaves every realized_r unchanged) at the node and again end-to-end through the executor. - Sizer (aura-std): `size = risk_budget / stop_distance` — flat-1R (risk_budget 1.0 => one risk unit per trade, size inversely proportional to the stop, not a constant). Reads bias (firing/presence) + stop_distance; the Stage-2 fixed-fractional sizer slots in unchanged (swap risk_budget for risk_fraction*equity, same node shape). - summarize_r enrichment: SQN (sqrt(n)*mean_R/sample-stdev_R; n<2 or zero-variance -> 0), conviction_terciles_r (E[R] by |bias_at_entry| tercile), net_expectancy_r (gross minus one round-trip cost per trade, charged in R via the latched_dist recovered from the entry_price/stop_price columns). summarize_r gains a `round_trip_cost` param (price units). The net-of-cost recovery is tested through the real producer->consumer seam, not just hand-built rows. - RunMetrics.r: Option<RMetrics> with #[serde(default, skip_serializing_if = "Option::is_none")] — legacy runs.jsonl (no `r` key) deserialise to None and a pip-only run's on-disk shape stays byte-unchanged (C14/C18 back-compat). Every RunMetrics literal threaded (report.rs, aura-registry, aura-engine/mc). - RiskExecutor (aura-engine integration-test fixture, sibling of vol_stop_composite): FixedStop -> Sizer -> PositionManagement behind open bias+price input roles, price fanned to both the stop and PM, the Sizer's size into PM. Bias is produced in-graph from the single price source (a second bias *source* would k-way-merge into separate cycles and mark stale prices). The Veto is a DOCUMENTED SEAM, not a runtime node (a pass-through identity is what C19/C23 DCE deletes). Tests: the composite bootstraps + runs + folds to the documented hand value; R invariant under risk_budget while the size column scales; a live-folded RMetrics survives the RunMetrics serde round-trip. The dense-record size column (10) is brought under the cross-crate layout guard (stage1_r_e2e r_col_indices_match_producer_field_layout) so the executor fixture's size-invariance read is drift-protected like the others. Scope: the CLI/recording surface (#129) is sub-split into a separate iteration 3 and is NOT in this commit. Verified: cargo build --workspace clean; cargo test --workspace 500 passed, 0 failed; cargo clippy --workspace --all-targets -D warnings clean. refs #117 #127 #128 #129 |
||
|
Brummel
|
0998f9aabf |
refactor(stage1-r): the vol stop is a composition, not a fused node
Applies the principle the user named: a node is a primitive only if it is NOT DAG-expressible from other primitives; a function that needs a missing primitive gets the primitive added, not fused away. - Remove the fused VolStop node (it was pure feed-forward arithmetic). - The volatility stop is now a composition of primitives, vol_stop(length, k): k * Sqrt(Ema(Mul(d,d), length)), d = Sub(price, Delay(price,1)) -- a rolling EWMA standard deviation. Built with GraphBuilder; proven end-to-end (bootstraps + runs + emits k*sigma) in tests/vol_stop_composite.rs. - Migrate the one VolStop caller (stage1_r_e2e.rs) off it: the R-is-stop-defined test now contrasts a tight vs a wide FixedStop (two constant distances still fold to different R, the property under test). Uses the Mul + Sqrt primitives added in the prior commit. FixedStop stays the only stop-rule primitive (a triggered constant). Full suite + clippy green. refs #117 #119 |
||
|
Brummel
|
831092841e |
feat(aura-std): add Mul + Sqrt primitives
The two genuinely-missing arithmetic primitives (Mul = two-stream f64 product, Sqrt = one-input f64 root, negatives clamped to 0). They are the building blocks for the volatility stop as a composition (rolling EWMA stddev), replacing the fused VolStop node. Also corrects plan 0066 Task 3 (the VolStop removal must migrate its stage1_r_e2e.rs caller — a false premise the implementer caught). refs #117 #119 |
||
|
Brummel
|
2c43296c2c |
feat(stage1-r): bias rename + stop-rule + position-management + summarize_r (iter 1)
Iteration 1 of the Stage-1 "R-based signal quality" cycle (spec 0065): a strategy's signal quality, measured in R on its unsized bias stream, feed-forward. New discrete-trade machinery, NOT a SimBroker extension. What lands: - exposure -> bias (refs #126): the Exposure node/type/file/output-field renamed to Bias (computation unchanged: clamp(signal/scale,-1,+1); the SEMANTICS change — the output is the unsized strategy bias, sizing leaves the strategy). Scoped to the semantic core; behaviour-preserving cosmetics are deferred (see below). - stop-rule nodes (refs #119): VolStop = k*EMA(|price - prev_price|) (one fused node; the volatility that defines 1R, close-to-close — true-range ATR deferred, needs OHLC) + FixedStop (constant, the test fixture / fixed-vs-vol structural-axis sibling). - PositionManagement (refs #127): the stateful heart. Latches the entry-cycle stop distance as the FROZEN R-denominator (never re-read), marks/exits no-look-ahead like SimBroker (a position earns from the next cycle; an exit realises against the cycle's close), and emits ONE dense per-cycle R-record (C8). Stop-outs are NOT capped at -1R (a gap through the stop realises R < -1 — the honest loss tail); R is computed size-invariantly (size = (exit-entry)*dir / latched_dist cancels). The trade ledger is the rows where closed_this_cycle; the R-equity is cum_realized + unrealized; a position open at window end is the last row (open=true) — explicit, never silent MtM. - summarize_r + RMetrics (refs #129): the post-run fold (NOT an in-graph node — recorders drain post-run). E[R], win-rate, profit-factor, avg win/loss R, by-trade max R-drawdown, n_open_at_end (window-end force-close). SQN / conviction terciles / net-of-cost / RunMetrics.r are iteration 2. Design adversarially hardened before implementation (12-juror refute panel; decisions on #117). Ratified implementer deviations from the plan snippet, verified by hand: - col-4 `direction` tracks the OPEN position at cycle end (window-end synthesis; names the reopened leg on a reversal), falling back to the closed trade's dir only when flat. summarize_r does not read col 4; the R-metrics are unaffected. - .named("exposure") kept on the 4 previously-unnamed Bias nodes so the auto-derived param-path stays exposure.scale (no manifest/dir-name drift) — the unnamed nodes would otherwise flip to bias.scale. - intra-doc links [`Exposure`] -> [`Bias`] in sim_broker/latch + the sample-model test pin (cosmetic, behaviour-neutral; broken intra-doc links fail cargo doc). - the E2E drives a full bootstrapped Harness (stronger than the plan's direct-node drive — exercises the real cross-crate producer->consumer seam). Deferred (behaviour-preserving, separate cosmetic pass — NOT this iteration): RunMetrics.exposure_sign_flips / Metric::ExposureSignFlips, SimBroker/LongOnly "exposure" input ports, the "exposure" tap/trace names, and the .named("exposure") instance labels. Iteration 2: the Sizer seam, the RiskExecutor composite (+ the Veto documented-seam), summarize_r enrichment, RunMetrics.r, and the CLI/recording surface. Verification (orchestrator-run, not agent-claimed): - cargo build --workspace: clean. - cargo test --workspace: all green, 0 failed (incl. the new bias/stop_rule/ position_management unit tests, the summarize_r arithmetic tests, and the stage1_r_e2e capstone + layout guard). - cargo clippy --workspace --all-targets -- -D warnings: clean (exit 0). - Keystone RED tests pass: no_lookahead_bias_exit_realises_the_held_move, stop_out_is_not_capped_at_minus_one_r, no_gap_stop_is_exactly_minus_one_r, reversal_closes_one_leg_and_reopens, open_at_window_end_is_carried_on_the_last_row. refs #117 #119 #126 #127 #129 |
||
|
Brummel
|
b91c89f964 |
feat(broker-foundation): InstrumentSpec deploy metadata + PositionEvent schema
The realistic-broker milestone's (C10 A-side) two foundation value types — no broker, no derivation yet, fully tested. #113 — InstrumentSpec gains six deploy-grade fields (instrument_id, contract_size, pip_value_per_lot, min_lot, lot_step, quote_currency) and the vetted table is populated for GER40/FRA40/EURUSD/GBPUSD/USDCAD. The struct stays Copy (quote_currency is &'static str); the _ => None refuse-don't-guess arm is the permanent floor. This is the milestone's permanent authored floor (recorded-metadata tier is forward work, #124). #114 — PositionAction { Buy, Sell, Close } (closed enum, serde-encoded as a bare i64 0/1/2 via From/TryFrom) + PositionEvent in aura-engine beside RunMetrics, faithful to the C10 column spec: event_ts, action, position_id, instrument_id, unsigned volume; no open_ts; direction is the action (no signed-volume trick). Re-exported from the crate root. Derived fork decisions (recorded on #113): quote_currency &'static str; action as ordinal i64; stable instrument_id + overridable vetted values. Verified by the orchestrator: cargo test --workspace = 458 passed / 0 failed; clippy --all-targets -D warnings clean. An adversarial 4-lens review (ledger faithfulness, downstream soundness, serde/wire robustness, test honesty) returned 3 SOUND + 1 CONCERN: quote_currency &'static str cannot hold a runtime-sourced currency at the #124 resolver — not a blocker (#115/#116 read only the authored floor), logged on #124 as a deliberate deferred design choice. The implementer's deviation from the plan snippet (vec![..] -> [..]) avoids clippy::useless_vec; semantics identical. Two tester-authored consumer-boundary E2E tests added beyond the inline units: instrument_spec_deploy_metadata.rs (public resolution seam) and position_event_table.rs (persisted bare-int wire shape + reversal table). closes #113 #114 |
||
|
Brummel
|
2957561c30 |
fix(chart): mean-decimate the bounded exposure series instead of a min/max band
The serve-time decimation (#108) reduced every series by per-bucket min/max. That is the right envelope for a cumulative equity curve but wrong for the bounded exposure stream (C10, f64 ∈ [-1,+1]): over a multi-year window each ~hundreds-of-bars bucket straddles sign flips, so min/max is ±1 in nearly every bucket and the exposure collapses to a solid -1..+1 band that reads as per-point oscillation — even for a calm strategy (a 50/200 member flips only 0.81% of bars yet still bands out). Make decimation tap-aware (RED-first, #111): a new `ReduceKind {MinMax, Mean}` on each Series, set by the chart builders via `reduce_for_tap` (exposure -> Mean, else MinMax). `decimate` honours it — a Mean series emits the per-bucket mean (its net/duty-cycle level) in both spine slots, a MinMax series keeps min/max. The shared spine, the equity rendering, and the page payload are unchanged (reduce is server-side only, `#[serde(skip)]`). Verified on real GER40 1y M1: the served exposure series goes from a ±1 band to a smooth net-level line in [-0.38, +0.34]. Rendering the min/max envelope honestly (range bars / OHLC, vs today's polyline) is the deferred other half — filed as #112. Verified: cargo test --workspace = 447 passed / 0 failed (incl. the RED-then-GREEN decimate_mean_reduces_a_bipolar_series_to_its_bucket_level); clippy -D warnings clean. closes #111 refs #112 |
||
|
Brummel
|
476342d7b1 |
feat(chart): decimate served pages + run-context header (visual World cut 2)
Hardens the families-comparison chart page (#107) so it is openable and legible on real multi-year data. Two halves, both confined to the CLI render path (engine + registry untouched, C14): #108 — serve-time min-max decimation. A pure `decimate(ChartData, buckets)` transform partitions the shared union-ts spine into <=~2000 buckets and emits each series' per-bucket min+max (nulls preserved), bounding the served page to a few thousand points regardless of the underlying M1 point count. Applied in emit_chart on both the single-run and family paths; a no-op under budget (every existing fixture). Full recorded data stays on disk; only the page is thinned. For a walk-forward family the null-fill blow-up collapses for free (an all-null bucket stays null). Deterministic (C1). #102 — a run-context header. A `ChartMeta` (name/commit/window/broker/seed/taps, + member count for a family, + bound params for a single run) is built from the RunManifest in build_chart_data / build_comparison_chart_data (which gain a `name` arg), injected into window.AURA_TRACES.meta, and rendered by a new pure buildHeader in chart-viewer.js (headless .mjs-guarded). The family window is the SPAN across members (min from, max to), not the first member's window — the only reading that labels a disjoint walk-forward family's true OOS coverage (it collapses to the shared window for sweep/MC). Reuses the existing render_value for param display (no new Scalar Display, keeping C7's tag-only param plane). The earlier "manifest is NOT carried into the page" render pin is flipped to a positive one. Verified: cargo test --workspace = 446 passed / 0 failed; cargo clippy --workspace --all-targets -D warnings clean. New coverage: 5 decimate unit tests, single-run + family meta wiring (incl. the span-window invariant), a buildHeader headless guard, and two end-to-end cli_run tests pinning the served-page meta at the binary boundary. closes #108 closes #102 |
||
|
Brummel
|
f51d54a1c1 |
docs(cell): name Scalar as the param-authoring surface; no tagged Cell ctors by design
Cell is the kind-erased hot-path carrier (C7): the value carries no tag, so there are deliberately no tuple-style/tagged constructors (`Cell::I64(..)`). Document on the struct that Scalar is the value to reach for when authoring a param / bind / sweep point / RunManifest field, and that the from_* ctors are the carrier-side entry. Resolves the discoverability papercut without softening the carrier split. closes #100 |
||
|
Brummel
|
4c64feb9ed |
feat(chart): aura chart <family> overlays a family's members; --tap + write-guard
`aura chart <name>` now branches on the name's on-disk kind: a single run charts
its taps as before; a FAMILY overlays one tap (default equity) across all its
members in one self-contained uPlot page, consistent across sweep / MC /
walk-forward. Members share ONE y-scale (they measure one quantity — that is what
makes the overlay comparable, unlike the single-run overlay of different taps),
and align on the union-ts spine via the existing join_on_ts: sweep/MC members
share the window (true overlay), walk-forward members are disjoint OOS windows
(null-complementary -> the stitched curve). One mechanism, three correct readings.
- build_comparison_chart_data (one labelled Series per member, shared y_scale_id);
filter_to_tap (single-run --tap restricts to one tap; no --tap = all taps,
unchanged); parse_chart_args (`<name> [--tap <t>] [--panels]`, any order);
emit_chart name-kind branch; USAGE updated.
- Write-guard: ensure_name_free is now called once per tracing command (run /
run --macd / run --real -> Run; sweep / mc / walkforward -> Family) so a name
cannot be reused across kinds — name resolution stays a total function.
Refuse-don't-guess throughout (exit 2, never panic). Engine untouched (C9/C14).
Tasks 2-3 of plan 0061; Task 1 (registry) landed in
|
||
|
Brummel
|
9b2adcbb1b |
feat(trace-store): name classification, family resolver, write-guard
aura-registry gains the read-side of total trace-name resolution and the
write-guard that makes it total:
- NameKind {Run, Family, NotFound} + TraceStore::name_kind — top-level index.json
is a single run; else ≥1 member subdir with index.json is a family; else absent.
- read_family -> Vec<FamilyMember{key, traces}> — every member of a family, read
via the existing read("<name>/<key>"), sorted by key (deterministic, C1); an
absent/empty family reads as empty (treat-as-absent), a malformed member
propagates Parse.
- ensure_name_free(name, WriteKind) + a TraceStoreError::NameTaken variant —
refuses only cross-kind name reuse, so the one ambiguous on-disk state (a name
used by both a run and a family) is unreachable; same-kind overwrite stays Ok.
Task 1 of plan 0061. Engine untouched (C9/C14). Tests: cargo test -p aura-registry
green (18 passed, incl. the 3 new), clippy -D warnings clean.
Note: the implement-loop quality gate exhausted its re-loop oscillating on the
method NAME (ensure_name_free "implies unused" vs a rename diverging from the
ratified plan/callers). Orchestrator overrule: the ratified name stands — it is
fixed by the spec/plan/#107 and the Task-3 call sites, and the doc comment already
states that Ok means "writable here", not "unused". The implementation is green,
clippy-clean, and spec-faithful.
refs #107
|
||
|
Brummel
|
e1ad0979de |
refactor(ger40-examples): source pip from instrument_spec, drop the baked literal
The runnable GER40/FRA40 examples baked `const PIP_SIZE = 1.0` and a dead pip column in the compare INSTRUMENTS table — a second source of pip truth parallel to the vetted `aura_ingest::instrument_spec` channel the CLI `--real` path already uses (#22). Route them all through one `pip_size_of(symbol)` helper: build_harness sources GER40 internally, ger40_breakout_blueprint gains a `pip_size` first arg fed from the lookup, and compare's run_cell sources each instrument's pip per cell. Value-identical (GER40/FRA40 both resolve to 1.0) and behaviour-preserving: the gated determinism / blueprint param-space / world tests stay green unchanged, and the frozen `sim-optimal(pip_size=1)` broker-label pins are untouched. closes #98 |
||
|
Brummel
|
094d63bc47 |
fix(family-stdout): unify family-wrapper key order with the store
The sweep / walk-forward / mc family-member stdout lines routed the embedded RunReport through serde_json::json! -> a serde_json::Value, which re-alphabetizes the manifest keys (broker-first) and, for mc, even reordered the top-level keys (seed after report). That diverged from the commit-first declaration order that RunReport::to_json and the stored families.jsonl already use. Splice the pre-serialized report into a hand-built line via family_member_line / mc_member_line so each family-member stdout line is byte-identical to its families.jsonl record (commit-first manifest, seed between family_id and report). RED-first: the two new key-order tests fail against the old json! helpers and pass after the format! rewrite. closes #99 |
||
|
Brummel
|
99fd32b1f9 |
feat(real-family): realistic strategy lengths over real data
The built-in family grid was calibrated for the synthetic demo streams (18/60
bars), where trend SMA 2-5 / MACD 2-4-3 are the only lengths that warm. Over real
M1 data those are noise — ~9000 exposure sign-flips per month, a strategy trading
the bid-ask wiggle, not a trend.
DataSource::strategy_lengths() makes the grid data-kind-dependent (like
wf_window_sizes): synthetic keeps {2,3}x{4,5} + MACD 2/4/3 (byte-unchanged); real
uses {50,100}x{200,400} trend SMAs + standard 12/26/9 MACD. Threaded through
sweep_family + sweep_over (the walk-forward in-sample sweep). Over real EURUSD the
monthly flip count drops from ~9000 to ~130-310 — a genuine trend cross.
This is a demo-strategy calibration patch (ledger C22 amendment notes it as such);
the real answer is project-authored strategies owning their own grid (C9), deferred
to the project-env work.
Verified: cargo test -p aura-cli (all green, synthetic byte-unchanged),
clippy -D warnings clean; real sweep probe shows ~130-310 flips/month.
refs #106
|
||
|
Brummel
|
8e5d14b2bb |
feat(real-family): --real parser + dispatch wiring; complete the feature
Plan 0060 Tasks 4-5 (#106): wire `aura sweep|walkforward --real` end to end. - A shared RealWindowGrammar accumulator holds the `--real <SYMBOL> [--from <ms>] [--to <ms>]` grammar (flag-repeat strictness, empty-symbol + window-without-real rejection) for both parsers, mirroring parse_real_args. - parse_sweep_args grows the DataChoice 4th element; parse_walkforward_args is new (no --strategy axis). The sweep + walkforward dispatch arms become rest-matched, build the DataSource via from_choice (build-or-refuse on un-vetted symbol / absent data), and run. USAGE documents the --real tails. mc stays exact-matched, so `aura mc --real` falls through to usage + exit(2) — Fork A: MC excluded (its seed varies a synthetic price-walk, undefined over real data). - Gated integration tests (skip on no local data): sweep --real EURUSD persists 4 portable member dirs over real bars and charts; walkforward --real EURUSD persists one oos<ns> dir per rolling window; same real sweep twice is byte-identical (C1). Un-gated: un-vetted symbol, window-without-real, and mc --real each refuse with exit 2. - Remove the now-redundant `#[allow(dead_code)]` from the Task-1 provider: every const, both enums, and every method are live once the wiring lands. Verified end-to-end over real EURUSD data: sweep --real -> 4 members; walkforward --real over 2024 -> 9 rolling OOS windows; all refusals exit 2. Full workspace test + clippy -D warnings green. closes #106 |
||
|
Brummel
|
2a6ca47acd |
feat(real-family): thread DataSource + pip through the family builders
Plan 0060 Tasks 2-3 (#106): the family builders now take a &DataSource and draw their source, pip, window, and walk-forward roller sizes from it instead of the hardcoded synthetic VecSource / SYNTHETIC_PIP_SIZE. - sample_blueprint_with_sinks / momentum_blueprint_with_sinks take a pip_size (un-hardcode the SimBroker); every caller threads it. - sweep_family / momentum_sweep_family / walkforward_family / sweep_over / run_oos take &DataSource; run_sweep / run_walkforward take a DataSource and the dispatch passes DataSource::Synthetic (the --real parser is the next step). Synthetic behaviour is byte-unchanged. - Fix a design gap the implementer correctly bounced on: walk-forward is a windowed consumer whose span comes from the 60-bar walkforward_prices, NOT the 18-bar showcase that full_window() returns. Added DataSource::wf_full_span() (synthetic walk-forward span; real = the probed --from..--to) and pointed walkforward_family at it. Without it the synthetic roller (24,12,12) over span (1,18) exits SpanTooShort. Verified: cargo test -p aura-cli (all green, synthetic byte-unchanged), cargo clippy -p aura-cli --all-targets -D warnings (clean). refs #106 |
||
|
Brummel
|
cf102172ec |
feat(real-family): DataSource provider + shared real-data helpers
Foundation for `aura sweep|walkforward --real` (#106, plan 0060 Task 1). - New CLI-side `DataChoice` (parsed) / `DataSource` (opened) provider: the synthetic-vs-real source choice as one type, supplying a member's source, pip, full/windowed sources, and walk-forward roller sizes. Not yet wired into the family builders (plan Tasks 2-5); `#[allow(dead_code)]` marks the wired-later items, to be removed once the builders thread `&DataSource`. - DRY: `run_sample_real`'s inline pip-refusal / no-data refusal / probe-window logic is extracted to shared `instrument_spec_or_refuse` / `no_real_data` / `probe_window` helpers, now reused by `DataSource::from_choice` / `full_window` instead of duplicated. - Real walk-forward roller-size constants (90d / 30d / 30d in ns, Fork D/F). The synthetic provider is consumer-dependent by design (full-window consumers draw showcase_prices; windowed consumers draw walkforward_prices) so the byte-unchanged guarantee holds across both family paths. Verified: cargo test -p aura-cli (all green incl. byte-unchanged synthetic path), cargo clippy -p aura-cli --all-targets -D warnings (clean). Full-workspace gate at iter close after Tasks 2-5. refs #106 |
||
|
Brummel
|
0b73a75d4b |
feat(momentum): bool-param demo strategy + aura sweep --strategy selector
Proves the #105 generic portable member key end-to-end: a demo strategy whose params have nothing in common with the SMA-cross demo, swept by the same machinery, with a bool axis surfacing — conformant — in the member dir names. - momentum strategy: price -> Ema(length:i64) -> Sub(price-ema) -> Exposure(scale:f64) -> LongOnly(enabled:bool) -> SimBroker. Three swept params of three kinds incl. a bool; the exposure sink taps the post-gate exposure so the bool's effect is in the trace. - aura sweep gains --strategy <sma|momentum> via a pure, unit-tested parse_sweep_args (mirroring parse_real_args); default = SMA-cross, so the existing bare / --name / --trace forms are unchanged. - `aura sweep --strategy momentum --trace mom` persists 8 member dirs named e.g. ema.length-5_exposure.scale-0.5_longonly.enabled-true — the bool in the dir name, every name matching [A-Za-z0-9._-], each chartable. - ledger (docs/design/INDEX.md): the C22/#101 member-key note's sweep clause amended to the generalised portable form (MC seed{N} / walk-forward oos{ns} unchanged). Self-verified: cargo build --workspace, cargo test --workspace (all green incl. momentum_param_space_is_ema_exposure_longonly, the 8-point determinism test, the parse_sweep_args grammar test, and the momentum_sweep_trace integration test asserting 8 portable bool-bearing dirs + a chartable member), cargo clippy --workspace --all-targets -D warnings. refs #105 |
||
|
Brummel
|
27f850dc52 |
feat(sweep-key): generic filesystem-portable member key + LongOnly bool node
Generalises the #105 foot-gun: the family-member trace key was derived from two hardcoded axis names (f{fast}s{slow}), collision-free only over the built-in SMA grid. It is now derived from the axes that actually VARY (SweepBinder::varying_axes) and rendered as a filesystem-conformant directory component: - charset restricted to [A-Za-z0-9._-] (valid on Linux/Windows/macOS, also URL-path- and cloud-sync-safe); any other byte sanitised; - case-less values (i64/timestamp decimal, bool true/false, f64 via Display, no scientific notation) so two members of one family never differ only by letter case -> no silent overwrite on case-insensitive filesystems (NTFS/APFS); - length-capped (MAX_KEY=200) with a version-stable FNV-1a fallback for the unbounded-grid edge. Pinned singleton axes are omitted; the SMA sweep's member dirs change from f2s4 to signals.trend.fast.length-2_signals.trend.slow.length-4 (its --trace integration test + the stale prose updated accordingly). Adds the first aura-std node with a bool *param*, LongOnly (a long-only exposure gate: enabled=true clamps short exposure to >=0, false passes through) — the block the momentum demo strategy (next commit) sweeps to prove the key generic over a bool axis. The engine change is one additive pure read accessor; no existing signature changes, trace state stays out of the engine. Self-verified: cargo build --workspace, cargo test --workspace (all green incl. the LongOnly node tests, the member_key worked-examples corpus, the varying_axes accessor test, and the updated SMA sweep-trace integration test), cargo clippy --workspace --all-targets -D warnings. refs #105 |
||
|
Brummel
|
d3cb5f8052 |
feat(family-traces): persist per-member streams for sweep/MC/walk-forward
Family runs (sweep, Monte-Carlo, walk-forward) ran a full harness per
member and drained its equity+exposure recorders, but immediately folded
the streams to metrics and discarded them — so the C21 family-comparison
view had no per-member data to chart. A new opt-in `--trace <name>` on
`aura sweep|mc|walkforward` now persists each member's streams to disk,
mirroring the single-run `aura run --trace` path. Any single member is
chartable today via `aura chart <name>/<member_key>`; the comparison view
itself is a later cycle.
Design:
- Persist INSIDE each per-member closure (aura-cli), reusing persist_traces
verbatim — the engine and the family types are untouched. The closures
already held the drained rows; they now bind them once, build the
manifest, persist when --trace is set, then fold as before.
- Content-derived, deterministic member keys (MC seed{N}, sweep f{fast}s{slow},
walk-forward oos{ns}). The engine HOFs are Fn + Sync — members run in
parallel, so a runtime ordinal counter would be non-deterministic (a C1
break); keys derive from what each closure receives deterministically.
sweep_member_key panics (naming the axis) if the grid lacks a key axis,
rather than silently collapsing two points to a shared dir.
- Each member is a standalone run-dir at runs/traces/<name>/<member_key>/
(persist_traces with a slash-name; TraceStore nests via create_dir_all),
so the existing single-run viewer charts a member with no view-side code.
- Opt-in: without --trace, stdout and the run registry are byte-unchanged;
the cardinality cost of N members x full resolution stays user-chosen.
Out of scope (deferred): the family-comparison view (overlay/small-multiples),
decimation/LOD, a binary trace container.
Verified locally: cargo build --workspace, cargo test --workspace (all green,
incl. 3 new family persistence tests + a sweep determinism test + 2
sweep_member_key unit tests), cargo clippy --workspace --all-targets
-D warnings (clean). The pre-existing family determinism and single-run
trace tests pass unchanged (no-trace path is byte-identical).
closes #104
|
||
|
Brummel
|
06fdafa925 |
feat(trace-charts): keyboard chart control — pan / zoom-toward-cursor / reset / toggle
Keyboard control for the served chart, deliberately the discrete counterpart to the
glitchy hand-rolled pointer drag that was just removed: each key is one clean
setScale("x", …), deterministic, with none of uPlot's pointer-pan jank.
- arrows pan (15% of the visible span), +/- zoom, 0/Home reset, x toggles the spine.
- zoom anchors on the CURSOR: the x-value under the mouse keeps its relative position,
so the point under the pointer stays put while zooming (centre when the mouse is off
the chart). The mouse position is tracked per chart via posToVal on mousemove.
- keys drive every chart instance together, so panels stay x-aligned.
- uPlot's default mouse (drag = box-zoom, double-click = reset) is retained — you have
both.
The load-bearing math is the pure keyXRange(key, lo, hi, anchor, ext) helper (the rest
is browser-only DOM wiring in mount); it is node-guarded by chart_viewer_keys.mjs, which
pins the zoom-toward-cursor invariant (anchor keeps its relative position) with an
off-centre anchor so a centre-zoom regression fails loudly. All five chart guards green.
refs #103
|
||
|
Brummel
|
08a272c73d |
fix(trace-charts): drop glitchy custom wheel/pan, restore uPlot default mouse control
The hand-rolled wheel-zoom + drag-pan (wheelZoomPlugin) was hakelig/glitchy — uPlot is not built to be driven that way. Remove the plugin and the cursor.drag override so uPlot's own defaults govern the mouse: drag = box-zoom on x, double-click = reset to full range, with the legend value-readout and crosshair unchanged. Everything else from the Tier-1 pass stays: the ordinal (gap-collapsed) x-axis default + continuous toggle, the ordinal-label honesty (ticks/legend report the real xs[i]), and panel cursor-sync (cursor.sync key retained; only the drag override and the plugin are dropped). The x-mode guard is flipped to pin the revert — overlay carries no custom plugin and no cursor.drag override, panels carry no plugin — and stays green alongside the other three chart guards. refs #103 |
||
|
Brummel
|
45cc1c44c9 |
feat(trace-charts): ordinal x-axis default + zoom/pan/cursor-sync (viewer Tier-1)
The served trace chart's x-axis no longer plots against raw timestamps by default — non-trading gaps (nights, weekends) left horizontal blanks that dominated the view. The x spine is now ordinal by default: every recorded slot sits at an even position, so gaps collapse and the signal fills the width. A header toggle flips to continuous (real-time-proportional) spacing and back; default ordinal honours the requested "x-axis should not be continuous", with continuous as the one-click option. Comfort, all on the read/render side (the engine and the Rust serve contract are untouched — window.AURA_TRACES still carries the real xs, the ordinal indices are derived client-side): - ordinal mode keeps the axis honest: ticks and the legend value-readout map a collapsed index back to the real xs[i], never the index itself. - wheel-zoom (x, cursor-anchored), drag-pan, double-click-reset on the plot (uPlot's default drag-box-zoom is disabled so drag pans). - panels share a cursor.sync key, so the crosshair lines up across stacked panels. Confined to chart-viewer.js (buildCharts gains an xMode param + a wheelZoomPlugin factory; mount re-renders on the toggle) and render.rs (CHART_HEAD toggle button + a chart-only style). The two twin node guards (build/gaps) re-point to the ordinal default in lockstep; a new x-mode guard pins ordinal-default / continuous / panel-sync / plugin wiring, and a label-honesty guard (gappy-traces fixture, xs far from the index spine) pins that ordinal ticks report the real timestamp. Verified: cargo test --workspace green (incl. the four chart guards + render inline tests), clippy --all-targets -D warnings clean, e2e (aura run --trace + aura chart) emits a self-contained page defaulting to ordinal with the toggle. Spec docs/specs/0057, plan docs/plans/0057 (boss-signed; grounding-check PASS). Derived-fork decision log in the issue comments. closes #103 |
||
|
Brummel
|
15ee6d5f4f |
feat(trace-charts): serve persisted traces as a web chart (iter 2)
Iteration 2 (serve half) of the visual face — spec 0056 / plan 0056:
read a persisted run's traces and serve them as a self-contained uPlot
HTML chart. Completes the goal "tap data from a graph, serve it as a
chart in the browser".
- chart-viewer.js (first-party): a pure buildCharts(traces, mode) that
maps the injected window.AURA_TRACES to uPlot configs (no DOM) — overlay
= one plot, every series on its own y-scale over the shared xs; panels =
one plot per series, all on the same timestamp x. A browser-only mount()
instantiates the vendored global uPlot. Guarded by a node .mjs DOM test
(+ a sparse/gaps variant) shelling out exactly like viewer_dot.rs.
- render_chart_html + ChartMode/ChartData/Series (aura-cli/render.rs):
mirrors render_html — self-contained page, uPlot (1.6.32, vendored
|
||
|
Brummel
|
d534f106da |
chore(aura-cli): vendor uPlot 1.6.32 for the trace-chart viewer
Vendors uPlot.iife.min.js (the IIFE bundle exposing the global `uPlot`) + uPlot.min.css under assets/, pinned at uplot@1.6.32 via unpkg, mirroring the existing Graphviz-WASM/pan-zoom vendoring (checked in verbatim so include_str! + the build stay hermetic/offline, C1/C8). refresh-assets.sh gains the matching pinned-fetch lines. Consumed by iteration 2's render_chart_html (serve half of spec 0056). refs #101 |
||
|
Brummel
|
8bb5256041 |
feat(trace-charts): persist recorder taps to disk (iter 1)
Iteration 1 (persist half) of the visual face — spec 0056 / plan 0055:
tap data from a graph and save it to disk so it can later be charted
(the serve/chart half is iteration 2).
- ColumnarTrace (aura-engine): a drained tap <-> columnar SoA form
({tap, kinds, ts, columns}); values coerced to f64 for plotting with
the base type preserved in `kinds` (C7); to_rows rebuilds uniformly-f64
rows for the serve-side join. Pure (C1), beside join_on_ts/summarize.
- TraceStore (aura-registry): a per-run directory under
runs/traces/<name>/ — one <tap>.json per tap + an index.json (manifest
+ tap order), beside the families.jsonl sibling store. Missing run =
NotFound (no panic); corrupt file = Parse{file}.
- CLI: a shared persist_traces helper threaded into run_sample /
run_macd / run_sample_real via an orthogonal `trace: Option<&str>`;
new arms `aura run [--macd] --trace <name>` and `--trace <name>` on
`run --real`. The default (no --trace) run is byte-for-byte unchanged.
Verified by the orchestrator: `cargo test --workspace` green (incl. the
4 columnar_trace, 3 trace_store, 4 cli_run e2e, and the 4-tuple
parse_real_args tests); `cargo clippy --workspace --all-targets -D
warnings` exit 0. Implementer deviations folded in, all
compiler-prescribed: #[derive(Debug)] on RunTraces (the RED tests
panic-format a Result over it), #[allow(clippy::type_complexity)] on
the parse_real_args 4-tuple (mirrors the crate's sample_harness idiom).
Default-run byte-identity stays pinned by run_prints_json_and_exits_zero.
refs #101
|
||
|
Brummel
|
3dac2c0c99 |
refactor(aura-cli): retire the dead-end runs list/rank CLI surface
Since cycle 0045 the flat runs.jsonl store has had no producer (sweep/mc/ walkforward persist to the family store; `aura run` does not persist), so `aura runs list` / `runs rank` read a store nothing populates — a live dead-end surface, the honesty gap the Runway milestone closes. Resolve the C18 deferred fork (INDEX.md) as (b) RETIRE: drop the two CLI commands + their dispatch arms, usage fragments, and the now-unused load_runs_or_exit. Families (sweep/mc/walkforward, C21) subsume standalone over-time comparison. The aura-registry flat lib API (append/load/rank_by/optimize) is RETAINED — optimize backs walk-forward's in-sample step, rank_by backs `runs family ... rank`. `runs families` / `runs family` are untouched. Decision recorded on #73 (user-directed (b), 2026-06-18). Cosmetic sub-item (json! family-wrapper -> to_json() stdout key-order unification) deferred to a follow-up: json! routes the embedded report through serde_json::Value, which re-alphabetizes keys, so it needs manual JSON construction, not a trivial swap. RED-first: the retire test asserted `runs list`/`rank` now exit 2 (was exit 0). Verified: cargo build --workspace, clippy --all-targets -D warnings, and cargo test --workspace all green. closes #73 |
||
|
Brummel
|
03f80f0a95 |
docs(aura-registry): note Registry::open owns two directory-co-located stores
Registry::append_family / load_family_members write+read a fixed-name families.jsonl sibling via self.path.with_file_name(...), so isolation between registries is per-directory, not per-filename — two Registry::open calls with different runs filenames in one directory share a single family store. The sibling was documented on append_family but not on Registry::open, where a caller binds the path; this records the per-directory-isolation consequence at the bind point. Pure rustdoc ratify of existing (cycle 0045) behaviour. closes #82 |
||
|
Brummel
|
26bec6d64a |
feat: per-instrument pip metadata channel for the sim-optimal broker
The data-server symbol stream carries only price bars, no instrument metadata.
So SimBroker was constructed with one global pip_size literal (0.0001, correct
only for 5-decimal FX), and `aura run --real <symbol>` emitted wrong-unit pip
equity for any non-FX instrument (AAPL.US, BTCUSD off by orders of magnitude).
Add the missing channel to *specify* per-instrument pip:
- aura-ingest: `InstrumentSpec { pip_size }` + `instrument_spec(symbol)` — a
typed, Rust-authored vetted lookup (NOT Aura.toml; the later Aura.toml schema
can subsume it). Seeded GER40/FRA40 = 1.0 (index points), EURUSD/GBPUSD/USDCAD
= 0.0001 (5-dp FX majors); None for an un-specced symbol.
- aura-cli: `sample_harness` and `sim_optimal_manifest` gain a `pip_size`
parameter; `run_sample_real` looks the pip up by symbol BEFORE any data access
and refuses (exit 2) an un-specced instrument rather than guessing — honest by
construction. The looked-up pip reaches both the broker divisor and the
manifest broker label (`format!`). Every synthetic caller passes the unchanged
0.0001, now named `SYNTHETIC_PIP_SIZE`. SimBroker (aura-std) is untouched.
Scope: narrowed to the CLI real path — the actual bug. The runnable GER40
examples already bake the correct 1.0 and are NOT buggy; centralizing them
through the lookup is deferred to #98.
Decisions (user-directed, recorded on #22): metadata channel = typed
InstrumentSpec at the source edge; un-specced real symbol refuses; seed =
GER40/FRA40 + FX majors; example refactor deferred. The spec went through the
grounding-check gate; the auto-sign panel surfaced the refusal-behaviour and
scope as genuine design decisions, escalated and resolved by the user.
Tests: instrument_spec unit; manifest pip rendering (1.0 -> "1", 0.0001
unchanged); non-gated CLI refusal (exit 2, no stdout leak, vetted symbol clears
the lookup); gated GER40 binary-boundary honesty (pip_size=1 in the emitted
manifest, deterministic). The pre-existing run_sample_real determinism test was
retargeted AAPL.US -> GER40 (AAPL.US is un-specced and would now refuse at the
lookup); the AAPL.US bounded-window streaming property still lives in
aura-ingest/tests/streaming_seam.rs (literal source, no spec lookup).
Verified: cargo build --workspace, clippy --all-targets -D warnings, and
cargo test --workspace all green; the gated GER40 path ran with local data.
closes #22
|
||
|
Brummel
|
35c5adc6f3 |
refactor(aura-ingest): fuse drain_trace taps via join_on_ts; doc the cadence rule
drain_trace now joins its four recording-sink taps through aura-engine's join_on_ts instead of a hand-rolled per-tap HashMap, resolving the #93 zip-by-index panic at its source: the four taps fire at different cardinalities (breakout one bar short via cold Delay(1); bars_since_open filtered by Session), so a positional zip misaligns or panics. The BarTrace mapping stays a thin layer with its caller-side defaults unchanged (held->0.0, bars_since_open->-1, breakout->false) — the engine reports presence, the consumer interprets absence. Behaviour is byte-preserved: the gated GER40 tests (ger40_breakout_real, ger40_breakout_blueprint, open_ohlc_seam) stay green against the real archive. The Recorder doc gains a prose note on the multi-tap cadence rule (join on ts, never by index), pointing at join_on_ts by name rather than rustdoc link since aura-std does not depend on aura-engine. Verified: cargo build/test/clippy --workspace --all-targets clean; the _by_ts hand-rolled join is gone (acceptance grep empty). closes #93 |
||
|
Brummel
|
74324d178f |
feat(aura-engine): join_on_ts — timestamp join for multi-tap sink traces
Ship a generic, spine-anchored join_on_ts helper + JoinedRow in the report layer (sibling of summarize/f64_field), re-exported from lib.rs. It fuses N recorded (Timestamp, Vec<Scalar>) tap streams on the recorded timestamp: exactly one JoinedRow per spine entry, each side looked up by ts as Some(row)/None where it did not fire. Option-per-side semantics keep the engine honest — it reports presence; the consumer interprets absence (the 0.0/-1/false defaults are consumer truths, not engine ones). C8/C18 (post-run reduction over recorded sink output), C3 (no in-graph join), C1 (one row per ts, documented precondition). Two unit tests pin the contract: cardinality-misalignment alignment (the #93 shape — a spine bar a side tap skipped resolves to None, not a zip-misalign) and the spine-anchored drop of a side row at a non-spine ts. RED accepted on the grounding-check record (both symbols were absent pre-cycle, so the tests are RED-by-construction) and verified green here. drain_trace's migration onto this helper + the Recorder doc note follow. refs #93 |
||
|
Brummel
|
8c9a1b4630 |
feat(aura-ingest): consolidate the real-data source-open seam
A real OHLC source now builds from aura-ingest alone, in the engine-native epoch-ns Timestamp currency, with the one ms<->ns crossing owned by the seam. Library surface (src/lib.rs), all additive — the ms-based `open` and `unix_ms_to_epoch_ns` are untouched, so every existing call site is preserved byte-for-byte: - epoch_ns_to_unix_ms (private): the seam-owned inverse; consumers never convert (#80). - M1FieldSource::open_window: the Timestamp-window mirror of `open`, mapping each bound through the private inverse and delegating to `open`. - open_ohlc: the canonical OHLC bundle opener — four M1FieldSources in the fixed open/high/low/close C4 merge order, sharing one Arc<DataServer> (one cache, C12); the order lives in one vetted place (#92). - default_data_server + `pub use data_server::{DataServer, DEFAULT_DATA_PATH}`: a real-data source builds without naming the external data_server crate (#81). Consumer migration: the shared breakout_real.rs helpers and every GER40 example/test move to the Timestamp-native surface — open_ohlc_sources removed, utc_month_window_ms -> utc_month_window (Timestamp), report_from_trace retyped, the per-consumer ns_to_ms hand-divides deleted, the direct data_server imports replaced by the re-exports. ger40_breakout_compare.rs (which opened OHLC by its own hand-loop, reached by neither retyped helper) is migrated by inspection. Return type is Vec, not [_; 4]: Harness::run consumes Vec<Box<dyn Source>>, so it feeds straight in; order-safety comes from the single helper either way. Tests: a hermetic round-trip pinning the inverse; a gated behaviour-preservation test (open_ohlc Timestamp path == ms-path open, bit-identical recorded series); a hermetic absent-archive fixture pinning the file-level None contract and the #81 re-export firewall. Verified: cargo build --workspace --all-targets clean; cargo test --workspace green (the gated GER40 tests ran against the local archive — open_ohlc_seam proves byte-identity on real data, not a skip); clippy --all-targets -D warnings clean. Acceptance greps pass: open_ohlc_sources gone, no consumer-side ns_to_ms, no data_server import in any migrated GER40 consumer, epoch_ns_to_unix_ms private. closes #80, #81, #92 |
||
|
Brummel
|
d858caf67b |
chore: scrub dangling references to deleted specs/plans from sources
docs/specs and docs/plans were retired (prior commit); the source/test
comments that cited them ("spec 0050 §4.1", "spec §Testing N", "per spec",
"the spec's ...") now point at nothing. Strip every such pointer while
preserving the technical substance, the design-ledger contract refs
(C1/C11/C20/C34/C12.1/...), and the Gitea issue refs (#41).
Comment/doc edits only across 20 files — no logic change; full workspace
suite green, clippy clean.
|