Five compile-gated tasks for spec 0060: (1) additive DataSource/DataChoice
provider + ns roller constants, (2) pip-parametrize the two blueprints + thread
all callers, (3) thread &DataSource into the five family builders + run_sweep/
run_walkforward (dispatch passes Synthetic), (4) --real parser grammar +
build-or-refuse dispatch + USAGE, (5) gated real-path integration tests + full
suite + clippy. Each task threads its signature change with all callers so
`cargo build -p aura-cli --all-targets` is green per task; the byte-unchanged
synthetic path is the regression guard.
refs #106
Opt-in `--real <SYMBOL> [--from <ms>] [--to <ms>]` for `aura sweep` and
`aura walkforward`, mirroring `aura run --real`: each family member streams real
M1 close bars from the local data-server archive instead of the synthetic stream.
MC is excluded — its seed varies a synthetic price-walk realization, undefined
over real data (one realization -> identical members; a real MC needs a
bootstrap-resampling axis, its own cycle).
Engine, ingest, and registry untouched: a CLI-side `DataSource` provider replaces
the hardcoded `VecSource`. `M1FieldSource` already impls `Source`; `WindowRoller`
already rolls over epoch-unit timestamps.
Boss-signed: Step-5 grounding-check PASS — all 8 load-bearing current-behaviour
assumptions ratified by named, currently-green tests.
refs #106
Cycle-close tidy for the generic portable sweep member-key + bool-param demo
strategy (spec/plan 0059, reference issue #105, implementations 27f850d + 0b73a75).
Architect drift review (range 1d6fafb..HEAD): clean — no drift, no debt.
- C1 (determinism): varying_axes + member_key are pure functions of the grid
point, no schedule state; the engine carries no trace state (consistent with
the #104 design + the C22/#101 ledger note).
- C7/C8/C9: LongOnly is a clean one-in/one-out f64 node, None until warmed (C8
filter), the first bool *param* through the already-ratified Cell::bool()
path; momentum is an ordinary composite of existing nodes; brokers stay
downstream (C10). C18/C21/C22 upheld.
- The engine accessor exposes the binder's OWN axis-cardinality knowledge (it
holds the value-lists), not a presentation concern — formatting stays in the
CLI. The hand-rolled fnv1a64 mirrors the ledger-blessed SplitMix64 precedent
(harness.rs): tiny, dependency-free, bit-stable-on-purpose (std DefaultHasher
is unstable across releases; on-disk-name stability is the per-case C16
reason). LongOnly is a genuine universal aura-std block (long-only is
asset-agnostic). The ledger amendment matches the code.
Regression: the project declares no dedicated regression command; the gate is
the architect review plus the full suite, verified green at 0b73a75 (cargo test
--workspace, clippy --all-targets -D warnings).
Spec docs/specs/0059 and plan docs/plans/0059 retired (git rm) per the ephemeral
active-cycle artefact lifecycle; the durable rationale (the portable member-key
convention) lives in the ledger amendment (docs/design/INDEX.md C22/#101 note)
landed in 0b73a75.
closes#105
Proves the #105 generic portable member key end-to-end: a demo strategy whose
params have nothing in common with the SMA-cross demo, swept by the same
machinery, with a bool axis surfacing — conformant — in the member dir names.
- momentum strategy: price -> Ema(length:i64) -> Sub(price-ema) ->
Exposure(scale:f64) -> LongOnly(enabled:bool) -> SimBroker. Three swept params
of three kinds incl. a bool; the exposure sink taps the post-gate exposure so
the bool's effect is in the trace.
- aura sweep gains --strategy <sma|momentum> via a pure, unit-tested
parse_sweep_args (mirroring parse_real_args); default = SMA-cross, so the
existing bare / --name / --trace forms are unchanged.
- `aura sweep --strategy momentum --trace mom` persists 8 member dirs named e.g.
ema.length-5_exposure.scale-0.5_longonly.enabled-true — the bool in the dir
name, every name matching [A-Za-z0-9._-], each chartable.
- ledger (docs/design/INDEX.md): the C22/#101 member-key note's sweep clause
amended to the generalised portable form (MC seed{N} / walk-forward oos{ns}
unchanged).
Self-verified: cargo build --workspace, cargo test --workspace (all green incl.
momentum_param_space_is_ema_exposure_longonly, the 8-point determinism test,
the parse_sweep_args grammar test, and the momentum_sweep_trace integration test
asserting 8 portable bool-bearing dirs + a chartable member), cargo clippy
--workspace --all-targets -D warnings.
refs #105
Generalises the #105 foot-gun: the family-member trace key was derived from two
hardcoded axis names (f{fast}s{slow}), collision-free only over the built-in SMA
grid. It is now derived from the axes that actually VARY
(SweepBinder::varying_axes) and rendered as a filesystem-conformant directory
component:
- charset restricted to [A-Za-z0-9._-] (valid on Linux/Windows/macOS, also
URL-path- and cloud-sync-safe); any other byte sanitised;
- case-less values (i64/timestamp decimal, bool true/false, f64 via Display, no
scientific notation) so two members of one family never differ only by letter
case -> no silent overwrite on case-insensitive filesystems (NTFS/APFS);
- length-capped (MAX_KEY=200) with a version-stable FNV-1a fallback for the
unbounded-grid edge.
Pinned singleton axes are omitted; the SMA sweep's member dirs change from f2s4
to signals.trend.fast.length-2_signals.trend.slow.length-4 (its --trace
integration test + the stale prose updated accordingly).
Adds the first aura-std node with a bool *param*, LongOnly (a long-only exposure
gate: enabled=true clamps short exposure to >=0, false passes through) — the
block the momentum demo strategy (next commit) sweeps to prove the key generic
over a bool axis. The engine change is one additive pure read accessor; no
existing signature changes, trace state stays out of the engine.
Self-verified: cargo build --workspace, cargo test --workspace (all green incl.
the LongOnly node tests, the member_key worked-examples corpus, the varying_axes
accessor test, and the updated SMA sweep-trace integration test), cargo clippy
--workspace --all-targets -D warnings.
refs #105
Generalises the family-member trace key (#105 foot-gun): derive it from the
axes that actually vary (not two hardcoded axis names), and render it as a
filesystem-conformant directory name on every major OS — charset [A-Za-z0-9._-],
case-less values (no case-insensitive-FS collision), length-bounded with an
FNV-1a fallback for the unbounded-grid edge.
Adds the first bool *param* node (LongOnly) and a momentum demo strategy
(Ema -> Sub -> Exposure -> LongOnly -> SimBroker) swept via a new
`aura sweep --strategy <sma|momentum>` selector — the empirical proof the
generic key works for a strategy whose params (incl. a bool) differ entirely
from the SMA-cross demo.
Boss-signed on the Step-5 grounding-check PASS (an independent fresh-context
agent): all 9 load-bearing current-behaviour assumptions ratified by green
tests; the end-to-end bool-param sweep path is correctly scoped as new
behaviour proven by new tests, not assumed. Workspace type-checks clean.
refs #105
Cycle-close tidy for family-member trace persistence (spec/plan 0058,
reference issue #104, implementation d3cb5f8).
Architect drift review (range 06fdafa..d3cb5f8): drift_found, low-grade
only — no contract violated. C1 (determinism), C7 (SoA), C8 (sinks),
C18/C21 (registry/families) all upheld; the opt-in --trace path is
genuinely byte-neutral. Two items, both resolved here:
- [high] Ledger gap: the C22/#101 disk-trace note recorded only the
single-run runs/traces/<name>/ layout. This cycle establishes a new,
load-bearing on-disk convention (per-member nested standalone run-dirs
runs/traces/<name>/<member_key>/) that the deferred family-comparison
view will build on. FIXED: amended the C22/#101 realization note in
docs/design/INDEX.md to record the layout, the content-derived
deterministic member keys, and the parallel-safe lock-free write.
- [medium] sweep_member_key hard-codes two grid axes; collision-free only
over today's built-in grid, a latent foot-gun if the grid varies an
unlisted axis. Not a current defect (one grid, covered, fail-loud on a
missing listed axis). CARRY-ON: filed #105 (idea) for generalised key
derivation when the grid generalises.
Regression: the project declares no dedicated regression command; the
gate is the architect review plus the full suite, verified green at
d3cb5f8 (cargo test --workspace, clippy --all-targets -D warnings).
Spec docs/specs/0058 and plan docs/plans/0058 retired (git rm) per the
ephemeral active-cycle artefact lifecycle; durable rationale lifted to
the ledger amendment above.
refs #104
Family runs (sweep, Monte-Carlo, walk-forward) ran a full harness per
member and drained its equity+exposure recorders, but immediately folded
the streams to metrics and discarded them — so the C21 family-comparison
view had no per-member data to chart. A new opt-in `--trace <name>` on
`aura sweep|mc|walkforward` now persists each member's streams to disk,
mirroring the single-run `aura run --trace` path. Any single member is
chartable today via `aura chart <name>/<member_key>`; the comparison view
itself is a later cycle.
Design:
- Persist INSIDE each per-member closure (aura-cli), reusing persist_traces
verbatim — the engine and the family types are untouched. The closures
already held the drained rows; they now bind them once, build the
manifest, persist when --trace is set, then fold as before.
- Content-derived, deterministic member keys (MC seed{N}, sweep f{fast}s{slow},
walk-forward oos{ns}). The engine HOFs are Fn + Sync — members run in
parallel, so a runtime ordinal counter would be non-deterministic (a C1
break); keys derive from what each closure receives deterministically.
sweep_member_key panics (naming the axis) if the grid lacks a key axis,
rather than silently collapsing two points to a shared dir.
- Each member is a standalone run-dir at runs/traces/<name>/<member_key>/
(persist_traces with a slash-name; TraceStore nests via create_dir_all),
so the existing single-run viewer charts a member with no view-side code.
- Opt-in: without --trace, stdout and the run registry are byte-unchanged;
the cardinality cost of N members x full resolution stays user-chosen.
Out of scope (deferred): the family-comparison view (overlay/small-multiples),
decimation/LOD, a binary trace container.
Verified locally: cargo build --workspace, cargo test --workspace (all green,
incl. 3 new family persistence tests + a sweep determinism test + 2
sweep_member_key unit tests), cargo clippy --workspace --all-targets
-D warnings (clean). The pre-existing family determinism and single-run
trace tests pass unchanged (no-trace path is byte-identical).
closes#104
Three tasks (MC, sweep, walk-forward) wiring per-member trace persistence
into the family closures + a full-suite/lint gate. Each task threads its
builder signature and every caller together so the workspace compiles
green per task.
refs #104
Persist each family member's equity/exposure streams to disk (opt-in
--trace on sweep|mc|walkforward), mirroring the single-run trace path, so
the C21 family-comparison view has per-member data and any single member
is chartable today. Write side only; the comparison view, decimation,
and a binary trace container are deferred.
Autonomously signed under /boss on the Step-5 grounding-check PASS.
refs #104
Keyboard control for the served chart, deliberately the discrete counterpart to the
glitchy hand-rolled pointer drag that was just removed: each key is one clean
setScale("x", …), deterministic, with none of uPlot's pointer-pan jank.
- arrows pan (15% of the visible span), +/- zoom, 0/Home reset, x toggles the spine.
- zoom anchors on the CURSOR: the x-value under the mouse keeps its relative position,
so the point under the pointer stays put while zooming (centre when the mouse is off
the chart). The mouse position is tracked per chart via posToVal on mousemove.
- keys drive every chart instance together, so panels stay x-aligned.
- uPlot's default mouse (drag = box-zoom, double-click = reset) is retained — you have
both.
The load-bearing math is the pure keyXRange(key, lo, hi, anchor, ext) helper (the rest
is browser-only DOM wiring in mount); it is node-guarded by chart_viewer_keys.mjs, which
pins the zoom-toward-cursor invariant (anchor keeps its relative position) with an
off-centre anchor so a centre-zoom regression fails loudly. All five chart guards green.
refs #103
The hand-rolled wheel-zoom + drag-pan (wheelZoomPlugin) was hakelig/glitchy —
uPlot is not built to be driven that way. Remove the plugin and the cursor.drag
override so uPlot's own defaults govern the mouse: drag = box-zoom on x,
double-click = reset to full range, with the legend value-readout and crosshair
unchanged.
Everything else from the Tier-1 pass stays: the ordinal (gap-collapsed) x-axis
default + continuous toggle, the ordinal-label honesty (ticks/legend report the
real xs[i]), and panel cursor-sync (cursor.sync key retained; only the drag
override and the plugin are dropped). The x-mode guard is flipped to pin the
revert — overlay carries no custom plugin and no cursor.drag override, panels
carry no plugin — and stays green alongside the other three chart guards.
refs #103
Architect drift review over 5bff7e0..HEAD: clean, no drift, no debt. C14/C7
firewall intact (main.rs byte-unchanged → build_chart_data still returns
ChartData{xs,series}, the injected window.AURA_TRACES payload shape untouched;
the ordinal index spine is derived client-side). C22 holds — the
ordinal/continuous toggle + wheel-zoom/drag-pan/dblclick-reset are view
affordances over a recorded trace (read-only setScale on the x range), not a
scene editor. The #102 manifest-header gap is neither closed nor regressed,
still tracked.
Regression gate: cargo test --workspace green (incl. the four chart guards +
render inline tests), clippy --all-targets -D warnings clean; aura has no
separate regression script, so the suite + architect are the gates. cycle
clean.
Spec/plan 0057 retired per the ephemeral-artifact lifecycle (durable record:
the feature commit 45cc1c4, the #103 decision-log comments, and git history).
refs #103
The served trace chart's x-axis no longer plots against raw timestamps by
default — non-trading gaps (nights, weekends) left horizontal blanks that
dominated the view. The x spine is now ordinal by default: every recorded slot
sits at an even position, so gaps collapse and the signal fills the width. A
header toggle flips to continuous (real-time-proportional) spacing and back;
default ordinal honours the requested "x-axis should not be continuous", with
continuous as the one-click option.
Comfort, all on the read/render side (the engine and the Rust serve contract
are untouched — window.AURA_TRACES still carries the real xs, the ordinal
indices are derived client-side):
- ordinal mode keeps the axis honest: ticks and the legend value-readout map a
collapsed index back to the real xs[i], never the index itself.
- wheel-zoom (x, cursor-anchored), drag-pan, double-click-reset on the plot
(uPlot's default drag-box-zoom is disabled so drag pans).
- panels share a cursor.sync key, so the crosshair lines up across stacked
panels.
Confined to chart-viewer.js (buildCharts gains an xMode param + a wheelZoomPlugin
factory; mount re-renders on the toggle) and render.rs (CHART_HEAD toggle button
+ a chart-only style). The two twin node guards (build/gaps) re-point to the
ordinal default in lockstep; a new x-mode guard pins ordinal-default / continuous
/ panel-sync / plugin wiring, and a label-honesty guard (gappy-traces fixture,
xs far from the index spine) pins that ordinal ticks report the real timestamp.
Verified: cargo test --workspace green (incl. the four chart guards + render
inline tests), clippy --all-targets -D warnings clean, e2e (aura run --trace +
aura chart) emits a self-contained page defaulting to ordinal with the toggle.
Spec docs/specs/0057, plan docs/plans/0057 (boss-signed; grounding-check PASS).
Derived-fork decision log in the issue comments.
closes#103
Tier-1 comfort pass for the served trace chart (#103): an ordinal x-axis
default (collapses non-trading gaps) with a one-click continuous toggle, plus
wheel-zoom / drag-pan / dblclick-reset, a value-readout legend, and panel
cursor-sync. Confined to the read/render side (chart-viewer.js + render.rs
CHART_HEAD); the Rust serve contract is unchanged.
Boss-signed: grounding-check PASS (five current-behaviour assumptions ratified
by green node guards + render.rs inline tests). Derived-fork decision log in
the #103 comments.
refs #103
Cycle-close audit for the trace-charts visual face (#101, shipped across
8bb5256 persist + 15ee6d5 serve). Architect drift review (13941a2..HEAD)
+ the regression gate (cargo build/test/clippy --workspace, all exit 0,
orchestrator-verified).
What holds (carry-on): C14 clean — aura-engine/aura-core carry zero
UI/HTML/uPlot knowledge; ColumnarTrace does struct->encoding only, file
I/O in aura-registry, rendering in aura-cli. C1/C3 clean — persistence
and serve-time alignment are post-run pure reductions (join_on_ts at
serve time only, never in-graph). C7 clean — on-disk f64 columns +
kinds-tagged base types, the documented coercion honest, no 5th scalar.
Resolutions:
- [fix] Added the C22 realization note now that the web-from-disk seam
shipped (the amendment 3b56efb had deferred it to "when the seam
ships").
- [ratify] The served page injects the chart SERIES but drops the run
manifest (no run-context header) — a deliberate, defensible naive-viewer
scope call (the manifest is on disk in index.json; a header matters most
for the deferred families/comparison work, C21). The deviation from spec
0056 §1/§4 landed unflagged in 15ee6d5; it is now flagged here and
tracked as a follow-up. refs #102
- [tidy] Retired this cycle's ephemera (spec 0056, plans 0055/0056) and
the stale debris from the prior Runway close (spec 0055-fieldtest) per
the ephemeral-artifact convention; the durable record is the ledger +
git history.
Cycle drift-clean (modulo the ratified, tracked manifest-header gap).
refs #101
Iteration 2 (serve half) of the visual face — spec 0056 / plan 0056:
read a persisted run's traces and serve them as a self-contained uPlot
HTML chart. Completes the goal "tap data from a graph, serve it as a
chart in the browser".
- chart-viewer.js (first-party): a pure buildCharts(traces, mode) that
maps the injected window.AURA_TRACES to uPlot configs (no DOM) — overlay
= one plot, every series on its own y-scale over the shared xs; panels =
one plot per series, all on the same timestamp x. A browser-only mount()
instantiates the vendored global uPlot. Guarded by a node .mjs DOM test
(+ a sparse/gaps variant) shelling out exactly like viewer_dot.rs.
- render_chart_html + ChartMode/ChartData/Series (aura-cli/render.rs):
mirrors render_html — self-contained page, uPlot (1.6.32, vendored
d534f10) + chart-viewer.js inlined via include_str!, data injected as
window.AURA_TRACES.
- emit_chart + build_chart_data (aura-cli/main.rs): the spec-§6 3-step
union-spine alignment — xs = sorted-dedup union of every tap's ts; a
synthetic empty-payload spine + ALL taps as symmetric join_on_ts sides
(no tap privileged, no point dropped); flatten each (tap,column) to a
Series of Option<f64>. New arms `aura chart <name> [--panels]`; a
missing run is stderr + exit 2 (TraceStoreError::NotFound), never a
panic. USAGE advertises the verb in lockstep.
Verified by the orchestrator: `cargo test --workspace` green (incl. the
node guards, render_chart_html, and the chart e2e); `cargo clippy
--workspace --all-targets -D warnings` exit 0; and a real end-to-end —
`aura run --trace demo && aura chart demo` emits a 57KB self-contained
page (doctype + inlined uPlot + AURA_TRACES carrying the equity/exposure
series), `--panels` reflects the panels mode, `aura chart nope` exits 2.
Implementer deviation folded in (compiler-prescribed): serde added to
aura-cli/Cargo.toml for the Series derive (Cargo.lock updated).
refs #101
Vendors uPlot.iife.min.js (the IIFE bundle exposing the global `uPlot`) + uPlot.min.css under assets/, pinned at uplot@1.6.32 via unpkg, mirroring the existing Graphviz-WASM/pan-zoom vendoring (checked in verbatim so include_str! + the build stay hermetic/offline, C1/C8). refresh-assets.sh gains the matching pinned-fetch lines. Consumed by iteration 2's render_chart_html (serve half of spec 0056).
refs #101
Iteration 1 (persist half) of the visual face — spec 0056 / plan 0055:
tap data from a graph and save it to disk so it can later be charted
(the serve/chart half is iteration 2).
- ColumnarTrace (aura-engine): a drained tap <-> columnar SoA form
({tap, kinds, ts, columns}); values coerced to f64 for plotting with
the base type preserved in `kinds` (C7); to_rows rebuilds uniformly-f64
rows for the serve-side join. Pure (C1), beside join_on_ts/summarize.
- TraceStore (aura-registry): a per-run directory under
runs/traces/<name>/ — one <tap>.json per tap + an index.json (manifest
+ tap order), beside the families.jsonl sibling store. Missing run =
NotFound (no panic); corrupt file = Parse{file}.
- CLI: a shared persist_traces helper threaded into run_sample /
run_macd / run_sample_real via an orthogonal `trace: Option<&str>`;
new arms `aura run [--macd] --trace <name>` and `--trace <name>` on
`run --real`. The default (no --trace) run is byte-for-byte unchanged.
Verified by the orchestrator: `cargo test --workspace` green (incl. the
4 columnar_trace, 3 trace_store, 4 cli_run e2e, and the 4-tuple
parse_real_args tests); `cargo clippy --workspace --all-targets -D
warnings` exit 0. Implementer deviations folded in, all
compiler-prescribed: #[derive(Debug)] on RunTraces (the RED tests
panic-format a Result over it), #[allow(clippy::type_complexity)] on
the parse_real_args 4-tuple (mirrors the crate's sample_harness idiom).
Default-run byte-identity stays pinned by run_prints_json_and_exits_zero.
refs #101
Iteration-1 (persist half) plan for spec 0056: ColumnarTrace (aura-engine), TraceStore (aura-registry), and the shared persist_traces helper threaded into all three run forms. RED-first; Task 3 folds all nine signature-change callers in before the build gate (per plan-recon's compile-gate warning). Serve/chart half deferred to iteration 2.
refs #101
First cut of aura's visual face (issue #101; the web-from-disk pivot
ratified into C14/C22 this session, commit 3b56efb): tap arbitrary
recorder streams from a run, persist them to disk as columnar (SoA)
JSON, and serve them as a self-contained uPlot HTML chart — feeds
overlaid on a shared timestamp axis, or timestamp-aligned in stacked
panels. `aura run [--macd] [--trace <name>]` and `aura run --real <SYM>
... [--trace <name>]` persist; `aura chart <name> [--panels]` serves.
Both modes share one synthetic-union-spine join_on_ts alignment.
Signed via the specify auto-sign gate: the user is away under an
explicit "work fully autonomously, document decisions" mandate, so no
manual signature was available. All objective gates green (precondition
clean, self-review clean, grounding-check PASS), and a unanimous 5-lens
adversarial spec-skeptic panel (criterion / grounding / scope-fork /
ambiguity / plan-readiness all SOUND).
The panel earned its keep — two self-correction rounds to unanimity:
- Round 0 caught (a) panels shipped NOT timestamp-aligned, contradicting
the user's verbatim "aligned nach Timestamps in Panels" — corrected to
conform to issue #101 Fork A; (b) the USAGE advertised `--macd --trace`
with no dispatch arm — corrected to an orthogonal `--trace` composing
with every run form via a shared persist helper.
- Round 1 caught the union-spine -> join_on_ts mapping under-specified
(which tap is the spine?) — corrected with an explicit 3-step
synthetic-empty-spine mechanism (no privileged tap, no points dropped),
plus a to_rows uniformly-f64 clarification.
The two delegated naive-viewer decisions (per-series y-scale, union
spine) are documented with provenance on issue #101.
refs #101
The visual face pivots from the ledger's egui-native, in-process
zero-copy direction to a web frontend served from disk-persisted
recorder traces: the engine writes recorded traces to disk, a browser
charts them. Settled in an in-context design discussion (provenance in
issue #101): raw per-tap trace persistence + serve-time join_on_ts
alignment + a static self-contained HTML chart (uPlot). The engine
stays headless (C14) — arguably more strictly, since a browser reading
a serialized file is further from the hot path than egui reaching into
the live SoA columns. Amends the C14 visual-face note and the open
"Playground & World UI surface" thread; the C22 realization note lands
when the seam ships.
refs #101
Milestone-close fieldtest for "Runway — real-data ergonomics & honesty
hardening". Four curated end-to-end scenarios derived top-down from the
milestone promise, run as a downstream consumer against real GER40/EURUSD M1
bars (public interface only; no crates/*/src read):
1. per-instrument pip honesty (#22) — GER40 index pip 1.0 AND EURUSD forex
pip 0.0001 both honest end-to-end; AAPL.US/FOOBAR un-vetted refused
exit 2, no stdout leak
2. real GER40 M1 open via open_ohlc from aura-ingest ALONE (#80/#81/#92) —
no data-server dep, 53173 real bars, C4 O/H/L/C order, ts-window inverse
3. multi-tap ts-join (#93) — two taps at different cadences, 27744 rows,
0 by-ts mismatches; warm-up None rows where a positional zip would corrupt
4. run-registry coherence (#73/#82) — list/rank retired exit 2, families/
family live, two-store Registry read-back + rustdoc
Findings: 4 working, 1 friction, 1 spec_gap.
- friction: Cell exposes only from_i64/from_f64/... while the side-by-side
Scalar is a tagged enum with public I64(..)/F64(..) variants. The natural
param-authoring entry point is Scalar (Cell is the hot-path buffer carrier),
so the wrong reach is a self-correcting compile snag — ergonomic only.
Filed as a follow-up idea.
- spec_gap: unknown family id exits 0 empty — ratified into the C18 note in
the preceding commit (treat-as-empty contract).
Gate verdict: GREEN. All seven axes mapped to a covering scenario and exercised
on real bars; both finding classifications and the one material coverage gap
were adversarially re-verified. That gap — #22's FX-vetted pip face, untested by
the original scenario set — was probed (EURUSD -> sim-optimal(pip_size=0.0001),
FX-scaled, not index-mis-scaled) and folded into scenario 1, so the pip promise
is demonstrated on both positive faces. Every scenario re-run independently on
HEAD before commit.
The Runway milestone fieldtest surfaced a spec_gap: `aura runs family <id>`
with an unknown-but-well-formed id exits 0 with empty output, and neither the
ledger nor the glossary constrained that case. The behaviour is already the
documented, tested CLI contract (treat-as-empty, mirroring Registry::load's
missing-store = Ok(empty)); this lifts that contract to the durable C18 note
so a consumer reasoning from the spec can derive it, and records that tightening
to a non-zero "no such family" exit (typo-safety) stays an available future UX
choice rather than a current contract.
Ratifies the spec_gap from docs/specs/0055-fieldtest-runway-real-data-seam.md.
The milestone-close architect flagged that the runs-retire note over-claimed
"retained because used": after dropping runs list/rank, Registry::append/load (the
flat-store half) have no in-tree caller — only rank_by/optimize keep live
consumers. Note now distinguishes the live half from the public-but-uncalled half
(a latent dead-code surface for a later sweep). Doc-only.
Since cycle 0045 the flat runs.jsonl store has had no producer (sweep/mc/
walkforward persist to the family store; `aura run` does not persist), so
`aura runs list` / `runs rank` read a store nothing populates — a live dead-end
surface, the honesty gap the Runway milestone closes.
Resolve the C18 deferred fork (INDEX.md) as (b) RETIRE: drop the two CLI commands
+ their dispatch arms, usage fragments, and the now-unused load_runs_or_exit.
Families (sweep/mc/walkforward, C21) subsume standalone over-time comparison. The
aura-registry flat lib API (append/load/rank_by/optimize) is RETAINED — optimize
backs walk-forward's in-sample step, rank_by backs `runs family ... rank`.
`runs families` / `runs family` are untouched.
Decision recorded on #73 (user-directed (b), 2026-06-18). Cosmetic sub-item
(json! family-wrapper -> to_json() stdout key-order unification) deferred to a
follow-up: json! routes the embedded report through serde_json::Value, which
re-alphabetizes keys, so it needs manual JSON construction, not a trivial swap.
RED-first: the retire test asserted `runs list`/`rank` now exit 2 (was exit 0).
Verified: cargo build --workspace, clippy --all-targets -D warnings, and
cargo test --workspace all green.
closes#73
Registry::append_family / load_family_members write+read a fixed-name
families.jsonl sibling via self.path.with_file_name(...), so isolation between
registries is per-directory, not per-filename — two Registry::open calls with
different runs filenames in one directory share a single family store. The
sibling was documented on append_family but not on Registry::open, where a caller
binds the path; this records the per-directory-isolation consequence at the bind
point. Pure rustdoc ratify of existing (cycle 0045) behaviour.
closes#82
Audit (architect drift review + cargo test --workspace) clean: code matches the
spec, the engine stayed domain-free (no instrument identity in Ctx / the hot
path), InstrumentSpec sits beside the hot path per C7/C15. No regression scripts
configured (no-op gate); the full suite is green.
Ratify: the architect flagged one ledger-realization gap — C10/C15 described
per-instrument pip as standing design intent but carried no realization note for
the now-shipped channel. Added dated realization notes to C10 (the per-instrument
pip channel + the refuse-don't-guess honesty rule) and C15 (instrument specs as
metadata, first realized by InstrumentSpec/instrument_spec).
Retire the ephemeral cycle artefacts (spec 0054, plan 0054) per the active-cycle
lifecycle. Deferred follow-up: #98 (route the runnable GER40 examples through the
lookup).
refs #22
The data-server symbol stream carries only price bars, no instrument metadata.
So SimBroker was constructed with one global pip_size literal (0.0001, correct
only for 5-decimal FX), and `aura run --real <symbol>` emitted wrong-unit pip
equity for any non-FX instrument (AAPL.US, BTCUSD off by orders of magnitude).
Add the missing channel to *specify* per-instrument pip:
- aura-ingest: `InstrumentSpec { pip_size }` + `instrument_spec(symbol)` — a
typed, Rust-authored vetted lookup (NOT Aura.toml; the later Aura.toml schema
can subsume it). Seeded GER40/FRA40 = 1.0 (index points), EURUSD/GBPUSD/USDCAD
= 0.0001 (5-dp FX majors); None for an un-specced symbol.
- aura-cli: `sample_harness` and `sim_optimal_manifest` gain a `pip_size`
parameter; `run_sample_real` looks the pip up by symbol BEFORE any data access
and refuses (exit 2) an un-specced instrument rather than guessing — honest by
construction. The looked-up pip reaches both the broker divisor and the
manifest broker label (`format!`). Every synthetic caller passes the unchanged
0.0001, now named `SYNTHETIC_PIP_SIZE`. SimBroker (aura-std) is untouched.
Scope: narrowed to the CLI real path — the actual bug. The runnable GER40
examples already bake the correct 1.0 and are NOT buggy; centralizing them
through the lookup is deferred to #98.
Decisions (user-directed, recorded on #22): metadata channel = typed
InstrumentSpec at the source edge; un-specced real symbol refuses; seed =
GER40/FRA40 + FX majors; example refactor deferred. The spec went through the
grounding-check gate; the auto-sign panel surfaced the refusal-behaviour and
scope as genuine design decisions, escalated and resolved by the user.
Tests: instrument_spec unit; manifest pip rendering (1.0 -> "1", 0.0001
unchanged); non-gated CLI refusal (exit 2, no stdout leak, vetted symbol clears
the lookup); gated GER40 binary-boundary honesty (pip_size=1 in the emitted
manifest, deterministic). The pre-existing run_sample_real determinism test was
retargeted AAPL.US -> GER40 (AAPL.US is un-specced and would now refuse at the
lookup); the AAPL.US bounded-window streaming property still lives in
aura-ingest/tests/streaming_seam.rs (literal source, no spec lookup).
Verified: cargo build --workspace, clippy --all-targets -D warnings, and
cargo test --workspace all green; the gated GER40 path ran with local data.
closes#22
Cycle 0053 (multi-tap trace join on timestamp, #93) closes drift-clean.
Audit (architect drift review over 066638e..HEAD against docs/design + the
CLAUDE.md domain invariants):
- C8/C18 honoured: join_on_ts is a post-run reduction over recorded
(Timestamp, Vec<Scalar>) sink streams, a true sibling of summarize/f64_field
in the report layer — correct architectural home.
- C3 not violated: the join runs post-run over already-recorded output, not
in-graph; no as-of join smuggled into the DAG. C1 ts-uniqueness is a
documented precondition, not a runtime check — consistent with how the
engine leaves wiring-time invariants to contract.
- Option-per-side keeps the engine honest (reports presence; the consumer owns
the 0.0/-1/false defaults). The hand-rolled *_by_ts HashMaps are fully gone;
the two RED unit tests pin the #93 cardinality-misalignment + spine-drop.
- No drift, no debt, no ledger entry needs updating. Recommendation: carry on.
Regression gate: no dedicated regression scripts configured (optional per the
conventions); build/test/clippy --workspace --all-targets is the gate and ran
green — the gated GER40 byte-identity tests executed against the local archive
(not skipped), proving drain_trace's trace is byte-preserved across the
refactor.
Per the artifact lifecycle (committed while the cycle is live, git rm at cycle
close), the ephemeral spec and plan are retired here; their durable rationale
lives in the implementation commits 74324d1 / 35c5adc and the git history.
drain_trace now joins its four recording-sink taps through aura-engine's
join_on_ts instead of a hand-rolled per-tap HashMap, resolving the #93
zip-by-index panic at its source: the four taps fire at different
cardinalities (breakout one bar short via cold Delay(1); bars_since_open
filtered by Session), so a positional zip misaligns or panics. The BarTrace
mapping stays a thin layer with its caller-side defaults unchanged
(held->0.0, bars_since_open->-1, breakout->false) — the engine reports
presence, the consumer interprets absence. Behaviour is byte-preserved: the
gated GER40 tests (ger40_breakout_real, ger40_breakout_blueprint,
open_ohlc_seam) stay green against the real archive.
The Recorder doc gains a prose note on the multi-tap cadence rule (join on
ts, never by index), pointing at join_on_ts by name rather than rustdoc link
since aura-std does not depend on aura-engine.
Verified: cargo build/test/clippy --workspace --all-targets clean; the
_by_ts hand-rolled join is gone (acceptance grep empty).
closes#93
Ship a generic, spine-anchored join_on_ts helper + JoinedRow in the report
layer (sibling of summarize/f64_field), re-exported from lib.rs. It fuses N
recorded (Timestamp, Vec<Scalar>) tap streams on the recorded timestamp:
exactly one JoinedRow per spine entry, each side looked up by ts as
Some(row)/None where it did not fire. Option-per-side semantics keep the
engine honest — it reports presence; the consumer interprets absence (the
0.0/-1/false defaults are consumer truths, not engine ones). C8/C18 (post-run
reduction over recorded sink output), C3 (no in-graph join), C1 (one row per
ts, documented precondition).
Two unit tests pin the contract: cardinality-misalignment alignment (the #93
shape — a spine bar a side tap skipped resolves to None, not a zip-misalign)
and the spine-anchored drop of a side row at a non-spine ts. RED accepted on
the grounding-check record (both symbols were absent pre-cycle, so the tests
are RED-by-construction) and verified green here.
drain_trace's migration onto this helper + the Recorder doc note follow.
refs #93
Bite-sized execution plan for spec 0053: Task 1 the join_on_ts helper +
JoinedRow + two RED unit tests in aura-engine report.rs (re-exported from
lib.rs); Task 2 the behaviour-preserving drain_trace refactor onto the helper;
Task 3 the Recorder prose cadence/join note. Final gate is workspace
build/test/clippy.
refs #93
Ratified design for a reusable, spine-anchored join_on_ts helper in
aura-engine that fuses multi-tap Recorder traces on the recorded timestamp
(Option-per-side: the engine reports presence, the consumer interprets
absence). Refactors drain_trace onto it and documents the cadence rule on
the Recorder API. Resolves the #93 zip-by-index panic.
refs #93
Cycle 0052 (real-data source-open seam, Runway-1) closes drift-clean.
Audit (architect drift review over aedaa5d..HEAD against docs/design + the
CLAUDE.md domain invariants):
- C3 (one ms<->ns crossing) strengthened — the inverse is seam-private and the
only consumer-side divide left; every consumer ns_to_ms / unix_ms_to_epoch_ns
wrap is gone.
- C4 (O/H/L/C merge tie-break order) now in exactly one vetted home (open_ohlc);
the hand-spelled four-field loops are gone.
- C12 (one shared Arc<DataServer>/cache) preserved; C7 surface additive.
- No drift, no debt, no ledger entry needs updating. Recommendation: carry on.
Regression gate: no dedicated regression scripts configured (optional per the
conventions); build/test/clippy --workspace is the gate and ran green — the gated
open_ohlc_seam behaviour-preservation test executed against the local archive
(not skipped), proving byte-identity vs the ms-path.
Per the artifact lifecycle (committed while the cycle is live, git rm at cycle
close), the ephemeral spec and plan are retired here; their durable rationale
lives in the implementation commit 8c9a1b4 and the git history.
A real OHLC source now builds from aura-ingest alone, in the engine-native
epoch-ns Timestamp currency, with the one ms<->ns crossing owned by the seam.
Library surface (src/lib.rs), all additive — the ms-based `open` and
`unix_ms_to_epoch_ns` are untouched, so every existing call site is preserved
byte-for-byte:
- epoch_ns_to_unix_ms (private): the seam-owned inverse; consumers never
convert (#80).
- M1FieldSource::open_window: the Timestamp-window mirror of `open`, mapping
each bound through the private inverse and delegating to `open`.
- open_ohlc: the canonical OHLC bundle opener — four M1FieldSources in the fixed
open/high/low/close C4 merge order, sharing one Arc<DataServer> (one cache,
C12); the order lives in one vetted place (#92).
- default_data_server + `pub use data_server::{DataServer, DEFAULT_DATA_PATH}`:
a real-data source builds without naming the external data_server crate (#81).
Consumer migration: the shared breakout_real.rs helpers and every GER40
example/test move to the Timestamp-native surface — open_ohlc_sources removed,
utc_month_window_ms -> utc_month_window (Timestamp), report_from_trace retyped,
the per-consumer ns_to_ms hand-divides deleted, the direct data_server imports
replaced by the re-exports. ger40_breakout_compare.rs (which opened OHLC by its
own hand-loop, reached by neither retyped helper) is migrated by inspection.
Return type is Vec, not [_; 4]: Harness::run consumes Vec<Box<dyn Source>>, so
it feeds straight in; order-safety comes from the single helper either way.
Tests: a hermetic round-trip pinning the inverse; a gated behaviour-preservation
test (open_ohlc Timestamp path == ms-path open, bit-identical recorded series);
a hermetic absent-archive fixture pinning the file-level None contract and the
#81 re-export firewall.
Verified: cargo build --workspace --all-targets clean; cargo test --workspace
green (the gated GER40 tests ran against the local archive — open_ohlc_seam
proves byte-identity on real data, not a skip); clippy --all-targets -D warnings
clean. Acceptance greps pass: open_ohlc_sources gone, no consumer-side ns_to_ms,
no data_server import in any migrated GER40 consumer, epoch_ns_to_unix_ms private.
closes#80, #81, #92
Consolidate aura-ingest's OHLC source-open surface onto the engine-native
epoch-ns Timestamp currency: a private seam-owned inverse, a Timestamp-window
opener, a canonical OHLC bundle opener (fixed C4 O/H/L/C order), and a
DataServer/DEFAULT_DATA_PATH re-export + default_data_server convenience — so a
real-data source builds from aura-ingest alone, with the one ms<->ns crossing
owned by the seam. Behaviour-preserving (byte-identical RunReports).
Grounded against the tree via the ground-spec-0052 workflow (25/28 assertions
confirmed; compare.rs compiler-invisible site + AC1 scope fix applied).
refs #80, #81, #92
Drop the gitignore entries for both dirs; align the CLAUDE.md note from "git-ignored, local-only" to "git-tracked — committed while the cycle is live, removed (git rm) at cycle close". The durable record stays the ledger + git history; specs/plans are transient in the repo, present only for their own cycle.
docs/specs and docs/plans were retired (prior commit); the source/test
comments that cited them ("spec 0050 §4.1", "spec §Testing N", "per spec",
"the spec's ...") now point at nothing. Strip every such pointer while
preserving the technical substance, the design-ledger contract refs
(C1/C11/C20/C34/C12.1/...), and the Gitea issue refs (#41).
Comment/doc edits only across 20 files — no logic change; full workspace
suite green, clippy clean.
Per-cycle specs/plans are step-scoped: every file maps to already-shipped
code, their snippets drift hard against current APIs (InputSpec->PortSpec,
Sim->Harness, schema() removed, C16 zero-dep reversed, ascii-dag retired),
and a stale spec read as an API reference misleads agents into non-compiling
code. No durable knowledge lost — rationale lives in the ledger + code
rustdoc, history in git + the tracker.
- gitignore docs/specs/ and docs/plans/
- delete all 117 existing specs/plans (recoverable via git history)
- CLAUDE.md: specs/plans are local-only ephemeral artifacts
The residency probe was an inherent method on the concrete M1FieldSource, so a Box<dyn Source> consumer could not read it without downcasting (the adjacent friction noted in #95). Hoist it onto the Source trait as resident_records(&self) -> Option<usize> with a default of None, mirroring bounds()'s Option-for-unknown idiom: None = "does not report" (the eager VecSource, which has no streaming ring), distinct from Some(0) = "reports, holds none" (e.g. exhausted).
M1FieldSource overrides it to return Some(chunk len). Callers updated (streaming_seam, the two deep-dive fieldtest bins). Tests: a hermetic VecSource default-is-None check, and a &dyn Source probe in the gated residency test proving M1FieldSource residency is readable without a downcast.
refs #95
The C12 realization and the 0041 spec claimed streaming residency is O(one chunk) at the process level ("a 20-year window streams in the same memory as a one-day one"). That holds only for the aura source ring (M1FieldSource::resident_records(), bounded by one chunk and correctly tested); whole-process RSS grows O(records-touched) because data-server's FileCache retains each window's parsed chunks (~56 B/record) read-only for the pass. Reproduced: 6 MiB @ 1mo -> 173 MiB @ ~10y, linear.
That retention is the replay-many optimization (load-once; close+volume share one parse via the field-agnostic FileKey), not a leak, and no always-on eviction can bound a single forward pass without regressing it. This narrows the docs/comments to the true per-source-ring property and names the FileCache per-window retention as the real, replay-amortized process cost. The streaming_seam assert is unchanged (it was always the ring bound; only its labelling confused ring vs process). The deferred opt-in non-retaining streaming read path is parked as Brummel/data-server#2.
closes#95
data-server does no timezone math (delphi_to_unix_ms is pure arithmetic), so the
archive carries whatever the raw Pepperstone files contain, labelled as UTC. The
Session node's UTC->Berlin DST conversion is unit-tested, but the assumption that
the raw timestamps are themselves UTC was unverified — a broker-local archive
would shift the whole session and the error would jump across the DST boundary.
Verified empirically via the DAX cash-open volume spike (GER40): it lands at
Berlin 09:00 in BOTH seasons, at 07:00 UTC (summer, CEST, UTC+2) and 08:00 UTC
(winter, CET, UTC+1) — a clean +1h DST shift, the genuine-UTC signature. This
gated test pins it so a future archive/convention drift cannot silently corrupt
every session strategy's bar alignment.
Add a C20 realization recording the resolution of the GER40 deep-dive's #94
finding: a hand-wired FlatGraph is not a shippable strategy (no param_space, so
World-opaque); the canonical form is the Composite blueprint, the FlatGraph its
compiled substrate (C23). Pins the bar-period-as-structural-construction-arg
decision (spec 0051) to the structural-axis-vs-tuning-param split.
refs #94
Brummel