Files
Aura/docs/design/INDEX.md
T
Brummel 3d33cc6fe5 docs: write the architecture design ledger (C1-C15)
Captures the rough-sketch interview as 15 discrete contracts (guarantee / forbids / why): determinism, causality, single-merge-at-ingestion, cycle granularity, freshness-gated recompute, firing policies A/B, SoA scalar payloads, node contract, fractal acyclic composition, the broker-is-part-of-strategy execution chain, generalized sources with the record-then-replay boundary, the atomic sim unit + four axes, authoring-only hot-reload, headless two-faced core, and resampling/sessions. Also corrects the tracker repo name to Brummel/Aura.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 01:33:06 +02:00

13 KiB
Raw Blame History

aura design ledger — INDEX

The ledger records the load-bearing design contracts and their rationale. Each contract states what it guarantees, what it forbids, and why. A change that breaks a contract is a design decision (amend the contract here with its new rationale), never a silent refactor.

Provenance: contracts C1C15 were settled in the initial rough-sketch design interview (2026-06-03), walking the design tree root-to-leaf. The CLAUDE.md Domain invariants section is the compressed, always-loaded summary of the subset that agents must never violate; this file is the fuller form with rationale.

Vocabulary: a contract is one ledger entry. A cycle is one pipeline round; a milestone is a tracker container spanning many cycles (the first milestone is the walking skeleton: ingest → one signal → deterministic backtest → equity metric).


Foundation — what aura is

aura is a framework and a playground for traders. A human and (primarily) LLMs author trading nodes directly in Rust; the engine backtests them deterministically and massively in parallel, composes them fractally, validates them (sweep / Monte-Carlo / walk-forward), and freezes a validated strategy into a standalone bot with a broker connection.

The predecessor RustAst (myc) tried this as a custom DSL and failed: too slow, too buggy, and LLMs author far better in Rust than in an unfamiliar DSL. aura inverts it — engine in Rust, strategies in Rust — but keeps RustAst's concepts (synchronous reactive streams, bounded-lookback series, run-counting, SoA). RustAst is a conceptual reference, not a dependency. The one reused component is data-server (the first data source).


Contracts

C1 — Determinism and disjoint parallelism

Guarantee. A backtest is a deterministic, synchronous, non-concurrent event loop that reaches a unique state after each input tick. Same input (incl. seed) → bit-identical run. Two backtests are fully disjoint and run concurrently without locking. Forbids. Concurrency within a single sim; any nondeterministic input that is not captured as an explicit input (see C11, C12). Why. Real money rides on backtest results; reproducibility and an audit trail are non-negotiable. Speed comes from parallelism across sims, which disjointness makes lock-free.

C2 — Causality / no look-ahead

Guarantee. A node sees only the past. Input history is a read-only window that ends at the current cursor; a resampler emits a bar only once it is complete. Forbids. Any node access to data with timestamp > now; emitting a partial / still-forming bar. Why. Look-ahead is the cardinal backtester bug — a fast backtester that leaks the future is worse than none. Making the future physically absent from what a node receives beats merely discouraging it.

C3 — One merge, at ingestion only

Guarantee. Heterogeneous timestamped sources are k-way-merged by time_ms into one chronological cycle stream at the ingestion boundary. Forbids. Any merge / as-of join inside the graph. Why. A single ordered timeline is the mechanism that makes heterogeneous-rate sources (news daily-bias + M5 + ticks) causally combinable without leaking the future. Keeping the merge at one boundary keeps the graph semantics simple.

C4 — Cycle granularity

Guarantee. The clock is data-driven: one input record = one cycle, advanced in global timestamp order, with a monotonic cycle_id. Ties (same timestamp, multiple sources) break by source declaration order. Forbids. A fixed time-grid clock; nondeterministic tie ordering. Why. The market is an irregular event sequence; a grid is arbitrary and either wastes empty cycles or clumps ticks. Backtest and live differ only in the origin of records, not the cycle semantics. Tie determinism preserves C1.

C5 — Freshness-gated recompute and sample-and-hold

Guarantee. The cycle_id advances everywhere (a cheap counter), but a node re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by run-count); otherwise it holds its last output. Stale inputs contribute their last (held) value. Forbids. Recomputing every node every cycle ("push all" is true for the clock, not for *recompute"); treating a held value as missing. Why. Total recompute does not scale to many sparse high-frequency sources; freshness-gating is the performance discipline that keeps the synchronous model fast.

C6 — Firing policy A and B, per input group

Guarantee. A node declares, per input group, one of two firing policies: A fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held daily-bias); B all-fresh barrier (synchronizing join — e.g. O/H/L/C from four separate 15m sources: the candle is complete only when all four are fresh). A single node may mix an A input and a B group. Forbids. A single global firing mode; forcing per-node-only granularity. Why. Both are genuinely needed; RustAst implemented only B. Per-input-group granularity is required by the OHLC-plus-bias case where one node needs both.

C7 — Four scalar base types, streamed as SoA

Guarantee. Only i64, f64, bool, timestamp (newtype over i64, epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams (OHLCV) are bundles of base columns. Edges are type-erased to these four kinds; the type check is paid once at wiring/sim-start, then the topology is frozen per sim → direct dispatch, no per-event allocation. Forbids. Streaming non-scalars (String, Records, tables, calendars) — those live as metadata beside the hot path; dyn Any payloads; per-event heap allocation; topology mutation mid-sim. Why. Maximal streaming performance (SIMD/cache) needs a tiny closed scalar set and SoA. The open set is composites (schemas of columns), not scalar types. Type-erasure at the edge is also forced by the cdylib boundary (C13).

C8 — The node contract

Guarantee. A node implements schema() (declares each input's scalar type, required lookback depth, and firing group) + eval(ctx) -> Option<Scalar>. The engine provides read-only, zero-copy windows into each input's SoA ring buffer (ctx.f64_in(x)[k], sized at wiring); a node may additionally keep its own mutable series for derived/intermediate state. None/Void return = filter / not-yet-warmed-up. A node has exactly one output (one series per node). Forbids. A node sizing/growing its input lookback at runtime; multiple named outputs per node (model as multiple nodes); copy-on-read of input history. Why. Engine-provided windows mean LLM-authored code cannot mis-manage lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop).

C9 — Fractal, acyclic composition

Guarantee. A composite is itself a Node that wires a sub-graph and exposes one output; signal, combined signal, and (with execution) strategy are all the same abstraction, nestable arbitrarily. The dataflow graph is a DAG; the only feedback path is an explicit delay/state node (the RTL "register"). Wiring is written in Rust (builder API); the built graph is introspectable runtime data. Forbids. Implicit dataflow cycles (combinational loops); special-casing "signal-of-signals" as separate mechanics. Why. Self-application of one contract gives unlimited composition with no adapter zoo. Acyclicity keeps the synchronous reactive model well-defined; forcing feedback through a visible delay node keeps the per-cycle determinism intact and the one legitimate feedback path explicit. Graph-as-data enables visualization, freezing, and re-parameterization for sweeps.

C10 — The execution chain; broker is part of the strategy

Guarantee. Execution is in-graph: signals (scores) → decision node (score → target position, sizing, risk) → broker node (intent → fills → position / trade / equity streams). The broker node is a fixed part of the strategy and identical in sim and live (one accounting path → one equity-curve shape). It is parameterized by a swappable broker profile (fees, spread/fill rules, order types, lot/margin); "same strategy, different broker" = swap the profile. The portfolio is multi-instrument (positions per symbol-key). Fills cross the real spread from data-server (buy @ ask, sell @ bid; M1 carries spread, tick carries ask/bid), causally (only prices that occurred ≤ now). Forbids. A SimBroker/LiveBroker swap; computing equity by two different code paths; synthetic spread when real bid/ask is available; in-graph live order routing. Why. Realistic P&L does not exist without committing to a broker model, so the broker is part of the testable artifact. One accounting path guarantees the equity curve looks the same in both modes by construction. Live order routing / reconciliation is an external adapter outside the strategy graph; the displayed equity stays the modeled one.

C11 — Generalized sources; record-then-replay determinism boundary

Guarantee. A source is anything that produces timestamped scalar streams — market data (data-server) and non-financial sources (e.g. a news-agent node emitting a bias) are treated identically. Anything nondeterministic, external, or slow (LLM/news/web) is materialized into a recorded, timestamped stream before it enters the engine; backtest replays the recording, live computes fresh in real time and records it for future backtests. A bias enters as a value held until the next event (firing policy A). Forbids. Any live external call inside a backtest replay. Why. It is the only model compatible with reproducible backtests — LLM calls are nondeterministic and far too slow per-cycle. Per ~/.claude/CLAUDE.md, external LLM (IONOS) calls happen only at the recording/live-source edge, with explicit per-session consent, never inside a sim.

C12 — The atomic sim unit and the four orchestration axes

Guarantee. The atomic unit is (frozen topology + param-set + data-window + RNG-seed) → deterministic run → metrics. Parameters are typed, ranged, runtime values injected at graph build (no recompile per param-set; the optimizer sees a generic vector of typed ranges). Raw data is shared read-only across sims via Arc<[T]> (data-server is built for this). Four axes orchestrate the atomic unit: (1) param-sweep (grid/random), (2) optimization (argmax metric), (3) walk-forward (rolling in-sample optimize + out-of-sample test), (4) Monte-Carlo (N seeded realizations perturbing input). MC = sweep over seeds; each realization is itself deterministic given its seed. Forbids. Baking a specific search strategy (Bayesian/genetic) into the primitive — those are pluggable policies atop the atomic unit; recompiling on a param change. Why. A stable primitive + orchestration axes keeps "wahnsinnig schnell" (embarrassingly parallel across the unit) cleanly separated from search policy. Seed-as-input reconciles Monte-Carlo with C1.

C13 — Hot-reload is authoring-only; deploy is frozen

Guarantee. A node/strategy is authored as a native Rust cdylib, hot-reloaded during the authoring loop (Rust-ABI; host and node built with the same toolchain). The live/deploy bot is a statically-linked, versioned, frozen artifact. Forbids. Hot-swapping a running live bot; loading third-party / foreign- toolchain plugins. Why. Hot-reload makes the research loop fast; a live artifact must be frozen and reproducible (audit trail: this bot = this commit). A sweep pays no hot-reload tax — params are runtime data (C12), so the cdylib loads once.

C14 — Headless core, two faces

Guarantee. The engine is a UI-agnostic library. Two faces sit on it: a programmatic/CLI face (the primary surface for the LLM and automation — author a node, run a sim/sweep, emit structured metrics) and a visual face for human exploration. Visualization is only a downstream consumer node on the streams. Forbids. Any UI/pixel knowledge inside the engine. Why. The LLM drives programmatically, the human visually; a headless core serves both and makes the visual face freely deferrable. (Visual face leaning egui-native, in-process zero-copy from the SoA columns — deferred decision, see Open threads.)

C15 — Resampling-as-node; sessions/calendars

Guarantee. A resampler is a node (finer stream → coarser bar stream), clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars and instrument specs are metadata (non-scalar, beside the hot path); session context is exposed as scalar streams via a SessionNode (bars_since_open: i64, in_session: bool, session_open_ts: timestamp). "3rd 15m candle after session open" is then a plain node checking bars_since_open == 3. Forbids. Streaming the calendar; special-casing session logic outside the stream model. Why. Keeps the line consistent — everything a signal needs arrives as a stream; reference data feeds source/session nodes from beside the hot path.


Open architectural threads not yet resolved

  • Visual playground form — leaning egui-native; deferred. The headless core (C14) makes deferring it free.
  • Parameter-space search strategies (Bayesian/genetic) — pluggable policies atop the atomic sim unit (C12), not yet designed.
  • strategies/ split — a later split of top-level strategies from reusable building blocks in nodes/; not a day-1 cut.
  • Sequencing — engine + CLI face first; walking-skeleton milestone before hot-reload, sweep, sessions, and the visual playground.