docs: write the architecture design ledger (C1-C15)
Captures the rough-sketch interview as 15 discrete contracts (guarantee / forbids / why): determinism, causality, single-merge-at-ingestion, cycle granularity, freshness-gated recompute, firing policies A/B, SoA scalar payloads, node contract, fractal acyclic composition, the broker-is-part-of-strategy execution chain, generalized sources with the record-then-replay boundary, the atomic sim unit + four axes, authoring-only hot-reload, headless two-faced core, and resampling/sessions. Also corrects the tracker repo name to Brummel/Aura. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -39,8 +39,8 @@ git:
|
||||
issue_tracker:
|
||||
kind: gitea
|
||||
close_marker: "closes #N"
|
||||
url: "http://192.168.178.103:3000/Brummel/aura/issues"
|
||||
list_cmd: "tea issues ls --repo Brummel/aura --state open"
|
||||
url: "http://192.168.178.103:3000/Brummel/Aura/issues"
|
||||
list_cmd: "tea issues ls --repo Brummel/Aura --state open"
|
||||
|
||||
notifications: {}
|
||||
|
||||
|
||||
+225
-11
@@ -1,21 +1,235 @@
|
||||
# aura design ledger — INDEX
|
||||
|
||||
The ledger records the load-bearing design contracts and their rationale. Each
|
||||
entry is one contract: what it guarantees, what it forbids, and why.
|
||||
contract states what it **guarantees**, what it **forbids**, and **why**. A
|
||||
change that breaks a contract is a design decision (amend the contract here with
|
||||
its new rationale), never a silent refactor.
|
||||
|
||||
This is the place where the architecture settled during the initial rough-sketch
|
||||
interview will be written up as discrete contracts. Until those entries land, the
|
||||
authoritative summary of the invariants is the **Domain invariants** section of
|
||||
`CLAUDE.md`, and the cross-session rationale notes live in the project memory.
|
||||
Provenance: contracts C1–C15 were settled in the initial rough-sketch design
|
||||
interview (2026-06-03), walking the design tree root-to-leaf. The `CLAUDE.md`
|
||||
**Domain invariants** section is the compressed, always-loaded summary of the
|
||||
subset that agents must never violate; this file is the fuller form with
|
||||
rationale.
|
||||
|
||||
Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a
|
||||
*milestone* is a tracker container spanning many cycles (the first milestone is
|
||||
the **walking skeleton**: ingest → one signal → deterministic backtest → equity
|
||||
metric).
|
||||
|
||||
---
|
||||
|
||||
## Foundation — what aura is
|
||||
|
||||
aura is a framework **and** a playground for traders. A human and (primarily)
|
||||
LLMs author trading **nodes** directly in Rust; the engine backtests them
|
||||
deterministically and massively in parallel, composes them fractally, validates
|
||||
them (sweep / Monte-Carlo / walk-forward), and freezes a validated strategy into
|
||||
a standalone bot with a broker connection.
|
||||
|
||||
The predecessor RustAst (`myc`) tried this as a custom DSL and failed: too slow,
|
||||
too buggy, and LLMs author far better in Rust than in an unfamiliar DSL. aura
|
||||
inverts it — engine in Rust, strategies in Rust — but keeps RustAst's *concepts*
|
||||
(synchronous reactive streams, bounded-lookback series, run-counting, SoA).
|
||||
RustAst is a conceptual reference, not a dependency. The one reused component is
|
||||
`data-server` (the first data source).
|
||||
|
||||
---
|
||||
|
||||
## Contracts
|
||||
|
||||
_(none yet — to be authored from the first spec onward)_
|
||||
### C1 — Determinism and disjoint parallelism
|
||||
**Guarantee.** A backtest is a deterministic, synchronous, non-concurrent event
|
||||
loop that reaches a unique state after each input tick. Same input (incl. seed)
|
||||
→ bit-identical run. Two backtests are fully disjoint and run concurrently
|
||||
without locking.
|
||||
**Forbids.** Concurrency *within* a single sim; any nondeterministic input that
|
||||
is not captured as an explicit input (see C11, C12).
|
||||
**Why.** Real money rides on backtest results; reproducibility and an audit
|
||||
trail are non-negotiable. Speed comes from parallelism *across* sims, which
|
||||
disjointness makes lock-free.
|
||||
|
||||
### C2 — Causality / no look-ahead
|
||||
**Guarantee.** A node sees only the past. Input history is a read-only window
|
||||
that ends at the current cursor; a resampler emits a bar only once it is
|
||||
complete.
|
||||
**Forbids.** Any node access to data with `timestamp > now`; emitting a partial
|
||||
/ still-forming bar.
|
||||
**Why.** Look-ahead is the cardinal backtester bug — a fast backtester that
|
||||
leaks the future is worse than none. Making the future *physically absent* from
|
||||
what a node receives beats merely discouraging it.
|
||||
|
||||
### C3 — One merge, at ingestion only
|
||||
**Guarantee.** Heterogeneous timestamped sources are k-way-merged by `time_ms`
|
||||
into one chronological cycle stream at the ingestion boundary.
|
||||
**Forbids.** Any merge / as-of join *inside* the graph.
|
||||
**Why.** A single ordered timeline is the mechanism that makes heterogeneous-rate
|
||||
sources (news daily-bias + M5 + ticks) causally combinable without leaking the
|
||||
future. Keeping the merge at one boundary keeps the graph semantics simple.
|
||||
|
||||
### C4 — Cycle granularity
|
||||
**Guarantee.** The clock is data-driven: one input record = one cycle, advanced
|
||||
in global timestamp order, with a monotonic `cycle_id`. Ties (same timestamp,
|
||||
multiple sources) break by source declaration order.
|
||||
**Forbids.** A fixed time-grid clock; nondeterministic tie ordering.
|
||||
**Why.** The market *is* an irregular event sequence; a grid is arbitrary and
|
||||
either wastes empty cycles or clumps ticks. Backtest and live differ only in the
|
||||
origin of records, not the cycle semantics. Tie determinism preserves C1.
|
||||
|
||||
### C5 — Freshness-gated recompute and sample-and-hold
|
||||
**Guarantee.** The `cycle_id` advances everywhere (a cheap counter), but a node
|
||||
re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by
|
||||
run-count); otherwise it holds its last output. Stale inputs contribute their
|
||||
last (held) value.
|
||||
**Forbids.** Recomputing every node every cycle ("push all" is true for the
|
||||
*clock*, not for *recompute"); treating a held value as missing.
|
||||
**Why.** Total recompute does not scale to many sparse high-frequency sources;
|
||||
freshness-gating is the performance discipline that keeps the synchronous model
|
||||
fast.
|
||||
|
||||
### C6 — Firing policy A and B, per input group
|
||||
**Guarantee.** A node declares, per input group, one of two firing policies:
|
||||
**A** fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held
|
||||
daily-bias); **B** all-fresh barrier (synchronizing join — e.g. O/H/L/C from
|
||||
four separate 15m sources: the candle is complete only when all four are fresh).
|
||||
A single node may mix an A input and a B group.
|
||||
**Forbids.** A single global firing mode; forcing per-node-only granularity.
|
||||
**Why.** Both are genuinely needed; RustAst implemented only B. Per-input-group
|
||||
granularity is required by the OHLC-plus-bias case where one node needs both.
|
||||
|
||||
### C7 — Four scalar base types, streamed as SoA
|
||||
**Guarantee.** Only `i64`, `f64`, `bool`, `timestamp` (newtype over i64,
|
||||
epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams
|
||||
(OHLCV) are bundles of base columns. Edges are type-erased to these four kinds;
|
||||
the type check is paid once at wiring/sim-start, then the topology is frozen per
|
||||
sim → direct dispatch, no per-event allocation.
|
||||
**Forbids.** Streaming non-scalars (String, Records, tables, calendars) — those
|
||||
live as metadata beside the hot path; `dyn Any` payloads; per-event heap
|
||||
allocation; topology mutation mid-sim.
|
||||
**Why.** Maximal streaming performance (SIMD/cache) needs a tiny closed scalar
|
||||
set and SoA. The open set is composites (schemas of columns), not scalar types.
|
||||
Type-erasure at the edge is also forced by the cdylib boundary (C13).
|
||||
|
||||
### C8 — The node contract
|
||||
**Guarantee.** A node implements `schema()` (declares each input's scalar type,
|
||||
required lookback depth, and firing group) + `eval(ctx) -> Option<Scalar>`. The
|
||||
engine provides read-only, zero-copy windows into each input's SoA ring buffer
|
||||
(`ctx.f64_in(x)[k]`, sized at wiring); a node may *additionally* keep its own
|
||||
mutable series for derived/intermediate state. `None`/Void return = filter /
|
||||
not-yet-warmed-up. A node has exactly one output (one series per node).
|
||||
**Forbids.** A node sizing/growing its input lookback at runtime; multiple named
|
||||
outputs per node (model as multiple nodes); copy-on-read of input history.
|
||||
**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage
|
||||
lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized
|
||||
buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop).
|
||||
|
||||
### C9 — Fractal, acyclic composition
|
||||
**Guarantee.** A composite is itself a `Node` that wires a sub-graph and exposes
|
||||
one output; signal, combined signal, and (with execution) strategy are all the
|
||||
same abstraction, nestable arbitrarily. The dataflow graph is a DAG; the only
|
||||
feedback path is an explicit delay/state node (the RTL "register"). Wiring is
|
||||
written in Rust (builder API); the built graph is introspectable runtime data.
|
||||
**Forbids.** Implicit dataflow cycles (combinational loops); special-casing
|
||||
"signal-of-signals" as separate mechanics.
|
||||
**Why.** Self-application of one contract gives unlimited composition with no
|
||||
adapter zoo. Acyclicity keeps the synchronous reactive model well-defined;
|
||||
forcing feedback through a visible delay node keeps the per-cycle determinism
|
||||
intact and the one legitimate feedback path explicit. Graph-as-data enables
|
||||
visualization, freezing, and re-parameterization for sweeps.
|
||||
|
||||
### C10 — The execution chain; broker is part of the strategy
|
||||
**Guarantee.** Execution is in-graph: `signals (scores) → decision node (score →
|
||||
target position, sizing, risk) → broker node (intent → fills → position / trade
|
||||
/ equity streams)`. The broker node is a fixed part of the strategy and
|
||||
identical in sim and live (one accounting path → one equity-curve shape). It is
|
||||
parameterized by a swappable **broker profile** (fees, spread/fill rules, order
|
||||
types, lot/margin); "same strategy, different broker" = swap the profile. The
|
||||
portfolio is multi-instrument (positions per symbol-key). Fills cross the real
|
||||
spread from `data-server` (buy @ ask, sell @ bid; M1 carries `spread`, tick
|
||||
carries ask/bid), causally (only prices that occurred ≤ now).
|
||||
**Forbids.** A `SimBroker`/`LiveBroker` swap; computing equity by two different
|
||||
code paths; synthetic spread when real bid/ask is available; in-graph live order
|
||||
routing.
|
||||
**Why.** Realistic P&L does not exist without committing to a broker model, so
|
||||
the broker is part of the testable artifact. One accounting path guarantees the
|
||||
equity curve looks the same in both modes by construction. Live order routing /
|
||||
reconciliation is an external adapter outside the strategy graph; the displayed
|
||||
equity stays the modeled one.
|
||||
|
||||
### C11 — Generalized sources; record-then-replay determinism boundary
|
||||
**Guarantee.** A source is anything that produces timestamped scalar streams —
|
||||
market data (`data-server`) and non-financial sources (e.g. a news-agent node
|
||||
emitting a bias) are treated identically. Anything nondeterministic, external,
|
||||
or slow (LLM/news/web) is materialized into a recorded, timestamped stream
|
||||
*before* it enters the engine; backtest replays the recording, live computes
|
||||
fresh in real time and records it for future backtests. A bias enters as a value
|
||||
held until the next event (firing policy A).
|
||||
**Forbids.** Any live external call *inside* a backtest replay.
|
||||
**Why.** It is the only model compatible with reproducible backtests — LLM calls
|
||||
are nondeterministic and far too slow per-cycle. Per `~/.claude/CLAUDE.md`,
|
||||
external LLM (IONOS) calls happen only at the recording/live-source edge, with
|
||||
explicit per-session consent, never inside a sim.
|
||||
|
||||
### C12 — The atomic sim unit and the four orchestration axes
|
||||
**Guarantee.** The atomic unit is `(frozen topology + param-set + data-window +
|
||||
RNG-seed) → deterministic run → metrics`. Parameters are typed, ranged, runtime
|
||||
values injected at graph build (no recompile per param-set; the optimizer sees a
|
||||
generic vector of typed ranges). Raw data is shared read-only across sims via
|
||||
`Arc<[T]>` (data-server is built for this). Four axes orchestrate the atomic
|
||||
unit: (1) param-sweep (grid/random), (2) optimization (argmax metric),
|
||||
(3) walk-forward (rolling in-sample optimize + out-of-sample test),
|
||||
(4) Monte-Carlo (N seeded realizations perturbing input). **MC = sweep over
|
||||
seeds**; each realization is itself deterministic given its seed.
|
||||
**Forbids.** Baking a specific search strategy (Bayesian/genetic) into the
|
||||
primitive — those are pluggable policies atop the atomic unit; recompiling on a
|
||||
param change.
|
||||
**Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell"
|
||||
(embarrassingly parallel across the unit) cleanly separated from search policy.
|
||||
Seed-as-input reconciles Monte-Carlo with C1.
|
||||
|
||||
### C13 — Hot-reload is authoring-only; deploy is frozen
|
||||
**Guarantee.** A node/strategy is authored as a native Rust `cdylib`,
|
||||
hot-reloaded during the authoring loop (Rust-ABI; host and node built with the
|
||||
same toolchain). The live/deploy bot is a statically-linked, versioned, frozen
|
||||
artifact.
|
||||
**Forbids.** Hot-swapping a running live bot; loading third-party / foreign-
|
||||
toolchain plugins.
|
||||
**Why.** Hot-reload makes the research loop fast; a live artifact must be frozen
|
||||
and reproducible (audit trail: this bot = this commit). A sweep pays no
|
||||
hot-reload tax — params are runtime data (C12), so the cdylib loads once.
|
||||
|
||||
### C14 — Headless core, two faces
|
||||
**Guarantee.** The engine is a UI-agnostic library. Two faces sit on it: a
|
||||
**programmatic/CLI** face (the primary surface for the LLM and automation —
|
||||
author a node, run a sim/sweep, emit structured metrics) and a **visual** face
|
||||
for human exploration. Visualization is only a downstream consumer node on the
|
||||
streams.
|
||||
**Forbids.** Any UI/pixel knowledge inside the engine.
|
||||
**Why.** The LLM drives programmatically, the human visually; a headless core
|
||||
serves both and makes the visual face freely deferrable. (Visual face leaning
|
||||
egui-native, in-process zero-copy from the SoA columns — deferred decision, see
|
||||
Open threads.)
|
||||
|
||||
### C15 — Resampling-as-node; sessions/calendars
|
||||
**Guarantee.** A resampler is a node (finer stream → coarser bar stream),
|
||||
clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars
|
||||
and instrument specs are metadata (non-scalar, beside the hot path); session
|
||||
*context* is exposed as scalar streams via a `SessionNode` (`bars_since_open:
|
||||
i64`, `in_session: bool`, `session_open_ts: timestamp`). "3rd 15m candle after
|
||||
session open" is then a plain node checking `bars_since_open == 3`.
|
||||
**Forbids.** Streaming the calendar; special-casing session logic outside the
|
||||
stream model.
|
||||
**Why.** Keeps the line consistent — everything a signal needs arrives as a
|
||||
stream; reference data feeds source/session nodes from beside the hot path.
|
||||
|
||||
---
|
||||
|
||||
## Open architectural threads not yet resolved
|
||||
|
||||
- Visual playground form (leaning egui-native; deferred — the headless core
|
||||
makes deferring it free).
|
||||
- Smarter parameter-space search strategies (Bayesian/genetic) as pluggable
|
||||
policies atop the atomic sim unit.
|
||||
- Top-level `strategies/` split from reusable building blocks in `nodes/`.
|
||||
- **Visual playground form** — leaning egui-native; deferred. The headless core
|
||||
(C14) makes deferring it free.
|
||||
- **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies
|
||||
atop the atomic sim unit (C12), not yet designed.
|
||||
- **`strategies/` split** — a later split of top-level strategies from reusable
|
||||
building blocks in `nodes/`; not a day-1 cut.
|
||||
- **Sequencing** — engine + CLI face first; walking-skeleton milestone before
|
||||
hot-reload, sweep, sessions, and the visual playground.
|
||||
|
||||
Reference in New Issue
Block a user