Aura/docs/design/INDEX.md

# aura design ledger — INDEX

The ledger records the load-bearing design contracts and their rationale. Each
contract states what it **guarantees**, what it **forbids**, and **why**. A
change that breaks a contract is a design decision (amend the contract here with
its new rationale), never a silent refactor.

Provenance: contracts C1–C18 were settled in the initial rough-sketch design
interview (2026-06-03), walking the design tree root-to-leaf (C16–C18 and the
C10 refinement to a broker-independent position table came in follow-up turns).
The `CLAUDE.md` **Domain invariants** section is the compressed, always-loaded
summary of the
subset that agents must never violate; this file is the fuller form with
rationale.

Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a
*milestone* is a tracker container spanning many cycles (the first milestone is
the **walking skeleton**: ingest → one signal → deterministic backtest →
position table → sim-optimal broker → synthetic pip-equity metric).

---

## Foundation — what aura is

aura is a framework **and** a playground for traders. A human and (primarily)
LLMs author trading **nodes** directly in Rust; the engine backtests them
deterministically and massively in parallel, composes them fractally, validates
them (sweep / Monte-Carlo / walk-forward), and freezes a validated strategy into
a standalone bot with a broker connection.

The predecessor RustAst (`myc`) tried this as a custom DSL and failed: too slow,
too buggy, and LLMs author far better in Rust than in an unfamiliar DSL. aura
inverts it — engine in Rust, strategies in Rust — but keeps RustAst's *concepts*
(synchronous reactive streams, bounded-lookback series, run-counting, SoA).
RustAst is a conceptual reference, not a dependency. The one reused component is
`data-server` (the first data source).

---

## Contracts

### C1 — Determinism and disjoint parallelism
**Guarantee.** A backtest is a deterministic, synchronous, non-concurrent event
loop that reaches a unique state after each input tick. Same input (incl. seed)
→ bit-identical run. Two backtests are fully disjoint and run concurrently
without locking.
**Forbids.** Concurrency *within* a single sim; any nondeterministic input that
is not captured as an explicit input (see C11, C12).
**Why.** Real money rides on backtest results; reproducibility and an audit
trail are non-negotiable. Speed comes from parallelism *across* sims, which
disjointness makes lock-free.

### C2 — Causality / no look-ahead
**Guarantee.** A node sees only the past. Input history is a read-only window
that ends at the current cursor; a resampler emits a bar only once it is
complete.
**Forbids.** Any node access to data with `timestamp > now`; emitting a partial
/ still-forming bar.
**Why.** Look-ahead is the cardinal backtester bug — a fast backtester that
leaks the future is worse than none. Making the future *physically absent* from
what a node receives beats merely discouraging it.

### C3 — One merge, at ingestion only
**Guarantee.** Heterogeneous timestamped sources are k-way-merged by timestamp
into one chronological cycle stream at the ingestion boundary; source-native
time units (e.g. data-server's Unix-`time_ms`) are normalized there to the
canonical epoch-ns `timestamp` of C7.
**Forbids.** Any merge / as-of join *inside* the graph.
**Why.** A single ordered timeline is the mechanism that makes heterogeneous-rate
sources (news daily-bias + M5 + ticks) causally combinable without leaking the
future. Keeping the merge at one boundary keeps the graph semantics simple.

### C4 — Cycle granularity
**Guarantee.** The clock is data-driven: one input record = one cycle, advanced
in global timestamp order, with a monotonic `cycle_id`. Ties (same timestamp,
multiple sources) break by source declaration order.
**Forbids.** A fixed time-grid clock; nondeterministic tie ordering.
**Why.** The market *is* an irregular event sequence; a grid is arbitrary and
either wastes empty cycles or clumps ticks. Backtest and live differ only in the
origin of records, not the cycle semantics. Tie determinism preserves C1.

### C5 — Freshness-gated recompute and sample-and-hold
**Guarantee.** The `cycle_id` advances everywhere (a cheap counter), but a node
re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by
run-count); otherwise it holds its last output. Stale inputs contribute their
last (held) value.
**Forbids.** Recomputing every node every cycle ("push all" is true for the
*clock*, not for *recompute"); treating a held value as missing.
**Why.** Total recompute does not scale to many sparse high-frequency sources;
freshness-gating is the performance discipline that keeps the synchronous model
fast.

### C6 — Firing policy A and B, per input group
**Guarantee.** A node declares, per input group, one of two firing policies:
**A** fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held
daily-bias); **B** all-fresh barrier (synchronizing join — e.g. O/H/L/C from
four separate 15m sources: the candle is complete only when all four are fresh).
A single node may mix an A input and a B group.
**Forbids.** A single global firing mode; forcing per-node-only granularity.
**Why.** Both are genuinely needed; RustAst implemented only B. Per-input-group
granularity is required by the OHLC-plus-bias case where one node needs both.

### C7 — Four scalar base types, streamed as SoA
**Guarantee.** Only `i64`, `f64`, `bool`, `timestamp` (newtype over i64,
epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams
(OHLCV) are bundles of base columns. Edges are type-erased to these four kinds;
the type check is paid once at wiring/sim-start, then the topology is frozen per
sim → direct dispatch, no per-event allocation.
**Forbids.** Streaming non-scalars (String, Records, tables, calendars) — those
live as metadata beside the hot path; `dyn Any` payloads; per-event heap
allocation; topology mutation mid-sim.
**Why.** Maximal streaming performance (SIMD/cache) needs a tiny closed scalar
set and SoA. The open set is composites (schemas of columns), not scalar types.
Type-erasure at the edge is also forced by the cdylib boundary (C13).

### C8 — The node contract
**Guarantee.** A node implements `schema()` (declares each input's scalar type,
required lookback depth, and firing group, **and the node's own tunable
parameters — typed, with ranges**, which aggregate into the blueprint's
param-space the optimizer sweeps, C12/C19/C20) + `eval(ctx) -> Option<Scalar>`. The
engine provides read-only, zero-copy windows into each input's SoA ring buffer
(`ctx.f64_in(x)[k]`, sized at wiring); a node may *additionally* keep its own
mutable series for derived/intermediate state. `None`/Void return = filter /
not-yet-warmed-up. A node is a **producer, a consumer, or both**: a
producer/transformer exposes **at most one** output (one series per node); a
**pure consumer (sink)** — chart, equity, logger — has **no** output. Sources
are pure producers; sinks are pure consumers.
**Forbids.** A node sizing/growing its input lookback at runtime; more than one
output per node (model as multiple nodes); copy-on-read of input history.
**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage
lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized
buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop).

### C9 — Fractal, acyclic composition
**Guarantee.** A composite is itself a `Node` that wires a sub-graph and exposes
one output; signal, combined signal, and (with execution) strategy are all the
same abstraction, nestable arbitrarily. The dataflow graph is a DAG; the only
feedback path is an explicit delay/state node (the RTL "register"). Wiring is
written in Rust (builder API); the built graph is introspectable runtime data.
**Forbids.** Implicit dataflow cycles (combinational loops); special-casing
"signal-of-signals" as separate mechanics.
**Why.** Self-application of one contract gives unlimited composition with no
adapter zoo. Acyclicity keeps the synchronous reactive model well-defined;
forcing feedback through a visible delay node keeps the per-cycle determinism
intact and the one legitimate feedback path explicit. Graph-as-data enables
visualization, freezing, and re-parameterization for sweeps.

### C10 — Strategy result is a broker-independent position table; brokers are downstream nodes
**Guarantee.** A strategy's result is **not** an equity curve but a
**broker-independent, time-ordered table of position events**. The chain is
`signals (scores) → decision/sizing node → position-event output`. An event is
pure scalar columns (C7): `event_ts: timestamp`, `action: i64` (buy / sell /
close), `position_id: i64`, `instrument_id: i64`, `volume: f64` (unsigned —
direction is the `action`). A position's open time is the `event_ts` of its
opening event (there is no separate `open_ts`); a `close` references a
`position_id` and may be partial via its own `volume`. The **set of open
positions at time t** (opens minus closes with `event_ts ≤ t`) is the strategy's
*state* at t; the ordered sequence of these states is the result. Position
sizing and risk live here (they set `volume`); the portfolio is multi-instrument.

A **broker is a downstream consumer node** (C8 / C9): it consumes the strategy's
position-event stream — plus the relevant price streams, to mark open positions —
and emits an **equity stream** as its output. It is *not* part of the strategy.
Because it is an ordinary node, **several brokers can be attached to the same
position table at once**, each emitting its own equity stream, so the resulting
curves are directly comparable. Two classes: **(a) the sim-optimal broker** —
deterministic, frictionless, perfect-fill execution producing a **synthetic
equity stream in pips** (no real currency, no real-broker constraints); the
neutral yardstick for comparing and optimizing strategy *logic*. **(b) realistic
broker nodes** (Pepperstone, …) — apply real spread / commission / slippage /
lot / margin, may reject or modify positions, and produce a currency equity
stream for viability and deployment. Pip PnL uses per-instrument pip metadata
(reference data beside the hot path, C7). Live: a realistic broker node consumes
the position events in real time and routes orders as a side effect;
reconciliation with the real account is an external adapter.
**Forbids.** Treating an equity curve as the strategy's output; baking a broker
into the strategy; a special external broker subsystem (a broker is an ordinary
node); storing `open_ts` (derive it from the opening event); a signed-volume
direction trick (use `action`); broker-specific assumptions leaking into the
strategy logic.
**Why.** A strategy can be judged neutrally only if its result is independent of
any real broker's frictions. The position table is that broker-independent
invariant: one table feeds many broker nodes, each yielding its own equity — so
attaching a synthetic and a real broker side by side gives two comparable
curves, and "same strategy, different broker" and "same decisions sim vs live"
both fall out. The sim-optimal pip curve is a level, currency-free playing field
for comparison; realistic broker nodes then test real-world viability. Modelling
the broker as a node (not a bespoke subsystem) keeps it within the one Node/graph
abstraction (C9). This supersedes the earlier "broker is part of the strategy"
framing.

### C11 — Generalized sources; record-then-replay determinism boundary
**Guarantee.** A source is anything that produces timestamped scalar streams —
market data (`data-server`) and non-financial sources (e.g. a news-agent node
emitting a bias) are treated identically. Anything nondeterministic, external,
or slow (LLM/news/web) is materialized into a recorded, timestamped stream
*before* it enters the engine; backtest replays the recording, live computes
fresh in real time and records it for future backtests. A bias enters as a value
held until the next event (firing policy A).
**Forbids.** Any live external call *inside* a backtest replay.
**Why.** It is the only model compatible with reproducible backtests — LLM calls
are nondeterministic and far too slow per-cycle. Per `~/.claude/CLAUDE.md`,
external LLM (IONOS) calls happen only at the recording/live-source edge, with
explicit per-session consent, never inside a sim.

### C12 — The atomic sim unit and the four orchestration axes
**Guarantee.** The atomic unit is `(frozen topology + param-set + data-window +
RNG-seed) → deterministic run → metrics`. Parameters are typed, ranged, runtime
values injected at graph build (no recompile per param-set; the optimizer sees a
generic vector of typed ranges). Raw data is shared read-only across sims via
`Arc<[T]>` (data-server is built for this). Four axes orchestrate the atomic
unit: (1) param-sweep (grid/random), (2) optimization (argmax metric),
(3) walk-forward (rolling in-sample optimize + out-of-sample test),
(4) Monte-Carlo (N seeded realizations perturbing input). **MC = sweep over
seeds**; each realization is itself deterministic given its seed.
**Forbids.** Baking a specific search strategy (Bayesian/genetic) into the
primitive — those are pluggable policies atop the atomic unit; recompiling on a
param change.
**Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell"
(embarrassingly parallel across the unit) cleanly separated from search policy.
Seed-as-input reconciles Monte-Carlo with C1. The "frozen topology" of the
atomic unit is one harness instance, selected by the harness's **structural
axes** (C20); the structural experiment matrix is the outer orchestration over
this dimension, the tuning sweep the inner (C19/C20).

### C13 — Hot-reload is authoring-only; deploy is frozen
**Guarantee.** A node/strategy is authored as a native Rust `cdylib`,
hot-reloaded during the authoring loop (Rust-ABI; host and node built with the
same toolchain). The live/deploy bot is a statically-linked, versioned, frozen
artifact.
**Forbids.** Hot-swapping a running live bot; loading third-party / foreign-
toolchain plugins.
**Why.** Hot-reload makes the research loop fast; a live artifact must be frozen
and reproducible (audit trail: this bot = this commit). A sweep pays no
hot-reload tax — params are runtime data (C12), so the cdylib loads once.

### C14 — Headless core, two faces
**Guarantee.** The engine is a UI-agnostic library. Two faces sit on it: a
**programmatic/CLI** face (the primary surface for the LLM and automation —
author a node, run a sim/sweep, emit structured metrics) and a **visual** face
for human exploration. Visualization is only a downstream consumer node on the
streams.
**Forbids.** Any UI/pixel knowledge inside the engine.
**Why.** The LLM drives programmatically, the human visually; a headless core
serves both and makes the visual face freely deferrable. (Visual face leaning
egui-native, in-process zero-copy from the SoA columns — deferred decision, see
Open threads.)

### C15 — Resampling-as-node; sessions/calendars
**Guarantee.** A resampler is a node (finer stream → coarser bar stream),
clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars
and instrument specs are metadata (non-scalar, beside the hot path); session
*context* is exposed as scalar streams via a `SessionNode` (`bars_since_open:
i64`, `in_session: bool`, `session_open_ts: timestamp`). "3rd 15m candle after
session open" is then a plain node checking `bars_since_open == 3`.
**Forbids.** Streaming the calendar; special-casing session logic outside the
stream model.
**Why.** Keeps the line consistent — everything a signal needs arrives as a
stream; reference data feeds source/session nodes from beside the hot path.

### C16 — Engine / project separation; three-tier node reuse
**Guarantee.** aura is the reusable **engine**; each research project is a
separate external repo that depends on aura via cargo (the game-engine / game
split). Node reuse is cargo-native, in three tiers: **`aura-std`** (universal
blocks, ship with the engine) / **shared node crates** (cross-project-reusable,
their own repos, pulled as cargo git deps) / **project-local `nodes/`**
(experimental, project-specific). A reusable node is an `rlib` dependency; the
hot-reload unit stays the project-side `cdylib` that composes it (consistent
with C13). Concretely a project is a **Rust crate** — a cdylib library of node /
strategy / experiment blueprints — plus a static `Aura.toml` (project context:
data paths, instrument/pip metadata, default broker & window, runs dir). During
research the `aura` host loads and runs it (C13 hot-reload); for deploy the
chosen strategy + broker freeze into a standalone binary. **A project is
therefore always a Rust program built on the engine.**
**Forbids.** Project-specific signals in the aura repo (it keeps at most
example/fixture nodes under `examples/` for its own tests); a multi-project
manager inside aura; a bespoke node registry/marketplace (cargo + Gitea *is* the
package mechanism).
**Why.** The engine/game split keeps the engine sharp and reusable while each
project versions its own research with its own forward-queue. Promotion
(local → shared → std) is the ordinary Rust reuse gradient, no new mechanism.

### C17 — Authoring surface
**Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses** —
is authored in native Rust through **Claude Code + the skills pipeline**: the
human describes, Claude writes the Rust, builds it, runs it via the `aura` CLI,
and reports metrics. Declarative config (`Aura.toml`) carries only **static
project context** (data paths, instrument/pip metadata, defaults, runs dir),
never logic. aura ships **no embedded coding-LLM**. IONOS LLMs are used only as a
*runtime data source* (news-agent bias, C11), gated by per-session consent,
never in the code path.
**Forbids.** An in-app LLM chat that generates node code inside aura; using
IONOS (weaker models) as the authoring brain.
**Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's
failure; making weaker models the coding brain reintroduces the very problem.
Keeps aura's scope an engine + playground, not an LLM-IDE.

### C18 — Project management: one repo = one project, plus a run registry
**Guarantee.** Management has two planes. (1) **Code & forward-queue:** git
(commit = identity; the frozen bot *is* a commit) + Gitea (ideas/hypotheses as
the forward-queue, a research thrust = a milestone, the
`idea → experimental → validated → deployed` label gradient). (2) **Experiments
& results:** an Aura-native **run registry** — one record per run = a *manifest*
(node-commit + params + data-window + seed + broker profile) + *metrics*,
queryable, with *lineage* (composite ← signals; run ← inputs). Determinism
(C1/C12) makes a run reproducible from its tiny manifest, so the registry stores
manifests + metrics and re-derives full results on demand. Depth: **structured**
(promotion/status, lineage, run-diff).
**Forbids.** Storing results not reproducible from a recorded manifest;
duplicating git/Gitea inside aura; a multi-project workspace manager.
**Why.** Comparing experiments over time is the heart of the research loop and
has no home in git/Gitea; determinism makes a structured registry cheap.
Sequencing: the walking skeleton emits a manifest + metrics per run from day
one; the registry/index is a later milestone over manifests that already exist.

### C19 — Bootstrap: blueprint → instance (recursive)
**Guarantee.** Construction is a distinct phase, recursive at every level. Each
node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)`
sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic
graph-as-data produced by running a Rust builder (C9); it carries *free* numeric
params (declared ranges) and *free* input roles. The **bootstrap** binds
`(blueprint + param-set + data bindings + seed)` into a concrete, **frozen
instance** — buffers sized, topology fixed. This is precisely the "wiring /
graph build" that C7 ("sized at wiring", "topology frozen per sim") and C12
("params injected at graph build") already reference. The same machinery applies
recursively up to the harness (C20). A sweep builds many instances from one
blueprint; instances are disjoint (C1).
**Forbids.** Params that change topology (a topology change is a *different*
blueprint — Fork A, C7 "frozen"); resizing buffers after bootstrap; running a
sim against an un-bootstrapped blueprint.
**Why.** Separating the param-generic blueprint from the param-bound instance is
what makes one strategy reusable across a whole sweep and lets the optimizer
mutate "the 20" by *rebuilding* an instance (cheap; no recompile, C12) instead
of rewriting code. Naming the build phase makes the implicit "wiring" of C7/C12
explicit.

### C20 — Strategy ↔ harness; the harness is the root sim graph
**Guarantee.** A **strategy** is a reusable composite-node blueprint (C9):
broker-, data-, and viz-independent, with inputs declared as named **roles**
(symbol-agnostic where possible) and the position-event stream (C10) as output.
A **harness** (the experimental setup) is the **root sim graph** — sources bound
to the strategy's input roles + the strategy + attached broker node(s) + sinks —
and is itself produced by the bootstrap (C19). A harness *instance* is C1's
disjoint unit (RustAst's "root scope"). The harness has **two kinds of
parameterization**: **structural axes** (which strategy, which instrument(s),
which broker(s), which window) whose variation selects *different* instances —
the **experiment matrix**; and **tuning params** (the strategy's numeric params)
swept *within* a fixed structure (Fork A). The same strategy blueprint is reused
across backtest, sweep, visual workspaces, and the frozen live bot — each a
different harness. **Both strategy and harness/experiment are authored in Rust**
via builder APIs (C17); the experiment matrix is ordinary Rust control flow
(loops/conditionals), not a config schema.
**Forbids.** Embedding data sources / brokers / sinks inside a strategy; a
declarative experiment mini-DSL (logic is Rust — C17); treating the harness as
anything but a (root) graph.
**Why.** Reusability needs the strategy to be a context-free blueprint that many
harnesses embed. Modelling the harness as a root graph keeps it within the one
Node/graph abstraction (C9) and makes "10 strategies in one environment" and
"one strategy × N instruments" plain nested loops over the structural axes. Rust
authoring (not config) preserves full programmatic power — conditional/adaptive
matrices, generated axes, custom wiring — and avoids re-introducing the DSL trap
C17 rejects.

---

## Open architectural threads not yet resolved

- **Visual playground form** — leaning egui-native; deferred. The headless core
  (C14) makes deferring it free.
- **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies
  atop the atomic sim unit (C12), not yet designed.
- **`aura new` scaffolder, the experiment-builder API, and `Aura.toml`'s
  static-context schema** — `aura new` scaffolds a Rust project *crate* (node /
  strategy / experiment blueprints) against the engine (C16/C20); the
  experiment-builder API surface (harness wiring, structural axes, sweep
  combinators) and `Aura.toml`'s schema (data paths, instrument/pip metadata,
  default broker & window, runs dir) are not yet designed.
- **`aura-std` contents** — the crate exists (doc-only); which universal blocks
  land first follows the walking-skeleton's needs.
- **`strategies/` split** — a later split, *inside a project*, of top-level
  strategies from reusable building blocks in `nodes/`; not a day-1 cut.
- **Sequencing** — engine + CLI face first; walking-skeleton milestone before
  hot-reload, sweep, sessions, the run registry, and the visual playground.