From 9eae43d30842f816fb2984d0deb9ebc8781a7f68 Mon Sep 17 00:00:00 2001 From: Brummel Date: Wed, 3 Jun 2026 09:54:44 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20C19/C20=20=E2=80=94=20bootstrap=20+=20s?= =?UTF-8?q?trategy/harness;=20a=20project=20is=20a=20Rust=20program?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Anchor the implementation-design branch: C19 (construction is a bootstrap phase — param-generic blueprint -> frozen instance; recursive up to the harness; params size/configure but never change topology). C20 (strategy = reusable context-free composite blueprint with role inputs + position-event output; harness = the root sim graph and C1's disjoint unit, with structural axes = experiment matrix vs tuning params = sweep; both strategy AND experiment authored in Rust via builder APIs, not a config DSL). Extend C8 (schema declares tunable params+ranges), C16 (a project is a Rust crate: cdylib of node/strategy/experiment blueprints + static Aura.toml; hosted by aura during research, frozen to a binary for deploy), C17 (all logic is Rust — nodes/strategies/experiments; Aura.toml = static context only), C12 (frozen topology = a harness instance; structural matrix is the outer axis). Add CLAUDE.md invariant 11; refresh project-layout.md (experiments/ dir, Rust experiments, day-in-the-life). Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 23 +++++++--- crates/aura-core/src/lib.rs | 3 +- docs/design/INDEX.md | 87 ++++++++++++++++++++++++++++++++----- docs/project-layout.md | 41 +++++++++++------ 4 files changed, 122 insertions(+), 32 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index c736094..e00d5b7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -92,8 +92,21 @@ design decision, not a refactor, and belongs in the ledger. crates; universal blocks in `aura-std` (shipped here). Reuse is cargo-native; the hot-reload unit is always the project-side `cdylib`. No user/project signals in this repo (only `examples/` fixtures for the engine's own tests); - no multi-project manager or node registry inside aura. -10. **Authoring surface.** Nodes are authored in native Rust via Claude Code + - the skills pipeline. aura ships no embedded coding-LLM. IONOS LLMs are used - only as a runtime data source (news bias), with per-session consent, never - in the code path. + no multi-project manager or node registry inside aura. **A project is always + a Rust crate** (a cdylib library of node / strategy / experiment blueprints + + a static `Aura.toml`), hosted by `aura` during research and frozen to a + standalone binary for deploy. +10. **Authoring surface — all logic is Rust.** Nodes, strategies, *and* + experiments/harnesses are authored in native Rust via Claude Code + the + skills pipeline, using builder APIs. Declarative config (`Aura.toml`) carries + only static project context, never logic — no experiment/strategy DSL (the + RustAst trap). aura ships no embedded coding-LLM; IONOS LLMs are used only as + a runtime data source (news bias), with per-session consent, never in the + code path. +11. **Construction is a bootstrap phase.** Blueprints (param-generic graph-as- + data from a Rust builder) are bootstrapped into frozen instances (buffers + sized, topology fixed) by binding params + data + seed. Params configure and + size nodes but never change topology (a topology change is a different + blueprint). The harness — sources + strategy + broker node(s) + sinks — is + the root sim graph, itself bootstrapped, and is C1's disjoint unit; its + structural axes form the experiment matrix, its tuning params the sweep. diff --git a/crates/aura-core/src/lib.rs b/crates/aura-core/src/lib.rs index 21de293..c6968c6 100644 --- a/crates/aura-core/src/lib.rs +++ b/crates/aura-core/src/lib.rs @@ -5,7 +5,8 @@ //! small and stable. It is the home of (to be specified): //! //! - the scalar base types streamed as SoA: `i64`, `f64`, `bool`, `timestamp`; -//! - the `Node` trait — the contract an authored node implements (`schema` + +//! - the `Node` trait — the contract an authored node implements (`schema` — +//! inputs, lookback, firing group, and tunable params with ranges — plus //! `eval`), with engine-provided read-only input windows and node-internal //! series; //! - the evaluation context `Ctx` (zero-copy indexed access into input columns, diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index 9ae0c21..554588e 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -115,7 +115,9 @@ Type-erasure at the edge is also forced by the cdylib boundary (C13). ### C8 — The node contract **Guarantee.** A node implements `schema()` (declares each input's scalar type, -required lookback depth, and firing group) + `eval(ctx) -> Option`. The +required lookback depth, and firing group, **and the node's own tunable +parameters — typed, with ranges**, which aggregate into the blueprint's +param-space the optimizer sweeps, C12/C19/C20) + `eval(ctx) -> Option`. The engine provides read-only, zero-copy windows into each input's SoA ring buffer (`ctx.f64_in(x)[k]`, sized at wiring); a node may *additionally* keep its own mutable series for derived/intermediate state. `None`/Void return = filter / @@ -216,7 +218,10 @@ primitive — those are pluggable policies atop the atomic unit; recompiling on param change. **Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell" (embarrassingly parallel across the unit) cleanly separated from search policy. -Seed-as-input reconciles Monte-Carlo with C1. +Seed-as-input reconciles Monte-Carlo with C1. The "frozen topology" of the +atomic unit is one harness instance, selected by the harness's **structural +axes** (C20); the structural experiment matrix is the outer orchestration over +this dimension, the tuning sweep the inner (C19/C20). ### C13 — Hot-reload is authoring-only; deploy is frozen **Guarantee.** A node/strategy is authored as a native Rust `cdylib`, @@ -261,7 +266,12 @@ blocks, ship with the engine) / **shared node crates** (cross-project-reusable, their own repos, pulled as cargo git deps) / **project-local `nodes/`** (experimental, project-specific). A reusable node is an `rlib` dependency; the hot-reload unit stays the project-side `cdylib` that composes it (consistent -with C13). +with C13). Concretely a project is a **Rust crate** — a cdylib library of node / +strategy / experiment blueprints — plus a static `Aura.toml` (project context: +data paths, instrument/pip metadata, default broker & window, runs dir). During +research the `aura` host loads and runs it (C13 hot-reload); for deploy the +chosen strategy + broker freeze into a standalone binary. **A project is +therefore always a Rust program built on the engine.** **Forbids.** Project-specific signals in the aura repo (it keeps at most example/fixture nodes under `examples/` for its own tests); a multi-project manager inside aura; a bespoke node registry/marketplace (cargo + Gitea *is* the @@ -271,11 +281,14 @@ project versions its own research with its own forward-queue. Promotion (local → shared → std) is the ordinary Rust reuse gradient, no new mechanism. ### C17 — Authoring surface -**Guarantee.** Nodes are authored in native Rust through **Claude Code + the -skills pipeline**: the human describes, Claude writes the node crate, builds it, -runs it via the `aura` CLI, and reports metrics. aura ships **no embedded -coding-LLM**. IONOS LLMs are used only as a *runtime data source* (news-agent -bias, C11), gated by per-session consent, never in the code path. +**Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses** — +is authored in native Rust through **Claude Code + the skills pipeline**: the +human describes, Claude writes the Rust, builds it, runs it via the `aura` CLI, +and reports metrics. Declarative config (`Aura.toml`) carries only **static +project context** (data paths, instrument/pip metadata, defaults, runs dir), +never logic. aura ships **no embedded coding-LLM**. IONOS LLMs are used only as a +*runtime data source* (news-agent bias, C11), gated by per-session consent, +never in the code path. **Forbids.** An in-app LLM chat that generates node code inside aura; using IONOS (weaker models) as the authoring brain. **Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's @@ -300,6 +313,54 @@ has no home in git/Gitea; determinism makes a structured registry cheap. Sequencing: the walking skeleton emits a manifest + metrics per run from day one; the registry/index is a later milestone over manifests that already exist. +### C19 — Bootstrap: blueprint → instance (recursive) +**Guarantee.** Construction is a distinct phase, recursive at every level. Each +node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)` +sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic +graph-as-data produced by running a Rust builder (C9); it carries *free* numeric +params (declared ranges) and *free* input roles. The **bootstrap** binds +`(blueprint + param-set + data bindings + seed)` into a concrete, **frozen +instance** — buffers sized, topology fixed. This is precisely the "wiring / +graph build" that C7 ("sized at wiring", "topology frozen per sim") and C12 +("params injected at graph build") already reference. The same machinery applies +recursively up to the harness (C20). A sweep builds many instances from one +blueprint; instances are disjoint (C1). +**Forbids.** Params that change topology (a topology change is a *different* +blueprint — Fork A, C7 "frozen"); resizing buffers after bootstrap; running a +sim against an un-bootstrapped blueprint. +**Why.** Separating the param-generic blueprint from the param-bound instance is +what makes one strategy reusable across a whole sweep and lets the optimizer +mutate "the 20" by *rebuilding* an instance (cheap; no recompile, C12) instead +of rewriting code. Naming the build phase makes the implicit "wiring" of C7/C12 +explicit. + +### C20 — Strategy ↔ harness; the harness is the root sim graph +**Guarantee.** A **strategy** is a reusable composite-node blueprint (C9): +broker-, data-, and viz-independent, with inputs declared as named **roles** +(symbol-agnostic where possible) and the position-event stream (C10) as output. +A **harness** (the experimental setup) is the **root sim graph** — sources bound +to the strategy's input roles + the strategy + attached broker node(s) + sinks — +and is itself produced by the bootstrap (C19). A harness *instance* is C1's +disjoint unit (RustAst's "root scope"). The harness has **two kinds of +parameterization**: **structural axes** (which strategy, which instrument(s), +which broker(s), which window) whose variation selects *different* instances — +the **experiment matrix**; and **tuning params** (the strategy's numeric params) +swept *within* a fixed structure (Fork A). The same strategy blueprint is reused +across backtest, sweep, visual workspaces, and the frozen live bot — each a +different harness. **Both strategy and harness/experiment are authored in Rust** +via builder APIs (C17); the experiment matrix is ordinary Rust control flow +(loops/conditionals), not a config schema. +**Forbids.** Embedding data sources / brokers / sinks inside a strategy; a +declarative experiment mini-DSL (logic is Rust — C17); treating the harness as +anything but a (root) graph. +**Why.** Reusability needs the strategy to be a context-free blueprint that many +harnesses embed. Modelling the harness as a root graph keeps it within the one +Node/graph abstraction (C9) and makes "10 strategies in one environment" and +"one strategy × N instruments" plain nested loops over the structural axes. Rust +authoring (not config) preserves full programmatic power — conditional/adaptive +matrices, generated axes, custom wiring — and avoids re-introducing the DSL trap +C17 rejects. + --- ## Open architectural threads not yet resolved @@ -308,10 +369,12 @@ one; the registry/index is a later milestone over manifests that already exist. (C14) makes deferring it free. - **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies atop the atomic sim unit (C12), not yet designed. -- **`aura new` scaffolder + `Aura.toml` schema** — the project-config surface - (symbols in scope, default data-window, broker profile, runs dir) and the - command that scaffolds a project repo against the engine (C16/C18); not yet - designed. +- **`aura new` scaffolder, the experiment-builder API, and `Aura.toml`'s + static-context schema** — `aura new` scaffolds a Rust project *crate* (node / + strategy / experiment blueprints) against the engine (C16/C20); the + experiment-builder API surface (harness wiring, structural axes, sweep + combinators) and `Aura.toml`'s schema (data paths, instrument/pip metadata, + default broker & window, runs dir) are not yet designed. - **`aura-std` contents** — the crate exists (doc-only); which universal blocks land first follows the walking-skeleton's needs. - **`strategies/` split** — a later split, *inside a project*, of top-level diff --git a/docs/project-layout.md b/docs/project-layout.md index 4ca9ee3..6697be8 100644 --- a/docs/project-layout.md +++ b/docs/project-layout.md @@ -16,25 +16,30 @@ from the things built with it: signals. - **A research project** is its **own external repo** (e.g. `~/dev/ger40-lab/`), with its own git history, its own Gitea repo (its own forward-queue), and a - cargo dependency on aura. This is where you and Claude author signals and where - the runs live. (See contract **C16**.) + cargo dependency on aura. **It is itself a Rust crate** (a `cdylib` library of + node / strategy / experiment blueprints); the `aura` host loads and runs it + during research (hot-reload), and freezes a chosen strategy + broker into a + standalone binary for deploy. This is where you and Claude author signals *and + experiments* — all in Rust — and where the runs live. (Contracts **C16**, **C20**.) The `aura` CLI is a tool you run *inside* a project directory — it finds the project root by walking up to an `Aura.toml`, the way `cargo` finds `Cargo.toml`. +A project is always a Rust program built on the engine. ## A project repo ``` -ger40-lab/ # your research project — a separate repo -├── Aura.toml # project config: symbols in scope, default data -│ # window, broker profile, runs dir (schema TBD) -├── Cargo.toml # depends on aura-core (+ shared node crates) +ger40-lab/ # your research project — a separate Rust crate (cdylib) +├── Aura.toml # STATIC context only: data paths, instrument/pip +│ # metadata, default broker & window, runs dir (no logic) +├── Cargo.toml # cdylib; depends on aura-core/aura-std (+ shared node crates) ├── CLAUDE.md # the project's own skills wiring (authoring discipline) ├── .claude/dev-cycle-profile.yml -├── nodes/ # project-local, experimental nodes (cdylib) +├── nodes/ # project-local node & strategy blueprints (Rust) │ └── third-candle-long/ +├── experiments/ # experiment/harness definitions (Rust): matrices, sweeps ├── runs/ # the run registry: manifest + metrics per run -└── (frozen bots, results, …) +└── (frozen bots, …) ``` ## Where reusable nodes live (three tiers) @@ -57,9 +62,11 @@ the dependent. ## Authoring happens in Claude Code (contract C17) aura has no built-in coding-LLM. You author by talking to Claude Code, which -writes the node crate, builds it, runs it, and reports back. IONOS LLMs appear -only as a *runtime data source* (e.g. a news-agent node emitting a bias), -recorded before it enters a backtest, and only with your per-session consent. +writes the Rust — nodes, strategies, *and experiments* — builds it, runs it, and +reports back. Declarative config (`Aura.toml`) holds only static context, never +logic. IONOS LLMs appear only as a *runtime data source* (e.g. a news-agent node +emitting a bias), recorded before it enters a backtest, and only with your +per-session consent. ## A day in the life @@ -77,11 +84,17 @@ recorded before it enters a backtest, and only with your per-session consent. attached at once: add `--broker pepperstone` to get a realistic currency curve *alongside* the default sim-optimal pip curve — two comparable equity curves from the same position table. (Contract C10.) -4. **Sweep + Monte-Carlo:** `aura sweep … --grid lookback=5..50,thresh=0.5..0.9 - --mc-seeds 1000` → a distribution of metrics, best params, robustness bands. +4. **Sweep / Monte-Carlo / matrix — a Rust experiment.** Anything beyond a + single backtest is an *experiment* in `experiments/` (Rust, builder API): a + parameter sweep, Monte-Carlo over seeds, or a structural matrix like "these 10 + strategies × these 3 instruments × {sim-optimal, pepperstone}". The matrix is + plain Rust loops, not a config schema (C20). `aura run experiments/compare` + bootstraps the matrix, fans the disjoint sims over all cores (C1), and writes + the comparable runs to `runs/`. 5. **Compose:** "combine it with `momentum-filter` as a weighted sum" → Claude writes a composite node (fractal, C9). -6. **Walk-forward:** `aura walkforward …` → an out-of-sample verdict. +6. **Walk-forward:** another experiment kind (rolling in-sample optimize + + out-of-sample test) → an out-of-sample verdict. 7. **Freeze:** `aura freeze nodes/strategy-y --broker pepperstone --out bots/strategy-y` → a standalone, statically-linked bot (C13). 8. **(Later) Look:** `aura play` opens the egui playground — equity curve, trade