diff --git a/CLAUDE.md b/CLAUDE.md index 8849a43..14886c1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -107,14 +107,19 @@ design decision, not a refactor, and belongs in the ledger. never hot-swapped (audit trail: this bot = this commit). 9. **Engine / project separation.** This repo is the reusable *engine*. Research projects are separate external repos that depend on it. Project-specific - signals live in a project's `nodes/`; cross-project-reusable nodes in shared - crates; universal blocks in `aura-std` (shipped here). Reuse is cargo-native; - the hot-reload unit is always the project-side `cdylib`. No user/project + *native* nodes live in the project's attached node crate; + cross-project-reusable nodes in shared crates; universal blocks in + `aura-std` (shipped here). Reuse is cargo-native. No user/project signals in this repo (only `examples/` fixtures for the engine's own tests); - no multi-project manager or node registry inside aura. **A project is always - a Rust crate** (a cdylib library of node / strategy / experiment blueprints + - a static `Aura.toml`), hosted by `aura` during research and frozen to a - standalone binary for deploy. + no multi-project manager or node registry inside aura. **A project is a + directory anchored by a static `Aura.toml`** — data-only by default + (blueprints + research documents over the std vocabulary, no build step), + hosted by `aura` during research and frozen to a standalone binary for + deploy. Native node logic lives in **node crates**: separate cdylib crates + referenced via `Aura.toml [nodes]`, scaffolded/attached by a dedicated verb + (the visible role-2 switch); the hot-reload unit is the referenced + node-crate cdylib. (#241 amendment, 2026-07-12 — re-opened deliberately by + the role model's diagnosis.) 10. **Authoring surface — all logic is Rust.** Node, strategy, and harness *logic* is authored in native Rust via Claude Code + the skills pipeline, using builder APIs. Declarative config (`Aura.toml`) carries only static diff --git a/crates/aura-cli/src/main.rs b/crates/aura-cli/src/main.rs index 17a400d..039a931 100644 --- a/crates/aura-cli/src/main.rs +++ b/crates/aura-cli/src/main.rs @@ -2489,7 +2489,7 @@ struct NodesCmd { #[derive(Subcommand)] enum NodesCommand { - /// Scaffold a node crate beside this project and attach it via [nodes]. + /// Scaffold a node crate beside this project and attach it via `[nodes]`. New(NodesNewCmd), } diff --git a/crates/aura-cli/src/project.rs b/crates/aura-cli/src/project.rs index 4bfca6c..cb36bba 100644 --- a/crates/aura-cli/src/project.rs +++ b/crates/aura-cli/src/project.rs @@ -475,8 +475,8 @@ fn newest_source_mtime(root: &Path) -> Option { newest } -/// Discover the project's tier and load it. Data-only (no [nodes], no root -/// crate) loads nothing; a [nodes] pointer or a root Cargo.toml routes into +/// Discover the project's tier and load it. Data-only (no `[nodes]`, no root +/// crate) loads nothing; a `[nodes]` pointer or a root Cargo.toml routes into /// the unchanged crate pipeline (`load_crate`). pub fn load(root: &Path, release: bool) -> Result { let toml = parse_aura_toml(root)?; diff --git a/docs/authoring-guide.md b/docs/authoring-guide.md index 5652711..a932b0e 100644 --- a/docs/authoring-guide.md +++ b/docs/authoring-guide.md @@ -35,13 +35,20 @@ the same shape every node in `aura-std` already follows. ### Where the code goes -- **A project's own signal** — the common case — lives in the project crate - `aura new` scaffolds (`docs/project-layout.md`, "Where reusable nodes - live"): one `src/lib.rs` (or a module under `src/`), registered under the - project's own `::` prefix. +- **The project's own signal doesn't need a crate by default.** `aura new` + scaffolds a **data-only** project (`docs/project-layout.md`, "A project + repo (two tiers)") — a strategy over the std vocabulary is a + blueprint/campaign document, not Rust: the scaffold ships a closed `signal.json` run target + and an open `signal_open.json` sweep target, both runnable immediately, no + build step. +- **A project-specific *native* node type** — the moment §0's three-part + pattern is actually needed — lives in an attached **node crate**: + `aura nodes new ` scaffolds a sibling cdylib crate (`Cargo.toml` + + `src/lib.rs`, registered under the project's own `::` prefix) + and appends its path to the project's `Aura.toml [nodes]` section. - **A block promoted to universal** — reused across projects and folded into the engine itself — lives in `crates/aura-std/src/`, unprefixed, and is - rostered in `crates/aura-std/src/vocabulary.rs` instead of a project's + rostered in `crates/aura-std/src/vocabulary.rs` instead of a node crate's `vocabulary()` function. The pattern below is identical either way; only the rostering call site differs (see "Rostering the type" below). @@ -92,9 +99,9 @@ Every node type — std or project-local — is three things: ### Worked example: `Scale`, a one-input, one-param node -This is the starter node `aura new` writes into every fresh project's -`src/lib.rs` (`__NS__` is the project's namespace) — copy-pasteable, and -already exercised end to end by the scaffold's own tests: +This is the starter node `aura nodes new` writes into every freshly attached +node crate's `src/lib.rs` (`__NS__` is the node crate's namespace) — +copy-pasteable, and already exercised end to end by the scaffold's own tests: ```rust use aura_core::{ diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index d8adcd0..1207e64 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -1318,6 +1318,15 @@ the one-method `MemberRunner` seam, so consumers (the `aura` CLI today; the play tests tomorrow) bind their own runners while the semantics live here once. It is explicitly NOT C21's World (a project-side program): it realizes one campaign document and owns no topology, no data sources, no UI. +**Realization (2026-07-12 — wiring-only tier, #241).** The smallest project +is now data-only: `Aura.toml` + `blueprints/` + `runs/`, no crate. The load +boundary tier-selects (a `[nodes]` pointer list in `Aura.toml` → load that +crate; a root `Cargo.toml` → the pre-#241 native project, unchanged; neither +→ std-vocabulary-only). `aura new` scaffolds the data-only tier; +`aura nodes new` scaffolds a node crate **beside** the project and attaches +it — the visible role-2 switch (C25 role model). Provenance widened +additively: a data-only run stamps commit-only. Invariant 9's "a project is +always a Rust crate" was deliberately amended (user decision, 2026-07-12). ### C17 — Authoring surface **Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses** — diff --git a/docs/glossary.md b/docs/glossary.md index 4f2820b..d909286 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -17,7 +17,7 @@ The primitive `(frozen topology + param-set + data-window + RNG-seed) → determ ### Aura.toml **Avoid:** — -The per-project declarative config holding only static context — paths only: `[paths] data` (the archive root; defaults to the built-in path when unset) and `[paths] runs` (the runs dir) — never logic, and never instrument geometry (that is the recorded sidecar, C15). Its presence marks the project root, the way `Cargo.toml` marks a cargo crate. +The per-project declarative config holding only static context — paths only: `[paths] data` (the archive root; defaults to the built-in path when unset) and `[paths] runs` (the runs dir) — never logic, and never instrument geometry (that is the recorded sidecar, C15). Its presence marks the project root, the way `Cargo.toml` marks a cargo crate. An optional `[nodes]` section (a `crates` pointer list, one entry honored) tier-selects a `node crate` in; absent, the project stays `data-only project`. ### backtest **Avoid:** — @@ -83,6 +83,10 @@ The validation read that grades how consistently one *brought* candidate holds a **Avoid:** — One data-driven clock step: one input record = one cycle, advanced in global timestamp order with a monotonic `cycle_id` — the time-axis clock step (mainstream backtesters call it a tick/event/bar), **not** the topology-axis directed loop that C5's DAG forbids. (In the pipeline-process sense "cycle" also names one milestone round; the engine sense is this clock step.) +### data-only project +**Avoid:** — +The default project tier (#241): a directory anchored by `Aura.toml` with blueprints and research documents over the std vocabulary — no crate, no build step. Runs, sweeps, and campaigns work immediately after `aura new`. + ### deflated score **Avoid:** trials-adjusted score A sweep winner's metric penalised for the number of configurations tried (trials-deflation — generalising the **Deflated Sharpe Ratio** deflation *concept*, Bailey & López de Prado 2014, not the analytic DSR formula), recorded additively on the winning member's `selection` without changing which member won. Paired with an **overfit probability**; the across-trials defence against a winner that is luck at sweep scale. @@ -155,7 +159,7 @@ A concrete, frozen graph produced by binding a blueprint to params + data + seed ### manifest **Avoid:** — -The reproducible metadata record of a run (node-commit + params + data-window + seed + broker profile + `topology hash`, plus — for a project-hosted run — the `project` provenance block `{namespace, dylib_sha256, commit}`), paired with metrics in the run registry — the run-output record, *not* the Cargo.toml/`Aura.toml`-style static *config* manifest. Determinism lets the full result be re-derived from this tiny record on demand. +The reproducible metadata record of a run (node-commit + params + data-window + seed + broker profile + `topology hash`, plus — for a project-hosted run — the `project` provenance block `{namespace, dylib_sha256, commit}`, with `namespace`/`dylib_sha256` present only when a `node crate` is attached — a `data-only project` run stamps `commit` alone), paired with metrics in the run registry — the run-output record, *not* the Cargo.toml/`Aura.toml`-style static *config* manifest. Determinism lets the full result be re-derived from this tiny record on demand. ### Monte-Carlo **Avoid:** MC @@ -177,6 +181,10 @@ The headline research artifact: the R-equity curve with the `cost model`'s cost- **Avoid:** block The universal composable dataflow unit, implementing `lookbacks()` + `eval(ctx)` — a producer, a pure consumer (sink), or both — with at most one output port; a producer's output is a **record of 1..K base-scalar columns** (a scalar is the degenerate K=1 case). Everything that plugs into the engine is fractally a node. +### node crate +**Avoid:** — +A separate cdylib crate holding a project's native node logic, attached via `Aura.toml [nodes]` (scaffolded by `aura nodes new`). Depends only on aura-core (no data-plane coupling); its namespace prefixes every type id it contributes; the manifest stamps its dylib sha alongside the project commit. + ### overfit probability **Avoid:** P(overfit), luck probability, Probability of Backtest Overfitting / PBO, CSCV The recorded chance that a deflated sweep winner is noise rather than edge — a rank of the winner's metric against a centred no-edge null over the family (a moving-block reality-check bootstrap; **not** CSCV-PBO — there is no in-sample/out-of-sample split). Paired with the **deflated score** on the winner's `selection`. diff --git a/docs/project-layout.md b/docs/project-layout.md index 5a3a880..64aa842 100644 --- a/docs/project-layout.md +++ b/docs/project-layout.md @@ -15,53 +15,80 @@ from the things built with it: ships at most `examples/` fixture nodes for its own tests — never your real signals. - **A research project** is its **own external repo** (e.g. `~/dev/ger40-lab/`), - with its own git history, its own Gitea repo (its own forward-queue), and a - cargo dependency on aura. **It is itself a Rust crate** (a `cdylib` library of - node / strategy / experiment blueprints); the `aura` host loads and runs it - during research (hot-reload), and freezes a chosen strategy + broker into a - standalone binary for deploy. This is where you and Claude author signals *and - experiments* — all in Rust — and where the runs live. (Contracts **C16**, **C20**.) + with its own git history and its own Gitea repo (its own forward-queue), + anchored by a static `Aura.toml`. Its default tier is **data-only** — + blueprints, process, and campaign documents over the std vocabulary, no + crate, no build step; the `aura` host loads and runs it during research as + is. The moment the project needs native node logic (role-2 work), it + attaches a **node crate** (a `cdylib`, scaffolded by `aura nodes new`) via + `Aura.toml [nodes]`; the `aura` host then hot-reloads that crate too, and + freezing a chosen strategy + broker still yields one standalone binary + spanning both halves for deploy. This is where you and Claude author + signals *and experiments*, and where the runs live. (Contracts **C16**, + **C20**, **#241**.) The `aura` CLI is a tool you run *inside* a project directory — it finds the project root by walking up to an `Aura.toml`, the way `cargo` finds `Cargo.toml`. -A project is always a Rust program built on the engine. +A project is a directory anchored by `Aura.toml`; native node logic never +lives in the project directory itself, only in an attached node crate (the +two tiers below). -## A project repo +## A project repo (two tiers) + +The default project is **data-only** — no crate, no build step. `aura new` +scaffolds exactly this: ``` -ger40-lab/ # your research project — a separate Rust crate (cdylib) -├── Aura.toml # STATIC context only, paths only: `[paths] runs` -│ # (the runs dir) and `[paths] data` (the archive -│ # root; defaults to the built-in Pepperstone path -│ # when unset); no logic — instrument -│ # geometry is the recorded sidecar (C15), never -│ # authored here -├── Cargo.toml # cdylib; depends on aura-core/aura-std (+ shared node crates); -│ # carries an empty [workspace] table (see note below) +ger40-lab/ # your research project — a plain directory, no crate +├── Aura.toml # STATIC context only: `[paths] runs` (the runs dir) +│ # and `[paths] data` (the archive root; defaults to +│ # the built-in Pepperstone path when unset); no +│ # logic — instrument geometry is the recorded +│ # sidecar (C15), never authored here. A `[nodes]` +│ # section, absent here, is what tier-selects a +│ # node crate in (see below). ├── CLAUDE.md # the project's own skills wiring (authoring discipline) -├── .claude/dev-cycle-profile.yml -├── nodes/ # project-local node & strategy blueprints (Rust) -│ └── third-candle-long/ -├── experiments/ # experiment/harness definitions (Rust): matrices, sweeps -├── runs/ # the run registry: manifest + metrics per run -└── (frozen bots, …) +├── blueprints/ # op-scripts / campaign / process documents (data, no build) +└── runs/ # the run registry: manifest + metrics per run ``` -> **The empty `[workspace]` table.** A project crate is its own cargo workspace, -> never a member of the engine's. This matters the moment a project lives *under* -> the engine repo (common while authoring, before it moves to its own repo): -> cargo otherwise walks up, finds the engine's root manifest, and refuses to -> build — +Runs, sweeps, and campaigns over the **std vocabulary** work immediately — +nothing to compile. The moment a project needs a *native* node type (role-2 +work), `aura nodes new ` scaffolds a sibling **node crate** and attaches +it via an `Aura.toml [nodes]` pointer: + +``` +ger40-lab/ # unchanged: still Aura.toml + blueprints/ + runs/ +├── Aura.toml # now carries `[nodes] crates = ["../ger40-lab-nodes"]` +└── … + +ger40-lab-nodes/ # a SIBLING directory — its own cdylib crate +├── Cargo.toml # cdylib; depends on aura-core (+ shared node crates); +│ # carries an empty [workspace] table (see note below) +├── CLAUDE.md # the node crate's own skills wiring +└── src/ + └── lib.rs # the project's native node types, `::`-prefixed +``` + +The `aura` host loads the pointed-at node crate during research (hot-reload); +freezing a chosen strategy + broker still yields one standalone binary for +deploy, spanning both halves. + +> **The empty `[workspace]` table (node-crate tier only).** A node crate is +> its own cargo workspace, never a member of the engine's. This matters the +> moment a node crate lives *under* the engine repo (common while authoring, +> before it moves to its own repo): cargo otherwise walks up, finds the +> engine's root manifest, and refuses to build — > > ``` > error: current package believes it's in a workspace when it's not: > ... to keep it out of the workspace, add an empty [workspace] table ... > ``` > -> The fix is cargo's own hint: an empty `[workspace]` table in the project's -> `Cargo.toml`. It pins the crate as a standalone workspace root and costs -> nothing when the project later sits in its own repo. The `aura new` -> scaffolder emits this line (cycle 0103); a hand-rolled project adds it +> The fix is cargo's own hint: an empty `[workspace]` table in the node +> crate's `Cargo.toml`. It pins the crate as a standalone workspace root and +> costs nothing when the node crate later sits in its own repo. The `aura +> nodes new` scaffolder emits this line; a hand-rolled node crate adds it > itself. ## Where reusable nodes live (three tiers) @@ -73,7 +100,9 @@ cargo, layered by maturity (contract C16): resamplers, the `SessionNode`, standard combinators, broker profiles). 2. **Shared node crates** — cross-project-reusable blocks in their own repos (e.g. `Brummel/aura-nodes-fx`), pulled in as cargo git dependencies. -3. **Project-local `nodes/`** — experimental, project-specific blocks. +3. **The project's own node crate** — experimental, project-specific blocks, + attached via `Aura.toml [nodes]` (scaffolded by `aura nodes new`; see the + two tiers above). Promotion is the ordinary Rust gradient: a node proves itself project-local → gets extracted into a shared crate → if it turns universal, into `aura-std`. @@ -110,10 +139,13 @@ command sequences. 1. **You** (in Claude Code, inside `ger40-lab/`): "I suspect the 3rd 15m candle after GER40 open is long-biased when the first two close bullish — build it as a signal." -2. **Claude** (via the skills pipeline) writes `nodes/third-candle-long/`, - implements `schema` + `eval` against `aura-core`. -3. **Backtest:** `aura backtest nodes/third-candle-long --symbol GER40 - --from 2020 --to 2024` → the strategy produces a broker-independent, unsized +2. **Claude** (via the skills pipeline) attaches the project's node crate — + once, `aura nodes new ger40-lab-nodes` — and implements the + `ThirdCandleLong` node's `schema` + `eval` in it, against `aura-core`. +3. **Backtest:** an op-script wiring `ger40_lab_nodes::ThirdCandleLong` (§1, + `docs/authoring-guide.md`) is built into `blueprints/third-candle-long.json` + (`aura graph build`); `aura run blueprints/third-candle-long.json --real + GER40 --from 2020 --to 2024` → the strategy produces a broker-independent, unsized **bias stream** (one signed, bounded `f64 ∈ [-1,+1]` per cycle — sign = direction, magnitude = optional conviction). A downstream **risk-based executor** (stop-rule → position-management, in **R**, the protective stop defining 1R) turns the bias into @@ -126,18 +158,20 @@ command sequences. and a currency curve are a later **live/deploy-edge** concern (the only reliable ground truth, measured forward — never an authored historical "realistic broker"); the legacy `SimBroker` pip yardstick survives only as an optional dual readout, not the model. -4. **Sweep / Monte-Carlo / matrix — a Rust experiment.** Anything beyond a - single backtest is an *experiment* in `experiments/` (Rust, builder API): a - parameter sweep, Monte-Carlo over seeds, or a structural matrix like "these 10 - strategies × these 3 instruments × {fixed-stop, vol-stop} risk-executors". The - matrix is plain Rust loops, not a config schema (C20). `aura run experiments/compare` - bootstraps the matrix, fans the disjoint sims over all cores (C1), and writes - the comparable runs to `runs/`. +4. **Sweep / Monte-Carlo / matrix — a campaign document.** Anything beyond a + single backtest is experiment *intent* named in a **campaign document** + (data, not Rust, §3 `docs/authoring-guide.md`): a parameter sweep, + Monte-Carlo over seeds, or a structural matrix like "these 10 strategies × + these 3 instruments × {fixed-stop, vol-stop} risk-executors" — instruments + × windows × strategy × param axes × process, headless-authorable and + registered under `blueprints/`. `aura campaign run ` + bootstraps the matrix, fans the disjoint sims over all cores (C1), and + writes the comparable runs to `runs/`. 5. **Compose:** "combine it with `momentum-filter` as a weighted sum" → Claude writes a composite node (fractal, C9). 6. **Walk-forward:** another experiment kind (rolling in-sample optimize + out-of-sample test) → an out-of-sample verdict. -7. **Freeze:** `aura freeze nodes/strategy-y --out bots/strategy-y` → a standalone, +7. **Freeze:** `aura freeze blueprints/strategy-y.json --out bots/strategy-y` → a standalone, statically-linked bot (C13); the live broker connection (e.g. cTrader Open API) is bound at this **deploy edge** — the only place account money appears, measured forward against a real venue, never an authored historical broker.