From f80571a46abb78a67220955ae91c9d65373b05d2 Mon Sep 17 00:00:00 2001 From: Brummel Date: Mon, 29 Jun 2026 12:33:38 +0200 Subject: [PATCH] =?UTF-8?q?docs(ledger):=20C24=20=E2=80=94=20blueprint=20a?= =?UTF-8?q?s=20serializable=20World-owned=20data;=20resolve=20#109?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Settle the #109 fork as topology-as-data (game-engine principle, C16): node logic stays Rust (C17), topology becomes a serializable value the World owns, generates, structurally searches, and reproduces. New contract C24 + refinements to C9/C17/C18/C20/C21/C22; threads: #109 resolved -> C24. Format itself unbuilt (next brainstorm/milestone). refs #109 --- docs/design/INDEX.md | 143 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 135 insertions(+), 8 deletions(-) diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index ade8d2d..3e0d069 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -13,7 +13,11 @@ C10 was reframed in cycle 0007 to an exposure stream, then again 2026-06-23 derived layer — see C10). C19–C22 were added as the construction / World / playground layer; C23 and the C9/C19 compilation refinements were settled 2026-06-05 for the **Construction-layer** milestone (the -blueprint→flat-graph reading of composites and graph optimisation). +blueprint→flat-graph reading of composites and graph optimisation). C24 (the +blueprint as a serializable, World-owned data value) was settled 2026-06-29, +resolving the long-deferred #109 fork in favour of **topology-as-data** (the +game-engine principle, C16): the engine owns topology as content it serializes / +loads / generates, not as Rust source baked into the binary. The `CLAUDE.md` **Domain invariants** section is the compressed, always-loaded summary of the subset that agents must never violate; this file is the fuller form with @@ -476,6 +480,13 @@ That breadth guarded **render identity**, dead since the renderer was retired in distinguishability check, if ever wanted (e.g. for the WASM graph view's #21 thread), is decoupled from param-space injectivity. Construction-phase only; the flat graph stays name-free (C23). +**Refinement (2026-06-29 — graph-as-data round-trips both ways, C24).** "The built +graph is introspectable runtime data" is now contract-level **bidirectional**: a +blueprint is not only *emitted* as data (`model_to_json`, the render half) but is +itself a **serializable data value with a load path** (data → blueprint → +`FlatGraph`), so topology is a value the World generates, stores, and reproduces — +see C24. The Rust builder API stays the primary *human / LLM* authoring surface +(C17); the data form is the *machine* surface the World owns. ### C10 — Strategy output is a bias stream; signal quality is measured in R; cost is a composable downstream graph (gross R → net R); money is decoupled to the live deploy edge **Guarantee.** A strategy's primary, backtestable output is **not** an equity @@ -1180,6 +1191,20 @@ IONOS (weaker models) as the authoring brain. **Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's failure; making weaker models the coding brain reintroduces the very problem. Keeps aura's scope an engine + playground, not an LLM-IDE. +**Refinement (2026-06-29, #109 resolved — node *logic* is Rust; *topology* is data, +C24).** "All logic … including experiments/harnesses … is Rust" governs +**computation** — a node's math, a meta-program's control flow — and that stays +native Rust (the RustAst lesson, sharpened: a small LLM cannot reliably *apply* a +computational DSL, and a strong one does not need one). It does **not** bind +**topology** — which node feeds which, the structural axes, the param-space — to +Rust *source*. Topology is a **serializable data value** the World owns (C24): a +non-Turing static DAG over the closed, compiled-in node vocabulary (C8/C16), +carrying no computation. This is *not* the RustAst trap re-opened: that trap is a +DSL the author must *apply*; a topology-data blueprint is machine-generated / owned +and applied by no one. The no-DSL guard therefore stands unbroken — it forbids a +*computational* experiment / strategy language, not a static topology-data format. +The experiment-matrix *generator* may stay Rust (C20); its *output* is +topology-data. ### C18 — Project management: one repo = one project, plus a run registry **Guarantee.** Management has two planes. (1) **Code & forward-queue:** git @@ -1251,6 +1276,17 @@ distinct from the retired `list`/`rank` exit-2, which is argv-shape rejection `no such family ` exit (typo-safety) is an available future UX choice, not a current contract. +**Refinement (2026-06-29 — a generated run's topology lives in / is addressed by +its manifest, C24).** The manifest identifies a run's topology today only via +`commit` (this engine + a hand-coded harness) — sufficient while harnesses are a +finite hand-coded menu. Once the World **generates or structurally searches** +topologies (C21/C24), `commit` no longer identifies the graph, so the manifest must +**carry or content-address the topology-data** to stay the re-derivation recipe +(C18's "reproducible from a recorded manifest"). This pulls the previously-deferred +**content-addressed identity** non-goal forward as the natural home for a generated +topology's identity. The format and the carrier are C24's design; recorded here as +the reproduction requirement it must meet. + ### C19 — Bootstrap: blueprint → instance (recursive) **Guarantee.** Construction is a distinct phase, recursive at every level. Each node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)` @@ -1441,6 +1477,16 @@ matrix axis (C12: a different period is a *different strategy*, not a sweep point), never a `param_space` entry, so a sweep cannot desync the two clocks. This is the concrete instance of the structural-axis-vs-tuning-param split above (and `delay.lag`, a C8 structural constant, is bound out of the space). +**Refinement (2026-06-29 — experiment-matrix *members* are topology-data, C24).** +"The experiment matrix is ordinary Rust control flow, not a config schema" holds for +the **generator** — the loop / conditional that *enumerates* structural variants may +stay Rust (C17). But each **member** it yields is a **topology-data value** (C24), +not a hand-coded blueprint nor engine-baked source: a structural axis selects among +*data* blueprints the World constructs, mutates, and serializes. This is what makes +structural variation **first-class and searchable** rather than a hand-enumerated +menu — `HarnessKind` and the per-strategy `*_sweep_family` functions in `aura-cli` +are the pre-C24 scaffolding (topology-as-engine-source, a C16 tension), retired as +C24 + the project-as-crate layer land. ### C21 — The World: the meta-level is the product **Guarantee.** Above the harness sits the **World** — the project's program / @@ -1461,6 +1507,17 @@ commodity; every quant system covers it. aura exists for the meta-level: a programmable space where harnesses are dynamically built and families of them orchestrated and explored. The deterministic single-harness engine (C1–C20) is the **substrate**; the World is the **product**. +**Refinement (2026-06-29, #109 resolved — the World owns topology as data, C24).** +"Harnesses are dynamically constructible, first-class objects" is sharpened: their +**topology is a serializable data value the World owns** (C24), not Rust source. +This is the precondition for the World's full ambition — it can **generate**, +**mutate**, **structurally search** (genetic / NEAT-style graph operators, the open +search-policy thread), **compare**, and **serialize-for-reproduction** (C18) +*families that vary structure*, not only numeric params. A param sweep over a fixed +skeleton is the degenerate case; varying the skeleton itself needs +topology-as-value. The game-engine principle (C16) made literal: the engine owns the +scene / blueprint graph as content, instantiates and runs it, the way aura owns +harness topology as data. ### C22 — The playground is a trace explorer; sinks are the recording mechanism **Guarantee.** The World is a *program*: nothing is displayable until it runs, @@ -1612,6 +1669,18 @@ the mean shows the net/duty-cycle level. Deferred refinements: rendering the min envelope honestly as range bars / OHLC rather than a polyline (#112); a `--width` budget flag and true intra-bucket min/max ordering for the non-default continuous x-mode (#110). +**Refinement (2026-06-29 — the visual face is orthogonal and optional; it does not +motivate C24).** The blueprint data format (C24) is required by C21 (generation / +structural search) and C18 (reproduction of a generated run) **at the CLI level, +autonomously** — independent of any visual surface. The playground's form (web or +egui, **editor or viewer, or not built at all**) is therefore **downstream and +optional**, and topology authoring does **not** live in it. This severs an earlier +conflation that treated the playground as where topology-editing would live. C22's +"never a scene editor" is refined to its true content — **no persistent +scene-at-rest** (a World is a *program*, C21) — which does **not** forbid a future +blueprint *editor* over the World-owned topology-data (C24); should the playground +ever become one, the data format is already the substrate it round-trips, not a new +mechanism. ### C23 — Graph compilation and behaviour-preserving optimisation **Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic, @@ -1670,6 +1739,57 @@ dropped at lowering (the flat graph is wired by raw index, unchanged). The check the by-name *authoring address space* injective; it does **not** make names load-bearing in the flat graph. +### C24 — The blueprint is a serializable, World-owned data value (the topology data format) +**Guarantee.** A **blueprint** — the param-generic, named graph-as-data a Rust +builder produces (C9/C19) — is a **first-class serializable data value** with a +stable format, and the engine has a **load path** (data → blueprint → `FlatGraph`, +C23), not only the Rust-builder construction path. Topology is therefore a **value +the World owns** (C21): constructed, **generated**, mutated, **structurally +searched**, serialized for **reproduction** (C18), and (optionally) rendered or +edited by a visual face (C22). `model_to_json` (C9) is the **existing reverse half** +(blueprint → data, for render); this contract closes the loop. The format carries +**topology + structural axes + the param-space**, referencing nodes by their +**compiled-in type identity** — the closed, typed node vocabulary of `aura-std` + the +project `cdylib` (C8/C16) — and carries **no node logic** (computation is compiled +Rust, C17). It is **non-Turing-complete by construction** (a static DAG, no control +flow), so it structurally cannot become a second RustAst. +**Forbids.** Putting **node logic / computation** in the format (the RustAst trap — +logic is Rust, C17); a **Turing-complete or control-flow-bearing** blueprint format +(it is static data; the *generator* that emits it may be Rust, C20, but the artifact +is not a language); a **by-name node marketplace / registry** (the vocabulary is the +compiled-in closed set referenced by type identity — the format is not a distribution +mechanism, C9/C16); treating the **visual face (C22) as the motivation or the +authoring brain** (the format is required by C21/C18 at the CLI level, independent of +any UI); a **generated / searched run whose topology is not recoverable from its +manifest** (C18 reproduction); baking topology as **engine source** compiled into the +binary (the hard-wired `aura-cli` harnesses are pre-C24 scaffolding, a C16 tension, +retired as this lands). +**Why.** The World (C21) is the product, and it cannot orchestrate, **structurally +search**, or **reproduce** *families that vary topology* while topology stays opaque +Rust source baked into the engine. The **game-engine principle** (C16's engine / game +split) makes it concrete: the engine is compiled native systems; the "game" is +**content** — and topology *is* content, a data value the engine owns, serializes, +instantiates, and mutates, exactly as a game engine owns its scene / prefab / +blueprint graphs. The #109 fork was never "*who types the wiring*" — a small LLM +cannot reliably *apply* a DSL and a strong one does not need one, so a +manifest-as-authoring-surface dies on both horns (C17). It is "**is topology a +compile-time *source* artifact or a runtime, serializable, World-owned *value*?**", +and determinism (C1) + reproduction (C18) + structural search (the open search-policy +thread) force the **value**. The engine already treats topology as a runtime value +(C9 graph-as-data, C19 "cheap graph re-compilation, not a code recompile", C23 the +flat graph as the optimisation target); C24 only adds that the value **serializes out +of Rust and loads back in**. +**Status (2026-06-29).** The **principle** is settled (this contract, ratified in an +in-context design discussion — the #109 resolution). The **format itself** — the +concrete data structure, serialization, the data → blueprint loader, and +content-addressing for identity (C18) — is **unbuilt**, the next brainstorm / +milestone. `model_to_json` (C9) is the one-directional half that exists. +Sequencing-coupled to the **project-as-crate authoring layer** (`aura new` / +`Aura.toml` / `cdylib` loading, C16/C17) and the **composable-orchestration** thread +(#109): topology-as-data is the substrate both stand on. The pre-C24 hard-wired +harnesses (`HarnessKind`, `run_stage1_r`, the per-strategy `*_sweep_family` in +`aura-cli`) are scaffolding retired as it lands. + --- ## Open architectural threads not yet resolved @@ -1686,7 +1806,9 @@ load-bearing in the flat graph. multi-strategy comparison, multi-column tap selection #47), a local server, and the run/replay clock controls. - **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies - atop the atomic sim unit (C12), not yet designed. + atop the atomic sim unit (C12), not yet designed. Search over *params* sits atop the + atomic unit; search over *structure* (graph mutation / crossover — NEAT-style) + additionally needs topology-as-data (C24). - **`aura new` scaffolder, the experiment-builder API, and `Aura.toml`'s static-context schema** — `aura new` scaffolds a Rust project *crate* (node / strategy / experiment blueprints) against the engine (C16/C20); the @@ -1702,12 +1824,17 @@ load-bearing in the flat graph. + cdylib loading, C16/C17). On the meta-level, authoring a strategy is *wiring existing nodes* (C9 composition) + *defining the analysis framework* (C21), not writing new nodes; the user wants it driven via the CLI, later the interactive - server / playground (C14/C22). **Load-bearing open fork — resolve in the brainstorm - before any cycle:** "driven via the CLI" = (a) *running* a Rust-authored experiment - via the CLI / playground (compatible with C16/C17/C22) vs (b) *wiring nodes through - the CLI itself* (a deliberate amendment to C17's no-DSL, C22's no-UI-topology, and - C9's no-by-name-node-registry). Deferred by the user (2026-06-22); #109 is its - durable home until promoted to a milestone seeded by that brainstorm. + server / playground (C14/C22). **Resolved (2026-06-29) → C24.** The fork was + mis-posed as "(a) run-a-Rust-experiment-via-CLI vs (b) wire-nodes-via-CLI". The real + axis is **topology = compile-time Rust source vs runtime, serializable, World-owned + data value**, settled as the **value** (C24, the game-engine principle): node + *logic* stays Rust (C17), *topology* becomes data the World generates / serializes / + reproduces. "Wire-nodes-via-CLI" is dropped — interactive wiring through the CLI is a + non-goal; it collapses to a file + parser, which *is* C24's load path. What + **remains** under #109: the **blueprint data format** itself (C24 Status — its own + brainstorm / milestone), and the **composable-orchestration** half — the axes + (sweep / MC / walk-forward) becoming tools applied to a World-constructed harness + instead of the hard-wired `aura-cli` verbs (`sweep_family` / `walkforward_family`). - **Inferential honesty of the World — family-selection without false-discovery control (tracked: milestone "Inferential validation (defend against false discovery at sweep scale)", #144 / #145 / #146; adjacent #139).** The World's