Aura/docs/design/contracts/c24-blueprint-data.md

C24 — The blueprint is a serializable, World-owned data value (the topology data format)

Guarantee. A blueprint — the param-generic, named graph-as-data a Rust builder produces (C9/C19) — is a first-class serializable data value with a stable, versioned format, and the engine has a load path (data → blueprint → FlatGraph, C23), not only the Rust-builder construction path. Topology is therefore a value the World owns (C21): constructed, generated, mutated, structurally searched, serialized for reproduction (C18), and (optionally) rendered or edited by a visual face (C22). model_to_json (C9) is the existing reverse half (blueprint → data, for render); this contract closes the loop. The format carries topology + structural axes + the param-space, referencing nodes by their compiled-in type identity — the closed, typed node vocabulary of the std set plus the project cdylib (C8/C16) — and carries no node logic (computation is compiled Rust, C17). It is non-Turing-complete by construction (a static DAG, no control flow), so it structurally cannot become a second RustAst.

Forbids. Putting node logic / computation in the format (the RustAst trap — logic is Rust, C17); a Turing-complete or control-flow-bearing blueprint format (it is static data; the generator that emits it may be Rust, C20, but the artifact is not a language); a by-name node marketplace / registry (the vocabulary is the compiled-in closed set referenced by type identity — the format is not a distribution mechanism, C9/C16); treating the visual face (C22) as the motivation or the authoring brain (the format is required by C21/C18 at the CLI level, independent of any UI); a generated / searched run whose topology is not recoverable from its manifest (C18 reproduction); baking topology as engine source compiled into the binary. The hard-wired aura-cli harnesses (HarnessKind, run_r_sma, r_sma_graph, sample_blueprint_with_sinks, the built-in *_sweep_family) were pre-C24 scaffolding, a C16 tension, and are now retired (#159/#220): topology reaches the engine only as data, resolved by the injected vocabulary. The wrap_r R-scaffolding wrap (aura-runner) survives — it is the r-sma wrap every run / sweep / mc / walkforward path applies to the loaded signal at run (supplied at run, not serialized); its full R-scaffolding retirement stays deferred (#159). Harness input roles bind archive columns by name, not by a single baked-in price weld (C26).

Why. The World (C21) is the product, and it cannot orchestrate, structurally search, or reproduce families that vary topology while topology stays opaque Rust source baked into the engine. The game-engine principle (C16's engine / game split) makes it concrete: the engine is compiled native systems; the "game" is content — and topology is content, a data value the engine owns, serializes, instantiates, and mutates, exactly as a game engine owns its scene / prefab / blueprint graphs. The #109 fork was never "who types the wiring" — a small LLM cannot reliably apply a DSL and a strong one does not need one, so a manifest-as-authoring-surface dies on both horns (C17). It is "is topology a compile-time source artifact or a runtime, serializable, World-owned value?", and determinism (C1) + reproduction (C18) + structural search (the open search-policy thread) force the value. The engine already treats topology as a runtime value (C9 graph-as-data, C19 "cheap graph re-compilation, not a code recompile", C23 the flat graph as the optimisation target); C24 only adds that the value serializes out of Rust and loads back in.

Current state

The principle is settled — the #109 resolution, ratified in an in-context design discussion. The format, its loader, the construction service, and the family orchestration all ship. The acceptance criterion is green: a serialized blueprint runs bit-identical (C1) to its Rust-built twin.

The format and load path

A Composite serializes to a canonical, versioned data value — a format_version envelope (BlueprintDoc), omit-defaults JSON — and loads back (data → blueprint → FlatGraph) via blueprint_to_json / blueprint_from_json (aura-engine::blueprint_serde). Nodes are referenced by compiled-in type identity through an injected resolver whose concrete closed match over the std vocabulary lives outside the engine (aura-vocabulary::std_vocabulary), so the engine stays domain-free, with no node registry (invariant 9). An unknown type fails clean as LoadError::UnknownNodeType — the data-plane face of invariant 9, never a silent wrong graph. model_to_json (C9) remains the render half; this closes the loop for the round-trippable vocabulary.

Out of the round-trippable set (deliberate; fails clean as UnknownNodeType): recording sinks (they capture an mpsc::Sender — runtime identity, not param-generic data, C19) and SimBroker (a scalar-typed baked value, a narrower gap than the args channel). LinComb / CostSum / Session — formerly listed here — entered the round-trippable set with #271's typed construction args (see the add-op args clause and the data-driven format_version below).

Runs and families are built FROM blueprint-data

aura run <blueprint.json> loads a serialized signal blueprint and emits a RunReport bit-identical (C1) to its Rust-built twin; the run scaffolding (sinks / broker / data) is supplied at run, not serialized. Beyond a single run, the World constructs and orchestrates families of harnesses from topology-data:

• aura sweep <blueprint.json> --axis <name>=<csv> → FamilyKind::Sweep. A sweep needs an open blueprint (a fully-bound one has an empty param_space).
• aura mc <blueprint.json> --seeds N → FamilyKind::MonteCarlo, each seed a distinct synthetic price walk drawn disjoint-parallel through the engine monte_carlo seam (invariant 1). MC binds no axis, so it needs a closed blueprint (the sweep's distinction inverted); an open one returns a named Err, rendered exit-2 at the builder boundary — no hidden exit in the pure builder.
• aura walkforward <blueprint.json> --axis <name>=<csv> [--select argmax|plateau:mean|plateau:worst] → FamilyKind::WalkForward: re-optimizes the loaded blueprint's params over the --axis grid on each 24/12/12 IS window, selects the winner by sqn_normalized, runs it out-of-sample. Reduce-mode members are R-measured (oos_r the meaningful summary; stitched pip-equity empty, C10).

The family builders live in aura-runner::family (blueprint_sweep_family / blueprint_mc_family / blueprint_walkforward_family). Every member manifest carries the shared topology_hash (one signal topology, only params vary; member_key distinguishes members), and each family stores its blueprint(s) content-addressed so aura reproduce re-derives every member bit-identically (aura-runner::reproduce, C18). The synthetic-walk DGP is the MC machinery, not trader-grade statistics; a real-data block-bootstrap — and retiring the synthetic_walk_sources len:60↔warm-up coupling — rides #172.

Reproduction identity

The RunManifest carries a topology_hash: SHA256 of the canonical blueprint_to_json (see Canonical form), the #158 reproducibility anchor, a Tier-1 optional field (#156). The hash and its helper live research-side (aura-cli + sha2), off the frozen engine (invariant 8). The hash covers the signal only — the fixed scaffolding is identified by commit; a content-id distinguishing structural-axis / whole-harness variants is a #158/#166 concern once scaffolding is itself blueprint-data. The identity id (#171) is an additive sibling: blueprint_identity_json (aura-engine::blueprint_serde) + graph introspect --identity-id. The byte-exact content id keeps the store/reproduce roles; the identity id is introspection-only until a dedup consumer exists.

Axis discovery

aura sweep <blueprint.json> --list-axes lists a loaded blueprint's open sweepable knobs (one <name>:<kind> per line, param_space() order) and exits; the printed names are exactly what --axis binds. Every listed name is mandatory on sweep / walkforward — the blueprint must be fully bound before it runs — so a subset grid is refused with the missing knob named (BindError::MissingKnob, aura-engine) and there is no default; pin a knob you do not want to vary with a single-value axis. A single blueprint_axis_probe (aura-runner) single-sources the wrapped probe for the sweep terminal, the MC closed-check, and the listing, so listed == swept by construction (and stays so across the harness retirement — the listing tracks whatever the sweep actually resolves). One raw namespace, <node>.<param> (a splice path keeps its interior path, e.g. anchor.sess.period_minutes), is the only user-facing axis name (#328): graph introspect --params and --list-axes are line-identical (open params bare, bound params with default=) and both print exactly what --axis binds, on both sweep routes. The wrapped <blueprint>.<node>.<param> form the probe still resolves against internally is retired from the surface, with a translation refusal naming the raw candidate on both intake seams (--axis and campaign validate). --trace on sweep / walkforward writes per-member traces on the real-data campaign path (depth-2 fan-out, chartable by the printed family handle, #224); the synthetic path still refuses (run / mc refuse --trace outright). The live trace-writer is the campaign presentation.persist_taps (persist_campaign_traces, aura-runner::runner).

The construction service (op-script)

aura graph build / introspect over a JSON op-list (#157) is the introspectable, replayable, by-identifier construction surface; it emits the serializable blueprint. The engine GraphSession / replay (aura-engine::construction) build a runnable blueprint through validated ops. Construction gates split into eager per-op checks — name resolution, edge kind-match (edge_kind_check), the double-wire arm, param bind, and acyclicity (a connect that would close a cycle is rejected eagerly at the closing op, GraphSession::closes_cycle, #161) — and holistic finalize checks — wiring totality (validate_wiring), param-namespace injectivity (check_param_namespace_injective). Both cadences call the same extracted predicates — no second validator. The holistic faults read by-identifier (#162): finish() translates the index-carrying CompileErrors (UnconnectedPort / RoleKindMismatch) into by-identifier OpError variants while still calling the unchanged holistic gates. Root-role boundness moved off this list (#317, open patterns): check_root_roles_bound no longer runs in finish() — a finish()ed op-script may still carry an open (unbound) root role, declaring a reusable pattern's formal parameter; only compile/bootstrap (the runnability gate, not a construction gate) still call it, unchanged. Build-free introspection answers over the closed vocabulary: graph introspect --vocabulary | --node <T> | --unwired. The engine Op stays serde-free; the wire DTO (OpDoc) is CLI-side (aura-cli::graph_construct). Acceptance: a graph built purely through the ops compiles identical (C1) to its Rust-built twin, an invalid op is rejected at the op naming the cause, introspection answers without a build.

Registered-blueprint splice (#317). The use op resolves a reference — a store content id, a unique content-id prefix (#302 semantics), or a registry name label (graph register --name, latest-wins, aura-registry's blueprint_names.jsonl sidecar) — to a previously registered blueprint and splices it into the building graph as a nested BlueprintNode::Composite under an instance identifier. The engine never resolves: GraphSession/replay take a second injected closure, subgraph: &dyn Fn(&str) -> Option<Composite>, mirroring vocab's closed-lookup posture exactly — this is the same enforcement-shift permission §"Enforcement shift" above already grants the node vocabulary, extended to registered-blueprint references. All store I/O — label/prefix resolution, the get_blueprint fetch, the C29 doc-gate re-walk over the fetched composite (before it ever reaches the session, so a doc-less fetched entry cannot enter a NEW composition — a backstop: the register verb already gates both input forms, so this fires only for store content written before C29 or through the raw in-crate path), and the resolution echo — happens CLI-side, at DTO conversion, before replay; build-free introspection paths pass &|_| None. The echo (aura: note: use "<instance>": <label-or-prefix> -> <full id>) is the existing aura: note: benign-diagnostic marker (C14) — a new instance of the existing class, not a new one, so the exit-code/marker taxonomy is unchanged.

Invariant-5 lockstep. The eager acyclicity gate is a second home of invariant 5 alongside the bootstrap Kahn sort (harness.rs, BootstrapError::Cycle); the two must co-evolve when the explicit delay/register node — invariant 5's sole legal feedback — lands, or a valid delay-feedback graph the bootstrap accepts would be rejected at construction.

Op-script grammar

The op-script is a JSON array of ops, each object internally tagged by "op", replayed in order; nodes are referenced by identifier, ports as dotted <identifier>.<port>. The eleven verbs:

• name — {"op":"name","name":<str>} — set the composite's render name (#331), script-level and at-most-once (a second refuses); omitted, the built composite keeps the CLI's own seed default ("graph"). Gated by a shared shape check (non-empty, single path segment, no /, \, . or ..) applied at this op intake plus every CLI intake that reads an authored blueprint envelope from a file — register, introspect --content-id <FILE>, the bare graph-file viewer, run, introspect --params <FILE>, and each of sweep/walkforward/mc/generalize's file-reading entry points (synthetic and --real alike) — one class of intake, one gate. Store read-back (reproduce, use resolution, introspect/--params by content id) stays deliberately ungated — C29: a registered artifact is never retroactively invalidated.
• source — {"op":"source","role":<str>,"kind":<ScalarKind>} — declare a root source role producing a base column of kind.
• input — {"op":"input","role":<str>} — declare a root input role (kind inferred from the slots it feeds) — a reusable pattern's formal parameter, left open until an enclosing graph wires it (#317's open patterns): finish() accepts an op-script that never binds it, only compile/bootstrap require it bound.
• add — {"op":"add","type":<TypeId>,"name":<str>?,"args":{<arg>:<str>}?,"bind":{<param>:<Scalar>}?} — add a node of compiled-in type identity type; name is its identifier (mirrors the builder's .named(...); defaults to the lowercased type label, so two unnamed nodes of one type collide); args (#271) configures an arg-bearing type's structural, non-scalar construction (Session's IANA timezone/open, LinComb/CostSum's arity) through the closed, load-time-validated ArgKind table (Tz/TimeOfDay/Count) — applied BEFORE bind, so a bound param args unlocks (e.g. LinComb's weights[i]) is only bindable once arity is consumed; an arg-bearing type given no args refuses (BadArg(MissingArg)), a Plain type given args it does not declare also refuses (BadArg(NotArgBearing)); bind sets the (now real) params.
• feed — {"op":"feed","role":<str>,"into":[<port>,…]} — fan a root role into interior input slots.
• connect — {"op":"connect","from":<port>,"to":<port>} — wire an interior output field to an input slot; a connect closing a cycle is rejected eagerly (invariant 5).
• expose — {"op":"expose","from":<port>,"as":<str>} — promote an interior output field to a boundary output, aliased by as (a real alias, not a naming; contrast add's name) — one of the two verbs that keep as (with tap).
• tap — {"op":"tap","from":<port>,"as":<str>} — declare a measurement tap on an interior output field under as (#284) — the output-side twin of expose: a recorded observation point (a Composite.taps entry, C27), not a boundary output. Name-addressed like every other op (no raw index); tap names are their own namespace (a duplicate refuses). The finish gate threads op-declared taps into the built Composite (.with_taps); a single aura run records each, a sweep leaves them inert.
• gang — {"op":"gang","as":"channel_length","into":["channel_hi.length","channel_lo.length"]} — fuse two or more sibling params into ONE public knob: the member addresses leave the sweepable param space and as replaces them; the bound or swept value fans out to every member at bootstrap. Members must share one scalar kind and stay open (un-bound).
• doc — {"op":"doc","text":<str>} — declare the composite's one-line meaning (C29, #316) — the op-script twin of the builder's .doc(...); at most one per script, a second is a fault (a declarative script carries no last-wins ambiguity).
• use — {"op":"use","ref":{"content_id":<str>}|{"name":<str>},"name":<str>?,"bind":{<path>:<Scalar>}?} — splice a registered blueprint into the building graph as a nested composite under an instance identifier (name, defaulting to the fetched composite's own name); bind path-qualifies the spliced instance's params (e.g. "sma.length"), applied after the splice (#317). An instance's open input roles become its <instance>.<role> in-ports, its output boundary aliases its <instance>.<field> out-fields — the same Composite arm every other add/wiring path walks, not a new mechanism. See §"The construction service" below for the resolution split.

Value forms are the serialized representations: a Scalar bind value is the typed tag form {"I64":2} / {"F64":0.5} / {"Bool":true} / {"Timestamp":<i64>}, and kind is the capitalized ScalarKind ("F64"). Worked example: fieldtests/cycle-0088-construction-op-script/c0088_1_sma_crossover.json (an SMA-crossover bias).

Canonical form (#164)

The canonical blueprint artifact is exactly the bytes blueprint_to_json returns — a JSON value with no trailing newline (serde_json::to_string, not framed with a display println!), so the CLI and library emit paths are byte-identical — a prerequisite for content-addressed topology identity (#158, a hash over the canonical form). The canonical JSON is also the blueprint's equality / identity surface: Composite (aura-engine::blueprint) and PrimitiveBuilder (aura-core::node) deliberately carry no in-memory PartialEq / Debug — the recipe holds a build: Box<dyn Fn> closure (non-comparable, non-printable), so equality could only be defined over the serialized data, of which blueprint_to_json is already the single source; a second in-memory notion would be a drift hazard against the very form #158 content-addresses. The loader stays lenient (a trailing newline on input still parses, Tier-1 robustness).

Forward-compat: the two-tier discipline (#156)

Tier-1 (additive-optional) is serde-default silent-ignore with no format_version bump — a new optional field defaults to prior behaviour (C1). Tier-2 (must-understand: a new node type, edge semantics, or structural-axis kind) bumps format_version so an old reader refuses cleanly (LoadError::UnsupportedVersion / UnknownNodeType). Pre-ship dormancy (#61, ended by #271): until the first external ship there were no out-of-repo readers — reader and writer changed atomically in one commit — so the Tier-2 bump discipline lay dormant and structurally-semantic additions (the gangs section) landed as additive-optional fields of v1. #271's construction args are the first exercised Tier-2 bump (data-driven, next paragraph); v1 remains the version of every args-free document, gangs included. The per-section required-flag scheme stays deferred (no per-section case yet; recorded on #156).

First data-driven bump (#271): construction args are the first Tier-2 case whose version is decided PER DOCUMENT, not a single global bump. The writer emits format_version: 1 for an args-free document (byte-identical to every pre-#271 document — content ids stable) and 2 the moment any primitive, at any composite nesting depth, carries args — the must-understand signal an old (pre-#271) reader needs, since it cannot construct a pending arg-bearing type at all. The loader accepts the closed range 1..=2.

Enforcement shift — invariant 9 on the data plane

Lifting topology onto the data plane relocates where the engine/project boundary (invariant 9, no bespoke node registry/marketplace) is enforced, without changing what it forbids. Pre-C24 a blueprint referencing another project's node was a compile error — topology was in-process Rust, so cross-boundary node resolution was structurally impossible, compiler-guaranteed for free. Post-C24 that same reference is a type-id string in a serialized blueprint, refused only by the injected resolver's closed set (UnknownNodeType). The anti-NIH rationale survives intact (node logic stays cargo+Gitea-only; the format distributes topology-as-content, never logic — C17), and std_vocabulary stays a compiled-in dispatch table, not a registry — but its enforcement degrades from compiler-guaranteed to resolver-seam discipline. Two consequences:

1. #160 (the guard that the closed vocabulary stays honest) is no longer droppable hygiene — it guards the data-plane face of the boundary (its own drift direction still fails safe, UnknownNodeType). The std_vocabulary_roster! macro (aura-vocabulary) expands the resolver match and the enumerable std_vocabulary_types() list from one roster, closing the resolver-vs-list drift mode by construction; the residual (a new zero-arg node never rostered at all) stays fail-safe, guarded by the one-line maintainer surface plus the count pins (the in-crate shape test and aura-cli's cross-boundary --vocabulary count e2e). The per-project merged resolver (project ∪ std, project first) lives in aura-runner::project.
2. The injected per-project resolver (C16) is the genuinely new, deferred invariant-9 surface: nothing structurally stops a project resolver from accepting arbitrary type-ids, or a project from layering a blueprint marketplace (store + type-discovery API) on top of the format — one layer above the engine contract. Invariant 9 holds at the engine; the project/ecosystem boundary (what an injected resolver may resolve, where closed-set discipline draws the line) is a World/C21-layer charter point, not an engine fix.

Invariant 8 (frozen) is likewise reframed by C24/C21: from "topology baked at build time" to "the World may generate topology at runtime (research plane), the chosen blueprint is frozen into the deploy artifact, and any run is deterministic once instantiated" (C1).

Adjacent / deferred

The project-as-crate load boundary has landed (Aura.toml discovery + cdylib loading + merged project ∪ std vocabulary; the aura new scaffolder emits a buildable project whose blueprint runs through the merged vocabulary) — see C13/C16. Canonical project shape (#181): the aura new templates (scaffold.rs) are the canonical authoring shape and evolve with the engine; the demo-project fixture is an intentionally frozen known-good twin. The two are deliberately not lockstep-guarded — no consumer requires them to match, each is e2e-guarded on its fitness for purpose (build → descriptor load → charter check → deterministic run; blueprint wiring content is pinned in neither, stated honestly), and an equality guard would convert every deliberate template improvement into forced churn of the frozen fixture (the same cross-purpose coupling that rules out regenerating the fixture from the scaffolder). What remains open is the composable-orchestration thread (#109): topology-as-data is the substrate it stands on.

See also

• C9 — graph-as-data; model_to_json render half
• C16 — engine / project split; the injected resolver
• C17 — logic is Rust; the closed-vocabulary-data-artifact clarification
• C18 — reproduction, the run registry, content-addressed store
• C19 — bootstrap: blueprint → frozen instance
• C21 — the World owns and orchestrates topology
• C23 — bootstrap-as-compilation to FlatGraph; the optimisation target
• C26 — role names bind archive columns
• C27 — declared taps as named measurement points
• C1 — determinism / bit-identity; C10, C13, C20, C22, C25, C28

History: c24-blueprint-data.history.md