Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

Serialize and load a blueprint as versioned canonical data (close the graph-as-data loop) #155

New Issue
Closed
opened 2026-06-29 12:58:03 +02:00 by Brummel · 2 comments
Brummel commented 2026-06-29 12:58:03 +02:00
Owner
Copy Link
Copy Source

C24 (the blueprint-as-data contract, docs/design/INDEX.md, settled f80571a) makes the blueprint a serializable, World-owned data value. Today only the forward half exists: model_to_json (crates/aura-engine/src/graph_model.rs) emits the graph model as data for the render. The inverse — data -> blueprint -> FlatGraph — does not exist (the engine builds a blueprint only from the Rust GraphBuilder / Composite path in crates/aura-engine/src/blueprint.rs).

Goal: a stable serialization of a Composite blueprint plus a loader, so a serialized blueprint compiles through compile_with_params (crates/aura-engine/src/blueprint.rs) and runs.

The format carries an envelope from its first version: a top-level format_version, and a canonical form — stable field ordering with defaults omitted (skip_serializing_if) — so an absent optional is byte-identical to the pre-extension bytes. Canonicality is a precondition for stable content-addressing and for the byte-identical-legacy-line discipline aura already pins in its serde tests.

Acceptance (validated property):

  • A serialized blueprint loads and runs to a bit-identical recorded trace (C1) against its Rust-built twin — same FlatGraph wiring, same sink output.
  • Serializer and loader are inverse on the in-repo example blueprints (round-trip identity).
  • The format carries format_version and a canonical omit-defaults encoding.

Scope: format + serializer + loader only. Out of scope, each its own issue: the forward-compat must-understand gate, the construction service, content-addressed identity.

C24 (the blueprint-as-data contract, docs/design/INDEX.md, settled f80571a) makes the blueprint a serializable, World-owned data value. Today only the forward half exists: model_to_json (crates/aura-engine/src/graph_model.rs) emits the graph model as data for the render. The inverse — data -> blueprint -> FlatGraph — does not exist (the engine builds a blueprint only from the Rust GraphBuilder / Composite path in crates/aura-engine/src/blueprint.rs). Goal: a stable serialization of a Composite blueprint plus a loader, so a serialized blueprint compiles through compile_with_params (crates/aura-engine/src/blueprint.rs) and runs. The format carries an envelope from its first version: a top-level format_version, and a canonical form — stable field ordering with defaults omitted (skip_serializing_if) — so an absent optional is byte-identical to the pre-extension bytes. Canonicality is a precondition for stable content-addressing and for the byte-identical-legacy-line discipline aura already pins in its serde tests. Acceptance (validated property): - [ ] A serialized blueprint loads and runs to a bit-identical recorded trace (C1) against its Rust-built twin — same FlatGraph wiring, same sink output. - [ ] Serializer and loader are inverse on the in-repo example blueprints (round-trip identity). - [ ] The format carries format_version and a canonical omit-defaults encoding. Scope: format + serializer + loader only. Out of scope, each its own issue: the forward-compat must-understand gate, the construction service, content-addressed identity.
Brummel added this to the Topology-as-data: blueprint serialization & loader (C24) milestone 2026-06-29 12:58:03 +02:00
Brummel added the feature label 2026-06-29 12:58:03 +02:00
Brummel commented 2026-06-29 13:16:28 +02:00
Author
Owner
Copy Link
Copy Source

Design reconciliation (specify, cycle 0087)

Spec: docs/specs/0087-blueprint-serialization-loader.md. Two load-bearing
forks were derived (no prior issue comment settled them); this records their
resolution before the spec is signed. Both are derived orchestrator
decisions (rationale, not user provenance) under the /boss bold-decide
stance — open to veto.

  • Fork: node type-identity → constructor resolution vs domain invariant 9
    ("no node registry inside aura").     The loader must map a serialized
    type_id (e.g. "SMA") back to a constructable node, but PrimitiveBuilder
    holds an opaque build: Box<dyn Fn> closure (crates/aura-core/src/node.rs:115)
    — there is no type_id → constructor resolution today (verified absent by
    recon: no registry/by_name/from_name/lookup in aura-std).
    → Decision: the loader is generic over an injected resolver
    Fn(&str) -> Option<PrimitiveBuilder>; the concrete closed match over
    the aura-std vocabulary lives in aura-std (a new std_vocabulary), not in
    the engine. This is not the forbidden registry — it realises the mechanism
    C24 already mandates.
    Basis (derived): C24 body (docs/design/INDEX.md:1751-1766, commit f80571a)
    mandates referencing nodes "by their compiled-in type identity — the
    closed, typed node vocabulary of aura-std + the project cdylib" and forbids
    only "a by-name node marketplace / registry … the format is not a
    distribution mechanism". Invariant 9's "no node registry" (in
    crates/aura/CLAUDE.md) governs multi-project management / distribution, not
    a crate's compile-time match over its own known nodes. The resolution is
    additionally forced by the crate graph: aura-engine → aura-core only
    (aura-std is a test-only dep — see the engine-domain-free comment in
    crates/aura-engine/Cargo.toml), so the engine cannot hold an aura-std
    node table; the resolver must be injected and the table must live outside
    the engine. A realisation of C24, not a contract change — a specify decision,
    not a brainstorm bounce.

  • Fork: are recording sinks part of the serialized blueprint? A recording
    sink is built as Recorder::builder(.., tx)
    (crates/aura-engine/src/test_fixtures.rs:67), capturing an mpsc::Sender in
    its build closure.
    → Decision: sinks are excluded from the serialized param-generic
    blueprint for #155; the round-trip test attaches identical sinks post-load
    to both twins and compares recorded traces.
    Basis (derived): a captured channel sender is runtime identity — neither a
    param nor topology — so it structurally cannot live in a param-generic,
    value-empty blueprint (C19, docs/design/INDEX.md:1290-1304). Sink-destination
    addressing is the C18-registry / trace-persistence concern (#101), out of
    #155's "format + serializer + loader" scope. This is also the correct long-term
    decomposition: the World owns the signal topology as data; observation is
    attached per run.

## Design reconciliation (specify, cycle 0087) Spec: `docs/specs/0087-blueprint-serialization-loader.md`. Two load-bearing forks were derived (no prior issue comment settled them); this records their resolution before the spec is signed. Both are **derived orchestrator decisions** (rationale, not user provenance) under the `/boss` bold-decide stance — open to veto. - **Fork: node type-identity → constructor resolution vs domain invariant 9 ("no node registry inside aura").** The loader must map a serialized `type_id` (e.g. `"SMA"`) back to a constructable node, but `PrimitiveBuilder` holds an opaque `build: Box<dyn Fn>` closure (`crates/aura-core/src/node.rs:115`) — there is no `type_id → constructor` resolution today (verified absent by recon: no `registry`/`by_name`/`from_name`/`lookup` in `aura-std`). → **Decision:** the loader is generic over an **injected** resolver `Fn(&str) -> Option<PrimitiveBuilder>`; the **concrete closed `match`** over the `aura-std` vocabulary lives in `aura-std` (a new `std_vocabulary`), not in the engine. This is **not** the forbidden registry — it realises the mechanism C24 already mandates. Basis (derived): C24 body (`docs/design/INDEX.md:1751-1766`, commit `f80571a`) mandates referencing nodes "by their **compiled-in type identity** — the closed, typed node vocabulary of `aura-std` + the project `cdylib`" and forbids only "a **by-name node marketplace / registry** … the format is not a distribution mechanism". Invariant 9's "no node registry" (in `crates/aura/CLAUDE.md`) governs *multi-project management / distribution*, not a crate's compile-time `match` over its own known nodes. The resolution is additionally **forced** by the crate graph: `aura-engine → aura-core` only (`aura-std` is a test-only dep — see the engine-domain-free comment in `crates/aura-engine/Cargo.toml`), so the engine *cannot* hold an `aura-std` node table; the resolver *must* be injected and the table *must* live outside the engine. A realisation of C24, not a contract change — a `specify` decision, not a `brainstorm` bounce. - **Fork: are recording sinks part of the serialized blueprint?** A recording sink is built as `Recorder::builder(.., tx)` (`crates/aura-engine/src/test_fixtures.rs:67`), capturing an `mpsc::Sender` in its build closure. → **Decision:** sinks are **excluded** from the serialized param-generic blueprint for #155; the round-trip test attaches identical sinks **post-load** to both twins and compares recorded traces. Basis (derived): a captured channel sender is runtime identity — neither a param nor topology — so it structurally cannot live in a param-generic, value-empty blueprint (C19, `docs/design/INDEX.md:1290-1304`). Sink-destination addressing is the C18-registry / trace-persistence concern (#101), out of #155's "format + serializer + loader" scope. This is also the correct long-term decomposition: the World owns the signal topology as data; observation is attached per run.
Brummel commented 2026-06-29 13:38:37 +02:00
Author
Owner
Copy Link
Copy Source

Design decision — variadic-builder scope (planner, cycle 0087)

A third load-bearing decision surfaced while planning #155, derived (not a user
preference), open to veto.

  • Fork: which node vocabulary does the #155 loader round-trip? Of aura-std's
    29 builder() factories, 22 are zero-arg Type::builder(); 4 take
    structural construction arguments — LinComb::builder(arity),
    CostSum::builder(n_costs), SimBroker::builder(pip_size),
    Session::builder(open_hour, open_minute, tz, period_minutes) — and 3 are
    recording sinks (already excluded). The injected resolver
    Fn(&str) -> Option<PrimitiveBuilder> receives only the type_id, so it
    cannot reconstruct a node whose construction depends on an argument the format
    does not carry. risk_executor/vol_stop (the composite the spec named as a
    "richer" round-trip candidate) contain LinComb::builder(1), so they hit this
    gap.
    → Decision: #155's round-trippable vocabulary is the 22 zero-arg
    builders     (std_vocabulary is a closed match over exactly those). The 4
    construction-arg builders are not in the set; a serialized composite
    containing one fails to load cleanly with LoadError::UnknownNodeType
    (never a silent wrong graph). The round-trip tests use only zero-arg nodes
    (the recursive "richer" test nests a zero-arg signal composite, not
    risk_executor).
    Basis (derived): a structural construction argument (an arity, a calendar
    config) is a structural axis (C20 — "Timestamp is a structural axis, never
    a numeric knob"; docs/design/INDEX.md), distinct from both a bound param and
    topology, and the param-generic format (C19) does not yet encode it — exactly
    parallel to the already-accepted sink exclusion (a captured value that is
    neither param nor topology). Serialising structural-axis construction args is
    its own concern, additively addable later (#156 forward-compat) without
    breaking existing serializations — so deferring it narrows #155's scope
    without foreclosing it. A scope-narrowing within "format + serializer +
    loader", not a contract change — a specify/planner decision, not a
    brainstorm bounce.
## Design decision — variadic-builder scope (planner, cycle 0087) A third load-bearing decision surfaced while planning #155, derived (not a user preference), open to veto. - **Fork: which node vocabulary does the #155 loader round-trip?** Of aura-std's 29 `builder()` factories, 22 are zero-arg `Type::builder()`; 4 take **structural construction arguments** — `LinComb::builder(arity)`, `CostSum::builder(n_costs)`, `SimBroker::builder(pip_size)`, `Session::builder(open_hour, open_minute, tz, period_minutes)` — and 3 are recording sinks (already excluded). The injected resolver `Fn(&str) -> Option<PrimitiveBuilder>` receives only the `type_id`, so it cannot reconstruct a node whose construction depends on an argument the format does not carry. `risk_executor`/`vol_stop` (the composite the spec named as a "richer" round-trip candidate) contain `LinComb::builder(1)`, so they hit this gap. → **Decision:** #155's round-trippable vocabulary is the **22 zero-arg builders** (`std_vocabulary` is a closed `match` over exactly those). The 4 construction-arg builders are **not** in the set; a serialized composite containing one fails to **load** cleanly with `LoadError::UnknownNodeType` (never a silent wrong graph). The round-trip tests use only zero-arg nodes (the recursive "richer" test nests a zero-arg signal composite, not `risk_executor`). Basis (derived): a structural construction argument (an arity, a calendar config) is a **structural axis** (C20 — "Timestamp is a structural axis, never a numeric knob"; `docs/design/INDEX.md`), distinct from both a bound param and topology, and the param-generic format (C19) does not yet encode it — exactly parallel to the already-accepted sink exclusion (a captured value that is neither param nor topology). Serialising structural-axis construction args is its own concern, **additively addable later** (#156 forward-compat) without breaking existing serializations — so deferring it narrows #155's scope without foreclosing it. A scope-narrowing within "format + serializer + loader", not a contract change — a `specify`/`planner` decision, not a `brainstorm` bounce.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BLOCKER
Core functionality blocked; preempts other selection
 bug
Observable misbehaviour
 feature
Substantial extension; no full spec required
 idea
Droppable; no commitment
No Label feature
Milestone
No items
No Milestone Topology-as-data: blueprint serialization & loader (C24)
Projects
Clear projects
No project
Assignees
Clear assignees
claude (Claude) Brummel
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#155
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?