Aura/docs/specs/0012-blueprint-compile-composites.md

Blueprint → flat graph: composite inlining — Design Spec

Date: 2026-06-05 Status: Draft — awaiting user spec review Authors: orchestrator + Claude

Goal

Introduce the construction layer that turns a hand-written, raw-index graph into named graph-as-data (a Blueprint) that compiles to the flat, type-erased instance the run loop already runs (the flat graph). The unit of reuse is the composite: a nestable sub-graph fragment that exposes one output port and named input roles, and that the compile step inlines into the flat (nodes, sources, edges) the unchanged Harness::bootstrap consumes.

This realizes C9 (fractal composition), C19 (bootstrap = compilation), and C23 (blueprint → flat graph, wired by raw index; names non-load-bearing). It is the anchor of the Construction layer milestone (Gitea #12).

What ships: a Blueprint / Composite representation in aura-engine, a compile() that inlines composites (recursive), and an SMA-cross composite that runs bit-identically to today's hand-wired sample harness (C1).

What does NOT ship (ratified non-goals — see §Acceptance criteria): behaviour-preserving optimisation passes (CSE/DCE, sweep-invariant hoisting, C23); any external optimisation crate (egg/egglog — C16); the ergonomic named-handle wiring / experiment-builder API (C20 scope 2/3 — interior wiring stays raw-index); the aura graph render (#13).

Architecture

The construction layer sits above Harness::bootstrap, not inside it. The run loop, bootstrap's signature, and the Edge / Target / SourceSpec data model are unchanged. A new module crates/aura-engine/src/blueprint.rs adds:

• Composite — a reusable sub-graph fragment (interior items + interior edges + input roles + one output port). It is not a Node: it is never eval'd; it is compiled away by inlining. It derives a NodeSchema-shaped interface so the enclosing graph can wire and kind-check it before compilation.
• BlueprintNode — a blueprint item: either a Leaf(Box<dyn Node>) or a nested Composite. Both present a declared interface (typed inputs + one output) to the enclosing graph.
• Blueprint — the root graph-as-data: blueprint items + sources + edges, all addressing blueprint-level indices. compile() lowers it to flat (Vec<Box<dyn Node>>, Vec<SourceSpec>, Vec<Edge>); bootstrap() is the convenience that compiles and then calls Harness::bootstrap.

Compilation is index lowering: each composite's interior items are appended to the flat node array at an offset, interior edges are rewritten by that offset, edges into a composite are resolved through its input roles (a fan-out: one blueprint edge may become several flat edges), and edges out of a composite are resolved to its interior output port. Nesting is handled by compiling inside-out / recursively. After lowering, bootstrap's existing kind-check and Kahn cycle-check cover the flat graph for free (C7/C8).

The flat graph is wired by raw index, not by name (C23): the composite boundary dissolves entirely; field/input-role names, where kept, are non-load-bearing debug symbols (as FieldSpec.name already is, crates/aura-core/src/node.rs:37).

Concrete code shapes

User-facing: authoring an SMA-cross composite (the acceptance evidence)

A composite is wired with the same raw local indices the engine already uses, scoped to the fragment:

use aura_std::{Sma, Sub};
use aura_engine::{Composite, OutPort, Edge, Target};

/// SMA-cross signal as a reusable composite: one input role (price), one output
/// (the fast-minus-slow spread). Interior wired with raw local indices 0..2.
pub fn sma_cross(fast: usize, slow: usize) -> Composite {
    Composite::new(
        // interior items, local indices 0..2
        vec![
            Sma::new(fast).into(),   // 0  (BlueprintNode::Leaf)
            Sma::new(slow).into(),   // 1
            Sub::new().into(),       // 2
        ],
        // interior edges (local indices into the items above)
        vec![
            Edge { from: 0, to: 2, slot: 0, from_field: 0 }, // fast SMA -> Sub.in0
            Edge { from: 1, to: 2, slot: 1, from_field: 0 }, // slow SMA -> Sub.in1
        ],
        // input roles: role 0 (price) fans into BOTH SMAs' slot 0
        vec![
            vec![Target { node: 0, slot: 0 }, Target { node: 1, slot: 0 }],
        ],
        // the one output port (C8): Sub's output field 0
        OutPort { node: 2, field: 0 },
    )
}

Wiring it into a harness — the composite is one blueprint item, like any node; the 7-node/6-edge hand-wiring of today's sample_harness (crates/aura-cli/src/main.rs:42-78) becomes:

use aura_engine::{Blueprint, BlueprintNode, SourceSpec, Target, Edge};
use aura_core::ScalarKind::F64;

// price -> [sma_cross] -> Exposure -> SimBroker -> 2 recording sinks
let bp = Blueprint::new(
    vec![
        BlueprintNode::Composite(sma_cross(2, 4)),  // 0  (interior: 3 nodes)
        Exposure::new(0.5).into(),                  // 1
        SimBroker::new(1e-4).into(),                // 2
        rec_equity.into(),                          // 3  sink
        rec_exposure.into(),                        // 4  sink
    ],
    // sources address blueprint-level (node, slot); a target into a composite
    // names the composite's input role
    vec![SourceSpec { kind: F64, targets: vec![
        Target { node: 0, slot: 0 },   // price -> sma_cross role 0
        Target { node: 2, slot: 1 },   // price -> SimBroker price slot
    ]}],
    // blueprint-level edges
    vec![
        Edge { from: 0, to: 1, slot: 0, from_field: 0 }, // composite out -> Exposure
        Edge { from: 1, to: 2, slot: 0, from_field: 0 }, // exposure -> broker
        Edge { from: 2, to: 3, slot: 0, from_field: 0 }, // equity -> sink
        Edge { from: 1, to: 4, slot: 0, from_field: 0 }, // exposure -> sink
    ],
);

let mut harness = bp.bootstrap()?;   // compile (inline composites) + Harness::bootstrap
harness.run(streams);

The headline test: composite ≡ hand-wired, bit-for-bit (C1)

#[test]
fn composite_sma_cross_runs_bit_identical_to_hand_wired() {
    let streams = sample_price_stream();

    // (a) today's flat, hand-wired graph (the sample_harness wiring)
    let (mut flat, flat_eq, flat_ex) = hand_wired_sma_cross_harness();
    flat.run(streams.clone());

    // (b) the same graph authored as a composite blueprint, compiled
    let (bp, comp_eq, comp_ex) = composite_sma_cross_harness();
    let mut composed = bp.bootstrap().unwrap();
    composed.run(streams);

    // both recording sinks captured the same equity + exposure traces, bit-for-bit
    assert_eq!(flat_eq.borrow().as_slice(), comp_eq.borrow().as_slice());
    assert_eq!(flat_ex.borrow().as_slice(), comp_ex.borrow().as_slice());
}

Implementation shape (secondary) — new types, before → after

Before: a graph exists only as the flat arguments to bootstrap; there is no composite (harness.rs:30-52, :114).

After: crates/aura-engine/src/blueprint.rs adds (sketch — exact bytes are the planner's job):

/// Which interior (node, output-field) is a composite's single output port (C8).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct OutPort { pub node: usize, pub field: usize }

/// A blueprint item: a leaf node or a nested composite. Both present a declared
/// interface (typed inputs + one output) to the enclosing graph.
pub enum BlueprintNode {
    Leaf(Box<dyn Node>),
    Composite(Composite),
}
// ergonomic lift used in the examples above
impl<N: Node + 'static> From<N> for BlueprintNode { /* Leaf(Box::new(n)) */ }

/// A reusable sub-graph fragment compiled away by inlining (C9/C23). NOT a Node.
pub struct Composite {
    nodes: Vec<BlueprintNode>,   // interior items, local indices
    edges: Vec<Edge>,            // interior wiring, local indices
    input_roles: Vec<Vec<Target>>, // role r -> interior targets it fans into
    output: OutPort,             // the one exposed output port
}
impl Composite {
    pub fn new(nodes: Vec<BlueprintNode>, edges: Vec<Edge>,
               input_roles: Vec<Vec<Target>>, output: OutPort) -> Self;
    /// Derived interface the enclosing graph wires against: input r's kind/
    /// lookback/firing from its role targets; output kind from the interior
    /// output port. (Derivation, not a Node impl.)
    pub fn schema(&self) -> NodeSchema;
}

/// The root graph-as-data, before compilation.
pub struct Blueprint {
    nodes: Vec<BlueprintNode>,
    sources: Vec<SourceSpec>,   // targets address blueprint-level (node, slot)
    edges: Vec<Edge>,           // blueprint-level indices
}
impl Blueprint {
    pub fn new(nodes: Vec<BlueprintNode>, sources: Vec<SourceSpec>,
               edges: Vec<Edge>) -> Self;
    /// Lower to the flat graph: inline every composite (recursive), offset
    /// interior indices, rewrite edges, fan input roles out. Run loop unchanged.
    pub fn compile(self) -> Result<(Vec<Box<dyn Node>>, Vec<SourceSpec>, Vec<Edge>), CompileError>;
    /// Compile, then hand the flat graph to the unchanged Harness::bootstrap.
    pub fn bootstrap(self) -> Result<Harness, CompileError>;
}

/// A construction-phase fault, caught before the flat graph reaches bootstrap.
#[derive(Debug, PartialEq, Eq)]
pub enum CompileError {
    BadInteriorIndex,                 // an interior edge/role/output index is out of range
    RoleKindMismatch { role: usize }, // a role fans into interior slots of differing kinds
    OutputPortOutOfRange,             // the output port names a missing interior node/field
    Bootstrap(BootstrapError),        // the lowered flat graph failed bootstrap's checks
}

Harness::bootstrap, the run loop, Edge, Target, SourceSpec, and the Node trait are unchanged.

Components

• OutPort — { node, field }, a composite's single output port (C8).
• BlueprintNode — Leaf(Box<dyn Node>) | Composite(Composite), plus a From<N: Node> lift for ergonomic authoring.
• Composite — interior items, interior edges, input roles, output port; new() and a derived schema() (kinds/lookback/firing from role targets + output port). Never eval'd.
• Blueprint — root items + sources + edges (blueprint-level indices); new(), compile(), bootstrap().
• The inliner (compile's core) — recursive index lowering: append interior items at an offset, rewrite interior edges, resolve edges into a composite via its input roles (fan-out), resolve edges out of a composite to its output port.
• CompileError — construction-phase faults, with Bootstrap(BootstrapError) for the lowered flat-graph failure.

Data flow

Authoring → Blueprint (graph-as-data, blueprint indices) → compile():

1. Walk blueprint items. A Leaf lowers to itself (one flat node). A Composite lowers by inlining: its interior items are appended to the flat array at base = flat.len(); interior edges are emitted as flat edges with both endpoints + base; a nested interior composite recurses (compiled before its parent's edges resolve, so its own boundary is already gone).
2. Edge into a composite (to: c, slot: s, where item c is a composite): replaced by one flat edge per target in input_roles[s] — the fan-out (price → both SMAs). Same rule for a SourceSpec.targets entry whose node is a composite.
3. Edge out of a composite (from: c, from_field: 0): rewritten to from: base_c + output.node, from_field: output.field.
4. Result: flat (nodes, sources, edges) with no composites, wired by raw index. Blueprint::bootstrap hands these to the unchanged Harness::bootstrap, whose kind-check + Kahn cycle-check (harness.rs:142-192) validate the flat graph.

Then Harness::run drives the flat graph exactly as today (C1 determinism preserved — the flat graph is the same flat topology the hand-wiring produced).

Error handling

• compile() returns CompileError for construction-phase faults before anything reaches bootstrap: an out-of-range interior edge/role/output index (BadInteriorIndex), a role fanning into interior slots of differing scalar kinds (RoleKindMismatch), an output port naming a missing interior node/field (OutputPortOutOfRange).
• The lowered flat graph is validated by bootstrap's existing checks; any BootstrapError (kind mismatch, bad index, cycle) is wrapped as CompileError::Bootstrap(..). No duplication of bootstrap's validation in the inliner — the flat graph is checked once, where it always was (C7/C8).
• A composite's derived schema() is what the enclosing graph kind-checks against; a same-name role fanning to mismatched kinds is caught at compile, not hidden until run.

Testing strategy

• Unit (inliner): a single composite lowers with correct index offset and edge rewrite; an input role fans to ≥2 interior targets (the price → both-SMAs case); an edge out of a composite resolves to its output port; a nested composite (composite-in-composite) inlines correctly.
• Unit (errors): BadInteriorIndex, RoleKindMismatch, OutputPortOutOfRange each provoked by a malformed composite; a lowered flat graph with a kind mismatch or cycle surfaces as CompileError::Bootstrap(..).
• Integration (headline, C1): the SMA-cross composite, compiled and run, produces equity + exposure sink traces bit-identical to the hand-wired flat graph (composite_sma_cross_runs_bit_identical_to_hand_wired).
• Regression: the existing bootstrap/run tests stay green unchanged (the run loop and signatures are untouched).

Forward-looking design notes (no code this cycle)

The representation is shaped so the C23 passes are later cheap, but none is built here: (1) interior structure stays inspectable so a future CSE pass can hash node identity (type-tag + bound params + input-sources) — building that identity, and per-node params, is C8 work not yet in the schema (node.rs:6-7); (2) sink reachability is readable from the flat edge list so a future DCE pass is trivial; (3) per-node provenance survives compilation so sweep-invariance analysis is possible once params and the sweep orchestration land (C12/C21).

Acceptance criteria

1. Blueprint, BlueprintNode, Composite, OutPort, CompileError exist in crates/aura-engine/src/blueprint.rs; Blueprint::{compile, bootstrap} work.
2. A composite wires ≥2 interior nodes, exposes exactly one output port (C8), and fans one input role to ≥2 interior targets.
3. A nested composite (composite-in-composite) inlines correctly (C9 self- application).
4. The SMA-cross composite produces a run bit-identical to the hand-wired graph (C1) — the headline test passes.
5. compile() rejects malformed wiring with a typed CompileError; the lowered flat graph is validated by bootstrap's existing kind/cycle checks (no re-implementation).
6. Harness::bootstrap, Harness::run, Edge, Target, SourceSpec, and the Node trait are unchanged; the run loop is untouched.
7. Zero new external dependencies (C16); pure-Rust, hand-rolled.
8. Non-goals respected: no optimisation pass, no egg/egglog, no named-handle ergonomics, no aura graph.