spec: 0012 blueprint-compile composites
The Construction-layer milestone's anchor spec: a Blueprint graph-as-data that compiles (inlines composites) to the flat (nodes, sources, edges) the unchanged Harness::bootstrap consumes (C9/C19/C23). The SMA-cross composite runs bit-identical to the hand-wired sample_harness (C1). Non-goals: optimisation passes, external deps, named-handle ergonomics, aura graph. Grounding-check PASS (13/13 current-behaviour assumptions ratified by green tests). refs #12
This commit is contained in:
@@ -0,0 +1,316 @@
|
||||
# Blueprint → compilat: composite inlining — Design Spec
|
||||
|
||||
**Date:** 2026-06-05
|
||||
**Status:** Draft — awaiting user spec review
|
||||
**Authors:** orchestrator + Claude
|
||||
|
||||
## Goal
|
||||
|
||||
Introduce the **construction layer** that turns a hand-written, raw-index graph
|
||||
into named **graph-as-data** (a `Blueprint`) that **compiles** to the flat,
|
||||
type-erased instance the run loop already runs (the **compilat**). The unit of
|
||||
reuse is the **composite**: a nestable sub-graph fragment that exposes one output
|
||||
port and named input roles, and that the compile step **inlines** into the flat
|
||||
`(nodes, sources, edges)` the unchanged `Harness::bootstrap` consumes.
|
||||
|
||||
This realizes C9 (fractal composition), C19 (bootstrap = compilation), and C23
|
||||
(blueprint → flat compilat, wired by raw index; names non-load-bearing). It is the
|
||||
anchor of the *Construction layer* milestone (Gitea #12).
|
||||
|
||||
**What ships:** a `Blueprint` / `Composite` representation in `aura-engine`, a
|
||||
`compile()` that inlines composites (recursive), and an SMA-cross composite that
|
||||
runs **bit-identically** to today's hand-wired sample harness (C1).
|
||||
|
||||
**What does NOT ship** (ratified non-goals — see §Acceptance criteria):
|
||||
behaviour-preserving optimisation passes (CSE/DCE, sweep-invariant hoisting, C23);
|
||||
any external optimisation crate (egg/egglog — C16); the ergonomic named-handle
|
||||
wiring / experiment-builder API (C20 scope 2/3 — interior wiring stays raw-index);
|
||||
the `aura graph` render (#13).
|
||||
|
||||
## Architecture
|
||||
|
||||
The construction layer sits **above** `Harness::bootstrap`, not inside it. The run
|
||||
loop, `bootstrap`'s signature, and the `Edge` / `Target` / `SourceSpec` data model
|
||||
are **unchanged**. A new module `crates/aura-engine/src/blueprint.rs` adds:
|
||||
|
||||
- `Composite` — a reusable sub-graph fragment (interior items + interior edges +
|
||||
input roles + one output port). It is **not** a `Node`: it is never `eval`'d; it
|
||||
is compiled away by inlining. It *derives* a `NodeSchema`-shaped interface so the
|
||||
enclosing graph can wire and kind-check it before compilation.
|
||||
- `BlueprintNode` — a blueprint item: either a `Leaf(Box<dyn Node>)` or a nested
|
||||
`Composite`. Both present a declared interface (typed inputs + one output) to the
|
||||
enclosing graph.
|
||||
- `Blueprint` — the root graph-as-data: blueprint items + sources + edges, all
|
||||
addressing **blueprint-level** indices. `compile()` lowers it to flat
|
||||
`(Vec<Box<dyn Node>>, Vec<SourceSpec>, Vec<Edge>)`; `bootstrap()` is the
|
||||
convenience that compiles and then calls `Harness::bootstrap`.
|
||||
|
||||
Compilation is **index lowering**: each composite's interior items are appended to
|
||||
the flat node array at an offset, interior edges are rewritten by that offset,
|
||||
edges *into* a composite are resolved through its input roles (a fan-out: one
|
||||
blueprint edge may become several flat edges), and edges *out of* a composite are
|
||||
resolved to its interior output port. Nesting is handled by compiling
|
||||
inside-out / recursively. After lowering, `bootstrap`'s existing kind-check and
|
||||
Kahn cycle-check cover the flat compilat for free (C7/C8).
|
||||
|
||||
The compilat is **wired by raw index, not by name** (C23): the composite boundary
|
||||
dissolves entirely; field/input-role names, where kept, are non-load-bearing debug
|
||||
symbols (as `FieldSpec.name` already is, `crates/aura-core/src/node.rs:37`).
|
||||
|
||||
## Concrete code shapes
|
||||
|
||||
### User-facing: authoring an SMA-cross composite (the acceptance evidence)
|
||||
|
||||
A composite is wired with the **same raw local indices** the engine already uses,
|
||||
scoped to the fragment:
|
||||
|
||||
```rust
|
||||
use aura_std::{Sma, Sub};
|
||||
use aura_engine::{Composite, OutPort, Edge, Target};
|
||||
|
||||
/// SMA-cross signal as a reusable composite: one input role (price), one output
|
||||
/// (the fast-minus-slow spread). Interior wired with raw local indices 0..2.
|
||||
pub fn sma_cross(fast: usize, slow: usize) -> Composite {
|
||||
Composite::new(
|
||||
// interior items, local indices 0..2
|
||||
vec![
|
||||
Sma::new(fast).into(), // 0 (BlueprintNode::Leaf)
|
||||
Sma::new(slow).into(), // 1
|
||||
Sub::new().into(), // 2
|
||||
],
|
||||
// interior edges (local indices into the items above)
|
||||
vec![
|
||||
Edge { from: 0, to: 2, slot: 0, from_field: 0 }, // fast SMA -> Sub.in0
|
||||
Edge { from: 1, to: 2, slot: 1, from_field: 0 }, // slow SMA -> Sub.in1
|
||||
],
|
||||
// input roles: role 0 (price) fans into BOTH SMAs' slot 0
|
||||
vec![
|
||||
vec![Target { node: 0, slot: 0 }, Target { node: 1, slot: 0 }],
|
||||
],
|
||||
// the one output port (C8): Sub's output field 0
|
||||
OutPort { node: 2, field: 0 },
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
Wiring it into a harness — the composite is one blueprint item, like any node; the
|
||||
7-node/6-edge hand-wiring of today's `sample_harness`
|
||||
(`crates/aura-cli/src/main.rs:42-78`) becomes:
|
||||
|
||||
```rust
|
||||
use aura_engine::{Blueprint, BlueprintNode, SourceSpec, Target, Edge};
|
||||
use aura_core::ScalarKind::F64;
|
||||
|
||||
// price -> [sma_cross] -> Exposure -> SimBroker -> 2 recording sinks
|
||||
let bp = Blueprint::new(
|
||||
vec![
|
||||
BlueprintNode::Composite(sma_cross(2, 4)), // 0 (interior: 3 nodes)
|
||||
Exposure::new(0.5).into(), // 1
|
||||
SimBroker::new(1e-4).into(), // 2
|
||||
rec_equity.into(), // 3 sink
|
||||
rec_exposure.into(), // 4 sink
|
||||
],
|
||||
// sources address blueprint-level (node, slot); a target into a composite
|
||||
// names the composite's input role
|
||||
vec![SourceSpec { kind: F64, targets: vec![
|
||||
Target { node: 0, slot: 0 }, // price -> sma_cross role 0
|
||||
Target { node: 2, slot: 1 }, // price -> SimBroker price slot
|
||||
]}],
|
||||
// blueprint-level edges
|
||||
vec![
|
||||
Edge { from: 0, to: 1, slot: 0, from_field: 0 }, // composite out -> Exposure
|
||||
Edge { from: 1, to: 2, slot: 0, from_field: 0 }, // exposure -> broker
|
||||
Edge { from: 2, to: 3, slot: 0, from_field: 0 }, // equity -> sink
|
||||
Edge { from: 1, to: 4, slot: 0, from_field: 0 }, // exposure -> sink
|
||||
],
|
||||
);
|
||||
|
||||
let mut harness = bp.bootstrap()?; // compile (inline composites) + Harness::bootstrap
|
||||
harness.run(streams);
|
||||
```
|
||||
|
||||
### The headline test: composite ≡ hand-wired, bit-for-bit (C1)
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn composite_sma_cross_runs_bit_identical_to_hand_wired() {
|
||||
let streams = sample_price_stream();
|
||||
|
||||
// (a) today's flat, hand-wired graph (the sample_harness wiring)
|
||||
let (mut flat, flat_eq, flat_ex) = hand_wired_sma_cross_harness();
|
||||
flat.run(streams.clone());
|
||||
|
||||
// (b) the same graph authored as a composite blueprint, compiled
|
||||
let (bp, comp_eq, comp_ex) = composite_sma_cross_harness();
|
||||
let mut composed = bp.bootstrap().unwrap();
|
||||
composed.run(streams);
|
||||
|
||||
// both recording sinks captured the same equity + exposure traces, bit-for-bit
|
||||
assert_eq!(flat_eq.borrow().as_slice(), comp_eq.borrow().as_slice());
|
||||
assert_eq!(flat_ex.borrow().as_slice(), comp_ex.borrow().as_slice());
|
||||
}
|
||||
```
|
||||
|
||||
### Implementation shape (secondary) — new types, before → after
|
||||
|
||||
**Before:** a graph exists only as the flat arguments to `bootstrap`; there is no
|
||||
composite (`harness.rs:30-52`, `:114`).
|
||||
|
||||
**After:** `crates/aura-engine/src/blueprint.rs` adds (sketch — exact bytes are the
|
||||
planner's job):
|
||||
|
||||
```rust
|
||||
/// Which interior (node, output-field) is a composite's single output port (C8).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct OutPort { pub node: usize, pub field: usize }
|
||||
|
||||
/// A blueprint item: a leaf node or a nested composite. Both present a declared
|
||||
/// interface (typed inputs + one output) to the enclosing graph.
|
||||
pub enum BlueprintNode {
|
||||
Leaf(Box<dyn Node>),
|
||||
Composite(Composite),
|
||||
}
|
||||
// ergonomic lift used in the examples above
|
||||
impl<N: Node + 'static> From<N> for BlueprintNode { /* Leaf(Box::new(n)) */ }
|
||||
|
||||
/// A reusable sub-graph fragment compiled away by inlining (C9/C23). NOT a Node.
|
||||
pub struct Composite {
|
||||
nodes: Vec<BlueprintNode>, // interior items, local indices
|
||||
edges: Vec<Edge>, // interior wiring, local indices
|
||||
input_roles: Vec<Vec<Target>>, // role r -> interior targets it fans into
|
||||
output: OutPort, // the one exposed output port
|
||||
}
|
||||
impl Composite {
|
||||
pub fn new(nodes: Vec<BlueprintNode>, edges: Vec<Edge>,
|
||||
input_roles: Vec<Vec<Target>>, output: OutPort) -> Self;
|
||||
/// Derived interface the enclosing graph wires against: input r's kind/
|
||||
/// lookback/firing from its role targets; output kind from the interior
|
||||
/// output port. (Derivation, not a Node impl.)
|
||||
pub fn schema(&self) -> NodeSchema;
|
||||
}
|
||||
|
||||
/// The root graph-as-data, before compilation.
|
||||
pub struct Blueprint {
|
||||
nodes: Vec<BlueprintNode>,
|
||||
sources: Vec<SourceSpec>, // targets address blueprint-level (node, slot)
|
||||
edges: Vec<Edge>, // blueprint-level indices
|
||||
}
|
||||
impl Blueprint {
|
||||
pub fn new(nodes: Vec<BlueprintNode>, sources: Vec<SourceSpec>,
|
||||
edges: Vec<Edge>) -> Self;
|
||||
/// Lower to the flat compilat: inline every composite (recursive), offset
|
||||
/// interior indices, rewrite edges, fan input roles out. Run loop unchanged.
|
||||
pub fn compile(self) -> Result<(Vec<Box<dyn Node>>, Vec<SourceSpec>, Vec<Edge>), CompileError>;
|
||||
/// Compile, then hand the flat compilat to the unchanged Harness::bootstrap.
|
||||
pub fn bootstrap(self) -> Result<Harness, CompileError>;
|
||||
}
|
||||
|
||||
/// A construction-phase fault, caught before the flat compilat reaches bootstrap.
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub enum CompileError {
|
||||
BadInteriorIndex, // an interior edge/role/output index is out of range
|
||||
RoleKindMismatch { role: usize }, // a role fans into interior slots of differing kinds
|
||||
OutputPortOutOfRange, // the output port names a missing interior node/field
|
||||
Bootstrap(BootstrapError), // the lowered flat compilat failed bootstrap's checks
|
||||
}
|
||||
```
|
||||
|
||||
`Harness::bootstrap`, the run loop, `Edge`, `Target`, `SourceSpec`, and the `Node`
|
||||
trait are **unchanged**.
|
||||
|
||||
## Components
|
||||
|
||||
- **`OutPort`** — `{ node, field }`, a composite's single output port (C8).
|
||||
- **`BlueprintNode`** — `Leaf(Box<dyn Node>) | Composite(Composite)`, plus a
|
||||
`From<N: Node>` lift for ergonomic authoring.
|
||||
- **`Composite`** — interior items, interior edges, input roles, output port;
|
||||
`new()` and a *derived* `schema()` (kinds/lookback/firing from role targets +
|
||||
output port). Never `eval`'d.
|
||||
- **`Blueprint`** — root items + sources + edges (blueprint-level indices);
|
||||
`new()`, `compile()`, `bootstrap()`.
|
||||
- **The inliner** (`compile`'s core) — recursive index lowering: append interior
|
||||
items at an offset, rewrite interior edges, resolve edges into a composite via
|
||||
its input roles (fan-out), resolve edges out of a composite to its output port.
|
||||
- **`CompileError`** — construction-phase faults, with `Bootstrap(BootstrapError)`
|
||||
for the lowered-compilat failure.
|
||||
|
||||
## Data flow
|
||||
|
||||
Authoring → `Blueprint` (graph-as-data, blueprint indices) → `compile()`:
|
||||
|
||||
1. Walk blueprint items. A `Leaf` lowers to itself (one flat node). A `Composite`
|
||||
lowers by **inlining**: its interior items are appended to the flat array at
|
||||
`base = flat.len()`; interior edges are emitted as flat edges with both
|
||||
endpoints `+ base`; a nested interior composite recurses (compiled before its
|
||||
parent's edges resolve, so its own boundary is already gone).
|
||||
2. **Edge into a composite** (`to: c, slot: s`, where item `c` is a composite):
|
||||
replaced by one flat edge per target in `input_roles[s]` — the fan-out (price →
|
||||
both SMAs). Same rule for a `SourceSpec.targets` entry whose `node` is a
|
||||
composite.
|
||||
3. **Edge out of a composite** (`from: c`, `from_field: 0`): rewritten to
|
||||
`from: base_c + output.node`, `from_field: output.field`.
|
||||
4. Result: flat `(nodes, sources, edges)` with no composites, wired by raw index.
|
||||
`Blueprint::bootstrap` hands these to the unchanged `Harness::bootstrap`, whose
|
||||
kind-check + Kahn cycle-check (`harness.rs:142-192`) validate the compilat.
|
||||
|
||||
Then `Harness::run` drives the flat compilat exactly as today (C1 determinism
|
||||
preserved — the compilat is the same flat topology the hand-wiring produced).
|
||||
|
||||
## Error handling
|
||||
|
||||
- `compile()` returns `CompileError` for construction-phase faults *before*
|
||||
anything reaches `bootstrap`: an out-of-range interior edge/role/output index
|
||||
(`BadInteriorIndex`), a role fanning into interior slots of differing scalar
|
||||
kinds (`RoleKindMismatch`), an output port naming a missing interior node/field
|
||||
(`OutputPortOutOfRange`).
|
||||
- The lowered flat compilat is validated by `bootstrap`'s **existing** checks; any
|
||||
`BootstrapError` (kind mismatch, bad index, cycle) is wrapped as
|
||||
`CompileError::Bootstrap(..)`. No duplication of bootstrap's validation in the
|
||||
inliner — the compilat is checked once, where it always was (C7/C8).
|
||||
- A composite's *derived* `schema()` is what the enclosing graph kind-checks
|
||||
against; a same-name role fanning to mismatched kinds is caught at compile, not
|
||||
hidden until run.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Unit (inliner):** a single composite lowers with correct index offset and edge
|
||||
rewrite; an input role fans to ≥2 interior targets (the price → both-SMAs case);
|
||||
an edge out of a composite resolves to its output port; a **nested** composite
|
||||
(composite-in-composite) inlines correctly.
|
||||
- **Unit (errors):** `BadInteriorIndex`, `RoleKindMismatch`, `OutputPortOutOfRange`
|
||||
each provoked by a malformed composite; a lowered compilat with a kind mismatch
|
||||
or cycle surfaces as `CompileError::Bootstrap(..)`.
|
||||
- **Integration (headline, C1):** the SMA-cross composite, compiled and run,
|
||||
produces equity + exposure sink traces **bit-identical** to the hand-wired flat
|
||||
graph (`composite_sma_cross_runs_bit_identical_to_hand_wired`).
|
||||
- **Regression:** the existing `bootstrap`/`run` tests stay green unchanged (the
|
||||
run loop and signatures are untouched).
|
||||
|
||||
## Forward-looking design notes (no code this cycle)
|
||||
|
||||
The representation is shaped so the C23 passes are later cheap, but **none is built
|
||||
here**: (1) interior structure stays inspectable so a future CSE pass can hash node
|
||||
identity (`type-tag + bound params + input-sources`) — building that identity, and
|
||||
per-node params, is C8 work not yet in the schema (`node.rs:6-7`); (2) sink
|
||||
reachability is readable from the flat edge list so a future DCE pass is trivial;
|
||||
(3) per-node provenance survives compilation so sweep-invariance analysis is
|
||||
possible once params and the sweep orchestration land (C12/C21).
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
1. `Blueprint`, `BlueprintNode`, `Composite`, `OutPort`, `CompileError` exist in
|
||||
`crates/aura-engine/src/blueprint.rs`; `Blueprint::{compile, bootstrap}` work.
|
||||
2. A composite wires ≥2 interior nodes, exposes exactly one output port (C8), and
|
||||
fans one input role to ≥2 interior targets.
|
||||
3. A nested composite (composite-in-composite) inlines correctly (C9 self-
|
||||
application).
|
||||
4. The SMA-cross composite produces a run **bit-identical** to the hand-wired
|
||||
graph (C1) — the headline test passes.
|
||||
5. `compile()` rejects malformed wiring with a typed `CompileError`; the lowered
|
||||
flat compilat is validated by `bootstrap`'s existing kind/cycle checks (no
|
||||
re-implementation).
|
||||
6. `Harness::bootstrap`, `Harness::run`, `Edge`, `Target`, `SourceSpec`, and the
|
||||
`Node` trait are **unchanged**; the run loop is untouched.
|
||||
7. **Zero** new external dependencies (C16); pure-Rust, hand-rolled.
|
||||
8. Non-goals respected: no optimisation pass, no egg/egglog, no named-handle
|
||||
ergonomics, no `aura graph`.
|
||||
Reference in New Issue
Block a user