Files
Aura/docs/specs/0018-composite-multi-output-record.md
T
Brummel 21c1621bd0 docs,engine: drop coined "compilat", use FlatGraph / "flat graph"
"Compilat" (German "Kompilat") was a coined noun for the product of the
bootstrap compilation — neither English nor a natural fit. The runtime
artifact already has a code identifier for exactly this thing: the
`FlatGraph` struct (harness.rs). Replace the coinage with that identifier:

- prose mentions          -> "flat graph" (mirrors the type, reads plainly)
- definitional anchors    -> `FlatGraph` (C11, C23, the running-graph line)
- `render_compilat`       -> `render_flat_graph` (historical render symbol;
                             keeps the `render_blueprint` / `render_flat_graph`
                             source-vs-product pairing)
- `intra-compilat`        -> `intra-graph`
- `from_compilat` (test)  -> `from_flat`

"compilation" / "re-compilation" / "bootstrap-as-compilation" (the process,
ordinary English) are deliberately left untouched. Behaviour-preserving:
only comments, design ledger, specs/plans, one test-local variable and its
assert messages change. Full workspace test suite green; clippy clean.
2026-06-14 17:02:15 +02:00

390 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Composite multi-output record — Design Spec
**Date:** 2026-06-08
**Status:** Draft — awaiting user spec review
**Authors:** orchestrator + Claude
**Closes:** #40
## Goal
A composite (`aura_engine::Composite`) currently exposes exactly **one** output
field (`output: OutPort`, a single interior `(node, field)`). Multi-line
indicators — MACD (`macd` / `signal` / `histogram`), Bollinger
(`upper` / `mid` / `lower`), Stochastic (`%K` / `%D`), Ichimoku (5 lines) —
therefore cannot be authored as a composition and re-exported as a unit: only
one line escapes the boundary. The MACD proof-of-concept (commit `d8b2a28`,
`crates/aura-cli/src/main.rs:206`) is forced to expose only the histogram —
`OutPort { node: 4, field: 0 }` — discarding the MACD line and signal line.
This cycle makes a composite's output a **named, ordered, multi-field record**:
`output: Vec<OutField { node, field, name }>`. Each entry re-exports one interior
`(node, output-field)` under a boundary name. A consumer selects which
re-exported field it reads via the existing `Edge::from_field` — exactly the
mechanism that already works for multi-output **leaf** producers.
This is a **boundary completion**, not an invariant change (see Architecture).
## Architecture
### Why this does not touch C8 / C7 / C4
The substrate is already multi-field everywhere *except* the composite boundary:
- `NodeSchema.output` is `Vec<FieldSpec>` — a K-column record through **one**
port (OHLCV = 5 columns, one port, supported today; C8 realization, cycle
0005). "One output **port**" (C8) and "≤1 record per `eval`" are about the
**port** and the **row**, never the column count.
- `Edge::from_field` already selects *which* output column a consumer reads.
- The inliner already routes `from_field` for **leaf** producers
(`rewrite_edge`, the `ItemLowering::Leaf` arm at `blueprint.rs:369`): a
multi-output leaf works today.
A 3-line MACD is therefore **one** record of 3 fields, **one** port, **one** row
per `eval` — identical in kind to OHLCV. The only cap is the *composite* boundary,
which asserts `from_field == 0` in three spots. Lifting that cap completes the
boundary; it does not widen the node contract.
- **C8** (one output port; record of 1..K base columns) — unchanged. A composite
re-exports a record, exactly as a leaf already may.
- **C7** (four scalar base types, SoA) — unchanged. Each re-exported field is one
base column; the record is a bundle of base columns, as today.
- **C4** (one record per cycle) — unchanged. One `eval`, one row, K columns,
co-fresh by construction.
- **C23** (flat graph wired by raw index; names are non-load-bearing debug
symbols) — **honoured**: the re-export *names* live at the blueprint boundary
only. The flat graph lowering drops them and carries raw `(node, field)` indices,
exactly as `FieldSpec.name` and `Composite.name` are dropped at inline today.
A strategy composite still outputs **one** exposure (C10/C7): it is simply a
composite whose output record happens to have one field. Indicator composites
have K fields, strategy composites have 1 — same mechanism, different arity.
### Scope boundary (output half only)
This cycle is the **output** half of giving a composite a typed, named boundary
signature. The output type is born **name-ready** (`name` is a field of
`OutField` from the start) so it is not reopened later merely to add names. The
**legibility** half — composite-level param aliasing and input-role naming on the
already-functioning param / input edges — is issue **#41**, which depends on this
cycle and reuses the same named-projection vocabulary. Input roles keep rendering
as `in:<index>` here; naming them is #41's job.
## Concrete code shapes
### Worked author example (the acceptance evidence)
The MACD composite, re-authored to re-export all three lines as a named record.
Today (`crates/aura-cli/src/main.rs:185`) it ends with
`OutPort { node: 4, field: 0 }` — only the histogram. After:
```rust
fn macd(name: &str) -> Composite {
Composite::new(
name,
vec![
Ema::factory().into(), // 0 fast EMA
Ema::factory().into(), // 1 slow EMA
Sub::factory().into(), // 2 MACD line = fast slow
Ema::factory().into(), // 3 signal EMA of the MACD line
Sub::factory().into(), // 4 histogram = MACD line signal
],
vec![
Edge { from: 0, to: 2, slot: 0, from_field: 0 }, // fast → line[0]
Edge { from: 1, to: 2, slot: 1, from_field: 0 }, // slow → line[1]
Edge { from: 2, to: 3, slot: 0, from_field: 0 }, // line → signal EMA
Edge { from: 2, to: 4, slot: 0, from_field: 0 }, // line → histogram[0]
Edge { from: 3, to: 4, slot: 1, from_field: 0 }, // signal → histogram[1]
],
vec![vec![
Target { node: 0, slot: 0 }, // price → fast EMA
Target { node: 1, slot: 0 }, // price → slow EMA
]],
// BEFORE: OutPort { node: 4, field: 0 } — only the histogram escaped
// AFTER: a 3-field record, all three lines re-exported under names
vec![
OutField { node: 2, field: 0, name: "macd".into() }, // the MACD line
OutField { node: 3, field: 0, name: "signal".into() }, // the signal line
OutField { node: 4, field: 0, name: "histogram".into() }, // the histogram
],
)
}
```
A downstream consumer now selects a re-exported field by `from_field` — the same
selector a leaf record already uses. E.g. a strategy that trades the histogram
reads `from_field: 2`, while a chart sink records the signal line via
`from_field: 1`:
```rust
// macd composite is interior item 0 of the enclosing blueprint
Edge { from: 0, to: /*Exposure*/ 1, slot: 0, from_field: 2 }, // histogram → strategy
Edge { from: 0, to: /*signal sink*/ 5, slot: 0, from_field: 1 }, // signal line → sink
```
This is the feature-acceptance evidence: the author reaches for the composition
exactly as for any record producer, and a behaviour the engine previously made
impossible (re-exporting >1 interior line) is now expressible without touching
the node contract.
### Change 1 — the output element type (`blueprint.rs:18`)
`OutPort` is replaced by `OutField`, carrying the boundary name. It is the only
use of `OutPort` (a composite's output port), so this is a rename + one added
field, not a parallel type.
```rust
// BEFORE
/// Which interior `(node, output-field)` is a composite's single output port (C8).
pub struct OutPort {
pub node: usize,
pub field: usize,
}
// AFTER
/// One re-exported field of a composite's output record: an interior
/// `(node, output-field)` surfaced at the boundary under `name`. `name` is a
/// non-load-bearing render/debug symbol (C23) — like `FieldSpec.name` and
/// `Composite.name`, it does not reach the flat graph.
pub struct OutField {
pub node: usize,
pub field: usize,
pub name: String,
}
```
`name` is `String` (not `&'static str`), consistent with the sibling
`Composite.name: String` and with author-supplied, possibly dynamic names. It is
non-load-bearing either way.
### Change 2 — `Composite.output` and `Composite::new` (`blueprint.rs:43`, `:56`)
```rust
// BEFORE
pub struct Composite {
name: String,
nodes: Vec<BlueprintNode>,
edges: Vec<Edge>,
input_roles: Vec<Vec<Target>>,
output: OutPort,
}
// AFTER — the output record = ordered re-exported interior fields
pub struct Composite {
name: String,
nodes: Vec<BlueprintNode>,
edges: Vec<Edge>,
input_roles: Vec<Vec<Target>>,
output: Vec<OutField>,
}
```
`Composite::new(… output: Vec<OutField>)` and the accessor
`output(&self) -> &[OutField]` (was `-> OutPort`) change in step.
### Change 3 — the lowering (`blueprint.rs:243`)
```rust
// BEFORE
enum ItemLowering {
Leaf { index: usize },
Composite { output: (usize, usize), roles: Vec<Vec<Target>> },
}
// AFTER — the composite lowers to K flat (node, field) producers, one per
// re-exported field, in declared order. Names are dropped here (C23).
enum ItemLowering {
Leaf { index: usize },
Composite { output: Vec<(usize, usize)>, roles: Vec<Vec<Target>> },
}
```
### Change 4 — `inline_composite` resolves each re-exported field (`blueprint.rs:292`)
The single `output` resolution (`blueprint.rs:304``333`) becomes a loop over the
record. Each `OutField.node` must be in range; each resolves to a flat
`(node, field)` and is range-checked against that producer's actual output arity
— the leaf arm checks `field` against `schema().output.len()`, the nested arm
indexes the nested composite's output `Vec` (instead of asserting `field == 0`).
```rust
// BEFORE (single port; nested arm asserts field == 0)
if output.node >= item_count {
return Err(CompileError::OutputPortOutOfRange);
}
// ... after lowering interior ...
let out = match &interior[output.node] {
ItemLowering::Leaf { index } => {
if output.field >= flat_nodes[*index].schema().output.len() {
return Err(CompileError::OutputPortOutOfRange);
}
(*index, output.field)
}
ItemLowering::Composite { output: nested, .. } => {
if output.field != 0 { // <-- the cap
return Err(CompileError::OutputPortOutOfRange);
}
*nested
}
};
Ok(ItemLowering::Composite { output: out, roles })
// AFTER (K fields; nested arm indexes by field)
let mut out: Vec<(usize, usize)> = Vec::with_capacity(output.len());
for of in &output {
if of.node >= item_count {
return Err(CompileError::OutputPortOutOfRange);
}
let resolved = match &interior[of.node] {
ItemLowering::Leaf { index } => {
if of.field >= flat_nodes[*index].schema().output.len() {
return Err(CompileError::OutputPortOutOfRange);
}
(*index, of.field)
}
ItemLowering::Composite { output: nested, .. } => {
*nested.get(of.field).ok_or(CompileError::OutputPortOutOfRange)?
}
};
out.push(resolved);
}
Ok(ItemLowering::Composite { output: out, roles })
```
Note the in-range check on `output.node` moves *inside* the loop (it was a single
pre-check before lowering); the rest of `inline_composite` — interior lowering,
edge rewrite, role resolution — is unchanged.
### Change 5 — `rewrite_edge` indexes the composite output by `from_field` (`blueprint.rs:360`)
The composite arm (`blueprint.rs:375``381`) drops the `from_field == 0`
assertion and indexes the output `Vec` with a range-check — identical in shape to
the leaf arm directly above it.
```rust
// BEFORE
ItemLowering::Composite { output, .. } => {
if e.from_field != 0 { // <-- the cap
return Err(CompileError::BadInteriorIndex);
}
*output
}
// AFTER
ItemLowering::Composite { output, .. } => {
*output.get(e.from_field).ok_or(CompileError::BadInteriorIndex)?
}
```
### Change 6 — render K named output markers (`crates/aura-cli/src/graph.rs:124`)
`render_definition` emits one `[out:<name>]` marker per re-exported field, each
wired from its interior source — analogous to the per-role `[in:k]` markers. The
output **name** is shown (this cycle's deliverable); input roles stay
index-labelled (`in:<index>`) until #41.
```rust
// BEFORE — one unnamed [out] marker
let out_id = labels.len();
labels.push("out".to_string());
edges.push((c.output().node, out_id));
// AFTER — one [out:<name>] marker per re-exported field
for of in c.output() {
let out_id = labels.len();
labels.push(format!("out:{}", of.name));
edges.push((of.node, out_id));
}
```
## Components
| Component | File | Change |
|---|---|---|
| `OutField` type (was `OutPort`) | `crates/aura-engine/src/blueprint.rs:18` | rename + `name: String` field |
| `Composite` struct + `new` + `output()` | `blueprint.rs:43`, `:56`, `:83` | `OutPort``Vec<OutField>` |
| `ItemLowering::Composite.output` | `blueprint.rs:243` | `(usize,usize)``Vec<(usize,usize)>` |
| `inline_composite` | `blueprint.rs:292` | loop over output record; nested arm indexes by field |
| `rewrite_edge` (composite arm) | `blueprint.rs:360` | index `output[from_field]` + range-check |
| `lib.rs` re-export | `crates/aura-engine/src/lib.rs:38` | `OutPort``OutField` |
| `render_definition` | `crates/aura-cli/src/graph.rs:105` | K `[out:<name>]` markers |
| MACD PoC author site | `crates/aura-cli/src/main.rs:185` | re-export all 3 lines as a record |
| All `OutPort { … }` literals | `blueprint.rs` tests, `main.rs:130,206,391,398` | → `OutField { …, name }` |
## Data flow
Authoring → compile → run is unchanged in shape; only the boundary arity widens.
1. **Author:** a composite declares `output: vec![OutField { node, field, name }, …]`
— an ordered record of re-exported interior fields.
2. **Compile (`inline_composite`):** each `OutField` resolves to a flat
`(node, field)`; the composite lowers to `ItemLowering::Composite { output:
Vec<(node,field)>, … }`. Names are dropped (C23).
3. **Edge rewrite (`rewrite_edge`):** a consumer edge reading the composite with
`from_field = k` resolves to `output[k]` — the k-th re-exported flat
`(node, field)`. Out-of-range `k``BadInteriorIndex`.
4. **Bootstrap:** the flat graph is unchanged in kind — ordinary
`(from_node, from_field) → (to_node, slot)` edges. Existing per-field
kind-checks at bootstrap (`slot_kind`) apply to each re-exported field as they
already do for any producer field. No new kind rule.
## Error handling
- `CompileError::OutputPortOutOfRange` — generalized from "the output port names
a missing interior node or output field" to "an output **record entry** names a
missing interior node or output field". Raised per `OutField` when its `node` is
out of interior range, its `field` exceeds a leaf producer's output arity, or
its `field` exceeds a nested composite's re-exported arity. (No new variant; the
existing one widens.)
- `CompileError::BadInteriorIndex` — a consumer edge whose `from_field` indexes
past the composite's output record (was: any non-zero `from_field` on a
composite). Same variant, range-checked instead of `== 0`.
- An **empty** output record (`output: vec![]`) is structurally a composite no
edge can read from (every `from_field` is out of range → `BadInteriorIndex`),
mirroring a leaf sink's `output: vec![]`. Not specially rejected; it is simply
unreadable, which is the correct semantics.
## Testing strategy
Unit tests in `blueprint.rs` (alongside the existing compile tests):
1. **Multi-output happy path:** a composite re-exporting 2 interior fields; two
downstream consumers read `from_field: 0` and `from_field: 1`; assert the
flat graph wires each consumer to the correct distinct flat `(node, field)`.
2. **Nested multi-output:** an outer composite re-exporting two fields of an
**inner** multi-output composite (exercises the nested arm's
`nested.get(field)`); assert correct flat resolution.
3. **Out-of-range re-export (`OutputPortOutOfRange`):** an `OutField` whose
`field` exceeds the interior producer's arity → `OutputPortOutOfRange`.
4. **Out-of-range consume (`BadInteriorIndex`):** a consumer edge with
`from_field` past the output record length → `BadInteriorIndex`.
5. **Single-field record is the old behaviour:** a composite with a one-field
output record (e.g. `sma_cross`) compiles identically to today — the
regression guard that strategy composites are unaffected.
Render test in `crates/aura-cli` (alongside the existing `[in:0]`/`[out]`
assertion at `main.rs:377`): assert a multi-output composite's definition renders
`[out:<name>]` markers for each re-exported field.
The existing `cargo test --workspace` suite is the regression boundary; all
current composite tests update their `OutPort { … }` literals to
`OutField { …, name }` and must stay green (single-field records are behaviour-
preserving).
## Acceptance criteria
1. `Composite::output` is `Vec<OutField>`; a composite re-exports K named interior
fields as one output record.
2. A downstream consumer selects a re-exported field via `Edge::from_field`,
range-checked at compile; out-of-range consume → `BadInteriorIndex`,
out-of-range re-export → `OutputPortOutOfRange`.
3. Nested multi-output composites resolve correctly (outer re-exports inner
fields by index).
4. The MACD PoC re-exports all three lines (`macd` / `signal` / `histogram`) as a
record; the histogram remains reachable as one selected field.
5. `aura graph` renders K `[out:<name>]` markers per multi-output composite.
6. Single-field output records (every strategy composite, `sma_cross`) compile
and run identically to today — C8 / C7 / C4 untouched; the flat graph for an
unchanged blueprint is byte-identical.
7. `cargo build --workspace`, `cargo test --workspace`, and
`cargo clippy --workspace --all-targets -- -D warnings` are green.