diff --git a/docs/specs/0018-composite-multi-output-record.md b/docs/specs/0018-composite-multi-output-record.md new file mode 100644 index 0000000..6b95108 --- /dev/null +++ b/docs/specs/0018-composite-multi-output-record.md @@ -0,0 +1,389 @@ +# Composite multi-output record — Design Spec + +**Date:** 2026-06-08 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude +**Closes:** #40 + +## Goal + +A composite (`aura_engine::Composite`) currently exposes exactly **one** output +field (`output: OutPort`, a single interior `(node, field)`). Multi-line +indicators — MACD (`macd` / `signal` / `histogram`), Bollinger +(`upper` / `mid` / `lower`), Stochastic (`%K` / `%D`), Ichimoku (5 lines) — +therefore cannot be authored as a composition and re-exported as a unit: only +one line escapes the boundary. The MACD proof-of-concept (commit `d8b2a28`, +`crates/aura-cli/src/main.rs:206`) is forced to expose only the histogram — +`OutPort { node: 4, field: 0 }` — discarding the MACD line and signal line. + +This cycle makes a composite's output a **named, ordered, multi-field record**: +`output: Vec`. Each entry re-exports one interior +`(node, output-field)` under a boundary name. A consumer selects which +re-exported field it reads via the existing `Edge::from_field` — exactly the +mechanism that already works for multi-output **leaf** producers. + +This is a **boundary completion**, not an invariant change (see Architecture). + +## Architecture + +### Why this does not touch C8 / C7 / C4 + +The substrate is already multi-field everywhere *except* the composite boundary: + +- `NodeSchema.output` is `Vec` — a K-column record through **one** + port (OHLCV = 5 columns, one port, supported today; C8 realization, cycle + 0005). "One output **port**" (C8) and "≤1 record per `eval`" are about the + **port** and the **row**, never the column count. +- `Edge::from_field` already selects *which* output column a consumer reads. +- The inliner already routes `from_field` for **leaf** producers + (`rewrite_edge`, the `ItemLowering::Leaf` arm at `blueprint.rs:369`): a + multi-output leaf works today. + +A 3-line MACD is therefore **one** record of 3 fields, **one** port, **one** row +per `eval` — identical in kind to OHLCV. The only cap is the *composite* boundary, +which asserts `from_field == 0` in three spots. Lifting that cap completes the +boundary; it does not widen the node contract. + +- **C8** (one output port; record of 1..K base columns) — unchanged. A composite + re-exports a record, exactly as a leaf already may. +- **C7** (four scalar base types, SoA) — unchanged. Each re-exported field is one + base column; the record is a bundle of base columns, as today. +- **C4** (one record per cycle) — unchanged. One `eval`, one row, K columns, + co-fresh by construction. +- **C23** (compilat wired by raw index; names are non-load-bearing debug + symbols) — **honoured**: the re-export *names* live at the blueprint boundary + only. The compilat lowering drops them and carries raw `(node, field)` indices, + exactly as `FieldSpec.name` and `Composite.name` are dropped at inline today. + +A strategy composite still outputs **one** exposure (C10/C7): it is simply a +composite whose output record happens to have one field. Indicator composites +have K fields, strategy composites have 1 — same mechanism, different arity. + +### Scope boundary (output half only) + +This cycle is the **output** half of giving a composite a typed, named boundary +signature. The output type is born **name-ready** (`name` is a field of +`OutField` from the start) so it is not reopened later merely to add names. The +**legibility** half — composite-level param aliasing and input-role naming on the +already-functioning param / input edges — is issue **#41**, which depends on this +cycle and reuses the same named-projection vocabulary. Input roles keep rendering +as `in:` here; naming them is #41's job. + +## Concrete code shapes + +### Worked author example (the acceptance evidence) + +The MACD composite, re-authored to re-export all three lines as a named record. +Today (`crates/aura-cli/src/main.rs:185`) it ends with +`OutPort { node: 4, field: 0 }` — only the histogram. After: + +```rust +fn macd(name: &str) -> Composite { + Composite::new( + name, + vec![ + Ema::factory().into(), // 0 fast EMA + Ema::factory().into(), // 1 slow EMA + Sub::factory().into(), // 2 MACD line = fast − slow + Ema::factory().into(), // 3 signal EMA of the MACD line + Sub::factory().into(), // 4 histogram = MACD line − signal + ], + vec![ + Edge { from: 0, to: 2, slot: 0, from_field: 0 }, // fast → line[0] + Edge { from: 1, to: 2, slot: 1, from_field: 0 }, // slow → line[1] + Edge { from: 2, to: 3, slot: 0, from_field: 0 }, // line → signal EMA + Edge { from: 2, to: 4, slot: 0, from_field: 0 }, // line → histogram[0] + Edge { from: 3, to: 4, slot: 1, from_field: 0 }, // signal → histogram[1] + ], + vec![vec![ + Target { node: 0, slot: 0 }, // price → fast EMA + Target { node: 1, slot: 0 }, // price → slow EMA + ]], + // BEFORE: OutPort { node: 4, field: 0 } — only the histogram escaped + // AFTER: a 3-field record, all three lines re-exported under names + vec![ + OutField { node: 2, field: 0, name: "macd".into() }, // the MACD line + OutField { node: 3, field: 0, name: "signal".into() }, // the signal line + OutField { node: 4, field: 0, name: "histogram".into() }, // the histogram + ], + ) +} +``` + +A downstream consumer now selects a re-exported field by `from_field` — the same +selector a leaf record already uses. E.g. a strategy that trades the histogram +reads `from_field: 2`, while a chart sink records the signal line via +`from_field: 1`: + +```rust +// macd composite is interior item 0 of the enclosing blueprint +Edge { from: 0, to: /*Exposure*/ 1, slot: 0, from_field: 2 }, // histogram → strategy +Edge { from: 0, to: /*signal sink*/ 5, slot: 0, from_field: 1 }, // signal line → sink +``` + +This is the feature-acceptance evidence: the author reaches for the composition +exactly as for any record producer, and a behaviour the engine previously made +impossible (re-exporting >1 interior line) is now expressible without touching +the node contract. + +### Change 1 — the output element type (`blueprint.rs:18`) + +`OutPort` is replaced by `OutField`, carrying the boundary name. It is the only +use of `OutPort` (a composite's output port), so this is a rename + one added +field, not a parallel type. + +```rust +// BEFORE +/// Which interior `(node, output-field)` is a composite's single output port (C8). +pub struct OutPort { + pub node: usize, + pub field: usize, +} + +// AFTER +/// One re-exported field of a composite's output record: an interior +/// `(node, output-field)` surfaced at the boundary under `name`. `name` is a +/// non-load-bearing render/debug symbol (C23) — like `FieldSpec.name` and +/// `Composite.name`, it does not reach the compilat. +pub struct OutField { + pub node: usize, + pub field: usize, + pub name: String, +} +``` + +`name` is `String` (not `&'static str`), consistent with the sibling +`Composite.name: String` and with author-supplied, possibly dynamic names. It is +non-load-bearing either way. + +### Change 2 — `Composite.output` and `Composite::new` (`blueprint.rs:43`, `:56`) + +```rust +// BEFORE +pub struct Composite { + name: String, + nodes: Vec, + edges: Vec, + input_roles: Vec>, + output: OutPort, +} + +// AFTER — the output record = ordered re-exported interior fields +pub struct Composite { + name: String, + nodes: Vec, + edges: Vec, + input_roles: Vec>, + output: Vec, +} +``` + +`Composite::new(… output: Vec)` and the accessor +`output(&self) -> &[OutField]` (was `-> OutPort`) change in step. + +### Change 3 — the lowering (`blueprint.rs:243`) + +```rust +// BEFORE +enum ItemLowering { + Leaf { index: usize }, + Composite { output: (usize, usize), roles: Vec> }, +} + +// AFTER — the composite lowers to K flat (node, field) producers, one per +// re-exported field, in declared order. Names are dropped here (C23). +enum ItemLowering { + Leaf { index: usize }, + Composite { output: Vec<(usize, usize)>, roles: Vec> }, +} +``` + +### Change 4 — `inline_composite` resolves each re-exported field (`blueprint.rs:292`) + +The single `output` resolution (`blueprint.rs:304`–`333`) becomes a loop over the +record. Each `OutField.node` must be in range; each resolves to a flat +`(node, field)` and is range-checked against that producer's actual output arity +— the leaf arm checks `field` against `schema().output.len()`, the nested arm +indexes the nested composite's output `Vec` (instead of asserting `field == 0`). + +```rust +// BEFORE (single port; nested arm asserts field == 0) +if output.node >= item_count { + return Err(CompileError::OutputPortOutOfRange); +} +// ... after lowering interior ... +let out = match &interior[output.node] { + ItemLowering::Leaf { index } => { + if output.field >= flat_nodes[*index].schema().output.len() { + return Err(CompileError::OutputPortOutOfRange); + } + (*index, output.field) + } + ItemLowering::Composite { output: nested, .. } => { + if output.field != 0 { // <-- the cap + return Err(CompileError::OutputPortOutOfRange); + } + *nested + } +}; +Ok(ItemLowering::Composite { output: out, roles }) + +// AFTER (K fields; nested arm indexes by field) +let mut out: Vec<(usize, usize)> = Vec::with_capacity(output.len()); +for of in &output { + if of.node >= item_count { + return Err(CompileError::OutputPortOutOfRange); + } + let resolved = match &interior[of.node] { + ItemLowering::Leaf { index } => { + if of.field >= flat_nodes[*index].schema().output.len() { + return Err(CompileError::OutputPortOutOfRange); + } + (*index, of.field) + } + ItemLowering::Composite { output: nested, .. } => { + *nested.get(of.field).ok_or(CompileError::OutputPortOutOfRange)? + } + }; + out.push(resolved); +} +Ok(ItemLowering::Composite { output: out, roles }) +``` + +Note the in-range check on `output.node` moves *inside* the loop (it was a single +pre-check before lowering); the rest of `inline_composite` — interior lowering, +edge rewrite, role resolution — is unchanged. + +### Change 5 — `rewrite_edge` indexes the composite output by `from_field` (`blueprint.rs:360`) + +The composite arm (`blueprint.rs:375`–`381`) drops the `from_field == 0` +assertion and indexes the output `Vec` with a range-check — identical in shape to +the leaf arm directly above it. + +```rust +// BEFORE +ItemLowering::Composite { output, .. } => { + if e.from_field != 0 { // <-- the cap + return Err(CompileError::BadInteriorIndex); + } + *output +} + +// AFTER +ItemLowering::Composite { output, .. } => { + *output.get(e.from_field).ok_or(CompileError::BadInteriorIndex)? +} +``` + +### Change 6 — render K named output markers (`crates/aura-cli/src/graph.rs:124`) + +`render_definition` emits one `[out:]` marker per re-exported field, each +wired from its interior source — analogous to the per-role `[in:k]` markers. The +output **name** is shown (this cycle's deliverable); input roles stay +index-labelled (`in:`) until #41. + +```rust +// BEFORE — one unnamed [out] marker +let out_id = labels.len(); +labels.push("out".to_string()); +edges.push((c.output().node, out_id)); + +// AFTER — one [out:] marker per re-exported field +for of in c.output() { + let out_id = labels.len(); + labels.push(format!("out:{}", of.name)); + edges.push((of.node, out_id)); +} +``` + +## Components + +| Component | File | Change | +|---|---|---| +| `OutField` type (was `OutPort`) | `crates/aura-engine/src/blueprint.rs:18` | rename + `name: String` field | +| `Composite` struct + `new` + `output()` | `blueprint.rs:43`, `:56`, `:83` | `OutPort` → `Vec` | +| `ItemLowering::Composite.output` | `blueprint.rs:243` | `(usize,usize)` → `Vec<(usize,usize)>` | +| `inline_composite` | `blueprint.rs:292` | loop over output record; nested arm indexes by field | +| `rewrite_edge` (composite arm) | `blueprint.rs:360` | index `output[from_field]` + range-check | +| `lib.rs` re-export | `crates/aura-engine/src/lib.rs:38` | `OutPort` → `OutField` | +| `render_definition` | `crates/aura-cli/src/graph.rs:105` | K `[out:]` markers | +| MACD PoC author site | `crates/aura-cli/src/main.rs:185` | re-export all 3 lines as a record | +| All `OutPort { … }` literals | `blueprint.rs` tests, `main.rs:130,206,391,398` | → `OutField { …, name }` | + +## Data flow + +Authoring → compile → run is unchanged in shape; only the boundary arity widens. + +1. **Author:** a composite declares `output: vec![OutField { node, field, name }, …]` + — an ordered record of re-exported interior fields. +2. **Compile (`inline_composite`):** each `OutField` resolves to a flat + `(node, field)`; the composite lowers to `ItemLowering::Composite { output: + Vec<(node,field)>, … }`. Names are dropped (C23). +3. **Edge rewrite (`rewrite_edge`):** a consumer edge reading the composite with + `from_field = k` resolves to `output[k]` — the k-th re-exported flat + `(node, field)`. Out-of-range `k` → `BadInteriorIndex`. +4. **Bootstrap:** the flat compilat is unchanged in kind — ordinary + `(from_node, from_field) → (to_node, slot)` edges. Existing per-field + kind-checks at bootstrap (`slot_kind`) apply to each re-exported field as they + already do for any producer field. No new kind rule. + +## Error handling + +- `CompileError::OutputPortOutOfRange` — generalized from "the output port names + a missing interior node or output field" to "an output **record entry** names a + missing interior node or output field". Raised per `OutField` when its `node` is + out of interior range, its `field` exceeds a leaf producer's output arity, or + its `field` exceeds a nested composite's re-exported arity. (No new variant; the + existing one widens.) +- `CompileError::BadInteriorIndex` — a consumer edge whose `from_field` indexes + past the composite's output record (was: any non-zero `from_field` on a + composite). Same variant, range-checked instead of `== 0`. +- An **empty** output record (`output: vec![]`) is structurally a composite no + edge can read from (every `from_field` is out of range → `BadInteriorIndex`), + mirroring a leaf sink's `output: vec![]`. Not specially rejected; it is simply + unreadable, which is the correct semantics. + +## Testing strategy + +Unit tests in `blueprint.rs` (alongside the existing compile tests): + +1. **Multi-output happy path:** a composite re-exporting 2 interior fields; two + downstream consumers read `from_field: 0` and `from_field: 1`; assert the + compilat wires each consumer to the correct distinct flat `(node, field)`. +2. **Nested multi-output:** an outer composite re-exporting two fields of an + **inner** multi-output composite (exercises the nested arm's + `nested.get(field)`); assert correct flat resolution. +3. **Out-of-range re-export (`OutputPortOutOfRange`):** an `OutField` whose + `field` exceeds the interior producer's arity → `OutputPortOutOfRange`. +4. **Out-of-range consume (`BadInteriorIndex`):** a consumer edge with + `from_field` past the output record length → `BadInteriorIndex`. +5. **Single-field record is the old behaviour:** a composite with a one-field + output record (e.g. `sma_cross`) compiles identically to today — the + regression guard that strategy composites are unaffected. + +Render test in `crates/aura-cli` (alongside the existing `[in:0]`/`[out]` +assertion at `main.rs:377`): assert a multi-output composite's definition renders +`[out:]` markers for each re-exported field. + +The existing `cargo test --workspace` suite is the regression boundary; all +current composite tests update their `OutPort { … }` literals to +`OutField { …, name }` and must stay green (single-field records are behaviour- +preserving). + +## Acceptance criteria + +1. `Composite::output` is `Vec`; a composite re-exports K named interior + fields as one output record. +2. A downstream consumer selects a re-exported field via `Edge::from_field`, + range-checked at compile; out-of-range consume → `BadInteriorIndex`, + out-of-range re-export → `OutputPortOutOfRange`. +3. Nested multi-output composites resolve correctly (outer re-exports inner + fields by index). +4. The MACD PoC re-exports all three lines (`macd` / `signal` / `histogram`) as a + record; the histogram remains reachable as one selected field. +5. `aura graph` renders K `[out:]` markers per multi-output composite. +6. Single-field output records (every strategy composite, `sma_cross`) compile + and run identically to today — C8 / C7 / C4 untouched; the compilat for an + unchanged blueprint is byte-identical. +7. `cargo build --workspace`, `cargo test --workspace`, and + `cargo clippy --workspace --all-targets -- -D warnings` are green.