plan: 0018 composite multi-output record

Four tasks: engine type migration (behaviour-preserving, caps kept) →
cap-lift with RED-first capability tests → CLI render + author sites +
golden re-capture → non-workspace fieldtest sweep + workspace gate.

refs #40
This commit is contained in:
2026-06-08 00:47:31 +02:00
parent 9ddeab8b7e
commit 2c33a8d74f
@@ -0,0 +1,623 @@
# Composite multi-output record — Implementation Plan
> **Parent spec:** `docs/specs/0018-composite-multi-output-record.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
> this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** Make a composite's output a named, ordered, multi-field record
(`output: Vec<OutField { node, field, name }>`) so multi-line indicators
(MACD/Bollinger/Stochastic/Ichimoku) can be authored as a composition and
re-exported as a unit, selected downstream by `Edge::from_field`.
**Architecture:** The substrate is already multi-field everywhere except the
composite boundary (`NodeSchema.output: Vec<FieldSpec>`, `Edge::from_field`
selects a column, leaf multi-output already works). This cycle replaces the
single `OutPort` with a `Vec<OutField>` and lifts the three `field == 0` caps at
the boundary (`inline_composite` nested arm, `rewrite_edge` composite arm). Names
live at the blueprint boundary only and are dropped in the compilat (C23); C8/C7/C4
are untouched — one port, one row, K columns.
**Tech Stack:** `crates/aura-engine/src/blueprint.rs` (type + compile logic),
`crates/aura-engine/src/lib.rs` (re-export), `crates/aura-cli/src/graph.rs`
(render), `crates/aura-cli/src/main.rs` (author sites + render tests),
`fieldtests/milestone-construction-layer/*.rs` (non-workspace stale-ref sweep).
**Files this plan creates or modifies:**
- Modify: `crates/aura-engine/src/blueprint.rs``OutPort``OutField`, `Composite`
struct/`new`/`output()`, `ItemLowering::Composite.output`, `inline_composite`,
`rewrite_edge`, all `#[cfg(test)]` `OutPort` literals; new capability tests.
- Modify: `crates/aura-engine/src/lib.rs:38` — re-export `OutPort``OutField`.
- Modify: `crates/aura-cli/src/graph.rs:105129``render_definition` K `[out:<name>]` markers.
- Modify: `crates/aura-cli/src/main.rs` — import `:13`, `sma_cross` `:130`, `macd` `:206`,
nested-render-test literals `:391`/`:398`, needle-list test `:377`, `blueprint_view_golden`.
- Modify: `fieldtests/milestone-construction-layer/mc_1..mc_4*.rs``OutPort``OutField` (8 sites, non-workspace).
**Naming decisions (orchestrator, fixed for this plan):**
- `sma_cross` single-field output name → `"cross"` (renders as `[out:cross]` in the re-captured golden).
- Engine test-fixture single-field outputs → `"out"`. New multi-output test fields → `"a"`, `"b"` (and `"c"` where a third is needed).
- Fieldtest single-field outputs → `"out"`.
---
## Task 1: Engine — introduce `OutField`, widen the type (behaviour-preserving)
This task replaces `OutPort` with `OutField` and widens `Composite.output` /
`ItemLowering::Composite.output` to vectors, threading **every** call site in
`blueprint.rs` so the crate compiles and **all existing tests stay green**. The
three `field == 0` caps are **kept** here (single-field composites only re-export
field 0, so behaviour is preserved); Task 2 lifts them test-first.
**Files:**
- Modify: `crates/aura-engine/src/blueprint.rs`
- Modify: `crates/aura-engine/src/lib.rs:38`
- [ ] **Step 1: Rename `OutPort` → `OutField` and add `name` (blueprint.rs:1823)**
Replace:
```rust
/// Which interior `(node, output-field)` is a composite's single output port (C8).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct OutPort {
pub node: usize,
pub field: usize,
}
```
with:
```rust
/// One re-exported field of a composite's output record: an interior
/// `(node, output-field)` surfaced at the boundary under `name`. `name` is a
/// non-load-bearing render/debug symbol (C23) — like `FieldSpec.name` and
/// `Composite.name`, it does not reach the compilat.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct OutField {
pub node: usize,
pub field: usize,
pub name: String,
}
```
(`Copy` is dropped — `String` is not `Copy`. Callers that relied on `output()`
returning by Copy are updated in Step 3 and Step 6.)
- [ ] **Step 2: Widen `Composite.output` to a record (blueprint.rs:4349)**
In `struct Composite`, change the field:
```rust
input_roles: Vec<Vec<Target>>,
output: Vec<OutField>,
```
(was `output: OutPort,` at `:48`.)
- [ ] **Step 3: Update `Composite::new` and `output()` (blueprint.rs:5685)**
In `Composite::new`, change the param type:
```rust
input_roles: Vec<Vec<Target>>,
output: Vec<OutField>,
) -> Self {
Self { name: name.into(), nodes, edges, input_roles, output }
}
```
(was `output: OutPort,` at `:61`.) Change the accessor (`:8385`):
```rust
/// The exposed output record: each entry re-exports one interior
/// `(node, output-field)` under a boundary name (C8 — one port, K columns).
pub fn output(&self) -> &[OutField] {
&self.output
}
```
(was `pub fn output(&self) -> OutPort { self.output }`.)
- [ ] **Step 4: Widen the lowering variant (blueprint.rs:243249)**
In `enum ItemLowering`, change the `Composite` variant:
```rust
/// A composite lowered to its interior: its output record is these flat
/// `(node, field)` producers (one per re-exported field, declared order), and
/// input role `r` fans into `roles[r]` (flat targets). Names dropped (C23).
Composite { output: Vec<(usize, usize)>, roles: Vec<Vec<Target>> },
```
(was `output: (usize, usize)`.)
- [ ] **Step 5: Make `inline_composite` build the output record, caps kept (blueprint.rs:292355)**
Remove the single pre-check at `:306308`:
```rust
if output.node >= item_count {
return Err(CompileError::OutputPortOutOfRange);
}
```
Replace the single-output resolution block (`:318333`, the `let out = match … ;`)
with a loop that builds the Vec. **The `!= 0` caps stay** (behaviour-preserving):
```rust
// resolve each re-exported field to a flat (node, field), in declared order
let mut out: Vec<(usize, usize)> = Vec::with_capacity(output.len());
for of in &output {
if of.node >= item_count {
return Err(CompileError::OutputPortOutOfRange);
}
let resolved = match &interior[of.node] {
ItemLowering::Leaf { index } => {
if of.field >= flat_nodes[*index].schema().output.len() {
return Err(CompileError::OutputPortOutOfRange);
}
(*index, of.field)
}
ItemLowering::Composite { output: nested, .. } => {
// cap kept in Task 1; Task 2 lifts this to nested.get(of.field)
if of.field != 0 {
return Err(CompileError::OutputPortOutOfRange);
}
nested[0]
}
};
out.push(resolved);
}
```
The terminal `Ok(ItemLowering::Composite { output: out, roles })` at `:354` is
unchanged (`out` is now a `Vec`).
- [ ] **Step 6: Thread `rewrite_edge`, cap kept (blueprint.rs:375381)**
Replace the composite arm:
```rust
ItemLowering::Composite { output, .. } => {
// cap kept in Task 1; Task 2 lifts this to output.get(e.from_field)
if e.from_field != 0 {
return Err(CompileError::BadInteriorIndex);
}
output[0]
}
```
(was `*output` after the `!= 0` guard; `output` is now a `Vec`, so `output[0]`.)
- [ ] **Step 7: Update `lib.rs` re-export (lib.rs:38)**
```rust
pub use blueprint::{Blueprint, BlueprintNode, CompileError, Composite, OutField};
```
(was `… Composite, OutPort};`.)
- [ ] **Step 8: Migrate all existing `OutPort` test literals (blueprint.rs `#[cfg(test)]`)**
Every existing test composite is single-field. Rewrite each literal at
`:532, :578, :611, :626, :641, :736, :916, :924, :967, :975` from
`OutPort { node: N, field: F }` to a one-element record:
```rust
vec![OutField { node: N, field: F, name: "out".into() }]
```
preserving each site's existing `node`/`field` values. (The site at `:641` is
`OutPort { node: 0, field: 5 }` — the OHLCV-field re-export — becomes
`vec![OutField { node: 0, field: 5, name: "out".into() }]`.) The comment mention
at `:549` is prose; update its wording from "OutPort" to "OutField" if it names
the type, otherwise leave.
- [ ] **Step 9: Build and test the engine crate**
Run: `cargo test -p aura-engine`
Expected: PASS — the crate compiles and every existing test is green
(single-field records are behaviour-preserving; the caps still hold).
- [ ] **Step 10: Clippy the engine crate**
Run: `cargo clippy -p aura-engine --all-targets -- -D warnings`
Expected: PASS — no warnings (watch for a needless `Vec::with_capacity` or
`clone` lint; none expected).
---
## Task 2: Engine — lift the caps, multi-output capability (RED-first)
The type is in place (Task 1); now lift the two `field == 0` caps, each gated by
a failing test written first. Tests reuse the existing `#[cfg(test)]` helpers
`pass1()` (1-in/1-out f64 leaf), `sink_f64()` (f64 sink), `Blueprint::new`,
`SourceSpec`, and the `bp.compile()` / `bp.compile().err()` assertion idiom — no
new helper is introduced. Spec test 3 (out-of-range re-export) is already covered
by the migrated `output_port_out_of_range_rejected` (Task 1 Step 8), so it is not
re-added here.
**Files:**
- Modify: `crates/aura-engine/src/blueprint.rs` (the two caps + three new `#[cfg(test)]` tests)
- [ ] **Step 1: Write the multi-output happy-path test (RED)**
Add to the `#[cfg(test)]` module, next to `single_composite_inlines_with_offset_fan_and_output`:
```rust
#[test]
fn composite_reexports_two_fields_to_distinct_consumers() {
// composite: two independent Pass1 leaves; role 0 -> leaf 0, role 1 -> leaf 1;
// output record re-exports leaf 0 as "a", leaf 1 as "b".
let c = Composite::new(
"two_out",
vec![pass1(), pass1()],
vec![],
vec![
vec![Target { node: 0, slot: 0 }],
vec![Target { node: 1, slot: 0 }],
],
vec![
OutField { node: 0, field: 0, name: "a".into() },
OutField { node: 1, field: 0, name: "b".into() },
],
);
// composite is item 0; two sinks (items 1, 2) read its two output fields by
// from_field; one source fans into both roles.
let bp = Blueprint::new(
vec![BlueprintNode::Composite(c), sink_f64(), sink_f64()],
vec![SourceSpec {
kind: ScalarKind::F64,
targets: vec![Target { node: 0, slot: 0 }, Target { node: 0, slot: 1 }],
}],
vec![
Edge { from: 0, to: 1, slot: 0, from_field: 0 }, // field "a" -> sink 1
Edge { from: 0, to: 2, slot: 0, from_field: 1 }, // field "b" -> sink 2
],
);
let (nodes, sources, edges) = bp.compile().expect("valid multi-output composite");
// flat layout: Pass1(0), Pass1(1), SinkF64(2), SinkF64(3)
assert_eq!(nodes.len(), 4);
// from_field 0 resolves to leaf 0, from_field 1 to leaf 1 — distinct producers
assert_eq!(
edges,
vec![
Edge { from: 0, to: 2, slot: 0, from_field: 0 },
Edge { from: 1, to: 3, slot: 0, from_field: 0 },
]
);
// the source fanned into both interior leaves
assert_eq!(
sources[0].targets,
vec![Target { node: 0, slot: 0 }, Target { node: 1, slot: 0 }]
);
}
```
Run: `cargo test -p aura-engine composite_reexports_two_fields_to_distinct_consumers`
Expected: FAIL — the consumer reading `from_field: 1` hits the `rewrite_edge` cap;
`compile` returns `Err(BadInteriorIndex)`, so `.expect(...)` panics.
- [ ] **Step 2: Lift the `rewrite_edge` cap (blueprint.rs composite arm)**
Replace the composite arm (the Step-6 Task-1 body) with the range-checked index:
```rust
ItemLowering::Composite { output, .. } => {
*output.get(e.from_field).ok_or(CompileError::BadInteriorIndex)?
}
```
Run: `cargo test -p aura-engine composite_reexports_two_fields_to_distinct_consumers`
Expected: PASS.
- [ ] **Step 3: Write the nested multi-output test (RED)**
An **outer** composite re-exporting two fields of an **inner** multi-output
composite (exercises the nested arm). Add next to `nested_composite_inlines`:
```rust
#[test]
fn outer_reexports_two_fields_of_inner_composite() {
// inner re-exports two leaves as "a","b"; outer re-exposes both inner roles
// and re-exports inner field 0 and field 1 (the latter exercises the nested arm).
let inner = Composite::new(
"inner_two",
vec![pass1(), pass1()],
vec![],
vec![
vec![Target { node: 0, slot: 0 }],
vec![Target { node: 1, slot: 0 }],
],
vec![
OutField { node: 0, field: 0, name: "a".into() },
OutField { node: 1, field: 0, name: "b".into() },
],
);
let outer = Composite::new(
"outer_two",
vec![BlueprintNode::Composite(inner)],
vec![],
vec![
vec![Target { node: 0, slot: 0 }], // outer role 0 -> inner role 0
vec![Target { node: 0, slot: 1 }], // outer role 1 -> inner role 1
],
vec![
OutField { node: 0, field: 0, name: "x".into() }, // inner field 0
OutField { node: 0, field: 1, name: "y".into() }, // inner field 1 (nested arm)
],
);
let bp = Blueprint::new(
vec![BlueprintNode::Composite(outer), sink_f64(), sink_f64()],
vec![SourceSpec {
kind: ScalarKind::F64,
targets: vec![Target { node: 0, slot: 0 }, Target { node: 0, slot: 1 }],
}],
vec![
Edge { from: 0, to: 1, slot: 0, from_field: 0 }, // outer field x -> sink 1
Edge { from: 0, to: 2, slot: 0, from_field: 1 }, // outer field y -> sink 2
],
);
let (nodes, _sources, edges) = bp.compile().expect("valid nested multi-output");
assert_eq!(nodes.len(), 4); // Pass1, Pass1, SinkF64, SinkF64
assert_eq!(
edges,
vec![
Edge { from: 0, to: 2, slot: 0, from_field: 0 },
Edge { from: 1, to: 3, slot: 0, from_field: 0 },
]
);
}
```
Run: `cargo test -p aura-engine outer_reexports_two_fields_of_inner_composite`
Expected: FAIL — `inline_composite`'s nested arm caps `of.field != 0` on the
outer's `field: 1` re-export; `compile` returns `Err(OutputPortOutOfRange)`.
- [ ] **Step 4: Lift the nested arm cap (blueprint.rs inline_composite)**
Replace the nested arm in the Step-5 Task-1 loop:
```rust
ItemLowering::Composite { output: nested, .. } => {
*nested.get(of.field).ok_or(CompileError::OutputPortOutOfRange)?
}
```
(was the `if of.field != 0 { … } nested[0]` cap.)
Run: `cargo test -p aura-engine outer_reexports_two_fields_of_inner_composite`
Expected: PASS.
- [ ] **Step 5: Write the out-of-range consume guard test**
```rust
#[test]
fn consume_of_missing_output_field_is_rejected() {
// a single-field composite; a consumer reads from_field 1 (past the 1-field
// record) -> the rewrite_edge range-check rejects it.
let c = Composite::new(
"c",
vec![pass1()],
vec![],
vec![vec![Target { node: 0, slot: 0 }]],
vec![OutField { node: 0, field: 0, name: "a".into() }],
);
let bp = Blueprint::new(
vec![BlueprintNode::Composite(c), sink_f64()],
vec![SourceSpec { kind: ScalarKind::F64, targets: vec![Target { node: 0, slot: 0 }] }],
vec![Edge { from: 0, to: 1, slot: 0, from_field: 1 }], // only field 0 exists
);
assert_eq!(bp.compile().err(), Some(CompileError::BadInteriorIndex));
}
```
Run: `cargo test -p aura-engine consume_of_missing_output_field_is_rejected`
Expected: PASS — `output.get(1)` on a 1-field record is `None``BadInteriorIndex`.
(This is a guard, not a RED gate: it also errored under the Task-1 cap, but now via
the range-check that replaced it.)
- [ ] **Step 6: Full engine regression + clippy**
Run: `cargo test -p aura-engine`
Expected: PASS — all old tests (single-field, behaviour-preserving) plus the four
new capability tests are green.
Run: `cargo clippy -p aura-engine --all-targets -- -D warnings`
Expected: PASS.
---
## Task 3: CLI — multi-output render + author sites + golden re-capture
`aura-cli` depends on `aura-engine`; after Task 1 it no longer compiles (`OutPort`
removed from the import and four literals). This task threads every CLI site,
re-exports the MACD's three lines, renders K named output markers, and re-captures
the one drifting golden — one atomic compile-and-test unit.
**Files:**
- Modify: `crates/aura-cli/src/main.rs` (import `:13`, `sma_cross` `:130`, `macd` `:206`, nested-render-test literals `:391`/`:398`, needle test `:377`, `blueprint_view_golden`)
- Modify: `crates/aura-cli/src/graph.rs:124126` (`render_definition`)
- [ ] **Step 1: Update the import (main.rs:13)**
Change `OutPort` to `OutField` in the `use` list:
```rust
f64_field, summarize, Blueprint, BlueprintNode, Composite, Edge, Harness, OutField,
```
- [ ] **Step 2: Render K named output markers (graph.rs:124126)**
Replace the single `[out]` marker block:
```rust
for of in c.output() {
let out_id = labels.len();
labels.push(format!("out:{}", of.name));
edges.push((of.node, out_id));
}
```
(was the three lines pushing one `"out"` label wired from `c.output().node`.)
Update the function's doc-comment at `:101104` to say "an `[out:<name>]` marker
per re-exported output field" (was "an `[out]` marker (wired from the output
port)").
- [ ] **Step 3: Migrate `sma_cross` to a single-field record (main.rs:130)**
```rust
vec![OutField { node: 2, field: 0, name: "cross".into() }],
```
(was `OutPort { node: 2, field: 0 }`.) Behaviour-preserving: one re-exported
field; renders as `[out:cross]`.
- [ ] **Step 4: Re-export all three MACD lines (main.rs:206)**
Replace the `macd` composite's output (was `OutPort { node: 4, field: 0 }, // the histogram`):
```rust
vec![
OutField { node: 2, field: 0, name: "macd".into() }, // the MACD line
OutField { node: 3, field: 0, name: "signal".into() }, // the signal line
OutField { node: 4, field: 0, name: "histogram".into() }, // the histogram
],
```
Update the `macd` doc-comment (`:178184`): the composite now exposes the **three
MACD lines** as a record; the strategy trades the histogram by reading
`from_field: 2`.
**Then update the consumer edge** in `macd_strategy_blueprint` (`:233`). The
histogram is now field index **2** of the 3-field record (was the sole output at
field 0), so the edge feeding `Exposure` must select it:
```rust
Edge { from: 0, to: 1, slot: 0, from_field: 2 }, // histogram → Exposure
```
(was `from_field: 0`.) This keeps the strategy trading the **histogram** — leaving
it at `0` would silently feed the MACD line instead, changing the run and breaking
`run_macd_compiles_from_nested_composite_and_is_deterministic`. The other three
edges (`:234236`) are unchanged.
- [ ] **Step 5: Migrate the nested-render-test literals (main.rs:391, :398)**
In `nested_composite_renders_without_panic`, both single-field composites:
```rust
vec![OutField { node: 0, field: 0, name: "out".into() }], // :391 inner
```
```rust
vec![OutField { node: 1, field: 0, name: "out".into() }], // :398 outer
```
- [ ] **Step 6: Build the CLI crate**
Run: `cargo build -p aura-cli`
Expected: PASS — every `OutPort` site threaded; crate compiles. (Golden/needle
tests may still be red — handled next.)
- [ ] **Step 7: Add the `[out:<name>]` needle assertion (main.rs:377)**
In `blueprint_view_defines_each_composite_once`, extend the needle list so the
multi-output marker is asserted. `sma_cross` is the rendered composite, so its
marker is `[out:cross]`:
```rust
for needle in ["[SMA]", "[Sub]", "[in:0]", "[out:cross]"] {
```
(was `… "[out]"`.)
Run: `cargo test -p aura-cli blueprint_view_defines_each_composite_once`
Expected: PASS — the render now emits `[out:cross]` for `sma_cross`.
- [ ] **Step 8: Re-capture the `blueprint_view_golden` (main.rs:464502)**
The `[out]``[out:cross]` change drifts this golden. Follow the test's own
re-capture protocol (comment at `:466468`): run the golden test, observe the
actual rendered string in the failure diff, and paste it verbatim into the golden
literal (`:497` region). Do **not** hand-edit the expected string field-by-field —
copy the actual output wholesale.
Run: `cargo test -p aura-cli blueprint_view_golden`
Expected: FAIL first (golden drift: `[out]``[out:cross]`), then PASS after the
literal is re-captured.
- [ ] **Step 9: Confirm the compilat golden is byte-identical (main.rs:505530)**
`compiled_view_golden` pins the flat compilat render; per C23 names are dropped at
inline, so it must **not** change (acceptance criterion 6 — the regression guard).
Run: `cargo test -p aura-cli compiled_view_golden`
Expected: PASS with **no** edit to the golden literal. If it drifts, a name leaked
into the compilat — a bug to fix, not a golden to re-capture.
- [ ] **Step 10: Full CLI test + clippy**
Run: `cargo test -p aura-cli`
Expected: PASS — including `run_macd_compiles_from_nested_composite_and_is_deterministic`
(the MACD run still produces the histogram, now via `from_field: 2`).
Run: `cargo clippy -p aura-cli --all-targets -- -D warnings`
Expected: PASS.
---
## Task 4: Non-workspace fieldtest sweep + full workspace gate
The construction-layer fieldtests are **not** workspace members, so
`cargo …--workspace` does not compile them — they hold stale `OutPort` references
to a now-deleted type. Sweep them for tree consistency (mechanical), then run the
full workspace gate.
**Files:**
- Modify: `fieldtests/milestone-construction-layer/mc_1_composite_build_run.rs:6,20,47`
- Modify: `fieldtests/milestone-construction-layer/mc_2_miswire_render.rs:15,47`
- Modify: `fieldtests/milestone-construction-layer/mc_3_nested_composite.rs:18,35,52`
- Modify: `fieldtests/milestone-construction-layer/mc_4_introspect_graph.rs:13,34`
- [ ] **Step 1: Sweep `OutPort` → `OutField` across the fieldtests**
In each file: update the `use … OutPort` import to `OutField`, and rewrite each
`OutPort { node: N, field: F }` literal to `vec![OutField { node: N, field: F, name: "out".into() }]`
**at the `Composite::new` output-arg position** (these are all single-field
composites). Update comment mentions of `OutPort` (e.g. mc_1 `:6`) to `OutField`.
- [ ] **Step 2: Verify no `OutPort` remains anywhere in the tree**
Run: `rg -n "OutPort" --type rust`
Expected: **no matches** — the type is fully retired (every reference is now
`OutField`).
- [ ] **Step 3: Full workspace build**
Run: `cargo build --workspace`
Expected: PASS.
- [ ] **Step 4: Full workspace test**
Run: `cargo test --workspace`
Expected: PASS — every crate green; single-field composites behaviour-preserving,
multi-output capability covered by Task 2, render by Task 3.
- [ ] **Step 5: Full workspace clippy**
Run: `cargo clippy --workspace --all-targets -- -D warnings`
Expected: PASS — zero warnings.
---
## Acceptance gate (whole plan)
- [ ] `cargo build --workspace` green.
- [ ] `cargo test --workspace` green (incl. the four new engine capability tests and the re-captured `blueprint_view_golden`).
- [ ] `cargo clippy --workspace --all-targets -- -D warnings` green.
- [ ] `rg "OutPort" --type rust` returns nothing (type fully retired).
- [ ] `compiled_view_golden` unchanged (compilat byte-identical — C23 / acceptance criterion 6).
- [ ] The MACD PoC re-exports `macd`/`signal`/`histogram`; the run still trades the histogram (via `from_field: 2`).
The implementation commit closes #40.