Blueprint grammar cannot reach sinks — add declared taps (output-side twin of input_roles) #282
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Raised in review discussion: hand-authored blueprints cannot express observability — the JSON roster excludes the sinks, so a measurement-shaped graph (e.g. the UK100 conditional-stats study as counter streams) can compute its results but never surface them; they stay trapped in the graph.
Diagnosis
Sinks ARE ordinary nodes in the execution semantics:
impl Node, evaluated in the deterministic cycle loop,output: vec —crates/aura-std/src/recorder.rs:62-. The roster exclusion is purely constructor-shaped:Recorder::new(kinds, firing, tx: Sender<(Timestamp, Vec<Scalar>)>)(recorder.rs:35) captures a live host resource (the drain channel) as a non-param construction arg, and a static JSON artifact cannot name a channel endpoint. An injection limitation, not an evaluation-model one.Why naked sinks in blueprints would still be wrong
wrap_r,crates/aura-cli/src/main.rs:1566-1575). A fragment-embedded recorder subverts that; sweeps would have to strip foreign sinks.Proposal: declared taps — the output-side twin of
input_rolesThe precedent already exists on the input side:
input_rolesname abstract inputs, the harness binds archive columns (crates/aura-cli/src/binding.rs). Symmetrically:--recordflag / campaign doc block) → construct aRecorderat the tap, injectingtxthere; sweep/reduce → taps stay unbound.With this, a measurement-shaped study (counters/rates via the #281 cells) becomes fully expressible as a hand-authored blueprint: compute in the graph, surface via taps, aggregate order statistics (median etc.) in the sink/analysis layer where they belong (C18).
Related: #281 (vocabulary cells that make the compute side composable); #280 (jitter fix both resamplers need). Design-ledger candidate: the tap/binding contract touches the blueprint format and the bootstrap, so the decision belongs in
docs/design/before implementation.Design reconciliation (specify) — fork decisions, autonomous 2026-07-17
A four-way subsystem recon established the current-code facts these decisions rest on. The load-bearing finding contradicts the issue body's premise: no FlatGraph DCE/CSE pass exists — it is explicitly deferred infrastructure (C23;
crates/aura-engine/src/blueprint.rs:14-15,:165-170). So the issue's framing ("an unbound tap is dead in the FlatGraph, so DCE removes the whole counter chain behind it") has no mechanism to rest on today. The forks below are decided against that reality.Fork 1 — Does #282 depend on DCE, or ship DCE-free? → Ship the tap mechanism DCE-free, via build-time elision; defer the chain-evaporation optimisation.
Options: (A) build DCE first/alongside so an unbound tap's whole counter chain is pruned (the issue's literal framing); (B) ship taps as declaration + run-mode-gated Recorder construction, where an unbound tap is simply not lowered into a Recorder (the
wrap_rif reduce {…} else {…}conditional-construction pattern,crates/aura-cli/src/main.rs:1516,1565), needing no DCE. Chosen: B. Basis (derived): (1) DCE is explicitly-deferred C23 infrastructure; building it is a separate large cycle touching the whole compile pipeline — pulling it into #282 is scope explosion. (2) The milestone's own Acceptance sketch gates on expressibility ("the origin question becomes expressible as a hand-authored blueprint (~50 nodes) plus oneaura run --real UK100with recording — zero lines of throwaway Rust"), not on zero-cost sweeps; the "reduce-mode sweep pays zero" line is forward-looking rationale predicated on DCE. (3) Taps are designed DCE-compatible (a bound tap is a natural DCE root; an unbound tap a dead declaration), so when DCE later lands, chain-pruning is automatic — nothing is foreclosed. An independent adversarial scope-fork review (spec-skeptic) tried to refute this deferral and returned SOUND:check_ports_connected(blueprint.rs:908-937) enforces only input-slot totality, never output consumption, and the Kahn sort (harness.rs:418) emits a no-out-edge node as a valid sink — so an unbound tap's chain leaves the FlatGraph valid and runnable with a dangling output; build-time elision needs no DCE for correctness. Residue (deferred, not lost): with an unbound tap, counter/aggregation nodes feeding only that tap still eval each cycle (dangling output, discarded) — pure runtime waste in sweeps until DCE ships. This is the deferred half of the milestone's stated benefit; it is sequenced to the future DCE cycle, not abandoned.Fork 2 — Tap declaration shape & location. → A new
taps: Vec<Tap>field onCompositeand its serde mirrorCompositeData, eachTap = { name: String, from: { node, field } }— the output-side twin ofinput_roles/Role. The wire references a flat-graph node index + output-field index, matchingOutField's{node, field}shape (blueprint.rs:25-34) andEdge.from_field. Additive under the Tier-1 field policy — noBLUEPRINT_FORMAT_VERSIONbump (blueprint_serde.rs:15-21). Basis (derived): structural symmetry with the settled input-sideRole(blueprint.rs:126-135); a pure data declaration with no captured endpoint, exactly as aRolenames an abstract input without naming a source.Fork 3 — Where the tap becomes a Recorder. → Declaration is engine-side data; Recorder construction is a run-mode-aware binding step, mirroring the input-side split (roles declared in the blueprint, bound to concrete sources by
bind_sources/ the CLI binding layer). The engine gains a seam to construct + wire aRecorderat a tap's{node, field}given a channel; the run-mode-owning layer decides per mode whether to invoke it (single-run path records —run_signal_r,reduce=false; sweep/reduce leaves taps unbound —run_blueprint_member,reduce=true). Basis (derived): this is the existing declaration-vs-binding seam forinput_roles, applied symmetrically; it keeps the effectful endpoint out of the serialized fragment (the reason naked sinks in blueprints are wrong — recording policy is run-mode authority).Fork 4 — Naming vs the existing
TapChannel/persist_taps. → Keep "tap" for the blueprint-declared concept (the term the issue and milestone establish). The pre-existing aura-cliTapChannel/persist_tapsis a different-layer artefact — a fixed closed vocabulary of conventional campaign-recorder channel names — reconciled by treating a bound blueprint-tap as a recorder that surfaces through that same persist/registry path. Basis (derived): the issue/milestone own the term for this concept; the collision is across layers (author-declared blueprint wire vs CLI execution channel-name), so the implementer reconciles with a doc note rather than a rename of either. Flagged for the implementer to keep the two clearly distinguished.Fork 5 — How tap output surfaces. → The same drain path existing recorders use (mpsc → run report / registry): a bound tap's recorded series lands beside the existing recorded series. Order statistics (median, etc.) stay sink/analysis-side per the milestone's deliberate tier boundary; multi-instrument inputs stay harness/World tier. Basis (derived): mirrors the existing recording surface and the milestone's explicitly stated boundary.
Ledger. The tap/binding contract is a new load-bearing contract touching the blueprint format and the bootstrap; it will be recorded in
docs/design/(a new contract entry) as part of the implementation, per this issue's own note that the decision belongs in the ledger before implementation.Status: design settled — ready for spec production.
Spec auto-signed (2026-07-17): the spec for declared taps (the output-side twin of input_roles — a blueprint declares named measurement points on interior output wires; the harness/run mode constructs a recorder at a bound tap and leaves an unbound tap inert) was signed autonomously. The signature is the grounding-check PASS: an independent fresh-context review ratified every load-bearing assumption against currently-green tests.
The first pass returned a single BLOCK — the design's load-bearing premise (a producer node wired to nothing runs inertly, so an unbound tap needs no DCE to be cost-safe) was true of the code but had no green test pinning it. That premise is now verified: a current-behaviour pin was written and confirmed green (a real producer fanned to nothing, bootstrapped + run, byte-identical to its removal), and the re-dispatched grounding-check returned PASS on all nine assumptions. So the DCE-deferral fork decided earlier is not merely reasoned — it is test-verified: build-time elision of an unbound tap is sound without DCE.
No human signed; planning proceeds. The spec is a git-ignored working file; its durable design record is the ledger contract to be added during implementation.
Spec refinement (specify) — four resolutions from plan recon, autonomous 2026-07-17
A file-structure recon surfaced four load-bearing points the first spec draft left under- or mis-specified. All four are decided by symmetry with existing patterns (none hangs on user preference); the spec is revised and the design log updated here.
Q1 —
bind_tapscannot construct aRecorderinside the engine (a genuine correction).aura-stdis a test-only dependency ofaura-engine(crates/aura-engine/Cargo.toml); productionaura-enginesees onlyaura-core, andRecorderlives inaura-std. So the seam is reshaped toFlatGraph::bind_tap(tap_name: &str, sink: Box<dyn Node>, sig: NodeSchema) -> Result<(), TapBindError>— it takes a caller-built sink node and appends it + an edge, constructing noRecorderitself. The CLI (which depends onaura-std) builds theRecorderand callsbind_tap. This is exactly symmetric with the input side:bind_sourceslikewise does not construct aVecSource— the caller hands in aBox<dyn Source>by role name.Box<dyn Node>/NodeSchemaareaura-coretypes, so the engine stays domain-agnostic.Q2 — Nested-composite tap hoisting. On the single-run path the loaded blueprint is nested as an interior composite inside the
wrap_rwrapper graph (crates/aura-cli/src/main.rs:1502), so its declared taps are interior to the wrapper. Decision: taps hoist to the rootFlatGraph.tapsthrough the sameloweringsoffset remap that edges andOutFieldre-exports already use on inline (theCompositedestructure atblueprint.rs:1169andlower_itemsgain atapsbinding). Basis (derived): a tap wire is structurally an edge-from reference; it must remap on inline exactly as edges do, or an interior study's taps would never reach the harness root.Q3 — Single-run trace persistence.
run_signal_rtoday prints aRunReport(manifest + metrics) and persists no per-cycle traces; only the campaign path writes aTraceStore(campaign_run::persist_campaign_traces,TraceStore.write). Decision: the single-run path, when any tap is bound, reuses that existing trace-store write to surface the tap series through the same tooling the campaign path feeds (build_chart_datareadsRunTraces.taps). Basis (derived): the milestone's acceptance sketch says "oneaura run --real UK100with recording" — recording must surface somewhere the author can see it, and reusing the campaign trace-store is the minimal delivery, not a new persistence format.Q4 —
Tap.nameidentity projection. Decision: the blueprint-declarationTap.nameis a debug symbol for the topological content-id and is blanked bystrip_debug_symbols(blueprint_serde.rs:150-176), exactly asRole.name/OutField.nameare — so blueprints differing only in tap names share a content-id. The resolvedFlatTap.name(post-compile) survives and is load-bearing for by-name binding, exactly asSourceSpec.rolesurvives forbind_sources(#275). Basis (derived): mirrors the settledRole.name(stripped) vsSourceSpec.role(kept) split precisely; identity stays topological while taps stay bindable by name.The spec is revised accordingly (bind seam, hoisting, persistence, identity). Re-grounding the revised spec before planning.