Files
Aura/fieldtests/cycle-0088-construction-op-script/FINDINGS.md
T
Brummel 8bb21b1c16 docs(0088): document op-script grammar; refresh stale cycle-0088 example corpus
#163 — the construction op-script (`aura graph build`) had no durable public
schema doc, and the only worked-example corpus had drifted out of sync with the
current binary, so a consumer copying the visible example was misled.

Grammar reference (durable): a compact op-script grammar — the six verbs
(source/input/add/feed/connect/expose), their keys, the typed-Scalar bind form
({"I64":2}) and the capitalized ScalarKind, the by-identifier / dotted-port
model, and the add-`name` vs expose-`as` distinction — added to the C24 ledger
entry, with a correct worked-example pointer. Surfacing the same grammar in
`aura graph build --help` is left to ride with the CLI-discoverability work
(#159), keeping this change decoupled from the deferred World/project-as-crate
layer.

Corpus refresh: the cycle-0088 fixture corpus replayed against the current
binary failed on two accumulated drifts. (1) `add`'s naming key was renamed
`as` -> `name` (3c4b667; `expose` keeps `as` as a real alias) — the inputs now
use `name` for `add` ops only. (2) The dataflow-cycle bug the 0088 fieldtest
found was fixed (1652042, closes #161), so `c0088_3m_two_cycle` is now a
rejection fixture: the gap-era cyclic-blueprint golden is removed and
`c0088_3m.err` carries the eager cycle-rejection message. FINDINGS.md gets a
dated refresh stamp; its findings are preserved as written at fieldtest time.

Verified: the four golden-bearing success fixtures (_1, _2_completed,
_2_partial via --unwired) reproduce byte-identically; all 15 error fixtures
produce the intended by-identifier `op N (kind): cause` (or finalize) message.
Doc + fixtures only, no engine/CLI code touched.

closes #163
2026-06-30 15:45:23 +02:00

214 lines
14 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Fieldtest — cycle 0088 (construction op-script service, #157) — 2026-06-29
**Status:** Triaged 2026-06-29 — filed to the forward queue (see Triage outcome)
**Author:** fieldtester (dispatched by fieldtest skill); triaged by orchestrator
> **Corpus refreshed 2026-06-30 (#163)** to replay against the current binary.
> Two drifts had accumulated since this run recorded the fixtures: `add`'s
> naming key was renamed `as` -> `name` (commit `3c4b667`; `expose` keeps `as`),
> and the dataflow-cycle bug below was fixed (`1652042`, closes #161). The `.json`
> inputs now use `name` for `add`; `c0088_3m_two_cycle` is a rejection fixture
> (`c0088_3m.err` carries the cycle-rejection message; the gap-era cyclic-blueprint
> golden was removed). The findings below are preserved as written at fieldtest time.
## Scope
Cycle 0088 shipped the **construction op-script service** (#157, C24): two CLI
subcommands over a declarative JSON op-list. `aura graph build` reads an op-list
from stdin, replays it op-by-op through the `aura-std` node vocabulary, and on
success prints a `format_version:1` #155 blueprint to stdout; on the first failing
op it prints `op N (kind): cause` (or `finalize: cause`) to stderr and exits
non-zero. `aura graph introspect --vocabulary | --node <T> | --unwired` answers,
build-free: the node-type roster, one type's input ports / output fields / param
paths, and (reading a partial op-list) the still-open interior slots. This field
test exercised the surface purely as a downstream consumer authoring topology as
data — public interface only (the C24/C17/C19/C23 ledger + the CLI's own output);
no implementation source read. Binary under test: freshly built `target/debug/aura`
(`cargo build --workspace`, nothing to recompile at HEAD).
## Examples
### fieldtests/cycle-0088-construction-op-script/c0088_1_sma_crossover.json — discover-then-author an SMA-crossover bias
- Used `introspect --vocabulary` (22 types) then `introspect --node SMA|Sub|Bias`
to learn ports/kinds, then hand-wrote a 10-op script: one `price` source → two
SMAs (lengths 2/4) → `Sub``Bias`, expose `bias.bias`.
- Fits scope: exercises `source`, `add`+`bind`, `feed` (fan-out into two slots),
`connect`, `expose` — the full op vocabulary in one realistic signal graph.
- Outcome: **built, ran, matched expected.** Emitted a `format_version:1` blueprint
with the expected 4 nodes / 3 edges / 1 input_role / 1 output
(`c0088_1_blueprint.out.json`). First try, no diagnostics.
### fieldtests/cycle-0088-construction-op-script/c0088_2_partial.json (+ _completed.json) — author with the --unwired helper
- Wrote a deliberately partial doc (left `sub.rhs` and `bias` unconnected), ran
`introspect --unwired`, finished wiring per its report, built.
- Fits scope: the cycle's headline build-free authoring aid.
- Outcome: **worked.** `--unwired` reported `sub.rhs:F64` / `bias.signal:F64`
by-identifier with kinds; the completed doc built to a #155 blueprint;
`--unwired` on the completed doc returned empty (exit 0) — a reliable
"am I done?" check.
### fieldtests/cycle-0088-construction-op-script/c0088_3*.json — error ergonomics (15 fixtures)
- Provoked the realistic hand-author mistakes: bad node type, wrong-kind connect,
wrong output field, unknown node id, double-wired slot, duplicate id, bad bind
kind, bad expose field, unknown role, bad feed port, left-open slot, dataflow
cycle (self-loop + explicit 2-cycle), unfed role, no-expose, naive/untyped bind,
lowercase kind.
- Fits scope: judges whether `op N (kind): cause` tells you *where* and *what*.
- Outcome: **eager per-op messages are excellent** (by-identifier, right op index);
three message-quality friction points and one acceptance gap surfaced (below).
### Round-trip sense-check (Task 4)
- Piped `c0088_1_blueprint.out.json` into `aura graph`; compared to bare `aura graph`.
- Outcome: byte-**identical** — `aura graph` ignores stdin and renders a hard-wired
sample. No CLI subcommand consumes an emitted #155 blueprint (see G2).
## Findings
### [working] Discover-then-author round-trips cleanly through the CLI
- Examples: c0088_1, all `introspect` runs.
- `introspect --vocabulary` lists 22 node types; `introspect --node SMA` prints
`in series:F64 / out value:F64 / param length:I64` — the exact dotted-port names
(`fast.series`, `fast.value`) and capitalized kinds the op-script consumes. An
author can go vocabulary → ports → op-script → `format_version:1` blueprint with
no source reading. This is the core promise of C24's data-authoring surface and
it holds.
- Recommended action: carry-on.
### [working] `--unwired` is a precise, by-identifier authoring aid
- Examples: c0088_2_partial / _completed.
- Reports open interior slots as `node.port:Kind` (`sub.rhs:F64`,
`bias.signal:F64`) and returns empty on a fully-wired doc. Exactly the
finish-the-wiring loop a hand-author wants; the kinds double as a connect hint.
- Recommended action: carry-on.
### [working] Eager per-op diagnostics name the op and the identifier
- Examples: c0088_3ah, 3j, 3k.
- Verbatim wins: `op 1 (add): unknown node type "Smaa"`; `op 4 (connect): kind
mismatch g.value -> fast.series (producer Bool, consumer F64)`; `op 4 (connect):
node "fast" has no output field "output"`; `op 4 (connect): unknown node
identifier "fst"`; `op 6 (connect): slot sub.lhs already wired`; `op 2 (add):
duplicate identifier "fast"`; `op 2 (feed): unknown role "pryce"`; `op 2 (feed):
node "fast" has no input port "serie"`. Every one names the op index and speaks
by-identifier — you can fix without counting. Lowercase `kind:"f64"` even
self-documents: `unknown variant 'f64', expected one of 'I64','F64','Bool',
'Timestamp'`.
- Recommended action: carry-on.
### [bug] `graph build` accepts a dataflow cycle and reports success
- Examples: c0088_3i_unfed_role (self-loop), c0088_3m_two_cycle.
- `aura graph build < c0088_3m_two_cycle.json` → **exit 0**, emits a blueprint
whose edges are `{from:0,to:1,slot:0},{from:1,to:0,slot:0}` — a combinational
2-cycle. A self-loop (`fast.value → fast.series`, edge `{from:0,to:0}`) is
likewise accepted. C9 / domain-invariant 5: "The graph is a DAG; the only
feedback path is an explicit delay/state node." The construction service's gate
set (kind, interior-slot totality, param-namespace injectivity, root-role
boundness) omits acyclicity, so it green-lights a non-runnable blueprint and
calls it a success. Caveat (cadence, unverifiable as a CLI consumer): acyclicity
may be intended as a bootstrap-time (compile-to-`FlatGraph`) gate — but the CLI
exposes no compile/run path for an emitted blueprint (see G2), so a CLI consumer
gets no signal at all that the graph is not a DAG.
- Repro: `aura graph build < fieldtests/cycle-0088-construction-op-script/c0088_3m_two_cycle.json; echo $?` → `0`.
- Recommended action: debug.
### [friction] The holistic `finalize` error speaks raw indices, breaking the by-identifier surface
- Examples: c0088_2_partial.
- `aura graph build < c0088_2_partial.json` → `aura: finalize: UnconnectedPort
{ node: 2, slot: 1 }`. The whole op-script is authored by-identifier, and the
sibling `introspect --unwired` reports the *same* open slot as `sub.rhs:F64` —
so the identifier mapping demonstrably exists. The build finalize path drops to
raw `node`/`slot` indices (and Debug-struct form), forcing the author to count
nodes (0=fast,1=slow,2=sub) and slots to decode it.
- Why friction: task completed (the error is non-zero and correct) but the surface
forced an index-counting dance the by-identifier design implies should be gone.
- Recommended action: plan — route finalize faults through the same by-identifier
formatter `--unwired` already uses.
### [friction] Bind-kind-mismatch leaks the Debug struct, inconsistent with the connect message
- Examples: c0088_3g_bad_bind_kind.
- `op 1 (add): bad param on fast: KindMismatch { param: "length", expected: I64,
got: F64 }` — the `KindMismatch { … }` is raw `#[derive(Debug)]`, whereas the
parallel connect mismatch is hand-phrased prose (`producer Bool, consumer F64`).
Two kind-mismatch errors, two registers.
- Recommended action: plan — phrase the bind mismatch like the connect one.
### [friction] The typed-Scalar bind form is not discoverable from introspection
- Examples: c0088_3n_naive_bind.
- `introspect --node SMA` shows `param length:I64` but never that a bind *value*
must be wrapped `{"I64": 2}`. The natural first guess `"bind":{"length":2}`
fails with a generic serde error: `invalid op-list: invalid type: integer 2,
expected string or map at line 4 column 2` — correct location, no domain hint
about the `{"I64": 2}` shape. (Capitalized `kind` *is* discoverable — both from
`introspect --node` output and the self-documenting lowercase-kind error.)
- Recommended action: plan/doc — show the bind-value form in `introspect --node`
(e.g. `param length:I64 (bind {"I64": <v>})`) or hint it in the serde error.
### [spec_gap] The construction surface is undiscoverable from the CLI
- Examples: bare `aura`, `aura --help`, `aura graph build --help`.
- Neither `aura` (no args, exit 2), `aura --help`, nor any public doc (glossary,
project-layout) mentions `aura graph build` or `aura graph introspect`. The
top-level usage string lists only the bare `aura graph` (HTML-render) arm. The
subcommands appear *only* in the C24 ledger Status prose. `aura graph build
--help` and `aura graph introspect --help` both print the giant top-level usage
at exit 0 (the `--help` token is swallowed); there is no per-subcommand help and
no statement anywhere of the op-list document shape. A genuine new user has no
in-CLI route to learn the surface exists or how a document is shaped.
- Why spec_gap: the surface shipped without a discovery path; the ledger is silent
on whether the CLI usage should advertise it. I read the surface only because the
carrier named the subcommands.
- Recommended action: tighten the design ledger / plan — add `graph build` /
`graph introspect` to the usage string and a `build --help` documenting the
op-list shape (the op kinds, the typed-Scalar bind, capitalized kind).
### [spec_gap] No CLI consumer for the emitted #155 blueprint — the round-trip dead-ends
- Examples: Task 4 (pipe c0088_1_blueprint into `aura graph`).
- `graph build` emits a blueprint to stdout, but no subcommand reads one back:
`aura graph` ignores stdin and renders a hard-wired sample (verified
byte-identical with and without a piped blueprint), and no `aura run` /
`graph render` accepts a blueprint file. C24 promises topology is a value
"constructed, generated, mutated, serialized for reproduction, and (optionally)
rendered"; the load path is realised only at the library level
(`blueprint_from_json`, cycle 0087). A pure-CLI consumer can author and build
topology-as-data but cannot run, render, or reload-validate it through the CLI —
which is also why the cycle bug (above) is undiscoverable. Equally plausible
reading: `build` is intentionally an emit-only producer feeding a library/World
consumer — hence spec_gap, not bug.
- Recommended action: ratify the emit-only scope, or planner — add a CLI edge that
consumes a #155 blueprint (render/run/validate). C24 Status already tracks #159
(retire hard-wired harnesses); note the missing build→consume edge there.
### [spec_gap] The validity floor of `graph build` is unstated
- Examples: c0088_3i_unfed_role, c0088_3l_no_expose.
- Beyond the cycle (bug above), build accepts at exit 0: an unfed `source` role
(emitted as `"targets":[]` — a dead role kept in the blueprint) and a graph with
no `expose`/output at all (no `output` key emitted). The ledger names the gates
build *does* enforce (kind, interior-slot totality, injectivity, root-role
boundness) but not the minimal well-formedness it *guarantees* — so a consumer
can't tell whether a dead role / output-less graph is intended-valid or a
silently-accepted slip.
- Recommended action: ratify (state the validity floor — what `build` does/does not
promise) or tighten.
## Recommendation summary
| Finding | Class | Action |
|---|---|---|
| Discover-then-author round-trips cleanly | working | carry-on |
| `--unwired` precise by-identifier aid | working | carry-on |
| Eager per-op diagnostics name op + identifier | working | carry-on |
| `graph build` accepts a dataflow cycle | bug | debug |
| `finalize` error speaks raw indices | friction | plan |
| Bind-kind-mismatch leaks Debug struct | friction | plan |
| Typed-Scalar bind form not discoverable | friction | plan / doc |
| Construction surface undiscoverable from CLI | spec_gap | tighten ledger / plan |
| No CLI consumer for emitted blueprint | spec_gap | ratify / plan |
| Validity floor of `graph build` unstated | spec_gap | ratify / tighten |
## Triage outcome (orchestrator, 2026-06-29)
The cycle bug was reproduced directly before filing. Findings routed to the forward queue:
| Finding | Disposition |
|---|---|
| `graph build` accepts a dataflow cycle (+ validity-floor: unfed role, no-`expose`) | filed **#161** (`bug`) — reject cyclic graphs, DAG invariant 5; validity floor decided alongside. RED-first via debug. |
| `finalize` raw indices + bind-mismatch Debug leak + typed-Scalar bind undiscoverable | filed **#162** (polish) — three message-quality defects on the by-identifier surface. |
| Construction surface undiscoverable from the CLI | commented on **#159** (CLI usage/help rework) — advertise the subcommands + per-subcommand `--help`. |
| No CLI consumer for the emitted #155 blueprint (round-trip dead-end) | commented on **#28** (render a consumer's OWN graph) — the build→consume CLI edge. |
| Three `working` findings | carry-on — the happy path, `--unwired`, and eager per-op diagnostics hold. |