f68258b044
Public-interface-only field test of the walking-skeleton closing seam (#8): the `aura run` CLI and the reusable aura-std::Recorder sink. A standalone consumer crate (path-deps on the three engine crates, as a C16 project would) drives the real binary as a subprocess and wires Recorder via the raw Harness::bootstrap API. Examples (all built from HEAD and ran green): - c0010_1: drive `aura run` as a downstream CLI/jq consumer — single-line JSON report, byte-identical across two runs (C1), bad-args → exit 2 + exact usage on stderr with empty stdout. - c0010_2: wire Recorder onto Sma(3) via raw bootstrap, drain the mpsc::Receiver — rows match the rustdoc warm-up model exactly. - c0010_3: Recorder over [I64, Bool, Timestamp] — the four-kind claim (never exercised by the CLI's f64-only path) holds, verified from rustdoc alone. Findings: 0 bugs, 4 working, 1 spec_gap, 1 friction. - [spec_gap] `aura run <trailing-arg>` is accepted (exit 0), not rejected — the arg parse keys only on the first token being `run` and ignores the rest. The cycle_scope's "any other or missing subcommand -> exit 2" does not unambiguously cover `run <junk>`; the binary chose the lenient reading. Tracked for a ratify-or-tighten decision. - [friction] verifying the documented {manifest, metrics} JSON nesting needs a hand-rolled string walker — the zero-dep consumer has only to_json() and no typed read-path. Low-priority tidy. Both non-bug findings filed to the forward queue; neither blocks the cycle.
164 lines
9.4 KiB
Markdown
164 lines
9.4 KiB
Markdown
# Fieldtest — cycle-0010 — 2026-06-04
|
|
|
|
**Status:** Draft — awaiting orchestrator triage
|
|
**Author:** fieldtester (dispatched by fieldtest skill)
|
|
|
|
## Scope
|
|
|
|
Cycle 0010 is the Walking-skeleton milestone's closing seam (#8). It shipped two
|
|
public surfaces:
|
|
|
|
1. **`aura_std::Recorder`** — a reusable recording-sink node. A pure consumer
|
|
(`output: vec![]`) over `kinds.len()` input columns, holding an
|
|
`mpsc::Sender<(Timestamp, Vec<Scalar>)>` as its out-of-graph destination
|
|
(C7/C8/C22). Constructor: `Recorder::new(kinds: &[ScalarKind], firing: Firing,
|
|
tx) -> Self`. Each fired cycle it sends `(ctx.now(), row)` with the newest
|
|
value of every column; returns `None` and records nothing until all columns
|
|
are warm. Supports all four base scalar kinds.
|
|
2. **The `aura run` CLI** (crate `aura-cli`, bin `aura`) — bootstraps a built-in
|
|
sample harness (synthetic source → SMA(2)/SMA(4) → Sub → Exposure → SimBroker
|
|
→ two Recorder sinks), runs it deterministically (C1), and prints the
|
|
cycle-0009 `RunReport` as canonical single-line JSON to stdout (exit 0). Any
|
|
other or missing subcommand prints `aura: usage: aura run` to stderr, exit 2.
|
|
|
|
Tested from the public interface only (rustdoc, design ledger, glossary, project
|
|
layout, the observable behaviour of the `aura` binary) — never `crates/*/src`.
|
|
The CLI's own functions are `pub(crate)`, so axis (a) drove the real binary as a
|
|
subprocess. The build exercised: workspace `cargo build` from HEAD
|
|
(`target/debug/aura`) for the binary; `cargo run --manifest-path
|
|
fieldtests/cycle-0010-aura-run/Cargo.toml` for the two library-consumer
|
|
fixtures.
|
|
|
|
## Examples
|
|
|
|
### fieldtests/cycle-0010-aura-run/c0010_1_cli_report.rs — `aura run` as a downstream CLI consumer (axis a)
|
|
- Spawns the real `aura` binary as a subprocess (path via `$AURA_BIN`, default
|
|
the sibling workspace `target/debug/aura`); captures stdout/stderr/exit for
|
|
`run`, no-args, an unknown subcommand, and `run` with a trailing junk arg.
|
|
- Fits the cycle's headline C14 move: a shell/jq/registry consumer parses the
|
|
single-line JSON report and relies on the exit-code + determinism contract.
|
|
- Outcome: built (from HEAD) and ran; exit 0 with `{manifest:{commit,params,
|
|
window,seed,broker}, metrics:{total_pips,max_drawdown,exposure_sign_flips}}`,
|
|
byte-identical across two runs, clean stderr; bad-args → exit 2 + exact usage.
|
|
Matched expected. Surfaced one spec_gap (trailing-arg leniency) and one
|
|
friction (no JSON parser in the consumer).
|
|
|
|
### fieldtests/cycle-0010-aura-run/c0010_2_recorder_sink.rs — `aura_std::Recorder` as a reusable sink block (axis b)
|
|
- Wires the shipped `Recorder` onto an `Sma(3)` output via the raw
|
|
`Harness::bootstrap(nodes, sources, edges)` API (exactly as a C16 project
|
|
would), runs 5 ticks, drains the `mpsc::Receiver`, asserts the recorded
|
|
`(Timestamp, Vec<Scalar>)` rows against the rustdoc warm-up model.
|
|
- Fits the cycle's first deliverable: Recorder used the way a downstream project
|
|
consumes it, not the way the CLI's built-in harness does.
|
|
- Outcome: built and ran first try; rows `[(3,[20.0]),(4,[30.0]),(5,[40.0])]`
|
|
exactly matched the hand model (no warm-up rows, one row per warm cycle,
|
|
ts = `ctx.now()`). Matched expected.
|
|
|
|
### fieldtests/cycle-0010-aura-run/c0010_3_recorder_four_kind.rs — Recorder four-kind support (axis c)
|
|
- Authors a tiny custom 3-field producer node (i64 counter / bool even? /
|
|
timestamp now) in Rust (C17), wires a multi-column
|
|
`Recorder::new(&[I64, Bool, Timestamp], …)` over it via three field-wise edges,
|
|
runs, drains, asserts the recorded Scalars are `I64`/`Bool`/`Ts`.
|
|
- Fits the cycle's four-kind claim: the CLI only ever exercises the f64 path, so
|
|
the non-f64 kinds are verifiable from the public rustdoc alone only this way.
|
|
- Outcome: built (after one trivial authoring fix — see working finding) and
|
|
ran; rows matched `[(100,[I64(1),Bool(false),Ts(100)]),(200,[I64(2),Bool(true),
|
|
Ts(200)]),(300,[I64(3),Bool(false),Ts(300)])]` exactly. Matched expected.
|
|
|
|
## Findings
|
|
|
|
### [working] `aura run` end-to-end: clean single-line JSON, deterministic, jq-parseable
|
|
- Example: c0010_1.
|
|
- What happened: `aura run` → exit 0, one line of JSON on stdout, empty stderr.
|
|
The documented `{manifest, metrics}` nesting and all eight documented fields
|
|
(`commit, params, window, seed, broker`; `total_pips, max_drawdown,
|
|
exposure_sign_flips`) are present. Two runs are byte-identical (C1). Piped
|
|
through `jq`, `.metrics.total_pips` is a number, `.manifest.params | keys`
|
|
resolves — a downstream C18/shell consumer can author against it.
|
|
- Why working: the headline C14 "run a sim, emit structured metrics" move is
|
|
reachable from a real binary, from rustdoc alone, on the first try, with the
|
|
determinism and shape the ledger promises.
|
|
- Recommended action: carry-on.
|
|
|
|
### [working] Bad-args contract exact and stdout-clean
|
|
- Example: c0010_1.
|
|
- What happened: `aura` (no args) and `aura play` (unknown) each → exit 2,
|
|
stderr exactly `aura: usage: aura run\n`, stdout empty (0 bytes). A consumer
|
|
keying off exit code never sees a half-report on stdout.
|
|
- Why working: matches the cycle_scope contract precisely; the stdout/stderr
|
|
split is clean (errors never contaminate the machine-readable channel).
|
|
- Recommended action: carry-on.
|
|
|
|
### [working] Recorder as a reusable sink: warm-up + row shape match rustdoc
|
|
- Example: c0010_2.
|
|
- What happened: a downstream wiring of `Recorder` over `Sma(3)` via raw
|
|
`Harness::bootstrap`, drained from `mpsc::Receiver`, produced exactly the rows
|
|
the rustdoc model predicts — no warm-up rows, one one-element `Vec<Scalar>`
|
|
per warm cycle, each tagged `ctx.now()`. The constructor signature on the
|
|
surface (`&[ScalarKind]`, `Firing`, `Sender`) is enough to author against with
|
|
no source reading.
|
|
- Why working: the cycle's primary deliverable is reachable and correct from the
|
|
public interface as a standalone consumer block (not just inside the CLI).
|
|
- Recommended action: carry-on.
|
|
|
|
### [working] Recorder four-kind claim holds on a non-f64 column set
|
|
- Example: c0010_3.
|
|
- What happened: a `Recorder` over `[I64, Bool, Timestamp]` recorded
|
|
`Scalar::I64` / `Scalar::Bool` / `Scalar::Ts` values verbatim, matching the
|
|
rustdoc's four-kind claim — a path the CLI never exercises. The trivial
|
|
authoring miss (`FieldSpec.name` is `&'static str`, I wrote `.to_string()`)
|
|
was caught by the compiler with an exact fix-it; the public `FieldSpec`
|
|
rustdoc states `pub name: &'static str` plainly. No source reading needed.
|
|
- Why working: the four-kind claim, otherwise unverifiable from the CLI, is
|
|
empirically true; the supporting types (`FieldSpec`, `Scalar`, multi-edge
|
|
field-wise binding) author cleanly from rustdoc.
|
|
- Recommended action: carry-on.
|
|
|
|
### [spec_gap] `aura run <trailing-arg>` is accepted (exit 0), not rejected
|
|
- Example: c0010_1.
|
|
- What happened: `aura run extra-garbage` printed the full JSON report and exited
|
|
0 — trailing args after `run` are silently ignored. The cycle_scope states
|
|
"Any **other** or missing subcommand prints `aura: usage: aura run` to stderr
|
|
and exits 2."
|
|
- Why spec_gap: `run <junk>` is neither a clean `run` nor a *different*
|
|
subcommand, so the contract does not unambiguously cover it. The binary picked
|
|
the lenient reading (key only on the first arg being `run`, ignore the rest);
|
|
the strict reading (any extra/unexpected token → usage + exit 2) is equally
|
|
plausible and is what a user who typos `aura run --sweep` would expect (they'd
|
|
get a silent full run instead of a hint). I asserted the lenient reading the
|
|
binary exhibits and recorded the ambiguity rather than treating it as a bug.
|
|
- Recommended action: ratify (document the arg-parse contract as
|
|
"first-arg-keyed, trailing args ignored") **or** tighten — decide whether
|
|
unexpected trailing tokens should be a usage error. A one-line note on the
|
|
intended arg contract in the `aura` crate rustdoc closes it.
|
|
|
|
### [friction] Verifying the documented JSON nesting needs a hand-rolled walker
|
|
- Example: c0010_1.
|
|
- What happened: the consumer crate is zero-external-dependency by design (no
|
|
serde/serde_json), and `RunReport` exposes only `to_json() -> String` with no
|
|
typed accessor. To confirm the documented `{manifest, metrics}` nesting and
|
|
field presence, the fixture hand-walks the string (`find("\"key\":")`) — a
|
|
consumer authoring real assertions against the report shape either pulls in a
|
|
JSON dep or writes this dance.
|
|
- Why friction: the task (confirm the structured-face shape) completed, but the
|
|
surface offered no programmatic handle on the report's structure short of
|
|
re-parsing the string the engine just serialized. The cycle's C14 framing
|
|
("machine-readable") implies a consumer can *inspect* the structure, not only
|
|
re-stringify it.
|
|
- Recommended action: plan (tidy) — consider a minimal typed read-path
|
|
(e.g. `RunReport` already round-trips its own fields; exposing the structured
|
|
values, or shipping a tiny documented parse helper, would let consumers assert
|
|
without a serde dep). Low priority; the JSON is correct and jq-parseable for
|
|
shell consumers today.
|
|
|
|
## Recommendation summary
|
|
|
|
| Finding | Class | Action |
|
|
|---|---|---|
|
|
| `aura run` end-to-end JSON + determinism | working | carry-on |
|
|
| Bad-args contract exact + stdout-clean | working | carry-on |
|
|
| Recorder reusable sink matches rustdoc | working | carry-on |
|
|
| Recorder four-kind claim holds | working | carry-on |
|
|
| `aura run <trailing-arg>` accepted | spec_gap | ratify / tighten the design ledger |
|
|
| JSON nesting needs a hand-rolled walker | friction | plan (tidy) |
|