Files
Aura/fieldtests/cycle-309-trace-handle/TRANSCRIPT.md
T
claude a09aaadcbd fieldtest: trace-handle cycle — 5 examples, 3 bugs / 4 friction / 2 spec-gaps / 4 working
Source-blind field test of the handle echo and the rewritten chart refusal
(range 9636b00..9f87e5a), driven from the public interface only: the built
binary's help and refusals, the authoring guide, the glossary, the ledger.
Replayable via c309_0_run_all.sh, which rebuilds its lab from scratch and
captures stdout/stderr per step.

The chain the cycle promised holds: take trace_name off a run's stdout, feed
it to chart and to measure ic, never once list the trace directory. Both legs
report the handle identically — including the measurement leg the audit
caught shipping without a pin — and a tap-free run emits no key at all,
consistently across stdout and the stored record. The family-id arm proved
its worth: the intuitive cut of a family id yields a REAL but wrong handle
(another run's data), and the arm refuses to make it.

Findings routed: the three bugs fixed in-cycle RED-first (51ac155 and its
successor); the friction and spec-gap items filed as a collective issue.

refs #309
2026-07-26 23:43:39 +02:00

8.3 KiB
Raw Blame History

cycle-309 (trace handle) — field-test transcript

Source-blind run against the public interface only: the built binary's help and refusals, docs/authoring-guide.md, docs/glossary.md, docs/design/, and git log. Nothing under crates/ was opened.

Binary: target/debug/aura, built from this working tree with cargo build (HEAD 9f87e5a, manifests stamp 9f87e5a…-dirty — the untracked corpus). Replay: ./c309_0_run_all.sh (rebuilds the lab from scratch; every step's stdout/stderr lands beside it as c309_*.out / c309_*.err).

Project: c309lab/, scaffolded by aura new. Its runs/ is git-ignored by the scaffold's own .gitignore, so the ~15 MB of recorded traces stay out of the commit; the authored fixtures (op-scripts, process + campaign documents) and the captured outputs are the corpus.


Axis 1 — the echo, on both legs

Strategy leg — c309_1_spread_strategy.ops.json

An SMA-crossover bias with two declared taps (spread on sub.value, signal on bias.bias) and a name op (c309_spread).

$ aura graph introspect --taps ../c309_1_spread_strategy.bp.json
spread  sub.value  F64
signal  bias.bias  F64
$ aura exec ../c309_1_spread_strategy.bp.json
{"manifest":{…},"metrics":{…},"trace_name":"c309_spread"}     # exit 0, stderr empty
$ find runs/traces/c309_spread
runs/traces/c309_spread/{index.json,spread.json,signal.json}

The handle is the last key of the same stdout object as the report; nothing is printed on stderr, and the report keys are unchanged.

Measurement leg — c309_3_measurement.ops.json

No expose, no bias output: a Scale(1.0) price passthrough tapped as price and the fastslow spread tapped as signal.

$ aura exec ../c309_3_measurement.bp.json
{"manifest":{…,"broker":"measurement",…},"taps":["price","signal"],"trace_name":"c309_measure"}

Same field, same position. (First attempt failed at build time on a port name I guessed — aura: op 4 (feed): node "px" has no input port "input"; graph introspect --node Scale names it signal. Clean diagnostic, one retry.)

Axis 2 — the chain, without knowing the name beforehand

$ H=$(jq -r '.trace_name' < ../c309_1_run.out)   # -> c309_spread
$ aura chart "$H"                                 # exit 0, HTML on stdout
   series: c309_spread, spread, signal
$ aura chart "$H" --tap spread                    # exit 0, series: c309_spread, spread

$ M=$(jq -r '.trace_name' < ../c309_3_run.out)   # -> c309_measure
$ aura measure ic "$M" --signal signal --price price
{"run":"c309_measure","signal_tap":"signal","price_tap":"price","horizon":1,
 "permutations":1000,"seed":0,"n_pairs":13,"information_coefficient":0.148…,
 "overfit_probability":0.300…}

Both destinations accept the printed value verbatim. No ls runs/traces/ anywhere in the loop.

Fold-only variant (--tap spread=mean) still reports trace_name; the one-row trace charts, and the previous run's leftover signal.json in the same directory is not resurrected — the rerun rewrites index.json ("taps":["spread"]) and the chart shows spread alone.

Unnamed blueprint (name op stripped): handle graph, aura chart graph exits 0.

Axis 3 — the two refusal arms

Arm A — not a family id

$ aura chart no_such_run                                         # exit 1
aura: no recorded run or family 'no_such_run' under runs/traces (a single run
prints its handle as `trace_name` on stdout; a campaign run prints it as
`campaign_run.trace_name`, one per run — check that value for a typo. A trace
is produced by `aura exec --tap <NODE.FIELD>=<FOLD>` on a blueprint or a
campaign's `presentation.persist_taps` section, not by naming a handle here)

Claim-by-claim:

claim verdict
a single run prints trace_name on stdout TRUE (both legs, above)
a campaign run prints campaign_run.trace_name, one per run TRUE (two runs → c364e8c3-0, c364e8c3-1)
a campaign's presentation.persist_taps produces a trace TRUE
aura exec --tap <NODE.FIELD>=<FOLD> produces a trace FALSE (see below)
$ aura exec ../c309_1_spread_strategy.bp.json --tap sub.value=record   # exit 2
aura: the tap plan names 'sub.value', but the blueprint declares no such tap — declared taps: spread, signal
$ aura exec ../c309_1_spread_strategy.bp.json --tap spread=mean        # exit 0

The flag takes the declared tap NAME, as exec --help says (--tap <TAP=FOLD>); and it is not needed at all — the two runs at the top of this transcript recorded their traces with no flag.

Arm B — a family id out of the families listing

$ aura runs families
{"family_id":"c364e8c3-0-GER40-w0-r0-s0-0","kind":"Sweep","members":4}
$ aura chart c364e8c3-0-GER40-w0-r0-s0-0                          # exit 1
aura: no recorded run or family 'c364e8c3-0-GER40-w0-r0-s0-0' under runs/traces — that
looks like a family id, not a trace handle. Recorded trace handles for campaign
'c364e8c3': c364e8c3-0.
aura: Did you mean: aura chart c364e8c3-0
$ aura chart c364e8c3-0                                            # exit 0
   series: c364e8c3-0, 88a221aa-GER40-w0/fast.length_2__slow.length_8__…  (4 members)

After a second run of the same campaign, the same arm lists both handles and suggests nothing:

$ aura chart c364e8c3-0-GER40-w0-r0-s0-1                           # exit 1
… Recorded trace handles for campaign 'c364e8c3': c364e8c3-0, c364e8c3-1.

That second case is the trap the arm exists for: the family id's trailing segment is the run (-s0-1 → run 1), so the naive cut to the leading pair gives c364e8c3-0 — a real, chartable handle holding a different run's data. The arm refuses instead of guessing.

Zero-handle sub-case (a campaign run with persist_taps: [], and a campaign head never recorded at all) — identical prose for both:

$ aura chart 94355db6-0-GER40-w0-r0-s0-0     # this campaign recorded nothing
$ aura chart deadbeef-0-GER40-w0-r0-s0-0     # no such campaign at all
aura: no recorded run or family '…' under runs/traces — that looks like a family id,
not a trace handle. The handle is the `trace_name` the campaign run printed; this id's
second segment counts strategies while the handle's counts runs, so cutting the id
down is not a way to derive it.

Following that advice: the run in question printed campaign_run without a trace_name key (has("trace_name") == false). The pointer is true but empty — the arm knows the campaign recorded nothing and does not say so.

Axis 4 — the tap-free case

$ aura exec blueprints/signal.json                       # the scaffold's own signal
{"manifest":{…},"metrics":{…}}                           # exit 0
$ jq 'has("trace_name")' < …                              false

No key, no note, no stderr. Same for the campaign leg with persist_taps: [] (campaign_run has no trace_name key) and for the stored campaign_runs.jsonl record. Absent-when-nothing is consistent across all three surfaces.

$ aura exec blueprints/signal.json --tap spread=record   # exit 2
aura: the tap plan names 'spread', but the blueprint declares no such tap — declared taps: 
$ aura graph introspect --taps blueprints/signal.json    # exit 0
aura: note: blueprints/signal.json declares no taps

The refusal's roster runs dry mid-sentence where the introspect verb says it plainly.

Beyond the arms — measure ic on a handle it will not take

$ aura measure ic c364e8c3-0 --signal equity --price r_equity                # exit 1
aura: reading run 'c364e8c3-0' traces failed: no recorded run 'c364e8c3-0' under runs/traces

runs/traces/c364e8c3-0/ exists — aura chart c364e8c3-0 renders it, and the member path underneath is accepted:

$ aura measure ic 'c364e8c3-0/88a221aa-GER40-w0/fast.length_2__slow.length_8__stop_length_3__stop_k_2' \
    --signal equity --price r_equity                                         # exit 0
{"run":"c364e8c3-0/88a221aa-…","n_pairs":24455,"information_coefficient":0.00796…,…}

And measure ic's own not-found refusal carries no route back to where a handle comes from, unlike chart's:

$ aura measure ic no_such_run --signal signal --price price                  # exit 1
aura: reading run 'no_such_run' traces failed: no recorded run 'no_such_run' under runs/traces

(Its wrong-tap refusal, by contrast, is exemplary: aura: run 'c309_measure' has no tap 'close' (taps: ["price", "signal"]).)