Files
Aura/docs/specs/fieldtest-sweep-dissolution.md
T
Brummel 36fbf492ee fieldtest: cycle-0110 sweep-dissolution corpus (GREEN-core)
Consumer scenario against the public surface only: selection-free
process doc round-trip, the dissolved real-data sweep (doc dedupe +
realization accumulation + C1 bit-identity), refusal edges, rank-later
over the persisted family. 0 bugs; the cycle's own axes all came back
working. 4 friction + 2 spec-gap findings on the verb entry ergonomics
and pre-existing guide drift — transcribed in
docs/specs/fieldtest-sweep-dissolution.md for triage.

refs #210
2026-07-04 19:33:58 +02:00

15 KiB
Raw Blame History

Fieldtest — cycle 0110 (sweep dissolution) — 2026-07-04

Status: Draft — awaiting orchestrator triage Author: fieldtester (dispatched by fieldtest skill) Binary exercised: target/debug/aura (workspace built from HEAD 2c72996; project cdylib liblab.so built from the scaffolded lab/ crate).

Scope

Cycle 0110 is cycle 1 of the "verb dissolution" milestone (#210). Two user-visible changes: (1) the std::sweep process block's selection triple (metric+select+deflate) became an optional, all-or-nothing selection group — omit it for a selection-free sweep, permitted only as the pipeline's terminal stage; (2) aura sweep <blueprint.json> --real SYM --from --to --axis k=v,… is now thin sugar over the campaign engine: it auto-registers a generated selection-free process document + a campaign document (content-addressed) and appends a campaign-run record, printing the same member lines as before (member manifests now carry an instrument stamp). This fieldtest exercises both, plus the two consumer paths the selection-free sweep exists for (refusal edges, rank-later).

Examples

All fixtures under fieldtests/cycle-0110-sweep-dissolution/; the scaffolded project is lab/ (its runs/ store is kept as evidence, target/ removed). Real data: GER40, Sep 2024 (--from 1725148800000 --to 1727740799999).

c0110_1_explore_sweep.json — selection-free single-stage process document

  • A bare {"block":"std::sweep"} pipeline authored straight from authoring-guide §2.
  • Fits scope: exercises the new optional-selection-group surface at its simplest (the omitted group).
  • Outcome: process validatevalid (intrinsic): 1 pipeline blocks; process register → id 78f2d8d7…, re-register dedupes (one file); stored canonical form is byte-exactly …"pipeline":[{"block":"std::sweep"}]. process introspect --block std::sweep matches the guide verbatim (metric/select/deflate all "optional"). Ran, matched expected.

c0110_2_smacross_open_op.json (+ built blueprint) — the dissolved verb

  • Unbound SMA-crossover op-script → graph build → blueprint with three open axes; aura sweep <bp> --real GER40 --from … --to … --axis graph.fast.length=2,4 --axis graph.slow.length=8,16 --axis graph.bias.scale=0.5 --name c0110-explore.
  • Fits scope: this is the dissolved verb; a realistic ad-hoc explore-sweep.
  • Outcome: prints 4 GER40-stamped member lines; auto-registers a generated process (9227c300, name "sweep", selection-free) + campaign (46e4f7ef, raw axis names, GER40, ms sub-range window) and appends one campaign_runs.jsonl record; identical re-run dedupes both documents (no new files, same ids) while appending a second realization (family suffix s0-0s0-1, metrics bit-identical — C1); campaign runs lists both. Ran, matched expected. (c0110_2_smacross_op.json, the bound variant, is kept as evidence for finding F3.)

c0110_3a_metric_only.json / c0110_3a_select_only.json / c0110_3b_* — refusal edges

  • Half-populated selection group (metric-only, select-only); a selection-free sweep followed by a std::gate stage, driven to the executor via c0110_3b_campaign_nonterminal.json.
  • Fits scope: the two new refusals (all-or-nothing group; terminal-only).
  • Outcome: half-group → clean intrinsic refusal naming the missing slot; non-terminal free sweep → passes intrinsic, refused at the executable tier and at campaign run with clear prose, no realization appended. Ran, matched expected.

(Task 4) rank-later over the persisted family — aura runs family … rank …

  • No new fixture; consumes the 46e4f7ef-0-GER40-w0-s0-0 family persisted by example 2.
  • Fits scope: the rank-later pattern the selection-free sweep exists for.
  • Outcome: rank sqn / rank expectancy_r return the 4 members best-first; ranking respects metric directionality (max_drawdown ascending, sqn descending); the rank roster equals the rankable metric roster (win_rate/nonexistent refused with the known list). Ran, matched expected.

Findings

[working] Selection-free process document round-trips cleanly

  • Example 1.
  • process validateprocess document valid (intrinsic): 1 pipeline blocks, 0 gate predicates; register id 78f2d8d7…, idempotent on re-register; stored form is the bare block; introspect --block std::sweep output is byte-for-byte the guide's transcription.
  • Working: the new optional group is reachable, authorable from the doc alone, and the introspection self-description matches the ledger/guide.
  • Action: carry-on.

[working] The dissolved sweep verb generates + dedupes a selection-free campaign

  • Example 2.
  • One invocation produced the generated process 9227c300 ({"name":"sweep","pipeline":[{"block":"std::sweep"}]}), campaign 46e4f7ef (raw axes fast.length/slow.length/bias.scale, instrument: GER40, window [1725236099999,1727726220000] — a ms sub-range of the request), and one campaign_runs.jsonl line. Identical re-run: no new documents (same content ids), a second realization (run 1, family suffix s0-1), member metrics bit-identical. campaign runs lists both runs.
  • Working: the sugar→campaign parity, content-addressed dedup, deterministic accumulation (C1), and the additive instrument stamp all hold end to end.
  • Action: carry-on.

[working] All-or-nothing + terminal-only refusals are clear and guide-matching

  • Examples 3a/3b.
  • metric-only → block std::sweep: slot "metric" without "select" — the selection group is all-or-nothing; select-only symmetric; non-terminal free sweep → campaign is not executable: process stage 0: a sweep without a selection group (metric + select) must be the last stage of its process (same prose at campaign run, no realization appended).
  • Working: both new refusals fire at the right tier with comprehensible prose that matches authoring-guide §2.
  • Action: carry-on.

[working] Rank-later over the persisted family, best-first, roster-enforced

  • Task 4.
  • rank sqn orders 1.054 > 0.849 > 0.052 > 0.545; rank max_drawdown orders 4.22M < 6.16M < 6.57M < 9.14M (ascending — lower drawdown ranked better); rank win_rateunknown metric 'win_rate' (known: total_pips, max_drawdown, bias_sign_flips, sqn, sqn_normalized, expectancy_r, net_expectancy_r).
  • Working: the selection-free sweep's reason to exist (defer selection to a rank-later step) works, and the rank roster equals the documented rankable set. (Direction convention noted as F-min below.)
  • Action: carry-on.

[spec_gap] aura sweep --axis requires the graph.-wrapped name; the docs teach the raw form

  • Example 2 (works with graph.fast.length) vs probe P2 (fails with fast.length).
  • graph introspect --params prints fast.length and the authoring guide (§1) calls that "the raw param-space namespace (what a campaign's axes bind against)". The stored campaign doc also uses the raw form. But the dissolved aura sweep --axis accepts only the wrapped graph.<param> form — discoverable solely via aura sweep <bp> --list-axes, which the authoring guide never mentions (the guide does not document aura sweep at all; the glossary and --help say only --axis name=<csv>).
  • Why spec_gap: the public interface documents the raw namespace as canonical and is silent on the wrapper; a consumer following it picks the wrong form. The wrapper leaks the root-composite instance name (graph) into the user's axis vocabulary.
  • Recommended action: document (glossary sweep + a note in authoring-guide) that aura sweep --axis uses the --list-axes (wrapped) names, or make the dispatch accept the raw --params names too. → tighten the design ledger / docs (+ plan if the surface is to accept raw).

[friction] The wrapped-name refusal mangles the axis the user typed

  • Probe P2: aura sweep … --axis fast.length=2,4 --axis slow.length=8,16 --axis bias.scale=0.5strategy 597d…: axis "length" is not in the param space / axis "scale" is not in the param space / open param "fast.length" is bound by no campaign axis
  • The #203 first-segment strip consumes the leading fast/slow/bias segment, so the diagnostic names length/scale — fragments the user never typed as a whole axis — while separately reporting the real params as "bound by no axis". The two same-stripped legs (fast.length,slow.length) even collapse to a single length key. The message never says "a leading segment was stripped as the graph name".
  • Why friction: the task can be completed (use the wrapped name), but the refusal actively misdirects — it names a mangled fragment instead of echoing the input and explaining the wrapper convention. (Same root cause as the spec_gap above; split per the one-class-per-finding rule.)
  • Recommended action: echo the typed axis and/or name the expected wrapper in the refusal. → plan (tidy iteration).

[spec_gap] Authoring-guide §1 inline SMA-crossover binds length, contradicting every id / --params / §3 axis it prints

  • Encountered building the sweep blueprint (Task 2); evidence c0110_2_smacross_op.json.
  • The guide's §1 worked op-script binds fast.length=2 and slow.length=4 and claims (line 72) it matches the on-disk corpus mra_1_strategy_smacross.json. It does not: the on-disk file leaves both unbound. The bound inline form hashes to e2be81b2… with --params = bias.scale only; the unbound corpus form hashes to 597d719b… (the id the guide later prints at §1/§3 and the id every campaign example references) with --params = fast.length,slow.length,bias.scale. A reader copying the inline example gets a blueprint whose id matches nothing in the guide and whose fast.length/slow.length are closed — so authoring-guide §3's campaign (axes fast.length,slow.length) would refuse against it.
  • Why spec_gap: the guide is internally inconsistent, and the underlying rule — that an op-script bind pins a param out of the sweepable space (not a sweep-overridable default) — is nowhere stated. Note the 0110 audit commit claimed "worked example verified against the live binary"; this §1 example (authored earlier, #197/#208) was not.
  • Recommended action: drop the bind from the §1 inline example (or state that binding closes the axis) and document bind-vs-axis interaction. → tighten docs.

[friction] aura sweep <op-script> rejects the op-script array with a raw serde error

  • Probe: aura sweep c0110_2_smacross_open_op.json --list-axesaura: …_op.json: Json(Error("invalid type: map, expected u32", line: 2, column: 2)).
  • graph introspect --content-id/--params/register auto-discriminate the op-script array vs the built envelope (guide §1), but aura sweep's blueprint slot accepts only the built envelope, and the mismatch surfaces as a leaked serde Json(Error(...)) rather than a house-style "expected a built blueprint; run aura graph build first". The guide never says aura sweep needs a pre-built blueprint.
  • Why friction: the path completes once you build first, but the diagnostic is an internal-type leak (contrast #162's Display-free-error convention seen elsewhere) and the required build step is undocumented for aura sweep.
  • Recommended action: shape-discriminate (as graph introspect does) or emit a house-style hint. → plan.

[friction] Refused dissolved sweeps still register their generated campaign document (store litter, some malformed)

  • Probes P2 (raw names) and P3 (subset axes) both refused at the referential tier, yet each left a generated campaign doc in runs/campaigns/ (92c66b18… "c0110-rawname", b0e3f90b… "c0110-subset"). campaign_runs.jsonl correctly stayed at 2 (no realization). The rawname doc is malformed: its axes are the stripped/collapsed {"length":[8,16], "scale":[0.5]} — a document the tool itself immediately rejects, now permanent in the content-addressed store.
  • Why friction: the sugar registers the generated process+campaign documents before referential/executable validation, so a fat-fingered axis pollutes the store with dead (occasionally malformed) documents. Content-addressing bounds the growth (identical bad invocations dedupe), and no realization is recorded, so nothing downstream breaks — hence friction, not bug. (campaign run <file> registering before an executable refusal — the Task-3 d10ed054… doc — is by contrast documented register-then-run behaviour and is not flagged.)
  • Recommended action: in the sweep sugar, validate the generated documents (referential/executable) before register_generated, registering only on a clean preflight. → plan.

[friction] process/campaign intrinsic validate needs the built project cdylib when inside a project

  • Task 1 setup: inside the fresh lab/ project, process validate c0110_1_explore_sweep.json (and register/introspect) first failed with aura: cargo metadata failed: no cdylib target …, then after cargo build with aura: project dylib not found … run cargo build. The very same process validate runs dylib-free outside any project (probe P1 → valid (intrinsic)).
  • Why friction: a std-only process document references no project nodes, so the intrinsic (and referential-over-std) check needs no compiled dylib; the CLI nonetheless loads the project env eagerly for any process/campaign subcommand once an Aura.toml is found. A newcomer following the guide inside a fresh project hits a build wall on a pure-shape check. (Likely pre-existing, not introduced by 0110; the error is actionable.)
  • Recommended action: let the intrinsic tier run before/without the dylib load inside a project, or document that any in-project process/campaign call requires a built cdylib. → plan or ratify (decide the intended coupling).

Recommendation summary

Finding Class Action
Selection-free process doc round-trips working carry-on
Dissolved sweep → deduped selection-free campaign working carry-on
All-or-nothing + terminal-only refusals clear working carry-on
Rank-later best-first, roster-enforced working carry-on
aura sweep --axis needs wrapped name, docs teach raw spec_gap tighten docs (+ plan if raw to be accepted)
Wrapped-name refusal mangles the typed axis friction plan
Guide §1 inline binds length, inconsistent with its ids/axes spec_gap tighten docs
aura sweep <op-script> leaks a serde error friction plan
Refused sweeps still register their generated campaign doc friction plan
In-project intrinsic validate needs the built cdylib friction plan / ratify