Aura/fieldtests/cycle-0107-campaign-executor/FINDINGS.md

Fieldtest — cycle 0107 — campaign executor + blueprint on-ramp — 2026-07-03

Role: downstream consumer (methodology / campaign designer), public interface only (ledger C16/C18/C20/C24/C25, glossary, CLI --help + introspection verbs, the 0106 fieldtest corpus as shape hints). No crates/ source read. Binary built from HEAD (cargo build --workspace, target/debug/aura, commit 428f83c). Real data: scaffolded demo_project/ (aura new), cargo build for the dylib, ran the executor over the local GER40 2024-09 Pepperstone archive.

Fixtures (this directory)

• c0107_1_smacross.json — SMA-cross op-script, lengths bound (content id 573262…)
• c0107_1_smacross_open.json / _open_built.json — lengths unbound (sweepable), op-script + its graph build envelope (content id 597d71…)
• c0107_2_process.json — sweep(sqn,argmax,deflate) → gate(expectancy_r>0) → walk_forward(rolling, ms lengths) (content id 5a88…)
• c0107_2_campaign.json — campaign missing a bias.scale axis (validates, then fails run)
• c0107_2_campaign_full.json — the runnable campaign (all 3 open params covered), GER40, 2024-09
• c0107_2_realization_campaign_runs.jsonl — the recorded realization (3 identical runs + 1 gate-all)
• c0107_3_process_{mc,gateall,badmetric}.json + c0107_3_campaign_*.json — refusal probes
• c0107_4_process_{plateau_wf,deflate_plateau,wf_toobig}.json + c0107_4_campaign_*.json + c0107_4_campaign_prefixref.json — trip-up probes
• demo_project/ — aura new scaffold + all docs (its runs/ store is project-gitignored; the realization is copied out to the corpus root as evidence)

Findings

F1 [working] The full loop closes — 0106 F7 is answered

aura campaign run c0107_2_campaign_full.json (file = register-then-run) and aura campaign run 533aed18… (id) both execute the cell GER40 × [2024-09-01,10-01] through std::sweep → std::gate → std::walk_forward over real data (exit 0). The gate expectancy_r > 0 kept survivors [0,1,2] (dropped the losing fast=4/slow=16 member, expectancy_r=-0.022); the sweep argmax winner was ordinal 0 (sqn=1.054); walk-forward rolled 2 OOS folds re-sweeping only the survivor points and stamped manifest.selection on the OOS winners. Emit honored: family_table + selection_report lines, always the campaign_run record. The stored CampaignRunRecord (runs/campaign_runs.jsonl) links {campaign, process, run, seed, cells[{stages: family_id / survivor_ordinals / selection}]} over untouched families.jsonl records, run-counted per campaign id (3 successful runs → run 0/1/2). Intent → execution → recorded realization is one continuous, reproducible path. → carry-on.

F2 [working] Determinism is bit-identical across re-run and addressing mode

Two file runs and one id-addressed run of the same campaign are byte-identical after normalizing only the per-campaign run counter in family ids (diff clean on both pairs). Deflation nulls seed from the doc seed=42 (overfit_probability, deflated_score reproduce). Realization is a pure function of doc + stores + data, as C1 promises. → carry-on.

F3 [working] 0106 F1 closed — the document envelope is now introspectable

aura campaign introspect --unwired <bare {}> enumerates the whole envelope, including the previously-invisible seed:

open slot: format_version (required, must be 1)
open slot: kind (required, must be "campaign")
open slot: name / data / strategies / process.ref / seed / presentation …

process introspect --unwired {} likewise names format_version/kind/name/pipeline. The field names (data, strategies, process.ref, presentation) that 0106 F1 could not map are now spelled out. → carry-on.

F4 [working] The honest-refusal surface is loud, precise, and correctly disposed

All probed refusals were clean, actionable prose with the right exit code:

• v1 boundary: process stage 1 (std::monte_carlo) is not executable in v1; executable pipeline shape: std::sweep [std::gate]* [std::walk_forward] (exit 1). std::generalize is symmetric.
• gate-everything-out (expectancy_r > 100): exit 0, family_table + sweep selection_report still emitted, walk-forward truncated, loud stderr cell …: gate at stage 1 left no survivors; cell realization truncated.
• plateau:mean inside walk_forward: walk_forward cannot use a plateau select (a gated survivor subset has no parameter lattice) (exit 1).
• deflate:true + non-argmax: sweep deflate: true composes only with select "argmax" (exit 1).
• old 0106 wf vocab (folds): block std::walk_forward: unknown slot "folds" (exit 1) — the 0107 machinery-true vocabulary correction is enforced.
• bogus file / unknown 64-hex id / uppercase-hex / outside-project: each exit 1 with a precise message (neither a readable .json file nor a 64-hex content id / no campaign <id> in the project store / campaign run needs a project: … no Aura.toml found up from <cwd>).
• persist_taps deferred LOUDLY on stderr before execution (persist_taps not yet honored (N tap(s) ignored)), the 0106 F7 lesson honored. → carry-on.

F5 [friction] The on-ramp rejects the op-script; the graph build bridge is undocumented

The corpus's canonical strategy form is an op-script (c0106_2_strategy.json, c0107_1_*). aura graph introspect --content-id FILE accepts it (shape-discriminated) — but aura graph register FILE and aura graph introspect --params FILE reject it with a raw serde message:

aura: blueprint document is not valid JSON: invalid type: map, expected u32 at line 2 column 2

Only the envelope emitted by aura graph build < op-script > envelope.json is accepted. Nothing in --help, the glossary blueprint entry, or the error names this graph build step, and the three on-ramp verbs disagree on which shapes they accept. (Once bridged, the content id is shape-invariant — op-script and envelope both hash to 573262…, so register and a ref computed from the op-script agree; this part is correct.) Repro: aura graph register c0107_1_smacross_open.json. → plan: shape-discriminate the op-list in register/--params (as --content-id already does), or make the error name the graph build bridge.

F6 [friction] The content: prefix trap persists and the on-ramp emits the wrong-to-paste form

graph|process|campaign register and introspect --content-id all print content:<hash>, but every ref wants the bare hash. A campaign whose process.ref carries the copied content:5a88… form passes intrinsic validation and then fails referential:

aura: campaign references do not resolve:
  process content:5a88…6ea not found in the project store

The error now echoes the offending prefixed ref (an improvement over 0106 F3) but never says "strip the content: prefix", and register still prints exactly the form that breaks. This directly taxes the on-ramp the cycle set out to smooth. Repro: aura campaign validate c0107_4_campaign_prefixref.json. → plan: one bare display form across verbs, or accept/strip the content: prefix in refs (carried from 0106 F3, still open: #194).

F7 [friction] campaign validate passes but campaign run aborts when an open param is uncovered

Referential validation reports all references resolve, axes are in the param space (exit 0), but it only checks that each axis name is in param_space — not that every open param is covered by an axis. The open blueprint has three open params (fast.length, slow.length, bias.scale); a campaign sweeping only the two lengths validates, then aborts at member-bind:

aura: a member failed to bind: open param "graph.bias.scale" of strategy 597d71… is bound by no campaign axis

The executor's "every open knob required, no default" rule (documented for sweep/walkforward in the glossary) is not mirrored in the validator, so "valid" ≠ "runnable". (Secondary: the bind error uses the wrapped graph.bias.scale path while --params and the axis use the raw bias.scale — two namespaces for one knob.) Repro: aura campaign validate c0107_2_campaign.json (passes) then aura campaign run c0107_2_campaign.json (aborts). → plan: extend the referential tier to require full open-param coverage, surfacing the missing knob at validate time; reconcile the raw-vs-wrapped param path in the bind message.

F8 [friction] The walk-forward window-fit refusal leaks a raw Rust Debug struct

A walk_forward whose in_sample_ms exceeds the campaign window refuses correctly, but in internal {:?} form, unlike its clean sibling refusals:

aura: process stage 1: walk_forward windows do not fit the campaign window:
  SpanTooShort { span: (Timestamp(1725148800000), Timestamp(1727740800000)), need: 315964799999 }

The SpanTooShort {…} struct, the Timestamp() wrapper, and the unit-unlabeled need (ms) break the "Debug-free house seam" the cycle otherwise maintains, and a user cannot easily see that need is in_sample_ms + out_of_sample_ms in ms against the window span. Repro: aura campaign run c0107_4_campaign_wf_toobig.json. → debug (RED-pin the prose form).

F9 [doc-gap] The rankable / gate metric roster is enumerable by no verb, nor by the refusal

Block hints say metric name (see metric_vocabulary), but no verb lists it (--vocabulary is the block list; --block metric_vocabulary → unknown block). process validate accepts any string (profit_factor passes intrinsic), and the run-time refusal names no roster:

aura: process stage 0: metric "profit_factor" is not rankable (winner selection needs one of the registry's rankable metrics)

profit_factor is itself emitted in every member's metrics block, so the exists-but-not-rankable split is invisible. A campaign author must guess the sweep/gate/wf metric names (0106 F6, still open: #197). → tighten docs: expose the rankable + gate metric rosters via process introspect (or enumerate them in the refusal).

F10 [spec_gap] Register-then-run stores the campaign before the gate — unrunnable docs accumulate

Every campaign run <file> issued left a runs/campaigns/<id>.json (9 stored), including campaigns that then failed the referential gate (prefixref) or preflight (mc/badmetric/plateauwf/deflateplateau/ wf_toobig). The store thus accumulates never-runnable and referentially-broken documents, with no cleanup. The ledger is silent on whether the file sugar should gate before storing, and whether the content-addressed store should hold referentially-broken docs. → ratify (accept intrinsic-keyed store as-is) or tighten (gate before storing).

Verdict

Yes — the campaign+process pair now delivers the full loop the 0106 verdict said was one wire short. F7 (0106) is closed end to end: a campaign document

• its referenced process document drive aura campaign run through sweep → gate → walk_forward over the real GER40 archive, and the run's realization is recorded as a thin, family-linking CampaignRunRecord that is content-addressed, run-counted, and bit-identical on re-run — persisted intent finally has a consumer, and every id, axis, gate, and selection earns its reason to exist. The authoring on-ramp that 0106 flagged is substantially repaired too: graph register + introspect --params (file and store-id) + --content-id file mode give a blueprint ref a resolvable path (F5-0106), and --unwired {} makes the whole envelope, seed included, derivable (F1-0106). The honest-refusal surface is a genuine strength — v1 boundary, empty gate (exit 0 + truncation), plateau/deflate composition, stale wf vocabulary, and every bad target are refused loudly, precisely, and with the correct disposition.

What remains is friction concentrated at the first-author edge, not the loop: the on-ramp verbs disagree on op-script vs envelope shape and hide the graph build bridge (F5); the content: prefix that every register/--content-id prints is the exact form that silently breaks a ref (F6); campaign validate blesses a campaign the executor then rejects for an uncovered open param (F7); one preflight fault leaks a raw Debug struct (F8); and the metric roster a process must name is still discoverable nowhere (F9). None block the loop; all tax the newcomer. One spec silence to settle: whether register-then-run should store before or after the gate (F10).

Counts: working 4 (F1–F4), friction 4 (F5–F8), doc-gap 1 (F9), spec-gap 1 (F10), bugs 0.

Orthogonal concern (out of 0107 scope)

aura new (cycle 0103) scaffolds a nested .git repo inside the project directory — a gitlink/submodule hazard when the scaffold is embedded in the tracked fieldtests/ corpus. demo_project/.git (and the large demo_project/target/) were removed so the corpus commits cleanly. Worth a scaffold-side note (--no-git or skip when nested under a repo).