Aura/fieldtests/cycle-0107-campaign-executor/FINDINGS.md

# Fieldtest — cycle 0107 — campaign executor + blueprint on-ramp — 2026-07-03

**Role:** downstream consumer (methodology / campaign designer), public interface
only (ledger C16/C18/C20/C24/C25, glossary, CLI `--help` + introspection verbs,
the 0106 fieldtest corpus as shape hints). No `crates/` source read. Binary
built from HEAD (`cargo build --workspace`, `target/debug/aura`, commit
`428f83c`). Real data: scaffolded `demo_project/` (`aura new`), `cargo build`
for the dylib, ran the executor over the local GER40 2024-09 Pepperstone
archive.

## Fixtures (this directory)

- `c0107_1_smacross.json` — SMA-cross op-script, lengths **bound** (content id `573262…`)
- `c0107_1_smacross_open.json` / `_open_built.json` — lengths **unbound** (sweepable), op-script + its `graph build` envelope (content id `597d71…`)
- `c0107_2_process.json` — sweep(sqn,argmax,deflate) → gate(expectancy_r>0) → walk_forward(rolling, ms lengths) (content id `5a88…`)
- `c0107_2_campaign.json` — campaign missing a `bias.scale` axis (validates, then fails run)
- `c0107_2_campaign_full.json` — the runnable campaign (all 3 open params covered), GER40, 2024-09
- `c0107_2_realization_campaign_runs.jsonl` — the recorded realization (3 identical runs + 1 gate-all)
- `c0107_3_process_{mc,gateall,badmetric}.json` + `c0107_3_campaign_*.json` — refusal probes
- `c0107_4_process_{plateau_wf,deflate_plateau,wf_toobig}.json` + `c0107_4_campaign_*.json` + `c0107_4_campaign_prefixref.json` — trip-up probes
- `demo_project/` — `aura new` scaffold + all docs (its `runs/` store is project-gitignored; the realization is copied out to the corpus root as evidence)

## Findings

### F1 [working] The full loop closes — 0106 F7 is answered
`aura campaign run c0107_2_campaign_full.json` (file = register-then-run) and
`aura campaign run 533aed18…` (id) both execute the cell
`GER40 × [2024-09-01,10-01]` through `std::sweep → std::gate →
std::walk_forward` over real data (exit 0). The gate `expectancy_r > 0` kept
survivors `[0,1,2]` (dropped the losing `fast=4/slow=16` member,
`expectancy_r=-0.022`); the sweep argmax winner was ordinal 0 (`sqn=1.054`);
walk-forward rolled 2 OOS folds re-sweeping **only** the survivor points and
stamped `manifest.selection` on the OOS winners. Emit honored: `family_table` +
`selection_report` lines, always the `campaign_run` record. The stored
`CampaignRunRecord` (`runs/campaign_runs.jsonl`) links
`{campaign, process, run, seed, cells[{stages: family_id / survivor_ordinals /
selection}]}` over untouched `families.jsonl` records, run-counted per campaign
id (3 successful runs → run 0/1/2). Intent → execution → recorded realization
is one continuous, reproducible path. → carry-on.

### F2 [working] Determinism is bit-identical across re-run and addressing mode
Two file runs and one id-addressed run of the same campaign are
**byte-identical** after normalizing only the per-campaign run counter in
family ids (`diff` clean on both pairs). Deflation nulls seed from the doc
`seed=42` (`overfit_probability`, `deflated_score` reproduce). Realization is a
pure function of doc + stores + data, as C1 promises. → carry-on.

### F3 [working] 0106 F1 closed — the document envelope is now introspectable
`aura campaign introspect --unwired <bare {}>` enumerates the whole envelope,
including the previously-invisible `seed`:
```
open slot: format_version (required, must be 1)
open slot: kind (required, must be "campaign")
open slot: name / data / strategies / process.ref / seed / presentation …
```
`process introspect --unwired {}` likewise names
`format_version/kind/name/pipeline`. The field names (`data`, `strategies`,
`process.ref`, `presentation`) that 0106 F1 could not map are now spelled out.
→ carry-on.

### F4 [working] The honest-refusal surface is loud, precise, and correctly disposed
All probed refusals were clean, actionable prose with the right exit code:
- v1 boundary: `process stage 1 (std::monte_carlo) is not executable in v1;
  executable pipeline shape: std::sweep [std::gate]* [std::walk_forward]`
  (exit 1). `std::generalize` is symmetric.
- gate-everything-out (`expectancy_r > 100`): exit **0**, `family_table` +
  sweep `selection_report` still emitted, walk-forward **truncated**, loud
  stderr `cell …: gate at stage 1 left no survivors; cell realization
  truncated`.
- `plateau:mean` inside walk_forward: `walk_forward cannot use a plateau select
  (a gated survivor subset has no parameter lattice)` (exit 1).
- `deflate:true` + non-argmax: `sweep deflate: true composes only with select
  "argmax"` (exit 1).
- old 0106 wf vocab (`folds`): `block std::walk_forward: unknown slot "folds"`
  (exit 1) — the 0107 machinery-true vocabulary correction is enforced.
- bogus file / unknown 64-hex id / uppercase-hex / outside-project: each exit 1
  with a precise message (`neither a readable .json file nor a 64-hex content
  id` / `no campaign <id> in the project store` / `campaign run needs a
  project: … no Aura.toml found up from <cwd>`).
- `persist_taps` deferred **LOUDLY** on stderr before execution
  (`persist_taps not yet honored (N tap(s) ignored)`), the 0106 F7 lesson
  honored.
→ carry-on.

### F5 [friction] The on-ramp rejects the op-script; the `graph build` bridge is undocumented
The corpus's canonical strategy form is an **op-script** (`c0106_2_strategy.json`,
`c0107_1_*`). `aura graph introspect --content-id FILE` accepts it
(shape-discriminated) — but `aura graph register FILE` and `aura graph
introspect --params FILE` **reject** it with a raw serde message:
```
aura: blueprint document is not valid JSON: invalid type: map, expected u32 at line 2 column 2
```
Only the envelope emitted by `aura graph build < op-script > envelope.json` is
accepted. Nothing in `--help`, the glossary `blueprint` entry, or the error
names this `graph build` step, and the three on-ramp verbs disagree on which
shapes they accept. (Once bridged, the content id is shape-invariant —
op-script and envelope both hash to `573262…`, so `register` and a ref computed
from the op-script agree; this part is correct.)
Repro: `aura graph register c0107_1_smacross_open.json`.
→ plan: shape-discriminate the op-list in `register`/`--params` (as
`--content-id` already does), or make the error name the `graph build` bridge.

### F6 [friction] The `content:` prefix trap persists and the on-ramp emits the wrong-to-paste form
`graph|process|campaign register` and `introspect --content-id` all print
`content:<hash>`, but every `ref` wants the **bare** hash. A campaign whose
`process.ref` carries the copied `content:5a88…` form **passes intrinsic
validation** and then fails referential:
```
aura: campaign references do not resolve:
  process content:5a88…6ea not found in the project store
```
The error now echoes the offending prefixed ref (an improvement over 0106 F3)
but never says "strip the `content:` prefix", and `register` still prints
exactly the form that breaks. This directly taxes the on-ramp the cycle set out
to smooth.
Repro: `aura campaign validate c0107_4_campaign_prefixref.json`.
→ plan: one bare display form across verbs, or accept/strip the `content:`
prefix in refs (carried from 0106 F3, still open: #194).

### F7 [friction] `campaign validate` passes but `campaign run` aborts when an open param is uncovered
Referential validation reports `all references resolve, axes are in the param
space` (exit 0), but it only checks that each axis **name** is in `param_space`
— not that **every** open param is covered by an axis. The open blueprint has
three open params (`fast.length`, `slow.length`, `bias.scale`); a campaign
sweeping only the two lengths **validates**, then aborts at member-bind:
```
aura: a member failed to bind: open param "graph.bias.scale" of strategy 597d71… is bound by no campaign axis
```
The executor's "every open knob required, no default" rule (documented for
`sweep`/`walkforward` in the glossary) is not mirrored in the validator, so
"valid" ≠ "runnable". (Secondary: the bind error uses the wrapped
`graph.bias.scale` path while `--params` and the axis use the raw `bias.scale`
— two namespaces for one knob.)
Repro: `aura campaign validate c0107_2_campaign.json` (passes) then
`aura campaign run c0107_2_campaign.json` (aborts).
→ plan: extend the referential tier to require full open-param coverage,
surfacing the missing knob at validate time; reconcile the raw-vs-wrapped param
path in the bind message.

### F8 [friction] The walk-forward window-fit refusal leaks a raw Rust `Debug` struct
A walk_forward whose `in_sample_ms` exceeds the campaign window refuses
correctly, but in internal `{:?}` form, unlike its clean sibling refusals:
```
aura: process stage 1: walk_forward windows do not fit the campaign window:
  SpanTooShort { span: (Timestamp(1725148800000), Timestamp(1727740800000)), need: 315964799999 }
```
The `SpanTooShort {…}` struct, the `Timestamp()` wrapper, and the
unit-unlabeled `need` (ms) break the "Debug-free house seam" the cycle
otherwise maintains, and a user cannot easily see that `need` is
`in_sample_ms + out_of_sample_ms` in ms against the window span.
Repro: `aura campaign run c0107_4_campaign_wf_toobig.json`.
→ debug (RED-pin the prose form).

### F9 [doc-gap] The rankable / gate metric roster is enumerable by no verb, nor by the refusal
Block hints say `metric name (see metric_vocabulary)`, but no verb lists it
(`--vocabulary` is the block list; `--block metric_vocabulary` → `unknown
block`). `process validate` accepts **any** string (`profit_factor` passes
intrinsic), and the run-time refusal names no roster:
```
aura: process stage 0: metric "profit_factor" is not rankable (winner selection needs one of the registry's rankable metrics)
```
`profit_factor` is itself emitted in every member's metrics block, so the
exists-but-not-rankable split is invisible. A campaign author must guess the
sweep/gate/wf metric names (0106 F6, still open: #197).
→ tighten docs: expose the rankable + gate metric rosters via `process
introspect` (or enumerate them in the refusal).

### F10 [spec_gap] Register-then-run stores the campaign before the gate — unrunnable docs accumulate
Every `campaign run <file>` issued left a `runs/campaigns/<id>.json` (9
stored), including campaigns that then failed the referential gate
(`prefixref`) or preflight (`mc`/`badmetric`/`plateauwf`/`deflateplateau`/
`wf_toobig`). The store thus accumulates never-runnable and
referentially-broken documents, with no cleanup. The ledger is silent on
whether the file sugar should gate before storing, and whether the
content-addressed store should hold referentially-broken docs.
→ ratify (accept intrinsic-keyed store as-is) or tighten (gate before storing).

## Verdict

**Yes — the campaign+process pair now delivers the full loop the 0106 verdict
said was one wire short.** F7 (0106) is closed end to end: a campaign document
+ its referenced process document drive `aura campaign run` through
`sweep → gate → walk_forward` over the real GER40 archive, and the run's
realization is recorded as a thin, family-linking `CampaignRunRecord` that is
content-addressed, run-counted, and **bit-identical on re-run** — persisted
intent finally has a consumer, and every id, axis, gate, and selection earns
its reason to exist. The authoring on-ramp that 0106 flagged is substantially
repaired too: `graph register` + `introspect --params` (file and store-id) +
`--content-id` file mode give a blueprint ref a resolvable path (F5-0106), and
`--unwired {}` makes the whole envelope, `seed` included, derivable (F1-0106).
The honest-refusal surface is a genuine strength — v1 boundary, empty gate
(exit 0 + truncation), plateau/deflate composition, stale wf vocabulary, and
every bad target are refused loudly, precisely, and with the correct
disposition.

What remains is friction concentrated at the *first-author* edge, not the
loop: the on-ramp verbs disagree on op-script vs envelope shape and hide the
`graph build` bridge (F5); the `content:` prefix that every
`register`/`--content-id` prints is the exact form that silently breaks a ref
(F6); `campaign validate` blesses a campaign the executor then rejects for an
uncovered open param (F7); one preflight fault leaks a raw `Debug` struct
(F8); and the metric roster a process must name is still discoverable nowhere
(F9). None block the loop; all tax the newcomer. One spec silence to settle:
whether register-then-run should store before or after the gate (F10).

Counts: working 4 (F1–F4), friction 4 (F5–F8), doc-gap 1 (F9), spec-gap 1
(F10), bugs 0.

## Orthogonal concern (out of 0107 scope)

`aura new` (cycle 0103) scaffolds a **nested `.git` repo** inside the project
directory — a gitlink/submodule hazard when the scaffold is embedded in the
tracked `fieldtests/` corpus. `demo_project/.git` (and the large
`demo_project/target/`) were removed so the corpus commits cleanly. Worth a
scaffold-side note (`--no-git` or skip when nested under a repo).