# Fieldtest — cycle 0108 — annotator stages execute (#200) — 2026-07-04

**Role:** downstream consumer (methodology / campaign designer), public
interface only (ledger C18 0107/0108 realization paragraphs + C1, glossary,
CLI `--help` + introspection verbs, the cycle-0107 fieldtest corpus as shape
hints). No `crates/` source read. Binary built from HEAD
(`cargo build --workspace`, `target/debug/aura`, commit
`ae2fac3212f931bf5a3b277ba36fbe4010bf7b9d`). Real data: scaffolded a fresh
`demo_project/` (`aura new` — nested `.git` correctly skipped, 0107 orthogonal
fix confirmed), `cargo build` for the dylib, ran the executor over the local
Pepperstone archive (`/mnt/tickdata/Pepperstone`, GER40 + FRA40, 2024-09).

## Fixtures (this directory)

- `c0108_strategy_smacross.json` / `_built.json` — the open (sweepable) SMA-cross
  strategy op-script + its `graph build` envelope (content id `597d71…`, 3 open
  params `fast.length` / `slow.length` / `bias.scale`)
- `c0108_process_full_pooled.json` — sweep(deflate) → gate → walk_forward →
  monte_carlo (the PooledOos path)
- `c0108_process_mc_persurvivor.json` — sweep → gate → monte_carlo (no wf → the
  PerSurvivor path)
- `c0108_process_full_generalize.json` — sweep → gate → wf → mc → generalize
  (the full v2 suffix)
- `c0108_campaign_full_pooled.json` / `_mc_persurvivor.json` — GER40 2024-09
- `c0108_campaign_2instr_generalize.json` — GER40 + FRA40, the end-to-end
  generalize
- `c0108_probe_{gate_after_mc,wf_after_mc,dup_mc,gen_not_last,zero_resamples,`
  `zero_blocklen,nonr_generalize}.json` — boundary probes + their
  `c0108_probe_campaign_*.json` runners; `c0108_probe_campaign_singleinstr_`
  `generalize.json`, `c0108_probe_generalize_maxrdrawdown.json`
- `c0108_realization_campaign_runs.jsonl` — the recorded realization (copied out
  of the project-gitignored `runs/` store): 3 pooled runs, 1 per-survivor, 1
  two-instrument-with-generalizations, 3 sweep→generalize metric probes

## Findings

### F1 [working] The full annotator suffix closes — pooled_oos over the wf OOS series
`aura campaign run c0108_campaign_full_pooled.json` executes
`sweep → gate → walk_forward → monte_carlo` over GER40 2024-09 (exit 0). The
`std::monte_carlo` stage records exactly the ledger's PooledOos semantics:
`"bootstrap":{"pooled_oos":{"e_r":{"mean":0.02572811864069487,…},
"prob_le_zero":0.274,"n_trades":1843,"block_len":5,"n_resamples":1000}}`.
`n_trades:1843` is exactly the two wf OOS windows pooled in roll order
(948 + 895 — the two walk_forward OOS member counts). One `r_bootstrap` over the
whole pooled series, seeded from the doc `seed=42`. Intent → execution →
recorded, honest inference. → carry-on.

### F2 [working] No-wf campaign records one bootstrap per survivor; the honesty seam bites
`aura campaign run c0108_campaign_mc_persurvivor.json` (sweep → gate → mc, no wf)
records `per_survivor` as `[[ordinal, bootstrap], …]` — one entry per gate
survivor `[0,1,2]`, each an `r_bootstrap` over that member's full in-memory R
series (`n_trades` 3903 / 2581 / 3811, matching the sweep members). Survivor 1
(fast=2/slow=16) passed the `expectancy_r > 0` gate at a razor-thin
`expectancy_r = 0.0019`, yet its bootstrap shows `prob_le_zero: 0.525` — the mc
annotator correctly flags a gate-passing member as coin-flip noise. This is the
inference layer earning its keep. → carry-on.

### F3 [working] Determinism is byte-identical across re-runs (modulo the run counter)
Three runs of `c0108_campaign_full_pooled.json` (run 0/1/2) produce
byte-identical `campaign_run` lines after normalizing only the per-campaign run
counter (the `"run"` field + the family-id trailing run index). Every bootstrap
percentile, `prob_le_zero`, gate survivor set, and selection reproduces exactly.
C1's "pure function of doc + stores + data" holds through the annotator stages.
→ carry-on.

### F4 [working] Campaign-scope generalize closes end-to-end over two instruments
`aura campaign run c0108_campaign_2instr_generalize.json`
(sweep → gate → wf → mc → generalize, GER40 + FRA40, exit 0) runs both cells then
records a campaign-scope `generalizations` entry keyed (strategy 0 × window 0):
`"generalization":{"selection_metric":"expectancy_r","n_instruments":2,
"worst_case":0.04367535583327283,"sign_agreement":2,
"per_instrument":[["GER40",0.0437…],["FRA40",0.0484…]]}`.
The nominees are each cell's last wf-OOS winner (fast=4/slow=8), not the
sweep winner (fast=2/slow=8) — matching the ledger. `worst_case` is the weaker
instrument (GER40); divergent winners would surface via `winners[]` params,
never averaged. The serde-default sparse widening is visible in the store: the
`generalizations` key appears only on this two-instrument line; the four
single-cell lines omit it entirely. → carry-on.

### F5 [working] The annotator tier boundary is honest and precise on both sides
Every boundary probe refuses cleanly with per-pair prose and the right exit code:
- gate after mc — intrinsic (`process validate`) — `stage 2: a gate cannot follow a terminal stage (monte_carlo/generalize)` — exit 1
- wf after mc — executor (`campaign run`; validate passes) — `process pipeline is not executable: stage 2: std::walk_forward cannot follow std::monte_carlo` — exit 1
- dup monte_carlo — executor — `stage 2: std::monte_carlo may appear at most once` — exit 1
- generalize not last — executor — `stage 2: std::monte_carlo cannot follow std::generalize` — exit 1
- zero resamples — executor — `process stage 1: monte_carlo resamples must be > 0` — exit 1
- zero block_len — executor — `process stage 1: monte_carlo block_len must be > 0` — exit 1
- non-R generalize metric — executor — `std::generalize metric "total_pips" is not an R metric (gross/net R families generalize; pip metrics do not)` — exit 1
- single-instrument generalize — executor — `campaign has 1 instrument(s); std::generalize needs at least 2` — exit 1

The intrinsic tier catches only gate-after-terminal; the executor preflight
catches the shape/param/metric faults — exactly the "test-pinned on both sides"
boundary the ledger describes. → carry-on.

### F6 [friction] `campaign validate` blesses process shapes the executor then refuses
`wf after mc` is a pure process-shape fault — no data or instrument count needed
to detect it — yet `aura campaign validate c0108_probe_campaign_wf_after_mc.json`
returns exit 0 (`valid (intrinsic) … valid (referential): all references
resolve`), and only `campaign run` refuses it. Same for dup-mc and
generalize-not-last. The author sees a green validate, authors on, and is
stopped only at run time. The three-tier model (intrinsic / referential /
executor-preflight) is invisible from the validate surface. (Kin to 0107 F7:
"valid" ≠ "runnable".) Repro: `campaign validate` then `campaign run` on any of
the three shape probes.
→ plan: run the executor shape/param preflight at `campaign validate` time (it is
a pure process property), or name the third tier in the validate output.

### F7 [doc-gap] The pooled-vs-per-survivor duality is invisible to the author surface
`aura process introspect --block std::monte_carlo` says only
`R-bootstrap over realised R (terminal annotator)` + `resamples` / `block_len`.
Nothing in `--block`, `--vocabulary`, `--metrics`, or the glossary tells the
author that the stage's output shape and semantics flip on whether a
walk_forward precedes it (one pooled-OOS bootstrap vs one bootstrap per
survivor over in-sample series). The distinction is documented only in the
design ledger (C18 0108) — not on the surface a headless campaign author reads.
A user cannot predict from introspection what `std::monte_carlo` will produce.
→ tighten docs: fold the position-dependent semantics into the `--block`
description (and/or a glossary `stage bootstrap` entry with the two output
shapes).

### F8 [doc-gap] The generalize R-metric set is unpredictable, and the refusal prose misleads
`std::generalize` requires an "R metric", but no introspection surface says which
of the 17 metrics qualify. `--metrics` tags are `rankable | gate | annotation` —
none marks R / generalizable. Probing the boundary:
- accepted: `expectancy_r`, `net_expectancy_r`, `sqn`, `sqn_normalized`
- refused: `total_pips`, `max_drawdown` (both rankable!), `win_rate`, and
  `max_r_drawdown` — the last is R-denominated (its name literally contains
  `_r`, it is a drawdown in R) yet is refused with
  `… is not an R metric (gross/net R families generalize; pip metrics do not)`.
  Calling `max_r_drawdown` a "pip metric" is wrong; the real rule is "the
  rankable gross/net-R expectancy family", which no surface states.
So the author must guess the metric, and the one honest hint (the refusal) both
under-describes the accepted set and mis-describes why an R metric was refused.
Repro: `aura campaign run c0108_probe_campaign_nonr_generalize.json` and the
`c0108_probe_generalize_maxrdrawdown.json` process.
→ tighten: add a `generalize`/R applicability tag to `--metrics`; correct the
refusal prose for R-denominated-but-unranked metrics.

### F9 [doc-gap] The mc slot hint says "non-negative" but the executor refuses 0
`--block std::monte_carlo` labels both `resamples` and `block_len`
`required, non-negative integer (u32)`. Non-negative includes 0, but 0 is
refused at run (`… must be > 0`). The hint describes the u32 type, not the
valid range, so it silently contradicts the guard the executor enforces.
Repro: `aura campaign run c0108_probe_campaign_zero_resamples.json`.
→ tighten the slot hint to "positive integer" (or "≥ 1").

### F10 [doc-gap] The recorded realization's output schema is undocumented
The realization a consumer reads — `prob_le_zero`, `per_survivor`, `pooled_oos`,
`e_r` percentiles, `worst_case`, `sign_agreement`, `n_instruments` — appears in
no public prose (grep of glossary + INDEX finds none of the wire-format field
names; only the internal enum names `PooledOos` / `PerSurvivor` are in the
ledger). A user seeing `prob_le_zero: 0.274` has no public definition (it is
P[bootstrap E[R] ≤ 0]). Also undocumented: the stdout emit wraps each record as
`{"campaign_run": …}` while the persisted `campaign_runs.jsonl` line is the
bare record — a consumer parsing the store must know to drop the wrapper.
→ tighten docs: define the campaign-run realization output fields (a glossary
entry or a `campaign introspect --realization` schema dump).

### F11 [friction] The annotator realizations are write-only — no CLI read-back
`aura runs` exposes only `families` / `family`; there is no verb to read back a
recorded `campaign_run`. The bootstrap distributions and generalization scores
the annotator stages compute are visible only in the run's own stdout; once the
terminal scrolls, the only access is hand-parsing `runs/campaign_runs.jsonl`
(which is also project-gitignored). Every cell × stage × per-survivor bootstrap
is concentrated into one JSONL line, which grows unboundedly with campaign size
and is unreadable raw. → plan: an `aura campaign runs` / `run <id>` read-back
verb, at least for the realized bootstrap / generalization records.

## Verdict

Yes — the annotator stages deliver honest, recorded, reproducible inference on
top of the 0107 loop. The v2 suffix `sweep [gate]* [wf]? [mc]? [generalize]?`
executes end to end over real data: `std::monte_carlo` bootstraps the stage's
incoming R-evidence with the exact position-dependent semantics the ledger
promises (PooledOos over the wf OOS series — `n_trades` provably the pooled OOS
count; PerSurvivor over each gate survivor's in-sample series), and
`std::generalize` runs at campaign scope over per-cell nominees, scoring the
worst-case R floor and exposing divergent winners rather than averaging them.
The inference is honest (a gate-passing survivor is flagged
`prob_le_zero=0.525`; a single-instrument generalize refuses rather than
fabricating a floor), recorded (the sparse serde widening lands cleanly — the
`generalizations` key appears only where earned), and reproducible
(byte-identical across three runs modulo the run counter, C1 intact). The
annotator tier boundary is a genuine strength: eight distinct
shape/param/metric/instrument faults each refuse loudly, precisely, and at the
right tier with exit 1.

Nothing here is a bug. What remains is discoverability, concentrated at the
author's first contact with the two new stages: the mc pooled-vs-per-survivor
duality (F7), the generalize R-metric set + misleading refusal prose (F8), the
"non-negative" slot hint that contradicts the `>0` guard (F9), the entirely
undocumented realization output schema (F10), a validate tier that greenlights
executor-unrunnable shapes (F6), and the absence of any read-back for the
realizations the stages so carefully record (F11). The engine is honest; the
map to it is thin.

Counts: working 5 (F1–F5), friction 2 (F6, F11), doc-gap 4 (F7–F10),
spec_gap 0, bugs 0.
