Aura/docs/specs/fieldtest-milestone-research-artifacts.md

Fieldtest — milestone: Research artifacts (process & campaign documents) — 2026-07-04

Status: Draft — awaiting orchestrator triage Author: fieldtester (dispatched by fieldtest skill, milestone-close gate)

Scope

The milestone "Research artifacts — process & campaign documents" (#189/#192–#207, cycles 0106–0109) ships roles 5/6b their artifacts: the process document (a closed std-stage validation methodology) and the campaign document (persisted experiment intent — instruments × windows × strategy refs × param axes × process ref × data-level presentation), authored/validated/executed headless, content- addressed beside blueprints. Over the four cycles the surface grew from v1 document vocabularies + two-tier validation (0106), through the v1 executor + blueprint on-ramp (0107), the annotator stages std::monte_carlo/std::generalize (0108), to persist_taps trace persistence + the third executable validation tier (0109). This milestone fieldtest runs ONE continuous downstream scenario over the whole span: introspection-only authoring → 3-tier validate → register → run by content id → judge the realization → re-run + byte-compare → boundary probes.

Examples

fieldtests/milestone-research-artifacts/mra_1_strategy_smacross.json — strategy op-script (role-6a input)

• Open SMA-cross (source→fast/slow SMA→Sub→Bias→expose bias); built via graph build to mra_1_strategy_built.json, param space fast.length/slow.length/bias.scale, registered (content id 597d71…).
• Fits: the campaign references it by content id; exercises graph introspect --vocabulary/--node/--unwired/--params/--content-id + graph register.
• Outcome: built, registered, content id stable across sessions. (Op grammar itself not CLI-discoverable — F8.)

fieldtests/milestone-research-artifacts/mra_2_process_full_v2.json — full v2 process document

• std::sweep(deflate) → std::gate(expectancy_r>0) → std::walk_forward(rolling) → std::monte_carlo → std::generalize; authored from process introspect --unwired/--vocabulary/--block/--metrics alone; validated, registered (cd9127…).
• Fits: the full anti-false-discovery pipeline the milestone promises.
• Outcome: valid (intrinsic): 5 pipeline blocks; registered.

fieldtests/milestone-research-artifacts/mra_3_campaign_full_v2.json — two-instrument campaign

• GER40 + FRA40, 2024-09, axes 2×2×1, process/strategy by content id, persist_taps [equity, r_equity], emit [family_table, selection_report]; authored from campaign introspect --unwired/--block alone; validated all three tiers, registered (42edeb…), run by content id.
• Fits: the whole milestone promise (intent → reproducible recorded evidence).
• Outcome: valid (executable); run exit 0; two nominee trace cells on disk; chart read-back OK; re-run byte-identical modulo counters.

Findings

See the recommendation table for classes. Full verbatim evidence is in the run capture fixtures (mra_run1_stdout.txt, mra_run2_stdout.txt, mra_realization_campaign_runs.jsonl) and the probe fixtures.

[working] F1 — full v2 pipeline closes end to end over two instruments

campaign run <content-id> executes sweep(deflate)→gate→walk_forward→monte_carlo per cell + campaign-scope generalize; every stage records its evidence (deflated_score/overfit_probability, survivor_ordinals, pooled_oos bootstrap with prob_le_zero, generalization worst_case/sign_agreement/winners). Headless, no compile.

[working] F2 — bare-{} envelope + block/metric introspection bootstrap authoring

introspect --unwired on a bare {} enumerates every required slot (campaign names the closed tap vocabulary inline); --vocabulary/--block/--metrics (with rankable|gate|generalize|annotation tags) suffice to author both documents.

[working] F3 — three tiers, legible degradation

Inside a project: intrinsic + referential + valid (executable). Outside: intrinsic only + referential checks skipped (no Aura.toml found up from …).

[working] F4 — C1 reproducibility + content-id stability

Two runs byte-identical modulo the campaign-run + family-run counters; metric number stream hashes identically; op-script re-hashes to the same content id.

[working] F5 — auditable / navigable / chartable realization

campaign runs list + byte-identical bare-line dump; trace_name → on-disk {campaign8}-{run}/{strategy8}-{instrument}-w{n}/{tap}.json family; aura chart renders a nominee cell.

[working] F6 — refuse-don't-guess + no-compile diffability

Bogus process ref → referential refusal (exit 1); unknown tap → intrinsic index-addressed refusal naming the closed vocabulary; wrong kind → exact-key prose. A one-value axis edit → new content id, still executable, no compile.

[doc-gap] F7 — the documented Aura.toml data-archive root is not honored

Glossary + docs/project-layout.md say the archive root is an Aura.toml path; aura new scaffolds none, a [paths] archive= key has no effect, and every real-data path resolves a built-in /mnt/tickdata/Pepperstone (named in the diagnostics). Cross-host reproducibility of recorded evidence is unreachable from the public surface. Peripheral to the document milestone; does not block it on this host. Repro: add archive= under [paths], run any --real/campaign path.

[doc-gap] F8 — role-6a op-script grammar is not CLI-discoverable

graph introspect surfaces node vocabulary + ports but not the op-list verbs (source/add/feed/connect/expose) or their JSON shape; graph build --help and project-layout.md do not describe them. Authoring the op-script required the fieldtest-corpus shape. Orthogonal (role 6a, shipped cycle 0088); does not block roles 5/6b (strategy ids arrive via C25's id interface).

Recommendation summary

Finding	class	action
F1 pipeline closes end to end	working	carry-on
F2 introspection bootstraps authoring	working	carry-on
F3 three-tier validate	working	carry-on
F4 C1 + content-id stability	working	carry-on
F5 auditable/navigable/chartable	working	carry-on
F6 refusals + diffability	working	carry-on
F7 Aura.toml archive root not honored	doc-gap	tighten glossary + project-layout (planner)
F8 op-script grammar undiscoverable	doc-gap	document op grammar (planner, orthogonal)

Milestone Verdict: GREEN — nameable, diffable, auditable research intent executes reproducibly into recorded, chartable evidence, end to end. The two doc-gaps are peripheral (archive configurability) and orthogonal (role-6a discoverability); no bug, friction, or spec_gap in the milestone's own surface.