chore(docs): remove ephemeral milestone fieldtest spec at cycle close
The research-artifacts milestone fieldtest closed GREEN and the milestone is formally closed; per the ephemeral-artifact convention its spec is git-rm'd at cycle close. The verdict survives durably in the tracked corpus (fieldtests/milestone-research-artifacts/FINDINGS.md), the closed milestone, and the ledger.
This commit is contained in:
@@ -1,110 +0,0 @@
|
||||
# Fieldtest — milestone: Research artifacts (process & campaign documents) — 2026-07-04
|
||||
|
||||
**Status:** Draft — awaiting orchestrator triage
|
||||
**Author:** fieldtester (dispatched by fieldtest skill, milestone-close gate)
|
||||
|
||||
## Scope
|
||||
The milestone "Research artifacts — process & campaign documents" (#189/#192–#207,
|
||||
cycles 0106–0109) ships roles 5/6b their artifacts: the **process document** (a
|
||||
closed std-stage validation methodology) and the **campaign document** (persisted
|
||||
experiment intent — instruments × windows × strategy refs × param axes × process
|
||||
ref × data-level presentation), authored/validated/executed headless, content-
|
||||
addressed beside blueprints. Over the four cycles the surface grew from v1
|
||||
document vocabularies + two-tier validation (0106), through the v1 executor +
|
||||
blueprint on-ramp (0107), the annotator stages `std::monte_carlo`/`std::generalize`
|
||||
(0108), to `persist_taps` trace persistence + the third executable validation tier
|
||||
(0109). This milestone fieldtest runs ONE continuous downstream scenario over the
|
||||
whole span: introspection-only authoring → 3-tier validate → register → run by
|
||||
content id → judge the realization → re-run + byte-compare → boundary probes.
|
||||
|
||||
## Examples
|
||||
### fieldtests/milestone-research-artifacts/mra_1_strategy_smacross.json — strategy op-script (role-6a input)
|
||||
- Open SMA-cross (source→fast/slow SMA→Sub→Bias→expose bias); built via
|
||||
`graph build` to `mra_1_strategy_built.json`, param space
|
||||
`fast.length/slow.length/bias.scale`, registered (content id `597d71…`).
|
||||
- Fits: the campaign references it by content id; exercises `graph introspect
|
||||
--vocabulary/--node/--unwired/--params/--content-id` + `graph register`.
|
||||
- Outcome: built, registered, content id stable across sessions. (Op grammar
|
||||
itself not CLI-discoverable — F8.)
|
||||
|
||||
### fieldtests/milestone-research-artifacts/mra_2_process_full_v2.json — full v2 process document
|
||||
- `std::sweep`(deflate) → `std::gate`(expectancy_r>0) → `std::walk_forward`(rolling)
|
||||
→ `std::monte_carlo` → `std::generalize`; authored from `process introspect
|
||||
--unwired/--vocabulary/--block/--metrics` alone; validated, registered (`cd9127…`).
|
||||
- Fits: the full anti-false-discovery pipeline the milestone promises.
|
||||
- Outcome: `valid (intrinsic): 5 pipeline blocks`; registered.
|
||||
|
||||
### fieldtests/milestone-research-artifacts/mra_3_campaign_full_v2.json — two-instrument campaign
|
||||
- GER40 + FRA40, 2024-09, axes 2×2×1, process/strategy by content id,
|
||||
`persist_taps [equity, r_equity]`, `emit [family_table, selection_report]`;
|
||||
authored from `campaign introspect --unwired/--block` alone; validated all three
|
||||
tiers, registered (`42edeb…`), run by content id.
|
||||
- Fits: the whole milestone promise (intent → reproducible recorded evidence).
|
||||
- Outcome: `valid (executable)`; run exit 0; two nominee trace cells on disk;
|
||||
chart read-back OK; re-run byte-identical modulo counters.
|
||||
|
||||
## Findings
|
||||
See the recommendation table for classes. Full verbatim evidence is in the run
|
||||
capture fixtures (`mra_run1_stdout.txt`, `mra_run2_stdout.txt`,
|
||||
`mra_realization_campaign_runs.jsonl`) and the probe fixtures.
|
||||
|
||||
### [working] F1 — full v2 pipeline closes end to end over two instruments
|
||||
`campaign run <content-id>` executes sweep(deflate)→gate→walk_forward→monte_carlo
|
||||
per cell + campaign-scope generalize; every stage records its evidence
|
||||
(deflated_score/overfit_probability, survivor_ordinals, pooled_oos bootstrap with
|
||||
prob_le_zero, generalization worst_case/sign_agreement/winners). Headless, no compile.
|
||||
|
||||
### [working] F2 — bare-`{}` envelope + block/metric introspection bootstrap authoring
|
||||
`introspect --unwired` on a bare `{}` enumerates every required slot (campaign
|
||||
names the closed tap vocabulary inline); `--vocabulary`/`--block`/`--metrics`
|
||||
(with `rankable|gate|generalize|annotation` tags) suffice to author both documents.
|
||||
|
||||
### [working] F3 — three tiers, legible degradation
|
||||
Inside a project: intrinsic + referential + `valid (executable)`. Outside:
|
||||
intrinsic only + `referential checks skipped (no Aura.toml found up from …)`.
|
||||
|
||||
### [working] F4 — C1 reproducibility + content-id stability
|
||||
Two runs byte-identical modulo the campaign-run + family-run counters; metric
|
||||
number stream hashes identically; op-script re-hashes to the same content id.
|
||||
|
||||
### [working] F5 — auditable / navigable / chartable realization
|
||||
`campaign runs` list + byte-identical bare-line dump; `trace_name` → on-disk
|
||||
`{campaign8}-{run}/{strategy8}-{instrument}-w{n}/{tap}.json` family; `aura chart`
|
||||
renders a nominee cell.
|
||||
|
||||
### [working] F6 — refuse-don't-guess + no-compile diffability
|
||||
Bogus process ref → referential refusal (exit 1); unknown tap → intrinsic
|
||||
index-addressed refusal naming the closed vocabulary; wrong `kind` → exact-key
|
||||
prose. A one-value axis edit → new content id, still executable, no compile.
|
||||
|
||||
### [doc-gap] F7 — the documented `Aura.toml` data-archive root is not honored
|
||||
Glossary + `docs/project-layout.md` say the archive root is an `Aura.toml` path;
|
||||
`aura new` scaffolds none, a `[paths] archive=` key has no effect, and every
|
||||
real-data path resolves a built-in `/mnt/tickdata/Pepperstone` (named in the
|
||||
diagnostics). Cross-host reproducibility of recorded evidence is unreachable from
|
||||
the public surface. Peripheral to the document milestone; does not block it on
|
||||
this host. Repro: add `archive=` under `[paths]`, run any `--real`/campaign path.
|
||||
|
||||
### [doc-gap] F8 — role-6a op-script grammar is not CLI-discoverable
|
||||
`graph introspect` surfaces node vocabulary + ports but not the op-list verbs
|
||||
(`source/add/feed/connect/expose`) or their JSON shape; `graph build --help` and
|
||||
`project-layout.md` do not describe them. Authoring the op-script required the
|
||||
fieldtest-corpus shape. Orthogonal (role 6a, shipped cycle 0088); does not block
|
||||
roles 5/6b (strategy ids arrive via C25's id interface).
|
||||
|
||||
## Recommendation summary
|
||||
| Finding | class | action |
|
||||
|---|---|---|
|
||||
| F1 pipeline closes end to end | working | carry-on |
|
||||
| F2 introspection bootstraps authoring | working | carry-on |
|
||||
| F3 three-tier validate | working | carry-on |
|
||||
| F4 C1 + content-id stability | working | carry-on |
|
||||
| F5 auditable/navigable/chartable | working | carry-on |
|
||||
| F6 refusals + diffability | working | carry-on |
|
||||
| F7 Aura.toml archive root not honored | doc-gap | tighten glossary + project-layout (planner) |
|
||||
| F8 op-script grammar undiscoverable | doc-gap | document op grammar (planner, orthogonal) |
|
||||
|
||||
**Milestone Verdict: GREEN** — nameable, diffable, auditable research intent
|
||||
executes reproducibly into recorded, chartable evidence, end to end. The two
|
||||
doc-gaps are peripheral (archive configurability) and orthogonal (role-6a
|
||||
discoverability); no bug, friction, or spec_gap in the milestone's own surface.
|
||||
Reference in New Issue
Block a user