Brummel
6f2cf443c4
MILESTONE VERDICT: GREEN — the whole span (0106-0109) holds end to
end as one continuous consumer scenario: authored from a bare {} via
introspection alone, every ref resolved from the public surface, three
validation tiers, run by content id over two real instruments through
the full v2 pipeline, realization recorded (selection/survivors/
bootstrap/generalizations/trace_name), traces on disk and charted,
read back via campaign runs, byte-identical re-runs (C1), refusals
precise. 0 bugs, 0 frictions, 0 spec gaps; 2 doc-gaps.
F7 fixed in this commit (the docs claimed an archive-root path without
naming the key; a guessed [paths] archive silently no-ops): project-
layout and the glossary Aura.toml entry now name [paths] data, and the
aura new template advertises the commented-out key. F8 (op-script
grammar not CLI-discoverable) filed as #208, sibling of the #197 docs
half — one docwriter pass once desired.
The milestone-close gate evidence is this corpus; the formal milestone
close and the push remain user-reserved.
refs #189
97 lines
5.3 KiB
Markdown
97 lines
5.3 KiB
Markdown
# Milestone fieldtest — Research artifacts — process & campaign documents (cycles 0106–0109) — 2026-07-04
|
||
|
||
**Role:** downstream consumer (methodology + campaign designer), public
|
||
interface only (ledger C18/C22/C25, glossary, CLI --help + introspection, the
|
||
fieldtests/ corpora as shape hints). No `crates/` source read. Build:
|
||
`cargo build --release -p aura-cli` (HEAD `01925e06`); project dylib built in
|
||
the scaffolded `mrp_lab/`. Real data: local Pepperstone archive (GER40 +
|
||
FRA40, 2024-09).
|
||
|
||
**Milestone Verdict: GREEN** — nameable, diffable, auditable research intent
|
||
executes reproducibly into recorded, chartable evidence, end to end. Both
|
||
findings are doc-gaps, neither a bug nor a broken contract; no
|
||
bug/friction/spec_gap in the milestone's own surface.
|
||
|
||
## The one continuous scenario
|
||
|
||
- `mra_1_strategy_smacross.json` (+`_built.json`) — op-script → `graph build`
|
||
→ register, content id `597d71…` (stable across sessions).
|
||
- `mra_2_process_full_v2.json` — sweep(deflate) → gate → walk_forward →
|
||
monte_carlo → generalize, authored from introspection alone; registered
|
||
`cd9127…`.
|
||
- `mra_3_campaign_full_v2.json` — GER40+FRA40, persist_taps [equity,
|
||
r_equity], both emit kinds; validated through all three tiers, registered
|
||
`42edeb…`, run by content id.
|
||
|
||
## Findings
|
||
|
||
### F1 [working] Full v2 pipeline closes end to end over two instruments
|
||
`campaign run <content-id>` (exit 0) runs sweep(deflate)→gate→walk_forward→
|
||
monte_carlo per cell + campaign-scope generalize. Records: sweep `selection`
|
||
(deflated_score/overfit_probability/n_trials), gate `survivor_ordinals:
|
||
[0,1,2]`, wf `family_id`, mc `pooled_oos` (GER40 `n_trades:1843,
|
||
prob_le_zero:0.274`; FRA40 `1272, 0.412`), `generalizations`
|
||
(`worst_case:0.04367…, sign_agreement:2`, per-instrument breakdown + `winners`
|
||
params). → carry-on.
|
||
|
||
### F2 [working] Bare-{} envelope + block/metric introspection bootstrap authoring
|
||
`process|campaign introspect --unwired mra_bare.json` enumerates every
|
||
required slot (campaign names the closed tap vocabulary inline:
|
||
`persist_taps (equity | exposure | r_equity | net_r_equity)`);
|
||
`--vocabulary`/`--block`/`--metrics` (tags `rankable|gate|generalize|
|
||
annotation`) suffice to author both docs. The 0108-F8 R-expectancy tagging is
|
||
correct. → carry-on.
|
||
|
||
### F3 [working] Three tiers, legible degradation
|
||
Inside a project: intrinsic + referential + `valid (executable): pipeline
|
||
shape and static guards pass`. Outside: intrinsic only + `referential checks
|
||
skipped (no Aura.toml found up from /tmp)` (exit 0). → carry-on.
|
||
|
||
### F4 [working] C1 reproducibility + content-id stability
|
||
Two runs byte-identical after normalizing only the campaign-run counter
|
||
(`42edebd2-<N>`) and the family-store per-name run index (`…-s0-<K>`, both
|
||
mint-fresh per ledger); the R-metric/bootstrap number stream hashes
|
||
identically (`cd9ad25…`). Op-script re-hashes to `597d71…`. → carry-on.
|
||
|
||
### F5 [working] Auditable / navigable / chartable realization
|
||
`campaign runs` → `run 0 — 2 cell(s), std::sweep -> std::gate ->
|
||
std::walk_forward -> std::monte_carlo`; `campaign runs <id>` dumps the BARE
|
||
store line, byte-identical to `campaign_runs.jsonl` (not the
|
||
`{"campaign_run":…}` wrapper). `trace_name:"42edebd2-0"` → on-disk
|
||
`runs/traces/42edebd2-0/597d719b-{GER40,FRA40}-w0/{equity,r_equity}.json`
|
||
(exactly the ledger layout); `aura chart 42edebd2-0/597d719b-GER40-w0`
|
||
renders a self-contained page. → carry-on.
|
||
|
||
### F6 [working] Refuse-don't-guess + no-compile diffability
|
||
Bogus process ref → `process 0000… not found in the project store`
|
||
(referential, exit 1). Unknown tap → `presentation.persist_taps[1]: unknown
|
||
tap "pnl" (taps: equity | exposure | r_equity | net_r_equity)` (exit 1).
|
||
Process doc validated as campaign → `the "kind" key must be "campaign"` (the
|
||
#193 exact-key prose, exit 1). One-value axis edit (`slow.length` 16→32) →
|
||
new content id `ce0de8…`, still `valid (executable)`, no compile/run.
|
||
→ carry-on.
|
||
|
||
### F7 [doc-gap] The documented Aura.toml data-archive root is not honored (as documented)
|
||
Glossary (`Aura.toml`: "paths only: the data archive root and the runs dir")
|
||
+ `docs/project-layout.md` claim the archive root is an `Aura.toml` path, but
|
||
neither names the KEY — and a guessed `[paths] archive = …` has no effect
|
||
(every real-data path then resolves the built-in default). The real key is
|
||
`[paths] data` (honored end-to-end incl. the campaign path). A consumer whose
|
||
archive lives elsewhere cannot configure it from the public surface without
|
||
the key name. → tighten docs (RESOLVED same-session: project-layout +
|
||
glossary now name `[paths] data`, and the `aura new` Aura.toml template
|
||
advertises the commented-out key).
|
||
|
||
### F8 [doc-gap] Role-6a op-script grammar not CLI-discoverable
|
||
`graph introspect` surfaces node vocabulary (`--vocabulary`) + ports
|
||
(`--node`) but NOT the op-list verbs (`source/add/feed/connect/expose`) or
|
||
their JSON shape; `graph build --help` and `project-layout.md` omit them too.
|
||
Authored the op-script only via the fieldtest-corpus shape hint. Orthogonal
|
||
to this milestone (role 6a shipped cycle 0088); does not block roles 5/6b
|
||
(strategy content ids arrive via C25's id interface), but breaks "author from
|
||
introspection alone" for the upstream input. → document the op grammar
|
||
(filed; docwriter territory with the #197 docs half).
|
||
|
||
Counts: working 6 (F1–F6), doc-gap 2 (F7 resolved same-session, F8 filed),
|
||
friction 0, spec_gap 0, bugs 0.
|