Brummel/Aura
1
0
Fork 0
Code Issues 32 Pull Requests Actions Packages Projects Releases Wiki Activity

audit: cycle 0108 tidy (drift resolved by ledger/glossary lift)

Browse Source 
Architect verdict: drift_found — the expected pre-audit doc lag; the
code holds (C1: doc-seeded bootstrap + BTreeMap-ordered generalize;
C14/C23 sparse widening pinned by the pre-0108-line parse test;
terminal-annotator dataflow confirmed — nothing flows out, no new
emit kind, no content-id movement). All items resolved here:

- C18: 0107's 'refuse loudly at preflight' points forward; new
  cycle-0108 realization paragraph records the v2 shape, the
  dual-input stage bootstrap, campaign-scope generalize, the tier
  boundary (test-pinned on both sides), the record widenings, and
  the stringly wf-detection debt.
- Glossary: campaign document -> v2 shape; bootstrap and Monte-Carlo
  entries disambiguate the three senses (construction vs statistical
  r_bootstrap vs seed axis — the std::monte_carlo stage IS the
  trade-sequence bootstrap, named for r_bootstrap's arguments);
  generalize gains the campaign-scope realization.

Regression: cargo test --workspace 1025/0; clippy -D warnings clean;
cargo doc 0 warnings. Spec + plan removed at close per convention.

closes #200
This commit is contained in:
Brummel
2026-07-04 01:33:00 +02:00
parent 7e4e9a8827
commit ae2fac3212
4 changed files with 34 additions and 3033 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/INDEX.md
+30 -2
View File
@@ -1399,8 +1399,9 @@ evidence), and the v1 executor shipped the next cycle (below).
`aura campaign run <file|content-id>` executes a campaign (a file is register-then-run
sugar; the content id is canonical): zero-fault referential gate, then the process
pipeline — v1 executable shape `std::sweep [std::gate]* [std::walk_forward]?`
(`std::monte_carlo`/`std::generalize` refuse loudly at preflight, as does every other
shape) — runs once per (strategy, instrument, window) cell in doc order. Execution
(`std::monte_carlo`/`std::generalize` refused loudly at preflight in this cycle; they
execute since cycle 0108, below) — runs once per (strategy, instrument, window) cell in
doc order. Execution
*semantics* live in the **`aura-campaign` library crate** (#198 home decision: reachable
beyond the CLI; NOT C21's project-side World): grid odometer over the campaign axes,
members through the engine `sweep` over a **`ListSpace`** (explicit point set beside
@@ -1428,6 +1429,33 @@ both `RollMode`s; the shipped `folds` slot mapped to nothing the machinery accep
a new `ZeroWalkForwardLength` intrinsic fault; wf-bearing process docs get new content ids
by design (the 0106 fieldtest corpus stays as the historical record). Known debt:
metric-roster triplication (#190), deflation-constant duplication (#199).
**Realization (cycle 0108, #200 — the annotator stages execute).** The v2 executable
shape is `std::sweep [std::gate]* [std::walk_forward]? [std::monte_carlo]?
[std::generalize]?` — an ordered optional annotator suffix, each at most once,
`std::generalize` strictly last; the executor preflight refuses what the intrinsic tier
deliberately admits (`[sweep, mc, walk_forward]` is intrinsically valid — the tier
boundary is test-pinned on both sides), plus three new static guards
(single-instrument generalize, non-R generalize metric via the registry's
`check_r_metric`, zero mc `resamples`/`block_len`). **Both annotators are terminal**
(unanimous #200 triage): nothing flows out of them; filtering stays the gate's monopoly.
`std::monte_carlo` bootstraps the stage's *incoming R-evidence* with one semantics,
input-shaped by position — after a walk_forward, ONE `r_bootstrap` over the wf family's
pooled per-window OOS `trade_rs` in roll order (`PooledOos`); after sweep/gates, one
`r_bootstrap` per surviving member's fresh in-memory series (`PerSurvivor`, ordinals into
the population family; a zero-trade member records the engine's defined all-zero
degenerate) — seeded from the campaign doc's `seed` (C1; `trade_rs` is `#[serde(skip)]`,
so annotators run in-executor or not at all). `std::generalize` executes at **campaign
scope**: after all cells, per (strategy, window) the per-cell *nominees* (last wf
window's OOS report, else the sweep winner; none on gate truncation) across instruments
feed the shipped `generalization()` when ≥ 2 exist — divergent per-instrument winners
are exposed via their params, never averaged away; a shortfall is recorded, not computed
around. Realization: `StageRealization.bootstrap` (`StageBootstrap::PerSurvivor |
PooledOos`) and `CampaignRunRecord.generalizations` (`CampaignGeneralization` keyed
strategy × window with `winners`/`missing`), both serde-default sparse — pre-0108
`campaign_runs.jsonl` lines parse unchanged (C14/C23). No document content id moved
(introspection doc-strings dropped "in v1" only). Noted debt: the mc arm detects the wf
family by the stringly `block == "std::walk_forward"` literal (a pre-existing lockstep
pattern with the realization's block strings).
**Guarantee.** Construction is a distinct phase, recursive at every level. Each
node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)`
sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic
docs/glossary.md
+4 -4
View File
@@ -33,7 +33,7 @@ The param-generic, input-role-generic graph-as-data produced by running a Rust b
### bootstrap
**Avoid:** —
The distinct, recursive construction phase that binds `(blueprint + param-set + data bindings + seed)` into a frozen instance — buffers sized, topology fixed. The explicit name for the "wiring / graph build" that C7/C12 reference — the construction/compilation sense, *not* the statistical resampling or yield-curve bootstrap (aura's resampling axis is `Monte-Carlo`).
The distinct, recursive construction phase that binds `(blueprint + param-set + data bindings + seed)` into a frozen instance — buffers sized, topology fixed. The explicit name for the "wiring / graph build" that C7/C12 reference — the construction/compilation sense. Disambiguation: the *statistical* moving-block bootstrap of a trade-R series (`r_bootstrap`, `RBootstrap`) is a different thing that keeps its statistics name — it appears as the deflation null, the `aura mc` R path, and since 0108 the `std::monte_carlo` stage's `stage bootstrap` annotation; context (construction vs annotation) disambiguates.
### bot
**Avoid:** —
@@ -45,7 +45,7 @@ A downstream consumer node, never part of the strategy: the signal-quality side
### campaign document
**Avoid:** experiment doc, campaign file
The role-6b research artifact (#188/#189): persisted experiment intent as closed-vocabulary canonical JSON — instruments × windows × strategy refs (by `content id` or `identity id`) × per-strategy param axes (each axis declares its `ScalarKind` once over bare values) × a process reference (content-id-only) × data-level presentation (taps to persist, tables to emit). Authored, validated, and executed headless (`aura campaign validate|introspect|register|run`), content-addressed beside blueprints in the registry store; carries P1 control constructs (bounded axes, gates, ladders) as intent, executed by `aura campaign run` (v1 pipeline shape `std::sweep [std::gate]* [std::walk_forward]?`, cycle 0107/#198) into a `campaign run` realization.
The role-6b research artifact (#188/#189): persisted experiment intent as closed-vocabulary canonical JSON — instruments × windows × strategy refs (by `content id` or `identity id`) × per-strategy param axes (each axis declares its `ScalarKind` once over bare values) × a process reference (content-id-only) × data-level presentation (taps to persist, tables to emit). Authored, validated, and executed headless (`aura campaign validate|introspect|register|run`), content-addressed beside blueprints in the registry store; carries P1 control constructs (bounded axes, gates, ladders) as intent, executed by `aura campaign run` (v2 pipeline shape `std::sweep [std::gate]* [std::walk_forward]? [std::monte_carlo]? [std::generalize]?` — the two annotators terminal, cycles 0107/0108, #198/#200) into a `campaign run` realization.
### campaign run
**Avoid:** campaign execution record, realized campaign
@@ -77,7 +77,7 @@ A composable downstream **C9 graph of cost nodes**, in **R**, that **approximate
### cross-instrument generalization
**Avoid:** cross-symbol pooling, pooled generalization
The validation read that grades how consistently one *brought* candidate holds across a set of instruments, scored on its weakest one — the across-instrument axis of the anti-false-discovery discipline. Realised by `aura generalize`; an aggregator (a recomputable family score), never a selector that picks a winner.
The validation read that grades how consistently one *brought* candidate holds across a set of instruments, scored on its weakest one — the across-instrument axis of the anti-false-discovery discipline. Realised by `aura generalize` and, since 0108 (#200), by the `std::generalize` process stage at campaign scope (per (strategy, window) over the cells' nominees across instruments, recorded in the `campaign run`'s `generalizations`); an aggregator (a recomputable family score), never a selector that picks a winner.
### cycle
**Avoid:** —
@@ -149,7 +149,7 @@ The reproducible metadata record of a run (node-commit + params + data-window +
### Monte-Carlo
**Avoid:** MC
An orchestration axis running N seeded realizations that perturb the input (the input stream, not the trade sequence); each realization is itself deterministic given its seed (Monte-Carlo = sweep over seeds).
An orchestration axis running N seeded realizations that perturb the input (the input stream, not the trade sequence); each realization is itself deterministic given its seed (Monte-Carlo = sweep over seeds). Distinct from the `std::monte_carlo` *process stage* (0108/#200): that block is the moving-block bootstrap of a per-trade R series (`r_bootstrap` — it perturbs the trade sequence, exactly what this axis does not), named for its fields (`resamples`, `block_len`) being that primitive's arguments; the seed-axis sense stays this entry's.
### neighbourhood score
**Avoid:** smoothed score
docs/plans/0108-annotator-stages.md
-2747
View File
File diff suppressed because it is too large Load Diff
docs/specs/0108-annotator-stages.md
-280
View File
@@ -1,280 +0,0 @@
# Annotator Stages — mc/generalize Execution — Design Spec (cycle 0108)
**Date:** 2026-07-03
**Status:** Draft — /boss autonomous run; sign-off gate = grounding-check PASS
**Authors:** orchestrator + Claude
**Reference issue:** #200 (this cycle's anchor). All five design forks are
decided on #200 (fork-triage comment 2026-07-03, five-lens swarm, all
convergent on cited ground); this spec encodes those decisions and adds none.
## Goal
`aura campaign run` executes the two annotator stages the v1 executor refused:
the v2 executable pipeline shape is
`std::sweep [std::gate]* [std::walk_forward]? [std::monte_carlo]? [std::generalize]?`.
`std::monte_carlo` bootstraps the stage's incoming R-evidence (per-survivor
after sweep/gates; the pooled per-window OOS series after a walk_forward) via
the shipped `r_bootstrap`; `std::generalize` runs at campaign scope over the
per-cell selection winners of each (strategy, window) across instruments via
the shipped `generalization`. Both are terminal annotators — nothing flows out
of them (#200 decision 3, unanimous) — and no document-schema bytes change:
every existing content id survives this cycle.
## Architecture
All execution changes live in **`aura-campaign`** (preflight v2 + the two
stage executions); **`aura-registry`** widens the realization records with
sparse serde-default fields (existing `campaign_runs.jsonl` lines stay
parseable — the C14/C23 widening convention) and derives serde on
`Generalization`; **`aura-engine`** derives serde on `RBootstrap`;
**`aura-cli`** gains prose for the new faults and loses the v1-boundary
refusal for mc/generalize (its pinned tests flip to the new semantics).
Determinism (C1): the bootstrap seeds from the doc's `seed` (the deflation
convention); all annotator inputs are the executor's fresh in-memory reports —
`trade_rs` is `#[serde(skip)]` and empty after any registry round-trip, so the
annotators run inside `execute` or not at all.
## Concrete code shapes
### The user-facing program
The 0107 fieldtest process grows the annotator suffix — no other document
changes (axes, campaign, data as in
`fieldtests/cycle-0107-campaign-executor/c0107_2_campaign_full.json`):
```json
{
"format_version": 1,
"kind": "process",
"name": "screen-wf-bootstrap-generalize",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn_normalized", "select": "argmax", "deflate": true },
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000,
"step_ms": 604800000, "mode": "rolling", "metric": "sqn_normalized", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
{ "block": "std::generalize", "metric": "net_expectancy_r" }
]
}
```
```
$ aura campaign run campaign.json # >= 2 instruments for generalize
…family/selection lines as today…
{"campaign_run":{"campaign":"…","process":"…","run":0,"seed":42,"cells":[…,
{"block":"std::walk_forward","family_id":"…"},
{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{"e_r":{…},"prob_le_zero":0.031,"n_trades":57,"block_len":5,"n_resamples":1000}}}…],
"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,"generalization":{"selection_metric":"net_expectancy_r","n_instruments":2,"worst_case":0.04,"sign_agreement":2,"per_instrument":[["EURUSD",0.04],["GER40",0.11]]},"winners":[["EURUSD",[["sma_cross.fast.length",{"I64":3}],…]],["GER40",[…]]]}]}}
$ echo $?
0
```
A process without walk_forward: the mc stage annotates **each surviving
member** instead (`"bootstrap":{"per_survivor":[[0,{…RBootstrap…}],[2,{…}]]}`,
ordinals indexing the population family — the `survivor_ordinals` convention).
### aura-engine
```rust
// mc.rs — RBootstrap currently derives Clone/Debug/PartialEq only:
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct RBootstrap { … unchanged fields … }
// (MetricStats, its field type, gains the same derives if it lacks them.)
```
### aura-registry
```rust
// lib.rs — Generalization gains serde derives (fields unchanged).
// lineage.rs — the mc annotation, payload mirroring the input shape (#200 d1/d4):
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum StageBootstrap {
/// After sweep/gates: one bootstrap per surviving member,
/// `(ordinal into the population family, result)`.
PerSurvivor(Vec<(usize, RBootstrap)>),
/// After a walk_forward: one bootstrap over the pooled per-window
/// OOS trade-R series (roll order).
PooledOos(RBootstrap),
}
pub struct StageRealization {
…existing fields…
/// std::monte_carlo stages only.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub bootstrap: Option<StageBootstrap>,
}
/// The campaign-scope generalize annotation (#200 d2): recorded at the scope
/// it is computed, keyed (strategy, window) across instruments.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct CampaignGeneralization {
pub strategy_ordinal: usize,
pub window_ordinal: usize,
/// None when fewer than 2 instrument cells produced a winner (gate
/// truncation) — the shortfall is recorded, never computed around.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub generalization: Option<Generalization>,
/// Per-instrument winning params — divergent winners are exposed,
/// never averaged away. Present whenever >= 1 winner exists.
pub winners: Vec<(String, Vec<(String, Scalar)>)>,
/// Instrument cells that contributed no winner (gate-truncated), by name.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub missing: Vec<String>,
}
pub struct CampaignRunRecord {
…existing fields…
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub generalizations: Vec<CampaignGeneralization>,
}
```
### aura-campaign — preflight v2
The v1 shape check is replaced by the v2 shape
`sweep (gate)* (walk_forward)? (monte_carlo)? (generalize)?` (each suffix
stage at most once, order fixed). `UnsupportedStage` disappears; new faults:
```rust
pub enum ExecFault {
…existing minus UnsupportedStage…
/// generalize needs >= 2 instruments in the campaign (static).
GeneralizeNeedsInstruments { available: usize },
/// generalize's metric must be an R metric (static — the intrinsic tier
/// accepts any vocabulary name; the shipped generalization() would only
/// refuse at runtime).
GeneralizeNonRMetric { metric: String },
/// monte_carlo with resamples == 0 or block_len == 0 (static, instead of
/// the engine's defined all-zero degenerate).
ZeroBootstrapParam { stage: usize, field: &'static str },
}
```
Shape violations (annotator before a population stage, duplicates, generalize
not last, mc before wf) stay `PipelineShape { detail }` with precise prose.
The generalize R-metric check reuses the registry's `check_r_metric`.
### aura-campaign — execution
```rust
// run_cell, new arm (replaces the unreachable! for MonteCarlo):
StageBlock::MonteCarlo { resamples, block_len } => {
let bootstrap = match &wf_family {
// after walk_forward: pool the wf family's OOS trade_rs in roll
// order (the shipped pooled_oos_trade_rs pattern), one bootstrap
Some(fam) => StageBootstrap::PooledOos(r_bootstrap(
&pooled_trade_rs(&fam.reports), *resamples as usize, *block_len as usize, seed)),
// after sweep/gates: one bootstrap per surviving member over its
// fresh in-memory trade_rs; zero-trade members record the defined
// all-zero degenerate explicitly
None => StageBootstrap::PerSurvivor(
survivors.iter().map(|(ordinal, _, report)| (*ordinal, r_bootstrap(
report.metrics.r.as_ref().map(|r| r.trade_rs.as_slice()).unwrap_or(&[]),
*resamples as usize, *block_len as usize, seed))).collect()),
};
stages.push(StageRealization { block: "std::monte_carlo".into(),
family_id: None, survivor_ordinals: None, selection: None,
bootstrap: Some(bootstrap) });
}
// execute(), after the cell loop — the campaign-scope generalize phase:
// for each (strategy_ordinal, window_ordinal): collect each instrument
// cell's FINAL selection winner report (the wf stage's per-window winners
// pooled? NO — the cell's nominated candidate is the last selection-bearing
// stage's winner REPORT: the sweep winner when no wf ran, else the wf
// family's last-window OOS report); >= 2 winners -> generalization(&pairs,
// metric) with winners+missing recorded; < 2 -> generalization: None,
// shortfall in `missing`.
```
The cell's **nominated report** for generalize is defined as: the OOS report
of the *last* walk-forward window when a wf stage ran (the freshest
out-of-sample evidence for the finally-selected params), else the sweep
stage's winner report. This is a definition this spec fixes so the executor,
the record, and the tests agree; it is recorded implicitly by `winners`
carrying the nominated params per instrument.
`run_cell` therefore returns (in `CellOutcome`) what the campaign phase needs:
the nominated winner report + params per cell (a small `nominee:
Option<(Vec<(String, Scalar)>, RunReport)>` on `CellOutcome`), `None` for
gate-truncated cells.
### aura-cli
- `exec_fault_prose`: arms for the three new faults (house style, e.g.
`campaign has {available} instrument(s); std::generalize needs at least 2`,
`process stage {stage}: monte_carlo {field} must be > 0`,
`std::generalize metric "{metric}" is not an R metric (gross/net R families
generalize; pip metrics do not)`); the `UnsupportedStage` arm and its
"not executable in v1" prose disappear (compile-driven by the variant
removal).
- The `campaign_run_v1_boundary_refuses_mc_process` seam test flips: an
mc-bearing process now RUNS (fake or real data permitting) — replaced by
pins on the new refusals (mc-before-wf shape, generalize single-instrument).
- `PROCESS_BLOCKS` doc strings in aura-research: "(terminal annotator in v1)"
→ "(terminal annotator)" — introspection TEXT only, no document bytes, no
content-id movement (verify: no golden pins the parenthetical; the describe
tests pin slot lines).
## Components
| Change | Crate |
|---|---|
| `RBootstrap` (+`MetricStats` if needed) serde derives | aura-engine |
| `Generalization` serde derives; `StageBootstrap`, `CampaignGeneralization`, `StageRealization.bootstrap`, `CampaignRunRecord.generalizations` | aura-registry |
| preflight v2 (shape + 3 static guards), mc arm, campaign-scope generalize phase, `CellOutcome.nominee`, `pooled_trade_rs` helper | aura-campaign |
| fault prose swap, seam-test flips, doc-string tidy | aura-cli, aura-research |
## Data flow
```
cell loop (unchanged) … wf stage
└─ mc stage: wf ran ? pooled OOS series → one RBootstrap
: per-survivor trade_rs → Vec<(ordinal, RBootstrap)>
→ StageRealization.bootstrap
after all cells: for each (strategy, window):
nominees across instruments (>= 2 ? generalization() : shortfall)
→ CampaignRunRecord.generalizations
stdout: the campaign_run line carries both (serde) — no new emit kinds
```
## Error handling
Exit codes unchanged (refusals 1 before any member runs; realization exit 0).
New refusals are static preflight faults with Debug-free prose. A zero-trade
bootstrap input is NOT an error: the engine's defined all-zero `RBootstrap`
is recorded explicitly (a null result is a valid research result).
## Testing strategy
1. **aura-registry:** record round-trip with `bootstrap`/`generalizations`
populated; a pre-widening `campaign_runs.jsonl` line (no new fields) still
parses (serde-default pin).
2. **aura-campaign (fake runner):** per-survivor mc (ordinals + per-member
RBootstrap against hand-computed `r_bootstrap` on planted `trade_rs`);
pooled-OOS mc after wf (input = wf family reports' trade_rs in roll
order); zero-trade degenerate recorded; deterministic twice (doc seed);
generalize over 2 fake instruments (worst_case/sign_agreement against the
shipped function, winners + params recorded); gate-truncated shortfall
(`generalization: None`, `missing` named); all new preflight refusals
(shape suffix order, duplicates, mc-before-wf, generalize-not-last,
single-instrument, non-R metric, zero resamples/block_len).
3. **aura-cli seam:** new-fault prose pins (Debug-free); the flipped
v1-boundary test; `--metrics`/introspection untouched.
4. **Gated real-data e2e:** extend the 0107 e2e process with the annotator
suffix where local data exists (2 instruments if a second archive exists,
else the mc-only suffix + a generalize-refusal assert); skip elsewhere.
## Acceptance criteria
1. The worked process above runs end-to-end (data permitting): exit 0, the
`campaign_run` line and stored record carry `bootstrap` and (with >= 2
instruments) `generalizations`.
2. A no-wf pipeline annotates every surviving member (`per_survivor`).
3. All new preflight refusals fire before any member runs, with precise
prose; `[sweep, mc, walk_forward]` — intrinsically valid — is refused.
4. Existing stored `campaign_runs.jsonl` lines parse unchanged; no document
content id moves this cycle.
5. Suite green, clippy clean, doc build clean.