The campaign walk_forward stage picked its per-window in-sample winner by
optimize_deflated (argmax) unconditionally, dropping the stage's `select`
field, and both the preflight and the `walkforward` verb's `--real` campaign
sub-branch refused plateau:* with a forward-pointer here (the #210 decision-4
deferral; the synthetic non-`--real` sub-branch has accepted plateau since
#173). Honour plateau selection when no gate precedes the wf stage.
- Preflight: the plateau-in-wf refusal is narrowed to gate-preceded stages.
With no preceding gate the survivors are the full sweep grid, so the
parameter lattice (grid.axis_lens) is exact and optimize_plateau is sound; a
gate filters survivors below the grid, breaking the lattice, so plateau after
any gate stays refused. The refusal is static — document validity must be
data-independent (a gate's filtering depends on the data) — and keeps the
existing "a gated survivor subset has no parameter lattice" message, which now
describes exactly the only case it fires.
- Executor: run_walk_forward_stage threads the per-cell grid.axis_lens and picks
each window's in-sample winner via select_sweep_winner — the same dispatch the
sweep stage uses. (Argmax, deflate=true) is byte-identical to the old
optimize_deflated call, so the argmax grade and content-id pins are unchanged;
the plateau arms read axis_lens and ignore deflate.
- CLI: the walkforward verb threads --select through translate_walkforward /
run_walkforward_sugar (a walkforward-local param, not the shared four-verb
SugarInvocation), mapping the parsed Selection to the research SelectRule via
select_rule_of; the refusal and its select_is_plateau helper are deleted.
Scope is walkforward-only — the mc verb's wf-stage select is untouched.
The plateau:worst grade anchor (stitched_total_pips = -9683776.67) is distinct
from the argmax anchor (-10398606.67), proving the plateau path genuinely
changes winner selection rather than falling back to argmax. A gate-free plateau
process validates clean while a gate-preceded one is refused (a new
campaign-validate e2e). Argmax defaults stay byte-identical.
closes#215
The persist-side C1 drift alarm re-ran every nominee under the hardcoded
default vol-stop because CellRealization dropped the regime at the record
boundary — a non-default risk regime + persist taps false-failed C1 and
refused to persist a correct trace; two regimes over the same cell would
also have collided in one trace dir.
Fix at the record boundary: CellRealization carries regime + regime_ordinal
(serde-default + skip-serializing keeps pre-existing stored records
byte-stable), populated where the cell is realized; the regime->StopRule
binding is single-sourced in stop_rule_for_regime, shared by the member
runner and the persist re-run so the two sites cannot drift apart again;
campaign_cell_key appends -r{regime_ordinal} for non-default regimes
(default keys unchanged, no stored dir renamed).
closes#219, closes#212
The regime becomes the fourth structural coordinate of the campaign matrix, a
kept-separate axis (a peer of window, not aggregated-over like instrument):
- aura-campaign: CellSpec gains `regime` + `regime_ordinal`; the executor
resolves the regime list (absent/empty risk -> one default cell, ordinal 0),
runs a cell per (strategy, instrument, window, regime), and keys the nominee
map by the 3-tuple including the regime ordinal — so generalize aggregates
across instruments WITHIN a regime, never across regimes. Family names gain a
`-r{regime_ordinal}` discriminator so two regimes' families cannot collide.
- aura-registry: `CampaignGeneralization` gains `regime_ordinal` (with
`#[serde(default)]`, so pre-feature stored records still parse — pinned by a
new backward-compat test).
The existing single-regime execute tests shift their family ids by `-r0`; a new
test pins that two regimes yield distinct `-r0`/`-r1` families. `CellSpec.regime`
is write-only in this slice — its sole reader is the deferred member runner
(the next slice) — a held quality nit, seam-prep as the plan prescribes. Full
workspace suite green.
refs #210
The first slice of the risk-regime structural axis (spec/plan abab1f1):
- aura-research: a serializable `RiskRegime` closed vocabulary (externally
tagged, sole variant `Vol { length, k }` — a content-addressable mirror of
the shipped two-variant StopRule, so a future Fixed regime is additive), and
an optional top-level `CampaignDoc.risk: Vec<RiskRegime>` that absent-
serializes (the `description` precedent) — content-id parity for every
risk-less document.
- validate_campaign refuses a non-positive regime (length < 1 or k <= 0, NaN
included) via a new `DocFault::BadRegime`; the CLI renders it as prose at the
research_docs seam (register/validate paths).
- The three CLI-seam E2E tests pin that a well-formed risk section passes
`campaign validate` and a bad one is refused with prose (exit 1, no Debug
leak) — the field is reachable at the seam a data author drives, not only in
library units.
Adding the field is compile-breaking at every `CampaignDoc { .. }` literal
(serde default does not cover Rust construction), so this slice also threads
`risk: vec![]` into the sites across aura-cli and aura-campaign — the
field-addition's compile-gate completion the plan should have scoped here. The
BadRegime prose reads `risk[i]: ...` (the implementer's terser wording over the
plan's illustrative string; the spec pinned no exact text). Full workspace
suite green.
refs #210
The std::sweep selection triple (metric/select/deflate) becomes ONE
optional group, Option<SweepSelection>, serde-flattened so the wire
form keeps the flat slots: every existing document parses to Some and
its canonical bytes — and content id — are unchanged (golden pin
byte-untouched). A half-populated group is refused by the schema-strict
parser (all-or-nothing); a bare {"block":"std::sweep"} is the
selection-free sweep.
Campaign side: preflight permits a selection-free sweep only as the
pipeline's terminal stage (new ExecFault::SelectionFreeSweepNotTerminal,
prose in aura-cli's exhaustive exec_fault_prose); the executor's
selection-free arm appends the family exactly as before but records no
StageSelection and produces no nominee (recording one would fabricate
intent — #210 fork decision 1).
Introspection follows: the sweep block's schema table marks the group
optional; CLI introspection pins updated. New tests: group-or-nothing
refusals, selection-free round-trip + distinct content id, terminal-only
preflight accept/refuse, selection-free execute realization.
refs #210
DEFLATION_N_RESAMPLES (1000) and DEFLATION_BLOCK_LEN (5) were duplicated
as independent copies in aura-cli and aura-campaign — a silent-drift risk
(an edit in one crate but not the other refuses nothing, it just produces
a wrong deflation result). Promote a single definition into aura-registry,
which owns optimize_deflated (the sole consumer of these counts); aura-cli
imports it and aura-campaign re-exports it to keep its public path stable.
DEFLATION_SEED stays cli-local (no campaign twin). Values unchanged;
behaviour-preserving, suite green unchanged (1043/0).
closes#199
Task 1: the tap namespace is a closed vocabulary — tap_vocabulary()
(the wrap convention's four sink names; escalation = new entry or an
authored blueprint sink, never an open node-path namespace) with
DocFault::UnknownTap in validate_campaign (index-addressed, prose
enumerating the vocabulary via the single source), SlotKind::TapKinds
on the persist_taps slot, open-slot hint naming the taps. The 0107
deferral seam test split into its two 0109 pins (unknown tap refuses
at validate; a valid tap passes validate and hits the member-data
seam) — the deferral eprintln itself survives until task 3.
Task 2: CampaignRunRecord.trace_name lands serde-default sparse; the
claim contract holds across the seam — execute stamps the sentinel iff
persist_taps is non-empty, append_campaign_run composes
Some("{campaign8}-{run}") onto the stored line (safe prefix take at
the registry seam), execute mirrors the same derivation onto the
returned copy; None stays None; the hand-seeded byte-identity dump pin
survives (absent -> skipped).
Gates: workspace 1037/0, clippy -D warnings clean.
refs #201
valid == runnable for every pure document/campaign property: inside a
project, once intrinsic + referential pass, validate fetches the
referenced process from the store and runs the data-free
aura_campaign::preflight (promoted to pub) — an executable shape gains
'campaign document valid (executable): pipeline shape and static
guards pass'; a shape/param/metric/instrument fault refuses under
'campaign is not executable:' with the exec_fault_prose detail, exit 1,
before any member runs. Outside a project the skip line is unchanged
(the third tier needs the store like the second).
Collateral the RED author predicted: the referential e2e's OK fixture
(PROCESS_DOC: deflate+plateau sweep, overfit_probability gate) was
intrinsically-valid-but-never-executable — that test now registers an
executable twin (argmax/no-deflate, per-member gate metric); the base
PROCESS_DOC fixture stays untouched for all other consumers.
RED-first (tdd-author handoff): the third-tier pin observed failing on
the two-line validate output.
Gates: workspace 1026/0, clippy -D warnings clean.
closes#205
The mc arm bootstraps the stage's incoming R-evidence with one
semantics, input-shaped by position (#200 decision 1): after a
walk_forward it pools the wf family's per-window OOS trade_rs in roll
order into ONE r_bootstrap (PooledOos); after sweep/gates it
bootstraps each surviving member's fresh in-memory series
(PerSurvivor, ordinals into the population family; a zero-trade
member records the engine's defined all-zero degenerate). Seeded from
the campaign doc's seed (C1). The wf family is read by reference from
the cell's families vec — no clone. run_cell's Generalize arm is an
explicit no-op naming the campaign scope.
The generalize phase runs after the cell loop (#200 decision 2):
nominees (last wf window's OOS report, else the sweep winner; None on
gate truncation) grouped by (strategy, window) across instruments via
a BTreeMap (deterministic order); >= 2 winners feed the shipped
generalization(), fewer record the shortfall (generalization: None,
missing named) — divergent per-instrument winners are exposed via
their params, never averaged away.
Gates: campaign 40/0 (4 execute_mc + 3 execute_generalize new),
workspace build clean; the known-RED cli integration test flips in
task 5.
refs #200
Task 1: RBootstrap and Generalization gain serde derives;
lineage.rs gains StageBootstrap (per_survivor | pooled_oos) and
CampaignGeneralization; StageRealization.bootstrap and
CampaignRunRecord.generalizations land serde-default sparse (a
pre-0108 campaign_runs.jsonl line still parses — pinned), threaded
None/empty through every literal site. No behaviour change.
Task 2: ExecFault::UnsupportedStage is gone; the preflight now admits
the v2 shape sweep [gate]* [wf]? [mc]? [generalize]? via a monotone
rank walk (precise per-pair PipelineShape prose) and statically
refuses: single-instrument generalize (GeneralizeNeedsInstruments),
non-R generalize metrics via check_r_metric (GeneralizeNonRMetric —
no fourth roster site), zero mc params (ZeroBootstrapParam). CLI
prose swapped accordingly. The loop's E2E phase added a tier-boundary
pin: process validate accepts [sweep, mc, walk_forward] while campaign
run refuses it data-free.
KNOWN RED (documented in the plan, flips in task 5):
crates/aura-cli/tests/research_docs.rs::campaign_run_v1_boundary_refuses_mc_process
still pins the retired 'not executable in v1' prose. Interim gates:
registry 57/0, campaign 33/0, cli --bin 111/0, workspace build clean.
refs #200
The roller refusal leaked its raw Rust form through ExecFault::Window
(SpanTooShort { span: (Timestamp(...), ...), need: ... }) — the one
break in the cycle's Debug-free prose seam. The detail now phrases
both roller refusals in the doc's own ms unit:
'in_sample_ms + out_of_sample_ms = N ms exceeds the campaign window
[a, b] (L ms)'. RED-first: the wf_roller_refusal_maps_to_window_fault
pin was flipped to the prose form (observed RED on the old Debug
output), then the map_err phrased; live repro confirmed against the
fieldtest fixture.
Gates: workspace 987/0, clippy -D warnings clean.
refs #198
campaign_run.rs: target resolution (file is register-then-run sugar;
bare 64-hex is the canonical id — #198 decision 1), project gate before
any store write, zero-fault referential gate, process fetch + v1
preflight, then aura_campaign::execute with the CLI MemberRunner over
the shipped loaded-blueprint convention: per-member blueprint reload,
reduce-mode wrap, unique suffix-join of raw campaign axis names onto
the wrapped param_space, windowed real data via the shipped ms->ns
source seam (absent archive/zero-bar windows -> NoData). Emission:
family_table / selection_report lines gated on the doc's emit list
(names debug_asserted against emit_vocabulary), the campaign_run
record line always, persist_taps deferred LOUDLY on stderr before
execution (F7 lesson), zero-survivor cells noted on stderr with exit 0.
exec_fault_prose/member_fault_prose keep the Debug-free house seam.
Seam tests: 8 campaign_run_* e2e tests incl. outside-project, bogus
target, unknown id, v1-boundary (mc process registers fine, run
refuses), persist_taps ordering, and the gated real-data
sweep->gate->walkforward e2e — which ran its full assert path on this
host (local GER40 2024-09 archive) rather than the data-less skip.
Also: exec.rs campaign-id guard tightened to lowercase hex, matching
is_content_id and the store's self-keyed form (review nit).
Gates: workspace 975/0, clippy -D warnings clean.
refs #198
The execution core: cells iterate strategy x instrument x window in doc
order (sequential — C1 keeps parallelism across sims); the sweep stage
enumerates the axes odometer (BTreeMap order, last axis fastest) through
the engine sweep over a ListSpace with lowest-index-deterministic fault
capture; selection dispatches optimize / optimize_plateau /
optimize_deflated (deflation nulls seeded from the campaign doc's seed);
gates filter per-member via member_metric (metrics.r == None fails an R
predicate, conservative); an empty survivor set truncates the cell's
realization and the run continues (exit-0 semantics at the caller). The
walk-forward stage rolls the declared cell window in ms (WfMode ->
RollMode), IS-sweeps ONLY the survivor points per window, stamps
manifest.selection on OOS winner reports, and appends a WalkForward
family; realization = CampaignRunRecord over untouched family records.
The loop's task-7 BLOCKED was review-exhaustion on the task-6 stub
wording the plan itself prescribed (task 7 deletes that stub); a fresh
post-loop quality review on the final diff returned one Minor — the
campaign_id prefix slice could panic on a malformed id — fixed as a
typed PipelineShape refusal + test (execute_refuses_malformed_campaign_id).
Known and accepted: a cross-cell member fault leaves earlier cells'
families as unreferenced records in the append-only store; nested
window/member parallelism can oversubscribe cores on large campaigns.
Gates: aura-campaign 29/0, workspace 961+/0, clippy -D warnings clean.
refs #198
New leaf library crate: the campaign-execution semantics home (#198
home decision — reusable beyond the CLI; NOT C21's project-side
World). CellSpec + the MemberRunner seam (the only thing a consumer
implements), MemberFault/ExecFault (Display-free, prose at the binary
seam), member_metric over the 14 per-member scalars (third roster
site, refs #190 — drift fails safe as a preflight refusal), and
preflight: v1 pipeline shape std::sweep [std::gate]* [std::walk_forward],
rankable-metric and gate-metric rosters, plateau-in-wf and
deflate+plateau refusals, i64-fit guard on wf lengths — all before any
member runs.
Post-loop: the held quality finding (untested i64-fit arm) verified
real and closed by hand — preflight_refuses_wf_length_exceeding_i64.
Gates: aura-campaign 14/0, clippy -D warnings clean.
refs #198
Brummel