tidy: clear the milestone fieldtest findings (refusal reasons, --help, ledger + glossary)
Resolves the four non-bug findings from the inferential-validation milestone
fieldtest (the bug — the broken-pipe panic — is fixed in 1088320):
- [friction] generalize refusals now carry a reason instead of a bare usage dump:
<2 instruments, a multi-value candidate flag, a missing knob, a duplicate
symbol, and a non-stage1-r strategy each get a one-line why (mirroring the
R-only metric refusal). Two parser tests strengthened to pin the message.
- [friction] --select <argmax|plateau:mean|plateau:worst> added to the
walkforward branch of the global USAGE (aura --help) — the plateau objective
was undiscoverable from the only help the CLI prints (the per-subcommand usage
already carried it).
- [spec_gap] C1 gains a scope note: bit-identity is per-run; a derived metric
recomputed by two different command paths may differ by floating-point
reassociation (<=1 ULP), which is not a C1 violation — ratifies the
fieldtester's reading of the sweep-vs-generalize sqn difference.
- [spec_gap] the milestone vocabulary is added to docs/glossary.md:
cross-instrument generalization, deflated score, overfit probability,
neighbourhood score, plateau selection, sign-agreement, worst-case floor.
The fieldtest triage spec (docs/specs/fieldtest-*) is removed now its findings
are resolved; the tracked fieldtests/ scenario corpus stays.
Verified: cargo test --workspace green (0 failed), clippy --workspace
--all-targets -D warnings clean; `aura --help` shows --select; the refusals
print their reason.
refs #146
This commit is contained in:
@@ -55,10 +55,18 @@ The type-erased 64-bit word holding one scalar-base-type value with its kind str
|
||||
**Avoid:** —
|
||||
A node that wires a sub-graph and exposes one output (a combined signal, or a strategy) — composition is fractal and acyclic. Also names the multi-column stream a node emits: the **record** a producer's `eval` returns, bundling 1..K base scalar columns (e.g. OHLCV), each bound field-wise by a consumer (C7/C8).
|
||||
|
||||
### cross-instrument generalization
|
||||
**Avoid:** cross-symbol pooling, pooled generalization
|
||||
The validation read that grades how consistently one *brought* candidate holds across a set of instruments, scored on its weakest one — the across-instrument axis of the anti-false-discovery discipline. Realised by `aura generalize`; an aggregator (a recomputable family score), never a selector that picks a winner.
|
||||
|
||||
### cycle
|
||||
**Avoid:** —
|
||||
One data-driven clock step: one input record = one cycle, advanced in global timestamp order with a monotonic `cycle_id`. (In the pipeline-process sense "cycle" also names one milestone round; the engine sense is this clock step.)
|
||||
|
||||
### deflated score
|
||||
**Avoid:** trials-adjusted score
|
||||
A sweep winner's metric penalised for the number of configurations tried (trials-deflation), recorded additively on the winning member's `selection` without changing which member won. Paired with an **overfit probability**; the across-trials defence against a winner that is luck at sweep scale.
|
||||
|
||||
### edge
|
||||
**Avoid:** —
|
||||
A directed wiring link in a harness that forwards **one field** (`from_field`) of a producer node's `eval` output record into a consumer node's input slot (the engine's `Edge`); the source-side variant binding the source value into an input slot is a **source target** (`Target`). Edges define the DAG the bootstrap topologically orders; a per-field scalar-kind mismatch across an edge is rejected at bootstrap.
|
||||
@@ -115,10 +123,22 @@ The reproducible metadata record of a run (node-commit + params + data-window +
|
||||
**Avoid:** MC
|
||||
An orchestration axis running N seeded realizations that perturb the input; each realization is itself deterministic given its seed (Monte-Carlo = sweep over seeds).
|
||||
|
||||
### neighbourhood score
|
||||
**Avoid:** smoothed score
|
||||
The metric of a sweep cell averaged (or worst-cased) over its closed grid neighbourhood — the surface a **plateau selection** argmaxes instead of the bare peak. Recorded on a plateau winner's `selection` alongside its neighbour count.
|
||||
|
||||
### node
|
||||
**Avoid:** block
|
||||
The universal composable dataflow unit, implementing `lookbacks()` + `eval(ctx)` — a producer, a pure consumer (sink), or both — with at most one output port; a producer's output is a **record of 1..K base-scalar columns** (a scalar is the degenerate K=1 case). Everything that plugs into the engine is fractally a node.
|
||||
|
||||
### overfit probability
|
||||
**Avoid:** P(overfit), luck probability
|
||||
The recorded chance that a deflated sweep winner is noise rather than edge — a rank of the winner's metric against a centred no-edge null over the family. Paired with the **deflated score** on the winner's `selection`.
|
||||
|
||||
### plateau selection
|
||||
**Avoid:** plateau-over-peak (as a noun)
|
||||
A selection objective that argmaxes the neighbourhood-smoothed metric surface (mean or worst-case) rather than the bare in-sample peak, preferring a robust parameter plateau to a lucky spike. Opt-in via `--select plateau:mean|plateau:worst`; the default selection stays a bare argmax.
|
||||
|
||||
### playground
|
||||
**Avoid:** —
|
||||
The egui-native visual face (`aura play`) that plays any harness — program structure before a run, live sink streams during, recorded traces and meta-views after. A trace explorer / execution viewer, never a scene editor.
|
||||
@@ -171,6 +191,10 @@ The four streamed scalar kinds — `i64`, `f64`, `bool`, `timestamp` (a newtype
|
||||
**Avoid:** —
|
||||
A node that exposes session context as scalar streams (`bars_since_open`, `in_session`, `session_open_ts`) so session logic stays inside the stream model. Calendars and instrument specs remain metadata beside the hot path.
|
||||
|
||||
### sign-agreement
|
||||
**Avoid:** sign consistency
|
||||
In a cross-instrument generalization, the count of instruments on which the candidate's R-metric is net positive. Reported beside the **worst-case floor** to express consistency of direction across instruments.
|
||||
|
||||
### signal
|
||||
**Avoid:** —
|
||||
A node whose output is a score, feeding the `signals (scores) → decision node → bias stream` chain. A specific node role — distinct from a general node and from a strategy.
|
||||
@@ -230,3 +254,7 @@ An orchestration axis: rolling in-sample optimize + out-of-sample test across mo
|
||||
### World
|
||||
**Avoid:** —
|
||||
The project's meta-level program that dynamically constructs and orchestrates *families* of harnesses (walk-forward / sweep / optimize / Monte-Carlo / comparison). aura's differentiator and product, as opposed to the single-backtest substrate.
|
||||
|
||||
### worst-case floor
|
||||
**Avoid:** min-over-instruments
|
||||
The headline cross-instrument generalization score: the minimum of a candidate's R-metric across the instruments, so a single strong market cannot flatter it (non-dominable by construction). The cross-instrument sibling of a worst-case **plateau selection**.
|
||||
|
||||
Reference in New Issue
Block a user