audit(0066): cycle close — lift R-sweep/SQN rationale to ledger, drop ephemera
Cycle 0066 (#133) closed. The close audit (architect) found the family ratified against C12 axis-2 (rank-by-metric over the C18 family store) and the C10 R framing, with one [high] gap (swept-member manifest omitted the fixed R-defining params) fixed forward in 499121e.
- Ledger: a C10 realization note records the durable rationale -- SQN as the single-number sweep objective, the signal-only grid (the stop defines R, so varying it breaks cross-member SQN comparability -- the SQN100 #130 motivation), the R-metric rank vocabulary, and the manifest-completeness fix.
- Glossary: new 'R metrics' and 'SQN' entries.
- The ephemeral spec + plan are removed; the ledger + git history are the durable record.
Follow-ups filed: #135 (per-member --trace for the stage1-r sweep). refs #133
This commit is contained in:
@@ -644,6 +644,22 @@ reconstruction loses precision; the stored `realized_r` is exact). Composites li
|
||||
acyclic. **Stage 2** (currency P&L, fixed-fractional compounding through the z⁻¹
|
||||
fill-edge register, realistic brokers consuming the position-event table) remains the
|
||||
deferred downstream layer, entered only after `E[R] > 0`.
|
||||
**Realization (cycle 0066).** SQN is now the operational single-number objective for
|
||||
ranking a Stage-1 sweep family by signal quality — C12 **axis-2 (argmax-metric)** over
|
||||
the C18 family store. `metric_cmp` (`aura-registry`) learns the higher-is-better R
|
||||
metrics `sqn`, `expectancy_r`, `net_expectancy_r`; a member with no `r` block sorts
|
||||
last (`NEG_INFINITY`), so a pip-only family ranked by an R metric degrades to ordinal
|
||||
order rather than erroring. `aura sweep --strategy stage1-r` produces the rankable R
|
||||
family, each member folding `summarize_r` into `RunMetrics.r`. The default grid varies
|
||||
**only the signal** (`fast`/`slow` SMA lengths), holding the stop and sizing fixed:
|
||||
`risk_budget` is R-invariant and `bias.scale` is sign-only under flat-1R (both
|
||||
degenerate axes for the ranked metric), and — load-bearing — the **stop defines 1R**,
|
||||
so varying it would change what R *means* per member and break cross-member SQN
|
||||
comparability (the motivation for the deferred n-normalized SQN100, #130). Each swept
|
||||
member's manifest records the fixed R-defining params (stop, scale) beside the floated
|
||||
knobs, so a member is reproducible from its own manifest (C18) and the constant-stop
|
||||
basis of comparability is auditable from the family record. Per-member `--trace` for
|
||||
the stage1-r sweep is refused explicitly (not a silent no-op) pending #135.
|
||||
|
||||
### C11 — Generalized sources; record-then-replay determinism boundary
|
||||
**Guarantee.** A source is anything that produces timestamped scalar streams —
|
||||
|
||||
@@ -131,6 +131,10 @@ A broker-independent, time-ordered table of position events (scalar columns: `ev
|
||||
**Avoid:** —
|
||||
The native unit of strategy performance: 1R = the loss if the protective stop is hit, so a trade's outcome is a multiple of R (−1R stopped out, +2R, …) and the primary question is *"how much R out per 1R risked?"*. Risk-normalized and account-/instrument-agnostic — the Stage-1 yardstick replacing pips, defined by the stop (Van Tharp R-multiples).
|
||||
|
||||
### R metrics
|
||||
**Avoid:** —
|
||||
The R-multiple summary metrics `summarize_r` folds into `RunMetrics.r`, which double as the higher-is-better rank vocabulary for `aura runs family <id> rank <metric>` (cycle 0066): **`sqn`** (the System Quality Number — see SQN), **`expectancy_r`** (mean realized R per trade, the headline E[R]), and **`net_expectancy_r`** (expectancy net of one round-trip cost per trade — churn-honest; equals `expectancy_r` while Stage 1 is frictionless). A family member with no `r` block ranks last under any of them, so a pip-only family ranked by an R metric falls back to ordinal order.
|
||||
|
||||
### R-evaluator
|
||||
**Avoid:** —
|
||||
The Stage-1 signal-quality node: integrates the per-trade R-outcomes of a flat-1R `RiskExecutor` into an R-expectancy / R-curve — the deterministic, account-/instrument-agnostic yardstick. The R-reframe successor of the `sim-optimal broker` (pip integral); unlike it the R-evaluator requires a stop, so it consumes a RiskExecutor, not raw bias.
|
||||
@@ -195,6 +199,10 @@ The columnar Structure-of-Arrays layout in which the four scalar base types are
|
||||
**Avoid:** —
|
||||
Anything that produces timestamped scalar streams — market data and non-financial feeds (e.g. a news-bias node) are treated identically; a pure producer node. `data-server` is aura's first source.
|
||||
|
||||
### SQN
|
||||
**Avoid:** system-quality-number (spell out once, then SQN)
|
||||
System Quality Number — `√n · mean_R / stdev_R`: the dispersion- and trade-count-aware single-number objective for ranking a sweep family by R signal quality (cycle 0066; van Tharp). Higher is better; `n < 2` or zero-variance → 0. One of the R metrics; an n-normalized variant (SQN100) is deferred (#130).
|
||||
|
||||
### Stage 1 / Stage 2
|
||||
**Avoid:** —
|
||||
The two-stage cleave of evaluation: **Stage 1** (R / signal quality) is flat-1R, feed-forward (no equity feedback), measured by the R-evaluator in R-expectancy — the primary research loop; **Stage 2** (currency / deploy viability) is fixed-fractional sizing that reads equity and compounds, with realistic brokers and real frictions, entered only once `E[R] > 0`. The cleave quarantines the only feedback into Stage 2 (flat-1R vs compounding is a structural axis, C11); aura's own synthesis, closest precedent vectorbt's signal vs money-management split.
|
||||
|
||||
@@ -1,542 +0,0 @@
|
||||
# R-sweep families + rank-by-SQN — Implementation Plan
|
||||
|
||||
> **Parent spec:** `docs/specs/0066-r-sweep-rank.md`
|
||||
>
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
|
||||
> this plan. Steps use `- [ ]` checkboxes for tracking.
|
||||
|
||||
**Goal:** Make SQN the single-number objective for ranking a Stage-1 sweep family
|
||||
by R signal quality: `aura sweep --strategy stage1-r` produces an R-bearing family,
|
||||
and `aura runs family <id> rank sqn` (also `expectancy_r`, `net_expectancy_r`) ranks
|
||||
it best-first.
|
||||
|
||||
**Architecture:** Two independent crates. `aura-registry` — `metric_cmp` learns
|
||||
three higher-is-better R metrics that reach into `RunReport.metrics.r`, with a
|
||||
`None` member sorting to the worst position. `aura-cli` — `enum Strategy` gains
|
||||
`Stage1R`; the stage1-r topology is extracted into one helper so the single run
|
||||
keeps its identical binding (byte-output unchanged, verified by a new golden) while
|
||||
the sweep floats `fast.length`/`slow.length` over a 4-point grid, folding `r:
|
||||
Some(..)` per member. The family store, its persistence, and the `runs family …
|
||||
rank …` routing are unchanged — they already carry full `RunReport`s and route any
|
||||
metric string through `metric_cmp`.
|
||||
|
||||
**Tech Stack:** `aura-registry/src/lib.rs` (Metric/metric_cmp/Display + tests),
|
||||
`aura-cli/src/main.rs` (Strategy/parse_sweep_args/run_sweep/stage1_r_graph/
|
||||
stage1_r_sweep_family + unit tests), `aura-cli/tests/cli_run.rs` (golden +
|
||||
integration test), read-only fixture source `aura-engine/src/report.rs`.
|
||||
|
||||
**Files this plan creates or modifies:**
|
||||
|
||||
- Modify: `crates/aura-cli/tests/cli_run.rs` — new golden char. test + new sweep+rank integration test
|
||||
- Modify: `crates/aura-cli/src/main.rs` — `stage1_r_graph` helper, `stage1_r_sweep_family`, `Strategy::Stage1R`, parse + dispatch, usage strings, unit test
|
||||
- Modify: `crates/aura-registry/src/lib.rs` — `Metric` variants, `metric_cmp` arms, `UnknownMetric` Display, R-metric tests
|
||||
- Read-only: `crates/aura-engine/src/report.rs` — `RMetrics` struct literal (fixture source)
|
||||
|
||||
**Task order (dependencies):** Task 1 (golden, GREEN on current HEAD) → Task 2
|
||||
(helper extraction; golden + existing stage1-r tests stay GREEN) → Task 3
|
||||
(registry metric_cmp, self-contained RED→GREEN) → Task 4 (sweep family + CLI
|
||||
wiring, RED→GREEN; needs Task 2's helper and Task 3's rank-by-sqn).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Golden characterization test for the stage1-r single run
|
||||
|
||||
Pins the EXACT current `metrics` output so Task 2's refactor is *verified*
|
||||
byte-preserving (the grounding-check found no such golden exists). Captured against
|
||||
current HEAD → GREEN immediately; Task 2 must keep it GREEN.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-cli/tests/cli_run.rs` (append a test near the stage1-r tests ~1600)
|
||||
|
||||
- [ ] **Step 1: Capture the current metrics literal**
|
||||
|
||||
Run: `cargo run -q -p aura-cli -- run --harness stage1-r`
|
||||
This prints one JSON line `{"manifest":{...},"metrics":{...}}`. Copy the substring
|
||||
from `"metrics":` to the end of the line (the trailing `}` included) — this is the
|
||||
golden literal for Step 2. Example shape (DO NOT use these numbers — paste the real
|
||||
captured bytes):
|
||||
`"metrics":{"total_pips":<f>,"max_drawdown":<f>,"bias_sign_flips":<n>,"r":{"expectancy_r":<f>,...,"conviction_terciles_r":[<f>,<f>,<f>]}}`
|
||||
|
||||
- [ ] **Step 2: Add the golden test, pasting the captured literal**
|
||||
|
||||
```rust
|
||||
/// Golden characterization (cycle 0066): pins the EXACT metric output of the
|
||||
/// stage1-r single run, so the `stage1_r_graph` helper extraction (Task 2) is
|
||||
/// VERIFIED byte-preserving, not assumed. Captured against HEAD before the
|
||||
/// refactor; the refactor must keep it identical (a drift is a behaviour change →
|
||||
/// debug). Commit-agnostic: pins only the `metrics` object, since the manifest
|
||||
/// carries the volatile build commit.
|
||||
#[test]
|
||||
fn stage1_r_single_run_output_golden() {
|
||||
let out = Command::new(BIN).args(["run", "--harness", "stage1-r"]).output().unwrap();
|
||||
assert!(out.status.success(), "exit: {:?}", out.status);
|
||||
let s = String::from_utf8(out.stdout).expect("utf-8 stdout");
|
||||
let key = "\"metrics\":";
|
||||
let start = s.find(key).expect("stdout has a metrics object");
|
||||
let metrics = s[start..].trim_end();
|
||||
assert_eq!(
|
||||
metrics,
|
||||
r#"PASTE_THE_CAPTURED_METRICS_LITERAL_FROM_STEP_1"#,
|
||||
"stage1-r single-run metric output drifted from the golden: {s}"
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the golden against current HEAD — must already pass**
|
||||
|
||||
Run: `cargo test -p aura-cli --test cli_run stage1_r_single_run_output_golden`
|
||||
Expected: PASS (it characterizes the unmodified code). If it FAILS here, the pasted
|
||||
literal is wrong — re-capture from Step 1.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Extract `stage1_r_graph` helper; float fast/slow; single run binds Some(2)/Some(4)
|
||||
|
||||
Behaviour-preserving refactor. One wiring source; the single-run binding is
|
||||
structurally identical, so the Task-1 golden + the existing stage1-r tests stay
|
||||
GREEN.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-cli/src/main.rs:1737-1788` (rename `stage1_r_blueprint` → `stage1_r_graph`, add the two `Option<i64>` params)
|
||||
- Modify: `crates/aura-cli/src/main.rs:1806-1808` (`run_stage1_r` calls the helper with `Some(2)`/`Some(4)`)
|
||||
|
||||
- [ ] **Step 1: Replace `stage1_r_blueprint` with `stage1_r_graph`**
|
||||
|
||||
Replace the whole `fn stage1_r_blueprint(...) -> Composite { ... }` (1737-1788)
|
||||
with the helper below. Only the `fast`/`slow` builder lines change (made
|
||||
conditional on the `Option`); every other line is verbatim from the original body.
|
||||
|
||||
```rust
|
||||
/// The Stage-1 R harness topology, shared by the single run and the sweep. The two
|
||||
/// signal knobs are bound when `Some` (single run — identical to the old
|
||||
/// build-time bind, output byte-unchanged) and left free when `None` (sweep — they
|
||||
/// land in `param_space` as `fast.length` / `slow.length`). Four taps: equity
|
||||
/// (SimBroker), exposure (Bias), the 14-column R-record (→ summarize_r), and
|
||||
/// r_equity = cum_realized_r + unrealized_r.
|
||||
#[allow(clippy::type_complexity, clippy::too_many_arguments)]
|
||||
fn stage1_r_graph(
|
||||
tx_eq: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
|
||||
tx_ex: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
|
||||
tx_r: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
|
||||
tx_req: mpsc::Sender<(Timestamp, Vec<Scalar>)>,
|
||||
fast_len: Option<i64>,
|
||||
slow_len: Option<i64>,
|
||||
) -> Composite {
|
||||
let mut g = GraphBuilder::new("stage1_r");
|
||||
// SMA-cross signal → Bias (the same signal the pip sample uses).
|
||||
let mut fast_b = Sma::builder().named("fast");
|
||||
if let Some(l) = fast_len {
|
||||
fast_b = fast_b.bind("length", Scalar::i64(l));
|
||||
}
|
||||
let fast = g.add(fast_b);
|
||||
let mut slow_b = Sma::builder().named("slow");
|
||||
if let Some(l) = slow_len {
|
||||
slow_b = slow_b.bind("length", Scalar::i64(l));
|
||||
}
|
||||
let slow = g.add(slow_b);
|
||||
let spread = g.add(Sub::builder());
|
||||
let exposure = g.add(Bias::builder().named("bias").bind("scale", Scalar::f64(0.5)));
|
||||
// pip branch (verbatim from sample_blueprint_with_sinks).
|
||||
let broker = g.add(SimBroker::builder(SYNTHETIC_PIP_SIZE));
|
||||
let eq = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_eq));
|
||||
let ex = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_ex));
|
||||
// R branch: bias + price → RiskExecutor(vol_stop) → dense R-record.
|
||||
let exec = g.add(risk_executor(
|
||||
StopRule::Vol { length: STAGE1_R_STOP_LENGTH, k: STAGE1_R_STOP_K },
|
||||
1.0,
|
||||
));
|
||||
let rrec = g.add(Recorder::builder(PM_RECORD_KINDS.to_vec(), Firing::Any, tx_r));
|
||||
// r_equity = cum_realized_r + unrealized_r — one tapped series for charting.
|
||||
let r_equity = g.add(
|
||||
LinComb::builder(2)
|
||||
.bind("weights[0]", Scalar::f64(1.0))
|
||||
.bind("weights[1]", Scalar::f64(1.0)),
|
||||
);
|
||||
let req = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_req));
|
||||
let price = g.source_role("price", ScalarKind::F64);
|
||||
g.feed(
|
||||
price,
|
||||
[fast.input("series"), slow.input("series"), broker.input("price"), exec.input("price")],
|
||||
);
|
||||
g.connect(fast.output("value"), spread.input("lhs"));
|
||||
g.connect(slow.output("value"), spread.input("rhs"));
|
||||
g.connect(spread.output("value"), exposure.input("signal"));
|
||||
// bias fans to: broker (pip), exposure sink, and the RiskExecutor (R).
|
||||
g.connect(exposure.output("bias"), broker.input("exposure"));
|
||||
g.connect(exposure.output("bias"), ex.input("col[0]"));
|
||||
g.connect(exposure.output("bias"), exec.input("bias"));
|
||||
g.connect(broker.output("equity"), eq.input("col[0]"));
|
||||
// tap the dense R-record (all PM fields) for summarize_r.
|
||||
for (i, field) in PM_FIELD_NAMES.iter().enumerate() {
|
||||
g.connect(exec.output(field), rrec.input(COL_PORTS[i].as_str()));
|
||||
}
|
||||
// r_equity sum + tap.
|
||||
g.connect(exec.output("cum_realized_r"), r_equity.input("term[0]"));
|
||||
g.connect(exec.output("unrealized_r"), r_equity.input("term[1]"));
|
||||
g.connect(r_equity.output("value"), req.input("col[0]"));
|
||||
g.build().expect("stage1_r wiring resolves")
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `run_stage1_r` to call the helper**
|
||||
|
||||
In `run_stage1_r` (~1806), change the build line:
|
||||
```rust
|
||||
let flat = stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, Some(2), Some(4))
|
||||
.compile_with_params(&[])
|
||||
.expect("valid stage1-r blueprint");
|
||||
```
|
||||
(was `stage1_r_blueprint(tx_eq, tx_ex, tx_r, tx_req)`.)
|
||||
|
||||
- [ ] **Step 3: Build — confirm no other caller of the old name**
|
||||
|
||||
Run: `cargo build -p aura-cli`
|
||||
Expected: clean (0 errors). A compile error naming `stage1_r_blueprint` means
|
||||
another caller exists — update it to `stage1_r_graph(.., Some(..), Some(..))`.
|
||||
|
||||
- [ ] **Step 4: Run the golden + all stage1-r tests — must stay GREEN (byte-preserved)**
|
||||
|
||||
Run: `cargo test -p aura-cli --test cli_run stage1_r`
|
||||
Expected: PASS — `stage1_r_single_run_output_golden`, `run_harness_stage1_r_*`,
|
||||
`stage1_r_trace_*` all green. A RED golden here means the extraction was NOT
|
||||
byte-preserving → STOP and bounce to `debug` (do not adjust the golden to match).
|
||||
|
||||
---
|
||||
|
||||
### Task 3: `metric_cmp` learns sqn / expectancy_r / net_expectancy_r (aura-registry)
|
||||
|
||||
Self-contained RED→GREEN in `aura-registry`. All three higher-is-better, reaching
|
||||
into `RunReport.metrics.r`; a `None` member sorts to the worst position.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-registry/src/lib.rs:100-104` (`enum Metric`), `:116-135` (`metric_cmp`), `:184-187` (`UnknownMetric` Display), `mod tests` (~200, fixture + tests)
|
||||
|
||||
- [ ] **Step 1: Add the RED tests + an R fixture helper**
|
||||
|
||||
In `mod tests`, extend the `use` and add the helper + four tests:
|
||||
```rust
|
||||
// add to the existing test `use aura_engine::{...}` line:
|
||||
use aura_engine::RMetrics;
|
||||
|
||||
/// A family member carrying an R block — mirrors `report_with` but sets
|
||||
/// `metrics.r = Some(..)`. Only sqn / expectancy_r / net_expectancy_r vary per
|
||||
/// call (the rank keys); the rest are fixed plausible values.
|
||||
fn report_with_r(sqn: f64, expectancy_r: f64, net_expectancy_r: f64) -> RunReport {
|
||||
let mut rep = report_with(0.0, 0.0, 0);
|
||||
rep.metrics.r = Some(RMetrics {
|
||||
expectancy_r,
|
||||
n_trades: 4,
|
||||
win_rate: 0.5,
|
||||
avg_win_r: 1.5,
|
||||
avg_loss_r: -0.5,
|
||||
profit_factor: 3.0,
|
||||
max_r_drawdown: 0.5,
|
||||
n_open_at_end: 0,
|
||||
sqn,
|
||||
net_expectancy_r,
|
||||
conviction_terciles_r: [0.0, 0.0, 0.0],
|
||||
});
|
||||
rep
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rank_by_sqn_orders_members_descending() {
|
||||
let reports = vec![
|
||||
report_with_r(0.5, 0.1, 0.1),
|
||||
report_with_r(2.0, 0.4, 0.4),
|
||||
report_with_r(1.0, 0.2, 0.2),
|
||||
];
|
||||
let ranked = rank_by(reports, "sqn").expect("rank sqn");
|
||||
let sqns: Vec<f64> = ranked.iter().map(|r| r.metrics.r.as_ref().unwrap().sqn).collect();
|
||||
assert_eq!(sqns, vec![2.0, 1.0, 0.5], "sqn must rank highest-first");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rank_by_expectancy_and_net_expectancy_r() {
|
||||
let reports = vec![report_with_r(1.0, 0.1, 0.05), report_with_r(1.0, 0.3, 0.25)];
|
||||
let by_exp = rank_by(reports.clone(), "expectancy_r").expect("rank expectancy_r");
|
||||
assert_eq!(by_exp[0].metrics.r.as_ref().unwrap().expectancy_r, 0.3);
|
||||
let by_net = rank_by(reports, "net_expectancy_r").expect("rank net_expectancy_r");
|
||||
assert_eq!(by_net[0].metrics.r.as_ref().unwrap().net_expectancy_r, 0.25);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rank_r_metric_sorts_none_member_last() {
|
||||
// a pip-only member (r: None) ranks below every member with an R block.
|
||||
let reports = vec![report_with(1.0, 0.5, 1), report_with_r(0.5, 0.1, 0.1)];
|
||||
let ranked = rank_by(reports, "sqn").expect("rank sqn with a None member");
|
||||
assert!(ranked[0].metrics.r.is_some(), "the R member ranks first");
|
||||
assert!(ranked[1].metrics.r.is_none(), "the None member sorts last");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_metric_message_lists_r_metrics() {
|
||||
let msg = RegistryError::UnknownMetric("nope".to_string()).to_string();
|
||||
assert!(msg.contains("sqn"), "msg: {msg}");
|
||||
assert!(msg.contains("expectancy_r"), "msg: {msg}");
|
||||
assert!(msg.contains("net_expectancy_r"), "msg: {msg}");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the new tests — confirm they FAIL**
|
||||
|
||||
Run: `cargo test -p aura-registry rank_by_sqn_orders_members_descending`
|
||||
Expected: FAIL — `rank_by(.., "sqn")` returns `Err(UnknownMetric("sqn"))`, so
|
||||
`.expect("rank sqn")` panics. (The other three fail likewise / on the message.)
|
||||
|
||||
- [ ] **Step 3: Extend `enum Metric`**
|
||||
|
||||
```rust
|
||||
enum Metric {
|
||||
TotalPips,
|
||||
MaxDrawdown,
|
||||
BiasSignFlips,
|
||||
Sqn,
|
||||
ExpectancyR,
|
||||
NetExpectancyR,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Extend `metric_cmp` (string arms + comparator arms + None→-inf)**
|
||||
|
||||
In `metric_cmp`, add the string arms before the `other =>` catch-all:
|
||||
```rust
|
||||
"sqn" => Metric::Sqn,
|
||||
"expectancy_r" => Metric::ExpectancyR,
|
||||
"net_expectancy_r" => Metric::NetExpectancyR,
|
||||
```
|
||||
Add a helper above the returned closure and the three comparator arms (a missing
|
||||
`r` block sorts to the bottom for these higher-is-better metrics):
|
||||
```rust
|
||||
fn r_get(rep: &RunReport, f: impl Fn(&aura_engine::RMetrics) -> f64) -> f64 {
|
||||
rep.metrics.r.as_ref().map(f).unwrap_or(f64::NEG_INFINITY)
|
||||
}
|
||||
Ok(move |a: &RunReport, b: &RunReport| match metric {
|
||||
// ... existing TotalPips / MaxDrawdown / BiasSignFlips arms unchanged ...
|
||||
Metric::Sqn => r_get(b, |r| r.sqn).total_cmp(&r_get(a, |r| r.sqn)),
|
||||
Metric::ExpectancyR => r_get(b, |r| r.expectancy_r).total_cmp(&r_get(a, |r| r.expectancy_r)),
|
||||
Metric::NetExpectancyR => r_get(b, |r| r.net_expectancy_r).total_cmp(&r_get(a, |r| r.net_expectancy_r)),
|
||||
})
|
||||
```
|
||||
(`total_cmp` so `NEG_INFINITY` orders deterministically and no `NaN` can panic. If
|
||||
`RMetrics` is not already in scope in `lib.rs`, use the fully-qualified
|
||||
`aura_engine::RMetrics` as written.)
|
||||
|
||||
- [ ] **Step 5: Extend the `UnknownMetric` Display known-list**
|
||||
|
||||
At `lib.rs:~186`, change the known-list literal:
|
||||
`(known: total_pips, max_drawdown, bias_sign_flips)` →
|
||||
`(known: total_pips, max_drawdown, bias_sign_flips, sqn, expectancy_r, net_expectancy_r)`
|
||||
|
||||
- [ ] **Step 6: Run the registry suite — all GREEN**
|
||||
|
||||
Run: `cargo test -p aura-registry`
|
||||
Expected: PASS (the four new tests + all existing, including
|
||||
`rank_by_orders_best_first_per_metric` and `rank_by_unknown_metric_is_an_error`).
|
||||
|
||||
---
|
||||
|
||||
### Task 4: `stage1_r_sweep_family` + `Strategy::Stage1R` + parse + dispatch
|
||||
|
||||
RED→GREEN. Adds the R sweep family and wires `--strategy stage1-r` end-to-end.
|
||||
Needs Task 2's `stage1_r_graph` and Task 3's `rank sqn`.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-cli/tests/cli_run.rs` (new integration test ~after 1033)
|
||||
- Modify: `crates/aura-cli/src/main.rs:1122-1125` (`enum Strategy`), `:1149` (parse), `:1228-1231` (dispatch), new `stage1_r_sweep_family` (~after 1098), usage strings (1128/1136/1215/1974)
|
||||
|
||||
- [ ] **Step 1: Add the RED integration test**
|
||||
|
||||
Append to `crates/aura-cli/tests/cli_run.rs`:
|
||||
```rust
|
||||
/// Property (#133): `aura sweep --strategy stage1-r` runs the stage1-r harness over
|
||||
/// the fast×slow signal grid (4 members), each carrying an R block, persisted as a
|
||||
/// rankable family; `runs family <id> rank sqn` orders them highest-SQN first.
|
||||
#[test]
|
||||
fn sweep_strategy_stage1_r_ranks_a_family_by_sqn() {
|
||||
let cwd = temp_cwd("sweep-stage1-r");
|
||||
let out = Command::new(BIN)
|
||||
.args(["sweep", "--strategy", "stage1-r"])
|
||||
.current_dir(&cwd)
|
||||
.output()
|
||||
.expect("spawn aura sweep --strategy stage1-r");
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"stage1-r sweep exit: {:?}; stderr: {}",
|
||||
out.status,
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
);
|
||||
let stdout = String::from_utf8(out.stdout).expect("utf-8");
|
||||
let lines: Vec<&str> = stdout.lines().collect();
|
||||
assert_eq!(lines.len(), 4, "stage1-r sweep must print 4 member lines: {stdout:?}");
|
||||
for line in &lines {
|
||||
assert!(line.contains("\"r\":{"), "member must carry an r block: {line}");
|
||||
assert!(line.contains("\"sqn\""), "member r block must carry sqn: {line}");
|
||||
}
|
||||
// rank the family by sqn — highest first.
|
||||
let rank = Command::new(BIN)
|
||||
.args(["runs", "family", "sweep-0", "rank", "sqn"])
|
||||
.current_dir(&cwd)
|
||||
.output()
|
||||
.expect("spawn rank sqn");
|
||||
assert!(rank.status.success(), "rank sqn exit: {:?}", rank.status);
|
||||
let rank_out = String::from_utf8(rank.stdout).expect("utf-8");
|
||||
assert_eq!(rank_out.lines().count(), 4, "rank: {rank_out:?}");
|
||||
let sqns: Vec<f64> = rank_out
|
||||
.lines()
|
||||
.map(|l| {
|
||||
let key = "\"sqn\":";
|
||||
let start = l.find(key).expect("line has sqn") + key.len();
|
||||
let tail = &l[start..];
|
||||
let end = tail.find([',', '}']).expect("token end");
|
||||
tail[..end].parse().expect("sqn is an f64")
|
||||
})
|
||||
.collect();
|
||||
assert!(sqns.windows(2).all(|w| w[0] >= w[1]), "rank not descending by sqn: {sqns:?}");
|
||||
// an unknown metric is a usage error (exit 2).
|
||||
let bogus = Command::new(BIN)
|
||||
.args(["runs", "family", "sweep-0", "rank", "bogus"])
|
||||
.current_dir(&cwd)
|
||||
.output()
|
||||
.expect("spawn bogus");
|
||||
assert_eq!(bogus.status.code(), Some(2), "bogus exit: {:?}", bogus.status);
|
||||
let _ = std::fs::remove_dir_all(&cwd);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run it — confirm it FAILS**
|
||||
|
||||
Run: `cargo test -p aura-cli --test cli_run sweep_strategy_stage1_r_ranks_a_family_by_sqn`
|
||||
Expected: FAIL — `--strategy stage1-r` is rejected by `parse_sweep_args` (exit 2),
|
||||
so `out.status.success()` is false. (Build is clean; this is a runtime failure.)
|
||||
|
||||
- [ ] **Step 3: Add `Strategy::Stage1R`**
|
||||
|
||||
```rust
|
||||
enum Strategy {
|
||||
SmaCross,
|
||||
Momentum,
|
||||
Stage1R,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Accept `--strategy stage1-r` in `parse_sweep_args`**
|
||||
|
||||
In the `--strategy` value `match *value` (~1149), add before the `_ =>` arm:
|
||||
```rust
|
||||
"stage1-r" => Strategy::Stage1R,
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Dispatch in `run_sweep`**
|
||||
|
||||
In the `match strategy` (~1228), add the arm:
|
||||
```rust
|
||||
Strategy::Stage1R => stage1_r_sweep_family(persist.then_some(name), &data),
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Add `stage1_r_sweep_family` (mirror `momentum_sweep_family`)**
|
||||
|
||||
Insert after `momentum_sweep_family` (~1098):
|
||||
```rust
|
||||
/// `aura sweep --strategy stage1-r`: sweep the stage1-r harness over a fast×slow
|
||||
/// SIGNAL grid (the stop + sizing stay fixed — see the cycle-0066 / #133 decision
|
||||
/// log: risk_budget is R-invariant and bias.scale is sign-only under flat-1R, both
|
||||
/// degenerate axes; the stop defines the R unit, so varying it would break
|
||||
/// cross-member SQN comparability). Each member folds the dense R-record via
|
||||
/// summarize_r, so its RunReport carries `r: Some(..)` and the family is rankable
|
||||
/// by sqn / expectancy_r / net_expectancy_r. Per-member tracing is a follow-up; the
|
||||
/// `_trace` slot keeps the dispatch shape uniform with the other sweeps.
|
||||
fn stage1_r_sweep_family(_trace: Option<&str>, data: &DataSource) -> SweepFamily {
|
||||
let pip = data.pip_size();
|
||||
let window = data.full_window();
|
||||
// a throwaway floated build, only to resolve param_space + the named axes
|
||||
// (its taps are never drained — the per-point run_one rebuilds with live ones).
|
||||
let floated = || {
|
||||
let (tx_eq, _) = mpsc::channel();
|
||||
let (tx_ex, _) = mpsc::channel();
|
||||
let (tx_r, _) = mpsc::channel();
|
||||
let (tx_req, _) = mpsc::channel();
|
||||
stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None)
|
||||
};
|
||||
let space = floated().param_space();
|
||||
floated()
|
||||
.axis("fast.length", [2, 3])
|
||||
.axis("slow.length", [6, 12])
|
||||
.sweep(|point| {
|
||||
let (tx_eq, rx_eq) = mpsc::channel();
|
||||
let (tx_ex, rx_ex) = mpsc::channel();
|
||||
let (tx_r, rx_r) = mpsc::channel();
|
||||
let (tx_req, _rx_req) = mpsc::channel();
|
||||
let mut h = stage1_r_graph(tx_eq, tx_ex, tx_r, tx_req, None, None)
|
||||
.bootstrap_with_cells(point)
|
||||
.expect("stage1-r grid points are kind-checked against param_space");
|
||||
h.run(data.run_sources());
|
||||
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
|
||||
let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
|
||||
let r_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_r.try_iter().collect();
|
||||
let named = zip_params(&space, point);
|
||||
let manifest = sim_optimal_manifest(named, window, 0, pip);
|
||||
let mut metrics = summarize(&f64_field(&eq_rows, 0), &f64_field(&ex_rows, 0));
|
||||
metrics.r = Some(summarize_r(&r_rows, 0.0));
|
||||
RunReport { manifest, metrics }
|
||||
})
|
||||
.expect("the stage1-r named grid matches the stage1-r param-space")
|
||||
}
|
||||
```
|
||||
Note: the `.axis(...)` calls must cover EVERY free knob in `param_space`. With only
|
||||
`fast`/`slow` floated, `param_space` is exactly `{fast.length, slow.length}`. If
|
||||
`.sweep(...)` panics on an unbound knob, an extra knob is free — inspect
|
||||
`floated().param_space()` and either bind it in `stage1_r_graph` or add a
|
||||
single-value axis. (Expected: exactly the two axes.)
|
||||
|
||||
- [ ] **Step 7: Update the sweep usage strings to list stage1-r**
|
||||
|
||||
At each `<sma|momentum>` in the sweep usage/doc text (`main.rs:1128`, `:1136` the
|
||||
`usage` closure, `:1215` doc, `:1974` the top-level usage literal), change to
|
||||
`<sma|momentum|stage1-r>`.
|
||||
|
||||
- [ ] **Step 8: Build + run the integration test — GREEN**
|
||||
|
||||
Run: `cargo build -p aura-cli`
|
||||
Expected: clean (the `run_sweep` match is now exhaustive).
|
||||
Run: `cargo test -p aura-cli --test cli_run sweep_strategy_stage1_r_ranks_a_family_by_sqn`
|
||||
Expected: PASS — 4 members each with an `r` block, ranked descending by sqn,
|
||||
unknown metric exits 2.
|
||||
|
||||
- [ ] **Step 9: Full workspace gate**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
Expected: PASS (no regressions; the new tests green).
|
||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: clean.
|
||||
|
||||
---
|
||||
|
||||
## Self-review
|
||||
|
||||
1. **Spec coverage:** Half A (metric_cmp R metrics + None-worst + extended message)
|
||||
= Task 3. Half B (stage1-r sweep family + --strategy + dispatch) = Tasks 2+4.
|
||||
Golden (byte-preservation guard) = Task 1. Acceptance #1–#5 all covered.
|
||||
2. **Placeholder scan:** the only deferred literal is the golden's captured metrics
|
||||
string (Task 1 Step 1→2) — a characterization value that is *by definition*
|
||||
captured from the running binary, with an exact capture command + paste slot.
|
||||
This is the legitimate golden-capture pattern, not a TBD; flagged here so it is
|
||||
not mistaken for one. No other placeholders.
|
||||
3. **Type consistency:** `stage1_r_graph` (not `stage1_r_blueprint`) used in both
|
||||
Task 2 and Task 4; `Strategy::Stage1R`, `report_with_r`, `r_get`, `RMetrics`
|
||||
consistent across tasks.
|
||||
4. **Step granularity:** each step is one action (one edit / one run).
|
||||
5. **No commit steps:** none — the orchestrator commits the iter.
|
||||
6. **Pin/replacement contiguity:** the golden pins the `metrics` substring captured
|
||||
from real output (contiguous by construction); the integration test extracts
|
||||
`"sqn":` tokens (no soft-wrap pin). OK.
|
||||
7. **Compile-gate ordering:** Task 2's signature change (`stage1_r_blueprint` →
|
||||
`stage1_r_graph` + 2 params) threads its one caller (`run_stage1_r`) in the same
|
||||
task before the build gate (Step 3). Task 4's `Strategy::Stage1R` threads both
|
||||
the dispatch arm (compiler-flagged) and the parse arm (silent) before the build
|
||||
gate (Step 8). No deferred caller past a build gate.
|
||||
8. **Verification-filter strings resolve:** `stage1_r_single_run_output_golden`,
|
||||
`rank_by_sqn_orders_members_descending`, `sweep_strategy_stage1_r_ranks_a_family_by_sqn`
|
||||
are the exact new test names; `stage1_r` / `-p aura-registry` are real targets;
|
||||
the final gates are unfiltered `--workspace`. One positional filter per `cargo
|
||||
test` (project memory). OK.
|
||||
@@ -1,283 +0,0 @@
|
||||
# R-sweep families + rank-by-SQN — Design Spec
|
||||
|
||||
**Date:** 2026-06-24
|
||||
**Status:** Draft — awaiting user spec review
|
||||
**Authors:** orchestrator + Claude
|
||||
**Cycle:** 0066 · **Reference issue:** #133 (decision log) · **Contract:** C12 (axes 1+2), C18 (family store), C10 (R)
|
||||
|
||||
## Goal
|
||||
|
||||
Make SQN the operational single-number objective for ranking a Stage-1 sweep
|
||||
family by R signal quality. Two halves, both deferred from cycle 0065:
|
||||
|
||||
1. **An R sweep family** — `aura sweep --strategy stage1-r` runs the stage1-r
|
||||
harness over a signal-param grid and persists each member's full `RunReport`
|
||||
(including the `r: Some(RMetrics)` block) into the `families.jsonl` store.
|
||||
2. **R-aware ranking** — `aura runs family <id> rank <metric>` learns three R
|
||||
metrics — `sqn`, `expectancy_r`, `net_expectancy_r` (all higher-is-better) —
|
||||
so a family is ranked best-first by signal quality.
|
||||
|
||||
This realizes C12 axis-2 (argmax-metric) over the C18 family store for the R
|
||||
yardstick. It is **not** a new mechanism: the store already persists the `r`
|
||||
block; the sweep builder is already harness-agnostic; only the CLI harness
|
||||
selection and the `metric_cmp` vocabulary need extending.
|
||||
|
||||
## Non-goals (explicit scope fence)
|
||||
|
||||
- **No new grid syntax.** The grid is a hardcoded default like the existing
|
||||
`sweep_family` / `momentum_sweep_family`. CLI-driven custom grids are a
|
||||
separate feature; no existing sweep has them.
|
||||
- **No sweep of the stop or sizing.** `bias.scale`, `risk_budget`, and the
|
||||
vol-stop `length`/`k` are held fixed — see the reference-issue decision log
|
||||
(degenerate axes for the ranked R metric; varying the stop changes what R
|
||||
means and motivates the deferred SQN100 #130).
|
||||
- **No `--harness`/`--strategy` flag unification.** The sweep keeps its own
|
||||
`--strategy` grammar; unifying it with `run --harness` is an orthogonal
|
||||
pre-existing CLI divergence, a candidate follow-up `idea`.
|
||||
- **No n-normalized SQN100** (#130, sibling) and **no friction** (`round_trip_cost`
|
||||
stays 0.0 — Stage-1 is frictionless; `net_expectancy_r` therefore equals
|
||||
`expectancy_r` on a stage1-r family, but the metric is wired so #120's friction
|
||||
lights it up unchanged).
|
||||
|
||||
## Architecture
|
||||
|
||||
Two crates change, independently:
|
||||
|
||||
- **`aura-registry`** (`src/lib.rs`) — `metric_cmp` gains three R-metric arms
|
||||
that reach into `RunReport.metrics.r: Option<RMetrics>`; a `None` member sorts
|
||||
to the worst position (treated as `f64::NEG_INFINITY` for the higher-is-better
|
||||
R metrics). The `UnknownMetric` "known: …" message is extended. No persistence
|
||||
change — the store already carries `r`.
|
||||
- **`aura-cli`** (`src/main.rs`) — `enum Strategy` gains `Stage1R`; the sweep
|
||||
argument parser accepts `--strategy stage1-r`; `run_sweep` dispatches to a new
|
||||
`stage1_r_sweep_family`. The stage1-r topology is extracted into a shared helper
|
||||
parameterized by the signal knobs: the **single run keeps binding** `fast=2` /
|
||||
`slow=4` (the same `.bind(...)` calls, now routed through the helper — its code
|
||||
path and recorded output are unchanged *by construction*), while the **sweep
|
||||
floats** `fast.length` / `slow.length` so they appear in `param_space` and are
|
||||
varied via `.axis(...)`. A new **golden characterization test** of the single-run
|
||||
output is added and committed against current HEAD *first*, so the refactor is
|
||||
*verified* byte-preserving (the grounding-check correctly noted no such golden
|
||||
exists today — adding it closes a real coverage gap).
|
||||
|
||||
The sweep-member run path, the `FamilyRunRecord` shape, `append_family`,
|
||||
`load_family_members`, `group_families`, and the `runs family … rank …` CLI
|
||||
routing are all **unchanged** — they already flow full `RunReport`s and route any
|
||||
metric string through `metric_cmp`.
|
||||
|
||||
## Concrete code shapes
|
||||
|
||||
### User-facing program (the acceptance evidence)
|
||||
|
||||
```console
|
||||
$ aura sweep --strategy stage1-r --name r-sweep
|
||||
# runs the stage1-r harness over the fast×slow signal grid (4 members),
|
||||
# persists 4 FamilyRunRecord lines to runs/families.jsonl, each with r: Some(..)
|
||||
|
||||
$ aura runs family r-sweep-0 rank sqn
|
||||
{"manifest":{...,"params":{"sma_fast":2,"sma_slow":6,...}},"metrics":{"total_pips":..,"r":{"sqn":1.83,"expectancy_r":0.42,...}}}
|
||||
{"manifest":{...,"params":{"sma_fast":3,"sma_slow":12,...}},"metrics":{...,"r":{"sqn":1.10,...}}}
|
||||
{"manifest":{...},"metrics":{...,"r":{"sqn":0.50,...}}}
|
||||
{"manifest":{...},"metrics":{...,"r":{"sqn":-0.20,...}}}
|
||||
# members printed best-first by SQN (descending). expectancy_r / net_expectancy_r likewise.
|
||||
|
||||
$ aura runs family r-sweep-0 rank wibble
|
||||
aura: unknown metric 'wibble' (known: total_pips, max_drawdown, bias_sign_flips, sqn, expectancy_r, net_expectancy_r)
|
||||
# exit 2 — vocabulary now lists the R metrics
|
||||
```
|
||||
|
||||
### `metric_cmp` — before → after (`aura-registry/src/lib.rs:99-135`)
|
||||
|
||||
Before: `enum Metric { TotalPips, BiasSignFlips, MaxDrawdown }`, all reading
|
||||
top-level `a.metrics.*`. After:
|
||||
|
||||
```rust
|
||||
enum Metric { TotalPips, BiasSignFlips, MaxDrawdown, Sqn, ExpectancyR, NetExpectancyR }
|
||||
|
||||
fn metric_cmp(metric: &str)
|
||||
-> Result<impl Fn(&RunReport, &RunReport) -> Ordering, RegistryError>
|
||||
{
|
||||
let metric = match metric {
|
||||
"total_pips" => Metric::TotalPips,
|
||||
"max_drawdown" => Metric::MaxDrawdown,
|
||||
"bias_sign_flips" | "exposure_sign_flips" => Metric::BiasSignFlips,
|
||||
"sqn" => Metric::Sqn,
|
||||
"expectancy_r" => Metric::ExpectancyR,
|
||||
"net_expectancy_r" => Metric::NetExpectancyR,
|
||||
other => return Err(RegistryError::UnknownMetric(other.to_string())),
|
||||
};
|
||||
// higher-is-better R metrics: a missing r block sorts to the bottom (-inf).
|
||||
fn r_get(rep: &RunReport, f: impl Fn(&RMetrics) -> f64) -> f64 {
|
||||
rep.metrics.r.as_ref().map(f).unwrap_or(f64::NEG_INFINITY)
|
||||
}
|
||||
Ok(move |a: &RunReport, b: &RunReport| match metric {
|
||||
Metric::TotalPips => b.metrics.total_pips.partial_cmp(&a.metrics.total_pips).unwrap(),
|
||||
Metric::MaxDrawdown => a.metrics.max_drawdown.partial_cmp(&b.metrics.max_drawdown).unwrap(),
|
||||
Metric::BiasSignFlips => a.metrics.bias_sign_flips.cmp(&b.metrics.bias_sign_flips),
|
||||
Metric::Sqn => r_get(b, |r| r.sqn).total_cmp(&r_get(a, |r| r.sqn)),
|
||||
Metric::ExpectancyR => r_get(b, |r| r.expectancy_r).total_cmp(&r_get(a, |r| r.expectancy_r)),
|
||||
Metric::NetExpectancyR => r_get(b, |r| r.net_expectancy_r).total_cmp(&r_get(a, |r| r.net_expectancy_r)),
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
`total_cmp` (not `partial_cmp().unwrap()`) for the R metrics so `NEG_INFINITY`
|
||||
orders deterministically and a stray `NaN` cannot panic. The `UnknownMetric`
|
||||
Display string gains `, sqn, expectancy_r, net_expectancy_r`.
|
||||
|
||||
### stage1-r blueprint — extract a shared helper, float the signal knobs only for the sweep (`aura-cli/src/main.rs:1745-1746, 1806-1808`)
|
||||
|
||||
Today `stage1_r_blueprint` binds `fast`/`slow` at build, and the single run
|
||||
compiles with an empty point:
|
||||
|
||||
```rust
|
||||
let fast = g.add(Sma::builder().named("fast").bind("length", Scalar::i64(2)));
|
||||
let slow = g.add(Sma::builder().named("slow").bind("length", Scalar::i64(4)));
|
||||
// ...
|
||||
let flat = stage1_r_blueprint(..).compile_with_params(&[]).expect(..);
|
||||
```
|
||||
|
||||
After: the topology moves into one helper; the ONLY difference between the
|
||||
single-run and the sweep build is whether the two signal knobs are bound or left
|
||||
free — one wiring source (no drift), and the single-run path keeps the *identical*
|
||||
`.bind(...)` calls (so its output is byte-unchanged by construction):
|
||||
|
||||
```rust
|
||||
// shared wiring; a None length floats the knob (it lands in param_space), a
|
||||
// Some(len) binds it at build exactly as today.
|
||||
fn stage1_r_graph(.., fast_len: Option<i64>, slow_len: Option<i64>) -> Composite {
|
||||
let mut fast_b = Sma::builder().named("fast");
|
||||
if let Some(l) = fast_len { fast_b = fast_b.bind("length", Scalar::i64(l)); }
|
||||
let fast = g.add(fast_b);
|
||||
// …slow likewise; the rest of the topology is verbatim from today's body…
|
||||
}
|
||||
|
||||
// single run — binds the same constants; code path + output byte-identical to today:
|
||||
let flat = stage1_r_graph(.., Some(2), Some(4)).compile_with_params(&[]).expect(..);
|
||||
|
||||
// sweep — floats them, binds per grid point via .axis(...):
|
||||
let bp = stage1_r_graph(.., None, None); // fast.length / slow.length now in param_space
|
||||
bp.axis("fast.length", [2, 3]).axis("slow.length", [6, 12]).sweep(run_one)
|
||||
```
|
||||
|
||||
(`bias.scale`, the `risk_executor` stop and `risk_budget` stay bound inside their
|
||||
builders — only `fast`/`slow` lengths are made floatable. The exact helper
|
||||
signature is the planner's to pin against the real `stage1_r_blueprint` body.)
|
||||
|
||||
### stage1-r sweep family — new, mirrors `momentum_sweep_family` (`aura-cli/src/main.rs`)
|
||||
|
||||
```rust
|
||||
enum Strategy { SmaCross, Momentum, Stage1R } // + Stage1R
|
||||
|
||||
// in run_sweep dispatch:
|
||||
let family = match strategy {
|
||||
Strategy::SmaCross => sweep_family(persist.then_some(name), &data),
|
||||
Strategy::Momentum => momentum_sweep_family(persist.then_some(name), &data),
|
||||
Strategy::Stage1R => stage1_r_sweep_family(persist.then_some(name), &data),
|
||||
};
|
||||
|
||||
fn stage1_r_sweep_family(trace: Option<&str>, data: &DataSource) -> SweepFamily {
|
||||
// build the stage1-r blueprint with fast/slow floated; bind a 4-point grid:
|
||||
// fast.length ∈ {2, 3} × slow.length ∈ {6, 12} (fast < slow always)
|
||||
// run_one bootstraps each point, drains the four taps, folds RunMetrics with
|
||||
// metrics.r = Some(summarize_r(&r_rows, 0.0)) // same fold as run_stage1_r
|
||||
// returns a SweepFamily of 4 members, each report carrying r: Some(..).
|
||||
}
|
||||
```
|
||||
|
||||
`--strategy stage1-r` is accepted by `parse_sweep_args` (it already tokenizes
|
||||
`--strategy <v>`; the value match gains a `stage1-r => Strategy::Stage1R` arm).
|
||||
|
||||
## Components
|
||||
|
||||
| Component | File | Change |
|
||||
|---|---|---|
|
||||
| `Metric` enum + `metric_cmp` | `aura-registry/src/lib.rs` | +3 variants, +3 string arms, +3 comparator arms with `None`→worst; extend `UnknownMetric` Display |
|
||||
| `Strategy` enum | `aura-cli/src/main.rs` | +`Stage1R` |
|
||||
| `parse_sweep_args` | `aura-cli/src/main.rs` | accept `--strategy stage1-r` |
|
||||
| `run_sweep` dispatch | `aura-cli/src/main.rs` | +`Strategy::Stage1R` arm |
|
||||
| `stage1_r_blueprint` → `stage1_r_graph` helper | `aura-cli/src/main.rs` | extract shared wiring; `fast`/`slow` lengths floatable via `Option` args |
|
||||
| `run_stage1_r` | `aura-cli/src/main.rs` | call the helper with `Some(2)`/`Some(4)` — binding + output unchanged |
|
||||
| golden char. test (new) | `aura-cli` (`tests/cli_run.rs` or unit) | pin current stage1-r single-run output against HEAD *before* the refactor |
|
||||
| `stage1_r_sweep_family` (new) | `aura-cli/src/main.rs` | the 4-point R sweep, folding `r: Some(..)` per member |
|
||||
|
||||
## Data flow
|
||||
|
||||
`aura sweep --strategy stage1-r --name r-sweep`
|
||||
→ `stage1_r_sweep_family` builds the floated blueprint, `.axis("fast.length",[2,3]).axis("slow.length",[6,12]).sweep(run_one)`
|
||||
→ each `run_one(point)` bootstraps a disjoint sim (C1), drains equity/exposure/R-record/r-equity taps, folds a `RunReport { manifest, metrics{ r: Some(summarize_r(..)) } }`
|
||||
→ `Registry::append_family("r-sweep", Sweep, reports)` writes 4 `FamilyRunRecord` lines to `runs/families.jsonl`.
|
||||
|
||||
`aura runs family r-sweep-0 rank sqn`
|
||||
→ `load_family_members` → `group_families` → find `r-sweep-0`
|
||||
→ `rank_by(reports, "sqn")` → `metric_cmp("sqn")` → sort descending by `r.sqn` (None→-inf)
|
||||
→ print each `RunReport` JSON line best-first.
|
||||
|
||||
## Error handling
|
||||
|
||||
- **Unknown metric** — unchanged path: `RegistryError::UnknownMetric`, `eprintln!`,
|
||||
`exit(2)`; the "known:" list now includes the three R metrics.
|
||||
- **Ranking a pip-only family by an R metric** — every member is `r: None` → all
|
||||
`NEG_INFINITY` → stable tie → ordinal order. No error (documented, by the
|
||||
None-sorts-worst decision).
|
||||
- **Mixed Some/None** (not reachable from a homogeneous family today, but defined):
|
||||
`Some` members rank above `None` members.
|
||||
- **Empty / degenerate R block** — `summarize_r` already returns `0.0` for
|
||||
`sqn`/`expectancy_r` on `n<2`/empty (cycle 0065); a member with no trades has a
|
||||
well-defined `r: Some(RMetrics{ sqn: 0.0, .. })` and ranks among the others.
|
||||
- **`fast >= slow`** — excluded by grid construction (all points keep fast<slow),
|
||||
so no inverted-cross degenerate member is produced.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
RED-first for all new behaviour.
|
||||
|
||||
**`aura-registry` unit tests (`src/lib.rs`):**
|
||||
- `rank_by_sqn_orders_members_descending` — a hand-built `Vec<RunReport>` with
|
||||
distinct `r.sqn` ranks highest-SQN first.
|
||||
- `rank_by_expectancy_r` and `rank_by_net_expectancy_r` — likewise.
|
||||
- `rank_r_metric_sorts_none_member_last` — a mix of `r: Some` and `r: None` ranks
|
||||
every `Some` above every `None`.
|
||||
- `rank_pip_only_family_by_r_metric_is_ordinal` — all-`None` reports keep input
|
||||
order (stable tie).
|
||||
- `unknown_metric_message_lists_r_metrics` — the `UnknownMetric` Display contains
|
||||
`sqn`, `expectancy_r`, `net_expectancy_r`.
|
||||
|
||||
**`aura-cli` tests:**
|
||||
- `stage1_r_single_run_output_golden` (added FIRST, before the refactor) — a
|
||||
characterization test pinning the EXACT current stage1-r single-run output (the
|
||||
emitted `RunReport` JSON, including the `r` block) as a committed golden literal,
|
||||
captured against current HEAD so it is GREEN immediately. The helper extraction
|
||||
must keep it green; a drift means the change was not behaviour-preserving →
|
||||
bounce to `debug`. This is the guard the grounding-check found missing.
|
||||
- `stage1_r_sweep_family_members_carry_r` — `stage1_r_sweep_family` returns 4
|
||||
members, each `report.metrics.r.is_some()`, and `fast<slow` on every member.
|
||||
- Integration (`tests/cli_run.rs`, via `Command::new(env!("CARGO_BIN_EXE_aura"))`):
|
||||
`aura sweep --strategy stage1-r --name <tmp>` then
|
||||
`aura runs family <tmp>-0 rank sqn` → exit 0, 4 JSON lines, SQN non-increasing
|
||||
down the output; an unknown metric → exit 2 with the extended message.
|
||||
- Determinism: re-running the same sweep yields byte-identical member reports (C1).
|
||||
|
||||
Commands: build `cargo build --workspace`; test `cargo test --workspace` (ONE
|
||||
positional filter substring per invocation); lint
|
||||
`cargo clippy --workspace --all-targets -- -D warnings`.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
1. `aura sweep --strategy stage1-r --name N` writes a 4-member family to
|
||||
`families.jsonl`, every member carrying `r: Some(RMetrics)`.
|
||||
2. `aura runs family N-0 rank sqn` (and `expectancy_r`, `net_expectancy_r`) prints
|
||||
members best-first by that metric; an unknown metric exits 2 with the extended
|
||||
"known:" list.
|
||||
3. The stage1-r single-run output is byte-unchanged — *verified* by a golden
|
||||
characterization test added against current HEAD before the refactor and kept
|
||||
green through it (not assumed; a drift bounces to `debug`).
|
||||
4. A pip-only (SMA/momentum) family ranked by an R metric degrades to ordinal
|
||||
order without error.
|
||||
5. Full workspace suite green, clippy `-D warnings` clean; the sweep is
|
||||
deterministic and reproducible (C1).
|
||||
|
||||
This serves the aura research loop directly (C12 axis-2 on the C18 family store
|
||||
for the R yardstick), measurably improves the research surface (the first
|
||||
R-based family ranking), and reintroduces no determinism/causality violation
|
||||
(members are disjoint deterministic sims).
|
||||
Reference in New Issue
Block a user