Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

R-sweep families + rank-by-sqn (SQN as the sweep objective) #133

New Issue
Closed
opened 2026-06-24 12:34:33 +02:00 by Brummel · 3 comments
Brummel commented 2026-06-24 12:34:33 +02:00
Owner
Copy Link
Copy Source

Deferred from cycle 0065 (the Stage-1 R single-run CLI surface, #129). SQN is documented as the dispersion-adjusted sweep objective, but no sweep currently produces R metrics or ranks by them.

Scope:

  • an R sweep-family builder (the stage1-r harness over a param grid);
  • extend runs family rank <metric> (metric_cmp in aura-registry) to understand the R metrics — sqn (higher-is-better), expectancy_r, net_expectancy_r.

This realizes SQN's role as the single-number objective for ranking a sweep family by signal quality. Larger than the single-run cut: touches the sweep builders + the registry rank vocabulary. Sibling of the n-normalized SQN idea (#130).

Deferred from cycle 0065 (the Stage-1 R single-run CLI surface, #129). SQN is documented as the dispersion-adjusted **sweep objective**, but no sweep currently produces R metrics or ranks by them. Scope: - an R sweep-family builder (the `stage1-r` harness over a param grid); - extend `runs family rank <metric>` (`metric_cmp` in `aura-registry`) to understand the R metrics — `sqn` (higher-is-better), `expectancy_r`, `net_expectancy_r`. This realizes SQN's role as the single-number objective for ranking a sweep family by signal quality. Larger than the single-run cut: touches the sweep builders + the registry rank vocabulary. Sibling of the n-normalized SQN idea (#130).
Brummel added the feature label 2026-06-24 12:34:33 +02:00
Brummel commented 2026-06-24 15:03:58 +02:00
Author
Owner
Copy Link
Copy Source

Design reconciliation (specify, cycle 0066) — /boss, signed below by the grounding-check

Spec: docs/specs/0066-r-sweep-rank.md. The forks below were derived from the
sources, the code, and the existing design; this records them on the reference
issue so the call made without a human in the loop is auditable and vetoable.

  • Fork: which knobs the stage1-r sweep grid varies -> the SIGNAL params
    (fast.length x slow.length); the stop and sizing stay fixed; bias.scale
    is also excluded.
    Basis (derived): risk_budget is R-invariant by construction (#128 — scaling
    the budget leaves every realized_r unchanged), and under flat-1R the R outcomes
    depend on the bias SIGN (direction/presence), not its magnitude, so bias.scale
    is degenerate for the ranked metric too — an axis that cannot move SQN/expectancy
    is a non-axis. The stop DEFINES the R unit (1R = the loss if stopped); varying it
    changes what R means per member and changes n_trades, making cross-member raw
    SQN an apples-to-oranges comparison — which is exactly the motivation for the
    deferred n-normalized SQN100 (#130). Holding the stop (the measurement yardstick)
    fixed and varying the signal makes SQN a coherent single-number signal-quality
    objective, the issue's stated purpose.

  • Fork: ranking a member whose r block is absent (RunMetrics.r == None)
    -> it sorts to the WORST position for an R metric (treated as -inf for the
    higher-is-better R metrics).
    Basis (derived): a member with no R block has no R signal quality, so ranking it
    below every member that does is the correct semantic; it also collapses harmlessly
    to ordinal order for a homogeneous all-None family (a pip-only SMA/momentum family
    ranked by an R metric), avoiding a new error path for a degenerate-but-harmless
    case. A stage1-r family is homogeneous r: Some; the None case is the cross-family
    guard, not the common path.

  • Fork: the CLI surface for sweeping stage1-r -> extend the sweep's own
    --strategy grammar with the value stage1-r; do NOT unify it with the run
    subcommand's --harness flag.
    Basis (derived): the sweep already selects its harness via --strategy {sma, momentum} (a different flag and a different value-set than run --harness {sma, macd,stage1-r} — a pre-existing, orthogonal divergence). The minimal coherent
    extension is one more --strategy value; unifying the two flag namespaces would
    touch the SMA/momentum paths and is its own CLI-UX change, out of scope here
    (a candidate follow-up idea).

  • Fork: how the swept knobs are floated without breaking the single run
    -> one blueprint with fast.length/slow.length left free in param_space; the
    single run binds the existing constants (fast=2, slow=4) via a compile-time param
    point (byte-output preserved, enforced by the single-run byte-pins); the sweep
    varies them via .axis(...).
    Basis (derived): a single wiring source avoids a drifting duplicate blueprint, and
    it mirrors the established momentum harness pattern (one *_blueprint_with_sinks,
    bound by params for the single run, by axes for the sweep).

  • Settled (issue + domain), not a fork: the three R metrics added to the rank
    vocabulary — sqn, expectancy_r, net_expectancy_r — are all higher-is-better.
    Contract fit: ranking a family by a metric is C12 axis-2 (argmax-metric) applied
    to the C18 family store (families.jsonl), which already persists the full
    RunReport incl. the r block — so no persistence change is needed, only the
    metric_cmp vocabulary reaching into r.

Cycle number 0066 (cycle 0065 closed; its spec/plan were git rm'd).

## Design reconciliation (specify, cycle 0066) — /boss, signed below by the grounding-check Spec: `docs/specs/0066-r-sweep-rank.md`. The forks below were derived from the sources, the code, and the existing design; this records them on the reference issue so the call made without a human in the loop is auditable and vetoable. - **Fork: which knobs the stage1-r sweep grid varies** -> the SIGNAL params (`fast.length` x `slow.length`); the stop and sizing stay fixed; `bias.scale` is also excluded. Basis (derived): `risk_budget` is R-invariant by construction (#128 — scaling the budget leaves every realized_r unchanged), and under flat-1R the R outcomes depend on the bias SIGN (direction/presence), not its magnitude, so `bias.scale` is degenerate for the ranked metric too — an axis that cannot move SQN/expectancy is a non-axis. The stop DEFINES the R unit (1R = the loss if stopped); varying it changes what R *means* per member and changes `n_trades`, making cross-member raw SQN an apples-to-oranges comparison — which is exactly the motivation for the deferred n-normalized SQN100 (#130). Holding the stop (the measurement yardstick) fixed and varying the signal makes SQN a coherent single-number signal-quality objective, the issue's stated purpose. - **Fork: ranking a member whose `r` block is absent (`RunMetrics.r == None`)** -> it sorts to the WORST position for an R metric (treated as -inf for the higher-is-better R metrics). Basis (derived): a member with no R block has no R signal quality, so ranking it below every member that does is the correct semantic; it also collapses harmlessly to ordinal order for a homogeneous all-None family (a pip-only SMA/momentum family ranked by an R metric), avoiding a new error path for a degenerate-but-harmless case. A stage1-r family is homogeneous `r: Some`; the None case is the cross-family guard, not the common path. - **Fork: the CLI surface for sweeping stage1-r** -> extend the sweep's own `--strategy` grammar with the value `stage1-r`; do NOT unify it with the `run` subcommand's `--harness` flag. Basis (derived): the sweep already selects its harness via `--strategy {sma, momentum}` (a different flag and a different value-set than `run --harness {sma, macd,stage1-r}` — a pre-existing, orthogonal divergence). The minimal coherent extension is one more `--strategy` value; unifying the two flag namespaces would touch the SMA/momentum paths and is its own CLI-UX change, out of scope here (a candidate follow-up `idea`). - **Fork: how the swept knobs are floated without breaking the single run** -> one blueprint with `fast.length`/`slow.length` left free in `param_space`; the single run binds the existing constants (fast=2, slow=4) via a compile-time param point (byte-output preserved, enforced by the single-run byte-pins); the sweep varies them via `.axis(...)`. Basis (derived): a single wiring source avoids a drifting duplicate blueprint, and it mirrors the established momentum harness pattern (one `*_blueprint_with_sinks`, bound by params for the single run, by axes for the sweep). - **Settled (issue + domain), not a fork:** the three R metrics added to the rank vocabulary — `sqn`, `expectancy_r`, `net_expectancy_r` — are all higher-is-better. Contract fit: ranking a family by a metric is C12 axis-2 (argmax-metric) applied to the C18 family store (`families.jsonl`), which already persists the full `RunReport` incl. the `r` block — so no persistence change is needed, only the `metric_cmp` vocabulary reaching into `r`. Cycle number 0066 (cycle 0065 closed; its spec/plan were `git rm`'d).
Brummel commented 2026-06-24 15:14:31 +02:00
Author
Owner
Copy Link
Copy Source

Grounding revision (specify, /boss auto-sign)

The first grounding-check BLOCKed one assumption: the spec claimed the stage1-r
single-run output was already byte-pinned and the blueprint refactor thereby
byte-protected — but no such golden exists (the only output test compares
run() == run() within one build, blind to a refactor drift vs the pre-refactor
build).

Resolution (not an override — the spec was corrected and re-grounded):

  • The single-run binding mechanism no longer changes. The stage1-r topology is
    extracted into a shared helper; the single run keeps its identical .bind(...)
    calls (output unchanged by construction), only the sweep floats fast/slow.
  • A golden characterization test of the single-run output is added FIRST, against
    current HEAD, so byte-preservation is verified rather than assumed; a drift
    bounces to debug.

Re-grounded → PASS. Spec committed 1c8678a, boss-signed via the grounding PASS.
Proceeding to planner → implement.

## Grounding revision (specify, /boss auto-sign) The first grounding-check BLOCKed one assumption: the spec claimed the stage1-r single-run output was already byte-pinned and the blueprint refactor thereby byte-protected — but no such golden exists (the only output test compares `run() == run()` within one build, blind to a refactor drift vs the pre-refactor build). Resolution (not an override — the spec was corrected and re-grounded): - The single-run binding mechanism no longer changes. The stage1-r topology is extracted into a shared helper; the single run keeps its identical `.bind(...)` calls (output unchanged by construction), only the sweep floats fast/slow. - A golden characterization test of the single-run output is added FIRST, against current HEAD, so byte-preservation is verified rather than assumed; a drift bounces to debug. Re-grounded → PASS. Spec committed 1c8678a, boss-signed via the grounding PASS. Proceeding to planner → implement.
Brummel commented 2026-06-24 15:28:06 +02:00
Author
Owner
Copy Link
Copy Source

Plan-stage decisions (planner recon flagged two; both derived)

  • Single-run (2,4) vs sweep grid {2,3}×{6,12} → the asymmetry is intended.
    The single run stays fast=2/slow=4 (byte-pinned by the new golden — aligning it
    to the grid would break the byte-preservation contract for no research benefit);
    the sweep grid spans distinct signal configs, not the smoke config. fast < slow
    holds in both.
  • Per-member --trace for the stage1-r sweep → out of scope this cycle.
    #133's deliverable is the rankable family store (append_family + rank); the
    family is persisted to families.jsonl regardless (the --name path). Per-member
    4-tap trace dirs (chartable members) are a separable follow-up; the
    stage1_r_sweep_family(_trace, data) slot stays for dispatch uniformity but is
    unused. (Candidate follow-up idea.)
  • Golden shape → the characterization golden pins only the metrics object
    (not the full line), because manifest.commit is a volatile build constant; the
    refactor can only move computed values (metrics), so a metrics-exact golden is the
    precise, commit-agnostic guard.
## Plan-stage decisions (planner recon flagged two; both derived) - **Single-run (2,4) vs sweep grid {2,3}×{6,12}** → the asymmetry is intended. The single run stays fast=2/slow=4 (byte-pinned by the new golden — aligning it to the grid would break the byte-preservation contract for no research benefit); the sweep grid spans distinct signal configs, not the smoke config. `fast < slow` holds in both. - **Per-member `--trace` for the stage1-r sweep** → out of scope this cycle. #133's deliverable is the rankable family store (`append_family` + rank); the family is persisted to `families.jsonl` regardless (the `--name` path). Per-member 4-tap trace dirs (chartable members) are a separable follow-up; the `stage1_r_sweep_family(_trace, data)` slot stays for dispatch uniformity but is unused. (Candidate follow-up `idea`.) - **Golden shape** → the characterization golden pins only the `metrics` object (not the full line), because `manifest.commit` is a volatile build constant; the refactor can only move computed values (metrics), so a metrics-exact golden is the precise, commit-agnostic guard.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BLOCKER
Core functionality blocked; preempts other selection
 bug
Observable misbehaviour
 feature
Substantial extension; no full spec required
 idea
Droppable; no commitment
No Label feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claude (Claude) Brummel
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#133
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?