Triage the findings of the 2026-07-22 external release-binary field test #314

Closed
opened 2026-07-23 06:36:23 +02:00 by claude · 1 comment
Collaborator

An external LLM agent (Gemini 3.6 via the Antigravity CLI) spent 2026-07-22 doing quantitative research against the aura release binary alone: no access to this repository's source, docs or design ledger; extension only via aura nodes new node crates; seven sandboxed research workspaces. The day exercised run, sweep, walkforward, mc, generalize, chart, measure, and native-node authoring, and surfaced four engine-relevant observations. This issue records their triage; the two actionable ones are filed as #312 and #313.

  1. Stop-and-re-enter churn under a continuous bias — intended semantics, not a bug. The agent observed a continuous +1 bias plus the default protective stop producing 1,316 trades where the signal flips sign only 66 times (EURUSD, EMA 14400/144000 cross, --stop-length 3 --stop-k 2.0) and described it as a "broker re-entry loop" defect. Verified as the intended decoupled target-position semantics — contract C10 / domain invariant 7; the position-management block re-opens whenever flat and bias != 0 (crates/aura-backtest/src/position_management.rs:176). Discriminating reproduction with the release binary on the identical signal: trade count is a smooth monotone function of stop distance (--stop-k 1 -> 2342 trades, 2 -> 1316, 5 -> 565, 10 -> 351, 30 -> 187, 100 -> 114, converging toward the 66-flip floor) while bias_sign_flips and pip totals stay constant — the extra round-trips are genuine stop-outs governed by stop geometry, not phantom churn. Not filed as a bug; the discoverability gap is recorded on #267 (the generated agent bootstrap card) as field evidence.

  2. measure ic is unreachable from the release binary — no producible trace carries a price or signal tap. Filed as #312.

  3. An all-zero-trades walk-forward completes silently — in the field run, 184 of 184 OOS windows with n_trades == 0 went unnoticed and fed downstream conclusions. Filed as #313.

  4. Claimed ~2-core ceiling for generalize with native node crates — refuted by measurement. On a 24-core machine, an 8-instrument generalize (AAPL.US, XAUUSD, BTCUSD, EURUSD, US500, Copper, NatGas, TSLA.US) shows the same CPU utilisation in all three configurations: native node used (347-372% CPU), std-only vocabulary in a project without a node crate (375-385%), std-only blueprint with the node crate loaded but unused (352%). The bound is per-instrument workload skew across 8 single-trial sims — parallelism is across sims by design (domain invariant 1) — not a cdylib effect. No issue filed.

An external LLM agent (Gemini 3.6 via the Antigravity CLI) spent 2026-07-22 doing quantitative research against the aura release binary alone: no access to this repository's source, docs or design ledger; extension only via `aura nodes new` node crates; seven sandboxed research workspaces. The day exercised `run`, `sweep`, `walkforward`, `mc`, `generalize`, `chart`, `measure`, and native-node authoring, and surfaced four engine-relevant observations. This issue records their triage; the two actionable ones are filed as #312 and #313. 1. **Stop-and-re-enter churn under a continuous bias — intended semantics, not a bug.** The agent observed a continuous +1 bias plus the default protective stop producing 1,316 trades where the signal flips sign only 66 times (EURUSD, EMA 14400/144000 cross, `--stop-length 3 --stop-k 2.0`) and described it as a "broker re-entry loop" defect. Verified as the intended decoupled target-position semantics — contract C10 / domain invariant 7; the position-management block re-opens whenever flat and bias != 0 (`crates/aura-backtest/src/position_management.rs:176`). Discriminating reproduction with the release binary on the identical signal: trade count is a smooth monotone function of stop distance (`--stop-k` 1 -> 2342 trades, 2 -> 1316, 5 -> 565, 10 -> 351, 30 -> 187, 100 -> 114, converging toward the 66-flip floor) while `bias_sign_flips` and pip totals stay constant — the extra round-trips are genuine stop-outs governed by stop geometry, not phantom churn. Not filed as a bug; the discoverability gap is recorded on #267 (the generated agent bootstrap card) as field evidence. 2. **`measure ic` is unreachable from the release binary** — no producible trace carries a price or signal tap. Filed as #312. 3. **An all-zero-trades walk-forward completes silently** — in the field run, 184 of 184 OOS windows with `n_trades == 0` went unnoticed and fed downstream conclusions. Filed as #313. 4. **Claimed ~2-core ceiling for `generalize` with native node crates — refuted by measurement.** On a 24-core machine, an 8-instrument `generalize` (AAPL.US, XAUUSD, BTCUSD, EURUSD, US500, Copper, NatGas, TSLA.US) shows the same CPU utilisation in all three configurations: native node used (347-372% CPU), std-only vocabulary in a project without a node crate (375-385%), std-only blueprint with the node crate loaded but unused (352%). The bound is per-instrument workload skew across 8 single-trial sims — parallelism is across sims by design (domain invariant 1) — not a cdylib effect. No issue filed.
claude self-assigned this 2026-07-23 06:36:23 +02:00
Author
Collaborator

Decision log — filed and decided autonomously (agent-driven run, 2026-07-23); no human review at filing time.

  • Churn finding routed to documentation, not bug. Options considered: (A) file the stop-and-re-enter behaviour as an engine bug; (B) treat it as intended semantics with a discoverability gap. Chosen: (B), on the evidence in item 1 of the issue description — the code path matching contract C10 / domain invariant 7, and the monotone stop-distance reproduction showing the round-trips track stop geometry. Field evidence recorded on #267 for the bootstrap card.

  • Two findings filed as feature rather than fixed in this run. #312 (persistable price/signal taps so measure ic is reachable) touches the closed tap vocabulary decided in #201; #313 (explicit notice on an all-zero-trades walk-forward) touches the stderr-notice taxonomy open in #278. Both are design surfaces rather than mechanical fixes; they enter the queue as feature work. Status: ready for design work.

  • Claimed 2-core ceiling: measured and refuted (item 4 of the issue description); no tracker item.

Intake complete: findings triaged, #312 and #313 filed, bootstrap-card field evidence recorded on #267 (comment).

Decision log — filed and decided autonomously (agent-driven run, 2026-07-23); no human review at filing time. - **Churn finding routed to documentation, not `bug`.** Options considered: (A) file the stop-and-re-enter behaviour as an engine bug; (B) treat it as intended semantics with a discoverability gap. Chosen: (B), on the evidence in item 1 of the issue description — the code path matching contract C10 / domain invariant 7, and the monotone stop-distance reproduction showing the round-trips track stop geometry. Field evidence recorded on #267 for the bootstrap card. - **Two findings filed as `feature` rather than fixed in this run.** #312 (persistable price/signal taps so `measure ic` is reachable) touches the closed tap vocabulary decided in #201; #313 (explicit notice on an all-zero-trades walk-forward) touches the stderr-notice taxonomy open in #278. Both are design surfaces rather than mechanical fixes; they enter the queue as feature work. Status: ready for design work. - **Claimed 2-core ceiling: measured and refuted** (item 4 of the issue description); no tracker item. Intake complete: findings triaged, #312 and #313 filed, bootstrap-card field evidence recorded on #267 ([comment](issues/267#issuecomment-4271)).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#314