Distinguish benign gate-emptied notices from cell-failure warnings on stderr #278

Closed
opened 2026-07-17 12:33:41 +02:00 by claude · 6 comments
Collaborator

Zero-survivor gate notices and per-cell failure notes share one undifferentiated aura: cell … stderr idiom:

  • Benign notice — a selection gate legitimately empties a cell's candidate set; the cell's realization is truncated and the run continues (exit stays 0: a null result is a valid research result, #198): crates/aura-cli/src/campaign_run.rs:672-677 prints aura: cell …: gate at stage N left no survivors; cell realization truncated.
  • Genuine fault — a cell fails, its fault is recorded, and the campaign continues over the remaining cells (#272 per-cell fault isolation): crates/aura-cli/src/campaign_run.rs:690-693 prints aura: cell (…) failed at stage N: … — recorded, campaign continues.

Both classes open with the same aura: cell prefix and differ only in prose wording, so neither a reader scanning a long campaign's stderr nor a tooling filter (grep, CI log scan) can separate expected null-result notices from real faults by form. The parallel cell loop (#277) raises the practical cell count per invocation, so mixed runs — some cells legitimately emptied, some faulted — produce stderr blocks where a fault line is easy to miss among benign notices.

Requirement: the two classes are machine-separable and visually distinct on stderr — e.g. a stable per-class marker (aura: note: … vs aura: warning: …) or an equivalent scheme; the mechanism is open. Exit-code semantics are out of scope and stay unchanged.

Provenance: parked as a non-benchmark side observation in the campaign-scaling measurement notes on #251 (the 2026-07-17 comment); promoted to its own issue so it survives #251's close.

Zero-survivor gate notices and per-cell failure notes share one undifferentiated `aura: cell …` stderr idiom: - Benign notice — a selection gate legitimately empties a cell's candidate set; the cell's realization is truncated and the run continues (exit stays 0: a null result is a valid research result, #198): `crates/aura-cli/src/campaign_run.rs:672-677` prints `aura: cell …: gate at stage N left no survivors; cell realization truncated`. - Genuine fault — a cell fails, its fault is recorded, and the campaign continues over the remaining cells (#272 per-cell fault isolation): `crates/aura-cli/src/campaign_run.rs:690-693` prints `aura: cell (…) failed at stage N: … — recorded, campaign continues`. Both classes open with the same `aura: cell` prefix and differ only in prose wording, so neither a reader scanning a long campaign's stderr nor a tooling filter (grep, CI log scan) can separate expected null-result notices from real faults by form. The parallel cell loop (#277) raises the practical cell count per invocation, so mixed runs — some cells legitimately emptied, some faulted — produce stderr blocks where a fault line is easy to miss among benign notices. Requirement: the two classes are machine-separable and visually distinct on stderr — e.g. a stable per-class marker (`aura: note: …` vs `aura: warning: …`) or an equivalent scheme; the mechanism is open. Exit-code semantics are out of scope and stay unchanged. Provenance: parked as a non-benchmark side observation in the campaign-scaling measurement notes on #251 ([the 2026-07-17 comment](issues/251#issuecomment-3709)); promoted to its own issue so it survives #251's close.
Author
Collaborator

Re-anchoring after cycle #295 (merged 4ed6455): the two stderr call sites this issue targets moved within crates/aura-cli/src/campaign_run.rs due to unrelated code (axes/binding/coverage/family/measure/member/project/reproduce/runner) being extracted out of aura-cli into the new aura-runner and aura-measurement crates. The file itself stayed in aura-cli.

Current locations:

  • Gate-emptied notice (was 672-677): now crates/aura-cli/src/campaign_run.rs:318-322, message unchanged ("aura: cell {}/{}/[{}, {}]: gate at stage {stage_ix} left no survivors; cell realization truncated").
  • Cell-failure warning (was 690-693): now crates/aura-cli/src/campaign_run.rs:336-339, message unchanged ("aura: cell ({}, {}, [{}, {}]) failed at stage {}: {} — recorded, campaign continues").

The requirement is still unmet: both lines still open with the same "aura: cell" prefix with no stable per-class marker. Substance is otherwise unaffected — this is a presentation-layer concern (stderr diagnostics), which stays with aura-cli under the #295 shell boundary (the crate's stated remaining scope is "argv/dispatch, argv-to-document translation, and presentation only"), and is orthogonal to the document-first migration of run-quintet flags decided in #300.

For context, the same undifferentiated cell-failure idiom also appears in crates/aura-cli/src/verb_sugar.rs:529 and :693 (walkforward/mc cell failures) — outside this issue's original two-line scope but worth the fix author's awareness if a shared marker helper is introduced. crates/aura-cli/src/scaffold.rs already sets a precedent for a marker prefix ("aura: warning: ..." at lines 283, 324, 357, 375) that a fix could align with.

Re-anchoring after cycle #295 (merged 4ed6455): the two stderr call sites this issue targets moved within crates/aura-cli/src/campaign_run.rs due to unrelated code (axes/binding/coverage/family/measure/member/project/reproduce/runner) being extracted out of aura-cli into the new aura-runner and aura-measurement crates. The file itself stayed in aura-cli. Current locations: - Gate-emptied notice (was 672-677): now crates/aura-cli/src/campaign_run.rs:318-322, message unchanged ("aura: cell {}/{}/[{}, {}]: gate at stage {stage_ix} left no survivors; cell realization truncated"). - Cell-failure warning (was 690-693): now crates/aura-cli/src/campaign_run.rs:336-339, message unchanged ("aura: cell ({}, {}, [{}, {}]) failed at stage {}: {} — recorded, campaign continues"). The requirement is still unmet: both lines still open with the same "aura: cell" prefix with no stable per-class marker. Substance is otherwise unaffected — this is a presentation-layer concern (stderr diagnostics), which stays with aura-cli under the #295 shell boundary (the crate's stated remaining scope is "argv/dispatch, argv-to-document translation, and presentation only"), and is orthogonal to the document-first migration of run-quintet flags decided in #300. For context, the same undifferentiated cell-failure idiom also appears in crates/aura-cli/src/verb_sugar.rs:529 and :693 (walkforward/mc cell failures) — outside this issue's original two-line scope but worth the fix author's awareness if a shared marker helper is introduced. crates/aura-cli/src/scaffold.rs already sets a precedent for a marker prefix ("aura: warning: ..." at lines 283, 324, 357, 375) that a fix could align with.
claude added this to the Surface honesty on stderr — notices, warnings, and the zero-trade walk-forward milestone 2026-07-23 13:42:57 +02:00
claude self-assigned this 2026-07-23 21:31:09 +02:00
Author
Collaborator

Design reconciliation (specify)

Spec: stderr-class-markers — one cycle covering #278 + #313 (milestone "Surface honesty on stderr"). The forks below are still listed open on this issue; this records their resolution. All are derived orchestrator decisions (2026-07-23), auditable and vetoable.

  • Fork: marker grammaraura: note: <text> / aura: warning: <text>.
    Basis: derived — extends the only class-tagged precedent in the binary (aura: warning: ×4 in scaffold.rs) and matches this issue's own proposal; no competing spelling has any precedent, and both markers keep the aura: namespace so the existing contains("aura:") contract test stays green.
  • Fork: site scope → the four undifferentiated lines are retagged (campaign_run.rs:318 gate-emptied → note; campaign_run.rs:336 cell failure → warning; verb_sugar.rs:529/:693 walkforward/mc cell failure → warning); scaffold's four aura: warning: literals migrate onto the shared helper; plus two prefix-normalization edits — campaign_run.rs:390 (record summary, today entirely unprefixed) gains the bare aura: namespace with no class tag, and aura-runner/src/project.rs:559 (stale-dylib, today warning: without namespace) becomes aura: warning: . The ~140 remaining bare aura: error/usage lines stay unchanged.
    Basis: derived — the class grammar targets diagnostics on continuing runs, where exit codes cannot discriminate; error lines that accompany a non-zero exit already have the C14 exit-code partition as their machine contract, so retagging them adds no separability and would balloon a presentation fix into a wholesale rewrite.
  • Fork: helper shape → a crate-internal diag module in aura-cli owning the two emitters; aura-runner keeps hand-written literals.
    Basis: derived — one source for the grammar inside the presentation crate (the #295 shell boundary names aura-cli as the presentation layer); a cross-crate helper would invert that boundary for one literal.
  • Fork: #313 class → note, wording aura: note: all N walk-forward windows recorded zero trades.
    Basis: derived — no fault occurred and exit stays 0; the glossary's #198 rule ("a null result is a valid research result", glossary.md:52/:286) forbids error/warning framing; #313's own body asks for a notice.
  • Fork: #313 path scope → both walk-forward paths (real/sugar summary at verb_sugar.rs:552 and synthetic-blueprint summary at main.rs:919), guarded on ≥1 window and every window's n_trades == 0.
    Basis: derived — both paths emit the identical {"walkforward":{"windows":N,…}} summary shape, so the silent-degenerate trap is path-independent; covering one and not the other would ship the same spec gap under a different flag combination.
  • Fork: ledger record → a new "stderr class markers" paragraph in C14's Current state (plus a glossary entry), no new C-number.
    Basis: derived — C14 already owns the CLI stderr conventions and the exit-code partition this grammar completes; a separate contract for two string prefixes would be record inflation.

Exit-code semantics are out of scope and unchanged (per this issue's own requirement).

## Design reconciliation (specify) Spec: `stderr-class-markers` — one cycle covering #278 + #313 (milestone "Surface honesty on stderr"). The forks below are still listed open on this issue; this records their resolution. All are derived orchestrator decisions (2026-07-23), auditable and vetoable. - **Fork: marker grammar** → `aura: note: <text>` / `aura: warning: <text>`. Basis: derived — extends the only class-tagged precedent in the binary (`aura: warning:` ×4 in `scaffold.rs`) and matches this issue's own proposal; no competing spelling has any precedent, and both markers keep the `aura:` namespace so the existing `contains("aura:")` contract test stays green. - **Fork: site scope** → the four undifferentiated lines are retagged (`campaign_run.rs:318` gate-emptied → note; `campaign_run.rs:336` cell failure → warning; `verb_sugar.rs:529`/`:693` walkforward/mc cell failure → warning); scaffold's four `aura: warning:` literals migrate onto the shared helper; plus two prefix-normalization edits — `campaign_run.rs:390` (record summary, today entirely unprefixed) gains the bare `aura: ` namespace with no class tag, and `aura-runner/src/project.rs:559` (stale-dylib, today `warning:` without namespace) becomes `aura: warning: `. The ~140 remaining bare `aura: ` error/usage lines stay unchanged. Basis: derived — the class grammar targets diagnostics on *continuing* runs, where exit codes cannot discriminate; error lines that accompany a non-zero exit already have the C14 exit-code partition as their machine contract, so retagging them adds no separability and would balloon a presentation fix into a wholesale rewrite. - **Fork: helper shape** → a crate-internal `diag` module in aura-cli owning the two emitters; aura-runner keeps hand-written literals. Basis: derived — one source for the grammar inside the presentation crate (the #295 shell boundary names aura-cli as the presentation layer); a cross-crate helper would invert that boundary for one literal. - **Fork: #313 class** → note, wording `aura: note: all N walk-forward windows recorded zero trades`. Basis: derived — no fault occurred and exit stays 0; the glossary's #198 rule ("a null result is a valid research result", glossary.md:52/:286) forbids error/warning framing; #313's own body asks for a notice. - **Fork: #313 path scope** → both walk-forward paths (real/sugar summary at `verb_sugar.rs:552` and synthetic-blueprint summary at `main.rs:919`), guarded on ≥1 window and every window's `n_trades == 0`. Basis: derived — both paths emit the identical `{"walkforward":{"windows":N,…}}` summary shape, so the silent-degenerate trap is path-independent; covering one and not the other would ship the same spec gap under a different flag combination. - **Fork: ledger record** → a new "stderr class markers" paragraph in C14's *Current state* (plus a glossary entry), no new C-number. Basis: derived — C14 already owns the CLI stderr conventions and the exit-code partition this grammar completes; a separate contract for two string prefixes would be record inflation. Exit-code semantics are out of scope and unchanged (per this issue's own requirement).
Author
Collaborator

Design reconciliation addendum (specify, post grounding-check)

The grounding-check on stderr-class-markers returned BLOCK on one finding: three benign continuing-run diagnostics were missing from the site table — aura-runner/src/runner.rs:370 (tap needs a cost model; skipped), :374 (tap not produced by this run; skipped), :390 (no-nominee cell; no traces persisted). All three are class 1 (note) by the spec's own taxonomy, so acceptance criterion 1 ("no diagnostic falls outside both classes") was unsatisfiable as written.

  • Fork: the three runner.rs notices → included, note class, as literal retags (aura: aura: note: , message bytes otherwise identical).
    Basis: derived — they are exactly the taxonomy's class 1 (benign, run continues, exit unaffected; the no-nominee line is the per-cell twin of the gate-emptied note), and the alternative — a taxonomy carve-out for trace-persistence notices — would weaken the machine-separability promise the milestone exists to deliver. Treatment mirrors the already-decided project.rs:559 warning retag: hand-written literal in aura-runner, no cross-crate helper (#295 boundary).
    Consequence: two existing pins move with the retag (crates/aura-cli/tests/research_docs.rs:3686, :3789 — the exact cost-model tap-skip string); no other test pins the three strings (workspace grep, 2026-07-23).

Spec edited accordingly; grounding-check re-dispatched on the edited bytes — the autonomous signature remains a fresh PASS, never a sign-over of the BLOCK.

## Design reconciliation addendum (specify, post grounding-check) The grounding-check on `stderr-class-markers` returned BLOCK on one finding: three benign continuing-run diagnostics were missing from the site table — `aura-runner/src/runner.rs:370` (tap needs a cost model; skipped), `:374` (tap not produced by this run; skipped), `:390` (no-nominee cell; no traces persisted). All three are class 1 (note) by the spec's own taxonomy, so acceptance criterion 1 ("no diagnostic falls outside both classes") was unsatisfiable as written. - **Fork: the three runner.rs notices** → included, note class, as literal retags (`aura: ` → `aura: note: `, message bytes otherwise identical). Basis: derived — they are exactly the taxonomy's class 1 (benign, run continues, exit unaffected; the no-nominee line is the per-cell twin of the gate-emptied note), and the alternative — a taxonomy carve-out for trace-persistence notices — would weaken the machine-separability promise the milestone exists to deliver. Treatment mirrors the already-decided `project.rs:559` warning retag: hand-written literal in aura-runner, no cross-crate helper (#295 boundary). Consequence: two existing pins move with the retag (`crates/aura-cli/tests/research_docs.rs:3686`, `:3789` — the exact cost-model tap-skip string); no other test pins the three strings (workspace grep, 2026-07-23). Spec edited accordingly; grounding-check re-dispatched on the edited bytes — the autonomous signature remains a fresh PASS, never a sign-over of the BLOCK.
Author
Collaborator

Autonomous spec sign (specify Step 6, /boss)

Spec stderr-class-markers (this issue + #313, milestone "Surface honesty on stderr") is signed for planning. The signature is the grounding-check PASS of 2026-07-23 on the edited spec bytes — an independent fresh-context agent ratifying every load-bearing current-behaviour assumption against currently-green tests (10 ratified assumptions, incl. the three runner.rs notice retags added after the first-round BLOCK, the two moving pins at research_docs.rs:3686/:3789, and the zero-trades trigger chain on both walk-forward paths). No human signed; the user was notified and can veto by reply. Proceeding to planner.

## Autonomous spec sign (specify Step 6, /boss) Spec `stderr-class-markers` (this issue + #313, milestone "Surface honesty on stderr") is signed for planning. The signature is the grounding-check **PASS** of 2026-07-23 on the edited spec bytes — an independent fresh-context agent ratifying every load-bearing current-behaviour assumption against currently-green tests (10 ratified assumptions, incl. the three runner.rs notice retags added after the first-round BLOCK, the two moving pins at research_docs.rs:3686/:3789, and the zero-trades trigger chain on both walk-forward paths). No human signed; the user was notified and can veto by reply. Proceeding to planner.
Author
Collaborator

Milestone fieldtest result + routing (decision log)

The milestone-close fieldtest for "Surface honesty on stderr" ran 2026-07-24, source-blind (knowledge floor = release binary + its own help/refusal surfaces only, per the user direction of 2026-07-23). Three missions, fixtures committed under fieldtests/milestone-stderr-honesty/.

Verdict: the promise holds on the real/campaign path — markers grep-separable with zero cross-talk, zero-trade note fires exactly when it should on both walk-forward paths, abort-path errors correctly stay outside both classes, exit codes coherent — but the synthetic path has one genuine hole: a member fault escapes as an uncaught panic (exit 101, no marker, internal source path leaked), where the real path contains the same fault as aura: warning: … — recorded, campaign continues + exit 3.

Routing (all findings, none worked around):

  • bug → debug (this run, milestone-gating): the synthetic-path panic escape. Derived contract decision for the RED test: same fault class, same verb, same surface ⇒ same treatment as the real path — contained fault, aura: warning: line, exit 3 (C14 partition), no panic text, no source-path leak. Basis: derived — the #272 containment is the shipped precedent and the C14 marker paragraph admits no unmarked fault class on a surviving run; a pre-flight axis validation (exit 2) would fix only statically-visible poisons and leave runtime member panics uncontained.
  • spec_gap → #324: null-result notes beyond walk-forward (sweep/generalize silent, mc's vacuous case exits 2 unmarked).
  • spec_gap → #325: per-verb marker/exit contract; generalize's partial failure bypasses the warning/exit-3 convention.
  • friction → #326 (bug): graph build silently drops unknown op-list fields.
  • friction → #323 (evidence comment): op-script surface undiscoverable from the binary.
  • friction → tidy (this run): "all 1 walk-forward windows" plural; fix via a pure message-builder + unit pin after the debug cycle lands.

The milestone's green roll-up therefore waits on the panic fix + a re-run of the affected mission; the formal milestone close remains the user's act.

## Milestone fieldtest result + routing (decision log) The milestone-close fieldtest for "Surface honesty on stderr" ran 2026-07-24, source-blind (knowledge floor = release binary + its own help/refusal surfaces only, per the user direction of 2026-07-23). Three missions, fixtures committed under `fieldtests/milestone-stderr-honesty/`. **Verdict:** the promise holds on the real/campaign path — markers grep-separable with zero cross-talk, zero-trade note fires exactly when it should on both walk-forward paths, abort-path errors correctly stay outside both classes, exit codes coherent — but the synthetic path has one genuine hole: a member fault escapes as an uncaught panic (exit 101, no marker, internal source path leaked), where the real path contains the same fault as `aura: warning: … — recorded, campaign continues` + exit 3. Routing (all findings, none worked around): - **bug → debug (this run, milestone-gating):** the synthetic-path panic escape. Derived contract decision for the RED test: same fault class, same verb, same surface ⇒ same treatment as the real path — contained fault, `aura: warning: ` line, exit 3 (C14 partition), no panic text, no source-path leak. Basis: derived — the #272 containment is the shipped precedent and the C14 marker paragraph admits no unmarked fault class on a surviving run; a pre-flight axis validation (exit 2) would fix only statically-visible poisons and leave runtime member panics uncontained. - **spec_gap → #324:** null-result notes beyond walk-forward (sweep/generalize silent, mc's vacuous case exits 2 unmarked). - **spec_gap → #325:** per-verb marker/exit contract; generalize's partial failure bypasses the warning/exit-3 convention. - **friction → #326 (bug):** `graph build` silently drops unknown op-list fields. - **friction → #323 (evidence comment):** op-script surface undiscoverable from the binary. - **friction → tidy (this run):** "all 1 walk-forward windows" plural; fix via a pure message-builder + unit pin after the debug cycle lands. The milestone's green roll-up therefore waits on the panic fix + a re-run of the affected mission; the formal milestone close remains the user's act.
Author
Collaborator

Milestone fieldtest roll-up: GREEN (run close)

After the containment fixes (51096a3 walkforward/sweep, a20b6f6 mc) and the pluralization tidy (4897919), all three mission drivers were re-run against the rebuilt release binary (6fb7caf refreshes the captured evidence in place):

  • the synthetic poison row now reads aura: warning: a member panicked: SMA length must be >= 1, exit 3 — zero raw panicked at lines anywhere in the survey;
  • the single-window case reads the single walk-forward window recorded zero trades;
  • the marker-separability log still greps into two clean classes; exit codes unchanged.

Final gates: full workspace suite green, clippy -D warnings clean, aura-bench exit 0 with all fingerprints OK (audit commit cc683fb carries the raw numbers).

Branch state: worktree-issue-278-stderr-markers, 11 commits on cfc2d9f, unmerged — merge into main is the user's ratification act, and the closes #278 / closes #313 markers (913d187) fire on that push. The formal close of milestone "Surface honesty on stderr" is likewise the user's act; its functional gate (green milestone fieldtest, binary-only knowledge floor) is satisfied.

Follow-ups already queued from the fieldtest: #324 (null-result notes beyond walk-forward), #325 (per-verb marker/exit contract), #326 (op-list silent unknown-field drop), evidence on #323, inventory note on #297.

## Milestone fieldtest roll-up: GREEN (run close) After the containment fixes (51096a3 walkforward/sweep, a20b6f6 mc) and the pluralization tidy (4897919), all three mission drivers were re-run against the rebuilt release binary (6fb7caf refreshes the captured evidence in place): - the synthetic poison row now reads `aura: warning: a member panicked: SMA length must be >= 1`, exit 3 — zero raw `panicked at` lines anywhere in the survey; - the single-window case reads `the single walk-forward window recorded zero trades`; - the marker-separability log still greps into two clean classes; exit codes unchanged. Final gates: full workspace suite green, clippy `-D warnings` clean, aura-bench exit 0 with all fingerprints OK (audit commit cc683fb carries the raw numbers). **Branch state:** `worktree-issue-278-stderr-markers`, 11 commits on `cfc2d9f`, unmerged — merge into main is the user's ratification act, and the `closes #278` / `closes #313` markers (913d187) fire on that push. The formal close of milestone "Surface honesty on stderr" is likewise the user's act; its functional gate (green milestone fieldtest, binary-only knowledge floor) is satisfied. Follow-ups already queued from the fieldtest: #324 (null-result notes beyond walk-forward), #325 (per-verb marker/exit contract), #326 (op-list silent unknown-field drop), evidence on #323, inventory note on #297.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#278