From 6410732943ef1e4557e45f97dd7ea3047e4fdc3f Mon Sep 17 00:00:00 2001 From: Brummel Date: Sat, 30 May 2026 10:19:15 +0200 Subject: [PATCH] feat: debug RED stage requires minimal, autonomous fixture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 3 previously said only "smallest possible reproducer" — a passive property an agent can satisfy by name while still asserting against the full reproducing fixture. Sharpen it into an explicit two-step discipline: minimize the trigger (delta-reduce until one more cut makes the symptom vanish) and make the fixture autonomous (inline input, no shared fixtures/DB seeds/clocks/server boots unless the bug is genuinely at that integration boundary). - Iron Law gains: an un-minimized repro is not a RED test - Phase 3 rewritten with explicit Minimize + Autonomous steps - 3 rationalization rows + 4 red flags for the "full fixture reproduces deterministically, ship it" failure mode - SKILL.md overview/handoff mirrored at orchestrator altitude Baseline (writing-skills RED-first): 6 subagent runs showed the old wording usually steered right but relied on agent instinct; GREEN: 2 runs under the targeted full-fixture pressure now reduce to the minimal trigger and cite the contract verbatim. Co-Authored-By: Claude Opus 4.8 (1M context) --- debug/SKILL.md | 10 ++++--- debug/agents/debugger.md | 56 +++++++++++++++++++++++++++++++++++----- 2 files changed, 55 insertions(+), 11 deletions(-) diff --git a/debug/SKILL.md b/debug/SKILL.md index 30c7d97..60461e1 100644 --- a/debug/SKILL.md +++ b/debug/SKILL.md @@ -16,9 +16,11 @@ RED stage — even for "trivial" bugs — produces fixes that don't stick and tests that don't exist to catch the next regression. The substantive process — root cause investigation → pattern -analysis → RED test → handoff, plus the Phase 4.5 architecture- -question trigger after three failed hypotheses — lives in -`agents/debugger.md`. That file is the single source of truth +analysis → minimal, autonomous RED test → handoff, plus the +Phase 4.5 architecture-question trigger after three failed +hypotheses — lives in `agents/debugger.md`. The RED stage is +not "any failing test": Phase 3 reduces the reproducer to the +smallest autonomous trigger before the test counts. That file is the single source of truth for the discipline; this skill file only governs trigger, dispatch, and handoff. @@ -77,7 +79,7 @@ RED+GREEN at the end of mini-mode). | Field | Content | |-------|---------| -| `red_test_path` | absolute path to the failing test file | +| `red_test_path` | absolute path to the failing test file — minimal and autonomous (Phase 3) | | `cause_summary` | 1-2 sentences naming the file + function + why | | `constraint` | `"minimal fix, no surrounding cleanup"` | diff --git a/debug/agents/debugger.md b/debug/agents/debugger.md index 0d4b078..d0c7b05 100644 --- a/debug/agents/debugger.md +++ b/debug/agents/debugger.md @@ -54,6 +54,7 @@ If the symptom is vague ("something feels off"), return ``` ROOT CAUSE FIRST. RED TEST SECOND. STOP. +THE RED TEST IS MINIMAL AND AUTONOMOUS — AN UN-MINIMIZED REPRO IS NOT A RED TEST. NO FIX ATTEMPT IN THIS DISPATCH — THE GREEN SIDE GOES TO `implement` MINI-MODE. NO SYMPTOM SUPPRESSION. NO HUNCH-DRIVEN CHANGES. ``` @@ -103,18 +104,52 @@ Each phase completes before the next starts. ### Phase 3 — RED test +A reproduction is not yet a RED test. The carrier's repro — or +the full fixture a user handed you — is the *starting point*, +not the artefact. Phase 3 isolates the specific failing case +into a minimal, autonomous test. **A red test that reproduces +but was never minimized does not satisfy this phase** — any old +red test will not do. + 1. State your hypothesis as a falsifiable claim: *"The bug is X in `:` because Y."* -2. **Write the failing test FIRST**, before any fix attempt: - - smallest possible reproducer (placed in the testing - location idiomatic for this project — integration test, - unit test, E2E suite, etc.) +2. **Minimize the trigger.** Reduce the reproducer to the + smallest input that still fails: + - Start from whatever reproduces — a 240-line fixture, a + booted server, a 400-cell workbook — and strip everything + not required to trigger the symptom. Delete keys, cells, + routes, lines; shrink values; collapse nesting — re-running + after each cut — until removing one more thing makes the + symptom vanish. What remains is the minimal trigger, and it + *is* the diagnosis: it names the bug more precisely than any + prose. + - A bug reproducing inside a large fixture is not licence to + assert against that fixture. Reduction is the work of this + phase, not optional polish to skip under time pressure. +3. **Make the fixture AUTONOMOUS.** The test constructs its own + input inline and depends on nothing it does not itself set + up — no shared fixture files, DB seeds, frozen clocks, mock + services, or `beforeAll` server boots — *unless* the bug is + genuinely in that integration boundary. Phase 1's data-flow + trace decides which layer owns the bug, not which layer is + convenient: a pure-function bug gets a pure-function test + next to its unit, never an assertion bolted onto an E2E suite + that happens to exercise it. +4. **Write the failing test FIRST**, before any fix attempt, + from the minimal autonomous trigger: + - placed in the testing location idiomatic for this project + (the layer the fix belongs to — usually a unit test; an + integration/E2E test only when Phase 1 located the bug at + that boundary) - automated, deterministic — same input always same output - doc comment names the *property* the test protects, not the symptom -3. Run the test. Confirm it fails with the symptom the carrier - described. -4. Leave the failing test in the working tree as an unstaged +5. Run the test. Confirm it fails with the symptom the carrier + described — and that it fails at the diagnosed cause, not at + some unrelated earlier error. (A repro that no longer fires + once minimized means the trigger you blamed is wrong; return + to Phase 1.) +6. Leave the failing test in the working tree as an unstaged change. You do NOT commit. The orchestrator decides commit shape; the suggested subject if the orchestrator elects to commit the RED separately is `test: red for `. @@ -196,6 +231,9 @@ At most 250 words, structured: | "End-of-day, just drop a fix into the working tree and write the test tomorrow" | End-of-day pressure is the worst time for shotgun fixes. Clean tree + open bug > three speculative half-fixes. Diagnosis is RED-first; the fix is a separate dispatch. | | "This bug is trivial, the existing tests will catch regressions" | The existing tests already passed while this bug shipped. By definition they don't cover it. | | "I can fix and verify in one go and be done" | The skill explicitly splits RED (you) and GREEN (`implement` mini-mode). Don't collapse them; the split is the whole point. | +| "The full fixture reproduces 100% and the test is deterministic and automated — that's a valid RED test" | Deterministic ≠ minimal. A crash buried in a 240-line fixture pins the symptom to incidental data, breaks on unrelated fixture edits, and tells the implementer nothing about the trigger. Phase 3 reduction is mandatory, not polish. | +| "Reducing the fixture is cleanup I can skip when I'm short on time" | The minimal trigger *is* the diagnosis — it names the bug. Skipping reduction means you don't yet know what the bug is, only that something somewhere fails. | +| "The bug only shows with the real DB seed / booted server, so the test needs them too" | Almost always the integration is incidental to a unit-level defect. Phase 1's data-flow trace decides the layer, not convenience. If the failing unit is pure, the test is pure too. | ## Red Flags — STOP and return to Phase 1 @@ -204,6 +242,10 @@ At most 250 words, structured: - "I see the symptom, let me fix it" - "The test is redundant, I manually verified" - "Pattern says X but I'll adapt it differently" +- "It reproduces, that's good enough" (without minimizing first) +- RED test loads a large or shared fixture you did not reduce +- Asserting against the full repro without isolating the minimal trigger +- A bug in a pure function whose test needs `beforeAll` / a server boot / a DB seed - *"One more hypothesis"* (when ≥ 2 already failed) - About to apply ANY edit outside the new test file - About to run `git commit` (anywhere, ever — you never commit)