feat: debug RED stage requires minimal, autonomous fixture
Phase 3 previously said only "smallest possible reproducer" — a passive property an agent can satisfy by name while still asserting against the full reproducing fixture. Sharpen it into an explicit two-step discipline: minimize the trigger (delta-reduce until one more cut makes the symptom vanish) and make the fixture autonomous (inline input, no shared fixtures/DB seeds/clocks/server boots unless the bug is genuinely at that integration boundary). - Iron Law gains: an un-minimized repro is not a RED test - Phase 3 rewritten with explicit Minimize + Autonomous steps - 3 rationalization rows + 4 red flags for the "full fixture reproduces deterministically, ship it" failure mode - SKILL.md overview/handoff mirrored at orchestrator altitude Baseline (writing-skills RED-first): 6 subagent runs showed the old wording usually steered right but relied on agent instinct; GREEN: 2 runs under the targeted full-fixture pressure now reduce to the minimal trigger and cite the contract verbatim. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+6
-4
@@ -16,9 +16,11 @@ RED stage — even for "trivial" bugs — produces fixes that don't
|
|||||||
stick and tests that don't exist to catch the next regression.
|
stick and tests that don't exist to catch the next regression.
|
||||||
|
|
||||||
The substantive process — root cause investigation → pattern
|
The substantive process — root cause investigation → pattern
|
||||||
analysis → RED test → handoff, plus the Phase 4.5 architecture-
|
analysis → minimal, autonomous RED test → handoff, plus the
|
||||||
question trigger after three failed hypotheses — lives in
|
Phase 4.5 architecture-question trigger after three failed
|
||||||
`agents/debugger.md`. That file is the single source of truth
|
hypotheses — lives in `agents/debugger.md`. The RED stage is
|
||||||
|
not "any failing test": Phase 3 reduces the reproducer to the
|
||||||
|
smallest autonomous trigger before the test counts. That file is the single source of truth
|
||||||
for the discipline; this skill file only governs trigger,
|
for the discipline; this skill file only governs trigger,
|
||||||
dispatch, and handoff.
|
dispatch, and handoff.
|
||||||
|
|
||||||
@@ -77,7 +79,7 @@ RED+GREEN at the end of mini-mode).
|
|||||||
|
|
||||||
| Field | Content |
|
| Field | Content |
|
||||||
|-------|---------|
|
|-------|---------|
|
||||||
| `red_test_path` | absolute path to the failing test file |
|
| `red_test_path` | absolute path to the failing test file — minimal and autonomous (Phase 3) |
|
||||||
| `cause_summary` | 1-2 sentences naming the file + function + why |
|
| `cause_summary` | 1-2 sentences naming the file + function + why |
|
||||||
| `constraint` | `"minimal fix, no surrounding cleanup"` |
|
| `constraint` | `"minimal fix, no surrounding cleanup"` |
|
||||||
|
|
||||||
|
|||||||
@@ -54,6 +54,7 @@ If the symptom is vague ("something feels off"), return
|
|||||||
|
|
||||||
```
|
```
|
||||||
ROOT CAUSE FIRST. RED TEST SECOND. STOP.
|
ROOT CAUSE FIRST. RED TEST SECOND. STOP.
|
||||||
|
THE RED TEST IS MINIMAL AND AUTONOMOUS — AN UN-MINIMIZED REPRO IS NOT A RED TEST.
|
||||||
NO FIX ATTEMPT IN THIS DISPATCH — THE GREEN SIDE GOES TO `implement` MINI-MODE.
|
NO FIX ATTEMPT IN THIS DISPATCH — THE GREEN SIDE GOES TO `implement` MINI-MODE.
|
||||||
NO SYMPTOM SUPPRESSION. NO HUNCH-DRIVEN CHANGES.
|
NO SYMPTOM SUPPRESSION. NO HUNCH-DRIVEN CHANGES.
|
||||||
```
|
```
|
||||||
@@ -103,18 +104,52 @@ Each phase completes before the next starts.
|
|||||||
|
|
||||||
### Phase 3 — RED test
|
### Phase 3 — RED test
|
||||||
|
|
||||||
|
A reproduction is not yet a RED test. The carrier's repro — or
|
||||||
|
the full fixture a user handed you — is the *starting point*,
|
||||||
|
not the artefact. Phase 3 isolates the specific failing case
|
||||||
|
into a minimal, autonomous test. **A red test that reproduces
|
||||||
|
but was never minimized does not satisfy this phase** — any old
|
||||||
|
red test will not do.
|
||||||
|
|
||||||
1. State your hypothesis as a falsifiable claim:
|
1. State your hypothesis as a falsifiable claim:
|
||||||
*"The bug is X in `<path>:<fn>` because Y."*
|
*"The bug is X in `<path>:<fn>` because Y."*
|
||||||
2. **Write the failing test FIRST**, before any fix attempt:
|
2. **Minimize the trigger.** Reduce the reproducer to the
|
||||||
- smallest possible reproducer (placed in the testing
|
smallest input that still fails:
|
||||||
location idiomatic for this project — integration test,
|
- Start from whatever reproduces — a 240-line fixture, a
|
||||||
unit test, E2E suite, etc.)
|
booted server, a 400-cell workbook — and strip everything
|
||||||
|
not required to trigger the symptom. Delete keys, cells,
|
||||||
|
routes, lines; shrink values; collapse nesting — re-running
|
||||||
|
after each cut — until removing one more thing makes the
|
||||||
|
symptom vanish. What remains is the minimal trigger, and it
|
||||||
|
*is* the diagnosis: it names the bug more precisely than any
|
||||||
|
prose.
|
||||||
|
- A bug reproducing inside a large fixture is not licence to
|
||||||
|
assert against that fixture. Reduction is the work of this
|
||||||
|
phase, not optional polish to skip under time pressure.
|
||||||
|
3. **Make the fixture AUTONOMOUS.** The test constructs its own
|
||||||
|
input inline and depends on nothing it does not itself set
|
||||||
|
up — no shared fixture files, DB seeds, frozen clocks, mock
|
||||||
|
services, or `beforeAll` server boots — *unless* the bug is
|
||||||
|
genuinely in that integration boundary. Phase 1's data-flow
|
||||||
|
trace decides which layer owns the bug, not which layer is
|
||||||
|
convenient: a pure-function bug gets a pure-function test
|
||||||
|
next to its unit, never an assertion bolted onto an E2E suite
|
||||||
|
that happens to exercise it.
|
||||||
|
4. **Write the failing test FIRST**, before any fix attempt,
|
||||||
|
from the minimal autonomous trigger:
|
||||||
|
- placed in the testing location idiomatic for this project
|
||||||
|
(the layer the fix belongs to — usually a unit test; an
|
||||||
|
integration/E2E test only when Phase 1 located the bug at
|
||||||
|
that boundary)
|
||||||
- automated, deterministic — same input always same output
|
- automated, deterministic — same input always same output
|
||||||
- doc comment names the *property* the test protects, not
|
- doc comment names the *property* the test protects, not
|
||||||
the symptom
|
the symptom
|
||||||
3. Run the test. Confirm it fails with the symptom the carrier
|
5. Run the test. Confirm it fails with the symptom the carrier
|
||||||
described.
|
described — and that it fails at the diagnosed cause, not at
|
||||||
4. Leave the failing test in the working tree as an unstaged
|
some unrelated earlier error. (A repro that no longer fires
|
||||||
|
once minimized means the trigger you blamed is wrong; return
|
||||||
|
to Phase 1.)
|
||||||
|
6. Leave the failing test in the working tree as an unstaged
|
||||||
change. You do NOT commit. The orchestrator decides commit
|
change. You do NOT commit. The orchestrator decides commit
|
||||||
shape; the suggested subject if the orchestrator elects to
|
shape; the suggested subject if the orchestrator elects to
|
||||||
commit the RED separately is `test: red for <symptom>`.
|
commit the RED separately is `test: red for <symptom>`.
|
||||||
@@ -196,6 +231,9 @@ At most 250 words, structured:
|
|||||||
| "End-of-day, just drop a fix into the working tree and write the test tomorrow" | End-of-day pressure is the worst time for shotgun fixes. Clean tree + open bug > three speculative half-fixes. Diagnosis is RED-first; the fix is a separate dispatch. |
|
| "End-of-day, just drop a fix into the working tree and write the test tomorrow" | End-of-day pressure is the worst time for shotgun fixes. Clean tree + open bug > three speculative half-fixes. Diagnosis is RED-first; the fix is a separate dispatch. |
|
||||||
| "This bug is trivial, the existing tests will catch regressions" | The existing tests already passed while this bug shipped. By definition they don't cover it. |
|
| "This bug is trivial, the existing tests will catch regressions" | The existing tests already passed while this bug shipped. By definition they don't cover it. |
|
||||||
| "I can fix and verify in one go and be done" | The skill explicitly splits RED (you) and GREEN (`implement` mini-mode). Don't collapse them; the split is the whole point. |
|
| "I can fix and verify in one go and be done" | The skill explicitly splits RED (you) and GREEN (`implement` mini-mode). Don't collapse them; the split is the whole point. |
|
||||||
|
| "The full fixture reproduces 100% and the test is deterministic and automated — that's a valid RED test" | Deterministic ≠ minimal. A crash buried in a 240-line fixture pins the symptom to incidental data, breaks on unrelated fixture edits, and tells the implementer nothing about the trigger. Phase 3 reduction is mandatory, not polish. |
|
||||||
|
| "Reducing the fixture is cleanup I can skip when I'm short on time" | The minimal trigger *is* the diagnosis — it names the bug. Skipping reduction means you don't yet know what the bug is, only that something somewhere fails. |
|
||||||
|
| "The bug only shows with the real DB seed / booted server, so the test needs them too" | Almost always the integration is incidental to a unit-level defect. Phase 1's data-flow trace decides the layer, not convenience. If the failing unit is pure, the test is pure too. |
|
||||||
|
|
||||||
## Red Flags — STOP and return to Phase 1
|
## Red Flags — STOP and return to Phase 1
|
||||||
|
|
||||||
@@ -204,6 +242,10 @@ At most 250 words, structured:
|
|||||||
- "I see the symptom, let me fix it"
|
- "I see the symptom, let me fix it"
|
||||||
- "The test is redundant, I manually verified"
|
- "The test is redundant, I manually verified"
|
||||||
- "Pattern says X but I'll adapt it differently"
|
- "Pattern says X but I'll adapt it differently"
|
||||||
|
- "It reproduces, that's good enough" (without minimizing first)
|
||||||
|
- RED test loads a large or shared fixture you did not reduce
|
||||||
|
- Asserting against the full repro without isolating the minimal trigger
|
||||||
|
- A bug in a pure function whose test needs `beforeAll` / a server boot / a DB seed
|
||||||
- *"One more hypothesis"* (when ≥ 2 already failed)
|
- *"One more hypothesis"* (when ≥ 2 already failed)
|
||||||
- About to apply ANY edit outside the new test file
|
- About to apply ANY edit outside the new test file
|
||||||
- About to run `git commit` (anywhere, ever — you never commit)
|
- About to run `git commit` (anywhere, ever — you never commit)
|
||||||
|
|||||||
Reference in New Issue
Block a user