feat: debug RED stage requires minimal, autonomous fixture
Phase 3 previously said only "smallest possible reproducer" — a passive property an agent can satisfy by name while still asserting against the full reproducing fixture. Sharpen it into an explicit two-step discipline: minimize the trigger (delta-reduce until one more cut makes the symptom vanish) and make the fixture autonomous (inline input, no shared fixtures/DB seeds/clocks/server boots unless the bug is genuinely at that integration boundary). - Iron Law gains: an un-minimized repro is not a RED test - Phase 3 rewritten with explicit Minimize + Autonomous steps - 3 rationalization rows + 4 red flags for the "full fixture reproduces deterministically, ship it" failure mode - SKILL.md overview/handoff mirrored at orchestrator altitude Baseline (writing-skills RED-first): 6 subagent runs showed the old wording usually steered right but relied on agent instinct; GREEN: 2 runs under the targeted full-fixture pressure now reduce to the minimal trigger and cite the contract verbatim. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -54,6 +54,7 @@ If the symptom is vague ("something feels off"), return
|
||||
|
||||
```
|
||||
ROOT CAUSE FIRST. RED TEST SECOND. STOP.
|
||||
THE RED TEST IS MINIMAL AND AUTONOMOUS — AN UN-MINIMIZED REPRO IS NOT A RED TEST.
|
||||
NO FIX ATTEMPT IN THIS DISPATCH — THE GREEN SIDE GOES TO `implement` MINI-MODE.
|
||||
NO SYMPTOM SUPPRESSION. NO HUNCH-DRIVEN CHANGES.
|
||||
```
|
||||
@@ -103,18 +104,52 @@ Each phase completes before the next starts.
|
||||
|
||||
### Phase 3 — RED test
|
||||
|
||||
A reproduction is not yet a RED test. The carrier's repro — or
|
||||
the full fixture a user handed you — is the *starting point*,
|
||||
not the artefact. Phase 3 isolates the specific failing case
|
||||
into a minimal, autonomous test. **A red test that reproduces
|
||||
but was never minimized does not satisfy this phase** — any old
|
||||
red test will not do.
|
||||
|
||||
1. State your hypothesis as a falsifiable claim:
|
||||
*"The bug is X in `<path>:<fn>` because Y."*
|
||||
2. **Write the failing test FIRST**, before any fix attempt:
|
||||
- smallest possible reproducer (placed in the testing
|
||||
location idiomatic for this project — integration test,
|
||||
unit test, E2E suite, etc.)
|
||||
2. **Minimize the trigger.** Reduce the reproducer to the
|
||||
smallest input that still fails:
|
||||
- Start from whatever reproduces — a 240-line fixture, a
|
||||
booted server, a 400-cell workbook — and strip everything
|
||||
not required to trigger the symptom. Delete keys, cells,
|
||||
routes, lines; shrink values; collapse nesting — re-running
|
||||
after each cut — until removing one more thing makes the
|
||||
symptom vanish. What remains is the minimal trigger, and it
|
||||
*is* the diagnosis: it names the bug more precisely than any
|
||||
prose.
|
||||
- A bug reproducing inside a large fixture is not licence to
|
||||
assert against that fixture. Reduction is the work of this
|
||||
phase, not optional polish to skip under time pressure.
|
||||
3. **Make the fixture AUTONOMOUS.** The test constructs its own
|
||||
input inline and depends on nothing it does not itself set
|
||||
up — no shared fixture files, DB seeds, frozen clocks, mock
|
||||
services, or `beforeAll` server boots — *unless* the bug is
|
||||
genuinely in that integration boundary. Phase 1's data-flow
|
||||
trace decides which layer owns the bug, not which layer is
|
||||
convenient: a pure-function bug gets a pure-function test
|
||||
next to its unit, never an assertion bolted onto an E2E suite
|
||||
that happens to exercise it.
|
||||
4. **Write the failing test FIRST**, before any fix attempt,
|
||||
from the minimal autonomous trigger:
|
||||
- placed in the testing location idiomatic for this project
|
||||
(the layer the fix belongs to — usually a unit test; an
|
||||
integration/E2E test only when Phase 1 located the bug at
|
||||
that boundary)
|
||||
- automated, deterministic — same input always same output
|
||||
- doc comment names the *property* the test protects, not
|
||||
the symptom
|
||||
3. Run the test. Confirm it fails with the symptom the carrier
|
||||
described.
|
||||
4. Leave the failing test in the working tree as an unstaged
|
||||
5. Run the test. Confirm it fails with the symptom the carrier
|
||||
described — and that it fails at the diagnosed cause, not at
|
||||
some unrelated earlier error. (A repro that no longer fires
|
||||
once minimized means the trigger you blamed is wrong; return
|
||||
to Phase 1.)
|
||||
6. Leave the failing test in the working tree as an unstaged
|
||||
change. You do NOT commit. The orchestrator decides commit
|
||||
shape; the suggested subject if the orchestrator elects to
|
||||
commit the RED separately is `test: red for <symptom>`.
|
||||
@@ -196,6 +231,9 @@ At most 250 words, structured:
|
||||
| "End-of-day, just drop a fix into the working tree and write the test tomorrow" | End-of-day pressure is the worst time for shotgun fixes. Clean tree + open bug > three speculative half-fixes. Diagnosis is RED-first; the fix is a separate dispatch. |
|
||||
| "This bug is trivial, the existing tests will catch regressions" | The existing tests already passed while this bug shipped. By definition they don't cover it. |
|
||||
| "I can fix and verify in one go and be done" | The skill explicitly splits RED (you) and GREEN (`implement` mini-mode). Don't collapse them; the split is the whole point. |
|
||||
| "The full fixture reproduces 100% and the test is deterministic and automated — that's a valid RED test" | Deterministic ≠ minimal. A crash buried in a 240-line fixture pins the symptom to incidental data, breaks on unrelated fixture edits, and tells the implementer nothing about the trigger. Phase 3 reduction is mandatory, not polish. |
|
||||
| "Reducing the fixture is cleanup I can skip when I'm short on time" | The minimal trigger *is* the diagnosis — it names the bug. Skipping reduction means you don't yet know what the bug is, only that something somewhere fails. |
|
||||
| "The bug only shows with the real DB seed / booted server, so the test needs them too" | Almost always the integration is incidental to a unit-level defect. Phase 1's data-flow trace decides the layer, not convenience. If the failing unit is pure, the test is pure too. |
|
||||
|
||||
## Red Flags — STOP and return to Phase 1
|
||||
|
||||
@@ -204,6 +242,10 @@ At most 250 words, structured:
|
||||
- "I see the symptom, let me fix it"
|
||||
- "The test is redundant, I manually verified"
|
||||
- "Pattern says X but I'll adapt it differently"
|
||||
- "It reproduces, that's good enough" (without minimizing first)
|
||||
- RED test loads a large or shared fixture you did not reduce
|
||||
- Asserting against the full repro without isolating the minimal trigger
|
||||
- A bug in a pure function whose test needs `beforeAll` / a server boot / a DB seed
|
||||
- *"One more hypothesis"* (when ≥ 2 already failed)
|
||||
- About to apply ANY edit outside the new test file
|
||||
- About to run `git commit` (anywhere, ever — you never commit)
|
||||
|
||||
Reference in New Issue
Block a user