skills: deduplicate single-agent SKILL.md files (debug, fieldtest)

The four-phase debug process and the five-phase fieldtest process
each lived twice — once in SKILL.md (orchestrator-facing) and once
in the dispatched agent's file (subagent-facing). The orchestrator
does not execute these phases; the subagent does. Duplicating them
in SKILL.md just bloated the orchestrator's main context with bytes
that only the subagent ever needs.

SKILL.md now carries only what the orchestrator must consult at
dispatch time: trigger gating, Iron Law as headline, the carrier
contract, the produced handoff, and cross-references. Iron Law in
operational form, full process, Common Rationalisations, Red Flags,
and (for fieldtest) the spec template now live solely in the agent
file. Net −160 LOC across the skills tree.

CLAUDE.md pointer for 'Bug fixes — TDD, always' updated to reflect
that the substantive discipline lives in the agent file, not the
skill file.
This commit is contained in:
2026-05-10 10:53:53 +02:00
parent bb6b52e7c3
commit 64b0841c5a
5 changed files with 97 additions and 254 deletions
+5 -2
View File
@@ -177,9 +177,12 @@ exception, not the rule.
## Bug fixes — TDD, always
Bug fixes are RED-first, autonomous, no orchestrator gate. Full
Bug fixes are RED-first, autonomous, no orchestrator gate. The
trigger and handoff are in `skills/debug/SKILL.md`; the full
discipline (Iron Law, four phases, Phase 4.5 architecture trigger,
common rationalisations) lives in `skills/debug/SKILL.md`. Headline:
common rationalisations) lives in
`skills/debug/agents/ailang-debugger.md` — that is the file the
dispatched subagent reads. Headline:
1. **RED first** — write a failing test that pins down the symptom,
commit it as `test: red for <symptom>` BEFORE any fix.
+22 -87
View File
@@ -15,6 +15,13 @@ skill then drives the fix to GREEN. Skipping the RED stage — even for
"trivial" bugs — produces fixes that don't stick and tests that don't
exist to catch the next regression.
The substantive process — root cause investigation → pattern analysis
→ RED test → handoff, plus the Phase 4.5 architecture-question trigger
after three failed hypotheses — lives in
`agents/ailang-debugger.md`. That file is the single source of truth
for the discipline; this skill file only governs trigger, dispatch,
and handoff.
## When to Use / Skipping
Trigger this skill on:
@@ -40,74 +47,24 @@ Both clauses are non-negotiable. The second was formerly the
CLAUDE.md "Bug fixes — TDD, always" section before the
2026-05-09 skill-system migration.
## The Process — four phases
## Dispatch
Each phase must complete before the next starts.
Dispatch `ailang-debugger` with:
### Phase 1 — Root Cause Investigation
| Carrier field | Content |
|---------------|---------|
| `symptom` | Exact error message, stack trace, wrong output, or repro command |
| `repro_known` | One-line repro if known, otherwise the agent finds one |
| `recent_iter` | Iteration that last touched the suspected area (often `git log` tail) |
1. Read the error message in full. Stack traces, line numbers, exit
codes — none of them get skimmed.
2. Reproduce with the shortest possible command. If you can't repro
reliably, gather more data — do NOT guess.
3. Check recent changes: `git log --oneline -20`,
`git diff <previous-iter-commit>..HEAD`.
4. For multi-component issues (compiler → emitter → linker; or
runtime → C glue → binary), instrument every boundary. Find the
layer where the data goes wrong before deciding which layer to
fix.
5. Trace data flow backward from symptom to source. Fix at source,
not at symptom.
### Phase 2 — Pattern Analysis
1. Find a similar working example in the codebase. What's different?
2. If the bug is in a pattern (e.g. ADT match lowering, RC drop
emission), read the reference implementation completely. No
skimming.
3. List every difference between working and broken — however small.
### Phase 3 — Hypothesis and Test
1. State the hypothesis as a single falsifiable claim:
*"X is the root cause because Y."*
2. **Write the failing test FIRST**, before any fix attempt:
- smallest possible reproducer (E2E in `crates/ail/tests/e2e.rs`,
or unit test in the affected crate)
- automated, deterministic — same input always same output
- doc comment names the property the test protects, not just
what it does
3. Run it to confirm it fails on the current code, with the symptom
the bug-report described.
4. Commit the failing test as a separate commit:
`git commit -m "test: red for <symptom>"`.
### Phase 4 — Hand off to `implement`
The fix itself is `implement`'s job, run in mini-mode. Hand it:
- path to the RED-test file
- 1-2 sentence cause summary from Phase 1
- hard constraint: *"minimal fix, no surrounding cleanup,
no opportunistic refactor"*
`implement` runs the GREEN side and commits `fix: <symptom>`.
### Phase 4.5 — 3+ failures = architecture question
If the third hypothesis fails:
```
STOP. Do not attempt Fix #4.
```
Three failed fixes in a row indicate the architecture is wrong, not
that the latest hypothesis is wrong. Each fix likely revealed new
shared state or coupling in a different place. Surface the
architecture question to the user; do not guess again.
The agent commits the RED test as `test: red for <symptom>` and reports
the handoff carrier for `implement` mini-mode. The agent does NOT write
the fix — splitting RED (this skill) and GREEN (`implement` mini-mode)
across two dispatches keeps the diagnosis honest.
## Handoff Contract
The carrier `debug` hands to `implement`:
`debug` produces, for `implement` mini-mode:
| Field | Content |
|-------|---------|
@@ -118,34 +75,12 @@ The carrier `debug` hands to `implement`:
Anything else (broader refactor, doc rewrite, new feature) is OUT of
scope for the bug-fix iteration and gets queued for a separate one.
## Common Rationalisations
| Excuse | Reality |
|--------|---------|
| "The implementer says it's obviously the new tag-extract emit" | Confident guesses paper over real causes. Repro, isolate, RED test, then fix. |
| "Trivial bug, existing tests will catch it if we break something" | The existing tests already passed while this bug shipped — by definition they don't cover it. |
| "30 seconds to repro, 30 seconds to fix, skip the test" | 30 seconds to repro is also 30 seconds to capture as a regression fixture. Don't trade durable coverage for nothing. |
| "I'll write the test after the fix works" | Test-after proves nothing about whether the test would have caught the bug pre-fix. |
| "Two fixes failed; the third hypothesis is the right one" | Two failures means the hypothesis space is wrong. STOP, dispatch the debugger agent, do not guess again. |
| "End-of-day, just push the fix and write the test tomorrow" | End-of-day pressure is exactly when shotgun fixes corrupt the tree. Clean tree + open bug > three speculative half-fixes merged. |
| "Architecture question is for next milestone, fix-and-revisit" | Phase 4.5 fires NOW or never. Carrying the wrong shape forward compounds. |
## Red Flags — STOP and follow process
- "Quick fix for now, investigate later"
- "Just try changing X and see if it works"
- "I see the symptom, let me fix it"
- "The test is redundant, I manually verified"
- "Pattern says X but I'll adapt it differently"
- *"One more fix attempt"* (when ≥ 2 already failed)
All of these mean: **return to Phase 1**.
## Cross-references
- **Agent dispatched:** `skills/debug/agents/ailang-debugger.md`
carries the actual diagnostic work (read CLAUDE.md / DESIGN.md /
JOURNAL, reproduce, write RED test, propose minimal fix).
carries the four-phase process, the Phase 4.5 escalation rule, the
Common Rationalisations table, and the Red Flags list. The
orchestrator does not execute these phases directly.
- **Hand-off target:** `skills/implement/SKILL.md` — runs the GREEN
side after the RED test is committed.
- **Project source:** former CLAUDE.md "Bug fixes — TDD, always"
+3 -1
View File
@@ -26,7 +26,9 @@ genuinely captures the symptom, not the post-fix code path.
2. `docs/DESIGN.md` — invariants the bug may have crossed.
3. `docs/JOURNAL.md` — most recent entries; the last iteration may have
introduced the bug.
4. `skills/debug/SKILL.md` — the four-phase process you follow.
The four-phase process below is the single source of truth — the
dispatching skill file does not duplicate it.
## Carrier contract — what the controller hands you
+32 -162
View File
@@ -22,6 +22,15 @@ The skill produces a friction-and-bug spec that the next iteration's
`plan` consumes as a reference. The spec sits next to milestone-design
specs at `docs/specs/<date>-fieldtest-<milestone>.md`.
The substantive process — read DESIGN.md + JOURNAL + milestone spec,
pick 2-4 real-world programming tasks per milestone axis, implement
each in `.ailx` Surface form, run via `ail check`/`build`/`run`,
classify findings, write the spec — lives in
`agents/ailang-fieldtester.md`. That file also carries the spec
template, the source-isolation discipline (no reading under
`crates/`, `runtime/`, `bench/`), and the per-finding classification
rules. This skill file only governs trigger, dispatch, and handoff.
## When to Use / Skipping
Triggers:
@@ -53,144 +62,24 @@ EVERY EXAMPLE IS WRITTEN IN .ailx (SURFACE) FIRST. RAW .ail.json IS NEVER HAND-A
EVERY FRICTION POINT AND BUG IS RECORDED. NONE IS WORKED AROUND.
```
The first clause is the load-bearing one. The whole point of the
field test is to simulate a downstream LLM author who has only the
specification and the example corpus. If the dispatched agent reads
`crates/ailang-check/`, `crates/ailang-codegen/`, or any other
implementation source, the test contaminates: it now reflects what
the code happens to support, not what DESIGN.md promises. The carrier
contract enforces this; the agent's standing reading list re-states
it.
The first clause is load-bearing: the whole point of the field test is
to simulate a downstream LLM author who has only the specification and
the example corpus. The agent file enforces this with a hard path
allowlist; the orchestrator must trust that contract and not feed the
agent compiler-internal hints in the carrier.
The second clause exists because `.ail.json` is the canonical storage
form, but `.ailx` is the form an LLM author is meant to reach for.
Hand-writing JSON skips exactly the part of the language we want to
field-test. If the surface form makes a particular pattern painful,
that is a finding — not a reason to drop down to JSON.
## Dispatch
## The Process — five phases
### Phase 1 — Scope the field test
The orchestrator collects:
- the milestone identifier and the JOURNAL entries describing what
shipped,
- the milestone's design spec (`docs/specs/<milestone>.md`) if one
exists,
- a 1-line scope hint per axis the milestone touched (e.g. "23
shipped typeclass dictionaries: try a Functor and an Eq instance",
"20 shipped prose projection: try one prose-first authoring
workflow").
Dispatch `ailang-fieldtester` with the carrier from the Handoff
Contract below. The agent picks 2-4 examples (one per axis the
milestone touched), implements them in `.ailx`, runs them through the
public `ail` CLI, classifies findings, writes the spec, and commits
both fixtures and spec in one commit
(`fieldtest: <milestone> — <N> examples, <K> findings`).
Two to four examples is the right size. One is too few (no signal on
variation); five is too many for one report to stay readable.
### Phase 2 — Dispatch `ailang-fieldtester`
Hand the agent the carrier (see Handoff Contract). The agent:
1. Reads DESIGN.md + JOURNAL tail + milestone spec + a representative
slice of `examples/` (to learn the *form* of valid AIL — not the
*content* of the compiler).
2. Picks the example tasks. Each task is a real, named programming
problem (FizzBuzz, polynomial root via Newton's method, JSON-ish
tree depth, BFS over a small graph, run-length encoder, …) chosen
so it exercises the milestone's deliveries without being a
compiler torture-test. Toy problems find toy bugs.
3. For each task:
- drafts the program in `.ailx` Surface form,
- saves it as `examples/fieldtest/<milestone>_<n>_<slug>.ailx`,
- runs `ail check` and `ail build` and `ail run`,
- records the experience: what the agent reached for, what it had
to back out of, what the diagnostic said, what surprised it,
where DESIGN.md was silent or ambiguous.
The agent has the `ail` CLI as a black box. It must NOT open
`crates/`, `runtime/`, or any internal source. It MAY read other
`.ailx`/`.ail.json` files in `examples/` to learn the form.
### Phase 3 — Friction & bug classification
Each finding is classified into exactly one of:
| Class | Meaning | Downstream action |
|-------|---------|-------------------|
| `bug` | Compiler/runtime misbehaves vs. DESIGN.md (panic, wrong stdout, misleading diagnostic, schema break). | Hand off to `debug` (RED-test first). |
| `friction` | Compiles and runs, but the LLM-author had to fight the surface to express the thing. Surface clumsy, redundancy forced, mode/clause dance ugly. | Feeds next milestone's `brainstorm` or a tidy iteration's `plan`. |
| `spec_gap` | DESIGN.md does not constrain the case; the agent had to guess. The compiler picked a reading; another reading was equally plausible. | Orchestrator decides: tighten DESIGN.md, or ratify the existing reading in DESIGN.md, with a JOURNAL entry. |
| `working` | The author reached for the milestone's new surface and got it right on the first try with a clean diagnostic when wrong. Worth recording — these are the wins that protect the feature from drift attempts later. |
### Phase 4 — Self-review the spec
Before commit, the agent (or the orchestrator on its behalf) checks:
1. **Source-isolation check:** is any finding phrased as "the
compiler does X"? If so, replace with "running `ail run` produced
X". Findings must reference observable behaviour, not internal
mechanism.
2. **Surface-only check:** every shipped example is a `.ailx` file.
No hand-written `.ail.json` for fieldtest examples.
3. **Classification check:** every finding has exactly one class. A
friction-and-bug item gets two entries.
4. **Reproducibility check:** every `bug` carries the one-line repro
command and the observed output.
### Phase 5 — Commit + handoff
```
examples/fieldtest/<milestone>_<n>_<slug>.ailx (Surface form)
examples/fieldtest/<milestone>_<n>_<slug>.ail.json (canonical, generated via `ail render --json` or equivalent)
docs/specs/<date>-fieldtest-<milestone>.md (the report)
```
Commit pattern (one commit per fieldtest run):
```bash
git add examples/fieldtest/*.ail* docs/specs/*-fieldtest-*.md
git commit -m "fieldtest: <milestone> — <N> examples, <K> findings"
```
Hand the orchestrator:
- the spec path,
- the per-finding classification,
- a one-line recommendation per finding (`debug` / `plan` / `ratify` /
`carry-on`).
The orchestrator dispatches the follow-ups; `fieldtest` does NOT
self-resolve.
## Spec structure
`docs/specs/<date>-fieldtest-<milestone>.md`:
```markdown
# Fieldtest — <milestone> — <date>
**Status:** Draft — awaiting orchestrator triage
**Author:** ailang-fieldtester (dispatched by skills/fieldtest)
## Scope
What the milestone shipped. One paragraph.
## Examples
Per example, one subsection:
### `examples/fieldtest/<milestone>_<n>_<slug>.ailx` — <task name>
- What it does
- Why this task fits the milestone's scope
- Outcome: compiles? runs? matches expected stdout?
## Findings
Per finding, one entry:
### [class] <one-line title>
- Example(s) where it surfaced
- What happened (verbatim diagnostic / wrong output / etc.)
- Why it is friction / bug / spec_gap / working
- (for `bug`) one-line repro
- Recommended downstream action
## Recommendation summary
A short table mapping finding → action (`debug` / `plan` / `ratify` /
`carry-on`).
```
## Handoff Contract
`fieldtest` consumes (from orchestrator at milestone close):
@@ -208,44 +97,25 @@ A short table mapping finding → action (`debug` / `plan` / `ratify` /
|-------|---------|
| `spec_path` | `docs/specs/<date>-fieldtest-<milestone>.md` |
| `examples_added` | list of `.ailx` paths committed |
| `findings` | list, each with class + recommendation |
| `findings` | list, each with class (`bug` / `friction` / `spec_gap` / `working`) + recommendation |
| `status` | `clean` / `friction_found` / `bugs_found` / `infra_blocked` |
The orchestrator drives downstream: bugs → `debug`; friction or
spec_gap → next `brainstorm` or a tidy iteration via `plan`.
The orchestrator drives downstream:
- `bug``debug` (RED-first; GREEN via `implement` mini-mode)
- `friction` → next `brainstorm` or a tidy iteration via `plan`
- `spec_gap` → ratify in JOURNAL + DESIGN.md, or tighten DESIGN.md
- `working` → carry-on (worth recording, no follow-up)
## Common Rationalisations
| Excuse | Reality |
|--------|---------|
| "I'll just peek at `crates/ailang-check/` to understand the new typeclass coherence rule" | Peeking ends the test. The whole skill exists because the downstream LLM author cannot peek. If DESIGN.md does not answer the question, that is a `spec_gap` finding — the most valuable kind. |
| "Hand-writing JSON is faster, the surface is just sugar" | The surface is the field-test target. JSON-direct skips it. If you want to skip the surface, you have skipped the field test. |
| "Diagnostic is unclear but I figured out the fix; the example is fine" | The diagnostic being unclear *is* the friction. Record it verbatim. |
| "Two examples is enough, both worked" | Two working examples on the milestone's primary axis is a thin signal. Pick at least one example per axis the milestone touched. |
| "DESIGN.md does not say what should happen here, I'll just pick the obvious reading" | The "obvious" reading IS the spec gap. Record it. The orchestrator decides whether to ratify or tighten. |
| "The example would be cleaner if I added one helper, even though the milestone didn't ship it" | The field test runs the language as shipped. Helpers you wish existed go in the friction list, not into the example. |
| "Bug found, I'll just fix it while I'm here, faster than handing off" | Fix-in-place violates the skill split. Record as `bug`, hand to `debug` (RED-first). The fix lands in a separate, tested commit. |
| "audit said clean, I'm sure the field test will too — skip" | audit and fieldtest answer different questions. audit: does code match DESIGN? fieldtest: can a downstream LLM use what shipped? Both gates exist for a reason. |
## Red Flags — STOP
- About to open any file under `crates/`, `runtime/`, or `bench/`'s
internals
- About to hand-write `.ail.json` for a fieldtest example
- About to silently swap an example to a smaller form because the
intended one didn't compile (the failure-to-compile IS the finding)
- About to merge a `bug` finding into a `friction` finding "because
it's borderline" — pick one class
- About to skip Phase 4 self-review because "I wrote it carefully"
- About to attempt the bug fix in this dispatch (the skill is RED-by-
observation; GREEN goes to `debug``implement`)
- About to recommend more than one downstream action per finding
`fieldtest` does NOT self-resolve.
## Cross-references
- **Agent dispatched:** `skills/fieldtest/agents/ailang-fieldtester.md`
— carries the five-phase process, the Iron Law in operational form,
the spec template, the per-finding classification rules, the Common
Rationalisations table, and the Red Flags list.
- **Upstream gate:** `skills/audit/SKILL.md` runs first; fieldtest
comes after a clean (or ratified) audit
comes after a clean (or ratified) audit.
- **Downstream targets:**
- `skills/debug/SKILL.md` — for each `bug` finding
- `skills/brainstorm/SKILL.md` or `skills/plan/SKILL.md` — for
+35 -2
View File
@@ -164,8 +164,7 @@ merged.
### Phase 5 — Write the spec, commit, hand back
Write `docs/specs/<YYYY-MM-DD>-fieldtest-<milestone>.md` using the
structure documented in `skills/fieldtest/SKILL.md` ("Spec
structure"). Then:
spec structure below. Then:
```bash
git add examples/fieldtest/*.ail* docs/specs/*-fieldtest-*.md
@@ -174,6 +173,40 @@ git commit -m "fieldtest: <milestone> — <N> examples, <K> findings"
Report `DONE` with the carrier (see Output format).
## Spec structure
`docs/specs/<date>-fieldtest-<milestone>.md`:
```markdown
# Fieldtest — <milestone> — <date>
**Status:** Draft — awaiting orchestrator triage
**Author:** ailang-fieldtester (dispatched by skills/fieldtest)
## Scope
What the milestone shipped. One paragraph.
## Examples
Per example, one subsection:
### `examples/fieldtest/<milestone>_<n>_<slug>.ailx` — <task name>
- What it does
- Why this task fits the milestone's scope
- Outcome: compiles? runs? matches expected stdout?
## Findings
Per finding, one entry:
### [class] <one-line title>
- Example(s) where it surfaced
- What happened (verbatim diagnostic / wrong output / etc.)
- Why it is friction / bug / spec_gap / working
- (for `bug`) one-line repro
- Recommended downstream action
## Recommendation summary
A short table mapping finding → action (`debug` / `plan` / `ratify` /
`carry-on`).
```
## Status protocol
End every report with exactly one of: