From 64b0841c5af682219a520d76b3d132d1e08aafe8 Mon Sep 17 00:00:00 2001 From: Brummel Date: Sun, 10 May 2026 10:53:53 +0200 Subject: [PATCH] skills: deduplicate single-agent SKILL.md files (debug, fieldtest) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The four-phase debug process and the five-phase fieldtest process each lived twice — once in SKILL.md (orchestrator-facing) and once in the dispatched agent's file (subagent-facing). The orchestrator does not execute these phases; the subagent does. Duplicating them in SKILL.md just bloated the orchestrator's main context with bytes that only the subagent ever needs. SKILL.md now carries only what the orchestrator must consult at dispatch time: trigger gating, Iron Law as headline, the carrier contract, the produced handoff, and cross-references. Iron Law in operational form, full process, Common Rationalisations, Red Flags, and (for fieldtest) the spec template now live solely in the agent file. Net −160 LOC across the skills tree. CLAUDE.md pointer for 'Bug fixes — TDD, always' updated to reflect that the substantive discipline lives in the agent file, not the skill file. --- CLAUDE.md | 7 +- skills/debug/SKILL.md | 109 ++-------- skills/debug/agents/ailang-debugger.md | 4 +- skills/fieldtest/SKILL.md | 194 +++--------------- skills/fieldtest/agents/ailang-fieldtester.md | 37 +++- 5 files changed, 97 insertions(+), 254 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 232127a..ab4cec1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -177,9 +177,12 @@ exception, not the rule. ## Bug fixes — TDD, always -Bug fixes are RED-first, autonomous, no orchestrator gate. Full +Bug fixes are RED-first, autonomous, no orchestrator gate. The +trigger and handoff are in `skills/debug/SKILL.md`; the full discipline (Iron Law, four phases, Phase 4.5 architecture trigger, -common rationalisations) lives in `skills/debug/SKILL.md`. Headline: +common rationalisations) lives in +`skills/debug/agents/ailang-debugger.md` — that is the file the +dispatched subagent reads. Headline: 1. **RED first** — write a failing test that pins down the symptom, commit it as `test: red for ` BEFORE any fix. diff --git a/skills/debug/SKILL.md b/skills/debug/SKILL.md index a1f318a..92df54b 100644 --- a/skills/debug/SKILL.md +++ b/skills/debug/SKILL.md @@ -15,6 +15,13 @@ skill then drives the fix to GREEN. Skipping the RED stage — even for "trivial" bugs — produces fixes that don't stick and tests that don't exist to catch the next regression. +The substantive process — root cause investigation → pattern analysis +→ RED test → handoff, plus the Phase 4.5 architecture-question trigger +after three failed hypotheses — lives in +`agents/ailang-debugger.md`. That file is the single source of truth +for the discipline; this skill file only governs trigger, dispatch, +and handoff. + ## When to Use / Skipping Trigger this skill on: @@ -40,74 +47,24 @@ Both clauses are non-negotiable. The second was formerly the CLAUDE.md "Bug fixes — TDD, always" section before the 2026-05-09 skill-system migration. -## The Process — four phases +## Dispatch -Each phase must complete before the next starts. +Dispatch `ailang-debugger` with: -### Phase 1 — Root Cause Investigation +| Carrier field | Content | +|---------------|---------| +| `symptom` | Exact error message, stack trace, wrong output, or repro command | +| `repro_known` | One-line repro if known, otherwise the agent finds one | +| `recent_iter` | Iteration that last touched the suspected area (often `git log` tail) | -1. Read the error message in full. Stack traces, line numbers, exit - codes — none of them get skimmed. -2. Reproduce with the shortest possible command. If you can't repro - reliably, gather more data — do NOT guess. -3. Check recent changes: `git log --oneline -20`, - `git diff ..HEAD`. -4. For multi-component issues (compiler → emitter → linker; or - runtime → C glue → binary), instrument every boundary. Find the - layer where the data goes wrong before deciding which layer to - fix. -5. Trace data flow backward from symptom to source. Fix at source, - not at symptom. - -### Phase 2 — Pattern Analysis - -1. Find a similar working example in the codebase. What's different? -2. If the bug is in a pattern (e.g. ADT match lowering, RC drop - emission), read the reference implementation completely. No - skimming. -3. List every difference between working and broken — however small. - -### Phase 3 — Hypothesis and Test - -1. State the hypothesis as a single falsifiable claim: - *"X is the root cause because Y."* -2. **Write the failing test FIRST**, before any fix attempt: - - smallest possible reproducer (E2E in `crates/ail/tests/e2e.rs`, - or unit test in the affected crate) - - automated, deterministic — same input always same output - - doc comment names the property the test protects, not just - what it does -3. Run it to confirm it fails on the current code, with the symptom - the bug-report described. -4. Commit the failing test as a separate commit: - `git commit -m "test: red for "`. - -### Phase 4 — Hand off to `implement` - -The fix itself is `implement`'s job, run in mini-mode. Hand it: -- path to the RED-test file -- 1-2 sentence cause summary from Phase 1 -- hard constraint: *"minimal fix, no surrounding cleanup, - no opportunistic refactor"* - -`implement` runs the GREEN side and commits `fix: `. - -### Phase 4.5 — 3+ failures = architecture question - -If the third hypothesis fails: - -``` -STOP. Do not attempt Fix #4. -``` - -Three failed fixes in a row indicate the architecture is wrong, not -that the latest hypothesis is wrong. Each fix likely revealed new -shared state or coupling in a different place. Surface the -architecture question to the user; do not guess again. +The agent commits the RED test as `test: red for ` and reports +the handoff carrier for `implement` mini-mode. The agent does NOT write +the fix — splitting RED (this skill) and GREEN (`implement` mini-mode) +across two dispatches keeps the diagnosis honest. ## Handoff Contract -The carrier `debug` hands to `implement`: +`debug` produces, for `implement` mini-mode: | Field | Content | |-------|---------| @@ -118,34 +75,12 @@ The carrier `debug` hands to `implement`: Anything else (broader refactor, doc rewrite, new feature) is OUT of scope for the bug-fix iteration and gets queued for a separate one. -## Common Rationalisations - -| Excuse | Reality | -|--------|---------| -| "The implementer says it's obviously the new tag-extract emit" | Confident guesses paper over real causes. Repro, isolate, RED test, then fix. | -| "Trivial bug, existing tests will catch it if we break something" | The existing tests already passed while this bug shipped — by definition they don't cover it. | -| "30 seconds to repro, 30 seconds to fix, skip the test" | 30 seconds to repro is also 30 seconds to capture as a regression fixture. Don't trade durable coverage for nothing. | -| "I'll write the test after the fix works" | Test-after proves nothing about whether the test would have caught the bug pre-fix. | -| "Two fixes failed; the third hypothesis is the right one" | Two failures means the hypothesis space is wrong. STOP, dispatch the debugger agent, do not guess again. | -| "End-of-day, just push the fix and write the test tomorrow" | End-of-day pressure is exactly when shotgun fixes corrupt the tree. Clean tree + open bug > three speculative half-fixes merged. | -| "Architecture question is for next milestone, fix-and-revisit" | Phase 4.5 fires NOW or never. Carrying the wrong shape forward compounds. | - -## Red Flags — STOP and follow process - -- "Quick fix for now, investigate later" -- "Just try changing X and see if it works" -- "I see the symptom, let me fix it" -- "The test is redundant, I manually verified" -- "Pattern says X but I'll adapt it differently" -- *"One more fix attempt"* (when ≥ 2 already failed) - -All of these mean: **return to Phase 1**. - ## Cross-references - **Agent dispatched:** `skills/debug/agents/ailang-debugger.md` — - carries the actual diagnostic work (read CLAUDE.md / DESIGN.md / - JOURNAL, reproduce, write RED test, propose minimal fix). + carries the four-phase process, the Phase 4.5 escalation rule, the + Common Rationalisations table, and the Red Flags list. The + orchestrator does not execute these phases directly. - **Hand-off target:** `skills/implement/SKILL.md` — runs the GREEN side after the RED test is committed. - **Project source:** former CLAUDE.md "Bug fixes — TDD, always" diff --git a/skills/debug/agents/ailang-debugger.md b/skills/debug/agents/ailang-debugger.md index 6d188d7..bd9aa94 100644 --- a/skills/debug/agents/ailang-debugger.md +++ b/skills/debug/agents/ailang-debugger.md @@ -26,7 +26,9 @@ genuinely captures the symptom, not the post-fix code path. 2. `docs/DESIGN.md` — invariants the bug may have crossed. 3. `docs/JOURNAL.md` — most recent entries; the last iteration may have introduced the bug. -4. `skills/debug/SKILL.md` — the four-phase process you follow. + +The four-phase process below is the single source of truth — the +dispatching skill file does not duplicate it. ## Carrier contract — what the controller hands you diff --git a/skills/fieldtest/SKILL.md b/skills/fieldtest/SKILL.md index 5664fe9..84606b1 100644 --- a/skills/fieldtest/SKILL.md +++ b/skills/fieldtest/SKILL.md @@ -22,6 +22,15 @@ The skill produces a friction-and-bug spec that the next iteration's `plan` consumes as a reference. The spec sits next to milestone-design specs at `docs/specs/-fieldtest-.md`. +The substantive process — read DESIGN.md + JOURNAL + milestone spec, +pick 2-4 real-world programming tasks per milestone axis, implement +each in `.ailx` Surface form, run via `ail check`/`build`/`run`, +classify findings, write the spec — lives in +`agents/ailang-fieldtester.md`. That file also carries the spec +template, the source-isolation discipline (no reading under +`crates/`, `runtime/`, `bench/`), and the per-finding classification +rules. This skill file only governs trigger, dispatch, and handoff. + ## When to Use / Skipping Triggers: @@ -53,144 +62,24 @@ EVERY EXAMPLE IS WRITTEN IN .ailx (SURFACE) FIRST. RAW .ail.json IS NEVER HAND-A EVERY FRICTION POINT AND BUG IS RECORDED. NONE IS WORKED AROUND. ``` -The first clause is the load-bearing one. The whole point of the -field test is to simulate a downstream LLM author who has only the -specification and the example corpus. If the dispatched agent reads -`crates/ailang-check/`, `crates/ailang-codegen/`, or any other -implementation source, the test contaminates: it now reflects what -the code happens to support, not what DESIGN.md promises. The carrier -contract enforces this; the agent's standing reading list re-states -it. +The first clause is load-bearing: the whole point of the field test is +to simulate a downstream LLM author who has only the specification and +the example corpus. The agent file enforces this with a hard path +allowlist; the orchestrator must trust that contract and not feed the +agent compiler-internal hints in the carrier. -The second clause exists because `.ail.json` is the canonical storage -form, but `.ailx` is the form an LLM author is meant to reach for. -Hand-writing JSON skips exactly the part of the language we want to -field-test. If the surface form makes a particular pattern painful, -that is a finding — not a reason to drop down to JSON. +## Dispatch -## The Process — five phases - -### Phase 1 — Scope the field test - -The orchestrator collects: -- the milestone identifier and the JOURNAL entries describing what - shipped, -- the milestone's design spec (`docs/specs/.md`) if one - exists, -- a 1-line scope hint per axis the milestone touched (e.g. "23 - shipped typeclass dictionaries: try a Functor and an Eq instance", - "20 shipped prose projection: try one prose-first authoring - workflow"). +Dispatch `ailang-fieldtester` with the carrier from the Handoff +Contract below. The agent picks 2-4 examples (one per axis the +milestone touched), implements them in `.ailx`, runs them through the +public `ail` CLI, classifies findings, writes the spec, and commits +both fixtures and spec in one commit +(`fieldtest: — examples, findings`). Two to four examples is the right size. One is too few (no signal on variation); five is too many for one report to stay readable. -### Phase 2 — Dispatch `ailang-fieldtester` - -Hand the agent the carrier (see Handoff Contract). The agent: -1. Reads DESIGN.md + JOURNAL tail + milestone spec + a representative - slice of `examples/` (to learn the *form* of valid AIL — not the - *content* of the compiler). -2. Picks the example tasks. Each task is a real, named programming - problem (FizzBuzz, polynomial root via Newton's method, JSON-ish - tree depth, BFS over a small graph, run-length encoder, …) chosen - so it exercises the milestone's deliveries without being a - compiler torture-test. Toy problems find toy bugs. -3. For each task: - - drafts the program in `.ailx` Surface form, - - saves it as `examples/fieldtest/__.ailx`, - - runs `ail check` and `ail build` and `ail run`, - - records the experience: what the agent reached for, what it had - to back out of, what the diagnostic said, what surprised it, - where DESIGN.md was silent or ambiguous. - -The agent has the `ail` CLI as a black box. It must NOT open -`crates/`, `runtime/`, or any internal source. It MAY read other -`.ailx`/`.ail.json` files in `examples/` to learn the form. - -### Phase 3 — Friction & bug classification - -Each finding is classified into exactly one of: - -| Class | Meaning | Downstream action | -|-------|---------|-------------------| -| `bug` | Compiler/runtime misbehaves vs. DESIGN.md (panic, wrong stdout, misleading diagnostic, schema break). | Hand off to `debug` (RED-test first). | -| `friction` | Compiles and runs, but the LLM-author had to fight the surface to express the thing. Surface clumsy, redundancy forced, mode/clause dance ugly. | Feeds next milestone's `brainstorm` or a tidy iteration's `plan`. | -| `spec_gap` | DESIGN.md does not constrain the case; the agent had to guess. The compiler picked a reading; another reading was equally plausible. | Orchestrator decides: tighten DESIGN.md, or ratify the existing reading in DESIGN.md, with a JOURNAL entry. | -| `working` | The author reached for the milestone's new surface and got it right on the first try with a clean diagnostic when wrong. Worth recording — these are the wins that protect the feature from drift attempts later. | - -### Phase 4 — Self-review the spec - -Before commit, the agent (or the orchestrator on its behalf) checks: -1. **Source-isolation check:** is any finding phrased as "the - compiler does X"? If so, replace with "running `ail run` produced - X". Findings must reference observable behaviour, not internal - mechanism. -2. **Surface-only check:** every shipped example is a `.ailx` file. - No hand-written `.ail.json` for fieldtest examples. -3. **Classification check:** every finding has exactly one class. A - friction-and-bug item gets two entries. -4. **Reproducibility check:** every `bug` carries the one-line repro - command and the observed output. - -### Phase 5 — Commit + handoff - -``` -examples/fieldtest/__.ailx (Surface form) -examples/fieldtest/__.ail.json (canonical, generated via `ail render --json` or equivalent) -docs/specs/-fieldtest-.md (the report) -``` - -Commit pattern (one commit per fieldtest run): - -```bash -git add examples/fieldtest/*.ail* docs/specs/*-fieldtest-*.md -git commit -m "fieldtest: — examples, findings" -``` - -Hand the orchestrator: -- the spec path, -- the per-finding classification, -- a one-line recommendation per finding (`debug` / `plan` / `ratify` / - `carry-on`). - -The orchestrator dispatches the follow-ups; `fieldtest` does NOT -self-resolve. - -## Spec structure - -`docs/specs/-fieldtest-.md`: - -```markdown -# Fieldtest — — - -**Status:** Draft — awaiting orchestrator triage -**Author:** ailang-fieldtester (dispatched by skills/fieldtest) - -## Scope -What the milestone shipped. One paragraph. - -## Examples -Per example, one subsection: -### `examples/fieldtest/__.ailx` — -- What it does -- Why this task fits the milestone's scope -- Outcome: compiles? runs? matches expected stdout? - -## Findings -Per finding, one entry: -### [class] -- Example(s) where it surfaced -- What happened (verbatim diagnostic / wrong output / etc.) -- Why it is friction / bug / spec_gap / working -- (for `bug`) one-line repro -- Recommended downstream action - -## Recommendation summary -A short table mapping finding → action (`debug` / `plan` / `ratify` / -`carry-on`). -``` - ## Handoff Contract `fieldtest` consumes (from orchestrator at milestone close): @@ -208,44 +97,25 @@ A short table mapping finding → action (`debug` / `plan` / `ratify` / |-------|---------| | `spec_path` | `docs/specs/-fieldtest-.md` | | `examples_added` | list of `.ailx` paths committed | -| `findings` | list, each with class + recommendation | +| `findings` | list, each with class (`bug` / `friction` / `spec_gap` / `working`) + recommendation | | `status` | `clean` / `friction_found` / `bugs_found` / `infra_blocked` | -The orchestrator drives downstream: bugs → `debug`; friction or -spec_gap → next `brainstorm` or a tidy iteration via `plan`. +The orchestrator drives downstream: +- `bug` → `debug` (RED-first; GREEN via `implement` mini-mode) +- `friction` → next `brainstorm` or a tidy iteration via `plan` +- `spec_gap` → ratify in JOURNAL + DESIGN.md, or tighten DESIGN.md +- `working` → carry-on (worth recording, no follow-up) -## Common Rationalisations - -| Excuse | Reality | -|--------|---------| -| "I'll just peek at `crates/ailang-check/` to understand the new typeclass coherence rule" | Peeking ends the test. The whole skill exists because the downstream LLM author cannot peek. If DESIGN.md does not answer the question, that is a `spec_gap` finding — the most valuable kind. | -| "Hand-writing JSON is faster, the surface is just sugar" | The surface is the field-test target. JSON-direct skips it. If you want to skip the surface, you have skipped the field test. | -| "Diagnostic is unclear but I figured out the fix; the example is fine" | The diagnostic being unclear *is* the friction. Record it verbatim. | -| "Two examples is enough, both worked" | Two working examples on the milestone's primary axis is a thin signal. Pick at least one example per axis the milestone touched. | -| "DESIGN.md does not say what should happen here, I'll just pick the obvious reading" | The "obvious" reading IS the spec gap. Record it. The orchestrator decides whether to ratify or tighten. | -| "The example would be cleaner if I added one helper, even though the milestone didn't ship it" | The field test runs the language as shipped. Helpers you wish existed go in the friction list, not into the example. | -| "Bug found, I'll just fix it while I'm here, faster than handing off" | Fix-in-place violates the skill split. Record as `bug`, hand to `debug` (RED-first). The fix lands in a separate, tested commit. | -| "audit said clean, I'm sure the field test will too — skip" | audit and fieldtest answer different questions. audit: does code match DESIGN? fieldtest: can a downstream LLM use what shipped? Both gates exist for a reason. | - -## Red Flags — STOP - -- About to open any file under `crates/`, `runtime/`, or `bench/`'s - internals -- About to hand-write `.ail.json` for a fieldtest example -- About to silently swap an example to a smaller form because the - intended one didn't compile (the failure-to-compile IS the finding) -- About to merge a `bug` finding into a `friction` finding "because - it's borderline" — pick one class -- About to skip Phase 4 self-review because "I wrote it carefully" -- About to attempt the bug fix in this dispatch (the skill is RED-by- - observation; GREEN goes to `debug` → `implement`) -- About to recommend more than one downstream action per finding +`fieldtest` does NOT self-resolve. ## Cross-references - **Agent dispatched:** `skills/fieldtest/agents/ailang-fieldtester.md` + — carries the five-phase process, the Iron Law in operational form, + the spec template, the per-finding classification rules, the Common + Rationalisations table, and the Red Flags list. - **Upstream gate:** `skills/audit/SKILL.md` runs first; fieldtest - comes after a clean (or ratified) audit + comes after a clean (or ratified) audit. - **Downstream targets:** - `skills/debug/SKILL.md` — for each `bug` finding - `skills/brainstorm/SKILL.md` or `skills/plan/SKILL.md` — for diff --git a/skills/fieldtest/agents/ailang-fieldtester.md b/skills/fieldtest/agents/ailang-fieldtester.md index 0eb3008..a929dda 100644 --- a/skills/fieldtest/agents/ailang-fieldtester.md +++ b/skills/fieldtest/agents/ailang-fieldtester.md @@ -164,8 +164,7 @@ merged. ### Phase 5 — Write the spec, commit, hand back Write `docs/specs/-fieldtest-.md` using the -structure documented in `skills/fieldtest/SKILL.md` ("Spec -structure"). Then: +spec structure below. Then: ```bash git add examples/fieldtest/*.ail* docs/specs/*-fieldtest-*.md @@ -174,6 +173,40 @@ git commit -m "fieldtest: — examples, findings" Report `DONE` with the carrier (see Output format). +## Spec structure + +`docs/specs/-fieldtest-.md`: + +```markdown +# Fieldtest — — + +**Status:** Draft — awaiting orchestrator triage +**Author:** ailang-fieldtester (dispatched by skills/fieldtest) + +## Scope +What the milestone shipped. One paragraph. + +## Examples +Per example, one subsection: +### `examples/fieldtest/__.ailx` — +- What it does +- Why this task fits the milestone's scope +- Outcome: compiles? runs? matches expected stdout? + +## Findings +Per finding, one entry: +### [class] +- Example(s) where it surfaced +- What happened (verbatim diagnostic / wrong output / etc.) +- Why it is friction / bug / spec_gap / working +- (for `bug`) one-line repro +- Recommended downstream action + +## Recommendation summary +A short table mapping finding → action (`debug` / `plan` / `ratify` / +`carry-on`). +``` + ## Status protocol End every report with exactly one of: