diff --git a/docs/specs/2026-05-09-skill-system.md b/docs/specs/2026-05-09-skill-system.md new file mode 100644 index 0000000..afc0a32 --- /dev/null +++ b/docs/specs/2026-05-09-skill-system.md @@ -0,0 +1,401 @@ +# AILang Skill System — Design Spec + +**Date:** 2026-05-09 +**Status:** Draft — awaiting user spec review +**Authors:** Brummel (orchestrator) + Claude + +## Goal + +Formalise the existing AILang development workflow as a five-skill system +that lives under `skills/`. The skills codify discipline that is currently +spread across `CLAUDE.md` and the orchestrator's habits, and they emit +durable artefacts (specs, plans) that future iterations can reference. + +The orchestrator (the user) remains the boss: skills are sharper tools, not +a replacement for judgement. Where the existing per-iter workflow already +works (small focused commits, JOURNAL discipline, agent delegation), it is +preserved verbatim. The new layer formalises the *pre-iter design step* +(spec) and the *iter handoff contract* (plan), and turns implicit +discipline (tidy-iter at family close, RED-first for bugs) into explicit +skill invocations. + +This is the bootstrap spec. It defines the system that future specs will +go through. Once the system is in place, every new family starts with a +spec under `docs/specs/`, and every iter starts with a plan under +`docs/plans/`. + +## Architecture + +### Five skills + +All five skills are project-local under `skills/`. Names are imperative +verbs without iter-/family- scope prefixes — the trigger condition is +encoded in each skill's `description` frontmatter, not in its name. + +| Skill | Trigger | Output | Verbindlichkeit | +|-------|---------|--------|-----------------| +| `brainstorm` | New family starting | `docs/specs/.md` | Hard-gate before plan | +| `plan` | New iter within an open family | `docs/plans/.md` | Hard-gate before implement | +| `implement` | Plan exists | Code + tests + per-task commits + JOURNAL entry | Standard iter path | +| `audit` | Family closing OR baseline drift suspected | Drift report + bench-regression report | **Mandatory** at family close | +| `debug` | Bug encountered (red test, segfault, wrong stdout) | RED-test commit + cause analysis | **Mandatory RED-first** for any bug | + +### Directory layout + +``` +skills/ + brainstorm/ + SKILL.md + plan/ + SKILL.md + implement/ + SKILL.md + agents/ + ailang-implementer.md + ailang-tester.md + audit/ + SKILL.md + agents/ + ailang-architect.md + ailang-bencher.md + debug/ + SKILL.md + agents/ + ailang-debugger.md +agents/ + ailang-docwriter.md # orphan — invoked directly by orchestrator + README.md # roster: pointers to skills/ + orphans +docs/ + specs/ # NEW — emitted by brainstorm + plans/ # NEW — emitted by plan + DESIGN.md # unchanged + JOURNAL.md # unchanged +``` + +`brainstorm` and `plan` have no private agents — both run as a dialogue +between orchestrator and user, producing the artefact directly. + +`implement`, `audit`, and `debug` each own the agents they dispatch. An +agent definition lives next to the skill that primarily uses it. + +`ailang-docwriter` stays in `agents/` as an orphan tool. It is rarely +invoked, exclusively for rustdoc drift, and does not slot into any of the +five skill pipelines. + +### `.claude/agents/` discovery + +Subagent-type discovery requires the agent definitions to be reachable +from `.claude/agents/`. Pragmatic resolution: one symlink per skill, +created at install time: + +``` +.claude/agents/implement -> ../../skills/implement/agents +.claude/agents/audit -> ../../skills/audit/agents +.claude/agents/debug -> ../../skills/debug/agents +.claude/agents/orphan -> ../../agents +``` + +This keeps the agent files visible in the project tree (per the existing +convention in `agents/README.md`) while still letting the Claude harness +find them. + +## Components + +### CLAUDE.md migration + +CLAUDE.md splits into **orchestrator identity** (stays) and **discipline +detail** (moves into skills, with a one-line reference back from CLAUDE.md). + +**Stays in CLAUDE.md:** + +- Project mission ("Invent your own programming language") +- Code layout table +- "My role: orchestrator" (delegation rules, when not to delegate) +- "Design rationale ≠ implementation effort" +- "Direction freedom" + "Notifications" +- "Roles of JOURNAL.md and DESIGN.md" + +**Moves into skills (CLAUDE.md keeps a one-line reference):** + +| CLAUDE.md section | Destination skill | +|-------------------|-------------------| +| "Bug fixes — TDD, always" (RED first / GREEN second / keep the test) | `debug` (Iron Law) | +| "Tidy-iter at family boundaries" (mandatory, drift review) | `audit` (trigger logic + escalation) | +| "Performance regressions" (bench/check.py + compile_check.py + cross_lang.py, exit codes 0/1/2) | `audit` (bench discipline) | +| "Feature acceptance: LLM utility" (full criterion, beyond the CLAUDE.md summary) | `brainstorm` (feature gate during spec writing) | + +CLAUDE.md keeps a short pointer in each migrated section, e.g.: + +> **Bug fixes** are RED-first TDD, autonomous, no orchestrator gate. +> Full discipline lives in `skills/debug/SKILL.md`. + +This way `CLAUDE.md` shrinks but a reader who only opens it still gets +the headline rule and a pointer to the binding source. + +### Agent migration + +Agent definitions move via `git mv`. Frontmatter (`name`, `description`, +`tools`) is preserved unchanged. The "mandatory reading order" sections +inside each agent stay — they are agent discipline, not skill discipline. + +The dirigierende skill provides the **task context** (plan extract, bug +symptom, drift item) so the agent does not have to rediscover it. This is +the controller-curates-context pattern from +`superpowers:subagent-driven-development`. + +Migration map: + +``` +agents/ailang-implementer.md -> skills/implement/agents/ailang-implementer.md +agents/ailang-tester.md -> skills/implement/agents/ailang-tester.md +agents/ailang-architect.md -> skills/audit/agents/ailang-architect.md +agents/ailang-bencher.md -> skills/audit/agents/ailang-bencher.md +agents/ailang-debugger.md -> skills/debug/agents/ailang-debugger.md +agents/ailang-docwriter.md -> agents/ailang-docwriter.md (no move) +agents/README.md -> agents/README.md (rewritten as skills+orphans roster) +``` + +## Data flow + +### Pipeline 1: family happy path + +``` +[orchestrator: new family starting] + | + v +brainstorm + Q&A dialogue with user (one question at a time, 2-3 approaches, design) + write docs/specs/.md + commit: "spec: " + | + v +[orchestrator: first iter of family] + | + v +plan + read docs/specs/.md + write docs/plans/.md (header references parent spec) + commit: "plan: " + | + v +implement + read docs/plans/.md + dispatch implementer (per task) -> spec-review -> code-quality-review + dispatch tester (E2E coverage) + per-task commits: "iter .: " + JOURNAL entry: "## YYYY-MM-DD — Iter : " +``` + +### Pipeline 2: family close path + +``` +[orchestrator: family-tidy due] + | + v +audit + dispatch architect -> drift report + run bench/check.py && bench/compile_check.py && bench/cross_lang.py + classify: 0 (clean) | 1 (drift/regression) | 2 (infra failure) + report to orchestrator + | + +--- exit 0 (clean) -------> JOURNAL: "Family-<X> tidy-iter (clean)" + | + +--- exit 1 (drift/regression) + | | + | v + | plan + implement (tidy-iter) OR ratify in DESIGN.md + | commit: "iter <X>.tidy: <fix>" OR JOURNAL ratify entry + | + +--- exit 2 (infra failure) + | + v + fix infrastructure FIRST, re-run audit +``` + +### Pipeline 3: bug autonomous TDD path + +``` +[orchestrator: bug observed] + | + v +debug + dispatch debugger + reproduce + isolate + write RED test (failing on current code) + commit: "test: red for <symptom>" + diagnosis report (Symptom, Cause, Fix-Vorschlag) + hand-off: bug-context + RED-test path + | + v +implement (mini-mode, no docs/plans/ file for trivial fixes) + dispatch implementer with RED-test as goal: "make this test pass, minimal change" + commit: "fix: <symptom>" + JOURNAL entry: "## YYYY-MM-DD — Bug fix: <symptom>" +``` + +### Handoff contracts + +Every handoff carries an explicit, named context. No implicit "the next +skill knows what to do": + +| Handoff | Carrier | +|---------|---------| +| `brainstorm` -> `plan` | path to `docs/specs/<family>.md` + iter scope ("the section X+Y of the spec is what this iter covers") | +| `plan` -> `implement` | path to `docs/plans/<iter>.md` + optional task focus ("only Tasks 1-3 this run") | +| `debug` -> `implement` | RED-test path + 1-2-sentence cause summary + hard constraint ("minimal fix, no surrounding cleanup") | +| `audit` -> orchestrator | prioritised drift items + bench exit code + raw bench numbers + recommendation ("fix N", "ratify M", "carry on") | + +### Skipping rules (codified in the skills, not ad-hoc) + +| Skill | May be skipped when | Never skipped when | +|-------|---------------------|--------------------| +| `brainstorm` | Iter is tidy / bug fix / trivial mechanic (≤1 file, ≤30 LOC, no design judgement) | New family starts | +| `plan` | Bug fix where RED-test is sufficient as plan; trivial mechanic | Standard iter within an active family | +| `implement` | — | Always required when there is code to ship | +| `audit` | — | **Always** at family close; deferral requires a JOURNAL entry naming the reason and the re-run date | +| `debug` | — | **Always** for any observable bug; trivial bugs still get RED first | + +Each skill's `SKILL.md` repeats the relevant skipping clause from this +table verbatim — the rule lives in the skill, not just here. + +### Commit-per-stage invariant + +Each pipeline stage commits its own artefact before handing off. Specs, +plans, RED-tests, and per-task implementation diffs are separate commits. +This makes every stage independently revertable: `git revert <plan-commit>` +discards the plan only, leaving the spec intact. The convention also +gives `audit` clean drift-review boundaries (commit ranges per family +correspond to spec ranges). + +## Error handling + +### Escalation ladder + +Common across all five skills: + +| Level | Decider | Trigger | +|-------|---------|---------| +| L1 — skill-internal recovery | The skill itself | Subagent reports `NEEDS_CONTEXT` -> provide more context, redispatch (max 2x with same model) | +| L2 — bounce to previous skill | The skill | `plan` cannot decompose spec -> back to `brainstorm`; implementer finds spec/code contradiction -> back to `plan` | +| L3 — orchestrator decision | Me | Task too large to fit one plan -> split; reasoning insufficient -> stronger model; design-rationale fork | +| L4 — user decision | User | Spec/DESIGN.md conflict; 3+ failed fix attempts in `debug` (architecture question); `audit` drift item where fix vs ratify is unclear | + +### Skill-specific stop conditions + +- **`brainstorm`** — After 3 approach iterations with no convergence, + STOP. User decides whether the family is dropped (commit message + `wip: drop spec, scope unclear`) or paused. +- **`plan`** — Spec containing a placeholder ("TBD", "implement later") + bounces back to `brainstorm`. The plan itself MUST contain no + placeholders (writing-plans Iron Rule, adopted verbatim). +- **`implement`** — implementer status `BLOCKED` with reason + "task too large" -> split + new plan. Status `BLOCKED` with reason + "design conflict" -> escalate to user. Never push past `BLOCKED` by + hand (context pollution). +- **`debug`** — Phase 4.5 trigger: 3+ failed fix attempts mean the + pattern is wrong, not the latest hypothesis. STOP, surface the + architecture question to the user. No bug fix without a RED-test + first; no "I'll write the test after I've confirmed the fix". +- **`audit`** — Bench exit code 2 (infrastructure failure) means fix + infrastructure FIRST and never report it as a regression. Drift items + where fix-vs-ratify is genuinely unclear escalate to the user. + +### Cross-skill rule + +> **Violating the letter of these rules is violating the spirit.** + +This single sentence (lifted verbatim from +`superpowers:systematic-debugging`) lives at the top of every SKILL.md. +"It's obvious, so we can skip the spec/plan/audit step" is not a valid +argument — the codified skipping rules in Section 3 are the only +sanctioned way to skip. Ad-hoc judgement that a step is unnecessary +is exactly the failure mode the system exists to prevent. + +## Testing strategy + +Skills are developed under TDD per `superpowers:writing-skills`: RED +baseline first, GREEN minimal skill second, REFACTOR loopholes third. + +### Per-skill pressure scenarios (for the RED phase) + +| Skill | Scenarios | +|-------|-----------| +| `brainstorm` | (a) "Idea is clear, write the plan now" — must hold the spec hard-gate. (b) "This family is trivial, no spec needed" — must cite the skipping rule or run anyway. (c) "Just sketch a one-paragraph spec, full design is overkill" — must produce a real spec or block. | +| `plan` | (a) "TBD steps are fine, I'll fill them in later" — No-Placeholders rule. (b) "Bundle these as one big step, faster" — bite-sized rule. (c) "Skip the spec, I know the family already" — bounce to `brainstorm`. | +| `implement` | (a) "Skip spec-review, the implementer self-reviewed" — both reviews still required. (b) "Commit straight to main" — never without explicit user consent. (c) "Implementer says DONE_WITH_CONCERNS, move on" — must read concerns first. | +| `debug` | (a) "Cause is obvious, fix it directly" — reproduce + RED first. (b) "Bug is trivial, no test needed" — TDD anyway. (c) "Fix #3 failed, one more try" — Phase 4.5 architecture question, no Fix #4. | +| `audit` | (a) "Tidy-iter later, we're in a hurry" — mandatory at family close. (b) "Bench is red, just bump the baseline" — only with `--update-baseline` AND a JOURNAL ratify entry. (c) "Drift item doesn't matter, ignore it" — must be either fixed or ratified. | + +### Verification sequence per skill + +1. **RED** — A general-purpose subagent receives the pressure scenario + without the skill's content. Document its rationalisations and + shortcuts verbatim. +2. **GREEN** — Write the SKILL.md with the documented rationalisations + as explicit counters in a "Common Rationalisations" table. Hand the + skill content + scenario to a fresh subagent. It must comply. +3. **REFACTOR** — A third subagent run, same scenario plus minor + variations, looking for new loopholes. Add explicit counters for + each found loophole. Re-test until bulletproof. + +Minimum **3 independent subagent runs** in the GREEN+REFACTOR phases +per skill. A single passing run is not evidence — random compliance is +possible. + +### Acceptance criterion per skill + +A skill ships (= is committed to `skills/`) only after: + +- 3 RED scenarios have been run and the rationalisations documented in + the skill's `Common Rationalisations` table. +- 3+ GREEN/REFACTOR runs all pass without further amendment. +- The skill's `description` frontmatter is technology-agnostic and + trigger-focused (no workflow summary — see CSO section in + `superpowers:writing-skills`). + +## Implementation order + +The skills depend on each other transitively (a `plan` skill cannot be +built without a working `brainstorm` skill upstream of it), but the test +discipline does not require the dependency chain to be live for testing +— the pressure scenarios are self-contained. + +Recommended order, scoped as iter increments under a new family: + +1. **22c.1** — `debug` skill (smallest scope, clearest Iron Law, no + upstream dependency since bugs do not flow through brainstorm/plan). +2. **22c.2** — `audit` skill (also independent of brainstorm/plan; + migrates the existing tidy-iter discipline). +3. **22c.3** — `implement` skill (consumes plans, dispatches + implementer + tester; depends on plan format being defined). +4. **22c.4** — `plan` skill (produces what `implement` consumes). +5. **22c.5** — `brainstorm` skill (top of the pipeline). +6. **22c.6** — CLAUDE.md split + agent migration + `.claude/agents/` + symlink setup. +7. **22c.tidy** — first audit on the new infrastructure itself. + +Each iter ships its skill behind the existing discipline (CLAUDE.md +rules apply during the build of the new system; the new skills do not +self-apply until 22c.6 lands). The first family to use the new system +end-to-end is the next family after 22c. + +## Acceptance criteria for the system as a whole + +The skill system is considered shipped when: + +- All five SKILL.md files exist under `skills/<name>/SKILL.md` with + passing pressure tests. +- Agent definitions have moved to their target locations and + `.claude/agents/` symlinks resolve. +- CLAUDE.md is split: orchestrator-identity stays, migrated sections + carry one-line pointers to the relevant skill. +- `docs/specs/` and `docs/plans/` exist with at least this spec and + the implementation plan derived from it as their first inhabitants. +- A JOURNAL entry records the system as live and names the next family + as the first end-to-end consumer. + +The headline test is: **the next family after 22c is brainstormed, +planned, implemented, and audited entirely through the skill system, +with no ad-hoc orchestrator improvisation.** If the orchestrator +catches themselves bypassing a skill mid-family, that is a skill bug +to be fixed in a follow-up iter — not a permitted shortcut.