From adcea02e8540e75e13672e210b59ecaa9c39c86f Mon Sep 17 00:00:00 2001 From: Brummel Date: Mon, 11 May 2026 10:56:24 +0200 Subject: [PATCH] spec: implement-orchestrator-agent Three coupled changes to /implement: - Boss-context offload via single subagent dispatch (target ~100x reduction) - Iter isolation via branches (eliminates parallel-implementer conflicts) - Per-iter journals under docs/journals/ replacing the JOURNAL.md monolith JOURNAL.md becomes docs/journal-archive.md at acceptance time. Workers stay at Opus 4.7; saving comes from token-volume relocation only. --- ...2026-05-11-implement-orchestrator-agent.md | 545 ++++++++++++++++++ 1 file changed, 545 insertions(+) create mode 100644 docs/specs/2026-05-11-implement-orchestrator-agent.md diff --git a/docs/specs/2026-05-11-implement-orchestrator-agent.md b/docs/specs/2026-05-11-implement-orchestrator-agent.md new file mode 100644 index 0000000..560470c --- /dev/null +++ b/docs/specs/2026-05-11-implement-orchestrator-agent.md @@ -0,0 +1,545 @@ +# `ailang-implement-orchestrator` — Design Spec + +**Date:** 2026-05-11 +**Status:** Draft — awaiting user spec review +**Authors:** Brummel (orchestrator) + Claude + +## Goal + +Three coupled changes to `/implement`, each individually motivated, +together resolving the dominant cost and workflow-coordination +problems observed on 2026-05-11: + +1. **Boss-context offload.** Reduce the Boss-Orchestrator's + per-`/implement` context cost from ~100k tokens to ~700 tokens + (factor ~100×) by delegating the entire per-task loop to a single + subagent — `ailang-implement-orchestrator` — running in its own + context. + +2. **Iter isolation via branches.** Every `/implement` run owns a + dedicated branch (`iter/`). The Boss-Orchestrator + decides per run whether and when to integrate. Eliminates the + "parallel implementer pushed 37 commits while I tried to commit + a spec"-class of problem. + +3. **Per-iter journals replace the JOURNAL.md monolith.** Iteration + reports move from a single 14k-line `docs/JOURNAL.md` to per-iter + files in `docs/journals/-iter-.md`. The existing + JOURNAL.md is archived as `docs/journal-archive.md`. A short + `docs/journals/INDEX.md` provides chronological navigation, edited + by the Boss as part of the merge step. + +Quality of decisions is non-negotiable: model selection stays at +Opus 4.7 for every role; the saving comes from token-volume +relocation, not from model downgrades. + +## Background — what the current `/implement` actually costs + +Anthropic-side usage statistics for the project (2026-05-11): + +``` +30% /implement (skill) +26% subagents under "implement" +12% /plan +50% of usage at >150k context +49% of usage in 8h+ sessions +``` + +A per-task breakdown of today's `/implement` flow in the Boss-context: + +| Step | Tokens (typical) | +|------|------------------| +| Plan loaded once (Step 1) — 478–1883 lines via `Read` | 12k–25k *per iter* | +| Per-task implementer dispatch — `task_text` verbatim | 2k–7k | +| Implementer report | ~300 | +| Per-task spec-reviewer dispatch — `task_text` AGAIN | 2k–7k *(duplicate)* | +| Spec-reviewer report | ~350 | +| Per-task quality-reviewer dispatch — SHAs only | ~100 | +| Quality-reviewer report | ~450 | +| Re-loop on `non_compliant` (~50% rate) | × 1.5 | + +For an 8-task iteration: ~12–25k (plan) + ~88k (per-task ×8 ×1.5) ≈ +**100k tokens in the Boss-context**. Combined with multi-iter +sessions this explains the >150k tail and the cost concentration the +statistics flag. + +In parallel, two structural issues surfaced: +- Concurrent `/implement` activity (separate session pushing to + `main` while the Boss was working) leaves the working tree in a + detached-HEAD state with no clean spot to commit a small artefact + like a spec file. +- `docs/JOURNAL.md` is 14309 lines. Skill steps that "read tail" + routinely pull thousands of tokens of which 90% is irrelevant to + the current iter; cross-iter searches require scrolling rather + than file-system tools. + +## Architecture + +``` +Boss-Orchestrator + │ + │ Agent("ailang-implement-orchestrator", carrier) + ▼ +ailang-implement-orchestrator ← runs in its own context + │ + │ git fetch origin main + │ git switch -c iter/ origin/main + │ + │ reads plan (once, in its context) + │ per task K = 1..N: + │ Agent("ailang-implementer", task_carrier_K) ── carrier points to /tmp/ail-iter//task-K.md + │ on DONE: Agent("ailang-spec-reviewer", review_carrier_K) + │ on compliant: Agent("ailang-quality-reviewer", quality_carrier_K) + │ loop on non_compliant / NEEDS_CONTEXT (≤ 2 retries) + │ on BLOCKED or 3rd retry: stop, emit PARTIAL bundle + │ Agent("ailang-tester", coverage_carrier) ── E2E coverage + │ write docs/journals/-iter-.md ── per-iter journal + │ commit journal file on the branch + │ write bench/orchestrator-stats/<...>.json + │ commit stats file on the branch + ▼ +compressed end-report (~500 tokens) — pointer to journal file + branch name + +then Boss-Orchestrator (separate, after reading end-report): + │ + │ reviews docs/journals/<...>.md on the branch + │ updates docs/journals/INDEX.md with one line + │ git rebase iter/ onto main ── linear history + │ git branch -D iter/ ── optional cleanup +``` + +The Boss-Orchestrator sees: one dispatch out, one compressed report +back, then a Boss-side merge step that touches one INDEX line. +Per-task loops, plan reads, review re-loops, journal writing, stats +writing — all happen inside the orchestrator-agent's context and +never reach the Boss. + +`skills/README.md` currently states "Agents do not call other +agents." `ailang-implement-orchestrator` becomes the documented +exception, named explicitly. The rule remains otherwise binding — +no other agent gains the `Agent` tool. + +## Components + +### `ailang-implement-orchestrator` (new agent) + +Location: `skills/implement/agents/ailang-implement-orchestrator.md`. + +Tools: `Read, Edit, Write, Bash, Glob, Grep, Agent`. The `Agent` +tool is the explicit exception — it dispatches the existing four +worker agents (implementer, spec-reviewer, quality-reviewer, +tester). + +Standing reading list: +1. `CLAUDE.md` — orchestrator framing. +2. `docs/DESIGN.md` — invariants the iteration must respect. +3. `docs/journals/INDEX.md` + the last 1–3 per-iter journal files + linked from there — recent state. (After the migration window, + no agent reads `docs/journal-archive.md` directly; long-tail + context lives there but is referenced by date if needed.) +4. `skills/implement/SKILL.md` — **canonical discipline** (Iron + Law, Step-2 sub-status mechanics, per-task scope rules). The + orchestrator reads this verbatim every dispatch; it does not + paraphrase. + +Model: Opus 4.7. + +First and last actions in every dispatch: +- **First**: `git fetch origin main && git switch -c iter/ origin/main`. + Ensures a clean base from current Remote regardless of what state + the local working tree was in. +- **Last (on DONE)**: write per-iter journal file, commit it on the + branch, write stats file, commit it on the branch, return + end-report. Do not merge — that's Boss-side. +- **Last (on BLOCKED)**: write per-iter journal file with the + partial status, commit it on the branch, write stats file with + `outcome: PARTIAL` or `outcome: BLOCKED`, commit it, return + end-report. Branch is left for the Boss to inspect, repair, or + delete. + +### `skills/implement/SKILL.md` (rewritten) + +Today: ~210 lines covering Step 1 (plan load), Step 2.1–2.5 +(per-task dispatch), Step 3 (tester), Step 4 (JOURNAL), Mini-Mode +handoff, Common Rationalisations, Red Flags. + +After: ~100 lines. Both Standard-Mode and Mini-Mode collapse to: + +> 1. Dispatch `ailang-implement-orchestrator` with the carrier +> below. +> 2. Read the returned end-report. +> 3. **Boss merge step (on DONE):** open the per-iter journal at +> `docs/journals/<...>.md` on the branch. Accept as-is or +> condense; append a one-line entry to `docs/journals/INDEX.md`; +> `git rebase iter/ onto main`; optionally delete +> branch. +> 4. **Boss handling (on BLOCKED):** inspect the partial branch, +> repair (re-dispatch with adjusted plan / extended context), +> or discard (`git branch -D iter/`). + +The Iron Law, per-task sub-status table, and Common Rationalisations +stay in this file — they remain the canonical discipline source, +now read by the orchestrator-agent. What goes away from the skill +body is the *procedural Boss-side* of each per-task step (no +Boss-side per-task step remains). + +### Worker agents — unchanged in role, minor carrier change + +`ailang-implementer`, `ailang-spec-reviewer`, +`ailang-quality-reviewer`, `ailang-tester` keep their current +frontmatter, tools, standing reading lists, and status protocols. +Only their *caller* changes from the Boss-Orchestrator to the +`ailang-implement-orchestrator`. + +One mechanical change to carriers: instead of `task_text` as a +verbatim string, implementer and spec-reviewer receive +`task_text_path` (`/tmp/ail-iter//task-.md`) and read +the file themselves. Eliminates per-task duplication of the task +text between the two agents. + +### `docs/journals/` — new subdir + +Replaces `docs/JOURNAL.md` going forward. + +Naming: `docs/journals/-iter-.md`. Example: +`docs/journals/2026-05-11-iter-ct.2.3.md`. + +File content, written by the orchestrator-agent: + +```markdown +# iter — + +**Date:** YYYY-MM-DD +**Branch:** iter/<id> +**Status:** DONE | PARTIAL | BLOCKED +**Tasks completed:** <N> of <total> + +## Summary + +<1-paragraph, Boss may rewrite during merge> + +## Per-task subjects + +- iter <id>.1: <subject> +- iter <id>.2: <subject> +- ... + +## Concerns + +(DONE_WITH_CONCERNS aggregate, one line each) + +## Known debt + +(one-liner each, with why-not-touched) + +## Blocked detail + +(only if BLOCKED / PARTIAL: task N, reason, worker-quote, suggested next step) + +## Commits + +<pre-iter SHA>..<head SHA> + +## Stats + +bench/orchestrator-stats/<file>.json +``` + +The Summary section is what the Boss may rewrite when integrating — +the agent's draft is a starting point. The rest is factual and +preserved verbatim. + +### `docs/journals/INDEX.md` — new file + +Boss-maintained chronological index. Append-only, one line per iter: + +```markdown +# Journal index + +- 2026-05-11 — iter ct.2.3: typecheck cleanup → 2026-05-11-iter-ct.2.3.md +- 2026-05-12 — bugfix sigsegv-pattern-ctor → 2026-05-12-iter-bugfix-sigsegv-pattern-ctor.md +- ... +``` + +Written by the Boss as part of the merge step, never by the +orchestrator-agent. Filenames are relative to `docs/journals/`. + +### `docs/journal-archive.md` — renamed historical JOURNAL.md + +At spec-acceptance time, `docs/JOURNAL.md` is renamed to +`docs/journal-archive.md`. Content is preserved verbatim — it +remains the canonical record for everything pre-migration. + +A single header note is added at the top: + +```markdown +> **Status:** archived 2026-05-11. New iter journals live under +> `docs/journals/`. See `docs/journals/INDEX.md` for the +> chronological pointer list. This file remains the historical +> record for entries prior to that date. +``` + +`CLAUDE.md` is updated in the same migration commit to reflect the +new layout. Existing cross-references to "JOURNAL.md" in skill +files and agent files are updated to point at the new layout +(`docs/journals/INDEX.md` + recent entries). + +### `bench/orchestrator-stats/<YYYY-MM-DD>-iter-<X>.json` — unchanged + +Per-iter JSON, written by the orchestrator before the journal +commit. Pointer in the end-report. No aggregation script ships +with this spec; once a few iters accumulate, a separate utility +can compute distributions. + +### `skills/README.md` (one-line update) + +The "Agents do not call other agents" subsection gains a sentence +naming `ailang-implement-orchestrator` as the dispatch-and-coordinate +exception. + +## Data flow + +### Carrier — Boss → `ailang-implement-orchestrator` + +| Field | Content | +|-------|---------| +| `mode` | `"standard"` or `"mini"` | +| `iter_id` | e.g. `"ct.2.3"` (standard) or `"bugfix-<short-symptom>"` (mini) — used for branch name, paths (stats file, scratch dir, journal file), JOURNAL header | +| `plan_path` | (standard only) `docs/plans/<file>.md` | +| `task_range` | (standard, optional) e.g. `[3, 8]` to run only Tasks 3..8 | +| `red_test_path` | (mini only) absolute path to the failing test from `debug` | +| `cause_summary` | (mini only) 1–2 sentences from the debugger agent | +| `constraint` | (mini only) `"minimal fix, no surrounding cleanup"` | + +### Carrier — `ailang-implement-orchestrator` → worker agents + +Implementer (per task): + +| Field | Content | +|-------|---------| +| `task_text_path` | `/tmp/ail-iter/<iter_id>/task-<N>.md` — orchestrator extracts the verbatim task block here on first reference | +| `scene_set` | parent milestone, position in iter, dependencies | +| `cross_task_context` | shared types, naming conventions for the iter | +| `mode` | `standard` or `mini` | + +Spec-reviewer: + +| Field | Content | +|-------|---------| +| `task_text_path` | same file as the implementer received | +| `pre_task_sha`, `head_sha` | for `git diff <pre>..<head>` | +| `status_from_implementer` | `DONE` or `DONE_WITH_CONCERNS` (with concerns) | + +Quality-reviewer: + +| Field | Content | +|-------|---------| +| `pre_task_sha`, `head_sha` | for the diff | +| `spec_review_status` | must be `compliant` | +| `task_subject` | one-line title, NOT the full task text | + +Tester (once per iter, at end, on DONE only): + +| Field | Content | +|-------|---------| +| `iteration_scope` | what shipped — feature name, commit range, invariants | +| `coverage_gap` | if known | +| `mode` | `e2e_after_iter` | + +### End-report — `ailang-implement-orchestrator` → Boss + +Plain-text report, ≤ 500 tokens, fixed structure: + +``` +Status: DONE | PARTIAL | BLOCKED +Iter: <iter_id> +Branch: iter/<iter_id> +Tasks completed: <N> of <total> + - <commit subject 1> + - <commit subject 2> + ... +Journal file: docs/journals/<file>.md (on branch) +Stats: bench/orchestrator-stats/<file>.json (on branch) +Commits: <pre-iter SHA>..<head SHA> +Tests: <count> green, <count> red +E2E coverage: <new fixture paths> +Blocked detail: (only if BLOCKED) + Task: <N> + Reason: context-exhausted | review-loop-exhausted | worker-blocked + Worker says: <verbatim reason> + Suggested next step: <one sentence> +``` + +The Boss reads this directly; the per-iter journal file is read +selectively only when integrating or when investigating BLOCKED. + +### Boss-side merge workflow (on DONE) + +1. Read `docs/journals/<file>.md` from the branch + (`git show iter/<iter_id>:docs/journals/<file>.md` or check out + if a closer look is needed). +2. Accept the agent's summary as-is, or rewrite the Summary section + to reflect Boss-level framing. +3. Append a one-line entry to `docs/journals/INDEX.md`. +4. `git rebase iter/<iter_id> onto main` for linear history. +5. Optionally `git branch -D iter/<iter_id>`. +6. If trigger is done-state and user-away: write `WhatsNew.md` entry + and `notify.sh` per CLAUDE.md. + +### Boss-side handling (on BLOCKED / PARTIAL) + +1. Read the per-iter journal on the branch — the `Blocked detail:` + section names the failure mode. +2. Decide: + - **Repair**: adjust plan, re-dispatch the orchestrator with the + same `iter_id` and `task_range` covering the remaining tasks. + The orchestrator picks up the existing branch (`git switch + iter/<iter_id>`) instead of creating a new one. + - **Discard**: `git branch -D iter/<iter_id>`. The per-iter + journal file disappears with the branch; INDEX.md is not + touched (no entry was added). + - **Escalate**: ask the user. The branch + journal file sit until + the conversation resumes. + +## Error handling — per-task sub-status mechanics + +The orchestrator follows this table verbatim. The same table is the +canonical reference in `skills/implement/SKILL.md`. + +| Sub-status | Orchestrator action | +|------------|---------------------| +| `DONE` | next task / next phase | +| `DONE_WITH_CONCERNS` | accumulate concern, next | +| `NEEDS_CONTEXT` (1st–2nd) | re-dispatch with expanded carrier | +| `NEEDS_CONTEXT` (3rd) | stop → `BLOCKED` to Boss, reason `context-exhausted` | +| `non_compliant` / `changes_requested` (1st–2nd) | implementer re-dispatch with the reviewer's report as repair brief | +| `non_compliant` / `changes_requested` (3rd) | stop → `BLOCKED` to Boss, reason `review-loop-exhausted` | +| `BLOCKED` (worker) | stop → `BLOCKED` to Boss, reason verbatim | +| Tool / infra error | stop → `BLOCKED` to Boss, reason `infra` + raw error | + +On `BLOCKED`: +- already-committed tasks stay on the branch +- the stats file is written with `outcome: PARTIAL` and the partial + per-task list +- the per-iter journal file IS written (with `Status: PARTIAL` or + `Status: BLOCKED`), committed on the branch +- end-report carries `Status: BLOCKED` with detail +- `docs/journals/INDEX.md` is **not** touched (Boss decides on + integration when handling the branch) + +`skip task K, continue` is intentionally NOT a mode — tasks have +implicit ordering dependencies and the orchestrator does not know +the plan dependency graph. Skip is a Boss decision. + +## Testing strategy + +This is a skill/agent refactor, not a language feature. No +`cargo test` coverage applies directly. Verification ladder: + +1. **Stand-alone agent file parses** — `Agent("ailang-implement-orchestrator", ...)` + resolves to a runnable spec. Check via a no-op dispatch on a + trivial fixture iter. +2. **Branch isolation** — dispatch the orchestrator; verify + `git branch` after run shows `iter/<test-iter>` distinct from + main; main is unchanged. +3. **Single-task standard-mode run** — dispatch the orchestrator on + a one-task plan. Expect: green build, green tests, journal file + on branch, stats file on branch, end-report follows the fixed + format. +4. **Multi-task standard-mode run** — same on a 3+ task plan. +5. **Mini-mode run** — dispatch from `debug` handoff. Expect: same + end-report shape, single-task stats, branch named `iter/bugfix-<>`. +6. **Re-loop run** — feed an intentionally non-compliant task. + Expect: 2 re-dispatches, then `BLOCKED` with reason + `review-loop-exhausted`. Branch preserved with partial commits. +7. **BLOCKED run** — feed an intentionally impossible task. Expect: + immediate `BLOCKED`, partial commits preserved on branch. +8. **Boss-merge step** — manually integrate a successful test run: + read journal file, write INDEX line, rebase, delete branch. + Verify linear history and INDEX coherence. + +Verification steps 6, 7, 8 are one-shot reproducers — they confirm +the discipline once. Future stats data answers "is the re-loop +limit too tight / too loose" empirically. + +## Acceptance criteria + +The refactor is accepted when: + +1. `skills/implement/agents/ailang-implement-orchestrator.md` exists, + following the standard agent template (frontmatter, spirit-letter + lead-in, standing reading list, carrier contract, Iron Law, + process, status protocol, output format, common rationalisations, + red flags). Contains the branch-creation step as Process Step 1 + and the journal-file-write step as a late-process action. +2. `skills/implement/SKILL.md` is rewritten to delegate both modes + to the orchestrator-agent and to document the Boss-side merge + step. The canonical discipline (Iron Law, sub-status table, + common rationalisations) remains in this file. +3. `skills/README.md` documents the orchestrator-agent as the named + exception to "agents do not call other agents". +4. `skills/implement/agents/ailang-implementer.md` and + `skills/implement/agents/ailang-spec-reviewer.md` accept + `task_text_path` in place of `task_text` in their carriers. +5. `docs/JOURNAL.md` is renamed to `docs/journal-archive.md` with + the archived-status header. `docs/journals/INDEX.md` exists + (initially with a single line linking back to + `journal-archive.md` for pre-migration context). +6. Skill files and agent files that reference `docs/JOURNAL.md` are + updated to point at the new layout. `CLAUDE.md` reflects the new + roles of `docs/journals/`, `docs/journals/INDEX.md`, and + `docs/journal-archive.md`. +7. The eight verification steps in Testing strategy pass. +8. The first real `/implement` run after the refactor lands shows a + Boss-context delta consistent with the predicted factor (target: + ≤ 1k tokens per `/implement` in the Boss-context for the agent + dispatch + end-report, plus ≤ 1k tokens for the Boss merge step). + +## Migration + +The migration touches three categories of files. All happen in a +single commit (or one commit per category, atomically). + +**1. Filesystem.** +- `git mv docs/JOURNAL.md docs/journal-archive.md` +- Add archived-status header to `docs/journal-archive.md` +- Create empty `docs/journals/` directory with `.gitkeep` (or with + the initial INDEX.md) +- Create `docs/journals/INDEX.md` with the header and a single + pointer line referencing the archive for pre-migration entries + +**2. References.** +- Scan all files for `JOURNAL.md` and update each reference: + - `CLAUDE.md` "Roles of …" section: new descriptions for + `docs/journals/`, `docs/journals/INDEX.md`, + `docs/journal-archive.md` + - `skills/*/SKILL.md` files: "read JOURNAL.md tail" → "read + `docs/journals/INDEX.md` and the latest 1–3 referenced files" + - `skills/*/agents/*.md` standing reading lists: same update + - `docs/DESIGN.md` if it references JOURNAL.md: same update + +**3. Workflow.** +- Existing plans in `docs/plans/` work unchanged — the plan format + is unchanged. +- Existing Iter-22+ plans can be re-run end-to-end through the new + orchestrator without rewriting; only the journal-write target + differs (per-iter file instead of appending to JOURNAL.md). +- WhatsNew.md and `notify.sh` workflow unchanged. + +## Out of scope (deferred to other specs) + +- **Worker model downgrade** (cheap models for simple subagents). + User has explicitly excluded this; all roles stay at Opus 4.7. +- **Stats aggregation tool** — written once empirical data exists. +- **Plan-drafter sub-agent** for the `/plan` skill. Separate hebel, + separate spec. +- **`/brainstorm` spec-drafter** — usage statistic shows + `/brainstorm` at 2%, not cost-effective. +- **`docs/DESIGN.md` changes**. The refactor is below the language + spec; the canonical AILang language definition is untouched. +- **External-platform integration** (Gitea PR via `tea`, GitHub + spiegelung). Decided 2026-05-11: the Per-Iter-File pattern in the + Git repo is the single source of truth; external platform UI is + redundant because Gitea's native file-browser already serves + these files read-only once pushed. May be reconsidered if + team-review workflow appears later.