AILang/docs/specs/2026-05-11-implement-orchestrator-agent.md

ailang-implement-orchestrator — Design Spec

Date: 2026-05-11 Status: Draft — awaiting user spec review Authors: Brummel (orchestrator) + Claude

Goal

Three coupled changes to /implement, each individually motivated, together resolving the dominant cost and workflow-coordination problems observed on 2026-05-11:

1. Boss-context offload. Reduce the Boss-Orchestrator's per-/implement context cost from ~100k tokens to ~700 tokens (factor ~100×) by delegating the entire per-task loop to a single subagent — ailang-implement-orchestrator — running in its own context.

2. Iter isolation via branches. Every /implement run owns a dedicated branch (iter/<iter_id>). The Boss-Orchestrator decides per run whether and when to integrate. Eliminates the "parallel implementer pushed 37 commits while I tried to commit a spec"-class of problem.

3. Per-iter journals replace the JOURNAL.md monolith. Iteration reports move from a single 14k-line docs/JOURNAL.md to per-iter files in docs/journals/<YYYY-MM-DD>-iter-<id>.md. The existing JOURNAL.md is archived as docs/journal-archive.md. A short docs/journals/INDEX.md provides chronological navigation, edited by the Boss as part of the merge step.

Quality of decisions is non-negotiable: model selection stays at Opus 4.7 for every role; the saving comes from token-volume relocation, not from model downgrades.

Background — what the current /implement actually costs

Anthropic-side usage statistics for the project (2026-05-11):

30% /implement (skill)
26% subagents under "implement"
12% /plan
50% of usage at >150k context
49% of usage in 8h+ sessions

A per-task breakdown of today's /implement flow in the Boss-context:

Step	Tokens (typical)
Plan loaded once (Step 1) — 478–1883 lines via Read	12k–25k per iter
Per-task implementer dispatch — task_text verbatim	2k–7k
Implementer report	~300
Per-task spec-reviewer dispatch — task_text AGAIN	2k–7k (duplicate)
Spec-reviewer report	~350
Per-task quality-reviewer dispatch — SHAs only	~100
Quality-reviewer report	~450
Re-loop on non_compliant (~50% rate)	× 1.5

For an 8-task iteration: ~12–25k (plan) + ~88k (per-task ×8 ×1.5) ≈ 100k tokens in the Boss-context. Combined with multi-iter sessions this explains the >150k tail and the cost concentration the statistics flag.

In parallel, two structural issues surfaced:

• Concurrent /implement activity (separate session pushing to main while the Boss was working) leaves the working tree in a detached-HEAD state with no clean spot to commit a small artefact like a spec file.
• docs/JOURNAL.md is 14309 lines. Skill steps that "read tail" routinely pull thousands of tokens of which 90% is irrelevant to the current iter; cross-iter searches require scrolling rather than file-system tools.

Architecture

Boss-Orchestrator
    │
    │ Agent("ailang-implement-orchestrator", carrier)
    ▼
ailang-implement-orchestrator         ← runs in its own context
    │
    │   git fetch origin main
    │   git switch -c iter/<iter_id> origin/main
    │
    │   reads plan (once, in its context)
    │   per task K = 1..N:
    │       Agent("ailang-implementer", task_carrier_K)        ── carrier points to /tmp/ail-iter/<iter_id>/task-K.md
    │       on DONE:   Agent("ailang-spec-reviewer", review_carrier_K)
    │       on compliant: Agent("ailang-quality-reviewer", quality_carrier_K)
    │       loop on non_compliant / NEEDS_CONTEXT (≤ 2 retries)
    │       on BLOCKED or 3rd retry: stop, emit PARTIAL bundle
    │   Agent("ailang-tester", coverage_carrier)      ── E2E coverage
    │   write docs/journals/<YYYY-MM-DD>-iter-<id>.md   ── per-iter journal
    │   commit journal file on the branch
    │   write bench/orchestrator-stats/<...>.json
    │   commit stats file on the branch
    ▼
compressed end-report (~500 tokens) — pointer to journal file + branch name

then Boss-Orchestrator (separate, after reading end-report):
    │
    │   reviews docs/journals/<...>.md on the branch
    │   updates docs/journals/INDEX.md with one line
    │   git rebase iter/<iter_id> onto main   ── linear history
    │   git branch -D iter/<iter_id>          ── optional cleanup

The Boss-Orchestrator sees: one dispatch out, one compressed report back, then a Boss-side merge step that touches one INDEX line. Per-task loops, plan reads, review re-loops, journal writing, stats writing — all happen inside the orchestrator-agent's context and never reach the Boss.

skills/README.md currently states "Agents do not call other agents." ailang-implement-orchestrator becomes the documented exception, named explicitly. The rule remains otherwise binding — no other agent gains the Agent tool.

Components

ailang-implement-orchestrator (new agent)

Location: skills/implement/agents/ailang-implement-orchestrator.md.

Tools: Read, Edit, Write, Bash, Glob, Grep, Agent. The Agent tool is the explicit exception — it dispatches the existing four worker agents (implementer, spec-reviewer, quality-reviewer, tester).

Standing reading list:

1. CLAUDE.md — orchestrator framing.
2. docs/DESIGN.md — invariants the iteration must respect.
3. docs/journals/INDEX.md + the last 1–3 per-iter journal files linked from there — recent state. (After the migration window, no agent reads docs/journal-archive.md directly; long-tail context lives there but is referenced by date if needed.)
4. skills/implement/SKILL.md — canonical discipline (Iron Law, Step-2 sub-status mechanics, per-task scope rules). The orchestrator reads this verbatim every dispatch; it does not paraphrase.

Model: Opus 4.7.

First and last actions in every dispatch:

• First: git fetch origin main && git switch -c iter/<iter_id> origin/main. Ensures a clean base from current Remote regardless of what state the local working tree was in.
• Last (on DONE): write per-iter journal file, commit it on the branch, write stats file, commit it on the branch, return end-report. Do not merge — that's Boss-side.
• Last (on BLOCKED): write per-iter journal file with the partial status, commit it on the branch, write stats file with outcome: PARTIAL or outcome: BLOCKED, commit it, return end-report. Branch is left for the Boss to inspect, repair, or delete.

skills/implement/SKILL.md (rewritten)

Today: ~210 lines covering Step 1 (plan load), Step 2.1–2.5 (per-task dispatch), Step 3 (tester), Step 4 (JOURNAL), Mini-Mode handoff, Common Rationalisations, Red Flags.

After: ~100 lines. Both Standard-Mode and Mini-Mode collapse to:

1. Dispatch ailang-implement-orchestrator with the carrier below.
2. Read the returned end-report.
3. Boss merge step (on DONE): open the per-iter journal at docs/journals/<...>.md on the branch. Accept as-is or condense; append a one-line entry to docs/journals/INDEX.md; git rebase iter/<iter_id> onto main; optionally delete branch.
4. Boss handling (on BLOCKED): inspect the partial branch, repair (re-dispatch with adjusted plan / extended context), or discard (git branch -D iter/<iter_id>).

The Iron Law, per-task sub-status table, and Common Rationalisations stay in this file — they remain the canonical discipline source, now read by the orchestrator-agent. What goes away from the skill body is the procedural Boss-side of each per-task step (no Boss-side per-task step remains).

Worker agents — unchanged in role, minor carrier change

ailang-implementer, ailang-spec-reviewer, ailang-quality-reviewer, ailang-tester keep their current frontmatter, tools, standing reading lists, and status protocols. Only their caller changes from the Boss-Orchestrator to the ailang-implement-orchestrator.

One mechanical change to carriers: instead of task_text as a verbatim string, implementer and spec-reviewer receive task_text_path (/tmp/ail-iter/<iter_id>/task-<N>.md) and read the file themselves. Eliminates per-task duplication of the task text between the two agents.

docs/journals/ — new subdir

Replaces docs/JOURNAL.md going forward.

Naming: docs/journals/<YYYY-MM-DD>-iter-<id>.md. Example: docs/journals/2026-05-11-iter-ct.2.3.md.

File content, written by the orchestrator-agent:

# iter <id> — <title>

**Date:** YYYY-MM-DD
**Branch:** iter/<id>
**Status:** DONE | PARTIAL | BLOCKED
**Tasks completed:** <N> of <total>

## Summary

<1-paragraph, Boss may rewrite during merge>

## Per-task subjects

- iter <id>.1: <subject>
- iter <id>.2: <subject>
- ...

## Concerns

(DONE_WITH_CONCERNS aggregate, one line each)

## Known debt

(one-liner each, with why-not-touched)

## Blocked detail

(only if BLOCKED / PARTIAL: task N, reason, worker-quote, suggested next step)

## Commits

<pre-iter SHA>..<head SHA>

## Stats

bench/orchestrator-stats/<file>.json

The Summary section is what the Boss may rewrite when integrating — the agent's draft is a starting point. The rest is factual and preserved verbatim.

docs/journals/INDEX.md — new file

Boss-maintained chronological index. Append-only, one line per iter:

# Journal index

- 2026-05-11 — iter ct.2.3: typecheck cleanup → 2026-05-11-iter-ct.2.3.md
- 2026-05-12 — bugfix sigsegv-pattern-ctor → 2026-05-12-iter-bugfix-sigsegv-pattern-ctor.md
- ...

Written by the Boss as part of the merge step, never by the orchestrator-agent. Filenames are relative to docs/journals/.

docs/journal-archive.md — renamed historical JOURNAL.md

At spec-acceptance time, docs/JOURNAL.md is renamed to docs/journal-archive.md. Content is preserved verbatim — it remains the canonical record for everything pre-migration.

A single header note is added at the top:

> **Status:** archived 2026-05-11. New iter journals live under
> `docs/journals/`. See `docs/journals/INDEX.md` for the
> chronological pointer list. This file remains the historical
> record for entries prior to that date.

CLAUDE.md is updated in the same migration commit to reflect the new layout. Existing cross-references to "JOURNAL.md" in skill files and agent files are updated to point at the new layout (docs/journals/INDEX.md + recent entries).

bench/orchestrator-stats/<YYYY-MM-DD>-iter-<X>.json — unchanged

Per-iter JSON, written by the orchestrator before the journal commit. Pointer in the end-report. No aggregation script ships with this spec; once a few iters accumulate, a separate utility can compute distributions.

skills/README.md (one-line update)

The "Agents do not call other agents" subsection gains a sentence naming ailang-implement-orchestrator as the dispatch-and-coordinate exception.

Data flow

Carrier — Boss → ailang-implement-orchestrator

Field	Content
mode	"standard" or "mini"
iter_id	e.g. "ct.2.3" (standard) or "bugfix-<short-symptom>" (mini) — used for branch name, paths (stats file, scratch dir, journal file), JOURNAL header
plan_path	(standard only) docs/plans/<file>.md
task_range	(standard, optional) e.g. [3, 8] to run only Tasks 3..8
red_test_path	(mini only) absolute path to the failing test from debug
cause_summary	(mini only) 1–2 sentences from the debugger agent
constraint	(mini only) "minimal fix, no surrounding cleanup"

Carrier — ailang-implement-orchestrator → worker agents

Implementer (per task):

Field	Content
task_text_path	/tmp/ail-iter/<iter_id>/task-<N>.md — orchestrator extracts the verbatim task block here on first reference
scene_set	parent milestone, position in iter, dependencies
cross_task_context	shared types, naming conventions for the iter
mode	standard or mini

Spec-reviewer:

Field	Content
task_text_path	same file as the implementer received
pre_task_sha, head_sha	for git diff <pre>..<head>
status_from_implementer	DONE or DONE_WITH_CONCERNS (with concerns)

Quality-reviewer:

Field	Content
pre_task_sha, head_sha	for the diff
spec_review_status	must be compliant
task_subject	one-line title, NOT the full task text

Tester (once per iter, at end, on DONE only):

Field	Content
iteration_scope	what shipped — feature name, commit range, invariants
coverage_gap	if known
mode	e2e_after_iter

End-report — ailang-implement-orchestrator → Boss

Plain-text report, ≤ 500 tokens, fixed structure:

Status:           DONE | PARTIAL | BLOCKED
Iter:             <iter_id>
Branch:           iter/<iter_id>
Tasks completed:  <N> of <total>
                  - <commit subject 1>
                  - <commit subject 2>
                  ...
Journal file:     docs/journals/<file>.md (on branch)
Stats:            bench/orchestrator-stats/<file>.json (on branch)
Commits:          <pre-iter SHA>..<head SHA>
Tests:            <count> green, <count> red
E2E coverage:     <new fixture paths>
Blocked detail:   (only if BLOCKED)
                  Task: <N>
                  Reason: context-exhausted | review-loop-exhausted | worker-blocked
                  Worker says: <verbatim reason>
                  Suggested next step: <one sentence>

The Boss reads this directly; the per-iter journal file is read selectively only when integrating or when investigating BLOCKED.

Boss-side merge workflow (on DONE)

1. Read docs/journals/<file>.md from the branch (git show iter/<iter_id>:docs/journals/<file>.md or check out if a closer look is needed).
2. Accept the agent's summary as-is, or rewrite the Summary section to reflect Boss-level framing.
3. Append a one-line entry to docs/journals/INDEX.md.
4. git rebase iter/<iter_id> onto main for linear history.
5. Optionally git branch -D iter/<iter_id>.
6. If trigger is done-state and user-away: write WhatsNew.md entry and notify.sh per CLAUDE.md.

Boss-side handling (on BLOCKED / PARTIAL)

1. Read the per-iter journal on the branch — the Blocked detail: section names the failure mode.
2. Decide:
   • Repair: adjust plan, re-dispatch the orchestrator with the same iter_id and task_range covering the remaining tasks. The orchestrator picks up the existing branch (git switch iter/<iter_id>) instead of creating a new one.
   • Discard: git branch -D iter/<iter_id>. The per-iter journal file disappears with the branch; INDEX.md is not touched (no entry was added).
   • Escalate: ask the user. The branch + journal file sit until the conversation resumes.

Error handling — per-task sub-status mechanics

The orchestrator follows this table verbatim. The same table is the canonical reference in skills/implement/SKILL.md.

Sub-status	Orchestrator action
DONE	next task / next phase
DONE_WITH_CONCERNS	accumulate concern, next
NEEDS_CONTEXT (1st–2nd)	re-dispatch with expanded carrier
NEEDS_CONTEXT (3rd)	stop → BLOCKED to Boss, reason context-exhausted
non_compliant / changes_requested (1st–2nd)	implementer re-dispatch with the reviewer's report as repair brief
non_compliant / changes_requested (3rd)	stop → BLOCKED to Boss, reason review-loop-exhausted
BLOCKED (worker)	stop → BLOCKED to Boss, reason verbatim
Tool / infra error	stop → BLOCKED to Boss, reason infra + raw error

On BLOCKED:

• already-committed tasks stay on the branch
• the stats file is written with outcome: PARTIAL and the partial per-task list
• the per-iter journal file IS written (with Status: PARTIAL or Status: BLOCKED), committed on the branch
• end-report carries Status: BLOCKED with detail
• docs/journals/INDEX.md is not touched (Boss decides on integration when handling the branch)

skip task K, continue is intentionally NOT a mode — tasks have implicit ordering dependencies and the orchestrator does not know the plan dependency graph. Skip is a Boss decision.

Testing strategy

This is a skill/agent refactor, not a language feature. No cargo test coverage applies directly. Verification ladder:

1. Stand-alone agent file parses — Agent("ailang-implement-orchestrator", ...) resolves to a runnable spec. Check via a no-op dispatch on a trivial fixture iter.
2. Branch isolation — dispatch the orchestrator; verify git branch after run shows iter/<test-iter> distinct from main; main is unchanged.
3. Single-task standard-mode run — dispatch the orchestrator on a one-task plan. Expect: green build, green tests, journal file on branch, stats file on branch, end-report follows the fixed format.
4. Multi-task standard-mode run — same on a 3+ task plan.
5. Mini-mode run — dispatch from debug handoff. Expect: same end-report shape, single-task stats, branch named iter/bugfix-<>.
6. Re-loop run — feed an intentionally non-compliant task. Expect: 2 re-dispatches, then BLOCKED with reason review-loop-exhausted. Branch preserved with partial commits.
7. BLOCKED run — feed an intentionally impossible task. Expect: immediate BLOCKED, partial commits preserved on branch.
8. Boss-merge step — manually integrate a successful test run: read journal file, write INDEX line, rebase, delete branch. Verify linear history and INDEX coherence.

Verification steps 6, 7, 8 are one-shot reproducers — they confirm the discipline once. Future stats data answers "is the re-loop limit too tight / too loose" empirically.

Acceptance criteria

The refactor is accepted when:

1. skills/implement/agents/ailang-implement-orchestrator.md exists, following the standard agent template (frontmatter, spirit-letter lead-in, standing reading list, carrier contract, Iron Law, process, status protocol, output format, common rationalisations, red flags). Contains the branch-creation step as Process Step 1 and the journal-file-write step as a late-process action.
2. skills/implement/SKILL.md is rewritten to delegate both modes to the orchestrator-agent and to document the Boss-side merge step. The canonical discipline (Iron Law, sub-status table, common rationalisations) remains in this file.
3. skills/README.md documents the orchestrator-agent as the named exception to "agents do not call other agents".
4. skills/implement/agents/ailang-implementer.md and skills/implement/agents/ailang-spec-reviewer.md accept task_text_path in place of task_text in their carriers.
5. docs/JOURNAL.md is renamed to docs/journal-archive.md with the archived-status header. docs/journals/INDEX.md exists (initially with a single line linking back to journal-archive.md for pre-migration context).
6. Skill files and agent files that reference docs/JOURNAL.md are updated to point at the new layout. CLAUDE.md reflects the new roles of docs/journals/, docs/journals/INDEX.md, and docs/journal-archive.md.
7. The eight verification steps in Testing strategy pass.
8. The first real /implement run after the refactor lands shows a Boss-context delta consistent with the predicted factor (target: ≤ 1k tokens per /implement in the Boss-context for the agent dispatch + end-report, plus ≤ 1k tokens for the Boss merge step).

Migration

The migration touches three categories of files. All happen in a single commit (or one commit per category, atomically).

1. Filesystem.

• git mv docs/JOURNAL.md docs/journal-archive.md
• Add archived-status header to docs/journal-archive.md
• Create empty docs/journals/ directory with .gitkeep (or with the initial INDEX.md)
• Create docs/journals/INDEX.md with the header and a single pointer line referencing the archive for pre-migration entries

2. References.

• Scan all files for JOURNAL.md and update each reference:
  • CLAUDE.md "Roles of …" section: new descriptions for docs/journals/, docs/journals/INDEX.md, docs/journal-archive.md
  • skills/*/SKILL.md files: "read JOURNAL.md tail" → "read docs/journals/INDEX.md and the latest 1–3 referenced files"
  • skills/*/agents/*.md standing reading lists: same update
  • docs/DESIGN.md if it references JOURNAL.md: same update

3. Workflow.

• Existing plans in docs/plans/ work unchanged — the plan format is unchanged.
• Existing Iter-22+ plans can be re-run end-to-end through the new orchestrator without rewriting; only the journal-write target differs (per-iter file instead of appending to JOURNAL.md).
• WhatsNew.md and notify.sh workflow unchanged.

Out of scope (deferred to other specs)

• Worker model downgrade (cheap models for simple subagents). User has explicitly excluded this; all roles stay at Opus 4.7.
• Stats aggregation tool — written once empirical data exists.
• Plan-drafter sub-agent for the /planner skill. Separate hebel, separate spec.
• /brainstorm spec-drafter — usage statistic shows /brainstorm at 2%, not cost-effective.
• docs/DESIGN.md changes. The refactor is below the language spec; the canonical AILang language definition is untouched.
• External-platform integration (Gitea PR via tea, GitHub spiegelung). Decided 2026-05-11: the Per-Iter-File pattern in the Git repo is the single source of truth; external platform UI is redundant because Gitea's native file-browser already serves these files read-only once pushed. May be reconsidered if team-review workflow appears later.