Brummel/AILang
1
0
Fork 0
Code Issues 29 Pull Requests Actions Packages Projects Releases Wiki Activity

spec: implement-orchestrator-agent

Browse Source 
Three coupled changes to /implement:
- Boss-context offload via single subagent dispatch (target ~100x reduction)
- Iter isolation via branches (eliminates parallel-implementer conflicts)
- Per-iter journals under docs/journals/ replacing the JOURNAL.md monolith

JOURNAL.md becomes docs/journal-archive.md at acceptance time. Workers
stay at Opus 4.7; saving comes from token-volume relocation only.
This commit is contained in:
Brummel
2026-05-11 10:56:24 +02:00
parent 0e6079d335
commit adcea02e85
1 changed files with 545 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specs/2026-05-11-implement-orchestrator-agent.md
+545
View File
@@ -0,0 +1,545 @@
# `ailang-implement-orchestrator` — Design Spec
**Date:** 2026-05-11
**Status:** Draft — awaiting user spec review
**Authors:** Brummel (orchestrator) + Claude
## Goal
Three coupled changes to `/implement`, each individually motivated,
together resolving the dominant cost and workflow-coordination
problems observed on 2026-05-11:
1. **Boss-context offload.** Reduce the Boss-Orchestrator's
per-`/implement` context cost from ~100k tokens to ~700 tokens
(factor ~100×) by delegating the entire per-task loop to a single
subagent — `ailang-implement-orchestrator` — running in its own
context.
2. **Iter isolation via branches.** Every `/implement` run owns a
dedicated branch (`iter/<iter_id>`). The Boss-Orchestrator
decides per run whether and when to integrate. Eliminates the
"parallel implementer pushed 37 commits while I tried to commit
a spec"-class of problem.
3. **Per-iter journals replace the JOURNAL.md monolith.** Iteration
reports move from a single 14k-line `docs/JOURNAL.md` to per-iter
files in `docs/journals/<YYYY-MM-DD>-iter-<id>.md`. The existing
JOURNAL.md is archived as `docs/journal-archive.md`. A short
`docs/journals/INDEX.md` provides chronological navigation, edited
by the Boss as part of the merge step.
Quality of decisions is non-negotiable: model selection stays at
Opus 4.7 for every role; the saving comes from token-volume
relocation, not from model downgrades.
## Background — what the current `/implement` actually costs
Anthropic-side usage statistics for the project (2026-05-11):
```
30% /implement (skill)
26% subagents under "implement"
12% /plan
50% of usage at >150k context
49% of usage in 8h+ sessions
```
A per-task breakdown of today's `/implement` flow in the Boss-context:
| Step | Tokens (typical) |
|------|------------------|
| Plan loaded once (Step 1) — 478–1883 lines via `Read` | 12k–25k *per iter* |
| Per-task implementer dispatch — `task_text` verbatim | 2k–7k |
| Implementer report | ~300 |
| Per-task spec-reviewer dispatch — `task_text` AGAIN | 2k–7k *(duplicate)* |
| Spec-reviewer report | ~350 |
| Per-task quality-reviewer dispatch — SHAs only | ~100 |
| Quality-reviewer report | ~450 |
| Re-loop on `non_compliant` (~50% rate) | × 1.5 |
For an 8-task iteration: ~12–25k (plan) + ~88k (per-task ×8 ×1.5) ≈
**100k tokens in the Boss-context**. Combined with multi-iter
sessions this explains the >150k tail and the cost concentration the
statistics flag.
In parallel, two structural issues surfaced:
- Concurrent `/implement` activity (separate session pushing to
`main` while the Boss was working) leaves the working tree in a
detached-HEAD state with no clean spot to commit a small artefact
like a spec file.
- `docs/JOURNAL.md` is 14309 lines. Skill steps that "read tail"
routinely pull thousands of tokens of which 90% is irrelevant to
the current iter; cross-iter searches require scrolling rather
than file-system tools.
## Architecture
```
Boss-Orchestrator
│
│ Agent("ailang-implement-orchestrator", carrier)
▼
ailang-implement-orchestrator ← runs in its own context
│
│ git fetch origin main
│ git switch -c iter/<iter_id> origin/main
│
│ reads plan (once, in its context)
│ per task K = 1..N:
│ Agent("ailang-implementer", task_carrier_K) ── carrier points to /tmp/ail-iter/<iter_id>/task-K.md
│ on DONE: Agent("ailang-spec-reviewer", review_carrier_K)
│ on compliant: Agent("ailang-quality-reviewer", quality_carrier_K)
│ loop on non_compliant / NEEDS_CONTEXT (≤ 2 retries)
│ on BLOCKED or 3rd retry: stop, emit PARTIAL bundle
│ Agent("ailang-tester", coverage_carrier) ── E2E coverage
│ write docs/journals/<YYYY-MM-DD>-iter-<id>.md ── per-iter journal
│ commit journal file on the branch
│ write bench/orchestrator-stats/<...>.json
│ commit stats file on the branch
▼
compressed end-report (~500 tokens) — pointer to journal file + branch name
then Boss-Orchestrator (separate, after reading end-report):
│
│ reviews docs/journals/<...>.md on the branch
│ updates docs/journals/INDEX.md with one line
│ git rebase iter/<iter_id> onto main ── linear history
│ git branch -D iter/<iter_id> ── optional cleanup
```
The Boss-Orchestrator sees: one dispatch out, one compressed report
back, then a Boss-side merge step that touches one INDEX line.
Per-task loops, plan reads, review re-loops, journal writing, stats
writing — all happen inside the orchestrator-agent's context and
never reach the Boss.
`skills/README.md` currently states "Agents do not call other
agents." `ailang-implement-orchestrator` becomes the documented
exception, named explicitly. The rule remains otherwise binding —
no other agent gains the `Agent` tool.
## Components
### `ailang-implement-orchestrator` (new agent)
Location: `skills/implement/agents/ailang-implement-orchestrator.md`.
Tools: `Read, Edit, Write, Bash, Glob, Grep, Agent`. The `Agent`
tool is the explicit exception — it dispatches the existing four
worker agents (implementer, spec-reviewer, quality-reviewer,
tester).
Standing reading list:
1. `CLAUDE.md` — orchestrator framing.
2. `docs/DESIGN.md` — invariants the iteration must respect.
3. `docs/journals/INDEX.md` + the last 1–3 per-iter journal files
linked from there — recent state. (After the migration window,
no agent reads `docs/journal-archive.md` directly; long-tail
context lives there but is referenced by date if needed.)
4. `skills/implement/SKILL.md` — **canonical discipline** (Iron
Law, Step-2 sub-status mechanics, per-task scope rules). The
orchestrator reads this verbatim every dispatch; it does not
paraphrase.
Model: Opus 4.7.
First and last actions in every dispatch:
- **First**: `git fetch origin main && git switch -c iter/<iter_id> origin/main`.
Ensures a clean base from current Remote regardless of what state
the local working tree was in.
- **Last (on DONE)**: write per-iter journal file, commit it on the
branch, write stats file, commit it on the branch, return
end-report. Do not merge — that's Boss-side.
- **Last (on BLOCKED)**: write per-iter journal file with the
partial status, commit it on the branch, write stats file with
`outcome: PARTIAL` or `outcome: BLOCKED`, commit it, return
end-report. Branch is left for the Boss to inspect, repair, or
delete.
### `skills/implement/SKILL.md` (rewritten)
Today: ~210 lines covering Step 1 (plan load), Step 2.1–2.5
(per-task dispatch), Step 3 (tester), Step 4 (JOURNAL), Mini-Mode
handoff, Common Rationalisations, Red Flags.
After: ~100 lines. Both Standard-Mode and Mini-Mode collapse to:
> 1. Dispatch `ailang-implement-orchestrator` with the carrier
> below.
> 2. Read the returned end-report.
> 3. **Boss merge step (on DONE):** open the per-iter journal at
> `docs/journals/<...>.md` on the branch. Accept as-is or
> condense; append a one-line entry to `docs/journals/INDEX.md`;
> `git rebase iter/<iter_id> onto main`; optionally delete
> branch.
> 4. **Boss handling (on BLOCKED):** inspect the partial branch,
> repair (re-dispatch with adjusted plan / extended context),
> or discard (`git branch -D iter/<iter_id>`).
The Iron Law, per-task sub-status table, and Common Rationalisations
stay in this file — they remain the canonical discipline source,
now read by the orchestrator-agent. What goes away from the skill
body is the *procedural Boss-side* of each per-task step (no
Boss-side per-task step remains).
### Worker agents — unchanged in role, minor carrier change
`ailang-implementer`, `ailang-spec-reviewer`,
`ailang-quality-reviewer`, `ailang-tester` keep their current
frontmatter, tools, standing reading lists, and status protocols.
Only their *caller* changes from the Boss-Orchestrator to the
`ailang-implement-orchestrator`.
One mechanical change to carriers: instead of `task_text` as a
verbatim string, implementer and spec-reviewer receive
`task_text_path` (`/tmp/ail-iter/<iter_id>/task-<N>.md`) and read
the file themselves. Eliminates per-task duplication of the task
text between the two agents.
### `docs/journals/` — new subdir
Replaces `docs/JOURNAL.md` going forward.
Naming: `docs/journals/<YYYY-MM-DD>-iter-<id>.md`. Example:
`docs/journals/2026-05-11-iter-ct.2.3.md`.
File content, written by the orchestrator-agent:
```markdown
# iter <id> — <title>
**Date:** YYYY-MM-DD
**Branch:** iter/<id>
**Status:** DONE | PARTIAL | BLOCKED
**Tasks completed:** <N> of <total>
## Summary
<1-paragraph, Boss may rewrite during merge>
## Per-task subjects
- iter <id>.1: <subject>
- iter <id>.2: <subject>
- ...
## Concerns
(DONE_WITH_CONCERNS aggregate, one line each)
## Known debt
(one-liner each, with why-not-touched)
## Blocked detail
(only if BLOCKED / PARTIAL: task N, reason, worker-quote, suggested next step)
## Commits
<pre-iter SHA>..<head SHA>
## Stats
bench/orchestrator-stats/<file>.json
```
The Summary section is what the Boss may rewrite when integrating —
the agent's draft is a starting point. The rest is factual and
preserved verbatim.
### `docs/journals/INDEX.md` — new file
Boss-maintained chronological index. Append-only, one line per iter:
```markdown
# Journal index
- 2026-05-11 — iter ct.2.3: typecheck cleanup → 2026-05-11-iter-ct.2.3.md
- 2026-05-12 — bugfix sigsegv-pattern-ctor → 2026-05-12-iter-bugfix-sigsegv-pattern-ctor.md
- ...
```
Written by the Boss as part of the merge step, never by the
orchestrator-agent. Filenames are relative to `docs/journals/`.
### `docs/journal-archive.md` — renamed historical JOURNAL.md
At spec-acceptance time, `docs/JOURNAL.md` is renamed to
`docs/journal-archive.md`. Content is preserved verbatim — it
remains the canonical record for everything pre-migration.
A single header note is added at the top:
```markdown
> **Status:** archived 2026-05-11. New iter journals live under
> `docs/journals/`. See `docs/journals/INDEX.md` for the
> chronological pointer list. This file remains the historical
> record for entries prior to that date.
```
`CLAUDE.md` is updated in the same migration commit to reflect the
new layout. Existing cross-references to "JOURNAL.md" in skill
files and agent files are updated to point at the new layout
(`docs/journals/INDEX.md` + recent entries).
### `bench/orchestrator-stats/<YYYY-MM-DD>-iter-<X>.json` — unchanged
Per-iter JSON, written by the orchestrator before the journal
commit. Pointer in the end-report. No aggregation script ships
with this spec; once a few iters accumulate, a separate utility
can compute distributions.
### `skills/README.md` (one-line update)
The "Agents do not call other agents" subsection gains a sentence
naming `ailang-implement-orchestrator` as the dispatch-and-coordinate
exception.
## Data flow
### Carrier — Boss → `ailang-implement-orchestrator`
| Field | Content |
|-------|---------|
| `mode` | `"standard"` or `"mini"` |
| `iter_id` | e.g. `"ct.2.3"` (standard) or `"bugfix-<short-symptom>"` (mini) — used for branch name, paths (stats file, scratch dir, journal file), JOURNAL header |
| `plan_path` | (standard only) `docs/plans/<file>.md` |
| `task_range` | (standard, optional) e.g. `[3, 8]` to run only Tasks 3..8 |
| `red_test_path` | (mini only) absolute path to the failing test from `debug` |
| `cause_summary` | (mini only) 1–2 sentences from the debugger agent |
| `constraint` | (mini only) `"minimal fix, no surrounding cleanup"` |
### Carrier — `ailang-implement-orchestrator` → worker agents
Implementer (per task):
| Field | Content |
|-------|---------|
| `task_text_path` | `/tmp/ail-iter/<iter_id>/task-<N>.md` — orchestrator extracts the verbatim task block here on first reference |
| `scene_set` | parent milestone, position in iter, dependencies |
| `cross_task_context` | shared types, naming conventions for the iter |
| `mode` | `standard` or `mini` |
Spec-reviewer:
| Field | Content |
|-------|---------|
| `task_text_path` | same file as the implementer received |
| `pre_task_sha`, `head_sha` | for `git diff <pre>..<head>` |
| `status_from_implementer` | `DONE` or `DONE_WITH_CONCERNS` (with concerns) |
Quality-reviewer:
| Field | Content |
|-------|---------|
| `pre_task_sha`, `head_sha` | for the diff |
| `spec_review_status` | must be `compliant` |
| `task_subject` | one-line title, NOT the full task text |
Tester (once per iter, at end, on DONE only):
| Field | Content |
|-------|---------|
| `iteration_scope` | what shipped — feature name, commit range, invariants |
| `coverage_gap` | if known |
| `mode` | `e2e_after_iter` |
### End-report — `ailang-implement-orchestrator` → Boss
Plain-text report, ≤ 500 tokens, fixed structure:
```
Status: DONE | PARTIAL | BLOCKED
Iter: <iter_id>
Branch: iter/<iter_id>
Tasks completed: <N> of <total>
- <commit subject 1>
- <commit subject 2>
...
Journal file: docs/journals/<file>.md (on branch)
Stats: bench/orchestrator-stats/<file>.json (on branch)
Commits: <pre-iter SHA>..<head SHA>
Tests: <count> green, <count> red
E2E coverage: <new fixture paths>
Blocked detail: (only if BLOCKED)
Task: <N>
Reason: context-exhausted | review-loop-exhausted | worker-blocked
Worker says: <verbatim reason>
Suggested next step: <one sentence>
```
The Boss reads this directly; the per-iter journal file is read
selectively only when integrating or when investigating BLOCKED.
### Boss-side merge workflow (on DONE)
1. Read `docs/journals/<file>.md` from the branch
(`git show iter/<iter_id>:docs/journals/<file>.md` or check out
if a closer look is needed).
2. Accept the agent's summary as-is, or rewrite the Summary section
to reflect Boss-level framing.
3. Append a one-line entry to `docs/journals/INDEX.md`.
4. `git rebase iter/<iter_id> onto main` for linear history.
5. Optionally `git branch -D iter/<iter_id>`.
6. If trigger is done-state and user-away: write `WhatsNew.md` entry
and `notify.sh` per CLAUDE.md.
### Boss-side handling (on BLOCKED / PARTIAL)
1. Read the per-iter journal on the branch — the `Blocked detail:`
section names the failure mode.
2. Decide:
- **Repair**: adjust plan, re-dispatch the orchestrator with the
same `iter_id` and `task_range` covering the remaining tasks.
The orchestrator picks up the existing branch (`git switch
iter/<iter_id>`) instead of creating a new one.
- **Discard**: `git branch -D iter/<iter_id>`. The per-iter
journal file disappears with the branch; INDEX.md is not
touched (no entry was added).
- **Escalate**: ask the user. The branch + journal file sit until
the conversation resumes.
## Error handling — per-task sub-status mechanics
The orchestrator follows this table verbatim. The same table is the
canonical reference in `skills/implement/SKILL.md`.
| Sub-status | Orchestrator action |
|------------|---------------------|
| `DONE` | next task / next phase |
| `DONE_WITH_CONCERNS` | accumulate concern, next |
| `NEEDS_CONTEXT` (1st–2nd) | re-dispatch with expanded carrier |
| `NEEDS_CONTEXT` (3rd) | stop → `BLOCKED` to Boss, reason `context-exhausted` |
| `non_compliant` / `changes_requested` (1st–2nd) | implementer re-dispatch with the reviewer's report as repair brief |
| `non_compliant` / `changes_requested` (3rd) | stop → `BLOCKED` to Boss, reason `review-loop-exhausted` |
| `BLOCKED` (worker) | stop → `BLOCKED` to Boss, reason verbatim |
| Tool / infra error | stop → `BLOCKED` to Boss, reason `infra` + raw error |
On `BLOCKED`:
- already-committed tasks stay on the branch
- the stats file is written with `outcome: PARTIAL` and the partial
per-task list
- the per-iter journal file IS written (with `Status: PARTIAL` or
`Status: BLOCKED`), committed on the branch
- end-report carries `Status: BLOCKED` with detail
- `docs/journals/INDEX.md` is **not** touched (Boss decides on
integration when handling the branch)
`skip task K, continue` is intentionally NOT a mode — tasks have
implicit ordering dependencies and the orchestrator does not know
the plan dependency graph. Skip is a Boss decision.
## Testing strategy
This is a skill/agent refactor, not a language feature. No
`cargo test` coverage applies directly. Verification ladder:
1. **Stand-alone agent file parses** — `Agent("ailang-implement-orchestrator", ...)`
resolves to a runnable spec. Check via a no-op dispatch on a
trivial fixture iter.
2. **Branch isolation** — dispatch the orchestrator; verify
`git branch` after run shows `iter/<test-iter>` distinct from
main; main is unchanged.
3. **Single-task standard-mode run** — dispatch the orchestrator on
a one-task plan. Expect: green build, green tests, journal file
on branch, stats file on branch, end-report follows the fixed
format.
4. **Multi-task standard-mode run** — same on a 3+ task plan.
5. **Mini-mode run** — dispatch from `debug` handoff. Expect: same
end-report shape, single-task stats, branch named `iter/bugfix-<>`.
6. **Re-loop run** — feed an intentionally non-compliant task.
Expect: 2 re-dispatches, then `BLOCKED` with reason
`review-loop-exhausted`. Branch preserved with partial commits.
7. **BLOCKED run** — feed an intentionally impossible task. Expect:
immediate `BLOCKED`, partial commits preserved on branch.
8. **Boss-merge step** — manually integrate a successful test run:
read journal file, write INDEX line, rebase, delete branch.
Verify linear history and INDEX coherence.
Verification steps 6, 7, 8 are one-shot reproducers — they confirm
the discipline once. Future stats data answers "is the re-loop
limit too tight / too loose" empirically.
## Acceptance criteria
The refactor is accepted when:
1. `skills/implement/agents/ailang-implement-orchestrator.md` exists,
following the standard agent template (frontmatter, spirit-letter
lead-in, standing reading list, carrier contract, Iron Law,
process, status protocol, output format, common rationalisations,
red flags). Contains the branch-creation step as Process Step 1
and the journal-file-write step as a late-process action.
2. `skills/implement/SKILL.md` is rewritten to delegate both modes
to the orchestrator-agent and to document the Boss-side merge
step. The canonical discipline (Iron Law, sub-status table,
common rationalisations) remains in this file.
3. `skills/README.md` documents the orchestrator-agent as the named
exception to "agents do not call other agents".
4. `skills/implement/agents/ailang-implementer.md` and
`skills/implement/agents/ailang-spec-reviewer.md` accept
`task_text_path` in place of `task_text` in their carriers.
5. `docs/JOURNAL.md` is renamed to `docs/journal-archive.md` with
the archived-status header. `docs/journals/INDEX.md` exists
(initially with a single line linking back to
`journal-archive.md` for pre-migration context).
6. Skill files and agent files that reference `docs/JOURNAL.md` are
updated to point at the new layout. `CLAUDE.md` reflects the new
roles of `docs/journals/`, `docs/journals/INDEX.md`, and
`docs/journal-archive.md`.
7. The eight verification steps in Testing strategy pass.
8. The first real `/implement` run after the refactor lands shows a
Boss-context delta consistent with the predicted factor (target:
≤ 1k tokens per `/implement` in the Boss-context for the agent
dispatch + end-report, plus ≤ 1k tokens for the Boss merge step).
## Migration
The migration touches three categories of files. All happen in a
single commit (or one commit per category, atomically).
**1. Filesystem.**
- `git mv docs/JOURNAL.md docs/journal-archive.md`
- Add archived-status header to `docs/journal-archive.md`
- Create empty `docs/journals/` directory with `.gitkeep` (or with
the initial INDEX.md)
- Create `docs/journals/INDEX.md` with the header and a single
pointer line referencing the archive for pre-migration entries
**2. References.**
- Scan all files for `JOURNAL.md` and update each reference:
- `CLAUDE.md` "Roles of …" section: new descriptions for
`docs/journals/`, `docs/journals/INDEX.md`,
`docs/journal-archive.md`
- `skills/*/SKILL.md` files: "read JOURNAL.md tail" → "read
`docs/journals/INDEX.md` and the latest 1–3 referenced files"
- `skills/*/agents/*.md` standing reading lists: same update
- `docs/DESIGN.md` if it references JOURNAL.md: same update
**3. Workflow.**
- Existing plans in `docs/plans/` work unchanged — the plan format
is unchanged.
- Existing Iter-22+ plans can be re-run end-to-end through the new
orchestrator without rewriting; only the journal-write target
differs (per-iter file instead of appending to JOURNAL.md).
- WhatsNew.md and `notify.sh` workflow unchanged.
## Out of scope (deferred to other specs)
- **Worker model downgrade** (cheap models for simple subagents).
User has explicitly excluded this; all roles stay at Opus 4.7.
- **Stats aggregation tool** — written once empirical data exists.
- **Plan-drafter sub-agent** for the `/plan` skill. Separate hebel,
separate spec.
- **`/brainstorm` spec-drafter** — usage statistic shows
`/brainstorm` at 2%, not cost-effective.
- **`docs/DESIGN.md` changes**. The refactor is below the language
spec; the canonical AILang language definition is untouched.
- **External-platform integration** (Gitea PR via `tea`, GitHub
spiegelung). Decided 2026-05-11: the Per-Iter-File pattern in the
Git repo is the single source of truth; external platform UI is
redundant because Gitea's native file-browser already serves
these files read-only once pushed. May be reconsidered if
team-review workflow appears later.