diff --git a/docs/pipeline.md b/docs/pipeline.md index 4fb0c0a..7d9f2ac 100644 --- a/docs/pipeline.md +++ b/docs/pipeline.md @@ -186,9 +186,11 @@ orchestrates from the top level, so the per-task phases are real, independently-invokable agent calls; this retired the former `implement-orchestrator` agent's inline-role-switch workaround (the four phase agents survive as the agent-types the script -dispatches). Writes code, tests, and stats files directly in the -working tree as unstaged changes; never commits. On `PARTIAL` or -`BLOCKED`, also writes `BLOCKED.md` at the repo root. +dispatches). Writes code and tests directly in the working tree as +unstaged changes; never commits. The end-report carries the finished +stats.json and — on `PARTIAL`/`BLOCKED` — `BLOCKED.md` contents, which +the orchestrator writes (issue #29 retired the finalize dispatch that +only templated the script's own aggregate). ### compiler-driven diff --git a/implement/SKILL.md b/implement/SKILL.md index 24a6879..6bf09e2 100644 --- a/implement/SKILL.md +++ b/implement/SKILL.md @@ -1,6 +1,6 @@ --- name: implement -description: Use when an implementation plan exists under docs/plans and is ready to execute, OR when a debug/tdd RED-test is handed off for the GREEN side. Runs the `implement-loop` Workflow — a deterministic script that executes the per-task loop (implementer → spec-compliance → quality, each a separate agent call) and aggregates in code, writing edits, tests, and a stats file to the working tree without committing (and `BLOCKED.md` on PARTIAL/BLOCKED). The orchestrator reads the end-report, inspects the working tree, decides commit shape, and performs all commits. Also documents the `compiler-driven` light-edit arm. +description: Use when an implementation plan exists under docs/plans and is ready to execute, OR when a debug/tdd RED-test is handed off for the GREEN side. Runs the `implement-loop` Workflow — a deterministic script that executes the per-task loop (implementer → spec-compliance → quality, each a separate agent call) and aggregates in code, writing edits and tests to the working tree without committing. The orchestrator reads the end-report, writes the stats file (and `BLOCKED.md` on PARTIAL/BLOCKED) from the contents the end-report carries, inspects the working tree, decides commit shape, and performs all commits. Also documents the `compiler-driven` light-edit arm. --- # implement — plan execution on the Workflow substrate @@ -30,11 +30,17 @@ real build/test outcome). The four phase agents survive unchanged as the agent-types the script dispatches; only the dispatch/aggregation prose the orchestrator-agent carried is gone. -All work lives in the working tree: code edits, the stats file, and — -on PARTIAL/BLOCKED — `BLOCKED.md` at the repo root. The script **never +All code edits live in the working tree. The stats file and — on +PARTIAL/BLOCKED — `BLOCKED.md` are **built by the script and carried in +the end-report** (`artifacts.stats_json` / `artifacts.blocked_md`): the +script has no filesystem access of its own, and dispatching an agent +just to template the script's own aggregate cost ~41k real tokens per +run (issue #29). The orchestrator writes them byte-identical, filling +the two runtime slots it owns anyway — `{{DATE}}` (`date +%F`) and +`{{FILES_TOUCHED}}` (`git status --porcelain`). The script **never commits** and never moves main HEAD. The orchestrator reads one -end-report (the workflow's return value), inspects the unstaged tree, -and decides commit shape. +end-report (the workflow's return value), writes the artifacts, +inspects the unstaged tree, and decides commit shape. ## When to Use / Skipping @@ -99,13 +105,13 @@ regression around the RED-first gate. ## The Iron Law ``` -THE LOOP NEVER COMMITS. CODE EDITS, TESTS, AND THE STATS FILE LIVE IN THE WORKING TREE AS UNSTAGED CHANGES UNTIL THE ORCHESTRATOR COMMITS THEM. +THE LOOP NEVER COMMITS. CODE EDITS AND TESTS LIVE IN THE WORKING TREE AS UNSTAGED CHANGES UNTIL THE ORCHESTRATOR COMMITS THEM; THE STATS FILE AND BLOCKED.md ARE WRITTEN BY THE ORCHESTRATOR FROM THE CONTENTS THE END-REPORT CARRIES. WITHIN THIS LOOP, MAIN HEAD IS SACROSANCT — NO RESET, NO REVERT; THE LOOP ONLY EVER WRITES UNSTAGED CHANGES. MAIN MOVES FORWARD ONLY VIA ORCHESTRATOR COMMITS. (THE `/boss` ROLLBACK SANDBOX WINDS BACK ALREADY-COMMITTED AUTONOMOUS WORK — A BOSS-LEVEL RECOVERY, NOT A STEP IN THIS LOOP.) NO IN-LOOP AGENT EVER DISCARDS WORKING-TREE STATE VIA `git checkout` / `git restore` — WHOLE-FILE OR `--patch`: NOTHING COMMITS BETWEEN TASKS, SO A SHARED FILE CARRIES EARLIER TASKS' UNCOMMITTED DONE WORK, AND A CHECKOUT SILENTLY DESTROYS IT (issue #23). RECOVERY IS FORWARD — EDIT BACK, OR REPORT `BLOCKED`. (THE ORCHESTRATOR'S OWN END-OF-ITERATION `git checkout -- .` DISPOSITION HAPPENS OUTSIDE THE LOOP.) PER-TASK PHASES ARE SEPARATE AGENT CALLS IN THE WORKFLOW SCRIPT — implementer, then spec-compliance, then quality. SPEC COMPLIANCE IS GATED BEFORE QUALITY. TASKS RUN SEQUENTIALLY; A BLOCKED TASK STOPS THE LOOP — NO SKIP-AHEAD (TASK ORDERING DEPENDENCIES ARE UNKNOWN). NEVER PUSH PAST `BLOCKED` BY HAND. -ON `PARTIAL` OR `BLOCKED`, THE SCRIPT WRITES `BLOCKED.md` AT THE REPO ROOT — UNCOMMITTED BY CONVENTION. ON `DONE`, NO SEPARATE FILE. (EXCEPTIONS THAT WRITE NO `BLOCKED.md`: A NO-OP ITERATION — ITS TREE IS ALREADY CLEAN — AND ANY PRE-LOOP EARLY EXIT (MALFORMED CARRIER, DIRTY-TREE PREFLIGHT, PLAN-EXTRACT GUARD): NOTHING RAN AND NOTHING WAS WRITTEN, SO THE STATUS + `infra:`-PREFIXED REASON RIDE THE END-REPORT AND THERE IS NOTHING TO CLEAN UP OR EXPLAIN.) +ON `PARTIAL` OR `BLOCKED`, THE END-REPORT CARRIES THE `BLOCKED.md` CONTENT (`artifacts.blocked_md`); THE ORCHESTRATOR'S FIRST ACTION ON READING SUCH A REPORT IS WRITING IT VERBATIM TO `BLOCKED.md` AT THE REPO ROOT — UNCOMMITTED BY CONVENTION. ON `DONE`, `artifacts.blocked_md` IS NULL. (EXCEPTIONS WHERE IT IS ALSO NULL: A NO-OP ITERATION — ITS TREE IS ALREADY CLEAN — AND ANY PRE-LOOP EARLY EXIT (MALFORMED CARRIER, DIRTY-TREE PREFLIGHT, PLAN-EXTRACT GUARD): NOTHING RAN AND NOTHING WAS WRITTEN, SO THE STATUS + `infra:`-PREFIXED REASON RIDE THE END-REPORT AND THERE IS NOTHING TO CLEAN UP OR EXPLAIN.) `DONE` REQUIRES POSITIVE EVIDENCE THAT AN EDIT LANDED — A NO-OP ITERATION (`git status --porcelain` EMPTY, untracked files INCLUDED) IS NEVER `DONE`, IT IS `BLOCKED` (the issue #11 vacuous-green shape, guarded on the main path). THE VERDICT IS THE GIT TREE COUNT, NEVER A SELF-REPORT. IN MINI MODE THE RED->GREEN IS OBSERVED, NOT ASSERTED: AN INDEPENDENT VERIFY RE-RUNS THE HANDED-OFF RED TEST AND THE SUITE; A STILL-RED TEST OR A REGRESSION IS `BLOCKED` (route back to debug, RED-first). THE COMPILER-DRIVEN ARM COMMITS ONLY ON CLEAN BUILD + SUITE GREEN UNCHANGED; ELSE IT BOUNCES (specify for a design hole, tdd for discovered new behaviour, debug for a regression). @@ -227,20 +233,28 @@ worktree branch). The workflow returns a small structured end-report (its return value): `status`, `iter_id`, `started_from`, `tasks_total` / `tasks_completed`, -per-task summaries, concerns, E2E fixtures, the stats path, and — on +per-task summaries, concerns, E2E fixtures, the `artifacts` block +(`stats_json`, `stats_path_hint`, `blocked_md`), and — on PARTIAL/BLOCKED — the blocked detail. Per-task chatter stayed inside the workflow's agent contexts and never reached the orchestrator. Read the end-report; it is the per-task summary you build the commit body -from. An **infra early-exit** (malformed or missing carrier, dirty-tree +from. **First action, before anything else:** materialise the +artifacts — write `artifacts.stats_json` to the stats path (fill +`{{DATE}}` with `date +%F`), and when `artifacts.blocked_md` is +non-null write it verbatim to `BLOCKED.md` at the repo root (fill +`{{DATE}}` and `{{FILES_TOUCHED}}` from `git status --porcelain`). The +contents are the script's single-sourced templates — do not rephrase, +restructure, or embellish them. An **infra early-exit** (malformed or missing carrier, dirty-tree preflight, a plan-extract guard) returns a minimal report instead: `status: BLOCKED` plus an `infra:`-prefixed `reason`, no `blocked_detail` and none of the per-task fields — nothing ran. ### Step 3 — Orchestrator inspect + commit step (on DONE) -The workflow returns with code edits and the stats file sitting in the -working tree as unstaged changes. Nothing is committed yet, and there -is no `BLOCKED.md` (DONE never writes one). +The workflow returns with code edits sitting in the working tree as +unstaged changes; the stats file was just written by you from +`artifacts.stats_json` (Step 2). Nothing is committed yet, and there +is no `BLOCKED.md` (`artifacts.blocked_md` is null on DONE). 1. Inspect: `git status` and `git diff` — confirm the changes match what the end-report claims. The end-report is the per-task summary; @@ -268,16 +282,16 @@ is no `BLOCKED.md` (DONE never writes one). ### Step 4 — Orchestrator handling (on PARTIAL or BLOCKED) -The workflow returns with whatever work-in-progress it managed plus -`BLOCKED.md` at the repo root carrying the diagnostic. Nothing is -committed. The orchestrator decides what to do with the dirty working -tree: +The workflow returns with whatever work-in-progress it managed; +`BLOCKED.md` at the repo root carries the diagnostic — written by you +in Step 2 from `artifacts.blocked_md`. Nothing is committed. The +orchestrator decides what to do with the dirty working tree: 0. **No-op `BLOCKED` special case** — if the end-report's status is - `BLOCKED` with `blocked_md` null and a no-op `blocked_detail` (the reason - names a no-op iteration or a clean-tree no-op fix; `files_touched` is 0 in - standard mode, null in mini), there is **no `BLOCKED.md`** and the tree is - already clean. The diagnostic is in the end-report's `blocked_detail`, not + `BLOCKED` with `artifacts.blocked_md` null and a no-op `blocked_detail` + (the reason names a no-op iteration or a clean-tree no-op fix; + `files_touched` is 0 in standard mode, null in mini), there is + **no `BLOCKED.md`** and the tree is already clean. The diagnostic is in the end-report's `blocked_detail`, not a file. Nothing to clean up or stash; re-plan or re-run from the same clean tree. The same no-file shape occurs for an **infra early-exit** (`reason` diff --git a/implement/workflows/implement-loop.js b/implement/workflows/implement-loop.js index 7d1d1aa..cb3fa9f 100644 --- a/implement/workflows/implement-loop.js +++ b/implement/workflows/implement-loop.js @@ -15,10 +15,12 @@ // encodes the project's real build/test outcome), not on a guess. // // Discipline preserved verbatim from the old loop: -// • The script NEVER commits and NEVER moves main HEAD. All code edits, -// the stats file, and (on PARTIAL/BLOCKED) BLOCKED.md live in the -// working tree as unstaged changes; the orchestrator that invoked -// this workflow inspects and commits. +// • The script NEVER commits and NEVER moves main HEAD. All code edits +// live in the working tree as unstaged changes; the end-report carries +// the finished stats.json and (on PARTIAL/BLOCKED) BLOCKED.md contents, +// which the orchestrator that invoked this workflow writes, inspects, +// and commits (issue #29 — the former finalize dispatch only templated +// the script's own aggregate). // • Tasks run SEQUENTIALLY — plan tasks carry implicit ordering // dependencies, and the script does not know the graph, so a BLOCKED // task stops the loop rather than skipping ahead. @@ -70,8 +72,9 @@ // files sit outside the threat model (`git checkout -- ` cannot // discard them). // • The script has no filesystem or shell access of its own — every -// working-tree mutation (code, stats.json, BLOCKED.md) happens inside -// an agent() call. +// working-tree mutation (code, tests) happens inside an agent() call; +// the stats.json / BLOCKED.md contents are built in-script and ride +// the end-report for the orchestrator to write. // • Model policy: every agent() call pins an explicit model — never the // session-model inherit (which could route to an unintended tier, e.g. // fable — banned for all plugin agents and workflows). Mechanical and @@ -81,7 +84,7 @@ // explicit effort — never the session inherit. Code-writing and review // steps run high, the opus quality gate runs xhigh, and schema-bound // extraction/check steps (preflight, plan-index, plan-extract, snapshot, -// mini-verify, tree-check, finalize) run medium. See +// mini-verify, tree-check) run medium. See // docs/agent-template.md § effort. // // Carrier — passed as the Workflow `args` object by the orchestrator: @@ -98,13 +101,12 @@ export const meta = { name: 'implement-loop', description: - 'Execute a plan iteration (standard) or a debug/tdd RED->GREEN handoff (mini) task-by-task: implementer -> spec-compliance -> quality, with deterministic re-loop and aggregation. Writes code, tests, stats.json, and (on PARTIAL/BLOCKED) BLOCKED.md to the working tree as unstaged changes; never commits, never moves main HEAD.', + 'Execute a plan iteration (standard) or a debug/tdd RED->GREEN handoff (mini) task-by-task: implementer -> spec-compliance -> quality, with deterministic re-loop and aggregation. Writes code and tests to the working tree as unstaged changes; the end-report carries the finished stats.json and (on PARTIAL/BLOCKED) BLOCKED.md contents for the orchestrator to write. Never commits, never moves main HEAD.', phases: [ { title: 'Preflight' }, { title: 'Per-task loop' }, { title: 'Verify' }, { title: 'E2E coverage' }, - { title: 'Report' }, ], } @@ -360,8 +362,9 @@ const MINI_VERIFY_SCHEMA = { } // Standard-mode no-op gate (issue #12, facets 1+2): the authoritative ground // truth that the iteration's per-task work actually landed. Runs after the loop -// but BEFORE E2E/finalize, so neither E2E fixtures nor the stats/BLOCKED.md -// artefacts can inflate the count. Uses `git status --porcelain` (counts brand-new +// but BEFORE E2E, so E2E fixtures cannot inflate the count (stats/BLOCKED.md +// are no longer written in-loop at all — the orchestrator writes them from the +// end-report, after this gate). Uses `git status --porcelain` (counts brand-new // UNTRACKED files — the implementer leaves edits unstaged, so a new-file deliverable // is untracked and `git diff HEAD` would miss it). A self-report can never reach // this verdict; it is git ground truth. @@ -403,16 +406,6 @@ const SNAP_SCHEMA = { }, }, } -const FINALIZE_SCHEMA = { - type: 'object', - additionalProperties: false, - required: ['stats_path', 'wrote_blocked_md'], - properties: { - stats_path: { type: 'string' }, - wrote_blocked_md: { type: 'boolean' }, - }, -} - // ---- Phase 0 — clean-tree preflight (mini mode only) -------------------- // Standard mode folds these checks into the plan-index dispatch (issue #29): // the check itself stays deterministic in the script (schema fields evaluated @@ -840,9 +833,9 @@ else if (completed.length > 0) outcome = 'PARTIAL' else outcome = 'BLOCKED' // An iteration-level guard (issue #12) may downgrade a DONE outcome to BLOCKED on -// positive-evidence grounds. It runs BEFORE finalize so the persisted stats.json -// records the final outcome (not a stale DONE), and so E2E does not run over a -// no-op. The verdict is git ground truth, never a per-task self-report — a +// positive-evidence grounds. It runs BEFORE the report assembly so the built +// stats.json content records the final outcome (not a stale DONE), and so E2E +// does not run over a no-op. The verdict is git ground truth, never a per-task self-report — a // self-report must never be able to FAIL a run that actually did the work. It // carries its own blocked_detail (no single offending task) and a `noopClean` // flag: a clean-tree no-op needs no BLOCKED.md (nothing to explain). @@ -964,8 +957,14 @@ if (e2e) { } } -// ---- Phase 4/5 — finalize: stats.json always, BLOCKED.md on non-DONE ---- -phase('Report') +// ---- Report assembly (issue #29): the script BUILDS the artefact contents — +// no finalize dispatch. The former `finalize` agent only templated the +// already-computed aggregate into two files (~41k real tokens per run); the +// script has no filesystem access, so the templates now live HERE (single +// source) and the END-REPORT carries the finished contents. The orchestrator +// writes them byte-identical, filling exactly the two runtime slots it owns +// anyway: {{DATE}} (`date +%F`) and, in BLOCKED.md, {{FILES_TOUCHED}} +// (`git status --porcelain` — its own Step-3/4 inspection duty). // A clean-tree no-op (mini or standard) gets NO BLOCKED.md (issue #12 / // consistency): there is no dirty tree to explain, so the status + reason ride // the end-report. `noopClean` was set by whichever no-op gate fired above. @@ -986,30 +985,58 @@ const aggregate = { synthBlock || (blocked ? { task: blocked.id, reason: blocked.reason, detail: blocked.detail || null } : null), } -const fin = await agent( - `${STANDING}\n\nFinalize an implement iteration. Using the aggregate below:\n` + - '1. Write a stats file at `/tmp/iter-' + iter_id + '/stats.json` (or under the project stats dir if its facts ' + - 'declare one) containing: iter_id, date (from `date +%F`), mode, outcome, tasks_total, tasks_completed, and ' + - 'blocked_reason (or null).\n' + - (!writeBlockedMd - ? '2. Do NOT write BLOCKED.md — ' + - (outcome === 'DONE' ? 'this run is DONE.\n' : 'this is a clean-tree no-op; there is nothing to explain.\n') - : '2. Write `BLOCKED.md` at the repo ROOT (uncommitted by convention; never staged). Sections: a `# BLOCKED — ' + - 'iter ' + iter_id + '` header with Date/Started-from/Status/Tasks-completed, `## What ran` (one line per DONE ' + - 'task), `## What did not` (the blocked task, its reason, and a one-line suggested next step), `## Concerns`, ' + - 'and `## Files touched` (from `git status --porcelain`).\n') + - 'Report the stats path and whether you wrote BLOCKED.md. Do NOT commit anything.\n\nAGGREGATE:\n' + - JSON.stringify(aggregate, null, 2), - { model: 'sonnet', effort: 'medium', label: 'finalize', phase: 'Report', schema: FINALIZE_SCHEMA }, -) +const statsJson = + JSON.stringify( + { + iter_id, + date: '{{DATE}}', + mode, + outcome, + tasks_total: tasks.length, + tasks_completed: completed.length, + blocked_reason: aggregate.blocked_detail ? aggregate.blocked_detail.reason : null, + }, + null, + 2, + ) + '\n' -// The no-op verdict was already taken before finalize (the tree-check / mini-verify -// gates set outcome + synthBlock + aggregate.blocked_detail), so the end-report -// just relays the final outcome. files_touched comes from the standard-mode -// tree-check (git status --porcelain, untracked-aware); it is null for mini / -// PARTIAL / BLOCKED, where the orchestrator reads BLOCKED.md and inspects the tree. +const blockedMd = writeBlockedMd + ? [ + `# BLOCKED — iter ${iter_id}`, + '', + 'Date: {{DATE}}', + `Started-from: ${startSha}`, + `Status: ${outcome}`, + `Tasks-completed: ${completed.length}/${tasks.length}`, + '', + '## What ran', + ...(completed.length ? completed.map((c) => `- #${c.id} ${c.title}`) : ['- (none)']), + '', + '## What did not', + ...(aggregate.blocked_detail + ? [ + `- ${aggregate.blocked_detail.task != null ? `task #${aggregate.blocked_detail.task}: ` : ''}${aggregate.blocked_detail.reason}`, + ...(aggregate.blocked_detail.detail ? [`- detail: ${aggregate.blocked_detail.detail}`] : []), + ] + : ['- (no blocked detail recorded)']), + '', + '## Concerns', + ...(aggregate.concerns.length ? aggregate.concerns.map((c) => `- ${c}`) : ['- (none)']), + '', + '## Files touched', + '{{FILES_TOUCHED}}', + '', + ].join('\n') + : null -// ---- End-report — the orchestrator reads this, inspects the tree, commits. +// files_touched comes from the standard-mode end-verify (git status +// --porcelain, untracked-aware); it is null for mini / PARTIAL / BLOCKED, +// where the orchestrator inspects the tree itself. + +// ---- End-report — the orchestrator reads this, WRITES the artifacts +// (artifacts.stats_json to the stats path; artifacts.blocked_md — when +// non-null — verbatim to BLOCKED.md at the repo root, filling {{DATE}} and +// {{FILES_TOUCHED}}), inspects the tree, and commits. return { status: outcome, iter_id, @@ -1020,8 +1047,11 @@ return { task_summaries: aggregate.task_summaries, concerns: aggregate.concerns, e2e_fixtures: e2e ? e2e.fixtures || [] : [], - stats_path: fin ? fin.stats_path : null, - blocked_md: fin && fin.wrote_blocked_md ? 'BLOCKED.md (uncommitted)' : null, + artifacts: { + stats_json: statsJson, + stats_path_hint: `/tmp/iter-${iter_id}/stats.json (or the project stats dir if its CLAUDE.md facts declare one)`, + blocked_md: blockedMd, + }, files_touched: treeState ? treeState.files_touched : null, blocked_detail: aggregate.blocked_detail, }