fe5ca8f8c0d89a21f8b052e9277c172dfd4625c8
15 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 claude
|
4996f2d919 |
Revert "feat(models): move the implementer to opus/high"
This reverts commit |
||
|
claude
|
cdbc2d03ae |
feat(models): move the implementer to opus/high
Owner-ratified 2026-07-20: authored code is the pipeline's primary output, so plan execution takes the judgment model — but keeps the in-loop effort tier (high, not xhigh): the implementer is the most-dispatched per-task role and volume x wall-clock still governs its thinking budget. Applies to the frontmatter pin and the three implement-loop dispatches (impl, impl-fix-spec, impl-fix-qual). The compiler-driven-edit arm deliberately keeps its implementer dispatch at sonnet/high: compiler-enumerated propagation is mechanical, and the arm's policy header pins its own tier. |
||
|
claude
|
44f45d16a9 |
feat(implement-loop): tier the quality review by independently-measured diff size
qual+repairs are the pipeline's largest cost pool (51M of 164.5M real tokens, 31%, in the 206-run corpus), with opus weighing ~5x on the quota side. The opus pin's own rationale (consequence-of-a-miss, agent-template § model) scales with diff size — so the tier now does too: the spec-reviewer, which runs before quality, reads the diff itself and is independent of the implementer, reports diff_magnitude (git diff HEAD --shortstat) and touches_contract; <=25 changed lines and no contract-referenced path -> sonnet/high, anything larger, contract-touching, or unmeasured -> opus/xhigh (fail conservative). The independent end-verify/mini-verify suite gate is the deterministic backstop that makes the lower tier defensible; the chosen tier is logged per task (qual_tier) in the end-report. Sampling phase per the issue's acceptance: the first ~10 real small-diff iterations get a one-off opus second review; important+ divergences are recorded on the issue before the threshold counts as proven. closes #30 |
||
|
claude
|
bff2120f42 |
fix(implement): spec-reviewer routes reality-contradicted task text to unclear
A requirement whose scripted literal is contradicted by verified reality (or by another requirement of the same task) is a plan defect, not a missing requirement: the persona and the loop's dispatch prompt now direct that case to status `unclear`, which the loop already maps to an immediate spec-ambiguous BLOCKED for the orchestrator to adjudicate. Grounding (aura runs, 2026-07-12): two of the last three cycles' four review-loop-exhausted BLOCKs were exactly this shape. wot-241-t34 (wf_2c988736) re-flagged the same empirically-false exit-code literal (task text Some(1), binary provably exits 2) as non_compliant across all three rounds; wot-241-t4 (wf_f8fe5359) did the same for an internally contradictory task (step 1 binds every param the step-5 sweep needs free) while its reviewer's own ambiguity field conceded the text was "technically infeasible". Both burned two repair rounds that could not converge before BLOCKing; the unclear route ends the same runs after round one with the contradiction named. The persona gains a "Task text vs reality" section (verify the implementer's documented deviation yourself; unforced deviation stays non_compliant), the matching rationalisation-table row, and a red flag; the status protocol states that unclear outranks non_compliant for verified contradictions. |
||
|
claude
|
5606ca0213 |
fix(implement-loop): minor-only quality rounds surface as concerns, not BLOCKED
A quality changes_requested verdict whose findings are all Minor,
arriving after at least one repair dispatch, now breaks the loop with
the findings surfaced as concerns instead of consuming the retry budget
toward a hard BLOCKED.
Grounding (aura runs, 2026-07-12/13): each quality round reads the diff
fresh, so late rounds keep surfacing NEW polish with severity trending
down (Important -> Minor -> comment-only Minor). Run sab-246-t56
(wf_643ee9fa) fixed its one Important finding in round 1, then hard-
BLOCKed an 85-minute green task in round 3 over two stale comment
cross-references; the sibling task was never attempted and the
re-dispatch cost another 32 minutes. Three of 14 runs across the last
three cycles ended review-loop-exhausted with no commit-blocking
residue.
Mechanically, the QUAL_SCHEMA issues array is now structured as
{severity: 'important'|'minor', text} so the valve keys on a declared
classification instead of parsing free-text prefixes; the fingerprint
backstop and the repair-brief prompt render the new shape, and the
quality-reviewer persona documents the honest-severity requirement.
Important findings still consume the full retry budget and BLOCK.
|
||
 Brummel
|
e7009bc304 |
fix(implement-loop): guard against a task discarding sibling tasks' uncommitted work
During a multi-task run nothing commits between tasks, so one task's file-level `git checkout -- <file>` / `git restore <file>` silently destroys an earlier task's uncommitted DONE work in a shared file. The loss was structurally invisible: spec- and quality-reviewers are scoped to the current task's footprint, diff_fingerprint only detects same-task repair cycles, and the end-of-iteration tree gate counts files (the surviving changes keep it nonzero). Observed in the wild as a PARTIAL run whose DONE reports did not match the tree. Two layers, both plugin-side (harness-level prevention is ruled out — decision log on the issue): Prose guard — implementer.md Step 8 no longer sanctions file checkout for scope curation: over-reach is undone by editing back, a broken intermediate state is repaired forward or reported BLOCKED (new fourth BLOCKED bucket in the status protocol). A matching Iron Law line in implement/SKILL.md covers all in-loop agents, whole-file and --patch; docs/conventions.md now marks the checkout discard idiom as the orchestrator's, between iterations. Mechanical guard — after every task of a multi-task run a snapshot agent records `git stash create` (a dangling commit; HEAD, index, and tree untouched — semantics verified empirically in a scratch repo) plus the `git diff HEAD --name-only --no-renames` path set. A path that was HEAD-modified at one boundary and gone at the next trips a hard BLOCKED naming the lost paths, the boundary, and the recovery snapshot sha (`git show <sha>:<path>`); the discard verdict outranks per-task outcomes in blocked_detail since the reports and the tree have diverged. Coarse by design, in both directions, and documented as such: a checkout-then-re-edit or a --patch hunk restore escapes the comparison (the snapshot keeps it diagnosable); a legitimate back-to-HEAD edit trips it (the verdict says adjudicate against the snapshot). --no-renames keeps a staged rename from reading as a loss; untracked files are outside the threat model (checkout cannot discard them). Cost: one sonnet/medium call per task, multi-task runs only; single-task and mini runs are unchanged. Verified: node --check on the async-wrapped script and a stub-agent harness — discard trips the hard BLOCKED and stops the loop; an accumulating happy path, a plan-intended deletion, a single-task run, and a dead snapshot agent all pass without a false positive. closes #23 |
||
|
Brummel
|
edbbb68f97 |
feat(agents): pin explicit reasoning effort on every agent and workflow call
Effort joins model as a mandatory pin: an omitted field inherits the session effort, coupling every dispatch's thinking budget to whatever the user happens to be chatting at (often xhigh) — the same session-state coupling the model pin removes. The assignment follows the model split: - xhigh on every opus agent (judgement roles are the pipeline's quality floor and must not degrade with the session); - high on every sonnet agent (tightly-scoped plan execution gains little from xhigh but pays its latency per dispatch, and these are the per-task in-loop roles — wall-clock is the efficiency metric; not lower than high, since re-loops cost more than saved thinking); - medium inline in the workflow scripts for schema-bound extraction/verification stages that author no code (preflight, plan-extract, mini-verify, tree-check, finalize, build/suite verify). Workflow agent() calls pass effort explicitly on every call — whether frontmatter effort propagates through an agentType dispatch is undocumented, so the scripts do not rely on it. Policy documented in docs/agent-template.md § effort, mirroring § model. |
||
|
Brummel
|
6bfec9655c |
feat(agents): pin explicit model on every agent and workflow call
Agents and workflows previously carried no model field, so every dispatch inherited the session model — including fable, which is banned for all plugin agents and workflows by owner decree. Every dispatch now pins opus or sonnet explicitly. - opus (low-volume judgment gates whose misses silently poison downstream work): architect, bencher, debugger, fieldtester, grounding-check, plan-recon, quality-reviewer, spec-skeptic, tdd-author - sonnet (mechanical scope, in-loop or fanned out): docwriter, glossary-extractor, implementer, spec-reviewer, synthetic-user, tester - workflows: all 13 agent() call sites pin a model — sonnet everywhere except the quality-reviewer gate in implement-loop, the loop's last correctness check (spec-reviewer only gates task-text correspondence; real-bug finding is the documented opus strength) - docs/agent-template.md: model is now a mandatory frontmatter field, with the assignment rule and the fable ban recorded |
||
|
Brummel
|
52db1abe10 |
fix(implement-loop): hold plan-contradicting quality findings instead of oscillating to a false BLOCKED
The per-task quality re-loop treated every `changes_requested` as "deviate to satisfy", and the quality-repair dispatch was not even given the task text. When a finding's only remedy contradicted a plan-prescribed name/signature, the implementer renamed off-plan, the next review flagged "diverges from plan", and the loop oscillated name-misleads <-> diverges-from-plan until the retry cap, emitting a false BLOCKED on code that was green the whole time. Fix (issue's option 2 — the plan knowledge lives with the implementer, which holds the task text; the quality-reviewer is deliberately blind to it, so option 1 would have breached that separation): - The quality-repair dispatch now receives the task text and a HOLD CLAUSE. A cosmetic finding (plan kept => build+tests green) is KEPT and recorded in a new `held` field; the loop surfaces it as a concern instead of chasing a deviation. A finding the implementer judges correctness-breaking escalates to BLOCKED, never a silent hold. - The hold is keyed on two structural signals, never on a self-reported status enum (which the implementer contract overloads): the `held` array, and a no-op backstop over a required `diff_fingerprint` (a changes_requested verdict over an already-seen diff-state means the repair was a no-op or cycled back — re-running quality is futile). Fingerprints are tracked in a Set so A-B-A edit-then-revert is caught. - A no-op-backstop concern is labelled neutrally (a byte-identical diff cannot tell a principled plan-hold from an ignored bug); implement SKILL.md Step 3 now routes a held/unresolved quality finding to orchestrator hand-verification before committing, even under /boss. The held/no-op partition is the implementer's judgement, not enforced in code — the residual fail-open is bounded to cosmetic-on-green findings and disclosed, with the orchestrator's Step-3 inspection as the backstop, the same trust placed in its other self-reports. Verified across three rounds of adversarial review (held-the-plan paths, fail- open laundering, schema/Set mechanics) — all closed. closes #10 |
||
|
Brummel
|
7a58a530b1 |
feat(pipeline): route to the lightest correct methodology; move execution loops onto the Workflow substrate
The selector forced every task through the heaviest methodology's
critical path: a behaviour-preserving, type-enumerable change paid the
same specify -> planner -> implement front-half as a novel feature,
because it was neither new behaviour (tdd) nor an observed bug (debug)
and so fell to specify by elimination. Two coupled defects — a selector
with no verification axis, and an all-or-nothing executor — kept the
existing lighter path unreachable and uneconomical. This fixes both.
Part A — verification-keyed selector (boss/SKILL.md):
- Replace the three-way "design line" with an ordered cascade that adds
a verification/enumeration axis ahead of the settled-vs-fork question.
Each lighter arm carries a positive trigger matched by signature, not
reached by elimination.
- New `compiler-driven` arm: a type/signature edit at a definition site
that propagates mechanically. Observe-then-bounce — make the edit,
build, run the suite; clean build AND suite green unchanged commits;
a hole bounces up (specify for a design choice, tdd for discovered
test-specifiable new behaviour); a regression bounces to debug.
- The observed-bug RED-first gate is first in the cascade, so a
mechanical-looking fix cannot bypass it.
- The straddle rule ("add an enum variant") is codified as a rule:
mechanical/forwarding -> compiler-driven; encodes new behaviour ->
tdd/spec; doubt routes up.
- The executor is the elevated inline carve-out plus a shipped workflow,
not a heavy new skill ("the largest concrete win is small").
Part B — Workflow substrate (implement/workflows/):
- implement-loop.js: the per-task loop as a deterministic script. Each
phase (implementer -> spec-compliance -> quality, + tester for E2E) is
a separate top-level agent() call, so a single phase is independently
invokable and inter-phase aggregation/re-loop is code. Retires the
implement-orchestrator agent's inline-role-switch workaround (the four
phase agents survive as the agent-types the script dispatches).
- compiler-driven-edit.js: the observe-then-bounce loop.
- install.sh / uninstall.sh symlink shipped workflows into
~/.claude/workflows/.
- specify and brainstorm stay prose + interactive (human-intent oracle);
only the autonomous/mechanical loops moved. try-and-error is deferred.
Docs (pipeline taxonomy, design, agent-template, migration, README) and
all selector<->executor cross-references updated; the arm and its
executor are co-located so a future re-route through the full loop is a
visible regression.
Verified by an adversarial multi-agent pass: PASS on all six acceptance
criteria; two coherence concerns fixed. The shipped scripts are
syntax-validated but exercised only in a downstream target project (the
skills repo is not itself a pipeline target).
closes #7
|
||
|
Brummel
|
26e9630496 |
refactor: drop dev-cycle-profile.yml for conventions + CLAUDE.md facts
The profile was never parsed — it was prose the skill bodies told the model to read, so most slots were dead, constant across every project, or fiction (the whole pipeline block, including the "tdd is opt-in" claim, was enforced by nothing). Split it in two: constants become fixed conventions named directly by the skills (new docs/conventions.md), and the few genuinely per-project facts move to each project's CLAUDE.md under '## Skills plugin: project facts'. tdd/fieldtest/docwriter are now always available; the only behavioural toggle left is spec auto-sign. Delete docs/profile-schema.md and templates/project-profile.yml; add docs/conventions.md and a project-facts section to templates/CLAUDE.md.fragment; rewrite all SKILL/agent prose and the pipeline/design/migration/README/INSTALL docs accordingly. |
||
|
Brummel
|
137ec21e26 |
docs(tdd): wire sibling skills to the new entry path
The tdd skill referenced its neighbours (implement, brainstorm, debug) but none referenced it back. Close the loop so the new executable-spec-first entry path is reachable and consistent from every skill that describes a relationship it now belongs to: - implement: mini-mode trigger + dispatch example now cover a RED-first handoff from `debug` OR `tdd` (was debug-only); the orchestrator's task template and Phase-3 skip note generalised. This was real drift — mini-mode is no longer debug-exclusive. - planner: skip rule gains the `tdd` case (it skips brainstorm AND planner — the RED executable-spec is the plan). - brainstorm: `tdd` added to the permitted-skip list as the profile-gated alternative entry path, plus a cross-ref marking brainstorm as the bounce-back target when behaviour stops being test-specifiable. - boss: pipeline diagram, Step-3 routing prose, and cross-refs. A test-specifiable feature issue is dispatched to `tdd` autonomously, the same way a bug issue goes to `debug`; this is NOT a new-cycle bounce-back (the test is the spec). The bounce-back fires only reactively, when tdd surfaces a genuine design fork. - debug: reciprocal sibling note + cross-ref (new behaviour is tdd's job; debug is for regressions of existing behaviour). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Brummel
|
0b969838c0 |
refactor: single-source carrier/handoff contracts across skills
Apply the debug-skill pattern (commits 6410732..de42974) to the remaining skills: the agent file is the single source for each interface field's semantics; the SKILL.md copy is reduced to field names plus a pointer, marking the agent's contract table authoritative. Because SKILL.md loads into orchestrator context and agents/*.md into the subagent's fresh system prompt with no transclusion, duplicated field tables drift apart over time. - brainstorm: carrier (spec_path/iteration_scope) -> grounding-check; the absolute-path requirement now lives only in the agent. - planner: carrier (spec_path/iteration_scope/focus_hint) -> plan-recon, including the mandatory/optional markers and the BLOCKED-on-missing rule SKILL.md had omitted. - fieldtest: carrier + produced fields -> fieldtester; the skill-level `status` roll-up (clean/friction_found/bugs_found/infra_blocked), which is not part of the agent's run-status protocol, stays defined in SKILL.md only. - docwriter: carrier + produced fields -> docwriter agent. - implement: carrier (iter_id scratch-dir/stats/not-a-branch semantics) -> implement-orchestrator; per-task sub-status vocabulary moved into the orchestrator-agent (it runs the loop in a fresh context and could not read SKILL.md at runtime, yet referenced "the sub-status table" by name); task_text_path single-sourced in implementer with spec-reviewer cross-referencing. audit was already the reference implementation (pointers, no restated contracts) and is unchanged. closes #2 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Brummel
|
540741e6e9 |
polish: six fixes from skill-migration reviews
Three minor-issues + three optional follow-ups identified by the three parallel reviewers of debug/docwriter/audit, fieldtest/planner/brainstorm, and implement. None blocking; all close calibrated gaps the bulk migration left. Minor-issues: - audit/SKILL.md: handoff field `regression_results` now explicitly carries "raw output verbatim (no rounding, no summarisation)" — restores the verbatim-numbers emphasis the original `bench_numbers` field name implied. - audit/agents/bencher.md: hypothesis examples grouped by axis (Tail-latency / Count reduction / Scalability / Overhead vs floor) with concrete numerical anchors (2× of median, ≥80%, 10-million-element, ±15%). Subjects stay domain-neutral but the calibrated bite returns. The preamble enumerates plausible subjects (allocators, query plans, request handlers, parsing strategies, codecs, caches) so the reader has concrete domains to anchor on. - implement/agents/implementer.md: "Architecture rules" section gains a 5-bullet kind-list (determinism contracts; backend / FFI constraints; schema versions; memory / resource models; effect / capability discipline) so the implementer has scaffolding to look for even when the project's CLAUDE.md is sparse. The "BLOCKED on contradiction" mechanism stays the load-bearing rule. Optional follow-ups: - brainstorm/SKILL.md Step 7.5: failure-mode procedure restores the structured backlog-issue fields (Title / Label / Body with `depends on:` and `context:` lines) and the milestone-container option for big deferred work (Gitea milestones, GitHub milestones, Linear projects). - templates/CLAUDE.md.fragment: adds an optional "Lockstep-invariant pairs" section so projects that have cross-file pairings can declare them in a way the architect agent and plan-recon agent already know to consult. Projects without such pairings omit the section; both agents handle absence gracefully. No discipline regressions; no Iron Law / Common Rationalisations / Red Flags edits. Pure scaffold-and-anchor improvements. |
||
|
Brummel
|
22653e410a |
implement: skill + 5 agents migrated — migration complete
Eighth and final skill via the boss pattern. implement carries the per-task loop mechanics; implementer + spec-reviewer + quality-reviewer + tester are phase references the implement-orchestrator inhales at each role-switch. All eight skills + their agents now live in the plugin. AILang can switch over by dropping a project profile + removing its in-tree skills/ in a follow-up. Vocabulary substitutions: - AILang → "this project" - milestone → cycle - Boss → orchestrator (consistently) - ailang-implement-orchestrator → implement-orchestrator - ailang-implementer → implementer - ailang-spec-reviewer → spec-reviewer - ailang-quality-reviewer → quality-reviewer - ailang-tester → tester AILang-specific bits dropped or generalized: - `docs/plans/<iteration>.md` / `docs/specs/<milestone>.md` → `paths.plan_dir` / `paths.spec_dir` - `/tmp/ail-iter/<iter_id>/` → `/tmp/iter-<iter_id>/` - `bench/orchestrator-stats/` → `/tmp/iter-<iter_id>/stats.json` with note that a project-configured stats dir can be used - `cargo build --workspace` / `cargo test --workspace` → `commands.build` / `commands.test` - `examples/<name>.ail.json` → "fixture in the project's canonical fixture form" - `crates/ail/tests/e2e.rs` → "the project's E2E test location" - The entire AILang "Architecture rules (binding)" block in implementer (determinism via BLAKE3, LLVM text IR, schema ailang/v0, RC + bump allocator, Type::Fn effects, Implicit-mode rule) → generic "the binding architectural rules are declared in CLAUDE.md and the design ledger" - `git log --all --grep='or.2'` reference in implement-orchestrator → dropped (was AILang project history) - `bench/orchestrator-stats/<YYYY-MM-DD>-iter-<iter_id>.json` → generic stats path with project-configured option - AILang-specific quality-bar examples (Implicit-mode RC unflagged, libllvm call, schema break without migration note) → generic "anything that violates a binding architectural rule from CLAUDE.md / design ledger" - AILang-specific tester examples (`(show 42)` Int dispatch, ailang/v0 schema) → generic "feature X variant" / "API response" / observable-behaviour framing - `Brummel` references → dropped - `notify.sh` hardcoded → "the project's configured notification command" - `iter/...` branches (legacy AILang concept) → "iteration branches are not used" Universal substance preserved verbatim across all six files: - implement SKILL Iron Law (7 lines including new-cycle triggers and BLOCKED.md convention) - Per-task sub-status table (9 rows) - implement-orchestrator: 7-phase process (Phase 0 clean tree, Phase 1 context load mode-dependent, Phase 2 per-task loop with 4 sub-phases, Phase 3 E2E, Phase 4 BLOCKED.md, Phase 5 stats, Phase 6 end-report) - implement-orchestrator: BLOCKED.md template, stats.json schema, end-report fixed structure - implementer: TDD-first discipline, RED-first exceptions, 6-step process, 4-state status protocol - spec-reviewer: missing/unrequested-extra definitions, 7-step process, 4-state status protocol - quality-reviewer: 8 quality-bar categories, severity definitions, 6-step process incl. Strengths-first rule - tester: 4-clause Iron Law, "named property" doctrine, 4-state status protocol Common Rationalisations + Red Flags tables preserved across all six files with vocabulary substitution only. No discipline compressed. All seven sibling skills (boss, brainstorm, planner, debug, audit, fieldtest, docwriter) and the AILang reference agents now resolve correctly via `../<skill>/...` relative paths post-flatten. |