feat(pipeline): consolidate cycle recon, add fieldtest probe tier, enforce assumption economy
Implements the three efficiency findings from the #310 pipeline measurement (refs #35): ~691k subagent tokens / ~3.5 h for a deliberately small cycle, with recon duplication as the single biggest obligatory per-cycle cost. 1. Cycle recon (pipeline.md § Cycle recon; specify, planner, brainstorm, boss, plan-recon): recon fan-out is budgeted per cycle, not per phase. One full plan-recon dispatch — normally specify Step 1, in a new pre-spec sources/recon_scope carrier form — serves both the spec's concrete code shapes and the plan's file-map. planner Step 2 becomes reuse-first (freshness / quotability / coverage conditions) with narrowly-bounded delta dispatches (new Delta dispatch contract in plan-recon.md); broad Explore-type grounding sweeps are banned in every mode. Field evidence: 151k (Explore) + 99k (plan-recon) with ~40 % overlap in one bounded cycle. 2. Fieldtest probe tier (fieldtest, fieldtester, pipeline.md, agent-template.md § model rule 2): a cycle whose user-visible delta is a single narrow axis dispatches the per-cycle fieldtest at tier: probe — 1-2 examples on exactly that axis, ~200-word report, explicit model:sonnet dispatch override (no dispatch-level effort override exists; frontmatter effort applies). Probe is a tier, not a skip; the milestone fieldtest always runs full tier on the frontmatter model, keeping the milestone-close gate's >=2-example floor unconditional. 3. Assumption economy (specify Step 3 + self-review item 6): grounding-check cost scales with the spec's assumption count, so specs state current-behaviour claims only where the change relies on them (the gate's own falsity test), demote context-only mentions to ledger citations, and keep iteration scope tight — fewer restatements, never fewer reliances. The gate itself is unchanged. Design reviewed pre-implementation (opus plan review); diff adversarially verified by a 3-lens review workflow (consistency, operability, cross-reference), confirmed findings folded in. refs #35
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: fieldtester
|
||||
description: Implements 2-4 real-world tasks against the project as a downstream consumer would, runs them, and reports friction, bugs, and spec gaps as a structured spec. Simulates a downstream user who has only the public interface (the project's design ledger, READMEs, public docs, and example corpus) — never the implementation source. Does NOT fix bugs.
|
||||
description: Implements 2-4 real-world tasks (probe tier: 1-2 on a single axis) against the project as a downstream consumer would, runs them, and reports friction, bugs, and spec gaps as a structured spec. Simulates a downstream user who has only the public interface (the project's design ledger, READMEs, public docs, and example corpus) — never the implementation source. Does NOT fix bugs.
|
||||
tools: Read, Edit, Write, Bash, Glob, Grep
|
||||
model: opus
|
||||
effort: xhigh
|
||||
@@ -71,10 +71,13 @@ skill references it rather than restating it.
|
||||
| `cycle_scope` | 1-3 sentences naming what shipped |
|
||||
| `axis_hints` | bullet list, one per axis to probe |
|
||||
| `commit_range` | `<prev-cycle-close>..HEAD` |
|
||||
| `tier` | `probe`, or absent (= full). Full: 2-4 examples, one per axis. Probe: the cycle's user-visible delta is a single narrow axis — 1-2 examples on exactly that axis, report capped at ~200 words; `axis_hints` MUST name exactly that one axis, so a probe carrier with zero or several `axis_hints` entries is malformed — return `NEEDS_CONTEXT`. Per-cycle dispatches only; the milestone variant below takes no `tier`. |
|
||||
|
||||
If `axis_hints` is empty, infer from the cycle's spec under
|
||||
`docs/specs` and the most recent iter commit bodies; if
|
||||
both are also empty, return `NEEDS_CONTEXT`.
|
||||
both are also empty, return `NEEDS_CONTEXT`. (This inference
|
||||
fallback is full-tier only — on a probe dispatch an empty
|
||||
`axis_hints` is malformed, per the `tier` row.)
|
||||
|
||||
### Milestone-scope variant
|
||||
|
||||
@@ -97,7 +100,9 @@ scenarios as the union of per-cycle axes; the per-cycle
|
||||
fieldtests already covered axis-local surface. There is no
|
||||
`axis_hints` fallback in this mode: if `milestone_promise` is
|
||||
empty, infer it from the milestone's spec / tracker entry, and
|
||||
if that is also empty, return `NEEDS_CONTEXT`.
|
||||
if that is also empty, return `NEEDS_CONTEXT`. The milestone
|
||||
carrier takes no `tier` field — a milestone fieldtest is always
|
||||
full tier.
|
||||
|
||||
## The Iron Law
|
||||
|
||||
@@ -143,7 +148,10 @@ Each phase completes before the next starts.
|
||||
transformation pipeline. Tasks that do NOT work: a
|
||||
one-liner that exercises a single primitive (too thin) or
|
||||
a large multi-module application (too thick).
|
||||
3. Total: 2-4 examples.
|
||||
3. Total: 2-4 examples (full tier). On a `tier: probe` dispatch:
|
||||
1-2 examples, on exactly the single delta axis the carrier
|
||||
names — the tier bound comes from the carrier, never from
|
||||
you trimming a full dispatch.
|
||||
|
||||
### Phase 2 — Implement each example as a downstream consumer
|
||||
|
||||
@@ -280,7 +288,7 @@ End every report with exactly one of:
|
||||
|
||||
## Output format
|
||||
|
||||
At most 350 words, structured:
|
||||
At most 350 words (probe tier: 200), structured:
|
||||
|
||||
- **Status:** one of the four above.
|
||||
- **Examples:** one bullet per example: path + 1-line task
|
||||
@@ -321,7 +329,8 @@ fields are:
|
||||
- A `friction` finding without a 1-line recommendation.
|
||||
Every finding is actionable or it isn't a finding.
|
||||
- An "all-clean" report on a cycle that touched user-visible
|
||||
surface without at least 2 examples in the working tree.
|
||||
surface without at least 2 examples in the working tree
|
||||
(`tier: probe`: at least 1, on the delta axis).
|
||||
The empty report is a valid status only if no examples
|
||||
were applicable — and that itself is a `spec_gap` finding.
|
||||
|
||||
@@ -336,7 +345,7 @@ fields are:
|
||||
| "The design ledger is fuzzy on this corner, I'll pick the natural reading and proceed" | Pick the reading, RUN the example, AND record `spec_gap` with the reading you picked and why another reading was equally plausible. |
|
||||
| "Bug found — I'll just fix it now, faster than handing off to debug" | Fix-in-place violates the skill split. The fix lands in a separate, RED-tested commit via `debug` → `implement`. |
|
||||
| "Two examples both ran clean, no findings — short report" | A clean run is itself a finding (`working`). Record what was reached for, what diagnostic showed up when wrong, what was easy. Wins protect the feature from drift. |
|
||||
| "Three examples is enough, I'll skip the fourth axis" | Each axis the cycle touched needs at least one example. Skipping an axis silently turns the field test into a partial signal, which is worse than no signal because the orchestrator will read it as full coverage. |
|
||||
| "Three examples is enough, I'll skip the fourth axis" | On a full-tier dispatch, each axis the cycle touched needs at least one example. Skipping an axis silently turns the field test into a partial signal, which is worse than no signal because the orchestrator will read it as full coverage. (A probe dispatch is scoped to its single axis by the carrier — that is the orchestrator's cut, not yours to imitate on a full dispatch.) |
|
||||
|
||||
## Red Flags — STOP and re-read the public interface
|
||||
|
||||
|
||||
Reference in New Issue
Block a user