feat(pipeline): consolidate cycle recon, add fieldtest probe tier, enforce assumption economy

Implements the three efficiency findings from the #310 pipeline
measurement (refs #35): ~691k subagent tokens / ~3.5 h for a
deliberately small cycle, with recon duplication as the single
biggest obligatory per-cycle cost.

1. Cycle recon (pipeline.md § Cycle recon; specify, planner,
   brainstorm, boss, plan-recon): recon fan-out is budgeted per
   cycle, not per phase. One full plan-recon dispatch — normally
   specify Step 1, in a new pre-spec sources/recon_scope carrier
   form — serves both the spec's concrete code shapes and the
   plan's file-map. planner Step 2 becomes reuse-first (freshness /
   quotability / coverage conditions) with narrowly-bounded delta
   dispatches (new Delta dispatch contract in plan-recon.md);
   broad Explore-type grounding sweeps are banned in every mode.
   Field evidence: 151k (Explore) + 99k (plan-recon) with ~40 %
   overlap in one bounded cycle.

2. Fieldtest probe tier (fieldtest, fieldtester, pipeline.md,
   agent-template.md § model rule 2): a cycle whose user-visible
   delta is a single narrow axis dispatches the per-cycle
   fieldtest at tier: probe — 1-2 examples on exactly that axis,
   ~200-word report, explicit model:sonnet dispatch override (no
   dispatch-level effort override exists; frontmatter effort
   applies). Probe is a tier, not a skip; the milestone fieldtest
   always runs full tier on the frontmatter model, keeping the
   milestone-close gate's >=2-example floor unconditional.

3. Assumption economy (specify Step 3 + self-review item 6):
   grounding-check cost scales with the spec's assumption count,
   so specs state current-behaviour claims only where the change
   relies on them (the gate's own falsity test), demote
   context-only mentions to ledger citations, and keep iteration
   scope tight — fewer restatements, never fewer reliances. The
   gate itself is unchanged.

Design reviewed pre-implementation (opus plan review); diff
adversarially verified by a 3-lens review workflow (consistency,
operability, cross-reference), confirmed findings folded in.

refs #35
This commit is contained in:
2026-07-25 13:52:04 +02:00
parent b2c92db725
commit fe5ca8f8c0
9 changed files with 353 additions and 67 deletions
+50 -11
View File
@@ -1,6 +1,6 @@
---
name: fieldtest
description: Orchestrator-dispatched only, after audit closes clean (or with ratified drift only), when the orchestrator judges the iteration is complete and wants a field test. Picks 2-4 real-world tasks within the cycle's scope, implements each as a downstream consumer of the project, runs them, and writes a friction-and-bug spec. The spec feeds the next plan as a reference. Implementer simulates a downstream user who has only the public interface — never the project's implementation source.
description: Orchestrator-dispatched only, after audit closes clean (or with ratified drift only), when the orchestrator judges the iteration is complete and wants a field test. Picks 2-4 real-world tasks within the cycle's scope (probe tier, for a single-axis surface delta: 1-2 tasks on that axis), implements each as a downstream consumer of the project, runs them, and writes a friction-and-bug spec. The spec feeds the next plan as a reference. Implementer simulates a downstream user who has only the public interface — never the project's implementation source.
---
# fieldtest — usability field test for a shipped cycle
@@ -24,8 +24,8 @@ next to cycle-design specs under `docs/specs`, with a
`fieldtest-` prefix in the slug.
The substantive process — read the design ledger + cycle spec
+ recent iter/audit commit bodies, pick 2-4 real-world tasks
per cycle axis, implement each as a downstream consumer, run
+ recent iter/audit commit bodies, pick the dispatched tier's
examples, implement each as a downstream consumer, run
the result, classify findings, write the spec — lives in
`agents/fieldtester.md`. That file also carries the spec
template, the source-isolation discipline (no reading under
@@ -61,6 +61,44 @@ here. The milestone fieldtest is skippable on the same terms as
the per-cycle one, lifted to milestone level: a milestone whose
entire scope is internal (no user-visible surface) is exempt.
## Two tiers: probe and full
Orthogonal to scope, the **per-cycle** fieldtest runs at one of
two tiers. The orchestrator picks the tier at dispatch and
carries it in the carrier's `tier` field — the field, its
example bounds, and its report cap are defined authoritatively
under **Carrier contract** in `agents/fieldtester.md` and
deliberately not restated here:
- **Full** (the default; `tier` absent): one example per axis
the cycle touched — the shape the rest of this file describes.
- **Probe**: for a cycle whose user-visible delta is a **single
narrow axis** — one flag, one diagnostic, one configuration
key. The carrier's `axis_hints` then names exactly that one
axis. Field evidence (issue #35): a full fieldtest cost ~89k
tokens on a one-flag cycle even at minimal example count, yet
that same dispatch found a real self-description bug — the
probe tier keeps that signal at a fraction of the cost.
**Probe is a tier, not a skip.** The skip rules below are
unchanged: a surface-touching cycle still runs its fieldtest;
probe merely right-sizes the dispatch when the surface delta is
one axis. A multi-axis cycle takes the full tier — when in
doubt, full.
**Probe dispatch mechanics.** A probe dispatch passes an
explicit `model: sonnet` override on the agent dispatch — the
dispatch-level model override takes precedence over the agent's
opus frontmatter; the conditioned exception is documented in
`../docs/agent-template.md` § model, rule 2. (The direct
agent-dispatch surface has no per-dispatch effort override, so
the frontmatter effort applies unchanged; the probe's economy
comes from the model tier and the one-axis scope.) The
judgement surface a probe covers is one axis, and the
orchestrator triages every finding anyway. The **milestone
fieldtest always runs full tier on the frontmatter model** — its
carrier takes no `tier` field.
## When to Use / Skipping
Triggers:
@@ -116,11 +154,12 @@ carrier.
## Dispatch
Dispatch `fieldtester` with the carrier from the Handoff
Contract below. The agent picks 2-4 examples (one per axis
the cycle touched; fewer loses the signal on variation, more
overflows one readable report), implements them as downstream
consumers, runs them, classifies findings, and writes the
spec. The two artefact kinds part ways at commit: the **fixtures are
Contract below. The agent picks its examples per the dispatched
tier (bounds in the `tier` carrier row; on full, one per axis
the cycle touched — fewer loses the signal on variation, more
overflows one readable report), implements them as
downstream consumers, runs them, classifies findings, and
writes the spec. The two artefact kinds part ways at commit: the **fixtures are
committed as normal code** (they are consumer test assets — suggested
subject `fieldtest: <cycle> — <N> examples, <K> findings`), while the
**fieldtest spec is a git-ignored working file that is never committed**.
@@ -137,10 +176,10 @@ tree as the next cycle's input; it plays no part in the delete decision
## Handoff Contract
`fieldtest` consumes (from orchestrator at cycle close) the carrier
fields `cycle_id`, `cycle_scope`, `axis_hints`, and `commit_range`,
defined authoritatively under **Carrier contract** in
fields `cycle_id`, `cycle_scope`, `axis_hints`, `commit_range`, and
`tier`, defined authoritatively under **Carrier contract** in
`agents/fieldtester.md` (which also specifies the empty-`axis_hints`
fallback).
fallback and the absent-`tier` default).
`fieldtest` produces, for the orchestrator, the fields `spec_path`,
`examples_added`, and `findings`, defined authoritatively under