Brummel
08bc2aa027
Per-cycle specs and plans stop being git-tracked commit-then-git-rm artefacts and become git-ignored working files: created on disk for the active cycle, read across sessions from the working tree, and shell-rm'd at cycle close by audit. Nothing is committed, so there is no git-history copy — the durable record of a cycle's intent is the design ledger only, and no durable artefact (code, test, doc, ledger) may cite a spec/plan by number or path. - conventions/pipeline/design/README: invert the git-tracked -> git-rm -> git-history-recovery model to gitignored -> shell-rm -> ledger-only. - naming: drop the NNNN file counter; files are slug.md (fieldtest specs keep a fieldtest- prefix as the cycle-close discriminator). - cycle number: separated from the (now-gone) file number, sourced from the feat(NNNN)/audit(NNNN) commit subjects, which cycle commits carry. - specify/planner/fieldtest/fieldtester: no commit step; fieldtest splits fixtures (committed as code) from its spec (gitignored), with the cross-cycle rule that a fieldtest- spec belongs to the next cycle. - boss: auto-sign auditability moves from the (boss-signed) commit subject to the run's reference issue; rollback reverts only code. - audit: new Step 5 shell-rm's the cycle spec+plan after the architect reads them, sparing fieldtest- specs. Companion machine config (outside this repo): the global git excludes (~/.config/git/ignore) gains docs/specs/ and docs/plans/ so the ignore applies to every project at once.
185 lines
8.0 KiB
Markdown
185 lines
8.0 KiB
Markdown
---
|
|
name: fieldtest
|
|
description: Orchestrator-dispatched only, after audit closes clean (or with ratified drift only), when the orchestrator judges the iteration is complete and wants a field test. Picks 2-4 real-world tasks within the cycle's scope, implements each as a downstream consumer of the project, runs them, and writes a friction-and-bug spec. The spec feeds the next plan as a reference. Implementer simulates a downstream user who has only the public interface — never the project's implementation source.
|
|
---
|
|
|
|
# fieldtest — usability field test for a shipped cycle
|
|
|
|
> **Violating the letter of these rules is violating the spirit.**
|
|
|
|
## Overview
|
|
|
|
`audit` measures drift between the design ledger and what the
|
|
codebase claims to do. `fieldtest` measures something audit
|
|
cannot: whether the project, *as it now ships*, is usable by
|
|
a downstream consumer who has only the public interface to
|
|
work from. If a freshly shipped feature is awkward to reach
|
|
for, or if the public docs leave a corner ambiguous that real
|
|
usage immediately exercises, the cycle has shipped a latent
|
|
debt — even when audit reports `clean`.
|
|
|
|
The skill produces a friction-and-bug spec that the next
|
|
iteration's `planner` consumes as a reference. The spec sits
|
|
next to cycle-design specs under `docs/specs`, with a
|
|
`fieldtest-` prefix in the slug.
|
|
|
|
The substantive process — read the design ledger + cycle spec
|
|
+ recent iter/audit commit bodies, pick 2-4 real-world tasks
|
|
per cycle axis, implement each as a downstream consumer, run
|
|
the result, classify findings, write the spec — lives in
|
|
`agents/fieldtester.md`. That file also carries the spec
|
|
template, the source-isolation discipline (no reading under
|
|
the project's code roots or its benchmark directory, if it has
|
|
one — its CLAUDE.md project facts), and the per-finding
|
|
classification rules. This skill file only governs trigger,
|
|
dispatch, and handoff.
|
|
|
|
## Two scopes: per-cycle and milestone
|
|
|
|
`fieldtest` runs at two scopes, both using the same fieldtester
|
|
agent — the carrier picks the scope:
|
|
|
|
- **Per-cycle fieldtest** (the default this file otherwise
|
|
describes): dispatched after a surface-touching cycle's audit,
|
|
carrier scoped to that cycle (`cycle_id` / `cycle_scope`).
|
|
Catches cycle-local friction early. Optional / orchestrator
|
|
judgement, as below.
|
|
- **Milestone fieldtest**: the closing gate for a tracker
|
|
milestone. Carrier scoped to the milestone's promise
|
|
(`milestone_id` / `milestone_promise`); the fieldtester
|
|
derives curated end-to-end scenarios top-down from that
|
|
promise and proves the shipped implementation delivers it. Its
|
|
green status roll-up is required before a surface-touching
|
|
milestone may be closed.
|
|
|
|
Both carriers are defined authoritatively under **Carrier
|
|
contract** in `agents/fieldtester.md` (the milestone variant
|
|
under its **Milestone-scope variant** sub-heading). The gate the
|
|
milestone fieldtest feeds is defined once in
|
|
`../docs/pipeline.md` § Milestone-close gate — not restated
|
|
here. The milestone fieldtest is skippable on the same terms as
|
|
the per-cycle one, lifted to milestone level: a milestone whose
|
|
entire scope is internal (no user-visible surface) is exempt.
|
|
|
|
## When to Use / Skipping
|
|
|
|
Triggers:
|
|
|
|
- Orchestrator-dispatched after audit closes cleanly, when
|
|
the orchestrator believes the iteration is correct and
|
|
wants a field test. The pre-condition is a clean (or
|
|
ratified) audit; the trigger is the orchestrator's
|
|
judgment-call that the iteration is complete.
|
|
- A shipped feature is suspected of being awkward in some
|
|
way the earlier brainstorm could not foresee.
|
|
- A user-facing surface change has just landed.
|
|
|
|
Findings cluster around two shapes — simple bugs (route via
|
|
`debug` → `implement` mini) or catastrophic architecture
|
|
problems (route via next `brainstorm`). Friction and spec-gap
|
|
items are also possible; the routing table below applies in
|
|
all cases.
|
|
|
|
**Skipping is permitted only** for:
|
|
|
|
- Bug-fix iterations (use `debug`).
|
|
- Tidy iterations whose scope is purely internal cleanup with
|
|
zero surface impact.
|
|
- Iterations whose entire scope is documentation.
|
|
|
|
**Skipping is not permitted** for:
|
|
|
|
- A cycle that introduced or changed surface syntax, schema,
|
|
effects, API, configuration format, or any user-visible
|
|
diagnostic.
|
|
- A cycle whose commit body or backlog issue claims "users
|
|
can now do X". `fieldtest` is the gate that empirically
|
|
substantiates such claims; an unverified claim is drift.
|
|
|
|
## The Iron Law
|
|
|
|
```
|
|
THE FIELDTESTER WORKS FROM THE PUBLIC INTERFACE ONLY — NOT FROM THE IMPLEMENTATION SOURCE.
|
|
EVERY EXAMPLE IS WRITTEN IN THE PROJECT'S CANONICAL AUTHORING / CONSUMER FORM.
|
|
EVERY FRICTION POINT AND BUG IS RECORDED. NONE IS WORKED AROUND.
|
|
```
|
|
|
|
The first clause is load-bearing: the whole point of the
|
|
field test is to simulate a downstream consumer who has only
|
|
the public interface. The agent file enforces this with a
|
|
hard path allowlist (computed from the project's public
|
|
interface, if it has one — its CLAUDE.md project facts); the
|
|
orchestrator must trust that contract
|
|
and not feed the agent implementation-internal hints in the
|
|
carrier.
|
|
|
|
## Dispatch
|
|
|
|
Dispatch `fieldtester` with the carrier from the Handoff
|
|
Contract below. The agent picks 2-4 examples (one per axis
|
|
the cycle touched; fewer loses the signal on variation, more
|
|
overflows one readable report), implements them as downstream
|
|
consumers, runs them, classifies findings, and writes the
|
|
spec. The two artefact kinds part ways at commit: the **fixtures are
|
|
committed as normal code** (they are consumer test assets — suggested
|
|
subject `fieldtest: <cycle> — <N> examples, <K> findings`), while the
|
|
**fieldtest spec is a git-ignored working file that is never committed**.
|
|
|
|
The fieldtest spec is written at cycle N's close but is an **input to
|
|
the next cycle's `planner`**, so it belongs to cycle N+1, not N: the
|
|
cycle-close sweep spares it (its `fieldtest-` filename prefix is the
|
|
discriminator) and it is shell-`rm`'d at N+1's close alongside N+1's own
|
|
spec and plan. If the consuming cycle closes without a `planner` (e.g. a
|
|
milestone close), the orchestrator discards it explicitly.
|
|
|
|
## Handoff Contract
|
|
|
|
`fieldtest` consumes (from orchestrator at cycle close) the carrier
|
|
fields `cycle_id`, `cycle_scope`, `axis_hints`, and `commit_range`,
|
|
defined authoritatively under **Carrier contract** in
|
|
`agents/fieldtester.md` (which also specifies the empty-`axis_hints`
|
|
fallback).
|
|
|
|
`fieldtest` produces, for the orchestrator, the fields `spec_path`,
|
|
`examples_added`, and `findings`, defined authoritatively under
|
|
**Output format** in `agents/fieldtester.md` — single-sourced in
|
|
the agent file, deliberately not restated here so the two files
|
|
cannot drift.
|
|
|
|
It additionally reports one skill-level `status` roll-up — one of
|
|
`clean` / `friction_found` / `bugs_found` / `infra_blocked`. This
|
|
field is owned here (it is not part of the agent's per-run status
|
|
protocol), so it is defined in this file only.
|
|
|
|
The orchestrator drives downstream:
|
|
|
|
- `bug` → `debug` (RED-first; GREEN via `implement` mini-mode)
|
|
- `friction` → next `brainstorm` or a tidy iteration via
|
|
`planner`
|
|
- `spec_gap` → ratify by updating the design ledger (and
|
|
naming the gap in the closing commit body), or tighten the
|
|
design ledger
|
|
- `working` → carry-on (worth recording, no follow-up)
|
|
|
|
`fieldtest` does NOT self-resolve.
|
|
|
|
## Cross-references
|
|
|
|
- **Agent dispatched:** `agents/fieldtester.md` — carries the
|
|
five-phase process, the Iron Law in operational form, the
|
|
spec template, the per-finding classification rules, the
|
|
Common Rationalisations table, and the Red Flags list.
|
|
- **Upstream gate:** `../audit/SKILL.md` runs first;
|
|
fieldtest comes after a clean (or ratified) audit.
|
|
- **Closing gate it feeds:** `../docs/pipeline.md`
|
|
§ Milestone-close gate — the milestone fieldtest's green
|
|
roll-up is the functional leg of that gate.
|
|
- **Cadence ordering:** fieldtest runs *before*
|
|
`../docwriter/SKILL.md`; docwriter happens at a later,
|
|
longer stability window.
|
|
- **Downstream targets:**
|
|
- `../debug/SKILL.md` — for each `bug` finding
|
|
- `../brainstorm/SKILL.md` or `../planner/SKILL.md` — for
|
|
`friction` and `spec_gap` findings worth a follow-up
|
|
iteration
|