Files
Skills/fieldtest/SKILL.md
T
Brummel 808044c0cf fieldtest: skill + fieldtester agent migrated
Fifth skill via the boss pattern. fieldtest empirically tests
whether a shipped cycle is usable by a downstream consumer
who has only the public interface.

Profile gains two slots:
- `paths.public_interface` — list of dirs/files the fieldtester
  may read (the project's outward-facing surface: README, design
  ledger, docs, examples)
- `paths.fieldtest_examples` — where the fieldtester writes its
  fixtures

The forbidden zone is derived: everything not in
`paths.public_interface` (especially `paths.code_roots` and
`paths.bench_dir`) is closed to the fieldtester. This is the
load-bearing source-isolation discipline.

Vocabulary substitutions:
- AILang → "this project"
- milestone → cycle
- Boss → orchestrator
- ailang-fieldtester → fieldtester
- "AIL Surface form / .ail" → "the project's canonical
  authoring or consumer form"
- "design/INDEX.md + design/models/" → `paths.public_interface`
- "crates/, runtime/, bench/" → `paths.code_roots`,
  `paths.bench_dir`

AILang-specific bits dropped:
- The form-a-default-authoring milestone history (2026-05-13)
  and the eight `.ail.json` carve-outs — AILang-specific
  schema-migration detail; the universal rule is just
  "canonical authoring form only"
- The ail check / ail build / ail emit-ir CLI commands — the
  fieldtester now uses "the project's user-facing entry point"
- Per-iter journals reference — replaced with recent iter
  commit bodies (universal)
- Example ideas calibrated to AILang (FizzBuzz, Newton's method
  on Float, JSON-tree depth, ADT functors) — generalised to
  "realistic tasks an end-user might be asked to do"
- `design/contracts/0004-feature-acceptance.md` cross-ref —
  dropped (AILang-specific contract); the empirical-vs-
  prospective brainstorm symmetry survives in prose

Universal substance preserved verbatim:
- All three Iron Law clauses (public-interface-only,
  canonical-form-only, record-don't-fix)
- Five-phase process structure
- Four-class finding taxonomy (bug / friction / spec_gap /
  working)
- Spec template
- Status protocol (DONE / DONE_WITH_CONCERNS / NEEDS_CONTEXT /
  BLOCKED)
- All 8 Common Rationalisations rows (vocabulary-substituted)
- All 10 Red Flags bullets (vocabulary-substituted, with the
  AILang-specific .ail.json hand-write bullet generalised to
  "intermediate representations")

The source-isolation contract (Iron Law clause 1) is the
load-bearing discipline. Profile slot
`paths.public_interface` makes it machine-checkable; the
agent's hard limit on opening paths outside that list is the
discipline.
2026-05-28 16:02:58 +02:00

150 lines
6.0 KiB
Markdown

---
name: fieldtest
description: Orchestrator-dispatched only, after audit closes clean (or with ratified drift only), when the orchestrator judges the iteration is complete and wants a field test. Picks 2-4 real-world tasks within the cycle's scope, implements each as a downstream consumer of the project, runs them, and writes a friction-and-bug spec. The spec feeds the next plan as a reference. Implementer simulates a downstream user who has only the public interface — never the project's implementation source.
---
# fieldtest — usability field test for a shipped cycle
> **Violating the letter of these rules is violating the spirit.**
## Overview
`audit` measures drift between the design ledger and what the
codebase claims to do. `fieldtest` measures something audit
cannot: whether the project, *as it now ships*, is usable by
a downstream consumer who has only the public interface to
work from. If a freshly shipped feature is awkward to reach
for, or if the public docs leave a corner ambiguous that real
usage immediately exercises, the cycle has shipped a latent
debt — even when audit reports `clean`.
The skill produces a friction-and-bug spec that the next
iteration's `planner` consumes as a reference. The spec sits
next to cycle-design specs under `paths.spec_dir`, with a
`fieldtest-` prefix in the slug.
The substantive process — read the design ledger + cycle spec
+ recent iter/audit commit bodies, pick 2-4 real-world tasks
per cycle axis, implement each as a downstream consumer, run
the result, classify findings, write the spec — lives in
`agents/fieldtester.md`. That file also carries the spec
template, the source-isolation discipline (no reading under
`paths.code_roots` or `paths.bench_dir`), and the per-finding
classification rules. This skill file only governs trigger,
dispatch, and handoff.
## When to Use / Skipping
Triggers:
- Orchestrator-dispatched after audit closes cleanly, when
the orchestrator believes the iteration is correct and
wants a field test. The pre-condition is a clean (or
ratified) audit; the trigger is the orchestrator's
judgment-call that the iteration is complete.
- A shipped feature is suspected of being awkward in some
way the earlier brainstorm could not foresee.
- A user-facing surface change has just landed.
Findings cluster around two shapes — simple bugs (route via
`debug``implement` mini) or catastrophic architecture
problems (route via next `brainstorm`). Friction and spec-gap
items are also possible; the routing table below applies in
all cases.
**Skipping is permitted only** for:
- Bug-fix iterations (use `debug`).
- Tidy iterations whose scope is purely internal cleanup with
zero surface impact.
- Iterations whose entire scope is documentation.
**Skipping is not permitted** for:
- A cycle that introduced or changed surface syntax, schema,
effects, API, configuration format, or any user-visible
diagnostic.
- A cycle whose commit body or backlog issue claims "users
can now do X". `fieldtest` is the gate that empirically
substantiates such claims; an unverified claim is drift.
## The Iron Law
```
THE FIELDTESTER WORKS FROM THE PUBLIC INTERFACE ONLY — NOT FROM THE IMPLEMENTATION SOURCE.
EVERY EXAMPLE IS WRITTEN IN THE PROJECT'S CANONICAL AUTHORING / CONSUMER FORM.
EVERY FRICTION POINT AND BUG IS RECORDED. NONE IS WORKED AROUND.
```
The first clause is load-bearing: the whole point of the
field test is to simulate a downstream consumer who has only
the public interface. The agent file enforces this with a
hard path allowlist (computed from `paths.public_interface`
in the profile); the orchestrator must trust that contract
and not feed the agent implementation-internal hints in the
carrier.
## Dispatch
Dispatch `fieldtester` with the carrier from the Handoff
Contract below. The agent picks 2-4 examples (one per axis
the cycle touched), implements them as downstream consumers,
runs them, classifies findings, and writes the spec. All
artefacts (fixtures + spec) stay in the working tree as
unstaged changes; the orchestrator commits them after
reviewing the report (suggested commit subject:
`fieldtest: <cycle> — <N> examples, <K> findings`).
Two to four examples is the right size. One is too few (no
signal on variation); five is too many for one report to
stay readable.
## Handoff Contract
`fieldtest` consumes (from orchestrator at cycle close):
| Field | Content |
|-------|---------|
| `cycle_id` | e.g. `cycle-22` |
| `cycle_scope` | 1-3 sentences naming what shipped |
| `axis_hints` | bullet list, one per cycle axis the test should probe |
| `commit_range` | `<prev-cycle-close>..HEAD` |
`fieldtest` produces:
| Field | Content |
|-------|---------|
| `spec_path` | spec file under `paths.spec_dir` with a `fieldtest-` prefix |
| `examples_added` | list of fixture paths committed |
| `findings` | list, each with class (`bug` / `friction` / `spec_gap` / `working`) + recommendation |
| `status` | `clean` / `friction_found` / `bugs_found` / `infra_blocked` |
The orchestrator drives downstream:
- `bug``debug` (RED-first; GREEN via `implement` mini-mode)
- `friction` → next `brainstorm` or a tidy iteration via
`planner`
- `spec_gap` → ratify by updating the design ledger (and
naming the gap in the closing commit body), or tighten the
design ledger
- `working` → carry-on (worth recording, no follow-up)
`fieldtest` does NOT self-resolve.
## Cross-references
- **Agent dispatched:** `agents/fieldtester.md` — carries the
five-phase process, the Iron Law in operational form, the
spec template, the per-finding classification rules, the
Common Rationalisations table, and the Red Flags list.
- **Upstream gate:** `../audit/SKILL.md` runs first;
fieldtest comes after a clean (or ratified) audit.
- **Cadence ordering:** fieldtest runs *before*
`../docwriter/SKILL.md`; docwriter happens at a later,
longer stability window.
- **Downstream targets:**
- `../debug/SKILL.md` — for each `bug` finding
- `../brainstorm/SKILL.md` or `../planner/SKILL.md` — for
`friction` and `spec_gap` findings worth a follow-up
iteration