--- name: fieldtest description: Use at milestone close after audit, OR when an LLM-usability concern about a shipped feature is suspected. Picks 2-4 real-world programming tasks within the milestone's scope, implements each in the AIL Surface form (.ailx — not raw JSON), runs the resulting binaries, and writes a friction-and-bug spec to docs/specs/-fieldtest-.md. The spec feeds the next plan as a reference. Implementer simulates a downstream LLM that has only DESIGN.md plus the public examples — never the language's own implementation. --- # fieldtest — LLM-usability field test for a shipped milestone > **Violating the letter of these rules is violating the spirit.** ## Overview `audit` measures drift between DESIGN.md and what the codebase claims to do. `fieldtest` measures something audit cannot: whether the language, *as it now ships*, is usable by an LLM author who has only DESIGN.md and public examples to work from. The shipping LLM-author *is* the user of AILang. If a freshly shipped feature is awkward for that author to reach for, or if DESIGN.md leaves a corner ambiguous that real code immediately exercises, the milestone has shipped a latent debt — even when audit reports `clean`. The skill produces a friction-and-bug spec that the next iteration's `planner` consumes as a reference. The spec sits next to milestone-design specs at `docs/specs/-fieldtest-.md`. The substantive process — read DESIGN.md + JOURNAL + milestone spec, pick 2-4 real-world programming tasks per milestone axis, implement each in `.ailx` Surface form, run via `ail check`/`build`/`run`, classify findings, write the spec — lives in `agents/ailang-fieldtester.md`. That file also carries the spec template, the source-isolation discipline (no reading under `crates/`, `runtime/`, `bench/`), and the per-finding classification rules. This skill file only governs trigger, dispatch, and handoff. ## When to Use / Skipping Triggers: - Boss-dispatched after audit closes cleanly, when the orchestrator believes the iteration is correct and wants a Praxistest. The pre-condition is a clean (or `ratify`-d) audit; the trigger is the Boss's judgment-call that the iteration is fertig. - A shipped feature is suspected of being LLM-hostile in some way the earlier brainstorm could not foresee. - A user-facing surface change (Decision 6, schema rev, new effect/typeclass mechanic) has just landed. Findings cluster around two shapes — simple bugs (route via `debug` → `implement` mini) or catastrophic architecture problems (route via next `brainstorm`). Friction and spec-gap items are also possible; the routing table below applies in all cases. **Skipping is permitted only** for: - Bug-fix iterations (use `debug`). - Tidy iterations whose scope is purely internal cleanup with zero surface impact. - Iterations whose entire scope is documentation in `docs/`. **Skipping is not permitted** for: - A milestone that introduced or changed surface syntax, schema, effects, types, modes, or any user-visible diagnostic. - A milestone whose JOURNAL entry claims "LLM author can now write X". `fieldtest` is the gate that empirically substantiates such claims; an unverified claim is drift. ## The Iron Law ``` THE FIELDTESTER WORKS FROM DESIGN.MD AND PUBLIC EXAMPLES — NOT FROM THE COMPILER SOURCE. EVERY EXAMPLE IS WRITTEN IN .ailx (SURFACE) FIRST. RAW .ail.json IS NEVER HAND-AUTHORED. EVERY FRICTION POINT AND BUG IS RECORDED. NONE IS WORKED AROUND. ``` The first clause is load-bearing: the whole point of the field test is to simulate a downstream LLM author who has only the specification and the example corpus. The agent file enforces this with a hard path allowlist; the orchestrator must trust that contract and not feed the agent compiler-internal hints in the carrier. ## Dispatch Dispatch `ailang-fieldtester` with the carrier from the Handoff Contract below. The agent picks 2-4 examples (one per axis the milestone touched), implements them in `.ailx`, runs them through the public `ail` CLI, classifies findings, writes the spec, and commits both fixtures and spec in one commit (`fieldtest: — examples, findings`). Two to four examples is the right size. One is too few (no signal on variation); five is too many for one report to stay readable. ## Handoff Contract `fieldtest` consumes (from orchestrator at milestone close): | Field | Content | |-------|---------| | `milestone_id` | e.g. `milestone-22` | | `milestone_scope` | 1-3 sentences naming what shipped | | `axis_hints` | bullet list, one per milestone axis the test should probe | | `commit_range` | `..HEAD` | `fieldtest` produces: | Field | Content | |-------|---------| | `spec_path` | `docs/specs/-fieldtest-.md` | | `examples_added` | list of `.ailx` paths committed | | `findings` | list, each with class (`bug` / `friction` / `spec_gap` / `working`) + recommendation | | `status` | `clean` / `friction_found` / `bugs_found` / `infra_blocked` | The orchestrator drives downstream: - `bug` → `debug` (RED-first; GREEN via `implement` mini-mode) - `friction` → next `brainstorm` or a tidy iteration via `planner` - `spec_gap` → ratify in JOURNAL + DESIGN.md, or tighten DESIGN.md - `working` → carry-on (worth recording, no follow-up) `fieldtest` does NOT self-resolve. ## Cross-references - **Agent dispatched:** `skills/fieldtest/agents/ailang-fieldtester.md` — carries the five-phase process, the Iron Law in operational form, the spec template, the per-finding classification rules, the Common Rationalisations table, and the Red Flags list. - **Upstream gate:** `skills/audit/SKILL.md` runs first; fieldtest comes after a clean (or ratified) audit. - **Cadence ordering:** fieldtest runs *before* `skills/docwriter`; docwriter happens at a later, longer stability window. - **Downstream targets:** - `skills/debug/SKILL.md` — for each `bug` finding - `skills/brainstorm/SKILL.md` or `skills/planner/SKILL.md` — for `friction` and `spec_gap` findings worth a follow-up iteration - **Project source:** `docs/DESIGN.md` "Feature-acceptance criterion" — the field test is the empirical complement to brainstorm's prospective application of that criterion. Brainstorm asks "would an LLM author reach for this?"; fieldtest asks "did one?"