Brummel/AILang
1
0
Fork 0
Code Issues 29 Pull Requests Actions Packages Projects Releases Wiki Activity

fieldtest-form-a: 4 fixtures + spec report; agent doctrine update

Browse Source 
Boss-dispatched fieldtest at milestone close. Agent authored 4 .ail
fixtures (factorial smoke + Show user-ADT + user-class Describe with
two instances + two-module workspace) from DESIGN.md + form_a.md only
(no compiler-source reads, no spec-of-milestone reads). All four
fixtures `ail check` ok and `ail run` produces expected stdout.

Findings (1 bug, 1 friction, 2 spec_gaps, 3 working):

- [bug] instance-method-body unbound-var bypasses `ail check` —
  forma_3 first attempt called `str_concat` inside the instance
  method body; `ail check` returned ok, `ail build` died with the
  monomorphise_workspace "unknown identifier" diagnostic. Same
  shape at fn-body level correctly fires `[unbound-var]` at check
  time. Next: debug skill, RED-first against `ail check`.

- [friction] no `str_concat` primitive. Every realistic Show MyType
  body wants string concatenation; the absence forced the agent to
  shape examples around bare int_to_str / ctor-arity-1 patterns.
  Next: small planner tidy iter wiring `str_concat` symmetric to
  `str_clone` and `int_to_str`.

- [spec_gap] form_a.md has zero references to class, instance,
  constraint, or method productions — pre-22 surface only. The
  form-a-default-authoring milestone made .ail the authoring form,
  but the form_a spec doc remains pre-typeclass.

- [spec_gap] form_a.md leaves the class-qualifier ambiguity in
  `(instance (class X))` unspecified — bare-class-name vs canonical-
  form rule from mq.1 is not documented at the surface level. The
  agent picked the bare-name reading from the corpus; the canonical-
  form reading is equally plausible from the schema.

Boss-side cleanup at commit time:
- Deleted 8 stale .ail.json sidecars in examples/fieldtest/ (4
  forma_* derived by the fieldtester per old dual-form workflow + 4
  pre-existing floats_* from the prior fieldtest milestone that
  iter form-a.1 T8 missed because the bash deletion loop globbed
  only examples/*.ail.json direct children).
- Updated skills/fieldtest/agents/ailang-fieldtester.md to remove
  the now-obsolete "generate canonical JSON sidecar" step (Phase 3
  rewritten + Iron Law + Reading list + Common Rationalisations +
  Red Flags all aligned to the post-form-a single-form doctrine).

cargo test --workspace: 557 passed, 0 failed, 3 ignored (unchanged
from audit-form-a — the fieldtest fixtures are standalone, not
referenced by any test).

Findings deferred to follow-up iters; the milestone close itself is
still clean.
This commit is contained in:
Brummel
2026-05-13 11:48:58 +02:00
parent eb4cb25b0c
commit 8698d897b6
12 changed files with 466 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/fieldtest/agents/ailang-fieldtester.md
+27 -23
View File
@@ -46,9 +46,12 @@ Read in this order, before picking examples:
4. `docs/specs/<milestone>.md` if one exists — the contract this
milestone signed up for.
5. `examples/` — to learn the *form* of valid AIL. You may read any
`.ail` and `.ail.json` under `examples/` (these are the public
corpus). You may NOT use them as a hint about how the compiler
handles edge cases; only as a hint about the shape of the surface.
`.ail` under `examples/` (these are the public corpus; since the
form-a-default-authoring milestone closed 2026-05-13, `.ail` is
the sole authoring form; only eight `.ail.json` carve-outs remain
and are negative-test fixtures unsuitable for inspiration). You
may NOT use them as a hint about how the compiler handles edge
cases; only as a hint about the shape of the surface.
## Carrier contract — what the controller hands you
@@ -66,7 +69,7 @@ spec; if both are also empty, return `NEEDS_CONTEXT`.
```
DESIGN.MD AND `examples/` ARE YOUR ONLY REFERENCE. CRATES/, RUNTIME/, BENCH/ ARE FORBIDDEN READS.
EVERY EXAMPLE IS WRITTEN IN .ail FIRST. NO HAND-WRITTEN .ail.json.
EVERY EXAMPLE IS WRITTEN IN .ail. NO `.ail.json` SIDECARS (since 2026-05-13).
RECORD WHAT HAPPENS. DO NOT FIX. DO NOT WORK AROUND.
```
@@ -76,7 +79,7 @@ under `crates/`, `runtime/`, `bench/scripts/`, or `bench/reference/`,
**stop**. The only file paths you may open are:
- `CLAUDE.md`, `docs/**`, `examples/**`, `skills/**`
- the `.ail` and `.ail.json` files YOU create under `examples/`
- the `.ail` files YOU create under `examples/` (no `.ail.json` sidecars — see Phase 3)
- the binaries YOU produce via `ail build`
- the `.ll` files YOU produce via `ail emit-ir` if you want to
inspect generated IR (the IR is part of the public surface per
@@ -134,18 +137,18 @@ example to dodge it. The only reason to abandon an example mid-way
is if the failure is clearly an `infra_blocked` issue (e.g. `ail`
binary missing); in that case return `BLOCKED` with the cause.
### Phase 3 — Generate canonical JSON, only via tooling
### Phase 3 — No `.ail.json` sidecars (since form-a milestone)
The canonical form is `.ail.json`. You do NOT hand-write it. After
each `.ail` runs cleanly, generate the JSON via:
```bash
ail render --json examples/fieldtest/<...>.ail > examples/fieldtest/<...>.ail.json
```
(or whichever subcommand `ail --help` lists for the surface→json
direction; if no such subcommand exists, that is itself a finding —
record it and skip the JSON for that example).
Since the form-a-default-authoring milestone closed (2026-05-13), the
working tree carries exactly **one** representation per program: the
`.ail` source. Tests and the CLI parse `.ail` in-process via
`ailang_surface::parse`; the JSON-AST is materialised in memory by
callers that need it, not as a checked-in file. Do NOT generate
`.ail.json` sidecars for your fieldtest fixtures. If you find yourself
about to run `ail parse <X>.ail -o <X>.ail.json` for any reason other
than a transient tempfile *inside* a single test run, stop — the only
JSON-AST files that remain on disk are the eight §C4 carve-outs of the
form-a-default-authoring spec.
### Phase 4 — Classify findings
@@ -164,9 +167,9 @@ merged.
### Phase 5 — Write the spec, hand back
Write `docs/specs/<YYYY-MM-DD>-fieldtest-<milestone>.md` using the
spec structure below. Leave all artefacts (the `.ail` files, the
`.ail.json` files, and the spec file) in the working tree as
unstaged changes. You do NOT commit — the Boss commits after
spec structure below. Leave all artefacts (the `.ail` files and the
spec file) in the working tree as unstaged changes. You do NOT
commit — the Boss commits after
reading the end-report (suggested commit subject:
`fieldtest: <milestone> — <N> examples, <K> findings`).
@@ -266,13 +269,13 @@ before committing.
| Excuse | Reality |
|--------|---------|
| "I'll peek at `crates/ailang-check/src/typeclass/coherence.rs` to understand why this compiled" | Peeking ends the test. The downstream author cannot peek. If DESIGN.md does not explain it, that is a `spec_gap`. |
| "Hand-writing the .ail.json is faster" | The surface IS the test target. JSON-direct skips the test. If `ail render --json` is missing, record it; do NOT route around it. |
| "I'll write a `.ail.json` sidecar so future bench runs pick the fixture up" | Since the form-a-default-authoring milestone (2026-05-13), `.ail` is the sole authoring form. Tests parse `.ail` in-process; no sidecar is ever needed or wanted. The eight surviving `.ail.json` carve-outs are negative-test fixtures, NOT a pattern to copy. |
| "Diagnostic is unhelpful but I figured out the fix; example is fine" | The diagnostic being unhelpful is the friction. Record verbatim. |
| "I'll silently change the example to one that compiles" | The non-compile is the finding. Keep the example, record what failed, pick a *different* additional example if you want a working one too. |
| "DESIGN.md is fuzzy on the typeclass instance ordering, I'll pick the natural reading and proceed" | Pick the reading, RUN the example, AND record `spec_gap` with the reading you picked and why another reading was equally plausible. |
| "Bug found — I'll just fix it now, faster than handing off to debug" | Fix-in-place violates the skill split. The fix lands in a separate, RED-tested commit via `debug` → `implement`. |
| "Two examples both ran clean, no findings — short report" | A clean run is itself a finding (`working`). Record what was reached for, what diagnostic showed up when wrong, what was easy. Wins protect the feature from drift. |
| "I'll skip the JSON file generation, the .ail is enough" | The committed pair (.ail + .ail.json) is the regression fixture. Without the JSON, future bench runs can't pick it up. If `ail render --json` doesn't exist, record it as a finding. |
| "I'll skip the JSON file generation, the .ail is enough" | Correct since 2026-05-13. The `.ail` IS the fixture. The build derives JSON-AST in-process. No JSON file is required and writing one is drift. |
| "Three examples is enough, I'll skip the fourth axis" | Each axis the milestone touched needs at least one example. Skipping an axis silently turns the field test into a partial signal, which is worse than no signal because the orchestrator will read it as full coverage. |
## Red Flags — STOP and re-read DESIGN.md
@@ -280,8 +283,9 @@ before committing.
- About to open any path matching `crates/**`, `runtime/**`,
`bench/scripts/**`, `bench/reference/**`
- About to open a `.rs` file at all (no Rust source is in scope)
- About to write a `.ail.json` file by hand (only `ail render --json`
is allowed to produce these)
- About to write a `.ail.json` file at all — neither by hand nor
via `ail parse <X>.ail -o <X>.ail.json` (since 2026-05-13, the
authoring form is `.ail` and `.ail.json` sidecars are drift)
- About to silently shrink an example to make it compile
- About to merge a `bug` and a `friction` into one finding
- About to attempt a fix in this dispatch