fieldtest: loop-recur — 10 examples, 6 findings (milestone CLOSE, clean on all axes)
Post-audit downstream-LLM-author field test of the shipped loop/recur surface (DESIGN.md + public examples only). 3 real iterative programs (Newton isqrt, Collatz, Euclidean gcd) + 5 plausible-mistake negatives + 2 no-termination probes, all run through the public ail CLI. 0 bugs. 4 working findings on the milestone's own axes: rejection diagnostics point-exact AND self-fixing; recur tail-position threads through match/let/outer-if (spec only showed if); loop composes as a value sub-expression + byte-stable round-trip; no-termination boundary exact. This empirically substantiates the "LLM author can now write iterative programs" claim. Two orthogonal non-blocking findings, neither in loop/recur scope, both routed to P2 todos (refused the scope creep into a loop/recur tidy): niladic (app f) spec_gap independently re-confirms the existing mut-local-F3 roadmap item (the design-fork decision deliberately NOT auto-ratified under /boss — parked, priority-strengthened); module-level (doc) diagnostic-hint friction (one-line tidy). Boss-verified independently (gcd->27; recur-outside-loop fires exact). The standalone loop/recur milestone is fully ratified and CLOSED: 3 iterations + tidy shipped, audit clean (drift resolved, bench pristine carry-on), fieldtest clean on every axis. Roadmap P0 marked closed.
This commit is contained in:
@@ -0,0 +1,233 @@
|
||||
# Fieldtest — loop-recur — 2026-05-18
|
||||
|
||||
**Status:** Draft — awaiting orchestrator triage
|
||||
**Author:** ailang-fieldtester (dispatched by skills/fieldtest)
|
||||
|
||||
## Scope
|
||||
|
||||
The `loop-recur` milestone shipped a standalone strict-iteration
|
||||
surface: the Form-A `(loop (NAME TYPE INIT)* BODY)` block and
|
||||
`(recur ARG*)` re-entry. `loop` is an expression whose value is the
|
||||
body's value on the iteration that exits via a non-`recur` branch;
|
||||
`recur` re-enters the lexically-innermost enclosing `loop`, rebinding
|
||||
its binders positionally. The single closed rule is that `recur` must
|
||||
be in tail position of its enclosing loop body. No totality is
|
||||
claimed — an infinite loop is legal and compiles. Four rejection
|
||||
diagnostics (`recur-outside-loop`, `recur-arity-mismatch`,
|
||||
`recur-type-mismatch`, `recur-not-in-tail-position`) plus the
|
||||
symmetric `loop-binder-captured-by-lambda` enforce the contract. The
|
||||
surface is fully shipped (parse + typecheck + native codegen + run +
|
||||
Roundtrip Invariant).
|
||||
|
||||
## Examples
|
||||
|
||||
### `examples/fieldtest/loop_recur_1_isqrt_newton.ail` — integer sqrt via Newton's method
|
||||
- Computes `floor(sqrt(152399025))` = 12345, then prints `r` and
|
||||
`r - 12345`. Two binders (`x` current guess, `prev` previous
|
||||
guess); the `recur` is buried two levels deep: inside an `if`,
|
||||
inside a `let next ...`, inside another `if`. The whole `(loop …)`
|
||||
is itself wrapped in an outer `if (< n 2)` guard, and its result
|
||||
feeds a `let r … (seq (print r) (print (- r 12345)))`.
|
||||
- Fits scope: axis 1 (multi-binder, non-trivial body, `recur` in a
|
||||
branch tail not at body toplevel) **and** axis 4 (`loop` result
|
||||
consumed by an enclosing `let`/`seq`, not as a whole fn body).
|
||||
- Outcome: `ail check` ok, `ail build` ok, runs, stdout `12345\n0` —
|
||||
matches expected on first try.
|
||||
|
||||
### `examples/fieldtest/loop_recur_2_collatz.ail` — Collatz step counter
|
||||
- `collatz_len(27)=111`, `collatz_len(97)=118`, `collatz_len(1)=0`.
|
||||
Two binders (`n` current value, `steps` accumulator). Parity
|
||||
dispatch is a `match (% n 2)` with `pat-lit 0` / `_` arms; the
|
||||
`recur` lives in the **tail of each `match` arm**, and the `match`
|
||||
is itself the tail of an `if (== n 1)`.
|
||||
- Fits scope: axis 1 — exercises `recur` tail-position through a
|
||||
`match` (the spec only spells out `if`; this confirms `match`-arm
|
||||
tails are handled identically).
|
||||
- Outcome: `ail check` ok, `ail build` ok, runs, stdout
|
||||
`111\n118\n0` — matches expected on first try.
|
||||
|
||||
### `examples/fieldtest/loop_recur_4_gcd_value_pos.ail` — Euclidean gcd as a value sub-expression
|
||||
- `gcd(48,18)=6`, `gcd(1071,462)=21`; prints their sum `27`. Two
|
||||
binders (`a`, `b`); `recur` in the tail of the else branch. The
|
||||
two `(loop …)`-bearing `gcd` calls are nested inside
|
||||
`(app print (app + (app gcd …) (app gcd …)))` — `loop` results
|
||||
flow as function arguments.
|
||||
- Fits scope: axis 4 — `loop` as a value sub-expression in
|
||||
argument position, two independent loop instantiations composed
|
||||
arithmetically.
|
||||
- Outcome: `ail check` ok, `ail build` ok, runs, stdout `27` —
|
||||
matches expected on first try.
|
||||
|
||||
### `examples/fieldtest/loop_recur_3a..3e` — the five rejection diagnostics
|
||||
Each is the *plausible wrong program an LLM author would actually
|
||||
write*, not a contrived minimal repro:
|
||||
- `3a_recur_outside_loop` — a self-recursive `countdown` helper that
|
||||
reaches for `recur` as a generic self-tail-call with no enclosing
|
||||
`loop`. → `[recur-outside-loop] countdown: …`
|
||||
- `3b_recur_arity` — a two-binder `(acc, i)` loop where the author
|
||||
writes `(recur (+ acc i))` and forgets to advance `i`. →
|
||||
`[recur-arity-mismatch] sum_to: passes 1 argument(s) but … 2
|
||||
binder(s)`
|
||||
- `3c_recur_type` — the author passes the `Bool` comparison result
|
||||
into the first `Int` binder slot. → `[recur-type-mismatch]
|
||||
count_down: argument 0 has type \`Bool\` but … is \`Int\``
|
||||
- `3d_recur_not_tail` — factorial written as `n * recur(n-1)`,
|
||||
treating `recur` as value-returning. → `[recur-not-in-tail-
|
||||
position] factorial: it is a back-jump, not a value-producing
|
||||
sub-expression`
|
||||
- `3e_binder_captured` — inside the loop body the author builds a
|
||||
`\d. acc + d` adder closing over loop binder `acc` and applies it.
|
||||
→ `[loop-binder-captured-by-lambda] sum_to: … Either move the
|
||||
lambda outside the loop, or restructure to pass \`acc\` as a lambda
|
||||
parameter …`
|
||||
- Fits scope: axes 2 and 3. Outcome: all five `ail check` exit 1
|
||||
with **exactly** the promised code; the function is named and the
|
||||
fix is described in the message text.
|
||||
|
||||
### `examples/fieldtest/loop_recur_5b_event_loop_noterm.ail` — non-terminating event loop (loop-axis-covering)
|
||||
- A two-binder tick/parity event loop with **no non-`recur` branch**.
|
||||
`ail check` ok, `ail build` ok. **Not executed** (infinite by
|
||||
design).
|
||||
- Fits scope: axis 5 — the no-termination boundary. Confirms an
|
||||
infinite `loop` typechecks and compiles cleanly with zero
|
||||
diagnostics, exactly as the spec promises.
|
||||
|
||||
### `examples/fieldtest/loop_recur_5_event_loop_noterm.ail` — friction artefact (kept, not worked around)
|
||||
- Same algorithm written the maximally-natural way for an infinite
|
||||
loop: a **niladic** `run_forever : fn() -> Int`, invoked as
|
||||
`(app run_forever)`. This is the LLM-natural shape — an event
|
||||
loop has no seed to thread — but it never reaches loop/recur
|
||||
semantics: it fails at parse with `surface-parse-error: parse
|
||||
error in app-term: expected at least one argument`. Retained
|
||||
verbatim as the finding artefact; `5b` is the variant that adds
|
||||
an honest `start` parameter so the no-termination *loop* axis is
|
||||
still genuinely exercised.
|
||||
|
||||
## Findings
|
||||
|
||||
### [working] All five rejection diagnostics are point-exact and self-fixing
|
||||
- Examples: `loop_recur_3a` … `loop_recur_3e`.
|
||||
- What happened: each plausible-mistake program produced exactly the
|
||||
promised code with no false positives and no crash. The wording
|
||||
goes beyond naming the rule — `3b` says "recur must rebind every
|
||||
binder positionally", `3d` says "it is a back-jump, not a
|
||||
value-producing sub-expression", `3e` spells out two concrete
|
||||
remediations. Every message names the enclosing function.
|
||||
- Why working: this is the milestone's clause-2 correctness claim
|
||||
(an entire silent-failure class becomes a compile error) made
|
||||
real. An author who makes the natural mistake is told what is
|
||||
wrong, where, and how to fix it, without reading DESIGN.md.
|
||||
- Recommended downstream action: carry-on. These diagnostics are a
|
||||
feature-defining strength; protect them against future
|
||||
rewording drift (a snapshot test on the message bodies, if not
|
||||
already present, would be cheap insurance — but that is an
|
||||
internal call, not a fieldtest demand).
|
||||
|
||||
### [working] `recur` tail-position threads correctly through `match`, nested `let`, and outer `if`
|
||||
- Examples: `loop_recur_1_isqrt_newton` (recur inside `if`/`let`/`if`,
|
||||
loop inside an outer `if`), `loop_recur_2_collatz` (recur in
|
||||
`match`-arm tails).
|
||||
- What happened: DESIGN.md and the §"Concrete code shapes" only
|
||||
demonstrate `recur` in `if` branches. I reached for `match`-arm
|
||||
tails and a `let`-then-`if` nest unprompted (both are the natural
|
||||
shapes for parity dispatch and Newton's fixpoint test). All
|
||||
compiled and ran correctly on the first try. The tail-position
|
||||
analysis correctly treats a `match` arm body and a `let` body as
|
||||
tail when the enclosing form is tail.
|
||||
- Why working: confirms the closed rule generalises exactly as an
|
||||
author would assume from the `if` examples — no surprise, no
|
||||
special-casing needed in author-side mental model.
|
||||
- Recommended downstream action: carry-on.
|
||||
|
||||
### [working] `loop` composes as a value sub-expression and round-trips
|
||||
- Examples: `loop_recur_1` (loop result into `let`/`seq`),
|
||||
`loop_recur_4` (two loop results into `(+ … …)` into `print`).
|
||||
- What happened: `loop` used purely as an expression — fed to a
|
||||
binding, summed with another `loop`, passed as a call argument —
|
||||
worked with no ceremony. Separately, `ail parse | ail render |
|
||||
ail parse` is canonical-JSON byte-identical for every
|
||||
loop/recur-bearing fixture (1, 2, 4, 5b), so the spec's Roundtrip
|
||||
Invariant acceptance criterion holds in practice.
|
||||
- Why working: the spec's "loop is an expression" claim is not just
|
||||
true for the whole-fn-body case the reference corpus shows; it
|
||||
holds in arbitrary expression position.
|
||||
- Recommended downstream action: carry-on.
|
||||
|
||||
### [working] No-termination boundary is exactly as specified
|
||||
- Example: `loop_recur_5b_event_loop_noterm`.
|
||||
- What happened: an infinite `loop` (every branch a `recur`)
|
||||
`ail check`s and `ail build`s with zero diagnostics. No spurious
|
||||
"unreachable", no "loop never exits", no totality complaint.
|
||||
- Why working: this is the precise difference from the reverted
|
||||
Iteration-discipline milestone, and it behaves as the spec's
|
||||
deliberate-boundary section promises.
|
||||
- Recommended downstream action: carry-on.
|
||||
|
||||
### [spec_gap] A niladic application `(app f)` is unrepresentable in Form A; DESIGN.md does not constrain it
|
||||
- Example: `loop_recur_5_event_loop_noterm` (kept as the artefact).
|
||||
- What happened: the maximally-natural infinite-loop shape is a
|
||||
zero-argument `run_forever : fn() -> Int` called as
|
||||
`(app run_forever)`. The surface parser rejects this with
|
||||
`error: [surface-parse-error] parse error in app-term: expected
|
||||
at least one argument`. DESIGN.md's `Term::App` schema is
|
||||
`{ "t":"app", "fn":Term, "args":[Term...] }` — `[Term...]` does
|
||||
not state a minimum, and the entire public `examples/*.ail`
|
||||
corpus contains **no** AIL-level call of a niladic function
|
||||
(`main` is the only zero-arg fn and is never called from source;
|
||||
`loop_forever_build.ail` deliberately gave `spin` an `Int`
|
||||
parameter to sidestep exactly this). So two readings are equally
|
||||
plausible from the spec alone: (a) `args:[]` is legal AST that
|
||||
the surface should be able to express, or (b) niladic application
|
||||
is forbidden by design and DESIGN.md should say so. The compiler
|
||||
picked (b) at the surface layer; I could not tell which reading
|
||||
is intended without reading `crates/`.
|
||||
- Why spec_gap: DESIGN.md neither permits nor forbids the empty-args
|
||||
application; the surface diagnostic ("expected at least one
|
||||
argument") states a rule that appears nowhere in DESIGN.md, and a
|
||||
downstream author writing an event loop / `unit -> a` thunk hits
|
||||
it with no spec guidance. It is orthogonal to loop/recur but was
|
||||
surfaced *by* the loop/recur no-termination axis, because an
|
||||
infinite loop is the canonical case with no state worth a
|
||||
parameter.
|
||||
- Recommended downstream action: ratify a decision and tighten
|
||||
DESIGN.md — either (a) state that niladic applications are
|
||||
forbidden in Form A and document the `unit`-param convention as
|
||||
the workaround, or (b) extend the surface to accept `(app f)`
|
||||
for `fn() -> a`. Until then, an author cannot express a
|
||||
niladic-call thunk and the diagnostic gives no DESIGN-backed
|
||||
remedy.
|
||||
|
||||
### [friction] Module-level `(doc …)` is rejected with a diagnostic that omits the actual location of `doc`
|
||||
- Examples: surfaced on the first draft of `loop_recur_3a..3e`
|
||||
(corrected in the committed fixtures to a leading `;` comment).
|
||||
- What happened: I wrote a module-level
|
||||
`(module NAME (doc "…") (fn …))` — a natural reach, since `fn`
|
||||
and `data` both accept a `doc` child, so an author reasonably
|
||||
assumes the module node does too. The diagnostic is
|
||||
`error: [surface-parse-error] parse error in module: unknown def
|
||||
head \`doc\`; expected \`data\`, \`fn\`, \`const\`, \`class\`,
|
||||
\`instance\`, or \`import\``. It correctly rejects, but the
|
||||
message lists the valid def heads **without** mentioning that
|
||||
`doc` is valid *inside* `fn`/`data` — an author can read this
|
||||
message and still not know where the doc string should go, and
|
||||
may conclude doc strings are unsupported entirely.
|
||||
- Why friction: it compiles-or-rejects correctly (no bug), but the
|
||||
verbose-by-design AILang diagnostic philosophy is not met here:
|
||||
the message names what is expected but not the one thing the
|
||||
author needs (move `doc` inside the def). Orthogonal to
|
||||
loop/recur; surfaced while writing the negative fixtures.
|
||||
- Recommended downstream action: plan a one-line tidy — append to
|
||||
the message a hint such as "(doc strings attach inside `fn`/`data`
|
||||
defs, not at module level)". Low cost, removes a dead-end for the
|
||||
author.
|
||||
|
||||
## Recommendation summary
|
||||
|
||||
| Finding | Class | Action |
|
||||
|---|---|---|
|
||||
| Five rejection diagnostics point-exact & self-fixing | working | carry-on |
|
||||
| `recur` tail-position threads through `match`/`let`/outer-`if` | working | carry-on |
|
||||
| `loop` composes as a value sub-expression & round-trips | working | carry-on |
|
||||
| No-termination boundary exactly as specified | working | carry-on |
|
||||
| Niladic `(app f)` unrepresentable; DESIGN.md silent | spec_gap | ratify + tighten DESIGN.md |
|
||||
| Module-level `(doc …)` diagnostic omits where `doc` belongs | friction | plan (one-line tidy) |
|
||||
Reference in New Issue
Block a user