Iter 14e: explicit, verified tail calls

Decision 8 ships. Term::App and Term::Do gain tail: bool with
serde-default false and skip-when-false serialisation. New
typecheck pass verify_tail_positions enforces tail-position rules
(Scheme-style propagation through match arms, seq.rhs, let body,
lam body). Codegen emits musttail call for marked App calls.

Hash invariance verified: only the two migrated print_list defs
(list_map_poly.print_list, sort.print_list) changed hashes; all
other defs across all 18 fixtures kept bit-identical hashes —
confirms the skip-when-false serialisation rule works.

Tests 76 -> 79: tail_call_in_non_tail_position_is_rejected,
tail_call_in_tail_position_is_accepted, plus an IR-grep e2e test
asserting that print_list's recursive call site emits musttail
in the lowered IR. Existing 25 e2e tests unchanged in behaviour
(map -> [2,3,4], sort -> sorted list).

IR evidence at the recursive site:
  %v7 = musttail call i8 @ail_list_map_poly_print_list(ptr %v6)
  ret i8 %v7

Two deviations called out in the implementer report and JOURNAL:
1. tail-do uses tail call, not musttail. Cross-type return
   (runtime helpers return i32, AILang Unit is i8) would have
   LLVM reject musttail. Path is implemented but not exercised
   by any current fixture; proper fix is runtime-helper signature
   change, punted.
2. block_terminated flag in codegen so tail-call emit
   (musttail call + ret) doesn't get a duplicate trailing ret
   from surrounding code (match-arm phi, fn-body, lambda thunk).
   Internal plumbing; required for IR well-formedness.

Form (A) productions now at ~30, exactly the constraint-1
budget. Future surface additions need to retire something or
explicit-budget-rebalance in DESIGN.md.

GC notes from implementer survey land in JOURNAL:
- Allocations cluster in lower_ctor; every term-ctor does
  malloc(8+8n).
- Tail recursion does not reduce alloc pressure, only stack.
  For map-style ctor-blocked recursions, allocation IS the
  bottleneck.
- Per-fn arena is sound only when fn return type contains no
  boxed ADT. Most current fixtures violate this.

Plan 14f: Boehm conservative GC (GC_malloc, -lgc) as a first
cut. Single-iter integration, no AST/schema change. Stress
test: build a 100k Cons list, observe RSS doesn't blow up.

After 14f the language is feature-complete enough for stdlib
work (15a).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 17:16:28 +02:00
parent 8d97a924de
commit d64031c234
16 changed files with 768 additions and 81 deletions
+72
View File
@@ -429,6 +429,21 @@ fixtures round-trip identically through `print → parse → canonical
JSON`; the three hand-written `.ailx` exhibits parse to canonical
JSON identical to their corresponding `.ail.json` files.
3. **Tail-call surface (Iter 14e).** Decision 8 ships two new
productions, both positional analogues of their non-tail
counterparts. Their result terms set `Term::App.tail = true` /
`Term::Do.tail = true`; the typechecker's `verify_tail_positions`
pass enforces that the marker is only used in tail position.
```
tail-app-term ::= "(" "tail-app" term term+ ")"
tail-do-term ::= "(" "tail-do" ident term* ")"
```
Production count after Iter 14e: ~30, still inside the 30-rule
constraint-1 budget. No new lexical rule (`tail-app` / `tail-do`
are bare ident tokens; no special casing).
## Decision 7: redundancy removal — `Term::If` is not a primitive
`Term::If { cond, then, else_ }` is semantically a subset of
@@ -454,6 +469,63 @@ No schema version bump (no third-party consumes `ailang/v0`).
Hash invalidation for the three migrated fixtures (`sum`, `sort`,
`max3`) is intentional; the new hashes become the new identity.
## Decision 8: explicit, verified tail calls
For an LLM author, recursion is the natural iteration form
(`\n. if n == 0 then () else loop(n-1)` is what I reach for, not
a `for`-loop). Without a tail-call guarantee, every recursive
program has a silent stack-depth ceiling that no compile-time
diagnostic warns about. That is exactly the class of correctness
hazard the language exists to eliminate.
Solution: explicit, verified tail calls.
- **AST.** `Term::App { fn, args, tail: bool }` and
`Term::Do { op, args, tail: bool }` gain a `tail` flag,
serde-defaulting to `false` so existing fixtures load with
`tail: false` and their hashes stay bit-identical.
- **Typecheck.** A new pass `verify_tail_positions(fn_body)`
runs after the main type-check. It walks the body with an
`is_tail_context: bool` threaded down. The flag is `true` at
the start, `true` for the body of every `Term::Match` arm,
`true` for the right operand of `Term::Seq`, `true` for the
body of `Term::Let`, `true` for the body of `Term::Lam` (each
Lam opens its own tail scope). The flag is `false` for: args
of any `App`/`Do`/`Ctor`, scrutinee of `Match`, left of
`Seq`, condition of `Let`-bound expression. When the walker
visits an `App { tail: true }` or `Do { tail: true }`, the
flag must be `true` at that visit; otherwise emit diagnostic
`tail-call-not-in-tail-position`.
- **Codegen.** Emit `musttail call` (LLVM IR) for marked calls
instead of plain `call`. LLVM rejects at IR-verification
time if the call cannot physically be a tail call (calling
convention mismatch, signature divergence, etc.). The reject
surfaces as a hard build error, not a silent runtime
surprise.
- **Form (A).** Two new keywords: `tail-app`, `tail-do`.
Productions are positional analogues of `app`/`do` with
`tail: true` set on the resulting term. EBNF gains 2 lines;
total production count goes from ~28 to ~30, still inside
the constraint-1 budget.
**What this does NOT promise.** Per the 14d tail-call survey,
many existing recursive calls are *not* in tail position
because they are arguments to constructor calls (e.g.
`Cons (f h) (map f t)`). 14e adds annotation + verification;
it does **not** add a CPS transform or accumulator-form
rewrite. Programs whose recursion is constructor-blocked
will continue to be stack-bounded by recursion depth. The
canonical authoring pattern in such cases is to write the
accumulator-form variant (`map_acc`, `fold_left`, etc.)
explicitly. The stdlib (15a onward) ships both forms where
relevant.
The 14d migration of existing fixtures will be partial: only
`print_list`-style terminal recursions get marked. The
constructor-blocked recursions in `map`, `sort`, `insert`
remain unmarked — they cannot benefit from `musttail`
without a source-level rewrite.
## Mangling scheme (Iter 5c)
All AILang functions are mangled to `@ail_<module>_<def>` — even in the