Iter 8c: docs (DESIGN.md + JOURNAL.md)
DESIGN.md: - Term schema: add `ctor`, `match`, `lam` rows. The schema fragment is now exhaustive for the supported language; prior versions silently dropped Iter 3/Iter 8 additions. - "What is not (yet) supported": closures-with-capture moves out of pending. New positive bullet for anonymous lambdas. Polymorphism added as an explicit pending bullet (Forall is parseable but not inferred). Smoke-test list extended with closure.ail.json. JOURNAL: append Iter 8 entry covering both 8a (closure-pair ABI) and 8b (Term::Lam + capture). Includes the rationale for skipping TIR (KISS won — `synth` already provides enough type info), the closure- conversion sketch, hash stability check (existing examples produce the same fn hashes pre vs post Iter 8), and the Iter 9 plan with two candidates ranked. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
+27
-11
@@ -178,9 +178,20 @@ hashes stay bit-identical.
|
||||
{ "t": "let", "name": "<id>", "value": Term, "body": Term }
|
||||
{ "t": "if", "cond": Term, "then": Term, "else": Term }
|
||||
{ "t": "do", "op": "<eff>/<op>", "args": [Term...] }
|
||||
{ "t": "ctor", "type": "<id>", "ctor": "<id>", "args": [Term...] }
|
||||
{ "t": "match", "scrutinee": Term, "arms": [Arm...] }
|
||||
{ "t": "lam",
|
||||
"params": ["<id>"...],
|
||||
"paramTypes": [Type...],
|
||||
"retType": Type,
|
||||
"effects": ["<id>"...],
|
||||
"body": Term }
|
||||
```
|
||||
|
||||
In the MVP, `do` is only a direct call to a built-in effect op (no handler).
|
||||
A `lam` term constructs an anonymous function value; free variables of
|
||||
its body are captured from the enclosing scope (see Iter 8 closure
|
||||
conversion in JOURNAL).
|
||||
|
||||
### Type
|
||||
|
||||
@@ -229,21 +240,22 @@ ail build <module> — full pipeline → binary
|
||||
|
||||
## What is not (yet) supported
|
||||
|
||||
Snapshot of the boundary at the end of Iter 7. Items move out of this list
|
||||
Snapshot of the boundary at the end of Iter 8. Items move out of this list
|
||||
as iterations land; the JOURNAL records the exact iteration.
|
||||
|
||||
- No closures with capture / no anonymous lambdas. Both will require a
|
||||
typed IR stage (TIR) and closure conversion in lowering. First-class
|
||||
function references (top-level fns as values, fn-typed parameters,
|
||||
indirect calls) **are** supported as of Iter 7 — see below.
|
||||
- No effect handlers — only the built-in IO and Diverge ops.
|
||||
- No refinements / SMT escalation.
|
||||
- No polymorphism in inference. `Type::Forall` is parseable but the
|
||||
typechecker rejects polymorphic uses inside a body (`PolymorphicNot
|
||||
Supported`). Generic functions must be monomorphised by the author
|
||||
via separate top-level defs.
|
||||
- No cross-module ADTs. ADTs are local to a module; ctor names must be
|
||||
unique within their module but may collide across modules.
|
||||
- No visibility rules in imports. Every top-level def of an imported module
|
||||
is reachable; there is no `pub` / `priv`.
|
||||
- No GC. The `Ctor` heap layout leaks. Acceptable for current example
|
||||
programs; required before any longer-running program.
|
||||
- No GC. ADT boxes, lambda envs, and closure pairs all leak. Acceptable
|
||||
for current example programs; required before any longer-running
|
||||
program.
|
||||
|
||||
What **is** supported (and used as the smoke test for the pipeline):
|
||||
|
||||
@@ -256,13 +268,17 @@ What **is** supported (and used as the smoke test for the pipeline):
|
||||
- **Imports + qualified cross-module references** via dotted names
|
||||
(Iter 5).
|
||||
- **First-class function references** (Iter 7). A top-level fn name (or
|
||||
qualified `prefix.def`) used as a `Term::Var` is a fn-pointer value of
|
||||
AILang type `Type::Fn { ... }` and LLVM type `ptr`. Fn-typed parameters
|
||||
may be called directly as `f(args)`. No capture, no lambdas — the only
|
||||
way to construct a fn-value is to refer to a top-level def.
|
||||
qualified `prefix.def`) used as a `Term::Var` is a fn-value.
|
||||
- **Anonymous lambdas with capture** (Iter 8). `Term::Lam` constructs a
|
||||
closure that captures any free variables of its body from the
|
||||
enclosing scope. All fn-values share a single ABI: a `ptr` to a
|
||||
closure pair `{ thunk_ptr, env_ptr }`. Top-level fns get an auto-
|
||||
generated adapter and a static closure pair (env = null) so they
|
||||
remain passable as values without heap overhead.
|
||||
|
||||
Pipeline regression smoke tests:
|
||||
|
||||
- `examples/sum.ail.json` → prints 55 (recursion, arithmetic).
|
||||
- `examples/list.ail.json` → prints 42 (ADTs + match).
|
||||
- `examples/hof.ail.json` → prints 42 (first-class fn-refs, indirect call).
|
||||
- `examples/closure.ail.json` → prints 42 (lambda capturing a let-bound var).
|
||||
|
||||
+138
@@ -519,3 +519,141 @@ working, the main delta is: a fn-value also needs an environment
|
||||
pointer, the sidetable becomes per-value (heap-allocated), and the
|
||||
calling convention shifts to `(env_ptr, args...)`. Touches every
|
||||
existing call path — that's why it gets its own iteration.
|
||||
|
||||
## 2026-05-07 — Iter 8 done: closures with capture (no TIR needed)
|
||||
|
||||
Iter 7's plan named TIR as the prerequisite for closures. On
|
||||
inspection that bundling was wrong — TIR is one possible
|
||||
implementation strategy, not a structural requirement. The
|
||||
typechecker already attaches enough type information through `synth`
|
||||
that the codegen can read capture types out of `self.locals`
|
||||
directly. So Iter 8 ships closures **without** introducing TIR. KISS
|
||||
won.
|
||||
|
||||
The work split into two commits:
|
||||
|
||||
**Iter 8a — closure-pair ABI flip.** Every fn-value is now a `ptr` to
|
||||
a heap or static closure pair `{ thunk_ptr, env_ptr }`, regardless of
|
||||
whether it came from a lambda or a top-level def reference. To keep
|
||||
top-level-fn references cheap, every top-level fn auto-emits:
|
||||
|
||||
```llvm
|
||||
define <ret> @ail_<m>_<f>_adapter(ptr %_env, <params>) {
|
||||
%r = call <ret> @ail_<m>_<f>(<args>)
|
||||
ret <ret> %r
|
||||
}
|
||||
@ail_<m>_<f>_clos = constant { ptr, ptr } { @adapter, null }
|
||||
```
|
||||
|
||||
`Term::Var` resolving to a top-level fn returns the address of
|
||||
`_clos`, never the bare fn pointer. `emit_indirect_call` was
|
||||
rewritten to GEP+load both halves and call `thunk(env, args...)`.
|
||||
Direct calls (statically-known callees in `Term::App`) bypass the
|
||||
adapter and stay at the original speed.
|
||||
|
||||
The Iter 7 hof example (`apply(inc, 41)`) continues to print 42
|
||||
unchanged — only the IR shape changed, not the source. IR snapshot
|
||||
files for sum/list/max3/hello/ws_main were refreshed.
|
||||
|
||||
**Iter 8b — Term::Lam + capture + lambda lifting.** New AST node:
|
||||
|
||||
```jsonc
|
||||
{ "t": "lam",
|
||||
"params": ["x"...],
|
||||
"paramTypes": [Type...],
|
||||
"retType": Type,
|
||||
"effects": ["..."],
|
||||
"body": Term }
|
||||
```
|
||||
|
||||
Param/return types are explicit. The typechecker accepts the
|
||||
declared `Type::Fn` shape, checks the body's type against `retType`,
|
||||
and verifies that body effects are a subset of the declared lambda
|
||||
effects (no row polymorphism in the MVP). Constructing a lambda is
|
||||
pure; the act of *calling* picks up the declared effects, via the
|
||||
existing App branch.
|
||||
|
||||
Codegen does textbook closure conversion:
|
||||
|
||||
1. **Free-variable analysis.** `collect_captures` walks the body
|
||||
skipping builtins (`+`, `==`, ...), the current module's top-
|
||||
level fns, and qualified `prefix.def` names. The remainder are
|
||||
captures. Inner lambdas contribute their own free vars upward.
|
||||
2. **Lift to thunk.** For each lambda, generate a fresh
|
||||
`@ail_<m>_<def>_lam<id>(ptr %env, params...)`. State the body
|
||||
into a side buffer (the emitter's `body`/`locals`/`counter` are
|
||||
saved and reset, then restored). Captures and lambda params are
|
||||
pushed as named locals so the body lowering finds them. The
|
||||
thunk text goes into a `deferred_thunks` queue and is appended
|
||||
after the parent fn's `}` — LLVM IR doesn't care about fn order.
|
||||
3. **Pack at the use site.** In the OUTER body emit:
|
||||
|
||||
```llvm
|
||||
%env = call ptr @malloc(i64 <8 * captures>)
|
||||
; for each capture i: store at offset 8*i
|
||||
%clos = call ptr @malloc(i64 16)
|
||||
; store thunk_ptr at offset 0, env at offset 8
|
||||
```
|
||||
|
||||
`%clos` is the value returned by the Lam term. Its sig is
|
||||
registered in the sidetable so subsequent indirect calls work.
|
||||
|
||||
4. **Capture sigs propagate.** A fn-typed capture (e.g. capturing a
|
||||
fn-typed param of an outer scope) keeps its FnSig in the thunk's
|
||||
sidetable, so the captured fn can still be indirect-called from
|
||||
inside the lambda.
|
||||
|
||||
Capture layout uses 8-byte slots regardless of LLVM type. Typed
|
||||
load/store reads only the bytes it needs — wasted padding for `i1`
|
||||
and `i8` is fine at this scale.
|
||||
|
||||
**Architecture self-check:**
|
||||
|
||||
- *Would I use this language now?* Yes for substantially more cases.
|
||||
`let n = 3 in apply(\\x. x + n, 39)` is the example I would have
|
||||
reached for in Iter 6 and bounced off. It now compiles and runs.
|
||||
`map`/`fold`/`filter` over user-supplied predicates are within
|
||||
reach — only the absence of polymorphism still forces author-side
|
||||
monomorphisation.
|
||||
- *Did I think of everything?* Hash stability checked manually:
|
||||
`examples/sum.ail.json` produced the same fn hashes
|
||||
(`db33f57cb329935e`, `d9a916a0ed10a3d3`) before and after Iter 8.
|
||||
Existing modules without `Term::Lam` serialise bit-identically. ✓
|
||||
- *Consistency:* DESIGN.md "What is not (yet) supported" rewritten
|
||||
in the same edit. The Term schema gained `lam`, `ctor`, `match`
|
||||
rows that were already supported but had been omitted from the
|
||||
schema fragment. Now the doc is exhaustive for the supported
|
||||
language.
|
||||
- *Visualisation:* `ail describe` already renders Lam terms (added
|
||||
pretty-printer rule), and the codegen IR for `closure.ail.json`
|
||||
reads as a textbook closure-conversion lowering.
|
||||
- *KISS check:* I considered three alternatives and all were
|
||||
strictly worse — fat-pointer ABI (aggregate-passing concerns), full
|
||||
TIR layer (large rewrite), uniform heap pair without static-closure
|
||||
optimisation (regressed Iter 7 to one malloc per fn-value escape).
|
||||
|
||||
**Tests:** 49 green (was 48 after Iter 7). One new e2e:
|
||||
`closure_captures_let_n` builds and runs `examples/closure.ail.json`
|
||||
asserting "42". IR snapshot files refreshed for the per-fn adapter +
|
||||
static-closure scaffold — only structural delta.
|
||||
|
||||
**Plan iteration 9:**
|
||||
|
||||
Two candidates, both real pain points:
|
||||
|
||||
1. **Polymorphic inference.** Make `Type::Forall` actually work in
|
||||
`synth` — instantiate fresh type variables at each use site, allow
|
||||
`let id = \\x. x in (id 1, id true)`. This unblocks generic
|
||||
`map`/`fold`/etc. without per-type clones. Probably small (~150
|
||||
LOC in the typechecker; codegen already monomorphises by
|
||||
instantiation when it lowers the call).
|
||||
|
||||
2. **GC / region reclamation.** Right now ADT boxes, lambda envs,
|
||||
and closure pairs all leak through the program's lifetime. A
|
||||
minimal mark-and-sweep over a tagged heap would let us run real
|
||||
programs. Bigger lift, ~400-600 LOC plus runtime support.
|
||||
|
||||
Leaning toward (1) for the next iteration: it's the smaller bite
|
||||
*and* the bigger expressivity unlock. (2) becomes acute only when
|
||||
someone tries to run an unbounded loop, which the current examples
|
||||
don't.
|
||||
|
||||
Reference in New Issue
Block a user