Files
AILang/docs/JOURNAL.md
T
Brummel 1918fbee7f Iter 13f: rustdoc polish for ailang-codegen + ail CLI
Third docwriter mission (combined). Pure rustdoc additions, no
API or behaviour change.

ailang-codegen (113 LOC of doc):
- Crate root: intra-doc links (emit_ir, lower_workspace,
  CodegenError::MissingEntryMain) + precondition note (both
  entry points assume type-checked input).
- /// on CodegenError + every variant, naming AST trigger.
- /// on emit_ir and lower_workspace (single vs multi-module
  split, cross-linked).

ail CLI (49 LOC of doc):
- Module-level //! expanded from 5 lines to full subcommand
  list with one-liners (11 subcommands verified against Cmd
  enum), clang-on-PATH note, design-intent paragraph.

Verification: cargo doc --no-deps zero warnings (also under
RUSTDOCFLAGS='-D rustdoc::broken_intra_doc_links'); build green;
tests 64/64 + 3 ignored doctests green. Diff is 100% doc lines
(verified by filtering).

Findings (not fixed; orchestrator-deferred):
- CodegenError::Internal is one catch-all variant for ~30
  invariant-violation sites; splitting would help test
  ergonomics but is out of doc scope.
- emit_ir synthesises a Workspace with root_dir="."; harmless
  today, surfaces if codegen ever reads root_dir.

Process note: my brief said the CLI had 9 subcommands; agent
found and documented 11. Useful counter-pressure on orchestrator
sloppiness — recorded in JOURNAL.

Workspace-wide rustdoc invariant (DESIGN.md item 6) now
load-bearing across all four crates. Docwriter shifts from
sweep to maintenance mode going forward.

Next: 14a (polymorphic list_map using Iter-13a parameterised
ADTs) is unblocked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 15:20:17 +02:00

1410 lines
65 KiB
Markdown

# JOURNAL
Chronological notes for myself. Not every change; only decisions, obstacles,
and observations that future iterations will need.
## 2026-05-07 — Day 0
- Repo initialised. Assignment in `CLAUDE.md`: LLM-native language, LLVM backend.
- Design decisions captured in `docs/DESIGN.md`.
- Toolchain: `rustc 1.94`, `llvm-config 22.1.3`, `clang` available.
- Decided against `inkwell` in favour of LLVM IR text emit. Rationale in DESIGN.md.
- Workspace layout:
- `crates/ailang-core` — AST, type, hash, JSON schema
- `crates/ailang-check` — typechecker (comes later)
- `crates/ailang-codegen` — lowering + LLVM IR emit
- `crates/ail` — CLI
- MVP goal: `examples/sum.ail.json` → binary that prints 55. **Achieved.**
## 2026-05-07 — architecture review after the MVP
Still on track? Broadly yes. Concrete observations:
**What holds:**
- JSON AST + canonical form + content hash are all lego bricks that later
tools can build on without a refactor (`ail deps`, `ail diff`).
- The LLVM IR text pipeline works as planned. No libllvm version pain.
- The effect set is wired into the type system from the start. Extensible to
row-poly without touching the core.
**Debt that accrues interest:**
1. **`current_block_label_for_phi` is a heuristic** (see codegen). On nested
`if` terms it will return the wrong block label, because it scans the body
backwards. Ticking, because no test cases trigger it yet. Must be fixed
next, before new language features arrive.
2. **No typed AST.** Codegen reads the source AST directly and relies on
the typechecker having run before. Fine for the MVP; once ADTs or
closures arrive, I will need a separate typed IR stage (TIR).
3. **The `hash` field is not in the AST.** Right now we hash the def object
directly. Once I serialise hashes as fields (caching), the hash will
need to exclude that field before computation.
**Plan iteration 2 (now):**
1. Clean up block-label tracking, with a nested-if test.
2. Strings as a literal + `io/print_str`.
3. Hello-world example as a second E2E test.
4. CLI: `--json` output for machine consumers wherever it fits.
**Plan iteration 3:**
ADTs + pattern matching. That is the next big jump. Requires a typed IR
stage (TIR), because pattern matching lowers into decision trees, which
have a different shape from the AST.
## 2026-05-07 — iteration 2 done
- Block-label tracking is now robust (nested `if`s work). Test
`max3_picks_largest` protects it.
- Strings as `Lit::Str { value }`, type `Str` -> LLVM `ptr`, with
`io/print_str` effect op. `examples/hello.ail.json` prints a string.
- CLI: `manifest --json`, `builtins --json` for tool consumers.
- `ail deps [--of NAME] [--json]` lists call edges. Effect ops are tagged
`effect:NAME` so a consumer can filter them.
**Architecture check:** no structural deviations. Codegen still reads the
source AST directly (a TIR stage will become necessary with ADTs in
iteration 3).
## 2026-05-07 — iteration 3 done: ADTs
- TypeDef in the AST with ctors. A ctor has `name` and `fields: [Type...]`.
- Term::Ctor (construction) and Term::Match (pattern matching).
- Patterns: `Wild`, `Var`, `Lit`, `Ctor { ctor, fields }`. In the MVP,
nested ctor patterns are NOT allowed — sub-patterns must be `Var` or
`Wild`.
- Typechecker with a type registry and `ctor_index` (ctor name → ADT). In
Match, exhaustiveness is checked against the full constructor set. A
negative test protects this.
- Codegen: boxed heap layout. Per ctor application, `malloc(8 + 8*n)`
bytes; tag at offset 0, fields from offset 8 (8-byte slots, native typed
load/store). Match: load tag + switch + arm blocks + phi at the join.
- `examples/list.ail.json` (Cons/Nil list, sum_list via match) returns 42.
**Surprisingly painless.** The architecture decisions from day 0 paid off:
opaque ptr in LLVM 22 makes the boxed layout almost glue-free; effect
tracking was untouched by ADTs; the JSON AST takes new node types
cleanly.
**Debt accrued:**
1. **Codegen still reads the source AST directly.** The temptation to push
on without TIR was strong — and worked, because my Match restrictions
are flat (no nested patterns). Once nested patterns arrive, decision-
tree lowering will not stay clean without TIR. Debt acknowledged; not
due now.
2. **No GC.** The heap leaks. Acceptable for demo programs; must be
addressed before any longer-running program. Options for Phase 4:
refcount, Boehm-GC linkage, region inference.
3. **No runtime pretty-printer for ADT values.** `io/print_int` is enough
for demos, but a generic `show :: a -> Str` for ADTs would be valuable.
Requires dispatch over the tag — feasible, but not now.
**Plan iteration 4:**
The next steps are less obvious. Three candidates in priority order:
1. **Module system (imports).** Right now everything is in a single module.
With multiple modules + cross-module hashing the language only becomes
practical for several defs.
2. **Structured error output (`ail check --json`).** So tools can react to
type errors without parsing text.
3. **Closures / higher-order functions.** Requires closure conversion and
is a bigger step.
Iteration 4 will be (1) + (2) — both strengthen the LLM tooling and have
moderate risk.
## 2026-05-07 — workflow change: orchestrator + agent repo
At the user's suggestion, switching to **orchestrator mode**: I delegate
clearly bounded implementation chunks to sub-agents and keep only
architecture decisions, reviews, and commit discipline. Four specialised
agents drafted: implementer, architect, tester, debugger.
**Important correction:** the user required the agents not to be hidden in
`.claude/agents/`, but versioned as a visible part of the project under
`agents/`. DESIGN.md gained a new section "Project ecosystem", which
records this: AILang is not just a language, but language core + CLI +
examples + agents + docs + tests, all of equal weight.
Invocation scheme: the system-prompt body from `agents/<name>.md` as a
prefix before the concrete task + sent to the `general-purpose` agent.
Functionally identical to subagent loading from `.claude/agents/`, but
visible in the repo.
**Plan iteration 4 (revised):**
The module system is more involved than expected (cross-module hashing,
import resolution). First the smaller tooling wins, then the module system
as iteration 5:
1. **Structured error output** (`ail check --json` with a Diagnostic
struct, stable codes like `unbound-var`, `type-mismatch`).
2. **`ail diff <a> <b>`** — semantic module diff via per-def hash
comparison.
3. **IR snapshot tests** — regression protection for the codegen pipeline.
## 2026-05-07 — iteration 4 done: LLM tooling consolidation
Three sub-commits, each produced by an `ailang-implementer` invocation
and spot-checked by the orchestrator:
- `93fe723` Iter 4a: `ail check --json` with a `Diagnostic` struct
(`severity`, `code`, `message`, `def`, `ctx`). Stable codes:
`unbound-var`, `type-mismatch`, `arity-mismatch`,
`non-exhaustive-match`, `unknown-ctor`, `unknown-ctor-in-pattern`,
`nested-ctor-pattern-not-allowed`, `duplicate-def`,
`unknown-effect-op`, `unknown-type`, `schema-mismatch`. API:
`check_module(&Module) -> Vec<Diagnostic>`.
- `c652b12` Iter 4b: `ail diff <a> <b> [--json]` as a structural top-level
def diff via BLAKE3 hash. Four categories (added/removed/changed/
unchanged), sorted alphabetically, exit code 1 on diff.
- `74a2005` Iter 4c: IR snapshot tests in
`crates/ail/tests/snapshots/{sum,max3,hello,list}.ll`.
Normalisation of `target triple`. Update via `UPDATE_SNAPSHOTS=1
cargo test ir_snapshot_`. Mismatch produces an `.actual` file.
Test count: 28 (previously 19). 7 E2E + 4 IR snapshot + 9 ailang-check + 1
ailang-codegen + 7 ailang-core.
**Closed from the debt register:**
- Block tracking in codegen has not been a heuristic risk since Iter 2;
the explicit `current_block: String` track is now additionally
protected against regression by Iter 4c snapshot tests. Debt closed.
**New / sharpened debt:**
1. `check_module` is **single-shot** — the first error aborts, no
multi-diagnostic gathering. The spec was that way, but the format
suggests Vec semantics. A real multi-diagnostic refactor will be
cheaper once TIR exists (a central error accumulator via a separate
stage). Not due now.
2. `source_filename` in the IR is hard-coded to `"<module>.ail"`. As long
as there is only one top-level module, that is platform-stable. With
Iter 5 (module system + imports) the path becomes relevant — keep it
path-independent at construction time, otherwise the snapshots will
tip over.
**Plan iteration 5:** module system with imports. Cross-module hashing,
import resolution, multiple `.ail.json` files in one build. The multi-
diagnostic refactor only after that.
Sub-steps:
- **5a — workspace loader.** `ailang_core::Workspace { modules:
BTreeMap<String, Module> }` plus `load_workspace(entry: &Path)`, which
follows `imports` recursively from the entry module. Convention:
`import { module: "foo" }` resolves to `<dir>/foo.ail.json` next to the
entry. Cycle detection. CLI: existing subcommands keep working on a
single module; a new `ail workspace <entry>` lists all reachable
modules with hash. Tests: two small example modules with an import
relation; cycle test.
- **5b — cross-module typecheck.** The typechecker takes `&Workspace`
instead of `&Module`. Imports are mounted in the env as a namespace
(`alias.def` or, with no alias, `module.def`). New diagnostic codes:
`unknown-module`, `unknown-import`, `import-cycle`, `ambiguous-name`.
Tests per code.
- **5c — cross-module codegen.** The emitter produces IR for all modules
in the workspace, prefix-mangled with `@ail_<module>_<def>`. E2E test:
a program that uses a function from module B in module A returns the
correct result in the binary.
- **5d — tooling adjustments.** `manifest`, `describe`, `deps`, `diff`
gain a `--workspace` mode (recursive). The single mode stays the
default for backwards compatibility.
During Iter 5, at construction time **keep `source_filename`
path-independent** (module name only, no directory prefix), otherwise
the IR snapshots will tip over.
## 2026-05-07 — Iter 5b done: cross-module typecheck
- `check_workspace(&Workspace) -> Vec<Diagnostic>` as the top-level API.
`check_module` is preserved and internally lifts the module into a
trivial workspace.
- Convention for qualified references (recorded in DESIGN.md):
`Term::Var { name }` with exactly one dot = `<prefix>.<def>`. Prefix is
an import alias or module name. No new AST node, no renamed fields ⇒
hashes stay stable; all `ir_snapshot_*` still green.
- Three new diagnostic codes: `unknown-module`, `unknown-import`,
`invalid-def-name` (with `ctx.reason: "contains-dot"`).
- CLI: `ail check <entry>` now **always** loads via `load_workspace`.
Workspace load failures become structured diagnostics in JSON mode with
codes `module-not-found`, `module-cycle`, `module-name-mismatch`,
`module-hash-mismatch`, `schema-mismatch`. `ail build` and
`ail emit-ir` stay per single module (cross-module codegen is 5c).
- Examples: `ws_main.ail.json` now calls `ws_lib.add` (observable). New:
`ws_broken.ail.json` (`unknown-import`),
`ws_unknown_module.ail.json` (`unknown-module`).
- Tests: 37 green (previously 32). 4 new workspace integration tests in
`crates/ailang-check/tests/workspace.rs`, one new e2e test
`check_workspace_resolves_import`.
- Debt: single-shot diagnostics still in place (multi-diagnostic after
5c). The dot convention covers exactly one dot — nested module paths
(`a.b.c`) do not exist; that would only be a topic with hierarchical
modules and currently falls through as `unbound-var`.
## 2026-05-07 — Iter 5c done: cross-module codegen
- **Mangling break (deliberate).** All AILang functions are now called
`@ail_<module>_<def>`, even in single-module programs. The old form
`@ail_<def>` is gone. Strings/const globals analogously
(`@.str_<module>_<hint>_<idx>`, `@ail_<module>_<const>`). The entry
point stays `main` as C ABI: a `define i32 @main()` trampoline calls
`@ail_<entry-module>_main()`. If the entry module has no
`main : () -> Unit !IO`, the build fails with `MissingEntryMain`.
- **Workspace lowering.** New top-level API
`ailang_codegen::lower_workspace(ws: &Workspace) -> Result<String>`
produces a single `.ll` for the whole workspace. Modules in
alphabetical order (BTreeMap order); defs in AST order. Cross-module
calls are resolved in codegen via the import map of the calling module
— same logic as in the typechecker, locally duplicated with a
cross-reference (no shared helper module, because the type worlds
differ: the typechecker handles `Type`, codegen handles `FnSig` from
llvm types).
- **CLI.** `ail build` and `ail emit-ir` now always load the workspace
and check/lower it fully. Single-module programs keep working
(trivial workspace with one module). `emit_ir(m)` stays in the codegen
crate as a convenience API and internally wraps into a trivial
workspace.
- **Snapshots regenerated.** `sum.ll`, `max3.ll`, `hello.ll`, `list.ll`
show the new mangling. New `ws_main.ll` snapshot documents the
cross-module build: `@ail_ws_main_main` calls `@ail_ws_lib_add`.
- **Tests.** 40 green (previously 37). New: `workspace_build_runs_imported_fn`
(e2e: prints 5), `ir_snapshot_ws_main`, `missing_entry_main_is_error`
(codegen unit). Existing behaviour tests
(`sum_1_to_10_is_55`, `max3_picks_largest`, `hello_world_str_lit`,
`list_sum_via_match`) stay green — behaviour unchanged, only the
mangling is new.
**Debt closed:**
- **#19 (`source_filename` hardening).** In the workspace world,
`source_filename` is now uniformly `<entry-module>.ail`, once per
workspace. The previous hard-coded path dot is gone with it.
**State:** the module system is closed end to end — loader + typecheck +
codegen + build see the workspace as a coherent unit. The multi-
diagnostic refactor and possibly cross-module ADTs remain for later.
## 2026-05-07 — Iter 5d done: tooling extended to the workspace
- `ail manifest|describe|deps|diff <entry> --workspace` now operate
across all modules of the workspace. The default without the flag stays
single-module for backwards compatibility. Manifest sorts by
`(module, name)`, describe accepts dotted notation `ws_lib.add`, deps
emits `{from_module, from_def, to_module, to_def}` edges, diff compares
workspace-wide with added/removed/changed/unchanged_modules and a
nested sub-diff per changed_module.
- Refactor: `diff_def_lists` is the single source of the four-category
logic; single and workspace diff share it.
- Tests: 44 green (previously 40). New: `manifest_workspace_lists_all_defs`,
`describe_workspace_resolves_qualified_name`,
`deps_workspace_includes_cross_module`, `diff_workspace_added_module`.
**Observation (debt):** `deps` does not filter builtins/locals/function
parameters. In workspace mode that becomes more visible than in single
mode — `ws_lib.add` lists edges to `ws_lib.+` (builtin) and
`ws_lib.a`/`ws_lib.b` (function parameters). A known pre-existing
issue from Iter 2; Task #22 in the backlog.
## 2026-05-07 — architecture review after Iter 5
Architect agent invoked. Findings:
1. **Mangling consistency holds.** `@ail_<module>_<def>` is consistent
across functions, constants, string globals, and cross-module calls.
The trampoline is correct. ADT constructors are deliberately
symbol-free (inline malloc).
2. **Module hashes bit-identical since Iter 4.** The Iter 5c snapshot
regeneration was a codegen-output change, not a hash break.
3. **Drift, due now:**
- DESIGN.md says `define i64 @main()`, codegen emits
`define i32 @main()` (see `sum.ll:35`).
- String-schema notation in DESIGN.md was shortened
(`@.str_<module>_<idx>` instead of `@.str_<module>_<hint>_<idx>`).
4. **Debt that accrues interest:** the `deps` builtin leak (Task #22) has
become a falsehood in workspace mode — close it before the next big
jump.
**Plan iteration 6 — clean-up:**
1. **Fix DESIGN.md drift.** Update the mangling-scheme block, correct the
`@main` signature, and note the string globals precisely.
2. **`deps` hardening (#22).** Build a top-level def table per workspace;
filter edges whose target is not a top-level symbol, or emit them as
separate `builtin:`/`local:` categories. Function parameters via
lexical scope tracking from walk_term.
3. **Multi-diagnostic refactor (#20).** `check_workspace` accumulates
`Vec<Diagnostic>` across all defs instead of short-circuiting on the
first error. Intra-def may still short-circuit — the value is "see
all broken defs at once", not "see all broken sub-terms of one def".
Order: 1 first (doc triviality), then 2 before 3 (deps is a tooling-
truth fix, multi-diag is a structural extension).
## 2026-05-07 — Iter 6 done: deps hardening + multi-diagnose + DESIGN audit
Three things landed together. All small, all KISS — no architecture move,
just paying off recorded debt.
**1. `ail deps` filters builtins, params, and let/match bindings (#22).**
Before: `sum -> +, -, ==, n, sum` and `ws_lib.add -> ws_lib.+`,
`ws_lib.a`, `ws_lib.b`. After: `sum -> sum`, `ws_lib.add -> ws_lib.add`
gone (no real deps; only the cross-module call from `ws_main` remains).
Implementation:
- New helper `ailang_check::builtins::value_names()`: derives the
Var-level builtin names (`+ - * / % == != < <= > >= not`) from
`list()`, so the install-list and the deps-filter share one source of
truth.
- `walk_term` in `crates/ail/src/main.rs` now threads a `scope` set:
fn-params seed it; `Let` adds the bound name for the body only;
`Match` arms add their pattern variables (`bind_pattern` helper, MVP
rule "ctor sub-patterns are Var/Wild") and roll them back after. Var
refs that hit `scope` or `builtins` are dropped; qualified names
(`prefix.def`) are passed through unconditionally — the typechecker
forbids dots in def names, so no shadowing risk.
- Tests added: `deps_filters_builtins_params_locals`,
`deps_workspace_filters_builtins_and_params`. The Iter 5d test
(`deps_workspace_includes_cross_module`) keeps passing — the only
edge it asserted is the legitimate one.
**2. `check_module` / `check_workspace` are multi-diagnose (#20).**
`check_in_workspace` returns `Vec<CheckError>` instead of `Result<()>`.
Pass-1 (top-level symbol table) stays fail-fast — corrupt globals would
taint every later diagnostic. Type-def installation is fail-fast within
a module (env corruption) but the outer module loop continues. The
body-check loop is the multi-diagnose layer: each def is checked against
the assembled env, errors accumulate, the next def is attempted.
Test: `body_errors_accumulate_across_defs` — one module with two
independent body errors (arity mismatch + unknown var) yields two
diagnostics with the right `def` field. The legacy single-error `check`
keeps working by `.into_iter().next()`-ing the Vec, so internal snapshot
tests in `crates/ailang-check/src/lib.rs` are unchanged.
Out of scope: intra-def collection. A single fn body with three type
errors still reports one. The "see all broken defs at once" goal is met;
intra-def will require unification deferral and isn't due now.
**3. DESIGN.md `What the MVP is NOT` audit (#24).**
The section was lying: it claimed "No ADTs / pattern matching" (delivered
Iter 3) and "Only ints + bools + unit" (strings landed Iter 2). Renamed
to `What is not (yet) supported`, restructured into "not yet" + "what is
supported (smoke-tested)". New invariant: this section is meant to be
the truth at the **end of the latest iteration**, not a 2026-05-07-day-0
scope statement.
**Architecture check (the user-asked self-questioning):**
- *Would I use this language now?* For non-recursive arithmetic + ADT
programs over int/bool/str: yes, comfortably. For anything that needs
mapping, folding, generic data structures: no, closures are the
blocker. That's the next big sprint, not Iter 7.
- *Consistency:* DESIGN.md, JOURNAL.md, code, and CLI output now agree
on what the language can do. The "What is not (yet) supported" block
is the canonical truth surface.
- *Visualisation:* `ail deps --workspace --json` is now a clean
cross-module call graph (no builtin noise). Good enough for an
external graph renderer to consume; a built-in DOT/ASCII renderer is
*possible future tooling*, not "we need it now". KISS.
- *Documentation:* the agents/ directory is the sub-prompt layer, the
JOURNAL is the iteration log, DESIGN.md is the contract. No new doc
axes needed at this scale.
**Tests:** 47 green (previously 44). +2 deps tests in `e2e.rs`, +1
multi-diag test in `crates/ailang-check/tests/workspace.rs`.
**Plan iteration 7:**
Closures + higher-order functions. This is the big jump that
DESIGN.md / Day 0 has been pointing at: it requires a typed IR (TIR)
stage, closure conversion in lowering, and a heap-aware ABI. The
multi-diag refactor in Iter 6 was scoped intentionally minimal — when
TIR lands, intra-def diagnostics become structurally cheap and Task #20
gets revisited.
## 2026-05-07 — Iter 7 done: first-class function references (no capture)
Iter 6 outlined Iter 7 as "closures + HOFs + TIR". KISS course-correct
on inspection: that bundle had three independent things in it, and the
HOF use-cases (passing functions around, calling through fn-typed
parameters) need none of TIR or capture. Splitting paid off — what
landed here is ~120 LOC of codegen, no TIR, no heap, no ABI churn.
Closures with capture stay queued for Iter 8 (where TIR is the
correct precondition).
**What works now:**
- Top-level fn name (or qualified `prefix.def`) used as a value yields
an LLVM fn-pointer (`@ail_<m>_<def>`, type `ptr`).
- Fn-typed parameters can be called as `f(args)` — the body emits an
indirect `call <ret> (<param-tys>) %f(...)`.
- Pass through `let`: `let g = inc in g(x)` works (the local just
aliases the global SSA, the sidetable lookup still hits).
- Pass to another fn: `apply(inc, 41) == 42` — see
`examples/hof.ail.json`, exercised end-to-end.
**What does not (yet) work — by design:**
- No anonymous lambdas. The only fn-value source is a top-level def
reference.
- No capture. A fn-value is always a constant pointer to a top-level
def; there is no environment to allocate.
- Both deferred to Iter 8 where they share the TIR + closure-conversion
preconditions.
**Implementation, in order of where the rubber meets the road:**
1. `llvm_type` learned `Type::Fn { .. } -> "ptr"`. The actual signature
travels separately. New helper `fn_sig_from_type` lifts an AILang
fn-type into an `FnSig` (LLVM types only).
2. `Emitter` got a sidetable: `ssa_fn_sigs: BTreeMap<String, FnSig>`,
keyed by SSA value (or `@global`). It's reset per function body.
3. At `emit_fn` entry, every fn-typed parameter registers
`(%arg_<name>, sig)` in the sidetable.
4. `lower_term(Term::Var)` now falls through to a top-level fn lookup
(`resolve_top_level_fn`) when the name isn't a local. The returned
SSA is the global symbol; the sidetable gets the sig.
5. `lower_term(Term::App)` dispatches:
- if callee is a `Var` AND not shadowed AND statically known
(`is_static_callee` covers builtin operators, qualified
`prefix.def`, current-module fns), keep the existing direct
`lower_app` path — no extra indirection in the IR;
- otherwise lower the callee, expect type `ptr`, look up the sig in
the sidetable, emit `emit_indirect_call`.
6. `Term::If` propagates the sig to its phi SSA when both branches are
fn-pointers with matching sigs (cheap two-line copy; no separate
test, falls out of the `apply`-on-conditional pattern).
**Why no typechecker change?** The typechecker already accepted
fn-typed locals (`Term::Var` against `env.globals`, App via `synth(callee)`
unifying with `Type::Fn`). The only blocker was `MVP: callee must be a
variable` in codegen.
**Tests:** 48 green (previously 47).
- `crates/ail/tests/e2e.rs::higher_order_apply_inc` builds and runs
`examples/hof.ail.json`, asserts the binary prints `42`.
- Existing tests unchanged (incl. snapshot tests around the IR
emission for `sum`, `list`, `max3`).
**Architecture self-check:**
- *Would I use this language now?* Yes for `apply`-style and "pass a
predicate" patterns. Still no for capturing closures (`let n = 3 in
map(\x -> x + n, xs)`-equivalent), but the ergonomic gap shrank.
- *Consistency:* DESIGN.md "What is not (yet) supported" rewritten in
the same edit; first-class fn-refs now have a positive bullet, the
closures bullet is precise about what it means (no capture, no
lambdas).
- *Visualisation:* `ail describe`/`manifest` already render fn-typed
params correctly via the existing `pretty::type_to_string`
(`((Int) -> Int, Int) -> Int`). No tooling change required.
- *KISS:* every alternative I considered (full `LocalType` enum,
swapping `(String, String)` returns to a typed wrapper, lifting
lambdas to defs as syntactic sugar) was strictly more code than the
sidetable approach, with no expressivity gain.
**Plan iteration 8:**
Closures with capture, anonymous lambdas, the typed IR (TIR) layer,
closure conversion in lowering. Now that we have indirect calls
working, the main delta is: a fn-value also needs an environment
pointer, the sidetable becomes per-value (heap-allocated), and the
calling convention shifts to `(env_ptr, args...)`. Touches every
existing call path — that's why it gets its own iteration.
## 2026-05-07 — Iter 8 done: closures with capture (no TIR needed)
Iter 7's plan named TIR as the prerequisite for closures. On
inspection that bundling was wrong — TIR is one possible
implementation strategy, not a structural requirement. The
typechecker already attaches enough type information through `synth`
that the codegen can read capture types out of `self.locals`
directly. So Iter 8 ships closures **without** introducing TIR. KISS
won.
The work split into two commits:
**Iter 8a — closure-pair ABI flip.** Every fn-value is now a `ptr` to
a heap or static closure pair `{ thunk_ptr, env_ptr }`, regardless of
whether it came from a lambda or a top-level def reference. To keep
top-level-fn references cheap, every top-level fn auto-emits:
```llvm
define <ret> @ail_<m>_<f>_adapter(ptr %_env, <params>) {
%r = call <ret> @ail_<m>_<f>(<args>)
ret <ret> %r
}
@ail_<m>_<f>_clos = constant { ptr, ptr } { @adapter, null }
```
`Term::Var` resolving to a top-level fn returns the address of
`_clos`, never the bare fn pointer. `emit_indirect_call` was
rewritten to GEP+load both halves and call `thunk(env, args...)`.
Direct calls (statically-known callees in `Term::App`) bypass the
adapter and stay at the original speed.
The Iter 7 hof example (`apply(inc, 41)`) continues to print 42
unchanged — only the IR shape changed, not the source. IR snapshot
files for sum/list/max3/hello/ws_main were refreshed.
**Iter 8b — Term::Lam + capture + lambda lifting.** New AST node:
```jsonc
{ "t": "lam",
"params": ["x"...],
"paramTypes": [Type...],
"retType": Type,
"effects": ["..."],
"body": Term }
```
Param/return types are explicit. The typechecker accepts the
declared `Type::Fn` shape, checks the body's type against `retType`,
and verifies that body effects are a subset of the declared lambda
effects (no row polymorphism in the MVP). Constructing a lambda is
pure; the act of *calling* picks up the declared effects, via the
existing App branch.
Codegen does textbook closure conversion:
1. **Free-variable analysis.** `collect_captures` walks the body
skipping builtins (`+`, `==`, ...), the current module's top-
level fns, and qualified `prefix.def` names. The remainder are
captures. Inner lambdas contribute their own free vars upward.
2. **Lift to thunk.** For each lambda, generate a fresh
`@ail_<m>_<def>_lam<id>(ptr %env, params...)`. State the body
into a side buffer (the emitter's `body`/`locals`/`counter` are
saved and reset, then restored). Captures and lambda params are
pushed as named locals so the body lowering finds them. The
thunk text goes into a `deferred_thunks` queue and is appended
after the parent fn's `}` — LLVM IR doesn't care about fn order.
3. **Pack at the use site.** In the OUTER body emit:
```llvm
%env = call ptr @malloc(i64 <8 * captures>)
; for each capture i: store at offset 8*i
%clos = call ptr @malloc(i64 16)
; store thunk_ptr at offset 0, env at offset 8
```
`%clos` is the value returned by the Lam term. Its sig is
registered in the sidetable so subsequent indirect calls work.
4. **Capture sigs propagate.** A fn-typed capture (e.g. capturing a
fn-typed param of an outer scope) keeps its FnSig in the thunk's
sidetable, so the captured fn can still be indirect-called from
inside the lambda.
Capture layout uses 8-byte slots regardless of LLVM type. Typed
load/store reads only the bytes it needs — wasted padding for `i1`
and `i8` is fine at this scale.
**Architecture self-check:**
- *Would I use this language now?* Yes for substantially more cases.
`let n = 3 in apply(\\x. x + n, 39)` is the example I would have
reached for in Iter 6 and bounced off. It now compiles and runs.
`map`/`fold`/`filter` over user-supplied predicates are within
reach — only the absence of polymorphism still forces author-side
monomorphisation.
- *Did I think of everything?* Hash stability checked manually:
`examples/sum.ail.json` produced the same fn hashes
(`db33f57cb329935e`, `d9a916a0ed10a3d3`) before and after Iter 8.
Existing modules without `Term::Lam` serialise bit-identically. ✓
- *Consistency:* DESIGN.md "What is not (yet) supported" rewritten
in the same edit. The Term schema gained `lam`, `ctor`, `match`
rows that were already supported but had been omitted from the
schema fragment. Now the doc is exhaustive for the supported
language.
- *Visualisation:* `ail describe` already renders Lam terms (added
pretty-printer rule), and the codegen IR for `closure.ail.json`
reads as a textbook closure-conversion lowering.
- *KISS check:* I considered three alternatives and all were
strictly worse — fat-pointer ABI (aggregate-passing concerns), full
TIR layer (large rewrite), uniform heap pair without static-closure
optimisation (regressed Iter 7 to one malloc per fn-value escape).
**Tests:** 49 green (was 48 after Iter 7). One new e2e:
`closure_captures_let_n` builds and runs `examples/closure.ail.json`
asserting "42". IR snapshot files refreshed for the per-fn adapter +
static-closure scaffold — only structural delta.
**Plan iteration 9:**
Two candidates, both real pain points:
1. **Polymorphic inference.** Make `Type::Forall` actually work in
`synth` — instantiate fresh type variables at each use site, allow
`let id = \\x. x in (id 1, id true)`. This unblocks generic
`map`/`fold`/etc. without per-type clones. Probably small (~150
LOC in the typechecker; codegen already monomorphises by
instantiation when it lowers the call).
2. **GC / region reclamation.** Right now ADT boxes, lambda envs,
and closure pairs all leak through the program's lifetime. A
minimal mark-and-sweep over a tagged heap would let us run real
programs. Bigger lift, ~400-600 LOC plus runtime support.
Leaning toward (1) for the next iteration: it's the smaller bite
*and* the bigger expressivity unlock. (2) becomes acute only when
someone tries to run an unbounded loop, which the current examples
don't.
## 2026-05-07 — Iter 9 done: dogfood + `ail run`
Course-corrected from the Iter-8 plan. Polymorphism is the bigger
expressivity unlock on paper, but I hadn't actually proved that the
language was sufficient for "small but real" programs without it. So
Iter 9 became a dogfood iteration: write a non-trivial program
that exercises everything Iter 1-8 shipped, and use `ail run` /
errors / type-checker output as the user would. If something broke,
fix it. If nothing broke, document the boundary moved.
**`examples/list_map.ail.json`**:
```jsonc
type IntList = Nil | Cons Int IntList
map_int :: ((Int) -> Int, IntList) -> IntList
map_int(f, xs) = match xs {
Nil -> Nil
Cons(h, t) -> Cons(f(h), map_int(f, t))
}
print_list :: (IntList) -> Unit !IO
print_list(xs) = match xs {
Nil -> ()
Cons(h, t) -> let _ = do io/print_int(h) in print_list(t)
}
main = let xs = Cons 1 (Cons 2 (Cons 3 Nil)) in
print_list(map_int(\\x. x * 2, xs))
```
**Result:** nothing broke. Output `2\\n4\\n6\\n`, exit 0. The full
pipeline (`ail run`) covers: ADTs with two ctors of different
arity; pattern matching with nested `Var` fields; recursion over
ADT; closures (with no captures here, so env is null but the
closure-pair plumbing still gets exercised); fn-typed parameters in
a top-level def; `do io/...` inside a match arm body, with `let _`
to sequence two effectful operations; effect propagation through
the call chain. This validates Iter 1-8 as a self-contained
foundation.
**Friction surfaced:** writing the AST by hand is tedious — the
JSON for this 4-def module is 200+ lines. That's not surprising
(the format is for LLMs, not humans), but it suggests an Iter 10
priority: a richer pretty-print form, or an `ail snippet` helper
for common boilerplate (`mk_list_int`, etc.). Not blocking; noted.
**`ail run` (Iter 9b):** Builds into a tempdir + execs the binary,
exit code passthrough. Saves a `cd && ./bin` step in the dogfood
loop. Tiny addition — `Cmd::Build`'s body factored into a shared
`build_to` helper.
**Architecture self-check:**
- *Would I use this language now?* For self-contained Int-typed
programs over recursive ADTs: yes. The list_map example is what
I would have wanted to write since Iter 6 and bounced off
repeatedly. It now compiles and runs without me adapting the
source — the language is what its authors said it was, end to
end.
- *Did I think of everything?* Two cracks observed during the
dogfood:
- `(Int)` parens around single-param fn-types in pretty-print are
visual noise. Cosmetic, can wait.
- `let _ = do <effect> in <body>` is the only way to sequence
effects today. Working as intended given KISS, but a `;`
operator (sequencing) would be cheap polish.
- *Consistency:* DESIGN.md CLI block + smoke-test list updated.
Iter 8c invariant — "What is not (yet) supported" ≡ truth at end
of latest iteration — held; no new pending items.
- *KISS:* Iter 9 added 0 LOC of language semantics. All gain came
from validating the existing surface and a small CLI helper.
**Tests:** 50 green (was 49). New e2e
`list_map_doubles_then_prints`. No test for `ail run` itself —
`build_and_run` already exercises the equivalent path.
**Plan iteration 10:**
The dogfood revealed two real-but-not-blocking pain points and one
big architectural gap. Candidates, ranked:
1. **Polymorphic let-bindings with monomorphisation at codegen.**
Allows `let id = \\x. x in (id 1, id true)` and ultimately
`map :: (a -> b) -> List a -> List b`. The ground truth-ier
answer for the "would I use it for X?" question, but a
non-trivial pipeline change (typechecker→codegen needs to thread
instantiation info to the call site).
2. **Sequencing operator `;` and richer effect ergonomics.** A
`Term::Seq { lhs, rhs }` (or compile sugar to `Let { name: "_",
value: lhs, body: rhs }`) plus a small pretty-print update.
Cheap, satisfying.
3. **GC.** Heap reclamation for ADT boxes, lambda envs, closure
pairs. Real architecture step. Becomes acute the moment someone
writes a long-running loop; the current examples don't.
Tentative pick: (2) for the next sprint as a satisfying small
polish, then (1) as Iter 11. (3) bides its time until a real
program needs it.
## 2026-05-07 — Iter 10 done: Term::Seq sequencing
Followed the Iter 9 plan and shipped (2). New AST node
`Term::Seq { lhs, rhs }` with serde tag "seq". Semantics: evaluate
lhs (which must be Unit), discard the value, return rhs. Effects
from both sides accumulate.
This is sugar for `let _ = lhs in rhs`, but it's a first-class node
because:
- The pretty-print renders cleanly (`(seq lhs rhs)` instead of
borrowing the `let` form with a discard binding).
- Diagnostics are sharper: a non-Unit lhs gets a "type mismatch"
error pointing at the seq site, not "binding `_` had type X" at
a let site.
- Future tooling (effect inference visualisation, dataflow) can
treat sequencing as a structural concept instead of a special-
cased let.
Codegen is trivial: lower lhs (drop SSA), lower rhs (return).
Refactored `examples/list_map.ail.json`'s `print_list` to use seq
instead of `let _ = ...`. Output unchanged (`2\\n4\\n6\\n`); the
JSON shed a few lines and reads more honestly.
**Architecture self-check:**
- *Would I use this language now?* Same answer as Iter 9 (yes for
small but real programs), but the seq node makes IO-heavy
recursion read better — closer to "call this effect, then this
one" instead of "bind this effect to nothing, then this one".
- *Did I break anything?* Hash stability check: existing examples
without `Term::Seq` serialise identically; their fn hashes are
unchanged. `list_map.ail.json`'s hashes shifted as expected since
its body changed.
- *KISS:* +30 LOC across AST/pretty/check/codegen/walker. One
unit test for the lhs-must-be-Unit rule. The dogfood example
proves the e2e path.
**Tests:** 51 green (was 50). New `seq_lhs_must_be_unit` unit test
in ailang-check. Existing list_map e2e still passes after the
refactor.
**Plan iteration 11:**
Polymorphism, as queued in the Iter 9 plan. Concretely: HM-style
unification + let-generalisation in the typechecker, monomorph-
isation at codegen time. Touches the typechecker→codegen pipeline.
Bigger commit than the recent stretch, will probably need to be
phased (typechecker substitution machinery, then codegen
specialisation, then docs).
## 2026-05-07 — Iter 11 done: deeper dogfood (insertion sort)
Pulled back from polymorphism for one more validation cycle before
the architectural step. Polymorphism is a substantial pipeline
change (typechecker substitution + codegen monomorphisation) and I
wanted one more "small but real" program to confirm the existing
foundation holds before disturbing it.
`examples/sort.ail.json` — insertion sort over `IntList`:
```jsonc
insert :: Int -> IntList -> IntList
insert(y, xs) = match xs {
Nil -> [y]
Cons(h, t) -> if y <= h then Cons(y, Cons(h, t))
else Cons(h, insert(y, t))
}
sort :: IntList -> IntList
sort(xs) = match xs {
Nil -> Nil
Cons(h, t) -> insert(h, sort(t))
}
print_list :: IntList -> Unit !IO // uses Iter 10 seq
main = print_list(sort([3,1,4,1,5,9,2,6,5,3,5]))
```
**Result:** typechecks first try, runs first try, prints
`1 1 2 3 3 4 5 5 5 6 9` (each on its own line). 11-element input,
correct sorted output. The combination of recursive ADT pattern
match + comparison ops + branching + leaf recursion + IO
sequencing all worked end to end without the language tripping me
up. Iter 10's seq made `print_list` notably cleaner than the
`let _ = ...` form would have been.
**Architecture self-check:**
- *Would I use this language now?* For "small but real"
monomorphic programs over Int, Bool, Unit, Str, and ADTs of
those: confidently yes. Insertion sort writes out as the
textbook recursion, no bookkeeping that the language couldn't
do for me.
- *Did I think of everything?* The remaining wall is still
polymorphism. Sort over `IntList` needs hand-monomorphisation;
a generic `sort :: (a -> a -> Bool) -> List a -> List a` is
what the language eventually wants. No new architectural cracks
surfaced from this dogfood.
- *Visualisation:* `ail describe sort.ail.json sort` reads the
way I'd expect a sort definition to read, with `IntList` types
inline and the recursive call rendered cleanly.
- *KISS:* Iter 11 added 0 LOC of language semantics and 1 e2e
test. The 250-line JSON for the example is verbose but
mechanical — no friction once you accept that the JSON is the
surface for LLM authors.
**Tests:** 52 green (was 51). New e2e
`insertion_sort_orders_list`. Pure addition; existing tests
untouched.
**Plan iteration 12:**
Now polymorphism. Two more dogfood programs would just keep
producing the "the language is fine for monomorphic programs"
result, which is already established. The real expressivity
unlock — and the answer to "would I use it for X?" for X that
actually needs generic data — is HM inference + let-generalisation
+ monomorphisation. Phased plan:
12a. Typechecker: introduce a `Subst` (type variable substitution)
and unification. Thread through `synth`. At `let`, generalise
syntactic values (lambdas) — no value-restriction subtlety
needed yet, the MVP has no mutable refs.
12b. Codegen: at each polymorphic call site, the typechecker
records the instantiation. Codegen walks the AST a second
time per (def, instantiation) pair and emits a specialised
version with the type variables substituted by concrete
types in fn signatures.
12c. Docs + a polymorphic `id` test + a generic `map :: (a -> b)
-> List a -> List b` rewrite of `list_map.ail.json`.
## 2026-05-07 — Iter 12a/b done: polymorphism reaches the binary
Skipped 12c's "polymorphic map" — without parameterised ADTs (which
the MVP doesn't have), the rewrite would still be over a concrete
`IntList`, defeating the purpose. So 12c becomes lighter: docs +
two new examples (`poly_id`, `poly_apply`) that prove polymorphism
end-to-end on primitive types and on fn-typed parameters. The big
test is whether *I* would use the language now for a poly-flavoured
program; the answer below.
**12a — typechecker:**
`Type::Forall { vars, body }` is now legal at top-level fn types.
Implementation is the textbook ML rule: peel the Forall when
checking the body (rigid vars go into `Env.rigid_vars` so
`check_type_well_formed` accepts them), instantiate fresh metavars
at every var-resolution site, unify on every formerly-`expect_eq`
edge.
The metavar encoding sidesteps an AST schema change: a metavar is
just `Type::Var { name: "$m<id>" }`. The `$` prefix can't collide
with source identifiers, the JSON layout doesn't shift, and module
hashes stay bit-identical (verified: `sum.ail.json` keeps
`db33f57cb329935e` / `d9a916a0ed10a3d3`). I considered adding a
new `Type::Meta` variant under `#[serde(skip)]` but that would
have pulled hashing concerns into serde; the naming convention
keeps the AST untouched.
`Subst` is a flat `BTreeMap<u32, Type>`; `unify` is the standard
occurs-check version with effects compared as a set. Constants
still reject Forall outright; ADT fields still reject vars. No
let-generalisation: lambdas inside fn bodies are checked
monomorphically against their declared types — keeps the
implementation small and matches DESIGN.md's "top-level types
must always be explicitly annotated".
**12b — codegen:**
Direct calls to a polymorphic def get monomorphised on demand.
Each unique (def, instantiation) pair emits a specialised LLVM fn
with mangling `@ail_<m>_<def>__<descriptor>`. Descriptor scheme:
`Int → I`, `Bool → B`, `Unit → U`, `Str → S`, ADT `Foo → FFoo`,
`Fn(a)→b → Fn_<a>__r_<b>`. So `id(42)` and `id(true)` produce
`@ail_poly_id_id__I` and `@ail_poly_id_id__B` side by side.
Pass 1 of `lower_workspace` now splits fn-typed defs into mono
(`module_user_fns`, LLVM-typed FnSig as before) and poly
(`module_polymorphic_fns`, full FnDef). A unified
`module_def_ail_types` carries AILang types for both, used by
the codegen-side type tracker.
The hard part was getting AILang types at call sites. The
typechecker has them but doesn't hand its annotations down (no
TIR yet). I considered three paths:
1. Typechecker sidetable keyed by AST node ids — would need
to assign ids deterministically, brittle.
2. Uniform representation (everything passes as ptr/i64) —
contradicts CLAUDE.md's "performance is extremely important".
3. Codegen replays the type derivation locally.
Picked (3). The trade-off is duplication (`synth_arg_type`
mirrors what the typechecker already did), but it's contained
to a small recursive walk and uses the same `locals`/`extras`
shadowing pattern. Worth it for the MVP — once a TIR stage
materialises (it's still on the debt list), the duplication
collapses into a single pass.
`locals` grew from 3-tuple to 4-tuple `(name, ssa, llvm_type,
ail_type)`. Six push sites updated mechanically. Lambda capture
metadata grew the same way. `CtorRef` got `ail_fields` so match
arm bindings inherit the AILang type.
The drain phase iterates until `mono_queue` is empty —
specialised bodies can themselves invoke polymorphic defs and
queue further entries. `apply_subst_to_term` substitutes rigid
vars in `Term::Lam` annotations (the only Term arm carrying
types).
**Architecture self-check:**
- *Would I use this language now?* For monomorphic programs:
yes (already established). For polymorphism over primitives
and fn-typed parameters: yes — `id` and `apply` write out the
way the textbook says they should, with no language-level
bookkeeping leaking into the source. The `poly_apply` example
was particularly revealing: the closure-pair ABI (Iter 8a)
composes cleanly with monomorphisation. Specialised body of
`apply__I_I` keeps `f` as a fn-typed local; the existing
indirect-call path already handles the lower from there.
- *Did I think of everything?* No, two known gaps:
1. **Polymorphic fn passed as a value** (`let f = id in f(42)`)
fails in codegen — `resolve_top_level_fn` looks in
`module_user_fns` only. Adding this means emitting one
closure-pair global per instantiation, possibly via the
same drain pass. Defer.
2. **Higher-rank polymorphism** (`apply(id, 42)`) trips
`unify_for_subst` which doesn't handle Forall on the param
side. Real higher-rank polymorphism is a substantial step
and not on the near horizon — deferred to a later iter.
- *Visualisation:* `ail manifest poly_id.ail.json` now shows
`forall a. (a) -> a` correctly. The pretty-printer carried
`Type::Forall` rendering since Iter 1; nothing to do.
- *KISS:* +1 typechecker file edit (~430 LOC inserted, mostly
Subst+unify+four tests), +1 codegen extension (~600 LOC
inserted, mostly the drain path + helpers + locals widening).
Two new examples, two new e2e tests. Could be smaller if I
bit the bullet on TIR; not yet worth the upfront cost.
**Tests:** 58/58 (was 56/56). Added 4 typechecker unit tests in
12a, 2 e2e tests in 12b. Hash invariant holds.
**Plan iteration 13 (queued, not started):**
The natural next step depends on what I want to use the language
for. Two candidates, in order of expected payoff:
13a. **Parameterised ADTs** — `List a`, `Maybe a`, etc. Without
these, polymorphism is half-useful: a generic `map` still
can't transform an `IntList` into a `BoolList`. ADT defs
would gain a `vars: Vec<String>` field; ctor field types
could mention them; codegen monomorphises ADT instances
just like fns. This is the bigger expressivity unlock.
13b. **GC or arena** — every ADT box, lambda env, and closure
pair currently leaks. For sort over an 11-element list,
fine. For anything longer-running, required. The current
lifetime model is "leak"; the right MVP is probably
bumpalloc per top-level fn invocation. Could be done
before parameterised ADTs but doesn't unlock new examples.
Leaning 13a — it's the more interesting architectural step and
makes the "polymorphic map" rewrite from the original 12c plan
finally meaningful.
## 2026-05-07 — Iter 13 done: parameterised ADTs reach the binary
**Why now.** End of Iter 12 left polymorphism half-useful: `id`
and `apply` worked, but every container was monomorphic
(`IntList`, `Maybe_Int`). A generic `map :: forall a b. ((a) -> b,
List a) -> List b` was unwritable. 13 lifts that.
**Three commits:**
- `0782622` 13a — schema (`TypeDef.vars`, `Type::Con.args`) +
checker (substitution at ctor + match + arity validation in
`check_fn`).
- `1631f60` 13b — codegen: per-use-site substitution of LLVM
field types in `lower_ctor` and `lower_match`. No mono-queue
for types — ctor code was already inlined at every use site,
so 13b only had to thread substitution through, not invent a
symbol scheme. `synth_arg_type` for `Term::Ctor` now returns
concrete type-args, and `llvm_type(Type::Var)` is a hard
error instead of a silent `ptr` fallback (the latter was
flagged by the architect review and is the most defensive
single change in 13).
- `<this>` 13c — DESIGN.md flipped (parameterised ADTs out of
the gap list, into the supported list); two new example lines.
**Hash invariant.** Both new fields are
`#[serde(default, skip_serializing_if = "Vec::is_empty")]`. A new
regression test in `crates/ailang-core/src/hash.rs` deserialises
the actual `examples/sum.ail.json` and `examples/list.ail.json`
from disk and asserts `db33f57cb329935e` and `b082192bd0c99202` —
the recorded pre-13a hashes. It's deliberately phrased against
the on-disk JSON rather than reconstructed code, so the test
fails if anyone resaves the examples in a way that drifts the
canonical bytes.
**Architect-flagged debt I deliberately did NOT touch in 13:**
- `is_static_callee` returns true for poly fns but
`resolve_top_level_fn` only consults `module_user_fns`. A
poly fn used as a value (`let f = id in f(42)`) passes the
static check then surfaces as `UnknownVar`. Would need one
closure-pair global per instantiation. Out of 13 scope; same
hole that was queued at the end of Iter 12.
- Triple source of truth for builtins (`builtins::install`,
`builtins::list`, `codegen::builtin_ail_type` /
`builtin_effect_op_ret`). Every new operator costs three
edits. Low interest today, escalates with every effect op.
Worth a future tidy iter — not blocking expressivity.
- `synth_arg_type` for `Term::If` returns `synth(then)` only;
for `Term::Match`, the first arm. Masked today by the
typechecker having already unified, but it's the kind of
duplication that decays. Same fundamental cost as the absence
of a TIR.
**Architecture self-check.**
- *Would I use this language now?* For polymorphism over
primitives, fn-typed values, AND parameterised containers —
yes. The `box.ail.json` and `maybe_int.ail.json` examples
read like the textbook says they should. No type-arg
bookkeeping leaks into the source.
- *KISS.* 13b was much smaller than I feared at the start of
the design phase: ~150 LOC in codegen, no new structures, no
mono-queue. The reason: ADT ctor code is already inlined.
The architect's recommendation to *not* mutate `ctor_index`
but derive `CtorRef` per use site was the right call —
preserved the static template, made the substitution local.
- *Did I think of everything?* Two known gaps remain. **(1)**
Polymorphic ADTs as the type-arg of a polymorphic fn —
works today because `unify_for_subst` recurses through
`Type::Con.args` (added in 13a). **(2)** A polymorphic fn
taking a polymorphic ADT and returning a different
parameterised ADT (`map : forall a b. ((a)->b, List a) -> List
b`) — should also work, but I haven't dogfooded it yet
because `List a`-as-a-rewrite-of-`list_map` would need the
schema bumps elsewhere (paramaterised list builder). Queued
for Iter 14.
- *Visualisation.* `ail manifest examples/box.ail.json` shows
`type Box :: forall a. MkBox(a)` and
`fn unbox :: forall a. (Box<a>) -> a`. The pretty-printer
picked up `args` and `vars` cleanly (Iter 13a).
**Tests:** 64/64 (was 58/58). Added: 1 hash-stability regression
(13a), 3 checker unit tests for parameterised ADTs (13a),
2 e2e tests over `box.ail.json` and `maybe_int.ail.json` (13b).
**Process note (orchestration).** First iter where I worked
strictly through the agents in `/agents/`: `ailang-architect`
ran a drift review on HEAD before 13b started; `ailang-implementer`
got a fixed brief that incorporated the architect's three
recommendations (don't mutate `ctor_index`, fix `synth_arg_type`
for `Term::Ctor`, harden `llvm_type`); 13c (this) is the
orchestrator's own work. The role split landed in `3df943d` after
I caught myself doing implementer work on 13a directly. The
agents pay off in proportion to iter size — for 13b they were
clearly worth the round-trip; for 13a's checker work, marginal.
**Plan iteration 14 (queued, not started):**
Two candidates, in order of expected payoff:
14a. **Polymorphic `List a` rewrite of `list_map`.** Replaces
`IntList` with `List a`, rewrites `list_map` to return
`List b`, and lets the polymorphic-map version be the
dogfood smoke test. Pure exercise — should fall out of
13b — but worth the dogfood beat. Also: `Maybe a` used
in a non-trivial fn (e.g. `find : forall a. ((a) -> Bool,
List a) -> Maybe a`).
14b. **GC or arena.** Same pitch as before: every ADT box,
lambda env, closure pair leaks. For `box.ail.json` and
`maybe_int.ail.json`, fine. For anything that allocates
in a loop, required. Bumpalloc per top-level fn
invocation is the natural MVP.
14c. **Poly fn as value.** Closes the asymmetry the architect
flagged; gates `let f = id in f(42)`. One closure-pair
global per instantiation, emitted via the same mono-queue
drain path. Smaller surface than 14a/b.
Leaning 14a — the dogfood payoff for one iter of polish is
high, and `Maybe`-in-a-real-fn is a missing piece I haven't
exercised yet. 14b stays second; 14c is a candidate if I want
a small palate cleanser.
---
## Iter 13d — rustdoc polish for `ailang-core` + new `ailang-docwriter` agent
User triggered: ran `cargo doc --open` for fun and reported
that the rendered docs were thin — crate headers existed,
but `pub` items had no `///` strings, there were no `# Examples`
sections, and intra-doc links were missing. Internal references
showed up as prose ("see Builtins") rather than as clickable
links. Two stale `[Builtins]` and `[code]` warnings had been
bleeding into every `cargo doc` invocation since Iter 6 or so.
Recurring task → new agent. Wrote `agents/ailang-docwriter.md`
with a tight mandate: rustdoc only, no API changes, no edits in
`docs/` or `agents/`, three verification gates (rustdoc clean,
build green, tests green incl. doctests). Updated
`agents/README.md`. Updated DESIGN.md item 6 of "Verification
and correctness" to make rustdoc cleanliness a project-wide
invariant rather than an iter-local cleanup.
First mission: `ailang-core` only. The foundation crate every
other crate depends on — biggest reader-leverage per diff. The
agent rewrote the crate root so a newcomer learns: what `core`
owns, where it sits in the pipeline (`core` → `check` →
`codegen` → `ail`), the central invariant (canonical JSON is
deterministic, hashes content-addressed, schema = `ailang/v0`),
and the entry points. Module roots in `ast.rs`, `canonical.rs`,
`hash.rs`, `pretty.rs`, `workspace.rs` got the same treatment.
Every `pub` struct / enum / variant / fn / const got a `///`
string. The Iter-13a additions (`TypeDef.vars`, `Type::Con.args`)
got an explicit backwards-compat note that points back to the
hash-stability regression test.
`# Examples` blocks landed where they shorten understanding
(`canonical::to_bytes`, `def_hash`, `Workspace`), all marked
`ignore` so the workspace doctest run stays cheap (3 ignored,
0 run, 0 failed). The two stale broken-link warnings in
`ailang-check` got prose-only fixes — out-of-scope for this
iter conceptually, but a one-line fix per file means rustdoc
is now globally clean.
**Findings reported by the docwriter** (judgement deferred to
me; none made it into the diff):
- `Term::Lam.param_tys` (JSON: `paramTypes`) is positionally
paired with `Term::Lam.params: Vec<String>`. Naming hints
at the convention but a cold reader has to deduce it. **Not
fixing**: renaming would touch the schema, breaks every hash.
The `///` string makes the convention explicit, which is
enough.
- `Type::Var` is overloaded: source-level rigid vars and
checker metavars (`$m<id>`) share the same variant. A reader
of `core` alone sees no hint of the metavar half — it's
documented in `ailang-check`'s lib doc instead. **Not fixing**:
splitting the variant would balloon the schema and
invalidate every hash. Acceptable as long as `check`'s lib
doc explains it (it does, post-13d).
- `Def::Type(TypeDef)` versus the type-expression enum `Type`
in the same module: name collision is real but unavoidable
without renaming `Type` (which would touch every crate). The
`///` strings now disambiguate at point of contact.
**Process note (orchestration).** Second iter where I worked
strictly through agents (after 13b). The docwriter brief was
written from the diagnostic in this conversation, not from a
DESIGN-doc design pass — there was no architecture decision to
make, just a discipline gap to close. That's the right shape
for a docwriter: low-judgement, repeatable, runs after every
iter that touches public surface. Cost ≈ 25 tool uses for ~390
LOC of doc additions across 9 files; smaller-grain than
`ailang-implementer` runs typically are.
**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests
(new). `cargo doc --no-deps`: 0 warnings (was 2). `cargo build
--workspace` green. `cargo test --workspace` green.
**Plan iteration 13e/13f (queued, not started).** Two natural
follow-ups for the docwriter:
13e. **`ailang-check` rustdoc**: type-checker is the next
biggest crate by `pub`-surface and the most algorithmically
dense. Rigid/metavar split, `Forall` instantiation, the
match-arm exhaustiveness logic, the Iter-13a substitution
machinery — all of it benefits more from prose explanation
than `core` did.
13f. **`ailang-codegen` + `ail` CLI rustdoc**: codegen is dense
but mechanical (mangling, ABI, block tracking); the CLI
is mostly clap derive macros. Lower payoff per LOC of doc
than 13e but rounds out the warning-free invariant across
the whole workspace.
After 13d there's no urgency on 13e/f — `cargo doc` is already
warning-free. They're "polish iterations to be slotted in
between feature iters when context budget is short". 14a
(`List a` rewrite) remains the next-feature default.
## Iter 13e — rustdoc polish for `ailang-check`
Second docwriter mission. Same mandate as 13d, applied to the
typechecker crate. `lib.rs` (1922 LOC) had a strong crate root
already — covers HM-with-effects, top-level forall, the
rigid-vs-metavar split, the `$m<id>` encoding — but its `pub`
surface (the `Env` struct, `CheckError` + 24 variants,
`CheckedModule`, `CtorRef`, the `check_module` entry point) was
mostly undocumented. `builtins.rs` had a one-line module
header and zero `///` strings on `EffectOpSig` or `install`.
`diagnostic.rs` had a strong module header (lists every stable
diagnostic code) but `Diagnostic`, `Severity`, and the
construction helpers were undocumented.
Agent added 188 LOC of pure rustdoc across the three files; no
non-doc lines changed (verified by filtering the diff). Each
`CheckError` variant now carries the AST term/type that
triggers it and the stable kebab-case `code` it maps to. `Env`
fields (`globals`, `effect_ops`, `types`, `module_globals`,
`current_module`) got individual `///` strings that name the
invariants — most importantly that `module_globals` includes
the current module, which the agent flagged as undocumented at
field level (now fixed). Crate-root prose got an upgraded
intra-doc link to `[`check_module`]`; `super::` reference in
`diagnostic.rs` rewritten to `crate::` (cosmetic, but the
canonical form).
**Findings reported** (judgement deferred to me):
- `Env` is `pub` with all-`pub` fields but `Env::new` is
private and there's no public builder — external callers can
only construct via field-by-field literal, which is fragile
if a field is added later. **Not fixing**: changing this is
an API decision, not a doc one. Worth raising next time we
touch the crate's public surface deliberately.
- `CheckError::CtorArity` and `CheckError::ArityMismatch` both
serialize to the public diagnostic code `arity-mismatch`. The
`///` strings now flag the collision per-variant; tooling
that consumes the diagnostic JSON sees only the merged code
and that's intentional from the iter-5b vintage. **Not
fixing.**
- `Diagnostic` and `Severity` are reachable both via the crate
re-export and via `crate::diagnostic::*` because the
`diagnostic` module is itself `pub`. Rustdoc renders both
pages; harmless but slightly noisy. **Not fixing**: the
re-export is the documented entry point and we don't want to
hide the module.
**Process note.** Same shape as 13d: I wrote a brief naming
the deficiencies (numbers of pub items, which files had thin
roots), the agent did the doc additions inside its mandate,
the orchestrator-side work was authoring DESIGN/JOURNAL and
verifying the diff. Cost ≈ 32 tool uses for 188 LOC of doc
across 3 files — denser than 13d (more sentences per pub item
because the typechecker invariants need explicit
articulation), but the per-LOC payoff for a future reader is
also higher.
**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests
(unchanged). `cargo doc --no-deps`: 0 warnings (was 0; the
agent introduced 4 transient broken-link warnings during the
work and resolved all of them before reporting done). `cargo
build --workspace` green. `cargo test --workspace` green.
**Plan 13f / 14a unchanged.** 13f (`ailang-codegen` + `ail`
CLI) is the natural next polish iter; 14a (`List a` rewrite of
`list_map`) remains the next-feature default. Auto-mode is on,
so I'll continue into 13f directly unless context budget
pressures a switch.
## Iter 13f — rustdoc polish for `ailang-codegen` + `ail` CLI
Combined docwriter mission. Codegen has a small public
surface (only 3 top-level `pub` items: `CodegenError` enum +
7 variants, `emit_ir`, `lower_workspace`); the CLI is a
binary with zero `pub` items, so the only useful rustdoc is
the module header. One agent run, both files.
What landed (148 LOC of doc additions, no non-doc lines
changed, verified by filtering the diff):
- `ailang-codegen` crate root got intra-doc-link upgrades
(`[`emit_ir`]`, `[`lower_workspace`]`,
`[`CodegenError::MissingEntryMain`]`) and a precondition
sentence — both entry points assume their input has already
passed the typechecker; codegen does not call the checker
itself.
- Every `CodegenError` variant got a `///` string naming the
AST term/condition that triggers it. Same shape as
`CheckError` post-13e, so the two error enums now read
similarly and a reader can grep across them. `Internal` is
flagged in its doc as a catch-all that covers ~30
invariant-violation sites.
- `emit_ir` and `lower_workspace` now make the
single-vs-multi-module split explicit and cross-link to
each other.
- `ail/main.rs` module header expanded from a 5-line stub to
a full subcommand list with one-liners (`manifest`,
`render`, `describe`, `deps`, `check`, `emit-ir`, `build`,
`run`, `builtins`, `diff`, `workspace`), the `clang`-on-PATH
prerequisite for `build`/`run`, the design-intent paragraph
about each subcommand being narrowly scoped for LLM
consumption (already partly there), and an explicit "no
`pub` items, `--help` text comes from clap" note for anyone
who lands here from rustdoc.
**Findings reported** (judgement deferred to me):
- `CodegenError::Internal(String)` is a single opaque catch-all
for ~30 distinct invariant-violation sites (mono-queue
desync, ctor-index miss, lambda-env shape, ...). Tests can
only substring-match on it. **Not fixing**: splitting is a
test-ergonomics decision, not a doc one. Worth raising the
next time codegen tests get a serious rewrite.
- `emit_ir` synthesises an internal `Workspace` with
`root_dir = "."`. No codegen path reads `root_dir` today, so
this is harmless; if a future feature reaches `root_dir`
from codegen, the assumption surfaces. **Not fixing**: the
agent flagged it correctly as "would change behaviour, out
of scope".
**Process note: brief drift caught by the agent.** I told the
docwriter the CLI had nine subcommands; it found eleven
(`Deps` and `Diff` were missing from my brief, which was
written off a `head -40` of the source). Agent silently
corrected and flagged the drift in its findings. Useful
counter-pressure to the orchestrator pattern: my survey was
sloppy and the agent did not propagate the sloppiness into
the doc. This is one of the things sub-agents are good at and
why I keep delegating even on small jobs.
**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests
(unchanged). `cargo doc --no-deps`: 0 warnings. Also verified
under `RUSTDOCFLAGS='-D rustdoc::broken_intra_doc_links'` per
agent report. `cargo build --workspace` green. `cargo test
--workspace` green.
**Workspace-wide rustdoc invariant achieved.** All four
crates (`ailang-core`, `ailang-check`, `ailang-codegen`,
`ail`) now have:
- crate-root `//!` that names ownership, position in
pipeline, and entry points;
- module-root `//!` on every file with non-trivial content;
- `///` on every public item (struct, enum, variant, fn,
field, const) that names the contract, not just the type;
- intra-doc links wherever prose previously referred to
another item by name.
DESIGN.md item 6 ("rustdoc cleanliness") is now load-bearing
across the whole workspace, not just `core`. The
`ailang-docwriter` agent's job from here is **maintenance**:
run after any iter that adds public surface, not full sweeps.
**Next.** 14a is now unblocked: write a polymorphic `list_map`
that uses `List a` (Iter-13a parameterised ADTs) and `Maybe a`,
then extend an existing demo program (`hello_print` or one of
the dogfood sources) to call it end-to-end. That exercises
the parameterised-ADT pipeline through type-check, codegen,
and runtime — the missing piece in 13a/b/c was that the
feature shipped but no real program used it. If context
budget at the start of 14a is tight, alternative is 14b
(GC/arena scaffolding) or 14c (poly fn as value); both have
real design questions that need an orchestrator design pass
first, so 14a stays the default.