Brummel
ca30606aec
Adds path-2 from the 16b.2 planning entry. Desugar now defers
LetRecs whose captures include Term::Let-bound names; a new
ailang-check::lift_letrecs pass runs after typecheck and uses the
elaborated env to resolve capture types. ailang-check learns a real
Term::LetRec typing rule in synth and verify_tail_positions.
- desugar: Term::LetRec arm gains a defer-arm; helpers promoted to
pub for reuse by the lift pass; find_non_callee_use moved before
classification.
- ailang-check::synth/verify_tail_positions: real LetRec rules
(effect-subset, locals install, recursive name in body+in_term).
- ailang-check::lift.rs (new, 720 LOC): post-typecheck lift with
post-order traversal, env-walk for capture-type resolution,
fast-path skip when no LetRec is present.
- ail::main.rs: build path now does load → check → desugar →
lift_letrecs → codegen.
- examples/local_rec_let_capture.{ailx,ail.json}: new fixture
capturing a let-bound `threshold` in a recursive helper.
- e2e + check unit tests: 106 → 110 (+4).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3890 lines
180 KiB
Markdown
3890 lines
180 KiB
Markdown
# JOURNAL
|
||
|
||
Chronological notes for myself. Not every change; only decisions, obstacles,
|
||
and observations that future iterations will need.
|
||
|
||
## 2026-05-07 — Day 0
|
||
|
||
- Repo initialised. Assignment in `CLAUDE.md`: LLM-native language, LLVM backend.
|
||
- Design decisions captured in `docs/DESIGN.md`.
|
||
- Toolchain: `rustc 1.94`, `llvm-config 22.1.3`, `clang` available.
|
||
- Decided against `inkwell` in favour of LLVM IR text emit. Rationale in DESIGN.md.
|
||
- Workspace layout:
|
||
- `crates/ailang-core` — AST, type, hash, JSON schema
|
||
- `crates/ailang-check` — typechecker (comes later)
|
||
- `crates/ailang-codegen` — lowering + LLVM IR emit
|
||
- `crates/ail` — CLI
|
||
- MVP goal: `examples/sum.ail.json` → binary that prints 55. **Achieved.**
|
||
|
||
## 2026-05-07 — architecture review after the MVP
|
||
|
||
Still on track? Broadly yes. Concrete observations:
|
||
|
||
**What holds:**
|
||
|
||
- JSON AST + canonical form + content hash are all lego bricks that later
|
||
tools can build on without a refactor (`ail deps`, `ail diff`).
|
||
- The LLVM IR text pipeline works as planned. No libllvm version pain.
|
||
- The effect set is wired into the type system from the start. Extensible to
|
||
row-poly without touching the core.
|
||
|
||
**Debt that accrues interest:**
|
||
|
||
1. **`current_block_label_for_phi` is a heuristic** (see codegen). On nested
|
||
`if` terms it will return the wrong block label, because it scans the body
|
||
backwards. Ticking, because no test cases trigger it yet. Must be fixed
|
||
next, before new language features arrive.
|
||
2. **No typed AST.** Codegen reads the source AST directly and relies on
|
||
the typechecker having run before. Fine for the MVP; once ADTs or
|
||
closures arrive, I will need a separate typed IR stage (TIR).
|
||
3. **The `hash` field is not in the AST.** Right now we hash the def object
|
||
directly. Once I serialise hashes as fields (caching), the hash will
|
||
need to exclude that field before computation.
|
||
|
||
**Plan iteration 2 (now):**
|
||
|
||
1. Clean up block-label tracking, with a nested-if test.
|
||
2. Strings as a literal + `io/print_str`.
|
||
3. Hello-world example as a second E2E test.
|
||
4. CLI: `--json` output for machine consumers wherever it fits.
|
||
|
||
**Plan iteration 3:**
|
||
|
||
ADTs + pattern matching. That is the next big jump. Requires a typed IR
|
||
stage (TIR), because pattern matching lowers into decision trees, which
|
||
have a different shape from the AST.
|
||
|
||
## 2026-05-07 — iteration 2 done
|
||
|
||
- Block-label tracking is now robust (nested `if`s work). Test
|
||
`max3_picks_largest` protects it.
|
||
- Strings as `Lit::Str { value }`, type `Str` -> LLVM `ptr`, with
|
||
`io/print_str` effect op. `examples/hello.ail.json` prints a string.
|
||
- CLI: `manifest --json`, `builtins --json` for tool consumers.
|
||
- `ail deps [--of NAME] [--json]` lists call edges. Effect ops are tagged
|
||
`effect:NAME` so a consumer can filter them.
|
||
|
||
**Architecture check:** no structural deviations. Codegen still reads the
|
||
source AST directly (a TIR stage will become necessary with ADTs in
|
||
iteration 3).
|
||
|
||
## 2026-05-07 — iteration 3 done: ADTs
|
||
|
||
- TypeDef in the AST with ctors. A ctor has `name` and `fields: [Type...]`.
|
||
- Term::Ctor (construction) and Term::Match (pattern matching).
|
||
- Patterns: `Wild`, `Var`, `Lit`, `Ctor { ctor, fields }`. In the MVP,
|
||
nested ctor patterns are NOT allowed — sub-patterns must be `Var` or
|
||
`Wild`.
|
||
- Typechecker with a type registry and `ctor_index` (ctor name → ADT). In
|
||
Match, exhaustiveness is checked against the full constructor set. A
|
||
negative test protects this.
|
||
- Codegen: boxed heap layout. Per ctor application, `malloc(8 + 8*n)`
|
||
bytes; tag at offset 0, fields from offset 8 (8-byte slots, native typed
|
||
load/store). Match: load tag + switch + arm blocks + phi at the join.
|
||
- `examples/list.ail.json` (Cons/Nil list, sum_list via match) returns 42.
|
||
|
||
**Surprisingly painless.** The architecture decisions from day 0 paid off:
|
||
opaque ptr in LLVM 22 makes the boxed layout almost glue-free; effect
|
||
tracking was untouched by ADTs; the JSON AST takes new node types
|
||
cleanly.
|
||
|
||
**Debt accrued:**
|
||
|
||
1. **Codegen still reads the source AST directly.** The temptation to push
|
||
on without TIR was strong — and worked, because my Match restrictions
|
||
are flat (no nested patterns). Once nested patterns arrive, decision-
|
||
tree lowering will not stay clean without TIR. Debt acknowledged; not
|
||
due now.
|
||
2. **No GC.** The heap leaks. Acceptable for demo programs; must be
|
||
addressed before any longer-running program. Options for Phase 4:
|
||
refcount, Boehm-GC linkage, region inference.
|
||
3. **No runtime pretty-printer for ADT values.** `io/print_int` is enough
|
||
for demos, but a generic `show :: a -> Str` for ADTs would be valuable.
|
||
Requires dispatch over the tag — feasible, but not now.
|
||
|
||
**Plan iteration 4:**
|
||
|
||
The next steps are less obvious. Three candidates in priority order:
|
||
|
||
1. **Module system (imports).** Right now everything is in a single module.
|
||
With multiple modules + cross-module hashing the language only becomes
|
||
practical for several defs.
|
||
2. **Structured error output (`ail check --json`).** So tools can react to
|
||
type errors without parsing text.
|
||
3. **Closures / higher-order functions.** Requires closure conversion and
|
||
is a bigger step.
|
||
|
||
Iteration 4 will be (1) + (2) — both strengthen the LLM tooling and have
|
||
moderate risk.
|
||
|
||
## 2026-05-07 — workflow change: orchestrator + agent repo
|
||
|
||
At the user's suggestion, switching to **orchestrator mode**: I delegate
|
||
clearly bounded implementation chunks to sub-agents and keep only
|
||
architecture decisions, reviews, and commit discipline. Four specialised
|
||
agents drafted: implementer, architect, tester, debugger.
|
||
|
||
**Important correction:** the user required the agents not to be hidden in
|
||
`.claude/agents/`, but versioned as a visible part of the project under
|
||
`agents/`. DESIGN.md gained a new section "Project ecosystem", which
|
||
records this: AILang is not just a language, but language core + CLI +
|
||
examples + agents + docs + tests, all of equal weight.
|
||
|
||
Invocation scheme: the system-prompt body from `agents/<name>.md` as a
|
||
prefix before the concrete task + sent to the `general-purpose` agent.
|
||
Functionally identical to subagent loading from `.claude/agents/`, but
|
||
visible in the repo.
|
||
|
||
**Plan iteration 4 (revised):**
|
||
|
||
The module system is more involved than expected (cross-module hashing,
|
||
import resolution). First the smaller tooling wins, then the module system
|
||
as iteration 5:
|
||
|
||
1. **Structured error output** (`ail check --json` with a Diagnostic
|
||
struct, stable codes like `unbound-var`, `type-mismatch`).
|
||
2. **`ail diff <a> <b>`** — semantic module diff via per-def hash
|
||
comparison.
|
||
3. **IR snapshot tests** — regression protection for the codegen pipeline.
|
||
|
||
## 2026-05-07 — iteration 4 done: LLM tooling consolidation
|
||
|
||
Three sub-commits, each produced by an `ailang-implementer` invocation
|
||
and spot-checked by the orchestrator:
|
||
|
||
- `93fe723` Iter 4a: `ail check --json` with a `Diagnostic` struct
|
||
(`severity`, `code`, `message`, `def`, `ctx`). Stable codes:
|
||
`unbound-var`, `type-mismatch`, `arity-mismatch`,
|
||
`non-exhaustive-match`, `unknown-ctor`, `unknown-ctor-in-pattern`,
|
||
`nested-ctor-pattern-not-allowed`, `duplicate-def`,
|
||
`unknown-effect-op`, `unknown-type`, `schema-mismatch`. API:
|
||
`check_module(&Module) -> Vec<Diagnostic>`.
|
||
- `c652b12` Iter 4b: `ail diff <a> <b> [--json]` as a structural top-level
|
||
def diff via BLAKE3 hash. Four categories (added/removed/changed/
|
||
unchanged), sorted alphabetically, exit code 1 on diff.
|
||
- `74a2005` Iter 4c: IR snapshot tests in
|
||
`crates/ail/tests/snapshots/{sum,max3,hello,list}.ll`.
|
||
Normalisation of `target triple`. Update via `UPDATE_SNAPSHOTS=1
|
||
cargo test ir_snapshot_`. Mismatch produces an `.actual` file.
|
||
|
||
Test count: 28 (previously 19). 7 E2E + 4 IR snapshot + 9 ailang-check + 1
|
||
ailang-codegen + 7 ailang-core.
|
||
|
||
**Closed from the debt register:**
|
||
|
||
- Block tracking in codegen has not been a heuristic risk since Iter 2;
|
||
the explicit `current_block: String` track is now additionally
|
||
protected against regression by Iter 4c snapshot tests. Debt closed.
|
||
|
||
**New / sharpened debt:**
|
||
|
||
1. `check_module` is **single-shot** — the first error aborts, no
|
||
multi-diagnostic gathering. The spec was that way, but the format
|
||
suggests Vec semantics. A real multi-diagnostic refactor will be
|
||
cheaper once TIR exists (a central error accumulator via a separate
|
||
stage). Not due now.
|
||
2. `source_filename` in the IR is hard-coded to `"<module>.ail"`. As long
|
||
as there is only one top-level module, that is platform-stable. With
|
||
Iter 5 (module system + imports) the path becomes relevant — keep it
|
||
path-independent at construction time, otherwise the snapshots will
|
||
tip over.
|
||
|
||
**Plan iteration 5:** module system with imports. Cross-module hashing,
|
||
import resolution, multiple `.ail.json` files in one build. The multi-
|
||
diagnostic refactor only after that.
|
||
|
||
Sub-steps:
|
||
|
||
- **5a — workspace loader.** `ailang_core::Workspace { modules:
|
||
BTreeMap<String, Module> }` plus `load_workspace(entry: &Path)`, which
|
||
follows `imports` recursively from the entry module. Convention:
|
||
`import { module: "foo" }` resolves to `<dir>/foo.ail.json` next to the
|
||
entry. Cycle detection. CLI: existing subcommands keep working on a
|
||
single module; a new `ail workspace <entry>` lists all reachable
|
||
modules with hash. Tests: two small example modules with an import
|
||
relation; cycle test.
|
||
- **5b — cross-module typecheck.** The typechecker takes `&Workspace`
|
||
instead of `&Module`. Imports are mounted in the env as a namespace
|
||
(`alias.def` or, with no alias, `module.def`). New diagnostic codes:
|
||
`unknown-module`, `unknown-import`, `import-cycle`, `ambiguous-name`.
|
||
Tests per code.
|
||
- **5c — cross-module codegen.** The emitter produces IR for all modules
|
||
in the workspace, prefix-mangled with `@ail_<module>_<def>`. E2E test:
|
||
a program that uses a function from module B in module A returns the
|
||
correct result in the binary.
|
||
- **5d — tooling adjustments.** `manifest`, `describe`, `deps`, `diff`
|
||
gain a `--workspace` mode (recursive). The single mode stays the
|
||
default for backwards compatibility.
|
||
|
||
During Iter 5, at construction time **keep `source_filename`
|
||
path-independent** (module name only, no directory prefix), otherwise
|
||
the IR snapshots will tip over.
|
||
|
||
## 2026-05-07 — Iter 5b done: cross-module typecheck
|
||
|
||
- `check_workspace(&Workspace) -> Vec<Diagnostic>` as the top-level API.
|
||
`check_module` is preserved and internally lifts the module into a
|
||
trivial workspace.
|
||
- Convention for qualified references (recorded in DESIGN.md):
|
||
`Term::Var { name }` with exactly one dot = `<prefix>.<def>`. Prefix is
|
||
an import alias or module name. No new AST node, no renamed fields ⇒
|
||
hashes stay stable; all `ir_snapshot_*` still green.
|
||
- Three new diagnostic codes: `unknown-module`, `unknown-import`,
|
||
`invalid-def-name` (with `ctx.reason: "contains-dot"`).
|
||
- CLI: `ail check <entry>` now **always** loads via `load_workspace`.
|
||
Workspace load failures become structured diagnostics in JSON mode with
|
||
codes `module-not-found`, `module-cycle`, `module-name-mismatch`,
|
||
`module-hash-mismatch`, `schema-mismatch`. `ail build` and
|
||
`ail emit-ir` stay per single module (cross-module codegen is 5c).
|
||
- Examples: `ws_main.ail.json` now calls `ws_lib.add` (observable). New:
|
||
`ws_broken.ail.json` (`unknown-import`),
|
||
`ws_unknown_module.ail.json` (`unknown-module`).
|
||
- Tests: 37 green (previously 32). 4 new workspace integration tests in
|
||
`crates/ailang-check/tests/workspace.rs`, one new e2e test
|
||
`check_workspace_resolves_import`.
|
||
- Debt: single-shot diagnostics still in place (multi-diagnostic after
|
||
5c). The dot convention covers exactly one dot — nested module paths
|
||
(`a.b.c`) do not exist; that would only be a topic with hierarchical
|
||
modules and currently falls through as `unbound-var`.
|
||
|
||
## 2026-05-07 — Iter 5c done: cross-module codegen
|
||
|
||
- **Mangling break (deliberate).** All AILang functions are now called
|
||
`@ail_<module>_<def>`, even in single-module programs. The old form
|
||
`@ail_<def>` is gone. Strings/const globals analogously
|
||
(`@.str_<module>_<hint>_<idx>`, `@ail_<module>_<const>`). The entry
|
||
point stays `main` as C ABI: a `define i32 @main()` trampoline calls
|
||
`@ail_<entry-module>_main()`. If the entry module has no
|
||
`main : () -> Unit !IO`, the build fails with `MissingEntryMain`.
|
||
- **Workspace lowering.** New top-level API
|
||
`ailang_codegen::lower_workspace(ws: &Workspace) -> Result<String>`
|
||
produces a single `.ll` for the whole workspace. Modules in
|
||
alphabetical order (BTreeMap order); defs in AST order. Cross-module
|
||
calls are resolved in codegen via the import map of the calling module
|
||
— same logic as in the typechecker, locally duplicated with a
|
||
cross-reference (no shared helper module, because the type worlds
|
||
differ: the typechecker handles `Type`, codegen handles `FnSig` from
|
||
llvm types).
|
||
- **CLI.** `ail build` and `ail emit-ir` now always load the workspace
|
||
and check/lower it fully. Single-module programs keep working
|
||
(trivial workspace with one module). `emit_ir(m)` stays in the codegen
|
||
crate as a convenience API and internally wraps into a trivial
|
||
workspace.
|
||
- **Snapshots regenerated.** `sum.ll`, `max3.ll`, `hello.ll`, `list.ll`
|
||
show the new mangling. New `ws_main.ll` snapshot documents the
|
||
cross-module build: `@ail_ws_main_main` calls `@ail_ws_lib_add`.
|
||
- **Tests.** 40 green (previously 37). New: `workspace_build_runs_imported_fn`
|
||
(e2e: prints 5), `ir_snapshot_ws_main`, `missing_entry_main_is_error`
|
||
(codegen unit). Existing behaviour tests
|
||
(`sum_1_to_10_is_55`, `max3_picks_largest`, `hello_world_str_lit`,
|
||
`list_sum_via_match`) stay green — behaviour unchanged, only the
|
||
mangling is new.
|
||
|
||
**Debt closed:**
|
||
|
||
- **#19 (`source_filename` hardening).** In the workspace world,
|
||
`source_filename` is now uniformly `<entry-module>.ail`, once per
|
||
workspace. The previous hard-coded path dot is gone with it.
|
||
|
||
**State:** the module system is closed end to end — loader + typecheck +
|
||
codegen + build see the workspace as a coherent unit. The multi-
|
||
diagnostic refactor and possibly cross-module ADTs remain for later.
|
||
|
||
## 2026-05-07 — Iter 5d done: tooling extended to the workspace
|
||
|
||
- `ail manifest|describe|deps|diff <entry> --workspace` now operate
|
||
across all modules of the workspace. The default without the flag stays
|
||
single-module for backwards compatibility. Manifest sorts by
|
||
`(module, name)`, describe accepts dotted notation `ws_lib.add`, deps
|
||
emits `{from_module, from_def, to_module, to_def}` edges, diff compares
|
||
workspace-wide with added/removed/changed/unchanged_modules and a
|
||
nested sub-diff per changed_module.
|
||
- Refactor: `diff_def_lists` is the single source of the four-category
|
||
logic; single and workspace diff share it.
|
||
- Tests: 44 green (previously 40). New: `manifest_workspace_lists_all_defs`,
|
||
`describe_workspace_resolves_qualified_name`,
|
||
`deps_workspace_includes_cross_module`, `diff_workspace_added_module`.
|
||
|
||
**Observation (debt):** `deps` does not filter builtins/locals/function
|
||
parameters. In workspace mode that becomes more visible than in single
|
||
mode — `ws_lib.add` lists edges to `ws_lib.+` (builtin) and
|
||
`ws_lib.a`/`ws_lib.b` (function parameters). A known pre-existing
|
||
issue from Iter 2; Task #22 in the backlog.
|
||
|
||
## 2026-05-07 — architecture review after Iter 5
|
||
|
||
Architect agent invoked. Findings:
|
||
|
||
1. **Mangling consistency holds.** `@ail_<module>_<def>` is consistent
|
||
across functions, constants, string globals, and cross-module calls.
|
||
The trampoline is correct. ADT constructors are deliberately
|
||
symbol-free (inline malloc).
|
||
2. **Module hashes bit-identical since Iter 4.** The Iter 5c snapshot
|
||
regeneration was a codegen-output change, not a hash break.
|
||
3. **Drift, due now:**
|
||
- DESIGN.md says `define i64 @main()`, codegen emits
|
||
`define i32 @main()` (see `sum.ll:35`).
|
||
- String-schema notation in DESIGN.md was shortened
|
||
(`@.str_<module>_<idx>` instead of `@.str_<module>_<hint>_<idx>`).
|
||
4. **Debt that accrues interest:** the `deps` builtin leak (Task #22) has
|
||
become a falsehood in workspace mode — close it before the next big
|
||
jump.
|
||
|
||
**Plan iteration 6 — clean-up:**
|
||
|
||
1. **Fix DESIGN.md drift.** Update the mangling-scheme block, correct the
|
||
`@main` signature, and note the string globals precisely.
|
||
2. **`deps` hardening (#22).** Build a top-level def table per workspace;
|
||
filter edges whose target is not a top-level symbol, or emit them as
|
||
separate `builtin:`/`local:` categories. Function parameters via
|
||
lexical scope tracking from walk_term.
|
||
3. **Multi-diagnostic refactor (#20).** `check_workspace` accumulates
|
||
`Vec<Diagnostic>` across all defs instead of short-circuiting on the
|
||
first error. Intra-def may still short-circuit — the value is "see
|
||
all broken defs at once", not "see all broken sub-terms of one def".
|
||
|
||
Order: 1 first (doc triviality), then 2 before 3 (deps is a tooling-
|
||
truth fix, multi-diag is a structural extension).
|
||
|
||
## 2026-05-07 — Iter 6 done: deps hardening + multi-diagnose + DESIGN audit
|
||
|
||
Three things landed together. All small, all KISS — no architecture move,
|
||
just paying off recorded debt.
|
||
|
||
**1. `ail deps` filters builtins, params, and let/match bindings (#22).**
|
||
|
||
Before: `sum -> +, -, ==, n, sum` and `ws_lib.add -> ws_lib.+`,
|
||
`ws_lib.a`, `ws_lib.b`. After: `sum -> sum`, `ws_lib.add -> ws_lib.add`
|
||
gone (no real deps; only the cross-module call from `ws_main` remains).
|
||
|
||
Implementation:
|
||
- New helper `ailang_check::builtins::value_names()`: derives the
|
||
Var-level builtin names (`+ - * / % == != < <= > >= not`) from
|
||
`list()`, so the install-list and the deps-filter share one source of
|
||
truth.
|
||
- `walk_term` in `crates/ail/src/main.rs` now threads a `scope` set:
|
||
fn-params seed it; `Let` adds the bound name for the body only;
|
||
`Match` arms add their pattern variables (`bind_pattern` helper, MVP
|
||
rule "ctor sub-patterns are Var/Wild") and roll them back after. Var
|
||
refs that hit `scope` or `builtins` are dropped; qualified names
|
||
(`prefix.def`) are passed through unconditionally — the typechecker
|
||
forbids dots in def names, so no shadowing risk.
|
||
- Tests added: `deps_filters_builtins_params_locals`,
|
||
`deps_workspace_filters_builtins_and_params`. The Iter 5d test
|
||
(`deps_workspace_includes_cross_module`) keeps passing — the only
|
||
edge it asserted is the legitimate one.
|
||
|
||
**2. `check_module` / `check_workspace` are multi-diagnose (#20).**
|
||
|
||
`check_in_workspace` returns `Vec<CheckError>` instead of `Result<()>`.
|
||
Pass-1 (top-level symbol table) stays fail-fast — corrupt globals would
|
||
taint every later diagnostic. Type-def installation is fail-fast within
|
||
a module (env corruption) but the outer module loop continues. The
|
||
body-check loop is the multi-diagnose layer: each def is checked against
|
||
the assembled env, errors accumulate, the next def is attempted.
|
||
|
||
Test: `body_errors_accumulate_across_defs` — one module with two
|
||
independent body errors (arity mismatch + unknown var) yields two
|
||
diagnostics with the right `def` field. The legacy single-error `check`
|
||
keeps working by `.into_iter().next()`-ing the Vec, so internal snapshot
|
||
tests in `crates/ailang-check/src/lib.rs` are unchanged.
|
||
|
||
Out of scope: intra-def collection. A single fn body with three type
|
||
errors still reports one. The "see all broken defs at once" goal is met;
|
||
intra-def will require unification deferral and isn't due now.
|
||
|
||
**3. DESIGN.md `What the MVP is NOT` audit (#24).**
|
||
|
||
The section was lying: it claimed "No ADTs / pattern matching" (delivered
|
||
Iter 3) and "Only ints + bools + unit" (strings landed Iter 2). Renamed
|
||
to `What is not (yet) supported`, restructured into "not yet" + "what is
|
||
supported (smoke-tested)". New invariant: this section is meant to be
|
||
the truth at the **end of the latest iteration**, not a 2026-05-07-day-0
|
||
scope statement.
|
||
|
||
**Architecture check (the user-asked self-questioning):**
|
||
|
||
- *Would I use this language now?* For non-recursive arithmetic + ADT
|
||
programs over int/bool/str: yes, comfortably. For anything that needs
|
||
mapping, folding, generic data structures: no, closures are the
|
||
blocker. That's the next big sprint, not Iter 7.
|
||
- *Consistency:* DESIGN.md, JOURNAL.md, code, and CLI output now agree
|
||
on what the language can do. The "What is not (yet) supported" block
|
||
is the canonical truth surface.
|
||
- *Visualisation:* `ail deps --workspace --json` is now a clean
|
||
cross-module call graph (no builtin noise). Good enough for an
|
||
external graph renderer to consume; a built-in DOT/ASCII renderer is
|
||
*possible future tooling*, not "we need it now". KISS.
|
||
- *Documentation:* the agents/ directory is the sub-prompt layer, the
|
||
JOURNAL is the iteration log, DESIGN.md is the contract. No new doc
|
||
axes needed at this scale.
|
||
|
||
**Tests:** 47 green (previously 44). +2 deps tests in `e2e.rs`, +1
|
||
multi-diag test in `crates/ailang-check/tests/workspace.rs`.
|
||
|
||
**Plan iteration 7:**
|
||
|
||
Closures + higher-order functions. This is the big jump that
|
||
DESIGN.md / Day 0 has been pointing at: it requires a typed IR (TIR)
|
||
stage, closure conversion in lowering, and a heap-aware ABI. The
|
||
multi-diag refactor in Iter 6 was scoped intentionally minimal — when
|
||
TIR lands, intra-def diagnostics become structurally cheap and Task #20
|
||
gets revisited.
|
||
|
||
## 2026-05-07 — Iter 7 done: first-class function references (no capture)
|
||
|
||
Iter 6 outlined Iter 7 as "closures + HOFs + TIR". KISS course-correct
|
||
on inspection: that bundle had three independent things in it, and the
|
||
HOF use-cases (passing functions around, calling through fn-typed
|
||
parameters) need none of TIR or capture. Splitting paid off — what
|
||
landed here is ~120 LOC of codegen, no TIR, no heap, no ABI churn.
|
||
Closures with capture stay queued for Iter 8 (where TIR is the
|
||
correct precondition).
|
||
|
||
**What works now:**
|
||
- Top-level fn name (or qualified `prefix.def`) used as a value yields
|
||
an LLVM fn-pointer (`@ail_<m>_<def>`, type `ptr`).
|
||
- Fn-typed parameters can be called as `f(args)` — the body emits an
|
||
indirect `call <ret> (<param-tys>) %f(...)`.
|
||
- Pass through `let`: `let g = inc in g(x)` works (the local just
|
||
aliases the global SSA, the sidetable lookup still hits).
|
||
- Pass to another fn: `apply(inc, 41) == 42` — see
|
||
`examples/hof.ail.json`, exercised end-to-end.
|
||
|
||
**What does not (yet) work — by design:**
|
||
- No anonymous lambdas. The only fn-value source is a top-level def
|
||
reference.
|
||
- No capture. A fn-value is always a constant pointer to a top-level
|
||
def; there is no environment to allocate.
|
||
- Both deferred to Iter 8 where they share the TIR + closure-conversion
|
||
preconditions.
|
||
|
||
**Implementation, in order of where the rubber meets the road:**
|
||
|
||
1. `llvm_type` learned `Type::Fn { .. } -> "ptr"`. The actual signature
|
||
travels separately. New helper `fn_sig_from_type` lifts an AILang
|
||
fn-type into an `FnSig` (LLVM types only).
|
||
2. `Emitter` got a sidetable: `ssa_fn_sigs: BTreeMap<String, FnSig>`,
|
||
keyed by SSA value (or `@global`). It's reset per function body.
|
||
3. At `emit_fn` entry, every fn-typed parameter registers
|
||
`(%arg_<name>, sig)` in the sidetable.
|
||
4. `lower_term(Term::Var)` now falls through to a top-level fn lookup
|
||
(`resolve_top_level_fn`) when the name isn't a local. The returned
|
||
SSA is the global symbol; the sidetable gets the sig.
|
||
5. `lower_term(Term::App)` dispatches:
|
||
- if callee is a `Var` AND not shadowed AND statically known
|
||
(`is_static_callee` covers builtin operators, qualified
|
||
`prefix.def`, current-module fns), keep the existing direct
|
||
`lower_app` path — no extra indirection in the IR;
|
||
- otherwise lower the callee, expect type `ptr`, look up the sig in
|
||
the sidetable, emit `emit_indirect_call`.
|
||
6. `Term::If` propagates the sig to its phi SSA when both branches are
|
||
fn-pointers with matching sigs (cheap two-line copy; no separate
|
||
test, falls out of the `apply`-on-conditional pattern).
|
||
|
||
**Why no typechecker change?** The typechecker already accepted
|
||
fn-typed locals (`Term::Var` against `env.globals`, App via `synth(callee)`
|
||
unifying with `Type::Fn`). The only blocker was `MVP: callee must be a
|
||
variable` in codegen.
|
||
|
||
**Tests:** 48 green (previously 47).
|
||
|
||
- `crates/ail/tests/e2e.rs::higher_order_apply_inc` builds and runs
|
||
`examples/hof.ail.json`, asserts the binary prints `42`.
|
||
- Existing tests unchanged (incl. snapshot tests around the IR
|
||
emission for `sum`, `list`, `max3`).
|
||
|
||
**Architecture self-check:**
|
||
|
||
- *Would I use this language now?* Yes for `apply`-style and "pass a
|
||
predicate" patterns. Still no for capturing closures (`let n = 3 in
|
||
map(\x -> x + n, xs)`-equivalent), but the ergonomic gap shrank.
|
||
- *Consistency:* DESIGN.md "What is not (yet) supported" rewritten in
|
||
the same edit; first-class fn-refs now have a positive bullet, the
|
||
closures bullet is precise about what it means (no capture, no
|
||
lambdas).
|
||
- *Visualisation:* `ail describe`/`manifest` already render fn-typed
|
||
params correctly via the existing `pretty::type_to_string`
|
||
(`((Int) -> Int, Int) -> Int`). No tooling change required.
|
||
- *KISS:* every alternative I considered (full `LocalType` enum,
|
||
swapping `(String, String)` returns to a typed wrapper, lifting
|
||
lambdas to defs as syntactic sugar) was strictly more code than the
|
||
sidetable approach, with no expressivity gain.
|
||
|
||
**Plan iteration 8:**
|
||
|
||
Closures with capture, anonymous lambdas, the typed IR (TIR) layer,
|
||
closure conversion in lowering. Now that we have indirect calls
|
||
working, the main delta is: a fn-value also needs an environment
|
||
pointer, the sidetable becomes per-value (heap-allocated), and the
|
||
calling convention shifts to `(env_ptr, args...)`. Touches every
|
||
existing call path — that's why it gets its own iteration.
|
||
|
||
## 2026-05-07 — Iter 8 done: closures with capture (no TIR needed)
|
||
|
||
Iter 7's plan named TIR as the prerequisite for closures. On
|
||
inspection that bundling was wrong — TIR is one possible
|
||
implementation strategy, not a structural requirement. The
|
||
typechecker already attaches enough type information through `synth`
|
||
that the codegen can read capture types out of `self.locals`
|
||
directly. So Iter 8 ships closures **without** introducing TIR. KISS
|
||
won.
|
||
|
||
The work split into two commits:
|
||
|
||
**Iter 8a — closure-pair ABI flip.** Every fn-value is now a `ptr` to
|
||
a heap or static closure pair `{ thunk_ptr, env_ptr }`, regardless of
|
||
whether it came from a lambda or a top-level def reference. To keep
|
||
top-level-fn references cheap, every top-level fn auto-emits:
|
||
|
||
```llvm
|
||
define <ret> @ail_<m>_<f>_adapter(ptr %_env, <params>) {
|
||
%r = call <ret> @ail_<m>_<f>(<args>)
|
||
ret <ret> %r
|
||
}
|
||
@ail_<m>_<f>_clos = constant { ptr, ptr } { @adapter, null }
|
||
```
|
||
|
||
`Term::Var` resolving to a top-level fn returns the address of
|
||
`_clos`, never the bare fn pointer. `emit_indirect_call` was
|
||
rewritten to GEP+load both halves and call `thunk(env, args...)`.
|
||
Direct calls (statically-known callees in `Term::App`) bypass the
|
||
adapter and stay at the original speed.
|
||
|
||
The Iter 7 hof example (`apply(inc, 41)`) continues to print 42
|
||
unchanged — only the IR shape changed, not the source. IR snapshot
|
||
files for sum/list/max3/hello/ws_main were refreshed.
|
||
|
||
**Iter 8b — Term::Lam + capture + lambda lifting.** New AST node:
|
||
|
||
```jsonc
|
||
{ "t": "lam",
|
||
"params": ["x"...],
|
||
"paramTypes": [Type...],
|
||
"retType": Type,
|
||
"effects": ["..."],
|
||
"body": Term }
|
||
```
|
||
|
||
Param/return types are explicit. The typechecker accepts the
|
||
declared `Type::Fn` shape, checks the body's type against `retType`,
|
||
and verifies that body effects are a subset of the declared lambda
|
||
effects (no row polymorphism in the MVP). Constructing a lambda is
|
||
pure; the act of *calling* picks up the declared effects, via the
|
||
existing App branch.
|
||
|
||
Codegen does textbook closure conversion:
|
||
|
||
1. **Free-variable analysis.** `collect_captures` walks the body
|
||
skipping builtins (`+`, `==`, ...), the current module's top-
|
||
level fns, and qualified `prefix.def` names. The remainder are
|
||
captures. Inner lambdas contribute their own free vars upward.
|
||
2. **Lift to thunk.** For each lambda, generate a fresh
|
||
`@ail_<m>_<def>_lam<id>(ptr %env, params...)`. State the body
|
||
into a side buffer (the emitter's `body`/`locals`/`counter` are
|
||
saved and reset, then restored). Captures and lambda params are
|
||
pushed as named locals so the body lowering finds them. The
|
||
thunk text goes into a `deferred_thunks` queue and is appended
|
||
after the parent fn's `}` — LLVM IR doesn't care about fn order.
|
||
3. **Pack at the use site.** In the OUTER body emit:
|
||
|
||
```llvm
|
||
%env = call ptr @malloc(i64 <8 * captures>)
|
||
; for each capture i: store at offset 8*i
|
||
%clos = call ptr @malloc(i64 16)
|
||
; store thunk_ptr at offset 0, env at offset 8
|
||
```
|
||
|
||
`%clos` is the value returned by the Lam term. Its sig is
|
||
registered in the sidetable so subsequent indirect calls work.
|
||
|
||
4. **Capture sigs propagate.** A fn-typed capture (e.g. capturing a
|
||
fn-typed param of an outer scope) keeps its FnSig in the thunk's
|
||
sidetable, so the captured fn can still be indirect-called from
|
||
inside the lambda.
|
||
|
||
Capture layout uses 8-byte slots regardless of LLVM type. Typed
|
||
load/store reads only the bytes it needs — wasted padding for `i1`
|
||
and `i8` is fine at this scale.
|
||
|
||
**Architecture self-check:**
|
||
|
||
- *Would I use this language now?* Yes for substantially more cases.
|
||
`let n = 3 in apply(\\x. x + n, 39)` is the example I would have
|
||
reached for in Iter 6 and bounced off. It now compiles and runs.
|
||
`map`/`fold`/`filter` over user-supplied predicates are within
|
||
reach — only the absence of polymorphism still forces author-side
|
||
monomorphisation.
|
||
- *Did I think of everything?* Hash stability checked manually:
|
||
`examples/sum.ail.json` produced the same fn hashes
|
||
(`db33f57cb329935e`, `d9a916a0ed10a3d3`) before and after Iter 8.
|
||
Existing modules without `Term::Lam` serialise bit-identically. ✓
|
||
- *Consistency:* DESIGN.md "What is not (yet) supported" rewritten
|
||
in the same edit. The Term schema gained `lam`, `ctor`, `match`
|
||
rows that were already supported but had been omitted from the
|
||
schema fragment. Now the doc is exhaustive for the supported
|
||
language.
|
||
- *Visualisation:* `ail describe` already renders Lam terms (added
|
||
pretty-printer rule), and the codegen IR for `closure.ail.json`
|
||
reads as a textbook closure-conversion lowering.
|
||
- *KISS check:* I considered three alternatives and all were
|
||
strictly worse — fat-pointer ABI (aggregate-passing concerns), full
|
||
TIR layer (large rewrite), uniform heap pair without static-closure
|
||
optimisation (regressed Iter 7 to one malloc per fn-value escape).
|
||
|
||
**Tests:** 49 green (was 48 after Iter 7). One new e2e:
|
||
`closure_captures_let_n` builds and runs `examples/closure.ail.json`
|
||
asserting "42". IR snapshot files refreshed for the per-fn adapter +
|
||
static-closure scaffold — only structural delta.
|
||
|
||
**Plan iteration 9:**
|
||
|
||
Two candidates, both real pain points:
|
||
|
||
1. **Polymorphic inference.** Make `Type::Forall` actually work in
|
||
`synth` — instantiate fresh type variables at each use site, allow
|
||
`let id = \\x. x in (id 1, id true)`. This unblocks generic
|
||
`map`/`fold`/etc. without per-type clones. Probably small (~150
|
||
LOC in the typechecker; codegen already monomorphises by
|
||
instantiation when it lowers the call).
|
||
|
||
2. **GC / region reclamation.** Right now ADT boxes, lambda envs,
|
||
and closure pairs all leak through the program's lifetime. A
|
||
minimal mark-and-sweep over a tagged heap would let us run real
|
||
programs. Bigger lift, ~400-600 LOC plus runtime support.
|
||
|
||
Leaning toward (1) for the next iteration: it's the smaller bite
|
||
*and* the bigger expressivity unlock. (2) becomes acute only when
|
||
someone tries to run an unbounded loop, which the current examples
|
||
don't.
|
||
|
||
## 2026-05-07 — Iter 9 done: dogfood + `ail run`
|
||
|
||
Course-corrected from the Iter-8 plan. Polymorphism is the bigger
|
||
expressivity unlock on paper, but I hadn't actually proved that the
|
||
language was sufficient for "small but real" programs without it. So
|
||
Iter 9 became a dogfood iteration: write a non-trivial program
|
||
that exercises everything Iter 1-8 shipped, and use `ail run` /
|
||
errors / type-checker output as the user would. If something broke,
|
||
fix it. If nothing broke, document the boundary moved.
|
||
|
||
**`examples/list_map.ail.json`**:
|
||
|
||
```jsonc
|
||
type IntList = Nil | Cons Int IntList
|
||
|
||
map_int :: ((Int) -> Int, IntList) -> IntList
|
||
map_int(f, xs) = match xs {
|
||
Nil -> Nil
|
||
Cons(h, t) -> Cons(f(h), map_int(f, t))
|
||
}
|
||
|
||
print_list :: (IntList) -> Unit !IO
|
||
print_list(xs) = match xs {
|
||
Nil -> ()
|
||
Cons(h, t) -> let _ = do io/print_int(h) in print_list(t)
|
||
}
|
||
|
||
main = let xs = Cons 1 (Cons 2 (Cons 3 Nil)) in
|
||
print_list(map_int(\\x. x * 2, xs))
|
||
```
|
||
|
||
**Result:** nothing broke. Output `2\\n4\\n6\\n`, exit 0. The full
|
||
pipeline (`ail run`) covers: ADTs with two ctors of different
|
||
arity; pattern matching with nested `Var` fields; recursion over
|
||
ADT; closures (with no captures here, so env is null but the
|
||
closure-pair plumbing still gets exercised); fn-typed parameters in
|
||
a top-level def; `do io/...` inside a match arm body, with `let _`
|
||
to sequence two effectful operations; effect propagation through
|
||
the call chain. This validates Iter 1-8 as a self-contained
|
||
foundation.
|
||
|
||
**Friction surfaced:** writing the AST by hand is tedious — the
|
||
JSON for this 4-def module is 200+ lines. That's not surprising
|
||
(the format is for LLMs, not humans), but it suggests an Iter 10
|
||
priority: a richer pretty-print form, or an `ail snippet` helper
|
||
for common boilerplate (`mk_list_int`, etc.). Not blocking; noted.
|
||
|
||
**`ail run` (Iter 9b):** Builds into a tempdir + execs the binary,
|
||
exit code passthrough. Saves a `cd && ./bin` step in the dogfood
|
||
loop. Tiny addition — `Cmd::Build`'s body factored into a shared
|
||
`build_to` helper.
|
||
|
||
**Architecture self-check:**
|
||
|
||
- *Would I use this language now?* For self-contained Int-typed
|
||
programs over recursive ADTs: yes. The list_map example is what
|
||
I would have wanted to write since Iter 6 and bounced off
|
||
repeatedly. It now compiles and runs without me adapting the
|
||
source — the language is what its authors said it was, end to
|
||
end.
|
||
- *Did I think of everything?* Two cracks observed during the
|
||
dogfood:
|
||
- `(Int)` parens around single-param fn-types in pretty-print are
|
||
visual noise. Cosmetic, can wait.
|
||
- `let _ = do <effect> in <body>` is the only way to sequence
|
||
effects today. Working as intended given KISS, but a `;`
|
||
operator (sequencing) would be cheap polish.
|
||
- *Consistency:* DESIGN.md CLI block + smoke-test list updated.
|
||
Iter 8c invariant — "What is not (yet) supported" ≡ truth at end
|
||
of latest iteration — held; no new pending items.
|
||
- *KISS:* Iter 9 added 0 LOC of language semantics. All gain came
|
||
from validating the existing surface and a small CLI helper.
|
||
|
||
**Tests:** 50 green (was 49). New e2e
|
||
`list_map_doubles_then_prints`. No test for `ail run` itself —
|
||
`build_and_run` already exercises the equivalent path.
|
||
|
||
**Plan iteration 10:**
|
||
|
||
The dogfood revealed two real-but-not-blocking pain points and one
|
||
big architectural gap. Candidates, ranked:
|
||
|
||
1. **Polymorphic let-bindings with monomorphisation at codegen.**
|
||
Allows `let id = \\x. x in (id 1, id true)` and ultimately
|
||
`map :: (a -> b) -> List a -> List b`. The ground truth-ier
|
||
answer for the "would I use it for X?" question, but a
|
||
non-trivial pipeline change (typechecker→codegen needs to thread
|
||
instantiation info to the call site).
|
||
|
||
2. **Sequencing operator `;` and richer effect ergonomics.** A
|
||
`Term::Seq { lhs, rhs }` (or compile sugar to `Let { name: "_",
|
||
value: lhs, body: rhs }`) plus a small pretty-print update.
|
||
Cheap, satisfying.
|
||
|
||
3. **GC.** Heap reclamation for ADT boxes, lambda envs, closure
|
||
pairs. Real architecture step. Becomes acute the moment someone
|
||
writes a long-running loop; the current examples don't.
|
||
|
||
Tentative pick: (2) for the next sprint as a satisfying small
|
||
polish, then (1) as Iter 11. (3) bides its time until a real
|
||
program needs it.
|
||
|
||
## 2026-05-07 — Iter 10 done: Term::Seq sequencing
|
||
|
||
Followed the Iter 9 plan and shipped (2). New AST node
|
||
`Term::Seq { lhs, rhs }` with serde tag "seq". Semantics: evaluate
|
||
lhs (which must be Unit), discard the value, return rhs. Effects
|
||
from both sides accumulate.
|
||
|
||
This is sugar for `let _ = lhs in rhs`, but it's a first-class node
|
||
because:
|
||
- The pretty-print renders cleanly (`(seq lhs rhs)` instead of
|
||
borrowing the `let` form with a discard binding).
|
||
- Diagnostics are sharper: a non-Unit lhs gets a "type mismatch"
|
||
error pointing at the seq site, not "binding `_` had type X" at
|
||
a let site.
|
||
- Future tooling (effect inference visualisation, dataflow) can
|
||
treat sequencing as a structural concept instead of a special-
|
||
cased let.
|
||
|
||
Codegen is trivial: lower lhs (drop SSA), lower rhs (return).
|
||
|
||
Refactored `examples/list_map.ail.json`'s `print_list` to use seq
|
||
instead of `let _ = ...`. Output unchanged (`2\\n4\\n6\\n`); the
|
||
JSON shed a few lines and reads more honestly.
|
||
|
||
**Architecture self-check:**
|
||
|
||
- *Would I use this language now?* Same answer as Iter 9 (yes for
|
||
small but real programs), but the seq node makes IO-heavy
|
||
recursion read better — closer to "call this effect, then this
|
||
one" instead of "bind this effect to nothing, then this one".
|
||
- *Did I break anything?* Hash stability check: existing examples
|
||
without `Term::Seq` serialise identically; their fn hashes are
|
||
unchanged. `list_map.ail.json`'s hashes shifted as expected since
|
||
its body changed.
|
||
- *KISS:* +30 LOC across AST/pretty/check/codegen/walker. One
|
||
unit test for the lhs-must-be-Unit rule. The dogfood example
|
||
proves the e2e path.
|
||
|
||
**Tests:** 51 green (was 50). New `seq_lhs_must_be_unit` unit test
|
||
in ailang-check. Existing list_map e2e still passes after the
|
||
refactor.
|
||
|
||
**Plan iteration 11:**
|
||
|
||
Polymorphism, as queued in the Iter 9 plan. Concretely: HM-style
|
||
unification + let-generalisation in the typechecker, monomorph-
|
||
isation at codegen time. Touches the typechecker→codegen pipeline.
|
||
Bigger commit than the recent stretch, will probably need to be
|
||
phased (typechecker substitution machinery, then codegen
|
||
specialisation, then docs).
|
||
|
||
## 2026-05-07 — Iter 11 done: deeper dogfood (insertion sort)
|
||
|
||
Pulled back from polymorphism for one more validation cycle before
|
||
the architectural step. Polymorphism is a substantial pipeline
|
||
change (typechecker substitution + codegen monomorphisation) and I
|
||
wanted one more "small but real" program to confirm the existing
|
||
foundation holds before disturbing it.
|
||
|
||
`examples/sort.ail.json` — insertion sort over `IntList`:
|
||
|
||
```jsonc
|
||
insert :: Int -> IntList -> IntList
|
||
insert(y, xs) = match xs {
|
||
Nil -> [y]
|
||
Cons(h, t) -> if y <= h then Cons(y, Cons(h, t))
|
||
else Cons(h, insert(y, t))
|
||
}
|
||
|
||
sort :: IntList -> IntList
|
||
sort(xs) = match xs {
|
||
Nil -> Nil
|
||
Cons(h, t) -> insert(h, sort(t))
|
||
}
|
||
|
||
print_list :: IntList -> Unit !IO // uses Iter 10 seq
|
||
|
||
main = print_list(sort([3,1,4,1,5,9,2,6,5,3,5]))
|
||
```
|
||
|
||
**Result:** typechecks first try, runs first try, prints
|
||
`1 1 2 3 3 4 5 5 5 6 9` (each on its own line). 11-element input,
|
||
correct sorted output. The combination of recursive ADT pattern
|
||
match + comparison ops + branching + leaf recursion + IO
|
||
sequencing all worked end to end without the language tripping me
|
||
up. Iter 10's seq made `print_list` notably cleaner than the
|
||
`let _ = ...` form would have been.
|
||
|
||
**Architecture self-check:**
|
||
|
||
- *Would I use this language now?* For "small but real"
|
||
monomorphic programs over Int, Bool, Unit, Str, and ADTs of
|
||
those: confidently yes. Insertion sort writes out as the
|
||
textbook recursion, no bookkeeping that the language couldn't
|
||
do for me.
|
||
- *Did I think of everything?* The remaining wall is still
|
||
polymorphism. Sort over `IntList` needs hand-monomorphisation;
|
||
a generic `sort :: (a -> a -> Bool) -> List a -> List a` is
|
||
what the language eventually wants. No new architectural cracks
|
||
surfaced from this dogfood.
|
||
- *Visualisation:* `ail describe sort.ail.json sort` reads the
|
||
way I'd expect a sort definition to read, with `IntList` types
|
||
inline and the recursive call rendered cleanly.
|
||
- *KISS:* Iter 11 added 0 LOC of language semantics and 1 e2e
|
||
test. The 250-line JSON for the example is verbose but
|
||
mechanical — no friction once you accept that the JSON is the
|
||
surface for LLM authors.
|
||
|
||
**Tests:** 52 green (was 51). New e2e
|
||
`insertion_sort_orders_list`. Pure addition; existing tests
|
||
untouched.
|
||
|
||
**Plan iteration 12:**
|
||
|
||
Now polymorphism. Two more dogfood programs would just keep
|
||
producing the "the language is fine for monomorphic programs"
|
||
result, which is already established. The real expressivity
|
||
unlock — and the answer to "would I use it for X?" for X that
|
||
actually needs generic data — is HM inference + let-generalisation
|
||
+ monomorphisation. Phased plan:
|
||
|
||
12a. Typechecker: introduce a `Subst` (type variable substitution)
|
||
and unification. Thread through `synth`. At `let`, generalise
|
||
syntactic values (lambdas) — no value-restriction subtlety
|
||
needed yet, the MVP has no mutable refs.
|
||
12b. Codegen: at each polymorphic call site, the typechecker
|
||
records the instantiation. Codegen walks the AST a second
|
||
time per (def, instantiation) pair and emits a specialised
|
||
version with the type variables substituted by concrete
|
||
types in fn signatures.
|
||
12c. Docs + a polymorphic `id` test + a generic `map :: (a -> b)
|
||
-> List a -> List b` rewrite of `list_map.ail.json`.
|
||
|
||
## 2026-05-07 — Iter 12a/b done: polymorphism reaches the binary
|
||
|
||
Skipped 12c's "polymorphic map" — without parameterised ADTs (which
|
||
the MVP doesn't have), the rewrite would still be over a concrete
|
||
`IntList`, defeating the purpose. So 12c becomes lighter: docs +
|
||
two new examples (`poly_id`, `poly_apply`) that prove polymorphism
|
||
end-to-end on primitive types and on fn-typed parameters. The big
|
||
test is whether *I* would use the language now for a poly-flavoured
|
||
program; the answer below.
|
||
|
||
**12a — typechecker:**
|
||
|
||
`Type::Forall { vars, body }` is now legal at top-level fn types.
|
||
Implementation is the textbook ML rule: peel the Forall when
|
||
checking the body (rigid vars go into `Env.rigid_vars` so
|
||
`check_type_well_formed` accepts them), instantiate fresh metavars
|
||
at every var-resolution site, unify on every formerly-`expect_eq`
|
||
edge.
|
||
|
||
The metavar encoding sidesteps an AST schema change: a metavar is
|
||
just `Type::Var { name: "$m<id>" }`. The `$` prefix can't collide
|
||
with source identifiers, the JSON layout doesn't shift, and module
|
||
hashes stay bit-identical (verified: `sum.ail.json` keeps
|
||
`db33f57cb329935e` / `d9a916a0ed10a3d3`). I considered adding a
|
||
new `Type::Meta` variant under `#[serde(skip)]` but that would
|
||
have pulled hashing concerns into serde; the naming convention
|
||
keeps the AST untouched.
|
||
|
||
`Subst` is a flat `BTreeMap<u32, Type>`; `unify` is the standard
|
||
occurs-check version with effects compared as a set. Constants
|
||
still reject Forall outright; ADT fields still reject vars. No
|
||
let-generalisation: lambdas inside fn bodies are checked
|
||
monomorphically against their declared types — keeps the
|
||
implementation small and matches DESIGN.md's "top-level types
|
||
must always be explicitly annotated".
|
||
|
||
**12b — codegen:**
|
||
|
||
Direct calls to a polymorphic def get monomorphised on demand.
|
||
Each unique (def, instantiation) pair emits a specialised LLVM fn
|
||
with mangling `@ail_<m>_<def>__<descriptor>`. Descriptor scheme:
|
||
`Int → I`, `Bool → B`, `Unit → U`, `Str → S`, ADT `Foo → FFoo`,
|
||
`Fn(a)→b → Fn_<a>__r_<b>`. So `id(42)` and `id(true)` produce
|
||
`@ail_poly_id_id__I` and `@ail_poly_id_id__B` side by side.
|
||
|
||
Pass 1 of `lower_workspace` now splits fn-typed defs into mono
|
||
(`module_user_fns`, LLVM-typed FnSig as before) and poly
|
||
(`module_polymorphic_fns`, full FnDef). A unified
|
||
`module_def_ail_types` carries AILang types for both, used by
|
||
the codegen-side type tracker.
|
||
|
||
The hard part was getting AILang types at call sites. The
|
||
typechecker has them but doesn't hand its annotations down (no
|
||
TIR yet). I considered three paths:
|
||
1. Typechecker sidetable keyed by AST node ids — would need
|
||
to assign ids deterministically, brittle.
|
||
2. Uniform representation (everything passes as ptr/i64) —
|
||
contradicts CLAUDE.md's "performance is extremely important".
|
||
3. Codegen replays the type derivation locally.
|
||
Picked (3). The trade-off is duplication (`synth_arg_type`
|
||
mirrors what the typechecker already did), but it's contained
|
||
to a small recursive walk and uses the same `locals`/`extras`
|
||
shadowing pattern. Worth it for the MVP — once a TIR stage
|
||
materialises (it's still on the debt list), the duplication
|
||
collapses into a single pass.
|
||
|
||
`locals` grew from 3-tuple to 4-tuple `(name, ssa, llvm_type,
|
||
ail_type)`. Six push sites updated mechanically. Lambda capture
|
||
metadata grew the same way. `CtorRef` got `ail_fields` so match
|
||
arm bindings inherit the AILang type.
|
||
|
||
The drain phase iterates until `mono_queue` is empty —
|
||
specialised bodies can themselves invoke polymorphic defs and
|
||
queue further entries. `apply_subst_to_term` substitutes rigid
|
||
vars in `Term::Lam` annotations (the only Term arm carrying
|
||
types).
|
||
|
||
**Architecture self-check:**
|
||
|
||
- *Would I use this language now?* For monomorphic programs:
|
||
yes (already established). For polymorphism over primitives
|
||
and fn-typed parameters: yes — `id` and `apply` write out the
|
||
way the textbook says they should, with no language-level
|
||
bookkeeping leaking into the source. The `poly_apply` example
|
||
was particularly revealing: the closure-pair ABI (Iter 8a)
|
||
composes cleanly with monomorphisation. Specialised body of
|
||
`apply__I_I` keeps `f` as a fn-typed local; the existing
|
||
indirect-call path already handles the lower from there.
|
||
- *Did I think of everything?* No, two known gaps:
|
||
1. **Polymorphic fn passed as a value** (`let f = id in f(42)`)
|
||
fails in codegen — `resolve_top_level_fn` looks in
|
||
`module_user_fns` only. Adding this means emitting one
|
||
closure-pair global per instantiation, possibly via the
|
||
same drain pass. Defer.
|
||
2. **Higher-rank polymorphism** (`apply(id, 42)`) trips
|
||
`unify_for_subst` which doesn't handle Forall on the param
|
||
side. Real higher-rank polymorphism is a substantial step
|
||
and not on the near horizon — deferred to a later iter.
|
||
- *Visualisation:* `ail manifest poly_id.ail.json` now shows
|
||
`forall a. (a) -> a` correctly. The pretty-printer carried
|
||
`Type::Forall` rendering since Iter 1; nothing to do.
|
||
- *KISS:* +1 typechecker file edit (~430 LOC inserted, mostly
|
||
Subst+unify+four tests), +1 codegen extension (~600 LOC
|
||
inserted, mostly the drain path + helpers + locals widening).
|
||
Two new examples, two new e2e tests. Could be smaller if I
|
||
bit the bullet on TIR; not yet worth the upfront cost.
|
||
|
||
**Tests:** 58/58 (was 56/56). Added 4 typechecker unit tests in
|
||
12a, 2 e2e tests in 12b. Hash invariant holds.
|
||
|
||
**Plan iteration 13 (queued, not started):**
|
||
|
||
The natural next step depends on what I want to use the language
|
||
for. Two candidates, in order of expected payoff:
|
||
|
||
13a. **Parameterised ADTs** — `List a`, `Maybe a`, etc. Without
|
||
these, polymorphism is half-useful: a generic `map` still
|
||
can't transform an `IntList` into a `BoolList`. ADT defs
|
||
would gain a `vars: Vec<String>` field; ctor field types
|
||
could mention them; codegen monomorphises ADT instances
|
||
just like fns. This is the bigger expressivity unlock.
|
||
13b. **GC or arena** — every ADT box, lambda env, and closure
|
||
pair currently leaks. For sort over an 11-element list,
|
||
fine. For anything longer-running, required. The current
|
||
lifetime model is "leak"; the right MVP is probably
|
||
bumpalloc per top-level fn invocation. Could be done
|
||
before parameterised ADTs but doesn't unlock new examples.
|
||
|
||
Leaning 13a — it's the more interesting architectural step and
|
||
makes the "polymorphic map" rewrite from the original 12c plan
|
||
finally meaningful.
|
||
|
||
## 2026-05-07 — Iter 13 done: parameterised ADTs reach the binary
|
||
|
||
**Why now.** End of Iter 12 left polymorphism half-useful: `id`
|
||
and `apply` worked, but every container was monomorphic
|
||
(`IntList`, `Maybe_Int`). A generic `map :: forall a b. ((a) -> b,
|
||
List a) -> List b` was unwritable. 13 lifts that.
|
||
|
||
**Three commits:**
|
||
|
||
- `0782622` 13a — schema (`TypeDef.vars`, `Type::Con.args`) +
|
||
checker (substitution at ctor + match + arity validation in
|
||
`check_fn`).
|
||
- `1631f60` 13b — codegen: per-use-site substitution of LLVM
|
||
field types in `lower_ctor` and `lower_match`. No mono-queue
|
||
for types — ctor code was already inlined at every use site,
|
||
so 13b only had to thread substitution through, not invent a
|
||
symbol scheme. `synth_arg_type` for `Term::Ctor` now returns
|
||
concrete type-args, and `llvm_type(Type::Var)` is a hard
|
||
error instead of a silent `ptr` fallback (the latter was
|
||
flagged by the architect review and is the most defensive
|
||
single change in 13).
|
||
- `<this>` 13c — DESIGN.md flipped (parameterised ADTs out of
|
||
the gap list, into the supported list); two new example lines.
|
||
|
||
**Hash invariant.** Both new fields are
|
||
`#[serde(default, skip_serializing_if = "Vec::is_empty")]`. A new
|
||
regression test in `crates/ailang-core/src/hash.rs` deserialises
|
||
the actual `examples/sum.ail.json` and `examples/list.ail.json`
|
||
from disk and asserts `db33f57cb329935e` and `b082192bd0c99202` —
|
||
the recorded pre-13a hashes. It's deliberately phrased against
|
||
the on-disk JSON rather than reconstructed code, so the test
|
||
fails if anyone resaves the examples in a way that drifts the
|
||
canonical bytes.
|
||
|
||
**Architect-flagged debt I deliberately did NOT touch in 13:**
|
||
|
||
- `is_static_callee` returns true for poly fns but
|
||
`resolve_top_level_fn` only consults `module_user_fns`. A
|
||
poly fn used as a value (`let f = id in f(42)`) passes the
|
||
static check then surfaces as `UnknownVar`. Would need one
|
||
closure-pair global per instantiation. Out of 13 scope; same
|
||
hole that was queued at the end of Iter 12.
|
||
- Triple source of truth for builtins (`builtins::install`,
|
||
`builtins::list`, `codegen::builtin_ail_type` /
|
||
`builtin_effect_op_ret`). Every new operator costs three
|
||
edits. Low interest today, escalates with every effect op.
|
||
Worth a future tidy iter — not blocking expressivity.
|
||
- `synth_arg_type` for `Term::If` returns `synth(then)` only;
|
||
for `Term::Match`, the first arm. Masked today by the
|
||
typechecker having already unified, but it's the kind of
|
||
duplication that decays. Same fundamental cost as the absence
|
||
of a TIR.
|
||
|
||
**Architecture self-check.**
|
||
|
||
- *Would I use this language now?* For polymorphism over
|
||
primitives, fn-typed values, AND parameterised containers —
|
||
yes. The `box.ail.json` and `maybe_int.ail.json` examples
|
||
read like the textbook says they should. No type-arg
|
||
bookkeeping leaks into the source.
|
||
- *KISS.* 13b was much smaller than I feared at the start of
|
||
the design phase: ~150 LOC in codegen, no new structures, no
|
||
mono-queue. The reason: ADT ctor code is already inlined.
|
||
The architect's recommendation to *not* mutate `ctor_index`
|
||
but derive `CtorRef` per use site was the right call —
|
||
preserved the static template, made the substitution local.
|
||
- *Did I think of everything?* Two known gaps remain. **(1)**
|
||
Polymorphic ADTs as the type-arg of a polymorphic fn —
|
||
works today because `unify_for_subst` recurses through
|
||
`Type::Con.args` (added in 13a). **(2)** A polymorphic fn
|
||
taking a polymorphic ADT and returning a different
|
||
parameterised ADT (`map : forall a b. ((a)->b, List a) -> List
|
||
b`) — should also work, but I haven't dogfooded it yet
|
||
because `List a`-as-a-rewrite-of-`list_map` would need the
|
||
schema bumps elsewhere (paramaterised list builder). Queued
|
||
for Iter 14.
|
||
- *Visualisation.* `ail manifest examples/box.ail.json` shows
|
||
`type Box :: forall a. MkBox(a)` and
|
||
`fn unbox :: forall a. (Box<a>) -> a`. The pretty-printer
|
||
picked up `args` and `vars` cleanly (Iter 13a).
|
||
|
||
**Tests:** 64/64 (was 58/58). Added: 1 hash-stability regression
|
||
(13a), 3 checker unit tests for parameterised ADTs (13a),
|
||
2 e2e tests over `box.ail.json` and `maybe_int.ail.json` (13b).
|
||
|
||
**Process note (orchestration).** First iter where I worked
|
||
strictly through the agents in `/agents/`: `ailang-architect`
|
||
ran a drift review on HEAD before 13b started; `ailang-implementer`
|
||
got a fixed brief that incorporated the architect's three
|
||
recommendations (don't mutate `ctor_index`, fix `synth_arg_type`
|
||
for `Term::Ctor`, harden `llvm_type`); 13c (this) is the
|
||
orchestrator's own work. The role split landed in `3df943d` after
|
||
I caught myself doing implementer work on 13a directly. The
|
||
agents pay off in proportion to iter size — for 13b they were
|
||
clearly worth the round-trip; for 13a's checker work, marginal.
|
||
|
||
**Plan iteration 14 (queued, not started):**
|
||
|
||
Two candidates, in order of expected payoff:
|
||
|
||
14a. **Polymorphic `List a` rewrite of `list_map`.** Replaces
|
||
`IntList` with `List a`, rewrites `list_map` to return
|
||
`List b`, and lets the polymorphic-map version be the
|
||
dogfood smoke test. Pure exercise — should fall out of
|
||
13b — but worth the dogfood beat. Also: `Maybe a` used
|
||
in a non-trivial fn (e.g. `find : forall a. ((a) -> Bool,
|
||
List a) -> Maybe a`).
|
||
14b. **GC or arena.** Same pitch as before: every ADT box,
|
||
lambda env, closure pair leaks. For `box.ail.json` and
|
||
`maybe_int.ail.json`, fine. For anything that allocates
|
||
in a loop, required. Bumpalloc per top-level fn
|
||
invocation is the natural MVP.
|
||
14c. **Poly fn as value.** Closes the asymmetry the architect
|
||
flagged; gates `let f = id in f(42)`. One closure-pair
|
||
global per instantiation, emitted via the same mono-queue
|
||
drain path. Smaller surface than 14a/b.
|
||
|
||
Leaning 14a — the dogfood payoff for one iter of polish is
|
||
high, and `Maybe`-in-a-real-fn is a missing piece I haven't
|
||
exercised yet. 14b stays second; 14c is a candidate if I want
|
||
a small palate cleanser.
|
||
|
||
---
|
||
|
||
## Iter 13d — rustdoc polish for `ailang-core` + new `ailang-docwriter` agent
|
||
|
||
User triggered: ran `cargo doc --open` for fun and reported
|
||
that the rendered docs were thin — crate headers existed,
|
||
but `pub` items had no `///` strings, there were no `# Examples`
|
||
sections, and intra-doc links were missing. Internal references
|
||
showed up as prose ("see Builtins") rather than as clickable
|
||
links. Two stale `[Builtins]` and `[code]` warnings had been
|
||
bleeding into every `cargo doc` invocation since Iter 6 or so.
|
||
|
||
Recurring task → new agent. Wrote `agents/ailang-docwriter.md`
|
||
with a tight mandate: rustdoc only, no API changes, no edits in
|
||
`docs/` or `agents/`, three verification gates (rustdoc clean,
|
||
build green, tests green incl. doctests). Updated
|
||
`agents/README.md`. Updated DESIGN.md item 6 of "Verification
|
||
and correctness" to make rustdoc cleanliness a project-wide
|
||
invariant rather than an iter-local cleanup.
|
||
|
||
First mission: `ailang-core` only. The foundation crate every
|
||
other crate depends on — biggest reader-leverage per diff. The
|
||
agent rewrote the crate root so a newcomer learns: what `core`
|
||
owns, where it sits in the pipeline (`core` → `check` →
|
||
`codegen` → `ail`), the central invariant (canonical JSON is
|
||
deterministic, hashes content-addressed, schema = `ailang/v0`),
|
||
and the entry points. Module roots in `ast.rs`, `canonical.rs`,
|
||
`hash.rs`, `pretty.rs`, `workspace.rs` got the same treatment.
|
||
Every `pub` struct / enum / variant / fn / const got a `///`
|
||
string. The Iter-13a additions (`TypeDef.vars`, `Type::Con.args`)
|
||
got an explicit backwards-compat note that points back to the
|
||
hash-stability regression test.
|
||
|
||
`# Examples` blocks landed where they shorten understanding
|
||
(`canonical::to_bytes`, `def_hash`, `Workspace`), all marked
|
||
`ignore` so the workspace doctest run stays cheap (3 ignored,
|
||
0 run, 0 failed). The two stale broken-link warnings in
|
||
`ailang-check` got prose-only fixes — out-of-scope for this
|
||
iter conceptually, but a one-line fix per file means rustdoc
|
||
is now globally clean.
|
||
|
||
**Findings reported by the docwriter** (judgement deferred to
|
||
me; none made it into the diff):
|
||
|
||
- `Term::Lam.param_tys` (JSON: `paramTypes`) is positionally
|
||
paired with `Term::Lam.params: Vec<String>`. Naming hints
|
||
at the convention but a cold reader has to deduce it. **Not
|
||
fixing**: renaming would touch the schema, breaks every hash.
|
||
The `///` string makes the convention explicit, which is
|
||
enough.
|
||
- `Type::Var` is overloaded: source-level rigid vars and
|
||
checker metavars (`$m<id>`) share the same variant. A reader
|
||
of `core` alone sees no hint of the metavar half — it's
|
||
documented in `ailang-check`'s lib doc instead. **Not fixing**:
|
||
splitting the variant would balloon the schema and
|
||
invalidate every hash. Acceptable as long as `check`'s lib
|
||
doc explains it (it does, post-13d).
|
||
- `Def::Type(TypeDef)` versus the type-expression enum `Type`
|
||
in the same module: name collision is real but unavoidable
|
||
without renaming `Type` (which would touch every crate). The
|
||
`///` strings now disambiguate at point of contact.
|
||
|
||
**Process note (orchestration).** Second iter where I worked
|
||
strictly through agents (after 13b). The docwriter brief was
|
||
written from the diagnostic in this conversation, not from a
|
||
DESIGN-doc design pass — there was no architecture decision to
|
||
make, just a discipline gap to close. That's the right shape
|
||
for a docwriter: low-judgement, repeatable, runs after every
|
||
iter that touches public surface. Cost ≈ 25 tool uses for ~390
|
||
LOC of doc additions across 9 files; smaller-grain than
|
||
`ailang-implementer` runs typically are.
|
||
|
||
**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests
|
||
(new). `cargo doc --no-deps`: 0 warnings (was 2). `cargo build
|
||
--workspace` green. `cargo test --workspace` green.
|
||
|
||
**Plan iteration 13e/13f (queued, not started).** Two natural
|
||
follow-ups for the docwriter:
|
||
|
||
13e. **`ailang-check` rustdoc**: type-checker is the next
|
||
biggest crate by `pub`-surface and the most algorithmically
|
||
dense. Rigid/metavar split, `Forall` instantiation, the
|
||
match-arm exhaustiveness logic, the Iter-13a substitution
|
||
machinery — all of it benefits more from prose explanation
|
||
than `core` did.
|
||
13f. **`ailang-codegen` + `ail` CLI rustdoc**: codegen is dense
|
||
but mechanical (mangling, ABI, block tracking); the CLI
|
||
is mostly clap derive macros. Lower payoff per LOC of doc
|
||
than 13e but rounds out the warning-free invariant across
|
||
the whole workspace.
|
||
|
||
After 13d there's no urgency on 13e/f — `cargo doc` is already
|
||
warning-free. They're "polish iterations to be slotted in
|
||
between feature iters when context budget is short". 14a
|
||
(`List a` rewrite) remains the next-feature default.
|
||
|
||
## Iter 13e — rustdoc polish for `ailang-check`
|
||
|
||
Second docwriter mission. Same mandate as 13d, applied to the
|
||
typechecker crate. `lib.rs` (1922 LOC) had a strong crate root
|
||
already — covers HM-with-effects, top-level forall, the
|
||
rigid-vs-metavar split, the `$m<id>` encoding — but its `pub`
|
||
surface (the `Env` struct, `CheckError` + 24 variants,
|
||
`CheckedModule`, `CtorRef`, the `check_module` entry point) was
|
||
mostly undocumented. `builtins.rs` had a one-line module
|
||
header and zero `///` strings on `EffectOpSig` or `install`.
|
||
`diagnostic.rs` had a strong module header (lists every stable
|
||
diagnostic code) but `Diagnostic`, `Severity`, and the
|
||
construction helpers were undocumented.
|
||
|
||
Agent added 188 LOC of pure rustdoc across the three files; no
|
||
non-doc lines changed (verified by filtering the diff). Each
|
||
`CheckError` variant now carries the AST term/type that
|
||
triggers it and the stable kebab-case `code` it maps to. `Env`
|
||
fields (`globals`, `effect_ops`, `types`, `module_globals`,
|
||
`current_module`) got individual `///` strings that name the
|
||
invariants — most importantly that `module_globals` includes
|
||
the current module, which the agent flagged as undocumented at
|
||
field level (now fixed). Crate-root prose got an upgraded
|
||
intra-doc link to `[`check_module`]`; `super::` reference in
|
||
`diagnostic.rs` rewritten to `crate::` (cosmetic, but the
|
||
canonical form).
|
||
|
||
**Findings reported** (judgement deferred to me):
|
||
|
||
- `Env` is `pub` with all-`pub` fields but `Env::new` is
|
||
private and there's no public builder — external callers can
|
||
only construct via field-by-field literal, which is fragile
|
||
if a field is added later. **Not fixing**: changing this is
|
||
an API decision, not a doc one. Worth raising next time we
|
||
touch the crate's public surface deliberately.
|
||
- `CheckError::CtorArity` and `CheckError::ArityMismatch` both
|
||
serialize to the public diagnostic code `arity-mismatch`. The
|
||
`///` strings now flag the collision per-variant; tooling
|
||
that consumes the diagnostic JSON sees only the merged code
|
||
and that's intentional from the iter-5b vintage. **Not
|
||
fixing.**
|
||
- `Diagnostic` and `Severity` are reachable both via the crate
|
||
re-export and via `crate::diagnostic::*` because the
|
||
`diagnostic` module is itself `pub`. Rustdoc renders both
|
||
pages; harmless but slightly noisy. **Not fixing**: the
|
||
re-export is the documented entry point and we don't want to
|
||
hide the module.
|
||
|
||
**Process note.** Same shape as 13d: I wrote a brief naming
|
||
the deficiencies (numbers of pub items, which files had thin
|
||
roots), the agent did the doc additions inside its mandate,
|
||
the orchestrator-side work was authoring DESIGN/JOURNAL and
|
||
verifying the diff. Cost ≈ 32 tool uses for 188 LOC of doc
|
||
across 3 files — denser than 13d (more sentences per pub item
|
||
because the typechecker invariants need explicit
|
||
articulation), but the per-LOC payoff for a future reader is
|
||
also higher.
|
||
|
||
**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests
|
||
(unchanged). `cargo doc --no-deps`: 0 warnings (was 0; the
|
||
agent introduced 4 transient broken-link warnings during the
|
||
work and resolved all of them before reporting done). `cargo
|
||
build --workspace` green. `cargo test --workspace` green.
|
||
|
||
**Plan 13f / 14a unchanged.** 13f (`ailang-codegen` + `ail`
|
||
CLI) is the natural next polish iter; 14a (`List a` rewrite of
|
||
`list_map`) remains the next-feature default. Auto-mode is on,
|
||
so I'll continue into 13f directly unless context budget
|
||
pressures a switch.
|
||
|
||
## Iter 13f — rustdoc polish for `ailang-codegen` + `ail` CLI
|
||
|
||
Combined docwriter mission. Codegen has a small public
|
||
surface (only 3 top-level `pub` items: `CodegenError` enum +
|
||
7 variants, `emit_ir`, `lower_workspace`); the CLI is a
|
||
binary with zero `pub` items, so the only useful rustdoc is
|
||
the module header. One agent run, both files.
|
||
|
||
What landed (148 LOC of doc additions, no non-doc lines
|
||
changed, verified by filtering the diff):
|
||
|
||
- `ailang-codegen` crate root got intra-doc-link upgrades
|
||
(`[`emit_ir`]`, `[`lower_workspace`]`,
|
||
`[`CodegenError::MissingEntryMain`]`) and a precondition
|
||
sentence — both entry points assume their input has already
|
||
passed the typechecker; codegen does not call the checker
|
||
itself.
|
||
- Every `CodegenError` variant got a `///` string naming the
|
||
AST term/condition that triggers it. Same shape as
|
||
`CheckError` post-13e, so the two error enums now read
|
||
similarly and a reader can grep across them. `Internal` is
|
||
flagged in its doc as a catch-all that covers ~30
|
||
invariant-violation sites.
|
||
- `emit_ir` and `lower_workspace` now make the
|
||
single-vs-multi-module split explicit and cross-link to
|
||
each other.
|
||
- `ail/main.rs` module header expanded from a 5-line stub to
|
||
a full subcommand list with one-liners (`manifest`,
|
||
`render`, `describe`, `deps`, `check`, `emit-ir`, `build`,
|
||
`run`, `builtins`, `diff`, `workspace`), the `clang`-on-PATH
|
||
prerequisite for `build`/`run`, the design-intent paragraph
|
||
about each subcommand being narrowly scoped for LLM
|
||
consumption (already partly there), and an explicit "no
|
||
`pub` items, `--help` text comes from clap" note for anyone
|
||
who lands here from rustdoc.
|
||
|
||
**Findings reported** (judgement deferred to me):
|
||
|
||
- `CodegenError::Internal(String)` is a single opaque catch-all
|
||
for ~30 distinct invariant-violation sites (mono-queue
|
||
desync, ctor-index miss, lambda-env shape, ...). Tests can
|
||
only substring-match on it. **Not fixing**: splitting is a
|
||
test-ergonomics decision, not a doc one. Worth raising the
|
||
next time codegen tests get a serious rewrite.
|
||
- `emit_ir` synthesises an internal `Workspace` with
|
||
`root_dir = "."`. No codegen path reads `root_dir` today, so
|
||
this is harmless; if a future feature reaches `root_dir`
|
||
from codegen, the assumption surfaces. **Not fixing**: the
|
||
agent flagged it correctly as "would change behaviour, out
|
||
of scope".
|
||
|
||
**Process note: brief drift caught by the agent.** I told the
|
||
docwriter the CLI had nine subcommands; it found eleven
|
||
(`Deps` and `Diff` were missing from my brief, which was
|
||
written off a `head -40` of the source). Agent silently
|
||
corrected and flagged the drift in its findings. Useful
|
||
counter-pressure to the orchestrator pattern: my survey was
|
||
sloppy and the agent did not propagate the sloppiness into
|
||
the doc. This is one of the things sub-agents are good at and
|
||
why I keep delegating even on small jobs.
|
||
|
||
**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests
|
||
(unchanged). `cargo doc --no-deps`: 0 warnings. Also verified
|
||
under `RUSTDOCFLAGS='-D rustdoc::broken_intra_doc_links'` per
|
||
agent report. `cargo build --workspace` green. `cargo test
|
||
--workspace` green.
|
||
|
||
**Workspace-wide rustdoc invariant achieved.** All four
|
||
crates (`ailang-core`, `ailang-check`, `ailang-codegen`,
|
||
`ail`) now have:
|
||
- crate-root `//!` that names ownership, position in
|
||
pipeline, and entry points;
|
||
- module-root `//!` on every file with non-trivial content;
|
||
- `///` on every public item (struct, enum, variant, fn,
|
||
field, const) that names the contract, not just the type;
|
||
- intra-doc links wherever prose previously referred to
|
||
another item by name.
|
||
|
||
DESIGN.md item 6 ("rustdoc cleanliness") is now load-bearing
|
||
across the whole workspace, not just `core`. The
|
||
`ailang-docwriter` agent's job from here is **maintenance**:
|
||
run after any iter that adds public surface, not full sweeps.
|
||
|
||
**Next.** 14a is now unblocked: write a polymorphic `list_map`
|
||
that uses `List a` (Iter-13a parameterised ADTs) and `Maybe a`,
|
||
then extend an existing demo program (`hello_print` or one of
|
||
the dogfood sources) to call it end-to-end. That exercises
|
||
the parameterised-ADT pipeline through type-check, codegen,
|
||
and runtime — the missing piece in 13a/b/c was that the
|
||
feature shipped but no real program used it. If context
|
||
budget at the start of 14a is tight, alternative is 14b
|
||
(GC/arena scaffolding) or 14c (poly fn as value); both have
|
||
real design questions that need an orchestrator design pass
|
||
first, so 14a stays the default.
|
||
|
||
## Iter 14a — polymorphic `List a` end-to-end + monomorphisation bug fixed
|
||
|
||
The dogfood payoff for parameterised ADTs (13a/b/c). Prior
|
||
to this iter, the only programs exercising the feature were
|
||
`box.ail.json` (single `MkBox(42)` round-trip) and
|
||
`maybe_int.ail.json` (single `or_else` call). That is a thin
|
||
slice — neither program builds a recursive parameterised ADT
|
||
nor calls a polymorphic higher-order fn. 14a closes that gap
|
||
with `data List a` + `map : forall a b. ((a) -> b, List<a>) ->
|
||
List<b>` recursive, then prints the result.
|
||
|
||
Three-agent run (tester → debugger → no implementer needed):
|
||
|
||
1. **Tester** wrote `examples/list_map_poly.ail.json` (5 defs:
|
||
`List`, `inc`, `map`, `print_list`, `main`) and a new e2e
|
||
test `list_map_poly_inc_then_prints` that asserts stdout
|
||
`["2", "3", "4"]`. Typecheck passed, build crashed:
|
||
`internal: monomorphisation: var \`a\` bound to two
|
||
distinct types`. Tester correctly stopped — fixture
|
||
encodes the contract; bug is in the compiler — and
|
||
reported with a hypothesis (recursion / shared
|
||
substitution slot in `map`).
|
||
|
||
2. **Debugger** refuted the hypothesis with a non-recursive
|
||
repro (`Cons(7, Nil)` triggers the same crash without
|
||
`map` in the picture at all). Real cause was much smaller:
|
||
in `synth_arg_type` (codegen, ~line 2087) for `Term::Ctor`,
|
||
any type var of the parent ADT that the ctor's args
|
||
couldn't pin was filled with `Type::unit()` as a
|
||
placeholder. For nullary ctors of a parameterised ADT
|
||
(`Nil : List<a>`, `None : Maybe<a>`) that placeholder
|
||
leaked upward. Inside a parent like `Cons(Int, Nil) :
|
||
List<a>`, `unify_for_subst` would walk
|
||
`cref.ail_fields = [Var{a}, Con{List,[Var{a}]}]` against
|
||
`[Int, Con{List,[Unit]}]`, bind `a = Int` from the head,
|
||
then collide with `a = Unit` from the tail. The recursive
|
||
`map` fixture surfaced it because it is the first program
|
||
to nest a nullary ctor of a parameterised ADT inside a
|
||
parent ctor — the existing 13b regressions never did. The
|
||
tester's hypothesis was reasonable from the symptom but
|
||
wrong on mechanism; refuting it via a smaller repro is
|
||
exactly the discipline the debugger role is for.
|
||
|
||
3. **Fix** (`crates/ailang-codegen/src/lib.rs`, +30/-9 LOC,
|
||
no new variant, no API change):
|
||
- Replace `Type::unit()` placeholder with a synth-only
|
||
wildcard `Type::Var { name: "$u" }`. The `$u` prefix is
|
||
a reserved-namespace convention that mirrors the
|
||
checker's `$m<id>` for instantiation metavars — same
|
||
trick (source-level identifiers can't start with `$`),
|
||
same goal (extra semantics without schema change).
|
||
- `unify_for_subst` short-circuits on a `$u`-prefixed
|
||
**arg-side** var: accept without binding, let a sibling
|
||
arg pin the type var instead. Param-side semantics
|
||
untouched. `derive_substitution`'s "var not pinned"
|
||
check still fires for genuinely under-determined calls
|
||
(those would have failed typechecking, so codegen never
|
||
sees them, but the safety net stands).
|
||
|
||
**Tests:** 25/25 e2e (was 24, +1 new poly test). All five
|
||
named regressions stayed green: `list_map_doubles_then_prints`,
|
||
`parameterised_box_round_trip`, `parameterised_maybe_match`,
|
||
`polymorphic_id_at_int_and_bool`, `polymorphic_apply_with_fn_param`.
|
||
Insertion sort still green. `cargo doc --no-deps`: 0 warnings
|
||
(workspace invariant from 13d/e/f preserved). `cargo build
|
||
--workspace` green.
|
||
|
||
**Process note: tester→debugger→done in one iter.** No
|
||
implementer dispatch needed — the debugger's mandate covers
|
||
"propose **and apply** a minimally invasive fix" once the
|
||
diagnosis is solid. 30 LOC across two sites in one file is
|
||
inside the role's scope (the role doc says "stop and report"
|
||
only on >50 LOC across multiple files). The orchestrator-side
|
||
work was the design (the source program), the scoped briefs,
|
||
and verification. This is the cleanest agent-flow shape so
|
||
far: each agent did exactly its job, the tester's wrong
|
||
hypothesis didn't propagate because the debugger tested it,
|
||
and the orchestrator never wrote compiler code.
|
||
|
||
**Findings flagged** (judgement deferred, not fixed):
|
||
|
||
- `CodegenError::Internal` (the catch-all string variant the
|
||
docwriter flagged in 13f) is now used by one more invariant
|
||
— the `$u` short-circuit could in principle be reached by a
|
||
malformed input; it's currently silent because typecheck
|
||
rejects under-determined calls upstream. Worth splitting
|
||
into typed variants the next time codegen tests get a real
|
||
rewrite. (Same finding as 13f, now reinforced by another
|
||
use site.)
|
||
- The `$u` / `$m` reserved-prefix convention (`$u` for
|
||
codegen synth wildcards, `$m<id>` for checker metavars)
|
||
is undocumented as a project-wide naming rule. Two
|
||
prefixes is fine; if a third appears, this should be
|
||
promoted to a DESIGN.md note.
|
||
|
||
**Next.** Parameterised ADTs are now genuinely usable for
|
||
real programs. The standard library starter set (an
|
||
`examples/std_*` series with `List`, `Maybe`, `Either`,
|
||
basic combinators `length`, `filter`, `fold`, `concat`)
|
||
becomes worthwhile in a way it wasn't pre-14a — that's a
|
||
plausible 14b alternative, smaller than GC/arena, and would
|
||
itself surface more dogfood bugs. The original 14b
|
||
(GC/arena) and 14c (poly fn as value) remain on the queue
|
||
but have not had a design pass.
|
||
|
||
## Iter 14b — design pass for the authoring surface
|
||
|
||
User redirected the project at the iter boundary. I had been
|
||
about to write a stdlib in JSON-AST form; user pushed back: do
|
||
I really program *best* in JSON, given the language is supposed
|
||
to be the one I'm most accurate in? Honest answer was no — JSON
|
||
was rationalisation. The constraint added in this iter:
|
||
|
||
> "Die Syntax sollte gut formalisierbar sein, damit man auch
|
||
> einem fremden LLM eine Spec geben kann, die es dann fehlerfrei
|
||
> befolgt."
|
||
|
||
That single sentence ruled out about half the design space:
|
||
anything with operator precedence (precedence is the #1 thing
|
||
an LLM gets wrong when handed a spec, because it requires
|
||
building a parse-tree mental model rather than just following
|
||
production rules), anything with semantic indentation, anything
|
||
with maximal-munch lexing or context-sensitive reductions.
|
||
|
||
What remains is roughly: S-expressions, or dialects close to
|
||
S-expressions. Decision 6 in DESIGN.md captures the constraints
|
||
in priority order, sketches three candidate forms (A: tagged
|
||
S-expression, B: indented record-style, C: pretty-printer-as-
|
||
source), and picks (A) as the first attempt with explicit
|
||
rollback to (C) if (A) hurts authoring more than it helps.
|
||
|
||
**Key shape of (A):** every AST node has a unique head keyword.
|
||
No case-disambiguation rule (no "capitalised head means ctor").
|
||
Bare atoms in positional slots get their sort from the parent
|
||
slot (inside `(con NAME args...)` second-and-later positions
|
||
are types; inside `(app HEAD args...)` first position is a term;
|
||
etc.). To construct a value with a ctor, write
|
||
`(term-ctor TypeName CtorName args...)` explicitly. To match,
|
||
`(pat-ctor CtorName fields...)`. The capitalisation/case of an
|
||
identifier carries no semantic weight to the parser.
|
||
|
||
This rules out the silent-error class "I forgot to capitalise
|
||
`Cons` and it parsed as a function call" — which I had been
|
||
relying on case-conventions to prevent in the JSON form too,
|
||
but only by hand-discipline. With explicit tags the discipline
|
||
becomes a parse rule.
|
||
|
||
**Empirical test (this iter).** Hand-encoded three examples in
|
||
form (A) as `examples/*.ailx`:
|
||
|
||
```
|
||
hello.ailx — 5 LOC (was JSON: 36 pretty / 21 canonical)
|
||
box.ailx — 25 LOC (was JSON: 160 pretty / 88 canonical)
|
||
list_map_poly.ailx — 50 LOC (was JSON: 394 pretty / 230 canonical)
|
||
```
|
||
|
||
Roughly 4–8× line reduction, ~4× character reduction. Bigger
|
||
gains on bigger programs (overhead is proportional to AST
|
||
depth, not to program size, so the form scales well). All three
|
||
mapped unambiguously to AST nodes by inspection — no
|
||
ambiguities surfaced during writing that the spec didn't
|
||
already cover.
|
||
|
||
Two small lex/grammar issues I discovered while writing and
|
||
folded back into DESIGN.md before committing:
|
||
|
||
1. **Operator idents** like `+`, `==`, `<=`. Initial spec had a
|
||
word-shaped `[A-Za-z_]...` regex; that excluded operators.
|
||
Fix: lexer recognises only `(`/`)`/whitespace as delimiters.
|
||
Every other maximal token is classified by first character
|
||
(digit ⇒ integer, `"` ⇒ string, else ⇒ ident). `+`, `42`,
|
||
`io/print_int`, `==` all become single ident or integer
|
||
tokens with no special rule.
|
||
2. **Bool literals.** Bare `true`/`false` are reserved in term
|
||
context; outside term context they would be ill-formed
|
||
anyway. Unit is explicit: `(lit-unit)`.
|
||
|
||
**Files touched:**
|
||
- `docs/DESIGN.md` — Decision 6 added (~140 lines), with
|
||
constraints, three candidates, first-choice rationale, and
|
||
an implementation outline for Iter 14c.
|
||
- `examples/hello.ailx`, `examples/box.ailx`,
|
||
`examples/list_map_poly.ailx` — three design exhibits. Not
|
||
parseable yet (header comment says so).
|
||
- `docs/JOURNAL.md` — this entry.
|
||
|
||
**Tests:** None new. Existing 25/25 e2e + 3 ignored doctests
|
||
unchanged (this iter is paper, not code). `cargo doc --no-deps`
|
||
0 warnings (workspace invariant from 13d/e/f preserved).
|
||
|
||
**Process note: orchestration with explicit licence to be
|
||
wrong.** User said "du darfst auch ausprobieren und dich irren
|
||
(und es dann rückgängig machen). Wir wollen das beste Design
|
||
für den propagierten Zweck." That changes the cost model for
|
||
design iteration: I should optimise for *information per iter*,
|
||
not for *correctness on first commit*. So I committed form (A)
|
||
as a working hypothesis with a documented rollback path to
|
||
form (C), rather than design-by-committee until I was sure.
|
||
|
||
**Plan 14c (next).**
|
||
1. New crate `ailang-surface` with a small PEG parser → existing
|
||
`ailang-core::ast` types. No new AST nodes.
|
||
2. Round-trip test gate: every existing `examples/*.ail.json`
|
||
gets a sibling `*.ailx` written by hand or by an
|
||
AST→surface emitter; the test parses the surface, canonises
|
||
to JSON, and asserts hash-equivalence to the original. If a
|
||
single fixture loses its hash, the form does not ship.
|
||
3. CLI: `ail parse <file.ailx> -o <file.ail.json>`. Symmetric
|
||
to existing `ail render`.
|
||
4. If round-trip works for all current fixtures, mark form (A)
|
||
confirmed and start the stdlib in `.ailx` directly. If it
|
||
fails, document why in JOURNAL and try (C).
|
||
|
||
**Plan 14d (after 14c).** First stdlib module: `std_list.ailx`
|
||
with `length`, `filter`, `fold`, `concat`, `reverse`, `head`,
|
||
`tail`. Each combinator is a fresh test vector for codegen and
|
||
for the still-young parameterised-ADT pipeline. Written in the
|
||
new surface from day one — no JSON authoring of stdlib.
|
||
|
||
## Iter 14c — `ailang-surface` parser + pretty-printer ships
|
||
|
||
Implements Decision 6's form (A). New crate `ailang-surface`
|
||
(~1843 LOC across `lex.rs`, `parse.rs`, `print.rs`, lib root,
|
||
plus tests) ships as a strictly additive producer of
|
||
`ailang-core::ast::Module` values. `ailang-check` and
|
||
`ailang-codegen` were not modified — projection-agnostic, as
|
||
the architectural pin requires.
|
||
|
||
**Implementer dispatch went clean.** Brief gave the EBNF, the
|
||
fixtures to round-trip, the lexer rule, and the architectural
|
||
constraints. Implementer hand-wrote a recursive-descent parser
|
||
(one Rust fn per EBNF production), a deterministic pretty-
|
||
printer, an integration test that runs the round-trip gate on
|
||
every fixture, and the `ail parse` CLI subcommand. No
|
||
parser-combinator dependency. No AST-shape changes.
|
||
|
||
**Two AST-driven form refinements** that were not in the 14b
|
||
sketch (both folded into DESIGN.md Decision 6 by the
|
||
implementer before commit):
|
||
|
||
1. `lam-term` had to carry `param_tys`, `ret_ty`, and
|
||
`effects` because the AST's `Term::Lam` stores parallel
|
||
typed-param data. Production became `(lam (params (typed
|
||
x Int) ...) (ret T) (effects ...) (body ...))`. Same
|
||
shape-style as the rest of the form; no new lex rules.
|
||
2. `import-clause` had to admit `Option<String>` aliases.
|
||
Production became `(import name (as alias)?)` with `as`
|
||
as a bare ident token. No new lex rules.
|
||
|
||
The 14b 30-production budget held: implementer reports ~28
|
||
named productions in the parser. Constraint 1 (formalisable
|
||
for foreign LLM) intact.
|
||
|
||
**Verification gates, all green:**
|
||
|
||
- `cargo build --workspace`: 0 warnings, finished.
|
||
- `cargo test --workspace`: 76 tests pass (was 64; +9 unit
|
||
tests in `ailang-surface`, +2 integration tests in
|
||
`tests/round_trip.rs`, +1 e2e regression preserved). All
|
||
17 `examples/*.ail.json` fixtures round-trip
|
||
byte-identical through `print → parse → canonical JSON`;
|
||
3 hand-written `.ailx` exhibits parse to canonical JSON
|
||
byte-identical to their `.ail.json` siblings.
|
||
- `cargo doc --no-deps`: 0 warnings (workspace invariant
|
||
from 13d/e/f preserved; new crate's rustdoc landed
|
||
correctly with crate-root `//!` plus all `pub` items
|
||
documented).
|
||
|
||
**Manual smoke test (orchestrator-side after agent reported
|
||
done):** `ail parse <file.ailx> -o <file.ail.json>` followed
|
||
by `ail run <file.ail.json>` for all three exhibits:
|
||
|
||
```
|
||
hello.ailx → "Hello, AILang."
|
||
box.ailx → "42"
|
||
list_map_poly.ailx → "2\n3\n4"
|
||
```
|
||
|
||
End-to-end pipeline form (A) → AST → typecheck → codegen →
|
||
clang → binary works on all three.
|
||
|
||
**Important non-issue.** `examples/*.ail.json` fixtures on
|
||
disk are hand-formatted JSON with non-canonical key order;
|
||
the parser produces canonical (lex-sorted) JSON. Diff at
|
||
the file-byte level is not zero. Diff at the canonical-byte
|
||
level is zero — which is the only thing that matters per
|
||
Decision 1, since hashing uses canonical form. The
|
||
round-trip test gates on canonical bytes, not file bytes.
|
||
This is correct behaviour; flagged here so a future reader
|
||
who runs `diff` doesn't think the surface is broken.
|
||
|
||
**Files created:**
|
||
|
||
- `crates/ailang-surface/Cargo.toml`
|
||
- `crates/ailang-surface/src/lib.rs` (~40 LOC, rustdoc heavy)
|
||
- `crates/ailang-surface/src/lex.rs` (~264 LOC)
|
||
- `crates/ailang-surface/src/parse.rs` (~1041 LOC, one fn
|
||
per production)
|
||
- `crates/ailang-surface/src/print.rs` (~371 LOC)
|
||
- `crates/ailang-surface/tests/round_trip.rs` (~128 LOC)
|
||
|
||
**Files modified:**
|
||
|
||
- `Cargo.toml` (workspace) — `ailang-surface` member +
|
||
workspace dep.
|
||
- `Cargo.lock` — refresh.
|
||
- `crates/ail/Cargo.toml` — `ailang-surface` dep.
|
||
- `crates/ail/src/main.rs` — new `Parse { path, output }`
|
||
subcommand (~36 LOC).
|
||
- `docs/DESIGN.md` — form refinements appendix to
|
||
Decision 6.
|
||
- `examples/list_map_poly.ailx` — implementer added doc
|
||
strings to match the JSON original (was a 14b design
|
||
exhibit, not byte-aligned).
|
||
|
||
**Known debt:** none reported. `ailang-check` /
|
||
`ailang-codegen` untouched per the architectural pin.
|
||
|
||
**Plan 14d (next, queued).** With form (A) now ergonomic
|
||
and round-trip-verified, the std-lib path opens up. First
|
||
target: `examples/std/std_list.ailx` with `length`,
|
||
`filter`, `fold` (left and right), `concat`, `reverse`,
|
||
`head`, `tail`. Each combinator a fresh test vector for
|
||
the parameterised-ADT pipeline (which 14a opened end-to-
|
||
end). Authored in form (A); the resulting `.ail.json` is
|
||
what tests consume. If 14d surfaces more codegen bugs in
|
||
the parameterised-ADT path (it likely will — 14a found
|
||
one already), debugger handles them inline.
|
||
|
||
The 14b/c form-A hypothesis has held under the empirical
|
||
test of round-tripping every existing fixture. The
|
||
documented rollback to form (C) is now off the table for
|
||
this iter cycle, though the architectural pin keeps it
|
||
open for future replacement of `ailang-surface` should
|
||
the form prove inadequate at stdlib scale.
|
||
|
||
## Language-completion sequence (14d → 14f, then stdlib in 15a)
|
||
|
||
User redirected at the 14d boundary: write the language to
|
||
"finished" before starting on a stdlib. Reasoning:
|
||
authoring a stdlib in an unfinished language wastes work —
|
||
each gap discovered later forces a rewrite of code already
|
||
written. The user also confirmed: **no schema version bump
|
||
needed**. AILang has exactly one consumer (me), so version
|
||
ceremony for compatibility management is pure overhead.
|
||
Edit AST and fixtures in place; pin new hashes where the
|
||
hash regression test demands it.
|
||
|
||
Updated planning sequence:
|
||
|
||
- **14d** — remove `Term::If` redundancy. Pure subtraction.
|
||
- **14e** — explicit tail-call annotation (`tail` flag on
|
||
`Term::App`/`Term::Do`, `musttail` in codegen, tail-position
|
||
verifier in checker).
|
||
- **14f** — memory management. Currently every ADT allocation
|
||
leaks. Likely Boehm conservative GC (`GC_malloc` + `-lgc`)
|
||
for minimum surface change; design pass first.
|
||
- **15a** — first stdlib module (`std_list`).
|
||
|
||
Deferred (not stdlib-blocking; can land later without
|
||
rewriting code that already exists): records/tuples (use
|
||
ADT pairs), nested patterns (use pyramid `match`), local
|
||
recursive `let` (hoist to top level).
|
||
|
||
## Iter 14d — `Term::If` removed
|
||
|
||
Subtraction iter. `Term::If { cond, then, else_ }` was
|
||
semantically a subset of `Term::Match` on `Bool`. CLAUDE.md
|
||
forbids redundancies; two AST nodes for the same operation
|
||
was an authoring decision with no semantic content and a
|
||
duplicate codegen path.
|
||
|
||
The migration shape was the canonical one named in the
|
||
brief:
|
||
|
||
```
|
||
(if c a b) → (match c (case (lit-bool true) a) (case _ b))
|
||
```
|
||
|
||
Wildcard arm satisfies the existing `primitive-needs-wildcard`
|
||
rule. A future iter may upgrade exhaustiveness to recognise
|
||
`true`+`false` as covering Bool without a wildcard, but the
|
||
wildcard form works with the current checker and that was
|
||
enough for 14d.
|
||
|
||
**Implementer dispatch went clean with one documented
|
||
deviation.** Removing the `Term::If` codegen path was not
|
||
sufficient on its own: the existing match codegen rejects
|
||
`i1` (Bool) scrutinees and `Pattern::Lit` patterns — both
|
||
of which the migration shape requires. The implementer
|
||
added a tightly-scoped `lower_bool_match` helper (~95 LOC)
|
||
that handles **only** the two-arm Bool migration shape
|
||
(`(lit-bool true) -> A | _ -> B` or its mirror), errors on
|
||
anything else, and emits the same `br i1`/phi IR the old
|
||
`Term::If` path emitted. No generalisation of the
|
||
ADT-match codegen.
|
||
|
||
The deviation was the right call. **Lesson for future
|
||
subtraction iters**: when removing a specialised AST node,
|
||
the codegen for the migration target may need a small
|
||
extension. Pre-emptively scope this in the brief next time.
|
||
|
||
**Diff size**: 13 files, +286/-221 LOC. Net +65 LOC across
|
||
the workspace, but the AST got smaller (one variant gone),
|
||
the form-(A) grammar got smaller (one production gone),
|
||
and the typechecker got smaller (one branch gone). Codegen
|
||
got slightly larger because of the bool-match helper, but
|
||
the alternative was reusing the existing match path and
|
||
generalising it — which would have been a bigger and
|
||
riskier change.
|
||
|
||
**Hash deltas** (intentional, per Decision 7):
|
||
|
||
| def | before | after |
|
||
|---|---|---|
|
||
| `sum.sum` | `db33f57cb329935e` | `7f5fe7f72c63a9fd` |
|
||
| `sort.insert` | `697fcb9f30f8633a` | `07ff6ee7db17565d` |
|
||
| `max3.max` | `65c45d6a45dd0a72` | `2aa1576f3fbf5b3d` |
|
||
| `max3.max3` | `624b14429bf302f5` | `c452ec2e36c0af27` |
|
||
|
||
Untouched defs (e.g. `sum.main`, all `sort.*` except
|
||
`insert`, `sort.IntList`, `sort.print_list`, `max3.main`)
|
||
keep bit-identical hashes. That's the canary that the
|
||
canonical-JSON byte format was not perturbed — only the
|
||
migrated bodies changed identity.
|
||
|
||
**Verification**: 76/76 tests green (unchanged count; no
|
||
new tests in this iter, by design — subtraction). Manual
|
||
smoke: `sum` → `55`, `max3` → `17`, `sort` → ordered list.
|
||
Identical to pre-migration stdout for all three. `cargo
|
||
doc --no-deps` 0 warnings.
|
||
|
||
**Tail-call survey from the implementer (gold finding,
|
||
informs 14e).** While reading the migrated fixtures the
|
||
implementer surveyed tail positions. Result:
|
||
|
||
- `print_list` (in both `sort.ail.json` and
|
||
`list_map_poly.ail.json`): the recursive call is the
|
||
rhs of a `seq` which is the body of a match arm —
|
||
**already in tail position**. TCO would convert these
|
||
to actual loops.
|
||
- `main` chains: the outer call is in tail position;
|
||
inner calls are not.
|
||
- `insert`, `sort`, `map`: the recursive calls are
|
||
**arguments to a `Cons` ctor construction** (e.g.
|
||
`Cons (f h) (map f t)`). NOT in tail position.
|
||
Constructor-blocking is the standard ML/Haskell case
|
||
where TCO does not apply without a CPS transform or an
|
||
accumulator-form rewrite.
|
||
|
||
**Implications for 14e.** Adding a `tail` flag to
|
||
`Term::App`/`Do` will work for `print_list`-style
|
||
recursions and for terminal call chains, but **will not**
|
||
help the `map`/`sort`/`insert`-style ctor-blocked
|
||
recursions. Those need either an accumulator-form rewrite
|
||
in the source program (the standard ML/Haskell move) or a
|
||
CPS transform (much more intrusive). 14e ships only the
|
||
annotation + verification; accumulator forms become an
|
||
authoring pattern in the stdlib, not a compiler feature.
|
||
|
||
This sharpens what 14e can promise: tail-call **wins**
|
||
will be visible in `print_list`-style terminal-recursion
|
||
patterns; `map`/`sort` style stays stack-bounded by depth,
|
||
which makes 14f (GC) the more important iter for handling
|
||
long lists than 14e by itself.
|
||
|
||
## Iter 14e — explicit, verified tail calls
|
||
|
||
Decision 8 ships. `Term::App` and `Term::Do` carry a `tail: bool`
|
||
flag (serde-default false, skip-when-false in serialisation).
|
||
A new `verify_tail_positions` typecheck pass walks each fn body
|
||
and Lam body with an `is_tail_context: bool` threaded down per
|
||
the standard Scheme rules, rejecting any `tail: true` call that
|
||
sits outside tail position. Codegen emits `musttail call` for
|
||
marked App calls.
|
||
|
||
**Hash deltas (intentional, only the migrated calls):**
|
||
|
||
| def | hash before | hash after |
|
||
|---|---|---|
|
||
| `list_map_poly.print_list` | unchanged 14c value | new |
|
||
| `sort.print_list` | unchanged 14d value | new |
|
||
|
||
All other defs across all 18 fixtures kept bit-identical hashes.
|
||
This is the canary for the `skip_serializing_if = "is_false"`
|
||
serde rule: an unmarked `App`/`Do` serialises identically to its
|
||
pre-14e form, so untouched defs cannot drift.
|
||
|
||
**Tests: 79/79 (was 76, +3).** New tests:
|
||
- `tail_call_in_non_tail_position_is_rejected` (check unit) —
|
||
asserts the diagnostic fires on a deliberately-misplaced
|
||
`tail: true` call (e.g. as a `Cons` arg).
|
||
- `tail_call_in_tail_position_is_accepted` (check unit) —
|
||
asserts the verifier accepts the canonical `print_list` shape.
|
||
- `iter14e_print_list_recursion_emits_musttail` (e2e IR-grep) —
|
||
builds `list_map_poly`, dumps IR, asserts the recursive call
|
||
site uses `musttail call`. This is the only direct evidence
|
||
that the AST flag actually reaches LLVM.
|
||
|
||
**IR-snapshot evidence** at the recursive site of
|
||
`print_list` after migration:
|
||
|
||
```
|
||
%v7 = musttail call i8 @ail_list_map_poly_print_list(ptr %v6)
|
||
ret i8 %v7
|
||
```
|
||
|
||
`musttail` followed immediately by `ret` of the same SSA value —
|
||
LLVM's terminator rule satisfied. Smoke: `list_map_poly` →
|
||
`2/3/4`, `insertion_sort_orders_list` → identical sorted list.
|
||
Behavior unchanged; only the calling shape did.
|
||
|
||
**Two implementer deviations (called out, both reasonable):**
|
||
|
||
1. **`tail-do` falls back to `tail call`, not `musttail`.** LLVM
|
||
`musttail` requires identical caller/callee return types. AILang
|
||
IO ops dispatch through runtime helpers (`printf`/`puts`)
|
||
returning `i32`, while AILang's `Unit` lowers to `i8`. Cross-type
|
||
`musttail` would be rejected by the verifier. So `Term::Do` with
|
||
`tail: true` lowers to `tail call` (the LLVM optimisation hint,
|
||
not the guarantee), then `ret i8 0`. No fixture currently uses
|
||
`tail-do`, so the path is implemented but not exercised
|
||
end-to-end. Proper fix: change runtime helper signatures to
|
||
return `i8`. Punted; not blocking.
|
||
|
||
2. **`block_terminated` plumbing in codegen.** A `tail-app` /
|
||
`tail-do` emits `musttail call ... ret ...` directly and sets
|
||
`self.block_terminated = true`. Surrounding code (match-arm phi
|
||
construction, fn-body trailing-ret, lambda-thunk trailing-ret)
|
||
checks the flag and skips the fall-through emit. When every
|
||
match arm is a tail call, the join block is omitted entirely.
|
||
This was unavoidable to keep the IR well-formed — adding a
|
||
second `ret` after a `musttail call`+`ret` would be a verifier
|
||
error. The flag is a small piece of state but it's the right
|
||
shape for "the current basic block has been definitively
|
||
terminated by a sub-emission".
|
||
|
||
**Form (A) at constraint ceiling.** Two new productions
|
||
(`tail-app-term`, `tail-do-term`) bring the count to ~30, which
|
||
is exactly the constraint-1 budget. Future productions need to
|
||
either retire something or accept an explicit budget rebalancing
|
||
in DESIGN.md.
|
||
|
||
**GC notes from the implementer (informs 14f).**
|
||
|
||
- **Allocations cluster in `lower_ctor`** (~line 850 of codegen).
|
||
Every `term-ctor` does `malloc(8 + 8 * n)`. In `print_list` we
|
||
allocate nothing per recursion (just match + read fields +
|
||
recurse); allocations come from `map`, from `main`'s
|
||
list-building Cons chain, and from any other user code that
|
||
builds ADTs.
|
||
- **Lambda envs and closure pairs allocate too** (`lower_lambda`).
|
||
Closure pair: `malloc(16)`. Env block: `malloc(env_size)`.
|
||
Direct-application closures (the common case for HOF args)
|
||
could be arena'd cleanly because the closure dies after the
|
||
call returns. Stored or returned closures escape.
|
||
- **Tail recursion does NOT reduce allocation pressure**, only
|
||
stack depth. For `print_list`-style recursions there's no
|
||
allocation to begin with, so the win is purely stack-bounded.
|
||
For `map`-style ctor-blocked recursions, each step allocates
|
||
one new `Cons` box — that's where allocation-side work pays
|
||
off.
|
||
- **The "obviously safe" arena boundary** is a fn whose return
|
||
type contains no boxed ADT (i.e. returns `Int`/`Bool`/`Unit`/
|
||
`Str` only). All ADT boxes allocated inside such a fn cannot
|
||
escape; an arena freed at fn return is sound by construction.
|
||
Most current fixtures violate this — `map`, `sort` return ADTs
|
||
— so a per-fn-arena scheme alone won't carry. Need a heap
|
||
with GC for escaping allocations.
|
||
|
||
This narrows 14f's design space: probably **Boehm conservative
|
||
GC across the board** as a first cut (`GC_malloc` substituted
|
||
for `malloc`, `-lgc` linked, no language change). Add a
|
||
per-fn-arena optimisation later for non-escaping cases if the
|
||
profile justifies it. Boehm is a one-iter shot; arenas would be
|
||
a multi-iter design pass with escape analysis.
|
||
|
||
**Plan 14f.** Boehm-GC integration. Concretely:
|
||
- `GC_malloc` instead of `malloc` in lowered IR.
|
||
- `-lgc` added to the clang link command in `ailang-codegen`'s
|
||
build path (probably in the CLI, since the codegen crate
|
||
emits IR text and clang is invoked downstream).
|
||
- Conservative scan handles AILang's stack and globals out of
|
||
the box.
|
||
- Verify on a stress test: build a list of 100k Cons cells in
|
||
`map`, run, observe RSS doesn't blow up. Boehm collects
|
||
unreachable boxes during allocation pressure.
|
||
- No AST or schema change. No language-level change.
|
||
|
||
After 14f, the language is "done enough" for stdlib (15a).
|
||
Anything else (records as a primitive, nested patterns, local
|
||
recursive let, type classes) can layer on later without forcing
|
||
stdlib rewrites.
|
||
|
||
## Iter 14f — Boehm conservative GC
|
||
|
||
Decision 9 ships. Through Iter 14e every ADT box, lambda env,
|
||
and closure pair was leaked. This iter substitutes
|
||
`GC_malloc` for `malloc` in all four IR allocation sites and
|
||
links `-lgc`. No language change, no AST change, no schema
|
||
change.
|
||
|
||
**Diff: 5 files, ~30 LOC net.**
|
||
|
||
- `crates/ailang-codegen/src/lib.rs`: 4× `@malloc` → `@GC_malloc`
|
||
(declare line + 3 call sites: `lower_ctor`, `lower_lambda`'s
|
||
env, `lower_lambda`'s closure pair).
|
||
- `crates/ail/src/main.rs`: `.arg("-lgc")` added to the clang
|
||
invocation in `build_to`.
|
||
- `crates/ail/tests/snapshots/{hello,sum,list,max3,ws_main}.ll`:
|
||
mechanical s/@malloc/@GC_malloc/, 9 occurrences across 5
|
||
files. The IR is bit-identical to pre-14f modulo this
|
||
substitution — exactly Decision 9's promise.
|
||
- `crates/ail/tests/e2e.rs`: new test
|
||
`gc_handles_recursive_list_construction` (+19 LOC).
|
||
- `examples/gc_stress.{ailx,ail.json}`: new fixture.
|
||
|
||
**Hash invariance verified.** Every existing fixture's def
|
||
hashes are unchanged. The codegen and link line are downstream
|
||
of canonical bytes; the AST schema didn't move; nothing on
|
||
disk in `examples/*.ail.json` was touched. The new
|
||
`gc_stress` module adds 4 new hashes, all unrelated.
|
||
|
||
**Tests: 80/80 (was 79).** Existing 79 produce byte-identical
|
||
stdout — only the allocator changed, semantics unchanged. New:
|
||
`gc_handles_recursive_list_construction` builds a List<Int> of
|
||
length 50 via recursive `Cons`, sums it (`1275`). Manual smoke:
|
||
|
||
- `gc_stress.ail.json` → `1275`.
|
||
- `list_map_poly` → `2 3 4` (unchanged).
|
||
- `sort` → sorted list (unchanged).
|
||
|
||
`cargo doc --no-deps`: 0 warnings (DESIGN.md item 6 invariant
|
||
preserved through nine iters of feature work).
|
||
|
||
**Pattern shape used in `gc_stress`** (caught a small typechecker
|
||
constraint). The first proposed shape `(case (lit-int 0) Nil)`
|
||
doesn't parse — `pat-lit` takes the bare literal token, not a
|
||
keyword-prefixed form, and `case` requires a pattern. Worked
|
||
shape: comparison-and-bool-match, mirroring `sort.ail.json`'s
|
||
`<=` arm:
|
||
|
||
```
|
||
(match (app == n 0)
|
||
(case (pat-lit true) (term-ctor List Nil))
|
||
(case (pat-wild) (term-ctor List Cons n (app build (app - n 1)))))
|
||
```
|
||
|
||
This is the canonical "if-then-else" pattern post-14d. Worth
|
||
flagging for the stdlib brief: predicates that need to branch
|
||
go through the `==` / `<` / `<=` builtin returning Bool, then
|
||
match on that Bool with a wildcard fallback. Three lines for
|
||
what `if` used to do in one — but uniform with the rest of the
|
||
language, no special case.
|
||
|
||
**GC integration notes.**
|
||
- `GC_INIT()` is **not needed** on this build host (Arch
|
||
with `libgc 1.5.6`). libgc auto-inits via
|
||
`__attribute__((constructor))`.
|
||
- No conservative-scan over-retention symptom observed: every
|
||
existing test's stdout byte-identical; behaviour preserved.
|
||
- `-lgc` alone is sufficient for the link; pthread/dl come in
|
||
transitively from libgc.so's NEEDED entries.
|
||
|
||
**Language is feature-complete enough for stdlib.** Iters 14d
|
||
(redundancy removal), 14e (explicit tail calls), 14f (GC) are
|
||
the three blockers identified at the 14b boundary. They are
|
||
all done. Anything else (records as primitive, nested patterns,
|
||
local rec let, type classes) layers on later without forcing
|
||
stdlib rewrite.
|
||
|
||
**Plan 15a.** First stdlib module: `examples/std/std_list.ailx`.
|
||
Combinators: `length`, `append`, `reverse`, `map`, `filter`,
|
||
`fold_left`, `fold_right`, `head`, `tail`, `is_empty`. Each
|
||
combinator a fresh test vector for the parameterised-ADT +
|
||
GC + tail-call combination. Authored in form (A) from day one;
|
||
`.ail.json` produced via `ail parse`. Each combinator gets a
|
||
dedicated e2e test.
|
||
|
||
If 15a surfaces compiler bugs (likely — every prior dogfood
|
||
iter has, see 14a's monomorphisation bug), debugger handles
|
||
them inline. If a compiler limitation surfaces that genuinely
|
||
blocks the stdlib (e.g. nested patterns turn out to be needed),
|
||
that becomes its own iter before 15a continues.
|
||
|
||
The architectural pin from Decision 6 governs: stdlib lives
|
||
under `examples/std/` as `.ailx` source; tests load the
|
||
generated `.ail.json`. `ailang-check` and `ailang-codegen`
|
||
remain projection-agnostic.
|
||
|
||
## Iter 14g — `Term::If` restored (revert of 14d)
|
||
|
||
Reconsidered 14d's removal of `Term::If`. The decision was wrong;
|
||
restored.
|
||
|
||
**Why 14d was wrong.** "No redundancies" from CLAUDE.md is a real
|
||
rule but it requires judgment to apply. `Term::If` reduces to
|
||
`Term::Match` on Bool, but reducibility is not redundancy in a
|
||
strong sense — `1 + 1` reduces to `2`, you don't remove `+` from
|
||
the language because of it. `Term::If` is a primitive control-
|
||
flow shape that every programming language has for good reason:
|
||
bool branching is the second most common control flow shape after
|
||
sequencing.
|
||
|
||
**Quantitative.** `(if c a b)` is 4 tokens. The post-14d
|
||
replacement `(match c (case (pat-lit true) a) (case (pat-wild) b))`
|
||
is 12. That's a 3× token-economy hit on every Bool branch — in
|
||
exactly the language whose authoring constraint was supposed to
|
||
be token-efficient. The match-on-Bool form is also asymmetric
|
||
(false case via `pat-wild` because the typechecker rejects
|
||
`pat-lit false` as exhaustive) and structurally lopsided.
|
||
|
||
**Meta-pattern that produced the wrong call.** I had been treating
|
||
user observations as directives. The user said "if is a subset of
|
||
match" — which is a factual observation; I jumped to remove it,
|
||
citing CLAUDE.md as cover. There was no independent conviction
|
||
behind the change, only doctrinal hooking-up of a user remark.
|
||
The leak showed up in 14f's JOURNAL prose ("three lines for what
|
||
`if` used to do in one"), which the user correctly read as me
|
||
regretting the decision.
|
||
|
||
Two feedback memories saved to head this off in future iters
|
||
(`/home/brummel/.claude/projects/-home-brummel-dev-ailang/memory/`):
|
||
- `feedback_user_suggestions_not_directives.md` — observations
|
||
are input, not output. Form an opinion before acting.
|
||
- `feedback_no_nostalgia_for_removed_features.md` — describe
|
||
canonical form on its own merits, not as compensation for
|
||
what was deleted.
|
||
|
||
**Implementation (revert).** Mechanically reverse-applied 14d's
|
||
diff at every site (`ast.rs`, `check/lib.rs` (incl. the new
|
||
14e `verify_tail_positions` arm), `codegen/lib.rs` (4 sites),
|
||
`surface/{parse,print}.rs`, `core/pretty.rs`, `ail/main.rs`,
|
||
`e2e.rs` test mutation). Removed the `lower_bool_match` helper
|
||
that 14d had introduced — it existed only because the 14d
|
||
migration shape needed codegen for non-`ptr` match scrutinees;
|
||
with `Term::If` back, match-on-Bool returns to its pre-14d
|
||
unsupported state and the helper is dead weight. Three fixtures
|
||
(`sum`, `sort`, `max3`) restored to their pre-14d shape.
|
||
|
||
**One additional fixture migration.** `gc_stress` was authored
|
||
in 14f using the 14d match-on-Bool migration shape (because
|
||
14f sat between 14d and this revert). After removing
|
||
`lower_bool_match` it would have failed to compile. Migrated
|
||
`gc_stress.{ail.json,ailx}` to use `(if ...)` directly. Output
|
||
unchanged: `1275`.
|
||
|
||
**14e and 14f are intact.** Verified by spot-emit of
|
||
`list_map_poly`'s IR: `musttail call i8 @ail_list_map_poly_print_list`
|
||
and `call ptr @GC_malloc(i64 8)` both present. The revert is
|
||
strictly local to `Term::If`-related code paths.
|
||
|
||
**Hash check.** All four pre-14d hashes returned:
|
||
|
||
| def | restored hash |
|
||
|---|---|
|
||
| `sum.sum` | `db33f57cb329935e` |
|
||
| `sort.insert` | `697fcb9f30f8633a` |
|
||
| `max3.max` | `65c45d6a45dd0a72` |
|
||
| `max3.max3` | `624b14429bf302f5` |
|
||
|
||
Untouched defs across all 18 fixtures (incl. the 14e print_list
|
||
hash deltas) keep their post-14f hashes. The revert's hash
|
||
movement is exactly the four 14d-migrated defs reverting plus
|
||
the one accidental 14f-victim (`gc_stress` defs, never previously
|
||
shipped under any other hash).
|
||
|
||
**DESIGN.md.** Decision 7 is preserved with a `Status: REVERTED`
|
||
header. Audit trail matters; future reads should see the
|
||
decision and its reversal both. Form-(A) productions in
|
||
Decision 6's appendix have `if-term` restored.
|
||
|
||
**Tests: 80/80 green.** Identical stdout for every existing
|
||
fixture. `cargo doc --no-deps` 0 warnings.
|
||
|
||
**LOC delta.** +265/-295 net −30. Net cleanup: the `lower_bool_match`
|
||
helper was bigger than the restored `Term::If` codegen.
|
||
|
||
**Plan.** Back to 15a — first stdlib module `std_maybe`. The
|
||
brief I had drafted included an "authoring note: post-14d
|
||
if-then-else" section that's now obsolete. Re-issue without
|
||
that, using `if` naturally where appropriate.
|
||
|
||
## Iter 14h — cross-module parameterised-ADT import (15a unblocked)
|
||
|
||
The 15a tester surfaced exactly the kind of bug a first-real-
|
||
stdlib-iter is supposed to surface: cross-module references to
|
||
types and ctors were not implemented. The Iter 5b cross-module
|
||
mechanism only carried fns + consts via `module_globals`; types
|
||
and ctors stayed module-local with an explicit comment in
|
||
`crates/ailang-check/src/lib.rs:703`: *"Register type defs (local
|
||
per module; cross-module ADT sharing is explicitly not part of
|
||
5b)"*. This iter completes that work using the same convention
|
||
as fns: **qualified-only access via `module.Name`**.
|
||
|
||
(Note on git tidiness: the `examples/std_maybe.ailx` file was
|
||
authored by the cancelled 15a tester dispatch and got swept into
|
||
the 14g commit by a `git add -A`. Should have spotted it pre-
|
||
commit. Not a correctness issue — the file was complete and
|
||
correctly authored — but a process-hygiene one. Will check the
|
||
diff carefully before staging next time.)
|
||
|
||
**The bug, surfaced by the 15a demo.**
|
||
|
||
```
|
||
ail check examples/std_maybe_demo.ail.json --json
|
||
[{"severity":"error","code":"unknown-type",
|
||
"message":"unknown type: `Maybe`","def":"main","ctx":{}}]
|
||
```
|
||
|
||
After qualifying the fn calls (`std_maybe.from_maybe`), fn refs
|
||
worked but the `(con Maybe (con Int))` and `(term-ctor Maybe
|
||
Just 7)` kept failing because `env.types` and `env.ctor_index`
|
||
are populated only from the current module.
|
||
|
||
**The fix.** Same shape as Iter 5b's fn solution, applied to types
|
||
and ctors:
|
||
|
||
- `Env` gains `module_types: BTreeMap<String, IndexMap<String,
|
||
TypeDef>>` populated by a sibling `build_module_types` to
|
||
`build_module_globals`. Lives in `check_in_workspace`'s
|
||
pre-check pass.
|
||
- Type resolution in `(con NAME args)`: if `NAME` contains exactly
|
||
one `.`, split into `module.type` parts and resolve via
|
||
`env.module_types[module]`. Else current behaviour.
|
||
- Term-ctor resolution in `Term::Ctor { type, ctor, args }`: same
|
||
split rule on the `type` field. The `ctor` field stays
|
||
unqualified — once the type is resolved, ctor lookup is
|
||
unambiguous within the type def.
|
||
- Pattern-ctor resolution: when the bare ctor name doesn't resolve
|
||
in the local `ctor_index`, fall back to scanning imported
|
||
modules' types. Conflict rule: local always wins; if multiple
|
||
imported modules declare the same ctor name, error with the
|
||
new diagnostic code `ambiguous-ctor`.
|
||
- Codegen mirrors: a workspace-level `module_ctor_index` replaces
|
||
the per-Emitter table. `lookup_ctor_by_type` / `lookup_ctor_in_pattern`
|
||
thread qualified type names through the box-tag and field-type
|
||
resolution paths.
|
||
|
||
**Diff size: 4 files, ~550 LOC net.** `ailang-check`: +311
|
||
(env + four resolution sites + 4 unit tests). `ailang-codegen`:
|
||
+230 (workspace ctor index + qualified type-name handling). One
|
||
new diagnostic code. Demo updated to use `std_maybe.Maybe` at
|
||
type-name slots.
|
||
|
||
**Tests: 85/85 (was 80, +5).** Four new unit tests in
|
||
`ailang-check` covering: qualified type ref, qualified term-ctor,
|
||
pat-ctor cross-module fallback, pat-ctor ambiguous-ctor
|
||
diagnostic. One new e2e test `cross_module_maybe_demo` asserts
|
||
stdout `["7", "99", "true", "true", "42"]`.
|
||
|
||
**Hash invariance: confirmed.** All five `std_maybe` def hashes
|
||
unchanged (`Maybe 0fb8eaacba5e1135`, `from_maybe caf8eeaca800c80d`,
|
||
`is_some c09002048ff1ff6e`, `is_none 144e131340b58bd3`,
|
||
`map_maybe 68d83d84799322fa`). All other 80-test-suite fixtures
|
||
retain bit-identical hashes — the cross-module support is purely
|
||
additive at the language level.
|
||
|
||
**14a-era regressions held.** Spot-checked
|
||
`parameterised_box_round_trip`, `parameterised_maybe_match`,
|
||
`list_map_poly_inc_then_prints`, `polymorphic_id_at_int_and_bool`
|
||
— all green. The 14h `derive_substitution` change (default
|
||
unpinned forall vars to `Unit` for the monomorphiser) sits on a
|
||
different layer than 14a's `synth_arg_type` `$u`-wildcard fix
|
||
(for nested ctor type synth). Both coexist:
|
||
|
||
- 14a's `$u`-wildcard short-circuits unification when a sibling
|
||
arg pins the same type var concretely.
|
||
- 14h's Unit default applies when no arg pins a forall var at
|
||
all (e.g. `is_none(Nothing)` — `a` in `Maybe<a>` is genuinely
|
||
unobservable from `Nothing`).
|
||
|
||
**Implementer note (flagged for future):** the Unit default
|
||
produces a single shared monomorphisation for all such
|
||
unconstrained-`a` call sites. Wasteful but correct. If the
|
||
stdlib grows toward overload-resolution-style cases where
|
||
unconstrained instantiations need to be distinguished, the
|
||
descriptor strategy needs rethinking. Punted; not a 15a blocker.
|
||
|
||
**std_maybe stdlib effectively ships.** Module + four
|
||
combinators + e2e-tested consumer demo. The cross-module
|
||
parameterised-ADT pipeline is the missing piece that 13a/b/c
|
||
(parameterised ADTs) and 5b (cross-module fns) could not by
|
||
themselves cover. This iter closes that loop.
|
||
|
||
**Plan 15b.** Now that `Maybe<a>` is reusable across modules,
|
||
write `std_list.ailx` importing `std_maybe`. Combinators:
|
||
`length`, `head` (returns `Maybe<a>`), `tail` (returns
|
||
`Maybe<List<a>>`), `is_empty`, `append`, `reverse`, `map`,
|
||
`filter`, `fold_left`, `fold_right`. `head`/`tail` exercise the
|
||
cross-module Maybe-returning case. `fold_left` is the
|
||
tail-recursive variant and gets `(tail-app ...)` markers; the
|
||
constructor-blocked combinators (`map`, `filter`, `append`,
|
||
`fold_right`) stay unmarked. If a new compiler bug surfaces
|
||
during stdlib construction (each prior dogfood iter has surfaced
|
||
one), debugger handles it inline.
|
||
|
||
## Iter 15b — `std_list` ships, three more compiler gaps closed
|
||
|
||
Second stdlib module. Tester wrote `std_list.ailx` (164 LOC, 10
|
||
combinators) plus `std_list_demo.ailx` (consumer importing both
|
||
`std_maybe` and `std_list`). `std_list.ail.json` typechecked
|
||
cleanly in isolation. The demo did not — the prediction "every
|
||
dogfood iter surfaces at least one compiler bug" held three
|
||
times over.
|
||
|
||
**Tester's diagnosis** was sharp enough that the orchestrator
|
||
could go straight to implementer without a debugger round:
|
||
|
||
> Iter 14h's `qualify_local_types` is applied at `Term::Var`
|
||
> cross-module lookup but **not** to ctor-field types when
|
||
> `Term::Ctor` is synthesized. For `Cons a (List a)`, `List`
|
||
> stays unqualified in `cdef.fields`. `std_maybe` slipped
|
||
> through because its ctors have no recursive Con field. First
|
||
> recursive ADT shared cross-module triggers it.
|
||
|
||
**The original-spec fix** was ~10 LOC across two sites in
|
||
`ailang-check/src/lib.rs` — apply `qualify_local_types` over
|
||
`cdef.fields` before substituting forall vars, in both
|
||
`Term::Ctor` synth and `Pattern::Ctor` resolution (the latter
|
||
symmetric, no current consumer hit it but it's the same
|
||
underlying gap).
|
||
|
||
**Two more bugs surfaced during implementation** of that fix:
|
||
|
||
1. **Codegen-side qualify-fields, four sites.** `ailang-codegen`
|
||
has its own field-type tracking that mirrored the check-side
|
||
bug. Symmetric fix needed in `Term::Ctor` synth + `lower_ctor`,
|
||
in `lower_match` for cross-module ADT scrutinees, and a tweak
|
||
to `unify_for_subst` to recurse instead of strict-equality
|
||
when re-binding a forall var that already has a previous
|
||
concrete binding (so a sibling-derived `List<Int>` accepts a
|
||
nullary ctor's `List<$u>` wildcard).
|
||
2. **Const codegen for non-literal values.** The demo defines
|
||
a top-level `xs : List<Int>` const whose body is a Cons
|
||
chain. `check_const` already accepts pure non-literal const
|
||
bodies, but `emit_const` rejected them. Fix: register
|
||
`module_consts` in pass 1, resolve const refs in `Term::Var`
|
||
(load from global for literal consts, inline body for
|
||
non-literal). Both bare and qualified const refs (`xs`
|
||
and `module.xs`) supported.
|
||
|
||
All three fixes together: ~349 / 25 LOC across `ailang-check`,
|
||
`ailang-codegen`, and the new e2e test. Each fix carries an
|
||
`Iter 15b` code comment at its site.
|
||
|
||
**Tests: 87/87 (was 85, +2).** New e2e `std_list_demo` asserts
|
||
the 11-line stdout sequence:
|
||
|
||
```
|
||
5 (length [1..5])
|
||
false (is_empty [1..5])
|
||
true (is_empty [])
|
||
1 (head [1..5] via from_maybe)
|
||
4 (length of tail)
|
||
10 (length of [1..5] ++ [1..5])
|
||
5 (head of reverse [1..5])
|
||
2 (head of map double [1..5] = [2,4,6,8,10])
|
||
2 (length of filter is_even [1..5] = [2,4])
|
||
15 (fold_left + 0 [1..5])
|
||
15 (fold_right + 0 [1..5])
|
||
```
|
||
|
||
New unit test `cross_module_recursive_adt_term_and_pat_ctor` in
|
||
`ailang-check` covers both Term::Ctor and Pattern::Ctor paths
|
||
against a recursive cross-module ADT. Catches the original bug
|
||
+ its symmetric pat-ctor latent twin.
|
||
|
||
**Hash invariance.** All 22 pre-15b fixture hashes plus
|
||
`std_maybe`'s five def hashes bit-identical. The 15b changes
|
||
were purely additive on the language side.
|
||
|
||
**14a-era and 14h-era regressions held.** Spot-checked
|
||
`parameterised_box_round_trip` (14a), `cross_module_maybe_demo`
|
||
(14h), `list_map_poly_inc_then_prints` (14e). All green.
|
||
|
||
**Authoring observation from the tester.** Form (A) holds up to
|
||
10 combinators in one module without breaking. The highest-
|
||
overhead pieces are typed `lam` (each closure carries `(typed
|
||
name type)` triples + return-type + effects-clause) and the
|
||
outer `(forall (vars a b) (fn-type ...))` wrapper. Repeated
|
||
paren-counting at the bottom of nested `seq` chains was the only
|
||
real friction during demo authoring. Suggests a future iter
|
||
might add a `seq*` n-ary form, but the n-ary case is sugar over
|
||
the current `seq` shape and not load-bearing.
|
||
|
||
**Cumulative state, post-15b.**
|
||
|
||
- Stdlib modules: 2 (`std_maybe`, `std_list`).
|
||
- Combinators: 14 (`Maybe` + 4; `List` + 10).
|
||
- Cross-module imports: type-side, ctor-side, fn-side all working.
|
||
- Recursive cross-module ADTs working.
|
||
- Tail-call markers used in production: `fold_left` in `std_list`,
|
||
`print_list` in two existing fixtures.
|
||
- Compiler bugs surfaced and fixed in dogfood:
|
||
- 14a: monomorphisation `Type::unit()` placeholder collision.
|
||
- 14h: cross-module type/ctor not implemented.
|
||
- 15b: qualify-fields gap (3 layers: check, codegen, plus const).
|
||
|
||
**Plan 15c.** Stress test on real-shape data. Build a list of
|
||
~1000 elements, run `fold_left` and `fold_right` over it, verify
|
||
both produce the expected sum. The point: empirically confirm
|
||
that `fold_left`'s tail-call marker actually prevents stack
|
||
overflow under load, while `fold_right` (constructor-blocked,
|
||
unmarked) can still run at this scale because it's only ~1000
|
||
deep, not 1M. If `fold_right` segfaults on this scale, that's a
|
||
useful boundary; if it works, we know the stack budget on this
|
||
host is at least a few thousand frames.
|
||
|
||
After 15c, optional: `std_pair` (2-tuple ADT), `std_either`
|
||
(disjoint union for error handling). Or move on to a
|
||
non-stdlib feature like nested patterns — at this scale of
|
||
language, the case for adding a feature can be made directly
|
||
from a stdlib annoyance.
|
||
|
||
## Iter 15c — empirical TCO + stack-budget validation at N=1000
|
||
|
||
Small empirical iter to confirm the TCO story works dogfood-
|
||
practical, not just on the contrived `print_list` recursion
|
||
that was the 14e regression target.
|
||
|
||
Fixture `examples/std_list_stress.ailx` builds a 1000-element
|
||
`List<Int>` via the recursive `build` fn (also used in 14f's
|
||
`gc_stress`), then folds it with both `std_list.fold_left` and
|
||
`std_list.fold_right`, prints both sums. Both should be
|
||
`500500` (1000 × 1001 / 2). E2E test asserts the two-line
|
||
output.
|
||
|
||
**IR evidence at the monomorphised fold-recursion sites:**
|
||
|
||
```
|
||
%v12 = musttail call i64 @ail_std_list_fold_left__I_I(...) ; tail
|
||
%v7 = call i64 @ail_std_list_fold_right__I_I(...) ; non-tail
|
||
```
|
||
|
||
The `tail: true` marker on `std_list.fold_left`'s recursive call
|
||
survives monomorphisation through the `__I_I` specialisation —
|
||
exactly what 14e's machinery was supposed to do. `fold_right`
|
||
correctly emits a plain `call` (its recursive call is the second
|
||
arg to `f`, not in tail position).
|
||
|
||
**Empirical findings.**
|
||
- `fold_left` at N=1000 runs in constant stack depth (musttail).
|
||
- `fold_right` at N=1000 runs with ~1000 stack frames. No
|
||
segfault. LLVM's frames at `-O0` are small enough that the
|
||
default 8MB Linux stack absorbs this comfortably.
|
||
- `build` (also unmarked, recursive) likewise fits.
|
||
- End-to-end binary execution time ~40ms wall, of which the
|
||
bulk is build + clang link; the actual program runs in <1ms.
|
||
- Two `build 1000` chains plus both folds = ~3000 frames total
|
||
for the unmarked recursions; still fine.
|
||
|
||
**Tests: 88/88 (was 87, +1 e2e).** Cumulative 30 e2e tests.
|
||
|
||
**Cumulative state, post-15c.**
|
||
|
||
- Stdlib modules: 2 (`std_maybe`, `std_list`).
|
||
- Combinators: 14.
|
||
- Cross-module imports: type-side, ctor-side, fn-side, recursive
|
||
ADT support — all working.
|
||
- TCO: marker propagates through monomorphisation; `musttail`
|
||
reaches LLVM at the right call sites.
|
||
- GC: Boehm runs at 1000-element scale without intervention
|
||
(no `GC_INIT()` needed, no symptom of conservative-scan
|
||
over-retention at this scale).
|
||
- Compiler bugs surfaced and fixed in dogfood since 14a: 4
|
||
(14a monomorphisation, 14h cross-module ADT, 15b qualify-
|
||
fields-codegen + non-literal-const-codegen).
|
||
|
||
**Natural pause point.** The language is now genuinely useful
|
||
for small-to-medium programs. The 14b-through-15c arc was
|
||
substantial: a textual surface, two language-completion iters
|
||
(tail calls + GC), one revert (14d→14g), a cross-module ADT
|
||
gap closure, two stdlib modules, and an empirical stress test
|
||
to validate TCO. Work since the user's "Lege los": 9 commits.
|
||
|
||
**Queue for future iters.**
|
||
|
||
- **15d (optional)**: `std_either : Either e a = Left e | Right a`
|
||
for error propagation. Small module (~5 combinators), would
|
||
exercise the cross-module-with-2-type-vars import path
|
||
(currently only Maybe<a> exercised at one type var).
|
||
- **15e (optional)**: `std_pair : Pair a b = MkPair a b` plus
|
||
`fst`, `snd`, `swap`, `map_first`, `map_second`. 2-type-var
|
||
parameterised ADT, no recursion. Smaller dogfood than List.
|
||
- **16a (language)**: nested patterns. Current pattern shape is
|
||
flat (a `pat-ctor`'s fields are `pat-var | pat-wild | pat-lit`,
|
||
not nested `pat-ctor`). This becomes painful when stdlib code
|
||
wants to match `Cons h (Cons h2 _)` directly. Manageable today
|
||
via nested `match` but would simplify several stdlib idioms.
|
||
- **16b (language)**: local recursive `let`. Currently must hoist
|
||
to top-level. Painful for stdlib helpers that are clearly
|
||
internal to one combinator (e.g. an accumulator-loop inside
|
||
`reverse`'s body).
|
||
- **17a (codegen optimisation)**: per-fn arena for non-escaping
|
||
ADT allocations (Decision 9's flagged future iter). Real win
|
||
for `map`/`filter`-style combinators where the intermediate
|
||
list is dropped immediately. Needs escape analysis; multi-iter
|
||
design pass.
|
||
|
||
None of these block further stdlib work. They are quality-of-
|
||
life improvements; the language is feature-complete enough that
|
||
the stdlib can grow without them.
|
||
|
||
|
||
|
||
|
||
|
||
---
|
||
|
||
## Iter 15d — `std_either`: 2-type-var ADT + 3-type-var eliminator
|
||
|
||
**Goal.** Third stdlib module. Either is the canonical 2-type-var
|
||
disjoint sum. The eliminator combinator `either : (e → c) → (a → c) →
|
||
Either<e, a> → c` introduces a third type var on top of the data,
|
||
making it the most polymorphism-dense fn shipped to date.
|
||
|
||
**What shipped.**
|
||
|
||
- `examples/std_either.ailx` (74 LOC). Defines `Either e a = Left e | Right a`
|
||
plus 5 combinators:
|
||
- `from_right : a → Either<e, a> → a` — eliminate Right or use default.
|
||
- `is_left`, `is_right : Either<e, a> → Bool`.
|
||
- `map_right : (a → b) → Either<e, a> → Either<e, b>` —
|
||
Functor-style on the Right side.
|
||
- `either : (e → c) → (a → c) → Either<e, a> → c` —
|
||
catamorphism / fold-on-sum.
|
||
- `examples/std_either_demo.ailx` exercises every combinator,
|
||
including the eliminator with two distinct concrete instantiations
|
||
(over `Left 5` and over `Right 100`).
|
||
- Canonical JSON for both, parsed via `ail parse`, type-checked,
|
||
built, and executed end-to-end.
|
||
|
||
**Output (deterministic).**
|
||
|
||
```
|
||
42 ; from_right 0 (Right 42)
|
||
99 ; from_right 99 (Left 7)
|
||
true ; is_left (Left 7)
|
||
true ; is_right (Right 42)
|
||
42 ; from_right 0 (map_right inc (Right 41))
|
||
6 ; either inc inc (Left 5)
|
||
101 ; either inc inc (Right 100)
|
||
```
|
||
|
||
**Monomorphisation evidence.** Six distinct instantiations across the
|
||
five combinators, all generated correctly with the `$u` wildcard for
|
||
unused type-var positions:
|
||
|
||
```
|
||
@ail_std_either_from_right__I_I ; e=Int, a=Int
|
||
@ail_std_either_from_right__U_I ; e=$u, a=Int
|
||
@ail_std_either_is_left__I_U ; e=Int, a=$u
|
||
@ail_std_either_is_right__U_I ; e=$u, a=Int
|
||
@ail_std_either_map_right__U_I_I ; e=$u, a=Int, b=Int
|
||
@ail_std_either_either__I_I_I ; e=Int, a=Int, c=Int
|
||
```
|
||
|
||
The two `from_right` variants are notable: same source fn, two
|
||
different concrete `e` per call site (Int when scrutinee is Left 7;
|
||
$u when scrutinee is Right _ since Left's payload is unused). The
|
||
14a monomorphisation machinery picks the right one per site without
|
||
extra ceremony.
|
||
|
||
The 3-type-var instantiation `either__I_I_I` is the deepest type
|
||
substitution shipped through monomorphisation so far. Both call
|
||
sites in main hit the same instantiation (`e=a=c=Int`) so a single
|
||
emit suffices.
|
||
|
||
**No new compiler bugs surfaced.** First stdlib iter without a fresh
|
||
codegen/check fix. Indicator that 14a / 14h / 15b coverage was wide
|
||
enough to handle 2-type-var data + 3-type-var fns out of the box.
|
||
|
||
**Discovered (not fixed in this iter):** `ail render` and `ail parse`
|
||
are not symmetric, despite the help text in the CLI claiming they
|
||
are. `render` calls the older `ailang_core::pretty::module`
|
||
human-pretty printer (form B-ish), while `parse` consumes form (A).
|
||
The form-(A) printer in `ailang_surface::print` is correctly the
|
||
inverse of `parse` — confirmed by the round-trip test in
|
||
`crates/ailang-surface/tests/round_trip.rs` which now covers 25
|
||
fixtures including `std_either.ail.json` — but it is not exposed via
|
||
the CLI. Logged as 15e.
|
||
|
||
**Tests: 89/89 (was 88, +1 e2e for std_either_demo).** Cumulative
|
||
31 e2e tests.
|
||
|
||
**Cumulative state, post-15d.**
|
||
|
||
- Stdlib modules: 3 (`std_maybe`, `std_list`, `std_either`).
|
||
- Combinators: 19.
|
||
- Type-system surface area exercised: 1-type-var data (Maybe, List
|
||
with recursion) and 2-type-var data (Either); 1- and 2- and
|
||
3-type-var polymorphic fns; cross-module recursive ADTs;
|
||
monomorphisation-with-wildcards across all.
|
||
|
||
**Queue update.** 15d done; remaining queue: 15e (CLI render/parse
|
||
symmetry — small, just discovered), then `std_pair` if more stdlib
|
||
is wanted, then 16a/16b language gaps.
|
||
|
||
---
|
||
|
||
## Iter 15e — `render` is the inverse of `parse` (CLI hygiene)
|
||
|
||
**Trigger.** Discovered during 15d: `ail render | ail parse` did not
|
||
round-trip. The help text claimed they were inverses but `render`
|
||
called `ailang_core::pretty::module` (an old human-readable
|
||
S-expression-ish form), while `parse` consumes form (A). Two
|
||
different "text projections" lived in the codebase, one of them
|
||
exposed via the CLI under a name that promised inversion.
|
||
|
||
The form-(A) printer in `ailang_surface::print` was already correct
|
||
and gated by `crates/ailang-surface/tests/round_trip.rs` across
|
||
every fixture (25 modules at the time of writing) — it was simply
|
||
not exposed at the CLI surface.
|
||
|
||
**Changes.**
|
||
|
||
1. `Cmd::Render` now calls `ailang_surface::print(&m)`. Help text
|
||
updated to state explicitly: form (A), exact inverse of `parse`,
|
||
round-trippable.
|
||
2. `Cmd::Describe` (both single-module and `--workspace` branches)
|
||
likewise switched to `ailang_surface::print` so that "text view
|
||
of one def" means the same thing everywhere in the CLI.
|
||
3. `ailang_core::pretty::module` deleted. With `module` gone its
|
||
helpers (`def_block`, `term_block`, `term_inline`) became dead
|
||
weight — also deleted. ~260 LOC of duplicate-purpose code gone;
|
||
`crates/ailang-core/src/pretty.rs` shrank from 457 to 196 LOC.
|
||
4. The remaining surface of `ailang_core::pretty` — `manifest`,
|
||
`type_to_string`, `pattern_to_string` — is the diagnostic
|
||
stringification surface (used in error messages and the
|
||
`ail manifest` summary, where one-line ML notation is more
|
||
readable than form (A)'s nested S-expression). Module-level
|
||
doc rewritten to make this single-purpose framing explicit;
|
||
no more "pretty-printer" / "render" overloading.
|
||
5. New e2e test `render_parse_round_trip_canonical` (in
|
||
`crates/ail/tests/e2e.rs`) gates the CLI shell pipeline:
|
||
shell out `ail render <fixture>` → write to tmp .ailx →
|
||
shell out `ail parse <tmp>` → assert canonical-byte equality
|
||
with the original fixture. The unit-level round-trip in
|
||
`ailang-surface/tests/round_trip.rs` covered the printer
|
||
function directly; this new test additionally guards the
|
||
CLI wiring.
|
||
6. The `ailang_core::pretty` unit test `pretty_print_does_not_panic`
|
||
exercised the deleted `module` fn — removed. `manifest_contains_
|
||
type_and_hash` stays.
|
||
|
||
**Doctrinal pin.** Form (A) is the only round-trippable text form.
|
||
The diagnostic stringification in `ailang_core::pretty` is
|
||
asymmetric and lossy by design — it exists for the *limited*
|
||
purpose of sticking a type or pattern into an error message in
|
||
human-friendlier shape than form (A). It does not roundtrip and
|
||
must not be used as an authoring or persistence target. Any
|
||
future add to it should add to that purpose, not back-fill toward
|
||
"another text projection".
|
||
|
||
**Tests: 89/89.** (e2e went from 31 to 32; ailang-core unit
|
||
tests went from 11 to 10 — the deleted unit test exercised the
|
||
removed fn. Net same.)
|
||
|
||
**Cumulative state, post-15e.** No new language features. Two
|
||
canonical text projections collapsed to one (form A). Internal
|
||
cleanup; no behavioural change for any program in the repo.
|
||
|
||
---
|
||
|
||
## Iter 16a — nested constructor patterns in `match`
|
||
|
||
**Goal.** Lift the gate that rejected `(pat-ctor Cons a (pat-ctor Cons b _))`
|
||
and similar nested-Ctor sub-patterns. Lit-in-Ctor stays rejected;
|
||
that's a separate iter.
|
||
|
||
**Approach: AST-level desugar before check + codegen.** Pure rewrite
|
||
that flattens nested Ctor patterns into chains of single-level
|
||
matches with let-bound fresh vars and duplicate fall-through.
|
||
Hash-relevant canonical bytes untouched because the pass runs
|
||
*after* `load_module`, in memory only. The checker / codegen always
|
||
see flat patterns.
|
||
|
||
**Algorithm sketch.**
|
||
|
||
For a `Match` whose arms contain nested-Ctor sub-patterns:
|
||
1. Bottom-up: recursively desugar scrutinee and arm bodies first.
|
||
2. Let-bind the scrutinee to `$mp_N` (single eval).
|
||
3. Build a chain `arm_1 (else arm_2 (else ... default))`. Default
|
||
is `Lit Unit`, unreachable for valid programs.
|
||
4. Each arm's nested Ctor sub-patterns are lifted to fresh vars,
|
||
then deepest-first wrapped via `wrap_sub`, which on a Ctor sub
|
||
recursively re-enters `desugar_match` — that recursion handles
|
||
arbitrary depth.
|
||
|
||
`fall_k` is cloned per inner level → O(arms × depth) terms in
|
||
worst case. Acceptable for typical patterns.
|
||
|
||
**Fresh-name safety.** `$` is a valid identifier character in form
|
||
(A) (the lexer's `Ident` token is "anything not paren/int/string"),
|
||
so `$mp_0` is theoretically user-writable. The Desugarer pre-walks
|
||
every `Term::Var` and `Pattern::Var` name in the module into a
|
||
`BTreeSet<String>` and bumps the counter past collisions.
|
||
|
||
**Files.**
|
||
|
||
- New: `crates/ailang-core/src/desugar.rs` (571 LOC including
|
||
doc-comments and two unit tests). Public surface is
|
||
`pub fn desugar_module(m: &Module) -> Module` — pure, idempotent,
|
||
no I/O.
|
||
- `crates/ailang-core/src/lib.rs` — `pub mod desugar;` plus a
|
||
module-doc bullet describing the new pipeline layer.
|
||
- `crates/ailang-check/src/lib.rs` — three public entries
|
||
(`check_module`, `check_workspace`, `check`) call
|
||
`desugar_module` first. The gate at the old line 1538 is now
|
||
narrowed: nested **Ctor** sub-patterns are `unreachable!()`
|
||
(desugared away); nested **Lit** still emits the existing
|
||
`nested-ctor-pattern-not-allowed` diagnostic. Crucially:
|
||
`check`'s returned `CheckedModule.symbols` keeps hashes of
|
||
the *original* defs so `ail diff` and `ail manifest` see the
|
||
on-disk identity, not a post-desugar one.
|
||
- `crates/ailang-codegen/src/lib.rs` — `lower_workspace` desugars
|
||
every module up front; `emit_ir` (single-file shortcut) goes
|
||
through `lower_workspace` so the desugar runs there too.
|
||
- New: `examples/nested_pat.ailx` + `nested_pat.ail.json`.
|
||
`first_two_sum` matches `(pat-ctor Cons a (pat-ctor Cons b _))`;
|
||
prints `30` for the input `[10, 20, 30]`.
|
||
- `crates/ail/tests/e2e.rs` — new
|
||
`nested_ctor_pattern_first_two_sum` test.
|
||
|
||
**Behavioural property of the desugar.** Already-flat matches go
|
||
through `is_flat()` and emit identical AST shapes. Empirically:
|
||
every existing fixture (std_maybe, std_list, std_either, list_map,
|
||
sort, etc.) produces the same observable output as before because
|
||
their patterns were already flat — the desugar is a no-op clone
|
||
on them.
|
||
|
||
**Tests: 92/92.**
|
||
- e2e: 33 (was 32, +1 for nested_pat).
|
||
- ailang-core unit: 12 (was 10, +2 for the desugar tests).
|
||
- All other crates unchanged.
|
||
|
||
**No new compiler bug surfaced.** The transform was straightforward
|
||
because the AST already supported nested sub-patterns at the type
|
||
level — only the checker gate and codegen drop-on-floor were
|
||
artificial walls. Removing them via desugaring (rather than
|
||
expanding the codegen) keeps the pattern-matching backend simple
|
||
and lets future iters (Lit-in-pattern, exhaustiveness, decision
|
||
trees) plug in at the same desugar layer.
|
||
|
||
**Cumulative state, post-16a.**
|
||
|
||
- Stdlib: 3 modules, 19 combinators (unchanged from 15d).
|
||
- Language: nested Ctor patterns now legal; nested Lit-in-pattern
|
||
still rejected (intentional follow-up scope).
|
||
- Pipeline: `load → desugar → check → codegen`. The desugar layer
|
||
is the natural home for further surface-level smoothing
|
||
(Lit-in-pattern, `if`-as-syntactic-sugar, etc.) without
|
||
enlarging the core AST.
|
||
- Compiler bugs surfaced and fixed in dogfood since 14a: still 4
|
||
(no fresh ones in 16a — the desugar landed clean).
|
||
|
||
**Queue update.** 16a done. Remaining: 15f (`std_pair`, optional);
|
||
16b (local recursive `let`); 17a (per-fn arena); future
|
||
"lit-in-Ctor" follow-up under 16c if and when needed.
|
||
|
||
---
|
||
|
||
## Iter 15f — `std_pair`: 2-type-var product, no recursion
|
||
|
||
**Goal.** Round out the small-ADT stdlib triad (Maybe, Either, Pair).
|
||
Pair is the canonical product with two type vars and a single
|
||
constructor — no recursion in the data def or any combinator.
|
||
Smallest dogfood for the parameterised-ADT path so far.
|
||
|
||
**What shipped.**
|
||
|
||
- `examples/std_pair.ailx` (~75 LOC, 5 combinators):
|
||
- `fst`, `snd` — projections.
|
||
- `swap : Pair<a, b> -> Pair<b, a>`.
|
||
- `map_first : (a -> c) -> Pair<a, b> -> Pair<c, b>`.
|
||
- `map_second : (b -> c) -> Pair<a, b> -> Pair<a, c>`.
|
||
Each combinator is a single-arm match on `MkPair` with no
|
||
fall-through, so the desugar pass added in 16a is a no-op
|
||
(the patterns are already flat).
|
||
- `examples/std_pair_demo.ailx` exercises every combinator. Output
|
||
(deterministic): 7, 9, 9, 7, 8, 18.
|
||
- `crates/ail/tests/e2e.rs::std_pair_demo` guards the path.
|
||
|
||
**No new compiler bug surfaced.** Stdlib iter #4 in a row that
|
||
landed clean. The only fixable issue was a paren-balance typo in
|
||
the demo's `seq` chain, surfaced immediately by the parser's
|
||
"unexpected end of input, expected `)`" diagnostic.
|
||
|
||
**Tests: 93/93** (e2e went from 33 to 34).
|
||
|
||
**Cumulative state, post-15f.**
|
||
|
||
- Stdlib: 4 modules (`std_maybe`, `std_list`, `std_either`,
|
||
`std_pair`); 24 combinators total.
|
||
- Type-system surface area exercised end-to-end: 1- and 2- and
|
||
3-type-var data; 1- and 2- and 3-type-var fns; recursive ADTs;
|
||
cross-module imports of all of the above; flat *and* nested
|
||
patterns (16a); TCO via monomorphised `musttail`; Boehm GC.
|
||
- Pipeline layers: `load → desugar → check → codegen → clang`.
|
||
Each layer has a public, narrow contract; the desugar layer
|
||
is the natural home for further surface-level smoothing
|
||
without enlarging the core AST.
|
||
|
||
**Queue update.** 15f done. Remaining: 16b (local recursive `let`),
|
||
16c (Lit-in-Ctor patterns), 17a (per-fn arena). None blocking
|
||
further stdlib growth; each is a quality-of-life improvement.
|
||
|
||
---
|
||
|
||
## Iter 16a-aux — DESIGN.md drift audit (post-15f)
|
||
|
||
**Goal.** After six feature iters (14g, 14h, 15a–15f, 16a) without a
|
||
docs sweep, the design doc had accumulated visible drift. Patch in
|
||
place rather than queueing the next codegen-heavy iter against a
|
||
stale spec — the user is unreachable for the broader memory-management
|
||
discussion that gates 17a, so this is the lowest-risk productive
|
||
move.
|
||
|
||
**Drift sites found and fixed (DESIGN.md +72 LOC).**
|
||
|
||
1. **Decision 6 status (L116).** Was: `(Iter 14b — WIP) ... Status: design pass in progress`.
|
||
Now: marks form (A) as shipped in Iter 14c, gated by the
|
||
round-trip test in `ailang-surface/tests/round_trip.rs`, and
|
||
notes that 15e made it the *sole* text projection (legacy
|
||
pretty-printer module helpers deleted). The body of the
|
||
section (constraints, candidate notations) remains as the
|
||
audit trail of the original design pass.
|
||
2. **Pipeline section (L687–696).** Inserted the **desugar pass**
|
||
between resolve+hash and typecheck, with the load-bearing
|
||
invariant explicit: `CheckedModule.symbols` hashes from the
|
||
*original* module, not the desugared one, so `ail diff` /
|
||
`ail manifest` keep reporting the on-disk identity. Also
|
||
noted libgc linkage on the clang line.
|
||
3. **CLI section (L700–708).** Added the four subcommands that
|
||
shipped in earlier iters but never made the doc: `deps`,
|
||
`diff` (single-module + `--workspace`), `workspace`, and
|
||
`builtins`. Reformatted to a two-column layout.
|
||
4. **"What is not (yet) supported" (L724–746).** Re-anchored
|
||
from "end of Iter 13" to "as of Iter 16a". Removed three
|
||
gates that had been lifted: cross-module ADTs (14h), no-GC
|
||
(14f, Decision 9), flat-pattern-only (16a). Replaced with
|
||
tighter follow-up gates: literal sub-patterns inside Ctor
|
||
patterns; local recursive `let`. Added a *one-paragraph*
|
||
"recently lifted" preamble so a future reader sees both the
|
||
delta and the iter that produced it without consulting JOURNAL.
|
||
5. **"What IS supported" (L782+).** Promoted four capabilities
|
||
into the smoke-test list: nested Ctor patterns via desugar
|
||
(16a); cross-module ADTs (14h); the form-(A) text surface as
|
||
shipped (14b/14c/15e); Boehm GC (Decision 9 / 14f). Replaced
|
||
"ADTs + flat pattern matching" line with one that names both
|
||
the original 3 and the 16a extension.
|
||
6. **Pipeline regression smoke tests (L819–).** Added the four
|
||
stdlib-demo fixtures (`std_list_demo`, `std_maybe_demo`,
|
||
`std_either_demo`, `std_pair_demo`) plus `nested_pat`. The
|
||
pre-existing entries (sum/list/hof/closure/list_map/sort/
|
||
poly_id/poly_apply/box/maybe_int) were preserved as-is.
|
||
|
||
**No code changes; tests still 93/93.** Verified with `cargo
|
||
build --workspace --quiet` — the doc-only edits do not affect
|
||
any compilation unit.
|
||
|
||
**What was *not* changed.** The Goal section, Decisions 1–5,
|
||
Decisions 7–9 (already accurate after their respective revert /
|
||
ship statuses), Mangling scheme, Convention for cross-module
|
||
references, Data model (Module/Def/Term/Type grammar), Verification
|
||
section. Spot-checked each — all match current implementation.
|
||
|
||
**Cumulative state, post-16a-aux.**
|
||
|
||
- Stdlib: unchanged (4 modules, 24 combinators).
|
||
- DESIGN.md: 804 → 876 LOC. JOURNAL.md grows by this entry.
|
||
- Drift baseline reset: any future iter that lifts a gate or
|
||
ships a new pipeline layer should patch the relevant section
|
||
in the same iter, not accumulate.
|
||
|
||
**Queue update.** 16a-aux done. Unchanged from post-15f:
|
||
16b (local recursive `let`), 16c (Lit-in-Ctor patterns),
|
||
17a (per-fn arena). 17a remains the explicit checkpoint before
|
||
any broader memory-management work — that decision is gated on
|
||
a joint conversation with the user.
|
||
|
||
---
|
||
|
||
## Iter 15g — std_either_list: first 3-way cross-module stdlib fn
|
||
|
||
**Goal.** Through 15f the stdlib had 4 modules but no fn imported
|
||
more than one foreign module (e.g. `std_list.head` returning
|
||
`std_maybe.Maybe<a>`). 15g introduces the first stdlib module whose
|
||
every fn imports three others — `std_list`, `std_either`,
|
||
`std_pair` — and returns a compound polymorphic ADT tree
|
||
(`Pair<List<e>, List<a>>`). Designed to stress monomorphisation
|
||
across `List × Either × Pair`, cross-module ctor resolution at
|
||
qualified `term-ctor` sites, and the 16a desugar layer's nested-
|
||
Ctor pattern handling at depth 2.
|
||
|
||
**What shipped.**
|
||
|
||
- `examples/std_either_list.ailx` — three combinators, all
|
||
`forall (vars e a)`, all using qualified cross-module names at
|
||
`con` and `term-ctor` sites:
|
||
- `lefts : List<Either<e, a>> -> List<e>` — depth-2 nested-Ctor
|
||
match per Cons arm (`Cons (Left l) t` / `Cons (Right _) t`)
|
||
plus a wildcard tail to anchor the desugar's fall-through
|
||
chain in `List<e>` rather than the synthetic `Unit` fallback.
|
||
- `rights : List<Either<e, a>> -> List<a>` — symmetric to
|
||
`lefts`.
|
||
- `partition_eithers : List<Either<e, a>> -> Pair<List<e>,
|
||
List<a>>` — `Nil → MkPair(Nil, Nil)`; `Cons h t →` let-bind
|
||
`rest = partition_eithers t`, then a flat match on `h` to
|
||
splice `l` / `r` onto `fst rest` / `snd rest`. Single pass.
|
||
- `examples/std_either_list_demo.ailx` — first demo importing four
|
||
stdlib modules. Drives all three combinators on the same five-
|
||
element list `[Left 1, Right 10, Left 2, Right 20, Right 30]`;
|
||
prints lengths via `std_list.length`. Output: `2 / 3 / 2 / 3`.
|
||
- `crates/ail/tests/e2e.rs::std_either_list_demo` (e2e count
|
||
34 → 35).
|
||
|
||
**16a desugar exercised.** Yes — `lefts` and `rights` use depth-2
|
||
nested Ctor patterns (`(pat-ctor Cons (pat-ctor Left l) t)`). Each
|
||
expands into a chain of single-level matches via the 16a pass; the
|
||
explicit trailing wildcard arm prevents the synthetic `Unit`
|
||
fallback (the desugar's documented "valid programs never reach it"
|
||
terminator) from leaking into the function's return type. Without
|
||
the wildcard the checker reports `expected std_list.List<e>, got
|
||
Unit` because the chain's `else`-arm body is `Lit Unit` — confirms
|
||
the design note in `desugar.rs::desugar_match` that exhaustiveness
|
||
is the caller's responsibility, not the desugar's.
|
||
|
||
**New compiler bug surfaced.** Yes — and it is **not** in the new
|
||
combinators themselves but in the existing monomorphiser, surfaced
|
||
the moment one tries to construct an inline list literal mixing
|
||
`(term-ctor std_either.Either Left n)` and `(term-ctor
|
||
std_either.Either Right n)`. Reduced repro is just `(app
|
||
std_list.length (term-ctor std_list.List Cons (Left 1) (term-ctor
|
||
std_list.List Cons (Right 10) (term-ctor std_list.List Nil))))` —
|
||
no `lefts` / `rights` / `partition_eithers` involved.
|
||
|
||
Cause: `synth_arg_type` produces `Either<Int, $u>` for `Left 1` and
|
||
`Either<$u, Int>` for `Right 10`. The outer `Cons`'s parameter
|
||
type `List<a>` unifies first against `List<Either<Int, $u>>`
|
||
(binds `a = Either<Int, $u>`) and then against `List<Either<$u,
|
||
Int>>` for the tail. `unify_for_subst` recurses into the prev
|
||
binding and ends up unifying param `$u` (from `Either<Int, $u>`)
|
||
against arg `Int` — the existing `if name.starts_with("$u") {
|
||
return Ok(()); }` early-return only fires when `$u` is on the
|
||
**arg** side. Param-side `$u` falls through to the catch-all error
|
||
`cannot match param `$u` to arg `Int``.
|
||
|
||
**Workaround in the demo.** Two monomorphic helpers `mkleft : Int
|
||
-> Either<Int, Int>` and `mkright : Int -> Either<Int, Int>` pin
|
||
both type vars at the call site, so each list element arrives with
|
||
fully concrete `Either<Int, Int>` and the synth-time `$u` never
|
||
appears. Documented in the demo's header comment with a pointer
|
||
back to this entry. The combinators themselves (`lefts`, `rights`,
|
||
`partition_eithers`) are bug-free — the demo just couldn't build
|
||
the input list inline without dodging the `$u`-on-param-side path.
|
||
|
||
**Bug fix sketch (queued, not landed).** A symmetric early-return
|
||
in `unify_for_subst` for param-side `$u` would close this — the
|
||
wildcard semantics ("don't care, defer") are direction-agnostic.
|
||
Filed as candidate iter 15g-aux. Single-line patch + a unit test
|
||
under `ailang-codegen/src/lib.rs`.
|
||
|
||
**Tests: 94/94.**
|
||
|
||
- e2e: 35 (was 34, +1 for `std_either_list_demo`).
|
||
- All other crates unchanged.
|
||
|
||
**Cumulative state, post-15g.**
|
||
|
||
- Stdlib: 5 modules (`std_maybe`, `std_list`, `std_either`,
|
||
`std_pair`, `std_either_list`); 27 combinators total (24 + 3).
|
||
- Deepest cross-module composition reached so far: `List × Either
|
||
× Pair` in a single fn (`partition_eithers`).
|
||
- Compiler bugs surfaced and fixed in dogfood since 14a: still 4
|
||
fixed; 1 new (the `$u`-on-param-side case above), worked around
|
||
in the demo, queued as 15g-aux.
|
||
|
||
**Queue update.** 15g done. Remaining: 15g-aux (param-side `$u`
|
||
acceptance — small, one-line in `unify_for_subst`); 16b (local
|
||
recursive `let`); 16c (Lit-in-Ctor patterns); 17a (per-fn arena,
|
||
gated on user discussion).
|
||
|
||
---
|
||
|
||
## Iter 15g-aux — symmetric `$u` early-return in `unify_for_subst`
|
||
|
||
**Goal.** Fix the codegen asymmetry that 15g surfaced: inline list
|
||
literals mixing `Left n` and `Right n` ctor calls of the same
|
||
`Either<e, a>` errored at synth time even though both elements have
|
||
fully-determined concrete-after-pinning types.
|
||
|
||
**Diagnosis.** `unify_for_subst` (`crates/ailang-codegen/src/lib.rs`)
|
||
had an arg-side-only early-return for `$u`-prefixed synth wildcards.
|
||
The function has a prev-binding recursion path that re-invokes
|
||
itself with the previously-bound type as the new param and the
|
||
fresh arg, which can swap a `$u` from arg position into param
|
||
position. Concretely, `length [Left 1, Right 10]` synths the list
|
||
elements as `Either<Int, $u>` and `Either<$u, Int>`. The first
|
||
element pins `a = Either<Int, $u>` for `Cons`'s parameter `a`. The
|
||
second element triggers a recursive unification of the previously-
|
||
bound `Either<Int, $u>` against `Either<$u, Int>`, walking
|
||
pairwise: `Int` vs `$u` (arg-side `$u`, ok) and `$u` vs `Int`
|
||
(param-side `$u`, **falls through** to the catch-all error).
|
||
|
||
**Fix.** Three lines in `unify_for_subst`: add a symmetric early-
|
||
return for param-side `$u`. Doc comment expanded to record the
|
||
asymmetry's origin and the symmetric extension's justification
|
||
(`$u` is a synth-only wildcard regardless of which side it ends
|
||
up on after the prev-binding swap).
|
||
|
||
**Demo refactor.** `examples/std_either_list_demo.ailx` no longer
|
||
uses the `mkleft`/`mkright` monomorphic helpers introduced as the
|
||
15g workaround. The list is now constructed inline by mixing
|
||
`(term-ctor std_either.Either Left 1)` and `(term-ctor
|
||
std_either.Either Right 10)` directly. Same expected output
|
||
(2, 3, 2, 3); the demo doubles as the 15g-aux regression fixture.
|
||
|
||
**Tests: 94/94, unchanged.** The fix expanded what compiles, did
|
||
not change observable behaviour for any prior fixture.
|
||
|
||
**Cumulative state, post-15g-aux.**
|
||
|
||
- Stdlib unchanged (5 modules, 27 combinators).
|
||
- One latent codegen bug retired. The bug count in the dogfood
|
||
audit since 14a stays at 4 surfaced + fixed (this is a fresh
|
||
surface from 15g, so 5 surfaced / 5 fixed).
|
||
- The std_either_list_demo workaround is gone, leaving the inline
|
||
cross-module mixed-ctor list as the canonical idiom.
|
||
|
||
**Queue update.** 15g-aux done. Unchanged: 16b (local recursive
|
||
let), 16c (Lit-in-Ctor patterns), 17a (per-fn arena, gated on
|
||
user discussion of memory management).
|
||
|
||
---
|
||
|
||
## Iter 15h — std_list extension: take, drop
|
||
|
||
**Goal.** Extend `std_list` with the two index-driven prefix
|
||
combinators `take` and `drop`. They are the first `std_list` fns to
|
||
combine `if` + Int arithmetic + recursive ADT pattern in a single
|
||
body — every previous `std_list` fn was either a fold/HOF
|
||
(fold_left, fold_right, length, map, filter, reverse) or a pure
|
||
match-on-Cons (head, tail, append, is_empty). 15h closes the
|
||
canonical-prefix-slicing gap and dogfoods the `if (<= n 0)` +
|
||
`(- n 1)` interaction inside a recursive Cons-pattern body.
|
||
|
||
**What shipped.**
|
||
|
||
- `examples/std_list.ailx` — `take` and `drop` appended after
|
||
`filter`. All ten pre-existing defs byte-identical (verified via
|
||
`cargo run -p ail -- check` on every downstream importer:
|
||
`std_list_demo`, `std_list_stress`, `std_either_list`,
|
||
`std_either_list_demo`, `list_map_poly`). Both new fns are
|
||
`forall (vars a). (Int, List<a>) -> List<a>`. `take` is
|
||
constructor-blocked recursive (`Cons h (take (n-1) t)`); `drop`
|
||
is direct recursive (the recursive call is the arm body, no
|
||
Cons wrap), so the `match` arm is in tail position relative to
|
||
the `if`'s `else` branch — but neither call site is marked
|
||
`tail-app` because the surrounding `if` is not the fn's
|
||
immediate body. Conservative; matches the rest of `std_list`'s
|
||
marking discipline.
|
||
- `examples/std_list_more_demo.ailx` — six prints exercising both
|
||
combinators at all three boundary regimes (n=0, 0<n<length,
|
||
n>length). Reuses `std_list_demo`'s `(const xs)` idiom for the
|
||
canonical `[1, 2, 3, 4, 5]`. Output: `0 / 3 / 5 / 5 / 3 / 0`.
|
||
- `crates/ail/tests/e2e.rs::std_list_more_demo` (e2e count
|
||
35 → 36).
|
||
|
||
**`(if (<= n 0) ...)` as the base-case guard.** Both fns gate the
|
||
recursion on the int counter at the top of the body, _outside_ the
|
||
match. The dual-arm `(case (pat-ctor Cons h t) ...)` then handles
|
||
only the n>0 path — so the n=0 branch never re-enters the match,
|
||
and the recursive call is reached only when the list is non-empty
|
||
and n is still positive. This is a deliberate workaround for the
|
||
absence of literal sub-patterns inside Ctor patterns: a
|
||
hypothetical `(case (pat-ctor Cons _ _) (case-when (== n 0) ...))`
|
||
or matching `n` against `0` directly is queued as 16c. With 16c
|
||
landed, both fns could collapse the `if` into the match.
|
||
|
||
**No new compiler bug surfaced.** The `if` + Int-arithmetic +
|
||
recursive ADT pattern shape is the first instance for `std_list`
|
||
in particular but not new for the compiler — `gc_stress` and
|
||
`std_list_stress` already exercised `(if (== n 0) ...)` with
|
||
`(- n 1)` recursion at the top level. `<=` on `Int` and the
|
||
forall-`a` instantiation through the `match` cleanly reused the
|
||
same monomorphisation paths. Round-trip `render | parse` stays
|
||
canonical for both new files.
|
||
|
||
**Tests: 95/95.**
|
||
|
||
- e2e: 36 (was 35, +1 for `std_list_more_demo`).
|
||
- All other crates unchanged.
|
||
|
||
**Cumulative state, post-15h.**
|
||
|
||
- Stdlib: 5 modules (`std_maybe`, `std_list`, `std_either`,
|
||
`std_pair`, `std_either_list`); **29** combinators total
|
||
(27 + 2). All four primary `std_list` operations — length, map,
|
||
filter, take/drop — are now present, plus the head/tail/append/
|
||
reverse/is_empty/fold_left/fold_right surface from 15b.
|
||
- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5
|
||
(unchanged from 15g-aux).
|
||
|
||
**Queue update.** 15h done. Remaining: 16b (local recursive let),
|
||
16c (Lit-in-Ctor patterns — would simplify `take`/`drop`), 17a
|
||
(per-fn arena, gated on user discussion).
|
||
|
||
---
|
||
|
||
## Iter 16b.1 — local recursive let (no-capture)
|
||
|
||
**Goal.** Let me write `(let-rec f (params x) (type ...) (body ...) (in ...))`
|
||
inline inside a fn body for the common no-capture case (the body's
|
||
free vars are all module-top-level def names, qualified imports,
|
||
effect-op names, or builtin operators). Stop forcing every recursive
|
||
helper to be a separate top-level fn — typical small kernels
|
||
(`fact`, `loop_n`, `gcd`) belong next to the use site, not on a
|
||
sibling line at module scope.
|
||
|
||
Sub-iter 16b.1 ships **no-capture only**. A LetRec whose body would
|
||
capture a name from the enclosing lexical scope (an enclosing fn's
|
||
params, a let-bound name from an outer `Term::Let`, or a pattern-
|
||
bound name from an outer `Term::Match` arm) is rejected at desugar
|
||
time with a clear panic that points at 16b.2 (closure conversion).
|
||
That keeps the iter at lift-via-desugar — no runtime closure changes,
|
||
no codegen plumbing, no LLVM IR change.
|
||
|
||
**Design choice.** Lift the LetRec to a synthetic top-level fn in
|
||
the same desugar pass that already runs between `load_module` and
|
||
typecheck (16a). The lifted fn gets a fresh name `<hint>$lr_N` that
|
||
is unique against both the original module's def names and against
|
||
any earlier lifts in the same pass. Every reference to the local
|
||
LetRec name (in the body and in the in-clause) is rewritten via a
|
||
`subst_var` helper to the lifted name. The lifted `FnDef` is
|
||
appended to `Module.defs`; from there the typechecker / codegen
|
||
treat it like any hand-written top-level fn.
|
||
|
||
Alternatives considered and rejected:
|
||
|
||
- **Runtime closures** (treat LetRec like an anonymous lambda
|
||
bound to a name). Would require a closure-pair allocation per
|
||
call, plus a self-reference field in the env block. Reaches the
|
||
same value but more LLVM IR per LetRec. Deferred to 16b.2 where
|
||
it becomes necessary anyway (capture support).
|
||
- **Open-coding the recursion at the LetRec site** via a Y-style
|
||
fixed-point combinator. Adds a non-local construct (the
|
||
combinator) and keeps the expansion at every LetRec; the lift-
|
||
and-substitute approach keeps the runtime shape identical to a
|
||
hand-written top-level fn.
|
||
|
||
The pipeline invariant from 16a (`CheckedModule.symbols` hashes
|
||
from the *original* on-disk module, not the desugared one) holds
|
||
unchanged: a lifted def has no on-disk identity, so it never
|
||
appears in `symbols`. `ail diff` and `ail manifest` see only the
|
||
original-source defs.
|
||
|
||
**What shipped.**
|
||
|
||
- `crates/ailang-core/src/ast.rs` (+16): `Term::LetRec { name, ty,
|
||
params, body, in_term }` variant inserted right after
|
||
`Term::Let`. JSON tag `"t": "letrec"`. `ty` and `in_term` use
|
||
serde renames (`type`, `in`) to keep the schema natural.
|
||
Additive — every pre-16b.1 fixture canonicalises bit-identically.
|
||
- `crates/ailang-core/src/desugar.rs` (+~570 of which ~310 are
|
||
the new helpers + LetRec arm; the rest is doc/test): three
|
||
additions plus the existing pass threaded through a `scope`
|
||
parameter. (a) `free_vars_in_term` walks a term collecting every
|
||
unbound `Term::Var` name, respecting all binders. (b) `subst_var`
|
||
rewrites `Term::Var { name == from }` to the lifted name,
|
||
respecting shadowing (a `Term::Let`/`Term::Lam`/`Pattern::Var`
|
||
that rebinds `from` blocks the substitution inside its scope).
|
||
(c) `Desugarer` gained `lifted: Vec<Def>` and
|
||
`module_top_names: BTreeSet<String>`; `desugar_module` seeds
|
||
the latter from every original def name and appends `lifted`
|
||
to `defs` after the per-def walk. The new `desugar_term` arm
|
||
for `Term::LetRec` recurses on body+in_term with an extended
|
||
scope, runs `free_vars_in_term` against `{name} ∪ params`,
|
||
intersects with the outer `scope`, panics if non-empty, then
|
||
generates `<hint>$lr_N`, substitutes, and appends the lifted
|
||
`FnDef`. Two new unit tests:
|
||
`let_rec_no_capture_lifts_to_top_level` and
|
||
`let_rec_with_capture_panics` (`#[should_panic]`).
|
||
- `crates/ailang-surface/src/parse.rs` (+~70): `let-rec-term`
|
||
production added to the EBNF (still inside the 30-rule
|
||
budget — count is now ~31, but `let-rec` is a positional
|
||
analogue of `let` and the increment is consistent with the
|
||
Decision-6 budget rationale). New `parse_let_rec` function;
|
||
dispatch keyword added in `parse_term`. Unit test
|
||
`parses_minimal_let_rec` round-trips a minimal shape.
|
||
- `crates/ailang-surface/src/print.rs` (+16): `Term::LetRec`
|
||
arm in `write_term`, single-line form mirroring the
|
||
`Term::Let` arm's compactness. The round-trip harness
|
||
(`tests/round_trip.rs`) picks up the new fixture
|
||
automatically.
|
||
- `crates/ailang-check/src/lib.rs` (+10): two `unreachable!`
|
||
arms in `verify_tail_positions` and `synth` —
|
||
`Term::LetRec` is eliminated by desugar before either runs.
|
||
- `crates/ailang-codegen/src/lib.rs` (+19): four `unreachable!`
|
||
arms in `lower_term`, `collect_captures`, `synth_with_extras`,
|
||
and `apply_subst_to_term`. Same rationale.
|
||
- `crates/ail/src/main.rs` (+22): `walk_term` (the `ail deps`
|
||
walker) gets a real arm — `deps` runs on the on-disk module
|
||
before desugar, so `Term::LetRec` is reachable there.
|
||
Treats it like a fn def for dependency purposes (name
|
||
shadows in body+in_term; params shadow inside body).
|
||
- `examples/local_rec_demo.ailx` + `.ail.json` — first
|
||
consumer fixture. A factorial helper bound by `(let-rec
|
||
fact ...)` inside `main`'s body; called at n=1, n=3, n=5;
|
||
prints `1\n6\n120\n`. The `fact` body has no captures
|
||
from `main`'s scope (referenced names: `<=`, `*`, `-`,
|
||
`fact`, `n`, `1` — all builtins, the LetRec's own name,
|
||
or its param), so it lifts cleanly.
|
||
- `crates/ail/tests/e2e.rs::local_rec_factorial_demo` (+18):
|
||
e2e count 36 → 37.
|
||
|
||
**Unreachable arms.** Every backend stage that pattern-matches
|
||
`Term` exhaustively now has a `Term::LetRec { .. } =>
|
||
unreachable!("Term::LetRec eliminated by desugar")` arm. The
|
||
phrase is identical across all five sites
|
||
(`ailang-check/src/lib.rs` × 2, `ailang-codegen/src/lib.rs` × 4)
|
||
so a future grep reaches every one of them. The `ail deps`
|
||
walker is the one site that intentionally has a real arm,
|
||
because `deps` runs on the on-disk module before any desugar
|
||
pass, and it is documented inline.
|
||
|
||
**Lift-name format.** `<hint>$lr_N` where `<hint>` is the
|
||
source-level LetRec name and `N` is an incrementing counter that
|
||
yields the first name not already in `Desugarer.used` or
|
||
`Desugarer.module_top_names`. Mirrors the `$mp_N` fresh-match-
|
||
pattern naming from 16a; `$lr_` is the namespace marker for
|
||
"lifted recursion". `$` is a valid ident character in form (A),
|
||
so the name reaches LLVM as is — but `clang` mangling chokes on
|
||
`$`, so the name is sanitised the same way every other ail name
|
||
is (the existing `@ail_<module>_<def>` mangling already handles
|
||
`$` because of the 16a `$mp_` precedent).
|
||
|
||
**Tests: 95 → 99 (+4).**
|
||
|
||
- e2e: 36 → 37 (`local_rec_factorial_demo`).
|
||
- `ailang-core::desugar::tests`: 2 → 4 (the two new LetRec tests).
|
||
- `ailang-surface::parse::tests`: 2 → 3
|
||
(`parses_minimal_let_rec`).
|
||
- All other crates unchanged.
|
||
|
||
**Cumulative state, post-16b.1.**
|
||
|
||
- Stdlib unchanged (5 modules, 29 combinators).
|
||
- `Term` enum: 11 variants (was 10) — additive, all pre-existing
|
||
fixture hashes bit-identical.
|
||
- 16a desugar pass now does two jobs: nested-ctor-pattern
|
||
flattening (16a) and LetRec lift (16b.1). The pass remains
|
||
the single AST-→-AST hop between `load_module` and typecheck.
|
||
- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5
|
||
(unchanged from 15h).
|
||
|
||
**Queue update.** 16b.1 done. Remaining: 16b.2 (LetRec capture —
|
||
closure conversion or env-passing rewrite, with the panic at
|
||
desugar time as the boundary-marker until then), 16c (Lit-in-
|
||
Ctor patterns — would simplify `take`/`drop`), 17a (per-fn
|
||
arena, gated on user discussion of memory management).
|
||
|
||
## Iter 16c — literal patterns via desugar
|
||
|
||
**Goal.** Lift the last gate that survived the 16a-aux audit:
|
||
`Pattern::Lit` was rejected at the codegen level (an internal
|
||
error in the Match lowering) and at the typechecker level when
|
||
nested inside a Ctor (`nested-ctor-pattern-not-allowed`). 16c
|
||
makes lit patterns work **everywhere they parse** — top-level
|
||
arms and Ctor sub-patterns alike — by extending the existing
|
||
16a desugar pass.
|
||
|
||
**Design choice.** Desugar `Pattern::Lit { lit }` to
|
||
`Term::If { cond = (== sv lit), then = body, else_ = fall_k }`,
|
||
where `sv` is the let-bound scrutinee (top-level) or the
|
||
field-bound fresh var (sub-pattern). After 16c, no
|
||
`Pattern::Lit` survives the desugar pass — the codegen and
|
||
typechecker never see one.
|
||
|
||
Alternatives considered and rejected:
|
||
|
||
- **Switch-on-i64 in codegen** (a `switch i64` LLVM instruction
|
||
per Match with at least one Int-lit arm). Smaller IR for
|
||
many-arm dispatch, but adds a second match-lowering path in
|
||
codegen and grows the typechecker's exhaustiveness checker
|
||
(now needs to reason about lit coverage). The desugar-to-If
|
||
approach hands every problem to the existing infrastructure:
|
||
`==` is a typed builtin, `Term::If` already lowers correctly,
|
||
and the chain machinery from 16a already produces a
|
||
fall-through structure that fits.
|
||
- **Codegen-level lit-arm rejection only** (allow lit arms past
|
||
typecheck and trap at codegen). Worse than today: today's
|
||
codegen rejects with an internal error; future codegen would
|
||
need a real lowering. Strictly more work for no gain.
|
||
|
||
The pipeline invariant from 16a/16b.1 holds unchanged: the
|
||
desugar runs after `load_module`, so on-disk module hashes are
|
||
untouched. `ail diff` and `ail manifest` see only original-source
|
||
defs.
|
||
|
||
**What shipped.**
|
||
|
||
- `crates/ailang-core/src/desugar.rs` (+~120, of which ~80 are
|
||
doc/test): three replacements plus one new helper.
|
||
(a) The `Pattern::Lit` arm of `desugar_one_arm` now emits a
|
||
`Term::If { cond = (== s_var lit), then = arm.body, else_ =
|
||
fall_k }` instead of the old single-arm `Term::Match` with
|
||
the lit pattern preserved (which the codegen rejected).
|
||
(b) The `Pattern::Lit` branch of `wrap_sub` mirrors (a) on
|
||
the field-bound fresh variable, replacing the old recursive
|
||
`desugar_match` call.
|
||
(c) `is_flat` no longer classifies `Pattern::Lit` as flat —
|
||
the early-return path in `desugar_match` would otherwise
|
||
leak a Pattern::Lit arm through to typecheck/codegen
|
||
unchanged. With the change, lit-arms always take the
|
||
let-bind + chain path.
|
||
(d) New free function `build_eq(scrutinee, lit) -> Term`:
|
||
produces `(app == scrutinee lit)` for Int/Bool/Str and a
|
||
`Bool(true)` literal for Unit (every Unit value is equal).
|
||
Three new unit tests:
|
||
`top_level_lit_desugars_to_if`,
|
||
`nested_lit_in_ctor_desugars_to_if`, and
|
||
`flat_arm_with_lit_is_no_longer_flat` (regression guard for
|
||
the deliberate change in `is_flat`). A new `any_lit_pattern`
|
||
walker mirrors `any_nested_ctor`.
|
||
- `examples/lit_pat.ailx` + `examples/lit_pat.ail.json` — first
|
||
consumer fixture. Two fns: `classify` (top-level lit arms
|
||
for 0/1/default → 100/200/999) and `categorize_first`
|
||
(`Cons (pat-lit 0) _` nested-lit demonstration over a local
|
||
`IntList` ADT). The trailing `_` catch-all in
|
||
`categorize_first` is required by the 16a chain
|
||
machinery — the chain's terminator is a `Unit` literal, so
|
||
a final `_` arm dominates it and keeps the match
|
||
type-checking. Output (per line): 100, 200, 999, -1, 0, 7.
|
||
- `crates/ail/tests/e2e.rs::lit_pat_demo` (+15 incl. doc): e2e
|
||
count 37 → 38.
|
||
- `docs/DESIGN.md`: moved the "no literal sub-patterns inside a
|
||
Ctor" line out of "What is not (yet) supported" into the
|
||
"Recently lifted gates" preamble and into the "What is
|
||
supported" list, with a note that `Bool`/`Str`/`Unit` lit
|
||
patterns are accepted by desugar but reach typecheck as a
|
||
type error today (because `==` is `(Int, Int) -> Bool`
|
||
only — extending `==` to those types is a separate iter).
|
||
|
||
**What deliberately did NOT change.**
|
||
|
||
- Codegen's `Pattern::Lit` arm at `crates/ailang-codegen/src/lib.rs:1302`
|
||
still returns `CodegenError::Internal("MVP: lit patterns in
|
||
match not supported")`. Left as a never-reached safety net —
|
||
the desugar is the contract; the codegen panic catches future
|
||
regressions where a Pattern::Lit slips through.
|
||
- Typechecker's `Pattern::Lit` arm in `check_pattern` still runs
|
||
(it accepts the pattern but contributes nothing to
|
||
exhaustiveness). Same rationale: dead code at the source-AST
|
||
level after desugar, but defensive against a future caller
|
||
that bypasses desugar.
|
||
- `std_list::take` and `std_list::drop` still hand-write the
|
||
base-case-via-arm-body workaround (`(case (pat-ctor Cons h t)
|
||
(if (== n 0) Nil (...)))`). Refactoring them to use lit
|
||
patterns is queued as 16c-aux.
|
||
|
||
**Tests: 99 → 103 (+4).**
|
||
|
||
- e2e: 37 → 38 (`lit_pat_demo`).
|
||
- `ailang-core::desugar::tests`: 4 → 7 (the three new lit tests).
|
||
- All other crates unchanged.
|
||
|
||
**Cumulative state, post-16c.**
|
||
|
||
- Stdlib unchanged (5 modules, 29 combinators).
|
||
- `Term`/`Pattern`/`Literal` enums unchanged — additive at the
|
||
desugar level only, so all pre-existing fixture hashes stay
|
||
bit-identical.
|
||
- 16a desugar pass now does three jobs: nested-ctor-pattern
|
||
flattening (16a), LetRec lift (16b.1), and lit-pattern → If
|
||
rewrite (16c). Pass remains the single AST-→-AST hop between
|
||
`load_module` and typecheck.
|
||
- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5
|
||
(unchanged).
|
||
|
||
**Queue update.** 16c done. Remaining: 16b.2 (LetRec capture —
|
||
closure conversion, unchanged), 16b.3 (LetRec let-binding
|
||
capture, unchanged), 17a (per-fn arena, gated on user
|
||
discussion of memory management, unchanged). 16c-aux
|
||
(`std_list::take`/`drop` refactor onto lit patterns) is a
|
||
separate iter, gated on orchestrator decision.
|
||
|
||
## Iter 16b.2 — planning entry (LetRec capture)
|
||
|
||
**Status: planning, not implemented.** This entry is orchestrator
|
||
work-product, not an iteration log. It captures the design space
|
||
for 16b.2 so the user can review the trade-offs before
|
||
implementation, and so a future agent has a sharp brief to work
|
||
against.
|
||
|
||
**Goal (when implemented).** Lift 16b.1's no-capture restriction.
|
||
Support `(let-rec name ...)` whose body references one or more
|
||
names bound in the enclosing lexical scope, by augmenting the
|
||
lifted top-level fn's signature with the captured names as extra
|
||
parameters and rewriting every call site of `name` (in body and
|
||
in_term) to pass the captures positionally.
|
||
|
||
**Why this is non-trivial.** The lifted fn's signature must
|
||
include the captures' types. Those types are not always
|
||
discoverable at desugar time:
|
||
|
||
- **Captured fn-param**: type is in `f.ty.params[i]`. Known.
|
||
- **Captured Lam-param**: type is in `param_tys[i]`. Known.
|
||
- **Captured Let-binding**: `Term::Let { name, value, body }`
|
||
carries no type annotation on `name`. Type is the inferred
|
||
type of `value`, which the typechecker computes — but the
|
||
desugar pass runs *before* the typechecker. **Unknown at
|
||
desugar time.**
|
||
- **Captured Match-arm pattern var**: type is the substituted
|
||
field type of the matched constructor, requiring an ADT-def
|
||
lookup plus arg-substitution. Computable in principle, but
|
||
the desugar pass would need to track the scrutinee's type
|
||
through the walk — also a small inference engine.
|
||
|
||
**Architectural choices.** Pick one path before implementing:
|
||
|
||
1. **Stay at desugar; restrict to fn/Lam-param captures only.**
|
||
Reject Let-binding and Match-arm captures with a clear
|
||
error → `16b.3` (Let captures) and `16b.4` (Match captures).
|
||
Lowest-cost path, covers the most common case (recursive
|
||
helpers that close over an enclosing fn's input).
|
||
|
||
2. **Move LetRec elimination to a post-typecheck pass.** By
|
||
then every `Term` has known types; capture types fall out.
|
||
Cost: a new pipeline stage, plus updating `DESIGN.md`'s
|
||
pipeline section. Pays off if the post-pass is also where
|
||
other future lowerings live (e.g. closure conversion).
|
||
|
||
3. **Run a lightweight inference inside desugar.** Just enough
|
||
to resolve `Term::Let`-bound names to their value's type.
|
||
Effectively a Hindley-Milner pass on a subset of the AST.
|
||
Cost: significant; basically duplicating part of
|
||
`ailang-check`. Rejected on principle — one source of truth.
|
||
|
||
**Recommended path.** (1), shipping incrementally. 16b.2 covers
|
||
fn/Lam-params; 16b.3 lifts Let-bindings via path (2). The
|
||
benefit of (1) first: it surfaces real-world usage and informs
|
||
how often Let-binding capture actually matters.
|
||
|
||
**Other restrictions for 16b.2 (under path 1).**
|
||
|
||
- **Direct-call only.** The LetRec name `f` may appear in
|
||
`body` and `in_term` only as the callee of a `Term::App`,
|
||
never as a `Term::Var` in any other position (e.g. passed as
|
||
an argument, bound to a `let`, or stored in an ADT field).
|
||
Reason: with captures-as-extra-params, every use site needs
|
||
the extras appended. Treating `f` as a value would require
|
||
a closure object that bundles `f` and its captures — that's
|
||
16b.5 / closure conversion.
|
||
- **Monomorphic enclosing fn.** If the enclosing fn is
|
||
`forall(...). ...`, the captures' types may mention the
|
||
outer's type vars. Constructing the lifted fn's `Forall` is
|
||
doable but error-prone; defer to 16b.6.
|
||
- **Single-level LetRec.** A LetRec whose body contains
|
||
another LetRec that captures the outer's name or params is
|
||
not supported in 16b.2; reject with a clear error → 16b.7.
|
||
|
||
**What 16b.2 ships (under the recommended scope).**
|
||
|
||
- `desugar_term`'s `scope` parameter changes from
|
||
`&BTreeSet<String>` to `&BTreeMap<String, ScopeEntry>`,
|
||
where `ScopeEntry` is `KnownType(Type) | LetBound | MatchArm`.
|
||
- Each binder extends the map: fn/Lam-params with `KnownType`,
|
||
Let-bindings with `LetBound`, Match-arm bindings with
|
||
`MatchArm`.
|
||
- LetRec arm's capture detection: free-vars ∩ scope-keys.
|
||
For each capture, look up `ScopeEntry`. If `KnownType(t)`,
|
||
proceed. If `LetBound` / `MatchArm`, error.
|
||
- Validate: walk body and in_term, assert `name` only appears
|
||
as `Term::App.callee`. Helper `validate_callee_only_use`.
|
||
- Build augmented fn type: `original.ty` with capture types
|
||
appended to `params`. Reject if `original.ty` is `Forall`
|
||
(out of scope per restriction 2 above).
|
||
- Rewrite call sites: `(app f a b)` → `(app f$lr_N a b cap0
|
||
cap1)`. Use a new helper `subst_call_with_extras` that
|
||
walks the term, recognizes `Term::App { callee = Var{f} }`
|
||
patterns, rewrites them, and recurses elsewhere.
|
||
- Substitute `f` → `f$lr_N` everywhere in body (for the
|
||
recursive self-reference at the lifted level). The existing
|
||
`subst_var` from 16b.1 handles this, but care needed: it
|
||
must run **after** `subst_call_with_extras` so that
|
||
recursive calls get both the rename and the extras.
|
||
- Append the lifted `FnDef` to `Module.defs`.
|
||
|
||
**Risk.** The most likely failure mode is the typechecker
|
||
rejecting a synthesized augmented fn signature that we believe
|
||
should type-check. Mitigation: implement under restriction 2
|
||
(no Forall) so we never construct a Forall; the augmented type
|
||
is plain `Type::Fn` with monomorphic capture types appended.
|
||
|
||
**Tests.**
|
||
|
||
- `examples/local_rec_capture.ailx`: an enclosing fn with one
|
||
Int param, a LetRec that recurses against that param. e2e
|
||
asserts an output value derived from that capture.
|
||
- Negative tests at the desugar unit-test level: Let-binding
|
||
capture errors; Match-arm capture errors; name-as-value
|
||
errors; nested LetRec mutual-capture errors. All
|
||
`#[should_panic]` (consistent with 16b.1's panic discipline).
|
||
|
||
**Adjacent open items.**
|
||
|
||
- **16d**: chain-machinery's `Unit` terminator forces a
|
||
trailing `_` arm in matches that are otherwise exhaustive,
|
||
surfaced by 16c (`categorize_first` fixture). Two design
|
||
paths: (a) introduce a polymorphic `__unreachable__`
|
||
builtin that codegen lowers to LLVM `unreachable`, used as
|
||
the chain default; (b) run an exhaustiveness pre-check in
|
||
desugar against the scrutinee's ADT and omit the
|
||
terminator when arms cover all ctors. Path (a) is broader
|
||
(gives users a primitive for panics/asserts). Path (b) is
|
||
purer (terminator never appears for exhaustive matches)
|
||
but needs ADT lookup in desugar.
|
||
- **16e**: extend `==` from Int-only to Int/Bool/Str. Bool
|
||
comparison is i1-equality (trivial). Str comparison
|
||
needs a runtime `strcmp`. Surfaced by 16c (`build_eq`
|
||
produces `(app == ...)` for Bool/Str/Unit but currently
|
||
fails at typecheck because `==` is not declared for them).
|
||
- **17a**: per-fn arena allocator. Gated on user
|
||
memory-management discussion (separate stored memory).
|
||
|
||
**Queue update post-16c.** 16c done. Open: 16b.2 (this entry —
|
||
planning only, awaiting user input on path 1 vs path 2), 16d
|
||
(planning needed — pick path a vs b), 16e (`==` extension),
|
||
17a (gated). All implementation work in this session-arc is
|
||
suspended at this planning checkpoint.
|
||
|
||
## Iter 16b.2 — LetRec captures of fn-params (path-1 safe subset)
|
||
|
||
**Goal.** Lift 16b.1's no-capture restriction for the **path-1 safe
|
||
subset** described in the 16b.2 planning entry: support a
|
||
`(let-rec ...)` whose body captures one or more names from the
|
||
enclosing scope, **provided every capture comes from a fn-param or
|
||
Lam-param** (whose types are statically declared at desugar time).
|
||
The lifted top-level fn's signature gets the capture types appended
|
||
to `params`; every call site of the LetRec name is rewritten to
|
||
pass the captures positionally as extra args. Anything outside the
|
||
safe subset is rejected at desugar time with a panic that names the
|
||
violation and points at the follow-up iter that will handle it
|
||
(16b.3 / 16b.4 / 16b.5 / 16b.6 / 16b.7).
|
||
|
||
**Path-1 restrictions in force (rejected at desugar with a
|
||
follow-up-iter pointer).**
|
||
|
||
- **Let-binding capture** (`Term::Let`-bound name, type unknown at
|
||
desugar). Queued for **16b.3**.
|
||
- **Match-arm pattern-binding capture** (constructor-field
|
||
substitution from the scrutinee's ADT not done at desugar).
|
||
Queued for **16b.4**.
|
||
- **Name-as-value use** (the LetRec name `f` appears anywhere
|
||
except as the callee of a `Term::App` — e.g. `(let g f ...)` or
|
||
`(app some_hof f)`). Needs closure conversion. Queued for
|
||
**16b.5**.
|
||
- **Polymorphic enclosing fn** (`Type::Forall`). Captured fn-param
|
||
types may mention outer type vars; constructing the lifted
|
||
signature's `Forall` is error-prone. Encoded as
|
||
`ScopeEntry::LetBound` for every fn-param of a `Forall`-typed
|
||
enclosing fn — caught at the same site as let-binding captures.
|
||
Queued for **16b.6**.
|
||
- **Nested LetRec mutual capture** (an inner LetRec captures the
|
||
outer LetRec's name or params). Queued for **16b.7**.
|
||
|
||
**What shipped.**
|
||
|
||
- `crates/ailang-core/src/desugar.rs` (1341 → 1853 LOC, +512). The
|
||
hot changes:
|
||
- New `enum ScopeEntry { KnownType(Type), LetBound, MatchArm,
|
||
EnclosingLetRec }`. The `scope` parameter threaded through
|
||
`desugar_term` and helpers became
|
||
`&BTreeMap<String, ScopeEntry>` (was `&BTreeSet<String>`).
|
||
Every binder ([`Term::Let`], [`Term::Lam`], [`Term::Match`]
|
||
arm, [`Term::LetRec`]) inserts the appropriate variant;
|
||
`desugar_module` seeds fn-param entries from `f.ty` (peeled
|
||
`Forall`).
|
||
- `Term::LetRec` arm rewritten end-to-end: peel `ty` to its
|
||
inner `Type::Fn` to learn the LetRec's own param types,
|
||
extend body-scope with `EnclosingLetRec` for `name` +
|
||
`KnownType(_)` for params, recurse on body and in_term, run
|
||
`free_vars_in_term` against `{name} ∪ params`, intersect
|
||
with the outer scope's keys, classify each capture's
|
||
`ScopeEntry` (KnownType → accept; everything else → panic
|
||
with the follow-up iter pointer), validate via
|
||
`find_non_callee_use` that `name` only appears as a callee,
|
||
build the augmented `Type::Fn` with capture types appended,
|
||
rewrite call sites via `subst_call_with_extras`, then
|
||
`subst_var` for any non-call references (defensive — none
|
||
survive the validator), append the lifted `FnDef`.
|
||
- Two new free helpers (~150 LOC together):
|
||
`subst_call_with_extras(t, name, lifted, extras)` rewrites
|
||
every `Term::App { callee = Var{name} }` to
|
||
`Term::App { callee = Var{lifted}, args ++ extras_as_vars }`,
|
||
walking through every variant; `find_non_callee_use(t, name)`
|
||
walks `t` and returns `Some(t)` at the first `Term::Var {
|
||
name == name }` reference in a non-callee position, `None`
|
||
otherwise. Plus `peel_forall_to_fn` (one-liner).
|
||
- The 16b.1 panic is gone for fn-param captures (replaced by
|
||
the lifter); it persists for every other capture kind, with
|
||
a sharper message.
|
||
- `crates/ail/tests/e2e.rs` (+18): `local_rec_capture_demo` —
|
||
e2e count 38 → 39. Asserts that the `local_rec_capture` fixture
|
||
prints `0\n10\n45\n`.
|
||
- `examples/local_rec_capture.ailx` + `.ail.json` (35 LOC source):
|
||
`sum_below(n)` returns the sum of integers `1 + ... + (n-1)`.
|
||
Body uses an inner `(let-rec loop (params i) ... (body ... (app
|
||
>= i n) ... (app loop (app + i 1))) (in (app loop 1)))`. The
|
||
helper captures `n` from `sum_below`'s param list; the desugar
|
||
pass lifts it to `loop$lr_0(i: Int, n: Int) -> Int` and rewrites
|
||
every `(app loop X)` to `(app loop$lr_0 X n)`. `main` drives
|
||
`sum_below` at 1, 5, 10 → outputs `0`, `10`, `45`.
|
||
- `crates/ailang-core/src/desugar.rs::tests` (3 new):
|
||
`let_rec_capture_fn_param_lifts_with_extra_arg` (positive),
|
||
`let_rec_capture_let_binding_panics`
|
||
(`#[should_panic(expected = "16b.3")]`),
|
||
`let_rec_name_as_value_panics`
|
||
(`#[should_panic(expected = "16b.5")]`). The 16b.1
|
||
`let_rec_with_capture_panics` test was repurposed into the
|
||
positive `let_rec_capture_fn_param_lifts_with_extra_arg`: its
|
||
fixture (a fn-param capture) is now legal and produces the
|
||
expected augmented signature.
|
||
|
||
**The augmented-signature mechanism.** Given
|
||
`(let-rec f (params p1..pk) (type Fn(t1..tk) -> tr) (body B) (in I))`
|
||
inside a fn whose scope contains
|
||
`{c1: T1, ..., cm: Tm}` (KnownType entries) used as free vars in
|
||
`B`:
|
||
|
||
- Lifted signature: `f$lr_N : Fn(t1..tk, T1..Tm) -> tr` with the
|
||
same `effects` set as the original.
|
||
- Lifted param-name list: `p1..pk, c1..cm` (capture names appended
|
||
unchanged so the lifted body's references resolve directly).
|
||
- Body rewrite: every `(app f a1..an)` in `B` → `(app f$lr_N
|
||
a1..an c1..cm)`. The `c1..cm` are `Term::Var` references that
|
||
resolve, in the lifted body, to the appended params with the
|
||
same names. Free uses of `f` in non-callee position are
|
||
pre-rejected by `find_non_callee_use`.
|
||
- In-term rewrite: every `(app f a1..an)` in `I` → `(app f$lr_N
|
||
a1..an c1..cm)`. The `c1..cm` are `Term::Var` references that
|
||
resolve, in the in-term's enclosing fn, to the captured names
|
||
themselves (still in scope).
|
||
|
||
The rewrite is order-sensitive: `subst_call_with_extras` runs
|
||
**before** `subst_var`, so a recursive self-call inside `B` first
|
||
becomes `(app f$lr_N ... c1..cm)`, then any leftover bare `Var{f}`
|
||
(none in 16b.2 — pre-rejected) gets renamed to `Var{f$lr_N}`. The
|
||
second pass is defensive.
|
||
|
||
**Tests: 102 → 106 (+4).**
|
||
|
||
- e2e: 38 → 39 (`local_rec_capture_demo`).
|
||
- `ailang-core::desugar::tests`: 6 → 9. Net +3 because the
|
||
16b.1-era `let_rec_with_capture_panics` test was repurposed
|
||
rather than removed (its fn-param-capture fixture is now a
|
||
positive lift), and two pure-negative tests were added.
|
||
- All other crates unchanged.
|
||
|
||
**Cumulative state, post-16b.2.**
|
||
|
||
- Stdlib unchanged (5 modules, 29 combinators).
|
||
- `Term` enum: 11 variants (unchanged; LetRec is still
|
||
desugar-eliminated, no schema impact). All pre-16b.2 fixtures
|
||
hash bit-identically.
|
||
- 16a desugar pass now does three jobs: nested-ctor-pattern
|
||
flattening (16a), literal-pattern lowering (16c), and LetRec
|
||
lift (16b.1 → 16b.2 path-1). Single AST→AST hop between
|
||
`load_module` and typecheck.
|
||
- The `unreachable!("Term::LetRec eliminated by desugar")` arms
|
||
in `ailang-check` × 2 and `ailang-codegen` × 4 remain correct
|
||
— every LetRec is still gone before either stage runs.
|
||
|
||
**Queue update post-16b.2 path-1.** 16b.2 path-1 done. Open:
|
||
**16b.3** (LetRec captures of `Term::Let`-bound names — would
|
||
need a post-typecheck re-run of the lift, or a small inference
|
||
on the let-value's type), **16b.4** (LetRec captures of match-arm
|
||
pattern bindings — needs ADT-def lookup + constructor-field
|
||
substitution), **16b.5** (closure conversion — lifts the
|
||
"name-as-value-only" restriction for both LetRec and Lam),
|
||
**16b.6** (LetRec inside a `Type::Forall`-quantified fn — needs
|
||
synthesised `Forall` for the lifted signature), **16b.7**
|
||
(nested LetRec mutual capture — generalised lifting that
|
||
accumulates captures across nesting). 16d (chain-machinery
|
||
exhaustiveness or `__unreachable__` — planning needed), 16e
|
||
(`==` extension to Bool/Str/Unit), 17a (per-fn arena, gated)
|
||
unchanged.
|
||
|
||
## Iter 16b.3 — LetRec captures of Let-bound names
|
||
|
||
**Goal.** Lift 16b.2's "fn/Lam-param captures only" restriction. A
|
||
`(let-rec ...)` may now capture names bound by a `Term::Let` in the
|
||
enclosing scope. Match-arm captures (16b.4), name-as-value (16b.5),
|
||
Forall enclosing fn (16b.6), and nested LetRec mutual capture
|
||
(16b.7) remain rejected at desugar time.
|
||
|
||
**Architectural choice (path-2: post-typecheck lift).** The desugar
|
||
pass cannot resolve a `Term::Let`-bound name's type — `Term::Let`
|
||
carries no annotation; the value's type is inferred at typecheck.
|
||
Three options were on the table (path-1 = stay-at-desugar with
|
||
restrictions, path-2 = post-typecheck lift, path-3 = run a private
|
||
inference inside desugar). Path-2 is the cleanest: typechecker is
|
||
the single source of truth, and the lift now becomes a small
|
||
AST-→-AST pass with O(1) type lookups against the typechecker's
|
||
env. `Term::LetRec` reaches the typechecker only when the desugar
|
||
pass deferred it — for the 16b.2 fast-path (KnownType captures
|
||
only) the desugar still does the lift in one hop.
|
||
|
||
**What shipped.**
|
||
|
||
- `crates/ailang-core/src/desugar.rs` (1853 → 1931 LOC, +78). The
|
||
`Term::LetRec` arm now has three exits:
|
||
(a) `KnownType`-only captures → existing 16b.2 lift (unchanged).
|
||
(b) Any `LetBound` capture → reconstruct the `Term::LetRec`
|
||
with desugared sub-terms and return it (defer to
|
||
post-typecheck pass).
|
||
(c) `MatchArm` / `EnclosingLetRec` → panic with the same
|
||
follow-up-iter pointers as 16b.2.
|
||
The `find_non_callee_use` (16b.5) check moved to run before
|
||
classification so the diagnostic fires consistently regardless
|
||
of which exit is taken. Four helpers (`free_vars_in_term`,
|
||
`subst_var`, `subst_call_with_extras`, `find_non_callee_use`,
|
||
`pattern_binds`) were promoted from `fn` to `pub fn` so the
|
||
post-typecheck pass can reuse them.
|
||
- `crates/ailang-check/src/lib.rs` (2887 → 3281 LOC, +394
|
||
including tests).
|
||
- `verify_tail_positions` and `synth` arms for `Term::LetRec`
|
||
replaced. `verify_tail_positions`: body is NOT in tail
|
||
position (it's a fn body — the LetRec name is what gets
|
||
tail-called); in_term inherits the enclosing context. `synth`:
|
||
peel any `Forall` defensively, validate param count, install
|
||
`name + params` in locals for the body, synth the body, unify
|
||
against `ret_ty`, check the effect-subset rule (same as
|
||
`Term::Lam`); restore locals; install `name` for the
|
||
in-clause; synth, return.
|
||
- New `pub fn check_and_lift(m) -> Result<(CheckedModule,
|
||
Module)>` runs check + desugar + `lift_letrecs` and returns
|
||
both the original-symbols `CheckedModule` and the lifted
|
||
module ready for codegen.
|
||
- `crates/ailang-check/src/lift.rs` (new file, 720 LOC). The
|
||
`pub fn lift_letrecs(m: &Module) -> Result<Module>` post-pass.
|
||
- Fast-path: `contains_any_letrec(m)` returns `false` →
|
||
return input unchanged. Skips env construction so cross-
|
||
module modules (which we don't fully wire up here) are not
|
||
touched unless they actually contain a deferred LetRec.
|
||
- Builds a single-module env (builtins + module type defs +
|
||
module globals + imports + current_module) and walks every
|
||
`Def::Fn`'s body, threading a parallel `IndexMap<String,
|
||
Type>` of locals as scope. At each `Term::Let`, synth the
|
||
value's type to populate locals; at each `Term::Match`
|
||
arm, run a minimal `type_check_pattern_for_lift` to infer
|
||
pattern-arm bindings.
|
||
- At every `Term::LetRec`: post-order recurse first (so
|
||
nested LetRecs are lifted before their enclosing one),
|
||
re-run `find_non_callee_use` defensively, recompute
|
||
captures via `free_vars_in_term`, look up each capture's
|
||
type from `locals`, build the lifted `FnDef` (capture types
|
||
appended to params), append to a `lifted: Vec<Def>`
|
||
accumulator, rewrite call sites in body and in_term via
|
||
`subst_call_with_extras`, then `subst_var` for any
|
||
leftover bare references. Synthetic name `<hint>$lr_N`
|
||
seeded past the highest existing `*$lr_N` suffix in
|
||
`m.defs` so it cannot collide with a 16b.2 lift. Synthetic
|
||
FnDefs carry a `doc` of the form
|
||
`"Lifted by 16b.3 from let-rec '<name>' inside '<enclosing>'."`.
|
||
- The `subst_call_with_extras` and `subst_var` helpers come
|
||
straight from `ailang-core::desugar` — re-used rather than
|
||
duplicated (the brief asked for one or the other; re-use
|
||
via `pub` keeps the rewrite logic single-sourced).
|
||
- `crates/ail/src/main.rs` (+25 LOC). `build_to` now goes
|
||
`load → check → per-module (desugar + lift_letrecs) → codegen`.
|
||
The `check` subcommand stays typecheck-only (no lift needed
|
||
for type-checking). Codegen's internal `desugar_module`
|
||
call is harmless on a lifted module — no `Term::LetRec`
|
||
remains, so the LetRec arm is never invoked.
|
||
- `examples/local_rec_let_capture.ailx` + `.ail.json` (48 LOC
|
||
source). `count_below(n)` returns how many `i` in `1..=n` are
|
||
strictly less than a `let threshold = (app + 5 5)` computed
|
||
inside the enclosing fn. The recursive helper `loop` captures
|
||
`threshold` (Let-bound; type unknown until typecheck) and
|
||
recurses on its own counter `i`. The lift produces
|
||
`loop$lr_0(i: Int, n: Int, threshold: Int) -> Int` and
|
||
rewrites every `(app loop X)` to `(app loop$lr_0 X n threshold)`.
|
||
The let-value is `(app + 5 5)` rather than a literal so the
|
||
type-synthesis path is exercised. Drives at 0, 5, 15 →
|
||
output `0\n5\n9\n`.
|
||
- `crates/ail/tests/e2e.rs::local_rec_let_capture_demo` (+18):
|
||
e2e count 39 → 40.
|
||
- `crates/ailang-core/src/desugar.rs::tests`. The 16b.2-era
|
||
`let_rec_capture_let_binding_panics` test was repurposed
|
||
into a positive test
|
||
`let_rec_capture_let_binding_is_deferred_to_post_typecheck`
|
||
that asserts the desugar leaves the LetRec in place (no
|
||
lifted fn appended; original LetRec still reachable). Net
|
||
test count unchanged.
|
||
- `crates/ailang-check/src/lib.rs::tests` (24 → 27, +3):
|
||
`letrec_with_let_binding_capture_typechecks` (positive),
|
||
`letrec_body_wrong_return_type_is_rejected` (negative — body
|
||
returns Bool but declared Int), and
|
||
`lift_letrecs_on_let_capture_produces_synthetic_fn` (asserts
|
||
synthetic FnDef added with augmented signature and call sites
|
||
rewritten).
|
||
- `docs/DESIGN.md` (+12 LOC). Pipeline section gains the
|
||
`lift_letrecs` stage between `check` and `codegen`, with a
|
||
paragraph clarifying it runs only on the `build` / `run`
|
||
paths and that synthetic FnDefs do not appear in
|
||
`CheckedModule.symbols`.
|
||
|
||
**The deferral mechanism.** Given
|
||
`(let y (app + 5 5) (let-rec helper (params x) (type Fn(Int) -> Int)
|
||
(body (app + x y)) (in (app helper 1))))`
|
||
inside an enclosing fn:
|
||
|
||
- Desugar runs. The LetRec's outer scope is `{n: Int (fn-param),
|
||
y: LetBound}`. Free vars of `body` minus `{name, params}` =
|
||
`{+, y}`; intersection with scope = `{y}`. `y` is `LetBound`,
|
||
so the all-or-nothing classifier sees `has_let_bound = true`
|
||
and reconstructs the LetRec with desugared sub-terms instead
|
||
of lifting.
|
||
- Typecheck runs. The new `Term::LetRec` arm in `synth` extends
|
||
locals with `helper: Fn(Int) -> Int` and `x: Int`, synths the
|
||
body to `Int`, unifies with `ret_ty`, and accepts.
|
||
- `lift_letrecs` runs. Walking outer's body, it threads locals.
|
||
At the `Term::Let`, `synth_type` resolves `(app + 5 5) → Int`
|
||
and inserts `y: Int` into locals. At the `Term::LetRec`, the
|
||
capture analyser collects `{y}` and reads its type as `Int`.
|
||
Builds `helper$lr_0(x: Int, y: Int) -> Int`, rewrites
|
||
`(app helper 1)` → `(app helper$lr_0 1 y)`, appends the
|
||
lifted `FnDef`. The Term::LetRec node is replaced by its
|
||
rewritten in-clause.
|
||
- Codegen runs on the lifted module. No `Term::LetRec` remains;
|
||
the four `unreachable!("Term::LetRec eliminated by desugar")`
|
||
arms in codegen stay valid (the message is a slight
|
||
misnomer post-16b.3 — by the time codegen runs, every LetRec
|
||
has been eliminated by desugar OR lift_letrecs — but the
|
||
invariant holds).
|
||
|
||
**Tests: 106 → 110 (+4).**
|
||
|
||
- e2e: 39 → 40 (`local_rec_let_capture_demo`).
|
||
- `ailang-check::tests`: 24 → 27 (the three new LetRec tests).
|
||
- `ailang-core::desugar::tests`: 9 → 9 (one `#[should_panic]`
|
||
test repurposed to a positive defer-check test — net 0).
|
||
|
||
**Cumulative state, post-16b.3.**
|
||
|
||
- Stdlib unchanged (5 modules, 29 combinators).
|
||
- `Term` enum: 11 variants (unchanged). All pre-16b.3 fixtures
|
||
hash bit-identically — additive at the typechecker / new-pass
|
||
level only.
|
||
- Compiler stages: load → desugar → typecheck →
|
||
**lift_letrecs (16b.3)** → codegen. The `check` subcommand
|
||
stops at typecheck and skips the lift; `build` / `run` go all
|
||
the way through.
|
||
- The four `unreachable!("Term::LetRec eliminated by desugar")`
|
||
arms in `ailang-codegen` remain correct: by the time codegen
|
||
runs, no LetRec survives — desugar lifts the 16b.1/16b.2
|
||
cases, lift_letrecs catches the 16b.3 case. The two
|
||
`unreachable!` arms in `ailang-check` were replaced with the
|
||
new typing rule (`verify_tail_positions` and `synth`).
|
||
- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5
|
||
(unchanged).
|
||
|
||
**Queue update post-16b.3.** 16b.3 done. Open: **16b.4** (LetRec
|
||
captures of match-arm pattern bindings — needs ADT-def lookup +
|
||
constructor-field substitution; would slot into the same
|
||
`lift_letrecs` pass with extended pattern-arm walk), **16b.5**
|
||
(closure conversion — lifts the "name-as-value-only" restriction
|
||
for both LetRec and Lam; out of scope for this iter line),
|
||
**16b.6** (LetRec inside a `Type::Forall`-quantified fn — needs
|
||
synthesised `Forall` for the lifted signature), **16b.7**
|
||
(nested LetRec mutual capture — generalised lifting that
|
||
accumulates captures across nesting; partly already handled by
|
||
the post-order traversal in `lift_letrecs` but the mutual case
|
||
is genuinely harder). 16d (chain-machinery exhaustiveness or
|
||
`__unreachable__`), 16e (`==` extension to Bool/Str/Unit),
|
||
17a (per-fn arena, gated) unchanged.
|