Iter 14e: explicit, verified tail calls

Decision 8 ships. Term::App and Term::Do gain tail: bool with
serde-default false and skip-when-false serialisation. New
typecheck pass verify_tail_positions enforces tail-position rules
(Scheme-style propagation through match arms, seq.rhs, let body,
lam body). Codegen emits musttail call for marked App calls.

Hash invariance verified: only the two migrated print_list defs
(list_map_poly.print_list, sort.print_list) changed hashes; all
other defs across all 18 fixtures kept bit-identical hashes —
confirms the skip-when-false serialisation rule works.

Tests 76 -> 79: tail_call_in_non_tail_position_is_rejected,
tail_call_in_tail_position_is_accepted, plus an IR-grep e2e test
asserting that print_list's recursive call site emits musttail
in the lowered IR. Existing 25 e2e tests unchanged in behaviour
(map -> [2,3,4], sort -> sorted list).

IR evidence at the recursive site:
  %v7 = musttail call i8 @ail_list_map_poly_print_list(ptr %v6)
  ret i8 %v7

Two deviations called out in the implementer report and JOURNAL:
1. tail-do uses tail call, not musttail. Cross-type return
   (runtime helpers return i32, AILang Unit is i8) would have
   LLVM reject musttail. Path is implemented but not exercised
   by any current fixture; proper fix is runtime-helper signature
   change, punted.
2. block_terminated flag in codegen so tail-call emit
   (musttail call + ret) doesn't get a duplicate trailing ret
   from surrounding code (match-arm phi, fn-body, lambda thunk).
   Internal plumbing; required for IR well-formedness.

Form (A) productions now at ~30, exactly the constraint-1
budget. Future surface additions need to retire something or
explicit-budget-rebalance in DESIGN.md.

GC notes from implementer survey land in JOURNAL:
- Allocations cluster in lower_ctor; every term-ctor does
  malloc(8+8n).
- Tail recursion does not reduce alloc pressure, only stack.
  For map-style ctor-blocked recursions, allocation IS the
  bottleneck.
- Per-fn arena is sound only when fn return type contains no
  boxed ADT. Most current fixtures violate this.

Plan 14f: Boehm conservative GC (GC_malloc, -lgc) as a first
cut. Single-iter integration, no AST/schema change. Stress
test: build a 100k Cons list, observe RSS doesn't blow up.

After 14f the language is feature-complete enough for stdlib
work (15a).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 17:16:28 +02:00
parent 8d97a924de
commit d64031c234
16 changed files with 768 additions and 81 deletions
+194 -58
View File
@@ -359,6 +359,12 @@ struct Emitter<'a> {
/// Current basic block label. Set by `start_block` and is
/// the single source of truth for `phi` operands.
current_block: String,
/// Iter 14e: true while the current block already ends in a
/// terminator (currently only `ret` after a `musttail call`).
/// Callers in the term lowering walk consult this to skip
/// fall-through `br` emission and to omit the value from a
/// surrounding match-arm phi. Reset by [`Self::start_block`].
block_terminated: bool,
/// Iter 7: SSA value (or `@global`) -> its FnSig, for first-class
/// function values. Populated whenever we lower a `Term::Var` to a
/// top-level fn pointer or when a fn-typed parameter is bound at
@@ -479,6 +485,7 @@ impl<'a> Emitter<'a> {
types,
ctor_index,
current_block: String::new(),
block_terminated: false,
ssa_fn_sigs: BTreeMap::new(),
current_def: String::new(),
lam_counter: 0,
@@ -490,6 +497,7 @@ impl<'a> Emitter<'a> {
self.body.push_str(label);
self.body.push_str(":\n");
self.current_block = label.to_string();
self.block_terminated = false;
}
fn emit_module(&mut self) -> Result<()> {
@@ -694,14 +702,21 @@ impl<'a> Emitter<'a> {
self.start_block("entry");
let (val, val_ty) = self.lower_term(&f.body)?;
if val_ty != llvm_ret {
return Err(CodegenError::Internal(format!(
"fn `{}`: body type {val_ty} != return type {llvm_ret}",
f.name
)));
if !self.block_terminated {
if val_ty != llvm_ret {
return Err(CodegenError::Internal(format!(
"fn `{}`: body type {val_ty} != return type {llvm_ret}",
f.name
)));
}
self.body
.push_str(&format!(" ret {val_ty} {val}\n}}\n\n"));
} else {
// Iter 14e: a `tail-app`/`tail-do` at the body root already
// emitted its own `ret` (after `musttail call`). Just close
// the function body — no fall-through ret.
self.body.push_str("}\n\n");
}
self.body
.push_str(&format!(" ret {val_ty} {val}\n}}\n\n"));
// Iter 8b: flush lambda thunks collected while lowering this fn's
// body. They go after the closing `}` of the parent fn, before
@@ -801,7 +816,7 @@ impl<'a> Emitter<'a> {
self.locals.pop();
r
}
Term::App { callee, args } => {
Term::App { callee, args, tail } => {
// Direct call when the callee is a `Var` referring to a
// statically-known target (builtin, current-module fn,
// qualified cross-module fn) AND not shadowed by a local.
@@ -811,7 +826,7 @@ impl<'a> Emitter<'a> {
if let Term::Var { name } = callee.as_ref() {
let shadowed = self.locals.iter().any(|(n, _, _, _)| n == name);
if !shadowed && self.is_static_callee(name) {
return self.lower_app(name, args);
return self.lower_app(name, args, *tail);
}
}
let (callee_ssa, callee_ty) = self.lower_term(callee)?;
@@ -829,9 +844,9 @@ impl<'a> Emitter<'a> {
"indirect call: no FnSig recorded for `{callee_ssa}`"
))
})?;
self.emit_indirect_call(&callee_ssa, &sig, args)
self.emit_indirect_call(&callee_ssa, &sig, args, *tail)
}
Term::Do { op, args } => self.lower_effect_op(op, args),
Term::Do { op, args, tail } => self.lower_effect_op(op, args, *tail),
Term::Ctor { type_name, ctor, args } => self.lower_ctor(type_name, ctor, args),
Term::Match { scrutinee, arms } => self.lower_match(scrutinee, arms),
Term::Lam { params, param_tys, ret_ty, effects: _, body } => {
@@ -840,6 +855,12 @@ impl<'a> Emitter<'a> {
Term::Seq { lhs, rhs } => {
// Iter 10: lower lhs for its effects, discard the SSA;
// lower rhs and return its value as the whole expression.
// Iter 14e: lhs may not legally be a `tail` call (the
// typechecker rejects that), so `block_terminated` is
// false after it. rhs is in the same tail context as the
// surrounding seq, so a `tail-app` there will set
// `block_terminated`; the outer match-arm/fn-body
// handler honours that.
let _ = self.lower_term(lhs)?;
self.lower_term(rhs)
}
@@ -1083,17 +1104,22 @@ impl<'a> Emitter<'a> {
for _ in 0..pushed {
self.locals.pop();
}
phi_inputs.push((val, self.current_block.clone()));
self.body
.push_str(&format!(" br label %{join_lbl}\n"));
if let Some(rt) = &result_ty {
if rt != &vty {
return Err(CodegenError::Internal(format!(
"match arm result type {vty} != {rt}"
)));
// Iter 14e: if the arm body lowered to a `musttail call` +
// `ret`, the block is already terminated. Skip the
// fall-through `br` and exclude this arm from the join phi.
if !self.block_terminated {
phi_inputs.push((val, self.current_block.clone()));
self.body
.push_str(&format!(" br label %{join_lbl}\n"));
if let Some(rt) = &result_ty {
if rt != &vty {
return Err(CodegenError::Internal(format!(
"match arm result type {vty} != {rt}"
)));
}
} else {
result_ty = Some(vty);
}
} else {
result_ty = Some(vty);
}
}
@@ -1112,17 +1138,19 @@ impl<'a> Emitter<'a> {
for _ in 0..pushed {
self.locals.pop();
}
phi_inputs.push((val, self.current_block.clone()));
self.body
.push_str(&format!(" br label %{join_lbl}\n"));
if let Some(rt) = &result_ty {
if rt != &vty {
return Err(CodegenError::Internal(format!(
"match default arm result type {vty} != {rt}"
)));
if !self.block_terminated {
phi_inputs.push((val, self.current_block.clone()));
self.body
.push_str(&format!(" br label %{join_lbl}\n"));
if let Some(rt) = &result_ty {
if rt != &vty {
return Err(CodegenError::Internal(format!(
"match default arm result type {vty} != {rt}"
)));
}
} else {
result_ty = Some(vty);
}
} else {
result_ty = Some(vty);
}
} else {
// Typechecker guarantees exhaustiveness, so unreachable.
@@ -1130,6 +1158,18 @@ impl<'a> Emitter<'a> {
}
// join
// Iter 14e: if every arm tail-called and terminated its own
// block, no predecessor branches into the join. Mark the whole
// match as block-terminated and emit no join body — the
// surrounding context (top-level fn body, seq rhs, etc.) checks
// `block_terminated` before any fall-through emission.
if phi_inputs.is_empty() {
self.block_terminated = true;
// Return a dummy SSA + type that won't be consumed. Use the
// result_ty if any arm produced one, else fall back to i8.
let rt = result_ty.unwrap_or_else(|| "i8".into());
return Ok(("0".into(), rt));
}
self.start_block(&join_lbl);
let phi = self.fresh_ssa();
let rt = result_ty.unwrap_or_else(|| "i64".into());
@@ -1195,20 +1235,43 @@ impl<'a> Emitter<'a> {
self.start_block(&then_lbl);
let (then_v, then_ty) = self.lower_term(true_body)?;
// Nested code in the `then` body may have changed the
// block label — phi must see the last actual block.
// Iter 14e: a tail-call in this arm already terminated its
// block; skip its branch to join and exclude from phi.
let then_terminated = self.block_terminated;
let then_block_end = self.current_block.clone();
self.body.push_str(&format!(" br label %{join_lbl}\n"));
if !then_terminated {
self.body.push_str(&format!(" br label %{join_lbl}\n"));
}
self.start_block(&else_lbl);
let (else_v, else_ty) = self.lower_term(false_body)?;
if then_ty != else_ty {
let else_terminated = self.block_terminated;
let else_block_end = self.current_block.clone();
if !then_terminated && !else_terminated && then_ty != else_ty {
return Err(CodegenError::Internal(format!(
"Bool-match arms type mismatch: {then_ty} vs {else_ty}"
)));
}
let else_block_end = self.current_block.clone();
self.body.push_str(&format!(" br label %{join_lbl}\n"));
if !else_terminated {
self.body.push_str(&format!(" br label %{join_lbl}\n"));
}
// Iter 14e: if both arms terminated, the whole match is
// terminated and no join is reachable. Mark and bail.
if then_terminated && else_terminated {
self.block_terminated = true;
return Ok(("0".into(), then_ty));
}
// If exactly one arm terminated, the join receives only the
// other arm's value — no phi node is needed.
if then_terminated {
self.start_block(&join_lbl);
return Ok((else_v, else_ty));
}
if else_terminated {
self.start_block(&join_lbl);
return Ok((then_v, then_ty));
}
self.start_block(&join_lbl);
let phi = self.fresh_ssa();
@@ -1235,7 +1298,7 @@ impl<'a> Emitter<'a> {
Ok((phi, then_ty))
}
fn lower_app(&mut self, name: &str, args: &[Term]) -> Result<(String, String)> {
fn lower_app(&mut self, name: &str, args: &[Term], tail: bool) -> Result<(String, String)> {
// Built-in arithmetic / comparison.
if let Some((instr, ret_ty)) = builtin_binop(name) {
if args.len() != 2 {
@@ -1249,6 +1312,11 @@ impl<'a> Emitter<'a> {
self.body.push_str(&format!(
" {dst} = {instr} i64 {a}, {b}\n"
));
// Builtins are not function calls in LLVM (they're inline
// arithmetic); `tail` annotation has nothing to act on.
// The typechecker accepts the marker but it is a no-op
// here. (Iter 14e survey: no fixture marks a builtin tail.)
let _ = tail;
return Ok((dst, ret_ty.into()));
}
if name == "not" {
@@ -1277,7 +1345,7 @@ impl<'a> Emitter<'a> {
.get(&target_module)
.is_some_and(|m| m.contains_key(suffix))
{
return self.lower_polymorphic_call(&target_module, suffix, args);
return self.lower_polymorphic_call(&target_module, suffix, args, tail);
}
let target_fns = self
.module_user_fns
@@ -1295,7 +1363,7 @@ impl<'a> Emitter<'a> {
"cross-module call `{name}`: def `{suffix}` not in module `{target_module}`"
))
})?;
return self.emit_call(&target_module, suffix, &sig, args);
return self.emit_call(&target_module, suffix, &sig, args, tail);
}
// Polymorphic def in the current module?
@@ -1305,7 +1373,7 @@ impl<'a> Emitter<'a> {
.is_some_and(|m| m.contains_key(name))
{
let owner = self.module_name.to_string();
return self.lower_polymorphic_call(&owner, name, args);
return self.lower_polymorphic_call(&owner, name, args, tail);
}
// User function in the current module?
@@ -1315,7 +1383,7 @@ impl<'a> Emitter<'a> {
.and_then(|m| m.get(name))
.cloned()
{
return self.emit_call(self.module_name, name, &sig, args);
return self.emit_call(self.module_name, name, &sig, args, tail);
}
Err(CodegenError::Internal(format!(
@@ -1332,6 +1400,7 @@ impl<'a> Emitter<'a> {
owner_module: &str,
def_name: &str,
args: &[Term],
tail: bool,
) -> Result<(String, String)> {
let fdef = self
.module_polymorphic_fns
@@ -1408,10 +1477,16 @@ impl<'a> Emitter<'a> {
.collect::<Vec<_>>()
.join(", ");
let dst = self.fresh_ssa();
let call_kw = if tail { "musttail call" } else { "call" };
self.body.push_str(&format!(
" {dst} = call {ret} @{mangled}({arglist})\n",
" {dst} = {call_kw} {ret} @{mangled}({arglist})\n",
ret = llvm_ret,
));
if tail {
self.body
.push_str(&format!(" ret {ret} {dst}\n", ret = llvm_ret));
self.block_terminated = true;
}
Ok((dst, llvm_ret))
}
@@ -1421,6 +1496,7 @@ impl<'a> Emitter<'a> {
target_def: &str,
sig: &FnSig,
args: &[Term],
tail: bool,
) -> Result<(String, String)> {
let mut compiled_args = Vec::new();
for (a, exp_ty) in args.iter().zip(sig.params.iter()) {
@@ -1438,12 +1514,23 @@ impl<'a> Emitter<'a> {
.collect::<Vec<_>>()
.join(", ");
let dst = self.fresh_ssa();
// Iter 14e: emit `musttail call ... ret` for `tail: true`. The
// call SSA flows directly into the `ret`, satisfying LLVM's
// "must immediately ret" rule. Same calling convention and
// signature as the surrounding fn (the typechecker enforces
// type compatibility).
let call_kw = if tail { "musttail call" } else { "call" };
self.body.push_str(&format!(
" {dst} = call {ret} @ail_{module}_{name}({arglist})\n",
" {dst} = {call_kw} {ret} @ail_{module}_{name}({arglist})\n",
ret = sig.ret,
module = target_module,
name = target_def,
));
if tail {
self.body
.push_str(&format!(" ret {ret} {dst}\n", ret = sig.ret));
self.block_terminated = true;
}
Ok((dst, sig.ret.clone()))
}
@@ -1457,6 +1544,7 @@ impl<'a> Emitter<'a> {
callee_ssa: &str,
sig: &FnSig,
args: &[Term],
tail: bool,
) -> Result<(String, String)> {
if args.len() != sig.params.len() {
return Err(CodegenError::Internal(format!(
@@ -1505,11 +1593,25 @@ impl<'a> Emitter<'a> {
param_tys.push_str(pt);
}
let dst = self.fresh_ssa();
// Iter 14e: indirect tail calls. Same `musttail`/`ret` shape as
// emit_call. The thunk's signature uniformly inserts an
// `env_ptr` first arg, but a `musttail call` to a thunk whose
// signature exactly matches the parent fn's prototype +
// env_ptr is malformed (parent has no env_ptr in its prototype).
// For the MVP no fixture marks an indirect tail call; we honour
// the flag by emitting `musttail call` (LLVM verifier will
// catch a real signature mismatch at IR-verification time).
let call_kw = if tail { "musttail call" } else { "call" };
self.body.push_str(&format!(
" {dst} = call {ret} ({ptys}) {thunk}({arglist})\n",
" {dst} = {call_kw} {ret} ({ptys}) {thunk}({arglist})\n",
ret = sig.ret,
ptys = param_tys,
));
if tail {
self.body
.push_str(&format!(" ret {ret} {dst}\n", ret = sig.ret));
self.block_terminated = true;
}
Ok((dst, sig.ret.clone()))
}
@@ -1594,6 +1696,8 @@ impl<'a> Emitter<'a> {
let saved_locals = std::mem::take(&mut self.locals);
let saved_counter = self.counter;
let saved_block = std::mem::take(&mut self.current_block);
let saved_terminated = self.block_terminated;
self.block_terminated = false;
let saved_sigs = std::mem::take(&mut self.ssa_fn_sigs);
// Lambdas inside lambdas are fine: they get their own counter
// namespace within the enclosing thunk. They share the
@@ -1653,13 +1757,17 @@ impl<'a> Emitter<'a> {
}
let (body_v, body_ty) = self.lower_term(lam_body)?;
if body_ty != llvm_ret {
return Err(CodegenError::Internal(format!(
"lambda `{thunk_name}`: body type {body_ty} != return type {llvm_ret}"
)));
if !self.block_terminated {
if body_ty != llvm_ret {
return Err(CodegenError::Internal(format!(
"lambda `{thunk_name}`: body type {body_ty} != return type {llvm_ret}"
)));
}
self.body
.push_str(&format!(" ret {body_ty} {body_v}\n}}\n\n"));
} else {
self.body.push_str("}\n\n");
}
self.body
.push_str(&format!(" ret {body_ty} {body_v}\n}}\n\n"));
// Park the thunk text in the deferred queue and restore outer
// emitter state.
@@ -1669,6 +1777,7 @@ impl<'a> Emitter<'a> {
self.locals = saved_locals;
self.counter = saved_counter;
self.current_block = saved_block;
self.block_terminated = saved_terminated;
self.ssa_fn_sigs = saved_sigs;
// 3. Emit allocation + capture filling + closure-pair packing
@@ -1751,7 +1860,7 @@ impl<'a> Emitter<'a> {
captures.push(name.clone());
}
}
Term::App { callee, args } => {
Term::App { callee, args, .. } => {
Self::collect_captures(callee, bound, captures, captures_set, builtins, top_level);
for a in args {
Self::collect_captures(a, bound, captures, captures_set, builtins, top_level);
@@ -1873,7 +1982,19 @@ impl<'a> Emitter<'a> {
))
}
fn lower_effect_op(&mut self, op: &str, args: &[Term]) -> Result<(String, String)> {
fn lower_effect_op(&mut self, op: &str, args: &[Term], tail: bool) -> Result<(String, String)> {
// Iter 14e: `musttail` requires identical caller/callee
// prototypes (same return type, same param types). The MVP's
// runtime print helpers (`printf`, `puts`) return `i32`, but the
// AILang fn enclosing a `tail-do io/print_*` returns `Unit`
// (`i8`). `musttail` would be rejected by the LLVM verifier.
// We therefore use the `tail` keyword (LLVM IR optimisation
// hint, NOT a guarantee) for `tail: true` do-ops. The optimiser
// is free to TCO it; if it can't, the call falls back to a
// normal call. The body of the AILang fn afterwards is empty
// (the op was the last thing), so we close it with `ret i8 0`.
let _ = tail;
let call_kw = if tail { "tail call" } else { "call" };
match op {
"io/print_int" => {
if args.len() != 1 {
@@ -1889,8 +2010,12 @@ impl<'a> Emitter<'a> {
}
let fmt = self.intern_string("fmt_int", "%lld\n");
self.body.push_str(&format!(
" call i32 (ptr, ...) @printf(ptr @{fmt}, i64 {v})\n"
" {call_kw} i32 (ptr, ...) @printf(ptr @{fmt}, i64 {v})\n"
));
if tail {
self.body.push_str(" ret i8 0\n");
self.block_terminated = true;
}
Ok(("0".into(), "i8".into()))
}
"io/print_str" => {
@@ -1906,7 +2031,11 @@ impl<'a> Emitter<'a> {
));
}
self.body
.push_str(&format!(" call i32 @puts(ptr {v})\n"));
.push_str(&format!(" {call_kw} i32 @puts(ptr {v})\n"));
if tail {
self.body.push_str(" ret i8 0\n");
self.block_terminated = true;
}
Ok(("0".into(), "i8".into()))
}
"io/print_bool" => {
@@ -1942,6 +2071,10 @@ impl<'a> Emitter<'a> {
));
self.body.push_str(&format!(" br label %{join_lbl}\n"));
self.start_block(&join_lbl);
if tail {
self.body.push_str(" ret i8 0\n");
self.block_terminated = true;
}
Ok(("0".into(), "i8".into()))
}
other => Err(CodegenError::Internal(format!(
@@ -2049,7 +2182,7 @@ impl<'a> Emitter<'a> {
ret: ret_ty.clone(),
effects: effects.clone(),
}),
Term::App { callee, args } => {
Term::App { callee, args, .. } => {
let cty = self.synth_with_extras(callee, extras)?;
match cty {
Type::Fn { ret, .. } => Ok(*ret),
@@ -2377,18 +2510,20 @@ fn apply_subst_to_type(t: &Type, subst: &BTreeMap<String, Type>) -> Type {
fn apply_subst_to_term(t: &Term, subst: &BTreeMap<String, Type>) -> Term {
match t {
Term::Lit { .. } | Term::Var { .. } => t.clone(),
Term::App { callee, args } => Term::App {
Term::App { callee, args, tail } => Term::App {
callee: Box::new(apply_subst_to_term(callee, subst)),
args: args.iter().map(|a| apply_subst_to_term(a, subst)).collect(),
tail: *tail,
},
Term::Let { name, value, body } => Term::Let {
name: name.clone(),
value: Box::new(apply_subst_to_term(value, subst)),
body: Box::new(apply_subst_to_term(body, subst)),
},
Term::Do { op, args } => Term::Do {
Term::Do { op, args, tail } => Term::Do {
op: op.clone(),
args: args.iter().map(|a| apply_subst_to_term(a, subst)).collect(),
tail: *tail,
},
Term::Ctor { type_name, ctor, args } => Term::Ctor {
type_name: type_name.clone(),
@@ -2559,6 +2694,7 @@ mod tests {
Term::Var { name: "a".into() },
Term::Var { name: "b".into() },
],
tail: false,
},
doc: None,
}),