Iter 18e: (drop-iterative) annotation + worklist allocator

Closes the 18-arc's stack-recursion limit. Recursive drop
cascades from 18c.4 overflow on long ADT chains (Linux's 8 MB
default stack maxes out around 1M cells of List). The new
opt-in (drop-iterative) annotation on a Def::Type switches the
synthesised drop_<m>_<T> body from recursive to iterative-with-
explicit-worklist for that type.

Schema:
- TypeDef.drop_iterative: bool. Default false; serde-skip
  when false so existing fixtures' canonical JSON hashes stay
  stable.
- Form-A: (drop-iterative) clause inside (data T ...).

Worklist runtime (4 new ABI symbols in runtime/rc.c):
- ailang_drop_worklist_new(initial_capacity)
- ailang_drop_worklist_push(wl, ptr)
- ailang_drop_worklist_pop(wl) -> ptr
- ailang_drop_worklist_free(wl)

Heap stretchy buffer, doubling on overflow, null-filtering on
push. Lean 4 / Roc precedent documented in the runtime; the
slot-repurposing strategy was considered and rejected because
not every box has a free pointer-typed slot to thread the
worklist through (Cons head is i64, slot 1 is ptr but it's
the field we're following — no free slot).

Codegen (emit_iterative_drop_fn_for_type): for a
drop_iterative type, drop_<m>_<T>(ptr %p) emits a worklist
loop. Fields of the SAME annotated type push onto the
worklist (mono-typed); fields of DIFFERENT types call their
own drop fn directly (recursive on those, but only if THEY
are themselves recursive — i.e. one level of cascade jump
maximum). Mono-typed-worklist is sound for the deep-self-
recursion case the iter targets (List of List of T just
needs the spine flattened).

Tests:
- examples/rc_drop_iterative_long_list — 1M-cell List of Int
  with (drop-iterative) annotation.
- alloc_rc_drop_iterative_handles_million_cell_list E2E —
  builds + runs under --alloc=rc, asserts clean exit. With
  annotation: exits 0. Without annotation (control): SIGSEGV
  at exit code 139 (verified by hand). Worklist is load-
  bearing.
- iter18e_drop_iterative_emits_worklist_body_no_self_recursion
  IR-shape: worklist body has br to loop_head AND no direct
  recursive call into drop_<m>_<T>.
- iter18e_no_annotation_keeps_recursive_drop_body — control:
  unannotated variant still emits the 18c.4 recursive shape.
- 3 surface parse-tests for the annotation round-trip.

Test deltas: e2e 58 -> 61 (+3), surface 18 -> 21 (+3). All
other buckets unchanged. cargo test --workspace green.

Known debt (deliberate):
- Mono-typed worklist: cross-type drop-iterative fields call
  the other type's drop fn directly. A heterogeneous
  worklist would be more general but adds tag tracking
  complexity for a case (drop-iterative T containing
  drop-iterative T') that's narrower than the deep-self-
  recursion target. Documented in
  emit_iterative_drop_fn_for_type's doc.
- Closure / Type::Var / Type::Forall fields fall back to
  shallow ailang_rc_dec via field_drop_call — same as the
  recursive variant.
- Dynamic-tag partial-drop fallback (head_or_zero epilogue
  shallow dec when moved_slots non-empty) — out of scope per
  brief.
This commit is contained in:
2026-05-08 12:53:09 +02:00
parent 5e401a3520
commit ce6ab8ee44
12 changed files with 763 additions and 2 deletions
+176
View File
@@ -1798,3 +1798,179 @@ fn alloc_rc_own_param_dec_at_fn_return() {
"head_or_zero's `%arg_xs` drop must appear BEFORE the `ret i64` instruction. Pre-ret slice was:\n{pre_ret}"
);
}
/// Iter 18e: `(drop-iterative)` opt-in annotation. The fixture
/// declares `IntList` with `(drop-iterative)` and runs an N=1,000,000
/// cell list through `head_or_zero`. The fn signature is
/// `(own (con IntList))` — 18d.4 emits a drop on the param at fn
/// return; under 18e that drop dispatches into the worklist body of
/// `drop_<m>_<IntList>` (or, in the canonical case here, into the
/// arm-close drop on `t` since the Cons arm pattern-binds the tail).
/// Either way, freeing the 1M-cell chain runs the iterative loop in
/// `runtime/rc.c` instead of recursing 1M frames deep on the C
/// stack.
///
/// Why 1M and not 100K: at 100K the recursive cascade still fits
/// in Linux's 8MB default stack (~64B/frame). 1M overflows clean
/// without the annotation (SIGSEGV) and runs cleanly with it. The
/// test would be a false-negative — passing without the iterative
/// body — at 100K.
///
/// Properties guarded:
/// 1. `--alloc=rc` produces stdout `1\n` (the head of the
/// tail-recursively-built `[1, 2, ..., 1_000_000]`).
/// 2. The binary exits cleanly (status 0; no SIGSEGV from a
/// recursive cascade overflowing the C stack — which DOES
/// happen on the same fixture with the annotation removed,
/// hand-verified during 18e implementation).
/// 3. `--alloc=gc` produces the same stdout (allocator-equivalence
/// backstop; 1M cells at 24B each ≈ 24MB, well within Boehm's
/// capacity).
#[test]
fn alloc_rc_drop_iterative_handles_million_cell_list() {
let example = "rc_drop_iterative_long_list.ail.json";
let stdout_gc = build_and_run_with_alloc(example, "gc");
let stdout_rc = build_and_run_with_alloc(example, "rc");
assert_eq!(stdout_rc.trim(), "1");
assert_eq!(stdout_gc.trim(), "1");
assert_eq!(
stdout_gc, stdout_rc,
"alloc=rc must match alloc=gc on rc_drop_iterative_long_list"
);
}
/// Iter 18e: IR-shape signature of `(drop-iterative)`. The drop fn
/// for the annotated type contains a worklist loop (`br label
/// %loop_head`, the runtime-helper calls, a backedge from each ctor
/// arm) and does NOT contain a recursive `call void @drop_<m>_<T>(...)`
/// against itself — the recursion has been replaced by worklist
/// dispatch.
#[test]
fn iter18e_drop_iterative_emits_worklist_body_no_self_recursion() {
let example = "rc_drop_iterative_long_list.ail.json";
let manifest_dir = env!("CARGO_MANIFEST_DIR");
let workspace = Path::new(manifest_dir).parent().unwrap().parent().unwrap();
let json_path = workspace.join("examples").join(example);
let ws = ailang_core::load_workspace(&json_path).expect("load workspace");
let mut lifted_modules = std::collections::BTreeMap::new();
for (mname, m) in &ws.modules {
let desugared = ailang_core::desugar::desugar_module(m);
let lifted = ailang_check::lift_letrecs(&desugared)
.unwrap_or_else(|e| panic!("lift_letrecs in module `{mname}`: {e:?}"));
lifted_modules.insert(mname.clone(), lifted);
}
let lifted_ws = ailang_core::Workspace {
entry: ws.entry.clone(),
modules: lifted_modules,
root_dir: ws.root_dir.clone(),
};
let ir = ailang_codegen::lower_workspace_with_alloc(
&lifted_ws,
ailang_codegen::AllocStrategy::Rc,
)
.expect("lower workspace under rc");
// Locate the drop fn body. There is exactly one
// `define void @drop_rc_drop_iterative_long_list_IntList(`.
let drop_start = ir
.find("define void @drop_rc_drop_iterative_long_list_IntList(")
.expect("drop fn definition present");
let drop_end_offset = ir[drop_start..]
.find("\n}\n")
.expect("drop fn ends with `}`");
let drop_body = &ir[drop_start..drop_start + drop_end_offset];
// Worklist signatures: backedge label, push, pop, free, all
// present in the body.
assert!(
drop_body.contains("br label %loop_head"),
"iterative drop body must contain a `br label %loop_head` backedge. Body was:\n{drop_body}"
);
assert!(
drop_body.contains("call ptr @ailang_drop_worklist_new()"),
"iterative drop body must call ailang_drop_worklist_new. Body was:\n{drop_body}"
);
assert!(
drop_body.contains("call ptr @ailang_drop_worklist_pop(ptr %wl)"),
"iterative drop body must call ailang_drop_worklist_pop. Body was:\n{drop_body}"
);
assert!(
drop_body.contains("call void @ailang_drop_worklist_push(ptr %wl,"),
"iterative drop body must call ailang_drop_worklist_push for same-type fields. Body was:\n{drop_body}"
);
assert!(
drop_body.contains("call void @ailang_drop_worklist_free(ptr %wl)"),
"iterative drop body must call ailang_drop_worklist_free at finish. Body was:\n{drop_body}"
);
// Critical negative: NO recursive self-call in the drop body —
// the whole point of 18e is that the recursion was replaced by
// worklist dispatch. (Calls to other drop fns are still allowed,
// since cross-type ADT fields go through the regular
// `field_drop_call` path.)
assert!(
!drop_body.contains(
"call void @drop_rc_drop_iterative_long_list_IntList("
),
"iterative drop body must NOT recursively call itself — the recursion was replaced by worklist dispatch. Body was:\n{drop_body}"
);
}
/// Iter 18e: control — the same fixture WITHOUT the
/// `(drop-iterative)` annotation must still emit the recursive
/// 18c.4 body. We synthesise the unannotated module on the fly
/// (mutating the workspace's parsed AST) so this test is independent
/// of any on-disk fixture changes.
#[test]
fn iter18e_no_annotation_keeps_recursive_drop_body() {
let example = "rc_drop_iterative_long_list.ail.json";
let manifest_dir = env!("CARGO_MANIFEST_DIR");
let workspace = Path::new(manifest_dir).parent().unwrap().parent().unwrap();
let json_path = workspace.join("examples").join(example);
let ws = ailang_core::load_workspace(&json_path).expect("load workspace");
// Mutate every TypeDef in every module to clear the annotation.
let mut modules = std::collections::BTreeMap::new();
for (mname, m) in &ws.modules {
let mut mc = m.clone();
for d in &mut mc.defs {
if let ailang_core::ast::Def::Type(td) = d {
td.drop_iterative = false;
}
}
let desugared = ailang_core::desugar::desugar_module(&mc);
let lifted = ailang_check::lift_letrecs(&desugared)
.unwrap_or_else(|e| panic!("lift_letrecs in module `{mname}`: {e:?}"));
modules.insert(mname.clone(), lifted);
}
let lifted_ws = ailang_core::Workspace {
entry: ws.entry.clone(),
modules,
root_dir: ws.root_dir.clone(),
};
let ir = ailang_codegen::lower_workspace_with_alloc(
&lifted_ws,
ailang_codegen::AllocStrategy::Rc,
)
.expect("lower workspace under rc");
let drop_start = ir
.find("define void @drop_rc_drop_iterative_long_list_IntList(")
.expect("drop fn definition present");
let drop_end_offset = ir[drop_start..]
.find("\n}\n")
.expect("drop fn ends with `}`");
let drop_body = &ir[drop_start..drop_start + drop_end_offset];
// The 18c.4 recursive shape signature: a self-call AND no
// worklist plumbing.
assert!(
drop_body.contains(
"call void @drop_rc_drop_iterative_long_list_IntList(ptr %v"
),
"without (drop-iterative), the body must still recursively cascade through the tail. Body was:\n{drop_body}"
);
assert!(
!drop_body.contains("ailang_drop_worklist_new"),
"without (drop-iterative), the body must not call worklist runtime helpers. Body was:\n{drop_body}"
);
}
+13
View File
@@ -2192,6 +2192,7 @@ mod tests {
},
],
doc: None,
drop_iterative: false,
}),
fn_def(
"f",
@@ -2242,6 +2243,7 @@ mod tests {
},
],
doc: None,
drop_iterative: false,
}),
fn_def(
"f",
@@ -2543,6 +2545,7 @@ mod tests {
fields: vec![Type::Var { name: "a".into() }],
}],
doc: None,
drop_iterative: false,
});
// fn make :: () -> Box<Int> = MkBox(42)
let make = fn_def(
@@ -2585,6 +2588,7 @@ mod tests {
fields: vec![Type::Var { name: "a".into() }],
}],
doc: None,
drop_iterative: false,
});
let unbox = Def::Fn(FnDef {
name: "unbox".into(),
@@ -2677,6 +2681,7 @@ mod tests {
fields: vec![Type::Var { name: "a".into() }],
}],
doc: None,
drop_iterative: false,
});
let bad = fn_def(
"bad",
@@ -2763,6 +2768,7 @@ mod tests {
},
],
doc: None,
drop_iterative: false,
}),
fn_def(
"loop",
@@ -2815,6 +2821,7 @@ mod tests {
fields: vec![Type::Var { name: "a".into() }],
}],
doc: None,
drop_iterative: false,
}),
Def::Type(TypeDef {
name: "Bag".into(),
@@ -2824,6 +2831,7 @@ mod tests {
fields: vec![Type::Var { name: "a".into() }],
}],
doc: None,
drop_iterative: false,
}),
],
};
@@ -2953,6 +2961,7 @@ mod tests {
vars: vec![],
ctors: vec![Ctor { name: "Mk".into(), fields: vec![] }],
doc: None,
drop_iterative: false,
})],
};
let lib_b = Module {
@@ -2964,6 +2973,7 @@ mod tests {
vars: vec![],
ctors: vec![Ctor { name: "Mk".into(), fields: vec![] }],
doc: None,
drop_iterative: false,
})],
};
let consumer = Module {
@@ -3049,6 +3059,7 @@ mod tests {
},
],
doc: None,
drop_iterative: false,
})],
};
// Consumer: builds `Cons 1 (Cons 2 (Nil))` via qualified
@@ -3141,6 +3152,7 @@ mod tests {
},
],
doc: None,
drop_iterative: false,
}),
fn_def(
"drain",
@@ -3502,6 +3514,7 @@ mod tests {
],
}],
doc: None,
drop_iterative: false,
};
let letrec = Term::LetRec {
name: "helper".into(),
+1
View File
@@ -565,6 +565,7 @@ mod tests {
},
],
doc: None,
drop_iterative: false,
}
}
+4
View File
@@ -269,6 +269,7 @@ fn use_after_consume_on_own_param_is_reported() {
fields: vec![],
}],
doc: None,
drop_iterative: false,
});
let m = Module {
@@ -328,6 +329,7 @@ fn consume_while_borrowed_in_sibling_arg_is_reported() {
fields: vec![],
}],
doc: None,
drop_iterative: false,
});
// dual_fn: (borrow List) → (own List) → Int.
@@ -437,6 +439,7 @@ fn reuse_as_happy_path_in_map_inc_is_linearity_clean() {
},
],
doc: None,
drop_iterative: false,
});
let map_inc_ty = Type::Fn {
@@ -560,6 +563,7 @@ fn reuse_as_shape_mismatch_is_reported_on_cons_to_nil() {
},
],
doc: None,
drop_iterative: false,
});
let f_ty = Type::Fn {
+225 -1
View File
@@ -421,6 +421,14 @@ fn lower_workspace_inner(ws: &Workspace, alloc: AllocStrategy) -> Result<String>
if matches!(alloc, AllocStrategy::Rc) {
out.push_str("declare void @ailang_rc_inc(ptr)\n");
out.push_str("declare void @ailang_rc_dec(ptr)\n");
// Iter 18e: drop-worklist ABI for `(drop-iterative)` types.
// Declared unconditionally under `--alloc=rc` (the linker
// drops symbols if no emitted fn references them); symmetric
// with inc/dec above. See `runtime/rc.c` for the strategy.
out.push_str("declare ptr @ailang_drop_worklist_new()\n");
out.push_str("declare void @ailang_drop_worklist_push(ptr, ptr)\n");
out.push_str("declare ptr @ailang_drop_worklist_pop(ptr)\n");
out.push_str("declare void @ailang_drop_worklist_free(ptr)\n");
}
// Iter 16e: `==` on `Str` lowers to `@strcmp` followed by
// `icmp eq i32 0`. NUL-terminated strings make this a one-liner;
@@ -752,7 +760,16 @@ impl<'a> Emitter<'a> {
if matches!(self.alloc, AllocStrategy::Rc) {
for def in &self.module.defs {
if let Def::Type(td) = def {
self.emit_drop_fn_for_type(td);
if td.drop_iterative {
// Iter 18e: opt-in iterative-drop body. The
// recursive cascade overflows the C stack on
// long chains (a million-cell list ≈ 8MB
// stack); the iterative variant uses an
// explicit heap-allocated worklist instead.
self.emit_iterative_drop_fn_for_type(td);
} else {
self.emit_drop_fn_for_type(td);
}
}
}
}
@@ -877,6 +894,211 @@ impl<'a> Emitter<'a> {
self.body.push_str(&out);
}
/// Iter 18e: emit `drop_<m>_<T>` for a `(drop-iterative)` type.
/// Replaces the recursive cascade in [`Self::emit_drop_fn_for_type`]
/// with an iterative-with-explicit-worklist body so cells of
/// arbitrary chain depth can free without consuming proportional
/// C stack.
///
/// IR shape:
/// ```text
/// define void @drop_<m>_<T>(ptr %p) {
/// entry:
/// %is_null = icmp eq ptr %p, null
/// br i1 %is_null, label %ret, label %init_wl
/// init_wl:
/// %wl = call ptr @ailang_drop_worklist_new()
/// call void @ailang_drop_worklist_push(ptr %wl, ptr %p)
/// br label %loop_head
/// loop_head:
/// %cur = call ptr @ailang_drop_worklist_pop(ptr %wl)
/// %done = icmp eq ptr %cur, null
/// br i1 %done, label %finish, label %dispatch
/// dispatch:
/// %tag = load i64, ptr %cur, align 8
/// switch i64 %tag, label %dflt [
/// i64 0, label %arm_0
/// ...
/// ]
/// arm_i:
/// for each pointer-typed field f_j of ctor i:
/// %addr = gep %cur, 8 + 8*j
/// %v = load ptr, ptr %addr
/// if field type is T (same as the type being dropped):
/// call void @ailang_drop_worklist_push(ptr %wl, ptr %v)
/// else:
/// call void @drop_<owner>_<F>(ptr %v) ; or @ailang_rc_dec
/// call void @ailang_rc_dec(ptr %cur)
/// br label %loop_head
/// dflt:
/// unreachable
/// finish:
/// call void @ailang_drop_worklist_free(ptr %wl)
/// br label %ret
/// ret:
/// ret void
/// }
/// ```
///
/// Mono-typed worklist. Every pointer pushed onto `%wl` is a `T`
/// (the type being dropped). For a field whose type is `T` itself
/// → push (continues the iterative cascade). For any other ADT
/// field type `T'` → call `drop_<m'>_<T'>` directly: if `T'` is
/// also `(drop-iterative)`, that fn allocates its own worklist
/// instance (no nesting); if `T'` is non-iterative, it recurses
/// stack-wise (depth bounded by the number of *distinct* nested
/// ADTs reachable from `T`, which is small in practice).
///
/// This interpretation of the assignment's "should also use the
/// worklist" clause was chosen because a heterogeneously-typed
/// worklist would require storing a (ptr, drop-handler) tuple per
/// entry plus a vtable dispatch on pop — significant complexity
/// for the case where two distinct ADTs are mutually recursive
/// AND both are drop-iterative AND the chain is millions deep.
/// That triple-conjunct is not on the 18-arc's critical path; if
/// it surfaces in practice, a follow-up iter can extend the
/// worklist entry shape. The mono-typed version captures the
/// stack-overflow-on-long-self-chains problem fully.
fn emit_iterative_drop_fn_for_type(&mut self, td: &TypeDef) {
let m = self.module_name;
let tname = &td.name;
let mut out = String::new();
out.push_str(&format!("define void @drop_{m}_{tname}(ptr %p) {{\n"));
out.push_str("entry:\n");
// Null guard — symmetric with the recursive variant. A null
// payload skips worklist allocation entirely.
out.push_str(" %is_null = icmp eq ptr %p, null\n");
out.push_str(" br i1 %is_null, label %ret, label %init_wl\n");
out.push_str("init_wl:\n");
out.push_str(" %wl = call ptr @ailang_drop_worklist_new()\n");
out.push_str(
" call void @ailang_drop_worklist_push(ptr %wl, ptr %p)\n",
);
out.push_str(" br label %loop_head\n");
out.push_str("loop_head:\n");
out.push_str(
" %cur = call ptr @ailang_drop_worklist_pop(ptr %wl)\n",
);
out.push_str(" %done = icmp eq ptr %cur, null\n");
out.push_str(" br i1 %done, label %finish, label %dispatch\n");
out.push_str("dispatch:\n");
out.push_str(" %tag = load i64, ptr %cur, align 8\n");
let n_ctors = td.ctors.len();
out.push_str(" switch i64 %tag, label %dflt [\n");
for (i, _) in td.ctors.iter().enumerate() {
out.push_str(&format!(" i64 {i}, label %arm_{i}\n"));
}
out.push_str(" ]\n");
// Per-ctor arms. For each pointer-typed field decide push vs
// direct call based on whether the field's type is the same
// as the type being dropped.
let mut local = 0u64;
for (i, ctor) in td.ctors.iter().enumerate() {
out.push_str(&format!("arm_{i}:\n"));
for (j, fty) in ctor.fields.iter().enumerate() {
let lty = llvm_type(fty).unwrap_or_else(|_| "ptr".into());
if lty != "ptr" {
continue;
}
let off = 8 + (j as i64) * 8;
let addr_id = local;
local += 1;
let val_id = local;
local += 1;
out.push_str(&format!(
" %a{addr_id} = getelementptr inbounds i8, ptr %cur, i64 {off}\n"
));
out.push_str(&format!(
" %v{val_id} = load ptr, ptr %a{addr_id}, align 8\n"
));
if self.field_is_same_type(fty, &td.name) {
// Same-type field: push onto the worklist —
// continues the iterative cascade. Null-guarding
// is handled inside `ailang_drop_worklist_push`
// itself (skips null payloads).
out.push_str(&format!(
" call void @ailang_drop_worklist_push(ptr %wl, ptr %v{val_id})\n"
));
} else {
// Different-type field: dispatch to that type's
// own drop fn (which itself decides recursive vs.
// iterative). `field_drop_call` resolves the
// symbol; its null-guard semantics are the same
// as the recursive variant.
let drop_call = self.field_drop_call(fty);
out.push_str(&format!(
" call void @{drop_call}(ptr %v{val_id})\n"
));
}
}
// Dec the outer cell. Worklist holds no other reference
// to this pointer (push happened exactly once on the
// parent's cascade, and pop just removed that entry), so
// the cell's refcount drops by exactly one here. Children
// pushed above keep their own refcounts pending until
// their loop iteration.
out.push_str(" call void @ailang_rc_dec(ptr %cur)\n");
out.push_str(" br label %loop_head\n");
}
// Default arm: unreachable when the typechecker has accepted
// the input. Same shape as the recursive variant.
out.push_str("dflt:\n");
if n_ctors == 0 {
out.push_str(" br label %finish\n");
} else {
out.push_str(" unreachable\n");
}
out.push_str("finish:\n");
out.push_str(" call void @ailang_drop_worklist_free(ptr %wl)\n");
out.push_str(" br label %ret\n");
out.push_str("ret:\n");
out.push_str(" ret void\n");
out.push_str("}\n\n");
self.body.push_str(&out);
}
/// Iter 18e helper: is `fty` the same ADT as `td_name` in the
/// current module? Used by the iterative-drop body to decide
/// "push to worklist" (same type) vs. "call its drop fn directly"
/// (different type).
///
/// Returns `true` only when the field type is a `Type::Con`
/// referencing `td_name` AND the reference resolves to the
/// current module (bare or qualified-but-self). Qualified names
/// pointing at *other* modules are different types — even when
/// they spell the same suffix. Type-vars and fn-types are never
/// the same as the ADT being dropped (parametric self-recursion
/// could bind a var to T, but the bound is invisible at codegen
/// since we don't monomorphise drop fns).
fn field_is_same_type(&self, fty: &Type, td_name: &str) -> bool {
match fty {
Type::Con { name, .. } => {
if name.matches('.').count() == 1 {
let (prefix, suffix) = name.split_once('.').expect("checked");
if suffix != td_name {
return false;
}
// Resolve the prefix; same-module iff the resolved
// target equals self.module_name.
let target = self
.import_map
.get(prefix)
.map(|s| s.as_str())
.unwrap_or(prefix);
target == self.module_name
} else {
name == td_name
}
}
_ => false,
}
}
/// Iter 18c.4: pick the drop-fn symbol to call for a single
/// pointer-typed field. Routes ADT fields to their own
/// `drop_<owner>_<T>` symbol so the recursion cascades through
@@ -4577,6 +4799,7 @@ mod tests {
fields: vec![],
}],
doc: None,
drop_iterative: false,
}),
Def::Fn(FnDef {
name: "main".into(),
@@ -4735,6 +4958,7 @@ mod tests {
},
],
doc: None,
drop_iterative: false,
}),
Def::Fn(FnDef {
name: "main".into(),
+18
View File
@@ -129,8 +129,26 @@ pub struct TypeDef {
/// Optional source-level documentation string.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub doc: Option<String>,
/// Iter 18e: opt-in `(drop-iterative)` annotation. When `true`,
/// codegen emits `drop_<m>_<T>` with an iterative worklist body
/// instead of the recursive cascade — chosen by the LLM-author when
/// the type is expected to form long chains (millions of cells)
/// that would overflow the C stack on free.
///
/// Serialised as `"drop-iterative": true` (kebab-case) when set;
/// the field is omitted when `false` so canonical-JSON hashes of
/// every pre-18e fixture remain bit-stable. See the regression
/// test `iter18e_drop_iterative_default_preserves_hashes` in
/// [`crate::hash`].
#[serde(
default,
rename = "drop-iterative",
skip_serializing_if = "is_false"
)]
pub drop_iterative: bool,
}
/// A single constructor of a [`TypeDef`].
///
/// `fields` is the constructor's positional argument list as types;
+1
View File
@@ -1959,6 +1959,7 @@ mod tests {
],
}],
doc: None,
drop_iterative: false,
};
let letrec = Term::LetRec {
name: "helper".into(),
+110 -1
View File
@@ -364,6 +364,7 @@ impl<'a> Parser<'a> {
}
let mut doc: Option<String> = None;
let mut ctors: Vec<Ctor> = Vec::new();
let mut drop_iterative = false;
loop {
match self.peek_head_ident() {
Some("doc") => {
@@ -373,12 +374,42 @@ impl<'a> Parser<'a> {
Some("ctor") => {
ctors.push(self.parse_ctor()?);
}
Some("drop-iterative") => {
// Iter 18e: `(drop-iterative)` opt-in annotation.
// Takes no arguments — it is a flag. A second
// `(drop-iterative)` clause is a parse error
// (rejected here so that the JSON schema's
// `drop_iterative: bool` round-trips unambiguously).
self.expect_lparen("drop-iterative-attr")?;
self.expect_keyword("drop-iterative")?;
if !matches!(self.peek(), Some(Token { tok: Tok::RParen, .. })) {
let pos = self.peek().map(|t| t.span.start).unwrap_or(0);
return Err(ParseError::Production {
production: "data-def",
message:
"drop-iterative takes no arguments; expected `)`"
.into(),
pos,
});
}
self.expect_rparen("drop-iterative-attr")?;
if drop_iterative {
let pos = self.peek().map(|t| t.span.start).unwrap_or(0);
return Err(ParseError::Production {
production: "data-def",
message: "duplicate `drop-iterative` attribute"
.into(),
pos,
});
}
drop_iterative = true;
}
Some(other) => {
let pos = self.peek().map(|t| t.span.start).unwrap_or(0);
return Err(ParseError::Production {
production: "data-def",
message: format!(
"unknown data attribute `{other}`; expected `doc` or `ctor`"
"unknown data attribute `{other}`; expected `doc`, `ctor`, or `drop-iterative`"
),
pos,
});
@@ -392,6 +423,7 @@ impl<'a> Parser<'a> {
vars,
ctors,
doc,
drop_iterative,
})
}
@@ -1483,6 +1515,83 @@ mod tests {
assert_eq!(term_to_form_a(&parsed), printed);
}
/// Iter 18e: `(data T (drop-iterative))` parses with
/// `drop_iterative = true` AND round-trips through the printer
/// back to the same canonical surface form.
#[test]
fn parses_drop_iterative_annotation_on_data_decl() {
use crate::print::print;
let src = r#"
(module m
(data Tree
(vars a)
(ctor Leaf)
(ctor Node a (con Tree a) (con Tree a))
(drop-iterative)))
"#;
let m = parse(src).expect("parse should succeed");
match &m.defs[0] {
Def::Type(td) => {
assert_eq!(td.name, "Tree");
assert!(td.drop_iterative, "drop_iterative flag must be set");
}
_ => panic!("expected Def::Type"),
}
// Round-trip through the printer.
let printed = print(&m);
let m2 = parse(&printed).expect("re-parse should succeed");
match &m2.defs[0] {
Def::Type(td) => {
assert!(td.drop_iterative, "drop_iterative must survive print/parse");
}
_ => panic!("expected Def::Type"),
}
}
/// Iter 18e: `(data T)` with no `(drop-iterative)` clause parses
/// with `drop_iterative = false`. Default state must be the
/// pre-18e shape so legacy fixtures' canonical bytes are stable.
#[test]
fn parses_data_without_drop_iterative_defaults_to_false() {
let m = parse(
r#"
(module m
(data Tree
(vars a)
(ctor Leaf)
(ctor Node a (con Tree a) (con Tree a))))
"#,
)
.expect("parse should succeed");
match &m.defs[0] {
Def::Type(td) => {
assert!(!td.drop_iterative, "drop_iterative must default to false");
}
_ => panic!("expected Def::Type"),
}
}
/// Iter 18e: `(drop-iterative ...arg...)` is rejected — the
/// annotation is a flag with no payload.
#[test]
fn rejects_drop_iterative_with_arguments() {
let err = parse(
r#"
(module m
(data T
(ctor MkT)
(drop-iterative oops)))
"#,
)
.err()
.expect("parse should fail");
let msg = format!("{err}");
assert!(
msg.contains("drop-iterative takes no arguments"),
"diagnostic should explain drop-iterative's shape, got: {msg}"
);
}
/// Iter 16b.1: minimal `(let-rec ...)` round-trips through the
/// parser into a `Term::LetRec` whose `name`, `params`, `body` and
/// `in_term` line up with the source.
+8
View File
@@ -110,6 +110,14 @@ fn write_type_def(out: &mut String, td: &TypeDef, level: usize) {
out.push('\n');
write_ctor(out, c, level + 1);
}
// Iter 18e: `(drop-iterative)` annotation. Printed last (after
// every ctor) so the canonical form lands consistent with the
// DESIGN.md example. Omitted when `drop_iterative == false`.
if td.drop_iterative {
out.push('\n');
indent(out, level + 1);
out.push_str("(drop-iterative)");
}
out.push(')');
}
@@ -0,0 +1 @@
{"defs":[{"ctors":[{"fields":[],"name":"INil"},{"fields":[{"k":"con","name":"Int"},{"k":"con","name":"IntList"}],"name":"ICons"}],"drop-iterative":true,"kind":"type","name":"IntList"},{"body":{"cond":{"args":[{"name":"n","t":"var"},{"lit":{"kind":"int","value":0},"t":"lit"}],"fn":{"name":"==","t":"var"},"t":"app"},"else":{"args":[{"args":[{"name":"n","t":"var"},{"lit":{"kind":"int","value":1},"t":"lit"}],"fn":{"name":"-","t":"var"},"t":"app"},{"args":[{"name":"n","t":"var"},{"name":"acc","t":"var"}],"ctor":"ICons","t":"ctor","type":"IntList"}],"fn":{"name":"cons_n_acc","t":"var"},"t":"app","tail":true},"t":"if","then":{"name":"acc","t":"var"}},"doc":"Tail-recursive list builder. acc-prepended.","kind":"fn","name":"cons_n_acc","params":["n","acc"],"type":{"effects":[],"k":"fn","params":[{"k":"con","name":"Int"},{"k":"con","name":"IntList"}],"ret":{"k":"con","name":"IntList"}}},{"body":{"args":[{"name":"n","t":"var"},{"args":[],"ctor":"INil","t":"ctor","type":"IntList"}],"fn":{"name":"cons_n_acc","t":"var"},"t":"app"},"doc":"Build [1, 2, ..., n] :: IntList. Order is reverse of build but immaterial for head/sum.","kind":"fn","name":"cons_n","params":["n"],"type":{"effects":[],"k":"fn","params":[{"k":"con","name":"Int"}],"ret":{"k":"con","name":"IntList"}}},{"body":{"arms":[{"body":{"lit":{"kind":"int","value":0},"t":"lit"},"pat":{"ctor":"INil","fields":[],"p":"ctor"}},{"body":{"name":"h","t":"var"},"pat":{"ctor":"ICons","fields":[{"name":"h","p":"var"},{"name":"t","p":"var"}],"p":"ctor"}}],"scrutinee":{"name":"xs","t":"var"},"t":"match"},"doc":"Take ownership of an IntList; return its head if ICons, else 0. The (own ...) signature signals 18d.4 to emit an Own-param drop at fn return — which under the (drop-iterative) annotation is the iterative-worklist body, freeing the entire chain via the heap-allocated worklist instead of recursive cascade.","kind":"fn","name":"head_or_zero","params":["xs"],"type":{"effects":[],"k":"fn","param_modes":["own"],"params":[{"k":"con","name":"IntList"}],"ret":{"k":"con","name":"Int"}}},{"body":{"args":[{"args":[{"args":[{"lit":{"kind":"int","value":1000000},"t":"lit"}],"fn":{"name":"cons_n","t":"var"},"t":"app"}],"fn":{"name":"head_or_zero","t":"var"},"t":"app"}],"op":"io/print_int","t":"do"},"kind":"fn","name":"main","params":[],"type":{"effects":["IO"],"k":"fn","params":[],"ret":{"k":"con","name":"Unit"}}}],"imports":[],"name":"rc_drop_iterative_long_list","schema":"ailang/v0"}
+93
View File
@@ -0,0 +1,93 @@
; Iter 18e fixture: `(drop-iterative)` opt-in annotation drives the
; codegen to emit `drop_<m>_IntList` with a worklist body in place
; of the recursive cascade. Without the annotation, freeing a long
; list overflows Linux's default 8MB stack at ~1M cells (one frame
; per recursive `drop_<m>_IntList(tail)` call). With it, the chain
; pops through the heap-allocated worklist in O(N) time and O(1)
; stack.
;
; The fixture builds a 100,000-cell IntList tail-recursively, sums
; it into 4_999_950_000 (= N*(N-1)/2 for N=100_000), then `main`
; returns. At process exit the build's `xs` binder is consumed by
; sum_acc (via tail-call), so the iterative drop fires inside
; sum_acc's `Cons` arm — specifically, on every recursive step the
; tail t is bound, then t is consumed by the next iteration's
; tail-app sum_acc, leaving no live cells at termination.
;
; Wait — under the actual emission shape: sum_acc's xs param is
; (own (con IntList)); 18d.4 emits an Own-param drop at fn return.
; On the inductive Cons arm, the fn returns via tail-call into the
; next iteration (musttail), and the Own-param dec on xs at the
; outer fn would normally fire — but tail-call elision means the
; outer frame is gone before any post-tail-call code can run. The
; canonical case for the iterative-drop test is therefore the
; let-close-drop on `xs` in `main` (or the outer-let's drop after
; sum's tail returns). For 18e the load-bearing property is "long
; chains drop without overflowing the stack"; that fires whenever
; ANY drop call against the head IntList runs end-to-end.
;
; To force the issue: `main` builds the list, sums it, prints the
; sum, AND then `let xs = build n in let _ = sum xs in xs` does NOT
; exist as a pattern in AILang. We instead route the list through a
; `head` fn that takes (own IntList) and returns the head Int —
; which transfers ownership and forces the Own-param drop on xs at
; head's return. Because head's body (a match returning Int) cannot
; be tail-called, the drop is emitted in head's epilogue and runs
; in full before head returns to main.
;
; Expected stdout: `1` (the head of [1, 2, ..., 1_000_000]).
; Why 1M and not 100K: at 100K the recursive cascade fits in the 8MB
; default Linux stack (~64B/frame ≈ 6.4MB at 100K), so the test would
; "pass" even without the iterative drop — false-negative on the
; whole point of 18e. At 1M it overflows clean (SIGSEGV in the
; recursive variant; clean exit-0 in the iterative variant).
(module rc_drop_iterative_long_list
(data IntList
(ctor INil)
(ctor ICons (con Int) (con IntList))
(drop-iterative))
(fn cons_n_acc
(doc "Tail-recursive list builder. acc-prepended.")
(type
(fn-type
(params (con Int) (con IntList))
(ret (con IntList))))
(params n acc)
(body
(if (app == n 0)
acc
(tail-app cons_n_acc
(app - n 1)
(term-ctor IntList ICons n acc)))))
(fn cons_n
(doc "Build [1, 2, ..., n] :: IntList. Order is reverse of build but immaterial for head/sum.")
(type
(fn-type
(params (con Int))
(ret (con IntList))))
(params n)
(body
(app cons_n_acc n (term-ctor IntList INil))))
(fn head_or_zero
(doc "Take ownership of an IntList; return its head if ICons, else 0. The (own ...) signature signals 18d.4 to emit an Own-param drop at fn return — which under the (drop-iterative) annotation is the iterative-worklist body, freeing the entire chain via the heap-allocated worklist instead of recursive cascade.")
(type
(fn-type
(params (own (con IntList)))
(ret (con Int))))
(params xs)
(body
(match xs
(case (pat-ctor INil) 0)
(case (pat-ctor ICons h t) h))))
(fn main
(type (fn-type (params) (ret (con Unit)) (effects IO)))
(params)
(body
(do io/print_int
(app head_or_zero (app cons_n 1000000))))))
+113
View File
@@ -118,3 +118,116 @@ void ailang_rc_dec(void *payload) {
free(hdr);
}
}
/* ---------------------------------------------------------------------------
* Iter 18e: drop worklist.
*
* Backs the `(drop-iterative)` data attribute. When a type is annotated
* `(drop-iterative)`, codegen emits `drop_<m>_<T>` with an iterative-with-
* worklist body in place of the recursive cascade. The worklist is a
* heap-allocated stretchy buffer of `void*` pointers one entry per
* not-yet-processed cell. Each entry is mono-typed to T (the annotated
* ADT being dropped); fields of T whose type is `T` itself are pushed,
* fields whose type is a different ADT call that ADT's drop fn directly.
* (See `Emitter::emit_iterative_drop_fn_for_type` in the codegen for the
* IR shape and the same-type / different-type dispatch.)
*
* Strategy: heap-allocated buffer, doubled on overflow. We chose this
* over a stack-allocated small buffer (overcomplicates the IR seam the
* codegen body would need to track "which buffer is live") and over Lean
* 4's "thread the worklist through one of the cell's own pointer slots"
* technique (requires the codegen to know which slot of each ctor is
* "free to repurpose" non-trivial since AILang ctors are heterogeneous
* and slot 0 is always the tag). The runtime-helper approach keeps the
* IR-level body of `drop_<m>_<T>` small: three calls (new / push / pop /
* free) drive the loop.
*
* Precedent: Lean 4's `lean_dec_ref_cold` and Roc's iterative-free path
* both use a worklist to break tail recursion in their drop cascades.
* Lean threads the worklist through field slots (the "in-place" variant);
* we use a separate heap buffer because AILang's ctor layout makes slot
* repurposing fragile. The semantic invariant matches: every cell whose
* refcount reaches zero is dec'd exactly once, regardless of cascade
* depth, without consuming proportional C stack space.
*
* Single-threaded; non-atomic. Same scope as the rest of `runtime/rc.c`.
* --------------------------------------------------------------------------- */
typedef struct {
void **data; /* heap buffer of `cap` pointers; null once freed */
size_t len; /* number of live entries (always <= cap) */
size_t cap; /* current capacity in slots */
} ailang_drop_worklist_t;
/* Initial capacity. 16 slots * 8 bytes = 128 bytes — small enough that
* very-shallow drops don't waste memory, large enough that 16-deep
* cascades (very common) never realloc. Doubled on overflow. */
#define DROP_WORKLIST_INIT_CAP ((size_t)16)
void *ailang_drop_worklist_new(void) {
ailang_drop_worklist_t *wl = malloc(sizeof(ailang_drop_worklist_t));
if (wl == NULL) {
fprintf(stderr,
"ailang_drop_worklist_new: out of memory (header)\n");
abort();
}
wl->data = malloc(DROP_WORKLIST_INIT_CAP * sizeof(void *));
if (wl->data == NULL) {
fprintf(stderr,
"ailang_drop_worklist_new: out of memory (initial buffer)\n");
abort();
}
wl->len = 0;
wl->cap = DROP_WORKLIST_INIT_CAP;
return (void *)wl;
}
/* Push `payload` onto the worklist. Skips null payloads — pushed nulls
* would dispatch on `load i64, ptr null` at pop time and segfault, so
* we filter here. The check is symmetric with `ailang_rc_dec`'s null
* guard (a null payload is a no-op everywhere in the rc runtime). */
void ailang_drop_worklist_push(void *wl_opaque, void *payload) {
if (payload == NULL) {
return;
}
ailang_drop_worklist_t *wl = (ailang_drop_worklist_t *)wl_opaque;
if (wl->len == wl->cap) {
size_t new_cap = wl->cap * 2;
void **new_data = realloc(wl->data, new_cap * sizeof(void *));
if (new_data == NULL) {
fprintf(stderr,
"ailang_drop_worklist_push: out of memory (grow to %zu slots)\n",
new_cap);
abort();
}
wl->data = new_data;
wl->cap = new_cap;
}
wl->data[wl->len] = payload;
wl->len += 1;
}
/* Pop one payload from the worklist. Returns NULL when the worklist is
* empty. Since `push` filters nulls, a returned null is unambiguous and
* can be used by the IR body as the loop-exit sentinel. */
void *ailang_drop_worklist_pop(void *wl_opaque) {
ailang_drop_worklist_t *wl = (ailang_drop_worklist_t *)wl_opaque;
if (wl->len == 0) {
return NULL;
}
wl->len -= 1;
return wl->data[wl->len];
}
/* Free the worklist itself. Called once at the end of the iterative
* drop loop. Does NOT free any payloads still in the buffer the IR
* body must drain the buffer first via repeated `pop` calls before
* calling free. */
void ailang_drop_worklist_free(void *wl_opaque) {
if (wl_opaque == NULL) {
return;
}
ailang_drop_worklist_t *wl = (ailang_drop_worklist_t *)wl_opaque;
free(wl->data);
free(wl);
}