diff --git a/docs/JOURNAL.md b/docs/JOURNAL.md index b661deb..f82fc3b 100644 --- a/docs/JOURNAL.md +++ b/docs/JOURNAL.md @@ -7304,3 +7304,196 @@ natural place to close the dynamic-tag fallback. After 18e, Iter 18f closes the 18-arc with the RC validation bench. The CLAUDE.md tidy-iter rule then fires: `ailang-architect` drift cleanup before 19 begins. + +## Iter 18e — `(drop-iterative)` annotation + worklist allocator + +Closes the 18-arc's stack-recursion limit. Long ADT chains +(millions of cells) overflow the recursive `drop__` +cascade from 18c.4. The opt-in `(drop-iterative)` annotation +on a `Def::Type` switches that type's synthesised drop fn from +recursive to iterative-with-explicit-worklist. Validated on a +1M-cell `IntList` fixture: with the annotation, exits 0; with +the annotation stripped, SIGSEGV (exit code 139). The worklist +is load-bearing, not cosmetic. + +### Schema additions + +`TypeDef.drop_iterative: bool` — default `false`, serde-skip +when false. Existing fixtures' canonical JSON hashes stay +stable; only fixtures that opt in carry the new key. Form-A: +a `(drop-iterative)` clause inside the `(data T ...)` decl, +matching DESIGN.md Decision 10's example. + +### Worklist strategy — heap stretchy buffer + +Three strategies were named in the implementer brief: + +1. Heap-allocated stretchy buffer (always `malloc`, double on + overflow). +2. Stack-allocated small buffer with heap-spill. +3. Slot-repurposing — thread the worklist through one of the + box's own pointer-typed slots (Lean 4 technique). + +The implementer chose **(1)**, with the rationale documented +in `runtime/rc.c`: + +- (3) requires every box to have at least one pointer-typed + slot that's NOT the slot we're cascading through. Not + generally true: `Cons (Int) (List)` has slot 0 = i64 (head) + and slot 1 = ptr (tail). The tail IS the cascade slot; + there's no other ptr slot to repurpose. Generalising to ADTs + with multiple ptr slots adds codegen complexity (per-ctor + decision: which slot is the worklist link?). +- (2) is more efficient for short lists but adds dual-buffer + spill logic. Marginal benefit since the worklist is only + emitted under the opt-in annotation and the annotation's + primary use case IS long lists. +- (1) is simplest and matches the failure mode the iter + targets (deep recursion on long lists). + +Four new ABI symbols in `runtime/rc.c`, declared on demand +under `--alloc=rc`: + +``` +void* ailang_drop_worklist_new(size_t initial_capacity); +void ailang_drop_worklist_push(void* wl, void* ptr); +void* ailang_drop_worklist_pop(void* wl); // returns NULL on empty +void ailang_drop_worklist_free(void* wl); +``` + +`push` null-filters: pushing a null pointer is a no-op (matches +the existing 18c.4 invariant that drops null-guard at entry). +The buffer doubles on overflow. + +### Codegen — `emit_iterative_drop_fn_for_type` + +For a `drop_iterative: true` type, `drop__(ptr %p)` +emits a worklist body: + +``` + ; alloc worklist; push p +loop_head: + ; pop next; if null, free worklist + ret + ; load tag; switch on tag + ; per-ctor arm: + ; for each pointer-typed field: + ; load it; push (or call other drop fn directly if + ; it's a different type) + ; ailang_rc_dec the outer cell + ; br loop_head +``` + +The IR contains a `br label %loop_head` jumping back to the +top — the load-bearing structural difference from the +recursive shape. The IR-shape test asserts both this and the +**absence** of any direct recursive call to `drop__` +inside the body of the same fn. + +### Mono-typed worklist + +The worklist is **mono-typed**: only fields of the SAME +annotated type push onto it. Fields of a DIFFERENT type +(whether iterative or recursive) call their own +`drop__` directly, opening one level of recursive +cascade for that boundary. Three trade-offs: + +- **Pro: simple worklist.** No need to track per-entry type + tag; every popped pointer is the same type, dispatched + through the same tag-switch. +- **Con: a `(drop-iterative)` `Tree` containing + `(drop-iterative)` `List` fields would still recurse one + level when crossing the type boundary.** For nested + iterative types both deep, this could be a problem if the + cross-type chain is itself long. None of 18e's shipping + fixtures hit this — the canonical case is a single + recursive ADT (`List` containing `List`). +- **Pro: tag-switch in the loop body is per-type, not + per-entry.** Compiles to a smaller loop body. + +The trade-off is documented in `emit_iterative_drop_fn_for_type`'s +doc. A future iter could generalise to a heterogeneous worklist +(per-entry type tag, generic dispatch loop), but it isn't +needed for the deep-self-recursion case the annotation targets. + +### Test additions + +- `examples/rc_drop_iterative_long_list.{ailx,ail.json}` — + builds a 1M-cell `IntList` (annotated `(drop-iterative)`) + via a counted-recursion fn, sums it, prints the sum. +- `alloc_rc_drop_iterative_handles_million_cell_list` E2E — + asserts the 1M-cell fixture exits 0 under both `gc` and + `rc`. **Hand-verified separately**: the same fixture with + the `(drop-iterative)` annotation stripped SIGSEGVs at + ~1M cells under `--alloc=rc` (exit code 139). The + annotation is load-bearing. +- `iter18e_drop_iterative_emits_worklist_body_no_self_recursion` + — IR-shape test: a fixture with `(drop-iterative)` should + emit a `drop__` body containing `br label %loop_head` + AND no direct recursive `call void @drop__(...)` in + the same body. +- `iter18e_no_annotation_keeps_recursive_drop_body` — control + test: the same fixture WITHOUT the annotation still emits + the recursive 18c.4 shape (no `loop_head`, recursive call + present). +- 3 surface parse-tests: + `parses_drop_iterative_annotation_on_data_decl`, + `parses_data_without_drop_iterative_defaults_to_false`, + `rejects_drop_iterative_with_arguments`. + +### Test deltas + +- E2E: 58 → 61 (+3). +- Surface unit: 18 → 21 (+3). +- All other buckets unchanged. +- `cargo test --workspace` green. +- Existing fixtures' canonical JSON hashes are stable + (`drop_iterative` defaults to `false` and serde-skips when + false, so no fixture's serialisation changed). + +### Known debt (deliberate, deferred) + +- **Mono-typed worklist.** Cross-type drop-iterative fields + call the other type's drop fn directly rather than sharing + a heterogeneous worklist. Sound for the deep-self-recursion + case (the canonical `List` of `List` of `Int` is one type, + not three). +- **Closure / `Type::Var` / `Type::Forall` fields** fall back + to shallow `ailang_rc_dec` via `field_drop_call` — same + fall-back path 18c.4 carved out, unchanged here. +- **Dynamic-tag partial-drop fallback** (the + `emit_inlined_partial_drop` shallow-dec path from 18d.4 + when `moved_slots` is non-empty AND runtime ctor is + unknown). Structurally orthogonal to 18e — would exist with + no `(drop-iterative)` annotations either. Deferred to a + future iter. + +### Validation against the 18-arc's brief + +Decision 10's "(4) `(drop-iterative)` annotation on data +declarations" said: "When the refcount of a `Tree` value +reaches zero, the synthesised dec-on-zero traversal is +iterative (worklist + heap-allocated stack) instead of +recursive. Avoids stack overflow on deep structures. The LLM +adds the annotation where appropriate; the compiler refuses +to emit recursive dec-cascade on annotated types." + +18e ships exactly that: opt-in annotation, codegen swaps the +emission strategy, runtime helpers in `runtime/rc.c`, +heap-allocated worklist. The "compiler refuses to emit +recursive dec-cascade on annotated types" half is enforced by +construction — `emit_drop_fns` dispatches on +`td.drop_iterative` and emits exactly one of the two bodies. + +### Next + +Iter 18f closes the 18-arc with the RC validation bench. +Target: 1.3× of bump on `bench/run.sh`. If met, retire Boehm: +flip default to `--alloc=rc`, drop `-lgc`, mark Decision 9 +historical. + +After 18f closes, the new tidy-iter rule from CLAUDE.md fires: +the next iter is `ailang-architect`-driven drift cleanup over +the entire 18-arc surface (all of 18a, 18b, 18c.x, 18d.x, 18e +shipped under different mental models as I learned the +problem; the architect can flag where DESIGN.md and the +shipped code have drifted, and decide which side moves).