JOURNAL: Iter 18e — (drop-iterative) annotation + worklist allocator
Records the schema additions (TypeDef.drop_iterative, serde-skip when false), why heap-stretchy-buffer worklist was chosen over slot-repurposing (not every box has a free ptr-slot to thread through), the mono-typed worklist trade-off (simpler loop body, small cross-type cascade penalty), the four new ABI symbols in runtime/rc.c, and the validation against Decision 10's brief. Validation note: hand-run confirmed the 1M-cell fixture with the annotation exits 0; same fixture without the annotation SIGSEGVs (exit 139). The annotation is load-bearing, not cosmetic. Closes with the dispatch line for 18f (RC validation bench + Boehm retirement decision) and reaffirms the post-18f tidy-iter will be ailang-architect-driven drift cleanup over the whole 18-arc surface.
This commit is contained in:
+193
@@ -7304,3 +7304,196 @@ natural place to close the dynamic-tag fallback.
|
||||
After 18e, Iter 18f closes the 18-arc with the RC validation
|
||||
bench. The CLAUDE.md tidy-iter rule then fires: `ailang-architect`
|
||||
drift cleanup before 19 begins.
|
||||
|
||||
## Iter 18e — `(drop-iterative)` annotation + worklist allocator
|
||||
|
||||
Closes the 18-arc's stack-recursion limit. Long ADT chains
|
||||
(millions of cells) overflow the recursive `drop_<m>_<T>`
|
||||
cascade from 18c.4. The opt-in `(drop-iterative)` annotation
|
||||
on a `Def::Type` switches that type's synthesised drop fn from
|
||||
recursive to iterative-with-explicit-worklist. Validated on a
|
||||
1M-cell `IntList` fixture: with the annotation, exits 0; with
|
||||
the annotation stripped, SIGSEGV (exit code 139). The worklist
|
||||
is load-bearing, not cosmetic.
|
||||
|
||||
### Schema additions
|
||||
|
||||
`TypeDef.drop_iterative: bool` — default `false`, serde-skip
|
||||
when false. Existing fixtures' canonical JSON hashes stay
|
||||
stable; only fixtures that opt in carry the new key. Form-A:
|
||||
a `(drop-iterative)` clause inside the `(data T ...)` decl,
|
||||
matching DESIGN.md Decision 10's example.
|
||||
|
||||
### Worklist strategy — heap stretchy buffer
|
||||
|
||||
Three strategies were named in the implementer brief:
|
||||
|
||||
1. Heap-allocated stretchy buffer (always `malloc`, double on
|
||||
overflow).
|
||||
2. Stack-allocated small buffer with heap-spill.
|
||||
3. Slot-repurposing — thread the worklist through one of the
|
||||
box's own pointer-typed slots (Lean 4 technique).
|
||||
|
||||
The implementer chose **(1)**, with the rationale documented
|
||||
in `runtime/rc.c`:
|
||||
|
||||
- (3) requires every box to have at least one pointer-typed
|
||||
slot that's NOT the slot we're cascading through. Not
|
||||
generally true: `Cons (Int) (List)` has slot 0 = i64 (head)
|
||||
and slot 1 = ptr (tail). The tail IS the cascade slot;
|
||||
there's no other ptr slot to repurpose. Generalising to ADTs
|
||||
with multiple ptr slots adds codegen complexity (per-ctor
|
||||
decision: which slot is the worklist link?).
|
||||
- (2) is more efficient for short lists but adds dual-buffer
|
||||
spill logic. Marginal benefit since the worklist is only
|
||||
emitted under the opt-in annotation and the annotation's
|
||||
primary use case IS long lists.
|
||||
- (1) is simplest and matches the failure mode the iter
|
||||
targets (deep recursion on long lists).
|
||||
|
||||
Four new ABI symbols in `runtime/rc.c`, declared on demand
|
||||
under `--alloc=rc`:
|
||||
|
||||
```
|
||||
void* ailang_drop_worklist_new(size_t initial_capacity);
|
||||
void ailang_drop_worklist_push(void* wl, void* ptr);
|
||||
void* ailang_drop_worklist_pop(void* wl); // returns NULL on empty
|
||||
void ailang_drop_worklist_free(void* wl);
|
||||
```
|
||||
|
||||
`push` null-filters: pushing a null pointer is a no-op (matches
|
||||
the existing 18c.4 invariant that drops null-guard at entry).
|
||||
The buffer doubles on overflow.
|
||||
|
||||
### Codegen — `emit_iterative_drop_fn_for_type`
|
||||
|
||||
For a `drop_iterative: true` type, `drop_<m>_<T>(ptr %p)`
|
||||
emits a worklist body:
|
||||
|
||||
```
|
||||
; alloc worklist; push p
|
||||
loop_head:
|
||||
; pop next; if null, free worklist + ret
|
||||
; load tag; switch on tag
|
||||
; per-ctor arm:
|
||||
; for each pointer-typed field:
|
||||
; load it; push (or call other drop fn directly if
|
||||
; it's a different type)
|
||||
; ailang_rc_dec the outer cell
|
||||
; br loop_head
|
||||
```
|
||||
|
||||
The IR contains a `br label %loop_head` jumping back to the
|
||||
top — the load-bearing structural difference from the
|
||||
recursive shape. The IR-shape test asserts both this and the
|
||||
**absence** of any direct recursive call to `drop_<m>_<T>`
|
||||
inside the body of the same fn.
|
||||
|
||||
### Mono-typed worklist
|
||||
|
||||
The worklist is **mono-typed**: only fields of the SAME
|
||||
annotated type push onto it. Fields of a DIFFERENT type
|
||||
(whether iterative or recursive) call their own
|
||||
`drop_<m>_<T'>` directly, opening one level of recursive
|
||||
cascade for that boundary. Three trade-offs:
|
||||
|
||||
- **Pro: simple worklist.** No need to track per-entry type
|
||||
tag; every popped pointer is the same type, dispatched
|
||||
through the same tag-switch.
|
||||
- **Con: a `(drop-iterative)` `Tree` containing
|
||||
`(drop-iterative)` `List` fields would still recurse one
|
||||
level when crossing the type boundary.** For nested
|
||||
iterative types both deep, this could be a problem if the
|
||||
cross-type chain is itself long. None of 18e's shipping
|
||||
fixtures hit this — the canonical case is a single
|
||||
recursive ADT (`List` containing `List`).
|
||||
- **Pro: tag-switch in the loop body is per-type, not
|
||||
per-entry.** Compiles to a smaller loop body.
|
||||
|
||||
The trade-off is documented in `emit_iterative_drop_fn_for_type`'s
|
||||
doc. A future iter could generalise to a heterogeneous worklist
|
||||
(per-entry type tag, generic dispatch loop), but it isn't
|
||||
needed for the deep-self-recursion case the annotation targets.
|
||||
|
||||
### Test additions
|
||||
|
||||
- `examples/rc_drop_iterative_long_list.{ailx,ail.json}` —
|
||||
builds a 1M-cell `IntList` (annotated `(drop-iterative)`)
|
||||
via a counted-recursion fn, sums it, prints the sum.
|
||||
- `alloc_rc_drop_iterative_handles_million_cell_list` E2E —
|
||||
asserts the 1M-cell fixture exits 0 under both `gc` and
|
||||
`rc`. **Hand-verified separately**: the same fixture with
|
||||
the `(drop-iterative)` annotation stripped SIGSEGVs at
|
||||
~1M cells under `--alloc=rc` (exit code 139). The
|
||||
annotation is load-bearing.
|
||||
- `iter18e_drop_iterative_emits_worklist_body_no_self_recursion`
|
||||
— IR-shape test: a fixture with `(drop-iterative)` should
|
||||
emit a `drop_<m>_<T>` body containing `br label %loop_head`
|
||||
AND no direct recursive `call void @drop_<m>_<T>(...)` in
|
||||
the same body.
|
||||
- `iter18e_no_annotation_keeps_recursive_drop_body` — control
|
||||
test: the same fixture WITHOUT the annotation still emits
|
||||
the recursive 18c.4 shape (no `loop_head`, recursive call
|
||||
present).
|
||||
- 3 surface parse-tests:
|
||||
`parses_drop_iterative_annotation_on_data_decl`,
|
||||
`parses_data_without_drop_iterative_defaults_to_false`,
|
||||
`rejects_drop_iterative_with_arguments`.
|
||||
|
||||
### Test deltas
|
||||
|
||||
- E2E: 58 → 61 (+3).
|
||||
- Surface unit: 18 → 21 (+3).
|
||||
- All other buckets unchanged.
|
||||
- `cargo test --workspace` green.
|
||||
- Existing fixtures' canonical JSON hashes are stable
|
||||
(`drop_iterative` defaults to `false` and serde-skips when
|
||||
false, so no fixture's serialisation changed).
|
||||
|
||||
### Known debt (deliberate, deferred)
|
||||
|
||||
- **Mono-typed worklist.** Cross-type drop-iterative fields
|
||||
call the other type's drop fn directly rather than sharing
|
||||
a heterogeneous worklist. Sound for the deep-self-recursion
|
||||
case (the canonical `List` of `List` of `Int` is one type,
|
||||
not three).
|
||||
- **Closure / `Type::Var` / `Type::Forall` fields** fall back
|
||||
to shallow `ailang_rc_dec` via `field_drop_call` — same
|
||||
fall-back path 18c.4 carved out, unchanged here.
|
||||
- **Dynamic-tag partial-drop fallback** (the
|
||||
`emit_inlined_partial_drop` shallow-dec path from 18d.4
|
||||
when `moved_slots` is non-empty AND runtime ctor is
|
||||
unknown). Structurally orthogonal to 18e — would exist with
|
||||
no `(drop-iterative)` annotations either. Deferred to a
|
||||
future iter.
|
||||
|
||||
### Validation against the 18-arc's brief
|
||||
|
||||
Decision 10's "(4) `(drop-iterative)` annotation on data
|
||||
declarations" said: "When the refcount of a `Tree` value
|
||||
reaches zero, the synthesised dec-on-zero traversal is
|
||||
iterative (worklist + heap-allocated stack) instead of
|
||||
recursive. Avoids stack overflow on deep structures. The LLM
|
||||
adds the annotation where appropriate; the compiler refuses
|
||||
to emit recursive dec-cascade on annotated types."
|
||||
|
||||
18e ships exactly that: opt-in annotation, codegen swaps the
|
||||
emission strategy, runtime helpers in `runtime/rc.c`,
|
||||
heap-allocated worklist. The "compiler refuses to emit
|
||||
recursive dec-cascade on annotated types" half is enforced by
|
||||
construction — `emit_drop_fns` dispatches on
|
||||
`td.drop_iterative` and emits exactly one of the two bodies.
|
||||
|
||||
### Next
|
||||
|
||||
Iter 18f closes the 18-arc with the RC validation bench.
|
||||
Target: 1.3× of bump on `bench/run.sh`. If met, retire Boehm:
|
||||
flip default to `--alloc=rc`, drop `-lgc`, mark Decision 9
|
||||
historical.
|
||||
|
||||
After 18f closes, the new tidy-iter rule from CLAUDE.md fires:
|
||||
the next iter is `ailang-architect`-driven drift cleanup over
|
||||
the entire 18-arc surface (all of 18a, 18b, 18c.x, 18d.x, 18e
|
||||
shipped under different mental models as I learned the
|
||||
problem; the architect can flag where DESIGN.md and the
|
||||
shipped code have drifted, and decide which side moves).
|
||||
|
||||
Reference in New Issue
Block a user