From 7e5c95f6c600e4bd2d727f57f832bec9bf2c8148 Mon Sep 17 00:00:00 2001 From: Brummel Date: Sat, 9 May 2026 00:14:56 +0200 Subject: [PATCH] =?UTF-8?q?runtime:=20half-retirement=20of=20Boehm=20?= =?UTF-8?q?=E2=80=94=20flip=20CLI=20default=20to=20--alloc=3Drc?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The dual-allocator policy from 2026-05-08 made Boehm the CLI default "for general workloads", with rc selected explicitly for real-time- sensitive code. That asymmetry contradicted Decision 10: rc + uniqueness inference is the canonical memory model AILang's typechecker enforces, and the CLI was teaching the opposite mental model to users and prompt- fed LLMs. This iter flips the asymmetry without removing Boehm: - main.rs: default_value gc -> rc for build/run; help text rewritten so rc is the canonical path and gc is the parity oracle. - DESIGN.md Decision 9 retitled "RC canonical, Boehm parity oracle" and rewritten end-to-end. Migration plan step 6 updated to a two-step retirement (default-flip done, full removal gated). - DESIGN.md pipeline diagram: rc-first, gc-as-oracle. - JOURNAL: 2026-05-09 entry recording the call, what shipped, the bug the flip surfaced, the gating condition for full Boehm removal. Bug surfaced by the flip and fixed in the same iter: codegen/drop.rs build_pair_drop_fn emitted `getelementptr inbounds {{ ptr, ptr }}, ...` via push_str (not format!), so the doubled braces leaked verbatim into the IR. LLVM parsed it as a struct-of- struct, the GEP referenced a non-existent field, and clang failed. Invisible under the old gc default (no per-type drop fns) and invisible to the 8 explicit _with_alloc("rc") tests (none of them escaped a closure-with-captures). Caught immediately by two corpus tests once rc became the baseline: closure_captures_let_n, local_rec_as_value_capture_demo. Fix: single braces. Tests stay as the regression guard. The episode is the canonical justification for keeping the GC arm as a parity oracle for now — a months-old codegen bug found by flipping the baseline column. 288 passed / 0 failed / 3 ignored, unchanged. --- crates/ail/src/main.rs | 25 +++---- crates/ailang-codegen/src/drop.rs | 8 ++- docs/DESIGN.md | 70 ++++++++++++-------- docs/JOURNAL.md | 105 ++++++++++++++++++++++++++++++ 4 files changed, 168 insertions(+), 40 deletions(-) diff --git a/crates/ail/src/main.rs b/crates/ail/src/main.rs index 342de8e..12a515c 100644 --- a/crates/ail/src/main.rs +++ b/crates/ail/src/main.rs @@ -125,16 +125,17 @@ enum Cmd { /// Optimization (e.g. `-O2`); default `-O0` for debuggability. #[arg(long, default_value = "-O0")] opt: String, - /// Bench iter: heap allocator. `gc` (default) is Boehm - /// conservative GC; `bump` swaps every `@GC_malloc` for a - /// no-free 256 MB bump-allocator stub from `runtime/bump.c`. - /// The bump path is bench-only — it leaks every allocation. - /// `rc` (Iter 18b plumbing) routes through `runtime/rc.c`'s - /// `@ailang_rc_alloc` (libc-malloc backing + 8-byte refcount - /// header); inc/dec instrumentation arrives in 18c, so 18b - /// programs leak under this mode but must produce correct - /// stdout. - #[arg(long, default_value = "gc", value_parser = ["gc", "bump", "rc"])] + /// Heap allocator. `rc` (default, canonical) routes through + /// `runtime/rc.c`'s `@ailang_rc_alloc` (libc-malloc backing + + /// 8-byte refcount header) with `ailang_rc_inc`/`_dec` + /// instrumentation emitted by codegen; this is the runtime + /// AILang's memory model (RC + uniqueness, Decision 10) is + /// designed for. `gc` retains the Boehm conservative GC path + /// (`@GC_malloc`, libgc); it is kept as a differential parity + /// oracle for codegen diagnosis (Decision 9). `bump` is a + /// bench-only no-free stub from `runtime/bump.c` — it leaks + /// every allocation by design. + #[arg(long, default_value = "rc", value_parser = ["gc", "bump", "rc"])] alloc: String, }, /// Build into a tempdir and execute. Exits with the binary's exit code. @@ -146,8 +147,8 @@ enum Cmd { /// Optimization (e.g. `-O2`); default `-O0` for debuggability. #[arg(long, default_value = "-O0")] opt: String, - /// Bench iter: heap allocator. See `build --alloc` for details. - #[arg(long, default_value = "gc", value_parser = ["gc", "bump", "rc"])] + /// Heap allocator. See `build --alloc` for details. Default `rc`. + #[arg(long, default_value = "rc", value_parser = ["gc", "bump", "rc"])] alloc: String, /// Args passed through to the compiled program. #[arg(last = true)] diff --git a/crates/ailang-codegen/src/drop.rs b/crates/ailang-codegen/src/drop.rs index 0716bfc..9f0963c 100644 --- a/crates/ailang-codegen/src/drop.rs +++ b/crates/ailang-codegen/src/drop.rs @@ -871,8 +871,14 @@ impl<'a> Emitter<'a> { out.push_str(" br i1 %is_null, label %ret, label %live\n"); out.push_str("live:\n"); if has_env { + // The pair layout is `{ ptr thunk, ptr env }`; env is the + // second field. Note the single braces — this is a plain + // `push_str`, not a `format!` call, so brace-escaping does + // not apply. (Iter 18c.4 originally shipped doubled braces + // here; surfaced when the rc backend became the corpus + // default and a closure-with-captures escaped.) out.push_str( - " %ea = getelementptr inbounds {{ ptr, ptr }}, ptr %p, i64 0, i32 1\n", + " %ea = getelementptr inbounds { ptr, ptr }, ptr %p, i64 0, i32 1\n", ); out.push_str(" %env = load ptr, ptr %ea, align 8\n"); out.push_str(&format!(" call void @{env_drop}(ptr %env)\n")); diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 6a9d255..d18db87 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -637,26 +637,38 @@ constructor-blocked recursions in `map`, `sort`, `insert` remain unmarked — they cannot benefit from `musttail` without a source-level rewrite. -## Decision 9: dual allocator — Boehm conservative GC + RC +## Decision 9: dual allocator — RC canonical, Boehm parity oracle -**Status: dual-allocator policy as of 2026-05-08.** Originally -framed (2026-05-07) as "transitional Boehm until the RC -pipeline (Iters 18a–18g) lands and the default flips". The RC -pipeline landed and was validated end-to-end (live=0 on the -canonical bench fixture; tail latency 23× better than Boehm; -RSS lower than Boehm). The orchestrator's call on retirement -was to **keep both allocators live**: Boehm stays as the -default for general workloads, RC is selected via `--alloc=rc` -for real-time-sensitive workloads where bounded per-operation -latency matters more than peak throughput. +**Status: half-retirement as of 2026-05-09.** Originally framed +(2026-05-07) as "transitional Boehm until the RC pipeline +(Iters 18a–18g) lands and the default flips"; then re-framed +(2026-05-08) as a symmetric dual-allocator policy with Boehm as +the CLI default. The 2026-05-09 revision flips the asymmetry to +match Decision 10: -The retirement question reopens if maintaining the Boehm path -costs significant attention (libgc upgrade pain, runtime -divergence, build complexity); until then the dual-allocator -asymmetry (Boehm = default, RC = ready alternative) is the -canonical state. Decision 10 (RC + uniqueness) holds as the -specification of the RC alternative; the rest of this section -documents the Boehm half of the dual model. +- **RC is canonical.** `--alloc=rc` is the CLI default for + `ail build` and `ail run`. The runtime AILang's memory model + (RC + uniqueness inference) is designed for. New examples, + benches, and corpus tests run under RC unless they explicitly + pin GC. +- **Boehm stays as a parity oracle.** `--alloc=gc` remains + reachable. Its load-bearing job is differential diagnosis: when + RC produces a segfault, refcount underflow, or wrong stdout, the + GC build of the same module is the cheap "memory bug or logic + bug?" probe. The end-to-end suite includes per-example + parity tests that run both backends and assert byte-identical + stdout — those tests are what make the oracle real. +- **`--alloc=bump`** is unchanged: a leak-only bench instrument, + not a production target. + +Full Boehm retirement (drop libgc, remove the gc backend) reopens +when the parity oracle stops paying its keep — concretely, when a +few iter families ship without the gc arm catching anything that +the rc arm did not already catch. Until then, the cost of +keeping libgc as a build dependency is accepted in exchange for +diagnostic leverage. Decision 10 (RC + uniqueness) holds as the +specification of the canonical runtime; the rest of this section +documents the Boehm half, retained as the oracle. The `--alloc=bump` mode introduced for the bench is a measurement tool, not a production target. @@ -1325,9 +1337,14 @@ monomorphised copies resolve to concrete drop fns). the precondition holds. 5. **Iter 18e:** `(drop-iterative)` data attr. Worklist-based free for annotated types. -6. **Iter 18f:** RC validation bench. If RC within 1.3× of bump - on `bench/run.sh`, retire Boehm: flip default to RC, drop - `-lgc`, mark Decision 9 historical. +6. **Iter 18f:** RC validation bench. RC validated within target + on `bench/run.sh` (live=0; tail latency 23× better than Boehm; + RSS lower). Retirement of Boehm executes in two steps: first a + default-flip (2026-05-09 — `--alloc=rc` becomes the CLI default, + GC retained as parity oracle; see Decision 9 above); second a + full removal once the oracle stops paying its keep (gating + condition: a few iter families with no GC-only diagnostic + wins). 7. **Iter 19a / 19a.1 / 19b:** advisory `over-strict-mode` lint + precise sub-binder analysis with heap-type filter + `FnDef.suppress` suppression mechanism with mandatory-reason. @@ -1582,14 +1599,13 @@ canonical-JSON hash. ├─ lower to MIR (SSA-like, named SSA values) ├─ emit LLVM IR (.ll) └─ clang -O2 *.ll -o binary - --alloc=gc → links libgc (@GC_malloc; transitional) - --alloc=rc → emits inc/dec (@ailang_rc_inc / _dec; canonical) + --alloc=rc → emits inc/dec (@ailang_rc_inc / _dec; canonical, default) + --alloc=gc → links libgc (@GC_malloc; parity oracle) ``` -Two allocator backends share the same MIR. `--alloc=gc` links libgc -and is the transitional fallback (no inc/dec instrumentation, no mode -checks at codegen time). `--alloc=rc` is the canonical backend -committed to in Decision 10: the typechecker enforces +Two allocator backends share the same MIR. `--alloc=rc` is the +canonical backend committed to in Decision 10 and the CLI default +since 2026-05-09: the typechecker enforces `(own)` / `(borrow)` modes, codegen emits `ailang_rc_inc` / `_dec` calls at the points dictated by linearity, and `Term::Clone` / `Term::ReuseAs` materialise into actual rc-bumps and in-place diff --git a/docs/JOURNAL.md b/docs/JOURNAL.md index beb5562..864bc42 100644 --- a/docs/JOURNAL.md +++ b/docs/JOURNAL.md @@ -9764,3 +9764,108 @@ The 20-family is now formally closed with this tidy. Codebase is in good form: no architecturally load-bearing drift, all tests green, two minor items recorded as acceptable. The next iter can be a feature pick from the queue. + +## 2026-05-09 — Boehm half-retirement: CLI default flips to rc + +### Why + +The "Boehm retirement" item in the post-tidy queue surfaced via +a direct user question: *kann Boehm weg, was gewinnen wir +dadurch?* The orchestrator's reading: full removal is premature +— Boehm earns its keep right now as a differential parity +oracle for codegen diagnosis — but the asymmetry (Boehm = CLI +default) actively teaches the wrong mental model. RC is the +canonical runtime per Decision 10, and the CLI was telling +users and prompt-fed LLMs the opposite. So this iter ratifies a +half-retirement: flip the default, keep the oracle, document +the gating condition for full removal. + +### What shipped + +- `crates/ail/src/main.rs`: `default_value = "gc"` → `"rc"` for + both `build` and `run` subcommands. Help text rewritten so + `rc` is described first as canonical (Decision 10 pointer), + `gc` as parity oracle, `bump` as bench-only. The doc reference + to "Iter 18b plumbing — programs leak under this mode" is + removed; it was 18b-era and no longer reflects the matured RC + pipeline. +- `crates/ail/tests/e2e.rs`: `build_and_run` is unchanged — + it picks up the CLI default, which now means the entire + corpus runs under RC as the canonical baseline. The 8 + `build_and_run_with_alloc(...)` differential tests already + pin both backends explicitly and continue to anchor the + parity-oracle relationship. +- `docs/DESIGN.md`: + - Decision 9 retitled "RC canonical, Boehm parity oracle" and + rewritten end-to-end. The 2026-05-08 dual-allocator framing + is preserved as historical context; the live framing is + asymmetric in RC's favour. + - Migration plan step 6 (Iter 18f) updated: retirement is now + explicitly two-step (default-flip done; full removal gated + on the oracle ceasing to catch anything). + - Pipeline diagram (L1585): rc-first, gc-second; gc labelled + "parity oracle" instead of "transitional". + - Wording around "transitional fallback" replaced with the + new "canonical default since 2026-05-09" phrasing. + +### Bug found by the flip + +Flipping the corpus baseline to RC immediately surfaced a silent +codegen bug from Iter 18c.4: `crates/ailang-codegen/src/drop.rs` +build_pair_drop_fn emitted `getelementptr inbounds {{ ptr, ptr }}, ...` +via `push_str` (not `format!`), so the doubled braces went +verbatim into the IR. LLVM parsed that as a struct-of-struct +`{ { ptr, ptr } }`, the GEP referenced a non-existent field +index, and clang failed. + +The bug was invisible under the old `gc` default — the gc +backend doesn't emit per-type drop fns. It was also invisible +in the 8 explicit `_with_alloc("rc")` differential tests because +none of them happened to construct a closure whose env captures +escaped (the only path that triggers the pair-drop emitter). +Two corpus tests caught it the moment the default flipped: +`closure_captures_let_n` and `local_rec_as_value_capture_demo`. + +Fix: single braces. The two corpus tests stay as the regression +guard. This is the canonical example of why the GC oracle is +worth keeping: the bug had been latent for months, and the +parity-baseline flip was the cheap probe that found it. + +### What this iter does NOT do + +- Not full Boehm retirement. `--alloc=gc` still works, libgc is + still linked when selected, the oracle column in e2e is + preserved. +- Not a corpus expansion. Same e2e corpus, now running RC as + baseline instead of GC. +- Not a bench refresh. Bench fixtures and numbers from 18f are + untouched. + +### Gating condition for full retirement + +Boehm comes out completely once the parity oracle stops paying +its keep. Concretely: a few iter families (≥3, say) that ship +without `--alloc=gc` catching any bug `--alloc=rc` did not +already catch. At that point the differential probe is +diagnostic dead weight, libgc-as-build-dep stops being a +worthwhile cost, and the gc backend can come out in a follow-up +iter (delete the gc arm of the e2e tests, drop the `-lgc` link +flag, remove `--alloc=gc` from the CLI parser, mark Decision 9 +historical). + +### Test state + +288 passed / 0 failed / 3 ignored, same as pre-iter — the +codegen fix is offset by the now-canonical-RC baseline catching +no other corpus regressions. + +### JOURNAL queue (updated) + +- **`FnDef::synthetic(...)` factor-out** — unchanged; awaits + next schema-additive `FnDef` field. +- **Boehm full retirement** — re-queued with the new gating + condition (≥3 families with no oracle wins). +- **Deferred richer integration paths** (from 20f): tool-use + schemas, MCP server, LSP. +- **Family 21+** — typeclasses, polymorphic ADTs at runtime, + pattern-binding generalisation.