diff --git a/docs/specs/2026-05-20-boehm-retirement.md b/docs/specs/2026-05-20-boehm-retirement.md new file mode 100644 index 0000000..35e66fe --- /dev/null +++ b/docs/specs/2026-05-20-boehm-retirement.md @@ -0,0 +1,471 @@ +# Boehm full retirement — Design Spec + +**Date:** 2026-05-20 +**Status:** Draft — awaiting user spec review +**Authors:** Brummel (orchestrator) + Claude + +## Goal + +Retire the transitional Boehm-Demers-Weiser GC backend from the +AILang toolchain. After this milestone, RC is the canonical +production allocator, `--alloc=bump` is retained as the raw-alloc +bench-floor, and there is no GC code path. The design ledger, +the CLI surface, the codegen pipeline, the test suite, and the +bench harness all reflect a single-canonical-allocator project. + +Resolves Gitea issue #4 (`Boehm full retirement`). + +## Architecture + +The Boehm backend was introduced as a transitional dual-allocator +during the pre-22 RC commitment window. Its standing role per +`design/models/rc-uniqueness.md` was twofold: (a) **production +fallback** before RC was mature; (b) **parity oracle** — +differential RC-vs-GC stdout assertions to catch RC bugs cheaply. +Both roles have expired: + +- (a) RC is the CLI default and has been the canonical production + allocator since the memory-model commitment landed. +- (b) No recent debug iteration has used the GC oracle to triage + an RC bug. The differential e2e tests have not caught any RC + bug that fixed-stdout assertions would not have caught. + +The retirement removes Boehm wholesale across six layers in one +cohesive iteration: + +1. **CLI surface** — `--alloc=gc` value rejected with + `unknown --alloc value` parser error; help text adjusted. +2. **Codegen** — `AllocStrategy::Gc` variant deleted; the + `runtime_alloc_fn` arm returning `"GC_malloc"` removed; the + `lower_workspace` default switches from `AllocStrategy::Gc` to + `AllocStrategy::Rc`. +3. **Build / link** — the `clang` invocation in `crates/ail/src/main.rs` + loses its libgc-link branch entirely. +4. **Test suite** — pure-differential e2e tests deleted; RC-feature + tests with GC-as-backstop converted to fixed-stdout (the GC + build + the `assert_eq!(stdout_gc, stdout_rc, ...)` line drop, + the absolute `assert_eq!(stdout_rc.trim(), "")` pin stays); + the codegen-internal negative-complement test that proves "no + drop fns under non-RC" retargets from `AllocStrategy::Gc` to + `AllocStrategy::Bump`; the M2 staticlib alloc-guard test loses + its gc-arm but keeps its bump-arm. +5. **Design ledger** — `design/models/rc-uniqueness.md` excises + the "Dual allocator — RC canonical, Boehm parity oracle" + section and the Boehm-Choice-block; the per-fn-alloca section's + language generalises from "on top of Boehm" to "on top of the + allocator". `design/models/pipeline.md` drops the + `--alloc=gc → libgc` arm of the pipeline diagram and the + accompanying prose. The 1.3× RC-over-bump number is retained + in `rc-uniqueness.md` but reframed as a **bench-health + regression gate** (not a retirement gate); the closure-chain + ±15% wider band is preserved analogously. +6. **Honesty pin** — `crates/ailang-core/tests/docs_honesty_pin.rs` + removes its present-tense assertion that requires + `"--alloc=gc selects the transitional Boehm backend"` in + `pipeline.md`; in its place, the `design_md_has_no_wunschdenken` + set gains absence-pins against Boehm-zombie strings + (`"transitional Boehm"`, `"parity oracle"`, `"GC_malloc"`, + `"libgc"`) so the narrative cannot quietly re-emerge. + +Bump survives unchanged — `AllocStrategy::Bump`, `runtime/bump.c`, +the `--alloc=bump` CLI flag, and `bench/run.sh`'s RC-vs-bump +comparison all remain. Bump's standing role is bench-floor for +RC-overhead measurement, not a production target. + +## Concrete code shapes + +This is a no-authoring-surface milestone — no new `.ail` +construct is added or removed. The clause-3 discriminator +(wrong code must fail) lives at the CLI layer: + +### Must-fail CLI fixture (clause-3 discriminator) + +``` +$ ail build --alloc=gc examples/hello.ail -o /tmp/h +error: unknown --alloc value `gc` (expected `rc` or `bump`) +exit 2 +``` + +### Unchanged-default north-star slice + +``` +$ cat examples/hello.ail +(module hello + (fn main + (type (fn-type (params) (ret (con Unit)) (effects IO))) + (params) + (body (do (io print_str "hello"))))) + +$ ail build examples/hello.ail -o /tmp/h # default --alloc=rc +$ /tmp/h +hello +``` + +After retirement, every example under `examples/` continues to +build and run unchanged under the default `--alloc=rc`. The +`--alloc=bump` invocation still works for leak-tolerant bench +contexts. + +### Implementation shape — codegen `AllocStrategy` + +Before: + +```rust +// crates/ailang-codegen/src/lib.rs +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum AllocStrategy { Gc, Bump, Rc } + +impl AllocStrategy { + fn runtime_alloc_fn(&self) -> &'static str { + match self { + AllocStrategy::Gc => "GC_malloc", + AllocStrategy::Bump => "bump_malloc", + AllocStrategy::Rc => "ailang_rc_alloc", + } + } +} + +pub fn lower_workspace(ws: &Workspace) -> Result { + lower_workspace_inner(ws, AllocStrategy::Gc, Target::Executable) +} +``` + +After: + +```rust +// crates/ailang-codegen/src/lib.rs +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum AllocStrategy { Bump, Rc } + +impl AllocStrategy { + fn runtime_alloc_fn(&self) -> &'static str { + match self { + AllocStrategy::Bump => "bump_malloc", + AllocStrategy::Rc => "ailang_rc_alloc", + } + } +} + +pub fn lower_workspace(ws: &Workspace) -> Result { + lower_workspace_inner(ws, AllocStrategy::Rc, Target::Executable) +} +``` + +### Implementation shape — CLI parse arm + +Before (`crates/ail/src/main.rs:2143-2148`): + +```rust +match alloc.as_str() { + "gc" => Ok(ailang_codegen::AllocStrategy::Gc), + "bump" => Ok(ailang_codegen::AllocStrategy::Bump), + "rc" => Ok(ailang_codegen::AllocStrategy::Rc), + other => bail!("unknown --alloc value `{other}` (expected `gc`, `bump`, or `rc`)"), +} +``` + +After: + +```rust +match alloc.as_str() { + "bump" => Ok(ailang_codegen::AllocStrategy::Bump), + "rc" => Ok(ailang_codegen::AllocStrategy::Rc), + other => bail!("unknown --alloc value `{other}` (expected `rc` or `bump`)"), +} +``` + +### Implementation shape — link branch + +Before (`crates/ail/src/main.rs:2389ff`): + +```rust +match strategy { + AllocStrategy::Gc => { + // libgc link; pthread/dl transitively + cmd.arg("-lgc"); + } + AllocStrategy::Bump => { + // compile runtime/bump.c, link static + ... + } + AllocStrategy::Rc => { ... } +} +``` + +After: the `Gc` arm is gone entirely; `match` exhausts on `Bump` +and `Rc`. The accompanying inline doc-comment block ("Boehm +conservative GC (the transitional dual-allocator). The lowered +IR calls @GC_malloc; libgc supplies it. …") is removed; the +remaining `Bump` and `Rc` arms keep their doc-comments unchanged. + +### Implementation shape — staticlib alloc-guard + +Before (`crates/ail/tests/embed_staticlib_alloc_guard.rs`): + +```rust +#[test] +fn staticlib_gc_is_rejected() { ... } // expects unknown-alloc OR RC-only diag + +#[test] +fn staticlib_bump_is_rejected() { + let out = build_staticlib_with_alloc("bump", "/tmp/ail_m2_guard_bump"); + assert!(!out.status.success(), ...); + assert!(stderr.contains("staticlib (swarm) artefact is RC-only"), ...); +} +``` + +After: `staticlib_gc_is_rejected` deleted (the CLI parser now +rejects `gc` *before* the build is reached, so the staticlib-guard +no longer governs that case). `staticlib_bump_is_rejected` kept +unchanged. File-level doc-comment updated to drop the GC reference. + +### Implementation shape — codegen negative-complement test + +Before (`crates/ailang-codegen/src/lib.rs:3571-3576`): + +```rust +// Negative complement: no drop fns under `--alloc=gc`. +let ir_gc = lower_workspace_with_alloc(&ws, AllocStrategy::Gc).unwrap(); +assert!(!ir_gc.contains("@drop_rclist_IntList"), ...); +``` + +After: + +```rust +// Negative complement: no drop fns under `--alloc=bump` +// (only RC emits per-type drop fns; bump leaks). +let ir_bump = lower_workspace_with_alloc(&ws, AllocStrategy::Bump).unwrap(); +assert!(!ir_bump.contains("@drop_rclist_IntList"), ...); +``` + +The test's semantic ("non-RC allocators emit no drop fns") is +preserved; the witness allocator shifts from `Gc` to `Bump`. + +### Implementation shape — honesty-pin inversion + +Before (`crates/ailang-core/tests/docs_honesty_pin.rs:116-117`): + +```rust +assert!(pipeline.contains("`--alloc=gc` selects the transitional Boehm backend"), + "models/pipeline.md must describe Boehm present-tense, not as 'on the path to retirement'"); +``` + +After: this assertion is **deleted**, and the +`design_md_has_no_wunschdenken` set gains four new absence-pins +in its place: + +```rust +assert!(!d.contains("transitional Boehm"), + "design/: Boehm narrative is retired — must not re-emerge in the ledger"); +assert!(!d.contains("parity oracle"), + "design/: Boehm-as-parity-oracle is retired narrative — git log carries the history"); +assert!(!d.contains("GC_malloc"), + "design/: GC_malloc references are retired — RC + bump are the only allocators"); +assert!(!d.contains("libgc"), + "design/: libgc references are retired — no GC link dependency exists anymore"); +``` + +The corpus that `design_md_has_no_wunschdenken` scans +(`design_corpus()`) already includes `rc-uniqueness.md` and the +other ledger files, so no path-list change is needed. The +header doc-comment for the test file is updated to drop the +Boehm/Decision-9 phrase. + +### Implementation shape — design/models excise (`rc-uniqueness.md`) + +The `## Dual allocator — RC canonical, Boehm parity oracle` section +(lines 3-32 of the file) is deleted entirely. The `## Per-fn arena +via stack alloca` section is kept; its references to "Boehm" and +"@GC_malloc" generalise to "the allocator" / "@" +respectively. The `## Memory model — RC + Uniqueness …` section +keeps the RC narrative; the parenthetical +"`(see the dual-allocator section above)`" is dropped; the +parenthetical naming Boehm as a transitional allocator is dropped; +the 1.3× target sentence is rewritten to drop the Boehm-retirement +framing: + +Before: + +> AILang's canonical memory model is reference counting with +> static uniqueness inference and explicit LLM-author annotations… +> Boehm becomes a transitional allocator and is retired when the +> RC pipeline matches the bump-allocator floor within an +> acceptable margin (target 1.3× on `bench/run.sh`). + +After: + +> AILang's canonical memory model is reference counting with +> static uniqueness inference and explicit LLM-author annotations. +> The RC pipeline tracks the bump-allocator raw-alloc floor: a +> bench-health regression gate requires RC overhead ≤ 1.3× bump +> on the linear/tree corpus, with a wider ±15% band on the +> closure-chain corpus (representational cost of the closure-pair +> layout). See `bench/run.sh` for the active check. + +### Implementation shape — design/models excise (`pipeline.md`) + +The pipeline diagram drops the libgc arm: + +Before: + +``` +└─ clang -O2 *.ll -o binary + --alloc=rc → emits inc/dec (@ailang_rc_inc / _dec; canonical, default) + --alloc=gc → links libgc (@GC_malloc; parity oracle) +``` + +After: + +``` +└─ clang -O2 *.ll -o binary + --alloc=rc → emits inc/dec (@ailang_rc_inc / _dec; canonical, default) + --alloc=bump → links bump-floor (@bump_malloc; raw-alloc bench-floor) +``` + +The accompanying prose ("Two allocator backends share the same +MIR. … `--alloc=gc` selects the transitional Boehm backend …") is +rewritten to describe the rc/bump pair as the present state. + +## Components + +| Component | Touchpoints (precise location-class, not byte-exact) | +|---|---| +| Codegen | `crates/ailang-codegen/src/lib.rs` — `AllocStrategy` enum variant; `runtime_alloc_fn` match arm; default-strategy entry-point fns (`lower_workspace`, `lower_workspace_staticlib`); doc-comment block on the enum; the inline negative-complement codegen test currently using `AllocStrategy::Gc` | +| Codegen | `crates/ailang-codegen/src/escape.rs` — module-level doc comment generalises GC-specific language to allocator-agnostic | +| CLI parse | `crates/ail/src/main.rs:2143ff` — `--alloc` match arm; error-text wording | +| CLI build | `crates/ail/src/main.rs:2389ff` — the `AllocStrategy::Gc` arm in the clang invocation; the surrounding doc-comment block; the staticlib-guard diagnostic string (the "links the shared Boehm collector" phrasing) | +| Runtime docs | `runtime/bump.c` header doc — drop "mirrors `GC_malloc` from libgc" and "default Boehm-GC build" wording; bump's signature description stays | +| Runtime docs | `runtime/rc.c`, `runtime/str.c` — comment references to Boehm / `--alloc=gc` | +| E2E pure-differential | `crates/ail/tests/e2e.rs` — `gc_handles_recursive_list_construction` (line 194), `alloc_rc_produces_same_stdout_as_gc` (1400), `alloc_rc_matches_gc_on_std_list_demo` (1564) — **deleted** | +| E2E RC-feature w/ differential backstop | `crates/ail/tests/e2e.rs` — `reuse_as_drop_demo` (1475ff), `rc_box_drop` (1602ff), `rc_list_drop` (1628ff), `rc_list_drop_borrow` (1674ff), `pat_extract_partial_drop` (1762ff), `rc_own_param_drop` (1858ff), `rc_drop_iterative_long_list` (1981ff), `rc_pin_recurse_implicit` (2168ff), `rc_let_alias_implicit_param` (2396ff) — **the `stdout_gc` build call and the `assert_eq!(stdout_gc, stdout_rc, ...)` differential line dropped**; the absolute `assert_eq!(stdout_rc.trim(), "")` pin stays; doc-comments cleaned of `--alloc=gc` references | +| E2E staticlib guard | `crates/ail/tests/embed_staticlib_alloc_guard.rs` — `staticlib_gc_is_rejected` deleted; `staticlib_bump_is_rejected` unchanged; file-level doc-comment updated | +| Bench harness | `bench/run.sh` — `bench_latency_implicit_gc` arm removed; all `--alloc=gc` build calls removed; comment "decisive number for Decision 10's Boehm-retirement target (1.3x)" → "RC-overhead-vs-bump bench-health gate (1.3× ceiling)"; comment "Boehm-fair Implicit @ gc" arm dropped | +| Design ledger | `design/models/rc-uniqueness.md` — full Boehm sections excised (per "Concrete code shapes" above); 1.3× reframed as bench-health regression gate | +| Design ledger | `design/models/pipeline.md` — pipeline diagram + accompanying prose updated (per above) | +| Design ledger | `design/contracts/embedding-abi.md:43-44` — the present-tense sentence `ail build --emit=staticlib rejects --alloc=gc/--alloc=bump (the shared Boehm collector is not swarm-safe)` is rewritten to drop the gc clause (gc is now a CLI-parser-level unknown-value, not a staticlib-guard rejection) and to reframe the swarm-safety justification: only `--alloc=bump` is rejected by the staticlib guard, on the grounds that bump is a leak-only bench instrument, not the historical Boehm-collector justification | +| Honesty pin | `crates/ailang-core/tests/docs_honesty_pin.rs` — present-tense Boehm pin deleted; four absence-pins added (per above); test-file doc-comment header updated to drop the Boehm/Decision-9 phrase | +| Bencher agent | `skills/audit/agents/ailang-bencher.md` — the worked example "under heap pressure where Boehm GC has stop-the-world pauses" is rewritten to use an allocator-agnostic hypothesis (RC overhead under a workload that doesn't exercise the rc path is a natural replacement) | + +The fixture `examples/gc_stress.ail` (target of the deleted +`gc_handles_recursive_list_construction` test) is **deleted**. +Verified at spec time: `grep -rn 'gc_stress' --include='*.rs' +--include='*.ail' --include='*.ail.json' --include='*.sh'` returns +exactly two hits — the e2e test call and the fixture itself — +both of which are removed by this iteration. + +## Data flow + +Removal-only milestone; no new data flow. The existing +post-retirement data flow is: + +``` +.ail.json → check (typecheck + uniqueness) → codegen + ├─ AllocStrategy::Rc → @ailang_rc_inc / _dec + rc.c linked + └─ AllocStrategy::Bump → bump.c linked (bench only) + → clang -O2 → binary +``` + +No surviving code path branches on Boehm/GC. + +## Error handling + +The only user-observable surface change is the CLI parser error: + +| Input | Old behaviour | New behaviour | +|---|---|---| +| `ail build --alloc=gc foo.ail` | succeeds; links libgc; produces a Boehm-backed binary | exit 2; stderr: `error: unknown --alloc value \`gc\` (expected \`rc\` or \`bump\`)` | +| `ail build foo.ail` (default) | succeeds; defaults to gc historically — but actual default in current code is rc per the memory-model commitment | unchanged: defaults to rc, succeeds | +| `ail build --alloc=rc foo.ail` | unchanged | unchanged | +| `ail build --alloc=bump foo.ail` | unchanged | unchanged | +| `ail build --emit=staticlib --alloc=bump foo.ail` | rejected with "staticlib artefact is RC-only" | unchanged | + +The host's libgc-installation status no longer affects any +`ail build`/`ail run` outcome. + +## Testing strategy + +The retirement RED-state is observable through several existing +green tests that *currently* depend on Boehm. Removal breaks +them; the iteration's GREEN-state is the simultaneous removal of +the Boehm path AND the breaking tests. + +**RED→GREEN transitions in the iteration:** + +1. `cargo test -p ailang-codegen` — the in-source codegen test + that exercises `AllocStrategy::Gc` fails to compile once the + enum variant is removed; the test is retargeted to `Bump` + inside the same iteration. +2. `cargo test -p ail --test e2e` — the pure-differential tests + are deleted; the RC-feature tests lose their `stdout_gc` build + call (which calls `build_and_run_with_alloc(example, "gc")`, + which would now panic). Each test is rewritten to drop the gc + call. +3. `cargo test -p ail --test embed_staticlib_alloc_guard` — + `staticlib_gc_is_rejected` is deleted; `staticlib_bump_is_rejected` + keeps passing. +4. `cargo test -p ailang-core --test docs_honesty_pin` — the + present-tense Boehm-anchor assertion fails the moment + `pipeline.md` drops the string; the assertion is deleted + inside the same iteration, and the four new absence-pins are + added to replace it. + +**New protective assertions:** + +- A `cargo test -p ail --test boehm_retirement_pin` (new file) + pins the CLI behaviour: `ail build --alloc=gc ` must + fail with exit code ≠ 0 and stderr containing + `unknown --alloc value`. This is the milestone-protecting E2E + for the retirement — guards against a future iter + reintroducing the gc arm without anyone noticing. + +**Standing tests that must remain green:** + +- All `workspace`-scope tests (`cargo test --workspace --quiet`). +- `design_index_pin`, `design_schema_drift`, `roundtrip-invariant`, + `effect_doc_honesty_pin`, `docs_honesty_pin` (with the new + absence-pins). +- Bench harness regression check `bench/check.py` — passes + unchanged (no compile-side compatibility break). +- `bench/compile_check.py`, `bench/cross_lang.py` — pass + unchanged. +- `bench/run.sh` runs to completion with rc-vs-bump arms only, + no gc arms. + +**Field test:** because this is a no-authoring-surface milestone, +no `.ail` fieldtest is needed. The CLI must-fail fixture is the +clause-3 discriminator. + +## Acceptance criteria + +The milestone closes CLEAN when all of the following hold: + +1. `cargo test --workspace --quiet` is fully green; the ~12 + removed/converted tests are accounted for in the iter commit + body; no new failure. +2. `grep -rn "Boehm\|libgc\|GC_malloc\|--alloc=gc\|AllocStrategy::Gc" \ + crates/ design/ runtime/ bench/run.sh skills/` returns matches + **only** in `crates/ailang-core/tests/docs_honesty_pin.rs` (the + four new absence-pins literally name the Boehm-zombie strings + to scan against). Anywhere else is a residual reference and + the iteration is not done. +3. `bench/check.py`, `bench/compile_check.py`, `bench/cross_lang.py` + all exit 0 against the existing baseline. +4. The CLI must-fail fixture exits ≠ 0 with the expected stderr + wording. +5. `design/models/rc-uniqueness.md` and `design/models/pipeline.md` + describe RC + bump as the present state; the honesty rule + (`design/contracts/honesty-rule.md`) is upheld — no past-tense + "Boehm was kept as oracle" narrative remains; Boehm's history + is recorded in the iter's git-log commit body. +6. Issue #4 is closed by the iter commit's `closes #4` trailer. + +Out of scope (deferred to separate work): + +- The "Closure-pair slab / pool" optimisation (Gitea #3) that + would tighten the closure-chain ±15% bench-health band. Not + blocked by retirement; can ship anytime. +- Any further `AllocStrategy::Bump` rework (still a single- + variant bench instrument). +- Any bench-harness *recalibration* — that just closed + (bench-harness-recalibration audit, 2026-05-20); this milestone + preserves the calibration.