spec: boehm-retirement — drop the transitional Boehm GC path

Resolves Gitea #4. Approach A (atomic single iteration). Six layers
in one cohesive cut: CLI `--alloc=gc` arm removed; codegen
`AllocStrategy::Gc` variant deleted with the default flipped to
`Rc`; libgc link branch removed; ~3 pure-differential e2e tests
plus the `gc_stress.ail` fixture deleted; ~9 RC-feature tests that
used GC-stdout as backstop lose only the differential assertion
(absolute fixed-stdout pin retained); M2 staticlib alloc-guard
drops its gc-arm (bump-arm preserved); design/models/rc-uniqueness
excises the Boehm-parity-oracle narrative; pipeline.md drops the
libgc pipeline-diagram arm; docs_honesty_pin flips from
Boehm-present-tense anchor to four absence-pins against
Boehm-zombie strings.

Three design forks were resolved with the user via brainstorm Q&A:

  (1) bump survives as bench-floor — `AllocStrategy::Bump`, the
      `--alloc=bump` CLI flag, and `runtime/bump.c` all stay; the
      enum keeps two variants (Rc, Bump); the codegen
      negative-complement test retargets from `AllocStrategy::Gc`
      to `AllocStrategy::Bump`. Bump's standing role is the
      raw-alloc bench-floor for RC-overhead measurement, not a
      production target.
  (2) the ~12 RC-vs-GC differential e2e tests are NOT all deleted
      wholesale — pure-differential ones (test name literally
      `*_matches_gc_*` or `*_same_stdout_as_gc`) are deleted; the
      RC-feature tests with GC as backstop keep their absolute
      stdout pin (`assert_eq!(stdout_rc.trim(), "<n>")`) and only
      lose the differential assertion. This nuance was added at
      spec time on top of the user's "delete the differential
      pattern" answer, because the differential was incidental to
      tests like `rc_box_drop` / `rc_list_drop_borrow` that pin
      drop-fn correctness under RC and would lose unrelated
      coverage if deleted entirely.
  (3) the 1.3× RC-over-bump number is retained in
      design/models/rc-uniqueness.md but reframed as a
      bench-health regression gate (not a Boehm-retirement gate);
      the closure-chain ±15% wider band is preserved analogously.

Grounding-check (ailang-grounding-check) PASS — 7 load-bearing
assumptions ratified, all spec-named paths and line numbers
verified (±2 lines), all proposed-for-removal strings present at
the spec-named locations. Assumption #7 (codegen default
currently Gc) is structurally self-evident: removing the variant
forces the default onto a surviving one by construction.

Out of scope, tracked separately:
  - Gitea #3 "Closure-pair slab / pool" — would tighten the
    closure-chain ±15% band; not blocked by retirement.
  - Any further `AllocStrategy::Bump` rework — still a
    single-variant bench instrument.

refs #4
This commit is contained in:
2026-05-20 20:02:32 +02:00
parent 7a42989b34
commit 50dc478ca5
+471
View File
@@ -0,0 +1,471 @@
# Boehm full retirement — Design Spec
**Date:** 2026-05-20
**Status:** Draft — awaiting user spec review
**Authors:** Brummel (orchestrator) + Claude
## Goal
Retire the transitional Boehm-Demers-Weiser GC backend from the
AILang toolchain. After this milestone, RC is the canonical
production allocator, `--alloc=bump` is retained as the raw-alloc
bench-floor, and there is no GC code path. The design ledger,
the CLI surface, the codegen pipeline, the test suite, and the
bench harness all reflect a single-canonical-allocator project.
Resolves Gitea issue #4 (`Boehm full retirement`).
## Architecture
The Boehm backend was introduced as a transitional dual-allocator
during the pre-22 RC commitment window. Its standing role per
`design/models/rc-uniqueness.md` was twofold: (a) **production
fallback** before RC was mature; (b) **parity oracle**
differential RC-vs-GC stdout assertions to catch RC bugs cheaply.
Both roles have expired:
- (a) RC is the CLI default and has been the canonical production
allocator since the memory-model commitment landed.
- (b) No recent debug iteration has used the GC oracle to triage
an RC bug. The differential e2e tests have not caught any RC
bug that fixed-stdout assertions would not have caught.
The retirement removes Boehm wholesale across six layers in one
cohesive iteration:
1. **CLI surface**`--alloc=gc` value rejected with
`unknown --alloc value` parser error; help text adjusted.
2. **Codegen**`AllocStrategy::Gc` variant deleted; the
`runtime_alloc_fn` arm returning `"GC_malloc"` removed; the
`lower_workspace` default switches from `AllocStrategy::Gc` to
`AllocStrategy::Rc`.
3. **Build / link** — the `clang` invocation in `crates/ail/src/main.rs`
loses its libgc-link branch entirely.
4. **Test suite** — pure-differential e2e tests deleted; RC-feature
tests with GC-as-backstop converted to fixed-stdout (the GC
build + the `assert_eq!(stdout_gc, stdout_rc, ...)` line drop,
the absolute `assert_eq!(stdout_rc.trim(), "<n>")` pin stays);
the codegen-internal negative-complement test that proves "no
drop fns under non-RC" retargets from `AllocStrategy::Gc` to
`AllocStrategy::Bump`; the M2 staticlib alloc-guard test loses
its gc-arm but keeps its bump-arm.
5. **Design ledger**`design/models/rc-uniqueness.md` excises
the "Dual allocator — RC canonical, Boehm parity oracle"
section and the Boehm-Choice-block; the per-fn-alloca section's
language generalises from "on top of Boehm" to "on top of the
allocator". `design/models/pipeline.md` drops the
`--alloc=gc → libgc` arm of the pipeline diagram and the
accompanying prose. The 1.3× RC-over-bump number is retained
in `rc-uniqueness.md` but reframed as a **bench-health
regression gate** (not a retirement gate); the closure-chain
±15% wider band is preserved analogously.
6. **Honesty pin**`crates/ailang-core/tests/docs_honesty_pin.rs`
removes its present-tense assertion that requires
`"--alloc=gc selects the transitional Boehm backend"` in
`pipeline.md`; in its place, the `design_md_has_no_wunschdenken`
set gains absence-pins against Boehm-zombie strings
(`"transitional Boehm"`, `"parity oracle"`, `"GC_malloc"`,
`"libgc"`) so the narrative cannot quietly re-emerge.
Bump survives unchanged — `AllocStrategy::Bump`, `runtime/bump.c`,
the `--alloc=bump` CLI flag, and `bench/run.sh`'s RC-vs-bump
comparison all remain. Bump's standing role is bench-floor for
RC-overhead measurement, not a production target.
## Concrete code shapes
This is a no-authoring-surface milestone — no new `.ail`
construct is added or removed. The clause-3 discriminator
(wrong code must fail) lives at the CLI layer:
### Must-fail CLI fixture (clause-3 discriminator)
```
$ ail build --alloc=gc examples/hello.ail -o /tmp/h
error: unknown --alloc value `gc` (expected `rc` or `bump`)
exit 2
```
### Unchanged-default north-star slice
```
$ cat examples/hello.ail
(module hello
(fn main
(type (fn-type (params) (ret (con Unit)) (effects IO)))
(params)
(body (do (io print_str "hello")))))
$ ail build examples/hello.ail -o /tmp/h # default --alloc=rc
$ /tmp/h
hello
```
After retirement, every example under `examples/` continues to
build and run unchanged under the default `--alloc=rc`. The
`--alloc=bump` invocation still works for leak-tolerant bench
contexts.
### Implementation shape — codegen `AllocStrategy`
Before:
```rust
// crates/ailang-codegen/src/lib.rs
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AllocStrategy { Gc, Bump, Rc }
impl AllocStrategy {
fn runtime_alloc_fn(&self) -> &'static str {
match self {
AllocStrategy::Gc => "GC_malloc",
AllocStrategy::Bump => "bump_malloc",
AllocStrategy::Rc => "ailang_rc_alloc",
}
}
}
pub fn lower_workspace(ws: &Workspace) -> Result<String, Error> {
lower_workspace_inner(ws, AllocStrategy::Gc, Target::Executable)
}
```
After:
```rust
// crates/ailang-codegen/src/lib.rs
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AllocStrategy { Bump, Rc }
impl AllocStrategy {
fn runtime_alloc_fn(&self) -> &'static str {
match self {
AllocStrategy::Bump => "bump_malloc",
AllocStrategy::Rc => "ailang_rc_alloc",
}
}
}
pub fn lower_workspace(ws: &Workspace) -> Result<String, Error> {
lower_workspace_inner(ws, AllocStrategy::Rc, Target::Executable)
}
```
### Implementation shape — CLI parse arm
Before (`crates/ail/src/main.rs:2143-2148`):
```rust
match alloc.as_str() {
"gc" => Ok(ailang_codegen::AllocStrategy::Gc),
"bump" => Ok(ailang_codegen::AllocStrategy::Bump),
"rc" => Ok(ailang_codegen::AllocStrategy::Rc),
other => bail!("unknown --alloc value `{other}` (expected `gc`, `bump`, or `rc`)"),
}
```
After:
```rust
match alloc.as_str() {
"bump" => Ok(ailang_codegen::AllocStrategy::Bump),
"rc" => Ok(ailang_codegen::AllocStrategy::Rc),
other => bail!("unknown --alloc value `{other}` (expected `rc` or `bump`)"),
}
```
### Implementation shape — link branch
Before (`crates/ail/src/main.rs:2389ff`):
```rust
match strategy {
AllocStrategy::Gc => {
// libgc link; pthread/dl transitively
cmd.arg("-lgc");
}
AllocStrategy::Bump => {
// compile runtime/bump.c, link static
...
}
AllocStrategy::Rc => { ... }
}
```
After: the `Gc` arm is gone entirely; `match` exhausts on `Bump`
and `Rc`. The accompanying inline doc-comment block ("Boehm
conservative GC (the transitional dual-allocator). The lowered
IR calls @GC_malloc; libgc supplies it. …") is removed; the
remaining `Bump` and `Rc` arms keep their doc-comments unchanged.
### Implementation shape — staticlib alloc-guard
Before (`crates/ail/tests/embed_staticlib_alloc_guard.rs`):
```rust
#[test]
fn staticlib_gc_is_rejected() { ... } // expects unknown-alloc OR RC-only diag
#[test]
fn staticlib_bump_is_rejected() {
let out = build_staticlib_with_alloc("bump", "/tmp/ail_m2_guard_bump");
assert!(!out.status.success(), ...);
assert!(stderr.contains("staticlib (swarm) artefact is RC-only"), ...);
}
```
After: `staticlib_gc_is_rejected` deleted (the CLI parser now
rejects `gc` *before* the build is reached, so the staticlib-guard
no longer governs that case). `staticlib_bump_is_rejected` kept
unchanged. File-level doc-comment updated to drop the GC reference.
### Implementation shape — codegen negative-complement test
Before (`crates/ailang-codegen/src/lib.rs:3571-3576`):
```rust
// Negative complement: no drop fns under `--alloc=gc`.
let ir_gc = lower_workspace_with_alloc(&ws, AllocStrategy::Gc).unwrap();
assert!(!ir_gc.contains("@drop_rclist_IntList"), ...);
```
After:
```rust
// Negative complement: no drop fns under `--alloc=bump`
// (only RC emits per-type drop fns; bump leaks).
let ir_bump = lower_workspace_with_alloc(&ws, AllocStrategy::Bump).unwrap();
assert!(!ir_bump.contains("@drop_rclist_IntList"), ...);
```
The test's semantic ("non-RC allocators emit no drop fns") is
preserved; the witness allocator shifts from `Gc` to `Bump`.
### Implementation shape — honesty-pin inversion
Before (`crates/ailang-core/tests/docs_honesty_pin.rs:116-117`):
```rust
assert!(pipeline.contains("`--alloc=gc` selects the transitional Boehm backend"),
"models/pipeline.md must describe Boehm present-tense, not as 'on the path to retirement'");
```
After: this assertion is **deleted**, and the
`design_md_has_no_wunschdenken` set gains four new absence-pins
in its place:
```rust
assert!(!d.contains("transitional Boehm"),
"design/: Boehm narrative is retired — must not re-emerge in the ledger");
assert!(!d.contains("parity oracle"),
"design/: Boehm-as-parity-oracle is retired narrative — git log carries the history");
assert!(!d.contains("GC_malloc"),
"design/: GC_malloc references are retired — RC + bump are the only allocators");
assert!(!d.contains("libgc"),
"design/: libgc references are retired — no GC link dependency exists anymore");
```
The corpus that `design_md_has_no_wunschdenken` scans
(`design_corpus()`) already includes `rc-uniqueness.md` and the
other ledger files, so no path-list change is needed. The
header doc-comment for the test file is updated to drop the
Boehm/Decision-9 phrase.
### Implementation shape — design/models excise (`rc-uniqueness.md`)
The `## Dual allocator — RC canonical, Boehm parity oracle` section
(lines 3-32 of the file) is deleted entirely. The `## Per-fn arena
via stack alloca` section is kept; its references to "Boehm" and
"@GC_malloc" generalise to "the allocator" / "@<runtime_alloc>"
respectively. The `## Memory model — RC + Uniqueness …` section
keeps the RC narrative; the parenthetical
"`(see the dual-allocator section above)`" is dropped; the
parenthetical naming Boehm as a transitional allocator is dropped;
the 1.3× target sentence is rewritten to drop the Boehm-retirement
framing:
Before:
> AILang's canonical memory model is reference counting with
> static uniqueness inference and explicit LLM-author annotations…
> Boehm becomes a transitional allocator and is retired when the
> RC pipeline matches the bump-allocator floor within an
> acceptable margin (target 1.3× on `bench/run.sh`).
After:
> AILang's canonical memory model is reference counting with
> static uniqueness inference and explicit LLM-author annotations.
> The RC pipeline tracks the bump-allocator raw-alloc floor: a
> bench-health regression gate requires RC overhead ≤ 1.3× bump
> on the linear/tree corpus, with a wider ±15% band on the
> closure-chain corpus (representational cost of the closure-pair
> layout). See `bench/run.sh` for the active check.
### Implementation shape — design/models excise (`pipeline.md`)
The pipeline diagram drops the libgc arm:
Before:
```
└─ clang -O2 *.ll -o binary
--alloc=rc → emits inc/dec (@ailang_rc_inc / _dec; canonical, default)
--alloc=gc → links libgc (@GC_malloc; parity oracle)
```
After:
```
└─ clang -O2 *.ll -o binary
--alloc=rc → emits inc/dec (@ailang_rc_inc / _dec; canonical, default)
--alloc=bump → links bump-floor (@bump_malloc; raw-alloc bench-floor)
```
The accompanying prose ("Two allocator backends share the same
MIR. … `--alloc=gc` selects the transitional Boehm backend …") is
rewritten to describe the rc/bump pair as the present state.
## Components
| Component | Touchpoints (precise location-class, not byte-exact) |
|---|---|
| Codegen | `crates/ailang-codegen/src/lib.rs``AllocStrategy` enum variant; `runtime_alloc_fn` match arm; default-strategy entry-point fns (`lower_workspace`, `lower_workspace_staticlib`); doc-comment block on the enum; the inline negative-complement codegen test currently using `AllocStrategy::Gc` |
| Codegen | `crates/ailang-codegen/src/escape.rs` — module-level doc comment generalises GC-specific language to allocator-agnostic |
| CLI parse | `crates/ail/src/main.rs:2143ff``--alloc` match arm; error-text wording |
| CLI build | `crates/ail/src/main.rs:2389ff` — the `AllocStrategy::Gc` arm in the clang invocation; the surrounding doc-comment block; the staticlib-guard diagnostic string (the "links the shared Boehm collector" phrasing) |
| Runtime docs | `runtime/bump.c` header doc — drop "mirrors `GC_malloc` from libgc" and "default Boehm-GC build" wording; bump's signature description stays |
| Runtime docs | `runtime/rc.c`, `runtime/str.c` — comment references to Boehm / `--alloc=gc` |
| E2E pure-differential | `crates/ail/tests/e2e.rs``gc_handles_recursive_list_construction` (line 194), `alloc_rc_produces_same_stdout_as_gc` (1400), `alloc_rc_matches_gc_on_std_list_demo` (1564) — **deleted** |
| E2E RC-feature w/ differential backstop | `crates/ail/tests/e2e.rs``reuse_as_drop_demo` (1475ff), `rc_box_drop` (1602ff), `rc_list_drop` (1628ff), `rc_list_drop_borrow` (1674ff), `pat_extract_partial_drop` (1762ff), `rc_own_param_drop` (1858ff), `rc_drop_iterative_long_list` (1981ff), `rc_pin_recurse_implicit` (2168ff), `rc_let_alias_implicit_param` (2396ff) — **the `stdout_gc` build call and the `assert_eq!(stdout_gc, stdout_rc, ...)` differential line dropped**; the absolute `assert_eq!(stdout_rc.trim(), "<n>")` pin stays; doc-comments cleaned of `--alloc=gc` references |
| E2E staticlib guard | `crates/ail/tests/embed_staticlib_alloc_guard.rs``staticlib_gc_is_rejected` deleted; `staticlib_bump_is_rejected` unchanged; file-level doc-comment updated |
| Bench harness | `bench/run.sh``bench_latency_implicit_gc` arm removed; all `--alloc=gc` build calls removed; comment "decisive number for Decision 10's Boehm-retirement target (1.3x)" → "RC-overhead-vs-bump bench-health gate (1.3× ceiling)"; comment "Boehm-fair Implicit @ gc" arm dropped |
| Design ledger | `design/models/rc-uniqueness.md` — full Boehm sections excised (per "Concrete code shapes" above); 1.3× reframed as bench-health regression gate |
| Design ledger | `design/models/pipeline.md` — pipeline diagram + accompanying prose updated (per above) |
| Design ledger | `design/contracts/embedding-abi.md:43-44` — the present-tense sentence `ail build --emit=staticlib rejects --alloc=gc/--alloc=bump (the shared Boehm collector is not swarm-safe)` is rewritten to drop the gc clause (gc is now a CLI-parser-level unknown-value, not a staticlib-guard rejection) and to reframe the swarm-safety justification: only `--alloc=bump` is rejected by the staticlib guard, on the grounds that bump is a leak-only bench instrument, not the historical Boehm-collector justification |
| Honesty pin | `crates/ailang-core/tests/docs_honesty_pin.rs` — present-tense Boehm pin deleted; four absence-pins added (per above); test-file doc-comment header updated to drop the Boehm/Decision-9 phrase |
| Bencher agent | `skills/audit/agents/ailang-bencher.md` — the worked example "under heap pressure where Boehm GC has stop-the-world pauses" is rewritten to use an allocator-agnostic hypothesis (RC overhead under a workload that doesn't exercise the rc path is a natural replacement) |
The fixture `examples/gc_stress.ail` (target of the deleted
`gc_handles_recursive_list_construction` test) is **deleted**.
Verified at spec time: `grep -rn 'gc_stress' --include='*.rs'
--include='*.ail' --include='*.ail.json' --include='*.sh'` returns
exactly two hits — the e2e test call and the fixture itself —
both of which are removed by this iteration.
## Data flow
Removal-only milestone; no new data flow. The existing
post-retirement data flow is:
```
.ail.json → check (typecheck + uniqueness) → codegen
├─ AllocStrategy::Rc → @ailang_rc_inc / _dec + rc.c linked
└─ AllocStrategy::Bump → bump.c linked (bench only)
→ clang -O2 → binary
```
No surviving code path branches on Boehm/GC.
## Error handling
The only user-observable surface change is the CLI parser error:
| Input | Old behaviour | New behaviour |
|---|---|---|
| `ail build --alloc=gc foo.ail` | succeeds; links libgc; produces a Boehm-backed binary | exit 2; stderr: `error: unknown --alloc value \`gc\` (expected \`rc\` or \`bump\`)` |
| `ail build foo.ail` (default) | succeeds; defaults to gc historically — but actual default in current code is rc per the memory-model commitment | unchanged: defaults to rc, succeeds |
| `ail build --alloc=rc foo.ail` | unchanged | unchanged |
| `ail build --alloc=bump foo.ail` | unchanged | unchanged |
| `ail build --emit=staticlib --alloc=bump foo.ail` | rejected with "staticlib artefact is RC-only" | unchanged |
The host's libgc-installation status no longer affects any
`ail build`/`ail run` outcome.
## Testing strategy
The retirement RED-state is observable through several existing
green tests that *currently* depend on Boehm. Removal breaks
them; the iteration's GREEN-state is the simultaneous removal of
the Boehm path AND the breaking tests.
**RED→GREEN transitions in the iteration:**
1. `cargo test -p ailang-codegen` — the in-source codegen test
that exercises `AllocStrategy::Gc` fails to compile once the
enum variant is removed; the test is retargeted to `Bump`
inside the same iteration.
2. `cargo test -p ail --test e2e` — the pure-differential tests
are deleted; the RC-feature tests lose their `stdout_gc` build
call (which calls `build_and_run_with_alloc(example, "gc")`,
which would now panic). Each test is rewritten to drop the gc
call.
3. `cargo test -p ail --test embed_staticlib_alloc_guard` —
`staticlib_gc_is_rejected` is deleted; `staticlib_bump_is_rejected`
keeps passing.
4. `cargo test -p ailang-core --test docs_honesty_pin` — the
present-tense Boehm-anchor assertion fails the moment
`pipeline.md` drops the string; the assertion is deleted
inside the same iteration, and the four new absence-pins are
added to replace it.
**New protective assertions:**
- A `cargo test -p ail --test boehm_retirement_pin` (new file)
pins the CLI behaviour: `ail build --alloc=gc <example>` must
fail with exit code ≠ 0 and stderr containing
`unknown --alloc value`. This is the milestone-protecting E2E
for the retirement — guards against a future iter
reintroducing the gc arm without anyone noticing.
**Standing tests that must remain green:**
- All `workspace`-scope tests (`cargo test --workspace --quiet`).
- `design_index_pin`, `design_schema_drift`, `roundtrip-invariant`,
`effect_doc_honesty_pin`, `docs_honesty_pin` (with the new
absence-pins).
- Bench harness regression check `bench/check.py` — passes
unchanged (no compile-side compatibility break).
- `bench/compile_check.py`, `bench/cross_lang.py` — pass
unchanged.
- `bench/run.sh` runs to completion with rc-vs-bump arms only,
no gc arms.
**Field test:** because this is a no-authoring-surface milestone,
no `.ail` fieldtest is needed. The CLI must-fail fixture is the
clause-3 discriminator.
## Acceptance criteria
The milestone closes CLEAN when all of the following hold:
1. `cargo test --workspace --quiet` is fully green; the ~12
removed/converted tests are accounted for in the iter commit
body; no new failure.
2. `grep -rn "Boehm\|libgc\|GC_malloc\|--alloc=gc\|AllocStrategy::Gc" \
crates/ design/ runtime/ bench/run.sh skills/` returns matches
**only** in `crates/ailang-core/tests/docs_honesty_pin.rs` (the
four new absence-pins literally name the Boehm-zombie strings
to scan against). Anywhere else is a residual reference and
the iteration is not done.
3. `bench/check.py`, `bench/compile_check.py`, `bench/cross_lang.py`
all exit 0 against the existing baseline.
4. The CLI must-fail fixture exits ≠ 0 with the expected stderr
wording.
5. `design/models/rc-uniqueness.md` and `design/models/pipeline.md`
describe RC + bump as the present state; the honesty rule
(`design/contracts/honesty-rule.md`) is upheld — no past-tense
"Boehm was kept as oracle" narrative remains; Boehm's history
is recorded in the iter's git-log commit body.
6. Issue #4 is closed by the iter commit's `closes #4` trailer.
Out of scope (deferred to separate work):
- The "Closure-pair slab / pool" optimisation (Gitea #3) that
would tighten the closure-chain ±15% bench-health band. Not
blocked by retirement; can ship anytime.
- Any further `AllocStrategy::Bump` rework (still a single-
variant bench instrument).
- Any bench-harness *recalibration* — that just closed
(bench-harness-recalibration audit, 2026-05-20); this milestone
preserves the calibration.