Files
AILang/design/contracts/memory-model.md
T
Brummel 14a91f0ae5 iter boehm-retirement.1 (DONE 10/10): retire the transitional Boehm GC backend
Closes Gitea #4. Removes the Boehm-Demers-Weiser conservative GC
backend wholesale across six layers in one atomic iteration. After
this iter, `AllocStrategy` has two variants (`Rc`, `Bump`),
`--alloc=gc` is rejected at CLI parse with `unknown --alloc value`,
the libgc link arm is gone, and the design ledger describes RC
(canonical) + bump (raw-alloc bench-floor) as the only allocators.

Layer-by-layer summary:

  CLI surface — `crates/ail/src/main.rs`:
    `parse_alloc_strategy` arm `"gc" => Ok(AllocStrategy::Gc)`
    removed; error wording updated to `(expected `rc` or `bump`)`;
    clap-derive `value_parser = ["gc","bump","rc"]` allowlist on
    BOTH `Build` and `Run` subcommands DROPPED so that
    `parse_alloc_strategy` remains the sole gatekeeper for the
    unknown-value diagnostic (otherwise clap shadows the runtime
    diagnostic with `invalid value 'gc' for '--alloc'`, which would
    miss the milestone-pin's stderr substring check). The
    `default_value = "rc"` stays.

  Codegen — `crates/ailang-codegen/src/lib.rs`:
    `AllocStrategy::Gc` variant + `Default` derive removed (no
    caller of `AllocStrategy::default()` existed in the workspace,
    so the trait derivation was dead). `fn_name` (spec called it
    `runtime_alloc_fn` loosely; actual identifier is `fn_name`)
    drops the `Gc => "GC_malloc"` arm. `lower_workspace` and
    `lower_workspace_staticlib` defaults flip from `Gc` to `Rc`.
    In-source negative-complement codegen test (mod tests, lib.rs:3571ff)
    retargets from `AllocStrategy::Gc` to `AllocStrategy::Bump`
    (bump also doesn't emit per-type drop fns; the test's semantic
    "no drop fns under non-RC" is preserved).

  Link branch — `crates/ail/src/main.rs:2389ff`:
    The `match strategy { AllocStrategy::Gc => { ... cmd.arg("-lgc"); ... } }`
    arm and its libgc-link block are entirely gone. The surviving
    match exhausts on `Bump` and `Rc` (Rust's exhaustiveness check
    confirms; no `error[E0004]`). Staticlib-guard diagnostic
    rewritten to drop the "shared Boehm collector" phrasing while
    preserving the prefix `staticlib (swarm) artefact is RC-only`
    verbatim (the surviving `staticlib_bump_is_rejected` test
    depends on that substring).

  Test suite — 3 pure-differential e2e tests deleted
    (`gc_handles_recursive_list_construction`,
    `alloc_rc_produces_same_stdout_as_gc`,
    `alloc_rc_matches_gc_on_std_list_demo`); 9 RC-feature tests
    stripped of their `stdout_gc` build call and differential
    `assert_eq!(stdout_gc, stdout_rc, ...)` (absolute
    `assert_eq!(stdout_rc.trim(), "<n>")` pin retained as
    correctness oracle); `staticlib_gc_is_rejected` deleted; new
    milestone-pin `crates/ail/tests/boehm_retirement_pin.rs`
    asserts `ail build --alloc=gc` exits ≠ 0 with stderr containing
    `unknown --alloc value` and `\`gc\``; `examples/gc_stress.ail`
    fixture deleted (no remaining references).

    Implementer expansion (not in plan): `iter17a_local_box_alloca`
    (in `e2e.rs`) carried an IR-shape assertion against
    `@GC_malloc`-absence as the witness for non-escaping
    allocation. After the Task-2 codegen default flip, the witness
    shifts to `@ailang_rc_alloc`-absence in escape-targeted
    positions; assertion + doc-comment updated. Property
    protected ("no heap allocation in non-escaping contexts") is
    unchanged; only the named allocator shifts.

  Bench harness — `bench/run.sh` 9→6 column compaction
    (workload + bump(s) + rc(s) + rc/bump + bump RSS + rc RSS);
    gc-arm `bench_latency_implicit_gc` build call + harness
    invocation dropped from latency block; header comment reframed
    from "GC-overhead bench harness" to "RC-overhead bench
    harness"; "Decision 10's Boehm-retirement target (1.3x)"
    rewording to "RC-overhead-vs-bump bench-health regression gate".

    `bench/check.py:62` header-sentinel changes from
    `"gc(s)" in line` to `"bump(s)" in line`; column-count check
    at `:72` flips from `!= 9` to `!= 6`; per-workload field set
    drops `gc_s`/`gc_over_bump`/`gc_rss_kb`; `ARM_LABEL_TO_KEY`
    drops the `"implicit @ gc": "implicit_at_gc"` entry.
    `bench/baseline.json` regenerated via `--update-baseline`.

    Implementer note (planner-defect): `write_new_baseline`
    iterated over the *existing* baseline's metric list when
    emitting the regenerated file, so even after parser-level
    `gc_*` removal, the fallback emitted them back into the JSON.
    Scrubbed post-update; the cleaner fix (have
    `write_new_baseline` emit only keys present in
    `parsed_throughput[workload]`) is a follow-up if the script
    becomes load-bearing for further allocator changes.

  Design ledger — `design/models/rc-uniqueness.md` excises the
    `## Dual allocator — RC canonical, Boehm parity oracle`
    section and the `Boehm-Demers-Weiser conservative GC` choice
    block + rationale + trade-offs; the per-fn-alloca section
    generalises Boehm-specific language to allocator-agnostic;
    the memory-model section's `## Choice.` paragraph reframes the
    1.3× target from "Boehm-retirement gate" to "bench-health
    regression gate".

    `design/models/pipeline.md` drops the `--alloc=gc → links libgc`
    arm of the pipeline diagram and replaces it with
    `--alloc=bump → links bump-floor`; the accompanying prose
    rewrites accordingly.

    `design/contracts/scope-boundaries.md` rewrites the
    "Memory management via Boehm conservative GC" bullet to
    describe RC + per-fn-arena present-tense; the dead reference
    to `examples/gc_stress.ail.json` (file never existed; the
    fixture only ever had a `.ail` form, deleted by this iter) is
    dropped along with the `examples/std_list_stress.ail.json`
    reference whose purpose was Boehm-only soak testing.
    `:67`'s `@printf` / `@GC_malloc` parenthetical updated.

    `design/contracts/memory-model.md:232` drops the
    "leaks like the pre-Boehm era" phrase; the RC inc/dec
    instrumentation is wired up, so the "until then" conditional
    that referenced pre-Boehm is closed.

    `design/contracts/embedding-abi.md:42-44` rewrites the
    staticlib-guard prose to drop the `--alloc=gc` clause (gc is
    now a CLI-parser-level unknown-value, not a staticlib-guard
    rejection) and reframe the swarm-safety justification around
    `--alloc=bump` (leak-only bench instrument) rather than the
    historical Boehm collector.

  Honesty pin — `crates/ailang-core/tests/docs_honesty_pin.rs`
    inverts the polarity: the present-tense Boehm-anchor assertion
    on `pipeline.md` (`:116-117`) is deleted, and four
    absence-pins are added to `design_md_has_no_wunschdenken`
    against the Boehm-zombie strings `transitional Boehm`,
    `parity oracle`, `GC_malloc`, `libgc`. The
    `design_corpus()` already includes `rc-uniqueness.md` so no
    path-list change was needed for the new pins to scan.

    `crates/ailang-core/tests/design_index_pin.rs:166` drops the
    `"pre-Boehm"` token from the protected-exception comment list
    (the phrase no longer appears in `memory-model.md` after this
    iter, so the exception is dead).

  Runtime docs — `runtime/bump.c`, `runtime/rc.c`, `runtime/str.c`
    header comments scrubbed of Boehm/`GC_malloc`/`libgc`
    references. `bump.c`'s function signature description still
    documents `void *bump_malloc(size_t)` as the bench-floor
    allocator interface, but no longer cross-references libgc.

  Example fixtures — `examples/bench_latency_implicit.ail`,
    `bench_latency_explicit.ail`, `escape_local_demo.ail`,
    `reuse_as_demo.ail`, `rc_pin_recurse_implicit.ail` doc-comment
    headers scrubbed of `--alloc=gc` / Boehm references. The
    `.ail` surface (AST) is untouched in every case; round-trip
    invariant holds (`cargo test -p ailang-surface --test round_trip`
    green).

  Skill / agent prompts — `skills/audit/agents/ailang-bencher.md`
    rewritten to use an RC-vs-bump worked example pattern for the
    hypothesis-driven bench tutorial, replacing the recurring
    "RC vs Boehm under heap pressure" example.
    `skills/implement/agents/ailang-implementer.md` Decision-10 /
    Boehm references replaced with present-tense RC-commitment
    framing.

  IR snapshots — the 5 checked-in snapshots
    (`crates/ail/tests/snapshots/{hello,list,max3,sum,ws_main}.ll`)
    regenerated via `UPDATE_SNAPSHOTS=1 cargo test -p ail --test
    ir_snapshot`. Each previously contained
    `declare ptr @GC_malloc(i64)` and (for `list.ll`) a `call ptr
    @GC_malloc(...)` invocation; post-flip the snapshots contain
    `declare ptr @ailang_rc_alloc(i64)` plus the rc inc/dec runtime
    declarations.

Spec-vs-acceptance addendum (caught at orchestrator end-report,
absorbed here rather than in a follow-up spec edit): spec §6
acceptance criteria said "Boehm-grep returns matches ONLY in
docs_honesty_pin.rs". The plan itself prescribed historical Boehm
references in 3 additional files: (a) the new milestone-pin
`boehm_retirement_pin.rs` (must literally invoke `--alloc=gc` to
assert its rejection), (b) `embed_staticlib_alloc_guard.rs` file
doc-comment historical note ("`--alloc=gc` no longer exists as a
CLI value"), (c) `embedding-abi.md:44-45` contract historical
clause ("see the Boehm-retirement iter"). All three are
prescribed; the spec's grep wording was too narrow. The four
absence-pins in `docs_honesty_pin.rs` catch the actual zombies
(Boehm-narrative re-emerging in the design ledger), which is the
substantive intent the spec was aiming at — the four extra
documented-by-design exceptions are the cost of having an
explicit milestone-pin and contract-level historical anchors.

Net delta:
  - 32 files modified, 2 new (boehm_retirement_pin.rs + stats),
    1 deleted (gc_stress.ail);
  - workspace tests: every binary `0 failed`. Pass-count delta:
    -3 net (4 e2e tests deleted, 1 new milestone-pin test added);
  - boehm-grep state: hits only in the four by-design exceptions
    documented above;
  - `bench/check.py` exit 0 against regenerated baseline;
  - CLI must-fail fixture: `ail build --alloc=gc examples/hello.ail`
    exits non-zero with stderr containing `unknown --alloc value`
    and `\`gc\``;
  - design ledger present-tense honest (Boehm-narrative gone from
    `rc-uniqueness.md` + `pipeline.md`; the few historical
    references in `embedding-abi.md` / `boehm_retirement_pin.rs` /
    `embed_staticlib_alloc_guard.rs` are explicit milestone-pins
    or contract anchors, not silent ledger residue).

Bench measurement variance noted: closure-chain and hof-pipeline
are ±1-5% jittery between runs; one regeneration flagged 2
metrics as `regressed` before a second run returned 0. The
captured baseline is within self-comparison range. Existing
per-metric tolerances absorb the jitter.

Stats file:
`bench/orchestrator-stats/2026-05-20-iter-boehm-retirement.1.json`.

closes #4
2026-05-20 20:51:53 +02:00

16 KiB

Memory model — schema, diagnostics, codegen contract

The four language-design constraints that make RC sound without a cycle-collector backstop (strict evaluation, no recursive value bindings, no shared mutable refs, acyclic ADTs) live in language constraints; this file covers the schema additions, advisory diagnostics, and codegen contract that build on them.

Schema additions

Parameter modes on Type::Fn (see Data model for the schema-level definition of Type::Fn).

The form-A surface (see authoring surface) for fn signatures gains mode wrappers:

(fn-type (params (borrow (List Int))) (ret (con Int)))
(fn-type (params (own (List Int))) (ret (own (List Int))))

Internally, this is not a new Type variant. Modes are metadata on Type::FnparamModes and retMode fields run parallel to params and ret (see Data model for the JSON schema). The substantive reasons for per-position metadata over a Type::Borrow / Type::Own variant approach:

  • Semantic locality. Modes are properties of fn-signature parameter positions, not of types in general. Int does not have a mode; a fn-parameter slot does. Embedding modes in Type would let the schema express forms like (con List (borrow Int)) — syntactically possible, semantically meaningless (you cannot separately own/borrow a list element from the list it lives in). The canonical-schema principle is "schema = data, schema permits exactly what is meaningful"; per-position metadata is the option that holds that line.
  • Compositional clarity. A Type value's identity should depend only on the type. Two functions with the same param / ret types but different calling conventions share Type::Fn.params and differ only in param_modes. That is the right factoring: "what data does this carry" is one axis, "how is it transferred" is another. Mixing them under a single hierarchy conflates the two and makes both harder to reason about.
  • Future-proof against more position metadata. If later iters add other per-position properties (streaming receiver, captured- by-closure, lifetime witness), they generalise as additional metadata fields on Type::Fn — one consistent hierarchy. The variant approach would force every new dimension into its own Type::* variant (Type::Streamed, Type::Captured, ...) and combinatorics blow up: Type::Borrow(Type::Streamed(T)) versus Type::Streamed(Type::Borrow(T)) raise questions of canonical ordering that don't exist when modes live in a flat metadata vector.

Implicit is the legacy / back-compat state — semantically equivalent to Own but printed bare ((con T), no wrapper). Own and Borrow are explicitly annotated.

JSON canonical hash for every existing fixture stays bit- identical: param_modes is skipped when every entry is Implicit, ret_mode is skipped when Implicit. Existing modules emit the same bytes as before.

Type::Con name scoping (canonical form). Within a .ail.json, a Type::Con.name is interpreted relative to the file's top-level "name" field (the owning module). Bare names (no .) refer to a TypeDef in the owning module's own defs. Cross-module references MUST be qualified <owning_module>.<TypeName> where <owning_module> is a known module in the workspace. Primitives (Int, Bool, Str, Unit, Float) are bare and have no module qualifier. Bare cross-module references are a schema violation (WorkspaceLoadError::BareCrossModuleTypeRef); qualified references whose owner is unknown are also a violation (WorkspaceLoadError::BadCrossModuleTypeRef). The same rule applies to Term::Ctor.type_name.

Class names follow the same canonical-form rule: bare for same-module references, <module>.<Class> for cross-module references — symmetric to Type::Con.name's rule above. Three schema fields carry class references in this form: InstanceDef.class, Constraint.class, and SuperclassRef.class. ClassDef.name itself stays bare (defining-site context, like TypeDef.name).

Method dispatch is type-driven (see Method dispatch): synth resolves a Term::Var { name: "show" } by consulting the workspace's method-to-candidate-class index, filtering by argument type (concrete) or by declared constraint (rigid-var), and routing the residual through the registry at fn-body-end discharge. Method-name collisions across classes are now structurally legal — they resolve at the call site via type-driven dispatch with explicit qualifier (<module>.<Class>.<method>) as the LLM-author's disambiguation tool.

The legacy (con T) form is treated as (own T) semantically.

(An incidental observation, not a design reason: keeping Type itself unchanged also avoids touching ~250 sites across the typechecker / desugar / codegen that match on Type variants. This is a tiebreaker, not a rationale — the substantive reasons above are what justify the choice.)

New Term variants.

Term::Clone { value: Box<Term> }                     ; `(clone X)` — explicit RC inc
Term::ReuseAs { source: Box<Term>, body: Box<Term> } ; `(reuse-as SRC NEW-CTOR)`

Term::ReuseAs is structured as a wrapper around a body term rather than as a reuse_from: Option<String> modifier on Term::Ctor. Two substantive reasons:

  1. Compositional flexibility. Reuse-as is conceptually a wrapper that says "this expression's allocation comes from <source>'s slot". The wrapper form generalises naturally if future iters introduce other allocating constructs (record literals, opaque box wrappers, capability cells) — they all become valid body positions. A modifier on Term::Ctor would have to be replicated on every constructible Term variant the language grows.
  2. Source-locality at the head. (reuse-as SRC NEW-CTOR) reads as a single sentence with the source-binder named at the head. The modifier form would scatter the reuse intent across a child position of the constructor's argument syntax, separating the source from the rest of the reuse-as semantics.

The trade-off this accepts: the schema permits Term::ReuseAs { body } where body is not an allocating form (e.g. a literal, a var). Such terms are caught at typecheck via a reuse-as-non-allocating-body diagnostic — structural rejection in the typechecker, not the schema. The principle: prefer composability over schema-level rejection where the typecheck rule is unambiguous.

TypeDef attribute.

TypeDef.drop_iterative: bool                         ; `(drop-iterative)`

All four are skipped during serialisation when absent / false / None so canonical-JSON hashes of every fixture remain stable until the fixture intentionally adopts the feature.

FnDef.suppress. The suppress field on FnDef carries a list of advisory-diagnostic suppress entries; each entry has a code (the diagnostic being suppressed) and a because (a mandatory non-empty reason). See Data model for the canonical schema.

Form-A surface: (suppress (code "...") (because "...")) clause between fn name and (type ...). Multiple clauses allowed; one per entry. Form-B (prose) renders one // @suppress <code>: <because> line per entry above the doc string — lossless, contract metadata.

Skipped from serialisation when empty so existing fixtures keep bit-identical canonical-JSON hashes (regression-pinned by iter19b_empty_suppress_preserves_pre_19b_hashes and iter19b_schema_extension_preserves_pre_19b_hashes). The canonical-form tightening for Type::Con.name shifted the hashes of two cross-module fixtures (ordering_match.ail.json and test_22b1_dup_a.ail.json); all intra-module fixtures, including the regression-pinned sum.ail.json and list.ail.json, remain bit-identical. The new pins are ct4_migrated_fixtures_have_canonical_form_hashes (locks the post-migration hashes) and ct4_unmigrated_fixtures_remain_bit_identical (re-asserts the pre-tightening hashes still hold).

Advisory diagnostics

The advisory-diagnostics arc introduces the language's first advisory typechecker diagnostic and the suppression mechanism that goes with it. The mandatory-annotation rule of this memory model is unchanged: param_modes and ret_mode remain author-required; the typechecker does not infer them. What's new is feedback when an authored annotation is stricter than necessary.

The lint: over-strict-mode. Fires on a fn-param p annotated (own T) when:

  1. p's consume_count == 0 (uniqueness pass: the body never consumes p as a whole).
  2. For every match arm whose scrutinee is p, no heap-typed pattern-binder has consume_count > 0.

The heap-type filter is load-bearing for soundness: match xs { Cons(h, t) => h } records consume_count(h) == 1, but h: Int is read by-value — no RC traffic, no heap data moved out of xs's allocation. Filtering primitive-typed binders is what lets the lint correctly identify head_or_zero as over-strict (could be borrow) while staying silent on sum_list where t: List is moved out.

Severity: Warning. ail check, ail build, ail emit-ir exit 1 only on at least one Error; warnings print but do not abort.

The suppression: mode-strict-because. Authors who want to keep an over-strict annotation deliberately (e.g. RC codegen-test fixtures, fns reserved for planned in-place mutation) attach a Suppress entry naming the diagnostic code and a non-empty reason. The typechecker drops matching diagnostics from the output. Empty because is a hard error (empty-suppress-reason); wrong-code suppresses are silent no-ops (open-set diagnostic registry — a suppress for a code that doesn't fire today may exist defensively for a code that might fire after a future edit).

Codegen contract

Memory layout:

  • Every heap allocation has an 8-byte refcount header, followed by the payload. ailang_rc_alloc(size) returns a pointer to the payload; the header is at ptr - 8.
  • ailang_rc_inc(ptr): load ptr - 8, +1, store. Non-atomic (single-threaded).
  • ailang_rc_dec(ptr): load, -1, store; if zero, recurse-dec child references and free(ptr - 8). For (drop-iterative) types, the recursion is replaced by a worklist loop (via drop-iterative).

Codegen for Term::Ctor / Term::Lam env / closure pair under --alloc=rc calls ailang_rc_alloc(SIZE); inc/dec instrumentation is emitted per the uniqueness inference. --alloc=bump selects the bench-floor allocator, which leaks by design (no inc/dec, no free); it is bench-only and never a production target.

Mode metadata is load-bearing for codegen

param_modes and ret_mode on Type::Fn are not merely typechecker metadata — codegen consults both to decide where to emit drop calls. They were promoted from "annotation that the typechecker enforces" to "annotation that codegen reads to keep RC correct". Recorded here so the schema metadata's role is explicit:

param_modes — drop-emission gates.

  • Iter B: Own-param dec at fn return. When a fn body fall-throughs to a ret (no tail-call), every parameter with param_modes[i] == Own is dec'd before the ret iff its uniqueness consume_count == 0 and the ret value is not the param itself. Borrow and Implicit parameters are skipped: Borrow retains the caller's ownership by contract; Implicit carries no static caller-handed-off-ownership signal (it's the back-compat lane).

  • Iter A: arm-close pattern-binder dec. When a match-arm's body terminates without a tail-call, every ptr-typed pattern-bound binder pushed by the arm is dec'd at arm close iff its consume_count == 0 and it is not the arm's tail value, gated on the scrutinee's static ownership. If the scrutinee is a fn-param, only Own-mode scrutinees enable the dec — Borrow and Implicit scrutinees would let the arm dec memory the caller still references.

  • Pre-tail-call shallow-dec. When a match-arm's body IS a tail call, both Iter A and Iter B are skipped (the block is terminated). A separate seam in lower_match emits a shallow ailang_rc_dec on the scrutinee outer cell BEFORE the tail call, gated identically on the scrutinee mode plus the requirement that every ptr-typed slot in the active ctor's pattern is in moved_slots[scrutinee].

ret_mode — let-binder trackability.

  • Term::App drop at let-scope close. A let-binder whose value is Term::App { callee, .. } is trackable for scope-close drop iff the callee's ret_mode == Own. The signal is the callee's static contract that ownership of the freshly heap-allocated cell flows to the caller. Borrow-returning calls remain non-trackable (the callee retains ownership; the caller holds a view, not an own ref). Implicit-returning calls remain non-trackable (back-compat lane).

The drop fn's symbol resolution for an Own-returning App: synthesise the call's return type, resolve Type::Con { name } to drop_<owner>_<T> (with cross-module qualification through the import map). Falls back to shallow ailang_rc_dec for returns that are not Type::Con (e.g. unresolved type vars on a polymorphic call's pre-monomorphisation site; the monomorphised copies resolve to concrete drop fns).

Arg-position policy for compound AST nodes

The uniqueness and linearity passes walk arguments of compound nodes with a fixed Position policy. For ownership-bearing nodes:

Node Arg position Reason
Term::Ctor.args[*] Consume constructor packs values into the cell; the cell owns them afterwards
Term::Do.args[*] Borrow effect-op observes its arguments; the caller still owns whatever pointer it passed in

The two policies are language rules, not per-op annotations. They do not appear as fields on EffectOpSig or Ctor; the AST node kind itself carries the default. The walkers that read this policy live at crates/ailang-check/src/uniqueness.rs and crates/ailang-check/src/linearity.rs (matched arms in both).

The Do = Borrow rule pairs with the ret_mode == Own letbinder- trackability rule above: when a built-in such as int_to_str is declared ret_mode: Own and its result is fed into an effect-op (io/print_str s), the let-binder is RC-tracked for scope-close drop and the effect-op does not consume it — the slab is freed exactly once at scope close, never zero-times (RC leak under the old Consume rule, which silenced the scope-close drop) and never twice (double-free under a hypothetical Consume + scope-close).

What this widening does NOT do.

  • Does not change the canonical hash. param_modes / ret_mode were already hash-load-bearing when introduced; subsequent work added codegen consumers, not new schema fields.
  • Does not introduce a new Type variant. Mode metadata stays flat on Type::Fn (see "Schema additions" above on why).
  • Does not cover let-aliases of borrowed values. A let-binder whose value is Term::Var referencing a Borrow-mode param is not yet propagated through; the param-mode gates treat such a binder as "owned" (its current_param_modes lookup misses, default = owned). This is a known carve-out shared by Iter A and the pre-tail-call shallow-dec arm; closing it is a propagation pass through let-bindings that has not shipped yet.

Ratified by: crates/ailang-check/src/uniqueness.rs.