Three drift items from ailang-architect, plus one false-positive
surfaced during verification:
1. DESIGN.md silent on closure-pair 4.14x finding (21'b).
Decision-10 ratified: "Workload scope of the 1.3x target"
paragraph scopes the retirement gate to linear/tree/poly-ADT
workloads; closure-pair carve-out documented as
representational cost (closure cell + env struct = 2 allocs
per step) until a slab/pool answer ships.
2. bench/compile_check.py corpus drift. Three fixtures added
(bench_compute_intsum, bench_compute_collatz,
bench_list_sum_explicit), re-baselined. Now 12 fixtures x
2 ops = 24 compile-time metrics.
3. baseline.json convention not codified. Note field gains
"max-of-distribution gets wider band than percentile"
convention discovered in 21'd.
4. (verification finding) bench_compute_intsum cross_lang
tolerances at 15%/12% fire on subprocess-spawn jitter for
sub-millisecond fixtures. Widened to 35% across all five
intsum metrics; convention recorded in baseline_cross_lang
note field (sub-ms fixtures need looser bands).
All three bench gates re-run sequentially after edits:
bench/check.py — 63 metrics; 63 stable
bench/compile_check.py — 24 metrics; 24 stable
bench/cross_lang.py — 25 metrics; 25 stable
Total: 112 metrics under regression coverage, all green.
288 tests passing, 3 ignored. No Rust changes.
Closes the apples-to-apples gap from 21'e. Adds:
- examples/bench_list_sum_explicit.ailx — same algorithm and sizes
as bench_list_sum, fully (borrow)/(own)/(drop-iterative)
annotated so codegen emits proper inc/dec instrumentation.
- bench/reference/list_sum_explicit_free.c — same algorithm
with explicit free() walking the chain after sum.
The full alloc+dec vs malloc+free comparison reveals two non-
trivial conclusions:
1. AILang's full RC pipeline is only 26% slower than glibc
malloc+free on this workload (rc/c = 1.26x). The implicit-
mode comparison's 1.42x was misleading — it counted neither
pipeline's free path. The fair ratio is 1.26x, materially
better than the previous read.
2. RC's dec is cheaper per cell than glibc free(). AILang
dec-tax: ~3 ns/cell. C free-tax: ~5.5 ns/cell. Plausible
cause: ailang_rc_dec operates on a known-shape cell with a
fixed-offset refcount and a static per-type drop fn — no
free-list bucketing, no header introspection, no global lock.
bump's advantage expresses fully: bench_list_sum_explicit.bump/c
= 0.42x means AILang at bump is 2.4x faster than C malloc+free.
Sets a useful upper bound on a slab/pool RC allocator's potential.
The 21'-family arc — bench-regression infrastructure — is now
substantively complete: 21'a (bench/check.py), 21'b (corpus
widening), 21'c (compile_check.py), 21'd (pure-compute fixtures
+ harness hardening), 21'e (cross-language hand-C), 21'f (explicit
apples-to-apples). 63 runtime metrics + 18 compile metrics + 25
cross-lang metrics under regression coverage. Any future iter
that regresses any axis beyond tolerance gets caught at the next
family close.
Remaining queue is back to substantive language work — Family 21
(typeclasses / polymorphic ADTs at runtime / pattern-binding
generalisation) is now an orchestrator-level fork that needs
direct user input.
Closes the question CLAUDE.md has carried since day one ("LLVM-
linkable, performance is extremely important") with data. Hand-C
variants of the four bench fixtures, compiled with clang -O2,
each carefully matching the AILang algorithm and explicitly
documenting representation differences (cell width, leak policy)
that affect the ratio.
Three substantive findings:
1. Pure-compute parity with C: bench_compute_collatz runs at
AILang/C = 0.99x across both allocators. AILang's IR composes
with LLVM's optimizer at the same level a hand-C source does.
This is the LLVM-linkable performance claim, backed by data
for the first time. bench_compute_intsum (1.05-1.18x) confirms.
2. AILang bump beats glibc malloc 2x on linear allocation:
bench_list_sum.bump/c = 0.50x. Bump's two-instruction inline
fastpath outperforms glibc's free-list-managed malloc on
no-free workloads. Quantitatively measured for the first time.
3. RC overhead vs C malloc quantified: bench_list_sum.rc/c =
1.49x, bench_tree_walk.rc/c = 2.61x. The 8-byte refcount
header + zero-init + libc backing add 50-160% over glibc
malloc on these implicit-mode workloads. Explicit-mode + a
free()-adding C variant (21'f, queued) will close the
apples-to-apples gap on dec-cost.
CLAUDE.md updated to list bench/cross_lang.py as the third
tidy-iter gate alongside bench/check.py and bench/compile_check.py.
20 new metrics in bench/baseline_cross_lang.json with 12-15%
tolerances (cross-language ratios are inherently noisier than
within-AILang ratios — two compiler stacks contribute variance).
Closes the third corpus blind spot (heap-allocation-only) by
adding two fixtures with no allocation pressure: bench_compute_
intsum (tail-recursive integer accumulator) and bench_compute_
collatz (Collatz step-counter, branchy).
Surprise on intsum: 50M-iteration loop runs in 1ms wall under
all three allocators. LLVM's induction-variable analysis applies
the closed-form triangular-sum reduction to AILang's IR — a
positive codegen finding (the IR composes with LLVM's optimizer
at the same level a hand-C loop would) but it makes intsum
useless as a runtime regression bench. Excluded from run.sh's
fixtures array; kept in examples/ as reference and as a future
cross-language comparison anchor.
Collatz survives optimization (data-dependent control flow). At
56ms wall, gc/bump/rc all within 2% — the canonical "pure-compute
is allocator-invariant" data point this fixture is meant to
prove. If a future codegen change leaks an allocation into the
inner loop, the 1.00x / 1.02x ratios diverge visibly.
Two infrastructure fixes the new fixtures forced:
- 6-decimal precision in run.sh's Python timing helper and median
averager (was 3-decimal; sub-ms times rounded to 0.000 and
crashed the ratio awk with Division durch Null).
- Zero-guard in the ratio awk (defensive even with the precision
bump, since LLVM-eliminated workloads can still hit zero).
Latency baseline: implicit_at_rc.max_us tolerance 25% -> 30%.
Three captures today (477 / 456 / 609 µs) show natural run-to-run
dispersion wider than the original tolerance accounts for. Not a
softening to dodge regression — the original baseline was the
first capture; a fairer tolerance across natural max-of-1000-
samples width is what the harness needed from the start.
Baseline file: 47 -> 55 metrics. 21'e (cross-language reference,
clang -O2 hand-C ratios) is the natural next dispatch.
Closes the second axis the user named: every typechecker / codegen
perf change was previously invisible to the tidy-iter gate. With
Family 21 typeclasses (queued) and 21'b's poly-ADT additions both
pushing on the typechecker, naive substitution loops would have
landed silently and decayed the compile path.
bench/compile_check.py is a separate script from bench/check.py
because the methodology differs: sub-process spawn timing on small
workloads (1ms scale for `ail check`, 65ms for `ail build`) vs.
allocator-stress on large ones (multi-second). Tolerances differ
by an order of magnitude (25% / 20% here vs. 5-15% there).
Empirically: ail check is sub-ms across the corpus, dominated by
subprocess spawn (~5-10ms on Linux); ail build is 63-69ms,
dominated by clang's link step. The bench is a catastrophe
detector (10x slowdowns visible) — finer regressions need a
profiler. CLAUDE.md updated to list both scripts as co-equal
tidy-iter gates alongside the architect drift report; exit 0/1/2
semantics are uniform across both.
JOURNAL queue: 21'd (pure-compute fixtures) and 21'e (cross-
language reference / hand-C ratio) remain to land the LLVM-
linkable performance claim.
Two new throughput fixtures targeting blind spots in the 21'a
corpus:
- bench_closure_chain exercises the build_pair_drop_fn codegen
path (the 18c.4 doubled-braces trigger). Each iteration of
run_loop allocates a {thunk, env} closure pair via the
let-rec-name-as-value escape route. Sizes 10k / 100k / 500k.
rc/bump = 4.14x — materially worse than the 2.91x / 2.59x of
the linear/tree fixtures, exposing that closure work pays the
RC alloc tax twice (pair + env-struct).
- bench_hof_pipeline exercises poly-ADT instantiation and
indirect dispatch via fold_with_fn over List<a>. Sizes 100k /
1M / 3M elements. Ratios essentially match bench_list_sum,
confirming the 13b static-template-plus-ctor-inline design
has zero measurable overhead at this scale.
Baseline file extends from 31 to 47 metrics. The two new fixtures
build clean under all three allocators; the rc-arm build exercises
the per-type drop fn for the closure-pair, providing a tripwire
for any future 18c.4-class IR malformedness.
JOURNAL records both surprises (4.14x closure tax, ~zero HOF/poly
overhead) and explicitly notes the dispersion observation on
explicit_at_rc.p99 — three captures today (357.5 / 294.6 / 251.5)
confirm wide run-to-run variance on that fixture. Methodology
upgrade (n>=10 captures or tighter fixture) deferred to 21'c.
bench/run.sh fixtures array updated. bench/check.py needed no
changes — its parser handles the wider table by metric name.
Closes the structural gap between bench/run.sh (one-shot capture
into JOURNAL) and a continuous tripwire. baseline.json records 31
metrics (16 throughput, 15 latency) with per-metric one-sided
tolerances tuned to absorb run-to-run noise on a quiet developer
machine; check.py spawns run.sh, parses both the throughput
pipe-table and the per-arm latency stanzas, diffs against the
baseline, prints a per-metric report, and exits non-zero on any
regression beyond tolerance.
User-facing flags: --from-file, --stdin, --baseline, and
--update-baseline (re-run + overwrite baseline.json after
intentional improvements).
Validation: captured the baseline, then re-ran end-to-end. All 31
metrics within tolerance. The harness also caught a single-capture
explicit-rc p99 spike (357.5 us) that was first read as drift vs.
yesterday's 18g.tidy.fu2 numbers but came in at 294.6 us on the
follow-up run — exactly the kind of noise the tolerance band is
there to absorb. Without 21'a we would have either chased a
phantom or buried the signal; with it, single noisy runs are data
points, not verdicts.
Tidy-iter discipline addition (run check.py at every family close
alongside the architect drift report) is recommended in JOURNAL
but not enacted in this iter — that's an orchestrator-level
update to CLAUDE.md.
The dual-allocator policy from 2026-05-08 made Boehm the CLI default
"for general workloads", with rc selected explicitly for real-time-
sensitive code. That asymmetry contradicted Decision 10: rc + uniqueness
inference is the canonical memory model AILang's typechecker enforces,
and the CLI was teaching the opposite mental model to users and prompt-
fed LLMs.
This iter flips the asymmetry without removing Boehm:
- main.rs: default_value gc -> rc for build/run; help text rewritten
so rc is the canonical path and gc is the parity oracle.
- DESIGN.md Decision 9 retitled "RC canonical, Boehm parity oracle"
and rewritten end-to-end. Migration plan step 6 updated to a
two-step retirement (default-flip done, full removal gated).
- DESIGN.md pipeline diagram: rc-first, gc-as-oracle.
- JOURNAL: 2026-05-09 entry recording the call, what shipped,
the bug the flip surfaced, the gating condition for full
Boehm removal.
Bug surfaced by the flip and fixed in the same iter:
codegen/drop.rs build_pair_drop_fn emitted `getelementptr inbounds
{{ ptr, ptr }}, ...` via push_str (not format!), so the doubled
braces leaked verbatim into the IR. LLVM parsed it as a struct-of-
struct, the GEP referenced a non-existent field, and clang failed.
Invisible under the old gc default (no per-type drop fns) and
invisible to the 8 explicit _with_alloc("rc") tests (none of them
escaped a closure-with-captures). Caught immediately by two corpus
tests once rc became the baseline: closure_captures_let_n,
local_rec_as_value_capture_demo. Fix: single braces. Tests stay
as the regression guard.
The episode is the canonical justification for keeping the GC arm
as a parity oracle for now — a months-old codegen bug found by
flipping the baseline column.
288 passed / 0 failed / 3 ignored, unchanged.
Architect drift review after Family 20 surfaced four items; (1) and
(2) are doc drift fixed here, (3) and (4) are recorded as acceptable.
DESIGN.md changes:
- Data model section rewritten from "Data model (MVP)" claiming
only fn/const exist to "Data model" reflecting all three Def
kinds. Added the missing schema fields the architect named:
`tail` flag on app/do, the `seq` / `clone` / `reuse-as` / `letrec`
terms, `paramModes` / `retMode` on Type::Fn, `suppress` on FnDef,
`drop-iterative` on TypeDef, `args` on Type::Con, `vars` on
TypeDef. Each additive field's hash-stable serialisation is
called out once at the top so individual mentions stay terse.
`Suppress`, `ParamMode`, `Literal`, `Pattern` get their own
sub-blocks.
- Pipeline diagram now shows `--alloc=gc` (links libgc;
transitional) and `--alloc=rc` (emits ailang_rc_inc / _dec;
canonical) as two backends sharing the same MIR.
- Ecosystem inventory gained a "Surface forms" bullet for
ailang-surface (Iter 14c, lossless Form-A printer/parser) and
ailang-prose (Family 20, lossy Form-B projection). CLI bullet
now lists parse, render, prose, merge-prose.
JOURNAL entry records the architect findings, the resolutions for
items (1) and (2), and the acceptable-drift status of (3)
FnDef::synthetic factor-out (defer to next schema-additive
FnDef field) and (4) linearity.rs spurious warning (not
corpus-triggered).
Pure documentation iter — no code changes. 288 workspace tests
unchanged and green.
Closes a design hole shipped in 20d: the merge-prose prompt instructed
the LLM to emit JSON-AST and gave a 12-line schema-essentials reminder.
JSON-AST is the canonical hashable artefact, not a writing surface; the
reminder was not a language spec. Foreign LLMs had no realistic shot.
20f makes two coupled changes:
1) The LLM now emits Form-A (the canonical authoring surface fixed by
Decision 6). merge-prose loads the original via ailang_core::load_module
and re-renders via ailang_surface::print before embedding (round-trip
is a gating contract on the surface crate, so this is lossless). The
user runs `ail parse foo.new.ailx` to recover JSON, then `ail check`.
2) crates/ailang-core/specs/form_a.md is the complete LLM-targeted
Form-A specification — grammar, every term/pattern/type/def keyword,
schema invariants, pitfall catalogue, four few-shot modules from
examples/*.ailx. Exported as ailang_core::FORM_A_SPEC via include_str!
and embedded verbatim in every merge-prose prompt.
Drift detection in crates/ailang-core/tests/spec_drift.rs: every variant
of Term, Pattern, Type, Def, Literal is reached via exhaustive `match`.
The arms are not the assertion — adding a new variant without updating
the match is a compile error before the test runs. Once matched, an
anchor string is asserted to appear in FORM_A_SPEC. 8 tests, all green.
The hand-written + mechanical-drift-test combo addresses the user's
"distance to code is too big" concern about a docs-only spec. Generator
overkill rejected: structural drift is mechanically caught, but prose
explanation, schema-invariant catalogue, pitfall list, and few-shot
corpus cannot be emitted from AST shape alone.
Tests:
- ailang-core: +8 spec_drift tests; existing 12 unchanged
- ail unit (3): rewritten in lockstep — assert (own T)/(effects IO)
landmarks, FORM-A SPECIFICATION header, FORM_A_SPEC body verbatim
- ail e2e merge_prose_prints_framed_prompt: rewritten — assert
`(module foo` + `FORM-A SPECIFICATION` instead of `ailang/v0`
- Workspace: all green
Richer integration paths (LLM tool-use, MCP server, LSP) were named
in the design discussion and deferred per "kiss". All three layer
additively on the static-prompt path; static prompt remains the
lowest-common-denominator fallback.
Per CLAUDE.md tidy-iter at family boundaries. ailang-architect
flagged the canonical 'DESIGN.md silent' finding for the entire
19a-arc — same shape as family-20's 20e tidy.
Decision 10 schema-additions block gained 'Iter 19b —
FnDef.suppress' entry. New subsection 'Advisory diagnostics —
Iter 19a-arc' documents the over-strict-mode lint rule (incl.
heap-type filter rationale), Severity::Warning introduction, CLI
exit-on-Error gating, and the suppress mechanism. New subsection
'Why advisory + suppress instead of inference' pins the three
substantive reasons (annotation-states-intent, drift-bremse,
LLM-authoring forcing function) so future iters can't reopen the
decision without engaging the recorded rationale.
Migration plan extended with step 7 covering 19a/19a.1/19b.
Decision 10's mandatory-annotation rule is explicitly reaffirmed
as unchanged.
No code changes. Two architect findings deferred (codegen FnDef
fan-out, snapshot-fixture coupling). Data model (MVP) drift
predates 19b — separate iter.
Corpus-signal upgrade: orchestrator ran 19a's lint across 65
fixtures, zero warnings fired. The conservative match-scrutinee
carve-out filtered every (own T) fixture because every such body
destructures the param via match. Real-corpus signal warranted
the precise variant.
Precise rule: for an Own param with consume_count == 0, walk each
match arm whose scrutinee is the param; if any HEAP-TYPED
pattern-binder has consume_count > 0, stay silent (sub-consume
forces ownership). Otherwise fire.
Heap-type filter is the implementer's design call (not in the
brief): reading a primitive (h: Int) is load-by-value, no RC, no
heap-data move; reading heap-typed t: IntList IS a pointer move
requiring outer-cell ownership. Documented in module-doc.
Corpus signal after upgrade: 5/65 fixtures fire — all RC
codegen-test fixtures with deliberate over-strictness for the
codegen path. Justifies 19b (mode-strict-because suppression).
ailang-check test count: 53 → 55. e2e + everything else green.
Known debt: deeply-nested-match-on-sub-binder (no fixture hits).
Family-20 tidy-iter: ailang-architect drift review surfaced
DESIGN.md silence on the new prose surface as the highest-leverage
gap. Decision 6 now has a 'Form (B) — human prose projection'
subsection naming the lossy-vs-lossless contract, the no-parser-
by-design choice, and the LLM-mediated round-trip. CLI block
gained 'ail prose' and 'ail merge-prose'.
No code changes. Form (B) does not weaken any Decision 6
invariant: JSON-AST remains the only hashable artefact, form (A)
the canonical authoring surface, 30-production grammar unchanged.
Two architect findings deferred (prompt-template lockstep test;
snapshot-fixture cross-family coupling). Family 20 closes.
JOURNAL queue: empty.
Closes family 20: prose-edit cycle is end-to-end. ail merge-prose
<original.ail.json> <edited.prose.txt> composes a shaped LLM prompt
to stdout. User pipes to their LLM (Claude/etc.), gets new .ail.json,
runs ail check, saves.
No built-in API client — AILang stays a compiler + tooling, the LLM
mediator is supplied externally. PROSE_ROUNDTRIP.md documents the
6-step cycle, the prompt contract (preserve modes/effects/tail
flags from original), failure modes, and the design choice to omit
an API client.
Family 20 now closes: 20a renderer + 20b polish + 20d mediator
shipped; 20c rolled into 20a. Three deferred polishes (let-inlining,
print sugar, doc-wrap widow control) wait for corpus signal.
Eleven binary builtins (+ - * / % == != < <= > >=) render as
lhs op rhs when called with two args (tail-flagged keeps prefix).
Standard Rust-aligned 4-level precedence table elides parens.
not(x) renders as !x. Long /// doc strings wrap at 80 cols on
word boundaries.
bench_list_sum: 'if ==(n, 0)' becomes 'if n == 0', '+(acc, h)'
becomes 'acc + h', tail keyword still visible on tail calls.
19 new unit tests (47 total). 4 snapshots (3 re-rendered + 1 new
bench fixture). Public API unchanged.
When a fn-param is annotated (own T) but the body never consumes
it (consume_count == 0) and never destructures it via match, the
linearity pass now emits a Severity::Warning diagnostic with a
form-A-rendered (borrow T) rewrite as suggested_rewrite. Pure
advisory, Decision 10 unchanged.
First Warning-severity diagnostic the typechecker emits; three
CLI exit paths gated on Error-only.
JOURNAL: design entry + iter 19a shipping entry.
Three sibling sites previously fell back to shallow `ailang_rc_dec`
when a binder had non-empty `moved_slots` and a dynamic runtime ctor
tag: Iter B Own-param dec at fn-return (lib.rs), Iter A arm-close
pattern-binder dec (match_lower.rs), and emit_inlined_partial_drop's
non-Ctor branch (drop.rs). Shallow freed the outer cell but did not
walk the unmoved ptr fields — silent leak.
Adds `emit_partial_drop_fn_for_type`: a per-ADT helper structurally
parallel to `emit_drop_fn_for_type` but taking an i64 moved-slots
bitmask. Each ptr-field's dec is gated on the corresponding mask
bit; the outer box is dec'd at join. Emitted alongside the per-type
drop fn for every ADT under --alloc=rc.
Three call sites rewritten to resolve the partial_drop symbol from
the binder's static type and build the mask from `moved_slots`.
Three RED-then-GREEN fixtures (rc_own_param_/rc_match_arm_/
rc_app_let_partial_drop_leak) pin the carve-outs in regression
coverage; live cell counts go from 3/1/3 (pre-fix) to 0/0/0.
e2e count: 66 → 69.
The pre-existing `alloc_rc_own_param_dec_at_fn_return` IR-shape
assertion is widened to accept `partial_drop_<m>_<T>(%arg_xs,
i64 mask)` as a third valid drop shape (the canonical fu2 case).
Closes the JOURNAL queue's only remaining iter; the "wait for
organic fixture" stance from the 18g tidy is retracted — known
leaks are bugs, CLAUDE.md's TDD rule covers them autonomously.
Records the four-phase reorganisation that moved 2470 lines
out of lib.rs into purpose-named sibling submodules. Captures
the visibility model (descendant-module privacy + selective
pub(crate)), the deliberate non-extractions (lower_term, the
app/call cluster), and the framing as a navigability tidy
rather than a feature-driven change.
Closes two of the three queued debts from the 18g tidy in the
same session. Tag-conditional partial-drop helper stays queued
— substantial, no observable fixture, all current fallback paths
cleanly use shallow ailang_rc_dec.
The 18-arc + 18g sub-arc + 18g tidy + post-tidy follow-ups are
all closed. Decision 9 has been re-framed to match the
orchestrator's dual-allocator stance (commit f10a77e). Next iter
is queue-driven; nothing load-bearing is open.
User's stance on the retirement question raised in 18f / 18g.2:
keep the dual-allocator setup. Boehm = default, RC = validated
alternative ready to flip. Re-opening trigger is operational
(maintenance cost), not performance.
Concrete consequences recorded: no default flip, -lgc stays, and
Decision 9's 'transitional' framing should be re-cast as 'dual
allocator' in a follow-up DESIGN.md edit (queued). Decision 10
holds as the canonical real-time path. The three RC-side debts
from the 18g tidy remain queued and don't block the dual stance.
Resolves the architect's drift report on the 18g sub-arc:
- DESIGN.md: ratify mode-metadata's codegen role. param_modes /
ret_mode were already on Type::Fn since 18a; with 18d.4 / 18g
they became load-bearing for drop-emission decisions in
codegen. The new 'Mode metadata is load-bearing for codegen'
subsection records the four seams (Iter A + Iter B + 18g.1 +
18g.2) and names the let-alias-of-borrow carve-out the gates
do not propagate through.
- bench/run.sh: post-throughput, invoke bench/latency_harness.py
on the three canonical arms (Implicit @ gc, explicit @ rc,
Implicit @ rc control). The Boehm-retirement bench numbers
are now reproducible by anyone running the harness, not just
by hand on a specific host.
- Negative coverage: examples/rc_let_implicit_returning_app
+ alloc_rc_let_binder_for_implicit_returning_app_does_not_drop
pin the asymmetry to the (own)-ret-mode test (live=0 there,
live=1 here). The Borrow-ret-mode case is covered by the
language design itself — typechecker rejects 'borrow-
passthrough' shapes with consume-while-borrowed.
JOURNAL entry records four items as deferred known debt:
emit_inlined_partial_drop dynamic-tag fallback, carve-out
diagnostics surface, bench-number stat-of-N, and the
cross-family ordering observation about CLAUDE.md's tidy-iter
rule.
The 18-arc is formally closed with this tidy. Next iter is
the orchestrator's Boehm-retirement decision (Path A vs Path B
from JOURNAL 2026-05-08 18f entry, joined by the post-18g.2
re-bench numbers).
Re-runs the latency bench post-18g.2 and records the result:
RC p99 = 311.6 µs (1.37× median), max = 348.2 µs (1.53× median);
Boehm p99 = 7249 µs (69.65× median), max = 7923 µs. RC is 23×
better on p99 and 23× better on max. RSS: RC 42 MB vs Boehm
66 MB — RC is now LOWER RSS, not higher. Median throughput: RC
is 2.18× slower (alloc-path tax).
Decision 10's real-time commitment has its evidence base. Boehm
retirement is now an orchestrator call between (A) 'retire,
accept the throughput tax until a slab allocator is built' and
(B) 'build the slab allocator first, retire after'. No commit
yet flips defaults.
Documents the 18-arc as complete on the correctness property:
explicit-mode RC reports live=0 on the canonical bench fixture
(allocs=11068575 frees=11068575). Remaining open work is
orchestrator-territory or queued (tag-conditional partial-drop,
let-alias-aware mode propagation, tidy-iter).
Records the diagnosis (18f.2 bencher finding -> tail-call drop
elision in match arms whose body consumes pattern-binders into
the tail-call), the new pre-tail-call seam in lower_match, the
opt-in rc-stats counter test infrastructure (18g.0), the TDD
record, and the end-to-end verification on bench_latency_explicit
(11M -> 1M live cells; the 1M residue is the persistent Tree
cache and queues as 18g.2).
Bencher ran the tail-latency bench post-RC-fix. Boehm's p99=7131µs
is 74× its median (96µs); explicit-mode RC's p99=453µs is 1.62× its
median (280µs). Decision 10's real-time claim has a first piece of
evidence on the correct axis (tail latency) — 18f's throughput
bench measured the wrong thing.
Surprise finding: explicit-mode RC leaks at the same RSS rate as
implicit-mode RC (511MB peak on both). The (own ...) +
(reuse-as) + (drop-iterative) annotations don't drive deallocation
in tail-recursive arms because pattern-binder t is consumed into
the tail-call (consume_count > 0, Iter A skips); the outer LCons
husk has all its ptr fields moved out but no drop site shallow-decs
it. Genuine 18d.4 implementation gap, queued as Iter 18g (#82).
Retirement decision (Path A vs Path B from 18f) still open; now
joined by the outer-cell-shallow-dec finding. Both feed the
eventual decision.
Iter B (Own-param dec at fn return) gates on param_modes[i] ==
ParamMode::Own and skips Implicit/Borrow because those modes carry
no caller-handed-off-ownership signal. Iter A (arm-close pattern-
binder dec) is the same shape — pattern-binders loaded out of a
scrutinee owe their drop-validity to the scrutinee's ownership —
but Iter A had no such gate and fired on every ptr-typed binder
with consume_count == 0.
Concrete failure (regression test): pin (params t) is Implicit,
caller loop holds t and re-passes it to itself. pin's (TNode v l r)
arm dec'd l and r, fragmenting the tree the caller still references.
Next recursion tripped ailang_rc_dec underflow.
Fix: thread current_param_modes onto the emitter, set it in
emit_fn from the fn type's param_modes (and reset/save it across
lambda thunk emission). In lower_match, derive scrutinee_is_owned
from the scrutinee's mode (Own only; let-binders and temps are
treated as owned) and skip Iter A when not owned.
Carve-out: a let-alias of a non-Own param is not yet detected;
the regression doesn't trigger it and a propagation pass belongs
in its own iter. Recorded in JOURNAL.
Red: alloc_rc_pattern_bind_in_implicit_fn_does_not_dec_borrowed_children.
Pre-fix the rc binary aborted with refcount underflow; post-fix
matches alloc=gc ("0").
Records the bench numbers (rc/bump 2.5–2.9x on the Implicit-mode
fixtures, missing Decision 10's 1.3x retirement target) and the
two paths forward — lower the threshold (Path A, pragmatic; rc
and gc are within 5% of each other so retirement under "RC ≤
Boehm ± 5%" qualifies) vs build a slab/pool allocator first
(Path B, larger iter, closes most of the gap to bump).
The 1.3x criterion was orchestrator-level commitment in
Decision 10 and not meeting it is a prompt for an orchestrator-
level discussion, not for the implementer to autonomously flip
the default. 18f deliberately stops without flipping default,
without dropping -lgc, without marking Decision 9 historical.
Records the schema additions (TypeDef.drop_iterative, serde-skip
when false), why heap-stretchy-buffer worklist was chosen over
slot-repurposing (not every box has a free ptr-slot to thread
through), the mono-typed worklist trade-off (simpler loop body,
small cross-type cascade penalty), the four new ABI symbols in
runtime/rc.c, and the validation against Decision 10's brief.
Validation note: hand-run confirmed the 1M-cell fixture with the
annotation exits 0; same fixture without the annotation SIGSEGVs
(exit 139). The annotation is load-bearing, not cosmetic.
Closes with the dispatch line for 18f (RC validation bench +
Boehm retirement decision) and reaffirms the post-18f tidy-iter
will be ailang-architect-driven drift cleanup over the whole
18-arc surface.
Records the two emission seams (pattern-binder at arm close,
Own-param at fn return), why they fit one iter (uniform
emission shape), the explicit dynamic-tag partial-drop fallback
to shallow ailang_rc_dec when moved_slots is non-empty for a
dynamic-runtime-ctor binder, the side effect on reuse_as_demo's
sum_list (correctness improvement, stdout unchanged, existing
assertions still hold), and the closing of 18d.3's regression.
Notes that 18d.3 + 18d.4 together close the move-aware-pattern
story for the static case; dynamic-tag partial-drop is the
remaining hygiene gap and is structurally orthogonal (would
exist even without moves). 18e (worklist) may be the natural
place to also close the dynamic-tag fallback.
Records strategy (b) (codegen-side bookkeeping, no source
mutation), why strategy (a) was retired (borrow contracts +
desugar re-scrutinise), the moved_slots side table and its
consumers (Term::Let close + reuse arm), and the narrow
memory-hygiene regression on borrow_only_recursive_list_drop
that 18d.4 will close. Also flags 18d.3 as the first iter to
ship with an undetected (no valgrind in CI) memory-hygiene
regression — accepted with explicit follow-up iter, not
papered over.
Records the runtime refcount-1 dispatch design (Lean 4 / Roc
lineage), why the dispatch lives at codegen rather than in
runtime/rc.c (per-site decision, no new ABI), the new
reuse_shape pre-codegen pass with its five reason sub-codes,
and — crucially — the explicit field-cleanup scope cut. The
reuse arm does not dec old pointer-typed fields because the
canonical fixture's pattern-bound subterms alias the new field
values (`(reuse-as xs (Cons _ (map_inc t)))` returns t's
in-place-rewritten box as the new tail). The root cause is
upstream: pattern-matching does not null out source slots. 18d.2
ships the perf-only half (alloc + outer cascade elided);
18d.3 / 18e closes the leak with move-aware patterns.
Records the schema floor (Term::ReuseAs wrapper, identity
codegen), the three diagnostics that close the
schema-permits-meaningless-forms gap (reuse-as-non-allocating-
body, reuse-as-source-not-bare-var, use-after-consume), and the
"source-not-bare-var rule lives in linearity, not typecheck"
design point — that constraint is a use-rule about reuse
semantics, not a type-rule, so it sits in the linearity pass
where it's gated on the all-explicit-mode activation. Closes
with the dispatch note for 18d.2 (in-place rewrite under
--alloc=rc, plus reuse-as-shape-mismatch diagnostic).
Records the drop-symbol scheme (uniform @drop_<m>_<T> for every
ADT, even no-boxed-children ones; pair+env drop fns for escaping
closures), field_drop_call's per-type dispatch and the three
fall-back cases (Str / Fn / Type::Var), the unchanged Term::Let
emission gates, and the explicit deferral of stack-recursion to
18e's worklist allocator. Also lists the four known-debt items
(stack-recursive drop, closure-typed ADT fields, Type::Var
fields, fn parameters) so 18d's design surface is clear.
Records the inference's max-over-paths semantics, the codegen
gates (consume_count == 0, body-tail-not-binder,
block-not-terminated, non-escape membership), the @-prefix elide
on closure-pair globals, the rc_box_drop fixture's choice of a
no-boxed-children ADT (shallow free sufficient), and the explicit
scope cut for per-type drop fn (18c.4). Closes with the dispatch
note for 18c.4 and the reminder that the new CLAUDE.md tidy-iter
rule fires after 18f when the 18-arc closes.
Records what shipped in 18c.2 (activation gate, two diagnostic
codes, position model, the new parse_term / term_to_form_a
round-trip in ailang-surface, gate on clean-typecheck), the
three known false negatives that are deliberate scope cuts
(missing-consume-of-Own, lam captures conservative, pattern
bindings start fresh), and the dispatch for 18c.3 (uniqueness
inference + codegen inc/dec — the iter where RC actually starts
collecting memory).
User correction during the post-RC-overnight check-in: the 18a
"Type::Fn metadata vs. new Type variant" call was justified
across DESIGN.md, JOURNAL, and the commit message primarily by
"avoids ~250 match-arm sites". That is an observation about the
current state of the code, not a design rationale.
CLAUDE.md gains two binding rules:
- Design rationale ≠ implementation effort. Effort is at most a
tiebreaker; a choice whose only stated reason is effort is
suspect. The rule names the 18a misstep as the canonical
anti-example so future sessions catch it earlier.
- Direction freedom + bounce-back conditions inlined (was
previously a cross-reference to a private auto-memory file
outside the repo, which the user couldn't see).
DESIGN.md Decision 10's Schema-additions block now leads with
the substantive reasons for per-position metadata: semantic
locality (modes belong to fn-parameter positions, not to types
in general — Decision 1 line); compositional clarity (type
identity vs. calling convention factor apart); future-proofing
(per-position metadata generalises; Type-variant approach
combinatoric blows up). The match-arm count remains parenthetical,
explicitly named a tiebreaker.
JOURNAL records the correction itself — the mistake stays as a
data point because how design discipline corrupts is informative.
Records that 18c.1 ran zero-surprise (mechanical recursion at
~10 sites across 6 files), the future-work seam in codegen for
18c.3 is a single line (emit one rc_inc call before returning
the inner SSA reg), and 18c.2's linearity check should start
as 'green for borrow_own_demo' since that fixture's shape is
exactly what the check needs to accept.
Records that the codegen change was a one-line extension because
the 18a bump path had centralised the allocator-symbol decision
on fn_name(). Documents the runtime ABI (8-byte refcount header,
ailang_rc_alloc/inc/dec) and the deliberate no-inc/dec posture:
programs leak intentionally under --alloc=rc; 18c lights up the
actual reference counting.
Next-step plan for 18c is split into three sub-iters
(clone schema / linearity check / inference + codegen) since
each wants its own verification cycle.
Records the schema-choice rationale (per-position metadata on
Type::Fn vs new Type variants — the latter would be a 250-site
mechanical migration, the former is one iter), the
padding-on-read trick that kept construction sites unchanged,
and the forward-compat note on the new fixture's borrow+own
sharing pattern.
Replaces the earlier "RC + inference, no annotations" position
with a two-layer architecture: inference (intra-fn) + mandatory
LLM-author mode annotations on fn signatures (inter-fn). Adds
structured rationale for rejecting region inference (regions
assume stack-shaped lifetimes; real programs need
non-stack-shaped lifetimes for caches / memo tables) and linear
types as primary mechanism.
Schema additions enumerated for the 18-series:
Type::Borrow / Type::Own (Iter 18a)
Term::Clone (Iter 18c)
Term::ReuseAs (Iter 18d)
TypeDef.drop_iterative (Iter 18e)
Migration plan extended from 18a-d (4 iters) to 18a-f (6 iters).
JOURNAL records the regions excursion, the scope-shape reduction,
and the LLM-author lever as the structural advantage Decision 10
cashes in on.
Adds --alloc=<gc|bump> to ail build/run. Bump path links a 256MB
no-free arena C stub instead of libgc; IR is byte-identical except
for the @GC_malloc → @bump_malloc symbol swap. Bench harness times
two allocation-heavy workloads (list cons/sum and balanced tree
build/walk) under both modes.
Numbers (RUNS=5, median of 4):
bench_list_sum gc 0.141s bump 0.048s +194%
bench_tree_walk gc 0.103s bump 0.041s +151%
Bucket: large. ~60% of runtime is Boehm on these workloads —
upper bound for any realistic program. Both fixtures hold the
heap fully live, so the cost we're seeing is Boehm's allocate
path itself, not collection work; that fact narrows the design
space for the GC discussion.
- crates/ailang-codegen: AllocStrategy enum, three callsites and
the IR header parameterised.
- crates/ail/src/main.rs: --alloc flag plumbed; bump runtime
located + compiled on demand.
- runtime/bump.c: 256MB static arena, abort-on-overflow.
- examples/bench_list_sum, bench_tree_walk: accumulator-form
fixtures (textbook recursive sum was constructor-blocked).
- bench/run.sh: harness with Python timing helper (Arch's
/usr/bin/time isn't part of the base install).
No language-level changes; default --alloc=gc, all 141 workspace
tests green, all 5 IR snapshots unchanged, 11 prior fixtures
produce identical stdout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a conservative escape analysis that flags Term::Ctor and
Term::Lam allocations as non-escaping when they are bound by a
let, never flow to a fn arg / ctor field / tail-return / lambda
capture, and remain inside the allocating fn's frame. Codegen
lowers flagged sites to LLVM `alloca` instead of `@GC_malloc`;
escaping sites continue to use Boehm.
- escape.rs (new): name-based taint propagation; let-only
candidates; tail-position / app-arg / ctor-field / lam-capture
all taint.
- codegen: three sites switched (lower_ctor, lam env, closure
pair). Save/restore non_escape across lambda thunk emit.
- examples/escape_local_demo.{ailx,ail.json}: focused fixture
exercising both branches (peek and recursive count).
- e2e + escape unit tests: 133 → 141 (+8).
Stop-gate: 17a closes the queue. Memory-management discussion
(GC scope) is the next user-driven step. Per-fn arena observations
appended to JOURNAL — including the headline finding that 0/270
shipped allocations are flagged non-escaping under this rule
(real AILang threads ctors directly between fns; build-locally /
consume-locally is not idiomatic).
All existing fixture stdouts and content hashes bit-identical;
IR snapshots unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lifts the Int-only restriction on `==`. Declared type becomes
forall a. Fn(a, a) -> Bool; codegen monomorphises and dispatches:
Int → icmp eq i64
Bool → icmp eq i1
Str → call @strcmp + icmp eq i32 0
Unit → constant true (operands still emitted for side effects)
ADT / Fn / other → CodegenError::Internal
This unblocks 16c's build_eq for non-Int lit patterns. == joins
__unreachable__ as the second polymorphic builtin (same Forall
machinery).
- check/builtins.rs: == registered as Forall(a, Fn(a, a) -> Bool).
- codegen: lower_eq dispatch table; @strcmp declared in IR header
alongside @printf/@GC_malloc/@puts.
- examples/eq_demo.{ailx,ail.json}: covers all four supported
scalars including a Str-lit-pattern match.
- IR snapshots refreshed: only +declare i32 @strcmp(ptr, ptr) in
the header; every define body bit-identical.
- e2e + check + codegen tests: 124 → 133 (+9, of which +1 is the
e2e fixture and the rest exercise the new dispatch / typecheck
surface).
Other comparison ops (<, <=, >, >=, !=) remain Int-only — out of
scope for this iter.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Eliminates the Unit-typed chain default that 16c left in place.
Adds __unreachable__ : forall a. a — codegen lowers to LLVM
unreachable. The 16a/16c chain machinery now uses it as the
deepest fall-through, so exhaustive matches with non-Unit arms
no longer need a (case _ ...) workaround.
- check/builtins.rs: register __unreachable__ as Forall(a, a).
- codegen: Term::Var "__unreachable__" emits LLVM unreachable
and sets block_terminated; If/Match/fn-body gate downstream
work on that flag.
- desugar: chain default switched from Lit{Unit} to Var.
- examples/lit_pat: categorize_first's trailing _ arm removed.
Hash changes from c4faec3abc2ed388 to 644de0c0ec15fc17.
- examples/unreachable_demo: positive fixture using safe_div
with __unreachable__ in the impossible branch.
- e2e + desugar tests: 122 → 124 (+2).
Path-a from 16b.2 planning: real primitive over a desugar-time
exhaustiveness pre-check (which would need cross-module type
registry access from desugar).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Inner LetRec capturing outer LetRec's params now lifts cleanly via
the existing post-order traversal — outer's params are KnownType
in inner's outer-scope, so 16b.2's fast-path handles them. Inner
LetRec capturing outer's NAME remains rejected (closure-of-self
in body, deferred separately).
Plus hardening: lift.rs's deferred path tracks the outer LetRec
name in a parallel `enclosing_letrec_names` set so the symmetric
inner-captures-outer-name rejection fires there too — without it
a silent lift produced a runtime arity mismatch.
- examples/nested_let_rec.{ailx,ail.json}: outer counts down via
inner that captures the outer's threshold param.
- desugar.rs: clarified panic message for outer-name capture.
- lift.rs: enclosing_letrec_names tracking.
- e2e + desugar tests: 120 → 122 (+2).
End-of-16b series: feature-complete except for closure-of-self
in body and closure-with-polymorphism, both queued separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lifts the monomorphic-only restriction. Lifted top-level fn now
becomes Forall(α..., Fn(t..., T...) -> tr) when the enclosing fn
is polymorphic; capture types may mention outer type vars and
the existing mono pipeline (Iter 12b/14a) specializes them
correctly at call sites.
- desugar.rs / lift.rs: scope-building unified — Forall fn-params
are now KnownType, not LetBound. Lift wraps augmented_ty in
Forall mirroring the enclosing fn.
- Name-as-value-in-in-term in a polymorphic enclosing fn is
rejected (eta-Lam wrap from 16b.5 has no Forall). Queued
separately as closure-conversion-with-polymorphism.
- examples/poly_rec_capture.{ailx,ail.json}: apply_n_times :
Forall(a). drives at Int and Bool to exercise two mono
instantiations.
- e2e + check + desugar tests: 116 → 120 (+4).
Codegen pipeline untouched — apply_subst_to_type substitutes
through Fn.params uniformly (original params + appended captures).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the direct-call-only restriction on the LetRec name when it
appears in `in_term`. No new ABI: the lift wraps the in-term in
`(let f (lam P (app f$lr_N P CAPTURES)) in_term')` so codegen's
existing Iter-8b closure-pair machinery handles it. Body-position
name-as-value still rejected (deferred — eta-of-self is harder).
- desugar.rs / lift.rs: split find_non_callee_use into body
(panic) and in_term (wrap).
- examples/local_rec_as_value.{ailx,ail.json}: no-capture path
(factorial passed to apply5 → 120).
- examples/local_rec_as_value_capture.{ailx,ail.json}: capture
path; lifted fn has captures, eta-Lam picks them up via 8b.
- e2e + desugar tests: 113 → 116 (+3).
Codegen Lam machinery handled the wrapped form on the first try.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Flips the desugar's MatchArm-capture rejection from panic to defer.
The 16b.3 lift_letrecs pass already handled match-arm bindings via
type_check_pattern_for_lift (Pattern::Var, Pattern::Ctor with sub-
patterns, ctor-field substitution against scrutinee args), so this
iter is a single-line classification change in desugar plus the
fixture and tests that exercise it.
- desugar.rs: MatchArm classification now defers (same arm as
LetBound); EnclosingLetRec panic remains for 16b.7.
- examples/local_rec_match_capture.{ailx,ail.json}: enclosing fn
pattern-matches on Pair<Int,Int>, inner LetRec captures both
match-arm bindings simultaneously. Lifts to
loop$lr_0(i: Int, threshold: Int, n: Int) -> Int.
- e2e + check + desugar tests: 110 → 113 (+3).
First fixture lifting more than one capture; subst_call_with_extras
already handled it generically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brummel