bench: 21'e — cross-language reference, AILang/C ratios

Closes the question CLAUDE.md has carried since day one ("LLVM-
linkable, performance is extremely important") with data. Hand-C
variants of the four bench fixtures, compiled with clang -O2,
each carefully matching the AILang algorithm and explicitly
documenting representation differences (cell width, leak policy)
that affect the ratio.

Three substantive findings:

1. Pure-compute parity with C: bench_compute_collatz runs at
   AILang/C = 0.99x across both allocators. AILang's IR composes
   with LLVM's optimizer at the same level a hand-C source does.
   This is the LLVM-linkable performance claim, backed by data
   for the first time. bench_compute_intsum (1.05-1.18x) confirms.

2. AILang bump beats glibc malloc 2x on linear allocation:
   bench_list_sum.bump/c = 0.50x. Bump's two-instruction inline
   fastpath outperforms glibc's free-list-managed malloc on
   no-free workloads. Quantitatively measured for the first time.

3. RC overhead vs C malloc quantified: bench_list_sum.rc/c =
   1.49x, bench_tree_walk.rc/c = 2.61x. The 8-byte refcount
   header + zero-init + libc backing add 50-160% over glibc
   malloc on these implicit-mode workloads. Explicit-mode + a
   free()-adding C variant (21'f, queued) will close the
   apples-to-apples gap on dec-cost.

CLAUDE.md updated to list bench/cross_lang.py as the third
tidy-iter gate alongside bench/check.py and bench/compile_check.py.
20 new metrics in bench/baseline_cross_lang.json with 12-15%
tolerances (cross-language ratios are inherently noisier than
within-AILang ratios — two compiler stacks contribute variance).
This commit is contained in:
2026-05-09 01:15:37 +02:00
parent 5a4a6de031
commit c897d2eef0
8 changed files with 678 additions and 2 deletions
+120
View File
@@ -10135,6 +10135,126 @@ fixture and baseline-file additions only.
- **Family 21+** — typeclasses, polymorphic ADTs at runtime,
pattern-binding generalisation. Orchestrator-level fork.
## 2026-05-09 — Iter 21'e: cross-language reference + AILang/C ratios
Closes the question CLAUDE.md has carried since day one — *"the
language must, in the end, be linkable to LLVM. Performance is
extremely important."* — by adding hand-C variants of the bench
corpus, building both with `clang -O2`, and comparing wall times
directly. Until this iter, every performance number AILang shipped
was internal (gc vs. bump vs. rc); none of them said anything
about absolute competitiveness.
### Hand-C corpus
`bench/reference/` — four C sources, one per fixture, each
carefully matching the AILang algorithm and explicitly
documenting representation choices (cell width, leak policy)
that affect the ratio:
- **`list_sum.c`** — linked list, malloc-and-leak (matches
AILang implicit-mode RC). 16-byte cell vs. AILang's 24-byte
ICons (tag overhead).
- **`tree_walk.c`** — balanced tree, malloc-and-leak. 24-byte
cell vs. AILang's 32-byte Tree::Node. NULL leaves (no alloc)
vs. AILang's tag-only Leaf cells.
- **`compute_intsum.c`** — pure-compute affine recurrence.
- **`compute_collatz.c`** — pure-compute, data-dependent control
flow.
### Headline numbers (5-run, drop-slowest, median of 4)
```
fixture | AILang_rc | AILang_bump | C | rc/c | bump/c
-----------------------+-----------+-------------+--------+-------+-------
bench_list_sum | 141.6 ms | 48.0 ms | 95.3 ms| 1.49× | 0.50×
bench_tree_walk | 97.0 ms | 38.9 ms | 37.2 ms| 2.61× | 1.05×
bench_compute_intsum | 0.4 ms | 0.4 ms | 0.4 ms| 1.18× | 1.05×
bench_compute_collatz | 56.9 ms | 56.6 ms | 57.5 ms| 0.99× | 0.98×
```
### Three substantive findings
**1. Pure-compute parity with C is real.** `bench_compute_collatz`
runs at AILang/C = 0.980.99× across both allocators. Same
algorithm, same `clang -O2`, same wall time. The IR AILang's
codegen emits composes with LLVM's optimizer at the same level
a hand-written C source does — both tail-recursive iteration,
both data-dependent branch prediction. This is the canonical
"LLVM-linkable, performance is extremely important" claim,
backed by data for the first time. `bench_compute_intsum` (1.05
1.18×) confirms the pattern; both fixtures get LLVM-folded /
optimized symmetrically.
**2. AILang bump beats glibc malloc on linear allocation.**
`bench_list_sum.bump/c = 0.50×` — AILang's `bump_malloc`
(`ptr += size; return old`) is twice as fast as glibc's
`malloc()` on dense Cons-cell allocation. Expected
qualitatively (bump is 2 instructions inline; glibc malloc has
free-list management even on the alloc path), but the
quantitative result is the first time it's been measured. No-
free workloads (bench fixtures) are exactly where bump shines;
production workloads that actually free are a separate story.
**3. RC overhead vs C malloc is now quantified.** Linear:
`bench_list_sum.rc/c = 1.49×` — RC's per-call cost (8-byte
refcount header + zero-init + libc malloc backing) is ~50%
above glibc malloc on this workload. Tree: 2.61×, larger
because the per-node fixed cost amortizes over a smaller
working set. These ratios are *implicit-mode* RC (no dec-tax);
explicit-mode would add the dec-cost on top, but the hand-C
reference also has no free, so the apples-to-apples comparison
needs an explicit-mode AILang fixture + a free()-adding C
variant to be honest about both sides. Queued.
### 20 new baselined metrics
`bench/baseline_cross_lang.json` — 4 fixtures × 5 metrics
(ail_rc_s, ail_bump_s, c_s, rc_over_c, bump_over_c). Tolerances
1215% across the board: cross-language ratios are inherently
less stable than within-AILang ratios because two compiler
stacks contribute noise.
### CLAUDE.md update
`Performance regressions` now lists three tidy-iter gates:
`bench/check.py`, `bench/compile_check.py`, and
`bench/cross_lang.py`. The cross-lang script is the
heaviest of the three (12 binary builds + 60 timed runs at
n=5), but it's the only mechanism that catches AILang/C ratios
drifting upward over time. Worth the seconds.
### What this iter does NOT do
- **No explicit-mode bench fixture pair.** `bench_list_sum`
and `bench_tree_walk` are implicit-mode-only; the C reference
is malloc-and-leak. To honestly compare RC's full alloc+dec
cost vs. C's full malloc+free cost would need a paired
explicit-mode AILang fixture + a `free()`-adding C variant.
Queued as 21'f.
- **No multi-platform reference.** Single x86-64 Linux
measurement. Cross-platform ratios may differ; not in scope.
- **No JIT comparison.** `clang -O2` AOT, AILang AOT — apples
to apples. JIT (LuaJIT, V8, etc.) is a different question.
### Test state
288 / 0 / 3, unchanged.
### JOURNAL queue (updated)
- **21'f — explicit-mode cross-lang pair.** Add `bench_list_sum_
explicit.ailx` (with `(borrow)` / `(own)` / `(drop-iterative)`)
and `list_sum_explicit_free.c` (matching `free()` calls).
Re-run cross_lang, capture rc-with-dec / c-with-free ratios.
Closes the apples-to-apples gap on the dec-cost axis.
- **`FnDef::synthetic(...)` factor-out** — unchanged.
- **Boehm full retirement** — unchanged.
- **Latency methodology upgrade** (n=10+ captures) — unchanged.
- **Deferred richer integration paths** (from 20f) — unchanged.
- **Family 21+** — typeclasses, polymorphic ADTs at runtime,
pattern-binding generalisation. Orchestrator-level fork.
## 2026-05-09 — Iter 21'd: pure-compute fixtures + harness hardening
Closes a third bench-corpus blind spot: every fixture so far has