bench: 21'e — cross-language reference, AILang/C ratios
Closes the question CLAUDE.md has carried since day one ("LLVM-
linkable, performance is extremely important") with data. Hand-C
variants of the four bench fixtures, compiled with clang -O2,
each carefully matching the AILang algorithm and explicitly
documenting representation differences (cell width, leak policy)
that affect the ratio.
Three substantive findings:
1. Pure-compute parity with C: bench_compute_collatz runs at
AILang/C = 0.99x across both allocators. AILang's IR composes
with LLVM's optimizer at the same level a hand-C source does.
This is the LLVM-linkable performance claim, backed by data
for the first time. bench_compute_intsum (1.05-1.18x) confirms.
2. AILang bump beats glibc malloc 2x on linear allocation:
bench_list_sum.bump/c = 0.50x. Bump's two-instruction inline
fastpath outperforms glibc's free-list-managed malloc on
no-free workloads. Quantitatively measured for the first time.
3. RC overhead vs C malloc quantified: bench_list_sum.rc/c =
1.49x, bench_tree_walk.rc/c = 2.61x. The 8-byte refcount
header + zero-init + libc backing add 50-160% over glibc
malloc on these implicit-mode workloads. Explicit-mode + a
free()-adding C variant (21'f, queued) will close the
apples-to-apples gap on dec-cost.
CLAUDE.md updated to list bench/cross_lang.py as the third
tidy-iter gate alongside bench/check.py and bench/compile_check.py.
20 new metrics in bench/baseline_cross_lang.json with 12-15%
tolerances (cross-language ratios are inherently noisier than
within-AILang ratios — two compiler stacks contribute variance).
This commit is contained in:
+120
@@ -10135,6 +10135,126 @@ fixture and baseline-file additions only.
|
||||
- **Family 21+** — typeclasses, polymorphic ADTs at runtime,
|
||||
pattern-binding generalisation. Orchestrator-level fork.
|
||||
|
||||
## 2026-05-09 — Iter 21'e: cross-language reference + AILang/C ratios
|
||||
|
||||
Closes the question CLAUDE.md has carried since day one — *"the
|
||||
language must, in the end, be linkable to LLVM. Performance is
|
||||
extremely important."* — by adding hand-C variants of the bench
|
||||
corpus, building both with `clang -O2`, and comparing wall times
|
||||
directly. Until this iter, every performance number AILang shipped
|
||||
was internal (gc vs. bump vs. rc); none of them said anything
|
||||
about absolute competitiveness.
|
||||
|
||||
### Hand-C corpus
|
||||
|
||||
`bench/reference/` — four C sources, one per fixture, each
|
||||
carefully matching the AILang algorithm and explicitly
|
||||
documenting representation choices (cell width, leak policy)
|
||||
that affect the ratio:
|
||||
|
||||
- **`list_sum.c`** — linked list, malloc-and-leak (matches
|
||||
AILang implicit-mode RC). 16-byte cell vs. AILang's 24-byte
|
||||
ICons (tag overhead).
|
||||
- **`tree_walk.c`** — balanced tree, malloc-and-leak. 24-byte
|
||||
cell vs. AILang's 32-byte Tree::Node. NULL leaves (no alloc)
|
||||
vs. AILang's tag-only Leaf cells.
|
||||
- **`compute_intsum.c`** — pure-compute affine recurrence.
|
||||
- **`compute_collatz.c`** — pure-compute, data-dependent control
|
||||
flow.
|
||||
|
||||
### Headline numbers (5-run, drop-slowest, median of 4)
|
||||
|
||||
```
|
||||
fixture | AILang_rc | AILang_bump | C | rc/c | bump/c
|
||||
-----------------------+-----------+-------------+--------+-------+-------
|
||||
bench_list_sum | 141.6 ms | 48.0 ms | 95.3 ms| 1.49× | 0.50×
|
||||
bench_tree_walk | 97.0 ms | 38.9 ms | 37.2 ms| 2.61× | 1.05×
|
||||
bench_compute_intsum | 0.4 ms | 0.4 ms | 0.4 ms| 1.18× | 1.05×
|
||||
bench_compute_collatz | 56.9 ms | 56.6 ms | 57.5 ms| 0.99× | 0.98×
|
||||
```
|
||||
|
||||
### Three substantive findings
|
||||
|
||||
**1. Pure-compute parity with C is real.** `bench_compute_collatz`
|
||||
runs at AILang/C = 0.98–0.99× across both allocators. Same
|
||||
algorithm, same `clang -O2`, same wall time. The IR AILang's
|
||||
codegen emits composes with LLVM's optimizer at the same level
|
||||
a hand-written C source does — both tail-recursive iteration,
|
||||
both data-dependent branch prediction. This is the canonical
|
||||
"LLVM-linkable, performance is extremely important" claim,
|
||||
backed by data for the first time. `bench_compute_intsum` (1.05–
|
||||
1.18×) confirms the pattern; both fixtures get LLVM-folded /
|
||||
optimized symmetrically.
|
||||
|
||||
**2. AILang bump beats glibc malloc on linear allocation.**
|
||||
`bench_list_sum.bump/c = 0.50×` — AILang's `bump_malloc`
|
||||
(`ptr += size; return old`) is twice as fast as glibc's
|
||||
`malloc()` on dense Cons-cell allocation. Expected
|
||||
qualitatively (bump is 2 instructions inline; glibc malloc has
|
||||
free-list management even on the alloc path), but the
|
||||
quantitative result is the first time it's been measured. No-
|
||||
free workloads (bench fixtures) are exactly where bump shines;
|
||||
production workloads that actually free are a separate story.
|
||||
|
||||
**3. RC overhead vs C malloc is now quantified.** Linear:
|
||||
`bench_list_sum.rc/c = 1.49×` — RC's per-call cost (8-byte
|
||||
refcount header + zero-init + libc malloc backing) is ~50%
|
||||
above glibc malloc on this workload. Tree: 2.61×, larger
|
||||
because the per-node fixed cost amortizes over a smaller
|
||||
working set. These ratios are *implicit-mode* RC (no dec-tax);
|
||||
explicit-mode would add the dec-cost on top, but the hand-C
|
||||
reference also has no free, so the apples-to-apples comparison
|
||||
needs an explicit-mode AILang fixture + a free()-adding C
|
||||
variant to be honest about both sides. Queued.
|
||||
|
||||
### 20 new baselined metrics
|
||||
|
||||
`bench/baseline_cross_lang.json` — 4 fixtures × 5 metrics
|
||||
(ail_rc_s, ail_bump_s, c_s, rc_over_c, bump_over_c). Tolerances
|
||||
12–15% across the board: cross-language ratios are inherently
|
||||
less stable than within-AILang ratios because two compiler
|
||||
stacks contribute noise.
|
||||
|
||||
### CLAUDE.md update
|
||||
|
||||
`Performance regressions` now lists three tidy-iter gates:
|
||||
`bench/check.py`, `bench/compile_check.py`, and
|
||||
`bench/cross_lang.py`. The cross-lang script is the
|
||||
heaviest of the three (12 binary builds + 60 timed runs at
|
||||
n=5), but it's the only mechanism that catches AILang/C ratios
|
||||
drifting upward over time. Worth the seconds.
|
||||
|
||||
### What this iter does NOT do
|
||||
|
||||
- **No explicit-mode bench fixture pair.** `bench_list_sum`
|
||||
and `bench_tree_walk` are implicit-mode-only; the C reference
|
||||
is malloc-and-leak. To honestly compare RC's full alloc+dec
|
||||
cost vs. C's full malloc+free cost would need a paired
|
||||
explicit-mode AILang fixture + a `free()`-adding C variant.
|
||||
Queued as 21'f.
|
||||
- **No multi-platform reference.** Single x86-64 Linux
|
||||
measurement. Cross-platform ratios may differ; not in scope.
|
||||
- **No JIT comparison.** `clang -O2` AOT, AILang AOT — apples
|
||||
to apples. JIT (LuaJIT, V8, etc.) is a different question.
|
||||
|
||||
### Test state
|
||||
|
||||
288 / 0 / 3, unchanged.
|
||||
|
||||
### JOURNAL queue (updated)
|
||||
|
||||
- **21'f — explicit-mode cross-lang pair.** Add `bench_list_sum_
|
||||
explicit.ailx` (with `(borrow)` / `(own)` / `(drop-iterative)`)
|
||||
and `list_sum_explicit_free.c` (matching `free()` calls).
|
||||
Re-run cross_lang, capture rc-with-dec / c-with-free ratios.
|
||||
Closes the apples-to-apples gap on the dec-cost axis.
|
||||
- **`FnDef::synthetic(...)` factor-out** — unchanged.
|
||||
- **Boehm full retirement** — unchanged.
|
||||
- **Latency methodology upgrade** (n=10+ captures) — unchanged.
|
||||
- **Deferred richer integration paths** (from 20f) — unchanged.
|
||||
- **Family 21+** — typeclasses, polymorphic ADTs at runtime,
|
||||
pattern-binding generalisation. Orchestrator-level fork.
|
||||
|
||||
## 2026-05-09 — Iter 21'd: pure-compute fixtures + harness hardening
|
||||
|
||||
Closes a third bench-corpus blind spot: every fixture so far has
|
||||
|
||||
Reference in New Issue
Block a user