From 9a609aecc7e7e4bc8b758b5b59501dba361e3c93 Mon Sep 17 00:00:00 2001 From: Brummel Date: Mon, 18 May 2026 17:53:13 +0200 Subject: [PATCH] =?UTF-8?q?roadmap:=20close=20Embedding=20ABI=20=E2=80=94?= =?UTF-8?q?=20M2=20([~]->[x]);=20WhatsNew?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit M2 (per-thread runtime context + concurrency safety) fully closed: spec 1c58055 (grounding-check PASS 9/9) -> plan b3388c8 (Boss-corrected) -> iter c9a84b3 (PARTIAL 4/9 + Boss spec-defect repair) + fbeeade (DONE 5-9) -> audit ad88dec (CLEAN-after-tidy; bench exit 1 causally exonerated, NO ratify) -> tidy a80d495 (doc-honesty lockstep rename). No fieldtest (zero authoring-surface change). Sanitiser-verified coherent stop. WhatsNew entry mirrors the done-state notify text verbatim. Next: Embedding ABI — M3 (a new milestone, no spec — a /boss new-milestone bounce-back). --- docs/WhatsNew.md | 10 +++++++ docs/roadmap.md | 70 ++++++++++++++++++++++++++++++------------------ 2 files changed, 54 insertions(+), 26 deletions(-) diff --git a/docs/WhatsNew.md b/docs/WhatsNew.md index e297f09..251fabe 100644 --- a/docs/WhatsNew.md +++ b/docs/WhatsNew.md @@ -214,3 +214,13 @@ The canonical specification and the prose-roundtrip guide have been swept so the To keep it that way, the specification now carries a short rule, in its own words, stating the honesty standard it holds itself to, and there is an automatic check that fails the build if a removed wishful-or-backward-looking phrasing ever creeps back in. The reviewer that guards the specification against drift was taught the same rule, so new violations get flagged going forward rather than only the ones cleaned up now. Nothing about how programs are written, type-checked, compiled, or run changed — this milestone touched documentation only. It was reviewed clean and the one flagged benchmark metric is, once again, the same pre-existing machine-drift item already on the list, proven unrelated here by before-and-after binaries that are byte-for-byte identical. + +## 2026-05-18 — Compiled kernels can now be called safely from many host threads at once + +A compiled AILang function can now be called safely from many host threads at the same time. The previous step made a single compiled numeric function callable from a C or Rust host, one caller at a time; this step makes that boundary swarm-safe — each host worker thread gets its own isolated runtime state, so a whole pool of threads can drive the same compiled kernel concurrently with no shared mutable state and no data race. + +This is verified, not asserted: the kernels run clean under a thread-race checker across many threads at once, and a deliberately-broken shared-state variant is correctly caught by that same checker, so the safety guarantee has teeth rather than being a claim on paper. + +Nothing about how programs are written changed — the source you write is byte-for-byte the same as before; only the generated calling boundary and its runtime became concurrency-safe, which is exactly the point: thread-safety here is a property of the compiled artefact, not a burden on the author. The build was reviewed clean; the one flagged benchmark number is, once again, the same pre-existing machine-drift item already on the list, proven unrelated here. + +The next milestone lets structured records — not just single numbers — cross that boundary with a frozen, stable memory layout. That is a fresh piece of work and a natural point to decide how to pick it up. diff --git a/docs/roadmap.md b/docs/roadmap.md index 3cd7188..dd1c785 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -218,32 +218,50 @@ work progresses. finding [spec_gap]#2; `docs/journals/2026-05-18-iter-emit-ir-staticlib.md`. -- [~] **\[milestone\]** Embedding ABI — M2: per-thread runtime - context + concurrency safety. `ailang_ctx_new()/_free` threaded - through every call; neutralise the two process-global hazards - the runtime read surfaced — the non-atomic - `g_rc_alloc_count` / `g_rc_free_count` counters + `atexit` - stats hook (`runtime/rc.c:86–98`), and the allocator backing - (toward pure per-thread RC, no shared Boehm collector; aligns - with the standing P2 "Boehm full retirement" todo). Still - scalar-only so any failure is attributable to concurrency, not - marshalling. IO-at-the-boundary (`io/print_str` via a - per-instance sink) explicitly deferred here: pure - `(State,Chunk)->State` kernels have no effects; revisit only - if a logging kernel appears — deliberately not a separate - roadmap entry, by feature-acceptance discipline. Coherent stop: - swarm-safe scalar kernels, tsan-clean under N threads. - - depends on: Embedding ABI — M1. - - context: 2026-05-18 chat; runtime hazard read - (`runtime/rc.c` global stats + `atexit`). - Spec'd + user-approved 2026-05-18 (grounding-check PASS 9/9): - `docs/specs/2026-05-18-embedding-abi-m2.md` — ctx as mandatory - explicit C-ABI param (TLS-forwarder, internal conv + `_adapter`/ - `_clos` byte-untouched); ctx near-empty by discipline - (`{alloc,free}_count` only); hazards neutralised in the swarm - artefact, `g_rc_*`+atexit retained as the null-ctx executable - fallback; staticlib RC-only enforced; no schema/surface change. - Plan next. +- [x] **\[milestone\]** Embedding ABI — M2: per-thread runtime + context + concurrency safety — CLOSED 2026-05-18. A compiled + scalar AILang kernel is now callable concurrently from a host + thread swarm without an RC-runtime data race. `ailang_ctx_t*` + is a mandatory leading parameter of the (M3-frozen-shape) C + `@(ctx, scalars…)` ABI; the M1 forwarder publishes it via + a `__thread` slot around the **byte-unchanged** internal + `@ail__` call (internal convention + `_adapter`/`_clos` + untouched — M1 decision held; TLS sound because the kernel call + is synchronous, ctx never held across a suspension). `runtime/rc.c` + de-globalised: the two RC-counter sites became `if (ctx) + ctx->{alloc,free}_count++ else g_rc_*++`, the `g_rc_*` statics + + `atexit` retained verbatim as the null-ctx single-threaded + executable fallback (executable-path leak readback byte-preserved). + ctx near-empty by discipline (`{alloc,free}_count` only — no + arena/lock/atomic/IO; the build-ahead trap named out-of-scope). + Staticlib is RC-only enforced (`--emit=staticlib` rejects + `--alloc=gc|bump`; Boehm never linked into a swarm artefact; + standing P2 Boehm-retirement untouched). No authoring-surface / + schema / `ailang-check` change — the `.ail` is byte-identical to + M1; the deliverable is a swarm-safe generated ABI + de-globalised + runtime, **sanitiser-verified** (per-thread-ctx scalar swarm + tsan-clean == oracle; direct rc-accounting per-ctx clean; + shared-ctx negative control a genuine `ThreadSanitizer` race at + the de-globalised counters, exit 66 — teeth non-vacuous). Process + note: the first implement dispatch correctly BLOCKED on a genuine + spec defect (a negative control on the non-allocating scalar + swarm is structurally impossible — the spec's own honesty point); + Boss adjudicated a spec-consistency repair (teeth relocated to the + rc-accounting harness) rather than a brainstorm bounce. spec + `1c58055` (grounding-check PASS 9/9) → plan `b3388c8` + (Boss-corrected) → iter `c9a84b3` (PARTIAL 4/9 + Boss repair) + + `fbeeade` (DONE 5–9); milestone-close `audit` `ad88dec` (architect + one `[medium]`+`[low]` doc-honesty drift; bench `check.py` exit 1 + decisively causally exonerated — bencher H0-refuted, all 3 firings + the two tracked-P2 noise families, rc.c hunk branchless+free, NO + ratify) → tidy `a80d495` (the `## Embedding ABI` lockstep rename, + drift resolved). No fieldtest (zero authoring-surface change). + Stays here briefly for context; remove once stale (full record in + `docs/journals/INDEX.md`). + - context: `docs/specs/2026-05-18-embedding-abi-m2.md`; + `docs/journals/2026-05-18-iter-embedding-abi-m2.1.md`; + `docs/journals/2026-05-18-audit-embedding-abi-m2.md`; + `docs/journals/2026-05-18-iter-embedding-abi-m2.tidy.md`. - [ ] **\[milestone\]** Embedding ABI — M3: frozen value layout + single ADT/record crossing + RC ownership contract. Host