diff --git a/docs/specs/2026-05-12-heap-str-abi.md b/docs/specs/2026-05-12-heap-str-abi.md index 2030c23..7c2be03 100644 --- a/docs/specs/2026-05-12-heap-str-abi.md +++ b/docs/specs/2026-05-12-heap-str-abi.md @@ -1,7 +1,7 @@ # Heap-`Str` ABI — Design Spec -**Date:** 2026-05-12 -**Status:** Draft — awaiting Step 7.5 grounding-check +**Date:** 2026-05-12 (amended same day after hs.2 empirical finding) +**Status:** Amended — awaiting re-dispatched Step 7.5 grounding-check **Authors:** Brummel (orchestrator) + Claude ## Goal @@ -9,8 +9,15 @@ Add a second realisation of `Str` values — `malloc`-backed, reference-counted heap slabs — alongside the existing static `@.str_*` globals, *without* enlarging the language surface. -From the LLM-author's view, `Str` remains a single type; from -the IR caller's view, a single ABI handles both realisations. +From the LLM-author's view, `Str` remains a single type. The +two realisations share the **consumer ABI** (pointer-to-len-field, +`+8` for bytes), so `@puts` / `@ail_str_eq` / `@ail_str_compare` / +`@strcmp` work uniformly on either. The producer side differs +(static via constexpr GEP into a `.rodata` packed-struct global; +heap via `ailang_int_to_str` / `ailang_float_to_str`), and the RC +discipline applies only to heap-Str — static-Str pointers never +flow into `ailang_rc_inc` / `ailang_rc_dec` along any shipping +execution path. Codegen invariant, not runtime guard. The milestone exists to unblock two builtins that are already visible at the type level but currently broken at codegen: @@ -84,11 +91,14 @@ the length into the first 8 bytes of the payload, and returns the payload pointer. The caller fills bytes at offset 8 and the `NUL` at offset `8 + len`. -`runtime/rc.c` `ailang_rc_inc` and `ailang_rc_dec` gain a -**sentinel-header fast path**: when `*header_of(payload) == UINT64_MAX`, -both return immediately. This is the discriminator between -"heap-allocated, real refcount" and "static literal, treat as -immortal". +`runtime/rc.c` is **not modified** in this milestone. The +runtime's RC primitives only ever see heap-Str pointers (real +`rc_header` at `payload - 8`, real refcount). Static-Str pointers +do not flow into `ailang_rc_inc` / `ailang_rc_dec` along any +shipping execution path — current codegen elides those calls +through two orthogonal optimisations (`emit_inlined_partial_drop` +move-tracking from iter 18d.3 + iter 18b non-escape lowering). +This is a codegen-level invariant; no runtime guard backs it up. ### Codegen layer (`crates/ailang-codegen/src/lib.rs`) @@ -96,12 +106,19 @@ Four change-points. 1. **Static-Str globals** (`intern_string`, ~`crates/ailang-codegen/src/lib.rs:2487`): emitted not as `[N+1 x i8] c"…\00"` but as a packed struct - `<{i64, i64, [N+1 x i8]}>` with values `<{i64 -1, i64 N, c"…\00"}>`. - The callsite pointer is `getelementptr` to the second field - of this struct (= where the `len` slot starts), not to the - slab start. The pointer that flows through the IR as a `Str` - value points at the *payload* (= len-field), exactly mirroring - the heap-Str case. + `<{i64, [N+1 x i8]}>` with values `<{i64 N, c"…\00"}>`. The + first field is the explicit byte length (excluding the + terminating `NUL`); the array tail carries the bytes and the + `NUL`. The callsite pointer is `getelementptr` to the first + field of this struct (= where the `len` slot starts). The + pointer that flows through the IR as a `Str` value points at + the *len-field*; for heap-Str, the analogous pointer points at + the `len`-field of the malloc-backed payload (with the real + `rc_header` at `payload - 8`). The two layouts share the + *consumer ABI* (len at offset 0, bytes at offset 8) but + physically differ in what sits at `payload - 8` — heap-Str has + the rc_header there, static-Str has whatever the linker placed + before the global (which codegen guarantees is never read). 2. **`float_to_str` lowering** (~`crates/ailang-codegen/src/lib.rs:1829`): the `CodegenError::Internal` fork is replaced by @@ -111,25 +128,29 @@ Four change-points. emits `call ptr @ailang_int_to_str(i64 %a)`. 4. **`drop__` walks** (per-type drop fns) — no code change. - `Str` fields are already included in the dec walk today: - `field_drop_call` at `crates/ailang-codegen/src/drop.rs:372` - maps `Type::Con { name: "Str", .. }` to bare `ailang_rc_dec`. - The comment at that site explicitly justifies the routing on - the assumption that *"Str payloads are NUL-terminated bytes in - static memory; nothing to recurse into"* — exactly the - assumption that breaks once heap-Str exists. With the current - static-only path, dropping an ADT field of type `Str` therefore - already calls `ailang_rc_dec(static_str_ptr)`, which reads - `header_of(payload) = payload - 8` — bytes that fall *outside* - the static `[N x i8]` global. This is latent undefined - behaviour today; no shipping test exercises an ADT-with-Str- - field under `--alloc=rc`, so it has not surfaced. The - sentinel-header check proposed in (item 1 above implicitly + - the `runtime/rc.c` change below) turns that latent UB into a - defined no-op: with the new packed-struct global, `payload - 8` - lands on a valid `i64` slot whose value is `UINT64_MAX`, and - `ailang_rc_dec` short-circuits. **The change here is in - correctness, not in code: no edit to `drop.rs` is required.** + `Str` fields are dispatched today via `field_drop_call` at + `crates/ailang-codegen/src/drop.rs:372`, which routes + `Type::Con { name: "Str", .. }` to bare `ailang_rc_dec`. The + comment at that site justifies the routing on the (now stale) + assumption that *"Str payloads are NUL-terminated bytes in + static memory; nothing to recurse into"* — true for static-Str, + correct-by-coincidence for heap-Str (heap-Str's rc_header is + exactly the `rc_header` shape `ailang_rc_dec` expects). + Empirically the dispatch site is **dead code along all + shipping execution paths** due to two orthogonal codegen + optimisations: `emit_inlined_partial_drop` (iter 18d.3) marks + match-arm-extracted `Str` slots as "already moved" and skips + their decs; non-escape lowering (iter 18b) stack-allocates + ADTs that never leave their fn frame, eliminating the + per-type drop call entirely. We rely on these elisions as the + *codegen-level* invariant that static-Str payload pointers + never reach `ailang_rc_dec`. If either elision were lost, the + call would read undefined bytes at `payload - 8` (the static + global has no `rc_header` slot at that offset; the linker + places arbitrary bytes there) and likely segfault — a loud, + locatable failure mode the testing + discipline would catch immediately. No runtime guard backs up + the invariant; the codegen-level guarantee is the protection. The existing extern declarations `@strcmp`, `@ail_str_eq`, `@ail_str_compare`, `@puts`, `@printf` stay. They still expect a @@ -186,9 +207,10 @@ Add: ### `runtime/rc.c` -Modify `ailang_rc_inc` and `ailang_rc_dec`: after the existing -`NULL` guard, add `if (*header_of(payload) == UINT64_MAX) return;` -to short-circuit on the sentinel. +Unchanged. The runtime's RC primitives only see heap-Str +pointers along shipping execution paths; static-Str pointers +are gated out at codegen via the elisions described in +§Architecture / Codegen layer item 4. ### `crates/ailang-codegen/src/lib.rs` @@ -196,10 +218,11 @@ Modify: - `intern_string` (~`:2487`) — switch the LLVM-IR emission of static-Str globals from `[N+1 x i8] c"…\00"` to - `<{i64, i64, [N+1 x i8]}> <{i64 -1, i64 N, c"…\00"}>`. The - global name stays the same; the way callsites materialise a - pointer to it changes to a `getelementptr` landing on the - `len`-field. + `<{i64, [N+1 x i8]}> <{i64 N, c"…\00"}>` (single `i64` length + prefix; bytes + `NUL` in the array tail). The global name + stays the same; the way callsites materialise a pointer to it + changes to a `getelementptr` landing on the `len`-field + (= field index 0 of the struct). - IR-header preamble (~`:455`–`:498`): add `declare ptr @ailang_int_to_str(i64)` and `declare ptr @ailang_float_to_str(double)`. @@ -238,12 +261,15 @@ Add an `int_to_str` arm parallel to the `float_to_str` arm at "What **is** supported": add `int_to_str : (Int) -> Str` next to `float_to_str`. - New top-level subsection "Str ABI" (or under an appropriate - Decision-10 / runtime-contracts anchor): document the unified - slab layout `{rc_header, len, bytes…, NUL}`, the sentinel- - header convention (`UINT64_MAX` = static / immortal), and the + Decision-10 / runtime-contracts anchor): document the heap-Str + slab layout `{rc_header, len, bytes…, NUL}` and the static-Str + global layout `<{len, bytes…, NUL}>` (no rc_header, since the + codegen-level elision invariant guarantees the runtime's RC + primitives never see static-Str pointers). Both layouts share + the consumer ABI "pointer-at-len-field, +8 for bytes". The inherited limitation that `==` / `<` use `strcmp` semantics - and do not support embedded `\0` bytes (this is a future- - milestone enhancement, not a regression). + and do not support embedded `\0` bytes is noted as a future- + milestone enhancement, not a regression. ## Data flow @@ -260,11 +286,11 @@ Three representative IR cases pin the after-state. After the milestone: ```llvm -@.str_M_hello_0 = private unnamed_addr constant <{i64, i64, [6 x i8]}> - <{ i64 -1, i64 5, [6 x i8] c"hello\00" }> +@.str_M_hello_0 = private unnamed_addr constant <{i64, [6 x i8]}> + <{ i64 5, [6 x i8] c"hello\00" }> … -%payload = getelementptr inbounds <{i64, i64, [6 x i8]}>, - ptr @.str_M_hello_0, i32 0, i32 1 +%payload = getelementptr inbounds <{i64, [6 x i8]}>, + ptr @.str_M_hello_0, i32 0, i32 0 %bytes = getelementptr inbounds i8, ptr %payload, i64 8 %1 = call i32 @puts(ptr %bytes) ``` @@ -297,29 +323,35 @@ define i1 @eq__Str(ptr %a, ptr %b) { } ``` -Either operand may be static (sentinel header) or heap (real -header); the body does not distinguish. `@ail_str_eq` keeps its -`strcmp`-based body unchanged. +Either operand may be a static-Str pointer (into a `.rodata` +packed-struct global) or a heap-Str pointer (into a malloc'd +slab); the body does not distinguish, because the consumer ABI +(len at offset 0, bytes at offset 8) is identical and +`@ail_str_eq` only reads the bytes via `strcmp`. **ADT drop with a `Str` field.** For `type Boxed = | Box(Str)`, -the per-type `drop_M_Boxed(ptr %cell)` already emits a -`call void @ailang_rc_dec(ptr %str_field)` today — `Str` fields -are dispatched to `ailang_rc_dec` by `field_drop_call` regardless -of whether the value originated from a static or a heap source. -What changes is the *meaning* of that call on a static-Str -pointer: pre-milestone it reads bytes outside the `[N x i8]` -global (latent UB; no shipping test exercises the path); -post-milestone it reads the new global's leading `i64 -1` slot, -hits the sentinel short-circuit in `ailang_rc_dec`, and returns -without modifying anything. For a heap-Str field, the normal -refcount drop runs unchanged. The single behavioural shift — -from latent UB to defined no-op on static fields — is gated by -the new `str_field_in_adt_drops_static_str_noop` E2E test (see -§Testing strategy). +the per-type `drop_M_Boxed(ptr %cell)` body in +`crates/ailang-codegen/src/drop.rs` dispatches the `Str` field +through `field_drop_call` to `ailang_rc_dec`. Empirically the +emitted call is **dead code along all shipping execution paths** +due to two orthogonal codegen optimisations: +`emit_inlined_partial_drop` (iter 18d.3) marks match-arm-extracted +`Str` slots as moved-out and skips their decs, and non-escape +lowering (iter 18b) stack-allocates ADTs that never leave the +fn frame, eliminating the per-type drop call entirely. The +codegen never produces an execution path that hands a static-Str +pointer to `ailang_rc_dec`. We rely on this as a codegen-level +invariant — no runtime guard backs it up. If a future codegen +change ever loses one of the two elisions, the resulting +`ailang_rc_dec(static_str_ptr)` call would read undefined memory +at `payload - 8` (the static-Str global has no `rc_header` slot) +and likely segfault — a loud failure that the test suite would +flag immediately, leading the implementer of that codegen change +to inspect the dispatch site. ## Error handling -Four failure modes; most are inherited or structurally impossible. +Three failure modes; most are inherited or structurally impossible. **Out-of-memory at heap-Str allocation.** `ailang_rc_alloc` already `abort()`s on OOM, matching Boehm. `str_alloc`, @@ -336,13 +368,6 @@ the buffer). The defensive check survives format-string changes that might widen the output without anyone noticing the buffer is too small. -**Sentinel collision.** A real refcount can never reach -`UINT64_MAX` — `2^64` references would require more memory than -exists on any machine the program can run on. The convention -`rc_header == UINT64_MAX` ⇒ static is structurally safe; no -runtime check beyond the equality test in `inc`/`dec` is needed. -DESIGN.md §"Str ABI" records this as an invariant. - **Embedded `\0` bytes in static-Str literals from JSON.** Status quo: a literal containing `""` ends up with a `NUL` in the middle of the slab; `strcmp`-based comparison treats it as the @@ -369,8 +394,9 @@ Three layers. Pin the structural guarantees of §"Data flow" with substring matches against emitted IR: -- `static_str_global_uses_packed_struct_with_sentinel_and_len` — - pins the new slab format. +- `static_str_global_uses_packed_struct_with_len` — pins the + new slab format (single `i64` length prefix, bytes + `NUL` in + the array tail). - `static_str_callsite_pointer_is_payload_not_slab` — pins the `getelementptr`-to-second-field convention. - `eq_str_calls_ail_str_eq_with_bytes_pointer`, @@ -417,14 +443,15 @@ Catch leaks and double-frees: - `str_field_in_adt_drops_heap_str_correctly` — a value of `type Boxed = | Box(Str)` is instantiated with a heap-Str (from `int_to_str`) and dropped; `live=0`. -- `str_field_in_adt_drops_static_str_noop` — same ADT - instantiated with a static literal under `--alloc=rc`; `live=0`, - no crash. **Load-bearing:** no precursor test today exercises - this path (the existing `alloc_rc_*` tests do not instantiate - an ADT with a `Str` field), so today's behaviour on this path - is undefined. The test gates the milestone-level safety - property that static-Str pointers in ADT fields drop safely - via the sentinel short-circuit. +No `str_field_in_adt_drops_static_str_noop` test ships. An +earlier draft of this spec proposed it as a load-bearing safety +gate for a runtime sentinel short-circuit; empirically the +codegen elides the static-Str-field-drop path entirely via +move-tracking and non-escape lowering, so the test could not be +made non-vacuous against any natural fixture shape. The +elision-based invariant is documented in §Architecture / +Codegen layer item 4 and stands or falls with the elision +itself, not with a runtime test. ### Bench gates @@ -466,13 +493,14 @@ The milestone closes when all the following are observable. - `runtime/str.c` exports `ailang_int_to_str(int64_t) -> char*` and `ailang_float_to_str(double) -> char*`. The private `str_alloc(uint64_t)` is *not* in the export symbol set. -- `runtime/rc.c` `ailang_rc_inc`/`ailang_rc_dec` short-circuit - on `*header == UINT64_MAX`. -- Static-Str globals are emitted as - `<{i64, i64, [N+1 x i8]}>` with values `{i64 -1, i64 N, c"…\00"}`. +- `runtime/rc.c` is unchanged from pre-milestone. +- Static-Str globals are emitted as `<{i64, [N+1 x i8]}>` with + values `{i64 N, c"…\00"}`. - DESIGN.md has a "Str ABI" anchor (or equivalent) documenting - the unified layout, sentinel convention, and inherited - `strcmp`-based comparison semantics. + the heap-Str and static-Str layouts, the shared consumer ABI + (len at offset 0, bytes at offset 8), the codegen-level + elision invariant for static-Str-in-RC-paths, and the + inherited `strcmp`-based comparison semantics. **Observable program behaviour.**