Brummel/AILang
1
0
Fork 0
Code Issues 29 Pull Requests Actions Packages Projects Releases Wiki Activity

spec: heap-str-abi — amend, drop sentinel-rc-header story

Browse Source 
hs.2's implementer-orchestrator BLOCKED with an empirical finding: the iter 18d.3 move-tracking and iter 18b non-escape lowering together prevent static-Str pointers from ever reaching ailang_rc_inc/_dec along any shipping execution path. The previously-proposed runtime sentinel guard and the i64 -1 sentinel slot in static-Str globals were both dead by construction.

Amendment is subtractive: remove the runtime/rc.c guard from §Architecture/Runtime layer + §Components/runtime/rc.c (none needed), shrink static-Str globals from <{i64, i64, [N+1 x i8]}> to <{i64, [N+1 x i8]}> (saves 8 bytes per literal), update GEP indices (i32 0, i32 1 → i32 0, i32 0), drop the §Error-handling 'Sentinel collision' subsection, drop the proposed str_field_in_adt_drops_static_str_noop safety-gate test from §Testing strategy (vacuous against natural fixtures), reframe §Architecture/Codegen-4 as 'codegen-level elision invariant, no runtime guard'. Goal/consumer-ABI claims tightened to 'unified along consumer paths, producer + RC sides differ'.

Acceptance: re-dispatched grounding-check PASS. The amendment leaves hs.1 (committed at c56498a) needing a forward-fix retrofit — that is the new hs.2's scope, planned next.
This commit is contained in:
Brummel
2026-05-12 17:34:41 +02:00
parent 8ca602dc1f
commit 2a72a4ad68
1 changed files with 119 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specs/2026-05-12-heap-str-abi.md
+119 -91
View File
@@ -1,7 +1,7 @@
# Heap-`Str` ABI — Design Spec
**Date:** 2026-05-12
**Status:** Draft — awaiting Step 7.5 grounding-check
**Date:** 2026-05-12 (amended same day after hs.2 empirical finding)
**Status:** Amended — awaiting re-dispatched Step 7.5 grounding-check
**Authors:** Brummel (orchestrator) + Claude
## Goal
@@ -9,8 +9,15 @@
Add a second realisation of `Str` values — `malloc`-backed,
reference-counted heap slabs — alongside the existing static
`@.str_*` globals, *without* enlarging the language surface.
From the LLM-author's view, `Str` remains a single type; from
the IR caller's view, a single ABI handles both realisations.
From the LLM-author's view, `Str` remains a single type. The
two realisations share the **consumer ABI** (pointer-to-len-field,
`+8` for bytes), so `@puts` / `@ail_str_eq` / `@ail_str_compare` /
`@strcmp` work uniformly on either. The producer side differs
(static via constexpr GEP into a `.rodata` packed-struct global;
heap via `ailang_int_to_str` / `ailang_float_to_str`), and the RC
discipline applies only to heap-Str — static-Str pointers never
flow into `ailang_rc_inc` / `ailang_rc_dec` along any shipping
execution path. Codegen invariant, not runtime guard.
The milestone exists to unblock two builtins that are already
visible at the type level but currently broken at codegen:
@@ -84,11 +91,14 @@ the length into the first 8 bytes of the payload, and returns
the payload pointer. The caller fills bytes at offset 8 and the
`NUL` at offset `8 + len`.
`runtime/rc.c` `ailang_rc_inc` and `ailang_rc_dec` gain a
**sentinel-header fast path**: when `*header_of(payload) == UINT64_MAX`,
both return immediately. This is the discriminator between
"heap-allocated, real refcount" and "static literal, treat as
immortal".
`runtime/rc.c` is **not modified** in this milestone. The
runtime's RC primitives only ever see heap-Str pointers (real
`rc_header` at `payload - 8`, real refcount). Static-Str pointers
do not flow into `ailang_rc_inc` / `ailang_rc_dec` along any
shipping execution path — current codegen elides those calls
through two orthogonal optimisations (`emit_inlined_partial_drop`
move-tracking from iter 18d.3 + iter 18b non-escape lowering).
This is a codegen-level invariant; no runtime guard backs it up.
### Codegen layer (`crates/ailang-codegen/src/lib.rs`)
@@ -96,12 +106,19 @@ Four change-points.
1. **Static-Str globals** (`intern_string`, ~`crates/ailang-codegen/src/lib.rs:2487`):
emitted not as `[N+1 x i8] c"…\00"` but as a packed struct
`<{i64, i64, [N+1 x i8]}>` with values `<{i64 -1, i64 N, c"…\00"}>`.
The callsite pointer is `getelementptr` to the second field
of this struct (= where the `len` slot starts), not to the
slab start. The pointer that flows through the IR as a `Str`
value points at the *payload* (= len-field), exactly mirroring
the heap-Str case.
`<{i64, [N+1 x i8]}>` with values `<{i64 N, c"…\00"}>`. The
first field is the explicit byte length (excluding the
terminating `NUL`); the array tail carries the bytes and the
`NUL`. The callsite pointer is `getelementptr` to the first
field of this struct (= where the `len` slot starts). The
pointer that flows through the IR as a `Str` value points at
the *len-field*; for heap-Str, the analogous pointer points at
the `len`-field of the malloc-backed payload (with the real
`rc_header` at `payload - 8`). The two layouts share the
*consumer ABI* (len at offset 0, bytes at offset 8) but
physically differ in what sits at `payload - 8` — heap-Str has
the rc_header there, static-Str has whatever the linker placed
before the global (which codegen guarantees is never read).
2. **`float_to_str` lowering** (~`crates/ailang-codegen/src/lib.rs:1829`):
the `CodegenError::Internal` fork is replaced by
@@ -111,25 +128,29 @@ Four change-points.
emits `call ptr @ailang_int_to_str(i64 %a)`.
4. **`drop_<m>_<T>` walks** (per-type drop fns) — no code change.
`Str` fields are already included in the dec walk today:
`field_drop_call` at `crates/ailang-codegen/src/drop.rs:372`
maps `Type::Con { name: "Str", .. }` to bare `ailang_rc_dec`.
The comment at that site explicitly justifies the routing on
the assumption that *"Str payloads are NUL-terminated bytes in
static memory; nothing to recurse into"* — exactly the
assumption that breaks once heap-Str exists. With the current
static-only path, dropping an ADT field of type `Str` therefore
already calls `ailang_rc_dec(static_str_ptr)`, which reads
`header_of(payload) = payload - 8` — bytes that fall *outside*
the static `[N x i8]` global. This is latent undefined
behaviour today; no shipping test exercises an ADT-with-Str-
field under `--alloc=rc`, so it has not surfaced. The
sentinel-header check proposed in (item 1 above implicitly +
the `runtime/rc.c` change below) turns that latent UB into a
defined no-op: with the new packed-struct global, `payload - 8`
lands on a valid `i64` slot whose value is `UINT64_MAX`, and
`ailang_rc_dec` short-circuits. **The change here is in
correctness, not in code: no edit to `drop.rs` is required.**
`Str` fields are dispatched today via `field_drop_call` at
`crates/ailang-codegen/src/drop.rs:372`, which routes
`Type::Con { name: "Str", .. }` to bare `ailang_rc_dec`. The
comment at that site justifies the routing on the (now stale)
assumption that *"Str payloads are NUL-terminated bytes in
static memory; nothing to recurse into"* — true for static-Str,
correct-by-coincidence for heap-Str (heap-Str's rc_header is
exactly the `rc_header` shape `ailang_rc_dec` expects).
Empirically the dispatch site is **dead code along all
shipping execution paths** due to two orthogonal codegen
optimisations: `emit_inlined_partial_drop` (iter 18d.3) marks
match-arm-extracted `Str` slots as "already moved" and skips
their decs; non-escape lowering (iter 18b) stack-allocates
ADTs that never leave their fn frame, eliminating the
per-type drop call entirely. We rely on these elisions as the
*codegen-level* invariant that static-Str payload pointers
never reach `ailang_rc_dec`. If either elision were lost, the
call would read undefined bytes at `payload - 8` (the static
global has no `rc_header` slot at that offset; the linker
places arbitrary bytes there) and likely segfault — a loud,
locatable failure mode the testing
discipline would catch immediately. No runtime guard backs up
the invariant; the codegen-level guarantee is the protection.
The existing extern declarations `@strcmp`, `@ail_str_eq`,
`@ail_str_compare`, `@puts`, `@printf` stay. They still expect a
@@ -186,9 +207,10 @@ Add:
### `runtime/rc.c`
Modify `ailang_rc_inc` and `ailang_rc_dec`: after the existing
`NULL` guard, add `if (*header_of(payload) == UINT64_MAX) return;`
to short-circuit on the sentinel.
Unchanged. The runtime's RC primitives only see heap-Str
pointers along shipping execution paths; static-Str pointers
are gated out at codegen via the elisions described in
§Architecture / Codegen layer item 4.
### `crates/ailang-codegen/src/lib.rs`
@@ -196,10 +218,11 @@ Modify:
- `intern_string` (~`:2487`) — switch the LLVM-IR emission of
static-Str globals from `[N+1 x i8] c"…\00"` to
`<{i64, i64, [N+1 x i8]}> <{i64 -1, i64 N, c"…\00"}>`. The
global name stays the same; the way callsites materialise a
pointer to it changes to a `getelementptr` landing on the
`len`-field.
`<{i64, [N+1 x i8]}> <{i64 N, c"…\00"}>` (single `i64` length
prefix; bytes + `NUL` in the array tail). The global name
stays the same; the way callsites materialise a pointer to it
changes to a `getelementptr` landing on the `len`-field
(= field index 0 of the struct).
- IR-header preamble (~`:455`–`:498`): add
`declare ptr @ailang_int_to_str(i64)` and
`declare ptr @ailang_float_to_str(double)`.
@@ -238,12 +261,15 @@ Add an `int_to_str` arm parallel to the `float_to_str` arm at
"What **is** supported": add `int_to_str : (Int) -> Str` next
to `float_to_str`.
- New top-level subsection "Str ABI" (or under an appropriate
Decision-10 / runtime-contracts anchor): document the unified
slab layout `{rc_header, len, bytes…, NUL}`, the sentinel-
header convention (`UINT64_MAX` = static / immortal), and the
Decision-10 / runtime-contracts anchor): document the heap-Str
slab layout `{rc_header, len, bytes…, NUL}` and the static-Str
global layout `<{len, bytes…, NUL}>` (no rc_header, since the
codegen-level elision invariant guarantees the runtime's RC
primitives never see static-Str pointers). Both layouts share
the consumer ABI "pointer-at-len-field, +8 for bytes". The
inherited limitation that `==` / `<` use `strcmp` semantics
and do not support embedded `\0` bytes (this is a future-
milestone enhancement, not a regression).
and do not support embedded `\0` bytes is noted as a future-
milestone enhancement, not a regression.
## Data flow
@@ -260,11 +286,11 @@ Three representative IR cases pin the after-state.
After the milestone:
```llvm
@.str_M_hello_0 = private unnamed_addr constant <{i64, i64, [6 x i8]}>
<{ i64 -1, i64 5, [6 x i8] c"hello\00" }>
@.str_M_hello_0 = private unnamed_addr constant <{i64, [6 x i8]}>
<{ i64 5, [6 x i8] c"hello\00" }>
…
%payload = getelementptr inbounds <{i64, i64, [6 x i8]}>,
ptr @.str_M_hello_0, i32 0, i32 1
%payload = getelementptr inbounds <{i64, [6 x i8]}>,
ptr @.str_M_hello_0, i32 0, i32 0
%bytes = getelementptr inbounds i8, ptr %payload, i64 8
%1 = call i32 @puts(ptr %bytes)
```
@@ -297,29 +323,35 @@ define i1 @eq__Str(ptr %a, ptr %b) {
}
```
Either operand may be static (sentinel header) or heap (real
header); the body does not distinguish. `@ail_str_eq` keeps its
`strcmp`-based body unchanged.
Either operand may be a static-Str pointer (into a `.rodata`
packed-struct global) or a heap-Str pointer (into a malloc'd
slab); the body does not distinguish, because the consumer ABI
(len at offset 0, bytes at offset 8) is identical and
`@ail_str_eq` only reads the bytes via `strcmp`.
**ADT drop with a `Str` field.** For `type Boxed = | Box(Str)`,
the per-type `drop_M_Boxed(ptr %cell)` already emits a
`call void @ailang_rc_dec(ptr %str_field)` today — `Str` fields
are dispatched to `ailang_rc_dec` by `field_drop_call` regardless
of whether the value originated from a static or a heap source.
What changes is the *meaning* of that call on a static-Str
pointer: pre-milestone it reads bytes outside the `[N x i8]`
global (latent UB; no shipping test exercises the path);
post-milestone it reads the new global's leading `i64 -1` slot,
hits the sentinel short-circuit in `ailang_rc_dec`, and returns
without modifying anything. For a heap-Str field, the normal
refcount drop runs unchanged. The single behavioural shift —
from latent UB to defined no-op on static fields — is gated by
the new `str_field_in_adt_drops_static_str_noop` E2E test (see
§Testing strategy).
the per-type `drop_M_Boxed(ptr %cell)` body in
`crates/ailang-codegen/src/drop.rs` dispatches the `Str` field
through `field_drop_call` to `ailang_rc_dec`. Empirically the
emitted call is **dead code along all shipping execution paths**
due to two orthogonal codegen optimisations:
`emit_inlined_partial_drop` (iter 18d.3) marks match-arm-extracted
`Str` slots as moved-out and skips their decs, and non-escape
lowering (iter 18b) stack-allocates ADTs that never leave the
fn frame, eliminating the per-type drop call entirely. The
codegen never produces an execution path that hands a static-Str
pointer to `ailang_rc_dec`. We rely on this as a codegen-level
invariant — no runtime guard backs it up. If a future codegen
change ever loses one of the two elisions, the resulting
`ailang_rc_dec(static_str_ptr)` call would read undefined memory
at `payload - 8` (the static-Str global has no `rc_header` slot)
and likely segfault — a loud failure that the test suite would
flag immediately, leading the implementer of that codegen change
to inspect the dispatch site.
## Error handling
Four failure modes; most are inherited or structurally impossible.
Three failure modes; most are inherited or structurally impossible.
**Out-of-memory at heap-Str allocation.** `ailang_rc_alloc`
already `abort()`s on OOM, matching Boehm. `str_alloc`,
@@ -336,13 +368,6 @@ the buffer). The defensive check survives format-string changes
that might widen the output without anyone noticing the buffer
is too small.
**Sentinel collision.** A real refcount can never reach
`UINT64_MAX` — `2^64` references would require more memory than
exists on any machine the program can run on. The convention
`rc_header == UINT64_MAX` ⇒ static is structurally safe; no
runtime check beyond the equality test in `inc`/`dec` is needed.
DESIGN.md §"Str ABI" records this as an invariant.
**Embedded `\0` bytes in static-Str literals from JSON.** Status
quo: a literal containing `""` ends up with a `NUL` in the
middle of the slab; `strcmp`-based comparison treats it as the
@@ -369,8 +394,9 @@ Three layers.
Pin the structural guarantees of §"Data flow" with substring
matches against emitted IR:
- `static_str_global_uses_packed_struct_with_sentinel_and_len` —
pins the new slab format.
- `static_str_global_uses_packed_struct_with_len` — pins the
new slab format (single `i64` length prefix, bytes + `NUL` in
the array tail).
- `static_str_callsite_pointer_is_payload_not_slab` — pins the
`getelementptr`-to-second-field convention.
- `eq_str_calls_ail_str_eq_with_bytes_pointer`,
@@ -417,14 +443,15 @@ Catch leaks and double-frees:
- `str_field_in_adt_drops_heap_str_correctly` — a value of
`type Boxed = | Box(Str)` is instantiated with a heap-Str
(from `int_to_str`) and dropped; `live=0`.
- `str_field_in_adt_drops_static_str_noop` — same ADT
instantiated with a static literal under `--alloc=rc`; `live=0`,
no crash. **Load-bearing:** no precursor test today exercises
this path (the existing `alloc_rc_*` tests do not instantiate
an ADT with a `Str` field), so today's behaviour on this path
is undefined. The test gates the milestone-level safety
property that static-Str pointers in ADT fields drop safely
via the sentinel short-circuit.
No `str_field_in_adt_drops_static_str_noop` test ships. An
earlier draft of this spec proposed it as a load-bearing safety
gate for a runtime sentinel short-circuit; empirically the
codegen elides the static-Str-field-drop path entirely via
move-tracking and non-escape lowering, so the test could not be
made non-vacuous against any natural fixture shape. The
elision-based invariant is documented in §Architecture /
Codegen layer item 4 and stands or falls with the elision
itself, not with a runtime test.
### Bench gates
@@ -466,13 +493,14 @@ The milestone closes when all the following are observable.
- `runtime/str.c` exports `ailang_int_to_str(int64_t) -> char*`
and `ailang_float_to_str(double) -> char*`. The private
`str_alloc(uint64_t)` is *not* in the export symbol set.
- `runtime/rc.c` `ailang_rc_inc`/`ailang_rc_dec` short-circuit
on `*header == UINT64_MAX`.
- Static-Str globals are emitted as
`<{i64, i64, [N+1 x i8]}>` with values `{i64 -1, i64 N, c"…\00"}`.
- `runtime/rc.c` is unchanged from pre-milestone.
- Static-Str globals are emitted as `<{i64, [N+1 x i8]}>` with
values `{i64 N, c"…\00"}`.
- DESIGN.md has a "Str ABI" anchor (or equivalent) documenting
the unified layout, sentinel convention, and inherited
`strcmp`-based comparison semantics.
the heap-Str and static-Str layouts, the shared consumer ABI
(len at offset 0, bytes at offset 8), the codegen-level
elision invariant for static-Str-in-RC-paths, and the
inherited `strcmp`-based comparison semantics.
**Observable program behaviour.**