AILang/docs/specs/0019-heap-str-abi.md

Heap-Str ABI — Design Spec

Date: 2026-05-12 (amended same day after hs.2 empirical finding) Status: Amended — awaiting re-dispatched Step 7.5 grounding-check Authors: Brummel (orchestrator) + Claude

Goal

Add a second realisation of Str values — malloc-backed, reference-counted heap slabs — alongside the existing static @.str_* globals, without enlarging the language surface. From the LLM-author's view, Str remains a single type. The two realisations share the consumer ABI (pointer-to-len-field, +8 for bytes), so @puts / @ail_str_eq / @ail_str_compare / @strcmp work uniformly on either. The producer side differs (static via constexpr GEP into a .rodata packed-struct global; heap via ailang_int_to_str / ailang_float_to_str), and the RC discipline applies only to heap-Str — static-Str pointers never flow into ailang_rc_inc / ailang_rc_dec along any shipping execution path. Codegen invariant, not runtime guard.

The milestone exists to unblock two builtins that are already visible at the type level but currently broken at codegen:

• float_to_str : (Float) -> Str is type-installed today but raises CodegenError::Internal on call (crates/ailang-codegen/src/lib.rs:1829) because there is no runtime-allocated Str path.
• int_to_str : (Int) -> Str is a natural counterpart that is not even type-installed today; its absence forces the LLM-author into awkward Bool-based formatting workarounds whenever an Int needs to flow into print.

Both ship together; both exercise the new heap-allocation path identically (produce a fresh refcounted slab containing runtime-formatted bytes).

Out of scope (each its own follow-up milestone or todo):

• ++ (Str concatenation operator). Would exercise the consume-and-produce heap-Str path; the operator does not exist in the language yet and needs its own feature-shape decision (surface operator vs. builtin fn vs. typeclass method).
• Show typeclass and print-rewire. Depends on this milestone per the roadmap (P1 entry "Post-22 Prelude — Show + print rewire").
• Length-aware ==/< semantics that handle embedded \0 bytes correctly. The status quo (@strcmp/@ail_str_eq/ @ail_str_compare) carries forward unchanged.

Feature-acceptance check

Per DESIGN.md §"Feature-acceptance criterion":

1. An LLM author naturally produces code that uses it. Yes. int_to_str(42) is the canonical way to convert an integer into a printable representation; float_to_str is already announced in DESIGN.md as a Float builtin. Both surface in real LLM-authored programs whenever a numeric value needs to reach io/print_str.

2. It measurably improves correctness or removes redundancy. Yes. Today float_to_str typechecks but fails codegen with a structured internal error; this milestone makes it a working path. int_to_str removes the redundant Bool-formatting workaround needed today.

Neither criterion rests on ergonomic appeal. The milestone ships infrastructure that turns two announced-but-broken builtins into working ones.

Architecture

The milestone adds work in three coordinated layers without enlarging the type-surface.

Runtime layer (runtime/str.c, runtime/rc.c)

Two new extern functions in runtime/str.c:

• char *ailang_int_to_str(int64_t n)
• char *ailang_float_to_str(double x)

Both format their input into a stack buffer, allocate a heap slab of matching size via a private static char *str_alloc(uint64_t len) helper, copy the bytes (plus terminating NUL), and return the payload pointer.

str_alloc itself calls ailang_rc_alloc(8 + len + 1), writes the length into the first 8 bytes of the payload, and returns the payload pointer. The caller fills bytes at offset 8 and the NUL at offset 8 + len.

runtime/rc.c is not modified in this milestone. The runtime's RC primitives only ever see heap-Str pointers (real rc_header at payload - 8, real refcount). Static-Str pointers do not flow into ailang_rc_inc / ailang_rc_dec along any shipping execution path — current codegen elides those calls through two orthogonal optimisations (emit_inlined_partial_drop move-tracking from iter 18d.3 + iter 18b non-escape lowering). This is a codegen-level invariant; no runtime guard backs it up.

Codegen layer (crates/ailang-codegen/src/lib.rs)

Four change-points.

1. Static-Str globals (intern_string, ~crates/ailang-codegen/src/lib.rs:2487): emitted not as [N+1 x i8] c"…\00" but as a packed struct <{i64, [N+1 x i8]}> with values <{i64 N, c"…\00"}>. The first field is the explicit byte length (excluding the terminating NUL); the array tail carries the bytes and the NUL. The callsite pointer is getelementptr to the first field of this struct (= where the len slot starts). The pointer that flows through the IR as a Str value points at the len-field; for heap-Str, the analogous pointer points at the len-field of the malloc-backed payload (with the real rc_header at payload - 8). The two layouts share the consumer ABI (len at offset 0, bytes at offset 8) but physically differ in what sits at payload - 8 — heap-Str has the rc_header there, static-Str has whatever the linker placed before the global (which codegen guarantees is never read).

2. float_to_str lowering (~crates/ailang-codegen/src/lib.rs:1829): the CodegenError::Internal fork is replaced by call ptr @ailang_float_to_str(double %a).

3. int_to_str lowering (new): parallel to float_to_str, emits call ptr @ailang_int_to_str(i64 %a).

4. drop_<m>_<T> walks (per-type drop fns) — no code change. Str fields are dispatched today via field_drop_call at crates/ailang-codegen/src/drop.rs:372, which routes Type::Con { name: "Str", .. } to bare ailang_rc_dec. The comment at that site justifies the routing on the (now stale) assumption that "Str payloads are NUL-terminated bytes in static memory; nothing to recurse into" — true for static-Str, correct-by-coincidence for heap-Str (heap-Str's rc_header is exactly the rc_header shape ailang_rc_dec expects). Empirically the dispatch site is dead code along all shipping execution paths due to two orthogonal codegen optimisations: emit_inlined_partial_drop (iter 18d.3) marks match-arm-extracted Str slots as "already moved" and skips their decs; non-escape lowering (iter 18b) stack-allocates ADTs that never leave their fn frame, eliminating the per-type drop call entirely. We rely on these elisions as the codegen-level invariant that static-Str payload pointers never reach ailang_rc_dec. If either elision were lost, the call would read undefined bytes at payload - 8 (the static global has no rc_header slot at that offset; the linker places arbitrary bytes there) and likely segfault — a loud, locatable failure mode the testing discipline would catch immediately. No runtime guard backs up the invariant; the codegen-level guarantee is the protection.

The existing extern declarations @strcmp, @ail_str_eq, @ail_str_compare, @puts, @printf stay. They still expect a NUL-terminated bytes pointer, which is not the same as the payload pointer that flows through the IR (the payload pointer points at the len field; bytes start 8 bytes later). The three codegen sites that today call these C functions on a Str value (compare__Str, eq__Str, io/print_str — at crates/ailang-codegen/src/lib.rs:2152, 2262, 2313) emit a getelementptr i8, ptr %s, i64 8 immediately before the call to land on the bytes pointer.

Checker layer (crates/ailang-check/src/builtins.rs)

int_to_str : (Int) -> Str is added to the builtin signature table next to the existing float_to_str entry (crates/ailang-check/src/builtins.rs:194). The unit test install_int_to_str_signature mirrors the existing install_float_to_str_signature (line 445).

No mode annotations, no effects — like int_to_float, this is a pure conversion.

What this milestone does not change

• The Type::Str variant, the Literal::Str JSON encoding, the Form-A / Form-B mapping, the prose serialisation.
• The uniqueness inferencer. Str is immutable; no Own/Bor modes apply.
• The @strcmp-based comparison semantics. Embedded \0 bytes in a Str value continue to cause comparison to stop early; this is the status quo, and the milestone explicitly inherits it.

Components

File-by-file inventory of the change surface.

runtime/str.c

Add:

• static char *str_alloc(uint64_t len) — private helper. Calls ailang_rc_alloc(8 + len + 1), writes len at payload offset 0, returns payload pointer.
• char *ailang_int_to_str(int64_t n) — extern. snprintf into a 64-byte stack buffer with format "%lld"; defensively asserts no truncation (the longest i64 is 20 chars, the buffer is 64); calls str_alloc with the resulting length; memcpy's bytes + writes the NUL; returns the payload pointer.
• char *ailang_float_to_str(double x) — extern. Same shape with format "%g". %g matches io/print_float's existing format for IEEE consistency; NaN/±Inf render via libc default.

runtime/rc.c

Unchanged. The runtime's RC primitives only see heap-Str pointers along shipping execution paths; static-Str pointers are gated out at codegen via the elisions described in §Architecture / Codegen layer item 4.

crates/ailang-codegen/src/lib.rs

Modify:

• intern_string (~:2487) — switch the LLVM-IR emission of static-Str globals from [N+1 x i8] c"…\00" to <{i64, [N+1 x i8]}> <{i64 N, c"…\00"}> (single i64 length prefix; bytes + NUL in the array tail). The global name stays the same; the way callsites materialise a pointer to it changes to a getelementptr landing on the len-field (= field index 0 of the struct).
• IR-header preamble (~:455–:498): add declare ptr @ailang_int_to_str(i64) and declare ptr @ailang_float_to_str(double).
• float_to_str arm (~:1829): drop the Err(CodegenError::Internal(…)), emit the call instead.
• New arm parallel to it for int_to_str.
• compare__Str, eq__Str, io/print_str codegen sites (~:2152, :2262, :2313): emit a +8 getelementptr to land on the bytes pointer before calling @ail_str_compare, @ail_str_eq, @puts respectively.
• Per-type drop walks are unchanged. Str is already routed to ailang_rc_dec by field_drop_call (crates/ailang-codegen/src/drop.rs:372). The stale comment at that site ("NUL-terminated bytes in static memory; nothing to recurse into") becomes incorrect once heap-Str ships and must be updated to reflect the new dual-realisation reality — but the dispatched symbol (ailang_rc_dec) is already correct.

crates/ailang-check/src/builtins.rs

Add int_to_str to the builtin signatures table (~:194, mirroring float_to_str's entry), and add the type-installation unit test parallel to install_float_to_str_signature.

crates/ailang-codegen/src/synth.rs

Add an int_to_str arm parallel to the float_to_str arm at :167 so the codegen's local type-replay knows the signature.

docs/DESIGN.md

• §"Float semantics", last paragraph: remove the "codegen lowering deferred" caveat on float_to_str.
• §"What is not (yet) supported" and the builtin inventory under "What is supported": add int_to_str : (Int) -> Str next to float_to_str.
• New top-level subsection "Str ABI" (or under an appropriate Decision-10 / runtime-contracts anchor): document the heap-Str slab layout {rc_header, len, bytes…, NUL} and the static-Str global layout <{len, bytes…, NUL}> (no rc_header, since the codegen-level elision invariant guarantees the runtime's RC primitives never see static-Str pointers). Both layouts share the consumer ABI "pointer-at-len-field, +8 for bytes". The inherited limitation that == / < use strcmp semantics and do not support embedded \0 bytes is noted as a future- milestone enhancement, not a regression.

Data flow

Three representative IR cases pin the after-state.

Static string literal. Today, "hello" is emitted as:

@.str_M_hello_0 = private unnamed_addr constant [6 x i8] c"hello\00"
…
%1 = call i32 @puts(ptr @.str_M_hello_0)

After the milestone:

@.str_M_hello_0 = private unnamed_addr constant <{i64, [6 x i8]}>
                  <{ i64 5, [6 x i8] c"hello\00" }>
…
%payload = getelementptr inbounds <{i64, [6 x i8]}>,
                                  ptr @.str_M_hello_0, i32 0, i32 0
%bytes   = getelementptr inbounds i8, ptr %payload, i64 8
%1       = call i32 @puts(ptr %bytes)

%payload is the value that flows through the IR as a Str. The %bytes adjustment is local to the @puts callsite.

Heap-allocated string from int_to_str(42).

%s = call ptr @ailang_int_to_str(i64 42)
; …print:
%bytes = getelementptr inbounds i8, ptr %s, i64 8
%1     = call i32 @puts(ptr %bytes)
; …at scope close:
call void @ailang_rc_dec(ptr %s)

%s is a fresh heap slab with rc_header == 1. ailang_rc_dec accesses the header at %s - 8, decrements to zero, frees.

eq Str with mixed-origin operands.

define i1 @eq__Str(ptr %a, ptr %b) {
  %a_bytes = getelementptr inbounds i8, ptr %a, i64 8
  %b_bytes = getelementptr inbounds i8, ptr %b, i64 8
  %r       = call zeroext i1 @ail_str_eq(ptr %a_bytes, ptr %b_bytes)
  ret i1 %r
}

Either operand may be a static-Str pointer (into a .rodata packed-struct global) or a heap-Str pointer (into a malloc'd slab); the body does not distinguish, because the consumer ABI (len at offset 0, bytes at offset 8) is identical and @ail_str_eq only reads the bytes via strcmp.

ADT drop with a Str field. For type Boxed = | Box(Str), the per-type drop_M_Boxed(ptr %cell) body in crates/ailang-codegen/src/drop.rs dispatches the Str field through field_drop_call to ailang_rc_dec. Empirically the emitted call is dead code along all shipping execution paths due to two orthogonal codegen optimisations: emit_inlined_partial_drop (iter 18d.3) marks match-arm-extracted Str slots as moved-out and skips their decs, and non-escape lowering (iter 18b) stack-allocates ADTs that never leave the fn frame, eliminating the per-type drop call entirely. The codegen never produces an execution path that hands a static-Str pointer to ailang_rc_dec. We rely on this as a codegen-level invariant — no runtime guard backs it up. If a future codegen change ever loses one of the two elisions, the resulting ailang_rc_dec(static_str_ptr) call would read undefined memory at payload - 8 (the static-Str global has no rc_header slot) and likely segfault — a loud failure that the test suite would flag immediately, leading the implementer of that codegen change to inspect the dispatch site.

Error handling

Three failure modes; most are inherited or structurally impossible.

Out-of-memory at heap-Str allocation. ailang_rc_alloc already abort()s on OOM, matching Boehm. str_alloc, ailang_int_to_str, ailang_float_to_str propagate. No new recovery path.

snprintf truncation in the runtime formatters. The 64-byte stack buffer is strictly larger than the worst-case width of either %lld (longest i64 = 20 chars) or %g on a double (~25 chars worst case). The runtime body asserts no truncation defensively (the snprintf return tells us how many bytes would have been written; an abort() fires if that exceeds the buffer). The defensive check survives format-string changes that might widen the output without anyone noticing the buffer is too small.

Embedded \0 bytes in static-Str literals from JSON. Status quo: a literal containing "�" ends up with a NUL in the middle of the slab; strcmp-based comparison treats it as the end of string. This is the current behaviour of static-Str and heap-Str inherits it unchanged. Heap-Str values produced by int_to_str and float_to_str themselves cannot contain embedded \0 (neither %lld nor %g emits one), so the milestone scope is not affected. DESIGN.md §"Str ABI" notes the limitation explicitly with a forward pointer to a future length-aware-compare milestone.

float_to_str's CodegenError::Internal path is removed entirely after the milestone — neither as a caught nor as an uncaught error. The symbol-installation in the builtin tier and the codegen lowering are landed in the same iteration; there is no intermediate state where one exists without the other.

Testing strategy

Three layers.

IR-shape tests in the codegen crate

Pin the structural guarantees of §"Data flow" with substring matches against emitted IR:

• static_str_global_uses_packed_struct_with_len — pins the new slab format (single i64 length prefix, bytes + NUL in the array tail).
• static_str_callsite_pointer_is_payload_not_slab — pins the getelementptr-to-second-field convention.
• eq_str_calls_ail_str_eq_with_bytes_pointer, compare_str_calls_ail_str_compare_with_bytes_pointer, print_str_calls_puts_with_bytes_pointer — pin the +8 GEP at each existing C-API callsite.
• int_to_str_lowers_to_ailang_int_to_str_call — pins the new builtin's lowering.
• float_to_str_no_longer_errors_internal — converts the existing "lowering deferred"-pinning test into a positive green test.

These tests are fast but sensitive to SSA-naming changes. The existing test style in crates/ailang-codegen/src/lib.rs already accepts this risk for analogous pins (e.g. eq_str_mono_symbol_emits_ail_str_eq_call); the new tests match.

Stdout E2E tests in crates/ail/tests/e2e.rs

Observable program behaviour:

• int_to_str_smoke — int_to_str(42) prints "42\n"; plus edge cases 0, -1, i64::MAX, i64::MIN.
• float_to_str_smoke — float_to_str(3.5) prints something starting with 3.5; plus 0.0, NaN, +Inf, -Inf (pinned against libc %g rendering — implementations are permitted to print NaN as nan, -nan, or other casing; the test pins the exact bytes observed on the dev target with a comment acknowledging the libc-defined freedom).
• static_str_print_unchanged — an existing example (examples/hello.ail or similar) produces byte-identical stdout before and after the milestone, sourced from pre-milestone expected/.

RC-discipline E2E tests via AILANG_RC_STATS

Catch leaks and double-frees:

• int_to_str_drop_balances_rc_stats — int_to_str is called N times in a loop, each result dropped without aliasing; AILANG_RC_STATS=1 reports allocs=K frees=K live=0 for some K matched to the loop bound + setup. Pins that heap-Str participates in RC normally.
• str_field_in_adt_drops_heap_str_correctly — a value of type Boxed = | Box(Str) is instantiated with a heap-Str (from int_to_str) and dropped; live=0. No str_field_in_adt_drops_static_str_noop test ships. An earlier draft of this spec proposed it as a load-bearing safety gate for a runtime sentinel short-circuit; empirically the codegen elides the static-Str-field-drop path entirely via move-tracking and non-escape lowering, so the test could not be made non-vacuous against any natural fixture shape. The elision-based invariant is documented in §Architecture / Codegen layer item 4 and stands or falls with the elision itself, not with a runtime test.

Bench gates

At audit close (per skills/audit/SKILL.md):

• bench/check.py — existing latency baselines green, with the known latency.explicit_at_rc.* nondeterminism tolerance documented in recent audits.
• bench/compile_check.py — entire examples/ corpus compiles.
• bench/cross_lang.py — byte-identical outputs vs. the C reference corpus in bench/reference/. This is the strongest "we did not break anything" gate because it compares output against a second-language implementation, not against AILang itself.

Out of testing scope

Embedded-\0 behaviour. Concurrent RC (AILang is single- threaded). Allocator stress beyond representative program sizes.

Acceptance criteria

The milestone closes when all the following are observable.

Language surface.

• int_to_str : (Int) -> Str is installed in crates/ailang-check/src/builtins.rs and resolved by the codegen builtin lookup.
• float_to_str no longer lowers to CodegenError::Internal; calling it produces a working binary.
• DESIGN.md §"Float semantics" carries no "codegen lowering deferred" caveat on float_to_str; the builtin inventory under "What is supported" lists int_to_str alongside float_to_str.

Runtime / ABI.

• runtime/str.c exports ailang_int_to_str(int64_t) -> char* and ailang_float_to_str(double) -> char*. The private str_alloc(uint64_t) is not in the export symbol set.
• runtime/rc.c is unchanged from pre-milestone.
• Static-Str globals are emitted as <{i64, [N+1 x i8]}> with values {i64 N, c"…\00"}.
• DESIGN.md has a "Str ABI" anchor (or equivalent) documenting the heap-Str and static-Str layouts, the shared consumer ABI (len at offset 0, bytes at offset 8), the codegen-level elision invariant for static-Str-in-RC-paths, and the inherited strcmp-based comparison semantics.

Observable program behaviour.

• A trivial .ail program do io/print_str(int_to_str(42)) builds, runs, and prints 42.
• The analogous float_to_str(3.5) program builds, runs, and prints %g-conformant output (exact bytes pinned by the E2E test).
• All existing examples/ programs produce byte-identical stdout vs. pre-milestone (no regressions).

RC discipline.

• The three new AILANG_RC_STATS-based E2E tests pass.
• No existing drop-relevant test breaks.

Bench gates.

• bench/check.py, bench/compile_check.py, bench/cross_lang.py all green at audit close.

Documentation / journal.

• One journal entry per iteration with per-task notes.
• WhatsNew.md gets a milestone-close entry (verbatim with the done-state Notify), written for the user-as-reader without iteration codes or crate names.

Explicitly not part of acceptance.

• ++ / Str concatenation operator (own milestone).
• Show typeclass and print rewire (own milestone, depends-on this milestone).
• Length-aware comparison semantics / embedded-\0 support.