b49f57d9c6
raw-buf.4 (RawBuf payload + Term::New desugar) implemented and about to
land, but it surfaced that the drop ratification was mis-scoped: the
flat drop FUNCTION is emittable in .4, but the drop CALL needs a codegen
resolution mechanism the .4 plan did not scope. An owned RawBuf binder's
value is a cross-module type-scoped intrinsic call ((app RawBuf.set ..) :
own (RawBuf a)); codegen's is_rc_heap_allocated only marks an App binder
drop-trackable when synth_callee_ret_mode resolves the callee's Own mode,
but codegen synth_arg_type has no TypeDef-first / cross-module-mono
resolution ladder (unlike the checker's lib.rs:3465), so the binder is
non-trackable and no drop call is inserted. A second site
(uniqueness::infer_module's per-module globals) misses the cross-module
borrow mode too.
Re-carve (six iterations; .1-.4 done, .5/.6 remain):
raw-buf.4 — RawBuf payload + Term::New desugar + flat drop FUNCTION
(DONE). Worked program prints 60; Float/Bool/reject ship.
raw-buf.5 — owned cross-module type-scoped drop-call resolution: a
TypeDef-first + cross-module-mono arm in codegen
synth_arg_type feeding is_rc_heap_allocated /
synth_callee_ret_mode / drop_symbol_for_binder, plus
cross-module op visibility in uniqueness::infer_module.
Ratified by raw_buf_no_leak (live == 0).
raw-buf.6 — kernel_stub retirement (was .5).
Also documents the raw-buf.4 diagnostic-behaviour change: because the
Term::New desugar now runs before check, (new T ..) with a missing
new-op surfaces type-scoped-member-not-found (more precise) rather than
the prep.2 new-type-not-constructible it supersedes; new-arg-kind-mismatch
is obsoleted (the desugar drops the type-arg). Same rejection conditions,
preserved. And corrects the @ailang_rc_release spec slip to the real
symbol @ailang_rc_dec.
The drop-call resolution is its own iteration for the same reason
raw-buf.3 became a mechanism prep: it is a codegen-resolution mechanism
(parity with the checker's type-scoped/cross-module ladder), separable
from the RawBuf payload, and isolating it keeps the resolution change
off the .4 payload diff.
594 lines
31 KiB
Markdown
594 lines
31 KiB
Markdown
# raw-buf — Design Spec
|
||
|
||
**Date:** 2026-05-29 (revised twice 2026-05-29)
|
||
**Status:** Draft — awaiting user spec review
|
||
**Authors:** orchestrator + Claude
|
||
|
||
> **Revision 1 note (post intrinsic-bodies).** raw-buf.1 (intercept
|
||
> registry) shipped (`140a0c0`). The `intrinsic-bodies` milestone
|
||
> (#9) then landed and removed the placeholder-body convention the
|
||
> original spec relied on — kernel-tier fn bodies are now
|
||
> typechecked, so `RawBuf.get` cannot ship a type-lying placeholder
|
||
> body; it must be `(intrinsic)`. And an `(intrinsic)` marker now
|
||
> requires a matching `INTERCEPTS` entry as a lockstep pair, so a
|
||
> "manifest-only / codegen-deferred" intermediate state is
|
||
> forbidden. The kernel-manifest and codegen therefore land
|
||
> together.
|
||
>
|
||
> **Revision 2 note (post raw-buf.2 + plan-recon).** raw-buf.2 (the
|
||
> family-crate rename) shipped (`fbdbe74`). Planning raw-buf.3 then
|
||
> surfaced a deeper gap: RawBuf's operations are a *third* intrinsic
|
||
> shape the `intrinsic-bodies` mechanism never handled. The bijection
|
||
> + symbol story was built for (a) monomorphic top-level intrinsics
|
||
> (`answer`, `float_*`) and (b) per-type class-method instances
|
||
> (`eq__Int`). RawBuf's ops are **type-scoped polymorphic top-level
|
||
> intrinsics** (`forall a` with `param-in {Int,Float,Bool}`, called
|
||
> as `RawBuf.get`). For that shape:
|
||
>
|
||
> 1. `mono_symbol_n` mints `<base>__<T>` from the bare fn name, so
|
||
> `RawBuf.get` @ Int becomes `get__Int` — the `RawBuf` scope never
|
||
> enters the symbol. `new`/`get`/`set`/`size` @ Int would collide
|
||
> with any other polymorphic free fn of those (very common) names.
|
||
> 2. The bijection collector's `Def::Fn` arm records the bare name
|
||
> `"get"` as one marker — matching neither the per-type entries nor
|
||
> their symbols. One polymorphic marker must map to N per-element
|
||
> entries, which the current 1-marker-to-1-entry bijection cannot
|
||
> express.
|
||
>
|
||
> So raw-buf.3 is no longer "add 12 table rows" — it is a new
|
||
> language mechanism (type-scoped polymorphic intrinsics) that RawBuf
|
||
> is the first consumer of. The remaining work is re-carved into a
|
||
> mechanism prep (raw-buf.3), the RawBuf payload (raw-buf.4), and the
|
||
> stub retirement (raw-buf.5). The decided naming scheme is
|
||
> scope-qualified mono symbols (`RawBuf_get__Int`); the mechanism is
|
||
> ratified standalone on the stub (the prep pattern that prep.1/.2/.3
|
||
> used) before RawBuf consumes it. The Goal, slab layout, and
|
||
> feature-acceptance argument are unchanged.
|
||
|
||
## Goal
|
||
|
||
Ship `RawBuf a` — the first kernel-tier *base* extension — as a
|
||
mutable, indexed, bounded-size flat buffer of primitive elements
|
||
(`{Int, Float, Bool}`). RawBuf is the storage primitive that
|
||
unblocks two downstream needs both already captured in the
|
||
backlog:
|
||
|
||
- **Series (milestone #8)** — a bounded ring buffer with
|
||
financial-style indexing, implemented as a *library extension*
|
||
(pure AILang) wrapping `RawBuf a` plus bookkeeping fields.
|
||
- **Embedding-ABI batch FFI (subsumes closed #2)** — the M5
|
||
friction-harvest measured per-tick FFI at ~206 ns/tick on real
|
||
EURUSD volume (~658 ms / 3.19M ticks). A contiguous primitive
|
||
slice amortises that per-tick cost via a batch crossing.
|
||
RawBuf is that primitive.
|
||
|
||
The milestone also fulfils the architectural commitment from the
|
||
kernel-extensions whitepaper § "Plugin contract": the migration
|
||
of the hardcoded primitive-instance intercepts into one registry,
|
||
triggered by the first real base extension shipping. raw-buf.1
|
||
already discharged that migration; RawBuf is now the first
|
||
*real-payload* consumer of that registry — and the first consumer
|
||
of the type-scoped polymorphic intrinsic mechanism raw-buf.3 adds.
|
||
|
||
## Architecture
|
||
|
||
Six iterations. raw-buf.1–.4 are done; .5/.6 remain. (The drop
|
||
ratification, originally bundled into .4, was re-carved into its own
|
||
iteration .5 once the implement surfaced that an owned cross-module
|
||
type-scoped intrinsic binder's drop-call needs a codegen
|
||
resolution mechanism the .4 plan did not scope — see iteration 5.)
|
||
|
||
1. **Intercept registry** (raw-buf.1) — **DONE** (`140a0c0`).
|
||
Lifted the hard-coded `try_emit_primitive_instance_body` match
|
||
into the registry table `INTERCEPTS` in
|
||
`crates/ailang-codegen/src/intercepts.rs`; the dispatch site is
|
||
an `intercepts::lookup` call. (The subsequent `intrinsic-bodies`
|
||
milestone added the `(intrinsic)` marker, the `Term::Intrinsic`
|
||
leaf, and the marker ↔ entry bijection pin on top of it.)
|
||
|
||
2. **Kernel family-crate rename** (raw-buf.2) — **DONE**
|
||
(`fbdbe74`). Renamed `crates/ailang-kernel-stub/` →
|
||
`crates/ailang-kernel/`, reshaped as a family-crate
|
||
(`src/lib.rs` hub re-exporting `src/kernel_stub/{mod.rs,
|
||
source.ail}`). Public symbol `STUB_AIL` preserved; zero
|
||
behavioural change.
|
||
|
||
3. **Type-scoped polymorphic intrinsic mechanism** (raw-buf.3) —
|
||
The new language mechanism RawBuf needs, built and ratified
|
||
standalone on the stub (no RawBuf yet), mirroring how prep.1/.2/.3
|
||
built kernel-extension mechanisms ratified by the stub. Two
|
||
coupled pieces:
|
||
- **Scope-qualified mono symbols.** When a type-scoped call
|
||
`T.f` (prep.1 spelling) resolves to a polymorphic op `f` of
|
||
TypeDef `T`, monomorphisation mints `T_f__<elem>` (e.g.
|
||
`StubT_peek__Int`) — the TypeDef scope is part of the symbol,
|
||
so two different types' identically-named ops never collide,
|
||
and the emitted IR is legible (it reads as "the `T.f`
|
||
specialised to `<elem>`"). The codegen intercept dispatch keys
|
||
on this scoped symbol.
|
||
- **Bijection expansion over `param-in`.** The bijection
|
||
collector (`intercepts::tests::workspace_intrinsic_markers`)
|
||
gains an arm: a top-level `(intrinsic)` op that is polymorphic
|
||
and type-scoped to a TypeDef with `param-in (a S1 S2 …)`
|
||
expands to one marker `T_f__Si` per allowed element type `Si`.
|
||
One polymorphic marker → N per-element entries, all pinned by
|
||
`intercepts_bijection_with_intrinsic_markers`.
|
||
Ratified by adding ONE polymorphic type-scoped `(intrinsic)` op
|
||
to the stub's `StubT` (`StubT.peek : (borrow (StubT a)) -> a`)
|
||
plus its 2 scope-qualified `INTERCEPTS` entries
|
||
(`StubT_peek__Int`, `StubT_peek__Float` — StubT's `param-in` is
|
||
`{Int, Float}`) and a unit test asserting the scoped symbols +
|
||
the bijection. `peek` retires with the stub in raw-buf.6, exactly
|
||
as `answer` does. No running E2E here (constructing a `StubT`
|
||
needs the Term::New desugar that lands in raw-buf.4); the
|
||
mechanism is unit-test ratified.
|
||
|
||
4. **RawBuf payload + Term::New desugar** (raw-buf.4) — **DONE**.
|
||
RawBuf as a clean consumer of the raw-buf.3 mechanism, plus the
|
||
`(new …)` construction sugar. Shipped:
|
||
- The `raw_buf` submodule (`src/raw_buf/{mod.rs, source.ail}`)
|
||
with the `RawBuf` TypeDef (`param-in (a Int Float Bool)`) and
|
||
the four ops `new`/`get`/`set`/`size`, each an `(intrinsic)`
|
||
marker.
|
||
- The 12 scope-qualified `INTERCEPTS` entries
|
||
(`RawBuf_{new,get,set,size}__{Int,Float,Bool}`) + emit fns
|
||
(`@ailang_rc_alloc` + getelementptr + load/store) — mechanical
|
||
on top of the raw-buf.3 naming + bijection machinery.
|
||
- The `Term::New` desugar `(new T <types…> <values…>)` →
|
||
`(app T.new <values…>)` (general — `(new StubT 42)` benefits
|
||
too), removing BOTH codegen `Term::New` deferral arms that
|
||
prep.2 left marked "milestone raw-buf". The desugar runs before
|
||
check, dropping the `NewArg::Type`; the element type is
|
||
recovered by inference from use.
|
||
- The flat intrinsic-storage drop *function* `@drop_raw_buf_RawBuf`
|
||
(a single `@ailang_rc_dec` on the slab pointer), emitted for
|
||
any TypeDef whose `new` op is `(intrinsic)`-bodied.
|
||
- `ailang-surface` wiring: `parse_raw_buf` + workspace injection
|
||
(raw_buf kernel-tier auto-imported; workspace count 4 → 5).
|
||
The worked consumer program runs end-to-end and prints `60`;
|
||
the Float/Bool variants and the `param-in` reject also ship.
|
||
`kernel_stub` (with `peek` + `answer`) stays alongside (count 5).
|
||
**Not** in .4: the drop *call* does not yet fire at scope close
|
||
for an owned RawBuf binder — that needs the codegen resolution
|
||
mechanism in .5 (the drop function is correct and waiting).
|
||
|
||
5. **Owned cross-module type-scoped drop-call resolution**
|
||
(raw-buf.5) — The codegen mechanism that makes the .4 flat drop
|
||
*fire*. An owned RawBuf binder's value is a cross-module
|
||
type-scoped intrinsic call (`(app RawBuf.set …) : own (RawBuf a)`);
|
||
codegen's `is_rc_heap_allocated` only marks an `App` binder
|
||
drop-trackable when `synth_callee_ret_mode` resolves the callee's
|
||
`Own` ret-mode, but codegen `synth_arg_type` has no TypeDef-first
|
||
/ cross-module-mono resolution ladder (unlike the checker's
|
||
`lib.rs:3465`), so the binder is classified non-trackable and no
|
||
drop call is inserted. A second site —
|
||
`uniqueness::infer_module`'s per-module globals map — likewise
|
||
misses the cross-module `borrow` mode, defaulting the
|
||
`RawBuf.get` arg to `Consume`. raw-buf.5 teaches codegen's
|
||
resolution a TypeDef-first + cross-module-mono arm (mirror of the
|
||
checker ladder) feeding `is_rc_heap_allocated` /
|
||
`synth_callee_ret_mode` / `drop_symbol_for_binder`, and extends
|
||
`uniqueness::infer_module` with cross-module op visibility.
|
||
Ratified by `raw_buf_no_leak` (the worked Int program under
|
||
`AILANG_RC_STATS` reaches `live == 0`). This is the drop
|
||
ratification the spec's acceptance criterion requires; isolating
|
||
it keeps the resolution-mechanism change off the .4 payload diff.
|
||
|
||
6. **kernel_stub retirement** (raw-buf.6) — RawBuf now subsumes
|
||
every ratification role the stub held: `Term::New` end-to-end
|
||
(the `(new RawBuf …)` consumer), `param-in` reject (the Str
|
||
reject fixture), kernel-tier auto-import, the monomorphic
|
||
intrinsic path (RawBuf has no nullary intrinsic, but the path is
|
||
exercised by the prelude's `float_*`), and — crucially — the
|
||
type-scoped polymorphic intrinsic mechanism (RawBuf's four ops).
|
||
Delete `src/kernel_stub/` + its re-export, drop
|
||
`parse_kernel_stub` + its injection, remove the `answer` and the
|
||
two `StubT_peek__*` `INTERCEPTS` entries + their emit fns (their
|
||
markers leave with the stub — the bijection lockstep holds),
|
||
delete `kernel_stub_module_round_trips`, re-baseline
|
||
`workspace_pin` to 4 (raw_buf in, kernel_stub out), and update
|
||
`design/INDEX.md` + `design/models/0007-kernel-extensions.md`.
|
||
The `ailang-kernel` crate stays as the family-crate home for
|
||
future kernel-tier modules.
|
||
|
||
Ordering rationale: .3 isolates the risky mono+bijection mechanism
|
||
change (proven on the stub) from RawBuf's payload diff; .4 lands
|
||
RawBuf as a mechanical consumer plus the orthogonal `(new …)`
|
||
desugar + the flat drop *function*; .5 isolates the codegen
|
||
drop-call *resolution* mechanism (cross-module type-scoped ret-mode
|
||
+ uniqueness visibility) off the .4 payload diff; .6 removes the
|
||
now-redundant stub without touching the new RawBuf code. Each
|
||
bijection-registry change is isolated to one diff (peek entries
|
||
added in .3, 12 RawBuf entries in .4, peek+answer removed in .6).
|
||
|
||
## Concrete code shapes
|
||
|
||
### Primary — the raw-buf.3 mechanism ratifier (StubT.peek)
|
||
|
||
raw-buf.3 adds one polymorphic type-scoped `(intrinsic)` op to the
|
||
stub to prove the mechanism. The op and its scope-qualified entries
|
||
are the iteration's concrete deliverable (the must-pass evidence
|
||
that scoped symbols + bijection expansion work). The fn is appended
|
||
to the existing `STUB_AIL` source (`crates/ailang-kernel/src/kernel_stub/source.ail`):
|
||
|
||
```text
|
||
(fn peek
|
||
(doc "Ratifies the type-scoped polymorphic intrinsic mechanism: StubT.peek is polymorphic over a (param-in {Int,Float}) and type-scoped to StubT. Codegen intercepts StubT_peek__Int / StubT_peek__Float emit a load from the Stub payload. Retires with the stub in raw-buf.5.")
|
||
(type (forall (vars a) (fn-type (params (borrow (con StubT a))) (ret a))))
|
||
(params s)
|
||
(intrinsic))
|
||
```
|
||
|
||
(Shown `text`, not `ail`: `kernel_stub` is a reserved built-in
|
||
module name, so the source cannot be `ail check`-ed standalone —
|
||
`reserved-module-name`. Ratification is the round-trip test + the
|
||
bijection/mono unit tests, not a standalone parse.) Its two
|
||
scope-qualified entries (`StubT_peek__Int`, `StubT_peek__Float`)
|
||
land in `INTERCEPTS` together with the bijection-collector
|
||
extension; the bijection test pins the 1-marker→2-entries mapping.
|
||
|
||
### Primary — the raw_buf kernel-tier module (raw-buf.4, gate-verified)
|
||
|
||
The exact Form-A source the `raw_buf` submodule embeds. It parses
|
||
and type-checks against the current tool (`ail check` → `ok`); the
|
||
four `(intrinsic)` markers are legal because the module is
|
||
`(kernel)`. `RawBuf.get` returns the bare element type `a`; `set`
|
||
is linear (`own` in, `own` out).
|
||
|
||
```ail
|
||
(module raw_buf
|
||
(kernel)
|
||
(data RawBuf (vars a)
|
||
(doc "Mutable, indexed, fixed-size flat buffer of primitive elements. Opaque single-ctor handle; codegen intercepts emit raw alloc/load/store over an @ailang_rc_alloc slab. Element type restricted to {Int, Float, Bool} via param-in.")
|
||
(ctor B a)
|
||
(param-in (a Int Float Bool)))
|
||
(fn new
|
||
(doc "Allocate an uninitialised RawBuf of capacity n. Compiler-supplied (intrinsic) body; codegen intercept RawBuf_new__<T> emits @ailang_rc_alloc plus the i64 size header. Element bytes are uninitialised; caller must set before get.")
|
||
(type (forall (vars a) (fn-type (params (con Int)) (ret (own (con RawBuf a))))))
|
||
(params n)
|
||
(intrinsic))
|
||
(fn get
|
||
(doc "Indexed read. UB if i >= (size b); caller checks bounds via size. Compiler-supplied (intrinsic) body; codegen intercept RawBuf_get__<T> emits getelementptr plus load.")
|
||
(type (forall (vars a) (fn-type (params (borrow (con RawBuf a)) (con Int)) (ret a))))
|
||
(params b i)
|
||
(intrinsic))
|
||
(fn set
|
||
(doc "Indexed write. Linear: own in, own out. Under uniqueness in-place; under shared copy-on-write (Issue #22). Compiler-supplied (intrinsic) body; codegen intercept RawBuf_set__<T> emits getelementptr plus store.")
|
||
(type (forall (vars a) (fn-type (params (own (con RawBuf a)) (con Int) a) (ret (own (con RawBuf a))))))
|
||
(params b i v)
|
||
(intrinsic))
|
||
(fn size
|
||
(doc "Element count. Compiler-supplied (intrinsic) body; codegen intercept RawBuf_size__<T> emits a single i64 load from the slab header.")
|
||
(type (forall (vars a) (fn-type (params (borrow (con RawBuf a))) (ret (con Int)))))
|
||
(params b)
|
||
(intrinsic)))
|
||
```
|
||
|
||
The `(ctor B a)` is a nominal fiction for the type system: it
|
||
gives `RawBuf` an identity and keeps `a` non-phantom. Real code
|
||
never constructs it — every operation is codegen-intercepted, and
|
||
`(new RawBuf …)` desugars to `(app RawBuf.new …)`, not to a
|
||
`(term-ctor RawBuf B …)`.
|
||
|
||
### Primary — the worked consumer program (raw-buf.4 acceptance)
|
||
|
||
The program an LLM author naturally writes for "I need a small
|
||
mutable indexed buffer to score N values and read them back." It is
|
||
the feature-acceptance criterion's empirical evidence. It uses the
|
||
`(new RawBuf …)` construction sugar (available once raw-buf.4 lands
|
||
the Term::New desugar). It references the `raw_buf` module, injected
|
||
only from raw-buf.4 — so it does **not** check against the current
|
||
tool (RawBuf is not yet a workspace module). Left unlabeled
|
||
deliberately: it describes post-raw-buf.4 surface.
|
||
|
||
```
|
||
(module raw_buf_demo
|
||
(fn main
|
||
(type (fn-type (params) (ret (con Int))))
|
||
(params)
|
||
(body
|
||
(let buf (new RawBuf (con Int) 3)
|
||
(let buf (app RawBuf.set buf 0 10)
|
||
(let buf (app RawBuf.set buf 1 20)
|
||
(let buf (app RawBuf.set buf 2 30)
|
||
(app + (app RawBuf.get buf 0)
|
||
(app + (app RawBuf.get buf 1)
|
||
(app RawBuf.get buf 2))))))))))
|
||
; ail check && ail build && ail run → prints 60
|
||
```
|
||
|
||
No `(import raw_buf)` needed: `RawBuf` is visible via kernel-tier
|
||
auto-import (prep.3). `RawBuf.set`/`.get`/`.new` resolve via
|
||
type-scoped namespacing (prep.1) and mono to the scope-qualified
|
||
symbols `RawBuf_set__Int` / `RawBuf_get__Int` / `RawBuf_new__Int`.
|
||
The element type `a = Int` is fixed by inference: it flows from the
|
||
`(app RawBuf.set buf 0 10)` value arg (`10 : Int`) through the
|
||
linear `own (RawBuf a)` threading. (For the direct-call equivalent
|
||
without the `(new …)` sugar — `(let buf (app RawBuf.new 3) …)` —
|
||
the same inference applies; the desugar reduces `(new RawBuf (con
|
||
Int) 3)` to exactly that.)
|
||
|
||
### Primary — the param-in reject fixture (raw-buf.4 must-fail)
|
||
|
||
```
|
||
(module raw_buf_reject_str
|
||
(fn cant_str
|
||
(type (fn-type (params (borrow (con RawBuf (con Str)))) (ret (con Str))))
|
||
(params buf)
|
||
(body (app RawBuf.get buf 0))))
|
||
; ail check → param-not-in-restricted-set: type Str not in {Int, Float, Bool}
|
||
```
|
||
|
||
Must fail at the checker with the `ParamNotInRestrictedSet`
|
||
diagnostic shipped in prep.3 (fieldtest F8) — the must-fail evidence
|
||
that `param-in` is alive on a real-payload extension. Depends on
|
||
`raw_buf` being injected, so not gate-checkable today (unlabeled).
|
||
|
||
### Secondary — scope-qualified mono symbol (raw-buf.3, implementation)
|
||
|
||
Current behaviour: `mono_symbol_n(base, types)`
|
||
(`crates/ailang-check/src/mono.rs:521`) joins `base` and each
|
||
type's mono suffix with `__`. For a bare-named poly free fn `get`
|
||
it yields `get__Int` — no type scope. The bijection collector's
|
||
`Def::Fn` arm (`intercepts.rs` ~490) inserts the bare `f.name`.
|
||
|
||
raw-buf.3 change (shape — exact threading mapped by plan-recon):
|
||
a type-scoped call `T.f` carries the resolved TypeDef `T`; mono
|
||
uses a scope-qualified base `T_f` so the symbol is `T_f__<elem>`
|
||
(`StubT_peek__Int`, later `RawBuf_get__Int`). The bijection
|
||
collector gains an arm that, for a polymorphic top-level
|
||
`(intrinsic)` op scoped to a TypeDef with `param-in (a S1 …)`,
|
||
emits `T_f__Si` for each `Si`. Both the mono mint and the collector
|
||
must compute the identical string (the bijection test is the pin).
|
||
|
||
### Secondary — INTERCEPTS entry shape (raw-buf.4, implementation)
|
||
|
||
The registry struct (raw-buf.1) is `Intercept { name,
|
||
expected_params, expected_ret, wants_alwaysinline, emit }`. One
|
||
representative of the 12 RawBuf entries (the exact symbol strings
|
||
are produced by the raw-buf.3 mechanism and pinned by the bijection
|
||
test):
|
||
|
||
```rust
|
||
// crates/ailang-codegen/src/intercepts.rs — appended to INTERCEPTS
|
||
Intercept {
|
||
name: "RawBuf_new__Int", // scope-qualified mono symbol of RawBuf.new @ Int
|
||
expected_params: &["i64"], // capacity n
|
||
expected_ret: "ptr", // slab base pointer
|
||
wants_alwaysinline: false,
|
||
emit: emit_rawbuf_new_int, // @ailang_rc_alloc(8 + n*8); store i64 n at offset 0
|
||
},
|
||
// + RawBuf_new__{Float,Bool}, RawBuf_get__{Int,Float,Bool},
|
||
// RawBuf_set__{Int,Float,Bool}, RawBuf_size__{Int,Float,Bool}
|
||
```
|
||
|
||
### Secondary — Term::New desugar (raw-buf.4, implementation)
|
||
|
||
`Term::New { type_name, args }` with `args: Vec<NewArg>`
|
||
(`NewArg::Type | NewArg::Value`, both parsed + round-trip ratified).
|
||
The desugar rewrites it to an application of the type's `new` fn,
|
||
passing only the `NewArg::Value` args:
|
||
|
||
```
|
||
(new RawBuf (con Int) 3) ; Term::New { type_name: "RawBuf",
|
||
; args: [Type(con Int), Value(3)] }
|
||
──desugar──▶
|
||
(app RawBuf.new 3) ; a = Int resolved by inference from use
|
||
```
|
||
|
||
Important: the AST has no type-ascription term and `Term::App`
|
||
carries no type-args, so the desugar **drops** the `NewArg::Type`
|
||
and the element type is recovered by ordinary inference (it flows
|
||
from how the result is used — see the worked program). This
|
||
suffices for every real program that uses the buffer; an isolated
|
||
`(new RawBuf (con Int) 3)` with no constraining use would leave `a`
|
||
ambiguous, which is acceptable (no real program does that). After
|
||
this pass no `Term::New` survives into codegen; both deferral arms
|
||
(`lib.rs` ~2094 `lower_term`, ~3296 synth) are removed. The prep.2
|
||
checker already accepts `Term::New` (it parses, round-trips, and
|
||
typechecks via the `new` def's signature); whether it needs any
|
||
extension for the `NewArg::Type`-bearing form is a raw-buf.4
|
||
verify-first task (plan-recon maps the current Term::New typecheck
|
||
path).
|
||
|
||
### Secondary — slab layout (raw-buf.4)
|
||
|
||
```
|
||
[ size : i64 ][ elem_0 ][ elem_1 ] ... [ elem_{size-1} ]
|
||
^ ^ ^
|
||
offset 0 offset 8 offset 8 + size*sizeof(T)
|
||
```
|
||
|
||
Element widths: `Int = 8`, `Float = 8`, `Bool = 1`. Header is i64,
|
||
8-byte aligned; element payload follows directly. `@ailang_rc_alloc`
|
||
is called with `8 + size * sizeof(T)` and returns the slab base
|
||
`ptr`. The size header is self-describing, so drop and `RawBuf.size`
|
||
need no side-table. Drop is a single `@ailang_rc_dec` on the
|
||
slab pointer — primitive elements carry no recursive drops; the
|
||
per-TypeDef drop generation gets a flat-slab arm (raw-buf.4 emits
|
||
the drop *function*; raw-buf.5 makes the drop *call* fire). (`@ailang_rc_dec`
|
||
is the real runtime symbol — the earlier `@ailang_rc_release`
|
||
spelling was a spec slip.)
|
||
|
||
## Components
|
||
|
||
| Component | Iter | Files |
|
||
|-----------|------|-------|
|
||
| Intercept registry | raw-buf.1 (DONE) | `crates/ailang-codegen/src/intercepts.rs` + dispatch in `lib.rs` |
|
||
| Kernel family-crate rename | raw-buf.2 (DONE) | `crates/ailang-kernel/` (renamed); `ailang-surface` dep + `loader.rs`; doc/ledger rows |
|
||
| Type-scoped polymorphic intrinsic mechanism | raw-buf.3 | scope-qualified mono mangling in `crates/ailang-check/src/mono.rs` (+ wherever prep.1 type-scoped resolution records the TypeDef, per recon); bijection-collector expansion arm in `crates/ailang-codegen/src/intercepts.rs` (`workspace_intrinsic_markers`); `StubT.peek` fn added to `crates/ailang-kernel/src/kernel_stub/source.ail` + 2 `StubT_peek__*` entries + emit fns; `kernel_stub_module_round_trips` updated for the new fn; mono/bijection unit tests |
|
||
| RawBuf payload + Term::New desugar | raw-buf.4 (DONE) | new `crates/ailang-kernel/src/raw_buf/{mod.rs, source.ail}` + `lib.rs` re-export; `parse_raw_buf` + injection in `crates/ailang-surface/src/loader.rs` + re-export in `lib.rs`; 12 entries + emit fns in `intercepts.rs`; `Term::New` desugar in `crates/ailang-core/src/desugar.rs` + removal of both codegen deferral arms in `crates/ailang-codegen/src/lib.rs`; flat-slab drop *function* in `drop.rs`/`lib.rs`; `workspace_pin` 4 → 5; round-trip + Int/Float/Bool/reject E2E |
|
||
| Owned cross-module type-scoped drop-call resolution | raw-buf.5 | TypeDef-first + cross-module-mono resolution arm in codegen `synth_arg_type` (`crates/ailang-codegen/src/lib.rs`) feeding `is_rc_heap_allocated` / `synth_callee_ret_mode` / `drop_symbol_for_binder`; cross-module op visibility in `uniqueness::infer_module` (`crates/ailang-check/src/uniqueness.rs`); `raw_buf_no_leak` E2E |
|
||
| kernel_stub retirement | raw-buf.6 | delete `crates/ailang-kernel/src/kernel_stub/` + re-export; drop `parse_kernel_stub` + injection; remove `answer` + `StubT_peek__*` entries + emit fns in `intercepts.rs`; delete `kernel_stub_module_round_trips`; `workspace_pin` 5 → 4; `design/INDEX.md` + `design/models/0007-kernel-extensions.md` |
|
||
|
||
## Data flow
|
||
|
||
**Mechanism (raw-buf.3):**
|
||
1. A consumer call `T.f` (type-scoped spelling, prep.1) resolves to
|
||
the polymorphic op `f` of TypeDef `T`, recording `T` as the
|
||
resolution scope.
|
||
2. Monomorphisation at element type `Si` mints the scope-qualified
|
||
symbol `T_f__Si`; codegen emits a fn of that name; the intercept
|
||
dispatch keys on it.
|
||
3. The bijection collector expands each polymorphic type-scoped
|
||
`(intrinsic)` op over its TypeDef's `param-in` set into the same
|
||
`T_f__Si` strings, and requires each to resolve to an `INTERCEPTS`
|
||
entry (and vice versa, minus `OPTIMISATION_ONLY`).
|
||
|
||
**Build-time (raw-buf.4 onwards):**
|
||
1. CLI loads workspace → `ailang-surface` injects `raw_buf` as a
|
||
kernel-tier module (`kernel: true`).
|
||
2. `check` walks consumer code; `(con RawBuf (con Int))` resolves
|
||
to `raw_buf.RawBuf` via kernel-tier auto-import (prep.3).
|
||
3. `param-in` on `a` rejects non-{Int,Float,Bool} instantiations
|
||
via the existing `ParamNotInRestrictedSet` diagnostic.
|
||
|
||
**Codegen-time (raw-buf.4 onwards):**
|
||
1. The desugar replaces every `Term::New` with `(app T.new
|
||
<values>)`; the element type is inferred from use.
|
||
2. Codegen reaches the registry dispatch site; if the fn's
|
||
scope-qualified mono symbol resolves via `intercepts::lookup`,
|
||
the entry's `emit` runs (the `(intrinsic)` marker has no body).
|
||
3. `RawBuf_new__Int` emits the alloc; `RawBuf_set__Int` a `store
|
||
i64`; `RawBuf_get__Int` a `load i64`; `RawBuf_size__Int` a header
|
||
load.
|
||
|
||
**Drop-call resolution (raw-buf.5):** the desugared
|
||
`(app RawBuf.set …) : own (RawBuf a)` binder is recognised as
|
||
rc-heap-trackable once codegen `synth_callee_ret_mode` resolves the
|
||
cross-module type-scoped callee's `Own` mode; the let-scope-close
|
||
drop call to `@drop_raw_buf_RawBuf` then fires.
|
||
|
||
**Workspace module count:** prelude + kernel_stub + 2 test fixtures
|
||
= 4 before raw-buf.4. raw-buf.4 adds raw_buf → 5. raw-buf.6 retires
|
||
kernel_stub → 4. The `workspace_pin` test pins each transition.
|
||
|
||
## Error handling
|
||
|
||
Two consumer-facing diagnostics; both already shipped, no new arms:
|
||
|
||
- **`param-not-in-restricted-set`** (prep.3, fieldtest F8) — fires
|
||
on `(con RawBuf (con Str))` or any non-{Int,Float,Bool} element.
|
||
- **`type-scoped-member-not-found`** — fires on `(new SomeType …)`
|
||
when `SomeType` has no `(fn new …)`. (raw-buf.4 note: because the
|
||
`Term::New` desugar now runs *before* check, the rejection is
|
||
raised by synth's type-scoped resolution of the desugared
|
||
`(app SomeType.new …)`, surfacing the more precise
|
||
`type-scoped-member-not-found` rather than the prep.2
|
||
`new-type-not-constructible` it superseded — same rejection
|
||
condition, more context. The prep.2 `new-arg-kind-mismatch` is
|
||
obsoleted: the desugar drops the type-arg, so there is no
|
||
arg-kind to mismatch.) raw_buf ships `new`, so the worked program
|
||
passes.
|
||
|
||
The codegen `Term::New` deferral arms (`lib.rs` ~2094 + ~3296,
|
||
message "… milestone raw-buf") are **removed** in raw-buf.4 once the
|
||
desugar eliminates `Term::New` before codegen. Neither is
|
||
user-facing — both are the prep.2 deferral marker. They have no test
|
||
pinning them today (internal `CodegenError::Internal`); raw-buf.4's
|
||
build-and-run E2E (the worked program printing `60`) is the forward
|
||
ratification.
|
||
|
||
Bounds checks are **not** in scope: `RawBuf.get/.set` are UB if `i
|
||
>= size(b)`; caller checks via `RawBuf.size`. A future axis, no
|
||
backlog item yet.
|
||
|
||
## Testing strategy
|
||
|
||
**raw-buf.3 (mechanism):**
|
||
- Mono unit test: a type-scoped poly intrinsic op mints the
|
||
scope-qualified symbol (`StubT_peek__Int`, `StubT_peek__Float`) —
|
||
asserts the new mangling directly, no running program needed.
|
||
- Bijection: `intercepts_bijection_with_intrinsic_markers` stays
|
||
green with the `peek` marker expanded to its 2 scope-qualified
|
||
entries (proves the 1-marker→N-entries expansion).
|
||
- `kernel_stub_module_round_trips` updated for the added `peek` fn
|
||
(parse → serialise → round-trip holds).
|
||
- Full suite green; no workspace-count change (peek is added to the
|
||
existing stub, no new module).
|
||
|
||
**raw-buf.4 (RawBuf payload + desugar):**
|
||
- Round-trip `raw_buf_module_round_trips` (mirror
|
||
`kernel_stub_module_round_trips`): assert `kernel == true`, name
|
||
`raw_buf`, `RawBuf` TypeDef `param_in` for `a` contains
|
||
Int/Float/Bool.
|
||
- E2E `raw_buf_int_e2e`: the worked program — `ail check && ail
|
||
build && ail run` → prints `60`.
|
||
- E2E `raw_buf_float_e2e` / `raw_buf_bool_e2e`: Float and Bool
|
||
variants (Bool catches any i1/i8 packing question).
|
||
- E2E `raw_buf_param_in_reject_e2e`: the Str reject fixture — `ail
|
||
check` exits non-zero with `param-not-in-restricted-set`.
|
||
- Term::New desugar ratified by the `(new RawBuf …)` form building
|
||
and running (it would hit the now-removed deferral arm otherwise);
|
||
plus `new_stubt_builds_and_runs` (`(new StubT 42)` via the real-body
|
||
`StubT.new`).
|
||
- Bijection green with the 12 RawBuf entries added.
|
||
- `workspace_pin` count 4 → 5 (+ `contains_key("raw_buf")`).
|
||
- (The drop / no-leak ratification moved to raw-buf.5 — the flat drop
|
||
function ships here but its call-insertion needs the .5 resolution
|
||
mechanism.)
|
||
|
||
**raw-buf.5 (drop-call resolution):**
|
||
- `raw_buf_no_leak`: the worked Int program under `AILANG_RC_STATS`
|
||
reaches `live == 0` (the slab is freed at let-scope close). RED
|
||
before the resolution arm, GREEN after — the iteration's ratifier.
|
||
- Full suite green; existing symbols unchanged (the resolution arm
|
||
fires only for owned cross-module type-scoped binders).
|
||
|
||
**raw-buf.6 (retirement):**
|
||
- `workspace_pin` re-baselines 5 → 4 (raw_buf in, kernel_stub out).
|
||
- `kernel_stub_module_round_trips` deleted;
|
||
`raw_buf_module_round_trips` carries the invariant forward.
|
||
- Bijection green after `answer` + `StubT_peek__*` markers + entries
|
||
leave together with the stub.
|
||
- Full suite green; the diff is a removal.
|
||
|
||
## Acceptance criteria
|
||
|
||
The milestone ships when:
|
||
|
||
1. The worked consumer program runs end-to-end: `ail check && ail
|
||
build && ail run` prints `60` (raw-buf.4 ✓).
|
||
2. The param-in reject fixture exits at `ail check` with the
|
||
`param-not-in-restricted-set` diagnostic (raw-buf.4 ✓).
|
||
3. RawBuf's four ops are `(intrinsic)` markers, each mono'd to a
|
||
scope-qualified symbol (`RawBuf_<op>__<T>`) with a matching
|
||
`INTERCEPTS` entry; `intercepts_bijection_with_intrinsic_markers`
|
||
is green (the mechanism in raw-buf.3, the 12 entries in raw-buf.4 ✓).
|
||
4. `Term::New` no longer reaches codegen — the desugar eliminates it
|
||
and both deferral arms are removed (raw-buf.4 ✓).
|
||
5. A RawBuf allocated and dropped at scope close does not leak
|
||
(`raw_buf_no_leak`, `live == 0`) — the drop call fires
|
||
(raw-buf.5).
|
||
6. `kernel_stub` is retired and `raw_buf` takes over its
|
||
ratification roles (raw-buf.6).
|
||
7. Workspace test suite green at each iteration boundary, plus the
|
||
new tests in § Testing strategy.
|
||
|
||
## Feature-acceptance argument
|
||
|
||
Applied per `design/contracts/0004-feature-acceptance.md`:
|
||
|
||
1. **LLM author reaches for it naturally.** RawBuf is the only path
|
||
to mutable indexed storage in AILang — the language has no other
|
||
"fill N slots and read them back" primitive. The worked consumer
|
||
program is what an LLM author writes for that need; there is no
|
||
alternative shape to reach for.
|
||
|
||
2. **Measurable improvement.** Embedding-ABI M5 measured per-tick
|
||
FFI at ~206 ns/tick (~658 ms / 3.19M EURUSD ticks). RawBuf as the
|
||
batch-FFI primitive amortises that per-tick cost across one
|
||
crossing. The Series milestone (#8) is also blocked on RawBuf;
|
||
shipping raw-buf removes that block.
|
||
|
||
3. **No reintroduced bug class.** Mutation stays mode-tracked:
|
||
`RawBuf.set` is `own → own`, not effect-tracked; no `RawBuf`
|
||
effect is declared. Uniqueness inference rewrites in-place; shared
|
||
ownership falls back to copy-on-write (Issue #22). The pure-core
|
||
invariant survives (code without a declared effect is pure); local
|
||
reasoning survives (a fn taking `borrow (RawBuf Int)` and
|
||
returning `Int` is pure from the caller's view). The new
|
||
scope-qualified mono symbols *remove* a latent bug class — the
|
||
`get__Int`-style collision between different types' identically
|
||
named polymorphic ops.
|