3ec406e687
raw-buf.1 (intercept registry) shipped (140a0c0). The intrinsic-bodies
milestone (#9) then landed and pulled the foundation out from under the
original raw-buf.2/.3 split, so this revises the spec.
Two foundation changes from intrinsic-bodies:
1. The placeholder-body convention the original spec leaned on
((body x) round-trip stubs) is gone. prelude.ail now uses
(intrinsic) markers, and check_fn (lib.rs:2173) typechecks every
non-intrinsic body including kernel-tier ones. RawBuf.get with a
placeholder body (declared (ret a), body constructs a RawBuf) is now
a hard type error — get/new/set/size MUST be (intrinsic).
2. An (intrinsic) marker now requires a matching INTERCEPTS entry as a
lockstep pair (intercepts_bijection_with_intrinsic_markers). The
"manifest visible / codegen deferred" intermediate state the original
raw-buf.2 was designed to ship is forbidden by that invariant — the
marker and its codegen entry must land in the same iteration.
Re-carve (3 remaining iterations, retirement isolated):
raw-buf.2 — kernel family-crate rename ailang-kernel-stub →
ailang-kernel. Pure refactor, zero behavioural change.
raw-buf.3 — RawBuf end-to-end: raw_buf module ((intrinsic) markers)
+ 12 INTERCEPTS entries + Term::New desugar + drop +
ailang-surface wiring + E2E. Manifest+codegen land
together. Workspace count 4 → 5.
raw-buf.4 — kernel_stub retirement: RawBuf subsumes every
ratification role (Term::New, param-in, kernel-tier
auto-import, intrinsic); answer entry + stub removed,
count 5 → 4.
Preserved unchanged: Goal, slab layout, feature-acceptance argument.
The raw_buf module Form-A source in § Concrete code shapes is
gate-verified: ail check → ok (35 symbols across 3 modules); the four
(intrinsic) markers are legal because the module is (kernel).
Grounding-check: 8/9 load-bearing current-behaviour assumptions
ratified by named green tests. The single override is the Term::New
codegen-deferral arm (lib.rs:2094 + sibling at lib.rs:3296, the latter
surfaced by the grounding agent) — a transient prep.2 marker that
raw-buf.3 deletes, not a relied-upon invariant; forward-ratified by
raw-buf.3's build-and-run E2E. Override logged, not a discard.
484 lines
24 KiB
Markdown
484 lines
24 KiB
Markdown
# raw-buf — Design Spec
|
||
|
||
**Date:** 2026-05-29 (revised 2026-05-29)
|
||
**Status:** Draft — awaiting user spec review
|
||
**Authors:** orchestrator + Claude
|
||
|
||
> **Revision note.** raw-buf.1 (intercept registry) shipped
|
||
> (`140a0c0`). This spec was then re-carved because the
|
||
> `intrinsic-bodies` milestone (#9) landed afterwards and removed
|
||
> the foundation the original raw-buf.2/.3 split stood on:
|
||
>
|
||
> 1. The **placeholder-body convention** the original spec relied
|
||
> on (`(body x)` round-trip stubs, "the existing convention from
|
||
> `examples/prelude.ail`") no longer exists. `prelude.ail` now
|
||
> uses `(intrinsic)` markers, and the checker (`check_fn`,
|
||
> `crates/ailang-check/src/lib.rs:2173`) typechecks every
|
||
> non-intrinsic body — including kernel-tier bodies. A
|
||
> placeholder body for `RawBuf.get` (declared `(ret a)`, body
|
||
> constructing a `RawBuf`) is now a hard type error. `get` (and
|
||
> `new`/`set`/`size`) MUST be `(intrinsic)`.
|
||
> 2. An `(intrinsic)` marker now requires a matching `INTERCEPTS`
|
||
> codegen entry as a **lockstep pair**, enforced by
|
||
> `intercepts::tests::intercepts_bijection_with_intrinsic_markers`.
|
||
> The "manifest visible, codegen deferred" intermediate state
|
||
> the original raw-buf.2 was designed to ship is therefore
|
||
> forbidden by an invariant. The marker and its codegen entry
|
||
> must land in the same iteration.
|
||
>
|
||
> Consequence: the kernel-manifest and the codegen now land
|
||
> together in one iteration. The remaining work is re-carved into
|
||
> three iterations (raw-buf.2/.3/.4) below. The Goal, slab layout,
|
||
> and feature-acceptance argument are unchanged.
|
||
|
||
## Goal
|
||
|
||
Ship `RawBuf a` — the first kernel-tier *base* extension — as a
|
||
mutable, indexed, bounded-size flat buffer of primitive elements
|
||
(`{Int, Float, Bool}`). RawBuf is the storage primitive that
|
||
unblocks two downstream needs both already captured in the
|
||
backlog:
|
||
|
||
- **Series (milestone #8)** — a bounded ring buffer with
|
||
financial-style indexing, implemented as a *library extension*
|
||
(pure AILang) wrapping `RawBuf a` plus bookkeeping fields.
|
||
- **Embedding-ABI batch FFI (subsumes closed #2)** — the M5
|
||
friction-harvest measured per-tick FFI at ~206 ns/tick on real
|
||
EURUSD volume (~658 ms / 3.19M ticks). A contiguous primitive
|
||
slice amortises that per-tick cost via a batch crossing.
|
||
RawBuf is that primitive.
|
||
|
||
The milestone also fulfils the architectural commitment from the
|
||
kernel-extensions whitepaper § "Plugin contract": the migration
|
||
of the hardcoded primitive-instance intercepts into one registry,
|
||
triggered by the first real base extension shipping. raw-buf.1
|
||
already discharged that migration (the registry is live and the
|
||
`intrinsic-bodies` bijection pin locks marker ↔ entry); RawBuf is
|
||
now the first *real-payload* consumer of that registry.
|
||
|
||
This makes raw-buf both a feature ship (a new type LLM authors
|
||
can reach for) and the proof that the registry path carries a
|
||
genuine domain type, not just the migrated primitive intercepts.
|
||
|
||
## Architecture
|
||
|
||
Four iterations. raw-buf.1 is done; .2/.3/.4 remain.
|
||
|
||
1. **Intercept registry** (raw-buf.1) — **DONE** (`140a0c0`).
|
||
Lifted the hard-coded `try_emit_primitive_instance_body` match
|
||
into the registry table `INTERCEPTS` in
|
||
`crates/ailang-codegen/src/intercepts.rs`; the dispatch site is
|
||
a `intercepts::lookup` call. Pure refactor, zero behavioural
|
||
change. (The subsequent `intrinsic-bodies` milestone added the
|
||
`(intrinsic)` marker, the `Term::Intrinsic` leaf, and the
|
||
marker ↔ entry bijection pin on top of this registry.)
|
||
|
||
2. **Kernel family-crate rename** (raw-buf.2) — Pure refactor,
|
||
zero behavioural change. Rename `crates/ailang-kernel-stub/` →
|
||
`crates/ailang-kernel/` and reshape it as a *family-crate*: one
|
||
Cargo crate hosting all kernel-tier modules as sibling Rust
|
||
submodules under `src/`, one submodule per `.ail` source. The
|
||
renamed crate carries `src/kernel_stub/{mod.rs, source.ail}`
|
||
(the existing stub, **verbatim including its current `answer`
|
||
intrinsic fn** — the source has grown since the original spec
|
||
was written). `src/lib.rs` re-exports each submodule's `SOURCE`
|
||
const under the stable name every consumer already binds
|
||
(`pub use kernel_stub::SOURCE as STUB_AIL;`). The acyclic chain
|
||
`ailang-surface → ailang-kernel → ailang-core` is preserved;
|
||
the parse hop `parse_kernel_stub` stays in `ailang-surface`. No
|
||
`raw_buf` submodule yet — this iteration is *only* the rename,
|
||
so its failure mode is "existing tests break", the cleanest
|
||
bisection target. Ratified by the existing suite green.
|
||
|
||
3. **RawBuf end-to-end** (raw-buf.3) — The feature ship, landing
|
||
the manifest and the codegen together (the lockstep pair the
|
||
bijection invariant requires). Adds:
|
||
- The `raw_buf` submodule (`src/raw_buf/{mod.rs, source.ail}`)
|
||
with the `RawBuf` TypeDef (`param-in (a Int Float Bool)`) and
|
||
the four operations `new`/`get`/`set`/`size`, each an
|
||
`(intrinsic)` marker (NOT a placeholder body).
|
||
- The 12 element-specialised `INTERCEPTS` entries (4 ops × 3
|
||
element types) lowering each via `@ailang_rc_alloc` +
|
||
getelementptr + load/store — no new C code. The entry `name`
|
||
of each MUST equal the mono-mangled symbol of its
|
||
`(intrinsic)` marker; the bijection test is the pin that
|
||
forces this and surfaces the exact expected strings.
|
||
- A desugar pass `(new T <types…> <values…>)` → an instantiated
|
||
`(app T.new <values…>)` that consumes the `NewArg::Type`
|
||
element type to drive monomorphisation to `new__RawBuf__<T>`,
|
||
so `Term::New` is eliminated before codegen. `Term::New`
|
||
becomes a checker-side-only construct (the codegen completion
|
||
the prep.2 deferral pointed at). raw-buf.3 removes **both**
|
||
`Term::New` deferral arms codegen currently carries — the
|
||
`lower_term` arm (`crates/ailang-codegen/src/lib.rs:2094`,
|
||
"milestone raw-buf" internal error) and the sibling
|
||
synth-path arm (`lib.rs:3296`). These arms are the deferral
|
||
marker prep.2 left; raw-buf.3 discharges them. They are not a
|
||
relied-upon current invariant — the milestone's correctness
|
||
depends on *removing* them, not on their persisting.
|
||
- Drop for primitive-element RawBuf: a single
|
||
`@ailang_rc_release` on the slab pointer (no element-walking;
|
||
elements are primitives).
|
||
- `ailang-surface` wiring: `parse_raw_buf` + the workspace
|
||
injection that makes `raw_buf` a kernel-tier auto-imported
|
||
module (workspace module count 4 → 5).
|
||
The worked consumer program runs end-to-end and prints `60`.
|
||
`kernel_stub` stays alongside this iteration (count 5).
|
||
|
||
4. **kernel_stub retirement** (raw-buf.4) — Focused removal.
|
||
RawBuf now subsumes every ratification role kernel_stub held:
|
||
`Term::New` end-to-end (the `(new RawBuf …)` consumer),
|
||
`param-in` reject (the Str-element reject fixture), kernel-tier
|
||
auto-import (RawBuf visible without `(import raw_buf)`), and the
|
||
`(intrinsic)` mechanism (RawBuf's four ops + their registry
|
||
entries). The stub is therefore pure redundancy. Delete
|
||
`src/kernel_stub/` + its re-export, drop `parse_kernel_stub` and
|
||
its surface injection, remove the `answer` `INTERCEPTS` entry +
|
||
`emit_answer` (its `(intrinsic)` marker leaves with the stub —
|
||
the bijection lockstep holds), delete the
|
||
`kernel_stub_module_round_trips` test, re-baseline `workspace_pin`
|
||
to 4 (raw_buf in, kernel_stub out — net unchanged), and update
|
||
`design/INDEX.md` + `design/models/0007-kernel-extensions.md`.
|
||
The `ailang-kernel` crate itself stays as the family-crate home
|
||
for future kernel-tier modules (series, matrix, …). Isolating
|
||
the retirement keeps each bijection-registry change (12 entries
|
||
added in .3, the `answer` entry removed in .4) in its own diff.
|
||
|
||
The ordering rationale: .2 is the only zero-behavioural-change
|
||
iteration (rename), the cleanest bisection target; .3 lands the
|
||
feature on a registry that has already absorbed every legacy
|
||
intercept (raw-buf.1) and on the renamed family-crate (raw-buf.2),
|
||
so the RawBuf manifest+codegen do not co-mingle with a registry
|
||
move or a crate move; .4 removes the now-redundant stub without
|
||
touching the new RawBuf code.
|
||
|
||
## Concrete code shapes
|
||
|
||
### Primary — the raw_buf kernel-tier module (raw-buf.3, gate-verified)
|
||
|
||
This is the exact Form-A source the `raw_buf` submodule embeds. It
|
||
parses and type-checks against the current tool (`ail check` →
|
||
`ok`); the four `(intrinsic)` markers are legal because the module
|
||
is `(kernel)`. The element type is restricted to `{Int, Float,
|
||
Bool}` via `param-in`. `RawBuf.get` returns the bare element type
|
||
`a`; `set` is linear (`own` in, `own` out).
|
||
|
||
```ail
|
||
(module raw_buf
|
||
(kernel)
|
||
(data RawBuf (vars a)
|
||
(doc "Mutable, indexed, fixed-size flat buffer of primitive elements. Opaque single-ctor handle; codegen intercepts emit raw alloc/load/store over an @ailang_rc_alloc slab. Element type restricted to {Int, Float, Bool} via param-in.")
|
||
(ctor B a)
|
||
(param-in (a Int Float Bool)))
|
||
(fn new
|
||
(doc "Allocate an uninitialised RawBuf of capacity n. Compiler-supplied (intrinsic) body; codegen intercept new__RawBuf__<T> emits @ailang_rc_alloc plus the i64 size header. Element bytes are uninitialised; caller must set before get.")
|
||
(type (forall (vars a) (fn-type (params (con Int)) (ret (own (con RawBuf a))))))
|
||
(params n)
|
||
(intrinsic))
|
||
(fn get
|
||
(doc "Indexed read. UB if i >= (size b); caller checks bounds via size. Compiler-supplied (intrinsic) body; codegen intercept emits getelementptr plus load.")
|
||
(type (forall (vars a) (fn-type (params (borrow (con RawBuf a)) (con Int)) (ret a))))
|
||
(params b i)
|
||
(intrinsic))
|
||
(fn set
|
||
(doc "Indexed write. Linear: own in, own out. Under uniqueness in-place; under shared copy-on-write (Issue #22). Compiler-supplied (intrinsic) body; codegen intercept emits getelementptr plus store.")
|
||
(type (forall (vars a) (fn-type (params (own (con RawBuf a)) (con Int) a) (ret (own (con RawBuf a))))))
|
||
(params b i v)
|
||
(intrinsic))
|
||
(fn size
|
||
(doc "Element count. Compiler-supplied (intrinsic) body; codegen intercept emits a single i64 load from the slab header.")
|
||
(type (forall (vars a) (fn-type (params (borrow (con RawBuf a))) (ret (con Int)))))
|
||
(params b)
|
||
(intrinsic)))
|
||
```
|
||
|
||
The `(ctor B a)` is a nominal fiction for the type system: it
|
||
gives `RawBuf` an identity and keeps `a` non-phantom. Real code
|
||
never constructs it — every operation is codegen-intercepted, and
|
||
`(new RawBuf …)` desugars to `(app RawBuf.new …)`, not to a
|
||
`(term-ctor RawBuf B …)`. The four `(intrinsic)` markers carry no
|
||
runnable body; codegen supplies the implementation via the
|
||
registry (raw-buf.3).
|
||
|
||
### Primary — the worked consumer program (raw-buf.3 acceptance)
|
||
|
||
This is the program an LLM author naturally writes for "I need a
|
||
small mutable indexed buffer to score N values and read them
|
||
back." It is the feature-acceptance criterion's empirical
|
||
evidence. It references the `raw_buf` module, which is only
|
||
injected once raw-buf.3 wires it through `ailang-surface` — so it
|
||
does **not** check against the current tool (RawBuf is not yet a
|
||
workspace module). Left unlabeled deliberately: it describes
|
||
post-raw-buf.3 surface, not current behaviour.
|
||
|
||
```
|
||
(module raw_buf_demo
|
||
(fn main
|
||
(type (fn-type (params) (ret (con Int))))
|
||
(params)
|
||
(body
|
||
(let buf (new RawBuf (con Int) 3)
|
||
(let buf (app RawBuf.set buf 0 10)
|
||
(let buf (app RawBuf.set buf 1 20)
|
||
(let buf (app RawBuf.set buf 2 30)
|
||
(app + (app RawBuf.get buf 0)
|
||
(app + (app RawBuf.get buf 1)
|
||
(app RawBuf.get buf 2))))))))))
|
||
; ail check && ail build && ail run → prints 60
|
||
```
|
||
|
||
No `(import raw_buf)` needed: `RawBuf` is visible via the
|
||
kernel-tier auto-import path (prep.3). `RawBuf.set` / `.get`
|
||
resolve via type-scoped namespacing (prep.1). `(new RawBuf (con
|
||
Int) 3)` is a `Term::New` whose `NewArg::Type` is `(con Int)`; the
|
||
raw-buf.3 desugar threads that into `new__RawBuf__Int`.
|
||
|
||
### Primary — the param-in reject fixture (raw-buf.3 must-fail)
|
||
|
||
```
|
||
(module raw_buf_reject_str
|
||
(fn cant_str
|
||
(type (fn-type (params (borrow (con RawBuf (con Str)))) (ret (con Str))))
|
||
(params buf)
|
||
(body (app RawBuf.get buf 0))))
|
||
; ail check → param-not-in-restricted-set: type Str not in {Int, Float, Bool}
|
||
```
|
||
|
||
Must fail at the checker with the `ParamNotInRestrictedSet`
|
||
diagnostic shipped in prep.3 (fieldtest F8). It is the must-fail
|
||
evidence that the `param-in` restriction is alive on a
|
||
real-payload extension, not just the prep.3 `StubT` fixture. Like
|
||
the worked program, it depends on `raw_buf` being injected, so it
|
||
is not gate-checkable against the current tool (unlabeled).
|
||
|
||
### Secondary — INTERCEPTS entry shape (raw-buf.3, implementation)
|
||
|
||
The real registry struct (raw-buf.1) is `Intercept { name,
|
||
expected_params, expected_ret, wants_alwaysinline, emit }`. The 12
|
||
RawBuf entries follow the existing rows' shape. One representative
|
||
(the exact symbol strings are derived by the implementer from the
|
||
bijection test, which mangles each `(intrinsic)` marker and
|
||
reports the expected name):
|
||
|
||
```rust
|
||
// crates/ailang-codegen/src/intercepts.rs — appended to INTERCEPTS
|
||
Intercept {
|
||
name: "new__RawBuf__Int", // == mono-mangled name of raw_buf.new @ a=Int
|
||
expected_params: &["i64"], // capacity n
|
||
expected_ret: "ptr", // slab base pointer
|
||
wants_alwaysinline: false,
|
||
emit: emit_new_raw_buf_int, // @ailang_rc_alloc(8 + n*8); store i64 n at offset 0
|
||
},
|
||
// + new__RawBuf__{Float,Bool}, RawBuf_get__{Int,Float,Bool},
|
||
// RawBuf_set__{Int,Float,Bool}, RawBuf_size__{Int,Float,Bool}
|
||
```
|
||
|
||
### Secondary — Term::New desugar (raw-buf.3, implementation)
|
||
|
||
`Term::New { type_name, args }` where `args: Vec<NewArg>`
|
||
(`NewArg::Type | NewArg::Value`, both already parsed + round-trip
|
||
ratified). The new desugar pass rewrites it to an application of
|
||
the type's `new` fn, instantiated at the element type carried by
|
||
the leading `NewArg::Type`:
|
||
|
||
```
|
||
(new RawBuf (con Int) 3) ; Term::New { type_name: "RawBuf",
|
||
; args: [Type(con Int), Value(3)] }
|
||
──desugar──▶
|
||
(app RawBuf.new 3) @ a = Int ; instantiation pins mono → new__RawBuf__Int
|
||
```
|
||
|
||
The desugar consumes the `NewArg::Type` for the instantiation and
|
||
passes only the `NewArg::Value` args as call arguments. After this
|
||
pass no `Term::New` survives into codegen; both deferral arms
|
||
(`lib.rs:2094` in `lower_term`, "milestone raw-buf" internal
|
||
error, and the sibling at `lib.rs:3296` in the synth path) become
|
||
unreachable and are removed. Whether
|
||
the prep.2 checker already accepts the `NewArg::Type`-bearing
|
||
`Term::New` semantically, or needs a small extension, is a
|
||
raw-buf.3 task to verify first (the form parses + round-trips
|
||
today; the lift pass passes `NewArg::Type` through untouched).
|
||
|
||
### Secondary — slab layout (raw-buf.3)
|
||
|
||
```
|
||
[ size : i64 ][ elem_0 ][ elem_1 ] ... [ elem_{size-1} ]
|
||
^ ^ ^
|
||
offset 0 offset 8 offset 8 + size*sizeof(T)
|
||
```
|
||
|
||
Element widths: `Int = 8`, `Float = 8`, `Bool = 1`. Header is
|
||
i64, 8-byte aligned; element payload follows directly.
|
||
`@ailang_rc_alloc` is called with `8 + size * sizeof(T)` and
|
||
returns the slab base `ptr`. The size header is self-describing,
|
||
so drop and `RawBuf.size` need no side-table. Drop is a single
|
||
`@ailang_rc_release` on the slab pointer — primitive elements
|
||
carry no recursive drops. The per-TypeDef drop generation gets a
|
||
"primitive-element-RawBuf" arm dispatched via the same registry.
|
||
|
||
## Components
|
||
|
||
| Component | Iter | Files |
|
||
|-----------|------|-------|
|
||
| Intercept registry | raw-buf.1 (DONE) | `crates/ailang-codegen/src/intercepts.rs` + dispatch site in `lib.rs` |
|
||
| Kernel family-crate rename | raw-buf.2 | `git mv crates/ailang-kernel-stub crates/ailang-kernel`; reshape to `src/lib.rs` + `src/kernel_stub/{mod.rs, source.ail}` (existing stub source incl. `answer` re-housed verbatim); `Cargo.toml` package + workspace member + dep-alias renames; `crates/ailang-surface/Cargo.toml` dep rename + `loader.rs` use-path renames; `design/INDEX.md` + `CLAUDE.md` code-layout row |
|
||
| RawBuf module + codegen + desugar + drop | raw-buf.3 | new `crates/ailang-kernel/src/raw_buf/{mod.rs, source.ail}` + `lib.rs` re-export; `parse_raw_buf` + workspace injection in `crates/ailang-surface/src/loader.rs` + re-export in `lib.rs`; 12 entries + emit fns in `crates/ailang-codegen/src/intercepts.rs`; `Term::New` desugar in `crates/ailang-core/src/desugar.rs` (and removal of the codegen `Term::New` error arm in `crates/ailang-codegen/src/lib.rs`); primitive-element-RawBuf drop arm; `workspace_pin` 4 → 5; round-trip + E2E tests |
|
||
| kernel_stub retirement | raw-buf.4 | delete `crates/ailang-kernel/src/kernel_stub/` + its `lib.rs` re-export; drop `parse_kernel_stub` + injection in `ailang-surface`; remove `answer` entry + `emit_answer` in `intercepts.rs`; delete `kernel_stub_module_round_trips` test; `workspace_pin` 5 → 4; `design/INDEX.md` + `design/models/0007-kernel-extensions.md` updates |
|
||
|
||
## Data flow
|
||
|
||
**Build-time** (raw-buf.3 onwards):
|
||
1. CLI loads workspace → `ailang-surface` injects `raw_buf` as a
|
||
kernel-tier module (`kernel: true`) just like `prelude`.
|
||
2. `check` walks consumer code; `(con RawBuf (con Int))` resolves
|
||
to `raw_buf.RawBuf` via the kernel-tier auto-import path
|
||
(prep.3).
|
||
3. `param-in` on RawBuf's `a` rejects non-{Int,Float,Bool}
|
||
instantiations at the consumer typecheck site, using the
|
||
existing `ParamNotInRestrictedSet` diagnostic.
|
||
|
||
**Codegen-time** (raw-buf.3 onwards):
|
||
1. The desugar pass replaces every `Term::New { type_name, args }`
|
||
with an instantiated `(app <type_name>.new <values>)`, pinning
|
||
the element type from the leading `NewArg::Type`.
|
||
2. For each fn, codegen reaches the registry dispatch site. If the
|
||
fn's mono name resolves via `intercepts::lookup`, the entry's
|
||
`emit` runs; the `(intrinsic)` marker carries no body to fall
|
||
back to. Otherwise the normal AILang lowering runs.
|
||
3. For RawBuf consumers, `new__RawBuf__Int` emits the alloc;
|
||
`RawBuf_set__Int` a `store i64`; `RawBuf_get__Int` a `load
|
||
i64`; `RawBuf_size__Int` a single header load.
|
||
|
||
**Bijection interaction** (raw-buf.3 / .4):
|
||
Each RawBuf `(intrinsic)` marker reachable from the workspace is
|
||
collected by `workspace_intrinsic_markers()` and must resolve to
|
||
an `INTERCEPTS` entry (and vice versa, minus the
|
||
`OPTIMISATION_ONLY` allowlist). raw-buf.3 adds 12 markers + 12
|
||
entries together. raw-buf.4 removes the `answer` marker (with the
|
||
stub) and the `answer` entry together. A half-landed change on
|
||
either side is a hard unit-test failure
|
||
(`intercepts_bijection_with_intrinsic_markers`).
|
||
|
||
**Workspace module count:** prelude + kernel_stub + 2 test
|
||
fixtures = 4 today. raw-buf.3 adds raw_buf → 5. raw-buf.4 retires
|
||
kernel_stub → 4. The `workspace_pin` test pins each transition.
|
||
|
||
## Error handling
|
||
|
||
Two consumer-facing diagnostics; both already shipped, no new arms
|
||
needed:
|
||
|
||
- **`param-not-in-restricted-set`** (prep.3, fieldtest F8). Fires
|
||
when a consumer writes `(con RawBuf (con Str))` or any
|
||
non-{Int,Float,Bool} element. Names the type, the variable, the
|
||
TypeDef, and the restricted set.
|
||
- **`new-type-not-constructible`** (prep.3, fieldtest F5). Fires
|
||
when a consumer writes `(new SomeType …)` and `SomeType` has no
|
||
`(fn new …)` in its home module. raw_buf ships `(fn new …)`, so
|
||
the worked program passes; the diagnostic still protects future
|
||
authors who write `(new Counter 0)` against a `new`-less type.
|
||
|
||
The codegen `Term::New` deferral arms (`lib.rs:2094` in
|
||
`lower_term`, message "… milestone raw-buf"; and the sibling at
|
||
`lib.rs:3296` in the synth path) are **removed** in raw-buf.3 once
|
||
the desugar eliminates `Term::New` before codegen. Neither is a
|
||
user-facing diagnostic — both are the deferral marker prep.2 left,
|
||
which raw-buf.3 discharges. They have no test pinning them today
|
||
(they are an internal `CodegenError::Internal`, not an asserted
|
||
behaviour); raw-buf.3's end-to-end build-and-run E2E (the worked
|
||
program printing `60`) is the forward ratification — it exercises
|
||
the `Term::New` → desugar → codegen path that replaces them.
|
||
|
||
Bounds checks are **not** in scope (whitepaper § "RawBuf — the
|
||
base extension"): `RawBuf.get/.set` are UB if `i >= size(b)`;
|
||
caller checks via `RawBuf.size`. Bounds-check insertion is a
|
||
future axis, no backlog item yet.
|
||
|
||
## Testing strategy
|
||
|
||
**raw-buf.2 (rename):**
|
||
- All existing workspace tests pass unchanged — the rename is a
|
||
pure refactor. Zero test-count change. The crate-rename's
|
||
failure mode is "the workspace does not build / existing tests
|
||
break", caught by `cargo build --workspace` + `cargo test
|
||
--workspace`.
|
||
|
||
**raw-buf.3 (RawBuf end-to-end):**
|
||
- Round-trip: `raw_buf_module_round_trips` (mirror of
|
||
`kernel_stub_module_round_trips` in `design_schema_drift.rs`) —
|
||
parse → serialise → round-trip; assert `kernel == true`, name
|
||
`raw_buf`, and `RawBuf` TypeDef `param_in` for `a` contains all
|
||
of Int/Float/Bool.
|
||
- E2E `raw_buf_int_e2e`: the worked consumer program — `ail check
|
||
&& ail build && ail run` → prints `60`.
|
||
- E2E `raw_buf_float_e2e` / `raw_buf_bool_e2e`: Float and Bool
|
||
element variants (Bool catches any i1/i8 packing question).
|
||
- E2E `raw_buf_param_in_reject_e2e`: the Str-element reject
|
||
fixture — `ail check` exits non-zero with
|
||
`param-not-in-restricted-set` in stderr.
|
||
- Drop ratification: a consumer that allocates a RawBuf and lets
|
||
it go out of scope without a leak — re-uses the existing
|
||
leak-check harness, no new infra.
|
||
- Bijection: `intercepts_bijection_with_intrinsic_markers` stays
|
||
green with the 12 new markers + 12 new entries (the test forces
|
||
the exact mangled-name match).
|
||
- `workspace_pin` count assertion 4 → 5 (+ `contains_key("raw_buf")`).
|
||
|
||
**raw-buf.4 (retirement):**
|
||
- `workspace_pin` re-baselines 5 → 4 (raw_buf in, kernel_stub
|
||
out); `contains_key("kernel_stub")` assertion removed,
|
||
`contains_key("raw_buf")` retained.
|
||
- `kernel_stub_module_round_trips` deleted (its `parse_kernel_stub`
|
||
callee is gone; `raw_buf_module_round_trips` carries the
|
||
round-trip invariant forward).
|
||
- Bijection stays green after the `answer` marker + entry leave
|
||
together.
|
||
- The full suite stays green; the diff is a removal, ratified by
|
||
"nothing references the stub and everything still builds".
|
||
|
||
## Acceptance criteria
|
||
|
||
The milestone ships when:
|
||
|
||
1. The worked consumer program runs end-to-end: `ail check && ail
|
||
build && ail run` prints `60` (raw-buf.3).
|
||
2. The param-in reject fixture exits at `ail check` with the
|
||
prep.3 `param-not-in-restricted-set` diagnostic (raw-buf.3).
|
||
3. RawBuf's four operations are `(intrinsic)` markers, each paired
|
||
with an `INTERCEPTS` entry; `intercepts_bijection_with_intrinsic_markers`
|
||
is green (raw-buf.3).
|
||
4. `Term::New` no longer reaches codegen — the desugar eliminates
|
||
it and both deferral arms (`lib.rs:2094` + `lib.rs:3296`) are
|
||
removed (raw-buf.3).
|
||
5. `ailang-kernel-stub` is renamed to the `ailang-kernel`
|
||
family-crate (raw-buf.2); `kernel_stub` is retired and `raw_buf`
|
||
takes over its ratification roles (raw-buf.4).
|
||
6. Workspace test suite green at each iteration boundary, plus the
|
||
new tests enumerated in § Testing strategy.
|
||
|
||
## Feature-acceptance argument
|
||
|
||
Applied per `design/contracts/0004-feature-acceptance.md`:
|
||
|
||
1. **LLM author reaches for it naturally.** RawBuf is the only
|
||
path to mutable indexed storage in AILang — the language has no
|
||
other "fill N slots and read them back" primitive. The worked
|
||
consumer program above is what an LLM author writes for that
|
||
need; there is no alternative shape to reach for.
|
||
|
||
2. **Measurable improvement.** Embedding-ABI M5 measured per-tick
|
||
FFI at ~206 ns/tick (~658 ms / 3.19M EURUSD ticks). RawBuf as
|
||
the batch-FFI primitive amortises that per-tick cost across one
|
||
crossing. The Series milestone (#8) is also blocked on RawBuf;
|
||
shipping raw-buf removes that block.
|
||
|
||
3. **No reintroduced bug class.** Mutation discipline stays
|
||
mode-tracked (whitepaper § "RawBuf — the base extension"):
|
||
`RawBuf.set` is `own → own`, not effect-tracked. No `RawBuf`
|
||
effect is declared. Uniqueness inference rewrites in-place;
|
||
shared ownership falls back to copy-on-write (Issue #22). The
|
||
pure-core invariant survives: code without a declared effect is
|
||
pure. Local reasoning survives: a fn taking `borrow (RawBuf
|
||
Int)` and returning `Int` is pure from the caller's view,
|
||
because the buffer is not mutated.
|