# Kernel extensions — plugin-style domain types whitepaper **STATUS.** Mechanisms milestone closed 2026-05-28 (`kernel-extension-mechanics`; iterations prep.1, prep.2, prep.3 landed). The four language-level mechanisms — type-scoped namespacing, `Term::New`, kernel-tier modules + auto-import, and `param-in` — are shipped and ratified by the stub kernel crate (`crates/ailang-kernel-stub/`). The base-extension and library- extension milestones (`raw-buf`, `series`) are pending. This whitepaper describes the design as a coherent whole; the per-milestone specs in `docs/specs/` carry the implementation- level detail. Sections describing the now-shipped mechanisms read in present-state per the [honesty-rule](../contracts/0007-honesty-rule.md); sections about the still-pending milestones keep their forward-looking framing. ## The problem AILang's core needs to stay small. Five primitive types (`Int`, `Bool`, `Unit`, `Str`, `Float`), algebraic data types, typeclasses, algebraic effects, RC + uniqueness — that is the language. New primitive types are a tax: they inflate the schema, the checker, the codegen, the documentation surface, and the LLM-author's mental model of what the language is. But: AILang authors will work in specific domains. Streaming analytics wants a bounded ring buffer. Numerical work wants flat arrays and matrices. Financial work wants fixed-precision decimals and timestamps with arithmetic. None of these belong in `ailang-core` — yet without them, the LLM author has to hand-roll linked-list approximations that are algorithmically wrong (O(N²) where O(N) is the natural complexity). The resolution: **kernel extensions** — a plugin contract that lets a domain-specific type live in its own crate, register with the compiler at boot time, and present itself to user code as if it were a built-in primitive. Core has no domain knowledge of any specific extension; extensions have no core dependency beyond the plugin contract. The first consumer is `Series` — a bounded ring buffer for streaming data. Future consumers are not specified here but the shape is intentional: each new domain type is one new crate, zero core changes. ## Worked example — `Series` for streaming analytics What an LLM-author writes when given the task "compute simple moving average over a stream of float values, window size 3": ``` (module sma_demo (data FloatList (ctor FNil) (ctor FCons (con Float) (con FloatList))) (fn sum_window_step (type (fn-type (params (borrow (Series (con Float))) (con Int) (con Int) (con Float)) (ret (con Float)))) (params s n i acc) (body (if (app eq i n) acc (tail-app sum_window_step s n (app + i 1) (app + acc (app Series.at s i)))))) (fn emit_if_full (type (fn-type (params (borrow (Series (con Float))) (con Int)) (ret (con Unit)) (effects IO))) (params s n) (body (if (app >= (app Series.total_count s) n) (app print (app / (app sum_window_step s n 0 0.0) (app int_to_float n))) (do io/print_str "")))) (fn run_stream (type (fn-type (params (own (Series (con Float))) (con Int) (con FloatList)) (ret (con Unit)) (effects IO))) (params s n input) (body (match input (case (pat-ctor FNil) (do io/print_str "")) (case (pat-ctor FCons v rest) (let s_new (app Series.push s v) (seq (app emit_if_full s_new n) (tail-app run_stream s_new n rest))))))) (fn main (type (fn-type (params) (ret (con Unit)) (effects IO))) (params) (body (let s (new Series (con Float) 3) (app run_stream s 3 (term-ctor FloatList FCons 1.0 (term-ctor FloatList FCons 5.0 (term-ctor FloatList FCons 3.0 (term-ctor FloatList FCons 8.0 (term-ctor FloatList FCons 6.0 (term-ctor FloatList FCons 2.0 (term-ctor FloatList FNil))))))))))) ``` Four constructs in this program are not in today's AILang. Each maps onto one of the four mechanisms this whitepaper defines: - `Series` appears in type position with no module qualifier (`(con Series (con Float))`) — *kernel-tier modules*. - `(new Series (con Float) 3)` constructs the series — *new term construct*. - `(app Series.at s i)`, `(app Series.total_count s)`, `(app Series.push s v)` — *type-scoped namespacing*. - The `(own (Series ...))` mode on `run_stream` plus the linear state-threading via `(let s_new ...)` is the *uniqueness mode* discipline that already exists in AILang. Series carries no separate algebraic effect; its mutation is mode-tracked, not effect-tracked. Under uniqueness inference (Gitea #22), the rebuild-on-push compiles to in-place mutation. A fifth mechanism, `param-in`, is not visible at the call site but enforces that `(con Series (con Str))` would be rejected. ## The four mechanisms ### 1. Type-scoped namespacing **Form.** `.` resolves to the `` definition in the home module of ``. The home module of a type `T` is the module whose `defs` contains a `TypeDef` named `T`. **Canonical form.** Type-scoped access is the *canonical* form for operations associated with a type. The pre-existing module-scoped form (`.`) remains a valid surface only for free-standing definitions not bound to a specific type receiver, and as a collision disambiguator if two types in the workspace ever shared a name (which is not the case today). **Examples.** ``` (app Series.at s i) ; series_at via type-scope (app Maybe.from_maybe 0 m) ; from_maybe via type-scope (was: std_maybe.from_maybe) (con Series (con Float)) ; type itself accessible bare under kernel-tier (con Maybe (con Int)) ; type itself, if Maybe is in scope via import ``` **Why this is LLM-natural.** When asked "what can I do with a Series?", an LLM produces `Series.` queries. When asked "what can I do with a Maybe?", it produces `Maybe.` queries. The type, not the module, is the LLM's mental anchor. Module-scoped access requires knowing where a function lives — extra cognitive load with no expressive benefit. **Schema impact.** Zero. The dotted form already parses today as `Term::Var { name: "Maybe.from_maybe" }` (the `.` is a permitted identifier character in the existing lexer). What changes is the *resolver*: it tries the type-namespace branch first, falls back to the module-namespace branch. **Diagnostics.** `TypeScopedMemberNotFound` (the receiver is a known type, but the member is not defined in its home module); `TypeScopedReceiverNotAType` (the receiver name is neither a module nor a type). ### 2. The `new` term construct **Form.** `(new T arg1 arg2 ...)` where `T` is a `TypeName` and `argN` is either a `Type` expression (e.g. `(con Float)`) or a `Term` expression. The construct desugars to: look up `new` in the home module of `T`, call it with the given args. **AST.** New variant `Term::New { type_name: TypeName, args: Vec }` where `NewArg = NewArg::Type(Type) | NewArg::Value(Term)`. Each arg carries its kind in serialised JSON. **Why complementary to `term-ctor`, not replacing it.** `term-ctor` constructs an ADT value via a *named data constructor* (a tag in the TypeDef's `ctors` list). `new` calls a *function* in the type's home module. They serve different purposes: - `(term-ctor IntList ICons 5 INil)` builds an `IntList` from its `ICons` ctor. This is structural — the AST node carries the ctor name and field args directly. - `(new Series (con Float) 3)` calls `Series.new : (Type, Int) -> Series a`. This is functional — there is no `Series` data ctor; the type is opaque and construction is a defined operation. A type can have both: an ADT could expose `term-ctor` for direct ctor access plus a `new` def for builder-style construction. The two paths are not in tension; they answer different design needs. **Diagnostics.** `NewTypeNotConstructible` (no `new` def in the type's home module); `NewArgKindMismatch` (a Type arg where the signature expects a Value, or vice versa). ### 3. Kernel-tier modules with auto-import **Form.** A new boolean field `kernel` on `Module` (omitted when false, so existing modules hash-stable). When `kernel: true`: - The module's top-level defs are accessible bare in every other module of the workspace, with no `(import ...)` declaration required. - The module's types are accessible bare in type position. **Why.** The architectural property the user wants is: the domain type lives outside core, but at the call site it feels like a primitive. Bare `Series` in `(con Series (con Float))` without seeing `(import series)` somewhere is the ergonomic property. **Single mechanism.** Auto-injection is a single load-time hop, generalised from a hardcoded single-name path (prelude only, pre-prep.3) to a flag-driven multi-module mechanism (since prep.3 of `kernel-extension-mechanics`). The loader at `crates/ailang-surface/src/loader.rs::load_workspace` injects `parse_prelude()` and `parse_kernel_stub()` programmatically, then derives the implicit-imports list from `modules.values().filter(|m| m.kernel)`. Every module in the workspace's modules map that carries `kernel: true` enters the auto-import set; consumers see those modules' top-level types bare without an `(import …)` declaration. Prelude carries `kernel: true` in `examples/prelude.ail`. Its 12 free fns (see `examples/prelude.ail:85-148`) remain callable bare in every consumer module — ratified by `crates/ail/tests/prelude_free_fns.rs` across the code-path migration. The ratifying stub kernel module (`crates/ailang-kernel-stub/`) and any future kernel-tier consumers (Series, Matrix) auto-import through the same filter. Class-method dispatch (a separate mechanism — see [method dispatch](../contracts/0016-method-dispatch.md)) is orthogonal: the dispatch is about how method calls are resolved; the auto-import is about which names are in scope. ### 4. `param-in` — closed-set type parameter restriction **Form.** Optional field on `TypeDef`: `param-in: Map>`. Skipped when absent. Each entry constrains a type variable to a fixed set of named types. **Example.** ```jsonc // TypeDef Series { "kind": "type", "name": "Series", "vars": ["a"], "ctors": [], "param-in": { "a": ["Int", "Float", "Bool"] } } ``` When the checker sees `Type::Con { name: "Series", args: [Type::Con { name: "Str", ... }] }`, it looks up `Series`'s `param-in`, finds that `a` must be in `{Int, Float, Bool}`, and rejects `Str`. **Why not a marker class.** `class Primitive a` with three instances (Int, Float, Bool) and no methods would express the same restriction via existing typeclass machinery. Reason it is not chosen: class constraints model *behaviour* (a type satisfies some interface). `param-in` models *structural identity* (the type's storage layout). Series's element-type restriction is structural — the C-runtime ring buffer has a `data[]` whose layout depends on whether the element is 8 bytes (Int/Float) or 1 byte (Bool). A marker class would model this as "Int is a Primitive" which is true but indirect. `param-in` says directly: "Series's element variable must be one of these three named types". The typecheck site is a set-membership lookup, not a class-instance discharge. **Diagnostic.** `ParamNotInRestrictedSet` names the offending type and the allowed set. ## Two-tier extension architecture Kernel extensions split into two tiers, distinguished by what they need to provide: ### Base extensions Provide primitives that are *not expressible in AILang itself* — typically because they require mutable indexed storage, hardware/OS interaction, or library-wrapping that has no AILang surface. A base extension ships: 1. **A kernel-tier module manifest** with the TypeDef (and `param-in` if applicable) plus the operation signatures. 2. **Codegen intercepts** — Rust code registered with the compiler that, at each call site of an operation, emits the appropriate LLVM IR text (using existing runtime symbols like `@ailang_rc_alloc` for allocation). This follows the `try_emit_primitive_instance_body` precedent: Rust returns IR text, no separate C glue. 3. **Optional C runtime support** in `runtime/.c` — only when the operations *truly cannot be expressed as IR over existing runtime primitives* (e.g. wrapping PCRE2 for a future Regex extension). For the foreseeable base extensions (RawBuf, Matrix), no C is needed. The first base extension is `RawBuf T` — a mutable, indexed, bounded-size flat buffer of primitive elements. Element type is restricted to `{Int, Float, Bool}` via `param-in` for the MVP. ### Library extensions Provide domain-specific types that *are* expressible in AILang once a base extension has been added. A library extension ships: 1. **A kernel-tier `.ail` module** (or `.ail.json`) — plain AILang code, no Rust intercepts, no C. The module declares an ADT and a set of fns; the fns call into a base extension's API. 2. **Nothing else.** No codegen intercepts (regular AILang codegen handles it), no C runtime (regular AILang RC handles it), no compiler-side plug-ins. The first library extension is `Series T` — a bounded ring buffer with financial-style indexing. Implemented as an ADT wrapping a `RawBuf T` plus four bookkeeping `Int` fields (lookback, head, count, total). All operations are AILang fns. ### Why split - **AILang-in-AILang where possible.** Domain types are written in the language they are for. A LLM author who needs to understand Series's eviction logic reads `series.ail`, not a Rust intercept registry. The implementation language matches the authoring language. - **Composability.** Multiple library extensions can share one base extension. Series, Matrix, Hashmap, and future Vector all build on RawBuf. Each is a few hundred lines of AILang, not a new Rust intercept. - **Cost localisation.** The Rust-intercept layer is small and centralised at the base. Domain growth (more library extensions) does not enlarge the Rust footprint. - **Forward axis: SoA for records.** When the day comes that Series should hold records (e.g. `{price: Float, volume: Int}`), the change is internal to RawBuf — its intercept dispatches on whether the element type is primitive (flat buffer) or record (struct-of-arrays). Series's AILang code does not change. ## The plugin contract (consolidated) Base extension: - Kernel-tier module manifest. - Rust codegen intercepts emitting LLVM IR for each operation. - Optional C runtime — only when LLVM IR over existing runtime symbols cannot express the operation. Library extension: - Kernel-tier `.ail` module. - Depends on one or more base extensions for its primitives. Both register with the compiler via the kernel-tier auto-injection mechanism described above. Core has no per-extension code. The four mechanisms (type-scoped namespacing, `new`, kernel-tier modules, `param-in`) are all general — the same checker logic handles RawBuf, Series, and any future extension. The migration of existing hardcoded primitive-instance intercepts (`try_emit_primitive_instance_body`) into the plugin registry is triggered by the first real base extension shipping (RawBuf). One registry mechanism for all intercepts, replacing the current single-case hardcoded list. CLI / discovery (`ail describe `) consults the workspace TypeDef registry and surfaces the `kernel: true` flag on the home module — uniform for base and library extensions; the user does not need to know which tier hosts the type. ## Series as the first library consumer (built on RawBuf) ### RawBuf — the base extension underneath `RawBuf a` is a mutable, indexed, fixed-size buffer. Element type `a` restricted to `{Int, Float, Bool}` for the MVP via `param-in`. API: - `(new RawBuf (con T) (size: Int)) : own (RawBuf T)` — allocates an uninitialised buffer. - `RawBuf.get : forall a. (borrow (RawBuf a), Int) -> a` — indexed read. UB if `index >= RawBuf.size(b)`; caller checks bounds. - `RawBuf.set : forall a. (own (RawBuf a), Int, a) -> own (RawBuf a)` — indexed write. Linear signature: caller hands over the buffer, callee returns it after mutation. Under uniqueness inference, the codegen rewrites this into in-place mutation; under shared ownership it falls back to a full buffer copy (Issue #22 territory). - `RawBuf.size : forall a. (borrow (RawBuf a)) -> Int`. Mutation discipline: ownership-mode-tracked, not effect-tracked. The own/borrow signatures *are* the mutation contract. No `RawBuf` effect is declared. This is consistent with AILang's existing memory-model — RC + uniqueness as the canonical mutation story. Storage at the LLVM-IR level is type-specialised: the Rust intercept registered for `RawBuf` dispatches on the element type at each call site and emits the appropriate `getelementptr` / `load` / `store` instructions over an opaque `ptr` allocated via `@ailang_rc_alloc`. No new C code; the existing RC machinery handles allocation, drop, and copy-on-share. ### Series — the library extension `Series a` is a bounded ring buffer with financial-style indexing (index 0 = newest). It is a plain AILang ADT in a kernel-tier `.ail` module: ``` (module series (kernel) (data Series (vars a) (param-in (a Int Float Bool)) (ctor S (own (RawBuf a)) ; the storage (con Int) ; lookback (con Int) ; head index (con Int) ; current count (con Int))) ; total pushes ever (fn new (type (fn-type (params (con Int)) (ret (own (Series a))))) (params lookback) (body (term-ctor Series S (new RawBuf lookback) lookback 0 0 0))) (fn push (type (fn-type (params (own (Series a)) (con a)) (ret (own (Series a))))) (params s v) (body (match s (case (pat-ctor S buf lookback head count total) (term-ctor Series S (app RawBuf.set buf head v) lookback (app % (app + head 1) lookback) (if (app < count lookback) (app + count 1) count) (app + total 1)))))) (fn at (type (fn-type (params (borrow (Series a)) (con Int)) (ret (con a)))) (params s i) (body (match s (case (pat-ctor S buf lookback head count total) (app RawBuf.get buf (app % (app + (app - head 1) (app + (app * lookback 2) i)) lookback)))))) (fn len (params s) (body (match s (case (pat-ctor S buf lookback head count total) count)))) (fn total_count (params s) (body (match s (case (pat-ctor S buf lookback head count total) total))))) ``` Push semantics: take Series by `own`, return a new Series with the buffer mutated (via `RawBuf.set`) and the bookkeeping fields updated. The match-and-rebuild pattern is the explicit linear state-threading style. Under uniqueness inference, both the RawBuf mutation and the Series ADT rebuild become in-place; the caller-side `(let s_new (app Series.push s v) ...)` pattern optimises to single-allocation, two-counter-update cost. Under shared ownership, push allocates a new Series wrapper (~40 bytes RC overhead) per call — the performance fallback that Issue #22 will narrow. No `Series` effect. The mutation is mode-tracked. Caller signatures do *not* gain a `[Series]` or `[RawBuf]` effect entry; they only need the linear threading discipline. `Series.at` uses financial-style indexing (0 = newest): `buf_index = (head - 1 - i + 2*lookback) mod lookback`. UB if `i >= count`; the caller checks via `Series.len` or `Series.total_count`. The `+ 2*lookback` is a non-negative adjustment so the modulo arithmetic stays well-defined for small `head` values. ### Forward axis: SoA for records (already accommodated) Extending Series to hold records (`Series {price: Float, volume: Int}`) is purely a RawBuf change. The RawBuf intercept gains a dispatch path for record element types — primitive types stay flat (AoS), record types switch to struct-of-arrays internally with per-field parallel storage. `RawBuf.get` for a record-typed buffer materialises the record on demand from the parallel slots; `RawBuf.set` distributes the record's fields across the parallel slots. Series's AILang code does not change; the API surface is identical. The Series milestone scope does *not* include record element types; that is a future axis. But the architecture accommodates it without revisiting Series itself. ## Feature-acceptance argument Series's three-clause feature-acceptance check is in the spec; the summary: LLM authors naturally reach for a bounded ring buffer for streaming workloads (clause 1 — the SMA worked example above is the evidence); the type structurally eliminates a class of off-by-one and warmup-handling bugs (clause 2); the bounded push-only mutation surface — gated by uniqueness mode-tracking on the owned `Series` value (not by an algebraic effect; see §"Coexistence" below) — does not reintroduce the iterated- mutable-state bug class that the `mut`/`var`/`assign` removal addressed (clause 3 — see `docs/specs/0052-kernel-extension-mechanics.md`'s clause-3 discussion). The same three-clause check applies to *each future kernel extension*. The whitepaper does not pre-justify Matrix or Decimal or Time — each gets its own feature-acceptance gate when its spec is written. ## Migration policy (pre-production frame) AILang is pre-production. No external author depends on a particular surface; no compiled artifact in the wild. The implication for kernel-extensions: **when a new mechanism is strictly better than an existing one for the use case it covers, the existing one is retired or repositioned, not preserved in parallel for compatibility.** Concrete state after the mechanisms milestone closed (2026-05-28): - Type-scoped namespacing makes `.` non-canonical for type-associated operations. The example and fixture corpus has been rewritten from `std_maybe.from_maybe`, `std_pair.from_pair`, etc., to type-scoped form. The hash-pin blast-radius audit at prep.1 plan time enumerated the affected pin files; only `crates/ailang-surface/tests/prelude_module_hash_pin.rs` needed refresh in this milestone (at prep.3, when prelude gained `(kernel)`). - The pre-existing `BareCrossModuleTypeRef` / `BadCrossModuleTypeRef` diagnostics are repurposed: what they considered "bad" before (a bare type ref to a foreign module's type) is still bad when the type is not in scope, but the *correct* form is now type-scoped (when feasible) rather than module-qualified. - The prelude module carries `kernel: true`. The previously hardcoded prelude-name paths in `crates/ailang-surface/src/loader.rs` (the `&["prelude"]` literal) and the matching reservation + contract docs in `crates/ailang-core/src/workspace.rs` have been replaced by a generic `modules.values().filter(|m| m.kernel)` derivation. Consumer-observable behaviour for prelude is unchanged (the 12 free fns remain callable bare in every consumer — `crates/ail/tests/prelude_free_fns.rs` is the regression pin); the code path no longer hardcodes any module name as special. - Codegen intercepts: the pre-existing `try_emit_primitive_instance_body` hardcoded list is **not yet** migrated into a plugin registry — that migration is deferred to the Series milestone, when there is the first real external consumer and the mechanism can graduate from "hardcoded for one case" to "registry for many cases". Single-consumer registries are premature mechanism. The reason migration cost is not a decision driver: there is nothing to break externally, and rewrites inside the workspace are cheap. Choosing the right design now is the priority; cost of refactoring tests and fixtures is the project's own problem and is amortised over zero external consumers. ## Coexistence with existing mechanisms The mechanisms in this whitepaper interact with several pre-existing parts of the language. Each interaction is named here so the spec can quote the position: - **Classes / method dispatch.** Orthogonal. Type-scoped namespacing accesses *top-level defs* in a type's home module; class-method dispatch (see [method dispatch](../contracts/0016-method-dispatch.md)) resolves *class method calls* by type-driven instance lookup. Both can coexist on the same type — e.g. `Series` could someday have a `Show` instance (class method `show` dispatches via the instance) and a `Series.dump` free-fn def (accessed via type-scope). Both mechanisms remain. - **Algebraic effects.** Not extended by Series. Mutation discipline lives in the uniqueness/mode system (see `(own Series)` in caller signatures); the algebraic-effects set is unchanged. - **RC + uniqueness.** Series values are RC-managed like any heap value; drop is via a kernel-supplied `@ailang_series_drop`. Future Uniqueness inference (Gitea #22) can in principle recognise unique Series and elide RC ops on push — but the baseline does not depend on that. - **Heap-Str ABI.** The Series runtime borrows the heap-Str RC-pointer-to-opaque-struct pattern (see [str-abi](../contracts/0011-str-abi.md)). Same pattern: opaque `ptr` from the compiler's view; dedicated `_new`, `_drop`, and operation symbols in C; RC header in the struct prefix. - **Term::Ctor + pattern matching.** Unchanged. `term-ctor` and `pat-ctor` keep their roles for named-ctor ADTs. - **`term-ctor` vs `new`.** Different constructs for different needs. The choice point for a hypothetical hybrid (e.g. "every named ctor implicitly defines a `new` fn") is out of scope for this whitepaper — the two coexist by design. ## Forward axes This whitepaper does not commit to any future kernel extension. The shape it enables — domain-specific types as plugins — is intended to make a future Matrix, Decimal, Time, etc., a self-contained crate-level proposal each gated by its own feature-acceptance and brainstorm pass. The plugin contract above is the only commitment. ## Cross-references - **Feature-acceptance gate** applied to each kernel extension: [feature-acceptance](../contracts/0004-feature-acceptance.md). - **Method dispatch** (orthogonal to type-scoped namespacing): [method-dispatch](../contracts/0016-method-dispatch.md). - **Honesty rule** (this whitepaper's STATUS contract): [honesty-rule](../contracts/0007-honesty-rule.md). - **Reference design** for Series storage layout: `/home/brummel/dev/RustAst/src/ast/rtl/series/` (external Rust project; not in this repo). - **Per-milestone specs:** `docs/specs/0052-kernel-extension-mechanics.md` (prep milestone); future `docs/specs/YYYY-MM-DD-series.md` (Series milestone, written after prep closes).