AILang/docs/journals/2026-05-19-design-decision-records.md

Design decision-records — relitigation guard (migrated 2026-05-19)

Why-X-chosen / why-Y-rejected / deliberately-does-not-do / rollback / empirical-addendum prose, migrated out of the former docs/DESIGN.md by the design-md-rolesplit milestone. A future brainstorm reads this so it does not re-propose a settled-and- rejected idea. Order preserved from the source. This file is append-only history; it is NOT a contract surface.

Decision 1: source = data, not text

A module is a JSON object with a fixed schema. There is no parser for free-form text. Typos in identifiers turn into hash-lookup errors that the compiler proposes a fix for directly.

A textual form exists (.ail, S-expression-like), but only as a bidirectional projection of the JSON form. It is intended for human reviews and diffs.

Canonical format: .ail.json with deterministic key order.

Decision 2: content-addressed definitions

Every top-level definition has a hash value (BLAKE3 over canonical JSON without the hash field itself). References between definitions go primarily by name — names are for readability. The hash is the canonical identity.

Advantages:

• Refactoring by adding new defs, not by in-place change. Old versions stay callable until manually removed.
• Caching of typecheck results and codegen per hash.
• Diffs show exactly which def has changed.

Decision 4: Hindley-Milner + optional refinements

MVP: HM with let-polymorphism. All types are inferable, but at the top level they must always be explicitly annotated (for local reasoning).

Later: refinement annotations that escalate to SMT. (i: Int | i >= 0). They are reserved in the AST from the start, but in the MVP they are simply passed through as opaque strings.

Decision 5: emit LLVM IR as text

Instead of inkwell or llvm-sys: AILang produces .ll files as strings and hands them to clang for linking.

Rationale:

• The LLVM IR text syntax is largely stable across versions.
• No build dependency on a specific libllvm version.
• Generated code is trivially inspectable, which makes debugging much easier.
• An LLM can read the generated IR directly, which is harder with opaque library calls.

Trade-off: no inline optimisations through the LLVM API. We rely on clang -O2 as the standard pipeline.

The rest of this section records the why of Decision 6 for the audit trail; the constraints listed below describe the surface as shipped.

Why this is opening up

Early development authored everything as raw *.ail.json. That worked for 17 fixture files (each ≤ 60 LOC of JSON) but does not scale. Two breaking signals:

1. The token-economy cost of the JSON-AST is massive: a single integer literal 1 is encoded as {"t":"lit","lit":{"kind":"int","value":1}} — ~38 tokens of structural overhead per bit of semantics. For a stdlib in the 200–500 def range this displaces real attention budget.
2. JSON-AST authoring exposes a class of errors (wrong field names, silently-accepted extra fields under #[serde(default)], inconsistent ctor casing) that surface only at load time. The schema is correct-by-construction in storage but error-prone in authoring.

Decision 1 anticipated this: it says a textual form exists "as a bidirectional projection of the JSON form." The pretty-printer already emits S-expression-style text (see crates/ailang-core/src/pretty.rs). What is missing is the inverse direction — text → AST. Decision 6 adds that inverse as one authoring projection alongside the existing pretty-printer; the JSON-AST remains the source of truth.

First choice and rollback plan

Try (A) first. Reasoning: constraint 1 (formalizable) outweighs constraint readability. (A) has a 3-rule core grammar with one lexical disambiguation rule. (B) doubles the rule count and re-introduces a soft form of precedence (-> inside forall). (C) is tempting because it is zero-design but the resulting spec is visibly heterogeneous, which is exactly what constraint 1 was meant to rule out.

If implementing (A) reveals that paren density actively hurts my authoring (measurable: I make more wrong-paren errors than the JSON-AST shape produced before), roll back and try (C). (B) stays on the shelf for a future iter only if both fail.

Implementation outline

• crates/ailang-surface — new crate. Strictly additive. PEG parser produces existing ailang-core::ast types. No new AST nodes, no schema changes, no new hashable form. Pretty-printer for form (A) lives here too (the round-trip is the contract).

• ailang-check, ailang-codegen are not modified. They continue to consume ailang-core::ast::Module values regardless of which projection produced them.

• Round-trip test: for every examples/*.ail.json, parse the corresponding hand-written *.ail, canonicalise, and assert hash-equivalence to the original. Hash equivalence is the truth check; the surface ships only if every fixture round-trips identically.

• CLI: ail parse <file.ail> -o <file.ail.json>. Symmetric to existing ail render. .ail.json remains a first-class input to every existing subcommand; the parser is a producer, not a gatekeeper.

  Both extensions accepted. Every path-taking CLI subcommand accepts either .ail (Form A) or .ail.json (Form B) as input. For .ail paths the subcommand parses through ailang_surface::parse in-line and then proceeds with the same loaded Module value that .ail.json would have produced. ail parse remains the explicit converter — it does nothing the implicit dispatch in the other subcommands does not, but it is the supported way to materialise a stable .ail.json snapshot from a .ail source for diff / hash / cache purposes.

• Stdlib (std_list, std_maybe, ...) authored in form (A) from day one because that is the AI authoring projection, not because JSON authoring is forbidden. The resulting .ail.json is what tests and downstream tools see.

What this Decision deliberately does not do

• It does not change which form is canonical: the JSON-AST remains the hashable, content-addressed representation, and all hashing, content-addressing, and cross-module references flow through it unchanged. The authored form is Form A; the JSON-AST is materialised in-process by callers that need it. The two forms are byte-isomorphic by the round-trip invariant. Form (A) is one projection; the JSON-AST stays canonical.
• It does not foreclose visual or graphical front-ends. The crate layout (core owns AST; surface/visual/... are siblings) reserves that lane.
• It does not remove .ail.json as input. Every existing CLI subcommand (check, render, describe, emit-ir, build, run, manifest, deps, diff, workspace, builtins) keeps its current .ail.json interface.

Form refinements during implementation

Two productions in the original sketch had to be widened during implementation to round-trip the existing AST faithfully. Captured here for the spec record:

1. lam-term carries types and effects. The AST's Term::Lam stores parallel params, param_tys, ret_ty, and effects fields. The original sketch had only names. The implemented form is

   lam-term ::= "(" "lam" "(" "params" typed-param* ")"
                          "(" "ret" type ")"
                          effects-clause? body-attr ")"
   typed-param ::= "(" "typed" ident type ")"
   

   This keeps the no-precedence / one-construct-per-token-list invariants and adds no new lexical rules.

2. import-clause admits an optional alias. The AST's Import.alias is Option<String>; the original sketch only supported the None case. The implemented form is

   import-clause ::= "(" "import" ident ("as" ident)? ")"
   

   as is a bare ident token in this position; no special lexical rule is needed.

Neither change extends the grammar's rule budget meaningfully: the 30-production ceiling of constraint 1 is intact (the parser implements ~28 named productions). All 17 examples/*.ail.json fixtures round-trip identically through print → parse → canonical JSON; the three hand-written .ail exhibits parse to canonical JSON identical to their corresponding .ail.json files.

3. Tail-call surface. Decision 8 ships two new productions, both positional analogues of their non-tail counterparts. Their result terms set Term::App.tail = true / Term::Do.tail = true; the typechecker's verify_tail_positions pass enforces that the marker is only used in tail position.

   tail-app-term ::= "(" "tail-app" term term+ ")"
   tail-do-term  ::= "(" "tail-do" ident term* ")"
   

   Production count: ~30, still inside the 30-rule constraint-1 budget. No new lexical rule (tail-app / tail-do are bare ident tokens; no special casing).

Empirical addendum — cross-model authoring measurement

Cross-model measurement against two foreign LLMs via IONOS, temperature=0, top_p=1, max-turns=5. Two blind cohorts on the same four MVP tasks (t1_add_three, t2_length, t3_main_prints, t4_count_zeros); each cohort sees only its own form's mini-spec. Subjects (both runs use the pipeline-error-formatting fix from ms.1 so JSON-cohort feedback carries the full anyhow cause chain):

• Qwen/Qwen3-Coder-Next — experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-080864/
• meta-llama/CodeLlama-13b-Instruct-hf — experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-9197fd/

metric	Qwen / JSON	Qwen / AIL	CodeLlama / JSON	CodeLlama / AIL
reached green	1/4	1/4	0/4	2/4
first-attempt green	1/4	1/4	0/4	2/4
mean turns-to-green (green only)	1.0	1.0	—	1.0
total prompt tokens	182,378	110,575	116,015	93,017
total completion tokens	14,972	3,474	2,711	2,234
top error class	check (×3)	parse (×3)	check (×2)	parse (×2)

Both subjects show the same direction on every metric the table tracks. AIL is cheaper than JSON on prompt tokens (Qwen 61%, CodeLlama 80% of the JSON cohort's spend) and cheaper on completion tokens (Qwen 23%, CodeLlama 82%). AIL reached-green is greater than or equal to JSON reached-green for each subject (Qwen ties at 1/4; CodeLlama strictly dominates, 2/4 vs 0/4). The failure-class symmetry observed on the original Qwen baseline holds for both subjects: JSON cohorts fail at the typecheck stage, AIL cohorts at the parse stage — each form's front-of-pipeline check. Three of the four first-attempt-green cells observed across the two subjects are AIL (Qwen-AIL t3, CodeLlama-AIL t1, CodeLlama-AIL t3); the fourth is Qwen-JSON t3, the same task the original baseline already flagged as the JSON-form's only first-attempt success. Two CodeLlama JSON-cohort cells (t2_length, t4_count_zeros) terminated as api_failure on turn 1 with zero tokens consumed — a transient IONOS-side terminal error, not a model-output failure; CodeLlama JSON-cohort numbers therefore average over 2 informative cells, not 4. The Qwen re-run also shifted by one cell against the cma.3 baseline (AIL 2/4 → 1/4, JSON 1/4 → 1/4) despite identical temperature=0 inputs; this is IONOS-side state-of-day noise on a single deterministic re-run, not an effect of the ms.1 feedback-formatting fix (which only changes JSON-cohort prompt content).

Scope of this addendum: two subjects, n=1 each, deterministic. This is a second data point pointing in the same direction as the first, not a verdict. The universal claim of this Decision (".ail is the AI authoring projection") would need ≥3 subjects with statistical robustness to ratify; both current points point the same way (AIL cohort cheaper and at-least-as-green) but neither the sample size nor the cross-call noise observed in the Qwen re-run is enough to close out the Decision.

Why not other memory models

Tracing GC. Open-ended. Boehm's bench overhead is structurally on the allocate path (JOURNAL bench notes); tuning Boehm cannot move that needle. A precise tracing GC would need read/write barriers, root maps, generational machinery — months of work, with irreducible pause-time variability at the end. The user's framing was decisive: "Am GC kannst du ewig rumschrauben (und bekommst trotzdem auch in 100 Jahren keine berechnbare Performance)."

Region inference (Tofte/Talpin / MLton-style). Considered seriously; rejected. Regions tie lifetimes to dynamic scope — every value lives in some region, regions stack on entry/exit, deallocation is bulk-by-region. The reduction: a region is an explicit allocator with sugar plus a static check that nothing escapes its lifetime. The deeper problem: regions assume stack-shaped lifetimes. Real programs have non-stack-shaped lifetimes — caches, memo tables, registries, lookup structures whose lifetimes are not nested. Regions would either force these into a letregion at the top of main (everyone's allocator becomes the root region — useless), or require a region per cache variant (combinatorial). RC has no lifetime-shape assumption; it works for any DAG.

Linear / ownership types as primary mechanism (Rust-style). Considered as a general-purpose alternative; rejected as primary. Rust's borrow-checker is a great mechanism but requires the author to thread lifetimes through every signature and accept that some programs cannot be expressed without unsafe. For an LLM-targeted language with recursion + closures everywhere, that cost is too high — most of the source surface would be lifetime annotations rather than logic. AILang uses a subset of these ideas (mode annotations, linear consumption discipline) selectively, layered on top of RC, where the annotations buy concrete optimisations and never have to thread lifetimes through callees.

Why advisory + suppress instead of inference

Three reasons, all anchored in Decision 10's framing:

1. Annotation states intent; inference picks weakest-supporting. These often coincide today but are conceptually different. The annotation captures what the author committed to (e.g. (own T) reserved for a planned mutation that hasn't landed); inference would silently relax it.
2. Annotation is a drift-bremse. Body change that flips the inferred mode produces caller-side breakage at remote sites. Annotation enforces the local-conflict-error pattern instead.
3. Forcing function for LLM authoring. Without mandatory annotation, the LLM never has to commit to ownership intent before writing the body. The advisory lint plus suppress lets us flag accidental over-strictness without weakening the contract.

The suppress mechanism with mandatory-reason mirrors Rust's #[allow(...)]-style escape hatch but sharpens it: the reason is required (not optional), and it becomes part of the contract the next reader sees. CLAUDE.md's "preserve correctness across development cycles" is what this directly serves.

Adjacent extensions for mutability (out of Decision 10's scope)

If future workloads need mutable arrays, hash tables, or other inherently mutable primitives, the answer is not a tracing GC backstop. The answer is a separate ownership/linear extension that gates mutability behind static single-owner discipline. RC

• uniqueness is the universal floor; ownership extends it for specific high-performance primitives without rebreaking the acyclicity invariant.

What this Decision deliberately does not do

• Does not infer everything. AILang demands annotations because the LLM author can produce them effortlessly. The compiler does inference plus verification of contracts.
• Does not require existing fixtures to migrate immediately. (con T) is treated as (own T). Existing JSON hashes stay bit-identical until the fixture is intentionally updated.
• Does not commit to atomic refcounts. AILang is single-threaded; refcounts are non-atomic. The embedding ABI's per-thread ailang_ctx_t keeps this correct under a host swarm: each thread's allocations are private to its ctx, no RC cell crosses a thread, so non-atomic refcounts stay sound.
• Does not introduce regions. Regions were considered and rejected; see "Why not other memory models" above.

What the typeclass design explicitly does NOT support

Each of the following is rejected by either schema (parser cannot express the construct) or by an enumerated diagnostic. The combination of axis-1 (single-param, no FunDeps, no assoc) and axis-5 (kind * only) covers the space.

• Multi-parameter classes (class Foo a b where ...). Schema rejects: ClassDef.param is a string, not a list.
• Higher-kinded class params (class Functor f). Rejected at workspace-load time: BareCrossModuleTypeRef from canonical-form validation fires before class-schema validation if any method body uses the param as a Type::Con head — bare non-primitive names must be declared as a TypeDef in the owning module, and a class param is not.
• Higher-rank polymorphism (forall a. (forall b. b -> b) -> a). Already rejected at parse time per the typeclass-conversation rationale recorded in JOURNAL; constraint-bearing signatures inherit that prohibition.
• Existential / dyn dispatch (exists a. Show a => a, heterogeneous lists like [Show]). Schema does not express existentials. The LLM-natural alternative is sum types.
• Associated types (class Container c where type Element c). Schema does not express type-level methods.
• Functional dependencies (class Convert a b | a -> b). Required only for multi-param classes; entails by axis 1.
• deriving / auto-derivation (data Foo = ... deriving (Eq)). Future iteration, gated separately on Feature-acceptance.
• Numeric literal defaulting. 1 does not auto-resolve to Int under an ambiguous Num a constraint. Bare polymorphic literals with multiple satisfying instances fire NoInstance with a hint to annotate. The LLM-author writes 1 : Int or 1 : Float.
• Orphan-with-warning mode. Coherence is hard. OrphanInstance is always an error, never a warning. The corollary --allow-orphans flag is not provided.

What this decision does NOT commit to

• Operator routing. ==, <, etc. stay primitive in milestone 22. Class-routing operators is a future-iteration option, not a Decision-11 commitment.
• Form-B (prose) projection of class/instance. The prose renderer for the class/instance schema nodes is one-way (no parser by design); the render in crates/ailang-prose/src/lib.rs is a placeholder and informational only.
• Specific monomorphised-symbol naming format. Milestone 22 uses <method>__<type-surface-name> for primitive type targets (e.g. show__Int) and <method>__<8-hex-prefix> for compound types (where <8-hex-prefix> is the BLAKE3 hash of the canonical type bytes). The __ separator was chosen over # and @ for LLVM IR identifier legality.
• Mode annotations on class methods. Class method signatures ARE full FnSigs and DO carry mode annotations per Decision 10; the convention is borrow for read-only methods. User-defined classes pick modes per method.
• Number of Prelude classes. Milestone 22 ships zero (no Prelude). A future Prelude milestone gates class additions on the Feature-acceptance criterion at the time of proposal.

Env construction — why two parallel overlays (rationale of the code-SoT §"Env construction")

Behaviour of the two-overlay check environment is code-authoritative (the env-construction ledger row is source-link only); only the why moves here so a future brainstorm does not re-propose collapsing the two maps:

The split is not a refactor wart. The two maps have distinct semantic roles: an owning data index, and a derived lookup accelerator over the same data. Collapsing them into one variant-keyed map (Map<Name, EntryKind>) would force every type-iterator and every ctor-lookup site to discriminate the variant tag, reversing the optimisation the reverse index exists to provide.

The mono-side overlay was narrowed to types-only at iter ct.3.2 because its consumer is the runtime ctor lookup, which became type-driven post-ct.2.2 — a different consumer story, hence a different overlay shape. The asymmetry between the check side and the mono side is by design and is pinned by crates/ailang-check/tests/duplicate_ctor_pin.rs.

Migrated from design/contracts/ at design-md-rolesplit.tidy (2026-05-19)

History/decision-record prose extracted from contract files by the post-audit tidy (the ###-whole relocation in iter .1 dragged these into contracts; they are relitigation-guard content, not contract).

typeclasses.md — Prelude-classes evolution

An earlier draft committed to a fixed Prelude (Show/Eq/Ord on the primitives), but the implementation work to wire int_to_str as a heap-allocated-string runtime primitive proved substantively separable from the typeclass machinery itself, and the LLM-utility case for primitive Show is weak (LLM-natural form is int_to_str x, not show x). Milestone 23 added Ordering/Eq/Ord + primitive instances + the five helpers; operator routing through Eq/Ord and print-rewire were held out for bench-rebaseline / post-mq-dispatcher reasons; Show shipped in milestone 24 (iters 24.2/24.3). MethodNameCollision was retired at iter mq.3 (cross-class method sharing became structurally legal). The dedicated KindMismatch diagnostic was retired at iter ctt.3 (BareCrossModuleTypeRef subsumes it).

str-abi.md — heap-Str builtin provenance + deferred-operator rationale

Builtin iter provenance: int_to_str/bool_to_str/float_to_str/ str_clone shipped iter 24.1; str_concat shipped iter str-concat (2026-05-13); the non-escape static-Str lowering pass landed iter 18b, move-tracking partial-drop iter 18d.3. Operator routing through classes is deliberately deferred — it would migrate every fixture and re-baseline the bench corpus (a new-baseline decision, not an iter detail). Num is not in milestone 22: class-based numeric overloading would invoke literal-defaulting which axis-7 excluded.

scope-boundaries.md — retired ops + deferral rationale

Polymorphic-fn-as-value is deferred because it would need one closure-pair global per instantiation. The per-type print ops io/print_int/io/print_bool/io/print_float were retired in iter rpe.1 (the polymorphic print is the canonical non-Str output path).

roundtrip-invariant.md / data-model.md — corpus + loop-recur provenance

The Form-A round-trip corpus was flipped from .ail.json to .ail at iter form-a.1. The loop/recur schema entries were added at loop-recur iter 1 (strictly additive; pre-existing fixtures hash bit-identically). Provenance only — the present-tense schema/contract is the entry shapes themselves.