706f90bacd
Strictly additive new crate per Decision 6 architectural pin. JSON-AST stays the source of truth; ailang-surface is one producer/consumer of ailang_core::ast::Module values. ailang-check and ailang-codegen unchanged. Crate contents: - src/lex.rs (~264 LOC): 3-rule lexical core. Whitespace and parens delimit tokens; semicolon to EOL is comment; first-character classifier (digit -> int, " -> string, else -> ident). - src/parse.rs (~1041 LOC): hand-written recursive descent. One Rust fn per EBNF production. No parser-combinator dep. - src/print.rs (~371 LOC): deterministic pretty-printer. Round-trip contract with parse() is the surface's correctness gate. - tests/round_trip.rs (~128 LOC): integration test runs every examples/*.ail.json fixture through print -> parse -> canonical JSON, asserts canonical-byte equality with the original. Two AST-driven form widenings beyond the 14b sketch (both folded into DESIGN.md Decision 6): - lam-term carries (typed name type) params, ret type, and optional effects (Term::Lam has parallel param_tys/ret_ty/ effects fields). - import-clause admits (import name (as alias)?) (Import.alias is Option<String>). Production count ~28, under 30-rule budget. Constraint 1 (formalisable for foreign LLM) intact. Verification: - cargo build --workspace green. - cargo test --workspace: 76 tests green (was 64; +9 surface unit, +2 round-trip integration). All 17 fixtures round-trip byte-identical at canonical level. 3 hand-written .ailx exhibits parse to canonical JSON identical to their .ail.json siblings. - cargo doc --no-deps zero warnings (DESIGN.md item 6 invariant). Manual smoke test (ail parse → ail run): hello, box, list_map_poly all produce expected output through the form-(A) authoring lane end-to-end. CLI: ail parse <file.ailx> [-o <file.ail.json>]. .ail.json remains a first-class input to every existing subcommand. Plan 14d: stdlib (std_list.ailx with length/filter/fold/concat/ reverse/head/tail), authored in form (A) from day one. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
645 lines
27 KiB
Markdown
645 lines
27 KiB
Markdown
# AILang — design decisions
|
||
|
||
This document records the core decisions for AILang. It is my contract with
|
||
myself across future iterations. Prefer cuts over growth.
|
||
|
||
## Goal
|
||
|
||
AILang is a programming language for LLM authors. It compiles to LLVM IR.
|
||
Performance: native, no GC for the MVP.
|
||
|
||
Optimised for:
|
||
|
||
- **Machine readability** over human ergonomics. The source is structured.
|
||
- **Local reasoning.** Every definition carries its full type and effects.
|
||
- **Provability.** Pure core language, explicit effects, optional refinements.
|
||
- **Robustness against hallucinations.** Symbols are hashable; tools can verify
|
||
existence without spending context window.
|
||
|
||
## Project ecosystem
|
||
|
||
AILang is not just a language but an ecosystem. The language on its own is
|
||
only valuable when its surroundings make it usable, checkable, and
|
||
extensible for its target user (LLM authors). The repo therefore contains
|
||
several equally important components — none of them optional, all of them
|
||
evolving in lockstep with the language:
|
||
|
||
- **Language core** (`crates/ailang-core`, `crates/ailang-check`,
|
||
`crates/ailang-codegen`): AST, type system, codegen.
|
||
- **CLI** (`crates/ail`): toolchain for tooling consumers — `manifest`,
|
||
`describe`, `deps`, `check`, `build`, etc., preferably with `--json` for
|
||
machine consumption.
|
||
- **Examples** (`examples/`): canonical `.ail.json` programs. They are
|
||
specification anchors, not demos — the E2E suite hangs off them.
|
||
- **Agents** (`agents/`): specialised sub-prompts (implementer,
|
||
architect, tester, debugger) that form the project's own LLM tooling.
|
||
They are a versioned part of the repo. See `agents/README.md`.
|
||
- **Docs** (`docs/`): DESIGN.md (what and why), JOURNAL.md (history).
|
||
- **Tests**: unit tests per crate plus E2E in `crates/ail/tests/e2e.rs`. Every
|
||
new compiler path needs a test, otherwise the feature does not count as done.
|
||
|
||
When the language grows, these components grow with it. New tools that
|
||
strengthen the LLM tooling (e.g. `ail diff`, IR snapshot diffs, new agents)
|
||
explicitly belong in the ecosystem inventory of this section and are added
|
||
here as soon as they are established.
|
||
|
||
## Project language: English
|
||
|
||
All in-tree content is written in English: source code (identifiers,
|
||
comments, string literals, CLI help), design documents, the journal, agent
|
||
prompts, READMEs, commit messages, examples, and `CLAUDE.md`. The live
|
||
conversation between user and me stays German for ergonomic reasons;
|
||
everything that lands in git is English. This keeps diffs and tooling output
|
||
uniform and matches the audience for AILang (LLM authors), for whom English
|
||
is the default.
|
||
|
||
## Decision 1: source = data, not text
|
||
|
||
A module is a JSON object with a fixed schema. There is no parser for
|
||
free-form text. Typos in identifiers turn into hash-lookup errors that the
|
||
compiler proposes a fix for directly.
|
||
|
||
A textual form exists (`.ail`, S-expression-like), but only as a
|
||
bidirectional projection of the JSON form. It is intended for human reviews
|
||
and diffs.
|
||
|
||
**Canonical format:** `.ail.json` with deterministic key order.
|
||
|
||
## Decision 2: content-addressed definitions
|
||
|
||
Every top-level definition has a `hash` value (BLAKE3 over canonical JSON
|
||
without the `hash` field itself). References between definitions go primarily
|
||
by name — names are for readability. The hash is the canonical identity.
|
||
|
||
Advantages:
|
||
|
||
- Refactoring by adding new defs, not by in-place change. Old versions stay
|
||
callable until manually removed.
|
||
- Caching of typecheck results and codegen per hash.
|
||
- Diffs show exactly which def has changed.
|
||
|
||
## Decision 3: pure core language + algebraic effects
|
||
|
||
The default is total, pure functions. Effects are declared as a set in the
|
||
function type: `(Int) -> Int ![IO]`. The effect set is row-polymorphic
|
||
(`![IO | r]`). In the MVP only the effects `IO` and `Diverge` (for infinite
|
||
loops) are wired up.
|
||
|
||
This is the most important LLM property: when I read a function, I can trust
|
||
its signature without reading the body.
|
||
|
||
## Decision 4: Hindley-Milner + optional refinements
|
||
|
||
MVP: HM with let-polymorphism. All types are inferable, but at the top level
|
||
they must always be explicitly annotated (for local reasoning).
|
||
|
||
Later: refinement annotations that escalate to SMT. `(i: Int | i >= 0)`. They
|
||
are reserved in the AST from the start, but in the MVP they are simply passed
|
||
through as opaque strings.
|
||
|
||
## Decision 5: emit LLVM IR as text
|
||
|
||
Instead of `inkwell` or `llvm-sys`: AILang produces `.ll` files as strings
|
||
and hands them to `clang` for linking.
|
||
|
||
Rationale:
|
||
|
||
- The LLVM IR text syntax is largely stable across versions.
|
||
- No build dependency on a specific libllvm version.
|
||
- Generated code is trivially inspectable, which makes debugging much easier.
|
||
- An LLM can read the generated IR directly, which is harder with opaque
|
||
library calls.
|
||
|
||
Trade-off: no inline optimisations through the LLVM API. We rely on
|
||
`clang -O2` as the standard pipeline.
|
||
|
||
## Decision 6: authoring surface (Iter 14b — WIP)
|
||
|
||
**Status: design pass in progress.** Reading without skipping the JOURNAL
|
||
will leave this section ahead of the implementation.
|
||
|
||
### Why this is opening up
|
||
|
||
Iters 1 through 14a authored everything as raw `*.ail.json`. That worked
|
||
for 17 fixture files (each ≤ 60 LOC of JSON) but does not scale. Two
|
||
breaking signals:
|
||
|
||
1. The token-economy cost of the JSON-AST is massive: a single integer
|
||
literal `1` is encoded as `{"t":"lit","lit":{"kind":"int","value":1}}`
|
||
— ~38 tokens of structural overhead per bit of semantics. For a stdlib
|
||
in the 200–500 def range this displaces real attention budget.
|
||
2. JSON-AST authoring exposes a class of errors (wrong field names,
|
||
silently-accepted extra fields under `#[serde(default)]`,
|
||
inconsistent ctor casing) that surface only at load time. The
|
||
schema is correct-by-construction in storage but **error-prone in
|
||
authoring**.
|
||
|
||
Decision 1 anticipated this: it says a textual form exists "as a
|
||
bidirectional projection of the JSON form." The pretty-printer already
|
||
emits S-expression-style text (see `crates/ailang-core/src/pretty.rs`).
|
||
What is missing is the inverse direction — text → AST. Decision 6
|
||
adds that inverse as **one** authoring projection alongside the
|
||
existing pretty-printer; the JSON-AST remains the source of truth.
|
||
|
||
### Architectural pin: data structure is the source of truth
|
||
|
||
The textual surface is **not** a replacement for the JSON-AST. It is
|
||
one projection among potentially many. Concretely:
|
||
|
||
- The JSON-AST keeps its role as the canonical, hashable, content-
|
||
addressed representation of a module. All hashing, content-
|
||
addressing, cross-module references, and typecheck/codegen input
|
||
flow through the JSON-AST. **No new hashable form is introduced.**
|
||
- The textual surface (form A, this Decision) is the **AI authoring
|
||
projection**: optimised for me producing programs token-efficiently
|
||
and for foreign LLMs producing programs from a spec. It is not
|
||
optimised for human authors and does not need to be human-pleasant.
|
||
- Future projections are explicitly anticipated: a visual / graphical
|
||
front-end is a plausible second projection for human review and
|
||
inspection (display being the one case where non-AI eyes matter).
|
||
The architecture leaves room: any producer of well-formed
|
||
`ailang-core::ast::Module` values is a valid front-end.
|
||
- **No human is expected to author AILang seriously.** Authoring is
|
||
AI work. Display and verification, by contrast, are concerns where
|
||
human-facing alternatives may be useful — and which can therefore
|
||
layer their own projections on top of the same AST without
|
||
touching the surface or the core.
|
||
|
||
In code terms: `ailang-core` owns the AST. `ailang-surface` (new in
|
||
Iter 14c) is one producer/consumer pair: text-form-A → AST → text-
|
||
form-A. A hypothetical `ailang-visual` would be a different producer
|
||
of the same AST. `ailang-check` and `ailang-codegen` consume only
|
||
the AST and remain projection-agnostic.
|
||
|
||
### Constraints (hard, in priority order)
|
||
|
||
1. **Formalizable for a foreign LLM.** The grammar must fit in an
|
||
EBNF/PEG spec of ≤ 30 productions. A model that has never seen
|
||
AILang must be able to read the spec and produce conforming source
|
||
zero-shot. Rules out: precedence between binary operators,
|
||
semantic indentation, maximal-munch lexing, context-sensitive
|
||
reductions.
|
||
2. **AST-isomorphic.** Every surface form maps to exactly one AST
|
||
shape. Round-trip surface → AST → canonical JSON → AST → surface
|
||
is the identity (modulo formatting). Hashes computed via the
|
||
round-tripped JSON must equal hashes of the same module written
|
||
directly in JSON.
|
||
3. **No external symbols.** ASCII only. No Greek (`∀`), no arrows
|
||
(`→`), no subscripts. Reasoning: I substitute mojibake for
|
||
non-ASCII characters under context pressure; foreign LLMs vary
|
||
in how they tokenize Unicode.
|
||
4. **No precedence.** Either everything is parenthesized, or there
|
||
are no infix operators. Prefer the latter — `add(x, 1)` over
|
||
`x + 1`. Removes a fail mode for both me and foreign LLMs.
|
||
5. **No semantic indentation.** Block structure expressed by paired
|
||
delimiters or terminator tokens. Indentation is informational
|
||
only; the parser ignores it.
|
||
6. **One construct per token-list.** Every AST node corresponds to
|
||
exactly one parenthesized form (or atom). No "sometimes you can
|
||
omit the parens" rules.
|
||
7. **AST surface stays frozen.** The surface adapts to the AST, not
|
||
the other way around. We do not change the JSON schema or
|
||
invalidate hashes to make the surface prettier.
|
||
|
||
### Candidate notations (same `map` encoded in each)
|
||
|
||
The reference target — the polymorphic `map` from `examples/list_map_poly.ail.json`:
|
||
|
||
```
|
||
data List a where Nil | Cons a (List a)
|
||
fn map : forall a b. ((a) -> b, List a) -> List b
|
||
= \f xs. match xs of Nil -> Nil
|
||
| Cons h t -> Cons(f(h), map(f, t))
|
||
```
|
||
|
||
#### (A) S-expression with fully-tagged AST nodes
|
||
|
||
```
|
||
(module list_map_poly
|
||
|
||
(data List (vars a)
|
||
(ctor Nil)
|
||
(ctor Cons a (con List a)))
|
||
|
||
(fn inc
|
||
(type (fn-type (params (con Int)) (ret (con Int))))
|
||
(params x)
|
||
(body (app + x 1)))
|
||
|
||
(fn map
|
||
(type
|
||
(forall (vars a b)
|
||
(fn-type
|
||
(params (fn-type (params a) (ret b)) (con List a))
|
||
(ret (con List b)))))
|
||
(params f xs)
|
||
(body
|
||
(match xs
|
||
(case (pat-ctor Nil) (term-ctor List Nil))
|
||
(case (pat-ctor Cons h t)
|
||
(term-ctor List Cons
|
||
(app f h)
|
||
(app map f t)))))))
|
||
```
|
||
|
||
Grammar core (3-rule lexical layer + ~25 named-form productions):
|
||
```
|
||
sexpr ::= atom | "(" sexpr* ")"
|
||
atom ::= integer | string | ident
|
||
ident ::= any maximal non-whitespace, non-paren run that is not
|
||
a recognised integer or string literal.
|
||
```
|
||
|
||
The lexer recognises one delimiter (`(` / `)`) and whitespace.
|
||
Every other maximal token is classified post-hoc:
|
||
- All-digit run with optional leading `-` → integer atom.
|
||
- `"`-delimited run → string atom.
|
||
- Otherwise → ident.
|
||
|
||
Consequence: operators like `+`, `==`, `<=`, `**`, qualified
|
||
names like `io/print_int`, and cross-module references like
|
||
`std_list.map` are all single ident tokens with no special lex
|
||
rule. The only reserved tokens are `(`, `)`, and whitespace.
|
||
Bool literals (`true`, `false`) and unit (`(lit-unit)`) are
|
||
disambiguated by parser context, not by lex.
|
||
|
||
Every AST node form has a unique head keyword (`module`, `data`,
|
||
`fn`, `forall`, `fn-type`, `con`, `var`, `app`, `lam`, `match`,
|
||
`case`, `pat-ctor`, `term-ctor`, `do`, `seq`, ...). A bare atom in
|
||
a positional slot (e.g. inside `(con List a)` second position) is
|
||
a name reference whose **sort** is determined by the parent slot:
|
||
|
||
- inside `(con NAME args...)` second-and-later positions → type
|
||
expression. Bare atom there ⇒ `Type::Var { name }`.
|
||
- inside `(app HEAD args...)` first position ⇒ `Term::Var`.
|
||
- inside `(pat-ctor CTOR fields...)` field positions ⇒
|
||
`Pattern::Var`.
|
||
- inside `(case PAT BODY)` second position ⇒ term.
|
||
|
||
There is **no lexical case rule**. To construct a value with a
|
||
ctor, write `(term-ctor TypeName CtorName args...)`. To match
|
||
against one, write `(pat-ctor CtorName fields...)`. Capitalised
|
||
identifiers carry no special meaning to the parser. This rules
|
||
out a class of silent errors ("I forgot to capitalise `Cons` and
|
||
it parsed as a function call").
|
||
|
||
**Pros:** smallest formal grammar of any candidate (the lexical
|
||
core is 3 rules; the named-form productions are uniform — every
|
||
node a tagged list). Foreign-LLM bar lowest. Round-trip with the
|
||
existing pretty-printer is a refactor of `pretty.rs` to emit this
|
||
tagged form, plus a new parser.
|
||
|
||
**Cons:** paren density is high. `(forall (vars a b) (fn-type
|
||
...))` has more visual nesting than the current pretty-printer's
|
||
`forall a. (...) -> ...`. Verbosity is ~2× JSON for the same node
|
||
when measured in characters, but ~8× shorter in lines (the
|
||
existing JSON `box.ail.json` of 160 lines becomes ~20 lines in
|
||
this form).
|
||
|
||
#### (B) Indented record-style with explicit terminators
|
||
|
||
```
|
||
module std_list
|
||
|
||
data List(a):
|
||
Nil
|
||
Cons(a, List(a))
|
||
end
|
||
|
||
fn map:
|
||
type: forall a b. fn(fn(a) -> b, List(a)) -> List(b)
|
||
params: f, xs
|
||
body:
|
||
match xs:
|
||
Nil => Nil
|
||
Cons(h, t) => Cons(f(h), map(f, t))
|
||
end
|
||
end
|
||
```
|
||
|
||
Grammar core (~20–30 productions): module-level (def/data/end), type
|
||
sub-grammar (forall, fn, con, var), term sub-grammar (lam, match,
|
||
ctor, app, lit, var, seq), pattern sub-grammar.
|
||
|
||
**Pros:** higher information density per line, closer to mainstream
|
||
ML/Haskell shape. **Cons:** four sub-grammars instead of one.
|
||
`forall a b. fn(...)` keeps a pseudo-precedence (`->` binds tighter
|
||
than the outer `fn(...)` wrapper). Foreign-LLM bar higher.
|
||
|
||
#### (C) Pretty-printer-as-source
|
||
|
||
Use exactly the format `pretty::module` already emits, plus a parser
|
||
that accepts it. The existing pretty-printer's quirks (`::` for
|
||
type-of, `[params]` for fn-params, `<a>` for type-args, `forall a. ...`,
|
||
`!IO`, `()` ambiguous between unit-arg-list and empty-form) become
|
||
the spec.
|
||
|
||
**Pros:** zero churn — the existing pretty-printer is already the
|
||
spec; only the inverse is missing. Round-trip is the identity by
|
||
construction. **Cons:** the existing format mixes four mini-dialects
|
||
(s-expr at term level, ML-shape at type level, square brackets for
|
||
params, `<>` for type args). Formalising it crisply is harder than
|
||
designing a uniform form from scratch.
|
||
|
||
### First choice and rollback plan
|
||
|
||
**Try (A) first.** Reasoning: constraint 1 (formalizable) outweighs
|
||
constraint readability. (A) has a 3-rule core grammar with one
|
||
lexical disambiguation rule. (B) doubles the rule count and
|
||
re-introduces a soft form of precedence (`->` inside `forall`).
|
||
(C) is tempting because it is zero-design but the resulting spec is
|
||
visibly heterogeneous, which is exactly what constraint 1 was meant
|
||
to rule out.
|
||
|
||
If implementing (A) reveals that paren density actively hurts my
|
||
authoring (measurable: I make more wrong-paren errors than the
|
||
JSON-AST shape produced before), roll back and try (C). (B) stays
|
||
on the shelf for a future iter only if both fail.
|
||
|
||
### Implementation outline (Iter 14c onwards, not done in 14b)
|
||
|
||
- `crates/ailang-surface` — new crate. **Strictly additive.** PEG
|
||
parser produces existing `ailang-core::ast` types. No new AST
|
||
nodes, no schema changes, no new hashable form. Pretty-printer
|
||
for form (A) lives here too (the round-trip is the contract).
|
||
- `ailang-check`, `ailang-codegen` are **not modified**. They
|
||
continue to consume `ailang-core::ast::Module` values regardless
|
||
of which projection produced them.
|
||
- Round-trip test: for every `examples/*.ail.json`, parse the
|
||
corresponding hand-written `*.ailx`, canonicalise, and assert
|
||
hash-equivalence to the original. Hash equivalence is the truth
|
||
check; the surface ships only if every fixture round-trips
|
||
identically.
|
||
- CLI: `ail parse <file.ailx> -o <file.ail.json>`. Symmetric to
|
||
existing `ail render`. **`.ail.json` remains a first-class input**
|
||
to every existing subcommand; the parser is a producer, not a
|
||
gatekeeper.
|
||
- Iter 14d: stdlib (`std_list`, `std_maybe`, ...) authored in
|
||
form (A) from day one **because that is the AI authoring
|
||
projection**, not because JSON authoring is forbidden. The
|
||
resulting `.ail.json` is what tests and downstream tools see.
|
||
|
||
### What this Decision deliberately does not do
|
||
|
||
- It does not promote form (A) to the source of truth. Form (A) is
|
||
one projection; the JSON-AST stays canonical.
|
||
- It does not foreclose visual or graphical front-ends. The crate
|
||
layout (`core` owns AST; `surface`/`visual`/... are siblings)
|
||
reserves that lane.
|
||
- It does not remove `.ail.json` as input. Every existing
|
||
CLI subcommand (`check`, `render`, `describe`, `emit-ir`,
|
||
`build`, `run`, `manifest`, `deps`, `diff`, `workspace`,
|
||
`builtins`) keeps its current `.ail.json` interface.
|
||
|
||
### Form refinements during Iter 14c implementation
|
||
|
||
Two productions in the original 14b sketch had to be widened
|
||
during implementation to round-trip the existing AST faithfully.
|
||
Captured here for the spec record:
|
||
|
||
1. **`lam-term` carries types and effects.** The AST's `Term::Lam`
|
||
stores parallel `params`, `param_tys`, `ret_ty`, and `effects`
|
||
fields. The 14b sketch had only names. The implemented form is
|
||
|
||
```
|
||
lam-term ::= "(" "lam" "(" "params" typed-param* ")"
|
||
"(" "ret" type ")"
|
||
effects-clause? body-attr ")"
|
||
typed-param ::= "(" "typed" ident type ")"
|
||
```
|
||
|
||
This keeps the no-precedence / one-construct-per-token-list
|
||
invariants and adds no new lexical rules.
|
||
|
||
2. **`import-clause` admits an optional alias.** The AST's
|
||
`Import.alias` is `Option<String>`; the original sketch only
|
||
supported the `None` case. The implemented form is
|
||
|
||
```
|
||
import-clause ::= "(" "import" ident ("as" ident)? ")"
|
||
```
|
||
|
||
`as` is a bare ident token in this position; no special lexical
|
||
rule is needed.
|
||
|
||
Neither change extends the grammar's rule budget meaningfully:
|
||
the 30-production ceiling of constraint 1 is intact (the parser
|
||
implements ~28 named productions). All 17 `examples/*.ail.json`
|
||
fixtures round-trip identically through `print → parse → canonical
|
||
JSON`; the three hand-written `.ailx` exhibits parse to canonical
|
||
JSON identical to their corresponding `.ail.json` files.
|
||
|
||
## Mangling scheme (Iter 5c)
|
||
|
||
All AILang functions are mangled to `@ail_<module>_<def>` — even in the
|
||
single-module case. Constants likewise (`@ail_<module>_<const>`). Global
|
||
string literals carry a short hint for readability:
|
||
`@.str_<module>_<hint>_<idx>` (e.g. `@.str_sum_fmt_int_0`). The entry point
|
||
is a `define i32 @main()` trampoline (C / LLVM ABI) that calls
|
||
`@ail_<entry-module>_main()`. `source_filename` exists exactly once per
|
||
workspace and carries the entry-module name (`<entry-module>.ail`).
|
||
|
||
## Convention: qualified cross-module references (Iter 5b)
|
||
|
||
Cross-module calls use **no** new AST node. Instead, a `Term::Var { name }`
|
||
with exactly one dot in the name is a qualified reference: `<prefix>.<def>`.
|
||
|
||
- `<prefix>` is an import alias (`import { module: "X", as: "<prefix>" }`)
|
||
or, when imported without an alias, the module name itself.
|
||
- `<def>` is the name of a top-level definition in the target module.
|
||
- Def names MUST NOT contain a dot — the typechecker reports
|
||
`invalid-def-name` with `ctx: { "reason": "contains-dot" }`.
|
||
- The workspace loader (Iter 5a) finds all reachable modules; the
|
||
typechecker (Iter 5b, `check_workspace`) resolves dotted names through the
|
||
import map. Diagnostic codes: `unknown-module` (prefix not imported),
|
||
`unknown-import` (module found, def not).
|
||
|
||
Hash stability: no new AST node, no renamed fields — all previous module
|
||
hashes stay bit-identical.
|
||
|
||
## Data model (MVP)
|
||
|
||
### Module
|
||
|
||
```jsonc
|
||
{
|
||
"schema": "ailang/v0",
|
||
"name": "<id>",
|
||
"imports": [{ "module": "<id>", "as": "<id>" }],
|
||
"defs": [Def...]
|
||
}
|
||
```
|
||
|
||
### Def
|
||
|
||
`kind ∈ { "fn", "type", "effect", "const" }`. In the MVP only `fn` and `const`.
|
||
|
||
```jsonc
|
||
{
|
||
"kind": "fn",
|
||
"name": "<id>",
|
||
"type": Type,
|
||
"params": ["<id>"...],
|
||
"body": Term,
|
||
"doc": "<optional string>"
|
||
}
|
||
```
|
||
|
||
### Term (expression)
|
||
|
||
```jsonc
|
||
{ "t": "lit", "lit": { "kind": "int" | "bool" | "unit", "value": ... } }
|
||
{ "t": "var", "name": "<id>" }
|
||
{ "t": "app", "fn": Term, "args": [Term...] }
|
||
{ "t": "let", "name": "<id>", "value": Term, "body": Term }
|
||
{ "t": "if", "cond": Term, "then": Term, "else": Term }
|
||
{ "t": "do", "op": "<eff>/<op>", "args": [Term...] }
|
||
{ "t": "ctor", "type": "<id>", "ctor": "<id>", "args": [Term...] }
|
||
{ "t": "match", "scrutinee": Term, "arms": [Arm...] }
|
||
{ "t": "lam",
|
||
"params": ["<id>"...],
|
||
"paramTypes": [Type...],
|
||
"retType": Type,
|
||
"effects": ["<id>"...],
|
||
"body": Term }
|
||
{ "t": "seq", "lhs": Term, "rhs": Term }
|
||
```
|
||
|
||
In the MVP, `do` is only a direct call to a built-in effect op (no handler).
|
||
A `lam` term constructs an anonymous function value; free variables of
|
||
its body are captured from the enclosing scope (see Iter 8 closure
|
||
conversion in JOURNAL). A `seq` term evaluates `lhs` for its effects
|
||
(its result must be Unit) and yields `rhs`'s value — equivalent to
|
||
`let _ = lhs in rhs`.
|
||
|
||
### Type
|
||
|
||
```jsonc
|
||
{ "k": "con", "name": "Int" }
|
||
{ "k": "con", "name": "Bool" }
|
||
{ "k": "con", "name": "Unit" }
|
||
{ "k": "fn", "params": [Type...], "ret": Type, "effects": ["IO"...] }
|
||
{ "k": "var", "name": "a" }
|
||
{ "k": "forall", "vars": ["a"...], "body": Type }
|
||
```
|
||
|
||
## Pipeline
|
||
|
||
```
|
||
.ail.json ─┐
|
||
├─ load + validate schema
|
||
├─ resolve names + assign hashes
|
||
├─ typecheck (HM, effect rows)
|
||
├─ lower to MIR (SSA-like, named SSA values)
|
||
├─ emit LLVM IR (.ll)
|
||
└─ clang -O2 *.ll -o binary
|
||
```
|
||
|
||
## CLI
|
||
|
||
```
|
||
ail check <module.ail.json> — loads, validates, typechecks
|
||
ail manifest <module.ail.json> — table: name :: type !effects [hash]
|
||
ail describe <module> <name> — detail of a definition
|
||
ail render <module> — JSON → pretty-print
|
||
ail parse <module.ail> — pretty-print → JSON (for bootstrapping)
|
||
ail emit-ir <module> — writes .ll
|
||
ail build <module> — full pipeline → binary
|
||
ail run <module> — build + execute (tempdir), passthrough exit code
|
||
```
|
||
|
||
## Verification and correctness (across cycles)
|
||
|
||
1. **Snapshot tests** for the pretty-printer and IR emit. The diff makes
|
||
regressions visible immediately.
|
||
2. **Property tests** for the JSON ↔ pretty-print roundtrip.
|
||
3. **End-to-end tests** for `examples/` with expected program output.
|
||
4. **Hash stability**: a test ensures the same def always produces the same
|
||
hash.
|
||
5. **CI pin** of the outputs in `tests/expected/`.
|
||
6. **Rustdoc cleanliness**: `cargo doc --no-deps` runs warning-free.
|
||
Maintained by the `ailang-docwriter` agent (Iter 13d onward); fixing a
|
||
rustdoc warning is part of the iter that introduced it, not a follow-up.
|
||
|
||
## What is not (yet) supported
|
||
|
||
Snapshot of the boundary at the end of Iter 13. Items move out of this list
|
||
as iterations land; the JOURNAL records the exact iteration.
|
||
|
||
- No effect handlers — only the built-in IO and Diverge ops.
|
||
- No refinements / SMT escalation.
|
||
- No HM inference inside bodies. Top-level def types are explicit;
|
||
polymorphism is opt-in via `Type::Forall { vars, body }`. Inside
|
||
a body, lambdas check monomorphically against their declared type.
|
||
- Polymorphic fns must be **directly called** at the use site.
|
||
Passing a polymorphic fn as a value (`let f = id in f(42)`) is
|
||
not yet supported — it would need one closure-pair global per
|
||
instantiation, deferred.
|
||
- No higher-rank polymorphism. Passing a polymorphic fn to another
|
||
polymorphic fn (`apply(id, 42)`) is not supported.
|
||
- No cross-module ADTs. ADTs are local to a module; ctor names must be
|
||
unique within their module but may collide across modules.
|
||
- No visibility rules in imports. Every top-level def of an imported module
|
||
is reachable; there is no `pub` / `priv`.
|
||
- No GC. ADT boxes, lambda envs, and closure pairs all leak. Acceptable
|
||
for current example programs; required before any longer-running
|
||
program.
|
||
|
||
What **is** supported (and used as the smoke test for the pipeline):
|
||
|
||
- Int, Bool, Unit, **Str** as primitive types.
|
||
- `if`, `let`, function calls, recursion.
|
||
- Effects on function signatures, with `do op(args)` for direct effect
|
||
ops (`io/print_int`, `io/print_bool`, `io/print_str`).
|
||
- **ADTs + flat pattern matching** (Iter 3). Sub-patterns of a Ctor
|
||
pattern are restricted to `Var` / `Wild`.
|
||
- **Imports + qualified cross-module references** via dotted names
|
||
(Iter 5).
|
||
- **First-class function references** (Iter 7). A top-level fn name (or
|
||
qualified `prefix.def`) used as a `Term::Var` is a fn-value.
|
||
- **Anonymous lambdas with capture** (Iter 8). `Term::Lam` constructs a
|
||
closure that captures any free variables of its body from the
|
||
enclosing scope. All fn-values share a single ABI: a `ptr` to a
|
||
closure pair `{ thunk_ptr, env_ptr }`. Top-level fns get an auto-
|
||
generated adapter and a static closure pair (env = null) so they
|
||
remain passable as values without heap overhead.
|
||
- **Polymorphism via `Type::Forall`** at top-level def types (Iter 12).
|
||
Use sites instantiate fresh metavars; unification pins them against
|
||
the concrete types of the call args. Codegen monomorphises on
|
||
demand: each unique instantiation emits a specialised LLVM fn
|
||
mangled `@ail_<m>_<def>__<descriptor>` (e.g. `id__I` for `id` at
|
||
`Int`, `apply__I_I` for `apply` at `(Int, Int)`).
|
||
- **Parameterised ADTs** (Iter 13). `TypeDef.vars: Vec<String>`
|
||
declares type parameters; `Type::Con.args: Vec<Type>` carries the
|
||
type arguments at use sites. Both fields default to empty and are
|
||
skipped during serialization, so canonical-JSON hashes of every
|
||
pre-13a definition stay bit-identical (regression test in
|
||
`crates/ailang-core/src/hash.rs`). Ctor and match codegen stay
|
||
inline at every use site — there is no specialised ADT symbol —
|
||
but LLVM field types are derived per use site by substituting
|
||
through `cdef.ail_fields`. The substitution is read off the call's
|
||
arg types (ctor) or the scrutinee's `Type::Con.args` (match). An
|
||
unresolved `Type::Var` reaching `llvm_type` is a hard error
|
||
rather than a silent fallback to `ptr`.
|
||
|
||
Pipeline regression smoke tests:
|
||
|
||
- `examples/sum.ail.json` → prints 55 (recursion, arithmetic).
|
||
- `examples/list.ail.json` → prints 42 (ADTs + match).
|
||
- `examples/hof.ail.json` → prints 42 (first-class fn-refs, indirect call).
|
||
- `examples/closure.ail.json` → prints 42 (lambda capturing a let-bound var).
|
||
- `examples/list_map.ail.json` → prints 2/4/6 (ADTs + closure + recursive
|
||
HOF + IO; the dogfood smoke test).
|
||
- `examples/sort.ail.json` → prints sorted [3,1,4,1,5,9,2,6,5,3,5]
|
||
one-per-line (insertion sort over an 11-element list).
|
||
- `examples/poly_id.ail.json` → prints 42 then "true" (polymorphic
|
||
identity at `Int` and `Bool`; two specialised fns emitted).
|
||
- `examples/poly_apply.ail.json` → prints 42 (polymorphic `apply`
|
||
with a fn-typed parameter; `apply(succ, 41)`).
|
||
- `examples/box.ail.json` → prints 42 (parameterised ADT round-
|
||
trip: `MkBox(42)` constructed, then projected by a polymorphic
|
||
`unbox : forall a. (Box<a>) -> a` and printed).
|
||
- `examples/maybe_int.ail.json` → prints 7 then 99 (pattern match
|
||
over `Maybe<Int>`: `or_else(Some(7), 99)` then
|
||
`or_else(None, 99)`).
|