Files
AILang/design/models/0001-authoring-surface.md
T
Brummel 7321826c66 audit: cycle-close tidy for #44$-lexer-reservation (refs #44)
Cycle-close audit for the #44 Form-A `$`-reservation cycle
(b151990..HEAD). One drift item fixed inline. Both regression scripts
exited 1; investigated and attributed to machine load this session, NOT
a code regression — baseline deliberately NOT updated.

Architect drift review:
- [fixed] design/models/0001-authoring-surface.md — the line "The only
  reserved tokens are `(`, `)`, and whitespace" was made stale by this
  cycle (the lexer now rejects any identifier token containing `$`).
  Corrected to distinguish token *delimiters* (`(`/`)`/whitespace) from
  the `$` *character reservation within a token*, scoped to the Form-A
  authoring surface, naming the enforcement site
  (`LexError::ReservedDollar`) and the intentional `.ail.json` non-guard.
- [carry-on] honesty sweep otherwise clean: the only other `$`-mentions
  in the ledger (0008-memory-model.md, 0003-pipeline.md) describe
  synthetic mint names (buf$1, <hint>$lr_N) and remain correct. No
  over-broad "no Module can contain `$`" prose exists. The corrected
  fresh_binder doc-comment (feat commit) is Form-A-scoped and honest.
- Decision: no new ledger *contract* added. 0015-language-constraints
  holds the four RC-soundness preconditions (strict eval, no recursive
  value bindings, no shared mutable refs, acyclic ADTs); a lexical
  character reservation is a different category and is documented in the
  authoring-surface model doc with its enforcement site, which is its
  natural home. Architect confirmed absence of `$`-contract prose is not
  itself drift.

Lockstep pairs: none apply (architect walked each against the diff):
- Pattern::Lit typecheck <-> pre_desugar_validation.rs — neither changed.
- lower_app <-> is_static_callee — codegen unchanged.
- INTERCEPTS <-> (intrinsic) markers — intercepts.rs + kernel sources
  unchanged.

Regression scripts:
- bench/cross_lang.py: EXIT 0. 25 metrics; 0 regressed, all stable.
- bench/compile_check.py: EXIT 1; "12 regressed" — ALL are check_ms.*
  (type-check wall-clock), baseline ~1.8-3.3ms vs actual ~2.5-4.4ms,
  ~+0.8ms absolute and roughly UNIFORM across every fixture regardless
  of size (hello +39%, bench_list_sum +51%). All 12 build_O0_ms.* (the
  ~93ms LLVM portion of the same `ail build`) are within tolerance but
  also uniformly shifted +12-15%. Attribution: NOT this cycle's code.
  The guard adds one `raw.contains('$')` byte-scan per token; a tiny
  module is ~50 tokens, so the added cost is nanoseconds — physically
  cannot account for +0.8ms on a 2ms check. The uniform ~12-15% shift
  across the WHOLE `ail` invocation (check AND build) is a global
  machine-slowdown fingerprint (heavy concurrent subagent/cargo load
  this session); it crosses the 25% tolerance only on the tiny-baseline
  check_ms metrics. Localised by reasoning, not by the bencher (the
  effect is environmental, nothing to localise in code).
- bench/check.py: EXIT 1; throughput metrics flipping run-to-run with
  identical code (rc_over_bump moved in/out of REGRESSION across four
  runs; bump_s swung -9.5/-11/-14/-15%). Same machine-load variance;
  rc_over_bump is a ratio of two independently-noisy wall-clock
  measurements, the noisiest metric in the set.

Baseline NOT updated on either script: ratifying a machine-load delta
would bake an under-load baseline in and mask a future real regression.
OPEN VERIFICATION: re-run bench/compile_check.py and bench/check.py on a
quiet machine to confirm green; tracked as the cycle's one open item.
Cycle #44 closes (code + docs clean; regression gate pending a
clean-machine confirmation, baseline untouched).
2026-05-30 15:18:49 +02:00

165 lines
6.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Authoring surface — notation rationale whitepaper
## Candidate notations (same `map` encoded in each)
The reference target — the polymorphic `map` from `examples/list_map_poly.ail.json`:
```
data List a where Nil | Cons a (List a)
fn map : forall a b. ((a) -> b, List a) -> List b
= \f xs. match xs of Nil -> Nil
| Cons h t -> Cons(f(h), map(f, t))
```
#### (A) S-expression with fully-tagged AST nodes
```
(module list_map_poly
(data List (vars a)
(ctor Nil)
(ctor Cons a (con List a)))
(fn inc
(type (fn-type (params (con Int)) (ret (con Int))))
(params x)
(body (app + x 1)))
(fn map
(type
(forall (vars a b)
(fn-type
(params (fn-type (params a) (ret b)) (con List a))
(ret (con List b)))))
(params f xs)
(body
(match xs
(case (pat-ctor Nil) (term-ctor List Nil))
(case (pat-ctor Cons h t)
(term-ctor List Cons
(app f h)
(app map f t)))))))
```
Grammar core (3-rule lexical layer + ~25 named-form productions):
```
sexpr ::= atom | "(" sexpr* ")"
atom ::= integer | string | ident
ident ::= any maximal non-whitespace, non-paren run that is not
a recognised integer or string literal.
```
The lexer recognises one delimiter (`(` / `)`) and whitespace.
Every other maximal token is classified post-hoc:
- All-digit run with optional leading `-` → integer atom.
- `"`-delimited run → string atom.
- Otherwise → ident.
Consequence: arithmetic operators like `+` / `-` / `*` / `/` / `%`,
qualified names like `io/print_str`, and cross-module references
like `std_list.map` are all single ident tokens with no special
lex rule. The only token *delimiters* are `(`, `)`, and whitespace.
One character is *reserved within* a token: `$`. Any identifier
token containing `$` is rejected by the lexer
(`LexError::ReservedDollar` in `crates/ailang-surface/src/lex.rs`),
because `$` is reserved for the compiler's synthetic namespace
(match-scrutinee, lifted-let-rec, and shadow-rename binders).
Making `$` unauthorable keeps that synthetic namespace
collision-free by construction. The reservation is on the
Form-(A) authoring surface only — `$` inside a string literal or
a comment is unaffected, and the canonical `.ail.json`
deserialization path is intentionally not guarded.
Bool literals (`true`, `false`) and unit (`(lit-unit)`) are
disambiguated by parser context, not by lex. Comparison and
equality are class methods (`eq` / `compare`) and named fns
(`float_eq` / `float_lt` / etc.), not operators — see
[Prelude classes](../contracts/0017-prelude-classes.md).
Every AST node form has a unique head keyword (`module`, `data`,
`fn`, `forall`, `fn-type`, `con`, `var`, `app`, `lam`, `match`,
`case`, `pat-ctor`, `term-ctor`, `do`, `seq`, ...). A bare atom in
a positional slot (e.g. inside `(con List a)` second position) is
a name reference whose **sort** is determined by the parent slot:
- inside `(con NAME args...)` second-and-later positions → type
expression. Bare atom there ⇒ `Type::Var { name }`.
- inside `(app HEAD args...)` first position ⇒ `Term::Var`.
- inside `(pat-ctor CTOR fields...)` field positions ⇒
`Pattern::Var`.
- inside `(case PAT BODY)` second position ⇒ term.
There is **no lexical case rule**. To construct a value with a
ctor, write `(term-ctor TypeName CtorName args...)`. To match
against one, write `(pat-ctor CtorName fields...)`. Capitalised
identifiers carry no special meaning to the parser. This rules
out a class of silent errors ("I forgot to capitalise `Cons` and
it parsed as a function call").
**Pros:** smallest formal grammar of any candidate (the lexical
core is 3 rules; the named-form productions are uniform — every
node a tagged list). Foreign-LLM bar lowest. Round-trip with the
existing pretty-printer is a refactor of `pretty.rs` to emit this
tagged form, plus a new parser.
**Cons:** paren density is high. `(forall (vars a b) (fn-type
...))` has more visual nesting than the current pretty-printer's
`forall a. (...) -> ...`. Verbosity is ~2× JSON for the same node
when measured in characters, but ~8× shorter in lines (the
existing JSON `box.ail.json` of 160 lines becomes ~20 lines in
this form).
#### (B) Indented record-style with explicit terminators
```
module std_list
data List(a):
Nil
Cons(a, List(a))
end
fn map:
type: forall a b. fn(fn(a) -> b, List(a)) -> List(b)
params: f, xs
body:
match xs:
Nil => Nil
Cons(h, t) => Cons(f(h), map(f, t))
end
end
```
Grammar core (~2030 productions): module-level (def/data/end), type
sub-grammar (forall, fn, con, var), term sub-grammar (lam, match,
ctor, app, lit, var, seq), pattern sub-grammar.
**Pros:** higher information density per line, closer to mainstream
ML/Haskell shape. **Cons:** four sub-grammars instead of one.
`forall a b. fn(...)` keeps a pseudo-precedence (`->` binds tighter
than the outer `fn(...)` wrapper). Foreign-LLM bar higher.
#### (C) Pretty-printer-as-source
Use exactly the format `pretty::module` already emits, plus a parser
that accepts it. The existing pretty-printer's quirks (`::` for
type-of, `[params]` for fn-params, `<a>` for type-args, `forall a. ...`,
`!IO`, `()` ambiguous between unit-arg-list and empty-form) become
the spec.
**Pros:** zero churn — the existing pretty-printer is already the
spec; only the inverse is missing. Round-trip is the identity by
construction. **Cons:** the existing format mixes four mini-dialects
(s-expr at term level, ML-shape at type level, square brackets for
params, `<>` for type args). Formalising it crisply is harder than
designing a uniform form from scratch.
The chosen form is (A), shipped as the `ailang-surface` crate (see
[authoring surface](../contracts/0001-authoring-surface.md)); the
canonical JSON-AST it round-trips against is documented in
[Data model](../contracts/0002-data-model.md), and the round-trip
property itself in [Roundtrip Invariant](../contracts/0009-roundtrip-invariant.md).
For the prose projection used for human review (Form B), see
[prose projection](0006-prose-projection.md).