Iter 14b: design pass for the authoring surface

User redirected at iter boundary: writing a stdlib in JSON was
the wrong move. The language is supposed to be the one I program
*best* in, and JSON-AST authoring is rationalisation, not
strength.

DESIGN.md Decision 6 captures the constraints that fall out of
the "formalisable for a foreign LLM" hard requirement (no
precedence, no semantic indentation, ASCII only, every AST node
a uniquely-tagged form), sketches three candidate notations with
the same `map` encoded in each, and picks form (A) — fully-tagged
S-expressions — as the first attempt with explicit rollback path
to form (C) if (A) hurts authoring.

Form (A) shape:
- 3-rule lexical core: sexpr / atom / token-classified-by-
  first-character.
- Every AST node has a unique head keyword. No case-rule (no
  "capitalised head means ctor"); ctors are explicit via
  `(term-ctor TypeName CtorName args)` and `(pat-ctor CtorName
  fields)`.
- Bare atoms get their sort from the parent slot (type-var
  inside `(con NAME args)`, term-var inside `(app HEAD args)`,
  pat-var inside `(pat-ctor CTOR fields)`, integer literal in
  term position, etc.).

Empirical exhibits, hand-encoded:
- examples/hello.ailx          5 LOC (JSON was 36 pretty / 21 canonical)
- examples/box.ailx           25 LOC (JSON was 160 / 88)
- examples/list_map_poly.ailx 50 LOC (JSON was 394 / 230)

4-8x line reduction, ~4x character reduction. Bigger gains on
bigger programs since overhead is proportional to AST depth.
None are parseable yet — header comments say "Iter 14b design
exhibit, parser lands in 14c".

Two small spec issues caught while writing the exhibits and
folded back into DESIGN.md before committing:
- Operator idents (`+`, `==`) need the token-by-first-char
  classification rule, not a word-shaped regex.
- Bool literals (`true`/`false`) reserved in term context;
  unit is explicit `(lit-unit)`.

Tests unchanged (this iter is paper). 25/25 e2e green.
cargo doc --no-deps zero warnings.

Plan 14c: new crate `ailang-surface` with PEG parser, round-trip
hash-equivalence gate against every existing `examples/*.ail.json`,
CLI subcommand `ail parse`. If round-trip holds, stdlib starts
in `.ailx` form (Iter 14d).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 15:57:24 +02:00
parent 747b7cd05c
commit 2bce825b69
5 changed files with 445 additions and 0 deletions
+115
View File
@@ -1513,3 +1513,118 @@ itself surface more dogfood bugs. The original 14b
(GC/arena) and 14c (poly fn as value) remain on the queue
but have not had a design pass.
## Iter 14b — design pass for the authoring surface
User redirected the project at the iter boundary. I had been
about to write a stdlib in JSON-AST form; user pushed back: do
I really program *best* in JSON, given the language is supposed
to be the one I'm most accurate in? Honest answer was no — JSON
was rationalisation. The constraint added in this iter:
> "Die Syntax sollte gut formalisierbar sein, damit man auch
> einem fremden LLM eine Spec geben kann, die es dann fehlerfrei
> befolgt."
That single sentence ruled out about half the design space:
anything with operator precedence (precedence is the #1 thing
an LLM gets wrong when handed a spec, because it requires
building a parse-tree mental model rather than just following
production rules), anything with semantic indentation, anything
with maximal-munch lexing or context-sensitive reductions.
What remains is roughly: S-expressions, or dialects close to
S-expressions. Decision 6 in DESIGN.md captures the constraints
in priority order, sketches three candidate forms (A: tagged
S-expression, B: indented record-style, C: pretty-printer-as-
source), and picks (A) as the first attempt with explicit
rollback to (C) if (A) hurts authoring more than it helps.
**Key shape of (A):** every AST node has a unique head keyword.
No case-disambiguation rule (no "capitalised head means ctor").
Bare atoms in positional slots get their sort from the parent
slot (inside `(con NAME args...)` second-and-later positions
are types; inside `(app HEAD args...)` first position is a term;
etc.). To construct a value with a ctor, write
`(term-ctor TypeName CtorName args...)` explicitly. To match,
`(pat-ctor CtorName fields...)`. The capitalisation/case of an
identifier carries no semantic weight to the parser.
This rules out the silent-error class "I forgot to capitalise
`Cons` and it parsed as a function call" — which I had been
relying on case-conventions to prevent in the JSON form too,
but only by hand-discipline. With explicit tags the discipline
becomes a parse rule.
**Empirical test (this iter).** Hand-encoded three examples in
form (A) as `examples/*.ailx`:
```
hello.ailx — 5 LOC (was JSON: 36 pretty / 21 canonical)
box.ailx — 25 LOC (was JSON: 160 pretty / 88 canonical)
list_map_poly.ailx — 50 LOC (was JSON: 394 pretty / 230 canonical)
```
Roughly 48× line reduction, ~4× character reduction. Bigger
gains on bigger programs (overhead is proportional to AST
depth, not to program size, so the form scales well). All three
mapped unambiguously to AST nodes by inspection — no
ambiguities surfaced during writing that the spec didn't
already cover.
Two small lex/grammar issues I discovered while writing and
folded back into DESIGN.md before committing:
1. **Operator idents** like `+`, `==`, `<=`. Initial spec had a
word-shaped `[A-Za-z_]...` regex; that excluded operators.
Fix: lexer recognises only `(`/`)`/whitespace as delimiters.
Every other maximal token is classified by first character
(digit ⇒ integer, `"` ⇒ string, else ⇒ ident). `+`, `42`,
`io/print_int`, `==` all become single ident or integer
tokens with no special rule.
2. **Bool literals.** Bare `true`/`false` are reserved in term
context; outside term context they would be ill-formed
anyway. Unit is explicit: `(lit-unit)`.
**Files touched:**
- `docs/DESIGN.md` — Decision 6 added (~140 lines), with
constraints, three candidates, first-choice rationale, and
an implementation outline for Iter 14c.
- `examples/hello.ailx`, `examples/box.ailx`,
`examples/list_map_poly.ailx` — three design exhibits. Not
parseable yet (header comment says so).
- `docs/JOURNAL.md` — this entry.
**Tests:** None new. Existing 25/25 e2e + 3 ignored doctests
unchanged (this iter is paper, not code). `cargo doc --no-deps`
0 warnings (workspace invariant from 13d/e/f preserved).
**Process note: orchestration with explicit licence to be
wrong.** User said "du darfst auch ausprobieren und dich irren
(und es dann rückgängig machen). Wir wollen das beste Design
für den propagierten Zweck." That changes the cost model for
design iteration: I should optimise for *information per iter*,
not for *correctness on first commit*. So I committed form (A)
as a working hypothesis with a documented rollback path to
form (C), rather than design-by-committee until I was sure.
**Plan 14c (next).**
1. New crate `ailang-surface` with a small PEG parser → existing
`ailang-core::ast` types. No new AST nodes.
2. Round-trip test gate: every existing `examples/*.ail.json`
gets a sibling `*.ailx` written by hand or by an
AST→surface emitter; the test parses the surface, canonises
to JSON, and asserts hash-equivalence to the original. If a
single fixture loses its hash, the form does not ship.
3. CLI: `ail parse <file.ailx> -o <file.ail.json>`. Symmetric
to existing `ail render`.
4. If round-trip works for all current fixtures, mark form (A)
confirmed and start the stdlib in `.ailx` directly. If it
fails, document why in JOURNAL and try (C).
**Plan 14d (after 14c).** First stdlib module: `std_list.ailx`
with `length`, `filter`, `fold`, `concat`, `reverse`, `head`,
`tail`. Each combinator is a fresh test vector for codegen and
for the still-young parameterised-ADT pipeline. Written in the
new surface from day one — no JSON authoring of stdlib.