Iter 14b: design pass for the authoring surface
User redirected at iter boundary: writing a stdlib in JSON was the wrong move. The language is supposed to be the one I program *best* in, and JSON-AST authoring is rationalisation, not strength. DESIGN.md Decision 6 captures the constraints that fall out of the "formalisable for a foreign LLM" hard requirement (no precedence, no semantic indentation, ASCII only, every AST node a uniquely-tagged form), sketches three candidate notations with the same `map` encoded in each, and picks form (A) — fully-tagged S-expressions — as the first attempt with explicit rollback path to form (C) if (A) hurts authoring. Form (A) shape: - 3-rule lexical core: sexpr / atom / token-classified-by- first-character. - Every AST node has a unique head keyword. No case-rule (no "capitalised head means ctor"); ctors are explicit via `(term-ctor TypeName CtorName args)` and `(pat-ctor CtorName fields)`. - Bare atoms get their sort from the parent slot (type-var inside `(con NAME args)`, term-var inside `(app HEAD args)`, pat-var inside `(pat-ctor CTOR fields)`, integer literal in term position, etc.). Empirical exhibits, hand-encoded: - examples/hello.ailx 5 LOC (JSON was 36 pretty / 21 canonical) - examples/box.ailx 25 LOC (JSON was 160 / 88) - examples/list_map_poly.ailx 50 LOC (JSON was 394 / 230) 4-8x line reduction, ~4x character reduction. Bigger gains on bigger programs since overhead is proportional to AST depth. None are parseable yet — header comments say "Iter 14b design exhibit, parser lands in 14c". Two small spec issues caught while writing the exhibits and folded back into DESIGN.md before committing: - Operator idents (`+`, `==`) need the token-by-first-char classification rule, not a word-shaped regex. - Bool literals (`true`/`false`) reserved in term context; unit is explicit `(lit-unit)`. Tests unchanged (this iter is paper). 25/25 e2e green. cargo doc --no-deps zero warnings. Plan 14c: new crate `ailang-surface` with PEG parser, round-trip hash-equivalence gate against every existing `examples/*.ail.json`, CLI subcommand `ail parse`. If round-trip holds, stdlib starts in `.ailx` form (Iter 14d). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
+115
@@ -1513,3 +1513,118 @@ itself surface more dogfood bugs. The original 14b
|
||||
(GC/arena) and 14c (poly fn as value) remain on the queue
|
||||
but have not had a design pass.
|
||||
|
||||
## Iter 14b — design pass for the authoring surface
|
||||
|
||||
User redirected the project at the iter boundary. I had been
|
||||
about to write a stdlib in JSON-AST form; user pushed back: do
|
||||
I really program *best* in JSON, given the language is supposed
|
||||
to be the one I'm most accurate in? Honest answer was no — JSON
|
||||
was rationalisation. The constraint added in this iter:
|
||||
|
||||
> "Die Syntax sollte gut formalisierbar sein, damit man auch
|
||||
> einem fremden LLM eine Spec geben kann, die es dann fehlerfrei
|
||||
> befolgt."
|
||||
|
||||
That single sentence ruled out about half the design space:
|
||||
anything with operator precedence (precedence is the #1 thing
|
||||
an LLM gets wrong when handed a spec, because it requires
|
||||
building a parse-tree mental model rather than just following
|
||||
production rules), anything with semantic indentation, anything
|
||||
with maximal-munch lexing or context-sensitive reductions.
|
||||
|
||||
What remains is roughly: S-expressions, or dialects close to
|
||||
S-expressions. Decision 6 in DESIGN.md captures the constraints
|
||||
in priority order, sketches three candidate forms (A: tagged
|
||||
S-expression, B: indented record-style, C: pretty-printer-as-
|
||||
source), and picks (A) as the first attempt with explicit
|
||||
rollback to (C) if (A) hurts authoring more than it helps.
|
||||
|
||||
**Key shape of (A):** every AST node has a unique head keyword.
|
||||
No case-disambiguation rule (no "capitalised head means ctor").
|
||||
Bare atoms in positional slots get their sort from the parent
|
||||
slot (inside `(con NAME args...)` second-and-later positions
|
||||
are types; inside `(app HEAD args...)` first position is a term;
|
||||
etc.). To construct a value with a ctor, write
|
||||
`(term-ctor TypeName CtorName args...)` explicitly. To match,
|
||||
`(pat-ctor CtorName fields...)`. The capitalisation/case of an
|
||||
identifier carries no semantic weight to the parser.
|
||||
|
||||
This rules out the silent-error class "I forgot to capitalise
|
||||
`Cons` and it parsed as a function call" — which I had been
|
||||
relying on case-conventions to prevent in the JSON form too,
|
||||
but only by hand-discipline. With explicit tags the discipline
|
||||
becomes a parse rule.
|
||||
|
||||
**Empirical test (this iter).** Hand-encoded three examples in
|
||||
form (A) as `examples/*.ailx`:
|
||||
|
||||
```
|
||||
hello.ailx — 5 LOC (was JSON: 36 pretty / 21 canonical)
|
||||
box.ailx — 25 LOC (was JSON: 160 pretty / 88 canonical)
|
||||
list_map_poly.ailx — 50 LOC (was JSON: 394 pretty / 230 canonical)
|
||||
```
|
||||
|
||||
Roughly 4–8× line reduction, ~4× character reduction. Bigger
|
||||
gains on bigger programs (overhead is proportional to AST
|
||||
depth, not to program size, so the form scales well). All three
|
||||
mapped unambiguously to AST nodes by inspection — no
|
||||
ambiguities surfaced during writing that the spec didn't
|
||||
already cover.
|
||||
|
||||
Two small lex/grammar issues I discovered while writing and
|
||||
folded back into DESIGN.md before committing:
|
||||
|
||||
1. **Operator idents** like `+`, `==`, `<=`. Initial spec had a
|
||||
word-shaped `[A-Za-z_]...` regex; that excluded operators.
|
||||
Fix: lexer recognises only `(`/`)`/whitespace as delimiters.
|
||||
Every other maximal token is classified by first character
|
||||
(digit ⇒ integer, `"` ⇒ string, else ⇒ ident). `+`, `42`,
|
||||
`io/print_int`, `==` all become single ident or integer
|
||||
tokens with no special rule.
|
||||
2. **Bool literals.** Bare `true`/`false` are reserved in term
|
||||
context; outside term context they would be ill-formed
|
||||
anyway. Unit is explicit: `(lit-unit)`.
|
||||
|
||||
**Files touched:**
|
||||
- `docs/DESIGN.md` — Decision 6 added (~140 lines), with
|
||||
constraints, three candidates, first-choice rationale, and
|
||||
an implementation outline for Iter 14c.
|
||||
- `examples/hello.ailx`, `examples/box.ailx`,
|
||||
`examples/list_map_poly.ailx` — three design exhibits. Not
|
||||
parseable yet (header comment says so).
|
||||
- `docs/JOURNAL.md` — this entry.
|
||||
|
||||
**Tests:** None new. Existing 25/25 e2e + 3 ignored doctests
|
||||
unchanged (this iter is paper, not code). `cargo doc --no-deps`
|
||||
0 warnings (workspace invariant from 13d/e/f preserved).
|
||||
|
||||
**Process note: orchestration with explicit licence to be
|
||||
wrong.** User said "du darfst auch ausprobieren und dich irren
|
||||
(und es dann rückgängig machen). Wir wollen das beste Design
|
||||
für den propagierten Zweck." That changes the cost model for
|
||||
design iteration: I should optimise for *information per iter*,
|
||||
not for *correctness on first commit*. So I committed form (A)
|
||||
as a working hypothesis with a documented rollback path to
|
||||
form (C), rather than design-by-committee until I was sure.
|
||||
|
||||
**Plan 14c (next).**
|
||||
1. New crate `ailang-surface` with a small PEG parser → existing
|
||||
`ailang-core::ast` types. No new AST nodes.
|
||||
2. Round-trip test gate: every existing `examples/*.ail.json`
|
||||
gets a sibling `*.ailx` written by hand or by an
|
||||
AST→surface emitter; the test parses the surface, canonises
|
||||
to JSON, and asserts hash-equivalence to the original. If a
|
||||
single fixture loses its hash, the form does not ship.
|
||||
3. CLI: `ail parse <file.ailx> -o <file.ail.json>`. Symmetric
|
||||
to existing `ail render`.
|
||||
4. If round-trip works for all current fixtures, mark form (A)
|
||||
confirmed and start the stdlib in `.ailx` directly. If it
|
||||
fails, document why in JOURNAL and try (C).
|
||||
|
||||
**Plan 14d (after 14c).** First stdlib module: `std_list.ailx`
|
||||
with `length`, `filter`, `fold`, `concat`, `reverse`, `head`,
|
||||
`tail`. Each combinator is a fresh test vector for codegen and
|
||||
for the still-young parameterised-ADT pipeline. Written in the
|
||||
new surface from day one — no JSON authoring of stdlib.
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user