ail: embed Form-A spec in merge-prose prompt (iter 20f)

Closes a design hole shipped in 20d: the merge-prose prompt instructed
the LLM to emit JSON-AST and gave a 12-line schema-essentials reminder.
JSON-AST is the canonical hashable artefact, not a writing surface; the
reminder was not a language spec. Foreign LLMs had no realistic shot.

20f makes two coupled changes:

1) The LLM now emits Form-A (the canonical authoring surface fixed by
   Decision 6). merge-prose loads the original via ailang_core::load_module
   and re-renders via ailang_surface::print before embedding (round-trip
   is a gating contract on the surface crate, so this is lossless). The
   user runs `ail parse foo.new.ailx` to recover JSON, then `ail check`.

2) crates/ailang-core/specs/form_a.md is the complete LLM-targeted
   Form-A specification — grammar, every term/pattern/type/def keyword,
   schema invariants, pitfall catalogue, four few-shot modules from
   examples/*.ailx. Exported as ailang_core::FORM_A_SPEC via include_str!
   and embedded verbatim in every merge-prose prompt.

Drift detection in crates/ailang-core/tests/spec_drift.rs: every variant
of Term, Pattern, Type, Def, Literal is reached via exhaustive `match`.
The arms are not the assertion — adding a new variant without updating
the match is a compile error before the test runs. Once matched, an
anchor string is asserted to appear in FORM_A_SPEC. 8 tests, all green.

The hand-written + mechanical-drift-test combo addresses the user's
"distance to code is too big" concern about a docs-only spec. Generator
overkill rejected: structural drift is mechanically caught, but prose
explanation, schema-invariant catalogue, pitfall list, and few-shot
corpus cannot be emitted from AST shape alone.

Tests:
  - ailang-core: +8 spec_drift tests; existing 12 unchanged
  - ail unit (3): rewritten in lockstep — assert (own T)/(effects IO)
    landmarks, FORM-A SPECIFICATION header, FORM_A_SPEC body verbatim
  - ail e2e merge_prose_prints_framed_prompt: rewritten — assert
    `(module foo` + `FORM-A SPECIFICATION` instead of `ailang/v0`
  - Workspace: all green

Richer integration paths (LLM tool-use, MCP server, LSP) were named
in the design discussion and deferred per "kiss". All three layer
additively on the static-prompt path; static prompt remains the
lowest-common-denominator fallback.
This commit is contained in:
2026-05-08 23:31:27 +02:00
parent 8375eb81ed
commit 72626fa94f
8 changed files with 1070 additions and 197 deletions
+106
View File
@@ -9535,3 +9535,109 @@ default):
handling).
- **Data model (MVP) refresh.** Cosmetic doc-tidy iter to
bring the schema snapshot current.
## 2026-05-08 — Iter 20f: Form-A spec embedding for prose round-trip
This iter closes a design hole shipped in 20d: the `merge-prose`
prompt instructed the LLM to emit JSON-AST, and gave it only a
12-line "schema essentials" reminder. JSON-AST is the canonical
hashable artefact, but it is not a writing surface, and a 12-line
hint is not a language reference. A foreign LLM with no AILang
exposure had no realistic shot at producing valid output.
20f makes two coupled changes.
### Form-A becomes the LLM's output target
The prompt now instructs the LLM to emit **Form-A** — the canonical
authoring surface fixed by Decision 6. The user's CLI cycle
becomes:
ail prose foo.ail.json > foo.prose.txt
$EDITOR foo.prose.txt
ail merge-prose foo.ail.json foo.prose.txt > prompt.txt
cat prompt.txt | <llm> > foo.new.ailx
ail parse foo.new.ailx > foo.new.ail.json
ail check foo.new.ail.json
mv foo.new.ail.json foo.ail.json # if check is clean
The original module is loaded via `ailang_core::load_module`
(schema-validating) and re-rendered via `ailang_surface::print` for
embedding. Form-A round-trip is a gating contract on the surface
crate, so this is lossless.
### Form-A spec embedded in every prompt
`crates/ailang-core/specs/form_a.md` is a hand-curated, complete
LLM-targeted specification of Form-A — grammar, every term /
pattern / type / def keyword with parenthesised syntax, schema
invariants enforced by the checker, a pitfall catalogue, and four
few-shot modules drawn from `examples/*.ailx`. It is exported as
`pub const FORM_A_SPEC: &str = include_str!("../specs/form_a.md")`
and embedded verbatim in the merge-prose prompt.
### Drift detection
The spec is hand-written, so drift was the load-bearing concern in
the design discussion (orchestrator note: user pushed back on
"docs/AIL_FORM_A_SPEC.md handgeschrieben" precisely because the
distance to the code was too big). The fix is mechanical:
`crates/ailang-core/tests/spec_drift.rs` walks every variant of
`Term`, `Pattern`, `Type`, `Def`, `Literal`, `ParamMode` via
exhaustive `match`. The arms are not the assertion — adding a new
variant without updating the match is a compile error in this
test, before the test even runs. Once the variant is matched, the
test asserts an anchor string for it appears in `FORM_A_SPEC`.
Eight tests, all green.
The exhaustive-match-as-trip-wire pattern is the same idea as the
`ailang-architect` drift review for DESIGN.md — both surfaces
encode load-bearing language semantics that must stay current —
but it runs on every `cargo test`, not just at family boundaries.
### Why hand-written and not generated
A generator (walking `syn` / proc-macro reflection on the AST,
emitting a tag table) would close the structural-drift channel
mechanically, but the spec is **not** just a tag table. It carries
prose explanation per construct, the schema-invariant catalogue,
the pitfall list, and the few-shot corpus. None of those can be
emitted from the AST shape alone. Hand-curated content + drift
test on the structural anchors is the right cost / value point;
generator overkill was rejected.
### What 20f does not address
Three larger integration paths got named in the discussion but
explicitly deferred per "kiss":
- **Tool-use schemas** — LLM calls `ail_parse` / `ail_check`
inside its turn, iterating until check is clean. Reduces
convergence from 12 rounds to ≈0 for cooperating clients.
- **MCP server** — Anthropic Model Context Protocol exposes
AILang resources (spec, examples, current module), tools
(parse, check, render, prose), prompts (canonical
round-trip templates) to any compatible client. Standard,
discoverable, vendor-neutral.
- **LSP** — editor integration; serves human authors more than
the round-trip flow. Bigger lift.
All three layer additively on the static-prompt path 20f ships.
The static prompt remains the lowest-common-denominator fallback
that always works without a tool-use-capable client.
### Test counts
- ailang-core: +8 (spec drift suite); existing 12 unchanged → 20
- ail: existing 70 e2e green after rewriting
`merge_prose_prints_framed_prompt` to assert Form-A landmarks
and `FORM-A SPECIFICATION` header instead of `ailang/v0`
- ail unit: existing 3 rewritten in lockstep
- Workspace: all green
### Family / arc state
20f closes the prose-roundtrip arc for now. JOURNAL queue is
empty again. Tidy iter for 20a / b / d / e / f together can wait
until the next family closes — the architect drift-review
discipline applies as before.