Files
AILang/docs/PROSE_ROUNDTRIP.md
T
Brummel 72626fa94f ail: embed Form-A spec in merge-prose prompt (iter 20f)
Closes a design hole shipped in 20d: the merge-prose prompt instructed
the LLM to emit JSON-AST and gave a 12-line schema-essentials reminder.
JSON-AST is the canonical hashable artefact, not a writing surface; the
reminder was not a language spec. Foreign LLMs had no realistic shot.

20f makes two coupled changes:

1) The LLM now emits Form-A (the canonical authoring surface fixed by
   Decision 6). merge-prose loads the original via ailang_core::load_module
   and re-renders via ailang_surface::print before embedding (round-trip
   is a gating contract on the surface crate, so this is lossless). The
   user runs `ail parse foo.new.ailx` to recover JSON, then `ail check`.

2) crates/ailang-core/specs/form_a.md is the complete LLM-targeted
   Form-A specification — grammar, every term/pattern/type/def keyword,
   schema invariants, pitfall catalogue, four few-shot modules from
   examples/*.ailx. Exported as ailang_core::FORM_A_SPEC via include_str!
   and embedded verbatim in every merge-prose prompt.

Drift detection in crates/ailang-core/tests/spec_drift.rs: every variant
of Term, Pattern, Type, Def, Literal is reached via exhaustive `match`.
The arms are not the assertion — adding a new variant without updating
the match is a compile error before the test runs. Once matched, an
anchor string is asserted to appear in FORM_A_SPEC. 8 tests, all green.

The hand-written + mechanical-drift-test combo addresses the user's
"distance to code is too big" concern about a docs-only spec. Generator
overkill rejected: structural drift is mechanically caught, but prose
explanation, schema-invariant catalogue, pitfall list, and few-shot
corpus cannot be emitted from AST shape alone.

Tests:
  - ailang-core: +8 spec_drift tests; existing 12 unchanged
  - ail unit (3): rewritten in lockstep — assert (own T)/(effects IO)
    landmarks, FORM-A SPECIFICATION header, FORM_A_SPEC body verbatim
  - ail e2e merge_prose_prints_framed_prompt: rewritten — assert
    `(module foo` + `FORM-A SPECIFICATION` instead of `ailang/v0`
  - Workspace: all green

Richer integration paths (LLM tool-use, MCP server, LSP) were named
in the design discussion and deferred per "kiss". All three layer
additively on the static-prompt path; static prompt remains the
lowest-common-denominator fallback.
2026-05-08 23:31:27 +02:00

6.5 KiB
Raw Blame History

Prose round-trip cycle

Why

The prose surface (ail prose, Iter 20a / 20b) is a deliberately lossy projection of an .ail.json module: it strips (con T) wrappings, drops redundant parentheses, infixes arithmetic, suppresses schema rigor that the LLM can re-derive. That makes prose pleasant to read and edit — but it also means there is no syntactic parser that turns edited prose back into a .ail.json.

Re-integration of free-text edits therefore requires an LLM mediator: a model that understands both the prose intent and the load-bearing detail in the original module, and emits a new module that respects both. This document describes the workflow, the prompt template ail merge-prose composes for that mediator, and the failure modes to watch for.

What the LLM produces

Iter 20f revised a load-bearing piece of this cycle: the LLM emits Form-A (an .ailx document — the canonical authoring surface fixed by Decision 6), not JSON-AST. JSON-AST is the canonical hashable artefact, but it is not a writing surface — every type reference wraps in {"k": "con", ...}, every term in {"t": "...", ...}, and a single missing param_modes entry is a schema error rather than a parse error with a position.

Form-A is the form examples/*.ailx are written in, the form ail render produces, and the form ail parse consumes. The full LLM-targeted specification is shipped inside the binary as ailang_core::FORM_A_SPEC (sourced from crates/ailang-core/specs/form_a.md) and embedded verbatim in every merge-prose prompt; the LLM does not need prior AILang exposure.

The cycle

1.  ail prose foo.ail.json > foo.prose.txt
2.  $EDITOR foo.prose.txt                     # human edits freely
3.  ail merge-prose foo.ail.json foo.prose.txt > prompt.txt
4.  cat prompt.txt | <your-llm-cli> > foo.new.ailx
5.  ail parse foo.new.ailx > foo.new.ail.json
6.  ail check foo.new.ail.json
7.  mv foo.new.ail.json foo.ail.json          # if check is clean

Step 1 is the deterministic projection (Iter 20a / 20b). Step 3 is the prompt composer this iter ships. The prompt embeds the Form-A spec, the original module rendered as Form-A, and the edited prose. The original module is loaded via ailang_core::load_module (which validates the schema) and rendered via ailang_surface::print. Step 4 is the user's external LLM client — Claude Code, the Anthropic API, OpenAI's CLI, anything that takes a prompt on stdin and emits text on stdout. AILang ships no client of its own; see Why no built-in API client below. Step 5 parses the LLM's Form-A back into a JSON-AST. Parser errors are positional and can be fed back to the LLM in a follow-up prompt. Step 6 typechecks. The full diagnostic catalogue applies — mode violations, exhaustiveness, effect closure, etc. Step 7 accepts the new module if check is clean.

The prompt template

ail merge-prose <original.ail.json> <edited.prose.txt> reads the original (parsing it, then re-rendering as Form-A), reads the prose file byte-for-byte, and prints the following structure to stdout — no fences, no JSON envelope. The shape is exactly what a human would compose by hand if they wanted to drive the same cycle without the CLI helper.

You are integrating prose edits back into an AILang module.

ROLE
Your job is to produce an updated AILang module in Form-A (an .ailx
file) that reflects the human's prose edits while preserving the
load-bearing semantic detail from the original module.

CONTRACT
The prose is the source of intent ... [list of preservation rules]

OUTPUT
Output ONLY the new Form-A bytes. ... [no fences, no commentary]

FORM-A SPECIFICATION
<<<FORM_A_SPEC
[ailang_core::FORM_A_SPEC, verbatim]
FORM_A_SPEC

ORIGINAL MODULE (Form-A)
<<<ORIGINAL_FORM_A
[ailang_surface::print(&original_module)]
ORIGINAL_FORM_A

EDITED PROSE
<<<EDITED_PROSE
[edited prose, verbatim]
EDITED_PROSE

Emit the updated Form-A module now.

The three payloads (spec, original Form-A, edited prose) are inserted between heredoc-style markers. merge-prose does not strip, re-encode, or otherwise transform the prose or the spec. The original is parsed and re-rendered (which is lossless for any module that has been through ail parse / ail render already; round-trip is a gating contract on the surface crate).

Failure modes

The mediator is an LLM, so the cycle is intentionally iterative. Common failure modes and their fix:

  • Output wrapped in markdown fences (```ailang ... ```). Strip the fences and re-run ail parse. If the LLM does this consistently, paste a corrective note ("emit raw Form-A, no fences") at the top of the next prompt.
  • Output contains commentary before/after the module. Same treatment. Most modern LLMs respect "OUTPUT ONLY" but not all.
  • ail parse rejects the output. The parser emits positional errors. Re-run with the parse error pasted into the prompt as a corrective note — "your previous output failed at byte N with expected X, got Y; fix that and re-emit".
  • Parse-clean but ail check fails. Re-run with the diagnostic messages pasted in. Common cases on first-time LLM use: missing mode annotation on a (fn ...) def (mode-required-on-fn-param), (app) used where (term-ctor) was needed, missing (effects IO) on a fn that calls (do io/...).
  • Preserved detail dropped (mode flipped, doc string lost, suppress clause stripped). Re-run with a corrective note naming the missing detail. The contract section calls these out explicitly, but a long edit may push the LLM to re-derive rather than preserve.

The cycle converges in 12 rounds for typical edits with a modern frontier model.

Why no built-in API client

AILang stays a compiler + tooling. Shipping an HTTP client to one particular LLM vendor would mean an API key story, rate limiting, streaming semantics, error mapping, version tracking — all carrying zero language-level value. The user already has Claude Code, the Anthropic SDK, the OpenAI CLI, curl, or any other client they prefer; ail merge-prose composes the prompt and gets out of the way. The Unix-pipe shape (ail merge-prose ... | client | ail parse | ail check) makes the cycle scriptable without lock-in to one vendor.

A future iter may layer a tool-use schema (LLM calls back into ail parse / ail check from inside its turn) or an MCP server (any MCP-compatible client discovers AILang's resources, tools, and prompts) on top of this cycle. Both are additive — the static-prompt fallback documented here remains the lowest-common-denominator path that always works.