ail: merge-prose subcommand + PROSE_ROUNDTRIP.md (iter 20d)

Closes family 20: prose-edit cycle is end-to-end. ail merge-prose <original.ail.json> <edited.prose.txt> composes a shaped LLM prompt to stdout. User pipes to their LLM (Claude/etc.), gets new .ail.json, runs ail check, saves. No built-in API client — AILang stays a compiler + tooling, the LLM mediator is supplied externally. PROSE_ROUNDTRIP.md documents the 6-step cycle, the prompt contract (preserve modes/effects/tail flags from original), failure modes, and the design choice to omit an API client. Family 20 now closes: 20a renderer + 20b polish + 20d mediator shipped; 20c rolled into 20a. Three deferred polishes (let-inlining, print sugar, doc-wrap widow control) wait for corpus signal.
2026-05-08 18:22:46 +02:00
parent 9cf0e3e81c
commit 9c09dfbc8d
4 changed files with 510 additions and 0 deletions
@@ -9057,3 +9057,113 @@ re-rendered to incorporate the wrapping.
  corpus shows it's needed.
 - **`do io/print_int(x)` → `print(x)`** — needs type context
  to be safe; deferred.
+
+## 2026-05-08 — Iter 20d: prose round-trip mediator shipped
+
+The closing piece of family 20: a documented prompt template and a
+thin CLI helper that compose the prompt for the prose-edit cycle. No
+LLM client of our own — the user pipes the prompt to whichever model
+they prefer.
+
+### The cycle
+
+```
+1.  ail prose foo.ail.json > foo.prose.txt
+2.  $EDITOR foo.prose.txt                     # human edits freely
+3.  ail merge-prose foo.ail.json foo.prose.txt > prompt.txt
+4.  cat prompt.txt | <your-llm-cli> > foo.new.ail.json
+5.  ail check foo.new.ail.json
+6.  mv foo.new.ail.json foo.ail.json          # if check is clean
+```
+
+Step 1 was 20a/20b. Step 3 is this iter. Steps 5/6 are existing
+`ail check` + plain `mv`. The mediator is the LLM, not the
+compiler — which is why the cycle is iterative (re-run step 4 with
+the diagnostic pasted in if `ail check` rejects the result).
+
+### What was built
+
+  - **`docs/PROSE_ROUNDTRIP.md`** — new top-level doc, 153 lines.
+    Sections: *Why* (prose is lossy, re-integration needs LLM
+    mediation), *The cycle* (the 7-step pipeline above), *The
+    prompt template* (the literal text the CLI emits, so a human
+    can compose by hand), *Failure modes* (markdown fences,
+    invalid JSON, schema-valid-but-`ail check`-fails — fix in each
+    case is to paste the corrective note and re-prompt), *Why no
+    built-in API client* (deliberate scope: AILang stays a
+    compiler, the user already has clients).
+  - **`Cmd::MergeProse { original, edited }`** in
+    `crates/ail/src/main.rs`. Reads both files, calls the helper,
+    prints to stdout. Errors via `anyhow` context on file-read
+    failures.
+  - **`fn compose_merge_prose_prompt(orig_ail_json: &str,
+    edited_prose: &str) -> String`** — pure helper in `main.rs`.
+    Both payloads insert verbatim between heredoc-style markers
+    (`<<<ORIGINAL_AIL_JSON ... ORIGINAL_AIL_JSON`,
+    `<<<EDITED_PROSE ... EDITED_PROSE`); the framing is the
+    role/contract/output/schema-essentials sections that match
+    the prose in `PROSE_ROUNDTRIP.md` byte-for-byte.
+
+### Test surface
+
+  - **3 unit tests** in `main.rs` (`#[cfg(test)] mod tests`):
+    `merge_prose_prompt_contains_both_inputs_verbatim`,
+    `merge_prose_prompt_has_all_framing_sections`,
+    `merge_prose_prompt_is_deterministic`.
+  - **1 e2e CLI smoke test** in `tests/e2e.rs`:
+    `merge_prose_prints_framed_prompt` — writes two temp files,
+    invokes `ail merge-prose`, asserts exit 0 and that stdout
+    carries both inputs verbatim plus the role-statement
+    landmark.
+
+### Why no built-in API client
+
+AILang stays a compiler + tooling. Shipping an HTTP client to a
+specific LLM vendor would mean an API-key story, rate limiting,
+streaming semantics, error mapping, and version tracking — all
+carrying zero language-level value. Users already have Claude
+Code, the Anthropic SDK, the OpenAI CLI, `curl`, or any other
+client they prefer; `ail merge-prose` composes the prompt and
+gets out of the way. The Unix-pipe shape
+(`ail merge-prose ... | client | ail check`) makes the cycle
+scriptable without lock-in to one vendor. This is documented as a
+design choice in `docs/PROSE_ROUNDTRIP.md` so it stays a
+deliberate stance, not a gap.
+
+### Files
+
+  - `docs/PROSE_ROUNDTRIP.md` — new (153 lines).
+  - `crates/ail/src/main.rs` — `Cmd::MergeProse` variant,
+    dispatch arm, `compose_merge_prose_prompt` helper, inline
+    `mod tests` (3 unit tests). +~180 lines.
+  - `crates/ail/tests/e2e.rs` — `merge_prose_prints_framed_prompt`
+    smoke test (~50 lines).
+
+### Build/test status
+
+- `cargo build --workspace` — green, no warnings.
+- `cargo test --workspace` — all green: 3 new unit tests on the
+  `ail` binary, e2e count 69 → 70, prose-unit / prose-snapshot /
+  ailang-check / ailang-surface all unchanged-green.
+
+### State of family 20
+
+20a + 20b + 20d done. 20c was rolled into 20a (the `ail prose`
+subcommand was a natural unit with the renderer skeleton, not a
+separate iter). The family closes pending real-corpus signal on
+20b's deferred polishes (let-inlining, `do io/print_int(x)` →
+`print(x)`, doc-wrap widow control). If those don't turn up in
+practice the family stays as-shipped.
+
+### Out of scope (explicit, not deferred)
+
+  - Built-in API client. AILang shells out to no LLM. The Unix
+    pipe is the integration surface.
+  - AST-aware merge (semantic diff between original and re-emitted
+    `.ail.json`). The mediator is the LLM; if a structural diff
+    becomes useful later, `ail diff` already exists for that
+    purpose post-merge.
+  - Schema validation in the `merge-prose` CLI itself. `ail check`
+    validates the merge product; `merge-prose` only composes the
+    prompt.
+
@@ -0,0 +1,153 @@
+# Prose round-trip cycle
+
+## Why
+
+The prose surface (`ail prose`, Iter 20a/20b) is a deliberately
+**lossy** projection of a `.ail.json` module: it strips `(con T)`
+wrappings, drops redundant parentheses, infixes arithmetic, suppresses
+schema rigor that the LLM can re-derive. That makes prose pleasant to
+read and edit — but it also means there is no syntactic parser that
+turns edited prose back into a `.ail.json`.
+
+Re-integration of free-text edits therefore requires an **LLM
+mediator**: a model that understands both the prose intent and the
+load-bearing detail in the original `.ail.json`, and emits a new
+`.ail.json` that respects both. This document describes the workflow,
+the prompt template `ail merge-prose` composes for that mediator, and
+the failure modes to watch for.
+
+## The cycle
+
+```
+1.  ail prose foo.ail.json > foo.prose.txt
+2.  $EDITOR foo.prose.txt                     # human edits freely
+3.  ail merge-prose foo.ail.json foo.prose.txt > prompt.txt
+4.  cat prompt.txt | <your-llm-cli> > foo.new.ail.json
+5.  ail check foo.new.ail.json
+6.  mv foo.new.ail.json foo.ail.json          # if check is clean
+```
+
+Step 1 is the deterministic projection (Iter 20a/20b).
+Step 3 is the prompt composer this iter ships.
+Step 4 is the user's external LLM client — Claude Code, the
+Anthropic API, OpenAI's CLI, anything that takes a prompt on stdin and
+emits text on stdout. AILang ships no client of its own; see *Why no
+built-in API client* below.
+Steps 5 and 6 reuse existing tooling: `ail check` validates schema +
+typechecks, then the user accepts the new module.
+
+## The prompt template
+
+`ail merge-prose <original.ail.json> <edited.prose.txt>` reads both
+files and prints the following text to stdout. Nothing more, nothing
+less — no fences, no JSON envelope. The shape is exactly what a human
+would compose by hand if they wanted to drive the same cycle without
+the CLI helper.
+
+```
+You are integrating prose edits back into an AILang module.
+
+ROLE
+Your job is to produce an updated AILang JSON-AST (.ail.json) that
+reflects the human's prose edits while preserving the load-bearing
+semantic detail from the original .ail.json.
+
+CONTRACT
+The prose is the source of intent: the human edited it to express
+what the program should now do. The original .ail.json carries
+load-bearing detail that the prose surface elides — preserve those
+details unless the prose explicitly contradicts them. Specifically:
+
+  - Mode annotations on fn parameters (`own T`, `borrow T`).
+    These are hard contracts (memory model). The prose shows them,
+    but if the prose is ambiguous, default to the original.
+  - Effect annotations on return types (`with IO`, `with Diverge`).
+    Prose shows these too, but again: if uncertain, keep what the
+    original had.
+  - `tail` flags on calls. The prose prints `tail f(x)`; if the
+    edit moved a call, decide whether the new position is still in
+    tail position and flag accordingly.
+  - Doc strings (`///` lines).
+  - Type annotations on signatures and lambdas.
+  - Constructor names and arities (the prose `Cons(h, t)` must
+    round-trip to a `Term::Ctor` with the right `type` field).
+
+OUTPUT
+Output ONLY the new .ail.json bytes. No commentary, no markdown
+fences, no preamble or postscript. The output must be valid JSON
+parsable by `ail parse` (i.e. by `serde_json` against the
+`ailang/v0` schema). The first byte should be `{` and the last
+non-whitespace byte should be `}`.
+
+SCHEMA ESSENTIALS
+A full reference lives in docs/DESIGN.md ("Data model (MVP)"). The
+shape is:
+
+  - Module: { "schema": "ailang/v0", "name": ..., "imports": [...],
+    "defs": [...] }
+  - Defs are tagged via `kind` ("fn" | "type" | "const").
+  - Types are tagged via `k` ("con" | "fn" | "var" | "forall").
+  - Terms are tagged via `t` ("lit" | "var" | "app" | "let" | "if"
+    | "do" | "ctor" | "match" | "lam" | "seq").
+  - Patterns are tagged via `p` ("var" | "lit" | "ctor" | "wild").
+  - On `fn-type`: `param_modes` (one of "own"/"borrow" per
+    parameter) and `effects` are mandatory. `ret_mode` is mandatory
+    when the return type carries a mode.
+  - Constructor application uses `Term::Ctor` (`"t": "ctor"`,
+    fields `type` + `ctor` + `args`), NOT `Term::App`. The prose
+    surface elides the `type` tag (`Cons(h, t)` instead of
+    `(term-ctor IntList Cons h t)`) — re-introduce it.
+  - Base type names that appear bare in prose (`Int`, `Bool`,
+    `Unit`, user types like `IntList`) must be wrapped as
+    `{"k": "con", "name": "Int"}` etc. in the JSON.
+
+ORIGINAL .ail.json
+<<<ORIGINAL_AIL_JSON
+{original_ail_json_here}
+ORIGINAL_AIL_JSON
+
+EDITED PROSE
+<<<EDITED_PROSE
+{edited_prose_here}
+EDITED_PROSE
+
+Emit the updated .ail.json now.
+```
+
+The two payloads (original JSON, edited prose) are inserted verbatim
+between the heredoc-style markers. `merge-prose` does not strip,
+re-encode, or otherwise transform either input — both go in
+byte-for-byte.
+
+## Failure modes
+
+The mediator is an LLM, so the cycle is intentionally iterative.
+Common failure modes and their fix:
+
+  - **Output wrapped in markdown fences (` ```json ... ``` `).** Strip
+    the fences (`sed -n '/^{/,/^}/p'`) and re-run `ail check`. If the
+    LLM does this consistently, paste a corrective note ("emit raw
+    JSON, no fences") at the top of the next prompt.
+  - **Output contains commentary before/after the JSON.** Same
+    treatment. Most modern LLMs respect "OUTPUT ONLY" but not all.
+  - **Invalid JSON / `serde_json` rejects.** Re-run with the parse
+    error pasted into the prompt as a corrective note.
+  - **Schema-valid but `ail check` fails.** Re-run with the
+    diagnostic messages pasted in. Common cases: missing
+    `param_modes`, `Term::App` instead of `Term::Ctor`, bare type
+    names instead of `{"k":"con","name":"…"}`.
+  - **Preserved detail dropped (mode flipped, doc string lost).**
+    Re-run with a corrective note naming the missing detail.
+
+The cycle converges in 1–3 rounds for typical edits.
+
+## Why no built-in API client
+
+AILang stays a compiler + tooling. Shipping an HTTP client to one
+particular LLM vendor would mean an API key story, rate limiting,
+streaming semantics, error mapping, version tracking — all carrying
+zero language-level value. The user already has Claude Code, the
+Anthropic SDK, the OpenAI CLI, `curl`, or any other client they
+prefer; `ail merge-prose` composes the prompt and gets out of the
+way. The Unix-pipe shape (`ail merge-prose ... | client | ail check`)
+also makes the cycle scriptable without lock-in to one vendor.