ail: merge-prose subcommand + PROSE_ROUNDTRIP.md (iter 20d)

Closes family 20: prose-edit cycle is end-to-end. ail merge-prose
<original.ail.json> <edited.prose.txt> composes a shaped LLM prompt
to stdout. User pipes to their LLM (Claude/etc.), gets new .ail.json,
runs ail check, saves.

No built-in API client — AILang stays a compiler + tooling, the LLM
mediator is supplied externally. PROSE_ROUNDTRIP.md documents the
6-step cycle, the prompt contract (preserve modes/effects/tail
flags from original), failure modes, and the design choice to omit
an API client.

Family 20 now closes: 20a renderer + 20b polish + 20d mediator
shipped; 20c rolled into 20a. Three deferred polishes (let-inlining,
print sugar, doc-wrap widow control) wait for corpus signal.
This commit is contained in:
2026-05-08 18:22:46 +02:00
parent 9cf0e3e81c
commit 9c09dfbc8d
4 changed files with 510 additions and 0 deletions
+199
View File
@@ -207,6 +207,111 @@ enum Cmd {
/// surface remains form (A) (`render` / `parse`) and the canonical
/// hashable artefact remains the JSON-AST.
Prose { path: PathBuf },
/// Iter 20d: composes a prompt for the prose-edit round-trip.
///
/// Given the original `.ail.json` and the edited `.prose.txt`,
/// prints a prompt that asks an external LLM to emit an updated
/// `.ail.json` integrating the prose edits. AILang ships no LLM
/// client of its own; the user pipes the prompt to their tool of
/// choice (Claude Code, Anthropic API, OpenAI CLI, etc.) and runs
/// `ail check` on the result.
///
/// See `docs/PROSE_ROUNDTRIP.md` for the full cycle and the
/// rationale for keeping the API client out of scope.
MergeProse {
/// Original `.ail.json` (carries load-bearing detail the prose elides).
original: PathBuf,
/// Edited `.prose.txt` (carries the human's intent).
edited: PathBuf,
},
}
/// Composes the prose-round-trip prompt described in
/// `docs/PROSE_ROUNDTRIP.md`.
///
/// The two payloads — the original `.ail.json` and the edited prose —
/// are inserted verbatim between heredoc-style markers; the rest of
/// the returned string is the role-statement, contract, output spec,
/// and schema-essentials reminder that frame the LLM's task.
///
/// Pure: same inputs always yield the same bytes. The CLI wrapper
/// (`Cmd::MergeProse`) is just a file-reader + `print!`.
fn compose_merge_prose_prompt(orig_ail_json: &str, edited_prose: &str) -> String {
// Keep this template in lockstep with `docs/PROSE_ROUNDTRIP.md` —
// the doc reproduces the literal text so a human can assemble the
// same prompt by hand.
format!(
"You are integrating prose edits back into an AILang module.
ROLE
Your job is to produce an updated AILang JSON-AST (.ail.json) that
reflects the human's prose edits while preserving the load-bearing
semantic detail from the original .ail.json.
CONTRACT
The prose is the source of intent: the human edited it to express
what the program should now do. The original .ail.json carries
load-bearing detail that the prose surface elides — preserve those
details unless the prose explicitly contradicts them. Specifically:
- Mode annotations on fn parameters (`own T`, `borrow T`).
These are hard contracts (memory model). The prose shows them,
but if the prose is ambiguous, default to the original.
- Effect annotations on return types (`with IO`, `with Diverge`).
Prose shows these too, but again: if uncertain, keep what the
original had.
- `tail` flags on calls. The prose prints `tail f(x)`; if the
edit moved a call, decide whether the new position is still in
tail position and flag accordingly.
- Doc strings (`///` lines).
- Type annotations on signatures and lambdas.
- Constructor names and arities (the prose `Cons(h, t)` must
round-trip to a `Term::Ctor` with the right `type` field).
OUTPUT
Output ONLY the new .ail.json bytes. No commentary, no markdown
fences, no preamble or postscript. The output must be valid JSON
parsable by `ail parse` (i.e. by `serde_json` against the
`ailang/v0` schema). The first byte should be `{{` and the last
non-whitespace byte should be `}}`.
SCHEMA ESSENTIALS
A full reference lives in docs/DESIGN.md (\"Data model (MVP)\"). The
shape is:
- Module: {{ \"schema\": \"ailang/v0\", \"name\": ..., \"imports\": [...],
\"defs\": [...] }}
- Defs are tagged via `kind` (\"fn\" | \"type\" | \"const\").
- Types are tagged via `k` (\"con\" | \"fn\" | \"var\" | \"forall\").
- Terms are tagged via `t` (\"lit\" | \"var\" | \"app\" | \"let\" | \"if\"
| \"do\" | \"ctor\" | \"match\" | \"lam\" | \"seq\").
- Patterns are tagged via `p` (\"var\" | \"lit\" | \"ctor\" | \"wild\").
- On `fn-type`: `param_modes` (one of \"own\"/\"borrow\" per
parameter) and `effects` are mandatory. `ret_mode` is mandatory
when the return type carries a mode.
- Constructor application uses `Term::Ctor` (`\"t\": \"ctor\"`,
fields `type` + `ctor` + `args`), NOT `Term::App`. The prose
surface elides the `type` tag (`Cons(h, t)` instead of
`(term-ctor IntList Cons h t)`) — re-introduce it.
- Base type names that appear bare in prose (`Int`, `Bool`,
`Unit`, user types like `IntList`) must be wrapped as
`{{\"k\": \"con\", \"name\": \"Int\"}}` etc. in the JSON.
ORIGINAL .ail.json
<<<ORIGINAL_AIL_JSON
{orig}
ORIGINAL_AIL_JSON
EDITED PROSE
<<<EDITED_PROSE
{edited}
EDITED_PROSE
Emit the updated .ail.json now.
",
orig = orig_ail_json,
edited = edited_prose,
)
}
fn main() -> Result<()> {
@@ -312,6 +417,17 @@ fn main() -> Result<()> {
let m = ailang_core::load_module(&path)?;
print!("{}", ailang_prose::module_to_prose(&m));
}
Cmd::MergeProse { original, edited } => {
// Iter 20d: read both inputs verbatim, compose the prompt,
// print to stdout. No schema validation here — `ail check`
// exists for that purpose, and merge-prose is intentionally
// a pure prompt-shaper.
let orig = std::fs::read_to_string(&original)
.with_context(|| format!("reading {}", original.display()))?;
let prose = std::fs::read_to_string(&edited)
.with_context(|| format!("reading {}", edited.display()))?;
print!("{}", compose_merge_prose_prompt(&orig, &prose));
}
Cmd::Describe { path, name, json, workspace } => {
if workspace {
let ws = ailang_core::load_workspace(&path)?;
@@ -1790,3 +1906,86 @@ fn build_to(
}
Ok(out_bin)
}
#[cfg(test)]
mod tests {
use super::compose_merge_prose_prompt;
/// The composed prompt must contain both inputs byte-for-byte —
/// the LLM has to see exactly what the user wrote, with no
/// re-encoding or stripping.
#[test]
fn merge_prose_prompt_contains_both_inputs_verbatim() {
let orig = "{\"schema\":\"ailang/v0\",\"name\":\"foo\",\"defs\":[]}";
let prose = "// module foo\n\nfn bar() -> Int { 42 }\n";
let out = compose_merge_prose_prompt(orig, prose);
assert!(
out.contains(orig),
"prompt missing original .ail.json verbatim; got:\n{out}"
);
assert!(
out.contains(prose),
"prompt missing edited prose verbatim; got:\n{out}"
);
}
/// The framing sections must all be present so the LLM has the
/// full task definition. Asserted by substring on a representative
/// landmark from each section.
#[test]
fn merge_prose_prompt_has_all_framing_sections() {
let out = compose_merge_prose_prompt("{}", "");
// Role statement.
assert!(
out.contains("ROLE")
&& out.contains("integrating prose edits"),
"prompt missing ROLE section"
);
// Contract about preserving load-bearing detail.
assert!(
out.contains("CONTRACT")
&& out.contains("preserve those")
&& out.contains("`own T`")
&& out.contains("`with IO`"),
"prompt missing CONTRACT section"
);
// Output spec — JSON only, no fences, no commentary.
assert!(
out.contains("OUTPUT")
&& out.contains("ONLY")
&& out.contains("no markdown"),
"prompt missing OUTPUT section"
);
// Schema-essentials reminder + DESIGN.md pointer.
assert!(
out.contains("SCHEMA ESSENTIALS")
&& out.contains("docs/DESIGN.md")
&& out.contains("`kind`")
&& out.contains("`k`")
&& out.contains("`t`")
&& out.contains("`p`")
&& out.contains("param_modes")
&& out.contains("Term::Ctor"),
"prompt missing SCHEMA ESSENTIALS section"
);
// Heredoc-style markers around both payloads.
assert!(
out.contains("<<<ORIGINAL_AIL_JSON")
&& out.contains("ORIGINAL_AIL_JSON\n")
&& out.contains("<<<EDITED_PROSE")
&& out.contains("EDITED_PROSE\n"),
"prompt missing payload markers"
);
}
/// Pure / deterministic: same inputs must yield byte-identical
/// output.
#[test]
fn merge_prose_prompt_is_deterministic() {
let orig = "{\"x\":1}";
let prose = "fn f() {}";
let a = compose_merge_prose_prompt(orig, prose);
let b = compose_merge_prose_prompt(orig, prose);
assert_eq!(a, b);
}
}
+48
View File
@@ -2372,3 +2372,51 @@ fn alloc_rc_partial_drop_at_app_let_close_does_not_leak_unmoved_fields() {
`partial_drop_<m>_<T>(p, mask)`"
);
}
/// Iter 20d: `ail merge-prose` reads two files and prints a prompt
/// that frames the prose-round-trip task for an external LLM. Smoke
/// test: writes two temp inputs, invokes the binary, asserts exit 0
/// and that the captured stdout carries the role-statement landmark
/// plus both inputs verbatim.
#[test]
fn merge_prose_prints_framed_prompt() {
let tmp = std::env::temp_dir().join(format!(
"ailang_e2e_merge_prose_{}",
std::process::id()
));
std::fs::create_dir_all(&tmp).unwrap();
let orig_path = tmp.join("orig.ail.json");
let prose_path = tmp.join("edited.prose.txt");
let orig_bytes =
b"{\"schema\":\"ailang/v0\",\"name\":\"foo\",\"imports\":[],\"defs\":[]}";
let prose_bytes = b"// module foo\n\nfn bar() -> Int { 42 }\n";
std::fs::write(&orig_path, orig_bytes).unwrap();
std::fs::write(&prose_path, prose_bytes).unwrap();
let output = Command::new(ail_bin())
.args(["merge-prose"])
.arg(&orig_path)
.arg(&prose_path)
.output()
.expect("ail merge-prose failed to run");
assert!(
output.status.success(),
"ail merge-prose exited non-zero: stderr={}",
String::from_utf8_lossy(&output.stderr)
);
let stdout = String::from_utf8(output.stdout).expect("stdout utf8");
assert!(!stdout.is_empty(), "empty stdout from merge-prose");
assert!(
stdout.contains("integrating prose edits"),
"prompt missing role-statement landmark; got:\n{stdout}"
);
// Both inputs must round-trip into the prompt verbatim.
assert!(
stdout.contains("ailang/v0"),
"prompt missing original .ail.json content"
);
assert!(
stdout.contains("fn bar() -> Int { 42 }"),
"prompt missing edited prose content"
);
}
+110
View File
@@ -9057,3 +9057,113 @@ re-rendered to incorporate the wrapping.
corpus shows it's needed.
- **`do io/print_int(x)``print(x)`** — needs type context
to be safe; deferred.
## 2026-05-08 — Iter 20d: prose round-trip mediator shipped
The closing piece of family 20: a documented prompt template and a
thin CLI helper that compose the prompt for the prose-edit cycle. No
LLM client of our own — the user pipes the prompt to whichever model
they prefer.
### The cycle
```
1. ail prose foo.ail.json > foo.prose.txt
2. $EDITOR foo.prose.txt # human edits freely
3. ail merge-prose foo.ail.json foo.prose.txt > prompt.txt
4. cat prompt.txt | <your-llm-cli> > foo.new.ail.json
5. ail check foo.new.ail.json
6. mv foo.new.ail.json foo.ail.json # if check is clean
```
Step 1 was 20a/20b. Step 3 is this iter. Steps 5/6 are existing
`ail check` + plain `mv`. The mediator is the LLM, not the
compiler — which is why the cycle is iterative (re-run step 4 with
the diagnostic pasted in if `ail check` rejects the result).
### What was built
- **`docs/PROSE_ROUNDTRIP.md`** — new top-level doc, 153 lines.
Sections: *Why* (prose is lossy, re-integration needs LLM
mediation), *The cycle* (the 7-step pipeline above), *The
prompt template* (the literal text the CLI emits, so a human
can compose by hand), *Failure modes* (markdown fences,
invalid JSON, schema-valid-but-`ail check`-fails — fix in each
case is to paste the corrective note and re-prompt), *Why no
built-in API client* (deliberate scope: AILang stays a
compiler, the user already has clients).
- **`Cmd::MergeProse { original, edited }`** in
`crates/ail/src/main.rs`. Reads both files, calls the helper,
prints to stdout. Errors via `anyhow` context on file-read
failures.
- **`fn compose_merge_prose_prompt(orig_ail_json: &str,
edited_prose: &str) -> String`** — pure helper in `main.rs`.
Both payloads insert verbatim between heredoc-style markers
(`<<<ORIGINAL_AIL_JSON ... ORIGINAL_AIL_JSON`,
`<<<EDITED_PROSE ... EDITED_PROSE`); the framing is the
role/contract/output/schema-essentials sections that match
the prose in `PROSE_ROUNDTRIP.md` byte-for-byte.
### Test surface
- **3 unit tests** in `main.rs` (`#[cfg(test)] mod tests`):
`merge_prose_prompt_contains_both_inputs_verbatim`,
`merge_prose_prompt_has_all_framing_sections`,
`merge_prose_prompt_is_deterministic`.
- **1 e2e CLI smoke test** in `tests/e2e.rs`:
`merge_prose_prints_framed_prompt` — writes two temp files,
invokes `ail merge-prose`, asserts exit 0 and that stdout
carries both inputs verbatim plus the role-statement
landmark.
### Why no built-in API client
AILang stays a compiler + tooling. Shipping an HTTP client to a
specific LLM vendor would mean an API-key story, rate limiting,
streaming semantics, error mapping, and version tracking — all
carrying zero language-level value. Users already have Claude
Code, the Anthropic SDK, the OpenAI CLI, `curl`, or any other
client they prefer; `ail merge-prose` composes the prompt and
gets out of the way. The Unix-pipe shape
(`ail merge-prose ... | client | ail check`) makes the cycle
scriptable without lock-in to one vendor. This is documented as a
design choice in `docs/PROSE_ROUNDTRIP.md` so it stays a
deliberate stance, not a gap.
### Files
- `docs/PROSE_ROUNDTRIP.md` — new (153 lines).
- `crates/ail/src/main.rs``Cmd::MergeProse` variant,
dispatch arm, `compose_merge_prose_prompt` helper, inline
`mod tests` (3 unit tests). +~180 lines.
- `crates/ail/tests/e2e.rs``merge_prose_prints_framed_prompt`
smoke test (~50 lines).
### Build/test status
- `cargo build --workspace` — green, no warnings.
- `cargo test --workspace` — all green: 3 new unit tests on the
`ail` binary, e2e count 69 → 70, prose-unit / prose-snapshot /
ailang-check / ailang-surface all unchanged-green.
### State of family 20
20a + 20b + 20d done. 20c was rolled into 20a (the `ail prose`
subcommand was a natural unit with the renderer skeleton, not a
separate iter). The family closes pending real-corpus signal on
20b's deferred polishes (let-inlining, `do io/print_int(x)`
`print(x)`, doc-wrap widow control). If those don't turn up in
practice the family stays as-shipped.
### Out of scope (explicit, not deferred)
- Built-in API client. AILang shells out to no LLM. The Unix
pipe is the integration surface.
- AST-aware merge (semantic diff between original and re-emitted
`.ail.json`). The mediator is the LLM; if a structural diff
becomes useful later, `ail diff` already exists for that
purpose post-merge.
- Schema validation in the `merge-prose` CLI itself. `ail check`
validates the merge product; `merge-prose` only composes the
prompt.
+153
View File
@@ -0,0 +1,153 @@
# Prose round-trip cycle
## Why
The prose surface (`ail prose`, Iter 20a/20b) is a deliberately
**lossy** projection of a `.ail.json` module: it strips `(con T)`
wrappings, drops redundant parentheses, infixes arithmetic, suppresses
schema rigor that the LLM can re-derive. That makes prose pleasant to
read and edit — but it also means there is no syntactic parser that
turns edited prose back into a `.ail.json`.
Re-integration of free-text edits therefore requires an **LLM
mediator**: a model that understands both the prose intent and the
load-bearing detail in the original `.ail.json`, and emits a new
`.ail.json` that respects both. This document describes the workflow,
the prompt template `ail merge-prose` composes for that mediator, and
the failure modes to watch for.
## The cycle
```
1. ail prose foo.ail.json > foo.prose.txt
2. $EDITOR foo.prose.txt # human edits freely
3. ail merge-prose foo.ail.json foo.prose.txt > prompt.txt
4. cat prompt.txt | <your-llm-cli> > foo.new.ail.json
5. ail check foo.new.ail.json
6. mv foo.new.ail.json foo.ail.json # if check is clean
```
Step 1 is the deterministic projection (Iter 20a/20b).
Step 3 is the prompt composer this iter ships.
Step 4 is the user's external LLM client — Claude Code, the
Anthropic API, OpenAI's CLI, anything that takes a prompt on stdin and
emits text on stdout. AILang ships no client of its own; see *Why no
built-in API client* below.
Steps 5 and 6 reuse existing tooling: `ail check` validates schema +
typechecks, then the user accepts the new module.
## The prompt template
`ail merge-prose <original.ail.json> <edited.prose.txt>` reads both
files and prints the following text to stdout. Nothing more, nothing
less — no fences, no JSON envelope. The shape is exactly what a human
would compose by hand if they wanted to drive the same cycle without
the CLI helper.
```
You are integrating prose edits back into an AILang module.
ROLE
Your job is to produce an updated AILang JSON-AST (.ail.json) that
reflects the human's prose edits while preserving the load-bearing
semantic detail from the original .ail.json.
CONTRACT
The prose is the source of intent: the human edited it to express
what the program should now do. The original .ail.json carries
load-bearing detail that the prose surface elides — preserve those
details unless the prose explicitly contradicts them. Specifically:
- Mode annotations on fn parameters (`own T`, `borrow T`).
These are hard contracts (memory model). The prose shows them,
but if the prose is ambiguous, default to the original.
- Effect annotations on return types (`with IO`, `with Diverge`).
Prose shows these too, but again: if uncertain, keep what the
original had.
- `tail` flags on calls. The prose prints `tail f(x)`; if the
edit moved a call, decide whether the new position is still in
tail position and flag accordingly.
- Doc strings (`///` lines).
- Type annotations on signatures and lambdas.
- Constructor names and arities (the prose `Cons(h, t)` must
round-trip to a `Term::Ctor` with the right `type` field).
OUTPUT
Output ONLY the new .ail.json bytes. No commentary, no markdown
fences, no preamble or postscript. The output must be valid JSON
parsable by `ail parse` (i.e. by `serde_json` against the
`ailang/v0` schema). The first byte should be `{` and the last
non-whitespace byte should be `}`.
SCHEMA ESSENTIALS
A full reference lives in docs/DESIGN.md ("Data model (MVP)"). The
shape is:
- Module: { "schema": "ailang/v0", "name": ..., "imports": [...],
"defs": [...] }
- Defs are tagged via `kind` ("fn" | "type" | "const").
- Types are tagged via `k` ("con" | "fn" | "var" | "forall").
- Terms are tagged via `t` ("lit" | "var" | "app" | "let" | "if"
| "do" | "ctor" | "match" | "lam" | "seq").
- Patterns are tagged via `p` ("var" | "lit" | "ctor" | "wild").
- On `fn-type`: `param_modes` (one of "own"/"borrow" per
parameter) and `effects` are mandatory. `ret_mode` is mandatory
when the return type carries a mode.
- Constructor application uses `Term::Ctor` (`"t": "ctor"`,
fields `type` + `ctor` + `args`), NOT `Term::App`. The prose
surface elides the `type` tag (`Cons(h, t)` instead of
`(term-ctor IntList Cons h t)`) — re-introduce it.
- Base type names that appear bare in prose (`Int`, `Bool`,
`Unit`, user types like `IntList`) must be wrapped as
`{"k": "con", "name": "Int"}` etc. in the JSON.
ORIGINAL .ail.json
<<<ORIGINAL_AIL_JSON
{original_ail_json_here}
ORIGINAL_AIL_JSON
EDITED PROSE
<<<EDITED_PROSE
{edited_prose_here}
EDITED_PROSE
Emit the updated .ail.json now.
```
The two payloads (original JSON, edited prose) are inserted verbatim
between the heredoc-style markers. `merge-prose` does not strip,
re-encode, or otherwise transform either input — both go in
byte-for-byte.
## Failure modes
The mediator is an LLM, so the cycle is intentionally iterative.
Common failure modes and their fix:
- **Output wrapped in markdown fences (` ```json ... ``` `).** Strip
the fences (`sed -n '/^{/,/^}/p'`) and re-run `ail check`. If the
LLM does this consistently, paste a corrective note ("emit raw
JSON, no fences") at the top of the next prompt.
- **Output contains commentary before/after the JSON.** Same
treatment. Most modern LLMs respect "OUTPUT ONLY" but not all.
- **Invalid JSON / `serde_json` rejects.** Re-run with the parse
error pasted into the prompt as a corrective note.
- **Schema-valid but `ail check` fails.** Re-run with the
diagnostic messages pasted in. Common cases: missing
`param_modes`, `Term::App` instead of `Term::Ctor`, bare type
names instead of `{"k":"con","name":"…"}`.
- **Preserved detail dropped (mode flipped, doc string lost).**
Re-run with a corrective note naming the missing detail.
The cycle converges in 13 rounds for typical edits.
## Why no built-in API client
AILang stays a compiler + tooling. Shipping an HTTP client to one
particular LLM vendor would mean an API key story, rate limiting,
streaming semantics, error mapping, version tracking — all carrying
zero language-level value. The user already has Claude Code, the
Anthropic SDK, the OpenAI CLI, `curl`, or any other client they
prefer; `ail merge-prose` composes the prompt and gets out of the
way. The Unix-pipe shape (`ail merge-prose ... | client | ail check`)
also makes the cycle scriptable without lock-in to one vendor.