From edd2558d35ba04991e755d89e311553111f0cb9f Mon Sep 17 00:00:00 2001 From: Brummel Date: Sun, 17 May 2026 23:57:46 +0200 Subject: [PATCH] =?UTF-8?q?iter=20loop-recur.3:=20codegen=20=E2=80=94=20re?= =?UTF-8?q?al=20LLVM-IR=20lowering=20+=20run-to-value=20E2E=20(milestone?= =?UTF-8?q?=20terminal)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third and terminal iteration of the standalone loop/recur milestone (plan eae73bf). Replaces the iter-1 lower_term CodegenError::Internal stub for Term::Loop/Term::Recur with real LLVM-IR lowering: loop binders as entry-block allocas (mut.3 pending_entry_allocas, reusing mut_var_allocas so the existing Term::Var load path is byte-unchanged), a fresh loop-header block, recur stores + back-edge br, a loop_frames stack saved/restored at the lambda boundary. clang -O2 mem2reg promotes the allocas to phi. Four Boss design calls implemented verbatim and journalled (alloca-not-hand-phi; mut_var_allocas reuse; emergent loop-exit via the if/match join; single block_terminated field + parallel SET site). Diff confirmed surgical: codegen/lib.rs 3 hunks (field + init + stub->2-arms), lambda.rs 2 hunks (save+restore); zero edits to any existing block_terminated SET/READ site, tail-app lowering, or verify_tail_positions (Boss call 4, the spec-pinned invariant). Three new .ail fixtures: sum_to->55, deep-n 1e6->500000500000 (clause-2 correctness made executable), infinite-loop build-only. Codegen-only: no schema/typecheck change; hash pins + drift trio stay green untouched. One DONE_WITH_CONCERNS (T3): the plan's `cargo test ... tail` filter resolved to no tests; ran via real names + the full 619/0 which subsumes it (feedback_plan_pseudo_vs_reality class, no behaviour change). Boss systemic fix folded in: planner SKILL.md Step-5 gains item 8 (verification-command filter strings must resolve) — the second planner-meta-gap this milestone surfaced. cargo test --workspace 616 -> 619 / 0 red (Boss-reran independently); the 3 loop/recur e2e explicitly green. All three components shipped: the loop/recur milestone is structurally complete. Milestone-close audit + fieldtest is the next step. --- .../2026-05-17-iter-loop-recur.3.json | 13 ++ crates/ail/tests/e2e.rs | 37 ++++ crates/ailang-codegen/src/lambda.rs | 8 + crates/ailang-codegen/src/lib.rs | 135 ++++++++++++- docs/journals/2026-05-17-iter-loop-recur.3.md | 188 ++++++++++++++++++ docs/journals/INDEX.md | 1 + examples/loop_forever_build.ail | 10 + examples/loop_sum_to_deep.ail | 14 ++ examples/loop_sum_to_run.ail | 14 ++ skills/planner/SKILL.md | 17 ++ 10 files changed, 434 insertions(+), 3 deletions(-) create mode 100644 bench/orchestrator-stats/2026-05-17-iter-loop-recur.3.json create mode 100644 docs/journals/2026-05-17-iter-loop-recur.3.md create mode 100644 examples/loop_forever_build.ail create mode 100644 examples/loop_sum_to_deep.ail create mode 100644 examples/loop_sum_to_run.ail diff --git a/bench/orchestrator-stats/2026-05-17-iter-loop-recur.3.json b/bench/orchestrator-stats/2026-05-17-iter-loop-recur.3.json new file mode 100644 index 0000000..ea5d282 --- /dev/null +++ b/bench/orchestrator-stats/2026-05-17-iter-loop-recur.3.json @@ -0,0 +1,13 @@ +{ + "iter_id": "loop-recur.3", + "date": "2026-05-17", + "mode": "standard", + "outcome": "DONE", + "tasks_total": 3, + "tasks_completed": 3, + "reloops_per_task": { "1": 0, "2": 0, "3": 0 }, + "review_loops_spec": 0, + "review_loops_quality": 0, + "blocked_reason": null, + "notes": "Terminal iteration of the standalone-loop/recur milestone (codegen Component 5 + run-to-value/deep-n/infinite-build E2E). Zero review re-loops across all 3 tasks; all four Boss design calls implemented verbatim (alloca-not-hand-phi; mut_var_allocas reuse; emergent loop-exit via the if/match join; single block_terminated field + parallel SET site) — each verified against actual read source before implementation, none reopened. T1 RED observed verbatim (iter-1 lower_term stub via direct ail build); GREEN sum_to->55. T1 plan's literal codegen arms compiled as-written, no plan-vs-reality substitution needed (pre-grounding confirmed mut_var_allocas:(String,Type) + lower_term->(String,String) matched). One DONE_WITH_CONCERNS on T3: the plan's literal `cargo test --workspace tail` filter substring matches no test name (real tail-app guards live under ailang-check/-prose lib + e2e iter14e_*musttail); ran the gate via real test names + the authoritative full-suite 619/0 which subsumes them — recurring feedback_plan_pseudo_vs_reality class (verification-command filter doesn't resolve), no behaviour change, intent (tail-app/verify_tail_positions byte-unchanged) preserved. Codegen-only iter: no schema/AST/serde/typecheck change; hash pins (loop_recur+iter13a) + drift trio + carve_out_inventory green untouched (empirical proof none scans the codegen region). cargo test --workspace 616 -> 619 (= baseline + exactly the 3 new e2e tests), zero FAILED/error. Milestone structurally closed; audit/fieldtest is the Boss post-iter call." +} diff --git a/crates/ail/tests/e2e.rs b/crates/ail/tests/e2e.rs index 9950cb6..1c4e425 100644 --- a/crates/ail/tests/e2e.rs +++ b/crates/ail/tests/e2e.rs @@ -82,6 +82,43 @@ fn sum_1_to_10_is_55() { assert_eq!(stdout.trim(), "55"); } +#[test] +fn loop_recur_sum_to_runs_to_value() { + let stdout = build_and_run("loop_sum_to_run.ail"); + assert_eq!(stdout.trim(), "55"); +} + +#[test] +fn loop_recur_deep_n_safe_by_construction() { + // 1..1_000_000 summed via the loop back-edge (no stack growth); + // a hand-written non-tail self-recursion to this depth would + // overflow. sum 1..1_000_000 = 500000500000 (fits i64). + let stdout = build_and_run("loop_sum_to_deep.ail"); + assert_eq!(stdout.trim(), "500000500000"); +} + +#[test] +fn loop_recur_infinite_loop_compiles() { + let manifest_dir = env!("CARGO_MANIFEST_DIR"); + let workspace = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); + let src = workspace.join("examples").join("loop_forever_build.ail"); + let tmp = std::env::temp_dir().join(format!( + "ailang_e2e_loopforever_{}", + std::process::id() + )); + std::fs::create_dir_all(&tmp).unwrap(); + let out = tmp.join("bin"); + let status = Command::new(ail_bin()) + .args(["build", src.to_str().unwrap(), "-o"]) + .arg(&out) + .status() + .expect("ail build failed to run"); + assert!( + status.success(), + "ail build must succeed on an infinite loop (typechecks AND compiles, no termination claim)" + ); +} + /// Guards block tracking in codegen: max3 has nested `if`s, and wrong /// phi block tracking would produce a wrong result here. #[test] diff --git a/crates/ailang-codegen/src/lambda.rs b/crates/ailang-codegen/src/lambda.rs index 25f9860..4570285 100644 --- a/crates/ailang-codegen/src/lambda.rs +++ b/crates/ailang-codegen/src/lambda.rs @@ -142,6 +142,13 @@ impl<'a> Emitter<'a> { // the three fields so the thunk emits its own hoisted- // alloca block at the thunk's entry, not the outer fn's. let saved_mut_allocas = std::mem::take(&mut self.mut_var_allocas); + // loop-recur iter 3: a lambda thunk is its own fn frame — + // a `recur` cannot target a loop enclosing the lambda + // (iter-2 typecheck already rejects it; codegen resets so + // the thunk's own loops work and the outer frames cannot + // leak in). Save + reset (mem::take empties the Vec), + // restored below alongside the mut.3 triple. + let saved_loop_frames = std::mem::take(&mut self.loop_frames); let saved_pending_allocas = std::mem::take(&mut self.pending_entry_allocas); let saved_entry_marker = self.entry_block_end_marker.take(); // Iter 18d.4 fix: a lambda thunk is its own fn frame for @@ -256,6 +263,7 @@ impl<'a> Emitter<'a> { self.current_param_modes = saved_param_modes; // Iter mut.3: restore outer fn's mut-var bookkeeping. self.mut_var_allocas = saved_mut_allocas; + self.loop_frames = saved_loop_frames; self.pending_entry_allocas = saved_pending_allocas; self.entry_block_end_marker = saved_entry_marker; diff --git a/crates/ailang-codegen/src/lib.rs b/crates/ailang-codegen/src/lib.rs index 2f73a92..4d847af 100644 --- a/crates/ailang-codegen/src/lib.rs +++ b/crates/ailang-codegen/src/lib.rs @@ -723,6 +723,16 @@ struct Emitter<'a> { /// walks `mut_scope_stack` innermost-first before falling /// through to locals/globals). mut_var_allocas: BTreeMap, + /// loop-recur iter 3: stack of enclosing `Term::Loop` codegen + /// frames. Each frame is `(header_block_label, binder_slots)` + /// where `binder_slots` is `(binder_name, alloca_ssa, llvm_ty)` + /// in declaration order. Pushed on `Term::Loop` body entry, + /// popped on exit; `Term::Recur` reads `.last()` for its target + /// header + the binder allocas to store into. Saved/reset/ + /// restored at the lambda-lowering boundary exactly as the + /// mut.3 `mut_var_allocas` triple (a `recur` cannot cross a + /// lambda boundary). + loop_frames: Vec<(String, Vec<(String, String, String)>)>, /// Iter mut.3: side buffer for `alloca` instructions emitted /// during body lowering but hoisted to the fn's entry block. /// Flushed once into `self.body` at `entry_block_end_marker` @@ -853,6 +863,7 @@ impl<'a> Emitter<'a> { moved_slots: BTreeMap::new(), current_param_modes: BTreeMap::new(), mut_var_allocas: BTreeMap::new(), + loop_frames: Vec::new(), pending_entry_allocas: String::new(), entry_block_end_marker: None, } @@ -1842,9 +1853,127 @@ impl<'a> Emitter<'a> { // iter stubs the dispatch so the workspace compiles; no // loop/recur program is codegen'd in iter 1. Mirrors // mut.1's lower_term stub. - Term::Loop { .. } | Term::Recur { .. } => Err(CodegenError::Internal( - "Term::Loop/Term::Recur lowering lands in loop-recur iter 3".into(), - )), + Term::Loop { binders, body } => { + // loop-recur iter 3: loop binders are loop-carried + // values lowered as entry-block allocas (the mut.3 + // `pending_entry_allocas` mechanism) registered in + // `mut_var_allocas` so the existing `Term::Var` load + // path resolves them with zero new Var code + // (representation-sharing; Boss calls 1+2). The loop + // header is a fresh block reached by an + // unconditional `br` from the pre-header (current) + // block; `recur` stores new values into the binder + // allocas and back-edges here. `clang -O2` mem2reg + // promotes the allocas to the phi nodes the spec's + // implementation-shape describes. The loop's value + // is the body's value on the non-`recur` exit, + // materialised by the existing `if`/`match` join + // once `recur` sets `block_terminated` (Boss call 3). + let id = self.fresh_id(); + let header = format!("loop.header.{id}"); + let mut saved: Vec<(String, Option<(String, Type)>)> = Vec::new(); + let mut frame_slots: Vec<(String, String, String)> = Vec::new(); + for b in binders { + let alloca_name = + format!("%loopv_{}_{}", b.name, self.fresh_id()); + let llvm_ty = llvm_type(&b.ty)?; + self.pending_entry_allocas + .push_str(&format!(" {alloca_name} = alloca {llvm_ty}\n")); + let (init_ssa, init_ty) = self.lower_term(&b.init)?; + if init_ty != llvm_ty { + return Err(CodegenError::Internal(format!( + "Term::Loop binder `{}`: init LLVM type {init_ty} != declared {llvm_ty}", + b.name + ))); + } + self.body.push_str(&format!( + " store {llvm_ty} {init_ssa}, ptr {alloca_name}\n" + )); + let prior = self + .mut_var_allocas + .insert(b.name.clone(), (alloca_name.clone(), b.ty.clone())); + saved.push((b.name.clone(), prior)); + frame_slots.push((b.name.clone(), alloca_name, llvm_ty)); + } + self.body.push_str(&format!(" br label %{header}\n")); + self.loop_frames.push((header.clone(), frame_slots)); + self.start_block(&header); + let body_result = self.lower_term(body); + self.loop_frames.pop(); + // Restore prior bindings unconditionally (mirrors the + // mut.3 `Term::Mut` arm — defensive on the `?` path). + for (name, prior) in saved.into_iter().rev() { + match prior { + Some(p) => { + self.mut_var_allocas.insert(name, p); + } + None => { + self.mut_var_allocas.remove(&name); + } + } + } + body_result + } + Term::Recur { args } => { + // loop-recur iter 3: re-enter the innermost + // enclosing loop. Typecheck (iter 2) pinned arity == + // binder count and per-arg type == binder type, so a + // miss/mismatch here is an internal error (the + // `Term::Assign`-arm precedent). ALL args are lowered + // to SSA values BEFORE any store, so a recur arg that + // reads a binder sees its OLD value (recur is a + // simultaneous positional rebind, e.g. + // `(recur (+ acc i) (+ i 1))`). The back-edge `br` is + // a block terminator: set `block_terminated` at + // recur's OWN emit site (the parallel setter — Boss + // call 4; tail-app's SET sites byte-unchanged) so the + // enclosing `if`/`match` join excludes this block + // exactly like a tail-app block. recur never falls + // through; its value is never used — return the + // canonical Unit/dead SSA (`("0","i8")`, the + // `Term::Assign` / both-if-branches-terminated + // precedent). + let (header, slots) = self + .loop_frames + .last() + .cloned() + .ok_or_else(|| { + CodegenError::Internal( + "Term::Recur reached codegen with no enclosing loop frame \ + — typecheck should have rejected this earlier" + .into(), + ) + })?; + if args.len() != slots.len() { + return Err(CodegenError::Internal(format!( + "Term::Recur arg count {} != enclosing loop binder count {} \ + — typecheck should have rejected this earlier", + args.len(), + slots.len() + ))); + } + let mut lowered: Vec<(String, String)> = + Vec::with_capacity(args.len()); + for a in args { + lowered.push(self.lower_term(a)?); + } + for ((arg_ssa, arg_ty), (_name, alloca_name, llvm_ty)) in + lowered.into_iter().zip(slots.iter()) + { + if &arg_ty != llvm_ty { + return Err(CodegenError::Internal(format!( + "Term::Recur arg LLVM type {arg_ty} != binder type {llvm_ty} \ + — typecheck should have rejected this earlier" + ))); + } + self.body.push_str(&format!( + " store {llvm_ty} {arg_ssa}, ptr {alloca_name}\n" + )); + } + self.body.push_str(&format!(" br label %{header}\n")); + self.block_terminated = true; + Ok(("0".into(), "i8".into())) + } } } diff --git a/docs/journals/2026-05-17-iter-loop-recur.3.md b/docs/journals/2026-05-17-iter-loop-recur.3.md new file mode 100644 index 0000000..c4e0371 --- /dev/null +++ b/docs/journals/2026-05-17-iter-loop-recur.3.md @@ -0,0 +1,188 @@ +# iter loop-recur.3 — Codegen (Component 5) + run-to-value E2E + +**Date:** 2026-05-17 +**Started from:** eae73bf320440f6b031715e72c1f20b0ac86e024 +**Status:** DONE +**Tasks completed:** 3 of 3 + +## Summary + +The **terminal iteration** of the standalone-`loop`/`recur` milestone: +the iter-1 `lower_term` `CodegenError::Internal` stub for +`Term::Loop`/`Term::Recur` is replaced with real LLVM-IR lowering, so +a `sum_to`-class loop program builds and runs to the correct printed +value, a deep-`n` variant is safe by construction, and an infinite +loop compiles. Loop binders are loop-carried values lowered as +entry-block allocas (the mut.3 `pending_entry_allocas` mechanism) +registered in the existing `mut_var_allocas` map, so the existing +`Term::Var` load path resolves them with zero new Var code +(representation-sharing); a fresh `loop.header.` block is reached +by an unconditional `br` from the pre-header, `recur` lowers each arg +to SSA *before* any store (simultaneous positional rebind), stores +into the binder allocas, back-edges `br` to the header, and sets the +single existing `block_terminated` field at its OWN new emit site so +the loop's exit value is the emergent product of the existing +`if`/`match` join (no separate loop-result phi/exit-block). A new +`loop_frames` codegen stack lets `recur` find its target; it is +saved/reset/restored at the single lambda-lowering boundary exactly +as mut.3's `mut_var_allocas` triple. `clang -O2` mem2reg promotes +the binder allocas to the phi nodes the spec's implementation-shape +describes. This is a **codegen-only iteration**: no schema/AST/serde +change, no typecheck change; hash pins (`loop_recur` + iter13a) and +the drift trio + `carve_out_inventory` stay green untouched +(confirmed, not modified — empirical proof none scans the codegen +region). `cargo test --workspace` 616 → 619 / 0 red (the 3 new e2e +tests). After this iter the milestone is structurally CLOSED; +milestone-close (audit/fieldtest) is the Boss's post-iter call. + +## Per-task notes + +- iter loop-recur.3.1: positive `sum_to` run-to-value E2E (RED → + GREEN). RED observed verbatim: `ail build` on `loop_sum_to_run.ail` + fails with the iter-1 stub `internal: Term::Loop/Term::Recur + lowering lands in loop-recur iter 3` (parse + iter-1/iter-2 + typecheck pass; only codegen stubbed — confirmed via direct `ail + build`). GREEN: `loop_frames` field + init, lambda-boundary + save/reset/restore, the two real `Term::Loop`/`Term::Recur` arms + replacing the single stub arm. `loop_sum_to_run.ail` → `55`; + `cargo build -p ailang-codegen` Finished; full e2e suite 88/0. + Plan's literal arms compiled as-written — no plan-vs-reality + substitution needed (pre-grounding confirmed `mut_var_allocas: + (String, Type)` and `lower_term -> (String /*ssa*/, String + /*llvm_ty*/)` matched the plan's assumptions exactly). +- iter loop-recur.3.2: deep-`n` safe-by-construction + + infinite-loop-compiles (test+fixture only; the codegen mechanism + shipped in 3.1, so no separate RED). `loop_sum_to_deep.ail` runs + 1e6 iterations via the back-edge → `500000500000` with no stack + growth (the spec clause-2 correctness claim made executable — + mem2reg promoted the binder allocas so the loop is an iterative + back-edge, not a stack-growing recursion). `loop_forever_build.ail` + (`spin`'s loop has no non-`recur` exit) compiles `ail build` exit + 0 — `recur` sets `block_terminated`, propagating through + `emit_fn`'s `if !self.block_terminated` fall-through guard so + `spin` emits no `ret` and is a well-formed never-returning fn; the + binary is never executed (spec "typechecks AND compiles, no + termination claim"). Full e2e suite 90/0. +- iter loop-recur.3.3: regression gates — pure verification, ZERO + source changes (confirmed `git diff --name-only HEAD` = only the + 3 T1/T2 source files). tail-app/tail-do/`verify_tail_positions` + non-regression green (the parallel `block_terminated` SET site + touched no existing SET/READ site — Boss call 4); `loop_recur` + + iter13a hash pins + drift trio + `carve_out_inventory` green + untouched (codegen-only iter, no schema change); full workspace + 619 passed = exactly the loop-recur.2 baseline (616) + the 3 new + e2e tests, zero FAILED/error; `round_trip` green (the 3 new `.ail` + fixtures auto-discovered, parse→print→parse idempotent — Roundtrip + Invariant holds for runnable loop/recur programs). + +## Boss design calls (mirrored from the plan header at iter close) + +All four were settled in the plan header on +architectural-consistency / spec-pinned-invariant grounds (NOT +effort); each verified against actual read source before +implementation; none reopened, re-derived, or "improved" in any +phase: + +1. **Loop binders → entry-block alloca + `mut_var_allocas` reuse, + NOT hand-emitted `phi`.** The spec's "phi" wording lives only in + the explicitly-*secondary* implementation-shape subsection. The + established in-repo loop-carried-value mechanism is + alloca-in-entry-block (mut.3 `pending_entry_allocas`) + `clang + -O2` mem2reg, which produces the identical optimized binary. The + linear-emit codegen architecture structurally cannot hand-emit a + header `phi` whose back-edge predecessors are discovered *during* + body lowering without a string-splice hack with no precedent (the + `if` arm sidesteps this by opening its join block *last* — a loop + header cannot be opened last). Rationale = architectural + consistency + absence of a linear-emit `phi` precedent + identical + -O2 output. Implemented verbatim; `loop_sum_to_run.ail`→55 and + `loop_sum_to_deep.ail`→500000500000 are the operational proof + mem2reg delivers the loop-carried-SSA intent. +2. **Binders ride the existing `mut_var_allocas` map + the existing + `Term::Var` load path (representation-sharing).** A loop binder's + codegen representation — a named alloca slot, loaded on each + `Term::Var` use, stored on `recur` — is structurally identical to + a mut-var's. The `Term::Loop` arm inserts each binder into + `mut_var_allocas` (value type `(String, Type)`, identical to the + `Term::Mut` precedent); the existing `lib.rs` `Term::Var` arm + resolves them via its pre-existing `mut_var_allocas.get(name)` → + `load` path **byte-unchanged** (zero edits to Var-resolution; no + parallel map). Verified against real source before implementing. + mut.3's shipped+E2E-green mut-var read path is the proof the load + works for loop binders. +3. **The loop's exit value is the emergent product of the existing + `if`/`match` join once `recur` sets `block_terminated` — no + separate loop-result phi/exit-block.** The body is typically + `(if c (recur …))`; the existing + one-branch-terminated logic at `lib.rs:1606-1613` already drops + the terminated (recur) branch and returns the non-recur branch's + `(ssa,ty)` directly — *iff* `recur` sets `block_terminated`. The + `Term::Loop` arm therefore returns `lower_term(body)`'s `(ssa,ty)` + directly; no separate loop-result phi was built. An infinite loop + (body = bare `recur`) compiles because `recur` sets + `block_terminated`, propagating through the `emit_fn` + `if !self.block_terminated` fall-through guard exactly as a + tail-app-terminated function. +4. **Reuse the single `block_terminated` field; `recur` sets it at + its OWN new emit site (a parallel SET call-site), zero edits to + any existing SET or READ site.** A back-edge `br` IS a block + terminator with exactly `block_terminated`'s semantics. `recur`'s + own `self.block_terminated = true;` is a NEW call-site parallel + to (not replacing) tail-app's SET at `lib.rs:2279`. The diff + confirms tail-app's SET and every READ site + (`:1124/1581/1594/1600-1613/...`) are byte-identical; the + tail-app/`verify_tail_positions` non-regression gate (T3 Step 1) + is the operational proof. + +## Concerns + +- DONE_WITH_CONCERNS (iter loop-recur.3.3): the plan's literal Task-3 + Step-1 command `cargo test --workspace tail` filters on the + test-name substring `tail`, which matches **no** test in the + workspace (the real tail-app guards are + `ailang-check::lib::tail_call_in_{non_,}tail_position_is_*`, + `ailang-prose::lib` `*_tail_*`, e2e + `iter14e_print_list_recursion_emits_musttail`). Ran the gate via + the real test names instead (2 + 3 + 1, all green); the + authoritative non-regression gate is Step 3's full-workspace + 619/0 which subsumes them. Recurring `feedback_plan_pseudo_vs_reality` + class (a literal verification command whose filter doesn't resolve + to its evident target); resolved by preserving the gate's evident + intent (tail-app/`verify_tail_positions` byte-unchanged proven), + no behaviour change. Observation, not a correctness risk — the + full-suite count `619 = 616 + 3` is independent confirmation + nothing regressed. + +## Known debt + +- None. The iteration's invariants (loop/recur lowers to real IR; + run-to-value 55; deep-`n` 500000500000 safe-by-construction; + infinite loop compiles; tail-app + hash pins + drift trio + byte-unchanged; round-trip on the 3 new fixtures) are fully pinned + by the 3 new e2e tests + the unchanged regression suite. + +## Files touched + +- Codegen: `crates/ailang-codegen/src/lib.rs` (`loop_frames` field + + init; the two real `Term::Loop`/`Term::Recur` arms replacing the + iter-1 single stub arm), `crates/ailang-codegen/src/lambda.rs` + (lambda-boundary `loop_frames` save/reset + restore, alongside the + mut.3 triple) +- Tests: `crates/ail/tests/e2e.rs` (3 new test fns: 2 build+run, 1 + build-only) +- New fixtures: `examples/loop_sum_to_run.ail`, + `examples/loop_sum_to_deep.ail`, `examples/loop_forever_build.ail` +- No edit / byte-frozen (confirmed, not modified): every existing + `block_terminated` SET/READ site + tail-app/tail-do lowering + + `verify_tail_positions`; the iter-2 typecheck (`synth` + Loop/Recur arms, `verify_loop_body`, `loop_stack`, the four + `Recur*` variants); the iter-1 `synth_with_extras` + Loop→body-type / Recur→Unit pass-through; `hash_pin.rs` + (`loop_recur` + iter13a pins) + the drift trio + + `carve_out_inventory.rs` (codegen-only iter; the 3 new `.ail` + fixtures are round-trip-auto-covered, NOT carve-outs — inventory + stays 8) + +## Stats + +bench/orchestrator-stats/2026-05-17-iter-loop-recur.3.json diff --git a/docs/journals/INDEX.md b/docs/journals/INDEX.md index a08fd81..081e970 100644 --- a/docs/journals/INDEX.md +++ b/docs/journals/INDEX.md @@ -83,3 +83,4 @@ - 2026-05-16 — iter effect-doc-honesty: standalone documentation-honesty tidy (split out from the retired effect-op-arg-modes bundle — user rejected bundling a real DESIGN.md fix with speculative build-ahead infra). Corrected three false effect-system claims the recon surfaced: DESIGN.md Decision 3 "row-polymorphic (`![IO | r]`)" (no `EffectRow`/row var exists — effect sets are flat closed sets unified by set-equality) + "`IO` and `Diverge` … are wired up" (`Diverge` is 100% vapour: zero code in any crate) → reconciled to IO-only/Diverge-reserved-unimplemented (modelled on Decision 4's reserved-refinements precedent); DESIGN.md §"What is not supported" "IO and Diverge ops" bullet; `ast.rs` `Term::Do` doc-comment "resolved against the effect-handler table at link time" → the real mechanism (typecheck `Env::effect_ops` lookup + `lower_effect_op` literal codegen match, no handler table). Satellite lockstep: `form_a.md:226` + rule-3, `main.rs:289` merge-prose CONTRACT example. Guarded by a new 4-test doc-presence pin `crates/ailang-core/tests/effect_doc_honesty_pin.rs` (fiction-absent + corrected-anchor-present, single-line substrings — wrap-robust). No language/checker/codegen change; `ail check`/`run` byte-unchanged by construction. `cargo test --workspace` 600 → 604; drift/coverage trio (`design_schema_drift`/`spec_drift`/`schema_coverage`) green, empirically confirming none scans the effect-prose region. One concern: the plan's Task-2 replacement body soft-wrapped a phrase the single-line pin asserts; orchestrator resolved correctly (exact wording + exact pin preserved, wrap column moved) — recurring-class planner gap, fixed forward by a Step-5 self-review tightening → 2026-05-16-iter-effect-doc-honesty.md - 2026-05-17 — iter loop-recur.1: standalone-`loop`/`recur` milestone (1 of 3) — the strictly-additive AST-node foundation. `Term::Loop { binders: Vec, body }` / `Term::Recur { args }` / `struct LoopBinder` (mirrors `MutVar`'s `(name,type,init)` triple; `binders` no `skip_serializing_if`) added end-to-end across all six crates' no-`_`-wildcard exhaustive `Term` matches (~30 sites incl. one recon-undercount integration-test pin): Form-A `parse_loop`/`parse_recur` (binder/body boundary disambiguated by an `is_term_head_kw` guard — the plan's flagged sole parser judgement) + print + `form_a.md` grammar/notes + prose render/free-var/subst lockstep + canonical-JSON serde/round-trip + DESIGN.md §Data-model `"t":"loop"`/`"t":"recur"` blocks + design_schema_drift/spec_drift/schema_coverage anchors + a new `loop_recur` hash pin (additivity proven: iter13a + new pin both green, pre-existing canonical-JSON bytes byte-stable) + `examples/loop_sum_to.ail` round-trip fixture. NO typecheck semantics + NO real codegen by design: the two semantic dispatch points stubbed `synth`→`CheckError::Internal` / `lower_term`→`CodegenError::Internal` exactly as mut.1; pure analysis walkers (escape/lambda/`synth_with_extras`) get real structural pass-through. Two binding Boss design calls (mirrored into the journal): (1) `verify_tail_positions` "byte-unchanged" = tail-app *role* unchanged not source-frozen — it is an exhaustive no-wildcard match so two additive descent-only arms are mandatory; acceptance evidence is the tail-app non-regression test (all `tail`-filtered tests byte-identical), mut.1 set the precedent; (2) codegen IS iter-1 scope as stubs/pass-throughs (no real loop-header/phi/back-edge — that is iter 3). Three plan-pseudo-vs-reality substitutions, all intent-preserving, recurring `feedback_specs_need_concrete_code`/`feedback_plan_pseudo_vs_reality` class: serde `Type::Con` literal `{"t":"con",…,"args":[]}`→real `{"k":"con","name":"Int"}`; bare `Int`→`(con Int)` in binder triples (bare `Int` parses as `Type::Var`); `ailang_core::ast::LoopBinder`→unqualified inside ailang-core. Boss forward-fix folded in: Concern 2 surfaced that the committed specs themselves (`2026-05-17-loop-recur.md` headline + `bad_recur`, `2026-05-17-llm-surface-discipline.md` §5) wrote bare `Int` — corrected all three snippets to `(con Int)` (doc-honesty, same class as the mono.rs header; no design/schema/assumption change, no re-brainstorm). `cargo test --workspace` 600→608 / 0 red (Boss-reran independently). Component 4 (typecheck) = iter 2; Component 5 (codegen) + positive/deep-`n` E2E = iter 3 → 2026-05-17-iter-loop-recur.1.md - 2026-05-17 — iter loop-recur.2: standalone-`loop`/`recur` milestone (2 of 3) — Component 4 typecheck semantics. The iter-1 `synth` `CheckError::Internal` stub for `Term::Loop`/`Term::Recur` is replaced with real binder typing (each init synthed in outer scope + already-declared binder names of THIS loop via ordinary `locals` save/restore mirroring `Term::Let` verbatim; loop's static type = body type) + positional `recur` arity/per-arg-type checking via a new `loop_stack: &mut Vec>` frame threaded exactly as mut.2's `mut_scope_stack` (every recursive + cross-module + test synth caller, compile-swept). New private `verify_loop_body(t,in_loop_tail)` pass — a SIBLING of the byte-frozen `verify_tail_positions` (0 deletions, tail-app role untouched), invoked in `check_fn` after it so synth-raised codes take precedence by pass ordering. Four `Recur*` `CheckError` variants (`RecurOutsideLoop`, `RecurArityMismatch{expected,got}`, `RecurTypeMismatch{position,expected,got}`, `RecurNotInTailPosition`) + `code()` + 2 `ctx()` arms, bracket-`[code]`-free Display (F2); four negative `.ail.json` fixtures fire their codes point-exactly (`assert_eq!`, non-vacuous) via a new `loop_recur_typecheck_pin.rs` (7 tests) + a ct1 F2 human-mode sibling; `carve_out_inventory` 13→17 with the pre-existing mut.4-tidy "Twelve"-vs-13 stale-header drift corrected in passing. iter-1's `loop_sum_to.ail` round-trip fixture now ALSO typechecks clean (one fixture, two properties — no near-duplicate); `loop_forever.ail` (loop whose only path is `recur`) typechecks clean, asserting the spec "no termination claim is made or enforced". Three binding Boss design calls (mirrored into the journal): (1) `RecurTypeMismatch` is an Assign-style `subst.apply`-both-then-structural-`!=` pre-check, NOT a `unify`-propagate (a propagate would surface generic `type-mismatch` and fail the point-exact-code acceptance); (2) `loop_stack` element type is positional `Vec` NOT name-keyed `IndexMap` (recur rebinds by position; binder names live in `locals`); (3) diagnostic-code precedence is by pass ordering (synth before verify_loop_body), no explicit logic. NO codegen (iter-1 `lower_term` `CodegenError::Internal` stub stays — iter 3); NO `Diverge`/`verify_structural_recursion`/guardedness (spec deliberate boundary). Two DONE_WITH_CONCERNS, both Task 2, both journalled: (a) a plan-ordering defect — Task 2's `cargo build` 0-errors gate is unsatisfiable while `check_fn` (deferred by the plan to Task 4) is an unthreaded caller; orchestrator pulled the byte-identical declaration+threading forward to Task 2, left only the `verify_loop_body` invocation in Task 4 (no semantic change, only sequencing); (b) recon-undercount of cross-module synth callers (`builtins.rs`×2, `lift.rs:746`, `mono.rs:720`/`:1361`), the recurring mut.2-class, resolved via the plan's own designated compile-sweep oracle. Boss systemic fix folded into this commit: planner SKILL.md Step-5 self-review gains item 7 (compile-gate vs. deferred-caller ordering) so the Concern-(a) class is scrubbed at plan time, not discovered at execution. `cargo test --workspace` 608→616 / 0 red (Boss-reran independently); `verify_tail_positions`/tail-app byte-frozen confirmed by diff-hunk inspection. Component 5 (codegen loop-header/phi/back-edge) + the positive RUN-to-value / deep-`n` E2E = iteration 3 → 2026-05-17-iter-loop-recur.2.md +- 2026-05-17 — iter loop-recur.3: standalone-`loop`/`recur` milestone (3 of 3, TERMINAL) — Component 5 codegen + run-to-value E2E. The iter-1 `lower_term` `CodegenError::Internal` stub for `Term::Loop`/`Term::Recur` is replaced with real LLVM-IR lowering: loop binders are loop-carried values lowered as entry-block allocas (the mut.3 `pending_entry_allocas` mechanism) registered in the EXISTING `mut_var_allocas` map so the EXISTING `Term::Var` load path resolves them with zero new Var code (representation-sharing only — Var-lowering byte-unchanged); a fresh `loop.header.` block reached by an unconditional `br` from the pre-header; `recur` lowers ALL args to SSA BEFORE any store (simultaneous positional rebind), stores into the binder allocas, back-edges `br` to the header, and sets the SINGLE existing `block_terminated` field at its OWN new emit site (a parallel SET call-site) so the loop's exit value is the emergent product of the existing `if`/`match` join — no separate loop-result phi/exit-block. A new `loop_frames` codegen stack lets `recur` find its target header+slots; saved/reset/restored at the single lambda-lowering boundary exactly as mut.3's `mut_var_allocas` triple. `clang -O2` mem2reg promotes the binder allocas to the phi nodes the spec's secondary implementation-shape describes. Four binding Boss design calls (mirrored into the journal), all on architectural-consistency / spec-pinned-invariant grounds (NOT effort): (1) alloca+mem2reg NOT hand-emitted phi (the linear-emit architecture has no phi-with-body-discovered-back-edge-predecessor precedent; the `if` arm sidesteps by opening its join last — a loop header cannot be opened last; mem2reg yields identical optimized output; spec's "phi" is the explicitly-secondary impl-shape the spec subordinates to the planner's exact-bytes authority); (2) reuse `mut_var_allocas` + the existing `Term::Var` load path (byte-unchanged; representation-sharing, the iter-2 AST/typecheck binder distinction is post-typecheck-irrelevant to codegen); (3) emergent loop-exit via the existing if/match join once recur terminates its block, no separate result phi; (4) reuse the single `block_terminated` field, recur sets it at its own emit site, ZERO edits to any existing SET/READ site (a second field would force editing every READ site — the opposite of the spec-pinned "tail-app's use byte-unchanged"). Diff confirmed surgical by hunk-header inspection: codegen/lib.rs = exactly 3 hunks (field + init + the stub→2-arms replacement), codegen/lambda.rs = exactly 2 hunks (save + restore); NO hunk at any existing `block_terminated` SET/READ site, at tail-app/tail-do lowering, or at `verify_tail_positions`. Three new `.ail` fixtures: `loop_sum_to_run.ail`→`55` (run-to-value), `loop_sum_to_deep.ail`→`500000500000` (1e6 iterations via the back-edge, no stack growth — the spec clause-2 correctness claim made executable), `loop_forever_build.ail` (infinite loop, no non-recur exit, `ail build` exit 0 — "typechecks AND compiles, no termination claim"; never executed). Codegen-ONLY iteration: no schema/AST/serde/typecheck change; hash pins (`loop_recur` + iter13a) + drift trio + `carve_out_inventory` confirmed green UNTOUCHED (not modified); the 3 new `.ail` fixtures are round-trip-auto-covered, NOT carve-outs. One DONE_WITH_CONCERNS (T3): the plan's literal `cargo test --workspace tail` filter resolves to ZERO tests (real guards are `*_tail_position_*`/`*musttail*`); orchestrator ran the gate via real names + the authoritative full-suite 619/0 which subsumes it — recurring `feedback_plan_pseudo_vs_reality` class, no behaviour change. Boss systemic fix folded into this commit: planner SKILL.md Step-5 gains item 8 (verification-command filter strings must resolve to ≥1 named test, or use the unfiltered suite + a count assertion) — the SECOND planner-meta-gap this milestone surfaced (iter-2 added item 7). `cargo test --workspace` 616→619 / 0 red (Boss-reran independently); the 3 loop/recur e2e explicitly green (55 / 500000500000 / infinite-compiles). **All three components shipped — the loop/recur milestone is structurally complete; milestone-close (mandatory `audit`, then `fieldtest` since loop/recur is user-visible Form-A surface) is the Boss's next pipeline step.** → 2026-05-17-iter-loop-recur.3.md diff --git a/examples/loop_forever_build.ail b/examples/loop_forever_build.ail new file mode 100644 index 0000000..0299257 --- /dev/null +++ b/examples/loop_forever_build.ail @@ -0,0 +1,10 @@ +(module loop_forever_build + (fn main + (doc "loop-recur iter 3 — an infinite loop (no non-recur exit) must COMPILE (typechecks AND compiles; no termination claim). Build-only; by design never returns, so the binary is never executed.") + (type (fn-type (params) (ret (con Unit)) (effects IO))) + (params) + (body (app print (app spin 0)))) + (fn spin + (type (fn-type (params (con Int)) (ret (con Int)))) + (params n) + (body (loop (i (con Int) 0) (recur (app + i 1)))))) diff --git a/examples/loop_sum_to_deep.ail b/examples/loop_sum_to_deep.ail new file mode 100644 index 0000000..0621dbd --- /dev/null +++ b/examples/loop_sum_to_deep.ail @@ -0,0 +1,14 @@ +(module loop_sum_to_deep + (fn main + (doc "loop-recur iter 3 — sum 1..1000000 iteratively. A non-tail hand-recursion to this depth would overflow; loop/recur is safe by construction. Expected stdout: 500000500000.") + (type (fn-type (params) (ret (con Unit)) (effects IO))) + (params) + (body (app print (app sum_to 1000000)))) + (fn sum_to + (type (fn-type (params (con Int)) (ret (con Int)))) + (params n) + (body + (loop (acc (con Int) 0) (i (con Int) 1) + (if (app > i n) + acc + (recur (app + acc i) (app + i 1))))))) diff --git a/examples/loop_sum_to_run.ail b/examples/loop_sum_to_run.ail new file mode 100644 index 0000000..a089e99 --- /dev/null +++ b/examples/loop_sum_to_run.ail @@ -0,0 +1,14 @@ +(module loop_sum_to_run + (fn main + (doc "loop-recur iter 3 — sum 1..10 via loop/recur. Expected stdout: 55.") + (type (fn-type (params) (ret (con Unit)) (effects IO))) + (params) + (body (app print (app sum_to 10)))) + (fn sum_to + (type (fn-type (params (con Int)) (ret (con Int)))) + (params n) + (body + (loop (acc (con Int) 0) (i (con Int) 1) + (if (app > i n) + acc + (recur (app + acc i) (app + i 1))))))) diff --git a/skills/planner/SKILL.md b/skills/planner/SKILL.md index 24ef2d9..050a2ed 100644 --- a/skills/planner/SKILL.md +++ b/skills/planner/SKILL.md @@ -208,6 +208,23 @@ Before handing the plan off, run this checklist inline: This is the recurring "compile-gate depends on a caller the plan defers past the gate" family (iter loop-recur.2 Concern 1) — scrub it here, every plan with a signature-change task. +8. **Verification-command filter strings must resolve.** Every + literal `cargo test … ` / `grep ` / `cargo test + --test ` in a Run step is itself a load-bearing artefact: + if its filter substring matches **no** test (or its `--test` + target file does not exist), the step silently asserts nothing — + a green "0 passed; 0 filtered" reads as success while proving + the opposite of the gate's intent. For every such command, + either (a) the filter substring must be one verified to match + ≥1 real named test/line in the current tree (name it, do not + guess from the feature word — e.g. tail-app guards are + `*_tail_position_*` / `*musttail*`, NOT `tail`), or (b) use the + unfiltered suite plus an explicit result-count assertion + (`… | awk '{p+=$4} END{print p}'`, expected ≥ a stated baseline) + so "nothing ran" cannot masquerade as "nothing regressed". This + is the recurring "literal verification filter doesn't resolve to + its evident target" family (iter loop-recur.3 Concern) — scrub + every Run step whose assertion lives in a filter string. Fix issues inline.