AILang/docs/journals/2026-05-15-iter-it.1.md

iter it.1 — loop/recur additive (parse/print/prose/check/codegen)

Date: 2026-05-15 Started from: 7381a4233b Status: DONE Tasks completed: 8 of 8

Summary

First of three iterations in the iteration-discipline milestone. Adds Term::Loop / Term::Recur / LoopBinder as first-class additive AST nodes end-to-end — parse, print, prose projection, canonical-JSON serde + round-trip, schema/spec-drift lockstep, typecheck (binder typing, recur arity/type unification via a loop_stack: &mut Vec<Vec<Type>> threaded exactly as mut.2 threaded mut_scope_stack, recur tail-position via a new private verify_loop_body helper with verify_tail_positions's public signature unchanged), and codegen (loop-header block with one phi per binder, recur as a back-edge br with a NEW parallel block_terminated = true setter). The change is strictly additive: zero deletions touch the existing tail-app/tail-do paths, verify_tail_positions' tail-app role, or any of the seven existing codegen block_terminated sites — the 32 within-iter deletion-lines are exclusively the Task-1 Internal(...) stubs and the Task-1 verify_tail_positions structural-passthrough being replaced by their real Task-5/Task-7 implementations (DD-3 stub-then-fill). This additive discipline is what makes the destructive it.3 safe later. Full cargo test --workspace green (63 ok result groups, 0 failed); loop_counter.ail builds and prints 55; the four Recur* diagnostics fire on their negatives; tail-app fixtures (bench_compute_intsum, list_map_poly) run byte-identically; zero IR/prose snapshot drift.

Per-task notes

• iter it.1.1: AST variants + every walker arm + serde/schema lockstep. Term::Loop { binders: Vec<LoopBinder>, body: Box<Term> }, Term::Recur { #[serde(default)] args: Vec<Term> }, and LoopBinder { name, #[serde(rename="type")] ty, init: Box<Term> } (DD-2 derives = Debug, Clone, Serialize, Deserialize, no PartialEq; note init is Box<Term> per the plan's literal code block, whereas MutVar.init is bare Term — a small divergence from "mirrors MutVar exactly", recorded under Concerns). ~40 walker arms across desugar (incl. 3 in-test any_* helpers not enumerated by the plan), workspace, all of ailang-check (lib/lift/linearity/mono/pre_desugar_validation/reuse_shape/ uniqueness), codegen (escape×3/lambda/lib), prose, surface parse/print, ail main + the test-side codegen_import_map_fallback_pin.rs walker the plan flagged (mut.1 missed its analogue; did not miss it here). Typecheck + codegen dispatch stubbed Internal(...) (tuple variant — the plan pseudo-code's Internal { msg } struct form does not exist; copied mut.1's verbatim tuple shape). synth_with_extras got real arms (loop → body's type, recur → unit) per plan. schema_coverage left RED on "variant never observed" until Task 3 (plan-expected); every other test green.

• iter it.1.2: Form-A parse. parse_loop / parse_recur modelled on parse_mut / parse_assign; the binder list is an explicitly parenthesised group (loop ((var ...)*) BODY+) (form_a.md production added in Task 1.10 lockstep). Plan pseudo-code named non-existent helpers (parse_term_str, parse_body_seq, parse_loop_binders); substituted the real harness (module-level parse_term) and inlined the per-(var …) decode exactly as parse_mut does — the plan explicitly forbade introducing a new body-folding helper. EBNF prologue + unknown-head error list extended. RED→GREEN confirmed.

• iter it.1.3: Form-A print + positive fixtures. Print arms landed early in Task 1 as a build-unblock necessity (documented mut.1 precedent — print arms are exhaustive-match neighbours of the synth/lower_term dispatchers). loop_smoke.ail / loop_nested_in_lambda.ail written in real Form-A matching mut_counter.ail's verbatim surface spellings ((con Int), (app == …), (app + …), (lam (params (typed …)) (ret …) (body …))) — the plan's fixture pseudo-code ((fn count_to : Int -> Int …)) is not valid Form-A and the plan's prose explicitly directed matching mut_counter.ail verbatim. parse→print→parse byte-idempotent; schema_coverage now GREEN (Task 1.11 deferred RED closed).

• iter it.1.4: Prose projection lockstep. Step 4.1's literal instruction ("render to prose and back, asserting AST equality") is not expressible — AILang has no Form-B→AST parser by design (CLAUDE.md / crates/ail/src/main.rs:207 "no parser exists for the prose surface"). Recorded as a plan defect (see Concerns). The substantive work (render/free-var/subst arms, landed in Task 1) is correct and verified by the full prose suite; added loop_and_recur_project_to_prose, a render-assertion pin using the same render_term harness the nearest existing prose tests (not_with_tail_flag_keeps_prefix_form) use — the feasible equivalent of the plan's intent. This is the documented mut.1-class plan-pseudo-vs-real-API substitution, not a silent swap. Status DONE_WITH_CONCERNS.

• iter it.1.5: Real typecheck (DD-1). Four CheckError variants (RecurOutsideLoop / RecurArityMismatch / RecurTypeMismatch / RecurNotInTailPosition) — no span field because no Span type is in scope in this crate and no existing CheckError variant carries one (the plan pseudo-code's span: Option<Span> references a non-existent-here type; mirrored the real MutAssignOutOfScope field-shape exactly as the plan's prose instructed). loop_stack: &mut Vec<Vec<Type>> threaded through synth's signature + every call site (signature + 22 single-line recursive + 3 multi-line recursive + 2 production entry points + lift.rs + 2 mono.rs + builtins.rs test helpers + 8 in-source #[cfg(test)] calls), exactly the mut.2 pattern. Real Loop arm binds binder names into self-… locals (save/restore like Term::Let — the correct precedent for immutable bindings, not the mut-var mechanism) and pushes the binder type-vector onto loop_stack. Term::Recur synth returns a fresh metavar (Subst::fresh(counter)), not the plan's flagged-risky Type::unit() — see Concerns; this resolves the plan's own self-review "Open risk" correctly (a fresh metavar is the codebase-idiomatic bottom that unifies anywhere, which is what makes (if c then (recur …)) typecheck — Type::unit() would have broken loop_smoke's if-branch unification). verify_loop_body added as a new private helper; verify_tail_positions' public signature unchanged (it.3 reworks it). The diagnostic-doc list in diagnostic.rs was NOT extended — the mut codes were never added there (false plan premise; followed the actual mut.2 precedent). RED→GREEN: loop_smoke clean, four negatives each return their exact code. loop_stack threading caused zero regressions.

• iter it.1.6: Carve-out inventory. EXPECTED extended with the four test_recur_*.ail.json negatives, 13→17, in an it.1 group (the const is milestone-grouped not alphabetical; the test uses a BTreeSet so order is functionally irrelevant). GREEN.

• iter it.1.7: Real codegen (loop-header + phi + recur back-edge + lambda boundary). loop_frames: Vec<LoopFrame> field added (mut.3 analogue to mut_var_allocas — init/clear/lambda-save-reset- restore symmetric). Loop lowers to loop.header.<id> with one phi per binder; binder phi SSA pushed into self.locals so Term::Var resolves to it via the existing lookup. Each phi's back-edge incoming list is emitted as a unique /*RECUR_EDGES…*/ LLVM-comment placeholder, then replacen(…, 1)-patched once the body and all its recur sites are lowered (the plan's patch_loop_phis mechanism; the placeholder is a comment so an unpatched one would be inert IR, not a syntax error). Term::Recur records (pred_block, [arg_ssa]), emits the back-edge br, sets block_terminated = true — a NEW parallel setter; the seven existing tail-driven setters are untouched (verified: zero deletions touch them). Reused start_block/fresh_ssa/fresh_id/ self.current_block instead of adding the plan-pseudo's sprawl of single-use helper methods (plan explicitly: "REUSE … grep before adding"). loop_counter.ail builds + prints 55.

• iter it.1.8: tail-app coexistence + IR snapshots + acceptance. bench_compute_intsum.ail → 3500003500000, list_map_poly.ail → 2\n3\n… (byte-identical to pre-it.1; no existing codegen path modified). Zero IR/prose snapshot drift (no blanket regen). All it.1 acceptance bullets verified — the spec's "five Recur diagnostics" is four in it.1 (the fifth, NonStructuralRecursion, is it.2; resolved per plan Step 8.3).

• Phase 3 (E2E): added loop_in_lambda_e2e.ail + e2e loop_in_lambda_runs_and_prints_49 — a loop inside a lam body invoked via a returned closure, proving the lambda-boundary loop_frames save/restore is codegen-sound (the no-main loop_nested_in_lambda.ail parse/print fixture cannot reach codegen via the CLI; this runnable fixture closes that gap).

Concerns

• LoopBinder.init is Box<Term> whereas the analogous MutVar.init is bare Term. Per the plan's literal Step 1.1 code block (which specified init: Box<Term>), not implementer drift. DD-2 says "mirrors MutVar exactly" — the divergence is in the plan's own pseudo-code vs. its own DD-2 prose. Functional impact: none (consistent (*b.init).clone() / Box::new(...) / &mut b.init handling everywhere). A future iter that wants strict parity could unbox; no it.1 test depends on the boxing.

• Term::Recur synth result: chose Subst::fresh(counter) (a fresh metavar) over the plan's named fallback Type::unit(). The plan's self-review flagged this exact point as an "Open risk" with Type::unit() as the fallback "because recur is always in tail position so its value is never consumed". But recur frequently appears as an if branch (loop_smoke: (if (== i n) i (recur …))), where the branch types MUST unify — unify(Int, unit) fails, so Type::unit() would have broken loop_smoke. A fresh metavar unifies with any type (codegen-idiomatic bottom; the same mechanism Subst::fresh serves everywhere). This is the correct resolution of the plan's own flagged risk, not a deviation from its intent.

• Step 4.1 plan defect: it asks for a prose round-trip asserting AST equality; AILang has no Form-B parser by design. Substituted a feasible render-assertion pin (mut.1-class plan-pseudo-vs-real substitution). The plan's substance (prose lockstep for the new variants) is fully met. Boss may want to tighten the planner template so it does not script prose round-trips.

• Step 5.2 plan premise defect: "Add the four code strings to the diagnostic-doc list … same list the mut codes were added to" — the mut codes were never added to diagnostic.rs's doc list. Followed the actual mut.2 precedent (not extended); the authoritative registry is CheckError::code(), which has all four.

• The plan's recon line numbers (e.g. desugar.rs:349/559/…, lib.rs:2590/2613/3593/3614, parse.rs:1212) had drifted vs. the live tree; drove every site off cargo build's E0004 enumeration as Step 1.2 instructs. Site count also differed (6 desugar E0004 sites vs. plan's 9; resolved via the build, which is authoritative). No behavioural impact — same class as mut.1's documented recon misindexing.

Known debt

• Prose-side (loop …) / (recur …) rendering is a minimal-correctness shape (loop { var n = init; …; body } / recur(a, b)), explicitly mirroring the deliberately-placeholder mut-block prose shape. The prose surface for loops is not yet designed; a follow-on prose iter refines it once the LLM-author signal arrives. Not drift — recorded for visibility.

• loop_nested_in_lambda.ail has no main, so it is a parse/print/typecheck fixture only (CLI codegen needs a main). Codegen-of-loop-in-lambda is instead proven by the runnable Phase-3 loop_in_lambda_e2e.ail. No gap remains; recorded so a future reader does not mistake the no-main fixture for dead coverage.

Files touched

AST/core: crates/ailang-core/src/ast.rs, crates/ailang-core/src/desugar.rs, crates/ailang-core/src/workspace.rs, crates/ailang-core/specs/form_a.md. Drift/coverage tests: crates/ailang-core/tests/{design_schema_drift, schema_coverage,spec_drift,carve_out_inventory}.rs. Check: crates/ailang-check/src/{lib,lift,linearity,mono, pre_desugar_validation,reuse_shape,uniqueness,builtins}.rs, crates/ailang-check/tests/loop_recur_pin.rs (new). Codegen: crates/ailang-codegen/src/{lib,escape,lambda}.rs. Surface/prose: crates/ailang-surface/src/{parse,print}.rs, crates/ailang-prose/src/lib.rs. CLI: crates/ail/src/main.rs, crates/ail/tests/{e2e,codegen_import_map_fallback_pin}.rs. Spec: docs/DESIGN.md. Fixtures (new): examples/loop_smoke.ail, examples/loop_nested_in_lambda.ail, examples/loop_counter.ail, examples/loop_in_lambda_e2e.ail, examples/test_recur_{outside_loop,arity_mismatch,type_mismatch, not_in_tail_position}.ail.json.

Stats

bench/orchestrator-stats/2026-05-15-iter-it.1.json