iter it.2: structural-guardedness checker + first real Diverge effect

Iteration-discipline milestone, 2 of 3. Strictly additive (nothing
tail-related removed; that is it.3). New whole-body pass
verify_structural_recursion sibling of verify_tail_positions
(DD-1): smaller-set algorithm with implicit candidate inference +
unconstrained accumulators (DD-2, foldl=structural), self/mutual
via inline ADT-family union-find (DD-3), it.2-only tail==false
grandfather. CheckError::NonStructuralRecursion. term_contains_loop
(stops at Term::Lam, DD-4) injects Diverge so existing
UndeclaredEffect enforces it, no new variant; lam-arrow + LetRec
sub-effect sites wired. DESIGN.md Decision 3 synced. Four it.1
loop fixtures gained !Diverge.

Two spec-premise boundary defects surfaced + resolved within the
additive invariant (corpus clean, check not weakened), recorded as
corrected it.3 corpus-migration scope: (1) the "21 tail-app
fixtures" grandfather premise under-counts the corpus —
no-ADT-candidate counter recursions have no structural position to
verify, deferred to it.3; (2) two RC-regression fixtures joined the
spec's transitional tail-app grandfather as the other 20 do (RC==GC
guards verified still green). cargo test --workspace 622/0; 9
acceptance pins non-vacuous. Spec fda9b78, plan bc9f512.

docs/DESIGN.md

@@ -164,8 +164,16 @@ Advantages:
 
 The default is total, pure functions. Effects are declared as a set in the
 function type: `(Int) -> Int ![IO]`. The effect set is row-polymorphic
-(`![IO | r]`). In the MVP only the effects `IO` and `Diverge` (for infinite
-loops) are wired up.
+(`![IO | r]`). Two effects are wired up: `IO` (observable side effects,
+raised by `do`-operations), and `Diverge` (non-termination). As of iter
+it.2 (2026-05-15) `Diverge` is no longer nominal: it is the effect carried
+by any function whose body contains a `loop` (or that calls a
+`Diverge`-declaring function), surfaced through the existing
+declared-vs-raised reconciliation (an undeclared `Diverge` is the existing
+`UndeclaredEffect`, no new diagnostic). Structural recursion is pure and
+total and carries no effect — the author who wants a `!Diverge`-free
+signature is structurally pulled toward structural recursion and pays
+`!Diverge` only when genuinely writing an unbounded `loop`.
 
 This is the most important LLM property: when I read a function, I can trust
 its signature without reading the body.
@@ -2446,8 +2454,16 @@ parse, print, prose-project, round-trip, typecheck (binder typing,
 recur arity/type unification, recur tail-position) and codegen
 (loop-header block with one phi per binder, `recur` as a back-edge
 `br`) without removing or modifying the existing `tail-app`/`tail-do`
-paths. The structural-recursion restriction and the `Diverge`
-effect land in it.2; `tail-app`/`tail-do` are retired in it.3. See
+paths. As of iter it.2 (2026-05-15) the structural-recursion
+guardedness restriction and the `Diverge` effect are in effect:
+a non-structural recursion-by-call is the compile error
+`NonStructuralRecursion` (directing the author to `(loop …)` /
+`recur`), and any fn whose body contains a `Term::Loop` (or calls a
+`Diverge`-declaring callee) must declare `!Diverge` in its effect
+row — structural recursion stays pure, total, and effect-free. A
+transitional grandfather exempts still-`tail: true`-marked recursive
+calls so the corpus type-checks unchanged through it.2.
+`tail-app`/`tail-do` are retired in it.3. See
 `docs/specs/2026-05-15-iteration-discipline.md`.
 
 **`Literal`**:

docs/journals/2026-05-15-iter-it.2.md

@@ -0,0 +1,241 @@
+# iter it.2 — structural-guardedness checker + first real Diverge effect
+
+**Date:** 2026-05-15
+**Started from:** bc9f5120034f2552ad7e78454bb198472404d4b1
+**Status:** DONE
+**Tasks completed:** 6 of 6
+
+## Summary
+
+Second of three iterations in the iteration-discipline milestone.
+Adds the structural-recursion guardedness checker
+(`CheckError::NonStructuralRecursion`) and the first real
+implementation of the Decision-3 `Diverge` effect, both strictly
+additive — nothing `tail`-related is removed (that is it.3). A new
+whole-body pass `verify_structural_recursion` runs as a sibling of
+`verify_tail_positions` in `check_fn`'s post-synth region (DD-1); it
+implements the `smaller`-set algorithm with implicit candidate
+inference and unconstrained accumulator positions (DD-2: the
+foldl-shape accumulator walk classifies as structural), self- and
+mutual-recursion identification with ADT-family connected components
+via an inline union-find (DD-3), and the it.2-only `tail==false`
+grandfather. A new `term_contains_loop` (stopping at `Term::Lam`
+boundaries, DD-4) injects `"Diverge"` into the raised effect set so
+the existing `UndeclaredEffect` reconciliation enforces it with no
+new diagnostic variant; the lam-arrow and `Term::LetRec` cases are
+wired at their respective sub-effect reconcile sites. DESIGN.md
+Decision 3 + the §Data-model hook are synced to present tense. All
+20 tail-app corpus fixtures stay grandfathered-clean;
+`cargo test --workspace` green at 622 passed / 0 failed; the it.1
+loop fixtures gained `!Diverge` (in scope per the plan).
+
+Two spec/plan boundary defects surfaced and were resolved inside
+the task's own stated invariants (corpus stays clean, structural
+check not weakened) — see Concerns. Both are recorded so the it.3
+planner and a future spec-tightening pass have the signal.
+
+## Per-task notes
+
+- iter it.2.1: `CheckError::NonStructuralRecursion { callee, arg }`
+  added beside the it.1 `Recur*` variants (bracket-`[code]`-free
+  Display per F2), `code()` → `"non-structural-recursion"`, `ctx()`
+  → `{callee, arg}`. New pin file `structural_recursion_pin.rs`
+  (loop_recur_pin harness verbatim). `diagnostic.rs` untouched
+  (plan-correctly: `code()` is the registry). RED→GREEN clean.
+
+- iter it.2.2: the pass + helpers next to `verify_loop_body`:
+  `adt_type_head` / `adt_param_positions` / `ctor_bound_names` /
+  `collect_rec_calls_walk` (single `smaller`-threaded walk folding
+  collection + guardedness, `tail==false` grandfather in the App
+  arm, stops at `Term::Lam`) / `call_guarded_at` /
+  `rec_call_arg_display`. `check_fn`/`check_def` gained a
+  `module_fns: &[&FnDef]` param (plan-anticipated in Step 3.2 for
+  mutual grouping; threaded from the per-module loop; synthetic
+  instance-method fns pass `&[]`). Wired
+  `verify_structural_recursion(f, &env, module_fns)?;` immediately
+  after `verify_tail_positions`. Two over-rejection classes found
+  against the corpus and fixed within the task's own acceptance
+  constraint (see Concerns §1, §2). Status DONE_WITH_CONCERNS.
+
+- iter it.2.3: ADT-family union-find (`TypeUnionFind`, inline
+  `BTreeMap`-backed, path-halving, no dependency) +
+  `mutual_structural_group` returning a `MutualGroup { members,
+  family_valid }` (a plan-pseudo-vs-real refinement — the plan's
+  `Vec<&FnDef>` could not express the cross-family-negative; an
+  empty group silently drops the cross-call instead of rejecting
+  it). `collect_direct_app_var_callees` for call-graph adjacency
+  (same lam boundary). `verify_structural_recursion` reworked to a
+  single `guarded` closure: self-calls vs callee-cand, cross-calls
+  guarded only if `family_valid`. tree/forest clean, cross-family
+  rejected. RED→GREEN.
+
+- iter it.2.4: `term_contains_loop` (exhaustive Term match;
+  `Loop=>true`, `Lam=>false`, all else recurse incl. Recur args).
+  Injection in `check_fn` before the declared-vs-raised reconcile.
+  Lam-arrow coherence wired at BOTH the `Term::Lam` synth
+  sub-effect site AND the `Term::LetRec` synth sub-effect site
+  (the plan named only the Lam site; recon line drifted and there
+  are now two structurally-identical reconcile sites — a
+  `Term::LetRec` clause is a fn-equivalent, same as the it.1
+  tail-position treatment, so a loop in a deferred LetRec body must
+  also carry Diverge; injecting at both is the sound completion,
+  not an extra). Four it.1 loop fixtures + two HOF signatures
+  updated to declare `!Diverge` (see Concerns §3). Status
+  DONE_WITH_CONCERNS.
+
+- iter it.2.5: DESIGN.md Decision 3 rewritten (`Diverge` no longer
+  nominal; carried by loop-bearing / Diverge-calling fns; structural
+  recursion pure+total; `IO` description kept + sharpened) +
+  §Data-model hook sentence → present tense, it.3 retirement
+  retained. Drift anchors (`"t":"loop"`/`"t":"recur"`) untouched —
+  `design_schema_drift`/`schema_coverage`/`spec_drift` green.
+
+- iter it.2.6: acceptance gate. Full workspace green; all 7 named
+  spec-acceptance pins green; `mut_counter.ail` still runs `55`
+  (grandfathered, unmigrated); all 20 tail-app corpus fixtures
+  grandfathered-clean (zero `non-structural-recursion`).
+
+- Phase 3 (E2E): `examples/struct_rec_sum_e2e.ail` +
+  `struct_rec_sum_runs_and_prints_15` — an it.2-clean structural
+  recursion (plain non-tail, no `!Diverge`, no `tail-app`) that
+  runs to 15, proving the "total" verdict is behaviourally sound,
+  not just a typecheck assertion. Plus
+  `loop_needs_diverge_runs_and_prints_55` — the Diverge-path twin,
+  proving the Diverge injection is a typecheck-only obligation with
+  zero codegen impact (the loop lowers/runs identically to it.1).
+
+## Concerns
+
+- **§1 — Spec/it.2-boundary defect: no-ADT-candidate recursion.**
+  The plan's load-bearing invariant ("the 21 tail-app corpus
+  fixtures stay clean through it.2 because `collect_rec_calls` only
+  collects `tail==false` calls") is insufficient: corpus fixtures
+  like `bench_latency_explicit.ail`'s `build_tree(depth: Int)`
+  recurse **non-tail** (inside a `term-ctor`) on a **primitive**
+  arg — the `tail==false` grandfather does not exempt them, and
+  they have no ADT candidate position. Strict DD-2 would reject
+  ~18 corpus fixtures in it.2, contradicting the additive mandate.
+  Resolution (does NOT weaken the ADT check): a def with no ADT
+  candidate position AND no mutual cycle has no *structural*
+  position to verify — it is integer-counter / other-shaped
+  recursion whose migration to `(loop …)` is the destructive it.3
+  corpus pass, not it.2's job. it.2 only rejects *misuse of an ADT
+  structural position*; the spec's load-bearing negative
+  (`f(xs: IntList) = … f(xs) …`, HAS an ADT candidate) still fires.
+  This is the honest reading of DD-2's "for each candidate
+  structural position" (vacuous when there are none); the plan's
+  pseudo-code only handled `calls.is_empty()`, not
+  `cand.is_empty()`. Recorded as a resolved it.2/it.3 boundary
+  clarification — the it.3 planner should expect these to be the
+  corpus-migration set, and a spec-tightening pass may want to
+  state the no-candidate boundary explicitly.
+
+- **§2 — Two RC-regression fixtures needed the spec's transitional
+  grandfather applied.** `rc_pin_recurse_implicit.ail` and
+  `rc_let_alias_implicit_param.ail` (NOT in the 20-fixture tail-app
+  set) contain `loop(n: Int, t: Tree) = … (app loop (- n 1) t)` —
+  an ADT param (`t: Tree`) held constant while decrementing an Int,
+  recursing **non-tail**. This is structurally identical to the
+  required negative (`f(xs)=…f(xs)…`): both pass the ADT param
+  unchanged at its candidate position. It therefore correctly fires
+  `NonStructuralRecursion` and CANNOT be exempted by broadening the
+  grandfather without also letting the required negative through
+  (which would weaken the check — explicitly forbidden by the
+  carrier). The recursive call is in tail position, so the
+  spec-prescribed transitional treatment applies: mark it
+  `tail-app` (the it.2 "Transitional grandfather" — identical to
+  the idiom 20 other corpus fixtures use; it.3 migrates to
+  `(loop …)`). The 18d.4 / 18g regressions these fixtures guard
+  live in `pin`/`pin_aliased`'s match arm-close, unaffected by the
+  `loop` recursion's lowering — so this is NOT adapting-a-test-to-
+  dodge-a-bug (the check is correct; the fixture joins the
+  transitional grandfather, with an in-fixture comment recording
+  why). Both fixtures' RC==GC e2e regression guards still pass.
+
+- **§3 — it.1 loop fixtures gained `!Diverge` (plan-in-scope, full
+  enumeration vs plan's two examples).** Plan Step 4.4 named
+  `loop_counter.ail` + `loop_in_lambda_e2e.ail` explicitly; the
+  same in-scope action applies to all it.1 loop fixtures.
+  `loop_counter.ail` (loop in `main` body) → `(effects IO Diverge)`.
+  `loop_smoke.ail` `count_to` (loop in fn body) → `(effects
+  Diverge)`. `loop_nested_in_lambda.ail` / `loop_in_lambda_e2e.ail`
+  (loop in a *lam* body) → `(effects Diverge)` on the **lam**, with
+  the enclosing fn / HOF param-and-return fn-types threaded to
+  carry `!Diverge` (effects are part of the function type — forced,
+  not optional). The lam-boundary rule held: `main` in
+  `loop_in_lambda_e2e` does NOT carry Diverge from its own body
+  (the loop is behind the lam edge) but DOES via callee-effect
+  propagation through `apply` (free per DD-4, identical to `!IO`).
+  The injection was not weakened. Both it.1 loop e2e tests
+  (55 / 49) still pass; round-trip green on every modified `.ail`.
+
+- **§4 — `MutualGroup` is a plan-pseudo-vs-real-API refinement.**
+  The plan's `mutual_structural_group -> Vec<&FnDef>` could not
+  express "cross-family mutual recursion must be rejected" — an
+  empty return drops the cross-call from collection instead of
+  flagging it. Split into `{ members: HashSet<String>,
+  family_valid: bool }`: membership drives collection (cross-calls
+  always collected), validity drives the verdict (a cross-call into
+  a family-invalid cycle is unconditionally unguarded). Minimal
+  shape satisfying both plan requirements; same documented
+  substitution class as it.1's repeated plan-pseudo-vs-real notes.
+
+## Known debt
+
+- The structural check's `rec_call_arg_display` has no real
+  term-pretty-printer (only `type_to_string` exists in
+  `ailang_core::pretty`); non-`Var` offending args render as terse
+  tags (`Ctor(…)`, `fn(…)`, `<expr>`). The canonical non-structural
+  case is always a bare `Var` (the param passed unchanged), so this
+  is cosmetic on the diagnostic only; a future term-pretty-printer
+  would improve the `arg` field. Not drift — recorded for
+  visibility.
+- Three exhaustive Term-walks now exist in the it.2 region
+  (`collect_rec_calls_walk`, `collect_direct_app_var_callees`,
+  `term_contains_loop`). Each collects a genuinely different thing
+  with different threading; a shared generic walker would be
+  speculative abstraction per the AILang quality bar. Recorded so a
+  future reader does not mistake the triplication for an oversight.
+- `mutual_structural_group`'s reaches/component BFS is O(n²)-ish
+  over a module's fns, run once per fn-check. Corpus modules are
+  <40 defs; not a hot path. No action; recorded for visibility.
+
+## Blocked detail
+
+None — all 6 tasks completed; the two spec/plan boundary defects
+(Concerns §1, §2) were resolved inside the task's own stated
+acceptance constraints (corpus stays clean, structural check not
+weakened), not escalated, because the carrier explicitly anticipated
+corpus-breakage handling and the resolutions are the spec's own
+transitional model rather than judgement substitutions.
+
+## Files touched
+
+Check core: `crates/ailang-check/src/lib.rs` (CheckError variant +
+code/ctx; `module_fns` thread through check_def/check_fn;
+`verify_structural_recursion` + helpers + `TypeUnionFind` +
+`MutualGroup` + `term_contains_loop`; Diverge injection at
+check_fn / Term::Lam / Term::LetRec sites).
+Tests (new): `crates/ailang-check/tests/structural_recursion_pin.rs`
+(9 pins). Carve-out: `crates/ailang-core/tests/carve_out_inventory.rs`
+(EXPECTED 17→20 + header comment rewritten accurate).
+E2E: `crates/ail/tests/e2e.rs` (2 new Phase-3 tests).
+Spec: `docs/DESIGN.md` (Decision 3 + §Data-model hook).
+Fixtures (new): `examples/struct_rec_list_len.ail`,
+`examples/struct_rec_foldl_sum.ail`,
+`examples/struct_rec_tree_forest.ail`,
+`examples/struct_rec_sum_e2e.ail`,
+`examples/loop_needs_diverge.ail`,
+`examples/test_non_structural_recursion.ail.json`,
+`examples/test_mutual_cross_family.ail.json`,
+`examples/test_loop_missing_diverge.ail.json`.
+Fixtures (modified): `examples/loop_counter.ail`,
+`examples/loop_smoke.ail`, `examples/loop_nested_in_lambda.ail`,
+`examples/loop_in_lambda_e2e.ail` (all four: it.1 loop fixtures →
+`!Diverge`); `examples/rc_pin_recurse_implicit.ail`,
+`examples/rc_let_alias_implicit_param.ail` (transitional `tail-app`
+grandfather on the `loop` fn — Concerns §2).
+
+## Stats
+
+bench/orchestrator-stats/2026-05-15-iter-it.2.json

docs/journals/INDEX.md

@@ -76,3 +76,4 @@
 - 2026-05-15 — iter mut.4-tidy: close mut-local audit drift. Architect's two `[high]` items addressed — new `CheckError::MutVarCapturedByLambda` rejects any lambda whose body's free vars hit the enclosing `mut_scope_stack` (via the existing `desugar::free_vars_in_term` helper, gated on non-empty stack), and `codegen/lambda.rs`'s blame-typechecker `Internal` path becomes `unreachable!` once typecheck is the gate. Two `[medium]` stale-history comments cleaned in DESIGN.md §"Term (expression)" and `ailang-codegen/src/lib.rs`. New negative fixture `examples/test_mut_var_captured_by_lambda.ail.json` + driver test extension; `carve_out_inventory.rs` EXPECTED 12→13. Spec §"Out of scope" amended with the lambda-capture rejection bullet. Bench: `compile_check.py` `check_ms` showed a uniform ~30-50% regression across the suite traced to a fixed-cost startup tax (mut-* added ~1400 lines of typecheck/codegen surface; the short-circuit on empty `mut_scope_stack` walk in Term::Var did not move the needle, falsifying the hot-path hypothesis); ratified as a feature tax with paired baseline update on `bench/baseline_compile.json`. `check.py` tail-noise envelope continues the audit-pd carve-out (no ratify needed). `cross_lang.py` clean. Tests 594 → 598. mut-local milestone audit closed → 2026-05-15-iter-mut.4-tidy.md
 - 2026-05-15 — bugfix mut-diag-double-code: fieldtest F2 — the four mut-local `CheckError` variants embedded `[<code>]` in their `#[error]` Display body while the non-JSON CLI formatter also prepends `[code]`, doubling it in human stderr; dropped the embedded prefix from the four strings, bringing them in line with all non-mut variants; RED-first via `debug` (`ct1_check_cli.rs::check_human_mode_renders_mut_diagnostic_code_exactly_once`), GREEN applied inline as a trivial mechanical edit; tests 598 → 599 → 2026-05-15-bugfix-mut-diag-double-code.md
 - 2026-05-15 — iter it.1: iteration-discipline milestone (1 of 3) — `Term::Loop` / `Term::Recur` / `LoopBinder` added as strictly-additive first-class AST nodes end-to-end: Form-A `parse_loop`/`parse_recur` + print + prose render/free-var/subst lockstep + canonical-JSON serde/round-trip + schema/spec-drift/schema-coverage/carve-out lockstep (13→17) + typecheck (binder typing + recur arity/type unification via `loop_stack: &mut Vec<Vec<Type>>` threaded exactly as mut.2's `mut_scope_stack`; recur-tail-position via a new private `verify_loop_body`, `verify_tail_positions` public signature unchanged) + codegen (loop-header block, one phi per binder, recur back-edge `br` with a NEW parallel `block_terminated` setter, lambda-boundary `loop_frames` save/restore mirroring mut.3). Four new `CheckError` variants `Recur{OutsideLoop,ArityMismatch,TypeMismatch,NotInTailPosition}` (bracket-`[code]`-free Display per the F2 convention). Strictly additive verified — zero deletions touch `tail-app`/`tail-do`, `verify_tail_positions`' tail-app role, or the seven existing codegen `block_terminated` sites (the 32 within-iter deletions are exclusively DD-3 stub-then-fill replacements). `Term::Recur` synth returns a fresh metavar (resolves the plan's flagged `Type::unit()` open risk: recur appears in `if` branches that must unify with the sibling type). `loop_counter.ail`→`55`, `loop_in_lambda_e2e.ail`→`49`, four negatives fire exact codes, `tail-app` fixtures byte-identical, zero IR/prose snapshot drift; `cargo test --workspace` green. Two mut.1-class plan-pseudo-vs-reality substitutions recorded (prose round-trip asserting AST-equality is impossible — no Form-B parser by design; the diagnostic.rs doc-list premise was false) → 2026-05-15-iter-it.1.md
+- 2026-05-15 — iter it.2: iteration-discipline milestone (2 of 3) — structural-recursion guardedness checker + first real `Diverge` effect, strictly additive (nothing `tail`-related removed; that is it.3). New whole-body pass `verify_structural_recursion` runs as a sibling of `verify_tail_positions` in `check_fn`'s post-synth region (DD-1): the `smaller`-set algorithm with implicit candidate-position inference + unconstrained accumulator positions (DD-2 — foldl-shape accumulator classifies as structural recursion, pure+total), self/mutual identification with an inline ADT-family connected-components union-find (DD-3), and the it.2-only `tail==false` grandfather. `CheckError::NonStructuralRecursion {callee,arg}` (bracket-`[code]`-free Display per F2). `term_contains_loop` (stops at `Term::Lam` boundaries, DD-4) injects `"Diverge"` into the raised effect set so the existing `UndeclaredEffect` machinery enforces it with no new diagnostic variant; lam-arrow + `Term::LetRec` sub-effect sites wired (loop behind a lam edge carries `!Diverge` on the lam's arrow, propagating via the free callee-effect path, not leaking to the enclosing fn — exactly as `!IO` scopes). DESIGN.md Decision 3 + §Data-model hook synced present-tense. Four it.1 loop fixtures gained `!Diverge`. Two spec-premise boundary defects surfaced and resolved inside the task's invariants (corpus clean, check not weakened): (§1) the spec's "21 tail-app fixtures" grandfather premise under-counts the corpus — no-ADT-candidate counter recursions (~18, e.g. `build_tree(depth:Int)`) have no structural position to verify and are deferred to it.3 migration; (§2) two RC-regression fixtures (`rc_pin_recurse_implicit`, `rc_let_alias_implicit_param`) hold an ADT param constant while decrementing an Int — genuinely non-structural, joined the spec's transitional `tail-app` grandfather exactly as the other 20 corpus fixtures do (their RC==GC regression guards verified still green; the regression lives in the unchanged `pin`/`pin_aliased` bodies). Both recorded as corrected it.3 corpus-migration scope. `cargo test --workspace` 622 green / 0 red; all 9 acceptance pins non-vacuous (negatives `.contains(code)`); struct_rec_sum→15, loop_needs_diverge→55, the it.1 55/49 e2e still green → 2026-05-15-iter-it.2.md