Commit Graph

9 Commits

Author SHA1 Message Date
Brummel 07f080256c iter remove-mut-var-assign.1: atomic removal of mut/var/assign
mut/var/assign removed from AILang entirely and atomically. Deleted:
Term::Mut/Term::Assign/struct MutVar; the three Form-A keywords +
parse_mut/parse_assign + grammar EBNF; the 4 mut CheckError variants;
the mut_scope_stack synth threading (param dropped from synth + every
internal/external/test caller); the two lower_term arms; and every
exhaustive no-_ Term::Mut/Term::Assign match arm across 17 source
files — cut in lockstep with DESIGN.md, fixtures, the drift trio,
carve-out and roadmap so the schema is honest at every commit. No
catch-all wildcard introduced (verified). loop/recur + let/if are
the surviving forms.

The shared codegen alloca machinery survives (loop reuses it):
mut_var_allocas renamed binder_allocas (representation-only, loop
codegen byte-identical) and the shared Term::Lam escape guard
simplified to !loop_stack.is_empty() with the loop half
(LoopBinderCapturedByLambda) byte-equivalent. Feature-acceptance
applied inverted: the removed feature fails clause 2 (redundant)
and clause 3 (IS the iterated-mutable-state bug class).

Behaviour preservation is executable: mut_counter/mut_sum_floats
still print 55 after the faithful let/if rewrite. The removal is
made executable by the new mut_removed_pin.rs (4 must-fail pins).
Independent verification: cargo test --workspace 605/0, zero
residual mut symbols in any crate source, loop/recur non-regression
all green (55 / 500000500000 / infinite-compiles / the
lambda_capturing_loop_binder pin), roundtrip_cli PASS.

One DONE_WITH_CONCERNS: a 4th recurrence of the recon-undercount
class (in-source mod tests + a drift-pin fn + 5 orphaned mut
.ail.json carve-outs + a non-enumerated E0599); all resolved within
implementer remit, no behaviour change. Milestone-close audit then
fieldtest remain.

spec docs/specs/2026-05-18-remove-mut-var-assign.md (grounding PASS)
plan docs/plans/remove-mut-var-assign.1.md
2026-05-18 11:06:17 +02:00
Brummel edd2558d35 iter loop-recur.3: codegen — real LLVM-IR lowering + run-to-value E2E (milestone terminal)
Third and terminal iteration of the standalone loop/recur
milestone (plan eae73bf). Replaces the iter-1 lower_term
CodegenError::Internal stub for Term::Loop/Term::Recur with real
LLVM-IR lowering: loop binders as entry-block allocas (mut.3
pending_entry_allocas, reusing mut_var_allocas so the existing
Term::Var load path is byte-unchanged), a fresh loop-header block,
recur stores + back-edge br, a loop_frames stack saved/restored at
the lambda boundary. clang -O2 mem2reg promotes the allocas to
phi. Four Boss design calls implemented verbatim and journalled
(alloca-not-hand-phi; mut_var_allocas reuse; emergent loop-exit
via the if/match join; single block_terminated field + parallel
SET site). Diff confirmed surgical: codegen/lib.rs 3 hunks (field
+ init + stub->2-arms), lambda.rs 2 hunks (save+restore); zero
edits to any existing block_terminated SET/READ site, tail-app
lowering, or verify_tail_positions (Boss call 4, the spec-pinned
invariant). Three new .ail fixtures: sum_to->55,
deep-n 1e6->500000500000 (clause-2 correctness made executable),
infinite-loop build-only. Codegen-only: no schema/typecheck
change; hash pins + drift trio stay green untouched.

One DONE_WITH_CONCERNS (T3): the plan's `cargo test ... tail`
filter resolved to no tests; ran via real names + the full 619/0
which subsumes it (feedback_plan_pseudo_vs_reality class, no
behaviour change). Boss systemic fix folded in: planner SKILL.md
Step-5 gains item 8 (verification-command filter strings must
resolve) — the second planner-meta-gap this milestone surfaced.

cargo test --workspace 616 -> 619 / 0 red (Boss-reran
independently); the 3 loop/recur e2e explicitly green. All three
components shipped: the loop/recur milestone is structurally
complete. Milestone-close audit + fieldtest is the next step.
2026-05-17 23:57:46 +02:00
Brummel a179ec30a0 iter loop-recur.1: additive Term::Loop / Term::Recur / LoopBinder foundation
First of three iterations of the standalone loop/recur milestone
(spec 97c1ed1). loop/recur become real parseable / printable /
round-trippable / hash-stable strictly-additive AST nodes across
every no-wildcard exhaustive Term match in all six crates, plus
parse/print, prose arms, DESIGN.md + form_a.md schema blocks,
drift/coverage/hash anchors, and the spec's worked sum_to program
as a green round-trip fixture. NO typecheck semantics and NO real
codegen this iter by design: synth and lower_term are stubbed
CheckError::Internal / CodegenError::Internal exactly as mut.1
(real typecheck = iter 2, real loop-header/phi/back-edge = iter 3).

Two binding Boss design calls recorded in the plan header and the
journal: (1) verify_tail_positions "byte-unchanged" = its tail-app
verification role is unchanged, not its source — it is an
exhaustive no-wildcard match so two descent-only arms are
mandatory; acceptance evidence is the tail-app non-regression test.
(2) codegen is in iter-1 scope as stubs/pass-throughs because the
workspace must compile for cargo test --workspace.

Three plan-pseudo-vs-reality substitutions (serde Type::Con
literal, bare Int -> (con Int), unqualified LoopBinder) and one
recon-undercount integration-test site, all intent-preserving and
journalled. Boss forward-fix folded in: the committed specs
themselves wrote bare Int in the loop-binder snippets (parses as
Type::Var) — corrected the three headline snippets to (con Int) for
doc-honesty (no design/schema change).

cargo test --workspace 600 -> 608 / 0 red (Boss-reran
independently); iter13a + new loop_recur hash pins green.
2026-05-17 23:00:15 +02:00
Brummel 37ac704bf3 iter revert: back out the Iteration-discipline milestone (it.1 + it.2)
One forward iteration; main never rewound. 1ff7e81 (the pre-9973546
commit) is the per-region byte oracle — every reverted source/test
file is byte-identical to it; crates/ailang-check/src/lib.rs fully so.

Root cause being corrected: the Iteration-discipline milestone was an
over-escalation of fieldtest finding F1 (a [friction] item whose own
minimal recommendation was a DESIGN.md note). Its totality dichotomy
made the maximally-LLM-natural build(d:Int)=Node(1,build(d-1),
build(d-1)) inexpressible (it.3 BLOCKED); the only in-thesis escape
(A1/it.2b) conceded the language's first documented-unenforced
totality precondition — a purity-pillar dilution the user rejected in
favour of a full revert + rebuild.

Removed: Term::Loop/Term::Recur/LoopBinder; the verify_structural_
recursion guardedness pass + term_contains_loop + Diverge-injection +
the transitively-it.2 module_fns plumbing; the five Recur*/
NonStructuralRecursion CheckError variants (+ code() + the 3 dedicated
ctx() arms); the it.1 codegen loop-header/phi/back-edge + parallel
block_terminated setter; all Loop/Recur walker arms; 16 it.1/it.2
fixtures; 2 pin files; bench/it3-oracle/. Restored: 2 RC fixtures to
1ff7e81 content.

Surgically kept (not in 1ff7e81, landed with the milestone but
independently sound): feature-acceptance clause 3 in DESIGN.md and
skills/brainstorm/SKILL.md, with its worked example de-claimed from
"shipped" to hypothetical-illustration form; the F3 P2 todo.
bench/orchestrator-stats/2026-05-15-iter-it.{1,2,3}.json kept as
historical record (like journals/plans).

Sole net addition: an honest F1/F4 documented-idiom note in DESIGN.md
(the tail-recursive accumulator fallback; examples/mut_counter.ail),
guarded by a doc-presence test — "a documentation note is not a
reshape", asserts nothing at the typecheck level.

Roadmap: the Iteration-discipline block + blocking-fork section
removed; the genuine total-Int-recursion ambition preserved as a
deferred P2 milestone sequenced behind a future Nat/refinement-types
milestone (not abandoned — correctly sequenced after the type
machinery it needs). 2026-05-15-iteration-discipline.md carries a
superseded header; it.1/it.2/it.3 journals + plans stay as history.

Correctness gate PRISTINE: 164 surviving 1ff7e81-era fixtures
ail check/ail run byte-identical to pre-milestone behaviour (verified
against a 1ff7e81 worktree reference compiler, zero drift);
cargo test --workspace 600/0; zero residual it.1/it.2 production
surface.

Spec docs/specs/2026-05-16-iteration-discipline-revert.md (b3853bf),
plan docs/plans/2026-05-16-iter-revert.md (abf0013).
2026-05-16 01:28:47 +02:00
Brummel 96db54d15d iter it.1: loop/recur additive — Term::Loop/Term::Recur/LoopBinder end-to-end
Iteration-discipline milestone, 1 of 3. Adds named loop + recur as
strictly-additive first-class AST nodes: parse/print/prose/serde/
round-trip/schema lockstep, typecheck (binder typing + recur
arity/type unification via loop_stack threaded as mut.2's
mut_scope_stack; recur-tail-position via verify_loop_body), codegen
(loop-header + one phi per binder; recur back-edge br with a NEW
parallel block_terminated setter; lambda-boundary loop_frames
save/restore). Four Recur* CheckError variants. Strictly additive:
zero deletions touch tail-app/tail-do or the seven existing
block_terminated sites — this is what makes the destructive it.3
safe. recur synth = fresh metavar (resolves the plan's flagged
Type::unit() risk). loop_counter->55, loop_in_lambda->49, four
negatives fire, tail-app byte-identical, cargo test --workspace
green. Spec fda9b78, plan 7381a42.
2026-05-15 14:38:55 +02:00
Brummel 20add51112 iter mut.4-tidy + audit close — mut-local milestone closed
Tidy iter addressing the audit-mut-local drift. Plus paired baseline
update on bench/baseline_compile.json (audit-skill discipline).

Architect [high] items closed:

1. CheckError::MutVarCapturedByLambda rejects lambdas whose body
   free vars include a mut-var of the enclosing mut_scope_stack.
   Uses the existing ailang_core::desugar::free_vars_in_term walker
   (which honours Term::Match pattern bindings). The scan runs only
   when mut_scope_stack is non-empty.

2. crates/ailang-codegen/src/lambda.rs: the capture-not-in-locals
   path that previously raised CodegenError::Internal blaming the
   typechecker now uses unreachable!. The companion comment block
   was rewritten to state the current reality.

Architect [medium] items closed (stale mut.1-stub history comments):

3. docs/DESIGN.md §'Term (expression)' mut/assign block: the
   trailing paragraph describing the iter mut.1 stub state was
   replaced with one describing the current mut.3-end-state. The
   inline jsonc comment on {'t': 'assign'} was updated to drop the
   'deferred to mut.2/mut.3' language.

4. crates/ailang-codegen/src/lib.rs: the stale comment block above
   the real Term::Mut arm describing the mut.1 stub was removed
   entirely.

Other:

- Short-circuit on empty mut_scope_stack in synth's Term::Var arm:
  the iter mut.2 prepend now skips the iter-and-find walk when
  the stack is empty, eliminating any per-Var-resolution overhead
  for the common case (programs with no mut blocks). The
  short-circuit did NOT close the check_ms regression — see ratify
  below.

- Spec docs/specs/2026-05-15-mut-local.md §'Out of scope' amended
  with the lambda-capture rejection bullet.

- Negative fixture examples/test_mut_var_captured_by_lambda.ail.json
  + driver test extension in crates/ailang-check/tests/
  mut_typecheck_pin.rs (6th test) +
  crates/ailang-core/tests/carve_out_inventory.rs EXPECTED bumped
  12 → 13.

Bench-regression ratify:

bench/compile_check.py check_ms showed a uniform ~30-50% relative
shift across the entire 11-fixture suite (~0.5ms absolute on a
1.4-1.5ms baseline). The uniformity across fixtures of very
different Var counts argues for a fixed-cost-per-invocation tax,
not a Var-proportional hot path. The Term::Var short-circuit
falsified the hot-path hypothesis. The plausible remaining causes
(synth-parameter-passing through ~19 recursive sites, and binary-
size startup tax from mut-* adding ~1400 LOC to typecheck/codegen)
are both feature-cost, not pathological. Ratified by paired
journal entry; bench/baseline_compile.json updated to the post-
mut-local numbers via 'bench/compile_check.py --update-baseline'.

bench/check.py continues to show the established tail-latency-noise
envelope from audit-pd (2026-05-14) — no separate ratify needed.

bench/cross_lang.py clean.

Tests: 594 → 598 green (4 new lib.rs mod tests + 1 driver test +
1 fixture-corpus uptake).

Journal: docs/journals/2026-05-15-iter-mut.4-tidy.md.

mut-local milestone end-to-end status:
  - mut.1 (7b92719): AST + Form A surface.
  - mut.2 (b24718a): typecheck.
  - mut.3 (03fb633): codegen + e2e.
  - mut.4-tidy (this commit): audit-drift close + bench ratify.
mut-local milestone CLOSED.

Refs: docs/specs/2026-05-15-mut-local.md,
docs/plans/2026-05-15-iter-mut.4-tidy.md.
2026-05-15 09:21:33 +02:00
Brummel 03fb633d85 iter mut.3: codegen + e2e — mut-local milestone end-to-end closed
Replaces the iter mut.1 dispatch stubs in lower_term with real LLVM
lowering: mut-vars become alloca slots hoisted to the fn's entry
block, assign lowers to store, mut-var reads lower to load. Two
example fixtures run end-to-end printing 55 (Int via mut_counter,
Float via mut_sum_floats).

Concretely:

- Emitter struct gains three new fields: mut_var_allocas:
  BTreeMap<String, (String, Type)> for per-fn name→(alloca SSA,
  AIL type) tracking; pending_entry_allocas: String as a side
  buffer accumulating alloca instructions during body lowering;
  entry_block_end_marker: Option<usize> recording the byte position
  in self.body immediately after the entry: label.

- start_block(label) captures the marker when label == 'entry'.
  emit_fn resets all three new fields per fn body. At the end of
  emit_fn, String::insert_str splices pending_entry_allocas into
  self.body at the marker — alloca instructions land in the entry
  block regardless of where in the source tree Term::Mut was
  encountered (mem2reg eligibility preserved).

- Term::Mut arm: per var, fresh-name an alloca SSA, push the
  alloca instruction into pending_entry_allocas, lower the init at
  the current body position, emit a store to bind. The mut-var
  binding is NOT visible during init (matches typecheck-side
  ordering at lib.rs:3576). After all vars are bound, lower the
  body in the extended scope. On block exit, restore any prior
  bindings via a per-block save stack — supports nested
  shadowing correctly.

- Term::Assign arm: look up the alloca + type via
  mut_var_allocas.get(name); lower the value; emit store; yield
  the canonical Unit SSA per the existing Term::Lit { lit:
  Literal::Unit } convention.

- Term::Var arm: prepended with a mut-var lookup that emits a
  load <ty>, ptr <alloca> and returns the load SSA. Innermost-wins
  shadowing falls out of the BTreeMap's insert-overwrites
  semantics combined with the per-block save/restore.

- examples/mut_counter.ail + examples/mut_sum_floats.ail:
  recursive sum_helper accumulates 1..10 (Int) or 1.0..10.0 (Float)
  inside a mut block whose single var is assigned the helper's
  result. Both run end-to-end printing 55.

- crates/ail/tests/e2e.rs: mut_counter_prints_55 and
  mut_sum_floats_prints_55 #[test] fns using the existing
  build_and_run helper.

- DESIGN.md 'What is supported' subsection: 'Local mutable state'
  bullet describes the new construct, points at the two example
  fixtures, and reaffirms the seal-by-construction invariant under
  Decision 10.

Beyond-plan adjustments absorbed in this iter:
  - synth_with_extras's parallel Term::Var arm in
    ailang-codegen/src/lib.rs needed the same mut-var lookup as
    lower_term's. Plan only specified lower_term's arm.
  - crates/ailang-codegen/src/lambda.rs needed save/restore of all
    three new Emitter fields across the lambda-body boundary plus
    in-thunk splice of pending_entry_allocas at the lambda's own
    entry marker, so mut-blocks inside a closure body hoist into
    the closure's entry block (not the outer fn's).

mut-local milestone end-to-end status:
  - mut.1 (7b92719): AST + Form A surface.
  - mut.2 (b24718a): typecheck.
  - mut.3 (this commit): codegen + e2e.
  Audit follows.

Tests: 592 → 594 green; cargo build green; both fixtures execute
and print 55.

Journal: docs/journals/2026-05-15-iter-mut.3.md.

Refs: docs/specs/2026-05-15-mut-local.md, docs/plans/2026-05-15-iter-mut.3.md.
2026-05-15 01:56:26 +02:00
Brummel 7b92719244 iter mut.1: AST extension + Form A surface for local mutable state
First iteration of the mut-local milestone (foundation step on the
Stateful-islands roadmap path). Lands the schema + surface tier:
Term::Mut, Term::Assign, and the nested MutVar struct become
first-class AST nodes that round-trip cleanly through Form A.
Typecheck and codegen recognition are deferred to mut.2 and mut.3
per the spec's out-of-iteration boundary; reaching either dispatch
entry point with these variants produces CheckError::Internal /
CodegenError::Internal with a 'deferred to iter mut.{2,3}' message.

Concretely:

- crates/ailang-core/src/ast.rs: two new Term variants behind
  #[serde(tag = 't')]; pub struct MutVar { name, ty, init } adjacent
  to Arm. Two canonical-bytes pin tests for the explicit-empty-vars
  serialisation and the assign round-trip.

- ~25 substantive Term-walker arms across ailang-core/desugar,
  ailang-core/workspace, ailang-check (lib + lift + linearity + mono
  + pre_desugar_validation + reuse_shape + uniqueness),
  ailang-codegen (escape + lambda + lib), ailang-prose, and
  crates/ail/src/main.rs. Universal policy: substantive recurse-into-
  children at every site; only the two dispatch entry points
  (synth in ailang-check, lower_term in ailang-codegen) stub with
  Internal-error. One test-side walker arm in
  crates/ail/tests/codegen_import_map_fallback_pin.rs not
  enumerated by the plan was added as well (defensive recursion).

- ailang-surface: parse_mut + parse_assign helpers; Term::Mut
  body desugared from a flat statement sequence into a right-folded
  Term::Seq chain inside the JSON-AST. Print arms in print.rs match
  the parser convention. EBNF prologue + crates/ailang-core/specs/
  form_a.md productions updated. Four new parser pin tests cover
  the empty-mut, single-var, body-required, and vars-only-no-body
  cases.

- Drift + coverage tests extended: design_schema_drift.rs adds two
  exemplars + match arms; schema_coverage.rs adds two VariantTag
  entries + EXPECTED_VARIANTS + visit_term arms; spec_drift.rs adds
  two exemplars + match arms. DESIGN.md §'Term (expression)' gets
  jsonc-blocked schemas for the two new variants.

- examples/mut.ail: six-fn round-trip fixture exercising empty mut,
  single-var, two-var, nested-shadow, and the four supported scalar
  return types (Int, Float, Bool, Unit). The round_trip auto-glob
  and schema_coverage corpus walker both pick it up.

Plan deviation: the plan named lib.rs:2572 as the typecheck
dispatch stub site, but that line is actually verify_tail_positions
(substantive walker). The real dispatch is synth (3403-area, stub
at 3489); the orchestrator routed correctly.

Tests: 564 → 579 green; cargo build green; round-trip green for
the new fixture; all drift + coverage tests green.

Journal: docs/journals/2026-05-15-iter-mut.1.md.

Refs: docs/specs/2026-05-15-mut-local.md, docs/plans/2026-05-15-iter-mut.1.md.
2026-05-15 01:10:56 +02:00
Brummel bea5c92591 codegen tidy: extract lambda lowering into lambda.rs
Fourth slice of the codegen split. Pulls the closure-emission
cluster into a dedicated module. No behaviour change.

Methods moved:
  - lower_lambda (~290 lines, pub(crate)): capture analysis +
    thunk fn lifting + heap-env construction + closure-pair
    packing + per-pair drop-fn emission under --alloc=rc.
  - collect_captures (~108 lines, private): recursive free-var
    walk used only as lower_lambda's prelude.
  - pattern_bound_names (~17 lines, private): pattern-binder
    enumeration used by collect_captures's match arm.

Imports: super::escape (for analyze_fn_body), super::synth
(for fn_sig_from_type, llvm_type), the usual Emitter/Result
re-exports.

lib.rs drops from 3237 → 2825 lines. lambda.rs is 447 lines.

Codegen split summary (5295 → 2825 lib.rs, ~47% reduction):
  lib.rs        2825 — Emitter struct, lifecycle (new,
                       start_block, emit_module, emit_*),
                       lower_term mass dispatcher, app/eq/effect
                       lowering, lookup helpers, synth helpers.
  match_lower.rs 935 — Term::Ctor / ReuseAs / Match.
  escape.rs      722 — escape analysis (predates the split).
  drop.rs        696 — per-type drop fns + per-let-close drop.
  lambda.rs      447 — Term::Lam + closure machinery.
  subst.rs       319 — monomorphisation substitution.
  synth.rs       216 — IR shaping helpers + builtin types.

cargo test --workspace clean across all four phases.
2026-05-08 15:54:20 +02:00