diff --git a/CLAUDE.md b/CLAUDE.md index 18214f3..aad3181 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,7 +43,7 @@ it measurably improves correctness or removes redundancy. | `bench/` | Regression harnesses (`check.py`, `compile_check.py`, `cross_lang.py`) and the throughput-and-latency runner (`run.sh`); `bench/reference/` holds the hand-C corpus for cross-language ratios | | `examples/` | AILang fixtures used by tests and benches | | `design/` | The canonical contract ledger — `design/INDEX.md` (sole addressable spine: a typed Contracts + Models table), `design/contracts/` (prose-authoritative test-linked invariants), `design/models/` (onboarding whitepapers) | -| `docs/` | Specs and roadmap — `docs/specs/` (per-milestone design specs), `docs/plans/` (per-iteration plans), `docs/roadmap.md` (forward queue), `docs/journal-archive.md` (pre-2026-05-11 history, content-frozen), `PROSE_ROUNDTRIP.md`. Per-iter history lives in `git log`. | +| `docs/` | Specs and roadmap — `docs/specs/` (per-milestone design specs), `docs/plans/` (per-iteration plans), `docs/roadmap.md` (forward queue), `PROSE_ROUNDTRIP.md`. Project history lives in `git log`. | | `skills/` | Project-local skill definitions and their agents. See `skills/README.md` for the skill table, agent roster, and discovery layout. | ## Skill system @@ -193,12 +193,11 @@ Work clusters into **milestones**, each subdivided into → fieldtest`), skipping rules, and bench-exit-code gating live in `skills/README.md` and the per-skill `SKILL.md` files. -Vocabulary note: legacy entries in `docs/journal-archive.md` -(pre-2026-05-09) use "iter" / "family"; current vocabulary is -"iteration" / "milestone". Existing archive entries are not -retroactively renamed. +Vocabulary note: pre-2026-05-09 git history uses "iter" / "family"; +current vocabulary is "iteration" / "milestone". Old commits are +not retroactively renamed. -## Roles of the `design/` ledger, `git log`, `docs/journal-archive.md`, `docs/roadmap.md`, `docs/specs/`, `docs/plans/` +## Roles of the `design/` ledger, `git log`, `docs/roadmap.md`, `docs/specs/`, `docs/plans/` - **The `design/` ledger** is the canonical specification. It describes what AILang *is*: schema, semantics, invariants, runtime @@ -222,10 +221,6 @@ retroactively renamed. with `git log --oneline -30`; per-milestone scope with `git log ..HEAD --format=full`. -- **`docs/journal-archive.md`** is the archived monolithic - decisions log for everything pre-2026-05-11. Content-frozen. - Read it only when chasing long-tail history; do not append. - - **`docs/roadmap.md`** (since 2026-05-10): the priority-ordered forward queue — milestones, features, todos, and ideas. The orchestrator owns this file and is responsible for keeping it @@ -243,6 +238,5 @@ retroactively renamed. by `skills/implement`. Together these answer three questions: "what is the language right -now?" (the `design/` ledger), "how did we get here?" (`git log`, -plus `docs/journal-archive.md` for pre-2026-05-11 context), and -"what's next?" (roadmap). +now?" (the `design/` ledger), "how did we get here?" (`git log`), +and "what's next?" (roadmap). diff --git a/crates/ailang-check/tests/duplicate_ctor_pin.rs b/crates/ailang-check/tests/duplicate_ctor_pin.rs index 9793e01..7116e12 100644 --- a/crates/ailang-check/tests/duplicate_ctor_pin.rs +++ b/crates/ailang-check/tests/duplicate_ctor_pin.rs @@ -3,11 +3,13 @@ //! consumer of the per-module `env.ctor_index` rebuild in //! `check_in_workspace`. Env construction is code-SoT (the //! `env-construction` ledger row in design/INDEX.md is source-link -//! only); the why-two-overlays rationale this test rests on as the -//! asymmetry-with-mono-side argument lives in -//! docs/journals/2026-05-19-design-decision-records.md. If the -//! per-module rebuild is ever removed without an equivalent -//! replacement path, this test goes red. +//! only). The two overlays — typecheck-side rebuild + mono-side +//! narrowed — are deliberately asymmetric: the typecheck-side +//! `ctor_index` half exists *for this very diagnostic*, while the +//! mono-side overlay is narrowed to types-only because the runtime +//! ctor lookup is type-driven. If the per-module rebuild is ever +//! removed without an equivalent replacement path, this test goes +//! red. use ailang_check::check_workspace; use ailang_core::ast::{Ctor, Def, Module, TypeDef}; diff --git a/crates/ailang-core/tests/docs_honesty_pin.rs b/crates/ailang-core/tests/docs_honesty_pin.rs index 79dd682..c0e6e2e 100644 --- a/crates/ailang-core/tests/docs_honesty_pin.rs +++ b/crates/ailang-core/tests/docs_honesty_pin.rs @@ -91,21 +91,16 @@ fn design_md_has_no_doc_archaeology() { #[test] fn design_md_present_tense_anchors_present() { - // Each anchor is read from its post-split home (spec Appendix - // relocation map). Three honest-reserved rationale anchors - // ("Regions were considered and rejected", the "does not infer - // everything" bullet, the Form-B prose-placeholder state) sat in - // `### What this Decision deliberately does not do` / `### What - // this decision does NOT commit to`, which the Appendix routes to - // the decision-record journal — so the journal is part of the - // canonical anchor corpus, not a contract surface. + // Each anchor is read from its post-split home. Phrases that only + // ever lived in the (now-removed) decision-records annex are + // dropped — anything the design/ ledger truly needs lives in + // design/contracts/ or design/models/. let honesty = norm(&read("design/contracts/honesty-rule.md")); let scope = norm(&read("design/contracts/scope-boundaries.md")); let memory = norm(&read("design/contracts/memory-model.md")); let pipeline = norm(&read("design/models/pipeline.md")); let str_abi = norm(&read("design/contracts/str-abi.md")); let prelude_classes = norm(&read("design/contracts/prelude-classes.md")); - let records = norm(&read("docs/journals/2026-05-19-design-decision-records.md")); // the discriminator meta-subsection -> contracts/honesty-rule.md assert!(honesty.contains("the honesty rule it holds itself to"), @@ -115,8 +110,6 @@ fn design_md_present_tense_anchors_present() { // protected honest-reserved exceptions — over-correction guard assert!(scope.contains("`Diverge` is a reserved effect name with no op and no codegen"), "the gold-form honest-reserved anchor must remain in scope-boundaries.md (do not over-strip)"); - assert!(records.contains("Regions were considered and rejected"), - "the present-tense design-rationale exclusion must remain in the decision-records (do not over-strip)"); assert!(memory.contains("a tiebreaker, not a rationale"), "the self-labelled tiebreaker is honest and stays in memory-model.md (do not over-strip)"); // corrected present-tense anchors @@ -124,12 +117,8 @@ fn design_md_present_tense_anchors_present() { "models/pipeline.md must describe Boehm present-tense, not as 'on the path to retirement'"); assert!(str_abi.contains("type-installed; codegen is reserved and not yet shipped"), "float_to_str must be present-tense honest-reserved in str-abi.md"); - assert!(records.contains("AILang demands annotations *because* the LLM author can produce them effortlessly"), - "the 'does not infer everything' bullet must be present-tense in the decision-records (no doc-archaeology)"); assert!(prelude_classes.contains("`io/print_str` is the only built-in direct-output effect-op"), "the print-op set must be stated present-tense in prelude-classes.md (post-split home), not as a retirement narrative"); - assert!(records.contains("the render in `crates/ailang-prose/src/lib.rs` is a placeholder and informational only"), - "the Form-B class/instance prose-projection state must be present-tense in the decision-records"); } #[test] diff --git a/crates/ailang-surface/src/parse.rs b/crates/ailang-surface/src/parse.rs index 3384eea..14dda86 100644 --- a/crates/ailang-surface/src/parse.rs +++ b/crates/ailang-surface/src/parse.rs @@ -76,13 +76,11 @@ //! ``` //! //! Notes on the form (the binding constraints are in -//! design/contracts/authoring-surface.md; the form-refinement -//! rationale is in docs/journals/2026-05-19-design-decision-records.md): +//! design/contracts/authoring-surface.md): //! //! - The `lam` form carries `paramTypes`, a `ret` type, and an -//! optional `effects` clause. The early form sketch left -//! these out; the AST stores them per-lambda and so the form must -//! round-trip them. +//! optional `effects` clause. The AST stores them per-lambda and +//! so the form must round-trip them. //! - The `import` form admits an optional `as` alias to round-trip //! [`ailang_core::ast::Import::alias`]. diff --git a/design/INDEX.md b/design/INDEX.md index 41fa3c8..b4dca43 100644 --- a/design/INDEX.md +++ b/design/INDEX.md @@ -56,11 +56,9 @@ evolving in lockstep with the language: sole addressable spine for canonical state), `design/contracts/` (test-linked invariants), `design/models/` (onboarding whitepapers). -- **Docs** (`docs/`): `docs/journal-archive.md` (archived monolith - for pre-2026-05-11 history, content-frozen), `roadmap.md` (forward - queue), `specs/` (per-milestone design specs), `plans/` (per- - iteration implementation plans). Current iter / audit history - lives in `git log`. +- **Docs** (`docs/`): `roadmap.md` (forward queue), `specs/` + (per-milestone design specs), `plans/` (per-iteration + implementation plans). Project history lives in `git log`. - **Tests**: unit tests per crate plus E2E in `crates/ail/tests/e2e.rs`. Every new compiler path needs a test, otherwise the feature does not count as done. diff --git a/docs/journal-archive.md b/docs/journal-archive.md deleted file mode 100644 index 6291327..0000000 --- a/docs/journal-archive.md +++ /dev/null @@ -1,14578 +0,0 @@ -# JOURNAL - -> **Status:** archived 2026-05-11. Content-frozen. This file is the -> historical record for entries prior to that date. Current iter and -> audit history lives in `git log` (commit bodies). - -Chronological notes for myself. Not every change; only decisions, obstacles, -and observations that future iterations will need. - -## 2026-05-07 — Day 0 - -- Repo initialised. Assignment in `CLAUDE.md`: LLM-native language, LLVM backend. -- Design decisions captured in `docs/DESIGN.md`. -- Toolchain: `rustc 1.94`, `llvm-config 22.1.3`, `clang` available. -- Decided against `inkwell` in favour of LLVM IR text emit. Rationale in DESIGN.md. -- Workspace layout: - - `crates/ailang-core` — AST, type, hash, JSON schema - - `crates/ailang-check` — typechecker (comes later) - - `crates/ailang-codegen` — lowering + LLVM IR emit - - `crates/ail` — CLI -- MVP goal: `examples/sum.ail.json` → binary that prints 55. **Achieved.** - -## 2026-05-07 — architecture review after the MVP - -Still on track? Broadly yes. Concrete observations: - -**What holds:** - -- JSON AST + canonical form + content hash are all lego bricks that later - tools can build on without a refactor (`ail deps`, `ail diff`). -- The LLVM IR text pipeline works as planned. No libllvm version pain. -- The effect set is wired into the type system from the start. Extensible to - row-poly without touching the core. - -**Debt that accrues interest:** - -1. **`current_block_label_for_phi` is a heuristic** (see codegen). On nested - `if` terms it will return the wrong block label, because it scans the body - backwards. Ticking, because no test cases trigger it yet. Must be fixed - next, before new language features arrive. -2. **No typed AST.** Codegen reads the source AST directly and relies on - the typechecker having run before. Fine for the MVP; once ADTs or - closures arrive, I will need a separate typed IR stage (TIR). -3. **The `hash` field is not in the AST.** Right now we hash the def object - directly. Once I serialise hashes as fields (caching), the hash will - need to exclude that field before computation. - -**Plan iteration 2 (now):** - -1. Clean up block-label tracking, with a nested-if test. -2. Strings as a literal + `io/print_str`. -3. Hello-world example as a second E2E test. -4. CLI: `--json` output for machine consumers wherever it fits. - -**Plan iteration 3:** - -ADTs + pattern matching. That is the next big jump. Requires a typed IR -stage (TIR), because pattern matching lowers into decision trees, which -have a different shape from the AST. - -## 2026-05-07 — iteration 2 done - -- Block-label tracking is now robust (nested `if`s work). Test - `max3_picks_largest` protects it. -- Strings as `Lit::Str { value }`, type `Str` -> LLVM `ptr`, with - `io/print_str` effect op. `examples/hello.ail.json` prints a string. -- CLI: `manifest --json`, `builtins --json` for tool consumers. -- `ail deps [--of NAME] [--json]` lists call edges. Effect ops are tagged - `effect:NAME` so a consumer can filter them. - -**Architecture check:** no structural deviations. Codegen still reads the -source AST directly (a TIR stage will become necessary with ADTs in -iteration 3). - -## 2026-05-07 — iteration 3 done: ADTs - -- TypeDef in the AST with ctors. A ctor has `name` and `fields: [Type...]`. -- Term::Ctor (construction) and Term::Match (pattern matching). -- Patterns: `Wild`, `Var`, `Lit`, `Ctor { ctor, fields }`. In the MVP, - nested ctor patterns are NOT allowed — sub-patterns must be `Var` or - `Wild`. -- Typechecker with a type registry and `ctor_index` (ctor name → ADT). In - Match, exhaustiveness is checked against the full constructor set. A - negative test protects this. -- Codegen: boxed heap layout. Per ctor application, `malloc(8 + 8*n)` - bytes; tag at offset 0, fields from offset 8 (8-byte slots, native typed - load/store). Match: load tag + switch + arm blocks + phi at the join. -- `examples/list.ail.json` (Cons/Nil list, sum_list via match) returns 42. - -**Surprisingly painless.** The architecture decisions from day 0 paid off: -opaque ptr in LLVM 22 makes the boxed layout almost glue-free; effect -tracking was untouched by ADTs; the JSON AST takes new node types -cleanly. - -**Debt accrued:** - -1. **Codegen still reads the source AST directly.** The temptation to push - on without TIR was strong — and worked, because my Match restrictions - are flat (no nested patterns). Once nested patterns arrive, decision- - tree lowering will not stay clean without TIR. Debt acknowledged; not - due now. -2. **No GC.** The heap leaks. Acceptable for demo programs; must be - addressed before any longer-running program. Options for Phase 4: - refcount, Boehm-GC linkage, region inference. -3. **No runtime pretty-printer for ADT values.** `io/print_int` is enough - for demos, but a generic `show :: a -> Str` for ADTs would be valuable. - Requires dispatch over the tag — feasible, but not now. - -**Plan iteration 4:** - -The next steps are less obvious. Three candidates in priority order: - -1. **Module system (imports).** Right now everything is in a single module. - With multiple modules + cross-module hashing the language only becomes - practical for several defs. -2. **Structured error output (`ail check --json`).** So tools can react to - type errors without parsing text. -3. **Closures / higher-order functions.** Requires closure conversion and - is a bigger step. - -Iteration 4 will be (1) + (2) — both strengthen the LLM tooling and have -moderate risk. - -## 2026-05-07 — workflow change: orchestrator + agent repo - -At the user's suggestion, switching to **orchestrator mode**: I delegate -clearly bounded implementation chunks to sub-agents and keep only -architecture decisions, reviews, and commit discipline. Four specialised -agents drafted: implementer, architect, tester, debugger. - -**Important correction:** the user required the agents not to be hidden in -`.claude/agents/`, but versioned as a visible part of the project under -`agents/`. DESIGN.md gained a new section "Project ecosystem", which -records this: AILang is not just a language, but language core + CLI + -examples + agents + docs + tests, all of equal weight. - -Invocation scheme: the system-prompt body from `agents/.md` as a -prefix before the concrete task + sent to the `general-purpose` agent. -Functionally identical to subagent loading from `.claude/agents/`, but -visible in the repo. - -**Plan iteration 4 (revised):** - -The module system is more involved than expected (cross-module hashing, -import resolution). First the smaller tooling wins, then the module system -as iteration 5: - -1. **Structured error output** (`ail check --json` with a Diagnostic - struct, stable codes like `unbound-var`, `type-mismatch`). -2. **`ail diff `** — semantic module diff via per-def hash - comparison. -3. **IR snapshot tests** — regression protection for the codegen pipeline. - -## 2026-05-07 — iteration 4 done: LLM tooling consolidation - -Three sub-commits, each produced by an `ailang-implementer` invocation -and spot-checked by the orchestrator: - -- `93fe723` Iter 4a: `ail check --json` with a `Diagnostic` struct - (`severity`, `code`, `message`, `def`, `ctx`). Stable codes: - `unbound-var`, `type-mismatch`, `arity-mismatch`, - `non-exhaustive-match`, `unknown-ctor`, `unknown-ctor-in-pattern`, - `nested-ctor-pattern-not-allowed`, `duplicate-def`, - `unknown-effect-op`, `unknown-type`, `schema-mismatch`. API: - `check_module(&Module) -> Vec`. -- `c652b12` Iter 4b: `ail diff [--json]` as a structural top-level - def diff via BLAKE3 hash. Four categories (added/removed/changed/ - unchanged), sorted alphabetically, exit code 1 on diff. -- `74a2005` Iter 4c: IR snapshot tests in - `crates/ail/tests/snapshots/{sum,max3,hello,list}.ll`. - Normalisation of `target triple`. Update via `UPDATE_SNAPSHOTS=1 - cargo test ir_snapshot_`. Mismatch produces an `.actual` file. - -Test count: 28 (previously 19). 7 E2E + 4 IR snapshot + 9 ailang-check + 1 -ailang-codegen + 7 ailang-core. - -**Closed from the debt register:** - -- Block tracking in codegen has not been a heuristic risk since Iter 2; - the explicit `current_block: String` track is now additionally - protected against regression by Iter 4c snapshot tests. Debt closed. - -**New / sharpened debt:** - -1. `check_module` is **single-shot** — the first error aborts, no - multi-diagnostic gathering. The spec was that way, but the format - suggests Vec semantics. A real multi-diagnostic refactor will be - cheaper once TIR exists (a central error accumulator via a separate - stage). Not due now. -2. `source_filename` in the IR is hard-coded to `".ail"`. As long - as there is only one top-level module, that is platform-stable. With - Iter 5 (module system + imports) the path becomes relevant — keep it - path-independent at construction time, otherwise the snapshots will - tip over. - -**Plan iteration 5:** module system with imports. Cross-module hashing, -import resolution, multiple `.ail.json` files in one build. The multi- -diagnostic refactor only after that. - -Sub-steps: - -- **5a — workspace loader.** `ailang_core::Workspace { modules: - BTreeMap }` plus `load_workspace(entry: &Path)`, which - follows `imports` recursively from the entry module. Convention: - `import { module: "foo" }` resolves to `/foo.ail.json` next to the - entry. Cycle detection. CLI: existing subcommands keep working on a - single module; a new `ail workspace ` lists all reachable - modules with hash. Tests: two small example modules with an import - relation; cycle test. -- **5b — cross-module typecheck.** The typechecker takes `&Workspace` - instead of `&Module`. Imports are mounted in the env as a namespace - (`alias.def` or, with no alias, `module.def`). New diagnostic codes: - `unknown-module`, `unknown-import`, `import-cycle`, `ambiguous-name`. - Tests per code. -- **5c — cross-module codegen.** The emitter produces IR for all modules - in the workspace, prefix-mangled with `@ail__`. E2E test: - a program that uses a function from module B in module A returns the - correct result in the binary. -- **5d — tooling adjustments.** `manifest`, `describe`, `deps`, `diff` - gain a `--workspace` mode (recursive). The single mode stays the - default for backwards compatibility. - -During Iter 5, at construction time **keep `source_filename` -path-independent** (module name only, no directory prefix), otherwise -the IR snapshots will tip over. - -## 2026-05-07 — Iter 5b done: cross-module typecheck - -- `check_workspace(&Workspace) -> Vec` as the top-level API. - `check_module` is preserved and internally lifts the module into a - trivial workspace. -- Convention for qualified references (recorded in DESIGN.md): - `Term::Var { name }` with exactly one dot = `.`. Prefix is - an import alias or module name. No new AST node, no renamed fields ⇒ - hashes stay stable; all `ir_snapshot_*` still green. -- Three new diagnostic codes: `unknown-module`, `unknown-import`, - `invalid-def-name` (with `ctx.reason: "contains-dot"`). -- CLI: `ail check ` now **always** loads via `load_workspace`. - Workspace load failures become structured diagnostics in JSON mode with - codes `module-not-found`, `module-cycle`, `module-name-mismatch`, - `module-hash-mismatch`, `schema-mismatch`. `ail build` and - `ail emit-ir` stay per single module (cross-module codegen is 5c). -- Examples: `ws_main.ail.json` now calls `ws_lib.add` (observable). New: - `ws_broken.ail.json` (`unknown-import`), - `ws_unknown_module.ail.json` (`unknown-module`). -- Tests: 37 green (previously 32). 4 new workspace integration tests in - `crates/ailang-check/tests/workspace.rs`, one new e2e test - `check_workspace_resolves_import`. -- Debt: single-shot diagnostics still in place (multi-diagnostic after - 5c). The dot convention covers exactly one dot — nested module paths - (`a.b.c`) do not exist; that would only be a topic with hierarchical - modules and currently falls through as `unbound-var`. - -## 2026-05-07 — Iter 5c done: cross-module codegen - -- **Mangling break (deliberate).** All AILang functions are now called - `@ail__`, even in single-module programs. The old form - `@ail_` is gone. Strings/const globals analogously - (`@.str___`, `@ail__`). The entry - point stays `main` as C ABI: a `define i32 @main()` trampoline calls - `@ail__main()`. If the entry module has no - `main : () -> Unit !IO`, the build fails with `MissingEntryMain`. -- **Workspace lowering.** New top-level API - `ailang_codegen::lower_workspace(ws: &Workspace) -> Result` - produces a single `.ll` for the whole workspace. Modules in - alphabetical order (BTreeMap order); defs in AST order. Cross-module - calls are resolved in codegen via the import map of the calling module - — same logic as in the typechecker, locally duplicated with a - cross-reference (no shared helper module, because the type worlds - differ: the typechecker handles `Type`, codegen handles `FnSig` from - llvm types). -- **CLI.** `ail build` and `ail emit-ir` now always load the workspace - and check/lower it fully. Single-module programs keep working - (trivial workspace with one module). `emit_ir(m)` stays in the codegen - crate as a convenience API and internally wraps into a trivial - workspace. -- **Snapshots regenerated.** `sum.ll`, `max3.ll`, `hello.ll`, `list.ll` - show the new mangling. New `ws_main.ll` snapshot documents the - cross-module build: `@ail_ws_main_main` calls `@ail_ws_lib_add`. -- **Tests.** 40 green (previously 37). New: `workspace_build_runs_imported_fn` - (e2e: prints 5), `ir_snapshot_ws_main`, `missing_entry_main_is_error` - (codegen unit). Existing behaviour tests - (`sum_1_to_10_is_55`, `max3_picks_largest`, `hello_world_str_lit`, - `list_sum_via_match`) stay green — behaviour unchanged, only the - mangling is new. - -**Debt closed:** - -- **#19 (`source_filename` hardening).** In the workspace world, - `source_filename` is now uniformly `.ail`, once per - workspace. The previous hard-coded path dot is gone with it. - -**State:** the module system is closed end to end — loader + typecheck + -codegen + build see the workspace as a coherent unit. The multi- -diagnostic refactor and possibly cross-module ADTs remain for later. - -## 2026-05-07 — Iter 5d done: tooling extended to the workspace - -- `ail manifest|describe|deps|diff --workspace` now operate - across all modules of the workspace. The default without the flag stays - single-module for backwards compatibility. Manifest sorts by - `(module, name)`, describe accepts dotted notation `ws_lib.add`, deps - emits `{from_module, from_def, to_module, to_def}` edges, diff compares - workspace-wide with added/removed/changed/unchanged_modules and a - nested sub-diff per changed_module. -- Refactor: `diff_def_lists` is the single source of the four-category - logic; single and workspace diff share it. -- Tests: 44 green (previously 40). New: `manifest_workspace_lists_all_defs`, - `describe_workspace_resolves_qualified_name`, - `deps_workspace_includes_cross_module`, `diff_workspace_added_module`. - -**Observation (debt):** `deps` does not filter builtins/locals/function -parameters. In workspace mode that becomes more visible than in single -mode — `ws_lib.add` lists edges to `ws_lib.+` (builtin) and -`ws_lib.a`/`ws_lib.b` (function parameters). A known pre-existing -issue from Iter 2; Task #22 in the backlog. - -## 2026-05-07 — architecture review after Iter 5 - -Architect agent invoked. Findings: - -1. **Mangling consistency holds.** `@ail__` is consistent - across functions, constants, string globals, and cross-module calls. - The trampoline is correct. ADT constructors are deliberately - symbol-free (inline malloc). -2. **Module hashes bit-identical since Iter 4.** The Iter 5c snapshot - regeneration was a codegen-output change, not a hash break. -3. **Drift, due now:** - - DESIGN.md says `define i64 @main()`, codegen emits - `define i32 @main()` (see `sum.ll:35`). - - String-schema notation in DESIGN.md was shortened - (`@.str__` instead of `@.str___`). -4. **Debt that accrues interest:** the `deps` builtin leak (Task #22) has - become a falsehood in workspace mode — close it before the next big - jump. - -**Plan iteration 6 — clean-up:** - -1. **Fix DESIGN.md drift.** Update the mangling-scheme block, correct the - `@main` signature, and note the string globals precisely. -2. **`deps` hardening (#22).** Build a top-level def table per workspace; - filter edges whose target is not a top-level symbol, or emit them as - separate `builtin:`/`local:` categories. Function parameters via - lexical scope tracking from walk_term. -3. **Multi-diagnostic refactor (#20).** `check_workspace` accumulates - `Vec` across all defs instead of short-circuiting on the - first error. Intra-def may still short-circuit — the value is "see - all broken defs at once", not "see all broken sub-terms of one def". - -Order: 1 first (doc triviality), then 2 before 3 (deps is a tooling- -truth fix, multi-diag is a structural extension). - -## 2026-05-07 — Iter 6 done: deps hardening + multi-diagnose + DESIGN audit - -Three things landed together. All small, all KISS — no architecture move, -just paying off recorded debt. - -**1. `ail deps` filters builtins, params, and let/match bindings (#22).** - -Before: `sum -> +, -, ==, n, sum` and `ws_lib.add -> ws_lib.+`, -`ws_lib.a`, `ws_lib.b`. After: `sum -> sum`, `ws_lib.add -> ws_lib.add` -gone (no real deps; only the cross-module call from `ws_main` remains). - -Implementation: -- New helper `ailang_check::builtins::value_names()`: derives the - Var-level builtin names (`+ - * / % == != < <= > >= not`) from - `list()`, so the install-list and the deps-filter share one source of - truth. -- `walk_term` in `crates/ail/src/main.rs` now threads a `scope` set: - fn-params seed it; `Let` adds the bound name for the body only; - `Match` arms add their pattern variables (`bind_pattern` helper, MVP - rule "ctor sub-patterns are Var/Wild") and roll them back after. Var - refs that hit `scope` or `builtins` are dropped; qualified names - (`prefix.def`) are passed through unconditionally — the typechecker - forbids dots in def names, so no shadowing risk. -- Tests added: `deps_filters_builtins_params_locals`, - `deps_workspace_filters_builtins_and_params`. The Iter 5d test - (`deps_workspace_includes_cross_module`) keeps passing — the only - edge it asserted is the legitimate one. - -**2. `check_module` / `check_workspace` are multi-diagnose (#20).** - -`check_in_workspace` returns `Vec` instead of `Result<()>`. -Pass-1 (top-level symbol table) stays fail-fast — corrupt globals would -taint every later diagnostic. Type-def installation is fail-fast within -a module (env corruption) but the outer module loop continues. The -body-check loop is the multi-diagnose layer: each def is checked against -the assembled env, errors accumulate, the next def is attempted. - -Test: `body_errors_accumulate_across_defs` — one module with two -independent body errors (arity mismatch + unknown var) yields two -diagnostics with the right `def` field. The legacy single-error `check` -keeps working by `.into_iter().next()`-ing the Vec, so internal snapshot -tests in `crates/ailang-check/src/lib.rs` are unchanged. - -Out of scope: intra-def collection. A single fn body with three type -errors still reports one. The "see all broken defs at once" goal is met; -intra-def will require unification deferral and isn't due now. - -**3. DESIGN.md `What the MVP is NOT` audit (#24).** - -The section was lying: it claimed "No ADTs / pattern matching" (delivered -Iter 3) and "Only ints + bools + unit" (strings landed Iter 2). Renamed -to `What is not (yet) supported`, restructured into "not yet" + "what is -supported (smoke-tested)". New invariant: this section is meant to be -the truth at the **end of the latest iteration**, not a 2026-05-07-day-0 -scope statement. - -**Architecture check (the user-asked self-questioning):** - -- *Would I use this language now?* For non-recursive arithmetic + ADT - programs over int/bool/str: yes, comfortably. For anything that needs - mapping, folding, generic data structures: no, closures are the - blocker. That's the next big sprint, not Iter 7. -- *Consistency:* DESIGN.md, JOURNAL.md, code, and CLI output now agree - on what the language can do. The "What is not (yet) supported" block - is the canonical truth surface. -- *Visualisation:* `ail deps --workspace --json` is now a clean - cross-module call graph (no builtin noise). Good enough for an - external graph renderer to consume; a built-in DOT/ASCII renderer is - *possible future tooling*, not "we need it now". KISS. -- *Documentation:* the agents/ directory is the sub-prompt layer, the - JOURNAL is the iteration log, DESIGN.md is the contract. No new doc - axes needed at this scale. - -**Tests:** 47 green (previously 44). +2 deps tests in `e2e.rs`, +1 -multi-diag test in `crates/ailang-check/tests/workspace.rs`. - -**Plan iteration 7:** - -Closures + higher-order functions. This is the big jump that -DESIGN.md / Day 0 has been pointing at: it requires a typed IR (TIR) -stage, closure conversion in lowering, and a heap-aware ABI. The -multi-diag refactor in Iter 6 was scoped intentionally minimal — when -TIR lands, intra-def diagnostics become structurally cheap and Task #20 -gets revisited. - -## 2026-05-07 — Iter 7 done: first-class function references (no capture) - -Iter 6 outlined Iter 7 as "closures + HOFs + TIR". KISS course-correct -on inspection: that bundle had three independent things in it, and the -HOF use-cases (passing functions around, calling through fn-typed -parameters) need none of TIR or capture. Splitting paid off — what -landed here is ~120 LOC of codegen, no TIR, no heap, no ABI churn. -Closures with capture stay queued for Iter 8 (where TIR is the -correct precondition). - -**What works now:** -- Top-level fn name (or qualified `prefix.def`) used as a value yields - an LLVM fn-pointer (`@ail__`, type `ptr`). -- Fn-typed parameters can be called as `f(args)` — the body emits an - indirect `call () %f(...)`. -- Pass through `let`: `let g = inc in g(x)` works (the local just - aliases the global SSA, the sidetable lookup still hits). -- Pass to another fn: `apply(inc, 41) == 42` — see - `examples/hof.ail.json`, exercised end-to-end. - -**What does not (yet) work — by design:** -- No anonymous lambdas. The only fn-value source is a top-level def - reference. -- No capture. A fn-value is always a constant pointer to a top-level - def; there is no environment to allocate. -- Both deferred to Iter 8 where they share the TIR + closure-conversion - preconditions. - -**Implementation, in order of where the rubber meets the road:** - -1. `llvm_type` learned `Type::Fn { .. } -> "ptr"`. The actual signature - travels separately. New helper `fn_sig_from_type` lifts an AILang - fn-type into an `FnSig` (LLVM types only). -2. `Emitter` got a sidetable: `ssa_fn_sigs: BTreeMap`, - keyed by SSA value (or `@global`). It's reset per function body. -3. At `emit_fn` entry, every fn-typed parameter registers - `(%arg_, sig)` in the sidetable. -4. `lower_term(Term::Var)` now falls through to a top-level fn lookup - (`resolve_top_level_fn`) when the name isn't a local. The returned - SSA is the global symbol; the sidetable gets the sig. -5. `lower_term(Term::App)` dispatches: - - if callee is a `Var` AND not shadowed AND statically known - (`is_static_callee` covers builtin operators, qualified - `prefix.def`, current-module fns), keep the existing direct - `lower_app` path — no extra indirection in the IR; - - otherwise lower the callee, expect type `ptr`, look up the sig in - the sidetable, emit `emit_indirect_call`. -6. `Term::If` propagates the sig to its phi SSA when both branches are - fn-pointers with matching sigs (cheap two-line copy; no separate - test, falls out of the `apply`-on-conditional pattern). - -**Why no typechecker change?** The typechecker already accepted -fn-typed locals (`Term::Var` against `env.globals`, App via `synth(callee)` -unifying with `Type::Fn`). The only blocker was `MVP: callee must be a -variable` in codegen. - -**Tests:** 48 green (previously 47). - -- `crates/ail/tests/e2e.rs::higher_order_apply_inc` builds and runs - `examples/hof.ail.json`, asserts the binary prints `42`. -- Existing tests unchanged (incl. snapshot tests around the IR - emission for `sum`, `list`, `max3`). - -**Architecture self-check:** - -- *Would I use this language now?* Yes for `apply`-style and "pass a - predicate" patterns. Still no for capturing closures (`let n = 3 in - map(\x -> x + n, xs)`-equivalent), but the ergonomic gap shrank. -- *Consistency:* DESIGN.md "What is not (yet) supported" rewritten in - the same edit; first-class fn-refs now have a positive bullet, the - closures bullet is precise about what it means (no capture, no - lambdas). -- *Visualisation:* `ail describe`/`manifest` already render fn-typed - params correctly via the existing `pretty::type_to_string` - (`((Int) -> Int, Int) -> Int`). No tooling change required. -- *KISS:* every alternative I considered (full `LocalType` enum, - swapping `(String, String)` returns to a typed wrapper, lifting - lambdas to defs as syntactic sugar) was strictly more code than the - sidetable approach, with no expressivity gain. - -**Plan iteration 8:** - -Closures with capture, anonymous lambdas, the typed IR (TIR) layer, -closure conversion in lowering. Now that we have indirect calls -working, the main delta is: a fn-value also needs an environment -pointer, the sidetable becomes per-value (heap-allocated), and the -calling convention shifts to `(env_ptr, args...)`. Touches every -existing call path — that's why it gets its own iteration. - -## 2026-05-07 — Iter 8 done: closures with capture (no TIR needed) - -Iter 7's plan named TIR as the prerequisite for closures. On -inspection that bundling was wrong — TIR is one possible -implementation strategy, not a structural requirement. The -typechecker already attaches enough type information through `synth` -that the codegen can read capture types out of `self.locals` -directly. So Iter 8 ships closures **without** introducing TIR. KISS -won. - -The work split into two commits: - -**Iter 8a — closure-pair ABI flip.** Every fn-value is now a `ptr` to -a heap or static closure pair `{ thunk_ptr, env_ptr }`, regardless of -whether it came from a lambda or a top-level def reference. To keep -top-level-fn references cheap, every top-level fn auto-emits: - -```llvm -define @ail___adapter(ptr %_env, ) { - %r = call @ail__() - ret %r -} -@ail___clos = constant { ptr, ptr } { @adapter, null } -``` - -`Term::Var` resolving to a top-level fn returns the address of -`_clos`, never the bare fn pointer. `emit_indirect_call` was -rewritten to GEP+load both halves and call `thunk(env, args...)`. -Direct calls (statically-known callees in `Term::App`) bypass the -adapter and stay at the original speed. - -The Iter 7 hof example (`apply(inc, 41)`) continues to print 42 -unchanged — only the IR shape changed, not the source. IR snapshot -files for sum/list/max3/hello/ws_main were refreshed. - -**Iter 8b — Term::Lam + capture + lambda lifting.** New AST node: - -```jsonc -{ "t": "lam", - "params": ["x"...], - "paramTypes": [Type...], - "retType": Type, - "effects": ["..."], - "body": Term } -``` - -Param/return types are explicit. The typechecker accepts the -declared `Type::Fn` shape, checks the body's type against `retType`, -and verifies that body effects are a subset of the declared lambda -effects (no row polymorphism in the MVP). Constructing a lambda is -pure; the act of *calling* picks up the declared effects, via the -existing App branch. - -Codegen does textbook closure conversion: - -1. **Free-variable analysis.** `collect_captures` walks the body - skipping builtins (`+`, `==`, ...), the current module's top- - level fns, and qualified `prefix.def` names. The remainder are - captures. Inner lambdas contribute their own free vars upward. -2. **Lift to thunk.** For each lambda, generate a fresh - `@ail___lam(ptr %env, params...)`. State the body - into a side buffer (the emitter's `body`/`locals`/`counter` are - saved and reset, then restored). Captures and lambda params are - pushed as named locals so the body lowering finds them. The - thunk text goes into a `deferred_thunks` queue and is appended - after the parent fn's `}` — LLVM IR doesn't care about fn order. -3. **Pack at the use site.** In the OUTER body emit: - - ```llvm - %env = call ptr @malloc(i64 <8 * captures>) - ; for each capture i: store at offset 8*i - %clos = call ptr @malloc(i64 16) - ; store thunk_ptr at offset 0, env at offset 8 - ``` - - `%clos` is the value returned by the Lam term. Its sig is - registered in the sidetable so subsequent indirect calls work. - -4. **Capture sigs propagate.** A fn-typed capture (e.g. capturing a - fn-typed param of an outer scope) keeps its FnSig in the thunk's - sidetable, so the captured fn can still be indirect-called from - inside the lambda. - -Capture layout uses 8-byte slots regardless of LLVM type. Typed -load/store reads only the bytes it needs — wasted padding for `i1` -and `i8` is fine at this scale. - -**Architecture self-check:** - -- *Would I use this language now?* Yes for substantially more cases. - `let n = 3 in apply(\\x. x + n, 39)` is the example I would have - reached for in Iter 6 and bounced off. It now compiles and runs. - `map`/`fold`/`filter` over user-supplied predicates are within - reach — only the absence of polymorphism still forces author-side - monomorphisation. -- *Did I think of everything?* Hash stability checked manually: - `examples/sum.ail.json` produced the same fn hashes - (`db33f57cb329935e`, `d9a916a0ed10a3d3`) before and after Iter 8. - Existing modules without `Term::Lam` serialise bit-identically. ✓ -- *Consistency:* DESIGN.md "What is not (yet) supported" rewritten - in the same edit. The Term schema gained `lam`, `ctor`, `match` - rows that were already supported but had been omitted from the - schema fragment. Now the doc is exhaustive for the supported - language. -- *Visualisation:* `ail describe` already renders Lam terms (added - pretty-printer rule), and the codegen IR for `closure.ail.json` - reads as a textbook closure-conversion lowering. -- *KISS check:* I considered three alternatives and all were - strictly worse — fat-pointer ABI (aggregate-passing concerns), full - TIR layer (large rewrite), uniform heap pair without static-closure - optimisation (regressed Iter 7 to one malloc per fn-value escape). - -**Tests:** 49 green (was 48 after Iter 7). One new e2e: -`closure_captures_let_n` builds and runs `examples/closure.ail.json` -asserting "42". IR snapshot files refreshed for the per-fn adapter + -static-closure scaffold — only structural delta. - -**Plan iteration 9:** - -Two candidates, both real pain points: - -1. **Polymorphic inference.** Make `Type::Forall` actually work in - `synth` — instantiate fresh type variables at each use site, allow - `let id = \\x. x in (id 1, id true)`. This unblocks generic - `map`/`fold`/etc. without per-type clones. Probably small (~150 - LOC in the typechecker; codegen already monomorphises by - instantiation when it lowers the call). - -2. **GC / region reclamation.** Right now ADT boxes, lambda envs, - and closure pairs all leak through the program's lifetime. A - minimal mark-and-sweep over a tagged heap would let us run real - programs. Bigger lift, ~400-600 LOC plus runtime support. - -Leaning toward (1) for the next iteration: it's the smaller bite -*and* the bigger expressivity unlock. (2) becomes acute only when -someone tries to run an unbounded loop, which the current examples -don't. - -## 2026-05-07 — Iter 9 done: dogfood + `ail run` - -Course-corrected from the Iter-8 plan. Polymorphism is the bigger -expressivity unlock on paper, but I hadn't actually proved that the -language was sufficient for "small but real" programs without it. So -Iter 9 became a dogfood iteration: write a non-trivial program -that exercises everything Iter 1-8 shipped, and use `ail run` / -errors / type-checker output as the user would. If something broke, -fix it. If nothing broke, document the boundary moved. - -**`examples/list_map.ail.json`**: - -```jsonc -type IntList = Nil | Cons Int IntList - -map_int :: ((Int) -> Int, IntList) -> IntList -map_int(f, xs) = match xs { - Nil -> Nil - Cons(h, t) -> Cons(f(h), map_int(f, t)) -} - -print_list :: (IntList) -> Unit !IO -print_list(xs) = match xs { - Nil -> () - Cons(h, t) -> let _ = do io/print_int(h) in print_list(t) -} - -main = let xs = Cons 1 (Cons 2 (Cons 3 Nil)) in - print_list(map_int(\\x. x * 2, xs)) -``` - -**Result:** nothing broke. Output `2\\n4\\n6\\n`, exit 0. The full -pipeline (`ail run`) covers: ADTs with two ctors of different -arity; pattern matching with nested `Var` fields; recursion over -ADT; closures (with no captures here, so env is null but the -closure-pair plumbing still gets exercised); fn-typed parameters in -a top-level def; `do io/...` inside a match arm body, with `let _` -to sequence two effectful operations; effect propagation through -the call chain. This validates Iter 1-8 as a self-contained -foundation. - -**Friction surfaced:** writing the AST by hand is tedious — the -JSON for this 4-def module is 200+ lines. That's not surprising -(the format is for LLMs, not humans), but it suggests an Iter 10 -priority: a richer pretty-print form, or an `ail snippet` helper -for common boilerplate (`mk_list_int`, etc.). Not blocking; noted. - -**`ail run` (Iter 9b):** Builds into a tempdir + execs the binary, -exit code passthrough. Saves a `cd && ./bin` step in the dogfood -loop. Tiny addition — `Cmd::Build`'s body factored into a shared -`build_to` helper. - -**Architecture self-check:** - -- *Would I use this language now?* For self-contained Int-typed - programs over recursive ADTs: yes. The list_map example is what - I would have wanted to write since Iter 6 and bounced off - repeatedly. It now compiles and runs without me adapting the - source — the language is what its authors said it was, end to - end. -- *Did I think of everything?* Two cracks observed during the - dogfood: - - `(Int)` parens around single-param fn-types in pretty-print are - visual noise. Cosmetic, can wait. - - `let _ = do in ` is the only way to sequence - effects today. Working as intended given KISS, but a `;` - operator (sequencing) would be cheap polish. -- *Consistency:* DESIGN.md CLI block + smoke-test list updated. - Iter 8c invariant — "What is not (yet) supported" ≡ truth at end - of latest iteration — held; no new pending items. -- *KISS:* Iter 9 added 0 LOC of language semantics. All gain came - from validating the existing surface and a small CLI helper. - -**Tests:** 50 green (was 49). New e2e -`list_map_doubles_then_prints`. No test for `ail run` itself — -`build_and_run` already exercises the equivalent path. - -**Plan iteration 10:** - -The dogfood revealed two real-but-not-blocking pain points and one -big architectural gap. Candidates, ranked: - -1. **Polymorphic let-bindings with monomorphisation at codegen.** - Allows `let id = \\x. x in (id 1, id true)` and ultimately - `map :: (a -> b) -> List a -> List b`. The ground truth-ier - answer for the "would I use it for X?" question, but a - non-trivial pipeline change (typechecker→codegen needs to thread - instantiation info to the call site). - -2. **Sequencing operator `;` and richer effect ergonomics.** A - `Term::Seq { lhs, rhs }` (or compile sugar to `Let { name: "_", - value: lhs, body: rhs }`) plus a small pretty-print update. - Cheap, satisfying. - -3. **GC.** Heap reclamation for ADT boxes, lambda envs, closure - pairs. Real architecture step. Becomes acute the moment someone - writes a long-running loop; the current examples don't. - -Tentative pick: (2) for the next sprint as a satisfying small -polish, then (1) as Iter 11. (3) bides its time until a real -program needs it. - -## 2026-05-07 — Iter 10 done: Term::Seq sequencing - -Followed the Iter 9 plan and shipped (2). New AST node -`Term::Seq { lhs, rhs }` with serde tag "seq". Semantics: evaluate -lhs (which must be Unit), discard the value, return rhs. Effects -from both sides accumulate. - -This is sugar for `let _ = lhs in rhs`, but it's a first-class node -because: -- The pretty-print renders cleanly (`(seq lhs rhs)` instead of - borrowing the `let` form with a discard binding). -- Diagnostics are sharper: a non-Unit lhs gets a "type mismatch" - error pointing at the seq site, not "binding `_` had type X" at - a let site. -- Future tooling (effect inference visualisation, dataflow) can - treat sequencing as a structural concept instead of a special- - cased let. - -Codegen is trivial: lower lhs (drop SSA), lower rhs (return). - -Refactored `examples/list_map.ail.json`'s `print_list` to use seq -instead of `let _ = ...`. Output unchanged (`2\\n4\\n6\\n`); the -JSON shed a few lines and reads more honestly. - -**Architecture self-check:** - -- *Would I use this language now?* Same answer as Iter 9 (yes for - small but real programs), but the seq node makes IO-heavy - recursion read better — closer to "call this effect, then this - one" instead of "bind this effect to nothing, then this one". -- *Did I break anything?* Hash stability check: existing examples - without `Term::Seq` serialise identically; their fn hashes are - unchanged. `list_map.ail.json`'s hashes shifted as expected since - its body changed. -- *KISS:* +30 LOC across AST/pretty/check/codegen/walker. One - unit test for the lhs-must-be-Unit rule. The dogfood example - proves the e2e path. - -**Tests:** 51 green (was 50). New `seq_lhs_must_be_unit` unit test -in ailang-check. Existing list_map e2e still passes after the -refactor. - -**Plan iteration 11:** - -Polymorphism, as queued in the Iter 9 plan. Concretely: HM-style -unification + let-generalisation in the typechecker, monomorph- -isation at codegen time. Touches the typechecker→codegen pipeline. -Bigger commit than the recent stretch, will probably need to be -phased (typechecker substitution machinery, then codegen -specialisation, then docs). - -## 2026-05-07 — Iter 11 done: deeper dogfood (insertion sort) - -Pulled back from polymorphism for one more validation cycle before -the architectural step. Polymorphism is a substantial pipeline -change (typechecker substitution + codegen monomorphisation) and I -wanted one more "small but real" program to confirm the existing -foundation holds before disturbing it. - -`examples/sort.ail.json` — insertion sort over `IntList`: - -```jsonc -insert :: Int -> IntList -> IntList -insert(y, xs) = match xs { - Nil -> [y] - Cons(h, t) -> if y <= h then Cons(y, Cons(h, t)) - else Cons(h, insert(y, t)) -} - -sort :: IntList -> IntList -sort(xs) = match xs { - Nil -> Nil - Cons(h, t) -> insert(h, sort(t)) -} - -print_list :: IntList -> Unit !IO // uses Iter 10 seq - -main = print_list(sort([3,1,4,1,5,9,2,6,5,3,5])) -``` - -**Result:** typechecks first try, runs first try, prints -`1 1 2 3 3 4 5 5 5 6 9` (each on its own line). 11-element input, -correct sorted output. The combination of recursive ADT pattern -match + comparison ops + branching + leaf recursion + IO -sequencing all worked end to end without the language tripping me -up. Iter 10's seq made `print_list` notably cleaner than the -`let _ = ...` form would have been. - -**Architecture self-check:** - -- *Would I use this language now?* For "small but real" - monomorphic programs over Int, Bool, Unit, Str, and ADTs of - those: confidently yes. Insertion sort writes out as the - textbook recursion, no bookkeeping that the language couldn't - do for me. -- *Did I think of everything?* The remaining wall is still - polymorphism. Sort over `IntList` needs hand-monomorphisation; - a generic `sort :: (a -> a -> Bool) -> List a -> List a` is - what the language eventually wants. No new architectural cracks - surfaced from this dogfood. -- *Visualisation:* `ail describe sort.ail.json sort` reads the - way I'd expect a sort definition to read, with `IntList` types - inline and the recursive call rendered cleanly. -- *KISS:* Iter 11 added 0 LOC of language semantics and 1 e2e - test. The 250-line JSON for the example is verbose but - mechanical — no friction once you accept that the JSON is the - surface for LLM authors. - -**Tests:** 52 green (was 51). New e2e -`insertion_sort_orders_list`. Pure addition; existing tests -untouched. - -**Plan iteration 12:** - -Now polymorphism. Two more dogfood programs would just keep -producing the "the language is fine for monomorphic programs" -result, which is already established. The real expressivity -unlock — and the answer to "would I use it for X?" for X that -actually needs generic data — is HM inference + let-generalisation -+ monomorphisation. Phased plan: - -12a. Typechecker: introduce a `Subst` (type variable substitution) - and unification. Thread through `synth`. At `let`, generalise - syntactic values (lambdas) — no value-restriction subtlety - needed yet, the MVP has no mutable refs. -12b. Codegen: at each polymorphic call site, the typechecker - records the instantiation. Codegen walks the AST a second - time per (def, instantiation) pair and emits a specialised - version with the type variables substituted by concrete - types in fn signatures. -12c. Docs + a polymorphic `id` test + a generic `map :: (a -> b) - -> List a -> List b` rewrite of `list_map.ail.json`. - -## 2026-05-07 — Iter 12a/b done: polymorphism reaches the binary - -Skipped 12c's "polymorphic map" — without parameterised ADTs (which -the MVP doesn't have), the rewrite would still be over a concrete -`IntList`, defeating the purpose. So 12c becomes lighter: docs + -two new examples (`poly_id`, `poly_apply`) that prove polymorphism -end-to-end on primitive types and on fn-typed parameters. The big -test is whether *I* would use the language now for a poly-flavoured -program; the answer below. - -**12a — typechecker:** - -`Type::Forall { vars, body }` is now legal at top-level fn types. -Implementation is the textbook ML rule: peel the Forall when -checking the body (rigid vars go into `Env.rigid_vars` so -`check_type_well_formed` accepts them), instantiate fresh metavars -at every var-resolution site, unify on every formerly-`expect_eq` -edge. - -The metavar encoding sidesteps an AST schema change: a metavar is -just `Type::Var { name: "$m" }`. The `$` prefix can't collide -with source identifiers, the JSON layout doesn't shift, and module -hashes stay bit-identical (verified: `sum.ail.json` keeps -`db33f57cb329935e` / `d9a916a0ed10a3d3`). I considered adding a -new `Type::Meta` variant under `#[serde(skip)]` but that would -have pulled hashing concerns into serde; the naming convention -keeps the AST untouched. - -`Subst` is a flat `BTreeMap`; `unify` is the standard -occurs-check version with effects compared as a set. Constants -still reject Forall outright; ADT fields still reject vars. No -let-generalisation: lambdas inside fn bodies are checked -monomorphically against their declared types — keeps the -implementation small and matches DESIGN.md's "top-level types -must always be explicitly annotated". - -**12b — codegen:** - -Direct calls to a polymorphic def get monomorphised on demand. -Each unique (def, instantiation) pair emits a specialised LLVM fn -with mangling `@ail____`. Descriptor scheme: -`Int → I`, `Bool → B`, `Unit → U`, `Str → S`, ADT `Foo → FFoo`, -`Fn(a)→b → Fn___r_`. So `id(42)` and `id(true)` produce -`@ail_poly_id_id__I` and `@ail_poly_id_id__B` side by side. - -Pass 1 of `lower_workspace` now splits fn-typed defs into mono -(`module_user_fns`, LLVM-typed FnSig as before) and poly -(`module_polymorphic_fns`, full FnDef). A unified -`module_def_ail_types` carries AILang types for both, used by -the codegen-side type tracker. - -The hard part was getting AILang types at call sites. The -typechecker has them but doesn't hand its annotations down (no -TIR yet). I considered three paths: - 1. Typechecker sidetable keyed by AST node ids — would need - to assign ids deterministically, brittle. - 2. Uniform representation (everything passes as ptr/i64) — - contradicts CLAUDE.md's "performance is extremely important". - 3. Codegen replays the type derivation locally. -Picked (3). The trade-off is duplication (`synth_arg_type` -mirrors what the typechecker already did), but it's contained -to a small recursive walk and uses the same `locals`/`extras` -shadowing pattern. Worth it for the MVP — once a TIR stage -materialises (it's still on the debt list), the duplication -collapses into a single pass. - -`locals` grew from 3-tuple to 4-tuple `(name, ssa, llvm_type, -ail_type)`. Six push sites updated mechanically. Lambda capture -metadata grew the same way. `CtorRef` got `ail_fields` so match -arm bindings inherit the AILang type. - -The drain phase iterates until `mono_queue` is empty — -specialised bodies can themselves invoke polymorphic defs and -queue further entries. `apply_subst_to_term` substitutes rigid -vars in `Term::Lam` annotations (the only Term arm carrying -types). - -**Architecture self-check:** - -- *Would I use this language now?* For monomorphic programs: - yes (already established). For polymorphism over primitives - and fn-typed parameters: yes — `id` and `apply` write out the - way the textbook says they should, with no language-level - bookkeeping leaking into the source. The `poly_apply` example - was particularly revealing: the closure-pair ABI (Iter 8a) - composes cleanly with monomorphisation. Specialised body of - `apply__I_I` keeps `f` as a fn-typed local; the existing - indirect-call path already handles the lower from there. -- *Did I think of everything?* No, two known gaps: - 1. **Polymorphic fn passed as a value** (`let f = id in f(42)`) - fails in codegen — `resolve_top_level_fn` looks in - `module_user_fns` only. Adding this means emitting one - closure-pair global per instantiation, possibly via the - same drain pass. Defer. - 2. **Higher-rank polymorphism** (`apply(id, 42)`) trips - `unify_for_subst` which doesn't handle Forall on the param - side. Real higher-rank polymorphism is a substantial step - and not on the near horizon — deferred to a later iter. -- *Visualisation:* `ail manifest poly_id.ail.json` now shows - `forall a. (a) -> a` correctly. The pretty-printer carried - `Type::Forall` rendering since Iter 1; nothing to do. -- *KISS:* +1 typechecker file edit (~430 LOC inserted, mostly - Subst+unify+four tests), +1 codegen extension (~600 LOC - inserted, mostly the drain path + helpers + locals widening). - Two new examples, two new e2e tests. Could be smaller if I - bit the bullet on TIR; not yet worth the upfront cost. - -**Tests:** 58/58 (was 56/56). Added 4 typechecker unit tests in -12a, 2 e2e tests in 12b. Hash invariant holds. - -**Plan iteration 13 (queued, not started):** - -The natural next step depends on what I want to use the language -for. Two candidates, in order of expected payoff: - -13a. **Parameterised ADTs** — `List a`, `Maybe a`, etc. Without - these, polymorphism is half-useful: a generic `map` still - can't transform an `IntList` into a `BoolList`. ADT defs - would gain a `vars: Vec` field; ctor field types - could mention them; codegen monomorphises ADT instances - just like fns. This is the bigger expressivity unlock. -13b. **GC or arena** — every ADT box, lambda env, and closure - pair currently leaks. For sort over an 11-element list, - fine. For anything longer-running, required. The current - lifetime model is "leak"; the right MVP is probably - bumpalloc per top-level fn invocation. Could be done - before parameterised ADTs but doesn't unlock new examples. - -Leaning 13a — it's the more interesting architectural step and -makes the "polymorphic map" rewrite from the original 12c plan -finally meaningful. - -## 2026-05-07 — Iter 13 done: parameterised ADTs reach the binary - -**Why now.** End of Iter 12 left polymorphism half-useful: `id` -and `apply` worked, but every container was monomorphic -(`IntList`, `Maybe_Int`). A generic `map :: forall a b. ((a) -> b, -List a) -> List b` was unwritable. 13 lifts that. - -**Three commits:** - -- `0782622` 13a — schema (`TypeDef.vars`, `Type::Con.args`) + - checker (substitution at ctor + match + arity validation in - `check_fn`). -- `1631f60` 13b — codegen: per-use-site substitution of LLVM - field types in `lower_ctor` and `lower_match`. No mono-queue - for types — ctor code was already inlined at every use site, - so 13b only had to thread substitution through, not invent a - symbol scheme. `synth_arg_type` for `Term::Ctor` now returns - concrete type-args, and `llvm_type(Type::Var)` is a hard - error instead of a silent `ptr` fallback (the latter was - flagged by the architect review and is the most defensive - single change in 13). -- `` 13c — DESIGN.md flipped (parameterised ADTs out of - the gap list, into the supported list); two new example lines. - -**Hash invariant.** Both new fields are -`#[serde(default, skip_serializing_if = "Vec::is_empty")]`. A new -regression test in `crates/ailang-core/src/hash.rs` deserialises -the actual `examples/sum.ail.json` and `examples/list.ail.json` -from disk and asserts `db33f57cb329935e` and `b082192bd0c99202` — -the recorded pre-13a hashes. It's deliberately phrased against -the on-disk JSON rather than reconstructed code, so the test -fails if anyone resaves the examples in a way that drifts the -canonical bytes. - -**Architect-flagged debt I deliberately did NOT touch in 13:** - -- `is_static_callee` returns true for poly fns but - `resolve_top_level_fn` only consults `module_user_fns`. A - poly fn used as a value (`let f = id in f(42)`) passes the - static check then surfaces as `UnknownVar`. Would need one - closure-pair global per instantiation. Out of 13 scope; same - hole that was queued at the end of Iter 12. -- Triple source of truth for builtins (`builtins::install`, - `builtins::list`, `codegen::builtin_ail_type` / - `builtin_effect_op_ret`). Every new operator costs three - edits. Low interest today, escalates with every effect op. - Worth a future tidy iter — not blocking expressivity. -- `synth_arg_type` for `Term::If` returns `synth(then)` only; - for `Term::Match`, the first arm. Masked today by the - typechecker having already unified, but it's the kind of - duplication that decays. Same fundamental cost as the absence - of a TIR. - -**Architecture self-check.** - -- *Would I use this language now?* For polymorphism over - primitives, fn-typed values, AND parameterised containers — - yes. The `box.ail.json` and `maybe_int.ail.json` examples - read like the textbook says they should. No type-arg - bookkeeping leaks into the source. -- *KISS.* 13b was much smaller than I feared at the start of - the design phase: ~150 LOC in codegen, no new structures, no - mono-queue. The reason: ADT ctor code is already inlined. - The architect's recommendation to *not* mutate `ctor_index` - but derive `CtorRef` per use site was the right call — - preserved the static template, made the substitution local. -- *Did I think of everything?* Two known gaps remain. **(1)** - Polymorphic ADTs as the type-arg of a polymorphic fn — - works today because `unify_for_subst` recurses through - `Type::Con.args` (added in 13a). **(2)** A polymorphic fn - taking a polymorphic ADT and returning a different - parameterised ADT (`map : forall a b. ((a)->b, List a) -> List - b`) — should also work, but I haven't dogfooded it yet - because `List a`-as-a-rewrite-of-`list_map` would need the - schema bumps elsewhere (paramaterised list builder). Queued - for Iter 14. -- *Visualisation.* `ail manifest examples/box.ail.json` shows - `type Box :: forall a. MkBox(a)` and - `fn unbox :: forall a. (Box) -> a`. The pretty-printer - picked up `args` and `vars` cleanly (Iter 13a). - -**Tests:** 64/64 (was 58/58). Added: 1 hash-stability regression -(13a), 3 checker unit tests for parameterised ADTs (13a), -2 e2e tests over `box.ail.json` and `maybe_int.ail.json` (13b). - -**Process note (orchestration).** First iter where I worked -strictly through the agents in `/agents/`: `ailang-architect` -ran a drift review on HEAD before 13b started; `ailang-implementer` -got a fixed brief that incorporated the architect's three -recommendations (don't mutate `ctor_index`, fix `synth_arg_type` -for `Term::Ctor`, harden `llvm_type`); 13c (this) is the -orchestrator's own work. The role split landed in `3df943d` after -I caught myself doing implementer work on 13a directly. The -agents pay off in proportion to iter size — for 13b they were -clearly worth the round-trip; for 13a's checker work, marginal. - -**Plan iteration 14 (queued, not started):** - -Two candidates, in order of expected payoff: - -14a. **Polymorphic `List a` rewrite of `list_map`.** Replaces - `IntList` with `List a`, rewrites `list_map` to return - `List b`, and lets the polymorphic-map version be the - dogfood smoke test. Pure exercise — should fall out of - 13b — but worth the dogfood beat. Also: `Maybe a` used - in a non-trivial fn (e.g. `find : forall a. ((a) -> Bool, - List a) -> Maybe a`). -14b. **GC or arena.** Same pitch as before: every ADT box, - lambda env, closure pair leaks. For `box.ail.json` and - `maybe_int.ail.json`, fine. For anything that allocates - in a loop, required. Bumpalloc per top-level fn - invocation is the natural MVP. -14c. **Poly fn as value.** Closes the asymmetry the architect - flagged; gates `let f = id in f(42)`. One closure-pair - global per instantiation, emitted via the same mono-queue - drain path. Smaller surface than 14a/b. - -Leaning 14a — the dogfood payoff for one iter of polish is -high, and `Maybe`-in-a-real-fn is a missing piece I haven't -exercised yet. 14b stays second; 14c is a candidate if I want -a small palate cleanser. - ---- - -## Iter 13d — rustdoc polish for `ailang-core` + new `ailang-docwriter` agent - -User triggered: ran `cargo doc --open` for fun and reported -that the rendered docs were thin — crate headers existed, -but `pub` items had no `///` strings, there were no `# Examples` -sections, and intra-doc links were missing. Internal references -showed up as prose ("see Builtins") rather than as clickable -links. Two stale `[Builtins]` and `[code]` warnings had been -bleeding into every `cargo doc` invocation since Iter 6 or so. - -Recurring task → new agent. Wrote `agents/ailang-docwriter.md` -with a tight mandate: rustdoc only, no API changes, no edits in -`docs/` or `agents/`, three verification gates (rustdoc clean, -build green, tests green incl. doctests). Updated -`agents/README.md`. Updated DESIGN.md item 6 of "Verification -and correctness" to make rustdoc cleanliness a project-wide -invariant rather than an iter-local cleanup. - -First mission: `ailang-core` only. The foundation crate every -other crate depends on — biggest reader-leverage per diff. The -agent rewrote the crate root so a newcomer learns: what `core` -owns, where it sits in the pipeline (`core` → `check` → -`codegen` → `ail`), the central invariant (canonical JSON is -deterministic, hashes content-addressed, schema = `ailang/v0`), -and the entry points. Module roots in `ast.rs`, `canonical.rs`, -`hash.rs`, `pretty.rs`, `workspace.rs` got the same treatment. -Every `pub` struct / enum / variant / fn / const got a `///` -string. The Iter-13a additions (`TypeDef.vars`, `Type::Con.args`) -got an explicit backwards-compat note that points back to the -hash-stability regression test. - -`# Examples` blocks landed where they shorten understanding -(`canonical::to_bytes`, `def_hash`, `Workspace`), all marked -`ignore` so the workspace doctest run stays cheap (3 ignored, -0 run, 0 failed). The two stale broken-link warnings in -`ailang-check` got prose-only fixes — out-of-scope for this -iter conceptually, but a one-line fix per file means rustdoc -is now globally clean. - -**Findings reported by the docwriter** (judgement deferred to -me; none made it into the diff): - -- `Term::Lam.param_tys` (JSON: `paramTypes`) is positionally - paired with `Term::Lam.params: Vec`. Naming hints - at the convention but a cold reader has to deduce it. **Not - fixing**: renaming would touch the schema, breaks every hash. - The `///` string makes the convention explicit, which is - enough. -- `Type::Var` is overloaded: source-level rigid vars and - checker metavars (`$m`) share the same variant. A reader - of `core` alone sees no hint of the metavar half — it's - documented in `ailang-check`'s lib doc instead. **Not fixing**: - splitting the variant would balloon the schema and - invalidate every hash. Acceptable as long as `check`'s lib - doc explains it (it does, post-13d). -- `Def::Type(TypeDef)` versus the type-expression enum `Type` - in the same module: name collision is real but unavoidable - without renaming `Type` (which would touch every crate). The - `///` strings now disambiguate at point of contact. - -**Process note (orchestration).** Second iter where I worked -strictly through agents (after 13b). The docwriter brief was -written from the diagnostic in this conversation, not from a -DESIGN-doc design pass — there was no architecture decision to -make, just a discipline gap to close. That's the right shape -for a docwriter: low-judgement, repeatable, runs after every -iter that touches public surface. Cost ≈ 25 tool uses for ~390 -LOC of doc additions across 9 files; smaller-grain than -`ailang-implementer` runs typically are. - -**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests -(new). `cargo doc --no-deps`: 0 warnings (was 2). `cargo build ---workspace` green. `cargo test --workspace` green. - -**Plan iteration 13e/13f (queued, not started).** Two natural -follow-ups for the docwriter: - -13e. **`ailang-check` rustdoc**: type-checker is the next - biggest crate by `pub`-surface and the most algorithmically - dense. Rigid/metavar split, `Forall` instantiation, the - match-arm exhaustiveness logic, the Iter-13a substitution - machinery — all of it benefits more from prose explanation - than `core` did. -13f. **`ailang-codegen` + `ail` CLI rustdoc**: codegen is dense - but mechanical (mangling, ABI, block tracking); the CLI - is mostly clap derive macros. Lower payoff per LOC of doc - than 13e but rounds out the warning-free invariant across - the whole workspace. - -After 13d there's no urgency on 13e/f — `cargo doc` is already -warning-free. They're "polish iterations to be slotted in -between feature iters when context budget is short". 14a -(`List a` rewrite) remains the next-feature default. - -## Iter 13e — rustdoc polish for `ailang-check` - -Second docwriter mission. Same mandate as 13d, applied to the -typechecker crate. `lib.rs` (1922 LOC) had a strong crate root -already — covers HM-with-effects, top-level forall, the -rigid-vs-metavar split, the `$m` encoding — but its `pub` -surface (the `Env` struct, `CheckError` + 24 variants, -`CheckedModule`, `CtorRef`, the `check_module` entry point) was -mostly undocumented. `builtins.rs` had a one-line module -header and zero `///` strings on `EffectOpSig` or `install`. -`diagnostic.rs` had a strong module header (lists every stable -diagnostic code) but `Diagnostic`, `Severity`, and the -construction helpers were undocumented. - -Agent added 188 LOC of pure rustdoc across the three files; no -non-doc lines changed (verified by filtering the diff). Each -`CheckError` variant now carries the AST term/type that -triggers it and the stable kebab-case `code` it maps to. `Env` -fields (`globals`, `effect_ops`, `types`, `module_globals`, -`current_module`) got individual `///` strings that name the -invariants — most importantly that `module_globals` includes -the current module, which the agent flagged as undocumented at -field level (now fixed). Crate-root prose got an upgraded -intra-doc link to `[`check_module`]`; `super::` reference in -`diagnostic.rs` rewritten to `crate::` (cosmetic, but the -canonical form). - -**Findings reported** (judgement deferred to me): - -- `Env` is `pub` with all-`pub` fields but `Env::new` is - private and there's no public builder — external callers can - only construct via field-by-field literal, which is fragile - if a field is added later. **Not fixing**: changing this is - an API decision, not a doc one. Worth raising next time we - touch the crate's public surface deliberately. -- `CheckError::CtorArity` and `CheckError::ArityMismatch` both - serialize to the public diagnostic code `arity-mismatch`. The - `///` strings now flag the collision per-variant; tooling - that consumes the diagnostic JSON sees only the merged code - and that's intentional from the iter-5b vintage. **Not - fixing.** -- `Diagnostic` and `Severity` are reachable both via the crate - re-export and via `crate::diagnostic::*` because the - `diagnostic` module is itself `pub`. Rustdoc renders both - pages; harmless but slightly noisy. **Not fixing**: the - re-export is the documented entry point and we don't want to - hide the module. - -**Process note.** Same shape as 13d: I wrote a brief naming -the deficiencies (numbers of pub items, which files had thin -roots), the agent did the doc additions inside its mandate, -the orchestrator-side work was authoring DESIGN/JOURNAL and -verifying the diff. Cost ≈ 32 tool uses for 188 LOC of doc -across 3 files — denser than 13d (more sentences per pub item -because the typechecker invariants need explicit -articulation), but the per-LOC payoff for a future reader is -also higher. - -**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests -(unchanged). `cargo doc --no-deps`: 0 warnings (was 0; the -agent introduced 4 transient broken-link warnings during the -work and resolved all of them before reporting done). `cargo -build --workspace` green. `cargo test --workspace` green. - -**Plan 13f / 14a unchanged.** 13f (`ailang-codegen` + `ail` -CLI) is the natural next polish iter; 14a (`List a` rewrite of -`list_map`) remains the next-feature default. Auto-mode is on, -so I'll continue into 13f directly unless context budget -pressures a switch. - -## Iter 13f — rustdoc polish for `ailang-codegen` + `ail` CLI - -Combined docwriter mission. Codegen has a small public -surface (only 3 top-level `pub` items: `CodegenError` enum + -7 variants, `emit_ir`, `lower_workspace`); the CLI is a -binary with zero `pub` items, so the only useful rustdoc is -the module header. One agent run, both files. - -What landed (148 LOC of doc additions, no non-doc lines -changed, verified by filtering the diff): - -- `ailang-codegen` crate root got intra-doc-link upgrades - (`[`emit_ir`]`, `[`lower_workspace`]`, - `[`CodegenError::MissingEntryMain`]`) and a precondition - sentence — both entry points assume their input has already - passed the typechecker; codegen does not call the checker - itself. -- Every `CodegenError` variant got a `///` string naming the - AST term/condition that triggers it. Same shape as - `CheckError` post-13e, so the two error enums now read - similarly and a reader can grep across them. `Internal` is - flagged in its doc as a catch-all that covers ~30 - invariant-violation sites. -- `emit_ir` and `lower_workspace` now make the - single-vs-multi-module split explicit and cross-link to - each other. -- `ail/main.rs` module header expanded from a 5-line stub to - a full subcommand list with one-liners (`manifest`, - `render`, `describe`, `deps`, `check`, `emit-ir`, `build`, - `run`, `builtins`, `diff`, `workspace`), the `clang`-on-PATH - prerequisite for `build`/`run`, the design-intent paragraph - about each subcommand being narrowly scoped for LLM - consumption (already partly there), and an explicit "no - `pub` items, `--help` text comes from clap" note for anyone - who lands here from rustdoc. - -**Findings reported** (judgement deferred to me): - -- `CodegenError::Internal(String)` is a single opaque catch-all - for ~30 distinct invariant-violation sites (mono-queue - desync, ctor-index miss, lambda-env shape, ...). Tests can - only substring-match on it. **Not fixing**: splitting is a - test-ergonomics decision, not a doc one. Worth raising the - next time codegen tests get a serious rewrite. -- `emit_ir` synthesises an internal `Workspace` with - `root_dir = "."`. No codegen path reads `root_dir` today, so - this is harmless; if a future feature reaches `root_dir` - from codegen, the assumption surfaces. **Not fixing**: the - agent flagged it correctly as "would change behaviour, out - of scope". - -**Process note: brief drift caught by the agent.** I told the -docwriter the CLI had nine subcommands; it found eleven -(`Deps` and `Diff` were missing from my brief, which was -written off a `head -40` of the source). Agent silently -corrected and flagged the drift in its findings. Useful -counter-pressure to the orchestrator pattern: my survey was -sloppy and the agent did not propagate the sloppiness into -the doc. This is one of the things sub-agents are good at and -why I keep delegating even on small jobs. - -**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests -(unchanged). `cargo doc --no-deps`: 0 warnings. Also verified -under `RUSTDOCFLAGS='-D rustdoc::broken_intra_doc_links'` per -agent report. `cargo build --workspace` green. `cargo test ---workspace` green. - -**Workspace-wide rustdoc invariant achieved.** All four -crates (`ailang-core`, `ailang-check`, `ailang-codegen`, -`ail`) now have: -- crate-root `//!` that names ownership, position in - pipeline, and entry points; -- module-root `//!` on every file with non-trivial content; -- `///` on every public item (struct, enum, variant, fn, - field, const) that names the contract, not just the type; -- intra-doc links wherever prose previously referred to - another item by name. - -DESIGN.md item 6 ("rustdoc cleanliness") is now load-bearing -across the whole workspace, not just `core`. The -`ailang-docwriter` agent's job from here is **maintenance**: -run after any iter that adds public surface, not full sweeps. - -**Next.** 14a is now unblocked: write a polymorphic `list_map` -that uses `List a` (Iter-13a parameterised ADTs) and `Maybe a`, -then extend an existing demo program (`hello_print` or one of -the dogfood sources) to call it end-to-end. That exercises -the parameterised-ADT pipeline through type-check, codegen, -and runtime — the missing piece in 13a/b/c was that the -feature shipped but no real program used it. If context -budget at the start of 14a is tight, alternative is 14b -(GC/arena scaffolding) or 14c (poly fn as value); both have -real design questions that need an orchestrator design pass -first, so 14a stays the default. - -## Iter 14a — polymorphic `List a` end-to-end + monomorphisation bug fixed - -The dogfood payoff for parameterised ADTs (13a/b/c). Prior -to this iter, the only programs exercising the feature were -`box.ail.json` (single `MkBox(42)` round-trip) and -`maybe_int.ail.json` (single `or_else` call). That is a thin -slice — neither program builds a recursive parameterised ADT -nor calls a polymorphic higher-order fn. 14a closes that gap -with `data List a` + `map : forall a b. ((a) -> b, List) -> -List` recursive, then prints the result. - -Three-agent run (tester → debugger → no implementer needed): - -1. **Tester** wrote `examples/list_map_poly.ail.json` (5 defs: - `List`, `inc`, `map`, `print_list`, `main`) and a new e2e - test `list_map_poly_inc_then_prints` that asserts stdout - `["2", "3", "4"]`. Typecheck passed, build crashed: - `internal: monomorphisation: var \`a\` bound to two - distinct types`. Tester correctly stopped — fixture - encodes the contract; bug is in the compiler — and - reported with a hypothesis (recursion / shared - substitution slot in `map`). - -2. **Debugger** refuted the hypothesis with a non-recursive - repro (`Cons(7, Nil)` triggers the same crash without - `map` in the picture at all). Real cause was much smaller: - in `synth_arg_type` (codegen, ~line 2087) for `Term::Ctor`, - any type var of the parent ADT that the ctor's args - couldn't pin was filled with `Type::unit()` as a - placeholder. For nullary ctors of a parameterised ADT - (`Nil : List`, `None : Maybe`) that placeholder - leaked upward. Inside a parent like `Cons(Int, Nil) : - List`, `unify_for_subst` would walk - `cref.ail_fields = [Var{a}, Con{List,[Var{a}]}]` against - `[Int, Con{List,[Unit]}]`, bind `a = Int` from the head, - then collide with `a = Unit` from the tail. The recursive - `map` fixture surfaced it because it is the first program - to nest a nullary ctor of a parameterised ADT inside a - parent ctor — the existing 13b regressions never did. The - tester's hypothesis was reasonable from the symptom but - wrong on mechanism; refuting it via a smaller repro is - exactly the discipline the debugger role is for. - -3. **Fix** (`crates/ailang-codegen/src/lib.rs`, +30/-9 LOC, - no new variant, no API change): - - Replace `Type::unit()` placeholder with a synth-only - wildcard `Type::Var { name: "$u" }`. The `$u` prefix is - a reserved-namespace convention that mirrors the - checker's `$m` for instantiation metavars — same - trick (source-level identifiers can't start with `$`), - same goal (extra semantics without schema change). - - `unify_for_subst` short-circuits on a `$u`-prefixed - **arg-side** var: accept without binding, let a sibling - arg pin the type var instead. Param-side semantics - untouched. `derive_substitution`'s "var not pinned" - check still fires for genuinely under-determined calls - (those would have failed typechecking, so codegen never - sees them, but the safety net stands). - -**Tests:** 25/25 e2e (was 24, +1 new poly test). All five -named regressions stayed green: `list_map_doubles_then_prints`, -`parameterised_box_round_trip`, `parameterised_maybe_match`, -`polymorphic_id_at_int_and_bool`, `polymorphic_apply_with_fn_param`. -Insertion sort still green. `cargo doc --no-deps`: 0 warnings -(workspace invariant from 13d/e/f preserved). `cargo build ---workspace` green. - -**Process note: tester→debugger→done in one iter.** No -implementer dispatch needed — the debugger's mandate covers -"propose **and apply** a minimally invasive fix" once the -diagnosis is solid. 30 LOC across two sites in one file is -inside the role's scope (the role doc says "stop and report" -only on >50 LOC across multiple files). The orchestrator-side -work was the design (the source program), the scoped briefs, -and verification. This is the cleanest agent-flow shape so -far: each agent did exactly its job, the tester's wrong -hypothesis didn't propagate because the debugger tested it, -and the orchestrator never wrote compiler code. - -**Findings flagged** (judgement deferred, not fixed): - -- `CodegenError::Internal` (the catch-all string variant the - docwriter flagged in 13f) is now used by one more invariant - — the `$u` short-circuit could in principle be reached by a - malformed input; it's currently silent because typecheck - rejects under-determined calls upstream. Worth splitting - into typed variants the next time codegen tests get a real - rewrite. (Same finding as 13f, now reinforced by another - use site.) -- The `$u` / `$m` reserved-prefix convention (`$u` for - codegen synth wildcards, `$m` for checker metavars) - is undocumented as a project-wide naming rule. Two - prefixes is fine; if a third appears, this should be - promoted to a DESIGN.md note. - -**Next.** Parameterised ADTs are now genuinely usable for -real programs. The standard library starter set (an -`examples/std_*` series with `List`, `Maybe`, `Either`, -basic combinators `length`, `filter`, `fold`, `concat`) -becomes worthwhile in a way it wasn't pre-14a — that's a -plausible 14b alternative, smaller than GC/arena, and would -itself surface more dogfood bugs. The original 14b -(GC/arena) and 14c (poly fn as value) remain on the queue -but have not had a design pass. - -## Iter 14b — design pass for the authoring surface - -User redirected the project at the iter boundary. I had been -about to write a stdlib in JSON-AST form; user pushed back: do -I really program *best* in JSON, given the language is supposed -to be the one I'm most accurate in? Honest answer was no — JSON -was rationalisation. The constraint added in this iter: - -> "Die Syntax sollte gut formalisierbar sein, damit man auch -> einem fremden LLM eine Spec geben kann, die es dann fehlerfrei -> befolgt." - -That single sentence ruled out about half the design space: -anything with operator precedence (precedence is the #1 thing -an LLM gets wrong when handed a spec, because it requires -building a parse-tree mental model rather than just following -production rules), anything with semantic indentation, anything -with maximal-munch lexing or context-sensitive reductions. - -What remains is roughly: S-expressions, or dialects close to -S-expressions. Decision 6 in DESIGN.md captures the constraints -in priority order, sketches three candidate forms (A: tagged -S-expression, B: indented record-style, C: pretty-printer-as- -source), and picks (A) as the first attempt with explicit -rollback to (C) if (A) hurts authoring more than it helps. - -**Key shape of (A):** every AST node has a unique head keyword. -No case-disambiguation rule (no "capitalised head means ctor"). -Bare atoms in positional slots get their sort from the parent -slot (inside `(con NAME args...)` second-and-later positions -are types; inside `(app HEAD args...)` first position is a term; -etc.). To construct a value with a ctor, write -`(term-ctor TypeName CtorName args...)` explicitly. To match, -`(pat-ctor CtorName fields...)`. The capitalisation/case of an -identifier carries no semantic weight to the parser. - -This rules out the silent-error class "I forgot to capitalise -`Cons` and it parsed as a function call" — which I had been -relying on case-conventions to prevent in the JSON form too, -but only by hand-discipline. With explicit tags the discipline -becomes a parse rule. - -**Empirical test (this iter).** Hand-encoded three examples in -form (A) as `examples/*.ailx`: - -``` -hello.ailx — 5 LOC (was JSON: 36 pretty / 21 canonical) -box.ailx — 25 LOC (was JSON: 160 pretty / 88 canonical) -list_map_poly.ailx — 50 LOC (was JSON: 394 pretty / 230 canonical) -``` - -Roughly 4–8× line reduction, ~4× character reduction. Bigger -gains on bigger programs (overhead is proportional to AST -depth, not to program size, so the form scales well). All three -mapped unambiguously to AST nodes by inspection — no -ambiguities surfaced during writing that the spec didn't -already cover. - -Two small lex/grammar issues I discovered while writing and -folded back into DESIGN.md before committing: - -1. **Operator idents** like `+`, `==`, `<=`. Initial spec had a - word-shaped `[A-Za-z_]...` regex; that excluded operators. - Fix: lexer recognises only `(`/`)`/whitespace as delimiters. - Every other maximal token is classified by first character - (digit ⇒ integer, `"` ⇒ string, else ⇒ ident). `+`, `42`, - `io/print_int`, `==` all become single ident or integer - tokens with no special rule. -2. **Bool literals.** Bare `true`/`false` are reserved in term - context; outside term context they would be ill-formed - anyway. Unit is explicit: `(lit-unit)`. - -**Files touched:** -- `docs/DESIGN.md` — Decision 6 added (~140 lines), with - constraints, three candidates, first-choice rationale, and - an implementation outline for Iter 14c. -- `examples/hello.ailx`, `examples/box.ailx`, - `examples/list_map_poly.ailx` — three design exhibits. Not - parseable yet (header comment says so). -- `docs/JOURNAL.md` — this entry. - -**Tests:** None new. Existing 25/25 e2e + 3 ignored doctests -unchanged (this iter is paper, not code). `cargo doc --no-deps` -0 warnings (workspace invariant from 13d/e/f preserved). - -**Process note: orchestration with explicit licence to be -wrong.** User said "du darfst auch ausprobieren und dich irren -(und es dann rückgängig machen). Wir wollen das beste Design -für den propagierten Zweck." That changes the cost model for -design iteration: I should optimise for *information per iter*, -not for *correctness on first commit*. So I committed form (A) -as a working hypothesis with a documented rollback path to -form (C), rather than design-by-committee until I was sure. - -**Plan 14c (next).** -1. New crate `ailang-surface` with a small PEG parser → existing - `ailang-core::ast` types. No new AST nodes. -2. Round-trip test gate: every existing `examples/*.ail.json` - gets a sibling `*.ailx` written by hand or by an - AST→surface emitter; the test parses the surface, canonises - to JSON, and asserts hash-equivalence to the original. If a - single fixture loses its hash, the form does not ship. -3. CLI: `ail parse -o `. Symmetric - to existing `ail render`. -4. If round-trip works for all current fixtures, mark form (A) - confirmed and start the stdlib in `.ailx` directly. If it - fails, document why in JOURNAL and try (C). - -**Plan 14d (after 14c).** First stdlib module: `std_list.ailx` -with `length`, `filter`, `fold`, `concat`, `reverse`, `head`, -`tail`. Each combinator is a fresh test vector for codegen and -for the still-young parameterised-ADT pipeline. Written in the -new surface from day one — no JSON authoring of stdlib. - -## Iter 14c — `ailang-surface` parser + pretty-printer ships - -Implements Decision 6's form (A). New crate `ailang-surface` -(~1843 LOC across `lex.rs`, `parse.rs`, `print.rs`, lib root, -plus tests) ships as a strictly additive producer of -`ailang-core::ast::Module` values. `ailang-check` and -`ailang-codegen` were not modified — projection-agnostic, as -the architectural pin requires. - -**Implementer dispatch went clean.** Brief gave the EBNF, the -fixtures to round-trip, the lexer rule, and the architectural -constraints. Implementer hand-wrote a recursive-descent parser -(one Rust fn per EBNF production), a deterministic pretty- -printer, an integration test that runs the round-trip gate on -every fixture, and the `ail parse` CLI subcommand. No -parser-combinator dependency. No AST-shape changes. - -**Two AST-driven form refinements** that were not in the 14b -sketch (both folded into DESIGN.md Decision 6 by the -implementer before commit): - -1. `lam-term` had to carry `param_tys`, `ret_ty`, and - `effects` because the AST's `Term::Lam` stores parallel - typed-param data. Production became `(lam (params (typed - x Int) ...) (ret T) (effects ...) (body ...))`. Same - shape-style as the rest of the form; no new lex rules. -2. `import-clause` had to admit `Option` aliases. - Production became `(import name (as alias)?)` with `as` - as a bare ident token. No new lex rules. - -The 14b 30-production budget held: implementer reports ~28 -named productions in the parser. Constraint 1 (formalisable -for foreign LLM) intact. - -**Verification gates, all green:** - -- `cargo build --workspace`: 0 warnings, finished. -- `cargo test --workspace`: 76 tests pass (was 64; +9 unit - tests in `ailang-surface`, +2 integration tests in - `tests/round_trip.rs`, +1 e2e regression preserved). All - 17 `examples/*.ail.json` fixtures round-trip - byte-identical through `print → parse → canonical JSON`; - 3 hand-written `.ailx` exhibits parse to canonical JSON - byte-identical to their `.ail.json` siblings. -- `cargo doc --no-deps`: 0 warnings (workspace invariant - from 13d/e/f preserved; new crate's rustdoc landed - correctly with crate-root `//!` plus all `pub` items - documented). - -**Manual smoke test (orchestrator-side after agent reported -done):** `ail parse -o ` followed -by `ail run ` for all three exhibits: - -``` -hello.ailx → "Hello, AILang." -box.ailx → "42" -list_map_poly.ailx → "2\n3\n4" -``` - -End-to-end pipeline form (A) → AST → typecheck → codegen → -clang → binary works on all three. - -**Important non-issue.** `examples/*.ail.json` fixtures on -disk are hand-formatted JSON with non-canonical key order; -the parser produces canonical (lex-sorted) JSON. Diff at -the file-byte level is not zero. Diff at the canonical-byte -level is zero — which is the only thing that matters per -Decision 1, since hashing uses canonical form. The -round-trip test gates on canonical bytes, not file bytes. -This is correct behaviour; flagged here so a future reader -who runs `diff` doesn't think the surface is broken. - -**Files created:** - -- `crates/ailang-surface/Cargo.toml` -- `crates/ailang-surface/src/lib.rs` (~40 LOC, rustdoc heavy) -- `crates/ailang-surface/src/lex.rs` (~264 LOC) -- `crates/ailang-surface/src/parse.rs` (~1041 LOC, one fn - per production) -- `crates/ailang-surface/src/print.rs` (~371 LOC) -- `crates/ailang-surface/tests/round_trip.rs` (~128 LOC) - -**Files modified:** - -- `Cargo.toml` (workspace) — `ailang-surface` member + - workspace dep. -- `Cargo.lock` — refresh. -- `crates/ail/Cargo.toml` — `ailang-surface` dep. -- `crates/ail/src/main.rs` — new `Parse { path, output }` - subcommand (~36 LOC). -- `docs/DESIGN.md` — form refinements appendix to - Decision 6. -- `examples/list_map_poly.ailx` — implementer added doc - strings to match the JSON original (was a 14b design - exhibit, not byte-aligned). - -**Known debt:** none reported. `ailang-check` / -`ailang-codegen` untouched per the architectural pin. - -**Plan 14d (next, queued).** With form (A) now ergonomic -and round-trip-verified, the std-lib path opens up. First -target: `examples/std/std_list.ailx` with `length`, -`filter`, `fold` (left and right), `concat`, `reverse`, -`head`, `tail`. Each combinator a fresh test vector for -the parameterised-ADT pipeline (which 14a opened end-to- -end). Authored in form (A); the resulting `.ail.json` is -what tests consume. If 14d surfaces more codegen bugs in -the parameterised-ADT path (it likely will — 14a found -one already), debugger handles them inline. - -The 14b/c form-A hypothesis has held under the empirical -test of round-tripping every existing fixture. The -documented rollback to form (C) is now off the table for -this iter cycle, though the architectural pin keeps it -open for future replacement of `ailang-surface` should -the form prove inadequate at stdlib scale. - -## Language-completion sequence (14d → 14f, then stdlib in 15a) - -User redirected at the 14d boundary: write the language to -"finished" before starting on a stdlib. Reasoning: -authoring a stdlib in an unfinished language wastes work — -each gap discovered later forces a rewrite of code already -written. The user also confirmed: **no schema version bump -needed**. AILang has exactly one consumer (me), so version -ceremony for compatibility management is pure overhead. -Edit AST and fixtures in place; pin new hashes where the -hash regression test demands it. - -Updated planning sequence: - -- **14d** — remove `Term::If` redundancy. Pure subtraction. -- **14e** — explicit tail-call annotation (`tail` flag on - `Term::App`/`Term::Do`, `musttail` in codegen, tail-position - verifier in checker). -- **14f** — memory management. Currently every ADT allocation - leaks. Likely Boehm conservative GC (`GC_malloc` + `-lgc`) - for minimum surface change; design pass first. -- **15a** — first stdlib module (`std_list`). - -Deferred (not stdlib-blocking; can land later without -rewriting code that already exists): records/tuples (use -ADT pairs), nested patterns (use pyramid `match`), local -recursive `let` (hoist to top level). - -## Iter 14d — `Term::If` removed - -Subtraction iter. `Term::If { cond, then, else_ }` was -semantically a subset of `Term::Match` on `Bool`. CLAUDE.md -forbids redundancies; two AST nodes for the same operation -was an authoring decision with no semantic content and a -duplicate codegen path. - -The migration shape was the canonical one named in the -brief: - -``` -(if c a b) → (match c (case (lit-bool true) a) (case _ b)) -``` - -Wildcard arm satisfies the existing `primitive-needs-wildcard` -rule. A future iter may upgrade exhaustiveness to recognise -`true`+`false` as covering Bool without a wildcard, but the -wildcard form works with the current checker and that was -enough for 14d. - -**Implementer dispatch went clean with one documented -deviation.** Removing the `Term::If` codegen path was not -sufficient on its own: the existing match codegen rejects -`i1` (Bool) scrutinees and `Pattern::Lit` patterns — both -of which the migration shape requires. The implementer -added a tightly-scoped `lower_bool_match` helper (~95 LOC) -that handles **only** the two-arm Bool migration shape -(`(lit-bool true) -> A | _ -> B` or its mirror), errors on -anything else, and emits the same `br i1`/phi IR the old -`Term::If` path emitted. No generalisation of the -ADT-match codegen. - -The deviation was the right call. **Lesson for future -subtraction iters**: when removing a specialised AST node, -the codegen for the migration target may need a small -extension. Pre-emptively scope this in the brief next time. - -**Diff size**: 13 files, +286/-221 LOC. Net +65 LOC across -the workspace, but the AST got smaller (one variant gone), -the form-(A) grammar got smaller (one production gone), -and the typechecker got smaller (one branch gone). Codegen -got slightly larger because of the bool-match helper, but -the alternative was reusing the existing match path and -generalising it — which would have been a bigger and -riskier change. - -**Hash deltas** (intentional, per Decision 7): - -| def | before | after | -|---|---|---| -| `sum.sum` | `db33f57cb329935e` | `7f5fe7f72c63a9fd` | -| `sort.insert` | `697fcb9f30f8633a` | `07ff6ee7db17565d` | -| `max3.max` | `65c45d6a45dd0a72` | `2aa1576f3fbf5b3d` | -| `max3.max3` | `624b14429bf302f5` | `c452ec2e36c0af27` | - -Untouched defs (e.g. `sum.main`, all `sort.*` except -`insert`, `sort.IntList`, `sort.print_list`, `max3.main`) -keep bit-identical hashes. That's the canary that the -canonical-JSON byte format was not perturbed — only the -migrated bodies changed identity. - -**Verification**: 76/76 tests green (unchanged count; no -new tests in this iter, by design — subtraction). Manual -smoke: `sum` → `55`, `max3` → `17`, `sort` → ordered list. -Identical to pre-migration stdout for all three. `cargo -doc --no-deps` 0 warnings. - -**Tail-call survey from the implementer (gold finding, -informs 14e).** While reading the migrated fixtures the -implementer surveyed tail positions. Result: - -- `print_list` (in both `sort.ail.json` and - `list_map_poly.ail.json`): the recursive call is the - rhs of a `seq` which is the body of a match arm — - **already in tail position**. TCO would convert these - to actual loops. -- `main` chains: the outer call is in tail position; - inner calls are not. -- `insert`, `sort`, `map`: the recursive calls are - **arguments to a `Cons` ctor construction** (e.g. - `Cons (f h) (map f t)`). NOT in tail position. - Constructor-blocking is the standard ML/Haskell case - where TCO does not apply without a CPS transform or an - accumulator-form rewrite. - -**Implications for 14e.** Adding a `tail` flag to -`Term::App`/`Do` will work for `print_list`-style -recursions and for terminal call chains, but **will not** -help the `map`/`sort`/`insert`-style ctor-blocked -recursions. Those need either an accumulator-form rewrite -in the source program (the standard ML/Haskell move) or a -CPS transform (much more intrusive). 14e ships only the -annotation + verification; accumulator forms become an -authoring pattern in the stdlib, not a compiler feature. - -This sharpens what 14e can promise: tail-call **wins** -will be visible in `print_list`-style terminal-recursion -patterns; `map`/`sort` style stays stack-bounded by depth, -which makes 14f (GC) the more important iter for handling -long lists than 14e by itself. - -## Iter 14e — explicit, verified tail calls - -Decision 8 ships. `Term::App` and `Term::Do` carry a `tail: bool` -flag (serde-default false, skip-when-false in serialisation). -A new `verify_tail_positions` typecheck pass walks each fn body -and Lam body with an `is_tail_context: bool` threaded down per -the standard Scheme rules, rejecting any `tail: true` call that -sits outside tail position. Codegen emits `musttail call` for -marked App calls. - -**Hash deltas (intentional, only the migrated calls):** - -| def | hash before | hash after | -|---|---|---| -| `list_map_poly.print_list` | unchanged 14c value | new | -| `sort.print_list` | unchanged 14d value | new | - -All other defs across all 18 fixtures kept bit-identical hashes. -This is the canary for the `skip_serializing_if = "is_false"` -serde rule: an unmarked `App`/`Do` serialises identically to its -pre-14e form, so untouched defs cannot drift. - -**Tests: 79/79 (was 76, +3).** New tests: -- `tail_call_in_non_tail_position_is_rejected` (check unit) — - asserts the diagnostic fires on a deliberately-misplaced - `tail: true` call (e.g. as a `Cons` arg). -- `tail_call_in_tail_position_is_accepted` (check unit) — - asserts the verifier accepts the canonical `print_list` shape. -- `iter14e_print_list_recursion_emits_musttail` (e2e IR-grep) — - builds `list_map_poly`, dumps IR, asserts the recursive call - site uses `musttail call`. This is the only direct evidence - that the AST flag actually reaches LLVM. - -**IR-snapshot evidence** at the recursive site of -`print_list` after migration: - -``` -%v7 = musttail call i8 @ail_list_map_poly_print_list(ptr %v6) -ret i8 %v7 -``` - -`musttail` followed immediately by `ret` of the same SSA value — -LLVM's terminator rule satisfied. Smoke: `list_map_poly` → -`2/3/4`, `insertion_sort_orders_list` → identical sorted list. -Behavior unchanged; only the calling shape did. - -**Two implementer deviations (called out, both reasonable):** - -1. **`tail-do` falls back to `tail call`, not `musttail`.** LLVM - `musttail` requires identical caller/callee return types. AILang - IO ops dispatch through runtime helpers (`printf`/`puts`) - returning `i32`, while AILang's `Unit` lowers to `i8`. Cross-type - `musttail` would be rejected by the verifier. So `Term::Do` with - `tail: true` lowers to `tail call` (the LLVM optimisation hint, - not the guarantee), then `ret i8 0`. No fixture currently uses - `tail-do`, so the path is implemented but not exercised - end-to-end. Proper fix: change runtime helper signatures to - return `i8`. Punted; not blocking. - -2. **`block_terminated` plumbing in codegen.** A `tail-app` / - `tail-do` emits `musttail call ... ret ...` directly and sets - `self.block_terminated = true`. Surrounding code (match-arm phi - construction, fn-body trailing-ret, lambda-thunk trailing-ret) - checks the flag and skips the fall-through emit. When every - match arm is a tail call, the join block is omitted entirely. - This was unavoidable to keep the IR well-formed — adding a - second `ret` after a `musttail call`+`ret` would be a verifier - error. The flag is a small piece of state but it's the right - shape for "the current basic block has been definitively - terminated by a sub-emission". - -**Form (A) at constraint ceiling.** Two new productions -(`tail-app-term`, `tail-do-term`) bring the count to ~30, which -is exactly the constraint-1 budget. Future productions need to -either retire something or accept an explicit budget rebalancing -in DESIGN.md. - -**GC notes from the implementer (informs 14f).** - -- **Allocations cluster in `lower_ctor`** (~line 850 of codegen). - Every `term-ctor` does `malloc(8 + 8 * n)`. In `print_list` we - allocate nothing per recursion (just match + read fields + - recurse); allocations come from `map`, from `main`'s - list-building Cons chain, and from any other user code that - builds ADTs. -- **Lambda envs and closure pairs allocate too** (`lower_lambda`). - Closure pair: `malloc(16)`. Env block: `malloc(env_size)`. - Direct-application closures (the common case for HOF args) - could be arena'd cleanly because the closure dies after the - call returns. Stored or returned closures escape. -- **Tail recursion does NOT reduce allocation pressure**, only - stack depth. For `print_list`-style recursions there's no - allocation to begin with, so the win is purely stack-bounded. - For `map`-style ctor-blocked recursions, each step allocates - one new `Cons` box — that's where allocation-side work pays - off. -- **The "obviously safe" arena boundary** is a fn whose return - type contains no boxed ADT (i.e. returns `Int`/`Bool`/`Unit`/ - `Str` only). All ADT boxes allocated inside such a fn cannot - escape; an arena freed at fn return is sound by construction. - Most current fixtures violate this — `map`, `sort` return ADTs - — so a per-fn-arena scheme alone won't carry. Need a heap - with GC for escaping allocations. - -This narrows 14f's design space: probably **Boehm conservative -GC across the board** as a first cut (`GC_malloc` substituted -for `malloc`, `-lgc` linked, no language change). Add a -per-fn-arena optimisation later for non-escaping cases if the -profile justifies it. Boehm is a one-iter shot; arenas would be -a multi-iter design pass with escape analysis. - -**Plan 14f.** Boehm-GC integration. Concretely: -- `GC_malloc` instead of `malloc` in lowered IR. -- `-lgc` added to the clang link command in `ailang-codegen`'s - build path (probably in the CLI, since the codegen crate - emits IR text and clang is invoked downstream). -- Conservative scan handles AILang's stack and globals out of - the box. -- Verify on a stress test: build a list of 100k Cons cells in - `map`, run, observe RSS doesn't blow up. Boehm collects - unreachable boxes during allocation pressure. -- No AST or schema change. No language-level change. - -After 14f, the language is "done enough" for stdlib (15a). -Anything else (records as a primitive, nested patterns, local -recursive let, type classes) can layer on later without forcing -stdlib rewrites. - -## Iter 14f — Boehm conservative GC - -Decision 9 ships. Through Iter 14e every ADT box, lambda env, -and closure pair was leaked. This iter substitutes -`GC_malloc` for `malloc` in all four IR allocation sites and -links `-lgc`. No language change, no AST change, no schema -change. - -**Diff: 5 files, ~30 LOC net.** - -- `crates/ailang-codegen/src/lib.rs`: 4× `@malloc` → `@GC_malloc` - (declare line + 3 call sites: `lower_ctor`, `lower_lambda`'s - env, `lower_lambda`'s closure pair). -- `crates/ail/src/main.rs`: `.arg("-lgc")` added to the clang - invocation in `build_to`. -- `crates/ail/tests/snapshots/{hello,sum,list,max3,ws_main}.ll`: - mechanical s/@malloc/@GC_malloc/, 9 occurrences across 5 - files. The IR is bit-identical to pre-14f modulo this - substitution — exactly Decision 9's promise. -- `crates/ail/tests/e2e.rs`: new test - `gc_handles_recursive_list_construction` (+19 LOC). -- `examples/gc_stress.{ailx,ail.json}`: new fixture. - -**Hash invariance verified.** Every existing fixture's def -hashes are unchanged. The codegen and link line are downstream -of canonical bytes; the AST schema didn't move; nothing on -disk in `examples/*.ail.json` was touched. The new -`gc_stress` module adds 4 new hashes, all unrelated. - -**Tests: 80/80 (was 79).** Existing 79 produce byte-identical -stdout — only the allocator changed, semantics unchanged. New: -`gc_handles_recursive_list_construction` builds a List of -length 50 via recursive `Cons`, sums it (`1275`). Manual smoke: - -- `gc_stress.ail.json` → `1275`. -- `list_map_poly` → `2 3 4` (unchanged). -- `sort` → sorted list (unchanged). - -`cargo doc --no-deps`: 0 warnings (DESIGN.md item 6 invariant -preserved through nine iters of feature work). - -**Pattern shape used in `gc_stress`** (caught a small typechecker -constraint). The first proposed shape `(case (lit-int 0) Nil)` -doesn't parse — `pat-lit` takes the bare literal token, not a -keyword-prefixed form, and `case` requires a pattern. Worked -shape: comparison-and-bool-match, mirroring `sort.ail.json`'s -`<=` arm: - -``` -(match (app == n 0) - (case (pat-lit true) (term-ctor List Nil)) - (case (pat-wild) (term-ctor List Cons n (app build (app - n 1))))) -``` - -This is the canonical "if-then-else" pattern post-14d. Worth -flagging for the stdlib brief: predicates that need to branch -go through the `==` / `<` / `<=` builtin returning Bool, then -match on that Bool with a wildcard fallback. Three lines for -what `if` used to do in one — but uniform with the rest of the -language, no special case. - -**GC integration notes.** -- `GC_INIT()` is **not needed** on this build host (Arch - with `libgc 1.5.6`). libgc auto-inits via - `__attribute__((constructor))`. -- No conservative-scan over-retention symptom observed: every - existing test's stdout byte-identical; behaviour preserved. -- `-lgc` alone is sufficient for the link; pthread/dl come in - transitively from libgc.so's NEEDED entries. - -**Language is feature-complete enough for stdlib.** Iters 14d -(redundancy removal), 14e (explicit tail calls), 14f (GC) are -the three blockers identified at the 14b boundary. They are -all done. Anything else (records as primitive, nested patterns, -local rec let, type classes) layers on later without forcing -stdlib rewrite. - -**Plan 15a.** First stdlib module: `examples/std/std_list.ailx`. -Combinators: `length`, `append`, `reverse`, `map`, `filter`, -`fold_left`, `fold_right`, `head`, `tail`, `is_empty`. Each -combinator a fresh test vector for the parameterised-ADT + -GC + tail-call combination. Authored in form (A) from day one; -`.ail.json` produced via `ail parse`. Each combinator gets a -dedicated e2e test. - -If 15a surfaces compiler bugs (likely — every prior dogfood -iter has, see 14a's monomorphisation bug), debugger handles -them inline. If a compiler limitation surfaces that genuinely -blocks the stdlib (e.g. nested patterns turn out to be needed), -that becomes its own iter before 15a continues. - -The architectural pin from Decision 6 governs: stdlib lives -under `examples/std/` as `.ailx` source; tests load the -generated `.ail.json`. `ailang-check` and `ailang-codegen` -remain projection-agnostic. - -## Iter 14g — `Term::If` restored (revert of 14d) - -Reconsidered 14d's removal of `Term::If`. The decision was wrong; -restored. - -**Why 14d was wrong.** "No redundancies" from CLAUDE.md is a real -rule but it requires judgment to apply. `Term::If` reduces to -`Term::Match` on Bool, but reducibility is not redundancy in a -strong sense — `1 + 1` reduces to `2`, you don't remove `+` from -the language because of it. `Term::If` is a primitive control- -flow shape that every programming language has for good reason: -bool branching is the second most common control flow shape after -sequencing. - -**Quantitative.** `(if c a b)` is 4 tokens. The post-14d -replacement `(match c (case (pat-lit true) a) (case (pat-wild) b))` -is 12. That's a 3× token-economy hit on every Bool branch — in -exactly the language whose authoring constraint was supposed to -be token-efficient. The match-on-Bool form is also asymmetric -(false case via `pat-wild` because the typechecker rejects -`pat-lit false` as exhaustive) and structurally lopsided. - -**Meta-pattern that produced the wrong call.** I had been treating -user observations as directives. The user said "if is a subset of -match" — which is a factual observation; I jumped to remove it, -citing CLAUDE.md as cover. There was no independent conviction -behind the change, only doctrinal hooking-up of a user remark. -The leak showed up in 14f's JOURNAL prose ("three lines for what -`if` used to do in one"), which the user correctly read as me -regretting the decision. - -Two feedback memories saved to head this off in future iters -(`/home/brummel/.claude/projects/-home-brummel-dev-ailang/memory/`): -- `feedback_user_suggestions_not_directives.md` — observations - are input, not output. Form an opinion before acting. -- `feedback_no_nostalgia_for_removed_features.md` — describe - canonical form on its own merits, not as compensation for - what was deleted. - -**Implementation (revert).** Mechanically reverse-applied 14d's -diff at every site (`ast.rs`, `check/lib.rs` (incl. the new -14e `verify_tail_positions` arm), `codegen/lib.rs` (4 sites), -`surface/{parse,print}.rs`, `core/pretty.rs`, `ail/main.rs`, -`e2e.rs` test mutation). Removed the `lower_bool_match` helper -that 14d had introduced — it existed only because the 14d -migration shape needed codegen for non-`ptr` match scrutinees; -with `Term::If` back, match-on-Bool returns to its pre-14d -unsupported state and the helper is dead weight. Three fixtures -(`sum`, `sort`, `max3`) restored to their pre-14d shape. - -**One additional fixture migration.** `gc_stress` was authored -in 14f using the 14d match-on-Bool migration shape (because -14f sat between 14d and this revert). After removing -`lower_bool_match` it would have failed to compile. Migrated -`gc_stress.{ail.json,ailx}` to use `(if ...)` directly. Output -unchanged: `1275`. - -**14e and 14f are intact.** Verified by spot-emit of -`list_map_poly`'s IR: `musttail call i8 @ail_list_map_poly_print_list` -and `call ptr @GC_malloc(i64 8)` both present. The revert is -strictly local to `Term::If`-related code paths. - -**Hash check.** All four pre-14d hashes returned: - -| def | restored hash | -|---|---| -| `sum.sum` | `db33f57cb329935e` | -| `sort.insert` | `697fcb9f30f8633a` | -| `max3.max` | `65c45d6a45dd0a72` | -| `max3.max3` | `624b14429bf302f5` | - -Untouched defs across all 18 fixtures (incl. the 14e print_list -hash deltas) keep their post-14f hashes. The revert's hash -movement is exactly the four 14d-migrated defs reverting plus -the one accidental 14f-victim (`gc_stress` defs, never previously -shipped under any other hash). - -**DESIGN.md.** Decision 7 is preserved with a `Status: REVERTED` -header. Audit trail matters; future reads should see the -decision and its reversal both. Form-(A) productions in -Decision 6's appendix have `if-term` restored. - -**Tests: 80/80 green.** Identical stdout for every existing -fixture. `cargo doc --no-deps` 0 warnings. - -**LOC delta.** +265/-295 net −30. Net cleanup: the `lower_bool_match` -helper was bigger than the restored `Term::If` codegen. - -**Plan.** Back to 15a — first stdlib module `std_maybe`. The -brief I had drafted included an "authoring note: post-14d -if-then-else" section that's now obsolete. Re-issue without -that, using `if` naturally where appropriate. - -## Iter 14h — cross-module parameterised-ADT import (15a unblocked) - -The 15a tester surfaced exactly the kind of bug a first-real- -stdlib-iter is supposed to surface: cross-module references to -types and ctors were not implemented. The Iter 5b cross-module -mechanism only carried fns + consts via `module_globals`; types -and ctors stayed module-local with an explicit comment in -`crates/ailang-check/src/lib.rs:703`: *"Register type defs (local -per module; cross-module ADT sharing is explicitly not part of -5b)"*. This iter completes that work using the same convention -as fns: **qualified-only access via `module.Name`**. - -(Note on git tidiness: the `examples/std_maybe.ailx` file was -authored by the cancelled 15a tester dispatch and got swept into -the 14g commit by a `git add -A`. Should have spotted it pre- -commit. Not a correctness issue — the file was complete and -correctly authored — but a process-hygiene one. Will check the -diff carefully before staging next time.) - -**The bug, surfaced by the 15a demo.** - -``` -ail check examples/std_maybe_demo.ail.json --json -[{"severity":"error","code":"unknown-type", - "message":"unknown type: `Maybe`","def":"main","ctx":{}}] -``` - -After qualifying the fn calls (`std_maybe.from_maybe`), fn refs -worked but the `(con Maybe (con Int))` and `(term-ctor Maybe -Just 7)` kept failing because `env.types` and `env.ctor_index` -are populated only from the current module. - -**The fix.** Same shape as Iter 5b's fn solution, applied to types -and ctors: - -- `Env` gains `module_types: BTreeMap>` populated by a sibling `build_module_types` to - `build_module_globals`. Lives in `check_in_workspace`'s - pre-check pass. -- Type resolution in `(con NAME args)`: if `NAME` contains exactly - one `.`, split into `module.type` parts and resolve via - `env.module_types[module]`. Else current behaviour. -- Term-ctor resolution in `Term::Ctor { type, ctor, args }`: same - split rule on the `type` field. The `ctor` field stays - unqualified — once the type is resolved, ctor lookup is - unambiguous within the type def. -- Pattern-ctor resolution: when the bare ctor name doesn't resolve - in the local `ctor_index`, fall back to scanning imported - modules' types. Conflict rule: local always wins; if multiple - imported modules declare the same ctor name, error with the - new diagnostic code `ambiguous-ctor`. -- Codegen mirrors: a workspace-level `module_ctor_index` replaces - the per-Emitter table. `lookup_ctor_by_type` / `lookup_ctor_in_pattern` - thread qualified type names through the box-tag and field-type - resolution paths. - -**Diff size: 4 files, ~550 LOC net.** `ailang-check`: +311 -(env + four resolution sites + 4 unit tests). `ailang-codegen`: -+230 (workspace ctor index + qualified type-name handling). One -new diagnostic code. Demo updated to use `std_maybe.Maybe` at -type-name slots. - -**Tests: 85/85 (was 80, +5).** Four new unit tests in -`ailang-check` covering: qualified type ref, qualified term-ctor, -pat-ctor cross-module fallback, pat-ctor ambiguous-ctor -diagnostic. One new e2e test `cross_module_maybe_demo` asserts -stdout `["7", "99", "true", "true", "42"]`. - -**Hash invariance: confirmed.** All five `std_maybe` def hashes -unchanged (`Maybe 0fb8eaacba5e1135`, `from_maybe caf8eeaca800c80d`, -`is_some c09002048ff1ff6e`, `is_none 144e131340b58bd3`, -`map_maybe 68d83d84799322fa`). All other 80-test-suite fixtures -retain bit-identical hashes — the cross-module support is purely -additive at the language level. - -**14a-era regressions held.** Spot-checked -`parameterised_box_round_trip`, `parameterised_maybe_match`, -`list_map_poly_inc_then_prints`, `polymorphic_id_at_int_and_bool` -— all green. The 14h `derive_substitution` change (default -unpinned forall vars to `Unit` for the monomorphiser) sits on a -different layer than 14a's `synth_arg_type` `$u`-wildcard fix -(for nested ctor type synth). Both coexist: - -- 14a's `$u`-wildcard short-circuits unification when a sibling - arg pins the same type var concretely. -- 14h's Unit default applies when no arg pins a forall var at - all (e.g. `is_none(Nothing)` — `a` in `Maybe` is genuinely - unobservable from `Nothing`). - -**Implementer note (flagged for future):** the Unit default -produces a single shared monomorphisation for all such -unconstrained-`a` call sites. Wasteful but correct. If the -stdlib grows toward overload-resolution-style cases where -unconstrained instantiations need to be distinguished, the -descriptor strategy needs rethinking. Punted; not a 15a blocker. - -**std_maybe stdlib effectively ships.** Module + four -combinators + e2e-tested consumer demo. The cross-module -parameterised-ADT pipeline is the missing piece that 13a/b/c -(parameterised ADTs) and 5b (cross-module fns) could not by -themselves cover. This iter closes that loop. - -**Plan 15b.** Now that `Maybe` is reusable across modules, -write `std_list.ailx` importing `std_maybe`. Combinators: -`length`, `head` (returns `Maybe`), `tail` (returns -`Maybe>`), `is_empty`, `append`, `reverse`, `map`, -`filter`, `fold_left`, `fold_right`. `head`/`tail` exercise the -cross-module Maybe-returning case. `fold_left` is the -tail-recursive variant and gets `(tail-app ...)` markers; the -constructor-blocked combinators (`map`, `filter`, `append`, -`fold_right`) stay unmarked. If a new compiler bug surfaces -during stdlib construction (each prior dogfood iter has surfaced -one), debugger handles it inline. - -## Iter 15b — `std_list` ships, three more compiler gaps closed - -Second stdlib module. Tester wrote `std_list.ailx` (164 LOC, 10 -combinators) plus `std_list_demo.ailx` (consumer importing both -`std_maybe` and `std_list`). `std_list.ail.json` typechecked -cleanly in isolation. The demo did not — the prediction "every -dogfood iter surfaces at least one compiler bug" held three -times over. - -**Tester's diagnosis** was sharp enough that the orchestrator -could go straight to implementer without a debugger round: - -> Iter 14h's `qualify_local_types` is applied at `Term::Var` -> cross-module lookup but **not** to ctor-field types when -> `Term::Ctor` is synthesized. For `Cons a (List a)`, `List` -> stays unqualified in `cdef.fields`. `std_maybe` slipped -> through because its ctors have no recursive Con field. First -> recursive ADT shared cross-module triggers it. - -**The original-spec fix** was ~10 LOC across two sites in -`ailang-check/src/lib.rs` — apply `qualify_local_types` over -`cdef.fields` before substituting forall vars, in both -`Term::Ctor` synth and `Pattern::Ctor` resolution (the latter -symmetric, no current consumer hit it but it's the same -underlying gap). - -**Two more bugs surfaced during implementation** of that fix: - -1. **Codegen-side qualify-fields, four sites.** `ailang-codegen` - has its own field-type tracking that mirrored the check-side - bug. Symmetric fix needed in `Term::Ctor` synth + `lower_ctor`, - in `lower_match` for cross-module ADT scrutinees, and a tweak - to `unify_for_subst` to recurse instead of strict-equality - when re-binding a forall var that already has a previous - concrete binding (so a sibling-derived `List` accepts a - nullary ctor's `List<$u>` wildcard). -2. **Const codegen for non-literal values.** The demo defines - a top-level `xs : List` const whose body is a Cons - chain. `check_const` already accepts pure non-literal const - bodies, but `emit_const` rejected them. Fix: register - `module_consts` in pass 1, resolve const refs in `Term::Var` - (load from global for literal consts, inline body for - non-literal). Both bare and qualified const refs (`xs` - and `module.xs`) supported. - -All three fixes together: ~349 / 25 LOC across `ailang-check`, -`ailang-codegen`, and the new e2e test. Each fix carries an -`Iter 15b` code comment at its site. - -**Tests: 87/87 (was 85, +2).** New e2e `std_list_demo` asserts -the 11-line stdout sequence: - -``` -5 (length [1..5]) -false (is_empty [1..5]) -true (is_empty []) -1 (head [1..5] via from_maybe) -4 (length of tail) -10 (length of [1..5] ++ [1..5]) -5 (head of reverse [1..5]) -2 (head of map double [1..5] = [2,4,6,8,10]) -2 (length of filter is_even [1..5] = [2,4]) -15 (fold_left + 0 [1..5]) -15 (fold_right + 0 [1..5]) -``` - -New unit test `cross_module_recursive_adt_term_and_pat_ctor` in -`ailang-check` covers both Term::Ctor and Pattern::Ctor paths -against a recursive cross-module ADT. Catches the original bug -+ its symmetric pat-ctor latent twin. - -**Hash invariance.** All 22 pre-15b fixture hashes plus -`std_maybe`'s five def hashes bit-identical. The 15b changes -were purely additive on the language side. - -**14a-era and 14h-era regressions held.** Spot-checked -`parameterised_box_round_trip` (14a), `cross_module_maybe_demo` -(14h), `list_map_poly_inc_then_prints` (14e). All green. - -**Authoring observation from the tester.** Form (A) holds up to -10 combinators in one module without breaking. The highest- -overhead pieces are typed `lam` (each closure carries `(typed -name type)` triples + return-type + effects-clause) and the -outer `(forall (vars a b) (fn-type ...))` wrapper. Repeated -paren-counting at the bottom of nested `seq` chains was the only -real friction during demo authoring. Suggests a future iter -might add a `seq*` n-ary form, but the n-ary case is sugar over -the current `seq` shape and not load-bearing. - -**Cumulative state, post-15b.** - -- Stdlib modules: 2 (`std_maybe`, `std_list`). -- Combinators: 14 (`Maybe` + 4; `List` + 10). -- Cross-module imports: type-side, ctor-side, fn-side all working. -- Recursive cross-module ADTs working. -- Tail-call markers used in production: `fold_left` in `std_list`, - `print_list` in two existing fixtures. -- Compiler bugs surfaced and fixed in dogfood: - - 14a: monomorphisation `Type::unit()` placeholder collision. - - 14h: cross-module type/ctor not implemented. - - 15b: qualify-fields gap (3 layers: check, codegen, plus const). - -**Plan 15c.** Stress test on real-shape data. Build a list of -~1000 elements, run `fold_left` and `fold_right` over it, verify -both produce the expected sum. The point: empirically confirm -that `fold_left`'s tail-call marker actually prevents stack -overflow under load, while `fold_right` (constructor-blocked, -unmarked) can still run at this scale because it's only ~1000 -deep, not 1M. If `fold_right` segfaults on this scale, that's a -useful boundary; if it works, we know the stack budget on this -host is at least a few thousand frames. - -After 15c, optional: `std_pair` (2-tuple ADT), `std_either` -(disjoint union for error handling). Or move on to a -non-stdlib feature like nested patterns — at this scale of -language, the case for adding a feature can be made directly -from a stdlib annoyance. - -## Iter 15c — empirical TCO + stack-budget validation at N=1000 - -Small empirical iter to confirm the TCO story works dogfood- -practical, not just on the contrived `print_list` recursion -that was the 14e regression target. - -Fixture `examples/std_list_stress.ailx` builds a 1000-element -`List` via the recursive `build` fn (also used in 14f's -`gc_stress`), then folds it with both `std_list.fold_left` and -`std_list.fold_right`, prints both sums. Both should be -`500500` (1000 × 1001 / 2). E2E test asserts the two-line -output. - -**IR evidence at the monomorphised fold-recursion sites:** - -``` -%v12 = musttail call i64 @ail_std_list_fold_left__I_I(...) ; tail -%v7 = call i64 @ail_std_list_fold_right__I_I(...) ; non-tail -``` - -The `tail: true` marker on `std_list.fold_left`'s recursive call -survives monomorphisation through the `__I_I` specialisation — -exactly what 14e's machinery was supposed to do. `fold_right` -correctly emits a plain `call` (its recursive call is the second -arg to `f`, not in tail position). - -**Empirical findings.** -- `fold_left` at N=1000 runs in constant stack depth (musttail). -- `fold_right` at N=1000 runs with ~1000 stack frames. No - segfault. LLVM's frames at `-O0` are small enough that the - default 8MB Linux stack absorbs this comfortably. -- `build` (also unmarked, recursive) likewise fits. -- End-to-end binary execution time ~40ms wall, of which the - bulk is build + clang link; the actual program runs in <1ms. -- Two `build 1000` chains plus both folds = ~3000 frames total - for the unmarked recursions; still fine. - -**Tests: 88/88 (was 87, +1 e2e).** Cumulative 30 e2e tests. - -**Cumulative state, post-15c.** - -- Stdlib modules: 2 (`std_maybe`, `std_list`). -- Combinators: 14. -- Cross-module imports: type-side, ctor-side, fn-side, recursive - ADT support — all working. -- TCO: marker propagates through monomorphisation; `musttail` - reaches LLVM at the right call sites. -- GC: Boehm runs at 1000-element scale without intervention - (no `GC_INIT()` needed, no symptom of conservative-scan - over-retention at this scale). -- Compiler bugs surfaced and fixed in dogfood since 14a: 4 - (14a monomorphisation, 14h cross-module ADT, 15b qualify- - fields-codegen + non-literal-const-codegen). - -**Natural pause point.** The language is now genuinely useful -for small-to-medium programs. The 14b-through-15c arc was -substantial: a textual surface, two language-completion iters -(tail calls + GC), one revert (14d→14g), a cross-module ADT -gap closure, two stdlib modules, and an empirical stress test -to validate TCO. Work since the user's "Lege los": 9 commits. - -**Queue for future iters.** - -- **15d (optional)**: `std_either : Either e a = Left e | Right a` - for error propagation. Small module (~5 combinators), would - exercise the cross-module-with-2-type-vars import path - (currently only Maybe exercised at one type var). -- **15e (optional)**: `std_pair : Pair a b = MkPair a b` plus - `fst`, `snd`, `swap`, `map_first`, `map_second`. 2-type-var - parameterised ADT, no recursion. Smaller dogfood than List. -- **16a (language)**: nested patterns. Current pattern shape is - flat (a `pat-ctor`'s fields are `pat-var | pat-wild | pat-lit`, - not nested `pat-ctor`). This becomes painful when stdlib code - wants to match `Cons h (Cons h2 _)` directly. Manageable today - via nested `match` but would simplify several stdlib idioms. -- **16b (language)**: local recursive `let`. Currently must hoist - to top-level. Painful for stdlib helpers that are clearly - internal to one combinator (e.g. an accumulator-loop inside - `reverse`'s body). -- **17a (codegen optimisation)**: per-fn arena for non-escaping - ADT allocations (Decision 9's flagged future iter). Real win - for `map`/`filter`-style combinators where the intermediate - list is dropped immediately. Needs escape analysis; multi-iter - design pass. - -None of these block further stdlib work. They are quality-of- -life improvements; the language is feature-complete enough that -the stdlib can grow without them. - - - - - ---- - -## Iter 15d — `std_either`: 2-type-var ADT + 3-type-var eliminator - -**Goal.** Third stdlib module. Either is the canonical 2-type-var -disjoint sum. The eliminator combinator `either : (e → c) → (a → c) → -Either → c` introduces a third type var on top of the data, -making it the most polymorphism-dense fn shipped to date. - -**What shipped.** - -- `examples/std_either.ailx` (74 LOC). Defines `Either e a = Left e | Right a` - plus 5 combinators: - - `from_right : a → Either → a` — eliminate Right or use default. - - `is_left`, `is_right : Either → Bool`. - - `map_right : (a → b) → Either → Either` — - Functor-style on the Right side. - - `either : (e → c) → (a → c) → Either → c` — - catamorphism / fold-on-sum. -- `examples/std_either_demo.ailx` exercises every combinator, - including the eliminator with two distinct concrete instantiations - (over `Left 5` and over `Right 100`). -- Canonical JSON for both, parsed via `ail parse`, type-checked, - built, and executed end-to-end. - -**Output (deterministic).** - -``` -42 ; from_right 0 (Right 42) -99 ; from_right 99 (Left 7) -true ; is_left (Left 7) -true ; is_right (Right 42) -42 ; from_right 0 (map_right inc (Right 41)) -6 ; either inc inc (Left 5) -101 ; either inc inc (Right 100) -``` - -**Monomorphisation evidence.** Six distinct instantiations across the -five combinators, all generated correctly with the `$u` wildcard for -unused type-var positions: - -``` -@ail_std_either_from_right__I_I ; e=Int, a=Int -@ail_std_either_from_right__U_I ; e=$u, a=Int -@ail_std_either_is_left__I_U ; e=Int, a=$u -@ail_std_either_is_right__U_I ; e=$u, a=Int -@ail_std_either_map_right__U_I_I ; e=$u, a=Int, b=Int -@ail_std_either_either__I_I_I ; e=Int, a=Int, c=Int -``` - -The two `from_right` variants are notable: same source fn, two -different concrete `e` per call site (Int when scrutinee is Left 7; -$u when scrutinee is Right _ since Left's payload is unused). The -14a monomorphisation machinery picks the right one per site without -extra ceremony. - -The 3-type-var instantiation `either__I_I_I` is the deepest type -substitution shipped through monomorphisation so far. Both call -sites in main hit the same instantiation (`e=a=c=Int`) so a single -emit suffices. - -**No new compiler bugs surfaced.** First stdlib iter without a fresh -codegen/check fix. Indicator that 14a / 14h / 15b coverage was wide -enough to handle 2-type-var data + 3-type-var fns out of the box. - -**Discovered (not fixed in this iter):** `ail render` and `ail parse` -are not symmetric, despite the help text in the CLI claiming they -are. `render` calls the older `ailang_core::pretty::module` -human-pretty printer (form B-ish), while `parse` consumes form (A). -The form-(A) printer in `ailang_surface::print` is correctly the -inverse of `parse` — confirmed by the round-trip test in -`crates/ailang-surface/tests/round_trip.rs` which now covers 25 -fixtures including `std_either.ail.json` — but it is not exposed via -the CLI. Logged as 15e. - -**Tests: 89/89 (was 88, +1 e2e for std_either_demo).** Cumulative -31 e2e tests. - -**Cumulative state, post-15d.** - -- Stdlib modules: 3 (`std_maybe`, `std_list`, `std_either`). -- Combinators: 19. -- Type-system surface area exercised: 1-type-var data (Maybe, List - with recursion) and 2-type-var data (Either); 1- and 2- and - 3-type-var polymorphic fns; cross-module recursive ADTs; - monomorphisation-with-wildcards across all. - -**Queue update.** 15d done; remaining queue: 15e (CLI render/parse -symmetry — small, just discovered), then `std_pair` if more stdlib -is wanted, then 16a/16b language gaps. - ---- - -## Iter 15e — `render` is the inverse of `parse` (CLI hygiene) - -**Trigger.** Discovered during 15d: `ail render | ail parse` did not -round-trip. The help text claimed they were inverses but `render` -called `ailang_core::pretty::module` (an old human-readable -S-expression-ish form), while `parse` consumes form (A). Two -different "text projections" lived in the codebase, one of them -exposed via the CLI under a name that promised inversion. - -The form-(A) printer in `ailang_surface::print` was already correct -and gated by `crates/ailang-surface/tests/round_trip.rs` across -every fixture (25 modules at the time of writing) — it was simply -not exposed at the CLI surface. - -**Changes.** - -1. `Cmd::Render` now calls `ailang_surface::print(&m)`. Help text - updated to state explicitly: form (A), exact inverse of `parse`, - round-trippable. -2. `Cmd::Describe` (both single-module and `--workspace` branches) - likewise switched to `ailang_surface::print` so that "text view - of one def" means the same thing everywhere in the CLI. -3. `ailang_core::pretty::module` deleted. With `module` gone its - helpers (`def_block`, `term_block`, `term_inline`) became dead - weight — also deleted. ~260 LOC of duplicate-purpose code gone; - `crates/ailang-core/src/pretty.rs` shrank from 457 to 196 LOC. -4. The remaining surface of `ailang_core::pretty` — `manifest`, - `type_to_string`, `pattern_to_string` — is the diagnostic - stringification surface (used in error messages and the - `ail manifest` summary, where one-line ML notation is more - readable than form (A)'s nested S-expression). Module-level - doc rewritten to make this single-purpose framing explicit; - no more "pretty-printer" / "render" overloading. -5. New e2e test `render_parse_round_trip_canonical` (in - `crates/ail/tests/e2e.rs`) gates the CLI shell pipeline: - shell out `ail render ` → write to tmp .ailx → - shell out `ail parse ` → assert canonical-byte equality - with the original fixture. The unit-level round-trip in - `ailang-surface/tests/round_trip.rs` covered the printer - function directly; this new test additionally guards the - CLI wiring. -6. The `ailang_core::pretty` unit test `pretty_print_does_not_panic` - exercised the deleted `module` fn — removed. `manifest_contains_ - type_and_hash` stays. - -**Doctrinal pin.** Form (A) is the only round-trippable text form. -The diagnostic stringification in `ailang_core::pretty` is -asymmetric and lossy by design — it exists for the *limited* -purpose of sticking a type or pattern into an error message in -human-friendlier shape than form (A). It does not roundtrip and -must not be used as an authoring or persistence target. Any -future add to it should add to that purpose, not back-fill toward -"another text projection". - -**Tests: 89/89.** (e2e went from 31 to 32; ailang-core unit -tests went from 11 to 10 — the deleted unit test exercised the -removed fn. Net same.) - -**Cumulative state, post-15e.** No new language features. Two -canonical text projections collapsed to one (form A). Internal -cleanup; no behavioural change for any program in the repo. - ---- - -## Iter 16a — nested constructor patterns in `match` - -**Goal.** Lift the gate that rejected `(pat-ctor Cons a (pat-ctor Cons b _))` -and similar nested-Ctor sub-patterns. Lit-in-Ctor stays rejected; -that's a separate iter. - -**Approach: AST-level desugar before check + codegen.** Pure rewrite -that flattens nested Ctor patterns into chains of single-level -matches with let-bound fresh vars and duplicate fall-through. -Hash-relevant canonical bytes untouched because the pass runs -*after* `load_module`, in memory only. The checker / codegen always -see flat patterns. - -**Algorithm sketch.** - -For a `Match` whose arms contain nested-Ctor sub-patterns: -1. Bottom-up: recursively desugar scrutinee and arm bodies first. -2. Let-bind the scrutinee to `$mp_N` (single eval). -3. Build a chain `arm_1 (else arm_2 (else ... default))`. Default - is `Lit Unit`, unreachable for valid programs. -4. Each arm's nested Ctor sub-patterns are lifted to fresh vars, - then deepest-first wrapped via `wrap_sub`, which on a Ctor sub - recursively re-enters `desugar_match` — that recursion handles - arbitrary depth. - -`fall_k` is cloned per inner level → O(arms × depth) terms in -worst case. Acceptable for typical patterns. - -**Fresh-name safety.** `$` is a valid identifier character in form -(A) (the lexer's `Ident` token is "anything not paren/int/string"), -so `$mp_0` is theoretically user-writable. The Desugarer pre-walks -every `Term::Var` and `Pattern::Var` name in the module into a -`BTreeSet` and bumps the counter past collisions. - -**Files.** - -- New: `crates/ailang-core/src/desugar.rs` (571 LOC including - doc-comments and two unit tests). Public surface is - `pub fn desugar_module(m: &Module) -> Module` — pure, idempotent, - no I/O. -- `crates/ailang-core/src/lib.rs` — `pub mod desugar;` plus a - module-doc bullet describing the new pipeline layer. -- `crates/ailang-check/src/lib.rs` — three public entries - (`check_module`, `check_workspace`, `check`) call - `desugar_module` first. The gate at the old line 1538 is now - narrowed: nested **Ctor** sub-patterns are `unreachable!()` - (desugared away); nested **Lit** still emits the existing - `nested-ctor-pattern-not-allowed` diagnostic. Crucially: - `check`'s returned `CheckedModule.symbols` keeps hashes of - the *original* defs so `ail diff` and `ail manifest` see the - on-disk identity, not a post-desugar one. -- `crates/ailang-codegen/src/lib.rs` — `lower_workspace` desugars - every module up front; `emit_ir` (single-file shortcut) goes - through `lower_workspace` so the desugar runs there too. -- New: `examples/nested_pat.ailx` + `nested_pat.ail.json`. - `first_two_sum` matches `(pat-ctor Cons a (pat-ctor Cons b _))`; - prints `30` for the input `[10, 20, 30]`. -- `crates/ail/tests/e2e.rs` — new - `nested_ctor_pattern_first_two_sum` test. - -**Behavioural property of the desugar.** Already-flat matches go -through `is_flat()` and emit identical AST shapes. Empirically: -every existing fixture (std_maybe, std_list, std_either, list_map, -sort, etc.) produces the same observable output as before because -their patterns were already flat — the desugar is a no-op clone -on them. - -**Tests: 92/92.** -- e2e: 33 (was 32, +1 for nested_pat). -- ailang-core unit: 12 (was 10, +2 for the desugar tests). -- All other crates unchanged. - -**No new compiler bug surfaced.** The transform was straightforward -because the AST already supported nested sub-patterns at the type -level — only the checker gate and codegen drop-on-floor were -artificial walls. Removing them via desugaring (rather than -expanding the codegen) keeps the pattern-matching backend simple -and lets future iters (Lit-in-pattern, exhaustiveness, decision -trees) plug in at the same desugar layer. - -**Cumulative state, post-16a.** - -- Stdlib: 3 modules, 19 combinators (unchanged from 15d). -- Language: nested Ctor patterns now legal; nested Lit-in-pattern - still rejected (intentional follow-up scope). -- Pipeline: `load → desugar → check → codegen`. The desugar layer - is the natural home for further surface-level smoothing - (Lit-in-pattern, `if`-as-syntactic-sugar, etc.) without - enlarging the core AST. -- Compiler bugs surfaced and fixed in dogfood since 14a: still 4 - (no fresh ones in 16a — the desugar landed clean). - -**Queue update.** 16a done. Remaining: 15f (`std_pair`, optional); -16b (local recursive `let`); 17a (per-fn arena); future -"lit-in-Ctor" follow-up under 16c if and when needed. - ---- - -## Iter 15f — `std_pair`: 2-type-var product, no recursion - -**Goal.** Round out the small-ADT stdlib triad (Maybe, Either, Pair). -Pair is the canonical product with two type vars and a single -constructor — no recursion in the data def or any combinator. -Smallest dogfood for the parameterised-ADT path so far. - -**What shipped.** - -- `examples/std_pair.ailx` (~75 LOC, 5 combinators): - - `fst`, `snd` — projections. - - `swap : Pair -> Pair`. - - `map_first : (a -> c) -> Pair -> Pair`. - - `map_second : (b -> c) -> Pair -> Pair`. - Each combinator is a single-arm match on `MkPair` with no - fall-through, so the desugar pass added in 16a is a no-op - (the patterns are already flat). -- `examples/std_pair_demo.ailx` exercises every combinator. Output - (deterministic): 7, 9, 9, 7, 8, 18. -- `crates/ail/tests/e2e.rs::std_pair_demo` guards the path. - -**No new compiler bug surfaced.** Stdlib iter #4 in a row that -landed clean. The only fixable issue was a paren-balance typo in -the demo's `seq` chain, surfaced immediately by the parser's -"unexpected end of input, expected `)`" diagnostic. - -**Tests: 93/93** (e2e went from 33 to 34). - -**Cumulative state, post-15f.** - -- Stdlib: 4 modules (`std_maybe`, `std_list`, `std_either`, - `std_pair`); 24 combinators total. -- Type-system surface area exercised end-to-end: 1- and 2- and - 3-type-var data; 1- and 2- and 3-type-var fns; recursive ADTs; - cross-module imports of all of the above; flat *and* nested - patterns (16a); TCO via monomorphised `musttail`; Boehm GC. -- Pipeline layers: `load → desugar → check → codegen → clang`. - Each layer has a public, narrow contract; the desugar layer - is the natural home for further surface-level smoothing - without enlarging the core AST. - -**Queue update.** 15f done. Remaining: 16b (local recursive `let`), -16c (Lit-in-Ctor patterns), 17a (per-fn arena). None blocking -further stdlib growth; each is a quality-of-life improvement. - ---- - -## Iter 16a-aux — DESIGN.md drift audit (post-15f) - -**Goal.** After six feature iters (14g, 14h, 15a–15f, 16a) without a -docs sweep, the design doc had accumulated visible drift. Patch in -place rather than queueing the next codegen-heavy iter against a -stale spec — the user is unreachable for the broader memory-management -discussion that gates 17a, so this is the lowest-risk productive -move. - -**Drift sites found and fixed (DESIGN.md +72 LOC).** - -1. **Decision 6 status (L116).** Was: `(Iter 14b — WIP) ... Status: design pass in progress`. - Now: marks form (A) as shipped in Iter 14c, gated by the - round-trip test in `ailang-surface/tests/round_trip.rs`, and - notes that 15e made it the *sole* text projection (legacy - pretty-printer module helpers deleted). The body of the - section (constraints, candidate notations) remains as the - audit trail of the original design pass. -2. **Pipeline section (L687–696).** Inserted the **desugar pass** - between resolve+hash and typecheck, with the load-bearing - invariant explicit: `CheckedModule.symbols` hashes from the - *original* module, not the desugared one, so `ail diff` / - `ail manifest` keep reporting the on-disk identity. Also - noted libgc linkage on the clang line. -3. **CLI section (L700–708).** Added the four subcommands that - shipped in earlier iters but never made the doc: `deps`, - `diff` (single-module + `--workspace`), `workspace`, and - `builtins`. Reformatted to a two-column layout. -4. **"What is not (yet) supported" (L724–746).** Re-anchored - from "end of Iter 13" to "as of Iter 16a". Removed three - gates that had been lifted: cross-module ADTs (14h), no-GC - (14f, Decision 9), flat-pattern-only (16a). Replaced with - tighter follow-up gates: literal sub-patterns inside Ctor - patterns; local recursive `let`. Added a *one-paragraph* - "recently lifted" preamble so a future reader sees both the - delta and the iter that produced it without consulting JOURNAL. -5. **"What IS supported" (L782+).** Promoted four capabilities - into the smoke-test list: nested Ctor patterns via desugar - (16a); cross-module ADTs (14h); the form-(A) text surface as - shipped (14b/14c/15e); Boehm GC (Decision 9 / 14f). Replaced - "ADTs + flat pattern matching" line with one that names both - the original 3 and the 16a extension. -6. **Pipeline regression smoke tests (L819–).** Added the four - stdlib-demo fixtures (`std_list_demo`, `std_maybe_demo`, - `std_either_demo`, `std_pair_demo`) plus `nested_pat`. The - pre-existing entries (sum/list/hof/closure/list_map/sort/ - poly_id/poly_apply/box/maybe_int) were preserved as-is. - -**No code changes; tests still 93/93.** Verified with `cargo -build --workspace --quiet` — the doc-only edits do not affect -any compilation unit. - -**What was *not* changed.** The Goal section, Decisions 1–5, -Decisions 7–9 (already accurate after their respective revert / -ship statuses), Mangling scheme, Convention for cross-module -references, Data model (Module/Def/Term/Type grammar), Verification -section. Spot-checked each — all match current implementation. - -**Cumulative state, post-16a-aux.** - -- Stdlib: unchanged (4 modules, 24 combinators). -- DESIGN.md: 804 → 876 LOC. JOURNAL.md grows by this entry. -- Drift baseline reset: any future iter that lifts a gate or - ships a new pipeline layer should patch the relevant section - in the same iter, not accumulate. - -**Queue update.** 16a-aux done. Unchanged from post-15f: -16b (local recursive `let`), 16c (Lit-in-Ctor patterns), -17a (per-fn arena). 17a remains the explicit checkpoint before -any broader memory-management work — that decision is gated on -a joint conversation with the user. - ---- - -## Iter 15g — std_either_list: first 3-way cross-module stdlib fn - -**Goal.** Through 15f the stdlib had 4 modules but no fn imported -more than one foreign module (e.g. `std_list.head` returning -`std_maybe.Maybe`). 15g introduces the first stdlib module whose -every fn imports three others — `std_list`, `std_either`, -`std_pair` — and returns a compound polymorphic ADT tree -(`Pair, List>`). Designed to stress monomorphisation -across `List × Either × Pair`, cross-module ctor resolution at -qualified `term-ctor` sites, and the 16a desugar layer's nested- -Ctor pattern handling at depth 2. - -**What shipped.** - -- `examples/std_either_list.ailx` — three combinators, all - `forall (vars e a)`, all using qualified cross-module names at - `con` and `term-ctor` sites: - - `lefts : List> -> List` — depth-2 nested-Ctor - match per Cons arm (`Cons (Left l) t` / `Cons (Right _) t`) - plus a wildcard tail to anchor the desugar's fall-through - chain in `List` rather than the synthetic `Unit` fallback. - - `rights : List> -> List` — symmetric to - `lefts`. - - `partition_eithers : List> -> Pair, - List>` — `Nil → MkPair(Nil, Nil)`; `Cons h t →` let-bind - `rest = partition_eithers t`, then a flat match on `h` to - splice `l` / `r` onto `fst rest` / `snd rest`. Single pass. -- `examples/std_either_list_demo.ailx` — first demo importing four - stdlib modules. Drives all three combinators on the same five- - element list `[Left 1, Right 10, Left 2, Right 20, Right 30]`; - prints lengths via `std_list.length`. Output: `2 / 3 / 2 / 3`. -- `crates/ail/tests/e2e.rs::std_either_list_demo` (e2e count - 34 → 35). - -**16a desugar exercised.** Yes — `lefts` and `rights` use depth-2 -nested Ctor patterns (`(pat-ctor Cons (pat-ctor Left l) t)`). Each -expands into a chain of single-level matches via the 16a pass; the -explicit trailing wildcard arm prevents the synthetic `Unit` -fallback (the desugar's documented "valid programs never reach it" -terminator) from leaking into the function's return type. Without -the wildcard the checker reports `expected std_list.List, got -Unit` because the chain's `else`-arm body is `Lit Unit` — confirms -the design note in `desugar.rs::desugar_match` that exhaustiveness -is the caller's responsibility, not the desugar's. - -**New compiler bug surfaced.** Yes — and it is **not** in the new -combinators themselves but in the existing monomorphiser, surfaced -the moment one tries to construct an inline list literal mixing -`(term-ctor std_either.Either Left n)` and `(term-ctor -std_either.Either Right n)`. Reduced repro is just `(app -std_list.length (term-ctor std_list.List Cons (Left 1) (term-ctor -std_list.List Cons (Right 10) (term-ctor std_list.List Nil))))` — -no `lefts` / `rights` / `partition_eithers` involved. - -Cause: `synth_arg_type` produces `Either` for `Left 1` and -`Either<$u, Int>` for `Right 10`. The outer `Cons`'s parameter -type `List` unifies first against `List>` -(binds `a = Either`) and then against `List>` for the tail. `unify_for_subst` recurses into the prev -binding and ends up unifying param `$u` (from `Either`) -against arg `Int` — the existing `if name.starts_with("$u") { -return Ok(()); }` early-return only fires when `$u` is on the -**arg** side. Param-side `$u` falls through to the catch-all error -`cannot match param `$u` to arg `Int``. - -**Workaround in the demo.** Two monomorphic helpers `mkleft : Int --> Either` and `mkright : Int -> Either` pin -both type vars at the call site, so each list element arrives with -fully concrete `Either` and the synth-time `$u` never -appears. Documented in the demo's header comment with a pointer -back to this entry. The combinators themselves (`lefts`, `rights`, -`partition_eithers`) are bug-free — the demo just couldn't build -the input list inline without dodging the `$u`-on-param-side path. - -**Bug fix sketch (queued, not landed).** A symmetric early-return -in `unify_for_subst` for param-side `$u` would close this — the -wildcard semantics ("don't care, defer") are direction-agnostic. -Filed as candidate iter 15g-aux. Single-line patch + a unit test -under `ailang-codegen/src/lib.rs`. - -**Tests: 94/94.** - -- e2e: 35 (was 34, +1 for `std_either_list_demo`). -- All other crates unchanged. - -**Cumulative state, post-15g.** - -- Stdlib: 5 modules (`std_maybe`, `std_list`, `std_either`, - `std_pair`, `std_either_list`); 27 combinators total (24 + 3). -- Deepest cross-module composition reached so far: `List × Either - × Pair` in a single fn (`partition_eithers`). -- Compiler bugs surfaced and fixed in dogfood since 14a: still 4 - fixed; 1 new (the `$u`-on-param-side case above), worked around - in the demo, queued as 15g-aux. - -**Queue update.** 15g done. Remaining: 15g-aux (param-side `$u` -acceptance — small, one-line in `unify_for_subst`); 16b (local -recursive `let`); 16c (Lit-in-Ctor patterns); 17a (per-fn arena, -gated on user discussion). - ---- - -## Iter 15g-aux — symmetric `$u` early-return in `unify_for_subst` - -**Goal.** Fix the codegen asymmetry that 15g surfaced: inline list -literals mixing `Left n` and `Right n` ctor calls of the same -`Either` errored at synth time even though both elements have -fully-determined concrete-after-pinning types. - -**Diagnosis.** `unify_for_subst` (`crates/ailang-codegen/src/lib.rs`) -had an arg-side-only early-return for `$u`-prefixed synth wildcards. -The function has a prev-binding recursion path that re-invokes -itself with the previously-bound type as the new param and the -fresh arg, which can swap a `$u` from arg position into param -position. Concretely, `length [Left 1, Right 10]` synths the list -elements as `Either` and `Either<$u, Int>`. The first -element pins `a = Either` for `Cons`'s parameter `a`. The -second element triggers a recursive unification of the previously- -bound `Either` against `Either<$u, Int>`, walking -pairwise: `Int` vs `$u` (arg-side `$u`, ok) and `$u` vs `Int` -(param-side `$u`, **falls through** to the catch-all error). - -**Fix.** Three lines in `unify_for_subst`: add a symmetric early- -return for param-side `$u`. Doc comment expanded to record the -asymmetry's origin and the symmetric extension's justification -(`$u` is a synth-only wildcard regardless of which side it ends -up on after the prev-binding swap). - -**Demo refactor.** `examples/std_either_list_demo.ailx` no longer -uses the `mkleft`/`mkright` monomorphic helpers introduced as the -15g workaround. The list is now constructed inline by mixing -`(term-ctor std_either.Either Left 1)` and `(term-ctor -std_either.Either Right 10)` directly. Same expected output -(2, 3, 2, 3); the demo doubles as the 15g-aux regression fixture. - -**Tests: 94/94, unchanged.** The fix expanded what compiles, did -not change observable behaviour for any prior fixture. - -**Cumulative state, post-15g-aux.** - -- Stdlib unchanged (5 modules, 27 combinators). -- One latent codegen bug retired. The bug count in the dogfood - audit since 14a stays at 4 surfaced + fixed (this is a fresh - surface from 15g, so 5 surfaced / 5 fixed). -- The std_either_list_demo workaround is gone, leaving the inline - cross-module mixed-ctor list as the canonical idiom. - -**Queue update.** 15g-aux done. Unchanged: 16b (local recursive -let), 16c (Lit-in-Ctor patterns), 17a (per-fn arena, gated on -user discussion of memory management). - ---- - -## Iter 15h — std_list extension: take, drop - -**Goal.** Extend `std_list` with the two index-driven prefix -combinators `take` and `drop`. They are the first `std_list` fns to -combine `if` + Int arithmetic + recursive ADT pattern in a single -body — every previous `std_list` fn was either a fold/HOF -(fold_left, fold_right, length, map, filter, reverse) or a pure -match-on-Cons (head, tail, append, is_empty). 15h closes the -canonical-prefix-slicing gap and dogfoods the `if (<= n 0)` + -`(- n 1)` interaction inside a recursive Cons-pattern body. - -**What shipped.** - -- `examples/std_list.ailx` — `take` and `drop` appended after - `filter`. All ten pre-existing defs byte-identical (verified via - `cargo run -p ail -- check` on every downstream importer: - `std_list_demo`, `std_list_stress`, `std_either_list`, - `std_either_list_demo`, `list_map_poly`). Both new fns are - `forall (vars a). (Int, List) -> List`. `take` is - constructor-blocked recursive (`Cons h (take (n-1) t)`); `drop` - is direct recursive (the recursive call is the arm body, no - Cons wrap), so the `match` arm is in tail position relative to - the `if`'s `else` branch — but neither call site is marked - `tail-app` because the surrounding `if` is not the fn's - immediate body. Conservative; matches the rest of `std_list`'s - marking discipline. -- `examples/std_list_more_demo.ailx` — six prints exercising both - combinators at all three boundary regimes (n=0, 0length). Reuses `std_list_demo`'s `(const xs)` idiom for the - canonical `[1, 2, 3, 4, 5]`. Output: `0 / 3 / 5 / 5 / 3 / 0`. -- `crates/ail/tests/e2e.rs::std_list_more_demo` (e2e count - 35 → 36). - -**`(if (<= n 0) ...)` as the base-case guard.** Both fns gate the -recursion on the int counter at the top of the body, _outside_ the -match. The dual-arm `(case (pat-ctor Cons h t) ...)` then handles -only the n>0 path — so the n=0 branch never re-enters the match, -and the recursive call is reached only when the list is non-empty -and n is still positive. This is a deliberate workaround for the -absence of literal sub-patterns inside Ctor patterns: a -hypothetical `(case (pat-ctor Cons _ _) (case-when (== n 0) ...))` -or matching `n` against `0` directly is queued as 16c. With 16c -landed, both fns could collapse the `if` into the match. - -**No new compiler bug surfaced.** The `if` + Int-arithmetic + -recursive ADT pattern shape is the first instance for `std_list` -in particular but not new for the compiler — `gc_stress` and -`std_list_stress` already exercised `(if (== n 0) ...)` with -`(- n 1)` recursion at the top level. `<=` on `Int` and the -forall-`a` instantiation through the `match` cleanly reused the -same monomorphisation paths. Round-trip `render | parse` stays -canonical for both new files. - -**Tests: 95/95.** - -- e2e: 36 (was 35, +1 for `std_list_more_demo`). -- All other crates unchanged. - -**Cumulative state, post-15h.** - -- Stdlib: 5 modules (`std_maybe`, `std_list`, `std_either`, - `std_pair`, `std_either_list`); **29** combinators total - (27 + 2). All four primary `std_list` operations — length, map, - filter, take/drop — are now present, plus the head/tail/append/ - reverse/is_empty/fold_left/fold_right surface from 15b. -- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5 - (unchanged from 15g-aux). - -**Queue update.** 15h done. Remaining: 16b (local recursive let), -16c (Lit-in-Ctor patterns — would simplify `take`/`drop`), 17a -(per-fn arena, gated on user discussion). - ---- - -## Iter 16b.1 — local recursive let (no-capture) - -**Goal.** Let me write `(let-rec f (params x) (type ...) (body ...) (in ...))` -inline inside a fn body for the common no-capture case (the body's -free vars are all module-top-level def names, qualified imports, -effect-op names, or builtin operators). Stop forcing every recursive -helper to be a separate top-level fn — typical small kernels -(`fact`, `loop_n`, `gcd`) belong next to the use site, not on a -sibling line at module scope. - -Sub-iter 16b.1 ships **no-capture only**. A LetRec whose body would -capture a name from the enclosing lexical scope (an enclosing fn's -params, a let-bound name from an outer `Term::Let`, or a pattern- -bound name from an outer `Term::Match` arm) is rejected at desugar -time with a clear panic that points at 16b.2 (closure conversion). -That keeps the iter at lift-via-desugar — no runtime closure changes, -no codegen plumbing, no LLVM IR change. - -**Design choice.** Lift the LetRec to a synthetic top-level fn in -the same desugar pass that already runs between `load_module` and -typecheck (16a). The lifted fn gets a fresh name `$lr_N` that -is unique against both the original module's def names and against -any earlier lifts in the same pass. Every reference to the local -LetRec name (in the body and in the in-clause) is rewritten via a -`subst_var` helper to the lifted name. The lifted `FnDef` is -appended to `Module.defs`; from there the typechecker / codegen -treat it like any hand-written top-level fn. - -Alternatives considered and rejected: - -- **Runtime closures** (treat LetRec like an anonymous lambda - bound to a name). Would require a closure-pair allocation per - call, plus a self-reference field in the env block. Reaches the - same value but more LLVM IR per LetRec. Deferred to 16b.2 where - it becomes necessary anyway (capture support). -- **Open-coding the recursion at the LetRec site** via a Y-style - fixed-point combinator. Adds a non-local construct (the - combinator) and keeps the expansion at every LetRec; the lift- - and-substitute approach keeps the runtime shape identical to a - hand-written top-level fn. - -The pipeline invariant from 16a (`CheckedModule.symbols` hashes -from the *original* on-disk module, not the desugared one) holds -unchanged: a lifted def has no on-disk identity, so it never -appears in `symbols`. `ail diff` and `ail manifest` see only the -original-source defs. - -**What shipped.** - -- `crates/ailang-core/src/ast.rs` (+16): `Term::LetRec { name, ty, - params, body, in_term }` variant inserted right after - `Term::Let`. JSON tag `"t": "letrec"`. `ty` and `in_term` use - serde renames (`type`, `in`) to keep the schema natural. - Additive — every pre-16b.1 fixture canonicalises bit-identically. -- `crates/ailang-core/src/desugar.rs` (+~570 of which ~310 are - the new helpers + LetRec arm; the rest is doc/test): three - additions plus the existing pass threaded through a `scope` - parameter. (a) `free_vars_in_term` walks a term collecting every - unbound `Term::Var` name, respecting all binders. (b) `subst_var` - rewrites `Term::Var { name == from }` to the lifted name, - respecting shadowing (a `Term::Let`/`Term::Lam`/`Pattern::Var` - that rebinds `from` blocks the substitution inside its scope). - (c) `Desugarer` gained `lifted: Vec` and - `module_top_names: BTreeSet`; `desugar_module` seeds - the latter from every original def name and appends `lifted` - to `defs` after the per-def walk. The new `desugar_term` arm - for `Term::LetRec` recurses on body+in_term with an extended - scope, runs `free_vars_in_term` against `{name} ∪ params`, - intersects with the outer `scope`, panics if non-empty, then - generates `$lr_N`, substitutes, and appends the lifted - `FnDef`. Two new unit tests: - `let_rec_no_capture_lifts_to_top_level` and - `let_rec_with_capture_panics` (`#[should_panic]`). -- `crates/ailang-surface/src/parse.rs` (+~70): `let-rec-term` - production added to the EBNF (still inside the 30-rule - budget — count is now ~31, but `let-rec` is a positional - analogue of `let` and the increment is consistent with the - Decision-6 budget rationale). New `parse_let_rec` function; - dispatch keyword added in `parse_term`. Unit test - `parses_minimal_let_rec` round-trips a minimal shape. -- `crates/ailang-surface/src/print.rs` (+16): `Term::LetRec` - arm in `write_term`, single-line form mirroring the - `Term::Let` arm's compactness. The round-trip harness - (`tests/round_trip.rs`) picks up the new fixture - automatically. -- `crates/ailang-check/src/lib.rs` (+10): two `unreachable!` - arms in `verify_tail_positions` and `synth` — - `Term::LetRec` is eliminated by desugar before either runs. -- `crates/ailang-codegen/src/lib.rs` (+19): four `unreachable!` - arms in `lower_term`, `collect_captures`, `synth_with_extras`, - and `apply_subst_to_term`. Same rationale. -- `crates/ail/src/main.rs` (+22): `walk_term` (the `ail deps` - walker) gets a real arm — `deps` runs on the on-disk module - before desugar, so `Term::LetRec` is reachable there. - Treats it like a fn def for dependency purposes (name - shadows in body+in_term; params shadow inside body). -- `examples/local_rec_demo.ailx` + `.ail.json` — first - consumer fixture. A factorial helper bound by `(let-rec - fact ...)` inside `main`'s body; called at n=1, n=3, n=5; - prints `1\n6\n120\n`. The `fact` body has no captures - from `main`'s scope (referenced names: `<=`, `*`, `-`, - `fact`, `n`, `1` — all builtins, the LetRec's own name, - or its param), so it lifts cleanly. -- `crates/ail/tests/e2e.rs::local_rec_factorial_demo` (+18): - e2e count 36 → 37. - -**Unreachable arms.** Every backend stage that pattern-matches -`Term` exhaustively now has a `Term::LetRec { .. } => -unreachable!("Term::LetRec eliminated by desugar")` arm. The -phrase is identical across all five sites -(`ailang-check/src/lib.rs` × 2, `ailang-codegen/src/lib.rs` × 4) -so a future grep reaches every one of them. The `ail deps` -walker is the one site that intentionally has a real arm, -because `deps` runs on the on-disk module before any desugar -pass, and it is documented inline. - -**Lift-name format.** `$lr_N` where `` is the -source-level LetRec name and `N` is an incrementing counter that -yields the first name not already in `Desugarer.used` or -`Desugarer.module_top_names`. Mirrors the `$mp_N` fresh-match- -pattern naming from 16a; `$lr_` is the namespace marker for -"lifted recursion". `$` is a valid ident character in form (A), -so the name reaches LLVM as is — but `clang` mangling chokes on -`$`, so the name is sanitised the same way every other ail name -is (the existing `@ail__` mangling already handles -`$` because of the 16a `$mp_` precedent). - -**Tests: 95 → 99 (+4).** - -- e2e: 36 → 37 (`local_rec_factorial_demo`). -- `ailang-core::desugar::tests`: 2 → 4 (the two new LetRec tests). -- `ailang-surface::parse::tests`: 2 → 3 - (`parses_minimal_let_rec`). -- All other crates unchanged. - -**Cumulative state, post-16b.1.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term` enum: 11 variants (was 10) — additive, all pre-existing - fixture hashes bit-identical. -- 16a desugar pass now does two jobs: nested-ctor-pattern - flattening (16a) and LetRec lift (16b.1). The pass remains - the single AST-→-AST hop between `load_module` and typecheck. -- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5 - (unchanged from 15h). - -**Queue update.** 16b.1 done. Remaining: 16b.2 (LetRec capture — -closure conversion or env-passing rewrite, with the panic at -desugar time as the boundary-marker until then), 16c (Lit-in- -Ctor patterns — would simplify `take`/`drop`), 17a (per-fn -arena, gated on user discussion of memory management). - -## Iter 16c — literal patterns via desugar - -**Goal.** Lift the last gate that survived the 16a-aux audit: -`Pattern::Lit` was rejected at the codegen level (an internal -error in the Match lowering) and at the typechecker level when -nested inside a Ctor (`nested-ctor-pattern-not-allowed`). 16c -makes lit patterns work **everywhere they parse** — top-level -arms and Ctor sub-patterns alike — by extending the existing -16a desugar pass. - -**Design choice.** Desugar `Pattern::Lit { lit }` to -`Term::If { cond = (== sv lit), then = body, else_ = fall_k }`, -where `sv` is the let-bound scrutinee (top-level) or the -field-bound fresh var (sub-pattern). After 16c, no -`Pattern::Lit` survives the desugar pass — the codegen and -typechecker never see one. - -Alternatives considered and rejected: - -- **Switch-on-i64 in codegen** (a `switch i64` LLVM instruction - per Match with at least one Int-lit arm). Smaller IR for - many-arm dispatch, but adds a second match-lowering path in - codegen and grows the typechecker's exhaustiveness checker - (now needs to reason about lit coverage). The desugar-to-If - approach hands every problem to the existing infrastructure: - `==` is a typed builtin, `Term::If` already lowers correctly, - and the chain machinery from 16a already produces a - fall-through structure that fits. -- **Codegen-level lit-arm rejection only** (allow lit arms past - typecheck and trap at codegen). Worse than today: today's - codegen rejects with an internal error; future codegen would - need a real lowering. Strictly more work for no gain. - -The pipeline invariant from 16a/16b.1 holds unchanged: the -desugar runs after `load_module`, so on-disk module hashes are -untouched. `ail diff` and `ail manifest` see only original-source -defs. - -**What shipped.** - -- `crates/ailang-core/src/desugar.rs` (+~120, of which ~80 are - doc/test): three replacements plus one new helper. - (a) The `Pattern::Lit` arm of `desugar_one_arm` now emits a - `Term::If { cond = (== s_var lit), then = arm.body, else_ = - fall_k }` instead of the old single-arm `Term::Match` with - the lit pattern preserved (which the codegen rejected). - (b) The `Pattern::Lit` branch of `wrap_sub` mirrors (a) on - the field-bound fresh variable, replacing the old recursive - `desugar_match` call. - (c) `is_flat` no longer classifies `Pattern::Lit` as flat — - the early-return path in `desugar_match` would otherwise - leak a Pattern::Lit arm through to typecheck/codegen - unchanged. With the change, lit-arms always take the - let-bind + chain path. - (d) New free function `build_eq(scrutinee, lit) -> Term`: - produces `(app == scrutinee lit)` for Int/Bool/Str and a - `Bool(true)` literal for Unit (every Unit value is equal). - Three new unit tests: - `top_level_lit_desugars_to_if`, - `nested_lit_in_ctor_desugars_to_if`, and - `flat_arm_with_lit_is_no_longer_flat` (regression guard for - the deliberate change in `is_flat`). A new `any_lit_pattern` - walker mirrors `any_nested_ctor`. -- `examples/lit_pat.ailx` + `examples/lit_pat.ail.json` — first - consumer fixture. Two fns: `classify` (top-level lit arms - for 0/1/default → 100/200/999) and `categorize_first` - (`Cons (pat-lit 0) _` nested-lit demonstration over a local - `IntList` ADT). The trailing `_` catch-all in - `categorize_first` is required by the 16a chain - machinery — the chain's terminator is a `Unit` literal, so - a final `_` arm dominates it and keeps the match - type-checking. Output (per line): 100, 200, 999, -1, 0, 7. -- `crates/ail/tests/e2e.rs::lit_pat_demo` (+15 incl. doc): e2e - count 37 → 38. -- `docs/DESIGN.md`: moved the "no literal sub-patterns inside a - Ctor" line out of "What is not (yet) supported" into the - "Recently lifted gates" preamble and into the "What is - supported" list, with a note that `Bool`/`Str`/`Unit` lit - patterns are accepted by desugar but reach typecheck as a - type error today (because `==` is `(Int, Int) -> Bool` - only — extending `==` to those types is a separate iter). - -**What deliberately did NOT change.** - -- Codegen's `Pattern::Lit` arm at `crates/ailang-codegen/src/lib.rs:1302` - still returns `CodegenError::Internal("MVP: lit patterns in - match not supported")`. Left as a never-reached safety net — - the desugar is the contract; the codegen panic catches future - regressions where a Pattern::Lit slips through. -- Typechecker's `Pattern::Lit` arm in `check_pattern` still runs - (it accepts the pattern but contributes nothing to - exhaustiveness). Same rationale: dead code at the source-AST - level after desugar, but defensive against a future caller - that bypasses desugar. -- `std_list::take` and `std_list::drop` still hand-write the - base-case-via-arm-body workaround (`(case (pat-ctor Cons h t) - (if (== n 0) Nil (...)))`). Refactoring them to use lit - patterns is queued as 16c-aux. - -**Tests: 99 → 103 (+4).** - -- e2e: 37 → 38 (`lit_pat_demo`). -- `ailang-core::desugar::tests`: 4 → 7 (the three new lit tests). -- All other crates unchanged. - -**Cumulative state, post-16c.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term`/`Pattern`/`Literal` enums unchanged — additive at the - desugar level only, so all pre-existing fixture hashes stay - bit-identical. -- 16a desugar pass now does three jobs: nested-ctor-pattern - flattening (16a), LetRec lift (16b.1), and lit-pattern → If - rewrite (16c). Pass remains the single AST-→-AST hop between - `load_module` and typecheck. -- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5 - (unchanged). - -**Queue update.** 16c done. Remaining: 16b.2 (LetRec capture — -closure conversion, unchanged), 16b.3 (LetRec let-binding -capture, unchanged), 17a (per-fn arena, gated on user -discussion of memory management, unchanged). 16c-aux -(`std_list::take`/`drop` refactor onto lit patterns) is a -separate iter, gated on orchestrator decision. - -## Iter 16b.2 — planning entry (LetRec capture) - -**Status: planning, not implemented.** This entry is orchestrator -work-product, not an iteration log. It captures the design space -for 16b.2 so the user can review the trade-offs before -implementation, and so a future agent has a sharp brief to work -against. - -**Goal (when implemented).** Lift 16b.1's no-capture restriction. -Support `(let-rec name ...)` whose body references one or more -names bound in the enclosing lexical scope, by augmenting the -lifted top-level fn's signature with the captured names as extra -parameters and rewriting every call site of `name` (in body and -in_term) to pass the captures positionally. - -**Why this is non-trivial.** The lifted fn's signature must -include the captures' types. Those types are not always -discoverable at desugar time: - -- **Captured fn-param**: type is in `f.ty.params[i]`. Known. -- **Captured Lam-param**: type is in `param_tys[i]`. Known. -- **Captured Let-binding**: `Term::Let { name, value, body }` - carries no type annotation on `name`. Type is the inferred - type of `value`, which the typechecker computes — but the - desugar pass runs *before* the typechecker. **Unknown at - desugar time.** -- **Captured Match-arm pattern var**: type is the substituted - field type of the matched constructor, requiring an ADT-def - lookup plus arg-substitution. Computable in principle, but - the desugar pass would need to track the scrutinee's type - through the walk — also a small inference engine. - -**Architectural choices.** Pick one path before implementing: - -1. **Stay at desugar; restrict to fn/Lam-param captures only.** - Reject Let-binding and Match-arm captures with a clear - error → `16b.3` (Let captures) and `16b.4` (Match captures). - Lowest-cost path, covers the most common case (recursive - helpers that close over an enclosing fn's input). - -2. **Move LetRec elimination to a post-typecheck pass.** By - then every `Term` has known types; capture types fall out. - Cost: a new pipeline stage, plus updating `DESIGN.md`'s - pipeline section. Pays off if the post-pass is also where - other future lowerings live (e.g. closure conversion). - -3. **Run a lightweight inference inside desugar.** Just enough - to resolve `Term::Let`-bound names to their value's type. - Effectively a Hindley-Milner pass on a subset of the AST. - Cost: significant; basically duplicating part of - `ailang-check`. Rejected on principle — one source of truth. - -**Recommended path.** (1), shipping incrementally. 16b.2 covers -fn/Lam-params; 16b.3 lifts Let-bindings via path (2). The -benefit of (1) first: it surfaces real-world usage and informs -how often Let-binding capture actually matters. - -**Other restrictions for 16b.2 (under path 1).** - -- **Direct-call only.** The LetRec name `f` may appear in - `body` and `in_term` only as the callee of a `Term::App`, - never as a `Term::Var` in any other position (e.g. passed as - an argument, bound to a `let`, or stored in an ADT field). - Reason: with captures-as-extra-params, every use site needs - the extras appended. Treating `f` as a value would require - a closure object that bundles `f` and its captures — that's - 16b.5 / closure conversion. -- **Monomorphic enclosing fn.** If the enclosing fn is - `forall(...). ...`, the captures' types may mention the - outer's type vars. Constructing the lifted fn's `Forall` is - doable but error-prone; defer to 16b.6. -- **Single-level LetRec.** A LetRec whose body contains - another LetRec that captures the outer's name or params is - not supported in 16b.2; reject with a clear error → 16b.7. - -**What 16b.2 ships (under the recommended scope).** - -- `desugar_term`'s `scope` parameter changes from - `&BTreeSet` to `&BTreeMap`, - where `ScopeEntry` is `KnownType(Type) | LetBound | MatchArm`. -- Each binder extends the map: fn/Lam-params with `KnownType`, - Let-bindings with `LetBound`, Match-arm bindings with - `MatchArm`. -- LetRec arm's capture detection: free-vars ∩ scope-keys. - For each capture, look up `ScopeEntry`. If `KnownType(t)`, - proceed. If `LetBound` / `MatchArm`, error. -- Validate: walk body and in_term, assert `name` only appears - as `Term::App.callee`. Helper `validate_callee_only_use`. -- Build augmented fn type: `original.ty` with capture types - appended to `params`. Reject if `original.ty` is `Forall` - (out of scope per restriction 2 above). -- Rewrite call sites: `(app f a b)` → `(app f$lr_N a b cap0 - cap1)`. Use a new helper `subst_call_with_extras` that - walks the term, recognizes `Term::App { callee = Var{f} }` - patterns, rewrites them, and recurses elsewhere. -- Substitute `f` → `f$lr_N` everywhere in body (for the - recursive self-reference at the lifted level). The existing - `subst_var` from 16b.1 handles this, but care needed: it - must run **after** `subst_call_with_extras` so that - recursive calls get both the rename and the extras. -- Append the lifted `FnDef` to `Module.defs`. - -**Risk.** The most likely failure mode is the typechecker -rejecting a synthesized augmented fn signature that we believe -should type-check. Mitigation: implement under restriction 2 -(no Forall) so we never construct a Forall; the augmented type -is plain `Type::Fn` with monomorphic capture types appended. - -**Tests.** - -- `examples/local_rec_capture.ailx`: an enclosing fn with one - Int param, a LetRec that recurses against that param. e2e - asserts an output value derived from that capture. -- Negative tests at the desugar unit-test level: Let-binding - capture errors; Match-arm capture errors; name-as-value - errors; nested LetRec mutual-capture errors. All - `#[should_panic]` (consistent with 16b.1's panic discipline). - -**Adjacent open items.** - -- **16d**: chain-machinery's `Unit` terminator forces a - trailing `_` arm in matches that are otherwise exhaustive, - surfaced by 16c (`categorize_first` fixture). Two design - paths: (a) introduce a polymorphic `__unreachable__` - builtin that codegen lowers to LLVM `unreachable`, used as - the chain default; (b) run an exhaustiveness pre-check in - desugar against the scrutinee's ADT and omit the - terminator when arms cover all ctors. Path (a) is broader - (gives users a primitive for panics/asserts). Path (b) is - purer (terminator never appears for exhaustive matches) - but needs ADT lookup in desugar. -- **16e**: extend `==` from Int-only to Int/Bool/Str. Bool - comparison is i1-equality (trivial). Str comparison - needs a runtime `strcmp`. Surfaced by 16c (`build_eq` - produces `(app == ...)` for Bool/Str/Unit but currently - fails at typecheck because `==` is not declared for them). -- **17a**: per-fn arena allocator. Gated on user - memory-management discussion (separate stored memory). - -**Queue update post-16c.** 16c done. Open: 16b.2 (this entry — -planning only, awaiting user input on path 1 vs path 2), 16d -(planning needed — pick path a vs b), 16e (`==` extension), -17a (gated). All implementation work in this session-arc is -suspended at this planning checkpoint. - -## Iter 16b.2 — LetRec captures of fn-params (path-1 safe subset) - -**Goal.** Lift 16b.1's no-capture restriction for the **path-1 safe -subset** described in the 16b.2 planning entry: support a -`(let-rec ...)` whose body captures one or more names from the -enclosing scope, **provided every capture comes from a fn-param or -Lam-param** (whose types are statically declared at desugar time). -The lifted top-level fn's signature gets the capture types appended -to `params`; every call site of the LetRec name is rewritten to -pass the captures positionally as extra args. Anything outside the -safe subset is rejected at desugar time with a panic that names the -violation and points at the follow-up iter that will handle it -(16b.3 / 16b.4 / 16b.5 / 16b.6 / 16b.7). - -**Path-1 restrictions in force (rejected at desugar with a -follow-up-iter pointer).** - -- **Let-binding capture** (`Term::Let`-bound name, type unknown at - desugar). Queued for **16b.3**. -- **Match-arm pattern-binding capture** (constructor-field - substitution from the scrutinee's ADT not done at desugar). - Queued for **16b.4**. -- **Name-as-value use** (the LetRec name `f` appears anywhere - except as the callee of a `Term::App` — e.g. `(let g f ...)` or - `(app some_hof f)`). Needs closure conversion. Queued for - **16b.5**. -- **Polymorphic enclosing fn** (`Type::Forall`). Captured fn-param - types may mention outer type vars; constructing the lifted - signature's `Forall` is error-prone. Encoded as - `ScopeEntry::LetBound` for every fn-param of a `Forall`-typed - enclosing fn — caught at the same site as let-binding captures. - Queued for **16b.6**. -- **Nested LetRec mutual capture** (an inner LetRec captures the - outer LetRec's name or params). Queued for **16b.7**. - -**What shipped.** - -- `crates/ailang-core/src/desugar.rs` (1341 → 1853 LOC, +512). The - hot changes: - - New `enum ScopeEntry { KnownType(Type), LetBound, MatchArm, - EnclosingLetRec }`. The `scope` parameter threaded through - `desugar_term` and helpers became - `&BTreeMap` (was `&BTreeSet`). - Every binder ([`Term::Let`], [`Term::Lam`], [`Term::Match`] - arm, [`Term::LetRec`]) inserts the appropriate variant; - `desugar_module` seeds fn-param entries from `f.ty` (peeled - `Forall`). - - `Term::LetRec` arm rewritten end-to-end: peel `ty` to its - inner `Type::Fn` to learn the LetRec's own param types, - extend body-scope with `EnclosingLetRec` for `name` + - `KnownType(_)` for params, recurse on body and in_term, run - `free_vars_in_term` against `{name} ∪ params`, intersect - with the outer scope's keys, classify each capture's - `ScopeEntry` (KnownType → accept; everything else → panic - with the follow-up iter pointer), validate via - `find_non_callee_use` that `name` only appears as a callee, - build the augmented `Type::Fn` with capture types appended, - rewrite call sites via `subst_call_with_extras`, then - `subst_var` for any non-call references (defensive — none - survive the validator), append the lifted `FnDef`. - - Two new free helpers (~150 LOC together): - `subst_call_with_extras(t, name, lifted, extras)` rewrites - every `Term::App { callee = Var{name} }` to - `Term::App { callee = Var{lifted}, args ++ extras_as_vars }`, - walking through every variant; `find_non_callee_use(t, name)` - walks `t` and returns `Some(t)` at the first `Term::Var { - name == name }` reference in a non-callee position, `None` - otherwise. Plus `peel_forall_to_fn` (one-liner). - - The 16b.1 panic is gone for fn-param captures (replaced by - the lifter); it persists for every other capture kind, with - a sharper message. -- `crates/ail/tests/e2e.rs` (+18): `local_rec_capture_demo` — - e2e count 38 → 39. Asserts that the `local_rec_capture` fixture - prints `0\n10\n45\n`. -- `examples/local_rec_capture.ailx` + `.ail.json` (35 LOC source): - `sum_below(n)` returns the sum of integers `1 + ... + (n-1)`. - Body uses an inner `(let-rec loop (params i) ... (body ... (app - >= i n) ... (app loop (app + i 1))) (in (app loop 1)))`. The - helper captures `n` from `sum_below`'s param list; the desugar - pass lifts it to `loop$lr_0(i: Int, n: Int) -> Int` and rewrites - every `(app loop X)` to `(app loop$lr_0 X n)`. `main` drives - `sum_below` at 1, 5, 10 → outputs `0`, `10`, `45`. -- `crates/ailang-core/src/desugar.rs::tests` (3 new): - `let_rec_capture_fn_param_lifts_with_extra_arg` (positive), - `let_rec_capture_let_binding_panics` - (`#[should_panic(expected = "16b.3")]`), - `let_rec_name_as_value_panics` - (`#[should_panic(expected = "16b.5")]`). The 16b.1 - `let_rec_with_capture_panics` test was repurposed into the - positive `let_rec_capture_fn_param_lifts_with_extra_arg`: its - fixture (a fn-param capture) is now legal and produces the - expected augmented signature. - -**The augmented-signature mechanism.** Given -`(let-rec f (params p1..pk) (type Fn(t1..tk) -> tr) (body B) (in I))` -inside a fn whose scope contains -`{c1: T1, ..., cm: Tm}` (KnownType entries) used as free vars in -`B`: - -- Lifted signature: `f$lr_N : Fn(t1..tk, T1..Tm) -> tr` with the - same `effects` set as the original. -- Lifted param-name list: `p1..pk, c1..cm` (capture names appended - unchanged so the lifted body's references resolve directly). -- Body rewrite: every `(app f a1..an)` in `B` → `(app f$lr_N - a1..an c1..cm)`. The `c1..cm` are `Term::Var` references that - resolve, in the lifted body, to the appended params with the - same names. Free uses of `f` in non-callee position are - pre-rejected by `find_non_callee_use`. -- In-term rewrite: every `(app f a1..an)` in `I` → `(app f$lr_N - a1..an c1..cm)`. The `c1..cm` are `Term::Var` references that - resolve, in the in-term's enclosing fn, to the captured names - themselves (still in scope). - -The rewrite is order-sensitive: `subst_call_with_extras` runs -**before** `subst_var`, so a recursive self-call inside `B` first -becomes `(app f$lr_N ... c1..cm)`, then any leftover bare `Var{f}` -(none in 16b.2 — pre-rejected) gets renamed to `Var{f$lr_N}`. The -second pass is defensive. - -**Tests: 102 → 106 (+4).** - -- e2e: 38 → 39 (`local_rec_capture_demo`). -- `ailang-core::desugar::tests`: 6 → 9. Net +3 because the - 16b.1-era `let_rec_with_capture_panics` test was repurposed - rather than removed (its fn-param-capture fixture is now a - positive lift), and two pure-negative tests were added. -- All other crates unchanged. - -**Cumulative state, post-16b.2.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term` enum: 11 variants (unchanged; LetRec is still - desugar-eliminated, no schema impact). All pre-16b.2 fixtures - hash bit-identically. -- 16a desugar pass now does three jobs: nested-ctor-pattern - flattening (16a), literal-pattern lowering (16c), and LetRec - lift (16b.1 → 16b.2 path-1). Single AST→AST hop between - `load_module` and typecheck. -- The `unreachable!("Term::LetRec eliminated by desugar")` arms - in `ailang-check` × 2 and `ailang-codegen` × 4 remain correct - — every LetRec is still gone before either stage runs. - -**Queue update post-16b.2 path-1.** 16b.2 path-1 done. Open: -**16b.3** (LetRec captures of `Term::Let`-bound names — would -need a post-typecheck re-run of the lift, or a small inference -on the let-value's type), **16b.4** (LetRec captures of match-arm -pattern bindings — needs ADT-def lookup + constructor-field -substitution), **16b.5** (closure conversion — lifts the -"name-as-value-only" restriction for both LetRec and Lam), -**16b.6** (LetRec inside a `Type::Forall`-quantified fn — needs -synthesised `Forall` for the lifted signature), **16b.7** -(nested LetRec mutual capture — generalised lifting that -accumulates captures across nesting). 16d (chain-machinery -exhaustiveness or `__unreachable__` — planning needed), 16e -(`==` extension to Bool/Str/Unit), 17a (per-fn arena, gated) -unchanged. - -## Iter 16b.3 — LetRec captures of Let-bound names - -**Goal.** Lift 16b.2's "fn/Lam-param captures only" restriction. A -`(let-rec ...)` may now capture names bound by a `Term::Let` in the -enclosing scope. Match-arm captures (16b.4), name-as-value (16b.5), -Forall enclosing fn (16b.6), and nested LetRec mutual capture -(16b.7) remain rejected at desugar time. - -**Architectural choice (path-2: post-typecheck lift).** The desugar -pass cannot resolve a `Term::Let`-bound name's type — `Term::Let` -carries no annotation; the value's type is inferred at typecheck. -Three options were on the table (path-1 = stay-at-desugar with -restrictions, path-2 = post-typecheck lift, path-3 = run a private -inference inside desugar). Path-2 is the cleanest: typechecker is -the single source of truth, and the lift now becomes a small -AST-→-AST pass with O(1) type lookups against the typechecker's -env. `Term::LetRec` reaches the typechecker only when the desugar -pass deferred it — for the 16b.2 fast-path (KnownType captures -only) the desugar still does the lift in one hop. - -**What shipped.** - -- `crates/ailang-core/src/desugar.rs` (1853 → 1931 LOC, +78). The - `Term::LetRec` arm now has three exits: - (a) `KnownType`-only captures → existing 16b.2 lift (unchanged). - (b) Any `LetBound` capture → reconstruct the `Term::LetRec` - with desugared sub-terms and return it (defer to - post-typecheck pass). - (c) `MatchArm` / `EnclosingLetRec` → panic with the same - follow-up-iter pointers as 16b.2. - The `find_non_callee_use` (16b.5) check moved to run before - classification so the diagnostic fires consistently regardless - of which exit is taken. Four helpers (`free_vars_in_term`, - `subst_var`, `subst_call_with_extras`, `find_non_callee_use`, - `pattern_binds`) were promoted from `fn` to `pub fn` so the - post-typecheck pass can reuse them. -- `crates/ailang-check/src/lib.rs` (2887 → 3281 LOC, +394 - including tests). - - `verify_tail_positions` and `synth` arms for `Term::LetRec` - replaced. `verify_tail_positions`: body is NOT in tail - position (it's a fn body — the LetRec name is what gets - tail-called); in_term inherits the enclosing context. `synth`: - peel any `Forall` defensively, validate param count, install - `name + params` in locals for the body, synth the body, unify - against `ret_ty`, check the effect-subset rule (same as - `Term::Lam`); restore locals; install `name` for the - in-clause; synth, return. - - New `pub fn check_and_lift(m) -> Result<(CheckedModule, - Module)>` runs check + desugar + `lift_letrecs` and returns - both the original-symbols `CheckedModule` and the lifted - module ready for codegen. -- `crates/ailang-check/src/lift.rs` (new file, 720 LOC). The - `pub fn lift_letrecs(m: &Module) -> Result` post-pass. - - Fast-path: `contains_any_letrec(m)` returns `false` → - return input unchanged. Skips env construction so cross- - module modules (which we don't fully wire up here) are not - touched unless they actually contain a deferred LetRec. - - Builds a single-module env (builtins + module type defs + - module globals + imports + current_module) and walks every - `Def::Fn`'s body, threading a parallel `IndexMap` of locals as scope. At each `Term::Let`, synth the - value's type to populate locals; at each `Term::Match` - arm, run a minimal `type_check_pattern_for_lift` to infer - pattern-arm bindings. - - At every `Term::LetRec`: post-order recurse first (so - nested LetRecs are lifted before their enclosing one), - re-run `find_non_callee_use` defensively, recompute - captures via `free_vars_in_term`, look up each capture's - type from `locals`, build the lifted `FnDef` (capture types - appended to params), append to a `lifted: Vec` - accumulator, rewrite call sites in body and in_term via - `subst_call_with_extras`, then `subst_var` for any - leftover bare references. Synthetic name `$lr_N` - seeded past the highest existing `*$lr_N` suffix in - `m.defs` so it cannot collide with a 16b.2 lift. Synthetic - FnDefs carry a `doc` of the form - `"Lifted by 16b.3 from let-rec '' inside ''."`. - - The `subst_call_with_extras` and `subst_var` helpers come - straight from `ailang-core::desugar` — re-used rather than - duplicated (the brief asked for one or the other; re-use - via `pub` keeps the rewrite logic single-sourced). -- `crates/ail/src/main.rs` (+25 LOC). `build_to` now goes - `load → check → per-module (desugar + lift_letrecs) → codegen`. - The `check` subcommand stays typecheck-only (no lift needed - for type-checking). Codegen's internal `desugar_module` - call is harmless on a lifted module — no `Term::LetRec` - remains, so the LetRec arm is never invoked. -- `examples/local_rec_let_capture.ailx` + `.ail.json` (48 LOC - source). `count_below(n)` returns how many `i` in `1..=n` are - strictly less than a `let threshold = (app + 5 5)` computed - inside the enclosing fn. The recursive helper `loop` captures - `threshold` (Let-bound; type unknown until typecheck) and - recurses on its own counter `i`. The lift produces - `loop$lr_0(i: Int, n: Int, threshold: Int) -> Int` and - rewrites every `(app loop X)` to `(app loop$lr_0 X n threshold)`. - The let-value is `(app + 5 5)` rather than a literal so the - type-synthesis path is exercised. Drives at 0, 5, 15 → - output `0\n5\n9\n`. -- `crates/ail/tests/e2e.rs::local_rec_let_capture_demo` (+18): - e2e count 39 → 40. -- `crates/ailang-core/src/desugar.rs::tests`. The 16b.2-era - `let_rec_capture_let_binding_panics` test was repurposed - into a positive test - `let_rec_capture_let_binding_is_deferred_to_post_typecheck` - that asserts the desugar leaves the LetRec in place (no - lifted fn appended; original LetRec still reachable). Net - test count unchanged. -- `crates/ailang-check/src/lib.rs::tests` (24 → 27, +3): - `letrec_with_let_binding_capture_typechecks` (positive), - `letrec_body_wrong_return_type_is_rejected` (negative — body - returns Bool but declared Int), and - `lift_letrecs_on_let_capture_produces_synthetic_fn` (asserts - synthetic FnDef added with augmented signature and call sites - rewritten). -- `docs/DESIGN.md` (+12 LOC). Pipeline section gains the - `lift_letrecs` stage between `check` and `codegen`, with a - paragraph clarifying it runs only on the `build` / `run` - paths and that synthetic FnDefs do not appear in - `CheckedModule.symbols`. - -**The deferral mechanism.** Given -`(let y (app + 5 5) (let-rec helper (params x) (type Fn(Int) -> Int) - (body (app + x y)) (in (app helper 1))))` -inside an enclosing fn: - -- Desugar runs. The LetRec's outer scope is `{n: Int (fn-param), - y: LetBound}`. Free vars of `body` minus `{name, params}` = - `{+, y}`; intersection with scope = `{y}`. `y` is `LetBound`, - so the all-or-nothing classifier sees `has_let_bound = true` - and reconstructs the LetRec with desugared sub-terms instead - of lifting. -- Typecheck runs. The new `Term::LetRec` arm in `synth` extends - locals with `helper: Fn(Int) -> Int` and `x: Int`, synths the - body to `Int`, unifies with `ret_ty`, and accepts. -- `lift_letrecs` runs. Walking outer's body, it threads locals. - At the `Term::Let`, `synth_type` resolves `(app + 5 5) → Int` - and inserts `y: Int` into locals. At the `Term::LetRec`, the - capture analyser collects `{y}` and reads its type as `Int`. - Builds `helper$lr_0(x: Int, y: Int) -> Int`, rewrites - `(app helper 1)` → `(app helper$lr_0 1 y)`, appends the - lifted `FnDef`. The Term::LetRec node is replaced by its - rewritten in-clause. -- Codegen runs on the lifted module. No `Term::LetRec` remains; - the four `unreachable!("Term::LetRec eliminated by desugar")` - arms in codegen stay valid (the message is a slight - misnomer post-16b.3 — by the time codegen runs, every LetRec - has been eliminated by desugar OR lift_letrecs — but the - invariant holds). - -**Tests: 106 → 110 (+4).** - -- e2e: 39 → 40 (`local_rec_let_capture_demo`). -- `ailang-check::tests`: 24 → 27 (the three new LetRec tests). -- `ailang-core::desugar::tests`: 9 → 9 (one `#[should_panic]` - test repurposed to a positive defer-check test — net 0). - -**Cumulative state, post-16b.3.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term` enum: 11 variants (unchanged). All pre-16b.3 fixtures - hash bit-identically — additive at the typechecker / new-pass - level only. -- Compiler stages: load → desugar → typecheck → - **lift_letrecs (16b.3)** → codegen. The `check` subcommand - stops at typecheck and skips the lift; `build` / `run` go all - the way through. -- The four `unreachable!("Term::LetRec eliminated by desugar")` - arms in `ailang-codegen` remain correct: by the time codegen - runs, no LetRec survives — desugar lifts the 16b.1/16b.2 - cases, lift_letrecs catches the 16b.3 case. The two - `unreachable!` arms in `ailang-check` were replaced with the - new typing rule (`verify_tail_positions` and `synth`). -- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5 - (unchanged). - -**Queue update post-16b.3.** 16b.3 done. Open: **16b.4** (LetRec -captures of match-arm pattern bindings — needs ADT-def lookup + -constructor-field substitution; would slot into the same -`lift_letrecs` pass with extended pattern-arm walk), **16b.5** -(closure conversion — lifts the "name-as-value-only" restriction -for both LetRec and Lam; out of scope for this iter line), -**16b.6** (LetRec inside a `Type::Forall`-quantified fn — needs -synthesised `Forall` for the lifted signature), **16b.7** -(nested LetRec mutual capture — generalised lifting that -accumulates captures across nesting; partly already handled by -the post-order traversal in `lift_letrecs` but the mutual case -is genuinely harder). 16d (chain-machinery exhaustiveness or -`__unreachable__`), 16e (`==` extension to Bool/Str/Unit), -17a (per-fn arena, gated) unchanged. - -## Iter 16b.4 — LetRec captures of match-arm pattern bindings - -**Goal.** Lift the 16b.3-era "match-arm capture rejected" panic. A -`(let-rec ...)` whose body captures a name bound by a `Term::Match` -arm pattern (a `Pattern::Var`, or a `Pattern::Var` sub-pattern of -a `Pattern::Ctor`) is now legal. Same architectural path as 16b.3: -desugar defers it; `lift_letrecs` resolves the capture's type from -the enclosing match's scrutinee type, applying constructor-field -substitution against the matched ctor's declared field types. - -**What changed in desugar.** Exactly one classification arm. The -LetRec capture loop used to treat `ScopeEntry::MatchArm` as a hard -panic (queued for 16b.4); it now sets `needs_defer = true` and -falls through to the same defer-arm 16b.3 wrote for `LetBound`. -The `has_let_bound` boolean was renamed to `needs_defer` so it -covers both LetBound and MatchArm captures uniformly. Mixed scopes -(some KnownType + some MatchArm, or some MatchArm + some LetBound) -defer just like the 16b.3 mixed case — the all-or-nothing decision -is preserved. `EnclosingLetRec` still panics (queued for 16b.7). -Total desugar diff: ~25 LOC of comment/classification changes; no -new helpers, no new traversals. - -**What `lift_letrecs` already supported.** The post-typecheck pass -needed **zero** code changes. 16b.3's `type_check_pattern_for_lift` -was already written to handle the full pattern grammar: - -- `Pattern::Wild` / `Pattern::Lit`: bind nothing (no-op). -- `Pattern::Var`: binds the entire scrutinee's type. -- `Pattern::Ctor`: looks up the ADT in `env.types` (or - `env.module_types` for a qualified `module.Type`), validates ctor - arity, builds the type-var → arg substitution from - `td.vars` zip `s_ty.args`, applies it to each `cdef.fields[i]`, - and recurses on each sub-pattern with the substituted field - type. The recursion handles arbitrarily-nested Ctor patterns - for free (though desugar's 16a flattening means lift_letrecs - rarely sees deeply-nested ctor patterns in practice). - -The only reason 16b.3 left the MatchArm path under a `panic!` was -to keep 16b.3's scope tight — the machinery for resolving the -type was already in place and exercised by 16b.3's own test -suite, just gated behind the desugar panic. - -**The fixture: `examples/local_rec_match_capture.ailx` (+ `.ail.json`).** -Defines `data Pair (vars a b) (ctor MkPair a b)`. `count_below(p)` -takes a `Pair Int Int` (threshold, n), pattern-matches with -`(pat-ctor MkPair threshold n)` (so `threshold` and `n` are both -match-arm bindings of type `Int` — the ADT's `[a, b]` ctor fields -substituted against the scrutinee's `[Int, Int]` type args), and -inside that arm runs a recursive `loop` that captures BOTH -match-arm bindings to count integers in `1..=n` strictly less than -`threshold`. The lift produces a synthetic top-level fn -`loop$lr_0(i: Int, threshold: Int, n: Int) -> Int` and rewrites -every `(app loop ARGS)` in the arm body to -`(app loop$lr_0 ARGS threshold n)`. Drives at `MkPair 10 0`, -`MkPair 10 5`, `MkPair 10 15` → output `0\n5\n9\n` (the same -numbers as the 16b.3 fixture so the comparison is direct). - -The fixture exercises the full 16b.4 chain: parameterised ADT -construction, match on the ctor, two simultaneous match-arm -captures (not just one), and `lift_letrecs` resolving both via -constructor-field substitution. - -**What this iter exercises that 16b.3 did not.** - -- The MatchArm branch of `type_check_pattern_for_lift` (was - unreachable in real programs because desugar always panicked - before the lift saw such captures; covered by the 16b.3 test - suite synthetically but never end-to-end). -- Multi-capture LetRec (16b.2 used a single fn-param capture; - 16b.3 used a single Let-capture; this is the first fixture with - two captures appended in the lifted signature, in a fixed - order driven by the BTreeSet's deterministic iteration). -- Constructor-field substitution at the lift site (the type args - of the scrutinee type `Pair Int Int` flow into the synthetic - fn's signature via the substitution map). - -**Tests: 113 → 116 (+3).** - -- e2e: 40 → 41 (`local_rec_match_capture_demo`, asserts the - `0/5/9` stdout from the new fixture). -- `ailang-check::tests`: 27 → 28 - (`lift_letrecs_on_match_arm_capture_produces_synthetic_fn`: - builds a Pair-using module, asserts the lifted FnDef's name - starts with `helper$lr_`, params are `[z, x]`, and the type is - `(Int, Int) -> Int`). -- `ailang-core::desugar::tests`: 9 → 10 - (`let_rec_capture_match_arm_is_deferred_to_post_typecheck`: - asserts no fn was lifted at desugar time and a Term::LetRec - still survives in the outer body for the post-typecheck pass). - -**Cumulative state, post-16b.4.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term` enum: 11 variants (unchanged). All pre-16b.4 fixture - hashes bit-identical (verified via `ail manifest` on the four - prior LetRec/lit_pat fixtures: hashes match pre-16b.4 values). -- Compiler stages: load → desugar → typecheck → lift_letrecs → - codegen (unchanged from 16b.3). -- The `unreachable!` arms in `ailang-codegen` and the typing rule - in `ailang-check` are unchanged. -- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5 - (unchanged). - -**Queue update post-16b.4.** 16b.4 done. Open: **16b.5** (closure -conversion — lifts the "name-as-value-only" restriction for both -LetRec and Lam; needs ABI-level work, out of scope for the 16b.x -sweep that's been about scope-of-capture restrictions only), -**16b.6** (LetRec inside a `Type::Forall`-quantified fn — needs a -synthesised `Forall` for the lifted signature plus rigid-var -threading; the `lift_letrecs` panic at the Forall-LetRec arm is -the entry point), **16b.7** (nested LetRec mutual capture — -generalised lifting that accumulates captures across nesting; -the post-order traversal in `lift_letrecs` already handles the -strictly-nested non-mutual case but the mutual case needs joint -lifting). 16d (chain-machinery exhaustiveness or -`__unreachable__`), 16e (`==` extension to Bool/Str/Unit), -17a (per-fn arena, gated) unchanged. - -## Iter 16b.5 — LetRec name as value (in_term only) via eta-Lam wrapper - -**Goal.** Lift the "callee-only use of the LetRec name" restriction -for the **in-clause** of a `(let-rec ...)`. After this iter, the -LetRec name `f` may appear in `in_term` as a value (e.g. passed to -a higher-order combinator, bound to another `let`, stored in an ADT -field). References to `f` as a value INSIDE the LetRec's own body -remain rejected — that case has a chicken-and-egg with the call-site -rewrite (the body's bare `f` references would need to call a name -that doesn't exist before the lift) and is queued as a non-trivial -extension. - -**Architectural choice (no new ABI).** Reuse the closure-pair -machinery from Iter 8b (`lower_lambda` in `ailang-codegen`). After -the existing 16b.2/16b.3 capture rewrite produces the lifted fn -`f$lr_N(p1..pk, c1..cm) -> rt` and rewrites every `(app f a..)` → -`(app f$lr_N a.. c1..cm)`, the lifter detects whether `in_term` -contains any non-callee use of `f`. If yes, it WRAPS the rewritten -in-term in - -``` -(let f - (lam (params p1..pk) (ret rt) (effects E) - (app f$lr_N p1..pk c1..cm)) - ) -``` - -The Lam's free variables (the captures `c1..cm`) are picked up -automatically by the existing 8b capture analysis in -`lower_lambda`. The Lam's signature mirrors the LetRec's declared -type, so any non-callee `Var{f}` in `in_term` resolves to a -`Fn(t1..tk) -> rt ![E]` value — usable wherever the typechecker -expects that type. Callee-position references in `in_term` are -already rewritten to `f$lr_N` (no closure indirection at the call -site; the wrap only affects value-position uses). - -The reduction is satisfying: 16b.5 reduces a problem that looked -like it needed a new ABI to a problem already solved by Iter 8b. -No closure ABI changes, no codegen plumbing, no LLVM IR change. - -**Where the wrap happens.** Two sites — both fast paths can -encounter LetRecs with non-callee uses in in_term: - -1. **`crates/ailang-core/src/desugar.rs`** (~+50 LOC inside the - existing `Term::LetRec` arm). The body-side `find_non_callee_use` - panic was kept (now points at the body-only restriction rather - than the generic 16b.5 deferral). The in_term-side check no - longer panics; instead it sets a flag `in_has_value_use`. After - the standard call-site rewrite, when `in_has_value_use` is true, - the in-term is wrapped in a `Term::Let { name: f, value: Term::Lam, - body: }`. The Lam's `effects` come from - `peel_forall_to_fn(ty)`. The defensive `subst_var` on `in_term` - is dropped — bare `Var{f}` references must reach the new - let-binding unchanged. - -2. **`crates/ailang-check/src/lift.rs`** (~+50 LOC inside the - existing post-typecheck LetRec arm). Symmetric to (1). The - wrap is identical in shape. The lift's effects come straight - from `ty` (already known to be `Type::Fn` at this point — - Forall is rejected upstream). - -**What stays rejected.** -- `find_non_callee_use(body, name)` returns Some → panic with the - 16b.5 body-only message: "name-as-value of a LetRec inside its - own body is not yet supported." Both sites. -- `EnclosingLetRec` capture (16b.7) — unchanged. -- Forall-typed LetRec (16b.6) — unchanged. - -**Did the codegen Lam machinery handle the wrapped form on the -first try?** Yes. The eta-Lam is a perfectly ordinary `Term::Lam` -whose body happens to be a single `Term::App` to a top-level -synthetic fn with the lam's params + free-var captures appended. -8b's `collect_captures` walks the Lam body, sees `f$lr_N` in -top-level fns (it's been pushed to `module.defs` already), sees -`p1..pk` in `bound`, and classifies the remaining `c1..cm` as -captures — exactly right. No AST massaging was needed. The -non-capture variant (no extras) reduces to the simplest possible -Lam (params-only body), which 8b also handles trivially. End-to-end -ran on first attempt for both fixtures. - -**What shipped.** - -- `crates/ailang-core/src/desugar.rs`: split `find_non_callee_use` - into body-only (panic) and in_term-only (sets a flag, then wraps - below). Drop the post-rewrite `subst_var` on the in-term — it - would otherwise rename the very `Var{f}` references that need to - resolve to the eta-Lam binding. Updated docstring on the - pre-existing panic test (`let_rec_name_as_value_panics` → - `let_rec_name_as_value_in_body_panics`) and added a new positive - unit test `let_rec_name_as_value_in_in_term_wraps_to_eta_lam` - that constructs an in_term-side value use and asserts the - resulting AST shape (lifted FnDef appended; main's body is - `Let { f, Lam{x -> App f$lr_0 x}, }`). -- `crates/ailang-check/src/lift.rs`: same split. Drop the - defensive `subst_var` on `in_term` when there's a value use; - keep it as no-op when there isn't (zero-capture symmetry). -- `examples/local_rec_as_value.ailx` + `.ail.json`: factorial - bound by `(let-rec ...)` inside main, then in the in-clause - passed as a VALUE to a higher-order combinator - `apply5 : (Fn(Int) -> Int) -> Int`. No captures. Expected - stdout: `120` (= apply5(factorial) = factorial(5)). -- `examples/local_rec_as_value_capture.ailx` + `.ail.json`: - variant with capture. `run_with_base(base)` builds a recursive - `factorial_plus` that uses `base` in its base case, then passes - the helper as a VALUE to `apply5`. The lifted fn is - `factorial_plus$lr_0(n: Int, base: Int) -> Int`; the eta-Lam - binds `factorial_plus` to a `(lam (n) (app - factorial_plus$lr_0 n base))` whose free var `base` becomes a - standard 8b closure-env capture. Expected stdout: `1320` - (= 5*4*3*2*(1+10)) and `12120` (= 5*4*3*2*(1+100)). -- `crates/ail/tests/e2e.rs`: `local_rec_as_value_demo` and - `local_rec_as_value_capture_demo` (e2e count 41 → 43). -- `crates/ailang-core/src/desugar.rs::tests`: - `let_rec_name_as_value_in_body_panics` (renamed from the - 16b.2-era panic test; same fixture) and the new positive test - `let_rec_name_as_value_in_in_term_wraps_to_eta_lam` (desugar - count 10 → 11; net +1). - -**Cross-iter regression check.** All five existing LetRec/lit_pat -fixtures (`local_rec_demo`, `local_rec_capture`, -`local_rec_let_capture`, `local_rec_match_capture`, `lit_pat`) -build, run, and produce identical stdout. Their on-disk hashes -(reported via `ail manifest`) are unchanged because the desugar -pass runs after `load_module` and never touches canonical bytes. - -**Tests: 113 → 116 (+3).** - -- e2e: 41 → 43 (`local_rec_as_value_demo`, - `local_rec_as_value_capture_demo`). -- `ailang-core::desugar::tests`: 10 → 11 (one rename + one new - positive test). Net +1. - -**Cumulative state, post-16b.5.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term` enum: 11 variants (unchanged). All pre-16b.5 fixture - hashes bit-identical. -- Compiler stages: load → desugar → typecheck → lift_letrecs → - codegen (unchanged from 16b.3). -- The four `unreachable!("Term::LetRec eliminated by desugar")` - arms in `ailang-codegen` are still correct — by the time - codegen runs, no LetRec survives. The eta-Lam is a `Term::Lam`, - which has its own well-trodden codegen path. -- DESIGN.md unchanged. Pipeline is unchanged. -- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5 - (unchanged). - -**Queue update post-16b.5.** 16b.5 done for the in_term-only -slice. Open: **16b.5-body** (informal — not formally numbered; -"non-trivial extension" per the orchestrator's brief), where the -LetRec name is used as a value INSIDE its own body. Solving that -needs either eta-conversion-of-self (the body's `f` would resolve -to a fresh Lam built around the lifted callee, but constructing it -inside the body before the lift completes is order-sensitive) or -a true closure conversion that doesn't lift to a top-level fn at -all. **16b.6** (Forall-typed LetRec), **16b.7** (nested mutual -LetRec capture), 16d, 16e, 17a unchanged. - -## Iter 16b.6 — LetRec inside polymorphic enclosing fn - -**Goal.** Lift the "monomorphic enclosing fn only" restriction -that 16b.2 introduced. A `(let-rec ...)` inside a fn whose -declared type is `Forall(α1..αn, Fn(...))` is now supported, -provided (a) the LetRec name is callee-only (any non-callee use -in `body` was already rejected by 16b.5; non-callee use in -`in_term` is rejected for the polymorphic case here — see -"closure-poly" below), and (b) other 16b.x restrictions still -apply (no nested LetRec mutual capture → 16b.7). - -**Architectural choice.** The lifted top-level fn is built as -`Forall(α1..αn, Fn(t1..tk, T1..Tm) → tr)` where `α1..αn` are -exactly the enclosing fn's type vars. Capture types `T1..Tm` -may mention any of those vars; the outer `Forall` binds them -back. Inside the enclosing fn's body every call site of the -lifted fn is monomorphic relative to the enclosing fn's current -instantiation, so codegen's existing Iter-12b/14a -monomorphisation machinery picks up `f$lr_N` from the -`mono_queue`, substitutes type args into both the original -params and the appended capture types (a single -`apply_subst_to_type` traversal handles both — captures live -in the same `Type::Fn.params` list as originals), and emits -specialised `f$lr_N__I`, `f$lr_N__B`, … per instantiation. -**No codegen changes were needed**: the lifted fn becomes an -ordinary `Forall` top-level fn and flows through the same -pipeline as a user-written polymorphic def. - -**Scope-building change.** 16b.2 entered fn-params of a -`Forall`-typed enclosing fn as `ScopeEntry::LetBound` — -defensive, because the param types could mention outer type -vars and the lift had no way to bind them. With the -`Forall(vars, Fn(t1..tk, T1..Tm) → tr)` synthesis above, that -bind site now exists. Fn-params of a polymorphic enclosing fn -therefore enter the scope as `ScopeEntry::KnownType(t)` -(matching the monomorphic case). The `LetBound` fallback is -reserved for `Term::Let`-bound names and for malformed fn -types (arity mismatch, non-Fn). Both `Desugarer` and `Lifter` -gained a `current_def_forall_vars: Vec` field that's -populated on entry to each `Def::Fn` and read by the LetRec -arm to decide whether to wrap the augmented type in a -`Forall`. - -**Why name-as-value-in-in-term is excluded for the polymorphic -case.** The 16b.5 wrap synthesises a `Term::Lam` whose body -calls the lifted fn positionally. `Term::Lam` has no -`Forall`-quantification slot — wrapping a polymorphic lifted -fn into a monomorphic Lam would lose the type vars. The -proper fix is closure conversion with polymorphism (a closure -pair generic over `α1..αn`); that's a separate, harder iter. -Both desugar and lift now panic with a clear message -referencing queue tag `closure-poly` (informally `16b.5b`) -when this combination is detected. Name-as-value-in-in-term -in a MONOMORPHIC enclosing fn continues to work via the -16b.5 eta-Lam wrap. - -**What shipped.** - -- `crates/ailang-core/src/desugar.rs` - (1931 → 1939 LOC + tests; +50/+135 net of test additions). - - New `Desugarer.current_def_forall_vars` field. Populated - on entry to each `Def::Fn` (cloned from `f.ty`'s - `Forall.vars`, or empty for monomorphic), restored on - exit. - - `desugar_module`'s per-def loop unified: instead of - branching `Type::Fn` vs `Type::Forall`, it peels the - inner `Type::Fn` once and treats both shapes uniformly - for fn-param scope-building. The defensive `LetBound` - fallback now fires only on a malformed fn type - (arity mismatch or non-Fn). - - `Term::LetRec` arm: when - `!current_def_forall_vars.is_empty() && in_has_value_use`, - panic with the new `Iter 16b.6` / `closure-poly` message. - When the fast-path lift fires, the `augmented_ty` is - `Type::Forall { vars: current_def_forall_vars.clone(), - body: Box::new(Type::Fn { ... }) }` if `vars` is non- - empty, else the bare `Type::Fn` as before. -- `crates/ailang-check/src/lift.rs` - (757 → 791 LOC; +34). Symmetric changes to `Lifter`: - new `current_def_forall_vars` field, populated alongside - the existing `rigid_vars` install in the per-def loop; - same Forall-wrap on `augmented_ty`; same `closure-poly` - panic for the polymorphic + name-as-value-in-in-term - combination. The lift's `synth_type` and - `type_check_pattern_for_lift` were unchanged — capture - types containing type vars flow through verbatim because - the typechecker installs the same rigid vars on entry. -- `examples/poly_rec_capture.ailx` + `.ail.json` (49 LOC - source). `apply_n_times : Forall(a). Fn(Int, a, Fn(a) -> a) -> a` - applies `f` to `x` exactly `n` times. The recursive helper - `loop` captures `f` from the enclosing fn's params. `f`'s - type is `Fn(a) -> a`, mentioning `a`. The lift produces - `loop$lr_0 : Forall(a). Fn(Int, a, Fn(a) -> a) -> a`. - `main` drives `apply_n_times` at TWO instantiations - (`a = Int` with `succ` to compute `succ` applied 5x to 0 - → 5; `a = Bool` with `flip` (a top-level wrapper around - `not`, since `not` is a builtin without a value-position - adapter) to compute `flip` applied 4x to false → false). - Codegen specialises `loop$lr_0` twice, emitting - `loop$lr_0__I` and `loop$lr_0__B` — confirmed in the - generated IR. The `flip` indirection is incidental (it - works around an unrelated builtin-as-value gap, not a - 16b.6 limitation). -- `crates/ail/tests/e2e.rs::poly_rec_capture_demo` (+25): - e2e count 43 → 44. -- `crates/ailang-core/src/desugar.rs::tests` (+2): - `let_rec_capture_under_polymorphic_enclosing_fn_lifts_to_forall_fn` - (positive — asserts the lifted FnDef's type is - `Forall(a, Fn(Int, a) -> a)` with the `a`-typed capture - appended), and - `let_rec_name_as_value_in_polymorphic_enclosing_fn_panics` - (`#[should_panic(expected = "16b.6")]`). -- `crates/ailang-check/src/lib.rs::tests` (+1): - `lift_letrecs_under_polymorphic_enclosing_fn_produces_forall_fn` - exercises the post-typecheck lift path (a `Term::Let`- - bound capture forces deferral to `lift_letrecs`); asserts - the lifted ty is `Forall(a, Fn(Int, a, Int) -> a)`. - -**Codegen integration: did anything need to change?** No. -Verified end-to-end: the lifted `Forall` fn is registered in -`module_polymorphic_fns` by `lower_workspace`'s pass-1 (no -distinction between user-written and synthetic Forall fns — -both are `FnDef`s with `Type::Forall` types). At each call -site inside the enclosing fn, `lower_polymorphic_call` -derives the substitution from the actual arg types, -specialises through `apply_subst_to_type` (which substitutes -through `Type::Fn.params` uniformly — original params and -appended captures share the same list), and queues a -specialisation. `emit_specialised_fn` consumes the queue, -substitutes type vars in both the body and the type, and -emits the LLVM fn. `poly_rec_capture` produces -`loop$lr_0__I` and `loop$lr_0__B` in the IR, both correctly -typed (capture `f: ptr` carries through unchanged because -`Fn(_) -> _` is `ptr` at the LLVM level). - -**Cross-iter regression check.** All seven prior fixtures -(`local_rec_demo`, `lit_pat`, `local_rec_capture`, -`local_rec_let_capture`, `local_rec_match_capture`, -`local_rec_as_value`, `local_rec_as_value_capture`) build, -run, and produce identical stdout. Their on-disk hashes -(`ail manifest`) are bit-identical: hashes flow from -canonical JSON, untouched by 16b.6's desugar/lift changes. - -**Tests: 116 → 120 (+4).** -- e2e: 43 → 44 (`poly_rec_capture_demo`). -- `ailang-check::tests`: 28 → 29 - (`lift_letrecs_under_polymorphic_enclosing_fn_produces_forall_fn`). -- `ailang-core::desugar::tests`: 21 → 23 (positive lift + - negative panic). Net +2. - -**Cumulative state, post-16b.6.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term` enum: 11 variants (unchanged). All pre-16b.6 - fixture hashes bit-identical. -- Compiler stages: load → desugar → typecheck → - lift_letrecs → codegen (unchanged pipeline; only the - per-stage logic at the LetRec arm changed). -- The four `unreachable!("Term::LetRec eliminated by - desugar")` arms in `ailang-codegen` remain correct: every - surviving LetRec is still gone before codegen runs (the - poly fast-path lifts here in desugar; the deferred path - is handled by `lift_letrecs`). The panic message in - codegen could be updated post-16b.3 to "by desugar OR - lift_letrecs"; left as-is for now (the invariant holds - either way). -- Compiler bugs surfaced and fixed in dogfood since 14a: - 5/5 (unchanged). - -**Queue update post-16b.6.** 16b.6 done. Open: -**`closure-poly`** (informally 16b.5b — name-as-value of a -LetRec inside a polymorphic enclosing fn; needs closure -pair generic over the enclosing fn's type vars). -**16b.5-body** (name-as-value INSIDE the LetRec's own body). -**16b.7** (nested LetRec mutual capture). 16d -(chain-machinery exhaustiveness or `__unreachable__`), 16e -(`==` extension to Bool/Str/Unit), 17a (per-fn arena, gated) -unchanged. - -## Iter 16b.7 — nested LetRec mutual capture (params, not name) - -**Goal.** Lift the "nested LetRec mutual capture" rejection, -but only for the case where the inner LetRec captures the -OUTER LetRec's PARAMS. Capturing the OUTER's NAME from an -inner LetRec body remains rejected — it is the same -chicken-and-egg as 16b.5-body / closure-of-self (the value -form of the outer LetRec doesn't exist before its own lift -completes), and a clearer error is now emitted in its place. - -**Architectural confirmation (the empirical result).** The -params-only case worked **out of the box**. The desugar pass -already enters outer's params into the inner's outer-scope as -`ScopeEntry::KnownType` while marking only the outer's NAME -as `EnclosingLetRec` (`desugar.rs:552-555`); the same shape -holds in the post-typecheck lifter via the `locals` map -(`lift.rs:373-378`). Post-order traversal then lifts the -inner LetRec first under the existing 16b.2 fast path, with -the outer's param appended to the inner's signature; once -the outer is then lifted, every call site inside outer's -body of the form `inner$lr_M(args, outer_param_x)` continues -to refer to `outer_param_x` because outer hasn't yet renamed -its params (outer's lift only renames its OWN name → lifted- -name, not its params). After outer's own lift, those param -references resolve to outer's lifted-fn's params (same name). - -So: no implementation change was needed for the supported -case. The work in 16b.7 is the test/fixture/JOURNAL surface -plus tightening the rejection of the still-unsupported -sub-case (inner-captures-outer-NAME) to point at the right -follow-up. - -**What shipped.** - -- `examples/nested_let_rec.ailx` + `.ail.json` (50 LOC - source). `nested_sum(n)` returns the sum of `1..=n` by - combining two recursive helpers: outer LetRec `outer` - iterates `i` over `1..=n`; for each `i` it calls inner - LetRec `inner(j)` to count one for each `j` in `1..=i`. - Inner captures `i` (outer's PARAM) and produces the count - `i`; outer captures `n` (the enclosing fn's param) and - sums those counts. Total = `n*(n+1)/2`. Drives at - `n ∈ {1, 3, 5}` → output `1\n6\n15\n`. The fixture - exercises the post-order lift, with inner lifted first as - `inner$lr_0(j: Int, i: Int) -> Int` and outer lifted next - as `outer$lr_1(i: Int, n: Int) -> Int`. -- `crates/ail/tests/e2e.rs::nested_let_rec_demo` (+18 LOC): - e2e count 44 → 45. -- `crates/ailang-core/src/desugar.rs`: tightened the - `ScopeEntry::EnclosingLetRec` panic (lines 653-660) from - the 16b.4-era "nested mutual-capture is not supported, - queued for 16b.7" message to a 16b.7 message that names - the precise constraint ("captures outer LetRec name `` - — closure conversion of a LetRec inside its own body - would be required") and points at the right follow-up - (`closure-of-self` / 16b.5-body / closure-poly). -- `crates/ailang-check/src/lift.rs`: added - `Lifter.enclosing_letrec_names: BTreeSet`, - populated on entry to each LetRec arm via - `BTreeSet::insert(name)` and removed on exit, mirroring - the desugar pass's `EnclosingLetRec` marker. The capture - classification step now panics with the same 16b.7 - message if any capture is in `enclosing_letrec_names`. - This closes a defensive hole the post-typecheck path - would otherwise leave open: without the marker, a - deferred outer LetRec (one with Let/Match captures) whose - inner LetRec captured the outer's name would silently - lift the inner with the outer's pre-lift fn-type as an - extra param, producing a type-correct AST that calls a - fn-value with the wrong arity at runtime once the outer - was itself lifted with captures. The new check rejects - that path symmetrically with the desugar path. -- `crates/ailang-core/src/desugar.rs::tests` (+2 net): - `nested_let_rec_inner_captures_outer_name_panics` - (`#[should_panic(expected = "16b.7")]`) and - `nested_let_rec_inner_captures_outer_param_lifts` - (positive — asserts two synthetic fns appended in - inner-first/outer-second order, and that the inner - lifted fn has params `[j, i]`). - -**What stays rejected.** Inner LetRec capturing the OUTER -LetRec's NAME (`EnclosingLetRec` in the desugar marker; -`enclosing_letrec_names` set in the lifter). The proper fix -is closure conversion of a LetRec inside its own body — the -same shape as 16b.5-body, queued there. Both desugar and -lift now panic with the same clarified message. - -**Tests: 120 → 122 (+2).** - -- e2e: 44 → 45 (`nested_let_rec_demo`). -- `ailang-core::desugar::tests`: 23 → 25 (+2: one negative - panic test + one positive lift test). -- All other crates unchanged. - -**Cross-iter regression check.** All eight prior LetRec -fixtures (`local_rec_demo`, `lit_pat`, -`local_rec_capture`, `local_rec_let_capture`, -`local_rec_match_capture`, `local_rec_as_value`, -`local_rec_as_value_capture`, `poly_rec_capture`) build, -run, and produce identical stdout. Their on-disk hashes -(verified via `ail manifest` on `local_rec_capture`) are -bit-identical: the desugar/lift edits change runtime -behaviour for previously-rejected cases only and never -touch canonical-bytes input. - -**Cumulative state, post-16b.7.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term` enum: 11 variants (unchanged). All pre-16b.7 - fixture hashes bit-identical. -- Compiler stages: load → desugar → typecheck → - lift_letrecs → codegen (unchanged). -- The four `unreachable!("Term::LetRec eliminated by - desugar")` arms in `ailang-codegen` remain correct. -- Compiler bugs surfaced and fixed in dogfood since 14a: - 5/5 (unchanged). - -**End-of-16b series note.** With 16b.7, the LetRec-capture -work is feature-complete except for the two deferred -closure-of-self sub-cases: - -- **`closure-of-self` (informally 16b.5-body)**: name-as- - value of a LetRec INSIDE its own body, OR an inner - LetRec capturing an outer LetRec's NAME. Both reduce to - the same problem: the LetRec's value-form does not exist - before its own lift completes, so any non-callee use - inside its body needs either eta-conversion-of-self - (constructing the value form before the lift, with the - unlifted name in scope) or a true closure conversion - that doesn't lift the LetRec to a top-level fn at all. -- **`closure-poly` (informally 16b.5b)**: name-as-value of - a LetRec inside a polymorphic enclosing fn. Needs a - closure pair generic over the enclosing fn's type vars - — a meaningful ABI extension. Independent of - `closure-of-self` but closely related. - -Every supported case is exercised by a fixture + -e2e + (where applicable) a desugar/lift unit test. The -16b.x sweep can be considered closed pending those two -deferrals; both are tracked separately in the queue and -neither blocks day-to-day authoring of recursive helpers. - -**Queue update post-16b.7.** 16b.7 done. Open: -**`closure-of-self`** (informally 16b.5-body — name-as- -value of a LetRec inside its own body, including nested- -LetRec inner-captures-outer-name; needs eta-conversion-of- -self or true closure conversion). **`closure-poly`** -(informally 16b.5b — name-as-value of a LetRec inside a -polymorphic enclosing fn; needs polymorphic closure -pairs). 16d (chain-machinery exhaustiveness or -`__unreachable__`), 16e (`==` extension to -Bool/Str/Unit), 17a (per-fn arena, gated) unchanged. - -## Iter 16d — chain-terminator via `__unreachable__` builtin - -**Goal.** Eliminate the synthetic `Unit`-typed chain terminator -that 16a/16c emitted for matches whose arms have non-Unit return -types. Pre-16d, the desugar pass's `desugar_match` used -`Term::Lit { lit: Literal::Unit }` as the deepest fall-through of -the let-bind + chain rewrite. That terminator unifies against -`Unit` only, so any match returning (say) `Int` had to carry a -trailing `(case _ )` arm whose sole purpose was to dominate -the terminator with a same-type value. Surfaced by 16c's -`categorize_first` fixture, where `IntList`'s two ctors (`Nil`, -`Cons`) are exhaustive on their own but the trailing `(case _ 0)` -was load-bearing for the chain machinery rather than for the -program's semantics. - -**Architectural decision (path a, by the orchestrator).** Two -paths were on the table per the 16c entry's "Adjacent open items": -**(a)** introduce a polymorphic `__unreachable__` builtin used as -the chain default; **(b)** run an exhaustiveness pre-check in -desugar against the scrutinee's ADT and omit the terminator -entirely for exhaustive matches. Path (b) is purer (terminator -never appears for exhaustive matches) but needs ADT lookup at -desugar time, which today runs single-module and would require -threading workspace type-registry access through the pass — a -non-trivial pipeline change. Path (a) is broader: besides -unblocking 16c's fixture, it gives users a real bottom primitive -for asserts, impossible branches, and panics. Path (a) chosen. - -**The new builtin.** `__unreachable__` is registered as a -**polymorphic value** (not a zero-arg fn) typed -`Type::Forall { vars: ["a"], body: Type::Var { name: "a" } }` — -i.e. `forall a. a`, the textbook bottom type. Reference site is -plain `(var __unreachable__)` (form-A bare ident -`__unreachable__`). The value-form was preferred over the -zero-arg-fn form because the obvious authoring shape (a name in -expression position) is then the right shape — `(app -__unreachable__)` would have rejected the natural use as a value -and added paren noise. Chosen the same way as how the -typechecker already treats every `Forall`-typed global: the -`Term::Var` resolver (`maybe_instantiate`) substitutes the -forall var with a fresh metavar at every use site, and -unification with the surrounding context's expected type pins -that metavar. - -**Codegen lowering.** `Term::Var { name = "__unreachable__" }` in -`lower_term` emits a single `unreachable\n` line, sets -`block_terminated = true`, and returns a dummy -`("0", "i8")` SSA + LLVM-type pair. The dummy type is sound -because the existing `Term::If`, `Term::Match`, and top-level -fn-body code paths all gate downstream emission on -`block_terminated`: the terminated branch's value/type is never -fed into a phi node. In an `if`, the live branch's value flows -through to the join unchanged (existing 14e behaviour). In a -`Term::Match` default block, the arm contributes nothing to -`phi_inputs` and the join collapses to whatever non-terminated -arms produced. No alternative lowering (e.g. `abort`/`trap`) -because LLVM `unreachable` lets the optimizer aggressively prune -the unreachable region. - -**What shipped.** - -- `crates/ailang-check/src/builtins.rs` (+12, of which ~7 are - doc): registers `__unreachable__` in `env.globals` with type - `forall a. a`; adds it to `list()` (consumed by the - `ail builtins` CLI subcommand and `value_names()`). Mirrors - how operators like `+` / `==` are installed. -- `crates/ailang-codegen/src/lib.rs` (+15, of which ~10 are - doc): special-cases `Term::Var { name = "__unreachable__" }` - in `lower_term` to emit `unreachable` + mark - `block_terminated`; adds the same `forall a. a` entry to - `builtin_ail_type` so `synth_arg_type` resolves it during - monomorphisation walks. -- `crates/ailang-core/src/desugar.rs` (+3 net): one-line - swap of the chain default from `Term::Lit { Unit }` to - `Term::Var { name: "__unreachable__".into() }` in - `desugar_match`; module-level doc updated to reflect the new - contract; one new unit test - (`chain_default_is_unreachable_builtin`) that walks the - desugared output for a two-lit-arm match and asserts the - deepest `else_` is `Term::Var { name = "__unreachable__" }`. -- `examples/lit_pat.ailx` + `examples/lit_pat.ail.json`: the - trailing `(case _ 0)` workaround on `categorize_first` is - removed. The remaining match (`Nil` arm + `Cons (pat-lit 0) - _` arm + `Cons h _` arm) is exhaustive on `IntList`; the - chain default is unreached. Header comment rewritten to - describe 16d's role. Output unchanged: `100, 200, 999, -1, - 0, 7`. -- `examples/unreachable_demo.ailx` + `.ail.json`: new fixture. - `safe_div(a, b)` returns `a / b` when `b != 0` and panics via - `__unreachable__` otherwise. Driver uses non-zero divisors - only, so the panic branch is never executed. Output: `4, 5`. - Exercises the typechecker's `forall a. a` instantiation - against `Int` and codegen's `unreachable` emission inside an - `if`'s then-branch. -- `crates/ail/tests/e2e.rs` (+18): `unreachable_demo` test; - `lit_pat_demo`'s doc updated to mention the 16d simplification. -- `docs/DESIGN.md`: new "Builtins" bullet under "What is - supported", listing every builtin (operators, `not`, IO ops) - and calling out `__unreachable__ : forall a. a` with its UB - semantics. - -**Hash determinism.** Of the 36 fixture defs across all -`examples/*.ail.json`, exactly one hash changed: the -`categorize_first` def of `lit_pat` (`c4faec3abc2ed388` → -`644de0c0ec15fc17`), because that's the only def whose source -text changed. All other defs — including `lit_pat`'s other -three (`IntList`, `classify`, `main`) — are bit-identical to -their pre-16d forms. The new `unreachable_demo` fixture has -two new hashes (`safe_div`, `main`) which obviously didn't -exist before. Verified by stash-diff of `ail manifest` output -on `sum`, `list`, `maybe_int`, and `lit_pat`. - -**Other fixtures that benefited.** Just `lit_pat` — -`categorize_first` was the only place in the shipped corpus -where a `(case _ ...)` arm existed solely to dominate the -chain terminator. The `(case _ 0)` arms in `nested_pat` and -`std_either_list` are semantically required (they handle -`Nil` and partial-coverage cases that the preceding nested -ctor patterns don't reach), not workarounds; they stay. -`std_list::take` / `drop` use a different workaround (base- -case-via-arm-body / `if (== n 0) Nil ...`) which 16c-aux -addresses, not 16d. - -**Tests: 122 → 124 (+2).** - -- e2e: 45 → 46 (`unreachable_demo`). -- `ailang-core::desugar::tests`: 25 → 26 - (`chain_default_is_unreachable_builtin`). -- All other crates unchanged. - -**Cumulative state, post-16d.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term`/`Pattern`/`Literal` enums unchanged. The new builtin - is purely a name-level addition (`env.globals` + a codegen - lowering branch); no AST shape changed and no schema bump. -- 16a desugar pass now does four jobs: nested-ctor flattening - (16a), LetRec lift (16b.1–16b.7), lit-pattern → If rewrite - (16c), and `__unreachable__` chain default (16d). Pass - remains the single AST-→-AST hop between `load_module` and - typecheck. -- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5 - (unchanged). - -**Queue update post-16d.** 16d done. Open: `closure-of-self` -(informally 16b.5-body — unchanged); `closure-poly` -(informally 16b.5b — unchanged); 16e (`==` extension to -Bool/Str/Unit — surfaced by 16c, unchanged); 17a (per-fn -arena, gated — unchanged); 16c-aux (`std_list::take`/`drop` -refactor onto lit patterns — unchanged). - -## Iter 16e — `==` extends to Bool/Str/Unit (polymorphic dispatch) - -**Goal.** Lift the last gate the 16c entry left open: `==` was -declared `(Int, Int) -> Bool`, so 16c's `build_eq` rewrite of a -non-Int `Pattern::Lit` (`(pat-lit "hi")`, `(pat-lit true)`, etc.) -desugared to a well-formed AST but failed at typecheck because -the `==` call's arg types could not unify with `Int`. 16e -extends `==` to a polymorphic operator usable on the four -language-level scalar types — `Int`, `Bool`, `Str`, `Unit` — -and cleans the gap left by 16c so every `Literal` kind that -the AST ships is now usable in a `(pat-lit ...)` position. - -**Pre-16e state.** From the 16d journal: `==` lived as -`("==", "(Int, Int) -> Bool")` in `ailang-check::builtins` (line -~140). 16c's `build_eq` already produced `(app == lhs rhs)` for -every non-Unit literal kind; for `Unit` it short-circuited to -`Term::Lit { Bool { true } }` (every `()` is equal). Without -16e, the codegen rejection of non-Int `==` was preceded by a -typecheck rejection: `forall a. (a, a) -> Bool` was simply not -the declared type, so unification of `Bool` against `Int` (via -`int_int_bool`) blew up first. Iter 16d's `categorize_first` -fixture sidestepped the issue by using only Int lit patterns; -no shipped fixture used a `Bool`/`Str`/`Unit` lit pattern. - -**Architectural decision (binding, by orchestrator).** - -- `==`'s declared type becomes - `Forall(["a"], Fn([Var("a"), Var("a")], Bool, []))` — - `forall a. (a, a) -> Bool`. The polymorphic-builtin pattern - mirrors `__unreachable__` from 16d (which is a `Forall`-typed - value rather than a `Forall`-typed fn, but lives in the same - `env.globals` slot and the same `builtin_ail_type` mirror in - codegen). User-facing surface stays one symbol; the existing - `(app == ...)` shape carries every supported type. -- Codegen monomorphises like any polymorphic builtin and - dispatches on the resolved AIL arg type at the call site. - The dispatch table (one row per supported type): - - | AIL type | LLVM lowering | - |----------|------------------------------------------------| - | `Int` | `icmp eq i64` | - | `Bool` | `icmp eq i1` | - | `Str` | `call i32 @strcmp(ptr, ptr)` then `icmp eq i32 0` | - | `Unit` | constant `i1 true` (single-inhabitant) | - | ADT/Fn | rejected with `CodegenError::Internal` | - -- Unit's lowering still evaluates both operands above the - comparison (preserving any side effects), then ignores the - resulting SSA values and returns the constant `true`. This - matches the standard pattern (eval-both-then-fold) and keeps - `Unit` `==` semantics-preserving in the presence of an - effectful sub-expression (none of the shipped fixtures - exercise that, but the invariant is documented in - `lower_eq`'s rustdoc). -- ADT and `Fn` arg types fall to a clear codegen error message - (`==` not supported for type X / function types`). Two design - reasons: (a) ADT structural equality requires either a derived - per-type equality fn or runtime field-by-field walk — neither - is in 16e's scope; (b) `Fn`-pointer equality on a closure pair - is meaningless without a normalisation pass (two thunks may - represent the same lambda). Either is its own iter, queued - outside this one. The negative path is gated by two new - codegen unit tests (`eq_on_adt_rejected_at_codegen`, - `eq_on_fn_rejected_at_codegen`). - -**Why polymorphic `==` over per-type `==int`/`==bool`/`==str`.** -Mirrors `__unreachable__`'s precedent (one polymorphic name in -`env.globals`, instantiated by the typechecker at every use -site). Three concrete benefits over the alternative: -(i) `build_eq` from 16c stays as-is — it already produces a -single `(app == lhs rhs)` term, type-driven; (ii) authoring -surface stays one symbol, no `==str` lookup table for the LLM -to memorise; (iii) the existing `Forall` instantiation -machinery (`maybe_instantiate` in `ailang-check`) does the -work — no new typechecker code path. - -**The `@strcmp` extern.** `Str` `==` lowers to libc's -`strcmp(ptr, ptr) -> i32`. Strings are NUL-terminated literals -in the AILang ABI (see `io/print_str` lowering, which calls -`@puts`), so `strcmp` is a one-liner. Declared in the LLVM IR -header next to `@printf` / `@puts` / `@GC_malloc`. No extra -clang link flag needed — `strcmp` is in libc. - -**What shipped.** - -- `crates/ailang-check/src/builtins.rs` (+22, of which ~10 - doc): `==` is hoisted out of the `int_int_bool` shape branch - into a dedicated `Forall` registration; the `list()` row - becomes `("==", "forall a. (a, a) -> Bool")` (consumed by the - `ail builtins` CLI subcommand). Other comparison ops (`<`, - `<=`, `>`, `>=`, `!=`) keep their Int-only registration - unchanged — extending those is queued separately if ever - needed (`!=` would be a one-line change after this iter). -- `crates/ailang-codegen/src/lib.rs` (+~110, of which ~50 - doc): (a) IR header gains `declare i32 @strcmp(ptr, ptr)`; - (b) `lower_app` short-circuits `name == "=="` before the - `builtin_binop` block, calling a new `lower_eq` helper that - takes the resolved `Type` of the first arg (via - `synth_arg_type`) and dispatches by `Type::Con.name`; - (c) `builtin_ail_type` mirror gets the `Forall` form for - `==` so the mono pipeline's arg-type inference still - resolves the symbol. The old `("==", "icmp eq", "i1")` row - in `builtin_binop` stays — `is_static_callee` queries the - table to decide whether `==` is a direct callee, and the - early-return in `lower_app` ensures the `i64`-emission code - is never reached. -- `examples/eq_demo.ailx` + `examples/eq_demo.ail.json`: new - fixture exercising `==` at all four supported types directly - via `(app == ...)` and indirectly via 16c's lit-pattern - desugar over a `Str` scrutinee. Drives `io/print_bool` for - the boolean results and `io/print_int` for the `classify_str` - fn (a 3-arm `(match s ... (case (pat-lit "hi") 1) ...)`). - Output (one per line): `true`, `false` (Int); - `false`, `true` (Bool); `true`, `false` (Str); `true` - (Unit); `1`, `2`, `0` (`classify_str` over `"hi"`/`"ho"`/ - `"??"`). The Str lit-pattern arm is the smoking gun for - 16c+16e working together — pre-16e, that `(pat-lit "hi")` - desugared to `(if (== sv "hi") 1 fall_k)` and the `==` call - failed to typecheck. -- `crates/ail/tests/e2e.rs`: new `eq_demo` test (+25 incl. - doc), e2e count 46 → 47. -- `crates/ailang-check/src/lib.rs` (+~85 incl. doc): five new - unit tests in the `tests` module — `eq_typechecks_at_int` - (regression), `eq_typechecks_at_bool`, `eq_typechecks_at_str`, - `eq_typechecks_at_unit` (positive), and - `eq_rejects_mixed_int_bool` (negative — confirms the rigid - var still demands the two sides agree). -- `crates/ailang-codegen/src/lib.rs` tests: two new unit tests - (`eq_on_adt_rejected_at_codegen`, - `eq_on_fn_rejected_at_codegen`) that drive `emit_ir` past - typecheck and assert a clear error message mentions `==` and - the offending type kind. ADT case constructs a tiny `data K = - Mk` and compares two `Mk` ctors; Fn case binds `f = main` and - compares the fn-value with itself. -- `crates/ail/tests/snapshots/*.ll`: the IR header in every - snapshot now contains `declare i32 @strcmp(ptr, ptr)`; the - five existing snapshots (`hello`, `list`, `max3`, `sum`, - `ws_main`) were refreshed via `UPDATE_SNAPSHOTS=1`. Body of - every `define` block is byte-identical to pre-16e — the - header is the only diff. -- `docs/DESIGN.md`: Builtins bullet rewritten to call out the - polymorphic `==` and the dispatch table; the lit-pattern - bullet updated to drop the "today that means Int" caveat; - the "Recently lifted gates" preamble extended. - -**What deliberately did NOT change.** - -- The other comparison ops (`<`, `<=`, `>`, `>=`, `!=`) stay - Int-only. `!=` could be made polymorphic by the same recipe - (one row in `builtins.rs`, one branch in `lower_app`), but - staying in scope for this iter — queued. -- Codegen `unreachable!` arms for `Term::LetRec` STAY - (architectural rule from the iter brief, unchanged from - 16b.x). -- ADT structural equality and `Fn`-pointer equality stay - rejected at codegen. Either could be lifted later, but - needs its own design. -- `==` at the LetRec / closure-pair / lambda boundary is - unchanged: those still produce `Fn` arg types and now hit - the codegen-level rejection with a clearer error message - than the pre-16e "wrong LLVM type for icmp" garble. - -**Hash determinism.** The ten fixtures listed in the iter -brief (`local_rec_demo`, `lit_pat`, `local_rec_capture`, -`local_rec_let_capture`, `local_rec_match_capture`, -`local_rec_as_value`, `local_rec_as_value_capture`, -`poly_rec_capture`, `nested_let_rec`, `unreachable_demo`) -all produce identical `ail manifest` output to their pre-16e -forms — verified via direct manifest dump. No fixture's -canonical JSON changed; no def hash changed. The new -`eq_demo` fixture introduces two new hashes (`classify_str`, -`main`) that obviously did not exist before. - -**Tests: 125 → 133 (+8).** - -- e2e: 46 → 47 (`eq_demo`). -- `ailang-check::tests`: 29 → 34 (five new `eq_*` tests). -- `ailang-codegen::tests`: 2 → 4 (two negative-path tests). -- All other crates unchanged. -- IR snapshot tests: 5 → 5 (unchanged count; bytes refreshed - for the new `@strcmp` header line). - -**Cumulative state, post-16e.** - -- Stdlib unchanged (5 modules, 29 combinators). -- `Term`/`Pattern`/`Literal` enums unchanged. The change is - entirely a builtin-registration shift (`Fn` → `Forall` - in `env.globals`) plus a codegen dispatch site — no AST - shape changed and no schema bump. -- 16a desugar pass keeps its four-job role from 16d. 16e's - contribution is a pre-existing desugar output (`(app == - s_var lit)` from 16c's `build_eq`) suddenly being typeable - for non-Int literals. -- Compiler bugs surfaced and fixed in dogfood since 14a: 5/5 - (unchanged). - -**Queue update post-16e.** 16e done. Open: `closure-of-self` -(informally 16b.5-body — unchanged); `closure-poly` -(informally 16b.5b — unchanged); 17a (per-fn arena, gated on -user GC discussion — unchanged); 16c-aux (`std_list::take`/ -`drop` refactor onto lit patterns — unchanged); a low-priority -follow-up that mirrors 16e for `!=` (one-line change in -`builtins.rs` plus a one-line dispatch arm in `lower_app`, -queued without a number). - -## Iter 17a — per-fn arena via stack alloca for non-escaping allocations - -**Goal.** Replace `@GC_malloc` with LLVM `alloca` for ADT/closure -allocations the compiler can prove do not escape their allocating -fn. Boehm GC stays linked and unchanged; this is purely an -optimisation layered on top of Decision 9's collector floor. - -**Architectural choice: alloca, not heap arena.** Stack -allocation matches "freed at fn return" exactly — no malloc/free -pair, no per-fn bump-allocator runtime, no recycling logic. LLVM -already optimises `alloca i8, i64 N` (mem2reg / SROA can promote -small allocas to registers when uses are simple). The simpler -mechanism wins; a heap-arena path would have meaningful -implementation cost with no obvious benefit at MVP scale. - -**Escape-analysis approach (conservative, name-based taint -propagation).** A new `crates/ailang-codegen/src/escape.rs` -walks each fn body once. For every `Let { name = X, value = -Term::Ctor | Term::Lam, body = B }` it asks: does any value -derived from X flow past the fn frame? - -Taint propagation: -- The bound name `X` is tainted in `B`. -- A `Term::Match` whose scrutinee is a `Var` referring to a - tainted name propagates taint to every pattern-bound name in - every arm. (Pattern bindings hold field projections of the - scrutinee, which live inside the same allocation.) -- A `Term::Let { name = Y, value = Var(t), body }` where `t` - is tainted makes `Y` tainted in the let's body. - -A tainted name escapes if it appears in: -- Tail position of `B` (the value of `B` is the value of the - outer region). -- The arg list of any `Term::App` / `Term::Do`. -- The field list of any `Term::Ctor`. -- The free-var capture set of any `Term::Lam`. - -Allowed (non-escaping) positions: -- Scrutinee of `Term::Match` (read-then-projected; the scrutinee - itself doesn't leak unless an arm leaks a pattern binding, - handled by the taint propagation above). -- The callee of `Term::App` when the callee is a bare Var to the - tainted name (calling locally just loads the closure pair via - GEP — the pointer isn't stored anywhere). - -Pessimism intentionally accepted: the analysis is not -flow-sensitive within an arm, not field-sensitive, not -inter-procedural. A pessimistic answer ("escapes" when it -doesn't) only loses optimisation opportunities, never -correctness. Sample of what's flagged escaping that maybe -shouldn't be: a let-bound `Pair(Int, Int)` where one field -projection is returned in tail position — the whole box flows -through the projection's taint, even though the Int value -itself doesn't share lifetime with the box. The journal section -"Observations" lists more examples. - -The Lam body is itself a fn frame for the analyzer's purposes — -each lifted thunk runs its own analysis when `lower_lambda` -emits its body. Escape-analysis state is saved/restored across -the thunk's emission alongside the rest of the per-fn emitter -state. - -**Codegen change.** Three allocation sites in -`crates/ailang-codegen/src/lib.rs` now branch on the per-fn -`non_escape: BTreeSet` (raw pointer addresses of -`Term::Ctor` / `Term::Lam` AST nodes flagged non-escaping): -1. `lower_ctor` — ADT box. -2. `lower_lambda` env block (when `cap_meta.len() > 0`). -3. `lower_lambda` closure pair (always 16 bytes). - -A hit emits ` = alloca i8, i64 , align 8`; a miss -emits ` = call ptr @GC_malloc(i64 )`. Tag stores, -field stores, and closure-pair packing are unchanged. The -closure-pair and its env share a single escape verdict — they -have parallel lifetimes; if the closure pair is non-escaping, -the env is too. - -The escape-analysis pass runs at the start of `emit_fn` (over -the fn body) and at the start of every lambda thunk emission -(over the thunk body). Cost: one tree walk per fn, O(node count). -Negligible compared to lower_term itself. - -**IR diff samples.** Across all 20 shipped `examples/*.ail.json` -fixtures (excluding the new Iter 17a fixture), the analysis -flagged **0 of 270 ctor / lambda allocations** as non-escaping. -This is structurally expected: typical AILang code passes -freshly-built ctors directly into another fn ("LLM-style" -threading of values), so the let-binding shape required by the -rule rarely appears, and when it does the let-body almost -always passes the value to a fn (immediate escape). Concrete -counts per fixture (alloca / `@GC_malloc`): -- `box.ail.json`: 0 / 1 -- `closure.ail.json`: 0 / 2 -- `gc_stress.ail.json`: 0 / 2 -- `list.ail.json`: 0 / 4 -- `list_map.ail.json`: 0 / 7 -- `list_map_poly.ail.json`: 0 / 6 -- `lit_pat.ail.json`: 0 / 5 -- `local_rec_*` (8 fixtures combined): 0 / 13 -- `maybe_int.ail.json`: 0 / 2 -- `nested_pat.ail.json`: 0 / 4 -- `sort.ail.json`: 0 / 18 -- `std_either_demo.ail.json`: 0 / 9 -- `std_either_list_demo.ail.json`: 0 / 64 -- `std_list_demo.ail.json`: 0 / 86 -- `std_list_more_demo.ail.json`: 0 / 41 -- `std_list_stress.ail.json`: 0 / 2 -- `std_maybe_demo.ail.json`: 0 / 7 -- `std_pair_demo.ail.json`: 0 / 9 - -The new Iter 17a fixture `examples/escape_local_demo.ail.json` -intentionally demonstrates the optimisation: 2 alloca / 0 -`@GC_malloc`. Both `peek` and `count` build a `Box(_)` -let-bound, scrutinise it with a wildcard pattern (no pattern -binding flows out), and return a literal Int derived from -neither the Box nor its payload. Each call to `count(N)` -recursively builds a fresh stack-allocated Box per frame; with -the optimisation, a million-deep recursion would not heap- -allocate a single byte for those Boxes (only the recursion's -stack frames themselves grow). Without the optimisation -(pre-17a), each `count` call would heap-allocate a Box, all of -which would live until Boehm's next sweep. - -**IR snapshot diffs.** The five existing IR snapshots -(`hello`, `sum`, `list`, `max3`, `ws_main`) are byte-identical -to pre-17a — none of those fixtures has a let-bound non- -escaping ctor allocation. Snapshot tests pass without refresh. - -**Tests: 133 → 141 (+8).** -- e2e: 47 → 48 (`iter17a_local_box_alloca`). -- `ailang-codegen::tests`: 4 → 11 (+7 new escape-analysis unit - tests in `escape::tests`: `local_ctor_match_only`, - `returned_ctor_escapes`, `ctor_passed_as_arg_escapes`, - `ctor_stored_in_ctor_field_escapes`, - `pattern_binding_returned_escapes`, `local_lam_call_only`, - `lam_passed_as_arg_escapes`). -- All other test counts unchanged. -- IR snapshots: 5 → 5, no refresh needed (no fixture's IR - changed). - -**Files touched.** -- `crates/ailang-codegen/src/escape.rs` (new, ~430 LOC incl. - doc and tests). -- `crates/ailang-codegen/src/lib.rs`: module declaration; new - `non_escape: NonEscapeSet` field on `Emitter`; analyse at - start of `emit_fn`; save/restore + re-analyse around lambda - thunk emission; new `term_ptr` parameter on `lower_ctor` and - `lower_lambda`; alloca-vs-GC_malloc branch at three sites - (ctor box, lambda env, closure pair). -- `crates/ail/tests/e2e.rs`: new test - `iter17a_local_box_alloca` (asserts stdout + IR contains - `alloca` / no `@GC_malloc` in the two demo fns). -- `examples/escape_local_demo.ailx` and `.ail.json`: new - fixture. -- `docs/DESIGN.md`: new "Per-fn arena via stack `alloca` - (Iter 17a)" subsection inside Decision 9; "Recently lifted - gates" preamble extended. - -**Hash invariance verified.** No existing fixture's `.ail.json` -was touched; no AST shape changed; no schema bumped. Every -shipped fixture's def hashes are unchanged. The new -`escape_local_demo` fixture introduces three new hashes -(`peek`, `count`, `main`). - -**Output bit-equality verified.** Manual smoke run of every -existing fixture: `sort` prints sorted list, `list_map_poly` -prints `2 3 4`, `gc_stress` prints `1275`, `std_list_demo` -prints the documented `5/false/true/1/4/10/5/2/2/15/15`, -etc. — all byte-identical to pre-17a stdout. The optimisation -is semantically transparent. - -**Observations for the GC discussion.** -- **Vanishingly few non-escaping allocations in shipped code.** - 0 / 270 alloca conversions across 20 existing fixtures. The - "build-locally, consume-locally" pattern (let X = Ctor in - match X of ... -> non-X) is not how AILang code is currently - written. Most fixtures thread freshly-built ctors directly - into another fn (immediate escape via the App-arg rule). -- **Most ctors are not let-bound at all.** They appear as - inline ctor field values (`Cons h (Cons (...) (...))`), as - the value of a fn return, or as direct args to a fn call. - An escape-analysis rule restricted to let-bound allocations - cannot catch these. A more aggressive rule that gives a - "name" to inline allocations and tracks their flow could - catch some of these — but at MVP scale most ctor data - honestly is shared between fns (linked-list spines, etc.), - so the precision gain may be small. -- **Polymorphic patterns make the escape-analysis question - harder.** `std_list.length`'s body is `fold_left (\c _. c+1) - 0 xs`. The lambda `\c _. c+1` is passed as App arg → - escapes. But it has no captures — the env block is empty - (null). Lifting this to a top-level fn (which the codegen - already does for top-level fns via the static closure-pair - global) would mean zero heap allocation for that lambda. A - "lift no-capture lambdas in arg position" optimisation is - adjacent to escape analysis but separate, and probably has - better hit rate than the current rule. -- **Pattern-binding taint is too pessimistic for product - types.** A `let p = Pair(x, y) in match p of (a, b) -> a + - b` is currently flagged escaping (because `a` and `b` are - tainted, both flow to tail of `+`). The Int values do not - share lifetime with the `Pair` box; once they're projected - to register they're free of the box. A field-sensitive - analysis would catch this. Refactor potential: the existing - `std_pair_demo` exercises exactly this shape repeatedly. -- **Ctor allocations as ctor fields look like escapes but are - recursive: the inner Cons in `Cons h (Cons t Nil)` could in - principle alloca alongside the outer Cons if the outer is - itself non-escaping (parallel lifetimes). The current rule - doesn't see this because only let-bound allocations are - candidates; an inline allocation in field position is never - considered. -- **Closures rarely qualify in real code.** Lambdas almost - always escape via being passed to a HOF (`map`, `fold_*`, - `filter`). The let-bound closure called only locally (e.g., - `let f = \x. body in f(arg)`) is the unit-test shape, not - the production shape. The closure-pair is currently the - most expensive single allocation per HOF use site (16 bytes - for the pair plus N×8 for the env). The all-or-nothing - GC-vs-alloca question is the wrong shape for closures — - many of them have very short, predictable lifetimes (HOF - arg → callee invokes → return) but cross enough fn frames - that pure stack alloca isn't sound. -- **Real-world wins concentrate in fns that locally - destructure intermediate ADTs.** The Iter 17a fixture - (`escape_local_demo`) is the canonical shape: build a - temporary box, look inside it, return something derived - from neither. This shape exists in real code (sentinel - values, scratch wrappers for type juggling) but is rare in - the current AILang stdlib because the stdlib is mostly - list/option combinators that propagate their input. -- **Tail-call interaction is benign.** The `musttail call` - shape from Iter 14e is unaffected: alloca'd memory is - freed at fn return, but `musttail` requires that no live - alloca's address escape into the call. The escape analysis - already rules out tainted values flowing into App args - (which is what tail-call args are), so any allocation that - reaches alloca cannot have its pointer used as a tail-call - arg. No regression. -- **Boehm conservative scan benefits from the shrinkage.** - Every alloca is one fewer GC root for the conservative - scan to potentially false-positive on. At small scale the - heap is small enough that this doesn't matter; at larger - scale the over-retention rate of conservative GC drops as - the heap shrinks, so even a small alloca conversion rate - improves collector precision. Hard to quantify without - Boehm-internal stats. - -**Anything noticed but did NOT touch (per scope discipline).** -- The "lift no-capture lambdas in arg position" optimisation - (turn `\c _. c+1` passed to `fold_left` into a static - closure-pair, no allocation at all). Adjacent and probably - higher hit rate; out of scope here. -- A field-sensitive escape analysis to recover product-type - precision (`Pair`, `Box`-of-scalar). Would require - per-field taint and projection tracking; out of scope. -- An aggressive analysis that names inline allocations and - tracks their flow through ctor fields. Would catch the - `Cons (Cons (...) (...)) ...` recursion case. Out of scope. -- ADT structural equality (still rejected at codegen). 16e - punted; Iter 17a does not change that. -- The `!=` polymorphism mirror (one-line change). Still - queued without a number; not picked up here. - -**Test count delta.** Workspace: 133 → 141 (+8). All green. - -**Queue update post-17a.** 17a closes. Per the iter brief, -**post-17a is gated on a user GC discussion — do not pick up -further iters until that conversation happens.** Open queue -items remaining (untouched): `closure-of-self` (informally -16b.5-body); `closure-poly` (informally 16b.5b); `16c-aux` -(`std_list::take`/`drop` refactor onto lit patterns); the -low-priority `!=` mirror of 16e. Adjacent ideas surfaced in -the Observations section above are explicitly NOT queued — -they require user sign-off on whether the project's GC -direction is "improve precision of stack-alloca", "replace -Boehm with a precise collector", or something else entirely. - -## Bench — GC overhead via bump-allocator comparison - -Single-purpose data-gathering iter, not a feature. Goal: quantify -how much of the runtime spent by AILang programs is paid to the -Boehm conservative collector by comparing the same program built -two ways — `--alloc=gc` (default, current behavior, links `-lgc`) -and `--alloc=bump` (a no-free 256 MB statically-allocated bump -arena from `runtime/bump.c`). The IR text for the two builds is -byte-identical except that every `@GC_malloc` callsite and the -`declare ptr @GC_malloc(i64)` declaration become `@bump_malloc`. -The link command swaps `-lgc` for `runtime/bump.o`. Nothing else -changes. - -### Methodology - -Two fixtures, both designed to drive heap allocation hard enough -that the collector / arena is on the hot path: - -- **`examples/bench_list_sum.ail.json`**. Local `IntList` ADT. - Builds three lists (lengths 100k / 1M / 3M) by tail-recursive - `cons_n_acc`, sums each via tail-recursive `sum_acc`, prints - the three sums. Both build and sum are written in accumulator - form with `tail-app` because at 3M elements a non-tail - recursion overflows the 8 MB system stack. Each `ICons` cell is - 24 B; total heap traffic ≈ 99 MB across the run (the bump - arena's 256 MB ceiling was the constraint that capped the - largest size at 3M, not 10M). -- **`examples/bench_tree_walk.ail.json`**. Local `Tree` ADT - (`Leaf | Node Int Tree Tree`). Builds and sums balanced trees - of depth 16 / 18 / 20. At depth 20 the tree has 2^20 − 1 nodes, - ~32 B per `Node`, ~64 MB heap traffic for the depth-20 phase - alone. Recursion in `build_tree` / `sum_tree` is constructor- - blocked so it cannot be `tail-app`'d, but the recursion depth - equals the tree depth (≤ 20), so it fits trivially. - -Build configuration: `clang -O2`, both modes. The harness -(`bench/run.sh`) runs each binary 5 times under a Python wrapper -that reads `getrusage(RUSAGE_CHILDREN).ru_maxrss` for peak RSS -and `time.monotonic()` deltas around `subprocess.Popen.wait` for -wall time. Slowest run is dropped, median wall over the kept 4 is -reported. The harness runs `cargo build --release -p ail` first, -then compiles each `(fixture, mode)` pair once before the timing -loop, so build time is excluded from measurements. - -### Numbers (Linux 7.0.3-1-cachyos, single machine, RUNS=5) - -``` -workload | gc median(s) | bump median(s) | overhead % | gc max RSS(KB) | bump max RSS(KB) ------------------------+--------------+--------------+--------------+----------------+---------------- -bench_list_sum | 0.145 | 0.050 | 190.0 | 103788 | 97640 -bench_tree_walk | 0.105 | 0.038 | 176.3 | 73452 | 55452 -``` - -A second run with RUNS=9 (median of 8) corroborates within noise: - -``` -bench_list_sum | 0.141 | 0.048 | 193.7 | 103784 | 97884 -bench_tree_walk | 0.103 | 0.039 | 164.1 | 73448 | 55396 -``` - -Overhead = `(gc - bump) / bump * 100` — i.e. the GC-mode runtime is -~2.7–2.9× the bump-mode runtime. Equivalently, ~63–65 % of the -GC-mode wall time is GC overhead (collector pauses + write -barriers + allocation-path complexity vs. a single bump pointer). - -### Bucket - -**Large.** GC takes roughly two-thirds of total runtime on these -allocation-heavy workloads. For comparison, the typical Boehm -conservative-GC overhead reported in the literature on -allocation-heavy workloads sits in the 20–60 % range; ~190 % puts -this firmly past that envelope. Caveat below. - -### Caveats - -- **Single-machine measurement.** No cross-machine confirmation, - no isolation from background load. Variance across the kept-4 - runs was ≤ 5 ms in absolute terms, but a bigger machine / - smaller machine / different libgc version could shift these - numbers materially. -- **Allocation-heavy workloads.** Both fixtures spend almost their - entire runtime in the allocator (Cons cell construction, Node - cell construction). Real programs that compute as well as - allocate would have a smaller GC-overhead share. The numbers - here are therefore an *upper bound* on the GC's share of any - realistic workload. -- **Bump leaks everything.** The bump-mode binary never frees a - byte; max RSS reflects the working-set after every allocation - the program ever made, plus committed pages from the 256 MB - arena. For `bench_list_sum`'s 99 MB heap traffic, GC's heap - (~100 MB RSS) is essentially identical to bump's (~97 MB). - Where the workload actually leaks past bump's arena (~256 MB - cells × any factor), GC would win on RSS by reusing freed - memory; this bench does not exhibit that regime. -- **No warmup theatrics.** Each timed run is a cold process start. - AILang has no JIT and no per-process allocation-path tuning, - so first-run / steady-state distinction does not apply here. - Variance was within noise even on the first kept run. -- **Hardcoded N.** No env-var / argv plumbing in AILang yet, so - the workload sizes are baked into the source. The three sizes - per fixture provide enough variety to detect a wildly - size-dependent overhead (none observed — both fixtures show a - flat ~2.8x ratio across all three calls). -- **The bench measures `GC_malloc` overhead, not full GC.** Boehm's - collector runs inline on allocation when the heap grows past a - threshold; we never observe it as a separate cost. A program - with a long-lived heap that causes repeated full marks would - see a different (likely larger) overhead share. Neither fixture - here triggers that. - -### Implementation summary - -- `crates/ailang-codegen/src/lib.rs`: new public `AllocStrategy` - enum (`Gc` / `Bump`); new public entry `lower_workspace_with_alloc`; - `lower_workspace` delegates to it with `Gc`. The single - declaration line and the three `@GC_malloc` callsites - (`lower_ctor`, lambda env, closure pair) all read the - Emitter's `alloc` field. -- `crates/ail/src/main.rs`: `--alloc=` flag added to - both `build` and `run`, default `gc`. Threaded into a now-four- - arg `build_to`; on `Bump`, the helper `locate_bump_runtime()` - walks up from the binary path / cwd to find `runtime/bump.c`, - compiles it inline (`clang -O2 -c`) into a tempdir-scoped - `bump.o`, and links that instead of `-lgc`. -- `runtime/bump.c`: 256 MB static arena, single bump pointer, - 8-byte alignment, `abort()` on overflow. Single function - `void *bump_malloc(size_t)`. -- `examples/bench_list_sum.{ailx,ail.json}` and - `examples/bench_tree_walk.{ailx,ail.json}`: the two fixtures - described above. List builder rewritten to accumulator form - to fit in 8 MB stack at 3M elements. -- `bench/run.sh`: harness as specified. Python helper for - monotonic clock + RUSAGE_CHILDREN max RSS (avoids the - `/usr/bin/time` dependency, which is not on Arch by default). - `awk` replaces `bc` for the same reason. -- `docs/DESIGN.md` not touched. The CLI flag is opt-in, the - default behavior is identical to pre-bench, and the bump path - is bench-only — it does not deserve language-spec status. - -### Cross-iter regression verified - -- Default `--alloc=gc` is byte-identical to pre-bench. The five - IR snapshots (`hello`, `sum`, `list`, `max3`, `ws_main`) pass - unchanged. All workspace tests pass: 141 → 141 (no test - count delta from this iter; no e2e additions). -- Manual smoke run of representative existing fixtures - (`sum`, `list`, `list_map`, `gc_stress`, `std_list_demo`, - `escape_local_demo`) under `--alloc=gc` produces identical - stdout to the documented expected outputs. -- The bump-mode IR, after a textual `s/GC_malloc/bump_malloc/g` - on the gc-mode IR, is `diff`-clean against a real `--alloc=bump` - build. The IR is byte-identical except for the allocator - symbol name. - -### Did anything surprise - -- **The overhead is large.** ~2.8x slowdown is at the high end of - what one expects for a modern conservative collector on - allocation-heavy code. Two factors likely contributing: (a) - Boehm's `GC_malloc` does conservative root scanning of the - C stack on every collection — for workloads that allocate - heavily, the collector triggers often; (b) AILang's escape - analysis (Iter 17a) flags 0 of 270 ctor sites in shipped code - as non-escaping, and 0 of the allocations in either bench - fixture, so the entire allocation traffic goes through the - collector. Workloads that converted more allocations to - `alloca` would see a smaller GC share. -- **No segfault from the bump leak.** The bump arena is - 256 MB; the heaviest workload (3M-element list) consumes - ~99 MB. We have headroom even at the largest configured size. - 10M elements (the original spec value) would have been - 240 MB — uncomfortably close to the ceiling, justified the - reduction to 3M. -- **GC's max RSS is barely larger than bump's.** I expected GC's - max RSS to be substantially smaller than bump's (because GC - reclaims dead memory). It isn't — the bump fixtures' working - sets are simply not large enough to pressure the collector - into reclaiming much. The list fixture builds the entire - 3M-element list before summing, so all allocations are live - at once anyway. Different workloads (e.g. a fold that builds - intermediate lists discarded between iterations) would surface - the RSS gap. -- **Tail-call discipline matters.** The original spec's "tail- - recursive sum" is a misnomer for `sum_list (Cons h t) = h + - sum_list t` — that's constructor-blocked, not tail-recursive. - Naïvely transcribing the spec produced a binary that - segfaulted at 3M elements. Both fixtures' linear-recursion fns - had to be rewritten in accumulator form with explicit - `tail-app` markers. Captured here because it is a real - consequence of how AILang is structured: an LLM author who - ports a textbook recursive sum into AILang at scale will - hit the stack ceiling unless they know about Decision 8. - -## 2026-05-08 — GC discussion: commit to RC + Uniqueness - -The bench numbers (~60% Boehm share, all of it in the allocate -path) precipitated the memory-management conversation that has -been gated since Iter 17a. The user reframed the question -sharply: tracing GC has irreducible variability that no amount -of tuning eliminates; RC must be committed to **now or never**, -because every iter shipped under the wrong model adds debt that -gets more expensive to migrate. With pre-stdlib code volume, -this is the cheapest moment. - -**Decision: AILang's canonical memory model is RC + Uniqueness.** -See `Decision 10` in `docs/DESIGN.md` for the architectural -write-up. Boehm becomes a transitional allocator (Decision 9 is -now annotated as superseded). The migration runs as Iters -18a–18d; the default flips when RC is within 1.3× of the bump -floor on `bench/run.sh`. - -**Why RC and not (a) keep Boehm, (b) build a precise tracing GC, -(c) linear / ownership types:** - -- **Keep Boehm.** Bench data shows the cost is structural in the - allocate path. Tuning Boehm cannot change that; the - `GC_malloc` fast path is what it is. Predictable performance - is unobtainable. -- **Precise tracing GC.** Larger investment than RC (read/write - barriers, root maps, generational/copying machinery, multiple - pause budgets). Solves a problem we don't have (cycle handling) - at the cost of one we do (predictable allocate cost). The - AILang language has no cycles by construction; paying for a - cycle-collector backstop is unjustified. -- **Linear / ownership types.** Push the bookkeeping to the - source surface. The author has to mark uniqueness explicitly. - For an LLM-targeted language, that is the worst possible - choice — LLMs are weak at long-range ownership reasoning, and - the value of AILang's "compiler does the bookkeeping" - positioning is exactly that the author doesn't think about it. - -**Why now and not later** (the user's framing, accepted in full): - -- RC and tracing GC produce fundamentally different LLVM IR - (inc/dec instrumentation everywhere vs. periodic safepoints - with root maps). They cannot be hot-swapped per binary; they - are the binary's runtime contract. A code corpus committed to - one cannot be migrated to the other except by recompilation - *and* re-validation. -- AILang has 11 demo fixtures and a small stdlib. The migration - cost is bounded. With 100k LOC it would not be. The cheapest - moment to commit is now. -- The four language-design constraints that make RC sound and - complete (strict / no recursive value bindings / no laziness / - no shared mutable refs) are already true of AILang - incidentally. Decision 10 makes them load-bearing — anything - that breaks them is rejected at design time, not handled by a - retrofit cycle-collector. - -**Lineage.** Lean 4 is the closest precedent (functional, RC + -uniqueness, ML-style type system, LLVM-ish backend, competitive -with OCaml's tracing GC). Roc has the most aggressive uniqueness -optimiser (LLVM, performance-focused dialect of Elm). Koka is -the effect-typed sibling, also with reuse analysis on LLVM. -AILang's profile (acyclic ADTs, strict, threaded ctors between -fns) matches all three; the Lean 4 / Roc shape is the more -direct fit because we share the no-effect-row-default -assumption. - -**What this Decision did NOT do.** - -- Did not start the implementation. Iter 18a (uniqueness - inference pass) is queued as the first concrete step. The - algorithm sketch in Decision 10 is orchestrator-level and - needs implementer-level refinement before it ships. -- Did not retire Boehm. Boehm stays under `--alloc=gc` (default) - through Iter 18d. Iter 18d's bench result triggers retirement; - the flip is mechanical at that point. -- Did not introduce annotations. Uniqueness is fully inferred. - Existing fixtures' AILang source is unchanged; their JSON - hashes remain bit-identical through the entire 18-series. -- Did not commit to atomic refcounts. AILang is single-threaded; - the question is deferred until concurrency primitives arrive - (which would themselves need their own design pass). - -**What changed in the repo.** - -- `docs/DESIGN.md`: Decision 9 header annotated as superseded by - Decision 10. Decision 10 added (~150 lines) covering the - commitment, the four binding constraints, the inference - algorithm sketch, the codegen contract sketch, the reuse - analysis sketch, the migration plan (18a–18d), and the - excluded mutability extensions. -- `docs/JOURNAL.md`: this entry. -- No code change. No fixture change. No test change. - -**Queue.** 16-series and 17a are closed. The 18-series queue is: -18a (uniqueness inference), 18b (codegen inc/dec behind -`--memory=rc`), 18c (reuse analysis), 18d (RC bench + Boehm -retirement decision). 18a is the first concrete iter; the -sketch in Decision 10 is the brief. - -## 2026-05-08 — Follow-up: regions considered, LLM-aware sharpening of RC - -Same day, same conversation thread, after the RC commitment was -written down. The user pumped the brakes: "Warte mal. Woher -wissen wir, dass GC wirklich so viel schneller ist? Und was ist -mit anderen Konzepten?" — and rejected my premature commit of -Decision 10. The conversation that followed widened the design -space, then narrowed back to RC on better grounds, and then -sharpened the RC design with LLM-specific mechanisms. The earlier -section of this entry (the moment of commitment) stays as it -was; this section records the surrounding discussion that -produced the version of Decision 10 currently in `DESIGN.md`. - -### Regions considered and rejected - -The user invoked Rust's borrow-checker as precedent: before -Rust, no one expected memory management to admit a third -non-{tracing-GC, manual-malloc} category. The follow-up question: -is there a similarly-non-obvious choice we are missing for -AILang? RC is the obvious next stop after rejecting GC; is it -also the *consequent* one? - -**Region inference (Tofte/Talpin / MLton-style)** got the most -serious look. The pitch: instead of per-object refcounts, every -value lives in some region; regions stack on entry/exit; when -a region exits, every allocation in it is bulk-freed. No -per-op cost, no cycle worries, and the compiler can synthesise -the regions automatically. - -The user reduced this in two moves. - -First: "ist dann eine region nicht einfach ein expliziter -allocator, der syntaktisch fancy in die Sprache eingebaut ist?" -Concession: yes. Regions = explicit arenas + inference of -which arena to use + a static check that nothing escapes its -arena. The mechanism is bog-standard. - -Second, on the three default cases I'd proposed (D1: fn-local -region, D2: caller-passed region, D3: explicit `letregion`): -"D1 ist exakt der Scope der Funktion. D2 ist der Scope der -aufrufenden Funktion. D3 ist ein malloc. Der Rest ist Zucker. -Oder?" Concession: exactly. Regions are scope-shaped lifetimes -with sugar. - -That reduction surfaced the genuinely consequential question: -**do AILang programs have stack-shaped lifetimes?** If yes, -regions work. If no, regions don't. - -The answer is no. Real programs need: - -- Caches whose lifetime is "until evicted by an LRU policy" — - not nested in any caller's scope. -- Memo tables whose lifetime is "across calls to the same - top-level fn" — also not stack-shaped. -- Lookup structures, registries, deduplication maps — all - similar. - -Regions cannot accommodate these without falling back to -"everything lives in the root region" (= no allocator) or -proliferating per-cache-variant regions (combinatorial). RC -makes no shape assumption; it works for any DAG. - -**Linear / ownership types as primary mechanism.** Briefly -considered. Rejected on the same grounds as in Decision 10: -threading lifetimes through every signature is a tax the LLM -would pay on every line of code, and some programs become -unexpressible without an `unsafe` escape hatch. Rust accepts -that trade-off; AILang doesn't have to. - -**RC + uniqueness wins** because it is the *universal* solution -— no lifetime-shape assumption, no inexpressibility frontier, -just one mechanism that works for any acyclic value graph. The -user closed the excursion: "Schätze wir sind wieder bei RC." - -### LLM-aware sharpening of RC - -With RC chosen, the user asked the question that mattered most: -"An welcher Stelle können wir ausnutzen, dass wir es mit LLMs -zu tun haben?" The mainstream RC implementation (Lean 4) infers -everything from naked AST plus a handful of optional hints; if -the inference is conservative, it falls back to runtime inc/dec. -AILang can do better because the LLM author can effortlessly -produce annotations that a human author would resist as -boilerplate. - -Five concrete mechanisms, all written into the new Decision 10: - -1. **Mandatory `(borrow T)` / `(own T)` on fn signatures.** The - single most consequential extension. Each fn parameter - declares whether it is borrowed (read-only, no inc/dec) or - owned (consumed, callee responsible for free). The compiler - gets a precise contract at every call site instead of a - probabilistic guess. The LLM treats it the same way it treats - the rest of the type — it writes the right annotation as a - matter of course. - -2. **Linear-by-default consumption with explicit `(clone X)`.** - Every binder is consumed by exactly one `own`-mode use. Two - uses → compile error with structured `suggested_rewrites` - (make first call borrow / insert explicit clone / fuse the - traversals). Sharing always costs visible source. - -3. **First-class `(reuse-as SRC NEW-CTOR)`.** Lean 4 / Roc / Koka - discover reuse opportunities by inference; AILang lifts it to - author assertion + compiler verification. The author writes - the hint everywhere it should fire; the compiler bounces it - when the precondition fails. Fewer compiler heuristics, more - author intent visible. - -4. **`(drop-iterative)` on data declarations.** Tells the - compiler to synthesise iterative dec-on-zero traversal - (worklist) instead of recursive cascade. Avoids stack - overflow on deep structures. The author marks types where - this matters. - -5. **Structured compiler diagnostics with `suggested_rewrites` - in form-A AILang.** Errors are JSON; the LLM consumes them - without prose-parsing and applies the rewrites directly. - This is the missing half of the LLM-as-author story. - -The combination: AILang's RC = Lean 4's RC with all the "be -strict" knobs flipped to mandatory because the LLM author can -tolerate them (and benefits from the precision they give the -compiler). - -### What changed - -- `docs/DESIGN.md` Decision 10: rewritten substantially. The - earlier "uniqueness is fully inferred / no annotations / no - schema change" framing is replaced with the LLM-aware - architecture above. New "Why not other memory models" section - records the regions excursion. Schema-additions section - enumerates the new wrappers (`Type::Borrow`, `Type::Own`), - new terms (`Term::Clone`, `Term::ReuseAs`), and new data attr - (`drop_iterative`). Migration plan extended from 4 iters - (18a–d) to 6 (18a–f). - -- Iter queue restructured: 18a is no longer "uniqueness - inference pass" but **"borrow/own annotations as a language - feature"** (schema + parser + typechecker, no codegen). 18b is - the RC runtime + alloc routing. 18c is the inference + naive - inc/dec instrumentation. 18d is reuse. 18e is drop-iterative. - 18f is bench + Boehm retirement. - -- `pre-rc` git tag on commit 65e280b (the bench commit) — marks - the last point at which Boehm was unambiguously the canonical - allocator. - -### Did anything surprise - -- **Three iterations on Decision 10 in one day.** Drafted as - "RC committed, no annotations". Committed prematurely (rejected - by user). Re-opened the design space (regions excursion). - Returned to RC with a sharper architecture (annotations - mandatory). The discipline this enforces: do not commit - decisions until the alternatives have been honestly walked. - -- **Regions were genuinely a candidate, not a strawman.** The - scope-shape reduction is the kind of insight that only surfaces - under adversarial questioning. If I had been left to write - Decision 10 alone, I would have shipped the inferior - inferred-only version because it is what the literature - defaults to. - -- **The LLM-author lever is real.** "Make annotations mandatory" - is a non-starter for human-targeted languages — every Rust - RFC fights about exactly this trade-off. For AILang it is - free: the LLM does not experience boilerplate as a cost. That - is a structural advantage of the target audience, and it is - the kind of advantage Decision 10 should be cashing in on - everywhere it can. - -### Queue (current) - -- **Iter 18a** (queued, in_progress): `(borrow T)` / `(own T)` - as schema + parser + JSON + typechecker. No codegen change. - `(con T)` ≡ `(own T)` for back-compat. New fixture exercises - both modes. -- **Iter 18b** (queued): `runtime/rc.c` (header layout + alloc/ - inc/dec). Codegen `--memory=rc` routes allocation through - `rc_alloc`; no inc/dec yet (deliberately leaks). -- **Iter 18c** (queued): uniqueness inference + naive inc/dec - instrumentation. `(clone X)` schema. Linear-by-default - enforcement turns on. -- **Iter 18d** (queued): reuse hints + reuse analysis. - `(reuse-as ...)`. -- **Iter 18e** (queued): `(drop-iterative)` + worklist free. -- **Iter 18f** (queued): RC bench + Boehm retirement decision. - -## Iter 18a — `(borrow T)` / `(own T)` mode annotations on fn signatures - -First concrete step of the RC migration plan from Decision 10. -Adds the form-A surface - -``` -(fn-type (params (borrow (List Int))) (ret (con Int))) -(fn-type (params (own (List Int))) (ret (own (List Int)))) -``` - -as a *language feature*: schema, parser, JSON, printer, -typechecker passthrough. Deliberately **not** included: codegen -change, linearity enforcement, mode-compatibility unification. - -### Schema choice - -Decision 10 sketched modes as new `Type` variants -(`Type::Borrow(Box)` / `Type::Own(Box)`). I rejected -that during the iter brief because adding new `Type` variants -requires touching every match-arm on `Type` in the entire -codebase — the typechecker alone has ~190 such arms, the -codegen/desugar/printer add another ~50. A new variant means a -~250-site migration, all of which would be "look through and -ignore", i.e. mechanical noise that degrades review signal. - -The chosen representation is per-position metadata on `Type::Fn`: - -```rust -Type::Fn { - params: Vec, - param_modes: Vec, // same length as params - ret: Box, - ret_mode: ParamMode, - effects: Vec, -} - -enum ParamMode { Implicit, Own, Borrow } -``` - -`Implicit` is the legacy state — semantically equivalent to -`Own` but printed bare (`(con T)`, no wrapper). `Own` and -`Borrow` are explicitly annotated. Type itself is unchanged, so -match-arm churn is zero. - -JSON canonical hash invariant: `param_modes` is skipped when -every entry is `Implicit`, `ret_mode` is skipped when -`Implicit`. Existing fixtures (sum, list, hof, closure, -list_map, std_*, etc.) emit byte-identical JSON. The hash -regression test in `crates/ailang-core/src/hash.rs` passes -unchanged. `git diff examples/` is empty after the patch. - -Decision 10's "Schema additions" subsection in `docs/DESIGN.md` -was rewritten to match this choice before the implementer ran; -it now describes the per-position layout, not the `Type` variant -sketch. - -### One subtle move: padding-on-read - -Construction sites in the typechecker / desugar / codegen -elide `param_modes` when none are non-`Implicit`, storing -`vec![]` even when `params.len() == n`. The `PartialEq` impl on -`Type::Fn` therefore pads the shorter slice with `Implicit` -before comparing — a `vec![]` and a `vec![Implicit; n]` compare -equal. - -This is the design lever that kept the patch from being a -~100-site mechanical migration. Every Fn-construction site that -already existed pre-18a stays a struct-literal with two new -elided fields (`param_modes: vec![], ret_mode: ParamMode::Implicit`) -rather than needing a per-call `vec![Implicit; n]`. New sites -that opt into mode info (the parser, eventually the inference -pass) write the mode vector explicitly. - -A `Type::fn_implicit(params, ret, effects)` constructor helper -exists for sites that want the explicit form. Currently unused -in 18a; will be the canonical constructor in 18c when -construction sites grow inference-derived modes. - -The "elide-and-pad" trick has one cost: 18c will need to decide -whether to keep the slack or normalise on construction. Keeping -the slack means consumers always need to be tolerant of partial -mode vectors. Normalising means every construction site has to -think about modes. Defer the call to 18c. - -### Parser - -`parse_param_with_mode` is the new helper. It peeks for a -leading `(borrow ...)` or `(own ...)` head and consumes the -wrapper, returning the inner `Type` plus the explicit `ParamMode`. -Otherwise it falls through to the regular `parse_type` and -returns the type with `ParamMode::Implicit`. `parse_fn_type` -calls it once per parameter slot and once for the return slot. - -`parse_type` (the top-level type parser) explicitly **rejects** -`borrow` / `own` as a type head outside fn-signature positions -with a clear ParseError. Two new unit tests in `parse.rs` cover -the rejection. Reasoning: modes outside fn signatures are -meaningless, and an LLM author who writes `(borrow Int)` as a -let-binding type expects an error — better to error at parse -time than typecheck time. - -### Printer - -`write_fn_type_slot` is the new helper in `print.rs`. It looks -up the mode for the param/ret position; on `Implicit` it prints -the inner type bare (preserving pre-18a output for every -existing fixture); on `Own` / `Borrow` it wraps with the -matching keyword. - -Round-trip verified by the existing -`ailang-surface/tests/round_trip.rs` (parses every `.ailx`, -prints, re-parses, demands canonical-byte equality). All -pre-existing fixtures pass unchanged. The new -`borrow_own_demo.ailx` round-trips identically. - -### Typechecker - -Modes are transparent for unification in 18a. The typechecker -matches `Type::Fn { params, ret, .. }` everywhere it already -did; the new fields are ignored under the `..` wildcard. No -change to substitution, instantiation, generalisation, occurs, -or apply. - -Mode-compatibility checking is deliberately deferred to 18c. -When two `Type::Fn`s unify, a future check will demand -`ParamMode::Borrow ≡ ParamMode::Borrow` (and `Implicit / Own` -are interchangeable). For 18a, the check is absent, so a -`(borrow T)` parameter can be passed where an `(own T)` was -expected. This is unsound under the future enforcement but -harmless under 18a's "all modes are Implicit-equivalent" -runtime contract. - -### Fixture - -`examples/borrow_own_demo.{ailx,ail.json}` exercises both -modes: - -``` -(fn list_length - (type (fn-type (params (borrow (con List))) (ret (con Int)))) - ...) - -(fn sum_list - (type (fn-type (params (own (con List))) (ret (con Int)))) - ...) -``` - -`main` builds `[1,2,3]`, prints `list_length(xs)` then -`sum_list(xs)`. Stdout is `3` then `6` (one per line via -`io/print_int`). - -The fixture exercises sharing — `xs` is used twice, once for -`list_length` (borrow, doesn't consume), once for `sum_list` -(own, consumes). Under 18c's linearity enforcement, this is -exactly the canonical "list_length doesn't take ownership so -sum_list still owns xs" pattern. Under 18a's no-enforcement -runtime, it just runs. - -The corresponding E2E test in `crates/ail/tests/e2e.rs` asserts -the stdout AND inspects the canonical JSON for the literal -strings `"param_modes":["borrow"]` and `"param_modes":["own"]`, -so a future serialisation regression flips the test red, not -just the runtime. - -### Build / test status - -`cargo build --workspace`: clean. `cargo test --workspace`: 154 -tests pass (+1 vs the 153 baseline at `pre-rc`), 0 failures, 3 -ignored doc-tests. Hash regression test in -`crates/ailang-core/src/hash.rs` passes — pre-18a fixtures all -produce identical canonical JSON, hashes unchanged. - -Touched files (15): - -- `crates/ailang-core/src/ast.rs` — `ParamMode` enum, `Type::Fn` - fields, `PartialEq` with `mode_eq` + `mode_slices_eq`, - `fn_implicit` helper. -- `crates/ailang-core/src/{desugar,pretty,hash}.rs`, - `crates/ailang-check/src/{lib,lift,builtins}.rs`, - `crates/ailang-check/tests/workspace.rs`, - `crates/ailang-codegen/src/lib.rs` — mechanical updates: `..` - on patterns, elided-default fields on struct literals. -- `crates/ailang-surface/src/parse.rs` — `parse_param_with_mode`, - fn-type integration, top-level rejection, two unit tests. -- `crates/ailang-surface/src/print.rs` — `write_fn_type_slot`. -- `crates/ail/tests/e2e.rs` — `borrow_own_demo_modes_are_metadata_only`. -- `examples/borrow_own_demo.{ailx,ail.json}` — new fixture. -- `docs/DESIGN.md` — Decision 10 schema-additions clarification + - `--memory=rc` → `--alloc=rc` flag rename for consistency with - existing CLI. - -### Did anything surprise - -- **The 250-site match-arm wall.** I had not internalised how - many sites destructure `Type` until I grepped for them. The - per-position-metadata representation chose itself once that - number became visible. The DESIGN.md sketch ("Type::Borrow as - variant") would have been a multi-iter implementation; the - metadata representation is one iter. - -- **The padding-on-read trick.** It feels hacky, and it is. - The honest alternative is normalisation on construction — - every Fn-builder site explicitly writes `vec![Implicit; n]`. - 18c can pay that cost when it has reason to, e.g. when - introducing a mode-compatibility check that wants to see - full-length vectors. For 18a the slack carries no cost. - -- **The fixture's "use xs twice" pattern.** The 18a fixture - *already* shows what 18c will need to enforce: a borrow call - followed by an own call on the same value. Under 18a this - works because nothing checks. Under 18c it should still work - because list_length's borrow declaration says "no consume". - The fixture is therefore a forward compatibility test for - the linearity-enforcement-with-borrow-correctly-spelled - story. If 18c lands and this fixture breaks, 18c got the - semantics wrong, not the fixture. - -### Next - -Iter 18b: RC runtime (`runtime/rc.c` already pre-staged in -working tree — `ailang_rc_alloc` / `inc` / `dec` with 8-byte -header layout) + codegen `--alloc=rc` routing. No inc/dec -emitted yet; programs leak intentionally. The point is to -validate that compiled programs run correctly under the new -allocator before 18c adds the inc/dec instrumentation that gives -the runtime contract its teeth. - -## Iter 18b — RC runtime (`runtime/rc.c`) + codegen `--alloc=rc` routing - -Pure plumbing iter. Establishes the RC runtime ABI and wires -`--alloc=rc` through the codegen + linker, but does NOT emit -any `inc` or `dec` calls. Programs running under `--alloc=rc` -allocate via `ailang_rc_alloc` and never free — the same -behaviour as pre-Boehm AILang. The point is to validate that -compiled programs still produce correct stdout under the new -allocator before 18c lights up the actual reference counting. - -### Runtime ABI (`runtime/rc.c`) - -Memory layout: an 8-byte `uint64_t` refcount header is prepended -to every payload. `ailang_rc_alloc(size)` allocates `size + 8` -bytes via libc `malloc`, sets the header to `1`, zero-initialises -the payload (matching `GC_malloc`'s contract), and returns a -pointer to the *payload*. The header is at `payload - 8`. - -`ailang_rc_inc(p)` increments the header at `p - 8`. -`ailang_rc_dec(p)` decrements; on zero-refcount it `free()`s -the underlying block. **Crucially**, dec does NOT recursively -dec child references in 18b — that requires per-type field- -layout info, which is 18c's job to wire up. For 18b, both `inc` -and `dec` are dead code from codegen's perspective; the codegen -emits zero calls to either. They exist as ABI placeholders so -18c can wire codegen against a stable surface. - -The runtime is single-threaded (counter ops are non-atomic); -when AILang acquires concurrency primitives, atomic-vs-non- -atomic becomes a separate decision per allocation kind, per -the "Does not commit to atomic refcounts" clause in Decision 10. - -### Codegen - -`AllocStrategy::Rc` was a one-line addition to the existing -enum. `fn_name()` returns `"ailang_rc_alloc"` for it. The rest -of the codegen — the `declare ptr @(i64)` line, the four -allocation sites (`lower_ctor`, lambda env, closure pair) — is -already parameterised on `alloc.fn_name()` from the bump-bench -work, so adding a third strategy required zero further codegen -edits. This is the "Iter 18a's bump path centralised the -allocator decision; 18b just adds a new value" payoff that the -implementer flagged for the JOURNAL. - -### CLI - -`--alloc=rc` is accepted by both `Build` and `Run`. -`locate_rc_runtime()` mirrors `locate_bump_runtime` — searches -upward from the binary and from CWD for `runtime/rc.c`. The -link arm compiles `runtime/rc.c` with `clang -O2 -c` and passes -the resulting `.o` to the main link command. **No `-lgc`** — -rc.c uses libc only. - -### E2E coverage - -Two new tests: - -- `alloc_rc_produces_same_stdout_as_gc` builds and runs - `list.ail.json` under both `--alloc=gc` and `--alloc=rc`, - asserts they produce identical stdout (`42`). -- `alloc_rc_matches_gc_on_std_list_demo` does the same for - `std_list_demo.ail.json` — broader allocation coverage - (length / sum / reverse / take/drop chained), every fold - goes through `ailang_rc_alloc`. - -E2E bundle: 51 tests (was 49 pre-18b, 50 with the 18a addition). -`cargo build/test --workspace` green. `git diff examples/` -empty — no fixture changes. - -### Hand-tested - -``` -$ cargo run --bin ail -- run examples/sum.ail.json --alloc=rc -55 - -$ cargo run --bin ail -- run examples/list.ail.json --alloc=rc -42 - -$ cargo run --bin ail -- run examples/borrow_own_demo.ail.json --alloc=rc -3 -6 - -$ cargo run --bin ail -- run examples/std_list_demo.ail.json --alloc=rc -... (matches --alloc=gc) -``` - -All correct. Programs allocate, produce output, exit. Memory -leaks all the way through (no `dec` is ever emitted), but at -the bench/test scales we run, the leak is bounded (<100 MB) and -the OS reclaims on process exit. - -### Did anything surprise - -- **Codegen change was a single line.** The 18a bump path had - already done the heavy lifting of centralising the allocator- - symbol decision on a single `fn_name()` method; 18b just adds - another arm to that match. This is the kind of compounding - return that pays for the bench iter retroactively — it built - the abstraction, 18b reuses it for free. - -- **No need for a `Type::fn_implicit` migration in 18b.** The - 18a JOURNAL flagged this as a possible cleanup; 18b confirmed - it's not blocking anything. The padding-on-read trick handles - every codegen-side construction site silently. Cleanup - remains optional, deferred to 18c if it has a reason. - -- **Boehm and RC have nearly the same emitted IR.** The two - binaries differ only in (a) the `declare ptr @(i64)` - symbol name and (b) the linker argument (`-lgc` vs. - `runtime/rc.o`). Allocation behaviour at the source level is - identical from codegen's perspective. This is what we wanted - — 18c's inc/dec emission can be added orthogonally without - re-architecting the allocation pipeline. - -### Next - -Iter 18c is the substantive next step. Three sub-pieces: - -1. **`Term::Clone { value }`** as a new schema variant. Author- - spelled explicit RC inc, like Lean 4's `(@Clone)` syntax. - Schema + parser + printer + typechecker passthrough. - -2. **Linearity check** as a new pass over the typechecked AST. - For functions with all-explicit-mode params, verify each - binder is consumed exactly once (or borrowed indefinitely). - Use-after-consume → structured diagnostic with - `suggested_rewrites`. Functions with any `Implicit` param - are exempt — that's the back-compat lane while existing - fixtures stay unannotated. - -3. **Uniqueness inference + codegen inc/dec emission.** - Post-typecheck dataflow over AST builds a side table - `BTreeMap`. Codegen consumes the table: - on every `Term::Var` of a shared reference that escapes its - binder, emit `call void @ailang_rc_inc(ptr %p)`. On every - binder going out of scope, emit `call void @ailang_rc_dec`. - The inference erases inc/dec on provably unique references - — that's the optimisation that closes the bump-allocator - gap on hot loops. - -18c is at least a 3-agent-run iter; tackle it as 18c.1 (clone -schema), 18c.2 (linearity check + suggested_rewrites), 18c.3 -(inference + codegen) sequentially. Don't attempt all three in -one pass — each builds on the previous and wants its own -verification cycle. - -## Iter 18c.1 — `Term::Clone` schema (no inc emission yet) - -Schema floor for explicit RC inc. Adds `Term::Clone { value }` -with serde tag `"clone"`, form-A `(clone X)`. Pure additive -schema — typechecker treats it as identity, codegen lowers it -exactly like its inner term. The runtime semantics of `(clone X)` -in 18c.1 are: nothing happens; it's an identity wrapper. - -The variant is the author-visible alternative to implicit -sharing under the LLM-aware RC design (Decision 10's -mechanism (2)). When 18c.2 ships linearity enforcement, an LLM -that wants two `own`-mode uses of the same binder will be -required to spell one of them as `(clone X)`. When 18c.3 ships -codegen, the `Term::Clone` arm in `lower_term` will emit a -single `call void @ailang_rc_inc(ptr %v)` before delegating -to the inner term's lowering. - -The 18c.3 emission seam is documented in the codegen arm with a -comment so the future-work site is greppable. - -### Sites touched - -The match-arm count was higher than the brief estimated -(~10 across 6 files) but every site was pure structural -recursion through `value`. Pretty.rs needed nothing — it has no -`match t {}` over Term. The non-obvious sites: - -- `crates/ailang-check/src/lib.rs::verify_tail_positions` — - records that `(clone tail-call)` is itself a tail call. Under - 18c.1's identity-of-clone semantics, the inner term keeps its - tail-position. (Under 18c.3, the inserted `inc` happens before - the call returns, so the tail-position story is preserved - there too — `inc` is not a function call from the LLVM-IR - tail-position perspective; it's an inline call after the value - is computed.) - -- `crates/ailang-codegen/src/escape.rs` — three sites in the - escape analysis: `walk`, `escapes`, `collect_free_vars`. The - pattern is the same as everywhere else — recurse through the - inner value. - -- `crates/ail/src/main.rs::walk_term` — the deps walker that - finds cross-module references for the workspace loader. Pass- - through. - -### Fixture - -`examples/clone_demo.{ailx,ail.json}`: - -``` -(module clone_demo - (fn main - (type (fn-type (params) (ret (con Unit)) (effects IO))) - (params) - (body - (let x 42 - (do io/print_int (clone x)))))) -``` - -Stdout: `42` under both `--alloc=gc` and `--alloc=rc`. The -fixture is intentionally minimal — the schema/parser/printer/ -round-trip path is exercised end-to-end, but the value being -"cloned" is a primitive `Int` where RC inc is meaningless even -in 18c.3. A richer fixture wrapping a list-bound clone is a -18c.3 concern (where clone actually does something). - -### Build / test - -`cargo build --workspace` clean. `cargo test --workspace` green: -52 E2E (+1), 14 surface parse (+2), all other buckets unchanged. -Hash regression test in `crates/ailang-core/src/hash.rs` passes -unchanged (the new serde tag `"clone"` appears in zero existing -fixtures). `git diff examples/` is empty for pre-existing -fixtures. - -### Did anything surprise - -- **Zero surprises.** The 18c.1 iter ran exactly like 18a in - shape: new variant + ~10 mechanical recursion sites + parser - + printer + fixture + test. Both iters lean on the same - underlying structure (an additive AST extension that - typechecker / codegen / desugar passes traverse without - semantic effect). When 18c.2 / 18c.3 land they will be - fundamentally different — those are real analysis passes - with semantic consequences. The schema-floor iter pattern is - rich because the project has good structural discipline - (every Term-traversal site uses pattern matching, not - reflection), and that's worth recording. - -- **The future-work seam is one line.** 18c.3 only needs to - flip the codegen `Term::Clone` arm from "lower inner, return - the SSA reg" to "lower inner, emit one `inc` call, return the - SSA reg". That's the entire emission cost of explicit clone. - The hard work in 18c.3 is everywhere else (uniqueness - inference + dec instrumentation across all binder-going-out- - of-scope sites); explicit clone is comparatively cheap. - -### Next - -Iter 18c.2: linearity check. Walk every fn body whose params -are all explicit-mode (`Borrow` or `Own`, no `Implicit`); track -each binder's consumption state through the AST; emit -structured `use-after-consume` / `consume-while-borrowed` -diagnostics with form-A `suggested_rewrites`. Functions with any -`Implicit` param stay exempt — that's the back-compat lane -while existing fixtures stay unannotated. - -The check is purely an addition to the diagnostic pipeline; it -emits no IR change. Existing fixtures (all `Implicit`) are -untouched. The new `borrow_own_demo` fixture (which has all- -explicit modes) becomes the first program subject to the check; -its current shape (borrow-then-own on `xs`) is exactly what the -check should accept, so 18c.2 starts as a "green for the -existing test" iter. - -## 2026-05-08 — Correction: 18a Schema-choice rationale - -User flagged that the 18a "per-position metadata vs. `Type` -variants" decision was justified in the JOURNAL / DESIGN.md / -commit message primarily by implementation effort ("avoids -~250 match-arm sites"). That is not a design rationale. The -choice may still be right (it is), but the *reasons* it is -right have to come from the language, not from the cost of an -alternative. New CLAUDE.md section "Design rationale ≠ -implementation effort" makes this a binding orchestrator rule. - -The substantive justification, retroactively recorded in -`docs/DESIGN.md` Decision 10's Schema-additions block: - -1. **Semantic locality.** Modes are properties of fn-signature - parameter positions, not of types in general. Embedding modes - in `Type` would let the schema permit forms like - `(con List (borrow Int))` — syntactically possible but - semantically meaningless. Decision 1's "schema permits - exactly what is meaningful" argues against the Type-variant - approach. -2. **Compositional clarity.** A `Type` value's identity should - depend only on the type. Calling-convention information - (own/borrow) is orthogonal to type identity; mixing them - conflates two axes that should be factored apart. -3. **Future-proof against more position metadata.** Per-position - metadata generalises naturally to additional dimensions - (streaming, captured, lifetime witness). The Type-variant - approach would force every new dimension into its own - `Type::*` variant and produce combinatoric ordering questions - (`Borrow(Streamed(T))` vs `Streamed(Borrow(T))`) that don't - arise when modes live in a flat metadata vector. - -The match-arm count remains true as an observation but appears -in DESIGN.md only parenthetically, marked explicitly as a -tiebreaker rather than a rationale. - -This entry stays as a record because the original mistake is -informative — design discipline corrupts faster than I notice -when I let "implementer-friendly" creep into the slot reserved -for "language-honest". The CLAUDE.md rule exists so the next -session catches this earlier. - -## Iter 18c.2 — linearity check + suggested_rewrites - -Pure diagnostic addition. New module `ailang-check::linearity` -walks every fn whose `param_modes` are all explicit (`Borrow` -or `Own`, no `Implicit`); tracks per-binder consume/borrow state -through the AST; emits `use-after-consume` and -`consume-while-borrowed` diagnostics with form-A -`suggested_rewrites`. No IR change, no codegen change, no runtime -change. Existing fixtures (all `Implicit`) untouched; the -all-explicit `borrow_own_demo` fixture passes the check -unchanged. - -### Activation gate - -The check is opt-in *by signature*: a fn has to have spelled -every param mode explicitly (none `Implicit`) and have at least -one param. Any `Implicit` in the signature skips the fn's body -entirely. This is the back-compat lane Decision 10 promised — -LLMs that want linearity guarantees opt in by writing modes; -hand-written or transitional code keeps the old behaviour. - -### Diagnostics - -Two codes, both at `Severity::Error`, both with -`ctx = {"binder": ""}`: - -- `use-after-consume` — a binder is referenced after a previous - reference already consumed it. Replacement: `(clone )` at - the *earlier* site, so the later site stays the unique - consume. - -- `consume-while-borrowed` — a binder is consumed while a borrow - of it is still live (a sibling subterm in the same call passed - it to a `Borrow` param earlier; or, for a `Borrow` parameter - of the enclosing fn, the caller's outer borrow is always live - for the body's whole duration). Replacement: `(clone )` at - the offending consume site. - -### Position model - -Each `Term::Var` occurrence is in either `Consume` or `Borrow` -position, computed from its parent term: - -- `App.callee` → Consume. -- `App.args[i]` → Borrow if the resolved callee type's - `param_modes[i] == Borrow`; otherwise Consume (Own / Implicit - / unknown all default to Consume). -- `Clone.value` → Borrow (clone reads + bumps RC, doesn't move). -- `Match.scrutinee` → Borrow. -- Everything else (`Let.value`, `If.cond`, `Ctor.args[*]`, - `Do.args[*]`, `Lam`-captures, …) → Consume. - -For `App` whose callee is a fn-typed `Var`: when an arg slot -takes a *bare* `Term::Var { name }` in Borrow position, we bump -`borrow_count[name]` *before* evaluating subsequent args, and -release after the call. So a sibling Consume of the same binder -within the same call triggers `consume-while-borrowed`. - -### `suggested_rewrites` - -New `Diagnostic` field, always serialised (`[]` when empty — -stable JSON shape for `ail check --json` consumers). Currently -populated only by the linearity codes; everything else emits -`[]`. Each `SuggestedRewrite` carries a free-form `description` -and a `replacement` string. The replacement is form-A AILang — -parseable by `ailang_surface::parse_term`, the new -single-term entrypoint added in this iter alongside the dual -`term_to_form_a`. The round-trip is a contract guarded by tests: -the workspace integration tests parse every emitted replacement -and panic if any fail. - -### Why a new `parse_term` / `term_to_form_a` pair - -Parser and printer previously only had whole-`Module` -entrypoints. The linearity check produces snippet-sized -suggestions, which forced either (a) ad-hoc string formatting -inside the check, or (b) a fully-fledged single-term -parser/printer pair. Choice (b) is right because the -`replacement` field is part of the diagnostic's *contract* — if -any future code path produces a string that the surface refuses -to parse, the round-trip test catches it before it ships. - -The pair is now public surface API of `ailang-surface` and is -the canonical way for any tool to produce form-A snippets that -round-trip cleanly. - -### Why gate on clean-typecheck before running - -`check_workspace` skips the linearity pass on any module that -already produced a typecheck diagnostic. Running on a partly- -defined IR (unresolved `Var` lookups, mismatched ctor arities) -would produce noise that competes with the upstream errors the -author actually has to fix first. Cleanly-typechecked modules -with at least one all-explicit fn are the surface this check is -designed to inspect. - -### Known false negatives (deferred to 18c.3) - -- **Missing-consume-of-Own.** An `Own` param matched by - `Term::Match` but never moved out of the body is not flagged. - Matching is a Borrow-position read by the position model; - catching "you declared `own`, you have to consume" requires a - must-consume analysis that the assignment kept out of scope. - 18c.3 will see it because uniqueness inference computes a - per-binder must-be-consumed bit anyway. - -- **Lam captures conservatively in Consume.** A `Term::Lam` body - is walked with each capture in Consume position; we don't - model "the closure borrows its capture for the closure's - lifetime". For 18c.2 this means closures over an explicit-mode - param effectively consume the param at the lam site, which - matches the conservative thing to flag if anyone tries to use - the param after closing over it. Full closure-borrow - discipline is 18c.3. - -- **Pattern bindings start fresh.** `Match` arm patterns bind - fresh names with default state; we don't propagate ownership - from the scrutinee into the pattern bindings. False positives - could only arise if a pattern re-binds a name already in scope - in the enclosing all-explicit fn — none in shipping fixtures. - -These three are deliberate scope cuts, not bugs. 18c.3's -uniqueness inference subsumes the first two by construction; the -third is rare enough that an explicit shadow-warning isn't worth -the schema cost. - -### Tests - -- `crates/ailang-check/src/linearity.rs` — 3 unit tests - (`explicit_fn_with_no_uses_is_clean`, `implicit_fn_is_exempt`, - `mixed_implicit_explicit_is_exempt`). -- `crates/ailang-check/tests/workspace.rs` — 3 integration tests - (`use_after_consume_on_own_param_is_reported`, - `consume_while_borrowed_in_sibling_arg_is_reported`, - `borrow_own_demo_is_linearity_clean`). Each negative test - asserts diagnostic kind, def, `ctx.binder`, AND that every - emitted `replacement` parses via `parse_term` (the contract - test). -- 1 unit test for `suggested_rewrites` JSON serialisation in - `diagnostic.rs`. - -`cargo build --workspace` clean, `cargo test --workspace` green. -E2E unchanged (52 → 52); ailang-check unit 34 → 38; ailang-check -integration 5 → 8. - -### Next - -Iter 18c.3: uniqueness inference + codegen `inc`/`dec` -emission. A dataflow over the AST computes, per binder, the -"unique" / "shared" status; codegen under `--alloc=rc` emits -`ailang_rc_inc` / `ailang_rc_dec` calls at the appropriate -seams (most notably: a single `inc` for every `Term::Clone` -inserted by 18c.2's suggested rewrites, and a `dec` at the last -use of each binder leaving scope). This is the iter where RC -actually starts collecting memory; 18b's leak-everything -behaviour ends here. - -The check from 18c.2 stays as the *user-visible* enforcement -surface; 18c.3's inference is internal codegen-side bookkeeping. -The two layers communicate only through the explicit -`Term::Clone` markers in the source — the linearity check -demands them, the codegen pass honours them. - -## Iter 18c.3 — uniqueness inference + non-recursive RC `inc`/`dec` - -End of the leak-everything era under `--alloc=rc`. Codegen now -emits `ailang_rc_inc` at every `Term::Clone` and `ailang_rc_dec` -at end-of-scope of trackable RC-allocated let-binders. The -default `--alloc=boehm` path is untouched. - -### Two parts, two files - -**Part A — `crates/ailang-check/src/uniqueness.rs`** (new module, -`pub mod uniqueness`). Post-typecheck dataflow producing -`UniquenessTable: BTreeMap<(def, binder), UniquenessInfo>` with -`Uniqueness::{Unique, Shared}` and a `consume_count: u32` -side-output. The walk reuses the position model from 18c.2's -linearity check (Consume vs Borrow), but runs unconditionally on -every fn (not gated on all-explicit modes) and produces no -diagnostics — the table is internal codegen input. - -`consume_count` is **max-over-paths**: at `Term::If` and -`Term::Match` the walker saves state, walks each arm against the -saved snapshot, and `merge_states` takes the max of the -per-binder counters. So `(let x e1 (if c x x))` gets -`consume_count = 1` (one arm consumes once, the other arm -consumes once, max = 1), correctly preventing codegen from -double-dec'ing on either branch. - -`Term::Clone { value }` is treated as a Borrow on the inner term -— the explicit clone is the user's signal that a fresh ref is -being produced via inc, not a fresh consume of the original -binder. - -5 unit tests: let unused / consumed once / consumed twice / -pattern bindings / `(clone)` is borrow. - -**Part B — codegen emission in -`crates/ailang-codegen/src/lib.rs`.** Two seams: - -- **`Term::Clone { value }`** (lib.rs:1256–1278): lower the inner - value, then under `--alloc=rc` and `val_ty == "ptr"` and - `!val_ssa.starts_with('@')` emit - `call void @ailang_rc_inc(ptr %v)`. The `@`-prefix gate elides - inc on top-level fn closure-pair globals — those live in the - LLVM data segment, not in `runtime/rc.c`'s 8-byte-header heap. - -- **`Term::Let { name, value, body }`** (lib.rs:1058–1110): a - new `is_rc_heap_allocated` predicate (lib.rs:984–1012) decides - whether `value` lowers through `ailang_rc_alloc` — true iff - `value` is `Term::Ctor` or `Term::Lam` AND the term is in the - current fn's escape-set (i.e. it heap-allocates rather than - `alloca`-allocates). When true AND `val_ty == "ptr"` AND the - uniqueness table reports `consume_count == 0` AND the body's - tail SSA is not the binder itself AND the current block is not - terminated, codegen emits - `call void @ailang_rc_dec(ptr %v)` after the body lowers, - before the let exits. The four extra gates each catch a - failure mode: - - `consume_count == 0`: a non-zero count means a callee/outer - site already took ownership; double-dec is undefined - behaviour against `runtime/rc.c`'s underflow guard. - - body-tail-is-binder: a let whose body returns the binder - transfers ownership to the caller — caller dec's, not us. - - block-not-terminated: a tail call or `unreachable` already - closed the block; emitting after is malformed LLVM IR. - - `non_escape` membership: stack `alloca`s have no header to - dec. - -Header declarations of `@ailang_rc_inc` / `@ailang_rc_dec` are -gated on `--alloc=rc` (lib.rs:421–429) so the Boehm and Bump -modules' IR shape is byte-identical to before this iter. - -### Fixture and E2E - -`examples/rc_box_drop.ail.json`: a single-cell `Box(Int)` ADT, -`main` does `(let b (MkBox 42) (match b (MkBox x) (print x)))`. -`b` has `consume_count == 0` (only borrow-position match -scrutinee), so codegen emits `dec` after the match join. The -`MkBox` cell has no boxed children, so the shallow `free()` in -`runtime/rc.c::ailang_rc_dec` is sufficient — this fixture -deliberately avoids the recursive-dec story (deferred to 18c.4). - -E2E test `alloc_rc_emits_dec_for_unique_let_bound_box` at -`crates/ail/tests/e2e.rs:1349` builds the fixture under both -`--alloc=gc` and `--alloc=rc`, asserts both emit `42`, and -asserts byte-identical stdout. Properties guarded: -1. dec does not free the box BEFORE the match reads its payload; -2. dec does not double-free or trip the underflow guard; -3. RC vs GC paths produce identical observable behaviour. - -### Test deltas - -- E2E: 52 → 53. -- ailang-check unit: 38 → 43 (5 new uniqueness tests). -- All other buckets unchanged. -- `cargo test --workspace` green. - -### Scope cuts (deliberate) - -- **Recursive `dec` cascades / per-type drop fns — Iter 18c.4.** - `runtime/rc.c::ailang_rc_dec` still frees the box without - recursing into boxed children (its top-of-file comment already - documents this). 18c.3 ships shallow free; recursive ADTs - (List, Tree) under `--alloc=rc` still leak everything except - the outermost cell. The fix is a per-type drop function the - codegen emits for each ADT — one `void @drop_(ptr)` - symbol that walks the boxed children, dec's each, then frees - the outer cell. - -- **Binders with `consume_count > 0`.** No `dec` is emitted — - the value moved to a callee or returned, and the receiving - site is responsible. This is correct under the current ABI - for callee-takes-ownership, but means a fixture that returns - a heap-allocated value still leaks the outer cell (the - caller's let-binding is a separate scope and a separate - question). 18c.4 / 18d will tighten this. - -- **Fn parameters.** The uniqueness table records them but - codegen does not emit `dec` on params — there's no static - signal yet for "this parameter is RC-allocated and the caller - has handed off ownership". `(own T)` parameters under explicit - modes carry exactly that semantic; wiring it through is part - of the wider mode-aware codegen story (likely 18d alongside - reuse hints). - -- **`runtime/rc.c::ailang_rc_inc` UB on static globals.** - Codegen now elides `inc` on `@`-prefixed SSAs, so the UB path - in `runtime/rc.c` is unreachable in practice. The runtime - itself still has no guard; a defensive check (e.g. flag bit - in the header word) is a tidy-iter concern, not load-bearing. - -### Why a separate inference pass from `linearity` - -Different scope, different output, different consumption point. -`linearity` is opt-in by signature (every param mode explicit) -and emits user diagnostics; uniqueness inference runs -unconditionally on every fn body and produces an internal side -table. They share the position model but nothing else, and -collapsing them would either force user-facing diagnostics on -fns that don't opt in, or hide the codegen input behind a gate -the codegen doesn't want. The two-pass design preserves -Decision 10's separation: linearity is the user-visible -*language* surface; uniqueness is the *implementation*. - -### Next - -Iter 18c.4 (queued, was implicit in 18c original splitting): -per-type drop fn / recursive `dec` cascade. Codegen emits a -`void @drop_(ptr)` for each `Type::Type` ADT that walks -boxed children, calls `ailang_rc_dec` on each, then frees the -outer cell. The `Term::Let` dec-emission seam from 18c.3 then -calls `@drop_` instead of `@ailang_rc_dec` directly -when the binder's type has boxed children. After 18c.4, RC -ADTs (List, Tree) under `--alloc=rc` should be allocation-leak- -free under `valgrind --leak-check=full`. - -After 18c.4 closes, the 18-arc continues with 18d (reuse hints -+ reuse analysis), 18e (drop-iterative + worklist free), 18f -(RC validation bench + Boehm retirement). After 18f the entire -18-arc closes — at which point the new tidy-iter rule from -CLAUDE.md kicks in: the next iter is `ailang-architect`-driven -drift cleanup before 19 starts. - -## Iter 18c.4 — per-type drop fn + recursive `dec` cascade - -Closes 18c.3's main scope cut. Codegen now emits a per-ADT and -per-closure drop function under `--alloc=rc`; the -`Term::Let`-scope-close call site routes through these drops -instead of `ailang_rc_dec` directly, so a recursive ADT (`List`, -`Tree`) under RC actually frees its tail cells when the outer -binder's drop fires. - -### The drop-symbol scheme - -For every `Def::Type` `T` in module `m`, codegen emits one -`define void @drop__(ptr %p)` under `--alloc=rc`. The body -is uniformly shaped: - -1. Null-guard the pointer (defensive — drop on null is a no-op). -2. Load tag from offset 0. -3. Switch on tag; one arm per ctor. -4. In each arm, for every pointer-typed field, load the field - and dispatch through `field_drop_call` (described below). -5. After every field is dec'd, `call void @ailang_rc_dec(ptr %p)` - to free the outer cell. -6. `ret void`. - -For closures, `Term::Lam` emits two drop fns when the closure -escapes (heap-allocated): `drop___env(ptr %env)` for -the captured-free-vars block, and `drop___pair(ptr -%pair)` for the `{ env, fn-ptr }` pair. The pair-drop dec's the -env via `drop___env`, then dec's the pair box. A new -`closure_drops: BTreeMap` field on `Emitter` -keys closure-pair SSA names to their pair-drop symbol so -`Term::Let` lookup is O(1). - -Decision (preempted in the brief): **always emit -`@drop__` for every ADT, even ones with no boxed -children.** A no-boxed-children ADT (like `Box(Int)`) gets a -drop fn whose body is just `null-guard; ailang_rc_dec; ret`. -Call sites uniformly call `drop__(ptr v)` and never -branch on "does this type have boxed children?". Two payoffs: -(1) the codegen call-site code is simpler; (2) any future -codegen pass that wants to thread additional cleanup through -the drop seam (e.g. atomic dec under threading) has one -canonical entry point per type. - -### `field_drop_call` — per-field dispatch - -Given a field's `Type`, `field_drop_call` produces the LLVM -instruction(s) that drop the field's value: - -- `Type::Con { name, .. }` (an ADT field) → call - `drop__` after qualifying `` via the - `import_map`. Recursive `IntList` → recursive `drop__IntList` - call, which is the cascade. -- `Type::Str` → `ailang_rc_dec` directly. Strings have no boxed - children and are not yet wrapped in their own per-type drop - (deferred — the per-type-drop seam is uniform but `Str` is a - primitive in DESIGN.md's eyes; no behaviour change). -- `Type::Fn` (a closure-typed field) → fall back to - `ailang_rc_dec`. The closure-pair dec route requires - `closure_drops` lookup which is keyed by *expression site*, - not *type*; a Fn-typed *field* doesn't carry the closure-pair - ID. Documented as a 18d/18e debt — closure-typed ADT fields - leak their captured envs. -- `Type::Var` (parameterised type, e.g. `Cons`'s head field - in a polymorphic `List`) → fall back to `ailang_rc_dec`. - We don't have monomorphisation yet, so `a` could be `Int` - (no-op) or `(con List Int)` (needs cascade). The conservative - choice is shallow free; full handling is monomorphisation - territory (orthogonal to RC, not in the 18-arc). - -The two fall-back cases are flagged at `field_drop_call` with -"# Iter 18c.4 debt" comments so they are greppable. - -### `Term::Let` switch-over - -The 18c.3 emission seam at `Term::Let` scope close now calls -`drop_symbol_for_binder(value, val_ssa)` instead of -`ailang_rc_dec` directly: - -- `Term::Ctor { type, .. }` binders → `drop__` - (looked up via `import_map`; falls back to current module if - the type is local). -- `Term::Lam` binders → the `closure_drops`-recorded pair-drop - symbol. -- Other expression shapes → unchanged from 18c.3 (codegen does - not own the lifecycle of a returned-from-callee box; that is - 18d's mode-aware story). - -The other emission gates from 18c.3 (`consume_count == 0`, -body-tail-not-binder, block-not-terminated, `non_escape` -membership) are unchanged. 18c.4 only swaps the *target* of the -call. - -### Recursion is stack-recursive (deferred to 18e) - -`drop__IntList`'s `Cons` arm calls `drop__IntList` on the -tail. For a 5-cell list, that's 5 stack frames. For a million- -cell list, that's a million frames and a stack overflow. **Iter -18e replaces this with a worklist allocator** — the recursive -call becomes a push-onto-worklist + iterative-pop loop. The -recursive call site in `field_drop_call` is commented with -"replaced in 18e by worklist-based iterative free". - -For 18c.4's shipping fixtures (5-element list), the depth is -safely below any reasonable stack. The worklist conversion is -mechanical once 18e introduces the worklist primitive. - -### Test additions - -- `examples/rc_list_drop.ail.json` — 5-element `IntList` summed - via `sum_list`; `xs` has `consume_count == 1` (passed to - `sum_list`), so codegen does NOT dec at main's let-close. - Fixture's purpose is to lock in the IR shape under - `--alloc=rc` (covered by the codegen unit test) and confirm - the recursive ADT compiles + runs correctly under both - allocators. Stdout: `15`. - -- `examples/rc_list_drop_borrow.ail.json` — same 5-element - `IntList`, but `main` only matches it (borrow) and prints the - head. `xs` has `consume_count == 0`, so codegen DOES emit - `call void @drop_rc_list_drop_borrow_IntList(ptr xs)` at - scope close. The drop fn's `Cons` arm recurses on the tail, - cascading through all 5 cells and freeing each in turn. The - E2E test asserts identical stdout vs `--alloc=gc` AND clean - exit (which is the implicit "no segfault, no underflow" check - for the recursive cascade). - -- `crates/ailang-codegen/src/lib.rs` unit test - `rc_alloc_emits_recursive_drop_fn_for_recursive_adt` — IR - shape lock-in. Asserts: - 1. `define void @drop_rclist_IntList(ptr %p)` is emitted - under `--alloc=rc`. - 2. The fn body contains a recursive call - `call void @drop_rclist_IntList(ptr %v...)` (the cascade). - 3. The fn body ends with - `call void @ailang_rc_dec(ptr %p)` (outer-box dec). - 4. Under `--alloc=gc`, NO `@drop_rclist_IntList` symbol - appears (negative complement — drop fns are an RC-only - emission). - -### Test deltas - -- E2E: 53 → 55 (+2: `alloc_rc_recursive_list_sum`, - `alloc_rc_borrow_only_recursive_list_drop`). -- ailang-codegen unit: 11 → 12 (+1: IR-shape test). -- All other buckets unchanged. -- `cargo test --workspace` green. - -### Known debt (deliberate, deferred) - -- **Stack-recursive drop** — Iter 18e replaces with worklist - allocator. Long lists overflow the stack today. -- **Closure-typed ADT fields** fall back to shallow - `ailang_rc_dec`. `field_drop_call`'s `Type::Fn` arm is the - greppable seam. -- **Parameterised type fields** (`Type::Var`) fall back to - shallow `ailang_rc_dec`. Full handling is monomorphisation, - orthogonal to RC. -- **Fn parameters** still don't get dec'd at fn return — the - caller-handed-off-ownership signal is the `(own T)` mode, but - wiring it through codegen is part of the wider mode-aware - story (18d alongside reuse hints). - -### Next - -Iter 18d: reuse hints + reuse analysis. The `(reuse-as -old-binder NewCtor)` form lets the author signal that a freed -cell can be overwritten in-place with a new ctor of the same -shape, avoiding the dec+malloc round-trip. Codegen lowers it to -a tag-overwrite + field-rewrite, eliding both the `@drop_` and -the `@ailang_rc_alloc` calls. Inference identifies the cases -where the rewrite is safe (the freed binder's last use is -exactly here, the new ctor has the same shape, no aliasing). - -## Iter 18d.1 — `Term::ReuseAs` schema + linearity check - -Schema floor for explicit reuse hints. Adds `Term::ReuseAs -{ source: Box, body: Box }` with serde tag -`"reuse-as"` and form-A `(reuse-as )`. Schema -shape (wrapper around a body, NOT a `reuse_from: Option` -modifier on `Term::Ctor`) and the substantive rationale were -ratified in DESIGN.md before this iter shipped — see the -"compositional flexibility" + "source-locality at the head" -paragraph in Decision 10's schema-additions block. - -Codegen is **identity** under every `--alloc` strategy — lower -`body`, drop `source` on the floor. The same staging the 18c.1 -→ 18c.3 split used: schema floor first so authors can write the -form, behaviour layered on later. Iter 18d.2 will replace the -identity codegen with the in-place rewrite under `--alloc=rc`. - -### User-visible diagnostics - -The wrapper schema permits `Term::ReuseAs` around a body that -isn't a constructor, which would be meaningless. Three -diagnostics close that gap: - -- **`reuse-as-non-allocating-body`** (typecheck) — `body` must - be `Term::Ctor` or `Term::Lam` (the two AST shapes that - allocate under `--alloc=rc`). A non-allocating body emits this - diagnostic with `ctx = {"got": ""}` and a - `suggested_rewrite` that drops the wrapper. - -- **`reuse-as-source-not-bare-var`** (linearity) — `source` - must be `Term::Var { name }` referring to an in-scope binder. - Anything else (literal, nested expression, out-of-scope var) - emits this with the same drop-the-wrapper suggested rewrite. - This rule is enforced by the linearity check rather than by - the typechecker because the constraint is about reuse - semantics, not type well-formedness; placing it in the - linearity pass also gates it on the all-explicit-mode - activation rule, so back-compat fixtures are unaffected. - -- **`use-after-consume`** (existing 18c.2 code, fired at a - reuse-as site) — `source`'s named binder must not have been - consumed earlier in the body. The fix is the same: drop the - wrapper. - -Visiting `Term::ReuseAs { source: Var { name }, body }` marks -`name` as consumed for the rest of the body's walk, so a -subsequent use of the same binder flags `use-after-consume` -against it. The linearity check thereby enforces "reuse-as -consumes the source exactly once". - -### Why the "source not bare var" rule is in linearity, not typecheck - -The schema permits `(reuse-as (some-expression) )` -syntactically. But reuse-as semantics says "free `source`'s -slot, write `body` into it" — that only makes sense if -`source` denotes a unique heap allocation we can name. A -binder name is the AILang way to name a unique heap -allocation; an arbitrary expression doesn't have that -property. Forcing `source` to be `Term::Var` is a *use-rule*, -not a *type-rule* — it's about the discipline of how reuse-as -is composed, not about whether the wrapping expression -typechecks. Linearity is where use-rules live. - -This pattern matches Decision 1's "schema permits exactly what -is meaningful" trade-off: the schema doesn't try to forbid -non-var sources structurally; the linearity pass does, with a -diagnostic the LLM author can act on. - -### Uniqueness awareness - -The 18c.3 uniqueness inference also walks `Term::ReuseAs` — -`source` is treated as Consume (counts toward `consume_count`), -`body` walks normally. The codegen consumer in 18d.2 will use -this to know when the source is in fact unique at the reuse-as -site (a precondition for the in-place rewrite to be safe). - -### Test additions - -- 4 surface parse-tests (round-trip, no-args rejection, one-arg - rejection, full `parse_term` / `term_to_form_a` round-trip). -- 1 typecheck unit test (`reuse_as_with_non_allocating_body_is_reported`). -- 2 linearity unit tests (`reuse_as_with_non_var_source_is_reported`, - `reuse_as_after_consume_is_use_after_consume`). -- 1 integration test (`reuse_as_happy_path_in_map_inc_is_linearity_clean`) - — exercises the canonical case: `(fn (own (con List)) → (own - (con List)))` whose body is `(match xs (Nil → Nil) (Cons h t → - (reuse-as xs (Cons (+ h 1) (map_inc t)))))`. Asserts no - diagnostics. -- 1 E2E test (`reuse_as_demo_is_identity_in_18d1`) — runs - `examples/reuse_as_demo.ail.json` under `--alloc=gc` and - asserts `9` (the sum of `[2, 3, 4]`). -- New fixture `examples/reuse_as_demo.{ailx,ail.json}` — the - canonical `map_inc`-via-reuse-as program. Identity codegen - produces the same output as the non-reuse-as version, so the - fixture's role in 18d.1 is to exercise the schema + linearity - + parser surface; 18d.2 will repurpose it as the in-place- - rewrite IR-shape lock-in. - -### Test deltas - -- E2E: 55 → 56. -- ailang-check unit: 43 → 46 (+3). -- ailang-check workspace: 8 → 9 (+1). -- surface: 14 → 18 (+4). -- All other buckets unchanged. -- `cargo test --workspace` green. -- No existing fixture's canonical JSON changed — - `Term::ReuseAs` is new, so no 18c-or-prior fixture serialises - to a different byte-string. - -### Next - -Iter 18d.2: codegen lowers `Term::ReuseAs { source, body=Ctor }` -under `--alloc=rc` to in-place tag-overwrite + field-rewrite, -eliding the `ailang_rc_alloc` call AND the per-field-drop -cascade for the source's old fields (the dec is replaced by -unconditional store of the new field values; ownership of the -old-field references must transfer to a per-field dec before -the store, since the old slot's pointer-typed values would -otherwise leak). Other allocators ignore the wrapper (same -identity behaviour as 18d.1 ships universally). - -Adds the `reuse-as-shape-mismatch` diagnostic when codegen -discovers the source's ctor and the body's ctor have -different sizes / layouts (only checkable once codegen has -resolved both ctors' field counts). - -## Iter 18d.2 — codegen reuse-as in-place rewrite - -The `Term::ReuseAs` schema-floor from 18d.1 now carries -behaviour under `--alloc=rc`. Two design points: a runtime -refcount-1 dispatch in the IR, and a static shape-compatibility -check in a new pre-codegen pass. - -### Runtime refcount-1 dispatch - -Following the Lean 4 / Roc precedent, codegen emits a runtime -branch on the source's refcount at the reuse-as site: - -``` -%hdr_ptr = getelementptr i8, ptr %src, i64 -8 -%refcnt = load i64, ptr %hdr_ptr -%is_one = icmp eq i64 %refcnt, 1 -br i1 %is_one, label %reuse, label %fresh -``` - -- **Reuse arm** (refcount == 1, the source IS unique at - runtime): overwrite the box in place. Store the new tag at - offset 0; store the new field values at offsets 8, 16, …. - No `ailang_rc_alloc`, no outer `@drop_` cascade — the - source's slot becomes the new ctor's slot. -- **Fresh arm** (refcount > 1, the source is shared): allocate - a fresh box via `ailang_rc_alloc`, store the new tag + fields - into it, and shallow-dec the source (so the source's - refcount drops by one — the OTHER live ref still points at - the old contents until that other ref's lifetime ends). -- Both arms phi-join to the same `ptr` value. The result of - `Term::ReuseAs` is a pointer to the new ctor's box, by - whichever path the runtime took. - -The branch is per reuse-as site, not per program path — every -reuse-as at runtime makes the runtime decision. That is the -correct trade-off for refcount-1 dispatch: an unconditional -in-place rewrite would corrupt sharing in the > 1 case; an -unconditional fresh-allocate would defeat the optimisation. -The trip cost is two loads + one compare + one branch on every -reuse-as — a one-digit-nanoseconds tax for a malloc-and-cascade -saving when the runtime path goes through the reuse arm. - -### Why the dispatch lives at codegen, not in `runtime/rc.c` - -The decision is per-site, not per-allocation: reuse-as -specifies WHERE to write (this slot, possibly aliased to other -data), not just WHEN to free. A runtime-helper function would -have to take an unbounded parameter list (the new field values -+ types) and either re-do work the codegen already did or push -that work back into a more general "type-driven copy" runtime -primitive. Embedding the branch in IR keeps the codegen -specialised per `Term::Ctor` shape and uses LLVM's existing -phi infrastructure, with no new runtime ABI. - -### Static shape compatibility — `crates/ailang-check/src/reuse_shape.rs` - -A new pre-codegen pass tracks the path-resolved ctor of every -let-bound and pattern-matched binder, then checks each -`Term::ReuseAs` site for shape compatibility between source's -ctor and body's ctor. The new diagnostic -`reuse-as-shape-mismatch` carries stable `ctx.reason` sub-codes -plus a drop-the-wrapper `SuggestedRewrite`: - -- `field-count-mismatch` — source's ctor has N fields, body's - ctor has M fields. The slot-by-slot rewrite cannot proceed - because the box sizes (or slot layouts) differ. -- `field-type-mismatch` — same field count, but field i in - source has a different LLVM type than field i in body. A - pointer-typed slot cannot be overwritten with an `i64` (or - vice versa) without breaking the layout invariant. -- `indeterminate-source-ctor` — the path that the reuse-as is - on does not statically determine a unique ctor for source. - Conservative reject — the user should restructure so the - ctor is locally evident, or remove the wrapper. -- `cross-module-body-ctor` — the body's ctor lives in a module - that the current module doesn't import in a position where - 18d.2 can resolve it. Out-of-scope today; deferrable. -- `ctor-not-in-module` — typecheck should have caught this - upstream; defensive guard. - -Build fails on any of these. The structured diagnostic shows -the user whether to fix the shapes or remove the hint. - -### Field-cleanup design — explicit scope cut - -The reuse arm does **NOT** dec old pointer-typed fields before -overwriting them with the new field values. The brief -specified `dec old before store new`, and the implementer -correctly identified that this schedule causes use-after-free -on the canonical fixture: - -``` -(reuse-as xs (term-ctor List Cons (app + h 1) (app map_inc t))) -``` - -Why: `(map_inc t)` recursively returns `t`'s in-place-rewritten -box (when reuse fires one level deeper) — so the "new tail" -about to be stored is the same pointer as the "old tail" we'd -be dec'ing. Dec-old-then-store-new would drop the box's -refcount to 0, free it via the cascade, then store the freed -pointer. - -The root cause is upstream of 18d.2: pattern-matching a `Cons` -binding `h` and `t` does NOT null out the source's slots. So -when the reuse-as site evaluates `(map_inc t)`, the t binder -holds a refcount on the box, but xs.tail STILL contains the -same pointer. From the slot's perspective, the field hasn't -been "moved out" — the slot still references the box. - -The chosen trade-off: skip the field-dec entirely. Pattern- -wildcarded fields (rare; e.g. `(match xs (Cons _ _ → reuse-as -xs (Cons 0 0)))`) leak. Pattern-moved fields are the body's -responsibility — and the body's evaluation has not consumed -them either (it's holding a refcount via the binder). The -result: 18d.2 leaks the SAME shape 18c.4 already leaks (Own -fn-parameters are not dec'd at fn-return, pattern-wildcarded -fields are not dec'd at scope close). No regression vs the -current baseline; the perf win on alloc + outer cascade ships. - -The proper fix is the move-aware-pattern story (18d.3 / 18e): -when a pattern binds a pointer-typed field, codegen should -either (a) null out the source's slot at the pattern point, or -(b) track at codegen-time which slots are "still owned by the -source" vs "transferred to a binder", and emit dec only on the -former at reuse-as / drop time. That work is queued; 18d.2 -ships the perf-only half. - -### Test additions - -- `examples/reuse_as_demo.ail.json` under `--alloc=rc` exits 0 - with stdout `9`. Same fixture 18d.1 ran under - `--alloc=gc`; under RC, 18d.2's reuse arm fires and the - runtime correctly produces the same observable behaviour. -- `crates/ail/tests/e2e.rs::reuse_as_demo_under_rc_uses_inplace_rewrite` - asserts stdout matches `--alloc=gc` AND inspects the LLVM IR - to confirm: the `icmp eq i64 …, 1` branch is present, AND - the `reuse.` block does NOT call `@ailang_rc_alloc`. Two - positive contracts: behavioural equivalence with GC; the - in-place rewrite is actually emitted, not silently bypassed. -- `crates/ailang-check/tests/workspace.rs::reuse_as_shape_mismatch_is_reported_on_cons_to_nil` - — a fixture using `(reuse-as cons_var (Nil))` triggers - `reuse-as-shape-mismatch` with `reason = field-count-mismatch`. -- `reuse_shape::tests` — 3 unit tests covering same-ctor-clean, - Cons→Nil reject, and indeterminate-source-ctor reject. - -### Test deltas - -- E2E: 56 (the 18d.1 identity test was converted into the rc - in-place test; net unchanged at 56). -- ailang-check unit: 46 → 49 (+3 reuse_shape tests). -- ailang-check workspace: 9 → 10 (+1 shape-mismatch). -- All other buckets unchanged. -- `cargo test --workspace` green. - -### Known debt (deliberate, deferred) - -- **Move-aware pattern bindings** — Iter 18d.3 / 18e. Pattern - matching a pointer-typed field should null out the source's - slot or otherwise mark it transferred, so reuse-as can dec - remaining pointer-typed slots safely. -- **Fn-parameter dec at fn return** — same upstream issue as - 18c.3/18c.4. `(own T)` parameters under explicit modes carry - the "caller handed off ownership" signal but codegen does - not yet emit a dec. -- **Atomic refcounts** — Decision 10's deferred concern; not - in any 18-arc iter. -- **The fresh-arm shallow-dec is conservative.** Like 18c.3's - let-close emission, the source's refcount drops by 1 but no - recursive cascade fires. For a refcount > 1 source, that's - correct (other refs still alive). For the rare path where - the source's refcount IS 1 but reuse-as still goes to fresh - (impossible today — the runtime branch correctly routes such - cases to the reuse arm), we'd want a full cascade. Not - reachable; flagged for grep. - -### Next - -Iter 18e: `(drop-iterative)` annotation + worklist allocator. -Closes the stack-recursion limit on 18c.4's drop fns. Long -lists (millions of cells) currently overflow the stack on -free; the annotation switches the synthesised drop fn from -recursive to iterative-with-explicit-worklist. Likely also -the place to land the move-aware-pattern story alongside — -both are about closing the deallocation surface, and the IR -infrastructure overlaps. - -After 18e, Iter 18f closes the 18-arc: RC validation bench -(target 1.3× of bump on `bench/run.sh`), and if met, retire -Boehm — flip default to `--alloc=rc`, drop `-lgc`, mark -Decision 9 historical. The CLAUDE.md tidy-iter rule then -fires: the next iter after 18f is `ailang-architect`-driven -drift cleanup before 19 begins. - -## Iter 18d.3 — move-aware pattern bindings (static tracking) - -Closes 18d.2's per-field-dec scope cut. Strategy (b) from the -JOURNAL queue: codegen-side bookkeeping rather than source -mutation. - -### How and why this strategy was chosen - -A first attempt at strategy (a) — "null out the source's slot -at the pattern-bind point" — was dispatched, implemented, and -correctly stopped by the implementer when two structural -problems surfaced: - -1. **Borrowed scrutinees must not be mutated.** A `(borrow T)` - parameter walked via `match` would have its slots null'd by - the body, breaking the borrow contract for any downstream - caller. 18a fixed this in the schema; 18c.2 enforces it in - linearity; null-out at codegen would have violated it - silently. - -2. **Desugar chains re-scrutinise.** `Desugarer::desugar_match` - produces chains where the same scrutinee binder is matched - in multiple consecutive matches (the canonical lowering for - nested patterns + multi-arm match). Arm 1 nulling the - source's slots makes arm 2's fall-through re-load see null - pointers and segfault. Systematic for any non-flat match. - -The implementer's report listed both as design-level issues -not fixable inside the iter's seam. Strategy (a) was retired. - -Strategy (b) — codegen-side bookkeeping — sidesteps both: the -source box stays bit-identical, and the move information rides -in a side table that's consulted only at top-level emission -sites that already know which binder they're emitting for. -Substantive reasons: - -- **Borrow-safe.** Source is never mutated. -- **Desugar-safe.** Re-loads after the move see the original - pointer, not null. -- **Allocator-uniform without observable IR change.** Under - `--alloc=boehm` / `--alloc=bump`, no per-field dec is - emitted; the bookkeeping is read but never produces output. - Under `--alloc=rc`, it informs which dec calls fire. -- **Bounded scope.** Per-binder, per-fn-body, lexically scoped - by binder lifetime. Same shape that `linearity.rs` and - `uniqueness.rs` already manage. - -Trade-off: more state in codegen. Acceptable; the alternative -introduced two structural bugs. - -### Implementation - -`Emitter` gains a new field: - -```rust -moved_slots: BTreeMap> -``` - -Keyed by binder name, valued by the set of field indices that -have been moved out via a pattern destructure. Reset per fn -body in `emit_fn`; saved/restored around `lower_lambda` thunk -emission so closure bodies don't see the parent fn's moves. - -`lower_match` captures `scrutinee_binder` (when the scrutinee -is a bare `Term::Var`). For each ctor arm, the pattern-binding -loop records `moved_slots[scrutinee_binder].insert(idx)` for -non-wildcard, pointer-typed slots — but does NOT emit any -store into the source. Pattern-bound binders (h, t) are -removed from `moved_slots` at arm body close (they're new -binders with their own clean scope). - -Two consumer sites: - -- **`Term::Let` scope close.** Existing 18c.4 behaviour: emit - `call void @drop__(ptr %binder)` if the binder is - RC-allocated and `consume_count == 0`. Now: `take` - `moved_slots[binder]` first. If empty, emit the same call - (no IR change for the common case — every shipping fixture - that doesn't pattern-extract anything follows this path). - If non-empty, route through a new helper - `emit_inlined_partial_drop` that emits per-field - `field_drop_call` for non-moved pointer-typed slots, then - `ailang_rc_dec` on the outer cell. - -- **`lower_reuse_as_rc` reuse arm.** Re-introduces the - per-field dec that 18d.2 punted on. For each pointer-typed - field of the source's old ctor: if its index is in - `moved_slots[source_binder]`, skip; otherwise load the field - and emit `field_drop_call`. The 18d.2 in-source comment - block is replaced with: "Iter 18d.3: per-field dec is now - safe — moved-out slots are skipped via `moved_slots`; non- - moved slots are dec'd via `field_drop_call`'s null-guarded - drop fns." - -`drop__` itself is unchanged. The cascade has no notion -of "partially moved" — whatever binder it's called on, the -caller has already decided that binder is fully owned. Partial- -move complexity is confined to the call sites with the static -info. - -### Tests - -- New fixture `examples/pat_extract_partial_drop.{ailx,ail.json}` - — `Pair(IntList, IntList)` destructured `(Pair a _)` — - exercises both a moved pointer slot (a) and a wildcarded - pointer slot (the `_` for the second IntList). -- New E2E `alloc_rc_partial_drop_skips_moved_keeps_wildcarded` - asserts: - 1. Stdout `6` under both `--alloc=gc` and `--alloc=rc`. - 2. The IR at `main`'s let-close inlines a `@drop__IntList` - call for the wildcarded slot, AND does NOT contain the - uniform `@drop__Pair` call (which would dec both - slots; the moved one must be skipped). -- Updated `reuse_as_demo_under_rc_uses_inplace_rewrite`'s IR- - shape assertion: the canonical fixture moves the only - pointer slot, so the reuse arm should now contain neither - `@drop_` nor `@ailang_rc_dec` for fields (just the new - field stores). - -### Test deltas - -- E2E: 56 → 57 (+1). -- All other buckets unchanged. -- `cargo test --workspace` green. - -### Known regression — narrow, deliberate, queued - -The 18c.4-shipped `alloc_rc_borrow_only_recursive_list_drop` -fixture had a pattern `(Cons h t)` where `t` was bound but -never consumed downstream. Under 18c.4 + `--alloc=rc`, -`drop__(xs)` at let-close cascaded through -`xs.tail` and freed the entire 5-cell chain. Under 18d.3 with -strategy (b), `xs.tail` is in `moved_slots`, so the cascade -is interrupted there — but `t` is never consumed, so its -4-cell tail now leaks. - -The fixture stdout is unchanged (still `11`); E2E test still -passes. The regression is in *memory hygiene*, which the -current test infrastructure does not assert against (no -valgrind / leak-check in CI). - -The fix is the symmetric counterpart to 18c.4's known -fn-parameter-dec debt: pattern-bound binders whose -`consume_count` is `0` at arm body close should also get a -drop call emitted. Same emission seam as let-close-drop, just -fired at arm boundary instead of let-body boundary. Queued as -18d.4. - -This is the first iter to ship with a documented memory- -hygiene regression that's not test-detectable. The trade-off: -strategy (b) is the only sound move-aware approach (strategy -(a) had two structural bugs); pattern-binder-dec at arm close -is mechanically the same shape as let-close-drop and is a -small standalone iter; shipping 18d.3 unblocks 18d.4 without -forcing them into one giant iter. Until 18d.4 ships, fixtures -with pattern-extracted unused pointer binders leak under -`--alloc=rc`. - -### Next - -Iter 18d.4: pattern-binder dec at arm close. Symmetric to -18c.4's let-close-drop emission but fired when a pattern arm -exits and binders go out of scope. Closes the regression -above and the symmetric `(own T)` parameter-dec gap from -18c.4 / 18c.3 (the param case is a parameter-list rather than -a pattern arm, but the emission seam is the same shape). -After 18d.4, the move-aware-pattern story is complete and -the 18-arc moves to 18e (worklist). - -## Iter 18d.4 — pattern-binder + Own-param dec at scope close - -Closes 18d.3's known memory-hygiene regression AND the -symmetric Own-param-leaks debt 18c.3 / 18c.4 left open. Both -emission sites share the same shape: a binder going out of -scope without being consumed (`consume_count == 0`) gets a -drop call emitted, gated on the value being `ptr`-typed and -the current block being open. - -### Two emission seams, one shape - -**(A) Pattern-binder dec at arm body close.** Inside -`lower_match`, after each arm's body lowers, the codegen -walks the arm's pattern-bound binders. For each whose value -is `ptr`-typed and uniqueness side table reports -`consume_count == 0`, emit a drop: - -- If `moved_slots[binder]` is empty (the common case — - pattern-bound binders rarely have moves of their own), - emit `call void @drop__(ptr %binder)` via - `field_drop_call` — exactly the 18c.4 path. -- If `moved_slots[binder]` is non-empty, fall back to shallow - `ailang_rc_dec` of the outer cell only. See "dynamic-tag - partial-drop" below. - -Wildcard slots are NOT dec'd here — they have no binder. The -source's drop (handled by 18d.3's `emit_inlined_partial_drop` -or the uniform `drop__`) takes care of them. - -**(B) Own-param dec at fn return.** Inside `emit_fn`, the -fn-type destructure now also pulls `param_modes`. After the -body's tail value lowers but before the `ret` instruction, -the codegen walks the parameter list. For each parameter `p` -where: - -- `param_modes[i] == ParamMode::Own`, -- the parameter type lowers to `ptr`, -- the uniqueness side table reports `consume_count == 0`, -- AND the parameter's SSA is NOT the body's tail value (that - would transfer ownership to the caller — caller dec's), - -emit a drop with the same routing as (A). - -`Borrow`-mode parameters are explicitly excluded — caller -still owns. `Implicit`-mode parameters are also excluded — -no static "caller handed off ownership" signal under -back-compat (a future iter could choose to opt-in this case -via uniqueness inference). - -### Why both in one iter - -The emission shape is uniform: "binder leaves scope, no one -consumed it, drop." Pattern-arm-close and fn-return are just -two lifecycle points where this fires. Splitting them into -separate iters would have produced two iters with effectively -the same emission code at different binder-lifecycle points; -the implementer noted no friction in handling them together. - -### `drop__` remains unchanged - -Same call as 18c.4 emits — same uniform per-type drop fn. -The partial-move complexity stays at the call sites with the -static info. The cascade has no notion of "partially moved"; -when a parent's drop walks into a child via -`field_drop_call`, the child is fully owned at that point. - -### Dynamic-tag partial-drop — debt continues - -`emit_inlined_partial_drop` (18d.3) requires a static -`Term::Ctor` for layout — its per-field load sequence is -keyed against a fixed ctor's field shape. Pattern-bound -binders and Own params often have only their static *type* -known at the dec site, not their runtime ctor (e.g. an `xs: -List` whose ctor is determined by the runtime value). When -`moved_slots[binder]` is non-empty for such a binder, the -partial-drop emission falls back to **shallow -`ailang_rc_dec`** of the outer cell only — it doesn't try to -walk fields. - -For 18d.4's shipping fixtures, this fallback is sound: - -- `rc_list_drop_borrow` (the 18d.3 regression): `t`'s - `moved_slots` is empty (the regression fixture doesn't - re-scrutinise `t`), so the per-type - `drop__(t)` fires cleanly via the (A) path. - Closes the leak. -- `rc_own_param_drop`: `xs` has `moved_slots[xs] = {1}`, but - the active ctor's only ptr field (`Cons.tail`) has already - been dec'd by the (A) path on `t`, and the `Nil` runtime - alternative has no ptr fields anyway. Shallow dec of the - outer cell is correct on every reachable runtime path. - -A general dynamic-tag partial-drop would emit a tag-switch -at the dec site (or pass a moved-mask parameter into a -parameterised `drop__`). That's a separate iter; not -load-bearing for the shipping fixtures of the 18-arc. - -### Side effect on `reuse_as_demo` (correctness improvement) - -`sum_list` in `examples/reuse_as_demo` has signature `(fn -((own (con List))) → Int)` and `consume_count(xs) == 0` -(match scrutinee is Borrow). (B) now fires shallow -`ailang_rc_dec(%arg_xs)` at each recursive return. Combined -with the recursive `sum_list(t)` chain, this dec's one cell -per stack frame on unwind, freeing the entire chain before -`main` exits. - -Stdout unchanged (`9`). The existing IR-shape assertions on -`map_inc`'s reuse arm continue to hold — they scope between -`reuse.:` and `\nfresh.:` labels and don't see -`sum_list`'s body. - -### Test deltas - -- E2E: 57 → 58 (+1: `alloc_rc_own_param_dec_at_fn_return`). -- All other buckets unchanged. -- `cargo test --workspace` green. -- The 18d.3 regression on `alloc_rc_borrow_only_recursive_list_drop` - is now closed: the IR-shape assertion confirms `t`'s drop - fires at arm close. Stdout unchanged (still `11`); memory - hygiene strictly improved. - -### Known debt (deliberate, deferred) - -- **Dynamic-tag partial-drop** — described above. Falls back - to shallow `ailang_rc_dec` for binders with both non-empty - `moved_slots` AND dynamic runtime ctor. -- **Closure-typed Own params** — fall through to - `field_drop_call`'s `Type::Fn` arm (shallow dec). Same path - 18c.4 carved out for closure-typed ADT fields. -- **Implicit-mode params with `consume_count == 0`** — not - dec'd. `Implicit` is the back-compat opt-out lane; its - parameters carry no caller-ownership signal. -- **`(drop-iterative)` worklist allocator** — Iter 18e. - -### What 18d.4 closes structurally - -The move-aware-pattern story spanning 18d.3 + 18d.4 is now -complete for the static cases: - -1. Pattern destructure of an owned scrutinee marks the slot - as moved (18d.3's `moved_slots`). -2. Drop of the source at let-close inlines a per-field dec - sequence skipping moved slots (18d.3's - `emit_inlined_partial_drop`). -3. The reuse-as reuse-arm dec's old non-moved fields safely - (18d.3, since moves are tracked statically and source is - never mutated). -4. Pattern-bound binders whose new homes don't consume them - get drop-emission at arm close (18d.4 (A)). -5. `(own T)` parameters that aren't consumed inside the fn - get drop-emission at fn return (18d.4 (B)). - -Together: the move-aware story closes the 18d.2 explicit -scope cut AND the implicit 18c.3 / 18c.4 Own-param-leaks gap. -The remaining hygiene gap is dynamic-tag partial-drop, which -is structurally orthogonal to the move-aware story (it'd -exist even with no moves anywhere — it's about not knowing -the runtime ctor at a partial-drop site). - -### Next - -Iter 18e: `(drop-iterative)` annotation + worklist allocator. -The recursive cascade in `drop__` overflows the stack -on long lists (millions of cells). The annotation switches -the synthesised drop fn from recursive to iterative-with- -explicit-worklist, allowing arbitrarily deep ADT chains to -free without stack growth. The IR infrastructure overlaps -with the dynamic-tag-partial-drop story (both want a -per-call-site customised free), so 18e may also be the -natural place to close the dynamic-tag fallback. - -After 18e, Iter 18f closes the 18-arc with the RC validation -bench. The CLAUDE.md tidy-iter rule then fires: `ailang-architect` -drift cleanup before 19 begins. - -## Iter 18e — `(drop-iterative)` annotation + worklist allocator - -Closes the 18-arc's stack-recursion limit. Long ADT chains -(millions of cells) overflow the recursive `drop__` -cascade from 18c.4. The opt-in `(drop-iterative)` annotation -on a `Def::Type` switches that type's synthesised drop fn from -recursive to iterative-with-explicit-worklist. Validated on a -1M-cell `IntList` fixture: with the annotation, exits 0; with -the annotation stripped, SIGSEGV (exit code 139). The worklist -is load-bearing, not cosmetic. - -### Schema additions - -`TypeDef.drop_iterative: bool` — default `false`, serde-skip -when false. Existing fixtures' canonical JSON hashes stay -stable; only fixtures that opt in carry the new key. Form-A: -a `(drop-iterative)` clause inside the `(data T ...)` decl, -matching DESIGN.md Decision 10's example. - -### Worklist strategy — heap stretchy buffer - -Three strategies were named in the implementer brief: - -1. Heap-allocated stretchy buffer (always `malloc`, double on - overflow). -2. Stack-allocated small buffer with heap-spill. -3. Slot-repurposing — thread the worklist through one of the - box's own pointer-typed slots (Lean 4 technique). - -The implementer chose **(1)**, with the rationale documented -in `runtime/rc.c`: - -- (3) requires every box to have at least one pointer-typed - slot that's NOT the slot we're cascading through. Not - generally true: `Cons (Int) (List)` has slot 0 = i64 (head) - and slot 1 = ptr (tail). The tail IS the cascade slot; - there's no other ptr slot to repurpose. Generalising to ADTs - with multiple ptr slots adds codegen complexity (per-ctor - decision: which slot is the worklist link?). -- (2) is more efficient for short lists but adds dual-buffer - spill logic. Marginal benefit since the worklist is only - emitted under the opt-in annotation and the annotation's - primary use case IS long lists. -- (1) is simplest and matches the failure mode the iter - targets (deep recursion on long lists). - -Four new ABI symbols in `runtime/rc.c`, declared on demand -under `--alloc=rc`: - -``` -void* ailang_drop_worklist_new(size_t initial_capacity); -void ailang_drop_worklist_push(void* wl, void* ptr); -void* ailang_drop_worklist_pop(void* wl); // returns NULL on empty -void ailang_drop_worklist_free(void* wl); -``` - -`push` null-filters: pushing a null pointer is a no-op (matches -the existing 18c.4 invariant that drops null-guard at entry). -The buffer doubles on overflow. - -### Codegen — `emit_iterative_drop_fn_for_type` - -For a `drop_iterative: true` type, `drop__(ptr %p)` -emits a worklist body: - -``` - ; alloc worklist; push p -loop_head: - ; pop next; if null, free worklist + ret - ; load tag; switch on tag - ; per-ctor arm: - ; for each pointer-typed field: - ; load it; push (or call other drop fn directly if - ; it's a different type) - ; ailang_rc_dec the outer cell - ; br loop_head -``` - -The IR contains a `br label %loop_head` jumping back to the -top — the load-bearing structural difference from the -recursive shape. The IR-shape test asserts both this and the -**absence** of any direct recursive call to `drop__` -inside the body of the same fn. - -### Mono-typed worklist - -The worklist is **mono-typed**: only fields of the SAME -annotated type push onto it. Fields of a DIFFERENT type -(whether iterative or recursive) call their own -`drop__` directly, opening one level of recursive -cascade for that boundary. Three trade-offs: - -- **Pro: simple worklist.** No need to track per-entry type - tag; every popped pointer is the same type, dispatched - through the same tag-switch. -- **Con: a `(drop-iterative)` `Tree` containing - `(drop-iterative)` `List` fields would still recurse one - level when crossing the type boundary.** For nested - iterative types both deep, this could be a problem if the - cross-type chain is itself long. None of 18e's shipping - fixtures hit this — the canonical case is a single - recursive ADT (`List` containing `List`). -- **Pro: tag-switch in the loop body is per-type, not - per-entry.** Compiles to a smaller loop body. - -The trade-off is documented in `emit_iterative_drop_fn_for_type`'s -doc. A future iter could generalise to a heterogeneous worklist -(per-entry type tag, generic dispatch loop), but it isn't -needed for the deep-self-recursion case the annotation targets. - -### Test additions - -- `examples/rc_drop_iterative_long_list.{ailx,ail.json}` — - builds a 1M-cell `IntList` (annotated `(drop-iterative)`) - via a counted-recursion fn, sums it, prints the sum. -- `alloc_rc_drop_iterative_handles_million_cell_list` E2E — - asserts the 1M-cell fixture exits 0 under both `gc` and - `rc`. **Hand-verified separately**: the same fixture with - the `(drop-iterative)` annotation stripped SIGSEGVs at - ~1M cells under `--alloc=rc` (exit code 139). The - annotation is load-bearing. -- `iter18e_drop_iterative_emits_worklist_body_no_self_recursion` - — IR-shape test: a fixture with `(drop-iterative)` should - emit a `drop__` body containing `br label %loop_head` - AND no direct recursive `call void @drop__(...)` in - the same body. -- `iter18e_no_annotation_keeps_recursive_drop_body` — control - test: the same fixture WITHOUT the annotation still emits - the recursive 18c.4 shape (no `loop_head`, recursive call - present). -- 3 surface parse-tests: - `parses_drop_iterative_annotation_on_data_decl`, - `parses_data_without_drop_iterative_defaults_to_false`, - `rejects_drop_iterative_with_arguments`. - -### Test deltas - -- E2E: 58 → 61 (+3). -- Surface unit: 18 → 21 (+3). -- All other buckets unchanged. -- `cargo test --workspace` green. -- Existing fixtures' canonical JSON hashes are stable - (`drop_iterative` defaults to `false` and serde-skips when - false, so no fixture's serialisation changed). - -### Known debt (deliberate, deferred) - -- **Mono-typed worklist.** Cross-type drop-iterative fields - call the other type's drop fn directly rather than sharing - a heterogeneous worklist. Sound for the deep-self-recursion - case (the canonical `List` of `List` of `Int` is one type, - not three). -- **Closure / `Type::Var` / `Type::Forall` fields** fall back - to shallow `ailang_rc_dec` via `field_drop_call` — same - fall-back path 18c.4 carved out, unchanged here. -- **Dynamic-tag partial-drop fallback** (the - `emit_inlined_partial_drop` shallow-dec path from 18d.4 - when `moved_slots` is non-empty AND runtime ctor is - unknown). Structurally orthogonal to 18e — would exist with - no `(drop-iterative)` annotations either. Deferred to a - future iter. - -### Validation against the 18-arc's brief - -Decision 10's "(4) `(drop-iterative)` annotation on data -declarations" said: "When the refcount of a `Tree` value -reaches zero, the synthesised dec-on-zero traversal is -iterative (worklist + heap-allocated stack) instead of -recursive. Avoids stack overflow on deep structures. The LLM -adds the annotation where appropriate; the compiler refuses -to emit recursive dec-cascade on annotated types." - -18e ships exactly that: opt-in annotation, codegen swaps the -emission strategy, runtime helpers in `runtime/rc.c`, -heap-allocated worklist. The "compiler refuses to emit -recursive dec-cascade on annotated types" half is enforced by -construction — `emit_drop_fns` dispatches on -`td.drop_iterative` and emits exactly one of the two bodies. - -### Next - -Iter 18f closes the 18-arc with the RC validation bench. -Target: 1.3× of bump on `bench/run.sh`. If met, retire Boehm: -flip default to `--alloc=rc`, drop `-lgc`, mark Decision 9 -historical. - -After 18f closes, the new tidy-iter rule from CLAUDE.md fires: -the next iter is `ailang-architect`-driven drift cleanup over -the entire 18-arc surface (all of 18a, 18b, 18c.x, 18d.x, 18e -shipped under different mental models as I learned the -problem; the architect can flag where DESIGN.md and the -shipped code have drifted, and decide which side moves). - -## Iter 18f — RC validation bench (Boehm retirement DEFERRED) - -Bench/run.sh extended with an `--alloc=rc` column. Decision -10's retirement criterion was: "RC within 1.3× of bump on -`bench/run.sh`. If met, retire Boehm: flip default to -`--alloc=rc`, drop `-lgc`, mark Decision 9 historical." - -Results on the 2 shipping bench fixtures (RUNS=5, drop slowest, -median of 4): - -``` -workload | gc(s) | bump(s) | rc(s) | gc/bump | rc/bump | gc RSS(KB) | bump RSS(KB) | rc RSS(KB) ------------------------+---------+----------+----------+---------+---------+------------+--------------+------------ -bench_list_sum | 0.142 | 0.049 | 0.140 | 2.90× | 2.86× | 103788 | 97628 | 193692 -bench_tree_walk | 0.101 | 0.038 | 0.096 | 2.66× | 2.53× | 73452 | 55452 | 109208 -``` - -`rc/bump = 2.5–2.9×`. **Target not met. Boehm retirement -deferred.** - -### Why RC is much slower than bump on these workloads - -The bench fixtures (`bench_list_sum`, `bench_tree_walk`) are -both Implicit-mode — they were authored before the 18a explicit- -mode infrastructure shipped, and they declare no `(borrow T)`, -`(own T)`, `(clone X)`, `(reuse-as ...)`, or `(drop-iterative)` -annotations. Under `--alloc=rc` with Implicit-mode params: - -- `ailang_rc_alloc` is called instead of `bump_malloc`. That - alone is a step from a hot-path bump-pointer (`ptr += size; - return old_ptr`) to a libc `malloc(size + 8)` call plus - header init plus payload memset. Libc malloc is a general- - purpose allocator, not optimised for the fixed-size ADT-cell - pattern AILang exercises hot. -- No `inc`/`dec` is emitted on Implicit-mode params (18c.3's - known debt — `Implicit` is the back-compat opt-out lane). - The cells leak. This means RC's TIME cost is purely the - allocate-path tax — no free-path tax — but the allocate - tax is large. -- Under bump, `bump_malloc` is a 2-instruction inline. Under - RC, `ailang_rc_alloc` is a libc call. The 2.5–2.9× ratio is - in line with this single-factor difference. - -### The relevant comparison: gc/bump ≈ rc/bump - -The numbers also show **gc and rc are within 5% of each other -on both fixtures.** Boehm's `GC_malloc` and `ailang_rc_alloc` -both go through libc-malloc-equivalents and pay similar -per-call costs. RC isn't slower than Boehm in any meaningful -sense — they're tied. - -So flipping the default from Boehm to RC would not regress -performance on Implicit-mode fixtures; it would just preserve -the current cost. The reason to *retire* Boehm rather than -just *make RC default* is: removing the `-lgc` link dependency -and Boehm's runtime overhead simplifies the toolchain. But -that simplification is independent of the 1.3× target. - -### Two paths forward - -**Path A: Lower the retirement threshold.** Decision 10's -1.3× was set in expectation of an inlined slab/pool allocator -in `runtime/rc.c`. A more honest target given the current -implementation is "RC ≤ Boehm ± 5%". On that bar, retirement -DOES qualify. The trade-off is that the language is committed -to RC at performance parity with Boehm rather than at -bump-allocator-floor performance. - -**Path B: Build the slab/pool allocator first.** Replace -`malloc(size + 8)` in `ailang_rc_alloc` with a fixed-size -slab allocator (one slab class per common cell size, free-list -recycling, batched OS allocation). This would close most of -the gap to bump on cell-allocation-heavy workloads. New iter -work, ~one full iter on its own. Then re-run the bench; if -within 1.3×, retire Boehm under the original criterion. - -**Path A is the pragmatic call.** Boehm's main defect was -unbounded GC pause variability and the `-lgc` dependency. Both -go away under either path. Path B's perf win is real but -narrow (helps the alloc-heavy hot path, doesn't help anything -else). Path B is an iter we can do later as a tuning pass when -real workloads show the alloc tax bites. - -### What this iter does NOT do (deliberate) - -- Does NOT flip the default to `--alloc=rc`. The retirement - decision is the orchestrator's call between Path A and - Path B; both are real options that need user input. -- Does NOT drop `-lgc`. Same. -- Does NOT mark Decision 9 historical. Decision 9 still - documents the transitional Boehm state we're in. - -### What it does ship - -- `bench/run.sh` extended with the `rc` column and the - `rc/bump` ratio (the decisive number). -- The bench numbers above, recorded in this entry. -- This entry, which decides the retirement question by - declining to decide it autonomously: the 1.3× criterion was - the orchestrator's commitment, and not meeting it is a - prompt for an orchestrator-level discussion (Path A vs - Path B), not for an implementer to ship a default flip. - -### Status of the 18-arc - -The 18-arc was: 18a (modes), 18b (runtime), 18c.1–4 (clone + -linearity + uniqueness + per-type drop), 18d.1–4 (reuse-as + -move-aware patterns + scope-close drops), 18e (drop-iterative -+ worklist), 18f (bench + retirement decision). - -All shipping infrastructure is now in place. The behaviour -under `--alloc=rc` for Implicit-mode fixtures is correct (same -output as `--alloc=gc`) and approximately as fast as Boehm. -Explicit-mode fixtures additionally benefit from in-place -reuse, iterative drop, and prompt deallocation at scope close; -the RC machinery covers them too. - -The retirement question is open pending Path A vs Path B, and -the new tidy-iter rule (CLAUDE.md) hasn't fired yet because -the family hasn't formally closed. Two paths forward: - -1. Resolve the Path A / Path B question, ship the corresponding - 18f.2, formally close the 18-arc, then do the tidy-iter. -2. Treat 18f's deferred-decision as the close, do the tidy- - iter NOW, then revisit the retirement question with a clean - slate. - -Both reasonable. Orchestrator's call. - -## 2026-05-08 — Bug fix: 18d.4 Iter A scrutinee-mode gate - -First bug fix shipped under the new TDD-for-bug-fixes rule -(CLAUDE.md, same commit). The bug was surfaced indirectly: the -new `ailang-bencher` agent could not run a tail-latency bench -under `--alloc=rc` because the bench fixture (Implicit-mode -recursive list/tree walk) crashed with `ailang_rc_dec` -refcount-underflow. Minimised to -`examples/rc_pin_recurse_implicit.ailx`: - -``` -(fn pin (params t) (body - (match t - (case (pat-ctor TLeaf) 0) - (case (pat-ctor TNode v l r) 1)))) - -(fn loop (params n t) (body - (if (app == n 0) 0 - (let _v (app pin t) (app loop (app - n 1) t))))) -``` - -`pin` is Implicit-mode. Caller `loop` passes `t` to `pin` and -then re-uses `t` itself in the recursive call. Under `--alloc= -rc`, `pin`'s pattern arm `(TNode v l r)` was emitting -`drop__Tree(l)` and `drop__Tree(r)` at arm close — but -the cells `l` and `r` were loaded out of `t`'s heap layout, and -`t` is still owned by `loop`. The dec'd children fragment the -tree the caller still references; on the recursive call into -`pin` again, the now-dangling children get re-loaded and the -ailang_rc_dec underflow trips. - -### Why Iter A had this bug and Iter B did not - -Iter B (Own-param dec at fn return, also 18d.4) gated correctly: -it consults `param_modes[i] == ParamMode::Own` and skips -`Implicit` and `Borrow`. The rationale was already in -`emit_fn`'s comment block: "Implicit-mode params do NOT get -this dec: they have no static caller-handed-off-ownership -signal." Iter A is the same shape — pattern-binders loaded out -of a scrutinee owe their drop-validity to the scrutinee's -ownership — but Iter A's emission seam in `lower_match` -predated the same gate and just fired on every ptr-typed -pattern-binder with `consume_count == 0`. - -The asymmetry was a copy-paste-of-rationale-without-copy- -paste-of-gate. Both sites need the same check; the fix is the -literal Iter B gate, threaded onto the emitter as -`current_param_modes` and consulted in `lower_match`. - -### Fix shape - -- New per-fn-body field on the emitter: - `current_param_modes: BTreeMap`. Set in - `emit_fn` from the fn type's `param_modes`; reset and saved - in `lower_lambda` (lambda thunks have their own param-mode - frame; thunk params default to `Implicit`). -- In `lower_match`, before Iter A's emission loop, derive - `scrutinee_is_owned`: - - If the scrutinee binder resolves to a fn-param: must be - `ParamMode::Own`. - - If non-fn-param (let-binder, temp expression): treat as - owned. -- Skip Iter A when `!scrutinee_is_owned`. - -`emit_fn`'s Iter B gate is unchanged. - -### TDD record - -Red: `crates/ail/tests/e2e.rs` — -`alloc_rc_pattern_bind_in_implicit_fn_does_not_dec_borrowed_children`. -Builds the fixture under both `--alloc=gc` (control: prints -`0`) and `--alloc=rc` (must match). Pre-fix: rc binary exited -with the underflow abort. Post-fix: both print `0`. Kept as -regression. - -### Carve-out: let-aliases of borrowed scrutinees - -A scrutinee that is a let-binder *holding* a non-Own param's -value (e.g. `(let x t (match x ...))` where `t` is Implicit) -would still mis-dec, because `current_param_modes` only knows -about params, not let-aliases. The fix gates only the direct- -fn-param case. The carve-out is recorded here rather than -fixed: the canonical regression doesn't trigger it (matches go -directly on `t`), and a let-alias-aware extension would either -need (a) propagation of "borrow-flavour" through let-bindings -in codegen, or (b) using uniqueness inference's `consume_count` -on the let-binder more aggressively. Both are widening of the -mode-as-ownership-signal apparatus and belong in their own -iter, not in this fix. - -### Out-of-scope: Implicit-mode safety more broadly - -This fix removes a refcount underflow on Implicit-mode -recursive shapes. It does NOT address the broader 18c.3 debt -that Implicit-mode params don't get *any* dec (Iter B already -documented this — `Implicit` is the back-compat lane, params -in this mode leak rather than trip). The fix here only ensures -arm-close pattern-binders don't dec what they shouldn't; it -does not start dec'ing what they should. Net effect on -Implicit fixtures: same correctness as `--alloc=gc` (no -regression), allocate-path tax only (matches the 18f bench). - -## 2026-05-08 — Iter 18f.2: tail-latency bench, Decision 10 supported on the latency axis - -`ailang-bencher` ran the tail-latency bench that the 18f -throughput bench couldn't speak to. New harness: -`bench/latency_harness.py` — PTY-line-buffered stdout + -`monotonic_ns` per line + inter-arrival distribution. Paired -fixtures: `bench_latency_{implicit,explicit}.ailx`. - -Hypothesis: under sustained alloc pressure with a large -persistent live working set (depth-19 tree, ~16 MB), Boehm has -a long STW tail (p99 ≫ median); explicit-mode RC has p99 within -a small constant factor of median. - -### Numbers (5 runs, AMD 5900X) - -``` -arm | wall(s) | RSS(MB) | median | p99 | p99.9 | max | p99/med -implicit-mode @ Boehm GC | 0.238 | 66 | 96.2 | 7131 | 7987 | 7989 | 74.1× -explicit-mode @ RC | 0.348 | 511 | 280.3 | 453 | 521 | 526 | 1.62× -implicit-mode @ RC (control) | 0.340 | 511 | 278.5 | 450 | 456 | 492 | 1.61× -``` - -Units µs unless stated. - -### Latency-axis verdict: SUPPORTED - -Boehm's max (~7.99 ms) is **83× its median**. RC's max is -**1.85× median**. Boehm's STW pauses are real and ~7 ms wide. -RC has no comparable tail. The signal sits two orders of -magnitude above the harness noise floor (~27 µs printf+pipe -roundtrip), so it's not an artefact. - -This is the first piece of evidence for Decision 10's -real-time claim. Boehm-retirement is *not* unblocked yet — see -the new finding below — but the latency axis, which 18f -admitted it didn't measure, now has signal. - -### Surprise finding: explicit-mode RC leaks at the same rate as implicit-mode RC - -Both RC arms peak at 511 MB RSS. Implicit-mode RC leaking is -expected (18c.3 documented debt). Explicit-mode RC leaking the -same amount is **not** what Decision 10 + the 18-arc promised. -The mode annotations + `(reuse-as)` + `(drop-iterative)` are -all present in `bench_latency_explicit.ailx`'s hot path -(`sum_list_acc`, `cons_n_acc`), and they should drive prompt -deallocation of each per-op IntList. - -Bencher's diagnosis: the hot path is tail-recursive -(`tail-app sum_list_acc t ...`); the pattern-binder `t` is -*consumed* into the tail-call, so its `consume_count > 0` at -arm-close, so 18d.4 Iter A skips. The outer LCons cell -(scrutinee `xs`) has all its pointer fields moved out (`h` is -Int, `t` was moved into the tail-call), so the outer cell is a -husk — but no drop site emits the shallow `ailang_rc_dec` for -it. moved_slots correctly records the fields are gone; nothing -wires "all fields moved → outer cell can shallow-dec". - -This is a genuine 18d.4 implementation gap, not a bug in the -existing emission seams. Iter A and Iter B are correct as -specified — they handle the *binder*-level drop. What's -missing is the *outer-cell* shallow-dec when the moved-from -cell becomes a husk inside a tail-recursive arm. - -### What this implies for the retirement question - -The 18f throughput bench failed its 1.3× target. This bench -shows that target was the wrong axis: on the *correct* axis -(tail latency), RC wins decisively. But the leak finding means -"explicit-mode RC works as designed" is not yet established. -Two open questions, both implementer territory: - -1. **Outer-cell shallow-dec for moved-from scrutinees.** The - immediate fix the bencher's finding points to. Should - close the leak path on the canonical tail-recursive - list/tree fixtures. -2. **Re-bench after the fix.** Confirm RSS drops to live-set - + small constant on the explicit arm. Median latency may - tick down (less alloc pressure → fewer libc-malloc calls). - Tail latency on the explicit arm should tighten further. - -After both land, the retirement decision (Path A vs Path B -from 18f's entry) gets a real evidence base. - -### Update (later same day, post-18g.1) - -The first item — outer-cell shallow-dec for moved-from -scrutinees — shipped as Iter 18g.1 (commit `ae2eb2e`, -preceded by `fc5f459` which added the `AILANG_RC_STATS=1` -counter infrastructure used by the regression test). Net -effect on `bench_latency_explicit` under `--alloc=rc`: -`allocs=11068575 frees=10020000 live=1048575`. The 10 M -freed cells correspond to the 20000 ops × 500-cell IntLists -that previously leaked. The remaining `live=1048575` is the -persistent depth-19 Tree cache (524287 TNode + 524288 TLeaf = -1048575). That's a separate at-program-exit cleanup issue, -not part of the same scope-close drop debt — queued as a -follow-up rather than a re-bench blocker. - -A re-run of the latency bench post-18g.1 is owed before the -retirement question can move; the Iter 18g.1 fix did not -touch the latency path's instruction shape (it only inserts -one extra `ailang_rc_dec` per consumed list element), so the -tail-latency win recorded above is not at risk, but the -median may shift slightly. - -### What this iter (18f.2) ships - -- `bench/latency_harness.py` (committed in `ac70011`). -- `examples/bench_latency_{implicit,explicit}.{ailx,ail.json}`. -- This entry. No DESIGN.md change yet — the leak is an impl - gap, not a Decision 10 revision. -- A new task (`#82` outer-cell shallow-dec) is queued. - -The 18f throughput entry's "Path A vs Path B" framing remains -open and is now joined by the outer-cell-shallow-dec finding; -both feed the eventual retirement decision. - -## 2026-05-08 — Iter 18g.1: outer-cell shallow-dec at tail-call sites - -`ailang-bencher`'s 18f.2 finding (above) localised: in -`(case (LCons h t) (tail-app sum_acc t (...)))`, the -pattern-binder `t` is consumed into the tail-call (uniqueness -records `consume_count > 0`), and the LCons outer cell becomes -a husk — all its ptr fields are moved into binders that the -downstream frame owns, but no drop site emits a shallow free -for the outer cell itself. The two existing 18d.4 emission -seams (Iter A: arm-close pattern-binder dec; Iter B: Own-param -dec at fn return) both run AFTER `lower_term(arm.body)`; for a -tail-call body, lower_term sets `block_terminated = true`, so -both seams skip. The husk leaks once per recursion step. - -### Fix - -A new pre-tail-call seam in `lower_match`, emitted *before* -`lower_term(arm.body)` so it lands ahead of the `musttail -call`. Conditions, all required: - -1. `alloc = Rc`. -2. `arm.body` is structurally `Term::App { tail: true, .. }` - or `Term::Do { tail: true, .. }`. -3. `scrutinee_is_owned` (existing 18d.4 Iter A param-mode - gate, hoisted to be shared between the new seam and Iter - A). -4. Every ptr-typed slot in this ctor's pattern is in - `moved_slots[scrutinee]`. A Wild-bound ptr would still hold - a live ref that the shallow free would strand; this - condition rules out that case. - -The dec is a *shallow* `ailang_rc_dec`, not a -`field_drop_call` / `drop__` — the per-type drop fn -would re-walk the ptr fields and dec'ing values now owned by -the downstream frame is exactly the bug 18d.3's `moved_slots` -infrastructure was set up to prevent. - -### Test infrastructure (Iter 18g.0, shipped first) - -`runtime/rc.c` gained two non-atomic uint64_t counters -(`g_rc_alloc_count`, `g_rc_free_count`) on the alloc / dec-to- -zero paths. An `__attribute__((constructor))` registers an -atexit handler IFF `AILANG_RC_STATS` is set in the -environment at startup; the handler prints - - ailang_rc_stats: allocs=N frees=M live=K - -to stderr. Default-disabled. The e2e test layer added a -`build_and_run_with_rc_stats(example) -> (stdout, allocs, -frees, live)` helper that compiles with `--alloc=rc`, runs -with `AILANG_RC_STATS=1`, and parses the atexit summary. This -is now the standard way to assert RC correctness invariants -("live == 0 at exit" for fixtures that own no live ADTs at -return). - -### TDD record - -Red: `alloc_rc_explicit_mode_tail_sum_does_not_leak_outer_cells` -on `examples/rc_tail_sum_explicit_leak.ail.json` (a 100-element -tail-recursive list-sum with full mode annotations). Pre-fix: -`live = 100` (one LCons leak per consumed element). Post-fix: -`live = 0`. Kept as regression coverage. - -### End-to-end verification - -`bench_latency_explicit` under `--alloc=rc`: -- Pre-fix: `allocs=11068575 frees=20000 live=11048575`. -- Post-fix: `allocs=11068575 frees=10020000 live=1048575`. - -The 10 M freed cells correspond to the 20000 × 500 IntList -cells that previously leaked. The remaining `live=1048575` = -524287 TNode + 524288 TLeaf, exactly matches the depth-19 -Tree cache the bench fixture intentionally keeps live across -the whole loop. That cache's deallocation is a separate -issue: main's let-scope close should dec the Tree at program -exit, but doesn't on this fixture — queued as Iter 18g.2 -follow-up, not a regression introduced here. - -### Carve-outs (deliberate) - -- The seam fires only at *match* arms whose body is a tail - call. A let-bound owned value passed into a tail-call as - an argument (e.g. `(let xs (cons_n n) (tail-app sum_acc xs))`) - has no match — its drop emission still happens at the - let-scope close, which IS reached because the let body's - result is the tail-call's return value. No fix needed there. -- Tail-app of `Term::Do` (tail-effects) is gated identically; - in practice no fixture exercises a moved-from scrutinee at - a tail-effect site, but the symmetry is correct. -- A scrutinee that is a let-binder *holding* a non-Own-param - alias (the same carve-out as 18d.4 Iter A's fix) still - evades the param-mode gate; the carve-out is unchanged - and still queued as a propagation-pass iter. - -## 2026-05-08 — Iter 18g.2: let-binder drop for Own-returning App - -18g.1 closed the per-op leak (LCons outer cells in tail- -recursive arms) but `bench_latency_explicit` still reported -`live = 1048575` at exit — exactly the depth-19 Tree cache -(524287 TNode + 524288 TLeaf) that main holds across the -whole run. Diagnosis: the existing `is_rc_heap_allocated` -predicate returned `false` for any `Term::App`, citing the -doc-comment "later iters tied to (own) ret-mode contracts". -We have those contracts (Iter 18a); this iter delivers the -deferred case. - -### Fix shape - -`is_rc_heap_allocated` widens: an `App` whose callee carries -`ret_mode == Own` is now trackable. The signal is the -callee's own static contract — the typechecker gates `Own` -ret-modes through the same machinery that gates `(own T)` -parameter modes (Iter 18a / 18c.2's linearity check), so the -predicate is sound by construction. `Borrow`-returning calls -remain non-trackable (the callee retains ownership) and -`Implicit`-returning calls remain non-trackable (back-compat -lane, leaks rather than mis-decs). - -`drop_symbol_for_binder` gains an `App` arm: synthesise the -return type, resolve `Type::Con { name }` to -`drop__` with cross-module qualification through -`import_map`. Falls back to `ailang_rc_dec` for non-`Type:: -Con` returns (e.g. unresolved type vars on a polymorphic -call's pre-monomorphisation site — the monomorphised copies -get the right symbol). - -`emit_inlined_partial_drop` now defaults to shallow -`ailang_rc_dec` when `value` is not `Term::Ctor`, instead of -panicking. With Iter 18g.2's wider input set, a -pattern-match against an `App`-bound let-binder can populate -`moved_slots`; the partial-drop helper has no static ctor to -key off, so the dynamic-tag carve-out fires (same family as -18d.4's `(case (LCons h t) ...)` debt). Closing this -remaining leak path requires a tag-conditional helper; queued -for later. - -### TDD record - -Red: `alloc_rc_let_binder_for_owned_returning_app_drops_at_scope_close` -on `examples/rc_let_owned_app_leak.ail.json` — pre-fix -`live = 3` (a depth-1 Tree: 1 TNode + 2 TLeaf children), -post-fix `live = 0`. Kept as regression coverage. - -### End-to-end on bench_latency_explicit - -``` -arm | wall(s) | RSS(KB) | live (allocs/frees) -implicit @ gc (Boehm) | 0.245 | 66400 | n/a (no rc-stats) -explicit @ rc (pre-18g.1) | 0.348 | 511000 | 11048575 (11068575 / 20000) -explicit @ rc (post-18g.1) | n/a | 511000 | 1048575 (11068575 / 10020000) -explicit @ rc (post-18g.2) | 0.285 | 42336 | 0 (11068575 / 11068575) -implicit @ rc (control) | 0.359 | 511632 | (leaks: Implicit-mode debt) -``` - -`live = 0` end-to-end. Decision 10's "RC frees promptly" property -now holds on the canonical bench fixture without exception. - -### Re-bench on the latency axis (post-18g.2) - -``` -arm | median(µs) | p99(µs) | p99/med | max(µs) | max/med -implicit @ gc (Boehm) | 104.1 | 7249.4 | 69.65× | 7923.4 | 76.13× -explicit @ rc (post-18g.2) | 227.4 | 311.6 | 1.37× | 348.2 | 1.53× -implicit @ rc (control) | 295.9 | 412.7 | 1.39× | 432.0 | 1.46× -``` - -Compared to 18f.2 (pre-18g.1): - -- RC's median tightened slightly (280 → 227 µs). The added - per-op `rc_dec` is more than offset by the cache effects of - not retaining 11M live cells. -- RC's p99 tightened (453 → 312 µs) and p99/median dropped - (1.62× → 1.37×). Boehm's tail unchanged at ~7 ms (its STW - pauses don't depend on whether the comparator is leaking). -- RC RSS dropped from 511 MB to 42 MB; explicit-mode RC is - now LOWER RSS than Boehm (42 vs 66 MB). Boehm reserves - conservatively; RC frees promptly. The "RC pays an RSS - premium" intuition is wrong on this workload. - -### Implications for the Boehm retirement decision - -The 18f throughput bench failed its 1.3× target. The 18f.2 -latency bench supported the real-time claim but had to -caveat that explicit-mode RC was leaking. Post-18g.2, both -caveats lift: - -- **Tail latency**: RC is 23× better than Boehm on p99 (312 - vs 7249 µs) and 23× better on max (348 vs 7923 µs). RC's - p99/median is 1.37× — meets even a strict real-time - determinism criterion. Boehm's 70× p99/median is the STW - pause signature. -- **RSS**: RC is now LOWER than Boehm (42 vs 66 MB) on this - fixture — promptly-freed cells beat Boehm's conservative - heap reservation. -- **Median throughput**: RC is 2.18× slower than Boehm. This - is the `ailang_rc_alloc` (libc malloc + 8-byte header init) - vs Boehm's bulk-page allocation cost. Path B (slab/pool - allocator in `runtime/rc.c`) would close most of this; Path - A (lower the throughput threshold) accepts it as the price - of bounded latency. - -Decision 10's central commitment — RC + uniqueness for real- -time-grade memory management — has its evidence base. The -remaining gap between "evidence base exists" and "Boehm -retired" is an orchestrator call, no longer a measurement -question. The two open paths from 18f's entry: - -1. **Path A**: accept that RC is throughput-slower, retire - Boehm anyway because the latency + RSS wins are decisive - and the real-time property is the canonical reason this - project exists. -2. **Path B**: build a slab/pool allocator first to close - the throughput gap, then retire. - -Both are available. Path A's "lower the threshold" framing -from 18f no longer applies — it presumed RC was on par with -Boehm; in fact RC is decisively better on the relevant axis -and merely throughput-slower. Path A is now better described -as "retire Boehm; throughput-tax is acceptable until Path B -is built later." - -The mechanical work to flip the default and drop `-lgc` is -small (a few lines in `bench/run.sh`, `crates/ail/src/main.rs`, -and a README pass). The semantic decision — does AILang -commit to the throughput-tax-for-determinism trade — is -canonical Decision 10 territory. Recorded here for the -orchestrator's review; no commit yet flips defaults. - -### Iter status / 18-arc closure - -18-arc as originally scoped: - 18a (modes), 18b (rc runtime), 18c.1–4 (clone + linearity + - uniqueness + per-type drop), 18d.1–4 (reuse-as + move-aware - patterns + scope-close drops), 18e (drop-iterative + worklist), - 18f (bench). - -Post-rename: 18f decided not to decide. 18f.2 ran the right -bench and surfaced two implementation gaps, both now closed: - - 18g.0 (rc-stats counter for diagnosing leaks) - 18g.1 (outer-cell shallow-dec at tail-call sites) - 18g.2 (let-binder drop for Own-returning App) - -The 18-arc's correctness property — "explicit-mode RC matches -explicit-mode RC's documented contracts; no leaks on the -canonical fixture" — now holds. The arc is complete. The -remaining open items are: - - - Boehm retirement (orchestrator decision, see above). - - Tag-conditional partial-drop helper (the dynamic-tag - carve-out shared by 18d.4 and 18g.2; not load-bearing - for the canonical fixture; queue for later). - - Let-alias-aware param-mode propagation (the 18d.4 Iter - A carve-out; same as above). - - Tidy-iter (the 18-arc's mandatory family-boundary - tidy; queued as #80). - -After the orchestrator's Boehm-retirement decision lands, -the tidy-iter is the right next step. - -## 2026-05-08 — 18g sub-arc tidy-iter - -`ailang-architect` ran a drift review of the 18g sub-arc -(commits `e8c6e99` through `97e793d`) and reported seven -items, prioritised. Resolved: - -### Ratified into DESIGN.md - -**Item 1: `param_modes` / `ret_mode` codegen role.** DESIGN.md -§ "Codegen contract" gained a "Mode metadata is load-bearing -for codegen (Iter 18d–18g)" subsection that lays out the four -seams that consume mode metadata: Iter B (Own-param dec at fn -return), Iter A (arm-close pattern-binder dec gated on -scrutinee mode), Iter 18g.1 (pre-tail-call shallow-dec), and -Iter 18g.2 (let-binder trackability for Own-returning App). -Also names the let-alias-of-borrow carve-out the gates do not -yet propagate through. - -The schema is unchanged (mode metadata was already on -`Type::Fn` since 18a); what's new is that codegen's drop- -emission behaviour now depends on those fields. Recording the -dependency in DESIGN.md was overdue — this entry closes that -gap. - -### Wired into the harness - -**Item 2: `bench/latency_harness.py` not in `bench/run.sh`.** -`run.sh` now invokes the latency harness on the three -canonical arms (Implicit @ gc Boehm-fair, explicit @ rc -RC-fair, Implicit @ rc control) after the throughput table. -Each invocation prints the median / p99 / p99.9 / max block -the harness already produces. The Boehm-retirement bench -numbers are now reproducible by anyone running `bench/run.sh`, -not just by hand on a specific host. - -The harness output is intentionally not folded into a single -table by the script — the per-arm block carries its own noise- -floor and read-coalescing context that the orchestrator should -see verbatim when capturing into a JOURNAL entry. - -### Negative coverage test added - -**Item 3 (partial): negative-side test for `Implicit`-ret-mode -App.** New fixture -`examples/rc_let_implicit_returning_app.ailx` and test -`alloc_rc_let_binder_for_implicit_returning_app_does_not_drop`. -The fixture is a let-binder whose value is a default-mode -(`Implicit` ret) App; the test asserts the binary exits -cleanly with `live = 1` (the cell leaks but does not crash). -This pins the asymmetry to the (own)-ret-mode test -(`live = 0`) and would fail loudly if a future iter -mistakenly widened `is_rc_heap_allocated` to all App shapes. - -The `Borrow`-ret-mode case was attempted but the typechecker -correctly rejects the only minimal repro shape (a -"borrow-passthrough" returning a borrowed view of an arg) -with `consume-while-borrowed` — meaning the language already -forbids the shape that would have been the negative test. The -gate is therefore covered by language-design constraint -rather than by a regression test, and a comment in the -fixture documents that path. - -### Carry-over (deferred, recorded as known debt) - -**Item 4: `emit_inlined_partial_drop` shallow-dec fallback -silent-leak path.** The fallback fires when an App-bound let- -binder accumulates `moved_slots` (the body pattern-matches -the binder). No fixture currently exercises that shape; if -one lands without surfacing the leak, the carve-out's debt -will compound silently. Queued as: a tag-conditional -partial-drop runtime helper that takes the dynamic ctor tag -as input and dispatches accordingly. Same family as 18d.4's -match-arm dynamic-tag carve-out; both close together. - -**Item 5: carve-outs accumulating without diagnostic -surface.** Three live carve-outs as of 18g.2 — let-aliases of -borrowed scrutinees (18d.4 fix), dynamic-tag pattern-binder -partial-drop (18d.4), dynamic-tag App-binder partial-drop -(18g.2). All three silently leak rather than diagnose. The -typechecker's `consume-while-borrowed` rule prevents the -worst class of these (e.g. the borrow-passthrough fixture -above), but the codegen-time carve-outs are not surfaced to -the user. Closing this requires a structured diagnostic -emitted from codegen when a carve-out path fires — feasible -within the existing `suggested_rewrites` framework. Queued -for the carve-out unification iter. - -**Item 6: bench numbers vs methodology.** The 18g.2 latency -table was a single-host hand-run on AMD 5900X; the harness -captures one run, not a stat-of-N. The qualitative claim -("Boehm has STW pauses, RC doesn't; RC is RSS-lower than -Boehm on this fixture") is robust at a 23× signal margin and -not at risk from run-to-run variance. The quantitative -numbers are not regression-locked. To make them so, the next -re-bench should record N runs and median + variance per cell; -the harness can be extended to do that without re-shaping the -output. Recorded as known limitation rather than fixed in -this tidy because the orchestrator's Boehm-retirement -decision (still open) does not require sub-percent precision -to resolve. - -### CLAUDE.md tidy-iter ordering - -**Item 7: Boehm retirement decision is staged ahead of the -tidy-iter in the 18g.2 entry.** CLAUDE.md "Tidy-iter at -family boundaries" requires the next iter after a family -closes to BE the tidy-iter, with explicit deferral -documented. The 18g.2 closing did not name the deferral -explicitly — implicit by the retirement-decision framing. -This tidy-iter (the entry you are reading) is in fact what -follows the 18g family, in order; the retirement decision -remains queued for the orchestrator's call. The ordering is -preserved in retrospect by this entry; future families -should not stage cross-family decisions before the tidy. - -### What this tidy ships - -- `docs/DESIGN.md` § "Mode metadata is load-bearing for - codegen" (~80 lines added). -- `bench/run.sh` post-throughput latency harness invocations. -- `examples/rc_let_implicit_returning_app.{ailx,ail.json}` + - one e2e test. -- This JOURNAL entry, which both closes the 18g sub-arc and - acknowledges the four deferred items as known debt. - -### Status of the 18-arc - -The 18-arc (a + b + c.1–4 + d.1–4 + e + f) was reported as -"complete on the correctness property" in the 18g.2 entry, -joined by sub-arc 18g (g.0 + g.1 + g.2). With this tidy -entry the family is formally closed; the next iter is the -orchestrator's call between Boehm-retirement Path A and -Path B (see 18g.2 entry for the framing). Three known debts -travel forward as queue-items, not as 18-arc loose ends. - -## 2026-05-08 — Decision: Boehm stays for now - -Orchestrator's call on the retirement question raised in the -18f / 18g.2 entries: **Boehm is NOT retired.** Both paths -remain alive on paper, but the operational stance is "keep -the dual-allocator setup — `--alloc=gc` (Boehm) stays default, -`--alloc=rc` is the validated alternative". The trigger for -re-opening the question is operational: if maintaining the -Boehm path costs significant attention (libgc upgrade pain, -runtime divergence, build complexity), the case for retirement -becomes the active queue item. Until then the asymmetry -(Boehm = default, RC = ready-to-flip) is the right one. - -Concrete consequences: - -- **No default flip.** `bench/run.sh`, `crates/ail/src/main.rs`'s - `--alloc` default, README guidance — all unchanged. -- **`-lgc` stays as a build dependency** for the gc path. The - optional-link footwork that retirement would require stays - un-attempted. -- **Decision 9 (Boehm transitional) is no longer "transitional" - in the original sense.** The Decision was framed in 2026-05-07 - as "transitional until RC is proven"; RC is now proven, but - the user's stance is to keep the Boehm path live. A - follow-up DESIGN.md edit should re-frame Decision 9 as "dual - allocator: Boehm GC for default workloads, RC for real-time- - sensitive workloads", removing the "transitional" framing - without touching the rest. Recorded as a tidy item; not - shipping in this entry. -- **Decision 10 (RC + uniqueness) holds as the canonical real- - time path.** The 18-arc's correctness invariants stand. Any - future RC iter is justified by the RC story itself, not by - retirement progress. - -The three known debts from the 18g tidy (tag-conditional -partial-drop, let-alias-aware mode propagation, bench-number -stat-of-N) remain queued. They are RC-side improvements; none -of them block the dual-allocator stance. - -## 2026-05-08 — 18g tidy follow-ups: stat-of-N + let-alias propagation - -Picked off the two smaller queued debts from the 18g tidy in -the same session. The third (tag-conditional partial-drop -helper) stays queued — substantively larger, no observable -fixture, and the carve-outs it would close all fall back to -`ailang_rc_dec` cleanly (potential leak, not crash). - -### Stat-of-N in the latency harness (commit `c2af5ad`) - -`bench/latency_harness.py` now accepts `--runs N`. Single-run -output is byte-identical for back-compat. With N≥2 the report -prints median + min..max per cell across runs; with N≥4 the -slowest run is dropped before aggregation, matching -`bench/run.sh`'s drop-slowest convention. `bench/run.sh` -invokes the harness with `--runs 5` for each of the three -latency arms. - -A 5-run smoke on the explicit-rc arm: - -``` -arm | median | p99(med) p99(range) | p99/median(med) -explicit @ rc (5-run) | 226.1 | 296.4 [288.7, 311.3] | 1.31× -``` - -The qualitative claim ("RC tail latency is 23× better than -Boehm; RC RSS is lower than Boehm") holds at this confidence -level — variance is well below the signal. The exact JOURNAL -2026-05-08 18g.2 numbers are now reproducible. - -### Let-alias-aware mode propagation (commit `2e00060`) - -Closes the carve-out shared by 18d.4 Iter A and 18g.1's -pre-tail-call seam: `(let a t (match a ...))` where `t` is a -non-Own fn-param defeated `scrutinee_is_owned`, because -`current_param_modes` only registered fn-params and the -let-binder `a` looked up as missing → default "owned" → arm- -close drop fired on pattern-binders whose underlying memory -the caller still owned. - -Fix shape: `Term::Let` lowering inspects `value`. If `value` -is `Term::Var { name: src }` and `src` is in -`current_param_modes`, the let-binder inherits that mode for -the duration of the body. Push/pop, symmetric with `locals`. -The Borrow-aliasing case is already prevented by the -typechecker's `consume-while-borrowed` rule; the Implicit-mode -case (default for unannotated params) is what the codegen -carve-out actually surfaces, and it's what the new fixture -`rc_let_alias_implicit_param.ailx` pins. - -Pre-fix on the new fixture: SIGSEGV / RC underflow on the -second iteration of `loop`. Post-fix: matches `alloc=gc` -(`0`). Kept as regression coverage. e2e count: 65 → 66. - -### Remaining queued debt - -- **Tag-conditional partial-drop helper.** Closes the two - dynamic-tag carve-outs (18d.4 match-arm shallow fallback - and 18g.2 App-binder partial-drop fallback). Substantial: - a per-type runtime helper that dispatches on the runtime - ctor tag and dec's non-moved ptr fields. No canonical - fixture currently hits the leak path; both fall back to - `ailang_rc_dec` (shallow) cleanly. Stays queued; will close - with the next iter that surfaces the leak in a benchmark or - a real workload. - -The 18-arc + the 18g sub-arc + the 18g tidy + the -post-18g-tidy follow-ups: all closed. Decision 9 has been -re-framed (commit `f10a77e`) to match the orchestrator's -dual-allocator stance. Next iter is queue-driven; nothing -load-bearing is open. - -## 2026-05-08 — codegen split: lib.rs from 5295 → 2825 lines - -User asked mid-session "wie groß ist die codebase mittlerweile? -Kannst du sie noch kontrollieren?" The honest answer was that -`crates/ailang-codegen/src/lib.rs` was the one file flagged for -navigability — 5295 lines monolithic, every codegen concern -sharing a single namespace. Decision delegated by the user -("Das musst du selbst entscheiden"). With the 18g family closed -and tests dense, the window for a mechanical move-only refactor -was as good as it gets. - -Executed as four sequential commits, full -`cargo test --workspace` between each, no behaviour change. -Each phase moved a coherent cluster into a sibling submodule -under `crates/ailang-codegen/src/`: - - - **Phase 1** (`84ba83d`): `synth.rs` (216 lines) + - `subst.rs` (319 lines). The free-function tail of lib.rs: - LLVM IR shaping helpers and the monomorphisation - substitution pipeline. - - **Phase 2** (`86989a7`): `drop.rs` (696 lines). All RC- - allocator drop work — per-type drop fns, per-let-close - drop dispatch, closure-pair drops. - - **Phase 3** (`86406ac`): `match_lower.rs` (935 lines). - `Term::Ctor`, `Term::ReuseAs`, `Term::Match` lowering. The - `lower_match` body (≈500 lines) is the single largest - method left in the project; moving it gave the biggest - navigability win. - - **Phase 4** (`bea5c92`): `lambda.rs` (447 lines). - `Term::Lam` + closure machinery (capture analysis, env - construction, deferred per-pair drop fns). - -Final layout (lib.rs is now the Emitter setup + the -`lower_term` mass dispatcher + the lookup helpers + tests): - -``` -lib.rs 2825 Emitter struct, lifecycle, lower_term - dispatcher, app/eq/effect lowering, - lookup helpers, synth_arg_type, - monomorphisation entry, tests. -match_lower.rs 935 Term::Ctor / ReuseAs / Match. -escape.rs 722 escape analysis (predates split). -drop.rs 696 per-type + per-let-close drops. -lambda.rs 447 Term::Lam + closure machinery. -subst.rs 319 substitution + unification. -synth.rs 216 IR shaping helpers + builtin types. -``` - -### Visibility model - -Rust's "private = visible to module + descendants" rule -made the split cheap. The `Emitter` struct and its private -fields stay private in `lib.rs`; submodules (drop, lambda, -match_lower) can read those fields directly through normal -descendant-module privacy. Only methods that lib.rs calls -back into the submodule needed `pub(crate)` (the entry-point -methods); helpers used only inside a submodule stayed -private. Mirroring this, a small set of lib.rs-side helpers -that submodules call back into were upgraded to -`pub(crate)`: `start_block`, `lower_term`, `fresh_ssa`, -`fresh_id`, `synth_arg_type`, `collect_owner_local_types`, -`lookup_ctor_by_type`, `lookup_ctor_in_pattern`. No fields -needed `pub(crate)`. - -### What was deliberately not done - -- **No behaviour change.** Not a single character of - generated IR is different. The split is purely a - reorganisation. (Verified by 66/66 e2e tests passing - byte-identically across all four commits.) -- **No method signatures changed.** Visibility upgrades only - (`fn` → `pub(crate) fn`) on helpers crossed by the new - module boundary. No parameter or return-type changes. -- **No further extraction.** `lower_term` is 395 lines of - pure dispatch into other methods; splitting its arms into - per-shape files would force every shape's dispatcher to - re-enter through a `pub(crate)` boundary and would split - the term-shape ↔ lowered-shape correspondence across - files. Kept whole. Same call for the - `lower_app`/`lower_polymorphic_call`/`emit_call`/ - `emit_indirect_call` cluster — they are too tightly - coupled to the `Emitter`'s dispatch state to extract - cleanly. - -### What this enables - -The biggest concrete win is for `ailang-architect` and -debugging conversations: a "what does the drop emission do?" -question now reads one 696-line file instead of grepping -through five thousand lines. Same for the closure machinery, -the match lowering, and the substitution pipeline. Future -iters that touch these subsystems get a smaller working -context. - -The split is not motivated by a coming feature, and no -queued iter required it. It is a tidy paid for navigability -alone — the kind of work that compounds across future -sessions without showing up in any single one's bench -numbers. - -## 2026-05-08 — Iter 18g.tidy.fu2: tag-conditional partial-drop helper - -User feedback closed an open question. The 18g tidy entry had -queued the tag-conditional partial-drop helper as "stays queued; -will close with the next iter that surfaces the leak in a -benchmark or a real workload." The user pushed back: *"Du weisst, -dass es einen Bug gibt — Leaks sind Bugs."* CLAUDE.md's TDD-for- -bug-fixes rule is unambiguous on a known bug: write a RED test, -ship the GREEN, keep as regression. The "wait for organic -fixture" framing was avoidance disguised as discipline. Fixed. - -### Three carve-outs, three sites, one helper - -The pre-fu2 codebase had three sibling sites that fell back to -shallow `ailang_rc_dec` when a binder had a non-empty -`moved_slots` *and* a dynamic runtime ctor tag: - - 1. `lib.rs` Iter B Own-param dec at fn-return (`emit_fn`'s - pre-ret seam). - 2. `match_lower.rs` Iter A arm-close pattern-binder dec - (outer-arm close when an arm-bound binder was scrutinised - by an inner match that moved out fields). - 3. `drop.rs::emit_inlined_partial_drop` non-Ctor branch - (`Term::Let` whose value is a `Term::App` Own-returning - call, body pattern-matches the binder). - -All three share a structural shape: a binder whose static type is -known (an ADT), but whose runtime ctor tag is *not* statically -recoverable from the AST node at the drop emission site. The -existing per-type drop fn `drop__(ptr)` is built around -"emit the dec for *every* ptr field of the active ctor"; what was -missing was the variant "emit the dec for every ptr field of the -active ctor *whose slot index is not in `moved`*." - -### `partial_drop__(ptr %p, i64 %mask)` - -The fu2 helper (`drop.rs::emit_partial_drop_fn_for_type`) is a -straight parallel to `emit_drop_fn_for_type`: - -```text - define void @partial_drop__(ptr %p, i64 %mask) { - %is_null = icmp eq ptr %p, null - br i1 %is_null, label %ret, label %live - live: - %tag = load i64, ptr %p - switch i64 %tag, label %dflt [...] - arm_i: - ; for each ptr field at slot j of ctor i: - %b = and i64 %mask, (1 << j) - %s = icmp ne i64 %b, 0 - br i1 %s, label %after, label %do - do: - %v = load ptr, gep %p, (8 + 8*j) - call void @(ptr %v) - br label %after - after: - ; next ptr field, or br label %join - dflt: unreachable - join: - call void @ailang_rc_dec(ptr %p) - br label %ret - ret: - ret void - } -``` - -One helper per ADT in the module, emitted alongside -`drop__`. We do *not* emit a `(drop-iterative)` partial-drop -variant: the helper runs once on the binder (carve-out sites are -not cascade points — the unmoved fields go through their own -`drop__` which itself decides recursive vs. iterative). - -The mask is `i64`, so ADTs with > 64 fields fall back to shallow -dec via the `build_moved_mask` cap. No language-level ADT has 64 -fields; the cap is load-bearing only as a defensive guard. - -### Three RED-then-GREEN fixtures - -Built before the helper to lock the carve-outs in regression -coverage. All three share the same `Wrap` / `Cell` / `Pair` shape -(Pair has 2 Cell slots, Cell has 2 Wrap slots) so the leak path -is observable through `AILANG_RC_STATS=1`'s -`allocs/frees/live` triple. - - - `examples/rc_own_param_partial_drop_leak.{ailx,ail.json}` — - site 1. `(fn use_first (params (own (con Pair))) ...)` matches - `p` as `MkPair(a, _)`. Pre-fu2: `live=3` (whole second Cell + - two Wraps leak through Iter B's shallow path). Post-fu2: - `live=0`. - - `examples/rc_match_arm_partial_drop_leak.{ailx,ail.json}` — - site 2. Outer match binds `a, b` from `p`; inner match - destructures `a` as `MkCell(w1, _)` (slot 1 wildcarded). - Pre-fu2: `live=1` (slot 1's MkWrap leaks through Iter A's - shallow path). Post-fu2: `live=0`. - - `examples/rc_app_let_partial_drop_leak.{ailx,ail.json}` — - site 3. `(let p (app build_pair 1) (match p ...))` with - `MkPair(a, _)`. Pre-fu2: `live=3` (slot 1's whole Cell + - Wraps leak through `emit_inlined_partial_drop`'s non-Ctor - fallback). Post-fu2: `live=0`. - -Each fixture has a corresponding e2e test in -`crates/ail/tests/e2e.rs`. e2e count: 66 → 69. - -### Routing the call sites - -Each of the three sites was rewritten symmetrically: resolve the -`partial_drop__` symbol from the binder's static type -(`partial_drop_symbol_for_type`), build the mask -(`build_moved_mask`), call. Falls back to the prior shallow -`ailang_rc_dec` only if the static type is non-ADT (Str, fn-typed, -type-var) — those shapes can't populate `moved_slots` in -practice, so the fallback is dead under the typechecker. - -The pre-existing test `alloc_rc_own_param_dec_at_fn_return` -asserted on the IR-level shape "either `drop__` or -`ailang_rc_dec` is called on `%arg_xs` before the ret"; widened -to also accept `partial_drop__(%arg_xs, i64 mask)` (the -canonical fu2 shape — the fixture's Cons arm moves slot 1 into -`t`, so `moved_slots[xs]={1}`). - -### What stays queued - -Carve-out diagnostic surface (item 5 from the 18g tidy entry): -the three carve-outs no longer leak, but they also don't surface -to the user when they fire. Closing this is a separate ergonomics -iter (structured diagnostics from codegen), not a correctness -fix. The fu2 helper makes the codegen-side leak path empty; -diagnostic surface is for the language-design layer. - -### Lesson recorded - -The "wait for organic fixture" stance from the 18g tidy entry is -now retracted: known leaks are bugs, and CLAUDE.md's TDD rule -covers them autonomously. Constructing a minimal fixture for a -known leak path is *not* "constructing fixtures to drive an -implementation" — it is the literal RED-step that the discipline -prescribes. The three fixtures shipped here are exactly that: -each pins one of the three carve-out sites, observable via the -RC stats line, kept as regression. JOURNAL queue: empty. - -## 2026-05-08 — Design: explicit annotations stay mandatory; `over-strict-mode` diagnostic - -User pushback on the queue-driven dispatch: the orchestrator -floated "uniqueness inference replaces fn-signature mode -annotations" as a candidate direction. Re-read of DESIGN.md -§ "Decision 10" (lines 685–795) shows that pitch was a -**Decision-10 reversal**, not an iter within it: - -> *"AILang makes it mandatory because the LLM author can carry -> the cognitive cost trivially, and the compiler gains a precise -> contract at every call site instead of a probabilistic guess."* - -Three reasons mandatory annotations are not redundancy in the -harmful sense: - - 1. **Inference picks weakest-supporting; annotation states - intent.** These often coincide today but are conceptually - different. The annotation captures what the author committed - to, not what the current body needs (e.g. `(own T)` reserved - for a planned in-place mutation that hasn't landed). - 2. **Annotation is a drift-bremse.** Body change that flips - the inferred mode → caller-side breakage at remote sites. - Annotation enforces the local-conflict-error pattern instead. - 3. **Forcing function specific to LLM authoring.** Without - mandatory annotation, an LLM never has to commit to ownership - intent before writing the body — the contract becomes a - by-product of local code choice. With it, the contract is a - first-order variable the body must satisfy. - -The "redundancy" between annotation and body is therefore the -feature, not the bug — analogous to test code redundantly -restating implementation behaviour, where the redundancy is what -catches drift. - -### What survives of the original pitch - -One narrow but real observation: today, when the author writes -`(own T)` and the body would compile under `(borrow T)`, the -typechecker silently accepts. No warning, no diagnostic. The -runtime cost is real (one inc/dec pair per call, plus potential -rec-cascade at fn return), but invisible. - -The Rust analogue is `clippy::needless_pass_by_value`: an -informational lint with a suppression mechanism. AILang sharpens -this to "suppression carries a mandatory reason" — consistent -with the explicit-intent philosophy. - -### Iter scope - -Two iters, dispatched separately so 19a's diagnostic frequency -on real code informs whether 19b's escape hatch is needed at all: - - - **Iter 19a — `over-strict-mode` diagnostic.** Linearity pass - detects: `param_modes[i] == Own` + `consume_count(p) == 0` + - no consume of any sub-binder of p in the body. Emits - `Severity::Warning` (the first warning-level diagnostic the - typechecker has emitted; reserves no longer reserved). Carries - a `suggested_rewrite` showing the relaxed `(borrow T)` - signature. No schema change; pure detection. - - - **Iter 19b — `mode-strict-because` suppression** (deferred - pending 19a's signal). Optional schema field on fn defs: - `suppress: [{code: String, because: String}]`. `because` is - non-empty (schema-validation error otherwise). The suppress - mechanism is generic across diagnostic codes but the only - consumer initially is `over-strict-mode`. - -### Why split - -19a alone is shippable: users who see the warning either accept -the rewrite or live with the noise on intentional-strict fns. -19b is only worth building if real code surfaces enough -intentional-strict cases to make the noise unbearable. This lets -the data drive whether 19b ships at all. - -### What this is NOT - -Not a relaxation of Decision 10. Annotations stay mandatory. -Authors cannot omit `param_modes` / `ret_mode`; the compiler -will not infer them. The diagnostic is purely advisory: "you -wrote this, here's a tighter alternative." If suppressed (19b), -the annotation stays exactly as authored. - -## 2026-05-08 — Iter 19a: `over-strict-mode` diagnostic shipped - -The diagnostic-side of the design entry above. Linearity pass -gained a post-walk loop over `Own`-annotated params; uniqueness -side-table provides `consume_count`. When `consume_count == 0` -*and* the body never destructures the param via match, the lint -fires with a form-A-rendered fn-type rewrite as `suggested_rewrite`. - -### Conservative cut - -The precise rule is "no consume of any sub-binder of `p`". The -shipped check approximates with "no `match` whose scrutinee is -`pname`" — sound (never false-positive) but incomplete (misses -match-on-p-without-sub-consume). Documented in `linearity.rs`'s -module doc and pinned by the test -`over_strict_mode_conservative_skips_match_on_param`. Acceptable -because the lint is purely advisory: a missed warning costs -nothing, a spurious warning would actively mislead. The precise -variant is queued for a follow-up if real-corpus signal warrants. - -### CLI side-effect - -This is the first `Severity::Warning` diagnostic the typechecker -emits. Three CLI exit paths (`Cmd::Check` non-JSON, `Cmd::EmitIr`, -`build_to`) previously aborted on any non-empty diagnostic list; -now they only abort when at least one diagnostic is `Severity::Error`. -The `--json` path was already correct (returns the list, doesn't -exit). No observable behaviour change for any pre-19a input — -nothing emitted Warning before this iter — but the gate is now -in place for future warning-level lints. - -### Tests - -Four new tests in `linearity.rs::tests`: - - - `over_strict_mode_fires_when_param_only_borrowed` — positive. - - `over_strict_mode_silent_when_body_consumes_param` — negative. - - `over_strict_mode_silent_when_param_is_borrow` — negative. - - `over_strict_mode_conservative_skips_match_on_param` — pins the - conservatism; flips to a positive assertion when the precise - variant lands. - -`ailang-check` test count: 49 → 53. e2e unchanged at 69. Workspace -build + test green. - -### Files touched - - - `crates/ailang-check/src/diagnostic.rs` — registered - `over-strict-mode` code, refreshed `Severity::Warning` doc, - added `Diagnostic::warning` ctor. - - `crates/ailang-check/src/linearity.rs` — module-doc § 19a, the - post-walk loop, helpers (`body_matches_on`, - `scrutinee_matches_param`, `make_over_strict_mode`, - `relax_param_to_borrow`), `check_fn` signature gained a - `&UniquenessTable` arg. - - `crates/ailang-surface/src/print.rs` — new - `pub fn type_to_form_a(&Type) -> String` (re-exported from `lib.rs`). - - `crates/ail/src/main.rs` — three exit-on-Error gates. - -### What stays queued - - - Iter 19b (`mode-strict-because` suppression): waits for - real-corpus signal on whether intentional-strict fns are - common enough to warrant the schema field. - - Precise sub-binder analysis for the lint: waits for evidence - that the conservative cut misses real cases. - -## 2026-05-08 — Pinned: human-readable prose surface (Family 20 candidate) - -User feature-request, pinned for autonomous design pass: - -> *"AILang ist für Menschen tatsächlich ziemlich unleserlich. Ich -> hätte gern einen Menschen-lesbaren Text-Output, der semantisch -> exakt identisch zu ail ist, aber von Menschen gut gelesen werden -> kann. Dabei sind Formatierung, Inlining, klare Identifier (keine -> Klammerhölle!), vllt. Infix-Notation, etc. wichtig. Und eventuell -> das Weglassen von Spezifika, die nur der Sprachstringenz dienen. -> Der Zweck ist: 1. Menschen sollen den Code schneller begreifen. -> 2. Sie sollen den Code durch FREITEXT editieren können. Der -> editierte Code und der Original-Code gehen dann wieder an das -> LLM, das dann in AIL die Freitext-Ergänzungen einbaut. Damit -> beginnt der Zyklus von vorne."* - -### Read of the request - -Two artefacts are wanted: - - 1. **A renderer.** `Module → human-prose String`. Indented, - infix-flavoured, parens dropped where unambiguous, Lispy - `(con T)` / `(term-ctor ...)` machinery suppressed in favour - of conventional notation (`T`, `Cons(1, Nil)`). - 2. **A round-trip mediator.** Original .ail.json + edited prose - → updated .ail.json. The mediator is the LLM, not the - compiler. Renderer is deterministic; round-trip is not (LLM - interprets free-text intent). - -The user's word "semantisch exakt identisch" applies to the -**renderer's output as a projection of the source**: re-reading -the prose alongside the .ail.json gives a human all the semantic -information needed to reason about the program. The user's -"Weglassen von Spezifika, die nur der Sprachstringenz dienen" -explicitly authorises *lossy* projection — the prose may omit -machinery the LLM can re-derive (e.g. `consume_count` is already -inferred, so the prose definitely doesn't show it; redundant -parens can go; `(con Int)` becomes `Int`). The full .ail.json -remains the canonical source. - -### What MUST stay visible in prose - -Anything semantically load-bearing that the LLM cannot trivially -re-derive from prose alone: - - - `(own T)` / `(borrow T)` mode annotations on fn signatures — - these are contracts (Decision 10), not buchhaltung. Render as - `own T` / `borrow T` or a sigil (`&T` / `~T`?). To be decided. - - Effect annotations (`IO`, `Diverge`) on user fns. - - Explicit `clone` calls — they're an author commitment, not a - free choice. - - Doc strings. - -### What MAY go - - - Outer `(module …)` wrap (filename or top-line is enough). - - `(con T)` wrapping around base types — render as `T`. - - `(term-ctor T C f1 f2)` — render as `C(f1, f2)` (or just - `Nil` for nullary). - - Redundant parentheses around expressions whose precedence is - unambiguous. - - Fully explicit `(fn-type (params ...) (ret ...))` — - render as `(p1: T1, p2: T2) -> R`. - - `(let x v body)` — render as `let x = v` (newline) `body` or - `x := v; body`. - - Non-essential schema-rigour fields (e.g. `consume_count` if - the AST ever exposed it; today it's already inferred so this - is a forward-proofing note). - - The implicit `(do io/print_int x)` ceremony when `print` would - do. - -### Round-trip cycle - -For the editor cycle: - -``` -.ail.json ──[render]──→ .prose.txt - ↓ user edits freely - .prose.txt' (edited) - ↓ -.ail.json + .prose.txt' ──[LLM mediator]──→ .ail.json' -``` - -The LLM mediator is the contract-enforcer here: it integrates -free-text edits while preserving annotations the prose may have -omitted (e.g. carries forward the original `(own T)` annotation -unless the user's edit obviously contradicts it). This is *not* -a compiler pass — the renderer ships first, the mediator is -a separate piece (probably a `prompt_template` + a calling -convention, not a Rust crate). - -### Iter scope - -Probably a small family. First cut design: - - - **Iter 20a — renderer skeleton.** New crate `ailang-prose` - with `pub fn module_to_prose(&Module) -> String`. Covers the - full AST with a conservative formatting policy (no infix yet, - no inlining, just kill the `(con ...)` / `(term-ctor ...)` - noise and use proper indentation + commas). One snapshot test - per fixture in `examples/`. - - - **Iter 20b — formatting polish.** Infix for arithmetic - (`(i64-add a b)` → `a + b`), drop redundant parens, inline - short let-bindings, prettier match arms. Snapshot tests - update; behavioural surface unchanged. - - - **Iter 20c — `ail prose` CLI subcommand.** `ail prose ` - prints the prose to stdout. Symmetric to `ail parse`'s role. - - - **Iter 20d — round-trip ergonomics.** This is where the LLM - mediator's input format gets specified. May be a documented - prompt template under `docs/`, may grow into a `ail merge-prose` - subcommand that wraps an LLM call. Defer until 20a–c are real - code we're using. - -### Why this is its own family - -Decision 10 / Family 18 was the memory model. Family 20 is a -**presentation layer** — it touches no semantics, no codegen, no -typechecker. Lives entirely above `ailang-core` as a sibling of -`ailang-surface` (which is the form-A *parser* / printer; prose -is the form-B *projection*). - -### What's deferred / open questions for design pass - - - How to render `(own T)` / `(borrow T)` in prose. Sigil vs. - keyword. Sigils are denser; keywords align with the - explicit-intent philosophy. Lean one way and document - rationale. - - Module headers / imports — render or suppress? - - Whether prose carries hash/version annotations for the - round-trip (so the mediator can detect drift between prose - and the .ail.json the user *thinks* they're editing). - - Whether snapshot tests live in `crates/ailang-prose/tests/` - or `examples/` (as `.prose.txt` siblings to the .ail.json). - - -## 2026-05-08 — Iter 20a: prose renderer skeleton shipped - -The renderer-side of the family-20 design pinning. New crate -`ailang-prose` with one public fn `module_to_prose(&Module) -> String`, -plus a `ail prose ` CLI subcommand that prints the -projection to stdout. - -### Style commitments (orchestrator-fixed for 20a, not up to renderer) - -- Rust-flavour with braces and `=>` for match arms. -- Mode keywords `own T` / `borrow T` (not sigils — Decision-10 consistency). -- Effects trailing the return type: `with IO`. -- Constructor application as `Cons(1, Nil)` (no `(term-ctor ...)` wrap). -- Types as bare names: `Int`, `List` (no `(con ...)` wrap). -- Doc strings as `///` lines above the def. -- `tail` flag on calls renders as a `tail ` prefix. -- All other load-bearing semantic detail stays visible: `clone`, - `reuse-as`, effects, doc strings, type annotations on signatures - and lambdas. -- No infix yet; arithmetic primitives stay as `i64-add(a, b)`. - That's 20b. - -### Snapshot coverage - -Three fixtures got `.prose.txt` siblings, asserted by -`crates/ailang-prose/tests/snapshot.rs`: - - - `examples/rc_own_param_drop.prose.txt` - - `examples/rc_match_arm_partial_drop_leak.prose.txt` - - `examples/rc_app_let_partial_drop_leak.prose.txt` - -Sample (`rc_own_param_drop.prose.txt`): - -``` -fn head_or_zero(xs: own IntList) -> Int { - match xs { - Nil => 0, - Cons(h, t) => h - } -} -``` - -vs. the 215-line `.ail.json` it projects from. - -The `head_or_zero` body is exactly the kind of thing the user -named: dramatically less syntactic noise, immediately legible to -a human, with `own` mode and `Cons(h, t)` as conventional source -forms. - -### Visible nits queued for 20b - -- Long doc strings stay on one wrap-less line. -- `do io/print_int(x)` could be `print(x)` for the IO/Int-print - default. -- Nested match arms could break across lines. -- Arithmetic primitives are still prefix. -- No precedence-aware paren elision yet. - -All explicitly out of 20a's scope; pinned for 20b. - -### Files - -- New crate `crates/ailang-prose/` (`Cargo.toml`, `src/lib.rs` - ~924 lines, `tests/snapshot.rs` 64 lines). -- `Cargo.toml` (root) + `crates/ail/Cargo.toml`: workspace + dep wiring. -- `crates/ail/src/main.rs`: `Cmd::Prose { path }` variant + dispatch. -- Three new `examples/*.prose.txt` snapshots. - -### Build/test status - -- `cargo build --workspace` — green, no warnings. -- `cargo test --workspace` — 28 prose-unit + 3 prose-snapshot pass; - existing 69 e2e + 53 ailang-check + others all unchanged-green. - -### What stays queued - -- **Iter 20b** — formatting polish: infix for arithmetic, paren - elision by precedence, let-inlining, prettier `do`, line-wrap - for long doc strings. -- **Iter 20c** — the CLI subcommand was bundled into 20a (one - natural unit). Originally pinned as a separate iter; collapsed. -- **Iter 20d** — the round-trip mediator (prose-edit → updated - AIL via LLM call). Specification + prompt template, not a - compiler pass. - -## 2026-05-08 — Iter 20b: prose formatting polish shipped - -The polish pass on top of 20a's renderer skeleton. Four polishes, -no public-API change. - -### Polishes - - 1. **Infix binary operators.** Eleven canonical builtins - (`+ - * / % == != < <= > >=`) when called via - `Term::App { args.len() == 2, tail: false }` render as - `lhs op rhs`. Tail-flagged binary ops keep prefix form so the - `tail` keyword stays visible. - 2. **Paren elision by precedence.** Standard 4-level Rust-aligned - table: `* / %` (mul) > `+ -` (add) > `< <= > >=` (cmp, - non-assoc) > `== !=` (eq, non-assoc). Atomic terms (Var, - Lit, Ctor, App-non-binary, etc.) bind tightest. Same-level - left-associative left side: no parens; same-level right - side: keeps parens. Non-assoc ops always keep parens at - same level. - 3. **Unary `not`.** `App { callee: Var "not", args: [x] }` - renders as `!x`. Atomic operand binds at level 5 (tightest); - `!(a == b)` keeps parens because the operand is a level-1 - binary op. - 4. **Long doc-string wrap.** `///` lines exceeding 80 columns - wrap at word boundaries. Explicit newlines split first, then - each piece word-wraps independently. - 5. **Nested-match formatting** (already structurally correct in - 20a; locked in by a 3-deep test). - -### Snapshot impressions - -`bench_list_sum.prose.txt` after 20b: - -``` -fn cons_n_acc(n: Int, acc: IntList) -> IntList { - if n == 0 { - acc - } else { - tail cons_n_acc(n - 1, ICons(n - 1, acc)) - } -} -``` - -vs. pre-20b `if ==(n, 0) { ... -(n, 1) ... ICons(-(n, 1), acc) }`. -The infix conversion is the headline win — arithmetic now reads -as arithmetic. - -`rc_own_param_drop.prose.txt` doc string now wraps: - -``` -/// Take ownership of an IntList; return its head if Cons, else 0. The tail is -/// loaded as a pattern binder but never consumed — iter A dec's it at arm -/// close. The outer cell is dec'd at fn return via iter B's Own-param emission. -``` - -### Tests - -Test count went from 28 → 47 (19 new unit tests in `lib.rs`), -plus a fourth snapshot (`examples/bench_list_sum.prose.txt`) that -exercises infix on a real benchmark fixture. Existing snapshots -re-rendered to incorporate the wrapping. - -### Files - - - `crates/ailang-prose/src/lib.rs` — `write_doc` rewrite, - `wrap_words` helper, `PREC_*` constants, `binop_info`, - `as_binop`, `as_unary_not` helpers, `write_term_prec` - threading `parent_prec` through term recursion. - - `crates/ailang-prose/tests/snapshot.rs` — new - `snapshot_bench_list_sum`. - - `examples/bench_list_sum.prose.txt` — new. - - `examples/rc_own_param_drop.prose.txt` — re-rendered. - -### Build/test status - -- `cargo build --workspace` — green, no warnings. -- `cargo test --workspace` — 47 prose-unit + 4 prose-snapshot - pass; everything else unchanged-green. - -### What stays queued - -- **Iter 20d** — round-trip mediator. Specification + prompt - template (the LLM bundles original .ail.json + edited prose - and emits the updated .ail.json). Likely a `ail merge-prose` - subcommand that prints a shaped prompt, the user mediates - through their LLM, then `ail parse` (or `ail check`) on the - re-emitted artefact. Possibly: skip the CLI shim, ship as a - documented prompt template under `docs/`. -- **Let-inlining** — questionable polish; deferred until the - corpus shows it's needed. -- **`do io/print_int(x)` → `print(x)`** — needs type context - to be safe; deferred. - -## 2026-05-08 — Iter 20d: prose round-trip mediator shipped - -The closing piece of family 20: a documented prompt template and a -thin CLI helper that compose the prompt for the prose-edit cycle. No -LLM client of our own — the user pipes the prompt to whichever model -they prefer. - -### The cycle - -``` -1. ail prose foo.ail.json > foo.prose.txt -2. $EDITOR foo.prose.txt # human edits freely -3. ail merge-prose foo.ail.json foo.prose.txt > prompt.txt -4. cat prompt.txt | > foo.new.ail.json -5. ail check foo.new.ail.json -6. mv foo.new.ail.json foo.ail.json # if check is clean -``` - -Step 1 was 20a/20b. Step 3 is this iter. Steps 5/6 are existing -`ail check` + plain `mv`. The mediator is the LLM, not the -compiler — which is why the cycle is iterative (re-run step 4 with -the diagnostic pasted in if `ail check` rejects the result). - -### What was built - - - **`docs/PROSE_ROUNDTRIP.md`** — new top-level doc, 153 lines. - Sections: *Why* (prose is lossy, re-integration needs LLM - mediation), *The cycle* (the 7-step pipeline above), *The - prompt template* (the literal text the CLI emits, so a human - can compose by hand), *Failure modes* (markdown fences, - invalid JSON, schema-valid-but-`ail check`-fails — fix in each - case is to paste the corrective note and re-prompt), *Why no - built-in API client* (deliberate scope: AILang stays a - compiler, the user already has clients). - - **`Cmd::MergeProse { original, edited }`** in - `crates/ail/src/main.rs`. Reads both files, calls the helper, - prints to stdout. Errors via `anyhow` context on file-read - failures. - - **`fn compose_merge_prose_prompt(orig_ail_json: &str, - edited_prose: &str) -> String`** — pure helper in `main.rs`. - Both payloads insert verbatim between heredoc-style markers - (`<< 0`, - `p` must be `Own` (sub-consume forces ownership). Lint stays - silent. - - Otherwise, lint fires. - -### The heap-type filter (design call by implementer) - -The orchestrator's literal brief said "any pattern-binder -`consume_count > 0` → silent." That contradicted the brief's own -positive test (`head_or_zero` MUST fire), because `match xs { -Cons(h, t) => h }` records `consume_count(h) == 1` regardless of -`h: Int` being a primitive. Implementer caught this, made the -design call, documented it in the module-doc rationale. - -The semantic justification: reading a primitive (`h: Int`) is a -load-by-value, no RC traffic, no heap data moved out of the -scrutinee's allocation. Reading a heap-typed sub-binder -(`t: IntList`) IS a heap-pointer move that requires owning the -outer cell. So the filter — "skip primitive-typed pattern-binders -when judging sub-consume" — is exactly what makes the lint -correct. - -This is a real design point that 19a's brief did not document. -Recording it here as a ratification of the implementer's call. - -### Corpus signal after upgrade - -Five of 65 fixtures now produce at least one `over-strict-mode` -warning: - - - `rc_app_let_partial_drop_leak` - - `rc_drop_iterative_long_list` - - `rc_match_arm_partial_drop_leak` - - `rc_own_param_drop` - - `rc_own_param_partial_drop_leak` - -All five are RC codegen-test fixtures — the `(own T)` annotation -is intentional, used to exercise the codegen path that drops Own- -params at fn return / arm close. They are NOT incorrect annotations; -they are deliberate test infrastructure. - -This is the signal that **justifies 19b** (`mode-strict-because` -suppression). Without it, every RC-codegen-test fixture would -permanently emit a warning despite being correct-by-design. With -it, each fixture annotates its strictness intentionally, the -warning suppresses, future-LLM reads the reason, future-iter -edits know whether to preserve or relax. - -19b is dispatched next. - -### Known debt - -**Deeply-nested-match-on-sub-binder** (recorded by implementer in -the module doc): `match p { Ctor(_, t) => match t { Ctor(_, t2) -=> consume(t2) } }` would produce a *spurious* warning under the -current rule — `t.consume_count == 0` (matching is Borrow), so -the inner consume of `t2` doesn't propagate up to `p`'s view. -None of the 65 fixtures hit this shape; queued for follow-up if -real corpus surfaces it. - -### Files / tests - - - `crates/ailang-check/src/linearity.rs` — module-doc § 19a.1 - rationale, `is_heap_type` helper, `any_sub_binder_consumed_for` - + `pattern_has_consumed_heap_binder` walkers, ctor-table - threading through `check_module` / `check_fn`. - - Tests: `over_strict_mode_conservative_skips_match_on_param` - flipped to positive `over_strict_mode_fires_when_match_arm_uses_no_sub_binder`. - New `over_strict_mode_silent_when_match_arm_consumes_sub_binder` - and `over_strict_mode_fires_when_match_arm_binders_unused`. - -`ailang-check` test count: 53 → 55. Workspace build/test green. - -## 2026-05-08 — Iter 19b: `mode-strict-because` suppression shipped - -The closer to the 19a/19a.1 arc. Corpus signal from 19a.1 -(5/65 fixtures fired `over-strict-mode`, all deliberate RC -codegen-test fixtures) justified shipping the suppress mechanism -end-to-end. - -### Schema - -`Suppress { code, because }` struct in `ailang-core::ast`. New -`FnDef::suppress: Vec` field with -`skip_serializing_if = "Vec::is_empty"` so pre-19b fixtures keep -bit-identical canonical-JSON hashes (regression-pinned by the -existing hash-stability test, plus 2 new ones). - -### Typechecker - -New `suppress_filter` module in `ailang-check`. Per-module -post-pass: - - - For each `Def::Fn` with non-empty `suppress`, drop diagnostics - matching `(def, code)` from the accumulated list. - - Emit `empty-suppress-reason` (Error severity) for any - suppress entry whose `because` is whitespace-only. - - Wrong codes (e.g. suppressing `type-mismatch` when only - `over-strict-mode` would fire) are silent no-ops — open-set - registry rationale documented. - -### Form-A surface - -Grammar: `(suppress (code "...") (because "..."))`, between fn -name and `(type ...)`. Multiple clauses allowed. Round-trip test -pinning (parse → print → re-parse → canonical-byte equality) holds -on all fixtures including the 5 RC ones that gained suppress. - -### Form-B prose - -Renders one `// @suppress : ` line per entry, ABOVE -the doc string. Lossless — contract metadata, not stringency -machinery; the LLM-reader of prose needs to see *why* an -annotation that looks over-strict is correct on this def. - -Concrete shape from `rc_own_param_drop.prose.txt`: - -``` -// @suppress over-strict-mode: RC codegen test: exercises Iter B Own-param dec at fn return -/// Take ownership of an IntList; return its head if Cons, else 0... -fn head_or_zero(xs: own IntList) -> Int { - ... -} -``` - -### Fixture migration - -Five `.ail.json` files gained suppress entries; `.ailx` siblings -regenerated via `ail render`; three of the four pinned `.prose.txt` -snapshots regenerated (the fourth, `bench_list_sum`, has no Own -params and is unchanged). The migration documented per-fixture -reason text matches the codegen-test path each fixture exercises. - -### Corpus signal after migration - -`over-strict-mode` warnings across all 65 fixtures: **5 → 0**. All -five RC codegen-test fixtures now check clean while preserving -their `(own T)` annotation as deliberate test infrastructure. The -lint still fires on any future fn that's accidentally over-strict -without an authored reason. - -### Test counts - -- `ailang-check`: 55 → 61 (6 new `suppress_filter` tests) -- `ailang-core`: 26 → 28 (2 hash-stability tests) -- `ailang-surface`: 21 → 26 (5 parse tests) -- `ailang-prose`: 49 → 52 (3 prose-render tests) -- `e2e`: 70 (unchanged) - -### Known debt - - - **`.ailx` comment headers lost.** The five regenerated `.ailx` - files lost their hand-written comment headers — `ail render`'s - contract excludes comment preservation. If those headers are - needed back, that's a separate iter (probably a comment- - preserving printer mode). - - **Duplicate-attr detection.** `parse_suppress_attr` accepts - `(code …)` and `(because …)` in either order but does not - detect duplicates within a single `(suppress …)` clause; a - second `(code …)` silently overwrites. The canonical printer - emits in fixed order, so the only way to construct duplicates - is hand-writing weird input — bounded. - - **Cross-iter snapshot coupling** (architect's deferred item 3 - from 20e): adding suppress to RC fixtures invalidated three - snapshots. They were already pinned by family 20; this iter - re-rendered them. Same pattern will recur on any future iter - that touches those fixtures. - -### Family / arc state - -The 19a/19a.1/19b arc is now closed: - - 19a: `over-strict-mode` lint + Severity::Warning surface - - 19a.1: precise sub-binder analysis (heap-type filter) - - 19b: `mode-strict-because` suppression + 5-fixture migration - -JOURNAL queue: empty again. Three pre-existing 20b deferrals -(let-inlining, `print` sugar, deeply-nested-match-on-sub-binder -in 19a.1) remain queued; all three need real-corpus signal that -hasn't surfaced. - -## 2026-05-08 — 19a-arc tidy-iter (DESIGN.md ratification) - -Per CLAUDE.md "Tidy-iter at family boundaries" — the 19a-arc -(19a + 19a.1 + 19b) closes with a tidy. `ailang-architect` ran -the drift review and reported the canonical "DESIGN.md silent" -finding, identical in shape to the one family-20's tidy-iter -(20e) caught: a substantial new mechanism shipped with no -ratification in the canonical spec. - -### Architect findings - - 1. **DESIGN.md silent on the entire 19a-arc.** Zero hits for - `suppress` / `over-strict-mode` / `empty-suppress-reason` / - `Severity::Warning`. Schema-additions block didn't list - `FnDef.suppress`. Highest-leverage fix. - 2. **Codegen carries `suppress: vec![]` in 7 synthetic FnDef - sites** (`crates/ailang-codegen/src/lib.rs` + lift / desugar - / reuse_shape / uniqueness / pretty). Mechanically correct; - a `FnDef::default()` or `synthetic` constructor would absorb - the fan-out. Bounded; observation, not blocker. - 3. **Snapshot-fixture coupling ratified by recurrence.** 19b - regenerated three pinned `.prose.txt` snapshots — exactly - the cross-family-coupling pattern 20e's item 3 named. - Promote to known structural cost, not a fix. - -### Resolved this iter - -**Item 1 — ratified.** Two edits to `docs/DESIGN.md`: - - - **Schema additions** subsection (Decision 10) gained an - "Iter 19b — `FnDef.suppress`" entry: schema shape, form-A - surface, form-B render, the `skip_serializing_if` / - bit-identity invariant, and the regression-pinning tests. - - **New subsection "Advisory diagnostics — Iter 19a-arc"** - placed between "Schema additions" and "Inference - algorithm". Documents the `over-strict-mode` lint rule - (incl. the heap-type filter rationale), the - `Severity::Warning` introduction, the CLI exit-on-Error - gating, and the `mode-strict-because` suppression with - mandatory-reason. - - **New subsection "Why advisory + suppress instead of - inference"** documents the three substantive reasons that - the user surfaced earlier in the design conversation — - annotation-states-intent vs. inference-picks-weakest, the - drift-bremse role, and the LLM-authoring forcing function. - This pins the rationale so future iters can't reopen the - decision without engaging with the recorded reasons. - - **Migration plan** gained a 7th step covering the 19a-arc. - -The ratification explicitly reaffirms that Decision 10's -mandatory-annotation rule is unchanged. The lint is *advisory*, -the suppress is an *escape hatch with reason*, neither weakens -the contract. - -### Deferred - -**Item 2 (codegen FnDef fan-out).** A `FnDef::default()` / -`synthetic_fn(...)` constructor would absorb the boilerplate -across the seven synthetic sites. Worth doing the next time a -schema-additive `FnDef` field lands; not worth doing speculatively. - -**Item 3 (snapshot-fixture coupling).** Recorded as known -structural cost. The pattern: any iter that touches a fixture -which is pinned by `ailang-prose/tests/snapshot.rs` must also -re-render the `.prose.txt`. Mitigation would require a more -abstract pinning mechanism (e.g. shape-checking instead of -byte-equality), which is itself a substantive design choice and -not on the queue today. - -**Data model (MVP) drift.** DESIGN.md's "Data model (MVP)" -section (around L1280–1325) shows pre-18a / pre-19b shapes for -`FnDef` and `Type::Fn` — no `param_modes`, no `ret_mode`, no -`suppress`. This drift predates 19b by several iters. Bringing -it current is its own iter (touches three or four shape blocks). - -### State of the world - -JOURNAL queue: empty. Both family-20 and the 19a-arc are now -tidied with their respective ratifications recorded in -DESIGN.md. - -Remaining longer-term substantive forks (none queued, none -default): - - - **Boehm retirement** (Decision 9 → Decision 10 endgame). - Validation bench shipped (18f); the orchestrator's call to - actually flip the default has not been made. - - **Family 21+** — language surface expansion (typeclasses, - polymorphic ADTs beyond the heutige form, IO/error - handling). - - **Data model (MVP) refresh.** Cosmetic doc-tidy iter to - bring the schema snapshot current. - -## 2026-05-08 — Iter 20f: Form-A spec embedding for prose round-trip - -This iter closes a design hole shipped in 20d: the `merge-prose` -prompt instructed the LLM to emit JSON-AST, and gave it only a -12-line "schema essentials" reminder. JSON-AST is the canonical -hashable artefact, but it is not a writing surface, and a 12-line -hint is not a language reference. A foreign LLM with no AILang -exposure had no realistic shot at producing valid output. - -20f makes two coupled changes. - -### Form-A becomes the LLM's output target - -The prompt now instructs the LLM to emit **Form-A** — the canonical -authoring surface fixed by Decision 6. The user's CLI cycle -becomes: - - ail prose foo.ail.json > foo.prose.txt - $EDITOR foo.prose.txt - ail merge-prose foo.ail.json foo.prose.txt > prompt.txt - cat prompt.txt | > foo.new.ailx - ail parse foo.new.ailx > foo.new.ail.json - ail check foo.new.ail.json - mv foo.new.ail.json foo.ail.json # if check is clean - -The original module is loaded via `ailang_core::load_module` -(schema-validating) and re-rendered via `ailang_surface::print` for -embedding. Form-A round-trip is a gating contract on the surface -crate, so this is lossless. - -### Form-A spec embedded in every prompt - -`crates/ailang-core/specs/form_a.md` is a hand-curated, complete -LLM-targeted specification of Form-A — grammar, every term / -pattern / type / def keyword with parenthesised syntax, schema -invariants enforced by the checker, a pitfall catalogue, and four -few-shot modules drawn from `examples/*.ailx`. It is exported as -`pub const FORM_A_SPEC: &str = include_str!("../specs/form_a.md")` -and embedded verbatim in the merge-prose prompt. - -### Drift detection - -The spec is hand-written, so drift was the load-bearing concern in -the design discussion (orchestrator note: user pushed back on -"docs/AIL_FORM_A_SPEC.md handgeschrieben" precisely because the -distance to the code was too big). The fix is mechanical: -`crates/ailang-core/tests/spec_drift.rs` walks every variant of -`Term`, `Pattern`, `Type`, `Def`, `Literal`, `ParamMode` via -exhaustive `match`. The arms are not the assertion — adding a new -variant without updating the match is a compile error in this -test, before the test even runs. Once the variant is matched, the -test asserts an anchor string for it appears in `FORM_A_SPEC`. -Eight tests, all green. - -The exhaustive-match-as-trip-wire pattern is the same idea as the -`ailang-architect` drift review for DESIGN.md — both surfaces -encode load-bearing language semantics that must stay current — -but it runs on every `cargo test`, not just at family boundaries. - -### Why hand-written and not generated - -A generator (walking `syn` / proc-macro reflection on the AST, -emitting a tag table) would close the structural-drift channel -mechanically, but the spec is **not** just a tag table. It carries -prose explanation per construct, the schema-invariant catalogue, -the pitfall list, and the few-shot corpus. None of those can be -emitted from the AST shape alone. Hand-curated content + drift -test on the structural anchors is the right cost / value point; -generator overkill was rejected. - -### What 20f does not address - -Three larger integration paths got named in the discussion but -explicitly deferred per "kiss": - - - **Tool-use schemas** — LLM calls `ail_parse` / `ail_check` - inside its turn, iterating until check is clean. Reduces - convergence from 1–2 rounds to ≈0 for cooperating clients. - - **MCP server** — Anthropic Model Context Protocol exposes - AILang resources (spec, examples, current module), tools - (parse, check, render, prose), prompts (canonical - round-trip templates) to any compatible client. Standard, - discoverable, vendor-neutral. - - **LSP** — editor integration; serves human authors more than - the round-trip flow. Bigger lift. - -All three layer additively on the static-prompt path 20f ships. -The static prompt remains the lowest-common-denominator fallback -that always works without a tool-use-capable client. - -### Test counts - -- ailang-core: +8 (spec drift suite); existing 12 unchanged → 20 -- ail: existing 70 e2e green after rewriting - `merge_prose_prints_framed_prompt` to assert Form-A landmarks - and `FORM-A SPECIFICATION` header instead of `ailang/v0` -- ail unit: existing 3 rewritten in lockstep -- Workspace: all green - -### Family / arc state - -20f closes the prose-roundtrip arc for now. JOURNAL queue is -empty again. The 20-family tidy-iter follows immediately per -CLAUDE.md ("the tidy-iter is non-optional, every named family -closes with one"); the previous note here ("can wait") was -incorrect and has been superseded by the actual tidy below. - -## 2026-05-08 — 20-family tidy-iter (DESIGN.md refresh) - -### Why - -Per CLAUDE.md, every named iter family closes with a tidy-iter: -run `ailang-architect`, resolve each item (fix the drift, ratify -in DESIGN.md, or record in JOURNAL why it is acceptable). Family -20 — 20a, 20b, 20d, 20e, 20f — added a whole new surface -(`ailang-prose`), a CLI subcommand pair (`ail prose` / -`ail merge-prose`), an embedded Form-A spec, and the drift-test -trip-wire. The architect surfaced four items. - -### Architect findings (verbatim, abridged) - - 1. **`docs/DESIGN.md` "Data model (MVP)" block stale** — - missing: `tail` flag on `app`/`do`, `seq`, `clone`, - `reuse-as`, `let-rec`, `paramModes`/`retMode` on `fn`, - `suppress` on `FnDef`, `drop_iterative` on `TypeDef`, - `Type::Con.args`, `TypeDef.vars`. Pipeline diagram still - said "links libgc" only — did not mention `--alloc=rc`. - The deferral was already noted in 19a-arc tidy. - 2. **Project ecosystem inventory missing `ailang-surface` and - `ailang-prose`** — the section's own rule says new tools - are added "as soon as they are established"; both crates - are well past that threshold (14c and 20a respectively). - `ail prose` and `ail merge-prose` were also missing from - the CLI bullet. - 3. **`FnDef { suppress: vec![] }` synthesis fan-out at 45 - sites** — `ailang-codegen` plus `ailang-core::desugar` - alone has 16. 19a-arc tidy noted 7 and deferred. Each - schema-additive `FnDef` / `Type::Fn` field hits every site; - a `FnDef::synthetic(...)` constructor is the absorbing fix. - 4. **`linearity.rs` known debt (L117–125)** — deeply-nested - match on a sub-binder produces a spurious - `over-strict-mode` warning. Not fired by current corpus; - module doc records the gap. Acceptable. - -### Resolutions - -**(1) and (2)** — fixed in this tidy. `docs/DESIGN.md`: - - - **Data model section rewritten end-to-end.** Was titled - "Data model (MVP)" with stale `MVP only fn and const`; now - "Data model" reflecting that all three kinds (`fn`, `const`, - `type`) are real surface forms. Added every additive field - documented in `crates/ailang-core/src/ast.rs` with the - schema-additive `skip_serializing_if` rationale called out - once at the top so each individual mention can stay terse. - The `Suppress`, `ParamMode`, `Literal`, and `Pattern` - sub-schemas now have their own blocks. - - **Pipeline diagram extended.** Now shows `--alloc=gc` (links - libgc; transitional) and `--alloc=rc` (emits - `ailang_rc_inc` / `_dec`; canonical) as two backends sharing - the same MIR. The 18a–18d additions (`Term::Clone`, - `Term::ReuseAs`) get a sentence pointing at where they - materialise under `--alloc=rc`. - - **Ecosystem inventory** gained a "Surface forms" bullet - between Language core and CLI, naming `ailang-surface` - (Iter 14c) as the lossless Form-A printer/parser fixed by - Decision 6 with `parse ∘ print = id` as gating contract, - and `ailang-prose` (Family 20) as the lossy Form-B - projection with no parser whose re-integration goes through - `docs/PROSE_ROUNDTRIP.md`. The CLI bullet now lists - `parse`, `render`, `prose`, `merge-prose`. - -**(3) `FnDef::synthetic(...)` constructor** — *deferred, not -addressed in this tidy.* Reason: the right factor-out shape is -not yet obvious. Most existing call sites differ on more than -just `suppress` — they have to compute `params`, build a -`Type::Fn`, etc. — so a one-arg constructor saves nothing; the -useful constructor is roughly `FnDef::new(name, ty, params, -body)` with `doc: None, suppress: vec![]` defaults. That is a -real change in 45 sites and should land as its own iter when the -schema next grows a `FnDef` field (forcing the touch anyway). -Recorded in the JOURNAL queue. - -**(4) `linearity.rs` sub-binder spurious warning** — -*acceptable, recorded.* The warning fires on a synthetic shape -that the current corpus does not produce; module doc names the -gap; an actual customer-impact trigger would warrant a fix, not -preemptive work. Status quo holds. - -### Acceptable drift summary - -- `FnDef::synthetic(...)` factor-out — deferred until the next - schema-additive `FnDef` field (rationale: existing fan-out - pattern works, change is large but mechanical, and the cost - is amortised by piggybacking on the next forced touch). -- `linearity.rs` deeply-nested-match-on-sub-binder spurious - warning — accepted (not corpus-triggered; cost of fix - exceeds cost of the false positive at this scale). - -### What this tidy does NOT do - -- No code changes. Pure documentation iter. -- No new tests. Existing 288 workspace tests stay green. -- No DESIGN.md content removed. The pre-tidy "Data model (MVP)" - block is replaced rather than amended because it had grown - factually incorrect; the historical context lives in - `git log` and in the iter-by-iter JOURNAL entries that - introduced each schema field. - -### JOURNAL queue (post-tidy) - -- **`FnDef::synthetic(...)` factor-out** — to be done when the - next schema-additive `FnDef` field forces the touch anyway. -- **Deferred richer integration paths** (from 20f): tool-use - schemas, MCP server, LSP. All additive over the static-prompt - round-trip; pick when prioritised. -- **Boehm retirement** — `--alloc=gc` is transitional; - `--alloc=rc` is canonical. Boehm-side scaffolding to be - removed once the rc backend covers the full corpus. -- **Family 21+** (typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation) — long-horizon language work. - -### Family / arc state - -The 20-family is now formally closed with this tidy. Codebase -is in good form: no architecturally load-bearing drift, all -tests green, two minor items recorded as acceptable. The next -iter can be a feature pick from the queue. - -## 2026-05-09 — Boehm half-retirement: CLI default flips to rc - -### Why - -The "Boehm retirement" item in the post-tidy queue surfaced via -a direct user question: *kann Boehm weg, was gewinnen wir -dadurch?* The orchestrator's reading: full removal is premature -— Boehm earns its keep right now as a differential parity -oracle for codegen diagnosis — but the asymmetry (Boehm = CLI -default) actively teaches the wrong mental model. RC is the -canonical runtime per Decision 10, and the CLI was telling -users and prompt-fed LLMs the opposite. So this iter ratifies a -half-retirement: flip the default, keep the oracle, document -the gating condition for full removal. - -### What shipped - -- `crates/ail/src/main.rs`: `default_value = "gc"` → `"rc"` for - both `build` and `run` subcommands. Help text rewritten so - `rc` is described first as canonical (Decision 10 pointer), - `gc` as parity oracle, `bump` as bench-only. The doc reference - to "Iter 18b plumbing — programs leak under this mode" is - removed; it was 18b-era and no longer reflects the matured RC - pipeline. -- `crates/ail/tests/e2e.rs`: `build_and_run` is unchanged — - it picks up the CLI default, which now means the entire - corpus runs under RC as the canonical baseline. The 8 - `build_and_run_with_alloc(...)` differential tests already - pin both backends explicitly and continue to anchor the - parity-oracle relationship. -- `docs/DESIGN.md`: - - Decision 9 retitled "RC canonical, Boehm parity oracle" and - rewritten end-to-end. The 2026-05-08 dual-allocator framing - is preserved as historical context; the live framing is - asymmetric in RC's favour. - - Migration plan step 6 (Iter 18f) updated: retirement is now - explicitly two-step (default-flip done; full removal gated - on the oracle ceasing to catch anything). - - Pipeline diagram (L1585): rc-first, gc-second; gc labelled - "parity oracle" instead of "transitional". - - Wording around "transitional fallback" replaced with the - new "canonical default since 2026-05-09" phrasing. - -### Bug found by the flip - -Flipping the corpus baseline to RC immediately surfaced a silent -codegen bug from Iter 18c.4: `crates/ailang-codegen/src/drop.rs` -build_pair_drop_fn emitted `getelementptr inbounds {{ ptr, ptr }}, ...` -via `push_str` (not `format!`), so the doubled braces went -verbatim into the IR. LLVM parsed that as a struct-of-struct -`{ { ptr, ptr } }`, the GEP referenced a non-existent field -index, and clang failed. - -The bug was invisible under the old `gc` default — the gc -backend doesn't emit per-type drop fns. It was also invisible -in the 8 explicit `_with_alloc("rc")` differential tests because -none of them happened to construct a closure whose env captures -escaped (the only path that triggers the pair-drop emitter). -Two corpus tests caught it the moment the default flipped: -`closure_captures_let_n` and `local_rec_as_value_capture_demo`. - -Fix: single braces. The two corpus tests stay as the regression -guard. This is the canonical example of why the GC oracle is -worth keeping: the bug had been latent for months, and the -parity-baseline flip was the cheap probe that found it. - -### What this iter does NOT do - -- Not full Boehm retirement. `--alloc=gc` still works, libgc is - still linked when selected, the oracle column in e2e is - preserved. -- Not a corpus expansion. Same e2e corpus, now running RC as - baseline instead of GC. -- Not a bench refresh. Bench fixtures and numbers from 18f are - untouched. - -### Gating condition for full retirement - -Boehm comes out completely once the parity oracle stops paying -its keep. Concretely: a few iter families (≥3, say) that ship -without `--alloc=gc` catching any bug `--alloc=rc` did not -already catch. At that point the differential probe is -diagnostic dead weight, libgc-as-build-dep stops being a -worthwhile cost, and the gc backend can come out in a follow-up -iter (delete the gc arm of the e2e tests, drop the `-lgc` link -flag, remove `--alloc=gc` from the CLI parser, mark Decision 9 -historical). - -### Test state - -288 passed / 0 failed / 3 ignored, same as pre-iter — the -codegen fix is offset by the now-canonical-RC baseline catching -no other corpus regressions. - -### JOURNAL queue (updated) - -- **`FnDef::synthetic(...)` factor-out** — unchanged; awaits - next schema-additive `FnDef` field. -- **Boehm full retirement** — re-queued with the new gating - condition (≥3 families with no oracle wins). -- **Deferred richer integration paths** (from 20f): tool-use - schemas, MCP server, LSP. -- **Family 21+** — typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation. - -## 2026-05-09 — Iter 21'a: bench-regression harness shipped - -### Why - -User asked: *"Mich würde interessieren, ob ailang wirklich wie -erwartet performt. Und vor allem wäre es wichtig, Performance -regressions zu erwischen."* Two distinct concerns. The first is -a one-shot validation question; the second is a structural gap. - -Going in: `bench/run.sh` and `bench/latency_harness.py` exist, -but every run was a one-shot manual capture into JOURNAL — no -baseline file, no diff, no tripwire. Five iter families had -shipped since the last canonical numbers (18f / 18g.tidy.fu2). A -perf regression that didn't break correctness would have gone -unnoticed until the next manual bencher invocation, with the -blame surface spread across every intervening commit. The -validation question gets answered as a side effect of building -the harness: capture a fresh baseline, then anything that drifts -from it gets caught immediately rather than weeks later. - -### What shipped - -- **`bench/baseline.json`** — flat per-metric record with - `baseline` + `tolerance_pct`. 31 metrics: 16 throughput (2 - fixtures × 8: gc_s / bump_s / rc_s / gc_over_bump / rc_over_bump - + 3 RSS values), 15 latency (3 arms × 5: median / p99 / p99.9 / - max / p99/median). -- **`bench/check.py`** — argparse front-end. Default behaviour: - spawn `bench/run.sh -n 5`, parse the throughput pipe-table and - the latency stanzas, diff against baseline, print a per-metric - report, exit 0 if every metric is within its one-sided tolerance, - exit 1 on any regression, exit 2 on parser misalignment (output - format changed). Flags: `--from-file PATH`, `--stdin`, - `--baseline PATH`, `--update-baseline` (re-run + overwrite - `baseline.json` with fresh numbers, used after intentional - improvements). - -Tolerances: throughput wall-time 10%, RSS 5%, ratios 8%; latency -median 15%, p99 20–25%, p99/median 20–25%. Tuned to absorb run- -to-run noise on a quiet developer machine, not as the language -correctness bar — Decision-10 thresholds (rc/bump ≤ 1.3× / -p99/median ≤ 5×) are a separate concern and continue to be -evaluated against absolute numbers. - -### Validation against the user's first question - -Captured the baseline (n=5), then re-ran the entire harness -back-to-back. First-run numbers vs. JOURNAL's last canonical -captures: - -``` - | 18f | 18g.tidy | now (1) | now (2) -list_sum.rc/bump | 2.86× | - | 2.89× | 2.96× -tree_walk.rc/bump | 2.53× | - | 2.50× | 2.59× -explicit_at_rc.median | - | 226.1 | 213.9 | 213.5 -explicit_at_rc.p99 | - | 296.4 | 357.5 | 294.6 -explicit_at_rc.p99/med | - | 1.31× | 1.66× | 1.37× -``` - -Throughput is stable across 5 iter families. Latency Boehm-arm -is stable. The explicit-rc p99 swings between captures -(296 → 357 → 295) — consistent with the 18g.tidy.fu2-recorded -p99 range `[288.7, 311.3]` expanding to `[273.2, 366.0]` today. -That is run-to-run dispersion on a single fixture, not a -regression. The all-green second run confirmed: AILang performs -as expected, no shift since 18g. - -Without 21'a we would have had one capture today, seen the -+20.6% p99 number, and either spent an iter chasing a phantom -or written it off without evidence. With 21'a, a single noisy -run is one data point inside a tolerance band, and the tripwire -fires only when a real regression accumulates. - -### What this iter does NOT do - -- **No corpus widening.** Same 4 fixtures (2 throughput, - 2 latency-implicit/explicit). Closure-with-escape-captures - (the 18c.4 trigger), polymorphic ADT pipelines, function-call- - heavy workloads — all queued as 21'b. The 18c.4 doubled-braces - bug remains the canonical demonstration that the corpus has - gaps; baseline-locking the existing 4 fixtures does not close - that. -- **No compile-time bench.** Typechecker / IR-builder slowdowns - from future iters (Family 21 typeclasses likely) would still - be invisible. Queued as 21'c. -- **No CI wiring.** Project has no CI today. `bench/check.py` - is invocable as a tidy-iter gate by hand or by the orchestrator - at family close. -- **No `cargo bench` / criterion.** Hot path for AILang perf - lives outside the Rust crate boundary (compiled AILang - binaries running their own runtime); criterion would be the - wrong instrument. Stays the right call until a Rust-side - hotpath becomes dominant. - -### Tidy-iter discipline addition (proposal, not yet enacted) - -Recommended: `bench/check.py` runs at every tidy-iter, alongside -the architect drift report. Green → family closes. Red → family -does not close until the regression is either fixed (revert / -refactor) or ratified (`--update-baseline` with a JOURNAL entry -naming what got intentionally slower and why). Orchestrator's -call to add to `CLAUDE.md` or the `agents/README.md` tidy-iter -checklist; not enacted in this iter. - -### Test state - -288 passed / 0 failed / 3 ignored. No Rust changes; the addition -is `bench/`-only. - -### JOURNAL queue (updated) - -- **21'b — bench corpus widening.** Closure-capture-escape fixture - (would have caught 18c.4), polymorphic-ADT-pipeline fixture, - call/return-churn fixture. Re-baseline after. -- **21'c — compile-time bench (optional).** Median `ail check` + - `ail build` over the corpus, with its own baseline. Catches - typechecker-complexity regressions before Family 21 lands. -- **`FnDef::synthetic(...)` factor-out** — unchanged. -- **Boehm full retirement** — unchanged. -- **Deferred richer integration paths** (from 20f) — unchanged. -- **Family 21+** — typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation. Orchestrator-level fork that - still wants direct user input. - -## 2026-05-09 — Iter 21'b: bench corpus widening (closure-pair + HOF/poly) - -User dispatch: *"Hätte man schon viel früher einbauen sollen ... -würde mich nicht wundern, wenn da bei den neuen Bench-Fixtures -schon ein paar Überraschungen warten."* The 21'a baseline had only -the historically-grown 4-fixture corpus (2 throughput list/tree, 2 -latency implicit/explicit). 18c.4's months-of-latency proved the -corpus had an unobservable closure-pair-drop blind spot. - -### Two new throughput fixtures - -**`bench_closure_chain`** — exercises the `build_pair_drop_fn` -codegen path (the 18c.4 trigger class). Each iteration of -`run_loop` introduces a fresh `let-rec helper` that captures the -outer fn-param and is passed-as-value to a HOF, forcing the -eta-Lam wrap and the `{ thunk, env }` closure-pair allocation. -Sizes 10k / 100k / 500k closure pairs. - -**`bench_hof_pipeline`** — exercises poly-ADT instantiation -(`(data List (vars a) ...)`) under load via a tail-recursive -`fold_with_fn` that takes `(fn-type (params a) (ret (con Int)))` -as its first parameter. Each fold step does an indirect call -through the f-arg. Sizes 100k / 1M / 3M elements. - -Both are implicit-mode for consistency with the existing -throughput corpus — gc/bump arms are the meaningful comparison, -the rc arm reports alloc-tax-only (Implicit-mode params are not -dec'd; the closure pairs leak by design, as in `bench_list_sum`). - -### What the data shows - -``` -workload | gc(s) | bump(s) | rc(s) | gc/bump | rc/bump | rc RSS(KB) ------------------------+--------+---------+--------+---------+---------+----------- -bench_list_sum | 0.142 | 0.046 | 0.134 | 3.09× | 2.91× | 193448 -bench_tree_walk | 0.098 | 0.037 | 0.096 | 2.65× | 2.59× | 108968 -bench_closure_chain | 0.013 | 0.007 | 0.029 | 1.86× | 4.14× | 39644 -bench_hof_pipeline | 0.134 | 0.048 | 0.136 | 2.79× | 2.83× | 193640 -``` - -**Surprise #1 — closure-pair RC tax is materially higher than -linear/tree alloc.** rc/bump = 4.14× on closure pairs vs 2.91× on -linked-list cells. Plausible cause: the closure pair carries a -two-pointer header (thunk + env) plus a separate env-struct -allocation, vs a Cons cell's single 24-byte alloc-and-init. RC -pays the per-call overhead twice for closures and once for cells. -Decision-10's 1.3× retirement target was set against the -linear-throughput corpus; closure-heavy workloads now have an -explicit 4.14× data point that should inform the eventual slab/ -pool allocator design (Path B in 18f's two-paths analysis). - -**Surprise #2 — HOF + poly costs essentially nothing on top of -direct iteration.** `bench_hof_pipeline` ratios (2.79× / 2.83×) -are within ~5% of `bench_list_sum` (3.09× / 2.91×). The -`fold_with_fn` indirect-call dispatch is dominated by per-cell -alloc; at this size the polymorphism-at-runtime instantiation -adds no measurable overhead on top of monomorph List. -Confirms the 13b "static template + ctor inline" design choice -is paying its keep — runtime poly is not a perf hazard at the -sizes actually exercised. - -**Non-surprise — Boehm vs RC gap on closure work is narrower -than on cells.** gc/bump = 1.86× on closures vs 2.91× on cells. -Boehm's mark-phase pointer-chasing dominates on dense Cons -chains; on sparse closure pairs (each touched once, no cache- -friendly traversal afterwards) Boehm's per-call overhead -amortizes better. - -### Build-path coverage of the 18c.4 trigger class - -Smoke-build of `bench_closure_chain` under all three allocators -succeeds. The rc-arm build exercises `build_pair_drop_fn` -emission for the closure-pair type; if the doubled-braces bug -were still present, the rc-arm build would fail at clang. It -doesn't — confirming the 2026-05-09 fix's reach. From now on, -any future reintroduction of a malformed-IR bug in the closure- -pair drop fn will surface immediately at `bench/check.py` time -rather than going months-undetected. - -### Baseline file: 31 → 47 metrics - -`bench/baseline.json` extended with 16 new metrics (8 per new -fixture: gc_s / bump_s / rc_s / gc_over_bump / rc_over_bump -+ 3 RSS values). Tolerances tuned for the absolute scales: - -- **closure_chain wall-time**: 20–25% (sub-30ms times are noisier - in relative terms; absolute drift of a few hundred μs is well - inside this band). -- **closure_chain ratios**: 15% (compounded run-to-run noise of - two short-time measurements). -- **closure_chain RSS**: 10–15% (small heaps have higher relative - RSS variance than the 100MB+ heaps of the larger fixtures). -- **hof_pipeline**: identical tolerances to `bench_list_sum` - (10/8/5%) — the absolute scale is the same as the existing - large-corpus throughput. - -### What this iter does NOT do - -- **No new latency fixtures.** PTY-line-arrival latency is - already covered by the implicit/explicit pair from 18f.2; the - new fixtures are throughput-shape only. -- **No re-baseline of explicit_at_rc.** Today's three captures of - explicit_at_rc.p99 came in at 357.5 / 294.6 / 251.5 — confirms - what 18g.tidy.fu2's range `[288.7, 311.3]` first hinted at: - this fixture has a wide run-to-run dispersion. The 21'a-set - baseline of 357.5 is on the high end; today's third capture - flags 29.65% improvement on p99 and 28.92% improvement on - p99/median. That's not real signal; it's noise. Re-baselining - to a "median of medians" requires either (a) wider run-count - (n=10+) per capture, or (b) a tighter-controlled fixture. Both - are 21'c+ scope; explicit_at_rc baseline stays at 357.5 for - now and the harness keeps surfacing the dispersion as - improvement until 21'c addresses the methodology. -- **No CLAUDE.md change** — the regression-discipline addition - shipped in commit `2e40699` and applies to this iter. - -### Test state - -288 / 0 / 3, unchanged. No Rust changes; the iter is bench- -fixture and baseline-file additions only. - -### JOURNAL queue (updated) - -- **21'c — compile-time bench.** Median `ail check` + `ail build` - over the corpus, with its own baseline. Catches typechecker - complexity regressions before Family 21 lands. Probably also - the right place to address the explicit_at_rc dispersion via - an n>=10 latency-harness option, since that's a methodology - upgrade more than a corpus addition. -- **21'd — pure-compute fixtures + cross-language reference.** - Mandelbrot / N-body / integer-loop workloads with hand-C - comparisons. Answers CLAUDE.md's "LLVM-linkable, performance - is extremely important" promise with absolute numbers. Likely - splits into 21'd (pure-compute fixtures) and 21'e (C reference - + cross-lang ratio). -- **`FnDef::synthetic(...)` factor-out** — unchanged. -- **Boehm full retirement** — unchanged. -- **Deferred richer integration paths** (from 20f) — unchanged. -- **Family 21+** — typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation. Orchestrator-level fork. - -## 2026-05-09 — Iter 21'f: explicit-mode pair, full alloc+dec vs malloc+free - -Closes the apples-to-apples gap from 21'e. Until this iter, every -AILang/C ratio compared *implicit-mode* AILang (alloc-tax-only, -no dec) against *malloc-and-leak* C — both leaking, both unfair to -the dec-cost question. 21'f ships the matched pair: - -- **`examples/bench_list_sum_explicit.ailx`** — same algorithm and - sizes as `bench_list_sum`, fully annotated with `(borrow)` / - `(own)` / `(drop-iterative)` so codegen emits proper `inc`/`dec` - instrumentation. Each cell allocated by `cons_n_acc` is dec'd as - `sum_acc` consumes the chain via the LCons-arm move-into-tail-call. -- **`bench/reference/list_sum_explicit_free.c`** — same C - algorithm with explicit `free()` walking the chain after sum. - -### The full alloc+dec vs malloc+free numbers - -``` -fixture | AILang_rc | AILang_bump | C | rc/c | bump/c ------------------------------+-----------+-------------+--------+-------+------- -bench_list_sum (implicit) | 138.9 ms | 50.3 ms | 97.5 ms| 1.42× | 0.52× -bench_list_sum_explicit | 150.8 ms | 49.5 ms |119.2 ms| 1.26× | 0.42× -``` - -### What this tells us - -**Subtraction reveals the per-axis tax:** - -- AILang RC dec-tax: `150.8 - 138.9 = ~12 ms` ≈ 8% of rc time. - This is the cost of emitting and executing `ailang_rc_dec` for - every consumed cell + the iterative-drop walker. -- C free-tax: `119.2 - 97.5 = ~22 ms` ≈ 18% of c+free time. - This is glibc's free-list-management overhead per free() call. - -Two non-trivial conclusions: - -**1. AILang's full RC pipeline is only 26% slower than glibc's -full malloc+free pipeline on this workload (rc/c = 1.26×).** -The implicit-mode comparison's 1.42× was misleading — it was -comparing AILang-with-alloc-tax-only vs C-with-malloc-only, which -counted neither pipeline's free path. The fair number is 1.26×, -materially better than the previous read. - -**2. RC's dec is cheaper than glibc's free.** AILang dec-tax is -~12 ms on 4M cells (3 ns/cell); C free-tax is ~22 ms (5.5 ns/cell). -Plausible cause: AILang's `ailang_rc_dec` operates on a known- -shape cell with a fixed-offset refcount header and a static -per-type drop fn — no free-list bucketing decision, no header -introspection, no global lock contention. glibc's `free()` is a -general-purpose allocator with all of those concerns. - -**3. bump's no-free advantage now expresses itself fully.** -`bench_list_sum_explicit.bump/c = 0.42×` means AILang at bump -allocator is **2.4× faster than C at malloc+free** on this -workload. The bump arm pays neither dec nor free; it's the -no-malloc-overhead floor. The 0.42× ratio sets a useful upper -bound on how fast a slab/pool RC allocator could plausibly run -(if Path B from 18f's two-paths analysis ever ships). - -### Runtime bench: bench_list_sum_explicit added - -`bench/run.sh`'s `fixtures` array extends to include the explicit -fixture. The runtime numbers show the dec-tax visible inside the -gc/rc/bump comparison too: - -``` - | gc(s) | bump(s) | rc(s) | gc/bump | rc/bump | rc RSS(KB) -bench_list_sum | 0.141 | 0.049 | 0.140 | 2.88× | 2.87× | 193640 -bench_list_sum_explicit| 0.139 | 0.048 | 0.152 | 2.89× | 3.14× | 142424 -``` - -`rc_over_bump` jumps from 2.87× (implicit, no dec) to 3.14× -(explicit, full dec). The dec-tax is now in the regression-check -band. RC RSS drops from 193 MB (leaking) to 142 MB (actually -freeing) — the first time a non-bump-baseline fixture has -demonstrated RC's free-path actually working under regression -coverage. - -### Baseline file: 55 → 63 metrics - -8 new metrics for `bench_list_sum_explicit` (same shape as the -implicit counterpart, slightly looser RSS tolerance at 8% because -the active-free working set is more variable than the leaking -peak). - -`bench/baseline_cross_lang.json` extended too — 5 new ratio -metrics for the explicit pair. - -### What this iter does NOT do - -- **No tree-walk explicit pair.** Could be done identically to - list_sum (add `bench_tree_walk_explicit.ailx` with full modes - + `tree_walk_explicit_free.c`). Useful but adds another 5 - metrics for a workload class already represented; deferred - unless a specific question motivates it. -- **No HOF / closure explicit pair.** Same reasoning. -- **No methodology upgrade for the latency dispersion.** Still - queued. - -### Test state - -288 / 0 / 3, unchanged. No Rust changes; iter is bench-fixture -additions only. - -### JOURNAL queue (updated) - -The 21' family arc — bench-regression infrastructure — is now -substantively complete: - - 21'a: bench/check.py + baseline.json (runtime regressions). - - 21'b: closure_chain + hof_pipeline corpus. - - 21'c: bench/compile_check.py (compile regressions). - - 21'd: pure-compute fixtures, harness hardening. - - 21'e: cross-language hand-C reference + bench/cross_lang.py. - - 21'f: explicit-mode pair, full apples-to-apples ratios. - -CLAUDE.md `Performance regressions` section codifies the three -scripts as co-equal tidy-iter gates. Any future iter that -regresses runtime, compile-time, or AILang/C ratios beyond -tolerance gets caught at the next family close. - -Remaining queue: - -- **`FnDef::synthetic(...)` factor-out** — unchanged; awaits next - schema-additive `FnDef` field. -- **Boehm full retirement** — unchanged; gating condition still - ≥3 families with no oracle wins. -- **Latency methodology upgrade** — n=10+ captures or tighter - fixture for `explicit_at_rc.p99` dispersion. Could ship as a - short standalone iter if the next tidy-iter sees the tolerance - regularly squeezed. -- **Optional explicit-mode pairs** — `bench_tree_walk_explicit`, - `bench_hof_pipeline_explicit`. Add when a specific question - demands the data. -- **Deferred richer integration paths** (from 20f) — tool-use, - MCP, LSP. Long-horizon. -- **Family 21+** — typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation. **Orchestrator-level fork - with multiple substantive options none of which is clearly - default; needs direct user input before dispatch.** - -## 2026-05-09 — Iter 21'e: cross-language reference + AILang/C ratios - -Closes the question CLAUDE.md has carried since day one — *"the -language must, in the end, be linkable to LLVM. Performance is -extremely important."* — by adding hand-C variants of the bench -corpus, building both with `clang -O2`, and comparing wall times -directly. Until this iter, every performance number AILang shipped -was internal (gc vs. bump vs. rc); none of them said anything -about absolute competitiveness. - -### Hand-C corpus - -`bench/reference/` — four C sources, one per fixture, each -carefully matching the AILang algorithm and explicitly -documenting representation choices (cell width, leak policy) -that affect the ratio: - -- **`list_sum.c`** — linked list, malloc-and-leak (matches - AILang implicit-mode RC). 16-byte cell vs. AILang's 24-byte - ICons (tag overhead). -- **`tree_walk.c`** — balanced tree, malloc-and-leak. 24-byte - cell vs. AILang's 32-byte Tree::Node. NULL leaves (no alloc) - vs. AILang's tag-only Leaf cells. -- **`compute_intsum.c`** — pure-compute affine recurrence. -- **`compute_collatz.c`** — pure-compute, data-dependent control - flow. - -### Headline numbers (5-run, drop-slowest, median of 4) - -``` -fixture | AILang_rc | AILang_bump | C | rc/c | bump/c ------------------------+-----------+-------------+--------+-------+------- -bench_list_sum | 141.6 ms | 48.0 ms | 95.3 ms| 1.49× | 0.50× -bench_tree_walk | 97.0 ms | 38.9 ms | 37.2 ms| 2.61× | 1.05× -bench_compute_intsum | 0.4 ms | 0.4 ms | 0.4 ms| 1.18× | 1.05× -bench_compute_collatz | 56.9 ms | 56.6 ms | 57.5 ms| 0.99× | 0.98× -``` - -### Three substantive findings - -**1. Pure-compute parity with C is real.** `bench_compute_collatz` -runs at AILang/C = 0.98–0.99× across both allocators. Same -algorithm, same `clang -O2`, same wall time. The IR AILang's -codegen emits composes with LLVM's optimizer at the same level -a hand-written C source does — both tail-recursive iteration, -both data-dependent branch prediction. This is the canonical -"LLVM-linkable, performance is extremely important" claim, -backed by data for the first time. `bench_compute_intsum` (1.05– -1.18×) confirms the pattern; both fixtures get LLVM-folded / -optimized symmetrically. - -**2. AILang bump beats glibc malloc on linear allocation.** -`bench_list_sum.bump/c = 0.50×` — AILang's `bump_malloc` -(`ptr += size; return old`) is twice as fast as glibc's -`malloc()` on dense Cons-cell allocation. Expected -qualitatively (bump is 2 instructions inline; glibc malloc has -free-list management even on the alloc path), but the -quantitative result is the first time it's been measured. No- -free workloads (bench fixtures) are exactly where bump shines; -production workloads that actually free are a separate story. - -**3. RC overhead vs C malloc is now quantified.** Linear: -`bench_list_sum.rc/c = 1.49×` — RC's per-call cost (8-byte -refcount header + zero-init + libc malloc backing) is ~50% -above glibc malloc on this workload. Tree: 2.61×, larger -because the per-node fixed cost amortizes over a smaller -working set. These ratios are *implicit-mode* RC (no dec-tax); -explicit-mode would add the dec-cost on top, but the hand-C -reference also has no free, so the apples-to-apples comparison -needs an explicit-mode AILang fixture + a free()-adding C -variant to be honest about both sides. Queued. - -### 20 new baselined metrics - -`bench/baseline_cross_lang.json` — 4 fixtures × 5 metrics -(ail_rc_s, ail_bump_s, c_s, rc_over_c, bump_over_c). Tolerances -12–15% across the board: cross-language ratios are inherently -less stable than within-AILang ratios because two compiler -stacks contribute noise. - -### CLAUDE.md update - -`Performance regressions` now lists three tidy-iter gates: -`bench/check.py`, `bench/compile_check.py`, and -`bench/cross_lang.py`. The cross-lang script is the -heaviest of the three (12 binary builds + 60 timed runs at -n=5), but it's the only mechanism that catches AILang/C ratios -drifting upward over time. Worth the seconds. - -### What this iter does NOT do - -- **No explicit-mode bench fixture pair.** `bench_list_sum` - and `bench_tree_walk` are implicit-mode-only; the C reference - is malloc-and-leak. To honestly compare RC's full alloc+dec - cost vs. C's full malloc+free cost would need a paired - explicit-mode AILang fixture + a `free()`-adding C variant. - Queued as 21'f. -- **No multi-platform reference.** Single x86-64 Linux - measurement. Cross-platform ratios may differ; not in scope. -- **No JIT comparison.** `clang -O2` AOT, AILang AOT — apples - to apples. JIT (LuaJIT, V8, etc.) is a different question. - -### Test state - -288 / 0 / 3, unchanged. - -### JOURNAL queue (updated) - -- **21'f — explicit-mode cross-lang pair.** Add `bench_list_sum_ - explicit.ailx` (with `(borrow)` / `(own)` / `(drop-iterative)`) - and `list_sum_explicit_free.c` (matching `free()` calls). - Re-run cross_lang, capture rc-with-dec / c-with-free ratios. - Closes the apples-to-apples gap on the dec-cost axis. -- **`FnDef::synthetic(...)` factor-out** — unchanged. -- **Boehm full retirement** — unchanged. -- **Latency methodology upgrade** (n=10+ captures) — unchanged. -- **Deferred richer integration paths** (from 20f) — unchanged. -- **Family 21+** — typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation. Orchestrator-level fork. - -## 2026-05-09 — Iter 21'd: pure-compute fixtures + harness hardening - -Closes a third bench-corpus blind spot: every fixture so far has -been heap-allocation-shaped, which makes the gc/bump/rc axis -informative but leaves AILang's IR-codegen quality on tight -integer loops unmeasured. This iter adds pure-compute fixtures -that have no heap pressure at all — the allocator axis flatlines -on them by design, and the absolute wall-time becomes the -codegen-quality signal. - -### Two new pure-compute fixtures - -**`bench_compute_intsum`** — tail-recursive `acc += i*7` loop. -Three sizes (1M / 10M / 50M iterations). No heap, no closure, -no pattern match. - -**`bench_compute_collatz`** — Collatz step-counter. Each step -does one `n % 2 == 0` branch and either `n / 2` or `3*n + 1`. -Two nested tail-recursions (sum over starting values, count -steps for one value). Heavy on integer math + branch -prediction. - -### Surprise on intsum: LLVM eats it whole - -Smoke-run timings under -O2: - -``` -bench_compute_intsum bump -> 0.001 s wall (50M iterations) -bench_compute_intsum rc -> 0.001 s wall -bench_compute_intsum gc -> 0.001 s wall -``` - -50M-iteration loops finishing in 1ms is not "the loop ran very -fast" — it's "LLVM recognized the affine recurrence and replaced -the entire loop with a closed-form constant fold". The wall time -is program startup + 3 print_int calls + already-precomputed -integer literals. - -This is a **positive codegen finding**: AILang's IR is good -enough that LLVM's induction-variable analysis applies the -standard triangular-sum reduction. The IR composes with LLVM's -optimizer at the same level a hand-written C loop would. The -fixture is therefore useless as a runtime regression bench -(absolute number is meaningless) but **is** a useful tripwire -for codegen-quality regressions: if AILang's IR ever stops being -fold-friendly (e.g., due to extra bookkeeping leaking into the -loop body, an opaque closure that breaks LLVM's analysis, or a -dec instruction emitted inside the inner loop), wall time would -jump by orders of magnitude and become trivially detectable. - -For now, `bench_compute_intsum` is excluded from -`bench/run.sh`'s `fixtures` array so its useless-as-regression -data doesn't pollute `bench/check.py`'s ratio tables. The -`.ailx` and `.ail.json` stay in `examples/` as reference, and -21'e (cross-language) will resurface the absolute number when -paired with a hand-C-baseline (also LLVM-folded — the comparison -will be at the level "both run at startup-dominated time, our -IR is at least as good as C's"). - -### Collatz works as intended - -`bench_compute_collatz` does survive optimization (data-dependent -control flow) and runs at 56ms wall time across all three -allocators: - -``` -bench_compute_collatz | gc=0.057 | bump=0.056 | rc=0.056 | gc/bump=1.02× | rc/bump=1.00× -``` - -The 1.00× / 1.02× ratios are the canonical "pure-compute is -allocator-invariant" data point — exactly what the fixture is -meant to assert. If a future codegen change accidentally injects -an allocation into the inner loop, those ratios would diverge -visibly, and that's the regression we'd want to catch. - -### Harness hardening (run.sh) - -Two infrastructure fixes the new fixtures forced: - -1. **Precision bump from %.3f to %.6f** in the Python timing - helper inside `run.sh` and in the awk median-of-even-N - averager. The old 3-decimal format printed `0.000` for - sub-millisecond runs (originally a non-issue when every - fixture ran for ≥10ms; sub-ms intsum trips it). 6-decimal - precision gives µs resolution. - -2. **Zero-guard in the ratio awk**. `gc/bump` and `rc/bump` - awk lines now check `b == 0` and emit `n/a` rather than - crashing with `Division durch Null`. Defensive even with - the precision fix, since LLVM-eliminated workloads can still - round to 0.000 in 3-decimal-formatted medians. - -### Latency tolerance recalibration - -`bench/check.py` flagged `implicit_at_rc.max_us` at +27.63% -during 21'd's bench. Investigation: no codegen-touching commits -since the 21'a baseline; pure-compute fixtures don't touch the -implicit_at_rc workload. The three captures of this metric -across today (477.3 / 456.0 / 609.2 µs) show the run-to-run -distribution is wider than the original 25% tolerance accounts -for — `max` is the single noisiest sample of a 1000-sample -distribution on a leaking control arm, and 30% tolerance is the -honest absorption band. - -Bumped tolerance from 25% to 30% with this rationale recorded -here. NOT a "tolerance softening to dodge a regression" — the -original baseline was the FIRST capture; a fairer tolerance -across natural distribution width is what the harness needed -from the start. p99 (20%) and p99.9 (25%) tolerances stay -unchanged; both came in well within during today's runs. - -### Baseline file: 47 → 55 metrics - -8 new metrics for `bench_compute_collatz`. Tolerances tuned -slightly looser than the heap-heavy fixtures (12% wall, 10% -ratio, 15% RSS) because the smaller absolute heap (~14 MB vs -100 MB+) and faster wall time (56ms vs 100-150ms) both amplify -relative noise. - -### What this iter does NOT do - -- **Does NOT add a cross-language comparison.** That's 21'e - (next iter): hand-C variants of the bench corpus + ratio - table. With 21'd's pure-compute fixtures in place, 21'e is - unblocked and natural. -- **Does NOT investigate the implicit_at_rc.max widening.** - Could be machine-state-dependent (cache, ASLR, system load) - rather than fixture-intrinsic. A clean-machine re-baseline - would clarify; deferred until that's available. -- **Does NOT re-baseline check.py at this run.** Existing - fixtures all stayed within tolerance (after the implicit_at_rc - recalibration); no need to bump the medians. - -### Test state - -288 / 0 / 3, unchanged. No Rust changes; iter is bench- -infrastructure additions only. - -### JOURNAL queue (updated) - -- **21'e — cross-language reference.** Hand-C variants of - bench_list_sum, bench_tree_walk, bench_compute_intsum, - bench_compute_collatz, compiled with `clang -O2`. AILang/C - ratio per fixture — the honest answer to CLAUDE.md's "LLVM- - linkable, performance is extremely important" claim. -- **`FnDef::synthetic(...)` factor-out** — unchanged. -- **Boehm full retirement** — unchanged. -- **Latency methodology upgrade** (n=10+ captures) — unchanged. -- **Deferred richer integration paths** (from 20f) — unchanged. -- **Family 21+** — typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation. Orchestrator-level fork. - -## 2026-05-09 — Iter 21'c: compile-time regression bench - -Closes the second axis the user explicitly named — until this -iter, every typechecker / codegen perf change was invisible to the -tidy-iter gate. Family 21 typeclasses (queued) plus 21'b's poly- -ADT additions both push on the typechecker; without a tripwire, -naive substitution loops or O(n²) constraint resolution would -land silently and decay the whole compile path. - -### What shipped - -**`bench/compile_check.py`** — separate from `bench/check.py` -because the methodology is different (sub-process spawn timing -on small workloads vs. allocator-stress on large ones) and the -relevant tolerances differ by an order of magnitude. Two ops -per fixture: `ail check FILE` and `ail build --opt=-O0 FILE -o T`. -Same drop-slowest-of-N, median-of-rest convention as -`bench/run.sh`. - -**`bench/baseline_compile.json`** — 18 metrics (9 fixtures × 2 -ops). Curated corpus: 5 surface-coverage examples (`hello`, -`list_map_poly`, `local_rec_capture`, `borrow_own_demo`, -`nested_pat`) + 4 bench-throughput fixtures (correlation with -`bench/check.py`). - -**Baselines on this machine**: - -``` -fixture | check(ms) | build(ms) -hello | 0.8 | 65.0 -list_map_poly | 1.1 | 67.3 -local_rec_capture | 0.9 | 65.3 -borrow_own_demo | 1.0 | 64.3 -nested_pat | 1.7 | 67.8 -bench_list_sum | 0.9 | 63.6 -bench_tree_walk | 0.9 | 65.8 -bench_closure_chain | 0.9 | 69.0 -bench_hof_pipeline | 1.0 | 66.6 -``` - -### What the data tells us - -`ail check` runs at **sub-millisecond per fixture** for everything -except `nested_pat` (1.7ms — its deeper pattern tree marginally -exceeds the noise floor). The typechecker is genuinely fast at -the current corpus scale; on this hardware the wall-clock is -dominated by subprocess spawn (~5-10ms on Linux), not by check -work. The bench detects catastrophes (10× slowdowns visible), -not subtler regressions — those want a profiler, not wall-clock. - -`ail build --opt=-O0` runs at **63-69ms per fixture**, dominated -by clang's link step. The variance across fixtures is small — -~9% spread between fastest (`bench_list_sum` 63.6) and slowest -(`bench_closure_chain` 69.0). This is fine for catastrophe- -detection but not informative about codegen-quality differences. -For codegen-quality questions the runtime bench (rc/bump ratios) -remains the right tool. - -### Tolerances - -- **`check_ms`**: 25% per fixture. Justified empirically: a - re-run captured a +17.35% diff on `bench_hof_pipeline check` - with no code changes. Sub-millisecond timing is noisy. -- **`build_O0_ms`**: 20% per fixture. Build noise is materially - lower; observed re-run drift was ≤7% on every fixture. - -These are catastrophe-detector tolerances. Tightening them -would mean false-positives on quiet-machine noise. - -### CLAUDE.md update - -The `Performance regressions` section now lists both -`bench/check.py` (runtime) and `bench/compile_check.py` (compile) -as co-equal tidy-iter gates alongside the architect drift report. -Exit 0 / 1 / 2 semantics are uniform across both scripts. - -### What this iter does NOT do - -- **No latency-harness methodology upgrade.** The wide - explicit_at_rc.p99 dispersion observed across today's three - captures (357.5 / 294.6 / 251.5) is a runtime-bench problem; - the compile bench is a different axis. Methodology upgrade - (n>=10 captures or tighter latency fixture) stays queued. -- **No O2 build bench.** `--opt=-O2` includes additional clang - passes that 2x-3x the build time. Useful for catching codegen - blowup-induced build slowdowns; not useful for detecting - AILang-side regressions, which are amply covered by the O0 - pass. Future addition if/when warranted. -- **No incremental check bench.** Today every `ail check` - rebuilds the entire context. If incremental compilation is - added later (no current plan), re-baseline. - -### Test state - -288 / 0 / 3, unchanged. No Rust changes; the iter is bench- -infrastructure additions only. - -### JOURNAL queue (updated) - -- **21'd — pure-compute fixtures.** Mandelbrot / N-body / integer- - loop workloads. Heap-light, codegen-quality-heavy. Pairs - naturally with 21'e. -- **21'e — cross-language reference.** Hand-C variants of the - bench corpus, compiled with `clang -O2`. AILang/C ratio is the - honest answer to CLAUDE.md's "LLVM-linkable, performance is - extremely important" claim, which today is unbacked by data. -- **Latency methodology upgrade** — n=10+ captures or tighter - fixture for `explicit_at_rc.p99`. Could fold into 21'd or be - its own short iter. -- **`FnDef::synthetic(...)` factor-out** — unchanged. -- **Boehm full retirement** — unchanged. -- **Deferred richer integration paths** (from 20f) — unchanged. -- **Family 21+** — typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation. Orchestrator-level fork. - -## 2026-05-09 — Tidy-iter 21'g: drift cleanup after 21'-arc close - -CLAUDE.md mandates a tidy-iter at every family boundary: run -`ailang-architect`, read its drift report, resolve every item by -either fixing the drift, ratifying it in DESIGN.md, or recording -acceptance in JOURNAL. The 21'-arc (21'a–f, six iters of bench- -regression infrastructure) is now closed; 21'g is the mandatory -cleanup pass. - -### What the architect found - -Three drift items, none codegen-affecting: - -1. **DESIGN.md silent on a finding it should reflect.** Iter 21'b - added `bench_closure_chain` to the corpus and recorded - `rc/bump = 4.14×` — a genuine language-level data point that - contradicts Decision-10's "1.3× target on bench/run.sh" - framing. The 1.3× target was set against linear/tree - workloads only; the closure-pair pattern's measured tax was - not anticipated when Decision 10 was committed. - -2. **Corpus drift between bench scripts.** `bench/run.sh` had - six fixtures by 21'f close (`bench_list_sum`, `bench_tree_walk`, - `bench_closure_chain`, `bench_hof_pipeline`, `bench_compute_collatz`, - `bench_list_sum_explicit`). `bench/compile_check.py` only - tracked the first four — compile-time regressions on the two - newest fixtures (collatz, list_sum_explicit) would not have - fired. Plus `bench_compute_intsum` was excluded from - `bench/run.sh` (LLVM-folded, useless as runtime regression - metric) but present in `bench/cross_lang.py`'s corpus — - defensible per 21'e but undocumented. - -3. **Tolerance-bump policy not codified.** Iter 21'd widened - `implicit_at_rc.max_us` from 25% to 30% with rationale - ("max-of-1000 has wider natural dispersion than p99"), but - `bench/baseline.json`'s "note" field did not encode the - convention. Next noisy max-metric would repeat the discussion. - -### How each was resolved - -**Drift 1 — ratified in DESIGN.md.** Decision 10 gains a -"Workload scope of the 1.3× target" paragraph that names the -linear / tree / poly-ADT subset as the retirement-gate scope and -records the closure-pair `4.14×` as a known representational -cost (each step is two allocations: closure cell + env struct). -The 1.3× retirement target therefore applies to the linear -subset; closure-heavy workloads get a wider band and are -explicitly excluded from the Boehm-retirement gate until a -slab/pool answer ships. Decision-10's commitment to RC is -unchanged; what is scoped is the *quantitative* retirement -criterion, not the choice of memory model. - -**Drift 2 — fixed.** `bench/compile_check.py` CORPUS extended by -three fixtures (`bench_compute_intsum`, `bench_compute_collatz`, -`bench_list_sum_explicit`) and re-baselined. The compile-time -bench now tracks 12 fixtures × 2 ops = 24 metrics (up from 18). -Including `bench_compute_intsum` in compile_check is intentional -and correct: intsum's runtime degeneracy (LLVM constant-folds -the loop) is a runtime-bench question, not a compile-bench one; -the typechecker and codegen still emit identical work. - -**Drift 3 — convention codified.** `bench/baseline.json`'s "note" -field gains a tolerance-convention sentence: max-of-distribution -metrics get a wider band than percentile metrics because the -maximum of a 1000-sample tail-latency distribution has wider -natural run-to-run dispersion than a percentile-of-distribution -does. This is the convention 21'd discovered; it is now the -documented policy for any future `*.max_us` tolerance call. - -While verifying, a fourth small drift surfaced: `bench/cross_lang.py`'s -`bench_compute_intsum` row at 15%/12% tolerance fired a false- -positive (sub-millisecond runtimes have wider relative noise -because subprocess spawn dominates). Tolerances widened to 35% -across all five intsum metrics with rationale recorded in -`baseline_cross_lang.json`'s note field. Same convention as the -max-metric one: time-axis noise scales inversely with absolute -runtime; sub-ms fixtures need looser bands than 50–1000ms ones. - -### Test state - -288 / 0 / 3, unchanged. No Rust changes; the iter is documentation -ratification + bench-script corpus catch-up + baseline JSON -edits. - -### Bench gates - -All three bench scripts re-run sequentially after edits: - -- `bench/check.py` — 63 metrics; 0 regressed, 0 improved, 63 stable -- `bench/compile_check.py` — 24 metrics; 0 regressed, 0 improved, 24 stable -- `bench/cross_lang.py` — 25 metrics; 0 regressed, 0 improved, 25 stable - -**Total under regression coverage: 112 metrics**, all green. - -### JOURNAL queue (updated) - -- **21'h (optional) — explicit-mode pairs for tree_walk and - hof_pipeline.** Same pattern as 21'f's list_sum_explicit: - `(borrow)` / `(own)` / `(drop-iterative)` annotations + paired - hand-C with `free()` walking. Provides apples-to-apples - rc/c ratios for the tree and HOF workload classes. Not - blocking; deferred unless a specific question demands the data. -- **Latency methodology upgrade** — n=10+ captures or tighter - fixture for `explicit_at_rc.p99` dispersion. Unchanged. -- **`FnDef::synthetic(...)` factor-out** — awaits next schema- - additive FnDef field. Unchanged. -- **Boehm full retirement** — unchanged. Now slightly easier to - evaluate: the retirement gate is explicitly scoped to linear - workloads. -- **Closure-pair slab/pool allocator** — newly explicit. The 21'b - finding that `bench_closure_chain` rc/bump = 4.14× points to a - representational improvement: a fixed-shape pair allocator that - compresses closure-cell + env-struct into one fast-path - allocation. Pairs with Decision-10's retirement gate; would - also lower the closure carve-out toward the 1.3× target. - Currently a JOURNAL queue item, not a Decision-level commitment. -- **Deferred richer integration paths** (from 20f) — unchanged. -- **Family 21+** — typeclasses, polymorphic ADTs at runtime, - pattern-binding generalisation. Orchestrator-level fork; needs - direct user input before dispatch. - -## 2026-05-09 — Feature-acceptance criterion codified - -Trigger: the typeclass-design conversation around 22a surfaced a -recurring meta-question — when is a proposed feature actually -worth shipping. The negative form was already in CLAUDE.md -("Design rationale ≠ implementation effort": cost is not a -reason for a feature). The positive form was implicit in many -decisions (Decision 10's reasoning explicitly invokes "what -LLMs are good at vs. not"; the JSON-over-text choice in -Decision 1 is justified by LLM-readability) but never stated as -a feature-acceptance gate. - -This entry codifies it. New top-level section in DESIGN.md -("Feature-acceptance criterion"): a feature ships only if (1) -an LLM author naturally produces code that uses it without -prompting toward it, AND (2) the feature measurably improves -correctness or removes redundancy. Aesthetic appeal — "feels -elegant", "is idiomatic" — does not count; neither does human -ergonomics. Two corollaries: human-attractive but LLM-neutral -features (point-free style, operator overloading, implicit -conversions) are cut; human-hostile but LLM-friendly features -(JSON authoring surface, mandatory mode annotations, mandatory -top-level signatures) are kept. - -Mirrored briefly in CLAUDE.md as a sub-section -"Feature acceptance: LLM utility", paired with the existing -"Design rationale ≠ implementation effort". The two together -fully narrow the space of valid feature rationales: not cost, -not aesthetics, only LLM-author utility. - -### Why now - -The typeclass conversation was the surfacing event. When asked -to construct two examples that pure monomorphisation cannot -handle (heterogeneous Show-able container; higher-rank -polymorphism), the natural response was: both are features that -an LLM author would not unprompted produce. Heterogeneous -containers reduce to closed-world sum types in practice; -higher-rank polymorphism reduces to two separate functions. -Without the rule explicitly named, the next instance of "should -we add feature X" would replay the same reasoning from scratch. -Codifying it now means future feature proposals get gated by an -articulated criterion, not by re-derivation. - -### Implications for 22a (next iter) - -The rule is the explicit basis for the typeclass-design choices -that 22a will make: - -- **Monomorphisation as default dispatch strategy.** A pure-mono - language with rank-1-only polymorphism is exactly what a - natural LLM author produces. Dictionaries would handle features - (heterogeneous containers, higher-rank) that are real but not - LLM-natural — so they don't ship. -- **Higher-rank polymorphism rejected at parse time.** Error - message proposes the canonical workaround (two separate - functions). LLM-friendly: clear cut over subtle codegen - fallback. -- **Heterogeneous containers via sum types, not `dyn Show`.** - Same reasoning. Sum types are what the LLM would produce - unprompted; type-erased existentials are not. - -If the rule were inverted — "ship every feature a sufficiently -sophisticated user might want" — 22a would commit to dictionary -passing and existential types from day one, and AILang would -gain the same dispatch overhead and codegen complexity that -make general-purpose languages opaque to the optimizer. The -rule cuts that off. - -### Test state - -288 / 0 / 3, unchanged. Documentation-only commit; no Rust, -schema, or bench changes. - -### JOURNAL queue - -Unchanged from 21'g. Next dispatch is 22a (typeclass design -iter), which is orchestrator-level work the orchestrator does -directly: DESIGN.md typeclass section, instantiation strategy, -schema nodes for `class` and `instance`, naming convention for -monomorphised functions. Implementer iter (22b) follows after -22a's design lands and is reviewed. - -## 2026-05-09 — 22a: typeclass design - -Trigger: 22a is the typeclass design iter queued in the previous -JOURNAL entry. Per CLAUDE.md, design iters are orchestrator work -done directly. This entry records the decisions and the reasoning; -the canonical specification lives in `docs/DESIGN.md` as Decision 11. - -The Feature-acceptance criterion (codified earlier the same day) -was applied as the primary gate. Each of the five committed -semantic axes traces to a single question: "would an LLM author -unprompted produce code that uses this mechanism, AND does the -mechanism measurably remove redundancy or improve correctness?" -Where the answer was no, the mechanism was rejected. - -### Five committed axes - -1. **Haskell-lite scope** (multi-method, single-param, optional - defaults, single-superclass). Multi-param classes were rejected - because LLM authors do not produce them unprompted, and because - they would require functional dependencies for tractable - resolution. Full Haskell scope (assoc types, GADTs-style - constraints) was rejected on the same grounds. - -2. **Constraints in signatures: explicit and mandatory.** Constraint - inference was rejected for the same reason mandatory mode - annotations are mandatory (Decision 10): explicit annotation - makes every commitment visible at the function boundary, so - reading a signature requires no body-level reasoning. The "alles - sichtbar" line of the project is preserved at this layer. - -3. **Resolution: orphan-free coherence.** Modeled on Rust's coherence - rule rather than Haskell's orphan-with-warning. The hard rule - makes registry resolution unambiguous by construction; no - `AmbiguousInstance` diagnostic exists. The trade — reduced - flexibility for third-party instance authors — is acceptable - because AILang's authoring surface is a single workspace, not - an open package ecosystem. - -4. **Defaults via explicit `default` keyword.** Haskell's - convention of mixing default and required methods in the class - body without syntactic distinction was rejected on visibility - grounds. The `default` keyword makes "what must this instance - implement" answerable from the class header alone. Same line - as axis 2. - -5. **Class-parameter kind: `*` only.** Higher-kinded class params - were rejected because the abstraction they enable (`Functor`, - `Monad`, `Applicative`) is not what an LLM author produces - unprompted. The natural LLM pattern is `List.map`, `Tree.map`, - `Option.map` as separate functions per type, which - monomorphisation handles directly without class machinery. The - kind-`*` restriction also removes the implementation cost of - higher-kinded constraint resolution. - -### Prelude scope - -Three classes ship in the 22b Prelude: `Show`, `Eq`, `Ord`, with -instances for the four primitive types (`Int`, `Float`, `Bool`, -`String`). `Ord` declares `Eq` as superclass. - -`print x` is rewired through `Show.show` at codegen — the one -operator-routing change in 22b. - -`==`, `<`, `<=`, `>`, `>=` stay primitive operators. Routing them -through `Eq`/`Ord` would require migrating every existing fixture -and would risk firing the bench gate during a feature iter. -Operator routing is deliberately deferred to a later iter, gated -on bench-stability. - -`Num` is NOT in the Prelude. Arithmetic operators stay primitive -and per-type. The LLM-natural pattern of distinct `Int` and -`Float` arithmetic is preserved. - -### What 22a does not commit to - -- The exact textual form of monomorphised-symbol names (e.g. - `show@Int` vs. `show#Int` vs. a hash-suffixed scheme). Naming - is deterministic from `(method, type-hash)`; the textual form - is fixed in 22b alongside the existing mangling scheme - (DESIGN.md §"Mangling scheme"). -- The Form-B (prose) projection of `ClassDef` and `InstanceDef`. - Prose-projector arms for the new nodes are 22b scope. -- Mode-annotation defaults for class methods. Class method - signatures are full FnSigs and carry mode annotations per - Decision 10; conventions (likely `borrow` for read-only methods - like `show`, `eq`, `lt`) settle in 22b. -- Auto-derivation of instances. `deriving` is a future-iter option, - gated on Feature-acceptance at proposal time. - -### Why this iter is design-only - -Per CLAUDE.md and the previous JOURNAL queue entry, 22a is design. -No Rust crates change in this iter; no schema-floor commit; no -bench corpus change. The iter ships a single edit to -`docs/DESIGN.md` (Decision 11 added between Decision 10 and the -Mangling-scheme section) and this JOURNAL entry. The schema and -implementation land in 22b. - -### Test state - -288 / 0 / 3, unchanged. Documentation-only. - -### Bench gates - -Not re-run. No runtime, codegen, or check-time path is touched. - -### JOURNAL queue (updated) - -- **22b — typeclass implementer.** Schema floor for `ClassDef`, - `InstanceDef`, `FnDef.type.constraints`. Workspace-load - registry build with the three coherence/uniqueness/completeness - checks. Class-schema validation (`KindMismatch`, - `InvalidSuperclassParam`, `ConstraintReferencesUnboundTypeVar`). - Typecheck arms for `MissingConstraint` and `NoInstance`. - Monomorphisation pass + naming convention. Prelude module with - `Show`/`Eq`/`Ord` and the four primitive instances. `print` - rewiring through `Show.show`. End-to-end fixture exercising - the full path. Bench gate at close. -- **22c — typeclass corpus expansion (deferred).** User-defined - classes/instances exercised by an example beyond the Prelude. - Optional `deriving` shorthand if Feature-acceptance gate passes. -- **Operator routing through Eq/Ord (deferred, no commitment).** - Migrating `==`, `<`, etc. to class methods. Big-bang fixture - refactor; gated on bench-stability and on a clear LLM-author - benefit ("less ceremony" alone is not a benefit; needs a - redundancy-removal claim). -- **21'h, latency methodology, FnDef::synthetic, Boehm full - retirement, closure-pair slab/pool, deferred richer integration - paths.** Unchanged from 21'g. - -## 2026-05-09 — Iter 22b.1: typeclass schema floor + workspace registry - -Trigger: 22b is the typeclass implementer arc queued by 22a. The -arc is sliced into four sub-iters (22b.1 schema floor + registry, -22b.2 typecheck arms, 22b.3 monomorphisation, 22b.4 prelude + -prose-projection arms); 22b.1 is the schema floor. Per the 22a -queue, the spec is Decision 11 §"Form-A schema" + §"Resolution -and monomorphisation" (the three coherence checks). - -### What 22b.1 shipped - -**AST.** `Def::Class(ClassDef)` and `Def::Instance(InstanceDef)` -landed as additive variants of `Def`. The struct shapes follow -Decision 11 §"Form-A schema": single-string `param` (multi-param -classes rejected by shape), optional `superclass: Option`, methods carry full FnSig signatures and an -`Option` `default` body, instances carry the concrete -instance type (`Type` variant, not `String`) plus their method -bodies. Every optional field uses `skip_serializing_if` per the -13a/19b additive-schema pattern, so canonical-JSON bytes — and -therefore `def_hash` — of every pre-22b fixture stay bit-identical. -`iter22b1_schema_extension_preserves_pre_22b_hashes` re-asserts the -two pinned hashes (`sum.ail.json`/sum → `db33f57cb329935e`, -`list.ail.json`/IntList → `b082192bd0c99202`); the -same-shape-different-paths test -(`iter22b1_classdef_empty_optionals_hash_stable`) asserts that a -`ClassDef` parsed from JSON without the optional keys hashes -identically to one constructed with `superclass: None` / `doc: -None`. - -**Downstream `match Def::*` sites.** Every match was extended with -explicit `Class`/`Instance` arms in `ailang-core` (desugar, -pretty), `ailang-check` (lib, lift, linearity, uniqueness), -`ailang-codegen` (lib), `ailang-prose` (lib), `ailang-surface` -(print), and `crates/ail/src/main.rs` (collect_refs, def_summary). -Behaviour for 22b.1 is placeholder: skip in typecheck/codegen, -one-line summary in pretty/manifest, placeholder marker in -prose/print. Each arm carries a TODO comment naming the -deferred sub-iter (22b.2 typecheck, 22b.3 codegen, 22b.4 -prose-projection). - -**Workspace registry.** `Workspace` gains a `registry: Registry` -field, populated at the end of `load_workspace` after the import -DFS. The registry is keyed by `(class-name, type-hash)` where -`type-hash` is a new `canonical::type_hash(t: &Type) -> String` -(16-hex prefix, parallel in shape to `def_hash` and `module_hash`). -`build_registry` enforces three coherence checks per Decision 11 -§"Resolution and monomorphisation": - -- **Coherence (orphan-freedom).** Every `instance C T` lives in - the module of `C` or in the module of `T`. Otherwise → - `WorkspaceLoadError::OrphanInstance` with the class, type, - defining module, and the modules where the class and type - actually live. -- **Uniqueness.** No two entries share a key. Otherwise → - `DuplicateInstance` with the two colliding modules. -- **Method completeness.** Each instance specifies a body for - every required (non-default) method of its class. Otherwise → - `MissingMethod` with the missing method name. - -The CLI's `workspace_error_to_diagnostic` carries three new error -codes (`orphan-instance`, `duplicate-instance`, `missing-method`) -into the JSON-mode diagnostic stream of `ail check`. - -**Test fixtures.** Seven fixtures under `examples/test_22b1_*`: - -- `test_22b1_orphan_class.ail.json` — class + instance in same - module (positive; one registry entry). -- `test_22b1_orphan_third_classmod.ail.json` + - `test_22b1_orphan_third.ail.json` — class in module A, instance - in third module declaring `instance Show Int` (Int is primitive, - so neither leg of coherence is satisfied → OrphanInstance). -- `test_22b1_dup_a.ail.json` + `test_22b1_dup_b.ail.json` + - `test_22b1_dup_entry.ail.json` — module A defines class Show - + instance Show MyInt (legal, A is class's module); module B - defines type MyInt + instance Show MyInt (legal, B is type's - module). Entry imports both → DuplicateInstance with two distinct - defining modules. -- `test_22b1_missing_method.ail.json` — class Eq with two - non-default methods (eq, ne); instance Eq Int specifies only - ne. Fires MissingMethod { method: "eq" }. - -The duplicate-instance setup was the trickiest: A→B import -("class A needs to know type B") works; B does not need to import -A because the registry build happens over the whole workspace, -not per-module — B just declares the instance with a string class -name. No import cycle. - -**Deviations from the plan.** The plan in -`docs/superpowers/plans/2026-05-09-22b.1-typeclass-schema-floor.md` -prescribed eight commits (one per task). Per the project's -iter-cadence convention (CLAUDE.md §"Iter cycle"), 22b.1 ships as -three commits: AST + downstream match arms (22b.1.1), workspace -registry skeleton + coherence checks + hash-stability tests -(22b.1.2), fixtures + workspace tests (22b.1.3). The JOURNAL -update is this commit (22b.1.4). - -### Surface round-trip gate - -The Form-B parser does not yet know `class` / `instance` head -keywords (parser arms are deferred to 22b.4 alongside the prose -projection). The round-trip test -(`crates/ailang-surface/tests/round_trip.rs`) iterates every -`examples/*.ail.json` and re-parses the printed text; without -a filter, `test_22b1_*` fixtures would block the gate on a -property the schema floor does not promise. The fixture filter -now excludes `test_22b1_*` until 22b.4 ratifies prose round-trip -for the new variants. - -### Test state - -288 → 295 (+7). New tests: -- `iter22b1_schema_extension_preserves_pre_22b_hashes` (hash.rs) -- `iter22b1_classdef_empty_optionals_hash_stable` (hash.rs) -- `iter22b1_workspace_with_no_classes_has_empty_registry` -- `iter22b1_instance_in_class_module_loads_clean` (positive) -- `iter22b1_orphan_instance_fires_diagnostic` -- `iter22b1_duplicate_instance_fires_diagnostic` -- `iter22b1_missing_method_fires_diagnostic` - -All workspace tests in `ailang-check/tests/workspace.rs` and the -e2e suite remain green; the registry threads through unchanged -(legacy callers default-construct an empty registry, codepaths -that hold a real workspace clone the field through). - -### Bench gates - -All three green at iter close: -- `bench/check.py` — 63 metrics, 0 regressed, 0 improved, 63 stable. -- `bench/compile_check.py` — 24 metrics, 0 regressed, 0 improved, - 24 stable. -- `bench/cross_lang.py` — 25 metrics, 0 regressed, 0 improved, - 25 stable. - -22b.1 touches workspace-load only; no hot path is exercised. The -registry-build pass adds two passes over all loaded modules (one -to collect class/type defining-module maps, one to register -instances) but only when class/instance defs are present — pre-22b -fixtures never enter the second pass at all. No measurable impact. - -### What 22b.1 does NOT ship - -Explicitly deferred to 22b.2: -- `FnDef.type.constraints` field (constraint annotations on - regular fns). -- Class-schema validation diagnostics: `KindMismatch`, - `InvalidSuperclassParam`, - `ConstraintReferencesUnboundTypeVar`. -- Typecheck arms: `MissingConstraint`, `NoInstance`. -- `OverridingNonExistentMethod`, `MethodNameCollision`. - -Deferred to 22b.3: -- Monomorphisation pass (the only mechanism for class-method - calls; replaces resolved class-method calls with synthesised - monomorphic FnDefs). - -Deferred to 22b.4: -- Prelude module (`Show`/`Eq`/`Ord` + four primitive instances). -- `print` rewiring through `Show.show`. -- Form-B (prose) projection arms for `ClassDef`/`InstanceDef` - and the matching parser arms (the round-trip-filter compensation - retires when 22b.4 lands). - -### JOURNAL queue (updated) - -- **22b.2 — typeclass typecheck arms.** `FnDef.type.constraints` - schema extension; class-schema validation; `MissingConstraint` - + `NoInstance` per call site; `OverridingNonExistentMethod` + - `MethodNameCollision`. -- **22b.3 — monomorphisation pass.** Synthesise monomorphic - FnDefs from `(method, type-hash)` pairs; rewrite class-method - calls; cache by `(method, type-hash)`. Determines the textual - monomorphised-symbol naming (open in 22a). -- **22b.4 — Prelude + prose round-trip.** Prelude module shipping - `Show`/`Eq`/`Ord` and the four primitive instances; `print` - rewiring through `Show.show`; Form-B parser/printer arms for - ClassDef/InstanceDef; remove the `test_22b1_*` filter from the - round-trip gate. -- **22c — typeclass corpus expansion (deferred).** Unchanged from - the 22a queue. -- **Operator routing through Eq/Ord (deferred, no commitment).** - Unchanged from the 22a queue. -- **21'h, latency methodology, FnDef::synthetic, Boehm full - retirement, closure-pair slab/pool, deferred richer integration - paths.** Unchanged from 21'g. - -## 2026-05-09 — Skill system live (orchestration meta-iteration) - -The five-skill development pipeline shipped today, formalising the -existing iter-cycle workflow as durable artefacts plus -context-isolated subagent dispatch. - -**Spec:** `docs/specs/2026-05-09-skill-system.md`. -**Plan:** `docs/plans/2026-05-09-skill-system-buildout.md`. - -### What landed - -- Five `SKILL.md` files under `skills//`: - - `skills/brainstorm/` — milestone spec generator (hard-gate - before plan) - - `skills/plan/` — spec → bite-sized plan (No-Placeholders rule) - - `skills/implement/` — plan execution with two-stage review - (spec compliance → code quality) - - `skills/audit/` — milestone-tidy with bench-regression gate - - `skills/debug/` — RED-first bug diagnoser, four-phase Iron Law -- All six existing agents migrated to `skills//agents/`: - implementer + tester under `implement`; architect + bencher + - docwriter under `audit`; debugger under `debug`. `agents/` - retains only `README.md` (rewritten as a pointer roster). -- `.claude/agents/{implement,audit,debug}` symlinks tracked in git - for subagent-type discovery on clone. -- `CLAUDE.md` split: 294 → 245 lines. Bug-fix TDD rules, - iter-cycle / tidy-iter / performance-regression sections, and - the feature-acceptance-criterion detail moved into the relevant - skill files; CLAUDE.md keeps headline rules and one-line - pointers. - -### Vocabulary change - -New artefacts use **milestone** (formerly "family") and -**iteration** (formerly "iter"). Legacy JOURNAL entries are NOT -retroactively renamed — they stay in their original wording. New -entries from this point forward use the new vocabulary; commit -messages and JOURNAL section titles will alternate during the -transition until the next milestone closes. - -### Pipeline contract - -``` -brainstorm -> docs/specs/.md -plan -> docs/plans/.md (per iteration) -implement -> per-task commits + JOURNAL entry -audit -> drift report + bench results (mandatory at - milestone close) -debug -> RED-test commit, hands to implement (mini-mode) -``` - -Each stage commits its artefact before handing off; this makes -every stage independently revertable. - -### Skipping rules (codified) - -- `brainstorm` may be skipped for tidy / bug-fix / trivial-mechanic - iterations; **never** at milestone start. -- `plan` may be skipped for bug fix (RED test is the plan) / - trivial mechanic; **never** for a standard iteration. -- `audit` is **mandatory** at milestone close; deferral requires - an explicit JOURNAL entry naming reason + re-run date. -- `debug` is **mandatory** for any observable bug; trivial bugs - still get RED first. - -### Why this exists (motivations recorded) - -Two concrete goals named by the user: - -1. **Formalisation of the development cycle.** Discipline that - was scattered across a 294-line CLAUDE.md, agent reading lists, - and the orchestrator's habits is now in named skills with - trigger conditions and skipping rules. Future-me cannot - accidentally diverge from existing practice without explicitly - violating a named rule. -2. **Context relief.** Skills route work to subagents (per - `skills/implement` and the upstream - `superpowers:subagent-driven-development` pattern). Subagents - work in isolated context windows; the orchestrator only sees - their reports. Long milestones no longer have to fit a single - main-context window. - -The user remains the boss — skills are sharper tools, not a -replacement for orchestrator judgement. The "Direction freedom" -and "When NOT to delegate" sections of CLAUDE.md still apply. - -### Rationalisation tables — pressure-tested baselines - -The five skills were drafted from baseline pressure scenarios run -against a general-purpose subagent without any skill loaded. The -baseline showed strong existing discipline (the project's -CLAUDE.md was already doing its job), so the skills' value-add is -discoverability + context-relief + survival of the CLAUDE.md split, -not novel rule-enforcement. The pressure-test transcripts informed -the `Common Rationalisations` tables in each SKILL.md. - -VERIFY/REFACTOR subagent passes per skill were skipped on the -build-out iteration — pragmatic, given the baselines were already -compliant. If a real-world milestone surfaces skill bugs the -baselines missed, those become follow-up iterations on the -relevant skill. - -### Open follow-ups - -- **First end-to-end exercise.** The next milestone (post-22b / - post-22c) is the first one routed entirely through the skill - pipeline. Expect minor skill bugs to surface; each becomes a - follow-up iteration on the relevant SKILL.md. -- **`ailang-docwriter` rare-tool check.** Now under - `skills/audit/agents/` but invoked rarely. If rustdoc drift - turns out to be its own recurring concern with its own cadence, - it may earn a dedicated skill (`document`?) later. For now, - audit owns it. -- **22b.2 typeclass typecheck arms.** Unchanged from the 22b - queue — picks up after this orchestration iteration closes. - -### Non-goals - -- Renaming `iter` to `iteration` across legacy JOURNAL entries. - Cost > benefit; legacy stays. -- Self-applying the new skills retroactively to in-flight 22b - iterations. 22b stays in its current shape; 22b.2 onwards run - through the new pipeline if 22b is paused; otherwise the next - milestone is the first consumer. -- Per-skill TDD pressure-tests (RED + 3 GREEN/REFACTOR runs each) - on the build-out iteration. Deferred to a follow-up audit if the - first end-to-end exercise reveals gaps. - -## 2026-05-09 — Iteration 22b.2: typecheck arms - -First milestone-iteration routed entirely through the new skill -pipeline (`brainstorm` was retrospective for milestone 22 in iter -22b.2 prelude; `plan` produced `docs/plans/2026-05-09-22b2-typecheck-arms.md`; -`implement` ran the 10 tasks plus E2E; `audit` will close the -milestone separately). - -**What shipped (8 new diagnostics + schema + infra):** - -- Schema: `Constraint { class, type_ }` struct; `Type::Forall` gains - `constraints: Vec` gated by - `#[serde(default, skip_serializing_if = "Vec::is_empty")]`. Pre-22b.2 - fixtures hash bit-identical (regression test in `hash.rs`). -- Class-schema (3, in `validate_classdefs` running before - `build_registry`): `kind-mismatch` (HKT use forbidden, Decision 11 - axis 5), `invalid-superclass-param` (superclass `type` must equal - class `param`, axis 1), `constraint-references-unbound-type-var` - (class-method constraint vars must be bound). -- Workspace coherence (3, in `build_registry`): - `overriding-non-existent-method` (instance bodies for undeclared - methods), `method-name-collision` (single code with `kind: "class-class"` - vs `"class-fn"` ctx field; backed by structural `enum Origin` not - string-prefix matching; fn-fn delegated to existing `DuplicateDef`), - `missing-superclass-instance` (post-loop superclass-chain walk with - `BTreeSet<&str>` cycle-termination guard). -- Per-FnDef typecheck (2, in `check_fn`): `missing-constraint` - (residual at class-method call site doesn't match expanded - declared constraints, where expansion walks the superclass chain - one step per Decision 11), `no-instance` (concrete-type residual - not in workspace registry; reuses `canonical::type_hash`). -- Infra: `ModuleGlobals { fns, class_methods }` two-channel split - (class methods get a separate map preserving definition order via - `IndexMap`); `ClassMethodEntry { class_name, class_param, method_ty, - defining_module }` accessor; `Env.class_methods`, - `Env.class_superclasses` (`BTreeMap` — absence - means no superclass), `Env.workspace_registry` threaded through - `check_in_workspace`. - -**Tests (cross-cutting + e2e):** 12 tests in -`crates/ail/tests/typeclass_22b2.rs` (4 task-level + 4 e2e -cross-module + 4 superclass-walk asymmetry). Workspace-load tests -under each diagnostic in `crates/ailang-core/src/workspace.rs`. -11 fixtures under `examples/test_22b2_*.ail.json`. - -**Per-task subjects (mirrors commit messages):** - -- iter 22b.2.1 — Constraint struct + Forall.constraints -- iter 22b.2.2 — kind-mismatch -- iter 22b.2.3 — invalid-superclass-param -- iter 22b.2.4 — constraint-references-unbound-type-var -- iter 22b.2.5 — overriding-non-existent-method -- iter 22b.2.6 — method-name-collision -- iter 22b.2.7 — missing-superclass-instance -- iter 22b.2.8 — register class methods in module globals -- iter 22b.2.9 — missing-constraint per-fn -- iter 22b.2.10 — no-instance per-fn -- iter 22b.2.e2e — cross-module class resolution + multi-fn aggregation - -19 commits total on branch `iter-22b2-typecheck-arms` (10 task -commits + 8 round-2 fixes from the spec/quality review loops + 1 -e2e commit). Two-stage review surfaced real issues the implementer -or task carrier alone wouldn't have caught — see "Skill-system -post-mortem" entry below. - -**Known debt deliberately not touched:** - -- The `constraint-references-unbound-type-var` walker is shallow: - it only inspects constraints whose `type_` is a bare `Type::Var`. - `(Bar, Maybe z)` with `z` unbound would slip past. Enough for - schema-level coherence per Decision 11's stated convention; - recursive walk is a follow-up if a real fixture needs it. -- Superclass-cycle DETECTION as a first-class diagnostic is - queued. The chain walks now terminate via `BTreeSet<&str>` - visited-set in `missing-superclass-instance` and via single-step - expansion in `expand_declared_constraints`; surfacing cycles as - their own error is a future arm. -- `Env.workspace_registry` is `Default::default()` in the - standalone `check_module` path. Acceptable because that path is - used only for diagnostics that don't need cross-module instance - lookup; if a future feature needs registry-backed checks in - module-only mode, the field must be made `Option` or - fed from a builder. - -**22b.3 next:** monomorphisation pass — synthesise FnDefs from -`(method, type-hash)` pairs, rewrite calls, with a synthetic -class+instance fixture for end-to-end mono validation before the -Prelude lands in 22b.4. - -## 2026-05-09 — Skill-system post-mortem (first end-to-end run) - -Iteration 22b.2 was the first milestone-iteration routed entirely -through the `brainstorm → plan → implement` pipeline (the build-out -iteration 22c.live shipped the system itself but didn't exercise -it on a real ten-task milestone). Observations: - -**What the skill system caught that an unstructured run wouldn't:** - -- **Spec drift in carrier construction.** Task 5 spec reviewer - flagged a placement issue (`OverridingNonExistentMethod` after - `MissingMethod` vs after `UnboundConstraintTypeVar`) that came - from the *orchestrator's* carrier text diverging from the literal - plan. The two-stage review made the divergence visible at the - point it landed, not three iters later. Cost: one fix-round - commit (`1f6d232`). Without spec review I'd have shipped the - drift. -- **Quality-stage caught real bugs spec-stage couldn't see.** - Task 6's first round used `prior.starts_with("class ")` to - decide the class-class-vs-class-fn discriminator — semantically - correct against the task text (so spec-compliant) but coupled - the discriminator to an ad-hoc display string. Quality reviewer - surfaced both that and the fn-fn-misreports-as-class-fn - correctness bug. The structural `enum Origin` rework - (`b252f83`) was ten lines of code but the only way the bug got - caught was by separating "did you build the right thing" from - "did you build it well". -- **Test-coverage gaps.** Task 9 shipped without a superclass-walk - exercise (the `expand_declared_constraints` code path was - unverified); quality stage flagged it as Important. Task 10 - shipped without a positive `instance-present` test (registry - threading bugs would have slipped past); also flagged. In both - cases the per-task plan was complete but the implementer didn't - add cross-cutting coverage at task level. Quality review caught - the gap right when it would otherwise have rotted. -- **Termination guards.** Task 7's chain-walk had no cycle bound; - quality stage surfaced the hang risk. Implementer's first round - was a clean implementation of the plan, but the plan didn't - ask for a guard and a malicious workspace would have hung - `build_registry` indefinitely. Quality reviewer caught the - defensive-validation question and the fix landed in round 2. - -**What the skill system cost:** - -- 8 round-2 fix commits across the iter (out of 19 total). Every - one was a justified correction. Review-loop overhead is real but - the marginal cost per finding is much lower than the downstream - cost of catching those issues weeks later. Per-iter latency: ~50% - more wall-clock vs a single-pass implementation, but that wall- - clock is delegated subagent time, not orchestrator context. -- **Carrier-text duplication.** Each implementer + spec reviewer - + quality reviewer dispatch carries the full task text and - scene-set. Token-cost-wise, this is the single biggest line item. - An optimisation to consider: have the spec reviewer accept a - "task_text_ref: docs/plans/.md#task-N" and load the file - itself. But this would break the "agents do not open - docs/plans/" convention from `skills/README.md`. Trade-off worth - revisiting if iter cost balloons. - -**What surprised me:** - -- **Implementer self-corrections were strong.** Three implementer - rounds returned `DONE_WITH_CONCERNS` flagging an unrequested - edit they made because the task plan was underspecified (e.g. - Task 2's `round_trip.rs` skip-list extension; Task 6's - `"kind": "string"` → `"str"` fixture fix). These were always - correct. The DONE_WITH_CONCERNS protocol works — the - implementer doesn't push through silently, but doesn't demand - permission either when the call is clear. -- **Spec-stage failed cheaply.** When non_compliant fired, the - fix-rounds were always single-file edits. The two-stage split - prevented quality-review from spending time on diffs that would - later be moved or reverted. -- **Quality stage's "trust the implementer to choose the fix" - discipline held.** Reviewers consistently described issues - rather than prescribing fixes. The implementer made structurally - sound rework calls (e.g. `enum Origin` instead of patching the - prefix-string check in place). The reviewer instructions in - `ailang-quality-reviewer.md` were doing real work. - -**Skill-system tweaks for the audit phase:** - -- The `ailang-spec-reviewer` carrier should default to "the - literal plan task text" rather than orchestrator paraphrase. - Task 5's drift came from my paraphrase adding "after X". The - fix is for me as orchestrator to copy plan-task text verbatim - into the carrier; not a SKILL.md change but a discipline note. -- `ailang-tester` Step 3 was excellent for this iter: identified - cross-module gaps + multi-fn aggregation that per-task tests - missed, AND honestly reported when a candidate property wasn't - E2E-testable rather than padding the suite. Worth preserving - the "DONE with no new tests if coverage is exhaustive" carve- - out in the agent file. - -**Net assessment:** the skill pipeline shipped 22b.2 with materially -higher quality than a single-pass run would have, at proportional -cost. First end-to-end exercise is a positive signal; the -remaining open follow-ups from 2026-05-09 (skill bugs surfaced by -real iter use) get queued as targeted SKILL.md tweaks rather than -a wholesale revisit. - -## 2026-05-09 — Iteration 22b.3: Monomorphisation pass - -Mono pass slots into `crates/ail/src/main.rs::build_to` between -`lift_letrecs` and `lower_workspace`. The pass is a `Workspace -> -Workspace` transformation living in `crates/ailang-check/src/mono.rs`, -mirroring `lift.rs`'s position and ownership model. For workspaces -with no `Def::Class` / `Def::Instance`, the early-out returns the -input clone byte-identically — pre-22b fixtures stay -hash-stable through the full pipeline. For typeclass-bearing -workspaces, the pass runs three phases: (1) build the full env via -`build_workspace_env` (same shape as `check_in_workspace`), (2) -fixpoint loop — each round walks every fn/const body, re-runs -`synth` to recover residual class constraints, applies `subst`, -filters fully-concrete residuals, dedupes via `(class, method, -type-hash)` key, synthesises one `Def::Fn` per unique target via -`synthesise_mono_fn` (instance body OR class default fallback, -class param substituted to instance type), and appends to the -registry's `defining_module`. Loop terminates when a round adds -no new entries — closes on chained class-method calls (e.g. an -instance body that itself calls a class method, exercised by -`test_22b3_chained_calls.ail.json`). (3) Phase 3 walks every -fn/const body in the post-fixpoint workspace and rewrites every -class-method `Term::Var` to its mono-symbol name, qualifying with -the defining module if cross-module. The walker is shadow-aware: -local bindings (Let / LetRec / Lam / Match-arm) suppress the -rewrite at locally-named Vars, mirroring synth's -locals-take-precedence rule. - -End-to-end gate met: `examples/test_22b3_mono_synthetic.ail.json` -declares `class Foo a where foo : (a) -> Int` + `instance Foo Int -where foo = lam i. i` + `main = do io/print_int (foo 5)`. `ail -run` produces stdout `5\n`. All three bench gates green: -`bench/check.py` 63/63, `bench/compile_check.py` 24/24, -`bench/cross_lang.py` 25/25 — mono pass is genuinely no-op for -class-free fixtures. - -Mono-symbol naming format decision: spec recommended -`#` ("but the implementer chooses"). `#` turns -out to be invalid in LLVM IR global identifiers — clang rejected -`@ail_..._foo#Int_clos`. Switched to `__` -(double underscore separator); same legibility, valid C-ABI. -DESIGN.md / spec amendment in the next sweep. - -Per-task subjects: - -- iter 22b.3.1: mono pass skeleton — identity for class-free workspaces -- iter 22b.3.2: mono_symbol — primitive surface forms + hash for compound -- iter 22b.3.3: collect_mono_targets — synth-replay residual gathering -- iter 22b.3.4: synthesise_mono_fn — instance body + class default fallback -- iter 22b.3.5: workspace fixpoint loop — dedup + synth-append (multi-round chained-call coverage) -- iter 22b.3.6: call-site rewrite — same/cross module qualified names (shadow-aware) -- iter 22b.3.7: synthetic fixture — class+instance compiles, runs, prints 5 -- iter 22b.3.tester: e2e for coherence (two instances), default keyword, cross-module mono - -Each task ran through the two-stage review (spec compliance → -code quality). Quality round caught: shadow-blind walker (Task -6), untested cross-module branch (Task 6), untested zero-arg -method branch (Task 4), untested multi-round chained-call case -(Task 5), defensive `unwrap_or_default` (Task 3), speculative -`Float` arm (Task 2), and module-doc-vs-implementation drift -(Task 1). All caught + fixed before iteration close. - -**Plan-text defects surfaced during execution** (collected here -so a 22-arc audit / next-iter brainstorm can act on them): - -- The plan's RED test for Task 5 used `Term::Seq` with `lhs: - Str`, but Seq requires `lhs: Unit`. Implementer substituted - `Term::Let { name: "_a", ... }`. The spec/plan template needs - a "discard-and-continue" idiom note. -- The plan's fixpoint loop scaffold filtered targets only across - rounds, not within a round. Two same-type call sites in round - 1 would both append; implementer added `seen_this_round`. -- `Float` was speculatively included in `mono_symbol`'s primitive - table — `Float` is not in the language. Removed. -- Pre-existing `examples/test_22b2_*` fixtures had bare-Lit - instance bodies for arrow-typed methods. Worked under 22b.1/2 - because instance-body typecheck is deferred follow-up. - Mono-pass synth requires Lam-shaped bodies; affected fixtures - Lam-wrapped during this iter (`test_22b2_instance_present`, - `test_22b2_xmod_classmod`). Instance-body typecheck remains - deferred — that's the right place to catch this user-side. - -**Known debt / out-of-scope deferrals:** - -- Form-B prose projection for `Def::Class` / `Def::Instance` - remains deferred to 22b.4. `crates/ailang-surface/tests/round_trip.rs` - skip-list excludes `test_22b2_*` and `test_22b3_*`. -- DESIGN.md mono-symbol naming amendment (`#` → `__`) needs a - small edit in 22b.4 alongside the Prelude wiring. -- Primitive-name set is hard-coded in three sites - (`linearity.rs`, `lib.rs` x2, `mono.rs`). Reviewer flagged the - duplication; consolidation deferred to milestone-22 audit. -- `pub mod mono` instead of `mod mono; pub use mono::...;` - pattern was the deliberate plan-level call (multiple pub items - reach by integration tests). Audit may revisit if the - convention drifts further. - -**Next: 22b.4 — Prelude module + Form-B parser/printer arms.** -Prelude declares `Show`, `Eq`, `Ord` over `Int`, `Bool`, `Str` -(default-bearing where Decision 11 mandates), `print` re-routes -through `Show.show`, Form-B class/instance prose lands, round-trip -filter retires. End-to-end gate: any prior `print 42`-style fixture -must keep working through `Show.show#Int`. - -## 2026-05-09 — Iteration 22b.4a: Form-A parser+printer arms for ClassDef/InstanceDef (and forall-constraints gap fix) - -Closes the round-trip gap that 22b.1 deliberately deferred: every -`examples/*.ail.json` fixture now round-trips through -`parse(print(M))` with canonical-equal JSON, including the -`test_22b1_*`, `test_22b2_*`, `test_22b3_*` typeclass fixtures that -the skip-list excluded. 106 fixtures green (up from 70). Bench gates -all 0/0/0. - -**Per-task subjects (mirror commit messages):** - -- 22b.4a.1: form-a parser arm for ClassDef -- 22b.4a.2: form-a parser arm for InstanceDef -- 22b.4a.3: form-a printer arm for ClassDef -- 22b.4a.4: form-a printer arm for InstanceDef -- 22b.4a.4.5: form-a printer+parser arms for Type::Forall constraints (gap fix) -- 22b.4a.5: retire round-trip skip-list for class/instance fixtures -- 22b.4a.6: spec + DESIGN amendments — 22b.4 split, terminology fix, '__' separator, forall-constraints gap - -**S-expression form locked:** - -ClassDef: - - (class - (param ) - [(superclass (class ) (type ))] - [(doc "...")] - (method (type ) [(default )])*) - -InstanceDef: - - (instance - (class ) - (type ) - [(doc "...")] - (method (body ))*) - -Type::Forall constraints (extension to existing forall-type form): - - (forall (vars …) - [(constraints (constraint )+)] - ) - -**Iteration split rationale:** the brainstorm-original 22b.4 bundled -surface arms with the Prelude module + `int_to_str` C-runtime -primitive + end-to-end `show 42` fixture. Surface arms touch one -crate with no runtime risk; the Prelude work touches `runtime/`, -`ailang-codegen`, `ailang-check::builtins`, plus codegen wiring for -the new primitive. Different review surface, different risk profile, -different bench-gate exposure. The split (22b.4a / 22b.4b) is -recorded in the spec amendments section and is the substantive -follow-up for 22b.4b. - -**Terminology fix:** the original spec called the surface arms -"Form-B parser/printer arms". `crates/ailang-surface` is in fact -**Form-A** (parseable s-expression); `crates/ailang-prose` is Form-B -(one-way prose, no parser by design). The 22b.1 round-trip-skip-list -comment carried the same conflation. Both corrected in the spec -amendment. - -**Mid-iter gap fix:** Task 5's pre-flight surfaced that the existing -Form-A arms dropped `Type::Forall.constraints` (the field added in -22b.2). Four `test_22b2_*` fixtures regressed when the filter was -lifted. Fixed inline via 22b.4a.4.5 by extending `parse_forall_type` -and the `Type::Forall` arm of `write_type` to round-trip the -constraints clause. The skip-list had silently masked this gap -alongside the ClassDef/InstanceDef gap; retiring it required closing -both in the same iter. - -**Quality-review carry-overs:** - -- Strict duplicate-clause detection in the new parser arms - (`(param ...)`, `(superclass ...)`, `(doc ...)`, `(method (type - ...))`, `(method (body ...))`) — sibling parsers (`parse_fn`, - `parse_data`, `parse_const`) silently overwrite earlier clauses - with later ones. The new strict pattern is correct; the silent - pattern is the wart. Promoting siblings to the strict pattern is - out of scope for 22b.4a (it would expand to all existing fixtures - and require their own RED tests). Filed as known debt. -- `pos: 0` fallback in missing-required-clause errors — pre-existing - pattern across all def parsers; mirroring it in the new code - preserves consistency. Targeted fix would be cross-crate and - belongs in a separate iter. - -**Known debt (deferred to 22b.4b or later):** - -- Form-B (prose) printer arms for ClassDef / InstanceDef. One-way, - not gating; queued for the audit cycle or 22b.4b cleanup. -- DESIGN.md line 1749 ("the exact textual form is fixed in 22b - alongside the existing mangling scheme") still does not name - `__` directly. The load-bearing rationale was added at line 1607 - (the `show__Int` example paragraph); line 1749 is a forward - reference whose update is audit-cycle work. -- Strict duplicate-clause detection in `parse_fn` / `parse_data` - / `parse_const` — file as known asymmetry, do not retrofit in - 22b.4a. - -**Bench gates:** all three (`bench/check.py`, `bench/compile_check.py`, -`bench/cross_lang.py`) exited 0/0/0 (matches 22b.3 close). - -**Next iter (queued):** 22b.4b — Prelude module with `class Show -a where show : (a borrow) -> Str`, `instance Show Int`, the -`int_to_str` C-runtime primitive, codegen wiring, and an end-to-end -`show 42` fixture printing `42` through the mono pass. - -## 2026-05-09 — Iteration 22c: Milestone-22 acceptance fixture (and a mono-pass bug) - -The 22c user-class e2e fixture lands `examples/test_22c_user_class_e2e.ail.json`: -a user `class Foo a where foo : (a borrow) -> Int`, a user ADT -`IntBox = MkIntBox Int`, an `instance Foo IntBox` whose body matches -on the ctor and returns the wrapped int, and `main = do io/print_int -(foo (MkIntBox 42))`. Stdout `42`. - -**The fixture surfaced a real bug** in the 22b.3 monomorphisation -pass: `build_workspace_env` (`crates/ailang-check/src/mono.rs:367+`) -populated `env.module_types` but not `env.types` or `env.ctor_index`, -so `synth`-replay on an instance method body that referenced a -user-defined ADT failed with `unknown type: IntBox`. Every prior -22b.3 fixture used primitive instance types (Int / Bool), so the -gap was invisible until 22c. Fixed RED-first per CLAUDE.md. - -**Per-task subjects (mirror commit messages):** - -- `test: red for monomorphise_workspace unknown type on user ADT` - (`59e86b3`) — fixture + e2e test, FAILED for the right reason - (`unknown type: IntBox`). -- `fix: mono pass seeds env.types and env.ctor_index for user ADTs` - (`5c5180f`) — minimal fix in `build_workspace_env`, mirrors - `check_in_workspace` (`lib.rs:1099-1128`) for the same two tables. - RED test now green; full workspace tests green; bench gates 0/0/0. - -**Why this is a meaningful close, not a "just-fixture iter":** -when the 22c plan was written it explicitly anticipated this -candidate ("Most likely candidates: mono pass on instance where -the type is a user ADT"). The fact that the gap surfaced exactly -where the plan flagged it is evidence the milestone scope was -understood; the fix is local (24 LOC, one function) and the -related sites in `lib.rs` are reachable only through paths that -already populate the env correctly. No collateral is implied. - -**Milestone-22 acceptance check (final):** - -1. ✓ 22b.1 + 22b.2 + 22b.3 + 22b.4a + 22c JOURNAL entries committed. -2. (pending) Audit suite — `audit` skill runs next. -3. ✓ 22c user-class fixture compiles, runs, prints `42`. -4. ✓ Round-trip filter retired in 22b.4a; all `examples/*.ail.json` - pass round-trip. - -**Bench gates:** all three exited 0/0/0. - -**Next step:** dispatch `audit` skill to close milestone 22. - -## 2026-05-09 — Iteration 22-tidy: DESIGN.md and spec drift after milestone-22 close - -Audit (architect drift review) flagged three doc-drift sites -where DESIGN.md and the spec described an as-originally-planned -milestone 22 (Prelude shipping, `print` rewired through Show.show, -`Float` type, Form-B class/instance render in 22b.4) instead of -what actually shipped (no Prelude, primitive print primitives, -no Float, prose render queued post-22). The fix is mechanical -across three files; no code logic changes. - -**Per-task subjects:** - -- 22-tidy.1: DESIGN.md — reconcile Decision 11 with milestone-22 outcome (no Prelude, '__' separator) -- 22-tidy.2: spec — components table reflects 22b.4a/b split + 22c shipped -- 22-tidy.3: prose-lib — drop 22b.4 reference from class/instance placeholder - -**Milestone-22 status: CLOSED.** The four acceptance criteria from -the spec are all satisfied: - -1. JOURNAL entries committed: 22b.1, 22b.2, 22b.3, 22b.4a, 22c. -2. Audit suite green — drift report addressed (this iter); bench - gates 0/0/0 at 22c close (rerun confirmed at milestone-22 audit - on 2026-05-09). -3. 22c user-class fixture compiles, runs, prints `42`. -4. Round-trip skip-list retired in 22b.4a; all 106 fixtures in - `examples/*.ail.json` pass round-trip. - -**Carried debt (not gating, not in this iter's scope):** - -- Primitive-name set duplicated across 4 sites - (`crates/ailang-check/src/mono.rs:316-327`, `linearity.rs:143`, - `lib.rs:1240`, `lib.rs:2222`). Flagged in 22b.3 JOURNAL, - re-flagged by milestone-22 audit. Consolidation is a real - refactor (one helper, four call sites) and benefits from being - its own iter. Queued. -- Strict duplicate-clause detection in `parse_fn` / `parse_data` - / `parse_const` (sibling parsers to the 22b.4a additions). The - asymmetry is documented; promoting siblings to the strict - pattern is queued. -- Form-B (prose) printer arms for ClassDef / InstanceDef. One-way, - not gating. -- Lib.rs gap-related sites at lib.rs:1266 / 1853-1856 / 1978 / - 2314-2316 — reachable only through paths that already populate - the env correctly, but the same discipline that was applied in - mono.rs (workspace-wide flat tables) could be hardened - defensively. Not currently broken. - -**What's next:** orchestrator's call. The JOURNAL queue's -substantive items per recent entries are: post-22 Prelude -milestone (gated on user-author demand for primitive -`Show`/`Eq`/`Ord`), operator routing through `Eq`/`Ord` (gated on -bench re-baselining), the primitive-name-set consolidation, or -unrelated work. The milestone-cycle dictates `brainstorm` for -whichever lands next. - -## 2026-05-10 — Bench: mono-vs-virtual-dispatch micro-benchmark - -Hypothesis-driven `ailang-bencher` run, prompted by the open -question "did monomorphisation actually buy us performance, or is -it purely a correctness/architectural choice?" Decision 11's -original framing in DESIGN.md (line 1462) reads "no runtime cost, -no dictionary passing, no vtables" — a true statement at the -mechanical level, but the **rationale** for *why* mono is faster -than vdisp had never been measured. This iter measures it. - -**Setup.** 100M-iter tail-recursive hot loop, body -`acc' = acc + foo(acc + i)` with `foo(x) = x*1103515245 + 12345` -(LCG step, defeats closed-form folding via serial dependency). -Int-only args → zero RC traffic, isolating dispatch-shape from -allocator effects. - -Five binaries, all clang -O2 (matches AILang lower path), Zen 3 -(Ryzen 5900X), 15 runs each (slowest dropped, median reported): - -| binary | median (s) | ratio | -|-------------------------|------------|----------| -| AILang mono | 0.0435 | 1.000x | -| C direct (inlinable) | 0.0435 | 1.000x | -| C direct (noinline) | 0.1439 | 3.310x | -| C indirect (mono fnptr) | 0.1440 | 3.310x | -| C indirect (4-fnptr) | 0.1739 | 4.000x | - -**Three substantive findings:** - -1. **AILang mono'd code is bit-for-perf-identical to hand-C - direct (1.000x).** No codegen-quality gap; LLVM treats the - mono'd direct call exactly as it does a normal C call. -2. **Inlining is the actual win (3.31x).** When the callee body - is visible, clang vectorises and unrolls the loop. The - `direct-noinline` variant — same direct call, but blocked from - inlining — runs identically to the indirect-monomorphic - variant, which is the empirical core of this iter. -3. **Indirect-monomorphic dispatch is essentially free on Zen 3 - (1.000x vs direct-noinline).** The branch predictor saturates - the BTB on the single target. Polymorphic indirect (4 distinct - fnptrs cycling) adds 21% on top — the real-world dict-passing - penalty in a multi-instance codebase. - -**The bench refines Decision 11's rationale.** The original -framing implicitly argued that mono avoids per-call indirect-jump -cost. That argument does not hold on modern x86 with a saturating -branch predictor. The correct argument is one level up: **mono -makes the call target visible to the optimiser, which unlocks -inlining and downstream loop transformations that virtual -dispatch prevents in principle.** The 3.3x measured here is an -*upper bound* (tiny callee, hot loop, ideal-case inlining); on -larger callee bodies or cold call sites the inlining win shrinks -toward zero. But the architectural claim — "mono enables -optimisations vdisp forbids" — survives across the whole -spectrum, while the original "saves an indirect call" framing -does not. - -**End-to-end win for the user on this fixture:** 3.3x vs -monomorphic vdisp, 4.0x vs polymorphic vdisp. - -**Limitations (binding):** - -- Synthetic micro-bench, friendliest possible case for inlining. - Real programs span the inlining-budget spectrum. -- Int-only args → zero RC traffic. Heap-typed args would - additionally cost dict-RC traffic under hypothetical vdisp; - this bench does not measure that. -- AMD Zen 3 has a strong predictor. Older x86 / ARM would show - larger indirect-vs-direct deltas on the monomorphic case. -- Single-arity, single-instance call site. A multi-instance - polymorphic call site would hit the 1.21x predictor-miss regime. - -**Side effect: mono-pass `env.globals`-seeding bug surfaced.** -While building the AILang fixture, a self-recursive top-level fn -in a class-bearing module trips -`monomorphise_workspace: unknown identifier`. Root cause: -`mono::collect_mono_targets` (`crates/ailang-check/src/mono.rs` -near line 475) does not seed `env.globals` from -`env.module_globals[mname]` before calling `synth`, unlike the -working pattern in `lib.rs:1135-1139`. The bencher worked around -it by wrapping the recursive loop as a class method; a regular -non-class recursive fn in a class-bearing module fails to -compile today. RED-first debug iter follows. - -**Files added:** - -- `examples/bench_mono_dispatch.ail.json` — AILang side fixture -- `bench/reference/bench_mono_direct.c` — direct-inlinable C -- `bench/reference/bench_mono_direct_noinline.c` — direct-noinline C -- `bench/reference/bench_mono_indirect.c` — indirect-monomorphic C -- `bench/reference/bench_mono_indirect_polymorphic.c` — indirect-poly C -- `bench/mono_dispatch.py` — harness (median-of-15, slowest-drop) - -**Action items consumed by this iter:** - -- DESIGN.md Decision 11 — performance-rationale paragraph appended, - pointing at this entry and reframing mono as inlining-enabler. - -**Action items spawned by this iter:** - -- RED-first debug iter for the `env.globals` mono bug. - -## 2026-05-10 — Bug fix: mono-pass `env.globals` not seeded — recursive top-level fn in class workspace - -Surfaced by the mono-vs-vdisp bench (entry above). RED-first per -`skills/debug` discipline; one RED-test commit, one GREEN-fix commit. - -**Symptom.** `ail build` on a workspace containing one `Def::Class`, -one `Def::Instance`, and a self-recursive top-level `Def::Fn` -exits with `monomorphise_workspace: unknown identifier `. -`ail check` on the same fixture passes — proof that the typechecker -is fine and the gap is in the mono pass. - -**Cause.** `crates/ailang-check/src/mono.rs::collect_mono_targets` -(line 463) and `collect_residuals_ordered` (line 819) both -re-run `crate::synth` on a fn body to recover residual class -constraints, but their `env` (built by `build_workspace_env`) only -populates `module_globals` (per-module index) — not `globals` (the -flat current-module table that `synth`'s `Term::Var` lookup at -lib.rs:1678 actually reads). The main check path handles this at -lib.rs:1135-1139, seeding `env.globals` per module before `synth` -runs; mono.rs did not. Sibling gap to commit 5c5180f's -`env.types` / `env.ctor_index` fix from milestone 22c — same -shape, different env table. - -**Fix.** Two-line seed in each of the two `synth`-rerun call -sites: `if let Some(g) = env.module_globals.get(module_name).cloned() { for (n, t) in g { env.globals.insert(n, t); } }`. -Per-module scope (not workspace-wide), because top-level fn names -are only per-module-unique — `build_module_globals` -(lib.rs:1011-1019) enforces uniqueness within a module, not -workspace-wide. A flat workspace seed would silently shadow -same-named fns across modules. The semantically-correct mirror of -lib.rs:1135-1139 is per-module, not the workspace-wide pattern -5c5180f used for types/ctors (which are workspace-unique). - -**Commits:** - -- `3b0bcf3` — `test: red for mono-pass unknown identifier on recursive fn in class workspace` - - RED test: `crates/ail/tests/mono_recursive_fn.rs::mono_pass_handles_recursive_fn_in_class_workspace` - - Fixture: `examples/test_mono_recursive_fn_bug.ail.json` -- `13b36cc` — `fix: mono pass seeds env.globals for recursive top-level fn in class workspace` - -**Verification:** RED test passes, full workspace tests green, -all three bench gates 0 regressed (`bench/check.py`, -`bench/compile_check.py`, `bench/cross_lang.py`). - -**Carried debt (untouched per Iron Law):** - -- Drift-risk comment in `build_workspace_env` says "If `synth` - starts reading a new `Env` field, update both paths" — accurate - but `env.globals` is now seeded in two further places (the - `collect_*` fns themselves) which the comment doesn't mention. - Cosmetic; left alone for the next consolidation pass. -- The four lib.rs gap-related sites flagged in 22-tidy - (`lib.rs:1266 / 1853-1856 / 1978 / 2314-2316`) and the - primitive-name-set consolidation remain queued. This bug - proved one of the env-seeding "gaps" from 22-tidy was - load-bearing, not defensive — a future iter that audits all - four flagged sites is worth its cost. - -## 2026-05-10 — Audit: mono-pass env-seeding gaps (lib.rs sites + env.imports) - -Targeted audit of the four `lib.rs` env-table reads that 22-tidy -flagged as gap-related, prompted by the env.globals fix (entry -above). Found a fifth gap that 22-tidy had not flagged. - -**The four flagged sites** all read from `env.types` or -`env.ctor_index`: - -- `lib.rs:1266` — `check_type_well_formed`, bare `Type::Con` lookup -- `lib.rs:1853-1856` — `Term::Ctor` bare-name path -- `lib.rs:1978` — `Term::Match` exhaustiveness, bare scrutinee type -- `lib.rs:2314-2316` — pattern-ctor resolution, local hit + type lookup - -`env.types` and `env.ctor_index` are seeded workspace-flat by -`build_workspace_env` since commit 5c5180f (milestone 22c). -**All four flagged sites are therefore defensive, already -covered.** They were load-bearing *for* the 22c fix; after the -fix, they are protected. - -**The unflagged fifth gap: `env.imports`.** Walking the env -struct (`lib.rs:2447-2509`) field by field against -`build_workspace_env` (`mono.rs:367`) revealed that -`env.imports` is read at four sites in `lib.rs` — -`1254` / `1697` / `1836` / `2320` — but `build_workspace_env` -never seeds it. The typecheck path at lib.rs:1147-1152 builds -`env.imports` per-module from `m.imports`; the mono pass had -no analogue. RED-first repro: - -- Two-module workspace: `test_mono_imports_classmod` - (class + instance) and `test_mono_imports_main` (imports - classmod, has a top-level fn body referencing - `test_mono_imports_classmod.foo` qualified). -- `ail check` passes, `ail build` fails with - `monomorphise_workspace: unknown module prefix - test_mono_imports_classmod in qualified reference`. - -**Commits:** - -- `2bf827f` — `test: red for mono-pass unknown module prefix on qualified cross-module reference in class workspace` - - Fixture: `examples/test_mono_imports_classmod.ail.json` + - `examples/test_mono_imports_main.ail.json` - - RED test: `crates/ail/tests/mono_xmod_qualified_ref.rs` -- `a9c685d` — `fix: mono pass seeds env.imports for qualified cross-module refs in class workspace` - - New `Env::module_imports: BTreeMap>` field - - `build_workspace_env` populates it from `Workspace::modules` - - `collect_mono_targets` and `collect_residuals_ordered` seed - `env.imports` per-module from it after setting `current_module` - - Mirrors the per-module pattern of `env.globals` (commit 13b36cc), - not the workspace-flat pattern of `env.types` (5c5180f) — import - aliases collide across modules. - -**Verification:** RED tests for both today's bugs (`mono_recursive_fn`, -`mono_xmod_qualified_ref`) green; full workspace tests green; bench -gates 0 regressed. - -**Architectural observation (not actioned this iter).** Three -consecutive commits (5c5180f, 13b36cc, a9c685d) have patched the -drift between `build_workspace_env` (mono.rs:367) and -`check_in_workspace` (lib.rs near 1095) at three different env -fields. The pattern is: every time `synth` learns to read a new -env table, both construction paths must be updated, but only one -gets it. The third occurrence is the signal — this is no longer -"audit one more place"; this is structural. Two paths to construct -the same env shape diverge under feature pressure. - -**Queued for the next iteration:** unify the two env-construction -paths, OR introduce a shared helper that both call. The shape that -fits the existing code is a `build_check_env(ws: &Workspace, -current_module: Option<&str>) -> Env` in `lib.rs` (or -`workspace.rs`), with `check_in_workspace` and the mono pass both -calling into it. That removes the drift entirely, and a future -"synth needs to read env.X" change touches one site. - -This is now the strongest item in the JOURNAL queue ahead of -post-22 Prelude / operator-routing / primitive-name-set -consolidation. It is not gating any user-facing feature, but it is -where the next "mysterious mono failure" will come from if not -addressed. - -## 2026-05-10 — Iteration env-construction unify - -Single-iteration milestone retiring the structural drift between -`check_in_workspace` and `mono::build_workspace_env`. Three -consecutive bug fixes (`5c5180f` env.types/ctor_index, `13b36cc` -env.globals, `a9c685d` env.imports) had been patches at three -different fields of the same drift class. The unify replaces both -construction paths with a single source-of-truth helper -`build_check_env(ws) -> Env` covering the workspace-flat fields -(builtins, module_globals + class_methods, module_types, -module_imports, class_superclasses, workspace_registry, plus -workspace-flat types/ctor_index for the mono pass). Per-call -overlay (`current_module`, `globals`, `imports`, `rigid_vars`, plus -per-module `types`/`ctor_index` for `Pattern::Ctor`'s local-first -resolution) stays at the call sites where it semantically belongs. - -After this iteration, when `synth` learns to read a new -workspace-flat `Env` field, exactly one construction site needs -the seeding edit. A new `crates/ailang-check/tests/env_construction_pin.rs` -integration test serves as the tripwire: any future divergence -between the helper and a frozen reproduction of the inline seeding -fails the test before the bug ships. - -Design corrections during execution (recorded for the audit -artifact): - -- The pin test originally asserted `env.globals.is_empty()` on the - helper output, treating `globals` as pure overlay. `builtins::install` - populates BOTH `globals` (with operator entries `+`, `-`, `==`, - `not`, `__unreachable__`) AND `effect_ops` from one call, so - `globals` is partially workspace-flat (operators) and partially - overlay (per-module fns). Pin test fixed to key-set equality. -- The original spec listed `types` and `ctor_index` as workspace-flat - fields. That was true for `mono::build_workspace_env` (since - `5c5180f`) but NOT for the pre-refactor `check_in_workspace`, - which seeded them per-module — `Pattern::Ctor`'s - local-first / imports-fallback logic depends on the per-module - shape to surface the qualified `module.Type` name on cross-module - pattern matches. Resolution: `build_check_env` keeps the - workspace-flat seed (mono needs it), and `check_in_workspace`'s - per-module overlay clears and rebuilds these two fields per-module - with the original in-band fail-fast `DuplicateType` / - `DuplicateCtor` diagnostics. - -Tasks (commit subjects): - -- `test: red for env-construction drift-shape pin` (Task 1) -- `test: clean red signal for env-construction pin (inline module_types)` (Task 1 fixup) -- `test: fix env-pin globals assertion (builtins seed operators workspace-flat)` (mid-Task-2 correction) -- `iter env-unify.1: extract build_check_env, mono uses it` (Task 2) -- `iter env-unify.1: rustdoc fixup (re-anchor check_in_workspace doc, drop transient task tag)` (Task 2 fixup) -- `iter env-unify.2: check_in_workspace consumes build_check_env` (Task 3) -- `iter env-unify.2: drop redundant overlay comments` (Task 3 fixup) - -Spec: `docs/specs/2026-05-10-env-construction-unify.md`. Plan: -`docs/plans/2026-05-10-env-construction-unify.md`. Tests: pin test -green; the three RED tests (`typeclass_22c`, `mono_recursive_fn`, -`mono_xmod_qualified_ref`) remain green; `cargo test --workspace` -green at 345 tests; bench gates 0/0/0 (`bench/check.py` 0 regressed + -4 improved beyond tolerance, `bench/compile_check.py` 0 regressed, -`bench/cross_lang.py` 0 regressed). - -Known debt: none introduced. Out-of-scope items from the spec -(pipeline-topology changes, per-module overlay refactor, -primitive-name-set consolidation, new env fields) remain queued -for separate milestones. - -## 2026-05-10 — Audit close: env-construction unify - -Architect drift review of diff `08abfdf..e414144` found three -low-medium items. Bench gates 0/0/0; rustdoc 16→15 warnings (one -removed by the milestone, none added); E2E coverage map produced -by tester confirmed existing 345-test suite + pin tripwire pin -every named invariant. - -Resolution: - -- `[medium]` Stale doc comment on `Env::module_imports` - (`crates/ailang-check/src/lib.rs:2508-2516`) said "populated only - by `mono::build_workspace_env`" and "`check_in_workspace` does - not populate this." Both clauses contradicted the post-unify - shape. Fixed inline as part of audit close (one-line - doc tidy). Same drift on `Env::module_globals` (line 2506) - fixed for consistency. -- `[low]` Pin test docstring - (`crates/ailang-check/tests/env_construction_pin.rs:26-30`) - claimed it reproduced "the pre-refactor `check_in_workspace` - workspace-flat seeding," but pre-refactor `check_in_workspace` - seeded `types`/`ctor_index` per-module, not workspace-flat. The - test still pins `build_check_env` against an independent - reproduction (its real value); the comment misnamed what was - pinned. Fixed inline. -- `[medium, queued]` `env.types.clear()` / `env.ctor_index.clear()` - immediately after `build_check_env` populates them - (`lib.rs:1187-1188`) is the visible scar of a deeper shape - mismatch: `types`/`ctor_index` are *per-module overlay* for - typecheck (`Pattern::Ctor`'s local-first / imports-fallback - resolution at lib.rs:2314-2316) but *workspace-flat* for the - mono pass. One field, two shapes. Not a fire (the wasteful - copy is small and bench gates are clean), but worth queuing - as a follow-up shape question for whichever iteration next - feels env-field pressure. Marker: "types/ctor_index overlay - shape" in the JOURNAL queue. - -Milestone closed: drift items 1 and 3 tidied; item 2 queued. - -## 2026-05-10 — Iteration 22-tidy.4: primitive-name-set consolidation - -Closing the milestone-22 carried-debt item flagged in the 22b.3 -JOURNAL and re-flagged at milestone-22 audit close: the -primitive-name set `{Int, Bool, Str, Unit}` was duplicated across -five sites (`crates/ailang-codegen/src/subst.rs:169`, -`crates/ailang-check/src/linearity.rs:143`, -`crates/ailang-check/src/lib.rs:1279` and `:2261`, -`crates/ailang-check/src/mono.rs:316-327`). - -This iteration introduces `crates/ailang-core/src/primitives.rs` -exporting `is_primitive_name(&str) -> bool` plus -`primitive_surface_name(&str) -> Option<&'static str>`. The four -`matches!` consumer sites now route through the predicate; the -mono-pass surface-name helper retains its outer zero-arity gating -and delegates the inner mapping to the same module. A unit test -`predicate_and_surface_name_agree` in `primitives.rs` pins the -lockstep invariant between the two functions (a future regression -where only one is extended fails the test before it can ship). - -Tasks (commit subjects): - -- 22-tidy.4: ailang-core::primitives — single home for the - primitive-name set -- 22-tidy.4: pin lockstep invariant + tighten module doc -- 22-tidy.4: route 4 matches! sites through is_primitive_name -- 22-tidy.4: mono::primitive_surface_name delegates to ailang-core - -Acceptance: 4 `matches!` sites + 1 mono.rs site routed; full -workspace test sweep 346 green (345 + the new lockstep test); -bench gates 0/0/0; grep confirms no remaining duplicate predicate -outside `crates/ailang-core/src/primitives.rs`. - -The audit-flagged milestone-22 carried-debt item closes; the -remaining carried items (parse-fn/data/const strict -duplicate-clause detection, Form-B prose printer arms for -ClassDef/InstanceDef, defensive lib.rs gap-related sites) stay -queued. - -Observation, not debt: `crates/ailang-codegen/src/drop.rs:379` has -a `matches!(name.as_str(), "Str")` single-element check (not the -4-set) — distinct invariant (heap-string-only semantics), out of -scope for this consolidation. - -## 2026-05-10 — Iteration 22-tidy.5: strict duplicate-clause detection in fn/const/data parsers - -Closing the second milestone-22 carried-debt item flagged in -`2026-05-09 — Iteration 22-tidy`: the strict duplicate-clause -detection that 22b.4a introduced for `parse_class` / -`parse_instance` / their method sub-parsers (10 sites) was missing -from the sibling parsers `parse_fn`, `parse_const`, `parse_data` — -those silently last-wins on duplicate clauses. This iteration -lifts the strict pattern to those three siblings. - -Sites promoted to strict (8 total): - -- `parse_fn`: `doc`, `type`, `params`, `body` -- `parse_const`: `doc`, `type`, `body` -- `parse_data`: `doc` (only — `ctor` is multi-instance, - `drop-iterative` was already strict from Iter 18e) - -Each new strict check uses the established 22b.4a pattern -(`if X.is_some() { return ParseError::Production { … } }`) and -the established message form -(`` " `` has duplicate `( ...)` clause" ``). -Eight new RED-first unit tests in `parse.rs::mod tests` pin each -diagnostic. - -Mid-iter fixture correction (Task 2): the planned `parse_const` -body fixtures used `(body (lit (int 42)))`, but bare integer -literals in the surface are tokenised directly (`Tok::Int`), not -as a `(lit (int N))` term. Implementer substituted bare integer -literals (`(body 42)`, etc.) — preserves the load-bearing -duplicate-clause assertion. - -Mid-iter fixup (Task 2 review): Task 1 originally added -`/// Iter 22-tidy.5: …` rationale doc-comments to the four -`parse_fn` tests, but Task 2 omitted them per CLAUDE.md comment -policy ("Don't reference the current task, fix, or callers"). -For consistency, the iter-tag doc-comments were stripped from -Task 1's tests in commit `9097b88`. - -Tasks (commit subjects): - -- 22-tidy.5.1: parse_fn rejects duplicate doc/type/params/body clauses -- 22-tidy.5.2: parse_const rejects duplicate doc/type/body clauses -- 22-tidy.5.2: drop transient iter-tags from parse_fn duplicate-clause tests -- 22-tidy.5.3: parse_data rejects duplicate doc clause - -Acceptance: 8 new tests RED-first then GREEN; full workspace test -sweep green (`cargo test --workspace` 0 FAILED across 23 test -binaries); bench gates 0/0/0; no behaviour change for valid -fixtures (the round-trip suite stays green). The strict-vs-loose -asymmetry between class/instance and fn/const/data parsers is -retired. - -Observation (out-of-scope, queued): the duplicate-clause check -pattern now appears at 18 sites across the parser (10 from 22b.4a -+ 8 from this iter). A `check_clause_unique!` macro or -`fn duplicate_clause_err(production, name, clause, pos)` helper -would consolidate them. Quality reviewer flagged as Nit during -Task 1 review; left for a future iter. Three similar lines beats -a premature abstraction; eighteen similar blocks may be the -tipping point. Re-evaluate next time the strict pattern needs -extending. - -Carried-debt status (after this iter): - -- ✅ Primitive-name-set consolidation (closed 22-tidy.4). -- ✅ Strict duplicate-clause detection in fn/const/data (this iter). -- ⏳ Form-B (prose) printer arms for ClassDef / InstanceDef — still - queued. -- ✅ Lib.rs gap-related sites at lib.rs:1266 / 1853-1856 / 1978 / - 2314-2316 — audit confirmed defensive (no action needed; see - `2026-05-10 — Audit: mono-pass env-seeding gaps`). - -## 2026-05-10 — Iteration 22-tidy.6: Form-B prose printer arms for ClassDef + InstanceDef - -Closing the third (and final gating) milestone-22 carried-debt -item: `crates/ailang-prose` previously rendered `Def::Class` / -`Def::Instance` as a one-line placeholder -(`// (class Foo a) -- full Form-B projection deferred (post-22)`). -This iteration replaces the placeholder with a full Rust-flavoured -projection covering class header, optional `extends `, -methods (signature plus optional `default { ... }` body), and -instance header + method bodies. - -`write_fn_def` was extended in the same iter to render forall -class constraints (`forall where Ord a`) so a class-constrained -fn signature survives the projection without semantic loss — a -prerequisite for the superclass snapshot to be useful. - -Three new snapshot fixtures pin every branch of the new render: - -- `examples/test_22b3_default_e2e.prose.txt` — class-with-default + - instance-with-no-method-overrides (no superclass, default Some). -- `examples/test_22b2_instance_present.prose.txt` — class-with-abstract - method + instance-with-method-override (default None, instance - methods non-empty). -- `examples/test_22b2_constraint_declared_via_superclass.prose.txt` - — class-with-superclass + class-constrained fn (extends Some, - forall constraints non-empty). - -Tasks (commit subjects): - -- 22-tidy.6.1: write_class_def + write_instance_def — full Form-B projection -- 22-tidy.6.1 fixup: cover abstract-method + instance-override branches; doc + fallback comments -- 22-tidy.6.2: write_fn_def renders forall class constraints - -Acceptance: 3 RED-first snapshot tests then GREEN; full workspace -test sweep green; bench gates 0/0/0; the four pre-existing prose -snapshots (rc_own_param_drop, rc_match_arm_partial_drop_leak, -rc_app_let_partial_drop_leak, bench_list_sum) stay green -(unaffected — none contain class/instance). The placeholder text -and comment are gone from `crates/ailang-prose/src/lib.rs` -(grep confirms zero hits for "full Form-B projection deferred"). - -Mid-iter design notes: - -- Class methods render with synthesised parameter slot names - (`x`, `x1`, `x2`, ...) because the AST stores method types - but no parameter names. Inline comment in `write_class_method` - records the rationale. -- Class-method `default` bodies render as their underlying AST - shape, which in practice is `Term::Lam` (the parser wraps the - surface `default` body as a closure). So `default { |x: a| -> Int - { 99 } }` is the faithful render — the closure form is the AST, - not the renderer wrapping it. -- Instance methods render as `fn { }` without a - signature. The signature is recoverable from the class - declaration via `InstanceDef.class` lookup - (typecheck-context-dependent substitution of `class.method.ty` - with `class.param -> instance.type_`). Per the prose policy - ("deliberately lossy where the LLM can re-derive the dropped - machinery from typecheck context"), this is the correct - projection — the class def is the authoritative signature - source. -- Plan's Step 4 prescribed adding `Constraint` to the - `use ailang_core::ast::{...}` line, but field access through - `c.class` / `&c.type_` does not require naming the type; - implementer correctly omitted the import to avoid an - `unused_imports` warning. - -Carried-debt status (after this iter, milestone 22 fully closed): - -- ✅ Primitive-name-set consolidation (closed 22-tidy.4). -- ✅ Strict duplicate-clause detection in fn/const/data - (closed 22-tidy.5). -- ✅ Form-B (prose) printer arms for ClassDef / InstanceDef - (closed this iter). -- ✅ Lib.rs gap-related sites at lib.rs:1266 / 1853-1856 / 1978 / - 2314-2316 — audit confirmed defensive (no action needed). - -Milestone 22 carried debt is now empty. Next milestone targets -(orchestrator's call): post-22 Prelude (gated on user-author -demand for primitive `Show`/`Eq`/`Ord`), operator routing through -`Eq`/`Ord` (gated on bench re-baselining), or the -`types/ctor_index` overlay shape question queued from -`2026-05-10 — Audit close: env-construction unify`. - -Out-of-scope observation (not new debt — quality-review nit -surfaced during Task 2): `write_type`'s `Type::Forall` arm -still silently drops constraints in inline-type rendering -(`crates/ailang-prose/src/lib.rs` `write_type` Type::Forall arm). -The inline path is unreachable for the surface forms emitted by -the parser today (forall only appears at fn-signature top level), -so the gap is dormant. If a future feature carries a forall in -inline position with constraints, this is the place to extend. - -## 2026-05-10 — Iteration 22-tidy.7: strict-clause helper consolidation - -Closing the queued observation from 22-tidy.5: the -duplicate-clause check pattern had grown to 17 inline blocks -(10 from 22b.4a + 8 from 22-tidy.5; the JOURNAL entry's "18 sites" -was an off-by-one) of identical 12-line shape across -`parse_data`, `parse_fn`, `parse_const`, `parse_class`, -`parse_class_method`, `parse_instance`, `parse_instance_method`. -17 × 12 LOC of textual duplication is past the tipping point; -this iter consolidates them. - -Introduced one private method -`Parser::duplicate_clause_err(&self, production: &'static str, -subject: &str, clause: &'static str) -> ParseError` that captures -the `pos` lookup, formats the canonical -`" has duplicate `( ...)` clause"` message, and -returns the `ParseError::Production` shape. All 17 inline blocks -reduced to one-line calls; net -99 LOC on `parse.rs` (31 ins, -130 del). - -Three subject shapes are unified under the one helper by passing -the formatted subject as a parameter: - -- `&format!(" \`{name}\`")` for the 13 named productions - (data ×1, fn ×4, const ×3, class ×3, class method ×2) -- literal `"instance"` for the 3 nameless instance forms -- `&format!("instance method \`{name}\`")` for the 1 instance method form - -The `production` tag stays separate (it's the diagnostic code, -not the human-readable text). All produced messages are -byte-identical to pre-refactor messages — the 8 duplicate-clause -unit tests from 22-tidy.5 (4 fn + 3 const + 1 data) continue -passing against verbatim assertion strings. - -Tasks (commit subjects): - -- 22-tidy.7: extract duplicate_clause_err helper, retire 17 inline blocks - -Acceptance: `cargo test --workspace` 0 FAILED; bench gates 0/0/0; -net LOC reduction confirmed (-99 LOC). Quality reviewer noted the -call-site lines now exceed ~120 cols at some sites, but the repo -has no enforced column limit and rustfmt accepts the shape. - -Observation (out-of-scope, queued): the 9 22b.4a-era duplicate- -clause sites (class ×3 + class method ×2 + instance ×3 + -instance method ×1) have no dedicated unit tests — they were -shipped without a regression pin. The 22-tidy.5 tests cover -fn/const/data only. Backfilling tests for the 22b.4a-era sites is -defensible but beyond the carried-debt scope; queue under -"parser-test backfill" if and when a 22b.4a-era diagnostic ever -needs to change. - -Carried-debt status (after this iter): - -- ✅ Primitive-name-set consolidation (22-tidy.4). -- ✅ Strict duplicate-clause detection in fn/const/data (22-tidy.5). -- ✅ Form-B (prose) printer arms for ClassDef / InstanceDef - (22-tidy.6). -- ✅ Strict-clause helper consolidation (this iter). -- ✅ Lib.rs gap-related sites — audit confirmed defensive. - -Milestone-22 carried debt + the queued tipping-point observation -are now both closed. Next milestone targets remain orchestrator's -call. - -## 2026-05-10 — Iteration design-md-consolidation 1: history-anchor sweep - -First iteration of the milestone defined in -`docs/specs/2026-05-10-design-md-consolidation.md`. Sweep 1 closes -the "history anchors in DESIGN.md" item: every iter tag (`Iter Nx`, -bare `Nx`, `pre-Nx`, `Nx sketch`, `21'g`), family tag (`Family 20`), -date anchor (`2026-MM-DD`), status marker (`**Status: …**`), and -historical bench data-point that anchored the document to a specific -moment is removed or condensed. DESIGN.md now reads as a state-only -document for these classes of anchors; the *narratives* of how -decisions changed (Decision 9's three-frame story, Decision 11's -mono-vs-vdisp correction, Decision 7's REVERTED body, the "Migration -plan" in Decision 10) remain in their narrative form this iter and -are condensed in sweep 2. - -Sites stripped (counts at iter start, full closure on commit): - -- Iter tags (`Iter [0-9]+[a-z]?(\.[0-9]+)?` strict): 93 sites (Task 4). -- Bare iter-id residues (`pre-Nx`, `Nx sketch`, plain `Nx` like `14d`, - `15a`, `20f`, `22a`, `22b`, `22c`, `18g.1`, `16b.2`): 4 + ~16 sites - (Task 4 fixups). -- Family tags: 4 sites (Task 2). -- Date anchors: 13 sites (Task 3). -- Status markers (`**Status: …**`): 4 sites (Task 1). -- Bench data-points: 12 sites — 5 KEPT (the 1.3× retirement target - and ±15% tolerance, which are policy contracts), 7 REMOVED or - CONDENSED (Task 5). - -Specific anchor examples discarded (representative): - -- Decision 6 opener: `**Status: shipped.** Form (A) was chosen in - Iter 14b and implemented as` → `Form (A) is implemented as`. -- Decision 9 opener: `**Status: half-retirement as of 2026-05-09.** - Originally framed (2026-05-07) as "transitional Boehm…"` → - `Originally framed as "transitional Boehm…"` (the - narrative-of-changes paragraph is sweep-2 territory). -- Decision 10 opener: `**Committed 2026-05-08, after the GC bench - showed Boehm contributing ~60% of runtime…` → `**The GC bench - showed Boehm contributing a substantial fraction of runtime - (bench notes in JOURNAL).` -- Decision 11 opener: `**Committed 2026-05-09 (Iter 22a), as the - design pass that gates 22b implementer work.**` → `**The design - pass that gates implementer work.**` - -Deliberate exception: -`docs/specs/2026-05-09-22-typeclasses.md` (line 1747) — date-prefixed -spec filename, kept verbatim because the on-disk file uses that -prefix and renaming would lose git history. The composite -acceptance grep masks paths beginning with `docs/`. - -Acceptance: - -- composite grep - `grep -nE 'Iter [0-9]+[a-z]?(\.[0-9]+)?|Family [0-9]+|^[^/]*2026-[0-9]{2}-[0-9]{2}|\*\*Status: |pre-[0-9]+[a-z]?|[0-9]+[a-z]? sketch|21.g' docs/DESIGN.md` - returns empty. -- bench-data residue grep returns only 5 KEEP-class lines (1.3× and - ±15%). -- `cargo test --workspace` 0 FAILED across all 23 test binaries. -- `bench/check.py` 0 regressed (63 metrics, 2 improved beyond - tolerance, 61 stable); `bench/compile_check.py` 0 regressed (24 - metrics, 24 stable). -- DESIGN.md size: 2262 → 2253 lines (small net reduction; sweeps 2-4 - expected to reduce further). - -The 1.3× Boehm-retirement target and the ±15% closure-band tolerance -are KEPT verbatim — they are policy contracts, not historical -measurements. The historical bench data-points that grounded those -targets (60% allocate-path overhead; 2.8× malloc slowdown; 4.14× -rc/bump on closure-chain; 3.31x / 4.00x mono-vs-vdisp; 1.000x Zen 3 -saturated-predictor anchor) are removed from DESIGN.md; their values -remain in JOURNAL bench entries (`bench/run.sh` baseline records, -mono-dispatch micro-bench entry). - -Tasks (commit subjects): - -- design-md-consolidation 1.1: drop **Status:** markers from DESIGN.md -- design-md-consolidation 1.2: drop Family-N tags from DESIGN.md -- design-md-consolidation 1.2 fixup: trim trailing whitespace on DESIGN.md:33 -- design-md-consolidation 1.3: drop date anchors from DESIGN.md -- design-md-consolidation 1.3 fixup: repair two date-strip quality issues -- design-md-consolidation 1.4: drop Iter-N tags from DESIGN.md -- design-md-consolidation 1.4 fixup: strip non-Iter-N iter anchors (14b sketch, pre-19b, pre-22a) -- design-md-consolidation 1.4 fixup: strip bare iter-id references (~16 sites) -- design-md-consolidation 1.4 nit: deduplicate 'fixture' word in user-class parenthetical -- design-md-consolidation 1.5: condense historical bench data-points in DESIGN.md -- design-md-consolidation 1.5 fixup: untangle line 863 run-on, deduplicate JOURNAL attribution, idiomatic 'larger still' - -Carried into sweep 2: Decision 7's body (still present, awaiting -removal), Decision 9's "Originally framed … then re-framed … -revision flips" narrative, Decision 10's "Migration plan", Decision -11's "Why mono, not virtual dispatch (the empirically-grounded -version)" correction history, and the "future iter may" -speculations. - -Process note: the plan's strict grep `Iter [0-9]+[a-z]?(\.[0-9]+)?` -caught 93 of the iter anchors but missed 20 bare-iter-id residues -(`pre-19b`, `14b sketch`, `14d`, `14e`, `15a`, `16b.2`, `18g.1`, -`20f`, `22a`, `22b`, `22b.4b`, `22c`). The implementer correctly -flagged these as "known debt" within the strict-grep boundary; the -orchestrator widened scope post-implementer-DONE because the spec's -intent (`all iter tags removed`) was clearly broader than the -plan's regex. Lesson for sweep 2-4 plan-writing: the acceptance -grep must match the spec intent, not just one specific anchor -form. Future plans for sweeps 2-4 will fold all anchor variants -(literal Iter-N, bare Nx, prefixed `pre-Nx`, possessive `Nx's`, -suffix `Nx sketch`, etc.) into one composite grep at plan-time. - -## 2026-05-10 — Iteration design-md-consolidation 2: REVERTED + migration + correction history + future speculations - -Second iteration of the milestone defined in -`docs/specs/2026-05-10-design-md-consolidation.md`. Sweep 2 closes -the "REVERTED + migration plans + correction history + future -speculations" item: the audit-trail preservation of Decision 7, -the 7-point Decision 10 Migration plan, the Decision 9 -narrative-of-changes opener, the Decision 11 mono-vs-vdisp -correction history, and the non-binding future-iter speculations -are removed or condensed. DESIGN.md now describes only the current -state plus the timeless rationale; the discarded narratives live -in JOURNAL (this entry plus the original iter entries that -recorded each change at the time). - -Sections deleted entirely: - -- Decision 7 (Term::If REVERTED block) — 32 lines. `Term::If` - exists in the language; the witness is the Term-schema - section. Decision numbering preserves the gap (Decision 6 → - Decision 8) so JOURNAL cross-references remain stable. One - orphan cross-reference at line 1141-1144 ("the same trade-off - Decision 7 made for `Term::If`'s relationship to nested - `Term::Match`") was caught in quality review and replaced with - the principle stated directly ("The principle: prefer - composability over schema-level rejection where the typecheck - rule is unambiguous"). -- Decision 10 §"Migration plan" — 32 lines, 7-point list of past - iters (annotation feature → RC runtime → uniqueness inference - → reuse hints → drop-iterative → validation bench → advisory - lint). The end state is described in the rest of Decision 10. - -Paragraphs condensed: - -- Decision 9 narrative-of-changes opener ("Originally framed as - X; then re-framed as Y; the revision flips Z") → one-line - state opener leading into the existing three RC/Boehm/bump - bullets. -- Decision 9 pre-Decision-9 history paragraphs ("Originally, - every ADT box, lambda env, and closure pair was allocated - with bare malloc and never freed…", and the bridging - "`--alloc=bump` mode introduced for the bench is a measurement - tool" sentence) — both removed. -- Decision 11 §"Why mono, not virtual dispatch (the - empirically-grounded version)" → state-only form. The - substantive rationale ("mono enables optimisations vdisp - forbids") survives; the correction history ("The original - rationale implicitly argued … the micro-benchmark refutes that - specific claim") is dropped. Header simplified to "Why mono, - not virtual dispatch." - -Future-iter speculations — per-site decisions: - -- `A future iter may layer a per-fn-arena optimisation` REMOVED - — the per-fn arena is shipped (next subsection describes it). - Announcement is history. -- `A future iter that proposes any of laziness, recursive value - bindings, shared mutable state, …` KEPT, TIGHTENED — restructured - from "future iter that proposes X must Y" to "X is rejected - unless Y" (assertive present-tense form of the same binding - constraint). -- `a future iteration (deferred) makes the explicit annotation - mandatory and rejects \`Implicit\`` REMOVED — non-binding - aspiration. (Plan named the site as "a later iter (deferred)"; - actual text was "a future iteration (deferred)" — variance - flagged by implementer, edit applied correctly.) -- `Concrete design deferred until the need materialises` REMOVED - — the binding content (no tracing GC backstop, ownership/linear - extension) precedes; the "deferred" sentence was the unneeded - punchline. -- `A future milestone may add a Prelude when concrete LLM-author - code surfaces a case…` REMOVED first sentence (aspiration), KEPT - second sentence (current-state description of how primitive - output works today via `io/print_int` / `io/print_bool` / - `io/print_str`). - -Acceptance: - -- composite grep - `grep -nE 'REVERTED|preserved for the audit trail|Migration plan|[Oo]riginally framed|original rationale|empirically-grounded version|A future (iter|iteration|milestone|Prelude) may|A future iter that|a (later|future) iter(ation)? \(deferred\)|Concrete design deferred' docs/DESIGN.md` - returns empty. -- Sweep-1 invariant grep stays empty (no regression). -- `Term::If` present in Term schema (line 1808 — `{ "t": "if", "cond": Term, … }`). -- `cargo test --workspace` 0 FAILED across all test binaries. -- `bench/check.py` 0 regressed (63 metrics; 4 improved beyond - tolerance, 59 stable); `bench/compile_check.py` 0 regressed (24 - metrics, 24 stable). -- DESIGN.md size: 2253 → 2155 lines (−98 lines this sweep, −107 - cumulatively from 2262 at milestone start). - -Tasks (commit subjects): - -- design-md-consolidation 2.1: delete Decision 7 (Term::If REVERTED block) -- design-md-consolidation 2.1 fixup: replace orphaned Decision 7 cross-reference at DESIGN.md:1141-1144 -- design-md-consolidation 2.2: condense Decision 9 narrative-of-changes opener + drop pre-Decision-9 history paragraphs -- design-md-consolidation 2.3: delete Decision 10 §Migration plan (7-point completed-iter list) -- design-md-consolidation 2.4: condense Decision 11 mono-vs-vdisp correction history to state-only rationale -- design-md-consolidation 2.5: future-iter speculations — remove aspirations, tighten binding prohibitions - -Carried into sweep 3: schema SoT inversion (DESIGN.md §Data-model -becomes the canonical schema; ast.rs gains a doc-comment pointing -to it; new drift test `design_schema_drift.rs` enforces structural -agreement between schema variants and ast.rs enums). Carried into -sweep 4: workflow / cross-reference cleanup ("Project ecosystem" -`agents/` path correction; "Verification and correctness" workflow -detail; "What is not (yet) supported" §"Recently lifted gates" -removal; cross-reference audit). - -Process note: Sweep 2's plan-time composite grep matched the -spec's full intent on the first attempt — only one fixup commit -(2.1, for the orphan Decision 7 cross-reference, which was a -content-not-grep issue caught by quality review). The Sweep 1 -lesson held. One plan-vs-actual phrasing variance ("a later iter" -vs "a future iteration") was caught by the implementer at edit -time and applied correctly without a fixup. - -## 2026-05-10 — Iteration design-md-consolidation 3: schema SoT inversion + data-model hardening - -Third iteration of the milestone defined in -`docs/specs/2026-05-10-design-md-consolidation.md`. Sweep 3 -inverts the schema source-of-truth between `docs/DESIGN.md` -§"Data model" and `crates/ailang-core/src/ast.rs`: DESIGN.md is -canonical, `ast.rs` is the projection, and a new drift test -catches divergence. - -Three substantive changes plus one new test: - -- **Two Rust code blocks removed from Decision 10.** The - `Type::Fn` + `ParamMode` block (around line 1027 at iter - start) and the `Suppress` struct block (around line 1132) are - replaced by prose pointers to §"Data model". Decision 10's - prose argument (per-position metadata vs `Type::Borrow` - variant) reads cleanly without the inline Rust. -- **§"Data model" SoT inversion.** The opener now reads "**This - section is the canonical schema.**" The Rust types in `ast.rs` - are framed as the in-memory projection, not the source. The - drift test is named as the enforcement mechanism in the - opener. -- **`ast.rs` module doc-comment.** The file-level `//!` block - bold-emphasises that DESIGN.md §"Data model" is canonical; - names the drift test as the enforcement; preserves the - serde-attribute description and the entry-type pointer. -- **`crates/ailang-core/tests/design_schema_drift.rs`** is the - new drift test (369 lines). Pattern follows the existing - `spec_drift.rs`: exhaustive `match` per enum (`Term`, - `Pattern`, `Type`, `Def`, `Literal`, `ParamMode`) ensures - adding a variant without a DESIGN.md anchor fails compilation; - the test asserts each anchor literally appears in DESIGN.md. - 7 tests total. All GREEN on first run after the schema-tag - alignment described below. - -**Real schema-vs-doc drift surfaced and closed inside the iter.** -The drift test exposed a pre-existing bug: `ast.rs`'s `Def` enum -uses `#[serde(tag = "kind", rename_all = "lowercase")]`, so -shipped `.ail.json` examples emit `"kind": "class"` / -`"kind": "instance"`. But Decision 11 §"Form-A schema" in -DESIGN.md said `"kind": "ClassDef"` / `"kind": "InstanceDef"`. -Confirmed by grepping `examples/test_22b1_*.ail.json` — every -shipped example uses lowercase tags, matching the serde output. -Per Sweep 3's commitment ("DESIGN.md is the canonical schema"), -DESIGN.md was wrong; closed with fixup `934a6e1`: Decision 11 -JSON code blocks now use lowercase tags, drift test anchors -updated to match. Prose references to the Rust type names -`ClassDef` / `InstanceDef` (e.g. line 1382's -"`**ClassDef**` — top-level definition kind, declares a class:") -are fine — those reference the Rust struct names, not the JSON -tag values. - -Quality-review nit closed inline: the test's file-level -doc-comment originally opened with `Sweep 3 / Task 4:` (a -task-reference prefix per CLAUDE.md comment policy is noise -once the iteration is closed). Doc-comment trimmed; def-kind -description list updated from PascalCase -(`ClassDef`/`InstanceDef`) to lowercase (`class`/`instance`) -to match the post-fixup anchors. - -Acceptance: - -- `grep -nE '^\s*(struct |enum |pub (struct|enum|fn))' docs/DESIGN.md` → empty. -- `grep -n 'whenever the two disagree\|ast.rs is the source of truth' docs/DESIGN.md` → empty. -- `grep -n 'This section is the canonical schema' docs/DESIGN.md` → 1 line (1708). -- `grep -n 'design_schema_drift' docs/DESIGN.md crates/ailang-core/src/ast.rs` → 1 match in each file. -- `cargo test -p ailang-core --test design_schema_drift` → 7 tests pass. -- `cargo test --workspace` → 0 FAILED. -- `bench/check.py` 0 regressed (63 metrics; 3 improved beyond - tolerance, 60 stable). `bench/compile_check.py` 0 regressed - (24 metrics, 24 stable). -- Sweep-1 + Sweep-2 invariants stay empty (no regression). -- DESIGN.md size: 2155 → 2139 lines (−16 this sweep, −123 - cumulatively from 2262 at milestone start). - -Tasks (commit subjects): - -- design-md-consolidation 3.1: remove 2 Rust code blocks from Decision 10 -- design-md-consolidation 3.2: invert §Data-model SoT — DESIGN.md canonical, ast.rs projection, drift test enforces -- design-md-consolidation 3.3: ast.rs doc-comment names DESIGN.md §Data-model as canonical schema, drift test as enforcement -- design-md-consolidation 3.4: add design_schema_drift.rs — exhaustive-match drift test for ast.rs vs DESIGN.md §Data-model -- design-md-consolidation 3.4 fixup: align DESIGN.md ClassDef/InstanceDef JSON tags with ast.rs lowercase serde rename -- design-md-consolidation 3.4 nit: drop task-ref doc-comment prefix + align def-kind list with lowercase tags - -Carried into sweep 4: workflow / cross-reference cleanup. -"Project ecosystem" `agents/` path correction; "Verification and -correctness" workflow detail; "What is not (yet) supported" -§"Recently lifted gates" removal; cross-reference audit. - -Process notes: -- The plan predicted "GREEN on first run — DESIGN.md already - aligned" for the drift test. Reality matched in the sense - that the test's grep-presence checks all passed on first run - — but the schema-vs-emit drift between DESIGN.md and ast.rs - was a different bug class that the grep-presence test wasn't - designed to catch. Surfacing happened anyway because the - implementer flagged it as known debt; the orchestrator - promoted it to in-scope and closed it. -- Open question for follow-up sweeps: the current drift test - guards DESIGN.md text presence, not serde-roundtrip fidelity. - A future hardening would round-trip a constructed `Def` value - through `serde_json::to_value` and assert the emitted JSON - matches one of DESIGN.md's anchors literally — that catches - the ClassDef/class class of bug structurally rather than - through grep-presence + spot inspection. Out of scope for - iter 3; queued. - -## 2026-05-10 — Iteration design-md-consolidation 4: workflow / cross-reference cleanup - -Fourth and final iteration of the milestone defined in -`docs/specs/2026-05-10-design-md-consolidation.md`. Sweep 4 -closes the "workflow / cross-reference cleanup" item: the stale -`agents/` ecosystem entry is replaced with a `Skills` entry -pointing to `skills//agents/`; the workflow-detail -attribution to `ailang-docwriter` in §"Verification and -correctness" is dropped (the rule survives, the agent assignment -lives in skills SoT); the "Recently lifted gates" history -paragraph is removed from §"What is not (yet) supported"; the -stale `see the JOURNAL queue` Pipeline pointer is updated to -`docs/roadmap.md` (per the user's 2026-05-10 split that moved the -forward queue out of JOURNAL); two additional residues caught by -the cross-reference audit are closed in a fixup ("Sharpened later -the same day" temporal anchor in Decision 10's opener — a -Sweep-2 missed pattern; supplementary "see closure conversion in -JOURNAL" pointer in the Term::Lam description — not load-bearing). - -Edits: - -- §"Project ecosystem" `Agents` bullet → `Skills` bullet: names - the six skills (`brainstorm`, `plan`, `implement`, `audit`, - `debug`, `fieldtest`) and points to `skills//agents/` as - the agent location; references `skills/README.md` instead of - the (now non-existent) `agents/README.md`. -- §"Project ecosystem" `Docs` bullet expanded: adds `roadmap.md` - (forward queue), `specs/` (per-milestone design specs), - `plans/` (per-iteration implementation plans), and tags - DESIGN.md as "canonical state" + JOURNAL.md as "chronological - decisions log". -- §"Verification and correctness" item 6: `ailang-docwriter` - agent attribution stripped; the "rustdoc warnings are fixed in - the iteration that introduced them, not as follow-up" rule - survives. -- §"What is not (yet) supported" opener: the "Recently lifted - gates that used to live here: cross-module ADTs … `==` - polymorphism" enumeration removed. The first sentence - ("Snapshot of the current boundary. Items move out of this list - as iterations land; the JOURNAL records when.") survives as a - state-only lead-in. -- Pipeline section: `retirement; see the JOURNAL queue.` → - `retirement; see \`docs/roadmap.md\` for the active queue.` -- Decision 10 opener (fixup): `Sharpened later the same day:` → - removed; "was extended" → "is extended" (present-tense state). -- Term::Lam description (fixup): `(see closure conversion in - JOURNAL)` parenthetical removed — the binding claim ("free - variables of its body are captured from the enclosing scope") - is self-sufficient. - -Cross-reference audit final inventory (post-fixup): - -- 3 JOURNAL bench-notes pointers retained as binding-evidence - anchors: - - line ~786 (Decision 10 opener: "bench notes in JOURNAL") - - line ~817 (closure-chain ratio: "current ratio recorded in - JOURNAL bench entries") - - line ~848 (tracing-GC argument: "JOURNAL bench notes") - - line ~1511 (mono-vs-vdisp ratio: "JOURNAL bench-notes entry - record the measured ratios") -- 1 JOURNAL rationale pointer retained (line ~1611, - higher-rank-polymorphism prohibition: "rationale recorded in - JOURNAL"). -- 1 `docs/roadmap.md` pointer added (Pipeline section). -- 1 `skills/README.md` pointer added (Project-ecosystem Skills - bullet). -- 1 `skills//agents/` pointer added (Project-ecosystem - Skills bullet). -- §"Project ecosystem" Docs bullet names DESIGN.md / JOURNAL.md / - roadmap.md / specs/ / plans/ as the doc family. -- §"What is not (yet) supported" lead-in retains - "the JOURNAL records when" as a binding pointer to where each - feature's ship-date lives. - -Acceptance: - -- composite Sweep-4 grep - `grep -nE '\(\`agents/\`\)|agents/README\.md|ailang-docwriter|Recently lifted|see the JOURNAL queue|later the same day|Sharpened later|closure conversion in JOURNAL' docs/DESIGN.md` - empty. -- Sweep-1 + Sweep-2 + Sweep-3 invariants stay empty (no - regression). -- `cargo test -p ailang-core --test design_schema_drift` 7 tests - pass. -- `cargo test --workspace` 0 FAILED. -- `bench/check.py` 0 regressed (63 metrics; 4 improved beyond - tolerance, 59 stable). `bench/compile_check.py` 0 regressed (24 - metrics, 24 stable). -- DESIGN.md size: 2139 → 2132 lines (−7 this sweep, −130 - cumulatively from 2262 at milestone start). - -Tasks (commit subjects): - -- design-md-consolidation 4.1: fix Project-ecosystem stale agents/ path; expand Docs bullet to name roadmap.md + specs/ + plans/ -- design-md-consolidation 4.2: strip ailang-docwriter agent workflow detail from Verification section; rule survives -- design-md-consolidation 4.3: drop 'Recently lifted gates' history paragraph from §What-is-not-yet-supported -- design-md-consolidation 4.4: cross-ref audit — JOURNAL queue → roadmap.md; bench pointers retained as binding-evidence anchors -- design-md-consolidation 4.4 fixup: drop 'Sharpened later the same day' temporal anchor + supplementary 'see closure conversion in JOURNAL' pointer - -Process note: the cross-reference audit (Task 4) surfaced two -additional residues — one Sweep-2 missed pattern ("Sharpened -later the same day") and one not-load-bearing supplementary -pointer ("see closure conversion in JOURNAL"). Both closed inside -Sweep 4 via fixup. Audit-as-discovery worked as intended for -Sweep 4 in the same way it worked for Sweep 3 (the schema-tag -drift caught by `design_schema_drift.rs`): the closing iter of a -milestone surfaces residues the earlier iters missed because -their grep coverage was scoped to specific patterns. - -## 2026-05-10 — Milestone close: design-md-consolidation - -Four sweeps shipped over a single working day. DESIGN.md -transitioned from a mixed state/history/workflow document -(2262 lines) to a state-only specification (2132 lines, −5.7%) -with three orthogonal SoTs cleanly separated: - -- `docs/DESIGN.md` — canonical state + timeless rationale. -- `docs/JOURNAL.md` — chronological decisions log. -- `docs/roadmap.md` — forward queue (introduced by user during - iter 1; integrated into the role table in iter 4). -- `skills//SKILL.md` + `skills//agents/` — workflow - disciplines and agent rosters. -- `crates/ailang-core/src/ast.rs` — Rust-side projection of - DESIGN.md §"Data model" (drift-tested). - -What changed across sweeps: - -- **Sweep 1 (history anchors):** 11 commits. Removed 93 iter - tags + ~16 bare iter-id residues + 4 family tags + 13 date - anchors + 4 status markers; condensed 7 of 12 historical bench - data-points (5 KEEPs grounded the 1.3× retirement target / - ±15% closure-band tolerance policy contracts). -- **Sweep 2 (REVERTED + migration):** 7 commits. Deleted - Decision 7 (Term::If REVERTED block, 32 lines), Decision 10 - Migration plan (32 lines), Decision 9 narrative-of-changes - opener + pre-Decision-9 history paragraphs, Decision 11 - mono-vs-vdisp correction history; condensed 5 future-iter - speculations (1 KEPT as a binding prohibition restructured to - assertive form; 4 REMOVED as non-binding aspirations). -- **Sweep 3 (schema SoT inversion):** 6 commits. Removed 2 Rust - code blocks from Decision 10; inverted §"Data model" SoT - relationship (DESIGN.md canonical, ast.rs projection, drift - test enforces); updated `ast.rs` module doc-comment to - bold-emphasise the inversion; created - `crates/ailang-core/tests/design_schema_drift.rs` (7 tests, - exhaustive match per enum); fixed real schema-vs-doc drift in - Decision 11 §"Form-A schema" (`"kind": "ClassDef"` / - `"InstanceDef"` → `"kind": "class"` / `"instance"` to match - the actual `rename_all = "lowercase"` serde output). -- **Sweep 4 (workflow / cross-reference cleanup):** 5 commits + - 1 close. Project-ecosystem Skills/Docs bullets modernised; - Verification workflow detail stripped; "Recently lifted gates" - history paragraph removed; stale JOURNAL queue pointer - updated to `docs/roadmap.md`; two cross-reference-audit - residues closed inside the iter via fixup. - -Cumulative commit count: 29 commits (4 spec/plan + 25 task / -fixup / nit / close). - -Open questions queued for follow-up: - -- **Drift test fidelity widening.** The current - `design_schema_drift.rs` guards DESIGN.md text presence, not - serde-roundtrip fidelity. A future hardening would round-trip - a constructed `Def` value through `serde_json::to_value` and - assert the emitted JSON matches one of DESIGN.md's anchors - literally — that catches the ClassDef/class class of bug - structurally rather than through grep-presence + spot - inspection. Out of scope for this milestone; queued. -- **Architect agent iron-law extension.** The architect agent's - drift-review iron law could be extended with the standing - greps from Sweeps 1, 2, and 4 so that future commits to - DESIGN.md cannot reintroduce history anchors / REVERTED - narratives / workflow detail without the architect flagging - it. Out of scope for this milestone; queued. - -Process retrospective: - -- The Sweep-1 lesson ("plan-time grep must match spec intent") - held across Sweeps 2-4. No fixup commits were required for - *missed* anchor variants in Sweeps 2, 3, or 4 — the variants - the spec named were all caught by their respective composite - greps on the first attempt. -- Three real bugs / residues were surfaced and closed inside the - milestone, each by a different mechanism: - - Sweep 1 closing: bare iter-id forms ("pre-19b", "14d", "22a" - etc.) — caught by the implementer's "known debt" note + - orchestrator's widened grep. - - Sweep 3 closing: schema-tag drift `ClassDef`/`InstanceDef` - vs serde-emitted `class`/`instance` — caught by the new - drift test created in the same iter. - - Sweep 4 closing: "Sharpened later the same day" + - "(see closure conversion in JOURNAL)" — caught by the - cross-reference audit walk-through in Task 4. - Each closure was an in-scope expansion of the discovering - iter, not deferred debt. -- Quality reviewer's nit-level findings (trailing whitespace, - dangling cross-references, doc-comment task-tag prefixes) - were closed inline as orchestrator edits where mechanical, or - dispatched as fixup commits where judgment was needed. The - two-stage review pattern (spec compliance → code quality) - caught residues that a single-stage review would have missed. -- DESIGN.md size reduction (130 lines, 5.7%) is observation, not - target. The substantive change is the *structure* — three - axes cleanly separated, no mixed framing, drift-test enforcing - the schema axis. A reader can now trust DESIGN.md as - state-only without filtering history-vs-state-vs-workflow on - every paragraph. - -## 2026-05-10 — Audit close: design-md-consolidation milestone - -Audit run on milestone-close commit `c6e4333`. Three substantive -architect findings closed inline as audit-tidy `63df0c0`; one low -flagged and acknowledged. - -### Architect drift review - -`[high]` `docs/DESIGN.md` §Data-model `### Def` enumeration was -incomplete: `kind ∈ { "fn", "const", "type" }` omitted the `class` -and `instance` kinds, which are real shipped schema forms (per -ast.rs `Def::Class` / `Def::Instance` with `rename_all = -"lowercase"`). The class/instance JSON shapes were documented in -Decision 11 §Form-A schema but absent from the §Data-model -canonical summary — a split-section gap. Sweep 3's drift test was -GREEN because anchor strings (`"kind": "class"`, -`"kind": "instance"`) appeared *somewhere* in DESIGN.md, but the -section that claims to be canonical (§Data-model, post-Sweep-3 -SoT inversion) was incomplete. Closed: §Data-model `### Def` -extended with `class` and `instance` JSON blocks (full method -table for class, instance method table, optional doc, superclass -and default fields). Drift test stays GREEN. - -`[high]` `docs/DESIGN.md` §Data-model `Type` block: the -`{ "k": "forall", "vars": [...], "body": Type }` JSON example -omitted the `constraints` field. ast.rs `Type::Forall` carries -`constraints: Vec` (serde -`skip_serializing_if = "Vec::is_empty"`) per Decision 11; the -field is exercised by the typeclass machinery. Same split-section -problem as the Def gap. Closed: the Forall block now lists the -`constraints` field as an optional element, with the -`omitted-when-empty / hash-stable` comment. - -`[medium]` `docs/DESIGN.md` §"What is not (yet) supported" said -"No local recursive `let`. `let f = ... in ...` … recursion needs -a top-level def." Factually wrong: `Term::LetRec` -(`{ "t": "letrec", ... }`) enables local recursive *fn* bindings -and is fully documented in §Data-model and the pipeline. An LLM -reading this item would believe recursion requires a top-level -def and would not reach for `letrec`. Closed: the entry is -rewritten to "No recursive `let` for non-fn values. … Recursive -*fn* bindings are supported via `Term::LetRec`; the desugar pass -lifts most occurrences to a synthetic top-level fn, with -`lift_letrecs` finishing the residue after typecheck." This -distinguishes the two cases correctly (no value-cycle recursion -to preserve Decision 10's acyclicity invariant; fn-recursion is -fine via letrec). - -`[low]` `docs/DESIGN.md` line 1641 contains the path -`docs/specs/2026-05-09-22-typeclasses.md` — a deliberate exception -to the Sweep-1 grep `2026-[0-9]{2}-[0-9]{2}` because it's a real -filename, not a date anchor in prose. Sweep 1 noted this exception -in its own JOURNAL entry; the architect's iron-law standing-grep -list (queued as a follow-up from milestone close) needs to allow -paths beginning with `docs/specs/` to match without firing. Not -fixed in this audit (the architect-iron-law extension is a -separate follow-up); flagged here for the implementer who picks -it up. - -### Bench regression - -`bench/check.py` exit 0 — 63 metrics, 0 regressed, 4 improved -beyond tolerance, 59 stable. -`bench/compile_check.py` exit 0 — 24 metrics, 0 regressed, 0 -beyond tolerance, 24 stable. -`bench/cross_lang.py` exit 0 — 25 metrics, 0 regressed, 0 beyond -tolerance, 25 stable. - -The 4 latency improvements beyond tolerance in `bench/check.py` -are noise within the noise floor (this milestone touched no -code paths that affect runtime); ratification not required. - -### Rustdoc audit - -`cargo doc --no-deps` reports 16 warnings (15 in `ailang-check`, -1 in `ailang-core`). Spot-checked: every warning predates this -milestone (private-item links from public doc, unresolved -intra-crate links). The milestone didn't introduce them; closing -them is out-of-scope. Queued in `docs/roadmap.md` as a follow-up -todo: "rustdoc warning sweep (15+1 pre-existing private/unresolved -links)". The rule the milestone preserved at §Verification item 6 -("rustdoc warnings are fixed in the iteration that introduced -them, not as follow-up") implies the queue item is for the -iteration that *introduced* each warning — but since they predate -the milestone, treating them as a sweep is the only feasible -catch-up. - -### Audit close - -- Architect: 3 high/medium drift items closed inline - (`63df0c0`); 1 low flagged for the architect-iron-law follow-up. -- Bench: green across all three scripts. -- Rustdoc: 16 pre-existing warnings queued in roadmap. - -Tasks (commit subjects): - -- design-md-consolidation audit-tidy: close 3 architect drift items (Def kinds class/instance + Type::Forall constraints + letrec correction) - -The audit-tidy commit's diff: 41 insertions, 6 deletions in -`docs/DESIGN.md`. Drift test stays GREEN; workspace tests stay -GREEN. - -Process note: the architect's findings confirm the queued -"drift-test fidelity widening" follow-up from the milestone-close -summary. The current `design_schema_drift.rs` checks anchor -*presence* in DESIGN.md as a whole, not anchor *placement* in the -section that claims to be canonical (§Data-model). The two `[high]` -findings (missing `class`/`instance` Def kinds; missing -`constraints` field in Type::Forall) were both invisible to the -drift test for exactly this reason — the anchors existed, just in -the wrong section. A future widening would either (a) constrain -the test to scan only §Data-model + ParamMode block, or (b) -extract the JSON-schema blocks from §Data-model into a -machine-readable file the test consumes. Queued. - -## 2026-05-10 — Iteration Floats.1: schema layer - -Added `Literal::Float { bits: u64 }` as the fifth `Literal` AST -variant. Bits are an IEEE-754 binary64 bit pattern, serialised as -a 16-character lowercase hex *string* in canonical JSON -(`{"bits":"","kind":"float"}`) via a private `hex_u64` serde -helper module on `ast.rs`. Routing the field through the JSON -string path bypasses `serde_json::Number::to_string` (not -bit-stable across `serde_json` versions for floats) and lets NaN / -±Inf survive canonicalisation at all — they collapse to `null` as -JSON numbers and are silently lost. - -`Float` is now registered as a primitive type name in -`primitives.rs`; the lockstep test was extended with explicit -`assert!` lines after the loop because the loop's `assert_eq!` -passes vacuously when both functions return `false` / `None` for -a missing primitive. - -Adding the variant broke Rust's enum exhaustiveness at eight -downstream `match Literal { … }` source sites and at two -drift-test exhaustive matches. Each got either the permanent -semantic arm (`ailang-check` typecheck: -`Literal::Float { .. } => Type::float()`) or a named-iteration -`unimplemented!("Floats milestone iter N: ")` arm -(`ailang-codegen` → iter 4, `ailang-surface` print → iter 2, -`ailang-prose` → iter 5). The arms are honest about which -iteration owns the semantics, so future debugging starts with the -right pointer rather than a generic `unreachable!`. - -The two drift-test exhaustive matches -(`spec_mentions_every_literal_variant`, -`design_md_anchors_every_literal_variant`) were extended along -with their exemplars and the corresponding spec anchors: -`crates/ailang-core/specs/form_a.md` gained a `` `FLOAT` `` atom -form line; `docs/DESIGN.md` gained a `{ "kind": "float", "bits": -"<16-lowercase-hex>" }` line in the Literal JSON-schema block at -line 1866. The drift tests are *the enforcement mechanism* of the -schema invariant — the original 5-iter plan put DESIGN.md changes -in iter 5, but the drift-test break revealed that schema-layer -DESIGN.md anchors belong in iter 1 (where they get added together -with the AST variant they document). Iter 5 retains the §"Float -semantics" subsection and the line-2033 "supported primitive -types" list update — those depend on later iterations being -shipped. - -Bit-stability tests pin the A1 / A5 spec guarantees: `-0` ≠ `+0` -at the canonical-bytes level (distinct hex strings); NaN bits -preserved; ±Inf bits preserved; serde round-trip is bit-exact for -the saturating boundary values (zero, sign-bit-only, 1.5, qNaN, -±Inf, all-ones). - -Pre-existing `def_hash` regression hashes (`db33f57cb329935e` for -`sum.sum`, `b082192bd0c99202` for `IntList`) stayed GREEN — -adding a `Literal` variant does not perturb any pre-existing -canonical bytes, because `serde(tag = "kind")` keeps the -discriminator-only-on-construct path. - -Rustdoc warning count stayed at the milestone-open baseline (1 -pre-existing warning on `desugar`). Two new private-intra-doc-link -warnings introduced by the initial `Literal::Float` doc-comment -(`[`hex_u64`]` linking to a private mod) were caught by the -acceptance gate and fixed inline by dropping the link form in -favour of a plain-text `hex_u64` reference. - -Per-task commits: - -- `ec28111` floats iter 1.1: Literal::Float variant + canonical hex serde + drift-test anchors -- `93fe2da` floats iter 1.1 fixup: trim form_a.md FLOAT bullet to match neighbour style -- `aa5b88e` floats iter 1.2: register Float as a primitive type name -- `93bae2d` floats iter 1.2 fixup: replace iter-N comment with durable rationale for explicit assertions -- `7c95a69` floats iter 1.3: bit-stability tests for Literal::Float -- `1a4e2f0` floats iter 1.4: refresh canonical.rs 'no floats' doc comment -- `b2d3182` floats iter 1.4 fixup: drop intra-doc-links to private hex_u64 (rustdoc baseline preservation) - -Known debt deliberately deferred to later iterations of this -milestone: - -- Surface lex / parse / print round-trip for `1.5`, `1.5e3`, - `1e10` — iter 2. -- Typecheck widening of `+`/`-`/`*`/`/`/`<`/`<=`/`>`/`>=` from - monomorphic Int to polymorphic-`{Int, Float}` — iter 3. -- Codegen Float lowering paths (`fadd`/`fsub`/`fmul`/`fdiv`, - `fcmp o*`/`une`, `sitofp`, `@llvm.fptosi.sat.i64.f64`, - `fcmp uno` for `is_nan`, hex-float literal constants for - `nan`/`inf`/`neg_inf`) — iter 4. -- Prose round-trip for Float literals — iter 5. -- DESIGN.md §"Float semantics" subsection (A5 determinism - contract) and line-2033 "supported primitive types" list - update — iter 5. - -## 2026-05-10 — Iteration Floats.2: surface layer - -Lexer recognises Float literals in form -`.(e[+-]?)?` and `e[+-]?` -per spec A2; bare leading/trailing dots, missing fraction-after- -dot, and missing/sign-only exponents reject with -`LexError::InvalidFloat { literal, start }`. Hex floats are -naturally rejected by `f64::from_str` and by the digit-lead caller -arm. The new private `looks_like_float` helper enforces the spec -grammar before delegating the bit-pattern conversion to -`f64::from_str` + `f64::to_bits`. Leading-dot tokens like `.5` -fall through the digit-lead rule (first byte not a digit) and -classify as `Tok::Ident(".5")` — the lexer does not reject them; -downstream stages produce the diagnostic. The 11 new lex tests -pin both the positive grammar (`1.5`, `1.5e3`, `1e10`, `-1.5`, -signed zero, uppercase `E`) and the rejection set (trailing dot, -missing fraction-before-exp, empty exp, sign-only exp, double -dot, leading dot routes to ident). - -The new `Tok::Float(u64)` is wired through the parser at two -sites — `parse_term` (atom literal position) and `parse_pat_lit` -(inside `(pat-lit ...)`). The parser ACCEPTS Float in pattern -position even though pattern-matching on Float is semantically -dubious (per spec line 723-735). Typecheck-level rejection is -iter 3's job; the parser-side diagnostic stays a generic "literal -form" error widened to mention float. `tok_label` exhaustiveness -forced the wiring on the same iteration the variant landed — -that's the same enforcement-via-exhaustive-match pattern iter 1 -exercised on the AST `Literal` enum. - -The two `unimplemented!("Floats milestone iter 2: surface print")` -arms left by iter 1 in `print.rs` are gone. The new -`write_float_lit` helper formats via `f64::to_string` (Grisu3 -shortest round-trippable) and appends `.0` if the rendered form -contains neither `.` nor `e`/`E` — preserving the -`lex(print(L)) == L` round-trip property pinned by a new -regression test over six representative bit patterns (`1.5`, -`0.0`, `-0.0`, `10.0`, `-0.375`, `1e10`). Non-finite bits panic; -surface lex cannot produce them, so the only path that would feed -NaN/Inf to the printer is a Form-A direct construction — and the -printer is not a Form-A escape hatch. - -Process note from this iteration: the original plan's Step 11 -caveat ("`cargo test -p ailang-surface --lib` bypasses parse.rs") -was technically incorrect — `--lib` does compile the whole -library. The implementer caught this and routed Task 1's GREEN -verification through Task 2 instead, where `tok_label` -exhaustiveness gets restored and the lex tests can finally run. -The intent (commit Task 1 with parse.rs broken at exactly one -site, fix in Task 2) was unambiguous and was followed; the -imprecise note in the plan template is queued as a one-line -correction the next plan should not repeat. - -Per-task commits: - -- `d0c9133` floats iter 2.1: Tok::Float + LexError::InvalidFloat + spec-A2 grammar validator -- `f960e39` floats iter 2.1 fixup: drop task-tags + refresh stale prose + uppercase-E + leading-dot tests -- `f62bff0` floats iter 2.2: parser accepts Tok::Float in term and pat-lit positions -- `c619697` floats iter 2.3: surface print emits shortest round-trippable decimal with .0 fallback - -Known debt deliberately deferred to later iterations of this -milestone: - -- Typecheck rejection of `Pattern::Lit { lit: Literal::Float - { .. } }` per spec line 723-735 recommendation (a). The parser - accepts Float patterns; iter 3 surfaces the type error. -- Typecheck widening of `+`/`-`/`*`/`/`/`<`/`<=`/`>`/`>=` from - monomorphic Int to polymorphic-`{Int, Float}` — iter 3. -- Codegen Float lowering paths - (`fadd`/`fsub`/`fmul`/`fdiv`, `fcmp o*`/`une`, `sitofp`, - `@llvm.fptosi.sat.i64.f64`, `fcmp uno`, hex-float constants) — - iter 4. -- Prose round-trip for Float literals — iter 5. -- DESIGN.md §"Float semantics" subsection (A5 determinism - contract) and line-2033 "supported primitive types" list - update — iter 5. - -## 2026-05-10 — Iteration Floats.3: typecheck + builtins - -Widened the existing arithmetic and comparison builtins (`+`, `-`, -`*`, `/`, `!=`, `<`, `<=`, `>`, `>=`) from monomorphic -`(Int, Int) -> {Int,Bool}` to polymorphic `forall a. (a, a) -> a` -(resp. `... -> Bool`). Same shape `==` already had since iter 16e. -The `{Int, Float}` filter is enforced at codegen (iter 4); typecheck -accepts any matching pair, mirroring how `==` works today. `%` stays -monomorphic-Int (no fmod yet — `%` semantics for Float require an -explicit decision on sign-of-result and ±0/±Inf edge cases that has -not been made). The widening is type-side only; codegen lowering for -`+` etc. continues to use the monomorphic-Int `synth::builtin_binop` -table (iter 4 converts that to type-dispatched). The lockstep partner -table at `crates/ailang-codegen/src/synth.rs::builtin_ail_type` was -widened in the same commit so no codegen call site sees an -out-of-date polymorphic shape. - -Stale comment caught and corrected during the iter-3.1 quality -review: the `==` install block carried "the other comparison ops -(`<`, `<=`, `>`, `>=`, `!=`) stay Int-only" since iter 16e. -Replaced with the actual current shape — and the same comment now -documents the codegen-dispatch table's Float arm that lands in -iter 4 (`Float → fcmp oeq double`), so the future-state pointer -sits next to the typecheck-side install rather than only in the -spec. - -Five new Float builtins installed: `neg : forall a. (a) -> a` -(polymorphic; parallel to widened `+`), `int_to_float : (Int) -> -Float`, `float_to_int_truncate : (Float) -> Int`, `float_to_str : -(Float) -> Str`, `is_nan : (Float) -> Bool`. The polymorphic `neg` -rationale: spec section A3 calls out that the `(- 0.0 x)` desugar -is wrong for `-0.0` (returns `+0.0` per IEEE rounding), so Float -negation needs its own builtin name. Reusing the `forall a. (a) -> a` -shape lets Int negation share the symbol. - -Three Float bit-pattern constants installed as bare values -(parallel to `__unreachable__` but with concrete `Type::float()` -rather than `forall a. a`): `nan`, `inf`, `neg_inf`. Reference site -is `(var nan)`, not `(app nan)`. Codegen emits `double 0x7FF8...` / -`double 0x7FF0...` / `double 0xFFF0...` at the use site in iter 4. - -One new effect op installed: `io/print_float : (Float) -> Unit !IO`. -Mirrors `io/print_int|bool|str`. Codegen lowering through runtime C -glue `@ail_print_float` lands in iter 4. - -Pattern-matching on Float literals is now typecheck-rejected via the -new `CheckError::FloatPatternNotAllowed` variant per spec line -723-735 recommendation (a). The surface lex / parser still ACCEPT -Float patterns (iter 2 left this open); the typecheck diagnostic -fires at the `Pattern::Lit { lit }` arm in -`crates/ailang-check/src/lib.rs`. The error message points the -LLM-author to the documented alternative — ordering operators and -`is_nan` for Float discrimination. The variant gets a parallel arm -in `CheckError::code()` returning `"float-pattern-not-allowed"` — -mechanical compile-completeness driven by the variant addition. - -`ail builtins` reflects all twelve new entries (4 widened arithmetic -display strings + 5 Float fn builtins + 3 Float constants + 1 -io/print_float effect op) via the refreshed `list()` table. - -Per-task commits: - -- `0981804` floats iter 3.1: widen +/-/*/// and !=//>= to polymorphic forall a -- `6890aa2` floats iter 3.1 fixup: correct stale ==-comparison comment + asymmetric Float test args + drop spec-section reference -- `60a4c68` floats iter 3.2: install neg/int_to_float/float_to_int_truncate/float_to_str/is_nan -- `fd3f74c` floats iter 3.3: install Float constants nan/inf/neg_inf + io/print_float effect op -- `d6da5c2` floats iter 3.4: typecheck rejects Pattern::Lit Float with FloatPatternNotAllowed - -Process note: the Floats milestone keeps centralising the lockstep -between `crates/ailang-check/src/builtins.rs::install` and -`crates/ailang-codegen/src/synth.rs::{builtin_ail_type, -builtin_effect_op_ret}`. After iter 3 these two tables carry four -near-identical `Forall { vars: ["a"], …, body: Fn { params, ret, … } }` -constructions per crate. The two-table convention is established and -deliberate (different crates, no `ailang-core` dependency in `synth.rs` -on `ailang-check`-style helpers), but if the duplication crosses a -fifth pair of entries in iter 4 a small `ailang-core::ast::poly_a_a_to_a` -constructor starts to earn its keep. Queued as a roadmap item, not -fix-now. - -Known debt deliberately deferred to later iterations of this -milestone: - -- Codegen LOWERING for the widened `+`/`-`/`*`/`/` etc. — iter 4 - converts the monomorphic-Int `synth::builtin_binop` table to - type-dispatched; emits `fadd double` / `fcmp olt double` / `fcmp - une double` (for `!=`!) / etc. for the Float arm. -- Codegen LOWERING for `neg` (Int → `sub i64 0, %x`, Float → - `fneg double %x`) — iter 4. The `fneg` instruction (LLVM 8+) is - needed for correct `-0.0` handling. -- Codegen LOWERING for `int_to_float` (`sitofp`), - `float_to_int_truncate` (`@llvm.fptosi.sat.i64.f64`), - `float_to_str` (runtime C glue), `is_nan` (`fcmp uno`) — iter 4. -- Codegen LOWERING for `nan`/`inf`/`neg_inf` constants (LLVM - hex-float literals at the use site) — iter 4. -- Codegen LOWERING for `io/print_float` (parallel to - `io/print_int`, runtime glue `@ail_print_float`) — iter 4. -- Prose round-trip for Float literals — iter 5. -- DESIGN.md §"Float semantics" subsection (A5 determinism contract) - and the line-2033 "supported primitive types" list update — iter 5. - -## 2026-05-10 — Iteration Floats.4: codegen + E2E - -Lowered every Float-related typecheck primitive that iter 3 installed -into LLVM IR. The end-to-end fixture `examples/floats.ail.json` builds -via `ail build`, runs, and produces the expected stdout -(`4\n42\n-1.5\n`) — this is the milestone-acceptance gate, and it -passed first try after the seven sub-tasks landed. - -`Float` is now a registered primitive in `synth.rs` (`llvm_type` → -`double`, `type_descriptor` → `Fl` to avoid the single-letter `F` ADT -prefix collision). Float literals lower as LLVM hex-float `double` -SSA constants (`format!("0x{:016X}", bits)` — uppercase per LLVM -convention). Both compile-completeness `unimplemented!("Floats -milestone iter 4: codegen")` arms left by iter 1 in `lib.rs:918` / -`:1215` are gone. - -Arithmetic and comparison ops dispatch on the resolved arg type. The -old monomorphic-Int `synth::builtin_binop` was replaced by a -3-tuple-returning `builtin_binop_typed(name, &Type) -> Option<(instr, -operand_ll_ty, result_ll_ty)>`. Returning the result type -explicitly eliminates the dual-meaning trap the original 2-tuple -shape carried (caller had to know whether arm was arithmetic -(operand-type == result-type) or comparison (result-type always -`i1`)). The `lower_app` dispatch site was simplified accordingly — -no more i1-result override at the caller. Float arithmetic emits -`fadd/fsub/fmul/fdiv double`; Float comparison emits `fcmp -olt/ole/ogt/oge/oeq double`. **`!=` Float emits `fcmp UNE double`, -NOT `fcmp one`** — `one` is "ordered and not equal" and would -return false for `nan != nan`, violating IEEE-`!=` and user-fixed -constraint #2. - -Five new fn-builtin lowering arms in `lower_app`: - -- `neg` polymorphic — Int arm `sub i64 0, %x`; Float arm `fneg - double %x` (LLVM 8+, correct for `-0.0`). The `fsub double 0.0, - %x` desugar would be wrong: IEEE `0.0 - 0.0 == +0.0`, so - `neg(+0.0)` would erroneously produce `+0.0` instead of `-0.0`. -- `int_to_float` → `sitofp i64 %x to double`. -- `float_to_int_truncate` → `call i64 @llvm.fptosi.sat.i64.f64(double - %x)` (LLVM 12+ saturating intrinsic; clang 22 always available). - NaN → 0, +Inf → i64::MAX, -Inf → i64::MIN, finite-out-of-range - saturates, finite-in-range truncates toward zero — exactly the - intrinsic's documented semantics, no wrapping needed. -- `is_nan` → `fcmp uno double %x, %x` (only NaN compares unordered - against itself). -- `float_to_str` → `Err(CodegenError::Internal("...not yet - implemented (requires dynamic Str allocation in the runtime)"))` - (deferred to iter 5+; the runtime's Str path currently uses only - static `@.str_*` globals — no malloc-backed dynamic-Str - infrastructure). The deferral surfaces as a structured codegen - error rather than a panic, mirroring the rest of the file's - error discipline. - -Float constants `nan`/`inf`/`neg_inf` resolve as bare `Term::Var` -references and lower to direct hex-float `double` SSA values at the -use site — `0x7FF8000000000000` (canonical qNaN), `0x7FF0000000000000` -(+Inf), `0xFFF0000000000000` (-Inf). Intercepted at the top of -`lower_term`'s `Term::Var` arm, before any other resolution. **NOT** -through the `__unreachable__` lowering pattern, which is a -terminator instruction; constants are SSA values. - -`io/print_float` lowers via inline `printf("%g\n", v)` parallel to -`io/print_int`. No new C runtime file needed — the `@printf` -declaration already exists. Format string interned as `fmt_float`. - -Two plan bugs caught and corrected during the iteration: - -1. **Iter-4.2 plan inconsistency** — Task 2 as literally written - would have stranded comparison ops (Step 4 said - `builtin_binop_typed` returns None for them, Step 5 narrowed - `lower_app` matches! to `+ - * / %` only — together that - collapses the no-regression invariant). The implementer caught - this and pulled forward Task 3 Steps 3-4 as a minimal repair. - The orchestrator endorsed; Task 3's residual scope was narrowed - to Float comparison arms + `lower_eq` Float arm. -2. **Iter-4.4 quality review** — `float_to_str` was specified as - `unimplemented!()` (panic), inconsistent with the rest of the - file's `CodegenError::Internal(...)` error discipline. A user - program calling the typecheck-installed `float_to_str` would - panic the compiler instead of producing a structured error. - Fixed in the 4.4 fixup commit. - -Two pull-forwards into iter-4.2 (`is_static_callee` extension for -`==`) and iter-4.4 (`is_static_callee` extension for the 5 new -fn-builtins) reflect the same pattern: any name that lowers via a -direct `lower_app` arm must also be recognised by `is_static_callee`, -or it falls through to the indirect-call path with `UnknownVar`. -Future iterations adding new builtin lowering arms must remember -this companion edit. - -The mono-pass and ADT slot-layout question from spec Components -(monomorphisation produces concrete `List_Fl` slots vs. polymorphic -erased slots needing bitcast) was NOT exercised by the E2E fixture -— `examples/floats.ail.json` does not use Float-typed containers. -This is not a regression: the descriptor `Fl` exists and the -mono pass would emit `List_Fl` correctly; the bitcast question -would only fire for an erased polymorphic container, which AILang -doesn't have today. Queued as a follow-up if a future fixture -exercises the path. - -Per-task commits: - -- `ac5e17e` floats iter 4.1: codegen primitive registration + Float literal hex-double lowering -- `8044a4d` floats iter 4.2: codegen arithmetic dispatch on arg type — fadd/fsub/fmul/fdiv double -- `2a29070` floats iter 4.2 fixup: 3-tuple return for builtin_binop_typed + classifier helper -- `3869641` floats iter 4.3: codegen Float comparison arms (fcmp olt/ole/ogt/oge/UNE) + lower_eq Float -- `581144a` floats iter 4.4: codegen neg/int_to_float/float_to_int_truncate/is_nan + float_to_str-deferred -- `613aa39` floats iter 4.4 fixup: float_to_str returns CodegenError::Internal instead of panic -- `bde5aaf` floats iter 4.5: codegen Float constants nan/inf/neg_inf as hex-double SSA values -- `9764b61` floats iter 4.6: codegen io/print_float + examples/floats.ail.json E2E fixture - -Workspace at iter-4 close: 402 tests passed, 0 failed (from 395 at -iter-3 close — added 6 codegen unit tests + 1 E2E test). - -Known debt deliberately deferred to iter 5: - -- Prose round-trip for Float literals. -- DESIGN.md §"Float semantics" subsection (A5 determinism contract). -- DESIGN.md line-2033 "supported primitive types" list update - (`Float` was not mentioned as a supported primitive there before; - iter 4's ship makes it true). -- `crates/ailang-prose/src/lib.rs::Literal::Float` arm replacement - (currently `unimplemented!("Floats milestone iter 5: prose")` - from iter 1). - -Known debt deliberately deferred BEYOND iter 5 (this milestone): - -- `float_to_str` codegen lowering (requires runtime-allocated Str; - AILang's Str path is currently static-only). Symbol is - type-installed and surface-callable, but lowering errors with a - structured `CodegenError::Internal`. Future milestone wires the - runtime Str-allocator path. -- Float-typed container slot layout (mono `List_Fl` etc.) — not - exercised by any current fixture; revisit if a future use case - surfaces. - -## 2026-05-10 — Iteration Floats.5: prose + DESIGN.md + milestone close - -Replaced the iter-1 prose `unimplemented!("Floats milestone iter 5: -prose")` arm with `write_float_lit` — finite values render as -shortest round-trippable decimal with `.0` suffix (parallel to -surface print); non-finite values render as `NaN` / `+Inf` / -`-Inf` (Mainstream-language spellings) because prose is one-way -render and CAN handle Form-A NaN / ±Inf bits that surface lex -cannot produce. - -DESIGN.md gained a new top-level §"Float semantics" subsection per -spec section A5: IEEE-754 binary64 commitment, per-op bit stability -on fixed (target, LLVM), NaN / ±Inf propagation, `-0`/`+0` hash-vs- -equality asymmetry, FMA-contraction / reassociation / subnormal -flushing UNSPECIFIED, conversions semantics, Form-A serialisation -shape, Pattern::Lit::Float rejection rationale, and the -`float_to_str` deferred-codegen note. The "What is supported" -bullet at line 2034 was extended to mention `Float` as a primitive -type; the builtins list was refreshed to include the widened ops -(`forall a. (a, a) -> a`), the 5 new fn builtins, the 3 Float -constants, and the 4th IO effect op (`io/print_float`). - -Roadmap flipped Floats from `[~]` to `[x]`; Post-22 Prelude is -unblocked (the `depends on: Floats` line dropped — the partial- -Eq/Ord-for-Float story is now settled and documented in -§"Float semantics", so the Prelude milestone can decide what's -instanced without bouncing back to Float design). - -Per-task commits: - -- `ea8988b` floats iter 5.1: prose renders Float literals (finite + NaN/Inf) -- `2d2646a` floats iter 5.2: DESIGN.md — Float in primitive types + refreshed builtins list -- `47b32bf` floats iter 5.3: DESIGN.md — new §Float semantics subsection (A5 determinism contract) -- `965e628` floats iter 5.4: roadmap — mark Floats [x] + unblock Post-22 Prelude - -## 2026-05-10 — Milestone close: Floats - -The Floats milestone is closed. `Float` is a fully supported -primitive type in AILang, end-to-end. `examples/floats.ail.json` -builds via `ail build`, runs, and produces predictable stdout — -the milestone-acceptance gate landed first try in iter 4.6. - -**Five-iteration arc:** - -- **Iter 1 (schema).** `Literal::Float { bits: u64 }` AST variant - with private `hex_u64` serde helper (16-char lowercase hex string - in canonical JSON; bypasses `serde_json::Number` for bit - stability + NaN/Inf representability). `Float` registered as - primitive name. 8 downstream `match Literal` sites wired with - permanent semantic arms or named-iteration `unimplemented!` - placeholders. Drift-test anchors added to `spec_drift.rs` / - `design_schema_drift.rs` / `form_a.md` / DESIGN.md JSON-schema - block. Bit-stability tests pin A1 / A5 properties. -- **Iter 2 (surface).** Lex `.(e[+-]?)?` - and `e[+-]?` per spec A2; reject bare leading / - trailing dots, missing fraction-after-dot, missing / sign-only - exponents. Parser accepts `Tok::Float` in atom + pat-lit - positions (typecheck rejects pat-lit Float in iter 3). Surface - print emits shortest-round-trippable decimal with `.0` suffix - fallback; non-finite bits panic per spec (cannot reach printer - via well-formed surface input). Round-trip property - `lex(print(L)) == L` pinned for 6 representative bit patterns. -- **Iter 3 (typecheck + builtins).** Widened `+`/`-`/`*`/`/` and - `!=`/`<`/`<=`/`>`/`>=` from monomorphic `(Int, Int) -> {Int, Bool}` - to polymorphic `forall a. (a, a) -> {a, Bool}` (same shape `==` - already had). `%` stays Int-only. 5 new builtins installed: - `neg` (poly), `int_to_float`, `float_to_int_truncate`, - `float_to_str`, `is_nan`. 3 bit-pattern constants installed as - bare-value globals: `nan`, `inf`, `neg_inf`. 1 new effect op: - `io/print_float`. `Pattern::Lit::Float` typecheck-rejected via - new `CheckError::FloatPatternNotAllowed`. -- **Iter 4 (codegen + E2E).** Float literals lower as LLVM hex- - float `double` constants. `synth::builtin_binop` replaced by - 3-tuple-returning `builtin_binop_typed(name, &Type)` — - `fadd/fsub/fmul/fdiv double` and `fcmp olt/ole/ogt/oge/oeq/UNE - double` arms (note `UNE`, not `one`). `lower_eq` Float arm. - 5 new fn-builtin lowering arms (`neg` poly with `fneg double`; - `int_to_float` `sitofp`; `float_to_int_truncate` - `@llvm.fptosi.sat.i64.f64`; `is_nan` `fcmp uno`; `float_to_str` - deferred via structured `CodegenError`). 3 Float constants - intercepted at `Term::Var` arm. `io/print_float` via inline - `printf("%g\n", v)`. `examples/floats.ail.json` E2E fixture + - `crates/ail/tests/floats_e2e.rs` E2E test — milestone gate. -- **Iter 5 (prose + DESIGN.md).** Prose renderer for Float - literals (finite + NaN/Inf). DESIGN.md §"Float semantics" + line- - 2034 + builtins-list refresh. Roadmap mark + Prelude unblock. - -**Key design decisions (rationale):** - -- **One float type only — `f64`, named `Float`** (no `f32`). LLM - authors don't reach for `f32` unprompted; shipping both would - surface a per-op type-pun question that adds friction without - measurable correctness gain. -- **IEEE-conformant equality** — `==` returns `false` for - `nan == nan`, no `Eq` instance for `Float` in the eventual - prelude. The partial-equality reality is real and surfaces - through builtins (`is_nan`, ordering ops returning `false` for - NaN-involved comparisons) rather than through a lying total - typeclass instance. -- **Polymorphic operators over `{Int, Float}` via codegen-dispatch** - — the same mechanism `==` already used. Mainstream alignment - beats the explicit-naming purity of the earlier `(fadd 1.5 2.5)` - draft. The hardcoded `{Int, Float}` filter is transitional; - cleanly replaceable by a `Num a` constraint when typeclasses - ship in 22c. -- **Form-A bit-pattern hex string** — bypasses `serde_json::Number` - (not bit-stable across versions for floats; cannot represent - NaN / ±Inf). Routing through the string path preserves bit-exact - determinism + lets non-finite Floats round-trip through - canonical JSON. -- **Pattern::Lit::Float hard-reject at typecheck** — IEEE semantics - make Float patterns semantically dubious (NaN never matches; - bit-exact equality is rarely the LLM-author's intent). Surface - lex / parser accept the syntax to keep the diagnostic at the - correct layer (typecheck, not parser). -- **`fcmp UNE` for `!=`, NOT `fcmp one`** — `one` is "ordered and - not equal" and returns false for `nan != nan`, violating IEEE- - `!=` and the user-fixed constraint #2. Caught during the - pre-implementation LLVM IR audit; documented in iter-4 codegen - arm + DESIGN.md §"Float semantics". -- **`fneg double` for `neg`, NOT `fsub double 0.0, x`** — IEEE - `0.0 - 0.0 = +0.0`, so the `fsub`-from-zero desugar would - wrongly produce `+0.0` for `neg(+0.0)` instead of `-0.0`. LLVM 8+ - intrinsic; clang 22 always available. -- **`@llvm.fptosi.sat.i64.f64` for `float_to_int_truncate`** — - saturating fp-to-int intrinsic exactly matches Rust `as i64` - semantics (NaN → 0, ±Inf → i64::{MIN,MAX}, saturating). Total, - no Maybe wrapper. - -**Process notes (orchestration lessons):** - -- **Drift tests are part of the schema layer.** The original 5-iter - plan put DESIGN.md anchor edits in iter 5; iter 1's drift-test - break revealed they belong in iter 1 (where they get added - together with the AST variant they document). The exhaustive- - match drift tests are *the enforcement mechanism* of the schema - invariant — adding a new `Literal` variant without their anchors - IS a schema-layer break. -- **Plan inconsistencies caught by the reviewer chain.** Iter 4.2 - shipped a plan that would have stranded comparison ops (Step 4 - said `builtin_binop_typed` returns None for them; Step 5 - narrowed the dispatch matches!). Implementer caught it; pulled - forward Task 3 Steps 3-4 as minimal repair; orchestrator - endorsed; Task 3's residual scope was narrowed accordingly. The - two-stage review (spec + quality) caught two more issues: - the dual-meaning `(instr, ll_ty)` 2-tuple → fixed via 3-tuple - return; the `unimplemented!()` panic for `float_to_str` → - fixed via structured `CodegenError::Internal`. -- **`is_static_callee` companion-edit.** Any new lowering arm in - `lower_app` MUST also be recognised by `is_static_callee` - (otherwise App dispatch falls through to indirect-call → - UnknownVar). This bit twice (iter 4.2 for `==`, iter 4.4 for - the 5 new fn-builtins). Both pulled forward as minimal repairs. -- **The `synth_term` test helper does not exist.** Iter 3 plan - named it; the actual fn is `crate::synth(...)` with 8 args. The - implementer flagged this; future plans should specify the - adapter pattern explicitly. - -**Workspace at milestone close:** 405 tests passing, 0 failed. -Iter-1-baseline rustdoc warnings preserved (1 warning on `desugar` -private link, pre-existing). `def_hash` regression hashes -(`db33f57cb329935e` for `sum.sum`, `b082192bd0c99202` for -`IntList`) preserved bit-identical — adding a Literal variant -does not perturb pre-existing canonical bytes. - -**Known debt deferred beyond this milestone:** - -- `float_to_str` codegen lowering — requires runtime-allocated Str - (the current Str path uses only static `@.str_*` globals; - no malloc-backed dynamic-Str infrastructure). Symbol is - type-installed and surface-callable, but lowering errors with a - structured `CodegenError::Internal`. Future milestone wires the - runtime Str-allocator path; revisit then. -- Float-typed container slot layout — mono pass produces - `List_Fl` etc. naturally per the iter-1 `type_descriptor` `Fl` - arm, but no current fixture exercises a Float-typed container. - Verify when a future use case surfaces. -- Pattern-matching on Float ranges (e.g. `(0.0..1.0)`) — out of - scope; would need a different Pattern variant entirely. -- Float-aware ADT layout for Float-only ADT fields — same - monomorphisation guarantee as `List` would extend; not - exercised by current corpus. - -**Roadmap effects:** Floats `[x]`. Post-22 Prelude unblocked (was -`depends on: Floats`); the Prelude milestone can now ship `Show` -for `Float` and decide-against `Eq` / `Ord` for Float (both -reflecting the partial-equality reality settled in this milestone). - -Next dispatch (orchestrator): `audit` skill — milestone-close -drift review + bench-regression diagnostics + rustdoc audit. -After audit closes clean: `fieldtest` skill — 2-4 `.ailx` -real-world examples exercising the new Float surface. - -## 2026-05-10 — Audit close: Floats milestone - -Audit ran in three stages: - -**Drift review (architect):** clean. No orphan -`unimplemented!("Floats milestone iter ...")` markers. The deferred -`float_to_str` correctly returns `CodegenError::Internal` (not a -panic) per the iter-4.4 fixup. Cross-crate lockstep intact: -`is_static_callee` recognises every name `lower_app` inlines; -`builtin_binop_typed`'s 3-tuple shape consistent at all consumer -sites; `CheckError::FloatPatternNotAllowed`'s `code()` arm present -(`"float-pattern-not-allowed"`). DESIGN.md sweep for stale -`(Int, Int) -> Int` builtin signatures returned only `%` itself -(correctly Int-only). 405 tests pass; 18 rustdoc warnings (= 16 -pre-existing in `ailang-check` registry/uniqueness/synth + 2 -summary lines), none referencing Float code. - -**Bench regression (bencher gate, no investigation):** all three -scripts exit 0. -- `bench/check.py`: 63 metrics, 0 regressed, 4 improved beyond - tolerance (`latency.explicit_at_rc.{p99, p99_9, max, - p99_over_median}`, all -33% to -39%), 59 stable. -- `bench/compile_check.py`: 24 metrics, 0 regressed, 0 improved - beyond tolerance, 24 stable. -- `bench/cross_lang.py`: 25 metrics, 0 regressed, 0 improved - beyond tolerance, 25 stable. - -The 4 explicit_at_rc tail-latency improvements are informational -(the audit gate is "0 regressed"). They are isolated to one -metric family and 3 of 4 are tail-latency p99/p99.9/max which are -notoriously high-variance — most likely environmental -(CPU thermal state, scheduler noise, page-cache warmth) rather -than a real code-path improvement from the Floats milestone. The -milestone did not intentionally touch the RC explicit-position -codegen path; if these improvements are real and reproduce on -next milestone's audit run, ratify with `--update-baseline` then. -For now: carry-on without baseline update. The improvements would -re-trigger the same "improved beyond tolerance" status next run -if real. - -**Rustdoc audit:** N/A — 0 new warnings introduced by the -milestone. The 16 pre-existing warnings (all in `ailang-check` -private-link / typeclass-registry / and `ailang-core` desugar -private-link) are the queued P2 sweep from -`docs/roadmap.md` and unchanged. - -**Verdict:** carry-on. Milestone closes clean. - -Next dispatch: `fieldtest` skill — 2-4 real-world `.ailx` examples -exercising the Float surface to surface friction / spec gaps from -an LLM-author-only-DESIGN.md perspective. - -## 2026-05-10 — Fieldtest close: Floats milestone - -Fieldtest dispatched `ailang-fieldtester` against the Floats -milestone surface (DESIGN.md + form_a.md only — agent did not -read implementation source). Four `.ailx` examples landed in -`examples/fieldtest/`: Newton's-method √2, Int-list mean, -safe-division-with-NaN, `float_to_str` reach-and-bounce. Spec -written at `docs/specs/2026-05-10-fieldtest-floats.md`. - -**Findings (1 bug, 1 friction, 1 spec_gap, 3 working):** - -- **B1 (bug):** `Pattern::Lit::Float` rejection unreachable through - `check_module`. DESIGN.md §"Float semantics" + JOURNAL Floats.3 + - Floats milestone-close entry all promised hard-reject via - `CheckError::FloatPatternNotAllowed`. Reality: surface-lex'd - `(case (pat-lit FLOAT-LIT) BODY)` typechecked, built, and ran - with IEEE-`==` semantics. The audit at `d6da5c2` reviewed the - iter-3.4 reject arm in isolation but missed that - `desugar::build_eq` (extended in iter 1.1 to include - `Literal::Float` in its OR-pattern) rewrites the Pattern::Lit::Float - arm into `(== scrut lit)` BEFORE typecheck runs, making the - reject arm at `lib.rs:2316` unreachable on the public `check` - API path. The iter-3.4 unit test in `builtins.rs` calls `synth` - directly and bypasses desugar, which is why the test passed - while the bug shipped. - - Fixed by adding `crates/ailang-check/src/pre_desugar_validation.rs` - — a private module with `reject_float_patterns_in_module` / - `_in_def` walkers that scan every `Term::Match`'s - `Pattern::Lit { lit: Literal::Float { .. } }` BEFORE - `desugar_module` runs, returning - `Err(CheckError::FloatPatternNotAllowed)` on the first hit. Call - sites: `check` (per-module, bare error) and `check_workspace` - (per-def, wrapped in `CheckError::Def` + `Diagnostic`). The - legacy iter-3.4 typecheck arm at `lib.rs:2316` stays as - defence-in-depth (the iter-3.4 unit test still passes — - defence-in-depth is real). - - Per-task commits: - - `23b625f` test: red for Pattern::Lit::Float typecheck-reject is unreachable through check_module - - `6be5abf` fix: Pattern::Lit::Float typecheck-reject is unreachable through check_module - -- **F1 (friction):** `io/print_float` strips `.0` (`%g\n` formats - `2.0` as `2`, indistinguishable from Int). Surface print always - emits `.` or `e/E`; runtime print doesn't — silent asymmetry - between LLM-authored output expectations (always-Float-shaped) - and runtime output (`%g`-formatted). Queued in `docs/roadmap.md` - as `[todo]` — either switch the runtime path to a `.0`-fallback - printer (matching surface) or document the `%g` contract in - DESIGN.md §"Float semantics" so the LLM-author knows - `io/print_float`'s output is for-humans not round-trip. - -- **G1 (spec_gap):** NaN sign-bit print form unspecified — - `printf("%g", nan)` emits `nan` / `-nan` / `NaN` etc. depending - on libc version + the NaN's sign bit. Both readings of DESIGN.md - A5 are plausible. Tightened DESIGN.md §"Float semantics" - Unspecified list with a one-sentence addition explaining that - AILang does not normalise NaN textual rendering by - `io/print_float`, since prose / surface-print render NaN as - `"NaN"` and `io/print_float` is for human-readable output not - round-trip. - -- **W1, W2, W3 (working):** `float_to_str` deferred-codegen - diagnostic gold-standard form (names builtin, layer, blocker — - worth mirroring for other deferred features); `is_nan` - discoverable via DESIGN.md alone; mixed `Int+Float` - diagnostic clear and actionable. - -**Architect-checklist follow-up the debugger flagged:** "for each -new typecheck reject arm on a pattern shape, verify the -corresponding desugar pass does not eat that shape first." This -is the same lockstep pattern as `is_static_callee` ↔ `lower_app` -flagged in the iter-4 milestone-close entry. Queued to -`docs/roadmap.md` as a roadmap entry under the architect-iron-law -extension that's already P2. - -**Workspace at fieldtest close:** 407 tests passing (= 405 prior + -1 new RED-now-GREEN test in `lib.rs::tests` + 1 fieldtest E2E that -the fieldtester didn't add a separate test wrapper for, so the -count may be 406 — verify post-commit), 0 failed. - -**Pipeline status:** Floats milestone is now well-and-truly -closed. The fieldtest is the empirical check on what brainstorm -prospectively committed; the bug it surfaced is the reason the -fieldtest skill exists at all. - -## 2026-05-10 — Architect iron-law extension: sweep script + lockstep-checklist - -`ailang-architect`'s drift-review iron law extended with two -standing checks, both motivated by failure modes that shipped in -the closing milestones of the typeclass-and-floats arc: - -1. **DESIGN.md history-anchor regrowth.** The four sweep regexes - from the design-md-consolidation milestone (history anchors, - REVERTED + migration, schema SoT, workflow + stale cross- - references) are now packaged as `bench/architect_sweeps.sh` and - run as Step 2.5 of the architect's process. Today's DESIGN.md - is clean against all four; the script's job is to keep it that - way against future commits. Non-zero exit is advisory: the - architect reads each match and decides legitimate-quote vs. fresh - drift. -2. **Lockstep invariants across files.** Two cross-file pairings - are pinned in the agent's "What you check" section as standing - walks: `Pattern::Lit::*` typecheck-rejects ↔ - `pre_desugar_validation.rs` walkers (the B1 lockstep — the - typecheck reject is unreachable through `check_module` if a - desugar pass rewrites the shape first), and `lower_app` arms ↔ - `is_static_callee` recognition (the iter-4.2 / 4.4 lockstep — - a direct lowering arm without `is_static_callee` recognition - falls through to indirect-call with `UnknownVar`). For each - milestone-scope commit-range arm landed in these files, the - architect now opens both files in the pair. - -Both pairings are real, both shipped silently broken at least once, -and both were caught only by post-hoc means (B1 by fieldtest, the -iter-4 ones by mid-iter implementer review). The architect was the -right role to catch them prospectively; the standing checklist is -how that role catches them in future. - -Per-task commits: - -- `67223c8` iter architect-iron-law.1: bench/architect_sweeps.sh runs the four design-md-consolidation sweeps against DESIGN.md -- `6047b38` iter architect-iron-law.2: extend ailang-architect with sweep-script invocation + lockstep-checklist for two known cross-file pairings - -The roadmap entry "Architect-iron-law standing-grep extension" -under P2 is closed by this iteration. Future lockstep pairings, as -they surface in JOURNAL entries, can extend the table inline -without further iteration ceremony. - -## 2026-05-10 — Iteration 23.1: Ordering ADT + prelude skeleton + auto-load - -First iteration of milestone 23 (Eq/Ord Prelude). Lays the -groundwork — no classes or instances ship yet — by establishing -the prelude module as a real loaded module that every workspace -implicitly contains, with bare-name resolution for its ctors via -the existing import-fallback machinery (Iter 15a, -`ailang-check/src/lib.rs::Pattern::Ctor` resolution). - -Three mechanisms compose: - -1. **Embedded prelude.** `examples/prelude.ail.json` is the - LLM-author-readable source; the loader embeds it via - `include_str!` at compile time. No runtime file IO, no CWD - dependency, no fixture-not-found failure mode. -2. **Loader injection.** `load_workspace` inserts the prelude - into `ws.modules` after the user's import DFS finishes and - before `validate_classdefs` / `build_registry`, so the - workspace-wide registry passes see it like any other module. - A user module trying to claim the reserved name `prelude` - fails fast with `WorkspaceLoadError::ReservedModuleName`. -3. **Implicit import.** `build_check_env` adds `prelude → prelude` - to every non-prelude module's `module_imports` map; the - parallel per-module overlay in `check_in_workspace` does the - same for the active `env.imports` table. The existing - bare-ctor fallback in `Pattern::Ctor` resolution finds prelude - ctors through this path, so the user writes `LT` instead of - `prelude.LT`. - -Verified end-to-end through `examples/ordering_match.ail.json` -(pattern-matches a hard-coded `LT` value, prints `1`). Coverage -spans the loader (`loads_workspace_auto_injects_prelude` + -collision via `user_module_named_prelude_is_rejected`), the -typechecker (`user_module_can_pattern_match_on_prelude_ordering_bare`), -the new error path (`cross_module_term_ctor_ambiguous_type_errors`), -and the full pipeline (`ordering_match_via_prelude_prints_1` -through `cargo test -p ail`). - -**Scope expansion authorised mid-iter.** Task 3 surfaced a -real asymmetry: `Pattern::Ctor` had Iter-15a imports-fallback -for bare ctor names, but `Term::Ctor` synth resolved bare -`type_name` via `env.types.get()` only — no fallback. The -spec's bare-name goal for prelude ctors required closing that -gap. Added local-first-then-imports-fallback to `Term::Ctor` -synth (`crates/ailang-check/src/lib.rs:1944-2076`), plus -`CheckError::AmbiguousType` mirroring `AmbiguousCtor`. The -codegen side (`crates/ailang-codegen/src/lib.rs::lookup_ctor_by_type`) -needed the same fallback in Task 4 — the typechecker accepts -the bare reference but codegen's ctor-resolution path was -strictly local. Both touched as part of the iter. - -**Three-site lockstep flagged.** The fix-pattern is now a -three-site invariant: `Pattern::Ctor` imports-fallback (Iter 15a) -↔ `Term::Ctor` typecheck imports-fallback (Iter 23.1.3) ↔ -`Term::Ctor` codegen imports-fallback (Iter 23.1.4). A new arm -in any one site without the matching extension in the other two -risks the silent-divergence failure mode the architect-iron-law -extension just shipped to catch. Candidate for adding to the -lockstep table in a future architect-iron-law iter, alongside -the two already there (`lower_app ↔ is_static_callee`, -`Pattern::Lit::Float reject ↔ pre_desugar_validation`). - -Out of scope for this iter (covered by later 23.x): -- Eq / Ord class definitions (23.2 / 23.3). -- Free top-level utility functions `ne` / `lt` / `le` / `gt` / `ge` (23.4). -- Float-NoInstance diagnostic + DESIGN.md amendment (23.5). - -Per-task commits: - -- `cce3d97` iter 23.1.1: examples/prelude.ail.json — Ordering ADT skeleton -- `3742583` iter 23.1.2: load_workspace auto-injects prelude module -- `927f7ea` iter 23.1.2 fixup: align workspace-root idiom + add collision test for ReservedModuleName -- `842df38` iter 23.1.3: implicit prelude import + symmetric bare-type-name imports-fallback in Term::Ctor synth -- `24af13e` iter 23.1.3 fixup: cover AmbiguousType branch with cross-module test -- `47d95d0` iter 23.1.3 fixup: clarify imports-fallback anchor + comment on env.imports vs env.module_imports duplication -- `aace5e3` iter 23.1.4: E2E fixture — bare LT match via implicit prelude import -- `1f24437` iter 23.1.4 fixup: local-hit with mismatching type_name falls through to imports-fallback - -Workspace at iter-23.1 close: 76 ailang-check tests + 5 env-pin -tests + the new ail-crate E2E + the new workspace-loader tests, -all green. Cross-language and the three regression scripts not -re-run for this iter (no codegen-shape change to the existing -fixtures); they'll run at milestone-23 close. - -## 2026-05-10 — Iteration 23.2: Eq class + three primitive instances - -Second iteration of milestone 23. Ships `class Eq a where eq : (a -borrow, a borrow) -> Bool` in the auto-loaded prelude plus three -primitive instances (Eq Int, Eq Bool, Eq Str). A user program that -calls `eq x y` on any of those three primitives now monomorphises -to `eq__Int` / `eq__Bool` / `eq__Str` synthesised in the prelude -module and codegen lowers them as: - -- `eq__Int` → `icmp eq i64` via existing `lower_eq` Int arm. -- `eq__Bool` → `icmp eq i1` via existing `lower_eq` Bool arm. -- `eq__Str` → `call zeroext i1 @ail_str_eq(...)` via a new codegen - intercept (`try_emit_primitive_instance_body` in - `crates/ailang-codegen/src/lib.rs`). - -The Str path is the only one needing new mechanism: a hand-rolled -body in `emit_fn` that bypasses normal lambda lowering and calls -the new C-runtime primitive `ail_str_eq` from `runtime/str.c`. -`runtime/str.c` is linked unconditionally for every alloc strategy -(Gc, Bump, Rc) — the `@ail_str_eq` IR declaration is unconditional -so the symbol must always resolve at link time; clang -O2 dead- -strips it when no caller exists. `==` on Str stays on `@strcmp + -icmp eq i32 0` this iter; only the mono-synthesised `eq__Str` -goes through `@ail_str_eq`. Routing primitive `==` through the -class methods is the declared P2 follow-up. - -**Two latent bugs surfaced.** Adding `class Eq` to the prelude -flipped the `workspace_has_typeclasses` gate from false to true -on every test workspace, exposing two long-dormant gaps: - -1. **`mono::build_workspace_env` ctor_index was workspace-flat.** - `check_in_workspace` clears the flat ctor_index after - `build_check_env` and rebuilds it per-module (lib.rs:1257-1287) - so `Pattern::Ctor` resolution at lib.rs:2486-2526 produces a - qualified `resolved_type_name` via the imports-fallback that - matches the scrutinee's qualified `Type::Con`. The mono entry - points (`collect_targets_workspace_wide`, the phase-3 rewrite - loop) consumed the flat-index env directly, so cross-module - `Cons` patterns resolved against bare `"List"` and mismatched - the qualified `"std_list.List"`. Fix in `84dcc46`: per-module - ctor_index overlay helper in mono.rs, applied at both sites. - Empirically `env.types` also had to be overlayed (the carrier's - "asymmetry" note was wrong — kept flat, the Term::Ctor and - Pattern::Ctor synth paths disagree on bare-vs-qualified - names). RED test: `crates/ail/tests/mono_xmod_ctor_pattern.rs` - (`e580f75`). Same family as commits 13b36cc / 5c5180f. - -2. **Codegen `lower_workspace_inner` import_map missed implicit - prelude.** Symmetric to the typechecker's implicit-prelude - injection at `build_check_env` / `check_in_workspace` (Iter - 23.1.3). When Iter 22b.3 monomorphisation rewrites a user - call `eq x y` to the cross-module symbol `prelude.eq__Int`, - `lower_call`'s prefix resolution looked up `"prelude"` in the - codegen import_map and found it missing — error - `cross-module call 'prelude.eq__Int': prefix 'prelude' not - in import map`. Fix in `be882c4`: mirror the typechecker's - guard (`m.name != "prelude"`) and inject the implicit entry. - -**Four-site lockstep now confirmed.** Iter 23.1 flagged a -three-site lockstep for implicit-prelude visibility (Pattern::Ctor -15a / Term::Ctor typecheck / `lookup_ctor_by_type` codegen). -Iter 23.2 just added the fourth site: -`lower_workspace_inner` import_map. The lockstep table is now -ripe for the architect-iron-law extension that already covers -`lower_app ↔ is_static_callee` and `Pattern::Lit::Float reject ↔ -pre_desugar_validation`. Adding it next time the architect role -takes an iter. - -**Fixture rename ripple.** Six existing test fixtures used a -local `class Eq` (some with `class Ord extends Eq`) for typeclass- -machinery tests. With the prelude's `class Eq.eq` injected, the -fixture-side methods collide on the global method-name uniqueness -check (`MethodNameCollision`). Five out of six were currently -visible failures (two surfaced first, three more after the -mono ctor_index bug was fixed). Renamed the colliding classes -and methods to `TEq` / `teq` / `TOrd` / `tlt` across the 6 -JSON fixtures (the prose snapshot for one of them too) and -updated the corresponding test assertions in workspace.rs. The -`iter22b1_workspace_with_no_classes_has_empty_registry` and -`iter22b1_instance_in_class_module_loads_clean` tests were -rewritten to filter `defining_module == "prelude"` rather than -counting raw registry entries — they now assert "no NON-prelude -entries" / "exactly one non-prelude entry", which is the -property they always meant. - -Coverage at iter close: -- Codegen unit + integration tests: `eq__Str`-intercept-shape - test, `eq__Str` closure-adapter test, three IR-shape tests - on the smoke fixture (`@ail_prelude_eq__Int` + `icmp eq i64`, - Bool / i1, Str / `call zeroext i1 @ail_str_eq`). -- E2E: `examples/eq_primitives_smoke.ail.json` → - `eq_primitives_smoke_compiles_and_runs` (full pipeline: - AST → typecheck → mono → codegen → clang → binary → stdout - `"1\n0\n1\n0\n1\n0\n"`). -- Mono RED-test: `mono_xmod_ctor_pattern.rs` (guards the - per-module ctor_index overlay against regression). -- Prelude-load assertion extended (`loads_workspace_auto_injects_prelude`) - to also check `class Eq` + Eq Int/Bool/Str instances. - -Per-task commits: - -- `cc2d694` iter 23.2.1: runtime/str.c with ail_str_eq + unconditional link -- `736064a` iter 23.2.2: codegen — declare @ail_str_eq + eq__Str body intercept -- `c6168ad` iter 23.2.2-fixup: emit closure adapter for primitive-instance-bodied fns -- `1618182` iter 23.2.2-fixup-doc: tighten try_emit_primitive_instance_body contract doc -- `e580f75` RED test (debug): mono_xmod_ctor_pattern.rs pins flat-ctor_index bug -- `84dcc46` fix: mono.rs per-module env.ctor_index overlay (same family as 13b36cc / 5c5180f) -- `7289bbc` iter 23.2.3-prep: reconcile workspace tests with auto-loaded prelude class Eq -- `65ab6c6` iter 23.2.3-prep2: rename remaining two 22b2 fixtures + their prose snapshot (orchestrator-inline) -- `7172e7e` iter 23.2.3: prelude — class Eq a + Eq Int/Bool/Str instances -- `be882c4` fix: codegen lower_workspace_inner implicit prelude import (4th lockstep site) -- `559e591` iter 23.2.4: e2e smoke fixture + ir-shape integration tests for eq__T mono symbols -- `a11cb2f` iter 23.2.4-fixup: symmetrize check_workspace error assertion across three IR-shape tests - -Out of scope for this iter (covered by later 23.x): -- Ord class + three Ord instances + `ail_str_compare` (23.3). -- Free top-level utility functions `ne` / `lt` / `le` / `gt` / `ge` (23.4). -- Float-NoInstance diagnostic + DESIGN.md amendment (23.5). - -Workspace at iter-23.2 close: full `cargo test --workspace` green -(416 tests, 0 failed). Cross-language and the three regression -scripts not re-run for this iter; they run at milestone-23 close. - -## 2026-05-11 — Iteration ct.1: canonical type names — validator + migration - -First iteration of the canonical-type-names milestone. Spec at -`docs/specs/2026-05-10-canonical-type-names.md`. The milestone -formalises an unambiguous rule for `Type::Con` names in `.ail.json`: -within a file's module context (`"name"` at the top), bare = -local, qualified `.` = explicit cross-module, -and primitives stay bare. The bug-class that motivated the -milestone (iter 23.3 Task 4: bare cross-module references silently -mismatching when one side resolves through imports-fallback and -the other stays bare) becomes structurally impossible because -there's no bare cross-module form in canonical JSON. - -What ct.1 shipped: - -- **Validator** (`validate_canonical_type_names` in - `crates/ailang-core/src/workspace.rs`): walks every `Type::Con` - in Type position (incl. `Term::Lam.param_tys`, `Term::Lam.ret_ty`, - `Term::LetRec.ty`) and every `Term::Ctor.type_name` in Term - position. Three new `WorkspaceLoadError` variants: - `BareCrossModuleTypeRef` (with `candidates` from imports-in-order - + implicit prelude), `BadCrossModuleTypeRef` (qualified owner - unknown or doesn't declare the named type), `QualifiedClassName` - (one of four fields — `ClassDef.name`, `InstanceDef.class`, - `SuperclassRef.class`, `Constraint.class` — contains a `.`). - Wired into `load_workspace` between prelude injection and - `validate_classdefs` so the canonical-form diagnostic fires - before any downstream check that would have papered over the - malformed shape. - -- **Migration tool** (`ail migrate-canonical-types `): dev-only - CLI subcommand that walks `/*.ail.json`, rewrites bare - cross-module Type::Con + Term::Ctor.type_name references to their - qualified form via the first-match-in-imports-order disambiguation - rule (mirrors iter 23.1.3's typecheck-side imports-fallback on - the happy path; explicitly weaker than 23.1.3's `ambiguous-type` - diagnostic since this is one-shot tooling). Prelude treated as - implicit last-resort fallback. Idempotent. After ct.1.5 ran - the tool over `examples/`, exactly two fixtures shifted: - `ordering_match.ail.json` (Term::Ctor `Ordering` → - `prelude.Ordering`) and `test_22b1_dup_a.ail.json` (Type::Con - `MyInt` → `test_22b1_dup_b.MyInt`). - -- **Registry-side normalize-before-hash** (ct.1.5a discovery — - not in the original plan). The migration exposed a real semantic - gap: under the new canonical-form rule, a type defined in - module B is referenced bare-local in B (`Type::Con { name: - "MyInt" }`) but qualified `B.MyInt` when referenced from - elsewhere. The registry's `canonical::type_hash(&inst.type_)` - compares Type::Con name bytes verbatim, so two coherent - instances of the same type-def (one bare-local, one - qualified-cross-module) would hash differently and the - duplicate-instance detection would silently miss. Fix: - `Registry::normalize_type_for_lookup` qualifies bare-locals - via a `type_def_module: BTreeMap` lookup - before hashing. Four consumer call sites in `ailang-check` - (one in `lib.rs::no-instance` lookup, three in `mono.rs`) - funneled through the same method so the registered-form ↔ - queried-form contract is enforceable from one helper rather - than by grep. The defining-module-qualifier choice (vs. - owning-module) is documented as the right call for the - canonical-form-compliant corpus; a docstring on - `Registry.type_def_module` flags the latent bare-name- - collision-across-modules edge case as future-tidy debt - (would require re-keying the map to `(owning_module, bare_name)`). - -**The dead `KindMismatch` arm.** The fixture -`test_22b2_kind_mismatch.ail.json` uses `Type::Con { name: "f", -args: [Type::Var{a}] }` where `f` is meant to be the class param -in applied position — a malformed shape that pre-ct.1 fired -`KindMismatch`. Under ct.1's validator, that same shape is now -caught earlier as `BareCrossModuleTypeRef` (because `f` is bare, -non-primitive, and not a TypeDef anywhere). The -`validate_classdefs::walk_kind_mismatch` path becomes -structurally unreachable through well-formed schema. The -regression test was renamed to -`class_param_in_applied_position_fires_canonical_form_rejection` -and re-targeted to the new diagnostic; the kind-mismatch code -stays in the codebase as dead-but-defensive for now (cleanup -queued as future tidy). - -**Friction surfaced — CLI human-mode diagnostic surface.** While -writing the ct.1.8 E2E tester noticed that non-JSON `ail check` -does NOT route ct.1 errors through `workspace_error_to_diagnostic`; -human-mode stderr lacks the bracketed `[diagnostic-code]` prefix -that JSON mode carries (it goes via the `anyhow`/`thiserror` -Display impl as `Error: ` instead). Plausibly the same -asymmetry applies to other CLI subcommands that call -`load_workspace(&path)?` directly. Not a ct.1 bug — pre-existing -CLI architecture issue — queued as P2 follow-up in roadmap. - -**Plan vs. reality.** The plan's Task 5 ("run migration on -`examples/`") expected `cargo test --workspace` to be green right -after. It wasn't. The duplicate-instance test broke because the -registry hash didn't normalize. ct.1.5a (the registry-normalize -fix) was a mid-iteration scope expansion. Lesson: the plan named -the four obsolete typechecker / codegen mechanisms as ct.2/ct.3 -targets, but missed a fifth consumer site (the registry's -type-hash key) that's neither typechecker nor codegen — it's -workspace-load metadata. Future spec-writing for cross-cutting -canonical-form changes should explicitly enumerate ALL consumer -sites of the hashed/keyed AST shape. - -**4 obsolete mechanisms NOT cleaned up here.** Per spec, the -four imports-fallback / overlay mechanisms named in §"Mechanisms -made obsolete" (Pattern::Ctor imports-fallback, Term::Ctor synth -imports-fallback, `lookup_ctor_by_type` fallback, mono -`ctor_index` overlay) stay in place for ct.1 — they continue to -work on the migrated form (qualified bare → direct lookup -succeeds before fallback fires). Removal is ct.2 (typechecker) -and ct.3 (codegen+mono) work. - -Coverage at iter close: -- 17 new unit tests under `ct1_validator_*` / `ct1_fixture_*` / - `ct1_registry_*` / `ct1_5a_*` in - `crates/ailang-core/src/workspace.rs::mod tests`. -- 2 E2E tests in `crates/ail/tests/migrate_canonical_types.rs` - (Term::Ctor + Type::Con rewrite paths, idempotence). -- 5 E2E CLI tests in `crates/ail/tests/ct1_check_cli.rs` - (one per new diagnostic code through `--json` mode, one for - human-mode surface, one happy-path on `ordering_match`). -- 3 on-disk fixtures under `examples/test_ct1_*.ail.json` pin - the diagnostic surface against schema drift. -- Existing `iter22b1_duplicate_instance_fires_diagnostic` test - continues to pass on the migrated state (via the registry - normalize). `test_22b2_kind_mismatch`'s regression test - renamed + retargeted to `BareCrossModuleTypeRef`. - -Per-task commits: - -- `000b84d` iter ct.1.1: validator scaffolding + Type::Con canonical-form rules -- `3723495` iter ct.1.2: validator extends to Term::Ctor.type_name -- `9e2e843` iter ct.1.3: validator rejects qualified class-name fields -- `219908e` iter ct.1.3-followup: test for Constraint.class in ClassDef-method Forall -- `d45977f` iter ct.1.4: dev-only ail migrate-canonical-types subcommand -- `a73f7d3` iter ct.1.4-followup: type-con rewrite test + doc clarity -- `7872ccf` iter ct.1.5a: registry-side normalize-before-hash -- `5e653ff` iter ct.1.5a-followup: forall-constraints recursion + docstring + drop iteration-tag comments -- `d3280e9` iter ct.1.5: migrate bare cross-module type refs in examples/ -- `727d2c1` iter ct.1.6: wire validator into load_workspace; refresh kind-mismatch test -- `f22f88b` iter ct.1.7: on-disk fixture tests for canonical-form diagnostics -- `c1783e0` iter ct.1.8: E2E coverage for canonical-form diagnostic surface + happy path - -Out of scope (covered by later ct.x): -- Removal of 4 obsolete imports-fallback / overlay mechanisms (ct.2 typechecker, ct.3 codegen+mono). -- DESIGN.md amendments to Decision 2 (content-addressed defs) + hash-stability promises review (ct.4). -- Re-baseline of hash-pinned regression tests affected by the 2 migrated fixtures (ct.4). -- Form-B prose round-trip test extension for the qualifier-trim-on-print behaviour (ct.4). - -Workspace at iter-ct.1 close: full `cargo test --workspace` green -(roughly 440+ tests across crates, 0 failed). Cross-language and -the three regression scripts not re-run for this iter (no codegen- -shape change to existing fixtures beyond the surgical migration; -they run at canonical-type-names milestone close after ct.4). - -## 2026-05-11 — Iteration ct.2: typechecker cleanup - -Three tasks landed in the canonical-type-names milestone's -typechecker-cleanup iteration, plus targeted follow-ups driven by -the quality-review pass. - -- **ct.2.1** (`78ccbce`): wired `qualify_local_types` into the - `env.class_methods` channel of `Term::Var` resolution, closing - the last cross-module fn-lookup boundary that did not run the - qualifier. The motivating shape: prelude declares - `class Ord a` with `compare : a -> a -> Ordering`; the method's - `Ordering` is bare-local to prelude; without the qualifier on - the class-method channel, a consumer module sees bare - `Ordering` and its pattern match against the post-ct.1 - canonical `prelude.Ordering` scrutinee no longer resolves. - Also fixed a latent gap in `qualify_local_types`'s - `Type::Forall` arm: it now recurses into `constraints[].type_`, - symmetric to the ct.1.5a-followup fix on - `normalize_type_for_registry`. Two new tests pin both - invariants. -- **ct.2.2** (`b3c3b60` + followups `0ca5b3d`, `0044467`, - `edb694f`): deleted the `Pattern::Ctor` imports-fallback (iter - 15a); replaced with type-driven lookup keyed on - `expected: Type::Con`'s canonical name. The lookup no longer - consults `env.ctor_index`; the per-module overlay now serves - only duplicate-detection at workspace build. The - `cross_module_pat_ctor_ambiguous_errors` test deleted - (ambiguity unreachable post-refactor; lookup is anchored to a - single TypeDef). The `CheckError::AmbiguousCtor` variant + - its `ambiguous-ctor` diagnostic code deleted as dead code - (quality-review pushed back on leaving it as - dead-but-defensive — the project rule is "no backwards-compat - shims for removed features"). The now-unused `Bag` test - fixture in `cross_module_ws` removed in `0044467`. Doc - comments on `UnknownCtorInPattern` and - `mono_xmod_ctor_pattern.rs` refreshed in `edb694f` to describe - the post-ct.2 mechanism. The `mono_xmod_ctor_pattern.rs` - regression test still passes — its scrutinee carries the - qualified type and the new lookup finds the TypeDef directly. -- **ct.2.3** (`97bb5e7`): deleted the `Term::Ctor` synth - imports-fallback (iter 23.1.3). Bare cross-module `type_name` - now fails closed with `UnknownType`; the post-ct.1 workspace - validator catches the shape at load time for canonical inputs. - The `CheckError::AmbiguousType` variant deleted (symmetric to - the `AmbiguousCtor` cleanup; same dead-code reasoning). Two - tests deleted (`cross_module_term_ctor_ambiguous_type_errors` - and `user_module_can_pattern_match_on_prelude_ordering_bare` - — both pinned behaviour that ct.2 removes); two tests added - (`ct2_term_ctor_bare_cross_module_fails_closed` and - `ct2_term_ctor_qualified_cross_module_resolves`). - -Remaining obsolete mechanisms (ct.3 scope): the codegen -`lookup_ctor_by_type` imports-fallback (iter 23.1.4) and the -mono `apply_per_module_ctor_index_overlay` (iter 23.2 fix -`84dcc46`). Surviving stale doc-comments at -`crates/ailang-codegen/src/lib.rs:1748` and -`crates/ail/src/main.rs:239` reference the now-removed -`AmbiguousType` / `AmbiguousCtor` paths; ct.3 (codegen) and -ct.4 (the migrate-canonical-types subcommand may itself retire -as one-shot tooling) will tidy those when the surrounding code -changes. - -Lesson from ct.2.2 quality review: the orchestrator's plan -explicitly said "leave `CheckError::AmbiguousCtor` as -dead-but-defensive, defer to a later tidy". The quality -reviewer correctly pushed back, citing CLAUDE.md's -"Avoid backwards-compatibility hacks like renaming unused -\_vars, re-exporting types, adding `// removed` comments for -removed code, etc. If you are certain that something is unused, -you can delete it completely." Lesson logged for future plan -writing: when an enum variant becomes unused as a result of -the iter's own changes, the delete belongs in the same iter, -not a later tidy. The roadmap was updated implicitly — both -variants are now gone, no roadmap entry needed. - -Workspace at iter-ct.2 close: full `cargo build --workspace` and -`cargo test --workspace` green (451 tests, 0 failed). Bench -scripts not re-run (no codegen-shape change, no fixture -migration this iter). E2E coverage skipped per implement-skill -carve-out (ct.2 ships internal refactor only; per-task pin -tests + the existing workspace test surface cover the -invariants; the canonical E2E demonstration of the -iter-23.3-Task-4 bug closure happens in ct.4 with the recreated -`compare_primitives_smoke.ail.json` fixture). - -## 2026-05-11 — Iteration ct.3: codegen + mono cleanup - -Three tasks landed, plus one orchestrator-level tidy from -quality-review feedback: - -- **ct.3.1** (`a8a58cc`, tidy `05d0cce`): deleted the - `lookup_ctor_by_type` imports-walk fallback in - `crates/ailang-codegen/src/lib.rs`. Two-branch direct lookup - remains: qualified via `import_map` → - `module_ctor_index[target_module]` (with `cref.type_name == - suffix` check); bare via `module_ctor_index[self.module_name]` - (with `cref.type_name == type_name` check). Both hard-error on - miss. Symmetric to ct.2.3's `Term::Ctor` synth fix on the - typecheck side. The lookup function's docstring and the - surrounding lockstep-rationale comment were refreshed; the - small `-tidy` followup dropped a dated "ct.3 Task 1:" comment - prefix flagged by the quality reviewer. -- **ct.3.2** (`9149801`): narrowed - `apply_per_module_ctor_index_overlay` to `env.types` only and - renamed it to `apply_per_module_types_overlay`. The - `env.ctor_index` half of the original 22c overlay became - decorative after ct.2.2 stopped consulting the index from - Pattern::Ctor; the `env.types` half is still load-bearing for - Term::Ctor bare-name lookup in mono's re-run synth. Both call - sites updated, the rationale comment rewritten to drop the - obsolete Pattern::Ctor framing, the now-unused `CtorRef` import - dropped, and the dangling rustdoc link in - `build_workspace_env`'s docstring re-pointed to the new - function name. -- **ct.3.3** (`666c784`): retired the obsolete "parity with - `ambiguous-type` diagnostic" paragraph in the - `MigrateCanonicalTypes` doc-comment - (`crates/ail/src/main.rs`). The diagnostic it referenced was - removed in ct.2.3. - -All four obsolete mechanisms named in the ct.1 JOURNAL entry are -now gone: -- Pattern::Ctor imports-fallback (ct.2.2). -- Term::Ctor synth imports-fallback (ct.2.3). -- codegen `lookup_ctor_by_type` imports-fallback (ct.3.1). -- mono `apply_per_module_ctor_index_overlay`'s `env.ctor_index` - half (ct.3.2 narrowing). - -Only ct.4 remains in the canonical-type-names milestone: -DESIGN.md amendments to Decision 2, hash-pinned regression test -re-baseline for the two migrated cross-module fixtures, prose -round-trip extension covering the qualifier-trim-on-print -behaviour, and the optional -`examples/compare_primitives_smoke.ail.json` recreation that -demonstrates end-to-end closure of the iter-23.3-Task-4 bug -through the full pipeline (parse → typecheck → mono → codegen -→ run). - -Out of scope this iter, deliberately: -- Codegen `lookup_ctor_in_pattern` (lib.rs:1790) — uses - imports-walk but has no scrutinee type to type-anchor; would - require plumbing Type through pattern lowering, a structurally - bigger refactor than ct.3's scope. -- The codegen `lower_workspace_inner` implicit-prelude - `import_map` entry (iter 23.2.4 fix `be882c4`) — different - problem (fn-name resolution for mono symbols, not Type::Con - names). -- `check_in_workspace`'s analogous per-module overlay in - `crates/ailang-check/src/lib.rs:1234` — its `env.ctor_index` - half still serves the duplicate-detection diagnostic at - workspace-build time, not the runtime ctor lookup; narrowing - it would be a separate concern with its own correctness - rationale. - -Workspace at iter-ct.3 close: full `cargo build --workspace` and -`cargo test --workspace` green (451 tests, 0 failed). Test count -unchanged from ct.2 — this iter is pure code deletion + doc tidy. - -## 2026-05-11 — Iteration ct.4: canonical-type-names milestone close - -Four tasks landed, plus one inline doc-tidy, closing the -canonical-type-names milestone: - -- **ct.4.1** (`61ad3e3`): amended DESIGN.md Decision 2 with the - canonical Type::Con name scoping rule. Bare = local to the - file's owning module; qualified `.` = - explicit cross-module; primitives are bare. The class-name - exception is named in DESIGN with a forward pointer to the - later milestone that will retire it. The hash-stability - paragraph at the surrounding "iter19b regression-pinned by" - citation was updated to name the two new ct.4 pins. - -- **ct.4.2** (`1d039b7`): added two pin tests in - `crates/ailang-core/src/hash.rs`. - `ct4_migrated_fixtures_have_canonical_form_hashes` locks the - post-migration hashes of `ordering_match.ail.json` - (`8d17235aa3d2e127`) and `test_22b1_dup_a.ail.json` - (`1c2573661ffd3da3` for the Show class def, - `cc8685f92f246e40` for the Show instance def — indexed access - because `Def::name()` returns the class name for both). - `ct4_unmigrated_fixtures_remain_bit_identical` re-asserts - `sum.sum` and `list.IntList` did not drift across the - canonical-form tightening. - -- **ct.4.3** (`06bc5f0`, doc-tidy `b651b1e`): threaded - `owning_module: &str` through every recursive `write_*` fn in - `crates/ailang-prose/src/lib.rs`. Type::Con's name now gets - its qualifier trimmed when the owner matches the current - file's module. Two new unit tests pin the trim-applies and - trim-doesn't-apply directions. One new snapshot fixture - `examples/ordering_match.prose.txt` is committed. - Implementer surfaced a plan-error: the prose printer ELIDES - `Term::Ctor.type_name` entirely (the type is recoverable - from typecheck context), so the trim has no analogue there. - The fixture's `prelude.Ordering` ref is on a Term::Ctor and - therefore does not appear in the snapshot — qualifier-trim - coverage is the two unit tests, the snapshot pins - the overall print shape. - -- **ct.4.4** (`ba48349`): recreated - `examples/compare_primitives_smoke.ail.json` — the - iter-23.3-Task-4 demonstration fixture. 9 calls to bare - `compare` (Int / Bool / Str × 3 pairings each), 9 prints, - expected stdout `1\n2\n3\n1\n2\n3\n1\n2\n3\n`. One new E2E - test in `crates/ail/tests/e2e.rs` asserts the stdout - verbatim. The IR-shape test originally planned was DROPPED: - `emit-ir` CLI bypasses `monomorphise_workspace`, so the - `compare__Int` / `compare__Bool` / `compare__Str` mono - symbols are unobservable via that path. The E2E stdout - assertion is the load-bearing proof — if any of the three - mono symbols were missing, the run would fail. The IR-shape - test is logged as a P3 follow-up. - - Plan-level adaptation: the plan said to use qualified - `prelude.compare` for the class-method call. Implementer - correctly switched to bare `compare`. Class methods resolve - via `env.class_methods` (workspace-flat, bare-keyed); the - qualified `prelude.compare` path would route through - `env.module_globals[prelude].fns`, which does NOT contain - class methods. The bare convention matches the existing - `eq_primitives_smoke.ail.json` precedent. - -The canonical-type-names milestone closes: - -- The structural invariant: every `Type::Con` and - `Term::Ctor.type_name` in canonical `.ail.json` is either - bare (= local to the file's module) or qualified - (`.`, with primitives bare). -- The four obsolete imports-fallback / overlay mechanisms named - in the ct.1 catalogue are gone: `Pattern::Ctor` - imports-fallback (ct.2.2), `Term::Ctor` synth imports-fallback - (ct.2.3), codegen `lookup_ctor_by_type` imports-fallback - (ct.3.1), mono `apply_per_module_ctor_index_overlay`'s - `env.ctor_index` half (ct.3.2 narrowing). -- The iter-23.1 / iter-23.3-Task-4 bug-class is empirically - closed: the smoke fixture compiles, runs, and prints the - expected 9-line output end-to-end without any further - compiler changes. - -Iter 23.3 Task 4 + Task 5 (the paused fixture-recreate + JOURNAL -work) are formally complete: the fixture is at -`examples/compare_primitives_smoke.ail.json`, the E2E test is at -`crates/ail/tests/e2e.rs::compare_primitives_smoke_prints_1_2_3_thrice`, -and the JOURNAL entry is this paragraph block. - -Skill-system pipeline next steps: - -1. `audit` (mandatory at milestone close): architect drift - review against `docs/specs/2026-05-10-canonical-type-names.md` - and DESIGN.md; bench regression diagnostics; optional - rustdoc cleanup. -2. `fieldtest` (after audit closes clean): real-world `.ailx` - examples written against DESIGN.md only, exercising the - canonical-form rule from the LLM-author surface. - -Out of scope at milestone close, logged as follow-ups: - -- **[P3]** Codegen `lookup_ctor_in_pattern` (lib.rs:1790) — - imports-walk with no scrutinee type to type-anchor. Plumbing - scrutinee Type through pattern lowering would type-anchor it - symmetric to the ct.2.2 typecheck-side fix. -- **[P3]** `compare_primitives_smoke` IR-shape assertion — - observe `compare__Int` / `compare__Bool` / `compare__Str` - symbols in the emitted IR. Blocked on `emit-ir` CLI not - running mono; three resolution paths (extend the CLI, use - library APIs directly in e2e.rs, add `--dump-ir` to `ail - build`) all need their own scope. -- **[P2]** Class-name canonical-form scoping — the - `MethodNameCollision` workaround at - `crates/ailang-core/src/workspace.rs:472` keeps bare class - names viable; type-driven method dispatch is a separate - milestone (already on the roadmap). -- **[P2]** `check_in_workspace`'s analogous per-module overlay - in `crates/ailang-check/src/lib.rs:1234` — the env.ctor_index - half there serves the duplicate-detection diagnostic at - workspace-build time, not the runtime ctor lookup; out of the - canonical-form scope but a clean tidy when context returns to - the typecheck pipeline. - -Workspace at iter-ct.4 close: full `cargo build --workspace` -and `cargo test --workspace` green (457 tests, 0 failed — net -+6 over ct.3's 451: 2 pin tests in hash.rs, 2 prose trim unit -tests, 1 prose snapshot, 1 E2E test). - -## 2026-05-11 — Audit close: canonical-type-names milestone - -Audit pipeline ran clean per skill protocol. - -**Architect drift review:** zero items. DESIGN.md's Decision 2 -amendment matches what the code enforces; the four obsolete -mechanisms in the ct.1 catalogue are all gone or correctly -narrowed; `qualify_local_types` is applied uniformly at all three -cross-module fn-lookup sites; the five remaining "bit-identical" -paragraphs in DESIGN.md refer to features orthogonal to the -canonical-form migration and remain factually correct. JOURNAL -test-count claim (457 at iter close) verified against -`cargo test --workspace`. Spec acceptance criteria 1-8 empirically -satisfied. - -**Bench-regression check:** `bench/check.py` and `bench/cross_lang.py` -green at exit 0 (0 / 88 metrics regressed). `bench/compile_check.py` -initially reported 23 / 24 regressed (compile-time +27-44% -across every fixture). Bencher dispatched to localise; the -result refuted the initial hypothesis (validator-causes-regression) -with N=15 median measurements: validator's measured CPU -contribution is 0.01-0.07 ms per `ail check`, within noise. - -Localised to **iter 23.3.3** (`12d9130` — `Ord extends Eq` + 3 -Ord instances added to `examples/prelude.ail.json`). The prelude -grew from 18 to 223 lines between `3742583` and `12d9130`. The -cost is paid on every `load_workspace`: parse, canonicalise, -typecheck the prelude module, register its 6 instances. The -bench fixtures are tiny (hello = 4 defs); the prelude is now -larger than every user fixture in the corpus and dominates load -time. ct.1.* contributes a residual +0.05-0.15 ms on top — not -material against the ~+20 ms build floor. - -**Resolution:** ratify. The prelude growth is a deliberate -post-iter-23.3 design state (Eq + Ord classes with three primitive -instances each are the canonical typeclass surface; the prelude -is auto-loaded for every workspace per iter 23.1.2). Constant -+17 ms per `ail build` invocation is the documented price of -the canonical typeclass prelude. Baseline rebased via -`bench/compile_check.py --update-baseline`; post-rebaseline run -is 24 / 24 stable. - -Lesson logged: when a milestone closes and a bench script reports -broad regressions, the canonical move is to localise BEFORE -ratifying. Without the bisect, ratifying would have written -"canonical-type-names caused +30% compile time" into the -JOURNAL — a false attribution that would mislead future -archaeology. - -**Rustdoc:** 20 pre-existing-pattern warnings (18 carried from -earlier, +2 introduced by ct.1.5a's `type_def_module` doc-link -and ct.3.2's `apply_per_module_types_overlay` rename). The -existing P2 roadmap entry "Rustdoc warning sweep" covers them; -not promoted to a tidy iteration on this audit. - -**Audit verdict:** carry-on. Canonical-type-names milestone is -closed. Skill pipeline next step: `fieldtest`. - -Follow-up not from this audit but worth flagging as a P3 idea: -optimising the prelude load path. A `OnceCell`-cached parsed -prelude (or a content-addressed disk cache) would recover most -of the ~17 ms per-invocation cost. Quantified by the bencher -above as worth doing only if the prelude-loading cost is judged -structurally permanent. Add to roadmap as P3 if the next -fieldtest surfaces it as friction. diff --git a/docs/journals/2026-05-11-fieldtest-canonical-type-names.md b/docs/journals/2026-05-11-fieldtest-canonical-type-names.md deleted file mode 100644 index eec23d7..0000000 --- a/docs/journals/2026-05-11-fieldtest-canonical-type-names.md +++ /dev/null @@ -1,75 +0,0 @@ -# fieldtest — canonical-type-names - -**Date:** 2026-05-11 -**Branch:** main (orchestrator-direct dispatch; the or.1 branch-per-iter rule lands on the *next* implement run) -**Status:** DONE_WITH_CONCERNS -**Examples:** 5 of 5 authored + committed -**Findings:** 9 (4 working, 4 friction, 1 spec_gap, 0 bugs) - -## Summary - -Standard skill-pipeline step after audit cleanly closed -canonical-type-names. The fieldtester wrote five `.ailx` examples -that exercise the milestone surface — one happy-path exhibit and -four diagnostic-triggers, one per new diagnostic plus a second -branch of the merged BadCrossModuleTypeRef case. Spec at -`docs/specs/2026-05-11-fieldtest-canonical-type-names.md`; fixtures -under `examples/ct_*`. - -All four milestone-scoped findings landed as **working**: the new -diagnostics fire on the LLM-natural authoring mistakes, the -diagnostics carry actionable recipes (BareCrossModuleTypeRef lists -import candidates + names the migration tool), and Form-A -round-tripping stays byte-stable across cross-module type refs. - -One milestone-internal friction: `BadCrossModuleTypeRef` collapses -two distinguishable cases (unknown owner vs. known owner / unknown -type name) into one diagnostic shape. A future tidy iter can split -these and, in the known-owner branch, list the available type names -the way `bare-cross-module-type-ref` lists candidates from imports. - -Three orthogonal friction items surfaced as pre-existing UX gaps -that every fieldtest invocation will keep tripping on: - -- Workspace search confined to the entry module's directory — no - way to organise fieldtest fixtures under - `examples/fieldtest/` while still importing prelude or std. -- `ail check foo.ailx` returns a misleading JSON-parse error - instead of either auto-parsing `.ailx` or pointing to `ail parse`. -- `(import prelude)` is rejected with a clear diagnostic but no - corpus example warns the author that prelude is auto-injected. - -One spec_gap: the canonical happy-path exhibit shipped with this -milestone (`examples/compare_primitives_smoke.ail.json`) exists -only as JSON. Decision 6 makes Surface the LLM-author surface; the -milestone should ship a `.ailx` counterpart (or retire the JSON -fixture in favour of `ct_1_ordering_signum.{ailx,ail.json}`). - -## Routing - -| Finding | Class | Action | -|---|---|---| -| BareCrossModuleTypeRef recipe + JSON ctx | working | carry-on | -| BadCrossModuleTypeRef catches Surface | working | carry-on | -| QualifiedClassName fires on `(class prelude.Eq)` | working | carry-on | -| Form-A round-trip byte-stable | working | carry-on | -| BadCrossModuleTypeRef merges two cases | friction | roadmap P2 | -| Workspace search confined to entry dir | friction | roadmap P2 | -| `ail check` on `.ailx` misleading | friction | roadmap P2 | -| `(import prelude)` corpus doc gap | friction | carry-on (one-line DESIGN.md hint sufficient) | -| No `.ailx` for `compare_primitives_smoke.ail.json` | spec_gap | roadmap P3 | - -No bug findings, so no debug dispatch. The five `.ailx` fixtures -stay in the corpus as live exhibits. - -## Lesson — orthogonal UX gaps surface together - -Three of four friction items are orthogonal to canonical-type-names -proper. They predate the milestone. None would have surfaced in a -brainstorm or planner pass because none touches the milestone's -substance. Fieldtest's role IS to surface this kind of cross-cutting -UX debt — DESIGN.md does not mention `.ailx` extension handling, -workspace-root flags, or the prelude doc-gap, but every downstream -LLM author trips on them within ten minutes of writing their first -program. The roadmap entries below are the persistent backlog this -fieldtest produced. diff --git a/docs/journals/2026-05-11-iter-23.4-prep.md b/docs/journals/2026-05-11-iter-23.4-prep.md deleted file mode 100644 index 8e65861..0000000 --- a/docs/journals/2026-05-11-iter-23.4-prep.md +++ /dev/null @@ -1,75 +0,0 @@ -# iter 23.4-prep — checker prerequisites for prelude free fns - -**Date:** 2026-05-11 -**Branch:** iter/23.4-prep -**Status:** DONE -**Tasks completed:** 2 of 2 - -## Summary - -Closes two localised checker gaps that BLOCKED iter 23.4 mid-flight: - -- **Gap 2 (Task 1):** `linearity::check_module`'s globals-construction - loop now iterates `Def::Class` methods and registers each method - type in the globals map. Previously, a borrow-mode fn that - forwarded its borrow-mode params into a class-method call fired - `consume-while-borrowed` false positives, because - `callee_arg_modes` defaulted to `Consume` when it could not find - the method's `param_modes`. `Def::Instance` stays a no-op: - instance method bodies are walked separately via `Def::Fn` after - monomorphisation, per the comment in the new arm. - -- **Gap 1 (Task 2):** the `Term::Var { name }` lookup ladder in - `synth` grows one new branch between the class-method arm and the - dot-qualified arm. It iterates `env.imports.values()` to find a - free-fn match in any implicitly-imported module's flat-globals - table; on hit, it qualifies any owner-module-local Type::Cons via - the existing `qualify_local_types` helper (mirror of the - dot-qualified arm). Today the prelude is the only implicit - import, so single-match-wins is unambiguous; the code comment - names the future revisit point for multi-implicit-import - ambiguity. - -Both fixes are intra-crate and additive — no schema change, no new -diagnostic, no new workspace plumbing. iter 23.4's five prelude -free fns (`ne`/`lt`/`le`/`gt`/`ge`) can now ship against the -existing plan unchanged. - -## Per-task subjects - -- iter 23.4-prep.1: linearity — register Def::Class method types in globals -- iter 23.4-prep.2: check — bare-name fall-through to implicit-imported free fns - -## Concerns - -- No E2E fixture added under `examples/`. The plan's own "Plan - self-review" identifies the two RED-first tests (one in - `linearity.rs`, one in `lib.rs`'s `check_workspace`-level test - module) as the iter-23.4-prep-shaped coverage. The natural E2E - surface (a fixture calling bare `ne 1 2` against the prelude) - cannot land until iter 23.4 ships the prelude free fns - themselves — both `examples/prelude.ail.json` and the consumer - fixture are out of scope for this iter. iter 23.4's E2E coverage - is the right place; flagging here so the next iter does not - inherit "no E2E for the prep iter" as unrecorded debt. - -## Known debt - -- The Task 2 branch's `mod_name` first-match-wins rule remains - ambiguous-by-design as long as `prelude` is the singleton - implicit import. If a second implicit import lands in a future - milestone, this branch needs a duplicate-detection diagnostic - ("name `X` resolves through implicit imports `A` and `B`, - qualify the call"). The code comment names the revisit point. - -## Blocked detail - -N/A — DONE. - -## Commits - -c10acf0a..aef4ab84 - -## Stats - -bench/orchestrator-stats/2026-05-11-iter-23.4-prep.json diff --git a/docs/journals/2026-05-11-iter-23.4.md b/docs/journals/2026-05-11-iter-23.4.md deleted file mode 100644 index 3e87e2f..0000000 --- a/docs/journals/2026-05-11-iter-23.4.md +++ /dev/null @@ -1,68 +0,0 @@ -# iter 23.4 — Mono-Pass Unification - -**Date:** 2026-05-11 -**Started from:** fab1685 -**Status:** DONE -**Tasks completed:** 11 of 11 - -## Summary - -Restructured `crates/ailang-check/src/mono.rs::monomorphise_workspace` -into a single specialiser over every `Type::Forall`-quantified -`Def::Fn` in one fixpoint pass, covering both class-method residuals -and polymorphic free-fn call sites. Removed the codegen-time -specialiser entirely (`lower_polymorphic_call`, `module_polymorphic_fns`, -`mono_queue`, `mono_emitted`, `emit_specialised_fn`, plus the dead -helpers `apply_subst_to_term`, `descriptor_for_subst`, `type_descriptor`). -Codegen now sees only monomorphic `Def::Fn`s — no polymorphic call -sites, no codegen-time queue. - -The unified pass produces bit-identical mono-symbol bodies for the -six existing primitive Eq/Ord symbols (hash-stability pin Task 1, -`crates/ail/tests/mono_hash_stability.rs`) and produces new free-fn -mono symbols on demand for fixtures like `cmp_max_smoke` (Tasks 2, -6, 9). All 73 e2e tests + 18 typeclass tests + 81 codegen tests pass -end-to-end against the new architecture. - -## Per-task notes - -- iter 23.4.1: hash-stability regression pin — `examples/mono_hash_pin_smoke.ail.json` + `mono_hash_stability.rs` with six pinned hashes (`eq__Int`/`Bool`/`Str`, `compare__Int`/`Bool`/`Str`). -- iter 23.4.2: `cmp_max_smoke` fixture + RED workspace-level acceptance gate. -- iter 23.4.3: `MonoTarget` struct → enum with `ClassMethod` / `FreeFn` variants; `mono_target_key` widened with `"class"` / `"free"` first component (guaranteed-disjoint keys). -- iter 23.4.4: free-fn arm in `collect_mono_targets`. NOTE: the plan's Step 4.3 sketch proposed a separate walker over `Term::App` plus a new `env.polymorphic_free_fns` index; instead I extended `synth` itself with a parallel `&mut Vec` channel (mirroring the existing `&mut Vec` mechanism). One less data flow, one less helper; substitution tracking comes "for free" via fresh metavars + post-synth `subst.apply`. Plan's Step 4.4 (adding `polymorphic_free_fns` to `Env`) is therefore obsolete; not landed. -- iter 23.4.5: `synthesise_mono_fn_for_free_fn` + N-ary `mono_symbol_n`. One substantive deviation from the plan's sketch: I also substitute rigid vars in the BODY via a new `crate::substitute_rigids_in_term` helper. The plan said "passes through unchanged" mirroring the class-method arm — but free-fn bodies (e.g. `std_list.length`) carry inner `Term::Lam`s whose `param_tys` reference outer Forall vars; without body substitution, Phase 3's re-synth would fail unification on unbound rigid `a`. -- iter 23.4.6: `rewrite_class_method_calls` renamed to `rewrite_mono_calls`; new helper `poly_free_fn_names_for_module` builds the per-module set of bare + qualified poly-fn names; `collect_residuals_ordered` walks the AST emitting interleaved slots via new `interleave_slots` helper. The walker and slot collector share the same predicate so the cursor invariant survives across the two channels. -- iter 23.4.7: `workspace_has_typeclasses` → `workspace_has_specialisable_targets`. NOTE: the plan's expected "FAIL → PASS after generalisation" was incorrect because the auto-injected prelude (iter 23.1) already keeps the old predicate true for every user workspace. The generalisation is principled-correctness only; documented in the predicate docstring. -- iter 23.4.8: codegen cleanup — removed `lower_polymorphic_call`, both call sites, the Emitter fields (`module_polymorphic_fns`, `mono_queue`, `mono_emitted`), `emit_specialised_fn`, Pass-1 poly population, and the unused dead helpers (`apply_subst_to_term`, `descriptor_for_subst`, `type_descriptor`). `is_static_callee` collapsed to its mono-only path. Two follow-on fixes were required beyond the plan: (a) Unit-default fallback for unpinned forall vars in the mono pass — mirrors pre-iter-23.4 codegen behaviour for sites like `is_empty(Nil)` where the elem type is unobservable from the args alone; (b) updated the `ail emit-ir` command pipeline to include `lift_letrecs` + `monomorphise_workspace` (pre-iter-23.4 the codegen-time poly path was masking this dependency — `emit-ir` was broken for poly fixtures without the explicit mono call now that codegen no longer handles it). -- iter 23.4.9: end-to-end verification — `cmp_max_smoke_runs_end_to_end` proves `cmp_max(3, 7)` at Int prints 7; the three existing poly fixtures (`polymorphic_id_at_int_and_bool`, `polymorphic_apply_with_fn_param`, `poly_rec_capture_demo`) still run. -- iter 23.4.10: `docs/DESIGN.md` §"Monomorphisation (post-typecheck, pre-codegen)" amended to describe the two-arm fixpoint, the dedup key with disjoint `"class"` / `"free"` prefixes, the cursor-aligned walker, and the removed codegen-time path. -- iter 23.4.11: bench-regression check — `bench/check.py`, `bench/compile_check.py`, `bench/cross_lang.py` all exit 0; no metrics regressed. - -## Concerns - -- Plan Step 4.4 was bypassed (no `env.polymorphic_free_fns` field added). The synth-channel approach is structurally cleaner; the plan's index is unneeded. If a future iter needs an O(1) "is this name a poly free fn" predicate at non-synth call sites, the helper `poly_free_fn_names_for_module` (in mono.rs) builds the set on demand. -- Plan Steps 5.3 / 7's "body unchanged" / "early-out fires for class-free workspaces" assumptions were both wrong against the actual codebase shape (free-fn bodies carry rigid vars in inner Lams; prelude is auto-injected so no workspace is class-free). Both were caught and adapted during implementation — no spec defect, just plan-draft drift against the live codebase. - -## Known debt - -- None tracked; all out-of-scope items from the plan (five prelude free fns ne/lt/le/gt/ge, `examples/prelude.ail.json` edits, the negative E2E fixtures `eq_ord_polymorphic` / `eq_ord_user_adt`, DESIGN.md §"Decision 11" amendment, Float-NoInstance diagnostic wording, roadmap update) remain owned by iter 23.5 as planned. - -## Files touched - -- `crates/ailang-check/src/lib.rs` — new `FreeFnCall` struct; `synth` signature gains `&mut Vec`; Var arm tracks `(owner_module, unqualified_name)` and pushes `FreeFnCall` for `Type::Forall`-resolved non-local refs; new `substitute_rigids_in_term` helper. -- `crates/ailang-check/src/mono.rs` — `MonoTarget` struct → enum (`ClassMethod` / `FreeFn`); `mono_target_key` widened; new `mono_symbol_n` (N-ary); new `synthesise_mono_fn_for_free_fn`; new `poly_free_fn_names_for_module`; new `interleave_slots`; rename `rewrite_class_method_calls` → `rewrite_mono_calls`; `workspace_has_typeclasses` → `workspace_has_specialisable_targets`; fixpoint loop matches on target variants. -- `crates/ailang-check/src/lift.rs`, `crates/ailang-check/src/builtins.rs` — synth call sites pass new `&mut free_fn_calls` Vec. -- `crates/ailang-codegen/src/lib.rs` — deletions per Task 8 (Emitter fields, `lower_polymorphic_call`, `emit_specialised_fn`, mono drain loop, Pass-1 poly population, `is_static_callee` poly arm). `use` import shrunk. -- `crates/ailang-codegen/src/subst.rs` — `apply_subst_to_term` and `descriptor_for_subst` removed (dead post-Task-8). -- `crates/ailang-codegen/src/synth.rs` — `type_descriptor` removed (dead post-Task-8). -- `crates/ail/src/main.rs` — `emit-ir` command gains `lift_letrecs` + `monomorphise_workspace` pre-passes (matches `build` shape). -- `crates/ail/tests/typeclass_22b3.rs` — updated three test sites to `MonoTarget::ClassMethod` enum form. -- `crates/ail/tests/mono_hash_stability.rs` (new) — regression pin for six primitive Eq/Ord mono-symbol body hashes. -- `crates/ail/tests/mono_unification.rs` (new) — six tests covering variant key disjointness, free-fn target emission, free-fn synthesis with rigid substitution, rewrite walker on free fns, identity for poly-free-fn-only workspaces, and end-to-end `cmp_max_smoke` runtime correctness. -- `examples/cmp_max_smoke.ail.json` (new) — `cmp_max : forall a. Ord a => (a, a) -> a` composition fixture. -- `examples/mono_hash_pin_smoke.ail.json` (new) — fixture exercising the six primitive Eq/Ord mono symbols for hash-stability pinning. -- `docs/DESIGN.md` — §"Monomorphisation (post-typecheck, pre-codegen)" amended. - -## Stats - -bench/orchestrator-stats/2026-05-11-iter-23.4.json diff --git a/docs/journals/2026-05-11-iter-cadence.md b/docs/journals/2026-05-11-iter-cadence.md deleted file mode 100644 index ceb7a51..0000000 --- a/docs/journals/2026-05-11-iter-cadence.md +++ /dev/null @@ -1,104 +0,0 @@ -# iter cadence — audit/fieldtest/docwriter cadence restructure - -**Date:** 2026-05-11 -**Branch:** main -**Status:** DONE -**Tasks completed:** 5 of 5 - -## Summary - -This iter sorts the milestone-close stretch of the pipeline into -three cadence buckets, packaged by stability-window rather than by -milestone-grain: - -1. **Per-milestone-mandatory — `audit`.** Runs every milestone close. - Architect drift review against DESIGN.md plus the three regression - scripts (`bench/check.py`, `bench/compile_check.py`, - `bench/cross_lang.py`). Step 3 ("Optional rustdoc audit") and the - `ailang-docwriter` cross-reference are removed; the rustdoc - warning line is gone from the Handoff Contract and the Common - Rationalisations table. Audit is now purely drift + bench. -2. **Boss-judgment post-audit — `fieldtest`.** Fires only when the - orchestrator decides the iteration is correct and wants a field - test, after audit closes clean (or with `ratify`-d drift only). - Trigger is reframed from "audit closes" to "Boss judgment call"; - the expected finding shape (simple bugs route via `debug`, - catastrophic architecture problems route via next `brainstorm`) - is named explicitly. Cross-references add the ordering rule: - fieldtest runs *before* docwriter. -3. **Boss-judgment post-fieldtest — `docwriter`.** New skill at - `skills/docwriter/SKILL.md`. Boss-dispatched only, never - triggered by audit closing. Pre-condition is clean audit plus any - pending fieldtest. The agent (`ailang-docwriter`) carries the - substantive rustdoc rules; the SKILL.md only governs trigger and - dispatch. Symlinks added at `.claude/skills/docwriter` and - `.claude/agents/docwriter`. - -`skills/README.md` is the one place that visibly mirrors the new -three-bucket taxonomy: the skill table grew from six rows to seven, -the pipeline diagram extends past `audit --(clean)-->` into -fieldtest and docwriter as two distinct Boss-decision points, and -the agent roster moves `ailang-docwriter` into its own dispatch -group. - -## Per-task subjects - -- iter cadence.1: skills/docwriter — new Boss-dispatched skill (split from audit) -- iter cadence.2: skills/audit — drop docwriter step (now own skill) -- iter cadence.3: skills/fieldtest — Boss-dispatched framing + ordering note -- iter cadence.3-fix: address quality-review minors (frontmatter desc + cross-ref path) -- iter cadence.4: skills/README — three-bucket cadence taxonomy -- iter cadence.4-fix: drop stale audit-dispatches-docwriter parenthetical (README Conventions) -- iter cadence.4-fix: address quality-review importants + German-loanword sweep (English-only in-tree) -- iter cadence.5: per-iter journal + INDEX - -## Lesson - -**Packaging cadence by stability-window outperforms packaging by -milestone-grain.** The previous arrangement bundled rustdoc cleanup -into `audit` as an "optional Step 3", which papered over the real -mismatch: rustdoc rot is a *post-stability* concern (documenting -something that gets rewritten three iterations later is waste), -while drift review and bench regression are per-iteration concerns -(letting them slip even one milestone compounds). Bundling those -two cadences inside one skill made the optional step easy to skip -on every milestone, which is exactly the failure mode the skill -discipline exists to prevent. Splitting docwriter out into its own -Boss-dispatched skill makes the cadence honest: audit is -non-negotiable per milestone, docwriter is non-negotiable per -*stability window*. The fieldtest framing flip — from "fires when -audit closes" to "Boss judgment after audit closes" — is the same -refactor at a smaller scale: the milestone clock was not the right -trigger; the orchestrator's judgement that the iteration is -*complete* is. - -## Concerns - -None worth carrying forward. Three quality-review-fix follow-ups on -cadence.3 and cadence.4 (frontmatter description drift, stale -audit-dispatches-docwriter parenthetical, one German-loanword in -in-tree text) — all caught by the standard implement-skill -two-stage review and resolved inline. - -## Known debt - -- The agent file `skills/docwriter/agents/ailang-docwriter.md` was - moved from `skills/audit/agents/` by the user before this iter - began. Its standing reading list still names `skills/audit` as - its home in one or two phrasings; a sweep of the agent's - cross-references is a candidate tidy iteration but did not - block this iter's narrower SKILL.md reshuffle. -- `docs/specs/2026-05-11-audit-fieldtest-docwriter-cadence.md` is a - dated artefact — the spec describes the design at time of - writing and is not retroactively updated as the implementation - shipped. Per the or.1 convention, the per-iter journal is the - running annotation. - -## Blocked detail - -N/A — DONE. - -## Commits - -`e351969^..HEAD` (8 commits including 3 quality-review-fix -follow-ups across cadence.3 and cadence.4). diff --git a/docs/journals/2026-05-11-iter-disc.1.md b/docs/journals/2026-05-11-iter-disc.1.md deleted file mode 100644 index 349856b..0000000 --- a/docs/journals/2026-05-11-iter-disc.1.md +++ /dev/null @@ -1,190 +0,0 @@ -# iter disc.1 — boss-only commits and main-as-quarantine - -**Date:** 2026-05-11 -**Started from:** 841d65d -**Status:** DONE -**Tasks completed:** 10 of 10 (Boss-direct implementation; no orchestrator dispatch) - -## Summary - -Skill-system-wide discipline update. Two project-wide rules are now -explicit: - -1. **Only the Boss commits.** No skill agent — implementer, - brainstormer, planner, debugger, fieldtester, docwriter, - architect, bencher — performs `git commit`. Agents write all - their artefacts into the working tree as unstaged changes; the - Boss inspects, decides commit shape (typically one cohesive iter - commit, occasionally a few logical commits), and commits. -2. **main HEAD is sacrosanct.** No actor — Boss or agent — runs - `git reset` or `git revert` on main. Bad work stays in the - working tree where it is still discardable via - `git checkout -- `; main moves forward only via Boss - commits. - -The two rules together force a working-tree-as-quarantine -discipline: nothing half-baked enters main, because nothing can -be taken back off main. The implement skill loses its -`iter/` branch-per-iter mechanic entirely; per-task agent -commits are also gone — Phase 4 of the orchestrator-agent now -writes the per-iter journal and the stats file into the working -tree (Write tool) but does not commit them. - -## Per-task notes - -- disc.1.1: `skills/implement/SKILL.md` — Iron Law rewritten - ("IMPLEMENTER NEVER COMMITS" replaces "ONE BRANCH PER ITER"; - "MAIN HEAD IS SACROSANCT" added). Step 3 "Boss merge step" - renamed and rewritten as "Boss inspect + commit step" (no - fast-forward; Boss reads `git status` / `git diff`, decides - commit shape, commits). Step 4 (BLOCKED) rewritten: Working-Tree - inspect, Repair re-dispatch or Discard via - `git checkout -- .`, no branch deletion. Description, Overview, - Rationalisations, Red Flags updated. -- disc.1.2: `skills/implement/agents/ailang-implement-orchestrator.md` - rewritten — Phase 0 is now a clean-tree check - (`git status --porcelain` must be empty), records - `start_sha` informationally (not as reset target), refuses to - proceed on a dirty tree. All `git commit` invocations removed. - Phase 4 (journal) and Phase 5 (stats) write to the working tree - via the Write tool, no commit. End-report Branch-field replaced - by `Working tree: dirty (N files)` and `(uncommitted)` markers - on file paths. -- disc.1.3: `skills/implement/agents/ailang-implementer.md`, - `ailang-tester.md` — commit steps in The Process removed. - Red-Flag "commit on red tests" replaced by the broader "never - commits, ever". Self-review now operates on `git diff HEAD` - with `git checkout -- ` for undo. -- disc.1.4: `skills/implement/agents/ailang-spec-reviewer.md`, - `ailang-quality-reviewer.md` — Carrier fields `pre_task_sha` / - `head_sha` replaced by `diff_command` (default `git diff HEAD`). - The full working-tree diff accumulates across tasks; reviewers - focus on the current task's footprint via the task block's file - list (or an optional `task_footprint_hint`). -- disc.1.5: `skills/brainstorm/SKILL.md` — Step 8 spec-commit - block removed; brainstorm writes the spec, user reviews, Boss - commits. The Step-7.5 retire-on-BLOCK procedure was rewritten - too: it now deletes the working-tree spec file (Bash `rm`, not - `git rm`, since the file was never committed) and appends a - roadmap entry; Boss commits the roadmap edit. -- disc.1.6: `skills/planner/SKILL.md` — Step 6 plan-commit - block removed; planner writes the plan, hand-off carries the - filepath, Boss commits. Plan-template tasks lose their - per-task "Step 5: Commit" — tasks are now 4 steps (write-failing- - test, verify-fail, write-impl, verify-pass) and leave work in - the working tree for the iter-level Boss commit. The "Bundle - Tasks 3 and 4" rationalisation reframed: tasks are units of - review, not units of commit. -- disc.1.7: `skills/debug/SKILL.md`, `skills/debug/agents/ailang-debugger.md` - — debugger writes the RED test to the working tree (uncommitted) - and hands off. The Boss chooses between two flows for - implement-mini: commit the RED separately for an audit-trail - before dispatching mini-mode (clean-tree mini gets the RED via - HEAD diff), or commit RED+GREEN together at the end (Boss - decides before dispatching). -- disc.1.8: `skills/fieldtest/SKILL.md`, - `skills/fieldtest/agents/ailang-fieldtester.md` — Phase 5 - commit block removed; fieldtester writes `.ailx` files, - `.ail.json` files, and the spec to the working tree; Boss - commits with the suggested subject - `fieldtest: — examples, findings`. -- disc.1.9: `skills/docwriter/agents/ailang-docwriter.md` — - Iron Law extended with "YOU NEVER COMMIT"; Red-Flag added. - `skills/audit/SKILL.md` — Step 4 reframed to name the Boss as - the actor who commits tidy fixes / ratify baselines / clean - closes. Other audit-agent files (architect, bencher) had no - commit invocations and are unchanged in this dispatch. -- disc.1.10: `skills/README.md` — Conventions section gained - two new bullets ("Only the Boss commits", "main HEAD is - sacrosanct"). Skill-table Output column updated for implement, - docwriter, fieldtest, debug to say "uncommitted in the working - tree". `CLAUDE.md` Direction-freedom subsection rewritten - (recovery is no longer about reverting commits; it is about - discarding bad working-tree state). New - "Commit discipline and main-branch sanctity" subsection added, - stating both rules in CLAUDE.md form. - -## Concerns - -(empty) - -## Known debt - -- Three historical files describe the obsolete branch-per-iter - discipline: `docs/specs/2026-05-11-implement-orchestrator-agent.md`, - `docs/journals/2026-05-11-iter-pr.1.md`, - `docs/plans/2026-05-11-orchestrator-refactor.md`. They are - zeitliche records and are left intact; this journal entry is the - forward-pointer that supersedes them. A future tidy iter could - add "Superseded by" banners, but the journal-archive convention is - append-only history. -- The implement orchestrator's Phase 0 refuses to run on a dirty - tree. The first real dispatch under the new discipline may - surface friction around this (e.g. Boss prepared scratch files - before dispatch). Capture in that iter's journal. - -## Blocked detail - -(none — DONE) - -## Files touched - -- `CLAUDE.md` -- `skills/README.md` -- `skills/audit/SKILL.md` -- `skills/brainstorm/SKILL.md` -- `skills/debug/SKILL.md` -- `skills/debug/agents/ailang-debugger.md` -- `skills/docwriter/agents/ailang-docwriter.md` -- `skills/fieldtest/SKILL.md` -- `skills/fieldtest/agents/ailang-fieldtester.md` -- `skills/implement/SKILL.md` -- `skills/implement/agents/ailang-implement-orchestrator.md` -- `skills/implement/agents/ailang-implementer.md` -- `skills/implement/agents/ailang-quality-reviewer.md` -- `skills/implement/agents/ailang-spec-reviewer.md` -- `skills/implement/agents/ailang-tester.md` -- `skills/planner/SKILL.md` -- `docs/journals/2026-05-11-iter-disc.1.md` (this file) -- `docs/journals/INDEX.md` - -## Motivation - -On 2026-05-11 iter 23.4 (eq/ord prelude — Free-Fns) BLOCKED three -times. A retrospective brainstorm produced a corrected spec; the -implement-orchestrator was dispatched on the revised plan and -immediately BLOCKED at Phase 0 with a branch-state mismatch: an -existing `iter/23.4` branch carried prep2 + prep3 commits that -the spec falsely claimed lived on main. They had never been -fast-forwarded; they were stranded on the iter-branch when the -iter was abandoned. The grounding-check agent had passed the -spec because it verified main-behaviour-via-green-tests, not -git-reachability of named commit SHAs. - -The user identified the structural root cause: branch-per-iter -plus manual Boss-merge plus iter-stacking allows work to strand -on a branch that nobody integrates. The discipline correction is -this iter: - -- Drop the branch mechanic in implement entirely. -- Centralise commit authority on the Boss across every skill. -- Make main forward-only by policy: no reset, no revert, by - anyone. - -Per-task agent commits were eh too many (user phrasing: "es wird -eh viel zu viel commitet") — the new policy eliminates them as -a side-effect. - -## Forward pointer - -Step 5 of the 2026-05-11 recovery plan: re-attempt the eq/ord -prelude mono-pass unification (iter 23.4 in the corrected spec -`docs/specs/2026-05-11-23-eq-ord-prelude.md`), now under the -discipline this iter installs. A fresh plan is written first; -the implement-orchestrator is dispatched against it; the Boss -inspects + commits at the end. No branches, no agent commits. - -## Stats - -None — this iter was Boss-direct implementation (no orchestrator -dispatch; no stats file written). diff --git a/docs/journals/2026-05-11-iter-gc.1.md b/docs/journals/2026-05-11-iter-gc.1.md deleted file mode 100644 index 9396c24..0000000 --- a/docs/journals/2026-05-11-iter-gc.1.md +++ /dev/null @@ -1,126 +0,0 @@ -# iter gc.1 — Grounding-Check Agent, brainstorm Step 7.5 hard-gate - -**Date:** 2026-05-11 -**Branch:** iter/gc.1 -**Status:** DONE -**Tasks completed:** 5 of 5 - -## Summary - -This iter ships `ailang-grounding-check`, a read-only spec reviewer -dispatched between brainstorm Step 7 (self-review) and Step 8 -(user-approval). The agent reads a freshly-written spec with no -sunk-cost bias, extracts the spec's load-bearing assumptions about -current compiler / checker / codegen / schema behaviour, and for -each one searches the workspace for a currently-green test that -ratifies it. PASS or BLOCK; no PARTIAL credit. The output is the -hybrid format the agent template prescribes: a ratified-assumptions -table for fast Boss scanning plus a per-assumption prose block for -the unratified set. - -Wiring is markdown-only. `skills/brainstorm/SKILL.md` gets a new -Step 7.5 hard-gate section, two new Handoff-Contract rows -(brainstorm → grounding-check, brainstorm → roadmap on no-override -BLOCK), and a Cross-references rewrite that splits the old "No -private agents" bullet into "Private agent" (the new grounding-check -dispatch) plus "Ad-hoc dispatch" (the existing plan-recon -opt-in). `skills/README.md` gets one new agent-roster row and one -new symlink-discovery line. `.claude/agents/brainstorm` is the -matching symlink. Spec-23 (Eq/Ord prelude) is retired because the -session that produced gc.1's spec is the same session that -discovered Spec-23's polymorphic-free-fn-monomorphisation assumption -was unratified — a fact gc.1's whole machinery exists to catch from -now on. The roadmap P1 Post-22 Prelude entry is amended to record -the retirement, name the unratified mechanism, and point forward to -the re-brainstorm. - -The bootstrap irony stands: the spec for `ailang-grounding-check` -did not itself run through `ailang-grounding-check`. The agent did -not exist when its own spec was written, so per Spec §Architecture's -bootstrap-exclusion the gc.1 plan was implemented without the -gating mechanism it introduces. The first real run of the new -pipeline is the re-brainstormed Spec-23 in a separate session, not -this iter. - -## Per-task subjects - -- iter gc.1.1: agent — ailang-grounding-check definition -- iter gc.1.2: brainstorm SKILL — insert Step 7.5 (grounding-check hard-gate) -- iter gc.1.3: skills README — roster + symlink for ailang-grounding-check -- iter gc.1.4: symlink .claude/agents/brainstorm -> skills/brainstorm/agents -- iter gc.1.5: retire Spec-23, roadmap forward-pointer to re-brainstorm - -## Concerns - -- Plan §Step 2.4 and §F.2 expect `grep -c "^### Step " == 9` after - the 7.5 splice, but the brainstorm SKILL has nine *original* step - headings (Step 1 through Step 9), so adding 7.5 lands at 10. The - plan's parenthetical comment in §F.2 enumerates ten labels (1, 2, - 3, 4, 5, 6, 7, 7.5, 8, 9) while asserting `wc -l = 9` — internally - inconsistent. Substantive intent (Step 7.5 between 7 and 8, leave - Step 9 in place) is unambiguous and honored; the off-by-one is a - plan-template miscount, not an implementation bug. Surfaced as a - cadence-flow signal to the Boss in the end-report. - -## Cadence-flow notes (for skillsystem postmortem) - -Two observations from the orchestrator's end-report, surfaced here -for the Boss's future skillsystem-postmortem entry — NOT in scope -for gc.1 itself: - -1. **Planner verify-step counts derived from offsets become stale - at plan-freeze time.** The §Step 2.4 and §F.2 miscount above is - one instance. The orchestrator suggested two fixes: (a) planner - re-runs grep counts against the current file state at - plan-freeze time, or (b) planner expresses such counts as - `≥ N` inequalities rather than exact-match equalities. The - latter is cheaper and survives subsequent edits to the same - file. - -2. **Markdown-only verbatim-content iters bottom out cleanly in the - spec-check + quality-check phases.** Zero review-loop fixups - landed this iter; all five tasks went DONE on first pass through - both phases. The spec-check phase effectively reduced to "is - the verbatim text from the plan in the diff?" and the - quality-check phase had little to flag beyond whitespace - hygiene. Both passes still ran (discipline preserved), but the - value they added was low. This is expected for verbatim-content - tasks and not a concern about the discipline itself — captured - here so it is not misread, in retrospect, as evidence that the - phases are too lenient. - -3. **First real run of grounding-check (2026-05-11 same-day, on - the revised Spec-23): PASS, 15 assumptions ratified.** The - agent's output format leaked an internal-thinking preamble - before the structured `GROUNDING-CHECK REPORT` block — the - Iron Law should be tightened to forbid any preamble before the - structured report. Substantively unproblematic (the judgement - and test pins were clear) but a real prompt-discipline gap. - -4. **Behaviour change requested for impending-BLOCK paths.** On a - miss (assumption with no green test), the agent should attempt - to write the missing GREEN test itself — strictly test code, - never `src/`-side app code — and only BLOCK if test creation - fails. This is two changes: (a) tools — add `Write` (or `Edit`) - to the agent frontmatter, today `Read, Glob, Grep, Bash` only; - (b) discipline — Iron Law gains "you may write tests under - `crates/*/tests/` or `#[cfg(test)] mod tests`; you may NOT touch - any file outside test scope." The first real PASS run did not - exercise this path, so the constraint surfaces here as a future - tightening rather than as a tested behaviour. - -## Known debt - -None. - -## Blocked detail - -N/A — DONE. - -## Commits - -`606a2eb..HEAD` (5 task commits, no review-loop fixups). - -## Stats - -bench/orchestrator-stats/2026-05-11-iter-gc.1.json diff --git a/docs/journals/2026-05-11-iter-or.1.md b/docs/journals/2026-05-11-iter-or.1.md deleted file mode 100644 index eedc572..0000000 --- a/docs/journals/2026-05-11-iter-or.1.md +++ /dev/null @@ -1,151 +0,0 @@ -# iter or.1 — orchestrator-refactor - -**Date:** 2026-05-11 -**Branch:** iter/or.1 -**Status:** DONE -**Tasks completed:** 7 of 7 - -## Summary - -This iter lands the three coupled changes from -`docs/specs/2026-05-11-implement-orchestrator-agent.md`: - -1. **Boss-context offload.** A new agent - `skills/implement/agents/ailang-implement-orchestrator.md` owns - one full `/implement` iter end-to-end — per-task implementer → - spec-reviewer → quality-reviewer loop, with the canonical - re-loop limits — running in its own context. The Boss now - dispatches one subagent and receives a ≤500-token end-report. -2. **Iter isolation via branches.** Each `/implement` run owns a - dedicated `iter/` branch, created from `origin/main` - in Phase 0 of the orchestrator-agent. Integration is a Boss-side - `git switch main && git merge --ff-only` after reading the - end-report. -3. **Per-iter journals.** `docs/JOURNAL.md` (14k lines) is renamed - to `docs/journal-archive.md` with an archived-status header. - Going forward, each iter writes its own file under - `docs/journals/-iter-.md`; the Boss appends a - one-line pointer to `docs/journals/INDEX.md` at merge time. - This file is the first such per-iter journal. - -The orchestrator-agent goes live on the *next* `/implement` run after -this iter lands. This iter itself was executed in the pre-refactor -mode (Boss dispatched workers directly) — a bootstrap necessity since -the new agent did not yet exist while creating it. - -Model selection was unchanged: every dispatch this iter used Opus 4.7. -The token savings come from token-volume relocation, not from -intelligence downgrade. - -## Per-task subjects - -- iter or.1.1: migrate JOURNAL.md to journal-archive.md + docs/journals/ -- iter or.1.2: repoint top-level docs to docs/journals/ + journal-archive.md -- iter or.1.2-fix: address quality-review minor + nit -- iter or.1.3: repoint skill + agent reading lists to docs/journals/ -- iter or.1.3-fix: address quality-review minors (architect / audit phrasing) -- iter or.1.4: carrier switch task_text -> task_text_path -- iter or.1.5: new agent ailang-implement-orchestrator -- iter or.1.6: rewrite skills/implement/SKILL.md to delegate to orchestrator-agent -- iter or.1.6-fix: correct git rebase syntax + README cross-ref wording -- iter or.1.6-fix2: SKILL.md description drift + self-contained exception ref -- iter or.1.7: document ailang-implement-orchestrator as named exception - -## Concerns - -Three `DONE_WITH_CONCERNS` observations across the iter, all -plan-template issues rather than scope deviations: - -- Task 1: the plan's Step 6 listed `git add docs/JOURNAL.md` after - a `git mv`; this errors because the path is already gone from the - working tree. The implementer dropped the dead path. Plan defect. -- Task 4: the plan's per-step instructions enumerated only the - carrier-contract row, but the plan's Step 6 verification clause - (`grep -n "task_text\b" → empty`) required visiting every body-prose - mention of `task_text` too. The implementer correctly expanded - scope to satisfy the verification; rewrites preserved semantics. -- Task 6: the plan's Step 1 verbatim content is 194 lines; Step 2's - sanity check expected 110–160. Step 1 is canonical; the sanity - range was my optimistic estimate when writing the plan. The - implementer correctly followed Step 1. -- Task 7: the plan's Step 3 grep expected 1 match for - `ailang-implement-orchestrator`, but the new exception paragraph - itself names the agent, so the actual count is 2. Same shape of - plan-template error. - -None of these affect correctness; all are notes for the next plan -template revision. - -## Known debt - -- Two bare `JOURNAL` uppercase mentions remain in `CLAUDE.md` at - lines 285/287 (the "answer three questions" closing paragraph). - They refer conceptually to the historical log, but the surface - reading is slightly stale now that JOURNAL is no longer a file. - Out of scope for Task 2's enumerated sites; carry-on observation. -- "Boss" / "Boss-Orchestrator" vocabulary is introduced in the new - `skills/implement/SKILL.md` and `ailang-implement-orchestrator.md` - to distinguish from "the orchestrator-agent". `CLAUDE.md` uses - "orchestrator" throughout. The distinction is intentional, but a - one-line glossary entry in `CLAUDE.md` would prevent term drift - in future docs. -- Verification ladder steps 1–5 (real-orchestrator-agent dispatch - exercise) are deferred to the first real `/implement` run after - this iter merges. Steps 6–8 (intentional-BLOCKED, intentional - re-loop, manual Boss-merge replay) get checked off when those - failure modes arise organically. - -## Blocked detail - -N/A — DONE. - -## Commits - -`1891030..92e39bb` (11 commits including 4 quality-review-fix -follow-ups; the per-task pattern was DONE/DONE-WITH-CONCERNS → -spec-reviewer compliant → quality-reviewer approved-or-fixed-then- -approved, per the implement skill's two-stage review). - -## Stats - -Not produced this iter — `bench/orchestrator-stats/` is a new -artefact introduced by the orchestrator-agent (Phase 5 of its -process). This iter was executed in pre-refactor mode by the Boss -directly, so no stats file exists. The first stats file will be -written by the orchestrator-agent on the next real `/implement` run. - -## Audit / milestone close - -Audit ran post-iter (skill-system mandatory at milestone close). - -- **Architect drift review:** `drift_found`. Live docs and a handful - of concept-noun sites still mentioned `JOURNAL.md` as if it were a - current path. Highest-impact misses were `docs/WhatsNew.md` (live - changelog framing-text — wrongly carved out of the Task 3 plan as - "dated artifact"), `docs/roadmap.md` lines 25-27 (live convention - text), and four bare `JOURNAL` mentions in `CLAUDE.md` (lines 108, - 267, 285, 287). Five concept-noun spots in - `skills/audit/agents/ailang-architect.md` (Iron Law line 100 was - already correct; the others were process descriptions). All - resolved inline via `iter or.1.tidy` (single commit, four files). - Spec-vs-shipped divergences (the spec still prescribes the invalid - `git rebase ... onto main` and lists 3 blocked reasons where the - agent ships 5) were intentionally left as-is — specs are dated - artefacts capturing design at time of writing; the per-iter journal - is the running annotation. -- **Bench-regression check:** `bench/check.py` 63 metrics stable - (exit 0); `bench/compile_check.py` 24 metrics stable (exit 0); - `bench/cross_lang.py` 25 metrics stable (exit 0). No language code - changed; the green result was expected and confirms it. -- **Rustdoc:** 18 pre-existing warnings (2 in `ailang-core`, 16 in - `ailang-check`) about private-item links and unresolved cross-refs. - None caused by this iter. Deferred to a future docwriter sweep. - -Milestone close: clean after `or.1.tidy`. Three roadmap follow-ups -worth noting: -- WhatsNew.md should NOT be carved out as "dated artifact" in any - future doc-sweep plan; it is a live changelog. -- A separate docwriter sweep for the 18 pre-existing rustdoc warnings. -- Once a real `/implement` run goes through the new orchestrator-agent, - ladders 1–5 of the verification ladder in `docs/specs/ - 2026-05-11-implement-orchestrator-agent.md` are checked off. diff --git a/docs/journals/2026-05-11-iter-or.2.md b/docs/journals/2026-05-11-iter-or.2.md deleted file mode 100644 index 2fce834..0000000 --- a/docs/journals/2026-05-11-iter-or.2.md +++ /dev/null @@ -1,189 +0,0 @@ -# iter or.2 — orchestrator-agent design correction - -**Date:** 2026-05-11 -**Branch:** none (trivial-mechanical carve-out — doc-only revision, single commit on main) -**Status:** DONE -**Tasks completed:** N/A (doc revision, not a plan-driven iter) - -## Summary - -This entry corrects a factual error in the `or.1` architecture that -surfaced during the first real dispatch (pr.1, same day). The -revision lands as one direct-on-main commit because the change is -doc-only: no code, no tests, no behaviour shift in any compiler -crate. The Iron-Law mini-fix from earlier today (`origin/main` → -`main`) is preserved unchanged — that fix remains correct. - -### What was wrong - -`or.1` and `pr.1` shipped the `ailang-implement-orchestrator` agent -with frontmatter `tools: Read, Edit, Write, Bash, Glob, Grep, Agent` -and an architecture that depended on the orchestrator-agent -spawning further subagents per task (implementer → spec-reviewer → -quality-reviewer → tester, each in fresh context). The architecture -was framed in `skills/README.md` as "agents do not call other -agents — one named exception" (the orchestrator-agent being the -exception). - -The exception never existed in Claude Code. The platform forbids -**all** nested-subagent dispatch, categorically: - -> "Subagents cannot spawn other subagents, so `Agent(agent_type)` -> has no effect in subagent definitions." -> — https://code.claude.com/docs/en/sub-agents - -> "Subagents cannot spawn their own subagents. Don't include -> `Agent` in a subagent's `tools` array." -> — https://code.claude.com/docs/en/agent-sdk/subagents - -The `Agent` tool was silently dropped from the orchestrator-agent's -tool set at dispatch time (no error, no warning). The pr.1 run -caught this empirically; the diagnosis closed the question today. - -### Why the failure took the form it did - -The Boss-Orchestrator (me) designed `or.1` on the false premise -that the project's house rule "agents do not call other agents" -was a local convention with a documented exception. In reality the -house rule was a *restatement of an external constraint* — and the -"named exception" was incompatible with the constraint itself. -This is a verification-of-capability failure: the architecture was -locked in before checking the platform docs for what subagents can -do. The right order is the other way around. (Memorised as a -feedback memory for future architecture decisions.) - -### The corrected architecture - -Same skill body, same skill triggers, same branch-and-merge -discipline, same Boss-side end-report contract. What changes is -*inside* the orchestrator-agent's run: - -- Per-task phases (implementer → spec-compliance check → quality - check) run sequentially as **role-switches in the orchestrator- - agent's own context**, not as nested subagents. -- The `ailang-implementer.md` / `ailang-spec-reviewer.md` / - `ailang-quality-reviewer.md` / `ailang-tester.md` files now serve - as **phase reference files** — the orchestrator-agent consults - them at each role-switch to inhale the discipline of that role, - but does not dispatch them as separate subagents. -- The Boss-context offload property is preserved (the Boss still - sees only the ≤500-token end-report; per-task chatter stays - inside the orchestrator-agent's context). -- The fresh-per-phase context property — which was the design goal - that drove `or.1` to attempt nested dispatch — is **given up**. - Pure role-switching inside one context is a weaker discipline - than fresh-context-per-phase. The trade-off is named openly: the - cost is the loss of fresh-eyes review per phase; the benefit is - that the architecture is buildable in Claude Code at all. -- The pr.1 "degraded mode" was not actually degraded — it was the - only mode this agent can run in. The pr.1 journal's degraded- - mode caveat is preserved as-is (historical record of how the - constraint was discovered). - -### Why option (A), not (B) or (C) - -Three options were on the table: - -- **(A)** Single-context inline phases (what landed) — preserves - Boss-context offload, gives up fresh-per-phase context. -- **(B)** Roll back to pre-or.1: Boss dispatches each per-task - subagent itself. Preserves fresh-per-phase context, gives up - Boss-context offload. -- **(C)** Hybrid: orchestrator-agent for setup / branch / journal, - Boss for per-task dispatches in between. Preserves both - theoretically but multiplies moving parts. - -(A) was chosen because the Boss-context offload is the scarce -resource (user chat budget). Empirically, the pr.1 run with the -plan template's verbatim per-task content blocks made fresh-per- -phase context contribute little additional signal beyond a careful -single-context phase loop. The decision was made by the user -2026-05-11 after the diagnosis was presented; the rationale belongs -in the user's hands, not the agent's. - -### Files changed - -- `skills/implement/agents/ailang-implement-orchestrator.md` — - frontmatter (`Agent` removed from tools, description rewritten), - Iron Law (sequential role-switches instead of fresh subagents), - Phase 2.1 / 2.2 / 2.3 rewritten as inline phases, Phase 3 likewise, - common rationalisations and red flags updated. -- `skills/implement/SKILL.md` — frontmatter description, Iron Law, - per-task sub-status mechanics note (vocabulary is preserved but - now describes inline role-phase status), one new - Common-Rationalisation row, Cross-references rewritten (role-files - recast as phase references, with a "Why inline phases" subsection - pointing here). -- `skills/README.md` — Conventions: "Agents do not call other - agents — one named exception" → "Agents do not call other agents" - (the exception is gone, with an explanation of why). Agent roster - rows for implementer / spec-reviewer / quality-reviewer / tester - recast as phase references for the orchestrator-agent. -- `docs/roadmap.md` — P1 entry "`Agent` tool wiring for the - dispatched orchestrator-agent" removed (resolved as - categorically-not-fixable). -- `docs/journals/INDEX.md` — this entry appended. -- `docs/WhatsNew.md` — user-facing correction entry appended. - -### Files NOT changed (and why) - -- `CLAUDE.md` — no stale references; the file does not encode the - named-exception story. -- `skills/implement/agents/ailang-implementer.md`, - `ailang-spec-reviewer.md`, `ailang-quality-reviewer.md`, - `ailang-tester.md` — these remain unchanged. Their content - (discipline, reading list, output format) is correct in the new - architecture; their *role* in the system has shifted from - "dispatched subagent" to "phase reference for the orchestrator- - agent". The shift is documented in the README roster row and the - SKILL.md Cross-references; no in-file edits are required for - correctness. A future cleanup iter may rewrite their frontmatter - to drop the "subagent" framing if it begins to mislead — for now - the cost of touching four files outweighs the marginal clarity - gain. -- `docs/specs/2026-05-11-implement-orchestrator-agent.md` and - `docs/specs/2026-05-11-plan-recon-subagent.md` — specs are - historical records of the design as conceived; they remain - accurate to that moment. Drift between spec and the corrected - architecture is acknowledged here, not by editing the specs. - -## Concerns - -None. The revision is deliberate and self-contained. - -## Known debt - -- **Phase reference files frame themselves as dispatched subagents.** - `ailang-implementer.md` and its three siblings still read like - files describing a subagent receiving a carrier from a dispatcher. - In the corrected architecture, the orchestrator-agent IS the - dispatcher and the worker for those phases. If the next - `/implement` run shows the orchestrator-agent confused by this - framing during phase reads, a follow-up cleanup iter rewrites - the four files. Not pre-emptive — wait for an empirical signal. -- **No verification ladder for the corrected design.** `or.1`'s - verification ladder (items 1–5 in - `docs/specs/2026-05-11-implement-orchestrator-agent.md`) was - written for the nested-dispatch architecture. Items 1–3 - (Boss-side dispatch, branch creation, journal write) carry over - unchanged. Items 4–5 (intentional BLOCKED, intentional re-loop) - are weaker in the inline-phase world — re-loop is a same-context - retry, not a fresh-context re-dispatch — and a new ladder entry - for the inline phase-switch should be added the next time - `/implement` runs against a non-trivial plan. Tracked - informally; no ticket. - -## Blocked detail - -N/A — DONE. - -## Commits - -This entry's revision lands as a single trivial-mechanical commit -on `main` (no iter branch, no plan template). The commit subject is -`or.2: orchestrator-agent design correction — no nested subagent dispatch`. - -## Stats - -N/A — no orchestrator-agent dispatch (this is a doc-only revision -made directly by the Boss-Orchestrator). diff --git a/docs/journals/2026-05-11-iter-pr.1.md b/docs/journals/2026-05-11-iter-pr.1.md deleted file mode 100644 index 896445a..0000000 --- a/docs/journals/2026-05-11-iter-pr.1.md +++ /dev/null @@ -1,171 +0,0 @@ -# iter pr.1 — plan-recon subagent - -**Date:** 2026-05-11 -**Branch:** iter/pr.1 -**Status:** DONE -**Tasks completed:** 5 of 5 - -## Summary - -This iter lands the changes from -`docs/specs/2026-05-11-plan-recon-subagent.md`: a new read-only -subagent `ailang-plan-recon` that the `planner` skill dispatches at -Step 2 (file-structure mapping), moving the read-heavy code-recon out -of Boss context. `brainstorm` gets a one-paragraph note in its -Cross-references allowing ad-hoc dispatch of the same agent when -entering unfamiliar code territory; it does NOT receive a dedicated -agent. Five sites changed: one new agent file, one new symlink, three -SKILL/README edits. No Rust, no benchmark impact. - -The recon agent goes live the next time `planner` runs after this -iter merges. This plan itself was written in pre-recon mode (Boss did -the file-mapping in-context) — a bootstrap necessity since the recon -agent did not yet exist while creating it. - -**Degraded-mode dispatch note (orchestrator-agent).** This was the -first real dispatch of `ailang-implement-orchestrator` after milestone -`or.1` (verification ladder steps 1–3). The dispatch was launched -without the `Agent` tool wired through to the orchestrator-agent's -tool set, so the canonical per-task implementer → spec-reviewer → -quality-reviewer sub-loop could not run. Two paths were available: -return `BLOCKED` with `reason: infra`, or execute the tasks inline -within the orchestrator's context as a degraded mode. The latter was -chosen because (a) every task in this plan carries verbatim content -blocks pre-written, leaving no judgement work for an implementer; (b) -the Iron-Law goal of *Boss-context offload* is still preserved (the -Boss sees one end-report regardless of how the per-task work was -executed inside the orchestrator); (c) Auto Mode is active and the -user explicitly named this run as a verification exercise, so -bailing on infra forces a re-dispatch for a tool-wiring oversight at -the dispatch layer, not at the agent-design layer. The per-task -discipline (RED check → write → GREEN check → commit) was preserved; -the fresh-subagent-per-task isolation was not. This degraded path is -*one-time*, not a precedent: the orchestrator-agent's standing tool -set must include `Agent`, and the next dispatch should restore the -canonical loop. Recorded for the verification ladder. - -## Per-task subjects - -- iter pr.1.1: new agent ailang-plan-recon -- iter pr.1.2: symlink .claude/agents/planner -- iter pr.1.3: planner Step 2 dispatches ailang-plan-recon -- iter pr.1.4: brainstorm allows ad-hoc plan-recon dispatch -- iter pr.1.5: roster + discovery entries for ailang-plan-recon - -## Concerns - -One `DONE_WITH_CONCERNS` observation, a plan-template issue rather -than a scope deviation: - -- Task 4's Step 1 RED-check regex is malformed: - `grep -q "No private agents.\*\*\* This skill is dialogue-driven\."` - expects three literal stars (`\*\*\*`) after the period, but the - file content has only two (markdown bold `**`). The check exits - non-zero against a correct pre-state. The implementer (the - orchestrator-agent in this degraded mode) verified RED by intent — - the line exists, "ad-hoc dispatch" does not — and proceeded. Plan - defect; note for the next plan template revision. - -## Known debt - -- Verification ladder items 1–3 from - `docs/specs/2026-05-11-implement-orchestrator-agent.md` were - exercised in degraded mode (see Summary). Restoring the full - per-task sub-loop is a precondition for ladder items 4–5 - (intentional-BLOCKED, intentional re-loop) to be meaningfully - testable. The Boss should verify `Agent` is in the - orchestrator-agent's tool wiring on the next `/implement` dispatch. -- The plan-recon agent's standing reading list points at - `docs/journals/INDEX.md` and "the latest entries" — once this iter - lands and INDEX is updated, the agent's reading list naturally - picks up the pr.1 entry. - -### Boss-side addendum - -Two follow-ups surfaced during this dispatch that are not pr.1's -own scope but were exposed by being the first real run of the -or.1-orchestrator-agent. Both go to the post-pr.1 fix queue: - -- **or.1 Iron-Law wording is wrong.** The orchestrator-agent's - Phase 0 Iron Law says `git switch -c iter/ origin/main`, - which presupposes that the plan-commit is on `origin/main`. The - Boss does not push between plan-commit and dispatch, so the - branch is created without the plan file. The agent worked around - this by mid-flight ff-merging local `main` into `iter/pr.1`, - which got the right end-state but rationalised itself through a - "branch already exists" clause that was actually meant for repair - re-dispatch. Correct fix: `git switch -c iter/ main` - (drop `origin/`). Iron Law's purpose is iter-isolation from main, - not push-state coupling. Touches - `skills/implement/agents/ailang-implement-orchestrator.md` Phase - 0 and the corresponding mention in `skills/implement/SKILL.md` - if any. One-commit mini-fix iter (no plan template needed, - trivial-mechanical carve-out). -- **`Agent` tool not reaching the orchestrator-agent.** Frontmatter - declares `tools: Read, Edit, Write, Bash, Glob, Grep, Agent`, but - the dispatched orchestrator-agent reports `Agent` was not in its - actual tool set, forcing the degraded-mode execution described - in Summary. Root-cause investigation needed: either a Claude - Code restriction on nested-subagent dispatch, or a frontmatter - parsing detail, or a deferred-tool issue at the harness layer. - This blocks the entire or.1 design goal (Boss-context offload - via fresh-subagent-per-task isolation), so it dominates the - post-pr.1 queue. Not fixable by a doc edit; needs diagnosis - before a fix can be scoped. -- **Plan-template RED-check regex.** Task 4 Step 1 used - `\*\*\*` (three literal stars) where the file content has two - (`**…**` markdown bold). Three plan iterations have now hit - plan-template defects (or.1 Tasks 1/4/6/7, pr.1 Task 4) — - cumulative signal that the plan-template needs a small revision - pass before the next planner run, or the planner skill needs - a self-test step. - -## Audit / milestone close - -Audit ran post-iter (skill-system mandatory at milestone close). - -- **Architect drift review:** `clean`. Spec → code mapping is 1:1 - across all five acceptance criteria; the new agent file is - template-compliant; `docs/DESIGN.md` was not touched; all four - `architect_sweeps.sh` sweeps are clean (the one sweep-1 hit at - `docs/DESIGN.md:50` is a pre-existing benign cross-reference last - modified in `or.1.2`, not pr.1 regrowth). The two follow-up items - in this journal's "Boss-side addendum" (or.1 Iron-Law wording; - `Agent` tool not reaching the orchestrator-agent) are confirmed as - fix-queue items, not DESIGN-level drift. A mild duplication - observation: the plan-template RED-check regex defect appears in - both "Concerns" and "Boss-side addendum" sections — accepted as-is - (different audiences read each section). -- **Bench-regression check:** `bench/check.py` 63 metrics stable - (exit 0); `bench/compile_check.py` 24 metrics stable (exit 0); - `bench/cross_lang.py` 25 metrics stable (exit 0). No language code - changed; the green result was expected and confirms it. -- **Rustdoc:** 20 pre-existing-style warnings (or.1 audit recorded - 18; the +2 are not caused by pr.1, which touched no Rust). All are - private-item links or unresolved cross-refs. Same `[P2] Rustdoc - warning sweep` roadmap entry still applies; the count will be - re-checked when that sweep runs. - -Milestone close: clean. Two post-pr.1 fix items already named in the -Boss-side addendum above: - -1. or.1 Iron-Law `origin/main` → `main` mini-fix (trivial one-commit - iter, no plan template needed; trivial-mechanical carve-out). -2. `Agent` tool wiring for the dispatched orchestrator-agent — needs - diagnosis before fix-scope can be set. Until resolved, the or.1 - design goal (per-task fresh-subagent isolation) is degraded; the - Boss-context-offload aspect still works because the per-task - detail collapses into one end-report regardless. - -## Blocked detail - -N/A — DONE. - -## Commits - -`14ac4ae..f06911d` (5 commits, one per task, no fix-passes). - -## Stats - -`bench/orchestrator-stats/2026-05-11-iter-pr.1.json` (created by -Phase 5 of this run; first stats file in the directory). diff --git a/docs/journals/2026-05-12-audit-23.md b/docs/journals/2026-05-12-audit-23.md deleted file mode 100644 index ebc13df..0000000 --- a/docs/journals/2026-05-12-audit-23.md +++ /dev/null @@ -1,134 +0,0 @@ -# audit-23 — Milestone 23 close - -**Date:** 2026-05-12 -**Started from:** 3fed372 -**Status:** DONE — milestone 23 closed (1 ratify + 1 nit-fix) -**Skill:** audit (architect + bencher) - -## Summary - -Milestone-close audit for Milestone 23 (Eq/Ord prelude). Architect -clean on substance — all 8 acceptance criteria of -`docs/specs/2026-05-11-23-eq-ord-prelude.md` met, no out-of-scope -creep, DESIGN.md two-arm-fixpoint description (line 1513-1556) -matches `mono.rs::collect_mono_targets` observable behaviour, roadmap -split internally consistent. Bencher found and ratified one -structural compile-time shift; the iter-23.5 journal's "one sub-ms -noise-band metric" framing was wrong on both counts (4 metrics, not -1; consistent +18-20% cohort shift, not noise). - -## Architect — drift items - -- **[important]** `docs/DESIGN.md:1692-1693` — sentence collision: - "Milestone 22 ships **no built-in Prelude classes**. The original - An earlier draft committed to …". Pre-existing since commit - `613d4d8` (2026-05-10, design-md-consolidation 1.4 fixup) which - edited `22a draft` → `An earlier draft` and left the dangling - fragment `The original` on the previous line. NOT introduced in - iter 23.5; architect's attribution corrected. Boss-fixed inline: - the dangling fragment removed. -- **[minor]** `crates/ailang-check/src/lib.rs:884-894` — workspace- - sibling linearity construction is O(N²) over module count. Bounded - for today's 2-module workspace; named here so a future Nx-module - workspace doesn't re-discover it. Bencher confirmed below that the - iter-23.5 H1 cost is negligible at N=2. No action. -- **[minor]** `crates/ailang-check/src/mono.rs:703-712` — - `contains_rigid_var` discriminates rigid vs. metavar by - `Type::Var.name.starts_with("$m")`. The journal already named this - as a future-fragility; a structural discriminator (separate - `Type::Meta` variant) would be the principled fix. Carry-on; not - blocking. -- **[nit]** `crates/ailang-check/src/lib.rs:709-726` — Float-NoInstance - addendum mutates `d.message` directly. The codebase has no - `with_hint` / `extend_message` convention yet, so this isn't - pre-existing drift, but the next class-specific addendum (e.g. - partial-Float `compare`) would compound `to_diagnostic` into a - conditional chain. Carry-on; refactor when a second addendum - arrives. - -## Bencher — compile_check regression diagnosis - -Bench-rerun at HEAD `3fed372` showed 4/12 check_ms metrics above -the 25% tolerance, with all 12 metrics in the +7 to +45% band — a -coherent cohort shift, not the journal-claimed "one sub-ms noise -metric". - -Two hypotheses tested via 5-run medians over 60 samples each across -four states (23.4 HEAD, 23.5 HEAD, 23.4 checker + 23.5 prelude, -23.5 checker + 23.4 prelude): - -- **H1** (linearity workspace-visibility + `contains_rigid_var`): - refuted. 23.5-checker + 23.4-prelude returns to baseline noise - (median +1.84% vs. baseline noise floor). -- **H2** (5 new polymorphic prelude `Def::Fn` bodies cost real - typecheck work): confirmed. 23.4-checker + 23.5-prelude reproduces - the full HEAD regression (median +18.62% vs. HEAD's +18.74%; gap - bounds the H1 interaction at <1%). - -Cost source: each workspace check now typechecks 5 additional -polymorphic `Def::Fn`s (each carrying `forall` + 1 class constraint -+ a `match` or small `not (eq …)` body) plus their constraint -resolution. ~0.2-0.3ms median additional work per `ail check` -subprocess; the ~5-10ms subprocess-spawn floor inflates the -relative-percent figure. - -**Verdict:** Ratify. The regression is the user-visible feature iter -23.5 shipped — five prelude helpers — accurately surfacing in the -bench as "typecheck does more work because the prelude has more -work to do". - -## Resolution - -- **Ratify path applied:** `python3 bench/compile_check.py - --update-baseline` re-measured (1 run, default 5 samples) and - wrote `bench/baseline_compile.json` with the new check_ms numbers. - Post-update rerun: 0/24 regressed, 0/24 improved beyond tolerance. -- **Important-drift fix applied:** the dangling `The original` - fragment removed from `docs/DESIGN.md:1692`. One-line tidy edit; - no behavioural change. -- **Minor / nit drift items:** carry-on. Linearity O(N²) is - observed-not-blocking; mono name-prefix discriminator is journal- - flagged; NoInstance addendum mutation pattern revisit when a - second class-specific addendum arrives. - -## Concerns - -- The iter-23.5 journal's bench framing was incorrect. The - implementer asserted "one sub-ms metric, two others stayed inside - on rerun" — fresh-rerun showed 4 above tolerance with consistent - cohort shift. Implementer-phase bench rerun count was likely too - small to see the cohort; orchestrator-audit pattern of running - bench 5× back-to-back at milestone close is the correct - granularity, and the 1-sample bench-check inside `implement` is - not. -- Implication for future iters: when `compile_check.py` exits 1 - during implement, the implementer should NOT classify it as - noise-band without doing the same multi-run, multi-state isolation - the bencher did here. Defer to audit if the framing is uncertain. - This is a soft convention — encoding it as a hard skill rule would - slow implement when the regression genuinely is noise; the - weaker form is "if bench exits 1, the journal Concerns must - describe what additional evidence was gathered, not just give a - hypothesis". - -## Known debt - -- DESIGN.md sentence-collision had been on main for two days - unnoticed. No drift-review ran between `613d4d8` (2026-05-10) - and this audit (2026-05-12); the implement-skill diff scope - (per-iter, narrow) does not catch unrelated pre-existing - fragments. Acceptable: drift-review at milestone close is exactly - the catch mechanism, and it caught it. - -## Files touched - -- `bench/baseline_compile.json` — re-measured baseline; numbers - reflect new prelude cost. -- `docs/DESIGN.md:1692` — dangling fragment removed. -- `docs/journals/2026-05-12-audit-23.md` (this file). -- `docs/journals/INDEX.md` — append (Boss-side). - -## Stats - -No stats file — audit is a Boss-direct skill, not an -implement-orchestrator dispatch. diff --git a/docs/journals/2026-05-12-audit-cma.md b/docs/journals/2026-05-12-audit-cma.md deleted file mode 100644 index 0e9eb32..0000000 --- a/docs/journals/2026-05-12-audit-cma.md +++ /dev/null @@ -1,122 +0,0 @@ -# audit-cma — Milestone close: Cross-model authoring-form test - -**Date:** 2026-05-12 -**Milestone:** Cross-model authoring-form test (cma.1 + cma.2 + cma.3) -**Status:** Closed, clean (after two `[low]` tidy fixes) - -## Architect drift review - -`ailang-architect` reported `drift_found` with two items over -`44c6e56..HEAD`: - -- `[low]` `docs/journals/INDEX.md` — commit `90512d5` ("brainstorm - Step 7.5 re-dispatch on any post-PASS spec edit") removed the - corresponding P1 todo from roadmap.md but did not add an INDEX - mirror. CLAUDE.md's roadmap convention requires a one-line mirror - in the journals when a roadmap entry is removed. -- `[low]` `experiments/.../README.md` — the "Total 8 passed" sentence - inherited from the cma.2 plan understates the actual harness - test sweep (13 passed across `--lib` unit + 4 integration suites). - Surface cosmetic; flagged in cma.3 known-debt. - -Both items confined to record-keeping, neither blocks milestone -close. Architect recommendation: carry on as planned. - -## Tidy fixes (boss-direct, inline) - -1. **INDEX backfill for `90512d5`.** Added a single-line entry in - `docs/journals/INDEX.md` pointing at the commit and noting that - no per-iter journal accompanies it (a small skill-discipline - tweak rather than an iter). The format mirrors existing entries - so future agents can navigate from INDEX to the commit history - without reconstructing. - -2. **README test-count correction.** The five-suite layout under - `experiments/2026-05-12-cross-model-authoring/README.md`'s - "Running the harness" section now lists all five suites - explicitly (inline `--lib` unit tests for strip_locations, plus - the four integration files) and reports the correct total - (13 passed). - -Both edits are ≤30 LOC and require no design judgement (CLAUDE.md -"trivial mechanical edits" carve-out); no planner+implement -dispatch needed. - -## Bench-regression check - -`bench/check.py && bench/compile_check.py && bench/cross_lang.py`: - -- `check.py`: exit 0; **63 metrics; 0 regressed, 5 improved beyond - tolerance, 58 stable**. -- `compile_check.py`: exit 0; 24 metrics; 0 regressed, 0 improved - beyond tolerance, 24 stable. -- `cross_lang.py`: exit 0; 25 metrics; 0 regressed, 0 improved - beyond tolerance, 25 stable. - -All three exit-code gates green. The five improvements on -`check.py` cluster on `latency.explicit_at_rc` (p99 -36.5%, p99_9 --41.8%, max -41.7%, p99_over_median -38.0%) plus -`throughput.bench_list_sum.gc_over_bump` -8.7%. - -**Decision: no baseline update (carry-on).** Rationale: - -- The cma milestone touched no in-workspace crate. Every line of - code change lives under `experiments/2026-05-12-cross-model-authoring/` - which is out of the root Cargo workspace by design. No mechanism - in the milestone could plausibly improve runtime latency or - GC-over-bump throughput. -- The improvements therefore reflect either (a) runtime variance - / scheduler / clang version drift between the prior audit-rt - baseline measurement and now, or (b) genuine system-level - improvement uncorrelated with the milestone. -- Ratifying an unexplained improvement would lock it into the - baseline and lose the ability to detect a future regression - back to the prior level. Better to leave the baseline pristine; - if the improvement holds across the next two or three audits, - the next audit can ratify with confidence. -- This is the audit-rt pattern carried forward: don't ratify what - the milestone didn't intentionally move. - -## Spec acceptance criteria — final check - -Reviewing the parent spec `docs/specs/2026-05-12-cross-model-authoring-form-test.md` -§"Acceptance criteria" against landed work: - -1. ✓ Directory `experiments/2026-05-12-cross-model-authoring/` - exists with all four sub-components (master/, render/, harness/, - runs/) plus top-level README. (cma.1 + cma.2) -2. ✓ `master/spec.md` is vollumfänglich; `spec_completeness.rs` - gate green; 34/34 AST variants covered by the curated subset. - (cma.1) -3. ✓ Both `rendered/json.md` and `rendered/ailx.md` are checked - in; all `render/` tests green. (cma.1) -4. ✓ `harness/` builds, passes 13/13 tests including the - mock_full_run end-to-end. (cma.2) -5. ✓ Four task definitions + reference solutions exist; the - `verify_references` integration test confirms each reaches - green through the real ail+clang pipeline locally. (cma.2) -6. ✓ Live run executed against IONOS (Qwen3-Coder-Next, - temperature=0, top_p=1, max-turns=5) and recorded under - `runs/2026-05-12-df7531/` with RUN_STATUS=ok, full per-turn raw - artefacts for all eight (cohort × task) combinations, - scores.csv, and summary.md. (cma.3) -7. ✓ DESIGN.md §"Decision 6" gained an "Empirical addendum - (2026-05-12)" subsection — 27 lines, model id + run date + - per-cohort means table + illustrative t3 contrast + explicit - scope note deferring the universal claim to a multi-subject - expansion. (cma.3) -8. ✓ Journal entry `2026-05-12-iter-cma.3.md` linked from - INDEX.md summarises the data point. (cma.3) -9. ✓ Roadmap P2 entry removed; P3 entry "Multi-subject expansion - (cross-model authoring-form follow-up)" added pointing at the - run as baseline. (cma.3) - -All nine criteria green. - -## Outcome - -Milestone closed clean after the two record-keeping tidies above. -The cross-model authoring-form test ships its harness, four MVP -tasks with verified reference solutions, a single-subject baseline -dataset, and a calibrated DESIGN.md addendum. The next step -(P3 roadmap: multi-subject expansion) is queued but not started. diff --git a/docs/journals/2026-05-12-audit-ct-tidy.md b/docs/journals/2026-05-12-audit-ct-tidy.md deleted file mode 100644 index 64c28e1..0000000 --- a/docs/journals/2026-05-12-audit-ct-tidy.md +++ /dev/null @@ -1,163 +0,0 @@ -# audit-ct-tidy — Milestone close: ct-tidy (closing iter ctt.3) - -**Date:** 2026-05-12 -**Milestone:** ct-tidy (ctt.1 + ctt.2 + ctt.3) -**Status:** Closed, three doc-drift items fixed inline (`ctt.tidy`) - -## Architect drift review - -`ailang-architect` reported `drift_found` across the commit range -`805bba3..0d3f44b`. Code-side changes match DESIGN.md / -spec / iters; three doc-side drift items surfaced (two high, -one low; one medium is acceptable per spec). - -**[high — spec acceptance]** `docs/DESIGN.md:1735-1736` — -`KindMismatch` listed as a live class-schema diagnostic. Stale -since ctt.3. - -**[high — spec acceptance]** `docs/DESIGN.md:1761-1763` — -`KindMismatch` cited as the rejection mechanism for -higher-kinded class params. Stale since ctt.3. - -**[high — spec acceptance]** `docs/roadmap.md:91-97` — the -`KindMismatch` retire todo unchecked with no forward-reference -to ctt.3. Spec acceptance criterion called for strikes on the -relevant todo lines; ctt.3 struck none. - -**[medium — record-keeping]** `examples/test_22b2_kind_mismatch.ail.json` -keeps the retired-diagnostic-name in its filename. Spec is silent -(acceptable as-is); rename is a future candidate, not in this -audit's scope. - -**[low — numerical drift]** Spec claim "`grep type_def_module` -returns 13 hits" is now 18 hits post-ctt.2. Intent honoured (every -consumer threaded); numerical figure is historical. - -### Tidy fixes applied inline (`ctt.tidy`) - -All three high-severity items addressed in this audit commit -(Boss-mechanical edits — three doc edits, no separate plan / -implement dispatch warranted): - -1. `docs/DESIGN.md:1733-1740` — dropped the `KindMismatch` bullet - from the class-schema diagnostics list; added a successor - paragraph naming `BareCrossModuleTypeRef` as the structural - rejection path for the malformed class-param-in-applied-position - shape; cross-references iter ctt.3. - -2. `docs/DESIGN.md:1761-1763` — rewrote the "Higher-kinded class - params" bullet under "What this typeclass design explicitly does - NOT support" to name `BareCrossModuleTypeRef` from canonical-form - validation instead of `KindMismatch` at class declaration. - -3. `docs/roadmap.md:91-97` — struck `[x]` with a forward-reference - to ctt.3 (canonical-form-rejection test stays green asserting - `BareCrossModuleTypeRef`). Adjacent ctt.2 todo at lines 98-105 - also struck (mechanical follow-up — ctt.2 closed the re-key it - tracks). - -Medium / low items skipped: filename rename + spec numerical claim -both fall outside ctt.tidy's scope (spec acceptance was on -intent-honour, not figure-precision; filename rename is a future -candidate the spec already permits as-is). - -## Bench-regression check - -Sequence run as Boss (the `&&` chain tripped at `check.py`'s exit -1; `compile_check.py` and `cross_lang.py` then ran separately): - -- `check.py`: **exit 1** — 63 metrics; 3 regressed, 3 improved - beyond tolerance, 57 stable. - - Regressions: - - `bench_list_sum.bump_s` +12.23% (tol 10%) — - **second consecutive sighting** (audit-eob first-sighted - this at +12.60%, this audit at +12.23%). - - `latency.explicit_at_rc.max_us` +44.31% (tol 25%) — - **first sighting**, rc-cohort tail-latency. - - `latency.implicit_at_rc.max_us` +35.07% (tol 30%) — - **first sighting**, rc-cohort tail-latency. - - Improvements: - - `bench_list_sum.gc_over_bump` -10.40% — mirror of the - bump_s regression (gc/bump ratio falls because bump rose). - - `bench_closure_chain.gc_over_bump` -15.05% — new improvement. - - `bench_list_sum_explicit.gc_over_bump` -8.30% — new improvement. -- `compile_check.py`: exit 0; 24 metrics, all stable within ±25%. -- `cross_lang.py`: exit 0; 25 metrics, all stable within ±15%. - -### Classification - -**bump_s persistence** (`bench_list_sum.bump_s`): second -consecutive sighting (audit-eob → audit-ct-tidy) at near-identical -magnitude (+12.60% → +12.23%). The audit-eob classification was -"first-sighting rule — observe in the next audit; if it persists, -investigate". This audit IS that next audit, and it persists. - -ctt.* milestone made zero codegen-relevant edits (the three iters -touched env-overlay doc + check-side registry key shape + -workspace-load-time diagnostic deletion). No plausible attribution -to ctt.*. The persistence is real but uncaptured — the underlying -cause predates this milestone and is independent of the work in -the audit window. - -Conservative call (same standing rule as previous three audits): -**do not** baseline-update. The persistence is now documented -across two audits; if it persists into the next audit -unchanged, ratify-without-attribution becomes the right call -(at three consecutive sightings the noise hypothesis is -unsupported). The latency tail-cluster regressions are first -sightings — observe. - -**Latency max_us regressions** (`explicit_at_rc.max_us`, -`implicit_at_rc.max_us`): first sighting in any recent audit. -Both metrics are rc-cohort tail-latency (the extreme-outlier -window). The bench harness runs ~50ms workloads; a single -outlier of +183µs (explicit) or +167µs (implicit) absolute -moves the max_us proportionally. ctt.* made zero codegen edits, -so the noise hypothesis is the most parsimonious explanation -on first sighting. - -Conservative call: first-sighting rule — **do not** -baseline-update; observe in the next audit. - -**Recurring latency.explicit_at_rc improvement cluster**: the -three-audit-long improvement cluster on `p99_us` / `p99_9_us` / -`p99_over_median` (previously at ~-37%, classified as -ratify-candidate after one more audit at audit-eob) now reports -within tolerance this audit: -- `p99_us` -24.87% (tol 25%) — stable (was: improvement) -- `p99_over_median` -24.10% (tol 25%) — stable (was: improvement) -- `p99_9_us` not flagged (within tolerance) - -The cluster has narrowed from ~-37% improvements to ~-24% -near-tolerance stable. The improvements partially reverted — -the metrics are still on the improvement side, but no longer -beyond tolerance. The audit-eob recommendation ("if the cluster -persists into the next two audits unchanged, ratify; if it -shifts or shrinks meaningfully, that is itself the attribution -signal") has fired: the cluster shrank meaningfully this audit. -This is itself attribution-evidence — the underlying cause is -variable rather than persistent. The ratify-pending plan is -withdrawn; the cluster is now noise-class, not signal-class. - -**Baseline left pristine on every script** for the fourth -consecutive audit. No `--update-baseline` invocation. - -## Status - -Closed. Ct-tidy milestone fully landed: - -- env-overlay shape ratified (ctt.1: DESIGN.md `## Env construction` - section + DuplicateCtor pinning test + 2 P2 todos closed). -- `Registry.type_def_module` re-keyed to `(owning_module, bare_name)` - with `caller_module` threaded through every consumer (ctt.2: - 3-module fixture + regression test). -- `KindMismatch` retired (ctt.3: variant + helper + dispatch + - Display arm deleted; test stays green via canonical-form - successor `BareCrossModuleTypeRef`). -- DESIGN.md tracks all three iters' anchors (ctt.tidy in this audit: - class-schema diagnostics list updated, higher-kinded params bullet - updated, ctt.3 retirement marker pattern). -- Roadmap reflects closure (ctt.tidy: two P2 todos struck `[x]`). - -Working tree exits audit clean. main HEAD advances by exactly -this audit commit. diff --git a/docs/journals/2026-05-12-audit-eob.md b/docs/journals/2026-05-12-audit-eob.md deleted file mode 100644 index e8a6b53..0000000 --- a/docs/journals/2026-05-12-audit-eob.md +++ /dev/null @@ -1,143 +0,0 @@ -# audit-eob — Milestone close: heap-str-abi (closing iter eob.1) - -**Date:** 2026-05-12 -**Milestone:** heap-str-abi (hs.1 + hs.2 + hs.3 + hs.4 + eob.1) -**Status:** Closed, three DESIGN.md tidies applied inline (`eob.tidy`) - -## Architect drift review - -`ailang-architect` reported `drift_found` across the commit range -`750f97e..78e8338`. Lockstep invariants and codegen ABI all -intact; three high-severity items in DESIGN.md plus one medium -record-keeping note. - -**[high — spec acceptance]** `docs/DESIGN.md:2338-2342` — -`float_to_str` described as "type-installed but codegen-deferred" -with `CodegenError::Internal`. Stale since hs.4. - -**[high — spec acceptance]** `docs/DESIGN.md:2387` — same caveat -inside the "What is supported" inventory; `int_to_str` was absent -from the inventory entirely. - -**[high — spec acceptance]** `docs/DESIGN.md` (missing) — no -"Str ABI" anchor documenting the heap-Str / static-Str dual -realisation, shared consumer ABI, or codegen-elision invariant -for static-Str-in-RC-paths. - -**[medium — record-keeping]** `docs/journals/2026-05-12-iter-hs.4.md` -closed `DONE_WITH_CONCERNS` with weakened RC assertions; eob.1 -retroactively restored the strict invariants. No forward-pointer -appended; readers cross-walk via INDEX.md. - -### Tidy fixes applied inline (`eob.tidy`) - -All three high-severity DESIGN.md drift items addressed in this -audit commit (Boss-mechanical edits — no separate plan / implement -dispatch warranted; the spec amend was small, the Str-ABI-anchor -content was already fully known from the just-closed milestone): - -1. `docs/DESIGN.md:2338-2343` (float_to_str caveat block): replaced - the codegen-deferred prose with a one-paragraph statement that - `float_to_str` and `int_to_str` are fully wired, allocate a - heap-Str slab, and carry `ret_mode: Own`. Forward-references the - new Str-ABI anchor. - -2. `docs/DESIGN.md:2387-2392` (inventory bullet): dropped the - codegen-deferred parenthetical from `float_to_str`; added - `int_to_str : (Int) -> Str` to the inventory; pointer to the - Str-ABI section for the dual realisation. - -3. `docs/DESIGN.md` (new "Str ABI" subsection placed after the - Float-semantics block, before "What is not (yet) supported"): - documents the two realisations (static-Str packed-struct - `<{ i64, [N+1 x i8] }>` globals in `.rodata`; heap-Str slabs - allocated via `ailang_rc_alloc(8 + len + 1)`); the shared - consumer ABI (Str pointer at len-field offset 0, bytes at - `payload + 8`, inherited `strcmp` semantics); and the - codegen-level non-RC invariant for static-Str (move-tracking - partial-drop + non-escape lowering + `Type::Con { name: "Str" }` - carve-outs at `field_drop_call` and `drop_symbol_for_binder`'s - App arm). - -Medium-severity hs.4-journal forward-pointer: skipped. INDEX.md -already links hs.4 → eob.1 chronologically; the eob.1 journal -explicitly cross-references hs.4's deferred fix as the iter's -motivation. A forward-pointer in hs.4's own file would push the -append-only convention; the cross-walk from INDEX is sufficient. - -## Bench-regression check - -Sequence run as Boss (the `&&` chain in audit-SKILL.md tripped at -`check.py`'s exit 1; `compile_check.py` and `cross_lang.py` then -run separately to read their exit codes cleanly): - -- `check.py`: **exit 1** — 63 metrics; 2 regressed, 5 improved - beyond tolerance, 56 stable. - - Regressions: `bench_list_sum.bump_s` +12.60% (tol 10%); - `bench_hof_pipeline.bump_s` +11.65% (tol 10%). Both only - marginally over tolerance; the other four `bump_s` metrics - (`bench_tree_walk`, `bench_closure_chain`, - `bench_compute_collatz`, `bench_list_sum_explicit`) are stable - within ±7%. - - Improvements: three on the `latency.explicit_at_rc` cluster - (`p99_us` -37.37%, `p99_9_us` -40.14%, `p99_over_median` - -37.35%); two `gc_over_bump` mirrors on `bench_list_sum` and - `bench_hof_pipeline` (ratio falls because the denominator — - `bump_s` — rose; same physical signal as the bump regressions). -- `compile_check.py`: exit 0; 24 metrics, all stable within ±25%. -- `cross_lang.py`: exit 0; 25 metrics, all stable within ±15%. - -### Classification - -**Latency cluster** (`latency.explicit_at_rc.p99_us` / -`p99_9_us` / `p99_over_median`): third consecutive sighting -across `audit-cma` → `audit-ms` → `audit-eob`. `audit-ms` flagged -this as "one more audit before considering ratification"; this is -that one more audit. - -eob.1's only codegen-side touchpoints are the `drop.rs` Str -carve-out (8 LOC) and the `ret_mode: Own` flip at four signature -sites; none of those should plausibly shift latency p99 by 37%. -The signal predates this milestone (audit-cma was three milestones -ago). Conservative call: still **do not** baseline-update on this -audit. The cluster is real and persistent, but the underlying -cause is unknown, and a baseline update without an identified -attribution would obscure the next signal that *does* have an -attributable cause. - -Recommendation forward: if the cluster persists into the next -two audits unchanged, ratify with `--update-baseline` and a -journal entry naming the persistence as the ratify rationale -(absence of identified cause being the standing fact). If the -cluster shifts or shrinks meaningfully, that is itself the -attribution signal. - -**bump_s cluster** (`bench_list_sum.bump_s`, -`bench_hof_pipeline.bump_s`): **first sighting** in any recent -audit. Both metrics only marginally over the 10% tolerance. -The other four `bump_s` measurements are stable, which makes -"shared underlying bump-allocator change" unlikely; per-fixture -system noise (12.6% on a ~50ms benchmark is ~6ms of jitter) is -the most parsimonious explanation. - -eob.1 made zero changes to bump-allocator codepaths. The -implementer's end-report flagged this same cluster as appearing -during the workspace bench sweep before commit. Conservative call: -first-sighting rule — **do not** baseline-update; observe in the -next audit; if it persists, investigate. - -**Baseline left pristine on every script** for the third -consecutive audit. No `--update-baseline` invocation. - -## Status - -Closed. Heap-str-abi milestone fully landed: -- Static-Str layout migrated (hs.1, hs.2). -- Heap-Str runtime additions wired (hs.3, hs.4). -- Effect-op-borrow rule + RC-discipline closes (eob.1). -- DESIGN.md anchors both arg-position rules + Str-ABI dual - realisation (eob.1 + eob.tidy in this audit). -- WhatsNew entry + roadmap P1 close (eob.1). - -Working tree exits audit clean. main HEAD advances by exactly -this audit commit. diff --git a/docs/journals/2026-05-12-audit-ms.md b/docs/journals/2026-05-12-audit-ms.md deleted file mode 100644 index eff50af..0000000 --- a/docs/journals/2026-05-12-audit-ms.md +++ /dev/null @@ -1,107 +0,0 @@ -# audit-ms — Milestone close: Multi-subject Authoring-Form Test (CodeLlama Replication) - -**Date:** 2026-05-12 -**Milestone:** Multi-subject Authoring-Form Test — CodeLlama Replication (ms.1 + ms.2) -**Status:** Closed, clean (no tidy fixes needed) - -## Architect drift review - -`ailang-architect` reported `clean` over the working tree -(`8689f7a..HEAD`, no commits yet). Working tree matches DESIGN.md -(the §Decision-6 addendum extension is the only DESIGN.md touch, -intentional and spec-mandated), matches the spec's seven acceptance -criteria, and matches the boss-only-commit / append-only-INDEX / -main-sacrosanct discipline. - -Two `[low]` items flagged, both expected: - -- The spec mandates an `audit-ms` INDEX entry on milestone close; - this audit pass is the one that produces it — written below. -- `docs/specs/2026-05-12-multi-subject-codellama.md:154` references - "the cma.3 baseline" (`runs/2026-05-12-df7531/`) which DESIGN.md - no longer cites after the addendum rewrite to a two-subject - view. The spec text is correct (the new runs are distinct from - cma.3) and the baseline dir is still on disk under the cma run - tree, so the asymmetry is purely cosmetic — ratified, no edit. - -Architect recommendation: carry on as planned. - -## Bench-regression check - -`bench/check.py && bench/compile_check.py && bench/cross_lang.py`: - -- `check.py`: exit 0; **63 metrics; 0 regressed, 5 improved beyond - tolerance, 58 stable**. -- `compile_check.py`: exit 0; 24 metrics; 0 regressed, 0 improved - beyond tolerance, 24 stable. -- `cross_lang.py`: exit 0; 25 metrics; 0 regressed, 0 improved - beyond tolerance, 25 stable. - -All three exit-code gates green. The five improvements on -`check.py` cluster on `latency.explicit_at_rc` (p99, p99_9, max, -p99_over_median) plus `throughput.bench_list_sum.gc_over_bump` — -the exact same cluster `audit-cma` (this morning's milestone close) -flagged and left unratified for the same reason. The ms milestone -touched only `experiments/2026-05-12-cross-model-authoring/harness/src/` -(plus docs); no in-workspace runtime / codegen / RC path changed. -The improvements therefore reflect runtime variance or -system-level drift uncorrelated with the milestone. - -**Decision: no baseline update (carry-on).** Same rationale as -audit-cma: ratifying an unexplained improvement would lock it into -the baseline and lose the ability to detect a future regression -back to the prior level. If the same improvement cluster persists -across the next two or three audits, it ratifies with confidence. -This is now the second audit in a row showing the same five-metric -improvement cluster; one more clean audit and the next audit can -ratify. - -## Spec acceptance criteria — final check - -Reviewing `docs/specs/2026-05-12-multi-subject-codellama.md` -§"Acceptance criteria" against landed work: - -1. ✓ `pipeline.rs:118` reads `format!("check: {e:#}")`; RED→GREEN - pinning unit test `check_format_preserves_anyhow_chain` present - in the same file. (ms.1) -2. ✓ Existing 13 harness tests still pass; suite now 14/14 with - the new pin test. (ms.1, re-confirmed at ms.2 close) -3. ✓ `meta-llama/CodeLlama-13b-Instruct-hf` ran to completion - against IONOS (8 cells, RUN_STATUS=ok at run level; 2 cells - `api_failure` on turn 1 disclosed in addendum). Chosen model id - recorded in ms.2 journal. (ms.2) -4. ✓ Qwen3-Coder-Next re-run with fixed pipeline completed (8 - cells, RUN_STATUS=ok). New dir `runs/2026-05-12-080864/` - distinct from cma.3 baseline `runs/2026-05-12-df7531/`; both new - runs checked into the working tree. (ms.2) -5. ✓ `docs/DESIGN.md` §"Decision 6 / Empirical addendum - (2026-05-12)" extended: framing paragraph + four-column per-cell - table (Qwen-JSON, Qwen-AILX, CodeLlama-JSON, CodeLlama-AILX) - covering prompt_tokens, completion_tokens, pass-rate, turn-count - distribution, top error class; api_failure anomaly disclosed; - scope paragraph updated to "two subjects, n=1 each". (ms.2) -6. ✓ Roadmap P3 multi-subject entry removed; one-line mirror in - `2026-05-12-iter-ms.2.md`. (ms.2) -7. ✓ Per-iter journal entries (ms.1, ms.2) linked from INDEX.md; - this audit-ms entry being added by this audit pass. (ms.{1,2, - audit}) - -All seven criteria green. - -## Outcome - -Milestone closed clean — no tidy fixes needed (both architect-flagged -`[low]` items are expected by-products of the audit step itself, -not drift). The multi-subject expansion ships: pipeline-error -formatting fixed (JSON-cohort now sees the full anyhow cause -chain), Qwen retroactive re-run plus first CodeLlama-13b-Instruct -run recorded, DESIGN.md §Decision-6 addendum now spans two -subjects with consistent direction (AILX cheaper + ≥ green for -both), roadmap P3 entry closed. Token spend across the milestone: -prompt 501,985 + completion 23,391 = 525,376. - -Decision 6 still pending universal ratification — the spec was -explicitly scoped to two subjects, not the ≥3-subjects + statistics -that a ratification would need. A future iteration may re-open the -queue if additional subjects are wanted; for now the second data -point is recorded and the queue is back to user-direction. diff --git a/docs/journals/2026-05-12-audit-rt.md b/docs/journals/2026-05-12-audit-rt.md deleted file mode 100644 index 0137d64..0000000 --- a/docs/journals/2026-05-12-audit-rt.md +++ /dev/null @@ -1,99 +0,0 @@ -# audit-rt — Milestone close: Roundtrip Invariant - -**Date:** 2026-05-12 -**Milestone:** Roundtrip Invariant (rt.1 + rt.2) -**Status:** Closed, clean (after rt.tidy fixes) - -## Architect drift review - -`ailang-architect` reported `drift_found` with four items over -`daf9f4f..HEAD`: - -- `[high]` DESIGN.md §"Roundtrip Invariant" Direction 2 (text → - JSON → text idempotency) is stated as a property but not - directly enforced by any of the four listed tests. Test 1 covers - Direction 1; test 2 covers `.ailx` → JSON ≡ counterpart; test 4 - covers CLI render → parse. None tests `parse → print → parse` - idempotency on a hand-authored `.ailx` text directly. Architect - recommendation: claim soften OR add a fifth test. -- `[high]` `docs/roadmap.md` P1 entry "Round-trip completeness - invariant" still `- [ ]` (open) after the milestone closed. - Stale per roadmap convention (finished entries → `[x]` then - removal with one-line journal mirror). -- `[low]` DESIGN.md wording: `<16-hex>` in new section vs - `<16-lowercase-hex>` in §"Data model". Casing constraint is - load-bearing for canonical-byte equality; should be stated - identically. -- `[low]` rt.1 plan errata note (planner emitted ast.rs-mismatching - destructure scaffold). Single occurrence absorbed cleanly by the - Step-3.4 escape hatch. Not a deeper drift pattern; informational. - -## Bench-regression check - -`bench/check.py && bench/compile_check.py && bench/cross_lang.py`: - -- `check.py`: exit 0; 63 metrics; 0 regressed, 0 improved beyond - tolerance, 63 stable. -- `compile_check.py`: exit 0; 24 metrics; 0 regressed, 0 improved - beyond tolerance, 24 stable. -- `cross_lang.py`: exit 0; 25 metrics; 0 regressed, 0 improved - beyond tolerance, 25 stable. - -All three bench gates green. Bench was the most plausible source -of a regression given rt.1 added three new tests but the tests -are pure readers and are not exercised by the bench harness; the -green signal matches that expectation. - -## Resolution (rt.tidy, boss-direct edits) - -Three fixes, mechanical, no plan/implement dispatch: - -1. **Direction-2 enforcement** — - `crates/ailang-surface/tests/round_trip.rs` gains a third test - `parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture`. - For every `.ailx` fixture asserts - `canonical_bytes(parse(t)) == canonical_bytes(parse(print(parse(t))))`. - For the 57 fixtures with a JSON counterpart this is logically - derivable from tests 1 + 2; the dedicated test stays robust for - future `.ailx` fixtures without a counterpart. DESIGN.md - §"Roundtrip Invariant" "Enforcement" list updated from four to - five tests; module doc-comment updated from "two complementary - checks" to "three complementary checks". Test PASS first-shot - across all 57 fixtures. - -2. **Roadmap P1 entry removed.** The "Round-trip completeness - invariant" milestone entry deleted; this journal entry is the - one-line mirror per roadmap convention. The P1 todo "Brainstorm - Step 7.5" remains; the P1 Show + print-rewire milestone remains. - -3. **Wording fix.** DESIGN.md §"Roundtrip Invariant" Float-bits-hex - description changed from `<16-hex>` to `<16-lowercase-hex>` to - match §"Data model". - -The `[low]` planner-errata carry-on item is retained in the rt.1 -journal "Concerns" section; it will surface again only if a future -tests-against-AST iteration repeats the pattern. - -## Spec acceptance criteria — final check - -1. ✓ Top-level `## Roundtrip Invariant` section in DESIGN.md; - Decision 6 Constraint 2 carries upward cross-reference. (rt.2) -2. ✓ `every_ailx_fixture_matches_its_json_counterpart` dynamic in - `crates/ailang-surface/tests/round_trip.rs`. (rt.1) -3. ✓ `crates/ailang-core/tests/schema_coverage.rs` with - exhaustive-match visitor; 34/34 variants observed. (rt.1) -4. ✓ `crates/ail/tests/roundtrip_cli.rs` with BLAKE3 identity - over `ail render` → tempfile → `ail parse`. (rt.1) -5. ✓ `cargo test --workspace` green with all four (now five) tests. - (rt.1 + rt.2 + rt.tidy) -6. ✓ No roundtrip gaps surfaced; no render/parse code changes - needed in this milestone. (rt.1) -7. ✓ Tests are pure readers; the CLI test's only filesystem side - effect is `tempfile::TempDir`. (rt.1) - -## Outcome - -Milestone closed clean. Five workspace-wide tests now anchor the -`.ail.json` ↔ `.ailx` bijection plus AST-variant coverage; the -property has a dedicated top-level section in DESIGN.md. No -follow-up iterations queued in this milestone. diff --git a/docs/journals/2026-05-12-iter-23.5.md b/docs/journals/2026-05-12-iter-23.5.md deleted file mode 100644 index f1153fc..0000000 --- a/docs/journals/2026-05-12-iter-23.5.md +++ /dev/null @@ -1,200 +0,0 @@ -# iter 23.5 — Prelude free fns + E2E (Eq/Ord milestone close) - -**Date:** 2026-05-12 -**Started from:** 35c6eb5 -**Status:** DONE -**Tasks completed:** 8 of 8 - -## Summary - -Closes milestone 23. Ships the five polymorphic prelude free fns -`ne` / `lt` / `le` / `gt` / `ge` (Tasks 1-2), three end-to-end -fixtures covering primitive composition + user-ADT integration + -the Float negative case (Tasks 3-5), the Float-aware `NoInstance` -diagnostic addendum (Task 5), the DESIGN.md §"Prelude classes" -amendment (Task 6), and the roadmap split between shipped Eq/Ord -and the pending Show + print-rewire half (Task 7). Bench check -runs clean except a noisy sub-millisecond `check_ms.local_rec_capture` -regression that sits just over the 25% tolerance — ratifiable per -the audit-tolerance rule (the prelude grew by 5 polymorphic -`Def::Fn`s, so the typecheck workload per workspace measurably -shifts). - -Two implementer-phase fixes were needed beyond the plan, both -symmetric to existing precedent (iter 23.4-prep). They surfaced -when the natural test fixtures (user-defined `at_most x y = not (gt -x y)` composing prelude poly fns) hit two distinct false-positive -shapes: - -1. **Linearity check needed workspace-wide visibility** for - imported polymorphic free fns + class methods. Previously the - linearity pass only saw the current module's globals, so a - user `at_most` forwarding its borrow params into `gt` (defined - in the prelude) hit `callee_arg_modes("gt", 2) → []` → - default-Consume → `consume-while-borrowed` false positive. Fix - extends `linearity::check_module` to a `check_module_with_visible` - entry that takes additional Module references whose top-level fns - + class methods are registered in globals; the workspace-side - driver passes all sibling modules. Pure precedent extension of - 23.4-prep, which added class-method visibility for the same - reason. - -2. **Mono pass needed to skip rigid-var-bearing free-fn targets.** - When a polymorphic free fn's body calls another polymorphic - free fn (e.g. `at_most`'s body calls `gt x y` with x, y of - type rigid `a`), the synth-channel observer recorded a - FreeFnCall whose type args were rigid `Type::Var { name: "a" - }`. The existing Unit-default fallback (correct for - `is_empty(Nil)`-shape unobservable metavars) was unsound here: - it produced `gt__Unit` whose body referenced `compare`, which - has no Unit instance → "unknown variable: compare" at the - synthesised body. Fix discriminates rigid vars - (Type::Var without `$m` prefix) from unbound metavars (`$m` - prefix) — rigids skip (they get re-collected with concrete - substitution when the enclosing fn itself monomorphises), - metavars keep their Unit-default. - -One existing fixture (`examples/test_22b1_missing_method.ail.json`) -had to rename its class method `ne → tne` because `ne` is now a -top-level prelude free fn and the workspace registry's -method-name uniqueness check rejected the collision. The -fixture's docstring already foreshadowed this case ("Class is -`TEq` rather than `Eq` to avoid colliding…"); extended to also -cover the `ne → tne` rename rationale. - -## Per-task notes - -- iter 23.5.1: `ne` free fn — added `Def::Fn ne` to prelude (body - `not (eq x y)`); created `crates/ail/tests/prelude_free_fns.rs` - with helpers `fixture()` and `mono_symbol_names()` + first test; - smoke fixture `examples/ne_at_int_smoke.ail.json`. RED→GREEN. - Adjusted `mono.modules.iter()` → `.values()` per the actual - `BTreeMap` shape (Boss pre-flight flagged). -- iter 23.5.2: `lt` / `le` / `gt` / `ge` free fns — four - `Def::Fn`s in prelude (each pattern-matches on `compare x y` - against one of LT/GT plus a wildcard); four extending tests in - the same `prelude_free_fns.rs`; four smoke fixtures. RED→GREEN - per fn. -- iter 23.5.3: positive E2E — `examples/eq_ord_polymorphic.ail.json` - with user-defined `at_most a a → Bool` composing prelude `not` - + `gt`; `crates/ail/tests/eq_ord_e2e.rs` with helpers - `fixture_path()` + `build_and_run()` plus - `eq_ord_polymorphic_runs_end_to_end`. Two implementer-phase - fixes (linearity + mono rigid-skip) landed during this task to - unblock the GREEN. Final stdout pinned at `"1\n0\n1"` (matches - `io/print_int`'s actual newline-separator behaviour; plan text - had `"101"` based on an incorrect concatenation assumption). -- iter 23.5.4: user-ADT E2E — `examples/eq_ord_user_adt.ail.json` - with `type IntBox = MkIntBox Int` + user-written - `instance Eq IntBox` + `instance Ord IntBox` + `main` calling - `eq` twice. Two extending tests: - `eq_ord_user_adt_runs_end_to_end` (stdout `"1\n0"`) and - `eq_ord_user_adt_eq_intbox_hash_stable` (record-then-pin). - Mono symbol name is `eq__cde77856` (8-hex-prefix mangling for - compound types per DESIGN.md §"Mangling scheme"); body hash - pinned at `9daaffa7528d2a1c`. The Ord-IntBox instance's - retType needed qualifying as `prelude.Ordering` per the - canonical-type-names rule. -- iter 23.5.5: Float-aware `NoInstance` addendum — implemented in - `CheckError::to_diagnostic()` (the per-iter spec-§"NoInstance" - conversion path; option-a from the plan's structural-choice - list). Addendum fires when `class ∈ {Eq, Ord}` AND `at_type == - "Float"`: appends `— Float has no Eq/Ord instance by design - (partial orderability per IEEE-754); see DESIGN.md §"Float - semantics".` New `examples/eq_float_noinstance.ail.json` + - `crates/ail/tests/eq_float_noinstance.rs`. RED→GREEN. -- iter 23.5.6: DESIGN.md amendment — inserted Milestone-23 - amendment paragraph in §"Prelude (built-in) classes". Grep - gate RED→GREEN. -- iter 23.5.7: roadmap split — replaced the single Post-22 - Prelude entry with two: `[x]` Eq/Ord (shipped 23.5) and `[ ]` - Show + print rewire (gated on heap-`Str` ABI). Also - disambiguated the `depends on: Post-22 Prelude` reference at - line 73 to point specifically at the Eq/Ord half (Op routing's - actual dependency). -- iter 23.5.8: bench gate — `bench/check.py` exit 0 (clean), - `bench/compile_check.py` exit 1 (one sub-ms `check_ms` - regression at the tolerance line — see Concerns), `bench/cross_lang.py` - exit 0 (clean). - -## Concerns - -- `bench/compile_check.py` exited 1 with one regression: - `check_ms.local_rec_capture` measured +26.36% over baseline - (1.1ms → 1.4ms, tolerance 25.0%). This is a sub-millisecond - noise-band metric, and the prelude grew by 5 polymorphic - `Def::Fn`s in this iter so a measurable typecheck-workload - shift is expected per the Boss-noted audit tolerance. Two - other check_ms metrics in the first run (`borrow_own_demo`, - `bench_list_sum`) crossed the line on one sample but stayed - inside on the rerun, confirming this is noise-band fluctuation - rather than a structural regression. Recommend ratifying at - iter close via `--update-baseline`; flagged as a Boss - judgement call. -- The mono-pass rigid-var skip (Task 3 inline fix) preserves - the existing Unit-default for unbound metavars exactly (the - `is_empty(Nil)` shape from iter 23.4.8 is untouched). The - discrimination is by name prefix (`$m`) which is the same - convention `Subst::meta_id` uses; a future schema refactor - that drops the prefix convention would silently break the - skip rule. Code comment names this. -- The linearity-side `check_module(&Module)` standalone entry - is now `#[cfg(test)]`-gated because production code uses - `check_module_with_visible(m, &visible_extra)` instead. Unit - tests still call the no-extra path. Acceptable; flagging in - case a future caller wants the standalone form back. - -## Known debt - -- The `eq_ord_user_adt` mono-symbol name (`eq__cde77856`) is - pinned by literal string match in the test. If the canonical - type-bytes encoding for `IntBox` ever changes (e.g. via a new - schema field on `TypeDef`), the test fails with a clear - type-mangling-regression message. That is the intended drift - alarm; no immediate debt. - -## Files touched - -- `crates/ailang-check/src/lib.rs` — Float-aware `NoInstance` - addendum in `to_diagnostic()`; linearity call site updated to - pass workspace siblings as visible-extra. -- `crates/ailang-check/src/linearity.rs` — new - `check_module_with_visible(m, visible_extra)` entry; - `check_module(&Module)` made test-only. -- `crates/ailang-check/src/mono.rs` — free-fn target collection - now distinguishes rigid `Type::Var` from `$m`-prefixed - metavars; rigid-bearing targets are skipped; new helper - `contains_rigid_var`. -- `crates/ailang-core/src/workspace.rs` — docstring update on - the `iter22b1_missing_method` test naming the new `ne` reason - for the `TEq`/`tne` rename. -- `crates/ail/tests/prelude_free_fns.rs` (new) — five workspace-level - tests, one per prelude free fn, plus helpers. -- `crates/ail/tests/eq_ord_e2e.rs` (new) — three E2E tests: - polymorphic composition, user-ADT integration, mono-symbol - hash stability. -- `crates/ail/tests/eq_float_noinstance.rs` (new) — Float-aware - NoInstance pin. -- `examples/prelude.ail.json` — five new `Def::Fn` entries - (ne / lt / le / gt / ge). -- `examples/ne_at_int_smoke.ail.json` (new), `lt_at_int_smoke.ail.json` - (new), `le_at_str_smoke.ail.json` (new), `gt_at_bool_smoke.ail.json` - (new), `ge_at_int_smoke.ail.json` (new) — five workspace-level - smoke fixtures. -- `examples/eq_ord_polymorphic.ail.json` (new) — positive E2E: - polymorphic helper composing prelude fns at Int / Bool / Str. -- `examples/eq_ord_user_adt.ail.json` (new) — user-ADT E2E: - `IntBox` with user-written Eq + Ord instances. -- `examples/eq_float_noinstance.ail.json` (new) — negative-Float - fixture firing the addended diagnostic. -- `examples/test_22b1_missing_method.ail.json` — `ne → tne` - rename to avoid collision with the new prelude `ne` free fn. -- `docs/DESIGN.md` — Milestone-23 amendment paragraph inserted - in §"Prelude (built-in) classes". -- `docs/roadmap.md` — Post-22 Prelude entry split into shipped - Eq/Ord + pending Show; Op-routing depends-on reference - disambiguated. - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-23.5.json diff --git a/docs/journals/2026-05-12-iter-24.1.md b/docs/journals/2026-05-12-iter-24.1.md deleted file mode 100644 index dcd5ec9..0000000 --- a/docs/journals/2026-05-12-iter-24.1.md +++ /dev/null @@ -1,216 +0,0 @@ -# iter 24.1 — `bool_to_str` + `str_clone` runtime + codegen wiring - -**Date:** 2026-05-12 -**Started from:** 8bfa09adc7774a1c94af609fc4c78a70ba77be48 -**Status:** DONE_WITH_CONCERNS -**Tasks completed:** 6 of 6 - -## Summary - -First iter of milestone 24 ("Show + print rewire"). Wires two new -heap-Str-producing primitives — `bool_to_str : (Bool) -> Str` and -`str_clone : (Str borrow) -> Str` — through the runtime C code, type -checker, codegen IR lowering, and IR-header preamble, mechanically -mirror-imaging the hs.4 wiring for `int_to_str` / `float_to_str`. The -two new symbols install lockstep in `builtins.rs::install` and -`synth.rs::builtin_value_type` with `ret_mode: Own`; codegen gains -two new IR-header `declare` lines, two `Emitter::lower_app` arms, and -two `is_static_callee` whitelist entries. Plan's pre-existing-drift -fix landed alongside: `int_to_str` row added to `builtins.rs::list()` -(hs.4 added the `env.globals` entry but missed the `list()` row). - -Nine new tests landed per plan: 2 builtins-install unit tests -(`install_bool_to_str_signature`, `install_str_clone_signature`), -2 IR-shape unit pins (`bool_to_str_emits_call_to_ailang_bool_to_str`, -`str_clone_emits_call_to_ailang_str_clone`), 5 E2E tests -(`bool_to_str_drop_balances_rc_stats`, `bool_to_str_emits_true_branch`, -`bool_to_str_emits_false_branch`, `str_clone_drop_balances_rc_stats`, -`str_clone_cross_realisation_uniform_abi`). Four `.ail.json` fixtures -shipped. Five IR snapshots regenerated for the two new declare lines. - -One substantive deviation from the plan's "purely-additive wiring" -framing surfaced during Task 6 fixture validation: the -`str_clone_cross_realisation_uniform_abi` test's literal assertion -`allocs == 3, frees == 3, live == 0` could not hold under the -pre-24.1 uniqueness analyser because the analyser's `globals` map -does not contain builtin signatures — `str_clone`'s `param_modes: -[Borrow]` is invisible to `callee_arg_modes`, so a `src_heap` let- -binder consumed by `str_clone(src_heap)` walks as `Position::Consume`, -incrementing `consume_count` and gating off the scope-close -`ailang_rc_dec`. Fixed inline by registering builtin signatures in -both `uniqueness.rs::infer_module` and -`linearity.rs::check_module_with_visible` (8 lines × 2 files, -symmetric to the existing class-method registration the linearity -pass added in iter 23.4-prep). The fix is substantively correct (it -applies a pre-existing pattern to a new class of callees) but is -NOT in the plan's literal Edit text — see Concerns for the -discussion. - -Acceptance verified: full `cargo test --workspace` green (513 tests -passing, 0 failures); the five new E2E tests pass with the plan's -literal numeric assertions; the two new IR-shape pins pass; the five -IR snapshot tests pass after `UPDATE_SNAPSHOTS=1` regen (diff is -exactly the two new `declare ptr @ailang_bool_to_str(i1)` and -`declare ptr @ailang_str_clone(ptr)` lines). `bench/compile_check.py` -24/24 stable, 0 regressions. `bench/cross_lang.py` 25/25 stable, 0 -regressions. Grep verification: 44 hits across runtime/str.c + -builtins.rs + synth.rs + lib.rs (well above the plan's ≥ 10 -threshold), 10 hits in snapshots (2 declares × 5 files), 0 build -warnings. - -## Per-task notes - -- iter 24.1.1: Appended `ailang_bool_to_str(bool b) -> char *` and - `ailang_str_clone(const char *src) -> char *` to `runtime/str.c` - after `ailang_float_to_str` (str.c:128-143). Both use the existing - static `str_alloc(uint64_t)` helper for heap-Str slab allocation; - `bool_to_str` writes either "true" (4 bytes) or "false" (5 bytes); - `str_clone` reads `len` from offset 0 of source and memcpy's bytes - + trailing NUL to a fresh slab. Standalone clang -c compile green. -- iter 24.1.2: Installed `bool_to_str : (Bool) -> Str` and - `str_clone : (Str borrow) -> Str` in `builtins.rs::install` - (builtins.rs:213-232, between the existing `int_to_str` block and - the `is_nan` block), both with `ret_mode: ParamMode::Own`. Mirror- - installed the same two signatures in `synth.rs::builtin_value_type` - (synth.rs:181-194, lockstep). Added three rows to - `builtins.rs::list()` (`int_to_str` drift fix from hs.4 + - `bool_to_str` + `str_clone`). Two new unit tests - (`install_bool_to_str_signature`, `install_str_clone_signature`) - appended to `builtins.rs::tests` using the fallback explicit - `Term::Lit` shape (plan-noted: `lit_bool` / `lit_str` helpers do - not exist; only `lit_int` / `lit_float` are defined). Both new - unit tests PASS. -- iter 24.1.3: Codegen IR wiring. (a) Extended the IR-header preamble - at `lib.rs:539-548` with two unconditional declares - `declare ptr @ailang_bool_to_str(i1)` and - `declare ptr @ailang_str_clone(ptr)`. (b) Inserted two new arms in - `Emitter::lower_app` at `lib.rs:1934-1969` (after the existing - `float_to_str` arm), each emitting a direct call to the runtime - C glue with arity check + `lower_term`+`fresh_ssa`+call triple - matching `int_to_str`'s structural shape. (c) Extended the - `is_static_callee` builtin-callee whitelist at `lib.rs:2133-2135` - with `bool_to_str` and `str_clone` (strictly required so - `Term::Var { name: "bool_to_str" }` reaches `lower_app`'s chain - instead of the UnknownVar fallback). Full workspace build clean. -- iter 24.1.4: Two IR-shape unit pin tests - (`bool_to_str_emits_call_to_ailang_bool_to_str`, - `str_clone_emits_call_to_ailang_str_clone`) appended at the bottom - of `lib.rs::mod tests`, mirroring the - `int_to_str_lowers_to_ailang_int_to_str_call` template. Both PASS. -- iter 24.1.5: Five IR snapshots (`hello.ll`, `list.ll`, `max3.ll`, - `sum.ll`, `ws_main.ll`) regenerated via `UPDATE_SNAPSHOTS=1`. RED - observed first as expected (5 ir_snapshot_* tests failed with - "run UPDATE_SNAPSHOTS=1 cargo test ir_snapshot_ to refresh"); - GREEN after regen. Diff is exactly the two new declare lines. - Each .ll file shows 2 matches for the new symbols. -- iter 24.1.6: Four `.ail.json` fixtures created - (`bool_to_str_drop_rc.ail.json`, `bool_to_str_smoke_false.ail.json`, - `str_clone_drop_rc.ail.json`, `str_clone_cross_realisation.ail.json`) - per the plan's verbatim JSON. Five new E2E tests appended to - `crates/ail/tests/e2e.rs` after `heap_str_repeated_print_balances_rc_stats`. - The `str_clone_cross_realisation_uniform_abi` test initially - FAILED with `frees=2, expected=3`; root-caused to the - uniqueness-analyser-globals scope gap (see Concerns); fixed by - registering builtin signatures in `uniqueness.rs::infer_module` - and `linearity.rs::check_module_with_visible`. After the fix all - five new E2E tests PASS with the plan's literal numeric - assertions. Full `cargo test --workspace`: 513 passed, 0 failed. - `bench/compile_check.py` 24/24 stable. `bench/cross_lang.py` 25/25 - stable. Grep verification per plan Step 9 met: 44 code-side hits, - 10 snapshot hits, 0 build warnings. - -## Concerns - -- **Analyser-globals registration extends the plan's "purely-additive" - scope.** The plan framed 24.1 as "Pure-additive wiring across four - crates / files." The actual landed change extends to two more - files (`uniqueness.rs`, `linearity.rs`) with 8 lines each — a fix - to the uniqueness / linearity analyser-globals scope so the - `str_clone` builtin's `param_modes: [Borrow]` is visible to the - `callee_arg_modes` walker. Without the fix, the - `str_clone_cross_realisation_uniform_abi` test's plan-literal - assertion `frees == 3` does not hold (one heap-Str slab leaks - because the App-arg walker can't see the Borrow mode → walks - `src_heap` as Consume → `consume_count == 1` → drop emission - gated off). - - The fix is **symmetric** to the iter 23.4-prep extension that - registered class-method signatures in the same `globals` maps for - the same reason (a builtin/class-method callee carrying - `param_modes` would not be visible to the analyser, false-firing - `consume-while-borrowed` lints AND, in this iter, leaking heap-Str - slabs). The plan's verbatim Edit text did not include the fix; the - plan's verbatim test assertion required it. Per the orchestrator- - agent's "make the reasonable call and continue" mandate I applied - the fix in-place; it is the substantively-correct repair, not a - design departure. - - The hs.4 journal's "Known debt" item on this is partially closed - by this iter (the App-side of the Borrow-walker visibility is now - fixed for builtin callees). The eob.1 fix (effect-op args walk as - Borrow regardless of callee `param_modes`) covered the `Term::Do` - side; the corresponding `Term::App` side is now covered for - builtins. User-fn `Term::App` callees still benefit from their - own def being registered in the analyser's `globals` from the - module's own def list (lines 113-115 in uniqueness.rs); imported - user-fn callees benefit from the iter 23.5 visible-extra - registration in linearity.rs (lines 220-238). So the only - remaining unknown-callee case is one this iter does not encounter. - -## Known debt - -- **`str_clone` could rc_inc on heap-Str input** instead of allocating - a fresh copy, but this requires distinguishing heap-Str from - static-Str at runtime — a tagging scheme the codebase deliberately - dropped post-hs.2 (UINT64_MAX sentinel removal). Out of scope per - the parent spec's "`str_clone` as rc_header bump" entry. - -- **`bool_to_str` as schema-`if`** with two static-Str literals would - avoid the per-call heap allocation but requires phi-of-static-Str - through the drop-elision pipeline that is not load-bearing-ratified - today. Open commitment per the parent spec. - -## Files touched - -- Modified (12 files): - - `runtime/str.c` — appended `ailang_bool_to_str` (16 LOC incl. - comment) and `ailang_str_clone` (18 LOC incl. comment) - - `crates/ailang-check/src/builtins.rs` — two new - `env.globals.insert` blocks (`bool_to_str` + `str_clone`), three - new rows in `list()` (drift fix + two new), two new unit tests - - `crates/ailang-codegen/src/synth.rs` — two new arms in - `builtin_value_type` (lockstep partner of the checker insert) - - `crates/ailang-codegen/src/lib.rs` — IR-header preamble extends - with 2 new `declare` lines; 2 new `lower_app` arms; 2-entry - `is_static_callee` whitelist extension; 2 new IR-shape pin tests - - `crates/ailang-check/src/uniqueness.rs` — register builtin - signatures in `infer_module`'s `globals` map so `callee_arg_modes` - resolves builtin `param_modes` correctly - - `crates/ailang-check/src/linearity.rs` — register builtin - signatures in `check_module_with_visible`'s `globals` map for the - same reason - - `crates/ail/tests/e2e.rs` — 5 new E2E tests appended - - `crates/ail/tests/snapshots/hello.ll`, - `crates/ail/tests/snapshots/list.ll`, - `crates/ail/tests/snapshots/max3.ll`, - `crates/ail/tests/snapshots/sum.ll`, - `crates/ail/tests/snapshots/ws_main.ll` — golden-IR regen via - `UPDATE_SNAPSHOTS=1`; diff is exactly the two new - `declare ptr @ailang_bool_to_str(i1)` and - `declare ptr @ailang_str_clone(ptr)` lines - -- Created (4 files): - - `examples/bool_to_str_drop_rc.ail.json` — RC-stats fixture for - `bool_to_str true` (4-byte slab, single drop) - - `examples/bool_to_str_smoke_false.ail.json` — stdout-smoke - fixture for `bool_to_str false` (5-byte slab) - - `examples/str_clone_drop_rc.ail.json` — RC-stats fixture for - `str_clone "hello"` (static-Str input → fresh heap-Str clone) - - `examples/str_clone_cross_realisation.ail.json` — cross- - realisation invariant: clones one heap-Str (output of - `int_to_str 42`) AND one static-Str (literal `"abc"`), printing - both; pins the "uniform consumer ABI" claim - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-24.1.json diff --git a/docs/journals/2026-05-12-iter-boss.md b/docs/journals/2026-05-12-iter-boss.md deleted file mode 100644 index b221b39..0000000 --- a/docs/journals/2026-05-12-iter-boss.md +++ /dev/null @@ -1,69 +0,0 @@ -# 2026-05-12 — iter boss: `/boss` skill — autonomous-mode discipline gated to user invocation - -## What shipped - -- New skill at `skills/boss/SKILL.md`. Consolidates the three - mode-specific subsections that previously lived in `CLAUDE.md`: - Direction freedom, Notifications, Done-state notifications: - WhatsNew.md. -- `CLAUDE.md` refactored. Three subsections removed (~95 lines). - Skill-system pointer paragraph extended with one sentence - naming `/boss` as the autonomous-mode gate. -- `.claude/skills/boss` symlink created (`../../skills/boss`), - matching the discoverability convention codified in - `skills/README.md`. -- `skills/README.md` updated: skill table extended with a `boss` - row, heading shifted from "seven skills" to "eight skills", - pipeline diagram captioned with a `/boss`-wraps note. -- Two cross-references repointed: `skills/implement/SKILL.md` - and `skills/implement/agents/ailang-implement-orchestrator.md` - both named the "Done-state notifications" subsection of - CLAUDE.md by sub-heading; both now point at `skills/boss/SKILL.md`. - -## Why - -CLAUDE.md is always loaded into context. Mixing universal facts -(agent role boundaries, commit discipline, design rationale, file -roles, TDD-for-bugs) with mode-specific autonomy rules (direction -freedom, notifications, WhatsNew procedure) was OK while -autonomous-by-default was the operating assumption — but it -conflicted with the user's intent that a fresh session should be -collaborative-interactive unless explicitly elevated. - -The mode-switch needs to be explicit. `/boss` is the explicit -elevation, and only the three genuinely mode-specific subsections -move with it. Universal orchestrator discipline stays in CLAUDE.md -because it applies whether `/boss` is active or not: agents need -to know role boundaries when dispatched in any session, design -discipline applies whenever a design choice is made (including a -`brainstorm` invoked outside `/boss`), and the orchestrator can -edit skills at any time. - -## Decisions - -- **Tightened the move-list from seven items to three.** The - initial spec proposed moving "My role: orchestrator", "Authority - over skills/", "Design rationale ≠ effort", "Feature acceptance", - "When NOT to delegate", plus the three mode-specific subsections. - Planner recon surfaced that the first five are universal, not - mode-specific — they apply in any session. The move-list was - narrowed to the three genuinely mode-specific items. The - brainstorm Step 7.5 grounding-check was re-dispatched on the - edited spec and returned PASS. -- **No private agents under `skills/boss/agents/`.** `/boss` is an - orchestration layer, not a worker. It dispatches the existing - skills (and their agents). The skill table reflects this by - listing `boss` as a meta-skill without an agent-roster row. -- **Cross-reference repoints kept tight.** Recon found 11+ - agent/skill files that reference "CLAUDE.md — orchestrator - framing". Most of those resolve correctly to the universal - content that stays in CLAUDE.md. Only two files name the moved - subsections by sub-heading (`implement/SKILL.md` and the - orchestrator-agent's standing reading list); only those two - were repointed. - -## Status - -Iter closed. No bench regressions to check (markdown-only change, -no compiler touch). No audit needed (single-iter milestone, no -DESIGN.md change). Acceptance criteria from the spec all met. diff --git a/docs/journals/2026-05-12-iter-cma.1.md b/docs/journals/2026-05-12-iter-cma.1.md deleted file mode 100644 index ed55add..0000000 --- a/docs/journals/2026-05-12-iter-cma.1.md +++ /dev/null @@ -1,128 +0,0 @@ -# iter cma.1 — Master mini-spec + renderer for the cross-model authoring experiment - -**Date:** 2026-05-12 -**Started from:** fd6efdbc -**Status:** DONE -**Tasks completed:** 9 of 9 - -## Summary - -Stands up the master mini-spec source and the renderer binary under a -new top-level `experiments/2026-05-12-cross-model-authoring/` tree -that lives outside the root Cargo workspace. The renderer projects -one form-agnostic `master/spec.md` into two form-specific files -(`rendered/json.md` and `rendered/ailx.md`) by walking three directive -forms (`{form-only: json}`, `{form-only: ailx}`, `{example: }`) -and emitting per-example fenced blocks from 13 curated `.ail.json` -fixtures. Four test gates land green: a 7-test splitter unit suite, -a `spec_completeness` test borrowing the AST-variant visitor verbatim -from `crates/ailang-core/tests/schema_coverage.rs` (34/34 variants -observed across the curated subset), an `example_roundtrip` test -mirroring the surface roundtrip contract, and a `token_balance` test -that gates form-only block tokens to ±5% under `tiktoken-rs::cl100k_base` -(final ratio: json=1800, ailx=1742, ratio 3.22%). Two follow-on iters -(cma.2: harness, cma.3: live IONOS run + DESIGN.md addendum) are out -of scope per the spec. - -## Per-task notes - -- cma.1.1: Bootstrap experiment dir + Cargo skeleton — created - `experiments/.../{README.md,render/Cargo.toml,render/src/main.rs}` - plus the four target directories. Spec-pinned dependency paths - resolve `ailang-core` and `ailang-surface` via relative - `../../../crates/...` references. -- cma.1.2: Directive splitter (TDD) — 7 unit tests RED first, then - implementation; added a `[lib]` target so the integration tests - can reach `xmodel_render::splitter`. GREEN on first impl run. -- cma.1.3: spec_completeness test — visitor + EXPECTED_VARIANTS - list lifted verbatim from `crates/ailang-core/tests/schema_coverage.rs`; - only `examples_dir()` body and the test name changed. Confirmed RED - on the empty master/examples/ directory (failure mode: "no .ail.json - fixtures found"). -- cma.1.4: 13 curated `.ail.json` fixtures (one per file as planned). - The plan's 13 fixture topics naturally covered 28 of the 34 AST - variants; the remaining 6 (Def::Const, Term::Let, Term::LetRec, - Term::If, Term::Clone, Term::ReuseAs) were folded into existing - fixtures per the plan's Step 4.14 instruction ("Extend the fixture - closest in scope … do not weaken the test"). Concrete absorptions: - Let + Clone into `fn_calls_prelude`; LetRec into `data_with_match` - (as a recursive `count_via_letrec` helper); If into - `match_literal_pattern` (as a `sign_if` helper); ReuseAs into - `data_simple` (the `wrap_one` body uses `(reuse-as src ...)`); - Const into `floats` (a `const pi : Float = 3.14`). All 34 variants - green on first run. -- cma.1.5: example_roundtrip test — verbatim from plan; all 13 - fixtures roundtrip through `ailang_surface::print` and - `ailang_surface::parse` to the same canonical bytes on first - implementation run. -- cma.1.6: master/spec.md — 13 sections of form-agnostic prose with - parallel JSON/AILX form-only blocks in §2-5, §7, §9, §10 plus the - prelude table in §11 (no form-only there; the table itself is the - content). Section §11's prelude tabulation reflects the live - `crates/ailang-check/src/builtins.rs` symbol set: `+`, `-`, `*`, - `/`, `%`, `==`, `!=`, `<`, `<=`, `>`, `>=`, `not`, `neg`, - `int_to_float`, `float_to_int_truncate`, `float_to_str`, `is_nan`, - `nan`/`inf`/`neg_inf`, `__unreachable__`, plus the effect ops - `io/print_{int,bool,str,float}`. `int_to_str` is explicitly named - as OUT (heap-Str ABI milestone). -- cma.1.7: Wire renderer end-to-end — examples.rs and main.rs filled - in verbatim from the plan. Renderer ran clean on first invocation; - json.md (33409 bytes) and ailx.md (23964 bytes) emit with the - expected 13 fenced blocks each (and zero of the wrong fence type). -- cma.1.8: token_balance test — first run reported json=1800, - ailx=1692, ratio 6.00% (above the 5% threshold). One substantive - AILX-side extension to §4 (mode annotations) added authoring - guidance on `over-strict-mode` and hash stability under future - default shifts; final ratio json=1800, ailx=1742, 3.22%. The - rebalancing is content, not filler. -- cma.1.9: Final render + full sweep — `cargo test --manifest-path - experiments/.../render/Cargo.toml` ends with 10 passed; 0 failed - across four test suites; `git status` shows the entire experiment - tree as untracked. - -## Concerns - -- Task 1: Cargo's automatic workspace discovery hits the root - manifest, so the experiment crate refuses to build on its own. The - fix is the standard Cargo idiom for a nested-but-out-of-workspace - crate: add an empty `[workspace]` table to the experiment's - `Cargo.toml`. The plan did not enumerate this; I added it with a - comment pointing at the spec authority (§Architecture lines 90-95) - for the standalone constraint. Boss should confirm this is the - intended mechanism vs. e.g. adding the experiment to a - `workspace.exclude` list in the root manifest (the spec is explicit - that the root Cargo.toml is not touched, which the chosen mechanism - respects). -- Task 1 ancillary: a `.gitignore` was added under - `experiments/.../render/` ignoring `/target` and `Cargo.lock`. Not - enumerated in the plan but a structural necessity to keep cargo's - build output and lockfile out of the repo. Boss can keep, replace - with a top-level rule, or move to the root `.gitignore`. - -## Known debt - -- None surfaced inside the iter. Two named deferrals remain out of - scope (cma.2 harness, cma.3 live run + DESIGN.md addendum), per - the spec. - -## Files touched - -27 new files, all under `experiments/2026-05-12-cross-model-authoring/`: - -- `README.md` -- `master/spec.md` -- `master/examples/*.ail.json` × 13 -- `render/Cargo.toml` -- `render/.gitignore` -- `render/src/{main,lib,splitter,examples}.rs` -- `render/tests/{splitter_unit,spec_completeness,example_roundtrip,token_balance}.rs` -- `rendered/json.md` -- `rendered/ailx.md` - -`git diff --name-only HEAD | wc -l`: 0 (untracked tree; use -`git status --porcelain --untracked-files=all experiments/...` to -enumerate). - -## Stats - -`bench/orchestrator-stats/2026-05-12-iter-cma.1.json` diff --git a/docs/journals/2026-05-12-iter-cma.2.md b/docs/journals/2026-05-12-iter-cma.2.md deleted file mode 100644 index 04cfba8..0000000 --- a/docs/journals/2026-05-12-iter-cma.2.md +++ /dev/null @@ -1,212 +0,0 @@ -# iter cma.2 — Harness binary + four tasks + reference solutions + three integration tests - -**Date:** 2026-05-12 -**Started from:** 21c2d2cfaa999980f727a069792e3bd073c1f1e7 -**Status:** DONE -**Tasks completed:** 13 of 13 - -## Summary - -Stands up the sibling `harness/` standalone Cargo crate under -`experiments/2026-05-12-cross-model-authoring/`, mirroring the -out-of-workspace idiom from cma.1's `render/`. Six modules under -`harness/src/`: `strip_locations` (regex pass for form-asymmetric -location info), `pipeline` (subprocess wrapper around -`ail parse|check|build` plus the model's compiled binary, with a -fail-fast preflight on `ail` + `clang`), `ionos` (blocking reqwest -client with retry policy), `mock` (canned-response loader keyed by -cohort/task/turn), `scoring` (CSV + summary.md writers), `tasks` -(task-definition struct + loader). `main.rs` ties them into the -per-(cohort, task) loop. Four task definitions (`t1_add_three`, -`t2_length`, `t3_main_prints`, `t4_count_zeros`) with reference -solutions that all reach green through the actual ail+clang -pipeline. 13/13 tests pass: 5 inline unit tests under `--lib` -(strip_locations), 5 strip_locations integration tests against -verbatim recon-captured stderr fixtures, 1 verify_references -integration test driving every reference through the real pipeline, -1 mock_full_run end-to-end with eight rows and per-cohort artefact -trees, 1 budget_abort with the harness aborting cleanly under a -1500-token cap. - -## Per-task notes - -- cma.2.1: Bootstrap harness Cargo project + skeleton — created - `harness/{Cargo.toml,.gitignore,src/lib.rs,src/main.rs}` plus - six stub module files. Empty `[workspace]` table at the top of - Cargo.toml keeps the crate out of the root workspace. Cargo.lock - gitignored per spec. First `cargo build` succeeded with the - expected unused-import warnings on the skeleton. -- cma.2.2: strip_locations module — written verbatim from the - plan with 5 inline unit tests (json_pointer, byte_offset, line/ - column, Caused-by chain, passthrough). All 5 green on first run. -- cma.2.3: pipeline module — written verbatim from the plan plus - a small implementer-phase repair: `ail check` enforces filename - stem == module name, so `pipeline::run_pipeline` extracts the - top-level `"name"` field from the JSON via a new - `module_name_from_json` helper and renames the working file to - `.ail.json` before invoking `ail check`. Without - this, every reference would have failed at the `ail check` step - with "module name in file does not match expected name from - path". See Concerns. -- cma.2.4: ionos module — written verbatim from the plan; added - `thiserror = "1"` to `[dependencies]`. Compile-time warnings - about three unused things on the `IonosClient` path (Auth, Usage, - `last_err` assignment) are expected and flagged in the plan. -- cma.2.5: mock module — written verbatim; compile clean. -- cma.2.6: scoring module — written verbatim; compile clean. -- cma.2.7: tasks module — written verbatim; compile clean. -- cma.2.8: Author four task definitions + reference solutions — - task.json files written verbatim from the plan text. Reference - `.ail.json` files authored fresh against the cma.1 templates - (hello.ail.json shape for t3; gc_stress.ail.json's polymorphic - `List a` shape for t2 and t4; canonical chained `+` for t1's - three-arg sum; canonical `eq` invocation pattern + branched `if` - for t4's count_zeros). All four references compile clean and - produce the expected stdout. Reference solutions intentionally - omit `param_modes` because every parameter is Implicit and the - canonical AILang form omits the field in that case — consistent - with hello.ail.json, list.ail.json, eq_primitives_smoke.ail.json, - and the explanatory doc in `master/examples/param_modes_all.ail.json`. - See Concerns. Added `tempfile = "3"` to `[dev-dependencies]` for - the `verify_references.rs` integration test. The test runs every - reference through `pipeline::run_pipeline(Cohort::Json, ...)` - end-to-end (ail parse-check-build-execute); passes when - `AIL_BIN` is set to the release binary. -- cma.2.9: Wire main.rs end-to-end — written verbatim from the - plan; added `tempfile = "3"` to `[dependencies]` (not only - dev-deps) because `main.rs::run_one` uses it for per-turn - workdir. Compile clean (modulo the three flagged warnings from - ionos). -- cma.2.10: Five stderr fixtures + strip_locations integration - test — all five fixture files written verbatim from the plan - (which itself captured them from recon's run against current HEAD). - Integration test asserts strip behaviour against each fixture; - 5/5 green on first run. -- cma.2.11: mock_full_run integration test + mock_full_run.json - fixture — fixture authored by inlining each - `master/tasks/.reference.ail.json` as the `content` field - for the green-path entries, plus `ail render` for the AILX-cohort - inlines (verbatim AILX serialisations of the same references). - Test runs the harness binary against the fixture; 8 rows total - (4 tasks × 2 cohorts), `(json, t3_main_prints)` reaches green on - turn 2 (matches the plan's intended cycle: broken-then-fixed), - `(ailx, t1_add_three)` runs to the 5-turn limit (`(module garbage)` - repeated). All artefacts (turn_N_program.ext, - turn_N_request.json, turn_N_response.json, - turn_N_check_stderr.txt, turn_N_run_stdout.txt) land in - `per_cohort///`. 1/1 green. -- cma.2.12: budget_abort integration test — test verbatim from - the plan, plus an implementer-phase repair to `main.rs`: with a - tiny budget (1500), the outer loop broke out after consuming - ~2400 tokens across tasks 1+2 (both green on turn 1), but - neither row carried `final_status = budget_abort` because the - inner `run_one` only marks BudgetAbort if the budget is exhausted - *during* the turn loop. The plan's test assertion expects at - least one row with `budget_abort`. Repair: after the outer loop - breaks with `run_status="budget_exceeded"`, fill in the not-yet-run - (cohort, task) pairs with synthetic `BudgetAbort` rows. mock_full_run - remains green (the new code only runs on budget_exceeded). - 1/1 green. See Concerns. -- cma.2.13: README update + final sweep — appended "Running the - harness" section to `experiments/.../README.md` verbatim from the - plan. Full `cargo test` sweep: 5 (lib unit, strip_locations) + 5 - (integ, strip_locations) + 1 (verify_references) + 1 - (mock_full_run) + 1 (budget_abort) = 13/13 green. `git status` - shows the README modification + the new harness/ tree + the new - master/tasks/ files, all unstaged. - -## Concerns - -- **Pipeline filename-matching: minimal repair landed in `pipeline.rs`, - not flagged in the plan.** `ail check` enforces filename stem == - module name; the plan's `run_pipeline` writes the model's program - to `prog.{ext}`, which would have been rejected at every `ail - check` invocation. The repair adds a 9-line helper - `module_name_from_json` (reads JSON, returns top-level `name`) plus - 14 lines in `run_pipeline` that rename `prog.ail.json` -> - `.ail.json` between parse and check. Behaviour - outside that rename is unchanged. The structural alternative - (telling the model to name its module `prog`) would have been - semantically wrong — the task description prompts say "module - named t1_add_three" etc. Recommend a forward-fix sweep in cma.3 - if the structural decision (whether the pipeline or the prompt - owns module naming) warrants spec attention. -- **`param_modes` omitted from every reference solution.** The plan's - Step 8.2 text says "mode annotations on every parameter", but the - templates the plan references (hello.ail.json, list.ail.json, - eq_primitives_smoke.ail.json, etc.) all omit `param_modes`, and - the `param_modes_all.ail.json` master example's own doc string - documents "Implicit mode is the legacy default — `param_modes` is - omitted from canonical JSON when every entry is Implicit". The - references stay in canonical form. If the plan's intent was - explicit `["implicit", "implicit", ...]` annotations on every - reference, that is a follow-on amendment. -- **Budget-abort row-filling, minimal repair landed in `main.rs`, - not flagged in the plan.** Without it, Task 12's test fails on - the assertion `expected at least one budget_abort row`. Repair - is gated on `run_status == "budget_exceeded"` so the green path - is unchanged. -- **README "Total 8 passed" text mismatches Step 13.2's "13 passed" - expectation.** The plan's README text (Step 13.1) advertises four - test suites with "Total 8 passed" — counts the four integration - suites but skips the 5 inline unit tests under `--lib`. Step 13.2 - expects 13 across `--lib` + 4 integration tests. The README text - is reproduced verbatim per plan; the actual sweep produces 13. - Recommend a docfix in cma.3 ("Total 13 passed across --lib + 4 - integration tests"), but the cma.2 plan locked the README copy. -- **`AIL_BIN` env required for `verify_references.rs` and - `mock_full_run.rs` to run from a fresh shell.** These tests shell - out to `ail`; without `ail` on PATH the preflight bails. The - README documents this. CI integration is out of scope. - -## Known debt - -- The harness has not yet been live-fired against IONOS (deferred - to cma.3 per spec). The retry/backoff path in - `ionos::IonosClient::post` has no test coverage in cma.2 (mock - mode bypasses it entirely). -- Three compile warnings in `ionos.rs` (`Auth` and one `Usage` are - read in tests/other code; `last_err` second assignment is - intentional dead-store for retry-loop symmetry). Not addressed. - -## Files touched - -Modified (1): -- `experiments/2026-05-12-cross-model-authoring/README.md` - -New — harness crate (8 source, 4 integration tests, 6 fixtures, 1 -manifest, 1 .gitignore): -- `experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml` -- `experiments/2026-05-12-cross-model-authoring/harness/.gitignore` -- `experiments/2026-05-12-cross-model-authoring/harness/src/lib.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/src/main.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/src/strip_locations.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/src/pipeline.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/src/ionos.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/src/mock.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/src/scoring.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/src/tasks.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/strip_locations.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/verify_references.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/mock_full_run.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/budget_abort.rs` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/check_unbound_var.stderr` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/check_type_mismatch.stderr` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/check_bare_xmod.stderr` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/check_schema_missing_field.stderr` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/parse_unclosed.stderr` -- `experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/mock_full_run.json` - -New — master tasks (4 task.json + 4 reference.ail.json): -- `experiments/2026-05-12-cross-model-authoring/master/tasks/t1_add_three.task.json` -- `experiments/2026-05-12-cross-model-authoring/master/tasks/t1_add_three.reference.ail.json` -- `experiments/2026-05-12-cross-model-authoring/master/tasks/t2_length.task.json` -- `experiments/2026-05-12-cross-model-authoring/master/tasks/t2_length.reference.ail.json` -- `experiments/2026-05-12-cross-model-authoring/master/tasks/t3_main_prints.task.json` -- `experiments/2026-05-12-cross-model-authoring/master/tasks/t3_main_prints.reference.ail.json` -- `experiments/2026-05-12-cross-model-authoring/master/tasks/t4_count_zeros.task.json` -- `experiments/2026-05-12-cross-model-authoring/master/tasks/t4_count_zeros.reference.ail.json` - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-cma.2.json diff --git a/docs/journals/2026-05-12-iter-cma.3.md b/docs/journals/2026-05-12-iter-cma.3.md deleted file mode 100644 index 99411e3..0000000 --- a/docs/journals/2026-05-12-iter-cma.3.md +++ /dev/null @@ -1,103 +0,0 @@ -# iter cma.3 — Live run + DESIGN.md addendum + milestone close - -**Date:** 2026-05-12 -**Started from:** fe1fb6b -**Status:** DONE (milestone Cross-Model Authoring-Form Test closed) -**Mode:** Boss-direct (no implement dispatch per spec) - -## Summary - -Live IONOS run against Qwen3-Coder-Next executed; raw dataset -under `experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-df7531/`. -AILX cohort reached green on 2/4 tasks at ~98k total tokens; JSON -cohort reached green on 1/4 tasks at ~207k total tokens. The only -first-attempt success was AILX on `t3_main_prints`. JSON-cohort -failures clustered in the typecheck stage (check×4); AILX-cohort -failures clustered in the parse stage (parse×3). DESIGN.md -§"Decision 6" gains a 27-line "Empirical addendum (2026-05-12)" -recording the data, the illustrative t3 case, and an explicit -single-subject single-run scope note. Multi-subject expansion -queued as a fresh roadmap P3 entry. - -## Run details - -- Endpoint: `https://openai.inference.de-txl.ionos.com/v1/chat/completions` -- Model: `Qwen/Qwen3-Coder-Next` -- Parameters: temperature=0, top_p=1, max-turns=5, - token-budget=500000. -- RUN_STATUS: `ok`. Total tokens consumed: 305,148 of 500,000 - budget (39% headroom). -- 40 raw artefact files (8 runs × 5 per-turn files on average) - preserved under `runs/2026-05-12-df7531/per_cohort///`. - -## Per-task notes - -- `json/t1_add_three`: turn_limit. Cohort consumed 5 turns trying - to satisfy the schema constraints around `do` / `seq` / effect - positioning. Top error class `check` throughout. -- `json/t2_length`: turn_limit. ADT + polymorphism stress; the - model emitted plausible-shape JSON that the typechecker - rejected. -- `json/t3_main_prints`: green on turn 2 (the only green in the - JSON cohort). Turn 1 was ~23 lines of structured JSON with a - small structural mistake the check loop pointed out (with - location info stripped); turn 2 was correct. -- `json/t4_count_zeros`: turn_limit. Typeclass surface + Eq - constraint shape; the model never converged. -- `ailx/t1_add_three`: green on turn 3. Got the structure right - but needed two correction turns to align with the prelude - function naming. -- `ailx/t2_length`: turn_limit. Errors mixed `check` and `parse` - (model occasionally produced malformed s-expressions on - correction turns). -- `ailx/t3_main_prints`: green on turn 1, ~5 lines of tagged - s-expr. Only first-attempt success in the entire run. -- `ailx/t4_count_zeros`: turn_limit. Parse-stage failures on - every turn; the model struggled with the typeclass-constraint - shape in AILX. - -## DESIGN.md edit - -§"Decision 6: authoring surface" gains a new -"Empirical addendum (2026-05-12)" subsection (~27 lines after the -existing prose, before §"Decision 8"). Records: model id, run -parameters, the 6-row metric table, an illustrative paragraph -naming the t3 contrast, and the explicit scope note that the -data is one subject + one deterministic run and that the -universal claim of Decision 6 stays pending multi-subject -follow-up. - -## Roadmap - -- Removed: P2 entry "Cross-model authoring-form test" (closed). -- Added: P3 entry "Multi-subject expansion - (cross-model authoring-form follow-up)" pointing at this run as - baseline. - -## Concerns - -- AILX cohort's high parse-error count is partly model-side - (small grammar mistakes the typechecker would have forgiven) - and partly the strip_locations regex erasing `at byte N` - offsets, which leaves the model without precise localisation - during correction turns. Symmetric degradation is the fairness - anchor; calibration trade-off is documented in the spec. -- The JSON cohort's mini-spec at ~15k system tokens vs AILX's - ~7k is an inherent form asymmetry (JSON is wordier per - construct). Token costs are partly a measurement of *the form*, - not just of how easy each is to author. This is part of what - the experiment measures, but worth naming. -- Only 3/8 (cohort, task) pairs reached green. The dataset's - signal-to-noise ratio is modest; the multi-subject expansion - needs more tasks and/or more permissive prelude coverage so - the failure clusters are less dominant. - -## Known debt - -- Multi-subject expansion (≥2 additional foreign LLMs, ≥4 more - tasks, repetitions for statistical robustness) — queued as the - P3 roadmap entry. Not started. -- README "Total 8 passed" inherited from the cma.2 plan still - understates the actual 13/13 test count; the prose was carried - verbatim per cma.2 plan discipline. Cosmetic; can be fixed in - a future tidy iter. diff --git a/docs/journals/2026-05-12-iter-ctt.1.md b/docs/journals/2026-05-12-iter-ctt.1.md deleted file mode 100644 index 553dacb..0000000 --- a/docs/journals/2026-05-12-iter-ctt.1.md +++ /dev/null @@ -1,47 +0,0 @@ -# iter ctt.1 — Env-overlay shape ratification - -**Date:** 2026-05-12 -**Started from:** 0a36294e61f08d8de85c084415d694d3736e0c67 -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -Ratifies the two-overlay shape of the check environment (`env.types` -+ `env.ctor_index`) and pins its load-bearing consumer. The split -decision now has a DESIGN.md anchor (§"Env construction") explaining -the owning-index / reverse-index roles, the rationale for not -collapsing into one variant-keyed map, and the intentional asymmetry -with the mono side (narrowed to types-only at ct.3.2, because its -runtime-ctor-lookup consumer story differs from the check side's -in-band `DuplicateCtor` diagnostic). A new behavioural-pin test -(`crates/ailang-check/tests/duplicate_ctor_pin.rs`) asserts -`code == "duplicate-ctor"` plus message-substring coverage for a -two-ADT-same-ctor fixture, going red if the per-module rebuild at -`crates/ailang-check/src/lib.rs:1342-1384` is ever silently removed. -Two P2 roadmap todos struck `[x]` with one-line forward-references to -this iter. No production-code edits. `cargo test --workspace` green. - -## Per-task notes - -- iter ctt.1.1: New pinning test `crates/ailang-check/tests/duplicate_ctor_pin.rs` — builds a `Workspace` inline with one module declaring `type Cat = Twins` and `type Dog = Twins`, calls `check_workspace`, asserts a `Diagnostic` with `code == "duplicate-ctor"` plus message substring for `Twins` / `Cat` / `Dog`. PASS on first run, as the plan predicted (behaviour pre-exists). -- iter ctt.1.2: Inserted new top-level section `## Env construction` in `docs/DESIGN.md` between `## Convention: qualified cross-module references` (line 1941) and `## Data model` (now line 1988, was 1959 pre-edit). Three paragraphs: split definition, rationale, check-side / mono-side asymmetry. Cross-references the pinning test. -- iter ctt.1.3: Two `[ ]` → `[x]` strikes in `docs/roadmap.md` at lines 79 (overlay shape question) and 183 (per-module overlay narrowing). Each gains a one-line forward-reference suffix on its `context:` line pointing at iter ctt.1 and the closing artefact. - -## Concerns - -(none) - -## Known debt - -(none) - -## Files touched - -- `crates/ailang-check/tests/duplicate_ctor_pin.rs` (new) -- `docs/DESIGN.md` (modified — `## Env construction` section inserted) -- `docs/roadmap.md` (modified — two strikes) - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-ctt.1.json diff --git a/docs/journals/2026-05-12-iter-ctt.2.md b/docs/journals/2026-05-12-iter-ctt.2.md deleted file mode 100644 index 4653bbc..0000000 --- a/docs/journals/2026-05-12-iter-ctt.2.md +++ /dev/null @@ -1,142 +0,0 @@ -# iter ctt.2 — `Registry.type_def_module` re-key to tuple-keyed map - -**Date:** 2026-05-12 -**Started from:** 548ebf8d51758ab9f156b266e0baa59c78b14034 -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -`Registry.type_def_module` re-keyed from `BTreeMap` to -`BTreeMap<(String, String), String>` keyed by `(owning_module, -bare_name)`. A new `caller_module: &str` parameter threads through -`normalize_type_for_registry` and `Registry::normalize_type_for_lookup`, -and all four `ailang-check` consumer sites pass the correct -module-context per the plan's Design Notes block. A regression test -plus a three-module fixture exercise the bare-name collision the -re-key fixes — two modules each declaring `type Foo` now register -under distinct canonical keys instead of tripping the silent-overwrite -defect. `cargo test --workspace` green (16 binary-test suites, -~600 tests total, zero failures). - -## Per-task notes - -- iter ctt.2.1: RED-first — three fixture JSON files - (`ctt2_collision_{cls,lib,main}.ail.json`) plus - `crates/ailang-core/tests/ctt2_registry_rekey.rs`. Test goes - RED today. Failure shape was `OrphanInstance` rather than the - plan's predicted `DuplicateInstance` (a) or missing-hash (b) — - see Concerns. -- iter ctt.2.2: GREEN — atomic re-key edit pass. Touched the field - type + doc-comment, `Registry::normalize_type_for_lookup` - signature + body, pass-1 declaration + insert, - pass-2 coherence-check lookup + canonical-form hashing call, - `normalize_type_for_registry` signature + doc-comment + body - (recursive calls thread caller_module everywhere), the inline - `ct1_5a_normalize_recurses_into_forall_constraints` test - at workspace.rs:2543, and all four `ailang-check` consumer sites - at the module-context dictated by the plan's Design Notes - (lib.rs:1640 → `env.current_module.as_str()`, mono.rs:121 → - `defining_module.as_str()`, mono.rs:624 and :1175 → `module_name`). - Clean `cargo build --workspace` at the end of the pass. -- iter ctt.2.3: workspace-wide test gate. New test goes GREEN. - Full `cargo test --workspace` green, no test in any crate - regressed. The plan's Step 2 hypothesis ("a fixture under - `examples/` might legitimately exercise a bare cross-module - type reference") did not materialise — the existing corpus - uses canonical-form (qualified) cross-module type refs - exclusively, so the tightening had no surface. - -## Concerns - -- Task 1 RED-failure shape diverged from the plan's predicted - shapes (a) `DuplicateInstance` and (b) missing-`contains_key`. - The plan's verbatim two-module fixture pair (lib imports main + - main imports lib + lib-side instance with qualified class - `ctt2_collision_main.MyC`) had two independent structural - defects against the loaded codebase invariants: - 1. The workspace loader rejects import cycles - (`WorkspaceLoadError::Cycle`) at module-name granularity — - the plan's claim "Two-way import is supported; workspace - loader DFS handles cycles at module-name granularity" is - empirically false. Confirmed by inspection at - `workspace.rs:1410-1413` and `:1451-1454`. - 2. `validate_canonical_type_names` rejects qualified - `InstanceDef.class` with `QualifiedClassName` (see test - `ct1_validator_rejects_qualified_instancedef_class` at - workspace.rs:2321) — the plan's lib-side instance with - `class: "ctt2_collision_main.MyC"` would have been rejected - even without the cycle. - - Implementer-phase repair: introduced a third module - `ctt2_collision_cls.ail.json` carrying `class MyC`, broke the - cycle (both lib and main import cls only; main also imports - lib so the loader DFS reaches it), and dropped the qualifier - from both instance heads (both declare bare `class: "MyC"` - satisfying coherence via the type's module). The plan's - *core* RED-condition is preserved: two modules each declaring - `type Foo` with `instance MyC Foo` collide on the pre-ctt.2 - bare-name `type_def_module` key. - - The actually-observed RED shape was `OrphanInstance` rather - than `DuplicateInstance` because the pass-2 coherence check - (`workspace.rs:656-659`) uses the bare-name lookup *before* - the duplicate-uniqueness check fires — lib's `Foo` and main's - `Foo` both resolve to whichever module won the alphabetical - race (`ctt2_collision_main` beats `ctt2_collision_lib` - lexicographically), so lib's instance is no longer coherent - ("type lives in main, class lives in cls, instance lives in - lib — neither matches"). The re-key fixes this consumer site - simultaneously with the duplicate-check site because both go - through the same `type_def_module` lookup, now tuple-keyed. - - The deviation from the plan's verbatim fixtures is an - implementer-phase NEEDS_CONTEXT repair that the - user-no-stop-instruction permitted in-line; the plan's *intent* - (RED-first against the bare-name collision) is achieved and - the GREEN-state matches the plan's acceptance criteria - unchanged. Boss may want to back-edit the plan or amend the - spec to note the multi-module fixture shape if this kind of - collision regression returns later. - -## Known debt - -- The `OrphanInstance` consumer site at `workspace.rs:656-659` - is fixed by ctt.2 but the diagnostic-shape change (orphan - instead of duplicate-on-loser) is not separately tested. The - new test asserts the GREEN state directly (both entries land - under distinct keys) which transitively exercises both - consumer sites, but neither the orphan nor the duplicate - diagnostic shapes are pinned by a dedicated RED test. Adding - one is plausibly out-of-scope for ctt.2. - -## Files touched - -Modified (3): -- `crates/ailang-check/src/lib.rs` — single call-site update at - the NoInstance check inside `check_fn`. -- `crates/ailang-check/src/mono.rs` — three call-site updates - (monomorphise_workspace at :121, collect_mono_targets at :624, - collect_residuals_ordered at :1175). -- `crates/ailang-core/src/workspace.rs` — field re-key, - `Registry::normalize_type_for_lookup` signature, - `normalize_type_for_registry` signature + body, - pass-1 declaration + insert, - pass-2 coherence-check lookup + canonical-form hashing call, - inline test `ct1_5a_normalize_recurses_into_forall_constraints` - at :2543; doc-comments at field-decl and at both - `normalize_…` fns. - -Added (4): -- `crates/ailang-core/tests/ctt2_registry_rekey.rs` — regression test. -- `examples/ctt2_collision_cls.ail.json` — class-bearing module - (not in plan; required to break the cycle in the plan's - verbatim fixture design). -- `examples/ctt2_collision_lib.ail.json` — `type Foo = MkLib` + - bare `instance MyC Foo`. -- `examples/ctt2_collision_main.ail.json` — `type Foo = MkMain` + - bare `instance MyC Foo`, plus imports of both `cls` and `lib`. - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-ctt.2.json diff --git a/docs/journals/2026-05-12-iter-ctt.3.md b/docs/journals/2026-05-12-iter-ctt.3.md deleted file mode 100644 index 1045a37..0000000 --- a/docs/journals/2026-05-12-iter-ctt.3.md +++ /dev/null @@ -1,74 +0,0 @@ -# iter ctt.3 — `KindMismatch` retire (pure deletion across 3 files) - -**Date:** 2026-05-12 -**Started from:** 0e556f085c14993441b7d25dcdc69077b77c83b0 -**Status:** DONE -**Tasks completed:** 2 of 2 - -## Summary - -`WorkspaceLoadError::KindMismatch`, its dispatch site in -`validate_classdefs`, its `walk_kind_mismatch` helper, its Display -arm in `crates/ail/src/main.rs:workspace_error_to_diagnostic`, and -the "dead-but-defensive" doc-comment on the canonical-form-rejection -test are all retired. Two adjacent textual-consistency edits ride -along: the `validate_classdefs` doc-comment narrows from "Three -diagnostics fire from here" to "Two" with a successor sentence -naming the new path, and the `crates/ailang-check/src/lib.rs:1428` -22b.1-era archaeology comment drops the `KindMismatch` token from -its enumeration of class-schema diagnostics and gains a one-line -ctt.3 retirement marker. The existing test -`class_param_in_applied_position_fires_canonical_form_rejection` -stays green unchanged — it asserts `BareCrossModuleTypeRef` on the -malformed fixture, which is the canonical-form-validator successor -the plan named. `cargo build --workspace` clean, no orphan match -arms, no dead-code warnings. `cargo test --workspace` green -(~600 tests across 16 binary-test suites, zero failures). - -## Per-task notes - -- iter ctt.3.1: atomic deletion pass — 6 Edits across 3 files. workspace.rs ×4 (variant delete; `validate_classdefs` doc + dispatch-loop delete; `walk_kind_mismatch` helper delete; "dead-but-defensive" doc delete). main.rs ×1 (`W::KindMismatch` arm in `workspace_error_to_diagnostic`). lib.rs ×1 (22b.1 archaeology comment). Build green at Step 7. -- iter ctt.3.2: workspace-wide test gate — grep substantively clean in `crates/` (one residual `KindMismatch` token at lib.rs:1431 is the ctt.3 retirement marker introduced by Step 6 itself, not a live consumer); canonical-form-rejection test PASS; full `cargo test --workspace` PASS. - -## Concerns - -- Plan-internal contradiction surfaced during Task 2 Step 1: the - plan's Step 6 `new_string` deliberately introduces a new - `KindMismatch` token in the ctt.3 retirement marker - (`// ctt.3: KindMismatch retired — ...`), but Task 2 Step 1's - verification text says "Expected: zero hits in `crates/`". The - parent spec at `docs/specs/2026-05-12-ct-tidy.md:384` is more - permissive ("`grep KindMismatch` returns the three deletion sites - only" — accepting the diff itself as the only mention, but - silent on a successor archaeology marker). Substantive intent - (retire the diagnostic) is met: variant, helper, dispatch, and - Display arm are all gone. Treated as `compliant` with this - concern recorded; Boss may choose to either tighten the - spec/plan rule for future tidies or accept the retirement-marker - pattern (which is already common in the codebase — cf. ctt.1 - successor-rationale comments in workspace.rs). -- Mid-deletion repair on workspace.rs:773 — the dispatch-loop - deletion in Step 2 orphaned the `mod_name` binding that - destructured `for (mod_name, m) in modules`. Plan Step 7's - acceptance criterion is "no dead-code warnings" — left as - `for (mod_name, m)` the build would warn `unused variable: - mod_name`. Minimal cleanup applied: rewrote to - `for m in modules.values()`. Same body, same `m: &Module` type - via `.values()` instead of tuple-destructure. Not in the plan's - verbatim edit list but a foreseeable side-effect of the - deletion and entirely mechanical. - -## Known debt - -- None new. The "future tidy may retire" promise in the deleted - doc-comment is now this iter; no successor obligation. - -## Files touched - -- `crates/ailang-core/src/workspace.rs` (variant delete + doc adjust + dispatch-loop delete + helper delete + test-doc delete + `mod_name` binding cleanup) -- `crates/ail/src/main.rs` (Display arm delete) -- `crates/ailang-check/src/lib.rs` (archaeology comment edit) - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-ctt.3.json diff --git a/docs/journals/2026-05-12-iter-eob.1.md b/docs/journals/2026-05-12-iter-eob.1.md deleted file mode 100644 index 44aa0a1..0000000 --- a/docs/journals/2026-05-12-iter-eob.1.md +++ /dev/null @@ -1,123 +0,0 @@ -# iter eob.1 — Effect-op args walked as Borrow; heap-Str RC discipline closes - -**Date:** 2026-05-12 -**Started from:** b5b0c2d7dcd8bec1819801257de7731edc542e0f -**Status:** DONE -**Tasks completed:** 7 of 7 - -## Summary - -Land the language rule that `Term::Do.args[*]` are walked in -`Position::Borrow` by the uniqueness and linearity passes, plus the -two heap-Str-specific shape fixes (`ret_mode: Own` for `int_to_str` -/ `float_to_str`; Str carve-out in the App arm of -`drop_symbol_for_binder`) that make the existing RED tests at -`crates/ail/tests/e2e.rs` flip to GREEN under `--alloc=rc`. Both -arg-position rules (Ctor = Consume, Do = Borrow) are anchored in -DESIGN.md §Decision 10. The heap-str-abi milestone closes here: -WhatsNew entry shipped, roadmap P1 entry checked off, downstream -Post-22-Prelude `depends on:` line removed. - -The five plan layers landed first-shot in lockstep — no review -re-loops fired on any of the 7 tasks. The pre-existing RED tests -(`int_to_str_drop_balances_rc_stats`, -`str_field_in_adt_drops_heap_str_correctly`) flipped to GREEN with -the predicted concrete numbers (`allocs == 1, frees == 1, live == 0` -and `allocs == 2, frees == 2, live == 0`). Two new positive tests -pin the rule's coverage at the primitive boundary (Int through -`io/print_int`: zero RC traffic) and at the linearity boundary -(heap-Str through two sequential `io/print_str` calls: allocs == 1, -frees == 1, live == 0). Workspace test --workspace + cross_lang.py -+ compile_check.py all green. check.py shows the same noisy -latency-max + bump_s cluster the audit-cma and audit-ms journals -both left pristine ("not coupled to milestone, baseline left -untouched for second consecutive audit") — same precedent applies -here; baseline left as-is. - -## Per-task notes - -- iter eob.1.1: Term::Do arg-walks flip Consume → Borrow in both - uniqueness.rs:289 and linearity.rs:506; linearity.rs:42 doc - comment split into two bullets matching the new code shape. - Inline clarifying comment added on the linearity-side flip - mirroring the Ctor-arm convention. cargo build + workspace - check-crate tests green. -- iter eob.1.2: builtins.rs `float_to_str` (:200) + `int_to_str` - (:210) and synth.rs `float_to_str` (:172) + `int_to_str` (:179) - all flip `ret_mode: Implicit` → `ret_mode: Own` in lockstep. - Workspace build clean. Target test now reports the predicted - link-error shape "undefined value @drop_int_to_str_drop_rc_Str" - — exactly the gap Task 3 closes. -- iter eob.1.3: drop.rs App arm of `drop_symbol_for_binder` gets - the Str carve-out `if name == "Str" return ailang_rc_dec`, - symmetric to `field_drop_call`'s existing Str arm. The - comment block mirrors `field_drop_call`'s wording. cargo build - clean. -- iter eob.1.4 (observational): both RED tests flip to GREEN with - the predicted concrete numbers (allocs == 1, frees == 1 / allocs - == 2, frees == 2; live == 0 on both). Full e2e suite (79 tests) - all green — no other fixture's RC trace shifted unexpectedly. -- iter eob.1.5: two new fixtures - (`examples/int_to_print_int_borrow.ail.json`, - `examples/heap_str_repeated_print_borrow.ail.json`) + two new - e2e tests (`int_arg_to_effect_op_does_not_rc_track`, - `heap_str_repeated_print_balances_rc_stats`) appended near - e2e.rs:2625. Both new tests PASS first-shot with exact predicted - RC stats. -- iter eob.1.6: DESIGN.md §Decision 10 gets a new subsection - `#### Arg-position policy for compound AST nodes` after the - `ret_mode` block. The two arg-position rules (Ctor = Consume, - Do = Borrow) are anchored together as language rules, not - per-op annotations. One added linking paragraph explains how - the Do = Borrow rule cooperates with the `ret_mode == Own` - letbinder-trackability rule above to balance heap-Str RC - discipline — within the plan's "Boss judgement on placement" - carve-out. -- iter eob.1.7: WhatsNew.md entry appended at bottom (user-facing - wording, no internals). roadmap.md heap-Str ABI entry flipped - `[ ]` → `[x]`. Post-22 Prelude `depends on:` line removed (the - only dep was Heap-Str, which has shipped). cargo test workspace - + cross_lang.py + compile_check.py all green. check.py noisy - latency cluster left pristine per audit-cma/audit-ms precedent. - -## Concerns - -(none — all 7 tasks DONE without review re-loops) - -## Known debt - -- check.py shows recurring noisy `latency.implicit_at_rc.max_us` - / `latency.explicit_at_rc.max_us` tail-jitter, plus - `throughput.bench_list_sum.bump_s` swinging +11-15% across - back-to-back runs (with reciprocal `gc_over_bump` improvement - on the same fixture). Neither family is reachable from this - iter's changes (bump_s on bench_list_sum has no effect-ops; the - list-sum hot path has no int_to_str / float_to_str calls). The - audit-cma and audit-ms journals already document the same - noise cluster as "not coupled to milestone, baseline left - pristine for two consecutive audits" — same precedent applies; - baseline untouched. - -## Files touched - -Code: -- crates/ailang-check/src/uniqueness.rs (Term::Do Consume → Borrow) -- crates/ailang-check/src/linearity.rs (Term::Do Consume → Borrow + doc-block split) -- crates/ailang-check/src/builtins.rs (int_to_str / float_to_str ret_mode → Own) -- crates/ailang-codegen/src/synth.rs (int_to_str / float_to_str ret_mode → Own) -- crates/ailang-codegen/src/drop.rs (App-arm Str carve-out) - -Tests + fixtures: -- crates/ail/tests/e2e.rs (2 new positive tests appended) -- examples/int_to_print_int_borrow.ail.json (new) -- examples/heap_str_repeated_print_borrow.ail.json (new) - -Docs: -- docs/DESIGN.md (Decision 10: arg-position policy subsection) -- docs/WhatsNew.md (heap-str-abi milestone close entry) -- docs/roadmap.md (Heap-Str ABI P1 closed; Post-22 Prelude - depends-on line removed) - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-eob.1.json diff --git a/docs/journals/2026-05-12-iter-ext-cli.1.md b/docs/journals/2026-05-12-iter-ext-cli.1.md deleted file mode 100644 index ccadd5f..0000000 --- a/docs/journals/2026-05-12-iter-ext-cli.1.md +++ /dev/null @@ -1,137 +0,0 @@ -# iter ext-cli.1 — CLI accepts `.ail` (Form A) source files alongside `.ail.json` - -**Date:** 2026-05-12 -**Started from:** efecbaa -**Status:** DONE -**Tasks completed:** 4 of 4 - -## Summary - -Closes the loop opened by this morning's `.ailx → .ail` rename. Every -path-taking `ail` subcommand (`check`, `build`, `run`, `manifest`, -`render`, `prose`, `merge-prose`, `describe`, `emit-ir`, `diff`, -`workspace`, `deps`) now accepts a `.ail` (Form A) input as well as -`.ail.json` (Form B). For `.ail` paths the subcommand parses through -`ailang_surface::parse` in-line and proceeds with the same loaded -`Module` value `.ail.json` would have produced; the binary semantics -are extension-agnostic. - -Architecture: a new `ailang_surface::{load_module, load_workspace}` -pair dispatches on file extension. A new injection point in -`ailang_core::workspace::load_workspace_with` lets the surface -crate plug in an extension-aware loader without `ailang-core` having -to import `ailang-surface` (which would close a crate cycle). Import -resolution in `workspace::visit` now prefers a `.ail` sibling over a -`.ail.json` sibling of the same module name. The CLI binary's 18 -`ailang_core::load_*` callsites all switched to the new -`ailang_surface::load_*`. Surface parse failures route through the -existing `workspace_error_to_diagnostic` channel as a new -`SurfaceParse` variant of `WorkspaceLoadError`, so -`ail check --json foo.ail` on a malformed `.ail` returns a parseable -JSON diagnostic with code `surface_parse_error` instead of the -pre-iter misleading `json: expected value at line 1 column 1` -fall-through. - -## Per-task notes - -- iter ext-cli.1.1: `WorkspaceLoadError::SurfaceParse { path, - message }` variant added to `ailang_core::workspace`; - `load_workspace` refactored into a thin wrapper around a new - `pub fn load_workspace_with(entry, loader: F)` with `F: - Fn(&Path) -> Result + Copy`; `visit`'s - signature gained the loader parameter and its import-path - construction now does `.ail`-first / `.ail.json`-fallback; new - test `load_workspace_with_custom_loader_is_called` pins the - injection. `tempfile` added as `ailang-core` dev-dependency. -- iter ext-cli.1.2: new `crates/ailang-surface/src/loader.rs` with - `pub fn load_module` (extension-dispatching: `.ail` → `parse`, - else delegate to `ailang_core::load_module`) and - `pub fn load_workspace` (calls `load_workspace_with(entry, - load_module)`); `lib.rs` re-exports both. Three integration tests - in `crates/ailang-surface/tests/loader.rs` pin the dispatch on - each form individually + the mixed-extension workspace case. - `tempfile` added as `ailang-surface` dev-dependency. -- iter ext-cli.1.3: every `ailang_core::load_module` / - `ailang_core::load_workspace` callsite in `crates/ail/src/main.rs` - (18 sites total) replaced with `ailang_surface::load_module` / - `ailang_surface::load_workspace`. Workspace-build placeholder - `W::SurfaceParse { .. } => None` arm added to - `workspace_error_to_diagnostic` (replaced in Task 4) so the - exhaustive match compiles. Two new E2E tests in - `crates/ail/tests/e2e.rs`: `ail_check_accepts_ail_source` and - `ail_run_accepts_ail_source_with_same_stdout_as_ail_json`. -- iter ext-cli.1.4: placeholder arm replaced with proper - `W::SurfaceParse { path, message } => Some(Diagnostic::error( - "surface_parse_error", message).with_ctx({"path": ...}))` — - mirroring the `SchemaMismatch` arm's builder pattern. DESIGN.md - §Decision 6 / "CLI" bullet gained the one-paragraph addendum - (verbatim from the plan, dated). New ct1 test - `ail_check_json_on_ail_with_syntax_error_returns_structured_diagnostic` - pins the JSON-diagnostic shape. - -Plus one E2E coverage test added during Phase 3: -`load_workspace_prefers_ail_when_both_extensions_exist` — -protects the `.ail`-first precedence when both extensions are -present for the same module (the existing 1.2 test only exercised -the fallback half). - -## Concerns - -- **Diagnostic code convention drift (Boss-corrected pre-commit).** - The plan-specified code `surface_parse_error` used snake_case; - every other diagnostic code in the codebase is kebab-case - (`type-mismatch`, `unbound-var`, `over-strict-mode`, - `reuse-as-shape-mismatch`, ...). Boss verified the convention - via `git grep` over `crates/ailang-check/src` and - `crates/ail/src` before commit and aligned the literal to - `surface-parse-error` at three sites: the `SurfaceParse` arm - in `workspace_error_to_diagnostic` (`crates/ail/src/main.rs`) - and the two matching strings in - `crates/ail/tests/ct1_check_cli.rs` (the rustdoc on the test + - the `assert_eq!` against `first["code"]`). The plan file is - left at the snake_case form as historical record; the source - of truth is now the corrected code. -- **bench/check.py first run noisy.** First execution showed 4 - regressions on `latency.{explicit,implicit}_at_rc.{p99_9,max}_us` - (tail-latency tier). Re-runs (×2) returned 0 regressed - (one even showed 4 improvements beyond tolerance). This is the - same `latency.explicit_at_rc.*` cluster audit-cma and audit-ms - already documented as nondeterministic; the iter's actual code - change introduces ONE extra `surface_path.is_file()` syscall per - import-resolution step plus a single-byte extension check per - load, neither of which can move us-scale tail latencies. Net: - baseline left pristine, consistent with the last two audits. - -## Known debt - -- The `ail parse` subcommand is still the only way to materialise - a stable `.ail.json` snapshot from a `.ail` source. Now that - every subcommand silently dispatches `.ail` through the parser, - one might argue for a "did you mean `ail parse`?" hint on paths - with non-`.ail`/non-`.ail.json` extensions. The roadmap note for - ext-cli.1 mentions this as a possible same-iter or follow-up - scope; nothing surfaced as load-bearing during impl, deferring. - -## Files touched - -Code: -- `crates/ailang-core/Cargo.toml` (+ tempfile dev-dep) -- `crates/ailang-core/src/workspace.rs` (SurfaceParse variant, - load_workspace_with, visit threading, .ail-first precedence, - new unit test) -- `crates/ailang-surface/Cargo.toml` (+ tempfile dev-dep) -- `crates/ailang-surface/src/lib.rs` (mod loader + re-exports) -- `crates/ailang-surface/src/loader.rs` (NEW) -- `crates/ailang-surface/tests/loader.rs` (NEW; 4 tests) -- `crates/ail/src/main.rs` (18 callsite rewires + SurfaceParse - diagnostic arm) -- `crates/ail/tests/e2e.rs` (2 new E2E tests) -- `crates/ail/tests/ct1_check_cli.rs` (1 new diagnostic-shape test) - -Spec / metadata: -- `docs/DESIGN.md` (§Decision 6 / CLI bullet addendum) -- `Cargo.lock` (tempfile dev-dep transitive) - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-ext-cli.1.json diff --git a/docs/journals/2026-05-12-iter-ext-rename.md b/docs/journals/2026-05-12-iter-ext-rename.md deleted file mode 100644 index 31b2bb4..0000000 --- a/docs/journals/2026-05-12-iter-ext-rename.md +++ /dev/null @@ -1,107 +0,0 @@ -# iter ext-rename — `.ailx` extension renamed to `.ail` across the live toolchain - -**Date:** 2026-05-12 -**Started from:** 17b370b (post-audit-ms close, working tree clean) -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -The surface-form file extension changes from `.ailx` to `.ail`. -AILang's authoring surface now uses the same `.ail` stem as its -canonical JSON form (`.ail.json`), giving the language a single -coherent extension family: `.ail` is the LLM-authored Form A; -`.ail.json` is the canonical JSON-AST Form B that -`ail check` / `build` / `run` actually loads today. - -The earlier `.ailx` extension carried an unmotivated `x`. The -cross-model authoring-form test (cma + ms milestones, closed -earlier today) treated `.ailx` purely as a cohort identifier; -once the empirical case for keeping a separate authoring surface -had been ratified (DESIGN.md §Decision-6 addendum), the awkward -extension was the last asymmetry between Form A and Form B. -User-initiated rename. - -## Scope - -In scope (touched): - -- 61 example files `examples/**/*.ailx → .ail` (57 top-level + - 4 `examples/fieldtest/`) renamed via `git mv`. -- `experiments/.../rendered/ailx.md → ail.md` rename. -- All live toolchain prose and code: `crates/ail/`, - `crates/ailang-surface/`, `crates/ailang-core/specs/form_a.md`, - `docs/DESIGN.md`, `docs/roadmap.md`, `docs/PROSE_ROUNDTRIP.md`, - `bench/reference/*.c`, all of `skills/`, the Cross-Model - Authoring experiment crates under - `experiments/.../{render,harness,master,rendered,README.md}`. -- Cohort identifier `ailx → ail` in the experiment crate (Rust - `Cohort::Ailx → Cohort::Ail`, `Form::Ailx → Form::Ail`, - per-cohort path component, `{form-only: ailx}` marker, - ```ailx codefence language tag). - -Out of scope (untouched, by design): - -- `docs/journal-archive.md` (content-frozen per CLAUDE.md). -- All `docs/journals/`, `docs/specs/`, `docs/plans/`, - `bench/orchestrator-stats/` — historical records describing - states as they were. -- `experiments/.../runs/` — frozen LLM-output artefacts; the - models in those runs actually saw `.ailx`, and renaming the - artefacts would falsify the experimental record. - -The exclusion preserves the journals as honest history. Anyone -chasing why a 2026-05-11 fieldtest names `.ailx` files will find -that the renaming happened today and that the names were correct -at the time of writing. - -## Why boss-direct edit (no planner / implement dispatch) - -Same rationale as iter rt.2: the user fully specified the -transformation (rename `.ailx → .ail`, rename cohort -`ailx → ail`, historical records untouched), down to the three -open questions that were resolved up-front via AskUserQuestion -(cohort name; treatment of frozen runs; treatment of historical -docs). The execution was two `git mv` passes plus one -heavily-scoped `sed` over an explicit 41-file whitelist (plus -two follow-up files surfaced by the negative-grep). No design -judgement deferred to execution time. - -## Files touched (working-tree counts) - -- 61 renames under `examples/`. -- 1 rename `experiments/.../rendered/ailx.md → ail.md`. -- 35 content-edited files: the 41-file SCOPE whitelist minus - the experiment-crate files that had no `ailx` content, plus - two follow-up files surfaced by the negative-grep - (`examples/bench_list_sum_explicit.ail`'s in-source comment; - `experiments/.../rendered/json.md` for three `AILX` acronyms - in prose). - -## Verification - -- `cargo build --workspace` clean. -- `cargo test --workspace` green; the renamed identifiers - `every_ail_fixture_matches_its_json_counterpart` and - `parse_then_print_then_parse_is_idempotent_on_every_ail_fixture` - in `crates/ailang-surface/tests/round_trip.rs` were run - explicitly and pass. -- Experiment crate `cargo test` (out-of-workspace) green. -- `bench/check.py`, `bench/compile_check.py`, `bench/cross_lang.py`: - 0 regressed across all three; 6 improved-beyond-tolerance on - `latency.explicit_at_rc` (same cluster the previous two audits - today already observed; decoupled from this rename — baseline - left pristine). -- Negative grep: `git grep -nIi 'ailx|Ailx|AILX'` outside the - out-of-scope paths returns no matches. - -## Roadmap implications - -The `.ail` extension is now canonical authoring surface but the -CLI does not yet accept it — `ail check foo.ail` still produces -the misleading JSON-parse error. The existing P2 todo -"`ail check`/`build`/`run` accept `.ail` extension" (roadmap.md -line 185) is the natural next iteration: it closes the loop -opened by today's rename and removes the largest first-command -friction for any LLM-author writing in Form A. Promoting that -todo to the immediate next dispatch. diff --git a/docs/journals/2026-05-12-iter-hs.1.md b/docs/journals/2026-05-12-iter-hs.1.md deleted file mode 100644 index 0b66fe9..0000000 --- a/docs/journals/2026-05-12-iter-hs.1.md +++ /dev/null @@ -1,118 +0,0 @@ -# iter hs.1 — Heap-Str ABI: Static-Str layout migration - -**Date:** 2026-05-12 -**Started from:** 69bb5f9952fb51af7cdf4e36f77cff1b6cd73c40 -**Status:** DONE -**Tasks completed:** 5 of 5 - -## Summary - -First iter of the heap-Str ABI milestone. Pure codegen-side refactor: -language `Str` literals now emit as packed-struct LLVM globals -`<{ i64, i64, [N+1 x i8] }> <{ i64 -1, i64 N, [N+1 x i8] c"...\00" }>` -carrying a `UINT64_MAX` sentinel rc-header in the first slot, an -explicit byte length in the second, and the bytes plus trailing NUL -in the array tail. The IR-Str-pointer convention is now uniform: a -language `Str` value flowing through the IR is a `ptr` landing on the -`len`-field (offset +8 from the header, -8 from the bytes). Every -codegen site that hands a `Str` pointer to a `NUL`-terminated-bytes -C-API consumer emits a `getelementptr inbounds i8, ptr , i64 8` -immediately before the call. The sentinel slot is hardcoded in the -global initialiser and not yet observed by any caller — `hs.2` will -add the `UINT64_MAX` short-circuit to `ailang_rc_inc` / -`ailang_rc_dec` in `runtime/rc.c` to exploit it. Format strings -(`%lld\n`, `%g\n`, `true\n`, `false\n`) are unchanged — they are -runtime-internal scaffolding, never flow as language `Str` values, -and remain in the raw `[N x i8]` shape via the original -`intern_string` / `all_strings` path. Acceptance verified -end-to-end: full `cargo test --workspace` green (no regressions -across 74 e2e fixtures), `bench/compile_check.py` exit 0, -`bench/cross_lang.py` exit 0 (byte-identical stdout across all -ail/c benchmark pairs). - -## Per-task notes - -- iter hs.1.1: introduced parallel `str_literals` interning table on - `Emitter`, `intern_str_literal` fn, workspace-level - `all_str_literals` aggregation, and a parallel global-emission - loop emitting the packed-struct shape. Switched both - `Literal::Str` arms (`emit_const_def`, `lower_term`) to call - `intern_str_literal` and materialise a constexpr - `getelementptr inbounds (<{ i64, i64, [N x i8] }>, ptr @g, i32 0, i32 1)` - landing on the `len`-field. Two pinning tests added. -- iter hs.1.2: `io/print_str` arm now emits a `+8 GEP` to land on - the bytes pointer before calling `@puts`. One pinning test added. -- iter hs.1.3: `eq__Str` arm of `try_emit_primitive_instance_body` - now emits `+8 GEP`s on both operands before calling - `@ail_str_eq`. One pinning test added. -- iter hs.1.4: `compare__Str` arm of `try_emit_primitive_instance_body` - now emits `+8 GEP`s on both operands before calling - `@ail_str_compare`. One pinning test added. -- iter hs.1.5: regenerated `crates/ail/tests/snapshots/hello.ll` - via `UPDATE_SNAPSHOTS=1`. Full `cargo test --workspace` revealed - a regression in `eq_demo` traced to a fourth codegen site not - enumerated in the plan: `lower_eq`'s `"Str"` arm calls `@strcmp` - inline (separate from the `eq__Str` instance-method intercept) - to lower `==` on `Str` literals and Str-pattern desugaring in - `classify_str`. Fixed inline with the same `+8 GEP` shape; one - additional pinning test added. - -## Concerns - -- The plan claimed three codegen sites pass a `Str` pointer to a - `NUL`-terminated-bytes C-API consumer (`@puts`, `@ail_str_eq`, - `@ail_str_compare`); in fact a fourth site exists in - `lower_eq`'s `"Str"` arm, calling `@strcmp` directly. This is - the dispatch path for the surface-level `==` operator on - `Str` and for `(pat-lit "...")` desugar against a `Str` - scrutinee. The fourth site is required to meet the iter's - stated acceptance criterion (byte-identical stdout); fixing it - was discovered via the e2e regression in `eq_demo` rather than - via the per-task IR-shape checks. The fix has the same shape - as Tasks 2-4 and the corresponding pinning test - (`lower_eq_str_calls_strcmp_with_bytes_pointer`) is added. The - plan's "Files this plan creates or modifies" enumeration - should list `crates/ailang-codegen/src/lib.rs:2520-2529` as a - fifth modify site for any future re-execution. -- The plan's two test fixtures in Task 1 contained schema-syntax - errors that prevented compilation: `ConstDef.body` (actual: - `value`), `Term::App { f, args }` (actual: `callee, args, tail`), - and `Term::App { Term::Var "io/print_str", ... }` for what is - actually a `Term::Do` effect op. Tests were adapted to compile - while preserving the substantive assertions verbatim. Same - applies to the test in Task 2 (`Term::App` → `Term::Do` for the - `io/print_str` invocation and added `effects: ["IO"]` on the - main fn so the typecheck-side dispatch finds the effect). - These were small per-test fixture adaptations, not changes to - the asserted IR shape. - -## Known debt - -- The sentinel value `-1` (`UINT64_MAX`) is hardcoded in the - global initialiser. `hs.2` adds the matching short-circuit - check in `ailang_rc_inc` / `ailang_rc_dec` so static-Str - pointers can flow through generic RC paths without - ref-count traffic. Until `hs.2` lands, no caller actually - observes the sentinel — static `Str` slabs are still - invisible to the RC runtime by construction (the call sites - for `inc` / `dec` against `Str` values are still gated by - the same uniqueness-inference pass that elided them - pre-iter). - -## Files touched - -- `crates/ailang-codegen/src/lib.rs` — Emitter field, intern fn, - workspace aggregation, global emission, two `Literal::Str` arm - switches, four C-API-callsite `+8 GEP` adjustments - (`io/print_str`, `eq__Str`, `compare__Str`, `lower_eq` Str arm), - six new IR-shape pinning tests in the test module. -- `crates/ail/tests/snapshots/hello.ll` — regenerated snapshot - showing the new packed-struct global and the `+8 GEP` before - `@puts`. No other snapshot under - `crates/ail/tests/snapshots/` changed (other snapshots contain - only format-string globals which kept the raw `[N x i8]` - shape). - -## Stats - -`bench/orchestrator-stats/2026-05-12-iter-hs.1.json` diff --git a/docs/journals/2026-05-12-iter-hs.2.md b/docs/journals/2026-05-12-iter-hs.2.md deleted file mode 100644 index 6685961..0000000 --- a/docs/journals/2026-05-12-iter-hs.2.md +++ /dev/null @@ -1,81 +0,0 @@ -# iter hs.2 — Static-Str sentinel-slot retrofit - -**Date:** 2026-05-12 -**Started from:** 51a096cf8354362783d8caf2ec4b35179d935a7a -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -Second iter of the heap-Str ABI milestone. Surgical codegen-side -retrofit of hs.1's static-`Str` global layout: per the amended parent -spec (`docs/specs/2026-05-12-heap-str-abi.md`) the sentinel rc-header -slot drops. Language `Str` literals now emit as -`<{ i64, [N+1 x i8] }> <{ i64 N, [N+1 x i8] c"…\00" }>` instead of -the hs.1 shape `<{ i64, i64, [N+1 x i8] }> <{ i64 -1, i64 N, …}>`. The -explicit `len` field becomes the first (and only non-byte) field on -which the IR-`Str` pointer lands via constexpr-GEP `i32 0, i32 0`. The -`+8 GEP` consumer-side calls (`@puts`, `@ail_str_eq`, `@ail_str_compare`, -inline `@strcmp`) do **not** change — the bytes still sit 8 bytes past -the `len`-field. Static-Str pointers are kept out of `ailang_rc_dec` -at the codegen level via move-tracking (iter 18d.3) and non-escape -lowering (iter 18b), so no header slot is needed at the global. - -Change surface: three format-string sites and two doc-comments in -`crates/ailang-codegen/src/lib.rs`, rename + assertion update of two -IR-shape tests in the same crate's test module, plus snapshot -regeneration of `crates/ail/tests/snapshots/hello.ll`. No runtime -changes, no checker changes, no new fixtures. Static-Str globals are -now 8 bytes smaller per literal. - -Acceptance verified end-to-end: `cargo test --workspace` green (every -test result `ok`, 0 failures across the workspace), `bench/cross_lang.py` -exit 0 (byte-identical stdout across all `ail/c` benchmark pairs), -`bench/compile_check.py` exit 0, `bench/check.py` exit 0 within the -known `latency.explicit_at_rc.*` nondeterminism tolerance. - -## Per-task notes - -- iter hs.2.1: renamed `static_str_global_uses_packed_struct_with_sentinel_and_len` - → `static_str_global_uses_packed_struct_with_len`; updated its - doc-comment and assertion substring (struct loses leading `i64, `, - initialiser loses leading `i64 -1, `). Updated assertion in - `static_str_callsite_pointer_is_payload_via_constexpr_gep` (name - preserved — "payload" still describes the `len`-field-as-pointer-target; - struct loses leading `i64, `, GEP indices `i32 0, i32 1` → `i32 0, i32 0`). - RED confirmed: both tests fail against hs.1 emission with the expected - assertion-substring mismatch. The four `+8 GEP` tests stayed green. -- iter hs.2.2: flipped the global-emission loop format string at - `lib.rs:471` to the 2-field shape; flipped both `Literal::Str` arms - (`emit_const_def` ~ `lib.rs:933`, `lower_term` ~ `lib.rs:1255`) to - emit constexpr-GEP `i32 0, i32 0` onto the 2-field struct type; - updated the two doc-comments on the `str_literals` field - (`lib.rs:561`) and on `intern_str_literal` (`lib.rs:2596`) to the - amended-spec wording. All five edits exactly per plan text. Static_str_ - tests GREEN, `_calls_` tests GREEN, full `cargo test -p ailang-codegen` - GREEN (33 tests). -- iter hs.2.3: regenerated `crates/ail/tests/snapshots/hello.ll` via - `UPDATE_SNAPSHOTS=1 cargo test -p ail --test ir_snapshot hello`. The - two affected lines (the `@.str_hello_str_0` global and its constexpr- - GEP at `%v1`) flipped to the new shape; the outer `i64 8` byte offset - preserved. Workspace `cargo test --workspace` green; `cross_lang.py`, - `compile_check.py`, `check.py` all green. - -## Concerns - -(none) - -## Known debt - -(none) - -## Files touched - -- `crates/ailang-codegen/src/lib.rs` — three format strings, two - doc-comments, two test names + assertions + doc-comments -- `crates/ail/tests/snapshots/hello.ll` — regenerated snapshot - (2 lines content change, byte offsets preserved) - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-hs.2.json diff --git a/docs/journals/2026-05-12-iter-hs.3.md b/docs/journals/2026-05-12-iter-hs.3.md deleted file mode 100644 index 26904d4..0000000 --- a/docs/journals/2026-05-12-iter-hs.3.md +++ /dev/null @@ -1,111 +0,0 @@ -# iter hs.3 — Heap-Str runtime additions - -**Date:** 2026-05-12 -**Started from:** 187d04814d4485bee1bb85f6f7433c2f3d146fb7 -**Status:** DONE -**Tasks completed:** 2 of 2 - -## Summary - -Third iter of the heap-Str ABI milestone. Single-file additive change -to `runtime/str.c`: three new symbols (private `static char -*str_alloc(uint64_t)` slab helper plus two extern formatters -`ailang_int_to_str(int64_t)` and `ailang_float_to_str(double)`), the -supporting `` / `` / `` includes, and an -`extern void *ailang_rc_alloc(size_t)` declaration. No IR-side caller -is wired yet — hs.4 will land codegen + checker + linker work that -makes the formatters live builtins. - -One deviation from the plan: the `extern` declaration of -`ailang_rc_alloc` was promoted from plain-external to -`__attribute__((weak))`. The plan's assertion that the new symbols -sit "present-but-unreferenced" across all `--alloc` strategies turned -out to be wrong at the link layer — `ailang_int_to_str` / -`ailang_float_to_str` are publicly-visible `T` symbols, so the linker -preserves them and their transitive callees (including -`ailang_rc_alloc`) in every binary regardless of program-level usage. -Under `--alloc=gc` and `--alloc=bump` the build pipeline does not -link `rc.c`, so the unresolved-symbol error fires at link time and -took out 11 e2e tests on the first regression sweep. The weak -attribute makes the reference link-safe in isolation: under gc/bump -it resolves to NULL (never dereferenced — no IR caller exists yet); -under rc the strong definition wins as usual. The attribute is -retained after hs.4 unconditionally links rc.c (where it becomes a -no-op) to keep `runtime/str.c` link-safe in isolation, matching the -spec's "single-file abstraction" framing of the runtime split. - -Acceptance verified: `cargo test --workspace` (zero failures across -the full test surface), `bench/cross_lang.py` (25/25 metrics stable, -exit 0), `bench/compile_check.py` (24/24 metrics stable, exit 0), -`bench/check.py` (zero regressed, exit 0; same five -`latency.explicit_at_rc.*` improvements outside tolerance that -audit-cma and audit-ms have already documented as -not-coupled-to-milestone — baseline left untouched per established -practice). `clang -c -O2 -Wall -Werror` on `runtime/str.c` standalone -also exits 0 with no warnings. - -## Per-task notes - -- iter hs.3.1: appended three `#include` lines (`stdint.h`, - `stdio.h`, `stdlib.h`), one weak `extern void *ailang_rc_alloc` - declaration, and three new function bodies (`str_alloc`, - `ailang_int_to_str`, `ailang_float_to_str`) to `runtime/str.c`. - Compile-check via `clang -c -O2 -Wall -Werror` green; nm - symbol-table inspection: 4 public `T` symbols (`ail_str_eq`, - `ail_str_compare`, `ailang_int_to_str`, `ailang_float_to_str`), - `str_alloc` absent at `-O2` (inlined into both callers — only - visible as `t str_alloc` at `-O0`; static-linkage invariant - preserved), `ailang_rc_alloc` shown as `w` (weak undefined). -- iter hs.3.2: ran the four regression gates. First-pass sweep - surfaced the 11-test linker failure under `--alloc=gc`/`bump` - caused by the plan's "rc_alloc reference is harmless if - unreferenced" assumption; weak-attribute repair re-ran clean. All - four gates green on the re-sweep. - -## Concerns - -- The plan's Task 1 Step 7 nm-output expected-line block ("`U - ailang_rc_alloc`" and "`0000000000000000 t str_alloc`") is - inaccurate at the optimisation level the plan itself specifies - (`-O2`): the grep regex does not match `ailang_rc_alloc`, and - `clang -O2` inlines the 2-line `str_alloc` helper into both - callers so no `t str_alloc` symbol survives the compilation unit. - Both deviations are `-O2` consequences, not bugs — the - semantically meaningful invariants (`str_alloc` is static and - absent from `T`; `ailang_rc_alloc` is a non-`T` undefined - reference) both hold. A future plan author writing nm-output - checks should inspect at `-O0` for symbol presence and at `-O2` - for what actually ships, or relax the expected text from - literal-match to invariant-check (e.g. "no `T str_alloc`", - "`ailang_rc_alloc` is `U` or `w`"). -- The plan's core "no build-pipeline change in hs.3" boundary held, - but the supporting claim "symbols are present in every binary - regardless of allocator choice" was false at the link layer. The - weak-attribute repair makes it true at the cost of one extra - attribute and a long-form doc-comment explaining the cross- - allocator linking story. A future plan that defers a build- - pipeline change should explicitly call out cross-translation-unit - symbol dependencies as a gating concern. - -## Known debt - -- hs.4 will wire `ailang_int_to_str` / `ailang_float_to_str` from - the IR side and unconditionally link `rc.c`; at that point the - `__attribute__((weak))` on the `ailang_rc_alloc` declaration - becomes a no-op (strong definitions always satisfy weak - references). The attribute is intentionally retained — see the - doc-comment in `runtime/str.c` — to keep the translation unit - link-safe in isolation, matching the parent spec's framing of - `runtime/str.c` as the heap-Str ABI's single point of contact. - -## Files touched - -- `runtime/str.c` — three new functions appended after - `ail_str_compare`, plus three new `#include` lines and one weak - `extern` declaration after the existing include block. Existing - `ail_str_eq` / `ail_str_compare` bodies and the file's top - comment block untouched. - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-hs.3.json diff --git a/docs/journals/2026-05-12-iter-hs.4.md b/docs/journals/2026-05-12-iter-hs.4.md deleted file mode 100644 index 1ad3c9c..0000000 --- a/docs/journals/2026-05-12-iter-hs.4.md +++ /dev/null @@ -1,260 +0,0 @@ -# iter hs.4 — IR + checker + linker wiring for int_to_str / float_to_str - -**Date:** 2026-05-12 -**Started from:** 1f832c028a419665ecc7ddbcbdd5dd1a16d35e83 -**Status:** DONE_WITH_CONCERNS -**Tasks completed:** 4 of 4 - -## Summary - -Fourth iter of the heap-Str ABI milestone. Wires the hs.3 runtime -symbols `ailang_int_to_str` and `ailang_float_to_str` into live -AILang builtins observable at every layer of the toolchain. After -this iter `do io/print_str(int_to_str(42))` builds and prints -`"42\n"` — the originating acceptance goal of the milestone is met. -Four pipeline layers landed in lockstep: (1) checker installs -`int_to_str : (Int) -> Str` next to the existing `float_to_str`, -synth.rs gets the parallel arm; (2) codegen IR-header unconditionally -declares both runtime externs, `Emitter::lower_app` gains an -`int_to_str` arm and replaces the `CodegenError::Internal` body of -`float_to_str` with the actual call emission, `is_static_callee`'s -builtin-callee whitelist extends to recognise `int_to_str`; (3) the -build pipeline hoists `runtime/rc.c`'s clang-compile-and-link out of -the `AllocStrategy::Rc` arm into the unconditional path next to -`runtime/str.c` (the `__attribute__((weak))` on str.c's -`ailang_rc_alloc` extern becomes the documented no-op the hs.3 -journal anticipated); (4) two IR-shape tests pin the new lowerings, -four E2E tests cover stdout + RC-stats, four `.ail.json` fixtures -exercise them, the stale Str-arm comment in `drop.rs`'s -`field_drop_call` is refreshed to the dual-realisation framing. - -One internal contradiction in the plan surfaced during Task 4 -fixture validation: the plan's literal `ret_mode: Implicit` for -both `int_to_str` and `float_to_str` (Task 1 Step 1 code block, -copied verbatim from the pre-hs.4 `float_to_str` stub) prevents the -plan's literal RC-stats assertion `allocs == frees && live == 0` -(Task 4 Step 5) from holding — under Implicit ret-mode the -let-binder for an `App`-shape value is not trackable in -`is_rc_heap_allocated` (drop.rs:464-475), so no scope-close -`ailang_rc_dec` is emitted. Per the carrier's "make the reasonable -call and continue" mandate, shipped is the conservative form -(plan's literal `Implicit`) with weakened test assertions -(`allocs >= 1` instead of `allocs == frees && live == 0`) and the -substantive design call queued as known debt. See Concerns for the -full failure mode + the speculative-fix attempt + revert sequence. - -Acceptance verified: full `cargo test --workspace` green; the four -new E2E tests pass; the two new IR-shape pins pass; the five IR -snapshot tests pass after `UPDATE_SNAPSHOTS=1` regen (the new -unconditional `declare ptr @ailang_int_to_str(i64)` / `declare ptr -@ailang_float_to_str(double)` lines now appear in every snapshot); -`bench/cross_lang.py` 25/25 stable; `bench/compile_check.py` 24/24 -stable; `bench/check.py` 0 regressed first run / 1 flapping -`bench_list_sum.bump_s` +10.51% vs 10.0% tolerance on second run, -matching the same noise-cluster pattern the hs.3 journal + audit-cma -audit-ms have documented as not-coupled-to-milestone, baseline left -pristine. Hello.ail builds and runs identically under all three -`--alloc` strategies post-rc.c-hoist. - -## Per-task notes - -- iter hs.4.1: installed `int_to_str : (Int) -> Str` in the checker - (builtins.rs:203-212, immediately after the existing `float_to_str` - entry); added the parallel `install_int_to_str_signature` unit - test (builtins.rs:461-466); added the lockstep arm in - `crates/ailang-codegen/src/synth.rs:174-180`. Both signature tests - PASS. Diff vs `float_to_str`: only the name and the param type - (`Type::int()` vs `Type::float()`) differ. `ret_mode: Implicit` kept - per the plan's literal text. -- iter hs.4.2: extended the IR-header preamble at - `crates/ailang-codegen/src/lib.rs:530-538` with two unconditional - `declare` lines for `@ailang_int_to_str(i64) -> ptr` and - `@ailang_float_to_str(double) -> ptr`; replaced the - `CodegenError::Internal` body of the `float_to_str` arm at - `lib.rs:1907-1925` with a `lower_term`+`fresh_ssa`+call-emission - triple matching the `int_to_float` arm's structural shape; added - the new `int_to_str` arm immediately above (lib.rs:1890-1906) so - both heap-Str formatters are co-located; extended the - `is_static_callee` builtin-callee whitelist at lib.rs:2124-2138 - with `"int_to_str"` (strictly required so `Term::Var { name: - "int_to_str" }` reaches lower_app's chain instead of the - UnknownVar fallback — the existing `float_to_str` entry was already - in the whitelist from iter 22-floats.3); appended the two IR-shape - tests at the bottom of `mod tests` (lib.rs:4089-4170), confirmed - RED-for-the-right-reason (UnknownVar then Internal-not-implemented) - before the lowering arms were wired, GREEN after; updated the - stale Str-arm comment in `drop.rs`'s `field_drop_call` at lines - 374-393 to the dual-realisation framing (heap-Str + static-Str - share consumer ABI; static-Str pointers kept out of `ailang_rc_dec` - via codegen-level move-tracking from iter 18d.3 + non-escape - lowering from iter 18b). Full `cargo test -p ailang-codegen` green - (35 tests). -- iter hs.4.3: hoisted the `runtime/rc.c` compile-and-link block out - of the `AllocStrategy::Rc` arm into the unconditional path next to - `runtime/str.c` at `crates/ail/src/main.rs:2294-2321`. The - `Rc`-arm body is now an empty doc-commented stub (retaining the - arm so `--alloc=rc` remains a valid CLI flag — it still drives - codegen's `@ailang_rc_alloc`-vs-`@GC_malloc` selection in the IR - header). Smoke-verified hello.ail builds + prints "Hello, AILang." - under all three alloc strategies. Five IR snapshot tests - regenerated via `UPDATE_SNAPSHOTS=1` to reflect the two new - unconditional `declare` lines added in Task 2; the snapshot diff - is exactly the two added lines (verified by inspecting hello.ll). - Full workspace re-sweep green afterwards. -- iter hs.4.4: created four fixtures (`int_to_str_smoke.ail.json`, - `float_to_str_smoke.ail.json`, `int_to_str_drop_rc.ail.json`, - `str_field_in_adt_heap.ail.json`) and appended four E2E tests at - the end of `crates/ail/tests/e2e.rs`. The float-literal JSON form - is `{"kind":"float","bits":"<16-char hex>"}` per the actual - `Literal::Float` schema (the plan's `value: ` - was a Rust-side description; the canonical wire form is the - hex-string per the `hex_u64` serde helper). 3.5_f64.to_bits() = - 0x400c000000000000. The two RC-stats tests assert - `allocs >= 1` / `allocs >= 2 && frees >= 1` instead of the plan's - literal `allocs == frees && live == 0` — the discovery + revert - + weakening sequence is in Concerns. Stdout pins: int → `"42\n"`, - float → `"3.5\n"` (libc %g, C locale default, glibc dev target). - All four E2E PASS. `bench/cross_lang.py` 25/25 stable; - `bench/compile_check.py` 24/24 stable; `bench/check.py` 0 - regressed first run, 1 flapping bench-noise second run within - documented variability. - -## Concerns - -- **Plan's literal assertions in Task 4 Step 5 cannot all hold - simultaneously given the plan's other choices.** The plan - installs `int_to_str` with `ret_mode: ailang_core::ast::ParamMode:: - Implicit` (Task 1 Step 1 code block), but `is_rc_heap_allocated` - at `crates/ailang-codegen/src/drop.rs:464-475` only fires on App - arms whose callee carries `ret_mode == Own`. With Implicit, the - let-binder `let s = int_to_str(42)` is not trackable, no - scope-close `ailang_rc_dec` is emitted, and the heap-Str slab - leaks. RC-stats observed on the plan-literal pipeline: - `int_to_str_drop_rc` produces `allocs=1 frees=0 live=1`; - `str_field_in_adt_heap` produces `allocs=2 frees=1 live=1` (the - inner heap-Str is freed via the ADT's match-arm pattern walk - through `field_drop_call`'s Str arm, but the outer ADT cell - leaks via the same Implicit-ret-mode story applied to its own - let-binder). The plan's literal Task 4 Step 5 asserts - `allocs == frees && live == 0` for both — over-strict given the - rest of the plan's literal choices. - - **Speculative-fix attempt + revert.** I tentatively switched both - `int_to_str` and `float_to_str` to `ret_mode: ParamMode::Own` in - both the checker (`builtins.rs`) and the codegen synth-arm - (`synth.rs`), and added a Str-primitive carve-out in - `drop_symbol_for_binder` at `drop.rs` (returning `ailang_rc_dec` - directly for `Type::Con { name: "Str" }` instead of the - `drop__Str` fallthrough that resolves to a non-existent - symbol). After rebuild the leak counts were unchanged. - Root cause is upstream: the uniqueness analyser at - `crates/ailang-check/src/uniqueness.rs:289-292` walks every - `Term::Do` arg in `Position::Consume`, so `consume_count` for - `s` in `do io/print_str(s)` is 1, and the let-arm dec at - `lib.rs:1440` is gated off by `consume_count == 0`. The - uniqueness analyser is treating an effect-op arg as a Consume - (transfer-of-ownership) when in fact `io/print_str` only reads - the pointer through libc's `@puts` and never dec's. Refining the - effect-op arg-mode story (do-args of pure-read effect-ops should - be Borrow for ptr-typed args) is a design call that goes beyond - hs.4's wiring scope. - - Reverted the three speculative edits to match the plan's literal - `Implicit` (more conservative, matches the existing `float_to_str` - signature convention). Weakened the test assertions to the - achievable invariant. Substantive fix is queued as known debt; - see below. - -- **bench/check.py flapping `bench_list_sum.bump_s` `+10.51%` on - the second run** (tolerance `10.0%`, overshoot 0.5pp). First run - was 0-regressed. The bump-allocator path was not touched by - hs.4. The hs.3 journal and audit-cma / audit-ms have documented - the same noise-cluster pattern (multi-metric +/- swings on - `latency.explicit_at_rc.*` and various `*.bump_s` / - `*.gc_over_bump` metrics) as not-coupled-to-milestone across the - past three iters + two audits, baseline left pristine. Treating - this as the same noise; baseline left pristine again. - -- **`drop_symbol_for_binder` Str-primitive fallthrough is wrong if - any future iter switches `int_to_str` to Own.** The current - fallback at `crates/ailang-codegen/src/drop.rs:534-549` for an - `App` whose ret-type is `Type::Con { name: "Str" }` (no dot) - produces `format!("drop_{m}_Str", m = self.module_name)` — - e.g. `drop_int_to_str_drop_rc_Str`, a non-existent symbol. - Currently dead code because Implicit-ret-mode gates the path off. - The known-debt item below covers the matched fix. - -## Known debt - -- **Refine the uniqueness-walker's effect-op arg-mode** so - `Term::Do` ptr-typed args of pure-read effect-ops (initially - `io/print_str` — only effect-op consuming ptr-typed Str) are - classified as Borrow rather than Consume. Pre-hs.4, no fixture - exercised the path "Own-returning App bound to a let, then - consumed by io/print_str" — `int_to_str`'s landing is the first - case. The wider question is which effect-ops carry which arg-modes - (most existing effect-ops take prim-typed args where the - consume/borrow distinction is moot; io/print_str + io/print_float - + io/print_bool + io/print_int all take by-value primitives; - io/print_str is the lone ptr-typed slot). A spec-level decision - is needed before the analyser change. - -- **Switch `int_to_str` / `float_to_str` to `ret_mode: Own` once the - uniqueness fix lands**, AND fix `drop_symbol_for_binder` to route - primitive-typed ret-cons (`Str` at a minimum) to `ailang_rc_dec` - shallow free — the speculative-fix attempt above showed both - pieces are needed in lockstep. Then strengthen the two RC-stats - test assertions from `allocs >= N` to `allocs == frees && live == 0` - per the plan's original Task 4 Step 5 intent. Either a follow-up - hs-tidy iter or a fresh hs.5+ slot. - -- **Single-value smokes only (per plan).** Edge cases for - `int_to_str` (0, -1, i64::MAX, i64::MIN) and `float_to_str` (NaN, - +Inf, -Inf, subnormals, exact-integer-doubles) are deferred to a - follow-up tidy iter if hs.4's regression sweep does not surface a - need. The current single-value smokes are sufficient to - demonstrate the codegen + runtime path is wired end-to-end. - -- **The `__attribute__((weak))` on `ailang_rc_alloc` in - `runtime/str.c`** is now a permanent no-op (strong definition - from rc.c always wins post-hs.4). Retained intentionally per the - hs.3 journal Concerns note + the doc-comment in str.c, to keep - the translation unit link-safe in isolation as the heap-Str ABI's - single point of contact. - -## Files touched - -- Modified (11 files): - - `crates/ailang-check/src/builtins.rs` — `int_to_str` signature - install + unit test - - `crates/ailang-codegen/src/synth.rs` — `int_to_str` type-replay - arm (lockstep partner of the checker insert) - - `crates/ailang-codegen/src/lib.rs` — IR-header preamble (2 new - declares); `is_static_callee` whitelist extension; `lower_app` - `int_to_str` arm + rewritten `float_to_str` arm; 2 new IR-shape - tests at the bottom of `mod tests` - - `crates/ailang-codegen/src/drop.rs` — `field_drop_call`'s - Str-arm comment refreshed to the dual-realisation framing - - `crates/ail/src/main.rs` — `runtime/rc.c` compile-and-link - hoisted to the unconditional path; Rc-arm body emptied with - doc comment - - `crates/ail/tests/e2e.rs` — 4 new E2E tests appended - - `crates/ail/tests/snapshots/hello.ll`, - `crates/ail/tests/snapshots/list.ll`, - `crates/ail/tests/snapshots/max3.ll`, - `crates/ail/tests/snapshots/sum.ll`, - `crates/ail/tests/snapshots/ws_main.ll` — golden-IR regen via - `UPDATE_SNAPSHOTS=1`; diff is exactly the two new unconditional - `declare` lines from the IR-header extension - -- Created (4 files): - - `examples/int_to_str_smoke.ail.json` — `do io/print_str(int_to_str(42))` - - `examples/float_to_str_smoke.ail.json` — `do io/print_str(float_to_str(3.5))` - - `examples/int_to_str_drop_rc.ail.json` — `let s = int_to_str(42) in - do io/print_str(s)` (RC-stats fixture) - - `examples/str_field_in_adt_heap.ail.json` — `type Boxed = | Box(Str)` - carrying an `int_to_str` result, pattern-matched + printed - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-hs.4.json diff --git a/docs/journals/2026-05-12-iter-ms.1.md b/docs/journals/2026-05-12-iter-ms.1.md deleted file mode 100644 index dc1c7f7..0000000 --- a/docs/journals/2026-05-12-iter-ms.1.md +++ /dev/null @@ -1,42 +0,0 @@ -# iter ms.1 — Pipeline anyhow-chain preservation - -**Date:** 2026-05-12 -**Started from:** 8689f7a2386e0d7993609c374d91984bc28d215c -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -Two-character fix at `pipeline.rs:118`: `format!("check: {e}")` → -`format!("check: {e:#}")`. Alternate-Display on `anyhow::Error` joins -the full cause chain with `: ` on one line, restoring the -`serde_json` parse-error leaf that plain Display dropped. Survives -`strip_locations` (regex only collapses `\nCaused by:` blocks). - -The bug surfaced during ms.2 planning recon: JSON-cohort feedback in -the cma.3 baseline showed uninformative top-line `"parsing JSON in -/tmp/..."` while AILX cohort saw the full `ail parse` diagnostic, an -asymmetry that flattered AILX. ms.1 lands the fix before ms.2's live -IONOS re-runs so the comparative dataset is internally consistent. - -A pinning unit test (`check_format_preserves_anyhow_chain`) asserts -both layers survive `{:#}`. Per the plan's Reviewer note, no helper -extracted for direct RED-first — two-character fix doesn't justify -the refactor. - -Test count: 14 (was 13). `cargo check` clean. - -## Concerns - -- Harness integration tests (`budget_abort`, `mock_full_run`, - `verify_references`) need `AIL_BIN` set or `ail` on PATH. The - 14-green count assumes `AIL_BIN=target/release/ail`. Pre-existing - cma.2 harness behaviour. - -## Files touched - -- `experiments/2026-05-12-cross-model-authoring/harness/src/pipeline.rs` - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-ms.1.json diff --git a/docs/journals/2026-05-12-iter-ms.2.md b/docs/journals/2026-05-12-iter-ms.2.md deleted file mode 100644 index 486f0ad..0000000 --- a/docs/journals/2026-05-12-iter-ms.2.md +++ /dev/null @@ -1,124 +0,0 @@ -# iter ms.2 — Multi-subject run + DESIGN.md addendum extension - -**Date:** 2026-05-12 -**Started from:** 8689f7a2386e0d7993609c374d91984bc28d215c -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -Two live IONOS harness invocations executed with the ms.1 fix in -place: a Qwen3-Coder-Next retroactive re-run -(`runs/2026-05-12-080864/`, RUN_STATUS=ok) and a first-ever -CodeLlama-13b-Instruct run (`runs/2026-05-12-9197fd/`, RUN_STATUS=ok -at the run level; 2 of 8 cells terminated as api_failure on turn 1 -with zero tokens — transient IONOS-side terminal error, disclosed -in the addendum and below). DESIGN.md §"Decision 6 / Empirical -addendum (2026-05-12)" replaced with a four-column per-cell table -plus a framing paragraph; roadmap P3 "Multi-subject expansion" -entry removed. - -Both subjects point the same way on every measured axis: AILX -cohort is cheaper than JSON cohort on prompt tokens (Qwen 61%, -CodeLlama 80%) and on completion tokens (Qwen 23%, CodeLlama 82%), -and AILX reached-green ≥ JSON reached-green for each subject (Qwen -ties at 1/4; CodeLlama strictly dominates 2/4 vs 0/4). Three of the -four first-attempt-green cells across the two subjects are AILX. -Decision 6 is not ratified — n=1 per subject across two subjects is -not statistical robustness — but the second data point points -consistently. The roadmap entry was removed because the brainstorm -spec was scoped to two subjects, not because the universal claim is -closed; a future ratification iteration would re-open it. - -Anomaly disclosure: the Qwen re-run shifted by one cell vs the cma.3 -baseline (AILX 2/4 → 1/4, JSON 1/4 → 1/4) at identical -temperature=0; this is IONOS-side state-of-day noise, not an effect -of the ms.1 feedback fix, which only changes JSON-cohort prompt -content. The two CodeLlama JSON api_failure cells (`t2_length`, -`t4_count_zeros`) were not re-run — the carrier constraint forbids -exploratory token spend, and the addendum discloses the partial -sample directly. - -Roadmap P3 "Multi-subject expansion" entry removed in this iter. - -## Per-task notes - -- iter ms.2.1: Qwen retroactive re-run via harness — new run dir - `experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-080864/`, - RUN_STATUS=ok, 8/8 cells populated. AIL_BIN set to - `target/release/ail` (no `~/.local/bin/ail`); harness picks it up - via env-var resolution at `pipeline.rs:42`. -- iter ms.2.2: CodeLlama-13b-Instruct first run via harness — new - run dir - `experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-9197fd/`, - model id `meta-llama/CodeLlama-13b-Instruct-hf` (Boss-pre-flighted - IONOS catalogue id), RUN_STATUS=ok. 6/8 cells informative; 2/8 - json cells (`t2_length`, `t4_count_zeros`) `api_failure` on turn 1 - zero tokens. -- iter ms.2.3: Per-cell metrics computed via direct scores.csv - read — see addendum table. Total milestone token spend: prompt - 501,985 + completion 23,391 = 525,376. -- iter ms.2.4: DESIGN.md lines 604-636 replaced with new - four-column addendum block (~50 lines including the framing - paragraph). Subjects named with canonical IONOS model ids and - run-dir pointers; framing paragraph reports direction agreement, - failure-class symmetry, first-attempt tally, api_failure anomaly, - and Qwen-rerun noise. Scope paragraph extended to "two subjects, - n=1 each". -- iter ms.2.5: roadmap.md lines 253-268 removed (P3 multi-subject - entry + context note); journal file written. INDEX.md is NOT - edited by this agent per the Iron Law; the Boss appends the - ms.2 line at commit time. Suggested INDEX line (per plan Task 5 - Step 3): - `- 2026-05-12 — iter ms.2: Qwen retroactive re-run + first CodeLlama-13b-Instruct run; DESIGN.md §Decision-6 addendum extended to four cells; roadmap P3 removed → 2026-05-12-iter-ms.2.md` -- iter ms.2.6: Spec acceptance criteria sweep (1-7 all green); 14 - harness tests still pass. - -## Spec acceptance sweep - -1. ms.1 `pipeline.rs:118` reads `format!("check: {e:#}")` + RED→GREEN unit test → green (ms.1 working tree). -2. ms.1 existing 14 harness tests still pass → green (verified Task 6 Step 2). -3. CodeLlama-13b-Instruct-hf runs to completion against IONOS endpoint, RUN_STATUS=ok at run level → green; model id recorded above. -4. Qwen3-Coder-Next re-run completes, RUN_STATUS=ok, 8 cells; distinct dir from cma.3 baseline → green. -5. DESIGN.md §"Decision 6 / Empirical addendum" extended with follow-up paragraph + four-cell-shaped table → green. -6. Roadmap P3 entry removed; one-line mirror above → green. -7. Per-iter journal (this file) written → green. INDEX.md ms.2 entry is Boss-side per the Iron Law; suggested line is in the end-report. - -## Concerns - -- 2 of 4 CodeLlama JSON cells are `api_failure` (terminal IONOS - error on turn 1). Per the Plan's branch (c) and the carrier's - "no exploratory re-runs" constraint, not re-attempted. The - addendum and journal disclose this directly; the - CodeLlama-JSON column's prompt-token average is therefore over 2 - informative cells, not 4. Effect on the AILX-cheaper conclusion: - none in direction; the prompt-token gap is preserved over the - two informative JSON cells. If a future iteration re-runs - CodeLlama at lower IONOS load, this should resolve. -- Qwen-re-run vs cma.3-baseline drift (1 cell) at identical - temperature=0 is IONOS noise on a single deterministic call. A - future ratification iteration should add ≥3 repetitions per - subject to bound this noise, not rely on a single - temperature-zero call. - -## Known debt - -- The pre-existing 3 warnings in `harness/src/ionos.rs` lines 86, - 125, 137 (`last_err` assigned but never read) survived ms.1 - and ms.2. Not in scope here; cma.2 inheritance. -- Decision 6 universal-claim status is still "second data point, - not verdict". A future ratification iteration would need ≥3 - subjects with statistical robustness. Out of scope for ms. - -## Files touched - -- `docs/DESIGN.md` (addendum extended) -- `docs/roadmap.md` (P3 entry removed) -- `experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-080864/` (created by Qwen run) -- `experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-9197fd/` (created by CodeLlama run) -- `docs/journals/2026-05-12-iter-ms.2.md` (this file) -- `bench/orchestrator-stats/2026-05-12-iter-ms.2.json` (stats) - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-ms.2.json diff --git a/docs/journals/2026-05-12-iter-rt.1.md b/docs/journals/2026-05-12-iter-rt.1.md deleted file mode 100644 index 6f4c59f..0000000 --- a/docs/journals/2026-05-12-iter-rt.1.md +++ /dev/null @@ -1,128 +0,0 @@ -# iter rt.1 — Roundtrip-Invariant audit tests - -**Date:** 2026-05-12 -**Started from:** 10ccd1406c886270d105cd7d5a08ec4b447030cd -**Status:** DONE -**Tasks completed:** 4 of 4 - -## Summary - -Three new tests landed; one existing test (handwritten exhibits) -was replaced with a dynamic counterpart. No production code or -DESIGN.md changes (those would be later iterations if any gap -surfaced). All three new tests passed on first observation — -the corpus carries enough breadth that the spec-intended audit -deliverable is the clean signal "no current gap" rather than a -defect list. - -## Per-task notes - -- iter rt.1.1: `crates/ailang-surface/tests/round_trip.rs` — - replaced static three-pair `handwritten_exhibits_match_json_fixtures` - with dynamic `every_ailx_fixture_matches_its_json_counterpart` - (and helper `list_ailx_fixtures`). Module doc rewritten to drop - the "three hand-written exhibits" phrasing. -- iter rt.1.2: `Cargo.toml` — `tempfile = "3"` added to - `[workspace.dependencies]`. `crates/ail/Cargo.toml` — new - `[dev-dependencies]` section with `tempfile.workspace = true` - and `blake3.workspace = true`. -- iter rt.1.3: `crates/ailang-core/tests/schema_coverage.rs` — new - file. Exhaustive AST visitor (`Def`, `Term`, `Pattern`, `Literal`, - `Type`, `ParamMode`) against the fixture corpus; flat `VariantTag` - enum + `EXPECTED_VARIANTS` array; assertion that every variant is - observed at least once. No `_` wildcards in outer match arms — - AST drift fails the build, not the test. -- iter rt.1.4: `crates/ail/tests/roundtrip_cli.rs` — new file. - Subprocess pipeline `ail render → tempfile → ail parse`, BLAKE3 - hash identity on canonical bytes, for every `examples/*.ail.json`. - -## Audit findings - -- `every_ailx_fixture_matches_its_json_counterpart`: - PASS. `.ailx cross-check ok (57 paired, 0 parse-only)`. - All 57 `.ailx` fixtures parse and produce canonical bytes - byte-equal to their same-stem `.ail.json` counterpart. Today - no `.ailx` exists without a JSON counterpart. - -- `every_ast_variant_is_observed_in_the_fixture_corpus`: - PASS. `schema coverage ok: 34 variants observed across 136 fixtures`. - All 34 expected `VariantTag` entries (5 `Def` + 13 `Term` + 4 - `Pattern` + 5 `Literal` + 4 `Type` + 3 `ParamMode`) are exercised - by at least one fixture. No load failures across the 136-file - corpus. - -- `cli_render_then_parse_preserves_canonical_bytes_on_every_fixture`: - PASS. `CLI roundtrip ok for 136 fixtures`. Every `examples/*.ail.json` - survives `ail render → tempfile → ail parse` with BLAKE3 hash - identity on canonical bytes. - -- `cargo test --workspace --quiet` (Step 4.5): PASS, no - regressions in the pre-existing test suite. - -## Concerns - -- Plan-text errata in Task 3's visitor scaffold. The plan in - `docs/plans/rt.1.md` Step 3.2 destructured AST nodes with field - names that do not match `crates/ailang-core/src/ast.rs` (the - authoritative source). Step 3.4 of the plan explicitly anticipated - this and prescribed inline correction by reading `ast.rs` and - adjusting the destructuring. The corrections applied: - - `Def::Fn(fd)`: dropped the `for m in &fd.params { visit_param_mode(m, …) }` - loop (`fd.params: Vec` is parameter names, not modes). - - `Def::Const(cd)`: walks `cd.value` (the actual field name), not - `cd.body`. - - `Def::Class(cd)`: walks `m.ty` plus optional `m.default: Option` - on each `ClassMethod`; no `m.body`/`m.params` fields exist on - `ClassMethod`. - - `Def::Instance(id)`: walks `id.type_: Type` (singular) and - `id.methods`; no `id.args: Vec` field exists. - `InstanceMethod` has only `name` and `body`. - - `Term::App`: `{ callee, args, .. }` (not `{ f, args }`). - - `Term::Let`: `{ value, body, .. }` (not `{ bound, body }`). - - `Term::LetRec`: `{ ty, body, in_term, .. }` (per actual AST: - `name, ty, params, body, in_term`), not the plan's - `{ bindings, body }`. - - `Term::If`: `{ cond, then, else_ }` (not `{ cond, then_b, else_b }`). - - `Term::Do`: `{ args, .. }` only (no `stmts`/`ret` fields). - - `Term::Lam`: `{ param_tys, ret_ty, body, .. }` (no `modes` field - exists on `Lam`). - - `Term::Seq`: `{ lhs, rhs }` (not `{ stmts, ret }`). - - ParamMode discovery routed through `visit_type` on - `Type::Fn { param_modes, ret_mode, .. }` — the only place - `ParamMode` appears in the AST. The plan tried to visit - ParamMode from `Def::Fn.params` (wrong type) and `Term::Lam.modes` - (non-existent field). - - The spec-compliance intent of Step 3.2 ("every AST enum variant is - visited at least once via an exhaustive match with no `_` wildcard") - is preserved; only the destructuring patterns adapted to the actual - AST. All 34 expected variants observe coverage in the corpus, and - the build remains a tripwire for future AST additions. - -- Plan template note: a future `planner`/`brainstorm` pass on - tests-against-AST might cheaply read `ast.rs` and emit a - destructuring-correct scaffold instead of relying on the - implementer to repair under the Step 3.4 escape hatch. Not - rt.1 work. - -## Known debt - -- None added by this iter. The three new tests are pure readers - and add no production-code surface; their existence pins three - invariants that future iters can use to detect regression - rather than rediscover. - -## Files touched - -- `Cargo.toml` — workspace dep `tempfile = "3"`. -- `Cargo.lock` — auto-updated by cargo for `tempfile`. -- `crates/ail/Cargo.toml` — new `[dev-dependencies]` section. -- `crates/ailang-surface/tests/round_trip.rs` — module doc - rewrite; `handwritten_exhibits_match_json_fixtures` → - `every_ailx_fixture_matches_its_json_counterpart` (+ helper). -- `crates/ailang-core/tests/schema_coverage.rs` — new file. -- `crates/ail/tests/roundtrip_cli.rs` — new file. - -## Stats - -bench/orchestrator-stats/2026-05-12-iter-rt.1.json diff --git a/docs/journals/2026-05-12-iter-rt.2.md b/docs/journals/2026-05-12-iter-rt.2.md deleted file mode 100644 index 5b0c6b0..0000000 --- a/docs/journals/2026-05-12-iter-rt.2.md +++ /dev/null @@ -1,71 +0,0 @@ -# iter rt.2 — Roundtrip Invariant anchored at top level in DESIGN.md - -**Date:** 2026-05-12 -**Started from:** b6046bbf (post-rt.1, working tree clean) -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -DESIGN.md gains a new top-level section `## Roundtrip Invariant`, -positioned between Decision 11 and `## Mangling scheme`. The -section states both directions of the `.ail.json` ↔ `.ailx` -bijection, names the Float-bits-hex encoding as the mechanism -that keeps Floats *inside* the invariant, lists the four -enforcement tests by path, and explains why the property is -top-level (load-bearing on language identity, not on Decision 6's -surface-design rationale). - -Decision 6 Constraint 2 is rewritten as an upward cross-reference -to the new top-level section. The substantive content of the -constraint now lives at the anchor; Decision 6 records that its -surface design must satisfy the anchored invariant. - -## Why boss-direct edit (no planner / implement dispatch) - -The spec's Architecture #1 fully specified the content (statement -both directions, Float-mechanism subsection, enforcement-points -list, why-top-level justification, Decision 6 cross-reference). -No design judgement was deferred to execution time; the iter is a -deterministic projection of the spec onto DESIGN.md prose. Plan + -implement dispatch for a pure docs edit with all content -pre-specified is overhead without value, per CLAUDE.md "trivial -mechanical edits" carve-out (the LOC budget is exceeded but the -judgement budget is zero). - -## Files touched - -- `docs/DESIGN.md`: - - Inserted new `## Roundtrip Invariant` section (Z. 1753+, just - before `## Mangling scheme`). - - Rewrote Decision 6 Constraint 2 (Z. 249-253) as an upward - cross-reference to the new section. - -## Verification - -`cargo test --workspace` — all tests green, no regressions. The -schema-drift tests in `crates/ailang-core/tests/design_schema_drift.rs` -scan DESIGN.md for anchors and were the most likely place for the -DESIGN.md edit to surface false-positive failures; they remain -green. - -## Milestone status - -The roundtrip-invariant milestone has now satisfied acceptance -criteria 1-7 from `docs/specs/2026-05-12-roundtrip-invariant.md`: - -1. ✓ Top-level `## Roundtrip Invariant` section in DESIGN.md; - Decision 6 Constraint 2 carries upward cross-reference. (rt.2) -2. ✓ `every_ailx_fixture_matches_its_json_counterpart` dynamic in - `crates/ailang-surface/tests/round_trip.rs`. (rt.1) -3. ✓ `crates/ailang-core/tests/schema_coverage.rs` with - exhaustive-match visitor; 34/34 variants observed. (rt.1) -4. ✓ `crates/ail/tests/roundtrip_cli.rs` with BLAKE3 identity - over `ail render` → tempfile → `ail parse`. (rt.1) -5. ✓ `cargo test --workspace` green with all four tests. (rt.1 + rt.2) -6. ✓ No roundtrip gaps surfaced; no render/parse code changes - needed in this milestone. (rt.1) -7. ✓ Tests are pure readers; the CLI test's only filesystem side - effect is `tempfile::TempDir`. (rt.1) - -Audit skill follows for milestone close. diff --git a/docs/journals/2026-05-13-audit-24.md b/docs/journals/2026-05-13-audit-24.md deleted file mode 100644 index 1ebe86b..0000000 --- a/docs/journals/2026-05-13-audit-24.md +++ /dev/null @@ -1,244 +0,0 @@ -# audit-24 — milestone close (Show + print rewire) - -**Date:** 2026-05-13 -**Milestone:** Show + print rewire (number-24 family) -**Iters covered:** 24.1 (`f38bad8`, shipped 2026-05-12) → 24.2 (`3286117`) → 24.3 (`246b5c7`) -**Range from previous close:** `0dcdaab..0cfb3f6` (audit-ct-tidy → iter-24.3 INDEX) -**Status:** drift_found — `24.tidy` queued - -## Summary - -The Show + print milestone shipped in three iters: 24.1 installed -`bool_to_str` + `str_clone` runtime + codegen primitives at heap-Str -ABI level (pre-mq audit lineage); 24.2 added `class Show` + four -primitive instances (Int/Bool/Str/Float) to the prelude and migrated -the 22b user-Show fixture corpus to `TShow`/`tshow` to eliminate -post-prelude.Show ambiguity; 24.3 added `fn print` with explicit-let -body, three E2E fixtures (positive 4-prim, user-ADT, negative -NoInstance), an IR-shape pin on `print__Int.body`, a Show-aware -NoInstance diagnostic, plus three compiler-path repairs surfaced by -the user-ADT E2E (canonical-form normalisation of -`MonoTarget::FreeFn::type_args`, codegen cross-module reference -fallback, FreeFnCall constraint-residual push). - -Architect drift review surfaces 7 items: three [high] (missing DESIGN.md -documentation of the three iter-24.3 strengthenings: canonical-form -`type_args`, post-mono cross-module-ref invariant, FreeFnCall constraint- -residual push; missing codegen-level pin for the import_map-fallback -path; constraint-residual push currently fires only in the dot-qualified -FreeFnCall branch), three [medium] (`unwrap_or_default` in the -method-name lookup masks class-index drift; `normalize_type_for_lookup` -duplicated across two `mono.rs` sites with manual-lockstep invariant; -negative-test coverage is single-shape — only `f : Int -> Int`, not -user-type-without-instance), and one [low] (bench `architect_sweeps.sh` -noise on three pre-milestone-24 DESIGN.md lines — not new drift). - -Bench: `bench/check.py` exit 1 with 2 regressed / 1 improved metrics -(metric identities migrate between runs — first run hit -`latency.implicit_at_rc.max_us +209%`, second run hit -`latency.explicit_at_rc.max_us +66%` + `bench_list_sum.bump_s +11.9%`). -`bench/compile_check.py` exit 1 with 3-5 regressed metrics across -runs (metric identities migrate: first run hit `build_O0_ms.*` -cluster within tolerance, second run hit `check_ms.*` cluster with -several +25-34% deltas). `bench/cross_lang.py` exit 0, 25/25 stable. -**9th consecutive audit observation** of the documented noise envelope -covering `latency.implicit_at_rc.*` / `latency.explicit_at_rc.*` / -`bench_list_sum.bump_s` / `check_ms.*` metric-cluster migration -across consecutive runs. Conservative-call convention has held the -baseline pristine across 8 prior audits since `audit-cma`; this -audit extends the lineage to 9 with the same conservative call — -the metric-migration-between-runs is itself attribution evidence -that the signal is variance, not regression. - -Resolution path: `24.tidy` iteration covers 5 actionable drift -items (three [high] + two [medium] documentable/refactorable). The -[medium] "negative-test coverage single-shape" item moves to the -roadmap P3 backlog. The [low] bench_sweep noise is carry-on. - -## Architect drift items - -### [high-1] Three iter-24.3 strengthenings undocumented in DESIGN.md - -**Paths:** `crates/ailang-check/src/mono.rs:687-696` + `:1287-1294`, -`crates/ailang-codegen/src/lib.rs:1980-2000` + `:2200-2244` + -`:2776-2816`, `crates/ailang-check/src/lib.rs:2820-2865`, -`docs/DESIGN.md` §"Monomorphisation" or new subsection. - -Iter 24.3 silently strengthened three load-bearing mono / codegen / -synth invariants without documenting them in DESIGN.md: - -- (a) `MonoTarget::FreeFn::type_args` carries canonical (`.`) - types post-collection via `normalize_type_for_lookup`. Pre-iter - invariant was "subst.apply result"; new invariant is - "subst.apply result normalised to registry-lookup form". -- (b) Post-mono synthesised bodies may carry cross-module references - to modules their source template didn't import. Codegen falls back - to direct `module_user_fns` / `module_def_ail_types` lookup when - the prefix isn't in the current module's `import_map`. -- (c) FreeFnCall synth pushes one `ResidualConstraint` per declared - forall-constraint with rigid vars substituted by fresh metavars. - Pre-iter, the arm had `constraints: _` (constraints ignored); - latent for milestone-23 fixtures because all primitive instances - discharged silently. - -Future planner / implementer work that touches these surfaces will -not see the strengthenings without a doc anchor. - -**Outcome:** fix in `24.tidy` (doc-only). - -### [high-2] Codegen import_map fallback path has no unit-level pin - -**Path:** `crates/ailang-codegen/src/lib.rs:2200-2244` + `:1980-2000` + -`:2776-2816`. - -The three cross-module-fallback arms (`resolve_top_level_fn`, -`lower_app` cross-module arm, `synth_with_extras` Var arm) introduce -a new dispatch path: try `import_map`, fall back to direct -`module_user_fns` / `module_def_ail_types` lookup. The fallback is -**only** justified by post-mono synthesised cross-module references. -No codegen-level unit test pins the failure mode; the only test that -would catch a regression is the E2E `show_user_adt.ail.json`. If a -future refactor tightens `resolve_top_level_fn` back to import_map-only, -the regression surfaces only at E2E (slow to bisect). - -**Outcome:** fix in `24.tidy` — add a unit pin asserting the fallback -resolves for a synthesised `prelude.print__T` → `.show__T` -pair without going through `import_map`. - -### [high-3] FreeFnCall constraint-residual push covers only dot-qualified branch - -**Path:** `crates/ailang-check/src/lib.rs:2820-2865`. - -The constraint-residual push happens **only** in the dot-qualified -`prefix.suffix` synth branch (the only branch that populates -`free_fn_owner = Some(...)`). Bare-name references to polymorphic free -fns that resolve through other branches (locals shadow, -`env.module_globals` direct hit at the current module, etc.) do not -push residuals for declared constraints. For prelude poly fns this is -fine because the auto-injected prelude call-path always lands in the -dot-qualified branch — but the invariant is implicit and undocumented. -The same "silent unknown variable from codegen" failure mode existed -pre-iter-24.3 for any milestone-23 negative case (`ne` on a type -without `Eq` instance, etc.); the iter-24.3 fix covers Show only -because that's what the negative E2E exercised. - -**Outcome:** fix in `24.tidy` — generalise the constraint-residual push -to all FreeFnCall synth branches, OR document the invariant -"poly free fns always reach synth via the dot-qualified branch" and -pin it with a unit test. - -### [medium-1] `unwrap_or_default` in method-name lookup masks class-index drift - -**Path:** `crates/ailang-check/src/lib.rs:2852-2858`. - -Method-name lookup in the FreeFnCall constraint-residual push uses -`unwrap_or_default()` (empty string fallback) when `class_methods` has -no entry for the residual class. A user constraint referencing a class -that does not exist would silently render `NoInstance` with an empty -`method:` field. The iter-22 `MissingClass` pre-pass plausibly rejects -this earlier, but the fallback masks the dependency. - -**Outcome:** fix in `24.tidy` — replace with explicit `Internal` or -`expect(...)` to surface registry/index drift on regression. - -### [medium-2] `normalize_type_for_lookup` duplicated across two sites - -**Path:** `crates/ailang-check/src/mono.rs:687-696` + `:1287-1294`. - -The two `normalize_type_for_lookup` call sites use a literal copy of -the body comment rather than extracting a helper. The two sites must -agree byte-identically (per iter-24.3 journal: "Must agree byte- -identically with that site or Phase 3 rewrite cursor produces a mono- -symbol name that differs from the Phase 2 synthesis name"). A -`normalize_free_fn_type_args(env, module_name, metas, subst)` helper -would make the lockstep enforceable by construction. - -**Outcome:** fix in `24.tidy` — extract helper. - -### [medium-3] Negative-test coverage single-shape (deferred) - -**Paths:** `examples/show_no_instance.ail.json` and the broader -NoInstance Show test surface. - -The negative E2E covers `f : Int -> Int` (function type without Show -instance). Other NoInstance shapes — user type without instance, -partial-application residual carrying both a missing constraint AND a -NoInstance, polymorphic call with two unsatisfied constraints — are -not pinned. The Show-aware NoInstance arm fires for any -non-prelude-Show type by construction, so the single fixture is -*structurally representative*, but coverage breadth is thin. - -**Outcome:** roadmap backlog P3 (not in `24.tidy`). Single-fixture -coverage is acceptable for the architecture-shipping milestone; -broader coverage waits for the corpus-migration milestone (the queued -P2 retirement of `io/print_int|bool|float`) when more downstream -fixtures naturally exercise these shapes. - -### [low-1] bench/architect_sweeps.sh noise on pre-milestone-24 DESIGN.md lines - -**Path:** `bench/architect_sweeps.sh` matches on `docs/DESIGN.md:50`, -`:449`, `:614`. - -All three matched lines predate milestone 24 (text not touched in the -milestone-24 diff). Legitimate references or empirical-addendum -headings, not new history anchors. Sweep noise from prior eras. - -**Outcome:** carry-on (not a milestone-24 drift item). - -## Bench - -- `bench/check.py`: exit 1 with metric identities migrating between - consecutive runs: - - Run 1: 2 regressed (`latency.implicit_at_rc.max_us +209%`, - `bench_list_sum.bump_s +11.9%`), 1 improved - (`bench_list_sum.gc_over_bump`). - - Run 2: 2 regressed (`latency.explicit_at_rc.max_us +66%`, - `bench_list_sum.bump_s +11.9%` again), 1 improved (different - metric). - - Pattern: the `latency.*_at_rc.max_us` metric cluster shifts - identity between runs; `bench_list_sum.bump_s` is the 5th - consecutive sighting since `audit-mq.tidy`. Same envelope the - iter-24.3 journal documented as the 8th consecutive observation; - this audit makes it 9. -- `bench/compile_check.py`: exit 1 with 3-5 regressed metrics across - runs; metric identity migrates between `build_O0_ms.*` cluster (run - 1) and `check_ms.*` cluster (run 2). All deltas are +5% to +34% - across runs; tolerance is 20-25% so multiple `check_ms.*` deltas - cross threshold sporadically. Same migration pattern. -- `bench/cross_lang.py`: exit 0; 25/25 stable. - -**Conservative call: baseline pristine for the 9th consecutive audit.** -The metric-identity-migration is itself attribution evidence that the -signal is variance, not regression. If signal were present, the metric -would not migrate between runs at this rate. Ratifying via -`--update-baseline` would pin a snapshot that does not characterise -the actual distribution (since the noise migrates, any single -snapshot is wrong). The right ratification path is the queued P3 -"Latency methodology rework — switch from per-run timing to a -histogram-based approach"; until that ships, conservative-call is the -correct response. - -No `--update-baseline` invocation this audit. - -## Resolution - -- `24.tidy` iter to land 5 drift fixes: `[high-1]` (DESIGN.md - documentation of the three iter-24.3 strengthenings), `[high-2]` - (codegen import_map-fallback pin), `[high-3]` (FreeFnCall - constraint-residual push generalisation OR documented invariant), - `[medium-1]` (replace `unwrap_or_default` with explicit error), - `[medium-2]` (extract `normalize_free_fn_type_args` helper). Routes - through `planner` → `implement`. -- `[medium-3]` (negative-test coverage single-shape): roadmap backlog - P3. -- `[low-1]` (bench sweep noise): carry-on, not actionable. -- Bench: extend lineage to 9th consecutive observation. Pristine - baseline. Document for future histogram-methodology landing. - -## Files touched - -- `docs/journals/2026-05-13-audit-24.md` (new, this file) -- `docs/journals/INDEX.md` (one new line) - -No production code or test edits in this audit; all recommendations -land in `24.tidy`. diff --git a/docs/journals/2026-05-13-audit-form-a.md b/docs/journals/2026-05-13-audit-form-a.md deleted file mode 100644 index b44eb2d..0000000 --- a/docs/journals/2026-05-13-audit-form-a.md +++ /dev/null @@ -1,145 +0,0 @@ -# audit-form-a — Milestone close (Form-A as the default authoring surface) - -**Date:** 2026-05-13 -**Audited range:** aabcadc (iter form-a.0) .. 9fdc4ca (iter form-a.1 T6-12) HEAD -**Spec:** `docs/specs/2026-05-13-form-a-default-authoring.md` (amended at 9fcda8b) -**Status:** drift_found (4 items, all documentary-only); bench clean (exit 0 across all three scripts) -**Recommendation:** carry-on — no form-a.tidy queued at architect's explicit recommendation; drift items deferred to documentary cleanup at next opportunity. - -## What holds (architect-verified) - -- DESIGN.md §"Roundtrip Invariant" restated correctly: four numbered - properties (parse-determinism + idempotency-under-print + CLI- - pipeline-idempotency + carve-out-anchor) replace the prior - Direction-1/Direction-2 framing; five surviving enforcement tests - named verbatim (parse_is_deterministic_over_every_ail_fixture, - parse_then_print_then_parse_is_idempotent_on_every_ail_fixture, - cli_parse_then_render_then_parse_is_idempotent, - every_ast_variant_is_observed_in_the_fixture_corpus, - examples_ail_json_inventory_matches_carve_outs). §"Float literals" - + §"Why anchored at top level" preserved. -- §A4 doctrine rewording landed verbatim per spec at CLAUDE.md:5-6 - + DESIGN.md:465-466 (canonical form = JSON-AST; authoring projection - = `.ail`; "Form (A) is one projection; the JSON-AST stays canonical" - preserved as the second sentence). -- Carve-out inventory matches the eight-file spec post-amendment: - `ls examples/*.ail.json | wc -l` returns 8, alphabetical order - `broken_unbound + prelude + 3× test_22b2_* + 3× test_ct1_*`. - `crates/ailang-core/tests/carve_out_inventory.rs::EXPECTED` - hardcodes the same eight entries. -- 156 non-carve-out `.ail.json` deleted; 157 `.ail` files present - (98 fresh-rendered in iter form-a.1 T2 + 58 pre-iter + the - prelude pilot from form-a.0). No regressions in - `cargo test --workspace` (557 green + 3 ignored). -- WhatsNew milestone-close entry editorial-clean per /boss - convention: English, no crates / iter-codes / agent names, leads - with the change, factual tone, telegram-pragmatic length. -- Roadmap milestone struck `[x]` with closing attribution - `(Closed 2026-05-13 by iter form-a.1.)`; follow-up milestone - `[Prelude embed: Form-A as compile-time source]` present with - motivation + problem + three resolution options. -- Per-iter journals (`2026-05-13-iter-form-a.0.md`, - `2026-05-13-iter-form-a.1.md`) both present; `INDEX.md` carries - both lines in chronological order. - -## Drift items (deferred per architect recommendation) - -| Severity | Path | Issue | -|----------|------|-------| -| medium | `docs/specs/2026-05-13-form-a-default-authoring.md` (§C1, §C2, §"Data flow") | Three sites still say "seven carve-outs" post-amendment; acceptance #1/#2 correctly say eight. Spec self-inconsistent against its own amendment commit 9fcda8b. | -| medium | `docs/plans/2026-05-13-iter-form-a.1.md` (two sites) | Same "seven carve-outs" wording surviving the amendment; harmless now but contradicts the eight-file reality the plan executed. | -| low | `crates/ailang-core/src/hash.rs:57` | Empty `mod tests {}` placeholder left after iter form-a.1 T5 relocation. Documentary-only today; drift if it stays past one further milestone. | -| low | `crates/ailang-surface/tests/round_trip.rs:16` | Module docstring still says "Direction 2 of the Roundtrip Invariant", but DESIGN.md T10 retired the Direction-1/Direction-2 framing. Stale cross-reference to a renamed concept. | - -All four items are documentary-only: the spec / plan orphans -contradict their own post-amendment sections but do not affect the -shipped implementation (which is eight-carve-out-correct); the -hash.rs placeholder and the round_trip.rs docstring are cleanup -hygiene that does not change behaviour or invariants. - -Architect's explicit recommendation: "carry on as planned. Queue a -documentary `form-a.tidy` only if a subsequent reader hits the -seven-vs-eight orphans; the implementation is correct and the spec -orphans are post-hoc retro-clean-up, not active drift." The audit -follows this recommendation. - -## Bench-regression report - -All three bench scripts exit 0; baseline left pristine. - -### `bench/check.py` (exit 0) - -63 metrics; 2 regressed-by-magnitude, 2 improved-by-magnitude, 59 -stable. Regressions both on `max_us` tail-latency metrics: - -- `latency.explicit_at_rc.max_us` +124.99% (tol 25%) — paired with - `latency.explicit_at_rc.p99_us` *improvement* −27.02%. -- `latency.implicit_at_rc.max_us` +126.13% (tol 30%). - -This is the 12th consecutive audit observation of the metric- -identity-migrating noise envelope on tail-latency metrics, dating -back to audit-cma (2026-05-12). The pattern across the lineage: -different metric identifiers fire each run, magnitudes oscillate, -median + p99 stay stable; the firing-metric set has migrated -through approximately a dozen distinct identifiers across the -twelve audits. The convention since the audit-cma originating -observation is to hold the baseline pristine — the metric-migration- -between-runs is itself attribution evidence that the cluster is -noise-class, not signal-class. The queued P3 latency-methodology- -histogram rework is the right ratification path; --update-baseline -is not. - -The form-a milestone is intentionally test-infra + corpus migration, -not language semantics. Zero language-semantic surface was touched. -The tail-latency regressions are not caused by the migration; they -are the noise envelope continuing. - -### `bench/compile_check.py` (exit 0) - -24 metrics; 2 regressed-by-magnitude, 0 improved, 22 stable. - -- `check_ms.hello` +32.44% (tol 25%) — 1.2ms → 1.5ms. -- `check_ms.borrow_own_demo` +26.18% (tol 25%) — 1.3ms → 1.7ms. - -Both on the smallest fixtures in the corpus where `check_ms` is -sub-2-millisecond. iter form-a.1 T6 journal already flagged this: -"First compile_check run reported exit-1 due to sub-millisecond -`check_ms` noise on `hello` and `borrow_own_demo`; second run -clean." Now both still fire at the current audit re-run. This is -sub-millisecond timing-jitter, not a migration-induced regression. -The migration touched only fixture file extensions and test-loader -call sites; the `ail check` binary path was not modified. - -### `bench/cross_lang.py` (exit 0) - -25 metrics; 0 regressed, 0 improved-beyond-tolerance, 25 stable. -Cross-language ratios (AILang RC / AILang bump / C reference) all -hold within tolerance. The most interesting movement is -`bench_tree_walk.rc_over_c` from 2.7394 to 2.5871 (−5.56%) — a -modest improvement still inside the noise envelope. - -## Recommendation summary - -| Item | Recommendation | -|------|----------------| -| 4 architect drift items | carry-on (per architect's explicit recommendation; documentary-only) | -| `bench/check.py` 2 regressions | noise-envelope continuation (12th audit); baseline pristine | -| `bench/compile_check.py` 2 regressions | sub-millisecond timing-jitter; baseline pristine | -| `bench/cross_lang.py` clean | no action | - -No `form-a.tidy` iter is queued. The four documentary drift items -are recorded here for future-reader visibility; they will be -addressed inline at the next iter that touches the affected files, -or in a dedicated cleanup pass if a subsequent reader hits them. - -The form-a-default-authoring milestone is fully closed. - -## Next-step pointer - -Per `/boss` and the milestone-close pipeline: the next milestone -candidate at the top of the roadmap is `[Retire io/print_int/_bool/_float]` -(P2, post-milestone-24 mechanical follow-up). The `[Prelude embed: -Form-A as compile-time source]` milestone is the second P2 entry, -queued behind the io/print retire. Starting either is a `/boss` -bounce-back per the new-milestone rule — the Boss surfaces the -candidate and the user picks the session shape. diff --git a/docs/journals/2026-05-13-audit-mq.md b/docs/journals/2026-05-13-audit-mq.md deleted file mode 100644 index 3b825c6..0000000 --- a/docs/journals/2026-05-13-audit-mq.md +++ /dev/null @@ -1,195 +0,0 @@ -# audit-mq — milestone close (module-qualified-class-names) - -**Date:** 2026-05-13 -**Milestone:** module-qualified-class-names -**Iters covered:** mq.1 (`0eb3323`) → mq.2 (`2e6a4ca`) → mq.3 (`99d3968`) -**Range from previous close:** `0dcdaab..99d3968` (audit-ct-tidy → mq.3) -**Status:** drift_found — `mq.tidy` queued - -## Summary - -The module-qualified-class-names milestone landed in three iters: -mq.1 lifted the canonical-form rule to the three class-ref schema -fields and qualified all workspace-internal class-name keys; mq.2 -installed the type-driven dispatch mechanism without exercising it -end-to-end; mq.3 retired `WorkspaceLoadError::MethodNameCollision`, -re-keyed `class_methods` to a `(class, method)` tuple, plumbed -`Env.active_declared_constraints`, added the -`class-method-shadowed-by-fn` warning, and shipped three positive -E2E fixtures exercising trajectories C (ambiguous), E (explicit -qualifier), and the class-fn shadow case. - -Architect drift review surfaces seven items: two [high] (rigid-var -refinement type-unification leg missing; same-module bare-class -qualifier shape unreachable), three [medium] (warning over-fires on -unrelated locals shadowing prelude method names; DESIGN.md Data -Model schema fragments don't reflect the canonical-form rule for -class-refs; `synth(...)` has grown to 10 mut-ref parameters), and -two [low] backlog items (trajectories B + D covered by unit tests -only — no E2E fixture; `collect_mono_targets` rebuilds env without -`active_declared_constraints`, currently latent). - -Bench: `check.py` exit 1 across four consecutive runs with metric -identity shifting between runs (3 → 1 → 1 → 0 regressions across -re-runs, different metrics each time; one run showed only -improvements). Pattern is consistent with 5th-consecutive audit -noise-class observation. `compile_check.py` exit 0 (24/24 stable); -`cross_lang.py` exit 0 (25/25 stable). Baseline left pristine per -the 5th-consecutive conservative-call convention; see Bench notes -below. - -Resolution path: `mq.tidy` iteration covers the four actionable -drift items (two [high] + two [medium]). The 5th [medium] -(`synth(...)` signature growth) is acknowledged as accepted -post-mq.3 cost and not fixed — refactoring `synth(...)` would -ripple through 15+ recursive callsites and 5 external callers, -disproportionate to the readability gain. The two [low] items -join the roadmap backlog. - -## Architect drift items - -### [high-1] Rigid-var refinement misses type-unification leg - -**Path:** `crates/ailang-check/src/lib.rs:2163-2168` - -The `refine_multi_candidate_residual` rigid-var branch filters -declared constraints on `dc.class == c` only. Spec §"Constraint- -discharge refinement" (line 130-138) requires the filter to also -unify the declared constraint's `type_` with the residual's -`type_`. Two same-class declared constraints on different typevars -(`forall a b. Show a, Show b => (a, b) -> String` style) cannot -currently be discriminated; the filter returns the first match -regardless of which typevar the residual actually came from. No -unit test pins the type-unification leg. - -**Outcome:** fix in `mq.tidy`. - -### [high-2] Same-module bare-class qualifier shape unreachable - -**Path:** `crates/ailang-check/src/lib.rs:2570-2575` - -`qualifier_is_class_shape = q.contains('.')` makes same-module bare- -class qualifiers (`Show.show`) unreachable; the gate only admits -cross-module class qualifiers (`..`). This -contradicts the canonical-form rule's symmetry — bare class refs -are legal at every other class-ref schema field (per mq.1's rule). -Trajectory E E2E coverage is cross-module-only as a consequence. - -**Outcome:** fix in `mq.tidy`. - -### [medium-1] `class-method-shadowed-by-fn` over-fires - -**Path:** `crates/ailang-check/src/lib.rs:2479-2502` - -The warning fires whenever any class declares the method, not -when a class candidate for the actual arg type exists per spec -§"Class-fn collisions" (line 161-162). Will spam on unrelated -locals shadowing prelude method names (e.g. `let show = …` inside -an arbitrary body where `show` has no Show-context). - -**Outcome:** fix in `mq.tidy`. - -### [medium-2] DESIGN.md Data Model schema fragments don't reflect canonical-form rule for class-refs - -**Path:** `docs/DESIGN.md:2139, 2152, 2273` - -Data Model schema fragments for `InstanceDef`, `Constraint`, -`SuperclassRef` describe class-ref `` without flagging the -canonical-form rule. Only the prose at lines 1156-1162 carries the -rule. A schema-section reader without the prose context sees -bare-only. - -**Outcome:** fix in `mq.tidy` (doc-only). - -### [medium-3] `synth(...)` signature growth (accepted) - -**Path:** `crates/ailang-check/src/lib.rs:2415` - -`synth(...)` now carries 10 mut-ref parameters (added `warnings` -in mq.3). Two re-entries (`mono.rs:614`, `lift.rs:700`) discard -the warnings channel by convention. - -**Outcome:** acknowledged debt, NOT fixed in `mq.tidy`. Rationale: -refactoring `synth(...)` (e.g. into a `SynthCtx` struct carrying -the mut-refs) would ripple through 15+ recursive callsites and 5 -external callers, with the only payoff a marginal readability -gain. The mut-ref pattern is consistent with the rest of the -crate (`check_in_workspace`'s `Vec` accumulator), so -this is project-style rather than drift. - -### [low-1] Trajectories B + D no E2E - -**Paths:** spec §"Data flow" trajectories B (type-driven narrow- -to-one) + D (rigid-var narrow-to-one) are covered by unit tests -only — `tests/method_dispatch_pin.rs` 5-step rule tests. No -on-disk fixture exercises either path end-to-end. - -**Outcome:** roadmap backlog (P3). Single-class-per-method -workspaces don't exercise these branches; the real coverage gap -opens when downstream code starts producing the multi-class -shapes the unit tests already pin. - -### [low-2] `collect_mono_targets` rebuilds env without `active_declared_constraints` - -**Path:** `crates/ailang-check/src/mono.rs:574-625` - -Structural divergence from `check_fn`'s env setup. Currently -latent: mono's residuals are concrete-type so the rigid-var -filter never runs. Env-shape invariant unpinned. - -**Outcome:** roadmap backlog (P3). Test that pins the invariant -(`env.active_declared_constraints` populated consistently -wherever `Env` is built) belongs with the [low-1] E2E work for -trajectory D. - -## Bench - -- `bench/check.py`: exit 1 across four consecutive re-runs with - metric identity shifting between runs: - - Run 1: 1 regression (`latency.implicit_at_rc.p99_over_median`), - 5 improvements (across `bench_list_sum.gc_over_bump` + - `bench_closure_chain.gc_over_bump` + 3 - `latency.explicit_at_rc.*` metrics). - - Run 2: 3 regressions (`bench_list_sum.bump_s` 3rd-consecutive - sighting + 2 `latency.implicit_at_rc.*` max-tail), 3 - improvements (same `latency.explicit_at_rc` cluster). - - Run 3: 1 regression (different metric again), 5 improvements. - - Run 4: 0 regressions, 2 improvements - (`bench_list_sum.gc_over_bump` + - `bench_closure_chain.gc_over_bump`). - - Pattern: metric identity is not stable; the recurring - `bench_list_sum.bump_s` 3rd-consecutive sighting per mq.3 - journal lineage shows up in 1 of 4 runs only. Audit-ct-tidy - (2026-05-12), audit-eob (2026-05-12), audit-ms (2026-05-12), - audit-cma (2026-05-12) all left baseline pristine on the same - `latency.explicit_at_rc` improvement cluster reappearing - across milestones — this is the 5th consecutive audit - observation of the same noise envelope. - - Conservative call: baseline pristine for 5th consecutive - audit. The latency cluster has narrowed-then-widened-then- - narrowed across audits, which is itself attribution evidence - that this is variance, not signal. If signal were present - the metric would not migrate between runs at this rate. -- `bench/compile_check.py`: exit 0; 24/24 stable. -- `bench/cross_lang.py`: exit 0; 25/25 stable. - -No `--update-baseline` invocation this audit. - -## Resolution - -- `mq.tidy` iter to land four drift fixes: `[high-1]`, `[high-2]`, - `[medium-1]`, `[medium-2]`. Routes through `planner` → - `implement`. -- `[medium-3]` (`synth(...)` signature growth): acknowledged - debt, no action. -- `[low-1]` (Trajectory B + D E2E) + `[low-2]` (mono env shape): - roadmap backlog P3. Same scope — single follow-up iter when the - multi-class workspace shape lands. - -## Files touched - -- `docs/journals/2026-05-13-audit-mq.md` (new, this file) -- `docs/journals/INDEX.md` (one new line) - -No production code or test edits in this audit; all -recommendations land in `mq.tidy`. diff --git a/docs/journals/2026-05-13-iter-24.2.md b/docs/journals/2026-05-13-iter-24.2.md deleted file mode 100644 index c7faa74..0000000 --- a/docs/journals/2026-05-13-iter-24.2.md +++ /dev/null @@ -1,189 +0,0 @@ -# iter 24.2 — Show class + 4 primitive instances + 22b TShow/tshow migration - -**Date:** 2026-05-13 -**Started from:** 64cea0e -**Status:** DONE -**Tasks completed:** 8 of 8 - -## Summary - -Ships `class Show a where show : (a borrow) -> Str` in -`examples/prelude.ail.json` alongside primitive instances `Show Int`, -`Show Bool`, `Show Str`, `Show Float`. Each instance body is a -single-application lambda invoking the corresponding runtime primitive -(`int_to_str`, `bool_to_str`, `str_clone`, `float_to_str`) — first -prelude instance bodies to call runtime primitives directly -(prior Eq/Ord bodies used operator + codegen intercept). The 22b -typeclass test corpus migrates `class Show` / `show` → -`class TShow` / `tshow` preemptively (Tasks 1–2 before Task 3 lands -`prelude.Show`) so the workspace stays green throughout. Three new -tests pin the post-mono shape: mono-synthesis existence -(`show_mono_synthesis.rs`), dispatch singleton (`show_dispatch_pin.rs` -× 2 cases), and body-hash stability (`mono_hash_stability.rs::primitive_show_mono_symbol_hashes_stay_bit_identical`). -DESIGN.md §"Prelude (built-in) classes" gains a milestone-24 paragraph -naming Show + instances + the Float caveat. `print`-rewire stays -deferred to iter 24.3. - -## Per-task notes - -- iter 24.2.1: Renamed `"Show"`/`"show"` → `"TShow"`/`"tshow"` across 21 - `test_22b*_*.ail.json` fixture files; cross-module qualifier forms - (`".Show"` → `".TShow"`) handled by regex in 3 - fixtures (`test_22b1_dup_a`, `test_22b1_dup_b`, - `test_22b1_orphan_third`). False-positive scan empty; fixture - parse round-trip green. -- iter 24.2.2: Renamed literals in `crates/ail/tests/typeclass_22b{2,3}.rs` - (`"Show"` → `"TShow"`, `"show"` → `"tshow"`, `show__T` → `tshow__T`), - including comment-prose forms (`` `Show` `` → `` `TShow` ``, - `class Show` → `class TShow`, etc.) and one assertion-message string - ("exactly one show__Int across the workspace" → "tshow__Int..."). - Plan undercount surfaced: the rename touches more than the 2 named - test files — see Concerns below. -- iter 24.2.3: Inserted 5 new top-level defs into `examples/prelude.ail.json` - immediately before `fn ne` (line 223 pre-edit). The `instance Show Str` - body invokes `str_clone` as `Term::Var` — first prelude instance body - to do so. Round-trip green; `primitive_eq_ord_mono_symbol_hashes_stay_bit_identical` - stays green (Eq/Ord hashes unaffected by Show defs). -- iter 24.2.4: New fixture `examples/show_mono_pin_smoke.ail.json` - exercises `show 42`, `show true`, `show "x"`, `show 1.5` in a - let-chain ending in `io/print_str s1`. New test - `crates/ail/tests/show_mono_synthesis.rs::primitive_show_mono_symbols_synthesise` - asserts `show__Int`/`show__Bool`/`show__Str`/`show__Float` materialise - as `Def::Fn` entries in the `prelude` post-mono module. `str_clone` - reachability in instance-lambda position verified implicitly: - `show__Str` materialises, which means the lambda body's `Term::Var - "str_clone"` resolved against the checker's builtin table during - typecheck. -- iter 24.2.5: New pin test `crates/ailang-check/tests/show_dispatch_pin.rs` - asserts `env.method_to_candidate_classes["show"] == {"prelude.Show"}` - (singleton) and `env.method_to_candidate_classes["tshow"] == {"*.TShow"}` - (singleton). API spelled `build_check_env(&ws)` not `Env::from_workspace` - (plan's placeholder name). 2/2 cases pass first-shot. -- iter 24.2.6: Extended `crates/ail/tests/mono_hash_stability.rs` with - a second test function `primitive_show_mono_symbol_hashes_stay_bit_identical` - loading `show_mono_pin_smoke.ail.json` (kept separate from the - Eq/Ord fixture to avoid coupling). Hashes captured first-run: - `show__Int 891eaebe64b3180d`, `show__Bool 1bdb621494e50607`, - `show__Str 3f1d57c90ddce081`, `show__Float 5ab0bdd40b9f8b8f`. -- iter 24.2.7: DESIGN.md §"Prelude (built-in) classes" amended: - milestone-23 paragraph drops "Show" from the deferred list and notes - Show ships in milestone 24; new milestone-24 paragraph names class - signature, instance set (including Float), body shape (single-app - lambda invoking runtime primitive), and the iter-24.3 carry-forward - for `print`-rewire. -- iter 24.2.8: Workspace tests 552 passed / 0 failed - (was 548 pre-iter, +4 new: 1 `show_mono_synthesis` + 2 `show_dispatch_pin` - + 1 hash-stability). `bench/compile_check.py` exit 0 (24/24 stable). - `bench/cross_lang.py` exit 0 (25/25 stable). `bench/check.py` exit 1 - with 2 noise-class metrics (`bench_list_sum.bump_s` +11.71% — 4th - consecutive sighting per `audit-mq` / `mq.tidy` lineage; - `latency.implicit_at_rc.max_us` +145.55% — 7th consecutive sighting of - the documented noise envelope, metric-identity-migrating across runs). - Baseline pristine per the conservative-call convention. - -## Concerns - -- **Plan undercount on 22b migration scope.** The plan's "2 Rust test - files" carrier (typeclass_22b{2,3}.rs) missed 4 additional consumer - sites that hardcode the renamed literals: (a) - `crates/ailang-core/src/hash.rs` ct4 canonical-form hash pin (2 - entries); (b) `crates/ailang-core/src/workspace.rs` 3 in-crate iter22b1 - tests; (c) `crates/ailang-prose/tests/snapshot.rs` (1 snapshot file at - `examples/test_22b2_instance_present.prose.txt`); (d) one - `mono_symbol_compound_type_uses_hash_suffix` assertion in typeclass_22b3.rs - that called `mono_symbol("tshow", ...)` (post my batch rename) but - still asserted `name.starts_with("show__")`. All fixed inline per the - precedent of mq.1 / mq.3 "Plan defects fixed inline" — fixture parse - was the only "green" pre-iter signal the plan's Step 3/4 anchored on, - which masked the broader downstream consumer surface. Suggested - Boss-side reaction: future migration plans run an upfront - `grep -rln '""' crates/` to enumerate the full - consumer set before committing to a per-file enumeration. -- **`str_clone` reachability is implicit.** Pre-flight note 3 flagged - this as needing first-round-trip verification; verification is - implicit in `primitive_show_mono_symbols_synthesise` passing - (mono of `show 42 / true / "x" / 1.5` requires the four `show__T` - bodies to typecheck, including the `str_clone` call in `show__Str`). - No explicit isolated unit test for `str_clone`-in-lambda was added - because the integration pin is strictly stronger; consider an - isolated unit pin only if a future regression makes the integration - test slow to bisect. -- **Hash-stability pin uses a separate fixture, not the existing - `mono_hash_pin_smoke.ail.json`.** Plan Task 6 offered both options; - the separate-fixture path was chosen because `show_mono_pin_smoke.ail.json` - already existed (minted in Task 4) and reusing it kept the Eq/Ord - pin structurally isolated. Both fixtures live side-by-side; the - second test function in `mono_hash_stability.rs` mirrors the first - byte-for-byte modulo the fixture path and pin table. -- **`bench/check.py` exit 1 is the documented recurring noise envelope.** - `latency.implicit_at_rc.*` max-tail metric has been migrating - between runs since `audit-cma` (2026-05-12) — 7th consecutive - observation; `bench_list_sum.bump_s` is the 4th since `audit-mq`. - Plan Task 8 Step 4 explicitly anticipates "1-3 regressed metrics - ratify-without-attribution per conservative-call convention"; this - iter follows that. The baseline stays pristine. - -## Known debt - -- **iter 24.3 still pending.** Carries `print : forall a. Show a => a -> () !IO` - rewire to route through `show` + `io/print_str`. Spec at - `docs/specs/2026-05-13-24-show-print.md`. 24.2 is structurally - complete; 24.3 is operationally distinct. -- **mq1 cross-module fixtures unmigrated by design.** - `examples/mq1_xmod_constraint_class.ail.json` and `..._dep.ail.json` - reference `mq1_xmod_constraint_class_dep.Show` (cross-module, fully - qualified) and ship zero `show` call sites; no dispatch ambiguity - arises once `prelude.Show` lands. They stay as-is per the plan's - pre-flight ratification. -- **mq3 fixtures deliberately retain two-Show shapes.** - `examples/mq3_two_show_*` fixtures test the multi-class dispatch - scenarios (ambiguity + qualifier resolution) and stay unchanged in - 24.2 by design. - -## Files touched - -**Created (3):** -- `crates/ail/tests/show_mono_synthesis.rs` -- `crates/ailang-check/tests/show_dispatch_pin.rs` -- `examples/show_mono_pin_smoke.ail.json` - -**Modified (29):** - -Prelude + DESIGN: -- `examples/prelude.ail.json` (+5 defs) -- `docs/DESIGN.md` (§"Prelude (built-in) classes" amendment) - -22b fixture migration (21 files): -- `examples/test_22b1_dup_a.ail.json` -- `examples/test_22b1_dup_b.ail.json` -- `examples/test_22b1_dup_classmod.ail.json` -- `examples/test_22b1_dup_same_module.ail.json` -- `examples/test_22b1_orphan_class.ail.json` -- `examples/test_22b1_orphan_third.ail.json` -- `examples/test_22b1_orphan_third_classmod.ail.json` -- `examples/test_22b2_class_method_lookup.ail.json` -- `examples/test_22b2_constraint_declared.ail.json` -- `examples/test_22b2_instance_present.ail.json` -- `examples/test_22b2_missing_constraint.ail.json` -- `examples/test_22b2_no_instance.ail.json` -- `examples/test_22b2_two_fns_missing_constraint.ail.json` -- `examples/test_22b2_xmod_classmod.ail.json` -- `examples/test_22b2_xmod_classmod_noinst.ail.json` -- `examples/test_22b2_xmod_instance_present.ail.json` -- `examples/test_22b2_xmod_missing_constraint.ail.json` -- `examples/test_22b2_xmod_no_instance.ail.json` -- `examples/test_22b3_dup_call_sites.ail.json` -- `examples/test_22b3_no_call_sites.ail.json` -- `examples/test_22b3_shadow_class_method.ail.json` - -22b consumer-test migration (5 source files + 1 snapshot): -- `crates/ail/tests/typeclass_22b2.rs` -- `crates/ail/tests/typeclass_22b3.rs` -- `crates/ailang-core/src/hash.rs` (ct4 hash pin) -- `crates/ailang-core/src/workspace.rs` (3 iter22b1 tests) -- `crates/ail/tests/mono_hash_stability.rs` (new show test fn) -- `examples/test_22b2_instance_present.prose.txt` (snapshot accept) - -## Stats - -`bench/orchestrator-stats/2026-05-13-iter-24.2.json` diff --git a/docs/journals/2026-05-13-iter-24.3.md b/docs/journals/2026-05-13-iter-24.3.md deleted file mode 100644 index 187a138..0000000 --- a/docs/journals/2026-05-13-iter-24.3.md +++ /dev/null @@ -1,238 +0,0 @@ -# iter 24.3 — fn print polymorphic free fn + E2E + DESIGN.md sync; milestone 24 close - -**Date:** 2026-05-13 -**Started from:** c04c07f -**Status:** DONE -**Tasks completed:** 8 of 8 - -## Summary - -Ships `fn print : forall a. Show a => (a borrow) -> () !IO` in -`examples/prelude.ail.json` with explicit-let body -`\x -> let s = show x in do io/print_str s`. The let-binder is -structurally pinned by `crates/ail/tests/print_mono_body_shape.rs` -(post-mono `print__Int.body` is `Term::Let → Term::App(show__Int) → -Term::Do(io/print_str)`). Three new E2E fixtures + tests verify the -full path: `show_print_smoke` (4 primitives) and `show_user_adt` -(`data IntBox` + `instance prelude.Show IntBox` + `print (MkIntBox 7)`) -both compile, run, and produce expected stdout; `show_no_instance` -fires a Show-aware `NoInstance` diagnostic with the canonical -DESIGN.md cross-reference. DESIGN.md §"Prelude (built-in) classes" -flips `print` from "ships in 24.3" to "shipped in iter 24.3" with the -explicit-let body + pin reference. DESIGN.md §"Float semantics" gains -a Show-Float NaN-spelling cross-reference paragraph. Roadmap P1 -"Post-22 Prelude — Show + print rewire" entry checked off; new P2 -entry "Retire `io/print_int|bool|float` effect-ops + migrate example -corpus to `print`" inserted at top of P2. **Three plan-defects-fixed- -inline** surfaced when running the user-ADT E2E: (a) `MonoTarget:: -FreeFn::type_args` were carrying bare type-cons references that broke -the post-mono walk's registry lookup when the synthesised body lived -in a different module from the type — fixed by normalising via -`workspace_registry.normalize_type_for_lookup` at two -`mono.rs` collection sites; (b) codegen's cross-module fn resolution -went through `import_map`, but post-mono synthesised bodies may -reference modules their source template didn't import — fixed by -falling back to direct `module_user_fns` lookup at three codegen -sites (`resolve_top_level_fn`, `lower_app` cross-module arm, -`synth_with_extras` Var arm); (c) the synth FreeFnCall arm didn't push -residuals for declared constraints, so `Show (Int -> Int)` never -reached the typecheck-time discharge logic — fixed by walking -`Type::Forall.constraints` at the FreeFnCall site and pushing one -residual per constraint with the fresh-metavar substitution. Full -`cargo test --workspace` 556 passed / 0 failed (was 552 post-24.2, -+4 new = 1 print_mono_body_shape + 2 show_print_e2e + 1 -show_no_instance_e2e). `bench/cross_lang.py` clean; `bench/compile_check.py` -+ `bench/check.py` exit 1 with documented noise-class metrics per the -audit-cma lineage envelope (baseline pristine, 8th consecutive -audit-grade observation). Milestone 24 closes structurally; only the -audit pipeline remains. - -## Per-task notes - -- iter 24.3.1: Appended `fn print` to `examples/prelude.ail.json` - after `fn ge` (last def pre-edit). 20 defs total (was 19). Body - is the literal `Term::Let { name: "s", value: App(show, [x]), body: - Do(io/print_str, [s]) }` with the constraint `Show a` bare per - mq.1 same-module canonical form. 552 tests stay green. -- iter 24.3.2: `crates/ail/tests/print_mono_body_shape.rs` asserts - post-mono `print__Int.body` shape. Initial test asserted outer - `Term::Lam` wrapper — corrected after the panic surfaced that - `FnDef.body: Term` is the *inner* function body for top-level fns - (params carried separately on `FnDef.params`). The test now asserts - directly `Term::Let → App(show__Int, [x]) → Do(io/print_str, [s])`, - faithful to the plan's intent (preserve the explicit-let-binder - discipline) while matching the actual AST shape. -- iter 24.3.3: `examples/show_print_smoke.ail.json` written with - nested `Term::Seq` of four `App(print, [lit])` calls (Int 42, Bool - true, Str "hello", Float bits=40091eb851eb851f for 3.14). `ail - build` clean; stdout: `42\ntrue\nhello\n3.14` — io/print_str - appends a newline via `@puts`. -- iter 24.3.4: `examples/show_user_adt.ail.json` written + 2-test - `crates/ail/tests/show_print_e2e.rs`. Surfaced three substantive - plan-defects-fixed-inline (Concerns below); after fix, both E2E - tests pass (`print (MkIntBox 7)` → stdout "7"). -- iter 24.3.5: Extended NoInstance Float-aware arm in - `crates/ailang-check/src/lib.rs:770-779` with a parallel - `class == "prelude.Show"` branch. The arm fires for ANY type - lacking a Show instance (function types, user types without - instance) — cross-references DESIGN.md §"Prelude (built-in) - classes" verbatim. Negative fixture `examples/show_no_instance.ail.json` - (let-bound `f : Int -> Int` + `print f`) and pin test in - `crates/ail/tests/show_no_instance_e2e.rs` (asserts code, - Show-substring, Prelude-(built-in)-classes substring). After - the constraint-residual-push fix in synth (Concerns below), the - diagnostic fires at typecheck. -- iter 24.3.6: DESIGN.md §"Prelude (built-in) classes" milestone-24 - paragraph amended: `print` flipped to past tense ("shipped in iter - 24.3"), body shape + pin file reference added, retirement - follow-up named. DESIGN.md §"Float semantics" gained a new - paragraph in the "Unspecified" block cross-referencing `show 1.5` - / `show nan` / `show inf` to `instance Show Float` via - `float_to_str`. ~10 lines added net (2697 → 2707). -- iter 24.3.7: Roadmap P1 "Post-22 Prelude — Show + print rewire" - entry flipped to `[x]` with a closing-summary block naming all - three shipped iters (24.1 f38bad8, 24.2 iter-24.2, 24.3 this - iter). New P2 entry "Retire `io/print_int|bool|float` effect-ops - + migrate example corpus to `print`" inserted at top of P2, - before the existing operator-routing-through-Eq/Ord entry. -- iter 24.3.8: Full workspace 556 passed / 0 failed. `prelude_free_fns` - 5/5 green; `mono_hash_stability` 2/2 green (Eq/Ord hashes - unchanged, Show hashes from 24.2 unchanged). `bench/cross_lang.py` - exit 0 (25/25 stable). `bench/compile_check.py` exit 1 with 1 - noise-class regression on a `check_ms.*` metric (regressed metric - identity migrates between runs — pattern consistent with the - documented noise envelope per audit-cma lineage, 8th consecutive - observation). `bench/check.py` exit 1 with 2 improvements beyond - tolerance on `latency.explicit_at_rc.*` — same envelope, same - conservative call. Baseline pristine. - -## Concerns - -- **Three plan-defects-fixed-inline surfaced during Task 4 (user-ADT - E2E).** All three are necessary repairs to make the spec's stated - user-ADT trajectory (§"Data flow / print x at user type IntBox" - lines 297-310) work end-to-end. The plan did not anticipate them - because the test architecture envisioned (Tasks 3+4 sharing one - test file) didn't actually exercise the cross-module mono synthesis - before this iter; primitive-only smoke worked through the existing - pipeline. - - (a) **Bare type-cons in `MonoTarget::FreeFn::type_args`.** At - mono target-collection (`collect_mono_targets` + `collect_residuals_ordered` - free-fn arms), `subst.apply(meta)` returns the unification result - in whatever qualification the caller-module's view used. For a - user-defined type referenced from `main` (in `show_user_adt`), - `IntBox` came out bare. The downstream `synthesise_mono_fn_for_free_fn` - then substituted bare `IntBox` everywhere in the synthesised body, - and Phase 3 rewrite of the synthesised body (run in PRELUDE - caller-module context, where `IntBox` is not a known TypeDef) saw - `Type::Con{IntBox}` that didn't normalise to the registry-keyed - `show_user_adt.IntBox`. Lookup missed → no `show__IntBox` target - scheduled → bare `Var "show"` persisted post-mono. Fix: - `mono.rs` lines 685-696 + 1267-1278 normalise via - `env.workspace_registry.normalize_type_for_lookup(module_name, &resolved)` - before pushing the type into `MonoTarget::FreeFn::type_args`. Same - invariant applies symmetric to the existing - `r_ty_norm = normalize_type_for_lookup(...)` calls already in - `collect_mono_targets`'s class-method arm. - - (b) **Codegen cross-module references via import_map.** Post - fix-(a), the synthesised body now correctly references - `show_user_adt.show__cb5eb497` from inside `prelude.print__cb5eb497`. - But `prelude` does not have `show_user_adt` in its `import_map` - (prelude is the base; user modules import IT, not the other way - around). Three codegen sites raised UnknownVar: - `resolve_top_level_fn:2200`, `lower_app` cross-module arm:1981, - `synth_with_extras` Var arm:2777. All three now fall back to a - direct `module_user_fns.contains_key(prefix)` (or - `module_def_ail_types.contains_key(prefix)`) lookup when the - prefix isn't in the current module's import_map. This is the - post-mono synthesised-body invariant: cross-module references in - synthesised bodies may bypass the source template's imports - (both ends were independently typechecked under their own module - contexts before mono ran; the cross-module reference is created - by mono, not by the source). Doc comments cross-reference all - three sites. - - (c) **Poly-free-fn constraint residuals not pushed at synth.** - Pre-iter-24.3, the FreeFnCall arm at `lib.rs:2820` had - `constraints: _` — declared constraints of the poly free fn were - ignored. For all milestone-23 fixtures (`ne`/`lt`/`le`/`gt`/`ge` - at primitive types), this was latent because every call site had - a registry entry; the mono pass silently scheduled the - corresponding `eq__T`/`compare__T` targets, and codegen succeeded. - But for negative cases — `print f` where `f : Int -> Int` has no - Show instance — the constraint never reached the typecheck-time - `NoInstance` discharge (lib.rs:1880-1905). The build would emit - a confusing post-mono `unknown variable: show` from codegen - rather than the right typecheck-phase diagnostic. Fix: at - `lib.rs:2820+`, walk `Type::Forall.constraints` at FreeFnCall - sites, substitute the rigid vars with their fresh metavars, and - push one `ResidualConstraint` per declared constraint. The - discharge loop then fires the correct NoInstance with full - diagnostic context (including the Show-aware addendum from - Task 5). Pre-existing milestone-23 tests stay green because - primitive-type Show/Eq/Ord registry entries discharge silently. -- **bench/compile_check.py + check.py exit 1.** Both report - noise-class metrics (1 regression / 2 improvements respectively). - Metric identity migrates between consecutive runs — pattern - consistent with the documented `latency.implicit_at_rc.*` / - `check_ms.*` noise envelope per audit-cma + audit-ms + audit-eob + - audit-ct-tidy + audit-mq + iter-24.2 lineage. 8th consecutive - observation. Baseline pristine per conservative-call convention. -- **Plan deviation on Task 2 (IR-shape pin).** The plan's test stub - asserted `Term::Lam { body: Term::Let { ... } }` for `print__Int`. - The actual `FnDef.body: Term` shape for top-level fns is the - *inner* function body (params separate); no outer `Term::Lam`. The - test was corrected to assert `Term::Let { ... }` directly. The - load-bearing property (explicit let-binder for heap-Str RC - discipline) is identical between the plan's intent and the - shipped assertion; only the wrapper assumption changed. Doc - comment in the test file explains the FnDef.body shape contract. - -## Known debt - -- **Milestone 24 closes structurally with this iter.** The standard - `audit` pipeline (architect drift review + bench regression - attribution) runs next. The three plan-defects-fixed-inline above - may merit architect commentary on whether the fix sites belong as - permanent invariants or are themselves transitional. Specifically: - fix (a) implies that `MonoTarget::FreeFn::type_args` carries - CANONICAL types post-collection — this is a strengthening of the - pre-iter invariant which was "subst.apply result"; future planner - work that touches the mono pass should keep that strengthening - visible. -- **P2 entry "Retire io/print_int|bool|float effect-ops" is queued.** - The new entry's `~86 fixtures affected` count is from a rough - grep against `do io/print_`; precise count + migration plan is - the planner's job when the milestone opens. -- **Negative-fixture coverage is single-shape.** `show_no_instance.ail.json` - uses `f : Int -> Int`. Other NoInstance shapes (user type without - instance, `print x` where `x : SomeUnsupportedClass`) are not - separately tested — the NoInstance arm fires for any non-prelude-Show - type by construction, so the single fixture is structurally - representative. - -## Files touched - -**Created (6):** -- `crates/ail/tests/print_mono_body_shape.rs` -- `crates/ail/tests/show_print_e2e.rs` -- `crates/ail/tests/show_no_instance_e2e.rs` -- `examples/show_print_smoke.ail.json` -- `examples/show_user_adt.ail.json` -- `examples/show_no_instance.ail.json` - -**Modified (6):** -- `examples/prelude.ail.json` (+1 def: `fn print`) -- `crates/ailang-check/src/lib.rs` (NoInstance Show-aware branch + - FreeFnCall constraint-residual push) -- `crates/ailang-check/src/mono.rs` (FreeFn type_args canonical-form - normalisation, 2 sites) -- `crates/ailang-codegen/src/lib.rs` (cross-module reference fallback - for post-mono synthesised bodies, 3 sites: `resolve_top_level_fn`, - `lower_app`'s cross-module call arm, `synth_with_extras`'s Var arm) -- `docs/DESIGN.md` (§"Prelude (built-in) classes" milestone-24 - paragraph + §"Float semantics" Show-Float cross-reference) -- `docs/roadmap.md` (P1 Show+print → [x]; new P2 entry at top) - -## Stats - -`bench/orchestrator-stats/2026-05-13-iter-24.3.json` diff --git a/docs/journals/2026-05-13-iter-24.tidy.md b/docs/journals/2026-05-13-iter-24.tidy.md deleted file mode 100644 index 41a2a1c..0000000 --- a/docs/journals/2026-05-13-iter-24.tidy.md +++ /dev/null @@ -1,157 +0,0 @@ -# iter 24.tidy — close 5 actionable drift items from audit-24 - -**Date:** 2026-05-13 -**Started from:** 0e27533 -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -Closes the 5 actionable drift items audit-24 surfaced (3× [high] + -2× [medium]): T1 adds a new DESIGN.md subsection -`### Cross-module references in synthesised bodies` under -`## Decision 11`'s `### Resolution and monomorphisation`, anchoring -the three iter-24.3 strengthenings as load-bearing lockstep -invariants (canonical-form type_args / import_map-fallback / FreeFnCall -constraint-residual push). T2 pins the codegen import_map-fallback -path with an integration test asserting the synthesised -`prelude.print__IntBox` body contains a `show_user_adt.show__IntBox` -Var AND the prelude source module does not import `show_user_adt`. -T3 pins the bare-name poly-fn dot-qualified-branch invariant by -asserting `show_no_instance.ail.json` produces exactly one -`no-instance` diagnostic and zero `unknown-variable` / -`internal` diagnostics. T4 tightens -`crates/ailang-check/src/lib.rs:2852` from `unwrap_or_default()` to -`.expect(...)` with a registry-coherence panic message; the strict -tightening surfaced no existing violations (558 tests pass post-edit). -T5 extracts the duplicated `subst.apply + is_fully_concrete + -normalize_type_for_lookup` body into a single `apply_subst_and_normalize` -helper returning `Option`, replacing the two byte-identical -sites at `mono.rs::collect_mono_targets` (685-714) and -`::collect_residuals_ordered` (1284-1303); the byte-identity -invariant is now enforced by construction, `mono_hash_stability` -green on both primitive Show + Eq/Ord hashes. T6 verifies: 558 -tests pass (was 556 + 2 new pins, exactly the expected count); -`bench/cross_lang.py` exit 0 (25/25 stable); `bench/compile_check.py` -exit 0 with 4 `check_ms.*` noise-class regressions in the documented -audit-24 lineage; `bench/check.py` exit 0 with 3 `latency.implicit_at_rc.*` -+ 4 `latency.explicit_at_rc.*` noise-class observations in the same -lineage (10th consecutive observation; baseline pristine). - -## Per-task notes - -- iter 24.tidy.1 (T1): DESIGN.md +68 lines, new `### Cross-module - references in synthesised bodies` subsection inserted between - `### Resolution and monomorphisation` (ends line 1693) and the - re-anchored `### Defaults and superclasses`. Subsection enumerates - three lockstep invariants with cross-references to source files - (`mono.rs::collect_mono_targets` / `::collect_residuals_ordered`, - `codegen/lib.rs::resolve_top_level_fn` / `::lower_app` / - `::synth_with_extras`, `check/lib.rs` synth FreeFnCall arm) and - to the protective test pins. Closing paragraph names the lockstep - partnership and the regression cost of loosening any one. - -- iter 24.tidy.2 (T2): New `crates/ail/tests/codegen_import_map_fallback_pin.rs`. - Loads `examples/show_user_adt.ail.json`, runs `check_workspace` + - `monomorphise_workspace`, finds the synthesised `Def::Fn` whose - name starts with `print__` in the `prelude` post-mono module, - recursively walks its body looking for a `Term::Var { name }` - whose name starts with `show_user_adt.` and contains `show__`, - and confirms the prelude source module's `Vec` does NOT - include `show_user_adt`. Plan draft adapted at three points: (a) - `Term::App` field name is `callee` not `fn` (the latter is the - serde rename; Rust field is `callee`); (b) `imports.iter()` gives - `&Import` not `&String` so the check is on `imp.module`; (c) - recursive walker extended with `Seq` / `Clone` / `ReuseAs` / - `LetRec` / `If` / `Match` / `Ctor` arms for exhaustiveness — all - data-shape adaptations the plan explicitly authorised. Test passes - first-shot (1 passed). - -- iter 24.tidy.3 (T3): New `crates/ail/tests/polyfn_dot_qualified_branch_pin.rs`. - Loads `examples/show_no_instance.ail.json`, asserts exactly one - `no-instance` diagnostic AND zero `unknown-variable` / `internal` - diagnostics. The plan's draft `d.code` field accesses match the - actual `Diagnostic` struct (verified against - `crates/ailang-check/src/diagnostic.rs:131`). Test passes first-shot. - -- iter 24.tidy.4 (T4): One-line tightening at - `crates/ailang-check/src/lib.rs:2858` — - `.unwrap_or_default()` → `.expect("class_methods registry coherence ...")` - with the verbatim plan message. The `.expect(...)` covers the - registry-coherence invariant: a declared constraint's class is - expected to be in `env.class_methods` because the pre-pass - `MissingClass` diagnostic should have rejected it earlier; reaching - this point with `None` means workspace-registry / class-index drift. - No existing test violates the new invariant — 558 tests pass. - -- iter 24.tidy.5 (T5): New `apply_subst_and_normalize` helper at - `crates/ailang-check/src/mono.rs:555-583` (immediately preceding - `collect_mono_targets`). Signature - `fn apply_subst_and_normalize(env: &crate::Env, module_name: &str, m: &Type, subst: &crate::Subst) -> Option`. - Returns `Some(normalised)` if `subst.apply(m)` is fully concrete - (via `crate::is_fully_concrete`); else `None`. Two call sites - retrofit: site 1 at `collect_mono_targets` (now ~ lines 707-727) - keeps its rigid-var / Unit-default branching at the call site - (helper-None → check `contains_rigid_var(&subst.apply(m))`; if - rigid set `has_rigid = true` and break; else push `Type::unit()`); - site 2 at `collect_residuals_ordered` (now ~ lines 1284-1290) uses - `apply_subst_and_normalize(&env, ...).unwrap_or_else(Type::unit)`. - Site 1 needed `&env` (local owned `Env`) where the plan draft had - bare `env` — one-character edit. `mono_hash_stability` both tests - pass (primitive Show + Eq/Ord mono-symbol hashes bit-identical), - full workspace 558 green. Byte-identity invariant now enforced by - construction at the helper boundary; each call site explicitly - documents its post-helper-None policy. - -- iter 24.tidy.6 (T6): Integration verification. Full workspace - `cargo test --workspace --no-fail-fast` = 558 passed, 0 failed - (matches plan's prediction: 556 baseline + 2 new pins). No - FAILED entries among milestone-24 specific test groups - (show_/print_/prelude_free_fns/mono_hash_stability/typeclass_22b/mq3). - Bench: `cross_lang.py` exit 0, 25/25 stable; `compile_check.py` - exit 0 with 4 `check_ms.*` ms-level noise-class regressions - (`hello`, `borrow_own_demo`, `bench_tree_walk`, `bench_hof_pipeline`); - `check.py` exit 0 with 3 noise-class regressions - (`latency.implicit_at_rc.{p99_9_us, max_us}` first-sightings in - this iter) + 4 noise-class improvements - (`latency.explicit_at_rc.{p99_us, p99_over_median}` improvements - continuing the documented audit-cma → audit-24 lineage). 10th - consecutive observation of metric-identity-migrating noise; - baseline pristine, conservative-call convention holds per the - documented lineage. - -## Concerns - -- T5 pre-extraction reading vs plan: the plan said site 1's unbound- - meta path "likely a `continue` without push" — actual pre-extraction - code did `type_args.push(Type::unit())` for unbound metas (iter 23.4 - behaviour, comment in place). I preserved the actual behaviour, not - the plan's guess. Lockstep with site 2 (which has the same Unit-default - policy) is improved post-extraction. Observation, not correctness. - -- T2 plan draft had three data-shape mismatches against actual Rust - schema (`Term::App.callee` vs plan's `fn`, `Vec` vs - `Vec`, exhaustive Term variants). The plan explicitly - flagged these as "implementer adapts to the actual field name" - cases; not plan defects but recon imprecisions. - -## Known debt - -- audit-24 [medium-3] negative-test coverage breadth → deferred to - P3 downstream-corpus-migration milestone (carried over from - audit-24, not closed here). -- audit-24 [low-1] bench/architect_sweeps.sh noise → carry-on, - pre-milestone-24 lines not new drift. - -## Files touched - -- `docs/DESIGN.md` — new subsection (T1), +68 lines. -- `crates/ail/tests/codegen_import_map_fallback_pin.rs` — new pin (T2). -- `crates/ail/tests/polyfn_dot_qualified_branch_pin.rs` — new pin (T3). -- `crates/ailang-check/src/lib.rs` — one-line tightening at :2858 (T4). -- `crates/ailang-check/src/mono.rs` — new helper + two call-site - replacements (T5). - -## Stats - -`bench/orchestrator-stats/2026-05-13-iter-24.tidy.json` diff --git a/docs/journals/2026-05-13-iter-bugfix-instance-body-unbound-var.md b/docs/journals/2026-05-13-iter-bugfix-instance-body-unbound-var.md deleted file mode 100644 index f941c3d..0000000 --- a/docs/journals/2026-05-13-iter-bugfix-instance-body-unbound-var.md +++ /dev/null @@ -1,94 +0,0 @@ -# iter bugfix-instance-body-unbound-var — instance method bodies now walk through check_fn - -**Date:** 2026-05-13 -**Started from:** 72f3f6541b3e41f2d4a7706dcad5bd784ec07d81 -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -Fixes the bug surfaced by the 2026-05-13 form-a fieldtest: an unbound -identifier inside an `instance` method body slipped past `ail check` -and surfaced only at `ail build` as the degraded internal-error -diagnostic `monomorphise_workspace: unknown identifier: ...`. Root -cause: `check_def`'s `Def::Class(_) | Def::Instance(_)` arm at -`crates/ailang-check/src/lib.rs:1574-1595` early-returned `Ok(())` -without ever invoking the body walker — the comment claimed "instance- -body typechecking (with class-method substitution) land[s] in 22b.2", -but that wiring was never implemented. Workspace-load coherence checks -(`build_registry`'s Orphan/Duplicate/MissingMethod) only inspect -instance schema, not method-body identifier graphs. - -Fix is the minimum-scope change at that call site: split the combined -arm so `Def::Class(_)` continues to be schema-only, while -`Def::Instance(inst)` calls a new helper `check_instance` that, for -each `InstanceMethod`, lifts the body `Term::Lam` into a synthetic -`FnDef` (class-method-substituted via `substitute_rigids` / -`substitute_rigids_in_term` against the class's `param` name as -declared in `env.class_methods[(qualified_class, method)]`) and -hands it to the existing `check_fn`. The reuse of `check_fn` is what -gets the body walked through the same identifier-resolution path as -fn bodies, so an unbound name fires `[unbound-var]` at `ail check` -with exit 1 and the standard structured diagnostic. - -Per the carrier's "minimal fix, no surrounding cleanup" constraint: -no widening to `Def::Class` (still schema-only); no changes to the -`monomorphise_workspace` diagnostic wording (the defence-in-depth -internal-error path stays as a backstop); no changes to mono.rs, -codegen, or any other crate. - -## Per-task notes - -- iter bugfix-instance-body-unbound-var.1: split the - `Def::Class(_) | Def::Instance(_)` arm into `Def::Class(_) => - Ok(())` and `Def::Instance(inst) => check_instance(inst, env, - out_warnings)`. New free fn `check_instance` reads - `env.class_methods[(qualify_class_ref_in_check(&inst.class, - &env.current_module), im.name)]` to recover the class's `param` - name, builds a `{class_param: inst.type_}` substitution, applies - it to the Lam's `param_tys` / `ret_ty` and to the Lam's body - Term (via the existing `substitute_rigids` / - `substitute_rigids_in_term` helpers), then constructs a synthetic - `FnDef` (`name = "::"`, `ty = - Type::fn_implicit(subst_param_tys, subst_ret_ty, - lam.effects)`, params from Lam, body = substituted Lam body) - and calls `check_fn` on it. - - First-attempt sketch wrapped the Lam's `param_tys` / `ret_ty` - unsubstituted on the assumption that the on-disk canonical form - was post-substitution — the `codegen_import_map_fallback_pin` - regression (and a sweep of the prelude `(typed x a) (typed y a)` - shape) immediately disproved that. The substitution step was - added at the second iteration of the implementer phase, before - reaching the spec / quality phases. - - Non-Lam method bodies are silently skipped — current schema - doesn't admit them and the pre-fix behaviour was also silent. - -## Concerns - -- None. - -## Known debt - -- `check_instance` does **not** verify that the substituted method - signature matches the corresponding `ClassMethodEntry.method_ty` - (with `class_param ↦ inst.type_`). The body-walk catches unbound - vars and effect-mismatches against the Lam's own declared types, - but a malformed instance whose Lam declares - `(typed x (con Int)) (typed y (con Int)) -> Bool` for an - `instance Eq Bool` body would currently typecheck cleanly. The - workspace-load `MissingMethod` / coherence checks only inspect - the (class, type) pair, not the method's declared type. Out of - scope per the carrier's "minimal fix" constraint; queued as a - follow-up roadmap candidate ("instance-method declared-type - vs class-method-type cross-check"). - -## Files touched - -- `crates/ailang-check/src/lib.rs` — `check_def` arm split, - new `check_instance` helper. - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-bugfix-instance-body-unbound-var.json diff --git a/docs/journals/2026-05-13-iter-drift-test-narrowing.md b/docs/journals/2026-05-13-iter-drift-test-narrowing.md deleted file mode 100644 index 698dbb5..0000000 --- a/docs/journals/2026-05-13-iter-drift-test-narrowing.md +++ /dev/null @@ -1,52 +0,0 @@ -# iter drift-test-narrowing — design_schema_drift.rs scans §"Data model" only - -**Date:** 2026-05-13 -**Started from:** rustdoc-sweep working-tree state -**Status:** DONE - -## Summary - -`crates/ailang-core/tests/design_schema_drift.rs` is the -drift-detection layer between AST enum definitions in -`crates/ailang-core/src/ast.rs` and the canonical schema -documentation in `docs/DESIGN.md` §"Data model". Each AST variant -must have its JSON-schema anchor (e.g. `"t": "lit"`, `"k": "fn"`) -documented; the tests assert presence of each anchor. - -Pre-iter, the assertion was `DESIGN_MD.contains(anchor)` — -substring search over the whole document. That was loose: an -anchor present only in §"Decision 11" (or any other discussion -section) was treated as documented even though §"Data model" — -the canonical schema reference section — was missing it. The -audit-form-a-precursor `[high]` drift report flagged this as the -silent-pass failure mode. - -This iter narrows the scan. New helper `data_model_section()` -extracts the slice of `DESIGN_MD` from the `## Data model` header -to the next top-level `## ` header. The 7 existing anchor tests -now call `data_model_section().contains(anchor)` instead of -`DESIGN_MD.contains(anchor)`. All 38 anchors verified present in -§"Data model" pre-edit (no regressions). A new test -`data_model_section_is_bounded` pins the extractor itself -(starts-with `## Data model`, does not bleed past `\n## Pipeline`, -length > 1 KB) so a future refactor cannot silently regress the -extractor to whole-document scanning. - -## Files touched - -- `crates/ailang-core/tests/design_schema_drift.rs` (M) - -## Verification - -- `cargo test -p ailang-core --test design_schema_drift` → - 8 passed (was 7; +1 for the bounded-section pin), 0 failed. -- `cargo test --workspace` → 563 passed (was 562; +1 from the - new pin), 0 failed. - -## Concerns - -- (none) - -## Known debt - -- (none) diff --git a/docs/journals/2026-05-13-iter-form-a.0.md b/docs/journals/2026-05-13-iter-form-a.0.md deleted file mode 100644 index 28a77c2..0000000 --- a/docs/journals/2026-05-13-iter-form-a.0.md +++ /dev/null @@ -1,71 +0,0 @@ -# Iter form-a.0 — Prelude Pilot - -**Date:** 2026-05-13 -**Parent spec:** `docs/specs/2026-05-13-form-a-default-authoring.md` -**Parent plan:** `docs/plans/2026-05-13-iter-form-a.0.md` -**Started from:** 4c2a3c5d0811aafb1bce5d6de8ad9a938fd98681 - -## Outcome - -`examples/prelude.ail` rendered from `examples/prelude.ail.json` via -`ail render` (116 lines / 6386 bytes — exact recon match). The two -auto-discovering roundtrip tests in -`crates/ailang-surface/tests/round_trip.rs` both pass on the new -fixture without test-code changes: - -- `every_ail_fixture_matches_its_json_counterpart` — green; the - `prelude.ail` parses to canonical bytes byte-equal to - `prelude.ail.json`. -- `parse_then_print_then_parse_is_idempotent_on_every_ail_fixture` - — green; the Form-A printer is idempotent over prelude. - -`prelude.ail.json` is retained on disk per spec §A2; iter 1 deletes -it alongside the bulk test-infrastructure refactor. This is the -only iteration in the milestone where a fixture is dual-form. - -## Why this iter is the pilot - -Spec §A2 names prelude as the pilot because it exercises the widest -cross-section of language features in a single file: typeclasses -with superclasses (`Ord extends Eq`), polymorphic free functions -with constraints (`forall where Show a fn print`), IO-effect -signatures, primitive ctor patterns, and the loader's auto-injection -path. A green roundtrip on prelude is strong evidence the bulk -migration in iter 1+ will not surface render or parse bugs. - -## Verifications run - -- `cargo run -q --bin ail -- render examples/prelude.ail.json > examples/prelude.ail` — exit 0 -- `wc -lc examples/prelude.ail` — `116 6386` (exact recon prediction) -- `head -n 5 examples/prelude.ail` — first line `(module prelude`, second ` (data Ordering` (exact recon prediction) -- `cargo test -p ailang-surface --test round_trip every_ail_fixture_matches_its_json_counterpart` — PASS (1/1) -- `cargo test -p ailang-surface --test round_trip parse_then_print_then_parse_is_idempotent_on_every_ail_fixture` — PASS (1/1) -- `cargo test --workspace` — PASS, **558 tests green**, identical to pre-iter baseline (audit-24 / iter-24.tidy lineage). - -## Scope NOT touched - -Per spec §"Iteration scope": no test-infrastructure changes, no -CLAUDE.md / DESIGN.md edits, no deletion of `prelude.ail.json`, -no migration script yet. Those land in iter 1. - -## INDEX append — Boss-side - -Plan Task 2 Step 2 asks for an INDEX.md append. Per the -`ailang-implement-orchestrator` Iron Law, `docs/journals/INDEX.md` -is Boss-only. The orchestrator does not touch it. The Boss appends -the line at commit time. Suggested INDEX line (from the plan -verbatim, byte/line counts confirmed by Task 1 Step 2): - -``` -- 2026-05-13 — iter form-a.0: prelude pilot for Form-A-as-default-authoring milestone — examples/prelude.ail rendered (116 lines / 6386 bytes) via `ail render`; two auto-discovering roundtrip tests (every_ail_fixture_matches_its_json_counterpart + parse_then_print_then_parse_is_idempotent_on_every_ail_fixture) green on the new fixture without test-code changes; prelude.ail.json retained per spec §A2 (singular dual-form iter, deletion lands iter 1 with bulk test-infra refactor); cargo test --workspace green → 2026-05-13-iter-form-a.0.md -``` - -## Files touched - -- `examples/prelude.ail` (new, ~6386 bytes / 116 lines) -- `docs/journals/2026-05-13-iter-form-a.0.md` (this file) -- `bench/orchestrator-stats/2026-05-13-iter-form-a.0.json` (stats) - -## Stats - -`bench/orchestrator-stats/2026-05-13-iter-form-a.0.json` diff --git a/docs/journals/2026-05-13-iter-form-a.1.md b/docs/journals/2026-05-13-iter-form-a.1.md deleted file mode 100644 index 327c526..0000000 --- a/docs/journals/2026-05-13-iter-form-a.1.md +++ /dev/null @@ -1,431 +0,0 @@ -# iter form-a.1 — Big-bang migration (Tasks 1-5: additive phase + relocation) - -**Date:** 2026-05-13 -**Started from:** 1a065b37 (post-iter 24.tidy) -**Status:** DONE (Tasks 1-5 only; Tasks 6-12 deferred to next dispatch) -**Tasks completed:** 5 of 12 - -## Summary - -First half of the form-a-default-authoring milestone-close iter: T1 -adds three new tests (parse-determinism + CLI-pipeline-idempotency + -carve-out-inventory-but-`#[ignore]`'d), T2 bulk-renders the 99 -missing `examples/.ail` files via `ail render` (corpus 58 → -157), T3 + T4 migrate ~26 test files from `ailang_core::load_workspace` -to `ailang_surface::load_workspace` (Group A) and flip subprocess -fixture-suffix strings from `.ail.json` to `.ail` (Group B), T5 -relocates `#[cfg(test)] mod tests` blocks from production source -(`ailang-core/src/hash.rs`, `ailang-core/src/workspace.rs` -non-carve-out subset, `ailang-codegen/src/lib.rs:3717-3799`) to -integration test crates that carry `ailang-surface` as a -dev-dependency, eliminating the production-source dependency on -`.ail.json` fixture reads. The remaining Tasks 6-12 (bench-driver -suffix flips, e2e diff-test rewrite, the actual 156-file -`.ail.json` deletion, retiring obsolete roundtrip tests + -schema_coverage corpus flip, DESIGN.md §Roundtrip Invariant -restatement, CLAUDE.md + DESIGN.md doctrine edits, WhatsNew + -roadmap close) run in the Boss's next dispatch on this iter ID. -`cargo test --workspace` is green at every per-task boundary (560 -passed + 4 ignored at iter-half close; 558 + 3 new − 1 transit -`#[ignore]` on the inventory test). One Task-2 plan defect surfaced -inline and was repaired by pre-emptive forward-pull of Task-3 -migrations. - -## Per-task notes - -- **iter form-a.1.1** — Added `parse_is_deterministic_over_every_ail_fixture` - (round_trip.rs) and `cli_parse_then_render_then_parse_is_idempotent` - (roundtrip_cli.rs); created `crates/ailang-core/tests/carve_out_inventory.rs` - with the 8-file hardcoded list and `#[ignore]`'d until Task 8. - The plan's `cli_parse_then_render_then_parse_is_idempotent` body - used an `ail()` helper that doesn't exist in the repo (assert_cmd - is not a dev-dep); adapted to use `Command::new(ail_bin())` - matching the file's existing pattern. Plan acknowledged this - possibility — "verify before writing the body; if `list_ail_fixtures` - is missing in roundtrip_cli.rs, add a parallel helper at lines 47+." -- **iter form-a.1.2** — Rendered the 99 missing `.ail` files. The - bulk render exposed a recon defect: `load_workspace` (since - ext-cli.1) prefers `.ail` siblings for *imports* even when the - entry is `.ail.json`. The seven Group-A test files whose entries - were `.ail.json` but whose imports gained newly-rendered `.ail` - siblings broke at Task 2's green-gate. Repair pulled forward five - Task-3 migrations (mono_xmod_ctor_pattern, 3 sites in - typeclass_22b2, 1 site in typeclass_22b3, env_construction_pin, - 4 sites in ailang-check/tests/workspace.rs, ctt2_registry_rekey) - plus `#[ignore]`'d the 4 fixture-loading tests in - `ailang-core/src/workspace.rs` `mod tests` with explicit - `form-a.1 transit: Task 5 relocates...` markers. Also added - `ailang-surface = { path = "../ailang-surface" }` to - `crates/ailang-core/Cargo.toml` `[dev-dependencies]` (required - for the ctt2_registry_rekey integration test to use - `ailang_surface::load_workspace`). -- **iter form-a.1.3** — Finished Group A migration across the - remaining 12 Group-A files (10 + the 2 already migrated in T2 - scope-pull). Two files (`mono_recursive_fn.rs`, - `mono_xmod_qualified_ref.rs`) were Group-B-pattern (subprocess - CLI, suffix-only flip) but listed in plan's Group A; treated - per their actual shape. -- **iter form-a.1.4** — Group-B migration: bulk regex flip of - `build_and_run("X.ail.json")` → `build_and_run("X.ail")` (3 - patterns) in e2e.rs across 59 invocations; plus targeted - subprocess `examples/.ail.json` → `.ail` flips at 11 - more e2e.rs sites (6× `ws_main`, 2× `sum`, 1× `escape_local_demo`, - 1× `list_map_poly`, 1× `std_either` left as-is per its raw-bytes - roundtrip semantics). Four additional sites in e2e.rs preserved - intentionally for raw-JSON inspection (`borrow_own_demo`, - `reuse_as_demo`, the `sum.ail.json` diff-test pending Task 7 - rewrite, and the `ail_run_accepts_ail_source...` dual-form smoke). - Four plan-Group-B files (`mono_hash_stability`, `prelude_free_fns`, - `print_mono_body_shape`, `show_mono_synthesis`) were - Group-A-pattern (consumers of `load_workspace`); migrated both - imports AND paths. -- **iter form-a.1.5** — Relocated 10 fixture-loading tests from - `ailang-core/src/workspace.rs` `mod tests` to new - `crates/ailang-core/tests/workspace_pin.rs` (integration test - crate); relocated all 8 tests from `ailang-core/src/hash.rs` - `mod tests` plus 2 in-memory unit tests to - `crates/ailang-core/tests/hash_pin.rs`; relocated 3 tests from - `ailang-codegen/src/lib.rs:3717-3799` to - `crates/ailang-codegen/tests/eq_primitives_pin.rs`; migrated - `ailang-prose/tests/snapshot.rs` (helper + 8 fixtures) to `.ail` - + `ailang_surface::load_module`. Carve-out tests stay in-place - (3× `class_param_in_applied_position` / `superclass_with_wrong_param` - / `constraint_with_unbound_var` for 22b2; 3× `ct1_fixture_*` - for ct.1). Tempdir-based loader tests - (`user_module_named_prelude_is_rejected`, `detects_import_cycle`, - `module_not_found_yields_structured_error`) stay in-place too — - they don't consume the examples corpus, so they don't need - migration for the Form-A goal. Added `ailang-surface` as a - `[dev-dependencies]` entry to `ailang-codegen/Cargo.toml` and - `ailang-prose/Cargo.toml`. Hash.rs has `mod tests {}` placeholder - kept; workspace.rs `mod tests` shrunk from 73 to 57 tests + - 6 carve-outs unchanged. - -## Concerns - -- **e2e.rs raw-JSON-inspect tests** (4 sites: `borrow_own_demo_modes_are_metadata_only`, - `reuse_as_demo_under_rc_uses_inplace_rewrite`, `diff_detects_changed_def`, - `render_parse_round_trip_canonical`, plus the dual-form smoke - `ail_run_accepts_ail_source_with_same_stdout_as_ail_json`) currently - read raw `.ail.json` bytes for substring asserts or byte-roundtrip - comparison. They pass today (the `.ail.json` files still exist - pre-T8) but will break at T8 when `.ail.json` siblings are deleted. - Plan Task 7 only addresses the `diff_detects_changed_def` test (the - one that mutates JSON). The other three are out of T1-5 scope and - inherit to Boss for T6-12 dispatch resolution (either parse-derived - JSON via `ail parse` per T7 pattern, or `#[ignore]` with explicit - marker). -- **`ailang-core/src/workspace.rs` `mod tests`**: 3 tempdir-based - loader-mechanism tests (lines :1743, :1759, :1776 pre-relocation; - current line numbers shifted by the deletions) remain in-place - per implementer-judgement (they don't consume `examples/`, so - the Form-A goal is trivially satisfied). Plan suggested - relocating them for doctrinal cleanup; scope reduction is - deliberate. They continue to test what they always tested - (loader cycle/missing-module/user-prelude-conflict behaviour) - and pass. -- **Plan recon defect on Group-A vs. Group-B classification**: - `mono_hash_stability.rs`, `prelude_free_fns.rs`, - `print_mono_body_shape.rs`, `show_mono_synthesis.rs` were listed - in plan's Group B but actually use `ailang_core::workspace::load_workspace` - directly (Group-A pattern). Migrated correctly by switching both - the import and the path. Recorded so a future recon-tightening - pass on the planner skill can review whether path-grep-only - recon is sufficient for Group A/B classification. -- **`mono_recursive_fn.rs` and `mono_xmod_qualified_ref.rs`** were - symmetric — listed in plan's Group A but actually CLI subprocess - callers (Group-B pattern). Suffix-only flip applied as if Group B. - -## Known debt - -- 4 raw-JSON-inspect tests in e2e.rs need T7-style rewrites or - `#[ignore]` markers before Task 8's `.ail.json` deletion lands - (see Concerns). -- 3 tempdir tests in `ailang-core/src/workspace.rs` `mod tests` - could be relocated as doctrinal cleanup but functionally don't - need to be (see Concerns). -- `mod tests {}` placeholder in `ailang-core/src/hash.rs` could - be deleted entirely; left as a structural breadcrumb pointing at - the integration test crate. - -## Files touched - -Working tree: 37 modified + 14 new test files + 99 new `.ail` -fixtures = 150 paths. Detail: - -### Modified production / test source (37) - -- `crates/ail/tests/`: codegen_import_map_fallback_pin.rs, - ct1_check_cli.rs, e2e.rs (extensive build_and_run + 11 - subprocess sites), eq_float_noinstance.rs, eq_ord_e2e.rs, - floats_e2e.rs, ir_snapshot.rs, mono_hash_stability.rs, - mono_recursive_fn.rs, mono_unification.rs, mono_xmod_ctor_pattern.rs, - mono_xmod_qualified_ref.rs, mq3_multi_class_e2e.rs, - polyfn_dot_qualified_branch_pin.rs, prelude_free_fns.rs, - print_mono_body_shape.rs, roundtrip_cli.rs, show_mono_synthesis.rs, - show_no_instance_e2e.rs, show_print_e2e.rs, typeclass_22b2.rs, - typeclass_22b3.rs, typeclass_22c.rs (23 files) -- `crates/ailang-check/tests/`: env_construction_pin.rs, - method_collision_pin.rs, show_dispatch_pin.rs, workspace.rs (4) -- `crates/ailang-codegen/`: Cargo.toml + src/lib.rs (2) -- `crates/ailang-core/`: Cargo.toml + src/hash.rs + src/workspace.rs - + tests/ctt2_registry_rekey.rs (4) -- `crates/ailang-prose/`: Cargo.toml + tests/snapshot.rs (2) -- `crates/ailang-surface/tests/round_trip.rs` (1) -- `Cargo.lock` (1) - -### New (14 test files + 99 rendered .ail fixtures) - -- `crates/ailang-codegen/tests/eq_primitives_pin.rs` (relocated T5) -- `crates/ailang-core/tests/hash_pin.rs` (relocated T5) -- `crates/ailang-core/tests/workspace_pin.rs` (relocated T5) -- `crates/ailang-core/tests/carve_out_inventory.rs` (new T1) -- 99× `examples/.ail` rendered via `ail render` (T2) - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-form-a.1.json - ---- - -# iter form-a.1 — Big-bang migration (Tasks 6-12: bench drivers, e2e rewrites, deletion, doctrine, milestone close) - -**Date:** 2026-05-13 -**Started from:** 9332d1e (post-T1-5 commit 77b28ad + plan-amendment 9332d1e) -**Status:** DONE (Tasks 6-12; iter form-a.1 fully closed) -**Tasks completed:** 7 of 7 (this dispatch); 12 of 12 (iter total) - -## Summary - -Second half of the form-a-default-authoring milestone-close iter: T6 -flips the bench-driver fixture suffix from `.ail.json` to `.ail` (4 -Python scripts + run.sh). T7 re-authors three e2e tests -(`diff_detects_changed_def`, `borrow_own_demo_modes_are_metadata_only`, -`reuse_as_demo_under_rc_uses_inplace_rewrite`) to derive their -canonical-JSON via `ail parse` on the `.ail` source rather than -raw-reading the soon-to-be-deleted `.ail.json` file, retires the -redundant `render_parse_round_trip_canonical` (subsumed by T1's -`cli_parse_then_render_then_parse_is_idempotent`), and re-authors -`ail_run_accepts_ail_source_with_same_stdout_as_ail_json` to derive -a tempdir-based `hello.ail.json` from `hello.ail` via `ail parse` -rather than reading a deleted `.ail.json` sibling. T8 deletes the -156 non-carve-out `.ail.json` files (count drops 164 → 8), removes -the `#[ignore]` on the carve-out inventory test, and pulls forward -20 stale-`.ail.json`-reference repairs that the T1-5 dispatch missed -(12 `let example = "X.ail.json"` Group-B sites + 5 -`ailang_core::load_workspace` Group-A sites in e2e.rs + 3 `ail_run` -Group-B sites in typeclass_22b3.rs); to keep T8's green-gate, T9 -Step 5 (schema_coverage corpus flip from `.ail.json` to `.ail`) is -also pulled forward into T8. T9 retires the 3 obsolete roundtrip -tests (`print_then_parse_round_trips_every_fixture`, -`every_ail_fixture_matches_its_json_counterpart`, -`cli_render_then_parse_preserves_canonical_bytes_on_every_fixture`) -plus the dead helpers `list_json_fixtures` (×2), -`round_trip_one`, and `strip_trailing_newlines`. T10 restates the -DESIGN.md §Roundtrip Invariant with parse-determinism + idempotency -+ CLI-pipeline-idempotency + carve-out-anchor framing, replacing -the old Direction-1/Direction-2 framing and the five enforcement -bullets with the five surviving test names. T11 replaces the -CLAUDE.md and DESIGN.md doctrine sentences per spec §A4. T12 -appends the WhatsNew milestone-close entry, strikes the roadmap -entry (`- [x]` + `(Closed 2026-05-13 by iter form-a.1.)`), and -verifies final inventory (8 `.ail.json` carve-outs + 157 `.ail`). -Final test count: 557 green + 3 ignored (3 pre-existing doctests). -The `cargo test --workspace` baseline holds green at every per-task -boundary; both `python3 bench/compile_check.py` and -`python3 bench/cross_lang.py` exit 0. The milestone -`[Form-A as the default authoring surface]` is fully closed. - -## Per-task notes - -- **iter form-a.1.6** — Bench-driver suffix flips in - `bench/compile_check.py:43,101`, `bench/cross_lang.py:48,80`, - `bench/mono_dispatch.py:10,112`, `bench/run.sh:68,182,183`. - Mechanical edit; comment text and path string updated at each - site. `python3 bench/compile_check.py` exit 0; `python3 - bench/cross_lang.py` exit 0. (First compile_check run reported - exit-1 due to sub-millisecond `check_ms` noise on `hello` and - `borrow_own_demo`; second run clean. The noise is unrelated to - the migration — the bench measures `ail check` runtime, and the - Python edits did not touch the binary path.) -- **iter form-a.1.7** — Re-authored 3 raw-JSON-inspect tests + - retired 1 + re-authored the dual-form smoke. Pattern for the - three substring-assert tests: replace - `std::fs::read*("examples/.ail.json")` with a `Command::new(ail_bin()).args(["parse", "examples/.ail", "-o", tmpfile])` - + `read_to_string(tmpfile)`. `reuse_as_demo_under_rc_uses_inplace_rewrite` - additionally switches its `ailang_core::load_workspace(&json_path)` - call to `ailang_surface::load_workspace(&ail_path)` (the - IR-lowering path no longer needs the JSON file). The dual-form - smoke (`ail_run_accepts_ail_source_with_same_stdout_as_ail_json`) - uses a per-process tempdir with `hello.ail.json` as the file - basename — necessary because the loader name-checks the module - name against the file stem, and the first attempt at a - tempfile-with-PID-suffix path tripped that check. Net test - count: −1 (retirement of `render_parse_round_trip_canonical`). - 559 green at task close. -- **iter form-a.1.8** — CRITICAL task. Before the deletion command, - 20 stale-`.ail.json`-reference repairs were pulled forward - (T1-5 dispatch missed these despite the plan's Group-A/B scope): - (a) 12 `let example = "X.ail.json"` patterns in `e2e.rs` flipped - to `"X.ail"` via sed; (b) 5 `ailang_core::load_workspace(&json_path)` - sites in `e2e.rs` switched to `ailang_surface::load_workspace` - (the `let example = "X.ail"` from (a) feeds into `workspace.join("examples").join(example)`, - which is `.ail` post-flip — needs the extension-dispatching loader); - (c) 3 `ail_run("X.ail.json")` sites in `typeclass_22b3.rs` flipped - via sed. Cargo-test green pre-deletion at 559. The plan's deletion - bash loop ran clean (exit 0); `ls examples/*.ail.json | wc -l` → - 8, `ls examples/*.ail | wc -l` → 157. The `#[ignore]` attribute - on `examples_ail_json_inventory_matches_carve_outs` was removed; - the test passes (8 files match `EXPECTED`). Post-deletion - cargo-test surfaced 1 failure - (`every_ast_variant_is_observed_in_the_fixture_corpus` in - `schema_coverage.rs`) because its corpus walker still looked at - `.ail.json` (now down to 8 carve-outs, insufficient to cover all - AST variants). The plan-T8 vs. T9 ordering implicitly assumed T9 - Step 5 (schema_coverage corpus flip) would land before T8 — but - T9 is plan-ordered after T8. To satisfy T8's green-gate, T9 - Step 5 was pulled forward inline (helper rename - `list_json_fixtures` → `list_ail_fixtures`, filter `.ail.json` - → `.ail`, loader `ailang_core::load_module` → - `ailang_surface::load_module`). Post-fix: 560 green + 3 ignored - (was 559 pre-T8 with 4 ignored including carve_out_inventory; - un-ignore + correct schema_coverage flip = +1 net = 560). -- **iter form-a.1.9** — Retired 3 obsolete roundtrip tests + - cleanups. `round_trip.rs`: retired - `print_then_parse_round_trips_every_fixture` (lines 57-101 in - pre-iter numbering) and `every_ail_fixture_matches_its_json_counterpart` - (lines 120-204); retired the `list_json_fixtures` helper and - the `round_trip_one` helper (only consumers were the two retired - tests); updated the file's doc-comment to describe the two - surviving tests (`parse_is_deterministic_*` and - `parse_then_print_then_parse_*`); removed the unused `Path` - import (only `PathBuf` remains). `roundtrip_cli.rs`: retired - `cli_render_then_parse_preserves_canonical_bytes_on_every_fixture` - (lines 116-146 pre-iter) and the `list_json_fixtures` + - `roundtrip_one` + `strip_trailing_newlines` helpers; updated the - doc-comment. T9 Step 5 (schema_coverage corpus flip) was already - done in T8; nothing to do here. 557 green + 3 ignored at task - close (560 pre-T9 − 3 retired = 557). -- **iter form-a.1.10** — DESIGN.md §Roundtrip Invariant restatement. - Lines 2029-2048 (Direction 1/Direction 2 framing) replaced with - the new 4-property framing (parse-determinism + - idempotency-under-print + CLI-pipeline-idempotency + - carve-out-anchor) verbatim from the plan. Lines 2061-2094 - (Enforcement bullets, five tests) replaced with the five - surviving tests verbatim. Float-literals subsection (lines - 2050-2059 pre-edit, 2065-2074 post-edit) preserved. Why-anchored-at-top-level - subsection (lines 2096-2108 pre-edit, 2108-2120 post-edit) - preserved. The numbered prose (4 properties) and the bulleted - enforcement (5 tests) are internally consistent; the carve-out - inventory is the 5th enforcement bullet, covering property 4 - ("carve-out anchor") as a meta-property. -- **iter form-a.1.11** — Doctrine edits. CLAUDE.md:5-6 sentence - replaced verbatim with spec §A4 wording. DESIGN.md:465-466 first - sentence replaced verbatim with spec §A4 wording; second - sentence ("Form (A) is one projection; the JSON-AST stays - canonical.") preserved as the trailing sentence of the new - replacement block. Doc-only; tests unaffected. -- **iter form-a.1.12** — Milestone close. WhatsNew.md entry - appended ("Form A is now the authoring surface", per-boss - editorial rules: no crate names, no iter codes, lead with the - change, factual). docs/roadmap.md entry at line 97 changed from - `- [ ]` to `- [x]` with `(Closed 2026-05-13 by iter form-a.1.)` - prepended. Final cargo-test: 557 green + 3 ignored. Final bench: - both compile_check and cross_lang exit 0. Final inventory: - exactly 8 `.ail.json` (carve-outs alphabetically: broken_unbound, - prelude, test_22b2_invalid_superclass_param, - test_22b2_kind_mismatch, test_22b2_unbound_constraint_var, - test_ct1_bad_qualifier, test_ct1_bare_xmod_rejected, - test_ct1_qualified_class_rejected) + 157 `.ail`. - -## Concerns - -- **Forward-pull of T9 Step 5 into T8.** The plan ordered T8 - (deletion) before T9 (test retirement + schema_coverage flip), - but T8's green-gate required schema_coverage to be flipped - *first* — otherwise the schema_coverage corpus walker shrinks - from 164 `.ail.json` to 8 carve-outs and the AST-variant - coverage assertion fails. Pulled T9 Step 5 inline into T8; - T9 then had no schema_coverage work left to do. Recorded so a - future planner-skill recon pass can detect this ordering - defect class (a Task K green-gate depending on an edit - scripted for Task K+N). -- **T1-5 Group-A/B classification gaps.** The T1-5 dispatch's - Concerns flagged 4 raw-JSON-inspect e2e.rs tests but missed - 20 additional sites of the same kind in two patterns: 12 - `let example = "X.ail.json"` Group-B-pattern sites in e2e.rs - (feeds `build_and_run_with_alloc`) + 5 - `ailang_core::load_workspace` Group-A-pattern sites in e2e.rs - + 3 `ail_run` Group-B sites in typeclass_22b3.rs. All repaired - inline in T8 before the deletion command ran. The pattern that - T1-5 missed is "binding pulled out of the call expression" — - the `let example = "X.ail.json"` introduces a local that the - Group-B sed for `build_and_run("X.ail.json")` doesn't match. - Worth noting for a future planner-recon tightening pass. -- **bench/compile_check.py first-run flakiness.** The first run - after T6 reported exit-1 because two `check_ms` metrics on - sub-millisecond timings (`hello`, `borrow_own_demo`) drifted - past the 25% tolerance; the second run was exit-0. The drift is - measurement noise (system load + ms-scale latency), not a - migration-induced regression. Final T12 bench run was exit-0 - on both scripts. - -## Known debt - -- **None known.** The iter closes the milestone cleanly. Two - follow-up roadmap items already exist: - - `[milestone] Prelude embed: Form-A as compile-time source` - (queued; retires carve-out (b) `prelude.ail.json`). - - The Show/print fieldtest example-corpus migration (separately - queued; orthogonal to this milestone). -- Cosmetic: the `mod tests {}` placeholder in - `ailang-core/src/hash.rs` from T5 is still in-tree; could be - deleted entirely in a doctrinal cleanup pass. Functionally - harmless. - -## Files touched (Tasks 6-12) - -### Modified bench (4) - -- `bench/compile_check.py` -- `bench/cross_lang.py` -- `bench/mono_dispatch.py` -- `bench/run.sh` - -### Modified test source (5) - -- `crates/ail/tests/e2e.rs` — T7 re-authoring of 5 tests; T8 - forward-pull of 12 Group-B + 5 Group-A sites -- `crates/ail/tests/typeclass_22b3.rs` — T8 forward-pull of 3 - Group-B sites -- `crates/ail/tests/roundtrip_cli.rs` — T9 retirement + - helpers cleanup + doc-comment update -- `crates/ailang-surface/tests/round_trip.rs` — T9 retirement + - helpers cleanup + doc-comment update + import cleanup -- `crates/ailang-core/tests/schema_coverage.rs` — T8 corpus flip - to `.ail` (pulled forward from T9 Step 5) - -### Modified test discipline (1) - -- `crates/ailang-core/tests/carve_out_inventory.rs` — T8 un-ignored - -### Modified doc (4) - -- `CLAUDE.md` — T11 doctrine sentence (lines 5-6) -- `docs/DESIGN.md` — T10 §Roundtrip Invariant restatement; T11 - §"What this Decision deliberately does not do" first sentence -- `docs/WhatsNew.md` — T12 milestone-close entry -- `docs/roadmap.md` — T12 strike of the Form-A milestone entry - -### Deleted (T8) — 156 files - -`examples/*.ail.json` — every non-carve-out fixture (the eight -carve-outs `broken_unbound`, `prelude`, -`test_22b2_invalid_superclass_param`, `test_22b2_kind_mismatch`, -`test_22b2_unbound_constraint_var`, `test_ct1_bad_qualifier`, -`test_ct1_bare_xmod_rejected`, `test_ct1_qualified_class_rejected` -preserved). - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-form-a.1.json diff --git a/docs/journals/2026-05-13-iter-form-a.tidy.md b/docs/journals/2026-05-13-iter-form-a.tidy.md deleted file mode 100644 index c91c392..0000000 --- a/docs/journals/2026-05-13-iter-form-a.tidy.md +++ /dev/null @@ -1,103 +0,0 @@ -# iter form-a.tidy — close 3 of 4 documentary drift items from audit-form-a + fix 5 contradictory "seven carve-outs" sites - -**Date:** 2026-05-13 -**Started from:** 5e94204c218c410d44fbb8e658b677a7294744b0 -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -Pure documentary tidy iteration. Three new sections landed in -`crates/ailang-core/specs/form_a.md` (Class declarations, Instance -declarations, Constraints on polymorphic fns), each anchored to a -real corpus fixture (`test_22c_user_class_e2e.ail`, -`mq3_class_eq_vs_fn_eq_classmod.ail`, `show_user_adt.ail`, -`cmp_max_smoke.ail`) and each fixture was `ail check`-verified to -produce the expected symbol counts before close. Six sites in -`docs/specs/2026-05-13-form-a-default-authoring.md` had "seven -carve-outs" tightened to "eight carve-outs" to match the §C4(b) -amendment. The empty `#[cfg(test)] mod tests {}` placeholder plus -its 6-line relocation comment was deleted from -`crates/ailang-core/src/hash.rs` (tests live in -`crates/ailang-core/tests/hash_pin.rs` since form-a.1 T5). The -`round_trip.rs` module-level and inner docstrings were rewritten -to use the post-T10 four-property framing (parse-determinism / -idempotency-under-print / CLI-pipeline-idempotency / -carve-out-anchor) instead of the retired Direction-1/Direction-2 -language, with cross-references to the sibling enforcement points -in `crates/ail/tests/roundtrip_cli.rs` and -`crates/ailang-core/tests/carve_out_inventory.rs`. - -No production-code behaviour changed. Workspace test count holds -at 559 passed at every per-task gate. - -## Per-task notes - -- iter form-a.tidy.1: `form_a.md` Class declarations section landed; §Definitions - intro updated `Three kinds` → `Five kinds`; example anchored to - `examples/test_22c_user_class_e2e.ail` (`ok (24 symbols across 2 modules)`). -- iter form-a.tidy.2: `form_a.md` Instance declarations section landed; two - examples (same-module abbreviated from `mq3_class_eq_vs_fn_eq_classmod.ail`, - cross-module from `show_user_adt.ail`); `bare-cross-module-class-ref` - diagnostic anchor named inline. -- iter form-a.tidy.3: `form_a.md` Constraints on polymorphic fns: extended - the four-shapes EBNF `(forall ...)` line with optional `(constraints ...)` - clause, added explanatory paragraph (with `no-instance` diagnostic anchor), - added fifth example to the Examples block anchored to `cmp_max_smoke.ail`. -- iter form-a.tidy.4: `docs/specs/2026-05-13-form-a-default-authoring.md` - six "seven" → "eight" edits across the preamble + §C1 + §C2 + §C3 + §"Data flow" - (two sites). Post-edit grep returned 4 surviving "seven" mentions at lines - 233, 238, 463, 469 — all correctly §C4(a)-scoped or arithmetic/future-state - (see Concerns below for one carrier-vs-plan transcription drift). -- iter form-a.tidy.5: `hash.rs` empty `mod tests {}` placeholder + 6-line - relocation comment block deleted (9 lines including the preceding blank). - `hash_pin.rs` integration tests still 10/10 passing. -- iter form-a.tidy.6: `round_trip.rs` module-level `//!` and inner `///` - rewritten to four-property framing with sibling-crate breadcrumbs. - `grep -n 'Direction [12]'` now zero hits. - -## Concerns - -- T4 carrier-vs-plan transcription drift on the Step 6 sanity-grep expected - output. The plan's Step 6 expected-output block listed four lines (233, - 238, 463, 469) but the carrier compressed this to "exactly the three - surviving lines named in T4 Step 6 (lines 238, 463, 469)". The actual - post-edit grep returns four lines matching the plan listing; line 233 - is §C4(a)-scope ("(seven files; the canonical") and is correctly retained. - No edit was missed. The carrier's three-line restatement was wrong; - the plan's four-line listing was right. Recording so a future auditor - re-reading the carrier doesn't second-guess the result. - -## Known debt - -- (none) - -## Files touched - -- `crates/ailang-core/specs/form_a.md` (+105/-) -- `crates/ailang-core/src/hash.rs` (-9) -- `crates/ailang-surface/tests/round_trip.rs` (+19/-12) -- `docs/specs/2026-05-13-form-a-default-authoring.md` (six 1-line edits) - -## Phase 3 (E2E coverage) - -Deliberately skipped per carrier: pure-documentary iter, no -behaviour change to cover. Verification gate was the per-task -`cargo test --workspace` smoke (559 green at every task close) -plus the per-task `ail check` runs on the four corpus fixtures -cited in the new form_a.md sections, all producing the expected -symbol counts (`test_22c_user_class_e2e.ail` ok 24/2, -`mq3_class_eq_vs_fn_eq_classmod.ail` ok 22/2, -`show_user_adt.ail` ok 23/2, `cmp_max_smoke.ail` ok 22/2). - -## Decision recorded — Drift item B2 (plan-file "seven carve-outs" orphans) - -Per plan §"Decision recorded" and the recon report: `docs/plans/2026-05-13-iter-form-a.1.md` -contains zero defective `seven` mentions; its four hits are all internally -scoped to §C4(a) (which IS seven), arithmetic, or future-state. The -audit-form-a journal's "two sites" claim against the plan file did not -match its contents at HEAD. No plan-file edit included in this iter. - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-form-a.tidy.json diff --git a/docs/journals/2026-05-13-iter-mq.1.md b/docs/journals/2026-05-13-iter-mq.1.md deleted file mode 100644 index bc3ca96..0000000 --- a/docs/journals/2026-05-13-iter-mq.1.md +++ /dev/null @@ -1,176 +0,0 @@ -# iter mq.1 — Canonical-form extension for class-ref fields + workspace internal qualification - -**Date:** 2026-05-13 -**Started from:** 1a5f8289b7fbad3ae5bdd6badde27c80eca6dad3 -**Status:** DONE -**Tasks completed:** 7 of 7 - -## Summary - -mq.1 lands the canonical-form rule for the three class-reference -schema fields (`InstanceDef.class`, `Constraint.class`, -`SuperclassRef.class`) — bare for same-module, `.` -for cross-module — symmetric to ct.1's `Type::Con.name` rule. -`ClassDef.name` stays bare (defining site, like `TypeDef.name`). -ct.1's `validate_canonical_type_names` validator gains a sibling -walk for class refs via the new `check_class_ref` helper, with -two new sibling diagnostics `BareCrossModuleClassRef` and -`BadCrossModuleClassRef`. Workspace-internal class-name keys -(`class_def_module`, `class_by_name`, registry `entries.0`, -`ClassMethodEntry.class_name`, `class_superclasses`, mono's -`class_index`, `Origin::Class.class_name`) all carry the qualified -form post-mq.1; the on-disk schema value is the canonical form (bare -or qualified) that `qualify_class_ref` lifts to the workspace key -at every lookup boundary. `MethodNameCollision` and the dispatch -path are unchanged in this iter (iter 2 + 3 land them). - -## Per-task notes - -- iter mq.1.1: two new diagnostic variants - `BareCrossModuleClassRef` / `BadCrossModuleClassRef` sibling to the - type-ref variants; two Diagnostic-shape Display arms in `ail/main.rs` - mirroring the existing arm. 2 RED unit tests, both GREEN. -- iter mq.1.2: doc-comments on the four class-reference fields - in `ast.rs` (`ClassDef.name` stays bare with cross-reference; - `SuperclassRef.class` / `InstanceDef.class` / `Constraint.class` - document canonical form). Zero semantic effect. -- iter mq.1.3: `check_class_ref` helper plus three per-def walks - inside `validate_canonical_type_names`; `check_class_name_fields` - narrowed to `ClassDef.name` only; four in-test pin tests inverted - (qualified is now accepted); on-disk `test_ct1_qualified_class_rejected` - fixture-test repurposed (now expects `OrphanInstance` — - the natural post-mq.1 rejection mode for the same shape); positive - on-disk fixture pair `mq1_xmod_constraint_class{,_dep}` shipped - plus the corresponding pin test. 4 new bare-xmod tests + 1 positive - qualified test, all GREEN. -- iter mq.1.4: `qualify_class_ref` helper at module scope; `build_registry` - Pass-1 re-keyed (qualified); Pass-2 lookups use `qualify_class_ref` to - lift `inst.class` to the registry-key shape; superclass walk threads - `current_class_module` to qualify bare `SuperclassRef.class`; - `Origin::Class.class_name` qualified (one construction site, not two as - the plan claimed); type-leg of coherence check now also handles - qualified head names symmetrically (split-and-lookup). -- iter mq.1.5: `ClassMethodEntry.class_name` insert qualifies in - `build_module_globals`; `class_superclasses` key + value both - qualified; mono's `build_class_index` keyed qualified (plan said - "no code change needed in mono.rs consumers" — that statement was - wrong, the build-side needed the matching qualification). -- iter mq.1.6: test assertions updated for the new qualified shape — - the two `MethodNameCollision` pin tests, `typeclass_22b2.rs:42` - (`class_method_class("show")` now `Some("...Show")`), - `typeclass_22b3.rs:151` (`MonoTarget.class` qualified). The other - lines plan flagged (217/232/295/301/341/353) are local-construction - tests that never round-trip through the workspace — left as-is. -- iter mq.1.7: full `cargo test --workspace` 520 passed / 0 failed; - `examples/prelude.ail.json` zero diff (confirms spec assumption 17: - intra-prelude refs stay bare); `bench/compile_check.py` exit 0, - `bench/cross_lang.py` exit 0; `bench/check.py` exit 1 due to 4 - improvements-beyond-tolerance (audit-ratifiable per convention, - not a regression). Roundtrip on the new fixture: `ail check - examples/mq1_xmod_constraint_class.ail.json` → "ok (16 symbols - across 3 modules)", exit 0. - -## Concerns - -- **Boss-note recon claim was empirically wrong.** The boss-note - said "every existing `examples/test_22b*.ail.json` class-ref field - is intra-module under the canonical-form rule, so existing fixtures - carry zero diff". Five test_22b* fixtures actually carried bare - cross-module class refs and required migration: `test_22b1_orphan_third.ail.json` - (bare `Show` from imported classmod), `test_22b1_dup_a.ail.json` + - `test_22b1_dup_b.ail.json` (the duplicate-instance pair structurally - relied on the pre-mq.1 bare cross-module shape), and - `test_22b2_unbound_constraint_var.ail.json` (constraint references - a class that doesn't exist anywhere → fired BareCrossModuleClassRef - before reaching UnboundConstraintTypeVar). Six non-test_22b - fixtures also needed migration (`eq_ord_polymorphic`, `eq_ord_user_adt`, - `cmp_max_smoke`, `ctt2_collision_lib`, `ctt2_collision_main`). All - migrations are minimal and aligned with the canonical-form rule; - no proactive "fixture cleanup" added. -- **Duplicate-instance test restructured by structural necessity.** - Pre-mq.1 the test relied on two coherent instances on the same - `(class, type)` declared from different modules; post-mq.1, the - orphan-freedom invariant prevents this shape (any second instance - in a module that owns neither the class nor the type fires - `OrphanInstance` first). The new `test_22b1_dup_same_module.ail.json` - fixture lands both instances in the same module — the only post-mq.1 - way to land two on the same canonical key. The old dup_a/dup_b/entry - fixtures stay as supporting modules (referenced by hash.rs's - canonical-form hash pin) but no longer participate in the duplicate - test. This is arguably a strengthening of the architecture: duplicate - instances are now structurally impossible across coherent modules. -- **Plan's "no code change needed" claims in Task 4 Step 8 and Task 5 - Step 5 were wrong.** Task 4 Step 8 said `Origin::Class.class_name` - qualified at "both sites" but there's only one construction site - (the plan's recon was off-by-one). Task 5 Step 5 said mono's - `class_index.get(class)` lookups would just work post-rekey, but - `build_class_index` itself was bare-keyed. Both fixed inline. -- **Several adjacent fixes were necessary that the plan didn't anticipate:** - (a) `build_module_globals` rejected `inst.class` containing `.` — - added a `Def::Instance` carve-out so qualified class refs survive. - (b) `check_fn`'s declared-constraint matching compared bare - `c.class` to qualified residual `r.class` → added an inline lift via - `qualify_class_ref_in_check`. (c) `NoInstance` Float-aware message - arm matched on bare `"Eq" / "Ord"` — updated to `"prelude.Eq" / - "prelude.Ord"`. (d) Coherence check's type-leg lookup couldn't - resolve qualified head names — extended with a split-and-lookup - fallback. None of these widened the contract; each is a one-line - bridge keeping the post-mq.1 qualified shape consistent across the - pipeline. -- **Tasks 3-6 ran as one coherent fix.** The plan structured them as - sequential tasks, but Task 3 alone left several existing tests red - (registry still bare-keyed, residual still bare). Walking the plan - literally and expecting "green after Task 3" per the plan's Step 9 - was impossible; the four tasks form one consistent rekey. - -## Known debt - -- The `test_22b1_dup_a` / `test_22b1_dup_b` / `test_22b1_dup_entry` - fixtures are now orphan from the duplicate-instance test (which - uses `test_22b1_dup_same_module` instead). They're still - referenced by `hash.rs:293` for the canonical-form hash pin. A - cleanup pass could either retire them entirely (and replace the - hash pin with the new fixture) or keep them as historical - reference. Deferred — not load-bearing. -- `qualify_class_ref_in_check` in `ailang-check/src/lib.rs` is a - one-line duplicate of `workspace::qualify_class_ref`. The - alternative is exposing `qualify_class_ref` as `pub(crate)` or - re-exporting; the duplication was the minimum-edit path. A - consolidation tidy could land in iter 2. - -## Files touched - -Code: -- crates/ail/src/main.rs -- crates/ail/tests/ct1_check_cli.rs -- crates/ail/tests/typeclass_22b2.rs -- crates/ail/tests/typeclass_22b3.rs -- crates/ailang-check/src/lib.rs -- crates/ailang-check/src/mono.rs -- crates/ailang-check/tests/env_construction_pin.rs -- crates/ailang-core/src/ast.rs -- crates/ailang-core/src/hash.rs -- crates/ailang-core/src/workspace.rs -- crates/ailang-core/tests/ctt2_registry_rekey.rs - -Fixtures (existing, migrated): -- examples/cmp_max_smoke.ail.json -- examples/ctt2_collision_lib.ail.json -- examples/ctt2_collision_main.ail.json -- examples/eq_ord_polymorphic.ail.json -- examples/eq_ord_user_adt.ail.json -- examples/test_22b1_dup_a.ail.json -- examples/test_22b1_dup_b.ail.json -- examples/test_22b1_dup_entry.ail.json -- examples/test_22b1_orphan_third.ail.json -- examples/test_22b2_unbound_constraint_var.ail.json - -Fixtures (new): -- examples/mq1_xmod_constraint_class.ail.json -- examples/mq1_xmod_constraint_class_dep.ail.json -- examples/test_22b1_dup_classmod.ail.json -- examples/test_22b1_dup_same_module.ail.json - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-mq.1.json diff --git a/docs/journals/2026-05-13-iter-mq.2.md b/docs/journals/2026-05-13-iter-mq.2.md deleted file mode 100644 index 43cd54d..0000000 --- a/docs/journals/2026-05-13-iter-mq.2.md +++ /dev/null @@ -1,169 +0,0 @@ -# iter mq.2 — Type-driven dispatch mechanism (installed, not yet exercised) - -**Date:** 2026-05-13 -**Started from:** e9e45c77affa635652505cc937b77b9349d18c8e -**Status:** DONE -**Tasks completed:** 9 of 9 - -## Summary - -mq.2 installs the type-driven dispatch infrastructure that mq.3 will -exercise end-to-end once `MethodNameCollision` retires. Three new -`CheckError` variants (`AmbiguousMethodResolution`, `UnknownClass`, -plus an additive `candidate_classes` field on `NoInstance`); a new -workspace-flat `method_to_candidate_classes` index on `Env` (inverse -of `class_methods`); an extended `ResidualConstraint` carrying an -optional `candidates: Option>` for the multi- -candidate dispatch path; a pure `resolve_method_dispatch` helper -implementing the spec's 5-step rule (qualifier → singleton → -type-driven filter → constraint-driven filter → `Multi` for discharge- -time refinement); a `refine_multi_candidate_residual` helper for -discharge / mono refinement; and a `resolve_residual_class_for_mono` -helper that wires the refinement into mono's residual-to-target -mapping. The synth `Term::Var` arm class-method branch is rewritten -to consult the new index via `parse_method_qualifier`, with a gate -that prevents capturing the existing cross-module-fn path -(`.`) by requiring class-qualifiers to carry an inner -dot (per mq.1's canonical-form rule, class qualifiers are always -`.`). - -With `MethodNameCollision` still active, the new multi-candidate -branch is exercised exclusively by 15 unit tests: 6 in -`tests/method_dispatch_pin.rs` (the 5-step rule's six cases), 6 in -`lib.rs` `mod tests` (the new `CheckError` variants + the -`ResidualConstraint`/`Env` field shapes + the discharge-side -refinement), and 3 in `mono.rs` `mod tests` (the mono-side helper). -Real workspaces continue producing single-class residuals -(`candidates: None`), and every pre-mq.2 fixture typechecks -unchanged. - -## Per-task notes - -- iter mq.2.1: Two new `CheckError` variants - `AmbiguousMethodResolution` and `UnknownClass`, added as siblings - after `NoInstance`. `code()` + `ctx()` table entries; module-doc - list in `diagnostic.rs` extended. 2 RED-then-GREEN smoke tests. -- iter mq.2.2: `NoInstance` extended with an additive - `candidate_classes: Vec` field. `ctx()` emits the field - only when non-empty (preserves pre-mq.2 JSON shape). 2 tests: - with + without populated candidates. -- iter mq.2.3: `ResidualConstraint` extended with `candidates: - Option>`. Struct bumped to `pub` (was - `pub(crate)`) for unit-test access from the dispatch-pin / Task 7 - + 8 tests. Single existing construction site updated with - `candidates: None`. 1 smoke test. -- iter mq.2.4: `Env.method_to_candidate_classes: - BTreeMap>` built workspace-flat in - `build_check_env` alongside the existing `class_methods` insert - (coalesced into one loop). 1 in-test fixture round-trip. -- iter mq.2.5: `MethodDispatchOutcome` enum + pure - `resolve_method_dispatch` helper. 6 unit tests cover unique - candidate, qualifier-match, qualifier-no-match (UnknownClass), - type-driven narrow-to-one, constraint-driven narrow-to-one - (rigid-var fallback), and true ambiguity. Helper uses - `ailang_core::pretty::type_to_string` for the at_type rendering - rather than the plan's invented `format_type_for_display` (one - fewer duplicate; the canonical pretty-printer is already in use - across other diagnostic surfaces). -- iter mq.2.6: Synth `Term::Var` arm class-method branch rewritten - to consult `method_to_candidate_classes` via - `parse_method_qualifier`. Branch gate: qualifier must be either - absent (bare form `"show"`) or contain an inner dot - (`.` per mq.1's canonical-form rule). 1-dot names - like `std_list.length` fall through to the existing qualified-fn - path. Plan's `active_declared_constraints` plumbing intentionally - skipped at synth time — synth's Var arm does not carry the body's - declared constraints; the constraint-driven filter runs at - discharge time (Task 7) where `expanded` is in scope. With - `MethodNameCollision` still active, the candidate set is always - singleton at this branch, so the `&[]` synth-time filter is - load-bearing only post-mq.3 and only for the rigid-var fallback; - on the post-mq.3 multi-candidate fully-concrete path discharge - will refine the residual on `r_ty` against the workspace registry. -- iter mq.2.7: `RefineOutcome` enum + `refine_multi_candidate_residual` - helper. Wired into `check_fn`'s residual loop BEFORE the existing - single-class discharge — multi-candidate residuals refine first, - `Resolved` falls through to `continue`, error variants return. - Used `expanded` (already-superclass-expanded declared constraints) - for the rigid-var path, sounder than `declared_constraints` direct. - 3 unit tests for the discharge path. -- iter mq.2.8: `resolve_residual_class_for_mono` helper in mono.rs - + 3 unit tests in a new `mod tests` block. Mono's residual-to- - target mapping (`collect_residuals_ordered`) now calls the helper - before constructing the registry key; resolved-class overrides - the residual's tentative `r.class` in `MonoTarget::ClassMethod`. - Single-class residuals flow through unchanged. -- iter mq.2.9: full `cargo test --workspace` 539 passed / 0 failed - (was 520 at start of mq.2; +19 = 15 new mq.2 tests + 4 pre- - existing mq.1 follow-on additions in cross-language test files - already shipped). All pre-mq.2 fixtures (prelude, mq1_xmod, - eq_ord_polymorphic) typecheck unchanged. Bench results: - `compile_check.py` 0 regressed / 0 improved / 24 stable; - `cross_lang.py` 0 regressed / 0 improved / 25 stable; - `check.py` 1 regressed / 2-4 improved / 58-60 stable (latency - noise — different metric regressed between two runs, runtime - benches cannot be touched by a typecheck-side iter). Prelude - roundtrip clean. - -## Concerns - -- **Helper-cost: `parse_method_qualifier` does an `rfind('.')` on - every `Term::Var` lookup that reaches the class-method branch.** - Today this is bounded by the per-Var-arm cost (already O(name- - length) due to the dot-counting and unification ladder), so it's - noise. If profiling ever shows the Var arm hot, the right fix is - to cache `(method_name, qualifier_opt)` on the Var node rather - than recompute — but that's a Term struct change and out of - scope for mq.2. - -- **`pub` bump on `ResidualConstraint`.** The unit-test crate - (`tests/method_dispatch_pin.rs`) needs to construct - `ResidualConstraint` for Task 5; Task 7's in-lib tests also - reference it. Bumping from `pub(crate)` to `pub` is the minimum - edit; the struct semantically belongs at the workspace boundary - anyway (the spec's published shape for what the dispatcher - produces). No risk: no external crate depends on `ailang-check` - outside this workspace. - -- **`MissingConstraint.class` field for multi-candidate rigid-var - case is best-effort.** When `refine_multi_candidate_residual` - emits `MissingConstraint`, the surfaced `class` is the first - candidate (BTreeSet order). The diagnostic ctx still names the - method, the type, and (via the diagnostic Display) the full - candidate set is reachable; the `class` is just a hint. Real - workspaces never reach this branch pre-mq.3. - -## Known debt - -- **mq.3 retires `MethodNameCollision` and exercises the - multi-candidate path end-to-end.** The unit tests in this iter - are the only coverage of the new branches until then. mq.3 will - add cross-class-collision fixtures (e.g. two `Show` classes in - separate modules) and the existing tests should continue to pass - unchanged. - -- **`class_methods` keyed by method name only.** Pre-mq.3 invariant - guarantees `env.class_methods.get(method_name)` is unambiguous; - post-mq.3 this lookup needs to be reworked (likely keyed by - `(class, method)` or by `method` returning a list). The synth - branch's `expect("invariant")` documents the dependency. Not - load-bearing in mq.2. - -- **Synth-time `declared_constraints: &[]` in the Var arm rewrite.** - This is correct for pre-mq.3 (singleton candidate set never - needs filtering) but means the post-mq.3 constraint-driven - filter cannot fire at synth time. The branch will need - `env.active_declared_constraints` plumbed through (Env field - + `check_fn` initialization) before mq.3's first multi-candidate - workspace with declared constraints. Cost: ~10-line edit. - -## Files touched - -- crates/ailang-check/src/diagnostic.rs -- crates/ailang-check/src/lib.rs -- crates/ailang-check/src/mono.rs -- crates/ailang-check/tests/method_dispatch_pin.rs (new) - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-mq.2.json diff --git a/docs/journals/2026-05-13-iter-mq.3.md b/docs/journals/2026-05-13-iter-mq.3.md deleted file mode 100644 index 56863ce..0000000 --- a/docs/journals/2026-05-13-iter-mq.3.md +++ /dev/null @@ -1,284 +0,0 @@ -# iter mq.3 — Retire MethodNameCollision + multi-class E2E + DESIGN.md sync - -**Date:** 2026-05-13 -**Started from:** 90075715d99c9e8c59ea79267c2615f5756520f2 -**Status:** DONE -**Tasks completed:** 9 of 9 - -## Summary - -mq.3 closes the module-qualified-class-names milestone by retiring -the `WorkspaceLoadError::MethodNameCollision` workaround that mq.1 + -mq.2 made redundant. The variant + pre-pass + Origin enum + Display -arm + two pin tests are deleted; the two on-disk fixtures that fired -the collision now load cleanly and the equivalent observations -relocate to `env.method_to_candidate_classes` (multi-entry set) plus -a per-call-site warning. Three positive E2E fixtures (Trajectory C -ambiguous, Trajectory E explicit-qualifier, class-fn shadow) exercise -the multi-candidate dispatch path end-to-end via `ail check --json`. - -Two out-of-band corrections that mq.2 surfaced as known debt land -here: (1) `Env.active_declared_constraints: Vec` is -plumbed pre-synth in `check_fn` so the post-superclass-expansion -constraint set is reachable at synth time by -`resolve_method_dispatch`'s constraint-driven filter; (2) -`ModuleGlobals.class_methods` + `Env.class_methods` re-keyed from -`BTreeMap` to -`BTreeMap<(QualifiedClass, MethodName), ClassMethodEntry>` so the -post-retirement world has O(1) `(class, method)` lookup, with -mono's two presence-check sites switched to consult the natively -method-keyed `method_to_candidate_classes`. A third -correction — `synth(...)` signature extended with a `warnings: &mut -Vec` out-parameter — installs the channel for the new -structured warning `class-method-shadowed-by-fn` that fires when a -free fn shadows a class method of the same name (per spec -§"Class-fn collisions": precedence + warning, the spec's default). - -DESIGN.md sync removes the `MethodNameCollision` bullet from -"Workspace-load (registry-build) diagnostics" with a forward-pointing -note, adds `AmbiguousMethodResolution` / `UnknownClass` / -`class-method-shadowed-by-fn` to the "Typecheck diagnostics" block, -rewords the `AmbiguousInstance` paragraph (per-class coherence stays -via `DuplicateInstance`; cross-class method ambiguity is the new -diagnostic at the call site), rewrites the class-names paragraph in -the canonical-form section to point at the mq.1 rule + the new -dispatch model, and adds a new `### Method dispatch` subsection -anchoring the 5-step rule with `method_to_candidate_classes` as the -load-bearing data structure. Roadmap P2 entry → `[x]` with a summary -of the three iters; milestone-24 (`Post-22 Prelude — Show + print -rewire`) `depends on:` line struck and the entry annotated -"ready for re-brainstorm". - -545 tests green (was 539 at start of mq.3; +6 net = 3 new mq.3.x lib -tests + 2 relocated method_collision_pin + 3 new mq3_multi_class_e2e -- 2 deleted workspace.rs pin tests). `bench/compile_check.py` and -`bench/cross_lang.py` exit 0; `bench/check.py` exit 1 with 2 -regressions both noise-class (bench_list_sum.bump_s persistence — -3rd consecutive sighting since audit-ct-tidy — and a max_us tail -metric, plus 4 improvements). Prelude zero-diff. - -## Per-task notes - -- **mq.3.1** — `Env.active_declared_constraints: Vec` - field plumbed; populated in `check_fn` pre-synth alongside the - rigid-vars install (consolidated with the existing post-synth - `expanded` computation — single source of truth, no clone-and- - duplicate as the plan literally suggested). Synth Var-arm - `resolve_method_dispatch` call site swapped `&[]` for - `&env.active_declared_constraints`. 1 new test mq3_env_active_declared_constraints_field_exists. - -- **mq.3.2** — `ModuleGlobals.class_methods` re-keyed - `IndexMap` → `IndexMap<(String, String), - ClassMethodEntry>`; sibling `Env.class_methods` similarly re-keyed - to `BTreeMap<(String, String), ClassMethodEntry>`. Accessor methods - on `ModuleGlobals` (`has_class_method`, `class_method_class`, - `class_method`) all take `(class, name)`; new - `class_method_candidates(name) -> Vec<(&class, &entry)>` for the - enumerate-all-classes-declaring-this-method use case (returning - `Vec` not `impl Iterator` because the borrow-checker rejected the - `'a`/`'_` capture without explicit `use<'a, '_>` syntax not yet - idiomatic in this crate). `build_module_globals` insert site + - `build_check_env` merge loop both threaded through the tuple key. - Synth Var-arm `class_methods.get(&(residual_class, method))` uses - the dispatcher-resolved tentative class. Mono's two presence-check - sites (`rewrite_mono_calls`, `interleave_slots`) switched to - consult `method_to_candidate_classes` natively — method-keyed, - preserves the existing presence-check signature shape; the four - call sites at module-walk + interleave_slots entry pass - `&env.method_to_candidate_classes` instead of `&env.class_methods`. - Test assertion in `crates/ail/tests/typeclass_22b2.rs:42` updated - to pass the qualified class explicitly. 1 new test - mq3_env_class_methods_tuple_keyed. - -- **mq.3.3** — `synth(...)` signature extended with `warnings: - &mut Vec` (15+ recursive synth callsites threaded). - Five external synth callers (check_fn, check_const, 3 in - builtins.rs, 1 in lift.rs, 2 in mono.rs, 1 in `check`) all - updated; check_fn / check_const pass through to caller via new - `out_warnings: &mut Vec` parameter on the caller - chain (`check_def`, `check_in_workspace`); `check_workspace` - extends per-module synth_warnings into `module_diags`. Plan - suggested `check_fn` return `Result<(CheckedFn, Vec)>` - but `check_fn`'s actual signature is `Result<()>` and the - mut-ref-accumulator pattern matches the existing `Vec` - accumulator at `check_in_workspace`'s caller. New diagnostic - `class-method-shadowed-by-fn` (warning, kebab-case code, - structured `ctx` carrying `name` + `method` + `fn_owner_module` + - `candidate_classes`) emitted by a closure at all three - fn-precedence branches (locals, same-module fn, implicit-import - fn). Implicit-import-fn branch reordered ABOVE the class-method - branch per spec §"Class-fn collisions" so fn-wins precedence is - structural, not just convention. Diagnostic doc updated in - `diagnostic.rs`. One existing test - `crates/ail/tests/typeclass_22b3.rs:rewrite_walker_skips_locally_shadowed_class_method` - was a known trip-wire — the fixture intentionally shadows a class - method with a local binding, which now correctly fires the new - warning; filtered out of the test's `assert!(diags.is_empty())` - with a comment naming why. 1 new test - mq3_class_method_shadowed_by_fn_warning_fires. - -- **mq.3.4** — `WorkspaceLoadError::MethodNameCollision` variant + - the Origin enum + the per-def collision loop in `build_registry` - (workspace.rs 596-681 pre-edit) + the Display arm in - `main.rs:1201` deleted. The two in-workspace.rs pin tests - (`class_class_method_name_collision_fires` + - `class_fn_method_name_collision_fires`) deleted; relocation lands - in mq.3.5. The on-disk fixtures - `test_22b2_method_name_collision_class_{class,fn}.ail.json` stay - on-disk (now exercised as positive-load by the relocated tests). - -- **mq.3.5** — `crates/ailang-check/tests/method_collision_pin.rs` - created with two repurposed pin tests: the class-class fixture - loads cleanly and `env.method_to_candidate_classes["foo"]` has - exactly two qualified-class entries (`...A` + `...B`); the - class-fn fixture loads cleanly (the warning is a call-site - concern, not a load-time concern; covered by mq.3.6 E2E). The - relocation from `ailang-core` to `ailang-check` is required - because `Env.method_to_candidate_classes` is built by - `build_check_env`, which lives in `ailang-check` (not visible - from `ailang-core`'s `mod tests`). - -- **mq.3.6** — Three new on-disk fixtures plus three integration - tests in `crates/ail/tests/mq3_multi_class_e2e.rs`. Fixture (a) - `mq3_two_show_ambiguous` exercises the bare-method ambiguous - case (two `Show` classes, both with `Show Int`, bare `show 42` - → `AmbiguousMethodResolution`). Fixture (b) - `mq3_two_show_qualified` exercises explicit-qualifier resolution - (same workspace, `mq3_two_show_ambiguous_a.Show.show 42` → - clean). Fixture (c) `mq3_class_eq_vs_fn_eq` exercises class-fn - shadow (`class MyEq { myeq }` + `fn myeq` + bare `myeq 1 2` → - fn wins, warning fires). The plan's draft fixtures used a bare - `Term::App` for instance method bodies; the existing-convention - shape (per `test_22b3_mono_synthetic.ail.json`) is `Term::Lam` - with `paramTypes`/`retType` — implementer-phase repair adjusted - inline. Naming `MyEq`/`myeq` (rather than `Eq`/`eq`) avoids an - extra prelude-vs-fn shadow that would have confused the warning - assertion. All three tests GREEN first-shot; spot-checked via - `ail check --json` for completeness. - -- **mq.3.7** — DESIGN.md sync: class-names paragraph (1156-1160) - rewritten to point at mq.1 canonical-form + mq.3 dispatch model; - `MethodNameCollision` bullet struck from "Workspace-load - (registry-build) diagnostics" with forward-pointing note; three - new typecheck diagnostics + `NoInstance.candidate_classes` - addendum added to "Typecheck diagnostics"; `AmbiguousInstance` - paragraph reworded to clarify the registry-level vs call-site - distinction; new `### Method dispatch` subsection anchors the - 5-step rule with `method_to_candidate_classes` as the - load-bearing data structure, the class-fn precedence rule, and - the post-mq.3 tuple-keyed `class_methods` shape. - -- **mq.3.8** — Roadmap P2 milestone "Module-qualified class names - + type-driven method dispatch" → `[x]` with a one-paragraph - summary of the three iters (mq.1 canonical-form, mq.2 mechanism, - mq.3 retirement) and the milestone's user-facing outcome (two - libraries can each declare `class Eq` with their own `eq`). - Milestone-24 (`Post-22 Prelude — Show + print rewire`) `depends - on:` line struck and the entry annotated "ready for - re-brainstorm" so the next time `/boss` walks the roadmap it - picks up the un-blocked spec re-derivation against the - post-retirement architecture. - -- **mq.3.9** — Integration verification. `cargo test --workspace` - 545 passed / 0 failed. `bench/compile_check.py` 0 regressed / 0 - improved / 24 stable. `bench/cross_lang.py` 0 regressed / 0 - improved / 25 stable. `bench/check.py` 2 regressed / 4 improved - / 57 stable: regression #1 `throughput.bench_list_sum.bump_s` - (3rd consecutive sighting since audit-ct-tidy 2026-05-12, audit- - ratified-class noise per the journal lineage); regression #2 - `latency.implicit_at_rc.max_us` (same noisy max-tail metric - that mq.2 journal flagged as runtime-uncoupled-to-typecheck- - iter, audit-ratified-class). Prelude zero-diff. All three new - E2E fixtures observable via `ail check` with the expected - diagnostic / no-diagnostic / warning outcomes. - -## Concerns - -- **Plan's `check_fn` signature suggestion was off.** Plan Step 4 - proposed `check_fn(...) -> Result<(CheckedFn, Vec), - CheckError>`; the actual signature is `Result<()>` (no - CheckedFn). The mut-ref-accumulator pattern adopted instead - matches the existing `check_in_workspace` shape (Vec - accumulator). Same outcome, different mechanism. - -- **`class_method_candidates` returns `Vec` not `impl Iterator`.** - The plan's accessor signature `impl Iterator` requires the `use<'a, '_>` opaque-type - capture syntax — not yet idiomatic in this crate (the rest of - the codebase uses `Vec`-returning accessors at module-graph - boundaries). The semantic contract is the same; performance is - one allocation per call, bounded by the per-method candidate - count (currently ≤ 2 across the corpus). - -- **One existing E2E test trips the new warning.** - `crates/ail/tests/typeclass_22b3.rs:rewrite_walker_skips_locally_shadowed_class_method` - uses a fixture (`test_22b3_shadow_class_method`) that - intentionally shadows the class method `show` with a local - `let show = "shadow"`. Pre-mq.3 this fixture typechecked clean; - post-mq.3 it correctly fires the `class-method-shadowed-by-fn` - warning. The test's `assert!(diags.is_empty())` was relaxed to - filter out the new warning with a comment naming why. This is - the warning behaving correctly, not a test workaround. - -- **Plan's instance-method body shape was wrong.** Plan Step 1 of - Task 6 drafted `body: { "t": "app", ... }` for instance - methods; the existing-convention shape (per `test_22b3_mono_synthetic`) - is `Term::Lam` with `paramTypes`/`retType` keys. Implementer-phase - repair adjusted all three fixtures inline. The plan's caveat - ("implementer adjusts to match the exact shape used by - `examples/prelude.ail.json`'s Eq/Ord instances") covered this - case; just exercising the caveat verbatim. - -## Known debt - -- **`bench_list_sum.bump_s` 3rd-consecutive regression.** Noise-class - per the audit-ct-tidy 2026-05-12 journal entry. No attribution - cause across three audits. The pattern is a candidate for - ratify-without-attribution at the milestone-close audit (audit-mq) - but this iter follows the conservative pristine-baseline - convention. - -- **`latency.implicit_at_rc.max_us` max-tail noise.** Continues from - mq.2's bench profile. Runtime metric, structurally uncoupled to - this typecheck-side iter. Audit-ratifiable. - -- **`bench/check.py` exit 1.** Same noise-class status as the two - regressions above. Conservative interpretation: leave the - baseline pristine; audit-mq decides whether to ratify. - -- **`class_method_candidates` allocation per call.** Bounded by - per-method candidate count (currently ≤ 2). A future tidy that - introduces `use<'a, '_>` syntax across the crate could revisit - the iterator-return shape. - -## Files touched - -Code: -- crates/ail/src/main.rs -- crates/ail/tests/typeclass_22b2.rs -- crates/ail/tests/typeclass_22b3.rs -- crates/ailang-check/src/builtins.rs -- crates/ailang-check/src/diagnostic.rs -- crates/ailang-check/src/lib.rs -- crates/ailang-check/src/lift.rs -- crates/ailang-check/src/mono.rs -- crates/ailang-core/src/workspace.rs - -New tests + fixtures: -- crates/ail/tests/mq3_multi_class_e2e.rs -- crates/ailang-check/tests/method_collision_pin.rs -- examples/mq3_two_show_ambiguous_a.ail.json -- examples/mq3_two_show_ambiguous_b.ail.json -- examples/mq3_two_show_ambiguous.ail.json -- examples/mq3_two_show_qualified.ail.json -- examples/mq3_class_eq_vs_fn_eq_classmod.ail.json -- examples/mq3_class_eq_vs_fn_eq_fnmod.ail.json -- examples/mq3_class_eq_vs_fn_eq.ail.json - -Docs: -- docs/DESIGN.md -- docs/roadmap.md - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-mq.3.json diff --git a/docs/journals/2026-05-13-iter-mq.tidy.md b/docs/journals/2026-05-13-iter-mq.tidy.md deleted file mode 100644 index aac987c..0000000 --- a/docs/journals/2026-05-13-iter-mq.tidy.md +++ /dev/null @@ -1,219 +0,0 @@ -# iter mq.tidy — Close 4 actionable drift items from audit-mq - -**Date:** 2026-05-13 -**Started from:** 64d3feeb972f4ddfe9bd48484585c69a457e7e1e -**Status:** DONE -**Tasks completed:** 5 of 5 - -## Summary - -mq.tidy closes the four actionable drift items audit-mq surfaced -(commit `f382931`), moving the module-qualified-class-names milestone -from "structurally complete" to "drift-clean ready for next milestone". -T1 extends the discharge-time `refine_multi_candidate_residual` -rigid-var filter from class-only to class + type-unification via the -existing `constraint_type_matches` helper, so two same-class declared -constraints on different typevars are correctly discriminated by the -residual's typevar identity (high-1). T2 extracts the inline -`qualifier_is_class_shape` predicate as a free `pub(crate)` helper and -broadens it to accept PascalCase single-segment qualifiers -(`"Show.show"`) alongside module-qualified ones (`"prelude.Show.show"`) -— same-module bare-class call sites are now reachable, symmetric to -the canonical-form rule at the schema level (high-2). T3 introduces -the `any_candidate_class_has_instance` predicate and gates the -`class-method-shadowed-by-fn` warning closure on it, so class methods -with zero registry instances anywhere no longer fire the warning; -conservative approximation of the full spec rule per Boss Q2 (the arg -type required for the strict version is unavailable at the Var-arm) -(medium-1). T4 annotates the four Data Model schema fragments -(`SuperclassRef`, `InstanceDef.class`, `Type::Con.name`, -`Constraint.class`) with the canonical-form cross-reference, so a -schema-section reader without the prose context no longer sees -bare-only (medium-2). - -T5 verification: 548 tests green (was 545 pre-iter + 3 new -mq_tidy_* unit tests). `bench/compile_check.py` exit 0 (24/24 stable -after a one-run noise blip absorbed under tolerance on re-run). -`bench/cross_lang.py` exit 0 (25/25 stable). `bench/check.py` exit 0 -both runs; metric-identity-migrating noise on the -`latency.implicit_at_rc.*` max-tail cluster — 5th-consecutive audit -observation per audit-mq's lineage, baseline pristine. The -`mq3_class_method_shadowed_by_fn_warning_fires` in-crate test was a -trip-wire: pre-tidy it used `Registry::default()` (no instances) and -the warning fired anyway because the old gate didn't check instances; -post-tidy it correctly suppresses, so the fixture was updated to ship -a `clsmod.Show Int` registry instance — fixture-repair as a coherence -consequence of the deliberately tightened spec rule. mq3 E2E (3 tests) -+ typeclass_22b3 (18 tests) all stay green; the E2E fixtures already -ship instances for the shadowed class. - -## Per-task notes - -- **mq.tidy.1** — Rigid-var refinement type-unification leg. - `refine_multi_candidate_residual` filter at lib.rs:2163-2180 extends - from `dc.class == c` to `dc.class == c && constraint_type_matches(&dc.type_, &residual.type_)`. - New unit test `mq_tidy_rigid_var_filter_uses_type_unification` pins - the same-class-different-typevar discrimination - (`prelude.Show a` + `userlib.Show b` declared, residual on Var `a` - → `Resolved("prelude.Show")`, `userlib.Show` drops because its - declared type `Var b` doesn't unify with the residual's `Var a`). - Architecture decision per plan: only the discharge-time call site - needs the type-unification leg; the synth-time `resolve_method_dispatch` - is invoked with `concrete_arg_type: None` and constructs the - residual's metavar AFTER the dispatch call, so the rigid-var leg - there has no residual type to unify against (fresh metavar would - trivially unify with any declared-constraint type, degenerating to - class-only filter). The class-only filter at synth time is therefore - semantically correct, not drift. - -- **mq.tidy.2** — Bare-class qualifier shape accepted. Inline - `qualifier_is_class_shape` predicate at the synth Var-arm dispatch - entry (originally lib.rs:2570-2575) extracted as a free - `pub(crate) fn qualifier_is_class_shape(&Option) -> bool` - helper adjacent to `parse_method_qualifier`. Predicate body: - `None` → true (bare method); `Some(q)` with inner dot → true - (`.`); `Some(q)` single-segment → first char - uppercase (PascalCase). Bare-fn qualifiers (`"std_list"`, lowercase) - correctly reject and fall through to the cross-module-fn arm. - Call site at the original location reduced to a one-liner. New - unit test `mq_tidy_bare_class_qualifier_shape_accepted` covers all - four shapes. - -- **mq.tidy.3** — `class-method-shadowed-by-fn` warning tightened. - New `pub(crate) fn any_candidate_class_has_instance(...)` helper - using BTreeMap range-scan (O(log n) per candidate, O(|candidates| - * log n) total) checks the workspace registry for at least one - instance under any candidate class. Warning closure at - lib.rs:2479-2502 (now ~2498-2540 post-T2-insertion-shift) builds - a `registry_unit_view: BTreeMap<(String, String), ()>` per - closure invocation and calls the helper before emitting. New - unit test `mq_tidy_shadow_warning_suppressed_when_no_instance` - pins both directions (empty registry → suppress; one instance - → fire). The existing `mq3_class_method_shadowed_by_fn_warning_fires` - in-crate test was a trip-wire — its fixture used - `Registry::default()` (no instances) and pre-tidy the warning - fired anyway because the old gate didn't check instances. Post-tidy - the spec rule correctly suppresses; fixture repaired by injecting - a `clsmod.Show Int` registry entry directly into `ws.registry.entries` - so the test continues to assert positive-fire behaviour under the - post-tidy rule. The three mq3 E2E fixtures (`mq3_class_eq_vs_fn_eq` - + `mq3_two_show_*`) already ship instances on the shadowed class, - so they stayed green without modification. - -- **mq.tidy.4** — DESIGN.md schema fragments. Four trailing-comment - annotations: `SuperclassRef` (`"superclass"` field at line 2139), - `InstanceDef.class` (line 2152), `Type::Con.name` (line 2253), and - `Constraint.class` (line 2273). Class-ref annotations cross-reference - §"Class names" / mq.1; the Type::Con annotation cross-references - §"Type::Con name scoping" / ct.1 (the actual section title — the - plan's "§Type names" / "Canonical type names" search did not match - exactly; used the canonical anchor title verified via grep). Doc - count of "canonical form" went from 1 → 5 (+4 as planned). All - annotations are inline trailing-comment appends; no line-count - change in DESIGN.md (2678 → 2678). - -- **mq.tidy.5** — Integration verification. - `cargo test --workspace --no-fail-fast` → 548 passed, 0 failed - (was 545 pre-iter + 3 new mq_tidy_* unit tests). `bench/compile_check.py` - exit 0; first run showed 1 regression - (`build_O0_ms.bench_list_sum_explicit` +5.43% within 20.0% tolerance, - classified `ok`; the summary's "1 regressed" count is a - false-positive on the first run, second run shows 0 regressed, - 24/24 stable). `bench/cross_lang.py` exit 0 both runs, 25/25 stable. - `bench/check.py` exit 0 both runs; run 1 had 3 regressed / 3 - improved (all `latency.implicit_at_rc.*` max-tail metrics — same - noise class the audit-mq journal documented across 4 consecutive - runs); run 2 had 0 regressed / 1 improved / 62 stable — exactly - the metric-identity-migration pattern the audit named as variance, - not signal. Baseline pristine per the conservative-call convention - (6th-consecutive observation now). Plan Step 5 (prelude roundtrip - via `cargo run --bin ail -- check examples/prelude.ail.json`) - returned exit 1 with "module name 'prelude' is reserved" — but - this is identical to pre-edit behaviour (verified via `git stash`), - not a regression from this iter. The plan's expectation was - inaccurate; the actual sanity gate (DESIGN.md doc edits do not - break prelude consistency) is satisfied by the workspace test - suite passing 548/548. - -## Concerns - -- **Plan Task 3 Step 6 anticipation was off for one in-crate test.** - The plan said: "Expected: PASS — the existing positive fixtures - (`mq3_class_eq_vs_fn_eq`, `typeclass_22b3::rewrite_walker_skips_locally_shadowed_class_method`) - all ship registry instances for the shadowed class, so the - tightened predicate continues to admit the warning emission." This - was correct for those two fixtures (both E2E), but the in-crate - unit test `mq3_class_method_shadowed_by_fn_warning_fires` builds - its workspace with `Registry::default()` (zero instances) and - pre-tidy relied on the looser rule. Post-tidy it correctly - suppresses; the fixture was repaired inline by injecting a - `clsmod.Show Int` registry entry. This is exactly analogous to the - `typeclass_22b3::rewrite_walker_skips_locally_shadowed_class_method` - trip-wire mq.3 documented — same pattern, different test. - Fixture-repair as a coherence consequence of a deliberately - tightened spec rule; the test's intent ("warning fires when a - free fn shadows a class method") is unchanged. - -- **Plan Task 5 Step 5 prelude check expectation was off.** The plan - said `cargo run --bin ail -- check examples/prelude.ail.json` - expected `ok` exit 0; the actual behaviour is exit 1 with the - "module name 'prelude' is reserved (auto-injected by the loader)" - error. This is identical to pre-edit behaviour (verified via - `git stash`); the prelude file is the loader's auto-injected - module, not a workspace entry, so `ail check` against it has - always failed by design. The right sanity gate for "DESIGN.md - doc edits do not break prelude consistency" is the workspace - test pass (548/548), which exercises every workspace test that - uses the prelude. The plan's prescribed command can be removed - from future tidy plans. - -- **T2 docstring update on `parse_method_qualifier` was not - explicitly in the plan.** The plan asked for adding the new - `qualifier_is_class_shape` helper but did not enumerate the - paragraph update on `parse_method_qualifier`'s docstring. The - old docstring said "Callers gate on the qualifier shape: a class - qualifier is always qualified per mq.1's canonical-form rule" — - which the broadened gate directly contradicts (bare PascalCase - qualifiers now go to class-dispatch). Updating the existing - helper's documentation to reference the new helper and remove - the now-incorrect claim is strictly required to keep the helper's - documentation coherent with the requested behaviour change; it - is the docstring analog of "code edits strictly required to make - the requested changes compile." Defensible as a coherence - consequence, not unrequested scope creep. - -## Known debt - -- **bench/check.py max-tail noise envelope.** 6th-consecutive - observation of the `latency.implicit_at_rc.*` max-tail metric - identity migrating between runs. The conservative-call convention - has held the baseline pristine across 5 prior audits; this iter - adds run 6 to the lineage. A future audit may consider ratification - via `--update-baseline` if the pattern persists for another 2-3 - audits; until then, the lineage is the documentation. - -- **mq.tidy did not deliver the [low-1] (Trajectory B + D E2E) - or [low-2] (mono env shape) roadmap-backlog items.** Same scope - as audit-mq's deferral; these wait for the multi-class workspace - shape to land downstream. - -- **mq.tidy did not refactor `synth(...)`'s 10-mut-ref signature.** - audit-mq's [medium-3] item was explicitly acknowledged-debt, not - in tidy scope. Same disposition; future iter may revisit if a - natural refactor opportunity arises. - -## Files touched - -Code: -- crates/ailang-check/src/lib.rs - -Docs: -- docs/DESIGN.md -- docs/journals/2026-05-13-iter-mq.tidy.md (this file, new) - -No new test files; the three new mq_tidy_* unit tests are inline in -`crates/ailang-check/src/lib.rs`'s `mod tests` block. - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-mq.tidy.json diff --git a/docs/journals/2026-05-13-iter-rustdoc-sweep.md b/docs/journals/2026-05-13-iter-rustdoc-sweep.md deleted file mode 100644 index 5558ee1..0000000 --- a/docs/journals/2026-05-13-iter-rustdoc-sweep.md +++ /dev/null @@ -1,61 +0,0 @@ -# iter rustdoc-sweep — clear all 23 cargo-doc warnings - -**Date:** 2026-05-13 -**Started from:** 48b1f77 (post-WhatsNew str-concat done-state) -**Status:** DONE - -## Summary - -Hygiene sweep against `cargo doc --workspace --no-deps`. Before: 23 -warnings (17 in `ailang-check` + 4 in `ailang-core` + 2 in -`ailang-surface`). After: zero. All edits are documentation-only — -no production code, no test code changed. Tests stay 562 green. - -Two warning classes, fixed with two different shapes: - -- **Public doc links to private items** (15 warnings). Pattern: - a `pub` item's `///` doc references `[`fn_name`]` where `fn_name` - is `pub(crate)` or private. Fix: replace ``[`fn_name`]`` with - the plain backtick-code-span ``` `fn_name` ```. The identifier - stays readable in rendered docs but rustdoc no longer tries to - resolve the link. -- **Unresolved links** (8 warnings). Sub-classes: - - ``[`Registry`]`` in `mono.rs` (×3) — `Registry` lives in - `ailang_core::workspace` and the path was bare. Fixed by - fully-qualifying as `[`ailang_core::workspace::Registry`]` / - `[`ailang_core::workspace::Registry::entries`]`. - - ``[`Env::class_methods`]`` — `Env` is in this crate but - bare; the doc string was demoted to plain backticks - (`` `crate::Env::class_methods` ``) because `Env` itself has - no `pub` re-export at the crate root that rustdoc could - follow. - - ``[`crate::parse`]`` ambiguous in `loader.rs` (×2) — rustdoc - saw both a function and a module with that path. Disambiguated - to the function form via ``[`crate::parse()`]`` per - rustdoc's own suggestion. - - `metas[i]` in `lib.rs` — rustdoc parsed `[i]` as a link. - Escaped as `metas\[i\]` (same fix for the sibling `vars\[i\]`). - -## Files touched - -- `crates/ailang-core/src/desugar.rs` (1 line) -- `crates/ailang-core/src/workspace.rs` (3 lines) -- `crates/ailang-surface/src/loader.rs` (2 lines) -- `crates/ailang-check/src/uniqueness.rs` (2 lines) -- `crates/ailang-check/src/diagnostic.rs` (1 line) -- `crates/ailang-check/src/mono.rs` (5 lines) -- `crates/ailang-check/src/lib.rs` (8 lines) - -## Verification - -- `cargo doc --workspace --no-deps 2>&1 | grep "^warning:" | wc -l` - → `0`. -- `cargo test --workspace` → 562 passed, 0 failed (baseline holds). - -## Concerns - -- (none) - -## Known debt - -- (none) diff --git a/docs/journals/2026-05-13-iter-str-concat.md b/docs/journals/2026-05-13-iter-str-concat.md deleted file mode 100644 index 018782f..0000000 --- a/docs/journals/2026-05-13-iter-str-concat.md +++ /dev/null @@ -1,149 +0,0 @@ -# iter str-concat — heap-Str concatenation primitive in four-site lockstep - -**Date:** 2026-05-13 -**Started from:** 679572a92d594d94b915d09fd60e7ef7f7bb83bd -**Status:** DONE -**Tasks completed:** 7 of 7 - -## Summary - -`str_concat : (borrow Str, borrow Str) -> Str` shipped as a heap-Str -primitive in the four-site-lockstep pattern established by `str_clone` -(iter 24.1). Closes fieldtest-form-a friction finding #4. The -LLM-natural Show-body shape -`(app str_concat "label=" (app int_to_str x))` now parses, checks, -builds, and runs end-to-end via the new -`examples/show_user_adt_with_label.ail` fixture (`Item 42\n` -through `print . MkItem 42`). - -Net delta: +3 tests (559 baseline → 562 green): the E2E pin -(`str_concat_e2e_show_user_adt_with_label`), the builtin-signature -unit test (`install_str_concat_signature` asserting the arity-2 -Borrow-Borrow / Own-return shape directly on `env.globals`), and the -codegen IR-pin (`str_concat_emits_call_to_ailang_str_concat` -asserting both the extern declaration AND the call instruction in -emitted IR). Five `crates/ail/tests/snapshots/*.ll` files were -regenerated to absorb the new unconditional -`declare ptr @ailang_str_concat(ptr, ptr)` line in the IR header -preamble — same upkeep pattern as hs.4 (which regenerated the same -5 snapshots for the unconditional `@ailang_int_to_str` / -`@ailang_float_to_str` declares). - -Task 5 repaired the bug-fixture / pin-test collision: the -`bugfix-instance-body-unbound-var` iter (commit 77f584a) used -`str_concat` as the literal unbound name because that was the -LLM-natural shape the fieldtester reached for; renamed to -`format_label` (LLM-author-realistic helper name) to preserve the -pin's intent (instance-method-body walked through unbound-var check) -while substituting a name that will never become a builtin. - -## Per-task notes - -- iter str-concat.1: RED fixture `examples/show_user_adt_with_label.ail` - (single-ctor `Item Int` ADT + `Show Item` body producing labelled - output) + E2E pin `crates/ail/tests/str_concat_e2e.rs` (check + - build + run asserting `Item 42\n` stdout). Confirmed RED with - `[unbound-var] prelude.Show: unknown identifier: 'str_concat'`. -- iter str-concat.2: `runtime/str.c` C helper `ailang_str_concat` - appended after `ailang_str_clone`; reads `len` from both source - payloads, `str_alloc(la+lb)`, memcpys both bytes, NUL-terminates. - Clean `clang -O2 -c` compile. -- iter str-concat.3: `crates/ailang-check/src/builtins.rs` — - `env.globals.insert("str_concat", ...)` install entry after - str_clone block (arity-2, both Borrow, Own return); list() row - `("str_concat", "(Str, Str) -> Str")` after str_clone row; - `install_str_concat_signature` test reaches into `env.globals` - directly to match the arity-2 / param_modes / ret_mode discipline - (deeper than the sibling `synth_in_builtins_env`-based tests - because str_concat is the first builtin with arity-2 Borrow params). -- iter str-concat.4: `crates/ailang-codegen/src/lib.rs` four-site lockstep: - extern declaration after `@ailang_str_clone`; lower_app arm after - str_clone arm (arity-2 check + two `lower_term` + `fresh_ssa` + - body push); `is_builtin_callable` list extended with `"str_concat"`; - IR-pin test mirrors the sibling `str_clone_emits_call_to_ailang_str_clone` - pattern (Module/FnDef/Term::Do scaffolding via `super::*` + - `SCHEMA`, not the plan's `ailang_surface::parse` structural sketch — - plan explicitly authorised this in Step 4 note). IR-pin test - asserts both the extern declare AND the call site. RED→GREEN flip - on the E2E pin happens at this task. Five IR snapshots regenerated - via `UPDATE_SNAPSHOTS=1` to absorb the new declare line (sole - diff verified at line 17 across all snapshots). -- iter str-concat.5: bug-fixture / pin-test collision repaired — - `examples/bug_unbound_in_instance_method.ail` line 14 rename - `str_concat` → `format_label`; pin test - `crates/ail/tests/unbound_in_instance_method_pin.rs` `replace_all` - rename (one test name changed to - `check_fires_unbound_var_for_format_label_in_instance_method_body`). - Both pin tests pass; new fixture `ail check` reports - `ok (23 symbols across 2 modules)` — matches the plan's empirical - guess exactly. -- iter str-concat.6: `docs/DESIGN.md` new §"Heap-Str primitives" - subsection between the milestone-24 paragraph and the - `Primitive output goes through ...` paragraph; lists all five - shipping primitives with their type, iter origin, and prelude - role, plus the user-visible vs prelude-internal distinction. -- iter str-concat.7: `docs/roadmap.md` struck `[x] [feature] str_concat` - entry at end of P2 cluster (right before `## P3 — Ideas`). All - four cited fixtures (`show_user_adt_with_label`, `show_user_adt`, - `test_22c_user_class_e2e`, `cmp_max_smoke`) `ail check` clean. - `ail builtins | grep str_concat` returns `str_concat (Str, Str) -> Str`. - -## Concerns - -- Task 3 (Step 5) test-count prediction was off: plan expected - `passed: 560 failed: 1` after registering the builtin but actual - was `passed: 558 failed: 3`. Reason: the two - `unbound_in_instance_method_pin` tests turn RED as soon as - `str_concat` is registered in the checker, NOT only at codegen. - The pins assert on `ail check`'s `[unbound-var]` diagnostic, which - depends only on the checker registry. Implementation is correct; - Task 5 fully repairs. No follow-up needed beyond noting the - prediction-vs-actual gap. -- Task 4 (Step 7) test-count prediction was off: plan expected - `passed: 562 failed: 1` after codegen lands; actual after - IR-snapshot regen was `passed: 560 failed: 2`. The difference is - fully accounted for by the 2 still-RED pin tests (repaired in - T5) and by the IR-snapshot regen being out-of-band of the - per-task counters in the plan. Final state at T5 Step 5 is - exactly `passed: 562 failed: 0`, matching the plan's iter target. -- IR snapshot regeneration was not explicitly scripted in the plan - for Task 4, but it follows the hs.4 precedent (which regenerated - the same 5 snapshots for the same reason: a new unconditional - declare line in the IR header). The sole diff in each snapshot - is the new `declare ptr @ailang_str_concat(ptr, ptr)` line on - IR line 17, immediately before the existing - `declare i64 @llvm.fptosi.sat.i64.f64(double)` declaration. - -## Known debt - -- (none) - -## Files touched - -Code: -- `runtime/str.c` (M) — append `ailang_str_concat` helper -- `crates/ailang-check/src/builtins.rs` (M) — install entry, list() - row, signature unit test -- `crates/ailang-codegen/src/lib.rs` (M) — extern declare, - lower_app arm, is_builtin_callable list, IR-pin unit test -- `crates/ail/tests/str_concat_e2e.rs` (new) — E2E pin -- `crates/ail/tests/unbound_in_instance_method_pin.rs` (M) — rename - `str_concat` → `format_label` (4 sites + 1 test name) -- `examples/show_user_adt_with_label.ail` (new) — Show body fixture -- `examples/bug_unbound_in_instance_method.ail` (M) — rename - `str_concat` → `format_label` (1 site) - -IR snapshots regenerated (5; sole diff = new declare line): -- `crates/ail/tests/snapshots/hello.ll` -- `crates/ail/tests/snapshots/list.ll` -- `crates/ail/tests/snapshots/max3.ll` -- `crates/ail/tests/snapshots/sum.ll` -- `crates/ail/tests/snapshots/ws_main.ll` - -Docs: -- `docs/DESIGN.md` (M) — new §"Heap-Str primitives" subsection -- `docs/roadmap.md` (M) — struck `[x] [feature] str_concat` line - -## Stats - -bench/orchestrator-stats/2026-05-13-iter-str-concat.json diff --git a/docs/journals/2026-05-14-audit-pd.md b/docs/journals/2026-05-14-audit-pd.md deleted file mode 100644 index 009a61c..0000000 --- a/docs/journals/2026-05-14-audit-pd.md +++ /dev/null @@ -1,119 +0,0 @@ -# audit-pd — milestone close (prelude-decouple) - -**Date:** 2026-05-14 -**Scope:** commits e6298f5..9a8d385 (spec → pd.1 plan → pd.1 → housekeeping → pd.2 plan → pd.2 → pd.3 plan → pd.3) -**Status:** DONE — drift fixed inline as `audit-pd-tidy`; bench ratified pristine - -## Architect drift review - -`ailang-architect` returned **drift_found** with one Important + one -Minor + one Nit: - -- `[Important]` `docs/DESIGN.md` lines 2095-2099 + 2140-2143 — §"Roundtrip - Invariant" point 4 ("Carve-out anchor") still asserted "Eight - `.ail.json`-only fixtures (seven subject-matter rejection tests + - one compile-time-embed for the prelude)" and §Enforcement still - said "exactly the eight named carve-out files exist", but - post-pd.3 the count is seven and the compile-time-embed clause - is empty. The form-a spec was retired in place during pd.3 (§C4 (b) - RETIRED status marker); DESIGN.md was missed. -- `[Minor]` `crates/ail/src/main.rs` lines 2438-2448 — `local_types.get("prelude")` - fallback in `rewrite_type` (the migrate-canonical-types subcommand's - helper) was dead-by-construction after pd.3 retired the synthetic - prelude load in the same subcommand. A literal `"prelude"` string - in CLI scope that survived a milestone whose explicit purpose was - to delete exactly that pattern in core; the same drift category - one crate over. -- `[Nit]` `crates/ailang-core/src/lib.rs` — three pre-existing cargo - doc warnings on dangling `[load_workspace]` references carried - from pd.1 (already recorded as known debt in pd.1 + pd.2 journals). - Pre-existing baseline; not in this audit's scope. The roadmap's - P2 "Rustdoc warning sweep" todo covers it. - -Both Important and Minor were fixed inline as `audit-pd-tidy` per -CLAUDE.md "trivial mechanical edit" carve-out (≤30 LOC, no -judgement). The Nit was deferred to its existing roadmap todo. - -### Inline fixes applied - -- `docs/DESIGN.md:2095-2099` rewritten: "Seven `.ail.json`-only - fixtures (subject-matter rejection tests) ... The prelude's - compile-time-embed carve-out, originally listed here as the eighth - fixture, was retired 2026-05-14 by milestone prelude-decouple; - the prelude is now embedded as `examples/prelude.ail` in - `ailang-surface` and parsed at compile time via - `ailang_surface::parse_prelude`." -- `docs/DESIGN.md:2141` rewritten: "exactly the seven named - carve-out files exist". -- `crates/ail/src/main.rs:2427-2448` — the `or_else` arm scanning - `local_types["prelude"]` was deleted (5 LOC body); replaced with - a multi-line comment explaining why (with a forward pointer for - any future migrate variant that needs prelude awareness: - repopulate `local_types["prelude"]` explicitly via - `parse_prelude`). The owner-search now scans `import_names` only. - No behavioural change against the current corpus (the `or_else` - arm could never fire post-pd.3). - -`cargo test -p ailang-core --test carve_out_inventory` PASS post-fix. -`cargo build -p ail` clean. - -## Bench-regression check - -| Script | Exit | Result | -|--------|------|--------| -| `bench/check.py` | 1 | 63 metrics; 1 regressed (`latency.implicit_at_rc.p99_us` +14.42%, p99_9 +16.17%, max +27.61%, p99_over_median +9.79% — same `implicit_at_rc` cluster), 4 improved (`latency.explicit_at_rc.*` cluster), 58 stable. **Established 14-audit envelope-noise.** | -| `bench/compile_check.py` | 1 | 24 metrics; 11 regressed, 0 improved, 13 stable. The build-time-O0 noise cluster (audit-form-a, clippy-sweep). All within the 20% tolerance, ratifying as noise. | -| `bench/cross_lang.py` | 0 | 25 metrics; 0 regressed, 0 improved, 25 stable. Clean. | - -`prelude-decouple` is a workspace-loader refactor + spec text + -file-system + test code only. Zero codegen / runtime / typecheck -path edited. The bench numbers cannot be causally affected by this -milestone; the `implicit_at_rc.p99_*` oscillation and the build-time -noise are the same long-running envelope every audit since audit-cma -has flagged. Baseline left pristine (15th consecutive audit decision -to do so on this envelope; the right resolution is a methodology -fix — already queued as P3 "Latency methodology rework" — not -per-audit baseline-juggling). - -## Resolution - -- `[Important]` DESIGN.md drift — FIXED inline (audit-pd-tidy). -- `[Minor]` `main.rs` dead prelude fallback — FIXED inline (audit-pd-tidy). -- `[Nit]` pre-existing cargo doc warnings — DEFERRED to P2 "Rustdoc - warning sweep" (existing roadmap todo). -- Bench check.py + compile_check.py exit 1 — RATIFIED as noise - envelope, baseline pristine, 15th consecutive observation. - -## Files touched - -- `docs/DESIGN.md` (two paragraphs in §"Roundtrip Invariant") -- `crates/ail/src/main.rs` (5 LOC body deletion + 8 LOC replacement - comment in `rewrite_type`) -- `docs/journals/2026-05-14-audit-pd.md` (this file) - -No bench baseline files modified. - -## Milestone status - -`prelude-decouple` milestone: **CLOSED**. The roadmap entry at P0 -moves to `[x]` and is removed from the active queue (the milestone -trace stays in the journal index for chronological context). The -post-milestone state of the working tree: - -- `examples/prelude.ail.json` deleted; the prelude exists on disk - only as `examples/prelude.ail`. -- `crates/ailang-core/src/` contains zero literal-`"prelude"` strings. -- `crates/ail/src/main.rs` contains zero literal-`"prelude"` strings - in production code (the migrate-subcommand fallback is gone post- - audit-tidy). -- The two-phase workspace-loading composition `(load_modules_with → - caller-side prelude inject → build_workspace)` is the canonical - shape, exposed as public API in `ailang-core` and composed by - `ailang-surface::load_workspace`. -- `ailang_surface::parse_prelude` + `PRELUDE_AIL` are the canonical - embed surface, re-exported from `ailang-surface/src/lib.rs`. -- The CLAUDE.md "authors write `.ail`; the build derives the - JSON-AST in-process via `ailang_surface::parse`" doctrine holds - without exception; the prelude was the last carve-out and is now - inside the doctrine. -- 573 tests green workspace-wide. diff --git a/docs/journals/2026-05-14-iter-bugfix-mono-cursor-print-with-class-method-arg.md b/docs/journals/2026-05-14-iter-bugfix-mono-cursor-print-with-class-method-arg.md deleted file mode 100644 index 20c3e2b..0000000 --- a/docs/journals/2026-05-14-iter-bugfix-mono-cursor-print-with-class-method-arg.md +++ /dev/null @@ -1,95 +0,0 @@ -# iter bugfix-mono-cursor-print-with-class-method-arg — `(app print (app eq …))` no longer rewrites the inner class-method to a stale `Show T` symbol - -**Date:** 2026-05-14 -**Started from:** 05e4c04e3ba74f3382216ff3ea285e3fb1e3a45d -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -The iter 24.3 `synth` Var-arm pushes BOTH a class `ResidualConstraint` -for each declared constraint AND a `FreeFnCall` observation at the -same poly-free-fn `Var` site (`crates/ailang-check/src/lib.rs` ≈ 2908– -2966). The two mono walkers in `crates/ailang-check/src/mono.rs`, -however, were each consuming exactly one slot per `Var`: at a -poly-free-fn `Var` they popped one `free_fn_slot` and zero -`class_slot`s. Every constraint on a poly-free-fn-with-constraints -left a leftover class slot, so the next class-method `Var` in the -same body consumed the wrong slot — the residual that belonged to -`print`'s `Show T` constraint got cursor-aligned with the inner `eq` -`Var`, and codegen reported `call prelude.show__Bool arg type -mismatch: expected i1, got i64`. - -The fix advances both walkers' cursors by `1 + N` at a poly-free-fn -`Var` whose source `Type::Forall` carries `N` class constraints. A -new sibling helper `poly_free_fn_constraint_counts_for_module` -mirrors `poly_free_fn_names_for_module`'s three-source spelling -contract (same-module / implicit-import bare / dot-qualified -`alias.name`) and produces a `BTreeMap` from synth- -visible name to declared-constraint count. The map is threaded into -the rewrite-phase loop alongside `poly_free_fns` and passed to both -`rewrite_mono_calls` and `interleave_slots`. - -In `interleave_slots`, a poly-free-fn `Var` now pushes the `FreeFn` -slot first (this is what the rewrite walker reads for the rename), -then `N` filler entries drawn in order from `class_slots` (consumed -only to keep `class_cur` aligned). In `rewrite_mono_calls`, the -`Var`-arm captures `n_filler` from the count map BEFORE the rename -mutates `name` (lookup keys are the synth-visible spelling), reads -the slot at `*cursor` as before, then advances by `1 + n_filler`. -Class-method `Var`s stay at advance-by-one (synth pushes exactly one -residual and no `FreeFnCall`). - -## Per-task notes - -- task 1: write/verify the cursor-alignment fix. - - Reproduced RED at `print_with_class_method_arg_does_not_misalign_mono_cursor` - with the exact failure mode stated in the carrier: - `call prelude.show__Bool arg type mismatch: expected i1, got i64`. - - Added `poly_free_fn_constraint_counts_for_module` helper - immediately after `poly_free_fn_names_for_module`; identical - name-source contract, returning constraint counts instead of a - presence set. - - Threaded `poly_free_fn_ccounts: &BTreeMap` into - `collect_residuals_ordered`, `interleave_slots`, and - `rewrite_mono_calls` (signatures + every recursive call site). - - `interleave_slots::Term::Var` poly-free-fn branch: after pushing - the `FreeFn` slot, drain `N` entries from `class_slots` and push - them as fillers in `out`, advancing `class_cur` by `N`. - - `rewrite_mono_calls::Term::Var`: compute `n_filler` from the - count map before the rename mutation, then `*cursor += 1 + - n_filler`. Class-method `Var`s are guarded with `if - is_poly_free_fn { … } else { 0 }`. - - RED `print_with_class_method_arg_does_not_misalign_mono_cursor` - flips to GREEN with stdout `false` as predicted by the fixture - docstring. - - Full `cargo test --workspace` passes 564 / 564 (= 563 baseline - from `clippy-sweep` + the 1 new mini-mode RED). Zero - regressions across any of the existing show / mono / typeclass - test suites. - -## Concerns - -(none) - -## Known debt - -(none — the constraint set is a single new sibling helper plus a -new threaded argument; the two-walker symmetry remains the invariant -the existing comments already document.) - -## Files touched - -- `crates/ailang-check/src/mono.rs` — the fix (helper + two walker - signatures + Var-arm cursor advancement + recursive-call - threading). -- `crates/ail/tests/show_print_e2e.rs` — RED pin - `print_with_class_method_arg_does_not_misalign_mono_cursor` - (committed by `/debug` upstream of this iter; flips RED → GREEN - here). -- `examples/print_eq_arg_repro.ail` — RED fixture (committed by - `/debug` upstream; consumed by the pin). - -## Stats - -bench/orchestrator-stats/2026-05-14-iter-bugfix-mono-cursor-print-with-class-method-arg.json diff --git a/docs/journals/2026-05-14-iter-bugfix-print-leak-show-ret-mode.md b/docs/journals/2026-05-14-iter-bugfix-print-leak-show-ret-mode.md deleted file mode 100644 index 5d183eb..0000000 --- a/docs/journals/2026-05-14-iter-bugfix-print-leak-show-ret-mode.md +++ /dev/null @@ -1,85 +0,0 @@ -# iter bugfix-print-leak-show-ret-mode — print() leaks heap-Str under --alloc=rc; Show class method's ret_mode stripped twice - -**Date:** 2026-05-14 -**Started from:** 301cbc33a0b2fed18f465c6c17041211ff7da35e -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -Two-layer bug: (i) the `Show` class method declaration in -`examples/prelude.ail.json` omitted the `ret_mode` key, so serde -defaulted it to `ParamMode::Implicit`; (ii) even with `ret_mode: own` -declared, `substitute_rigids` in `crates/ailang-check/src/lib.rs` -was hard-resetting `param_modes` to `vec![]` and `ret_mode` to -`Implicit` in its `Type::Fn` arm — silently stripping the mode -metadata during class-method mono synthesis (`show__Int`, `eq__T`, -`compare__T`, user-instance `eq__`, etc.). Codegen's -`is_rc_heap_allocated` App-arm requires the callee's `ret_mode == Own` -to mark the let-binder trackable, so print's `let s = show x` binder -was never `ailang_rc_dec`'d at let-scope-close. The fix declares -`ret_mode: "own"` on the Show class method (parallel to the -`int_to_str`/`bool_to_str`/`float_to_str`/`str_clone`/`str_concat` -builtins which already carry `ret_mode: Own`) and repairs -`substitute_rigids`'s `Type::Fn` arm to clone `param_modes` and -preserve `ret_mode` through rigid substitution. The Form-A sibling -`examples/prelude.ail` is regenerated via `ail render` to stay in -lock-step per spec §C4 (b). Ten pinned mono-body hashes -(`eq__Int/Bool/Str`, `compare__Int/Bool/Str`, `show__Int/Bool/Str/Float`, -plus `eq__IntBox`) are re-pinned to the new GREEN baselines because -the mono bodies now carry the previously-stripped `param_modes` / -`ret_mode` metadata; bodies are semantically identical. - -## Per-task notes - -- bugfix.1: applied the fix. - - `examples/prelude.ail.json`: Show class method now has - `"ret_mode": "own"`. - - `examples/prelude.ail`: regenerated via - `cargo run -p ail -- render examples/prelude.ail.json - > examples/prelude.ail`; round-trips parse-isomorphic. - - `crates/ailang-check/src/lib.rs` `substitute_rigids` Type::Fn - arm: pattern-match all five fields explicitly; clone - `param_modes` and pass `*ret_mode` through (`ParamMode: Copy`). - Comment cites the bugfix iter id and the cause. - - `crates/ail/tests/mono_hash_stability.rs`: re-pinned 10 hashes - across the Eq/Ord and Show test fns; both bodies now report - `captured: {h}` on assertion failure (the Eq/Ord arm was - missing the captured-suffix; small in-place tidy folded into - the re-pin since the assertion-message string was being - rewritten anyway). - - `crates/ail/tests/eq_ord_e2e.rs`: re-pinned `eq__IntBox` - body hash to `3c4cf040cb4e8bb2`. - -## Concerns - -The debugger's cause analysis (carrier) pointed only at the JSON -omission. The JSON-only fix leaves the test RED — `substitute_rigids` -was silently stripping modes too. Suggest: at the next /audit pass, -verify there are no *other* call sites that build a `Type::Fn` with -explicit `..` field-spread and end up dropping modes. The pattern -`Type::Fn { ..., .. }` is the smell. - -## Known debt - -None — the fix is local. The wider question of "should `Type::Fn`'s -mode fields be field-init-by-name everywhere to prevent this class -of bug at the compiler level" is out of scope for this minimal fix. - -## Files touched - -- `examples/prelude.ail.json` — Show class: added `ret_mode: own`. -- `examples/prelude.ail` — regenerated Form-A sibling. -- `crates/ailang-check/src/lib.rs` — `substitute_rigids` Type::Fn - arm: preserve `param_modes` + `ret_mode`. -- `crates/ail/tests/mono_hash_stability.rs` — re-pinned 10 hashes - (6 Eq/Ord + 4 Show); added cause comments. -- `crates/ail/tests/eq_ord_e2e.rs` — re-pinned `eq__IntBox` hash. -- `crates/ail/tests/print_no_leak_pin.rs` (from /debug) — RED-pin - for the leak; now GREEN. -- `examples/print_int_no_leak_pin.ail` (from /debug) — fixture for - the RED-pin; now GREEN. - -## Stats - -bench/orchestrator-stats/2026-05-14-iter-bugfix-print-leak-show-ret-mode.json diff --git a/docs/journals/2026-05-14-iter-cli-diag-human.md b/docs/journals/2026-05-14-iter-cli-diag-human.md deleted file mode 100644 index 40c381a..0000000 --- a/docs/journals/2026-05-14-iter-cli-diag-human.md +++ /dev/null @@ -1,72 +0,0 @@ -# iter cli-diag-human — non-JSON CLI surfaces `WorkspaceLoadError` with bracketed `[code]` - -**Date:** 2026-05-14 -**Started from:** 6755060 (post-rpe.1.tidy) -**Status:** DONE - -## Summary - -Closes the P2 todo "CLI human-mode diagnostic surface for -`WorkspaceLoadError`" surfaced by the ct.1.8 tester (2026-05-11). -Pre-iter, the non-JSON path of every CLI subcommand that called -`ailang_surface::load_workspace(&path)?` propagated the error via -anyhow's Display — producing `Error: ` -on stderr — while the JSON path of `ail check --json` correctly -routed through `workspace_error_to_diagnostic` and emitted the -bracketed `[code]` prefix. The asymmetry made the human-mode -output harder to triage and less consistent across the toolchain. - -Fix: new private helper `load_workspace_human(path) -> Result` -in `crates/ail/src/main.rs` that wraps `ailang_surface::load_workspace`, -threads any `WorkspaceLoadError` through the existing -`workspace_error_to_diagnostic`, formats the resulting `Diagnostic` -into a single stderr line of shape -`error: [] : `, and exits non-zero. Pure I/O -errors (where `workspace_error_to_diagnostic` returns `None` per -the loader's own contract) still propagate as anyhow errors so the -caller's `?`-flow stays well-behaved. - -Nine call sites in `main.rs` swapped from -`ailang_surface::load_workspace()?` to -`load_workspace_human()?`: the `ail check` (non-JSON arm), -`build`, `run`, `emit-ir`, `prose`, `describe`, `deps`, `diff`, -`manifest` paths. The JSON arm of `ail check` (line 596) kept its -explicit `match` shape — it builds a structured diagnostics array -on stdout, which is a different output contract. - -The `BareCrossModuleTypeRef` arm of `workspace_error_to_diagnostic` -gained the existing migration-command hint -("Run `ail migrate-canonical-types` to fix legacy fixtures.") so -that the ct.1.8 actionable-hint test -(`check_human_mode_emits_actionable_message_to_stderr`) stays -green. The hint was previously carried by `WorkspaceLoadError`'s -thiserror Display impl; making `workspace_error_to_diagnostic` the -single source for both JSON and human paths required moving it -into the Diagnostic message body. - -## Files touched - -- `crates/ail/src/main.rs` — new `load_workspace_human` helper - (~25 lines, including doc comment); 9 call-site swaps; one - Diagnostic message gained the migration-hint suffix. -- `crates/ail/tests/cli_diag_human_workspace_load_error.rs` — - new RED-pin test (tempdir-based fixture asserting - `[module-not-found]` in stderr on a missing-import workspace). -- `docs/roadmap.md` — entry struck through. - -## Verification - -- RED test `check_non_json_emits_bracketed_module_not_found` - flipped from RED to GREEN. -- `cargo test --workspace` → 565 / 0 / 3 (was 564 pre-iter; +1 - from this iter's RED pin). -- `cargo clippy --workspace --all-targets` → 0 warnings. -- `cargo doc --workspace --no-deps` → 0 warnings. - -## Concerns - -- (none) - -## Known debt - -- (none) diff --git a/docs/journals/2026-05-14-iter-clippy-sweep.md b/docs/journals/2026-05-14-iter-clippy-sweep.md deleted file mode 100644 index 71e23bf..0000000 --- a/docs/journals/2026-05-14-iter-clippy-sweep.md +++ /dev/null @@ -1,134 +0,0 @@ -# iter clippy-sweep — clear all 61 `cargo clippy` warnings - -**Date:** 2026-05-14 -**Started from:** 04258c5 (post-WhatsNew of rustdoc-sweep + drift-test-narrowing) -**Status:** DONE - -## Summary - -Hygiene sweep against `cargo clippy --workspace --all-targets`. Before: -61 warnings across all eight crates. After: zero. Tests stay 563 green; -`cargo doc --workspace --no-deps` stays at zero warnings (the -rustdoc-sweep baseline); all three bench scripts (`bench/check.py`, -`bench/compile_check.py`, `bench/cross_lang.py`) all exit 0 against -the existing baselines — a free Stabilitätsbeweis that the sweep -touched no semantics. - -Twelve warning classes, fixed with three different shapes: - -- **Pure documentation fixes** (~32 warnings). `doc_lazy_continuation` - on multi-line `///` and `//!` comments where clippy parsed the - second line as a list continuation; resolved by either adding a - blank line, escaping the marker, or rephrasing the offending line. - Affected: `lex.rs:235-243` (3-line bullet list), `lib.rs:2360-2368` - in `ailang-check` (3-line bullet list), `lib.rs:3736` in - `ailang-check` (a `> 1` numeric comparison parsed as a quote - marker; rephrased to "any cardinality above one"), `drop.rs:570-585` - (numbered list + paragraph mix), `e2e.rs:915-920` (a `+ three - primitive instances` line where `+` was parsed as bullet marker), - `e2e.rs:1607-1614`, `typeclass_22b3.rs:706-708`, `eq_ord_e2e.rs:13`. - Plus four `empty_line_after_doc_comments` hits in - `workspace.rs:1675-2506` — these were `///` doc-comments left - orphaned by form-a.1 Task 5 test relocation; converted to plain - `//` comments (eight blocks total) since they no longer document - any function. - -- **Idiomatic refactors** (8 warnings). `err_expect` (5 hits in - `parse.rs`): `.err().expect("…")` → `.expect_err("…")`. - `useless_conversion` (3 hits): `.extend(x.into_iter())` → `.extend(x)` - in `desugar.rs:275` + `lift.rs:201`; `vec!["x".into()]` → - `vec!["x"]` in `lib.rs:5522` (where the call-site `fn_def` - parameter is `Vec<&str>` so the conversion was identity). - `redundant_closure` (2 hits): `|f| collect_pattern_binders(f)` → - `collect_pattern_binders` in `linearity.rs:717`; same pattern in - `loader.rs:108`. `trim_split_whitespace`: `.trim().split_whitespace()` - → `.split_whitespace()` in `prose/src/lib.rs:2103` (since - `split_whitespace` already ignores leading/trailing whitespace). - `single_char_add_str`: `out.push_str("x")` → `out.push('x')` in - `prose/src/lib.rs:383`. `collapsible_match` (2 hits): nested - `if let Some(_) = …` collapsed to `if let Some(Some(_)) = …` in - `mono.rs:1028` + `drop.rs:535-536`. - -- **`derivable_impls`** (2 hits). `impl Default for ParamMode { - fn default() -> Self { ParamMode::Implicit } }` in `ast.rs:731` - → `#[derive(Default)]` + `#[default] Implicit`. Same shape for - `AllocStrategy` in `codegen/src/lib.rs:164`. The derive form is - strictly more explicit (the `#[default]` attribute marks which - variant in-source rather than the impl body referencing it by - name) so the change is also a small clarity gain. - -- **`blocks_in_conditions`** (1 hit). The synth `Term::Var` arm in - `check/src/lib.rs` had a 13-line block inside an `else if` - condition (the mq.2 dispatch entry: `else if { let (m, q) = - parse_method_qualifier(name); qualifier_is_class_shape(&q) && - env.method_to_candidate_classes.contains_key(m) }`). Extracted - to a new helper `fn is_class_method_dispatch(name, env) -> bool` - next to `qualifier_is_class_shape`. The helper inherits the - block's full doc-comment so the why-each-branch reasoning is - preserved at the call site. - -- **`only_used_in_recursion`** (1 hit). `walk_pattern(p, f)` in - `workspace.rs:1399` carries an `f: &mut F` parameter that - Pattern walking never invokes (ct.1 left this seam unused: no - Pattern variant carries a cross-module reference). Rather than - delete `f` (which would break the symmetry with the five other - `walk_*` framework functions in this file that all share the - same `(_, f: &mut F) -> Result<…>` signature), added - `#[allow(clippy::only_used_in_recursion)]` with an inline - comment naming the framework-uniformity rationale. - -- **`if_same_then_else`** (2 hits). Both in `qualify_local_types` - shapes (`check/src/lib.rs:3457` + `codegen/src/subst.rs:165`): - the if-chain has two branches returning `name.clone()` for - semantically distinct reasons (`name.contains('.')` = already - qualified; `is_primitive_name(name)` = primitive needs no - qualification). Combining the branches with `||` would obscure - the reasoning. Added `#[allow(clippy::if_same_then_else)]` with - an inline comment naming why each branch disqualifies the name. - -- **`too_many_arguments`** (1 hit). `Emitter::new` takes 8 of the - workspace-flat tables the emitter needs at construction time. - Bundling them into a context struct would just rename the - boilerplate without reducing it. Added - `#[allow(clippy::too_many_arguments)]` with the rationale - inline. - -## Files touched - -- `crates/ailang-core/src/ast.rs` (1 derive flip) -- `crates/ailang-core/src/desugar.rs` (1 .extend) -- `crates/ailang-core/src/workspace.rs` (1 walk_pattern allow + 8 `///`→`//` blocks) -- `crates/ailang-surface/src/lex.rs` (1 doc-list blank line) -- `crates/ailang-surface/src/parse.rs` (5 `.expect_err`) -- `crates/ailang-surface/tests/loader.rs` (1 redundant closure) -- `crates/ailang-prose/src/lib.rs` (push_str + trim removal) -- `crates/ailang-check/src/lib.rs` (helper extraction + 1 allow + 1 doc rephrase + 1 doc-list blank line + 1 vec literal) -- `crates/ailang-check/src/lift.rs` (1 .extend) -- `crates/ailang-check/src/linearity.rs` (1 redundant closure) -- `crates/ailang-codegen/src/lib.rs` (1 derive flip + 1 too_many_args allow) -- `crates/ailang-codegen/src/subst.rs` (1 if_same_then_else allow) -- `crates/ailang-codegen/src/drop.rs` (1 doc-list blank line + 1 collapsible_match) -- `crates/ailang-check/src/mono.rs` (1 collapsible_match) -- `crates/ail/tests/e2e.rs` (2 doc rephrasings) -- `crates/ail/tests/typeclass_22b3.rs` (1 doc rephrasing) -- `crates/ail/tests/eq_ord_e2e.rs` (1 doc rephrasing) - -## Verification - -- `cargo clippy --workspace --all-targets 2>&1 | grep -c '^warning:'` - → `0` (was 61). -- `cargo test --workspace` → 563 passed + 3 ignored, 0 failed - (baseline holds). -- `cargo doc --workspace --no-deps 2>&1 | grep -c '^warning:'` → - `0` (rustdoc-sweep baseline holds — no regression). -- `bench/check.py` → exit 0. -- `bench/compile_check.py` → exit 0. -- `bench/cross_lang.py` → exit 0. - -## Concerns - -- (none) - -## Known debt - -- (none) diff --git a/docs/journals/2026-05-14-iter-pd.1.md b/docs/journals/2026-05-14-iter-pd.1.md deleted file mode 100644 index 5e27eef..0000000 --- a/docs/journals/2026-05-14-iter-pd.1.md +++ /dev/null @@ -1,120 +0,0 @@ -# iter pd.1 — core API split (load_modules_with + build_workspace) — surface unchanged via shim - -**Date:** 2026-05-14 -**Started from:** 61167a4ef0ade5dd65dadfbdde1a44b65c6693ca -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -pd.1 carved out two new public fns in `crates/ailang-core/src/workspace.rs` -(`load_modules_with` for DFS-only loading, `build_workspace` for validation + -registry construction with a caller-supplied `implicit_imports: &[&str]`), -threaded the new parameter through `validate_canonical_type_names` / -`check_class_ref` / `check_type_con_name`, and replaced the four -hardcoded literal-`"prelude"` fallback blocks in the diagnostic helpers -with a parameterised iter-loop over `implicit_imports`. The surviving -`load_workspace_with` collapses from a ~70-line inline DFS+inject+validate -block to a 3-line composition (`load_modules_with` → caller-side prelude -inject → `build_workspace`). The legacy JSON-only wrapper `pub fn -load_workspace` retired with zero production callers (only doc-comment -references and 9 in-mod test callers; the latter migrated to -`load_workspace_with(&entry, load_one)`). Surface (`ailang-surface`) and -all downstream crates compile + test unchanged because the shim -preserves today's `load_workspace_with` semantics; the regression-pin -test asserts the prelude module hash matches `load_prelude()` post-refit. -Production-code literal-`"prelude"` count in workspace.rs dropped from 8 -to exactly the 4 expected sites (all in the shim, none in the -diagnostic helpers). pd.2 will move the prelude-embed + the inject step -into `ailang-surface` and retire the shim; pd.3 will delete -`examples/prelude.ail.json`. - -## Per-task notes - -- iter pd.1.1: added `pub fn load_modules_with` above `load_workspace_with` - (extracted DFS pre-amble verbatim from the old inline body); RED test - `load_modules_with_returns_modules_without_prelude` proves the new fn - returns modules without auto-injecting prelude. First-shot GREEN. -- iter pd.1.2: added `pub fn build_workspace` next to `load_modules_with` - (validation + registry construction; takes pre-assembled modules-map - and `implicit_imports: &[&str]`). RED test - `build_workspace_accepts_assembled_modules_and_runs_validation` ends - the task RED-on-purpose per the plan (the inner call to - `validate_canonical_type_names` doesn't have the parameter yet); - Task 3 lands the green. -- iter pd.1.3: added `implicit_imports: &[&str]` parameter to - `validate_canonical_type_names`, `check_class_ref`, - `check_type_con_name`; threaded it through 6 internal call sites in - `validate_canonical_type_names` (2 in the type-walk + 4 in the - class-ref match-arms); rewrote the two literal-`"prelude"` fallback - blocks in the diagnostic helpers as iter-loops over `implicit_imports`. - RED test `implicit_imports_arg_drives_prelude_fallback_in_diagnostics` - exercises both branches (with-prelude includes `prelude.Eq` candidate; - without-prelude excludes it). Migrated 18 in-mod test callers + 1 - production callsite (the still-inline shim body) to - `&["prelude"]`. Task 2's test now passes. New helpers - `module_with_bare_classref` and `module_with_class_def` use the - existing `serde_json::from_value(json!(...))` pattern (the rest of - mod tests' convention) rather than the plan's struct-literal style - (which would have failed against actual `ClassDef.superclass: - Option<...>` field shape). -- iter pd.1.4: replaced `load_workspace_with`'s ~70-line body with the - 3-line shim composition; doc-comment rewritten to describe the shim - role + retain the ext-cli.1 cycle-avoidance rationale on the - loader-injection point. Pin test - `load_workspace_with_shim_preserves_today_semantics` asserts - prelude-module hash equality with `load_prelude()` post-refit. -- iter pd.1.5: deleted `pub fn load_workspace`; removed it from - `lib.rs` re-export; rewrote the workspace.rs module rustdoc example - + 3 lib.rs `//!` references to point at `load_workspace_with`; - hoisted the orphaned "Algorithm: DFS over imports" doc-comment block - onto `load_modules_with` (where it now belongs). Migrated 9 in-mod - test callers from `load_workspace(&entry)` to - `load_workspace_with(&entry, load_one)`. Required `#[cfg(test)]` - gating of `fn load_one` and the `load_module` import (now consumed - only by the test mod) to silence dead-code warnings. -- iter pd.1.6: verification — `cargo test --workspace` all green - (every crate's tests pass); `bench/check.py` + `bench/compile_check.py` - + `bench/cross_lang.py` all exit 0; production literal-`"prelude"` - count in workspace.rs is exactly 4 (the four expected shim sites at - lines 553/555/558/560 — all in `load_workspace_with` body — none in - the diagnostic helpers). - -## Concerns - -- pd.1.2 ends RED-on-purpose per the plan. This is intentional and - documented in the plan, but if a future iter ever bisects through - this plan the workspace WILL NOT compile at the Task 2 boundary — - Task 3 must land in the same Boss commit. The Boss-side commit - shape for pd.1 is therefore "one cohesive iter commit" not "per-task - commits". -- pd.1.5 needed an unanticipated `#[cfg(test)]` gating step on the - `load_one` fn + the `load_module` import. The plan didn't anticipate - this because it framed the `load_workspace` retirement as "doc + lib - cleanup only", but `load_one` was the only consumer of those imports - in production. Recording so a future planner adds an "if removing a - caller, check whether its callees become test-only" check. - -## Known debt - -- pd.2 will move prelude-embed (`PRELUDE_JSON` const + `load_prelude` fn) - into `ailang-surface` and rewire surface's `load_workspace` to call - `load_modules_with` + `build_workspace` directly, retiring the shim. - Out of scope for pd.1 by spec. -- pd.3 will delete `examples/prelude.ail.json`. Out of scope for pd.1. -- The latency.implicit_at_rc improvement cluster appearing in - `bench/check.py` (4 metrics regressed, 2 improved) reappears for - the 4th audit in a row; cause not localised. pd.1 is a workspace.rs - refactor with zero codegen / runtime touch, so cannot be the source. - Baseline left pristine (consistent with audit-eob and earlier - decisions). - -## Files touched - -- `crates/ailang-core/src/workspace.rs` (extraction + threading + - shim refit + dead-fn delete + test-mod migration) -- `crates/ailang-core/src/lib.rs` (re-export drop + 3 rustdoc updates) - -## Stats - -`bench/orchestrator-stats/2026-05-14-iter-pd.1.json` diff --git a/docs/journals/2026-05-14-iter-pd.2.md b/docs/journals/2026-05-14-iter-pd.2.md deleted file mode 100644 index d440683..0000000 --- a/docs/journals/2026-05-14-iter-pd.2.md +++ /dev/null @@ -1,176 +0,0 @@ -# iter pd.2 — surface owns prelude embed; pd.1 shim retired - -**Date:** 2026-05-14 -**Started from:** 116157a2b84a98060c1b6c04eb4ec071fe40eb93 -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -pd.2 moves the prelude embed (`PRELUDE_AIL` const + `parse_prelude` fn) and -the prelude-injection step (with the `ReservedModuleName` reservation) out -of `ailang-core` into `ailang-surface`, rewires `surface::load_workspace` -from the 3-line shim-call into the 7-line composition `(load_modules_with -→ reserved-name check → inject parse_prelude → build_workspace(&["prelude"]))`, -and retires the pd.1 compatibility shim `core::load_workspace_with` along -with `load_one`, `load_prelude`, `PRELUDE_JSON`, and the `#[cfg(test)] use -crate::load_module;` import. Cross-form-identity preflight (hash-equality -between `parse(prelude.ail)` and `serde_json::from_str(prelude.ail.json)`) -PASSES at hash `3abe0d3fa3c11c99` — the load-bearing assumption that -justifies pd.3's deletion of `prelude.ail.json` is verified. Production -literal-`"prelude"` count in `core/src/` is now zero. Test count: 573 -green (from the pd.1 baseline of 564, +9 net = 3 new surface pins -[parse + hash + injection] + 6 tempdir/fixture relocations to -workspace_pin.rs minus 6 in-mod placeholders that became comments; -in-mod test deletion of the pd.1 shim test is offset by the -`load_modules_with_custom_loader_is_called` rename + 2 pd.1 tests that -now use an inline test-private `load_one` helper). `bench/cross_lang.py` -+ `bench/compile_check.py` exit 0; `bench/check.py` exit 1 with the -12-audit noise envelope (the `bench_list_sum.bump_s` + tail-latency -oscillation that has been observed in audits since audit-cma; pd.2 is -a workspace-loader refactor with zero codegen / runtime touch). pd.3 -will delete `examples/prelude.ail.json` plus the -`prelude_ail_and_json_parse_to_identical_module` preflight, the -`include_str!` carve-out in `crates/ail/src/main.rs:472-484`, the -`carve_out_inventory.rs` §C4(b) row, the form-a §C4(b) spec text, and -the `migrate-bare-cross-module-refs` defensive include. - -## Per-task notes - -- iter pd.2.1: `pub const PRELUDE_AIL: &str = include_str!("../../../examples/prelude.ail");` - + `pub fn parse_prelude() -> Module` added near the top of - `crates/ailang-surface/src/loader.rs`; both re-exported from - `ailang-surface/src/lib.rs`. RED test - `parse_prelude_returns_module_named_prelude_with_min_defs` failed - with the expected unresolved-import message, then GREEN. First-shot - clean. -- iter pd.2.2: `crates/ailang-surface/tests/prelude_module_hash_pin.rs` - with `prelude_ail_and_json_parse_to_identical_module` (cross-form- - identity preflight, load-bearing for pd.3) + `prelude_parse_yields_canonical_hash` - (literal-hex anchor). Preflight PASSED first-run (the round-trip - invariant from form-a.0 + form-a.1 was already exercising - parse-then-print idempotency on every `.ail` fixture, so - parse(prelude.ail) ≡ deserialize(prelude.ail.json) was not an - accident). Hash `3abe0d3fa3c11c99` captured at pd.2 close. -- iter pd.2.3: `surface::load_workspace` rewritten 3-line → 7-line - composition; imports updated `load_workspace_with` → bare - `Workspace, WorkspaceLoadError` with `load_modules_with` / - `build_workspace` fully-qualified inline; module rustdoc rewritten - to name the new injection point + the implicit-imports list. - Regression-pin tests `surface_load_workspace_injects_prelude_into_every_workspace` - + `surface_load_workspace_rejects_user_module_named_prelude` PASS - before AND after the rewrite (semantics preserved). DONE_WITH_CONCERNS: - the plan's example used `examples/hello.ail.json` which doesn't - exist post-form-a.1 (deleted in T8); switched to `examples/hello.ail`. -- iter pd.2.4: 10 in-mod tests RELOCATED to - `crates/ailang-core/tests/workspace_pin.rs` rather than aliased per - the plan's literal text — the dev-dep cycle (`ailang-core` ↔ - `ailang-surface` via `[dev-dependencies]`) makes - `surface::load_workspace` calls from `ailang-core`'s lib-test crate - structurally impossible (two distinct compilations of `ailang-core` - appear in the test build, types don't match). The integration-test - crate consumes both crates against the same `ailang-core` artefact, - so the cycle never materialises — same pattern form-a.1 T5 used for - similar relocations. The 11th test (`load_workspace_with_custom_loader_is_called`) - renamed to `load_modules_with_custom_loader_is_called` with the - closure adapted to wrap `crate::load_module` directly via the same - Io/Schema mapping `load_one` had. The 12th test - (`load_workspace_with_shim_preserves_today_semantics`) deleted - outright with a forwarding comment to the surface-side hash pin. -- iter pd.2.5: 5 deletions (`PRELUDE_JSON`, `load_prelude`, - `load_workspace_with`, top-level `load_one`, `#[cfg(test)] use - crate::load_module;`); module rustdoc reworked to two-phase - composition framing; lib.rs three rustdoc sites (lines 16-21, 50-51, - 120) re-pointed at `ailang_surface::load_workspace`. ONE plan-vs-actual - deviation: the plan didn't enumerate the 2 pd.1-introduced in-mod - tests at the bottom of `mod tests` that consume `load_one` directly - (`load_modules_with_returns_modules_without_prelude`, - `build_workspace_accepts_assembled_modules_and_runs_validation`); - preserved by adding a test-mod-private `load_one` helper inside `mod - tests` (production-symbol-level deletion preserved; helper is - scoped only to the test module, not exported). The - `build_workspace...` test also called `load_prelude()` directly; - replaced inline with `serde_json::from_str(include_str!(...))` of - `prelude.ail.json` (the same one-line code `load_prelude` was). Zero - literal-`"prelude"` strings in production `core/src/` confirmed by - grep. cargo doc warnings: 3 (unchanged from pre-pd.2 baseline; all - three are dangling `[`load_workspace`]` references in the `Workspace` - / `Registry` rustdoc that pd.1 left behind when it deleted - `load_workspace`; out of scope per the implementer's - no-surrounding-cleanup discipline, recorded as known debt). -- iter pd.2.6: bench verification — `cross_lang.py` exit 0 (25/25 - stable), `compile_check.py` exit 0 (24/24, "11 regressed" reported - but exit code says baseline-tolerance check passes — this is the - audit-form-a + clippy-sweep-named noise on the corpus's small build - fixtures), `check.py` exit 1 with 1 regression on `bench_list_sum.bump_s` - +13.61% (within established noise envelope; second run showed - same `bump_s` plus a tail-latency `implicit_at_rc.p99_us` blip). - All three are runtime/compile metrics that cannot be touched by a - workspace.rs/loader.rs/lib.rs refactor (no codegen / no runtime / - no typecheck path edited). - -## Concerns - -- pd.2.3: plan's regression-pin used `examples/hello.ail.json` which - doesn't exist post-form-a.1 T8. Switched to `examples/hello.ail`. - Recommend a planner check: when a plan references a `.ail.json` - fixture, verify the fixture exists post-form-a.1 (the carve-out - inventory is the right reference). -- pd.2.4: plan's "alias to surface_load_workspace and stay in-mod" was - structurally invalid because of the `ailang-core` ↔ `ailang-surface` - dev-dep cycle inside the lib-test target. Relocated to - `tests/workspace_pin.rs` instead — same pattern form-a.1 T5 used. - Recommend: planner recon-check should detect when a planned in-mod - test is meant to consume an external dev-dep cycle that the lib-test - target can't resolve. -- pd.2.5: plan's "delete `load_one`" preserved at production-symbol - level (the top-level `#[cfg(test)] fn load_one` is gone) by - introducing a test-mod-private `fn load_one` helper inside `mod - tests` for the 2 pd.1 in-mod tests that still consume it. The - symbol's name is preserved for clarity; its scope is now narrower - than pre-pd.2. -- pd.2.6: `check.py` exit 1 — established noise envelope, 13th - consecutive observation pattern (audit-cma onwards). pd.2 is a - workspace-loader refactor with zero codegen / runtime / typecheck - touch, so cannot be the source. Baseline left pristine consistent - with all prior audit decisions on this envelope. - -## Known debt - -- 3 cargo doc warnings on `[`load_workspace`]` references in the - `Workspace` (line 44, 49) + `Registry` (line 64) rustdoc — pd.1 - deleted `load_workspace` but didn't sweep these references. Pre- - pd.2 baseline already carried them; pd.2 didn't add to the count; - not within pd.2 scope (no-surrounding-cleanup discipline). -- pd.3 work explicitly out of scope: deleting `examples/prelude.ail.json`, - the `prelude_ail_and_json_parse_to_identical_module` preflight test - (which depends on the JSON file's existence), the `include_str!` of - `prelude.ail.json` in `crates/ail/src/main.rs:472-484`, the - `carve_out_inventory.rs` §C4(b) row, the form-a §C4(b) spec text - edits, and the `migrate-bare-cross-module-refs` defensive include. - -## Files touched - -Modified: -- `crates/ailang-core/src/lib.rs` (3 rustdoc sites updated) -- `crates/ailang-core/src/workspace.rs` (5 deletions + module rustdoc - reframe + 2 pd.1 in-mod test repairs + 6 in-mod test placeholders + - test-mod-private `load_one` helper restored at narrow scope) -- `crates/ailang-core/tests/workspace_pin.rs` (10 relocated tests - appended + helper fn `write_simple_module_json` + preamble extension) -- `crates/ailang-surface/src/lib.rs` (re-export of `parse_prelude` + - `PRELUDE_AIL`) -- `crates/ailang-surface/src/loader.rs` (`PRELUDE_AIL` const + - `parse_prelude` fn added; `load_workspace` body 3-line → 7-line - composition rewrite; module rustdoc reframed; imports updated) - -New: -- `crates/ailang-surface/tests/prelude_parse_pin.rs` (1 test) -- `crates/ailang-surface/tests/prelude_module_hash_pin.rs` (2 tests: - cross-form-identity preflight + literal-hex anchor `3abe0d3fa3c11c99`) -- `crates/ailang-surface/tests/prelude_injection_pin.rs` (2 tests: - inject + reservation pin) - -## Stats - -`bench/orchestrator-stats/2026-05-14-iter-pd.2.json` diff --git a/docs/journals/2026-05-14-iter-pd.3.md b/docs/journals/2026-05-14-iter-pd.3.md deleted file mode 100644 index ea85097..0000000 --- a/docs/journals/2026-05-14-iter-pd.3.md +++ /dev/null @@ -1,209 +0,0 @@ -# iter pd.3 — prelude.ail.json deleted; prelude-decouple milestone closed - -**Date:** 2026-05-14 -**Started from:** c0dd2bc334d490e5e90ad10501c33e48076ff2bd -**Status:** DONE -**Tasks completed:** 8 of 8 - -## Summary - -pd.3 closes the **prelude-decouple milestone** (pd.1 + pd.2 + pd.3) by -deleting `examples/prelude.ail.json` from the working tree, sweeping the -three remaining `include_str!` consumers (one in-mod test in -`workspace.rs`, the cross-form-identity preflight in -`prelude_module_hash_pin.rs`, the defensive include in `ail/src/main.rs`'s -`migrate-canonical-types` subcommand) and the lockstep skip-branch the -defensive include necessitated, dropping the §C4 (b) entry from -`carve_out_inventory.rs` (8 → 7 named carve-outs), retiring §C4 (b) of -`docs/specs/2026-05-13-form-a-default-authoring.md` in place with a -"Status: RETIRED 2026-05-14" marker that preserves the original framing -as historical context, and adding `prelude_decouple_carve_out_pin.rs` -as the milestone's permanent guard against regression. At milestone -close: the prelude exists on disk only as `examples/prelude.ail`; core -embeds zero prelude bytes; the `migrate-canonical-types` subcommand no -longer auto-injects a synthetic prelude entry; the doctrine "authors -write `.ail`; the build derives the JSON-AST in-process via -`ailang_surface::parse`" (CLAUDE.md) holds without exception. Test count: -573 green workspace-wide (preserved from pd.2; +1 carve-out pin -1 -cross-form-identity preflight +1 relocated `build_workspace_accepts...` -test -1 in-mod twin = net 0). `bench/cross_lang.py` + `bench/compile_check.py` -exit 0; `bench/check.py` exit 1 with the same envelope-noise pattern -observed since audit-cma (14th consecutive observation; pd.3 is a -file-system + spec-text + test deletion + dead-code cleanup with zero -codegen / runtime / typecheck touch, so cannot be the source). - -## Per-task notes - -- iter pd.3.1: `build_workspace_accepts_assembled_modules_and_runs_validation` - RELOCATED from `crates/ailang-core/src/workspace.rs::tests` (in-mod) to - `crates/ailang-core/tests/workspace_pin.rs` (integration crate) instead - of the literal call-site swap the plan prescribed. The plan specified - "use the fully-qualified `ailang_surface::parse_prelude()` at the call - site" claiming it was the workaround pd.2 used; in fact pd.2 worked - around this same dev-dep cycle by inlining - `serde_json::from_str(include_str!("prelude.ail.json"))` precisely - BECAUSE `parse_prelude()` doesn't typecheck from in-mod (the lib-test - target sees two distinct `ailang-core` compilations and the `Module` - types fail to unify). pd.3 cannot use the JSON workaround because the - JSON file is being deleted. The structural fix is the relocation - pattern pd.2.4 already used for 10 sibling tests — same crate edge, - same outcome. A 6-line stub-comment in `workspace.rs::tests` documents - the dev-dep cycle so future maintainers don't try to put the test - back. New test `load_one_adapter` helper at the top of `workspace_pin.rs` - mirrors the in-mod `load_one` exactly (Io/Schema mapping). DONE_WITH_CONCERNS: - plan defect (treated under pd.2.4 lesson recommendation but not - caught at planning time). -- iter pd.3.2: 5 deletions in `prelude_module_hash_pin.rs` (cross-form- - identity preflight test fn, `PRELUDE_JSON` const, `PRELUDE_AIL` import, - `const _: &str = PRELUDE_AIL;` shape-pin, lines 1-15 of original - module doc-comment); module doc-comment replaced verbatim with the - surviving-only version (preserves the historical record of why the - preflight existed, what it discharged, when it was retired). Hash - `3abe0d3fa3c11c99` still asserted by the surviving - `prelude_parse_yields_canonical_hash`. Zero orphan-import warnings. - First-shot clean. -- iter pd.3.3: defensive `include_str!` block (12 lines) deleted from - `Cmd::MigrateCanonicalTypes` arm; lockstep skip-branch (4 lines) - deleted from the rewrite loop's Pass 2. The plan referenced the - subcommand by its variant tag `MigrateBareCrossModuleRefs` but the - user-facing subcommand name is `migrate-canonical-types` — spot-test - ran under the correct name and the help text printed. DONE_WITH_CONCERNS: - plan referenced obsolete variant name; doc-comment update on - `MigrateCanonicalTypes` (one paragraph) was needed beyond strict task - text because the prelude-fallback capability that the doc-comment - asserted ("Prelude is consulted as an implicit last-resort fallback - when no listed import owns the name") was silently removed by Step 2 - — leaving the doc-comment in would have been an active mis-statement - in `--help` output. Update names pd.3 as the retirement point. -- iter pd.3.4: 3 edits to `carve_out_inventory.rs` (header sentence - "eight" → "seven"; comment block re-published per plan; `"prelude.ail.json"` - entry + its `// §C4 (b) — compile-time-embed` comment dropped from - EXPECTED). Test RED-on-purpose post-edit (file still on disk); - flips GREEN at Task 7. -- iter pd.3.5: Status marker inserted verbatim above §C4 (b) heading - in `docs/specs/2026-05-13-form-a-default-authoring.md`. Original §C4 - (b) text preserved as historical context (per the spec's convention - for retired sections). Reads coherently top-to-bottom; the marker - reads as a forward-pointer to the resolution and the original text - reads as historical motivation. -- iter pd.3.6: `crates/ailang-surface/tests/prelude_decouple_carve_out_pin.rs` - created verbatim per the plan. RED-on-purpose at this task (file still - exists); flips GREEN at Task 7. Failure message names the four things - a future maintainer must do to legitimately reintroduce the file — - pre-empts the "it's just one file" rationalisation. -- iter pd.3.7: pre-deletion verification clean (zero `include_str!` - consumers; the two non-`include_str!` `.rs` references are inert — - `print_no_leak_pin.rs:110` is a string-literal in an assertion - message, `prelude_injection_pin.rs:28` constructs a tempdir-relative - path that does not read the in-tree file). `rm examples/prelude.ail.json`. - Both pin tests flip RED → GREEN at this step. Workspace test count - unchanged at 573 green. -- iter pd.3.8: `bench/compile_check.py` exit 0; `bench/cross_lang.py` - exit 0; `bench/check.py` exit 1 with the established noise envelope - (the `bench_list_sum.bump_s` and `implicit_at_rc` tail-latency - oscillation observed since audit-cma — 14th consecutive observation). - Acceptance criterion sweep: zero `include_str!` of `prelude.ail.json` - workspace-wide; zero literal-`"prelude"` in `crates/ailang-core/src/` - production scope (the matches in `workspace.rs` are all inside - `#[cfg(test)] mod tests`); CLAUDE.md "Authors write `.ail`" sentence - is consistent with the post-milestone state without modification (the - prelude exception it implicitly tolerated pre-milestone is gone); - `docs/DESIGN.md` carries no production-doctrine assertions about core - embedding the prelude or referencing `prelude.ail.json`. - -## Concerns - -- pd.3.1: plan defect — the task text instructed a call-site swap - ("In each affected test fn body, replace the pattern ... with - `ailang_surface::parse_prelude()`") that doesn't typecheck due to - the dev-dep cycle. The plan also misattributed pd.2's actual - workaround (`serde_json::from_str(include_str!(...))`) as - successful use of `parse_prelude()` from in-mod. Recommend planner - recon-check: when a plan proposes a call-site change involving - `ailang_surface::*` from in-mod tests, verify the test target can - resolve the dev-dep edge (the lib-test target generally cannot; - the integration-test crate generally can — same lesson pd.2.4 - recorded but didn't propagate to the planner skill). -- pd.3.3: plan referenced subcommand by its internal variant tag - (`MigrateBareCrossModuleRefs`) instead of its user-facing name - (`migrate-canonical-types`). Trivial repair (test under correct - name) but recommend planner check: verify subcommand-arm references - in spot-test commands by enumeration-after-rename. -- pd.3.3 (deeper): the plan's "Architecture" preamble called the - defensive include "a documented no-op" because the inserted entry - was unconditionally skipped by the lockstep guard at lines 500-505. - This is correct about WRITING the rewritten module file back, but - not about CONSULTING `local_types["prelude"]` for fallback rewrites - (which the inserted entry populated, lines 487-497, before the Pass - 2 skip). Removing both blocks silently retired the prelude-fallback - capability of the rewrite logic. The plan called the change net-zero - behaviour; it isn't. The doc-comment update I made on - `MigrateCanonicalTypes` makes the new behaviour visible. The - rewrite-helper's prelude-fallback branch at lines ~2436-2446 in - `main.rs` is now dead-by-construction (the `local_types` map will - never have a `"prelude"` key, since no prelude module is loaded by - this subcommand) — left in place per the implementer's - no-surrounding-cleanup discipline; tagged below as known debt. - -## Known debt - -- `crates/ail/src/main.rs:~2436-2446`: the `local_types.get("prelude")` - fallback in `rewrite_def`'s `rewrite_type_con_owner` (or analogue) - is dead-by-construction post-pd.3 since the `local_types` map for - the `migrate-canonical-types` subcommand no longer contains a - `"prelude"` key. Out of pd.3's literal scope (no surrounding cleanup); - candidate for a follow-up tidy iter or for the post-pd.3 audit's - drift sweep. -- 3 cargo doc warnings on dangling `[`load_workspace`]` references in - `Workspace` and `Registry` rustdoc — pre-existing baseline carried - forward from pd.2; out of pd.3 scope. (Identical entry was in the - pd.2 known-debt section.) -- The `migrate-canonical-types` subcommand's substantive behaviour - (rewriting bare cross-module refs across a directory of `.ail.json` - files) hasn't been exercised since the canonical-type-names - milestone. No in-tree fixtures known to need rewriting today; spot- - tested via `--help` only. If a future migration is requested, the - prelude-fallback removal + the rewrite-helper dead-branch should - be reviewed together. - -## Files touched - -Modified: -- `crates/ail/src/main.rs` (delete defensive include 472-484, delete - skip-branch 500-505, update `MigrateCanonicalTypes` doc-comment to - reflect retired prelude-fallback) -- `crates/ailang-core/src/workspace.rs` (delete in-mod - `build_workspace_accepts_assembled_modules_and_runs_validation` test; - replace with 6-line stub comment documenting the dev-dep cycle as - invariant) -- `crates/ailang-core/tests/carve_out_inventory.rs` (header sentence - + comment block + EXPECTED list — 8 → 7) -- `crates/ailang-core/tests/workspace_pin.rs` (add `load_one_adapter` - helper at top; append the relocated test at end) -- `crates/ailang-surface/tests/prelude_module_hash_pin.rs` (delete - cross-form-identity preflight + supporting `PRELUDE_JSON` / - `PRELUDE_AIL` references; trim doc-comment to surviving-only - framing) -- `docs/specs/2026-05-13-form-a-default-authoring.md` (insert "Status: - RETIRED" marker above §C4 (b) heading) - -New: -- `crates/ailang-surface/tests/prelude_decouple_carve_out_pin.rs` - (single test asserting `examples/prelude.ail.json` does not exist) - -Deleted: -- `examples/prelude.ail.json` (the milestone's primary artefact - retirement) - -## Stats - -`bench/orchestrator-stats/2026-05-14-iter-pd.3.json` - -## Note on pre-existing dirty INDEX.md - -When this iter dispatched, `docs/journals/INDEX.md` was already -modified in the working tree with one Boss-side line for pd.2 (the -prior iter); that edit was not committed before pd.3 dispatch. This -agent did NOT touch INDEX.md (it is Boss-only per skill discipline). -The Boss should commit the pd.2 INDEX line either with the pd.3 -commit or as a separate prior commit — both options are clean. diff --git a/docs/journals/2026-05-14-iter-rpe.1.md b/docs/journals/2026-05-14-iter-rpe.1.md deleted file mode 100644 index 23c4cdb..0000000 --- a/docs/journals/2026-05-14-iter-rpe.1.md +++ /dev/null @@ -1,175 +0,0 @@ -# iter rpe.1 — Retire per-type print effect-ops - -**Date:** 2026-05-14 -**Started from:** 8b455bee4ca2b09bcadc56d95e0d79dcef75d284 -**Status:** DONE -**Tasks completed:** 10 of 10 - -## Summary - -The retire-per-type-print-effect-ops milestone shipped in one iter, -covering: a RED test pinning the deletion gate (`crates/ailang-check/ -tests/no_per_type_print_ops.rs`); 92 `examples/*.ail` fixture -migrations from `(do io/print_ x)` → `(app print x)` plus 6 -`.prose.txt` snapshot regenerations; four-site lockstep compiler -deletion (`crates/ailang-check/src/builtins.rs` × 5 regions, -`crates/ailang-codegen/src/lib.rs::lower_app` × 3 arms + 1 test, -`crates/ailang-codegen/src/synth.rs::builtin_effect_op_ret` × 1 -match arm), with the dead `intern_string` helper removed as a -clean follow-up; six test-body migrations (`ailang-check` × 2, -`ailang-core` tests × 2, `ailang-surface/src/lex.rs`, `ailang-prose`); -the Cat B test-harness patch in `crates/ail/tests/e2e.rs` (6 -IR-shape tests gained a `monomorphise_workspace` call so they -follow the same desugar → lift → mono → lower pipeline as `ail -build`); 6 doc-comment touch-ups; 7-site DESIGN.md sweep; 3 E2E -test-comment polish; bench-run-and-no-ratification (all 3 scripts -exit 0 against existing baselines, drift envelope within -tolerance); plus IR-snapshot refresh (4 `.ll` files: `sum`, -`list`, `max3`, `ws_main`) and 1 canonical-hash pin update -(`ordering_match::main`) as plan-unanticipated downstream -consequences of the corpus migration. Tests 564/564 green -post-iter (was 563 at start; +1 from the new RED test). - -## Per-task notes - -- iter rpe.1.1: RED test `no_per_type_print_ops.rs` — asserts - `io/print_int|bool|float` absent from `env.effect_ops`, - `io/print_str` present. -- iter rpe.1.2: 92 `.ail` corpus migrated (89 single-line + - 3 multi-line shapes); 5 ancillary comment touch-ups in - `borrow_own_demo`, `eq_demo`, `int_to_print_int_borrow`, - `floats_1_newton_sqrt`, `floats_3_safe_division` to satisfy - the §H acceptance criterion `grep -rl ... examples/ == 0`. -- iter rpe.1.3: 6 `.prose.txt` snapshots regenerated via - `cargo run -p ail -- prose`. -- iter rpe.1.4: builtins.rs deletes (3 effect_ops.insert blocks - + 3 `list()` rows + 1 unit test + 2 doc-comment swaps); - codegen lib.rs deletes (3 `lower_app` arms + 1 test); - synth.rs `builtin_effect_op_ret` match-arm narrowed. - `intern_string` removed as orphan helper. `Emitter.strings` - field left in place (still consumed by emit-time loop; - cycles over empty map; non-functional). RED test flips - GREEN. -- iter rpe.1.5: 6 test-body sites swapped to `io/print_str` + - `Str` args; Cat B harness patch at all 6 e2e.rs sites - inserts `ailang_check::monomorphise_workspace(&lifted_ws)` - between the existing lift loop and the lower call (the plan - said "replace the desugar+lift loop with mono", but mono's - precondition is "already lifted" — interpreting per spirit - and matching the `ail build` pipeline). E2E 85/85 green - after this step. -- iter rpe.1.6: doc-comment swaps in 6 sites (`lex.rs`, - `parse.rs`, `main.rs` × 2, `runtime/str.c`, `form_a.md`). -- iter rpe.1.7: DESIGN.md 7 sites swept; the iter's - past-tense paragraph at line 1990-1992 anchors the - milestone close. -- iter rpe.1.8: E2E test-comment polish at `eq_ord_e2e.rs` × 2, - `floats_e2e.rs`. Plus three plan-unanticipated sites: - `int_arg_to_effect_op_does_not_rc_track` test assertions - shifted from `0/0/0` to `1/1/0` (post-iter `print n` for - Int now goes via Show Int → int_to_str → io/print_str, - exactly one heap-Str slab cycle per spec §E); 4 IR - snapshots refreshed via `UPDATE_SNAPSHOTS=1` because the - format-string globals (`@.str__fmt_int_0` / - `%lld\n`) no longer appear in IR; 1 canonical-hash pin - update for `ordering_match::main` (body migration changes - the def's canonical-form hash). -- iter rpe.1.9: bench run — all 3 scripts exit 0 against - existing baselines. No ratification needed; spec §C4 (a) - anticipated possible latency drift but actual fell within - tolerance. Largest single deltas: `latency.explicit_at_rc. - max_us` +154.84% (a tail metric in the noise-class - envelope observed across the last ~12 audits); `throughput. - bench_list_sum.bump_s` +11.85% (REGRESSION, single bench, - 3rd-iter-recurring; tracked as background noise per - audit-24 lineage). Baseline pristine for the 13th - consecutive audit-equivalent. -- iter rpe.1.10: roadmap entry struck `[x]`; journal written; - stats written. - -## Concerns - -- Cat B harness patch deviates from plan-literal: plan said - "replace the desugar+lift loop with monomorphise_workspace"; - I inserted mono after the existing lift loop. Mono's - documented precondition is "already lifted", so the literal - reading would have broken mono's contract. Spirit-reading - was unambiguous: match `ail build`'s pipeline. -- Plan Task 8 under-scoped: it named 3 E2E-comment sites, - but the corpus migration also forced (a) one assertion - update on `int_arg_to_effect_op_does_not_rc_track` (allocs - 0→1, frees 0→1, fully documented in spec §E), (b) IR - snapshot refresh × 4, (c) one canonical-hash pin update. - All three are mechanical downstream consequences of Task 2. -- `Emitter.strings` field in `crates/ailang-codegen/src/lib.rs` - is now functionally dead (no caller after `intern_string` - removal) but the field + the per-module collection loop + - the emit-time globals loop are all still in place. They - cycle over an empty map and are harmless. Cleaning them - up cascades through `emit_workspace_with_alloc` and - parallel `str_literals` infrastructure; out of scope for - this iter. - -## Known debt - -- The `int_arg_to_effect_op_does_not_rc_track` test name now - describes a pre-iter property; the body pins the post-iter - shape (one Show-Int slab cycle). Renaming to e.g. - `print_int_one_slab_cycle` was floated and explicitly - declined (plan Task 8 step 2 said "lean toward keep-name- - update-doc"). If a future iter wants the rename, the test - body is already a clean reference. -- The `intern_string` helper is gone but `Emitter.strings` - field remains — see Concerns above. - -## Files touched - -121 files in the working tree (1 new + 120 modified). - -Code (compiler): -- `crates/ailang-check/src/builtins.rs` (3 effect_ops + 3 list() rows + 1 test + 2 doc-comments deleted/swapped) -- `crates/ailang-check/src/lib.rs` (2 test-body sites swapped) -- `crates/ailang-codegen/src/lib.rs` (3 lower_app arms + 1 test + `intern_string` helper removed) -- `crates/ailang-codegen/src/synth.rs` (builtin_effect_op_ret match-arm narrowed) -- `crates/ailang-surface/src/lex.rs` (1 test-body + 1 module doc swap) -- `crates/ailang-surface/src/parse.rs` (1 diagnostic example swap) -- `crates/ailang-prose/src/lib.rs` (1 round-trip test-body swap) -- `crates/ail/src/main.rs` (2 doc-comments swapped) -- `runtime/str.c` (1 comment re-anchored on %g) - -Tests: -- `crates/ailang-check/tests/no_per_type_print_ops.rs` (NEW) -- `crates/ailang-core/tests/spec_drift.rs` (1 swap) -- `crates/ailang-core/tests/design_schema_drift.rs` (1 swap) -- `crates/ailang-core/tests/hash_pin.rs` (1 hash pin updated) -- `crates/ail/tests/e2e.rs` (6 Cat B harness patches + 1 assertion update + 1 doc-comment) -- `crates/ail/tests/eq_ord_e2e.rs` (2 doc-comments) -- `crates/ail/tests/floats_e2e.rs` (1 doc-comment) -- `crates/ail/tests/snapshots/{sum,list,max3,ws_main}.ll` (regenerated; format-string globals removed) - -Docs: -- `docs/DESIGN.md` (7 sites) -- `docs/roadmap.md` (P2 entry struck [x]) -- `crates/ailang-core/specs/form_a.md` (1 example swap) - -Examples: 92 `.ail` migrations + 6 `.prose.txt` regenerations + 5 ancillary comment touch-ups. - -## Verification - -- `cargo test --workspace`: 564 passed, 0 failed (was 563 at - start; +1 = new RED test, no test counts shifted otherwise). -- `cargo build --workspace`: clean. -- `python3 bench/check.py` exit 0 (drift within tolerance, - no ratification). -- `python3 bench/compile_check.py` exit 0. -- `python3 bench/cross_lang.py` exit 0 (25/25 stable). -- `grep -rl "io/print_int\|io/print_bool\|io/print_float" - examples/` returns 0 results (acceptance criterion met). -- `grep -n "io/print_int\|io/print_bool\|io/print_float" - crates/ailang-check/src/builtins.rs` returns 0 lines. -- `crates/ailang-codegen/src/lib.rs::lower_app` has no arm - for any of the three retired op names. - -## Stats - -`bench/orchestrator-stats/2026-05-14-iter-rpe.1.json` diff --git a/docs/journals/2026-05-14-iter-rpe.1.tidy.md b/docs/journals/2026-05-14-iter-rpe.1.tidy.md deleted file mode 100644 index eb47eb4..0000000 --- a/docs/journals/2026-05-14-iter-rpe.1.tidy.md +++ /dev/null @@ -1,58 +0,0 @@ -# iter rpe.1.tidy — Close audit-rpe.1 [medium] drift - -**Date:** 2026-05-14 -**Started from:** 973f50b (post-rpe.1 + audit-rpe.1) -**Status:** DONE - -## Summary - -Closes the single `[medium]` drift item surfaced by audit-rpe.1: -`crates/ailang-codegen/src/subst.rs:188,217` — `qualify_local_types_codegen` -and `apply_subst_to_type` both carried the exact -`Type::Fn { params, ret, effects, .. }` field-spread that drops -`param_modes` and `ret_mode` to `vec![]` / `ParamMode::Implicit`. This -is the same shape the print-leak bugfix at commit feb9413 corrected -in `crates/ailang-check/src/lib.rs::substitute_rigids`, and the -feb9413 commit message explicitly flagged any other instance as a -latent bug in the same class. - -Fix: preserve both fields through the rebuild — `param_modes.clone()` -(it's a `Vec`) and `*ret_mode` (Copy). Symmetric to the feb9413 fix. - -The audit's `[nit]` finding (dead `Emitter.strings` field in -`crates/ailang-codegen/src/lib.rs`, an orphan after `intern_string` -was removed in rpe.1 Task 4) is left as a follow-up tidy candidate — -it cycles over an empty map harmlessly and isn't an audit gate. - -## Files touched - -- `crates/ailang-codegen/src/subst.rs:188` — - `qualify_local_types_codegen` Type::Fn arm now binds - `param_modes` and `ret_mode` and threads them through. -- `crates/ailang-codegen/src/subst.rs:217` — - `apply_subst_to_type` Type::Fn arm same shape. - -## Verification - -- `cargo test --workspace` → 564 / 0 / 3 (baseline holds). -- `cargo clippy --workspace --all-targets` → 0 warnings. -- `cargo doc --workspace --no-deps` → 0 warnings. - -## Concerns - -- (none) - -## Known debt - -- `Emitter.strings` field in `crates/ailang-codegen/src/lib.rs` - remains dead. Cycles over empty map; no behavioural impact. - Roll into a future opportunistic cleanup. -- Three other field-spread `Type::Fn` rebuild sites - (`crates/ailang-check/src/lib.rs:93` and `:3492`, - `crates/ailang-check/src/lift.rs:875`) ALSO drop `param_modes` - / `ret_mode` to `vec![]` / `Implicit`. They were not flagged - by audit-rpe.1 because no currently-failing test pins them. - Leaving conservatively in place — a flag for a future - fieldtest or audit cycle. The pattern of field-spread-and- - drop is the bug class; whether each site is correctness- - reachable is what needs verification before changing. diff --git a/docs/journals/2026-05-15-bugfix-mut-diag-double-code.md b/docs/journals/2026-05-15-bugfix-mut-diag-double-code.md deleted file mode 100644 index 14d6411..0000000 --- a/docs/journals/2026-05-15-bugfix-mut-diag-double-code.md +++ /dev/null @@ -1,59 +0,0 @@ -# bugfix mut-diag-double-code — doubled `[code]` prefix in mut-local human diagnostics - -**Date:** 2026-05-15 -**Status:** DONE -**Trigger:** fieldtest finding F2 (`docs/specs/2026-05-15-fieldtest-mut-local.md`). -**Started from:** 1faee67 - -## Symptom - -Non-JSON `ail check` stderr rendered the kebab diagnostic code -twice for the four mut-local `CheckError` variants, e.g. - -``` -error: [mut-var-captured-by-lambda] main: [mut-var-captured-by-lambda] mut-var `x` cannot be captured... -``` - -## Cause - -`MutAssignOutOfScope`, `AssignTypeMismatch`, -`UnsupportedMutVarType`, `MutVarCapturedByLambda` were authored in -iters mut.2 / mut.4-tidy with their `#[error("[] ...")]` -`thiserror` Display bodies embedding the bracketed code. The -non-JSON `Cmd::Check` formatter in `crates/ail/src/main.rs` -independently prepends `[{d.code}]` from `CheckError::code()`. -Embedded + prepended = doubled. The non-mut `CheckError` variants -never embedded the bracket, so only these four doubled — they were -the inconsistent ones, brought in line here. - -## Fix - -RED-first via the `debug` skill (`ailang-debugger` wrote the -failing test; the GREEN side was a trivial four-string mechanical -edit applied inline by the Boss per the CLAUDE.md -trivial-mechanical-edit carve-out — no review-and-commit -discipline shed). - -- RED test: `crates/ail/tests/ct1_check_cli.rs` — - `check_human_mode_renders_mut_diagnostic_code_exactly_once`. - Pins the *observable* property: the code appears exactly once in - the rendered human diagnostic, exercising the same formatter path - the CLI uses (not a brittle assertion on the raw `#[error]` body). -- GREEN: dropped the leading `[] ` literal from the four - `#[error(...)]` Display strings in `crates/ailang-check/src/lib.rs`. - -The mut-typecheck pin tests (`mut_typecheck_pin.rs`) and the -in-source unit assertions match on the struct variants / `code()`, -not the Display string, so the prefix removal did not touch them. - -## Tests - -598 → 599 green (the new RED-now-GREEN human-formatter test). -Full workspace clean, zero failures. - -## Not in scope (stays queued) - -F1/F4 (accumulator-over-iteration: the milestone motivation is -unmet without a loop construct) and F3 (zero-arg `(app f)` rejected -at parse) from the same fieldtest are NOT addressed here — they are -design/roadmap items, not this bug. diff --git a/docs/journals/2026-05-15-iter-it.1.md b/docs/journals/2026-05-15-iter-it.1.md deleted file mode 100644 index ca5cc7d..0000000 --- a/docs/journals/2026-05-15-iter-it.1.md +++ /dev/null @@ -1,251 +0,0 @@ -# iter it.1 — loop/recur additive (parse/print/prose/check/codegen) - -**Date:** 2026-05-15 -**Started from:** 7381a4233b0ea34b23871c59e3a3e802441751ef -**Status:** DONE -**Tasks completed:** 8 of 8 - -## Summary - -First of three iterations in the iteration-discipline milestone. -Adds `Term::Loop` / `Term::Recur` / `LoopBinder` as first-class -additive AST nodes end-to-end — parse, print, prose projection, -canonical-JSON serde + round-trip, schema/spec-drift lockstep, -typecheck (binder typing, recur arity/type unification via a -`loop_stack: &mut Vec>` threaded exactly as mut.2 threaded -`mut_scope_stack`, recur tail-position via a new private -`verify_loop_body` helper with `verify_tail_positions`'s public -signature unchanged), and codegen (loop-header block with one phi -per binder, `recur` as a back-edge `br` with a NEW parallel -`block_terminated = true` setter). The change is **strictly -additive**: zero deletions touch the existing `tail-app`/`tail-do` -paths, `verify_tail_positions`' tail-app role, or any of the seven -existing codegen `block_terminated` sites — the 32 within-iter -deletion-lines are exclusively the Task-1 `Internal(...)` stubs and -the Task-1 `verify_tail_positions` structural-passthrough being -replaced by their real Task-5/Task-7 implementations (DD-3 -stub-then-fill). This additive discipline is what makes the -destructive it.3 safe later. Full `cargo test --workspace` green -(63 ok result groups, 0 failed); `loop_counter.ail` builds and -prints `55`; the four `Recur*` diagnostics fire on their negatives; -`tail-app` fixtures (`bench_compute_intsum`, `list_map_poly`) run -byte-identically; zero IR/prose snapshot drift. - -## Per-task notes - -- iter it.1.1: AST variants + every walker arm + serde/schema - lockstep. `Term::Loop { binders: Vec, body: Box }`, - `Term::Recur { #[serde(default)] args: Vec }`, and - `LoopBinder { name, #[serde(rename="type")] ty, init: Box }` - (DD-2 derives = `Debug, Clone, Serialize, Deserialize`, no - PartialEq; note `init` is `Box` per the plan's literal code - block, whereas `MutVar.init` is bare `Term` — a small divergence - from "mirrors MutVar exactly", recorded under Concerns). ~40 - walker arms across desugar (incl. 3 in-test `any_*` helpers not - enumerated by the plan), workspace, all of ailang-check - (lib/lift/linearity/mono/pre_desugar_validation/reuse_shape/ - uniqueness), codegen (escape×3/lambda/lib), prose, surface - parse/print, ail main + the test-side - `codegen_import_map_fallback_pin.rs` walker the plan flagged - (mut.1 missed its analogue; did not miss it here). Typecheck + - codegen *dispatch* stubbed `Internal(...)` (tuple variant — - the plan pseudo-code's `Internal { msg }` struct form does not - exist; copied mut.1's verbatim tuple shape). `synth_with_extras` - got real arms (loop → body's type, recur → unit) per plan. - schema_coverage left RED on "variant never observed" until - Task 3 (plan-expected); every other test green. - -- iter it.1.2: Form-A parse. `parse_loop` / `parse_recur` modelled - on `parse_mut` / `parse_assign`; the binder list is an explicitly - parenthesised group `(loop ((var ...)*) BODY+)` (form_a.md - production added in Task 1.10 lockstep). Plan pseudo-code named - non-existent helpers (`parse_term_str`, `parse_body_seq`, - `parse_loop_binders`); substituted the real harness (module-level - `parse_term`) and inlined the per-`(var …)` decode exactly as - `parse_mut` does — the plan explicitly forbade introducing a new - body-folding helper. EBNF prologue + unknown-head error list - extended. RED→GREEN confirmed. - -- iter it.1.3: Form-A print + positive fixtures. Print arms landed - early in Task 1 as a build-unblock necessity (documented mut.1 - precedent — print arms are exhaustive-match neighbours of the - synth/lower_term dispatchers). `loop_smoke.ail` / - `loop_nested_in_lambda.ail` written in real Form-A matching - `mut_counter.ail`'s verbatim surface spellings (`(con Int)`, - `(app == …)`, `(app + …)`, `(lam (params (typed …)) (ret …) - (body …))`) — the plan's fixture pseudo-code (`(fn count_to : - Int -> Int …)`) is not valid Form-A and the plan's prose - explicitly directed matching mut_counter.ail verbatim. - parse→print→parse byte-idempotent; schema_coverage now GREEN - (Task 1.11 deferred RED closed). - -- iter it.1.4: Prose projection lockstep. Step 4.1's literal - instruction ("render to prose and back, asserting AST equality") - is **not expressible** — AILang has no Form-B→AST parser by - design (CLAUDE.md / `crates/ail/src/main.rs:207` "no parser - exists for the prose surface"). Recorded as a plan defect (see - Concerns). The substantive work (render/free-var/subst arms, - landed in Task 1) is correct and verified by the full prose - suite; added `loop_and_recur_project_to_prose`, a render-assertion - pin using the same `render_term` harness the nearest existing - prose tests (`not_with_tail_flag_keeps_prefix_form`) use — the - feasible equivalent of the plan's intent. This is the documented - mut.1-class plan-pseudo-vs-real-API substitution, not a silent - swap. Status DONE_WITH_CONCERNS. - -- iter it.1.5: Real typecheck (DD-1). Four `CheckError` variants - (`RecurOutsideLoop` / `RecurArityMismatch` / `RecurTypeMismatch` - / `RecurNotInTailPosition`) — **no `span` field** because no - `Span` type is in scope in this crate and no existing CheckError - variant carries one (the plan pseudo-code's `span: Option` - references a non-existent-here type; mirrored the real - `MutAssignOutOfScope` field-shape exactly as the plan's prose - instructed). `loop_stack: &mut Vec>` threaded through - `synth`'s signature + every call site (signature + 22 single-line - recursive + 3 multi-line recursive + 2 production entry points + - lift.rs + 2 mono.rs + builtins.rs test helpers + 8 in-source - `#[cfg(test)]` calls), exactly the mut.2 pattern. Real Loop arm - binds binder names into `self`-… `locals` (save/restore like - `Term::Let` — the correct precedent for *immutable* bindings, - not the mut-var mechanism) and pushes the binder type-vector onto - `loop_stack`. **`Term::Recur` synth returns a fresh metavar - (`Subst::fresh(counter)`), not the plan's flagged-risky - `Type::unit()`** — see Concerns; this resolves the plan's own - self-review "Open risk" correctly (a fresh metavar is the - codebase-idiomatic bottom that unifies anywhere, which is what - makes `(if c then (recur …))` typecheck — `Type::unit()` would - have broken `loop_smoke`'s if-branch unification). `verify_loop_body` - added as a new private helper; `verify_tail_positions`' public - signature unchanged (it.3 reworks it). The diagnostic-doc list in - `diagnostic.rs` was NOT extended — the mut codes were never added - there (false plan premise; followed the actual mut.2 precedent). - RED→GREEN: loop_smoke clean, four negatives each return their - exact code. `loop_stack` threading caused zero regressions. - -- iter it.1.6: Carve-out inventory. EXPECTED extended with the four - `test_recur_*.ail.json` negatives, 13→17, in an it.1 group - (the const is milestone-grouped not alphabetical; the test uses - a BTreeSet so order is functionally irrelevant). GREEN. - -- iter it.1.7: Real codegen (loop-header + phi + recur back-edge + - lambda boundary). `loop_frames: Vec` field added (mut.3 - analogue to `mut_var_allocas` — init/clear/lambda-save-reset- - restore symmetric). Loop lowers to `loop.header.` with one - `phi` per binder; binder phi SSA pushed into `self.locals` so - `Term::Var` resolves to it via the existing lookup. Each phi's - back-edge incoming list is emitted as a unique `/*RECUR_EDGES…*/` - LLVM-comment placeholder, then `replacen(…, 1)`-patched once the - body and all its recur sites are lowered (the plan's - `patch_loop_phis` mechanism; the placeholder is a comment so an - unpatched one would be inert IR, not a syntax error). `Term::Recur` - records `(pred_block, [arg_ssa])`, emits the back-edge `br`, sets - `block_terminated = true` — a NEW parallel setter; the seven - existing tail-driven setters are untouched (verified: zero - deletions touch them). Reused `start_block`/`fresh_ssa`/`fresh_id`/ - `self.current_block` instead of adding the plan-pseudo's sprawl of - single-use helper methods (plan explicitly: "REUSE … grep before - adding"). `loop_counter.ail` builds + prints `55`. - -- iter it.1.8: tail-app coexistence + IR snapshots + acceptance. - `bench_compute_intsum.ail` → `3500003500000`, - `list_map_poly.ail` → `2\n3\n…` (byte-identical to pre-it.1; no - existing codegen path modified). Zero IR/prose snapshot drift (no - blanket regen). All it.1 acceptance bullets verified — the spec's - "five Recur diagnostics" is four in it.1 (the fifth, - `NonStructuralRecursion`, is it.2; resolved per plan Step 8.3). - -- Phase 3 (E2E): added `loop_in_lambda_e2e.ail` + e2e - `loop_in_lambda_runs_and_prints_49` — a `loop` inside a `lam` - body invoked via a returned closure, proving the lambda-boundary - `loop_frames` save/restore is codegen-sound (the no-`main` - `loop_nested_in_lambda.ail` parse/print fixture cannot reach - codegen via the CLI; this runnable fixture closes that gap). - -## Concerns - -- `LoopBinder.init` is `Box` whereas the analogous - `MutVar.init` is bare `Term`. Per the plan's literal Step 1.1 - code block (which specified `init: Box`), not implementer - drift. DD-2 says "mirrors MutVar exactly" — the divergence is in - the plan's own pseudo-code vs. its own DD-2 prose. Functional - impact: none (consistent `(*b.init).clone()` / `Box::new(...)` / - `&mut b.init` handling everywhere). A future iter that wants - strict parity could unbox; no it.1 test depends on the boxing. - -- `Term::Recur` synth result: chose `Subst::fresh(counter)` (a - fresh metavar) over the plan's named fallback `Type::unit()`. - The plan's self-review flagged this exact point as an "Open - risk" with `Type::unit()` as the fallback "because recur is - always in tail position so its value is never consumed". But - `recur` frequently appears as an `if` branch (`loop_smoke`: - `(if (== i n) i (recur …))`), where the branch types MUST - unify — `unify(Int, unit)` fails, so `Type::unit()` would have - broken `loop_smoke`. A fresh metavar unifies with any type - (codegen-idiomatic bottom; the same mechanism `Subst::fresh` - serves everywhere). This is the correct resolution of the plan's - own flagged risk, not a deviation from its intent. - -- Step 4.1 plan defect: it asks for a prose round-trip asserting - AST equality; AILang has no Form-B parser by design. Substituted - a feasible render-assertion pin (mut.1-class plan-pseudo-vs-real - substitution). The plan's *substance* (prose lockstep for the new - variants) is fully met. Boss may want to tighten the planner - template so it does not script prose round-trips. - -- Step 5.2 plan premise defect: "Add the four code strings to the - diagnostic-doc list … same list the mut codes were added to" — - the mut codes were never added to `diagnostic.rs`'s doc list. - Followed the actual mut.2 precedent (not extended); the - authoritative registry is `CheckError::code()`, which has all - four. - -- The plan's recon line numbers (e.g. `desugar.rs:349/559/…`, - `lib.rs:2590/2613/3593/3614`, `parse.rs:1212`) had drifted - vs. the live tree; drove every site off `cargo build`'s - E0004 enumeration as Step 1.2 instructs. Site *count* also - differed (6 desugar E0004 sites vs. plan's 9; resolved via the - build, which is authoritative). No behavioural impact — - same class as mut.1's documented recon misindexing. - -## Known debt - -- Prose-side `(loop …)` / `(recur …)` rendering is a - minimal-correctness shape (`loop { var n = init; …; body }` / - `recur(a, b)`), explicitly mirroring the deliberately-placeholder - mut-block prose shape. The prose surface for loops is not yet - designed; a follow-on prose iter refines it once the LLM-author - signal arrives. Not drift — recorded for visibility. - -- `loop_nested_in_lambda.ail` has no `main`, so it is a - parse/print/typecheck fixture only (CLI codegen needs a `main`). - Codegen-of-loop-in-lambda is instead proven by the runnable - Phase-3 `loop_in_lambda_e2e.ail`. No gap remains; recorded so a - future reader does not mistake the no-main fixture for dead - coverage. - -## Files touched - -AST/core: `crates/ailang-core/src/ast.rs`, -`crates/ailang-core/src/desugar.rs`, -`crates/ailang-core/src/workspace.rs`, -`crates/ailang-core/specs/form_a.md`. -Drift/coverage tests: `crates/ailang-core/tests/{design_schema_drift, -schema_coverage,spec_drift,carve_out_inventory}.rs`. -Check: `crates/ailang-check/src/{lib,lift,linearity,mono, -pre_desugar_validation,reuse_shape,uniqueness,builtins}.rs`, -`crates/ailang-check/tests/loop_recur_pin.rs` (new). -Codegen: `crates/ailang-codegen/src/{lib,escape,lambda}.rs`. -Surface/prose: `crates/ailang-surface/src/{parse,print}.rs`, -`crates/ailang-prose/src/lib.rs`. -CLI: `crates/ail/src/main.rs`, -`crates/ail/tests/{e2e,codegen_import_map_fallback_pin}.rs`. -Spec: `docs/DESIGN.md`. -Fixtures (new): `examples/loop_smoke.ail`, -`examples/loop_nested_in_lambda.ail`, `examples/loop_counter.ail`, -`examples/loop_in_lambda_e2e.ail`, -`examples/test_recur_{outside_loop,arity_mismatch,type_mismatch, -not_in_tail_position}.ail.json`. - -## Stats - -bench/orchestrator-stats/2026-05-15-iter-it.1.json diff --git a/docs/journals/2026-05-15-iter-it.2.md b/docs/journals/2026-05-15-iter-it.2.md deleted file mode 100644 index 8917641..0000000 --- a/docs/journals/2026-05-15-iter-it.2.md +++ /dev/null @@ -1,241 +0,0 @@ -# iter it.2 — structural-guardedness checker + first real Diverge effect - -**Date:** 2026-05-15 -**Started from:** bc9f5120034f2552ad7e78454bb198472404d4b1 -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -Second of three iterations in the iteration-discipline milestone. -Adds the structural-recursion guardedness checker -(`CheckError::NonStructuralRecursion`) and the first real -implementation of the Decision-3 `Diverge` effect, both strictly -additive — nothing `tail`-related is removed (that is it.3). A new -whole-body pass `verify_structural_recursion` runs as a sibling of -`verify_tail_positions` in `check_fn`'s post-synth region (DD-1); it -implements the `smaller`-set algorithm with implicit candidate -inference and unconstrained accumulator positions (DD-2: the -foldl-shape accumulator walk classifies as structural), self- and -mutual-recursion identification with ADT-family connected components -via an inline union-find (DD-3), and the it.2-only `tail==false` -grandfather. A new `term_contains_loop` (stopping at `Term::Lam` -boundaries, DD-4) injects `"Diverge"` into the raised effect set so -the existing `UndeclaredEffect` reconciliation enforces it with no -new diagnostic variant; the lam-arrow and `Term::LetRec` cases are -wired at their respective sub-effect reconcile sites. DESIGN.md -Decision 3 + the §Data-model hook are synced to present tense. All -20 tail-app corpus fixtures stay grandfathered-clean; -`cargo test --workspace` green at 622 passed / 0 failed; the it.1 -loop fixtures gained `!Diverge` (in scope per the plan). - -Two spec/plan boundary defects surfaced and were resolved inside -the task's own stated invariants (corpus stays clean, structural -check not weakened) — see Concerns. Both are recorded so the it.3 -planner and a future spec-tightening pass have the signal. - -## Per-task notes - -- iter it.2.1: `CheckError::NonStructuralRecursion { callee, arg }` - added beside the it.1 `Recur*` variants (bracket-`[code]`-free - Display per F2), `code()` → `"non-structural-recursion"`, `ctx()` - → `{callee, arg}`. New pin file `structural_recursion_pin.rs` - (loop_recur_pin harness verbatim). `diagnostic.rs` untouched - (plan-correctly: `code()` is the registry). RED→GREEN clean. - -- iter it.2.2: the pass + helpers next to `verify_loop_body`: - `adt_type_head` / `adt_param_positions` / `ctor_bound_names` / - `collect_rec_calls_walk` (single `smaller`-threaded walk folding - collection + guardedness, `tail==false` grandfather in the App - arm, stops at `Term::Lam`) / `call_guarded_at` / - `rec_call_arg_display`. `check_fn`/`check_def` gained a - `module_fns: &[&FnDef]` param (plan-anticipated in Step 3.2 for - mutual grouping; threaded from the per-module loop; synthetic - instance-method fns pass `&[]`). Wired - `verify_structural_recursion(f, &env, module_fns)?;` immediately - after `verify_tail_positions`. Two over-rejection classes found - against the corpus and fixed within the task's own acceptance - constraint (see Concerns §1, §2). Status DONE_WITH_CONCERNS. - -- iter it.2.3: ADT-family union-find (`TypeUnionFind`, inline - `BTreeMap`-backed, path-halving, no dependency) + - `mutual_structural_group` returning a `MutualGroup { members, - family_valid }` (a plan-pseudo-vs-real refinement — the plan's - `Vec<&FnDef>` could not express the cross-family-negative; an - empty group silently drops the cross-call instead of rejecting - it). `collect_direct_app_var_callees` for call-graph adjacency - (same lam boundary). `verify_structural_recursion` reworked to a - single `guarded` closure: self-calls vs callee-cand, cross-calls - guarded only if `family_valid`. tree/forest clean, cross-family - rejected. RED→GREEN. - -- iter it.2.4: `term_contains_loop` (exhaustive Term match; - `Loop=>true`, `Lam=>false`, all else recurse incl. Recur args). - Injection in `check_fn` before the declared-vs-raised reconcile. - Lam-arrow coherence wired at BOTH the `Term::Lam` synth - sub-effect site AND the `Term::LetRec` synth sub-effect site - (the plan named only the Lam site; recon line drifted and there - are now two structurally-identical reconcile sites — a - `Term::LetRec` clause is a fn-equivalent, same as the it.1 - tail-position treatment, so a loop in a deferred LetRec body must - also carry Diverge; injecting at both is the sound completion, - not an extra). Four it.1 loop fixtures + two HOF signatures - updated to declare `!Diverge` (see Concerns §3). Status - DONE_WITH_CONCERNS. - -- iter it.2.5: DESIGN.md Decision 3 rewritten (`Diverge` no longer - nominal; carried by loop-bearing / Diverge-calling fns; structural - recursion pure+total; `IO` description kept + sharpened) + - §Data-model hook sentence → present tense, it.3 retirement - retained. Drift anchors (`"t":"loop"`/`"t":"recur"`) untouched — - `design_schema_drift`/`schema_coverage`/`spec_drift` green. - -- iter it.2.6: acceptance gate. Full workspace green; all 7 named - spec-acceptance pins green; `mut_counter.ail` still runs `55` - (grandfathered, unmigrated); all 20 tail-app corpus fixtures - grandfathered-clean (zero `non-structural-recursion`). - -- Phase 3 (E2E): `examples/struct_rec_sum_e2e.ail` + - `struct_rec_sum_runs_and_prints_15` — an it.2-clean structural - recursion (plain non-tail, no `!Diverge`, no `tail-app`) that - runs to 15, proving the "total" verdict is behaviourally sound, - not just a typecheck assertion. Plus - `loop_needs_diverge_runs_and_prints_55` — the Diverge-path twin, - proving the Diverge injection is a typecheck-only obligation with - zero codegen impact (the loop lowers/runs identically to it.1). - -## Concerns - -- **§1 — Spec/it.2-boundary defect: no-ADT-candidate recursion.** - The plan's load-bearing invariant ("the 21 tail-app corpus - fixtures stay clean through it.2 because `collect_rec_calls` only - collects `tail==false` calls") is insufficient: corpus fixtures - like `bench_latency_explicit.ail`'s `build_tree(depth: Int)` - recurse **non-tail** (inside a `term-ctor`) on a **primitive** - arg — the `tail==false` grandfather does not exempt them, and - they have no ADT candidate position. Strict DD-2 would reject - ~18 corpus fixtures in it.2, contradicting the additive mandate. - Resolution (does NOT weaken the ADT check): a def with no ADT - candidate position AND no mutual cycle has no *structural* - position to verify — it is integer-counter / other-shaped - recursion whose migration to `(loop …)` is the destructive it.3 - corpus pass, not it.2's job. it.2 only rejects *misuse of an ADT - structural position*; the spec's load-bearing negative - (`f(xs: IntList) = … f(xs) …`, HAS an ADT candidate) still fires. - This is the honest reading of DD-2's "for each candidate - structural position" (vacuous when there are none); the plan's - pseudo-code only handled `calls.is_empty()`, not - `cand.is_empty()`. Recorded as a resolved it.2/it.3 boundary - clarification — the it.3 planner should expect these to be the - corpus-migration set, and a spec-tightening pass may want to - state the no-candidate boundary explicitly. - -- **§2 — Two RC-regression fixtures needed the spec's transitional - grandfather applied.** `rc_pin_recurse_implicit.ail` and - `rc_let_alias_implicit_param.ail` (NOT in the 20-fixture tail-app - set) contain `loop(n: Int, t: Tree) = … (app loop (- n 1) t)` — - an ADT param (`t: Tree`) held constant while decrementing an Int, - recursing **non-tail**. This is structurally identical to the - required negative (`f(xs)=…f(xs)…`): both pass the ADT param - unchanged at its candidate position. It therefore correctly fires - `NonStructuralRecursion` and CANNOT be exempted by broadening the - grandfather without also letting the required negative through - (which would weaken the check — explicitly forbidden by the - carrier). The recursive call is in tail position, so the - spec-prescribed transitional treatment applies: mark it - `tail-app` (the it.2 "Transitional grandfather" — identical to - the idiom 20 other corpus fixtures use; it.3 migrates to - `(loop …)`). The 18d.4 / 18g regressions these fixtures guard - live in `pin`/`pin_aliased`'s match arm-close, unaffected by the - `loop` recursion's lowering — so this is NOT adapting-a-test-to- - dodge-a-bug (the check is correct; the fixture joins the - transitional grandfather, with an in-fixture comment recording - why). Both fixtures' RC==GC e2e regression guards still pass. - -- **§3 — it.1 loop fixtures gained `!Diverge` (plan-in-scope, full - enumeration vs plan's two examples).** Plan Step 4.4 named - `loop_counter.ail` + `loop_in_lambda_e2e.ail` explicitly; the - same in-scope action applies to all it.1 loop fixtures. - `loop_counter.ail` (loop in `main` body) → `(effects IO Diverge)`. - `loop_smoke.ail` `count_to` (loop in fn body) → `(effects - Diverge)`. `loop_nested_in_lambda.ail` / `loop_in_lambda_e2e.ail` - (loop in a *lam* body) → `(effects Diverge)` on the **lam**, with - the enclosing fn / HOF param-and-return fn-types threaded to - carry `!Diverge` (effects are part of the function type — forced, - not optional). The lam-boundary rule held: `main` in - `loop_in_lambda_e2e` does NOT carry Diverge from its own body - (the loop is behind the lam edge) but DOES via callee-effect - propagation through `apply` (free per DD-4, identical to `!IO`). - The injection was not weakened. Both it.1 loop e2e tests - (55 / 49) still pass; round-trip green on every modified `.ail`. - -- **§4 — `MutualGroup` is a plan-pseudo-vs-real-API refinement.** - The plan's `mutual_structural_group -> Vec<&FnDef>` could not - express "cross-family mutual recursion must be rejected" — an - empty return drops the cross-call from collection instead of - flagging it. Split into `{ members: HashSet, - family_valid: bool }`: membership drives collection (cross-calls - always collected), validity drives the verdict (a cross-call into - a family-invalid cycle is unconditionally unguarded). Minimal - shape satisfying both plan requirements; same documented - substitution class as it.1's repeated plan-pseudo-vs-real notes. - -## Known debt - -- The structural check's `rec_call_arg_display` has no real - term-pretty-printer (only `type_to_string` exists in - `ailang_core::pretty`); non-`Var` offending args render as terse - tags (`Ctor(…)`, `fn(…)`, ``). The canonical non-structural - case is always a bare `Var` (the param passed unchanged), so this - is cosmetic on the diagnostic only; a future term-pretty-printer - would improve the `arg` field. Not drift — recorded for - visibility. -- Three exhaustive Term-walks now exist in the it.2 region - (`collect_rec_calls_walk`, `collect_direct_app_var_callees`, - `term_contains_loop`). Each collects a genuinely different thing - with different threading; a shared generic walker would be - speculative abstraction per the AILang quality bar. Recorded so a - future reader does not mistake the triplication for an oversight. -- `mutual_structural_group`'s reaches/component BFS is O(n²)-ish - over a module's fns, run once per fn-check. Corpus modules are - <40 defs; not a hot path. No action; recorded for visibility. - -## Blocked detail - -None — all 6 tasks completed; the two spec/plan boundary defects -(Concerns §1, §2) were resolved inside the task's own stated -acceptance constraints (corpus stays clean, structural check not -weakened), not escalated, because the carrier explicitly anticipated -corpus-breakage handling and the resolutions are the spec's own -transitional model rather than judgement substitutions. - -## Files touched - -Check core: `crates/ailang-check/src/lib.rs` (CheckError variant + -code/ctx; `module_fns` thread through check_def/check_fn; -`verify_structural_recursion` + helpers + `TypeUnionFind` + -`MutualGroup` + `term_contains_loop`; Diverge injection at -check_fn / Term::Lam / Term::LetRec sites). -Tests (new): `crates/ailang-check/tests/structural_recursion_pin.rs` -(9 pins). Carve-out: `crates/ailang-core/tests/carve_out_inventory.rs` -(EXPECTED 17→20 + header comment rewritten accurate). -E2E: `crates/ail/tests/e2e.rs` (2 new Phase-3 tests). -Spec: `docs/DESIGN.md` (Decision 3 + §Data-model hook). -Fixtures (new): `examples/struct_rec_list_len.ail`, -`examples/struct_rec_foldl_sum.ail`, -`examples/struct_rec_tree_forest.ail`, -`examples/struct_rec_sum_e2e.ail`, -`examples/loop_needs_diverge.ail`, -`examples/test_non_structural_recursion.ail.json`, -`examples/test_mutual_cross_family.ail.json`, -`examples/test_loop_missing_diverge.ail.json`. -Fixtures (modified): `examples/loop_counter.ail`, -`examples/loop_smoke.ail`, `examples/loop_nested_in_lambda.ail`, -`examples/loop_in_lambda_e2e.ail` (all four: it.1 loop fixtures → -`!Diverge`); `examples/rc_pin_recurse_implicit.ail`, -`examples/rc_let_alias_implicit_param.ail` (transitional `tail-app` -grandfather on the `loop` fn — Concerns §2). - -## Stats - -bench/orchestrator-stats/2026-05-15-iter-it.2.json diff --git a/docs/journals/2026-05-15-iter-it.3.md b/docs/journals/2026-05-15-iter-it.3.md deleted file mode 100644 index 98bfba7..0000000 --- a/docs/journals/2026-05-15-iter-it.3.md +++ /dev/null @@ -1,187 +0,0 @@ -# iter it.3 — retire tail-app/tail-do + corpus migration - -**Date:** 2026-05-15 -**Started from:** c992eb93df4c5ec0a0686c924535cb9951be552f -**Status:** BLOCKED -**Tasks completed:** 1 of 7 (Task 1 complete; BLOCKED at the -Task-1.3 → Task-2 boundary before any destructive task) - -## Summary - -Third and terminal iteration of the iteration-discipline milestone -(destructive: retire `tail-app`/`tail-do`, migrate the corpus). -Task 1 (the non-destructive pre-migration oracle + the spec-delegated -class-(b) live enumeration) ran to completion: 40 recursive corpus -fixtures classified, 40 behavioural oracles captured under -`bench/it3-oracle/`, `MANIFEST.tsv` written. Task 1.3's mandated -live sweep surfaced a **real spec-signal** that the plan's own -self-review (lines 550-560) and the carrier explicitly defined as a -BLOCKED-to-Boss condition, not a silent reshape: **six fixtures -contain a recursive fn that resists BOTH a natural `(loop)`/`recur` -form AND a structural-plain form.** Per DD-2/DD-3 (migrate before -remove; Task 4's completeness gate must not be softened and a RED -there is a Task-2 defect, never a restored exemption) there is **no -in-plan migration target** for these six. The destructive tasks -(4/5) were therefore not started — correctly, since the migration -they depend on is impossible for this subset and DD-3 forbids -working around it. No production code was modified; the working -tree contains only the new `bench/it3-oracle/` artefacts. - -## Per-task notes - -- iter it.3.1: Pre-migration oracle + migration-set enumeration — - **complete.** `cargo build -p ail` clean at HEAD c992eb9. - Class-(a): 23 `(tail-app|tail-do)` fixtures (recon-confirmed). - Class-(b) live sweep (Task 1.3, spec-delegated): read - `verify_structural_recursion` + the two it.2 exemptions - (`lib.rs:3027` `if !*tail` grandfather; `lib.rs:3197` - `if cand.is_empty() && group.members.is_empty()` no-ADT skip), - then classified every self-recursive corpus fn against the - post-Task-4 verdict logic (`lib.rs:3219-3267`). Authoritative - result: 13 no-ADT-candidate single-recursion fns are - loop-migratable; 11 fixtures recurse structurally on an ADT param - (clean before AND after Task 4 — no migration, only the redundant - `tail-app` marker if class-(a) is dropped); **6 fixtures have a - no-ADT-candidate DOUBLE non-tail recursion (binary-tree build) - that is neither structural nor loop-expressible.** Oracles - captured for all 40 (`bench/it3-oracle/.out` + `MANIFEST.tsv`). - Task 1.5 fallout list recorded below for the eventual unblocked - re-dispatch. No code changed (Task 1 is non-destructive by spec). - -- iter it.3.2 … it.3.7: not started. Task 2 (migration) has no - valid action for the 6 blocked fixtures; Tasks 4/5 are - destructive and gated behind a complete Task-2 migration that - cannot be completed. Halted at the spec-signal per carrier. - -## Concerns - -- **The RC-RSS / 18g.1 load-bearing risk (DD-4 / Task 5.5 / 5b) was - NOT reached.** The iter blocked before Task 5, so the 18g.1 - husk-dec deletion and the RC-RSS bench gate were never exercised. - No RC-RSS evidence exists from this dispatch. The milestone-close - audit must NOT treat the RC-RSS contingency as resolved — it is - untested. When the spec defect below is resolved and it.3 is - re-dispatched, Task 5.5 (and conditionally 5b) is still the - milestone's load-bearing risk and must be run + its evidence - journalled then. - -## Known debt - -- `bench/it3-oracle/` currently holds 40 `.out` snapshots + - `MANIFEST.tsv` (41 files). They are the Task-1 deliverable and - stay valid for a re-dispatch from the same `start_sha`. Boss - call at commit whether to keep them under `bench/` (plan §"Files - this plan creates": "deleted in Task 6 … or kept under `bench/` - if cheap") — but do NOT delete before the re-dispatch; the - re-dispatched Task 7.3 diffs against them. -- The 11 class-(a) `plaincall` fixtures (structural on an ADT param, - carrying only a redundant `tail-app` marker) and the 11 - `already-structural` fixtures need no semantic migration — only - marker-stripping (class-a) — and are unaffected by the blocker. - Recorded so the re-dispatch does not re-derive this. - -## Blocked detail - -**Task:** 1 (the Task-1.3 spec-delegated sweep) → halts entry to -Task 2 / Task 4. - -**Reason:** `spec-ambiguous` / spec defect surfaced by the mandated -live sweep — six corpus fixtures contain a recursive fn that is -**neither structural recursion (no ADT-typed parameter — `cand` -empty in `verify_structural_recursion`) nor expressible as -`(loop …)`/`recur`** (the recursion is two non-tail self-calls -building both children of a binary `Tree`; `recur` is -tail-position-only with a single back-edge, so a balanced-tree -build cannot be a tail loop). After Task 4 removes the -no-ADT-candidate skip (`lib.rs:3197`, mandated by DD-3 and the -spec's it.3 acceptance bullet "no grandfather/no-candidate skip -remains"), each of these deterministically fires -`CheckError::NonStructuralRecursion` (verdict logic `lib.rs:3219-3267`: -`cand` empty ⇒ `self_clear=false` ⇒ NSR). DD-3 forbids softening -Task 4 or restoring the exemption; the carrier and project memory -(`feedback_dont_adapt_tests_to_bugs`) forbid reshaping the -LLM-natural fixture (e.g. changing `depth: Int` to an ADT-encoded -`Nat`, or hand-rolling an explicit stack/continuation). There is -**no in-plan migration target**, so the destructive split cannot -proceed. - -**The six fixtures (fn :: shape):** - -1. `examples/bench_tree_walk.ail` :: `build_tree` - (`(if (== depth 0) (Tree Leaf) (Tree Node 1 (build_tree (- depth 1)) (build_tree (- depth 1))))`) -2. `examples/bench_latency_explicit.ail` :: `build_tree` (same shape; file is class-(a) for other fns but `build_tree` itself is non-tail double recursion) -3. `examples/bench_latency_implicit.ail` :: `build_tree` (same) -4. `examples/rc_let_owned_app_leak.ail` :: `build` (same Tree shape on `d: Int`) -5. `examples/rc_pin_recurse_implicit.ail` :: `build` (same; the RC-regression-guard fixture from it.2 Concerns §2) -6. `examples/rc_let_alias_implicit_param.ail` :: `build` (same; the RC-regression-guard fixture from it.2 Concerns §2) - -This is exactly the spec-signal the it.2 journal §1 foreshadowed -(it named `bench_latency_explicit:build_tree` as a no-ADT-candidate -case "whose migration to `(loop …)` is the destructive it.3 corpus -pass") and the plan self-review's "Recorded risk" anticipated -("a genuine non-structural recursion with no natural loop form … -surfaces as a Task-2 BLOCKED to the Boss with the fixture named"). -The spec/plan assumed every no-ADT-candidate recursion is a -loopable counter/accumulator; balanced-binary-tree construction by -double recursion on a depth counter is the counter-example. - -**Suggested next step:** Boss-level spec decision required (this is -not an implementer or planner judgement call — see -`feedback_spec_over_plan_patches`: a spec defect goes back through -`brainstorm`, not a fourth plan patch). Candidate resolutions for -the Boss to weigh, language-first: -(a) **Add a structural accommodation for tree recursion** — extend -`verify_structural_recursion` so a non-tail recursive call on a -*structurally-derived smaller value* is guarded even when the -decreasing quantity is a primitive depth counter feeding an ADT -build (i.e. recognise the `build_tree` shape as bounded). This -keeps the six fixtures verbatim and is the only option that does -not reshape LLM-natural code, but it widens it.2's structural -checker — a Decision-10/Decision-3 spec change, brainstorm-gated. -(b) **Carve the six into a permanent allow-list / carve-out** (like -the existing `.ail.json` carve-outs) with an explicit -"constructor-blocked recursion, depth-bounded by construction" -annotation, leaving the no-ADT-candidate skip retired for -everything else. Requires a spec sentence defining the carve-out -predicate. -(c) **Retire the bench fixtures that exist only to exercise the -removed `musttail`/RC-RSS path** (`bench_latency_*`, -`rc_pin_recurse_implicit`, `rc_let_alias_implicit_param`, -`rc_let_owned_app_leak`) and replace `bench_tree_walk`'s -`build_tree` coverage with a structural equivalent — but this is -the "adapt the fixture to dodge" path the carrier and -`feedback_dont_adapt_tests_to_bugs` explicitly forbid for -language-limitation cases; named only for completeness, not -recommended. -Once the Boss resolves the spec, re-dispatch it.3 from the same -`start_sha` (the `bench/it3-oracle/` artefacts remain valid). - -## Files touched - -Created (untracked, non-code): -- `bench/it3-oracle/MANIFEST.tsv` (40-fixture classification) -- `bench/it3-oracle/.out` × 40 (behavioural oracle: - `ail run` stdout for `main` fixtures, `ail check` output for - check-only) - -No production code, no spec, no test files modified (Task 1 is -non-destructive by plan; the BLOCKED halted before any code-edit -task). - -## Task 1.5 fallout map (recorded for the unblocked re-dispatch) - -Tail-app-specific tests the plan's Task 6.4 will delete: -`crates/ailang-prose` `app_tail_renders_with_keyword`, -`binop_with_tail_flag_keeps_prefix_form`, -`not_with_tail_flag_keeps_prefix_form`; -`crates/ailang-core/tests/spec_drift.rs::spec_mentions_tail_variants`; -`crates/ailang-check` `TailCallNotInTailPosition` pins -(`tail_call_in_tail_position_is_accepted` / -`..._non_tail_position_is_rejected` if present); any -`iter14e_*musttail*` e2e. (Enumeration deferred to the -re-dispatch's Task 1.5 against the then-current tree, since the -exact test names will drift; recorded here as the recon seed.) - -## Stats - -bench/orchestrator-stats/2026-05-15-iter-it.3.json diff --git a/docs/journals/2026-05-15-iter-mut.1.md b/docs/journals/2026-05-15-iter-mut.1.md deleted file mode 100644 index 1166ee9..0000000 --- a/docs/journals/2026-05-15-iter-mut.1.md +++ /dev/null @@ -1,215 +0,0 @@ -# iter mut.1 — schema + surface for local mutable state - -**Date:** 2026-05-15 -**Started from:** 60e4559e31d7f09de77e2679aab2d524de257a61 -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -Lands the foundational AST + surface forms for the mut-local -milestone (first milestone on the Stateful-islands roadmap path). -Two new `Term` variants (`Mut` and `Assign`) plus the nested -`MutVar` struct enter `ailang-core::ast`; canonical-JSON serde -round-trips them by virtue of the existing `#[serde(tag = "t")]` -machinery. Form A gains `(mut …)` / `(var …)` / `(assign …)` -productions in the parser and the corresponding write arms in the -printer; the parser right-folds the trailing body sequence into -`Term::Seq` so the canonical JSON-AST always sees a single -`body: Term`. Every `Term` exhaustive match in the workspace (~25 -walker sites) gains substantive arms, except the two dispatch -entry points (`synth` in `ailang-check`, `lower_term` in -`ailang-codegen`) which stub with `CheckError::Internal` / -`CodegenError::Internal` per the spec's "out of iteration" -boundary — typecheck recognition lands in mut.2, codegen lowering -in mut.3. The three drift tests (`design_schema_drift`, -`spec_drift`, `schema_coverage`) all flip green after the -exemplar / variant-tag / visitor extensions, and `examples/mut.ail` -ships as the round-trip fixture covering empty / single-var / -two-var / nested-shadow / Bool / Unit cases. Full -`cargo test --workspace` 579/579 green (was 564 + 2 new AST pins + -4 new parser pins + 9 from the extended drift exemplars and the -mut.ail-corpus contributions netting to +15). - -## Per-task notes - -- iter mut.1.1 — AST extension: added `Term::Mut { vars: Vec, - body: Box }`, `Term::Assign { name: String, value: Box }`, - and the `MutVar { name, ty, init }` struct to `crates/ailang-core/src/ast.rs`. - Two new unit tests pin the canonical-bytes of the empty-vars mut - serialisation and the Assign round-trip. `MutVar` carries Arm's - derives (`Debug, Clone, Serialize, Deserialize`) — the plan's - literal pseudo-code asked for `PartialEq + Eq` but its own - justification ("derives match Arm's for cross-tree consistency") - points at Arm, which does not derive those. Concern recorded; - no functional impact (the Task 1 tests do not depend on - `PartialEq`, and `Term` itself uses a custom `PartialEq` impl on - `Type` only). - -- iter mut.1.2 — substantive walker arms: 9 sites in `desugar.rs`, - 2 in `workspace.rs`, 3 in `ailang-check/src/lib.rs` (1 in - `substitute_rigids_in_term`, 1 in `verify_tail_positions`, 1 - variant-name string emitter inside `synth`'s `Term::ReuseAs` sub- - match), 2 in `lift.rs`, 3 in `linearity.rs` (one being a - variant-name string emitter), 2 in `mono.rs`, 1 each in - `pre_desugar_validation.rs`, `reuse_shape.rs`, `uniqueness.rs`, - 3 in `escape.rs`, 1 in `lambda.rs`, 1 in `codegen/src/lib.rs` - (the `synth_with_extras` type-synthesis helper), 3 in - `ailang-prose/src/lib.rs`, 2 in `ail/src/main.rs`. Plus one - unenumerated site in `crates/ail/tests/codegen_import_map_fallback_pin.rs` - (an exhaustive walker inside a test) that the build required. - Each arm follows the appropriate shape per existing convention - at its site: rebuilder shapes for `substitute_*` / `subst_*` / - `desugar_term` / `lift_in_term` / `substitute_rigids_in_term` / - `rewrite_term`, visitor shapes for `collect_used_in_term` / - `free_vars_in_term` / `find_non_callee_use` / `walk_term*` / - `verify_tail_positions` / `term_has_letrec` / the linearity - walkers / `walk` in `reuse_shape.rs` / `walk` in `uniqueness.rs` - / `walk` in `escape.rs` / `collect_captures` / `count_free_var` / - `subst_var_with_term` (rebuilder-shape in prose). Shadowing - semantics across mut-var bindings replicates the - `Term::Let`/`Term::Lam`/`Term::LetRec`-shaped convention at each - site: var-init terms see the outer scope plus already-declared - vars; later var-inits and the body see all earlier var bindings. - -- iter mut.1.3 — dispatch stubs: typecheck dispatch at `synth` in - `ailang-check/src/lib.rs` (Plan line 2572 was wrong — that line - is `verify_tail_positions`, which is substantive walker - territory; the actual dispatch entry is `synth` at line 3403's - former `Term::ReuseAs` arm, where the two new arms now sit) - returns `CheckError::Internal("Term::Mut/Assign not yet - supported in typecheck (deferred to iter mut.2)")`. Codegen - dispatch at `lower_term` in `ailang-codegen/src/lib.rs:1649` - returns `CodegenError::Internal("Term::Mut/Assign not yet - supported in codegen (deferred to iter mut.3)")`. Both errors - use the single-field tuple-variant shape (`Internal(String)`). - -- iter mut.1.4 — Form A parser + printer: dispatcher arms added - for `"mut" => parse_mut` and `"assign" => parse_assign` in - `parse_term`'s head-keyword match; the two helpers parse - positionally (mut: pull leading `(var ...)` entries, then ≥ 1 - body terms right-folded into `Term::Seq`; assign: positional - ident-then-value pair). Plan pseudo-code used helper names - (`peek_is_open_paren_with_head`, `expect_head`, etc.) that don't - exist; substituted the actual API (`expect_lparen`, - `expect_keyword`, `expect_ident`, `parse_type`, `parse_term`, - `expect_rparen`, `peek_head_ident`, `matches!(self.peek(), - Tok::RParen)`). Four RED-first parser tests added covering empty - mut + 1-var-1-assign-1-final + missing-body rejection (both - shapes). Print arms in `print.rs` walk the right-spine of - `Term::Seq` and emit each lhs as a top-level statement; the - terminal expression closes the `(mut …)` form. EBNF prologue in - `parse.rs` extended with `mut-term`, `var-decl`, `assign-term` - productions. The print arms had to land during the Task 1+2 - build-unblock phase rather than waiting for Task 4 (they are - exhaustive-match neighbours of the synth/lower_term dispatchers); - this is a benign sequencing shift and the plan's "Task 2 build- - green verification gate" still holds. - -- iter mut.1.5 — drift + coverage extensions: two new exemplars - added to `design_schema_drift.rs` and two new match arms; two - new `VariantTag` entries (`TermMut`, `TermAssign`), - `EXPECTED_VARIANTS` extended, two new `visit_term` arms in - `schema_coverage.rs`; two new exemplars + two new match arms in - `spec_drift.rs`. `form_a.md` gains three new lines in the - Parenthesised-forms block plus a prose paragraph naming the - empty-vars and post-vars-body invariants and the diagnostic - codes. `DESIGN.md` §"Term (expression)" gains the two jsonc - schema blocks immediately after the `reuse-as` block plus a - closing prose paragraph naming the mut.1 dispatch-stub semantics - and the deferred mut.2 / mut.3 iterations. Post-edit: - `design_schema_drift` and `spec_drift` both green; - `schema_coverage` is RED on the missing `examples/mut.ail` - fixture, per plan Step 6 expectation. - -- iter mut.1.6 — fixture + round-trip: `examples/mut.ail` shipped - with the six fns from plan Step 1 verbatim (empty / single-var - with assign / two-var with combine / nested-shadow / Bool / - Unit). The corpus-globbing round-trip test - (`parse_then_print_then_parse_is_idempotent_on_every_ail_fixture`) - and the parse-determinism test pick the fixture up automatically; - both green. `schema_coverage` also green now that both new - variants are observed in the corpus. Full `cargo test - --workspace` 579/579 green. - -## Concerns - -- `MutVar` derives diverge from the plan's literal pseudo-code - (`Debug, Clone, PartialEq, Eq, Serialize, Deserialize`); the - shipped form is `Debug, Clone, Serialize, Deserialize` to match - Arm's pattern. The plan's own justification ("match Arm's - derives for cross-tree consistency") points at Arm which does - not derive PartialEq/Eq. Functional impact: none in mut.1 - (the Task 1 tests do not depend on PartialEq, and `Term` uses - a custom PartialEq impl scoped to `Type` only). A future iter - that needs to compare `MutVar` (e.g. for in-place rewriting in - mut.3 codegen) can add the derives at that time without breaking - any mut.1 test. - -- Plan recon for the typecheck dispatch entry was misindexed. - The plan listed `crates/ailang-check/src/lib.rs:2572` as the - "check_term dispatch — STUBBED" site; that line is actually in - `verify_tail_positions`, a substantive walker. The real - typecheck dispatch entry is `synth` (defined at lib.rs:2613, - with its `Term::ReuseAs` arm at line 3403 before the new mut - arms). The stub was placed at the actual dispatch entry; line - 2572 received a substantive arm. The variant-name string - emitter sub-match referenced as line 3430 is inside `synth`'s - former `Term::ReuseAs` arm and was extended as Task 2 Step 13 - asks. Net: the spec intent (stub at typecheck dispatch) is - preserved; only the line-number routing differs. - -- One test-side walker site (`crates/ail/tests/codegen_import_map_fallback_pin.rs:62`) - carries an exhaustive match on `Term` and was not enumerated - by the plan's Task 2 file list. The site is the body-walker of - a regression-pin test from iter 24.tidy; it required two new - arms to build green. Added with appropriate visitor-shape - recursion. The plan-recon miss is recorded here as a - documentary item; no behavioural impact. - -## Known debt - -- Prose-side `(mut ...)` rendering in `ailang-prose/src/lib.rs` is - a minimal-correctness shape (`mut { var = ; ...; }`). - The prose surface for mut blocks is not yet fully designed; a - future prose-iter once the LLM-author signal arrives will refine - it. This is a deliberate placeholder, not drift — recording for - visibility. - -- `subst_var` in `desugar.rs` does NOT rename the `Term::Assign.name` - field. The desugar pass's `subst_var` rewrites let-rec captures - (KnownType / LetBound / MatchArm), none of which can be mut-vars - (mut-vars are first-order scalars, not captureable). Leaving - the assign-name untouched is semantically correct; documented in - the arm's doc-comment. Mentioned here for visibility. - -## Files touched - -- `crates/ailang-core/src/ast.rs` — `Term::Mut`, `Term::Assign`, `MutVar`, two new unit tests -- `crates/ailang-core/src/desugar.rs` — 9 walker arms -- `crates/ailang-core/src/workspace.rs` — 2 walker arms -- `crates/ailang-check/src/lib.rs` — 3 walker arms (substantive + variant-name) + 2 dispatch stubs in `synth` -- `crates/ailang-check/src/lift.rs` — 2 walker arms -- `crates/ailang-check/src/linearity.rs` — 3 walker arms (2 substantive + 1 variant-name) -- `crates/ailang-check/src/mono.rs` — 2 walker arms (1 mut-ref rebuilder, 1 visitor) -- `crates/ailang-check/src/pre_desugar_validation.rs` — 1 walker arm -- `crates/ailang-check/src/reuse_shape.rs` — 1 walker arm -- `crates/ailang-check/src/uniqueness.rs` — 1 walker arm -- `crates/ailang-codegen/src/escape.rs` — 3 walker arms -- `crates/ailang-codegen/src/lambda.rs` — 1 walker arm -- `crates/ailang-codegen/src/lib.rs` — 1 walker arm (`synth_with_extras`) + 2 dispatch stubs in `lower_term` -- `crates/ailang-prose/src/lib.rs` — 3 walker arms (1 substantive, 1 `count_free_var`, 1 `subst_var_with_term` rebuilder) -- `crates/ail/src/main.rs` — 2 walker arms (dep-walker + `rewrite_term`) -- `crates/ail/tests/codegen_import_map_fallback_pin.rs` — 1 walker arm (test-side, unenumerated by plan) -- `crates/ailang-surface/src/parse.rs` — dispatcher arms, `parse_mut` / `parse_assign` helpers, EBNF prologue extension, 4 new parser tests -- `crates/ailang-surface/src/print.rs` — 2 `write_term` arms -- `crates/ailang-core/tests/design_schema_drift.rs` — 2 exemplars + 2 match arms -- `crates/ailang-core/tests/schema_coverage.rs` — 2 VariantTag + EXPECTED_VARIANTS + 2 visit_term arms -- `crates/ailang-core/tests/spec_drift.rs` — 2 exemplars + 2 match arms -- `crates/ailang-core/specs/form_a.md` — 3 new productions + prose paragraph -- `docs/DESIGN.md` — 2 jsonc schema blocks + prose paragraph -- `examples/mut.ail` — round-trip fixture (6 fns) - -## Stats - -bench/orchestrator-stats/2026-05-15-iter-mut.1.json diff --git a/docs/journals/2026-05-15-iter-mut.2.md b/docs/journals/2026-05-15-iter-mut.2.md deleted file mode 100644 index ccb79bc..0000000 --- a/docs/journals/2026-05-15-iter-mut.2.md +++ /dev/null @@ -1,256 +0,0 @@ -# iter mut.2 — typecheck recognition of `Term::Mut` and `Term::Assign` - -**Date:** 2026-05-15 -**Started from:** 3a5b1d33472e5c35c4917702e30fa5a12b749e31 -**Status:** DONE -**Tasks completed:** 7 of 7 - -## Summary - -Replaces the mut.1 dispatch stubs in `synth` (`ailang-check`) for -`Term::Mut` and `Term::Assign` with real typecheck logic. Three new -`CheckError` variants ship (`MutAssignOutOfScope`, -`AssignTypeMismatch`, `UnsupportedMutVarType`) with kebab-coded -`code()` arms and structured `ctx()` payloads, following the -cli-diag-human bracketed-`[code]` convention. A new -`mut_scope_stack: &mut Vec>` parameter -threads through every `synth` call site (24 sites across lib.rs + -mono.rs:712 / mono.rs:1337 + lift.rs:723 + builtins.rs in two test -helpers); each entry-point caller (`check_fn`, `check_const`, mono -re-synth, lift letrec-capture synth) opens a fresh empty stack at -top-of-body. `Term::Var` resolution prepends the mut-scope check -ahead of the existing locals/globals chain (innermost-first wins on -shadowing). `Term::Mut` pushes a frame after gating each -`MutVar.ty` against `is_supported_mut_var_type` (Int / Float / -Bool / Unit), synths inits in the in-progress scope, unifies init -type with declared, and synths the body in the extended scope. -`Term::Assign` resolves the target name through the stack, returns -`MutAssignOutOfScope` (with a flat-deduplicated `available` list) -on miss, compares value type against the declared via -`subst.apply` + equality (manual to preserve the -`AssignTypeMismatch` diagnostic identity), and yields `Type::unit()` -on success. Five `.ail.json` fixtures + a 5-test driver -(`mut_typecheck_pin.rs`) cover each diagnostic plus the positive -nested-shadow case; the carve-out inventory at -`carve_out_inventory.rs` is extended 7 → 12 to accept them. -`examples/mut.ail` (the mut.1 six-fn round-trip fixture) now -typechecks clean via `ail check`. Codegen stubs unchanged — -`examples/mut.ail` still cannot run end-to-end; that lands in mut.3. -Full `cargo test --workspace`: 592 passed / 0 failed (was 579 at -start of mut.2; +13 = 5 driver pins + 7 lib.rs `mod tests` pins + -1 inventory test count unchanged — the inventory entry count went -from `seven` to `twelve` but the test itself was already counted). - -## Per-task notes - -- iter mut.2.1 — `CheckError` variants + `code()` + `ctx()`: - Three new variants land before the `Internal` variant with - thiserror `#[error("[code] message")]` attributes per the - cli-diag-human convention. `code()` arms return the canonical - kebab strings (`mut-assign-out-of-scope`, `assign-type-mismatch`, - `mut-var-unsupported-type`). `ctx()` arms emit the structured - payloads named in the plan (`{name, available}`, - `{name, expected, actual}`, `{name, type}`). One in-crate pin - (`mut_typecheck_diagnostics_have_kebab_codes`) gates the code() - + ctx() shape; RED → GREEN observed. - -- iter mut.2.2 — `synth` signature threads `mut_scope_stack`: - New parameter `mut_scope_stack: &mut Vec>` - sits immediately after `locals` (canonical mid-position; matches - the existing convention for per-fn-body lexical state). 18 - recursive call sites in `lib.rs::synth` updated via - pattern-by-pattern `replace_all` (each first-arg form had a - unique-enough wrapper). Entry-point callers in `check_fn:1934` - and `check_const:2687` declare a fresh empty stack. mono.rs's - two re-synth sites at 712 + 1337 also declare a fresh empty - stack (they begin from a top-of-body position). Plan-recon - missed two more call sites: `lift.rs:723` (letrec-capture - re-synth) and `builtins.rs` (two test helpers - `synth_in_builtins_env` + a FloatPatternNotAllowed pin) — both - required for build green; both get fresh empty stacks. The - in-test mq.3 call site at `lib.rs:6663` also got the new - parameter. Build was red structurally (E0061 at 23+ sites) until - every call site was updated, then green. - -- iter mut.2.3 — `Term::Var` prepends mut-scope resolution: A 5- - line block at the top of the Var arm walks - `mut_scope_stack.iter().rev()` innermost-first, returning the - matched type cloned. Mut-vars are monomorphic per spec §"var - element types" — no Forall instantiation, no FreeFnCall. Two - pins (`mut_var_resolution_shadows_outer_local`, - `mut_var_resolution_picks_innermost_frame`) directly populate - the stack to isolate this change from the Mut-arm work in - Task 4 (the plan's literal Let { Mut { ... } } shape would have - hit the still-stubbed Mut arm at this stage; the property - protected — mut-scope wins over locals + innermost-wins on - shadowing — is identical). TDD discipline gap: the Var-arm edit - landed before the pin tests were observed RED on disk; the - reasoning is analytical (the pre-edit code consults - `locals.get(name)` first, so with `locals = { x → Int }` the - Var arm returned Int, failing the assert-Float; same logic for - the two-frame test). Recorded as Concern. - -- iter mut.2.4 — `Term::Mut` typecheck arm: Replaces the stub - with real logic per plan Step 3. A file-scope helper - `is_supported_mut_var_type(&Type) -> bool` gates each var's - declared type to `Int | Float | Bool | Unit` (no args). Per-var - loop synths the init in the outer scope plus the in-progress - frame (the in-flight frame is cloned and pushed during the init - synth, then popped — keeps `mut_scope_stack` truthful at every - nested call), unifies init type with declared, inserts the - (name → declared ty) entry into the frame. After all vars: push - the completed frame, synth body in extended scope, pop frame - (regardless of body outcome — the `body_result` is captured - before pop). Two pins: - `mut_block_typechecks_with_int_var_and_returns_int` (positive, - initially RED because Assign was still stubbed — flipped GREEN - by Task 5) and `mut_block_rejects_str_var_with_unsupported_type` - (negative, GREEN at Task 4 close). - -- iter mut.2.5 — `Term::Assign` typecheck arm: Replaces the stub - with real logic per plan Step 3. Resolves the target name via - `mut_scope_stack.iter().rev().find_map(|f| f.get(name).cloned())`. - None case: flattens every name in every frame innermost-first, - dedup-preserving-order via a manual `contains` check (small N, - no need for a Set), returns `MutAssignOutOfScope`. Some case: - synths the value, compares `subst.apply(&declared)` vs - `subst.apply(&value)` via `!=` (manual equality, NOT `unify` — - unify would have emitted `TypeMismatch`, and the spec demands - the distinct `AssignTypeMismatch` code), returns - `AssignTypeMismatch` with rendered type strings on mismatch or - `Ok(Type::unit())` on match. Three pins: - `assign_to_in_scope_mut_var_synths_unit` (positive), - `assign_to_out_of_scope_name_fires_diagnostic` (None case), - `assign_with_wrong_value_type_fires_diagnostic` (Some-mismatch - case). RED → GREEN observed for all three. - -- iter mut.2.6 — Five fixtures + driver test: Five `.ail.json` - fixtures land under `examples/` (carve-out style, per the - Boss decision in the carrier overriding the spec's literal - `crates/ailang-check/tests/`-side placement). Each fixture is - minimal — single-fn `main`, no imports, body shape exactly the - one the plan named. The driver test - `crates/ailang-check/tests/mut_typecheck_pin.rs` uses the - canonical `load_workspace` + `check_workspace` + - `diagnostics.iter().map(|d| d.code.clone()).collect()` pattern - from `method_collision_pin.rs` and `typeclass_22b2.rs` — no - `todo!()` survives. Carve-out inventory extended from seven to - twelve to accept the five new fixtures (alphabetised; the - comment block names mut.2 as the lineage). All 5 driver tests - GREEN; carve-out inventory GREEN. - -- iter mut.2.7 — Verify `examples/mut.ail` typechecks clean: - `cargo run --quiet --bin ail -- check examples/mut.ail` reports - `ok (26 symbols across 2 modules)` — zero diagnostics across - all six fns (`mut_empty`, `mut_single_var`, `mut_two_vars`, - `mut_nested_shadow`, `mut_returns_bool`, `mut_returns_unit`). - Full `cargo test --workspace`: 592 / 0. - -## Concerns - -- Task 3 TDD-ordering gap: the `Term::Var` arm prepend landed - before the two pin tests were observed RED on disk. The - reasoning that the pre-edit code would have failed both pins - is analytical (locals-first resolution returns Int when - `locals = { x → Int }`, contradicting both `assert_eq!(Float)` - and `assert_eq!(Bool)`). The discipline gap is recorded - per the implementer's Iron Law ("the test you wrote in RED MUST - pass; no other test may regress" — and the spirit-not-ritual - clause that tests-after prove nothing about whether they would - have caught the regression pre-edit). Subsequent tasks - (4 + 5) restored the RED-first sequencing; on those, the Mut / - Assign stubs already emit `CheckError::Internal` so the pins - were observed RED with the canonical pattern. No correctness - impact; the pins protect the property correctly post-edit. - -- Plan-recon missed three synth call sites beyond the four the - plan enumerated (lib.rs 1885 + 2634; mono.rs 712 + 1337). The - build-red structural signal flushed them out via E0061: - `lib.rs:6663` (in-test mq.3 helper), `lift.rs:723` (letrec- - capture re-synth), `builtins.rs` (`synth_in_builtins_env` at - 351 + a FloatPatternNotAllowed test pin at 594). Each gets a - fresh empty stack with the same "begins from top-of-body" - rationale as mono.rs. No behavioural impact; documentary - concern for the planner's next plan-recon dispatch. - -- The carve-out inventory extension was strictly required (the - inventory test enforces a closed list of `examples/*.ail.json` - files) but was not explicitly named in the Boss carrier or the - plan. Recording here so the planner's recon for future iters - knows to enumerate the side-effect when fixtures land under - `examples/`. The boundary call: the fixtures match §C4(a)'s - precedent (negative typecheck fixtures as `.ail.json` so the - diagnostic-code assertion is the load-bearing form, not the - surface). The positive `test_mut_nested_shadow_legal.ail.json` - is the one stretch — its load-bearing assertion is "zero - diagnostics", which the form-A `.ail` form could equally - carry. Landing it as `.ail.json` keeps all five mut.2 fixtures - on one driver and one extension on the carve-out list; the - alternative (splitting positive to `.ail`, negatives to - `.ail.json`) would have produced two drivers or a hybrid - loader. Conservative call documented for visibility. - -## Known debt - -- Codegen stubs unchanged in mut.2 per the spec's iteration - boundary. A `Term::Mut` or `Term::Assign` reaching - `lower_term` still produces `CodegenError::Internal`. The - full mut.ail E2E (build + run) lands in mut.3. The spec's - Acceptance criterion #3 (`mut_counter.ail` + `mut_sum_floats.ail` - build + run + print expected stdout) is mut.3-scoped. - -- The Mut-arm's in-progress-frame clone-on-each-init is O(n²) - over the var count. n is realistically ≤ 2 (the mut.ail - fixture's `mut_two_vars` is the largest in the corpus). Not - worth a more elaborate structure (e.g. pushing a single in- - progress frame and mutating it across inits); the clone-pop - shape keeps the stack truthful at every nested call and makes - the per-var init scope obvious. Recording for visibility. - -## Files touched - -Modified: -- `crates/ailang-check/src/lib.rs` — 3 new CheckError variants + - 3 code() arms + 3 ctx() arms; `synth` signature gains - `mut_scope_stack` after `locals`; 18 recursive synth call - sites updated; 2 entry-point callers (`check_fn`, - `check_const`) declare fresh empty stacks; in-test synth call - (mq.3 helper) gets new param; `Term::Var` arm prepends - mut-scope resolution (5-line block); `Term::Mut` stub replaced - with real logic (push/pop frame, per-var init, body synth); - `Term::Assign` stub replaced with real logic - (find/None-MutAssignOutOfScope/Some-AssignTypeMismatch-or-Unit); - file-scope helper `is_supported_mut_var_type`; 7 new tests in - `mod tests` (`mut_typecheck_diagnostics_have_kebab_codes`, - `mut_var_resolution_shadows_outer_local`, - `mut_var_resolution_picks_innermost_frame`, - `mut_block_typechecks_with_int_var_and_returns_int`, - `mut_block_rejects_str_var_with_unsupported_type`, - `assign_to_in_scope_mut_var_synths_unit`, - `assign_to_out_of_scope_name_fires_diagnostic`, - `assign_with_wrong_value_type_fires_diagnostic`) plus the - `synth_standalone` helper. -- `crates/ailang-check/src/mono.rs` — 2 re-synth call sites - (712, 1337) each get a fresh `mut_scope_stack`. -- `crates/ailang-check/src/lift.rs` — letrec-capture re-synth at - :723 gets a fresh `mut_scope_stack`. -- `crates/ailang-check/src/builtins.rs` — 2 in-test synth calls - (synth_in_builtins_env + FloatPatternNotAllowed pin) get a - fresh `mut_scope_stack`. -- `crates/ailang-core/tests/carve_out_inventory.rs` — EXPECTED - list 7 → 12, comment-block updated naming mut.2 as the - lineage of the 5 new entries. - -New: -- `crates/ailang-check/tests/mut_typecheck_pin.rs` — 5-test - driver via canonical load_workspace + check_workspace + - collect-codes pattern. -- `examples/test_mut_assign_out_of_scope.ail.json` -- `examples/test_mut_assign_outside_mut.ail.json` -- `examples/test_mut_assign_type_mismatch.ail.json` -- `examples/test_mut_nested_shadow_legal.ail.json` -- `examples/test_mut_var_unsupported_type.ail.json` - -## Stats - -bench/orchestrator-stats/2026-05-15-iter-mut.2.json diff --git a/docs/journals/2026-05-15-iter-mut.3.md b/docs/journals/2026-05-15-iter-mut.3.md deleted file mode 100644 index d0752f1..0000000 --- a/docs/journals/2026-05-15-iter-mut.3.md +++ /dev/null @@ -1,139 +0,0 @@ -# iter mut.3 — codegen + e2e for Term::Mut / Term::Assign - -**Date:** 2026-05-15 -**Started from:** b057789b55a86cd664e2d7fee2c998bb855cad98 -**Status:** DONE -**Tasks completed:** 9 of 9 - -## Summary - -Replaced the iter mut.1 codegen stubs for `Term::Mut` and `Term::Assign` -with real LLVM lowering: mut-vars become `alloca` slots hoisted to the -fn's entry block via a per-fn side buffer `pending_entry_allocas` -spliced into `self.body` at a byte marker captured during -`start_block("entry")`; `Term::Assign` lowers to `store`; references to -a mut-var name lower to `load` through a new lookup precedence in both -`lower_term`'s `Term::Var` arm and `synth_with_extras`'s `Term::Var` -arm (precedence: extras → mut-vars → locals → globals → builtins, -symmetric to the typecheck-side `mut_scope_stack` walk in -`ailang-check`). Lambda emission saves/restores the three new -`Emitter` fields across the thunk boundary and splices the thunk's own -hoisted allocas at its own entry marker so nested mut blocks inside -lambdas hoist into the lambda's entry, not the outer fn's. Two e2e -fixtures shipped (`mut_counter.ail` Int / `mut_sum_floats.ail` Float), -both printing `55` end-to-end (Float's `%g` formatter strips the `.0`). -DESIGN.md gains a "Local mutable state" bullet under "What **is** -supported". Closes the mut-local milestone end-to-end; the audit step -follows. - -## Per-task notes - -- iter mut.3.1: three `Emitter` fields (`mut_var_allocas`, - `pending_entry_allocas`, `entry_block_end_marker`) added with - doc comments; initialised in `Emitter::new`; cleared per-fn at the - top of `emit_fn` next to the existing per-fn-reset block. -- iter mut.3.2: `start_block` extended — when `label == "entry"` it - captures `self.body.len()` as `entry_block_end_marker` (the splice - point for `pending_entry_allocas`). -- iter mut.3.3: `Term::Var` arm of `lower_term` gained a mut-var - lookup before the `self.locals` walk — on hit, emits - `%v_N = load , ptr ` and returns the load SSA - + LLVM type. -- iter mut.3.4: `Term::Mut` arm of `lower_term` real lowering — for - each var: alloca into the side buffer, lower init in outer-plus- - earlier-vars scope (matches typecheck ordering at lib.rs:3578), - store init into alloca, then install the binding (saving any prior - entry for restoration on block exit). Lowers body, restores saved - bindings (unconditional restoration even on error path). -- iter mut.3.5: `Term::Assign` arm of `lower_term` real lowering — - look up the alloca in `mut_var_allocas`, lower value, emit store, - return the canonical Unit SSA form `("0", "i8")` matching - `Literal::Unit` in the same `match`. Tasks 4 and 5 were applied - in one `Edit` call because both arms were adjacent stubs that - needed simultaneous replacement (the `match` arm signature changed - from `{ .. }` to `{ name, value }` / `{ vars, body }`); the review - separation still held — each arm was spec-checked independently. -- iter mut.3.6: splice of `pending_entry_allocas` at the marker — - inserted into `self.body` at the end of `emit_fn` (after body - finalisation, before the deferred-thunks flush). Marker absence - while `pending` is non-empty fires `CodegenError::Internal` with - the fn name. -- iter mut.3.7: `examples/mut_counter.ail` (Int) and - `examples/mut_sum_floats.ail` (Float) authored; both use an - accumulator-style tail-recursive helper since while-loops are - deferred. Initial draft used the natural `(app + lo (tail-app ...))` - shape and was rejected by the tail-call analysis ("call marked - `tail` is not in tail position") — restructured to pass `acc` as - a third arg and yield it directly in the base case. Both fixtures - typecheck clean and print `55` (Float via `%g` strips `.0`). -- iter mut.3.8: `mut_counter_prints_55` + `mut_sum_floats_prints_55` - appended to `crates/ail/tests/e2e.rs`. The plan's placeholder - `expected = "55"` resolved by inspection of `examples/floats.ail` - + `crates/ail/tests/floats_e2e.rs` (where `1.5 + 2.5` prints as - `4`, confirming `%g`-driven trailing-zero strip — so `55.0` - prints as `55`); test pinned to `"55"`. -- iter mut.3.9: DESIGN.md "Local mutable state" bullet appended - after the "Parameterised ADTs" bullet at the end of the "What - **is** supported" subsection. Schema-drift tests pass (the bullet - lives outside §"Data model"). - -Two beyond-plan adjustments emerged during execution and were -absorbed inline (recorded as Concerns below): - -- `synth_with_extras`'s `Term::Var` arm also needed mut-var lookup - precedence. The plan's Task 3 only touched `lower_term`'s - `Term::Var` arm; without the parallel `synth_with_extras` update, - every polymorphic-call-site arg-type derivation that reaches a - mut-var reference fails with `UnknownVar`. Discovered by running - `examples/mut.ail` through codegen and observing the - `mut_single_var` case fail at `(app + x 1)`. -- Lambda emission (`lambda.rs`) needed save/restore of the three - new fields PLUS an in-thunk splice of `pending_entry_allocas` at - the thunk's own entry marker. Without this, a `Term::Mut` inside - a lambda body would push its alloca into the outer fn's side - buffer (or worse, leak into whichever fn's body the splice runs - on). Mirrors how `lambda.rs` already saves/restores - `non_escape` / `moved_slots` / `current_param_modes` across the - thunk boundary. - -## Concerns - -- Synth-side `Term::Var` mut-var-lookup parallel was not in the - plan but is structurally required. Recorded as a plan defect to - feed into the iter-mut.4 lessons (if any) and into the milestone- - close audit. -- Lambda-boundary save/restore + in-thunk splice was not in the - plan but is structurally required for correctness of any - `Term::Mut` inside a lambda body. Same defect category as above. - -## Known debt - -- None for the milestone-local scope. The spec's deferred items - (while-loops, heap-RC mut-var element types, `ref a` + `!Mut` - effect, `Stateful a b` + `pipe`) are explicitly named as future - milestones in the new DESIGN.md bullet. - -## Blocked detail - -n/a — Status: DONE. - -## Files touched - -- `crates/ailang-codegen/src/lib.rs` — three `Emitter` fields, init - + per-fn reset, `start_block` marker capture, `Term::Var` arm - mut-var precedence (both `lower_term` and `synth_with_extras`), - `Term::Mut` + `Term::Assign` arms in `lower_term`, splice at end - of `emit_fn`. -- `crates/ailang-codegen/src/lambda.rs` — save/restore of the three - new fields across the thunk boundary, in-thunk splice of - `pending_entry_allocas`. -- `crates/ail/tests/e2e.rs` — two new tests pinning - `mut_counter.ail` → `"55"` and `mut_sum_floats.ail` → `"55"`. -- `docs/DESIGN.md` — "Local mutable state" bullet at end of - "What **is** supported" subsection. -- `examples/mut_counter.ail` — new Int fixture. -- `examples/mut_sum_floats.ail` — new Float fixture. - -## Stats - -bench/orchestrator-stats/2026-05-15-iter-mut.3.json diff --git a/docs/journals/2026-05-15-iter-mut.4-tidy.md b/docs/journals/2026-05-15-iter-mut.4-tidy.md deleted file mode 100644 index 425ad87..0000000 --- a/docs/journals/2026-05-15-iter-mut.4-tidy.md +++ /dev/null @@ -1,137 +0,0 @@ -# iter mut.4-tidy — close mut-local audit drift - -**Date:** 2026-05-15 -**Status:** DONE -**Spec / Plan:** `docs/specs/2026-05-15-mut-local.md` (no spec changes -beyond a §"Out of scope" amendment); `docs/plans/2026-05-15-iter-mut.4-tidy.md`. -**Started from:** 6966cce -**Trigger:** audit close on the mut-local milestone. - -## What this iter shipped - -Tidy work closing the audit drift the architect surfaced after -mut.3 landed. Two `[high]` items, two `[medium]` items, plus a -substantive bench regression on `bench/compile_check.py` `check_ms`. - -### Architect [high] 1 — lambda-captures-of-mut-var diagnostic - -The spec's "mut-vars cannot escape their enclosing block" -invariant was not enforced by any check pass — codegen detected -the violation via a `CodegenError::Internal` whose comment blamed -the typechecker for letting it through. The fix lands the -diagnostic at the typecheck layer: - -- New variant `CheckError::MutVarCapturedByLambda { name: String }` - with kebab-case code `mut-var-captured-by-lambda`, `ctx()` - emitting `{"name": }`, and the canonical bracketed-`[code]:` - Display attribute. -- `Term::Lam`'s synth arm now walks the lambda body's free vars - (via the existing `ailang_core::desugar::free_vars_in_term` - helper, which already honours pattern-binding subtraction in - `Term::Match` arms) and rejects any free var whose name appears - in any enclosing `mut_scope_stack` frame. -- The walk is gated by `!mut_scope_stack.is_empty()` so lambdas - outside any mut block pay no overhead. - -### Architect [high] 2 — codegen comment cleanup - -The codegen lambda.rs path that previously raised -`CodegenError::Internal("lambda capture {c} not in locals — -typechecker bug?")` now uses `unreachable!` with a message -naming the typecheck rejection that gates it. The companion -comment block at `lambda.rs:474` was rewritten to state -the current reality (typecheck rejects via -`MutVarCapturedByLambda`) instead of the stale promise that -"mut.2 typecheck will enforce this" (mut.2 didn't). - -### Architect [medium] 3 + 4 — stale mut.1-stub history comments - -- `docs/DESIGN.md` §"Term (expression)" mut/assign block: the - trailing paragraph describing the iter mut.1 stub state was - replaced with one describing the current end-to-end mut.3 - reality. The `// Iter mut.1` comment on the `{"t": "assign"}` - jsonc fragment was updated to drop the "deferred to mut.2/mut.3" - language. -- `crates/ailang-codegen/src/lib.rs:1749-1758` — the comment block - above the real `Term::Mut` codegen arm still described the mut.1 - stub. Removed entirely; the real `Iter mut.3` comment immediately - below it is now the only descriptive header. - -### Bench regression ratify — `check_ms` ~30-50% uniform shift - -`bench/compile_check.py` flags an 11-fixture-wide `check_ms` -regression on the mut-local milestone (30-50% relative; ~0.5ms -absolute on a 1.4-1.5ms baseline). The pattern is **uniform across -fixtures of very different Var counts** (`hello.ail` 7 lines and -`bench_list_sum_explicit.ail` 100+ lines both shift by ~0.5ms), -which indicates a fixed-cost-per-`ail check`-invocation tax rather -than a hot-path-per-Var cost. - -Root-cause analysis: - -- **Hypothesis A: `mut_scope_stack` walk at every Var resolution.** - Tested by the Task 3 short-circuit (`if !mut_scope_stack.is_empty()`). - Re-bench: regression unchanged. **Falsified.** -- **Hypothesis B: synth-parameter-passing overhead.** Threading - `&mut Vec>` through ~19 recursive synth calls per - Term-tree walk. Each call pays a 64-bit reference push to the - stack frame. Not directly testable without rolling the - threading back. Plausible but speculative. -- **Hypothesis C: binary size / startup cost.** mut.* added - ~1400 lines of code across `ailang-check` and `ailang-codegen`. - Larger binary → slightly slower process startup, dynamic linker - resolution, instruction-cache warmup. ~0.5ms is in the right - ballpark for that kind of cost on a small binary. The uniform- - across-fixtures pattern is consistent with this — it's not - Var-proportional. - -The pragmatic call: hypothesis B and C are both consistent with the -observation; neither has a cheap remedy at this milestone. The -short-circuit in Task 3 closes hypothesis A's contribution if any, -and remains the right shape for future workloads that DO have -non-empty `mut_scope_stack`. The remaining shift is the cost of -the feature. - -**Disposition: ratify** as a fixed-cost feature tax. The journal -records this disposition; the `bench/compile_check.py` baseline is -updated to the post-mut.4-tidy numbers. Future bench observations -will tolerance-check against the new baseline. If a future iter -finds a substantive hot-path that retroactively explains the shift, -the ratify can be revisited. - -`bench/check.py` regressions are tail-latency metrics -(`latency.implicit_at_rc.p99_9_us / max_us`) consistent with the -"established noise envelope, 15th consecutive observation" finding -from audit-pd (2026-05-14). No ratify needed — they fall under -the existing noise carve-out. - -`bench/cross_lang.py` is clean (25/0/0/25). Runtime is unchanged -by mut-local. - -## Working tree - -Eight files: 7 modified + 1 new (`examples/test_mut_var_captured_by_lambda.ail.json`). - -## Tests - -594 → 598 green (4 new: the variant code-pin smoke test, the -positive `lambda_outside_mut_still_typechecks_clean` regression -guard, the negative `lambda_inside_mut_capturing_mut_var_emits_mut_var_captured_by_lambda` -in lib.rs, and the integration-test -`lambda_capturing_mut_var_emits_mut_var_captured_by_lambda`). - -## Concerns - -- **`Term::Match` pattern binding over-approximation.** The - `free_vars_in_term` walker correctly subtracts pattern-bound - names; reused without modification. No false positives - encountered in the existing test corpus. -- **Bench-ratify decision.** Documented above. The 30-50% - uniform shift is plausible but not proven to be feature-cost. - A future bencher dispatch could test hypothesis B by branching - off a "remove the mut_scope_stack parameter, use a thread-local - / Env field instead" prototype and measuring the difference. Out - of scope for this tidy. -- **`bench/compile_check.py` baseline updated** in this commit. - The audit-skill discipline ("NO BASELINE UPDATE WITHOUT A - PAIRED JOURNAL RATIFY ENTRY") is satisfied by this entry. diff --git a/docs/journals/2026-05-16-audit-iteration-discipline-revert.md b/docs/journals/2026-05-16-audit-iteration-discipline-revert.md deleted file mode 100644 index 1855794..0000000 --- a/docs/journals/2026-05-16-audit-iteration-discipline-revert.md +++ /dev/null @@ -1,107 +0,0 @@ -# audit — iteration-discipline-revert milestone close - -**Date:** 2026-05-16 -**Milestone:** iteration-discipline-revert (HEAD `37ac704`) -**Verdict:** Milestone-iteration-discipline-revert tidy — **clean** - -## Step 1 — Architect drift review - -The revert is byte-pristine: all 18 reverted production source files -+ 5 reverted test files byte-identical to `git show 1ff7e81:` -(`crates/ailang-check/src/lib.rs` fully so); zero residual it.1/it.2 -production surface (`Term::Loop`/`Recur`/`LoopBinder`/ -`verify_structural_recursion`/`term_contains_loop`/`module_fns`/ -`"Diverge"`-injection / the five `Recur*`+`NonStructuralRecursion` -variants all absent from `crates/*/src/`). DESIGN.md coherent: -Decision 3 & 8 byte-identical to oracle, no loop/recur in §"Data -model", clause 3 retained (hunk-edited, not wholesale-restored — the -`1ff7e81` DESIGN.md had zero clause-3 text) with its worked example -de-claimed to explicit hypothetical, `skills/brainstorm/SKILL.md` -de-claimed in lockstep, F1/F4 idiom note present and guarded by a -passing `design_schema_drift.rs` test. Roadmap: Iteration-discipline -block + blocking-fork section gone, single deferred P2 entry with -correct `depends on:`/`context:`, F3 todo + Stateful-islands intact, -old spec carries the superseded header. - -One `[medium]` drift item: - -- `docs/roadmap.md` Stateful-islands `[~]` scope bullet still - asserted *"the 'Iteration discipline' P1 milestone supersedes the - earlier 'while-loops legal' scope … repetition is structural - recursion or named `loop`/`recur`"* — a dangling cross-reference - into the now-reverted milestone from a **live planning doc** that - would mislead the next brainstorm into treating `loop`/`recur` as - the committed repetition story. **Resolved inline** (Boss - doc-hygiene, full context already loaded): reworded to keep the - real, still-true scope guard (AILang has no `while`/`for`; this - milestone introduces none; repetition stays recursion exactly as - the language has it today; a `mut` block is a sealed expression, - never a loop) and drop the reverted-milestone reference. - -## Step 2 — Bench-regression check - -- `bench/compile_check.py` — exit 0 (24/24 stable). -- `bench/cross_lang.py` — exit 0 (25/25 stable; notably - `bench_list_sum.ail_bump_s` only +2.44% vs baseline). -- `bench/check.py` — exit 1: single metric - `throughput.bench_list_sum.bump_s` baseline 0.046s → 0.052s, - +12.71% (tol 10%); the 4 "improvements" are `gc_over_bump` ratio - shifts downstream of that one number. - -**Localisation (bencher, hypothesis-driven).** Built release `ail` -at HEAD `37ac704` and at the byte-oracle `1ff7e81` (read-only -`git worktree`, removed after). The compiled `bench_list_sum` -bump binaries are **`cmp`-identical** (byte-for-byte same machine -code, identical stdout). Interleaved 3 trials × 60 runs/arm: head -median 0.051233s vs oracle median 0.051206s; head−oracle delta -oscillates around zero (+0.05% / +0.61% / −0.20%, within ~1.5% -stdev); **both** arms read ~+11% over the 0.046 baseline on this -machine *now*. Verdict: **environmental/hardware drift relative to -the 2026-05-09 baseline-capture machine state, NOT a revert -regression** — confirmed independently of the architect byte-diff -and the implement Task-11 behavioural-equivalence sweep (third -independent proof the revert's codegen == `1ff7e81`'s codegen). - -## Step 3 — Classification & resolution - -- Architect `[medium]` roadmap dangling xref → **fix**, done inline - (Boss doc-hygiene; no separate tidy iteration warranted for a - one-bullet prose correction with a single correct outcome). -- `check.py` exit 1 → **carry-on**, baseline **NOT ratified**. - Ratify is categorically wrong here: nothing *intentionally* moved - the metric (the revert returned codegen to byte-identical - `1ff7e81`), and the audit Iron Law forbids a baseline bump without - an iter that intentionally moved it. Bumping 0.052 in would bake a - noise/environment value into `bench/baseline.json`. The spec's - "bench PRISTINE, not ratified" mandate is **satisfied**: pristine - means the revert introduced no regression, and it provably did not - (byte-identical binaries vs the clean `1ff7e81` oracle). -- The pre-existing `*.bump_s` baseline-vs-hardware staleness (the - `1ff7e81` oracle itself trips the same +11% on current hardware — - predates this milestone, revert-foreign) recorded as a separate - **P2 roadmap `[todo]`** (bench-harness recalibration; no language - change), so it is tracked without being mis-attributed to the - revert or laundered through a ratify. - -## Fieldtest - -**Skipped** (Boss judgement; fieldtest is non-mandatory). A revert -restores the already-field-tested pre-`9973546` user-visible surface -(structural / tail recursion; `tail-app` intact) — the surface that -the mut-local fieldtest already exercised and that started this -corrective arc. A fieldtest here would re-test a known, well-trodden -state and add no signal; the implement Task-11 byte-equivalence -sweep over 164 fixtures is the stronger correctness proof for a -revert. - -## Close - -Milestone **iteration-discipline-revert** closed clean: -byte-pristine vs `1ff7e81` across all production surface; behaviour -byte-identical over 164 surviving fixtures; `cargo test --workspace` -600/0; one roadmap drift fixed inline; the single `check.py` metric -localised to environmental variance (baseline untouched) with the -pre-existing staleness filed forward. AILang's iteration story is -exactly the pre-`9973546` state; the genuine total-recursion -ambition is preserved on the roadmap, sequenced behind a future -`Nat`/refinement-types milestone. diff --git a/docs/journals/2026-05-16-iter-effect-doc-honesty.md b/docs/journals/2026-05-16-iter-effect-doc-honesty.md deleted file mode 100644 index 896d0ad..0000000 --- a/docs/journals/2026-05-16-iter-effect-doc-honesty.md +++ /dev/null @@ -1,81 +0,0 @@ -# iter effect-doc-honesty — effect-system documentation-honesty tidy - -**Date:** 2026-05-16 -**Started from:** c41e0e5a9e48fd0099eda8c286f10ec7a30e8e8a -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -Documentation-honesty tidy: corrected three false effect-system -claims plus their two satellite mentions across the project docs, -guarded by one new doc-presence regression pin. No language, -checker, or codegen code changed — only a doc-comment, two markdown -spec files, an internal CLI guidance string, and a new integration -test. `ail check`/`ail run` behaviour is byte-unchanged by -construction; `cargo test --workspace` 600 → 604 (the four new pin -tests, zero regressions), `cargo build --workspace` clean. The -three sentinel drift/coverage tests (`design_schema_drift`, -`spec_drift`, `schema_coverage`) stay green, empirically confirming -none scans the effect-prose region. - -## Per-task notes - -- iter effect-doc-honesty.1: new pin `crates/ailang-core/tests/effect_doc_honesty_pin.rs` - (4 tests asserting fiction strings absent + corrected anchors - present across the 4 edited files); authored first, ran RED 0/4 - for the right reason (TDD: RED before corrections). -- iter effect-doc-honesty.2: `docs/DESIGN.md` Decision 3 row-polymorphic - block → flat/unordered/closed set-equality + Diverge-reserved - wording; line-2722 "IO and Diverge ops" bullet → IO-only + - Diverge-reserved bullet. `design_md_effect_prose_is_true` GREEN. -- iter effect-doc-honesty.3: `crates/ailang-core/src/ast.rs` `Term::Do` - doc-comment effect-handler-table-at-link-time fiction → real - mechanism (typecheck against `Env::effect_ops`, `lower_effect_op` - literal-match codegen). `term_do_doc_comment_is_true` GREEN. -- iter effect-doc-honesty.4: `crates/ailang-core/specs/form_a.md` - line-226 "currently `IO` and `Diverge`" → IO-built-in-op / - Diverge-reserved; rule-3 "`Diverge` for `diverge/*`" → - io/print_str-is-the-only-built-in-op. `form_a_spec_effect_names_are_true` - GREEN. -- iter effect-doc-honesty.5: `crates/ail/src/main.rs` merge-prose - CONTRACT effect-annotation bullet dropped the non-existent - `(effects Diverge)` example. Whole pin 4/4 GREEN. -- iter effect-doc-honesty.6: full verification — whole pin 4/4, - `cargo test --workspace` 604/0 (600 prior + 4), drift/coverage - trio green, `cargo build --workspace` clean. - -## Concerns - -- Task 2: the plan's verbatim Task-2 replacement body soft-wrapped - the phrase "flat, unordered,\nclosed set of effect names" - mid-phrase, which contradicts Task-1's single-line pin substring - `"flat, unordered, closed set of effect names"` AND the plan's - own self-review item 6 ("every pin substring is single-line — the - documented iter-revert grep-wrap failure mode is avoided"). The - two authoritative artefacts (exact replacement text vs. exact - pin) could not both be applied verbatim. Resolved by preserving - the exact corrected *wording* (byte-identical to the plan's - intent) and the exact pin (Task 1, unchanged), moving only the - soft-wrap column so the asserted phrase stays contiguous on one - line. No wording added/dropped/changed. This is a planner - self-review-item-3 cross-check gap (the asserted-substring vs. - replacement-body check did not account for the replacement body's - own line wrap) — orchestrator may want to tighten the planner's - pin/replacement-body wrap cross-check. - -## Known debt - -- (none) — the tidy is self-contained; no deferred follow-up. - -## Files touched - -- `crates/ailang-core/tests/effect_doc_honesty_pin.rs` (new — the pin) -- `docs/DESIGN.md` (Decision 3 + "What is not supported" bullet) -- `crates/ailang-core/src/ast.rs` (`Term::Do` doc-comment) -- `crates/ailang-core/specs/form_a.md` (line 226 + rule 3) -- `crates/ail/src/main.rs` (merge-prose CONTRACT string) - -## Stats - -bench/orchestrator-stats/2026-05-16-iter-effect-doc-honesty.json diff --git a/docs/journals/2026-05-16-iter-revert.md b/docs/journals/2026-05-16-iter-revert.md deleted file mode 100644 index 6bddafd..0000000 --- a/docs/journals/2026-05-16-iter-revert.md +++ /dev/null @@ -1,174 +0,0 @@ -# iter revert — Iteration-discipline milestone surgical forward revert - -**Date:** 2026-05-16 -**Started from:** abf00131c10ff9287b419439c565480c96fa861b -**Status:** DONE -**Tasks completed:** 11 of 11 - -## Summary - -The Iteration-discipline milestone (it.1 `96db54d` + it.2 `a4be1e5`) -is fully backed out by a single forward iteration, with `1ff7e81` -(the last commit before milestone-start `9973546`) as the per-region -byte oracle. `main` was never rewound — every restore is a forward -working-tree edit, net-new it.1/it.2 surface is `rm`'d, DESIGN.md is -hunk-edited in place (clause 3 / criterion section KEPT, never -wholesale-restored). Feature-acceptance clause 3 and the F3 todo are -retained; the clause-3 worked example is de-claimed from "shipped" to -hypothetical-illustration form (in DESIGN.md and brainstorm SKILL.md -in lockstep); one honest F1/F4 documentation note is added (the only -net addition) and guarded by a doc-presence test. The correctness -gate (Task 11) proved compiler behaviour **pristine**: all 164 -surviving `1ff7e81`-era fixtures `ail check`/`ail run` byte-identical -to their pre-milestone behaviour; `cargo test --workspace` 600/0; -zero residual it.1/it.2 production surface. AILang's iteration story -is exactly the pre-`9973546` state (structural / tail recursion; -`tail-app` intact); the genuine total-recursion ambition is preserved -on the roadmap as a deferred P2 milestone sequenced behind a future -`Nat`/refinement-types milestone. - -## Per-task notes - -- revert.1: AST node family + drift anchors + DESIGN.md §Data-model — - removed `Term::Loop`/`Term::Recur`/`struct LoopBinder` + 2 serde - units (ast.rs byte-identical to oracle); loop/recur exemplars + - match arms removed from design_schema_drift.rs / spec_drift.rs / - schema_coverage.rs (byte-identical); DESIGN.md §Data-model - loop/recur JSON + it.1/it.2 prose removed (region byte-identical). -- revert.2: `CheckError` surface — 5 variants + doc comments, 5 - `code()` arms, 3 dedicated `ctx()` arms deleted; `_ =>` catch-all - preserved; diagnostic.rs confirmed genuine no-op. -- revert.3: it.2 guardedness pass + Diverge-injection — STOP-gate - cleared (all 6 named helpers it.2-net-new at oracle → all delete, - no ambiguity). Removed the entire 687-line guardedness cluster - (verify_structural_recursion, term_contains_loop, adt_families, - referenced_con_names, mutual_structural_group, adt_param_positions, - RecCall, TypeUnionFind, MutualGroup, et al.) **plus** the - transitively-it.2 `module_fns` plumbing through - check_module/check_def/check_fn signatures (oracle proved net-new; - resolved via the plan's Oracle convention + recon-by-name mandate — - see Concerns). 3 Diverge-injection sites restored byte-identical; - lib.rs `"Diverge"` count 0 == oracle. -- revert.4: it.1 check surface — synth Loop/Recur arms + - verify_loop_body deleted; loop_stack param + ~30 threaded calls + 9 - local decls removed; verify_tail_positions survives, it.1 arms - removed, byte-identical to oracle; substitute_rigids_in_term + - post-mono name arms cleaned. Result: - `crates/ailang-check/src/lib.rs` FULLY byte-identical to `1ff7e81`. -- revert.5: 9 remaining walker files (lift/mono/linearity/uniqueness/ - reuse_shape/pre_desugar_validation/builtins + core desugar/ - workspace) restored to oracle, all byte-identical; pre-restore diff - classification proved every hunk it.1/it.2-only. -- revert.6: surface + prose + form-A grammar — parse.rs/print.rs/ - prose lib.rs/form_a.md restored to oracle byte-identical; each - touched only by 96db54d (it.1) since 1ff7e81; surface+prose build - clean. -- revert.7: codegen — lib.rs/escape.rs/lambda.rs + ail/main.rs - restored to oracle byte-identical; `block_terminated = true` count - == 8 == oracle (it.1 parallel setter gone; the 4 tail-driven + 4 - control-flow setters survive). `cargo build --workspace` clean - (first full-build gate, lockstep clustering worked as designed). -- revert.8: STOP-gate re-verify cleared (live `git diff --stat` - matched plan's net-new/modified/absent classification exactly). - Deleted 2 pin files + 16 fixtures + bench/it3-oracle via plain - `rm` (no git-index touch). 2 RC fixtures restored byte-identical. - e2e.rs + codegen_import_map_fallback_pin.rs + carve_out_inventory.rs - restored byte-identical (carve = "Twelve carve-outs", last entry - test_mut_var_captured_by_lambda.ail.json). it.{1,2,3} stats json - KEPT. cargo test --workspace 599/0. -- revert.9: DESIGN.md Decision 3 → 1ff7e81 placeholder (byte-identical, - hunk-only); clause-3 worked example de-claimed to hypothetical form - (criterion section + clause-3 discriminator KEPT); brainstorm - SKILL.md de-claimed in lockstep; F1/F4 note added verbatim; - doc-presence test added verbatim, PASSES. -- revert.10: roadmap Iteration-discipline [~] block + blocking-fork - sub-section removed; deferred "Iteration-totality story" entry - added verbatim at top of P2 (per plan + spec sequencing); F3 + - Stateful-islands untouched; superseded header prepended to the old - spec; Step-4 STOP-check clean (no residual A1 amendment). -- revert.11 (correctness gate): 1ff7e81 ref compiler built in - throwaway worktree; 164-fixture check/run oracle captured; - post-revert compiler diffed byte-for-byte — fail=0, ZERO DRIFT, - behaviour PRISTINE. cargo test --workspace 600/0. Residual - it.1/it.2 production-surface grep clean. Worktree removed; main - HEAD never moved. - -## Concerns - -- revert.3: the plan's Task-3 file-map enumerated the guardedness - helpers + call site but did NOT explicitly list the `module_fns` - signature threading through check_module/check_def/check_fn. The - `1ff7e81` oracle unambiguously proved that threading it.2-net-new; - leaving it would have been dead it.2 surface referencing deleted - helpers. Resolved correctly via the plan's own "Oracle convention" - + "re-locate by symbol name, oracle is authority on every byte" - mandate + the spec §Data-flow ("restored to pure pre-it.1 role"). - This is the recon-map-drifts-but-oracle-is-authority mechanism - working as designed, not a plan defect — no bounce. Flagged so the - Boss is aware the plan file-map under-enumerated one signature - edge; the end-state is verified correct (lib.rs byte-identical to - oracle). -- revert.9: the plan's Task-9 Step-6 single-line grep heuristic - (`grep -c 'clauses 1 and 2 yet fail clause 3' ...`) returned 0 - because the verbatim-applied replacement text line-wraps ("yet - fail\n clause 3"). The substantive acceptance criterion - (clause-3 block retained, only de-claimed) is positively verified - multiline + by the intact `## Feature-acceptance criterion` section - and "criterion 3 is the discriminator" sentence. Not a defect; - flagged so the Boss does not misread the heuristic. - -## Known debt - -(none — the revert is byte-pristine against `1ff7e81` for all -production surface; behaviour is byte-identical over the surviving -corpus.) - -## Blocked detail - -(none — DONE.) - -## Files touched - -90 working-tree paths (Boss commits all as one cohesive revert iter): - -- Deleted (59): `crates/ailang-check/tests/loop_recur_pin.rs`, - `crates/ailang-check/tests/structural_recursion_pin.rs`; 16 it.1/it.2 - fixtures under `examples/` (loop_*.ail, struct_rec_*.ail, - test_recur_*.ail.json, test_loop_missing_diverge / test_mutual_cross_family - / test_non_structural_recursion .ail.json); `bench/it3-oracle/` - (MANIFEST.tsv + 40 .out). -- Modified — source revert to 1ff7e81 (byte-identical): - `crates/ailang-core/src/ast.rs`, `desugar.rs`, `workspace.rs`; - `crates/ailang-check/src/lib.rs`, `lift.rs`, `mono.rs`, - `linearity.rs`, `uniqueness.rs`, `reuse_shape.rs`, - `pre_desugar_validation.rs`, `builtins.rs`; - `crates/ailang-codegen/src/lib.rs`, `escape.rs`, `lambda.rs`; - `crates/ail/src/main.rs`; `crates/ailang-surface/src/parse.rs`, - `print.rs`; `crates/ailang-prose/src/lib.rs`; - `crates/ailang-core/specs/form_a.md`. -- Modified — tests revert to 1ff7e81 (byte-identical): - `crates/ailang-core/tests/spec_drift.rs`, `schema_coverage.rs`, - `carve_out_inventory.rs`; `crates/ail/tests/e2e.rs`, - `codegen_import_map_fallback_pin.rs`. -- Modified — net-add / hunk-edit: - `crates/ailang-core/tests/design_schema_drift.rs` (loop/recur - anchors removed + new doc-presence test added), - `docs/DESIGN.md` (Decision 3 placeholder + clause-3 de-claim + - F1/F4 note; hunk-edited, clause 3 kept), `docs/roadmap.md` - (Iteration-discipline block removed, deferred P2 entry added), - `docs/specs/2026-05-15-iteration-discipline.md` (superseded - header), `skills/brainstorm/SKILL.md` (worked-example de-claim). -- Modified — fixtures restored to 1ff7e81: - `examples/rc_pin_recurse_implicit.ail`, - `examples/rc_let_alias_implicit_param.ail`. -- KEPT (no-op, Boss decision): - `bench/orchestrator-stats/2026-05-15-iter-it.{1,2,3}.json`. - -E2E coverage: none (revert iteration — the Task-11 behavioural- -equivalence sweep over all 164 surviving fixtures is a stronger -correctness proof than new E2E; the doc-presence test guards the -only net-new artefact). - -## Stats - -bench/orchestrator-stats/2026-05-16-iter-revert.json diff --git a/docs/journals/2026-05-17-iter-loop-recur.1.md b/docs/journals/2026-05-17-iter-loop-recur.1.md deleted file mode 100644 index 23b266d..0000000 --- a/docs/journals/2026-05-17-iter-loop-recur.1.md +++ /dev/null @@ -1,180 +0,0 @@ -# iter loop-recur.1 — Additive AST-Node Foundation (Term::Loop / Term::Recur / LoopBinder) - -**Date:** 2026-05-17 -**Started from:** a5eebcd5fd9d703cc4f8a6374f95e00061d86c55 -**Status:** DONE -**Tasks completed:** 7 of 7 - -## Summary - -`loop` / `recur` become real, parseable, printable, round-trippable, -hash-stable, strictly-additive AST nodes across all six crates plus -the drift tests — with NO typecheck semantics and NO real codegen -lowering (those are loop-recur iters 2 and 3). Added `Term::Loop { -binders, body }`, `Term::Recur { args }`, and `struct LoopBinder` -(mirroring `MutVar` exactly); Form-A `(loop (NAME TYPE INIT)* BODY+)` -/ `(recur ARG*)` parse + print + grammar/notes; prose render / -count_free_var / subst_var arms; ~30 structural pass-through walker -arms; the two semantic dispatch points stubbed with -`CheckError::Internal` (`synth`) / `CodegenError::Internal` -(`lower_term`) exactly as mut.1 did; DESIGN.md + form_a.md schema -blocks; drift/coverage/hash anchors; and the spec's worked `sum_to` -program as a green `examples/loop_sum_to.ail` round-trip fixture. -`cargo test --workspace` 608 green / 0 red; the iter13a + new -loop-recur hash pins both green (additivity proven — pre-existing -canonical-JSON hashes byte-stable); all `tail`-filtered tests green -(Boss-call-1 operational evidence). - -## Per-task notes - -- iter loop-recur.1.1: Core AST — `Term::Loop`/`Term::Recur` variants - + `struct LoopBinder` + 2 serde round-trip unit tests. RED-first - (`error[E0599]`); GREEN after Task 4 (desugar/workspace are *in* - ailang-core, so the lib's own exhaustive matches gate it — see - Concerns). Both serde tests green. -- iter loop-recur.1.2: Surface parser (`parse_loop`/`parse_recur` + - head dispatch + unknown-head message), printer arms, form_a.md - grammar+notes. Binder/body boundary disambiguated via an - `is_term_head_kw` discriminator (the plan's flagged "one genuine - parser judgement"). `parse_loop_and_recur_round_trip_via_term` - green (exactly 2 binders + `if` body + 2-arg recur). -- iter loop-recur.1.3: Prose projection — `write_term_prec`, - `count_free_var`, `subst_var_with_term` Loop/Recur arms (verbatim - plan code, lexical-shadow-aware mirroring the Mut arms). -- iter loop-recur.1.4: ailang-core walkers — 9 desugar.rs sites + 2 - workspace.rs sites. `cargo build -p ailang-core` GREEN. -- iter loop-recur.1.5: ailang-check — 14 steps across 7 files - including the Boss-call-1 `verify_tail_positions` two mandatory - additive arms and the Boss-call-2 `synth` `CheckError::Internal` - stub. `cargo build -p ailang-check` GREEN. -- iter loop-recur.1.6: ailang-codegen + ail/main.rs — `lower_term` - `CodegenError::Internal` stub + real structural pass-through at - the pure analysis walkers (lambda/escape ×3) + `synth_with_extras` - pass-through + main.rs walk_term/rewrite_term. `cargo build - --workspace` GREEN. -- iter loop-recur.1.7: DESIGN.md + form_a.md schema blocks; - design_schema_drift + spec_drift + schema_coverage anchors; new - loop_recur hash pin; `examples/loop_sum_to.ail` positive fixture. - Full suite 608/0; tail-app non-regression green. - -## Boss design calls (mirrored from the plan header at iter close) - -1. **`verify_tail_positions` "byte-unchanged" = tail-app role - unchanged, NOT source frozen.** `verify_tail_positions` - (`crates/ailang-check/src/lib.rs`) is an exhaustive no-`_`-wildcard - `match`-on-`Term`; the two additive variants *require* two new - arms there or `ailang-check` does not compile. The added arms - only define how the tail-app walker descends through two - brand-new node kinds that no pre-existing fixture contains — - zero behaviour change for any pre-existing construct. The - acceptance evidence is the tail-app non-regression test (all - `tail`-filtered tests green), NOT a frozen-source diff. mut.1 set - the exact precedent (it added `Term::Mut`/`Term::Assign` arms - inside the same function). Settled; not reopened — the - spec-compliance phase did NOT emit `unclear`/BLOCKED on the - anticipated "spec says byte-unchanged but the function changed" - tension, per the plan header's authority. -2. **Codegen IS in iter-1 scope as stubs/pass-throughs (mirrors - mut.1 site-for-site).** "No codegen in iter 1" means no real - loop-header/phi/back-edge lowering (that is iter 3). But - `ailang-codegen` carries no-wildcard exhaustive `Term` matches; - the workspace would not compile without arms. Resolved exactly - as mut.1: `CodegenError::Internal` stub at the `lower_term` - dispatch + real structural pass-through at the pure analysis - walkers (lambda.rs collect_captures, escape.rs walk/escapes/ - collect_free_vars) and `synth_with_extras`. No real loop lowering - written this iter. - -## Concerns - -- DONE_WITH_CONCERNS (iter loop-recur.1.1/1.2/1.4): three - plan-pseudo-vs-reality substitutions, all of the recurring - "specs need concrete code / plan pseudo vs reality" class - (`feedback_specs_need_concrete_code`, `feedback_plan_pseudo_vs_reality`), - each resolved by preserving the arm/test's evident intent and - fixing to the real codebase: - 1. **Task-1 serde expected-bytes literal.** The plan asserted - `"type":{"t":"con","name":"Int","args":[]}`; the real canonical - `Type::Con` form is `{"k":"con","name":"Int"}` (discriminator - `"k"`, empty `args` omitted — established since mut.2). The - test's stated intent (pin one-binder Loop canonical bytes; - prove `binders` has no `skip_serializing_if`) is preserved; - only the mis-transcribed expected string was corrected. NOT a - bug-dodge — `{"k":"con",…}` is the canonical form every Type - fixture uses; the plan author mis-recalled the Type JSON. - 2. **Bare `Int` vs `(con Int)` in binder triples.** The spec's - worked example and the plan's Step-1 test source / Step-6 - fixture write `(loop (acc Int 0) …)` with bare `Int`. The real - `parse_type` parses a bare `Int` ident as `Type::Var{"Int"}` - (a type *variable*), not the Int constructor — semantically - wrong and inconsistent with every `mut` fixture - (`(var x (con Int) 0)`), the spec schema, and the Task-1 serde - pin. Corrected to `(con Int)` in the parser test source and the - `examples/loop_sum_to.ail` fixture. This is the LLM-natural AND - correct Form-A type (not a fixture-adapts-to-bug case — every - sibling fixture uses `(con Int)`). - 3. **`ailang_core::ast::LoopBinder` path inside ailang-core.** The - plan's literal desugar.rs/workspace.rs arms used the - downstream-crate path `ailang_core::ast::LoopBinder`; inside - ailang-core itself (`use crate::ast::*;`) the correct path is - unqualified `LoopBinder` (consistent with the adjacent bare - `MutVar` usage). `subst_call_with_extras`'s plan arm also used - `(target, extras)`; the real signature is `(name, lifted, - extras)` — mirrored from the local Mut arm per the plan's own - instruction. ailang-check (downstream) correctly keeps the - `ailang_core::ast::LoopBinder` path. - -## Boss addendum (post-orchestrator, at commit) - -Concern 2 surfaced that the bare-`Int`-in-binder imprecision is not -just in the plan/test but in the **committed specs themselves**: -`docs/specs/2026-05-17-loop-recur.md`'s headline clause-1 worked -example + the must-fail `bad_recur` fixture, and -`docs/specs/2026-05-17-llm-surface-discipline.md` §5's illustrative -anchor, all wrote `(loop (acc Int 0) (i Int 1) …)`. Bare `Int` -parses as `Type::Var{"Int"}`, so the spec's *headline evidence* — -the program an LLM author would copy verbatim — does not parse to -the typed loop it claims. Same doc-honesty class as the mono.rs -header / effect-doc-honesty: a spec must show code that actually -parses to what it asserts. Boss forward-fixed all three snippets to -`(con Int)` (the established convention every `mut` fixture + the -spec's own schema use); folded into this iter's commit. No design, -semantics, schema, or load-bearing assumption changed — a -surface-syntax precision fix, so no re-brainstorm / re-grounding. - -## Known debt - -- No additional E2E fixture written (Phase 3). iter-1 ships NO - typecheck and NO codegen by design; an `ail run` e2e would only - exercise the deliberately-temporary `synth`/`lower_term` - `Internal` stubs (a negative-value test asserting a transient - error). The milestone invariant worth protecting at iter-1 — - additivity (hash pins), round-trip (the new fixture), drift trio, - tail-app non-regression — is fully covered by tests added in - Tasks 1/2/7. The spec's positive sum_to-runs-to-a-value E2E is - explicitly iter-3 scope. - -## Files touched - -- AST + core walkers: `crates/ailang-core/src/ast.rs`, - `desugar.rs`, `workspace.rs` -- Surface: `crates/ailang-surface/src/parse.rs`, `print.rs`; - `crates/ailang-core/specs/form_a.md` -- Prose: `crates/ailang-prose/src/lib.rs` -- Check: `crates/ailang-check/src/{lib,lift,mono,linearity, - uniqueness,reuse_shape,pre_desugar_validation}.rs` -- Codegen + CLI: `crates/ailang-codegen/src/{lambda,escape,lib}.rs`, - `crates/ail/src/main.rs` -- Docs/drift/coverage/hash: `docs/DESIGN.md`, - `crates/ailang-core/tests/{design_schema_drift,spec_drift, - schema_coverage,hash_pin}.rs`, - `crates/ail/tests/codegen_import_map_fallback_pin.rs` - (recon-undercount: an integration-test exhaustive `Term` match - not in the plan's site inventory; same additive-arm class, - mirrors the Mut defensive-recursion shape — Boss-call-2-class - "recon under-scoped it") -- New fixture: `examples/loop_sum_to.ail` - -## Stats - -bench/orchestrator-stats/2026-05-17-iter-loop-recur.1.json diff --git a/docs/journals/2026-05-17-iter-loop-recur.2.md b/docs/journals/2026-05-17-iter-loop-recur.2.md deleted file mode 100644 index 0dd1f77..0000000 --- a/docs/journals/2026-05-17-iter-loop-recur.2.md +++ /dev/null @@ -1,185 +0,0 @@ -# iter loop-recur.2 — Typecheck Semantics (Component 4) - -**Date:** 2026-05-17 -**Started from:** 5ac57fe8dec2d83ba135dd5ff6dd9caa1165c700 -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -The iter-1 `synth` `CheckError::Internal` stub for `Term::Loop` / -`Term::Recur` is replaced with real typecheck semantics: binder -typing (each init synthed in outer scope + already-declared binder -names of THIS loop; loop's static type = body type), positional -`recur` arity + per-arg type checking via a new `loop_stack: &mut -Vec>` frame threaded exactly as mut.2's `mut_scope_stack`, -the four `Recur*` `CheckError` variants (bracket-`[code]`-free -Display per F2) + `code()` + two `ctx()` arms, and a NEW private -`verify_loop_body` tail-position pass (a *sibling* of the -spec-frozen `verify_tail_positions`, run after `synth` in `check_fn` -so synth-raised codes take precedence by pass ordering). Four -negative `.ail.json` fixtures fire their codes point-exactly -(`assert_eq!`, non-vacuous); the iter-1 `loop_sum_to.ail` -round-trip fixture now ALSO typechecks clean; an infinite `loop` -typechecks (no termination claim). NO codegen this iter (the iter-1 -`lower_term` `CodegenError::Internal` stub stays — iter 3); NO -`Diverge`/`verify_structural_recursion`/guardedness (spec deliberate -boundary). `verify_tail_positions` is byte-frozen (0 deletions). -`cargo test --workspace` 616 green / 0 red (iter-1 baseline 608). - -## Per-task notes - -- iter loop-recur.2.1: the four `Recur*` `CheckError` variants - (`RecurOutsideLoop`, `RecurArityMismatch{expected,got}`, - `RecurTypeMismatch{position,expected,got}`, - `RecurNotInTailPosition`) + 4 `code()` arms + 2 `ctx()` arms - (Arity/Type only; OutsideLoop/NotInTailPosition use the `{}` - catch-all). RED-first via `recur_checkerror_codes_are_exact` - (observed 4× E0599); GREEN exact. Bracket-`[code]`-free Display. -- iter loop-recur.2.2: `synth` signature gains - `loop_stack: &mut Vec>` after `mut_scope_stack`; - threaded at every recursive call site (17 single-line canonical + - 3 multi-line `Term::Mut`-init/body & `Term::Assign`-value + 2 - letrec-capture `&mut body_effects`-shape) + `check_const` (fresh - empty stack) + cross-module callers + test-module callers. The - compile-driven sweep (`cargo build`/`cargo test --no-run`) was the - exhaustive caller oracle as the plan specifies; iterated to 0 - errors. Stub intact, lib suite green. -- iter loop-recur.2.3: iter-1 `Internal` stub replaced with the two - real `Term::Loop`/`Term::Recur` arms. Binder-name save/restore - mirrors `Term::Let` (`lib.rs:3205-3218`) verbatim; per-arg type - check mirrors `Term::Assign` (`lib.rs:3716-3730`) verbatim per - Boss call 1; `recur`'s own type = `Subst::fresh(counter)`. - RED `loop_sum_to_typechecks_clean` observed `["internal"]`; - GREEN zero diagnostics. -- iter loop-recur.2.4: new private `verify_loop_body(t, in_loop_tail)` - immediately after `verify_tail_positions`'s closing brace — - exhaustive no-`_` match over all 16 `Term` variants (verified - equal to `verify_tail_positions`' variant set). Invoked from - `check_fn` after `verify_tail_positions(&f.body, true)?;`. RED - `recur_not_in_tail_position` observed `left: []`; GREEN - `["recur-not-in-tail-position"]`, positive still clean. -- iter loop-recur.2.5: 3 remaining negative fixtures - (`test_recur_outside_loop` / `_arity_mismatch` / `_type_mismatch`) - + 3 pins (all 5 green) + `carve_out_inventory` 13→17 with header - refresh (`Twelve…mut.2` → `Seventeen…loop-recur.2`, pre-existing - Twelve-vs-13 mut.4-tidy drift corrected in passing) + ct1 F2 - sibling `check_human_mode_renders_recur_diagnostic_code_exactly_once`. -- iter loop-recur.2.6: `examples/loop_forever.ail` (loop whose only - path is `recur`, no exit) typechecks clean — asserts the spec - "no termination claim is made or enforced". Form-A grammar - verified verbatim against the shipped `loop_sum_to.ail`. Full - suite 616/0; tail-app non-regression green (9 `tail`-named tests - ok incl. the iter-1 verify_tail_positions guards). - -## Boss design calls (mirrored from the plan header at iter close) - -1. **`RecurTypeMismatch` is an Assign-style structural pre-check, - NOT a `unify`-propagate.** `Term::Recur` per-arg checking mirrors - the `Term::Assign` arm (`lib.rs:3716-3730`) verbatim in - mechanism: synth the arg, `subst.apply` both the arg type and the - binder type, structural `!=`, on mismatch - `Err(RecurTypeMismatch{position,expected,got})`. A - `unify`-propagate would surface the generic `type-mismatch` code - and fail the spec acceptance ("dedicated `recur-type-mismatch` - fires point-exactly"). Rationale semantic (one in-repo mechanism - for "declared-vs-actual at a binding site with its own - point-exact code", consistent with `AssignTypeMismatch`), not - effort. Implemented verbatim; the `test_recur_type_mismatch` - fixture fires exactly `["recur-type-mismatch"]`. -2. **`loop_stack` element type is `Vec` (ordered binder - types), NOT `IndexMap`.** `recur` rebinds binders - *positionally*, so the frame models the ordered binder types for - arity (`.len()`) + per-position type checks. Binder *names* enter - the ordinary `locals: &mut IndexMap` exactly as - `Term::Let` binds its name (save/restore in reverse on loop - exit). The spec phrase "threaded exactly as mut.2's - `mut_scope_stack`" governs the *threading discipline* - (a `&mut Vec<…>` pushed/popped around the body, passed through - every recursive `synth`), not the element type. Semantic, not - effort. Implemented verbatim. -3. **Diagnostic-code precedence is by pass ordering, no explicit - logic.** `RecurOutsideLoop`/`RecurArityMismatch`/`RecurTypeMismatch` - are raised in `synth` (runs at the `check_fn` synth call); - `RecurNotInTailPosition` is raised in `verify_loop_body`, invoked - *after* `verify_tail_positions(&f.body, true)?;`. A `recur` - outside any loop therefore fires `RecurOutsideLoop` (synth, - first), not `RecurNotInTailPosition`. No precedence logic added; - the pass sequence IS the mechanism. `verify_loop_body` is entered - only on synth success, so every `recur` reached there is already - inside a loop with matching arity/types — the pass adds only the - tail rule. `verify_tail_positions` is byte-frozen (0 deletions); - `verify_loop_body` is a new sibling, never a repurpose. - -## Concerns - -- DONE_WITH_CONCERNS (iter loop-recur.2.2): **plan ordering defect — - `check_fn` threading must be in Task 2, not Task 4.** The plan's - Task 2 Step 3 bullet says "check_fn :1970: handled in Task 4 Step - 4 (it declares its own fresh loop_stack)", but Task 2 Step 4's - gate is `cargo build -p ailang-check` reaching `Finished` with 0 - errors. That gate is *unsatisfiable* while `check_fn` remains an - unthreaded `synth` caller (hard `error[E0061]`). Resolution: - `check_fn`'s `loop_stack` declaration + synth-call threading - (verbatim the plan's Task 4 Step 3 block + Step 4 first half) were - pulled into Task 2 where the compile gate forces them; only the - `verify_loop_body` *invocation* (Task 4 Step 4 second half) - remained for Task 4. This is the unique resolution satisfying both - Task 2 Step 4 AND preserving all plan code verbatim — the "fix to - compile while preserving evident intent + Boss calls" repair the - carrier explicitly authorises. Correctness: the declaration code - is byte-identical to the plan's Task 4 Step 3 block; no semantic - change, only sequencing. Recurring planner-gap class (a - compile-gate task whose gate depends on a caller the plan defers - past the gate) — candidate for a planner Step-5 self-review check. -- DONE_WITH_CONCERNS (iter loop-recur.2.2): **recon under-counted - cross-module `synth` callers** — `builtins.rs` ×2 (test helpers), - `lift.rs:746` (letrec-capture re-entry), `mono.rs:720` & - `mono.rs:1361` (mono re-synth) were not in the plan's site - inventory (`:3161…:3717` + the 8 test sites). Exactly the - mut.2-class recon-undercount the iter-1 journal flagged - (Boss-call-2-class "recon under-scoped it"). Resolved by mirroring - the established mut.2 fresh-stack pattern at each (declare a fresh - empty `loop_stack` alongside the existing fresh `mut_scope_stack`, - pass `&mut loop_stack` immediately after `&mut mut_scope_stack`). - Observation, not a correctness risk — the compile-driven sweep IS - the plan's designated exhaustive oracle and surfaced every site; - the plan's literal site list is advisory, the compiler is - authoritative (as the plan itself states). - -## Known debt - -- No additional `ail run`/binary-stdout E2E fixture written (Phase - 3). iter-2 ships typecheck semantics only; the `lower_term` - `CodegenError::Internal` stub stays by design (real loop lowering - + the positive sum_to-runs-to-a-value / deep-`n` E2E is explicitly - iter-3 scope). An `ail run` this iter would only exercise the - deliberately-transient codegen stub (a negative-value test of a - known-temporary error) — same justified Phase-3 reasoning as - iter-1. The iter-2 invariants worth protecting (four codes - point-exact, positive + infinite typecheck clean, - carve-out/tail-app non-regression) are fully pinned by the tests - added in Tasks 1/3/4/5/6 + the ct1 F2 sibling. - -## Files touched - -- Check core: `crates/ailang-check/src/lib.rs` (4 variants + - code/ctx, `synth` signature + threading, real Loop/Recur arms, - new `verify_loop_body`, `check_fn` + `check_const` wiring), - `crates/ailang-check/src/lift.rs`, `crates/ailang-check/src/mono.rs`, - `crates/ailang-check/src/builtins.rs` (cross-module synth-caller - threading) -- Tests/fixtures: `crates/ailang-check/tests/loop_recur_typecheck_pin.rs` - (new — 7 tests), `crates/ail/tests/ct1_check_cli.rs` (F2 sibling), - `crates/ailang-core/tests/carve_out_inventory.rs` (13→17 + header) -- New fixtures: `examples/test_recur_outside_loop.ail.json`, - `examples/test_recur_arity_mismatch.ail.json`, - `examples/test_recur_type_mismatch.ail.json`, - `examples/test_recur_not_in_tail_position.ail.json`, - `examples/loop_forever.ail` -- Reused (no new file): `examples/loop_sum_to.ail` (iter-1 - round-trip fixture, now also the positive `ail check` evidence) - -## Stats - -bench/orchestrator-stats/2026-05-17-iter-loop-recur.2.json diff --git a/docs/journals/2026-05-17-iter-loop-recur.3.md b/docs/journals/2026-05-17-iter-loop-recur.3.md deleted file mode 100644 index c4e0371..0000000 --- a/docs/journals/2026-05-17-iter-loop-recur.3.md +++ /dev/null @@ -1,188 +0,0 @@ -# iter loop-recur.3 — Codegen (Component 5) + run-to-value E2E - -**Date:** 2026-05-17 -**Started from:** eae73bf320440f6b031715e72c1f20b0ac86e024 -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -The **terminal iteration** of the standalone-`loop`/`recur` milestone: -the iter-1 `lower_term` `CodegenError::Internal` stub for -`Term::Loop`/`Term::Recur` is replaced with real LLVM-IR lowering, so -a `sum_to`-class loop program builds and runs to the correct printed -value, a deep-`n` variant is safe by construction, and an infinite -loop compiles. Loop binders are loop-carried values lowered as -entry-block allocas (the mut.3 `pending_entry_allocas` mechanism) -registered in the existing `mut_var_allocas` map, so the existing -`Term::Var` load path resolves them with zero new Var code -(representation-sharing); a fresh `loop.header.` block is reached -by an unconditional `br` from the pre-header, `recur` lowers each arg -to SSA *before* any store (simultaneous positional rebind), stores -into the binder allocas, back-edges `br` to the header, and sets the -single existing `block_terminated` field at its OWN new emit site so -the loop's exit value is the emergent product of the existing -`if`/`match` join (no separate loop-result phi/exit-block). A new -`loop_frames` codegen stack lets `recur` find its target; it is -saved/reset/restored at the single lambda-lowering boundary exactly -as mut.3's `mut_var_allocas` triple. `clang -O2` mem2reg promotes -the binder allocas to the phi nodes the spec's implementation-shape -describes. This is a **codegen-only iteration**: no schema/AST/serde -change, no typecheck change; hash pins (`loop_recur` + iter13a) and -the drift trio + `carve_out_inventory` stay green untouched -(confirmed, not modified — empirical proof none scans the codegen -region). `cargo test --workspace` 616 → 619 / 0 red (the 3 new e2e -tests). After this iter the milestone is structurally CLOSED; -milestone-close (audit/fieldtest) is the Boss's post-iter call. - -## Per-task notes - -- iter loop-recur.3.1: positive `sum_to` run-to-value E2E (RED → - GREEN). RED observed verbatim: `ail build` on `loop_sum_to_run.ail` - fails with the iter-1 stub `internal: Term::Loop/Term::Recur - lowering lands in loop-recur iter 3` (parse + iter-1/iter-2 - typecheck pass; only codegen stubbed — confirmed via direct `ail - build`). GREEN: `loop_frames` field + init, lambda-boundary - save/reset/restore, the two real `Term::Loop`/`Term::Recur` arms - replacing the single stub arm. `loop_sum_to_run.ail` → `55`; - `cargo build -p ailang-codegen` Finished; full e2e suite 88/0. - Plan's literal arms compiled as-written — no plan-vs-reality - substitution needed (pre-grounding confirmed `mut_var_allocas: - (String, Type)` and `lower_term -> (String /*ssa*/, String - /*llvm_ty*/)` matched the plan's assumptions exactly). -- iter loop-recur.3.2: deep-`n` safe-by-construction + - infinite-loop-compiles (test+fixture only; the codegen mechanism - shipped in 3.1, so no separate RED). `loop_sum_to_deep.ail` runs - 1e6 iterations via the back-edge → `500000500000` with no stack - growth (the spec clause-2 correctness claim made executable — - mem2reg promoted the binder allocas so the loop is an iterative - back-edge, not a stack-growing recursion). `loop_forever_build.ail` - (`spin`'s loop has no non-`recur` exit) compiles `ail build` exit - 0 — `recur` sets `block_terminated`, propagating through - `emit_fn`'s `if !self.block_terminated` fall-through guard so - `spin` emits no `ret` and is a well-formed never-returning fn; the - binary is never executed (spec "typechecks AND compiles, no - termination claim"). Full e2e suite 90/0. -- iter loop-recur.3.3: regression gates — pure verification, ZERO - source changes (confirmed `git diff --name-only HEAD` = only the - 3 T1/T2 source files). tail-app/tail-do/`verify_tail_positions` - non-regression green (the parallel `block_terminated` SET site - touched no existing SET/READ site — Boss call 4); `loop_recur` + - iter13a hash pins + drift trio + `carve_out_inventory` green - untouched (codegen-only iter, no schema change); full workspace - 619 passed = exactly the loop-recur.2 baseline (616) + the 3 new - e2e tests, zero FAILED/error; `round_trip` green (the 3 new `.ail` - fixtures auto-discovered, parse→print→parse idempotent — Roundtrip - Invariant holds for runnable loop/recur programs). - -## Boss design calls (mirrored from the plan header at iter close) - -All four were settled in the plan header on -architectural-consistency / spec-pinned-invariant grounds (NOT -effort); each verified against actual read source before -implementation; none reopened, re-derived, or "improved" in any -phase: - -1. **Loop binders → entry-block alloca + `mut_var_allocas` reuse, - NOT hand-emitted `phi`.** The spec's "phi" wording lives only in - the explicitly-*secondary* implementation-shape subsection. The - established in-repo loop-carried-value mechanism is - alloca-in-entry-block (mut.3 `pending_entry_allocas`) + `clang - -O2` mem2reg, which produces the identical optimized binary. The - linear-emit codegen architecture structurally cannot hand-emit a - header `phi` whose back-edge predecessors are discovered *during* - body lowering without a string-splice hack with no precedent (the - `if` arm sidesteps this by opening its join block *last* — a loop - header cannot be opened last). Rationale = architectural - consistency + absence of a linear-emit `phi` precedent + identical - -O2 output. Implemented verbatim; `loop_sum_to_run.ail`→55 and - `loop_sum_to_deep.ail`→500000500000 are the operational proof - mem2reg delivers the loop-carried-SSA intent. -2. **Binders ride the existing `mut_var_allocas` map + the existing - `Term::Var` load path (representation-sharing).** A loop binder's - codegen representation — a named alloca slot, loaded on each - `Term::Var` use, stored on `recur` — is structurally identical to - a mut-var's. The `Term::Loop` arm inserts each binder into - `mut_var_allocas` (value type `(String, Type)`, identical to the - `Term::Mut` precedent); the existing `lib.rs` `Term::Var` arm - resolves them via its pre-existing `mut_var_allocas.get(name)` → - `load` path **byte-unchanged** (zero edits to Var-resolution; no - parallel map). Verified against real source before implementing. - mut.3's shipped+E2E-green mut-var read path is the proof the load - works for loop binders. -3. **The loop's exit value is the emergent product of the existing - `if`/`match` join once `recur` sets `block_terminated` — no - separate loop-result phi/exit-block.** The body is typically - `(if c (recur …))`; the existing - one-branch-terminated logic at `lib.rs:1606-1613` already drops - the terminated (recur) branch and returns the non-recur branch's - `(ssa,ty)` directly — *iff* `recur` sets `block_terminated`. The - `Term::Loop` arm therefore returns `lower_term(body)`'s `(ssa,ty)` - directly; no separate loop-result phi was built. An infinite loop - (body = bare `recur`) compiles because `recur` sets - `block_terminated`, propagating through the `emit_fn` - `if !self.block_terminated` fall-through guard exactly as a - tail-app-terminated function. -4. **Reuse the single `block_terminated` field; `recur` sets it at - its OWN new emit site (a parallel SET call-site), zero edits to - any existing SET or READ site.** A back-edge `br` IS a block - terminator with exactly `block_terminated`'s semantics. `recur`'s - own `self.block_terminated = true;` is a NEW call-site parallel - to (not replacing) tail-app's SET at `lib.rs:2279`. The diff - confirms tail-app's SET and every READ site - (`:1124/1581/1594/1600-1613/...`) are byte-identical; the - tail-app/`verify_tail_positions` non-regression gate (T3 Step 1) - is the operational proof. - -## Concerns - -- DONE_WITH_CONCERNS (iter loop-recur.3.3): the plan's literal Task-3 - Step-1 command `cargo test --workspace tail` filters on the - test-name substring `tail`, which matches **no** test in the - workspace (the real tail-app guards are - `ailang-check::lib::tail_call_in_{non_,}tail_position_is_*`, - `ailang-prose::lib` `*_tail_*`, e2e - `iter14e_print_list_recursion_emits_musttail`). Ran the gate via - the real test names instead (2 + 3 + 1, all green); the - authoritative non-regression gate is Step 3's full-workspace - 619/0 which subsumes them. Recurring `feedback_plan_pseudo_vs_reality` - class (a literal verification command whose filter doesn't resolve - to its evident target); resolved by preserving the gate's evident - intent (tail-app/`verify_tail_positions` byte-unchanged proven), - no behaviour change. Observation, not a correctness risk — the - full-suite count `619 = 616 + 3` is independent confirmation - nothing regressed. - -## Known debt - -- None. The iteration's invariants (loop/recur lowers to real IR; - run-to-value 55; deep-`n` 500000500000 safe-by-construction; - infinite loop compiles; tail-app + hash pins + drift trio - byte-unchanged; round-trip on the 3 new fixtures) are fully pinned - by the 3 new e2e tests + the unchanged regression suite. - -## Files touched - -- Codegen: `crates/ailang-codegen/src/lib.rs` (`loop_frames` field + - init; the two real `Term::Loop`/`Term::Recur` arms replacing the - iter-1 single stub arm), `crates/ailang-codegen/src/lambda.rs` - (lambda-boundary `loop_frames` save/reset + restore, alongside the - mut.3 triple) -- Tests: `crates/ail/tests/e2e.rs` (3 new test fns: 2 build+run, 1 - build-only) -- New fixtures: `examples/loop_sum_to_run.ail`, - `examples/loop_sum_to_deep.ail`, `examples/loop_forever_build.ail` -- No edit / byte-frozen (confirmed, not modified): every existing - `block_terminated` SET/READ site + tail-app/tail-do lowering + - `verify_tail_positions`; the iter-2 typecheck (`synth` - Loop/Recur arms, `verify_loop_body`, `loop_stack`, the four - `Recur*` variants); the iter-1 `synth_with_extras` - Loop→body-type / Recur→Unit pass-through; `hash_pin.rs` - (`loop_recur` + iter13a pins) + the drift trio + - `carve_out_inventory.rs` (codegen-only iter; the 3 new `.ail` - fixtures are round-trip-auto-covered, NOT carve-outs — inventory - stays 8) - -## Stats - -bench/orchestrator-stats/2026-05-17-iter-loop-recur.3.json diff --git a/docs/journals/2026-05-17-iter-loop-recur.tidy.md b/docs/journals/2026-05-17-iter-loop-recur.tidy.md deleted file mode 100644 index e827fcd..0000000 --- a/docs/journals/2026-05-17-iter-loop-recur.tidy.md +++ /dev/null @@ -1,133 +0,0 @@ -# iter loop-recur.tidy — lambda capturing a loop binder must fail `ail check`, not panic codegen - -**Date:** 2026-05-18 -**Started from:** 39380d361d530a95a4f1c718c0e201f8312d3ac4 -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -A `Term::Lam` whose body captured an enclosing `Term::Loop` binder -passed `ail check` (exit 0) and then panicked `unreachable!()` at -`crates/ailang-codegen/src/lambda.rs:102` — type-correct input -crashing the compiler. This iter adds the symmetric escape guard: -such a capture now fails `ail check` with the stable kebab code -`loop-binder-captured-by-lambda`, exactly as mut.4-tidy did for -`mut-var-captured-by-lambda`. The fix is the minimal mechanism that -reuses the scope structure the `Term::Loop` arm already maintains. - -## Per-task notes - -- iter loop-recur.tidy.1: GREEN side of the debug RED test - `lambda_capturing_loop_binder_emits_loop_binder_captured_by_lambda`. - Added `CheckError::LoopBinderCapturedByLambda { name }` - (bracket-free F2 Display), its `code()` arm - (`loop-binder-captured-by-lambda`), and its `ctx()` arm - (`{name}`, mirroring `MutVarCapturedByLambda`). Extended - `loop_stack`'s element type from `Vec` to - `Vec<(String, Type)>` so the existing per-loop scope frame also - carries binder names (the `Term::Recur` arm reads `.1` for the - positional type check; the `Term::Lam` escape guard reads `.0`). - The `Term::Lam` arm's existing free-var scan (shared - `free_vars_in_term` walker) gained a parallel pass over - `loop_stack` next to the `mut_scope_stack` pass; its gate widened - to `!mut_scope_stack.is_empty() || !loop_stack.is_empty()`. - Mirrored the two mut.4-tidy in-source unit tests - (kebab-code test + synth-level capturing test). Lockstep: - `carve_out_inventory.rs` 17→18 (header count word + new bullet + - EXPECTED entry), `ct1_check_cli.rs` mut-F2 sibling `cases` gained - the new fixture (where `mut-var-captured-by-lambda` is asserted — - the exact precedent). `docs/DESIGN.md` one-line note adjacent to - the `MutVarCapturedByLambda` mention. - -## Root cause - -The `Term::Lam` escape guard iterated only `mut_scope_stack` to -reject lambda captures of alloca-resident locals. Loop binders are -inserted into the ordinary `locals` by the `Term::Loop` arm and -never entered any escape-checked scope, so a lambda body referencing -a loop binder was accepted by `ail check`. Codegen's lambda capture -resolver searches only `self.locals` after `mut_var_allocas` is -`std::mem::take`-emptied at the lambda frame boundary, so the -loop-binder reference resolved to nothing and hit `unreachable!()`. -Loop binders are alloca-resident under the same scheme as mut-vars -and obey the same no-escape-into-lambda rule. - -## Design note — why extend `loop_stack` rather than add a parallel stack - -The carrier offered "reuse whatever scope structure the `Term::Loop` -arm already maintains" as an explicit minimal option. `loop_stack` -is already threaded through every `synth` recursive call and is -pushed/popped at exactly the loop scope boundary. Extending its -frame element from `Type` to `(String, Type)` preserves all existing -push/pop discipline and `Term::Recur` positional-type semantics -unchanged — it is not a redesign. The alternative (a parallel -`mut_scope_stack`-shaped parameter) would have touched all 35 -recursive `synth` call sites plus declarations, strictly larger. -Element-type extension is the minimal correct mechanism. - -## Concerns - -(none) - -## Known debt - -(none) - -## Files touched - -- `crates/ailang-check/src/lib.rs` — variant, code/ctx/Display, the - `Term::Loop` push + `Term::Recur` read + `Term::Lam` escape guard, - two in-source unit tests -- `crates/ailang-check/src/builtins.rs` — `loop_stack` decl type -- `crates/ailang-check/src/lift.rs` — `loop_stack` decl type -- `crates/ailang-check/src/mono.rs` — `loop_stack` decl type (x2) -- `crates/ailang-core/tests/carve_out_inventory.rs` — 17→18 lockstep -- `crates/ail/tests/ct1_check_cli.rs` — mut-F2 sibling `cases` -- `docs/DESIGN.md` — one-line symmetric-rule note - -## Milestone-close audit resolution (Boss-side) - -The loop/recur milestone-close `audit` (architect drift review + -bench gate) produced four items; this tidy commit resolves all: - -- **`[high]` lambda-captures-loop-binder codegen crash** — fixed by - this iter's GREEN (the symmetric `loop-binder-captured-by-lambda` - guard). RED test committed separately as the audit trail - (`39380d3`); GREEN + the items below fold into this commit. -- **`[medium]` `mut_var_allocas` rustdoc lied about dual use** — - `crates/ailang-codegen/src/lib.rs` rustdoc said "Populated when - entering a `Term::Mut` block" / "matching the typecheck-side - mut-scope-stack precedence"; as of iter-3 loop binders ride the - same map. Boss-edited to present-tense dual-use truth (mut-vars - AND loop binders; the shared map is a representation detail, not - a scope-precedence claim). Doc-honesty class. -- **`[medium]` `Term::Loop` doc-comment stale "per-binder phi"** — - `crates/ailang-core/src/ast.rs` still described iter-1's - forward-looking `synth`/`lower_term` `Internal` stubs + "per-binder - phi"; iters 2/3 shipped real typecheck + alloca+mem2reg lowering - (Boss-call-1 explicitly rejected hand-emitted phi). Boss-edited to - the real shipped state. Doc-honesty class. -- **`[low]` recon-undercount 3× pattern** — added a P2 `[todo]` to - `docs/roadmap.md` (`ailang-plan-recon` cross-crate-caller-undercount - countermeasure; pairs with the planner Step-5 items 7+8 added this - milestone — those scrub the plan, this scrubs the recon). Not a - milestone-blocking fix; tracked forward. -- **Bench gate (audit Step 2)** — `bench/check.py` / - `compile_check.py` / `cross_lang.py` all exit 0; 25 metrics, 0 - regressed, 25 stable. **Pristine** — exactly as the spec's - acceptance criteria expect for a strictly-additive surface no - hot-path bench exercises; the pre-existing P2 `*.bump_s` - hardware-staleness drift did not trip this run. **carry-on**: no - baseline update, no ratify entry (Iron Law — nothing intentionally - moved a metric). - -**Milestone-loop-recur tidy: resolved.** Architect drift cleared -(the `[high]` is fixed code, the two `[medium]` are doc-honest, the -`[low]` is roadmap-tracked); bench pristine carry-on. The loop/recur -milestone is audit-clean; the only remaining pipeline step before -full close is `fieldtest` (user-visible Form-A surface). - -## Stats - -bench/orchestrator-stats/2026-05-17-iter-loop-recur.tidy.json diff --git a/docs/journals/2026-05-18-audit-docs-honesty-lint.md b/docs/journals/2026-05-18-audit-docs-honesty-lint.md deleted file mode 100644 index 4bc9fb8..0000000 --- a/docs/journals/2026-05-18-audit-docs-honesty-lint.md +++ /dev/null @@ -1,133 +0,0 @@ -# audit docs-honesty-lint — milestone close (CLEAN, carry-on) - -**Date:** 2026-05-18 -**Scope:** `928e1c0..e50b400` (spec `aff25cd`, plan `de66eb7`, iter -`7580d43`, INDEX `e50b400`) -**Status:** CLEAN — carry-on, no fix iteration, NO baseline ratify -**Milestone:** docs-honesty-lint (single iteration) — **CLOSED** - -## Outcome - -| Gate | Result | -|------|--------| -| Architect drift | `clean` (one `[medium]` standing advisory wart, see below) | -| `bench/check.py` | EXIT 1 — **causally exonerated**, do NOT ratify | -| `bench/compile_check.py` | EXIT 0 (24/24 stable) | -| `bench/cross_lang.py` | EXIT 0 (25/25 stable) | -| Recommendation | **carry-on** | - -## Step 1 — Architect drift review - -Status `clean`. Verified: - -- **Scope claim exact.** `git diff 928e1c0..e50b400 --stat -- - 'crates/**/src/**' 'runtime/'` empty; sole `crates/` change is the - additive `crates/ailang-core/tests/docs_honesty_pin.rs` (RED-first, - now GREEN 4/4). The two lockstep-invariant pairs (`Pattern::Lit`/ - `pre_desugar_validation`, `lower_app`/`is_static_callee`) untouched. -- **DESIGN.md edits are themselves honest.** All 14 corrections - genuinely present-tense; no Wunschdenken/post-mortem reintroduced. - All four protected-exception KEEPs survive un-stripped: `Diverge` - reserved (L2730), `Regions … considered and rejected` at the - Decision site (L1530), `a tiebreaker, not a rationale` (L1240), - `No deriving` (L1850). The `### What this document is — and the - honesty rule it holds itself to` meta-subsection is itself - present-tense (no "we will / planned"). -- **Agent-def lockstep complete.** `skills/audit/agents/ailang-architect.md` - "four"→"five sweeps" purge verified (`grep "four sweeps…"` empty); - new "DESIGN.md honesty drift" bullet present and citing the - meta-subsection. -- **Sweep residue is exactly the 3 documented deliberate KEEPs.** - `bash bench/architect_sweeps.sh` EXIT 1 with: Sweep-1 L50 - (`docs/journal-archive.md … pre-2026-05-11 history` — present-tense - doc-roles pointer, incidental date) + L2060 (the true - `(iter str-concat, 2026-05-13)` provenance tag — asserts nothing - false, consistent with the sibling `(iter 24.1)` tags); Sweep-5 L62 - (the discriminator subsection's OWN forbidden-phrasing catalogue — - a self-referential false-positive, it IS the rule definition). Each - correctly adjudicated KEEP per the new bullet's discriminator. - -**`[medium]` standing advisory wart (architect-flagged, orchestrator- -adjudicated → filed, not fixed now).** Sweep 5's regex matches the -discriminator subsection's own example catalogue (DESIGN.md L62) -permanently: every future `architect_sweeps.sh` run inherits a -guaranteed self-referential EXIT 1 that must be re-adjudicated by -hand. This does not introduce a *new* failure mode — the script has -never been EXIT 0 in practice (Sweeps 1-4 already retain legitimate -date-anchors; the script header states non-zero is advisory, -architect-adjudicated by design). It is a minor signal-erosion, not -current drift. Filed as a P3 roadmap todo (optional Sweep-5 -self-anchor exclusion); explicitly NOT a fix iteration — consistent -with "trivial drift recorded, not ignored, but priced honestly at -P3". - -## Step 2 — Bench-regression check - -`check.py` EXIT 1, three firings: `throughput.bench_list_sum.bump_s` -+13.42% (tol 10%); `latency.implicit_at_rc.p99_9_us` +37.65% (tol -25%); `latency.implicit_at_rc.max_us` +134.70% (tol 30%). 58 stable, -2 improvements. `compile_check.py` EXIT 0; `cross_lang.py` EXIT 0 — -and `cross_lang.py` independently measures the *same* `bench_list_sum` -workload at `ail_bump_s +5.95% ok`, `ail_rc_s +5.08% ok` (the fixture -`check.py` calls a +13.42% REGRESSION reads within tolerance under a -fresher anchor / wider band). - -## Step 3 — Bencher causal exoneration (decisive) - -Hypothesis: the firings were *caused by this milestone's diff*. -Refuting test: HEAD bench binaries byte-identical to a pre-milestone -commit ⇒ no causal mechanism. Result (verbatim): - -- `cmp` of `bench_list_sum_{bump,gc,rc}` + `bench_latency_implicit_{rc,gc}` - at HEAD `e50b400` vs `5bb7211` (last milestone close) vs `de66eb7` - (immediate pre-iter): **0 differing bytes, all 5 binaries identical - across both baselines.** `ail` release binary sha256 identical at - all three commits (zero compiled-source delta — the sole `crates/` - change is the non-bench test file). -- Latency re-measure on the byte-identical binary (5×`--runs 5`): - median/p99 rock-steady (≤0.6% / 1.1% spread); p99.9 swings - 503.8→651.4, max 565.6→692.1 — classic stable-center / volatile-tail - on identical machine code. = the tracked-P2 `-n 5` tail-jitter - false-positive, **4th occurrence on a zero-runtime-change - milestone**. -- `bench_list_sum_bump` (byte-identical) 20-run median 0.0531s, - persistent ~+15% (not transient) = the tracked-P2 `*.bump_s` - environmental-baseline-staleness item; the 0.046s anchor is stale - vs this machine's current steady state. - -**Verdict: EXONERATED (environmental, do NOT ratify).** Per the audit -Iron Law ("NO BASELINE UPDATE WITHOUT A PAIRED JOURNAL RATIFY ENTRY"; -ratifying noise is forbidden) and the spec's zero-codegen invariant, a -`--update-baseline` is foreclosed. Neither firing family is -attributable to docs-honesty-lint. - -## Step 4 — Resolution: carry-on (clean) - -No fix iteration. No ratify. Milestone **CLOSED**. No fieldtest: this -milestone changed zero authoring surface (documentation prose + one -non-bench test file + an agent-def + a bench shell sweep) — the -fieldtest trigger (surface-touching milestone) does not apply; the -wrap-robust `docs_honesty_pin.rs` (4 tests) + Sweep 5 ARE the -milestone's regression coverage. - -Two pre-existing standing P2 items remain orchestrator-owned and -were explicitly NOT bundled into this close (the bencher's -recommendation, honoured): (a) the stale `*.bump_s` anchor — a -deliberate quiescent-machine baseline refresh, separately journaled -when pursued, never a ratification rider on an unrelated close; (b) -the recurring `-n 5` tail false-positive — the durable fix is a -structural harness change (higher default N or a tail-metric gating -policy), already a tracked roadmap todo, not a bench-output artefact. - -## Milestone close summary - -docs-honesty-lint is fully ratified and CLOSED: spec (grounding-check -PASS 10/10) + plan + single iter + this audit. The canonical docs -(`docs/DESIGN.md`, `docs/PROSE_ROUNDTRIP.md`) now mirror only the -current state, present-tense; the tense+modality discriminator is -codified in DESIGN.md and the architect drift-checks against it via -Sweep 5 + the new bullet; the wrap-robust enumerated pin is the hard -forward gate. The procedural-enumeration design proved itself: the -Task-4 Step-14 catch-all surfaced one genuine extra FIX site (the -Form-B prose-projection bullet) that the spec's illustrative regex -and a frozen orchestrator list would both have missed. diff --git a/docs/journals/2026-05-18-audit-embedding-abi-m1.md b/docs/journals/2026-05-18-audit-embedding-abi-m1.md deleted file mode 100644 index 74e2b59..0000000 --- a/docs/journals/2026-05-18-audit-embedding-abi-m1.md +++ /dev/null @@ -1,155 +0,0 @@ -# audit embedding-abi-m1 — milestone close (CLEAN — carry-on, NO ratify) - -**Date:** 2026-05-18 -**Scope:** `064599e..e406d07` -**Verdict:** Milestone audit **CLEAN**. Architect `clean` (one -`[low]` evidence-adjudicated to carry-on); bench `check.py` exit 1 -**decisively causally exonerated** (21/21 byte-identical bench -binaries vs pre-milestone), `compile_check.py` / `cross_lang.py` -exit 0. No fix iteration, **no baseline ratify**. - -## Commit map - -- `064599e` — pre-milestone HEAD (bench/architect diff baseline). -- `51160e9` spec, `b0bd7ef` plan — docs only. -- `123ae1b` — USER-authored roadmap-P2 ("DESIGN.md → `design/` - role-split"), `docs/roadmap.md` only, path-disjoint — explicitly - out of M1 audit scope (Boss roadmap maintenance, architect told - to skip it; it is not M1 code/doc work). -- `818177d` — iter partial (Tasks 1–3) + a Boss plan Repair - (Design-decision 6, the module≠stem fixture defect). -- `e406d07` — iter completion (Tasks 4–7) + INDEX line. - -## Step 1 — Architect drift review - -**Status: `clean`.** All four focus areas + standing concerns -verified against the `064599e..e406d07` diff: - -- **Invariant 1 (clean core) holds semantically.** No - finance/`data-server` vocabulary, logic, or dependency entered - `ailang-core` / `ailang-codegen` / `runtime/`. `backtest_step` - is an opaque author-chosen string flowing through generic - `FnDef.export`; the only "backtest" occurrences in core/codegen - are test-fixture filenames + mangled-symbol assertions in test - files, not domain knowledge. AILang↔host meeting point correctly - deferred (no `ail-embed`, no runtime lifecycle API) — M1 scope - respected. -- **DESIGN.md honesty rule satisfied.** New §"Embedding ABI (M1)" - + fn-JSON `"export"` line + `FnDef.export` rustdoc are - present-tense current state; "provisional until M3" / "M2 will - change this C signature" are correctly *labelled* reserved- - unimplemented, not stated as present fact, no - post-mortem/Wunschdenken. Sweep hits (DESIGN.md:50/:2060/:62) - all pre-existing, out of M1 range. -- **Lockstep intact.** `FnDef` ↔ DESIGN.md §Data-model fn-JSON ↔ - `design_schema_drift.rs`/`spec_drift.rs` all carry the field. - The `@` forwarder is additive raw-IR emission *beside* the - unchanged `@ail__` mangling — does not route through - `lower_app`, so the `lower_app`/`is_static_callee` and - `Pattern::Lit` lockstep pairs are not engaged. Schema stays - exhaustively-constructed (~85 sites uniform `export: None`, no - `_ =>` / `..Default::default()` — plan Design-decision 1 - honoured). In-source `missing_entry_main_is_error` pin (now - L3411) behaviourally byte-unchanged (only the mandatory field - added). `_adapter`/`_clos` closure pair untouched (spec "Out of - scope"). -- **Design-decision 6 confirmed correct, not re-flagged.** The - gate/headline fixtures' `(module embed_*)` == file-stem; the - divergence from the spec's *illustrative* `(module backtest)` / - `(module bad)` blocks is expected under the loader's - module==stem invariant + spec Decision 2 (C symbol - `backtest_step` decoupled via `(export …)`). The fixtures - *demonstrate* the decoupling rather than violate the spec. - -**Drift / debt — one item, adjudicated:** - -- `[low]` ~85 inserted `export: None,` lines indented to the - struct-literal brace column rather than the field column - (`rustfmt`-divergent struct literals; compiles 611/0, semantics - unaffected). **Orchestrator adjudication → carry-on (NOT a fix, - NOT pending).** Evidence: `cargo fmt --all --check` reports - **1137 fmt-divergent locations tree-wide** (pre-existing - `builtins.rs` / `diagnostic.rs` / dozens of `lib.rs` sites - predating M1). rustfmt is demonstrably **not** a project - convention — the hand-formatted codebase tolerates 1137 - divergences. The architect's premise ("a `cargo fmt` pass would - churn later") is therefore moot: no such gate exists or is - coming, and selectively rustfmt-aligning 85 of 1137 sites would - *impose* a standard the project does not hold, which is noise, - not tidy. This is an evidence-based adjudication of the - architect's finding (premise falsified), not "trivial, ignore - it". If AILang ever adopts a fmt gate that is a tree-wide - decision (1137 sites), unrelated to M1. - -## Step 2 — Bench-regression check - -| Script | Exit | Result | -|---|---|---| -| `bench/check.py` | **1** | 63 metrics; 7 regressed, 2 improved, 54 stable | -| `bench/compile_check.py` | 0 | 24 metrics; 0 regressed | -| `bench/cross_lang.py` | 0 | 25 metrics; 0 regressed | - -**The 7 `check.py` firings:** -`throughput.bench_list_sum.bump_s` +12.27%, -`throughput.bench_hof_pipeline.bump_s` +12.67%, -`throughput.bench_list_sum_explicit.bump_s` +13.74%, -`latency.explicit_at_rc.p99_9_us` +36.13%, -`latency.explicit_at_rc.max_us` +101.43%, -`latency.implicit_at_rc.p99_9_us` +67.32%, -`latency.implicit_at_rc.max_us` +123.57%. - -**Decisive causal exoneration (bencher localisation).** Hypothesis: -M1's `Target::Executable` default leaves the entire bench corpus's -emitted machine code byte-unchanged; the firings are pre-existing -environmental/sampling noise, not M1. - -- **21/21 bench binaries `cmp`-byte-identical** between `064599e` - and `e406d07` (`bench_list_sum_bump` sha `dce18c288904c9f0` at - both commits; every one of the 6 fixtures × 3 modes + 3 latency - arms IDENTICAL, 0 DIFFER). `ail` self-hash differs (expected — - Rust source changed for the embedding ABI) but the *emitted - machine code* for every bench fixture is identical because every - fixture builds an executable via the unchanged `Target::Executable` - default. **Zero codegen/runtime causal mechanism exists.** -- The RC-latency tails reproduce *on the byte-identical oracle - binary itself*, swinging 3× run-to-run (`explicit_at_rc.max_us` - per-run 376→1200 us; `implicit_at_rc.max_us` 562→1742 us) while - median/p99 hold steady — the documented `-n 5` single-sample- - jitter signature. -- Fresh `check.py` re-run: the `*.bump_s` trio → ok, 4 of 7 - original firings → ok, only `explicit_at_rc.max_us` still trips - (+26%, byte-oracle-proven environmental). -- Both firing families map (confirmed, not assumed) onto the two - pre-existing **roadmap-P2** tracked items: `*.bump_s` baseline- - vs-current-hardware staleness, and the RC-latency `*.max_us` / - `*.p99_9_us` `-n 5` structural false-positive (this is the ~4th - zero-runtime-change milestone close on which they fire and - collapse on identical-code re-run). - -**Classification: carry-on, NO baseline ratify.** Per the audit -Iron Law a baseline update requires a paired JOURNAL ratify entry -naming an iter that *intentionally* moved the metric; M1 by -construction moved **zero** default-path bytes (21/21 binaries -identical), so a ratify would bake noise into `baseline.json` — -forbidden. The spec forecloses a zero-codegen baseline bump. The -residual flapping is pre-existing, already roadmap-P2-tracked; the -genuinely next-best action (re-capture `baseline.json` on current -hardware at higher N, or widen single-sample tail tolerances) is a -separate bench-harness iteration, **not** an M1 concern — the two -P2 items remain orchestrator-owned and are NOT bundled into this -close (bencher recommendation honoured, consistent with the -remove-mut-var-assign / docs-honesty-lint closes). - -## Step 3/4 — Resolution - -**carry-on (clean).** No fix iteration, no ratify, no commit beyond -this journal + its INDEX line. Architect clean; the sole `[low]` is -evidence-adjudicated non-actionable; bench firings decisively -non-causal (21/21 byte-identical binaries) and mapped to existing -P2 noise. Both bench gates that can pass on the merits do -(`compile_check`/`cross_lang` exit 0; `check.py` exit 1 causally -exonerated). - -**M1 milestone audit CLEAN.** Next pipeline step: `fieldtest` — -M1 added user-visible authoring surface (`(export "")`), so -the surface-touch fieldtest gate applies before milestone close. diff --git a/docs/journals/2026-05-18-audit-embedding-abi-m2.md b/docs/journals/2026-05-18-audit-embedding-abi-m2.md deleted file mode 100644 index ece7877..0000000 --- a/docs/journals/2026-05-18-audit-embedding-abi-m2.md +++ /dev/null @@ -1,106 +0,0 @@ -# Audit — Embedding ABI — M2 (milestone close) - -**Date:** 2026-05-18 -**Scope:** `2db737c..fbeeade` (M2 = spec `1c58055` → plan `b3388c8` → -iter `c9a84b3` PARTIAL 4/9 + Boss spec-defect repair → iter `fbeeade` -DONE 5–9). HEAD `fbeeade`, working tree clean at audit time. -**Outcome:** DRIFT (one `[medium]` + coupled `[low]` doc-honesty -item → fix via tidy iteration). Bench `check.py` exit 1 — **all -firings causally exonerated as the two tracked roadmap-P2 noise -families; NO baseline ratified, NOT an M2 regression.** - -## Step 1 — Architect drift review - -Status: `drift_found`. No DESIGN.md/spec-contract drift, no -unrequested extras, Invariant 1 intact, internal-convention -byte-invariance held, Decision-10 atomicity note present-tense- -correct (sanitiser-verified this iter), spec→DESIGN.md→code lockstep -complete, `docs_honesty_pin.rs` green unmodified, the Boss -spec-defect repair correctly reflected in both spec and shipped -tests. - -Actionable drift (one item, two coupled sites): - -- **`[medium]` `docs/DESIGN.md:2266`** — the section header reads - `## Embedding ABI (M1)` while the body (`:2285–2295`) now - describes the post-M2 ctx ABI as present-tense current state - ("Every exported entrypoint takes a mandatory leading - `ailang_ctx_t*` (M2)… the ctx-threaded C signature is the M2 - shape"). Current-state-mirror honesty mismatch: a header that - misdescribes its own present-tense body. Not a Sweep-5 catch (the - body prose itself is correctly present-tense — the - docs-honesty-lint machinery does not auto-flag a header/body - desync; only a read catches it). The iter journal's Concerns + - Known-debt flagged it accurately and truthfully — Task 7's edit - regions were *deliberately* scope-limited (provisional-until-M3 - sentence + Decision 10) to preserve the `docs_honesty_pin.rs:135` - verbatim "Export parameters are written **bare**…" assertion 12 - lines below the header. Deferral reasoning sound; the mismatch is - real drift that must not persist past this milestone close. -- **`[low]` `docs/DESIGN.md:2354`** — schema-block cross-reference - comment still reads `see §"Embedding ABI (M1)"`; stale xref - coupled to the header above. Must move in lockstep with any - header rename so the fix is scoped header + xref + - `docs_honesty_pin.rs:134–136` `(M1)`-token audit together, not - the header alone. - -Architect recommendation: one scoped tidy iteration. - -## Step 2 — Bench regression - -| Script | Exit | -|---|---| -| `bench/check.py` | **1** | -| `bench/compile_check.py` | 0 (clean — 24/24 stable) | -| `bench/cross_lang.py` | 0 (clean — 25/25 stable, **every `rc_over_c` ratio within tol**) | - -`check.py` 63 metrics: 3 regressed, 2 improved-beyond-tol, 58 stable. -The 3 firings (verbatim): `throughput.bench_list_sum.bump_s` -0.046→0.052 +13.57% (tol 10%); `latency.explicit_at_rc.max_us` -413.0→751.5 +81.96% (tol 25%); `latency.implicit_at_rc.max_us` -477.3→661.4 +38.57% (tol 30%). Every `*.rc_s`/`*.gc_s` throughput and -every rc-latency median/p99/p99.9 stable within tol. - -Because M2 genuinely changed the benched runtime path -(`runtime/rc.c`: the two RC-counter sites gained a `__thread` load + -null-branch ahead of `g_rc_*++`), the byte-identical exoneration -method did **not** apply — `ailang-bencher` was dispatched for -hypothesis-driven localisation (H0 = real M2 regression; H1 = -tracked-P2 noise, M2 bench-neutral). - -**Bencher verdict: H0 refuted; all three firings -tracked-noise-causally-exonerated.** Method: swapped *only* `rc.c` -between `2db737c` and `fbeeade` (same `ail` binary, same fixtures), -disassembled the delta, interleaved core-pinned A/B. -- `bump_s`: causally **impossible** from M2 — `bump_malloc` - disassembly has 0 refs to `__ail_tls_ctx`/rc counters; the - M2-inert bump binary independently reproduces ~+12% vs the stale - 0.046 baseline. Roadmap-P2 `*.bump_s` environmental staleness. -- M2's rc.c hunk compiled **branchless** (3 insns: `mov %fs`, - `test`, `cmovne`) and measured **free** (HEAD median −0.63% vs - pre-M2, inside intra-arm spread) — the H0 mechanism does not - exist; no per-alloc cost. -- `*.max_us`: median/p99 flat across rc.c variants (a systematic - per-alloc cost must move them and does not); max_us - non-reproducible within a single fixed binary (3.2× spread). - Roadmap-P2 `*.max_us` `-n 5` structural false-positive (now its - 4th consecutive no-runtime-change-attributable firing). - -Limitations (bencher-stated): single host; N=15 moderate for -max-tail — median/p99 verdict solid, max verdict rests on the -(strong, N-bounded) non-reproducibility argument; gc latency arm not -re-measured (no gc firing; rc/gc share the identical M2 hunk). - -## Step 3 — Classification & resolution - -| Item | Class | Resolution | -|---|---|---| -| `[medium]` DESIGN.md:2266 stale `(M1)` header + `[low]` :2354 xref | **fix** | one tidy iteration `embedding-abi-m2.tidy`: rename the `## Embedding ABI (M1)` header to drop the milestone tag (current-state-mirror), lockstep the `:2354` cross-reference and the `docs_honesty_pin.rs:134–136` comment/assert-message `(M1)` token — the pinned *substring* (`:135`, the "Export parameters are written **bare**…" sentence) stays verbatim untouched. planner → implement. | -| `check.py` exit 1 — 3 firings | **carry-on** | causally exonerated as the two pre-existing roadmap-P2 noise families (`*.bump_s` staleness; `*.max_us` `-n 5` false-positive). **No baseline ratified** (bencher wrote nothing to baseline; these are not M2 artefacts). The P2 items remain tracked on the roadmap; retiring the `bump_s` stale baseline is a separate roadmap-P2 action against that item, explicitly NOT this milestone. | -| `compile_check.py` / `cross_lang.py` | carry-on | clean (exit 0). | - -**Milestone M2 is substantively closed and sound** (capability -sanitiser-verified, all contract invariants held). The one open -drift is a doc-honesty header desync, fixed by the tidy iteration -dispatched next; the roadmap `[~]`→`[x]` flip follows the tidy -landing clean. No ratify. No re-audit deferral. diff --git a/docs/journals/2026-05-18-audit-embedding-abi-m3.md b/docs/journals/2026-05-18-audit-embedding-abi-m3.md deleted file mode 100644 index e23d7dc..0000000 --- a/docs/journals/2026-05-18-audit-embedding-abi-m3.md +++ /dev/null @@ -1,136 +0,0 @@ -# audit Embedding ABI — M3 (milestone close) - -**Date:** 2026-05-18 -**Milestone range:** `9a609ae..4ea8bc5` (M2 roadmap-close → M3 DONE) -**Outcome:** DRIFT — one `[medium]` + one `[low]` doc-honesty item -→ a single docs-honesty tidy iteration; bench `check.py` exit 1 -**decisively causally exonerated** (byte-identical generated IR), -NO ratify. compile_check / cross_lang clean. - -## Step 1 — Architect drift review (`drift_found`) - -Dispatched `ailang-architect` over `9a609ae..HEAD`. - -**What holds (audited clean):** -- **Invariant 1 (clean core):** zero `Cargo.toml`/runtime dep - changes; `ailang-core`/`ailang-codegen`/`runtime/` touches are - only the spec-scoped widen + lockstep comments + pins. No - `data-server`/finance knowledge entered. -- **M1/M2 internal-convention byte-invariant held:** the M2 - `Target::StaticLib` forwarder body and the internal - `@ail__` / `_adapter` / `_clos` convention are *absent - from the diff* (untouched). The `llvm_scalar` / `is_c_abi_type` - widen is exactly the spec scope — no new `CheckError` variant, no - schema field, no Form-A change (M2-style honoured). -- **Frozen-layout SSOT consistent:** DESIGN.md - `### Frozen value layout` (`:2321-2359`) is present-tense and - matches both `match_lower.rs` `lower_ctor` (`size = 8 + n*8`, - tag@0, fields@8+i*8) and the enforcing byte-pin - (`@ailang_rc_alloc(i64 24)`, tag@0, @8/@16). All three lockstep - `// FROZEN ABI` pointers present and aimed at the SSOT. The Boss - spec-consistency repair note + §"Testing strategy" 3/4 + the - journal Boss-adjudication / Re-dispatch sections are mutually - consistent and honest about the M2-TLS cross-attribution (correct - behaviour, not a leak). - -**Drift / debt:** -- **`[medium]` `docs/DESIGN.md:2299-2302`** — a surviving M1-era - paragraph still asserts modes "apply only to heap-shaped types, - which the **scalar-only rule above forbids** at an export - boundary anyway". `:2280-2284` was rewritten to accept a - single-ctor record, so there is no longer a scalar-only rule; - this sentence now contradicts the section's own freeze and the - `(own (con State))`/`(borrow (con State))` mode contract at - `:2345-2351`. Canonical-spec internal contradiction the milestone - owned. → **fix** (docs-honesty tidy). -- **`[low]` `crates/ailang-codegen/src/lib.rs:608-610`** — the - forwarder-body comment still says the gate "guarantees every - param and the ret are `Int`/`Float`; map Int→i64, Float→double", - now false (records→ptr). The architect classified this "carried - debt, not a scope miss" — the forwarder *body* is contractually - byte-frozen, so the comment was genuinely outside any M3 edit - region. Comment-only; correcting it changes no emitted IR (the - byte-pin + forwarder-IR pin assert IR, not source comments). → - **fix**, folded into the same tidy (doc-honesty hygiene; the IR - pins are the guard that generated code stays byte-frozen). - -(Boss already caught + fixed in-line one sibling stale `//` -gate-comment block at `check/lib.rs:1917-1922` during the DONE -inspect; the architect found no further siblings.) - -## Step 2 — Bench-regression trio - -| script | exit | summary | -|---|---|---| -| `bench/check.py` | **1** | 63 metrics; 3 regressed, 2 improved, 58 stable | -| `bench/compile_check.py` | 0 | 24 metrics; 0 regressed, 24 stable | -| `bench/cross_lang.py` | 0 | 25 metrics; 0 regressed, 25 stable (every `rc_over_c` within tol) | - -`check.py` exit 1 — the 3 firings: - -| metric | baseline | actual | diff | tol | -|---|---|---|---|---| -| `throughput.bench_list_sum.bump_s` | 0.046 | 0.052 | **+12.62%** | 10.0% | -| `latency.implicit_at_rc.p99_9_us` | 452.0 | 580.6 | **+28.45%** | 25.0% | -| `latency.implicit_at_rc.max_us` | 477.3 | 645.2 | **+35.18%** | 30.0% | - -Both are the two **standing tracked P2 noise families** (roadmap -P2): the `*.bump_s` stale-baseline (~+5–13% environmental band on -current hardware) and the `*.max_us`/`p99_9` `-n 5` tail-jitter -structural false-positive (signature: same benchmark's `median_us` -+5.11% ok, `p99_us` +15.08% ok — only the extreme tail quantiles -fire, the worst-of-~5000-samples-over-5-runs pattern). - -**Causal exoneration (decisive, not hand-waved — M2-audit method).** -M3 changed *no* executable-path codegen (architect-confirmed: M2 -forwarder body byte-unchanged; the only codegen touches are the -`llvm_scalar` record→ptr arm — reachable solely for an `(export)` -single-ctor record, and no `bench_*` program has an `(export)` — the -`ailang-check` `is_c_abi_type` widen, and lockstep *comments*). Built -`ail` at HEAD and at the pre-M3 commit `9a609ae` in an isolated -worktree; the **generated IR is byte-identical** (`cmp -s`) for every -firing benchmark and its neighbours: - -``` -bench_list_sum: IR BYTE-IDENTICAL (HEAD == 9a609ae) -bench_list_sum_explicit: IR BYTE-IDENTICAL -bench_latency_implicit: IR BYTE-IDENTICAL -bench_tree_walk: IR BYTE-IDENTICAL -``` - -The IR is what is handed to `clang -O2` (deterministic), so the -bench binaries are byte-identical and M3 causation is logically -impossible. **NO baseline ratify** (these are not M3 artefacts; the -two P2 items stay separately tracked). Bench → **carry-on**. - -## Step 3 — Classification - -| item | class | recommendation | -|---|---|---| -| `[medium]` DESIGN.md:2299-2302 scalar-only-rule contradiction | drift | **fix** — docs-honesty tidy | -| `[low]` codegen forwarder-comment Int/Float-only staleness | drift | **fix** — folded into the same tidy | -| `check.py` exit 1 (3 firings) | tracked-noise | **carry-on**, NO ratify (byte-identical-IR causal exoneration) | -| compile_check / cross_lang | clean | carry-on | - -## Step 4 — Resolution (Boss) - -- **Drift → one docs-honesty tidy iteration `embedding-abi-m3.tidy`** - (the M2-audit `[medium]+[low] doc-honesty → tidy` precedent - shape): reconcile the stale DESIGN.md:2299-2302 "scalar-only rule - above forbids" paragraph against the frozen SSOT + the own/borrow - mode contract, **pin-safe** (the `docs_honesty_pin.rs:135` pinned - sentence — "Export parameters are written **bare**…", shifted to - ~`:2299` by the M3.1 DONE — sits in/adjacent to the edit zone; - the tidy plan must keep it byte-verbatim+contiguous, planner - Step-5 item-6 / M2.tidy precedent); plus the `[low]` codegen - forwarder-comment honesty fix (comment-only, the byte-pin + - forwarder-IR pin stay green as the guard that generated code is - untouched). Routed via `planner` → `implement`. -- **Bench → carry-on, NO ratify.** Causally exonerated; the two - tracked P2 noise todos stay as-is. - -Milestone is substantively closed + sound (record crosses the C ABI -in/out, ownership follows the declared mode, the value layout is -frozen + byte-pin-enforced, all contract invariants held, -Invariant 1 clean). The roadmap M3 `[~]`→`[x]` follows the tidy -landing clean. diff --git a/docs/journals/2026-05-18-audit-prose-loop-binders.md b/docs/journals/2026-05-18-audit-prose-loop-binders.md deleted file mode 100644 index ea35322..0000000 --- a/docs/journals/2026-05-18-audit-prose-loop-binders.md +++ /dev/null @@ -1,91 +0,0 @@ -# audit prose-loop-binders — Milestone-prose-loop-binders tidy (clean) - -**Date:** 2026-05-18 -**Milestone:** prose-loop-binders (single-iteration) -**Commit range:** `6cbd0fe..c9355d7` (prev close = loop/recur at `6cbd0fe`) -**Status:** CLOSED — clean, carry-on (no fix, no ratify) - -## Step 1 — Architect drift review - -`ailang-architect` over `6cbd0fe..HEAD`: **clean**, zero drift, zero -debt. Verified against the diff (not journal trust): - -- Single hunk in `crates/ailang-prose/src/lib.rs` (the `Term::Loop` - arm of `write_term`, lines 936–953). The two non-render - `Term::Loop` sites (`count_free_var`, `subst_var_with_term`) and - all `Term::Recur` arms are absent from the diff — untouched. -- DESIGN.md and PROSE_ROUNDTRIP.md make **no** claim about how - prose renders `loop` (grep: no match), so the spec's "no doc - edit needed" is correct, not a skipped obligation. `DESIGN.md:714` - `loop(n-1)` is an ordinary recursive-call illustration in - Decision 8, unrelated to the `Term::Loop` construct — not stale. -- Diff touches only the four named code/fixture files plus expected - docs (spec/plan/journal/INDEX/roadmap/stats). No carve-out / - hash-pin / round-trip lockstep bypassed (prose is off the - Form-A↔JSON round-trip path by construction). -- The `architect_sweeps.sh` Sweep-1 DESIGN.md hits all pre-date - `6cbd0fe` and are outside this milestone's diff (zero DESIGN.md - lines changed) — flagged for the record, not attributable here. - -Architect recommendation: carry on as planned. - -## Step 2 — Bench-regression check - -- `bench/compile_check.py` → **exit 0**, 24 metrics, 0 regressed. -- `bench/cross_lang.py` → **exit 0**, 25 metrics, 0 regressed. - Notably its independent `bench_list_sum.ail_bump_s` reads +1.50% - (±15% tol) → **ok** — corroborates that check.py's bump_s firing - is baseline-staleness, not a real bump regression. -- `bench/check.py` → exit 1. First run: 3 regressed - (`bench_list_sum.bump_s` +13.17%; `latency.implicit_at_rc.p99_9_us` - +37.30%; `latency.implicit_at_rc.max_us` +114.14%), 4 improved, - 56 stable. Confirmatory identical re-run (no code change): - `p99_9_us` +10.22% → **ok**, `max_us` +22.17% → **ok** (median - and p99 within tol on both runs) — the two latency-tail firings - were single-sample 5-run RC-mode jitter, proven by re-run. - `bench_list_sum.bump_s` persisted at +11.79% (consistent with - the +13.17% first run): summary `63 metrics; 1 regressed`. - -## Step 3 — Classification - -| Item | Class | Rationale | -|------|-------|-----------| -| Architect drift | carry-on | clean — nothing actionable | -| `throughput.bench_list_sum.bump_s` +~12% | carry-on | The already-tracked P2 roadmap todo "`*.bump_s` throughput baseline is stale vs current hardware" — byte-oracle-proven environmental (not codegen) at the 2026-05-16 iteration-discipline-revert audit. Causally impossible to attribute to a projection-only `ailang-prose` change (prose is not in the bench runtime path; architect verified the diff is one render-arm + test fixtures). | -| `latency.implicit_at_rc` p99.9 / max | carry-on | Single-sample 5-run RC-mode tail jitter; vanished on identical re-run with median/p99 always within tolerance. Not a regression; the queued P3 "latency methodology rework" (histogram-based) exists precisely to de-noise this metric family. | - -## Step 4 — Resolution - -**Carry-on** on every item. No fix iteration, no baseline ratify. - -Explicitly **not ratified**: the `*.bump_s` baseline is NOT bumped -here. The P2 roadmap todo scopes a holistic recapture+recalibration -of the entire `*.bump_s` set on current hardware from a known-clean -commit as a dedicated bench-harness iteration; opportunistically -bumping one metric's baseline inside an unrelated projection-only -milestone is exactly the "bump the baseline, the regression is -expected" rationalisation the audit Iron Law forbids, and would -fragment the recapture the P2 todo owns. The pointer stands; the -todo is unchanged. - -## Fieldtest applicability - -**Not applicable / not dispatched.** `fieldtest` is Boss-dispatched -for a milestone that touched the *authoring* surface — the -fieldtester writes `.ail` Form-A programs and runs them through the -public `ail` CLI. This milestone changed only the Form-B *reading* -projection (`ail prose` output); the Form-A authoring surface, the -typechecker, codegen, and runtime are byte-unchanged. A fieldtester -authoring `.ail` programs would not exercise the prose-render change -at all — there is no friction/bug/spec-gap axis for it to probe. -The two committed whole-file byte-equality `.prose.txt` snapshots + -the live-CLI `ail prose | diff` acceptance checks (iter journal) are -the end-to-end gate for a projection-only milestone. - -## Milestone close - -The **prose-loop-binders** milestone is **CLOSED, clean**. Architect -drift clear; bench carry-on (sole persistent firing is the tracked -P2 `*.bump_s` environmental staleness, latency-tail proven sampling -noise by re-run). Roadmap P0 entry flipped to closed; no follow-up -iteration spawned. diff --git a/docs/journals/2026-05-18-audit-remove-mut-var-assign.md b/docs/journals/2026-05-18-audit-remove-mut-var-assign.md deleted file mode 100644 index 1415b36..0000000 --- a/docs/journals/2026-05-18-audit-remove-mut-var-assign.md +++ /dev/null @@ -1,108 +0,0 @@ -# audit — remove-mut-var-assign milestone close - -**Date:** 2026-05-18 -**Scope:** `48e7774..07f0802` (prev close → the single terminal iter) -**Outcome:** CLEAN after one inline `.tidy` fix; bench causally -exonerated; two P2 roadmap todos filed/strengthened. Fieldtest -remains (surface-touching milestone). - -## Step 1 — Architect drift review - -**`[high]` — `crates/ailang-core/specs/form_a.md` (EBNF 286-288, -prose 308-328, dangling `like mut` at 332).** The live in-tree -Form-A grammar reference still documented the full `mut`/`var`/ -`assign` construct. The spec (Components #2, Architecture table -row 2) explicitly named "form_a.md" for deletion; the -schema-vs-doc honesty invariant the milestone advertises was -broken. **Root cause = the recon-undercount class (4th -recurrence), and specifically a Boss error:** at plan time the -Boss grepped `docs/form_a.md` (dir-scoped), got "No such file", -concluded "form_a.md does not exist", and propagated that -unverified non-existence claim into the recon brief. The file -lives at `crates/ailang-core/specs/form_a.md` and is actively -maintained (form-a.tidy edited it) — not frozen historical -record. A compile-driven sweep cannot catch this: a stale doc -never produces `E0004`/`E0061`. - -**Resolution — inline Boss `.tidy`, not planner+implement.** -Rationale (CLAUDE.md "When NOT to delegate": context already -loaded, an agent would redo all milestone reading for a ~40-line -single-file doc deletion whose design was already settled by the -approved spec): deleted the three EBNF productions (keeping -`reuse-as`→`loop`→`recur`), the three mut/var/assign prose -bullets, and reworded the surviving `loop` bullet so its -"right-folded into `Term::Seq`" no longer back-references the -deleted `mut`. Full review-and-commit discipline kept (no -discipline shed for the carve-out): verified tree-wide that the -only remaining `Term::Mut`/`Term::Assign`/`MutVar` strings in -`crates/` + `docs/DESIGN.md` + `docs/roadmap.md` are the roadmap -entry *describing the milestone itself* (not residue), and -form_a.md's two remaining `mut`/`var` substrings are the unrelated -`[unbound-var]` / `reuse-as-source-not-bare-var` diagnostic names. -Grammar + notes read coherently after the cut. - -**`[medium]` — process-drift, recon-undercount 4th recurrence.** -The existing loop-recur.tidy P2 (`ailang-plan-recon` -cross-crate-caller-undercount) is demonstrably insufficient: the -class now recurs through a *new* site type each time (test bodies, -drift pins, carve-out files, and now a spec-named non-compiled -doc + an unverified existence claim feeding the recon brief). -Escalated that P2 in place — broadened from "cross-crate caller" -to a structural recon-contract defect with two named gaps -(non-compile-checked spec-named sites; unverified "X does not -exist" claims inheriting into a recon brief), and a concrete -agent-definition fix (compile-driven enumeration as authoritative -code-site set **plus** an `ls`-verified spec-named-path existence -table). Not a code defect; an agent-discipline fix, parked P2. - -## Step 2 — Bench-regression - -`bench/check.py` **exit 1** (3 firings); `compile_check.py` -**exit 0** (24/24 stable — no check-time tax, consistent with a -pure removal); `cross_lang.py` **exit 0** (25/25 stable, incl. the -independent `bench_list_sum.ail_bump_s` +2.95% ok). - -check.py firings: `throughput.bench_list_sum.bump_s` +12.51%; -`latency.explicit_at_rc.max_us` +108.38%; -`latency.implicit_at_rc.max_us` +66.46%. All medians/p99 stable. - -**Bencher localisation — causal exoneration (decisive).** Built -`ail` at HEAD and at pre-milestone `48e7774`; the three -firing-relevant bench binaries (`list_sum_bump`, `lat_expl_rc`, -`lat_impl_rc`) are **`cmp`-byte-identical** across the milestone. -The −2520-line removal diff does not alter one byte of these -fixtures' machine code (none contain `mut`); no causal mechanism -exists. By elimination every firing is non-code-caused. - -Per-firing verdict: `bump_s` = the tracked P2 environmental -staleness (stable ~+11–14% across 4 runs, byte-identical binary, -cross_lang corroborates); both `*.max_us` = single-sample `-n 5` -jitter (`explicit_at_rc.max_us` collapsed +108%→**-2.30% ok** by -an identical-code rerun3; `implicit_at_rc.max_us` swings -477→1172→843→756 on the identical binary while its median holds -+3.1…3.9%). - -**Resolution: PRISTINE met on the merits. NO baseline ratify.** -The Iron Law forbids ratifying noise into the baseline and the -spec explicitly forecloses opportunistic ratification here; the -non-pristine exit code is bench-harness noise, not a code defect, -and does not block close. Filed a **new, distinct P2** (separate -from the `*.bump_s` staleness P2): `check.py` RC-latency -`*.max_us` at `-n 5` is a structural false-positive — 3-for-3 on -consecutive no-runtime-change milestones — recommending drop -`max_us` from gating / raise `-n` / cpuset-pin (mitigation choice -left to a future iteration). - -## Step 3/4 — Classification & resolution - -| Item | Class | Outcome | -|------|-------|---------| -| form_a.md construct doc | fix | inline Boss `.tidy` (this commit) | -| recon-undercount 4th recurrence | process-drift | P2 escalated in place (roadmap) | -| `bump_s` +12.51% | carry-on | tracked P2 (no ratify) | -| `*.max_us` ×2 | carry-on | new P2 filed (no ratify) | - -Milestone audit **CLEAN** post-tidy. Next pipeline step: -`fieldtest` (the removal is a user-visible Form-A surface change — -`mut`/`var`/`assign` no longer lex; an LLM author must now reach -for `let`/`if`/`loop`/`recur`). diff --git a/docs/journals/2026-05-18-brainstorm-embedding-abi-m4-retired.md b/docs/journals/2026-05-18-brainstorm-embedding-abi-m4-retired.md deleted file mode 100644 index 9ff504b..0000000 --- a/docs/journals/2026-05-18-brainstorm-embedding-abi-m4-retired.md +++ /dev/null @@ -1,122 +0,0 @@ -# 2026-05-18 — brainstorm embedding-abi-m4 → RETIRED (never speced) - -`/boss` picked the top P0 item "Embedding ABI — M4: sequence -crossing via the existing `List` ADT" (a new milestone, no spec). -User green-lit a continue-here brainstorm. The brainstorm -terminated **without a spec** at the Step-2/3 Q&A: the milestone's -premise collapsed under its own feature-acceptance gate. This file -records why, so the retired entry is not silently erased. - -## What was explored before the collapse - -Plan-recon (`ailang-plan-recon`) returned a full fact sheet. Two -user forks were resolved in Q&A and are recorded here as context -(now moot, but they shaped the analysis): - -- **own-only** for the `List` parameter — kernel consumes each - chunk; its existing internal recursive drop frees it; no - host-side recursive-free symbol (M3 explicitly deferred that as - the build-ahead trap it refused). Keeps M4 "no language change". -- **`List Record` only** — element an M3-shaped single-ctor - all-scalar record crossing as ptr-to-M3-box, one uniform - cons-cell layout; sidesteps the poly/mono inline-scalar-element - divergence the recon flagged (open question #3). - -Approach **A (structural recognition)** — extend `is_c_abi_type` -with a recursive list-shaped arm (2 ctors: one nullary + one -`(C-ABI, recursive-self)`), no name match, consistent with M3's -structural single-ctor rule and AILang's content-addressing -thesis — was the recommendation. B (name-anchored) rejected on -language grounds (name fragility; no canonical `List` — 8 -divergent decls, no `std/`); C (land a `std_list` SSOT first) -rejected on scope grounds (a separate milestone; breaks the -established self-contained embed-fixture discipline). - -## Why M4 was struck (the substantive reason — feature-acceptance clause 2) - -The user asked "what is M4 even *for*?" then "but the host has to -build the list first, no?" then narrowed: a *minimal* data-server -binding would be satisfied by the host pushing each tick -individually — no list at all. - -Held against `docs/DESIGN.md` §"Feature-acceptance criterion": - -- **Clause 2 (improves correctness / removes redundancy) — FAILS.** - The shipped M3 export gate `is_c_abi_type` - (`crates/ailang-check/src/lib.rs:1934-1953`) runs as a **per- - parameter loop** (`for p in ¶m_tys { if !is_c_abi_type(p, - &env) … }`) accepting, *independently per param*, a C scalar OR - a single-ctor all-scalar record. The forwarder's `llvm_scalar` - maps every non-scalar `Type::Con` → `ptr` and passes it - straight through (M3-frozen). Therefore `(State, Tick) -> State` - with both as single-ctor all-scalar records is **already - gate-accepted and forwarder-supported today** — the minimal - data-server binding is M3 (shipped) + a host-side per-tick loop - over each chunk's records. Cons-list crossing would *add* a - 2N+1-box-per-chunk host builder and pull in the deferred - flat-array perf debt (the cons-list IS the P2-deferred perf - problem), removing no redundancy. -- **No named consumer.** M5's adapter (the sole data-server↔AILang - meeting point, Invariant 1) can wire `while let next_chunk` by - unrolling each chunk host-side into per-tick calls — a clean - adapter, not a layering violation; the adapter is exactly where - host data shape meets the kernel. Whole-chunk in-kernel - visibility buys nothing: `State` threads across calls regardless - of chunk boundaries, so a windowed indicator behaves identically - whether ticks arrive one-per-call or chunk-per-call; chunk - boundaries are a host artefact, semantically invisible to the - fold. - -M4 (cons-list crossing) is therefore **speculative infra by -AILang's own criterion**: it builds something no named consumer -needs and adds cost rather than removing redundancy. The -brainstorm discipline at this point is *not* to ratify a known- -unneeded shape into the roadmap (Common Rationalisations: -"Approaches A,B,C — proceed with A" → surface the mis-framing), -so no spec was written, no grounding-check dispatched, no planner -handoff. - -## Honest correction (lesson) - -Mid-Q&A I asserted "M5 cannot wire the real data-server loop -without M4." That was wrong and I corrected it on the record: M5 -*can* wire it with M3 + a host-side `for tick in chunk` loop. The -earlier "O(records) vs O(chunks) FFI calls" framing was also -misleading — the per-tick forwarder is thin; the expensive thing -in the M4 alternative is the cons-list marshalling the scalar loop -does not have at all. Re-deriving against the code (not defending -the roadmap I had written) is what surfaced the clause-2 failure; -this is the "user suggestions ≠ directives, form own judgment" -discipline working in the user's favour. - -## Decision / outcome - -- **M4 retired** in `docs/roadmap.md` (kept one cycle as a struck - entry for context, then delete — never `[x]`, never shipped). -- **M5 reconciled**: `depends on:` M4 → M3 (+ the Tick-coverage - todo); body now states the adapter unrolls each chunk host-side - into per-tick `(State, Tick) -> State` calls; the friction - harvest now feeds the host-per-tick-FFI vs. batch-crossing perf - decision (the P2 flat-array item) rather than an array-vs-cons - decision. -- **Residual = a `[todo]`**: Tick-coverage on M3 — an E2E + - fixture pinning the two-record-param per-tick crossing - `(State, Tick) -> State` on the shipped M3 ABI. Capability is - present today but only E2E-proven for a *single* record param - (`State`): every shipped M3 fixture - (`embed_backtest_step_record{,_borrow}.ail`, - `embed_export_record_ok.ail`) pushes a scalar `Float` sample, - none a record `Tick`. A second record param is the identical - mechanism (same gate arm, same `→ptr`, same forwarder, zero new - code path) — worth pinning by fixture, but a test backfill, not - a milestone (no brainstorm). This is the actual "minimal - data-server binding" the user asked for. - -## Forward note (not edited now — avoid scope creep) - -The P2 "Flat array/slice primitive — performance" item's framing -("1024 RC cons-cells per chunk on the hot path") is now partly -stale — the cons-list path is dropped, so the residual perf -question is host-side per-tick FFI vs. an eventual batch crossing. -Flagged here; reconcile that entry's wording when it is picked up, -not now. diff --git a/docs/journals/2026-05-18-iter-bugfix-over-strict-mode-ctor-rebuild-consume.md b/docs/journals/2026-05-18-iter-bugfix-over-strict-mode-ctor-rebuild-consume.md deleted file mode 100644 index e123760..0000000 --- a/docs/journals/2026-05-18-iter-bugfix-over-strict-mode-ctor-rebuild-consume.md +++ /dev/null @@ -1,74 +0,0 @@ -# iter bugfix-over-strict-mode-ctor-rebuild-consume — over-strict-mode FP on ctor-rebuild-from-primitive-fields - -**Date:** 2026-05-18 -**Started from:** a11cb7cc9f912a0d7ee8a56b3f53255bd101e327 -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -The `over-strict-mode` lint spuriously fired `(borrow ...) would -suffice` on `(own (con T))` params whose body destructures `T` into -purely *primitive* fields and then feeds those fields into a -`Term::Ctor` that builds a fresh allocation. The dismantle+rebuild -is a genuine consume of the scrutinee's box (a borrowed value's -payload cannot be taken apart and re-packaged), but the lint's -consume-detection only recognised a *heap-typed* pattern-binder -being moved out — primitive fields are filtered by -`pattern_has_consumed_heap_binder`, so the rebuild was invisible. -This is the false positive behind `embed_backtest_step_tick.ail` -warning on both `st` and `tick`. The fix adds a second recognition -path in `any_sub_binder_consumed_for`: for a `match p` arm that -destructures binders out of `p`'s ctor, if any of those binders -(primitive or not) is mentioned anywhere inside a `Term::Ctor`'s -argument subtrees in the arm body, the param is genuinely consumed -and the lint stays silent. Over-strict-only: a param merely read -(no ctor build) still warns, proven by a negative control. No -existing test changed; doc-honesty corrections applied to the two -stale comments that mis-attributed this defect to nested `match`. - -## Per-task notes - -- iter bugfix-over-strict-mode-ctor-rebuild-consume.1: added - `collect_pattern_binders`-based ctor-rebuild-consume recognition - to `any_sub_binder_consumed_for`'s matched-scrutinee arm loop in - `crates/ailang-check/src/linearity.rs`, plus two pure private - helpers (`ctor_uses_any_binder` — finds a fresh-allocation - `Term::Ctor` reachable in the arm body; `term_mentions_any_binder` - — deep free-var scan of that ctor's args, so the binder reaching - the ctor *through* an expression like `(+ acc px)` is recognised, - not only a bare `Var` arg). Corrected the stale "Known limit" doc - on `any_sub_binder_consumed_for` and the matching "Known debt: - deeply-nested-match-on-sub-binder" block in the module header — - both previously mis-attributed the FP to a nested match; the real - mechanism is the unrecognised ctor-rebuild consume, independent of - match nesting. - -## Concerns - -(none) - -## Known debt - -- `term_mentions_any_binder`'s binder scan is by name and does not - track shadowing. A destructured primitive binder shadowed and - rebound before reaching the ctor would still be counted. This is - not a corpus shape, and the resulting imprecision is strictly in - the over-strict (extra-silence) direction the cause explicitly - permits — never under-strict, so no soundness risk. Documented in - the function's own doc comment; left as-is per the minimal-fix - constraint. - -## Blocked detail - -(not blocked) - -## Files touched - -- crates/ailang-check/src/linearity.rs (+166 / −20): lint-detection - fix + two helpers + two doc-honesty corrections. Check-only; no - codegen / runtime / ABI / schema / DESIGN.md change. - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-bugfix-over-strict-mode-ctor-rebuild-consume.json diff --git a/docs/journals/2026-05-18-iter-docs-honesty-lint.1.md b/docs/journals/2026-05-18-iter-docs-honesty-lint.1.md deleted file mode 100644 index 0b00e16..0000000 --- a/docs/journals/2026-05-18-iter-docs-honesty-lint.1.md +++ /dev/null @@ -1,131 +0,0 @@ -# iter docs-honesty-lint.1 — canonical docs made present-tense-honest, guarded by a wrap-robust pin + Sweep 5 - -**Date:** 2026-05-18 -**Started from:** de66eb7ab2e457bfe7e5dbe2f87c5da15a786cd7 -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -`docs/DESIGN.md` and `docs/PROSE_ROUNDTRIP.md` carried Wunschdenken -(forward intent stated as fact) and non-citation post-mortem / -doc-archaeology narrative. This iter strips both classes, codifies -the honesty discriminator as a `### What this document is — and the -honesty rule it holds itself to` meta-subsection inside DESIGN.md -§"Project ecosystem", and guards the invariant two ways: a -wrap-robust enumerated absent/present regression pin -(`crates/ailang-core/tests/docs_honesty_pin.rs`, 4 tests) and a -wrap-robust advisory Sweep 5 in `bench/architect_sweeps.sh` with all -sweep-count lockstep wording + a new `ailang-architect.md` "DESIGN.md -honesty drift" bullet. Genuine forward intent (LLM tool-use / MCP / -LSP) relocated to `docs/roadmap.md` P3. No language/checker/codegen/ -runtime change — the sole `crates/` change is the additive pin file; -`ail check`/`run`/`build` are byte-unchanged by construction. - -## Per-task notes - -- iter docs-honesty-lint.1.1: created the wrap-robust RED pin - (`docs_honesty_pin.rs`, `norm()` whitespace-collapse helper, 4 - tests). Verified RED for the right reasons (compiles, 4 fail on the - enumerated still-present strings + the absent discriminator). -- iter docs-honesty-lint.1.2: appended Sweep 5 to - `architect_sweeps.sh` + tail "All four"→"All five" + header - "four"→"five sweeps"; added the "DESIGN.md honesty drift" architect - bullet and fixed the sibling "four sweeps"→"five / Sweeps 1-4" - wording. Sweep 5 confirmed firing on the uncorrected tree - (EXIT=1). Concern-1 lockstep shipped whole, no split. -- iter docs-honesty-lint.1.3: inserted the discriminator - meta-subsection. Tree had drifted (a `- **Tests**:` bullet now - follows the Docs bullet, no blank line; closing prose is "When the - language grows…"); anchored on the verbatim closing-prose line per - the plan's own Step-1 "quote it, do not assume" caveat. -- iter docs-honesty-lint.1.4: applied the 13 byte-exact corrections - (each verbatim "current text" re-confirmed by grep first; lines had - drifted +~22 from Task 3). Step-14 catch-all surfaced one genuine - extra FIX site (see Concerns); classified + handled per procedure. -- iter docs-honesty-lint.1.5: stripped the PROSE_ROUNDTRIP tool-use/ - MCP Wunschdenken paragraph (present-tense replacement); appended - the relocated forward-intent idea to roadmap P3 with spec - provenance. Full pin GREEN (4/4). -- iter docs-honesty-lint.1.6: full verification — pin GREEN isolated; - workspace zero failures (609 passed, delta = +4 pin tests only); - sentinel trio (design_schema_drift/spec_drift/schema_coverage) - green; Sweep 5 reduced to the single self-referential L62 KEEP; - build clean; `ONLY THE PIN` scope confirmation. - -## Concerns - -- T3: plan Task-1 pin string `whether the document asserts something - exists, works, or changed that does not` omitted the `**…**` - Markdown bold that plan Task-3 body wrapped around `asserts … - not`. Plan self-review item 3 cross-check missed it. Resolved by - dropping the incidental bold from the inserted body (the pin is the - contract, written first; the emphasis is non-load-bearing). Prose - meaning + present-tense framing unchanged. Plan-internal - inconsistency, not a tree-drift; flagged for planner calibration. -- T4: Step-14 catch-all surfaced one genuine extra Sweep-5 FIX site - not in Steps 1-13 — DESIGN.md Form-B class/instance prose-projection - bullet ("was deferred from milestone 22 entirely; … A future iter - ships the full prose projection if/when …"). The Boss's "zero extra - hits" pre-verification was against the spec's illustrative regex; - the plan's broader Sweep-5 regex correctly catches this soft-wrapped - site. Handled exactly per Step-14: rewritten present-tense (kept the - load-bearing facts — one-way renderer, no parser by design, - placeholder render in `crates/ailang-prose/src/lib.rs`) + matching - absent-string asserts + present-anchor added to the pin. This is the - procedural catch-all working as designed. - -## Catch-all classification (Task 4 Step 14) - -- KEEP: DESIGN.md L62 `- **Forward intent** ("planned / will back / - on the path to / if-when X arrives")` — the discriminator - subsection's own enumerated catalogue of forbidden phrasings; - asserts nothing false, present-tense rule definition. A - self-referential Sweep-5 false-positive (the regex matches the - literal "on the path to" inside the rule's own example list). -- FIX: DESIGN.md L2093 Form-B prose-projection bullet — see Concerns; - before→after recorded there; pin pair added. - -## Known debt - -- (none) — the iter is self-contained; the pin + Sweep 5 are the - forward guard against regrowth. - -## Post-iter sweep output (Task 6 Step 4) - -``` -=== Sweep: 1 (history anchors) === -50: `docs/journal-archive.md` (archived monolith for pre-2026-05-11 history), -2060: 2026-05-13) — combines two `Str` values into a single owned `Str`. - -=== Sweep: 5 (honesty: wunschdenken + non-citation post-mortem) === -62:- **Forward intent** ("planned / will back / on the path to / - -EXIT=1 -``` - -Sweep-1 residue = the two documented deliberate KEEPs (L50 doc-roles -pointer date, L2060 true `(iter str-concat, 2026-05-13)` provenance). -Sweep-5 residue = the single L62 self-referential discriminator- -catalogue KEEP. EXIT=1 is expected + architect-adjudicated per plan -Task 6 Step 4 (deliberate-KEEP date-anchors remain by design). - -## Blocked detail - -(none — DONE) - -## Files touched - -- `crates/ailang-core/tests/docs_honesty_pin.rs` (new — the pin, 4 - tests; extended in T4 Step 14 per the mandated catch-all procedure) -- `bench/architect_sweeps.sh` (Sweep 5 + header/tail lockstep) -- `skills/audit/agents/ailang-architect.md` ("four"→"five" + new - "DESIGN.md honesty drift" bullet) -- `docs/DESIGN.md` (discriminator subsection + 13 enumerated - corrections + 1 catch-all correction) -- `docs/PROSE_ROUNDTRIP.md` (tool-use/MCP Wunschdenken stripped) -- `docs/roadmap.md` (relocated forward-intent idea, P3) - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-docs-honesty-lint.1.json diff --git a/docs/journals/2026-05-18-iter-embedding-abi-m1.1.md b/docs/journals/2026-05-18-iter-embedding-abi-m1.1.md deleted file mode 100644 index afa57a8..0000000 --- a/docs/journals/2026-05-18-iter-embedding-abi-m1.1.md +++ /dev/null @@ -1,474 +0,0 @@ -# iter embedding-abi-m1.1 — Embedding ABI M1: linkable artifact + scalar C entrypoint - -**Date:** 2026-05-18 -**Started from:** b0bd7efeaff137af22bb12f5e751260804e770c3 -(Tasks 1–3 committed at 818177d; Tasks 4–7 re-dispatched from 818177d) -**Status:** DONE — all 7 tasks complete. Tasks 1–3 committed -(818177d); Tasks 4–7 implemented on the corrected plan and sit -unstaged in the working tree. The E2E coherent-stop proof -(`embed_e2e.rs`: `cc` link + run, `s == 25`, exit 0) is GREEN. -**Tasks completed:** 7 of 7 (Tasks 1–3 DONE + committed; Task 4 -initially BLOCKED on a Boss plan-defect — fixture module≠stem — -resolved by Boss plan Repair [Design-decision 6] + re-dispatch; -Tasks 4–7 then completed cleanly on the corrected plan) - -## Summary - -The Embedding-ABI M1 iter is complete across all 5 pipeline -layers. Tasks 1–3 landed in the first dispatch and were committed -(818177d) as a verified known-good subset: the CLI build-path -`MissingEntryMain` baseline pin (Task 1), the additive -`FnDef.export: Option` schema field threaded across the -whole workspace via compile-driven enumeration with a hash-stability -golden pin (Task 2), and the Form-A `(export "")` surface -modifier with full parse/print/grammar lockstep and a byte-identical -round-trip (Task 3). - -Task 4 was BLOCKED in the first dispatch on a Boss plan-defect -(fixtures declared `module ≠ file-stem`; the loader hard-enforces -`module == file-stem` at `workspace.rs:438`). The Boss Repaired the -plan (Design-decision 6: fixture `(module …)` == file stem; the -C symbol stays `backtest_step` via `(export …)`, demonstrating spec -Decision 2's mangling-decoupling) and re-dispatched Tasks 4–7. On -the corrected plan all four landed cleanly: the check-side -scalar-only + effect-free export gate with two new `CheckError` -variants (`export-non-scalar-signature` / `export-has-effects`, the -clause-3 discriminator *in code*, Task 4); the codegen -`Target::StaticLib` mode suppressing `@main`/`MissingEntryMain` and -emitting one external `@` forwarder per export to -`@ail__` (Task 5); the CLI `ail build --emit=staticlib` -two-archive emit (`libembed_backtest_step.a` + `libailang_rt.a`, -Task 6); and the C-host E2E coherent-stop proof — `cc` links both -archives, calls `backtest_step(0,3)=9`, `backtest_step(9,4)=25`, -the `s == 25` assertion holds, exit 0 (Task 7), plus the DESIGN.md -§"Embedding ABI (M1)" current-state subsection. Full workspace -green (0 failures); `roundtrip_cli` green; `compile_check.py` / -`cross_lang.py` exit 0; `check.py` fires only the tracked P2 -`*.bump_s` / `*.max_us` known-noise envelope (plan Step 6: audit- -adjudicated, not a plan-level gate — the default-executable -codegen/runtime path is byte-unchanged). No clean-core dependency -added (Invariant 1). - -## Per-task notes - -- iter embedding-abi-m1.1.1: CLI build-path `MissingEntryMain` - baseline pin. Created `examples/embed_noentry_baseline.ail` - (main-free module) + `crates/ail/tests/embed_missing_main_baseline.rs` - (triple-assertion: non-zero exit + verbatim `has no \`main\` def` - Display substring [codegen lib.rs:129] + absent binary). GREEN on - write; not-a-no-op confirmed; `roundtrip_cli` green with the new - fixture. DONE. -- iter embedding-abi-m1.1.2: additive `FnDef.export` schema field + - workspace-wide threading. Added the field to `FnDef` after `doc` - (verbatim plan doc-comment, `#[serde(default, - skip_serializing_if = "Option::is_none")]`). Compile-driven - enumeration via `cargo build --workspace --all-targets - --keep-going` over two dependency-ordered waves threaded - `export: None` into ~85 `FnDef`/`AstFnDef` literals across 18 - files incl. all named blind-spot sites (codegen lib.rs:3320 - `missing_entry_main_is_error`, design_schema_drift.rs:278, - spec_drift.rs:256, hash_pin.rs, the in-source `#[cfg(test)] - mod tests` literals, the `mono.rs` `AstFnDef` alias, the - `parse.rs`/`print.rs` literals). `cargo build --workspace` - exits 0 (planner self-review item 7 gate met). DESIGN.md - fn-JSON `"export"` line + drift test green (Recon Q2 confirmed: - no nested-struct-key anchor forced). Hash pin green (golden - `b4662aa70839f60b`). Full workspace test gate: 611/0. - DONE_WITH_CONCERNS. -- iter embedding-abi-m1.1.3: Form-A `(export "")` surface - modifier. `parse_export` helper (faithful `parse_doc` mirror); - `parse_fn` threaded (`let mut export`, `Some("export")` arm with - duplicate-clause guard, unknown-attr string updated, `Ok(FnDef{…})` - uses parsed `export`); `write_fn_def` export emission block (doc → - export → suppress → type order); form_a.md production + - `(export STRING)?` + prose paragraph. RED verified - (`unknown fn attribute \`export\``), then GREEN: round-trip gate - byte-identical, surface crate 72/0, ailang-core 111/0. DONE. -- iter embedding-abi-m1.1.4 (FIRST dispatch — BLOCKED, audit - trail): wrote the five plan fixtures + the verbatim gate pin; ran - it → RED for the WRONG reason (a `ModuleNameMismatch` load - failure, not the absence of the gate). BLOCKED — see Blocked - detail + Boss resolution. No gate production code written; the - WIP fixtures + RED test were discarded by the Boss for the - re-dispatch. -- iter embedding-abi-m1.1.4 (RE-dispatch — DONE): check-side - export-signature gate on the corrected plan (Design-decision 6, - module==stem). Recreated the five fixtures (module == file stem; - `embed_export_float_ok.ail` uses `(app * a b)` — no `*.` head) - + the verbatim gate pin `crates/ailang-check/tests/embed_export_gate.rs`. - RED verified for the RIGHT reason (fixtures load cleanly, 4 - `*_rejected` fail because no gate exists, 2 positives already - pass). Added two `CheckError` Error variants - (`ExportNonScalarSignature(String,&'static str,String)` / - `ExportHasEffects(String,Vec)`) after `ConstHasEffects`, - their two `code()` arms, and the gate in `check_fn` after the - param/ret well-formedness checks (gate order params→ret→effects, - first-violation-wins, unconditional on `f.export.is_some()`). - GREEN 6/6; combined fixture fails signature-first. Round-trip + - full workspace green. DONE. -- iter embedding-abi-m1.1.5: codegen `Target::StaticLib`. Added the - `Target` enum next to `AllocStrategy`; threaded `target` through - `lower_workspace_inner`; `lower_workspace_with_alloc` / - `lower_workspace` pin `Target::Executable`; new public - `lower_workspace_staticlib_with_alloc`. Gated the - `MissingEntryMain` shape-check + the `@main` trampoline behind - `Target::Executable`; the `StaticLib` arm emits one external - `define {cret} @(...)` per `(export …)` fn forwarding to - `@ail__` via the new `fn_scalar_sig` / `llvm_scalar` - helpers. RED→GREEN IR-shape pin (no `@main`, has `@backtest_step`, - forwards to `@ail_embed_backtest_step_step`; exe target still - `MissingEntryMain`). In-source `missing_entry_main_is_error` pin - byte-unchanged; codegen crate 0 failures. DONE_WITH_CONCERNS - (the `emit_ir`-routes-via-`lower_workspace` plan note). -- iter embedding-abi-m1.1.6: CLI `ail build --emit=staticlib`. - Added the `--emit` clap field (`value_parser = ["exe", - "staticlib"]`, default `exe`) after `alloc`; branched - `Cmd::Build` (`emit == "staticlib"` → `build_staticlib`); added - `build_staticlib` (verbatim `build_to:2239–2294` shared prefix + - `has_export` guard + `lower_workspace_staticlib_with_alloc` + - two-archive `clang -c`/`ar rcs` tail) and the new shared `run_cmd` - helper (confirmed no pre-existing equivalent). RED→GREEN CLI pin - (`libembed_backtest_step.a` + `libailang_rt.a` produced; - export-less module rejected with the zero-export error). - Default-exe regression (floats E2E + Task-1 baseline) green. - DONE. -- iter embedding-abi-m1.1.7: E2E host harness + DESIGN.md - §"Embedding ABI (M1)". Wrote `crates/ail/tests/embed/host.c` - (spec headline host, verbatim) + `crates/ail/tests/embed_e2e.rs`. - E2E GREEN — `cc` links `libembed_backtest_step.a` + - `libailang_rt.a`, runs the host, `backtest_step(0,3)=9` / - `backtest_step(9,4)=25`, `s == 25` assertion holds, exit 0 (the - milestone coherent stop). Added the DESIGN.md §"Embedding ABI - (M1)" subsection (verbatim prose, at `##` level for consistency - with §"Mangling scheme"). Drift tests 8/8 + 8/8; full workspace - 0 failures; bench trio per plan Step 6. DONE_WITH_CONCERNS (the - dead-`Path`-import correction + the `###`-vs-`##` heading-level - resolution). - -## Concerns - -- iter embedding-abi-m1.1.2: the plan's RED-pin symbol - `ailang_core::hash::content_hash_fn(&FnDef)` does not exist. The - real entry point is `def_hash(&Def) -> String` (the one - `crates/ailang-core/tests/hash_pin.rs` uses). Resolved by - mirroring `hash_pin.rs` exactly (wrap the `FnDef` in - `Def::Fn(...)`, call `def_hash`) — this is explicitly authorised - by the plan's own Step-1 NOTE ("the implementer mirrors the exact - hashing entry point that `hash_pin.rs` uses if the name differs"). - Property and shape unchanged; recorded because the plan's literal - symbol was wrong (planner pseudo-vs-reality class — the - `feedback_specs_need_concrete_code` / `feedback_plan_pseudo_vs_reality` - family). -- iter embedding-abi-m1.1.4: the plan fixture `embed_export_float_ok.ail` - uses `(app *. a b)`. There is no `*.` Float-multiply head in - AILang — Float arithmetic uses the same typeclass-generic head as - Int (`examples/floats.ail` / `examples/mut_sum_floats.ail` both - use `(app + …)`; codegen lowers to `fadd/fmul double` by type). - Per the plan NOTE ("if the Float-multiply surface differs, use the - same head that example uses"), the fixture as written uses - `(app * a b)`. The fixture's role ("scalar+pure Float export - passes the gate") is preserved. Recorded; not yet exercised - (Task 4 BLOCKED before the gate ran). -- INFRA (not iter-authored): `docs/roadmap.md` became dirty in the - working tree during this run (a ~98-line P2 "DESIGN.md → design/ - role-split" milestone proposal, mtime mid-session). Phase 0's - `git status --porcelain` was empty (clean start). Nothing in this - iter writes `roadmap.md`; it was modified by an external/concurrent - writer. It is path-disjoint from every iter file. Left untouched - (roadmap.md is Boss-owned per CLAUDE.md; main HEAD sacrosanct). - The Boss must decide independently whether to keep or discard it; - it is NOT part of this iter's commit. (RESOLVED at Boss - re-dispatch: committed separately as Boss roadmap maintenance — - see Boss resolution. Not part of the Tasks 4–7 working tree.) -- iter embedding-abi-m1.1.5 (plan pseudo-vs-reality, behaviour- - neutral): plan Task-5 Step 3(b) third bullet says "`emit_ir` - (line 197): its internal `lower_workspace_inner` call passes - `Target::Executable`". Reality: `emit_ir` does NOT call - `lower_workspace_inner` directly — it calls `lower_workspace` - (`codegen lib.rs:214`), which now passes - `lower_workspace_inner(ws, AllocStrategy::Gc, Target::Executable)`. - The plan's *intent* (the in-source `missing_entry_main_is_error` - pin, which calls `emit_ir`, stays byte-unchanged) is delivered - transitively and verified GREEN at Task-5 Step 7. Recorded - because the plan's literal call-chain description was inaccurate - (planner pseudo-vs-reality family); no behavioural deviation. -- iter embedding-abi-m1.1.7 (plan-verbatim transcription defect, - corrected): the plan's verbatim Task-7 `embed_e2e.rs` imports - `std::path::{Path, PathBuf}` but the test body uses only - `PathBuf` — a transcribed unused import that raises - `unused_imports` on every build. Implementer-mindset judgement: - a verbatim copy that introduces a compiler warning is not the - plan's intent; reduced to `use std::path::PathBuf;`. Test - behaviour byte-identical (E2E still GREEN). Recorded as a - deviation from plan-literal text in service of plan intent. -- iter embedding-abi-m1.1.7 (plan code-block vs plan prose, design - consistency): plan Task-7 Step 4's code block shows the new - section as `### Embedding ABI (M1)`, but the same step's prose - instruction says "at the same heading level as §"Mangling - scheme"" — §"Mangling scheme" is `## ` (level-2) in DESIGN.md. - Used `## Embedding ABI (M1)` for structural consistency with the - surrounding §"Mangling scheme" / §"Data model" level-2 sections, - following the plan's explicit prose directive over its - illustrative code-block `#`-count. Section content byte-verbatim. -- iter embedding-abi-m1.1 (cross-checks, plan-confirming, no - deviation): the six named implementer-judgement cross-checks were - executed. (a) `Type::Con` has exactly `{name: String, args: - Vec}` — no mode field; the plan's `matches!` guard is - correct. (b) `declared_effs` is already `Vec` (per - `lib.rs` `declared_effs.into_iter().collect()`), so the plan's - conditional `.into_iter().collect()` conversion did NOT apply — - `.clone()` matches `ExportHasEffects(_, Vec)` exactly. - (c) `synth::llvm_type` lowers `Int`→`"i64"`, `Float`→`"double"` - (`synth.rs:17,21`) — the plan's `llvm_scalar` is an exact mirror - for the 2-scalar M1 subset; forwarder `call` types match - `@ail__`'s emitted signature (also proven by the - Task-7 E2E real `cc` link + run). (d) no pre-existing `run_cmd` - helper in `main.rs` (only inline `Command::new` blocks) — adding - it was correct per the plan. (e) `ailang_core::Def` is crate-root - re-exported (`ailang-core lib.rs:76`) — `ailang_core::Def::Fn` - valid in `main.rs`. (f) `ailang_surface::load_workspace` + - `ailang_check::check_workspace` + `Diagnostic.code: String` are - the exact `main.rs:580–581` public symbols. All confirmations of - plan assumptions, not contradictions. - -## Known debt - -- None blocking. The M1 ABI is deliberately scalar-only and - provisional until M3 (no `Bool`/`Str`/ADT/record marshalling, no - per-thread context lifecycle API); this is reserved-unimplemented - scope, labelled in DESIGN.md §"Embedding ABI (M1)", not debt. -- `check.py` fires the tracked P2 `*.bump_s` / `*.max_us` - known-noise envelope (3 regressed / 2 improved beyond tolerance / - 58 stable; e.g. `throughput.bench_list_sum.bump_s` +13.95%, - `latency.explicit_at_rc.max_us` +60.61%). This iter touched only - check-side gating, a codegen `Target` enum (every existing path - pinned `Executable`), and CLI dispatch — none on the runtime hot - path these metrics measure; the default-executable codegen/runtime - path is byte-unchanged (Task-5 Step 7/8 + Task-6 Step 6 confirm). - Per plan Step 6 this is milestone-close-`audit`-adjudicated, NOT - a plan-level gate. `compile_check.py` / `cross_lang.py` exit 0. -- `bench/architect_sweeps.sh` exits 1 both with and without the - DESIGN.md edit (pre-existing advisory residue, established - carry-on adjudicated at milestone-close `audit`); the new - §"Embedding ABI (M1)" introduced no new Wunschdenken / post- - mortem phrasing (the "provisional until M3" / "M2 will change" - clauses are labelled reserved-unimplemented, which the DESIGN.md - honesty rule explicitly permits). - -## Blocked detail - -> **RESOLVED (2026-05-18, re-dispatch).** This block was a -> first-dispatch event; it was resolved by the Boss plan Repair -> (see "## Boss resolution" below — Design-decision 6, Option 2, -> module==stem) and the Tasks 4–7 re-dispatch completed all four -> tasks cleanly. The detail below is **preserved verbatim as the -> iter's audit trail** and is no longer an open block. Iter status -> is DONE. - -**Task:** 4 (check-side export-signature gate) -**Reason:** spec-ambiguous (a spec-internal contradiction that -rippling-binds Tasks 4–7, surfaced as `unclear` per the per-task -sub-status table → BLOCKED to Boss). - -**Worker says:** The plan's Task-4 gate pin -(`crates/ailang-check/tests/embed_export_gate.rs`) is verbatim and -correct Rust; it calls `ailang_surface::load_workspace(&path)` on -each fixture (the exact call shape `crates/ail/src/main.rs:580–581` -uses). `load_workspace` → -`ailang_core::workspace::load_modules_with` → -`module_name_from_path` (`workspace.rs:1566`) hard-enforces -`in-file module name == file stem`, returning -`WorkspaceLoadError::ModuleNameMismatch` (`workspace.rs:182`) -otherwise. The plan's verbatim fixtures deliberately violate this: - - - `examples/embed_backtest_step.ail` has `(module backtest …)` — - spec-mandated, because the entry module name `backtest` is what - produces `libbacktest.a` (Task 6), `@ail_backtest_step` and the - exported C symbol `backtest_step` (Tasks 5/7, the spec headline - host `host.c` declares `extern int64_t backtest_step(...)`). - Stem is `embed_backtest_step` ⇒ `ModuleNameMismatch`. - - `embed_export_str_param_rejected.ail` → module `bad_str_param`; - `embed_export_io_rejected.ail` → `bad_io`; - `embed_export_effectful_rejected.ail` → `bad`; - `embed_export_adt_ret_rejected.ail` → `bad_adt_ret`; - `embed_export_float_ok.ail` → `embed_float_ok`. All - stem-mismatched ⇒ every `check_codes()` call panics on load - before the gate is ever consulted. - -These three plan facts cannot all hold simultaneously: (a) fixture -files named `examples/embed_*` (every Task 4/5/6/7 path cites these -exact names), (b) the spec-headline module/C-symbol naming -(`backtest` → `libbacktest.a` / `backtest_step`), (c) a -`load_workspace`-based check-side gate pin. `roundtrip_cli` (Task 1/3) -sidesteps this only because it drives `ail parse` (single-file, no -workspace stem-match). The contradiction is therefore invisible until -the first `load_workspace`-based fixture test — exactly Task 4. - -This is not one of the six named implementer-judgement cross-checks -(Float `*.` head, public-symbol confirmation, `Type::Con` field set, -`declared_effs` element type, `run_cmd` pre-existence, `llvm_scalar` -callee-ABI). It is a structural spec/plan defect requiring a -Boss/spec decision among at least: - - 1. Rename each fixture **file** to its module stem - (`examples/backtest.ail`, `examples/bad_str_param.ail`, …) and - update every Task 4/5/6/7 path citation accordingly (preserves - the spec headline module/C-symbol naming; ripples ~12 plan - path references + the Task-6/7 `libbacktest.a` math still holds - since module stays `backtest`). - 2. Rename each fixture **module** to its filename stem - (`(module embed_backtest_step …)`, …) and re-derive the spec's - C-symbol / `lib.a` story from `embed_backtest_step` - (changes the spec headline; `libembed_backtest_step.a`, - `@ail_embed_backtest_step_step` — almost certainly not what the - spec wants). - 3. Change the Task-4 gate pin to a loader that does not - stem-enforce (e.g. parse the single file directly), which - diverges from the plan's verbatim `load_workspace`-based test - and from how `ail check` actually loads (`main.rs:580`). - -Option 1 is the most spec-faithful (it keeps the headline module -`backtest` and only moves files), but it is a cross-task plan edit -(≥12 path citations) and therefore the planner's / Boss's call, not -an inline implementer substitution. I did NOT silently pick one -(reverted my own speculative fixture-module renames so the Boss sees -the contradiction unobscured); no gate production code was written. - -**Suggested next step:** Boss adopts Option 1 — re-issue the plan -for Tasks 4–7 with fixture files renamed to their module stems -(`examples/backtest.ail` etc.) and the ~12 path citations updated; -keep module `backtest` so the spec's `libbacktest.a` / -`backtest_step` headline is preserved — then re-dispatch this iter -with `task_range: [4, 7]` (Tasks 1–3 are clean and can be committed -now or carried). - -## Boss resolution (2026-05-18) - -The BLOCK was verified accurate against source: -`crates/ailang-core/src/workspace.rs:438` (`entry_module.name != -module_name_from_path(entry_path)` → `ModuleNameMismatch`, no -exception) — module name MUST equal file stem. The defect is a -**Boss plan-write error**: the plan's headline + gate fixtures -declared `(module backtest)` / `(module bad)` / … in `embed_*.ail` -files, malformed under that invariant. The orchestrator correctly -stopped at the Iron Law instead of guessing. Recon maps file -structure, not fixture-content invariants; the Boss source- -verification focused on code-edit anchors, not cross-checking each -fixture's module name against its filename against the loader — -this is the `feedback_specs_need_concrete_code` / -`feedback_plan_pseudo_vs_reality` family (the fixture as written -would not load), now also a fixture-well-formedness lesson. - -**Decision: Option 2, overriding the orchestrator-recommended -Option 1.** (Agent recommendations are not directives — own -judgement formed first.) Keep the descriptive `embed_*` filenames; -rename the fixture *modules* to their file stems -(`embed_backtest_step`, `embed_export_str_param_rejected`, …). -Rationale from semantics, not effort: - -- The spec's load-bearing contract (Decision 2, spec lines - 52–59 / 167–169) is that the exported C symbol is *author-chosen - and decoupled from `ail_` mangling* — - `(export "backtest_step")` is the contract; `host.c` asserts on - `backtest_step`, which is invariant under module rename. So - spec-faithfulness does **not** discriminate between the options; - `libbacktest.a` / `@ail_backtest_step` are *illustrative* - (build-output filename + internal mangling — the latter is - exactly what the spec decouples from), not frozen-ABI contracts. - The orchestrator's "Option 1 most spec-faithful" over-weighted - the illustrative `libbacktest.a` string. -- `examples/` is a flat shared corpus auto-enrolled in whole-corpus - tests (`roundtrip_cli`); Option 1's `examples/bad.ail` / - `backtest.ail` / `bad_io.ail` pollute it with collision-prone, - non-self-describing names. Option 2 keeps the self-describing - grouped `embed_*` set — a real maintainability argument in a - ~100-file flat dir read by future LLM authors. -- Option 2 (module = stem, C symbol stays `backtest_step` via the - `(export …)` string) is a *concrete demonstration of the spec's - own Decision-2 decoupling thesis* — the first fixture proving - module-name ≠ C-symbol — not a violation of it. The spec's - `(module backtest)` / `(module bad)` blocks are shape- - illustrative (planner contract: spec owns shape, not bytes). - -Consequence: entry module `embed_backtest_step` ⇒ archive -`libembed_backtest_step.a`, internal symbol -`@ail_embed_backtest_step_step`; the external C symbol -`backtest_step` and `host.c` are UNCHANGED (the actual coherent- -stop contract). Architect-at-close note recorded in the plan -(Design-decision 6): a fixture module name differing from the -spec's illustrative block is expected under module==stem + -Decision 2, not drift. - -Actions taken: (1) plan `docs/plans/embedding-abi-m1.1.md` fixed — -Design-decision 6 added, Task-4 fixtures → module==stem, Task-5/6/7 -dependent strings updated (`libembed_backtest_step.a`, -`@ail_embed_backtest_step_step`; C symbol `@backtest_step` -unchanged), the two iter-1 concerns folded (no `*.` Float head → -`(app * a b)`; `content_hash_fn` → `def_hash(&Def)`); (2) the -already-created headline `examples/embed_backtest_step.ail` -corrected to `(module embed_backtest_step)`; (3) the 5 blocked- -Task-4 fixtures + the RED `embed_export_gate.rs` discarded -(untracked WIP — recreated correctly by the re-dispatch from the -fixed plan); (4) Tasks 1–3 committed as a verified known-good -partial subset (`cargo build --workspace` exit 0, full workspace -green with zero FAILED after the WIP discard, `roundtrip_cli` -green); (5) orchestrator re-dispatched with `task_range: [4, 7]` -on the clean tree + corrected plan. The user's path-disjoint -roadmap addition (P2 "DESIGN.md → `design/` role-split") was -committed separately as Boss roadmap maintenance, not bundled -into this iter. - -## Files touched - -### Committed at 818177d (Tasks 1–3, first dispatch) - -Schema / canonical docs: -- crates/ailang-core/src/ast.rs (FnDef.export field) -- docs/DESIGN.md (fn-JSON "export" line) -- crates/ailang-core/specs/form_a.md (grammar production + prose) - -Workspace-wide compile-gate threading (~85 `export: None,` literals): -- crates/ailang-core/src/desugar.rs, pretty.rs -- crates/ailang-core/tests/{design_schema_drift,hash_pin,spec_drift}.rs -- crates/ailang-check/src/{lib,lift,linearity,mono,reuse_shape,suppress_filter,uniqueness}.rs -- crates/ailang-check/tests/workspace.rs -- crates/ailang-codegen/src/lib.rs -- crates/ailang-prose/src/lib.rs - -Surface modifier (Task 3): -- crates/ailang-surface/src/parse.rs (parse_export + parse_fn thread) -- crates/ailang-surface/src/print.rs (write_fn_def export block) - -New files committed at 818177d: -- examples/embed_noentry_baseline.ail -- examples/embed_backtest_step.ail (corrected to `(module embed_backtest_step)`) -- crates/ail/tests/embed_missing_main_baseline.rs -- crates/ailang-core/tests/embed_export_hash_stable.rs - -### Tasks 4–7 re-dispatch working tree (UNCOMMITTED — this hand-off) - -Modified (4 source/doc files): -- crates/ailang-check/src/lib.rs (2 CheckError variants + 2 code() arms + check_fn gate; +51) -- crates/ailang-codegen/src/lib.rs (Target enum + threaded target + forwarder loop + fn_scalar_sig/llvm_scalar; +130/-22) -- crates/ail/src/main.rs (--emit clap field + Cmd::Build branch + build_staticlib + run_cmd; +123) -- docs/DESIGN.md (new §"Embedding ABI (M1)" subsection; +24) - -New files (untracked, this re-dispatch): -- examples/embed_export_str_param_rejected.ail -- examples/embed_export_adt_ret_rejected.ail -- examples/embed_export_io_rejected.ail -- examples/embed_export_effectful_rejected.ail -- examples/embed_export_float_ok.ail -- crates/ailang-check/tests/embed_export_gate.rs (GREEN 6/6) -- crates/ailang-codegen/tests/embed_staticlib_lowering.rs (GREEN 2/2) -- crates/ail/tests/embed_staticlib_cli.rs (GREEN 2/2) -- crates/ail/tests/embed/host.c -- crates/ail/tests/embed_e2e.rs (GREEN 1/1 — coherent-stop proof) - -(`docs/roadmap.md` is NOT in this iter's working tree — the -first-dispatch external modification was Boss-committed separately -as roadmap maintenance, per the Boss resolution.) - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-embedding-abi-m1.1.json diff --git a/docs/journals/2026-05-18-iter-embedding-abi-m2.1.md b/docs/journals/2026-05-18-iter-embedding-abi-m2.1.md deleted file mode 100644 index 442d7ec..0000000 --- a/docs/journals/2026-05-18-iter-embedding-abi-m2.1.md +++ /dev/null @@ -1,214 +0,0 @@ -# iter embedding-abi-m2.1 — per-thread runtime context + concurrency safety - -**Date:** 2026-05-18 -**Started from:** c9a84b33b3e7806491153a5bd3037be1b8ef9d31 (Tasks 1–4 committed) -**Original start (first dispatch):** b3388c83506a314fbd87f3354f80ae8cc212f501 -**Status:** DONE -**Tasks completed:** 9 of 9 (Tasks 1–4 in the first dispatch, committed at c9a84b3; Tasks 5–9 in this re-dispatch) - -## Summary - -The per-thread embedding-ctx ABI is complete. Tasks 1–4 (first -dispatch, committed at c9a84b3): the FIXED-FIRST alloc-guard baseline -pin, the de-globalised `ailang_ctx_t` runtime (per-thread TLS slot + -conditional accounting, `g_rc_*`+atexit retained verbatim as the -null-ctx fallback), the codegen forwarder gaining a leading -`ptr %ctx` + `@__ail_tls_ctx` save/store/restore around the -byte-unchanged internal call, and the CLI RC-only guard. Tasks 5–9 -(this re-dispatch): the per-thread-ctx scalar-swarm capability demo -(`swarm.c`, no SHARED_CTX, GREEN), the de-globalisation -negative-control standing test in the rc-accounting harness -(`rc_accounting_shared_ctx_is_flagged_by_tsan` — shared-ctx genuinely -raced under tsan, teeth proven), the M1 `host.c` migration to the ctx -ABI (`s == 25` holds through the ctx-threaded forwarder), the -DESIGN.md current-state update (provisional-until-M3 narrowed to -value/record layout; Decision-10 per-thread-ctx note; pinned -"Export parameters are written **bare**…" sentence + `(con Int)` -snippet preserved verbatim), and the full regression + bench gate -(workspace 0 failed; bench-trio causally exonerated). The milestone's -concurrency-safety invariant is sanitiser-verified. - -## Boss spec-defect repair (2026-05-18) - -The first dispatch (Tasks 1–4 clean) correctly **BLOCKED on Task 5**, -surfacing a genuine spec defect: the original Task 5 paired a -`-DSHARED_CTX` negative control onto `swarm.c`. That is structurally -impossible — `examples/embed_backtest_step.ail` is a non-allocating -scalar kernel (`state += sample*sample`); it never calls -`ailang_rc_alloc`/`ailang_rc_dec`, never writes a `ctx` field, and -`@__ail_tls_ctx` is `__thread` (per-thread, never shared even when the -`ailang_ctx_t*` value is passed to a shared ctx). A shared-ctx scalar -swarm therefore has no shared-memory write to race on; tsan correctly -stays clean. Asserting a structurally-unreachable race would have been -a vacuous (and dishonest) test. The Boss adjudicated: the -de-globalisation negative-control teeth belong to the **item-1 -rc-accounting harness** (`rc_accounting_tsan.c -DSHARED_CTX`, where a -shared ctx genuinely races on `ctx->alloc_count`), NOT `swarm.c`. The -spec was amended in lockstep and the plan's Task 5 restructured: -`swarm.c` is the per-thread-ctx capability demo ONLY (one green -positive test); the negative control became -`rc_accounting_shared_ctx_is_flagged_by_tsan` added to the committed -`embed_rc_accounting_tsan.rs`. This re-dispatch implemented the -corrected Task 5 — no swarm.c negative control reintroduced. The -plan's Task 3 expected-string transcription error (recorded below) -was likewise Boss-corrected (`@ail_embed_backtest_step_step`, the -module-qualified convention-correct symbol). - -## Per-task notes - -- iter embedding-abi-m2.1.1 (first dispatch, committed): FIXED-FIRST - alloc-guard pin. New `crates/ail/tests/embed_staticlib_alloc_guard.rs` - (verbatim from plan); RED captured (build currently succeeds), the - deliverable. RED→GREEN across the iter (Task 4 closes it). -- iter embedding-abi-m2.1.2 (first dispatch, committed): `runtime/rc.c` - gains `ailang_ctx_t` {alloc_count, free_count} + `ailang_ctx_new`/ - `ailang_ctx_free` + `__thread __ail_tls_ctx`; the two increment - sites become `if (_ctx) _ctx->… else g_rc_…`. `g_rc_*` statics + - atexit + constructor retained verbatim. New `rc_accounting_tsan.c` + - `embed_rc_accounting_tsan.rs`: RED (link error) → GREEN (per-ctx, - 8×200000, tsan-clean). -- iter embedding-abi-m2.1.3 (first dispatch, committed): codegen - `Target::StaticLib` arm emits `@__ail_tls_ctx = external - thread_local global ptr` once; forwarder signature gains leading - `ptr %ctx`; body does load-save / store-ctx / unchanged - `call @ail__(args)` / store-restore. `_adapter`/`_clos` - + internal arg vector byte-untouched. Doc-comment narrowed to - value/record-layout provisionality. `embed_e2e` left RED by design - (Task 6 migrates it). -- iter embedding-abi-m2.1.4 (first dispatch, committed): - `crates/ail/src/main.rs build_staticlib` rejects `--alloc != Rc` - after the has_export check, before lowering. Task-1 pin RED→GREEN; - `embed_staticlib_cli` default path unaffected. -- iter embedding-abi-m2.1.5 (this re-dispatch): per-thread-ctx - capability demo. New `crates/ail/tests/embed/swarm.c` (no - SHARED_CTX) + `crates/ail/tests/embed_swarm_tsan.rs`; - `scalar_swarm_per_ctx_is_tsan_clean_and_matches_oracle` GREEN - (8 threads × 200k, tsan-clean, every result == serial oracle). - De-globalisation negative control - `rc_accounting_shared_ctx_is_flagged_by_tsan` appended to the - committed `embed_rc_accounting_tsan.rs` (reuses its `ws_root()`/ - `cc()` helpers); GREEN — shared-ctx build genuinely raced under - tsan (teeth proven, not vacuous). No swarm.c negative control - reintroduced (the Boss spec-defect repair). -- iter embedding-abi-m2.1.6 (this re-dispatch): `crates/ail/tests/ - embed/host.c` replaced with the ctx-ABI version (opaque - `ailang_ctx_t`, `ailang_ctx_new`/`_free` lifecycle, ctx-prefixed - `backtest_step`). Confirmed expected RED post-Task-3 first - (`s != 25` from the garbage-ctx arg shift). `embed_e2e.rs` - intentionally NOT modified: the M1 harness already links - `libailang_rt.a` by absolute path (`.arg(outdir.join( - "libailang_rt.a"))`), so the ctx symbols resolve without an - `-lailang_rt` add — the task block's own "no change is needed - beyond Step 2" conditional branch. GREEN, `s == 25` holds. -- iter embedding-abi-m2.1.7 (this re-dispatch): `docs/DESIGN.md` — - provisional-until-M3 sentence (~:2282) replaced with the M2 ctx - prose (mandatory leading `ailang_ctx_t*`, per-thread, RC-only - swarm artefact, value/record layout still provisional); - Decision-10 atomicity bullet (~:1528) gains the per-thread-ctx - soundness note. Pinned "Export parameters are written **bare**…" - sentence + canonical `(con Int)` snippet preserved verbatim (the - edit ends before them; no soft-wrap split risk). `docs_honesty_pin` - GREEN unmodified (5 passed), `design_schema_drift` GREEN (8 - passed) — edits outside the `## Data model` scan window. -- iter embedding-abi-m2.1.8 (this re-dispatch): regression gate, no - code. `print_no_leak_pin` (1) + `e2e -- rc_stats` (4) GREEN — - null-ctx fallback prints unchanged. `embed_export_hash_stable` - (1, `fn_without_export_hash_is_unchanged`) + `embed_export_gate` - (6, all pass) GREEN — no schema field, export gate unchanged. - Full `cargo test --workspace`: every suite ok, 0 failing suites. -- iter embedding-abi-m2.1.9 (this re-dispatch): bench gate, no code. - `compile_check.py` exit=0 (24/24 stable); `cross_lang.py` exit=0 - (25/25 stable); `check.py` exit=1 on exactly 4 tracked-P2 noise - metrics (`bench_list_sum.bump_s` +16.53%, - `latency.explicit_at_rc.max_us` +40.41%, - `latency.implicit_at_rc.p99_9_us` +31.66%, - `latency.implicit_at_rc.max_us` +92.35%). Causally exonerated by - the source-level invariant (see Concerns); no baseline ratified. - -## Concerns - -- iter embedding-abi-m2.1.3 (first dispatch — carried forward): - PLAN TRANSCRIPTION ERROR. Task 3 Step 1's plan code block asserted - `ir.contains("call i64 @ail_backtest_step(i64 %a0, i64 %a1)")`. - The internal symbol is `@ail_embed_backtest_step_step` - (module-qualified `@ail__`), which the plan's own Step-4 - parenthetical and Boss constraint 3 require to stay byte-unchanged. - Written instead with the convention-correct symbol (Boss-corrected - in the re-dispatched plan). The plan file should remain corrected. -- iter embedding-abi-m2.1.6: `embed_e2e.rs` is in the task's `Files:` - block but was intentionally left unchanged — the M1 harness already - links `libailang_rt.a` by absolute path, so the ctx symbols resolve - without an `-lailang_rt` add. This is the task block's own sanctioned - "no change is needed beyond Step 2" conditional branch, not a missed - requirement (observation, not correctness). -- iter embedding-abi-m2.1.7: the section header is still - `## Embedding ABI (M1)` while the body now describes the M2 ctx ABI - as current state — a header/content honesty mismatch. NOT fixed: - renaming the header is outside the task block's named edit regions - (`:2282–2285`, `:1528–1529`) and could perturb a doc-honesty pin - tracking the `(M1)` token; the controller curates scope, not the - implementer. Routed here for Boss adjudication — a candidate - follow-up (header rename + fieldtest/pin audit) but explicitly NOT - in this iter's plan scope. -- iter embedding-abi-m2.1.8: the task block expected - `embed_export_gate` to be "8 passed"; the suite at HEAD c9a84b3 has - exactly 6 tests, all passing, 0 filtered. The literal "8" is a - plan-transcription artefact (pre-existing — M2 added/removed no test - in this suite). The protected invariant (export gate unchanged by - M2) holds unambiguously; the implementer correctly did NOT pad the - suite to 8 (that would fabricate coverage). Plan number vs reality, - not a regression. -- iter embedding-abi-m2.1.9: `check.py` exit=1 on 4 metrics, all in - the tracked-P2 noise families (`*.bump_s`, `latency.*.max_us`, - `latency.*.p99_9_us`). CAUSALLY EXONERATED by a source-level - invariant stronger than the plan's binary `cmp`: this re-dispatch's - entire working-tree footprint is test fixtures + test drivers + - `docs/DESIGN.md` — ZERO files under `crates/*/src/` or `runtime/`. - The `ail` codegen binary and the linked `rc.c`/`str.c` runtime are - byte-identical to HEAD c9a84b3 by construction, so the firing - metrics cannot be caused by this iter. The plan's `git stash`/ - `git stash pop` recipe was deliberately NOT run: on a re-dispatch - it would discard the iter's uncommitted Tasks 5–8 work and violate - the working-tree-as-handoff contract. No baseline ratified - (consistent with roadmap-P2 noise policy + the 2026-05-14 audit-pd - "baseline pristine, Nth consecutive observation" envelope). - -## Known debt - -- `## Embedding ABI (M1)` section header in DESIGN.md still says - "(M1)" though its body is now M2 current-state. Out of this iter's - plan scope (Task 7 named only `:2282–2285` and `:1528–1529`). - Candidate follow-up: header rename with a doc-honesty-pin audit - for any `(M1)` token dependency. Not touched to keep the diff - scope-compliant and avoid perturbing a pin. - -## Blocked detail - -None. All 9 tasks completed (Tasks 1–4 committed at c9a84b3 in the -first dispatch; Tasks 5–9 in this re-dispatch, all reaching -`approved` quality with zero review re-loops). The first dispatch's -Task-5 BLOCK was a genuine spec defect, Boss-adjudicated and repaired -before this re-dispatch (see "Boss spec-defect repair"). - -## Files touched (this re-dispatch — Tasks 5–9 working tree) - -Modified: -- `crates/ail/tests/embed/host.c` (Task 6: ctx-ABI migration) -- `crates/ail/tests/embed_rc_accounting_tsan.rs` (Task 5: + - `rc_accounting_shared_ctx_is_flagged_by_tsan` negative control) -- `docs/DESIGN.md` (Task 7: M2 current-state + Decision-10 note) - -Created: -- `crates/ail/tests/embed/swarm.c` (Task 5: per-thread-ctx - capability demo, no SHARED_CTX) -- `crates/ail/tests/embed_swarm_tsan.rs` (Task 5: one green - positive test) - -(Tasks 1–4's files — `embed_staticlib_alloc_guard.rs`, `rc.c`, -codegen `lib.rs`, `embed_staticlib_lowering.rs`, `main.rs`, -`rc_accounting_tsan.c`, the Task-2 `embed_rc_accounting_tsan.rs` -base — are already committed at c9a84b3.) - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-embedding-abi-m2.1.json diff --git a/docs/journals/2026-05-18-iter-embedding-abi-m2.tidy.md b/docs/journals/2026-05-18-iter-embedding-abi-m2.tidy.md deleted file mode 100644 index d5c0c06..0000000 --- a/docs/journals/2026-05-18-iter-embedding-abi-m2.tidy.md +++ /dev/null @@ -1,58 +0,0 @@ -# iter embedding-abi-m2.tidy — lockstep section rename `Embedding ABI (M1)` → `Embedding ABI` - -**Date:** 2026-05-18 -**Started from:** d1241b12c759765f12713584ce2aab21e2a74f74 -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -Audit-driven docs-honesty tidy (no parent spec; source: -`docs/journals/2026-05-18-audit-embedding-abi-m2.md`, the M2 -milestone-close audit). The DESIGN.md section `## Embedding ABI (M1)` -had a stale `(M1)` qualifier — its body now describes the post-M2 ctx -ABI present-tense, so the qualifier contradicts the current-state-mirror -discipline. Renamed the header and moved all four *live* coupled -references in lockstep so no pointer dangles at the vanished `(M1)` -title. Pure docs/test-string tidy: zero language / checker / codegen / -schema change; `ail check`/`run` byte-unchanged by construction. The -`docs_honesty_pin.rs:135` asserted substring is rename-immune (no -section-name token, matched file-wide via `d.contains`) and was left -byte-verbatim untouched; the pin staying green before AND after -empirically confirms the rename did not perturb it. - -## Per-task notes - -- iter embedding-abi-m2.tidy.1: lockstep rename across 5 live-coupling - sites in 3 files — `docs/DESIGN.md:2266` (section header) + - `docs/DESIGN.md:2354` (schema-block xref comment) + - `crates/ailang-core/tests/docs_honesty_pin.rs:134` (test comment) + - `:136` (assert-message) + `crates/ailang-core/specs/form_a.md:89` - (compiled-in forward-xref). Each site: one-line in-place `(M1)`-token - deletion, no reflow. Step-1 baseline grep showed exactly the 5 sites - present (`:135` correctly absent — no `(M1)` token); Step-6 grep - returned zero (exit 1). `docs_honesty_pin` 5 passed and - `design_schema_drift` 8 passed both before and after. Footprint - gated at exactly 3 files, 5 insertions / 5 deletions — no - `docs/journals/**`, `docs/specs/**`, `docs/plans/**`, - `bench/orchestrator-stats/**` touched (committed append-only history - retains `Embedding ABI (M1)` as correct record of the name at the - time; renaming it would falsify the record). - -## Concerns - -(none) - -## Known debt - -(none) - -## Files touched - -- `docs/DESIGN.md` (header `:2266`, schema-block xref `:2354`) -- `crates/ailang-core/tests/docs_honesty_pin.rs` (comment `:134`, assert-message `:136`; `:135` untouched) -- `crates/ailang-core/specs/form_a.md` (forward-xref `:89`) - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-embedding-abi-m2.tidy.json diff --git a/docs/journals/2026-05-18-iter-embedding-abi-m3.1.md b/docs/journals/2026-05-18-iter-embedding-abi-m3.1.md deleted file mode 100644 index 94920ad..0000000 --- a/docs/journals/2026-05-18-iter-embedding-abi-m3.1.md +++ /dev/null @@ -1,404 +0,0 @@ -# iter embedding-abi-m3.1 — freeze the value layout + record across the C ABI - -**Date:** 2026-05-18 -**Started from:** 15ee3c5c8fbb871a9cde6cc7d57c0801bae9ae85 -**Status:** DONE (Tasks 1–5 GREEN after the Boss spec-consistency -repair below; Tasks 6–7 completed GREEN in the re-dispatch) -**Tasks completed:** 7 of 7 (Tasks 1–4 DONE/GREEN by the orchestrator; -Task 5 correctly BLOCKED on a genuine spec-defect, M2.1-precedent -class, resolved GREEN by the Boss spec-consistency repair — -global-leak-freedom proof model; Tasks 6–7 DONE/GREEN in the -`task_range:[6,7]` re-dispatch on the amended plan) - -## Summary - -> **Boss note (read first):** the paragraphs below are the -> orchestrator's *first-dispatch* account (present-tense as written -> then) — Task 5's BLOCK was real and is preserved for its -> documentary value. It was resolved by the Boss spec-consistency -> repair and Tasks 6–7 completed in the re-dispatch; the iter is -> **DONE 7/7**. See "## Boss adjudication" and "## Re-dispatch" for -> the resolution. Do not read the "Task 5 is BLOCKED / Task 7 not -> reached" sentences below as the final state. - -Tasks 1–4 are complete and GREEN in the working tree: the FIXED-FIRST -baseline pins (1a re-point annotation on `adt_ret_export_rejected`; -1b the new `@ailang_rc_alloc` heap-box byte-pin proving size=8+n*8 / -tag@0 / fields@8,16), the export-gate widen to a single-ctor scalar -record (`is_c_scalar` → two-level `is_c_abi_type`, 10/10 gate suite), -the codegen forwarder widen (`llvm_scalar` maps a gate-guaranteed -record `Type::Con` → `ptr`; M2 forwarder body byte-unchanged; 3/3 -staticlib-lowering pins), and the `own` E2E record round-trip -(1,000,000-iter balanced ctx readback `allocs=1000000 frees=1000000 -live=0`, exit 0 — the spec's "mechanically almost free" ownership -claim substantiated, ratified by `e2e.rs:1855`, NOT the Iter-18a -metadata-only pin). - -Task 5 (the `borrow` E2E arm) is BLOCKED on a genuine -plan-assertion-model defect (detail below), structurally identical to -the M2.1 first-dispatch swarm.c/-DSHARED_CTX spec defect that the -Boss adjudicated and repaired. No production-code bug: the borrow -mode is leak-free and value-correct; the plan's *assertion shape* is -unsatisfiable given the M2 TLS-ctx accounting mechanism. Returned -BLOCKED for Boss adjudication per the Iron Law (never push past -BLOCKED by hand; spec/plan amendment is a Boss/brainstorm decision). - -audit owns the bench + architect milestone-close (spec Testing items -8/9 — architect Invariant 1, bench trio); this implement run asserts -only the test-suite GREEN state an implement run can, and Task 7 -(workspace-green gate) was not reached because Task 5 blocks the -sequence. - -## Per-task notes - -- iter embedding-abi-m3.1.1 (DONE_WITH_CONCERNS): re-point annotation - added verbatim above `adt_ret_export_rejected` - (`crates/ailang-check/tests/embed_export_gate.rs`); gate suite 6/6 - inert. New byte-pin carrier `examples/embed_record_layout_carrier.ail` - + new pin `crates/ailang-codegen/tests/embed_record_layout_pin.rs` - (`lower_workspace_with_alloc(&ws, AllocStrategy::Rc)`, mirroring - `embed_staticlib_lowering.rs`'s `load()` helper). Pin GREEN on the - pre-M3 tree. Carrier required two grounding corrections vs the - plan's literal pseudo-code (see Concerns). -- iter embedding-abi-m3.1.2 (DONE_WITH_CONCERNS): `is_c_scalar` → - two-level `is_c_abi_type` (`crates/ailang-check/src/lib.rs:1921+`), - both call sites (param loop + ret) repointed; `:452` `#[error]` - widened; `:458` correctly left byte-unchanged (no `M1` token). 5 - fixtures (`embed_export_record_ok` + 3 `_rejected` + re-pointed - `embed_export_adt_ret_rejected` → multi-ctor+Str `Reading`) + 4 new - gate tests. RED observed (`record_export_passes` failed pre-widen); - GREEN 10/10; full `ailang-check` 0 failed. -- iter embedding-abi-m3.1.3 (DONE_WITH_CONCERNS): `llvm_scalar` - widened (`Float`→double, `Int`→i64, other `Type::Con`→`ptr`, `_`→ - i64) + doc-comment lockstep; `fn_scalar_sig` clarifying comment only - (body recon-confirmed filter-free). 2 fixtures - (`embed_backtest_step_record{,_borrow}.ail`) + forwarder-IR pin - `staticlib_record_forwarder_is_ptr_passthrough`. RED observed - (State→i64 pre-widen); GREEN 3/3; full `ailang-codegen` 0 failed - (Task-1 byte-pin unchanged — box layout did not move). -- iter embedding-abi-m3.1.4 (DONE): `crates/ail/tests/embed/ - record_roundtrip.c` + `crates/ail/tests/embed_record_e2e.rs`. - `build_link_run_embed` composed from verbatim-mirrored real - incantations (`embed_e2e.rs` build/link + `e2e.rs:2190` - `build_and_run_with_rc_stats` stats-parse — `embed_e2e.rs` exposes - no reusable helper; mirroring, not inventing). `own` E2E GREEN - immediately once Tasks 2+3 in tree (ctx readback `allocs=1000000 - frees=1000000 live=0`, exit 0). No production code (plan: none - expected). Ratifier `e2e.rs::alloc_rc_own_param_dec_at_fn_return: - 1855` cited; `borrow_own_demo_modes_are_metadata_only:1330` NOT - cited (Boss constraint 2 honoured). -- iter embedding-abi-m3.1.5 (BLOCKED — plan-assertion-model defect): - `record_roundtrip_borrow_alloc_eq_free` added to - `embed_record_e2e.rs`. RED and stays RED — see Blocked detail. Left - in the working tree as evidence (the M2.1-precedent handling: the - BLOCKED task's state is left for Boss adjudication, not discarded). -- iter embedding-abi-m3.1.6 / .7: NOT attempted in the first dispatch - (Task 5 was a hard predecessor in the plan's sequencing; `skip task - K, continue` is intentionally not a mode). Completed GREEN in the - `task_range:[6,7]` re-dispatch — see the Re-dispatch section below. - -## Concerns - -- iter embedding-abi-m3.1.1: PLAN PSEUDO-CODE DEFECT (class: - "plan pseudo-code vs reality"). The plan's carrier body used - `(app Pt a b)` for ADT construction — non-compiling (`ail check`: - `[not-a-function] mk: \`Pt\` is not a function`). AILang construction - is `(term-ctor args…)` (grounded on - `examples/box.ail:36`, `borrow_own_demo.ail`). Corrected to - `(term-ctor Pt Pt a b)`. Second correction: the plan's - `(app print …)` IO sink is unresolvable by the codegen library API - the byte-pin must use (`lower_workspace_with_alloc` does not run the - mono/prelude pass that resolves polymorphic `print` — - empirically: `UnknownVar("print")`; `io/print_int` is not a known - effect-op). Corrected to `(let s (app int_to_str …) (do io/print_str - s))` (monomorphic builtin, library-resolvable; grounded on - `int_to_str_drop_rc.ail`). Structural intent (single-ctor 2-Int - record forced onto the `@ailang_rc_alloc` heap path) preserved - exactly; plan Step-4/5 explicitly sanctions carrier adjustment to - force the heap box. Observation, not correctness — the byte-pin - asserts the literal emitted forms and is a genuine RED-on-offset- - move guard (proven enforceable would be Task 6 Step 7, not reached). -- iter embedding-abi-m3.1.2: same plan-pseudo-code class — all 5 - fixture bodies used `(app Ctor …)`; corrected to `(term-ctor …)`. - The test assertion idiom was matched to the file's *existing* - `.iter().any(|c| c == …)` pattern (plan showed - `.contains(&"…".to_string())`); `record_export_passes` written as - `!any(gate-code)` not the plan's `.is_empty()` (a single - `over-strict-mode` warning makes the diagnostics vec non-empty — - `.is_empty()` would false-fail; the protected property is - "no gate rejection"). Semantic intent preserved. -- iter embedding-abi-m3.1.2 (carry to Task 6): the - `CheckError::ExportNonScalarSignature` rustdoc at - `crates/ailang-check/src/lib.rs:448-450` still says "M1's embedding - ABI is scalar-only; `Bool`/`Str`/every ADT is rejected" — now stale - (single-ctor scalar records are accepted). Outside Task 2's named - edit regions (`:452`/`:458` only); Task 6 owns the broader - stale-doc / "provisional until M3" sweep and should absorb this - line. Not gating; recorded for Boss/Task-6 scope. -- iter embedding-abi-m3.1.3: PLAN TRANSCRIPTION DEFECT (same class as - the M2.1 journal's `@ail_embed_backtest_step_step` correction). The - plan's fixtures used `(module backtest …)` while the file stems are - `embed_backtest_step_record{,_borrow}` — the loader enforces - module==file-stem (`[module-name-mismatch]`). Module names corrected - to the file stems; the forwarder-IR pin's internal-symbol assertion - corrected from the plan's draft `@ail_backtest_step` to the - convention-correct module-qualified - `@ail_embed_backtest_step_record_step` (per the plan's own Step-3 - parenthetical "tighten to literal emitted forms" + Boss constraint / - M2.1 module-qualified-symbol mandate). `llvm_scalar` doc-comment - updated in lockstep with its widened body (coherence-required, not - surrounding cleanup — a function whose doc says "the only two M1 - scalars" while its body maps a third case is self-contradictory - within the same task). `llvm_scalar` is now a slightly - over-narrow *name* (also maps records→ptr) — renaming is a 4-site - cross-cut out of Task-3 scope; the doc disambiguates; advisory Nit - only. - -## Known debt - -- `crates/ailang-check/src/lib.rs:448-450` stale variant rustdoc — - Task-6-scope (see Concerns); not touched to keep Task 2 - scope-compliant. -- `llvm_scalar` identifier slightly narrower than its post-M3 - behaviour — out of Task-3 scope; doc disambiguates. -- Tasks 6 (DESIGN.md SSOT + lockstep freeze pointers + the - match_lower.rs enforceability demonstration) and 7 (workspace-green - gate) unexecuted — blocked behind Task 5. - -## Blocked detail - -**Task:** 5 (`record_roundtrip_borrow_alloc_eq_free`). -**Reason:** worker-blocked — design/plan contradiction -(plan-assertion-model defect; the implementer-phase sub-status is -`BLOCKED`, mapped to the Boss per the SKILL.md sub-status table). - -**Worker says (verbatim):** The plan's Task-5 assertion -`assert_eq!(stats.allocs, stats.frees, "borrow: kernel kept input; -host freed every input + the return")` is structurally unsatisfiable. -`stats` is defined (Task 4 Step 2, plan) as the *single* -`ailang_ctx_free` RC-stats readback — the kernel-internal ctx -accounting. In `borrow` mode the kernel allocates the new return box -during the forwarder call (lands on `ctx->alloc_count` because the -codegen forwarder publishes `ctx` into `__ail_tls_ctx` for the -synchronous call duration only) but does NOT consume/drop the borrowed -input (correct borrow semantics, ratified by -`e2e.rs::alloc_rc_borrow_only_recursive_list_drop:1671`). The -balancing frees are the host's per-iter `ailang_rc_dec(st)`, which run -*outside* any forwarder call → `__ail_tls_ctx` is NULL → they land on -`g_rc_free_count` (the atexit fallback line), NOT on `ctx`. There is -no public runtime API (`ailang_ctx_enter`/`set`/`bind`) for the C -host to publish a ctx to TLS around its own decs (`__thread -ailang_ctx_t *__ail_tls_ctx` is runtime-internal; the only writer is -the codegen-emitted forwarder, by M2 design). Empirical readback -(`AILANG_RC_STATS=1`, borrow): ctx line `allocs=1000000 frees=0 -live=1000000`; g_rc atexit line `allocs=1 frees=1000001 -live=-1000000`. Globally leak-free and value-correct (exit 0, -`assert(n==1000000)` held; `allocs_total=1000001 == -frees_total=1000001`, net live=0). The plan's own assertion *message* -("host freed every input") contradicts its own `stats` *source* (the -ctx readback, which by construction cannot see host-side frees) — an -internal contradiction in the plan. This is NOT a production-code bug -(the spec's substantive invariant — ownership follows the declared -mode, no leak — holds); it is a plan-assertion-model defect of the -same class the M2.1 journal records (the swarm.c/-DSHARED_CTX -spec defect: a plan assertion structurally impossible given the M2 -TLS-ctx mechanism, Boss-adjudicated + spec-amended, NOT -agent-self-amended). Per the Iron Law ("never push past BLOCKED by -hand") and memory ("when an iter needs a spec change, the spec is -wrong — update via brainstorm, not a self-patch"), returned BLOCKED -rather than silently substituting a global-balance / `live==0` -assertion (which would be the correct invariant but is a *different* -assertion than the plan scripts — a Boss/brainstorm adjudication). - -**Suggested next step:** Boss adjudicates the spec/plan accounting -model — likely: amend the spec + Task 5 (and re-check Task 4's -coincidentally-passing ctx-line assertion) so the proven invariant is -global leak-freedom (`live==0` across the ctx readback AND the g_rc -atexit line, or `allocs_total==frees_total` summed) for *both* modes, -rather than per-line ctx `allocs==frees` (which only holds for `own` -by the accident that own-decs happen inside the forwarder call). Then -re-dispatch with `task_range:[5,7]` against the amended plan. Tasks -1–4 are sound and GREEN; the working tree (including the failing -Task-5 test as evidence) is left intact per the M2.1 precedent. - -## Files touched - -Modified: -- `crates/ailang-check/src/lib.rs` (Task 2: gate predicate + message) -- `crates/ailang-check/tests/embed_export_gate.rs` (Tasks 1,2) -- `crates/ailang-codegen/src/lib.rs` (Task 3: llvm_scalar + - fn_scalar_sig comment) -- `crates/ailang-codegen/tests/embed_staticlib_lowering.rs` (Task 3) -- `examples/embed_export_adt_ret_rejected.ail` (Task 2: re-point) - -Created: -- `crates/ailang-codegen/tests/embed_record_layout_pin.rs` (Task 1) -- `examples/embed_record_layout_carrier.ail` (Task 1) -- `examples/embed_export_record_ok.ail` (Task 2) -- `examples/embed_export_str_field_record_rejected.ail` (Task 2) -- `examples/embed_export_list_field_record_rejected.ail` (Task 2) -- `examples/embed_export_multictor_rejected.ail` (Task 2) -- `examples/embed_backtest_step_record.ail` (Task 3) -- `examples/embed_backtest_step_record_borrow.ail` (Task 3) -- `crates/ail/tests/embed/record_roundtrip.c` (Task 4/5) -- `crates/ail/tests/embed_record_e2e.rs` (Task 4 GREEN + - Task 5 RED-evidence) - -## Boss adjudication (2026-05-18 — M2.1-precedent class) - -The orchestrator's Task-5 BLOCK was correct and well-reasoned, and -its diagnosis was independently verified by the Boss (not -agent-trusted): built+ran both E2E modes by hand, full -`AILANG_RC_STATS` stderr — - -- own: ctx `allocs=1000000 frees=1000000 live=0`; g_rc `allocs=1 frees=1 live=0`; exit 0 -- borrow: ctx `allocs=1000000 frees=0 live=1000000`; g_rc `allocs=1 frees=1000001 live=-1000000`; exit 0 - -Both modes are **globally** leak-free (Σallocs = Σfrees = 1,000,001, -Σ`live` = 0) and value-correct. The defect is purely the spec's -proof *instrument*: §"Testing strategy" items 3/4 + §"Coherent stop" -asserted the single `ailang_ctx_free` ctx readback shows -`allocs == frees`, which is false **by M2's shipped TLS-ctx design** -(ctx bound to `__ail_tls_ctx` only for the synchronous forwarder -call; the host's `make_state`/`dec` run outside it → `g_rc_*`). The -roadmap's own coherent-stop wording is "a record in/out with -**alloc==free** across many calls" = *global*, exactly what holds. - -Adjudicated as a **Boss spec-consistency repair, not a brainstorm -bounce** — M2.1-precedent class (genuine spec defect; deliverable + -substantive invariant sound; only verification mechanics -mis-specified; first BLOCKED task in M3, not 2+-in-a-family). The -repair, folded into this partial (M2.1 `c9a84b3` shape): - -1. Spec amended (dated repair note + §"Testing strategy" 3/4 + - §"Coherent stop"): proof model → **global leak-freedom** - (Σallocs == Σfrees across *all* `ailang_rc_stats:` lines, - Σ`live` = 0, exit 0) for both modes; the M2-TLS cross-attribution - documented as correct behaviour, not a leak. Strictly stronger - and honest. No fresh grounding-check (removes an over-strong - measurement assumption, adds none about compiler behaviour). -2. Plan amended (Task 4/5 + the harness step): the harness sums - *every* `ailang_rc_stats:` line, not just the first. -3. Code repair applied (mechanical, decided-invariant): the - `build_link_run_embed` parse now sums all stat lines; T4/T5 - docstrings + assertion messages corrected to the global model. - Verified GREEN by the Boss: `embed_record_e2e` 2/2 (own+borrow), - gate 10/10, byte-pin 1/1, forwarder 3/3; regression-green set - (`embed_e2e`, `embed_staticlib_cli`, `embed_export_hash_stable`, - `design_schema_drift`, `docs_honesty_pin`) unmodified-green. - -Plan-pseudo-code defects the orchestrator self-corrected inline -(`feedback_plan_pseudo_vs_reality` class — `(app Ctor …)` → -`(term-ctor …)` construction, library-`print` non-resolution, -module==file-stem) are noted in Concerns; a planner Step-5 scrub -item is warranted (recorded by the Boss outside this journal). - -audit owns the bench + architect milestone-close (spec Testing -items 8/9). This run asserts only the test-suite GREEN state. - -## Re-dispatch (2026-05-18 — `task_range:[6,7]`, Tasks 6–7 GREEN) - -Re-dispatched from HEAD `d5c565d` (the committed PARTIAL 5/7 + Boss -spec-defect repair). Tasks 1–5 NOT re-implemented or re-verified. - -- iter embedding-abi-m3.1.6 (DONE): the substantive freeze. - DESIGN.md §"Embedding ABI" — both "provisional until M3" sentences - rewritten to one-way-freeze wording (`:2280-2282` → embedding ABI - accepts `Int`/`Float`/single-ctor record + "**Frozen as of M3**"; - `:2293-2295` → "**frozen as of M3** (see "Frozen value layout" - below)") and the new `### Frozen value layout (M3 — one-way - commitment)` SSOT subsection inserted before `## Data model` (byte - table `p-8` header / `p+0` tag / `p+8+i*8` fields, size `8+n*8`, - construction/ownership/free contract). Lockstep `// FROZEN ABI` - pointers added at the three independent encoders: `runtime/rc.c` - layout comment block, `match_lower.rs` `lower_ctor` `:107` size - site, `drop.rs` `:88` tag-load + `:113` field-offset. Three stale - rustdocs corrected — the two plan-named (`codegen/lib.rs:178`, - `ailang-core/ast.rs:211`) PLUS the Concerns carry-forward - `ailang-check/lib.rs:448-450` ("scalar-only…every ADT rejected" → - frozen-as-of-M3, Boss constraint 2). docs_honesty_pin GREEN - baseline AND after the rewrite (5/5 both); the pinned line - ("Export parameters are written **bare**…") shifted `:2297`→`:2299` - but stays byte-verbatim+contiguous — the pin asserts content not - line, so pin-safe (planner Step-5 item-6 family honoured). Step-7 - enforceability: perturbed `match_lower.rs` `size_bytes` `8`→`16` - (the unanchored sed-class literal hit both `:107` and `:400`; both - feed the byte-pin through `lower_ctor`'s heap path), byte-pin went - RED (`0 passed; 1 failed`, IR emitted `i64 32` not `i64 24`), - restored via `git checkout -- crates/ailang-codegen/src/match_lower.rs` - (working-tree restore only — main HEAD untouched; the single - legitimate git history-touching op of the run), byte-pin GREEN - again (1/1). The freeze is proven enforceable, not aspirational. -- iter embedding-abi-m3.1.7 (DONE): milestone-close verification - gate (no code change). Regression-green-unmodified set GREEN - (`embed_e2e` 1, `embed_rc_accounting_tsan` 2, `embed_swarm_tsan` - 1, `embed_staticlib_cli` 2, `embed_staticlib_alloc_guard` 2, - `print_no_leak_pin` 1; `design_schema_drift` 8/8; - `embed_export_hash_stable` 1/1). Full workspace **639 passed, 0 - failed**, zero compile errors; M3 net-new present + GREEN - (`embed_record_layout_pin` 1, `embed_export_gate` 10 = orig 6 + 4, - `embed_staticlib_lowering` 3 = orig 2 + 1, `embed_record_e2e` 2), - nothing pre-existing removed/weakened. `round_trip` 2/2 (new - `.ail` fixtures round-trip; no Form-A surface added). - -## Concerns (re-dispatch addendum) - -- iter embedding-abi-m3.1.7: PLAN TRANSCRIPTION DEFECT (class: - "plan pseudo-code vs reality" — same family as the iter's existing - `(app Ctor)`→`(term-ctor)` / module==file-stem Concerns). Plan - Task-7 Step 1 names `cargo test -p ailang-check --test - embed_export_hash_stable`, but that test target lives in - **`ailang-core`** (`crates/ailang-core/tests/embed_export_hash_ - stable.rs`), not `ailang-check`. The substantive requirement - ("no schema field added; hashes/drift byte-unchanged") is - unaffected — ran the pin in its actual crate, 1/1 GREEN. Crate - name corrected at execution; observation not correctness. The - recon "named this exact set" claim was wrong for this one target - (recurring `feedback_grounding_check_misses_insource_tests` / - `feedback_plan_pseudo_vs_reality` family — a planner Step-5 scrub - item, recorded here for Boss visibility). -- iter embedding-abi-m3.1.6: the plan places the Step-5 match_lower.rs - `// FROZEN ABI` comment edit and the Step-7 `git checkout -- - match_lower.rs` enforceability restore in the SAME task on the SAME - file. The checkout necessarily reverts the uncommitted Step-5 - comment along with the perturbation (it restores to HEAD `d5c565d`, - where neither exists). Re-applied the Step-5 comment after the - enforceability demo so Step 5 stays satisfied — this is a plan - step-ordering wrinkle, not a defect: the freeze demo MUST run on a - tree where the only delta is the perturbation, and the comment is - re-addable post-demo. Recorded for planner-scrub awareness. - -## Known debt (re-dispatch addendum) - -- `llvm_scalar` identifier still slightly narrower than its post-M3 - behaviour (carried from the first dispatch's Concerns) — out of - scope for Tasks 6/7; the doc-comment disambiguates. Unchanged. -- Bench + architect milestone-close (spec Testing items 8/9) remain - the `audit` skill's job — NOT this implement run, as the original - account and Boss adjudication both record. audit is the next - pipeline step. - -## Files touched (re-dispatch addendum — Tasks 6–7) - -Modified (Task 6; all uncommitted in the working tree): -- `docs/DESIGN.md` (two "provisional until M3" rewrites + the new - "### Frozen value layout" SSOT subsection) -- `runtime/rc.c` (lockstep `// FROZEN ABI` comment in the layout - block) -- `crates/ailang-codegen/src/match_lower.rs` (`lower_ctor` `:107` - lockstep comment) -- `crates/ailang-codegen/src/drop.rs` (`:88` tag-load + `:113` - field-offset lockstep comments) -- `crates/ailang-codegen/src/lib.rs` (`Target::StaticLib` rustdoc - "provisional until M3" → frozen) -- `crates/ailang-core/src/ast.rs` (`FnDef.export` rustdoc - "provisional until M3" → frozen) -- `crates/ailang-check/src/lib.rs` (`ExportNonScalarSignature` - rustdoc "scalar-only…every ADT rejected" → frozen — Concerns - carry-forward) - -Task 7: verification only, no files. - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-embedding-abi-m3.1.json diff --git a/docs/journals/2026-05-18-iter-embedding-abi-m3.tidy.md b/docs/journals/2026-05-18-iter-embedding-abi-m3.tidy.md deleted file mode 100644 index 63a4444d..0000000 --- a/docs/journals/2026-05-18-iter-embedding-abi-m3.tidy.md +++ /dev/null @@ -1,97 +0,0 @@ -# iter embedding-abi-m3.tidy — close the two M3-audit doc-honesty DRIFT items - -**Date:** 2026-05-18 -**Started from:** 4fc34705528333de765251be9c26f7bb3182e175 -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -Post-audit docs-honesty tidy that reconciles the two DRIFT items the -M3 milestone-close audit (`docs/journals/2026-05-18-audit-embedding-abi-m3.md`, -committed `b8a60b1`) routed here, in the M2.tidy `[medium]+[low] -doc-honesty → tidy` precedent shape. Two surgical, independent text -edits, zero language/checker/codegen behaviour change. (1) DESIGN.md -§"Embedding ABI" `:2300-2301` — replaced *only* the contradicted -M1-era parenthetical (`modes apply only to heap-shaped types, which -the scalar-only rule above forbids at an export boundary anyway`), -which since the M3 frozen-layout SSOT accepts a single-ctor record -export type contradicted both that freeze and the `(own/borrow -(con T))` mode contract at `:2345-2351`; the replacement states the -present-tense truth and points at the frozen-value-layout SSOT. The -`docs_honesty_pin.rs:135`-pinned sentence ("Export parameters are -written **bare**: a scalar type carries no `own`/`borrow` mode", -`norm()`-whitespace-collapsed) shares physical line `:2300` with the -stale clause — the edit kept every pinned word contiguous; the pin -stayed GREEN (5 passed) across the edit, no RED→restore needed. -(2) `crates/ailang-codegen/src/lib.rs:608-610` — comment-only -honesty fix: the forwarder-body comment claimed the gate guarantees -`Int`/`Float`-only, false post-M3 (a gate-guaranteed single-ctor -record lowers to `ptr`); the IR byte-pin + 3 forwarder-IR pins -stayed byte-identically GREEN before and after, the guard that no -generated code moved. Four standing pins at exact recon baseline -(`docs_honesty_pin` 5, `design_schema_drift` 8, -`embed_record_layout_pin` 1, `embed_staticlib_lowering` 3); full -workspace 639/77 byte-unchanged from the M3-DONE baseline. Diff is -exactly the two intended files. This closes the M3 audit's -`[medium]`+`[low]` doc-honesty drift; bench was already carry-on at -the audit (decisively causally exonerated by byte-identical -generated IR — M3 changed no executable-path codegen; NO ratify). -No audit/fieldtest gate after this tidy — the four pins + the -639/77 baseline ARE the regression coverage (M2.tidy precedent). - -## Per-task notes - -- iter embedding-abi-m3.tidy.1: reconcile the contradicted DESIGN.md - parenthetical (`[medium]`). Replaced only the parenthetical on - `:2300-2301`; pinned bare-scalar sentence kept verbatim+contiguous - (`docs_honesty_pin` 5 passed, `design_schema_drift` 8 passed). -- iter embedding-abi-m3.tidy.2: honesty-fix the stale forwarder-body - comment (`[low]`). Comment-only edit at `lib.rs:608-610`; - `embed_record_layout_pin` 1 + `embed_staticlib_lowering` 3 passed - byte-identically before and after (no IR delta); full codegen - crate green. -- iter embedding-abi-m3.tidy.3: milestone-close verification gate. - Four standing pins at recon baseline; workspace 639/77 unchanged; - diff exactly two files (DESIGN.md, codegen/src/lib.rs). One - plan-vs-actual grep discrepancy recorded under Concerns (Step 1 - pattern collides with the plan's own Task 2 replacement text; - substantive intent satisfied via the discriminating fragment). - -## Concerns - -- Task 3 Step 1's verification grep pattern `guarantees every param` - is a plan defect: it is a substring of the plan's *own* Task 2 - prescribed NEW replacement text (`// The check-side export gate - guarantees every param and`), so it cannot return "no output" as - the plan's Step 1 expects without contradicting Task 2. The - substantive intent of Step 1 — the stale `Int`/`Float`-only - comment is gone — was verified via the discriminating fragments - instead: the stale `check-side gate (Task 4)` marker is ABSENT - (grep exit 1) and the stale `guarantees every param` + immediately - following `and the ret are \`Int\`/\`Float\`; map` two-line claim - is ABSENT (Pzo exit 1), while the new correct M3 comment with - `single-constructor` record framing is PRESENT (`:609`). The - implementation followed Task 2's verbatim replacement exactly as - mandated; the retained words "guarantees every param" are correct - (the gate genuinely still guarantees every param — the M3 change - is *what* it guarantees, not *whether*). No edit was made to - dodge the grep; the plan's pattern is the defect, not the code. - -## Known debt - -(none — the two audit-flagged drift items are both closed; no -deferred follow-up.) - -## Blocked detail - -(none — DONE.) - -## Files touched - -- `docs/DESIGN.md` — §"Embedding ABI" parenthetical replaced (`:2300-2301`) -- `crates/ailang-codegen/src/lib.rs` — `Target::StaticLib` forwarder-body comment honesty (`:608-610`) - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-embedding-abi-m3.tidy.json diff --git a/docs/journals/2026-05-18-iter-emit-ir-staticlib.md b/docs/journals/2026-05-18-iter-emit-ir-staticlib.md deleted file mode 100644 index 66639cc..0000000 --- a/docs/journals/2026-05-18-iter-emit-ir-staticlib.md +++ /dev/null @@ -1,74 +0,0 @@ -# iter emit-ir-staticlib — `ail emit-ir --emit=staticlib` (M1 fieldtest spec_gap#2) - -**Date:** 2026-05-18 -**Started from:** 03493c9b316038f6bfe7f86678457bfa223451a5 -**Status:** DONE -**Tasks completed:** 4 of 4 - -## Summary - -Resolves M1 fieldtest [spec_gap]#2 ("no public path emits the -staticlib IR though Decision 5 makes IR readability a first-class -LLM affordance") by adding `--emit=staticlib` to `ail emit-ir`, -symmetric with the M1-shipped `ail build --emit=staticlib`. A -one-line codegen convenience `lower_workspace_staticlib(ws)` -(mirroring `lower_workspace`) routes through the M1-audited, -unchanged `Target::StaticLib` forwarder-emission path; the CLI -arm gains an `emit: String` clap field (symmetric with -`Cmd::Build`'s) and a zero-export guard byte-identical to -`build_staticlib`'s. DESIGN.md is *widened* (never narrowed) with -a present-tense affordance sentence in §"Embedding ABI (M1)" and a -synopsis correction that also fixes a pre-existing M1 omission for -`ail build --emit=staticlib`. No new doc pin (architecture -decision): the new E2E file pins the behaviour; the existing -`docs_honesty_pin` (5 tests) confirmed non-regressed. - -## Per-task notes - -- iter emit-ir-staticlib.1: codegen — added `pub fn - lower_workspace_staticlib` in `ailang-codegen/src/lib.rs` between - `lower_workspace` and `lower_workspace_inner`; delegates to - `lower_workspace_inner(ws, Gc, Target::StaticLib)`. `cargo build - -p ailang-codegen` clean. -- iter emit-ir-staticlib.2: CLI — RED-first E2E file - `crates/ail/tests/emit_ir_staticlib_cli.rs` (3 tests), observed - RED "1 passed; 2 failed" exactly as predicted; added `emit` - clap field to `Cmd::EmitIr`, destructured it in the arm, branched - the lowering call with the byte-identical zero-export guard - (Steps 3+4 honoured as a single compile unit — no cargo between - the E0027 window). GREEN: 3 passed; `embed_staticlib_cli` 2 - passed (build-side symmetry, no regression). -- iter emit-ir-staticlib.3: DESIGN.md — added the present-tense - `--emit=staticlib` affordance sentence after the M1 export - snippet; synopsis: both `emit-ir` and `build` lines gained - `[--emit=staticlib]` (the `build` correction discharges a - pre-existing M1 current-state-honesty gap). `docs_honesty_pin` - 5/5 green — no new pin, no regression. -- iter emit-ir-staticlib.4: regression gate — `ail` suite 186 - passed / 0 failed; workspace `TOTAL_PASSED=626` = pre-iter 623 - + 3 (exactly the new tests, no pre-existing test lost); scope - check clean (only the 4 in-scope paths). - -## Concerns - -(none) - -## Known debt - -(none — scoped feature fully closed; no fixture minted, no doc -pin added, all by explicit plan decision.) - -## Blocked detail - -(none) - -## Files touched - -- `crates/ailang-codegen/src/lib.rs` — `lower_workspace_staticlib` convenience -- `crates/ail/src/main.rs` — `Cmd::EmitIr` `emit` clap field + arm branch + zero-export guard -- `crates/ail/tests/emit_ir_staticlib_cli.rs` — new, 3 E2E tests (positive forwarder-IR / zero-export guard / default-exe regression) -- `docs/DESIGN.md` — §"Embedding ABI (M1)" affordance sentence + CLI synopsis (both lines) - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-emit-ir-staticlib.json diff --git a/docs/journals/2026-05-18-iter-form-a-scalar-param-mode-carveout.md b/docs/journals/2026-05-18-iter-form-a-scalar-param-mode-carveout.md deleted file mode 100644 index f79a606..0000000 --- a/docs/journals/2026-05-18-iter-form-a-scalar-param-mode-carveout.md +++ /dev/null @@ -1,73 +0,0 @@ -# iter form-a-scalar-param-mode-carveout — form_a.md scalar-parameter mode carve-out (docs-honesty tidy) - -**Date:** 2026-05-18 -**Started from:** 4c266a64b478c6a6106958d577faea5ac30dfa3e -**Status:** DONE -**Tasks completed:** 4 of 4 - -## Summary - -Docs-honesty tidy in the `docs-honesty-lint` class resolving the -fieldtest `[friction]` + `[spec_gap]#1` from -`docs/specs/2026-05-18-fieldtest-embedding-abi-m1.md`: -`crates/ailang-core/specs/form_a.md` and `docs/DESIGN.md` §"Embedding -ABI (M1)" stated an unconditional "every `(fn ...)` parameter MUST -carry a mode" rule that contradicts the shipped checker behaviour -(scalar `Int`/`Bool`/`Unit`/`Str` params take — and require — no -mode, mirroring the existing return-type carve-out). RED-first: the -new pin `form_a_scalar_param_carveout_present_and_old_rule_absent` -was written asserting the post-edit state and failed correctly -(Task 1), then the four form_a.md sites + one DESIGN.md insertion -turned it green (Tasks 2–3). Zero language/checker/codegen change; -exactly three files touched. Workspace suite 622 → 623 (delta = +1, -the new pin), zero failures. - -## Per-task notes - -- iter form-a-scalar-param-mode-carveout.1: RED pin — - `use ailang_core::FORM_A_SPEC;` added + new `#[test] fn - form_a_scalar_param_carveout_present_and_old_rule_absent` - (4 ABSENT + 4 PRESENT form_a + 1 PRESENT DESIGN assertions); - failed at the first ABSENT check (site 3) as designed; 4 - pre-existing pins unaffected (4 passed / 1 failed). -- iter form-a-scalar-param-mode-carveout.2: form_a.md four sites — - site 1 `### Function` prose, site 2 grammar-block comment, site 3 - Schema-invariants item 1 (+ checker-behaviour rationale), site 4 - Pitfalls bullet split into heap-shaped-forget + inverse - scalar-over-wrap. Pin then failed only on the DESIGN.md assertion; - spec_drift all 8 green (not a lockstep partner, confirmed). -- iter form-a-scalar-param-mode-carveout.3: DESIGN.md §"Embedding - ABI (M1)" — inserted the bare-scalar export-param rule + the - corpus-grounded `step` Form-A snippet (byte-identical to - `examples/embed_backtest_step.ail` lines 3-11). Pin fully green; - docs_honesty_pin 5 passed / 0 failed. -- iter form-a-scalar-param-mode-carveout.4: regression gate — - `cargo test -p ailang-core` 112 passed / 0 failed; workspace - TOTAL_PASSED=623 (baseline 622, +1 expected) / 0 failed; zero - src-code diff under ailang-check / ailang-codegen / - ailang-core/src / ailang-surface. - -## Concerns - -(none) - -## Known debt - -(none — site 5 form_a.md:521-524 already-correct few-shot annotation -left verbatim by design; the carve-out text deliberately reuses its -heap-shaped/primitive vocabulary so the document is internally -consistent end to end.) - -## Blocked detail - -(none — DONE) - -## Files touched - -- `crates/ailang-core/specs/form_a.md` (4 carve-out sites) -- `docs/DESIGN.md` (1 §"Embedding ABI (M1)" insertion) -- `crates/ailang-core/tests/docs_honesty_pin.rs` (1 `use` + 1 new `#[test]`) - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-form-a-scalar-param-mode-carveout.json diff --git a/docs/journals/2026-05-18-iter-prose-loop-binders.1.md b/docs/journals/2026-05-18-iter-prose-loop-binders.1.md deleted file mode 100644 index 4c49a2a..0000000 --- a/docs/journals/2026-05-18-iter-prose-loop-binders.1.md +++ /dev/null @@ -1,72 +0,0 @@ -# iter prose-loop-binders.1 — Form-B `loop` renders binders as a parenthesised init-list on the keyword - -**Date:** 2026-05-18 -**Started from:** 6533134dab888605c24fb2e50a65fcbf99c1897e -**Status:** DONE -**Tasks completed:** 2 of 2 - -## Summary - -Projection-only single-iteration milestone in `ailang-prose`. The -`Term::Loop` arm of `write_term` (`crates/ailang-prose/src/lib.rs`) -now renders loop binders as a parenthesised init-list on the keyword -line — `loop(acc = 0, i = 1) { … }` — instead of bare -`name = init;` statements inside the body block. The old shape -implied C/Rust re-init-every-iteration semantics; the new shape -mirrors the adjacent (untouched) `Term::Recur` arm's -`enumerate()` + `", "`-separator idiom and renders init expressions -at `level` (they sit on the keyword line, not the indented block). -RED-first: two committed `examples/*.prose.txt` snapshot fixtures -plus two `check_snapshot`-calling `#[test]` fns were added and -verified failing against the old renderer before the arm rewrite -made them pass. Loop/recur AST, Form-A, JSON-AST, typecheck, -codegen and the Form-A↔JSON round-trip invariant are untouched by -construction (single render-arm body rewrite, no signature change, -no caller touched). - -## Per-task notes - -- iter prose-loop-binders.1.1: RED — created - `examples/loop_sum_to_run.prose.txt` (multi-binder + non-recur - exit) and `examples/loop_forever_build.prose.txt` (single-binder - recur-only) with the exact Option-A bytes from the plan; appended - `snapshot_loop_sum_to_run` + `snapshot_loop_forever_build` after - `snapshot_ordering_match` in `tests/snapshot.rs`. `cargo test -p - ailang-prose snapshot_loop` → `0 passed; 2 failed; 8 filtered - out` (filter resolved to exactly the two new tests; both panic - with the expected `prose snapshot mismatch` message). RED - confirmed. -- iter prose-loop-binders.1.2: GREEN — rewrote the `Term::Loop` - arm at `lib.rs:938-955` verbatim per the plan (parenthesised - init-list, init at `level`, stale "not yet designed" comment - removed). `cargo test -p ailang-prose snapshot_loop` → `2 - passed; 0 failed`; full `ailang-prose` suite → `10 passed; 0 - failed` (no pre-existing snapshot regressed); both `ail prose | - diff` CLI byte-matches → `MATCH`; post-cleanup `git status - --porcelain` is EXACTLY the four expected paths. - -## Concerns - -(none) - -## Known debt - -(none — projection-only single-arm rewrite, fully gated by -whole-file byte-equality snapshots + live-CLI diff) - -## Files touched - -- `crates/ailang-prose/src/lib.rs` — `Term::Loop` arm of - `write_term` rewritten (single hunk, lines 938-953) -- `crates/ailang-prose/tests/snapshot.rs` — two `#[test]` fns - appended after `snapshot_ordering_match` -- `examples/loop_sum_to_run.prose.txt` — new committed snapshot -- `examples/loop_forever_build.prose.txt` — new committed snapshot - -## Blocked detail - -(n/a — Status DONE) - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-prose-loop-binders.1.json diff --git a/docs/journals/2026-05-18-iter-remove-mut-var-assign.1.md b/docs/journals/2026-05-18-iter-remove-mut-var-assign.1.md deleted file mode 100644 index 5444614..0000000 --- a/docs/journals/2026-05-18-iter-remove-mut-var-assign.1.md +++ /dev/null @@ -1,63 +0,0 @@ -# iter remove-mut-var-assign.1 — atomic removal of `mut`/`var`/`assign` - -**Date:** 2026-05-18 -**Started from:** f355899fdf6071c0aeb7de182e93c08568ca743a -**Status:** DONE -**Tasks completed:** 6 of 6 - -## Summary - -The local-mutable-state construct (`Term::Mut` / `Term::Assign` / -`struct MutVar`, the `mut`/`var`/`assign` Form-A keywords, the 4 -`mut` `CheckError` variants, the `mut_scope_stack` synth threading, -the two `lower_term` arms) is removed from AILang entirely and -atomically — AST + surface + checker + codegen + DESIGN.md + -fixtures + drift trio + carve-out + roadmap cut together so the -schema is honest at every commit. `loop`/`recur` + `let`/`if` are -the surviving forms. The shared codegen alloca machinery survives -(loop needs it); its now-misnamed `mut_var_allocas` field is renamed -`binder_allocas` and the shared `Term::Lam` escape guard is -simplified to `!loop_stack.is_empty()` with the loop half kept -byte-equivalent. Removal is symmetric to the additive mut.1 -introduction: serde is tag-based so no surviving module's -canonical-JSON hash moves; `loop`/`recur` codegen is byte-identical -(the rename is representation-only). Full `cargo test --workspace` -605/0; loop/recur non-regression hard gates all green -(55 / 500000500000 / infinite-compiles / the -`lambda_capturing_loop_binder` pin). The behaviour-preservation -proof is executable: `mut_counter`/`mut_sum_floats` still print `55` -after the faithful `let`/`if` rewrite. The "removal made executable" -gate is the new `mut_removed_pin.rs` (4 must-fail pins: `mut`/ -`assign` keyword rejected, `{"t":"mut"}`/`{"t":"assign"}` serde -unknown-variant). - -## Per-task notes - -- remove-mut-var-assign.1.1: created `crates/ailang-surface/tests/mut_removed_pin.rs` — 4 RED→GREEN must-fail pins; harness compiled against the real `ailang_surface::parse` + `ailang_core::ast::Term` serde public API unchanged (no adaptation needed). RED-verified (4 fail with the exact expected messages), GREEN after T2. -- remove-mut-var-assign.1.2: the atomic Rust cut. Deleted `Term::Mut`/`Term::Assign`/`MutVar` from ast.rs; every exhaustive no-`_` `Term::Mut`/`Term::Assign` match arm across 17 source files; parse.rs dispatch + `parse_mut`/`parse_assign` + reservation + grammar EBNF (incl. the line-43 `| mut-term | assign-term` alternation that referenced the deleted productions); print.rs arms; the 4 `CheckError` variants + `code()`/`ctx()`; the synth arms + `Term::Var` mut-scope lookup; the `mut_scope_stack` param removed from `synth` + every internal/external/test caller; escape guard simplified to `loop_stack`-only; `lower_term` Mut/Assign arms deleted; `mut_var_allocas`→`binder_allocas` renamed at all surviving sites + lambda.rs save/restore. Dead `is_supported_mut_var_type` deleted. `cargo build --workspace` 0 errors + 4 Task-1 pins GREEN (the only T2 gate). NO `_ =>` wildcard introduced. -- remove-mut-var-assign.1.3: deleted `mut_typecheck_pin.rs` (whole file); dropped the 4 mut rows in `ct1_check_cli.rs` (kept the surviving loop-binder row, renamed the test + reworded doc honestly); dropped the mut/assign entries + match arms in the drift trio (`schema_coverage`, `design_schema_drift`, `spec_drift`). -- remove-mut-var-assign.1.4: DESIGN.md hard-delete — the `Term::Mut`/`Term::Assign` JSONC blocks, the mut post-schema paragraph (loop-binder half kept + reworded standalone, present-tense, points at the live loop-recur spec), the "Local mutable state" feature bullet, the accumulator-idiom paragraph. Clause-3 rationale ("iterated-mutable-state reasoning", 99/107) untouched. Zero construct residue (only `resolve names + assign hashes` survives — a hashing-pipeline verb, plan-allowed). -- remove-mut-var-assign.1.5: deleted the 3 rejection-probe fixtures; rewrote `mut.ail` (6 fns), `mut_counter.ail`/`mut_sum_floats.ail`, `mut-local_1..4` to the plan's exact faithful `let`/`if` shapes; present-tense docs, no nostalgia. Verified: `mut_counter`/`mut_sum_floats`→55, factorial→120, horner→18, `classify 22`→2, `has_small_factor 91`→true; zero `mut`/`var`/`assign` construct tokens remain. -- remove-mut-var-assign.1.6: `carve_out_inventory.rs` 18→12 (EXPECTED + header doc) AND deleted the 5 orphaned mut `.ail.json` carve-out files the test's disk-inventory assertion requires gone; roadmap false-foundation `**Progress.**` mut-local sub-bullet deleted (milestone header kept). FINAL full-suite green gate 605/0; loop/recur non-regression all green; `roundtrip_cli` PASS. - -## Concerns - -- DONE_WITH_CONCERNS (T2/T3/T6, recon-undercount class, `feedback_plan_pseudo_vs_reality` / `feedback_recon_undercount`): the plan's Step-7 line-list framed several sites as "delete the *argument* at the test-only synth callers", but those test fns' *bodies* construct deleted `Term::Mut`/`MutVar`/deleted-`CheckError` types — arg-trimming alone cannot make them compile. The faithful execution (identical rationale to Task-3's `mut_typecheck_pin.rs` whole-file deletion) was to delete the pure-mut-behaviour test fns wholesale: ~10 in-source `#[cfg(test)] mod tests` fns in `ailang-check/src/lib.rs` (mut_var_resolution_*, mut_block_*, assign_*, mut_typecheck_diagnostics_have_kebab_codes, mut_var_captured_by_lambda_*, lambda_inside_mut_*), the `design_md_documents_the_accumulator_over_iteration_idiom` test fn (pins deleted DESIGN.md prose), the 5 orphaned mut `.ail.json` carve-out files (Task 5 only deleted 1 of 6; the inventory test asserts disk==EXPECTED so all 6 must go), and the `Term::Mut`/`Term::Assign` arms in two non-enumerated test helpers (`ail/tests/codegen_import_map_fallback_pin.rs` — a hard E0599 the recon missed entirely; `ail/tests/e2e.rs` doc-comments). ~30 now-dangling doc-comments/comments that cross-referenced deleted symbols (`mirrors Term::Mut`, `MutVarCapturedByLambda`, the `loop_recur_typecheck_pin.rs` RED-state doc) were reworded so the source has zero trace/post-mortem (spec acceptance). The kept e2e bodies `mut_counter_prints_55`/`mut_sum_floats_prints_55` were preserved byte-identical per plan line-57's explicit "must stay green" directive — only their lying doc-comments were corrected. None of these are gratuitous; every one is strictly entailed by the atomic removal + the plan-stated final green gate. The class recurred enough (in-source tests, drift-pin test fn, orphaned carve-out files, non-enumerated test helpers) to be a planner-recon defect worth a follow-up countermeasure, not a per-iter fix. - -## Known debt - -- `crates/ailang-core/src/workspace.rs:1620` `tmp_dir` is a pre-existing `never used` test-helper warning, NOT introduced by this iter (it surfaces only in the test-target build; `cargo build --workspace` is 0 warnings). Out of scope (no surrounding cleanup); left untouched. - -## Files touched - -Source (17): ail/src/main.rs, ailang-check/src/{builtins,lib,lift,linearity,mono,pre_desugar_validation,reuse_shape,uniqueness}.rs, ailang-codegen/src/{escape,lambda,lib}.rs, ailang-core/src/{ast,desugar,workspace}.rs, ailang-prose/src/lib.rs, ailang-surface/src/{parse,print}.rs -Tests (modified): ailang-core/tests/{carve_out_inventory,design_schema_drift,schema_coverage,spec_drift}.rs, ail/tests/{ct1_check_cli,codegen_import_map_fallback_pin,e2e}.rs, ailang-check/tests/loop_recur_typecheck_pin.rs -Tests (created): ailang-surface/tests/mut_removed_pin.rs -Tests (deleted): ailang-check/tests/mut_typecheck_pin.rs -Docs: docs/DESIGN.md, docs/roadmap.md -Fixtures (rewritten): examples/mut.ail, examples/mut_counter.ail, examples/mut_sum_floats.ail, examples/fieldtest/mut-local_{1,2,3,4}*.ail -Fixtures (deleted, 8): examples/fieldtest/mut-local_{5,6}*.ail, examples/test_mut_{assign_out_of_scope,assign_outside_mut,assign_type_mismatch,nested_shadow_legal,var_captured_by_lambda,var_unsupported_type}.ail.json - -## Stats - -bench/orchestrator-stats/2026-05-18-iter-remove-mut-var-assign.1.json diff --git a/docs/journals/2026-05-19-audit-design-ledger-formal-links.md b/docs/journals/2026-05-19-audit-design-ledger-formal-links.md deleted file mode 100644 index 136d609..0000000 --- a/docs/journals/2026-05-19-audit-design-ledger-formal-links.md +++ /dev/null @@ -1,117 +0,0 @@ -# audit — milestone close: formal cross-links in the design/ ledger - -**Date:** 2026-05-19 -**Scope:** `2ee4087..8ad91e7` (5 commits — 3 spec commits incl. 2 corpus-grounded amendments, 1 plan, 1 iter) -**Status:** CLEAN (no drift; bench pristine; no tidy) - -## Architect (clean — drift_items: none) - -Every claimed commitment and invariant verified positively: - -- **clause-5** (`design_body_links_are_durable_and_resolve`) implements - all four predicates (resolves-from-containing-file; durable-tier; - no-fragment; fence-skip via toggle-`strip_fences`); RED-first - synthetic-vector mechanism honest; scope is `design/contracts/` + - `design/models/`, excludes `design/INDEX.md` per commitment 4. -- The 7 prose conversions (8 link tokens) match the spec's - Acceptance-3 closed enumeration verbatim; file-relative arithmetic - correct (siblings → bare filename; `crates/` reach via `../../`; - cross-`design/`-subtree via `../models/`). Zero `docs/` link - targets; zero `#fragment`. -- Disposition (b) on `pipeline.md:61` + `authoring-surface.md:180` - correctly applied: cross-tier pointer removed, surrounding - behavioural `ail merge-prose` prose preserved (the - authoring-surface case is slightly rephrased to keep the meaning - without the pointer). -- `honesty-rule.md` positive-half paragraph pin-safe: both - `docs_honesty_pin.rs`-pinned phrases byte-identical (L14 and now - L25, shifted from L19 by the +6 lines); the - `design_index_pin.rs clause-5` mention is a nominal identifier in - the existing "Ratified by:" footer style (no `[](...)` form, no - navigational pointer) — consistent with the spec's §Scope - navigational-vs-nominal definition. No clause-3 PHRASE tripwire, - no Sweep-1 anchor. -- **Commitment 4** honored: `design/INDEX.md` byte-unchanged - (zero-line diff vs HEAD); `design_index_pin.rs` clauses 1–4 source - byte-unchanged (the only `-` lines are the two-line `//!` header - rewrite delivered by Task 1 Step 5 itself). -- **Commitment 7** honored: - `docs/journals/2026-05-19-design-decision-records.md` - byte-unchanged. -- **Composition invariant** holds on the post-milestone tree: - clause-3 GREEN ∧ clause-5 GREEN simultaneously for every - `design/contracts/*.md` (independently verified by running both - in the same suite). -- **Out-of-scope set untouched:** `data-model.md` 38/66/79/206/226 - verified inside ```jsonc fences (30–87, 203–228); - `embedding-abi.md:51` "frozen value layout below specifies" - untouched; intra-file `above/below` directional prose untouched - (sample-checked `float-semantics.md:70` "The NaN-spelling caveat - above" still intra-file). -- **No commitment-2 violation:** no agent/SKILL reading list adds a - `docs/PROSE_ROUNDTRIP.md` design/-link target. The sole remaining - mention is `design/INDEX.md:43` — prose mention only (no `[](...)` - form), outside clause-5's scope and not a navigational link. -- `bench/architect_sweeps.sh` exit 0 ("All five sweeps clean"); - Sweep-1 shares no scope or regex with clause-5 (orthogonality - asserted in spec confirmed empirically). - -Two minor non-drift observations recorded without raising drift: - -- `design/INDEX.md:43`'s prose mention of `docs/PROSE_ROUNDTRIP.md` - is now the sole remaining `docs/` reference in `design/`. The - spec consciously scopes clause-5 to body files only; this is a - nominal mention in the spine, not a formal link — commitment-2 - governs *formal links*. Recorded for future-tightening awareness, - not as drift. -- The roadmap entry stands `[~]` IN FLIGHT; flipping to `[x] - CLOSED` is the standard post-audit ratification step, executed - in this same done-state commit. Not drift. - -## Bench (check.py exit 0; compile_check exit 0; cross_lang exit 0) - -``` -check.py 63 metrics: 0 regressed (within tolerance), 0 improved beyond tolerance, 63 stable; exit 0 -compile_check 24 metrics: 0 regressed, 0 improved beyond tolerance, 24 stable; exit 0 -cross_lang 25 metrics: 0 regressed, 0 improved beyond tolerance, 25 stable; exit 0 -aggregate 0 / 0 / 0 -``` - -Cleanest possible bench outcome. Two metric movements were flagged -in the per-metric output (1 `regressed` and 1 `improved` in the -check.py summary line) but both are within their tolerance bands — -the `ok` status the script prints means the threshold check passed. -No metric exceeds tolerance; baseline pristine; no -`--update-baseline` and no paired JOURNAL ratify entry warranted. - -This matches the spec's expectation precisely: the milestone makes -**zero codegen / runtime / IR-emitting changes** (every code edit is -either a new test, a `//!` doc comment, or design/ prose). The -firings inside the bands are the standing P2 environmental jitter -already familiar from the M2/M3/M5/rolesplit closes. - -## Resolution (orchestrator) - -- **Architect:** carry-on. Zero drift items. -- **Bench:** carry-on. All three scripts exit 0; baseline pristine; - no ratify warranted. -- **No tidy iteration.** No fix path. -- **No fieldtest.** The milestone touches no authoring surface (no - `.ail` program, no language construct, no compiler/checker/codegen - path) — spec's reasoned exclusion stands. -- **Milestone closes immediately.** Done-state procedure (roadmap - `[~]` → `[x]`, WhatsNew append, notify) runs in this same - cohesive commit. - -This is the cleanest milestone close pattern in the project's -recent history: zero spec defects surviving to audit (the two were -caught and amended at planner-recon and plan-corpus-fetch time, -before any byte moved), zero drift, zero bench movement, zero -re-loop. The two amendment cycles (clause-6 + cross-ref definition; -clause-5 fence-skip + closed convert-set enumeration) were the -discipline working — recon and corpus-fetching surfaced gaps the -brainstorm-sample missed, the spec was grounded properly *before* -implementation, and the implementation then ran clean 5/5 with one -non-blocking concern (a planner self-review-item-8 arithmetic miss -on the `//!` header rewrite line count — substantive assertion -unaffected, recorded as lesson). diff --git a/docs/journals/2026-05-19-audit-design-md-rolesplit.md b/docs/journals/2026-05-19-audit-design-md-rolesplit.md deleted file mode 100644 index e6392ba..0000000 --- a/docs/journals/2026-05-19-audit-design-md-rolesplit.md +++ /dev/null @@ -1,171 +0,0 @@ -# audit — milestone close: DESIGN.md → design/ role-split - -**Date:** 2026-05-19 -**Scope:** `deeffb1..176821c` (spec a64b2cc/314e5e4, plan deeffb1, iter 176821c) -**Status:** DRIFT (one tidy iteration) + bench causally-exonerated (baseline pristine) - -## Architect (drift_found — one [medium] spirit-finding; relocation faithful) - -**What holds:** relocation byte-faithful at `###` granularity (MIXED -Decisions, dual-link frozen-value-layout, source-link-only -mangling/env/qualified-xref, rewritten honesty-rule all match the -spec Appendix vs `git show deeffb1:docs/DESIGN.md`); build-atomicity -holds (`design_schema_drift.rs` `include_str!` retargeted, slicer -removed exactly as spec'd; OQ7 cite deleted); `design_index_pin.rs` -4/4 GREEN; tree-wide live-ref grep clean; `honesty-rule.md` -present-tense, names `docs/journals/` as the rationale home, both -`docs_honesty_pin.rs:70,72` phrases verbatim+contiguous; agent -contracts coherent (no contract instructs reading a missing file). - -**Drift:** -- `[medium]` `design/contracts/typeclasses.md:182,199,280-317`, - `str-abi.md:23,38-47`, `scope-boundaries.md:17,42` — faithfully - migrated but contract-class-violating decision-record/history/ - cross-milestone-amendment prose ("An earlier draft committed - to…", "Milestone 23/24 amends the above", "was retired at iter - mq.3/ctt.3", iter/date provenance stamps, "deliberately deferred", - "NOT in milestone 22", "new-baseline decision"). Dodges the 6 - *literal* clause-3 markers (case + wording variance: "An earlier - draft" ≠ "an earlier draft"; "retired at iter" ≠ "retired in - iter") yet is exactly the relitigation-guard content the split's - *spirit* sends to journals. -- `[low/known-debt]` `design/contracts/float-semantics.md` - stale-direction `(see "Str ABI" below …)` — the prose moved to - `str-abi.md`; pre-existing, faithfully migrated; wrong-direction - anchor inside the ledger whose reason to exist is doc honesty. - -**Routed-item adjudication (both):** strippable doc-archaeology to -**tidy, NOT ratify**. The advisory `architect_sweeps.sh` Sweep-1 -exit-1 on `str-abi.md:23` is *correctly* diagnosing real -history-anchor residue (Boss-confirmed byte-identical to -DESIGN.md@deeffb1:2062-2065 — pre-existing, faithfully migrated, not -split-introduced; the iter/date stamps are journal-class metadata, -the present-tense contract is the signature itself). - -## Bench (check.py exit 1; compile_check 24/24 exit 0; cross_lang 25/25 exit 0) - -6 `check.py` firings: `bench_list_sum.{gc_s,bump_s}`, -`bench_hof_pipeline.gc_s`, `bench_list_sum_explicit.bump_s`, -`latency.{explicit,implicit}_at_rc.max_us`. - -**Bencher verdict — causally-exonerated, decisive on byte-evidence.** -H0 (no causal mechanism) supported: emitted IR **and** final `-O2` -native bench binaries **byte-identical** HEAD `176821c` vs -pre-milestone `dd5b183` for every firing fixture (sha256 + `cmp -s` -table in the bencher report). Bencher audited *all ~11* changed -code/runtime files (not just the 2 named) — every change is -comment / rustdoc / diagnostic-string-literal (`DESIGN.md §"…"` → -`design/contracts/….md`), none on an IR-emitting path; checker -diagnostic strings are type-check-time only, absent from the -binary. The 6 firings are the recurring tracked-P2 families -(`*.bump_s` environmental staleness; `*_at_rc.max_us` `-n 5` -single-sample tail jitter; `*.gc_s` Boehm-scan wall-time variance — -NOT the M5-ratified `gc_rss_kb` trio). `compile_check`/`cross_lang` -pristine corroborates (no codegen surface moved). - -**Disposition: NO-ratify / carry-on causally-exonerated — baseline -stays pristine.** Identical to the M2/M3/M5 closes; ratifying would -bake measurement variance into the baseline. No `--update-baseline`, -no paired ratify entry (none is warranted — nothing moved). - -Bench-design limitations recorded (not milestone-close actions): -`*_at_rc.max_us` is a `bench/run.sh` `-n 5` artifact (max over 5 -has no tail confidence); `implicit_at_rc` leaks by construction so -its `max_us` is inherently unstable. Both pre-existing, tracked-P2, -candidates for the standing latency-methodology rework — not this -milestone. - -## Resolution (orchestrator) - -- **Bench:** carry-on, baseline pristine, no commit beyond this - entry. Recorded causally-exonerated as M2/M3/M5. -- **Architect drift [medium]+[low]:** **fix path — one tidy - iteration** `design-md-rolesplit.tidy`. Orchestrator-decided - scope (the architect named the gap; the clause-3-widening - decision is mine): - 1. Move the decision-record/history prose out of the affected - contract files into - `docs/journals/2026-05-19-design-decision-records.md` (append — - same milestone's relitigation archive). Each removed history - sentence is replaced by its present-tense contract equivalent - (the builtin *signature* is the contract; the `(iter …)` stamp - is journal metadata). **Scope corrected on planning-time - evidence (see amendment below):** the architect's `[medium]` - was a 3-file spot-check - (`typeclasses`/`str-abi`/`scope-boundaries`); the exhaustive - plan-recon + Boss verification scan found two more contract - files carrying lowercase `iter ` provenance the - spot-check missed — `roundtrip-invariant.md:73` ("at iter - form-a.1") and `data-model.md:149,161` ("loop-recur iter 1:"). - The true strip set is **5 contract files**. - 2. Fix `float-semantics.md` stale-direction cross-ref → - `(see design/contracts/str-abi.md)`. - 3. **Re-scope the `architect_sweeps.sh` honesty sweeps from - `design/contracts design/models` to `design/contracts` only.** - Substantive reason (not effort): post-split, `design/models/` - is the *explicitly narrative* tier — a whitepaper legitimately - carries "as of milestone N" context; scanning it for - history-anchors is a category error. The honesty surface is - `design/contracts/` (the hot, test-linked tier). Update - `ailang-architect.md` + any sweep-scope doc in lockstep. - 4. **Widen `design_index_pin.rs` clause-3** so, over the - `design/contracts/` scope, it *subsumes* `architect_sweeps.sh` - Sweep-1's history-anchor regex. Invariant established: - **clause-3 GREEN ⟹ Sweep-1 finds nothing in `contracts/`** — - the in-code hard gate enforces the spirit; the advisory then - only legitimately fires on `models/` (out of its scope after - point 3). This permanently closes the spirit-vs-letter gap the - literal 6-marker list left open. **Mechanism corrected on - planning-time evidence (see amendment below):** the original - "case-insensitive iter-code regex" sketch is *unworkable* and - is rejected — a blanket case-insensitive `iter`/`milestone` - detector conflates the memory-model rule-names "Iter A"/"Iter - B" and the ordinary words "pre-existing"/"pre-set"/"pre-Boehm" - with provenance, over-firing on legitimate present-tense - contract prose. The workable design: clause-3 = a **faithful - reproduction** of Sweep-1's actual alternatives - (case-*sensitive*, digit-anchored — confirmed ZERO across all - contracts) + Sweep-1's `^[^/]*` **path-excluded** date - (so `docs/specs/2026-…` citations are not flagged) + the - audit-named decision-record **phrases** (case-insensitive — the - deliberate widening that closes the capital-variance dodge). - The lowercase `(iter )` provenance is removed by the - strip tasks (point 1), not by a fragile hard-gate regex; the - invariant holds without the rejected detector. - 5. Exit state: `architect_sweeps.sh` exit 0; the widened - (faithful-superset) `design_index_pin.rs` clause-3 GREEN; whole - `cargo test --workspace` GREEN; the decision-record journal - grows; no contract carries history residue. - -### Resolution amendment (2026-05-19, planning-time evidence) - -The tidy plan's gate-first Task 1 (RED-verification) + Boss -independent verification surfaced two defects in this Resolution as -first written — caught exactly where the planner→Boss loop is -designed to catch them, before any contract byte moved: - -- **Mechanism (point 4):** "case-insensitive iter-code regex" is - unworkable (conflates rule-names + ordinary words with - provenance). Replaced by the faithful-Sweep-1 + path-excluded-date - + phrases design above. The load-bearing invariant (clause-3 ⟹ - Sweep-1 clean) is *preserved* — it never required the blanket - detector; faithful-Sweep-1 + phrases suffices and faithful-Sweep-1 - is confirmed already ZERO across every contract. -- **Scope (point 1):** the `[medium]` was a 3-file spot-check; the - exhaustive scan found `roundtrip-invariant.md` + `data-model.md` - also carry lowercase provenance. The strip set is 5 files (the - plan's Task 5b covers the 2 additions; they are spirit-cleanup, - not gate-blocking — neither Sweep-1 nor the corrected clause-3 - trips on lowercase `iter form-a.1`, so the invariant/gate close - regardless; stripping them is the audit's *spirit* fully honored). - -This amendment is the upstream-artifact correction the discipline -mandates over patching the plan a third time (the "two+ defects in -one iteration ⇒ the upstream artifact is wrong" rule). The plan -`docs/plans/design-md-rolesplit.tidy.md` is re-derived consistently -with it. - -Milestone closes after the tidy iteration lands Boss-verified. -No fieldtest (zero authoring-surface change — the only author-facing -delta is the 2 diagnostic pointers, a doc-pointer not a language -change; recorded by reasoned exclusion, not omission). diff --git a/docs/journals/2026-05-19-audit-embedding-abi-m5.md b/docs/journals/2026-05-19-audit-embedding-abi-m5.md deleted file mode 100644 index 7fc65f1..0000000 --- a/docs/journals/2026-05-19-audit-embedding-abi-m5.md +++ /dev/null @@ -1,167 +0,0 @@ -# audit embedding-abi-m5 — milestone close - -**Date:** 2026-05-19 -**Scope:** `c75fb80..HEAD` (M3-close → m5.3 `0900f3f`), focus the M5 arc -**Outcome:** DRIFT (one doc-honesty tidy) + RATIFY (gc_rss trio, -evidenced) + causally-exonerated pre-existing P2 bench noise - -## Architect (drift review) — `drift_found` - -**Gates that PASS:** -- **Invariant 1 — PASS (hard M5 spec Acceptance gate, spec:328-330).** - `cargo metadata --no-deps` on the AILang root manifest = zero - `data-server`; `ail-embed/` is its own `[workspace]` root, not an - AILang member. No compiler crate / `runtime/` gains a - `data_server`/finance symbol. The only "finance" hits in compiler - crates are the M3 ABI export *symbol name* `backtest_step_tick` in - tests/docs — ABI knowledge, not data-server knowledge (correct). -- **M3 frozen-layout SSOT unmoved.** The one `runtime/rc.c` change in - range (`7bfa11e`) touches only the two GLOBAL `g_rc_*` *statistics* - counters; box offsets, `p-8` header, ctor tag, host-`ailang_rc_dec` - rule byte-unchanged; DESIGN.md §"Frozen value layout" correctly not - touched. -- **rc.c header-comment honesty accurate.** The - bugfix-rc-global-stats-race GREEN rewrite (rc.c ~:42-58 / ~:85-101) - matches the actual `_Atomic` decls + `atomic_fetch_add_explicit`/ - `atomic_load_explicit` sites + plain per-ctx `++`; the rc.c:~270 - drop-worklist "single-threaded" note is correctly still true - (per-call worklist). - -**Drift list (prioritised):** -- `[medium]` `docs/DESIGN.md:2358-2360` — "an additive M4 concern" - for a recursive typed-free of boxed-field records is stale: M4 was - RETIRED 2026-05-18 (never speced), will never ship. - **PRE-EXISTING** (M4-retirement era), correctly scoped out by the - M5 spec §"Out of scope" and the M4-retirement journal forward note; - NOT an M5 regression. Direction: doc-honesty tidy (drop the M4 - reference; the boxed-field record is simply export-gate-rejected). -- `[medium]` `docs/DESIGN.md` §"Embedding ABI" — DESIGN.md already - asserts "no shared mutable runtime state … data-race-free, - sanitiser-verified". After `7bfa11e` the global RC-stats fallback - *is* shared mutable runtime state under concurrency (atomic-relaxed), - and M5 (the embedding swarm) is AILang's first concurrent runtime - consumer. The doc asserts a concurrency property whose mechanism it - no longer fully/accurately describes — real silence-drift the - contract line reaches. Direction: additive present-tense note (the - global RC-stats fallback is atomic-relaxed; per-ctx counters + the - per-object refcount header stay non-atomic by design because a box - never crosses a thread and each ctx is single-thread-per-ctx) — a - current-state mirror, not a history retelling. -- `[low]` `runtime/rc.c:88` — "two unconditional `++`" is now - imprecise (the global path is a relaxed atomic add, conditional on - `_ctx == NULL`). Pre-existing, self-flagged known debt in - `2026-05-19-iter-bugfix-rc-global-stats-race.md`; a performance - aside, not an atomicity claim. Fold into the same tidy if `rc.c` - is reopened. - -Architect recommendation (adopted): ONE doc-honesty tidy iteration -reconciling the stale-M4 phrasing **and** the atomic-global-stats / -first-concurrent-consumer note (carrying rc.c:88 if `rc.c` is -reopened) — same surface, same honesty axis; do not split. - -## Bencher (bench-regression localisation) — exit 1, hypothesis-driven - -`bench/check.py` exit 1: 6 regressed / 1 improved / 56 stable on the -first audit run. Bencher built `ail` at HEAD (`0900f3f`) and -pre-atomic (`427b687`) in an isolated worktree; `ail` binary -sha256-identical; emitted IR for the gc_rss fixtures byte-identical -both sides (rc.o links into bench *binaries*, not the compiler). - -- **3× `*.gc_rss_kb +32%` (list_sum, hof_pipeline, list_sum_explicit) - — GENUINE, DETERMINISTIC, M5-ATTRIBUTABLE.** My pre-audit causal - read (allocator-orthogonal → environmental) was **refuted by - evidence**. Mechanism: `7bfa11e`'s atomic split a branch in - `ailang_rc_alloc` (called by `str.c` int→str at `print`, reached by - GC-mode programs too), spilling `%r14`/`%rbx`; a stale interior - list pointer in the new spill footprint is **conservatively - retained by Boehm**, pinning a ~32 MB dead list spine across a - collection (Boehm internal: collection #12 freed PRE 32,000,032 B - vs HEAD 32 B). Bisects exactly on the commit with IR byte-identical - — causation established, not inferred. RC/bump RSS dead-stable. -- `bench_list_sum.bump_s` / `bench_list_sum_explicit.bump_s ~+12%` — - pre-existing tracked-P2 stale-0.046/0.048s-anchor; bump path is - `g_rc_*`-orthogonal; HEAD-vs-PRE delta ~2%. NOT M5. -- `latency.{explicit,implicit}_at_rc.max_us` — 1-sample worst-case - tail; median/p99/p99.9 flat both sides; PRE frequently *worse*; - tracked-P2 `-n5` tail-jitter. Re-run minutes later swung - +65→+104% / +33→+71% — dispositively noise, NOT M5. - -## Adjudication (orchestrator) - -### RATIFY — the gc_rss trio (`7bfa11e`, evidenced) - -**Item:** `throughput.bench_list_sum.gc_rss_kb`, -`bench_hof_pipeline.gc_rss_kb`, `bench_list_sum_explicit.gc_rss_kb`, -each ~+32% (103980→137448, 103788→137768, 103772→137636). - -**Iter that moved the metric:** `7bfa11e` -(`fix(rc): GREEN — atomic global g_rc_* stats fallback counters`), -the user-approved Option-A swarm-safety fix in the -`bugfix-rc-global-stats-race` chain — necessary and load-bearing for -M5's deterministic leak-proof (reverting it reintroduces the -non-deterministic swarm-leak race). - -**Language reason (not effort):** AILang's committed memory model is -RC + uniqueness inference; **Boehm is explicitly transitional** -(Decision 9). The measurable cost of the correct, necessary atomic -fix is *exclusively* a Boehm **conservative-stack-scan -false-retention artifact** on the transitional GC allocator — the -bencher is explicit it is "a Boehm conservatism artefact, not an -RC-semantics cost"; RC and bump RSS are dead-stable; no memory is -actually leaked by AILang (Boehm conservatively retains a dead stack -slot). The mechanism is "codegen-layout-fragile — a future unrelated -rc.c edit could equally flip it back." Spending an out-of-scope -codegen change (clear the dead root before `print`) to placate the -conservative scanner of the allocator the language is migrating away -from is investing in the path being removed; the committed RC path — -the one that matters — is unaffected. Therefore ratify the new -deterministic steady-state rather than chase it. - -**Action taken:** selective baseline re-anchor — `bench/baseline.json` -lines 14/44/64 only (the 3 evidenced M5-caused `gc_rss_kb`), set to -the bencher-confirmed stable HEAD values (137448 / 137768 / 137636); -every other metric byte-pristine (a wholesale `--update-baseline` -would have absorbed the unrelated P2 noise and broken the -established-envelope discipline). Re-run confirms the 3 are now `ok` -(±0.3% vs the new anchor, ≪ 5% tol). - -### NO-RATIFY / causally exonerated — the pre-existing P2 noise - -`bench_list_sum.bump_s`, `bench_list_sum_explicit.bump_s`, -`latency.explicit_at_rc.max_us`, `latency.implicit_at_rc.max_us` -remain firing (check.py still exit 1 on these). Bencher-established -NOT M5 (bump path RC-orthogonal; max_us 1-sample tail-jitter, PRE -often worse, +65→+104% swing between runs minutes apart). Baseline -left **pristine** for these — the established tracked-P2 noise -envelope, identical disposition to the M2 / M3 / prelude-decouple -milestone closes ("check.py exit 1 causally exonerated, NO ratify, -baseline pristine"). The standing P2 items (the stale bump_s anchor; -the `-n5` max_us tail) are not M5's and are not bundled here. - -### DRIFT → tidy iteration - -The architect's `[medium]`+`[medium]`+`[low]` doc-honesty drift is -resolved via one tidy iteration `embedding-abi-m5.tidy` (planner → -implement; doc-only — DESIGN.md stale-M4 phrasing + the -atomic-global-stats / first-concurrent-consumer present-tense note -reconciling the "no shared mutable runtime state" claim; rc.c:88 -carried along). Same surface, same honesty axis, not split. - -## Friction-harvest (recorded — the P2 flat-array-decision input) - -m5.3 measured host-per-tick-FFI ≈ 658 ms / 3 192 562 ticks ≈ -**~206 ns/tick** at real EURUSD volume (one symbol-window per thread, -own ctx, the M3 ABI per tick). This is the observation the P2 -"flat-array/batch-crossing" roadmap item is now informed by: -per-tick FFI at ~200 ns is the cost a future batch crossing would -amortise; whether that is worth the cons-list-free flat-array path is -the P2 decision, now anchored on a real number rather than the -stale cons-list framing the M4 retirement obsoleted. - -## Close posture - -Invariant 1 PASS; M3 frozen layout unmoved; bench dispositioned -(gc_rss ratified evidenced, P2 noise exonerated baseline-pristine). -M5 closes after the doc-honesty tidy lands and is Boss-verified. -The baseline edit + this journal are committed together (ratify -discipline: updated baseline JSON + paired JOURNAL ratify entry). diff --git a/docs/journals/2026-05-19-design-decision-records.md b/docs/journals/2026-05-19-design-decision-records.md deleted file mode 100644 index 4b22652..0000000 --- a/docs/journals/2026-05-19-design-decision-records.md +++ /dev/null @@ -1,479 +0,0 @@ -# Design decision-records — relitigation guard (migrated 2026-05-19) - -Why-X-chosen / why-Y-rejected / deliberately-does-not-do / rollback -/ empirical-addendum prose, migrated out of the former -`docs/DESIGN.md` by the design-md-rolesplit milestone. A future -brainstorm reads this so it does not re-propose a settled-and- -rejected idea. Order preserved from the source. This file is -append-only history; it is NOT a contract surface. - - -## Decision 1: source = data, not text - -A module is a JSON object with a fixed schema. There is no parser for -free-form text. Typos in identifiers turn into hash-lookup errors that the -compiler proposes a fix for directly. - -A textual form exists (`.ail`, S-expression-like), but only as a -bidirectional projection of the JSON form. It is intended for human reviews -and diffs. - -**Canonical format:** `.ail.json` with deterministic key order. - -## Decision 2: content-addressed definitions - -Every top-level definition has a `hash` value (BLAKE3 over canonical JSON -without the `hash` field itself). References between definitions go primarily -by name — names are for readability. The hash is the canonical identity. - -Advantages: - -- Refactoring by adding new defs, not by in-place change. Old versions stay - callable until manually removed. -- Caching of typecheck results and codegen per hash. -- Diffs show exactly which def has changed. - -## Decision 4: Hindley-Milner + optional refinements - -MVP: HM with let-polymorphism. All types are inferable, but at the top level -they must always be explicitly annotated (for local reasoning). - -Later: refinement annotations that escalate to SMT. `(i: Int | i >= 0)`. They -are reserved in the AST from the start, but in the MVP they are simply passed -through as opaque strings. - -## Decision 5: emit LLVM IR as text - -Instead of `inkwell` or `llvm-sys`: AILang produces `.ll` files as strings -and hands them to `clang` for linking. - -Rationale: - -- The LLVM IR text syntax is largely stable across versions. -- No build dependency on a specific libllvm version. -- Generated code is trivially inspectable, which makes debugging much easier. -- An LLM can read the generated IR directly, which is harder with opaque - library calls. - -Trade-off: no inline optimisations through the LLVM API. We rely on -`clang -O2` as the standard pipeline. - -The rest of this section -records the *why* of Decision 6 for the audit trail; the constraints -listed below describe the surface as shipped. - -## Why this is opening up - -Early development authored everything as raw `*.ail.json`. That worked -for 17 fixture files (each ≤ 60 LOC of JSON) but does not scale. Two -breaking signals: - -1. The token-economy cost of the JSON-AST is massive: a single integer - literal `1` is encoded as `{"t":"lit","lit":{"kind":"int","value":1}}` - — ~38 tokens of structural overhead per bit of semantics. For a stdlib - in the 200–500 def range this displaces real attention budget. -2. JSON-AST authoring exposes a class of errors (wrong field names, - silently-accepted extra fields under `#[serde(default)]`, - inconsistent ctor casing) that surface only at load time. The - schema is correct-by-construction in storage but **error-prone in - authoring**. - -Decision 1 anticipated this: it says a textual form exists "as a -bidirectional projection of the JSON form." The pretty-printer already -emits S-expression-style text (see `crates/ailang-core/src/pretty.rs`). -What is missing is the inverse direction — text → AST. Decision 6 -adds that inverse as **one** authoring projection alongside the -existing pretty-printer; the JSON-AST remains the source of truth. - -## First choice and rollback plan - -**Try (A) first.** Reasoning: constraint 1 (formalizable) outweighs -constraint readability. (A) has a 3-rule core grammar with one -lexical disambiguation rule. (B) doubles the rule count and -re-introduces a soft form of precedence (`->` inside `forall`). -(C) is tempting because it is zero-design but the resulting spec is -visibly heterogeneous, which is exactly what constraint 1 was meant -to rule out. - -If implementing (A) reveals that paren density actively hurts my -authoring (measurable: I make more wrong-paren errors than the -JSON-AST shape produced before), roll back and try (C). (B) stays -on the shelf for a future iter only if both fail. - -## Implementation outline - -- `crates/ailang-surface` — new crate. **Strictly additive.** PEG - parser produces existing `ailang-core::ast` types. No new AST - nodes, no schema changes, no new hashable form. Pretty-printer - for form (A) lives here too (the round-trip is the contract). -- `ailang-check`, `ailang-codegen` are **not modified**. They - continue to consume `ailang-core::ast::Module` values regardless - of which projection produced them. -- Round-trip test: for every `examples/*.ail.json`, parse the - corresponding hand-written `*.ail`, canonicalise, and assert - hash-equivalence to the original. Hash equivalence is the truth - check; the surface ships only if every fixture round-trips - identically. -- CLI: `ail parse -o `. Symmetric to - existing `ail render`. **`.ail.json` remains a first-class input** - to every existing subcommand; the parser is a producer, not a - gatekeeper. - - **Both extensions accepted.** Every path-taking CLI subcommand - accepts either `.ail` (Form A) - or `.ail.json` (Form B) as input. For `.ail` paths the subcommand - parses through `ailang_surface::parse` in-line and then proceeds - with the same loaded `Module` value that `.ail.json` would have - produced. `ail parse` remains the explicit converter — it does - nothing the implicit dispatch in the other subcommands does not, - but it is the supported way to materialise a stable `.ail.json` - snapshot from a `.ail` source for diff / hash / cache purposes. -- Stdlib (`std_list`, `std_maybe`, ...) authored in - form (A) from day one **because that is the AI authoring - projection**, not because JSON authoring is forbidden. The - resulting `.ail.json` is what tests and downstream tools see. - -## What this Decision deliberately does not do - -- It does not change which form is canonical: the JSON-AST remains - the hashable, content-addressed representation, and all hashing, - content-addressing, and cross-module references flow through it - unchanged. The authored form is Form A; the JSON-AST is - materialised in-process by callers that need it. The two - forms are byte-isomorphic by the round-trip invariant. Form (A) - is one projection; the JSON-AST stays canonical. -- It does not foreclose visual or graphical front-ends. The crate - layout (`core` owns AST; `surface`/`visual`/... are siblings) - reserves that lane. -- It does not remove `.ail.json` as input. Every existing - CLI subcommand (`check`, `render`, `describe`, `emit-ir`, - `build`, `run`, `manifest`, `deps`, `diff`, `workspace`, - `builtins`) keeps its current `.ail.json` interface. - -## Form refinements during implementation - -Two productions in the original sketch had to be widened -during implementation to round-trip the existing AST faithfully. -Captured here for the spec record: - -1. **`lam-term` carries types and effects.** The AST's `Term::Lam` - stores parallel `params`, `param_tys`, `ret_ty`, and `effects` - fields. The original sketch had only names. The implemented form is - - ``` - lam-term ::= "(" "lam" "(" "params" typed-param* ")" - "(" "ret" type ")" - effects-clause? body-attr ")" - typed-param ::= "(" "typed" ident type ")" - ``` - - This keeps the no-precedence / one-construct-per-token-list - invariants and adds no new lexical rules. - -2. **`import-clause` admits an optional alias.** The AST's - `Import.alias` is `Option`; the original sketch only - supported the `None` case. The implemented form is - - ``` - import-clause ::= "(" "import" ident ("as" ident)? ")" - ``` - - `as` is a bare ident token in this position; no special lexical - rule is needed. - -Neither change extends the grammar's rule budget meaningfully: -the 30-production ceiling of constraint 1 is intact (the parser -implements ~28 named productions). All 17 `examples/*.ail.json` -fixtures round-trip identically through `print → parse → canonical -JSON`; the three hand-written `.ail` exhibits parse to canonical -JSON identical to their corresponding `.ail.json` files. - -3. **Tail-call surface.** Decision 8 ships two new - productions, both positional analogues of their non-tail - counterparts. Their result terms set `Term::App.tail = true` / - `Term::Do.tail = true`; the typechecker's `verify_tail_positions` - pass enforces that the marker is only used in tail position. - - ``` - tail-app-term ::= "(" "tail-app" term term+ ")" - tail-do-term ::= "(" "tail-do" ident term* ")" - ``` - - Production count: ~30, still inside the 30-rule - constraint-1 budget. No new lexical rule (`tail-app` / `tail-do` - are bare ident tokens; no special casing). - -## Empirical addendum — cross-model authoring measurement - -Cross-model measurement against two foreign LLMs via IONOS, -temperature=0, top_p=1, max-turns=5. Two blind cohorts on the same -four MVP tasks (`t1_add_three`, `t2_length`, `t3_main_prints`, -`t4_count_zeros`); each cohort sees only its own form's mini-spec. -Subjects (both runs use the pipeline-error-formatting fix from ms.1 -so JSON-cohort feedback carries the full anyhow cause chain): - -- `Qwen/Qwen3-Coder-Next` — - `experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-080864/` -- `meta-llama/CodeLlama-13b-Instruct-hf` — - `experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-9197fd/` - -| metric | Qwen / JSON | Qwen / AIL | CodeLlama / JSON | CodeLlama / AIL | -|---|---|---|---|---| -| reached green | 1/4 | 1/4 | 0/4 | 2/4 | -| first-attempt green | 1/4 | 1/4 | 0/4 | 2/4 | -| mean turns-to-green (green only) | 1.0 | 1.0 | — | 1.0 | -| total prompt tokens | 182,378 | 110,575 | 116,015 | 93,017 | -| total completion tokens | 14,972 | 3,474 | 2,711 | 2,234 | -| top error class | check (×3) | parse (×3) | check (×2) | parse (×2) | - -Both subjects show the same direction on every metric the table -tracks. AIL is cheaper than JSON on prompt tokens (Qwen 61%, -CodeLlama 80% of the JSON cohort's spend) and cheaper on completion -tokens (Qwen 23%, CodeLlama 82%). AIL reached-green is greater than -or equal to JSON reached-green for each subject (Qwen ties at 1/4; -CodeLlama strictly dominates, 2/4 vs 0/4). The failure-class -symmetry observed on the original Qwen baseline holds for both -subjects: JSON cohorts fail at the typecheck stage, AIL cohorts at -the parse stage — each form's front-of-pipeline check. Three of the -four first-attempt-green cells observed across the two subjects are -AIL (Qwen-AIL t3, CodeLlama-AIL t1, CodeLlama-AIL t3); the -fourth is Qwen-JSON t3, the same task the original baseline already -flagged as the JSON-form's only first-attempt success. Two CodeLlama -JSON-cohort cells (`t2_length`, `t4_count_zeros`) terminated as -`api_failure` on turn 1 with zero tokens consumed — a transient -IONOS-side terminal error, not a model-output failure; CodeLlama -JSON-cohort numbers therefore average over 2 informative cells, not -4. The Qwen re-run also shifted by one cell against the cma.3 -baseline (AIL 2/4 → 1/4, JSON 1/4 → 1/4) despite identical -temperature=0 inputs; this is IONOS-side state-of-day noise on a -single deterministic re-run, not an effect of the ms.1 -feedback-formatting fix (which only changes JSON-cohort prompt -content). - -**Scope of this addendum:** two subjects, n=1 each, deterministic. -This is a second data point pointing in the same direction as the -first, not a verdict. The universal claim of this Decision (".ail -is the AI authoring projection") would need ≥3 subjects with -statistical robustness to ratify; both current points point the -same way (AIL cohort cheaper and at-least-as-green) but neither -the sample size nor the cross-call noise observed in the Qwen -re-run is enough to close out the Decision. - -## Why not other memory models - -**Tracing GC.** Open-ended. Boehm's bench overhead is structurally on the allocate path (JOURNAL bench notes); tuning Boehm cannot move that -needle. A precise tracing GC would need read/write barriers, -root maps, generational machinery — months of work, with -irreducible pause-time variability at the end. The user's -framing was decisive: "Am GC kannst du ewig rumschrauben (und -bekommst trotzdem auch in 100 Jahren keine berechnbare -Performance)." - -**Region inference (Tofte/Talpin / MLton-style).** Considered -seriously; rejected. Regions tie lifetimes to dynamic scope — -every value lives in some region, regions stack on entry/exit, -deallocation is bulk-by-region. The reduction: a region is an -explicit allocator with sugar plus a static check that nothing -escapes its lifetime. The deeper problem: regions assume -**stack-shaped lifetimes**. Real programs have non-stack-shaped -lifetimes — caches, memo tables, registries, lookup structures -whose lifetimes are not nested. Regions would either force these -into a `letregion` at the top of `main` (everyone's allocator -becomes the root region — useless), or require a region per -cache variant (combinatorial). RC has no lifetime-shape -assumption; it works for any DAG. - -**Linear / ownership types as primary mechanism (Rust-style).** -Considered as a general-purpose alternative; rejected as primary. -Rust's borrow-checker is a great mechanism but requires the -author to thread lifetimes through every signature and accept -that some programs cannot be expressed without `unsafe`. For an -LLM-targeted language with recursion + closures everywhere, that -cost is too high — most of the source surface would be lifetime -annotations rather than logic. AILang uses a *subset* of these -ideas (mode annotations, linear consumption discipline) -selectively, layered on top of RC, where the annotations buy -concrete optimisations and never have to thread lifetimes -through callees. - -## Why advisory + suppress instead of inference - -Three reasons, all anchored in Decision 10's framing: - - 1. **Annotation states intent; inference picks - weakest-supporting.** These often coincide today but are - conceptually different. The annotation captures what the - author committed to (e.g. `(own T)` reserved for a planned - mutation that hasn't landed); inference would silently - relax it. - 2. **Annotation is a drift-bremse.** Body change that flips - the inferred mode produces caller-side breakage at remote - sites. Annotation enforces the local-conflict-error pattern - instead. - 3. **Forcing function for LLM authoring.** Without mandatory - annotation, the LLM never has to commit to ownership intent - before writing the body. The advisory lint plus suppress - lets us flag accidental over-strictness without weakening - the contract. - -The suppress mechanism with mandatory-reason mirrors Rust's -`#[allow(...)]`-style escape hatch but sharpens it: the reason -is required (not optional), and it becomes part of the contract -the next reader sees. CLAUDE.md's "preserve correctness across -development cycles" is what this directly serves. - -## Adjacent extensions for mutability (out of Decision 10's scope) - -If future workloads need mutable arrays, hash tables, or other -inherently mutable primitives, the answer is **not** a tracing GC -backstop. The answer is a separate ownership/linear extension -that gates mutability behind static single-owner discipline. RC -+ uniqueness is the universal floor; ownership extends it for -specific high-performance primitives without rebreaking the -acyclicity invariant. - -## What this Decision deliberately does not do - -- **Does not infer everything.** AILang demands annotations - *because* the LLM author can produce them effortlessly. The - compiler does inference *plus* verification of contracts. -- **Does not require existing fixtures to migrate immediately.** - `(con T)` is treated as `(own T)`. Existing JSON hashes - stay bit-identical until the fixture is intentionally updated. -- **Does not commit to atomic refcounts.** AILang is - single-threaded; refcounts are non-atomic. The embedding ABI's - per-thread `ailang_ctx_t` keeps this correct under a host swarm: - each thread's allocations are private to its ctx, no RC cell - crosses a thread, so non-atomic refcounts stay sound. -- **Does not introduce regions.** Regions were considered and - rejected; see "Why not other memory models" above. - -## What the typeclass design explicitly does NOT support - -Each of the following is rejected by either schema (parser cannot -express the construct) or by an enumerated diagnostic. The -combination of axis-1 (single-param, no FunDeps, no assoc) and -axis-5 (kind `*` only) covers the space. - -- **Multi-parameter classes** (`class Foo a b where ...`). Schema - rejects: `ClassDef.param` is a string, not a list. -- **Higher-kinded class params** (`class Functor f`). Rejected at - workspace-load time: `BareCrossModuleTypeRef` from canonical-form - validation fires before class-schema validation if any method - body uses the param as a `Type::Con` head — bare non-primitive - names must be declared as a `TypeDef` in the owning module, and - a class param is not. -- **Higher-rank polymorphism** (`forall a. (forall b. b -> b) -> a`). - Already rejected at parse time per the typeclass-conversation - rationale recorded in JOURNAL; constraint-bearing - signatures inherit that prohibition. -- **Existential / dyn dispatch** (`exists a. Show a => a`, - heterogeneous lists like `[Show]`). Schema does not express - existentials. The LLM-natural alternative is sum types. -- **Associated types** (`class Container c where type Element c`). - Schema does not express type-level methods. -- **Functional dependencies** (`class Convert a b | a -> b`). Required - only for multi-param classes; entails by axis 1. -- **`deriving` / auto-derivation** (`data Foo = ... deriving (Eq)`). - Future iteration, gated separately on Feature-acceptance. -- **Numeric literal defaulting.** `1` does not auto-resolve to `Int` - under an ambiguous `Num a` constraint. Bare polymorphic literals - with multiple satisfying instances fire `NoInstance` with a hint - to annotate. The LLM-author writes `1 : Int` or `1 : Float`. -- **Orphan-with-warning mode.** Coherence is hard. `OrphanInstance` - is always an error, never a warning. The corollary `--allow-orphans` - flag is not provided. - -## What this decision does NOT commit to - -- **Operator routing.** `==`, `<`, etc. stay primitive in milestone - 22. Class-routing operators is a future-iteration option, not a - Decision-11 commitment. -- **Form-B (prose) projection of class/instance.** The prose - renderer for the class/instance schema nodes is one-way (no - parser by design); the render in - `crates/ailang-prose/src/lib.rs` is a placeholder and - informational only. -- **Specific monomorphised-symbol naming format.** Milestone 22 - uses `__` for primitive type targets - (e.g. `show__Int`) and `__<8-hex-prefix>` for compound - types (where `<8-hex-prefix>` is the BLAKE3 hash of the canonical - type bytes). The `__` separator was chosen over `#` and `@` for - LLVM IR identifier legality. -- **Mode annotations on class methods.** Class method signatures - ARE full FnSigs and DO carry mode annotations per Decision 10; - the convention is `borrow` for read-only methods. User-defined - classes pick modes per method. -- **Number of Prelude classes.** Milestone 22 ships zero (no - Prelude). A future Prelude milestone gates class additions on - the Feature-acceptance criterion at the time of proposal. - -## Env construction — why two parallel overlays (rationale of the code-SoT §"Env construction") - -Behaviour of the two-overlay check environment is code-authoritative -(the `env-construction` ledger row is source-link only); only the -*why* moves here so a future brainstorm does not re-propose -collapsing the two maps: - -The split is not a refactor wart. The two maps have distinct -semantic roles: an owning data index, and a derived lookup -accelerator over the same data. Collapsing them into one -variant-keyed map (`Map`) would force every -type-iterator and every ctor-lookup site to discriminate the variant -tag, reversing the optimisation the reverse index exists to -provide. - -The mono-side overlay was narrowed to types-only at iter ct.3.2 because its -consumer is the runtime ctor lookup, which became type-driven -post-ct.2.2 — a different consumer story, hence a different -overlay shape. The asymmetry between the check side and the mono -side is by design and is pinned by -`crates/ailang-check/tests/duplicate_ctor_pin.rs`. - -## Migrated from design/contracts/ at design-md-rolesplit.tidy (2026-05-19) - -History/decision-record prose extracted from contract files by the -post-audit tidy (the `###`-whole relocation in iter .1 dragged these -into contracts; they are relitigation-guard content, not contract). - -### typeclasses.md — Prelude-classes evolution - -An earlier draft committed to a fixed Prelude (Show/Eq/Ord on the -primitives), but the implementation work to wire `int_to_str` as a -heap-allocated-string runtime primitive proved substantively -separable from the typeclass machinery itself, and the LLM-utility -case for primitive `Show` is weak (LLM-natural form is `int_to_str -x`, not `show x`). Milestone 23 added `Ordering`/`Eq`/`Ord` + -primitive instances + the five helpers; operator routing through -`Eq`/`Ord` and `print`-rewire were held out for bench-rebaseline / -post-mq-dispatcher reasons; `Show` shipped in milestone 24 (iters -24.2/24.3). `MethodNameCollision` was retired at iter mq.3 -(cross-class method sharing became structurally legal). The -dedicated `KindMismatch` diagnostic was retired at iter ctt.3 -(`BareCrossModuleTypeRef` subsumes it). - -### str-abi.md — heap-Str builtin provenance + deferred-operator rationale - -Builtin iter provenance: `int_to_str`/`bool_to_str`/`float_to_str`/ -`str_clone` shipped iter 24.1; `str_concat` shipped iter str-concat -(2026-05-13); the non-escape static-Str lowering pass landed iter -18b, move-tracking partial-drop iter 18d.3. Operator routing through -classes is deliberately deferred — it would migrate every fixture -and re-baseline the bench corpus (a new-baseline decision, not an -iter detail). `Num` is not in milestone 22: class-based numeric -overloading would invoke literal-defaulting which axis-7 excluded. - -### scope-boundaries.md — retired ops + deferral rationale - -Polymorphic-fn-as-value is deferred because it would need one -closure-pair global per instantiation. The per-type print ops -`io/print_int`/`io/print_bool`/`io/print_float` were retired in iter -rpe.1 (the polymorphic `print` is the canonical non-Str output path). - -### roundtrip-invariant.md / data-model.md — corpus + loop-recur provenance - -The Form-A round-trip corpus was flipped from `.ail.json` to `.ail` -at iter form-a.1. The `loop`/`recur` schema entries were added at -loop-recur iter 1 (strictly additive; pre-existing fixtures hash -bit-identically). Provenance only — the present-tense schema/contract -is the entry shapes themselves. diff --git a/docs/journals/2026-05-19-iter-bugfix-rc-global-stats-race.md b/docs/journals/2026-05-19-iter-bugfix-rc-global-stats-race.md deleted file mode 100644 index 3e901a3..0000000 --- a/docs/journals/2026-05-19-iter-bugfix-rc-global-stats-race.md +++ /dev/null @@ -1,73 +0,0 @@ -# iter bugfix-rc-global-stats-race — atomic global RC-stats fallback counters - -**Date:** 2026-05-19 -**Started from:** 427b687b9588ac8e1f96070ffc20f2644e9cd28c -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -The two null-ctx fallback RC-stats counters `g_rc_alloc_count` / -`g_rc_free_count` (`runtime/rc.c`) were plain `static uint64_t` with a -non-atomic `++` at the `__ail_tls_ctx == NULL` arm of `ailang_rc_alloc` -and the to-zero branch of `ailang_rc_dec`. A multi-threaded host that -uses the global fallback path (no `ailang_ctx_new`) raced the -read-modify-write and lost increments, so the `AILANG_RC_STATS` atexit -Σ under-counted non-deterministically. Not a memory bug — no box -crosses a thread; only the global statistics Σ was wrong. Fix -(user-approved Option A, 2026-05-19): the two global counters are now -`_Atomic uint64_t`, incremented via -`atomic_fetch_add_explicit(.., memory_order_relaxed)` and read via -`atomic_load_explicit(.., memory_order_relaxed)` in the atexit printer -(its only reader). Relaxed is correct and sufficient: pure statistics -with no happens-before obligation, and the atexit reader runs after all -worker threads have joined (single-threaded at exit). The per-ctx -counters and the per-object refcount header stay non-atomic by design -(single-thread-per-ctx; boxes never cross threads) — explicitly out of -scope. Scope held to `runtime/rc.c` only. - -## Per-task notes - -- bugfix-rc-global-stats-race.1: made `g_rc_alloc_count` / - `g_rc_free_count` `_Atomic uint64_t`; added ``; - `atomic_fetch_add_explicit(.., relaxed)` at the two null-ctx - fallback `++` sites; `atomic_load_explicit(.., relaxed)` snapshot - in `ailang_rc_stats_atexit` (the sole reader). Corrected the two - now-stale doc blocks (file header comment + the 18g.0 stats block) - to state the global counters are atomic while per-ctx + header - stay non-atomic by design. Left the rc.c drop-worklist - "Single-threaded; non-atomic" note unchanged — it refers only to - the per-call drop-worklist, which is genuinely still - single-threaded, so correcting it would be a false correction the - constraint explicitly forbids. RED `embed_rc_global_stats_race` - went GREEN deterministically (3/3 runs, allocs == frees == - 16_000_000); full `cargo test -p ail` green (39/39 result lines - ok, all embed_* incl. the per-ctx tsan harnesses unaffected). - -## Concerns - -(none) - -## Known debt - -- `runtime/rc.c` comment near the 18g.0 block ("adds only two - unconditional `++` operations to the hot path") is now slightly - imprecise: the global path is a relaxed atomic add and is - conditional on `_ctx == NULL`. Left verbatim deliberately — it is - a performance aside (not an atomicity claim), it predates and was - already imprecise after the M2 per-ctx split, and the atomicity - statement is now correctly carried by the adjacent rewritten - block; editing it would exceed the named doc-honesty drift the - minimal-fix constraint scoped this iter to. - -## Blocked detail - -(n/a — DONE) - -## Files touched - -- runtime/rc.c - -## Stats - -bench/orchestrator-stats/2026-05-19-iter-bugfix-rc-global-stats-race.json diff --git a/docs/journals/2026-05-19-iter-bugfix-swarm-rc-alloc-undercount.md b/docs/journals/2026-05-19-iter-bugfix-swarm-rc-alloc-undercount.md deleted file mode 100644 index 8d36bae..0000000 --- a/docs/journals/2026-05-19-iter-bugfix-swarm-rc-alloc-undercount.md +++ /dev/null @@ -1,97 +0,0 @@ -# iter bugfix-swarm-rc-alloc-undercount — build.rs missed runtime/ as a rerun-if-changed input - -**Date:** 2026-05-19 -**Started from:** 483117d39be6a38333cb5fae8c4b5b96e108d1c2 -**Status:** DONE -**Tasks completed:** 1 of 1 - -## Summary - -The M5 swarm leak-proof `symbol_fan_swarm_leak_free` was -non-deterministic: `Σfrees` stable-exact (12000003) every run while -`Σallocs` was short by a jittering amount in ~1/3 of runs. The -m5.2-resume-attempt journal framed this as a *residual runtime -defect* distinct from `7bfa11e` (the atomic-global-counter fix). The -`debug` cycle root-caused it instead to a build-dependency -completeness gap, not a runtime bug: `ail-embed/build.rs` declared -`cargo:rerun-if-changed` only for the kernel `.ail` and `build.rs` -itself — NOT the `runtime/` C sources that -`ail build --emit=staticlib` compiles into `libailang_rt.a`. Cargo -therefore never re-ran `build.rs` after `7bfa11e` landed, so -`ail-embed` kept linking a stale pre-`7bfa11e` non-atomic -`libailang_rt.a` whose `g_rc_alloc_count++` raced the M5 swarm's -concurrent TLS-NULL host allocs (the isolated 8×2M pure-global RED -`embed_rc_global_stats_race` structurally could not model this -because it never touches the stale ail-embed archive — it links the -workspace runtime fresh). `runtime/rc.c` at HEAD was already correct; -the entire production fix is one line in the build script. - -The fix adds a directory-level -`println!("cargo:rerun-if-changed={}", repo.join("runtime").display())`. -Confirmed by inspecting `crates/ail/src/main.rs::build_staticlib` -(lines 2519-2534): `libailang_rt.a = ar(rc.o, str.o)` where the -sources are exactly `runtime/rc.c` + `runtime/str.c`, both under -`runtime/`. The directory-level mechanism (Cargo recurses mtimes -under a tracked directory) is preferred over enumerating individual -files precisely because a per-file list would silently re-introduce -the same staleness for any future runtime source (e.g. a new -`runtime/foo.c`). Existing rerun-if-changed lines (kernel `.ail`, -`build.rs`, `AIL_BIN`) were kept. The build.rs edit triggers its own -`rerun-if-changed=build.rs`, which relinks the archive from the -current atomic `runtime/rc.c` — the RED's self-bootstrapping -mechanism. - -The cohesive, in-scope consequence: `symbol_fan_swarm_leak_free` is -un-`#[ignore]`d (the `#[ignore]` attribute and its reason string -removed; the test BODY is byte-for-byte unchanged — it was -quarantined, never weakened). The two surviving doc breadcrumbs -(module-doc bullet, fn-doc) are rewritten to present-tense-resolved -with the accurate build-dependency-staleness rationale and the -corrected journal reference. This test is now the integration-level -acceptance of both `7bfa11e` and the build-dependency fix; the leak Σ -balances deterministically (`Σallocs == Σfrees == 12000003`) across 6 -consecutive runs. - -## Per-task notes - -- iter bugfix-swarm-rc-alloc-undercount.1: added directory-level - `cargo:rerun-if-changed=/runtime` to `ail-embed/build.rs` - (existing kernel/`build.rs`/`AIL_BIN` lines kept); un-`#[ignore]`d - `symbol_fan_swarm_leak_free` in `ail-embed/tests/swarm.rs` (body - verbatim) and corrected its module-doc + fn-doc breadcrumbs to the - resolved build-dep-staleness rationale. RED - `rt_archive_freshness` went 0-lock-insn → GREEN (≥2 lock insns, - matches fresh cross-check). Gate-2 determinism: 6/6 runs, both - `symbol_fan_swarm_leak_free` and `symbol_fan_swarm_bit_exact` - GREEN every run, 0 failed / 0 ignored. - -## Concerns - -(none) - -## Known debt - -- (resolved) `ail-embed/tests/swarm.rs` carried a pre-existing - `warning: unused import: data_server::records::DataFormat` (present - since the m5.2 split at `b724cd1`; `DataFormat` is used only by - `swarm_runner.rs`, never the test). The orchestrator correctly did - not chase it under the minimal-fix constraint; the Boss removed the - dead import inline at commit time (trivial mechanical edit per the - CLAUDE.md carve-out — `swarm.rs` was being committed this iter - regardless, and leaving a warning in the now-final, no-longer- - quarantined M5 leak-proof file is its own debt). `cargo test - --manifest-path ail-embed/Cargo.toml` is warning-clean for this - crate after removal. - -## Blocked detail - -(n/a — DONE) - -## Files touched - -- ail-embed/build.rs -- ail-embed/tests/swarm.rs - -## Stats - -bench/orchestrator-stats/2026-05-19-iter-bugfix-swarm-rc-alloc-undercount.json diff --git a/docs/journals/2026-05-19-iter-design-ledger-formal-links.1.md b/docs/journals/2026-05-19-iter-design-ledger-formal-links.1.md deleted file mode 100644 index b9da768..0000000 --- a/docs/journals/2026-05-19-iter-design-ledger-formal-links.1.md +++ /dev/null @@ -1,109 +0,0 @@ -# iter design-ledger-formal-links.1 — formal cross-links + hard-gate clause-5 - -**Date:** 2026-05-19 -**Started from:** 36599be -**Status:** DONE -**Tasks completed:** 5 of 5 - -## Summary - -Positive-half completion of the DESIGN.md → design/ split: `design/` -body cross-references are now formal, file-relative Markdown links -into the durable tier (`design/` or source), and a new in-tree hard -gate (`design_index_pin.rs` clause-5, -`design_body_links_are_durable_and_resolve`) walks every -`design/contracts/*.md` + `design/models/*.md`, strips fenced code, -extracts every `](path)`, and asserts the target resolves -file-relative to a real file under `design/`-or-`crates/`-or- -`runtime/` — never `docs/`, never an in-file `#anchor`. RED-first was -established via four embedded synthetic-vector asserts on an -identity-stubbed `strip_fences`; replacing the stub with the real -toggle-on-fence impl turns the test GREEN. Seven prose cross-refs -were converted to `[label](path)` links (8 link tokens total — the -mixed-referent `scope-boundaries.md:88` split carries two), the two -homeless `(see docs/PROSE_ROUNDTRIP.md)` pointers were resolved via -clause-6 disposition (b) (pointer removed, surrounding behavioural -prose preserved), and a pin-safe positive-half paragraph was inserted -into `honesty-rule.md` after the existing clause-3 paragraph, -explicitly naming `design_index_pin.rs` clause-5 as the gate and -keeping both `docs_honesty_pin.rs`-pinned phrases byte-identical. -`design/INDEX.md` and the decision-records journal are byte-unchanged -(commitment 4 / Acceptance 5). Whole-workspace tests: **647 passed / -0 failed** (+1 from the pre-milestone 646: the new clause-5). - -**Composition-invariant assertion** (Acceptance 5; spec -testing-strategy §Composition with clause-3): on the post-milestone -tree, for every `design/contracts/*.md`, clause-3 GREEN ∧ clause-5 -GREEN — independently verified by running both tests in the same -suite. Every cross-reference in a contract is therefore either a -resolving durable file-link **or** it is gone (clause-3 would -otherwise have rejected it as decision-record prose). The two -clauses now form the complete invariant the split promised. - -## Per-task notes - -- iter design-ledger-formal-links.1.1: clause-5 hard gate added to - `crates/ailang-core/tests/design_index_pin.rs` (RED-first via - identity-stubbed `strip_fences`; GREEN after real toggle-on-fence - impl; `//!` header extended one sentence; 5/5 tests pass). -- iter design-ledger-formal-links.1.2: seven prose cross-refs - converted to file-relative `[label](path)` Markdown links - (`float-semantics.md:69, :100`; `embedding-abi.md:45`; - `memory-model.md:44, :105-106`; `scope-boundaries.md:48, :88` — - the last splits into two links, source-link + cross-models link). -- iter design-ledger-formal-links.1.3: clause-6 disposition (b) - applied to the two homeless `(see docs/PROSE_ROUNDTRIP.md)` - pointers (`pipeline.md:60-61`, `authoring-surface.md:178-181`) — - pointer removed, present-tense behavioural prose about - `ail merge-prose` and the six-step cycle preserved. -- iter design-ledger-formal-links.1.4: positive-half paragraph - inserted into `honesty-rule.md` between lines 14 and 15 — names - formal links into the durable tier as the surviving - cross-reference form and `design_index_pin.rs` clause-5 as the - gate. Pin-safe (both `docs_honesty_pin.rs`-pinned phrases stay - byte-identical, no clause-3 PHRASE or Sweep-1 anchor introduced). -- iter design-ledger-formal-links.1.5: whole-workspace gate + - acceptance-criteria spot-checks (zero `docs/`-targeting links, - zero `#fragment` links, exactly 8 `](path)` tokens introduced, 5 - in-fence refs untouched, INDEX + decision-records journal - byte-unchanged). - -## Concerns - -- Task 5 Step 7 planner predicted-count discrepancy: the plan - expected exactly **1** removed line in the test-file diff - (interpreting Step 5's `//!` header replacement as a single-line - edit), but the actual diff shows **2** removed lines because Step - 5's verbatim before→after rewords lines 4 *and* 5 of the original - `//!` block (5-line block → 8-line block). Clauses 1–4 source - content (line range 7+ of the file) is byte-unchanged, which is - the load-bearing Acceptance 1; the 2-vs-1 mismatch is purely in - the planner's predicted byte-count of the `//!` header rewording - it itself mandated. Plan-pseudo-vs-reality class (cf. - feedback_plan_pseudo_vs_reality), Step-5 verbatim taking precedence - over Step-7 predicted-count. - -## Known debt - -- (none — the milestone closes the positive-half side of the - contract/decision-record split; the spec's named acceptance set is - exhaustive and closed; intra-file directional prose remains - explicitly out of scope, with file-split as the documented future - remedy under commitment 1 if a file ever grows enough that - "above"/"below" loses clarity). - -## Files touched - -- `crates/ailang-core/tests/design_index_pin.rs` (header + appended - clause-5) -- `design/contracts/float-semantics.md` (lines 69, 100) -- `design/contracts/embedding-abi.md` (line 45) -- `design/contracts/memory-model.md` (lines 44, 105-106) -- `design/contracts/scope-boundaries.md` (lines 48, 88) -- `design/contracts/honesty-rule.md` (new paragraph after L14) -- `design/models/pipeline.md` (drop line 61) -- `design/models/authoring-surface.md` (lines 178-181 reworded) - -## Stats - -`bench/orchestrator-stats/2026-05-19-iter-design-ledger-formal-links.1.json` diff --git a/docs/journals/2026-05-19-iter-design-md-rolesplit.1.md b/docs/journals/2026-05-19-iter-design-md-rolesplit.1.md deleted file mode 100644 index 04d28a4..0000000 --- a/docs/journals/2026-05-19-iter-design-md-rolesplit.1.md +++ /dev/null @@ -1,91 +0,0 @@ -# iter design-md-rolesplit.1 — DESIGN.md → design/ ledger role-split (whole milestone) - -**Date:** 2026-05-19 -**Started from:** deeffb1872cd13a9fed2a0812d69e7273f49671b -**Status:** DONE -**Tasks completed:** 9 of 9 - -## Summary - -The 3020-line `docs/DESIGN.md` is replaced by the `design/` ledger: -`design/INDEX.md` (sole addressable spine, a typed Contracts+Models -table), 14 `design/contracts/*.md` (test-linked invariants), 5 -`design/models/*.md` (onboarding whitepapers), plus -`docs/journals/2026-05-19-design-decision-records.md` (the -relitigation-guard archive of every why/rejected/does-not-do/rollback -`###`). Content was sliced from DESIGN.md by the spec Appendix line -ranges with a deterministic Python slicer (`/tmp/ail-iter/ -design-md-rolesplit.1/slice.py`, not committed) — byte-faithful -except (a) heading-level re-level, (b) the rewritten -`honesty-rule.md`, (c) the OQ7 cite deletion. The RED-first -`design_index_pin.rs` anti-regrowth spine (4 clauses) was -demonstrably RED before and is GREEN after. Every live `DESIGN.md` -reference in code, scripts, agents, SKILL bodies, CLAUDE.md, README, -runtime C, `.ail` fixtures, and `form_a.md` was retargeted to the -Appendix destination (or to a code-SoT note for the source-link -contracts). `docs/DESIGN.md` deleted via `git rm` (staged only). -Build-atomic by task ordering; whole `cargo test --workspace` GREEN, -zero failures. - -## Per-task notes - -- design-md-rolesplit.1.1: RED-first `crates/ailang-core/tests/design_index_pin.rs` — 4-clause structural pin written; verified RED (DESIGN.md present + design/ absent). Dropped one unused `Path` import (plan-verbatim code had an unused-import warning) — recorded as a concern. -- design-md-rolesplit.1.2: `design/INDEX.md` (spec-verbatim 2-table ledger + "## Project framing" preamble absorbing DESIGN.md:6-18/19-55/83-92) + 14 `design/contracts/*.md` sliced from the Appendix ranges + the rewritten `honesty-rule.md` (two `docs_honesty_pin.rs:70,72` phrases each contiguous on one physical line). INDEX link cells made workspace-root-relative (`design/contracts/...`) so the authoritative pin resolves them. -- design-md-rolesplit.1.3: 5 `design/models/*.md` sliced from the Appendix model-tagged ranges; `every_index_link_resolves` PASS for all 20 link rows. -- design-md-rolesplit.1.4: `docs/journals/2026-05-19-design-decision-records.md` — every Appendix `D` range concatenated in source order + the genuine Env-construction "why" rationale (source-link sections' behaviour dropped as code-SoT). Acceptance-criterion-2 completeness verified programmatically: every one of 66 `##`/`###` headings mapped exactly once, no overlap (only uncovered lines are DESIGN.md's own `#`-preamble :1-5 and one blank separator :244). Journals-INDEX pointer line (Step 2) **deferred to Boss** — see Concerns. -- design-md-rolesplit.1.5: retargeted `design_schema_drift.rs` (const `DESIGN_MD`→`DATA_MODEL` include_str! → data-model.md; deleted `data_model_section()` + `data_model_section_is_bounded()`; all anchor tests use `DATA_MODEL`), `docs_honesty_pin.rs` (per-test reads point at the design/ homes; three present-anchors found to live in the decision-records journal — read from there), `effect_doc_honesty_pin.rs:21` (normalised join of effects.md + scope-boundaries.md; added `norm()`). All three GREEN; build green with DESIGN.md still present. -- design-md-rolesplit.1.6: retargeted the Float-branch (`design/contracts/float-semantics.md`) and Show-branch (`design/contracts/typeclasses.md`, contiguous across the `\`-continuation) diagnostics in `ailang-check/src/lib.rs` + the 2 lockstep E2E assertions + module rustdocs; both E2Es GREEN. -- design-md-rolesplit.1.7: `bench/architect_sweeps.sh` repointed (`INDEX`/`DESIGN_GLOB`, recursive grep over design/contracts+models, regexes unchanged, exit-code contract preserved, exit-2 verified). Sweep-5 honesty-rule self-quote reworded; one pre-existing advisory Sweep-1 match remains (see Concerns). -- design-md-rolesplit.1.8: retargeted buckets (b) 12 agent files, (c) 5 SKILL bodies + skills/README.md, (d) CLAUDE.md (layout table + the "Roles of …" section, current-state-mirror discipline preserved & re-homed), (e) ~25 code/C/.ail/spec comment xrefs to the Appendix destinations (source-link contracts → code-SoT/INDEX note); OQ7 "Iter 13b" cite deleted (behaviour stated directly, no pointer). Build green. -- design-md-rolesplit.1.9: `git rm docs/DESIGN.md` (staged); workspace build green (build-atomicity proven — the only compile-time consumer was retargeted in Task 5); `cargo test --workspace` ALL GREEN incl. `design_index_pin`'s 4 clauses (RED→GREEN); acceptance grep CLEAN of live refs (residuals = the spec-mandated clause-4 pin only); architect_sweeps contract preserved. - -## Concerns - -- iter .1: dropped an unused `use std::path::Path` from the plan-verbatim `design_index_pin.rs` (it produced an unused-import warning). Minimal quality fix; the RED-first property is unchanged. Deviation from plan-verbatim text, recorded. -- iter .2: **Plan/spec internal contradiction (repaired).** Task 1's clause-3 marker list (spec-verbatim) includes `"an earlier draft"`; Task 2 Step 4's plan-verbatim `honesty-rule.md` text quoted `"an earlier draft said"` as a *forbidden-pattern example*, which `contracts_carry_no_decision_record_prose` flags forever. The spec's own sample honesty-rule.md (spec §"Concrete code shapes") does NOT contain that phrase. Repaired by rewording honesty-rule.md's History bullet to name the category without the tripwire substring, preserving the two `docs_honesty_pin.rs:70,72` pinned phrases verbatim+contiguous. Spec-faithful to §Architecture-4 ("rewritten to resolve the internal contradiction"). -- iter .2: INDEX link cells changed from spec-verbatim `contracts/…` to workspace-root-relative `design/contracts/…` so the authoritative `design_index_pin.rs` (resolves `root().join(link)`) is GREEN. Ratifying-test column resolved to exact paths (`skills/brainstorm/SKILL.md`, `crates/ailang-surface/tests/round_trip.rs`) per spec lines 183-187 ("Boss-resolved to exact recon-verified paths"); the spec's short labels (`… Step 4`, `… round_trip.rs`) would not resolve under clause-2. -- iter .7 / .9 (DONE_WITH_CONCERNS): `architect_sweeps.sh` exits **1**, not the plan-expected **0**. Sole match: `design/contracts/str-abi.md:23` `(iter str-concat, 2026-05-13)` — an API-origin date faithfully migrated verbatim inside the `### Heap-Str primitives` block (spec Architecture-2: `###` moves whole, no sentence-surgery; the pre-split sweep against DESIGN.md flagged the same line identically — NOT a split-introduced regression). The sweep is explicitly advisory; this is a milestone-close `audit` adjudication item, not a relocation defect. Faithful-migration over content-dodging (CLAUDE.md memory). - -## Known debt - -- `design/contracts/float-semantics.md` retains the cross-prose `(see "Str ABI" below for the dual realisation)` inside the `float_to_str`/`int_to_str` paragraph (DESIGN.md:2795-2800) — the Str ABI section moved to `str-abi.md`, so this internal "below" cross-ref is now stale-direction. Not strippable at relocation time (no task prescribes intra-prose cross-ref rewriting; the Appendix is heading-level). `str-abi.md` contract exists; the phrase still names a real contract. Audit-adjudication candidate. -- `design/contracts/str-abi.md:23` iter-origin date stamp (the architect_sweeps Sweep-1 advisory match above) — audit adjudicates whether the iter/date origin annotations in the migrated `### Heap-Str primitives` API list are honest API-provenance or strippable history. - -## Blocked detail - -(none — DONE) - -## Boss action required (NOT done by this agent — role + project rule) - -- **Journals INDEX pointer (Task 4 Step 2):** the orchestrator role - Red-Flags any edit to `docs/journals/INDEX.md`, and - `skills/implement/SKILL.md` makes INDEX.md strictly Boss-only with - no exception. The decision-record *file* is created (the - substantive deliverable); the Boss appends, in the same commit, - newest-last to `docs/journals/INDEX.md`: - `- 2026-05-19 — design-decision-records (migration): relitigation-guard archive — every why/rejected/does-not-do/rollback/empirical ### moved out of the former docs/DESIGN.md by the design-md-rolesplit milestone. Companion to spec 2026-05-19-design-md-rolesplit.` - plus this iter's own pointer line per the standard Boss procedure. -- **Acceptance-grep plan-defect:** Task 9 Step 4's verbatim grep - filter `grep -vE '^\./docs/(...)'` uses a `^\./` anchor that does - NOT match this system's `grep -rIn` path output (`docs/...`, no - `./`), so the history-doc exclusion silently fails on the literal - command. The spec-criterion-8-faithful filter (anchor `^(\./)?`, - plus excluding `design_index_pin.rs` — the absence-enforcing - clause-4 pin the spec's own sketch mandates) returns - **CLEAN: zero live DESIGN.md references**. The 3 residual matches - are all `crates/ailang-core/tests/design_index_pin.rs` (the - spec-mandated `docs/DESIGN.md was resurrected` guard) — the - deletion *enforcer*, categorically not a live content xref. - -## Files touched - -- New: `design/INDEX.md`, `design/contracts/{14 files}.md`, `design/models/{5 files}.md`, `crates/ailang-core/tests/design_index_pin.rs`, `docs/journals/2026-05-19-design-decision-records.md` -- Deleted (staged): `docs/DESIGN.md` -- Modified — tests: `design_schema_drift.rs`, `docs_honesty_pin.rs`, `effect_doc_honesty_pin.rs`, `eq_float_noinstance.rs`, `show_no_instance_e2e.rs`, `duplicate_ctor_pin.rs`, `codegen_import_map_fallback_pin.rs`, `polyfn_dot_qualified_branch_pin.rs`, `embed_record_layout_pin.rs`, `embed_staticlib_lowering.rs`, `eq_ord_e2e.rs`, `e2e.rs`, `round_trip.rs` -- Modified — source: `ailang-check/src/lib.rs`, `ailang-codegen/src/{lib,drop,match_lower}.rs`, `ailang-core/src/ast.rs`, `ailang-surface/src/{lib,parse,print}.rs`, `ail/src/main.rs`, `ail-embed/src/lib.rs`, `ailang-core/specs/form_a.md` -- Modified — runtime/fixtures: `runtime/{rc,str}.c`, `crates/ail/tests/embed/{record,tick}_roundtrip.c`, `examples/fieldtest/floats_{3,4}*.ail` -- Modified — bench/skills/docs: `bench/architect_sweeps.sh`, `CLAUDE.md`, `skills/README.md`, 5 SKILL.md, 12 agent .md - -## Stats - -bench/orchestrator-stats/2026-05-19-iter-design-md-rolesplit.1.json diff --git a/docs/journals/2026-05-19-iter-design-md-rolesplit.tidy.md b/docs/journals/2026-05-19-iter-design-md-rolesplit.tidy.md deleted file mode 100644 index 56def13..0000000 --- a/docs/journals/2026-05-19-iter-design-md-rolesplit.tidy.md +++ /dev/null @@ -1,96 +0,0 @@ -# iter design-md-rolesplit.tidy — contract-history strip + clause-3 widening - -**Date:** 2026-05-19 -**Started from:** f2cdd67e69574f564b63bfd4b7f5715046d75a5a -**Status:** DONE -**Tasks completed:** 7 of 7 (Tasks 1, 2, 3, 4, 5, 5b, 6, 7 — task_range [1,7] covers the 5b sub-task) - -## Summary - -Closes the `[medium]`+`[low]` spirit-vs-letter drift from -audit-design-md-rolesplit: the `###`-whole relocation in iter .1 -dragged decision-record/history prose into 5 contract files. This -tidy extracts that prose into -`docs/journals/2026-05-19-design-decision-records.md` (each removed -history sentence replaced by its present-tense contract equivalent), -fixes the one stale `(see "Str ABI" below)` cross-ref in -float-semantics.md, re-scopes `architect_sweeps.sh` from -`design/contracts design/models` to `design/contracts` only (the -narrative tier is not honesty-bound) with its lockstep -`ailang-architect.md` sentence, and widens `design_index_pin.rs` -clause-3 into a hand-rolled **faithful superset** of Sweep-1 over -the contracts scope (case-sensitive digit-anchored `sweep1_line_anchor` -+ path-EXCLUDED `date_anchor` + case-insensitive decision-record -PHRASES; no `regex` dep, no blanket iter detector). Gate-first TDD: -clause-3 was verified RED against the un-stripped tree (PHRASES hit -`"an earlier draft"` in typeclasses.md) before any strip, then -flipped GREEN after Tasks 2–5b. End state: design_index_pin 4/4, -`architect_sweeps.sh` exit 0, `cargo test --workspace` 646 passed / -0 failed, acceptance grep CLEAN. - -## Per-task notes - -- Task 1: replaced `contracts_carry_no_decision_record_prose` - (:130-157) with the widened provable-superset matcher (verbatim - from plan; other 3 clauses + helpers untouched). RED verified - against un-stripped tree; other 3 clauses + build GREEN. -- Task 2: typeclasses.md — 6 spans rewritten present-tense (:77-78, - :182-186, :197-200, :279-289 deleted wholesale, :291-300, :302-317); - Prelude-classes evolution history appended to the decision-record - journal. `io/print_str` pin contiguous at :297; docs_honesty_pin 5/5. -- Task 3: str-abi.md — `(iter …)` parentheticals + deferred-operator - + `Num`/axis-7 rationale stripped (:14, :16, :18, :20, :22-23 - soft-wrap collapsed, :74, :38-43, :45-47); journal appended. - `type-installed; codegen is reserved…` pin contiguous at :19. -- Task 4: scope-boundaries.md — process-meta line plain-deleted - (:5-6), closure-pair-global + rpe.1-retirement rationale stripped - (:14-17, :36-38); journal appended. Diverge over-strip guard: - pin byte-untouched, effect_doc_honesty_pin 4/4 (pin line shifted - :9→:8 as the expected consequence of the authorized :5-6 two-line - collapse — content match unaffected). -- Task 5: float-semantics.md:99-100 stale `(see "Str ABI" below …)` - → `(see design/contracts/str-abi.md …)`. eq_float_noinstance 1/1. -- Task 5b: roundtrip-invariant.md:73 corpus-flip parenthetical - dropped; data-model.md:149/161 `// loop-recur iter 1:` → `// loop:` - / `// recur:`; :154 `pre-existing` left untouched (ordinary word); - journal appended. No lowercase iter-code provenance remains; - design_schema_drift 7/7, round_trip 2/2. -- Task 6: architect_sweeps.sh DESIGN_GLOB narrowed to - `design/contracts` with substantive-reason inline comment + header - prose; ailang-architect.md `:77-84` bullet gains the scope sentence. - All five sweeps clean, exit 0. -- Task 7: whole-tree gate — design_index_pin 4/4 (clause-3 RED→GREEN), - 3 pins + 2 E2Es PASS, workspace 646/0, sweep exit 0, acceptance - grep CLEAN. - -## Concerns - -(none) - -## Known debt - -(none — the audit Resolution scope is fully covered: Resolution-1 -history→journal across the corrected 5-file set, Resolution-2 stale -below, Resolution-3 sweep re-scope+lockstep, Resolution-4 faithful- -superset clause-3, Resolution-5 exit state.) - -## Blocked detail - -(none — DONE) - -## Files touched - -- Test gate: `crates/ailang-core/tests/design_index_pin.rs` -- Contract strips: `design/contracts/typeclasses.md`, - `design/contracts/str-abi.md`, - `design/contracts/scope-boundaries.md`, - `design/contracts/float-semantics.md`, - `design/contracts/roundtrip-invariant.md`, - `design/contracts/data-model.md` -- Decision-record archive: `docs/journals/2026-05-19-design-decision-records.md` -- Sweep re-scope + lockstep: `bench/architect_sweeps.sh`, - `skills/audit/agents/ailang-architect.md` - -## Stats - -bench/orchestrator-stats/2026-05-19-iter-design-md-rolesplit.tidy.json diff --git a/docs/journals/2026-05-19-iter-embedding-abi-m5.1.md b/docs/journals/2026-05-19-iter-embedding-abi-m5.1.md deleted file mode 100644 index 793844b..0000000 --- a/docs/journals/2026-05-19-iter-embedding-abi-m5.1.md +++ /dev/null @@ -1,112 +0,0 @@ -# iter embedding-abi-m5.1 — lean `ail-embed` core + build.rs + hermetic smoke - -**Date:** 2026-05-19 -**Started from:** 22f02aa26b57c478760265dfc6468b54d35c7cdc -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -Stood up the workspace-excluded `ail-embed` crate: a zero-dependency -embedding core that links the M3-frozen staticlib via a `build.rs` -and folds a price stream through the frozen `(State, Tick) -> State` -C ABI, plus a hermetic `data-server` smoke test proving the real -adapter wiring without `/mnt`. The crate is its own cargo workspace -root (not an AILang `[workspace]` member); `data-server` is a -dev-dependency only, so Invariant 1 holds in the dependency graph, -mechanically verified by both the `cargo metadata` grep (Task 1 -Step 7 → `0`) and the `git status` path filter (Task 3 Step 4 → -empty). Zero diff to `crates/ailang-*`, `crates/ail/`, `runtime/`, -`examples/*.ail`. Both ratifying tests are green -(`kernel_run_sums_prices` unit + `hermetic_smoke_data_server_roundtrip` -integration; 2/0). Two minimal, toolchain-forced corrections to the -plan's verbatim `ail-embed/Cargo.toml` were required (see Concerns) — -neither alters the iter's intent or any acceptance gate. - -## Per-task notes - -- iter embedding-abi-m5.1.1: Scaffold the workspace-excluded crate + - build.rs. Created `ail-embed/{Cargo.toml,.gitignore,src/lib.rs - (stub),build.rs}`; appended the deliberate-non-membership comment - to the root `Cargo.toml` members block (verbatim). Build gate - PASS, `cargo metadata … grep -c data-server` == 0. RED-first N/A - (build-infra de-risking task; verification is the build+metadata - gate, which the plan scripts). -- iter embedding-abi-m5.1.2: The lean embedding core. RED-first - honored: wrote the test-only `src/lib.rs` first, observed RED - (`unresolved import \`super::Kernel\``, exactly the plan's - expected reason), then prepended the verbatim impl block - (`extern "C"` ABI, `Ctx` + frozen-layout box helpers + `Drop`, - `Kernel::new`/`run`, `Default`). GREEN: `kernel_run_sums_prices` - 1/0. -- iter embedding-abi-m5.1.3: Hermetic data-server smoke. Created - `ail-embed/tests/smoke.rs` verbatim (synthetic Pepperstone ZIP - fixture → real `DataServer` → `Kernel` fold → bit-exact assertion - vs host reference and against the closed-form `55.0`). Test-only - task (no separate RED). Full suite 2/0; Invariant-1 path filter - empty. - -## Concerns - -- iter .1 (DONE_WITH_CONCERNS): the plan's verbatim - `ail-embed/Cargo.toml` could not satisfy its own Step 6 build gate - as written. Two minimal toolchain-forced corrections, both confined - to the in-scope plan-created `ail-embed/Cargo.toml`, neither - changing iter intent or any acceptance gate: - 1. Added an empty `[workspace]` table. Cargo hard-errors on a - nested package that "believes it's in a workspace when it's - not"; the empty table is cargo's documented mechanism for a - deliberately workspace-excluded nested crate (and is what makes - the Step 7 metadata grep return `0` — the AILang root manifest - no longer traverses into `ail-embed`). A comment at the table - records the rationale. - 2. Corrected the `data-server` dev-dep path from - `../libs/data-server` to `../../libs/data-server`. The crate - lives at `/home/brummel/dev/libs/data-server` — a sibling of - the repo *directory*, one level above the repo root. From - `ail-embed/Cargo.toml`, `../` is the repo root, so the plan's - literal resolved to the non-existent - `repo/libs/data-server`. The plan's read-only cross-reference - anchors (`../libs/data-server/...`) are written from the - repo-root cwd and are correct *there*; only the manifest path - literal, resolved relative to `ail-embed/`, needed the extra - `../`. `build.rs` is unaffected (it never references - data-server; it computes `repo = manifest.parent()`). - - Plan-template implication for the orchestrator: planner path-recon - for a workspace-excluded nested crate should (a) emit the empty - `[workspace]` table in the manifest template and (b) resolve - sibling-dependency paths relative to the *manifest's* directory, - not the repo-root cwd the anchors are written from. - -## Known debt - -- No `examples/*.ail.json` + `crates/ail/tests/e2e.rs` E2E fixture - added. Deliberate, not a gap: this iter ships zero compiler-surface - change (mechanically enforced by Task 3 Step 4 — adding a fixture - under `examples/` or `crates/ail/tests/` would itself break the - iter's Invariant-1 acceptance). The milestone invariant - ("M3-frozen staticlib embeds + folds bit-exactly via the C ABI; - data-server is a dev-only meeting point") is already protected by - the two tests this iter ships. Standard E2E protects compiler - invariants; there is none to protect here. -- Adapter API + thread-swarm are explicitly deferred to embedding-abi - iter 2+ per the plan and spec — not touched, not debt of this iter. - -## Blocked detail - -N/A — DONE. - -## Files touched - -- Modified: `Cargo.toml` (root — 4-line non-membership comment after - the `members` array; no other change) -- Created: `ail-embed/Cargo.toml`, `ail-embed/.gitignore`, - `ail-embed/build.rs`, `ail-embed/src/lib.rs`, - `ail-embed/tests/smoke.rs` -- Untracked build artefacts under `ail-embed/target/` are - `.gitignore`d by `ail-embed/.gitignore`. - -## Stats - -bench/orchestrator-stats/2026-05-19-iter-embedding-abi-m5.1.json diff --git a/docs/journals/2026-05-19-iter-embedding-abi-m5.2-resume-attempt.md b/docs/journals/2026-05-19-iter-embedding-abi-m5.2-resume-attempt.md deleted file mode 100644 index a2ff0ab..0000000 --- a/docs/journals/2026-05-19-iter-embedding-abi-m5.2-resume-attempt.md +++ /dev/null @@ -1,91 +0,0 @@ -# iter embedding-abi-m5.2-resume-attempt — atomic-global fix is necessary but INSUFFICIENT for the swarm leak-proof - -**Date:** 2026-05-19 -**Started from:** 7bfa11e (GREEN of bugfix-rc-global-stats-race) -**Status:** RE-QUARANTINED — residual handed to a new debug cycle -**Tasks completed:** 0 of 1 (the un-`#[ignore]` was attempted, verified -unsound, and reverted; the honest-rationale correction is the only -shipped change) - -## Summary - -After the runtime fix `bugfix-rc-global-stats-race` (`7bfa11e`, -Boss-verified: the isolated 8×2M pure-global RED -`embed_rc_global_stats_race` deterministically green 3/3; full -`cargo test -p ail` green), I un-`#[ignore]`d the M5 swarm leak-proof -`symbol_fan_swarm_leak_free` — that un-ignore was *designed* as the -integration-level acceptance of the runtime fix. **Boss verification -falsified the premise that the runtime fix was sufficient.** Across 4 -swarm runs: runs 1–2 GREEN, runs 3–4 FAILED; a 6-run capture: 3 -GREEN, run 4 FAILED `left: 11998383, right: 12000003`. Signature: -`Σfrees` is **stable-exact (12000003) every run**; `Σallocs` is short -by a **jittering ~1600–2260 in ~1/3 of runs**, across the 4 -`ailang_rc_stats:` lines (3 per-ctx + 1 global atexit). - -The atomic-global fix is real, committed, and **necessary** — it -removed the *pure* global-counter contention (the isolated RED is now -deterministic). It is **not sufficient** for the swarm: a *second, -distinct* alloc-undercount remains that the isolated 8×2M pure-global -RED did not model. The un-ignore was reverted (`git checkout` — it -was never committed); the now-stale `#[ignore]` rationale that said -"un-ignore when the runtime makes the counters atomic" was corrected -to the accurate necessary-but-insufficient state (this honesty fix is -the only shipped diff — a future agent seeing `7bfa11e` must NOT -un-ignore on that basis). - -## The asymmetry (orchestrator framing for the debug carrier, NOT a fix) - -`Σfrees` perfectly stable while `Σallocs` jitters is the load-bearing -clue. Both global counters are now `_Atomic`; both per-ctx counters -are plain `++` but single-writer-per-thread (each worker owns its -`Ctx`; `Ctx: !Send`). If a per-ctx counter raced, *both* alloc and -free would jitter (the dec path also `++`s a per-ctx counter) — it -does not. So the lost increments are specifically on the **alloc** -side, and not pure global contention (RED green). Boss reasoning -could not localise it further; per the debug Iron Law that is -where Boss speculation stops. - -Two framings the debug cycle must both consider (do not pre-judge): - -1. **A genuine residual runtime defect** — some alloc events on the - swarm path are uncounted non-deterministically (a TLS-ctx - set/restore window across the M3 forwarder boundary; an - interaction with data-server's prefetch background threads; a - path the 8×2M ctx-less RED structurally cannot reach because the - swarm mixes TLS-NULL host allocs with TLS-ctx kernel-internal - allocs). -2. **A test-methodology unsoundness** — `Σ` over a *heterogeneous* - set of lines (one process-global atexit line + N per-ctx - `ailang_ctx_free` lines, with per-box alloc/free attribution - split asymmetrically between global and ctx) may not be a sound - leak invariant for a multi-thread host even with correct - counters. If so the fix is a methodology redesign (e.g. - per-symbol single-thread subprocess leak-proof, or assert each - line balances under a single-thread-per-process shape), not - another runtime change — and that becomes a design decision, - possibly a bounce-back, only AFTER root cause. - -## Disposition - -- `symbol_fan_swarm_leak_free` STAYS `#[ignore]`d, body verbatim - (still quarantined, NOT weakened). Rationale corrected in all - three breadcrumbs (module doc, fn doc, `#[ignore]` reason). -- `symbol_fan_swarm_bit_exact` remains live & GREEN — the M5 - existence proof is unaffected (the swarm IS actually correct and - leak-free; only the *accounting* is wrong). -- The residual is handed to a fresh `debug` RED-first cycle - (`bugfix-swarm-rc-alloc-undercount`). main stays green at the - honesty-fix commit. -- M5 (the milestone) stays open `[~]`; m5.3 (time-shard + - friction-harvest) remains downstream of a sound leak-proof. - -## Files touched - -- `ail-embed/tests/swarm.rs` (the three `#[ignore]`-rationale - breadcrumbs corrected for doc-honesty; the test body and the - `#[ignore]` itself unchanged) - -## Stats - -(no orchestrator-stats file — this is a Boss-side honesty correction + -finding record, not an implement-orchestrator iteration) diff --git a/docs/journals/2026-05-19-iter-embedding-abi-m5.2.md b/docs/journals/2026-05-19-iter-embedding-abi-m5.2.md deleted file mode 100644 index 3bd4fe0..0000000 --- a/docs/journals/2026-05-19-iter-embedding-abi-m5.2.md +++ /dev/null @@ -1,234 +0,0 @@ -# iter embedding-abi-m5.2 — data-server adapter + symbol-fan swarm; leak-free gate surfaced a runtime race - -**Date:** 2026-05-19 -**Started from:** 9cc9d9c5170eb8d52152dcef071726c7a6ffd05d -**Status:** PARTIAL -**Tasks completed:** 2 of 3 - -## Summary - -Tasks 1 and 2 landed clean and verbatim-to-plan: `data-server` was -promoted dev-dep → real dep, the additive `adapter` module -(`tick_to_px` / `MidPriceStream` / `fold_symbol`) was added with a -genuine compile-error RED → GREEN cycle, and the `swarm_runner` bin -compiled — that compile *is* the compile-time per-thread-ctx proof -(`Ctx: !Send`; a shared-ctx design is `error[E0277]` and could not -have built). Invariant 1 holds independently: the AILang workspace -graph has zero `data-server` (Step 4 = `0`) and there is zero -compiler-surface diff (Step 5 path filter empty — only `ail-embed/**` -touched, not even the root `Cargo.toml`). - -Task 3 is BLOCKED. The plan's verbatim leak-free assertion -(`Σallocs == Σfrees` across all `ailang_rc_stats:` lines) is correct -and is doing its job: it surfaced a genuine concurrency defect. The -swarm E2E passed once in isolation by luck (the racing counter -happened to land on the balanced value) and then FAILED in the -full-suite run and on 4 of 5 direct re-runs. The host-side box -allocations in `ail-embed/src/lib.rs` (`make_state` / `make_tick` / -the final `ailang_rc_dec`) run on the Rust worker thread where -`__ail_tls_ctx == NULL` — the M3 forwarder only sets the TLS ctx for -the synchronous duration of the kernel call, not for the host's own -allocs. With `__ail_tls_ctx` null those allocs fall through to the -**non-atomic global statics** `g_rc_alloc_count` / `g_rc_free_count` -(`runtime/rc.c:161,212`), incremented concurrently by 3 worker -threads with no synchronisation. The atexit `g_rc` line's `allocs` -is consequently non-deterministic across runs -(5,993,321 … 5,999,318 over 5 runs; should be a stable 6,000,000) -while `frees` stays at a stable 3. The bit-exactness half of the -test is sound and passed every run; only the global leak-accounting -is racy. - -This is exactly the class of finding the symbol-fan swarm milestone -exists to expose — and it is out of m5.2 scope to fix (see Blocked -detail). The Boss decides: extend the spec / m5.3 to cover the -host-side-alloc TLS-ctx routing (or atomic global counters), or -re-scope the leak-free acceptance. - -## Per-task notes - -- iter embedding-abi-m5.2.1: promote `data-server` dev-dep → real - dep + additive `adapter` module. RED (`E0432 unresolved import - super::tick_to_px`) → impl `tick_to_px`/`MidPriceStream`/ - `fold_symbol` → GREEN (`tick_to_px_is_mid`, + - `kernel_run_sums_prices` unregressed under `--lib`). Cargo.toml + - `lib.rs:7` verbatim-to-plan. Stale m5.1 "ZERO dependencies" - manifest comment corrected for present-tense honesty (quality - phase). -- iter embedding-abi-m5.2.2: `src/bin/swarm_runner.rs` written - verbatim; `cargo build --bin swarm_runner` → exit 0. The clean - compile IS the compile-time per-thread-ctx proof (`Ctx: !Send`; - no `E0277`; per-thread-owned-Kernel structure intact, not - collapsed to a shared ctx). -- iter embedding-abi-m5.2.3: `tests/swarm.rs` written verbatim. The - E2E genuinely executed (NOT the skip path — 3 tick symbols - present: EURUSD, GER40, XAUUSD; ~4 s, 3×2M FFI folds). Per-symbol - bit-exactness vs independent host reference: sound, passed every - run. Global `Σallocs==Σfrees` leak-free assertion: BLOCKED — - correctly detects a non-atomic global-counter race (see Summary + - Blocked detail). Not re-loopable (the plan's verbatim code is - exactly as specified; no re-attempt makes a racy C counter - deterministic) and not fixable in-scope (Step 5 mandates zero - `runtime/` diff; `runtime/` is M3-frozen). - -## Concerns - -- Plan Task 1 Step 6's literal command - (`cargo test --manifest-path ail-embed/Cargo.toml`, unfiltered) - is unsatisfiable in Task 1's isolation: the `[[bin]] swarm_runner` - declaration added by Step 2's verbatim Cargo.toml replacement - references `src/bin/swarm_runner.rs`, which Task 2 Step 1 creates. - The plan Self-review point 7 ("No later task is required to make - an earlier compile gate pass") is incorrect for this command. - Task 1's no-core-regression intent was fully proven via `--lib` - (`kernel_run_sums_prices` + `tick_to_px_is_mid` GREEN); the - smoke + full-suite gate is covered by Task 3 Step 3 once the bin - exists. The verbatim Cargo.toml was NOT split to dodge this - (splitting it would be the spec-compliance violation; the - `[[bin]]` + dep-promotion is one cohesive manifest change). No - code defect — purely a plan command-ordering defect. -- Quality phase corrected the pre-existing m5.1 Cargo.toml comment - ("the library itself has ZERO dependencies … data-server is - dev-only") which became false the moment `data-server` moved into - `[dependencies]`. The plan's verbatim replacement boundary started - *at* `[dependencies]` and did not script the preceding comment's - removal; leaving a now-false manifest comment violates the - DESIGN.md honesty rule (comments mirror current state). The - replacement comment states the present-tense m5.2 framing, - consistent with the plan's own new block + Boss decision. - -## Known debt - -- The host-side embedding allocation path (`ail-embed/src/lib.rs` - `make_state`/`make_tick`/final `ailang_rc_dec`) executes with - `__ail_tls_ctx == NULL` (the M3 `@` forwarder sets the TLS - ctx only for the synchronous kernel-call duration). Under the - multi-thread swarm these allocs/frees land on the non-atomic - global `g_rc_alloc_count`/`g_rc_free_count` fallback - (`runtime/rc.c:161,212`), which races. Not touched here: - `runtime/` is M3-frozen and Step 5 mandates zero `runtime/` diff; - the host-side-routing alternative is a design change beyond - "additive adapter + swarm bin". Reserved for Boss/m5.3 design. - -## Blocked detail - -- **Task:** 3 (Step 2 / Step 3 — the leak-free `Σallocs==Σfrees` - gate). -- **Reason:** `worker-blocked` — the plan's own verbatim acceptance - gate correctly detects a runtime concurrency defect whose fix is - out of m5.2 scope and would violate this iter's Invariant-1 - Step-5 zero-`runtime/`-diff constraint. -- **Worker says (verbatim):** Task 3's `tests/swarm.rs` is verbatim - to plan and genuinely executes (bit-exact half sound, GREEN every - run). The `Σallocs==Σfrees` half is non-deterministic: full-suite - run FAILED `left: 11997743, right: 12000003`; 5 direct child runs - of the atexit `g_rc` line gave `allocs=` 5993699/5996029/5993321/ - 5999318/5996019 with `frees=3` stable. Root cause: host-side - `ailang_rc_alloc`/`ailang_rc_dec` from `lib.rs` run with - `__ail_tls_ctx==NULL`, falling through to the non-atomic global - `g_rc_*` counters (`rc.c:161,212`) raced by 3 worker threads. Not - re-loopable (verbatim plan code; a racy C counter is not made - deterministic by re-attempting the same code) and not - in-scope-fixable (zero `runtime/` diff mandated; `runtime/` is - M3-frozen). -- **Suggested next step:** Boss decision — either (a) extend the - spec/m5.3 to route host-side `ail-embed` allocs through a - per-thread `ailang_ctx_t` so the swarm's leak accounting is - per-thread-attributed (no global fallback), or make - `g_rc_alloc_count`/`g_rc_free_count` atomic in a *separate* - runtime iteration outside m5.2's zero-`runtime/`-diff constraint; - or (b) re-scope the m5.2 leak-free acceptance. Tasks 1 + 2 are - clean and committable as a PARTIAL; the architect should see the - TSan-discharge reasoning below at milestone close. - -## Boss-decision rationale carried forward (mirrored from the plan, for the architect at milestone close) - -- **TSan discharge (compile-time + cited green precedent, no new - harness).** The spec's "sanitiser-clean (no data race across - per-thread ctxs)" acceptance is discharged by (1) `Ctx`/`Kernel` - being `!Send` (raw `*mut c_void`) — the swarm is *structurally - forced* one-ctx-per-thread; a shared-ctx design is `error[E0277]` - and cannot compile, so Task 2 Step 2's clean build is a - compile-time race-freedom proof for the *per-ctx kernel calls*; - (2) the C-runtime race-freedom of that exact per-thread-ctx - pattern is ratified green by `crates/ail/tests/embed_swarm_tsan.rs` - (C host, 8 pthreads, own ctx each, `clang -fsanitize=thread`, - exit-clean), cited not re-run. NOTE FOR THE ARCHITECT: this - discharge covers the *per-thread-ctx kernel path*. The Task 3 - finding is a *different* surface — the host-side `ail-embed` - allocs that bypass the TLS ctx and hit the non-atomic global - `g_rc_*` fallback. The TSan precedent does not cover that path - (the C host in `embed_swarm_tsan.rs` does not exercise a Rust - host allocating boxes with a null TLS ctx). The compile-time - `!Send` proof and the cited green TSan remain valid for what they - claim; they simply do not reach the global-fallback race m5.2 - newly exposed. -- **`data-server` dev-dep → real dep is sanctioned.** Invariant 1 - governs the *compiler* crates (`ailang-*`, `crates/ail`, - `runtime/`), which gain nothing — verified clean this iter - (Step 4 = `0`, Step 5 empty). `ail-embed` is explicitly the - *sole* data-server↔AILang meeting point. -- **Deferred to m5.3 (not this plan):** time-shard - boundary-invisibility, friction-harvest timing, milestone - close-out. Not implemented here (scope discipline held — no - time-shard, no perf timing, no close-out crept in). - -## Files touched - -- `ail-embed/Cargo.toml` (modified — dep promotion + `[[bin]]` + - stale-comment honesty fix) -- `ail-embed/src/lib.rs` (modified — `pub mod adapter;`, additive) -- `ail-embed/src/adapter.rs` (new — the sole data-server-knowing layer) -- `ail-embed/src/bin/swarm_runner.rs` (new — symbol-fan swarm bin) -- `ail-embed/tests/swarm.rs` (new — symbol-fan leak-free + bit-exact - E2E; bit-exact half sound, leak-free half surfaces the race) - -## Stats - -bench/orchestrator-stats/2026-05-19-iter-embedding-abi-m5.2.json - -## Boss disposition (2026-05-19) - -Boss independently read `runtime/rc.c` and confirms the diagnosis is -**precise and the finding is an instrumentation-measurement race, NOT -a memory bug**: the per-object refcount header ops (`*hdr +=/-= 1`, -rc.c:180/208) are non-atomic but **no box crosses a thread** in the -swarm (`Ctx: !Send` structurally forces alloc/use/free within one -worker), so there is no real refcount race, no double-free, no leak — -the bit-exact half is GREEN every run, confirming the memory -management is actually correct. The *only* raced object is the global -`g_rc_*` statistics counter (rc.c:90-91,161,212), **non-atomic by -rc.c's own explicit design** (header comment rc.c:44-49,85: "AILang -has no concurrency primitives yet; when it acquires them, -atomic-vs-non-atomic becomes a separate decision"). M5 is AILang's -first concurrent consumer — that deferred decision is now due. - -Committed as a PARTIAL (Tasks 1+2 + the GREEN existence proof) so the -sound work + the finding land on the record, main stays green: -`tests/swarm.rs` was Boss-split into `symbol_fan_swarm_bit_exact` -(live, GREEN — the real-data existence proof) and -`symbol_fan_swarm_leak_free` (`#[ignore]`, **body preserved -verbatim** — quarantined, not weakened; un-`#[ignore]` is the -acceptance criterion for the runtime fix). Earlier plan/journal claim -that `embed_swarm_tsan.rs` covers this path was wrong and is -corrected here: that test uses the *scalar* kernel (zero box allocs) -so it never exercised the host-side global-counter path; the -compile-time `!Send` proof + the cited green TSan remain valid only -for the per-thread-ctx *kernel* path, which they always claimed. - -Escalated to the user as an M5 **bounce-back** (touches M3-frozen -`runtime/`; re-frames M5's "zero runtime change" commitment; -multiple substantive options — not unilaterally Boss's to pick in -frozen-runtime territory). Boss recommendation on the record: -**Option A** — make *only the global fallback* `g_rc_*` counters -atomic (leave the per-thread ctx path plain — no contention there), -as its own RED-first runtime micro-iteration (`debug` skill), outside -M5's "zero runtime change" framing; then resume the M5 leak-proof on -the now-valid measurement (un-`#[ignore]` `symbol_fan_swarm_leak_free` -as that iter's GREEN). Rationale: it discharges exactly the deferred -decision rc.c's own comment names, at the defect's actual location, -without touching the frozen value layout / ABI offsets / host-free -rule (the literal content of the M3 freeze); Option B (host-side -TLS-ctx routing) adds a new runtime API surface; Option C (per-symbol -single-thread subprocess leak-proof) is sound but weaker and leaves -the runtime's documented single-thread-stats footgun for every future -concurrent consumer. User picks the direction. diff --git a/docs/journals/2026-05-19-iter-embedding-abi-m5.3.md b/docs/journals/2026-05-19-iter-embedding-abi-m5.3.md deleted file mode 100644 index 3af3547..0000000 --- a/docs/journals/2026-05-19-iter-embedding-abi-m5.3.md +++ /dev/null @@ -1,101 +0,0 @@ -# iter embedding-abi-m5.3 — time-shard boundary-invisibility proof + friction harvest - -**Date:** 2026-05-19 -**Started from:** 67027ab0f98210926ed30ffd551306d3db3645ee -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -Final functional M5 iteration. Proves the chunk/window-boundary-invisibility -claim the M4 retirement rests on: one symbol (EURUSD), three disjoint -contiguous single-month windows (2017-03 / 2017-04 / 2017-05), each folded -on its own thread owning its own `Kernel` (`Ctx: !Send` makes -one-ctx-per-thread a compile-time guarantee), each shard's `(acc,n)` -asserted **bit-exact** vs a single-thread host fold of that exact window in -data-server stream order. Adds the windowed adapter sibling `fold_window` -(purely additive after `fold_symbol`), a new single-responsibility -`[[bin]] timeshard_runner`, and `tests/timeshard.rs` carrying the spec §3 -strength-ordered (a)/(b)/(c) assertions plus the deterministic leak-Σ and a -journal-only friction-timing capture (no bench gate, no timing assertion). -The shipped m5.2 symbol-fan path (`swarm_runner.rs`, `swarm.rs`) and m5.1 -core stay byte-untouched and green. Invariant 1 holds (the AILang workspace -graph has zero `data-server`; `chrono` enters only as an `ail-embed` -dev-dep). Determinism re-verified 6× (5× standalone + 1× in full suite), -all identical. Friction-harvest observed: -`TIMING total_ticks=3192562 elapsed_ms=658` → derived per-tick-FFI cost -≈ 658 ms / 3,192,562 ≈ **~206 ns/tick** (the Boss records this as the P2 -flat-array-decision input in the close-out journal). - -## Per-task notes - -- iter embedding-abi-m5.3.1: windowed adapter + chrono dev-dep + bin - declaration — `fold_window(server, symbol, from_ms, to_ms)` appended to - `ail-embed/src/adapter.rs` immediately after `fold_symbol`, before the - `#[cfg(test)] mod tests` block (windowed sibling, same - `MidPriceStream`+`Kernel` path, differs only by `stream_tick_windowed`); - `chrono = "0.4"` added to `[dev-dependencies]` + `[[bin]] timeshard_runner` - declared in `ail-embed/Cargo.toml`. `--lib` regression check - `2 passed; 0 failed` (exactly the plan's prediction). Spec compliant, - quality approved, 0 re-loops. -- iter embedding-abi-m5.3.2: the time-shard E2E test (RED) — - `ail-embed/tests/timeshard.rs` created verbatim from the plan - (boundary-invisibility (a)/(b)/(c) + leak-Σ + friction capture). - RED observed: the declared-but-absent `[[bin]] timeshard_runner` source - makes the test binary fail to build. Spec compliant, quality approved, - 0 re-loops. DONE_WITH_CONCERNS — see Concerns (RED-text discrepancy, - same cause). -- iter embedding-abi-m5.3.3: the `timeshard_runner` bin (GREEN) — - `ail-embed/src/bin/timeshard_runner.rs` created verbatim (one thread per - window, each building its own `Kernel`; prints `RESULT`/`TIMING`). - Step 2 GREEN (`1 passed; 0 failed`, ran 2.71s against real /mnt data — - did NOT skip). Step 3 determinism ×5: 5/5 `ok`. Step 4 full suite: - 0 failed AND 0 ignored across all binaries (m5.2 `symbol_fan_swarm_*` - un-ignored & green). Step 5: `cargo metadata` data-server count = 0; - path filter empty. Step 6: `embed_rc_global_stats_race` - `1 passed; 0 failed`. Spec compliant, quality approved, 0 re-loops. - -## Concerns - -- iter embedding-abi-m5.3.2 (DONE_WITH_CONCERNS): the Task 2 Step 2 RED - surfaced as `error: couldn't read \`src/bin/timeshard_runner.rs\`: No such - file or directory` rather than the plan's predicted `error: environment - variable \`CARGO_BIN_EXE_timeshard_runner\` not defined at compile time`. - Same root cause (the declared `[[bin]] timeshard_runner` target has no - source until Task 3), same trigger, same resolution (disappears exactly - when Task 3 lands the bin) — Cargo fails at the missing-source check - before reaching the `env!` expansion, so the surface error differs but - the RED is genuine, deterministic, and not a false-green. Observation - only; the plan's intent (a deterministic compile-failure RED isolating - Task 2 from Task 3) is fully satisfied. - -## Known debt - -- (none introduced by this iter) -- Pre-existing `docs/DESIGN.md:2358-2360` "additive M4 concern" drift is - explicitly out of m5.3 scope (spec §"Out of scope" / Boss decision) and - was deliberately not touched — the milestone-close audit handles it, - not this iter. - -## Blocked detail - -(not applicable — Status DONE) - -## Files touched - -- `ail-embed/src/adapter.rs` (modified — additive `fold_window`) -- `ail-embed/Cargo.toml` (modified — `chrono` dev-dep + `timeshard_runner` bin) -- `ail-embed/Cargo.lock` (modified — single line: `chrono` direct-dep edge; - automatic, strictly required by the Cargo.toml dep add, already resolved - transitively via data-server) -- `ail-embed/src/bin/timeshard_runner.rs` (created) -- `ail-embed/tests/timeshard.rs` (created) - -All within the plan's declared file set and the Step 5b allowed path scope -(`ail-embed/**`, `docs/`, `bench/orchestrator-stats/`). Zero diff to -`crates/ailang-*`, `crates/ail/`, `runtime/`, `examples/*.ail`, root -`Cargo.toml`, `docs/DESIGN.md`. - -## Stats - -bench/orchestrator-stats/2026-05-19-iter-embedding-abi-m5.3.json diff --git a/docs/journals/2026-05-19-iter-embedding-abi-m5.tidy.md b/docs/journals/2026-05-19-iter-embedding-abi-m5.tidy.md deleted file mode 100644 index 8a40e35..0000000 --- a/docs/journals/2026-05-19-iter-embedding-abi-m5.tidy.md +++ /dev/null @@ -1,81 +0,0 @@ -# iter embedding-abi-m5.tidy — milestone-close doc-honesty drift reconciliation - -**Date:** 2026-05-19 -**Started from:** e81fb90d88bda546829a35496a02ee9c79f80150 -**Status:** DONE -**Tasks completed:** 3 of 3 - -## Summary - -The M5 milestone-close audit (`2026-05-19-audit-embedding-abi-m5.md`) -found three doc-honesty drift items — two `docs/DESIGN.md` prose -inaccuracies and one `runtime/rc.c` stale performance-aside comment. -This tidy reconciles all three: DOC/COMMENT-ONLY, pin-safe, zero -language/checker/codegen/schema/runtime-behaviour change. Mirrors the -M2.tidy (`a80d495`) / M3.tidy (`63d7d60`) precedent — the existing pin -tests (`design_schema_drift`, `docs_honesty_pin`, -`effect_doc_honesty_pin`, `embed_record_layout_pin`) staying green -before AND after is the mechanical proof the edits moved no pinned -byte; that is the coverage, so no RED test, no audit/fieldtest gate. -This is the final M5 iteration; M5 closes after Boss-verify -(roadmap [~]→[x] + done-state are Boss-side). - -## Per-task notes - -- iter embedding-abi-m5.tidy.1: `docs/DESIGN.md` doc-honesty (edit-1 - + edit-2). edit-1 §"Free (host side)" (~:2356-2360): dropped the - retired-M4 forward-reference ("an additive M4 concern") — M4 was - retired 2026-05-18; the boxed-field record is simply - export-gate-rejected (present-tense current fact). edit-2 - §"Embedding ABI" (~:2290-2293): reconciled the stale "no shared - mutable runtime state … data-race-free, sanitiser-verified" claim - with the now-atomic global RC-stats fallback — the per-allocation - hot path (per-ctx counters + per-object refcount header) is - non-atomic and never shared (`Ctx: !Send`, single-thread-per-ctx); - the one shared datum is the atomic-relaxed global RC-stats fallback - counter. Pin-safety proof green (8/5/4 passed, 0 failed across the - three ailang-core pin suites); the adjacent separately-pinned - line (`Export parameters are written **bare**:`, - `docs_honesty_pin.rs:135`) is byte-identical, shifted line 2299→2305 - by the net-added lines above it but content unchanged (the pin is a - substring assertion, not a line-number assertion — it passes). -- iter embedding-abi-m5.tidy.2: `runtime/rc.c` 18g.0-block (~:88-89) - comment-only honesty fix. Replaced the stale "two unconditional - `++` operations" performance aside with the accurate "one counter - bump per alloc/free … relaxed atomic add on the global null-ctx - fallback, a plain `++` on the per-ctx path", consistent with the - adjacent already-correct :93-106 atomic paragraph and :45-55 - Threading header. Mechanical comment-only filter clean (every - changed line is a `*`-prefixed C comment); frozen-IR pin - `embed_record_layout_pin` byte-identical (1 passed, 0 failed — it - asserts generated IR, never rc.c comment text; M3.tidy precedent). -- iter embedding-abi-m5.tidy.3: consolidated doc-only + scope gate - (verification only, no files). All pins green across both crates - (`grep -c 'test result: ok'` == 3 + `embed_record_layout_pin` ok); - `git status --porcelain` shows ONLY `docs/DESIGN.md` + `runtime/rc.c` - as content changes (zero `crates/**`, `examples/**`, `ail-embed/**`, - `Cargo.toml`, `bench/baseline.json`); `cargo build --workspace` - Finished — confirms no behaviour change. - -## Concerns - -(none) - -## Known debt - -(none — this iteration *clears* the M5-audit doc-honesty debt; no new -debt introduced.) - -## Blocked detail - -(n/a — DONE) - -## Files touched - -- `docs/DESIGN.md` (§"Embedding ABI" edit-2 ~:2290-2293; §"Free (host - side)" edit-1 ~:2356-2360 — prose-only) -- `runtime/rc.c` (18g.0 comment block ~:88-89 — C-comment text only) - -## Stats - -bench/orchestrator-stats/2026-05-19-iter-embedding-abi-m5.tidy.json diff --git a/docs/journals/INDEX.md b/docs/journals/INDEX.md deleted file mode 100644 index 63ddfd2..0000000 --- a/docs/journals/INDEX.md +++ /dev/null @@ -1,122 +0,0 @@ -# Journal index - -> Chronological pointer list of per-iter journal files. Append-only. -> One line per iter, newest at the bottom. Filenames are relative to -> `docs/journals/`. - -- pre-2026-05-11 — see `../journal-archive.md` for all prior history -- 2026-05-11 — iter or.1: orchestrator-refactor → 2026-05-11-iter-or.1.md -- 2026-05-11 — fieldtest canonical-type-names → 2026-05-11-fieldtest-canonical-type-names.md -- 2026-05-11 — iter pr.1: plan-recon subagent → 2026-05-11-iter-pr.1.md -- 2026-05-11 — iter or.2: orchestrator-agent design correction (no nested subagent dispatch) → 2026-05-11-iter-or.2.md -- 2026-05-11 — iter cadence: audit/fieldtest/docwriter cadence restructure → 2026-05-11-iter-cadence.md -- 2026-05-11 — iter 23.4-prep: checker prerequisites for prelude free fns → 2026-05-11-iter-23.4-prep.md -- 2026-05-11 — iter gc.1: grounding-check agent + brainstorm Step 7.5 → 2026-05-11-iter-gc.1.md -- 2026-05-11 — iter disc.1: boss-only commits + main-as-quarantine (no branches in implement) → 2026-05-11-iter-disc.1.md -- 2026-05-11 — iter 23.4: mono-pass unification (class methods + polymorphic free fns in one fixpoint; codegen-time specialiser removed) → 2026-05-11-iter-23.4.md -- 2026-05-12 — iter 23.5: prelude free fns (ne/lt/le/gt/ge) + E2E (positive / user-ADT / Float-NoInstance); milestone 23 closed → 2026-05-12-iter-23.5.md -- 2026-05-12 — audit-23: milestone 23 close (architect clean, compile_check ratified per H2 / 5-fn workload), DESIGN.md stub-fix tidy → 2026-05-12-audit-23.md -- 2026-05-12 — iter rt.1: roundtrip invariant audit tests (3 new tests + ailx-pair dynamic); all PASS first-shot, no gaps surfaced → 2026-05-12-iter-rt.1.md -- 2026-05-12 — iter rt.2: DESIGN.md anchor for Roundtrip Invariant (top-level section + Decision 6 Constraint 2 upward cross-reference) → 2026-05-12-iter-rt.2.md -- 2026-05-12 — audit-rt: milestone close (Roundtrip Invariant) — architect drift fixed in rt.tidy (3 items: Direction-2 5th test, roadmap P1 removed, wording sync); bench all-green → 2026-05-12-audit-rt.md -- 2026-05-12 — iter boss: `/boss` skill — autonomous-mode discipline (Direction freedom, Notifications, WhatsNew procedure) extracted from CLAUDE.md into a user-invoked skill → 2026-05-12-iter-boss.md -- 2026-05-12 — brainstorm Step 7.5: re-dispatch grounding-check on any post-PASS spec edit (skill SKILL.md edit + roadmap P1 todo removed) → see commit `90512d5` (no per-iter journal — small skill-discipline tweak) -- 2026-05-12 — iter cma.1: master mini-spec + render binary for cross-model authoring experiment (13 fixtures cover 34/34 AST variants, 4 test gates green, rendered/{json,ailx}.md checked in; out-of-workspace nested crate) → 2026-05-12-iter-cma.1.md -- 2026-05-12 — iter cma.2: harness binary + 4 tasks + reference solutions + 4 integration tests (six modules, real-pipeline preflight via verify_references, 13/13 tests green; pipeline.rs renames program file to module-name for ail check) → 2026-05-12-iter-cma.2.md -- 2026-05-12 — iter cma.3: live Qwen3-Coder-Next run + DESIGN.md §Decision-6 empirical addendum + roadmap close (AILX 2/4 green vs JSON 1/4; ~98k vs ~207k tokens; single-subject scope explicit, multi-subject expansion queued as P3) → 2026-05-12-iter-cma.3.md -- 2026-05-12 — audit-cma: milestone close (Cross-model authoring-form test) — architect drift fixed in 2 record-keeping items (INDEX backfill for brainstorm Step 7.5, README test-count correction); bench all-green (5 unexplained improvements on latency.explicit_at_rc not coupled to milestone, baseline left pristine) → 2026-05-12-audit-cma.md -- 2026-05-12 — iter ms.1: pipeline anyhow-chain preservation — `{e}` → `{e:#}` at `pipeline.rs:118` restores `serde_json` parse-error leaf in JSON-cohort feedback; pinning unit test added (14/14 green) → 2026-05-12-iter-ms.1.md -- 2026-05-12 — iter ms.2: Qwen3-Coder-Next retroactive re-run + first CodeLlama-13b-Instruct run via IONOS; DESIGN.md §Decision-6 addendum extended from 2-col single-subject to 4-col two-subject; both subjects agree on direction (AILX cheaper + ≥ green); roadmap P3 multi-subject entry removed → 2026-05-12-iter-ms.2.md -- 2026-05-12 — audit-ms: milestone close (Multi-subject Authoring-Form Test — CodeLlama Replication) — architect clean, bench all-green (same 5-metric latency.explicit_at_rc improvement cluster as audit-cma earlier today, baseline left pristine for second consecutive audit) → 2026-05-12-audit-ms.md -- 2026-05-12 — iter ext-rename: `.ailx` → `.ail` extension rename across the live toolchain (61 example renames + 35 content edits + experiment-crate cohort rename `ailx → ail`); historical docs (journals, archive, specs, plans, frozen experiment runs) deliberately untouched; cargo+bench all-green; opens follow-up `.ail`-CLI-acceptance iter → 2026-05-12-iter-ext-rename.md -- 2026-05-12 — iter ext-cli.1: `ail check`/build/run/...all 12 path-taking subcommands now accept `.ail` (Form A) inputs via `ailang_surface::{load_module, load_workspace}` (extension-dispatching loaders + injection point `core::load_workspace_with`); workspace imports prefer `.ail` sibling over `.ail.json`; new `WorkspaceLoadError::SurfaceParse` variant routes surface parse errors as `surface-parse-error` structured diagnostics through `ail check --json`; DESIGN.md §Decision 6 CLI addendum; +7 new tests; closes the loop opened by iter ext-rename → 2026-05-12-iter-ext-cli.1.md -- 2026-05-12 — iter hs.1: heap-Str ABI iter 1 — static-Str LLVM globals migrate from `[N x i8]` to packed-struct `<{ i64, i64, [N x i8] }>` carrying a `UINT64_MAX` sentinel rc-header + explicit `len`; IR-Str pointers now land on the `len`-field; `@puts` / `@ail_str_eq` / `@ail_str_compare` / `@strcmp` callsites GEP +8 to recover the bytes pointer (4 sites; plan listed 3, Task-5 e2e regression surfaced the 4th); 6 IR-shape pins; format strings stay raw `[N x i8]`; cross_lang.py + compile_check.py + full cargo test --workspace all green; byte-identical stdout across every example → 2026-05-12-iter-hs.1.md -- 2026-05-12 — spec amendment: heap-str-abi — sentinel-rc-header story dropped after hs.2 BLOCKED finding (codegen-level elision via move-tracking + non-escape lowering keeps static-Str pointers out of `ailang_rc_dec` along every shipping execution path; no runtime guard needed); re-dispatched grounding-check PASS → see commit `2a72a4a` (no per-iter journal — spec edit) -- 2026-05-12 — iter hs.2: static-Str layout retrofit — drop the `UINT64_MAX` sentinel slot from packed-struct globals (`<{ i64, i64, [N x i8] }>` → `<{ i64, [N x i8] }>`); IR-`Str` pointer now lands on the `len`-field at struct-index 0 (was 1) via constexpr-GEP `i32 0, i32 0`; `+8` consumer-side GEPs at the four C-API callsites invariant across the switch; renamed `..._sentinel_and_len` test to `..._with_len`; updated 5 sites in `crates/ailang-codegen/src/lib.rs` (3 format strings + 2 doc-comments); regenerated `hello.ll` snapshot; full `cargo test --workspace`, cross_lang.py, compile_check.py, check.py all green; static-Str globals are now 8 bytes smaller per literal → 2026-05-12-iter-hs.2.md -- 2026-05-12 — iter hs.3: heap-Str runtime additions — three new symbols appended to `runtime/str.c` (`static str_alloc(uint64_t)` slab helper, `ailang_int_to_str(int64_t)`, `ailang_float_to_str(double)`); supporting ``/``/`` includes + `extern void *ailang_rc_alloc(size_t)` declaration; the extern was promoted from plain-external to `__attribute__((weak))` after the first regression sweep surfaced 11 e2e link failures under `--alloc=gc`/`bump` (public `T` symbols pull in their transitive callees regardless of upstream usage; rc.c is not linked under gc/bump in hs.3 — repair makes str.c link-safe in isolation, no-op once hs.4 lands the unconditional rc.c link); cargo test --workspace + cross_lang.py + compile_check.py + check.py all green on re-sweep; no IR-side caller wired yet (hs.4) → 2026-05-12-iter-hs.3.md -- 2026-05-12 — iter hs.4: IR + checker + linker wiring — `int_to_str : (Int) -> Str` installed in checker + synth.rs lockstep partner; IR-header preamble unconditionally declares both `@ailang_int_to_str` / `@ailang_float_to_str`; `Emitter::lower_app` gets new `int_to_str` arm and replaces `float_to_str`'s `CodegenError::Internal` with the actual call emission; `is_static_callee` whitelist extends to `int_to_str`; `runtime/rc.c` hoisted out of `AllocStrategy::Rc` arm to the unconditional link path (the `__attribute__((weak))` on str.c's `ailang_rc_alloc` extern becomes the documented no-op); 2 new IR-shape pins + 4 new E2E (2 stdout-smoke + 2 RC-stats) + 4 `.ail.json` fixtures; `drop.rs` Str-arm comment refreshed to dual-realisation framing; 5 IR snapshots regen for the 2 new declare lines. Status: DONE_WITH_CONCERNS — heap-Str RC-discipline incomplete (slabs leak at program end) because plan's literal `ret_mode: Implicit` interacts with uniqueness analyser walking `Term::Do` args as `Position::Consume`. Test asserts weakened from `allocs == frees && live == 0` to `allocs >= 1`. Substantive fix requires spec-level effect-op arg-mode decision (deferred, bounce-back to user) → 2026-05-12-iter-hs.4.md -- 2026-05-12 — iter eob.1: Effect-op args walked as Borrow (uniqueness.rs + linearity.rs + linearity.rs doc-comment); heap-Str RC discipline closes (int_to_str / float_to_str `ret_mode: Implicit` → `Own` at 4 lockstep sites across builtins.rs + synth.rs; `drop_symbol_for_binder` App-arm gets `Str` carve-out symmetric to `field_drop_call`'s existing one); 2 pre-existing RED tests at e2e.rs (commit 592d87b) flipped to GREEN unchanged with predicted concrete numbers (`allocs==1,frees==1,live==0` and `allocs==2,frees==2,live==0`); 2 new positive tests + fixtures (primitive-Int through `io/print_int`: zero RC traffic; heap-Str repeated borrow through 2× `io/print_str`: `allocs==1,frees==1,live==0`); DESIGN.md §Decision 10 anchors both arg-position rules together (Ctor=Consume, Do=Borrow, plus a paragraph linking Do=Borrow to ret_mode==Own letbinder-trackability); WhatsNew entry shipped, roadmap P1 `[milestone] Heap-Str ABI` checked off, Post-22-Prelude `depends on:` line removed; 7/7 tasks first-shot, zero review re-loops; lint side-effect surface (over-strict-mode on (own T) params used solely via effect-ops) was empty for the current corpus → 2026-05-12-iter-eob.1.md -- 2026-05-12 — audit-eob: milestone close (heap-str-abi) — architect drift fixed inline as `eob.tidy` (3 DESIGN.md edits: float_to_str / int_to_str caveats dropped, "Str ABI" anchor added documenting both heap-Str and static-Str realisations with their shared consumer ABI and codegen-level non-RC invariant for static-Str); bench mixed (latency.explicit_at_rc improvement cluster reappears for the 3rd consecutive audit + new marginal bump_s regressions on list_sum/hof_pipeline 12% over 10% tol), baseline pristine for 3rd consecutive audit (conservative call: latency cluster has no identified cause across 3 audits — ratify-without-attribution would obscure future signal; bump_s cluster is first-sighting, observe next audit) → 2026-05-12-audit-eob.md -- 2026-05-12 — iter ctt.1: env-overlay shape ratification — new DESIGN.md top-level section `## Env construction` anchors the `env.types` (owning) / `env.ctor_index` (reverse-index) split with the semantic rationale, plus the intentional check-side / mono-side asymmetry (mono-side narrowed at ct.3.2, check-side retains both halves because in-band `DuplicateCtor` consumes the per-module `env.ctor_index` rebuild); new behavioural-pin test `crates/ailang-check/tests/duplicate_ctor_pin.rs` ratifies the consumer; two P2 roadmap todos struck `[x]` (overlay shape question, per-module overlay narrowing); 3/3 tasks first-shot, zero review re-loops, no production-code edits → 2026-05-12-iter-ctt.1.md -- 2026-05-12 — iter ctt.2: `Registry.type_def_module` re-key from `BTreeMap` to `BTreeMap<(String, String), String>` keyed by `(owning_module, bare_name)`; new `caller_module: &str` parameter threads through `normalize_type_for_registry` + `Registry::normalize_type_for_lookup`; four `ailang-check` consumer sites (lib.rs:1640, mono.rs:121/624/1175) pass the correct module-context (env.current_module / defining_module / module_name) per the plan's Design Notes; new regression test plus three-module fixture (`ctt2_collision_{cls,lib,main}.ail.json`) exercises the cross-module bare-name collision shape that pre-ctt.2 silently overwrote; RED-failure observed as `OrphanInstance` rather than the plan's predicted `DuplicateInstance` because the pass-2 coherence check (workspace.rs:656-659) is also a bare-name consumer and fires earlier; both consumers fixed by the same re-key; plan's verbatim two-module fixture had two structural defects (two-way imports → Cycle, qualified `InstanceDef.class` → QualifiedClassName) that the implementer-phase repair handled by introducing the third cls-module; spec-intent (RED-first against bare-name collision) preserved; `cargo test --workspace` green (16 binary-test suites, ~600 tests, zero failures) → 2026-05-12-iter-ctt.2.md -- 2026-05-12 — iter ctt.3: `KindMismatch` retire — pure deletion across 3 files (workspace.rs: variant + dispatch + helper + "dead-but-defensive" doc; main.rs: Display arm; lib.rs: 22b.1 archaeology comment). Existing test `class_param_in_applied_position_fires_canonical_form_rejection` stays green unchanged (asserts `BareCrossModuleTypeRef` on the malformed fixture — the canonical-form-validator successor path). Two adjacent textual-consistency edits ride along (validate_classdefs doc-comment "Three → Two", lib.rs archaeology comment gains a ctt.3 retirement marker). One mid-deletion mechanical fix: dispatch-loop deletion orphaned `mod_name` binding → rewrote `for (mod_name, m) in modules` to `for m in modules.values()` (same body type, mechanical). 2/2 tasks, ~600 tests green across 16 binary-test suites → 2026-05-12-iter-ctt.3.md -- 2026-05-12 — audit-ct-tidy: milestone close (ct-tidy) — architect drift fixed inline as `ctt.tidy` (3 doc-side edits: DESIGN.md §"Class-schema diagnostics" drops the retired `KindMismatch` bullet + adds successor paragraph naming `BareCrossModuleTypeRef`; DESIGN.md §"Higher-kinded class params" rewritten to name `BareCrossModuleTypeRef` from canonical-form validation instead; roadmap.md two P2 todos struck `[x]` with forward-reference to ctt.3 and ctt.2). Bench mixed (check.py exit 1: bump_s persistence on `bench_list_sum` second consecutive sighting at +12.23% → +12.60%; two new first-sighting rc-cohort max_us latency regressions classified as noise pending next audit; the recurring 3-audit `latency.explicit_at_rc` improvement cluster narrowed to within tolerance this audit, withdrawing the ratify-pending plan from audit-eob — the meaningful shrinkage is itself attribution evidence that the cluster is noise-class not signal-class), baseline pristine for the 4th consecutive audit → 2026-05-12-audit-ct-tidy.md -- 2026-05-12 — iter 24.1: `bool_to_str` + `str_clone` runtime + codegen wiring — 2 new C functions in `runtime/str.c` (slab-allocating heap-Str primitives via existing `str_alloc`), lockstep checker + synth.rs installs with `ret_mode: Own`, 2 IR-header declares + 2 `lower_app` arms + `is_static_callee` whitelist extension, 5 IR snapshots regen (2 declares × 5 files), 9 new tests (2 builtins-install unit + 2 IR-shape pin + 5 E2E all green), pre-existing-drift fix included (int_to_str row added to `builtins.rs::list()`). One substantive deviation: builtin-signatures registered in `uniqueness.rs::infer_module` + `linearity.rs::check_module_with_visible` (8 LOC × 2 files) so `str_clone`'s `param_modes: [Borrow]` is visible to the App-arg walker; symmetric to 23.4-prep's class-method registration; necessary for the plan's literal `frees == 3` cross-realisation assertion. Full `cargo test --workspace` 513 passed, 0 failed. `bench/compile_check.py` + `bench/cross_lang.py` green → 2026-05-12-iter-24.1.md -- 2026-05-13 — iter mq.1: class-ref canonical-form extension + workspace-internal class-name qualification — three schema fields (`InstanceDef.class`, `Constraint.class`, `SuperclassRef.class`) move bare → canonical (bare for same-module, `.` for cross-module) symmetric to ct.1's `Type::Con.name` rule; `ClassDef.name` stays bare (defining site, like `TypeDef.name`). `validate_canonical_type_names` gains three field-walks via new `check_class_ref` helper plus two sibling diagnostics `BareCrossModuleClassRef` / `BadCrossModuleClassRef`; `check_class_name_fields` narrowed to `ClassDef.name`-only. Workspace-internal class-name keys all qualified: `class_def_module`, `class_by_name`, registry `entries.0`, `ClassMethodEntry.class_name`, `class_superclasses`, mono's `class_index`, `Origin::Class.class_name`. New `qualify_class_ref` helper in `workspace.rs` plus sibling `qualify_class_ref_in_check` in `ailang-check`. Two new positive on-disk fixtures (`mq1_xmod_constraint_class{,_dep}`). Recon claim that all existing fixtures' class-refs are intra-module was empirically wrong — 5 test_22b* (`orphan_third`, `dup_a/b/entry`, `unbound_constraint_var`) + 5 other (`eq_ord_polymorphic`, `eq_ord_user_adt`, `cmp_max_smoke`, `ctt2_collision_{lib,main}`) fixtures required minimal canonical-form migration. Duplicate-instance test architecturally restructured: post-mq.1 orphan-freedom makes two-modules-on-same-`(class, type)` structurally impossible (any second instance in a non-owning module fires `OrphanInstance` first); new `test_22b1_dup_same_module.ail.json` lands both instances in the class's module — the only post-mq.1 way to land two on the same canonical key. Plan defects fixed inline: `Origin::Class.class_name` had one construction site not two (plan recon off-by-one); `build_class_index` in `mono.rs` needed qualifying (plan said "no code change needed"); `build_module_globals` needed a `Def::Instance` carve-out for qualified `inst.class`; `check_fn`'s declared-constraint matching needed inline canonical-form lifting; `NoInstance` Float-aware message arm needed `prelude.Eq`/`prelude.Ord` match; coherence type-leg needed qualified-head split-and-lookup. Tasks 3-6 ran as one coherent fix (Task 3 alone left tests RED). 7/7 tasks, 520 tests green, `bench/compile_check.py` + `bench/cross_lang.py` exit 0; `bench/check.py` exit 1 due to 4 improvements-beyond-tolerance (audit-ratifiable per convention, not a regression). `MethodNameCollision` + dispatch path unchanged; iters mq.2 + mq.3 land them → 2026-05-13-iter-mq.1.md -- 2026-05-13 — iter mq.2: type-driven dispatch mechanism installed (mechanism-before-exercise) — `Env.method_to_candidate_classes` workspace-flat inverse index built alongside `class_methods`; two new `CheckError` variants `AmbiguousMethodResolution` + `UnknownClass` (Display+code+ctx) plus additive `NoInstance.candidate_classes` field; `ResidualConstraint` extended with `candidates: Option>` (visibility bumped `pub(crate)` → `pub` for test-crate access); pure `resolve_method_dispatch` helper implementing the spec's 5-step rule (qualifier → singleton → type-driven filter → constraint-driven filter → `Multi` for discharge-time refinement); synth Var-arm class-method branch rewritten via `parse_method_qualifier` with inner-dot gate (qualifier must be `.` form, single-dot names like `std_list.length` fall through to the existing qualified-fn path); `refine_multi_candidate_residual` wired into `check_fn` discharge using `expanded` (post-superclass-expansion constraints) for the rigid-var path; `resolve_residual_class_for_mono` wired into mono's `collect_residuals_ordered` residual-to-target mapping. With `MethodNameCollision` still gating real workspaces, the new multi-candidate branches are exercised exclusively by 15 unit tests: 6 in `tests/method_dispatch_pin.rs` covering the 5-step rule's six cases, 6 in `lib.rs` `mod tests` covering variants + field shapes + discharge refinement, 3 in `mono.rs` `mod tests` covering the mono helper. Real workspaces continue producing single-class residuals (`candidates: None`); every pre-mq.2 fixture typechecks unchanged. Plan-invented `format_type_for_display` replaced with `ailang_core::pretty::type_to_string` (one less duplicate). Synth-time `declared_constraints: &[]` is a deliberate gap documented as known debt — load-bearing only post-mq.3 for the rigid-var fallback (env-plumbing the active fn's constraints into the Var arm is a ~10-line edit slated for mq.3). 9/9 tasks, 539 tests green (was 520 at start of mq.2; +19 = 15 new + 4 pre-existing); `bench/compile_check.py` + `cross_lang.py` clean; `bench/check.py` 1 regression (latency noise, runtime cannot be touched by typecheck-side iter). `MethodNameCollision` retirement lands in mq.3 → 2026-05-13-iter-mq.2.md -- 2026-05-13 — iter mq.3: `MethodNameCollision` retired + multi-class E2E + DESIGN.md sync — milestone close. Deletes `WorkspaceLoadError::MethodNameCollision` variant + `Origin` enum + per-def collision loop in `build_registry` (workspace.rs 596-681) + Display arm in `main.rs:1201`; the two in-workspace.rs pin tests relocate to `crates/ailang-check/tests/method_collision_pin.rs` with inverted assertions (loads cleanly + `env.method_to_candidate_classes["foo"]` has two qualified-class entries). Resolves both mq.2 known-debt items: (1) `Env.active_declared_constraints: Vec` plumbed pre-synth in `check_fn` so the post-superclass-expansion constraint set reaches the synth Var-arm's `resolve_method_dispatch` constraint-driven filter; (2) `ModuleGlobals.class_methods` + `Env.class_methods` re-keyed from `BTreeMap` to `BTreeMap<(QualifiedClass, MethodName), ClassMethodEntry>` with new `class_method_candidates(name) -> Vec<(&class, &entry)>` accessor; mono's two presence-check sites (`rewrite_mono_calls`, `interleave_slots`) switched to consult `method_to_candidate_classes` natively (method-keyed, preserves the presence-check signature shape). New `synth(...)` warnings channel via `warnings: &mut Vec` out-parameter threaded through 15+ recursive callsites + 5 external callers (check_fn, check_const, 3 in builtins.rs, 1 in lift.rs, 2 in mono.rs, 1 in `check`); new structured warning `class-method-shadowed-by-fn` (kebab-case code, structured ctx carrying `name`/`method`/`fn_owner_module`/`candidate_classes`) fires at all three fn-precedence branches (locals, same-module fn, implicit-import fn). Implicit-import-fn branch reordered ABOVE the class-method branch per spec §"Class-fn collisions" so fn-wins precedence is structural. Three new positive E2E fixtures + integration tests in `crates/ail/tests/mq3_multi_class_e2e.rs`: (a) `mq3_two_show_ambiguous` (two `Show` classes, both with `Show Int`, bare `show 42` → `AmbiguousMethodResolution`); (b) `mq3_two_show_qualified` (same workspace, `mq3_two_show_ambiguous_a.Show.show 42` → clean); (c) `mq3_class_eq_vs_fn_eq` (`class MyEq { myeq }` + `fn myeq` + bare `myeq 1 2` → fn wins, warning fires). DESIGN.md sync: class-names paragraph rewritten to point at mq.1 canonical-form + mq.3 dispatch model; `MethodNameCollision` bullet struck from "Workspace-load (registry-build) diagnostics" with forward-pointing note; `AmbiguousMethodResolution` / `UnknownClass` / `class-method-shadowed-by-fn` + `NoInstance.candidate_classes` added to "Typecheck diagnostics"; `AmbiguousInstance` paragraph reworded to distinguish registry-level (per-class coherence via `DuplicateInstance`) from call-site (cross-class method ambiguity, new diagnostic); new `### Method dispatch` subsection anchors the 5-step rule with `method_to_candidate_classes` as the load-bearing data structure, the class-fn precedence rule, and the post-mq.3 tuple-keyed `class_methods` shape. Roadmap P2 milestone → `[x]` with three-iter summary; milestone-24 `depends on:` line struck and entry annotated "ready for re-brainstorm". Plan defects fixed inline: `check_fn` signature is `Result<()>` not `Result<(CheckedFn, Vec)>` — adopted mut-ref-accumulator pattern matching existing `Vec` shape; `class_method_candidates` returns `Vec<(...)>` not `impl Iterator<...>` (the `use<'a, '_>` opaque-type-capture syntax not yet idiomatic in this crate); instance-method body shape draft was `Term::App` but existing-convention is `Term::Lam` with `paramTypes`/`retType` — three fixtures repaired inline. One existing E2E test (`crates/ail/tests/typeclass_22b3.rs:rewrite_walker_skips_locally_shadowed_class_method`) now correctly fires the new warning (the fixture intentionally shadows a class method); test's `assert!(diags.is_empty())` relaxed to filter the new warning with a naming comment. 9/9 tasks, 545 tests green (was 539; +6 net = 3 new mq.3.x lib + 2 method_collision_pin + 3 mq3_multi_class_e2e − 2 deleted workspace.rs pin tests); `bench/compile_check.py` + `cross_lang.py` exit 0; `bench/check.py` exit 1 with 2 noise-class regressions (3rd-consecutive `bench_list_sum.bump_s` persistence + max_us tail metric, both runtime-uncoupled-to-typecheck-iter); prelude zero-diff. Module-qualified-class-names milestone structurally closed → 2026-05-13-iter-mq.3.md -- 2026-05-13 — audit-mq: milestone close (module-qualified-class-names) — architect drift report surfaces 4 actionable items routing to `mq.tidy` (2× [high]: rigid-var refinement type-unification leg missing in `refine_multi_candidate_residual` for `forall a b. Show a, Show b => ...` shapes; same-module bare-class qualifier `Show.show` unreachable because `qualifier_is_class_shape = q.contains('.')` excludes the no-dot case contradicting mq.1 canonical-form symmetry; 2× [medium]: `class-method-shadowed-by-fn` warning over-fires on locals shadowing prelude method names without class-candidate-for-arg-type check; DESIGN.md Data Model schema fragments don't carry the canonical-form rule for `InstanceDef.class` / `Constraint.class` / `SuperclassRef.class`), plus 1× [medium] acknowledged debt without fix (`synth(...)` 10-mut-ref-parameter growth — consistent with crate's existing accumulator pattern; refactor cost disproportionate to gain), plus 2× [low] roadmap-backlog (no E2E for Trajectories B + D; `collect_mono_targets` rebuilds env without `active_declared_constraints` — currently latent because mono residuals are concrete-type). Bench mixed: `compile_check.py` + `cross_lang.py` exit 0; `check.py` exit 1 across 4 consecutive re-runs with metric identity shifting between runs (3 → 1 → 1 → 0 regressions, different metrics each) — pattern consistent with 5th-consecutive audit noise-class observation since audit-cma; baseline pristine for 5th consecutive audit (the metric-migration-between-runs is itself attribution evidence variance not signal) → 2026-05-13-audit-mq.md -- 2026-05-13 — iter mq.tidy: close 4 actionable drift items from audit-mq — T1 extends `refine_multi_candidate_residual`'s rigid-var filter at `lib.rs:2163-2168` from class-only (`dc.class == c`) to class + type-unification (`dc.class == c && constraint_type_matches(&dc.type_, &residual.type_)`) via the existing `constraint_type_matches` helper at `lib.rs:2273`, so `forall a b. prelude.Show a, userlib.Show b => (a, b) -> String`-shape declared-constraint sets correctly discriminate by which typevar the residual is on (high-1; spec §"Constraint-discharge refinement" 130-138). Plan revision noted at the architecture paragraph: synth-time `resolve_method_dispatch` is invoked with `concrete_arg_type: None` and constructs the residual metavar AFTER the dispatch call, so the rigid-var leg there has no residual type to unify against — class-only filter at synth time is semantically correct, not drift; fix lands only at the discharge-time site. T2 extracts the inline `qualifier_is_class_shape` predicate from synth Var-arm at `lib.rs:2570-2575` as a free `pub(crate) fn qualifier_is_class_shape(&Option) -> bool` adjacent to `parse_method_qualifier`; broadened to accept PascalCase single-segment qualifiers (`"Show.show"`) alongside module-qualified ones (`"prelude.Show.show"`), so same-module bare-class call sites are now reachable — symmetric to mq.1's canonical-form rule at the schema level (high-2). Discriminator: class names start with uppercase per PascalCase convention; bare-fn qualifiers (`"std_list"`, lowercase) correctly reject and fall through to the cross-module-fn arm. T3 introduces `pub(crate) fn any_candidate_class_has_instance(...)` using a BTreeMap range-scan (O(|candidates| * log n)) and gates the `class-method-shadowed-by-fn` warning closure at `lib.rs:2479-2502` on it, so class methods with zero registry instances anywhere no longer fire the warning — conservative Boss-Q2-decision tightening of the spec rule (full rule would require the arg type, unavailable at the Var-arm before App-arm unification) (medium-1). T4 annotates four DESIGN.md Data Model schema fragments (`SuperclassRef` 2139, `InstanceDef.class` 2152, `Type::Con.name` 2253, `Constraint.class` 2273) with trailing-comment canonical-form cross-references — Type::Con annotation points at the actual existing section title `§"Type::Con name scoping"` (ct.1 anchor verified via grep, plan's text guess was off) (medium-2). Two trip-wires fixed inline: (a) `parse_method_qualifier` docstring updated to remove the now-incorrect "class qualifier is always qualified per mq.1" claim that the broadened gate directly contradicts — coherence-required not unrequested; (b) in-crate `mq3_class_method_shadowed_by_fn_warning_fires` test built workspace with `Registry::default()` (no instances) and pre-tidy fired anyway because old gate didn't check instances — post-tidy correctly suppresses, fixture repaired by injecting a `clsmod.Show Int` registry entry directly into `ws.registry.entries` (analogous to mq.3's `typeclass_22b3` trip-wire pattern). 5/5 tasks, 548 tests green (was 545 + 3 new `mq_tidy_*` unit tests); `bench/compile_check.py` exit 0 (24/24 stable after one-run noise blip absorbed under tolerance on re-run), `cross_lang.py` exit 0 (25/25 stable), `check.py` exit 0 both runs with metric-identity-migrating noise on `latency.implicit_at_rc.*` max-tail cluster — 6th-consecutive observation of the audit-mq-named noise envelope, baseline pristine. Plan Step 5 expectation `ail check examples/prelude.ail.json` exit 0 was inaccurate (actual: exit 1 "module name 'prelude' is reserved", identical to pre-edit, prelude is loader-auto-injected and never a workspace entry); 548/548 workspace test pass is the right sanity gate → 2026-05-13-iter-mq.tidy.md -- 2026-05-13 — iter 24.2: class Show + 4 primitive instances (Int/Bool/Str/Float) shipped in prelude; 22b user-Show fixtures migrated to TShow/tshow → 2026-05-13-iter-24.2.md -- 2026-05-13 — iter 24.3: fn print polymorphic free fn + 3 E2E fixtures (positive 4-prim smoke / user-ADT IntBox+instance prelude.Show IntBox / negative print f:Int→Int firing Show-aware NoInstance) + IR-shape pin asserting post-mono print__Int.body preserves explicit let-binder + 3 compiler-path repairs (mono.rs canonical-form normalisation of MonoTarget::FreeFn::type_args 2 sites; codegen/lib.rs cross-module reference fallback for post-mono synthesised bodies 3 sites: resolve_top_level_fn / lower_app cross-module arm / synth_with_extras Var arm; check/lib.rs synth FreeFnCall constraint-residual push) + Show-aware NoInstance diagnostic addendum cross-referencing DESIGN.md §Prelude(built-in)classes + DESIGN.md §Prelude(built-in)classes mq24 paragraph flipped to past tense + §Float semantics Show-Float NaN paragraph + roadmap P1 Post-22 Prelude → [x] + new P2 Retire io/print_int|bool|float at top; 556 tests pass (was 552+4 new); milestone 24 structurally closes → 2026-05-13-iter-24.3.md -- 2026-05-13 — audit-24: milestone close (Show + print rewire) — architect drift report surfaces 5 actionable items routing to `24.tidy` (3× [high]: DESIGN.md §"Monomorphisation" doc-anchor missing for three iter-24.3 strengthenings — MonoTarget::FreeFn::type_args canonical-form normalisation, post-mono synthesised-body cross-module-ref import_map-bypass invariant, FreeFnCall constraint-residual push; codegen import_map-fallback path has no unit-level pin — only E2E catches regression; FreeFnCall constraint-residual push covers only dot-qualified branch, bare-name poly-fn refs uncovered; 2× [medium]: unwrap_or_default in method-name lookup masks class-index drift, normalize_type_for_lookup duplicated across two mono.rs sites with manual-lockstep invariant), plus 1× [medium] deferred to roadmap P3 (negative-test coverage single-shape — only f:Int→Int, broader shapes wait for downstream corpus-migration milestone), plus 1× [low] carry-on (bench/architect_sweeps.sh noise on pre-milestone-24 DESIGN.md lines, not new drift). Bench: cross_lang.py exit 0 (25/25 stable); check.py + compile_check.py exit 1 with metric-identity-migrating noise envelope per audit-cma → audit-ms → audit-eob → audit-ct-tidy → audit-mq → audit-mq.tidy → iter-24.2 → iter-24.3 lineage — 9th consecutive observation; conservative-call convention holds baseline pristine across all 9, the metric-migration-between-runs is itself attribution evidence variance not signal; right ratification path is the queued P3 latency-methodology-histogram rework, not --update-baseline → 2026-05-13-audit-24.md -- 2026-05-13 — iter 24.tidy: close 5 actionable drift items from audit-24 — T1 DESIGN.md new subsection §Cross-module references in synthesised bodies (3 invariants installed iter 24.3: canonical-form MonoTarget::FreeFn::type_args via normalize_type_for_lookup, post-mono synthesised body import_map-bypass + module_user_fns fallback, FreeFnCall constraint-residual push per declared forall-constraint); T2 codegen_import_map_fallback_pin (integration test asserts prelude.print__ body references user_module.show__ AND prelude imports do not contain user_module); T3 polyfn_dot_qualified_branch_pin (asserts bare-name print f fires exactly one no-instance + zero unknown-variable, proving constraint-residual push fires via dot-qualified branch); T4 check/lib.rs:2858 unwrap_or_default → expect(class_methods registry coherence) — surfaces class-index drift on regression; T5 mono.rs apply_subst_and_normalize helper extracted from byte-identical sites at :685-714 + :1284-1303 with Option return — each call site keeps own rigid-var/unit-default policy. Tests 558/558 (was 556+2 pins). Bench cross_lang exit 0 stable, compile_check + check both exit 0 this run (10th consecutive lineage observation unobserved-firing). audit-24 medium-3 negative-test breadth defers to roadmap P3; low-1 sweep noise carry-on → 2026-05-13-iter-24.tidy.md -- 2026-05-13 — iter form-a.0: prelude pilot for Form-A-as-default-authoring milestone — examples/prelude.ail rendered (116 lines / 6386 bytes) via `ail render`; two auto-discovering roundtrip tests (every_ail_fixture_matches_its_json_counterpart + parse_then_print_then_parse_is_idempotent_on_every_ail_fixture) green on the new fixture without test-code changes; prelude.ail.json retained per spec §A2 (singular dual-form iter, deletion lands iter 1 with bulk test-infra refactor); cargo test --workspace green at 558 tests → 2026-05-13-iter-form-a.0.md -- 2026-05-13 — iter form-a.1: Form-A-as-default-authoring milestone close (Boss-decided strategy C, big-bang) — 12 tasks across two implementer dispatches; T1 added 3 new tests (parse-determinism + CLI-pipeline-idempotency + carve_out_inventory#[ignore]) + T2 bulk-rendered 99 missing examples/.ail (corpus 58→157 .ail) + T3-4 migrated 26 fixture-loading test files (Group A load_workspace→ailang_surface::load_workspace + Group B subprocess suffix flip) + T5 relocated 23 #[cfg(test)] mod tests blocks from ailang-core/-codegen production source to integration test crates with ailang-surface dev-dep (hash_pin.rs + workspace_pin.rs + eq_primitives_pin.rs + prose snapshot migration) + T6 bench-driver suffix flips (4 Python + run.sh, both compile_check.py + cross_lang.py exit 0) + T7 re-authored 4 raw-JSON-inspect e2e tests via `ail parse`-derived tempfiles + retired 1 (render_parse_round_trip_canonical subsumed by T1) + T8 deleted 156 non-carve-out .ail.json (inventory 8 carve-outs + 157 .ail post-T8, alphabetical: broken_unbound + prelude + 3× test_22b2_* + 3× test_ct1_*) + 20 forward-pulled stale-.ail.json-ref repairs missed by T1-5 recon + T9 retired 3 obsolete roundtrip tests (print_then_parse_round_trips_every_fixture + every_ail_fixture_matches_its_json_counterpart + cli_render_then_parse_preserves_canonical_bytes_on_every_fixture) + dead helpers (list_json_fixtures ×2, round_trip_one, strip_trailing_newlines) + flipped schema_coverage corpus to .ail (forward-pulled into T8 for green-gate) + T10 §C3 DESIGN.md §Roundtrip Invariant restated with parse-determinism + idempotency + CLI-pipeline-idempotency + carve-out-anchor framing (5 surviving enforcement tests named) + T11 §A4 CLAUDE.md + DESIGN.md doctrine sentences replaced (canonical form is JSON-AST; authoring form is .ail) + T12 WhatsNew milestone-close entry + roadmap [milestone] form-a struck [x] (Closed 2026-05-13 by iter form-a.1). 4 carve-outs added beyond the original 7 §C4(a): only 1 — prelude.ail.json — landed as §C4(b) compile-time-embed (rationale: include_str! at workspace.rs:417 + main.rs:474; resolution queued as separate milestone Prelude embed: Form-A as compile-time source). Spec amendment + grounding-recheck PASS at 9fcda8b. Final tests: 557 green + 3 ignored (561 from T1 − 4 retired = 557); bench compile_check.py + cross_lang.py exit 0; check.py noise envelope continues 11th consecutive audit, baseline pristine. Milestone structurally closed → 2026-05-13-iter-form-a.1.md -- 2026-05-13 — audit-form-a: milestone close (Form-A as the default authoring surface) — architect status drift_found with 4 documentary-only items (2× [medium] spec + plan "seven carve-outs" orphans contradicting the §C4(b) amendment commit 9fcda8b — the implementation is eight-carve-out-correct; 2× [low] hash.rs:57 empty `mod tests {}` placeholder after T5 relocation + round_trip.rs:16 stale "Direction 2" docstring contradicting T10's framing rewrite); architect explicit recommendation carry-on — no form-a.tidy queued, drift deferred to documentary cleanup at next opportunity (architect verbatim: "the implementation is correct and the spec orphans are post-hoc retro-clean-up, not active drift"). Bench: all 3 scripts exit 0, baseline pristine for the 12th consecutive audit. check.py 2 regressions in tail-latency max_us (latency.explicit_at_rc + latency.implicit_at_rc, both >120% over baseline, both paired with p99 improvements on same metric stem) — 12th consecutive observation of the metric-identity-migrating noise envelope since audit-cma; compile_check.py 2 regressions in sub-millisecond check_ms on hello + borrow_own_demo — timing-jitter on the corpus's two smallest fixtures, iter form-a.1 T6 already flagged the same pair as transient; cross_lang.py 25/25 stable, bench_tree_walk.rc_over_c improved 2.74→2.59 still inside noise. What holds (verified): §C3 §Roundtrip Invariant restated with 4 properties + 5 surviving enforcement tests named verbatim; §A4 CLAUDE.md + DESIGN.md doctrine reworded per spec; 8-carve-out inventory matches spec post-amendment (alphabetical: broken_unbound + prelude + 3× test_22b2_* + 3× test_ct1_*); 156 non-carve-out .ail.json deleted; 157 .ail post-iter; cargo test --workspace 557 green + 3 ignored; WhatsNew editorial-clean (user-facing); roadmap milestone struck [x] with close attribution + follow-up [Prelude embed] milestone present with motivation + 3 resolution options. Milestone fully closed → 2026-05-13-audit-form-a.md -- 2026-05-13 — fieldtest-form-a: 4 hand-authored `.ail` fixtures (factorial smoke, Show user-ADT, user-class Describe with two instances, two-module workspace) + spec report. Findings: 1 bug (instance-method-body unbound-var bypasses `ail check`; surfaces at mono with degraded "monomorphise_workspace: unknown identifier" diagnostic — `ail check` should fire `[unbound-var]` like at fn-body level → debug RED-first), 1 friction (`str_concat` primitive missing — first attempt at user-Show body wanted "prefix=" ++ int_to_str-style concatenation, repaired by dropping prefix → planner tidy symmetric to `str_clone`/`int_to_str` wiring), 2 spec_gaps (`form_a.md` has zero `class`/`instance`/`constraint`/`method` sections — the form-a milestone made `.ail` the authoring form but the form_a spec doc still only covers pre-22 surface; canonical-form rule for `(instance (class X))` class-qualifier not in form_a — `class X` vs `class some_module.X` reading is ambiguous from the spec, observed canonical reading via examples corpus). Plus 3 working (4 fixtures, 3 first-shot-clean + 1 first-attempt-bug-then-clean). Boss-side cleanup at commit time: deleted 8 stale `.ail.json` sidecars in `examples/fieldtest/` (4 forma_* derived by fieldtester per old workflow, 4 pre-existing floats_* from prior milestone that T8 missed because the bash deletion loop globbed only `examples/*.ail.json` direct children); plus updated `skills/fieldtest/agents/ailang-fieldtester.md` Phase 3 to remove the now-obsolete "generate canonical JSON" step (the agent was calibrated to dual-form pre-form-a; post-form-a `.ail` is the sole authoring form). Findings deferred to follow-up iters: bug → next `debug` dispatch, friction → small planner tidy, spec_gaps → form_a.md tightening pass → 2026-05-13-fieldtest-form-a.md -- 2026-05-13 — iter bugfix-instance-body-unbound-var: instance method bodies now walk through `check_fn` — `check_def`'s `Def::Class(_) | Def::Instance(_)` arm at `crates/ailang-check/src/lib.rs:1574-1595` was early-returning `Ok(())` for both variants (the comment claimed iter 22b.2 landed instance-body typechecking; the wiring was never implemented), so an unbound identifier inside an instance-method lambda body slipped past `ail check` (false-OK exit 0 with `ok (N symbols across M modules)`) and surfaced only at `ail build` as the degraded internal-error diagnostic `monomorphise_workspace: unknown identifier: ` without source location, symbol kind, or "did you mean" candidates. Fix: split the combined arm so `Def::Class(_)` stays schema-only at this layer while `Def::Instance(inst)` routes through a new helper `check_instance` that, for each `InstanceMethod`, lifts the body `Term::Lam` into a synthetic `FnDef` and applies class-method substitution (class param → instance type via the existing `substitute_rigids` / `substitute_rigids_in_term` helpers — looked up in `env.class_methods[(qualify_class_ref_in_check(&inst.class, &env.current_module), im.name)]` for the class's `param` name) before handing to `check_fn`. The reuse of `check_fn` is what gets the body walked through the same identifier-resolution path as fn bodies, so an unbound name fires `[unbound-var]` at `ail check` with exit 1 and the standard structured diagnostic. RED: 2 tests in `crates/ail/tests/unbound_in_instance_method_pin.rs` (text-mode + `--json` shape, both committed in 72f3f65 ahead of the GREEN side per the audit-trail flow). GREEN: 557+2 = 559 green; both pins PASS; fn-body-level unbound-var path stays green; all 8 carve-out fixtures classify unchanged. Known debt (out of scope per "minimal fix" constraint, recorded in journal): `check_instance` does not yet cross-check the substituted method signature against the class's `ClassMethodEntry.method_ty` — a malformed instance declaring wrong types in its Lam would still typecheck cleanly. Closes the bug finding from fieldtest-form-a → 2026-05-13-iter-bugfix-instance-body-unbound-var.md -- 2026-05-13 — iter form-a.tidy: post-fieldtest documentary tidy — 6 tasks; closes 2/3 fieldtest-form-a spec_gaps + 3/4 audit-form-a drift items. T1-T3 add three new sections to `crates/ailang-core/specs/form_a.md`: `### Class — (class ...)` (EBNF with optional `(superclass …)?` schema-matched cardinality + abstract-required vs default-bearing method clauses, anchored to `examples/test_22c_user_class_e2e.ail` ok 24/2), `### Instance — (instance ...)` (EBNF with canonical-form CLASS-REF rule explicitly stated + same-module bare example abbreviated from `mq3_class_eq_vs_fn_eq_classmod.ail` + cross-module qualified example from `show_user_adt.ail` + `bare-cross-module-class-ref` diagnostic anchor), and `(forall ...)` extension with optional `(constraints (constraint CLASS-REF TYPE)+)?` clause + `no-instance` diagnostic anchor + fifth Examples-block entry anchored to `cmp_max_smoke.ail` ok 22/2. §Definitions intro flipped `Three kinds` → `Five kinds`. T4 fixes 6 contradictory "seven carve-outs" sites in `docs/specs/2026-05-13-form-a-default-authoring.md` (preamble + §C1 + §C2 + §C3 + §"Data flow" two sites) to match the §C4(b) amendment commit 9fcda8b; post-edit grep returns 4 correctly-scoped surviving "seven" mentions (lines 233, 238, 463, 469 — all §C4(a)-scope or arithmetic/future-state). T5 deletes empty `#[cfg(test)] mod tests {}` placeholder + 6-line relocation-comment block from `crates/ailang-core/src/hash.rs:50-57` (the unit tests live in `crates/ailang-core/tests/hash_pin.rs` since form-a.1 T5 relocation; placeholder served no purpose). T6 rewrites `crates/ailang-surface/tests/round_trip.rs` module-level `//!` (lines 1-26) and inner `///` (lines 63-72) docstrings to the post-T10 four-property framing (parse-determinism + idempotency-under-print + CLI-pipeline-idempotency + carve-out-anchor) instead of retired Direction-1/Direction-2 framing, with sibling-crate breadcrumbs pointing at the other two enforcement points (`crates/ail/tests/roundtrip_cli.rs::cli_parse_then_render_then_parse_is_idempotent` for CLI-pipeline-idempotency, `crates/ailang-core/tests/carve_out_inventory.rs::examples_ail_json_inventory_matches_carve_outs` for carve-out-anchor). Drift item B2 ("plan-file seven-orphans, two sites" per audit-form-a) dropped from the iter: recon verified `docs/plans/2026-05-13-iter-form-a.1.md` contains zero defective `seven` mentions; all four hits are internally scoped to §C4(a), arithmetic, or future-state — the audit-form-a journal's "two sites" claim against the plan file did not match its contents at HEAD; decision recorded in journal §"Decision recorded — Drift item B2". Tests 559 green at every per-task gate (no test-file changes; T5 deletes a no-op `mod tests {}`). Pure-documentary iter, Phase 3 deliberately skipped per carrier; verification was per-task `cargo test --workspace` + `ail check` on the four corpus fixtures cited in T1-T3. Remaining open from form-a fieldtest queue: friction (no `str_concat` primitive — symmetric tidy iter queued) → 2026-05-13-iter-form-a.tidy.md -- 2026-05-13 — iter str-concat: `str_concat : (borrow Str, borrow Str) -> Str` heap-Str concatenation primitive shipped in four-site lockstep, closing fieldtest-form-a friction finding #4. T1 RED-pin `crates/ail/tests/str_concat_e2e.rs` + new corpus fixture `examples/show_user_adt_with_label.ail` (Show user-ADT body using `(app str_concat "Item " (app int_to_str n))` through prelude `print`; ok 23/2). T2 `runtime/str.c` C helper `ailang_str_concat(a, b)` appended after `ailang_str_clone`: reads `len` headers from both source payloads, `str_alloc(la+lb)`, memcpys both bytes, NUL-terminates. T3 `crates/ailang-check/src/builtins.rs` install entry (Type::Fn arity-2 Borrow-Borrow params, Own return, effects empty) + list() row + `install_str_concat_signature` unit test reaching into `env.globals` directly (deeper than synth_in_builtins_env-based siblings because first builtin with arity-2 Borrow params). T4 `crates/ailang-codegen/src/lib.rs` four-site lockstep: extern `declare ptr @ailang_str_concat(ptr, ptr)` after str_clone declare + lower_app arm after str_clone arm (arity-2 check + two `lower_term` + `fresh_ssa` + body push) + `is_builtin_callable` match-list extended with `"str_concat"` + IR-pin unit test `str_concat_emits_call_to_ailang_str_concat` asserting both extern declaration AND call instruction in emitted IR. Five IR snapshots regenerated (`hello.ll`, `list.ll`, `max3.ll`, `sum.ll`, `ws_main.ll`) to absorb the new unconditional declare line in IR header — same upkeep pattern as hs.4 (sole diff verified at IR line 17 across all snapshots). T5 lockstep-collision repair: `examples/bug_unbound_in_instance_method.ail` had used `str_concat` as the literal UNBOUND name (because that was the fieldtester's LLM-natural repro); renamed to `format_label` (LLM-author-realistic helper name that will never become a builtin) and updated `crates/ail/tests/unbound_in_instance_method_pin.rs` (`replace_all` on `str_concat` → `format_label`, including one test name flipped to `check_fires_unbound_var_for_format_label_in_instance_method_body`), preserving the regression guard's intent (instance-method-body walked through unbound-var check) while substituting a name with stable unbound semantics. T6 `docs/DESIGN.md` new §"Heap-Str primitives" subsection (line 1994) between the milestone-24 Show-backer enumeration and the existing `Primitive output goes through ...` paragraph, cataloguing all five heap-Str primitives (`int_to_str`, `bool_to_str`, `float_to_str`, `str_clone`, `str_concat`) with type signatures, iter origins, and the user-visible-vs-prelude-internal distinction; Show-backer block unchanged. T7 `docs/roadmap.md` struck `[x] [feature] str_concat` line at end of P2 cluster. Two minor plan-vs-actual discrepancies recorded in journal Concerns: (a) T3 Step 5 test-count prediction `passed: 560 failed: 1` was actual `passed: 558 failed: 3` because the unbound_in_instance_method_pin tests turn RED at checker registration not at codegen (assertion is on the `[unbound-var]` diagnostic which depends only on the checker registry) — implementation correct, T5 fully repairs; (b) IR snapshot regen for T4 was not explicitly scripted in the plan but follows hs.4 precedent. Final iter state: 562/562 green (559 baseline + 3 new pins), zero re-loops across all 7 tasks. Bench impact none (no new code paths touched by bench corpus; bench fixtures use `int_to_str` / `bool_to_str` only). Closes fieldtest-form-a queue: bug (closed bugfix-instance-body-unbound-var), spec_gaps 2+3 (closed form-a.tidy), friction (closed here). Remaining open: spec_gap 1 dropped per recon (audit-form-a "plan two sites" claim did not match HEAD) → 2026-05-13-iter-str-concat.md -- 2026-05-13 — iter rustdoc-sweep: cleared all 23 `cargo doc --workspace --no-deps` warnings (17 in `ailang-check` + 4 in `ailang-core` + 2 in `ailang-surface`). Two warning classes: (a) `pub` doc-comments that linked-via-brackets to `pub(crate)`/private items (15 hits) — replaced ``[`fn`]`` with plain backtick-code-span ``` `fn` ``` so the identifier stays readable but rustdoc no longer tries to resolve the link; (b) unresolved/ambiguous links (8 hits) — `mono.rs` `[`Registry`]`/`[`Registry::entries`]` ×3 fully-qualified to `[`ailang_core::workspace::Registry…`]`; `loader.rs` `[`crate::parse`]` ×2 (ambiguous between fn + mod) disambiguated to `[`crate::parse()`]`; `lib.rs` `metas[i]` (parsed as a link) escaped to `metas\[i\]`. Tests 562 → 562, no behaviour change → 2026-05-13-iter-rustdoc-sweep.md -- 2026-05-13 — iter drift-test-narrowing: `crates/ailang-core/tests/design_schema_drift.rs` now scans §"Data model" only, not the whole DESIGN.md. New helper `data_model_section()` slices `DESIGN_MD` from `## Data model` to the next top-level `## ` header; all 7 anchor-presence tests switched from `DESIGN_MD.contains(anchor)` to `data_model_section().contains(anchor)`. Closes the audit-form-a-precursor `[high]` "anchors-elsewhere-pass-silently" failure mode (an anchor present only in §"Decision 11" or another discussion section was previously treated as documented). All 38 anchors verified pre-edit to live in §"Data model" so the tightening doesn't regress to red. New pin `data_model_section_is_bounded` (starts-with `## Data model`, no bleed past `\n## Pipeline`, > 1 KB) guards the extractor itself against silent regression to whole-document scanning. Tests 562 → 563 (+1) → 2026-05-13-iter-drift-test-narrowing.md -- 2026-05-14 — iter clippy-sweep: cleared all 61 `cargo clippy --workspace --all-targets` warnings across 12 lint classes. Bulk: 32× `doc_lazy_continuation` + 4× `empty_line_after_doc_comments` (8 orphan `///` blocks in `workspace.rs` left by form-a.1 T5 relocation converted to plain `//`) — pure documentation hygiene. Idiomatic refactors: 5× `err_expect` (`.err().expect()` → `.expect_err()`), 3× `useless_conversion`, 2× `redundant_closure`, 2× `collapsible_match`, 1× `single_char_add_str`, 1× `trim_split_whitespace`. Two `derivable_impls` (manual `Default` for `ParamMode` + `AllocStrategy` flipped to `#[derive(Default)]` + `#[default]`). One `blocks_in_conditions` extracted to a new helper `is_class_method_dispatch(name, env)` next to `qualifier_is_class_shape` in `check/src/lib.rs`. Three `#[allow]`s with inline rationale: `only_used_in_recursion` on `walk_pattern` (preserves the 5-fn walker-framework signature uniformity), 2× `if_same_then_else` on `qualify_local_types` shapes (`check/src/lib.rs:3457` + `codegen/src/subst.rs:165` — the two `name.clone()` branches encode semantically distinct disqualification reasons), `too_many_arguments` on `Emitter::new` (8 workspace-flat tables; bundling would just rename boilerplate). Tests 563 → 563 (no behavior change); `cargo doc` stays at 0 warnings (rustdoc-sweep baseline holds); all 3 bench scripts exit 0 against existing baselines (free stability check that the sweep touched no semantics) → 2026-05-14-iter-clippy-sweep.md -- 2026-05-14 — iter bugfix-mono-cursor-print-with-class-method-arg: mono cursor advance by 1+N at poly-free-fn Var with class-constrained Forall → 2026-05-14-iter-bugfix-mono-cursor-print-with-class-method-arg.md -- 2026-05-14 — iter bugfix-print-leak-show-ret-mode: prelude Show.show ret_mode + substitute_rigids preserves param_modes/ret_mode → 2026-05-14-iter-bugfix-print-leak-show-ret-mode.md -- 2026-05-14 — iter rpe.1: retire per-type io/print_int|bool|float; corpus migrated to polymorphic print → 2026-05-14-iter-rpe.1.md -- 2026-05-14 — iter rpe.1.tidy: subst.rs preserves Type::Fn modes through rebuild → 2026-05-14-iter-rpe.1.tidy.md -- 2026-05-14 — iter cli-diag-human: WorkspaceLoadError thru diagnostic in non-JSON CLI path → 2026-05-14-iter-cli-diag-human.md -- 2026-05-14 — iter pd.1: core API split — load_modules_with + build_workspace + implicit_imports threading; load_workspace_with kept as 3-line shim so surface unchanged; production literal-"prelude" count in workspace.rs dropped 8→4 → 2026-05-14-iter-pd.1.md -- 2026-05-14 — iter pd.2: surface owns prelude embed (PRELUDE_AIL + parse_prelude in ailang-surface); pd.1 shim load_workspace_with retired along with PRELUDE_JSON / load_prelude / load_one; cross-form-identity preflight PASSED at module_hash 3abe0d3fa3c11c99; production literal-"prelude" in core/src/ now zero → 2026-05-14-iter-pd.2.md -- 2026-05-14 — iter pd.3: prelude.ail.json deleted from working tree; cross-form-identity preflight + supporting bytes deleted from prelude_module_hash_pin (purpose discharged); migrate-bare-cross-module-refs defensive include + lockstep skip-branch deleted; carve_out_inventory.rs §C4(b) row dropped (8→7); form-a-default-authoring spec §C4(b) RETIRED status marker added; new prelude_decouple_carve_out_pin.rs asserts JSON file does not exist; milestone prelude-decouple closed → 2026-05-14-iter-pd.3.md -- 2026-05-14 — audit-pd: milestone close (prelude-decouple) — architect drift fixed inline as audit-pd-tidy (DESIGN.md §"Roundtrip Invariant" carve-out count 8→7 + main.rs migrate-subcommand dead prelude fallback removed); bench mixed (check.py + compile_check.py established noise envelope, 15th consecutive observation, baseline pristine; cross_lang clean) → 2026-05-14-audit-pd.md -- 2026-05-15 — iter mut.1: AST extension `Term::Mut` + `Term::Assign` + `MutVar` struct; Form A `(mut ...) / (var ...) / (assign ...)` productions parse + print + round-trip; ~25 substantive Term-walker arms across the codebase; dispatch stubs in `synth` (typecheck) + `lower_term` (codegen) return `CheckError::Internal` / `CodegenError::Internal` per the spec's out-of-iteration boundary; `examples/mut.ail` six-fn fixture (Int/Float/Bool/Unit + empty + nested-shadow); drift + coverage + spec-drift tests + DESIGN.md §"Term (expression)" + `crates/ailang-core/specs/form_a.md` amendments; tests 564 → 579 → 2026-05-15-iter-mut.1.md -- 2026-05-15 — iter mut.2: typecheck for `Term::Mut` + `Term::Assign` replacing the iter mut.1 dispatch stubs; three new `CheckError` variants (`MutAssignOutOfScope`, `AssignTypeMismatch`, `UnsupportedMutVarType`) with bracketed-`[code]:` Display + `code()` + `ctx()` arms; `synth` signature threads `mut_scope_stack: &mut Vec>` after `locals`, propagated through all recursive call sites in `lib.rs` plus `mono.rs:712/1337` + `lift.rs:723` + `builtins.rs` test helpers + the mq.3 in-test helper at `lib.rs:6663`; `Term::Var` resolution prepended with innermost-first mut-scope lookup; `Term::Mut` arm gates var types to {Int,Float,Bool,Unit} and push/pops a fresh frame; `Term::Assign` arm walks the stack innermost-first, emits `AssignTypeMismatch` on type mismatch and `MutAssignOutOfScope` (with `available` flattened across all frames) on miss; five `.ail.json` fixtures under `examples/` (four negative + one positive nested-shadow) + driver `crates/ailang-check/tests/mut_typecheck_pin.rs`; `carve_out_inventory.rs` EXPECTED extended 7→12 for the new negative-fixture set; `examples/mut.ail` typechecks clean (`ok (26 symbols across 2 modules)`); tests 579 → 592 → 2026-05-15-iter-mut.2.md -- 2026-05-15 — iter mut.3: codegen + e2e for `Term::Mut` + `Term::Assign` closing the mut-local milestone end-to-end; mut-var allocas hoisted to fn entry block via a per-`Emitter` side buffer `pending_entry_allocas: String` + a captured `entry_block_end_marker: Option` byte position spliced via `String::insert_str` at the end of `emit_fn`; `Term::Mut` arm in `lower_term` emits one alloca per var into the side buffer, lowers each init at the current position, emits a `store` to bind, push/pops a per-block save stack for proper shadowing of outer mut-vars; `Term::Assign` arm emits `store , ptr ` and yields the canonical Unit SSA; `Term::Var` arm prepends mut-var lookup with a `load` emission. Two beyond-plan adjustments: (1) `synth_with_extras`'s parallel `Term::Var` arm needed the same mut-var lookup precedence as `lower_term`, (2) `lambda.rs` thunk emission needed save/restore of all three new `Emitter` fields across the lambda-body boundary plus an in-thunk splice at the lambda's own entry marker so mut-blocks inside a closure body hoist to the closure's entry not the outer fn's. Two e2e fixtures `examples/mut_counter.ail` (Int) + `examples/mut_sum_floats.ail` (Float); both run end-to-end and print `55`. DESIGN.md "What **is** supported" subsection gains a "Local mutable state" bullet. mut-local milestone end-to-end closed. Tests 592 → 594 → 2026-05-15-iter-mut.3.md -- 2026-05-15 — iter mut.4-tidy: close mut-local audit drift. Architect's two `[high]` items addressed — new `CheckError::MutVarCapturedByLambda` rejects any lambda whose body's free vars hit the enclosing `mut_scope_stack` (via the existing `desugar::free_vars_in_term` helper, gated on non-empty stack), and `codegen/lambda.rs`'s blame-typechecker `Internal` path becomes `unreachable!` once typecheck is the gate. Two `[medium]` stale-history comments cleaned in DESIGN.md §"Term (expression)" and `ailang-codegen/src/lib.rs`. New negative fixture `examples/test_mut_var_captured_by_lambda.ail.json` + driver test extension; `carve_out_inventory.rs` EXPECTED 12→13. Spec §"Out of scope" amended with the lambda-capture rejection bullet. Bench: `compile_check.py` `check_ms` showed a uniform ~30-50% regression across the suite traced to a fixed-cost startup tax (mut-* added ~1400 lines of typecheck/codegen surface; the short-circuit on empty `mut_scope_stack` walk in Term::Var did not move the needle, falsifying the hot-path hypothesis); ratified as a feature tax with paired baseline update on `bench/baseline_compile.json`. `check.py` tail-noise envelope continues the audit-pd carve-out (no ratify needed). `cross_lang.py` clean. Tests 594 → 598. mut-local milestone audit closed → 2026-05-15-iter-mut.4-tidy.md -- 2026-05-15 — bugfix mut-diag-double-code: fieldtest F2 — the four mut-local `CheckError` variants embedded `[]` in their `#[error]` Display body while the non-JSON CLI formatter also prepends `[code]`, doubling it in human stderr; dropped the embedded prefix from the four strings, bringing them in line with all non-mut variants; RED-first via `debug` (`ct1_check_cli.rs::check_human_mode_renders_mut_diagnostic_code_exactly_once`), GREEN applied inline as a trivial mechanical edit; tests 598 → 599 → 2026-05-15-bugfix-mut-diag-double-code.md -- 2026-05-15 — iter it.1: iteration-discipline milestone (1 of 3) — `Term::Loop` / `Term::Recur` / `LoopBinder` added as strictly-additive first-class AST nodes end-to-end: Form-A `parse_loop`/`parse_recur` + print + prose render/free-var/subst lockstep + canonical-JSON serde/round-trip + schema/spec-drift/schema-coverage/carve-out lockstep (13→17) + typecheck (binder typing + recur arity/type unification via `loop_stack: &mut Vec>` threaded exactly as mut.2's `mut_scope_stack`; recur-tail-position via a new private `verify_loop_body`, `verify_tail_positions` public signature unchanged) + codegen (loop-header block, one phi per binder, recur back-edge `br` with a NEW parallel `block_terminated` setter, lambda-boundary `loop_frames` save/restore mirroring mut.3). Four new `CheckError` variants `Recur{OutsideLoop,ArityMismatch,TypeMismatch,NotInTailPosition}` (bracket-`[code]`-free Display per the F2 convention). Strictly additive verified — zero deletions touch `tail-app`/`tail-do`, `verify_tail_positions`' tail-app role, or the seven existing codegen `block_terminated` sites (the 32 within-iter deletions are exclusively DD-3 stub-then-fill replacements). `Term::Recur` synth returns a fresh metavar (resolves the plan's flagged `Type::unit()` open risk: recur appears in `if` branches that must unify with the sibling type). `loop_counter.ail`→`55`, `loop_in_lambda_e2e.ail`→`49`, four negatives fire exact codes, `tail-app` fixtures byte-identical, zero IR/prose snapshot drift; `cargo test --workspace` green. Two mut.1-class plan-pseudo-vs-reality substitutions recorded (prose round-trip asserting AST-equality is impossible — no Form-B parser by design; the diagnostic.rs doc-list premise was false) → 2026-05-15-iter-it.1.md -- 2026-05-15 — iter it.2: iteration-discipline milestone (2 of 3) — structural-recursion guardedness checker + first real `Diverge` effect, strictly additive (nothing `tail`-related removed; that is it.3). New whole-body pass `verify_structural_recursion` runs as a sibling of `verify_tail_positions` in `check_fn`'s post-synth region (DD-1): the `smaller`-set algorithm with implicit candidate-position inference + unconstrained accumulator positions (DD-2 — foldl-shape accumulator classifies as structural recursion, pure+total), self/mutual identification with an inline ADT-family connected-components union-find (DD-3), and the it.2-only `tail==false` grandfather. `CheckError::NonStructuralRecursion {callee,arg}` (bracket-`[code]`-free Display per F2). `term_contains_loop` (stops at `Term::Lam` boundaries, DD-4) injects `"Diverge"` into the raised effect set so the existing `UndeclaredEffect` machinery enforces it with no new diagnostic variant; lam-arrow + `Term::LetRec` sub-effect sites wired (loop behind a lam edge carries `!Diverge` on the lam's arrow, propagating via the free callee-effect path, not leaking to the enclosing fn — exactly as `!IO` scopes). DESIGN.md Decision 3 + §Data-model hook synced present-tense. Four it.1 loop fixtures gained `!Diverge`. Two spec-premise boundary defects surfaced and resolved inside the task's invariants (corpus clean, check not weakened): (§1) the spec's "21 tail-app fixtures" grandfather premise under-counts the corpus — no-ADT-candidate counter recursions (~18, e.g. `build_tree(depth:Int)`) have no structural position to verify and are deferred to it.3 migration; (§2) two RC-regression fixtures (`rc_pin_recurse_implicit`, `rc_let_alias_implicit_param`) hold an ADT param constant while decrementing an Int — genuinely non-structural, joined the spec's transitional `tail-app` grandfather exactly as the other 20 corpus fixtures do (their RC==GC regression guards verified still green; the regression lives in the unchanged `pin`/`pin_aliased` bodies). Both recorded as corrected it.3 corpus-migration scope. `cargo test --workspace` 622 green / 0 red; all 9 acceptance pins non-vacuous (negatives `.contains(code)`); struct_rec_sum→15, loop_needs_diverge→55, the it.1 55/49 e2e still green → 2026-05-15-iter-it.2.md -- 2026-05-15 — iter it.3 (BLOCKED, bounce-back): destructive terminal iteration of iteration-discipline. Task 1 (non-destructive: pre-migration oracle + the spec-delegated class-(b) live sweep) ran complete — 40 recursive corpus fixtures classified + oracled under `bench/it3-oracle/`; zero production code changed; HEAD clean at c992eb9. The mandated Task-1.3 sweep surfaced a **fundamental milestone-design flaw**, not a migration nuisance: 6 fixtures contain `build(d: Int) = if d==0 then Leaf else Node(1, build(d-1), build(d-1))` — a terminating, maximally-LLM-natural, **non-structural (Int param, no ADT-candidate) non-tail BRANCHING (double) recursion** that the milestone's totality dichotomy ("structural-over-ADT OR loop/recur, else error") makes **inexpressible**: not ADT-structural, and not `recur`-able (branching ≠ single tail back-edge). Post-it.3 it is a hard `NonStructuralRecursion` with no expressible alternative — the milestone as-specified would fail its own feature-acceptance criterion (clause 1: the LLM reaches for `build(d-1),build(d-1)`; clause 2: the milestone would *remove* expressivity). The orchestrator correctly refused a 4th plan-patch and bounced (`feedback_spec_over_plan_patches`). Boss verdict: the totality story overlooked a second canonical total-by-construction scheme — well-founded recursion on a strictly-decreasing non-negative Int measure. Resolution requires a new additive iteration **it.2b** (widen the it.2 guardedness checker to accept Int-bounded recursion incl. branching) BEFORE it.3 re-dispatches; the purity sub-fork (Int≠Nat: total only for non-negative entry) touches the user-co-designed purity pillar → user design decision, recommended option A1 (accept with a documented non-negative-entry obligation, analogous to array-bounds). Notify sent; loop stopped. The RC-RSS/18g.1 load-bearing risk (it.3 Task 5.5/5b) was NOT reached and must NOT be treated as resolved by the milestone-close audit → 2026-05-15-iter-it.3.md -- 2026-05-16 — iter revert: Iteration-discipline milestone (it.1 `96db54d` + it.2 `a4be1e5`) fully backed out by one **forward** iteration (`main` sacrosanct — never rewound; `1ff7e81`, the pre-`9973546` commit, is the per-region byte oracle). Root cause: the milestone was an over-escalation of fieldtest finding F1 (a `[friction]` item whose own minimal recommendation was a DESIGN.md note); its totality dichotomy made the maximally-LLM-natural `build(d:Int)=Node(1,build(d-1),build(d-1))` inexpressible (it.3 BLOCKED), and the only in-thesis escape (A1/it.2b) conceded the language's first documented-unenforced totality precondition — a purity-pillar dilution the user rejected. `Term::Loop`/`Term::Recur`/`LoopBinder`, the `verify_structural_recursion` guardedness pass + `term_contains_loop` + `Diverge`-injection + the transitively-it.2 `module_fns` plumbing, the five `Recur*`/`NonStructuralRecursion` `CheckError` variants (+ `code()` + 3 dedicated `ctx()` arms), the it.1 codegen loop-header/phi/back-edge + parallel `block_terminated` setter, and all walker arms are removed — `crates/ailang-check/src/lib.rs` and every reverted source/test file byte-identical to `1ff7e81`. Surgical keeps: feature-acceptance **clause 3** (DESIGN.md + brainstorm SKILL.md, worked example de-claimed "shipped"→hypothetical) and the F3 P2 todo. Sole net addition: an honest F1/F4 documented-idiom note (tail-recursive accumulator fallback; `examples/mut_counter.ail`) guarded by a doc-presence test. 16 it.1/it.2 fixtures + 2 pin files + `bench/it3-oracle/` deleted; 2 RC fixtures restored to `1ff7e81`; `bench/orchestrator-stats/2026-05-15-iter-it.{1,2,3}.json` kept (historical record). Roadmap: Iteration-discipline block + blocking-fork section removed; the genuine total-Int-recursion ambition preserved as a deferred P2 milestone sequenced behind a future `Nat`/refinement-types milestone (not abandoned — correctly sequenced). Correctness gate PRISTINE: 164 surviving `1ff7e81`-era fixtures `ail check`/`ail run` byte-identical to pre-milestone behaviour (1ff7e81 worktree reference compiler, zero drift); `cargo test --workspace` 600/0; zero residual it.1/it.2 production surface. The old it.* journals/plans + the superseded-headered `2026-05-15-iteration-discipline.md` stay as historical record → 2026-05-16-iter-revert.md -- 2026-05-16 — audit iteration-discipline-revert (milestone close, clean): architect confirmed the revert byte-pristine (18 src + 5 test files byte-identical to `1ff7e81`, zero residual it.1/it.2 production surface, DESIGN.md coherent, roadmap/spec hygiene correct); one `[medium]` drift — a dangling `loop`/`recur` cross-reference into the reverted milestone inside the *live* Stateful-islands roadmap scope bullet — fixed inline (Boss doc-hygiene; reworded to keep the real "no `while`; repetition stays recursion" scope guard, drop the reverted-milestone reference). Bench: `compile_check.py`/`cross_lang.py` exit 0; `check.py` exit 1 on the single metric `bench_list_sum.bump_s` +12.71%. Bencher localisation: HEAD and `1ff7e81` bump binaries `cmp`-identical, interleaved 3×60-run measurement shows head−oracle delta ~0 (±1.5% stdev) with *both* ~+11% over the 2026-05-09 baseline → environmental/hardware drift, NOT a revert regression (third independent proof codegen == `1ff7e81`). Resolution: carry-on, baseline NOT ratified (Iron Law — nothing intentionally moved the metric; byte-identical codegen must not move the baseline; the spec's PRISTINE mandate is satisfied because pristine = "no regression introduced", proven). Pre-existing `*.bump_s` baseline-vs-hardware staleness (the `1ff7e81` oracle trips the same +11%) filed forward as a separate P2 `[todo]` (bench-harness recalibration, no language change), not mis-attributed to the revert. Fieldtest skipped (revert restores the already-field-tested pre-`9973546` surface; Task-11 164-fixture byte-equivalence is the stronger proof). Milestone closed clean → 2026-05-16-audit-iteration-discipline-revert.md -- 2026-05-16 — iter effect-doc-honesty: standalone documentation-honesty tidy (split out from the retired effect-op-arg-modes bundle — user rejected bundling a real DESIGN.md fix with speculative build-ahead infra). Corrected three false effect-system claims the recon surfaced: DESIGN.md Decision 3 "row-polymorphic (`![IO | r]`)" (no `EffectRow`/row var exists — effect sets are flat closed sets unified by set-equality) + "`IO` and `Diverge` … are wired up" (`Diverge` is 100% vapour: zero code in any crate) → reconciled to IO-only/Diverge-reserved-unimplemented (modelled on Decision 4's reserved-refinements precedent); DESIGN.md §"What is not supported" "IO and Diverge ops" bullet; `ast.rs` `Term::Do` doc-comment "resolved against the effect-handler table at link time" → the real mechanism (typecheck `Env::effect_ops` lookup + `lower_effect_op` literal codegen match, no handler table). Satellite lockstep: `form_a.md:226` + rule-3, `main.rs:289` merge-prose CONTRACT example. Guarded by a new 4-test doc-presence pin `crates/ailang-core/tests/effect_doc_honesty_pin.rs` (fiction-absent + corrected-anchor-present, single-line substrings — wrap-robust). No language/checker/codegen change; `ail check`/`run` byte-unchanged by construction. `cargo test --workspace` 600 → 604; drift/coverage trio (`design_schema_drift`/`spec_drift`/`schema_coverage`) green, empirically confirming none scans the effect-prose region. One concern: the plan's Task-2 replacement body soft-wrapped a phrase the single-line pin asserts; orchestrator resolved correctly (exact wording + exact pin preserved, wrap column moved) — recurring-class planner gap, fixed forward by a Step-5 self-review tightening → 2026-05-16-iter-effect-doc-honesty.md -- 2026-05-17 — iter loop-recur.1: standalone-`loop`/`recur` milestone (1 of 3) — the strictly-additive AST-node foundation. `Term::Loop { binders: Vec, body }` / `Term::Recur { args }` / `struct LoopBinder` (mirrors `MutVar`'s `(name,type,init)` triple; `binders` no `skip_serializing_if`) added end-to-end across all six crates' no-`_`-wildcard exhaustive `Term` matches (~30 sites incl. one recon-undercount integration-test pin): Form-A `parse_loop`/`parse_recur` (binder/body boundary disambiguated by an `is_term_head_kw` guard — the plan's flagged sole parser judgement) + print + `form_a.md` grammar/notes + prose render/free-var/subst lockstep + canonical-JSON serde/round-trip + DESIGN.md §Data-model `"t":"loop"`/`"t":"recur"` blocks + design_schema_drift/spec_drift/schema_coverage anchors + a new `loop_recur` hash pin (additivity proven: iter13a + new pin both green, pre-existing canonical-JSON bytes byte-stable) + `examples/loop_sum_to.ail` round-trip fixture. NO typecheck semantics + NO real codegen by design: the two semantic dispatch points stubbed `synth`→`CheckError::Internal` / `lower_term`→`CodegenError::Internal` exactly as mut.1; pure analysis walkers (escape/lambda/`synth_with_extras`) get real structural pass-through. Two binding Boss design calls (mirrored into the journal): (1) `verify_tail_positions` "byte-unchanged" = tail-app *role* unchanged not source-frozen — it is an exhaustive no-wildcard match so two additive descent-only arms are mandatory; acceptance evidence is the tail-app non-regression test (all `tail`-filtered tests byte-identical), mut.1 set the precedent; (2) codegen IS iter-1 scope as stubs/pass-throughs (no real loop-header/phi/back-edge — that is iter 3). Three plan-pseudo-vs-reality substitutions, all intent-preserving, recurring `feedback_specs_need_concrete_code`/`feedback_plan_pseudo_vs_reality` class: serde `Type::Con` literal `{"t":"con",…,"args":[]}`→real `{"k":"con","name":"Int"}`; bare `Int`→`(con Int)` in binder triples (bare `Int` parses as `Type::Var`); `ailang_core::ast::LoopBinder`→unqualified inside ailang-core. Boss forward-fix folded in: Concern 2 surfaced that the committed specs themselves (`2026-05-17-loop-recur.md` headline + `bad_recur`, `2026-05-17-llm-surface-discipline.md` §5) wrote bare `Int` — corrected all three snippets to `(con Int)` (doc-honesty, same class as the mono.rs header; no design/schema/assumption change, no re-brainstorm). `cargo test --workspace` 600→608 / 0 red (Boss-reran independently). Component 4 (typecheck) = iter 2; Component 5 (codegen) + positive/deep-`n` E2E = iter 3 → 2026-05-17-iter-loop-recur.1.md -- 2026-05-17 — iter loop-recur.2: standalone-`loop`/`recur` milestone (2 of 3) — Component 4 typecheck semantics. The iter-1 `synth` `CheckError::Internal` stub for `Term::Loop`/`Term::Recur` is replaced with real binder typing (each init synthed in outer scope + already-declared binder names of THIS loop via ordinary `locals` save/restore mirroring `Term::Let` verbatim; loop's static type = body type) + positional `recur` arity/per-arg-type checking via a new `loop_stack: &mut Vec>` frame threaded exactly as mut.2's `mut_scope_stack` (every recursive + cross-module + test synth caller, compile-swept). New private `verify_loop_body(t,in_loop_tail)` pass — a SIBLING of the byte-frozen `verify_tail_positions` (0 deletions, tail-app role untouched), invoked in `check_fn` after it so synth-raised codes take precedence by pass ordering. Four `Recur*` `CheckError` variants (`RecurOutsideLoop`, `RecurArityMismatch{expected,got}`, `RecurTypeMismatch{position,expected,got}`, `RecurNotInTailPosition`) + `code()` + 2 `ctx()` arms, bracket-`[code]`-free Display (F2); four negative `.ail.json` fixtures fire their codes point-exactly (`assert_eq!`, non-vacuous) via a new `loop_recur_typecheck_pin.rs` (7 tests) + a ct1 F2 human-mode sibling; `carve_out_inventory` 13→17 with the pre-existing mut.4-tidy "Twelve"-vs-13 stale-header drift corrected in passing. iter-1's `loop_sum_to.ail` round-trip fixture now ALSO typechecks clean (one fixture, two properties — no near-duplicate); `loop_forever.ail` (loop whose only path is `recur`) typechecks clean, asserting the spec "no termination claim is made or enforced". Three binding Boss design calls (mirrored into the journal): (1) `RecurTypeMismatch` is an Assign-style `subst.apply`-both-then-structural-`!=` pre-check, NOT a `unify`-propagate (a propagate would surface generic `type-mismatch` and fail the point-exact-code acceptance); (2) `loop_stack` element type is positional `Vec` NOT name-keyed `IndexMap` (recur rebinds by position; binder names live in `locals`); (3) diagnostic-code precedence is by pass ordering (synth before verify_loop_body), no explicit logic. NO codegen (iter-1 `lower_term` `CodegenError::Internal` stub stays — iter 3); NO `Diverge`/`verify_structural_recursion`/guardedness (spec deliberate boundary). Two DONE_WITH_CONCERNS, both Task 2, both journalled: (a) a plan-ordering defect — Task 2's `cargo build` 0-errors gate is unsatisfiable while `check_fn` (deferred by the plan to Task 4) is an unthreaded caller; orchestrator pulled the byte-identical declaration+threading forward to Task 2, left only the `verify_loop_body` invocation in Task 4 (no semantic change, only sequencing); (b) recon-undercount of cross-module synth callers (`builtins.rs`×2, `lift.rs:746`, `mono.rs:720`/`:1361`), the recurring mut.2-class, resolved via the plan's own designated compile-sweep oracle. Boss systemic fix folded into this commit: planner SKILL.md Step-5 self-review gains item 7 (compile-gate vs. deferred-caller ordering) so the Concern-(a) class is scrubbed at plan time, not discovered at execution. `cargo test --workspace` 608→616 / 0 red (Boss-reran independently); `verify_tail_positions`/tail-app byte-frozen confirmed by diff-hunk inspection. Component 5 (codegen loop-header/phi/back-edge) + the positive RUN-to-value / deep-`n` E2E = iteration 3 → 2026-05-17-iter-loop-recur.2.md -- 2026-05-17 — iter loop-recur.3: standalone-`loop`/`recur` milestone (3 of 3, TERMINAL) — Component 5 codegen + run-to-value E2E. The iter-1 `lower_term` `CodegenError::Internal` stub for `Term::Loop`/`Term::Recur` is replaced with real LLVM-IR lowering: loop binders are loop-carried values lowered as entry-block allocas (the mut.3 `pending_entry_allocas` mechanism) registered in the EXISTING `mut_var_allocas` map so the EXISTING `Term::Var` load path resolves them with zero new Var code (representation-sharing only — Var-lowering byte-unchanged); a fresh `loop.header.` block reached by an unconditional `br` from the pre-header; `recur` lowers ALL args to SSA BEFORE any store (simultaneous positional rebind), stores into the binder allocas, back-edges `br` to the header, and sets the SINGLE existing `block_terminated` field at its OWN new emit site (a parallel SET call-site) so the loop's exit value is the emergent product of the existing `if`/`match` join — no separate loop-result phi/exit-block. A new `loop_frames` codegen stack lets `recur` find its target header+slots; saved/reset/restored at the single lambda-lowering boundary exactly as mut.3's `mut_var_allocas` triple. `clang -O2` mem2reg promotes the binder allocas to the phi nodes the spec's secondary implementation-shape describes. Four binding Boss design calls (mirrored into the journal), all on architectural-consistency / spec-pinned-invariant grounds (NOT effort): (1) alloca+mem2reg NOT hand-emitted phi (the linear-emit architecture has no phi-with-body-discovered-back-edge-predecessor precedent; the `if` arm sidesteps by opening its join last — a loop header cannot be opened last; mem2reg yields identical optimized output; spec's "phi" is the explicitly-secondary impl-shape the spec subordinates to the planner's exact-bytes authority); (2) reuse `mut_var_allocas` + the existing `Term::Var` load path (byte-unchanged; representation-sharing, the iter-2 AST/typecheck binder distinction is post-typecheck-irrelevant to codegen); (3) emergent loop-exit via the existing if/match join once recur terminates its block, no separate result phi; (4) reuse the single `block_terminated` field, recur sets it at its own emit site, ZERO edits to any existing SET/READ site (a second field would force editing every READ site — the opposite of the spec-pinned "tail-app's use byte-unchanged"). Diff confirmed surgical by hunk-header inspection: codegen/lib.rs = exactly 3 hunks (field + init + the stub→2-arms replacement), codegen/lambda.rs = exactly 2 hunks (save + restore); NO hunk at any existing `block_terminated` SET/READ site, at tail-app/tail-do lowering, or at `verify_tail_positions`. Three new `.ail` fixtures: `loop_sum_to_run.ail`→`55` (run-to-value), `loop_sum_to_deep.ail`→`500000500000` (1e6 iterations via the back-edge, no stack growth — the spec clause-2 correctness claim made executable), `loop_forever_build.ail` (infinite loop, no non-recur exit, `ail build` exit 0 — "typechecks AND compiles, no termination claim"; never executed). Codegen-ONLY iteration: no schema/AST/serde/typecheck change; hash pins (`loop_recur` + iter13a) + drift trio + `carve_out_inventory` confirmed green UNTOUCHED (not modified); the 3 new `.ail` fixtures are round-trip-auto-covered, NOT carve-outs. One DONE_WITH_CONCERNS (T3): the plan's literal `cargo test --workspace tail` filter resolves to ZERO tests (real guards are `*_tail_position_*`/`*musttail*`); orchestrator ran the gate via real names + the authoritative full-suite 619/0 which subsumes it — recurring `feedback_plan_pseudo_vs_reality` class, no behaviour change. Boss systemic fix folded into this commit: planner SKILL.md Step-5 gains item 8 (verification-command filter strings must resolve to ≥1 named test, or use the unfiltered suite + a count assertion) — the SECOND planner-meta-gap this milestone surfaced (iter-2 added item 7). `cargo test --workspace` 616→619 / 0 red (Boss-reran independently); the 3 loop/recur e2e explicitly green (55 / 500000500000 / infinite-compiles). **All three components shipped — the loop/recur milestone is structurally complete; milestone-close (mandatory `audit`, then `fieldtest` since loop/recur is user-visible Form-A surface) is the Boss's next pipeline step.** → 2026-05-17-iter-loop-recur.3.md -- 2026-05-18 — iter loop-recur.tidy: milestone-close audit resolution (RED `39380d3` + this GREEN). Architect drift review found a `[high]` correctness defect: a `(lam ...)` body capturing an enclosing `Term::Loop` binder passed `ail check` (exit 0) then panicked `unreachable!()` at `crates/ailang-codegen/src/lambda.rs:102` on type-correct input — a DESIGN.md "Robustness against hallucinations" violation. Root cause (debugger-confirmed RED-first): the `Term::Lam` escape guard at `crates/ailang-check/src/lib.rs:3608` iterated only `mut_scope_stack`; loop binders live in the ordinary `locals` (iter-2 design) and never entered any escape-checked scope, so the mut.4-tidy `MutVarCapturedByLambda`-class rejection had no loop-binder counterpart. Fix (GREEN, implement mini-mode): new `CheckError::LoopBinderCapturedByLambda { name }` (bracket-`[code]`-free F2 Display) + `code()` `loop-binder-captured-by-lambda` + `ctx() {name}`, symmetric to `MutVarCapturedByLambda`; the `Term::Lam` escape guard gained a parallel pass over `loop_stack` (gate widened to `!mut_scope_stack.is_empty() || !loop_stack.is_empty()`). Minimal-mechanism design call: `loop_stack`'s frame element extended `Vec` → `Vec<(String, Type)>` so the already-threaded per-loop scope frame also carries binder names (`Term::Recur` still reads `.1` BY POSITION — Boss-call-2's positional-recur invariant preserved verbatim, the name is a second field consumed only by the escape guard, a consumer iter-2 did not anticipate; the parallel-stack alternative would touch all 35 `synth` call sites, strictly larger). Codegen byte-unchanged (typecheck-only fix; the lambda.rs:102 `unreachable!()` is now genuinely unreachable, kept as a defensive assertion). Two mut.4-tidy-mirrored in-source unit tests; lockstep `carve_out_inventory` 17→18 + `ct1_check_cli` mut-F2 sibling `cases`; DESIGN.md one-line symmetric-rule note. Boss folded into this GREEN commit: the two `[medium]` doc-honesty edits the audit also surfaced (`mut_var_allocas` rustdoc now states its mut-var+loop-binder dual use truthfully; `Term::Loop` doc-comment now describes the real shipped iter-2/3 typecheck+alloca+mem2reg state, not iter-1's stale "per-binder phi" stub forward-look) + a `[low]` P2 roadmap `[todo]` (the recon-undercount 3×-pattern → an `ailang-plan-recon` agent-definition countermeasure; pairs with planner Step-5 items 7+8). Bench gate (audit Step 2): `check.py`/`compile_check.py`/`cross_lang.py` all exit 0, 25 metrics 0 regressed — **pristine** (strictly-additive surface, no hot-path; the pre-existing P2 `*.bump_s` hardware-staleness drift did not trip) → carry-on, no baseline/ratify. `cargo test --workspace` 619→622 / 0 red (Boss-reran independently incl. hash_pin 11/0 — the ast.rs/codegen doc-comment edits moved no canonical-JSON hash). **The loop/recur milestone is audit-clean** — architect drift cleared, bench pristine; the only remaining pipeline step before full close is `fieldtest` (loop/recur is user-visible Form-A surface) → 2026-05-17-iter-loop-recur.tidy.md -- 2026-05-18 — fieldtest loop-recur (milestone CLOSE, clean on all milestone axes): post-audit downstream-LLM-author field test of the shipped loop/recur surface. The fieldtester (DESIGN.md + public examples only, never compiler source) wrote 3 real iterative programs — integer Newton sqrt (multi-binder, recur buried in if/let/if, loop result into let/seq), Collatz length (recur in match-arm tails), Euclidean gcd (loop as a value sub-expression in argument position) — plus 5 plausible-mistake negatives and 2 no-termination probes, all run through the public `ail` CLI. **0 bugs; 4 working findings, all on the milestone's own axes**: the five rejection diagnostics (recur-outside-loop / -arity / -type / -not-in-tail / loop-binder-captured-by-lambda) are point-exact AND self-fixing (name the fn, describe the fix, no false positives, no crash — the clause-2 silent-failure-class-becomes-compile-error claim made real); recur tail-position threads correctly through match-arm/let/outer-if (the spec only demonstrated `if` — generalises exactly as an author assumes); loop composes as a value sub-expression everywhere + every loop/recur fixture is `ail parse|render|parse` byte-identical (Roundtrip Invariant holds in practice); the no-termination boundary is exactly as specified (infinite loop check+build clean, zero spurious diagnostics — the precise difference from the reverted Iteration-discipline milestone). This empirically substantiates the milestone's "an LLM author can now write iterative programs with loop/recur" claim — the gate fieldtest exists to provide. Two ORTHOGONAL non-blocking findings surfaced incidentally while probing the no-termination/negative axes, neither in loop/recur scope, both routed to P2 todos (not a loop/recur tidy — refusing the scope creep): (spec_gap) niladic `(app f)` is unrepresentable in Form A and DESIGN.md's `Term::App args:[Term...]` states no minimum — INDEPENDENTLY re-confirms the existing mut-local-F3 roadmap todo (two fieldtests now hit it; the accept-`(app f)`-vs-ratify-DESIGN.md decision is a genuine design fork deliberately NOT auto-ratified under autonomous /boss — parked, priority-strengthened); (friction) the module-level `(doc …)` rejection lists valid def heads but omits that doc attaches inside fn/data — a one-line diagnostic-hint tidy. Independent Boss verification: `loop_recur_4_gcd_value_pos.ail` re-run → stdout `27`; `loop_recur_3a` re-run → `ail check` exit 1 `[recur-outside-loop] countdown: …`. **The standalone loop/recur milestone is fully ratified and CLOSED**: 3 iterations + a tidy shipped, audit clean (architect drift resolved, bench pristine carry-on), fieldtest clean on every milestone axis. Fixtures `examples/fieldtest/loop_recur_*.ail` + spec `docs/specs/2026-05-18-fieldtest-loop-recur.md` committed; roadmap P0 entry marked closed; the two orthogonal findings tracked as P2 todos → 2026-05-18-fieldtest-loop-recur.md -- 2026-05-18 — iter prose-loop-binders.1 (single-iteration milestone, DONE): Form-B prose `loop` now renders binders as a parenthesised init-list on the keyword line — `loop(acc = 0, i = 1) { … }` — instead of bare `name = init;` statements inside the body block, which had implied C/Rust re-init-every-iteration semantics. Projection-only single-arm rewrite of the `Term::Loop` case of `write_term` (`crates/ailang-prose/src/lib.rs`); init exprs render at `level` (keyword line, not the indented block), mirroring the untouched adjacent `Term::Recur` `enumerate()`+`", "` idiom — making the binder list positionally isomorphic to `recur(...)`, which teaches the correct "recur re-binds these positionally" model instead of obscuring it. RED-first via the existing stem-explicit `check_snapshot` harness: two new committed `examples/{loop_sum_to_run,loop_forever_build}.prose.txt` whole-file byte-equality fixtures + two `snapshot_loop_*` `#[test]` fns, verified `0 passed; 2 failed` against the old renderer before the arm rewrite turned them green. Loop/recur AST, Form-A, JSON-AST, typecheck, codegen and the Form-A↔JSON round-trip invariant untouched by construction (no signature change, no caller touched, prose is a one-way lossy projection off the round-trip path — spec §Architecture states this explicitly so milestone-close audit does not chase a phantom). Full `ailang-prose` suite 10/0; both `ail prose | diff` live-CLI byte-matches `MATCH`; post-cleanup porcelain exactly the expected paths. Surfaced from the 2026-05-18 user prose review of the just-closed loop/recur milestone; brainstorm spec `docs/specs/2026-05-18-prose-loop-binders.md` (Step-7.5 grounding-check PASS), plan `docs/plans/prose-loop-binders.1.md`. Both implement review phases first-pass, 0 re-loops. Milestone-close `audit` is the Boss's next pipeline step (projection-only, no Form-A surface change → no fieldtest). → 2026-05-18-iter-prose-loop-binders.1.md -- 2026-05-18 — audit prose-loop-binders (milestone CLOSE, clean / carry-on): milestone-close tidy for the single-iteration prose-loop-binders milestone (range `6cbd0fe..c9355d7`). Architect drift review **clean** — zero drift, zero debt, verified against the diff not journal trust: single `Term::Loop` `write_term` hunk in `crates/ailang-prose/src/lib.rs`, the two non-render `Term::Loop` sites + all `Term::Recur` arms absent from the diff, DESIGN.md/PROSE_ROUNDTRIP.md make no claim about prose `loop` rendering (so the spec's "no doc edit needed" is correct not a skipped obligation), no carve-out/hash-pin/round-trip lockstep bypassed (prose is off the Form-A↔JSON round-trip path by construction). Bench: `compile_check.py` 0/24 + `cross_lang.py` 0/25 both **exit 0** (cross_lang's independent `ail_bump_s` +1.50% ±15% ok corroborates check.py's bump_s firing is baseline-staleness not a real regression); `check.py` exit 1, first run 3 regressed but a confirmatory identical re-run (no code change) returned `latency.implicit_at_rc.p99_9_us` +37.30%→+10.22% **ok** and `.max_us` +114.14%→+22.17% **ok** (median/p99 within tol both runs) — the two latency-tail firings proven single-sample 5-run RC-mode jitter by re-run; `bench_list_sum.bump_s` persisted ~+12% = the already-tracked P2 roadmap todo (byte-oracle-proven environmental at the 2026-05-16 iteration-discipline-revert audit, causally impossible to attribute to a projection-only `ailang-prose` change). **All items carry-on**: no fix iteration, no baseline ratify — explicitly NOT bumping the `*.bump_s` baseline opportunistically inside an unrelated projection-only milestone (the P2 todo owns a holistic recapture; piecemeal bump is the "regression is expected" rationalisation the Iron Law forbids). Fieldtest **not applicable / not dispatched**: projection-only change to the Form-B *reading* surface; the fieldtester authors `.ail` Form-A and would not exercise the prose-render change (no friction/bug/spec-gap axis); the two whole-file byte-equality `.prose.txt` snapshots + live-CLI `ail prose|diff` checks are the projection-only E2E gate. Milestone **CLOSED clean**; roadmap P0 entry flipped `[~]`→`[x]`, WhatsNew.md user-facing entry appended (user present → no notify.sh fired per notification policy). → 2026-05-18-audit-prose-loop-binders.md -- 2026-05-18 — iter remove-mut-var-assign.1 (single terminal iteration, DONE): `mut`/`var`/`assign` removed from AILang entirely and atomically — `Term::Mut`/`Term::Assign`/`struct MutVar`, the three Form-A keywords + `parse_mut`/`parse_assign` + grammar EBNF, the 4 `mut` `CheckError` variants, the `mut_scope_stack` synth threading (param dropped from `synth` + every internal/external/test caller), the two `lower_term` arms, and every exhaustive no-`_` `Term::Mut`/`Term::Assign` match arm across 17 source files — cut in lockstep with DESIGN.md + fixtures + drift trio + carve-out + roadmap so the schema is honest at every commit (no `_ =>` wildcard introduced — Boss-verified). `loop`/`recur` + `let`/`if` are the surviving forms. Shared codegen alloca machinery survives (loop reuses it): `mut_var_allocas`→`binder_allocas` (representation-only, loop codegen byte-identical) and the shared `Term::Lam` escape guard simplified to `!loop_stack.is_empty()` with the loop half (`LoopBinderCapturedByLambda`) kept byte-equivalent. Feature-acceptance applied inverted (removed feature fails clause 2 redundant + clause 3 IS the iterated-mutable-state bug class). Behaviour-preservation proof executable: `mut_counter`/`mut_sum_floats` still print `55` after the faithful `let`/`if` rewrite (factorial→120, horner→18, classify 22→2, has_small_factor 91→true); the "removal made executable" gate is the new `mut_removed_pin.rs` (4 must-fail pins: `mut`/`assign` keyword rejected, `{"t":"mut"}`/`{"t":"assign"}` serde unknown-variant). Independent Boss verification: full `cargo test --workspace` **605/0**, zero residual mut symbols in any crate source, loop/recur non-regression all green (55 / 500000500000 / infinite-compiles / `lambda_capturing_loop_binder` pin), `roundtrip_cli` PASS. One systemic DONE_WITH_CONCERNS — 4th recurrence of the recon-undercount `feedback_plan_pseudo_vs_reality` class (in-source `mod tests` fns + a drift-pin fn + 5 orphaned mut `.ail.json` carve-outs + a non-enumerated `codegen_import_map_fallback_pin.rs` E0599 the recon missed; all resolved within implementer remit via Task-3's identical whole-file-deletion rationale, no behaviour change, e2e bodies preserved per plan) — worth a planner/recon-agent countermeasure, pairs with the existing loop-recur.tidy P2 todo. Milestone-close `audit` then `fieldtest` (surface-touching) are the Boss's next pipeline steps. spec `docs/specs/2026-05-18-remove-mut-var-assign.md` (grounding-check PASS), plan `docs/plans/remove-mut-var-assign.1.md` → 2026-05-18-iter-remove-mut-var-assign.1.md -- 2026-05-18 — audit remove-mut-var-assign (milestone close, CLEAN after one inline `.tidy`): scope `48e7774..07f0802`. Architect drift: one `[high]` — `crates/ailang-core/specs/form_a.md` (live in-tree Form-A grammar still documented the full `mut`/`var`/`assign` construct: EBNF 286-288, prose 308-328, dangling `like mut` at 332); the spec named it for deletion, the iter missed it. Root cause = recon-undercount class **4th recurrence + a Boss error**: a plan-time grep checked only `docs/form_a.md` (dir-scoped), got "No such file", and the Boss propagated that unverified non-existence claim into the recon brief — the real file is `crates/ailang-core/specs/form_a.md`, actively maintained, and a stale doc never produces a compile error so the implement compile-sweep could not catch it. Fixed inline as a Boss `.tidy` (CLAUDE.md context-already-loaded carve-out; full review discipline kept — tree-wide verified zero construct residue except the roadmap entry that names the milestone itself; grammar reads coherently reuse-as→loop→recur). One `[medium]` process-drift: escalated the existing loop-recur.tidy `ailang-plan-recon` P2 in place — broadened from cross-crate-caller to a structural recon-contract defect with two named gaps (non-compile-checked spec-named sites; unverified "X does not exist" claims inheriting into a recon brief) + a concrete fix (compile-driven enumeration as authoritative code-site set PLUS an `ls`-verified spec-named-path existence table). Bench: `compile_check.py` 0/24 + `cross_lang.py` 0/25 exit 0; `check.py` exit 1 (3 firings: `bench_list_sum.bump_s` +12.51%, `latency.{explicit,implicit}_at_rc.max_us` +108%/+66%). Bencher causal exoneration **decisive**: HEAD vs `48e7774` `list_sum_bump`/`lat_expl_rc`/`lat_impl_rc` binaries `cmp`-byte-identical — the −2520-line removal alters zero bytes of these fixtures' machine code, no causal mechanism; `bump_s` = the tracked P2 environmental staleness, both `*.max_us` = single-sample `-n 5` jitter (`explicit_at_rc.max_us` +108%→**-2.30% ok** on identical-code rerun, medians held). **PRISTINE met on the merits; NO baseline ratify** (Iron Law forbids ratifying noise, spec forecloses it). Filed a NEW distinct P2 (separate from `*.bump_s` staleness): `check.py` RC-latency `*.max_us` at `-n 5` is a 3-for-3 structural false-positive on no-runtime-change milestones → drop from gating / raise `-n` / cpuset-pin. Milestone audit **CLEAN**; only `fieldtest` (surface-touching) remains before close → 2026-05-18-audit-remove-mut-var-assign.md -- 2026-05-18 — fieldtest remove-mut-var-assign (milestone CLOSE, clean — removal thesis empirically confirmed): post-audit downstream-LLM-author field test of the post-removal Form-A surface. The fieldtester (DESIGN.md + form_a.md + public examples only, never compiler source) wrote 4 FRESH real-world programs an imperative-trained author would instinctively reach for a mutable local for — running sum-of-squares accumulator (`loop`/`recur`, 2 binders → `385`), grade-classification cascade (let-threaded flag through 4 `if` → `4`/`2`/`0`), Horner polynomial straight-line build-up (5 `let acc` shadows → `73`), multi-state bracket-balance scanner threading `(rest,depth,ok)` by tail recursion → `true`/`false`) plus an out-of-tree `mut`-keyword rejection probe. **0 bugs, 0 friction, 0 spec_gap, 4 working**: every task expressed cleanly and correctly on the FIRST try with only `let`/`if`/`loop`/`recur`; all 4 `ail parse|render|parse` byte-identical. The closest-to-friction case (the multi-state machine must restate the full state tuple at every tail-call even when a branch changes one component) was carefully classified `working` not `spec_gap` — that explicit-dataflow cost IS the local-reasoning pillar working as designed, and `mut`'s implicit cross-iteration persistence of un-restated state is precisely the clause-3 failure the removal eliminates; the fieldtester explicitly noted the only conceivable sugar (a record to bundle state) is an orthogonal additive future question, NOT evidence against the removal (refused the scope-creep). The `mut` rejection is fail-closed with an actively-guiding diagnostic that enumerates the surviving term heads incl. `loop`/`recur`/`let` (no tombstone, no-nostalgia, as spec'd). Independent Boss verification: all 4 fixtures re-run → `385`/`4 2 0`/`73`/`true false`, zero `mut`/`var`/`assign` construct tokens in any fixture (a doc-faithful author did not produce the removed construct — the affirmative clause-1 evidence). **Verdict: the removal thesis is confirmed** — `mut` was redundant with `let`/`if`/`loop`/`recur` in 100% of the fresh real tasks; no expressivity lost. **The remove-mut-var-assign milestone is fully ratified and CLOSED**: spec+plan+iter, audit clean (architect `[high]` form_a.md fixed in `.tidy`, bench causally exonerated), fieldtest clean on the removal axis. Fixtures `examples/fieldtest/remove-mut_*.ail` + spec `docs/specs/2026-05-18-fieldtest-remove-mut-var-assign.md`; roadmap P0 flipped `[~]`→`[x]`, WhatsNew.md user-facing entry appended; next queue item (DESIGN.md/docs honesty-lint) is a NEW milestone = a /boss new-milestone bounce-back (not auto-started) → 2026-05-18-fieldtest-remove-mut-var-assign.md -- 2026-05-18 — iter docs-honesty-lint.1 (single-iteration milestone, DONE): canonical docs made present-tense-honest. Stripped Wunschdenken (forward intent stated as fact) + non-citation post-mortem/doc-archaeology from `docs/DESIGN.md` (14 corrections: L484 iter-anchor, L503/L1318/L1504/L2027/L2065/L2135 post-mortem, L650 malformed-Wunschdenken, L655 dated-heading, L1513/L1837/L2046/L2519 Wunschdenken, + the Task-4-Step-14 procedural-catch-all FIX at the Form-B prose-projection bullet "was deferred from milestone 22 entirely … A future iter ships" — a soft-wrapped site the spec's illustrative regex would have missed, the procedural enumeration working exactly as designed) + `docs/PROSE_ROUNDTRIP.md` (tool-use/MCP paragraph); genuine forward intent (LLM tool-use/MCP/LSP) relocated to roadmap P3. Codified the two-pronged tense+modality discriminator as a present-tense `### What this document is — and the honesty rule it holds itself to` meta-subsection in DESIGN.md §"Project ecosystem". Anti-regrowth two ways: wrap-robust enumerated absent/present pin `crates/ailang-core/tests/docs_honesty_pin.rs` (a `norm()` whitespace-collapse helper structurally discharges the recurring grep/contains line-wrap failure family — strictly better than the effect-doc-honesty "keep-on-one-line" precedent; resolves spec Testing-item 0 + planner self-review item 6 at once) + wrap-robust advisory Sweep 5 in `bench/architect_sweeps.sh` (contiguous-fragment regex, since DESIGN.md is hard-wrapped ~70col) with full sweep-count lockstep (header + "All four"→"All five" tail + `ailang-architect.md` "four"→"five sweeps") + a new `ailang-architect.md` "DESIGN.md honesty drift" bullet citing the meta-subsection. Protected-exception KEEPs preserved + pinned-present (Diverge-reserved, regions-considered-and-rejected, the self-labelled "tiebreaker, not a rationale"); `form_a.md` correctly left untouched/unpinned. One plan-internal DONE_WITH_CONCERNS (Task-1 pin string omitted a `**…**` bold the Task-3 body added; resolved by dropping the non-load-bearing emphasis, pin is the contract) — planner self-review-item-3 calibration note. Independent Boss verification: pin 4/4 GREEN, full `cargo test --workspace` 0 failures (delta = +4 pin tests only), sentinel trio (`design_schema_drift`/`spec_drift`/`schema_coverage`) green, `cargo build --workspace` clean, scope = ONLY THE PIN in `crates/` (zero language/checker/codegen/runtime change, `ail check`/`run`/`build` byte-unchanged by construction). Post-iter sweep residue = only the 3 documented deliberate KEEPs (Sweep-1 L50 doc-roles-pointer date + L2060 true `(iter str-concat)` provenance; Sweep-5 L62 self-referential discriminator-catalogue), EXIT=1 architect-adjudicated-expected. Milestone-close `audit` is the next pipeline step (no fieldtest — zero authoring-surface change). spec `docs/specs/2026-05-18-docs-honesty-lint.md` (grounding-check PASS 10/10), plan `docs/plans/docs-honesty-lint.1.md` → 2026-05-18-iter-docs-honesty-lint.1.md -- 2026-05-18 — audit docs-honesty-lint (milestone close, CLEAN — carry-on, NO ratify): scope `928e1c0..e50b400`. Architect `clean`: scope claim exact (zero `crates/**/src/**` + `runtime/`; sole crates/ change the additive RED-first-now-GREEN `docs_honesty_pin.rs`; lockstep pairs untouched), all 14 DESIGN.md corrections genuinely present-tense with the 4 protected KEEPs (`Diverge`-reserved/`Regions`-rejected/`tiebreaker`/`No deriving`) un-stripped, agent-def "four→five" lockstep complete, sweep residue exactly the 3 documented deliberate KEEPs. One `[medium]` standing advisory wart — Sweep 5's regex self-matches the discriminator subsection's own forbidden-phrasing catalogue (DESIGN.md L62) forever (architect-flagged; orchestrator-adjudicated → filed as a P3 todo, NOT a fix iteration — not a new failure mode since the script is advisory-with-residue by design). Bench: `compile_check.py` 0/24 + `cross_lang.py` 0/25 EXIT 0 (cross_lang independently reads the same `bench_list_sum` bump path +5.95% within tol); `check.py` EXIT 1, 3 firings (`bench_list_sum.bump_s` +13.42%, `latency.implicit_at_rc.{p99_9,max}_us` +37.65%/+134.70%). Bencher causal exoneration **decisive**: HEAD vs `5bb7211` vs `de66eb7` — all 5 firing bench binaries `cmp`-byte-identical, `ail` sha256 identical at all three commits ⇒ zero causal mechanism; `*.max_us` = tracked-P2 `-n 5` tail-jitter (4th occurrence on a zero-runtime-change milestone, median/p99 rock-steady), `*.bump_s` = tracked-P2 environmental-anchor-staleness (persistent ~+15%, stale 0.046s anchor). **EXONERATED, NO baseline ratify** (Iron Law forbids ratifying noise; spec forecloses a zero-codegen baseline bump). Two pre-existing standing P2 items kept orchestrator-owned, explicitly NOT bundled into this close (bencher recommendation honoured). Milestone audit **CLEAN**; no fieldtest (zero authoring-surface change — the wrap-robust pin + Sweep 5 ARE the regression coverage). **docs-honesty-lint fully ratified and CLOSED**: spec (grounding-check PASS 10/10) + plan + single iter + audit clean; roadmap P0 flipped `[~]`→`[x]`, WhatsNew.md user-facing entry appended → 2026-05-18-audit-docs-honesty-lint.md -- 2026-05-18 — iter embedding-abi-m1.1 (Embedding-ABI M1, DONE across 2 dispatches + a Boss plan Repair): a compiled AILang scalar `fn` is now callable in-process from a C/Rust host. Five pipeline layers shipped: additive `FnDef.export: Option` (serde `default`+`skip_serializing_if`, modelled on `FnDef.doc`, ~85-site compile-driven thread incl. all in-source `#[cfg(test)]`/drift/hash literals, hash-stable golden `b4662aa70839f60b`); Form-A `(export "")` modifier (parse/print/`form_a.md` lockstep, byte-identical round-trip); check-side scalar-only+effect-free export gate — two new `CheckError` Error variants `export-non-scalar-signature`/`export-has-effects`, params→ret→effects first-violation-wins, unconditional on `f.export.is_some()` (the feature-acceptance clause-3 discriminator *in code*); codegen `Target::StaticLib` (suppresses `@main`+`MissingEntryMain`, emits one external `@` forwarder per export to `@ail__`); CLI `ail build --emit=staticlib` (`build_staticlib`: `libembed_backtest_step.a` program-only + separate `libailang_rt.a`). Coherent stop PROVEN: `embed_e2e.rs` `cc`-links both archives, `backtest_step(0,3)=9`/`(9,4)=25`, `s==25`, exit 0. Task 1 (fixed-first CLI `MissingEntryMain` baseline pin) green before any staticlib code. Process: Task 4 BLOCKED on the first dispatch — a Boss plan-defect (gate/headline fixtures declared `module ≠ file-stem`; the loader hard-enforces `module==stem` at `workspace.rs:438`, so `ail check` panicked on load before the gate). Boss verified, chose **Option 2 over the orchestrator-recommended Option 1** (keep descriptive `embed_*` filenames, rename fixture *modules* to their stems — the C symbol `backtest_step` stays via `(export …)`, *demonstrating* spec Decision 2's mangling-decoupling rather than violating it; archive→`libembed_backtest_step.a`, internal→`@ail_embed_backtest_step_step`, `host.c` unchanged), Repaired the plan (Design-decision 6 + Task-4 module==stem + Task-5/6/7 dependent strings + folded the no-`*.`-Float-head and `content_hash_fn`→`def_hash` concerns), committed Tasks 1–3 as a verified known-good partial (`818177d`), re-dispatched [4,7]. Independent Boss verification post-completion: E2E 1/1, gate 6/6, lowering 2/2, CLI 2/2, full `cargo test --workspace` 0 failures, `roundtrip_cli`+`design_schema_drift`+`spec_drift` green, Invariant 1 holds (no new dep in `ailang-core`/`ailang-codegen`/`runtime`). 3 behaviour-neutral plan pseudo-vs-reality concerns recorded (`emit_ir` routes via `lower_workspace`; dead `Path` import dropped; `##` vs `###` heading). Bench: `compile_check.py`/`cross_lang.py` exit 0; `check.py` exit 1 = tracked-P2 `*.bump_s`/`*.max_us` known-noise (default-exe codegen/runtime byte-unchanged — audit-adjudicated, not a plan gate). spec `docs/specs/2026-05-18-embedding-abi-m1.md` (grounding-check PASS) → plan `docs/plans/embedding-abi-m1.1.md` (Repaired) → commits `818177d` (partial 1–3) + this completion. Milestone-close `audit` then `fieldtest` (surface-touching — `(export …)` is new authoring surface) are the next pipeline steps. → 2026-05-18-iter-embedding-abi-m1.1.md -- 2026-05-18 — audit embedding-abi-m1 (milestone close, CLEAN — carry-on, NO ratify): scope `064599e..e406d07`. Architect `clean` — Invariant 1 (clean core) holds semantically (no finance/`data-server` vocab/logic/dep in `ailang-core`/`ailang-codegen`/`runtime`; `backtest_step` is an opaque author string through generic `FnDef.export`; AILang↔host meeting point correctly M5-deferred), DESIGN.md honesty satisfied (§"Embedding ABI (M1)" + fn-JSON `"export"` + field rustdoc present-tense; "provisional until M3"/"M2 changes the C sig" correctly *labelled* reserved-unimplemented), lockstep intact (`FnDef`↔fn-JSON↔drift/spec-drift carry the field; `@` forwarder additive raw-IR *beside* unchanged `@ail__` mangling — not through `lower_app`; schema exhaustively-constructed, no `_=>`/`..Default`; in-source `missing_entry_main_is_error` byte-unchanged; `_adapter`/`_clos` untouched), Design-decision 6 confirmed correct (fixture module==stem differs from the spec's *illustrative* `(module backtest)`/`(module bad)` by the loader invariant + spec Decision 2 — the fixtures *demonstrate* the C-symbol decoupling, not violate the spec). One `[low]` — ~85 `export: None,` lines brace-column-indented (rustfmt-divergent) — **orchestrator-adjudicated carry-on on evidence** (not pending, not ignored): `cargo fmt --all --check` shows **1137 fmt-divergent locations tree-wide predating M1**, so rustfmt is demonstrably not a project convention and the architect's "cargo fmt would churn later" premise is moot; selectively aligning 85/1137 would impose an absent standard (noise, not tidy). Bench: `compile_check.py` 0/24 + `cross_lang.py` 0/25 EXIT 0; `check.py` EXIT 1, 7 firings (`*.bump_s` trio +12–14%; `explicit_at_rc`/`implicit_at_rc` `p99_9`/`max_us` +36–124%). Bencher causal exoneration **decisive**: 21/21 bench binaries `cmp`-byte-identical `064599e`↔`e406d07` (`bench_list_sum_bump` sha `dce18c288904c9f0` both) — M1's `Target::Executable` default leaves every bench fixture's emitted machine code unchanged ⇒ zero causal mechanism; RC-latency tails reproduce *on the byte-identical oracle itself* (3× run-to-run swing, median/p99 steady = `-n 5` single-sample jitter); fresh re-run collapsed 6/7 firings to ok. Both families map (confirmed) onto the two pre-existing roadmap-P2 tracked items (~4th zero-runtime-change milestone they fire+collapse on). **EXONERATED, NO baseline ratify** (Iron Law forbids ratifying noise against a zero-default-path-byte iter; spec forecloses a zero-codegen baseline bump); the two P2 bench-harness-recalibration items stay orchestrator-owned, NOT bundled (bencher recommendation honoured). M1 milestone audit **CLEAN**; `fieldtest` (surface-touching — `(export "")` is new authoring surface) is the next pipeline step before close → 2026-05-18-audit-embedding-abi-m1.md -- 2026-05-18 — fieldtest embedding-abi-m1 (milestone CLOSE, surface-touch — thesis substantiated, 3 non-blocking follow-ups routed): post-audit downstream-LLM-author field test of M1's `(export "")` + `ail build --emit=staticlib` + scalar/effect-free gate, DESIGN.md + public examples only (never compiler source). 4 fresh programs: Int fixed-point EMA `(State,Sample)->State` fold + Float leaky integrator (both author→`--emit=staticlib`→C-host link→typed scalar return, `s==67`/`s==1.875`, exit 0 — **clause-1 confirmed first-try**), IntList-param export (correctly rejected `export-non-scalar-signature`), `!IO` scalar-clean export (correctly rejected `export-has-effects`) — **clause-3 confirmed, both diagnostics precise+self-correcting**. **0 bugs**; the M1 capability + gate are sound and the milestone thesis ("an LLM author makes a scalar fold callable") is empirically substantiated. 3 working + 1 friction + 2 spec_gap. The friction + spec_gap#1 share a root: `crates/ailang-core/specs/form_a.md` "Schema invariants" item 1 + Pitfalls state an *unconditional* "every fn param MUST be `(own/borrow T)`" but scalars require **bare** `(con Int)` (`(own (con Int))`→`use-after-consume`, `(borrow)`→`consume-while-borrowed`); the doc the LLM reads first misdirects M1's headline task into a body-pointing linearity diagnostic — a **pre-existing form_a.md docs-honesty defect M1 made acute, not introduced** (the positive examples only succeeded because the fieldtester imitated the public `embed_*` corpus over the form_a.md prose rule). spec_gap#2: no public `emit-ir --emit=staticlib` though Decision 5 makes kernel-IR readability load-bearing. Boss independently re-verified via the public CLI (positive E2E builds both archives; both rejections fire the documented codes). Orchestrator routing (own judgement, not the fieldtester's recommended-action column): friction+spec_gap#1 → ONE P1 `[todo]` docs-honesty tidy (form_a.md scalar-param carve-out symmetric with the existing return-type carve-out + DESIGN.md mirror; behaviour settled → tidy, no brainstorm; ahead of M2 because it undermines the clause-1 ergonomics of the surface M2–M5 extend); spec_gap#2 → P1 `[feature]` add `--emit=staticlib` to `ail emit-ir` (restore the affordance, NOT narrow DESIGN.md). None are bugs (no debug); none M1-blocking. **embedding-abi-m1 fully ratified and CLOSED**: spec (grounding-check PASS) + plan (Boss-Repaired) + iter (`818177d` partial+Repair / `e406d07` DONE) + audit `425c4eb` CLEAN + fieldtest thesis-substantiated; roadmap P1 `[~]`→`[x]`, 2 follow-ups added P1-ahead-of-M2. Next *milestone* is Embedding ABI — M2 (no spec) = a `/boss` new-milestone bounce-back (not auto-started); the 2 P1 follow-ups are autonomous-eligible tidies/feature but the session is context-deep — surfaced to the user for the session-shape/next-item call. → 2026-05-18-fieldtest-embedding-abi-m1.md -- 2026-05-18 — iter form-a-scalar-param-mode-carveout (M1-fieldtest follow-up #1, docs-honesty tidy in the docs-honesty-lint class, DONE): resolved the M1 fieldtest `[friction]` + `[spec_gap]#1` shared root. `crates/ailang-core/specs/form_a.md` stated an *unconditional* "every `(fn ...)` param MUST carry an `(own/borrow)` mode" rule in four places (`### Function` prose L97, grammar-block comment L230, "Schema invariants" item 1 L347, Pitfalls bullet L382) that contradicts shipped checker behaviour — scalar `Int`/`Bool`/`Unit`/`Str` params take **and require** bare `(con Int)` (a mode on a scalar makes the linearity pass hold the primitive to linear discipline → body-pointing `use-after-consume`/`consume-while-borrowed`, the exact fieldtest misdirection). All four sites rewritten symmetric to the pre-existing return-type carve-out; site 5 (L521 few-shot annotation) already correct, left verbatim by design (carve-out reuses its heap-shaped/primitive vocabulary for end-to-end internal consistency). `docs/DESIGN.md` §"Embedding ABI (M1)" gained the bare-scalar export-param rule + a corpus-grounded `step` Form-A snippet (byte-identical to `examples/embed_backtest_step.ail:3-11`). RED-first anti-regrowth pin `form_a_scalar_param_carveout_present_and_old_rule_absent` in `crates/ailang-core/tests/docs_honesty_pin.rs` (reads the canonical `ailang_core::FORM_A_SPEC` `include_str!` const — single-source, compile-checked — + `read("docs/DESIGN.md")`; `norm()` whitespace-collapse structurally discharges the grep/contains line-wrap family; 4 ABSENT + 4 PRESENT form_a + 1 PRESENT DESIGN). Boss scope calls: 4 form_a sites not 2 (internal-consistency, on the merits — a tidy whose thesis is "consistent current-state mirror" cannot fix some and leave others contradicting); pin idiom = `FORM_A_SPEC` const (semantic single-source, not effort). `spec_drift.rs` recon-confirmed NOT a lockstep partner (anchors are short keyword tokens; `(own`/`(borrow` survive an additive carve-out). Zero language/checker/codegen change by construction; independent Boss verification: docs_honesty_pin 5/0, spec_drift 8/8, ailang-core 112/0, workspace 622→623 (+1 = the pin), zero `crates/**/src/**` diff. spec carrier `docs/specs/2026-05-18-fieldtest-embedding-abi-m1.md` + roadmap P1 entry → plan `docs/plans/form-a-scalar-param-mode-carveout.md` (`4c266a6`). Pure docs+pin tidy — no audit/fieldtest gate (zero authoring-behaviour change; the pin IS the regression coverage). → 2026-05-18-iter-form-a-scalar-param-mode-carveout.md -- 2026-05-18 — iter emit-ir-staticlib (M1-fieldtest follow-up #2, scoped [feature], DONE): resolved M1 fieldtest [spec_gap]#2 — `ail emit-ir` had no `--emit=staticlib` (only `ail build` did) and on a `main`-free kernel hit the executable-path `MissingEntryMain` rejection, so an author could not read the generated `@` forwarder for an exported kernel — the exact Decision-5 IR-readability affordance, for the exact artefact M1 introduced. Added a one-line symmetric codegen convenience `lower_workspace_staticlib(ws)` (mirrors `lower_workspace`: `--emit=exe`↔`lower_workspace`, `--emit=staticlib`↔`lower_workspace_staticlib`; delegates to the M1-audited unchanged `Target::StaticLib` forwarder path), an `emit: String` clap field on `Cmd::EmitIr` symmetric with `Cmd::Build`'s, and a zero-export guard byte-identical to `build_staticlib`'s (so the error is symmetric across both subcommands and the existing `embed_staticlib_cli.rs` substring assertion holds for both). Recon resolved the only integration risk negative (both codegen entrypoints return `Result`, the emit-ir printer already consumes `String` — no adapter). DESIGN.md *widened* not narrowed (roadmap resolution honoured): a present-tense affordance sentence in §"Embedding ABI (M1)" + a CLI-synopsis correction that also discharges a pre-existing M1 omission (`ail build --emit=staticlib` was never in the synopsis). Boss editorial calls: synopsis fixes BOTH lines (docs-as-current-state-mirror consistency, on the merits); NO new docs pin (the E2E pins behaviour; the architect's mandatory DESIGN.md read at milestone close catches stale prose — pinning every feature sentence is noise); Decision-5 prose left untouched (already present-tense correct). New E2E `crates/ail/tests/emit_ir_staticlib_cli.rs` (3 tests: positive — IR contains external `@backtest_step(` + internal `@ail_embed_backtest_step_step`, NOT `@main(`, NOT `has no main def`; zero-export guard symmetric with build's, reusing the identical `embed_noentry_baseline.ail` fixture; default-exe regression — `--emit` defaults to `exe`, a `main`-free kernel without the flag still hits `MissingEntryMain`, so the new branch cannot silently reroute the default path). RED-first observed exactly ("1 passed; 2 failed" pre-edit — the default-exe test passes because that IS the spec_gap behaviour). Independent Boss verification: the 3 new tests 3/0, `embed_staticlib_cli` 2/0 (build-side untouched), `docs_honesty_pin` 5/0 (no regression), `ail` suite 186/0, workspace 623→626 (+3 = exactly the new tests), zero diff outside the 4 in-scope paths, no fixture minted. Functional spot-check from the user-facing CLI: `ail emit-ir examples/embed_backtest_step.ail --emit=staticlib` emits `define i64 @backtest_step(...)` (the readable external forwarder) + `@ail_embed_backtest_step_step` (internal), no `@main` — the affordance is genuinely delivered. spec carrier `docs/specs/2026-05-18-fieldtest-embedding-abi-m1.md` [spec_gap]#2 + roadmap P1 entry → plan `docs/plans/emit-ir-staticlib.md` (`03493c9`) → iter this commit. Scoped feature, no audit/fieldtest gate (the E2E is the coverage; no authoring-surface or invariant change — reuses the M1-audited codegen path). → 2026-05-18-iter-emit-ir-staticlib.md -- 2026-05-18 — iter embedding-abi-m2.1 (Embedding ABI — M2, DONE 9/9 across one Boss-repaired split dispatch): per-thread runtime context + concurrency safety. No authoring-surface change (the `.ail` is byte-identical to M1); the deliverable is a swarm-safe generated C ABI + de-globalised runtime, sanitiser-verified. Shipped: `runtime/rc.c` gains `ailang_ctx_t {alloc_count,free_count}` + `ailang_ctx_new`/`_free` + `__thread __ail_tls_ctx`, the two increment sites become `if (_ctx) _ctx->… else g_rc_…` (the `g_rc_*` statics + `ailang_rc_stats_atexit` + constructor RETAINED VERBATIM as the null-ctx single-threaded executable fallback — `print_no_leak_pin`/`e2e` rc_stats stay green unmodified); codegen `Target::StaticLib` forwarder gains a mandatory leading `ptr %ctx` + one `@__ail_tls_ctx = external thread_local global ptr` decl + TLS save/store/restore around the BYTE-UNCHANGED internal `@ail__` call (`_adapter`/`_clos` + internal arg vector untouched — M1 decision held; the frozen-at-M3 surface is the C `@(ctx,…)` signature, internal convention free); `build_staticlib` rejects `--alloc != rc` (RC-only swarm artefact, no shared Boehm collector by construction; default already rc; P2 Boehm-retirement untouched); DESIGN.md §"Embedding ABI (M1)" updated to post-M2 current state (provisional-until-M3 narrowed to the value/record layout; Decision-10 per-thread-ctx atomicity note) with the docs_honesty-pinned "Export parameters are written **bare**…" sentence + `(con Int)` snippet preserved verbatim; M1 `host.c`/`embed_e2e` migrated to the ctx ABI (`s==25` holds). Coherent stop sanitiser-verified by the Boss directly (not inferred): per-thread-ctx scalar swarm `-fsanitize=thread`-clean with every thread == serial oracle (capability demo); direct rc-accounting per-ctx exit 0 / 0 tsan warnings (de-globalisation positive); shared-ctx negative control a genuine `ThreadSanitizer: data race` at `rc.c:157` (`_ctx->alloc_count++` in `ailang_rc_alloc`) + `rc.c:208` (`_ctx->free_count++` in `ailang_rc_dec`), exit 66 — teeth non-vacuous, precisely targeting the de-globalised counters. Process note: the first dispatch (Tasks 1–4 clean, committed `c9a84b3` as the known-good subset) correctly BLOCKED on Task 5 having surfaced a genuine spec defect — the original Task 5 paired a `-DSHARED_CTX` negative control onto `swarm.c`, structurally impossible because a non-allocating scalar kernel writes no ctx field and `__ail_tls_ctx` is `__thread` (the spec's OWN item-1 honesty point). Boss adjudication (not a brainstorm bounce — a consistency-repair of an internally-contradictory clause whose intent is already realised by item 1): spec amended in lockstep (Goal coherent-stop, must-fail-axis item 2, Testing item 3, Acceptance) so the negative-control teeth are item-1's by the de-globalisation-proof / capability-demo split; plan Task 5 restructured (`swarm.c` per-ctx capability demo only; negative control relocated into the item-1 `rc_accounting` harness driver) + Task-3 plan-transcription error corrected (`@ail_backtest_step`→`@ail_embed_backtest_step_step`); re-dispatch `[5,9]` DONE. spec `docs/specs/2026-05-18-embedding-abi-m2.md` (user-approved, grounding-check PASS 9/9, `1c58055`) → plan `docs/plans/embedding-abi-m2.1.md` (`b3388c8`, Boss-corrected) → iter `c9a84b3` (PARTIAL 4/9 + Boss repair) + this commit (DONE 5–9). Known debt routed: DESIGN.md `## Embedding ABI (M1)` section header still says "(M1)" while the body now mirrors M2 — out of Task 7's scope-limited edit regions by design (preserve the docs_honesty_pin), a post-audit doc-honesty tidy candidate. Milestone-close `audit` next. → 2026-05-18-iter-embedding-abi-m2.1.md -- 2026-05-18 — audit embedding-abi-m2 (milestone close — DRIFT: one [medium]+[low] doc-honesty item → tidy; bench exit 1 causally exonerated, NO ratify): architect `drift_found` — no DESIGN.md/spec-contract drift, Invariant 1 intact, internal-convention byte-invariant, Decision-10 atomicity note present-tense-correct (sanitiser-verified), lockstep complete, docs_honesty_pin green, no unrequested extras, Boss spec-defect repair correctly reflected; the ONE actionable item is `[medium]` DESIGN.md:2266 — section header still `## Embedding ABI (M1)` while the body now mirrors the post-M2 ctx ABI present-tense (current-state-mirror honesty mismatch; NOT a Sweep-5 catch — header/body desync only a read finds; iter-journal Known-debt flagged it truthfully, Task-7-scope-deferred to preserve the docs_honesty_pin:135 verbatim sentence) + coupled `[low]` DESIGN.md:2354 stale `see §"Embedding ABI (M1)"` xref. Bench: `check.py` exit 1 (3 regressed: bench_list_sum.bump_s +13.57%, latency.explicit_at_rc.max_us +81.96%, latency.implicit_at_rc.max_us +38.57%), compile_check/cross_lang exit 0 clean (every rc_over_c ratio within tol). M2 genuinely changed the benched runtime path (rc.c null-ctx-fallback branch), so byte-identical exoneration did NOT apply → `ailang-bencher` dispatched: swapped only rc.c pre-M2↔HEAD, disassembled the delta (3 branchless insns mov %fs/test/cmovne), interleaved core-pinned A/B — **H0 refuted, all 3 firings tracked-noise causally-exonerated**: bump_s causally impossible from M2 (bump_malloc 0 rc-counter refs; reproduces ~+12% on M2-inert code = P2 *.bump_s stale-baseline), the rc.c hunk compiled branchless + measured free (HEAD −0.63% median), max_us median/p99 flat across rc.c variants + non-reproducible within a fixed binary (3.2× spread) = P2 *.max_us -n5 structural false-positive (4th consecutive null-attributable firing). NO baseline ratified (not M2 artefacts; P2 items stay separately tracked). Resolution: `[medium]`+`[low]` → tidy iter embedding-abi-m2.tidy (header rename + :2354 xref + docs_honesty_pin.rs:134–136 (M1)-token lockstep; the pinned :135 substring stays verbatim); bench → carry-on. Milestone substantively closed + sound (capability sanitiser-verified, all contract invariants held); roadmap [~]→[x] follows the tidy landing clean. → 2026-05-18-audit-embedding-abi-m2.md -- 2026-05-18 — iter embedding-abi-m2.tidy (M2 audit `[medium]`+`[low]` doc-honesty fix, DONE 1/1, zero re-loops): lockstep-renamed the DESIGN.md section `## Embedding ABI (M1)` → `## Embedding ABI` (current-state-mirror — the body had described the post-M2 ctx ABI present-tense while the header still carried the milestone tag) across all 5 live-coupling sites: `docs/DESIGN.md:2266` (header) + `:2354` (schema-block `see §"Embedding ABI"` xref), `crates/ailang-core/tests/docs_honesty_pin.rs:134` (comment) + `:136` (assert-message), and the Boss-added 5th site `crates/ailang-core/specs/form_a.md:89` (a live forward-xref into the renamed section in the compiled-in `FORM_A_SPEC` — same dangling-xref class as :2354, surfaced by the plan-recon-undercount countermeasure, NOT in the architect's named 3-site scope; Boss folded it in on the merits because leaving it stale is a half-rename contradicting the tidy's own thesis). The `docs_honesty_pin.rs:135` asserted substring (the DESIGN.md body sentence "Export parameters are written **bare**…", 31 lines below the header, section-name-free, file-wide `d.contains`) is rename-immune and stayed byte-verbatim untouched — recon proved this definitively pre-plan; the diff confirms :135 is unchanged context between the :134/:136 hunks. Committed append-only history (journals/specs/plans/orchestrator-stats) NOT falsified — Boss-verified the diff footprint is exactly the 3 content files + journal + stats, zero history files. Pure docs/test-string tidy, zero language/checker/codegen/schema change; no audit/fieldtest gate (the docs_honesty_pin IS the regression guard — Boss-verified 5/0 before AND after, design_schema_drift 8/0, the :2354 edit moved no JSON-schema drift anchor). Recon also corrected an audit inaccuracy (the :2354 xref IS inside the `## Data model` window :2319–2558, not outside — non-blocking, the xref string is not a drift anchor). audit source `docs/journals/2026-05-18-audit-embedding-abi-m2.md` (`ad88dec`) → plan `docs/plans/embedding-abi-m2.tidy.md` (`d1241b1`) → iter this commit. → 2026-05-18-iter-embedding-abi-m2.tidy.md -- 2026-05-18 — iter embedding-abi-m3.1 (Embedding ABI — M3, DONE 7/7 across one Boss-repaired split dispatch): freeze the value layout + a single-constructor record of `Int`/`Float` fields crosses the C ABI in and out, ownership following the declared `own`/`borrow` mode. No authoring-surface / schema / Form-A / `CheckError` change (M2-style — the `.ail` is the minimal evolution of the M1/M2 scalar kernel into a record `State`, existing `data`+`(own (con T))`+`(export …)`). Shipped: the export-gate scalar predicate `is_c_scalar` widened to a two-level `is_c_abi_type` (a bare `Int`/`Float`, OR a `Type::Con` resolving via the in-scope `env.types` to a single-constructor `data` whose every field is a bare `Int`/`Float` — NOT deep: a record-typed field stays rejected = M4; multi-ctor sums / `Str` / `List` / effectful stay RED; the M1 `embed_export_adt_ret_rejected.ail` must-fail RE-POINTED from the now-M3-valid single-ctor `Pair` to a genuinely-still-rejected multi-ctor+`Str` `Reading`, the pin's meaning sharpened not silently inverted; gate suite 10/10); codegen `llvm_scalar` maps a gate-guaranteed record `Type::Con` → `ptr` (the M2 `Target::StaticLib` forwarder body BYTE-UNCHANGED — a record crosses as a bare `ptr`, no `sret`/`byval`; all mode-driven RC stays in the byte-unchanged internal `@ail__`; 3/3 staticlib-lowering pins, no `fn_scalar_sig` signature change so no caller ripple); the record memory layout FROZEN as a one-way commitment — DESIGN.md §"Embedding ABI" gains a `### Frozen value layout (M3 — one-way commitment)` SSOT subsection (header@p-8 / i64 tag@0 / fields i64-strided@8+i*8 / size=8+n*8 / `ailang_rc_alloc` construction MUST / `ailang_rc_dec` host-free leak-free because scalar-field / ownership follows the declared mode), the two "provisional until M3" sentences rewritten to one-way-freeze wording, lockstep `// FROZEN ABI` pointers added at the three independent encoders (`runtime/rc.c`, `match_lower.rs` `lower_ctor`, `drop.rs`), 3 stale rustdocs fixed (`codegen/lib.rs`, `core/ast.rs`, the Concerns-carry `check/lib.rs:448-450` variant doc) + the Boss-caught stale `//` gate-comment block (`check/lib.rs:1917-1922`, "M1/scalar-only" → "M1/M2/M3 / C-ABI-permitted incl. single-ctor record"); a new `@ailang_rc_alloc` heap-box byte-pin (`embed_record_layout_pin.rs`) proven RED-on-offset-perturbation / GREEN-on-restore (the freeze made enforceable, not aspirational); the `(State,Float)->State` record fold E2E round-trip proven globally leak-free for BOTH `own` and `borrow`. Process note: the first dispatch (Tasks 1–4 clean) correctly BLOCKED on Task 5 having surfaced a genuine spec defect — the spec's proof instrument asserted the single `ailang_ctx_free` ctx readback shows `allocs==frees`, structurally unsatisfiable for `borrow` (and only coincidentally passing for `own`) because M2's `__ail_tls_ctx` is bound only for the synchronous forwarder call, so host-side `ailang_rc_dec`s land on `g_rc_*` not ctx. Boss adjudication (M2.1-precedent class — genuine spec defect, deliverable+invariant sound, only verification mechanics mis-specified, first BLOCKED task not 2+-in-family ⇒ consistency-repair not a brainstorm bounce): Boss independently rebuilt+ran both modes and confirmed globally leak-free + value-correct (own ctx live=0/g_rc live=0; borrow ctx live=+N/g_rc live=−N, Σlive=0; exit 0 both); spec amended (dated repair note + §"Testing strategy" 3/4 + §"Coherent stop" → global-leak-freedom proof model, the M2-TLS cross-attribution documented as correct behaviour) + plan amended (harness sums ALL `ailang_rc_stats:` lines) + the mechanical code repair applied; partial committed `d5c565d` (PARTIAL 5/7 + Boss spec-defect repair); re-dispatch `task_range:[6,7]` DONE. No fresh grounding-check (the repair removes an over-strong measurement assumption, adds none about compiler behaviour). Independent Boss verification: workspace 639/0, `embed_record_e2e` 2/2 (own+borrow global model), byte-pin 1/1, gate 10/10, forwarder 3/3, `docs_honesty_pin` 5/5 pin-safe (the freeze rewrite kept the pinned "Export parameters are written **bare**…" sentence byte-verbatim+contiguous, shifted :2297→:2299, content-asserted), `design_schema_drift`/`embed_export_hash_stable`/`embed_e2e`/M2 swarm regression-green-unmodified. spec `docs/specs/2026-05-18-embedding-abi-m3.md` (user-approved, grounding-check PASS 8/8, `1fbb9c4`) → plan `docs/plans/embedding-abi-m3.1.md` (`15ee3c5`, Boss-amended) → iter `d5c565d` (PARTIAL 5/7 + Boss repair) + this commit (DONE 6–7). Milestone-close `audit` next (architect Invariant 1 + bench trio — spec Testing items 8/9, explicitly NOT the implement run's job). → 2026-05-18-iter-embedding-abi-m3.1.md -- 2026-05-18 — audit embedding-abi-m3 (Embedding ABI — M3 milestone close — DRIFT: one [medium]+[low] doc-honesty → one tidy; bench check.py exit 1 decisively causally exonerated, NO ratify): architect `drift_found` — Invariant 1 clean (zero Cargo/runtime dep change; core/codegen/runtime touches only the spec-scoped widen + lockstep comments + pins; no data-server/finance knowledge), the M1/M2 internal-convention byte-invariant held (M2 forwarder body + @ail__/_adapter/_clos absent from the diff — the spec's load-bearing "transparent ptr pass-through" claim verified), the frozen-layout SSOT (DESIGN.md `### Frozen value layout` :2321-2359) present-tense + matching `match_lower.rs lower_ctor` (size=8+n*8/tag@0/fields@8+i*8) AND the enforcing byte-pin (@ailang_rc_alloc(i64 24)/tag@0/@8/@16), all three // FROZEN ABI lockstep pointers aimed at the SSOT, the Boss spec-consistency repair note + Testing 3/4 + journal Boss-adjudication/Re-dispatch mutually consistent and honest about the M2-TLS cross-attribution (correct behaviour, not a leak), no new CheckError/schema/Form-A (M2-style honoured). Two residual doc-honesty drift items, both milestone-owned: `[medium]` DESIGN.md:2299-2302 — a surviving M1-era paragraph asserts modes "apply only to heap-shaped types, which the **scalar-only rule above forbids** at an export boundary anyway", now self-contradicting the section's own freeze (`:2280-2284` was rewritten to accept a single-ctor record — there is no scalar-only rule) and the (own/borrow (con State)) mode contract at :2345-2351 → fix (docs-honesty tidy); `[low]` crates/ailang-codegen/src/lib.rs:608-610 — forwarder-body comment still says the gate "guarantees Int/Float; map Int→i64 Float→double", now false (records→ptr), classified by the architect "carried debt, not a scope miss" (the body is contractually byte-frozen so the comment was outside any M3 edit region; comment-only, no emitted-IR change) → fix, folded into the same tidy. (The Boss had already caught+fixed one sibling stale `//` gate-comment block at check/lib.rs:1917-1922 during the DONE inspect; architect found no further siblings.) Bench: check.py exit 1 (3 regressed: throughput.bench_list_sum.bump_s +12.62%/tol10; latency.implicit_at_rc.p99_9_us +28.45%/tol25; .max_us +35.18%/tol30), compile_check exit 0 (24/0), cross_lang exit 0 (25/0, every rc_over_c within tol). The 3 firings are the two standing tracked P2 noise families (the *.bump_s ~+5-13% stale-baseline + the *.max_us/p99_9 -n5 tail-jitter structural false-positive — signature: same bench median +5.11% ok / p99 +15.08% ok, only the extreme tail quantiles fire). DECISIVE causal exoneration (M2-audit method, not hand-waved): M3 changed no executable-path codegen (the llvm_scalar record→ptr arm is reachable only for an (export) single-ctor record; no bench_* program has an (export)), so built `ail` at HEAD and at pre-M3 9a609ae in an isolated worktree and cmp'd the GENERATED IR — byte-identical for bench_list_sum, bench_list_sum_explicit, bench_latency_implicit, bench_tree_walk; the IR is what is handed to deterministic `clang -O2`, so the bench binaries are byte-identical and M3 causation is logically impossible. NO baseline ratify (not M3 artefacts; the two P2 todos stay separately tracked). Resolution: `[medium]`+`[low]` → one docs-honesty tidy iter embedding-abi-m3.tidy (pin-safe re docs_honesty_pin.rs:135, planner Step-5 item-6 / M2.tidy precedent), routed planner→implement; bench → carry-on. Milestone substantively closed + sound; roadmap [~]→[x] follows the tidy landing clean. → 2026-05-18-audit-embedding-abi-m3.md -- 2026-05-18 — iter embedding-abi-m3.tidy (M3 audit [medium]+[low] doc-honesty fix, DONE 3/3, pin-safe): closed the two DRIFT items the M3 milestone-close audit routed here (M2.tidy `[medium]+[low] doc-honesty → tidy` precedent). [medium] docs/DESIGN.md §"Embedding ABI" — surgically replaced ONLY the contradicted M1-era parenthetical "(modes apply only to heap-shaped types, which the scalar-only rule above forbids at an export boundary anyway)" with the present-tense truth "(a single-constructor record export parameter, by contrast, carries `own`/`borrow` — the ownership contract the frozen value layout below specifies)"; the parenthetical shared physical line :2300 with the docs_honesty_pin.rs:135 pinned bare-scalar sentence ("Export parameters are written **bare**: a scalar type carries no `own`/`borrow` mode", norm()-whitespace-collapsed, fn form_a_scalar_param_carveout_present_and_old_rule_absent) — the edit kept every pinned word (line :2299 + the `own`/`borrow` mode` continuation untouched), the planner Step-5 item-6 presence-pin-vs-verbatim-edit collision the M2.tidy precedent ran. [low] crates/ailang-codegen/src/lib.rs:608-610 — comment-only honesty fix ("gate guarantees Int/Float; map Int→i64,Float→double" → "...Int/Float or a single-constructor record of those (M3); map Int→i64, Float→double, a record → ptr"); tree-wide-grep-confirmed no test pins the comment text; the byte-pin (embed_record_layout_pin) + forwarder-IR pin (embed_staticlib_lowering) assert generated IR not source comments → byte-identical before/after (the guard that no codegen moved). Boss-verified independently: both stale fragments grep-ABSENT; the 4 standing pins green at the exact recon baseline (docs_honesty_pin 5/0 ⇒ pin-safety held, design_schema_drift 8/0, embed_record_layout_pin 1/0, embed_staticlib_lowering 3/0); workspace 639/77 byte-unchanged from the M3-DONE baseline (docs/comment tidy, zero behaviour/test delta); diff exactly 2 files (DESIGN.md 5±, codegen/src/lib.rs 7±). No language/checker/codegen behaviour change; no audit/fieldtest gate (the 4 pins + 639/77 ARE the regression coverage, M2.tidy precedent). One non-gating planner-quality defect recorded in Concerns: Task-3 Step-1's `guarantees every param` verification grep is a substring of the plan's own Task-2 replacement text (non-discriminating) — the orchestrator correctly verified the substantive intent via discriminating fragments instead of bending code (same family as planner Step-5 item-8). bench: already carry-on / NO ratify at the M3 audit (causally exonerated by byte-identical generated IR; this tidy touches no executable path). audit source docs/journals/2026-05-18-audit-embedding-abi-m3.md (b8a60b1) → plan docs/plans/embedding-abi-m3.tidy.md (44ced51) → iter this commit. M3 milestone substantively closed + sound; roadmap [~]→[x] follows. → 2026-05-18-iter-embedding-abi-m3.tidy.md -- 2026-05-18 — brainstorm embedding-abi-m4 → RETIRED, never speced (premise collapsed under its own feature-acceptance gate during Step-2/3 Q&A; no spec, no grounding-check, no planner handoff — the "problem mis-framed → don't ratify a known-unneeded shape" brainstorm path): `/boss` picked top-P0 "Embedding ABI — M4: sequence crossing via `List`"; user green-lit a continue-here brainstorm; recon (`ailang-plan-recon`) returned a full fact sheet; two user forks resolved in Q&A (own-only `List` param; `List Record`-only element) and Approach A (structural list-shaped `is_c_abi_type` arm, no name-match; B name-anchored / C `std_list`-SSOT-first rejected on language/scope grounds) recommended — all now moot. Struck on **feature-acceptance clause 2**: the shipped M3 gate `is_c_abi_type` (`crates/ailang-check/src/lib.rs:1934-1953`) is a per-parameter loop accepting a C scalar OR a single-ctor all-scalar record *independently per param*, and the forwarder's `llvm_scalar` maps every non-scalar `Type::Con`→`ptr` (M3-frozen), so `(State, Tick) -> State` (both single-ctor all-scalar records) is **already gate-accepted + forwarder-supported today** — the minimal data-server binding is M3 (shipped) + a host-side per-tick loop; cons-list crossing would *add* a 2N+1-box-per-chunk host builder + the deferred flat-array perf debt and removes no redundancy, with no named consumer (M5's adapter unrolls each chunk host-side — a clean adapter, the sole data-server↔AILang meeting point per Invariant 1; whole-chunk in-kernel visibility is semantically void since `State` threads across calls regardless of chunk boundaries). Honest mid-Q&A correction recorded: I had asserted "M5 cannot wire data-server without M4" — false (M5 wires it on M3 + `for tick in chunk`); re-deriving against the code rather than defending the roadmap I wrote is what surfaced the clause-2 failure ("user suggestions ≠ directives, form own judgment"). Outcome: M4 retired in `docs/roadmap.md` (struck entry kept one cycle, never `[x]`); M5 reconciled (`depends on:` M4→M3 + Tick-coverage todo; adapter unrolls host-side; friction feeds the host-per-tick-FFI-vs-batch P2 perf decision); residual = a new `[todo]` "Tick-coverage on M3" (E2E+fixture pinning the two-record-param per-tick `(State, Tick) -> State` shape — capability present today but only E2E-proven for a single record param `State`; every shipped M3 fixture pushes a scalar `Float` sample, none a record `Tick`; test backfill, no brainstorm — the actual "minimal data-server binding"). Forward note: the P2 flat-array item's "1024 cons-cells/chunk" framing is now partly stale (cons-list path dropped) — reconcile when picked up, not now. → 2026-05-18-brainstorm-embedding-abi-m4-retired.md -- 2026-05-18 — iter bugfix-over-strict-mode-ctor-rebuild-consume (RED→GREEN, debug→implement mini, DONE 1/1): fixed a conservative `[over-strict-mode]` false-positive surfaced by the Tick-coverage fixtures. The lint's consume-detection (`any_sub_binder_consumed_for`/`pattern_has_consumed_heap_binder`, `crates/ailang-check/src/linearity.rs`) only recognised a consume of an `(own (con T))` param when a *heap-typed* pattern-binder was moved out of `match p`; when `p` was destructured into purely *primitive* fields fed into a `Term::Ctor` rebuilding `p`'s own ctor, that genuine dismantle+rebuild consume was invisible, so the lint spuriously advised `(borrow ...)`. Real harm: an LLM author "fixing" the spurious warning by flipping an export's declared mode `own`→`borrow` would silently invert the ABI ownership contract. RED-first: debugger disproved the carrier's initial nested-`match` hypothesis (the M3 `embed_backtest_step_record.ail` is silent only because its implicit-mode scalar `Float` param disables the lint via the activation gate, linearity.rs:327 — NOT because it handles the rebuild; the defect reproduces single-param, no nesting), wrote the synthetic RED unit `over_strict_mode_silent_when_ctor_rebuilt_from_primitive_fields` committed as its own audit-trail commit `a11cb7c`. GREEN (implement mini): added a 2nd recognition path to `any_sub_binder_consumed_for` — a `match p` arm that destructures binders out of `p`'s ctor and references any of them (primitive or not) inside a `Term::Ctor`'s args in the arm body genuinely consumes `p`; two pure helpers (`ctor_uses_any_binder` + deep `term_mentions_any_binder`, so `(+ acc px)`-mediated flow counts), conservative toward NOT suppressing (a ctor ignoring `p`'s payload still warns — negative-control proven), over-strict-only (by-name shadowing imprecision is extra-silence, never under-strict; recorded as Known debt). Both stale doc comments that mis-attributed the FP to nested `match` corrected for doc-honesty (debugger concern #2, same code region — in-scope, not opportunistic). +166/−20 in linearity.rs only; check-only, zero codegen/runtime/ABI/schema/DESIGN.md change. Boss-verified independently: RED→GREEN, full `cargo test -p ailang-check` 108/0 lib + every binary 0-failed with NO existing test modified, `ail check embed_backtest_step_tick.ail` no longer over-strict on `st`/`tick` (exit 0), `_tick_borrow.ail` + M3 `embed_backtest_step_record.ail` still clean, the already-green `embed_tick_e2e` + bench posture untouched. No audit/fieldtest gate (lint-precision bugfix; the RED test + the green check-suite ARE the regression coverage). RED `a11cb7c` → GREEN this commit. → 2026-05-18-iter-bugfix-over-strict-mode-ctor-rebuild-consume.md -- 2026-05-19 — iter embedding-abi-m5.1 (DONE 3/3): M5 iter 1 — stood up the workspace-excluded `ail-embed` crate. A zero-dependency embedding core (Rust port of the audited `crates/ail/tests/embed/tick_roundtrip.c`: `extern "C"` to the M3-frozen ABI + frozen-layout `State`/`Tick` box helpers + a `Kernel` price fold, raw pointers never escaping the type) links the M3 staticlib via a new `build.rs` (no in-repo precedent — `AIL_BIN` env override else nested `cargo build -p ail` against the parent workspace, separate target dir so no cargo-lock deadlock). Plus a hermetic `data-server` smoke: a synthetic Pepperstone-format ZIP fixture written via the crate's own public `RawTickRecord` type (correct-by-construction, mirrors `data-server/src/loader.rs:110-124`) → real `DataServer` → `Kernel`, asserting bit-exact vs a same-order host reference fold and the closed-form `55.0`; runs with no `/mnt`. `ail-embed` is its own cargo workspace root (empty `[workspace]` table) so the AILang compiler workspace owes it nothing; `data-server` is a dev-dependency only ⇒ Invariant 1 holds in the dependency graph, not just on paper. Boss-verified independently: ail-embed suite 2/2 green (`kernel_run_sums_prices` unit RED-first + `hermetic_smoke_data_server_roundtrip` integration), full+no-deps `cargo metadata` on the AILang workspace shows `data-server` count 0, `git status` path filter empty (zero diff to `crates/ailang-*`/`crates/ail/`/`runtime/`/`examples/*.ail`), `src/lib.rs` zero code-level `data_server`, AILang `cargo build --workspace` still clean. Two toolchain-forced corrections to the plan's verbatim `ail-embed/Cargo.toml` (added the empty `[workspace]` table; sibling dev-dep path `../libs`→`../../libs` manifest-relative) — confined to the plan-created manifest, no acceptance gate altered, 0 review re-loops; planner-recon implication recorded in the journal Concerns. Adapter API + thread-swarm explicitly deferred to M5 iter 2+. → 2026-05-19-iter-embedding-abi-m5.1.md -- 2026-05-19 — iter embedding-abi-m5.2 (PARTIAL 2/3; M5 bounce-back): data-server adapter + symbol-fan swarm; the leak-proof did its job and surfaced a real concurrency finding. Tasks 1+2 landed clean: `data-server` promoted dev-dep→real dep (Invariant-1 sanctioned — compiler workspace graph still data-server-count 0, zero compiler-surface diff), additive `adapter` module (`tick_to_px`/`MidPriceStream` lazy `Iterator`/`fold_symbol`, RED-first), and a `swarm_runner` `[[bin]]` whose clean compile *is* the compile-time per-thread-ctx proof (`Ctx: !Send` ⇒ a shared-ctx swarm is `E0277`, cannot build). The real-data symbol-fan swarm runs (EURUSD/GER40/XAUUSD, ~4s) and is **bit-exact vs an independent same-order host reference every run** (`symbol_fan_swarm_bit_exact`, GREEN, live). Task 3's `Σallocs==Σfrees` leak gate is BLOCKED: Boss independently read `runtime/rc.c` and confirms it is an **instrumentation race, not a memory bug** — no box crosses a thread (`!Send`), the real RC is correct (bit-exact green), only the *global* `g_rc_*` stat counters (rc.c:90-91,161,212) are non-atomic *by rc.c's own documented single-threaded design* and lose `++`s when 3 worker threads hit the null-`__ail_tls_ctx` host-side path. M5 is AILang's first concurrent consumer → the deferred "concurrency arrived: atomic-vs-non-atomic" decision rc.c:44-49 names is now due. The leak assertion was Boss-split into `symbol_fan_swarm_leak_free` (`#[ignore]`, **body verbatim** — quarantined not weakened; un-ignore = the runtime fix's acceptance) so main stays green and the finding is pinned. Prior claim that `embed_swarm_tsan.rs` covers this was corrected (it uses the scalar kernel — zero box allocs — never exercised this path). Escalated as a **bounce-back** (touches M3-frozen `runtime/`; re-frames M5's "zero runtime change"; multiple substantive options); Boss recommendation on the record = Option A (atomic *global-fallback* counters only, standalone RED-first runtime micro-iter, M5 framing amended) — user picks the direction. Time-shard + friction-harvest remain m5.3. → 2026-05-19-iter-embedding-abi-m5.2.md -- 2026-05-19 — iter bugfix-rc-global-stats-race (RED→GREEN, debug→implement mini, DONE 1/1): the M5 iter-2 bounce-back resolution (user-approved Option A). The two null-ctx fallback RC-stats counters `g_rc_alloc_count`/`g_rc_free_count` (`runtime/rc.c:90-91`) were plain `static uint64_t` with a non-atomic `++` on the `__ail_tls_ctx == NULL` arm of `ailang_rc_alloc` (rc.c:161) / the to-zero branch of `ailang_rc_dec` (rc.c:212); a multi-threaded host driving the global fallback (no `ailang_ctx_new`) raced the read-modify-write and lost increments, so the `AILANG_RC_STATS` atexit Σ under-counted non-deterministically — exactly the deferred "concurrency arrived: atomic-vs-non-atomic" decision rc.c's own header (rc.c:44-49) names, M5 being AILang's first concurrent consumer. NOT a memory bug (`Ctx: !Send` keeps every box on one thread; the real refcount/free is correct; programs bit-exact). RED (debugger, audit-trail commit `427b687`): a C host — 8 threads × 2M alloc-then-dec, no ctx so the global path is taken, no box crossing a thread — + integration test asserting the atexit Σ is exact; Boss-verified RED `allocs=2141382 expected 16000000, live=-131242` (deterministic-fail under that contention, not flaky). GREEN (implement mini, this commit): the two globals are now `_Atomic uint64_t`, `atomic_fetch_add_explicit(.., memory_order_relaxed)` at the two fallback `++` sites, `atomic_load_explicit(.., relaxed)` in `ailang_rc_stats_atexit` (its sole reader) — relaxed is correct (pure stats, no happens-before; atexit reader runs post-join). Per-ctx counters + the per-object refcount header left non-atomic BY DESIGN (single-thread-per-ctx; boxes never cross threads) — explicitly out of scope, untouched; frozen value layout / ABI offsets / host-free rule untouched; the genuinely-still-single-threaded drop-worklist doc note correctly left unchanged (a false correction was refused); the two genuinely-stale atomicity doc blocks corrected for honesty. Scope held to `runtime/rc.c` ONLY (37+/14−). Boss-verified independently: RED→GREEN deterministically 3/3 (jitter gone, `allocs==frees==16_000_000`), full `cargo test -p ail` green (every binary 0 failed — the per-ctx tsan harnesses embed_swarm_tsan/embed_rc_accounting_tsan + embed_tick_e2e/embed_record_e2e unaffected since the per-ctx path is untouched; rc.c links into every ail binary so this is the strong regression gate), git-status scope = rc.c + journal + stats only, the RED files unchanged. Unblocks resuming the M5 leak-proof (un-`#[ignore]` `ail-embed`'s `symbol_fan_swarm_leak_free`). RED `427b687` → GREEN this commit. → 2026-05-19-iter-bugfix-rc-global-stats-race.md -- 2026-05-19 — iter embedding-abi-m5.2-resume-attempt (RE-QUARANTINED; honesty fix only): attempted to un-`#[ignore]` the M5 swarm leak-proof `symbol_fan_swarm_leak_free` as the integration-level acceptance of the runtime fix `7bfa11e` (bugfix-rc-global-stats-race). Boss verification FALSIFIED the sufficiency premise: across 4–6 swarm runs the leak Σ is still non-deterministic — `Σfrees` stable-exact (12000003) every run, `Σallocs` short by a jittering ~1600–2260 in ~1/3 of runs (e.g. left:11998383 right:12000003 across 4 stat lines). The atomic-global fix is real, committed, NECESSARY (the isolated 8×2M pure-global RED `embed_rc_global_stats_race` is deterministically green) but NOT sufficient — a second, distinct alloc-side undercount remains that the isolated RED did not model (frees never lose ⇒ not a per-ctx race; not pure global contention ⇒ RED green; localisation beyond that is where Boss speculation stops per the debug Iron Law). Un-ignore reverted (never committed); the only shipped diff is the three now-stale `#[ignore]`-rationale breadcrumbs in `ail-embed/tests/swarm.rs` corrected from "un-ignore when the runtime is atomic" to the accurate necessary-but-insufficient state (a future agent seeing `7bfa11e` must NOT un-ignore on that basis — doc-honesty). Test body + the `#[ignore]` itself unchanged (still quarantined, NOT weakened); `symbol_fan_swarm_bit_exact` stays live+GREEN (the swarm IS actually correct and leak-free — only the accounting is wrong). Residual handed to a fresh debug RED-first cycle; two framings flagged for it (genuine residual runtime defect vs Σ-over-heterogeneous-global+per-ctx-lines methodology unsoundness — the latter, if so, is a post-root-cause design decision). main green at the honesty-fix commit; M5 stays open `[~]`; m5.3 remains downstream of a sound leak-proof. → 2026-05-19-iter-embedding-abi-m5.2-resume-attempt.md -- 2026-05-19 — iter bugfix-swarm-rc-alloc-undercount (RED→GREEN, debug→implement mini, DONE 1/1): root-caused and fixed the m5.2-resume-attempt residual. The M5 swarm leak-proof was still non-deterministic after the atomic-counter runtime fix `7bfa11e` — NOT a second runtime bug and NOT test-methodology unsoundness, but a **build-dependency staleness gap**: `ail-embed/build.rs` declared `cargo:rerun-if-changed` only for the kernel `.ail` + `build.rs`, NOT the `runtime/` C sources `ail build --emit=staticlib` compiles into `libailang_rt.a`. So Cargo never re-ran `build.rs` after `7bfa11e`; `ail-embed` kept linking a stale pre-`7bfa11e` non-atomic `libailang_rt.a` whose `g_rc_alloc_count++` raced the swarm's concurrent TLS-NULL host allocs (explaining the frees-stable/allocs-jitter asymmetry, and why the isolated `crates/ail` RED — which rebuilds the staticlib fresh per run — was green while the `ail-embed` swarm was not). `runtime/rc.c` at HEAD was always correct; the entire production fix is one directory-level `cargo:rerun-if-changed=/runtime` line (preferred over a per-file list — the implementer inspected `crates/ail` `build_staticlib` confirming `libailang_rt.a = ar(rc.o,str.o)`, both under `runtime/`; directory tracking is future-proof against any new runtime source). RED (debugger, audit-trail `483117d`): `ail-embed/tests/rt_archive_freshness.rs` pins the *cause* deterministically (objdump the linked archive for `lock`-prefixed atomic increments + a fresh-build cross-check) since the *symptom* is non-deterministic by construction; Boss-verified RED = 0 lock insns ("STALE"). GREEN (implement mini, this commit): the build.rs fix + the cohesive consequence — `symbol_fan_swarm_leak_free` un-`#[ignore]`d (body byte-for-byte verbatim — it was always quarantined, never weakened; the three doc breadcrumbs rewritten to the resolved build-dep-staleness rationale), now the integration-level acceptance of both `7bfa11e` and this fix. Boss removed a pre-existing dead `DataFormat` import in swarm.rs inline (trivial; file committed this iter regardless). Boss-verified independently: RED→GREEN; **swarm determinism 5/5 — `symbol_fan_swarm_leak_free` + `symbol_fan_swarm_bit_exact` GREEN every run, 0 ignored, jitter gone, Σallocs==Σfrees==12000003**; isolated `embed_rc_global_stats_race` still GREEN (untouched); Invariant 1 data-server count 0; scope = `ail-embed/build.rs` + `ail-embed/tests/swarm.rs` + journal + stats only. The M5 swarm leak-proof bounce-back is now fully resolved end-to-end (bounce-back → Option A `7bfa11e` → resume-attempt "necessary-but-insufficient" → this build-dep root-cause+fix). RED `483117d` → GREEN this commit. M5 stays open `[~]`; m5.3 (time-shard + friction-harvest + close-out) remains. → 2026-05-19-iter-bugfix-swarm-rc-alloc-undercount.md -- 2026-05-19 — iter embedding-abi-m5.3 (DONE 3/3): final functional M5 iteration — the time-shard boundary-invisibility proof + friction harvest. One symbol (EURUSD), three disjoint contiguous single-month windows (2017-03/04/05), each folded on its own thread owning its own `Kernel` (`Ctx: !Send` ⇒ one-ctx-per-thread compile-time-enforced). Adds the purely-additive windowed adapter sibling `fold_window` (same `MidPriceStream`+`Kernel` path as `fold_symbol`, differs only by `stream_tick_windowed`), a new single-responsibility `[[bin]] timeshard_runner`, `chrono` as a **dev-dep** (the time-shard test derives `(y,m)→Unix-ms` bounds via data-server's own `NaiveDate…timestamp_millis()` precedent and passes raw ms to the bin, keeping the bin date-math-free and Invariant 1 untouched), and `tests/timeshard.rs` carrying the spec §3 strength-ordered assertions: (a) each shard's `(acc,n)` **bit-exact** vs a single-thread host fold of THAT EXACT window in data-server stream order — the first actual proof of the chunk/window-boundary-invisibility claim the entire M4 retirement rests on; (b) host-summed partials bit-exact by construction; (c) the whole-window [2017-03..2017-05] single-thread total within `REL_TOL=1e-6` only (cross-shard f64 re-association is host arithmetic, NOT a kernel property — asserting it bit-exact would be the bug; tolerance derived from the recursive-summation error bound with ~4 orders of safety, shown in the test). Plus the now-deterministic global leak-Σ (post-`dbd76e5`) and a journal-only friction-timing capture (no bench gate, no timing assertion — flake-free). RED was a genuine deterministic compile failure (declared-but-absent `[[bin]] timeshard_runner` source; surfaced as Cargo's missing-source error rather than the plan's predicted `env!` error — same root cause, same trigger, no false-green; recorded as a benign plan-text imprecision in Concerns). The shipped m5.2 symbol-fan path (`swarm_runner.rs`, `swarm.rs`) and m5.1 core stay byte-untouched and green. Boss-verified independently: **timeshard determinism 5/5** (consistent ~2.7s, no jitter); full ail-embed suite **0 failed AND 0 ignored** across all binaries (lib 2, swarm 2 — the m5.2 leak-proof stays un-ignored & green, the whole saga's resolution intact —, smoke 1, rt_archive 1, timeshard 1); isolated `embed_rc_global_stats_race` still green; AILang workspace `cargo metadata` data-server count 0; compiler-surface diff empty (zero diff to crates/ailang-*, crates/ail/, runtime/, examples/*.ail, root Cargo.toml, DESIGN.md). **Friction-harvest deliverable (P2 flat-array-decision input): host-per-tick-FFI ≈ 658 ms / 3 192 562 ticks ≈ ~206 ns/tick at real EURUSD volume.** M5's functional work is complete; the milestone closes next via the mandatory `audit` (which also handles the pre-existing `docs/DESIGN.md:2358-2360` "additive M4 concern" drift, explicitly out of m5.3 scope). → 2026-05-19-iter-embedding-abi-m5.3.md -- 2026-05-19 — audit embedding-abi-m5 (milestone close): DRIFT (one doc-honesty tidy) + RATIFY (gc_rss trio, evidenced) + causally-exonerated pre-existing P2 bench noise. Architect: **Invariant 1 PASS** (zero data-server in any compiler crate/runtime; ail-embed its own workspace root; only "finance" hits are the M3 ABI export *symbol name* — ABI not data-server knowledge), M3 frozen-layout SSOT unmoved (the lone rc.c change `7bfa11e` touched only the global stats counters), rc.c header-comment honesty accurate; drift = `[medium]` DESIGN.md:2358-2360 stale "additive M4 concern" (PRE-EXISTING, M4 retired, correctly scoped out of M5) + `[medium]` DESIGN.md §"Embedding ABI" silent-drift (it asserts "no shared mutable runtime state … data-race-free" but the now-atomic global RC-stats fallback IS shared mutable state under concurrency — M5 is AILang's first concurrent consumer) + `[low]` rc.c:88 "two unconditional ++" imprecise — one tidy, same honesty axis, not split. Bencher (my pre-audit "gc_rss is allocator-orthogonal → environmental" hypothesis was **refuted by evidence** — the verification discipline working): the 3× `*.gc_rss_kb +32%` are GENUINE, DETERMINISTIC, M5-attributable — `7bfa11e`'s atomic split a branch in `ailang_rc_alloc` (reached by GC programs via `str.c` int→str at `print`), spilling `%r14/%rbx`; a stale interior list pointer in the new spill slot is conservatively retained by Boehm, pinning a ~32 MB dead spine (Boehm collection #12 freed PRE 32 MB vs HEAD 32 B); bisects exactly on the commit, IR byte-identical, `ail` sha256-identical; RC/bump RSS dead-stable. `bump_s`/`max_us` = pre-existing tracked-P2 (bump RC-orthogonal; max_us 1-sample tail-jitter, PRE often worse, +65→+104% swing between runs minutes apart — dispositively noise), NOT M5. Adjudication: **RATIFY** the gc_rss trio — language reason (Decision 9: Boehm is explicitly transitional; the cost is purely a Boehm conservative-scan false-retention artifact on the transitional allocator of a *correct, necessary, user-approved swarm-safety fix*, codegen-layout-fragile, RC path the committed model and unaffected; chasing it with an out-of-scope codegen change invests in the path being removed) — selective baseline re-anchor `bench/baseline.json` lines 14/44/64 ONLY (137448/137768/137636, bencher-confirmed steady-state; every other metric byte-pristine — a wholesale --update-baseline would have absorbed the P2 noise and broken the established-envelope discipline); re-run confirms the 3 now `ok` at ±0.3%. **NO-ratify / causally exonerated** for bump_s×2 + max_us×2 — baseline pristine, identical disposition to the M2/M3/prelude-decouple closes. Friction-harvest recorded (P2 input): host-per-tick-FFI ≈ ~206 ns/tick at real EURUSD volume. M5 closes after the doc-honesty tidy lands + Boss-verified. Ratify discipline honored: updated baseline JSON + this paired JOURNAL ratify entry committed together. → 2026-05-19-audit-embedding-abi-m5.md -- 2026-05-19 — iter embedding-abi-m5.tidy (DONE 3/3): resolves the M5 milestone-close audit DRIFT (28ab56a). DOC/COMMENT-ONLY, recon-pin-verified, single cohesive commit (M2.tidy a80d495 / M3.tidy 63d7d60 precedent — pins are the coverage, no RED, no audit/fieldtest gate). `docs/DESIGN.md` edit-1 §"Free (host side)": dropped the retired-M4 forward-reference ("an additive M4 concern, not a contradiction of this freeze" — M4 retired 2026-05-18) → present-tense current fact (a boxed-field record is **not** an M3 embedding type, export-gate-rejected; the freeze covers exactly the all-scalar single-ctor record). edit-2 §"Embedding ABI": reconciled the now-inaccurate "(no shared mutable runtime state — … data-race-free, sanitiser-verified)" blanket with the real post-7bfa11e state — the per-allocation hot path (per-ctx counters + per-object refcount header) is non-atomic by design and never shared (`Ctx: !Send`, single-thread-per-ctx); the one datum a multi-threaded host shares is the global RC-stats fallback counter, atomic-relaxed so the swarm's leak accounting is exact; "data-race-free, sanitiser-verified" retained (still true). `runtime/rc.c:88` 18g.0-block comment-only: stale "two unconditional `++` operations" → accurate "one counter bump per alloc/free … relaxed atomic add on the global null-ctx fallback, a plain `++` on the per-ctx path" (consistent with the adjacent already-correct :93-106 atomic block + :45-55 Threading header). Boss-verified independently: all pins green (design_schema_drift 8/0, docs_honesty_pin 5/0, effect_doc_honesty_pin 4/0, embed_record_layout_pin 1/0) — the mechanical proof the edits moved no pinned/hashed byte; the adjacent separately-pinned bare-scalar sentence is byte-identical (shifted 2299→2305 by net-added lines; `docs_honesty_pin.rs:135` is a substring pin, passes); rc.c diff strictly comment-only (filter empty); `cargo build --workspace` Finished; git scope = only docs/DESIGN.md + runtime/rc.c + journal/stats. Clears the M5-audit doc-honesty debt; no new debt. This is the FINAL M5 iteration — M5 (the M1–M5 Embedding ABI arc) is now functionally complete, audited, ratified, and doc-honest. → 2026-05-19-iter-embedding-abi-m5.tidy.md -- 2026-05-19 — iter design-md-rolesplit.1 (DONE 9/9, whole milestone): the 3020-line `docs/DESIGN.md` replaced by the `design/` ledger — `design/INDEX.md` (sole addressable spine, typed Contracts+Models tables, polymorphic links: prose file OR authoritative source `//!`), 14 `design/contracts/*.md` test-linked invariants + 3 source-link-only contracts (mangling/env-construction/qualified-xref, no prose file — code is SoT), 5 `design/models/*.md` onboarding whitepapers, and `docs/journals/2026-05-19-design-decision-records.md` (the relitigation-guard archive: every why/rejected/does-not-do/rollback/empirical `###` moved out at `###` granularity). Clean cut (`git rm docs/DESIGN.md`, no stub). RED-first `crates/ailang-core/tests/design_index_pin.rs` 4-clause anti-regrowth spine (DESIGN.md-gone / every-INDEX-link-resolves / every-contract-names-a-resolvable-ratifier / contracts-carry-no-decision-record-prose) demonstrably RED→GREEN. Build-atomic by task ordering (the only compile-time consumer, `design_schema_drift.rs` `include_str!`, retargeted to `design/contracts/data-model.md` BEFORE the deletion; the `## Data model`/`## Pipeline` slicer dropped — a simplification the split enables). 2 NoInstance diagnostics + their 2 lockstep E2Es retargeted to `design/contracts/{float-semantics,typeclasses}.md` (the contiguity-across-`\`-continuation hazard scrubbed). ~12 agent reading lists + 5 SKILL bodies + CLAUDE.md + skills/README.md + ~25 code/C/.ail/spec comment xrefs retargeted to the Appendix destinations; OQ7 dangling "Iter 13b" cite deleted (no forward target — a pointer would be fiction). honesty-rule.md rewritten so the rule names the new home (rationale→journals), resolving the recon-found internal contradiction; the two `docs_honesty_pin.rs:70,72` pinned phrases preserved verbatim+contiguous. Boss-verified independently: whole `cargo test --workspace` **646 passed / 0 failed**, `design_index_pin` 4/4, acceptance grep **CLEAN of live DESIGN.md refs** (residuals = only the spec-mandated clause-4 deletion-enforcer), honesty-rule repair carries no clause-3 marker. Spec grounding-check PASS ×2 (one re-dispatch after a corrected commitment-4 pin-status claim; one after the Boss-adjudicated relocation-appendix amendment resolving plan-recon's 7 open questions — ledger completed to 17 contract rows incl. the qualified-xref/str-abi/scope-boundaries additions the 12-list under-counted). 2 DONE_WITH_CONCERNS routed to the mandatory milestone-close `audit`: (a) `design/contracts/str-abi.md:23` `(iter str-concat, 2026-05-13)` API-provenance stamp trips advisory `architect_sweeps.sh` Sweep-1 — Boss-confirmed **byte-identical to DESIGN.md@deeffb1:2062-2065**, a faithfully-migrated PRE-EXISTING anchor (sweep regexes verbatim, only the path retargeted), NOT a split-introduced regression — RATIFY-or-tidy at audit; (b) the now stale-direction `(see "Str ABI" below)` intra-prose cross-ref in `float-semantics.md` (the Appendix is heading-level; no task prescribes intra-prose cross-ref rewrite) — audit-adjudication candidate. One plan defect recorded (Task 9 Step 4's verbatim acceptance grep used a `^\./` anchor not matching the system's `grep -rIn` output; substance independently re-verified CLEAN). → 2026-05-19-iter-design-md-rolesplit.1.md -- 2026-05-19 — design-decision-records (migration): relitigation-guard archive — every why/rejected/does-not-do/rollback/empirical ### moved out of the former docs/DESIGN.md by the design-md-rolesplit milestone. Companion to spec 2026-05-19-design-md-rolesplit. → 2026-05-19-design-decision-records.md -- 2026-05-19 — audit design-md-rolesplit (milestone close): DRIFT (one tidy iteration) + bench causally-exonerated (baseline pristine). Architect drift_found, relocation byte-faithful (MIXED Decisions / dual-link / source-link / rewritten honesty-rule all match the spec Appendix vs deeffb1; build-atomicity holds; design_index_pin 4/4; honesty-rule.md correct + pins retargeted; agent contracts coherent) — one [medium] spirit-finding: faithfully-migrated decision-record/history prose in design/contracts/{typeclasses,str-abi,scope-boundaries}.md DODGES the 6 literal clause-3 markers (case+wording variance) but IS the relitigation content the split's spirit sends to journals; plus [low] float-semantics.md stale-direction "see Str ABI below". Both routed items adjudicated strippable-doc-archaeology-to-TIDY-not-ratify (the str-abi.md:23 advisory Sweep-1 exit-1 correctly diagnoses real pre-existing history residue, byte-identical DESIGN.md@deeffb1:2062-2065, faithfully migrated — not split-introduced). Bencher: NO-ratify, causally-exonerated DECISIVE on byte-evidence — emitted IR AND final -O2 binaries byte-identical 176821c vs pre-milestone dd5b183 for all 6 check.py firings (sha256+cmp), all ~11 changed code/runtime files audited comment/docstring/diagnostic-string-only, zero IR-path change; the firings are tracked-P2 (*.bump_s staleness, *_at_rc.max_us -n5 tail jitter, *.gc_s Boehm-scan variance — NOT the M5 gc_rss trio); compile_check 24/24 + cross_lang 25/25 pristine corroborate; baseline untouched (identical to M2/M3/M5). Resolution: one tidy iter design-md-rolesplit.tidy — (1) move the history prose to the decision-record journal, (2) fix the stale "below", (3) re-scope architect_sweeps honesty sweeps to design/contracts only (models/ is the explicitly-narrative tier — scanning it for history-anchors is a post-split category error), (4) widen design_index_pin.rs clause-3 to SUBSUME Sweep-1's history-anchor regex over contracts/ (invariant: clause-3 GREEN ⟹ Sweep-1 clean in contracts/ — the hard gate enforces the spirit, closing the literal-marker dodge permanently). No fieldtest (zero authoring-surface change). Milestone closes after the tidy lands Boss-verified. → 2026-05-19-audit-design-md-rolesplit.md -- 2026-05-19 — iter design-md-rolesplit.tidy (DONE 7/7): resolves the milestone-close audit DRIFT (2ba5e16). Gate-first TDD: widened design_index_pin.rs clause-3 to a hand-rolled FAITHFUL Sweep-1 superset — case-sensitive digit-anchored Sweep-1 line anchors (confirmed ZERO across all contracts) + Sweep-1's `^[^/]*` path-EXCLUDED date_anchor (so docs/specs/2026-.. citations are not flagged) + the audit-named decision-record PHRASES (case-insensitive, closing the capital-variance dodge); NO regex dep, the blanket case-insensitive iter-detector REJECTED as unworkable (it conflated the memory-model rule-names "Iter A/B" + ordinary "pre-existing/pre-set/pre-Boehm" with provenance). Sentence-level strip of the faithfully-migrated history/decision-record prose out of 5 contract files (typeclasses/str-abi/scope-boundaries — the audit's spot-check — PLUS roundtrip-invariant.md:73 + data-model.md:149,161, the 2 the exhaustive plan-time scan found the spot-check missed) into docs/journals/2026-05-19-design-decision-records.md, each removed sentence replaced by its present-tense contract equivalent; float-semantics.md stale `(see "Str ABI" below)` → `(see design/contracts/str-abi.md)`; architect_sweeps.sh honesty sweeps re-scoped design/contracts+design/models → design/contracts only (models/ is the explicitly-narrative tier; scanning it for history-anchors is a post-split category error) + skills/audit/agents/ailang-architect.md lockstep. Invariant established: clause-3 GREEN ⟹ Sweep-1 finds nothing in contracts/. The prior dispatch correctly BLOCKED on a real plan defect (iso_date lacked Sweep-1's path-exclusion → over-fired on legit spec-path citations); per the "two+ defects in one iteration ⇒ fix the upstream artifact, not a third patch" discipline the audit Resolution mechanism+scope were corrected in lockstep (f2cdd67) before re-dispatch. Three docs_honesty_pin pinned-byte runs survived contiguous (str-abi.md:19, typeclasses.md:297, scope-boundaries.md:8 — shifted from :9 by the authorized :5-6 collapse, content byte-untouched). Boss-verified independently: cargo test --workspace 646/0, design_index_pin 4/4 (clause-3 RED→GREEN), architect_sweeps.sh exit 0 "All five sweeps clean" (acceptance criterion 9 met), acceptance grep CLEAN, 3 pins each exactly 1 contiguous match. Zero spec/quality re-loops. This is the FINAL design-md-rolesplit iteration — the milestone (DESIGN.md → design/ ledger role-split) is now functionally complete, audited, drift-resolved, and the in-code hard gate enforces the honesty spirit. → 2026-05-19-iter-design-md-rolesplit.tidy.md -- 2026-05-19 — iter design-ledger-formal-links.1 (DONE 5/5, whole milestone): positive-half completion of the DESIGN.md → design/ split — design/ body cross-references are now formal, file-relative Markdown links `[label](path)` into the durable tier (design/ + crates/** + runtime/**), and a new in-tree hard gate `design_index_pin.rs` **clause-5** (`design_body_links_are_durable_and_resolve`) walks every design/contracts/*.md + design/models/*.md, strips fenced code (`strip_fences` toggles on ```/~~~), extracts every `](path)`, and asserts it resolves file-relative to a real durable file — never docs/, never an in-file #anchor. RED-first via identity-stubbed strip_fences (four embedded synthetic vectors — first FAILS) → real toggle-on-fence impl → GREEN. **clause-5 ∘ clause-3 = complete invariant** the milestone delivers: every contract cross-reference is either a resolving durable file-link OR clause-3-forbidden decision-record prose. Convert-set (recon-and-corpus-verified, closed): 7 prose refs / 8 link tokens — float-semantics:69/100, embedding-abi:45, memory-model:44/105 (the genuine cross-file stale-direction wart: "Method dispatch below" but the heading lives in typeclasses.md:227, not memory-model.md), scope-boundaries:48/88 (mixed-referent split: `ailang-core::desugar` → source link + Pipeline → ../models/pipeline.md). Disposition-(b) on the 2 PROSE_ROUNDTRIP homeless pointers (pipeline.md:61, authoring-surface.md:180) — pointer removed, ail merge-prose behavioural prose preserved (recon proved roundtrip-invariant.md does NOT carry the merge-prose-cycle content; the prior clause-6 sample's "(a) retarget" was thus wrong, corrected in spec amendment 1 to the three-disposition rule). honesty-rule.md positive-half paragraph inserted pin-safe between L14 and L16; both `docs_honesty_pin.rs` phrases byte-identical. Spec went through TWO corpus-grounded amendments before plan-write (grounding-check PASS ×3): (1) recon-driven — clause-6 incomplete + missing navigational/nominal cross-reference definition + OQ2/OQ3 byte-policies; (2) plan-time-corpus-driven — clause-5 fence-skip + the in-fence schema-annotation carve-out (data-model.md 38/66/79/206/226 are inside ```jsonc fences 30–87 + 203–228, where Markdown links are literal text on every renderer; the inline analog of the nominal-mention carve-out — schema-example documentation, not browsable cross-references). Per the "two+ surfaced ⇒ ground properly once, don't iter-patch" discipline, amendment 2 was the definitive corpus-grounded pass (every convert-set ref's prose-vs-fence status + exact bytes personally verified before amending), not a fourth patch returning later. INDEX-spine sub-fork resolved by user MC during brainstorm Step 2: spine stays repo-root-relative (registry tier is structurally invariant — move-fragility absent), clause-1 byte-unchanged; body uses file-relative (the converged-design path-form decision). Boss-verified independently: `cargo test --workspace` 647/0 (+1: clause-5), `design_index_pin` 5/5, `docs_honesty_pin` 5/5, design/INDEX.md + decision-records journal byte-unchanged, clauses 1–4 source byte-unchanged (the 2 `-` lines are the two-line //! header rewrite Task 1 Step 5 itself delivers — clauses are #[test] fns, untouched), 0 docs/ link targets, 0 #fragments, 8 `](` link tokens (= the closed convert-set), embedding-abi.md:48 docs_honesty_pin pinned phrase byte-identical. One Concerns item: Task-5 Step-7 plan-predicted `-`-line count was 1, actual 2 — planner self-review-item-8 arithmetic miss on the //! header rewrite (Steps 5 + 7 inconsistent within Task 1+5 about how many lines the header rewrite removes); harmless (substantive assertion *clauses 1–4 source unchanged* fully holds), recorded as lesson. Next: mandatory milestone-close audit; no fieldtest (zero authoring-surface change — reasoned exclusion). → 2026-05-19-iter-design-ledger-formal-links.1.md -- 2026-05-19 — audit design-ledger-formal-links (milestone close): **CLEAN** — no drift, no tidy iteration, bench trio 0/0/0 pristine, no ratify warranted, no fieldtest (zero authoring-surface change — reasoned exclusion stands). Architect: every claimed commitment + invariant verified positively — clause-5 implements all four predicates (resolves-from-containing-file / durable-tier / no-fragment / fence-skip via toggle-`strip_fences`); RED-first synthetic-vector mechanism honest; scope excludes design/INDEX.md per commitment 4. The 7 prose conversions (8 link tokens) match the spec's Acceptance-3 closed enumeration verbatim; file-relative arithmetic correct (siblings → bare filename; `crates/` via `../../`; cross-`design/`-subtree via `../models/`); zero docs/ link targets; zero #fragment. Disposition (b) on the 2 PROSE_ROUNDTRIP homeless pointers correctly applied: cross-tier pointer removed, behavioural prose preserved (authoring-surface case slightly rephrased to keep meaning without the pointer). honesty-rule.md positive-half paragraph pin-safe: both `docs_honesty_pin.rs` phrases byte-identical (L14 + L25, the latter shifted from L19 by the +6 lines); the `design_index_pin.rs clause-5` mention is a nominal identifier in the existing "Ratified by:" footer style (no `[](...)` form, no navigational pointer — consistent with §Scope navigational-vs-nominal definition); no clause-3 PHRASE tripwire, no Sweep-1 anchor. **Commitment 4 honored:** design/INDEX.md byte-unchanged (zero-line diff vs HEAD); clauses 1–4 of design_index_pin.rs source byte-unchanged (the only `-` lines are the two-line `//!` header rewrite the milestone explicitly delivers). **Commitment 7 honored:** decision-records journal byte-unchanged. **Composition invariant holds:** clause-3 GREEN ∧ clause-5 GREEN simultaneously for every design/contracts/*.md. **Out-of-scope set untouched:** data-model.md 38/66/79/206/226 inside ```jsonc fences (30–87, 203–228); embedding-abi.md:51; every intra-file `above/below` (sample-checked float-semantics.md:70 still intra-file). No commitment-2 violation: no agent/SKILL reading list adds docs/PROSE_ROUNDTRIP.md as a design/-link target; the sole remaining `docs/` mention is design/INDEX.md:43 — prose nominal mention only, no `[](...)` form, outside clause-5's INDEX-excluded scope, recorded as a future-tightening observation not as drift. bench/architect_sweeps.sh exit 0 "All five sweeps clean"; Sweep-1 shares no scope/regex with clause-5 (orthogonality empirically confirmed). Bench: check.py 63 metrics 0 regressed / 0 improved beyond tolerance / 63 stable (exit 0); compile_check 24/0/24 (exit 0); cross_lang 25/0/25 (exit 0) — aggregate 0/0/0. Baseline pristine, no `--update-baseline`, no paired ratify entry. **The cleanest milestone close pattern in the project's recent history:** zero spec defects surviving to audit (the two were caught and amended at planner-recon and plan-corpus-fetch time, before any byte moved — grounding-check PASS ×3 across the corpus-grounded amendments), zero drift, zero bench movement, zero re-loop. The two amendment cycles were the discipline working — recon and corpus-fetching surfaced gaps the brainstorm-sample missed, the spec was grounded properly *before* implementation, the iter then ran clean 5/5 with one non-blocking concern (a planner self-review-item-8 arithmetic miss on the `//!` header rewrite line count — substantive assertion unaffected, recorded as lesson). Milestone closes immediately; no tidy, no fieldtest. → 2026-05-19-audit-design-ledger-formal-links.md diff --git a/docs/roadmap.md b/docs/roadmap.md index 812a7eb..f7948a6 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -21,11 +21,10 @@ work progresses. - **\[idea\]** — not yet decision-ready, no commitment. - Optional `depends on:` line names another entry that has to land first. -- Optional `context:` line points to the rationale. For current - work this is a spec file under `docs/specs/` or a commit hash - (`git show `); for pre-2026-05-11 entries it points to the - archived `docs/journal-archive.md`. The roadmap is intentionally - terse; rationale stays in those sources. +- Optional `context:` line points to the rationale: a spec file + under `docs/specs/`, a commit hash (`git show `), or a + `git log --grep=""` pointer for older entries. The + roadmap is intentionally terse; rationale stays in those sources. - Priority buckets: - **P0** — in flight. Spec or plan already exists. - **P1** — next up. Decision made; not yet started. @@ -365,9 +364,8 @@ work progresses. polymorphic links — prose file OR authoritative source `//!`), 14 `design/contracts/*.md` test-linked invariants + 3 source-link-only contracts (mangling/env-construction/ - qualified-xref — code is SoT), 5 `design/models/*.md` - whitepapers, and `docs/journals/2026-05-19-design-decision-records.md` - (the relitigation-guard archive). RED-first + qualified-xref — code is SoT), and 5 `design/models/*.md` + whitepapers. RED-first `design_index_pin.rs` 4-clause anti-regrowth spine; clause-3 is a hand-rolled **faithful Sweep-1 superset** so the in-code hard gate enforces the honesty spirit (clause-3 GREEN ⟹ Sweep-1 @@ -418,9 +416,9 @@ work progresses. - **Decision-record / relitigating-guard** — why X was chosen, why Y/Z were rejected (region inference, tracing GC, …). Consumer: a future `brainstorm`, so it does not re-propose a - settled-and-rejected idea. This already has a home — - `docs/journals/` — and sits in DESIGN.md only because the - audit-trail invariant predates per-iter addressable journals. + settled-and-rejected idea. (This tier was migrated out of + DESIGN.md at split time and later retired entirely; the + project's history lives in `git log`.) Underlying principle: the code is authoritative for *what it does* — delete every description of behaviour, the code speaks; @@ -440,8 +438,8 @@ work progresses. single-owner `//!` source header (mangling, env, roundtrip, float, tail-calls, data-model), or a `design/models/…` whitepaper. INDEX.md is the sole addressable entry point. - Decision-records move out to `docs/journals/`; the honesty-lint - audit-trail invariant is repointed there in the same milestone. + Decision-records move out of DESIGN.md; the honesty-lint + audit-trail invariant is repointed in the same milestone. **Rejected — do not relitigate.** Quarto / `.qmd` was evaluated and rejected: every distinctive Quarto feature (rendered @@ -618,13 +616,14 @@ work progresses. `<` etc. resolved via the typeclass instead of the built-in primitive comparators. No commitment; gated on bench re-baselining to make sure the indirection doesn't tank latency. - - context: `docs/journal-archive.md` (2026-05-09 entry). + - context: `git log --before=2026-05-11 --grep=...` for the + pre-archive-cutoff rationale. - depends on: Post-22 Prelude — Eq/Ord (shipped 23.5) - [x] **\[todo\]** `types` / `ctor_index` overlay shape question — decide whether the env's two parallel ctor maps should collapse into one overlay, or stay split. Surfaced during the env-construction unify audit. - - context: `docs/journal-archive.md` (2026-05-10 "Audit close: env-construction unify"); closed by iter ctt.1 — DESIGN.md §"Env construction" anchors the split decision. + - context: closed by iter ctt.1 — DESIGN.md §"Env construction" anchors the split decision. - [x] **\[todo\]** CLI human-mode diagnostic surface for `WorkspaceLoadError`. Shipped 2026-05-14 as iter cli-diag-human — a new `load_workspace_human` helper in `crates/ail/src/main.rs` routes 9 non-JSON @@ -639,7 +638,7 @@ work progresses. the malformed `Type::Con { name: param }` shape earlier). The enum variant + `walk_kind_mismatch` helper stay as dead-but-defensive code; a future tidy can delete both. - - context: `docs/journal-archive.md` (2026-05-11 "Iteration ct.1"); closed by iter ctt.3 — variant + helper + dispatch + Display arm all deleted; canonical-form-rejection test stays green asserting `BareCrossModuleTypeRef`. + - context: closed by iter ctt.3 — variant + helper + dispatch + Display arm all deleted; canonical-form-rejection test stays green asserting `BareCrossModuleTypeRef`. - [x] **\[todo\]** Re-key `Registry.type_def_module` to handle bare-name-collision-across-modules — `BTreeMap` keyed by bare type name silently overwrites when two modules each define @@ -647,11 +646,12 @@ work progresses. and `N.Foo` to whichever insert won. Acceptable for current corpus (distinct bare type names across modules), but the proper fix is to key by `(owning_module, bare_name) → defining_module`. - - context: `docs/journal-archive.md` (2026-05-11 "Iteration ct.1") — flagged by ct.1.5a quality reviewer. + - context: flagged by the ct.1.5a quality reviewer. - [ ] **\[feature\]** 22c — typeclass corpus expansion. User-defined classes beyond the prelude four; multi-parameter classes; superclass chains; richer instance bodies. Deferred from milestone 22. - - context: `docs/journal-archive.md` (2026-05-09 entry). + - context: `git log --before=2026-05-11 --grep=...` for the + pre-archive-cutoff rationale. - [x] **\[milestone\]** Module-qualified class names + type-driven method dispatch — retired the `MethodNameCollision` workaround; shipped 2026-05-13 as iters mq.1 (canonical-form extension for @@ -671,7 +671,8 @@ work progresses. - [ ] **\[todo\]** Boehm full retirement — remove the transitional Boehm GC path now that RC + uniqueness is the canonical memory story. - - context: `docs/journal-archive.md` (pre-22, "Boehm transitional"). + - context: pre-milestone-22 "Boehm transitional" decision — + see commits from that window. - [ ] **\[feature\]** Closure-pair slab / pool — codegen tweak to pool the env+code closure pairs instead of one-shot heap allocs. Bench-gated. @@ -705,7 +706,8 @@ work progresses. `ailang-core`: private-item links from public doc, unresolved intra-crate links). All predate the design-md-consolidation milestone; treat as a one-off sweep. - - context: `docs/journal-archive.md` (2026-05-10 "Audit close: design-md-consolidation"). + - context: surfaced in the design-md-consolidation audit close + (2026-05-10). - [ ] **\[todo\]** `ailang-plan-recon` site-undercount countermeasure (P2, **escalated** — now a structural recon-contract defect, no longer a single-milestone curiosity) — the recon agent hand-lists @@ -744,7 +746,7 @@ work progresses. Constrain the test to scan only §"Data model" + ParamMode block, or extract JSON-schema blocks into a machine-readable file the test consumes. - - context: `docs/journal-archive.md` (2026-05-10 "Audit close"). + - context: surfaced in the 2026-05-10 audit close. - [ ] **\[todo\]** Split `BadCrossModuleTypeRef` into two diagnostics — the current single shape collapses unknown-owner and known-owner / unknown-type-in-owner into one message. The two @@ -835,8 +837,8 @@ work progresses. time, not the runtime ctor lookup (which is type-driven post-ct.2.2). Narrowing is mechanical but needs a careful read to confirm no other consumer survives. - - context: `docs/journal-archive.md` (2026-05-11 "Iteration ct.4") — - milestone close follow-up; closed by iter ctt.1 — recon confirmed the rebuild is the load-bearing consumer of in-band DuplicateCtor (pinned by `crates/ailang-check/tests/duplicate_ctor_pin.rs`); the asymmetry with the mono side is intentional. + - context: milestone close follow-up; closed by iter ctt.1 — + recon confirmed the rebuild is the load-bearing consumer of in-band DuplicateCtor (pinned by `crates/ailang-check/tests/duplicate_ctor_pin.rs`); the asymmetry with the mono side is intentional. - [x] **\[feature\]** `str_concat : (borrow Str, borrow Str) -> Str` — heap-Str concatenation primitive. Shipped 2026-05-13 as iter str-concat (closes fieldtest-form-a friction #4). Symmetric to the @@ -1004,7 +1006,7 @@ work progresses. qualified `Type::Con` through pattern lowering would type-anchor it symmetric to the ct.2.2 typecheck-side fix. Not load-bearing (uniqueness is enforced at typecheck), but cleaner. - - context: `docs/journal-archive.md` (2026-05-11 "Iteration ct.3" + ct.4 close). + - context: surfaced in iter ct.3 + ct.4 close (2026-05-11). - [ ] **\[todo\]** `compare_primitives_smoke` IR-shape assertion — observe `compare__Int` / `compare__Bool` / `compare__Str` symbols in the emitted IR. Blocked on `emit-ir` CLI not running mono; @@ -1013,8 +1015,7 @@ work progresses. build`. The E2E stdout assertion already covers correctness; the IR-shape test would catch refactor regressions that rename mono symbols. - - context: `docs/journal-archive.md` (2026-05-11 "Iteration ct.4") — - dropped from ct.4.4 when the BLOCKED condition surfaced. + - context: dropped from ct.4.4 when the BLOCKED condition surfaced. - [ ] **\[idea\]** Latency methodology rework — switch from per-run timing to a histogram-based approach so tail-latency regressions @@ -1023,13 +1024,13 @@ work progresses. sites (class × 3, class method × 2, instance × 3, instance method × 1). No regression pin today; only worth it if a 22b.4a-era diagnostic needs to change. - - context: `docs/journal-archive.md` (2026-05-10 "Iteration 22-tidy.7"). + - context: surfaced in iter 22-tidy.7 (2026-05-10). - [ ] **\[idea\]** `write_type` `Type::Forall` arm in `crates/ailang-prose/src/lib.rs` silently drops constraints in inline-type rendering. Dormant — surface forms today only carry forall at fn-signature top level. Fix only if a future feature carries forall + constraints inline. - - context: `docs/journal-archive.md` (2026-05-10 "Iteration 22-tidy.6"). + - context: surfaced in iter 22-tidy.6 (2026-05-10). - [ ] **\[idea\]** Richer integration paths between RC and uniqueness — deferred from the 21' arc; revisit once the uniqueness inference covers more program shapes. diff --git a/skills/README.md b/skills/README.md index 3b45ed6..3040fa5 100644 --- a/skills/README.md +++ b/skills/README.md @@ -7,9 +7,8 @@ discipline so the orchestrator (me) cannot accidentally diverge from established practice without explicitly violating a named rule. The system was bootstrapped on 2026-05-09. See -`docs/specs/2026-05-09-skill-system.md` for the design and -`docs/journal-archive.md` ("Skill system live") for the rationale — -both content-frozen and reachable only as historical context. +`docs/specs/2026-05-09-skill-system.md` for the design; +the bootstrap rationale lives in the commits from that week. ## The eight skills