AILang/docs/JOURNAL.md

# JOURNAL

Chronological notes for myself. Not every change; only decisions, obstacles,
and observations that future iterations will need.

## 2026-05-07 — Day 0

- Repo initialised. Assignment in `CLAUDE.md`: LLM-native language, LLVM backend.
- Design decisions captured in `docs/DESIGN.md`.
- Toolchain: `rustc 1.94`, `llvm-config 22.1.3`, `clang` available.
- Decided against `inkwell` in favour of LLVM IR text emit. Rationale in DESIGN.md.
- Workspace layout:
  - `crates/ailang-core` — AST, type, hash, JSON schema
  - `crates/ailang-check` — typechecker (comes later)
  - `crates/ailang-codegen` — lowering + LLVM IR emit
  - `crates/ail` — CLI
- MVP goal: `examples/sum.ail.json` → binary that prints 55. **Achieved.**

## 2026-05-07 — architecture review after the MVP

Still on track? Broadly yes. Concrete observations:

**What holds:**

- JSON AST + canonical form + content hash are all lego bricks that later
  tools can build on without a refactor (`ail deps`, `ail diff`).
- The LLVM IR text pipeline works as planned. No libllvm version pain.
- The effect set is wired into the type system from the start. Extensible to
  row-poly without touching the core.

**Debt that accrues interest:**

1. **`current_block_label_for_phi` is a heuristic** (see codegen). On nested
   `if` terms it will return the wrong block label, because it scans the body
   backwards. Ticking, because no test cases trigger it yet. Must be fixed
   next, before new language features arrive.
2. **No typed AST.** Codegen reads the source AST directly and relies on
   the typechecker having run before. Fine for the MVP; once ADTs or
   closures arrive, I will need a separate typed IR stage (TIR).
3. **The `hash` field is not in the AST.** Right now we hash the def object
   directly. Once I serialise hashes as fields (caching), the hash will
   need to exclude that field before computation.

**Plan iteration 2 (now):**

1. Clean up block-label tracking, with a nested-if test.
2. Strings as a literal + `io/print_str`.
3. Hello-world example as a second E2E test.
4. CLI: `--json` output for machine consumers wherever it fits.

**Plan iteration 3:**

ADTs + pattern matching. That is the next big jump. Requires a typed IR
stage (TIR), because pattern matching lowers into decision trees, which
have a different shape from the AST.

## 2026-05-07 — iteration 2 done

- Block-label tracking is now robust (nested `if`s work). Test
  `max3_picks_largest` protects it.
- Strings as `Lit::Str { value }`, type `Str` -> LLVM `ptr`, with
  `io/print_str` effect op. `examples/hello.ail.json` prints a string.
- CLI: `manifest --json`, `builtins --json` for tool consumers.
- `ail deps [--of NAME] [--json]` lists call edges. Effect ops are tagged
  `effect:NAME` so a consumer can filter them.

**Architecture check:** no structural deviations. Codegen still reads the
source AST directly (a TIR stage will become necessary with ADTs in
iteration 3).

## 2026-05-07 — iteration 3 done: ADTs

- TypeDef in the AST with ctors. A ctor has `name` and `fields: [Type...]`.
- Term::Ctor (construction) and Term::Match (pattern matching).
- Patterns: `Wild`, `Var`, `Lit`, `Ctor { ctor, fields }`. In the MVP,
  nested ctor patterns are NOT allowed — sub-patterns must be `Var` or
  `Wild`.
- Typechecker with a type registry and `ctor_index` (ctor name → ADT). In
  Match, exhaustiveness is checked against the full constructor set. A
  negative test protects this.
- Codegen: boxed heap layout. Per ctor application, `malloc(8 + 8*n)`
  bytes; tag at offset 0, fields from offset 8 (8-byte slots, native typed
  load/store). Match: load tag + switch + arm blocks + phi at the join.
- `examples/list.ail.json` (Cons/Nil list, sum_list via match) returns 42.

**Surprisingly painless.** The architecture decisions from day 0 paid off:
opaque ptr in LLVM 22 makes the boxed layout almost glue-free; effect
tracking was untouched by ADTs; the JSON AST takes new node types
cleanly.

**Debt accrued:**

1. **Codegen still reads the source AST directly.** The temptation to push
   on without TIR was strong — and worked, because my Match restrictions
   are flat (no nested patterns). Once nested patterns arrive, decision-
   tree lowering will not stay clean without TIR. Debt acknowledged; not
   due now.
2. **No GC.** The heap leaks. Acceptable for demo programs; must be
   addressed before any longer-running program. Options for Phase 4:
   refcount, Boehm-GC linkage, region inference.
3. **No runtime pretty-printer for ADT values.** `io/print_int` is enough
   for demos, but a generic `show :: a -> Str` for ADTs would be valuable.
   Requires dispatch over the tag — feasible, but not now.

**Plan iteration 4:**

The next steps are less obvious. Three candidates in priority order:

1. **Module system (imports).** Right now everything is in a single module.
   With multiple modules + cross-module hashing the language only becomes
   practical for several defs.
2. **Structured error output (`ail check --json`).** So tools can react to
   type errors without parsing text.
3. **Closures / higher-order functions.** Requires closure conversion and
   is a bigger step.

Iteration 4 will be (1) + (2) — both strengthen the LLM tooling and have
moderate risk.

## 2026-05-07 — workflow change: orchestrator + agent repo

At the user's suggestion, switching to **orchestrator mode**: I delegate
clearly bounded implementation chunks to sub-agents and keep only
architecture decisions, reviews, and commit discipline. Four specialised
agents drafted: implementer, architect, tester, debugger.

**Important correction:** the user required the agents not to be hidden in
`.claude/agents/`, but versioned as a visible part of the project under
`agents/`. DESIGN.md gained a new section "Project ecosystem", which
records this: AILang is not just a language, but language core + CLI +
examples + agents + docs + tests, all of equal weight.

Invocation scheme: the system-prompt body from `agents/<name>.md` as a
prefix before the concrete task + sent to the `general-purpose` agent.
Functionally identical to subagent loading from `.claude/agents/`, but
visible in the repo.

**Plan iteration 4 (revised):**

The module system is more involved than expected (cross-module hashing,
import resolution). First the smaller tooling wins, then the module system
as iteration 5:

1. **Structured error output** (`ail check --json` with a Diagnostic
   struct, stable codes like `unbound-var`, `type-mismatch`).
2. **`ail diff <a> <b>`** — semantic module diff via per-def hash
   comparison.
3. **IR snapshot tests** — regression protection for the codegen pipeline.

## 2026-05-07 — iteration 4 done: LLM tooling consolidation

Three sub-commits, each produced by an `ailang-implementer` invocation
and spot-checked by the orchestrator:

- `93fe723` Iter 4a: `ail check --json` with a `Diagnostic` struct
  (`severity`, `code`, `message`, `def`, `ctx`). Stable codes:
  `unbound-var`, `type-mismatch`, `arity-mismatch`,
  `non-exhaustive-match`, `unknown-ctor`, `unknown-ctor-in-pattern`,
  `nested-ctor-pattern-not-allowed`, `duplicate-def`,
  `unknown-effect-op`, `unknown-type`, `schema-mismatch`. API:
  `check_module(&Module) -> Vec<Diagnostic>`.
- `c652b12` Iter 4b: `ail diff <a> <b> [--json]` as a structural top-level
  def diff via BLAKE3 hash. Four categories (added/removed/changed/
  unchanged), sorted alphabetically, exit code 1 on diff.
- `74a2005` Iter 4c: IR snapshot tests in
  `crates/ail/tests/snapshots/{sum,max3,hello,list}.ll`.
  Normalisation of `target triple`. Update via `UPDATE_SNAPSHOTS=1
  cargo test ir_snapshot_`. Mismatch produces an `.actual` file.

Test count: 28 (previously 19). 7 E2E + 4 IR snapshot + 9 ailang-check + 1
ailang-codegen + 7 ailang-core.

**Closed from the debt register:**

- Block tracking in codegen has not been a heuristic risk since Iter 2;
  the explicit `current_block: String` track is now additionally
  protected against regression by Iter 4c snapshot tests. Debt closed.

**New / sharpened debt:**

1. `check_module` is **single-shot** — the first error aborts, no
   multi-diagnostic gathering. The spec was that way, but the format
   suggests Vec semantics. A real multi-diagnostic refactor will be
   cheaper once TIR exists (a central error accumulator via a separate
   stage). Not due now.
2. `source_filename` in the IR is hard-coded to `"<module>.ail"`. As long
   as there is only one top-level module, that is platform-stable. With
   Iter 5 (module system + imports) the path becomes relevant — keep it
   path-independent at construction time, otherwise the snapshots will
   tip over.

**Plan iteration 5:** module system with imports. Cross-module hashing,
import resolution, multiple `.ail.json` files in one build. The multi-
diagnostic refactor only after that.

Sub-steps:

- **5a — workspace loader.** `ailang_core::Workspace { modules:
  BTreeMap<String, Module> }` plus `load_workspace(entry: &Path)`, which
  follows `imports` recursively from the entry module. Convention:
  `import { module: "foo" }` resolves to `<dir>/foo.ail.json` next to the
  entry. Cycle detection. CLI: existing subcommands keep working on a
  single module; a new `ail workspace <entry>` lists all reachable
  modules with hash. Tests: two small example modules with an import
  relation; cycle test.
- **5b — cross-module typecheck.** The typechecker takes `&Workspace`
  instead of `&Module`. Imports are mounted in the env as a namespace
  (`alias.def` or, with no alias, `module.def`). New diagnostic codes:
  `unknown-module`, `unknown-import`, `import-cycle`, `ambiguous-name`.
  Tests per code.
- **5c — cross-module codegen.** The emitter produces IR for all modules
  in the workspace, prefix-mangled with `@ail_<module>_<def>`. E2E test:
  a program that uses a function from module B in module A returns the
  correct result in the binary.
- **5d — tooling adjustments.** `manifest`, `describe`, `deps`, `diff`
  gain a `--workspace` mode (recursive). The single mode stays the
  default for backwards compatibility.

During Iter 5, at construction time **keep `source_filename`
path-independent** (module name only, no directory prefix), otherwise
the IR snapshots will tip over.

## 2026-05-07 — Iter 5b done: cross-module typecheck

- `check_workspace(&Workspace) -> Vec<Diagnostic>` as the top-level API.
  `check_module` is preserved and internally lifts the module into a
  trivial workspace.
- Convention for qualified references (recorded in DESIGN.md):
  `Term::Var { name }` with exactly one dot = `<prefix>.<def>`. Prefix is
  an import alias or module name. No new AST node, no renamed fields ⇒
  hashes stay stable; all `ir_snapshot_*` still green.
- Three new diagnostic codes: `unknown-module`, `unknown-import`,
  `invalid-def-name` (with `ctx.reason: "contains-dot"`).
- CLI: `ail check <entry>` now **always** loads via `load_workspace`.
  Workspace load failures become structured diagnostics in JSON mode with
  codes `module-not-found`, `module-cycle`, `module-name-mismatch`,
  `module-hash-mismatch`, `schema-mismatch`. `ail build` and
  `ail emit-ir` stay per single module (cross-module codegen is 5c).
- Examples: `ws_main.ail.json` now calls `ws_lib.add` (observable). New:
  `ws_broken.ail.json` (`unknown-import`),
  `ws_unknown_module.ail.json` (`unknown-module`).
- Tests: 37 green (previously 32). 4 new workspace integration tests in
  `crates/ailang-check/tests/workspace.rs`, one new e2e test
  `check_workspace_resolves_import`.
- Debt: single-shot diagnostics still in place (multi-diagnostic after
  5c). The dot convention covers exactly one dot — nested module paths
  (`a.b.c`) do not exist; that would only be a topic with hierarchical
  modules and currently falls through as `unbound-var`.

## 2026-05-07 — Iter 5c done: cross-module codegen

- **Mangling break (deliberate).** All AILang functions are now called
  `@ail_<module>_<def>`, even in single-module programs. The old form
  `@ail_<def>` is gone. Strings/const globals analogously
  (`@.str_<module>_<hint>_<idx>`, `@ail_<module>_<const>`). The entry
  point stays `main` as C ABI: a `define i32 @main()` trampoline calls
  `@ail_<entry-module>_main()`. If the entry module has no
  `main : () -> Unit !IO`, the build fails with `MissingEntryMain`.
- **Workspace lowering.** New top-level API
  `ailang_codegen::lower_workspace(ws: &Workspace) -> Result<String>`
  produces a single `.ll` for the whole workspace. Modules in
  alphabetical order (BTreeMap order); defs in AST order. Cross-module
  calls are resolved in codegen via the import map of the calling module
  — same logic as in the typechecker, locally duplicated with a
  cross-reference (no shared helper module, because the type worlds
  differ: the typechecker handles `Type`, codegen handles `FnSig` from
  llvm types).
- **CLI.** `ail build` and `ail emit-ir` now always load the workspace
  and check/lower it fully. Single-module programs keep working
  (trivial workspace with one module). `emit_ir(m)` stays in the codegen
  crate as a convenience API and internally wraps into a trivial
  workspace.
- **Snapshots regenerated.** `sum.ll`, `max3.ll`, `hello.ll`, `list.ll`
  show the new mangling. New `ws_main.ll` snapshot documents the
  cross-module build: `@ail_ws_main_main` calls `@ail_ws_lib_add`.
- **Tests.** 40 green (previously 37). New: `workspace_build_runs_imported_fn`
  (e2e: prints 5), `ir_snapshot_ws_main`, `missing_entry_main_is_error`
  (codegen unit). Existing behaviour tests
  (`sum_1_to_10_is_55`, `max3_picks_largest`, `hello_world_str_lit`,
  `list_sum_via_match`) stay green — behaviour unchanged, only the
  mangling is new.

**Debt closed:**

- **#19 (`source_filename` hardening).** In the workspace world,
  `source_filename` is now uniformly `<entry-module>.ail`, once per
  workspace. The previous hard-coded path dot is gone with it.

**State:** the module system is closed end to end — loader + typecheck +
codegen + build see the workspace as a coherent unit. The multi-
diagnostic refactor and possibly cross-module ADTs remain for later.

## 2026-05-07 — Iter 5d done: tooling extended to the workspace

- `ail manifest|describe|deps|diff <entry> --workspace` now operate
  across all modules of the workspace. The default without the flag stays
  single-module for backwards compatibility. Manifest sorts by
  `(module, name)`, describe accepts dotted notation `ws_lib.add`, deps
  emits `{from_module, from_def, to_module, to_def}` edges, diff compares
  workspace-wide with added/removed/changed/unchanged_modules and a
  nested sub-diff per changed_module.
- Refactor: `diff_def_lists` is the single source of the four-category
  logic; single and workspace diff share it.
- Tests: 44 green (previously 40). New: `manifest_workspace_lists_all_defs`,
  `describe_workspace_resolves_qualified_name`,
  `deps_workspace_includes_cross_module`, `diff_workspace_added_module`.

**Observation (debt):** `deps` does not filter builtins/locals/function
parameters. In workspace mode that becomes more visible than in single
mode — `ws_lib.add` lists edges to `ws_lib.+` (builtin) and
`ws_lib.a`/`ws_lib.b` (function parameters). A known pre-existing
issue from Iter 2; Task #22 in the backlog.

## 2026-05-07 — architecture review after Iter 5

Architect agent invoked. Findings:

1. **Mangling consistency holds.** `@ail_<module>_<def>` is consistent
   across functions, constants, string globals, and cross-module calls.
   The trampoline is correct. ADT constructors are deliberately
   symbol-free (inline malloc).
2. **Module hashes bit-identical since Iter 4.** The Iter 5c snapshot
   regeneration was a codegen-output change, not a hash break.
3. **Drift, due now:**
   - DESIGN.md says `define i64 @main()`, codegen emits
     `define i32 @main()` (see `sum.ll:35`).
   - String-schema notation in DESIGN.md was shortened
     (`@.str_<module>_<idx>` instead of `@.str_<module>_<hint>_<idx>`).
4. **Debt that accrues interest:** the `deps` builtin leak (Task #22) has
   become a falsehood in workspace mode — close it before the next big
   jump.

**Plan iteration 6 — clean-up:**

1. **Fix DESIGN.md drift.** Update the mangling-scheme block, correct the
   `@main` signature, and note the string globals precisely.
2. **`deps` hardening (#22).** Build a top-level def table per workspace;
   filter edges whose target is not a top-level symbol, or emit them as
   separate `builtin:`/`local:` categories. Function parameters via
   lexical scope tracking from walk_term.
3. **Multi-diagnostic refactor (#20).** `check_workspace` accumulates
   `Vec<Diagnostic>` across all defs instead of short-circuiting on the
   first error. Intra-def may still short-circuit — the value is "see
   all broken defs at once", not "see all broken sub-terms of one def".

Order: 1 first (doc triviality), then 2 before 3 (deps is a tooling-
truth fix, multi-diag is a structural extension).

## 2026-05-07 — Iter 6 done: deps hardening + multi-diagnose + DESIGN audit

Three things landed together. All small, all KISS — no architecture move,
just paying off recorded debt.

**1. `ail deps` filters builtins, params, and let/match bindings (#22).**

Before: `sum -> +, -, ==, n, sum` and `ws_lib.add -> ws_lib.+`,
`ws_lib.a`, `ws_lib.b`. After: `sum -> sum`, `ws_lib.add -> ws_lib.add`
gone (no real deps; only the cross-module call from `ws_main` remains).

Implementation:
- New helper `ailang_check::builtins::value_names()`: derives the
  Var-level builtin names (`+ - * / % == != < <= > >= not`) from
  `list()`, so the install-list and the deps-filter share one source of
  truth.
- `walk_term` in `crates/ail/src/main.rs` now threads a `scope` set:
  fn-params seed it; `Let` adds the bound name for the body only;
  `Match` arms add their pattern variables (`bind_pattern` helper, MVP
  rule "ctor sub-patterns are Var/Wild") and roll them back after. Var
  refs that hit `scope` or `builtins` are dropped; qualified names
  (`prefix.def`) are passed through unconditionally — the typechecker
  forbids dots in def names, so no shadowing risk.
- Tests added: `deps_filters_builtins_params_locals`,
  `deps_workspace_filters_builtins_and_params`. The Iter 5d test
  (`deps_workspace_includes_cross_module`) keeps passing — the only
  edge it asserted is the legitimate one.

**2. `check_module` / `check_workspace` are multi-diagnose (#20).**

`check_in_workspace` returns `Vec<CheckError>` instead of `Result<()>`.
Pass-1 (top-level symbol table) stays fail-fast — corrupt globals would
taint every later diagnostic. Type-def installation is fail-fast within
a module (env corruption) but the outer module loop continues. The
body-check loop is the multi-diagnose layer: each def is checked against
the assembled env, errors accumulate, the next def is attempted.

Test: `body_errors_accumulate_across_defs` — one module with two
independent body errors (arity mismatch + unknown var) yields two
diagnostics with the right `def` field. The legacy single-error `check`
keeps working by `.into_iter().next()`-ing the Vec, so internal snapshot
tests in `crates/ailang-check/src/lib.rs` are unchanged.

Out of scope: intra-def collection. A single fn body with three type
errors still reports one. The "see all broken defs at once" goal is met;
intra-def will require unification deferral and isn't due now.

**3. DESIGN.md `What the MVP is NOT` audit (#24).**

The section was lying: it claimed "No ADTs / pattern matching" (delivered
Iter 3) and "Only ints + bools + unit" (strings landed Iter 2). Renamed
to `What is not (yet) supported`, restructured into "not yet" + "what is
supported (smoke-tested)". New invariant: this section is meant to be
the truth at the **end of the latest iteration**, not a 2026-05-07-day-0
scope statement.

**Architecture check (the user-asked self-questioning):**

- *Would I use this language now?* For non-recursive arithmetic + ADT
  programs over int/bool/str: yes, comfortably. For anything that needs
  mapping, folding, generic data structures: no, closures are the
  blocker. That's the next big sprint, not Iter 7.
- *Consistency:* DESIGN.md, JOURNAL.md, code, and CLI output now agree
  on what the language can do. The "What is not (yet) supported" block
  is the canonical truth surface.
- *Visualisation:* `ail deps --workspace --json` is now a clean
  cross-module call graph (no builtin noise). Good enough for an
  external graph renderer to consume; a built-in DOT/ASCII renderer is
  *possible future tooling*, not "we need it now". KISS.
- *Documentation:* the agents/ directory is the sub-prompt layer, the
  JOURNAL is the iteration log, DESIGN.md is the contract. No new doc
  axes needed at this scale.

**Tests:** 47 green (previously 44). +2 deps tests in `e2e.rs`, +1
multi-diag test in `crates/ailang-check/tests/workspace.rs`.

**Plan iteration 7:**

Closures + higher-order functions. This is the big jump that
DESIGN.md / Day 0 has been pointing at: it requires a typed IR (TIR)
stage, closure conversion in lowering, and a heap-aware ABI. The
multi-diag refactor in Iter 6 was scoped intentionally minimal — when
TIR lands, intra-def diagnostics become structurally cheap and Task #20
gets revisited.