Commit Graph

10 Commits

Author SHA1 Message Date
Brummel 99df14c792 workflow: scrub residual JOURNAL/journal refs missed in the first sweep
A re-grep found 15 live references (excluding docs/specs/ and
docs/plans/, which stay historical) the previous two commits
missed — all in source comments, doctests, bench helpers, and one
agent file.

Per-file:
- crates/ail/src/main.rs: typeclass-coherence diagnostic comment
  pointed at "the JOURNAL queue's wording" — points at
  design/contracts/typeclasses.md alone.
- crates/ailang-prose/src/lib.rs: `//!` header referred to
  docs/JOURNAL.md "Pinned: human-readable prose surface" —
  retargeted to design/contracts/authoring-surface.md.
- crates/ailang-check/src/lib.rs: bugfix tag + "see iter
  method-dispatch-refactor journal" prose collapsed to a clean
  rationale paragraph; the canonical shape is stated inline.
- crates/ailang-surface/tests/prelude_decouple_carve_out_pin.rs:
  "(d) record the rationale in a per-iter journal" →
  "(d) record the rationale in the commit body".
- crates/ailang-surface/tests/prelude_module_hash_pin.rs: header
  collapsed (pd.2/pd.3 milestone narrative dropped — the test's
  purpose is self-evident from its body); both "per-iter journal"
  drift-instructions point at the commit body.
- runtime/rc.c: "bench numbers in JOURNAL 18f.2" →
  "original profiling bench numbers".
- bench/run.sh: two "JOURNAL entry" comments → "commit body".
- bench/architect_sweeps.sh: header rewritten (cross-ref to the
  design-md-consolidation milestone commits, not a JOURNAL entry).
  Sweep-4 extended with two new anti-regrowth phrases
  ("see the per-iter journal", "in a per-iter journal") so journal
  prose can't grow back into design/contracts/ silently.
- skills/audit/agents/ailang-architect.md: "unlike a journal it
  lives on main" → "since it lives on main".
- ail-embed/src/bin/timeshard_runner.rs: "records it in the
  close-out journal" → "prints it to stderr".
- ail-embed/tests/timeshard.rs: two "journal-only friction timing"
  → "stderr-only friction timing".

Verification (after the edit):
- `grep -rin '\bjournal\b'` against live tree returns exactly two
  hits: the Sweep-4 regex itself (intentional — TABU phrases that
  prevent regrowth) and the PHRASES array in design_index_pin.rs
  (intentional — the same regrowth guard at test level). Both are
  load-bearing negative assertions.
- `bash bench/architect_sweeps.sh` exits 0 ("All five sweeps
  clean") — no design/-side regrowth.
- `cargo build --workspace` green; `cargo test --workspace` green.
2026-05-20 11:32:37 +02:00
Brummel 176821c2e7 iter design-md-rolesplit.1 (DONE 9/9): DESIGN.md -> design/ ledger role-split
The 3020-line docs/DESIGN.md is replaced by the design/ ledger:
design/INDEX.md (sole addressable spine, typed Contracts+Models tables,
polymorphic links — prose file OR authoritative source //!), 14
design/contracts/*.md test-linked invariants + 3 source-link-only
contracts (mangling/env-construction/qualified-xref, no prose file —
code is SoT), 5 design/models/*.md whitepapers, and
docs/journals/2026-05-19-design-decision-records.md (the
relitigation-guard archive — every why/rejected/does-not-do/rollback/
empirical ### moved out at ###-granularity). Clean cut: git rm
docs/DESIGN.md, no stub.

RED-first crates/ailang-core/tests/design_index_pin.rs — the 4-clause
anti-regrowth spine (DESIGN.md-gone / every-INDEX-link-resolves /
every-contract-names-a-resolvable-ratifier /
contracts-carry-no-decision-record-prose) — demonstrably RED before,
GREEN after. Build-atomic by task ordering: design_schema_drift.rs's
include_str! (the only compile-time consumer) retargeted to
design/contracts/data-model.md BEFORE the deletion; its
## Data model/## Pipeline slicer dropped (a simplification the split
enables). 2 NoInstance diagnostics + 2 lockstep E2Es retargeted to
design/contracts/{float-semantics,typeclasses}.md. ~12 agent reading
lists + 5 SKILL bodies + CLAUDE.md + skills/README.md + ~25
code/C/.ail/spec comment xrefs retargeted; OQ7 dangling 'Iter 13b'
cite deleted (no forward target — a pointer would be fiction).
honesty-rule.md rewritten so the rule names the new home
(rationale->journals), resolving the recon-found internal
contradiction; the two docs_honesty_pin.rs:70,72 pinned phrases kept
verbatim+contiguous.

Boss-verified independently: cargo test --workspace 646 passed /
0 failed; design_index_pin 4/4; acceptance grep CLEAN of live
DESIGN.md refs (residuals = only the spec-mandated clause-4
deletion-enforcer). 2 DONE_WITH_CONCERNS routed to the mandatory
milestone-close audit: (a) str-abi.md:23 '(iter str-concat,
2026-05-13)' provenance stamp trips advisory architect_sweeps Sweep-1
— Boss-confirmed byte-identical to DESIGN.md@deeffb1:2062-2065, a
faithfully-migrated PRE-EXISTING anchor (regexes verbatim, only path
retargeted), NOT split-introduced — RATIFY-or-tidy at audit; (b) a
now stale-direction intra-prose 'see Str ABI below' cross-ref in
float-semantics.md — audit-adjudication candidate. Plan defect noted:
Task 9 Step 4's verbatim acceptance grep used a ^./ anchor not
matching the system's grep -rIn output; substance re-verified CLEAN.

Spec grounding-check PASS x2. Journals INDEX + decision-records
pointer appended (Boss-only).
2026-05-19 13:04:22 +02:00
Brummel 70f2a318e0 iter embedding-abi-m5.tidy (DONE 3/3): milestone-close doc-honesty drift — pin-safe, doc/comment-only
Resolves the M5 milestone-close audit DRIFT (audit journal
28ab56a). Single cohesive commit, M2.tidy a80d495 / M3.tidy
63d7d60 precedent (pins are the coverage, no RED, no
audit/fieldtest gate).

- docs/DESIGN.md edit-1 §"Free (host side)": dropped the retired-M4
  forward-reference ("an additive M4 concern, not a contradiction
  of this freeze"; M4 retired 2026-05-18) -> present-tense current
  fact (a boxed-field record is not an M3 embedding type,
  export-gate-rejected; the freeze covers exactly the all-scalar
  single-ctor record).
- docs/DESIGN.md edit-2 §"Embedding ABI": reconciled the now-
  inaccurate "(no shared mutable runtime state - ... data-race-free,
  sanitiser-verified)" blanket with the real post-7bfa11e state:
  the per-allocation hot path (per-ctx counters + per-object
  refcount header) is non-atomic by design and never shared
  (Ctx:!Send, single-thread-per-ctx); the one shared datum is the
  atomic-relaxed global RC-stats fallback counter; "data-race-free,
  sanitiser-verified" retained (still true).
- runtime/rc.c:88 comment-only: stale "two unconditional ++
  operations" -> accurate "one counter bump per alloc/free ...
  relaxed atomic add on the global null-ctx fallback, a plain ++
  on the per-ctx path" (consistent with the adjacent already-
  correct :93-106 atomic block + :45-55 Threading header).

Boss-verified independently: all pins green (design_schema_drift
8/0, docs_honesty_pin 5/0, effect_doc_honesty_pin 4/0,
embed_record_layout_pin 1/0) = mechanical proof no pinned/hashed
byte moved; the adjacent separately-pinned bare-scalar sentence is
byte-identical (shifted 2299->2305 by net-added lines; substring
pin, passes); rc.c strictly comment-only (filter empty);
cargo build --workspace Finished; git scope = only docs/DESIGN.md
+ runtime/rc.c + journal/stats. Clears the M5-audit doc-honesty
debt; no new debt.

FINAL M5 iteration. The M1-M5 Embedding ABI arc is functionally
complete, audited, ratified, and doc-honest. Includes the per-iter
journal, stats, and INDEX line.
2026-05-19 03:07:40 +02:00
Brummel 7bfa11e838 fix(rc): GREEN — atomic global g_rc_* stats fallback counters (swarm-safe)
bugfix-rc-global-stats-race, GREEN stage. RED is the separate
audit-trail commit 427b687.

g_rc_alloc_count / g_rc_free_count (runtime/rc.c:90-91) were plain
static uint64_t; the __ail_tls_ctx == NULL fallback at rc.c:161
(alloc) and rc.c:212 (dec-to-zero) did a non-atomic ++. A
multi-threaded host using the global fallback path (no
ailang_ctx_new) raced the read-modify-write and lost increments, so
the AILANG_RC_STATS atexit Σ under-counted non-deterministically.
Not a memory bug — Ctx:!Send keeps every box on one thread; only
the global statistics Σ was wrong (the M5 iter-2 symbol-fan
leak-proof finding, b724cd1; user-approved Option A bounce-back
resolution).

Fix (runtime/rc.c only, 37+/14-): the two globals are now
_Atomic uint64_t; atomic_fetch_add_explicit(.., memory_order_relaxed)
at the two null-ctx fallback ++ sites; atomic_load_explicit(..,
relaxed) snapshot in ailang_rc_stats_atexit (its sole reader).
Relaxed is correct and sufficient — pure statistics, no
happens-before obligation, atexit reader runs after all worker
threads joined. Per-ctx counters and the per-object refcount header
stay non-atomic BY DESIGN (single-thread-per-ctx; boxes never cross
threads) — out of scope, untouched. Frozen value layout / ABI
offsets / host-free rule untouched. The genuinely-still-
single-threaded drop-worklist doc note was correctly left unchanged
(false correction refused); the two stale atomicity doc blocks
corrected for honesty.

Boss-verified independently: RED -> GREEN deterministically 3/3
(jitter gone, allocs==frees==16_000_000); full cargo test -p ail
green (every binary 0 failed — per-ctx tsan harnesses + embed
e2e unaffected; rc.c links into every ail binary, strong regression
gate); scope = runtime/rc.c + journal + stats only; RED files
unchanged. Unblocks resuming the M5 leak-proof.

Includes the per-iter journal, stats, and the INDEX.md line.
2026-05-19 01:57:06 +02:00
Brummel 4ea8bc5faf iter embedding-abi-m3.1 (DONE 6-7): freeze the value layout — DESIGN.md frozen-layout SSOT + lockstep pointers + enforceable byte-pin
Tasks 6-7 of the Boss-repaired split dispatch (Tasks 1-5 committed d5c565d).
Task 6: DESIGN.md §"Embedding ABI" gains the `### Frozen value layout
(M3 — one-way commitment)` SSOT subsection; both "provisional until M3"
sentences rewritten to one-way-freeze wording (docs_honesty_pin-safe — the
pinned "written **bare**" sentence stayed byte-verbatim, :2297->:2299
content-asserted); // FROZEN ABI lockstep pointers at runtime/rc.c +
match_lower.rs lower_ctor + drop.rs; 3 plan-named stale rustdocs fixed +
the Boss-caught stale // gate-comment block (check/lib.rs:1917-1922,
M1/scalar-only -> M1/M2/M3/C-ABI-permitted-incl-record); byte-pin
enforceability demonstrated (RED on a local offset perturbation, GREEN on
git-checkout restore — the only legitimate git op, working-tree only).
Task 7: milestone-close verification gate.

Boss-verified independently (agent report = intent, not outcome): workspace
639 passed / 0 failed; embed_record_e2e 2/2 (own+borrow, global
leak-freedom model); byte-pin 1/1; gate 10/10; forwarder 3/3;
docs_honesty_pin 5/5 pin-safe; design_schema_drift + embed_export_hash_stable
+ embed_e2e + M2 swarm regression-green-unmodified.

INDEX.md line added (deferred from the PARTIAL — one line per iter, final
title). Milestone-close audit next (architect Invariant 1 + bench trio,
spec Testing items 8/9 — audit-owned, not the implement run's job).
2026-05-18 21:29:50 +02:00
Brummel c9a84b33b3 iter embedding-abi-m2.1 (PARTIAL 4/9 + Boss spec-defect repair): ctx ABI + de-globalisation
Tasks 1-4 land fully review-green and are the cohesive shippable
subset (the per-thread embedding ABI, fully wired + proven):
- T1: FIXED-FIRST alloc-guard baseline pin (RED captured -> GREEN at T4).
- T2: runtime/rc.c gains ailang_ctx_t {alloc_count,free_count} +
  ailang_ctx_new/_free + __thread __ail_tls_ctx; the two increment
  sites become 'if (_ctx) _ctx->... else g_rc_...'. g_rc_* statics +
  ailang_rc_stats_atexit + the constructor RETAINED VERBATIM as the
  null-ctx (single-threaded executable) fallback. rc_accounting_tsan.c
  + driver: per-ctx 8x200000 tsan-clean (de-globalisation positive proof).
- T3: codegen Target::StaticLib forwarder gains a leading 'ptr %ctx'
  + one '@__ail_tls_ctx = external thread_local global ptr' decl +
  TLS save/store/restore around the BYTE-UNCHANGED internal call;
  _adapter/_clos + internal arg vector untouched (M1 decision held);
  doc-comment provisionality narrowed to the value/record layout.
- T4: build_staticlib rejects --alloc != rc (RC-only swarm artefact).

Boss adjudication of the orchestrator's Task-5 BLOCKED (spec-defect,
correctly surfaced not hacked): swarm.c's -DSHARED_CTX negative
control is structurally impossible — examples/embed_backtest_step.ail
is a non-allocating scalar kernel that never writes a ctx field, and
__ail_tls_ctx is __thread, so a shared-ctx scalar swarm has no
shared-memory write to race on (the spec's OWN item-1 honesty point).
The de-globalisation teeth belong to the item-1 rc_accounting harness
(shared ctx genuinely races on ctx->alloc_count; orchestrator
independently confirmed tsan exit 66). Spec amended in lockstep
(Goal coherent-stop, must-fail-axis item 2, Testing item 3,
Acceptance) so item 3's negative control is item-1's by the
de-globalisation-proof / capability-demo split; plan Task 5
restructured (swarm.c per-ctx capability demo only; negative-control
standing test relocated into the item-1 harness driver) + Task 3's
plan-transcription error (@ail_backtest_step ->
@ail_embed_backtest_step_step) corrected. This is a Boss
consistency-repair of an internally-contradictory clause whose intent
is already correctly realised in the same spec — not a redesign; no
brainstorm bounce. Task-5 working files discarded (corrected plan
reproduces them; a structurally-RED test must not enter main).
INDEX.md + final journal land at iter completion (re-dispatch [5,9]).
2026-05-18 17:16:40 +02:00
Brummel fc5f4590f8 rc: opt-in alloc/free stats counter for diagnosing leaks
Two non-atomic uint64_t counters in runtime/rc.c, incremented from
ailang_rc_alloc and the to-zero branch of ailang_rc_dec. An
__attribute__((constructor)) registers an atexit handler IFF the
AILANG_RC_STATS env var is non-empty at startup; the handler prints

    ailang_rc_stats: allocs=N frees=M live=K

to stderr. Default-disabled so production binaries stay quiet.

Used by the e2e test infrastructure for assertions about RC
correctness (e.g. tail-recursive list-sum must not leak outer cells)
and by the bencher / debugger when diagnosing leak shape from a
fixture run. Single-threaded; non-atomic — same scope as the rest
of runtime/rc.c. The two unconditional increments on the hot paths
are negligible relative to the libc malloc/free already there.
2026-05-08 14:18:02 +02:00
Brummel 6b3ff3bbed tidy: rewrite stale 18b/18c.x doc-headers to match shipped state
First half of the post-18-arc tidy-iter (per the new CLAUDE.md
iter-cycle rule). Architect's drift review flagged module-doc
headers describing 18b's leak-everything snapshot or 18c.x's
"deferred" debt that has since shipped. Doc-only changes; cargo
build clean, cargo test --workspace green at e2e=61, no
behavioural change.

- runtime/rc.c top-of-file header: rewrote from "Iter 18b
  deliberately stops at the layout and the alloc... programs
  leak every allocation" (false post-18c.3) to a stage summary
  spanning 18b–18e. Fixed `--memory=rc` reference (renamed to
  `--alloc=rc` in 18b's CLI work). Updated ailang_rc_inc /
  ailang_rc_dec block comments to point at `drop_<m>_<T>` and
  the worklist as the cascade owners, not at "18c will wire
  this up".

- ailang-check uniqueness.rs module-doc: replaced the "deferred
  to later iters" block (which named 18c.4 + 18d as future
  work, both shipped) with a current "what this pass does NOT
  do" block. Cross-fn reasoning is still genuinely deferred;
  per-type drop fns and recursive cascades are NOT this pass's
  job by design (codegen does them, not the inference).

- ailang-codegen emit_drop_fn_for_type doc + in-body comment:
  rewrote "Iter 18e replaces the recursive call with an
  iterative worklist free" to describe the actual shipped
  behaviour — the 18e (drop-iterative) annotation routes
  annotated types through emit_iterative_drop_fn_for_type;
  unannotated types stay recursive by orchestrator design
  (cheaper IR, no worklist alloc).

Held back for the second half of the tidy-iter (pending the
ailang-bencher determinism result):

- DESIGN.md Decision 10 line 700 says modes are "mandatory"
  but lines 940–952 admit they're opt-in with deferred
  mandatoriness. The bench result either supports tightening
  the mandatoriness claim or backs down to "opt-in with
  performance benefit" — orchestrator-level decision blocked
  on the bench data.
- Dynamic-tag partial-drop debt is captured in JOURNAL but
  should be surfaced in DESIGN as a known precision gap.
2026-05-08 13:34:32 +02:00
Brummel ce6ab8ee44 Iter 18e: (drop-iterative) annotation + worklist allocator
Closes the 18-arc's stack-recursion limit. Recursive drop
cascades from 18c.4 overflow on long ADT chains (Linux's 8 MB
default stack maxes out around 1M cells of List). The new
opt-in (drop-iterative) annotation on a Def::Type switches the
synthesised drop_<m>_<T> body from recursive to iterative-with-
explicit-worklist for that type.

Schema:
- TypeDef.drop_iterative: bool. Default false; serde-skip
  when false so existing fixtures' canonical JSON hashes stay
  stable.
- Form-A: (drop-iterative) clause inside (data T ...).

Worklist runtime (4 new ABI symbols in runtime/rc.c):
- ailang_drop_worklist_new(initial_capacity)
- ailang_drop_worklist_push(wl, ptr)
- ailang_drop_worklist_pop(wl) -> ptr
- ailang_drop_worklist_free(wl)

Heap stretchy buffer, doubling on overflow, null-filtering on
push. Lean 4 / Roc precedent documented in the runtime; the
slot-repurposing strategy was considered and rejected because
not every box has a free pointer-typed slot to thread the
worklist through (Cons head is i64, slot 1 is ptr but it's
the field we're following — no free slot).

Codegen (emit_iterative_drop_fn_for_type): for a
drop_iterative type, drop_<m>_<T>(ptr %p) emits a worklist
loop. Fields of the SAME annotated type push onto the
worklist (mono-typed); fields of DIFFERENT types call their
own drop fn directly (recursive on those, but only if THEY
are themselves recursive — i.e. one level of cascade jump
maximum). Mono-typed-worklist is sound for the deep-self-
recursion case the iter targets (List of List of T just
needs the spine flattened).

Tests:
- examples/rc_drop_iterative_long_list — 1M-cell List of Int
  with (drop-iterative) annotation.
- alloc_rc_drop_iterative_handles_million_cell_list E2E —
  builds + runs under --alloc=rc, asserts clean exit. With
  annotation: exits 0. Without annotation (control): SIGSEGV
  at exit code 139 (verified by hand). Worklist is load-
  bearing.
- iter18e_drop_iterative_emits_worklist_body_no_self_recursion
  IR-shape: worklist body has br to loop_head AND no direct
  recursive call into drop_<m>_<T>.
- iter18e_no_annotation_keeps_recursive_drop_body — control:
  unannotated variant still emits the 18c.4 recursive shape.
- 3 surface parse-tests for the annotation round-trip.

Test deltas: e2e 58 -> 61 (+3), surface 18 -> 21 (+3). All
other buckets unchanged. cargo test --workspace green.

Known debt (deliberate):
- Mono-typed worklist: cross-type drop-iterative fields call
  the other type's drop fn directly. A heterogeneous
  worklist would be more general but adds tag tracking
  complexity for a case (drop-iterative T containing
  drop-iterative T') that's narrower than the deep-self-
  recursion target. Documented in
  emit_iterative_drop_fn_for_type's doc.
- Closure / Type::Var / Type::Forall fields fall back to
  shallow ailang_rc_dec via field_drop_call — same as the
  recursive variant.
- Dynamic-tag partial-drop fallback (head_or_zero epilogue
  shallow dec when moved_slots non-empty) — out of scope per
  brief.
2026-05-08 12:53:09 +02:00
Brummel 1eed78c41e Iter 18b: RC runtime + --alloc=rc routing
Wires up reference-counting allocator end-to-end without any
inc/dec emission. Programs run under --alloc=rc and produce
correct stdout (validated against --alloc=gc); they leak every
allocation, exactly like the pre-Boehm era. The point of 18b is
to establish the runtime contract before 18c adds the
inc/dec-emission codegen pass.

runtime/rc.c — new file. 8-byte uint64 refcount header
prepended to every payload; ailang_rc_alloc(size) returns a
ptr to the payload (header at ptr-8). ailang_rc_inc / dec are
declared but never called by codegen yet; they exist so 18c
can wire codegen against a stable runtime ABI. dec frees on
zero refcount but does NOT recursively dec child references —
that's 18c's job once it has per-ctor type info.

crates/ailang-codegen/src/lib.rs — AllocStrategy::Rc variant
added; fn_name() returns "ailang_rc_alloc". Single-line
extension because Iter 18a's bump path had already centralised
the allocator-symbol decision on fn_name() for all four
allocation sites.

crates/ail/src/main.rs — --alloc=rc accepted by Build/Run;
parse_alloc_strategy extended; locate_rc_runtime() helper
mirrors locate_bump_runtime; new Rc arm in build_to compiles
runtime/rc.c with clang -O2 -c and links the resulting .o
into the final binary (no -lgc).

E2E coverage: alloc_rc_produces_same_stdout_as_gc on
list.ail.json (42), alloc_rc_matches_gc_on_std_list_demo for
broader allocation-site coverage. Total e2e bundle: 51 tests
(was 49).

Hand-verified on:
  sum.ail.json --alloc=rc → 55
  list.ail.json --alloc=rc → 42
  borrow_own_demo.ail.json --alloc=rc → 3 then 6
  std_list_demo.ail.json --alloc=rc → matches --alloc=gc

cargo build/test --workspace green; git diff examples/ empty.
2026-05-08 02:13:08 +02:00