Commit Graph

1006 Commits

Author SHA1 Message Date
Brummel 93887aa03b workflow: replace docs/roadmap.md with Gitea issue backlog
The forward queue moves out of the in-tree markdown file and into
Gitea issues at http://192.168.178.103:3000/Brummel/AILang/issues.
Labels: kind:{milestone,feature,todo,idea} + prio:{p1,p2,p3}
+ state:in-progress. Big chunks live as Gitea milestones
(containers) with full prose in the description; smaller items are
standalone issues. Browse-and-filter scales constant against
growing item count; the previous markdown file was 1059 lines, of
which ~850 were already-closed-entry verlauf (the same failure
class the JOURNAL cut removed).

Sync-drift Code<>Tracker mitigation: Soft-convention `closes #N`
/ `refs #N` in commit bodies — Gitea auto-closes the issue on
push. Captured in user-level CLAUDE.md (~/.claude/CLAUDE.md, not
in this commit) as the durable rule; no hook enforcement.

In-repo changes:

- docs/roadmap.md deleted.
- CLAUDE.md (project): Code-layout drops roadmap; /boss gating
  retargeted; Roles section rewritten with a new "Gitea issues"
  bullet (URL + tea-CLI snippet) and the closes-#N trailer note.
- skills/boss/SKILL.md: 10 sites retargeted, plus Step 1 now
  prescribes `tea issues ls --labels prio:p1` as the queue read.
- skills/brainstorm/SKILL.md: Step 7.5 no-override BLOCK now
  files a Gitea issue via `tea issues create` instead of
  appending a roadmap entry; spec deletion stays.
- skills/audit/SKILL.md + ailang-architect.md: deferral
  requirement and debt-heuristic retargeted; forward-intent
  belongs in the Gitea backlog.
- skills/fieldtest/SKILL.md, skills/docwriter/SKILL.md +
  ailang-docwriter.md: roadmap → backlog.
- design/contracts/honesty-rule.md: forward intent lives in
  the Gitea backlog (pinned phrases unchanged).
- design/INDEX.md: Docs bullet drops roadmap, adds the backlog
  URL.
- crates/ailang-core/tests/docs_honesty_pin.rs: two assert
  messages retargeted (assertion bodies unchanged).
- bench/architect_sweeps.sh: Sweep-4 TABU extended with
  `docs/roadmap\.md` so the deleted path cannot quietly regrow
  as a cross-reference in design/contracts/.

Verification:

- cargo build --workspace clean.
- cargo test --workspace: 647 passed, 0 failed, 2 ignored.
- bench/architect_sweeps.sh exit 0 (all five sweeps clean, incl.
  new TABU).
- grep over the live tree (excluding docs/specs/, docs/plans/)
  shows zero residual docs/roadmap.md refs.

Not touched: ~55 historical files under docs/specs/ and
docs/plans/ that mention docs/roadmap.md. Snapshot-character,
analogous to the JOURNAL-cut precedent — historical specs are
not mass-edited just because a live file was retired; their
mentions were correct at write time.
2026-05-20 14:48:27 +02:00
Brummel 99df14c792 workflow: scrub residual JOURNAL/journal refs missed in the first sweep
A re-grep found 15 live references (excluding docs/specs/ and
docs/plans/, which stay historical) the previous two commits
missed — all in source comments, doctests, bench helpers, and one
agent file.

Per-file:
- crates/ail/src/main.rs: typeclass-coherence diagnostic comment
  pointed at "the JOURNAL queue's wording" — points at
  design/contracts/typeclasses.md alone.
- crates/ailang-prose/src/lib.rs: `//!` header referred to
  docs/JOURNAL.md "Pinned: human-readable prose surface" —
  retargeted to design/contracts/authoring-surface.md.
- crates/ailang-check/src/lib.rs: bugfix tag + "see iter
  method-dispatch-refactor journal" prose collapsed to a clean
  rationale paragraph; the canonical shape is stated inline.
- crates/ailang-surface/tests/prelude_decouple_carve_out_pin.rs:
  "(d) record the rationale in a per-iter journal" →
  "(d) record the rationale in the commit body".
- crates/ailang-surface/tests/prelude_module_hash_pin.rs: header
  collapsed (pd.2/pd.3 milestone narrative dropped — the test's
  purpose is self-evident from its body); both "per-iter journal"
  drift-instructions point at the commit body.
- runtime/rc.c: "bench numbers in JOURNAL 18f.2" →
  "original profiling bench numbers".
- bench/run.sh: two "JOURNAL entry" comments → "commit body".
- bench/architect_sweeps.sh: header rewritten (cross-ref to the
  design-md-consolidation milestone commits, not a JOURNAL entry).
  Sweep-4 extended with two new anti-regrowth phrases
  ("see the per-iter journal", "in a per-iter journal") so journal
  prose can't grow back into design/contracts/ silently.
- skills/audit/agents/ailang-architect.md: "unlike a journal it
  lives on main" → "since it lives on main".
- ail-embed/src/bin/timeshard_runner.rs: "records it in the
  close-out journal" → "prints it to stderr".
- ail-embed/tests/timeshard.rs: two "journal-only friction timing"
  → "stderr-only friction timing".

Verification (after the edit):
- `grep -rin '\bjournal\b'` against live tree returns exactly two
  hits: the Sweep-4 regex itself (intentional — TABU phrases that
  prevent regrowth) and the PHRASES array in design_index_pin.rs
  (intentional — the same regrowth guard at test level). Both are
  load-bearing negative assertions.
- `bash bench/architect_sweeps.sh` exits 0 ("All five sweeps
  clean") — no design/-side regrowth.
- `cargo build --workspace` green; `cargo test --workspace` green.
2026-05-20 11:32:37 +02:00
Brummel 54f0ced148 workflow: delete docs/journals/ and docs/journal-archive.md
Strict application of the "Future-Use, not Verlauf" criterion to the
two remaining journal artefacts:

- `docs/journals/` (110 files): the per-iter and audit journals from
  2026-05-11 onward. Pure history; no live reader after the previous
  sweep. Entire directory removed.
- `docs/journals/2026-05-19-design-decision-records.md`: the one file
  that had live readers (`docs_honesty_pin.rs:108`, `parse.rs:80`,
  `duplicate_ctor_pin.rs:8`, three roadmap.md mentions) and was framed
  as a "relitigation guard". On re-examination its three asserted
  pinned phrases ("Regions were considered and rejected", the
  "demands annotations *because*" rationale, the prose-render
  placeholder statement) are rationale-prose, not load-bearing
  invariants — the test pinned itself, not a system property. Any
  Decision that still holds lives in the code, `design/contracts/`,
  or `design/models/`. Removed with the rest.
- `docs/journal-archive.md` (pre-2026-05-11 history): content-frozen
  long-tail history with no live reader. Removed; if anyone ever
  needs the pre-cutoff rationale they can `git log --before=2026-05-11
  --grep=<keyword>`.

Live-file consequences:
- `docs_honesty_pin.rs` `design_md_present_tense_anchors_present`:
  the three `records.*` assertions and the `read("docs/journals/…")`
  are dropped; the contract/model anchor pins remain.
- `duplicate_ctor_pin.rs`: the "why-two-overlays rationale lives in
  docs/journals/…" comment is replaced with the rationale inline.
- `parse.rs`: the "form-refinement rationale lives in docs/journals/…"
  comment is dropped (the rule body already states the refinement).
- `roadmap.md`: the decision-records mention in the DESIGN.md →
  design/ entry's body is dropped; the journal-archive.md `context:`
  pointers across ~12 closed entries are either rephrased to point
  at the relevant iter/audit (no path), or simplified to a one-line
  comment when the iter name alone carries the story.
- `CLAUDE.md` Roles section: the `docs/journal-archive.md` slot is
  removed; vocabulary note rephrased.
- `design/INDEX.md`: the Docs bullet drops `journal-archive.md`.
- `skills/README.md` bootstrap-rationale pointer rephrased.

`docs/specs/` and `docs/plans/` (per-milestone specs and per-iteration
plans) are unaffected — they remain the structured-design artefacts
they were before this sweep.

Workspace builds, full test suite green.
2026-05-20 11:25:15 +02:00
Brummel 8e586f493f workflow: replace per-iter journal system with git log + BLOCKED.md
The per-iter journal under docs/journals/ duplicated the iter commit
body's substance and accumulated as Verlauf-Doku with no Future-Use.
Sweep across all live control documents: CLAUDE.md, the 7 SKILL.md
files, the 11 agent files, design/INDEX.md and the contracts/models
that referenced journals, docs/roadmap.md, and the handful of source
comments + tests that pointed at journal files for rationale.

Mechanism changes:
- Standing-reading-lists in every agent now read `git log -N --format=full`
  for recent project state, never per-iter journal files. The architect
  reads `git log <prev-milestone-close>..HEAD --format=full` for audit
  scope.
- implement-orchestrator no longer writes a journal file. DONE outcomes
  emit just code + stats; the end-report is the per-task summary the
  Boss uses to write the commit body. PARTIAL/BLOCKED outcomes emit
  BLOCKED.md at the repo root — uncommitted by convention, Boss removes
  on repair or discard. New iron-law line + four-rationalisation row
  + red-flag bullet codify it.
- audit ratify mechanic: --update-baseline is now paired with an explicit
  ratify paragraph in the audit-close commit body, not a separate
  JOURNAL ratify entry.
- design/contracts/honesty-rule.md: "history and rationale lives in
  docs/journals/" → "lives in git log (iter and audit commit bodies)".
  Pinned phrase preserved verbatim.
- CLAUDE.md "Roles of …" section reframed: design/, git log,
  journal-archive.md (content-frozen), roadmap.md, specs/, plans/.
  No docs/journals/ slot anymore.
- roadmap.md context-lines that pointed at per-iter journals are
  dropped where the spec/commit already carries the rationale, or
  rephrased to "shipped in the <iter> iter commit" / "docs/journal-
  archive.md (<date> entry)" for pre-2026-05-11 references.

What stays (this commit):
- docs/journals/ directory and contents are NOT touched. Removing the
  contents is a separate follow-up.
- docs/journals/2026-05-19-design-decision-records.md still has live
  readers (docs_honesty_pin.rs Z 108 + parse.rs + duplicate_ctor_pin.rs
  + 3 roadmap mentions) — also follow-up.
- docs/journal-archive.md still exists; its self-pointer header has
  been updated to drop the "see docs/journals/INDEX.md" mention.

Workspace builds, full test suite green.
2026-05-20 11:21:37 +02:00
Brummel 55ad0fa37f workflow: drop docs/WhatsNew.md (Notify carries done-state on its own)
WhatsNew.md duplicated the Notify text into a file the only reader
(the user) does not consult. Removed the file and trimmed every
reference in the live control docs (CLAUDE.md, skills/boss,
skills/implement). The editorial rules (no internals,
telegram-pragmatic, factual) are preserved in skills/boss as
notify-text discipline. Historical specs/plans/journals are not
rewritten — they show the contemporaneous state.
2026-05-20 10:27:33 +02:00
Brummel ac4d545570 source: scrub iter-code / Decision-N residue from inline comments
Follow-up to bcd4181: the remaining ~530 inline `//` and `///`
comments still carrying opaque shorthand are now reformulated to
their content phrases. The only surviving `iter-<code>` reference
in source is the literal filename
`docs/journals/2026-05-13-iter-mq.3.md` (a real journal file).

Sweep covered:
  - `// Iter X.Y: <text>` prefixes (Iter 13a / 14a / 14e / 15g-aux /
    16b.x / 16d / 16e / 18b / 18c.x / 18d.x / 18e / 18g.x / 19a /
    19a.1 / 19b / 20a / 20f / 22-floats.x / 22b.x / 22c / 23.x /
    24.1 / cli-diag-human / hs.x / str-concat / etc.) — fully
    removed; the descriptive text that followed each prefix stays.
  - `// (Decision N)` and `per Decision N` and `Decision N axis 3` —
    replaced with the content phrase plus the relevant contract
    file (`design/contracts/tail-calls.md` for Decision 8,
    `design/contracts/memory-model.md` for Decision 10,
    `design/contracts/typeclasses.md` and `design/models/typeclasses.md`
    for Decision 11, `design/contracts/authoring-surface.md` for
    Decision 6, "the transitional dual-allocator" for Decision 9,
    "Effect prose" for Decision 3).
  - `// mq.X / mq.X (Task N) / mq.X journal / mq.X invariant` ->
    "the canonical-class-form rule / Class-class repurpose /
    method-dispatch-refactor journal / canonical-class-form
    invariant".
  - `// ct.X / ct.X (canonical-type-names) / ct.1.5a + ctt.2 /
    ct.2 Task N` -> "the canonical-form rule for type references /
    the canonical-form normalisation step / canonical-type-lookup
    refactor".
  - `// eob.X` -> "heap-Str-ABI" / "the Str carve-out".
  - `// rpe.X` -> "the per-type-print-op retirement".
  - `// post-mq.X / Pre-ct.X / pre-mq.X` -> "post-canonical-class-form" /
    "Pre-canonical-type-form" etc.
  - `/// Iter X regression: / /// Iter X.Y: / /// Iter A arm-close /
    /// Iter ct.4 (...): / /// Iter rpe.1 ...` -> descriptive
    phrases.

The journal filename in `crates/ailang-core/src/workspace.rs:573`
stays verbatim because it points at an actual file under
`docs/journals/`.

Tests: full `cargo test --workspace` green (80/80 test-result blocks
clean, no FAILED line). design_index_pin 5/5 + docs_honesty_pin 5/5
gating tests pass.
2026-05-20 09:58:04 +02:00
Brummel bcd41810f4 design/ + source rustdoc: replace opaque shorthand with content phrases + links
Reader-facing prose and rustdoc carried opaque shorthand like
"Decision 10", "clause-5", "mq.1", "ct.1", "eob.1", "rpe.1",
"post-mq.3", and "Iter 22b.1:" with no in-repo definition the reader
could follow. This commit replaces every such occurrence in the
durable tier the reader is most likely to land on (design/ ledger +
source //! module headers + the central /// public-item rustdoc) with
an inline content phrase plus, where applicable, a Markdown link to
the file that defines the referenced concept.

design/ ledger — 16 files:
  Definition-site headings demoted from "Decision N: <title>" to
  "<title>": authoring-surface, tail-calls, memory-model section in
  rc-uniqueness.md, dual-allocator section, typeclass design,
  effects "pure core + algebraic effects".
  Cross-reference sites: "Decision 1" -> canonical-schema principle
  (data-model); "Decision 3/4" -> effects + scope-boundaries; "Decision
  6" -> authoring-surface; "Decision 8" -> tail-calls; "Decision 9" ->
  rc-uniqueness (dual-allocator); "Decision 10" -> memory-model;
  "Decision 11" -> typeclasses (model). "clause-5" -> body-link
  durability gate. "clause-3" (in language-constraints) ->
  bug-class-reintroduction discriminator. "mq.1/2/3", "ct.1/4",
  "eob.1", "rpe.1" -> the canonical-form rule / the type-driven
  dispatch / the Str carve-out / etc. "post-mq.3" -> "type-driven".

design/contracts/feature-acceptance.md: file-local "clauses 1/2/3"
-> "criteria 1/2/3" (sprachliche Kohärenz mit der File-Überschrift
"Feature-acceptance criterion"); "the clause-3 mechanism" -> "the
bug-class-reintroduction discriminator".

Source //! module headers — 24 files:
  Stripped "Iter X.Y:" prefixes and "(Decision N)" / "(mq.X)" tags
  from spec_drift, uniqueness, reuse_shape, migrate_canonical_types,
  typeclass_22b{2,3,c}, suppress_filter, lift, mono, linearity,
  diagnostic, method_dispatch_pin, method_collision_pin,
  no_per_type_print_ops, mq3_multi_class_e2e, print_mono_body_shape,
  print_no_leak_pin, cli_diag_human_workspace_load_error,
  ct1_check_cli, prose snapshot, unbound_in_instance_method_pin,
  mono_xmod_ctor_pattern, desugar.

Central /// public-item rustdoc:
  ast.rs (full sweep — every "Iter X" + "Decision N" prefix
  reformulated; mode/Type::Fn rustdoc now points at memory-model.md;
  Constraint / SuperclassRef / InstanceDef / ClassDef rustdoc points
  at typeclasses contract).
  diagnostic.rs (all "(Iter X)" / "(mq.X)" tags on diagnostic codes
  removed).
  lib.rs (FORM_A_SPEC rustdoc points at authoring-surface.md
  instead of "Decision 6").
  canonical.rs (type_hash + Float-literal rustdoc).

Still outstanding (for a follow-up commit): ~500 inline `//`
code-body comments with `Iter X.Y` markers across the workspace, and
a handful of `///` rustdoc items in hash_pin / workspace_pin / lift /
mono / suppress_filter test-pin and internal-function bodies. Code
identifiers (test filenames like `mq3_multi_class_e2e.rs`, function
names like `iter18e_drop_iterative_default_preserves_hashes`) stay
verbatim per the user's "code identifiers stay verbatim" rule.

Tests: design_index_pin 5/5 + docs_honesty_pin 5/5; workspace builds
clean; full `cargo test --workspace` previously green (every
`test result: ok` line, no FAILED line).
2026-05-20 09:47:33 +02:00
Brummel 3e087d759a design/ ledger: prose-bridges for in-fence §-refs in data-model.md
Five §"..." cross-refs survived the main sweep because they live
inside jsonc Schema-comments — clause-5 fence-skip would treat any
[label](path) there as literal text, not as a navigable link.

Instead of touching the schema-comment style (it is self-documenting
where it sits, against the field it qualifies), this commit adds
prose paragraphs just outside the fence that turn the same
cross-references into real links:

  - Before the Def jsonc block: pointer to memory-model.md
    (canonical-form rule for cross-module class/type names) and
    embedding-abi.md (exported fn surface).
  - Before the Type jsonc block: pointer to memory-model.md for
    Type::Con.name scoping and Type::Fn parameter-mode metadata.

Also re-phrases an intra-file §"Str ABI" self-reference in
str-abi.md to "the Str ABI table below" — file-only granularity
makes a self-link sense-less; the reader wants intra-file
direction, which prose carries fine without a link.

Tests: design_index_pin (5/5) + docs_honesty_pin (5/5) stay green.
2026-05-20 00:46:37 +02:00
Brummel 19dc42f5ca design/ ledger: dense cross-linking via three topical splits + 88-link sweep
The first formal-links milestone shipped clause-5 + 8 links across the
existing file layout. Browsing surfaced that file-only granularity is
only as precise as the file boundaries — three files mixed two or
three navigation targets under one address, so the 8 links could not
multiply without ambiguity. This commit fixes the substrate, then
applies the sweep the original milestone deferred.

Splits (each extracts an already-self-contained section into its own
file so links land on the topic, not the parent doc's TOC):

  contracts/typeclasses.md
    → +contracts/prelude-classes.md  (Eq/Ord/Show ships, polymorphic `print`)
    → +contracts/method-dispatch.md  (5-step dispatch rule, candidate index)

  contracts/memory-model.md
    → +contracts/language-constraints.md  (the 4 binding constraints
                                           making RC sound without a
                                           cycle collector)

  models/authoring-surface.md
    → +models/prose-projection.md  (Form-B / `ail prose` / merge-prose)

Each new file enters design/INDEX.md as its own row (three contracts
share show_no_instance_e2e.rs / uniqueness.rs as ratifying tests;
prose-projection is a model). Two pre-existing links rebind to the
new topic-files (memory-model.md → method-dispatch.md;
float-semantics.md → prelude-classes.md).

Link sweep: 8 → 88 formal Markdown links over 23 files. Every file
in design/contracts/ + design/models/ now has at least one outgoing
link; the tree is fully connected. Links are file-relative
`[label](path)` per the established convention, fenced code blocks
are skipped (a `](` inside ```jsonc``` is literal text), the durable
tier (design/ + crates/ + runtime/) is enforced by clause-5.

Tests:
  - design_index_pin.rs (5/5 clauses): clean-cut, INDEX resolution,
    ratifying-test resolution, no decision-record prose in contracts/,
    body links durable + resolving.
  - docs_honesty_pin.rs (5/5): one assertion rebinds from typeclasses.md
    to prelude-classes.md (where the gated sentence now lives);
    design_corpus widens to include the 4 new files so the Wunschdenken
    / doc-archaeology sweeps continue to cover everything that used to
    live in the parents.

No spec/plan/journal for this batch — interactive collaboration after
the milestone closed; the user gated the splits explicitly before the
sweep.
2026-05-20 00:24:08 +02:00
Brummel cca6e3959d audit + close: design-ledger-formal-links CLEAN milestone close (no drift, bench 0/0/0, no tidy)
Cleanest close pattern in recent project history. Architect: zero
drift -- every commitment + invariant verified positively (clause-5
implements all 4 predicates incl. fence-skip, 8 link tokens match
the closed convert-set exactly, disposition-(b) applied to the 2
homeless pointers, honesty-rule positive-half pin-safe, commitments
4/7 byte-unchanged, composition invariant clause-3 ∧ clause-5 GREEN
simultaneously, out-of-scope set untouched incl. the 5 in-fence
data-model annotations, no commitment-2 violation in any reading
list, architect_sweeps clean). Bench: check.py exit 0 (63/0/0/63),
compile_check exit 0 (24/0/0/24), cross_lang exit 0 (25/0/0/25);
baseline pristine, no --update-baseline, no paired ratify.

The two spec amendments (clause-6 + cross-ref definition;
clause-5 fence-skip + closed convert-set enumeration) were the
discipline working: planner-recon and plan-corpus-fetch surfaced
gaps the brainstorm-sample missed; the spec was corpus-grounded
*before* any byte moved (grounding-check PASS x3); implementation
then ran clean 5/5 with one non-blocking concern (planner
self-review-item-8 arithmetic miss on the //! header line count --
substantive assertion unaffected; recorded as lesson).

No tidy iteration. No fieldtest -- zero authoring-surface change
(no .ail, no language construct, no compiler/checker/codegen path)
is the spec-stated reasoned exclusion, audit-confirmed.

This commit ratifies the milestone close:
- audit journal: docs/journals/2026-05-19-audit-design-ledger-formal-links.md
- journals/INDEX.md: audit pointer appended
- roadmap.md: [~] -> [x] CLOSED 2026-05-19 with full pipeline record
- WhatsNew.md: user-facing done-state entry (verbatim with notify)

Spec: docs/specs/2026-05-19-design-ledger-formal-links.md
Plan: docs/plans/design-ledger-formal-links.1.md
Iter journal: docs/journals/2026-05-19-iter-design-ledger-formal-links.1.md
Pipeline: brainstorm (PASS x3) -> planner (recon + plan +
self-review 8/8) -> implement (DONE 5/5, Boss-verified independently)
-> audit (CLEAN, no tidy).
2026-05-19 23:37:59 +02:00
Brummel 8ad91e7f24 iter design-ledger-formal-links.1 (DONE 5/5): clause-5 hard gate + 7 prose-ref conversions + 2 disposition-(b) homeless removals + honesty-rule positive-half (whole milestone in one iter)
Positive-half completion of the DESIGN.md -> design/ split: design/
body cross-references are now formal, file-relative Markdown links
into the durable tier (design/ or source), and a new in-tree hard
gate (design_index_pin.rs clause-5,
design_body_links_are_durable_and_resolve) walks every
design/contracts/*.md + design/models/*.md, strips fenced code
(strip_fences toggles on ```/~~~ lines so a ](  inside a fence is not
treated as a link), extracts every ](path), and asserts the target
resolves file-relative to a real file under design/-or-crates/-or-
runtime/; never docs/, never an in-file #anchor.

RED-first via identity-stubbed strip_fences (four embedded synthetic
vectors -- first one FAILS); replacing the stub with the real
toggle-on-fence impl turns the test GREEN. clause-5 composes with
clause-3 into the complete invariant the milestone establishes:
every contract cross-reference is EITHER a resolving durable
file-link OR clause-3-forbidden decision-record prose.

Conversions (recon-and-corpus-verified closed set):
  Task 2 (7 prose refs, 8 link tokens):
    float-semantics.md:69    Prelude classes -> [..](typeclasses.md)
    float-semantics.md:100   bare-path -> [Str ABI](str-abi.md)
    embedding-abi.md:45      "Frozen value layout" -> [..](frozen-value-layout.md)  (drop stale "below")
    memory-model.md:44       Data model -> [..](data-model.md)
    memory-model.md:105-106  Method dispatch -> [..](typeclasses.md)  (drop stale "below"; the target heading lives in typeclasses.md:227, not in this file)
    scope-boundaries.md:48   Str ABI -> [..](str-abi.md)
    scope-boundaries.md:88   mixed split: ailang-core::desugar -> source link + Pipeline -> ../models/pipeline.md (drop stale "above")
  Task 3 (2 disposition-(b) homeless removals):
    pipeline.md:60-61            (see docs/PROSE_ROUNDTRIP.md) pointer removed, CLI prose preserved
    authoring-surface.md:178-181 cross-tier pointer clause removed, ail merge-prose sentence preserved
  Task 4: honesty-rule.md positive-half paragraph inserted between L14 and the existing L15-blank-L16; both docs_honesty_pin.rs-pinned phrases byte-identical at L14/L19 (now shifted to L19 -> L25 by the +6 lines).

Out of scope, preserved (asserted independently): every intra-file
"above/below"; embedding-abi.md:51 "frozen value layout below
specifies" (no quoted title, no (see) form); data-model.md
38/66/79/206/226 (in-fence ```jsonc schema annotations -- the inline
analog of the nominal-mention carve-out). INDEX.md and the
decision-records journal byte-unchanged; clauses 1-4 of
design_index_pin.rs source byte-unchanged (the only `-` lines in
the diff are the two-line //! header rewrite Task 1 Step 5 itself
delivers).

Boss-verified independently (not on agent report alone):
  cargo test --workspace               647 passed / 0 failed
                                       (+1 vs pre-milestone 646:
                                        the new clause-5)
  cargo test --test design_index_pin   5 / 5 passed
  cargo test --test docs_honesty_pin   5 / 5 passed (additive
                                       paragraph is pin-safe)
  grep ](.../docs/.../) under design/  zero
  grep ](#)        under design/  zero
  ](-link count under design/          8 (closed convert-set)
  git diff --quiet design/INDEX.md     ok
  git diff --quiet decision-records    ok
  embedding-abi.md:48 pinned phrase    byte-identical

One Concerns item: Task-5 Step-7's plan-predicted "`-` line count = 1"
was actually 2 because Task 1 Step 5 rewrote the //! header 5 -> 8
lines (removing the original L4 + L5, not just L5). Planner self-
review-item-8 miss on my part -- a verification-arithmetic error in
the plan, NOT an implementation defect. The substantive assertion
(clauses 1-4 source byte-unchanged) is fully satisfied; the
implementer correctly flagged it and proceeded. The plan stands as
written; the assertion's `1` should have been `2`. Lesson noted for
future header-rewrite tasks.

Spec: docs/specs/2026-05-19-design-ledger-formal-links.md
(grounding-check PASS x3 across two corpus-grounded amendments --
clause-6 + cross-ref definition; clause-5 fence-skip + closed
convert-set enumeration).

Next: mandatory milestone-close audit (no fieldtest -- zero
authoring-surface change, reasoned exclusion).
2026-05-19 23:31:30 +02:00
Brummel 36599bedd9 plan: design-ledger-formal-links.1 — clause-5 + 7 conversions + 2 disposition-(b) + honesty-rule positive-half
Single iteration covering the whole milestone (same shape as
rolesplit.1; spec is small, no build-atomicity issue — clause-5 is
isolated additive code, prose edits don't affect compilation).
Five tasks, each at task-level review granularity:

  Task 1: clause-5 added to design_index_pin.rs RED-first via
          identity-stubbed strip_fences (synthetic vector FAILS) →
          correct toggle-on-fence impl (GREEN); module //!-header
          extended to name clause-5. Test count 4 → 5.
  Task 2: 7 prose-ref conversions to file-relative Markdown links
          (float-semantics:69/100, embedding-abi:45, memory-model:44/
          105, scope-boundaries:48/88 with the source+Pipeline split).
          Exact verbatim before→after for every line. clause-5 stays
          GREEN by construction (every introduced link resolves
          durable).
  Task 3: 2 disposition-(b) homeless-ref removals (pipeline:60-61
          in-fence CLI block + authoring-surface:178-181 prose).
          Pointer dropped, behavioural prose preserved.
  Task 4: honesty-rule.md pin-safe positive-half paragraph between
          L14 and the existing L15-blank (the two docs_honesty_pin
          phrases at L14/L19 each stay verbatim on their physical
          line; the additive insertion splits neither). The new
          paragraph names design_index_pin.rs clause-5 by name
          (nominal mention, not a link — consistent with the
          existing 'Ratified by:' footer style and with §Scope).
  Task 5: Whole-suite gate (646 → 647) + 5 grep+diff verifications
          (no docs/ link, no #fragment, link count = 8, in-fence
          schema-annotations intact, INDEX.md + decision-records
          journal + clauses 1-4 byte-unchanged).

Self-review 8/8: spec coverage complete; zero placeholders; names
consistent; bite-sized; no commit steps; pin contiguity verified
(L14 + L19 stay verbatim on their physical lines, replacement body
shows L14 contiguous); no signature change ⇒ no compile-gate
ordering issue; every cargo-test filter substring is a real test
name + unfiltered runs have explicit count assertions.
2026-05-19 23:21:29 +02:00
Brummel 42ff44adf6 spec: design-ledger-formal-links — corpus-grounded amendment (clause-5 fence-skip + closed enumeration)
Plan-time corpus verification surfaced a third distinct gap the
brainstorm-sample missed: data-model.md:38/66/79/206/226 are 'see §"..."'
annotations inside fenced jsonc code blocks (verified: fences at
30-87 and 203-228; the 5 refs fall inside), where a Markdown link is
literal text — not a navigable link on any renderer.

Per the "two+ surfaced ⇒ ground the spec properly once, don't iter-patch"
discipline: amendment 2 is the definitive corpus-grounded pass, not a
fourth-patch returning later. Every convert-set ref's prose-vs-fence
status + exact bytes personally verified before amending.

- clause-5 (Concrete a): adds a strip_fences helper toggling on
  triple-backtick / tilde lines; the per-file scan runs
  targets(&strip_fences(&raw)); the RED-first synthetic vector gains
  two fence-skip assertions (identity-stub of strip_fences ⇒ first
  assert fails ⇒ genuine RED). Contract grows from 3 to 4 predicates
  (+ fenced-code-not-scanned).
- Scope section: third out-of-scope carve-out — a 'see §"..."' inside
  a fenced code block is schema-example documentation = the inline-
  annotation analog of the nominal-mention carve-out (a // comment in
  a code example is code documentation, not a 'go browse there'
  pointer). Clean extension of the converged navigational-vs-nominal
  principle.
- Acceptance 3: convert-set CLOSED and exhaustive — 8 prose refs
  (float-semantics:69/100, embedding-abi:45, memory-model:44/105,
  scope-boundaries:48/88), 2 disposition-(b) homeless removals
  (pipeline:61, authoring-surface:180), 6 stay-prose (data-model
  in-fence x5, embedding-abi:51). Iter-provenance suffixes stripped
  on conversion. scope-boundaries:88 splits into a source link plus a
  Pipeline cross-file link.

grounding-check third dispatch on the amended bytes: PASS, all 8
assumptions ratified (incl. corpus-state-verified A4-A7 per the
rolesplit precedent for text-enumeration claims).
2026-05-19 23:14:56 +02:00
Brummel b1a0364bf2 spec: design-ledger-formal-links — recon-driven amendment (clause-6 + cross-ref definition)
Planner Step-2 recon (ailang-plan-recon) surfaced two spec defects;
forward-fix amendment (63b669f stands, main forward-only):

- clause-6 homeless-ref remedy was an incomplete binary. Recon proved
  roundtrip-invariant.md does NOT carry the ail-merge-prose-cycle
  content that pipeline.md:61 / authoring-surface.md:180 cite — so the
  prior canonical retarget sample was wrong. Replaced with three
  priority-ordered dispositions; PROSE_ROUNDTRIP refs take (b) (drop
  the cross-tier pointer, preserve the present-tense behavioural prose).
- Added a precise 'what counts as a cross-reference' definition:
  navigational pointer = in scope; nominal source-symbol mention = out
  of scope (keeps the milestone 'formalise the cross-references', not
  'hyperlink the ledger'). Source link only for a genuine navigational
  (see crates/...). Folds OQ2 (strip iter-provenance suffixes on
  conversion) + OQ3 (no-title directional stays prose) as byte-policies.

grounding-check re-dispatched on the amended bytes: PASS, all
assumptions ratified incl. the amendment-introduced ones.
2026-05-19 23:03:03 +02:00
Brummel 63b669fa8f spec: design-ledger-formal-links — formal cross-links in the design/ ledger
Positive completion of the DESIGN.md -> design/ split: informal prose
cross-references become formal file-relative Markdown links, gated by a
new RED-first design_index_pin.rs clause-5 (every design/ body link
resolves into the durable tier or fails the build). INDEX spine stays
repo-root-relative (registry tier, clause-1 unchanged). No authoring
surface. Eight converged commitments + the user-decided INDEX sub-fork;
grounding-check PASS 7/7. roadmap [~] in-flight.
2026-05-19 22:51:21 +02:00
Brummel 2ee408770e roadmap: close DESIGN.md->design/ ([~]->[x]); WhatsNew: design-spec-restructure done-state 2026-05-19 13:46:14 +02:00
Brummel f683f1aec8 iter design-md-rolesplit.tidy (DONE 7/7): resolve milestone-close audit drift
Gate-first TDD: widened design_index_pin.rs clause-3 to a hand-rolled
FAITHFUL Sweep-1 superset (case-sensitive digit-anchored line anchors
+ Sweep-1's ^[^/]* path-excluded date + the audit-named
decision-record phrases, case-insensitive); no regex dep; the blanket
iter-detector rejected as unworkable. Sentence-level strip of
faithfully-migrated history/decision-record prose out of 5 contract
files (the audit's 3 spot-checked + roundtrip-invariant.md +
data-model.md the exhaustive scan found) into the decision-record
journal, each replaced by its present-tense contract equivalent;
float-semantics.md stale 'see Str ABI below' -> str-abi.md;
architect_sweeps honesty sweeps re-scoped to design/contracts only
(models/ is the narrative tier) + ailang-architect.md lockstep.
Invariant: clause-3 GREEN => Sweep-1 clean in contracts/.

The prior dispatch correctly BLOCKED on a real plan defect (iso_date
lacked Sweep-1's path-exclusion, over-firing on legit
docs/specs/2026-.. citations); per the two+-defects-in-one-iteration
discipline the audit Resolution mechanism+scope were corrected
upstream in lockstep (f2cdd67) before re-dispatch, not patched a
third time.

Boss-verified independently: cargo test --workspace 646/0,
design_index_pin 4/4 (clause-3 RED->GREEN), architect_sweeps.sh exit
0 'All five sweeps clean' (acceptance criterion 9 met), acceptance
grep CLEAN, 3 docs_honesty_pin pinned runs each exactly 1 contiguous
match. Zero spec/quality re-loops. FINAL design-md-rolesplit
iteration — milestone functionally complete, audited, drift-resolved,
hard gate enforces the honesty spirit.
2026-05-19 13:44:37 +02:00
Brummel f2cdd67e69 plan+audit: design-md-rolesplit.tidy re-derived on planning-time evidence (mechanism + scope corrected)
The tidy's gate-first Task 1 RED-verification + Boss independent
verification surfaced two defects in the audit Resolution as first
written — caught exactly where the planner->Boss loop is designed to,
before any contract byte moved. Per the 'two+ defects in one
iteration => fix the upstream artifact, do not patch a third time'
discipline, the audit Resolution is corrected in lockstep with the
plan re-derivation:

- Mechanism (Resolution-4): the 'case-insensitive iter-code regex'
  clause-3 design is REJECTED as unworkable — a blanket
  iter/milestone detector conflates the memory-model rule-names
  'Iter A'/'Iter B' and ordinary words 'pre-existing'/'pre-set'/
  'pre-Boehm' with provenance, over-firing on legit present-tense
  contract prose (4 no-strip files). Replaced by a FAITHFUL Sweep-1
  superset: case-sensitive digit-anchored Sweep-1 line anchors
  (confirmed ZERO across all contracts) + Sweep-1's ^[^/]*
  path-excluded date (so docs/specs/2026-.. citations are not
  flagged — repairs the iso_date over-fire the implement
  orchestrator correctly BLOCKED on) + the audit-named
  decision-record phrases (case-insensitive). No regex dep. The
  load-bearing invariant (clause-3 GREEN => Sweep-1 clean) is
  preserved; it never required the blanket detector.
- Scope (Resolution-1): the architect's [medium] was a 3-file
  spot-check; the exhaustive scan found roundtrip-invariant.md:73
  ('at iter form-a.1') and data-model.md:149,161 ('loop-recur iter
  1:') also carry lowercase provenance. True strip set = 5 files;
  plan Task 5b adds the 2 (spirit-cleanup, not gate-blocking).

Plan self-review 8/8 re-run; clause-3 = hand-rolled faithful Sweep-1
superset, no regex dep, no blanket iter detector.
2026-05-19 13:36:14 +02:00
Brummel 4f61a7aa4f plan: design-md-rolesplit.tidy — audit drift fix (history->journal, sweep re-scope, clause-3 widen)
7 tasks executing the audit Resolution's 5 points: (1) gate-first
RED — widen design_index_pin.rs clause-3 to a hand-rolled PROVABLE
SUPERSET of architect_sweeps Sweep-1 over design/contracts/ (no
regex dep; subsumption proof inline), verify RED vs the un-stripped
tree; (2-4) sentence-level strip the faithfully-migrated
history/decision-record prose out of typeclasses.md / str-abi.md /
scope-boundaries.md into the decision-record journal, per recon's
per-sentence contract-residue map, each docs_honesty_pin pinned run
guarded contiguous; (5) fix float-semantics.md stale 'see Str ABI
below' -> str-abi.md; (6) re-scope architect_sweeps DESIGN_GLOB to
design/contracts only (models/ is the narrative tier) + lockstep
ailang-architect.md; (7) whole-tree GREEN gate (clause-3 RED->GREEN,
4/4, sweep exit 0, suite >=646/0, acceptance grep CLEAN).
Self-review 8/8.
2026-05-19 13:22:22 +02:00
Brummel 2ba5e16806 audit design-md-rolesplit (milestone close): DRIFT (one tidy iter) + bench causally-exonerated (baseline pristine)
Architect: relocation byte-faithful; one [medium] spirit-finding —
faithfully-migrated decision-record/history prose in
design/contracts/{typeclasses,str-abi,scope-boundaries}.md dodges the
6 literal clause-3 markers but is the relitigation content the
split's spirit sends to journals; [low] float-semantics.md
stale-direction 'see Str ABI below'. Both routed items: tidy not
ratify (the str-abi.md:23 advisory Sweep-1 exit-1 correctly
diagnoses real pre-existing history residue, byte-identical
DESIGN.md@deeffb1 — faithfully migrated, not split-introduced).

Bencher: causally-exonerated, DECISIVE on byte-evidence — emitted IR
AND final -O2 binaries byte-identical 176821c vs dd5b183 for all 6
check.py firings; all ~11 changed files audited
comment/docstring/diagnostic-string-only; tracked-P2 families;
compile_check/cross_lang pristine. NO-ratify, baseline pristine
(M2/M3/M5 disposition).

Resolution: one tidy iter (move history prose to the decision-record
journal; fix stale 'below'; re-scope architect_sweeps honesty sweeps
to design/contracts only — models/ is the narrative tier; widen
design_index_pin.rs clause-3 to subsume Sweep-1's history-anchor
regex over contracts/ so the hard gate enforces the spirit). No
fieldtest (zero authoring-surface change). Milestone closes after
the tidy lands Boss-verified.
2026-05-19 13:14:15 +02:00
Brummel 176821c2e7 iter design-md-rolesplit.1 (DONE 9/9): DESIGN.md -> design/ ledger role-split
The 3020-line docs/DESIGN.md is replaced by the design/ ledger:
design/INDEX.md (sole addressable spine, typed Contracts+Models tables,
polymorphic links — prose file OR authoritative source //!), 14
design/contracts/*.md test-linked invariants + 3 source-link-only
contracts (mangling/env-construction/qualified-xref, no prose file —
code is SoT), 5 design/models/*.md whitepapers, and
docs/journals/2026-05-19-design-decision-records.md (the
relitigation-guard archive — every why/rejected/does-not-do/rollback/
empirical ### moved out at ###-granularity). Clean cut: git rm
docs/DESIGN.md, no stub.

RED-first crates/ailang-core/tests/design_index_pin.rs — the 4-clause
anti-regrowth spine (DESIGN.md-gone / every-INDEX-link-resolves /
every-contract-names-a-resolvable-ratifier /
contracts-carry-no-decision-record-prose) — demonstrably RED before,
GREEN after. Build-atomic by task ordering: design_schema_drift.rs's
include_str! (the only compile-time consumer) retargeted to
design/contracts/data-model.md BEFORE the deletion; its
## Data model/## Pipeline slicer dropped (a simplification the split
enables). 2 NoInstance diagnostics + 2 lockstep E2Es retargeted to
design/contracts/{float-semantics,typeclasses}.md. ~12 agent reading
lists + 5 SKILL bodies + CLAUDE.md + skills/README.md + ~25
code/C/.ail/spec comment xrefs retargeted; OQ7 dangling 'Iter 13b'
cite deleted (no forward target — a pointer would be fiction).
honesty-rule.md rewritten so the rule names the new home
(rationale->journals), resolving the recon-found internal
contradiction; the two docs_honesty_pin.rs:70,72 pinned phrases kept
verbatim+contiguous.

Boss-verified independently: cargo test --workspace 646 passed /
0 failed; design_index_pin 4/4; acceptance grep CLEAN of live
DESIGN.md refs (residuals = only the spec-mandated clause-4
deletion-enforcer). 2 DONE_WITH_CONCERNS routed to the mandatory
milestone-close audit: (a) str-abi.md:23 '(iter str-concat,
2026-05-13)' provenance stamp trips advisory architect_sweeps Sweep-1
— Boss-confirmed byte-identical to DESIGN.md@deeffb1:2062-2065, a
faithfully-migrated PRE-EXISTING anchor (regexes verbatim, only path
retargeted), NOT split-introduced — RATIFY-or-tidy at audit; (b) a
now stale-direction intra-prose 'see Str ABI below' cross-ref in
float-semantics.md — audit-adjudication candidate. Plan defect noted:
Task 9 Step 4's verbatim acceptance grep used a ^./ anchor not
matching the system's grep -rIn output; substance re-verified CLEAN.

Spec grounding-check PASS x2. Journals INDEX + decision-records
pointer appended (Boss-only).
2026-05-19 13:04:22 +02:00
Brummel deeffb1872 plan: design-md-rolesplit.1 — the whole milestone, build-atomic by task ordering
9 tasks: (1) RED-first design_index_pin.rs 4-clause guard; (2)
design/INDEX.md + 14 prose contracts; (3) 5 model whitepapers; (4)
decision-record journal + INDEX pointer; (5) retarget the 3
doc-reading tests GREEN vs design/; (6) 2 diagnostics + 2 E2Es
lockstep; (7) architect_sweeps.sh spine repoint; (8) ~12 agent lists
+ 5 SKILL bodies + CLAUDE.md + README + bucket-(e) comment xrefs +
OQ7 cite deletion; (9) clean cut (git rm DESIGN.md) + whole-tree gate
+ acceptance grep. Build-atomicity by construction: include_str!
retarget (Task 5) precedes DESIGN.md deletion (Task 9), data-model.md
exists from Task 2 — every intermediate cargo build is green, no
monster task. Self-review 8/8 (contiguity + compile-gate-ordering +
filter-resolution scrubbed).
2026-05-19 12:16:21 +02:00
Brummel 314e5e4920 spec: design-md-rolesplit — Boss-adjudicated relocation appendix (planner recon resolution)
plan-recon surfaced 7 open questions (genuine Boss design judgement,
not architectural forks). Adjudicated and folded into the spec as an
authoritative placeholder-free Appendix relocation map (every ##/###
of DESIGN.md -> exactly one destination):

- design_index_pin.rs clause-2 widened to accept crates/**/src/*.rs
  in-source #[cfg(test)] mod tests as first-class ratifiers.
- INDEX ratifier tokens resolved to recon-verified paths; memory-model
  -> uniqueness.rs (in-source), tail-calls -> lib.rs:5044
  tail_call_in_non_tail_position_is_rejected (NOT loop_recur — different
  construct), env-construction path corrected to ailang-check/tests.
- 3 ledger additions the 12-list under-counted: qualified-xref
  (source-link only), str-abi (prose+source; the one sanctioned
  sentence-level move, 2802 bold para), scope-boundaries (present-tense
  honesty-pinned). Net 15 contracts / 5 models / 1 decision-record journal.
- OQ7: lib.rs:103 dangling 'Iter 13b' cite is deleted not retargeted
  (no forward target exists; a pointer would be fiction).

grounding-check re-dispatched after the material edit: PASS, all 6
changed/new ratifiers resolve to named currently-green tests.
2026-05-19 12:11:53 +02:00
Brummel a64b2ccb2a spec: design-md-rolesplit — DESIGN.md role-split into design/ contracts+models ledger
Role-split the 3020-line docs/DESIGN.md into design/contracts/ (the
hot, test-linked invariant set), design/models/ (one whitepaper per
model), and design/INDEX.md as the sole addressable spine; decision-
records rehomed to docs/journals/. Clean cut (DESIGN.md deleted, no
stub), ###-subsection relocation granularity, build-atomic landing
(design_schema_drift.rs include_str! forces one commit), polymorphic
INDEX links (code-SoT contracts point at source //!, no prose dup),
typed-INDEX anti-regrowth pin (design_index_pin.rs). grounding-check
PASS 13/13 after one re-dispatch (commitment-4 pin-status claim
corrected: Decision-6 :242 audit-trail sentence is pinned by no test,
retired to the decision-record journal, no successor pin by design).

roadmap: DESIGN.md->design/ marked [~] in flight (spec landed).
2026-05-19 11:58:07 +02:00
Brummel dd5b18378f docs: DESIGN.md §Embedding ABI — current-state honesty (no general-ABI claim)
The §"Embedding ABI" prose presented the M3 contract as "the
embedding ABI" — a general boundary — and justified the layout
freeze with "this is what codegen emits today". Both overclaim:

- The accepted set (bare scalar | single-ctor all-scalar record)
  is the *whole* contract, narrow by construction. The host
  hand-constructs and hand-reads the box layout directly; there
  is no general value-marshalling layer. The text now says this
  plainly instead of framing the set as a subset of a wider ABI.
- The freeze is reframed as the consequence of that direct
  layout exposure (the host is the contract surface), not as
  "frozen because that is what lower_ctor emits".
- "MUST NOT move these offsets" is corrected to "for an exported
  type" — the prior wording overclaimed the freeze as binding
  every record layout; the commitment binds exported types and
  thereby constrains codegen repacking of exported records.

Present-tense Ist-state mirror only: no lockstep/wire-contract
design, no post-mortem, no aspirational "should become". Schema
section untouched. Verified green: design_schema_drift (8),
docs_honesty_pin (5), effect_doc_honesty_pin (4),
embed_record_layout_pin (1), embed_staticlib_lowering (3).
2026-05-19 11:22:25 +02:00
Brummel d160f2c977 roadmap: close Embedding ABI — M5 ([~]->[x]) — M1–M5 arc complete, audited+ratified+doc-honest; WhatsNew + P2 reframed
M5 done 2026-05-19: real backtest over real Pepperstone tick data
through the real external data-server into the shipped M3-frozen
kernel, across a per-thread-ctx thread swarm; symbol-fan (headline
existence proof) + time-shard (first actual proof of chunk-boundary
invisibility); globally leak-free, deterministic. Zero
language/compiler change; one runtime change (7bfa11e atomic
global RC-stats fallback — user-adjudicated bounce-back, RED-first,
+ a build-dep staleness root-caused & fixed). Audit: Invariant 1
PASS, M3 frozen-layout unmoved, gc_rss trio evidenced-RATIFY
(Boehm conservative-scan artefact, Decision-9 transitional), P2
noise causally exonerated baseline-pristine, doc-honesty tidy
clean. Friction-harvest: host-per-tick-FFI ~= ~206 ns/tick at real
EURUSD volume.

- roadmap M5 [~]->[x] with the done-state summary (stays one cycle
  for context, then prune).
- roadmap P2 "Flat array/slice primitive" reframed: the
  M4-retirement forward note's deferred reconciliation is now
  done — the obsolete "1024-cell cons-chain per chunk / M4 ships
  it" framing replaced with the real residual (host-per-tick-FFI
  vs. batch crossing) anchored on M5's measured ~206 ns/tick; the
  M5-dependency is now satisfied (the justification input exists).
- WhatsNew.md done-state entry (verbatim == the notify text):
  user-facing, no internals — the embedding arc is complete.

Done-state notification fired (user away per /boss). The next
roadmap item is a new milestone (the P2 flat-array decision or
another P0) — a deliberate user call, not autonomous.
2026-05-19 03:10:04 +02:00
Brummel 70f2a318e0 iter embedding-abi-m5.tidy (DONE 3/3): milestone-close doc-honesty drift — pin-safe, doc/comment-only
Resolves the M5 milestone-close audit DRIFT (audit journal
28ab56a). Single cohesive commit, M2.tidy a80d495 / M3.tidy
63d7d60 precedent (pins are the coverage, no RED, no
audit/fieldtest gate).

- docs/DESIGN.md edit-1 §"Free (host side)": dropped the retired-M4
  forward-reference ("an additive M4 concern, not a contradiction
  of this freeze"; M4 retired 2026-05-18) -> present-tense current
  fact (a boxed-field record is not an M3 embedding type,
  export-gate-rejected; the freeze covers exactly the all-scalar
  single-ctor record).
- docs/DESIGN.md edit-2 §"Embedding ABI": reconciled the now-
  inaccurate "(no shared mutable runtime state - ... data-race-free,
  sanitiser-verified)" blanket with the real post-7bfa11e state:
  the per-allocation hot path (per-ctx counters + per-object
  refcount header) is non-atomic by design and never shared
  (Ctx:!Send, single-thread-per-ctx); the one shared datum is the
  atomic-relaxed global RC-stats fallback counter; "data-race-free,
  sanitiser-verified" retained (still true).
- runtime/rc.c:88 comment-only: stale "two unconditional ++
  operations" -> accurate "one counter bump per alloc/free ...
  relaxed atomic add on the global null-ctx fallback, a plain ++
  on the per-ctx path" (consistent with the adjacent already-
  correct :93-106 atomic block + :45-55 Threading header).

Boss-verified independently: all pins green (design_schema_drift
8/0, docs_honesty_pin 5/0, effect_doc_honesty_pin 4/0,
embed_record_layout_pin 1/0) = mechanical proof no pinned/hashed
byte moved; the adjacent separately-pinned bare-scalar sentence is
byte-identical (shifted 2299->2305 by net-added lines; substring
pin, passes); rc.c strictly comment-only (filter empty);
cargo build --workspace Finished; git scope = only docs/DESIGN.md
+ runtime/rc.c + journal/stats. Clears the M5-audit doc-honesty
debt; no new debt.

FINAL M5 iteration. The M1-M5 Embedding ABI arc is functionally
complete, audited, ratified, and doc-honest. Includes the per-iter
journal, stats, and INDEX line.
2026-05-19 03:07:40 +02:00
Brummel e81fb90d88 plan: embedding-abi-m5.tidy — milestone-close doc-honesty drift (pin-safe, doc/comment-only)
Resolves the M5 milestone-close audit DRIFT (audit journal
28ab56a). 3 tasks, doc/comment-only, recon-verified pin-safe:
T1 docs/DESIGN.md — drop the retired-M4 forward-reference
(§"Free (host side)" ~:2354-2360) + reconcile the "no shared
mutable runtime state … data-race-free" claim with the now-atomic
global RC-stats fallback (§"Embedding ABI" ~:2287-2298, must not
disturb the adjacent line-2299 pinned bare-scalar sentence);
T2 runtime/rc.c:88 — comment-only fix of the stale "two
unconditional ++" performance aside (consistent with the adjacent
already-corrected :93-106 atomic block); T3 consolidated
doc-only/pin-safe/scope gate. All 3 sites recon-verified OUTSIDE
every pinned/hashed region (design_schema_drift scans only the
post-line-2362 data-model slice; docs_honesty_pin:135 pins the
distinct line-2299 sentence; the IR-byte pin asserts generated IR
not rc.c comments). Pins green before+after = the mechanical
pin-safety proof; no RED, no audit/fieldtest gate — mirrors the
M2.tidy a80d495 / M3.tidy 63d7d60 precedent (single cohesive
commit, pins are the coverage). Last step before M5 fully closes.
2026-05-19 03:02:51 +02:00
Brummel 28ab56af7e audit embedding-abi-m5: milestone close — RATIFY gc_rss trio (evidenced) + DRIFT (one doc-honesty tidy); P2 bench noise causally exonerated, baseline pristine for it
Architect: Invariant 1 PASS (zero data-server in any compiler
crate/runtime; ail-embed its own workspace root); M3 frozen-layout
SSOT unmoved (the lone rc.c change 7bfa11e touched only the global
stats counters); rc.c header-comment honesty accurate. Drift =
[medium] DESIGN.md:2358-2360 stale "additive M4 concern" (PRE-
EXISTING — M4 retired; correctly scoped out of every M5 iter) +
[medium] DESIGN.md §"Embedding ABI" silent-drift (it asserts "no
shared mutable runtime state … data-race-free" but the now-atomic
global RC-stats fallback IS shared mutable state under concurrency;
M5 is AILang's first concurrent consumer) + [low] rc.c:88 "two
unconditional ++" imprecise → one doc-honesty tidy iteration
(embedding-abi-m5.tidy), same honesty axis, not split.

Bencher refuted my pre-audit "gc_rss allocator-orthogonal →
environmental" hypothesis WITH EVIDENCE: the 3x *.gc_rss_kb +32%
are genuine, deterministic, M5-attributable — 7bfa11e's atomic
split a branch in ailang_rc_alloc (reached by GC programs via
str.c int->str at print), spilling %r14/%rbx; a stale interior
list pointer in the new spill slot is conservatively retained by
Boehm, pinning a ~32 MB dead spine (collection #12 freed PRE 32 MB
vs HEAD 32 B); bisects exactly on the commit, IR byte-identical,
ail sha256-identical; RC/bump RSS dead-stable. bump_s + max_us are
pre-existing tracked-P2 noise (bump RC-orthogonal; max_us 1-sample
tail-jitter, PRE often worse, +65->+104% swing between runs) — NOT
M5.

RATIFY the gc_rss trio. Language reason (not effort): Decision 9 —
Boehm is explicitly transitional; AILang's committed model is RC +
uniqueness. The measurable cost of the correct, necessary,
user-approved swarm-safety fix is exclusively a Boehm
conservative-stack-scan false-retention artifact on the
transitional allocator (codegen-layout-fragile; no memory actually
leaked; RC/bump RSS dead-stable). Chasing it with an out-of-scope
codegen change invests in the path being migrated away from; the
committed RC path is unaffected. Selective baseline re-anchor:
bench/baseline.json lines 14/44/64 ONLY -> 137448/137768/137636
(bencher-confirmed steady-state); every other metric byte-pristine
(a wholesale --update-baseline would absorb the P2 noise and break
the established-envelope discipline). Re-run confirms the 3 now ok
at +-0.3%.

bump_s x2 + max_us x2: NO-ratify, causally exonerated, baseline
left pristine — identical disposition to the M2/M3/prelude-decouple
milestone closes.

Friction-harvest recorded (P2 flat-array-decision input):
host-per-tick-FFI ~= ~206 ns/tick at real EURUSD volume.

Ratify discipline: updated baseline JSON + this paired JOURNAL
ratify entry committed together (bench/baseline.json + the audit
journal + INDEX line). M5 closes after the doc-honesty tidy lands
and is Boss-verified.
2026-05-19 02:56:43 +02:00
Brummel 0900f3f413 iter embedding-abi-m5.3 (DONE 3/3): time-shard boundary-invisibility proof + friction harvest
Final functional M5 iteration (spec ae905de, plan 67027ab; on
green m5.1 204c171 + m5.2 b724cd1 whose leak-proof resolved via
dbd76e5). Proves the chunk/window-boundary-invisibility claim the
M4 retirement rests on:

- adapter::fold_window — purely additive windowed sibling of
  fold_symbol (same MidPriceStream+Kernel path; stream_tick_windowed).
  fold_symbol/MidPriceStream/Kernel byte-untouched.
- chrono as a DEV-dep only — the time-shard test derives
  (y,m)->Unix-ms bounds (data-server's own precedent) and passes
  raw ms to the bin; the bin is date-math-free; Invariant 1
  unaffected.
- new single-responsibility [[bin]] timeshard_runner — one thread
  per disjoint window, each owning its Kernel (Ctx:!Send =>
  compile-time per-thread-ctx).
- tests/timeshard.rs — spec §3 strength-ordered: (a) per-shard
  (acc,n) BIT-EXACT vs single-thread host fold of that exact window
  in stream order (THE boundary-invisibility proof); (b) Σ partials
  bit-exact by construction; (c) whole-window within REL_TOL=1e-6
  only (cross-shard f64 reassociation is host arithmetic, never a
  kernel property — bit-exact there would be the bug; tolerance
  derived from the recursive-summation error bound). Plus the
  now-deterministic leak-Σ + a journal-only friction timing capture
  (no bench gate, no timing assertion).

Concrete pin: EURUSD 2017-03/04/05 (recon-verified present,
contiguous, no gaps; data-server has no month-listing API). RED was
a genuine deterministic compile failure (declared-but-absent bin
source; Cargo missing-source error vs the plan's predicted env!
error — same cause, no false-green; benign plan-text imprecision in
Concerns).

Boss-verified independently: timeshard determinism 5/5 (no jitter);
full ail-embed suite 0 failed AND 0 ignored across all binaries
(m5.2 symbol_fan_swarm_leak_free stays un-ignored & green — the
saga's resolution intact); isolated embed_rc_global_stats_race
still green; AILang workspace data-server count 0; compiler-surface
diff empty (zero diff to crates/ailang-*, crates/ail/, runtime/,
examples/*.ail, root Cargo.toml, DESIGN.md).

Friction-harvest deliverable (P2 flat-array-decision input):
host-per-tick-FFI ~= 658 ms / 3,192,562 ticks ~= ~206 ns/tick at
real EURUSD volume.

M5 functional work complete; milestone closes next via the
mandatory audit (which also handles the pre-existing
DESIGN.md:2358-2360 "additive M4 concern" drift, out of m5.3 scope).
Includes the per-iter journal, stats, and the INDEX.md line.
2026-05-19 02:40:16 +02:00
Brummel 67027ab0f9 plan: embedding-abi-m5.3 — time-shard boundary-invisibility proof + friction harvest
Final functional M5 iteration (spec ae905de; builds on green
m5.1 204c171 + m5.2 b724cd1 whose leak-proof is resolved through
dbd76e5). 3 tasks: (1) additive adapter::fold_window (windowed
sibling of fold_symbol) + chrono dev-dep + a second
[[bin]] timeshard_runner declaration; (2) tests/timeshard.rs
written FIRST — RED is the deterministic `env!(CARGO_BIN_EXE_
timeshard_runner) not defined` compile failure (bin absent); (3)
the timeshard_runner bin → GREEN, with the spec §3 strength-ordered
assertions (a) per-shard bit-exact vs single-thread windowed host
fold (THE boundary-invisibility proof), (b) Σ bit-exact by
construction, (c) whole-window within REL_TOL=1e-6 (derived from
the recursive-summation error bound, never bit-exact — spec
explicit), plus the now-deterministic leak-Σ and a journal-only
friction-timing capture.

Boss decisions in-plan: new bin not an arg-mode (shipped symbol-fan
path byte-untouched, no regression surface); chrono dev-only (date
math in the test, bin date-math-free; Invariant 1 unaffected);
concrete pin EURUSD 2017_03/04/05 (recon-verified present, no
gaps; data-server has no month-listing API); friction is a journal
observation, no bench gate, no timing assertion (flake-free);
pre-existing DESIGN.md:2358-2360 M4-drift explicitly out of scope.
Milestone close + mandatory audit run after Boss-verification.
2026-05-19 02:30:46 +02:00
Brummel dbd76e5503 fix(ail-embed): GREEN — build.rs tracks runtime/ as rerun-if-changed; M5 swarm leak-proof armed & deterministic
bugfix-swarm-rc-alloc-undercount, GREEN stage. RED is the separate
audit-trail commit 483117d.

Root cause of the m5.2-resume-attempt residual: ail-embed/build.rs
declared cargo:rerun-if-changed only for the kernel .ail + build.rs
itself — NOT the runtime/ C sources ail build --emit=staticlib
compiles into libailang_rt.a. Cargo never re-ran build.rs after the
atomic-counter fix 7bfa11e, so ail-embed kept linking a stale
pre-7bfa11e non-atomic libailang_rt.a; the swarm's concurrent
TLS-NULL host allocs raced the stale non-atomic g_rc_alloc_count++
(frees-stable/allocs-jitter). NOT a second runtime bug and NOT
test-methodology unsoundness — runtime/rc.c at HEAD is correct; the
isolated crates/ail RED was green only because it rebuilds the
staticlib fresh per run.

Fix (ail-embed/build.rs, +7 lines): directory-level
cargo:rerun-if-changed=<repo>/runtime (Cargo recurses mtimes under
the dir). Preferred over a per-file list — the implementer inspected
crates/ail build_staticlib (libailang_rt.a = ar(rc.o, str.o), both
under runtime/) and chose the directory mechanism so any future
runtime source cannot silently re-introduce the same staleness.
Existing kernel/build.rs/AIL_BIN rerun-if-changed lines kept.

Cohesive consequence (in-scope): symbol_fan_swarm_leak_free
un-#[ignore]d — the test BODY is byte-for-byte unchanged (it was
quarantined, never weakened); the module-doc + fn-doc breadcrumbs
rewritten to the resolved build-dep-staleness rationale. It is now
the integration-level acceptance of both 7bfa11e and this fix. Boss
also removed a pre-existing unused `DataFormat` import in swarm.rs
inline (trivial; the file is committed this iter regardless;
warning-clean after).

Boss-verified independently: RED rt_archive_freshness -> GREEN;
swarm determinism 5/5 (symbol_fan_swarm_leak_free +
symbol_fan_swarm_bit_exact GREEN every run, 0 ignored, jitter gone,
Sallocs==Sfrees==12000003); isolated embed_rc_global_stats_race
still GREEN (untouched, runtime untouched); Invariant 1 data-server
count 0; scope = ail-embed/build.rs + ail-embed/tests/swarm.rs +
journal + stats only.

The M5 swarm leak-proof bounce-back is fully resolved end-to-end.
M5 stays open [~]; m5.3 (time-shard + friction-harvest + close-out)
remains. Includes the per-iter journal, stats, and INDEX.md line.
2026-05-19 02:21:47 +02:00
Brummel 483117d39b test(ail-embed): RED — build.rs links a stale libailang_rt.a (missing rerun-if-changed on runtime sources)
bugfix-swarm-rc-alloc-undercount, RED stage (audit trail; GREEN
follows separately via implement mini-mode).

Root cause of the m5.2-resume-attempt residual (the M5 swarm
leak-proof still non-deterministic after the atomic-counter fix
7bfa11e): ail-embed/build.rs declares cargo:rerun-if-changed only
for the kernel .ail + build.rs itself — NOT runtime/rc.c. So Cargo
never re-ran build.rs after 7bfa11e; the swarm keeps linking a
stale pre-7bfa11e libailang_rt.a whose g_rc_*count++ is the old
non-atomic racing increment. 7bfa11e was correct and is unchanged;
it just never reached the ail-embed binary. This explains why the
isolated crates/ail RED (rebuilds the staticlib fresh per run) is
green while the ail-embed swarm is not, and the
frees-stable/allocs-jitter asymmetry (few TLS-NULL frees = no
contention; millions of TLS-NULL host allocs = race on the stale
non-atomic counter).

Framing: a genuine defect (build-dependency completeness gap), NOT
test-methodology unsoundness — with a fresh archive the swarm
Σ-over-stat-lines balances deterministically (Σallocs==Σfrees==
12000003, 5/5 per the debugger). runtime/rc.c at HEAD is correct;
the GREEN fix is in the build script, not the runtime.

RED pins the CAUSE deterministically (the symptom is
non-deterministic by construction): objdump the libailang_rt.a that
build.rs emitted for this crate; assert >=2 lock-prefixed
instructions (the atomic lock incq for g_rc_alloc_count +
g_rc_free_count from current rc.c) and that it matches a
freshly-built archive. Boss-verified RED: linked archive has 0 lock
insns ("STALE pre-7bfa11e archive linked").

Does NOT touch the committed #[ignore] of symbol_fan_swarm_leak_free
nor the isolated embed_rc_global_stats_race RED (both correct,
unchanged).
2026-05-19 02:14:12 +02:00
Brummel 6500ca0493 ail-embed: re-quarantine symbol_fan_swarm_leak_free — atomic-global fix necessary but insufficient (doc-honesty + finding record)
Attempted to un-#[ignore] the M5 swarm leak-proof as the
integration-level acceptance of bugfix-rc-global-stats-race
(7bfa11e). Boss verification falsified the sufficiency premise:
across 4-6 swarm runs the leak Σ is STILL non-deterministic —
Σfrees stable-exact (12000003) every run, Σallocs short by a
jittering ~1600-2260 in ~1/3 of runs (e.g. left:11998383
right:12000003, 4 stat lines).

The atomic-global runtime fix (7bfa11e) is real, committed, and
NECESSARY — the isolated 8x2M pure-global RED
embed_rc_global_stats_race is deterministically green. It is NOT
sufficient for the swarm: a second, distinct alloc-side undercount
remains that the isolated RED did not model (Σfrees never loses =>
not a per-ctx race; not pure global contention => RED green).

The un-#[ignore] was reverted (never committed). The ONLY shipped
diff is the three now-stale #[ignore]-rationale breadcrumbs in
ail-embed/tests/swarm.rs corrected from "un-ignore when the runtime
makes the counters atomic" to the accurate necessary-but-insufficient
state — a future agent seeing 7bfa11e must NOT un-ignore on that
basis (doc-honesty rule). The test body and the #[ignore] attribute
itself are unchanged: still quarantined, NOT weakened.
symbol_fan_swarm_bit_exact stays live and GREEN — the swarm IS
actually correct and leak-free; only the accounting is wrong.

Residual handed to a fresh debug RED-first cycle. Two framings
recorded for it (genuine residual runtime defect vs Σ-over-
heterogeneous-lines test-methodology unsoundness; the latter, if so,
is a post-root-cause design decision). main stays green; M5 open
[~]; m5.3 remains downstream of a sound leak-proof.

Includes the resume-attempt journal + INDEX line.
2026-05-19 02:06:43 +02:00
Brummel 7bfa11e838 fix(rc): GREEN — atomic global g_rc_* stats fallback counters (swarm-safe)
bugfix-rc-global-stats-race, GREEN stage. RED is the separate
audit-trail commit 427b687.

g_rc_alloc_count / g_rc_free_count (runtime/rc.c:90-91) were plain
static uint64_t; the __ail_tls_ctx == NULL fallback at rc.c:161
(alloc) and rc.c:212 (dec-to-zero) did a non-atomic ++. A
multi-threaded host using the global fallback path (no
ailang_ctx_new) raced the read-modify-write and lost increments, so
the AILANG_RC_STATS atexit Σ under-counted non-deterministically.
Not a memory bug — Ctx:!Send keeps every box on one thread; only
the global statistics Σ was wrong (the M5 iter-2 symbol-fan
leak-proof finding, b724cd1; user-approved Option A bounce-back
resolution).

Fix (runtime/rc.c only, 37+/14-): the two globals are now
_Atomic uint64_t; atomic_fetch_add_explicit(.., memory_order_relaxed)
at the two null-ctx fallback ++ sites; atomic_load_explicit(..,
relaxed) snapshot in ailang_rc_stats_atexit (its sole reader).
Relaxed is correct and sufficient — pure statistics, no
happens-before obligation, atexit reader runs after all worker
threads joined. Per-ctx counters and the per-object refcount header
stay non-atomic BY DESIGN (single-thread-per-ctx; boxes never cross
threads) — out of scope, untouched. Frozen value layout / ABI
offsets / host-free rule untouched. The genuinely-still-
single-threaded drop-worklist doc note was correctly left unchanged
(false correction refused); the two stale atomicity doc blocks
corrected for honesty.

Boss-verified independently: RED -> GREEN deterministically 3/3
(jitter gone, allocs==frees==16_000_000); full cargo test -p ail
green (every binary 0 failed — per-ctx tsan harnesses + embed
e2e unaffected; rc.c links into every ail binary, strong regression
gate); scope = runtime/rc.c + journal + stats only; RED files
unchanged. Unblocks resuming the M5 leak-proof.

Includes the per-iter journal, stats, and the INDEX.md line.
2026-05-19 01:57:06 +02:00
Brummel 427b687b95 test(rc): RED — non-atomic global g_rc_* stats counters race under a multi-threaded host
bugfix-rc-global-stats-race, RED stage (audit trail; GREEN follows
separately via implement mini-mode).

`g_rc_alloc_count`/`g_rc_free_count` (runtime/rc.c:90-91) are plain
`static uint64_t`; the `__ail_tls_ctx == NULL` fallback at rc.c:161
and rc.c:212 does a non-atomic `++`. Concurrent host-side
ailang_rc_alloc/ailang_rc_dec outside a bound ctx race the
read-modify-write and silently drop updates. New C host (8 threads
x 2_000_000 alloc-then-dec, no ailang_ctx_new so the global path is
taken, no box crosses a thread) + integration test asserting the
atexit Σ is exact. Boss-verified RED: allocs=2141382 expected
16000000, live=-131242 (deterministic-fail under this contention,
not flaky).

NOT a memory bug — Ctx:!Send keeps every box on one thread, the
per-object refcount header op is correct, programs are bit-exact;
the only defect is the under-counted statistics Σ (the M5 iter 2
symbol-fan leak-proof finding, b724cd1). Existing green per-ctx
harnesses (embed_swarm_tsan.rs, embed_rc_accounting_tsan.rs) always
bind __ail_tls_ctx, exercising the zero-contention per-thread
counters and never this global fallback — which is why the suite is
green while the bug ships.
2026-05-19 01:50:36 +02:00
Brummel b724cd17a1 iter embedding-abi-m5.2 (PARTIAL 2/3): data-server adapter + symbol-fan swarm; leak-proof surfaced a non-atomic global g_rc_* stats race
M5 iteration 2 (spec ae905de, plan 9cc9d9c). Tasks 1+2 clean and
committable; Task 3 BLOCKED on a real finding → M5 bounce-back.

Shipped:
- data-server promoted [dev-dependencies] -> [dependencies]
  (Invariant-1 sanctioned: ail-embed is the sole meeting point; the
  AILang workspace graph still has data-server count 0; zero
  compiler-surface diff — not even root Cargo.toml this iter).
- additive `adapter` module (ail-embed/src/adapter.rs): tick_to_px,
  MidPriceStream (lazy Iterator<f64> over SymbolChunkIter),
  fold_symbol; RED-first (E0432 -> GREEN). m5.1 core untouched
  except `pub mod adapter;`.
- swarm_runner [[bin]]: one thread per symbol, each owning its
  Kernel. The clean compile IS the compile-time per-thread-ctx
  proof (Ctx: !Send => a shared-ctx swarm is E0277).
- tests/swarm.rs: real-data symbol-fan E2E. `symbol_fan_swarm_bit_exact`
  is GREEN and live — per-symbol kernel (acc,n) bit-exact vs an
  independent same-order host reference (EURUSD/GER40/XAUUSD, ~4s).

The Task 3 finding (Boss independently confirmed by reading
runtime/rc.c): the global Σallocs==Σfrees leak measurement is an
INSTRUMENTATION race, not a memory bug. No box crosses a thread
(Ctx: !Send), the real refcount/free is correct (bit-exact GREEN
every run); only the global g_rc_* stat counters (rc.c:90-91,
161,212) are non-atomic BY rc.c's own documented single-threaded
design, losing ++s when 3 worker threads hit the null-__ail_tls_ctx
host-side path. M5 is AILang's first concurrent consumer; rc.c's
header (rc.c:44-49) explicitly defers exactly this atomic-vs-non-
atomic decision to "when it acquires concurrency primitives".

The leak assertion was Boss-split into `symbol_fan_swarm_leak_free`
(#[ignore], body preserved VERBATIM — quarantined not weakened;
un-ignore = the runtime fix's acceptance criterion) so main stays
green and the finding is pinned as a regression marker. The earlier
plan/journal claim that embed_swarm_tsan.rs covers this path was
wrong and is corrected on the record (that test uses the scalar
kernel — zero box allocs — so it never exercised the host-side
global-counter path).

Escalated as an M5 bounce-back: resolution touches M3-frozen
runtime/ and re-frames M5's "zero runtime change" commitment;
multiple substantive options, not unilaterally Boss's to pick in
frozen-runtime territory. Boss recommendation on the record =
Option A (make only the global-fallback g_rc_* counters atomic, as
a standalone RED-first runtime micro-iteration; M5 framing amended).
Time-shard + friction-harvest remain m5.3.

Includes the per-iter journal (with Boss disposition), stats, and
the INDEX.md line.
2026-05-19 01:37:50 +02:00
Brummel 9cc9d9c517 plan: embedding-abi-m5.2 — data-server adapter + symbol-fan swarm + leak-proof
M5 iteration 2 (spec ae905de; builds on m5.1 204c171). 3 tasks:
(1) promote data-server dev-dep -> real dep + additive `adapter`
module (tick_to_px / MidPriceStream lazy Iterator / fold_symbol),
RED-first on tick_to_px; (2) `swarm_runner` [[bin]] — one thread per
symbol, each owning its Kernel (Ctx: !Send makes one-ctx-per-thread
a compile-time guarantee), prints RESULT <sym> <acc_bits> <n>;
(3) integration test spawning that bin as a subprocess under
AILANG_RC_STATS=1 (atexit stat line is unobservable in-process —
runtime/rc.c:132-138), asserting per-symbol bit-exact vs an
independent same-order host reference + Σallocs==Σfrees (ported from
embed_tick_e2e.rs:84-104), skip-if-absent mirroring data-server's
own precedent.

Boss decisions recorded in-plan for the audit: TSan needs NO new
nightly harness — per-thread-ctx race-freedom is compile-time
enforced by Ctx:!Send AND already green-tested by the C-host
embed_swarm_tsan.rs (same pattern); m5.2 adds no new AILang-side
concurrency primitive. data-server dev->real dep is Invariant-1
sanctioned (compiler crates untouched; gate shifts to "AILang
workspace graph data-server count == 0" + zero compiler-surface
diff). Time-shard + friction-harvest deferred to m5.3.
2026-05-19 01:22:19 +02:00
Brummel 204c171e60 iter embedding-abi-m5.1 (DONE 3/3): lean ail-embed core + build.rs + hermetic smoke
M5 iteration 1 (spec ae905de, plan 22f02aa). Stands up the
workspace-excluded `ail-embed` crate:

- zero-dependency embedding core (`ail-embed/src/lib.rs`): extern "C"
  to the M3-frozen ABI + frozen-layout State/Tick box helpers + a
  Kernel price fold; the Rust port of the audited
  crates/ail/tests/embed/tick_roundtrip.c. Raw pointers never escape
  the type.
- build.rs (no in-repo precedent): AIL_BIN env override else nested
  `cargo build -p ail` against the parent workspace (separate target
  dir → no cargo-lock deadlock), `ail build --emit=staticlib`, link
  directives.
- hermetic data-server smoke (ail-embed/tests/smoke.rs): synthetic
  Pepperstone-format ZIP fixture via data-server's own public
  RawTickRecord type → real DataServer → Kernel, bit-exact vs a
  same-order host reference fold; runs with no /mnt.
- `ail-embed` is its own cargo workspace root (empty [workspace]
  table); data-server is a dev-dependency only. Root Cargo.toml gains
  only a 4-line non-membership comment.

Invariant 1 Boss-verified independently: full+no-deps cargo metadata
on the AILang workspace shows data-server count 0; git status path
filter empty (zero diff to crates/ailang-*, crates/ail/, runtime/,
examples/*.ail); src/lib.rs zero code-level data_server; AILang
cargo build --workspace still clean. ail-embed suite 2/2 green
(kernel_run_sums_prices unit RED-first + hermetic_smoke integration),
verified by me, not just the agent report.

Two toolchain-forced corrections to the plan's verbatim
ail-embed/Cargo.toml (added empty [workspace] table; sibling dev-dep
path ../libs -> ../../libs, manifest-relative) — confined to the
plan-created manifest, no acceptance gate altered, 0 review re-loops.
Journal Concerns records the planner-recon implication for the next
workspace-excluded-nested-crate plan. Adapter API + thread-swarm
deferred to M5 iter 2+ per spec/plan.

Includes the per-iter journal, stats, and the INDEX.md line.
2026-05-19 01:12:19 +02:00
Brummel 22f02aa26b plan: embedding-abi-m5.1 — lean ail-embed core + build.rs + hermetic smoke
First iteration of M5 (spec ae905de). 3 tasks, RED-first on the
core: (1) scaffold the workspace-EXCLUDED ail-embed crate +
build.rs (de-risks the no-in-repo-precedent ail-binary resolution
in isolation with a stub lib), (2) the lean embedding core
(Ctx/Kernel/frozen-layout helpers, Rust port of the audited
tick_roundtrip.c:31-75, zero data_server use) with an in-source
data-server-free RED unit test, (3) hermetic data-server smoke
(synthetic Pepperstone-format ZIP fixture via the public
RawTickRecord type → real DataServer → Kernel, bit-exact vs host
reference).

Boss decomposition calls recorded in-plan: data-server is dev-only
(iter-1 library has ZERO deps — Invariant 1 maximally honest);
build.rs resolves ail via AIL_BIN-else-cargo-build-parent-workspace
(separate target dir, no nested-cargo lock); fixture writer mirrors
data-server/src/loader.rs:110-124. Invariant-1/no-language-change
mechanically gated (cargo metadata graph check + git status path
check). Adapter + swarm explicitly deferred to iter 2+.
2026-05-19 01:01:09 +02:00
Brummel ae905de00c spec: embedding-abi-m5 — ail-embed adapter + data-server wiring + thread-swarm backtest
User-approved 2026-05-19; grounding-check PASS (all load-bearing
assumptions are already-shipped M3 ABI behaviour, each pinned by a
green non-ignored integration test: embed_tick_e2e own+borrow,
embed/tick_roundtrip.c, embed_staticlib_cli, embed_swarm_tsan,
embed_staticlib_alloc_guard). Terminal embedding milestone, zero
language/compiler/runtime change — ail-embed is a lean reusable
embedding module (Rust port of the audited tick_roundtrip.c) + the
real E2E; in-repo, [workspace]-excluded so the compiler workspace
owes nothing to ../libs/data-server or /mnt (Invariant 1 in the
dependency graph, not just on paper). Real data-server + real
Pepperstone ticks; symbol-fan headline + time-shard
boundary-invisibility proof (per-shard bit-exact; whole-window only
within f64-reassociation tolerance — self-review fix).

roadmap: M5 [ ]->[~] open in-flight, spec pointer added, dangling
"todo above" dependency reconciled to shipped 170464f; M4-retired
and Tick-coverage struck/done entries pruned (their own text
scheduled deletion at next-milestone-start; permanent rationale
survives in 2026-05-18-brainstorm-embedding-abi-m4-retired.md, still
pointed to by M5's context line). Mirrors the M3 1fbb9c4 precedent
(spec + roadmap-flip in one commit).
2026-05-19 00:52:38 +02:00
Brummel 208d7095bc fix(check): GREEN — over-strict-mode recognises ctor-rebuild-from-primitive-fields as a consume
Drives the RED test from a11cb7c to GREEN. Bug-fix iter
bugfix-over-strict-mode-ctor-rebuild-consume (debug -> implement
mini). Check-only lint-precision fix; zero codegen / runtime / ABI /
schema / DESIGN.md change.

The [over-strict-mode] lint's consume-detection
(any_sub_binder_consumed_for / pattern_has_consumed_heap_binder,
crates/ailang-check/src/linearity.rs) only recognised a consume of an
(own (con T)) param when a heap-typed pattern-binder was moved out of
`match p`. When p was destructured into purely primitive fields fed
into a Term::Ctor rebuilding p's own ctor, that genuine
dismantle+rebuild consume was invisible, so the lint spuriously
advised "(borrow ...) would suffice". An LLM author "fixing" that by
flipping an export own->borrow would silently invert the ABI
ownership contract — why a low-severity advisory FP got a real
RED-first fix.

Fix: a second recognition path in any_sub_binder_consumed_for — a
`match p` arm destructuring binders out of p's ctor that references
any of them (primitive or not) inside a Term::Ctor's args in the arm
body genuinely consumes p. Two pure helpers: ctor_uses_any_binder
(finds a fresh-allocation Term::Ctor reachable in the arm body) +
term_mentions_any_binder (deep free-var scan, so binder flow through
an intervening expr like (+ acc px) is recognised, not only a bare
Var arg). Conservative toward NOT suppressing: a ctor ignoring p's
payload still warns (negative-control verified). Over-strict-only —
by-name-shadowing imprecision is extra-silence, never under-strict
(Known debt in the journal + the fn doc). The two stale doc comments
that mis-attributed this FP to a nested `match` corrected for
doc-honesty (debugger concern #2 — same code region, in-scope).

Boss-verified independently: RED test now GREEN; full
cargo test -p ailang-check 108/0 lib + every binary 0-failed with NO
existing test modified; ail check examples/embed_backtest_step_tick.ail
no longer warns over-strict on st/tick (exit 0);
embed_backtest_step_tick_borrow.ail + M3 embed_backtest_step_record.ail
still clean; embed_tick_e2e + bench posture untouched.

+166/-20 in crates/ailang-check/src/linearity.rs only. Journal +
stats + INDEX line in this commit.
2026-05-18 23:50:13 +02:00
Brummel a11cb7cc9f test(check): RED — over-strict-mode false-positive on ctor-rebuild consume
RED-first audit trail (debug skill; fix follows as a separate GREEN
commit). Test-only, +97 lines in the linearity in-source test module,
no production change.

Bug: the [over-strict-mode] lint emits a conservative false-positive.
The consume-detection (any_sub_binder_consumed_for /
pattern_has_consumed_heap_binder, crates/ailang-check/src/linearity.rs
:853,942) only recognises a consume of an (own (con T)) param when a
heap-typed pattern-binder is moved out of `match p`. When p is
destructured into purely primitive fields (e.g. Float/Int) that are
fed into a Term::Ctor rebuilding p's own ctor, that rebuild's
re-consumption of the dismantled allocation is invisible, so the lint
wrongly tells the author `(borrow ...)` would suffice. It is
over-strict, never under-strict (exit 0; codegen/ABI unaffected) — but
an LLM author "fixing" the spurious warning by flipping an export's
declared mode own->borrow would silently invert the ABI ownership
contract, which is why a low-severity advisory FP gets a real
RED-first fix.

The carrier's initial hypothesis (nested inner `match` is the
discriminator) was disproved during diagnosis: the M3
embed_backtest_step_record.ail is silent only because its
implicit-mode scalar Float param disables the lint via the activation
gate (linearity.rs:327), not because it handles the rebuild; the
defect reproduces with a single param and no nesting. The RED unit is
therefore the general synthetic shape — an (own (con T)) param
consumed by a term-ctor rebuild of its own ctor — not the embed
fixtures.

RED test (fails now, goes GREEN when consume-detection sees the
ctor-rebuild re-consumption):
crates/ailang-check/src/linearity.rs ::
  linearity::tests::over_strict_mode_silent_when_ctor_rebuilt_from_primitive_fields
2026-05-18 23:39:31 +02:00
Brummel 32e59766f9 roadmap: Tick-coverage on M3 [todo] -> [x] (170464f) — keep one cycle for context 2026-05-18 23:29:31 +02:00
Brummel 170464fca8 test(embed): pin two-record-param per-tick (State, Tick) -> State on the shipped M3 ABI
Coverage backfill for an already-shipped capability — NO language /
checker / codegen / schema / runtime change (4 new files, all under
examples/ and crates/ail/tests/).

Property protected: the M3 export gate is_c_abi_type runs as a
per-parameter loop, accepting a single-ctor all-scalar record
independently per param; the staticlib forwarder maps every
non-scalar Type::Con to a bare ptr (M3-frozen). So a kernel
(State, Tick) -> State with BOTH params single-ctor all-scalar
records is callable from C today — a record Tick pushed per call,
not just a scalar sample. Every shipped M3 fixture pushes a scalar
Float; none pinned the two-record-param shape (the actual minimal
data-server binding). This is the residual of the retired M4.

Added:
- examples/embed_backtest_step_tick.ail            (own Tick)
- examples/embed_backtest_step_tick_borrow.ail     (borrow Tick)
- crates/ail/tests/embed/tick_roundtrip.c          (C host, -DBORROW)
- crates/ail/tests/embed_tick_e2e.rs               (2 E2E tests)

Both E2E tests pass on HEAD (Boss-verified, not just agent-claimed):
own + borrow, globally leak-free (Sigma allocs == Sigma frees across
the ctx readback + g_rc atexit lines, the M3 dual-stat-line model),
acc/n value-correct, exit 0. Independently re-confirms the M4
retirement reasoning: the capability is genuinely present in M3.

Honest, deliberately NOT silenced: the new fixtures emit an
advisory [over-strict-mode] warning (ail check still exits 0,
non-gating) that the M3 scalar-sample original does not. Root cause
characterized: a conservative over-strict-mode false-positive whose
discriminator is the nested second (match tick ...) — structurally
identical `st` handling warns when its term-ctor rebuild is nested
inside an inner match. It is advisory only and does not affect
codegen, the ABI, or the proven property. The fixture is kept in
its LLM-natural form (dont-adapt-tests-to-bugs); the lint precision
wart is surfaced as a separate finding, not papered over.
2026-05-18 23:28:52 +02:00
Brummel 38150ae923 roadmap: retire Embedding ABI — M4 (never speced) — speculative infra by feature-acceptance clause 2
M4 (sequence crossing via the List ADT) was struck during its own
/boss brainstorm, in the Step-2/3 Q&A, before any spec — the
"problem mis-framed -> don't ratify a known-unneeded shape into the
roadmap" brainstorm path. No spec, no grounding-check, no planner
handoff.

Why: held against DESIGN.md feature-acceptance criterion clause 2,
M4 removes no redundancy. The shipped M3 export gate is_c_abi_type
(crates/ailang-check/src/lib.rs:1934-1953) is a per-parameter loop
accepting a C scalar OR a single-ctor all-scalar record
independently per param; the forwarder's llvm_scalar maps every
non-scalar Type::Con to ptr (M3-frozen). So (State, Tick) -> State
with both as single-ctor all-scalar records is already
gate-accepted and forwarder-supported today. The minimal
data-server binding is M3 (shipped) + a host-side per-tick loop;
cons-list crossing would add a 2N+1-box-per-chunk host builder and
the deferred flat-array perf debt, with no named consumer (M5's
adapter unrolls each chunk host-side; whole-chunk in-kernel
visibility is semantically void — State threads across calls
regardless of chunk boundaries).

Outcome: M4 retired in docs/roadmap.md (struck entry kept one
cycle for context, never [x] — never shipped); M5 reconciled
(depends on M4 -> M3 + the Tick-coverage todo; adapter unrolls
host-side; friction feeds the host-per-tick-FFI vs. batch-crossing
P2 perf decision); residual recorded as a new [todo] Tick-coverage
on M3 — an E2E + fixture pinning the two-record-param per-tick
(State, Tick) -> State shape (test backfill, no brainstorm).

Rationale + the honest mid-Q&A correction (I had wrongly asserted
M5 needs M4) in
docs/journals/2026-05-18-brainstorm-embedding-abi-m4-retired.md;
INDEX.md mirrors it.
2026-05-18 23:22:39 +02:00
Brummel c75fb805a6 roadmap: close Embedding ABI — M3 ([~]->[x]) — milestone substantively closed + sound (record crosses C ABI in/out, ownership follows declared mode, value layout frozen+byte-pin-enforced, all invariants held, Invariant 1 clean, bench causally exonerated NO-ratify); no WhatsNew (next item M4 is a new milestone — a /boss bounce-back, not done-state) 2026-05-18 21:51:01 +02:00
Brummel 63d7d60dd1 iter embedding-abi-m3.tidy (DONE 3/3): close M3 audit [medium]+[low] doc-honesty drift, pin-safe
[medium] DESIGN.md §"Embedding ABI": surgically replaced ONLY the
contradicted M1-era parenthetical (asserted the export boundary is
scalar-only — false post-M3, a single-ctor record is an accepted
export type) with the present-tense truth pointing at the frozen-layout
SSOT + the own/borrow mode contract. The parenthetical shared physical
line :2300 with the docs_honesty_pin:135 pinned bare-scalar sentence;
the edit kept every pinned word (planner Step-5 item-6 pin-safety, the
M2.tidy precedent — docs_honesty_pin stayed 5/5 green, RED-restore path
never triggered). [low] codegen forwarder-body comment honesty fix
(comment-only; the IR byte-pins assert generated code not comments —
byte-identical before/after).

Boss-verified: both stale fragments grep-absent; 4 standing pins green
at the recon baseline; workspace 639/77 byte-unchanged; diff exactly
2 files. No behaviour change; the pins are the coverage (M2.tidy
precedent, no audit/fieldtest gate). bench already carry-on/NO-ratify
at the M3 audit (causally exonerated). One non-gating planner-quality
grep defect recorded in the journal Concerns (orchestrator handled it
correctly, no code bent).
2026-05-18 21:50:15 +02:00
Brummel 4fc3470552 gitignore: .claude/scheduled_tasks.lock (Claude Code harness runtime artefact; the tracked .claude/ skill+agent symlinks are unaffected) 2026-05-18 21:43:34 +02:00
Brummel 44ced513dc plan: embedding-abi-m3.tidy — reconcile M3-audit [medium]+[low] doc-honesty drift (3 tasks, pin-safe parenthetical-only edit + comment-only fix, M2.tidy precedent) 2026-05-18 21:42:52 +02:00