10 Commits

Author	SHA1	Message	Date
Brummel	832375f2ac	convention: counter-prefix file naming across docs/specs/, docs/plans/, design/contracts/, design/models/ ... All 176 files in the four accumulating directories now use a zero-padded 4-digit counter prefix that reflects creation order (`NNNN-slug.md`). The counter is assigned per directory in strict git-log creation order; ties broken alphabetically by original name. The old `YYYY-MM-DD-` prefix on docs/specs/ and docs/plans/ files is dropped — the date is recoverable from git log and the counter carries the ordering. A file's counter is stable for the life of the file: never reassigned, never reused, never compacted. Deleted files retire their counter; subsequent files do not fill the gap. This is the property that lets cross-references stay literal — refs use the full filename including the counter (`design/contracts/0007-honesty-rule.md`) so they grep cleanly and resolve directly without a glob step. 313 cross-references updated across .md/.rs/.toml/.c/.json files (test pins, include_str! paths, design-INDEX entries, baseline notes, runtime C comments, inter-contract markdown links incl. bare basename and `../models/foo.md` forms). CLAUDE.md gets a new "File-naming convention" section spelling out the rule and rationale. skills/brainstorm/SKILL.md and skills/planner/SKILL.md updated so new spec/plan creation produces counter-prefixed names from the start. The full test suite (cargo test --workspace) passes.	2026-05-28 13:31:31 +02:00
Brummel	99df14c792	workflow: scrub residual JOURNAL/journal refs missed in the first sweep ... A re-grep found 15 live references (excluding docs/specs/ and docs/plans/, which stay historical) the previous two commits missed — all in source comments, doctests, bench helpers, and one agent file. Per-file: - crates/ail/src/main.rs: typeclass-coherence diagnostic comment pointed at "the JOURNAL queue's wording" — points at design/contracts/typeclasses.md alone. - crates/ailang-prose/src/lib.rs: `//!` header referred to docs/JOURNAL.md "Pinned: human-readable prose surface" — retargeted to design/contracts/authoring-surface.md. - crates/ailang-check/src/lib.rs: bugfix tag + "see iter method-dispatch-refactor journal" prose collapsed to a clean rationale paragraph; the canonical shape is stated inline. - crates/ailang-surface/tests/prelude_decouple_carve_out_pin.rs: "(d) record the rationale in a per-iter journal" → "(d) record the rationale in the commit body". - crates/ailang-surface/tests/prelude_module_hash_pin.rs: header collapsed (pd.2/pd.3 milestone narrative dropped — the test's purpose is self-evident from its body); both "per-iter journal" drift-instructions point at the commit body. - runtime/rc.c: "bench numbers in JOURNAL 18f.2" → "original profiling bench numbers". - bench/run.sh: two "JOURNAL entry" comments → "commit body". - bench/architect_sweeps.sh: header rewritten (cross-ref to the design-md-consolidation milestone commits, not a JOURNAL entry). Sweep-4 extended with two new anti-regrowth phrases ("see the per-iter journal", "in a per-iter journal") so journal prose can't grow back into design/contracts/ silently. - skills/audit/agents/ailang-architect.md: "unlike a journal it lives on main" → "since it lives on main". - ail-embed/src/bin/timeshard_runner.rs: "records it in the close-out journal" → "prints it to stderr". - ail-embed/tests/timeshard.rs: two "journal-only friction timing" → "stderr-only friction timing". Verification (after the edit): - `grep -rin '\bjournal\b'` against live tree returns exactly two hits: the Sweep-4 regex itself (intentional — TABU phrases that prevent regrowth) and the PHRASES array in design_index_pin.rs (intentional — the same regrowth guard at test level). Both are load-bearing negative assertions. - `bash bench/architect_sweeps.sh` exits 0 ("All five sweeps clean") — no design/-side regrowth. - `cargo build --workspace` green; `cargo test --workspace` green.	2026-05-20 11:32:37 +02:00
Brummel	8e586f493f	workflow: replace per-iter journal system with git log + BLOCKED.md ... The per-iter journal under docs/journals/ duplicated the iter commit body's substance and accumulated as Verlauf-Doku with no Future-Use. Sweep across all live control documents: CLAUDE.md, the 7 SKILL.md files, the 11 agent files, design/INDEX.md and the contracts/models that referenced journals, docs/roadmap.md, and the handful of source comments + tests that pointed at journal files for rationale. Mechanism changes: - Standing-reading-lists in every agent now read `git log -N --format=full` for recent project state, never per-iter journal files. The architect reads `git log <prev-milestone-close>..HEAD --format=full` for audit scope. - implement-orchestrator no longer writes a journal file. DONE outcomes emit just code + stats; the end-report is the per-task summary the Boss uses to write the commit body. PARTIAL/BLOCKED outcomes emit BLOCKED.md at the repo root — uncommitted by convention, Boss removes on repair or discard. New iron-law line + four-rationalisation row + red-flag bullet codify it. - audit ratify mechanic: --update-baseline is now paired with an explicit ratify paragraph in the audit-close commit body, not a separate JOURNAL ratify entry. - design/contracts/honesty-rule.md: "history and rationale lives in docs/journals/" → "lives in git log (iter and audit commit bodies)". Pinned phrase preserved verbatim. - CLAUDE.md "Roles of …" section reframed: design/, git log, journal-archive.md (content-frozen), roadmap.md, specs/, plans/. No docs/journals/ slot anymore. - roadmap.md context-lines that pointed at per-iter journals are dropped where the spec/commit already carries the rationale, or rephrased to "shipped in the <iter> iter commit" / "docs/journal- archive.md (<date> entry)" for pre-2026-05-11 references. What stays (this commit): - docs/journals/ directory and contents are NOT touched. Removing the contents is a separate follow-up. - docs/journals/2026-05-19-design-decision-records.md still has live readers (docs_honesty_pin.rs Z 108 + parse.rs + duplicate_ctor_pin.rs + 3 roadmap mentions) — also follow-up. - docs/journal-archive.md still exists; its self-pointer header has been updated to drop the "see docs/journals/INDEX.md" mention. Workspace builds, full test suite green.	2026-05-20 11:21:37 +02:00
Brummel	176821c2e7	iter design-md-rolesplit.1 (DONE 9/9): DESIGN.md -> design/ ledger role-split ... The 3020-line docs/DESIGN.md is replaced by the design/ ledger: design/INDEX.md (sole addressable spine, typed Contracts+Models tables, polymorphic links — prose file OR authoritative source //!), 14 design/contracts/*.md test-linked invariants + 3 source-link-only contracts (mangling/env-construction/qualified-xref, no prose file — code is SoT), 5 design/models/*.md whitepapers, and docs/journals/2026-05-19-design-decision-records.md (the relitigation-guard archive — every why/rejected/does-not-do/rollback/ empirical ### moved out at ###-granularity). Clean cut: git rm docs/DESIGN.md, no stub. RED-first crates/ailang-core/tests/design_index_pin.rs — the 4-clause anti-regrowth spine (DESIGN.md-gone / every-INDEX-link-resolves / every-contract-names-a-resolvable-ratifier / contracts-carry-no-decision-record-prose) — demonstrably RED before, GREEN after. Build-atomic by task ordering: design_schema_drift.rs's include_str! (the only compile-time consumer) retargeted to design/contracts/data-model.md BEFORE the deletion; its ## Data model/## Pipeline slicer dropped (a simplification the split enables). 2 NoInstance diagnostics + 2 lockstep E2Es retargeted to design/contracts/{float-semantics,typeclasses}.md. ~12 agent reading lists + 5 SKILL bodies + CLAUDE.md + skills/README.md + ~25 code/C/.ail/spec comment xrefs retargeted; OQ7 dangling 'Iter 13b' cite deleted (no forward target — a pointer would be fiction). honesty-rule.md rewritten so the rule names the new home (rationale->journals), resolving the recon-found internal contradiction; the two docs_honesty_pin.rs:70,72 pinned phrases kept verbatim+contiguous. Boss-verified independently: cargo test --workspace 646 passed / 0 failed; design_index_pin 4/4; acceptance grep CLEAN of live DESIGN.md refs (residuals = only the spec-mandated clause-4 deletion-enforcer). 2 DONE_WITH_CONCERNS routed to the mandatory milestone-close audit: (a) str-abi.md:23 '(iter str-concat, 2026-05-13)' provenance stamp trips advisory architect_sweeps Sweep-1 — Boss-confirmed byte-identical to DESIGN.md@deeffb1:2062-2065, a faithfully-migrated PRE-EXISTING anchor (regexes verbatim, only path retargeted), NOT split-introduced — RATIFY-or-tidy at audit; (b) a now stale-direction intra-prose 'see Str ABI below' cross-ref in float-semantics.md — audit-adjudication candidate. Plan defect noted: Task 9 Step 4's verbatim acceptance grep used a ^./ anchor not matching the system's grep -rIn output; substance re-verified CLEAN. Spec grounding-check PASS x2. Journals INDEX + decision-records pointer appended (Boss-only).	2026-05-19 13:04:22 +02:00
Brummel	0900f3f413	iter embedding-abi-m5.3 (DONE 3/3): time-shard boundary-invisibility proof + friction harvest ... Final functional M5 iteration (spec ae905de, plan 67027ab; on green m5.1 204c171 + m5.2 b724cd1 whose leak-proof resolved via dbd76e5). Proves the chunk/window-boundary-invisibility claim the M4 retirement rests on: - adapter::fold_window — purely additive windowed sibling of fold_symbol (same MidPriceStream+Kernel path; stream_tick_windowed). fold_symbol/MidPriceStream/Kernel byte-untouched. - chrono as a DEV-dep only — the time-shard test derives (y,m)->Unix-ms bounds (data-server's own precedent) and passes raw ms to the bin; the bin is date-math-free; Invariant 1 unaffected. - new single-responsibility [[bin]] timeshard_runner — one thread per disjoint window, each owning its Kernel (Ctx:!Send => compile-time per-thread-ctx). - tests/timeshard.rs — spec §3 strength-ordered: (a) per-shard (acc,n) BIT-EXACT vs single-thread host fold of that exact window in stream order (THE boundary-invisibility proof); (b) Σ partials bit-exact by construction; (c) whole-window within REL_TOL=1e-6 only (cross-shard f64 reassociation is host arithmetic, never a kernel property — bit-exact there would be the bug; tolerance derived from the recursive-summation error bound). Plus the now-deterministic leak-Σ + a journal-only friction timing capture (no bench gate, no timing assertion). Concrete pin: EURUSD 2017-03/04/05 (recon-verified present, contiguous, no gaps; data-server has no month-listing API). RED was a genuine deterministic compile failure (declared-but-absent bin source; Cargo missing-source error vs the plan's predicted env! error — same cause, no false-green; benign plan-text imprecision in Concerns). Boss-verified independently: timeshard determinism 5/5 (no jitter); full ail-embed suite 0 failed AND 0 ignored across all binaries (m5.2 symbol_fan_swarm_leak_free stays un-ignored & green — the saga's resolution intact); isolated embed_rc_global_stats_race still green; AILang workspace data-server count 0; compiler-surface diff empty (zero diff to crates/ailang-*, crates/ail/, runtime/, examples/*.ail, root Cargo.toml, DESIGN.md). Friction-harvest deliverable (P2 flat-array-decision input): host-per-tick-FFI ~= 658 ms / 3,192,562 ticks ~= ~206 ns/tick at real EURUSD volume. M5 functional work complete; milestone closes next via the mandatory audit (which also handles the pre-existing DESIGN.md:2358-2360 "additive M4 concern" drift, out of m5.3 scope). Includes the per-iter journal, stats, and the INDEX.md line.	2026-05-19 02:40:16 +02:00
Brummel	dbd76e5503	fix(ail-embed): GREEN — build.rs tracks runtime/ as rerun-if-changed; M5 swarm leak-proof armed & deterministic ... bugfix-swarm-rc-alloc-undercount, GREEN stage. RED is the separate audit-trail commit 483117d. Root cause of the m5.2-resume-attempt residual: ail-embed/build.rs declared cargo:rerun-if-changed only for the kernel .ail + build.rs itself — NOT the runtime/ C sources ail build --emit=staticlib compiles into libailang_rt.a. Cargo never re-ran build.rs after the atomic-counter fix 7bfa11e, so ail-embed kept linking a stale pre-7bfa11e non-atomic libailang_rt.a; the swarm's concurrent TLS-NULL host allocs raced the stale non-atomic g_rc_alloc_count++ (frees-stable/allocs-jitter). NOT a second runtime bug and NOT test-methodology unsoundness — runtime/rc.c at HEAD is correct; the isolated crates/ail RED was green only because it rebuilds the staticlib fresh per run. Fix (ail-embed/build.rs, +7 lines): directory-level cargo:rerun-if-changed=<repo>/runtime (Cargo recurses mtimes under the dir). Preferred over a per-file list — the implementer inspected crates/ail build_staticlib (libailang_rt.a = ar(rc.o, str.o), both under runtime/) and chose the directory mechanism so any future runtime source cannot silently re-introduce the same staleness. Existing kernel/build.rs/AIL_BIN rerun-if-changed lines kept. Cohesive consequence (in-scope): symbol_fan_swarm_leak_free un-#[ignore]d — the test BODY is byte-for-byte unchanged (it was quarantined, never weakened); the module-doc + fn-doc breadcrumbs rewritten to the resolved build-dep-staleness rationale. It is now the integration-level acceptance of both 7bfa11e and this fix. Boss also removed a pre-existing unused `DataFormat` import in swarm.rs inline (trivial; the file is committed this iter regardless; warning-clean after). Boss-verified independently: RED rt_archive_freshness -> GREEN; swarm determinism 5/5 (symbol_fan_swarm_leak_free + symbol_fan_swarm_bit_exact GREEN every run, 0 ignored, jitter gone, Sallocs==Sfrees==12000003); isolated embed_rc_global_stats_race still GREEN (untouched, runtime untouched); Invariant 1 data-server count 0; scope = ail-embed/build.rs + ail-embed/tests/swarm.rs + journal + stats only. The M5 swarm leak-proof bounce-back is fully resolved end-to-end. M5 stays open [~]; m5.3 (time-shard + friction-harvest + close-out) remains. Includes the per-iter journal, stats, and INDEX.md line.	2026-05-19 02:21:47 +02:00
Brummel	483117d39b	test(ail-embed): RED — build.rs links a stale libailang_rt.a (missing rerun-if-changed on runtime sources) ... bugfix-swarm-rc-alloc-undercount, RED stage (audit trail; GREEN follows separately via implement mini-mode). Root cause of the m5.2-resume-attempt residual (the M5 swarm leak-proof still non-deterministic after the atomic-counter fix 7bfa11e): ail-embed/build.rs declares cargo:rerun-if-changed only for the kernel .ail + build.rs itself — NOT runtime/rc.c. So Cargo never re-ran build.rs after 7bfa11e; the swarm keeps linking a stale pre-7bfa11e libailang_rt.a whose g_rc_*count++ is the old non-atomic racing increment. 7bfa11e was correct and is unchanged; it just never reached the ail-embed binary. This explains why the isolated crates/ail RED (rebuilds the staticlib fresh per run) is green while the ail-embed swarm is not, and the frees-stable/allocs-jitter asymmetry (few TLS-NULL frees = no contention; millions of TLS-NULL host allocs = race on the stale non-atomic counter). Framing: a genuine defect (build-dependency completeness gap), NOT test-methodology unsoundness — with a fresh archive the swarm Σ-over-stat-lines balances deterministically (Σallocs==Σfrees== 12000003, 5/5 per the debugger). runtime/rc.c at HEAD is correct; the GREEN fix is in the build script, not the runtime. RED pins the CAUSE deterministically (the symptom is non-deterministic by construction): objdump the libailang_rt.a that build.rs emitted for this crate; assert >=2 lock-prefixed instructions (the atomic lock incq for g_rc_alloc_count + g_rc_free_count from current rc.c) and that it matches a freshly-built archive. Boss-verified RED: linked archive has 0 lock insns ("STALE pre-7bfa11e archive linked"). Does NOT touch the committed #[ignore] of symbol_fan_swarm_leak_free nor the isolated embed_rc_global_stats_race RED (both correct, unchanged).	2026-05-19 02:14:12 +02:00
Brummel	6500ca0493	ail-embed: re-quarantine symbol_fan_swarm_leak_free — atomic-global fix necessary but insufficient (doc-honesty + finding record) ... Attempted to un-#[ignore] the M5 swarm leak-proof as the integration-level acceptance of bugfix-rc-global-stats-race (7bfa11e). Boss verification falsified the sufficiency premise: across 4-6 swarm runs the leak Σ is STILL non-deterministic — Σfrees stable-exact (12000003) every run, Σallocs short by a jittering ~1600-2260 in ~1/3 of runs (e.g. left:11998383 right:12000003, 4 stat lines). The atomic-global runtime fix (7bfa11e) is real, committed, and NECESSARY — the isolated 8x2M pure-global RED embed_rc_global_stats_race is deterministically green. It is NOT sufficient for the swarm: a second, distinct alloc-side undercount remains that the isolated RED did not model (Σfrees never loses => not a per-ctx race; not pure global contention => RED green). The un-#[ignore] was reverted (never committed). The ONLY shipped diff is the three now-stale #[ignore]-rationale breadcrumbs in ail-embed/tests/swarm.rs corrected from "un-ignore when the runtime makes the counters atomic" to the accurate necessary-but-insufficient state — a future agent seeing 7bfa11e must NOT un-ignore on that basis (doc-honesty rule). The test body and the #[ignore] attribute itself are unchanged: still quarantined, NOT weakened. symbol_fan_swarm_bit_exact stays live and GREEN — the swarm IS actually correct and leak-free; only the accounting is wrong. Residual handed to a fresh debug RED-first cycle. Two framings recorded for it (genuine residual runtime defect vs Σ-over- heterogeneous-lines test-methodology unsoundness; the latter, if so, is a post-root-cause design decision). main stays green; M5 open [~]; m5.3 remains downstream of a sound leak-proof. Includes the resume-attempt journal + INDEX line.	2026-05-19 02:06:43 +02:00
Brummel	b724cd17a1	iter embedding-abi-m5.2 (PARTIAL 2/3): data-server adapter + symbol-fan swarm; leak-proof surfaced a non-atomic global g_rc_* stats race ... M5 iteration 2 (spec ae905de, plan 9cc9d9c). Tasks 1+2 clean and committable; Task 3 BLOCKED on a real finding → M5 bounce-back. Shipped: - data-server promoted [dev-dependencies] -> [dependencies] (Invariant-1 sanctioned: ail-embed is the sole meeting point; the AILang workspace graph still has data-server count 0; zero compiler-surface diff — not even root Cargo.toml this iter). - additive `adapter` module (ail-embed/src/adapter.rs): tick_to_px, MidPriceStream (lazy Iterator<f64> over SymbolChunkIter), fold_symbol; RED-first (E0432 -> GREEN). m5.1 core untouched except `pub mod adapter;`. - swarm_runner [[bin]]: one thread per symbol, each owning its Kernel. The clean compile IS the compile-time per-thread-ctx proof (Ctx: !Send => a shared-ctx swarm is E0277). - tests/swarm.rs: real-data symbol-fan E2E. `symbol_fan_swarm_bit_exact` is GREEN and live — per-symbol kernel (acc,n) bit-exact vs an independent same-order host reference (EURUSD/GER40/XAUUSD, ~4s). The Task 3 finding (Boss independently confirmed by reading runtime/rc.c): the global Σallocs==Σfrees leak measurement is an INSTRUMENTATION race, not a memory bug. No box crosses a thread (Ctx: !Send), the real refcount/free is correct (bit-exact GREEN every run); only the global g_rc_* stat counters (rc.c:90-91, 161,212) are non-atomic BY rc.c's own documented single-threaded design, losing ++s when 3 worker threads hit the null-__ail_tls_ctx host-side path. M5 is AILang's first concurrent consumer; rc.c's header (rc.c:44-49) explicitly defers exactly this atomic-vs-non- atomic decision to "when it acquires concurrency primitives". The leak assertion was Boss-split into `symbol_fan_swarm_leak_free` (#[ignore], body preserved VERBATIM — quarantined not weakened; un-ignore = the runtime fix's acceptance criterion) so main stays green and the finding is pinned as a regression marker. The earlier plan/journal claim that embed_swarm_tsan.rs covers this path was wrong and is corrected on the record (that test uses the scalar kernel — zero box allocs — so it never exercised the host-side global-counter path). Escalated as an M5 bounce-back: resolution touches M3-frozen runtime/ and re-frames M5's "zero runtime change" commitment; multiple substantive options, not unilaterally Boss's to pick in frozen-runtime territory. Boss recommendation on the record = Option A (make only the global-fallback g_rc_* counters atomic, as a standalone RED-first runtime micro-iteration; M5 framing amended). Time-shard + friction-harvest remain m5.3. Includes the per-iter journal (with Boss disposition), stats, and the INDEX.md line.	2026-05-19 01:37:50 +02:00
Brummel	204c171e60	iter embedding-abi-m5.1 (DONE 3/3): lean ail-embed core + build.rs + hermetic smoke ... M5 iteration 1 (spec ae905de, plan 22f02aa). Stands up the workspace-excluded `ail-embed` crate: - zero-dependency embedding core (`ail-embed/src/lib.rs`): extern "C" to the M3-frozen ABI + frozen-layout State/Tick box helpers + a Kernel price fold; the Rust port of the audited crates/ail/tests/embed/tick_roundtrip.c. Raw pointers never escape the type. - build.rs (no in-repo precedent): AIL_BIN env override else nested `cargo build -p ail` against the parent workspace (separate target dir → no cargo-lock deadlock), `ail build --emit=staticlib`, link directives. - hermetic data-server smoke (ail-embed/tests/smoke.rs): synthetic Pepperstone-format ZIP fixture via data-server's own public RawTickRecord type → real DataServer → Kernel, bit-exact vs a same-order host reference fold; runs with no /mnt. - `ail-embed` is its own cargo workspace root (empty [workspace] table); data-server is a dev-dependency only. Root Cargo.toml gains only a 4-line non-membership comment. Invariant 1 Boss-verified independently: full+no-deps cargo metadata on the AILang workspace shows data-server count 0; git status path filter empty (zero diff to crates/ailang-*, crates/ail/, runtime/, examples/*.ail); src/lib.rs zero code-level data_server; AILang cargo build --workspace still clean. ail-embed suite 2/2 green (kernel_run_sums_prices unit RED-first + hermetic_smoke integration), verified by me, not just the agent report. Two toolchain-forced corrections to the plan's verbatim ail-embed/Cargo.toml (added empty [workspace] table; sibling dev-dep path ../libs -> ../../libs, manifest-relative) — confined to the plan-created manifest, no acceptance gate altered, 0 review re-loops. Journal Concerns records the planner-recon implication for the next workspace-excluded-nested-crate plan. Adapter API + thread-swarm deferred to M5 iter 2+ per spec/plan. Includes the per-iter journal, stats, and the INDEX.md line.	2026-05-19 01:12:19 +02:00