Milestone "Cross-model authoring-form test" closed. Live IONOS run
against Qwen/Qwen3-Coder-Next executed at temperature=0, max-turns=5,
token-budget=500000. Raw dataset under runs/2026-05-12-df7531/:
RUN_STATUS=ok, scores.csv with 8 rows (4 tasks × 2 cohorts),
summary.md, plus 156 per-turn raw artefacts (request/response/program/
stderr/stdout) preserved for any future analysis.
Headline numbers:
- AILX cohort: 2/4 green, 1/4 first-attempt, ~95k prompt + ~2.6k completion
- JSON cohort: 1/4 green, 0/4 first-attempt, ~192k prompt + ~15k completion
- Only first-attempt success was AILX on t3_main_prints (~5 lines of
tagged s-expr vs ~23 lines of structured JSON for the JSON cohort's
turn-1 attempt that needed correction).
- Per spec, single subject + single deterministic run = data point,
not verdict.
DESIGN.md §"Decision 6: authoring surface" gains an "Empirical
addendum (2026-05-12)" subsection: 6-row metric table, illustrative
t3 contrast, explicit single-subject scope note pointing at the
roadmap follow-up entry for multi-subject expansion.
Roadmap: P2 entry "Cross-model authoring-form test" removed (this
journal is the convention's one-line mirror); P3 entry
"Multi-subject expansion (cross-model authoring-form follow-up)"
added with the run dir named as baseline.
Milestone artefacts span three iters (cma.1 master mini-spec +
renderer; cma.2 harness + tasks + reference solutions; cma.3 this
live run + DESIGN.md addendum + close).
Sibling standalone Cargo crate `harness/` under
experiments/2026-05-12-cross-model-authoring/ (out-of-workspace
idiom carried forward from cma.1 verbatim). Six modules:
strip_locations (regex pass for form-asymmetric location info,
calibrated against five real `ail check`/`ail parse` stderr
captures), pipeline (subprocess wrapper for parse|check|build +
5s-timeout exec; preflight on ail+clang), ionos (blocking reqwest
client + retry policy per spec), mock (canned-response loader),
scoring (CSV + summary.md), tasks (definition struct + loader).
main.rs ties them into the per-(cohort,task) loop with budget
accounting and per-turn artefact recording.
Four MVP tasks land with reference solutions that compile, build,
and execute green through the actual ail+clang pipeline:
t1_add_three (chained `+` + io/print_int), t2_length (polymorphic
List + recursion), t3_main_prints (minimal IO module), t4_count_zeros
(prelude Eq Int + branched if). Reference solutions stay in canonical
AILang form — param_modes is omitted when every parameter is the
Implicit default, consistent with the existing examples/ corpus.
13/13 tests green: 5 lib unit (strip_locations) + 5 integration
(strip_locations against verbatim captured fixtures) + 1
verify_references (drives every reference through the real
ail+clang pipeline) + 1 mock_full_run (full eight-row sweep with
mixed green-on-turn-2 + turn-limit cycles) + 1 budget_abort
(synthetic budget exhaustion with budget_abort rows + run_status).
Two implementer-phase repairs beyond the plan, both small and
surfaced in the iter journal Concerns:
1. pipeline.rs renames the program file to `<module-name>.ail.json`
between parse and check because `ail check` enforces filename
stem == module name (compiler contract the plan did not anticipate).
2. main.rs fills synthetic budget_abort rows for tasks the outer
loop never reached so scores.csv preserves the expected eight-row
shape on budget exhaustion.
One plan/text mismatch carried over: README "Total 8 passed" reflects
the plan's four-suite count; actual sweep produces 13. Doc-fix
candidate for cma.3.
cma.3 (live IONOS run + DESIGN.md addendum + roadmap edits) remains
out of scope.
New top-level experiments/2026-05-12-cross-model-authoring/ (out of
root Cargo workspace; nested-crate standalone via empty [workspace]
table per Cargo idiom). The renderer projects one canonical
master/spec.md into two form-specific files (rendered/json.md and
rendered/ailx.md) by walking three directive forms ({form-only:
json}, {form-only: ailx}, {example: <id>}) and emitting per-example
fenced blocks sourced from 13 curated .ail.json fixtures.
Four test gates green (10/10 total):
- splitter_unit (7) — directive parser, malformed inputs panic
- spec_completeness (1) — AST-variant visitor lifted verbatim from
schema_coverage.rs; 34/34 variants covered by the curated subset
- example_roundtrip (1) — every fixture roundtrips through
ailang_surface::{print, parse} to identical canonical bytes
- token_balance (1) — form-only block totals balanced to 3.22% under
tiktoken-rs::cl100k_base (gate is ±5%)
Six of the 34 variants did not have a dedicated fixture topic in the
plan and were absorbed into the closest existing fixture per the
plan's Step 4.14 escape hatch (Let/Clone → fn_calls_prelude; LetRec
→ data_with_match; If → match_literal_pattern; ReuseAs →
data_simple; Const → floats). Surfaced as a journal note, not a
plan revision.
Two ancillary additions vs. plan, both structural Cargo necessities,
spec-compatible: render/Cargo.toml carries an empty [workspace] table
so the nested crate refuses parent-workspace discovery, and
render/.gitignore drops /target + Cargo.lock. Both surfaced inline
in the per-iter journal's Concerns section.
cma.2 (harness) and cma.3 (live IONOS run + DESIGN.md addendum +
journal + roadmap edits) remain out of scope per the spec.
CLAUDE.md previously mixed universal facts (agent role boundaries,
commit discipline, design rationale, file roles, TDD-for-bugs) with
mode-specific autonomy rules (direction freedom, notifications,
WhatsNew procedure). Autonomous-by-default conflicted with the user's
intent that a fresh session should be collaborative-interactive unless
explicitly elevated.
Add `skills/boss/` containing only the three genuinely mode-specific
subsections — Direction freedom, Notifications, Done-state notifications:
WhatsNew.md. Trim CLAUDE.md from 343 to 243 lines; extend the skill-
system pointer paragraph with a one-sentence /boss gate. Universal
orchestrator discipline stays in CLAUDE.md because it applies whether
/boss is active or not.
Two cross-references that named the moved subsections by sub-heading
are repointed: skills/implement/SKILL.md and the implement-orchestrator
agent's standing reading list. The other ~11 agent-file references to
"orchestrator framing" still resolve correctly because that framing
stays in CLAUDE.md.
skills/README.md skill table extended with a `boss` row (now eight
skills); pipeline-diagram caption notes /boss wraps the pipeline.
.claude/skills/boss symlink follows the existing relative-path
convention.
Milestone-close audit found three drift items (architect) plus
two carry-on items; bench all-green.
Fixes (rt.tidy, boss-direct):
- 5th test parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture
enforces Direction 2 of the Roundtrip Invariant directly. The
DESIGN.md enforcement list grows from four to five tests.
- docs/roadmap.md P1 'Round-trip completeness invariant' entry
retired with one-line journal mirror per roadmap convention.
- DESIGN.md wording sync: <16-hex> → <16-lowercase-hex> for the
Float-bits-hex spelling in the new section (matches §Data model).
Milestone closed clean; five workspace-wide tests anchor the
.ail.json ↔ .ailx bijection plus AST-variant coverage.
New top-level section '## Roundtrip Invariant' between Decision 11
and '## Mangling scheme'. States both directions of the .ail.json
↔ .ailx bijection, names Float-bits-hex encoding as the in-
invariant mechanism (not an exception), lists the four enforcement
tests by path, and explains why the property is top-level
(load-bearing on language identity, not on Decision 6's surface-
design rationale).
Decision 6 Constraint 2 rewritten as upward cross-reference to the
new section; the substantive content now lives at the anchor.
Direct boss edit (no planner/implement dispatch): spec Architecture
#1 fully specified the content, no design judgement deferred to
execution. The iter is a deterministic projection of the spec
onto DESIGN.md prose.
Three new reader-only tests pin the .ail.json / .ailx bijection
plus AST-variant coverage. All three passed first observation
across the current corpus — no roundtrip, schema-coverage, or
CLI-drift gaps surfaced.
- every_ailx_fixture_matches_its_json_counterpart (replaces the
3-pair handwritten exhibits): 57 paired .ailx fixtures pass.
- every_ast_variant_is_observed_in_the_fixture_corpus (new): 34/34
AST variants exercised across 136 fixtures; exhaustive matches
without _ wildcard so AST drift fails the build.
- cli_render_then_parse_preserves_canonical_bytes_on_every_fixture
(new): 136 fixtures round-trip through ail render → tempfile →
ail parse with BLAKE3 identity on canonical bytes.
No production code, no DESIGN.md changes (those follow in later
iterations). Tests are pure readers of the repo per spec
acceptance #7 — tempfile crate added as workspace dep for the CLI
roundtrip's intermediate file outside the repo.
Milestone-23 close audit. Architect clean on substance; one
pre-existing DESIGN.md grammar stub (line 1692, dangling 'The
original' from 2026-05-10 design-md-consolidation) caught and
fixed.
Bencher localised the compile_check.py regression to H2 (the 5 new
polymorphic prelude Def::Fns iter 23.5 added — each carries forall
+ class constraint + match/not body, all typechecked on every
workspace check). H1 (linearity workspace-visibility extension +
contains_rigid_var) is negligible at today's 2-module workspace.
The regression is the feature's measured cost. Ratified per audit
skill (--update-baseline + journal entry). 0/24 metrics regressed
post-update.
Milestone 23 (Eq/Ord prelude) is closed.
Five polymorphic prelude free fns (ne/lt/le/gt/ge) plus three E2E
fixtures: positive composition over Int/Bool/Str, user-ADT with
user-written Eq/Ord on IntBox, and Float-NoInstance with a
Float-aware diagnostic addendum cross-referencing DESIGN.md §"Float
semantics".
Two checker-side fixes were needed alongside the prelude additions,
both symmetric to the iter 23.4-prep visibility fix:
- linearity::check_module_with_visible — workspace-aware callee-mode
resolution so a Borrow-mode user fn forwarding into a prelude poly
fn (e.g. `at_most x y = not (gt x y)`) doesn't false-fire
consume-while-borrowed.
- mono::collect_mono_targets — distinguish rigid forall vars from
unbound metavars; rigid-bearing free-fn targets skip (they get
re-collected with concrete subst when the enclosing fn itself
monomorphises). Without this, a polymorphic body calling another
polymorphic free fn produced a spurious __Unit specialisation
whose body referenced compare at Unit.
Roadmap split: shipped Eq/Ord half checked, Show + print-rewire half
remains pending behind the heap-Str ABI prerequisite. DESIGN.md
§"Prelude classes" amended.
One pre-existing fixture (test_22b1_missing_method) renamed its
class method ne → tne to avoid collision with the new prelude `ne`.
Bench: check.py + cross_lang.py exit 0; compile_check.py exits 1
with one sub-ms check_ms.local_rec_capture at +26% (tolerance 25%) —
prelude-growth-related noise-band fluctuation, ratifiable per spec
§248-249 and journal Concerns.
Restructure ailang-check/src/mono.rs::monomorphise_workspace into a
single specialiser over every Type::Forall-quantified Def::Fn, covering
both class-method residuals AND polymorphic free-fn call sites in one
fixpoint pass. Remove the codegen-time specialiser (lower_polymorphic_call,
module_polymorphic_fns, mono_queue, mono_emitted, emit_specialised_fn,
plus the now-dead helpers apply_subst_to_term, descriptor_for_subst,
type_descriptor) entirely. Codegen sees only monomorphic Def::Fns after
this iter — no polymorphic call sites, no codegen-time queue.
Architecture changes:
- MonoTarget: struct → enum with ClassMethod / FreeFn variants; dedup
keys are guaranteed-disjoint via 'class'/'free' first component.
- collect_mono_targets: new FreeFnCall channel through synth (cleaner
than the plan's separate-walker proposal — one less data flow,
substitution tracking comes 'for free' via fresh metavars +
post-synth subst.apply). Plan Step 4.4 (polymorphic_free_fns env
field) consequently obsolete.
- synthesise_mono_fn_for_free_fn: new free-fn entry point; substitutes
rigid vars in BOTH the type AND the body (body substitution required
because free-fn bodies carry inner Lams whose param_tys reference
outer Forall vars — adapted from plan's 'body unchanged' sketch).
- mono_symbol_n: N-ary extension of mono_symbol; bit-stable for the
single-type-var case (hash-stability pin Task 1 fires zero times
through all eight refactor tasks).
- rewrite_class_method_calls → rewrite_mono_calls; interleaved-slot
collector preserves the positional-cursor invariant across both
channels.
- workspace_has_typeclasses → workspace_has_specialisable_targets
(principled correctness; old predicate happened to already be true
for every user workspace because the prelude is auto-injected).
Codegen cleanup:
- Two call sites of lower_polymorphic_call deleted (codegen/src/lib.rs
lines ~1939-1945, ~1965-1973).
- lower_polymorphic_call body, module_polymorphic_fns field +
population + Emitter wiring, mono_queue / mono_emitted, drain loop,
emit_specialised_fn — all removed.
- is_static_callee collapsed to module_user_fns-only.
- subst.rs apply_subst_to_term + descriptor_for_subst removed; synth.rs
type_descriptor removed (all dead post-removal).
- ail emit-ir command pipeline gains lift_letrecs + monomorphise_workspace
pre-passes (pre-iter-23.4 the codegen-time poly path was masking
this dependency — emit-ir is now consistent with build).
- Net codegen reduction: -285 lines lib.rs, -93 lines subst.rs.
Tests:
- mono_hash_stability.rs (new): regression pin for six primitive Eq/Ord
mono-symbol body hashes; stayed bit-stable through all eight refactor
tasks.
- mono_unification.rs (new): six tests covering variant-key disjointness,
free-fn target emission, free-fn synthesis with rigid substitution,
rewrite walker on free fns, identity for poly-free-fn-only workspaces,
end-to-end cmp_max_smoke runtime correctness.
- typeclass_22b3.rs: three test sites updated to MonoTarget::ClassMethod
enum form.
- 73 e2e + 18 typeclass + 81 codegen tests all green workspace-wide.
DESIGN.md §'Monomorphisation (post-typecheck, pre-codegen)' amended:
two-arm fixpoint described, disjoint-keyed dedup, cursor-aligned
walker, removed codegen-time path.
Bench check (all exit 0, 112 metrics stable): bench/check.py +
compile_check.py + cross_lang.py.
Plan parent: docs/plans/2026-05-11-iter-23.4.md (fab1685).
Spec parent: docs/specs/2026-05-11-23-eq-ord-prelude.md (841d65d).
Per-iter journal documents two adapted deviations from the plan
(synth-channel vs separate walker; body substitution; emit-ir fix);
no spec defects.
Two project-wide rules are now explicit across every skill:
1. Only the Boss commits. No skill agent (implementer,
brainstormer, planner, debugger, fieldtester, docwriter,
architect, bencher) runs `git commit`. Agents write their
artefacts to the working tree as unstaged changes; the Boss
inspects, decides commit shape, and commits.
2. main HEAD is sacrosanct. No actor runs `git reset` or
`git revert` on main. Bad work stays in the working tree
where it is still discardable via `git checkout -- <paths>`.
Implement loses the `iter/<iter_id>` branch mechanic entirely;
Phase 0 of the orchestrator-agent now does a clean-tree check
and refuses to start on a dirty tree. Per-task agent commits
are removed everywhere; reviewers operate against
`git diff HEAD` instead of `pre_task_sha..head_sha`.
Motivation: 2026-05-11 iter 23.4 stranded prep2/prep3 commits on
an iter-branch that never integrated to main, then a corrected
spec falsely claimed those commits had shipped. Branch-per-iter
+ manual-Boss-merge + iter-stacking made the strand structurally
possible. See docs/journals/2026-05-11-iter-disc.1.md for the
full per-task notes and motivation.
Note 3: agent output format leaked internal-thinking preamble before
the structured report block — Iron Law should forbid preamble.
Note 4: Boss-requested behaviour change for impending-BLOCK paths —
on miss, agent attempts to write missing GREEN test (test code only,
never src/) before falling to BLOCK; requires tool + Iron Law update.
Both notes are for the future skillsystem-postmortem entry; out of
gc.1 scope.
Claude Code categorically forbids subagents from spawning other
subagents (code.claude.com/docs/en/sub-agents and the Agent SDK
subagents page). The `or.1` architecture (and `pr.1` that ran on
top of it) presumed a named-exception for `ailang-implement-
orchestrator` to dispatch implementer / spec-reviewer / quality-
reviewer / tester per task. That exception never existed at the
platform level; the `Agent` tool was silently dropped from the
orchestrator-agent's tool set at dispatch time.
Architecture revised, doc-only:
- Per-task phases (implementer → spec-compliance check → quality
check) now run as sequential role-switches in the orchestrator-
agent's own context, not as nested subagents.
- The four role-files (implementer / spec-reviewer / quality-
reviewer / tester) become phase reference files the
orchestrator-agent consults at role-switch boundaries.
- Boss-context offload preserved; fresh-per-phase context given
up (the property that drove or.1's nested-dispatch design is
not buildable in Claude Code).
Files touched:
- skills/implement/agents/ailang-implement-orchestrator.md
(frontmatter, Iron Law, Phase 2/3 rewrite, rationalisations,
red flags)
- skills/implement/SKILL.md (frontmatter, Iron Law, sub-status
vocabulary note, cross-references)
- skills/README.md (Conventions: no more named exception; agent
roster: role-files recast as phase references)
- docs/journals/INDEX.md (or.2 entry appended)
- docs/journals/2026-05-11-iter-or.2.md (new — full rationale)
- docs/roadmap.md (P1 tool-wiring item removed; resolved as
categorically-not-fixable)
- docs/WhatsNew.md (user-facing correction entry appended)
No code changes; no bench impact; CLAUDE.md untouched (no stale
references to fix there).
Per-iter journal entry for the fieldtest run plus INDEX pointer.
Roadmap absorbs four actionable findings as P2/P3 todos (split the
merged BadCrossModuleTypeRef diagnostic, workspace subdir search,
`ail check` on `.ailx` extension, `.ailx` counterpart for the
canonical happy-path JSON exhibit). The two carry-on items
(`(import prelude)` doc-gap, the working ones) stay as notes in the
fieldtest spec only. P0 Floats milestone removed — closed
2026-05-10 and no longer load-bearing context.
No bug findings, so no debug dispatch. canonical-type-names is fully
shipped.
Brummel