Brummel
525112e32e
Tested the user's bracket-free ideas (indent, YAML) against the annotated-paren result. Hypothesis was that removing the closing token removes the repetition tail. The opposite held. Instrument note (the watch-out paid off twice): a real YAML library auto-quotes and gives *,-,true special meaning, which collides with the AILang operators — Qwen wrote `- *` (multiply), a YAML alias marker, and the library crashed. That is a measurement bug, not a model failure. Rebuilt with a hand-written literal indent parser (atoms verbatim, no quoting); `- *` then round-trips fine. With the instrument fixed, the result is monotone across four formats: format 4 annotated parens 7/8 > plain Form-A parens 6/8 > yamlish indent 3/8 > YAML library 2/8. More explicit structure marking helps this model; less hurts. Qwen's weakness is structure-tracking at depth, and indentation makes it track the nesting itself — as error-prone as matching parens but without the redundant cue. The bracket-free surfaces fail EARLIER (L3/L4), before the L7 repetition ceiling is even reached. Lever for an LLM-authored language: redundant explicit structure (depth- annotated brackets), not a lighter surface. Synthesis in format-findings.md; raw ladders qwen-yaml.md (PyYAML, discarded) and qwen-yamlish.md (own parser).
Cross-model authoring-form test
Empirical measurement of whether .ail.json or .ail is the form a
foreign LLM author reaches for and succeeds with. Single subject for
v1: Qwen3-Coder-Next via IONOS. Two blind cohorts; same four tasks.
Parent spec: docs/specs/0017-cross-model-authoring-form-test.md.
Layout
master/spec.md— canonical mini-spec source (form-agnostic prose +{form-only: X}blocks +{example: id}markers).master/examples/*.ail.json— AST source-of-truth; each example prints to either form via the existing roundtrip machinery.master/tasks/*.task.json— task definitions consumed by the harness. Authored in cma.2, not cma.1.render/— standalone Cargo crate, outside the root workspace, builds the renderer binary.rendered/json.md,rendered/ail.md— projected mini-specs, checked into the repo for review.harness/— Authored in cma.2.runs/<date>-<hash>/— populated byharnessduring a live run.
Running the renderer
cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml -- \
--master experiments/2026-05-12-cross-model-authoring/master \
--rendered experiments/2026-05-12-cross-model-authoring/rendered
Running the tests
cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml
Three integration tests: example_roundtrip (each example loads,
prints to AIL, reparses to the same canonical bytes), spec_completeness
(every AST variant in ailang_core::ast is exercised by at least
one example), token_balance (form-only blocks balanced within ±5%
across the two rendered files).
Running the harness
Live mode (one full eight-run sweep, ~480k tokens budget by default):
export IONOS_API_TOKEN="<token>" # see roadmap entry for token provenance
cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml -- \
--rendered experiments/2026-05-12-cross-model-authoring/rendered \
--tasks experiments/2026-05-12-cross-model-authoring/master/tasks \
--out experiments/2026-05-12-cross-model-authoring/runs \
--model Qwen/Qwen3-Coder-Next
The harness pre-flights ail --version and clang --version before
the first API call. Set AIL_BIN if ail is not on PATH.
Mock mode (offline; CI-friendly; bypasses the IONOS API):
cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml -- \
--rendered experiments/2026-05-12-cross-model-authoring/rendered \
--tasks experiments/2026-05-12-cross-model-authoring/master/tasks \
--out /tmp/mock-runs \
--model mock \
--mock experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/mock_full_run.json
Tests:
cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml
Five suites: inline --lib unit tests for strip_locations (5),
plus integration tests strip_locations against captured stderr
fixtures (5), verify_references (1, drives every reference through
the real ail+clang pipeline), mock_full_run (1, full eight-row
mock E2E), budget_abort (1). Total 13 passed.