Brummel
72e54f4fd3
The surface-form file extension changes from .ailx to .ail. AILang's
authoring surface now uses the same .ail stem as its canonical JSON
form (.ail.json), giving the language a single coherent extension
family: .ail is the LLM-authored Form A, .ail.json is the canonical
JSON-AST Form B.
Scope (touched):
- 61 example renames examples/**/*.ailx → .ail (git mv)
- 1 rename experiments/.../rendered/ailx.md → ail.md
- 35 content-edited live-toolchain files (crates/, docs/DESIGN.md,
docs/roadmap.md, docs/PROSE_ROUNDTRIP.md, skills/, bench/reference/*.c,
experiment crates under experiments/.../{render,harness,master})
- Experiment-crate cohort rename Cohort::Ailx → Cohort::Ail,
Form::Ailx → Form::Ail, per_cohort/ailx → per_cohort/ail,
{form-only: ailx} → {form-only: ail}, ```ailx → ```ail
Out of scope (deliberately untouched, to preserve honest history):
- docs/journal-archive.md (content-frozen per CLAUDE.md)
- docs/journals/, docs/specs/, docs/plans/, bench/orchestrator-stats/
- experiments/.../runs/ (frozen LLM-output artefacts; models actually
saw .ailx — renaming would falsify the experimental record)
Verification: cargo build/test --workspace green; experiment crate
cargo test green; bench/check.py + compile_check.py + cross_lang.py
all 0-regressed; negative grep for ailx|Ailx|AILX outside the
out-of-scope paths returns zero matches.
Opens immediate follow-up: roadmap.md P2 todo `ail check`/build/run
accept .ail extension — after this rename, .ail is canonical
authoring surface but the CLI still produces a misleading JSON-parse
error on `ail check foo.ail`. That's the next iter.
82 lines
3.1 KiB
Markdown
82 lines
3.1 KiB
Markdown
# Cross-model authoring-form test
|
|
|
|
Empirical measurement of whether `.ail.json` or `.ail` is the form a
|
|
foreign LLM author reaches for and succeeds with. Single subject for
|
|
v1: Qwen3-Coder-Next via IONOS. Two blind cohorts; same four tasks.
|
|
|
|
Parent spec: `docs/specs/2026-05-12-cross-model-authoring-form-test.md`.
|
|
|
|
## Layout
|
|
|
|
- `master/spec.md` — canonical mini-spec source (form-agnostic prose +
|
|
`{form-only: X}` blocks + `{example: id}` markers).
|
|
- `master/examples/*.ail.json` — AST source-of-truth; each example
|
|
prints to either form via the existing roundtrip machinery.
|
|
- `master/tasks/*.task.json` — task definitions consumed by the
|
|
harness. **Authored in cma.2**, not cma.1.
|
|
- `render/` — standalone Cargo crate, outside the root workspace,
|
|
builds the renderer binary.
|
|
- `rendered/json.md`, `rendered/ail.md` — projected mini-specs,
|
|
checked into the repo for review.
|
|
- `harness/` — **Authored in cma.2**.
|
|
- `runs/<date>-<hash>/` — populated by `harness` during a live run.
|
|
|
|
## Running the renderer
|
|
|
|
```
|
|
cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml -- \
|
|
--master experiments/2026-05-12-cross-model-authoring/master \
|
|
--rendered experiments/2026-05-12-cross-model-authoring/rendered
|
|
```
|
|
|
|
## Running the tests
|
|
|
|
```
|
|
cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml
|
|
```
|
|
|
|
Three integration tests: `example_roundtrip` (each example loads,
|
|
prints to AIL, reparses to the same canonical bytes), `spec_completeness`
|
|
(every AST variant in `ailang_core::ast` is exercised by at least
|
|
one example), `token_balance` (form-only blocks balanced within ±5%
|
|
across the two rendered files).
|
|
|
|
## Running the harness
|
|
|
|
Live mode (one full eight-run sweep, ~480k tokens budget by default):
|
|
|
|
```
|
|
export IONOS_API_TOKEN="<token>" # see roadmap entry for token provenance
|
|
cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml -- \
|
|
--rendered experiments/2026-05-12-cross-model-authoring/rendered \
|
|
--tasks experiments/2026-05-12-cross-model-authoring/master/tasks \
|
|
--out experiments/2026-05-12-cross-model-authoring/runs \
|
|
--model Qwen/Qwen3-Coder-Next
|
|
```
|
|
|
|
The harness pre-flights `ail --version` and `clang --version` before
|
|
the first API call. Set `AIL_BIN` if `ail` is not on PATH.
|
|
|
|
Mock mode (offline; CI-friendly; bypasses the IONOS API):
|
|
|
|
```
|
|
cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml -- \
|
|
--rendered experiments/2026-05-12-cross-model-authoring/rendered \
|
|
--tasks experiments/2026-05-12-cross-model-authoring/master/tasks \
|
|
--out /tmp/mock-runs \
|
|
--model mock \
|
|
--mock experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/mock_full_run.json
|
|
```
|
|
|
|
Tests:
|
|
|
|
```
|
|
cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml
|
|
```
|
|
|
|
Five suites: inline `--lib` unit tests for `strip_locations` (5),
|
|
plus integration tests `strip_locations` against captured stderr
|
|
fixtures (5), `verify_references` (1, drives every reference through
|
|
the real ail+clang pipeline), `mock_full_run` (1, full eight-row
|
|
mock E2E), `budget_abort` (1). Total 13 passed.
|