AILang/docs/specs/0025-form-a-default-authoring.md

Form-A as the Default Authoring Surface — Design Spec

Date: 2026-05-13 Status: Draft — awaiting user spec review Authors: Brummel (orchestrator) + Claude

Goal

Make Form A (.ail) the sole authoring surface for every program in the working tree. After this milestone, the only .ail.json files that remain in the repository are eight specific negative-test fixtures whose subject matter cannot be expressed in Form A by construction. Every other .ail.json is rendered to its .ail sibling via ail render, and the now-redundant .ail.json is deleted — not retained as a generated artefact, not stashed in a build directory, not regenerated at test time. The working tree contains exactly one representation per program: the .ail source.

The language doctrine does not change. The canonical, hashable, content-addressed form remains the JSON-AST (Decision 6, architectural pin, §"data structure is the source of truth"). What changes is the authoring doctrine: authors write .ail; the build derives the JSON-AST in-process when a consumer needs it, gated by the round-trip invariant.

Driving evidence: the cross-model authoring-form study (audit-cma, audit-ms, 2026-05-12) measured that Form A is cheaper in tokens (~98k vs ~207k for the same four tasks) and produces strictly more first-shot green programs than JSON-AST authoring, across two distinct foreign LLMs (Qwen3-Coder-Next and CodeLlama-13b-Instruct). The internal authoring practice is now drift from the empirical "this is the form LLMs reach for unprompted" finding.

Architecture

Four architectural decisions land in this spec; each is the substantive answer to one of the open questions in the roadmap entry.

A1. In-process parse, no build-time artefacts

Tests and benches consume .ail fixtures by calling ailang_surface::parse (library, in-process) — not by spawning ail parse as a build prelude that emits .ail.json to a target directory.

Why. A build-prelude approach reintroduces a second representation: examples/foo.ail (source) plus target/parsed/foo.ail.json (derived). Even with the derived form gitignored, it would be a working-tree fact during tests and a shadow corpus for downstream tools to drift against. The single- representation discipline that motivates this milestone is broken the moment a derived .ail.json exists on disk anywhere a tool might read it.

The in-process path is also already plumbed: since iter ext-cli.1 (2026-05-12), every path-taking CLI subcommand accepts .ail and parses through ailang_surface::parse in-line. The Python bench drivers (bench/check.py, bench/compile_check.py, bench/cross_lang.py, bench/mono_dispatch.py) call ail check etc. via subprocess and inherit the .ail acceptance for free — they need only a path-suffix change. The integration tests under crates/ail/tests/*.rs and the workspace tests under crates/ailang-check/tests/*.rs currently fs::read_to_string a .ail.json and pass the bytes to a loader; they will switch to reading the .ail and calling ailang_surface::parse to recover the same Module value.

A2. prelude.ail ships as iter-0 pilot

Iteration 0 converts exactly one file: examples/prelude.ail.json → examples/prelude.ail. Nothing else moves. The roundtrip-CI gate covers it (every .ail with a same-stem .ail.json counterpart is already byte-compared by every_ail_fixture_matches_its_json_counterpart), which proves the conversion path end-to-end before the bulk migration commits to it.

Why. Prelude exercises the widest cross-section of language features in a single file: typeclasses with superclasses, polymorphic free functions with constraints, IO-effect signatures, primitive ctor patterns, and the loader's auto-injection path. If prelude round-trips identically through ail render → ail parse → canonical bytes, no other fixture in the corpus introduces a feature this file does not. The pilot validates the migration mechanism on the hardest single file before the bulk migration commits the test- infrastructure refactor.

Iter 0 keeps prelude.ail.json on disk alongside prelude.ail — the deletion of redundant JSON-AST starts in iter 1 once the test-infrastructure refactor lands. This is the only iteration where a fixture is dual-form; it is justified by the existing every_ail_fixture_matches_its_json_counterpart gate, which is exactly the byte-comparison this iteration wants to leverage as a witness.

A3. Per-file deletion cadence with iteration-end checkpoints

.ail.json files are deleted on the same iteration that materialises their .ail counterpart, not at milestone end. The bulk migration is sequenced across ~3 iterations of ~30-40 files each, each iteration fully reaping its own .ail.json files before close. There is no intermediate state where the bulk of fixtures are dual-form.

The exception is iter 0 (the prelude pilot, A2 above), which is the singular dual-form window: the JSON counterpart is the witness the existing CI gate uses, and the gate is the safety net for the pilot itself.

Why. A milestone-end bulk delete would mean the working tree spends weeks in a "which form is the source?" ambiguity. Anyone editing a fixture in the interim — to repair an unrelated bug, add a regression test, fix a hash drift — has no canonical guide to which form to edit. The single-representation property must hold on a per-commit basis, not as a milestone-end snapshot.

The trade-off is iteration sequencing pressure: every iteration that ships in this milestone must leave the tree in a coherent single-representation state. This is enforced by the per-iteration roundtrip-CI rewrite (see Testing strategy below): once the infrastructure flips in iter 1, fixtures without .ail.json are the new normal, and fixtures with .ail.json are visibly in-flight.

A4. CLAUDE.md and DESIGN.md rewording

Two documents need substantive rewording in this milestone. Both edits land in the same commit as the iteration that retires the last non-carve-out .ail.json.

CLAUDE.md §"AILang — a language for LLM authors" opening:

Currently:

Machine readability over human readability. Source of truth is structured data (.ail.json), not text.

After this milestone:

Machine readability over human readability. The canonical, hashable, content-addressed form is structured data (.ail.json); the authoring projection is Form A (.ail). Authors write .ail; the build derives the JSON-AST in-process via ailang_surface::parse, gated by the round-trip invariant. The two forms are byte-isomorphic — picking either does not change the identity of the module.

DESIGN.md §"What this Decision deliberately does not do" (line 465-466), currently:

It does not promote form (A) to the source of truth. Form (A) is one projection; the JSON-AST stays canonical.

The first sentence is replaced; the second stays. New text:

It does not change which form is canonical: the JSON-AST remains the hashable, content-addressed representation, and all hashing, content-addressing, and cross-module references flow through it unchanged. What did change in 2026-05-13 (this milestone) is which form is authored: authors now write Form A; the JSON-AST is materialised in-process by callers that need it. The two forms are byte-isomorphic by the round-trip invariant.

The "Form (A) is one projection; the JSON-AST stays canonical" phrasing is preserved verbatim in the second sentence because it remains true: Form A and the eventual visual / graphical projections are still parallel projections of the same AST. What the milestone changes is which projection is the entry point, not the projection's relation to the AST.

Components

C1. Migration script

A one-shot Bash script scripts/migrate-to-form-a.sh (new file) walks examples/*.ail.json, skips the eight carve-outs (named explicitly in the script body), invokes ail render to produce the sibling .ail, and rms the JSON counterpart. The script is idempotent (re-running on a partially migrated tree is a no-op for already-migrated files). The script is retired at milestone close — it has no second use; keeping it would invite drift the next time someone wonders "where did the JSON-AST come from".

C2. Test infrastructure refactor

48 files under crates/ and bench/ reference .ail.json paths. The refactor is mechanical but spans three classes of consumer:

1. Roundtrip tests (crates/ailang-surface/tests/round_trip.rs, crates/ail/tests/roundtrip_cli.rs). The five-test invariant restructures (see Testing strategy §T below).
2. Fixture-loading integration tests (crates/ail/tests/*.rs, ~35 files). Each test currently reads examples/<stem>.ail.json with fs::read_to_string and passes the bytes to Module::from_json or the workspace loader. The refactor reads examples/<stem>.ail and parses via ailang_surface::parse_module_from_str. The eight carve-outs continue to use the .ail.json path (they have no .ail form by construction).
3. Bench drivers (bench/*.py, bench/run.sh). The drivers spawn ail check etc. with fixture paths. Since ext-cli.1 the subcommands accept both extensions; the drivers' fixture walks change from glob("examples/*.ail.json") to glob("examples/*.ail") plus an explicit allow-list for the carve-outs.

C3. Roundtrip invariant restatement

§"Roundtrip Invariant" at DESIGN.md:2027 currently anchors on a two-way .ail.json ↔ .ail bijection. After this milestone the JSON-AST counterpart no longer exists for the bulk of the corpus, so the invariant restates as:

• Parse-determinism. For every well-formed .ail text t, parse(t) produces a unique AST (the loader is a pure function; no randomness, no time dependence, no environment leak).
• Idempotency under print. canonical(parse(t)) == canonical(parse(print(parse(t)))) — printing and re-parsing is a no-op on the canonical bytes.
• CLI-pipeline idempotency. The public CLI pipeline ail render | ail parse (one round) is byte-identical to direct ail parse of the source .ail. Pins the user-facing CLI against drift internal tests cannot see.
• Carve-out anchor. The eight .ail.json-only fixtures are negative-tests (load-time / typecheck-time rejection); they participate only in the existing rejection-shape tests, not in the roundtrip gate. Their presence is the only surviving reason .ail.json remains a first-class file extension.

The "JSON → text → JSON" direction (Direction 1 in the current spec) loses its corpus once the JSON files are deleted. The property is replaced by parse-determinism plus idempotency — the two together discharge the load-bearing concern (hashing is well- defined for any .ail source).

C4. The carve-outs (immutable list)

Two structural reasons keep .ail.json alive post-milestone: (a) subject-matter carve-outs (seven files; the canonical list as drafted) and (b) compile-time-embed carve-outs (one file as of 2026-05-13; subject to a separate milestone). Both gain no .ail sibling.

(a) Subject-matter carve-outs (immutable, seven files):

examples/test_ct1_bare_xmod_rejected.ail.json
examples/test_ct1_qualified_class_rejected.ail.json
examples/test_ct1_bad_qualifier.ail.json
examples/broken_unbound.ail.json
examples/test_22b2_invalid_superclass_param.ail.json
examples/test_22b2_kind_mismatch.ail.json
examples/test_22b2_unbound_constraint_var.ail.json

Justification per file:

• test_ct1_* (3): assert canonical-form rejection of malformed type references. The Form-A parser would either accept the reference (test loses its subject) or reject it at parse time with a different diagnostic (test loses its meaning). The fixture's purpose is to test the loader/checker rejection shape, which requires the input to bypass the parser.
• broken_unbound: asserts unbound-variable rejection at typecheck. The .ail.json form encodes a structurally well-formed but semantically broken module that the Form-A parser would also parse cleanly; the file is retained in JSON-form not because the surface cannot express it but because the test already pins the exact JSON bytes, and re-deriving from .ail would add no value while adding a regeneration step.
• test_22b2_* (3): assert class-definition rejection shapes (invalid superclass param, kind mismatch, unbound constraint var) that the Form-A parser would either reject at parse time (changing the diagnostic) or refuse to encode at all (changing the test subject).

Status: RETIRED 2026-05-14 by milestone prelude-decouple. The §C4 (b) compile-time-embed carve-out is empty post that milestone; the prelude is now authored exclusively as examples/prelude.ail and parsed at compile time via ailang_surface::parse_prelude. The original §C4 (b) framing below is preserved as historical context; it documents the doctrine inconsistency that prelude-decouple resolved. See docs/specs/0028-prelude-decouple.md and the per-iter journals 2026-05-14-iter-pd.{1,2,3}.md for the resolution.

(b) Compile-time-embed carve-outs (one file as of 2026-05-13):

examples/prelude.ail.json

Justification:

• prelude.ail.json is include_str!-embedded at crates/ailang-core/src/workspace.rs:417 and crates/ail/src/main.rs:474, then parsed via serde_json::from_str at compile-time-fixed sites in production code. Switching the embed to prelude.ail would require parsing via ailang_surface::parse, but ailang-core cannot import ailang-surface (crate-cycle constraint documented at crates/ailang-surface/src/loader.rs:9-13). The mechanism is a structurally distinct from the runtime-loaded-program corpus the rest of this milestone migrates: .ail.json carve-outs in category (a) are kept because their content cannot be expressed in Form A; prelude.ail.json is kept because its embed mechanism requires the JSON-AST form.

  This is a real spec gap relative to §A4's authoring-doctrine ("authors write .ail; the build derives the JSON-AST in-process via ailang_surface::parse"). The prelude is currently authored as .ail.json and embedded as bytes; the doctrine says it should be authored as .ail and parsed at compile time. Resolving the inconsistency requires either (i) splitting the workspace loader so prelude-init lives in ailang-surface, (ii) generating prelude.ail.json at build time from prelude.ail via a build.rs, or (iii) some other refactor — all of which exceed the scope of this milestone.

  The resolution lives in a separate roadmap milestone: "Prelude embed: Form-A as compile-time source" (see docs/roadmap.md). After that milestone closes, this carve-out category becomes empty and the spec text drops it.

  Note: examples/prelude.ail was rendered in iter form-a.0 and remains the authoring projection (it stays in the tree; both forms coexist for prelude specifically, mirroring the iter-0 pilot's dual-form witness arrangement, until the prelude-embed-refactor milestone resolves the inconsistency).

Future additions to either list require brainstorm-level justification on the same structural grounds — not "this test is annoying to migrate" or "the diff is large", but a named mechanism (canonical-form rejection / compile-time embed / etc.) that the surface cannot satisfy.

Data flow

Before this milestone:

author writes:          examples/foo.ail.json
                              |
                              v
                    [loader / serde_json]
                              |
                              v
                   Module value -> typecheck, codegen, ...

After iter 0 (prelude pilot only):

author writes:          examples/prelude.ail            (new)
                              |
                              v
                  [ailang_surface::parse]
                              |
                              v
                   Module value -> typecheck, codegen, ...

                                                  [byte-witness]
                        examples/prelude.ail.json  ----+   (retained
                              |                         |    for iter 0,
                              v                         |    deleted iter 1)
                    [loader / serde_json]               |
                              |                         |
                              v                         |
                   Module value -----> canonical -------+

After milestone close (bulk migration done, JSON counterparts deleted):

author writes:          examples/<name>.ail             (every program)
                              |
                              v
                  [ailang_surface::parse]
                              |
                              v
                   Module value -> typecheck, codegen, ...

except for the eight carve-outs:
                        examples/<carve-out>.ail.json
                              |
                              v
                    [loader / serde_json]   (negative-test only)

Cross-module imports continue to use module names, not file paths; the workspace loader walks examples/* and accepts either extension for any module it loads. Since the bulk of the corpus is single- extension post-migration, the loader's extension-dispatch is exercised only by the eight carve-outs (and by any test that explicitly mixes the two forms for migration testing).

Error handling

The migration is mechanical and the round-trip invariant discharges correctness. There are three failure modes the spec must name explicitly because they could escape the existing CI net:

E1. A fixture round-trips before migration but the migrated .ail fails to parse. This would be a bug in ail render or ail parse; the round-trip CI gate (every_ail_fixture_matches_its_json_counterpart) already covers it for every fixture with a hand-written .ail sibling. The migration script extends the same property to every fixture by construction (the script's output is exactly the witness the gate demands). If iter 0 (prelude pilot) fails this gate, the bug is in render/parse; the milestone halts and the bug fix takes precedence.

E2. A test that depends on the exact byte shape of .ail.json breaks after migration. Some tests (e.g. snapshot tests) compare byte-equal canonical JSON to a checked-in expected blob. These tests do not break — the canonical bytes are derived from the AST, and the AST is form-agnostic. The migration touches only the fixture-loading path, not the canonical-encoding path.

E3. A bench driver receives a .ail path and emits stale .ail.json paths in its report. The bench drivers' report format echoes the fixture name; the suffix change from .ail.json to .ail propagates into report bytes. The audit-* JSONs under bench/orchestrator-stats/ will start carrying .ail paths after the next milestone-close audit; this is intended and not a regression.

A failure mode the spec explicitly does not worry about: parse non-determinism. The roundtrip-CI gate has been green across 9 consecutive audits (audit-cma onward); the parser is well- exercised.

Testing strategy

T1. Roundtrip-CI test restatement

crates/ailang-surface/tests/round_trip.rs currently runs three tests against the .ail.json corpus. Post-migration, the three restate:

Pre-migration test	Post-migration test	Behaviour
print_then_parse_round_trips_every_fixture (over .ail.json)	replaced by parse_is_deterministic_over_every_ail_fixture (over .ail)	Reads every .ail, parses twice, asserts canonical-byte equality between the two parses. Asserts pure-function-of-input.
parse_then_print_then_parse_is_idempotent_on_every_ail_fixture	unchanged	Reads every .ail, asserts parse == parse ∘ print ∘ parse.
every_ail_fixture_matches_its_json_counterpart	retired	Loses its corpus. The property it pinned (no drift between two hand-authored forms) is no longer expressible because only one form is hand-authored.

crates/ail/tests/roundtrip_cli.rs::cli_render_then_parse_preserves_canonical_bytes_on_every_fixture restates as cli_parse_then_render_then_parse_is_idempotent: reads every .ail, runs ail parse (to canonical JSON tempfile), runs ail render (back to .ail tempfile), runs ail parse again, and asserts byte equality across the two parses.

crates/ailang-core/tests/schema_coverage.rs::every_ast_variant_is_observed_in_the_fixture_corpus flips its corpus walk from .ail.json to .ail (load via ailang_surface::parse_module_from_str); the assertion is unchanged. The schema-coverage property is the strictest gate that survives: a new AST variant cannot land without a fixture that uses it.

T2. Per-iteration regression discipline

Every iteration in this milestone must leave cargo test --workspace green and bench/compile_check.py exit-0. The migration is mechanical; non-green at iteration close is a defect in the iteration's test-infrastructure refactor, not in the language. The iteration does not close until the refactor is green.

T3. Test-naming discipline for the migration

A few existing tests carry .ail.json in their test-function names (e.g. mono_recursive_fn_loads_from_json_ast). These names are renamed in the same iteration that migrates the corresponding fixture: the new name should not lie about what file it reads. This is mechanical, but it has to land in the migration commit, not as a follow-up rename.

T4. Carve-out coverage check

A new test crates/ailang-core/tests/carve_out_inventory.rs asserts that exactly the eight named carve-out files exist under examples/*.ail.json after milestone close (seven from §C4 (a) + one from §C4 (b)). The test fails if a new .ail.json is added or a carve-out goes missing. The list is hardcoded in the test body — this is intentional: the carve-out list is part of the language's negative-test surface and any change to it is a deliberate, brainstorm-level decision. The test's pass-count drops to seven once the prelude-embed-refactor milestone retires prelude.ail.json from category (b); the test body is then re-published with the smaller list in that milestone's iter.

Acceptance criteria

The milestone closes when all of the following are true:

1. ls examples/*.ail.json | wc -l returns 8, and the eight files match the carve-out list verbatim (§C4 category (a) + category (b)).
2. Every program in examples/ other than the eight carve-outs exists as a .ail file with no .ail.json sibling. examples/prelude.ail exists in addition to examples/prelude.ail.json per §C4 (b) (dual-form for prelude only, pending the prelude-embed-refactor milestone).
3. The roundtrip-CI restatement (T1) is green: parse is deterministic, idempotent under print, CLI-pipeline idempotent, and the carve-out inventory test (T4) passes.
4. cargo test --workspace is green at milestone close (no regressions introduced by the test-infrastructure refactor).
5. bench/compile_check.py and bench/cross_lang.py exit 0 against the migrated corpus.
6. bench/check.py is either exit 0 or exit 1 with audit- ratifiable noise only (consistent with the 9-audit lineage since audit-cma); a new regression class attributable to the migration triggers brainstorm-level escalation.
7. CLAUDE.md and DESIGN.md carry the reworded paragraphs (§A4 above) verbatim.
8. The migration script scripts/migrate-to-form-a.sh is deleted from the working tree (one-shot, retired).
9. docs/WhatsNew.md carries a milestone-close entry framed for the user-as-reader: what changed in the authoring experience, not what changed in the internal representation.
10. The [milestone] Form-A as the default authoring surface entry in docs/roadmap.md is checked off and removed.

Iteration scope

The first iteration covers §A2 (prelude pilot) only:

• Render examples/prelude.ail.json → examples/prelude.ail via ail render.
• Do not delete prelude.ail.json yet (iter 1 deletes it alongside the bulk migration of test infrastructure).
• Verify the existing every_ail_fixture_matches_its_json_counterpart test is green on the new .ail.
• No test-infrastructure changes; no CLAUDE.md / DESIGN.md edits yet.

Iter 1 onward is out of scope for the first plan; the planner re-derives iter scope from this spec each iteration.