AILang/docs/specs/2026-05-13-form-a-default-authoring.md

# Form-A as the Default Authoring Surface — Design Spec

**Date:** 2026-05-13
**Status:** Draft — awaiting user spec review
**Authors:** Brummel (orchestrator) + Claude

## Goal

Make Form A (`.ail`) the sole authoring surface for every program in
the working tree. After this milestone, the only `.ail.json` files
that remain in the repository are eight specific negative-test
fixtures whose subject matter cannot be expressed in Form A by
construction. Every other `.ail.json` is rendered to its `.ail`
sibling via `ail render`, and the now-redundant `.ail.json` is
**deleted** — not retained as a generated artefact, not stashed in a
build directory, not regenerated at test time. The working tree
contains exactly one representation per program: the `.ail` source.

The language doctrine does not change. The canonical, hashable,
content-addressed form remains the JSON-AST (Decision 6,
architectural pin, §"data structure is the source of truth"). What
changes is the **authoring doctrine**: authors write `.ail`; the
build derives the JSON-AST in-process when a consumer needs it,
gated by the round-trip invariant.

Driving evidence: the cross-model authoring-form study (audit-cma,
audit-ms, 2026-05-12) measured that Form A is cheaper in tokens
(~98k vs ~207k for the same four tasks) and produces strictly
more first-shot green programs than JSON-AST authoring, across two
distinct foreign LLMs (Qwen3-Coder-Next and CodeLlama-13b-Instruct).
The internal authoring practice is now drift from the empirical
"this is the form LLMs reach for unprompted" finding.

## Architecture

Four architectural decisions land in this spec; each is the
substantive answer to one of the open questions in the roadmap
entry.

### A1. In-process parse, no build-time artefacts

Tests and benches consume `.ail` fixtures by calling
`ailang_surface::parse` (library, in-process) — not by spawning
`ail parse` as a build prelude that emits `.ail.json` to a target
directory.

**Why.** A build-prelude approach reintroduces a second
representation: `examples/foo.ail` (source) plus
`target/parsed/foo.ail.json` (derived). Even with the derived form
gitignored, it would be a working-tree fact during tests and a
shadow corpus for downstream tools to drift against. The single-
representation discipline that motivates this milestone is broken
the moment a derived `.ail.json` exists on disk anywhere a tool
might read it.

The in-process path is also already plumbed: since iter ext-cli.1
(2026-05-12), every path-taking CLI subcommand accepts `.ail` and
parses through `ailang_surface::parse` in-line. The Python bench
drivers (`bench/check.py`, `bench/compile_check.py`,
`bench/cross_lang.py`, `bench/mono_dispatch.py`) call `ail check`
etc. via subprocess and inherit the `.ail` acceptance for free —
they need only a path-suffix change. The integration tests under
`crates/ail/tests/*.rs` and the workspace tests under
`crates/ailang-check/tests/*.rs` currently `fs::read_to_string` a
`.ail.json` and pass the bytes to a loader; they will switch to
reading the `.ail` and calling `ailang_surface::parse` to recover
the same `Module` value.

### A2. `prelude.ail` ships as iter-0 pilot

Iteration 0 converts exactly one file: `examples/prelude.ail.json`
→ `examples/prelude.ail`. Nothing else moves. The roundtrip-CI gate
covers it (every `.ail` with a same-stem `.ail.json` counterpart is
already byte-compared by `every_ail_fixture_matches_its_json_counterpart`),
which proves the conversion path end-to-end before the bulk
migration commits to it.

**Why.** Prelude exercises the widest cross-section of language
features in a single file: typeclasses with superclasses, polymorphic
free functions with constraints, IO-effect signatures, primitive
ctor patterns, and the loader's auto-injection path. If prelude
round-trips identically through `ail render → ail parse → canonical
bytes`, no other fixture in the corpus introduces a feature this
file does not. The pilot validates the migration mechanism on the
hardest single file before the bulk migration commits the test-
infrastructure refactor.

Iter 0 keeps `prelude.ail.json` on disk alongside `prelude.ail` —
the deletion of redundant JSON-AST starts in iter 1 once the
test-infrastructure refactor lands. This is the *only* iteration
where a fixture is dual-form; it is justified by the existing
`every_ail_fixture_matches_its_json_counterpart` gate, which is
exactly the byte-comparison this iteration wants to leverage as a
witness.

### A3. Per-file deletion cadence with iteration-end checkpoints

`.ail.json` files are deleted on the same iteration that materialises
their `.ail` counterpart, not at milestone end. The bulk migration is
sequenced across ~3 iterations of ~30-40 files each, each iteration
fully reaping its own `.ail.json` files before close. There is no
intermediate state where the bulk of fixtures are dual-form.

The exception is iter 0 (the prelude pilot, A2 above), which is
the singular dual-form window: the JSON counterpart is the witness
the existing CI gate uses, and the gate is the safety net for the
pilot itself.

**Why.** A milestone-end bulk delete would mean the working tree
spends weeks in a "which form is the source?" ambiguity. Anyone
editing a fixture in the interim — to repair an unrelated bug,
add a regression test, fix a hash drift — has no canonical guide
to which form to edit. The single-representation property must hold
on a per-commit basis, not as a milestone-end snapshot.

The trade-off is iteration sequencing pressure: every iteration that
ships in this milestone must leave the tree in a coherent
single-representation state. This is enforced by the per-iteration
roundtrip-CI rewrite (see Testing strategy below): once the
infrastructure flips in iter 1, fixtures without `.ail.json` are
the new normal, and fixtures with `.ail.json` are visibly in-flight.

### A4. CLAUDE.md and DESIGN.md rewording

Two documents need substantive rewording in this milestone. Both
edits land in the same commit as the iteration that retires the
last non-carve-out `.ail.json`.

**CLAUDE.md** §"AILang — a language for LLM authors" opening:

Currently:
> *Machine readability over human readability. Source of truth is
> structured data (`.ail.json`), not text.*

After this milestone:
> *Machine readability over human readability. The canonical,
> hashable, content-addressed form is structured data (`.ail.json`);
> the authoring projection is Form A (`.ail`). Authors write `.ail`;
> the build derives the JSON-AST in-process via
> `ailang_surface::parse`, gated by the round-trip invariant. The
> two forms are byte-isomorphic — picking either does not change
> the identity of the module.*

**DESIGN.md** §"What this Decision deliberately does not do" (line
465-466), currently:
> *It does not promote form (A) to the source of truth. Form (A)
> is one projection; the JSON-AST stays canonical.*

The first sentence is replaced; the second stays. New text:
> *It does not change which form is canonical: the JSON-AST remains
> the hashable, content-addressed representation, and all hashing,
> content-addressing, and cross-module references flow through it
> unchanged. What did change in 2026-05-13 (this milestone) is
> which form is authored: authors now write Form A; the JSON-AST
> is materialised in-process by callers that need it. The two
> forms are byte-isomorphic by the round-trip invariant.*

The "Form (A) is one projection; the JSON-AST stays canonical"
phrasing is preserved verbatim in the second sentence because it
remains true: Form A and the eventual visual / graphical
projections are still parallel projections of the same AST. What
the milestone changes is which projection is the **entry point**,
not the projection's relation to the AST.

## Components

### C1. Migration script

A one-shot Bash script `scripts/migrate-to-form-a.sh` (new file)
walks `examples/*.ail.json`, skips the eight carve-outs (named
explicitly in the script body), invokes `ail render` to produce
the sibling `.ail`, and `rm`s the JSON counterpart. The script is
idempotent (re-running on a partially migrated tree is a no-op for
already-migrated files). The script is retired at milestone close
— it has no second use; keeping it would invite drift the next
time someone wonders "where did the JSON-AST come from".

### C2. Test infrastructure refactor

48 files under `crates/` and `bench/` reference `.ail.json` paths.
The refactor is mechanical but spans three classes of consumer:

1. **Roundtrip tests** (`crates/ailang-surface/tests/round_trip.rs`,
   `crates/ail/tests/roundtrip_cli.rs`). The five-test invariant
   restructures (see Testing strategy §T below).
2. **Fixture-loading integration tests** (`crates/ail/tests/*.rs`,
   ~35 files). Each test currently reads
   `examples/<stem>.ail.json` with `fs::read_to_string` and
   passes the bytes to `Module::from_json` or the workspace
   loader. The refactor reads `examples/<stem>.ail` and parses via
   `ailang_surface::parse_module_from_str`. The eight carve-outs
   continue to use the `.ail.json` path (they have no `.ail` form
   by construction).
3. **Bench drivers** (`bench/*.py`, `bench/run.sh`). The drivers
   spawn `ail check` etc. with fixture paths. Since ext-cli.1
   the subcommands accept both extensions; the drivers' fixture
   walks change from `glob("examples/*.ail.json")` to
   `glob("examples/*.ail")` plus an explicit allow-list for the
   carve-outs.

### C3. Roundtrip invariant restatement

§"Roundtrip Invariant" at DESIGN.md:2027 currently anchors on a
two-way `.ail.json` ↔ `.ail` bijection. After this milestone the
JSON-AST counterpart no longer exists for the bulk of the corpus,
so the invariant restates as:

- **Parse-determinism.** For every well-formed `.ail` text `t`,
  `parse(t)` produces a unique AST (the loader is a pure function;
  no randomness, no time dependence, no environment leak).
- **Idempotency under print.**
  `canonical(parse(t)) == canonical(parse(print(parse(t))))` —
  printing and re-parsing is a no-op on the canonical bytes.
- **CLI-pipeline idempotency.** The public CLI pipeline `ail render
  | ail parse` (one round) is byte-identical to direct
  `ail parse` of the source `.ail`. Pins the user-facing CLI
  against drift internal tests cannot see.
- **Carve-out anchor.** The eight `.ail.json`-only fixtures are
  negative-tests (load-time / typecheck-time rejection); they
  participate only in the existing rejection-shape tests, not in
  the roundtrip gate. Their presence is the *only* surviving
  reason `.ail.json` remains a first-class file extension.

The "JSON → text → JSON" direction (Direction 1 in the current
spec) loses its corpus once the JSON files are deleted. The
property is replaced by parse-determinism plus idempotency — the
two together discharge the load-bearing concern (hashing is well-
defined for any `.ail` source).

### C4. The carve-outs (immutable list)

Two structural reasons keep `.ail.json` alive post-milestone:
(a) **subject-matter** carve-outs (seven files; the canonical
list as drafted) and (b) **compile-time-embed** carve-outs (one
file as of 2026-05-13; subject to a separate milestone). Both
gain no `.ail` sibling.

**(a) Subject-matter carve-outs (immutable, seven files):**

```
examples/test_ct1_bare_xmod_rejected.ail.json
examples/test_ct1_qualified_class_rejected.ail.json
examples/test_ct1_bad_qualifier.ail.json
examples/broken_unbound.ail.json
examples/test_22b2_invalid_superclass_param.ail.json
examples/test_22b2_kind_mismatch.ail.json
examples/test_22b2_unbound_constraint_var.ail.json
```

Justification per file:
- `test_ct1_*` (3): assert canonical-form rejection of malformed
  type references. The Form-A parser would either accept the
  reference (test loses its subject) or reject it at parse time
  with a different diagnostic (test loses its meaning). The
  fixture's purpose is to test the **loader/checker** rejection
  shape, which requires the input to bypass the parser.
- `broken_unbound`: asserts unbound-variable rejection at typecheck.
  The `.ail.json` form encodes a structurally well-formed but
  semantically broken module that the Form-A parser would also
  parse cleanly; the file is retained in JSON-form not because the
  surface cannot express it but because the test already pins the
  exact JSON bytes, and re-deriving from `.ail` would add no value
  while adding a regeneration step.
- `test_22b2_*` (3): assert class-definition rejection shapes
  (invalid superclass param, kind mismatch, unbound constraint var)
  that the Form-A parser would either reject at parse time
  (changing the diagnostic) or refuse to encode at all (changing
  the test subject).

**(b) Compile-time-embed carve-outs (one file as of 2026-05-13):**

```
examples/prelude.ail.json
```

Justification:
- `prelude.ail.json` is `include_str!`-embedded at
  `crates/ailang-core/src/workspace.rs:417` and
  `crates/ail/src/main.rs:474`, then parsed via
  `serde_json::from_str` at compile-time-fixed sites in production
  code. Switching the embed to `prelude.ail` would require parsing
  via `ailang_surface::parse`, but `ailang-core` cannot import
  `ailang-surface` (crate-cycle constraint documented at
  `crates/ailang-surface/src/loader.rs:9-13`). The mechanism is a
  structurally distinct from the runtime-loaded-program corpus the
  rest of this milestone migrates: `.ail.json` carve-outs in
  category (a) are kept because their *content* cannot be expressed
  in Form A; `prelude.ail.json` is kept because its *embed mechanism*
  requires the JSON-AST form.

  This is a real spec gap relative to §A4's authoring-doctrine
  ("authors write `.ail`; the build derives the JSON-AST in-process
  via `ailang_surface::parse`"). The prelude is currently authored
  as `.ail.json` and embedded as bytes; the doctrine says it should
  be authored as `.ail` and parsed at compile time. Resolving the
  inconsistency requires either (i) splitting the workspace loader
  so prelude-init lives in `ailang-surface`, (ii) generating
  `prelude.ail.json` at build time from `prelude.ail` via a
  `build.rs`, or (iii) some other refactor — all of which exceed
  the scope of this milestone.

  The resolution lives in a separate roadmap milestone:
  **"Prelude embed: Form-A as compile-time source"** (see
  `docs/roadmap.md`). After that milestone closes, this carve-out
  category becomes empty and the spec text drops it.

  Note: `examples/prelude.ail` was rendered in iter form-a.0 and
  remains the authoring projection (it stays in the tree; both
  forms coexist for prelude specifically, mirroring the iter-0
  pilot's dual-form witness arrangement, until the
  prelude-embed-refactor milestone resolves the inconsistency).

**Future additions to either list require brainstorm-level
justification on the same structural grounds** — not "this test
is annoying to migrate" or "the diff is large", but a named
mechanism (canonical-form rejection / compile-time embed / etc.)
that the surface cannot satisfy.

## Data flow

Before this milestone:
```
author writes:          examples/foo.ail.json
                              |
                              v
                    [loader / serde_json]
                              |
                              v
                   Module value -> typecheck, codegen, ...
```

After iter 0 (prelude pilot only):
```
author writes:          examples/prelude.ail            (new)
                              |
                              v
                  [ailang_surface::parse]
                              |
                              v
                   Module value -> typecheck, codegen, ...

                                                  [byte-witness]
                        examples/prelude.ail.json  ----+   (retained
                              |                         |    for iter 0,
                              v                         |    deleted iter 1)
                    [loader / serde_json]               |
                              |                         |
                              v                         |
                   Module value -----> canonical -------+
```

After milestone close (bulk migration done, JSON counterparts
deleted):
```
author writes:          examples/<name>.ail             (every program)
                              |
                              v
                  [ailang_surface::parse]
                              |
                              v
                   Module value -> typecheck, codegen, ...

except for the eight carve-outs:
                        examples/<carve-out>.ail.json
                              |
                              v
                    [loader / serde_json]   (negative-test only)
```

Cross-module imports continue to use module names, not file paths;
the workspace loader walks `examples/*` and accepts either extension
for any module it loads. Since the bulk of the corpus is single-
extension post-migration, the loader's extension-dispatch is exercised
only by the eight carve-outs (and by any test that explicitly mixes
the two forms for migration testing).

## Error handling

The migration is mechanical and the round-trip invariant
discharges correctness. There are three failure modes the spec
must name explicitly because they could escape the existing CI net:

**E1. A fixture round-trips before migration but the migrated `.ail`
fails to parse.** This would be a bug in `ail render` or
`ail parse`; the round-trip CI gate
(`every_ail_fixture_matches_its_json_counterpart`) already covers
it for every fixture with a hand-written `.ail` sibling. The
migration script extends the same property to every fixture by
construction (the script's output is exactly the witness the gate
demands). If iter 0 (prelude pilot) fails this gate, the bug is
in render/parse; the milestone halts and the bug fix takes
precedence.

**E2. A test that depends on the exact byte shape of `.ail.json`
breaks after migration.** Some tests (e.g. snapshot tests) compare
byte-equal canonical JSON to a checked-in expected blob. These
tests do not break — the canonical bytes are derived from the AST,
and the AST is form-agnostic. The migration touches only the
fixture-loading path, not the canonical-encoding path.

**E3. A bench driver receives a `.ail` path and emits stale
`.ail.json` paths in its report.** The bench drivers' report format
echoes the fixture name; the suffix change from `.ail.json` to
`.ail` propagates into report bytes. The audit-* JSONs under
`bench/orchestrator-stats/` will start carrying `.ail` paths after
the next milestone-close audit; this is intended and not a
regression.

A failure mode the spec explicitly does *not* worry about: parse
non-determinism. The roundtrip-CI gate has been green across 9
consecutive audits (audit-cma onward); the parser is well-
exercised.

## Testing strategy

### T1. Roundtrip-CI test restatement

`crates/ailang-surface/tests/round_trip.rs` currently runs three
tests against the `.ail.json` corpus. Post-migration, the three
restate:

| Pre-migration test | Post-migration test | Behaviour |
|--------------------|---------------------|-----------|
| `print_then_parse_round_trips_every_fixture` (over `.ail.json`) | replaced by `parse_is_deterministic_over_every_ail_fixture` (over `.ail`) | Reads every `.ail`, parses twice, asserts canonical-byte equality between the two parses. Asserts pure-function-of-input. |
| `parse_then_print_then_parse_is_idempotent_on_every_ail_fixture` | unchanged | Reads every `.ail`, asserts `parse == parse ∘ print ∘ parse`. |
| `every_ail_fixture_matches_its_json_counterpart` | retired | Loses its corpus. The property it pinned (no drift between two hand-authored forms) is no longer expressible because only one form is hand-authored. |

`crates/ail/tests/roundtrip_cli.rs::cli_render_then_parse_preserves_canonical_bytes_on_every_fixture`
restates as `cli_parse_then_render_then_parse_is_idempotent`:
reads every `.ail`, runs `ail parse` (to canonical JSON tempfile),
runs `ail render` (back to `.ail` tempfile), runs `ail parse`
again, and asserts byte equality across the two parses.

`crates/ailang-core/tests/schema_coverage.rs::every_ast_variant_is_observed_in_the_fixture_corpus`
flips its corpus walk from `.ail.json` to `.ail` (load via
`ailang_surface::parse_module_from_str`); the assertion is
unchanged. The schema-coverage property is the strictest gate
that survives: a new AST variant cannot land without a fixture
that uses it.

### T2. Per-iteration regression discipline

Every iteration in this milestone must leave `cargo test
--workspace` green and `bench/compile_check.py` exit-0. The
migration is mechanical; non-green at iteration close is a defect
in the iteration's test-infrastructure refactor, not in the
language. The iteration does not close until the refactor is
green.

### T3. Test-naming discipline for the migration

A few existing tests carry `.ail.json` in their test-function names
(e.g. `mono_recursive_fn_loads_from_json_ast`). These names are
renamed in the same iteration that migrates the corresponding
fixture: the new name should not lie about what file it reads.
This is mechanical, but it has to land in the migration commit, not
as a follow-up rename.

### T4. Carve-out coverage check

A new test `crates/ailang-core/tests/carve_out_inventory.rs`
asserts that exactly the eight named carve-out files exist under
`examples/*.ail.json` after milestone close (seven from §C4 (a) +
one from §C4 (b)). The test fails if a new `.ail.json` is added
or a carve-out goes missing. The list is hardcoded in the test
body — this is intentional: the carve-out list is part of the
language's negative-test surface and any change to it is a
deliberate, brainstorm-level decision. The test's pass-count
drops to seven once the prelude-embed-refactor milestone retires
`prelude.ail.json` from category (b); the test body is then
re-published with the smaller list in that milestone's iter.

## Acceptance criteria

The milestone closes when all of the following are true:

1. `ls examples/*.ail.json | wc -l` returns `8`, and the eight
   files match the carve-out list verbatim (§C4 category (a) +
   category (b)).
2. Every program in `examples/` other than the eight carve-outs
   exists as a `.ail` file with no `.ail.json` sibling.
   `examples/prelude.ail` exists in addition to
   `examples/prelude.ail.json` per §C4 (b) (dual-form for prelude
   only, pending the prelude-embed-refactor milestone).
3. The roundtrip-CI restatement (T1) is green: parse is
   deterministic, idempotent under print, CLI-pipeline idempotent,
   and the carve-out inventory test (T4) passes.
4. `cargo test --workspace` is green at milestone close (no
   regressions introduced by the test-infrastructure refactor).
5. `bench/compile_check.py` and `bench/cross_lang.py` exit 0
   against the migrated corpus.
6. `bench/check.py` is either exit 0 or exit 1 with audit-
   ratifiable noise only (consistent with the 9-audit lineage
   since audit-cma); a new regression class attributable to the
   migration triggers brainstorm-level escalation.
7. CLAUDE.md and DESIGN.md carry the reworded paragraphs (§A4
   above) verbatim.
8. The migration script `scripts/migrate-to-form-a.sh` is deleted
   from the working tree (one-shot, retired).
9. `docs/WhatsNew.md` carries a milestone-close entry framed for
   the user-as-reader: what changed in the authoring experience,
   not what changed in the internal representation.
10. The `[milestone] Form-A as the default authoring surface`
    entry in `docs/roadmap.md` is checked off and removed.

## Iteration scope

The first iteration covers **§A2 (prelude pilot)** only:
- Render `examples/prelude.ail.json` → `examples/prelude.ail` via
  `ail render`.
- Do not delete `prelude.ail.json` yet (iter 1 deletes it
  alongside the bulk migration of test infrastructure).
- Verify the existing
  `every_ail_fixture_matches_its_json_counterpart` test is green
  on the new `.ail`.
- No test-infrastructure changes; no CLAUDE.md / DESIGN.md edits
  yet.

Iter 1 onward is out of scope for the first plan; the planner
re-derives iter scope from this spec each iteration.