fieldtest: canonical-type-names — 5 examples, 9 findings
This commit is contained in:
@@ -0,0 +1,289 @@
|
||||
# Fieldtest — canonical-type-names — 2026-05-11
|
||||
|
||||
**Status:** Draft — awaiting orchestrator triage
|
||||
**Author:** ailang-fieldtester (dispatched by skills/fieldtest)
|
||||
|
||||
## Scope
|
||||
|
||||
The canonical-type-names milestone made cross-module type references
|
||||
load-time validated. Within a `.ail.json` module, bare `Type::Con`
|
||||
names refer to the file's own definitions; cross-module references
|
||||
MUST be qualified `<owning_module>.<TypeName>`; primitives stay bare.
|
||||
Three new workspace-load diagnostics were added: `bare-cross-module-type-ref`,
|
||||
`bad-cross-module-type-ref`, `qualified-class-name`. The prose Form-B
|
||||
printer trims the owning-module qualifier when printing a definition
|
||||
file's own types and keeps cross-module qualifiers verbatim.
|
||||
|
||||
## Examples
|
||||
|
||||
Five fixtures were authored, all `.ailx` first, parsed via
|
||||
`ail parse`, then checked via `ail check`. All live at
|
||||
`examples/` top-level rather than `examples/fieldtest/` because they
|
||||
need cross-module imports (`prelude`, `std_maybe`) which the
|
||||
workspace loader only finds beside the entry module — see Findings
|
||||
for that friction.
|
||||
|
||||
### `examples/ct_1_ordering_signum.ailx` — sign-profile of an Int list
|
||||
|
||||
Reduces a list of Int to a sequence of `-1` / `0` / `+1` by pattern-
|
||||
matching the result of `compare(n, 0)` against `prelude.Ordering`'s
|
||||
three constructors. Drives recursion over a local `IntList` ADT.
|
||||
|
||||
- **Why it fits**: canonical happy-path exercise of cross-module
|
||||
type reference (`prelude.Ordering`) AND local-type reference
|
||||
(`IntList`) in one file.
|
||||
- **Outcome**: parse OK; check OK ("ok (13 symbols across 2 modules)");
|
||||
build OK; runs cleanly; stdout matches expected
|
||||
`-1 / 0 / 1 / -1 / 1`.
|
||||
- **Form-A round-trip**: `render → parse → JSON` is byte-identical.
|
||||
|
||||
### `examples/ct_2_bare_cross_module.ailx` — bare `Maybe` instead of `std_maybe.Maybe`
|
||||
|
||||
A consumer of `std_maybe` writes `(con Maybe (con Int))` in a fn
|
||||
signature instead of `(con std_maybe.Maybe (con Int))`. This is what
|
||||
an LLM author who forgot the qualification rule would naturally
|
||||
produce.
|
||||
|
||||
- **Why it fits**: probes axis 1 (BareCrossModuleTypeRef).
|
||||
- **Outcome**: parse OK; check fails with:
|
||||
> Error: module `ct_2_bare_cross_module` contains bare type name
|
||||
> `Maybe` that does not resolve to a local type. AILang's
|
||||
> `.ail.json` requires cross-module type references to be
|
||||
> qualified. Candidates from imports: `["std_maybe.Maybe"]`. Run
|
||||
> `ail migrate-canonical-types` to fix legacy fixtures.
|
||||
|
||||
Diagnostic code: `bare-cross-module-type-ref`. JSON ctx includes
|
||||
`module`, `name`, `candidates`. Exit code 1.
|
||||
|
||||
### `examples/ct_3_bad_qualified.ailx` — qualified `mystery.Widget` to unknown module
|
||||
|
||||
A consumer references `(con mystery.Widget)` where the module
|
||||
`mystery` is not in the workspace.
|
||||
|
||||
- **Why it fits**: probes axis 2 (BadCrossModuleTypeRef — unknown
|
||||
owner).
|
||||
- **Outcome**: parse OK; check fails with:
|
||||
> Error: module `ct_3_bad_qualified` references qualified type
|
||||
> `mystery.Widget` but the owner module is not known or does not
|
||||
> declare a type by that name
|
||||
|
||||
Diagnostic code: `bad-cross-module-type-ref`. Exit code 1.
|
||||
|
||||
### `examples/ct_3b_bad_qualified_known_module.ailx` — qualified `std_maybe.Widget` to known module
|
||||
|
||||
A consumer references `(con std_maybe.Widget)`. The module `std_maybe`
|
||||
exists and is imported, but it has no `Widget` type def.
|
||||
|
||||
- **Why it fits**: probes axis 2 (BadCrossModuleTypeRef — known
|
||||
owner, unknown type). This is the second branch of the
|
||||
diagnostic's `or` clause.
|
||||
- **Outcome**: parse OK; check fails with the identical-shape
|
||||
diagnostic as ct_3, just with `std_maybe.Widget` substituted.
|
||||
See finding **friction: BadCrossModuleTypeRef does not
|
||||
distinguish unknown-owner from unknown-type-in-known-owner**.
|
||||
|
||||
### `examples/ct_4_qualified_class.ailx` — qualified `prelude.Eq` in an instance
|
||||
|
||||
A consumer defines a local `Box` ADT and writes
|
||||
`(instance (class prelude.Eq) (type (con Box)) ...)` instead of
|
||||
bare `(class Eq)`. Per the milestone's Out-of-scope note, class
|
||||
names remain bare; a qualified form is a schema violation.
|
||||
|
||||
- **Why it fits**: probes axis 3 (QualifiedClassName).
|
||||
- **Outcome**: parse OK; check fails with:
|
||||
> Error: module `ct_4_qualified_class` contains qualified class
|
||||
> name `prelude.Eq` in field `InstanceDef.class`. Class names are
|
||||
> not module-qualified in this milestone; keep the bare form.
|
||||
|
||||
Diagnostic code: `qualified-class-name`. JSON ctx includes
|
||||
`module`, `name`, `field`. Exit code 1.
|
||||
|
||||
## Findings
|
||||
|
||||
### [working] BareCrossModuleTypeRef diagnostic carries the full fix recipe
|
||||
|
||||
Example: `ct_2`. The diagnostic names the module, the offending bare
|
||||
name, the qualified candidate(s) from imports, the migration tool
|
||||
that automates the rewrite. An LLM author can act on this without
|
||||
re-reading the spec. The `ctx` field on the JSON diagnostic is
|
||||
structured (`module`, `name`, `candidates`) — machine-actionable.
|
||||
|
||||
Recommended downstream action: **carry-on**.
|
||||
|
||||
### [working] BadCrossModuleTypeRef catches both Surface and JSON authoring of qualified type refs
|
||||
|
||||
Examples: `ct_3`, `ct_3b`. The Surface syntax `(con mystery.Widget)`
|
||||
DOES parse cleanly, which is the right behavior — Surface should
|
||||
be the canonical authoring path, and authoring a deliberately wrong
|
||||
qualifier should not be rejected at the lexer level. The validator
|
||||
catches it later at workspace load. The diagnostic is clear about
|
||||
what's wrong.
|
||||
|
||||
Recommended downstream action: **carry-on**.
|
||||
|
||||
### [working] QualifiedClassName fires on `(instance (class prelude.Eq) ...)`
|
||||
|
||||
Example: `ct_4`. The Surface form for an instance with a qualified
|
||||
class is parseable (`(class prelude.Eq)` produces JSON
|
||||
`"class": "prelude.Eq"`). The validator rejects it with a clear
|
||||
"keep the bare form" recommendation and names the offending field
|
||||
(`InstanceDef.class`).
|
||||
|
||||
Recommended downstream action: **carry-on**.
|
||||
|
||||
### [working] Form-A round-trip is byte-stable across cross-module refs
|
||||
|
||||
Example: `ct_1`. `ail render → ail parse → JSON` reproduces the
|
||||
input file byte-for-byte. The prose Form-B printer trims the
|
||||
owning-module qualifier for local types correctly (`IntList`
|
||||
renders bare in `ct_1`'s prose, since `IntList` is a local def).
|
||||
Cross-module ctors elide the type-name qualifier in prose
|
||||
(`std_either_demo.prose` shows `Right(42)` not
|
||||
`std_either.Either::Right(42)`), but the canonical JSON underneath
|
||||
keeps the qualified form. This matches the spec's "prose drops
|
||||
qualifiers for surface readability while JSON stays canonical"
|
||||
intent.
|
||||
|
||||
Recommended downstream action: **carry-on**.
|
||||
|
||||
### [friction] BadCrossModuleTypeRef does not distinguish unknown-owner from unknown-type-in-known-owner
|
||||
|
||||
Examples: `ct_3` (unknown owner `mystery`) and `ct_3b` (known owner
|
||||
`std_maybe`, unknown type `Widget`) produce *identical-shape*
|
||||
diagnostics:
|
||||
|
||||
```
|
||||
references qualified type `<owner>.<name>` but the owner module is
|
||||
not known or does not declare a type by that name
|
||||
```
|
||||
|
||||
The two cases suggest different fixes:
|
||||
|
||||
- Unknown owner: add an `(import <owner>)` clause, or fix the
|
||||
typo in the owner prefix.
|
||||
- Known owner, unknown type: fix the typo in the type name, or
|
||||
check what types the owner actually exports.
|
||||
|
||||
The merged diagnostic leaves the LLM author guessing. When the owner
|
||||
IS known, the diagnostic could list available type defs in the
|
||||
owner's module the way `bare-cross-module-type-ref` lists candidates
|
||||
from imports.
|
||||
|
||||
Recommended downstream action: **plan** (tidy iteration to split the
|
||||
two cases and, in the known-owner branch, list available type names
|
||||
as candidates).
|
||||
|
||||
### [friction] Workspace search does not look beyond the entry-module's directory
|
||||
|
||||
Encountered while placing fixtures. The agent spec convention says
|
||||
fieldtest fixtures go under `examples/fieldtest/`. But every fixture
|
||||
that imports `prelude` or `std_maybe` had to be moved to
|
||||
`examples/` top-level because the workspace loader only finds
|
||||
sibling `.ail.json` files in the same directory as the entry module.
|
||||
There is no `--workspace-root` flag on `ail check` / `ail build`,
|
||||
and the diagnostic
|
||||
> module `prelude` not found (expected at <entry-dir>/prelude.ail.json)
|
||||
|
||||
names the precise lookup path but offers no remediation. The result
|
||||
is that subdirectories are effectively forbidden for any consumer
|
||||
of stdlib or prelude — an LLM author hoping to organise example
|
||||
files into subdirectories has to either (a) keep everything flat or
|
||||
(b) duplicate stdlib `.ail.json` files into each subdir.
|
||||
|
||||
This is orthogonal to canonical-type-names per se (it predates the
|
||||
milestone) but it surfaced naturally during the field test and
|
||||
shapes how the fixtures could be placed.
|
||||
|
||||
Recommended downstream action: **plan** (a small iteration to add
|
||||
either a `--workspace-root` flag, or upward-search from the entry
|
||||
module's directory; or **ratify** the current behavior in DESIGN.md
|
||||
if the flat-workspace assumption is intentional).
|
||||
|
||||
### [friction] `ail check` on a `.ailx` source fails with a misleading JSON-parse error
|
||||
|
||||
Encountered immediately on the first invocation: `ail check
|
||||
foo.ailx` produced:
|
||||
> Error: schema/parse error in foo.ailx: json: expected value at
|
||||
> line 1 column 1
|
||||
|
||||
The diagnostic blames `foo.ailx` for not being valid JSON, when the
|
||||
real situation is that `ail check` only accepts `.ail.json`. The
|
||||
LLM author has to learn (from `ail --help`) that `ail parse` does
|
||||
the surface→JSON step, and then chain
|
||||
`ail parse foo.ailx > foo.ail.json && ail check foo.ail.json`.
|
||||
|
||||
The clean fix would be either (a) `ail check` recognises `.ailx`
|
||||
and parses internally, or (b) the diagnostic special-cases the
|
||||
`.ailx` extension and points to `ail parse`. Today, neither holds,
|
||||
and an LLM author's natural first command produces a misleading
|
||||
diagnostic.
|
||||
|
||||
This is orthogonal to canonical-type-names per se. Orthogonal but
|
||||
load-bearing — every fieldtest invocation hits this.
|
||||
|
||||
Recommended downstream action: **plan** (a small iteration to teach
|
||||
`ail check` and `ail build` to accept `.ailx`, or to detect the
|
||||
extension and emit a "did you mean `ail parse foo.ailx`?" hint).
|
||||
|
||||
### [friction] `(import prelude)` is rejected as reserved
|
||||
|
||||
Encountered while writing `ct_1`. The LLM-natural reach when writing
|
||||
a cross-module program that consumes prelude is to write
|
||||
`(import prelude)` in the module header. This is rejected:
|
||||
> Error: module name `prelude` is reserved (auto-injected by the
|
||||
> loader)
|
||||
|
||||
The diagnostic is clear and short. But the fact that prelude is
|
||||
auto-injected and must NOT be explicitly imported is not documented
|
||||
in any `.ailx` corpus example I can read (no example uses
|
||||
`(import prelude)`, but also no comment explains why). DESIGN.md
|
||||
mentions prelude as the autodiscovered module but does not say
|
||||
"do not write `(import prelude)`". An author has to discover this
|
||||
by trying and being told.
|
||||
|
||||
This is mild — it's a one-trip teaching diagnostic, and the
|
||||
message is plain. But worth recording.
|
||||
|
||||
Recommended downstream action: **carry-on** (the diagnostic itself
|
||||
is clear; the deeper fix is a DESIGN.md or per-module-comment hint,
|
||||
not a code change).
|
||||
|
||||
### [spec_gap] No `.ailx` counterpart for `compare_primitives_smoke.ail.json`
|
||||
|
||||
Observation: the canonical happy-path example shipped with this
|
||||
milestone (`examples/compare_primitives_smoke.ail.json`) exists
|
||||
only as JSON. There is no `.ailx` counterpart. Surface IS the
|
||||
LLM-author surface per Decision 6, so a milestone whose "happy
|
||||
path" demonstrates cross-module type usage SHOULD have a `.ailx`
|
||||
exhibit — otherwise the LLM author has to read raw JSON to
|
||||
understand the pattern. My `ct_1_ordering_signum.ailx` partly fills
|
||||
this gap (it exercises prelude.Ordering pattern-matching from a
|
||||
Surface program) but it is not bit-identical to
|
||||
`compare_primitives_smoke`.
|
||||
|
||||
Recommended downstream action: **plan** (tidy iteration: author
|
||||
`examples/compare_primitives_smoke.ailx` to round-trip into the
|
||||
existing JSON, OR retire `compare_primitives_smoke.ail.json` in
|
||||
favour of `ct_1_ordering_signum.{ailx,ail.json}` as the new
|
||||
canonical happy-path exhibit).
|
||||
|
||||
## Recommendation summary
|
||||
|
||||
| Finding | Class | Action |
|
||||
|---|---|---|
|
||||
| BareCrossModuleTypeRef carries full fix recipe | working | carry-on |
|
||||
| BadCrossModuleTypeRef catches Surface-authored qualified refs | working | carry-on |
|
||||
| QualifiedClassName fires on `(class prelude.Eq)` | working | carry-on |
|
||||
| Form-A round-trip byte-stable | working | carry-on |
|
||||
| BadCrossModuleTypeRef merges two distinguishable cases | friction | plan |
|
||||
| Workspace search confined to entry dir | friction | plan or ratify |
|
||||
| `ail check` on `.ailx` produces misleading JSON-parse error | friction | plan |
|
||||
| `(import prelude)` rejected — diagnostic clear, doc-gap mild | friction | carry-on |
|
||||
| No `.ailx` counterpart for `compare_primitives_smoke.ail.json` | spec_gap | plan |
|
||||
|
||||
The four milestone-scoped findings are all **working**. The three
|
||||
**friction** items orthogonal to canonical-type-names are
|
||||
pre-existing UX gaps surfaced by the field test. The two
|
||||
canonical-type-names-internal friction items (BadCrossModuleTypeRef
|
||||
case-merging; missing `.ailx` exhibit) are recommendable tidy
|
||||
iterations.
|
||||
Reference in New Issue
Block a user