Files
AILang/crates/ailang-core/tests/spec_drift.rs
T
Brummel 72626fa94f ail: embed Form-A spec in merge-prose prompt (iter 20f)
Closes a design hole shipped in 20d: the merge-prose prompt instructed
the LLM to emit JSON-AST and gave a 12-line schema-essentials reminder.
JSON-AST is the canonical hashable artefact, not a writing surface; the
reminder was not a language spec. Foreign LLMs had no realistic shot.

20f makes two coupled changes:

1) The LLM now emits Form-A (the canonical authoring surface fixed by
   Decision 6). merge-prose loads the original via ailang_core::load_module
   and re-renders via ailang_surface::print before embedding (round-trip
   is a gating contract on the surface crate, so this is lossless). The
   user runs `ail parse foo.new.ailx` to recover JSON, then `ail check`.

2) crates/ailang-core/specs/form_a.md is the complete LLM-targeted
   Form-A specification — grammar, every term/pattern/type/def keyword,
   schema invariants, pitfall catalogue, four few-shot modules from
   examples/*.ailx. Exported as ailang_core::FORM_A_SPEC via include_str!
   and embedded verbatim in every merge-prose prompt.

Drift detection in crates/ailang-core/tests/spec_drift.rs: every variant
of Term, Pattern, Type, Def, Literal is reached via exhaustive `match`.
The arms are not the assertion — adding a new variant without updating
the match is a compile error before the test runs. Once matched, an
anchor string is asserted to appear in FORM_A_SPEC. 8 tests, all green.

The hand-written + mechanical-drift-test combo addresses the user's
"distance to code is too big" concern about a docs-only spec. Generator
overkill rejected: structural drift is mechanically caught, but prose
explanation, schema-invariant catalogue, pitfall list, and few-shot
corpus cannot be emitted from AST shape alone.

Tests:
  - ailang-core: +8 spec_drift tests; existing 12 unchanged
  - ail unit (3): rewritten in lockstep — assert (own T)/(effects IO)
    landmarks, FORM-A SPECIFICATION header, FORM_A_SPEC body verbatim
  - ail e2e merge_prose_prints_framed_prompt: rewritten — assert
    `(module foo` + `FORM-A SPECIFICATION` instead of `ailang/v0`
  - Workspace: all green

Richer integration paths (LLM tool-use, MCP server, LSP) were named
in the design discussion and deferred per "kiss". All three layer
additively on the static-prompt path; static prompt remains the
lowest-common-denominator fallback.
2026-05-08 23:31:27 +02:00

317 lines
10 KiB
Rust

//! Iter 20f: drift detection between the AST and `specs/form_a.md`.
//!
//! The spec is hand-curated, but it cannot silently fall behind the
//! language. Every AST enum (`Term`, `Pattern`, `Type`, `Def`, `Literal`,
//! `ParamMode`) discriminates on a `#[serde(rename = "...")]` tag.
//! These tests construct a sample of every variant, then check that the
//! corresponding tag string (or its parenthesised Form-A keyword) appears
//! in the spec.
//!
//! The exhaustive `match` is the load-bearing piece: adding a new variant
//! without a spec entry fails compilation here long before the test runs.
//! Once the variant is matched, the test asserts the spec mentions it.
use ailang_core::ast::{
ConstDef, Ctor, Def, FnDef, Literal, Pattern, Suppress, Term, Type, TypeDef,
};
use ailang_core::FORM_A_SPEC;
/// Every `Term` variant must be reachable from the spec. The Form-A
/// keyword for each variant is what the spec is supposed to teach an
/// LLM; if it is missing here, the LLM cannot produce that term.
#[test]
fn spec_mentions_every_term_variant() {
let exemplars: Vec<(&str, Term)> = vec![
("(lit-unit", Term::Lit { lit: Literal::Unit }),
// The Var form has no parenthesised keyword (a bare ident is a
// var-ref). The spec calls it out under "Atom forms"; we look for
// that anchor.
("Atom forms", Term::Var { name: "x".into() }),
(
"(app",
Term::App {
callee: Box::new(Term::Var { name: "f".into() }),
args: vec![Term::Var { name: "x".into() }],
tail: false,
},
),
(
"(let ",
Term::Let {
name: "x".into(),
value: Box::new(Term::Lit { lit: Literal::Int { value: 1 } }),
body: Box::new(Term::Var { name: "x".into() }),
},
),
(
"(let-rec",
Term::LetRec {
name: "f".into(),
ty: Type::fn_implicit(vec![], Type::int(), vec![]),
params: vec![],
body: Box::new(Term::Lit { lit: Literal::Int { value: 0 } }),
in_term: Box::new(Term::Var { name: "f".into() }),
},
),
(
"(if",
Term::If {
cond: Box::new(Term::Lit { lit: Literal::Bool { value: true } }),
then: Box::new(Term::Lit { lit: Literal::Int { value: 1 } }),
else_: Box::new(Term::Lit { lit: Literal::Int { value: 0 } }),
},
),
(
"(do ",
Term::Do {
op: "io/print_int".into(),
args: vec![],
tail: false,
},
),
(
"(term-ctor",
Term::Ctor {
type_name: "List".into(),
ctor: "Nil".into(),
args: vec![],
},
),
(
"(match",
Term::Match {
scrutinee: Box::new(Term::Var { name: "x".into() }),
arms: vec![],
},
),
(
"(lam",
Term::Lam {
params: vec![],
param_tys: vec![],
ret_ty: Box::new(Type::int()),
effects: vec![],
body: Box::new(Term::Lit { lit: Literal::Int { value: 0 } }),
},
),
(
"(seq",
Term::Seq {
lhs: Box::new(Term::Var { name: "a".into() }),
rhs: Box::new(Term::Var { name: "b".into() }),
},
),
(
"(clone",
Term::Clone {
value: Box::new(Term::Var { name: "x".into() }),
},
),
(
"(reuse-as",
Term::ReuseAs {
source: Box::new(Term::Var { name: "x".into() }),
body: Box::new(Term::Var { name: "y".into() }),
},
),
];
for (anchor, term) in exemplars {
// Force the exhaustive match: the body is just a tag string that
// we will not actually use, but the compiler will refuse to
// compile this file once a new Term variant is added without a
// matching arm.
let _: &'static str = match term {
Term::Lit { .. } => "lit",
Term::Var { .. } => "var",
Term::App { .. } => "app",
Term::Let { .. } => "let",
Term::LetRec { .. } => "letrec",
Term::If { .. } => "if",
Term::Do { .. } => "do",
Term::Ctor { .. } => "ctor",
Term::Match { .. } => "match",
Term::Lam { .. } => "lam",
Term::Seq { .. } => "seq",
Term::Clone { .. } => "clone",
Term::ReuseAs { .. } => "reuse-as",
};
assert!(
FORM_A_SPEC.contains(anchor),
"spec is missing anchor `{anchor}` for a Term variant — \
update crates/ailang-core/specs/form_a.md"
);
}
}
/// Every `Pattern` variant must appear in the spec.
#[test]
fn spec_mentions_every_pattern_variant() {
let exemplars: Vec<(&str, Pattern)> = vec![
("_", Pattern::Wild),
// pat-var is again the bare-ident form. The spec discusses it
// under "Patterns". Use the heading as the anchor.
("## Patterns", Pattern::Var { name: "x".into() }),
("(pat-lit", Pattern::Lit { lit: Literal::Int { value: 0 } }),
(
"(pat-ctor",
Pattern::Ctor { ctor: "Nil".into(), fields: vec![] },
),
];
for (anchor, pat) in exemplars {
let _: &'static str = match pat {
Pattern::Wild => "wild",
Pattern::Var { .. } => "var",
Pattern::Lit { .. } => "lit",
Pattern::Ctor { .. } => "ctor",
};
assert!(
FORM_A_SPEC.contains(anchor),
"spec is missing anchor `{anchor}` for a Pattern variant"
);
}
}
/// Every `Type` variant must appear in the spec.
#[test]
fn spec_mentions_every_type_variant() {
let exemplars: Vec<(&str, Type)> = vec![
("(con ", Type::int()),
(
"(fn-type",
Type::fn_implicit(vec![], Type::unit(), vec![]),
),
(
"TYVAR-NAME",
Type::Var { name: "a".into() },
),
(
"(forall",
Type::Forall {
vars: vec!["a".into()],
body: Box::new(Type::Var { name: "a".into() }),
},
),
];
for (anchor, ty) in exemplars {
let _: &'static str = match ty {
Type::Con { .. } => "con",
Type::Fn { .. } => "fn",
Type::Var { .. } => "var",
Type::Forall { .. } => "forall",
};
assert!(
FORM_A_SPEC.contains(anchor),
"spec is missing anchor `{anchor}` for a Type variant"
);
}
}
/// Every `Literal` variant must appear in the spec.
#[test]
fn spec_mentions_every_literal_variant() {
// Anchors describe how the literal renders in Form-A.
let exemplars: Vec<(&str, Literal)> = vec![
("`INT`", Literal::Int { value: 0 }),
("`true`, `false`", Literal::Bool { value: true }),
("`STRING`", Literal::Str { value: "x".into() }),
("(lit-unit)", Literal::Unit),
];
for (anchor, lit) in exemplars {
let _: &'static str = match lit {
Literal::Int { .. } => "int",
Literal::Bool { .. } => "bool",
Literal::Str { .. } => "str",
Literal::Unit => "unit",
};
assert!(
FORM_A_SPEC.contains(anchor),
"spec is missing anchor `{anchor}` for a Literal variant"
);
}
}
/// Every `Def` kind must appear in the spec.
#[test]
fn spec_mentions_every_def_kind() {
let fn_def = FnDef {
name: "f".into(),
doc: None,
suppress: vec![],
ty: Type::fn_implicit(vec![], Type::int(), vec![]),
params: vec![],
body: Term::Lit { lit: Literal::Int { value: 0 } },
};
let const_def = ConstDef {
name: "k".into(),
doc: None,
ty: Type::int(),
value: Term::Lit { lit: Literal::Int { value: 0 } },
};
let type_def = TypeDef {
name: "T".into(),
doc: None,
vars: vec![],
ctors: vec![Ctor {
name: "C".into(),
fields: vec![],
}],
drop_iterative: false,
};
let exemplars: Vec<(&str, Def)> = vec![
("(fn ", Def::Fn(fn_def)),
("(const ", Def::Const(const_def)),
("(data ", Def::Type(type_def)),
];
for (anchor, def) in exemplars {
let _: &'static str = match def {
Def::Fn(_) => "fn",
Def::Const(_) => "const",
Def::Type(_) => "type",
};
assert!(
FORM_A_SPEC.contains(anchor),
"spec is missing anchor `{anchor}` for a Def kind"
);
}
}
/// The mode keywords (Decision 10) must appear so an LLM knows the
/// Form-A wrapper syntax.
#[test]
fn spec_mentions_mode_keywords() {
assert!(FORM_A_SPEC.contains("(own"), "spec missing `(own ...)`");
assert!(FORM_A_SPEC.contains("(borrow"), "spec missing `(borrow ...)`");
}
/// `tail-app` and `tail-do` are distinct keywords from `app`/`do`. The
/// spec must mention both, otherwise an LLM cannot produce tail-correct
/// code at scale.
#[test]
fn spec_mentions_tail_variants() {
assert!(FORM_A_SPEC.contains("tail-app"), "spec missing `tail-app`");
assert!(FORM_A_SPEC.contains("tail-do"), "spec missing `tail-do`");
}
/// `suppress` is part of the surface and the LLM must know how to
/// preserve it. Empty-because is itself a diagnostic; the spec calls
/// it out so the LLM does not produce empty justifications.
#[test]
fn spec_mentions_suppress_clause() {
assert!(FORM_A_SPEC.contains("(suppress"), "spec missing `(suppress ...)`");
assert!(
FORM_A_SPEC.contains("empty-suppress-reason"),
"spec missing the empty-suppress-reason diagnostic"
);
// Make sure `Suppress` in the AST can still be constructed — the
// exhaustive-match property carries through to the surface.
let _ = Suppress {
code: "x".into(),
because: "y".into(),
};
}