Iter 14c: ailang-surface ships — form (A) parser + pretty-printer

Strictly additive new crate per Decision 6 architectural pin.
JSON-AST stays the source of truth; ailang-surface is one
producer/consumer of ailang_core::ast::Module values. ailang-check
and ailang-codegen unchanged.

Crate contents:
- src/lex.rs (~264 LOC): 3-rule lexical core. Whitespace and parens
  delimit tokens; semicolon to EOL is comment; first-character
  classifier (digit -> int, " -> string, else -> ident).
- src/parse.rs (~1041 LOC): hand-written recursive descent. One Rust
  fn per EBNF production. No parser-combinator dep.
- src/print.rs (~371 LOC): deterministic pretty-printer. Round-trip
  contract with parse() is the surface's correctness gate.
- tests/round_trip.rs (~128 LOC): integration test runs every
  examples/*.ail.json fixture through print -> parse -> canonical
  JSON, asserts canonical-byte equality with the original.

Two AST-driven form widenings beyond the 14b sketch (both folded
into DESIGN.md Decision 6):
- lam-term carries (typed name type) params, ret type, and
  optional effects (Term::Lam has parallel param_tys/ret_ty/
  effects fields).
- import-clause admits (import name (as alias)?) (Import.alias
  is Option<String>).

Production count ~28, under 30-rule budget. Constraint 1
(formalisable for foreign LLM) intact.

Verification:
- cargo build --workspace green.
- cargo test --workspace: 76 tests green (was 64; +9 surface unit,
  +2 round-trip integration). All 17 fixtures round-trip
  byte-identical at canonical level. 3 hand-written .ailx
  exhibits parse to canonical JSON identical to their .ail.json
  siblings.
- cargo doc --no-deps zero warnings (DESIGN.md item 6 invariant).

Manual smoke test (ail parse → ail run): hello, box,
list_map_poly all produce expected output through the form-(A)
authoring lane end-to-end.

CLI: ail parse <file.ailx> [-o <file.ail.json>]. .ail.json
remains a first-class input to every existing subcommand.

Plan 14d: stdlib (std_list.ailx with length/filter/fold/concat/
reverse/head/tail), authored in form (A) from day one.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 16:22:14 +02:00
parent 5e8342df02
commit 706f90bacd
13 changed files with 2065 additions and 4 deletions
Generated
+10
View File
@@ -9,6 +9,7 @@ dependencies = [
"ailang-check",
"ailang-codegen",
"ailang-core",
"ailang-surface",
"anyhow",
"clap",
"serde_json",
@@ -46,6 +47,15 @@ dependencies = [
"thiserror",
]
[[package]]
name = "ailang-surface"
version = "0.0.1"
dependencies = [
"ailang-core",
"serde_json",
"thiserror",
]
[[package]]
name = "anstream"
version = "1.0.0"
+2
View File
@@ -4,6 +4,7 @@ members = [
"crates/ailang-core",
"crates/ailang-check",
"crates/ailang-codegen",
"crates/ailang-surface",
"crates/ail",
]
@@ -26,6 +27,7 @@ indexmap = { version = "2", features = ["serde"] }
ailang-core = { path = "crates/ailang-core" }
ailang-check = { path = "crates/ailang-check" }
ailang-codegen = { path = "crates/ailang-codegen" }
ailang-surface = { path = "crates/ailang-surface" }
[profile.release]
lto = "thin"
+1
View File
@@ -12,6 +12,7 @@ path = "src/main.rs"
ailang-core.workspace = true
ailang-check.workspace = true
ailang-codegen.workspace = true
ailang-surface.workspace = true
serde_json.workspace = true
clap.workspace = true
anyhow.workspace = true
+36
View File
@@ -171,6 +171,17 @@ enum Cmd {
#[arg(long)]
json: bool,
},
/// Parses a `.ailx` source file (form (A)) into canonical
/// `.ail.json`. Iter 14c addition; symmetric to `render`.
///
/// The form-(A) projection is one of potentially many producers of
/// `Module` values. The JSON-AST remains the source of truth and
/// is what every other subcommand consumes.
Parse {
path: PathBuf,
#[arg(short, long)]
output: Option<PathBuf>,
},
}
fn main() -> Result<()> {
@@ -531,6 +542,31 @@ fn main() -> Result<()> {
}
}
}
Cmd::Parse { path, output } => {
// Read .ailx, parse via the surface crate, emit canonical
// JSON. Symmetric to `render` (which goes the other way).
let src = std::fs::read_to_string(&path)
.with_context(|| format!("reading {}", path.display()))?;
let module = match ailang_surface::parse(&src) {
Ok(m) => m,
Err(e) => {
eprintln!("parse error: {e}");
std::process::exit(1);
}
};
let bytes = ailang_core::canonical::to_bytes(&module);
match output {
Some(p) => {
std::fs::write(&p, &bytes)?;
eprintln!("wrote {}", p.display());
}
None => {
use std::io::Write;
std::io::stdout().write_all(&bytes)?;
println!();
}
}
}
Cmd::Deps { path, of, json, workspace } => {
if workspace {
let ws = ailang_core::load_workspace(&path)?;
+10
View File
@@ -0,0 +1,10 @@
[package]
name = "ailang-surface"
version.workspace = true
edition.workspace = true
license.workspace = true
[dependencies]
ailang-core.workspace = true
serde_json.workspace = true
thiserror.workspace = true
+264
View File
@@ -0,0 +1,264 @@
//! Lexer for form (A).
//!
//! The lexical layer has three rules:
//!
//! 1. Whitespace (` `, `\t`, `\n`, `\r`) and parens (`(`, `)`) are the
//! only delimiters. Any maximal non-whitespace, non-paren run is a
//! single token.
//! 2. `;` introduces a comment to end-of-line. Comments produce no
//! tokens.
//! 3. After tokenisation, classify by first character:
//! - `"` ⇒ string literal (consume until closing `"`; `\"` and
//! `\n` escapes supported).
//! - digit, or `-` followed by digit ⇒ integer literal.
//! - otherwise ⇒ ident.
//!
//! Operators (`+`, `==`, `<=`, `**`), qualified names like
//! `io/print_int`, and dotted cross-module references like
//! `std_list.map` are all single-token idents — there is no special
//! lexical rule for any of them. `(`, `)`, and whitespace are the only
//! reserved tokens. Bool literals (`true`, `false`) and unit
//! (`(lit-unit)`) are disambiguated by the parser via context.
use thiserror::Error;
/// Source byte offset (start, end] into the original input. `end` is
/// one past the last byte. Used for error reporting only.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Span {
pub start: usize,
pub end: usize,
}
/// One lexical token.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Tok {
/// `(`
LParen,
/// `)`
RParen,
/// Integer literal. Original textual form preserved for error messages;
/// parsed `i64` value carried for direct use.
Int(i64),
/// String literal contents (after escape processing).
Str(String),
/// Identifier. Anything that isn't a paren, integer, or string.
Ident(String),
}
/// A token plus its span.
#[derive(Debug, Clone)]
pub struct Token {
pub tok: Tok,
pub span: Span,
}
/// Lexer error.
#[derive(Debug, Error)]
pub enum LexError {
#[error("unterminated string literal at byte {start}")]
UnterminatedString { start: usize },
#[error("invalid integer literal {literal:?} at byte {start}")]
InvalidInteger { literal: String, start: usize },
#[error("invalid escape sequence \\{ch} in string literal at byte {pos}")]
InvalidEscape { ch: char, pos: usize },
}
/// Tokenise an input string into a flat token stream.
///
/// Whitespace and comments are dropped. Returns a [`LexError`] on
/// unterminated string literals or numeric tokens that fail to parse.
pub fn tokenize(input: &str) -> Result<Vec<Token>, LexError> {
let bytes = input.as_bytes();
let mut out = Vec::new();
let mut i = 0usize;
while i < bytes.len() {
let b = bytes[i];
// Whitespace.
if b == b' ' || b == b'\t' || b == b'\n' || b == b'\r' {
i += 1;
continue;
}
// Comment to end-of-line.
if b == b';' {
while i < bytes.len() && bytes[i] != b'\n' {
i += 1;
}
continue;
}
// Parens.
if b == b'(' {
out.push(Token { tok: Tok::LParen, span: Span { start: i, end: i + 1 } });
i += 1;
continue;
}
if b == b')' {
out.push(Token { tok: Tok::RParen, span: Span { start: i, end: i + 1 } });
i += 1;
continue;
}
// String literal. Iterate over UTF-8 chars (not raw bytes) so
// multi-byte glyphs in the source survive round-trip.
if b == b'"' {
let start = i;
i += 1;
let mut value = String::new();
// Walk via char_indices on the unread tail to keep i a byte offset.
loop {
if i >= bytes.len() {
return Err(LexError::UnterminatedString { start });
}
// Quick byte-level checks for ASCII control bytes that
// can never start a multi-byte UTF-8 sequence.
let c0 = bytes[i];
if c0 == b'"' {
i += 1;
break;
}
if c0 == b'\\' {
if i + 1 >= bytes.len() {
return Err(LexError::UnterminatedString { start });
}
let esc = bytes[i + 1];
match esc {
b'"' => value.push('"'),
b'\\' => value.push('\\'),
b'n' => value.push('\n'),
b't' => value.push('\t'),
b'r' => value.push('\r'),
other => {
return Err(LexError::InvalidEscape {
ch: other as char,
pos: i,
});
}
}
i += 2;
continue;
}
// Decode one full UTF-8 character starting at i. The
// input is guaranteed to be valid UTF-8 by Rust's
// `&str` invariant, so a single `chars()` step on the
// tail gives us the next scalar.
if let Some(ch) = input[i..].chars().next() {
value.push(ch);
i += ch.len_utf8();
} else {
return Err(LexError::UnterminatedString { start });
}
}
out.push(Token {
tok: Tok::Str(value),
span: Span { start, end: i },
});
continue;
}
// Otherwise: maximal non-paren / non-whitespace / non-comment run.
let start = i;
while i < bytes.len() {
let c = bytes[i];
if c == b' '
|| c == b'\t'
|| c == b'\n'
|| c == b'\r'
|| c == b'('
|| c == b')'
|| c == b';'
{
break;
}
i += 1;
}
let raw = &input[start..i];
// Classify.
let first = raw.as_bytes()[0];
let is_int = first.is_ascii_digit()
|| (first == b'-' && raw.len() > 1 && raw.as_bytes()[1].is_ascii_digit());
if is_int {
// Numeric parse must succeed; else parse error.
match raw.parse::<i64>() {
Ok(v) => out.push(Token {
tok: Tok::Int(v),
span: Span { start, end: i },
}),
Err(_) => {
return Err(LexError::InvalidInteger {
literal: raw.to_string(),
start,
});
}
}
} else {
out.push(Token {
tok: Tok::Ident(raw.to_string()),
span: Span { start, end: i },
});
}
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parens_and_idents() {
let toks = tokenize("(foo bar)").unwrap();
assert_eq!(toks.len(), 4);
assert!(matches!(toks[0].tok, Tok::LParen));
assert!(matches!(&toks[1].tok, Tok::Ident(s) if s == "foo"));
assert!(matches!(&toks[2].tok, Tok::Ident(s) if s == "bar"));
assert!(matches!(toks[3].tok, Tok::RParen));
}
#[test]
fn comments_are_dropped() {
let toks = tokenize("(a ; this is a comment\n b)").unwrap();
assert_eq!(toks.len(), 4);
}
#[test]
fn integers() {
let toks = tokenize("42 -7 0").unwrap();
assert_eq!(toks.len(), 3);
assert!(matches!(toks[0].tok, Tok::Int(42)));
assert!(matches!(toks[1].tok, Tok::Int(-7)));
assert!(matches!(toks[2].tok, Tok::Int(0)));
}
#[test]
fn operators_are_idents() {
let toks = tokenize("+ == <= io/print_int std_list.map").unwrap();
let names: Vec<_> = toks
.iter()
.map(|t| match &t.tok {
Tok::Ident(s) => s.clone(),
_ => panic!("not ident"),
})
.collect();
assert_eq!(names, vec!["+", "==", "<=", "io/print_int", "std_list.map"]);
}
#[test]
fn string_with_escapes() {
let toks = tokenize(r#""hello\nworld""#).unwrap();
assert_eq!(toks.len(), 1);
assert!(matches!(&toks[0].tok, Tok::Str(s) if s == "hello\nworld"));
}
#[test]
fn negative_lone_minus_is_ident() {
// `-` alone (no digit follows) is an ident, not a (failed) integer.
let toks = tokenize("-").unwrap();
assert_eq!(toks.len(), 1);
assert!(matches!(&toks[0].tok, Tok::Ident(s) if s == "-"));
}
#[test]
fn string_preserves_multibyte_utf8() {
let toks = tokenize(r#""em — dash and naïve""#).unwrap();
assert_eq!(toks.len(), 1);
assert!(matches!(&toks[0].tok, Tok::Str(s) if s == "em — dash and naïve"));
}
}
+39
View File
@@ -0,0 +1,39 @@
//! AILang authoring surface — form (A) S-expression projection.
//!
//! This crate is **one of potentially many** producers / consumers of
//! [`ailang_core::ast::Module`] values. The JSON-AST in `ailang-core`
//! remains the canonical, hashable, content-addressed source of truth;
//! form (A) is the AI-authoring projection optimised for token-efficient
//! production by LLMs (see `docs/DESIGN.md` Decision 6).
//!
//! ## Modules
//!
//! - [`mod@lex`] — hand-written lexer for the 3-rule lexical core
//! (whitespace / parens / comments delimit tokens; first character
//! classifies into integer / string / ident).
//! - [`mod@parse`] — hand-written recursive-descent parser. One Rust
//! function per EBNF production, line-by-line auditable. No
//! parser-combinator dependency.
//! - [`mod@print`] — deterministic pretty-printer that emits form (A).
//! Round-trip with the parser is the contract that gates the
//! surface (see the integration test `tests/round_trip.rs`).
//!
//! ## Round-trip contract
//!
//! For every well-formed [`ailang_core::ast::Module`] value `m`,
//! [`parse()`] applied to [`print()`]`(m)` produces a module whose
//! canonical JSON bytes equal those of `m`. This is enforced by the
//! integration test against every `examples/*.ail.json` fixture.
//!
//! ## Architectural pin
//!
//! No new AST nodes, no schema changes, no new hashable form.
//! `ailang-check` and `ailang-codegen` are projection-agnostic — they
//! consume `Module` values regardless of which front-end produced them.
pub mod lex;
pub mod parse;
pub mod print;
pub use parse::{parse, ParseError};
pub use print::print;
File diff suppressed because it is too large Load Diff
+371
View File
@@ -0,0 +1,371 @@
//! Pretty-printer for form (A).
//!
//! Mirrors the parser in [`mod@crate::parse`]. Output is deterministic
//! and parseable by the parser — round-trip is the gating contract
//! (see `tests/round_trip.rs`).
//!
//! Indentation is informational only (the lexer ignores it). Two-space
//! per level. Comments are NOT emitted.
use ailang_core::ast::{
Arm, ConstDef, Ctor, Def, FnDef, Import, Literal, Module, Pattern, Term, Type, TypeDef,
};
/// Print a module in form (A).
pub fn print(module: &Module) -> String {
let mut out = String::new();
out.push('(');
out.push_str("module ");
out.push_str(&module.name);
for imp in &module.imports {
out.push('\n');
write_import(&mut out, imp, 1);
}
for def in &module.defs {
out.push('\n');
write_def(&mut out, def, 1);
}
out.push(')');
out.push('\n');
out
}
// ---- helpers --------------------------------------------------------------
fn indent(out: &mut String, level: usize) {
for _ in 0..level {
out.push_str(" ");
}
}
fn write_string_lit(out: &mut String, s: &str) {
out.push('"');
for c in s.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\t' => out.push_str("\\t"),
'\r' => out.push_str("\\r"),
other => out.push(other),
}
}
out.push('"');
}
// ---- imports & defs -------------------------------------------------------
fn write_import(out: &mut String, imp: &Import, level: usize) {
indent(out, level);
out.push_str("(import ");
out.push_str(&imp.module);
if let Some(alias) = &imp.alias {
out.push_str(" as ");
out.push_str(alias);
}
out.push(')');
}
fn write_def(out: &mut String, def: &Def, level: usize) {
match def {
Def::Type(td) => write_type_def(out, td, level),
Def::Fn(fd) => write_fn_def(out, fd, level),
Def::Const(cd) => write_const_def(out, cd, level),
}
}
fn write_type_def(out: &mut String, td: &TypeDef, level: usize) {
indent(out, level);
out.push_str("(data ");
out.push_str(&td.name);
if !td.vars.is_empty() {
out.push_str(" (vars");
for v in &td.vars {
out.push(' ');
out.push_str(v);
}
out.push(')');
}
if let Some(doc) = &td.doc {
out.push('\n');
indent(out, level + 1);
out.push_str("(doc ");
write_string_lit(out, doc);
out.push(')');
}
for c in &td.ctors {
out.push('\n');
write_ctor(out, c, level + 1);
}
out.push(')');
}
fn write_ctor(out: &mut String, c: &Ctor, level: usize) {
indent(out, level);
out.push_str("(ctor ");
out.push_str(&c.name);
for f in &c.fields {
out.push(' ');
write_type(out, f);
}
out.push(')');
}
fn write_fn_def(out: &mut String, fd: &FnDef, level: usize) {
indent(out, level);
out.push_str("(fn ");
out.push_str(&fd.name);
if let Some(doc) = &fd.doc {
out.push('\n');
indent(out, level + 1);
out.push_str("(doc ");
write_string_lit(out, doc);
out.push(')');
}
out.push('\n');
indent(out, level + 1);
out.push_str("(type ");
write_type(out, &fd.ty);
out.push(')');
out.push('\n');
indent(out, level + 1);
out.push_str("(params");
for p in &fd.params {
out.push(' ');
out.push_str(p);
}
out.push(')');
out.push('\n');
indent(out, level + 1);
out.push_str("(body ");
write_term(out, &fd.body, level + 2);
out.push(')');
out.push(')');
}
fn write_const_def(out: &mut String, cd: &ConstDef, level: usize) {
indent(out, level);
out.push_str("(const ");
out.push_str(&cd.name);
if let Some(doc) = &cd.doc {
out.push('\n');
indent(out, level + 1);
out.push_str("(doc ");
write_string_lit(out, doc);
out.push(')');
}
out.push('\n');
indent(out, level + 1);
out.push_str("(type ");
write_type(out, &cd.ty);
out.push(')');
out.push('\n');
indent(out, level + 1);
out.push_str("(body ");
write_term(out, &cd.value, level + 2);
out.push(')');
out.push(')');
}
// ---- types ----------------------------------------------------------------
fn write_type(out: &mut String, t: &Type) {
match t {
Type::Var { name } => out.push_str(name),
Type::Con { name, args } => {
out.push_str("(con ");
out.push_str(name);
for a in args {
out.push(' ');
write_type(out, a);
}
out.push(')');
}
Type::Fn {
params,
ret,
effects,
} => {
out.push_str("(fn-type (params");
for p in params {
out.push(' ');
write_type(out, p);
}
out.push_str(") (ret ");
write_type(out, ret);
out.push(')');
if !effects.is_empty() {
out.push_str(" (effects");
for e in effects {
out.push(' ');
out.push_str(e);
}
out.push(')');
}
out.push(')');
}
Type::Forall { vars, body } => {
out.push_str("(forall (vars");
for v in vars {
out.push(' ');
out.push_str(v);
}
out.push_str(") ");
write_type(out, body);
out.push(')');
}
}
}
// ---- terms ----------------------------------------------------------------
fn write_term(out: &mut String, t: &Term, level: usize) {
match t {
Term::Lit { lit } => write_lit(out, lit),
Term::Var { name } => out.push_str(name),
Term::App { callee, args } => {
out.push_str("(app ");
write_term(out, callee, level);
for a in args {
out.push(' ');
write_term(out, a, level);
}
out.push(')');
}
Term::Let { name, value, body } => {
out.push_str("(let ");
out.push_str(name);
out.push(' ');
write_term(out, value, level);
out.push(' ');
write_term(out, body, level);
out.push(')');
}
Term::If { cond, then, else_ } => {
out.push_str("(if ");
write_term(out, cond, level);
out.push(' ');
write_term(out, then, level);
out.push(' ');
write_term(out, else_, level);
out.push(')');
}
Term::Do { op, args } => {
out.push_str("(do ");
out.push_str(op);
for a in args {
out.push(' ');
write_term(out, a, level);
}
out.push(')');
}
Term::Ctor { type_name, ctor, args } => {
out.push_str("(term-ctor ");
out.push_str(type_name);
out.push(' ');
out.push_str(ctor);
for a in args {
out.push(' ');
write_term(out, a, level);
}
out.push(')');
}
Term::Match { scrutinee, arms } => {
out.push_str("(match ");
write_term(out, scrutinee, level);
for arm in arms {
out.push('\n');
indent(out, level);
write_arm(out, arm, level);
}
out.push(')');
}
Term::Lam {
params,
param_tys,
ret_ty,
effects,
body,
} => {
out.push_str("(lam (params");
for (name, ty) in params.iter().zip(param_tys.iter()) {
out.push_str(" (typed ");
out.push_str(name);
out.push(' ');
write_type(out, ty);
out.push(')');
}
out.push_str(") (ret ");
write_type(out, ret_ty);
out.push(')');
if !effects.is_empty() {
out.push_str(" (effects");
for e in effects {
out.push(' ');
out.push_str(e);
}
out.push(')');
}
out.push_str(" (body ");
write_term(out, body, level);
out.push_str("))");
}
Term::Seq { lhs, rhs } => {
out.push_str("(seq ");
write_term(out, lhs, level);
out.push(' ');
write_term(out, rhs, level);
out.push(')');
}
}
}
fn write_arm(out: &mut String, arm: &Arm, level: usize) {
out.push_str("(case ");
write_pattern(out, &arm.pat);
out.push(' ');
write_term(out, &arm.body, level + 1);
out.push(')');
}
fn write_lit(out: &mut String, lit: &Literal) {
match lit {
Literal::Int { value } => out.push_str(&value.to_string()),
Literal::Bool { value } => out.push_str(if *value { "true" } else { "false" }),
Literal::Str { value } => write_string_lit(out, value),
Literal::Unit => out.push_str("(lit-unit)"),
}
}
fn write_pattern(out: &mut String, p: &Pattern) {
match p {
Pattern::Wild => out.push('_'),
Pattern::Var { name } => out.push_str(name),
Pattern::Lit { lit } => {
out.push_str("(pat-lit ");
// Inside pat-lit, write the bare literal form (no `(lit-unit)`
// — Unit is not allowed in pat-lit per the grammar).
match lit {
Literal::Int { value } => out.push_str(&value.to_string()),
Literal::Bool { value } => out.push_str(if *value { "true" } else { "false" }),
Literal::Str { value } => write_string_lit(out, value),
Literal::Unit => {
// Unit is not a valid pat-lit; fall back to a bare unit-lit
// form. This is unreachable for any well-formed AST that
// came through the typechecker.
out.push_str("(lit-unit)");
}
}
out.push(')');
}
Pattern::Ctor { ctor, fields } => {
out.push_str("(pat-ctor ");
out.push_str(ctor);
for f in fields {
out.push(' ');
write_pattern(out, f);
}
out.push(')');
}
}
}
+128
View File
@@ -0,0 +1,128 @@
//! Round-trip gate for the form-(A) projection.
//!
//! For every `examples/*.ail.json` fixture, this test:
//!
//! 1. Loads the original module via `ailang_core::load_module`.
//! 2. Prints it with `ailang_surface::print`.
//! 3. Re-parses the printed text with `ailang_surface::parse`.
//! 4. Canonicalises both modules and asserts byte-equal canonical JSON.
//!
//! In addition, the three hand-written exhibits (`hello.ailx`,
//! `box.ailx`, `list_map_poly.ailx`) are parsed and asserted to produce
//! canonical JSON identical to their corresponding `.ail.json`
//! fixtures. This is the human/AI authoring ground-truth check that
//! the form is writeable without going through the printer.
use std::path::{Path, PathBuf};
fn examples_dir() -> PathBuf {
// `CARGO_MANIFEST_DIR` is the surface crate root; examples live two
// levels up.
let crate_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
crate_dir.parent().unwrap().parent().unwrap().join("examples")
}
fn list_json_fixtures() -> Vec<PathBuf> {
let dir = examples_dir();
let mut paths: Vec<PathBuf> = std::fs::read_dir(&dir)
.unwrap_or_else(|e| panic!("read_dir({}): {e}", dir.display()))
.filter_map(|entry| entry.ok())
.map(|e| e.path())
.filter(|p| {
p.file_name()
.and_then(|n| n.to_str())
.map(|n| n.ends_with(".ail.json"))
.unwrap_or(false)
})
.collect();
paths.sort();
paths
}
#[test]
fn print_then_parse_round_trips_every_fixture() {
let fixtures = list_json_fixtures();
assert!(
!fixtures.is_empty(),
"no .ail.json fixtures found under {}",
examples_dir().display()
);
let mut failures = Vec::<String>::new();
let mut passed = 0usize;
for path in &fixtures {
match round_trip_one(path) {
Ok(()) => passed += 1,
Err(msg) => failures.push(format!("{}: {msg}", path.display())),
}
}
if !failures.is_empty() {
panic!(
"round-trip failed for {} of {} fixtures (passed: {}):\n{}",
failures.len(),
fixtures.len(),
passed,
failures.join("\n")
);
}
eprintln!("round-trip ok for {passed} fixtures");
}
fn round_trip_one(path: &Path) -> Result<(), String> {
let original = ailang_core::load_module(path).map_err(|e| format!("load: {e}"))?;
let text = ailang_surface::print(&original);
let parsed = ailang_surface::parse(&text)
.map_err(|e| format!("re-parse failed: {e}\n--- printed text ---\n{text}"))?;
let bytes_orig = ailang_core::canonical::to_bytes(&original);
let bytes_round = ailang_core::canonical::to_bytes(&parsed);
if bytes_orig != bytes_round {
let s_orig = String::from_utf8_lossy(&bytes_orig).into_owned();
let s_round = String::from_utf8_lossy(&bytes_round).into_owned();
return Err(format!(
"canonical bytes differ.\noriginal: {s_orig}\nround: {s_round}"
));
}
Ok(())
}
#[test]
fn handwritten_exhibits_match_json_fixtures() {
let dir = examples_dir();
let pairs: &[(&str, &str)] = &[
("hello.ailx", "hello.ail.json"),
("box.ailx", "box.ail.json"),
("list_map_poly.ailx", "list_map_poly.ail.json"),
];
let mut failures = Vec::new();
for (ailx, json) in pairs {
let ailx_path = dir.join(ailx);
let json_path = dir.join(json);
let text = std::fs::read_to_string(&ailx_path)
.unwrap_or_else(|e| panic!("read {}: {e}", ailx_path.display()));
let parsed = match ailang_surface::parse(&text) {
Ok(m) => m,
Err(e) => {
failures.push(format!("{ailx}: parse failed: {e}"));
continue;
}
};
let original = ailang_core::load_module(&json_path)
.unwrap_or_else(|e| panic!("load {}: {e}", json_path.display()));
let bytes_orig = ailang_core::canonical::to_bytes(&original);
let bytes_parsed = ailang_core::canonical::to_bytes(&parsed);
if bytes_orig != bytes_parsed {
let s_orig = String::from_utf8_lossy(&bytes_orig).into_owned();
let s_round = String::from_utf8_lossy(&bytes_parsed).into_owned();
failures.push(format!(
"{ailx} vs {json}: canonical bytes differ.\noriginal: {s_orig}\nparsed: {s_round}"
));
}
}
if !failures.is_empty() {
panic!(
"{} hand-written exhibit(s) failed:\n{}",
failures.len(),
failures.join("\n\n")
);
}
}
+38
View File
@@ -391,6 +391,44 @@ on the shelf for a future iter only if both fail.
`build`, `run`, `manifest`, `deps`, `diff`, `workspace`,
`builtins`) keeps its current `.ail.json` interface.
### Form refinements during Iter 14c implementation
Two productions in the original 14b sketch had to be widened
during implementation to round-trip the existing AST faithfully.
Captured here for the spec record:
1. **`lam-term` carries types and effects.** The AST's `Term::Lam`
stores parallel `params`, `param_tys`, `ret_ty`, and `effects`
fields. The 14b sketch had only names. The implemented form is
```
lam-term ::= "(" "lam" "(" "params" typed-param* ")"
"(" "ret" type ")"
effects-clause? body-attr ")"
typed-param ::= "(" "typed" ident type ")"
```
This keeps the no-precedence / one-construct-per-token-list
invariants and adds no new lexical rules.
2. **`import-clause` admits an optional alias.** The AST's
`Import.alias` is `Option<String>`; the original sketch only
supported the `None` case. The implemented form is
```
import-clause ::= "(" "import" ident ("as" ident)? ")"
```
`as` is a bare ident token in this position; no special lexical
rule is needed.
Neither change extends the grammar's rule budget meaningfully:
the 30-production ceiling of constraint 1 is intact (the parser
implements ~28 named productions). All 17 `examples/*.ail.json`
fixtures round-trip identically through `print → parse → canonical
JSON`; the three hand-written `.ailx` exhibits parse to canonical
JSON identical to their corresponding `.ail.json` files.
## Mangling scheme (Iter 5c)
All AILang functions are mangled to `@ail_<module>_<def>` — even in the
+118
View File
@@ -1627,4 +1627,122 @@ with `length`, `filter`, `fold`, `concat`, `reverse`, `head`,
for the still-young parameterised-ADT pipeline. Written in the
new surface from day one — no JSON authoring of stdlib.
## Iter 14c — `ailang-surface` parser + pretty-printer ships
Implements Decision 6's form (A). New crate `ailang-surface`
(~1843 LOC across `lex.rs`, `parse.rs`, `print.rs`, lib root,
plus tests) ships as a strictly additive producer of
`ailang-core::ast::Module` values. `ailang-check` and
`ailang-codegen` were not modified — projection-agnostic, as
the architectural pin requires.
**Implementer dispatch went clean.** Brief gave the EBNF, the
fixtures to round-trip, the lexer rule, and the architectural
constraints. Implementer hand-wrote a recursive-descent parser
(one Rust fn per EBNF production), a deterministic pretty-
printer, an integration test that runs the round-trip gate on
every fixture, and the `ail parse` CLI subcommand. No
parser-combinator dependency. No AST-shape changes.
**Two AST-driven form refinements** that were not in the 14b
sketch (both folded into DESIGN.md Decision 6 by the
implementer before commit):
1. `lam-term` had to carry `param_tys`, `ret_ty`, and
`effects` because the AST's `Term::Lam` stores parallel
typed-param data. Production became `(lam (params (typed
x Int) ...) (ret T) (effects ...) (body ...))`. Same
shape-style as the rest of the form; no new lex rules.
2. `import-clause` had to admit `Option<String>` aliases.
Production became `(import name (as alias)?)` with `as`
as a bare ident token. No new lex rules.
The 14b 30-production budget held: implementer reports ~28
named productions in the parser. Constraint 1 (formalisable
for foreign LLM) intact.
**Verification gates, all green:**
- `cargo build --workspace`: 0 warnings, finished.
- `cargo test --workspace`: 76 tests pass (was 64; +9 unit
tests in `ailang-surface`, +2 integration tests in
`tests/round_trip.rs`, +1 e2e regression preserved). All
17 `examples/*.ail.json` fixtures round-trip
byte-identical through `print → parse → canonical JSON`;
3 hand-written `.ailx` exhibits parse to canonical JSON
byte-identical to their `.ail.json` siblings.
- `cargo doc --no-deps`: 0 warnings (workspace invariant
from 13d/e/f preserved; new crate's rustdoc landed
correctly with crate-root `//!` plus all `pub` items
documented).
**Manual smoke test (orchestrator-side after agent reported
done):** `ail parse <file.ailx> -o <file.ail.json>` followed
by `ail run <file.ail.json>` for all three exhibits:
```
hello.ailx → "Hello, AILang."
box.ailx → "42"
list_map_poly.ailx → "2\n3\n4"
```
End-to-end pipeline form (A) → AST → typecheck → codegen →
clang → binary works on all three.
**Important non-issue.** `examples/*.ail.json` fixtures on
disk are hand-formatted JSON with non-canonical key order;
the parser produces canonical (lex-sorted) JSON. Diff at
the file-byte level is not zero. Diff at the canonical-byte
level is zero — which is the only thing that matters per
Decision 1, since hashing uses canonical form. The
round-trip test gates on canonical bytes, not file bytes.
This is correct behaviour; flagged here so a future reader
who runs `diff` doesn't think the surface is broken.
**Files created:**
- `crates/ailang-surface/Cargo.toml`
- `crates/ailang-surface/src/lib.rs` (~40 LOC, rustdoc heavy)
- `crates/ailang-surface/src/lex.rs` (~264 LOC)
- `crates/ailang-surface/src/parse.rs` (~1041 LOC, one fn
per production)
- `crates/ailang-surface/src/print.rs` (~371 LOC)
- `crates/ailang-surface/tests/round_trip.rs` (~128 LOC)
**Files modified:**
- `Cargo.toml` (workspace) — `ailang-surface` member +
workspace dep.
- `Cargo.lock` — refresh.
- `crates/ail/Cargo.toml` — `ailang-surface` dep.
- `crates/ail/src/main.rs` — new `Parse { path, output }`
subcommand (~36 LOC).
- `docs/DESIGN.md` — form refinements appendix to
Decision 6.
- `examples/list_map_poly.ailx` — implementer added doc
strings to match the JSON original (was a 14b design
exhibit, not byte-aligned).
**Known debt:** none reported. `ailang-check` /
`ailang-codegen` untouched per the architectural pin.
**Plan 14d (next, queued).** With form (A) now ergonomic
and round-trip-verified, the std-lib path opens up. First
target: `examples/std/std_list.ailx` with `length`,
`filter`, `fold` (left and right), `concat`, `reverse`,
`head`, `tail`. Each combinator a fresh test vector for
the parameterised-ADT pipeline (which 14a opened end-to-
end). Authored in form (A); the resulting `.ail.json` is
what tests consume. If 14d surfaces more codegen bugs in
the parameterised-ADT path (it likely will — 14a found
one already), debugger handles them inline.
The 14b/c form-A hypothesis has held under the empirical
test of round-tripping every existing fixture. The
documented rollback to form (C) is now off the table for
this iter cycle, though the architectural pin keeps it
open for future replacement of `ailang-surface` should
the form prove inadequate at stdlib scale.
+7 -4
View File
@@ -1,20 +1,21 @@
; Iter 14b design exhibit — Decision 6 form (A).
; Companion to box.ailx. Same hash-equivalence gate in Iter 14c.
; Iter 14b design exhibit, finalised in Iter 14c.
; Hash-equivalent to list_map_poly.ail.json (round-trip-tested).
(module list_map_poly
(data List (vars a)
(doc "Singly-linked polymorphic list.")
(doc "Polymorphic singly-linked list.")
(ctor Nil)
(ctor Cons a (con List a)))
(fn inc
(doc "Add 1 to an Int.")
(type (fn-type (params (con Int)) (ret (con Int))))
(params x)
(body (app + x 1)))
(fn map
(doc "Apply f to every element. Recursive on the tail.")
(doc "Polymorphic map: apply f to every element, recursing on the tail.")
(type
(forall (vars a b)
(fn-type
@@ -32,6 +33,7 @@
(app map f t))))))
(fn print_list
(doc "Print each Int on its own line.")
(type
(fn-type
(params (con List (con Int)))
@@ -48,6 +50,7 @@
(app print_list t))))))
(fn main
(doc "Map inc over [1,2,3] via polymorphic List, then print: 2,3,4.")
(type (fn-type (params) (ret (con Unit)) (effects IO)))
(params)
(body