Iter 13d: rustdoc polish for ailang-core + new docwriter agent
Adds ailang-docwriter to /agents/ — a recurring role for keeping crate-, module-, and pub-item-level rustdoc accurate. First mission: ailang-core. Crate root, every module root, every pub item documented; intra-doc links throughout; Iter-13a additions (TypeDef.vars, Type::Con.args) get an explicit backwards-compat note. Two stale broken-link warnings in ailang-check fixed in passing. cargo doc --no-deps now warning-free across the workspace; promoted to verification invariant 6 in DESIGN.md.
This commit is contained in:
@@ -14,6 +14,7 @@ the repo.
|
||||
| `ailang-architect.md` | Read-only reviewer; checks for drift against DESIGN.md after iterations. |
|
||||
| `ailang-tester.md` | Writes examples and E2E tests. |
|
||||
| `ailang-debugger.md` | Diagnoses compiler or codegen bugs. |
|
||||
| `ailang-docwriter.md` | Writes and maintains rustdoc on the Rust crates. |
|
||||
|
||||
## Invocation scheme
|
||||
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: ailang-docwriter
|
||||
description: Writes and maintains rustdoc for the AILang crates. Brings crate, module, and public-item docs up to a level where a newcomer can navigate `cargo doc --open` without having read DESIGN.md first. NOT for changing APIs, NOT for editing files in `docs/`.
|
||||
tools: Read, Edit, Write, Bash, Glob, Grep
|
||||
---
|
||||
|
||||
You are the **docwriter** for the AILang project at `/home/brummel/dev/ailang`.
|
||||
|
||||
Your output is `///` and `//!` doc comments inside the Rust source. The
|
||||
audience is an LLM (or human) who has just opened `cargo doc --open` and
|
||||
clicked into one of the crates — they have NOT read `docs/DESIGN.md`.
|
||||
|
||||
## Mandatory reading order
|
||||
|
||||
1. `CLAUDE.md`, `docs/DESIGN.md`, the most recent entries in `docs/JOURNAL.md`.
|
||||
2. The crate(s) the assignment names — read every `pub` item before you write a single doc line.
|
||||
3. Run `cargo doc --no-deps 2>&1` and read all warnings. Every warning the assignment touches must be gone when you're done.
|
||||
|
||||
## Documentation rules (binding)
|
||||
|
||||
- **Crate root (`//!` in `src/lib.rs` / `src/main.rs`)** explains, in this order:
|
||||
what this crate is, how it fits into the pipeline (`core` → `check` → `codegen` → `ail` CLI), the most important entry points (with intra-doc links), and the key invariants.
|
||||
- **Module root (`//!` at the top of every `src/<mod>.rs`)** answers: what does this module own, what does it not own, what is the typical entry point.
|
||||
- **Every `pub` item** (struct, enum, fn, type alias, trait, const) gets a `///` doc string. One sentence is fine if the name is self-evident. Two-to-five sentences when the item carries an invariant, a non-obvious cost, or a precondition the caller must respect.
|
||||
- **Use intra-doc links** ([`Type`], [`fn_name`], [`module::item`]). No prose-only references to types — a reader should be able to click.
|
||||
- **Add `# Examples` sections** sparingly: only where an example genuinely shortens the path to understanding. Keep them in plain markdown unless the crate already runs doctests; if you write a code block, mark it ` ```ignore ` or ` ```no_run ` so it doesn't have to compile against the workspace.
|
||||
- **Cross-repo references** (DESIGN.md, JOURNAL.md, the `ail` CLI subcommands) are fine as plain prose mentions — those are NOT in rustdoc, so don't try to link them.
|
||||
|
||||
## Hard limits
|
||||
|
||||
- No API changes, no renames, no signature tweaks. If a name is so confusing it needs renaming, raise it in your report instead of changing it.
|
||||
- No new `pub` exports. Visibility stays as-is.
|
||||
- No edits in `docs/`. The orchestrator owns DESIGN.md and JOURNAL.md.
|
||||
- Don't paper over broken behaviour with prose — if doc-writing surfaces a real bug, stop and report it.
|
||||
|
||||
## Verification (all must pass before reporting done)
|
||||
|
||||
- `cargo doc --no-deps 2>&1` — zero warnings on every line you touched.
|
||||
- `cargo build --workspace` — green.
|
||||
- `cargo test --workspace` — green (doc-tests count).
|
||||
|
||||
## Output format
|
||||
|
||||
At most 200 words:
|
||||
- **Files touched** (paths + which level: crate / module / item docs).
|
||||
- **Warnings cleared** (count, plus the rustdoc check status).
|
||||
- **Findings** — items whose names or behaviour seemed confusing while documenting them. One line each, no prescriptions. The orchestrator decides whether they become a follow-up.
|
||||
@@ -3,7 +3,7 @@
|
||||
//! A [`Diagnostic`] is the machine-readable representation of a problem
|
||||
//! reported by the typechecker (or an upstream load step).
|
||||
//! [`Severity`] serializes as a lowercase string
|
||||
//! (`"error"` / `"warning"`); [`code`] is a stable kebab-case identifier
|
||||
//! (`"error"` / `"warning"`); the `code` field is a stable kebab-case identifier
|
||||
//! that tooling can consume without parsing the `message` text.
|
||||
//!
|
||||
//! Convention: each call to [`super::check_module`] reports at most one
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
//! Effects are propagated as a set and reconciled against the annotation
|
||||
//! on the function type.
|
||||
//!
|
||||
//! Built-in operations are resolved via [`Builtins`].
|
||||
//! Built-in operations are resolved via the internal `Builtins` table.
|
||||
//!
|
||||
//! ## Internals
|
||||
//!
|
||||
|
||||
+163
-15
@@ -1,32 +1,78 @@
|
||||
//! AST nodes. Serde layout matches the JSON schema in `docs/DESIGN.md`.
|
||||
//! AST nodes for the AILang language.
|
||||
//!
|
||||
//! Every type in this module is the in-memory mirror of a node in the
|
||||
//! AILang JSON schema documented in `docs/DESIGN.md`. The serde
|
||||
//! attributes carry the schema: field renames (`as`, `type`, `fn`,
|
||||
//! `paramTypes`, `retType`), enum tags (`kind`, `t`, `k`, `p`), and
|
||||
//! `skip_serializing_if` predicates that keep the canonical-JSON
|
||||
//! representation backwards compatible across schema extensions.
|
||||
//!
|
||||
//! The entry type is [`Module`]. The two helpers [`def_name`] and
|
||||
//! [`def_kind`] are intended for tools (`ail diff`, `ail manifest`) that
|
||||
//! consume a [`Def`] without going through method calls.
|
||||
//!
|
||||
//! This module does **not** typecheck, evaluate, or hash anything — see
|
||||
//! [`crate::canonical`] for canonical bytes, [`crate::hash`] for content
|
||||
//! hashes, and the `ailang-check` crate for typechecking.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A complete AILang translation unit.
|
||||
///
|
||||
/// Carries the schema tag (always [`crate::SCHEMA`] for this version),
|
||||
/// a module name, the list of imports, and the list of top-level
|
||||
/// definitions. A module is the smallest unit that can be loaded,
|
||||
/// hashed, and emitted as LLVM IR.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Module {
|
||||
/// Schema identifier; must equal [`crate::SCHEMA`] at load time.
|
||||
pub schema: String,
|
||||
/// Module name. By convention matches the file stem
|
||||
/// (`<name>.ail.json`); the [`crate::workspace`] loader enforces
|
||||
/// this on import.
|
||||
pub name: String,
|
||||
/// Imports of other modules. Resolved by the workspace loader as
|
||||
/// `<root_dir>/<module>.ail.json`.
|
||||
#[serde(default)]
|
||||
pub imports: Vec<Import>,
|
||||
/// Top-level definitions in declaration order.
|
||||
pub defs: Vec<Def>,
|
||||
}
|
||||
|
||||
/// An import statement.
|
||||
///
|
||||
/// `module` is the bare module name (no path, no extension). `alias`
|
||||
/// renames it for use in the body; absent means the module is referred
|
||||
/// to under its own name.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Import {
|
||||
/// Imported module name, resolved relative to the entry file's
|
||||
/// directory.
|
||||
pub module: String,
|
||||
/// Optional rename (`import foo as bar`). Serialized as `as` in
|
||||
/// JSON; omitted when absent.
|
||||
#[serde(rename = "as", default, skip_serializing_if = "Option::is_none")]
|
||||
pub alias: Option<String>,
|
||||
}
|
||||
|
||||
/// A top-level definition — the unit that gets a content hash.
|
||||
///
|
||||
/// Discriminated by the JSON `kind` tag: `"fn"`, `"const"`, or
|
||||
/// `"type"`. Use [`def_name`] / [`def_kind`] (or the [`Def::name`]
|
||||
/// method) to inspect generically without matching every variant.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", rename_all = "lowercase")]
|
||||
pub enum Def {
|
||||
/// Function definition; see [`FnDef`].
|
||||
Fn(FnDef),
|
||||
/// Constant definition; see [`ConstDef`].
|
||||
Const(ConstDef),
|
||||
/// Type (ADT) definition; see [`TypeDef`].
|
||||
Type(TypeDef),
|
||||
}
|
||||
|
||||
impl Def {
|
||||
/// Source-level name of the definition, regardless of kind.
|
||||
pub fn name(&self) -> &str {
|
||||
match self {
|
||||
Def::Fn(f) => &f.name,
|
||||
@@ -36,14 +82,20 @@ impl Def {
|
||||
}
|
||||
}
|
||||
|
||||
/// External helper: name of a definition (for tools like `ail diff`
|
||||
/// that consume the def node decoupled from the method call).
|
||||
/// External helper: name of a definition.
|
||||
///
|
||||
/// Equivalent to [`Def::name`], exposed as a free function so tools
|
||||
/// like `ail diff` that hold a [`Def`] node can read its name without
|
||||
/// importing the inherent-method namespace.
|
||||
pub fn def_name(def: &Def) -> &str {
|
||||
def.name()
|
||||
}
|
||||
|
||||
/// External helper: discriminator tag of a definition (`fn`, `const`, `type`).
|
||||
/// Identical to the `kind` field in the JSON representation.
|
||||
/// External helper: discriminator tag of a definition (`"fn"`,
|
||||
/// `"const"`, `"type"`).
|
||||
///
|
||||
/// Identical to the `kind` field in the JSON representation. Useful
|
||||
/// for tools that group or filter definitions by kind.
|
||||
pub fn def_kind(def: &Def) -> &'static str {
|
||||
match def {
|
||||
Def::Fn(_) => "fn",
|
||||
@@ -52,75 +104,129 @@ pub fn def_kind(def: &Def) -> &'static str {
|
||||
}
|
||||
}
|
||||
|
||||
/// An algebraic data type definition.
|
||||
///
|
||||
/// A `TypeDef` introduces a named type with one or more constructors.
|
||||
/// Monomorphic types (`vars` empty) and parameterised types (`vars`
|
||||
/// non-empty) share the same node — see the field docs for the
|
||||
/// schema-compatibility note.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct TypeDef {
|
||||
/// Type name (capitalised by convention).
|
||||
pub name: String,
|
||||
/// Type parameters (Iter 13a). A monomorphic ADT has `vars` empty
|
||||
/// and is serialized identically to the pre-13a schema (the field is
|
||||
/// skipped when empty), preserving the canonical-JSON hash of every
|
||||
/// existing module on disk.
|
||||
/// Type parameters (Iter 13a parameterised-ADT support). A
|
||||
/// monomorphic ADT has `vars` empty and is serialized identically
|
||||
/// to the pre-13a schema (the field is **omitted** when empty),
|
||||
/// preserving the canonical-JSON hash of every existing module on
|
||||
/// disk. See the regression test
|
||||
/// `iter13a_schema_extension_preserves_pre_13a_hashes` in
|
||||
/// [`crate::hash`].
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub vars: Vec<String>,
|
||||
/// Constructors in declaration order. Pattern-match arms in
|
||||
/// `ailang-check` are validated against this list.
|
||||
pub ctors: Vec<Ctor>,
|
||||
/// Optional source-level documentation string.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub doc: Option<String>,
|
||||
}
|
||||
|
||||
/// A single constructor of a [`TypeDef`].
|
||||
///
|
||||
/// `fields` is the constructor's positional argument list as types;
|
||||
/// nullary constructors have an empty list.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Ctor {
|
||||
/// Constructor name (capitalised by convention; unique within its
|
||||
/// `TypeDef`).
|
||||
pub name: String,
|
||||
/// Positional field types. Empty for nullary constructors.
|
||||
#[serde(default)]
|
||||
pub fields: Vec<Type>,
|
||||
}
|
||||
|
||||
/// A function definition.
|
||||
///
|
||||
/// `ty` is the full function type (including effect set and any
|
||||
/// `Forall` quantifier for top-level polymorphism). `params` are the
|
||||
/// names bound in `body`, in order.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct FnDef {
|
||||
/// Function name.
|
||||
pub name: String,
|
||||
/// Declared function type. Top-level polymorphism is opt-in via
|
||||
/// [`Type::Forall`].
|
||||
#[serde(rename = "type")]
|
||||
pub ty: Type,
|
||||
/// Parameter names, in the order they appear in `ty.params`.
|
||||
pub params: Vec<String>,
|
||||
/// Function body.
|
||||
pub body: Term,
|
||||
/// Optional source-level documentation string.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub doc: Option<String>,
|
||||
}
|
||||
|
||||
/// A constant (top-level binding to a value).
|
||||
///
|
||||
/// Differs from a zero-arg [`FnDef`] in that it is evaluated once and
|
||||
/// has no parameter list; the codegen emits it as a global.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ConstDef {
|
||||
/// Constant name.
|
||||
pub name: String,
|
||||
/// Declared type of the value.
|
||||
#[serde(rename = "type")]
|
||||
pub ty: Type,
|
||||
/// The value expression. Must be pure (no effects).
|
||||
pub value: Term,
|
||||
/// Optional source-level documentation string.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub doc: Option<String>,
|
||||
}
|
||||
|
||||
/// An expression node ("Term" = computation that produces a value).
|
||||
///
|
||||
/// The JSON discriminator is the `t` field. Each variant's doc
|
||||
/// describes its concrete-syntax intent; the typing rules live in
|
||||
/// `ailang-check`.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "t", rename_all = "lowercase")]
|
||||
pub enum Term {
|
||||
/// Literal value (see [`Literal`]).
|
||||
Lit { lit: Literal },
|
||||
/// Variable reference (parameter, local, top-level def, or
|
||||
/// imported alias).
|
||||
Var { name: String },
|
||||
/// Function application. `callee` is evaluated to a function value;
|
||||
/// `args` are evaluated left-to-right.
|
||||
App {
|
||||
#[serde(rename = "fn")]
|
||||
callee: Box<Term>,
|
||||
args: Vec<Term>,
|
||||
},
|
||||
/// Let-binding: `value` is evaluated and bound to `name` in `body`.
|
||||
Let {
|
||||
name: String,
|
||||
value: Box<Term>,
|
||||
body: Box<Term>,
|
||||
},
|
||||
/// If-expression. Both branches must have the same type.
|
||||
If {
|
||||
cond: Box<Term>,
|
||||
then: Box<Term>,
|
||||
#[serde(rename = "else")]
|
||||
else_: Box<Term>,
|
||||
},
|
||||
/// Effect operation invocation (e.g. `do print "hi"`). The `op` is
|
||||
/// resolved against the effect-handler table at link time.
|
||||
Do {
|
||||
op: String,
|
||||
args: Vec<Term>,
|
||||
},
|
||||
/// Constructor application. `type_name` binds the ADT, `ctor` the variant.
|
||||
/// Example: `Some(42)` -> `{ "t": "ctor", "type": "Option", "ctor": "Some",
|
||||
/// Constructor application. `type_name` binds the ADT, `ctor` the
|
||||
/// variant. Example: `Some(42)` ->
|
||||
/// `{ "t": "ctor", "type": "Option", "ctor": "Some",
|
||||
/// "args": [{"t":"lit","lit":{"kind":"int","value":42}}] }`.
|
||||
Ctor {
|
||||
#[serde(rename = "type")]
|
||||
@@ -129,7 +235,8 @@ pub enum Term {
|
||||
#[serde(default)]
|
||||
args: Vec<Term>,
|
||||
},
|
||||
/// Pattern matching over a value.
|
||||
/// Pattern matching over a value. Arms are tried top-to-bottom;
|
||||
/// exhaustiveness is checked by `ailang-check`.
|
||||
Match {
|
||||
scrutinee: Box<Term>,
|
||||
arms: Vec<Arm>,
|
||||
@@ -158,12 +265,22 @@ pub enum Term {
|
||||
},
|
||||
}
|
||||
|
||||
/// One arm of a [`Term::Match`].
|
||||
///
|
||||
/// `pat` is the pattern; `body` is evaluated when the pattern matches,
|
||||
/// with any pattern-bound variables in scope.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Arm {
|
||||
/// Pattern to match the scrutinee against.
|
||||
pub pat: Pattern,
|
||||
/// Body to evaluate on a successful match.
|
||||
pub body: Term,
|
||||
}
|
||||
|
||||
/// A match pattern.
|
||||
///
|
||||
/// The JSON discriminator is the `p` field. Patterns are linear: each
|
||||
/// pattern variable may appear at most once.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "p", rename_all = "lowercase")]
|
||||
pub enum Pattern {
|
||||
@@ -181,36 +298,63 @@ pub enum Pattern {
|
||||
},
|
||||
}
|
||||
|
||||
/// A literal value.
|
||||
///
|
||||
/// The JSON discriminator is the `kind` field. `Unit` carries no
|
||||
/// payload and serializes as `{"kind":"unit"}`.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", rename_all = "lowercase")]
|
||||
pub enum Literal {
|
||||
/// Signed 64-bit integer.
|
||||
Int { value: i64 },
|
||||
/// Boolean.
|
||||
Bool { value: bool },
|
||||
/// UTF-8 string.
|
||||
Str { value: String },
|
||||
/// The unit value `()`.
|
||||
Unit,
|
||||
}
|
||||
|
||||
/// A type expression.
|
||||
///
|
||||
/// The JSON discriminator is the `k` field. The four variants are
|
||||
/// type constructor application ([`Type::Con`]), function type
|
||||
/// ([`Type::Fn`]), type variable ([`Type::Var`]), and universal
|
||||
/// quantifier ([`Type::Forall`] — only valid at the top level of an
|
||||
/// [`FnDef`] / [`ConstDef`] type).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "k", rename_all = "lowercase")]
|
||||
pub enum Type {
|
||||
/// Type-constructor application: `Int`, `Bool`, user ADT names,
|
||||
/// and parameterised forms like `List<Int>`.
|
||||
Con {
|
||||
name: String,
|
||||
/// Type arguments (Iter 13a). For pre-13a uses (`Int`, `Bool`,
|
||||
/// non-parameterised user ADTs) this stays empty and is skipped
|
||||
/// during serialization, so the canonical-JSON hash of every
|
||||
/// pre-existing module remains bit-identical.
|
||||
/// non-parameterised user ADTs) this stays empty and is
|
||||
/// **omitted** during serialization, so the canonical-JSON
|
||||
/// hash of every pre-existing module remains bit-identical.
|
||||
/// See the regression test in [`crate::hash`].
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
args: Vec<Type>,
|
||||
},
|
||||
/// Function type with optional effect annotation. `effects` is a
|
||||
/// set; equality compares it modulo order (see the [`PartialEq`]
|
||||
/// impl below).
|
||||
Fn {
|
||||
params: Vec<Type>,
|
||||
ret: Box<Type>,
|
||||
#[serde(default)]
|
||||
effects: Vec<String>,
|
||||
},
|
||||
/// Type variable. During checking, names with the prefix `$m`
|
||||
/// denote checker-internal metavariables; source-level names
|
||||
/// cannot collide because identifiers may not start with `$`.
|
||||
Var {
|
||||
name: String,
|
||||
},
|
||||
/// Universal quantifier (top-level polymorphism only). `body`
|
||||
/// is the quantified type; `vars` are the bound type-variable
|
||||
/// names, instantiated fresh at each use site.
|
||||
Forall {
|
||||
vars: Vec<String>,
|
||||
body: Box<Type>,
|
||||
@@ -218,15 +362,19 @@ pub enum Type {
|
||||
}
|
||||
|
||||
impl Type {
|
||||
/// Convenience constructor for `Int` (no args).
|
||||
pub fn int() -> Type {
|
||||
Type::Con { name: "Int".into(), args: vec![] }
|
||||
}
|
||||
/// Convenience constructor for `Bool` (no args).
|
||||
pub fn bool_() -> Type {
|
||||
Type::Con { name: "Bool".into(), args: vec![] }
|
||||
}
|
||||
/// Convenience constructor for `Unit` (no args).
|
||||
pub fn unit() -> Type {
|
||||
Type::Con { name: "Unit".into(), args: vec![] }
|
||||
}
|
||||
/// Convenience constructor for `Str` (no args).
|
||||
pub fn str_() -> Type {
|
||||
Type::Con { name: "Str".into(), args: vec![] }
|
||||
}
|
||||
|
||||
@@ -1,11 +1,44 @@
|
||||
//! Canonical JSON serialization.
|
||||
//!
|
||||
//! Writes JSON without whitespace and with lexicographically sorted object keys.
|
||||
//! This makes the representation deterministic and suitable for hashing.
|
||||
//! Writes JSON without whitespace and with lexicographically sorted
|
||||
//! object keys. This makes the representation deterministic across
|
||||
//! runs, platforms, and `serde_json` versions, which is what makes it
|
||||
//! suitable as the pre-image for [`crate::def_hash`] /
|
||||
//! [`crate::module_hash`].
|
||||
//!
|
||||
//! The single entry point is [`to_bytes`]. This module deliberately
|
||||
//! does **not** parse, validate, or hash — it only emits bytes. It also
|
||||
//! does not attempt to be a general-purpose canonical-JSON library;
|
||||
//! number-formatting follows whatever `serde_json::Number::to_string`
|
||||
//! does, which is fine for the AILang AST (integers and short strings;
|
||||
//! no floats).
|
||||
//!
|
||||
//! # Examples
|
||||
//!
|
||||
//! ```ignore
|
||||
//! use ailang_core::canonical;
|
||||
//! use serde_json::json;
|
||||
//!
|
||||
//! // Object keys are sorted; no whitespace appears in the output.
|
||||
//! let bytes = canonical::to_bytes(&json!({ "b": 1, "a": 2 }));
|
||||
//! assert_eq!(std::str::from_utf8(&bytes).unwrap(), r#"{"a":2,"b":1}"#);
|
||||
//! ```
|
||||
|
||||
use std::io::Write;
|
||||
|
||||
/// Canonical bytes for any Serializable object.
|
||||
/// Serialize `value` to canonical JSON bytes.
|
||||
///
|
||||
/// Object keys are sorted lexicographically, no whitespace is emitted,
|
||||
/// and arrays preserve their input order. The output is the byte
|
||||
/// pre-image used by [`crate::def_hash`] and
|
||||
/// [`crate::workspace::module_hash`]; if you change what bytes this
|
||||
/// function emits for a given value, all on-disk hashes change with it.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics if `value` cannot be converted to a `serde_json::Value`
|
||||
/// (i.e. its `Serialize` impl errors). For the AST types in
|
||||
/// [`crate::ast`] this never happens.
|
||||
pub fn to_bytes<T: serde::Serialize>(value: &T) -> Vec<u8> {
|
||||
let v = serde_json::to_value(value).expect("serializable");
|
||||
let mut out = Vec::new();
|
||||
|
||||
@@ -1,14 +1,45 @@
|
||||
//! Content-addressed hashing for definitions.
|
||||
//!
|
||||
//! Hash = BLAKE3 over the canonical JSON form (see `canonical`).
|
||||
//! The `hash` field in the input is removed before hashing.
|
||||
//! Hash = BLAKE3 over the canonical JSON bytes of a [`Def`] (see
|
||||
//! [`crate::canonical`]). The hash function takes a [`Def`] by
|
||||
//! reference, so there is no `hash` field to strip — the in-memory
|
||||
//! struct does not carry one.
|
||||
//!
|
||||
//! The single entry point at this level is [`def_hash`]. The parallel
|
||||
//! entry point at module granularity is
|
||||
//! [`crate::workspace::module_hash`].
|
||||
|
||||
use crate::ast::Def;
|
||||
use crate::canonical;
|
||||
|
||||
/// 16-hex-char (64-bit) prefix of the BLAKE3 hash.
|
||||
/// Enough for uniqueness within realistic codebases and compact
|
||||
/// enough for visual inspection.
|
||||
/// Content hash of a single [`Def`] — the 16-hex-char (64-bit) prefix
|
||||
/// of its BLAKE3 hash over canonical JSON bytes.
|
||||
///
|
||||
/// 64 bits is wide enough to be unique across realistic AILang
|
||||
/// codebases and short enough to read at a glance in pretty-printed
|
||||
/// manifests. The hash is computed over the **canonical JSON byte
|
||||
/// pre-image**, not over the in-memory struct, so any change to the
|
||||
/// canonical form (new fields, different `skip_serializing_if`
|
||||
/// behaviour, key reordering bug) changes every hash. The Iter 13a
|
||||
/// regression test below pins concrete hashes for two example
|
||||
/// definitions to catch that.
|
||||
///
|
||||
/// # Examples
|
||||
///
|
||||
/// ```ignore
|
||||
/// use ailang_core::{ast::*, def_hash};
|
||||
///
|
||||
/// let def = Def::Const(ConstDef {
|
||||
/// name: "answer".into(),
|
||||
/// ty: Type::int(),
|
||||
/// value: Term::Lit { lit: Literal::Int { value: 42 } },
|
||||
/// doc: None,
|
||||
/// });
|
||||
///
|
||||
/// // Stable across runs: same canonical bytes -> same hash.
|
||||
/// assert_eq!(def_hash(&def), def_hash(&def));
|
||||
/// assert_eq!(def_hash(&def).len(), 16);
|
||||
/// ```
|
||||
pub fn def_hash(def: &Def) -> String {
|
||||
let bytes = canonical::to_bytes(def);
|
||||
let h = blake3::hash(&bytes);
|
||||
|
||||
@@ -1,7 +1,48 @@
|
||||
//! AILang core data model.
|
||||
//! Foundation crate of the AILang toolchain.
|
||||
//!
|
||||
//! The source of an AILang translation unit is a `Module` as JSON. This
|
||||
//! crate defines the schema, serialization, and content-addressed hashing.
|
||||
//! `core` owns the data model that every other crate consumes:
|
||||
//!
|
||||
//! - **AST** — see [`ast`]. The source of an AILang translation unit is an
|
||||
//! [`ast::Module`] expressed as JSON; the types here mirror that schema
|
||||
//! one-to-one via serde.
|
||||
//! - **Canonical JSON** — see [`canonical`]. A deterministic byte form
|
||||
//! (no whitespace, lexicographically sorted object keys) used as the
|
||||
//! pre-image for hashing.
|
||||
//! - **Content-addressed hashes** — see [`hash`] and [`def_hash`]. Each
|
||||
//! definition gets a stable 16-hex-char BLAKE3 prefix derived from its
|
||||
//! canonical bytes, which is how the rest of the pipeline refers to it.
|
||||
//! - **Pretty-printing** — see [`pretty`]. AST → human-readable text for
|
||||
//! diff/review. The JSON form remains the source of truth.
|
||||
//! - **Workspace loading** — see [`workspace`] and [`load_workspace`].
|
||||
//! Resolves transitive imports of an entry module and returns a
|
||||
//! [`Workspace`].
|
||||
//!
|
||||
//! ## Place in the pipeline
|
||||
//!
|
||||
//! `core` → `check` (typecheck) → `codegen` (LLVM emission) → `ail` CLI.
|
||||
//! `core` has no upstream dependencies inside the workspace; everything
|
||||
//! else depends on it.
|
||||
//!
|
||||
//! ## Central invariant
|
||||
//!
|
||||
//! Canonical JSON is **deterministic**: the byte output of
|
||||
//! [`canonical::to_bytes`] for a given value never changes across runs,
|
||||
//! versions of `serde_json`, or platforms. Hashes are
|
||||
//! **content-addressed**: equal canonical bytes ⇒ equal hash. Schema
|
||||
//! identifier is `ailang/v0`; a [`Module`] whose `schema` field disagrees
|
||||
//! is rejected at load time. Schema extensions added in later iterations
|
||||
//! (e.g. parameterised ADTs in Iter 13a) are written so that all
|
||||
//! pre-existing on-disk hashes remain bit-identical — see
|
||||
//! [`ast::TypeDef::vars`] and the `args` field of [`ast::Type::Con`].
|
||||
//!
|
||||
//! ## Entry points
|
||||
//!
|
||||
//! - Read a single module from disk: [`load_module`].
|
||||
//! - Read an entry module plus all imports: [`load_workspace`].
|
||||
//! - Hash a [`Def`]: [`def_hash`]. Hash a whole [`Module`]:
|
||||
//! [`workspace::module_hash`].
|
||||
//! - Render a [`Module`] for review: [`pretty::module`] /
|
||||
//! [`pretty::manifest`].
|
||||
|
||||
pub mod ast;
|
||||
pub mod canonical;
|
||||
@@ -15,22 +56,41 @@ pub use ast::{
|
||||
pub use hash::def_hash;
|
||||
pub use workspace::{load_workspace, module_hash, Workspace, WorkspaceLoadError};
|
||||
|
||||
/// Errors raised while loading a single module file.
|
||||
///
|
||||
/// Workspace-level loading uses a richer enum that wraps this one; see
|
||||
/// [`WorkspaceLoadError`].
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum Error {
|
||||
/// Module file parses as JSON but its `schema` field does not match
|
||||
/// [`SCHEMA`].
|
||||
#[error("schema mismatch: expected {expected:?}, got {got:?}")]
|
||||
SchemaMismatch { expected: String, got: String },
|
||||
|
||||
/// Module file failed to parse as JSON or did not match the AST
|
||||
/// schema (missing/unexpected fields).
|
||||
#[error("json: {0}")]
|
||||
Json(#[from] serde_json::Error),
|
||||
|
||||
/// File could not be read.
|
||||
#[error("io: {0}")]
|
||||
Io(#[from] std::io::Error),
|
||||
}
|
||||
|
||||
/// Result alias scoped to the crate-level [`Error`].
|
||||
pub type Result<T> = std::result::Result<T, Error>;
|
||||
|
||||
/// Schema identifier this version of `core` accepts. Every loaded
|
||||
/// [`Module`] must carry exactly this string in its `schema` field, or
|
||||
/// loading fails with [`Error::SchemaMismatch`].
|
||||
pub const SCHEMA: &str = "ailang/v0";
|
||||
|
||||
/// Load a single module file, validate its schema, and deserialize it
|
||||
/// into a [`Module`].
|
||||
///
|
||||
/// This is the lowest-level entry point and does **not** resolve imports
|
||||
/// — for the transitive closure use [`load_workspace`]. Errors are
|
||||
/// returned as [`Error::Io`] / [`Error::Json`] / [`Error::SchemaMismatch`].
|
||||
pub fn load_module(path: &std::path::Path) -> Result<Module> {
|
||||
let bytes = std::fs::read(path)?;
|
||||
let module: Module = serde_json::from_slice(&bytes)?;
|
||||
|
||||
@@ -1,12 +1,29 @@
|
||||
//! Pretty-printer: AST → human-readable text form.
|
||||
//!
|
||||
//! The text form is intended as a diff and review tool. The
|
||||
//! canonical source remains the JSON form. Every pretty output is
|
||||
//! deterministic.
|
||||
//! The text form is intended for diff and review (LLM and human eyes).
|
||||
//! The canonical source remains the JSON form, and the pretty output
|
||||
//! is deliberately one-way: there is no parser back from text into AST
|
||||
//! in this crate. Every pretty output is deterministic.
|
||||
//!
|
||||
//! Two top-level renderers are provided:
|
||||
//! - [`module`] — full S-expression dump of every definition.
|
||||
//! - [`manifest`] — one-line summary per def with its type and
|
||||
//! [`crate::def_hash`].
|
||||
//!
|
||||
//! Helpers [`pattern_to_string`] and [`type_to_string`] are exposed
|
||||
//! because diagnostics in `ailang-check` reuse them in error messages.
|
||||
//!
|
||||
//! This module deliberately does **not** roundtrip text back to AST
|
||||
//! and does not attempt to mirror canonical JSON byte-for-byte.
|
||||
|
||||
use crate::ast::*;
|
||||
use std::fmt::Write;
|
||||
|
||||
/// Render a complete [`Module`] as an S-expression-style text block.
|
||||
///
|
||||
/// Output format is stable but not part of the on-disk schema —
|
||||
/// consumers should treat it as documentation, not a serialization
|
||||
/// format. Used by `ail print` and by the pretty-print review flow.
|
||||
pub fn module(m: &Module) -> String {
|
||||
let mut s = String::new();
|
||||
writeln!(s, "(module {}", m.name).unwrap();
|
||||
@@ -31,6 +48,12 @@ pub fn module(m: &Module) -> String {
|
||||
s
|
||||
}
|
||||
|
||||
/// Render a one-line-per-def manifest of a [`Module`].
|
||||
///
|
||||
/// Each line carries the def kind keyword (`fn` / `const` / `type`),
|
||||
/// the name padded to a fixed width, the rendered type, and the
|
||||
/// 16-hex-char [`crate::def_hash`] in brackets. Used by `ail manifest`
|
||||
/// and as the canonical "what's in this module" summary.
|
||||
pub fn manifest(m: &Module) -> String {
|
||||
let mut s = String::new();
|
||||
writeln!(s, "module {}", m.name).unwrap();
|
||||
@@ -246,6 +269,10 @@ fn term_block(t: &Term, indent: usize) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
/// Render a [`Pattern`] as a single-line string.
|
||||
///
|
||||
/// Used both internally by [`module`] for match arms and externally by
|
||||
/// `ailang-check` when constructing diagnostic messages.
|
||||
pub fn pattern_to_string(p: &Pattern) -> String {
|
||||
match p {
|
||||
Pattern::Wild => "_".into(),
|
||||
@@ -340,6 +367,13 @@ fn lit_to_string(l: &Literal) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
/// Render a [`Type`] as a single-line string.
|
||||
///
|
||||
/// The format mirrors typical ML-family notation: type-constructor
|
||||
/// applications use angle brackets (`List<Int>`), function types use
|
||||
/// arrow syntax (`(Int, Int) -> Int`) with optional `!eff1,eff2`
|
||||
/// effect suffixes, and `Forall` becomes `forall a b. ...`. Used by
|
||||
/// [`manifest`] and by `ailang-check` diagnostics.
|
||||
pub fn type_to_string(t: &Type) -> String {
|
||||
match t {
|
||||
Type::Con { name, args } => {
|
||||
|
||||
@@ -1,10 +1,35 @@
|
||||
//! Workspace loader: loads an entry module and recursively follows its
|
||||
//! `imports`. Convention: an `import { module: "foo" }` is resolved
|
||||
//! relative to the entry file's directory as `<root_dir>/foo.ail.json`.
|
||||
//! `imports`.
|
||||
//!
|
||||
//! Iter 5a is responsible for **finding** and consistently **loading**
|
||||
//! all reachable modules. Cross-module typecheck (5b) and codegen (5c)
|
||||
//! build on top, but are not part of this step.
|
||||
//! Convention: an `import { module: "foo" }` is resolved relative to
|
||||
//! the entry file's directory as `<root_dir>/foo.ail.json`. Module
|
||||
//! names must match the file stem (the loader rejects mismatches).
|
||||
//!
|
||||
//! The single entry point is [`load_workspace`]; it returns a fully
|
||||
//! populated [`Workspace`] or a structured [`WorkspaceLoadError`].
|
||||
//! [`module_hash`] is the module-granularity counterpart to
|
||||
//! [`crate::def_hash`] and is used both internally (to detect a module
|
||||
//! re-loaded from disk with different content) and by the CLI.
|
||||
//!
|
||||
//! This module is responsible only for **finding** and consistently
|
||||
//! **loading** all reachable modules. Cross-module typechecking lives
|
||||
//! in `ailang-check`; codegen in `ailang-codegen`. Neither is run from
|
||||
//! here.
|
||||
//!
|
||||
//! # Examples
|
||||
//!
|
||||
//! ```ignore
|
||||
//! use ailang_core::load_workspace;
|
||||
//! use std::path::Path;
|
||||
//!
|
||||
//! let ws = load_workspace(Path::new("examples/ws_main.ail.json"))?;
|
||||
//! assert_eq!(ws.entry, "ws_main");
|
||||
//! // All transitively imported modules are now in `ws.modules`.
|
||||
//! for (name, m) in &ws.modules {
|
||||
//! println!("{name}: {} defs", m.defs.len());
|
||||
//! }
|
||||
//! # Ok::<(), ailang_core::workspace::WorkspaceLoadError>(())
|
||||
//! ```
|
||||
|
||||
use crate::ast::Module;
|
||||
use crate::canonical;
|
||||
@@ -20,8 +45,14 @@ use std::path::{Path, PathBuf};
|
||||
/// in; all imports are resolved relative to it.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Workspace {
|
||||
/// Name of the entry module (the one passed to [`load_workspace`]).
|
||||
pub entry: String,
|
||||
/// Every module reachable from `entry`, indexed by module name.
|
||||
/// `BTreeMap` is used so iteration order is deterministic, which
|
||||
/// matters for downstream codegen and reporting.
|
||||
pub modules: BTreeMap<String, Module>,
|
||||
/// Directory the entry file lives in; all imports are resolved
|
||||
/// relative to it.
|
||||
pub root_dir: PathBuf,
|
||||
}
|
||||
|
||||
@@ -31,6 +62,8 @@ pub struct Workspace {
|
||||
/// closed — the last element is the name already present in `visiting`.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum WorkspaceLoadError {
|
||||
/// File I/O failed while reading a module file (typically: file
|
||||
/// missing, permission denied).
|
||||
#[error("io error for {path}: {source}")]
|
||||
Io {
|
||||
path: PathBuf,
|
||||
@@ -38,6 +71,9 @@ pub enum WorkspaceLoadError {
|
||||
source: std::io::Error,
|
||||
},
|
||||
|
||||
/// File contents failed to parse as a [`Module`] or had the wrong
|
||||
/// schema tag. Wraps a [`CoreError`] (the single-module loader's
|
||||
/// error type).
|
||||
#[error("schema/parse error in {path}: {source}")]
|
||||
Schema {
|
||||
path: PathBuf,
|
||||
@@ -45,9 +81,14 @@ pub enum WorkspaceLoadError {
|
||||
source: CoreError,
|
||||
},
|
||||
|
||||
/// An `import { module: "foo" }` did not resolve to an existing
|
||||
/// file at the expected path.
|
||||
#[error("module `{name}` not found (expected at {expected_path})")]
|
||||
ModuleNotFound { name: String, expected_path: PathBuf },
|
||||
|
||||
/// The `name` field in a loaded module file disagrees with the
|
||||
/// file-stem-derived name the loader expected (i.e. the
|
||||
/// `<name>.ail.json` convention is broken).
|
||||
#[error(
|
||||
"module name in file ({name_in_file:?}) does not match expected name from path ({name_from_path:?})"
|
||||
)]
|
||||
@@ -56,9 +97,15 @@ pub enum WorkspaceLoadError {
|
||||
name_from_path: String,
|
||||
},
|
||||
|
||||
/// An import cycle was detected. `path` is the chain of module
|
||||
/// names in which the cycle was closed — the last element is the
|
||||
/// name that was already on the visit stack.
|
||||
#[error("import cycle detected: {}", path.join(" -> "))]
|
||||
Cycle { path: Vec<String> },
|
||||
|
||||
/// A module was reachable through two import paths, and its
|
||||
/// on-disk content (compared via [`module_hash`]) differs between
|
||||
/// the two reads. This typically means the file changed mid-load.
|
||||
#[error(
|
||||
"module `{name}` was loaded twice with differing content (hashes differ)"
|
||||
)]
|
||||
|
||||
@@ -241,6 +241,9 @@ ail run <module> — build + execute (tempdir), passthrough exit
|
||||
4. **Hash stability**: a test ensures the same def always produces the same
|
||||
hash.
|
||||
5. **CI pin** of the outputs in `tests/expected/`.
|
||||
6. **Rustdoc cleanliness**: `cargo doc --no-deps` runs warning-free.
|
||||
Maintained by the `ailang-docwriter` agent (Iter 13d onward); fixing a
|
||||
rustdoc warning is part of the iter that introduced it, not a follow-up.
|
||||
|
||||
## What is not (yet) supported
|
||||
|
||||
|
||||
@@ -1146,3 +1146,99 @@ Leaning 14a — the dogfood payoff for one iter of polish is
|
||||
high, and `Maybe`-in-a-real-fn is a missing piece I haven't
|
||||
exercised yet. 14b stays second; 14c is a candidate if I want
|
||||
a small palate cleanser.
|
||||
|
||||
---
|
||||
|
||||
## Iter 13d — rustdoc polish for `ailang-core` + new `ailang-docwriter` agent
|
||||
|
||||
User triggered: ran `cargo doc --open` for fun and reported
|
||||
that the rendered docs were thin — crate headers existed,
|
||||
but `pub` items had no `///` strings, there were no `# Examples`
|
||||
sections, and intra-doc links were missing. Internal references
|
||||
showed up as prose ("see Builtins") rather than as clickable
|
||||
links. Two stale `[Builtins]` and `[code]` warnings had been
|
||||
bleeding into every `cargo doc` invocation since Iter 6 or so.
|
||||
|
||||
Recurring task → new agent. Wrote `agents/ailang-docwriter.md`
|
||||
with a tight mandate: rustdoc only, no API changes, no edits in
|
||||
`docs/` or `agents/`, three verification gates (rustdoc clean,
|
||||
build green, tests green incl. doctests). Updated
|
||||
`agents/README.md`. Updated DESIGN.md item 6 of "Verification
|
||||
and correctness" to make rustdoc cleanliness a project-wide
|
||||
invariant rather than an iter-local cleanup.
|
||||
|
||||
First mission: `ailang-core` only. The foundation crate every
|
||||
other crate depends on — biggest reader-leverage per diff. The
|
||||
agent rewrote the crate root so a newcomer learns: what `core`
|
||||
owns, where it sits in the pipeline (`core` → `check` →
|
||||
`codegen` → `ail`), the central invariant (canonical JSON is
|
||||
deterministic, hashes content-addressed, schema = `ailang/v0`),
|
||||
and the entry points. Module roots in `ast.rs`, `canonical.rs`,
|
||||
`hash.rs`, `pretty.rs`, `workspace.rs` got the same treatment.
|
||||
Every `pub` struct / enum / variant / fn / const got a `///`
|
||||
string. The Iter-13a additions (`TypeDef.vars`, `Type::Con.args`)
|
||||
got an explicit backwards-compat note that points back to the
|
||||
hash-stability regression test.
|
||||
|
||||
`# Examples` blocks landed where they shorten understanding
|
||||
(`canonical::to_bytes`, `def_hash`, `Workspace`), all marked
|
||||
`ignore` so the workspace doctest run stays cheap (3 ignored,
|
||||
0 run, 0 failed). The two stale broken-link warnings in
|
||||
`ailang-check` got prose-only fixes — out-of-scope for this
|
||||
iter conceptually, but a one-line fix per file means rustdoc
|
||||
is now globally clean.
|
||||
|
||||
**Findings reported by the docwriter** (judgement deferred to
|
||||
me; none made it into the diff):
|
||||
|
||||
- `Term::Lam.param_tys` (JSON: `paramTypes`) is positionally
|
||||
paired with `Term::Lam.params: Vec<String>`. Naming hints
|
||||
at the convention but a cold reader has to deduce it. **Not
|
||||
fixing**: renaming would touch the schema, breaks every hash.
|
||||
The `///` string makes the convention explicit, which is
|
||||
enough.
|
||||
- `Type::Var` is overloaded: source-level rigid vars and
|
||||
checker metavars (`$m<id>`) share the same variant. A reader
|
||||
of `core` alone sees no hint of the metavar half — it's
|
||||
documented in `ailang-check`'s lib doc instead. **Not fixing**:
|
||||
splitting the variant would balloon the schema and
|
||||
invalidate every hash. Acceptable as long as `check`'s lib
|
||||
doc explains it (it does, post-13d).
|
||||
- `Def::Type(TypeDef)` versus the type-expression enum `Type`
|
||||
in the same module: name collision is real but unavoidable
|
||||
without renaming `Type` (which would touch every crate). The
|
||||
`///` strings now disambiguate at point of contact.
|
||||
|
||||
**Process note (orchestration).** Second iter where I worked
|
||||
strictly through agents (after 13b). The docwriter brief was
|
||||
written from the diagnostic in this conversation, not from a
|
||||
DESIGN-doc design pass — there was no architecture decision to
|
||||
make, just a discipline gap to close. That's the right shape
|
||||
for a docwriter: low-judgement, repeatable, runs after every
|
||||
iter that touches public surface. Cost ≈ 25 tool uses for ~390
|
||||
LOC of doc additions across 9 files; smaller-grain than
|
||||
`ailang-implementer` runs typically are.
|
||||
|
||||
**Tests:** 64/64 unit + e2e (unchanged), 3 ignored doctests
|
||||
(new). `cargo doc --no-deps`: 0 warnings (was 2). `cargo build
|
||||
--workspace` green. `cargo test --workspace` green.
|
||||
|
||||
**Plan iteration 13e/13f (queued, not started).** Two natural
|
||||
follow-ups for the docwriter:
|
||||
|
||||
13e. **`ailang-check` rustdoc**: type-checker is the next
|
||||
biggest crate by `pub`-surface and the most algorithmically
|
||||
dense. Rigid/metavar split, `Forall` instantiation, the
|
||||
match-arm exhaustiveness logic, the Iter-13a substitution
|
||||
machinery — all of it benefits more from prose explanation
|
||||
than `core` did.
|
||||
13f. **`ailang-codegen` + `ail` CLI rustdoc**: codegen is dense
|
||||
but mechanical (mangling, ABI, block tracking); the CLI
|
||||
is mostly clap derive macros. Lower payoff per LOC of doc
|
||||
than 13e but rounds out the warning-free invariant across
|
||||
the whole workspace.
|
||||
|
||||
After 13d there's no urgency on 13e/f — `cargo doc` is already
|
||||
warning-free. They're "polish iterations to be slotted in
|
||||
between feature iters when context budget is short". 14a
|
||||
(`List a` rewrite) remains the next-feature default.
|
||||
|
||||
Reference in New Issue
Block a user