AILang/docs/plans/0112-reserved-dollar-in-names.md

Reserve $ in the Form-A lexer — Implementation Plan

Parent spec: docs/specs/0057-reserved-dollar-in-names.md

For agentic workers: REQUIRED SUB-SKILL: use the implement skill to run this plan. Steps use - [ ] checkboxes for tracking.

Goal: Reject $ in any Form-A identifier token at the lexer, so the compiler's $-synthetic namespace ($mp_N, <hint>$lr_N, <base>$<n>) can never collide with an authored name (closes the latent collision class behind #44).

Architecture: A single new LexError::ReservedDollar variant plus a one-line guard inline in tokenize, raised on any token run containing $, before int/float/ident classification. The error surfaces through the existing ParseError::Lex → W::SurfaceParse → surface-parse-error channel with zero new wiring. String- and comment-internal $ stay legal because both are consumed by earlier branches of the scan loop, before a run is ever sliced. A stale doc-comment on fresh_binder is corrected to state the now-enforced invariant.

Tech Stack: crates/ailang-surface/src/lex.rs (lexer + in-source tests), crates/ailang-surface/tests/ (integration test), crates/ailang-core/src/desugar.rs (doc-comment).

---

Files this plan creates or modifies:

• Modify: crates/ailang-surface/src/lex.rs:75 — add ReservedDollar { token: String, start: usize } to the LexError enum (after the InvalidEscape variant at line 75, before the enum's closing } at line 76).
• Modify: crates/ailang-surface/src/lex.rs:184 — insert the $-reject guard between let raw = &input[start..i]; (line 183) and let first = raw.as_bytes()[0]; (line 185).
• Test: crates/ailang-surface/src/lex.rs:~509 — add 3 in-source tests inside mod tests (before its closing } at line 510): dollar_in_ident_is_reserved, dollar_in_string_literal_is_allowed, dollar_in_comment_is_allowed.
• Create: crates/ailang-surface/tests/reserved_dollar_pin.rs — integration test feeding a full (module …) with a $ binder through parse, asserting Err(ParseError::Lex(LexError::ReservedDollar { … })).
• Modify: crates/ailang-core/src/desugar.rs:528-536 — correct the fresh_binder doc-comment (the now-false "an authored binder may legally contain $" / "tracked as a follow-up" claims).

---

Task 1: Lexer $ reservation (variant + guard + in-source tests)

Files:

• Modify: crates/ailang-surface/src/lex.rs:75 (enum variant)

• Modify: crates/ailang-surface/src/lex.rs:184 (guard)

• Test: crates/ailang-surface/src/lex.rs:~509 (3 in-source tests)

• Step 1: Add the ReservedDollar variant to LexError

In crates/ailang-surface/src/lex.rs, the LexError enum currently ends:

    #[error("invalid escape sequence \\{ch} in string literal at byte {pos}")]
    InvalidEscape { ch: char, pos: usize },
}

Add the new variant immediately before the closing } (after InvalidEscape):

    #[error("invalid escape sequence \\{ch} in string literal at byte {pos}")]
    InvalidEscape { ch: char, pos: usize },
    #[error("reserved character `$` in token {token:?} at byte {start}; \
             `$` is reserved for compiler-synthetic names")]
    ReservedDollar { token: String, start: usize },
}

• Step 2: Write the failing test dollar_in_ident_is_reserved

In the #[cfg(test)] mod tests block at the bottom of crates/ailang-surface/src/lex.rs (closes at line 510), add this test before the module's closing }:

    #[test]
    fn dollar_in_ident_is_reserved() {
        let err = tokenize("x$1").expect_err("`$` in an ident must be a lex error");
        assert!(
            matches!(err, LexError::ReservedDollar { ref token, .. } if token == "x$1"),
            "expected ReservedDollar(\"x$1\"), got {err:?}",
        );
    }

• Step 3: Run the test to verify it fails

Run: cargo test -p ailang-surface dollar_in_ident_is_reserved Expected: FAIL — the test panics with `$` in an ident must be a lex error (because the guard does not exist yet, tokenize("x$1") returns Ok, so expect_err panics).

• Step 4: Add the $-reject guard inline in tokenize

In crates/ailang-surface/src/lex.rs, the run-classification currently reads (lines 183-186):

        let raw = &input[start..i];
        // Classify.
        let first = raw.as_bytes()[0];
        let is_int = first.is_ascii_digit()

Insert the guard immediately after the let raw line, before the // Classify. comment:

        let raw = &input[start..i];
        // raw-buf.#44: `$` is reserved for compiler-synthetic names
        // (`$mp_N`, `<hint>$lr_N`, `<base>$<n>`). Reject any authored
        // token run containing it, before int/float/ident classification.
        if raw.contains('$') {
            return Err(LexError::ReservedDollar { token: raw.to_string(), start });
        }
        // Classify.
        let first = raw.as_bytes()[0];
        let is_int = first.is_ascii_digit()

• Step 5: Run the test to verify it passes

Run: cargo test -p ailang-surface dollar_in_ident_is_reserved Expected: PASS.

• Step 6: Add the two exemption tests

In the same mod tests block (before its closing }), add:

    #[test]
    fn dollar_in_string_literal_is_allowed() {
        // The `$` lives inside a string; it must NOT trip the reservation.
        let toks = tokenize(r#""price: $5""#).expect("string-internal `$` is legal");
        assert!(matches!(toks.as_slice(), [Token { tok: Tok::Str(s), .. }] if s == "price: $5"));
    }

    #[test]
    fn dollar_in_comment_is_allowed() {
        // Comment is stripped before tokenization; the ident `x` survives.
        let toks = tokenize("x ; mentions loop$lr_0\n").expect("comment `$` is legal");
        assert!(matches!(toks.as_slice(), [Token { tok: Tok::Ident(s), .. }] if s == "x"));
    }

• Step 7: Run the exemption tests + the full lexer suite

Run: cargo test -p ailang-surface --lib Expected: PASS — all lexer tests green, including the 19 pre-existing ones plus the 3 new (dollar_in_ident_is_reserved, dollar_in_string_literal_is_allowed, dollar_in_comment_is_allowed). Confirm the run reports 22 lexer tests, not 19 (the count proves the 3 new tests were collected and ran).

---

Task 2: Correct the fresh_binder doc-comment

Files:

• Modify: crates/ailang-core/src/desugar.rs:528-536

• Step 1: Replace the stale doc paragraph

In crates/ailang-core/src/desugar.rs, the fresh_binder doc-comment currently reads (lines 528-536):

    /// `$` is the project's convention for synthetic names, but it is
    /// NOT lexer-enforced — an authored binder may legally contain `$`.
    /// The probe rejects a `<base>$<n>` that is already in `used` or
    /// currently in `scope`, but it does NOT see an authored
    /// `<base>$<n>` that is out of scope at mint time yet later binds
    /// under the same `(def, name)` key. No fixture exercises an
    /// authored `$` binder; closing that latent gap (enforce the
    /// reservation, or seed `used` with every authored binder name in
    /// the def) is tracked as a follow-up.

Replace those 9 lines with:

    /// `$` is reserved for the compiler's synthetic namespace and is
    /// rejected in any authored Form-A identifier by the lexer
    /// (`ailang_surface::lex::tokenize` → `LexError::ReservedDollar`).
    /// So a `<base>$<n>` candidate minted here cannot alias any
    /// authored name — no authored name can contain `$`. The probe
    /// against `used` and `scope` therefore guards only against prior
    /// synthetic mints and currently in-scope renamed binders. (The
    /// reservation is on *Form-A authored* identifiers; the `.ail.json`
    /// deserialization path is intentionally not guarded — it is the
    /// orchestrator's self-responsible channel, not client-reachable.)

• Step 2: Verify ailang-core still builds

Run: cargo build -p ailang-core Expected: build succeeds (doc-comment-only change; 0 errors).

---

Task 3: Integration test — the surfacing chain end-to-end

Files:

• Create: crates/ailang-surface/tests/reserved_dollar_pin.rs

• Step 1: Write the integration test

Create crates/ailang-surface/tests/reserved_dollar_pin.rs with:

//! #44: a `$` in an authored Form-A identifier is rejected at the
//! lexer and surfaces through `parse` as `ParseError::Lex`. Pins that
//! the reservation reaches the public `parse` entry point (not just
//! the internal `tokenize`), so a hallucinating Form-A author cannot
//! land a `$` binder in a `Module`.

// `parse`, `ParseError`, `LexError` are all re-exported at the crate
// root (`pub use` in lib.rs); the `lex` / `parse` modules themselves
// are private, so import from the crate root, not the module paths.
use ailang_surface::{parse, LexError, ParseError};

#[test]
fn dollar_binder_in_module_is_rejected_at_parse() {
    // (module bad (fn main (type (fn-type (params) (ret (con Int)))) (params) (body (let x$1 7 x$1))))
    let src = "(module bad\n  (fn main\n    (type (fn-type (params) (ret (con Int))))\n    (params)\n    (body\n      (let x$1 7 x$1))))\n";
    let err = parse(src).expect_err("authored `$` binder must be rejected at parse");
    assert!(
        matches!(err, ParseError::Lex(LexError::ReservedDollar { ref token, .. }) if token == "x$1"),
        "expected ParseError::Lex(ReservedDollar(\"x$1\")), got {err:?}",
    );
}

#[test]
fn dollar_in_string_in_module_parses_clean() {
    // The exemption survives at the `parse` level too: `$` inside a
    // string literal in a real module is fine.
    let src = "(module dollar_ok\n  (fn main\n    (type (fn-type (params) (ret (con Str))))\n    (params)\n    (body \"price: $5\")))\n";
    parse(src).expect("string-internal `$` must parse clean");
}

• Step 2: Confirm the imports resolve against the crate's public API

Run: cargo build -p ailang-surface --tests Expected: build succeeds (0 errors). The imports use the crate-root re-exports ailang_surface::{parse, LexError, ParseError} (confirmed present in crates/ailang-surface/src/lib.rs:8-9); the lex / parse modules are private, so the module-qualified paths would NOT compile.

• Step 3: Run the integration test

Run: cargo test -p ailang-surface --test reserved_dollar_pin Expected: PASS — both dollar_binder_in_module_is_rejected_at_parse and dollar_in_string_in_module_parses_clean green (2 tests ran).

---

Task 4: No-regression sweep

Files: none (verification only)

• Step 1: Full workspace test suite

Run: cargo test --workspace Expected: PASS — all crates green. In particular crates/ailang-surface/tests/round_trip.rs (parse_then_print_then_parse_is_idempotent_on_every_ail_fixture, parse_is_deterministic_over_every_ail_fixture) stays green: no examples/*.ail carries an authored $ ident, and the six comment-only $ files (e.g. local_rec_let_capture.ail) survive because comments are stripped before a run is sliced.

• Step 2: CLI check — comment-$ example survives end-to-end

Run: cargo run -q -p ail -- check examples/local_rec_let_capture.ail Expected: exit 0 (ok (… symbols …)) — a real checked-in file whose comments mention loop$lr_0 still checks clean, proving the guard does not reach comment text at the CLI level.

• Step 3: CLI check — the #43 shadowing idiom still compiles clean

Run: cargo run -q -p ail -- build examples/raw_buf_int.ail -o /tmp/rawbuf_int_check Expected: exit 0 — raw_buf_int.ail is the real nested-let-shadow idiom ((let buf (new …) (let buf (app RawBuf.set buf …) …))); its inner buf → buf$1 rename is minted at desugar, after parse, and is never re-lexed, so the reservation does not regress it. (If ail build needs different flags in this tree, fall back to cargo run -q -p ail -- check examples/raw_buf_int.ail and assert exit 0 — check alone still exercises parse → desugar → typecheck over the shadow rename.)