plan: reserved-dollar-in-names — dollar-lexer-reservation (refs #44)

Executable plan for spec 0057 (committed c760570). Single iteration, four tasks: (1) LexError::ReservedDollar variant + inline tokenize guard + 3 in-source lexer tests (RED-first on the must-fail dollar_in_ident_is_reserved); (2) fresh_binder doc-comment correction; (3) a surface integration test pinning the parse -> ParseError::Lex(ReservedDollar) surfacing chain; (4) no-regression sweep (workspace suite + CLI checks on a comment-dollar example and the #43 shadow idiom). plan-recon verified all line anchors against HEAD (lex.rs:183 run-slice, lex.rs:67-76 enum, 19 in-source tests, desugar.rs:528-536 doc-comment, parse.rs:103/126/149, main.rs:1339, ct1_check_cli.rs:310) and confirmed none of the three CLAUDE.md lockstep pairs apply. Self-review caught and fixed two defects before hand-off: the integration test imported the private lex/parse module paths (corrected to the crate-root re-exports ailang_surface::{parse, LexError, ParseError}), and the compile-gate check confirmed no external exhaustive match on LexError exists, so adding a variant cannot break compilation and the Task 1 RED is a genuine runtime expect_err panic.
2026-05-30 14:51:41 +02:00
parent c76057008e
commit 559531806d
1 changed files with 291 additions and 0 deletions
@@ -0,0 +1,291 @@
+# Reserve `$` in the Form-A lexer — Implementation Plan
+
+> **Parent spec:** `docs/specs/0057-reserved-dollar-in-names.md`
+>
+> **For agentic workers:** REQUIRED SUB-SKILL: use the
+> `implement` skill to run this plan. Steps use `- [ ]`
+> checkboxes for tracking.
+
+**Goal:** Reject `$` in any Form-A identifier token at the lexer, so
+the compiler's `$`-synthetic namespace (`$mp_N`, `<hint>$lr_N`,
+`<base>$<n>`) can never collide with an authored name (closes the
+latent collision class behind #44).
+
+**Architecture:** A single new `LexError::ReservedDollar` variant plus
+a one-line guard inline in `tokenize`, raised on any token run
+containing `$`, before int/float/ident classification. The error
+surfaces through the existing `ParseError::Lex` → `W::SurfaceParse` →
+`surface-parse-error` channel with zero new wiring. String- and
+comment-internal `$` stay legal because both are consumed by earlier
+branches of the scan loop, before a run is ever sliced. A stale
+doc-comment on `fresh_binder` is corrected to state the now-enforced
+invariant.
+
+**Tech Stack:** `crates/ailang-surface/src/lex.rs` (lexer + in-source
+tests), `crates/ailang-surface/tests/` (integration test),
+`crates/ailang-core/src/desugar.rs` (doc-comment).
+
+---
+
+**Files this plan creates or modifies:**
+
+- Modify: `crates/ailang-surface/src/lex.rs:75` — add `ReservedDollar { token: String, start: usize }` to the `LexError` enum (after the `InvalidEscape` variant at line 75, before the enum's closing `}` at line 76).
+- Modify: `crates/ailang-surface/src/lex.rs:184` — insert the `$`-reject guard between `let raw = &input[start..i];` (line 183) and `let first = raw.as_bytes()[0];` (line 185).
+- Test: `crates/ailang-surface/src/lex.rs:~509` — add 3 in-source tests inside `mod tests` (before its closing `}` at line 510): `dollar_in_ident_is_reserved`, `dollar_in_string_literal_is_allowed`, `dollar_in_comment_is_allowed`.
+- Create: `crates/ailang-surface/tests/reserved_dollar_pin.rs` — integration test feeding a full `(module …)` with a `$` binder through `parse`, asserting `Err(ParseError::Lex(LexError::ReservedDollar { … }))`.
+- Modify: `crates/ailang-core/src/desugar.rs:528-536` — correct the `fresh_binder` doc-comment (the now-false "an authored binder may legally contain `$`" / "tracked as a follow-up" claims).
+
+---
+
+## Task 1: Lexer `$` reservation (variant + guard + in-source tests)
+
+**Files:**
+- Modify: `crates/ailang-surface/src/lex.rs:75` (enum variant)
+- Modify: `crates/ailang-surface/src/lex.rs:184` (guard)
+- Test: `crates/ailang-surface/src/lex.rs:~509` (3 in-source tests)
+
+- [ ] **Step 1: Add the `ReservedDollar` variant to `LexError`**
+
+In `crates/ailang-surface/src/lex.rs`, the `LexError` enum currently
+ends:
+
+```rust
+    #[error("invalid escape sequence \\{ch} in string literal at byte {pos}")]
+    InvalidEscape { ch: char, pos: usize },
+}
+```
+
+Add the new variant immediately before the closing `}` (after
+`InvalidEscape`):
+
+```rust
+    #[error("invalid escape sequence \\{ch} in string literal at byte {pos}")]
+    InvalidEscape { ch: char, pos: usize },
+    #[error("reserved character `$` in token {token:?} at byte {start}; \
+             `$` is reserved for compiler-synthetic names")]
+    ReservedDollar { token: String, start: usize },
+}
+```
+
+- [ ] **Step 2: Write the failing test `dollar_in_ident_is_reserved`**
+
+In the `#[cfg(test)] mod tests` block at the bottom of
+`crates/ailang-surface/src/lex.rs` (closes at line 510), add this test
+before the module's closing `}`:
+
+```rust
+    #[test]
+    fn dollar_in_ident_is_reserved() {
+        let err = tokenize("x$1").expect_err("`$` in an ident must be a lex error");
+        assert!(
+            matches!(err, LexError::ReservedDollar { ref token, .. } if token == "x$1"),
+            "expected ReservedDollar(\"x$1\"), got {err:?}",
+        );
+    }
+```
+
+- [ ] **Step 3: Run the test to verify it fails**
+
+Run: `cargo test -p ailang-surface dollar_in_ident_is_reserved`
+Expected: FAIL — the test panics with
+`` `$` in an ident must be a lex error `` (because the guard does not
+exist yet, `tokenize("x$1")` returns `Ok`, so `expect_err` panics).
+
+- [ ] **Step 4: Add the `$`-reject guard inline in `tokenize`**
+
+In `crates/ailang-surface/src/lex.rs`, the run-classification currently
+reads (lines 183-186):
+
+```rust
+        let raw = &input[start..i];
+        // Classify.
+        let first = raw.as_bytes()[0];
+        let is_int = first.is_ascii_digit()
+```
+
+Insert the guard immediately after the `let raw` line, before the
+`// Classify.` comment:
+
+```rust
+        let raw = &input[start..i];
+        // raw-buf.#44: `$` is reserved for compiler-synthetic names
+        // (`$mp_N`, `<hint>$lr_N`, `<base>$<n>`). Reject any authored
+        // token run containing it, before int/float/ident classification.
+        if raw.contains('$') {
+            return Err(LexError::ReservedDollar { token: raw.to_string(), start });
+        }
+        // Classify.
+        let first = raw.as_bytes()[0];
+        let is_int = first.is_ascii_digit()
+```
+
+- [ ] **Step 5: Run the test to verify it passes**
+
+Run: `cargo test -p ailang-surface dollar_in_ident_is_reserved`
+Expected: PASS.
+
+- [ ] **Step 6: Add the two exemption tests**
+
+In the same `mod tests` block (before its closing `}`), add:
+
+```rust
+    #[test]
+    fn dollar_in_string_literal_is_allowed() {
+        // The `$` lives inside a string; it must NOT trip the reservation.
+        let toks = tokenize(r#""price: $5""#).expect("string-internal `$` is legal");
+        assert!(matches!(toks.as_slice(), [Token { tok: Tok::Str(s), .. }] if s == "price: $5"));
+    }
+
+    #[test]
+    fn dollar_in_comment_is_allowed() {
+        // Comment is stripped before tokenization; the ident `x` survives.
+        let toks = tokenize("x ; mentions loop$lr_0\n").expect("comment `$` is legal");
+        assert!(matches!(toks.as_slice(), [Token { tok: Tok::Ident(s), .. }] if s == "x"));
+    }
+```
+
+- [ ] **Step 7: Run the exemption tests + the full lexer suite**
+
+Run: `cargo test -p ailang-surface --lib`
+Expected: PASS — all lexer tests green, including the 19 pre-existing
+ones plus the 3 new (`dollar_in_ident_is_reserved`,
+`dollar_in_string_literal_is_allowed`, `dollar_in_comment_is_allowed`).
+Confirm the run reports 22 lexer tests, not 19 (the count proves the 3
+new tests were collected and ran).
+
+---
+
+## Task 2: Correct the `fresh_binder` doc-comment
+
+**Files:**
+- Modify: `crates/ailang-core/src/desugar.rs:528-536`
+
+- [ ] **Step 1: Replace the stale doc paragraph**
+
+In `crates/ailang-core/src/desugar.rs`, the `fresh_binder` doc-comment
+currently reads (lines 528-536):
+
+```rust
+    /// `$` is the project's convention for synthetic names, but it is
+    /// NOT lexer-enforced — an authored binder may legally contain `$`.
+    /// The probe rejects a `<base>$<n>` that is already in `used` or
+    /// currently in `scope`, but it does NOT see an authored
+    /// `<base>$<n>` that is out of scope at mint time yet later binds
+    /// under the same `(def, name)` key. No fixture exercises an
+    /// authored `$` binder; closing that latent gap (enforce the
+    /// reservation, or seed `used` with every authored binder name in
+    /// the def) is tracked as a follow-up.
+```
+
+Replace those 9 lines with:
+
+```rust
+    /// `$` is reserved for the compiler's synthetic namespace and is
+    /// rejected in any authored Form-A identifier by the lexer
+    /// (`ailang_surface::lex::tokenize` → `LexError::ReservedDollar`).
+    /// So a `<base>$<n>` candidate minted here cannot alias any
+    /// authored name — no authored name can contain `$`. The probe
+    /// against `used` and `scope` therefore guards only against prior
+    /// synthetic mints and currently in-scope renamed binders. (The
+    /// reservation is on *Form-A authored* identifiers; the `.ail.json`
+    /// deserialization path is intentionally not guarded — it is the
+    /// orchestrator's self-responsible channel, not client-reachable.)
+```
+
+- [ ] **Step 2: Verify `ailang-core` still builds**
+
+Run: `cargo build -p ailang-core`
+Expected: build succeeds (doc-comment-only change; 0 errors).
+
+---
+
+## Task 3: Integration test — the surfacing chain end-to-end
+
+**Files:**
+- Create: `crates/ailang-surface/tests/reserved_dollar_pin.rs`
+
+- [ ] **Step 1: Write the integration test**
+
+Create `crates/ailang-surface/tests/reserved_dollar_pin.rs` with:
+
+```rust
+//! #44: a `$` in an authored Form-A identifier is rejected at the
+//! lexer and surfaces through `parse` as `ParseError::Lex`. Pins that
+//! the reservation reaches the public `parse` entry point (not just
+//! the internal `tokenize`), so a hallucinating Form-A author cannot
+//! land a `$` binder in a `Module`.
+
+// `parse`, `ParseError`, `LexError` are all re-exported at the crate
+// root (`pub use` in lib.rs); the `lex` / `parse` modules themselves
+// are private, so import from the crate root, not the module paths.
+use ailang_surface::{parse, LexError, ParseError};
+
+#[test]
+fn dollar_binder_in_module_is_rejected_at_parse() {
+    // (module bad (fn main (type (fn-type (params) (ret (con Int)))) (params) (body (let x$1 7 x$1))))
+    let src = "(module bad\n  (fn main\n    (type (fn-type (params) (ret (con Int))))\n    (params)\n    (body\n      (let x$1 7 x$1))))\n";
+    let err = parse(src).expect_err("authored `$` binder must be rejected at parse");
+    assert!(
+        matches!(err, ParseError::Lex(LexError::ReservedDollar { ref token, .. }) if token == "x$1"),
+        "expected ParseError::Lex(ReservedDollar(\"x$1\")), got {err:?}",
+    );
+}
+
+#[test]
+fn dollar_in_string_in_module_parses_clean() {
+    // The exemption survives at the `parse` level too: `$` inside a
+    // string literal in a real module is fine.
+    let src = "(module dollar_ok\n  (fn main\n    (type (fn-type (params) (ret (con Str))))\n    (params)\n    (body \"price: $5\")))\n";
+    parse(src).expect("string-internal `$` must parse clean");
+}
+```
+
+- [ ] **Step 2: Confirm the imports resolve against the crate's public API**
+
+Run: `cargo build -p ailang-surface --tests`
+Expected: build succeeds (0 errors). The imports use the crate-root
+re-exports `ailang_surface::{parse, LexError, ParseError}` (confirmed
+present in `crates/ailang-surface/src/lib.rs:8-9`); the `lex` / `parse`
+modules are private, so the module-qualified paths would NOT compile.
+
+- [ ] **Step 3: Run the integration test**
+
+Run: `cargo test -p ailang-surface --test reserved_dollar_pin`
+Expected: PASS — both `dollar_binder_in_module_is_rejected_at_parse`
+and `dollar_in_string_in_module_parses_clean` green (2 tests ran).
+
+---
+
+## Task 4: No-regression sweep
+
+**Files:** none (verification only)
+
+- [ ] **Step 1: Full workspace test suite**
+
+Run: `cargo test --workspace`
+Expected: PASS — all crates green. In particular
+`crates/ailang-surface/tests/round_trip.rs`
+(`parse_then_print_then_parse_is_idempotent_on_every_ail_fixture`,
+`parse_is_deterministic_over_every_ail_fixture`) stays green: no
+`examples/*.ail` carries an authored `$` ident, and the six
+comment-only `$` files (e.g. `local_rec_let_capture.ail`) survive
+because comments are stripped before a run is sliced.
+
+- [ ] **Step 2: CLI check — comment-`$` example survives end-to-end**
+
+Run: `cargo run -q -p ail -- check examples/local_rec_let_capture.ail`
+Expected: exit 0 (`ok (… symbols …)`) — a real checked-in file whose
+comments mention `loop$lr_0` still checks clean, proving the guard
+does not reach comment text at the CLI level.
+
+- [ ] **Step 3: CLI check — the #43 shadowing idiom still compiles clean**
+
+Run: `cargo run -q -p ail -- build examples/raw_buf_int.ail -o /tmp/rawbuf_int_check`
+Expected: exit 0 — `raw_buf_int.ail` is the real nested-`let`-shadow
+idiom (`(let buf (new …) (let buf (app RawBuf.set buf …) …))`); its
+inner `buf` → `buf$1` rename is minted at desugar, after parse, and is
+never re-lexed, so the reservation does not regress it. (If `ail build`
+needs different flags in this tree, fall back to
+`cargo run -q -p ail -- check examples/raw_buf_int.ail` and assert
+exit 0 — check alone still exercises parse → desugar → typecheck over
+the shadow rename.)