832375f2ac
All 176 files in the four accumulating directories now use a zero-padded 4-digit counter prefix that reflects creation order (`NNNN-slug.md`). The counter is assigned per directory in strict git-log creation order; ties broken alphabetically by original name. The old `YYYY-MM-DD-` prefix on docs/specs/ and docs/plans/ files is dropped — the date is recoverable from git log and the counter carries the ordering. A file's counter is stable for the life of the file: never reassigned, never reused, never compacted. Deleted files retire their counter; subsequent files do not fill the gap. This is the property that lets cross-references stay literal — refs use the full filename including the counter (`design/contracts/0007-honesty-rule.md`) so they grep cleanly and resolve directly without a glob step. 313 cross-references updated across .md/.rs/.toml/.c/.json files (test pins, include_str! paths, design-INDEX entries, baseline notes, runtime C comments, inter-contract markdown links incl. bare basename and `../models/foo.md` forms). CLAUDE.md gets a new "File-naming convention" section spelling out the rule and rationale. skills/brainstorm/SKILL.md and skills/planner/SKILL.md updated so new spec/plan creation produces counter-prefixed names from the start. The full test suite (cargo test --workspace) passes.
794 lines
29 KiB
Markdown
794 lines
29 KiB
Markdown
# Floats Iteration 2 — Surface Layer (Implementation Plan)
|
|
|
|
> **Parent spec:** `docs/specs/0005-floats.md` (committed
|
|
> `e37366f`, approved 2026-05-10) — sections A2 and the
|
|
> `crates/ailang-surface/` Components subsection.
|
|
>
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: use `skills/implement`
|
|
> to run this plan. Steps use `- [ ]` checkboxes for tracking.
|
|
|
|
**Goal:** Lex, parse, and print Float literals in surface form — `1.5`,
|
|
`1.5e3`, `1e10`, `-1.5e-3`. The two `unimplemented!("Floats milestone
|
|
iter 2: surface print")` arms left by iter 1 are replaced with a
|
|
shortest-round-trippable decimal printer that always emits `.` or
|
|
`e`/`E` so the output unambiguously re-lexes as `Tok::Float`. The
|
|
round-trip property `lex(print(L)) == L` becomes a regression test.
|
|
|
|
**Architecture:** A new `Tok::Float(u64)` carries the IEEE-754
|
|
binary64 bit pattern from lexer to parser. The lexer's existing
|
|
digit-lead arm gets a sibling float path: when the raw token contains
|
|
`.` or `e`/`E`, validate against the spec A2 grammar (`<digits>.<digits>
|
|
(e[+-]?<digits>)?` OR `<digits>e[+-]?<digits>`; bare leading/trailing
|
|
dot rejected; hex floats naturally rejected by `f64::from_str`),
|
|
parse via `f64::from_str`, store as `Tok::Float(f64::to_bits)`. The
|
|
parser wraps `Tok::Float(bits)` as `Term::Lit { lit: Literal::Float
|
|
{ bits } }` everywhere a literal is allowed (including `pat-lit` —
|
|
typecheck-level rejection for Float patterns is iter 3's job, not the
|
|
parser's). The printer reconstructs `f64::from_bits(bits)`, formats
|
|
via `f64::to_string()` (shortest round-trippable), and appends `.0` if
|
|
neither `.` nor `e`/`E` is in the rendered string. NaN / ±Inf bits
|
|
panic per spec — the surface lexer cannot produce them, so they
|
|
cannot reach the printer through any well-formed surface→AST path.
|
|
|
|
**Tech Stack:** `crates/ailang-surface/` — `lex.rs`, `parse.rs`,
|
|
`print.rs`. No other crate touched (the iter-1 `unimplemented!` arms
|
|
in `ailang-codegen`, `ailang-prose`, etc. remain — those are iters
|
|
4 and 5 respectively).
|
|
|
|
---
|
|
|
|
## Files this plan creates or modifies
|
|
|
|
- Modify: `crates/ailang-surface/src/lex.rs` — add `Tok::Float(u64)`
|
|
variant and `LexError::InvalidFloat { literal, start }` variant;
|
|
add the `looks_like_float` private helper; extend the digit-lead
|
|
classification arm at lines 175-196 to dispatch Int vs Float on
|
|
presence of `.` or `e`/`E`; extend the `mod tests` block with
|
|
positive (1.5, 1e10, -1.5, etc.) and rejection (`5.`, `1.5e`,
|
|
`1..5`, `1.5e+`) cases.
|
|
- Modify: `crates/ailang-surface/src/parse.rs` — extend the
|
|
`tok_label` exhaustive match at line 161 with a `Tok::Float` arm;
|
|
extend `parse_term` (around line 1232) to wrap `Tok::Float(bits)`
|
|
as `Term::Lit { lit: Literal::Float { bits } }`; extend
|
|
`parse_pat_lit` (around line 1626) with the same wrap, and update
|
|
its error message to mention "float". Add a parse-level unit test.
|
|
- Modify: `crates/ailang-surface/src/print.rs` — add
|
|
`write_float_lit(out, bits)` private helper that handles the
|
|
shortest-round-trippable decimal output and the `.0` suffix
|
|
guarantee, plus the panic on non-finite bits; replace both
|
|
`unimplemented!("Floats milestone iter 2: surface print")` arms
|
|
(one at `write_lit` line 567, one inside `write_pattern`'s
|
|
`Pattern::Lit` arm around line 588) with calls to the helper. Add
|
|
the `lex(print(L)) == L` round-trip property test using the
|
|
existing `round_trip(Module, &str)` test infrastructure.
|
|
- Modify: `docs/JOURNAL.md` — append iteration-close entry.
|
|
|
|
---
|
|
|
|
## Task 1: Lex — `Tok::Float` + `LexError::InvalidFloat` + spec-A2 grammar validation
|
|
|
|
**Files:**
|
|
- Modify: `crates/ailang-surface/src/lex.rs:34-47` (Tok enum),
|
|
`lex.rs:56-65` (LexError enum), `lex.rs:172-196` (digit-lead
|
|
classification arm), `lex.rs:202-264` (tests mod).
|
|
|
|
- [ ] **Step 1: Write the RED lex test for `1.5`**
|
|
|
|
In `crates/ailang-surface/src/lex.rs`, inside `mod tests`, append:
|
|
|
|
```rust
|
|
/// Iter 22-floats.2 RED: a `<digits>.<digits>` token lexes as
|
|
/// `Tok::Float` carrying the IEEE-754 binary64 bit pattern of
|
|
/// `f64::from_str(raw).unwrap().to_bits()`. Pinned literal:
|
|
/// `1.5_f64` → bit pattern `0x3FF8000000000000`.
|
|
#[test]
|
|
fn float_basic_decimal() {
|
|
let toks = tokenize("1.5").unwrap();
|
|
assert_eq!(toks.len(), 1);
|
|
assert!(matches!(toks[0].tok, Tok::Float(0x3ff8_0000_0000_0000)));
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify it fails (RED)**
|
|
|
|
Run: `cargo test -p ailang-surface --lib float_basic_decimal`
|
|
Expected: COMPILE FAIL with rustc error E0599 / E0432 naming
|
|
`Tok::Float` (the variant does not yet exist).
|
|
|
|
- [ ] **Step 3: Add `Tok::Float(u64)` variant**
|
|
|
|
In `crates/ailang-surface/src/lex.rs`, replace the `Tok` enum
|
|
declaration at lines 34-47:
|
|
|
|
```rust
|
|
/// One lexical token.
|
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
pub enum Tok {
|
|
/// `(`
|
|
LParen,
|
|
/// `)`
|
|
RParen,
|
|
/// Integer literal. Original textual form preserved for error messages;
|
|
/// parsed `i64` value carried for direct use.
|
|
Int(i64),
|
|
/// IEEE-754 binary64 literal (Floats milestone iter 2). The
|
|
/// payload is the bit pattern produced by `f64::to_bits`. The
|
|
/// surface forms recognised by the lexer are
|
|
/// `<digits>.<digits>(e[+-]?<digits>)?` and
|
|
/// `<digits>e[+-]?<digits>` (per spec A2). Hex floats and bare
|
|
/// leading/trailing dots are not recognised.
|
|
Float(u64),
|
|
/// String literal contents (after escape processing).
|
|
Str(String),
|
|
/// Identifier. Anything that isn't a paren, integer, or string.
|
|
Ident(String),
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Add `LexError::InvalidFloat { literal, start }` variant**
|
|
|
|
In `crates/ailang-surface/src/lex.rs`, replace the `LexError` enum
|
|
declaration at lines 56-65:
|
|
|
|
```rust
|
|
/// Lexer error.
|
|
#[derive(Debug, Error)]
|
|
pub enum LexError {
|
|
#[error("unterminated string literal at byte {start}")]
|
|
UnterminatedString { start: usize },
|
|
#[error("invalid integer literal {literal:?} at byte {start}")]
|
|
InvalidInteger { literal: String, start: usize },
|
|
#[error("invalid float literal {literal:?} at byte {start}")]
|
|
InvalidFloat { literal: String, start: usize },
|
|
#[error("invalid escape sequence \\{ch} in string literal at byte {pos}")]
|
|
InvalidEscape { ch: char, pos: usize },
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Add the `looks_like_float` private helper**
|
|
|
|
In `crates/ailang-surface/src/lex.rs`, immediately AFTER the
|
|
`tokenize` function (after line 199, before the `#[cfg(test)] mod tests`
|
|
block), add:
|
|
|
|
```rust
|
|
/// Spec A2 grammar validator for a digit-lead token that already
|
|
/// contains `.` or `e`/`E`. Returns `true` iff `raw` matches one of
|
|
/// - `[-]?<digits>.<digits>`
|
|
/// - `[-]?<digits>.<digits>(e|E)[+-]?<digits>`
|
|
/// - `[-]?<digits>(e|E)[+-]?<digits>`
|
|
/// and contains at least one of `.`, `e`, `E` (otherwise the caller
|
|
/// would dispatch to the integer path). Rejects bare leading/trailing
|
|
/// dots (`5.`, `5.e10`), missing exponent digits (`1.5e`), and any
|
|
/// trailing junk.
|
|
///
|
|
/// Hex floats (`0x1.8p4`) are naturally rejected because the
|
|
/// digit-lead caller arm only runs for tokens whose first digit
|
|
/// character is decimal; an `x` mid-token would fail validation here.
|
|
fn looks_like_float(raw: &str) -> bool {
|
|
let body = raw.strip_prefix('-').unwrap_or(raw);
|
|
let bytes = body.as_bytes();
|
|
if bytes.is_empty() || !bytes[0].is_ascii_digit() {
|
|
return false;
|
|
}
|
|
let mut i = 0;
|
|
while i < bytes.len() && bytes[i].is_ascii_digit() {
|
|
i += 1;
|
|
}
|
|
let mut saw_dot = false;
|
|
if i < bytes.len() && bytes[i] == b'.' {
|
|
saw_dot = true;
|
|
i += 1;
|
|
let frac_start = i;
|
|
while i < bytes.len() && bytes[i].is_ascii_digit() {
|
|
i += 1;
|
|
}
|
|
if i == frac_start {
|
|
return false; // `5.` or `5.e10` — trailing dot
|
|
}
|
|
}
|
|
let mut saw_exp = false;
|
|
if i < bytes.len() && (bytes[i] == b'e' || bytes[i] == b'E') {
|
|
saw_exp = true;
|
|
i += 1;
|
|
if i < bytes.len() && (bytes[i] == b'+' || bytes[i] == b'-') {
|
|
i += 1;
|
|
}
|
|
let exp_start = i;
|
|
while i < bytes.len() && bytes[i].is_ascii_digit() {
|
|
i += 1;
|
|
}
|
|
if i == exp_start {
|
|
return false; // `1.5e` or `1.5e+`
|
|
}
|
|
}
|
|
i == bytes.len() && (saw_dot || saw_exp)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 6: Extend the digit-lead classification arm**
|
|
|
|
In `crates/ailang-surface/src/lex.rs`, replace the `if is_int { ... }
|
|
else { ... }` block at lines 177-196 with:
|
|
|
|
```rust
|
|
if is_int {
|
|
let has_dot = raw.contains('.');
|
|
let has_exp = raw.contains('e') || raw.contains('E');
|
|
if has_dot || has_exp {
|
|
if !looks_like_float(raw) {
|
|
return Err(LexError::InvalidFloat {
|
|
literal: raw.to_string(),
|
|
start,
|
|
});
|
|
}
|
|
match raw.parse::<f64>() {
|
|
Ok(f) => out.push(Token {
|
|
tok: Tok::Float(f.to_bits()),
|
|
span: Span { start, end: i },
|
|
}),
|
|
Err(_) => {
|
|
return Err(LexError::InvalidFloat {
|
|
literal: raw.to_string(),
|
|
start,
|
|
});
|
|
}
|
|
}
|
|
} else {
|
|
// Pure integer: the existing path.
|
|
match raw.parse::<i64>() {
|
|
Ok(v) => out.push(Token {
|
|
tok: Tok::Int(v),
|
|
span: Span { start, end: i },
|
|
}),
|
|
Err(_) => {
|
|
return Err(LexError::InvalidInteger {
|
|
literal: raw.to_string(),
|
|
start,
|
|
});
|
|
}
|
|
}
|
|
}
|
|
} else {
|
|
out.push(Token {
|
|
tok: Tok::Ident(raw.to_string()),
|
|
span: Span { start, end: i },
|
|
});
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 7: Verify the RED test now passes (GREEN)**
|
|
|
|
Run: `cargo test -p ailang-surface --lib float_basic_decimal`
|
|
Expected: PASS — exactly one test reported, GREEN.
|
|
|
|
- [ ] **Step 8: Add positive lex tests**
|
|
|
|
In `crates/ailang-surface/src/lex.rs`, inside `mod tests`, append:
|
|
|
|
```rust
|
|
/// Spec A2 grammar: scientific-notation forms with a fractional
|
|
/// part (`1.5e3`, `2.0e-10`).
|
|
#[test]
|
|
fn float_scientific_with_fraction() {
|
|
let toks = tokenize("1.5e3 2.0e-10").unwrap();
|
|
assert_eq!(toks.len(), 2);
|
|
let bits1500 = 1.5e3_f64.to_bits();
|
|
let bits_neg10 = 2.0e-10_f64.to_bits();
|
|
match toks[0].tok {
|
|
Tok::Float(b) => assert_eq!(b, bits1500),
|
|
ref other => panic!("expected Tok::Float, got {other:?}"),
|
|
}
|
|
match toks[1].tok {
|
|
Tok::Float(b) => assert_eq!(b, bits_neg10),
|
|
ref other => panic!("expected Tok::Float, got {other:?}"),
|
|
}
|
|
}
|
|
|
|
/// Spec A2 grammar: exponent-only forms (`1e10`, `3e-5`).
|
|
#[test]
|
|
fn float_pure_exponent() {
|
|
let toks = tokenize("1e10 3e-5").unwrap();
|
|
assert_eq!(toks.len(), 2);
|
|
match toks[0].tok {
|
|
Tok::Float(b) => assert_eq!(b, 1e10_f64.to_bits()),
|
|
ref other => panic!("expected Tok::Float, got {other:?}"),
|
|
}
|
|
match toks[1].tok {
|
|
Tok::Float(b) => assert_eq!(b, 3e-5_f64.to_bits()),
|
|
ref other => panic!("expected Tok::Float, got {other:?}"),
|
|
}
|
|
}
|
|
|
|
/// Negative float literals follow the existing negative-int rule
|
|
/// (`-` followed by a digit is part of the numeric token), so
|
|
/// `-1.5` and `-1.5e-3` lex as single Tok::Float values.
|
|
#[test]
|
|
fn float_negative() {
|
|
let toks = tokenize("-1.5 -1.5e-3").unwrap();
|
|
assert_eq!(toks.len(), 2);
|
|
match toks[0].tok {
|
|
Tok::Float(b) => assert_eq!(b, (-1.5_f64).to_bits()),
|
|
ref other => panic!("expected Tok::Float, got {other:?}"),
|
|
}
|
|
match toks[1].tok {
|
|
Tok::Float(b) => assert_eq!(b, (-1.5e-3_f64).to_bits()),
|
|
ref other => panic!("expected Tok::Float, got {other:?}"),
|
|
}
|
|
}
|
|
|
|
/// Edge case: `0.0` and `-0.0` lex as distinct Tok::Float bit
|
|
/// patterns (per spec A5: -0 and +0 are distinct at the bit level).
|
|
#[test]
|
|
fn float_signed_zero() {
|
|
let toks = tokenize("0.0 -0.0").unwrap();
|
|
assert_eq!(toks.len(), 2);
|
|
match toks[0].tok {
|
|
Tok::Float(b) => assert_eq!(b, 0u64),
|
|
ref other => panic!("expected Tok::Float, got {other:?}"),
|
|
}
|
|
match toks[1].tok {
|
|
Tok::Float(b) => assert_eq!(b, 0x8000_0000_0000_0000u64),
|
|
ref other => panic!("expected Tok::Float, got {other:?}"),
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 9: Add rejection tests**
|
|
|
|
In `crates/ailang-surface/src/lex.rs`, inside `mod tests`, append:
|
|
|
|
```rust
|
|
/// Spec A2 rejects bare trailing dot — `5.` is not a float literal.
|
|
#[test]
|
|
fn float_rejects_trailing_dot() {
|
|
let err = tokenize("5.").unwrap_err();
|
|
assert!(
|
|
matches!(err, LexError::InvalidFloat { ref literal, .. } if literal == "5."),
|
|
"expected InvalidFloat for `5.`, got {err:?}"
|
|
);
|
|
}
|
|
|
|
/// Spec A2 rejects missing fractional part before exponent — `5.e10`
|
|
/// has the `<digits>.` shape with no fraction digits.
|
|
#[test]
|
|
fn float_rejects_missing_fraction_before_exponent() {
|
|
let err = tokenize("5.e10").unwrap_err();
|
|
assert!(
|
|
matches!(err, LexError::InvalidFloat { ref literal, .. } if literal == "5.e10"),
|
|
"expected InvalidFloat for `5.e10`, got {err:?}"
|
|
);
|
|
}
|
|
|
|
/// Spec A2 rejects empty exponent — `1.5e` has no exponent digits.
|
|
#[test]
|
|
fn float_rejects_empty_exponent() {
|
|
let err = tokenize("1.5e").unwrap_err();
|
|
assert!(
|
|
matches!(err, LexError::InvalidFloat { ref literal, .. } if literal == "1.5e"),
|
|
"expected InvalidFloat for `1.5e`, got {err:?}"
|
|
);
|
|
}
|
|
|
|
/// Spec A2 rejects sign-only exponent — `1.5e+` has no exponent
|
|
/// digits after the sign.
|
|
#[test]
|
|
fn float_rejects_sign_only_exponent() {
|
|
let err = tokenize("1.5e+").unwrap_err();
|
|
assert!(
|
|
matches!(err, LexError::InvalidFloat { ref literal, .. } if literal == "1.5e+"),
|
|
"expected InvalidFloat for `1.5e+`, got {err:?}"
|
|
);
|
|
}
|
|
|
|
/// Spec A2 rejects double dot — `1..5`.
|
|
#[test]
|
|
fn float_rejects_double_dot() {
|
|
let err = tokenize("1..5").unwrap_err();
|
|
assert!(
|
|
matches!(err, LexError::InvalidFloat { ref literal, .. } if literal == "1..5"),
|
|
"expected InvalidFloat for `1..5`, got {err:?}"
|
|
);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 10: Run all lex tests**
|
|
|
|
Run: `cargo test -p ailang-surface --lib lex`
|
|
Expected: all lex tests GREEN — pre-existing 7 + 9 new = 16 tests pass, 0 fail.
|
|
|
|
- [ ] **Step 11: Verify full workspace still builds**
|
|
|
|
Run: `cargo build --workspace`
|
|
Expected: clean — adding `Tok::Float` to the `Tok` enum breaks
|
|
exhaustiveness in `parse.rs::tok_label` (line 161). This is expected
|
|
to fail compile in `ailang-surface`. Task 2 immediately fixes it.
|
|
|
|
If `cargo build --workspace` fails ONLY at `tok_label` in parse.rs,
|
|
that is the expected RED for Task 2 and you may commit Task 1
|
|
anyway (the lex.rs unit-tests use `--lib` and bypass parse.rs).
|
|
|
|
If `cargo build --workspace` fails anywhere else, raise NEEDS_CONTEXT.
|
|
|
|
- [ ] **Step 12: Commit**
|
|
|
|
```bash
|
|
git add crates/ailang-surface/src/lex.rs
|
|
git commit -m "floats iter 2.1: Tok::Float + LexError::InvalidFloat + spec-A2 grammar validator"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 2: Parser — accept `Tok::Float` in literal positions
|
|
|
|
**Files:**
|
|
- Modify: `crates/ailang-surface/src/parse.rs:161-167` (tok_label),
|
|
`parse.rs:1232-1234` (parse_term Tok::Int arm),
|
|
`parse.rs:1626-1628` (parse_pat_lit Tok::Int arm),
|
|
`parse.rs:1643` (error-message string in parse_pat_lit fallback),
|
|
plus a new test inside the existing `mod tests` block (or a new
|
|
`mod tests` block if there is none — verify by reading the bottom
|
|
of `parse.rs`).
|
|
|
|
- [ ] **Step 1: Write the RED parse test for `1.5`**
|
|
|
|
In `crates/ailang-surface/src/parse.rs`, inside its `mod tests`
|
|
block (use `Bash` `grep -n "^mod tests\|#\[cfg(test)\]" crates/ailang-surface/src/parse.rs`
|
|
to locate; if no `mod tests` exists, append one at the file's end),
|
|
add:
|
|
|
|
```rust
|
|
/// Iter 22-floats.2 RED: `1.5` parses as
|
|
/// `Term::Lit { lit: Literal::Float { bits: 0x3ff8_0000_0000_0000 } }`.
|
|
#[test]
|
|
fn parses_float_atom_term() {
|
|
let toks = crate::lex::tokenize("1.5").expect("lex 1.5");
|
|
let mut p = Parser::new(&toks);
|
|
let t = p.parse_term().expect("parse term");
|
|
match t {
|
|
Term::Lit { lit: Literal::Float { bits } } => {
|
|
assert_eq!(bits, 0x3ff8_0000_0000_0000u64);
|
|
}
|
|
other => panic!("expected Term::Lit::Float, got {other:?}"),
|
|
}
|
|
}
|
|
```
|
|
|
|
If `Parser::new` is private (i.e. no `pub fn new`) OR `parse_term` is
|
|
private, use the public `crate::parse::parse_term(input: &str)` /
|
|
`crate::parse::parse(input: &str)` style entry point instead — read
|
|
the top of `parse.rs` to find the right public fn. Do NOT make
|
|
internals public for the test.
|
|
|
|
- [ ] **Step 2: Run test to verify it fails (RED)**
|
|
|
|
Run: `cargo test -p ailang-surface --lib parses_float_atom_term`
|
|
Expected: COMPILE FAIL because `tok_label` at line 161 is exhaustive
|
|
over `Tok` and is now missing the `Tok::Float` arm. The error names
|
|
the missing variant. (Both `cargo build` and `cargo test` will fail
|
|
at the same `tok_label` site — this is the genuine RED for Task 2.)
|
|
|
|
- [ ] **Step 3: Extend `tok_label`**
|
|
|
|
In `crates/ailang-surface/src/parse.rs`, replace the `tok_label`
|
|
function at lines 160-168:
|
|
|
|
```rust
|
|
fn tok_label(t: &Tok) -> String {
|
|
match t {
|
|
Tok::LParen => "`(`".into(),
|
|
Tok::RParen => "`)`".into(),
|
|
Tok::Int(v) => format!("integer `{v}`"),
|
|
Tok::Float(bits) => format!("float `0x{bits:016x}`"),
|
|
Tok::Str(s) => format!("string {s:?}"),
|
|
Tok::Ident(s) => format!("ident `{s}`"),
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Add `Tok::Float` arm to `parse_term`**
|
|
|
|
In `crates/ailang-surface/src/parse.rs`, in the literal-dispatch
|
|
match block around lines 1232-1235, add a `Tok::Float` arm
|
|
immediately AFTER the existing `Tok::Int` arm:
|
|
|
|
```rust
|
|
Some(Token { tok: Tok::Int(v), .. }) => {
|
|
self.cur += 1;
|
|
Ok(Term::Lit { lit: Literal::Int { value: v } })
|
|
}
|
|
Some(Token { tok: Tok::Float(bits), .. }) => {
|
|
self.cur += 1;
|
|
Ok(Term::Lit { lit: Literal::Float { bits } })
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Add `Tok::Float` arm to `parse_pat_lit`**
|
|
|
|
In `crates/ailang-surface/src/parse.rs`, in `parse_pat_lit` around
|
|
lines 1625-1648, add a `Tok::Float` arm immediately AFTER the
|
|
existing `Tok::Int` arm:
|
|
|
|
```rust
|
|
Some(Token { tok: Tok::Int(v), .. }) => {
|
|
self.cur += 1;
|
|
Literal::Int { value: v }
|
|
}
|
|
Some(Token { tok: Tok::Float(bits), .. }) => {
|
|
self.cur += 1;
|
|
Literal::Float { bits }
|
|
}
|
|
```
|
|
|
|
The parser ACCEPTS `Tok::Float` here even though pattern-matching
|
|
on Float is semantically dubious (per spec line 723-735). The
|
|
typechecker will reject it in iter 3 with a clean diagnostic — the
|
|
parser's job is surface→AST translation, not semantics.
|
|
|
|
- [ ] **Step 6: Update the `parse_pat_lit` error message**
|
|
|
|
In `crates/ailang-surface/src/parse.rs`, replace the two occurrences
|
|
of the string `"literal form (integer, string, `true`, or `false`)"`
|
|
inside `parse_pat_lit` (currently at lines 1643 and 1651) with:
|
|
|
|
```
|
|
"literal form (integer, float, string, `true`, or `false`)"
|
|
```
|
|
|
|
Use `Edit` with `replace_all = true` for the file, or two separate
|
|
`Edit` calls if the surrounding context is needed for uniqueness.
|
|
|
|
- [ ] **Step 7: Verify the RED test now passes (GREEN)**
|
|
|
|
Run: `cargo test -p ailang-surface --lib parses_float_atom_term`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 8: Verify the workspace builds**
|
|
|
|
Run: `cargo build --workspace`
|
|
Expected: clean — `tok_label` exhaustiveness restored.
|
|
|
|
- [ ] **Step 9: Commit**
|
|
|
|
```bash
|
|
git add crates/ailang-surface/src/parse.rs
|
|
git commit -m "floats iter 2.2: parser accepts Tok::Float in term and pat-lit positions"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 3: Print — shortest round-trippable decimal + `lex(print(L)) == L` round-trip
|
|
|
|
**Files:**
|
|
- Modify: `crates/ailang-surface/src/print.rs:562-569` (write_lit
|
|
Literal::Float arm), `print.rs:579-590` (write_pattern Pattern::Lit
|
|
Literal::Float arm), and add a new helper `write_float_lit` plus a
|
|
new round-trip test.
|
|
|
|
- [ ] **Step 1: Write the RED round-trip test for several Float literals**
|
|
|
|
In `crates/ailang-surface/src/print.rs`, inside `mod tests` (after the
|
|
existing `print_then_parse_round_trip_*` tests around line 720),
|
|
append:
|
|
|
|
```rust
|
|
/// Iter 22-floats.2 RED: `lex(print(L)) == L` round-trip property
|
|
/// for Float literals. Six representative bit patterns:
|
|
/// `1.5`, `0.0`, `-0.0`, `10.0`, `-0.375`, `1e10`. The round-trip
|
|
/// uses the existing `round_trip(Module, &str)` helper, which
|
|
/// prints the module to surface form, re-parses it, and asserts
|
|
/// canonical bytes match. The Float arm in surface print MUST emit
|
|
/// at least one of `.` or `e`/`E` so the lexer recognises the
|
|
/// output as a Float (not an Int) — `f64::to_string` returns "1"
|
|
/// for `1.0_f64`, "10000000000" for `1e10_f64`, and the printer's
|
|
/// post-pass adds `.0` in those cases.
|
|
#[test]
|
|
fn print_then_parse_round_trip_float_literal() {
|
|
use ailang_core::ast::*;
|
|
for &bits in &[
|
|
0x3ff8_0000_0000_0000u64, // 1.5
|
|
0x0000_0000_0000_0000u64, // 0.0
|
|
0x8000_0000_0000_0000u64, // -0.0
|
|
0x4024_0000_0000_0000u64, // 10.0
|
|
0xbfd8_0000_0000_0000u64, // -0.375
|
|
0x4202_a05f_2000_0000u64, // 1e10
|
|
] {
|
|
let m = Module {
|
|
schema: ailang_core::SCHEMA.to_string(),
|
|
name: "M".into(),
|
|
imports: vec![],
|
|
defs: vec![Def::Const(ConstDef {
|
|
name: "k".into(),
|
|
ty: Type::float(),
|
|
value: Term::Lit { lit: Literal::Float { bits } },
|
|
doc: None,
|
|
})],
|
|
};
|
|
round_trip(m, &format!("float_literal_{bits:#018x}"));
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify it fails (RED)**
|
|
|
|
Run: `cargo test -p ailang-surface --lib print_then_parse_round_trip_float_literal`
|
|
Expected: FAIL with `panic` originating from
|
|
`crates/ailang-surface/src/print.rs:567` — the existing
|
|
`unimplemented!("Floats milestone iter 2: surface print")` arm
|
|
fires the moment the printer reaches the Float literal.
|
|
|
|
- [ ] **Step 3: Add the `write_float_lit` helper**
|
|
|
|
In `crates/ailang-surface/src/print.rs`, add a new private helper
|
|
function immediately BEFORE `write_lit` (around line 562). The
|
|
helper handles both the `f64::to_string` shortest-round-trippable
|
|
output and the `.0` suffix guarantee:
|
|
|
|
```rust
|
|
/// Render a Float literal's bit pattern as surface text. The output
|
|
/// is the shortest round-trippable decimal produced by
|
|
/// `f64::to_string` (Grisu3). If the rendered form contains
|
|
/// neither `.` nor `e`/`E` (`f64::to_string` returns "1" for
|
|
/// `1.0_f64`, "10000000000" for `1e10_f64`), append `.0` so the
|
|
/// lexer's `is_int`/`has_dot|has_exp` dispatch routes the token to
|
|
/// the Float path on re-lex. This preserves the
|
|
/// `lex(print(L)) == L` round-trip property pinned by
|
|
/// [`print_then_parse_round_trip_float_literal`].
|
|
///
|
|
/// Non-finite bit patterns (NaN, ±Inf) cannot be expressed in
|
|
/// surface form (per spec section A2 — the lexer rejects all such
|
|
/// inputs). A Float literal in the AST whose bits are non-finite
|
|
/// must therefore have arrived via Form-A directly. The printer's
|
|
/// job is surface output, not a Form-A escape hatch, so we panic
|
|
/// rather than emit a non-round-trippable token.
|
|
fn write_float_lit(out: &mut String, bits: u64) {
|
|
let f = f64::from_bits(bits);
|
|
if !f.is_finite() {
|
|
panic!(
|
|
"write_float_lit: non-finite Float literal {bits:#018x} \
|
|
cannot be expressed in surface form"
|
|
);
|
|
}
|
|
let s = f.to_string();
|
|
out.push_str(&s);
|
|
if !s.contains('.') && !s.contains('e') && !s.contains('E') {
|
|
out.push_str(".0");
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Replace the `unimplemented!` arm in `write_lit`**
|
|
|
|
In `crates/ailang-surface/src/print.rs`, in `write_lit` at line
|
|
562-569, replace the `Literal::Float { .. } => unimplemented!(...)`
|
|
arm with:
|
|
|
|
```rust
|
|
Literal::Float { bits } => write_float_lit(out, *bits),
|
|
```
|
|
|
|
- [ ] **Step 5: Replace the `unimplemented!` arm in `write_pattern`'s `Pattern::Lit` block**
|
|
|
|
In `crates/ailang-surface/src/print.rs`, in `write_pattern` around
|
|
line 579-590 (the inner match inside the `Pattern::Lit { lit }` arm),
|
|
replace the `Literal::Float { .. } => unimplemented!(...)` arm with:
|
|
|
|
```rust
|
|
Literal::Float { bits } => write_float_lit(out, *bits),
|
|
```
|
|
|
|
- [ ] **Step 6: Verify the RED test now passes (GREEN)**
|
|
|
|
Run: `cargo test -p ailang-surface --lib print_then_parse_round_trip_float_literal`
|
|
Expected: PASS — all 6 bit patterns round-trip cleanly (canonical
|
|
bytes pre-print match canonical bytes post-reparse).
|
|
|
|
- [ ] **Step 7: Verify no `unimplemented!` arms remain in print.rs**
|
|
|
|
Run: `grep -n "Floats milestone iter 2: surface print" crates/ailang-surface/src/print.rs`
|
|
Expected: no matches — both arms replaced.
|
|
|
|
- [ ] **Step 8: Verify full workspace tests still GREEN**
|
|
|
|
Run: `cargo test --workspace`
|
|
Expected: 0 failures across all crates. Iter 1's pre-existing tests
|
|
all stay GREEN; the new lex / parse / print tests all GREEN.
|
|
|
|
- [ ] **Step 9: Commit**
|
|
|
|
```bash
|
|
git add crates/ailang-surface/src/print.rs
|
|
git commit -m "floats iter 2.3: surface print emits shortest round-trippable decimal with .0 fallback"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 4: Iteration close — JOURNAL entry
|
|
|
|
**Files:**
|
|
- Modify: `docs/JOURNAL.md` — append entry at the file end under
|
|
today's date.
|
|
|
|
- [ ] **Step 1: Append the iteration-close entry**
|
|
|
|
In `docs/JOURNAL.md`, append a new section at the end of the file:
|
|
|
|
```markdown
|
|
### Floats milestone — iter 2: surface layer
|
|
|
|
Lexer recognises Float literals in form `<digits>.<digits>(e[+-]?<digits>)?`
|
|
and `<digits>e[+-]?<digits>` per spec A2; bare leading/trailing
|
|
dots, missing fraction-after-dot, and missing/sign-only exponents
|
|
all reject with `LexError::InvalidFloat { literal, start }`. Hex
|
|
floats are naturally rejected by `f64::from_str` (and by the
|
|
digit-lead caller arm). The new private helper `looks_like_float`
|
|
in `lex.rs` enforces the spec grammar before delegating the
|
|
bit-pattern conversion to `f64::from_str` + `f64::to_bits`.
|
|
|
|
The new `Tok::Float(u64)` is wired through the parser at two sites
|
|
— `parse_term` (atom literal position) and `parse_pat_lit` (inside
|
|
`(pat-lit ...)`). The parser ACCEPTS Float in pattern position
|
|
even though pattern-matching on Float is semantically dubious;
|
|
typecheck-level rejection is iter 3's job, so the parser-side
|
|
diagnostic stays a generic "literal form" error.
|
|
|
|
The two `unimplemented!("Floats milestone iter 2: surface print")`
|
|
arms left by iter 1 in `print.rs` are gone. The new `write_float_lit`
|
|
helper formats via `f64::to_string` (shortest round-trippable
|
|
Grisu3) and appends `.0` if the rendered form contains neither `.`
|
|
nor `e`/`E` — this preserves the `lex(print(L)) == L` round-trip
|
|
property, which is pinned by a new regression test over six
|
|
representative bit patterns (`1.5`, `0.0`, `-0.0`, `10.0`, `-0.375`,
|
|
`1e10`). Non-finite bits panic; surface lex cannot produce them, so
|
|
the only path that would feed NaN/Inf to the printer is a Form-A
|
|
direct construction — and the printer is not a Form-A escape hatch.
|
|
|
|
Per-task commits:
|
|
|
|
- `<sha>` floats iter 2.1: Tok::Float + LexError::InvalidFloat + spec-A2 grammar validator
|
|
- `<sha>` floats iter 2.2: parser accepts Tok::Float in term and pat-lit positions
|
|
- `<sha>` floats iter 2.3: surface print emits shortest round-trippable decimal with .0 fallback
|
|
|
|
Known debt deliberately deferred to later iterations:
|
|
|
|
- Typecheck rejection of `Pattern::Lit { lit: Literal::Float { .. } }`
|
|
per spec line 723-735 recommendation (a). The parser accepts
|
|
Float patterns; iter 3 surfaces the type error.
|
|
- Typecheck widening of `+`/`-`/`*`/`/`/`<`/`<=`/`>`/`>=` from
|
|
monomorphic Int to polymorphic-`{Int, Float}` — iter 3.
|
|
- Codegen Float lowering paths — iter 4.
|
|
- Prose round-trip for Float literals — iter 5.
|
|
- DESIGN.md §"Float semantics" subsection and the line-2033
|
|
"supported primitive types" list update — iter 5.
|
|
```
|
|
|
|
Replace each `<sha>` with the actual commit SHA at commit time.
|
|
|
|
- [ ] **Step 2: Commit**
|
|
|
|
```bash
|
|
git add docs/JOURNAL.md
|
|
git commit -m "floats iter 2: JOURNAL — iteration close (surface layer)"
|
|
```
|
|
|
|
---
|
|
|
|
## Iteration acceptance
|
|
|
|
- [ ] `cargo build --workspace` is clean.
|
|
- [ ] `cargo test --workspace` is GREEN — pre-existing tests stay
|
|
GREEN; the new lex unit tests, parse unit test, and print
|
|
round-trip test all PASS.
|
|
- [ ] No `unimplemented!("Floats milestone iter 2: surface print")`
|
|
arms remain in `crates/ailang-surface/src/print.rs`.
|
|
- [ ] `Tok::Float` and `LexError::InvalidFloat` are public in
|
|
`crates/ailang-surface/src/lex.rs`.
|
|
- [ ] Round-trip property `lex(print(L)) == L` holds for the six
|
|
pinned Float bit patterns.
|
|
- [ ] No file touched outside the four listed in the file map.
|
|
- [ ] JOURNAL entry committed with the actual per-task commit SHAs.
|
|
|
|
When all four tasks are committed and the acceptance checklist is
|
|
green, hand back to the orchestrator. Iteration 3 (Type + builtins
|
|
WIDEN) is the next dispatch.
|