feat: Add AST emitter and comment support

Introduces a new AST emitter module responsible for serializing AST
nodes back into Myc source code. This emitter preserves leading comments
and whitespace, enabling a round-trip guarantee for the parser and
emitter.

The documentation in `docs/BNF.md` has been updated to reflect the new
syntax rules for comments, including single-line comments (`;`) and
documentation comments (`;;`). The rules clarify that comments must be
whole lines and that inline comments are disallowed.

The lexer and AST node structures have been updated to accommodate
`CommentLine` variants, allowing for different types of comments
(regular, documentation, and blank lines) to be stored and processed.

A new test suite `tests/comment_roundtrip.rs` has been added to verify
the round-trip property of the parser and emitter across all example
`.myc` files. This test ensures that parsing source code, emitting it,
and then parsing the emitted code again results in the same output
string.

The `ast.rs` binary now includes a `--emit` flag that uses the new
emitter to output the parsed source code, facilitating debugging and
testing of the emitter itself.
This commit is contained in:
2026-03-25 19:15:40 +01:00
parent ee89d2330d
commit e757b453a4
10 changed files with 501 additions and 32 deletions
+36 -5
View File
@@ -66,6 +66,19 @@ This document defines the syntax (BNF) and semantics of the Myc language, a Lisp
```
*(Lexical Note: `<ident-start>` consists of alphabetic characters or `+-*/<=>!$%&_?.`. `<ident-char>` includes digits as well. Identifiers evaluating to a field accessor begin with a `.` followed by one or more valid identifier characters.)*
### Comments
```ebnf
<comment> ::= ";" " "? <line-text> <newline>
<doc-comment> ::= ";;" " "? <line-text> <newline>
```
**Rules:**
- Comments must be **whole lines** — they start at the beginning of a line. Inline comments (after code on the same line) are a compile error.
- Comment lines directly preceding an expression form a **comment block** attached to that expression.
- Blank lines between comment lines are preserved as separators within the block. The first and last line of a block must not be blank.
- `;;` (doc comment) before a `def` or `macro` registers the text in the symbol documentation registry (available via `--dump-docs`). Before other expressions it is allowed but has no registry effect.
## 2. Core Semantics & Data Types
- **Integers and Floats:** Standard 64-bit numeric types.
@@ -138,7 +151,8 @@ The Myc runtime environment provides a collection of built-in functions and macr
```clojure
(do
(def user {:id 101 :name "Alice" :role :admin})
(def role (.role user)) ; Evaluates to :admin
; Evaluates to :admin
(def role (.role user))
)
```
@@ -156,7 +170,8 @@ The Myc runtime environment provides a collection of built-in functions and macr
(def prices (.price my_ticks))
;; Series indexing: 0 is the newest (15.5), 1 is the previous (11.2)
(+ (prices 0) (prices 1)) ;; Output: 26.7
;; Output: 26.7
(+ (prices 0) (prices 1))
)
```
@@ -223,6 +238,19 @@ The `check_syntax` tool uses the single-expression compiler pass. Wrap multiple
(do (def x 1) (+ x 2))
```
### Inline comments are not allowed
Comments must be **whole lines** starting with `;` or `;;`. Placing a comment after code on the same line is a compile error.
```clojure
;; WRONG — inline comment causes a compile error:
(def x 42) ; this is x
;; CORRECT — comment on its own line before the expression:
; this is x
(def x 42)
```
### Field accessors are functions, not property syntax
```clojure
@@ -247,7 +275,8 @@ user.name
acc
(again (- n 1) (* acc n)))))
(factorial 10 1) ;; Output: 3628800
;; Output: 3628800
(factorial 10 1)
)
```
@@ -258,7 +287,8 @@ user.name
(macro unless [c body]
`(if ~c ... ~body))
(unless false "executed") ;; Output: "executed"
;; Output: "executed"
(unless false "executed")
)
```
@@ -276,6 +306,7 @@ user.name
;; Read back the two most recent prices
(def p (.price ticks))
(+ (p 0) (p 1)) ;; newest + second-newest
;; newest + second-newest
(+ (p 0) (p 1))
)
```