feat: Add AST emitter and comment support
Introduces a new AST emitter module responsible for serializing AST nodes back into Myc source code. This emitter preserves leading comments and whitespace, enabling a round-trip guarantee for the parser and emitter. The documentation in `docs/BNF.md` has been updated to reflect the new syntax rules for comments, including single-line comments (`;`) and documentation comments (`;;`). The rules clarify that comments must be whole lines and that inline comments are disallowed. The lexer and AST node structures have been updated to accommodate `CommentLine` variants, allowing for different types of comments (regular, documentation, and blank lines) to be stored and processed. A new test suite `tests/comment_roundtrip.rs` has been added to verify the round-trip property of the parser and emitter across all example `.myc` files. This test ensures that parsing source code, emitting it, and then parsing the emitted code again results in the same output string. The `ast.rs` binary now includes a `--emit` flag that uses the new emitter to output the parsed source code, facilitating debugging and testing of the emitter itself.
This commit is contained in:
@@ -0,0 +1,63 @@
|
||||
use myc::ast::compiler::emitter;
|
||||
use myc::ast::parser::Parser;
|
||||
use std::fs;
|
||||
|
||||
/// Verifies the round-trip property for all example scripts:
|
||||
///
|
||||
/// 1. Parse source → `ast1`
|
||||
/// 2. Emit `ast1` → `code1`
|
||||
/// 3. Parse `code1` → `ast2`
|
||||
/// 4. Emit `ast2` → `code2`
|
||||
/// 5. Assert `code1 == code2`
|
||||
///
|
||||
/// If the emitter is idempotent (a second pass produces the same output),
|
||||
/// the AST is structurally stable across round-trips.
|
||||
#[test]
|
||||
fn test_roundtrip_examples() {
|
||||
let entries = fs::read_dir("examples").expect("examples/ directory not found");
|
||||
|
||||
let mut tested = 0;
|
||||
for entry in entries.filter_map(|e| e.ok()) {
|
||||
let path = entry.path();
|
||||
if path.extension().is_none_or(|ext| ext != "myc") {
|
||||
continue;
|
||||
}
|
||||
|
||||
let name = path.file_name().unwrap().to_string_lossy().to_string();
|
||||
let source = fs::read_to_string(&path)
|
||||
.unwrap_or_else(|e| panic!("Failed to read {}: {}", name, e));
|
||||
|
||||
// Pass 1: parse original source, emit to code1
|
||||
let mut parser1 = Parser::new(&source);
|
||||
let ast1 = parser1.parse_expression();
|
||||
assert!(
|
||||
!parser1.diagnostics.has_errors(),
|
||||
"{}: parse errors on original source: {:?}",
|
||||
name,
|
||||
parser1.diagnostics.items
|
||||
);
|
||||
let code1 = emitter::emit(&ast1);
|
||||
|
||||
// Pass 2: parse emitted code1, emit to code2
|
||||
let mut parser2 = Parser::new(&code1);
|
||||
let ast2 = parser2.parse_expression();
|
||||
assert!(
|
||||
!parser2.diagnostics.has_errors(),
|
||||
"{}: parse errors on round-tripped code:\n---\n{}\n---\nErrors: {:?}",
|
||||
name,
|
||||
code1,
|
||||
parser2.diagnostics.items
|
||||
);
|
||||
let code2 = emitter::emit(&ast2);
|
||||
|
||||
assert_eq!(
|
||||
code1, code2,
|
||||
"{}: emitter is not idempotent (AST changed after second round-trip)",
|
||||
name
|
||||
);
|
||||
|
||||
tested += 1;
|
||||
}
|
||||
|
||||
assert!(tested > 0, "No .myc example files found in examples/");
|
||||
}
|
||||
Reference in New Issue
Block a user