Introduces a new AST emitter module responsible for serializing AST
nodes back into Myc source code. This emitter preserves leading comments
and whitespace, enabling a round-trip guarantee for the parser and
emitter.
The documentation in `docs/BNF.md` has been updated to reflect the new
syntax rules for comments, including single-line comments (`;`) and
documentation comments (`;;`). The rules clarify that comments must be
whole lines and that inline comments are disallowed.
The lexer and AST node structures have been updated to accommodate
`CommentLine` variants, allowing for different types of comments
(regular, documentation, and blank lines) to be stored and processed.
A new test suite `tests/comment_roundtrip.rs` has been added to verify
the round-trip property of the parser and emitter across all example
`.myc` files. This test ensures that parsing source code, emitting it,
and then parsing the emitted code again results in the same output
string.
The `ast.rs` binary now includes a `--emit` flag that uses the new
emitter to output the parsed source code, facilitating debugging and
testing of the emitter itself.
This commit introduces bidirectional type inference, enabling the
compiler to infer lambda parameter types based on the context of their
usage. This is particularly useful for functions like `pipe`, where a
lambda's signature can be deduced from the types of the streams it
operates on.
Key changes include:
- **`extract_lambda_param_hints` function:** This new helper function in
`type_checker.rs` is responsible for extracting expected parameter
types for a lambda based on the callee's signature and already known
argument types.
- **`check_lambda_with_param_hints` function:** This function allows
type-checking a lambda node using provided parameter type hints,
preserving upvalue types from the enclosing scope.
- **Call expression type checking:** The `check_call` function now
phases its argument type checking. Non-lambda arguments are typed
first, and then lambda arguments are typed using hints derived from
`extract_lambda_param_hints`.
- **`pipe_arg_hint_resolver`:** This new resolver for the `pipe`
function's `PolymorphicFn` type allows it to provide specific type
hints for its lambda argument based on the types of its stream inputs
(single stream, vector of streams, or tuple of streams).
- **`StaticType::PolymorphicFn` update:** The `PolymorphicFn` enum
variant has been extended to include `resolve_arg_hints`, enabling
functions to provide lambda parameter hints during bidirectional type
inference.
- **New tests:** Added tests to verify the correct inference of lambda
parameters for `pipe` with vector and tuple inputs, ensuring that
inner stream types are correctly propagated. Also, a test to confirm
`pipe` with an optional return still produces `Stream(Float)`.
The `bound_nodes.rs` file has been removed and its contents have been
moved to `src/ast/nodes.rs`. This consolidates all AST node definitions
into a single module, improving organization and maintainability.
The `compiler` modules now import these definitions from
`crate::ast::nodes` instead of `crate::ast::compiler::bound_nodes`.
This commit refactors the test suite by removing the monolithic
`integration_test.rs` file.
The tests have been categorized and moved into dedicated modules within
the `tests/` directory:
- `tests/parser.rs`: Contains tests for parsing various literal values.
- `tests/destructuring.rs`: Houses tests related to destructuring
syntax.
- `tests/closures.rs`: Includes tests for closure behavior and related
optimizations.
- `tests/optimizer.rs`: Contains tests specifically targeting optimizer
logic and regressions.
- `tests/records.rs`: Holds tests for record syntax and associated
optimizations.
- `tests/rtl.rs`: Contains tests for runtime library operators and
functions.
- `tests/pipeline.rs`: Tests for pipeline functionality.
- `tests/macros.rs`: Tests for macro expansion and related issues.
- `tests/error_recovery.rs`: Includes tests for compiler error handling
and recovery mechanisms.
- `tests/examples.rs`: Verifies the execution of example scripts.
This reorganization improves test discoverability and maintainability.
The `lib.rs` file has also been updated to reflect these changes by
removing the reference to the old `integration_test` module.
The `type_checker.rs` file has had some tests removed, as they are now
covered by the more specific tests in `tests/error_recovery.rs`.
Brummel