The `unless` macro's expansion was updated to use `...` (NOP) instead of
`void`. This change also involved removing the `void` keyword from the
parser, as it is no longer used.
Boolean literals `true` and `false` are now parsed as identifiers,
matching the behavior of other constants like `NaN`. This change aligns
the AST representation of boolean literals with their runtime constants.
This commit introduces a new module `ast::rtl` to house the standard
runtime library functions for the language. It includes implementations
for arithmetic operators (+, -, *, /, //, %), logical operators (and,
or, xor, not), bitwise shifts (<<, >>), and comparison operators (=, <>,
<, >, <=, >=).
Additionally, a `date` function for parsing datetime strings has been
added. This enhances the language's capability to handle basic
mathematical and logical operations.
Add RTL functions for arithmetic and comparisons
This commit introduces the runtime library (RTL) functions for basic
arithmetic operations, logical/bitwise operations, and comparisons. It
also includes a `date` function for parsing datetime strings.
Integration tests have been updated to cover these new functionalities.
This commit introduces the `DateTime` type to the language, enabling
users to work with dates and times. It includes:
- A new `DateTime` variant in the `Value` and `StaticType` enums.
- A `date` function for parsing date strings into `DateTime` values.
- Overloads for `+` and `-` operators to support `DateTime` arithmetic.
- Comparison operators (`>`, `<`) for `DateTime` values.
- Corresponding unit tests for `DateTime` operations.
- Updates to the `gemini.md` documentation to reflect the new
functionality.
Add DateTime type and operations
Introduce a new `DateTime` type to the language, allowing for date and
time manipulation. This includes:
* Parsing dates and datetimes from strings.
* Performing arithmetic operations (addition and subtraction) between
`DateTime` and `Int` (representing milliseconds) and between two
`DateTime` values (resulting in an `Int` duration).
* Enabling comparison operations (`>`, `<`) between `DateTime` values.
This commit refactors the type checking logic for functions and
operators to improve type safety and expressiveness.
Key changes include:
- Introduced `StaticType::FunctionOverloads` to represent functions with
multiple possible signatures, enabling better handling of operator
overloading.
- Updated the `TypeChecker` to correctly infer function types and
resolve calls with overloaded functions.
- Modified the `Environment` to register standard library functions with
specific `FunctionOverloads` signatures, replacing the previous
`StaticType::Any` approach.
- Enhanced the `StaticType::resolve_call` method to intelligently match
argument types against expected parameters, including handling of
overloaded functions.
- Added new tests to verify the correctness of type inference for lambda
returns and operator overloading.
Introduces the `TypeChecker` struct and its associated `TypeContext` for
performing static type analysis on the abstract syntax tree.
This change includes:
- A new `type_checker.rs` module.
- Integration of `TypeChecker` into the `Environment::compile` method.
- Updates to `Environment` to manage global types.
- Renaming `bound_nodes.rs`'s `BoundNode` to `TypedNode` to reflect its
type-checked nature.
- Refinements in binder and macro expansion to accommodate type
information.
The Binder has been refactored to simplify its internal types and reduce
redundancy.
Key changes include:
- Removed StaticType from LocalInfo, as it's not strictly needed for the
binder.
- Simplified `define` in CompilerScope to not take a StaticType.
- Removed StaticType from upvalues in FunctionCompiler.
- Adjusted global mappings to store only the index, not the type.
- Updated BoundKind to use a generic type parameter `T` for metadata,
defaulting to `()` for untyped bound nodes.
- Introduced `BoundNode` and `TypedNode` type aliases for clarity.
- Minor adjustments to dumper and VM to accommodate the new generic
BoundKind.
The UI layout has been significantly refactored to use
`egui::TopBottomPanel` and `egui::SidePanel` for a more organized
structure. A new menu bar has been introduced with File, Build, and
Tools menus, along with a toolbar for quick access to common actions.
Keyboard shortcuts have been updated and consolidated:
- `Shift + Enter` now compiles the code and is accessible from anywhere
in the app.
- `Ctrl + S` saves the current file.
The `Save As` functionality has been moved into a dedicated group within
the bottom panel and is now triggered by a button in the File menu. The
output log and trace logs now use `egui::ScrollArea::both` for better
scrolling experience.
Minor adjustments to default panel heights and scroll area behaviors
have been made for improved usability.
This commit replaces the `lazy_static` crate with `std::sync::OnceLock`.
This removes an external dependency and utilizes a standard library
feature for lazy initialization.
Additionally, a benchmark regression test has been added to
`src/ast/tester.rs`.
Introduces a `Symbol` struct to represent identifiers, incorporating
macro hygiene context. Updates various parts of the AST compiler and
environment to use `Symbol` instead of raw `Rc<str>` for identifiers,
improving robustness for macro expansions.
Also includes:
- Adds a new example `macro_hygiene.myc`.
- Updates `.gitignore`.
- Refactors `MacroExpander` to use `ExpansionState` for cleaner template
expansion.
- Adjusts `VM::eval_observed` to ensure `after_eval` is called
correctly.
- Resets `Environment` for each test run and compilation/dumping in
`main.rs`.
Ensure that variables are not redefined within the same scope, both for
global and function-local variables. This commit modifies the `define`
method in `CompilerScope` to return a `Result` and includes checks for
existing definitions. The `Binder`'s logic for handling global variables
and function parameters is also updated to incorporate these new checks
and return appropriate errors.
Truncate trace logs to improve GUI performance and prevent excessive
display. This change also introduces line length limits for individual
log entries, adding "..." for truncated lines. The display logic for
trace logs has been updated to use `egui::ScrollArea` for better user
experience.
This commit introduces support for AST macros and templates, enabling
users to define and use custom, reusable components within the visual
DSL.
Key changes include:
- A new `MacroRegistry` to manage macro definitions.
- A `MacroExpander` to process macro calls and expand templates.
- A `MacroEvaluator` trait for evaluating expressions during expansion.
- The `Expansion` node in `BoundKind` to preserve the original macro
call and its expanded form for debugging.
- Updates to the `Binder`, `Dumper`, and `UpvalueAnalyzer` to handle the
new macro constructs.
- New examples demonstrating various macro functionalities like
`unless`, splicing, and nested macros.
The `eval` and `eval_observed` methods in the `VM` struct were very
similar, with the primary difference being the call to the `VMObserver`
trait. This commit refactors these methods into a single macro,
`dispatch_eval!`, which reduces code duplication and improves
maintainability.
The `Value::TailCallRequest` variant was also updated to use `Box` for
its contents, which helps keep the `Value` enum size consistent.
Finally, the benchmarks in the `examples` directory have been updated to
reflect minor performance changes resulting from these code
modifications.
This commit introduces several improvements:
- Pre-compiles regular expressions in `ast/tester.rs` to avoid redundant
compilation, improving performance.
- Replaces `.extension().map_or(false, |ext| ext == "myc")` with the
more concise and readable `.extension().is_some_and(|ext| ext ==
"myc")`.
- Simplifies upvalue capture logic by using `.or_default()` instead of
`.or_insert_with(HashSet::new)`.
- Adds a `Default` implementation for `TracingObserver`.
- Optimizes string literal parsing in `highlight_myc` to correctly
handle escape sequences.
- Uses `saturating_sub` for bracket depth to prevent underflow.
Introduce a debug mode that collects and displays execution trace logs.
This involves adding an `AppTab` enum to manage UI state and modifying
the
`compile` function to handle debug execution.
Introduce a `debug_mode` flag to the `Environment` and a new `run_debug`
method. This method uses a `TracingObserver` to log the VM's execution
flow, including node evaluation and scope state changes. This allows for
detailed inspection of script execution.
The `BoundKind` enum now also includes a `display_name` method for
better log readability.
The `UpvalueAnalyzer` now correctly identifies which lambdas capture
which variable declarations, rather than just marking declarations that
need boxing. This information is stored in a `capture_map`.
The `Binder` has been updated to use this new `capture_map` instead of a
`HashSet` of boxed declarations. The `DefLocal` node in `bound_nodes.rs`
now stores a `captured_by` field, which is a list of lambda identities
that capture the local variable.
A new `dumper` module has been added to provide a human-readable
representation of the bound AST, including information about captured
variables.
The `VM` has been updated to use the `captured_by` field to determine if
a local variable needs to be stored in a `Cell`, rather than relying on
a boolean `is_boxed` flag.
An example script `extreme_capture.myc` has been added to test deep
nesting and variable capture.
A new "Dump AST" button has been added to the UI, which uses the new
`Dumper` to display the bound AST.
The binder now uses an `UpvalueAnalyzer` to identify local variables
that are captured by nested functions. These identified variables are
marked with `is_boxed: true` during `DefLocal` node creation. The VM
then uses this flag to wrap such variables in a `Value::Cell` (using
`Rc<RefCell<_>>`) to ensure they can be mutated across function calls.
feat: Introduce boxing for captured variables
Add UpvalueAnalyzer to identify variables captured by nested lambdas.
Modify Binder to use the analyzer and mark captured local variables for
boxing.
Update BoundKind::DefLocal to include an `is_boxed` flag.
Update VM to box captured variables when they are defined.
Add tests for upvalue capture detection.
Ensure that a script does not contain any tokens after the main
expression. This prevents accidental trailing expressions and enforces a
clearer script structure.
This commit introduces a new `tester` module to the `ast` crate,
enabling functional tests and performance benchmarks for MYC scripts.
Functional tests are executed by reading `.myc` files from the
`examples` directory, parsing expected output comments, and comparing
them against the actual script execution results.
Benchmarking involves measuring the execution time of scripts,
calculating median durations, and comparing them against stored
baselines. The commit also adds support for updating these baselines.
The `ast.rs` CLI and the `main.rs` GUI application have been updated to
expose these new testing and benchmarking features. The `Cargo.toml` and
`Cargo.lock` files have been updated to include the `regex` dependency
required for parsing benchmark comments.
This commit introduces the `chrono` dependency to the project, enabling
the use of time-related functionalities. Additionally, it adds several
example files (`closure.myc`, `fib.myc`, `hof.myc`) to showcase the
language's features like closures, recursion, and higher-order
functions.
The `Cargo.lock` and `Cargo.toml` files have been updated to reflect the
new dependency. The `integration_test.rs` file now includes a test to
run all `.myc` files found in the `examples` directory, ensuring their
correctness. The `main.rs` file has also been updated to load and
display these examples in the UI.
The project now uses the `clap` crate for handling command-line
arguments, replacing the previous manual parsing approach. This includes
adding `clap` as a dependency and creating a `Cli` struct to define the
command-line interface. The `main` function has been updated to parse
these arguments and handle file input or direct script evaluation.
This commit introduces two main changes:
1. **Binder Refactoring**: The `Binder` has been refactored to support
recursive definitions by pre-declaring names in the scope before
binding their values. This ensures that a name is visible to itself
when its definition is being processed.
2. **Stdlib Operator Implementation**: Several standard library
operators, including `+`, `-`, `*`, `/`, `>`, and `<`, have been
implemented. These operators now correctly handle both `Int` and
`Float` types for numerical operations and comparisons.
Additionally, the display formatting for `Value::List` and
`Value::Record` has been improved for better readability. The
default source code in `main.rs` has also been updated to include a
Fibonacci example, demonstrating the new recursive capabilities.
This commit introduces support for tuple and map literals in the AST.
Tuples are now represented by `UntypedKind::Tuple` and maps by
`UntypedKind::Map`.
The `Binder` has been updated to correctly handle these new node types
and infer their types.
The `VM` now also supports evaluating tuple and map literals.
This commit replaces `Arc<Mutex<T>>` with `Rc<RefCell<T>>` for managing
shared mutable state within the AST and VM. This change is primarily an
internal refactoring to leverage Rust's standard library more
effectively for single-threaded scenarios, improving performance by
avoiding the overhead of mutexes.
The following types and their usage have been updated:
- `Environment.global_names` and `Environment.global_values`
- `Binder.globals`
- `bound_nodes::BoundKind::Lambda.body` (now `Rc<Node>`)
- `ast::types::NodeIdentity` (now `Rc<NodeIdentity>`)
- `ast::types::Value::List`, `Value::Record`, `Value::Function`,
`Value::Object`, `Value::Cell`
- `ast::nodes::Scope` and `ast::nodes::Context`
- `ast::vm::Closure.function_node` and `Closure.upvalues`
- `ast::vm::VM.globals`
This change does not alter the external behavior of the library but
streamlines internal data management.
modify captured variables.
Implement `capture_upvalue` and `get_value`/`set_value` methods in the
`VM` to handle these mutable upvalues.
Refactor number parsing to support integers and floats
Separate the `TokenKind::Number` into `TokenKind::Integer` and
`TokenKind::Float`.
Update the lexer to distinguish between integer and float literals.
Update the parser to correctly map these new token kinds to their
respective `Value` types.
Add integration tests for parsing integers and floats, and for closure
modification of captured variables.
This commit introduces `StaticType` and enhances the `Binder` to perform
basic type checking during the binding phase.
Key changes include:
- **`StaticType` enum:** Represents static types such as `Any`, `Void`,
`Bool`, `Int`, `Float`, `Text`, `List`, `Record`, and `Function`.
- **`Node<K, T>`:** The generic `Node` now includes a `ty` field to
store its inferred static type.
- **Binder enhancements:**
- `CompilerScope` now stores `LocalInfo` containing the variable's
slot and `StaticType`.
- `FunctionCompiler` stores upvalues with their associated
`StaticType`.
- `Binder::bind` now returns `Node<BoundKind, StaticType>`,
propagating type information.
- Type checking is added for `If` conditions and `Assign`
operations.
- `Def` nodes now infer and store the type of the defined variable.
- **`Value::static_type()`:** A new method to determine the `StaticType`
of a `Value`.
- **Environment modifications:** `global_names` now stores `(u32,
StaticType)` to associate global variables with their types.
- **Parser modifications:** The `ty` field of nodes is initialized to
`()` by default.
- **VM modifications:** `VM::run` and `VM::eval` now operate on
`Node<BoundKind, StaticType>`.
- **Tests:** Added basic evaluation and type error tests.
feat: Add StaticType and type checking
Introduces `StaticType` enum and integrates it into the AST. The binder
now performs type checking during compilation, ensuring that expressions
conform to expected types, especially for control flow structures like
`if`. This lays the groundwork for static type analysis and error
reporting.
Introduces a new `is_first_frame` boolean flag to the `CompilerApp`
struct to manage initial UI state, specifically for requesting focus on
the source code editor on the first frame. This commit also moves the
compilation logic into a dedicated `compile` method for better
organization and introduces a Shift+Enter keyboard shortcut for
compilation.
Adds a new `Binder` struct that traverses the AST and resolves variable
references to concrete `Address` types (Local, Upvalue, Global). This
information is crucial for the Virtual Machine's execution phase.
Introduces the `BoundKind` enum to represent the AST after binding.
The `VM` is updated to handle `BoundKind` nodes and execute the bound
AST.
It now manages a call stack, local variables, and closures for function
calls and upvalue capturing.
The `Environment` struct is enhanced to manage global variables and
provide
a unified interface for parsing, binding, and executing scripts. It also
includes basic standard library functions.
Introduces the `Assign` AST node and a corresponding `assign` method on
the `Scope` struct. This allows for updating existing variables within
the current scope or any parent scope.
The `assign` method recursively traverses the scope chain until it finds
the variable or determines it's undefined. This enables reassigning
variables defined in outer scopes.
Introduces `UntypedKind::Lambda` to represent function closures.
Adds `parse_fn` to the parser to handle lambda syntax.
Updates `Node::eval` to correctly evaluate lambda expressions, creating
new scopes for each invocation with captured variables.
Improves handling of call expressions by passing the AST node's
identity.
Removes unnecessary comments and simplifies some error handling.
Introduces a `Scope` struct for dynamic scope management and a `Context`
struct to hold the current scope.
Implements a basic standard library with arithmetic operations and a
greater-than comparison.
Modifies `Node::eval` to handle identifiers by resolving them within the
current scope and `Call` nodes for function execution.
Updates the parser to use `Context::new()` for evaluating vector
elements, ensuring a fresh scope.
Enhances the main loop to handle potential runtime errors during
evaluation.
Introduces the lexer and parser modules, enabling the conversion of
source code into an Abstract Syntax Tree (AST).
This includes:
- Defining token kinds and structures.
- Implementing lexer logic to tokenize input.
- Defining AST node kinds and structures.
- Implementing parser logic to construct the AST from tokens.
- Adding support for basic expressions, lists, keywords, and
identifiers.
- Adding the `lazy_static` dependency for keyword interning.
- Refactoring `Value::Nil` to `Value::Void`.
Sets up the basic structure for the compiler GUI application using
eframe.
This includes the main application loop, a default UI layout with a
source code editor and an output log panel, and the foundational AST
(Abstract Syntax Tree) definitions for types, nodes, and an evaluation
mechanism.
Michael Schimmel