Refine the BNF for `series` to explicitly include the `lookback_limit`
and `schema`.
Introduce scope management within the `bind` function for `if/else`
branches to ensure correct context handling during compilation.
Add `Again` and `GetField` node kinds to the `Specializer`.
Improve lexer to ignore invisible characters within identifiers,
demonstrated by a new test case.
This commit introduces support for comments within the AST nodes.
Comments are now parsed by the lexer and stored within the `Token`
struct.
These comments are then propagated through various compiler phases,
including
the `Node` struct, ensuring they are preserved in the Abstract Syntax
Tree.
This change enhances the AST's ability to retain source code
information,
which can be valuable for debugging and analysis.
The `bound_nodes.rs` file has been removed and its contents have been
moved to `src/ast/nodes.rs`. This consolidates all AST node definitions
into a single module, improving organization and maintainability.
The `compiler` modules now import these definitions from
`crate::ast::nodes` instead of `crate::ast::compiler::bound_nodes`.
The Binder and related types have been refactored to use the new
`NodeKind` enum instead of the previous `BoundKind`. This commit updates
all references to use the new structure, ensuring consistency across the
compiler's AST representation.
Key changes include:
- Replacing `BoundKind` with `NodeKind` in the Binder's `bind` and
`visit` methods.
- Updating pattern matching and field access to reflect the new enum
variants (e.g., `NodeKind::Def` instead of `BoundKind::Define`).
- Adjusting identifier bindings to use the new `IdentifierBinding` enum.
- Reflecting changes in `DefBinding`, `AssignBinding`, and
`LambdaBinding` structures.
- Ensuring all newly created nodes use `NodeKind` and the appropriate
metadata.
Introduce generic `CompilerPhase` trait to unify different stages of the
bound AST. Rename `LocalSlot` to `VirtualId` and introduce `StackOffset`
for clearer distinction between compile-time and run-time addressing.
Update type aliases and implementations to reflect these changes.
This commit replaces the `UntypedNode` enum with the more accurately
named `SyntaxNode`. This change is primarily for clarity and better
reflects the role of these nodes as representing the structure of the
source code prior to semantic analysis.
The corresponding enum `UntypedKind` has also been renamed to
`SyntaxKind` to maintain consistency.
No functional changes are introduced by this refactoring; it is purely a
renaming and organizational update.
Simplified the AST architecture by removing the overly complex Node<K,
T> structure and replacing it with specialized types for
each phase:
- Introduced UntypedNode in nodes.rs for the Parser and Macro system,
eliminating redundant ty: () initializations.
- Defined a new generic Node<T> in bound_nodes.rs for all compiler
stages, where T represents the phase-specific metadata.
- Updated the Binder to act as the bridge between UntypedNode and
Node<()>.
- Simplified type aliases: TypedNode, AnalyzedNode, and ExecNode now
leverage the streamlined Node<T> structure.
- Updated Parser, Macros, Type-Checker, Optimizer, and VM to reflect
the architectural changes.
- Fixed import chains and resolved needless borrows via clippy.
This change improves type safety, reduces boilerplate code in the
parser, and makes the compiler stages more idiomatic and
readable.
The specializer now correctly identifies when a compiled value is a
scalar and avoids folding it into a constant node. Instead, it updates
the callee to the specialized version if it's an object. This ensures
that scalar folding does not interfere with the program's execution.
This commit refactors various AST node types to use `Rc` (Reference
Counting) instead of `Box`. This change is primarily driven by the need
for shared ownership and more efficient handling of potentially
recursive or shared data structures within the AST.
The main benefits of this change include:
- **Reduced cloning:** `Rc` allows multiple parts of the AST to refer to
the same node without needing to clone the entire node. This is
particularly beneficial for optimization passes where nodes might be
visited multiple times.
- **Enabling sharing:** It makes it easier to represent scenarios where
a single AST node might be a child of multiple parent nodes.
- **Performance improvements:** By avoiding unnecessary deep copies of
AST nodes, this change can lead to performance gains, especially
during complex compilation phases.
Introduces `GlobalFunctionRegistry` and `GlobalAnalyzedRegistry` type
aliases to clarify the usage of `HashMap`s for storing global functions
and their analyzed versions. This change also refactors the
`LambdaCollector` and `Optimizer` to use these new aliases, improving
code readability and maintainability.
This commit refactors the internal representation of `BoundKind::Record`
to store a `RecordLayout` and a `Vec` of values, rather than a `Vec` of
key-value pairs. This change simplifies the representation and improves
efficiency by decoupling the record's structure from its specific values
during compilation and analysis.
The `RecordLayout` now defines the structure of the record, and the
values are stored in a separate vector, ordered according to the layout.
This allows for better optimization and type checking, as the record's
shape is explicitly defined and immutable once created.
This commit consolidates `DefLocal` and `DefGlobal` into a single
`Define` bound kind. This simplifies the AST and makes it more
consistent.
It also introduces `DeclarationKind` to differentiate between variable
and parameter definitions.
The `flatten_tuple` function was unnecessarily recursive. It can now
simply return the elements of the tuple directly. The VM's destructuring
logic has been updated to handle nested destructuring more efficiently.
A new integration test case for multi-level destructuring has been
added.
The Analyzer has been refactored to decorate `TypedNode`s with their
purity and recursion status. This involves creating a new `AnalyzedNode`
type and a `NodeMetrics` struct to hold this information. The `Analyzer`
now returns an `AnalyzedNode` instead of a separate `Analysis` struct.
This change lays the groundwork for future optimizations and analysis
passes.
This commit introduces a new AST analysis pass that identifies function
purity and recursion. This information is then used by the optimizer and
specializer to make more informed decisions, particularly regarding
inlining.
The `Analyzer` struct and its associated `Analysis` struct are
responsible for traversing the AST and collecting this data.
Key changes include:
- A new `analyzer` module is added to `ast::compiler`.
- `Analyzer::analyze` performs a two-pass traversal to collect
global-to-lambda mappings and then analyze purity and recursion.
- The `Optimizer` and `Specializer` are updated to accept and utilize
the `Analysis` data.
- Recursion checks in `Optimizer` and `Specializer` are replaced with
checks against the pre-computed `Analysis.is_recursive` set.
- The `Environment` now stores and passes the `Analysis` results to the
compiler stages.
The `is_tail` field on the `BoundKind::Call` node was an intermediate
representation for tail-call optimization and is no longer needed as a
distinct field. The TCO pass now embeds this information directly into
the `RuntimeMetadata` of the `ExecNode`, which is the VM's internal AST.
This simplifies the `BoundKind::Call` structure and removes redundant
information.
The `TailCall` variant has been merged into `Call` by adding an
`is_tail` boolean field. This simplifies the AST by reducing the number
of node variants and makes it easier to handle tail call optimization.
The `tco.rs` module is responsible for setting this flag when a call
occurs in tail position.
Introduces a new optimizer pass that can "crack" closures, allowing for
more aggressive specialization. It also enables inlining of upvalues
that point to immutable global variables. This removes overhead for
higher-order functions and currying when arguments are statically
resolvable.
This commit renames `Map` to `Record` and updates all related AST nodes,
binders, type checkers, and runtime values to reflect this change. This
is a semantic change to better align with common programming language
terminology.
Introduce `BoundKind::Parameter` to represent function parameters.
Modify `Binder` to correctly handle parameters within lambda
definitions, ensuring they are only defined within function scopes.
Update `LambdaCollector` to register the body of lambdas as templates
for global definitions.
Adjust `Dumper` to accurately represent lambda parameters.
Update `Specializer` and `TCO` to handle the new `BoundKind::Parameter`.
Refactor `Call` and `TailCall` to use a single `args` node, often a
tuple.
Adjust type signatures in RTL to use `StaticType::Tuple` for function
parameters.
This commit updates benchmark values in several example files to reflect
minor performance variations. It also includes adjustments to
integration
and unit tests to align with recent changes in the type checking and VM
logic, specifically concerning closures and TCO handling.
correctly initialize the `TypeChecker` with argument types during
macro expansion.
- Refactor `Specializer::compile` to perform type checking with provided
arguments before specialization and to correctly extract the return
type.
- Enhance the `Dumper` to introspect and display specialized closure
bodies.
- Update `LambdaCollector` to use `BoundNode` consistently.
- Modify `TypeChecker` to accept and inject specialized argument types
for lambdas.
Introduces `LambdaCollector` to gather lambda functions and populate the
function registry. This enables the `Specializer` to work with
user-defined functions.
The `Environment` struct is updated to manage the `function_registry`
and `monomorph_cache`, which are essential for the specialization
process.
The `link` method in `Environment` now incorporates lambda collection
and node specialization before applying TCO optimization. This ensures
that lambdas are properly processed and specialized for potential
performance gains.
The `Specializer`'s `new` constructor has been modified to accept and
initialize the `MonoCache` through an `Rc<RefCell<MonoCache>>`. This
allows the cache to be shared across different specialized functions.
Also includes minor refactoring and type adjustments in `specializer.rs`
for better clarity and consistency.
The specialization logic has been refactored to include a call to
`rtl_lookup` for host functions. This allows the specializer to
recognize and optimize calls to built-in runtime functions by directly
returning their specialized values.
Additionally, the `_rtl_lookup` field in `Specializer` has been renamed
to `rtl_lookup` for clarity. The `visit_call` method has also been
renamed to `specialize_call_logic` to better reflect its purpose, and it
now correctly handles `TailCall` nodes by preserving them.
The `Get`, `DefLocal`, and `DefGlobal` bound kinds now store the symbol
name associated with the variable. This information is useful for
debugging and provides more context in the compiled output.
Update the testing section to clarify when warnings should be eliminated
and tests run.
Add the new `specializer` module to the compiler's public API.
Add the `type_registry` module to the `rtl` module's public API.
Brummel
Michael Schimmel