This commit refactors the `Environment` struct to better manage the Rust
Runtime Library (RTL) bootstrap state.
A new `Rtl` struct is introduced to hold a frozen snapshot of the
initial RTL state, including scopes, types, purity, values, and
documentation. This snapshot is created once after `rtl::register()`
completes.
The `Environment::new()` constructor is updated to perform a two-phase
bootstrap: first, it registers RTL functions, and then it freezes the
scope 0 and captures the RTL state into the new `Rtl` struct.
A new constructor, `Environment::from_rtl()`, is added. This allows
creating independent execution environments that all share the same
underlying RTL state. This is crucial for isolation between different
execution contexts, as user definitions and loaded modules will not leak
between environments created from the same `Rtl` snapshot.
The `Environment::list_bindings` and `Environment::list_rtl_docs`
methods are updated to use the `rtl` field for accessing the RTL scope
and documentation, reinforcing the concept of a shared, immutable RTL
state.
A new test, `test_environments_from_shared_rtl_are_independent`, is
added to verify that environments created from the same `Rtl` snapshot
are indeed independent and do not leak user-defined bindings.
The `instantiate` method in `Environment` has been refactored to return
`Result<Rc<Closure>, String>` instead of `Rc<NativeFunction>`. This
change separates the creation of a `Closure` from its execution,
allowing the caller to manage the `VM` lifecycle and choose the
appropriate execution strategy (e.g., single run vs. repeated benchmark
iterations).
The `NativeFunction` type is now exclusively used for true Rust
intrinsics. The benchmark runner has also been updated to reuse a single
`VM` across iterations, improving performance by avoiding repeated VM
allocations.
This commit updates all example files to use `assert_eq!` for verifying
output, replacing the previous `Output:` comments. This change makes the
examples more robust and self-testing.
The benchmark results in some examples have also been slightly adjusted,
reflecting minor performance variations after the refactoring.
Additionally, a runtime panic handling mechanism has been introduced in
the VM for function calls, which improves error reporting for unexpected
panics.
This commit refactors several modules to use the defined types from
`ast::types` and `ast::nodes` directly, rather than using fully
qualified paths. This improves code readability and reduces redundancy.
Specifically, the following changes were made:
- In `analyzer.rs`, `crate::ast::types::Identity` and
`crate::ast::types::Purity` are now used directly.
- In `binder.rs`, `crate::ast::types::NodeIdentity`,
`crate::ast::types::SourceLocation`,
`crate::ast::types::RecordLayout`, and `crate::ast::types::Value` are
now used directly.
- In `macros.rs`, types like `Address`, `VirtualId`, `NodeIdentity`,
`SourceLocation`, `StaticType`, and `Purity` are now used directly.
- In `optimizer/engine.rs`, `Address<VirtualId>` is now used directly.
- In `type_checker.rs`, `BoundPhase`, `Signature`, `Keyword`, and
`RecordLayout` are now used directly.
- In `environment.rs`, `CompilerScope`, `LocalInfo`, `CapturePass`,
`AnalyzedPhase`, `NativeFunction`, and `Closure` are now used
directly.
- In `nodes.rs`, `RecordLayout`, `Keyword`, `Purity`, and `StaticType`
are now used directly.
- In `parser.rs`, `SourceLocation` and `NodeIdentity` are now used
directly.
- In `rtl/math.rs`, `NativeFunction`, `Purity`, `Signature`,
`StaticType`, `Value`, `RefCell`, and `Rc` are now used directly.
- In `rtl/streams.rs`, `ScalarValue`, `SeriesMember`, `Object`,
`PipeFn`, `Value`, `VM`, `RingBuffer`, `RecordSeries`, `SeriesView`,
`build_map_stream`, and `build_pipeline_node` are now used directly.
- In `rtl/type_registry.rs`, `RecordLayout`, `Keyword`, `Purity`,
`Signature`, `StaticType`, and `Value` are now used directly.
- In `vm.rs`, `RecordSeries`, `SeriesView`, `build_map_stream`,
`build_pipeline_node`, `StreamNode`, `Object`, and `PipeFn` are now
used directly.
- In `utils/tester.rs`, `TypedNode` is now used directly.
This commit replaces the `UntypedNode` enum with the more accurately
named `SyntaxNode`. This change is primarily for clarity and better
reflects the role of these nodes as representing the structure of the
source code prior to semantic analysis.
The corresponding enum `UntypedKind` has also been renamed to
`SyntaxKind` to maintain consistency.
No functional changes are introduced by this refactoring; it is purely a
renaming and organizational update.
The `system.myc` file, containing macros and core utilities, has been
removed as its functionality is now handled by an embedded string. This
simplifies the build process and eliminates a file that was no longer
strictly necessary.
Additionally, the `with_cwd()` method on the `Environment` has been
removed. The environment now automatically adds the current working
directory and its `rtl` subdirectory to the search paths during
initialization. This streamlines the setup for file-based usage and
ensures standard search paths are always considered.
The `#use` directive now supports referencing entire directories,
automatically including all `.myc` files within them. Additionally,
environments are now initialized with the current working directory as a
default search path, simplifying module resolution.
Introduces the `#use` preprocessor directive, allowing Myc scripts to
explicitly declare dependencies on other Myc libraries. This change
moves dependency management from a command-line argument to an in-script
declaration, ensuring scripts are self-contained and can correctly
resolve macros and other symbols.
Key features include:
- Declarations at the beginning of a file, evaluated before parsing.
- Support for relative paths and a new `->` separator.
- Automatic resolution of dependencies from specified search paths.
- Idempotent loading to prevent duplicate parsing and evaluation.
- No AST pollution; dependency management is a compile-time concern.
The compiler's lexer has been updated to recognize `#` as a comment
character, and the environment now manages search paths and loaded
modules. This lays the groundwork for more complex library structures
and improved code organization.
The Binder and its helper functions have been updated to accept and
propagate a `Diagnostics` struct. This allows for better error reporting
during the binding phase, enabling the compiler to continue processing
even after encountering errors and collect all issues before halting.
The `BoundKind::Error` node and `StaticType::Error` are introduced as
"poison" nodes/types. These nodes indicate that an error occurred during
compilation, preventing further valid processing of that specific AST
fragment but allowing the compiler to continue with other parts of the
code.
The `Parser` has also been updated to return a `Diagnostics` struct,
enabling it to report errors during the initial parsing stage while
still attempting to build a partial AST. This adheres to the same error
recovery strategy.
Introduces `register_native` for direct registration of `NativeFunction`
and `register_native_fn` for convenience from closures. The
`Environment::run`
method is removed, and its functionality is now handled by
`Environment::instantiate`,
which packages the linked AST into an invokable `NativeFunction`. This
streamlines
the execution path and better separates compilation/linking from runtime
execution.
The optimizer now uses a simple boolean flag (`optimization`) instead of
an `optimization_level` (u32). This simplifies the optimizer's logic and
makes it easier to enable or disable optimizations.
The command-line interface and internal testing have been updated to
reflect this change.
Add a new example file `examples/def_local_inlining.myc` to demonstrate
and test local inlining.
Refine the optimizer to handle local inlining more robustly by:
- Passing a `SubstitutionMap` to `visit_node` to track local variable
substitutions.
- Enabling constant propagation for local variables by checking
`sub.locals` in `BoundKind::Get`.
- Updating `BoundKind::DefLocal` and `BoundKind::Set` to maintain the
substitution map for local variables.
- Adjusting `try_beta_reduce` to use the optimizer's `visit_node` with
the substitution map for inlining.
- Re-indexing upvalues correctly when inlining captured variables into
lambdas.
The optimizer has been refactored to separate its phases and introduce
more aggressive collapsing capabilities.
Phase 2 (Cracking) now focuses on stateless transformations, making it
easier to reason about and potentially parallelize.
Phase 2.5 (Aggressive Collapsing) has been introduced, enabling
optimizations like:
- Beta-reduction for lambda literals.
- Cracking and inlining for constant closures.
- Folding intrinsics for constant arithmetic.
- Short-circuiting conditional expressions.
These changes aim to improve performance by reducing redundant
computations and code bloat. The maximum number of optimization passes
has also been increased to 5 to allow for more complex transformations.
Extracts the logic for running example tests into a new function
`run_functional_tests` in `src/utils/tester.rs`. This function is then
called from `src/integration_test.rs` to simplify the test setup and
improve code organization. The changes also adjust the benchmark
thresholds slightly.
This commit updates the benchmark timings and repeat counts for various
example files. The `tester.rs` script has been modified to:
- Correctly parse and use the `Benchmark-Repeat` directive.
- Implement an adaptive sampling mechanism for more accurate median
calculation, especially for long-running benchmarks.
- Improve the logic for updating and inserting benchmark and repeat
lines in example files.
- Handle potential compilation and runtime errors during benchmark
execution.
The `Value::List` and `Value::Record` variants have been consolidated
into a single `Value::Tuple` variant. This new variant uses a
`TupleData` struct to store values and an optional `Rc<Vec<Keyword>>`
for keys, allowing it to represent both ordered lists/tuples and
key-value records.
This change simplifies the internal representation and improves
performance by allowing schema sharing for records. It also includes
updates to the compiler, runtime, and tests to reflect the new
structure.
Introduce `BoundKind::Parameter` to represent function parameters.
Modify `Binder` to correctly handle parameters within lambda
definitions, ensuring they are only defined within function scopes.
Update `LambdaCollector` to register the body of lambdas as templates
for global definitions.
Adjust `Dumper` to accurately represent lambda parameters.
Update `Specializer` and `TCO` to handle the new `BoundKind::Parameter`.
Refactor `Call` and `TailCall` to use a single `args` node, often a
tuple.
Adjust type signatures in RTL to use `StaticType::Tuple` for function
parameters.
Brummel
Michael Schimmel