- Use a `make-random` factory to create isolated random number
generators.
- Remove the global `prng` from the `Environment`.
- Ensure `make-random` can be called with or without a seed.
Implements the `create-ticker` function, which allows for the creation
of a ticker stream based on a provided condition closure. This function
is crucial for synchronizing pipeline execution based on specific events
or conditions.
Adds support for pipelines, allowing streams to be chained together and
transformed.
This includes new types for `PipeStream` and `SourceAdapter`, along with
modifications
to the `Environment` to manage pipeline execution. A new example
`pipeline.myc`
demonstrates its usage.
Introduces `StaticType::Optional` to represent values that can be either
of a specific type `T` or `Void`. This is crucial for handling
situations in filter pipes where an expression might not always produce
a value.
The type checker now correctly deduces and propagates this `Optional`
type, and the pipeline operator (`pipe`) is updated to unwrap
`Optional(T)` results, yielding `T`. This ensures that the type system
accurately reflects the potential absence of values in intermediate
pipeline steps.
Also includes a minor rename in the tuple-struct example from `pipe` to
`p` for clarity, and registers the `streams` module.
This commit introduces the `get_record` method to `RecordSeries`,
allowing for the dynamic reconstruction of a full record from its
constituent fields at a given index. This enables idiomatic access to
series data using indexing like `(my_ticks 0)`.
The `push-series` function has also been updated to correctly handle the
structure of records when pushing data into a `RecordSeries`.
Introduces a new RTL module for handling series data, enabling efficient
storage and retrieval of time-series data in a Struct-of-Arrays (SoA)
format.
Includes `SeriesView`, a zero-copy wrapper that provides a fast,
read-only interface to individual columns within a `RecordSeries`,
optimizing performance for data access in the VM.
Registers new native functions `create-series` and `push-series` for
managing series data.
This commit refactors the internal representation of `BoundKind::Record`
to store a `RecordLayout` and a `Vec` of values, rather than a `Vec` of
key-value pairs. This change simplifies the representation and improves
efficiency by decoupling the record's structure from its specific values
during compilation and analysis.
The `RecordLayout` now defines the structure of the record, and the
values are stored in a separate vector, ordered according to the layout.
This allows for better optimization and type checking, as the record's
shape is explicitly defined and immutable once created.
These changes update the benchmark numbers and repeat counts in the
example files. The benchmark results have slightly varied, and these
updates reflect the most recent measurements.
This commit updates the benchmark results for various examples. The
benchmark times and repeat counts have been adjusted to reflect current
performance characteristics.
This commit introduces a new prelude file that defines the `while` macro
and registers it during environment bootstrapping. The `again` macro has
been updated to accept multiple arguments for its recursive call, and
several examples have been adjusted to reflect this change.
Additionally, the `MacroRegistry` in the compiler is now cloneable and
can be moved out of the `MacroExpander`.
Add a new Markdown document detailing the AST node structure as it
evolves through the compiler's different stages. This includes a diagram
illustrating the data flow and relationships between the generic
`Node<K, T>` structure and its specialized forms (`UntypedNode`,
`BoundNode`, `TypedNode`, `AnalyzedNode`). The document explains the
`kind` and `ty` payloads for each stage and their significance.
Also removes a deleted file related to a previous optimization plan and
adds new files for future optimization plans.
Introduces a new `RecordLayout` system for efficient and type-safe
record handling.
Key changes:
- `RecordLayout` struct with `O(1)` field lookup using FMap
optimization.
- Field accessors (`.name`) are now first-class callable values.
- Parser recognizes dot-prefixed identifiers as field accessors.
- Binder and TypeChecker handle field accessors.
- Optimizer transforms field accessor calls into specialized `GetField`
nodes.
- VM executes `GetField` efficiently by directly accessing record
values.
- Adds a new `records.myc` example showcasing record features.
- Improves comparison logic for records to use pointer equality for
layout.
These tests were for record destructuring, which has been removed.
The code for record destructuring was also removed from the type checker
and types modules.
This commit introduces newtype wrappers for `LocalSlot`, `UpvalueIdx`,
and `GlobalIdx` to improve type safety and clarity. These wrappers
replace direct use of `u32` for indices, making the code more robust and
easier to understand.
The changes include:
- Defining `LocalSlot`, `UpvalueIdx`, and `GlobalIdx` structs.
- Implementing `Display` for these new types to provide user-friendly
output.
- Updating the `Address` enum to use these new types.
- Modifying various compiler components (analyzer, binder, optimizer,
type checker, environment, VM) to use the new types.
- Adjusting tests to accommodate the changes.
This commit consolidates `DefLocal` and `DefGlobal` into a single
`Define` bound kind. This simplifies the AST and makes it more
consistent.
It also introduces `DeclarationKind` to differentiate between variable
and parameter definitions.
The `UsageInfo` struct has been updated to include tracking for
`used_upvalues` and `assigned_upvalues`. The `collect_usage` function
has been modified to handle these new fields when encountering
`Address::Upvalue`.
The `map_params_to_args` function now takes `body_usage` as an argument
to ensure that parameters assigned to or used in the body are correctly
substituted. A new helper function, `collect_parameter_slots_set`, has
been added to gather all parameter slots within a pattern.
The inlining logic in `Optimizer::inline_call` has been enhanced to
prevent inlining if a parameter is used or assigned in the function body
but cannot be substituted. This addresses a bug where aggressive
inlining could lead to incorrect code generation when parameters were
reassigned.
A new integration test, `test_closure_reassignment_optimization_bug`,
has been added to specifically target and verify the fix for this
inlining issue.
Refine the type checker to correctly handle destructuring of various
collection types (tuples, vectors, matrices, lists, records).
Additionally, prevent implicit type coercion for function arguments,
ensuring that only tuples are passed to functions expecting tuple
arguments. This avoids unexpected behavior where vectors or matrices
might be treated as tuples.
Add a new example file demonstrating pattern matching and destructuring
rules.
The `flatten_tuple` function was unnecessarily recursive. It can now
simply return the elements of the tuple directly. The VM's destructuring
logic has been updated to handle nested destructuring more efficiently.
A new integration test case for multi-level destructuring has been
added.
This commit introduces optimizations for nested destructuring, allowing
tuples and records to be flattened and matched directly against function
arguments. This significantly improves performance by enabling more
constant folding and reducing intermediate allocations.
The changes include:
- Modifying the `Binder` to correctly count nested parameters.
- Enhancing `flatten_tuple` in the `Optimizer` to handle records and NOP
nodes.
- Updating `map_params_to_args` to recursively destructure nested
compound arguments.
- Adding integration tests to verify the correctness of tuple-to-tuple
and record-to-tuple destructuring optimizations.
This commit updates the benchmark numbers in various example files. The
changes reflect recent performance optimizations or adjustments to the
benchmarking environment.
This commit updates the benchmark metadata in several example files. The
benchmark runs and repeat counts have been slightly adjusted due to
optimizations or minor variations in execution.
This commit introduces purity inference to the compiler's optimizer.
This allows for more aggressive constant folding and dead code
elimination by tracking which functions and global variables
are free of side effects.
Key changes include:
- Added `global_purity` field to `Optimizer` and `Environment`.
- Modified `Optimizer::is_pure` to recursively determine if an
AST node represents a pure computation.
- Introduced `Optimizer::try_fold_pure` to replace the old
`try_fold_intrinsic`, enabling folding of pure function calls
with constant arguments.
- Updated `Environment::register_native` and
`Environment::register_constant`
to optionally record purity.
- Added purity flags to several built-in functions in `core.rs` and
`datetime.rs`.
- A new example `optimizer_purity.myc` demonstrates the new feature.
Add a new example file `examples/def_local_inlining.myc` to demonstrate
and test local inlining.
Refine the optimizer to handle local inlining more robustly by:
- Passing a `SubstitutionMap` to `visit_node` to track local variable
substitutions.
- Enabling constant propagation for local variables by checking
`sub.locals` in `BoundKind::Get`.
- Updating `BoundKind::DefLocal` and `BoundKind::Set` to maintain the
substitution map for local variables.
- Adjusting `try_beta_reduce` to use the optimizer's `visit_node` with
the substitution map for inlining.
- Re-indexing upvalues correctly when inlining captured variables into
lambdas.
Introduces a new optimizer pass that can "crack" closures, allowing for
more aggressive specialization. It also enables inlining of upvalues
that point to immutable global variables. This removes overhead for
higher-order functions and currying when arguments are statically
resolvable.
The benchmark results in several example files have been updated. This
commit reflects slight variations in performance measurements for
various test cases, including closure scope, data structures,
destructuring, higher-order functions, macros, and tuple operations.
This commit updates the benchmark timings and repeat counts for various
example files. The `tester.rs` script has been modified to:
- Correctly parse and use the `Benchmark-Repeat` directive.
- Implement an adaptive sampling mechanism for more accurate median
calculation, especially for long-running benchmarks.
- Improve the logic for updating and inserting benchmark and repeat
lines in example files.
- Handle potential compilation and runtime errors during benchmark
execution.
The `Value::List` and `Value::Record` variants have been consolidated
into a single `Value::Tuple` variant. This new variant uses a
`TupleData` struct to store values and an optional `Rc<Vec<Keyword>>`
for keys, allowing it to represent both ordered lists/tuples and
key-value records.
This change simplifies the internal representation and improves
performance by allowing schema sharing for records. It also includes
updates to the compiler, runtime, and tests to reflect the new
structure.
Replaces the use of `BTreeMap` and `HashMap` for maps and records with
`Vec<(Keyword, Value)>` and `Vec<(Keyword, StaticType)>`. This
simplifies the data structure and allows for ordered iteration, which is
important for serialization and comparison.
Also updates the `unpack` function in `vm.rs` to handle records as well
as lists when destructuring.
The type inference for collections (tuples, vectors, matrices, and
records) has been significantly refactored.
This includes:
- Introducing distinct `StaticType` variants for `Tuple`, `Vector`, and
`Matrix`.
- Implementing more robust type checking for these collections, allowing
for homogeneous and heterogeneous structures.
- Enhancing the `is_assignable_from` method to handle type compatibility
between these collection types.
- Updating `Value::static_type()` to correctly infer the types of
literal collections.
- Improving the type inference for map literals to create `Record`
types.
- Adding comprehensive unit tests for tuple, vector, matrix, and record
type inference.
This commit updates benchmark values in several example files to reflect
minor performance variations. It also includes adjustments to
integration
and unit tests to align with recent changes in the type checking and VM
logic, specifically concerning closures and TCO handling.
The `unless` macro's expansion was updated to use `...` (NOP) instead of
`void`. This change also involved removing the `void` keyword from the
parser, as it is no longer used.
Introduces a `Symbol` struct to represent identifiers, incorporating
macro hygiene context. Updates various parts of the AST compiler and
environment to use `Symbol` instead of raw `Rc<str>` for identifiers,
improving robustness for macro expansions.
Also includes:
- Adds a new example `macro_hygiene.myc`.
- Updates `.gitignore`.
- Refactors `MacroExpander` to use `ExpansionState` for cleaner template
expansion.
- Adjusts `VM::eval_observed` to ensure `after_eval` is called
correctly.
- Resets `Environment` for each test run and compilation/dumping in
`main.rs`.
This commit introduces support for AST macros and templates, enabling
users to define and use custom, reusable components within the visual
DSL.
Key changes include:
- A new `MacroRegistry` to manage macro definitions.
- A `MacroExpander` to process macro calls and expand templates.
- A `MacroEvaluator` trait for evaluating expressions during expansion.
- The `Expansion` node in `BoundKind` to preserve the original macro
call and its expanded form for debugging.
- Updates to the `Binder`, `Dumper`, and `UpvalueAnalyzer` to handle the
new macro constructs.
- New examples demonstrating various macro functionalities like
`unless`, splicing, and nested macros.
The `eval` and `eval_observed` methods in the `VM` struct were very
similar, with the primary difference being the call to the `VMObserver`
trait. This commit refactors these methods into a single macro,
`dispatch_eval!`, which reduces code duplication and improves
maintainability.
The `Value::TailCallRequest` variant was also updated to use `Box` for
its contents, which helps keep the `Value` enum size consistent.
Finally, the benchmarks in the `examples` directory have been updated to
reflect minor performance changes resulting from these code
modifications.
The `UpvalueAnalyzer` now correctly identifies which lambdas capture
which variable declarations, rather than just marking declarations that
need boxing. This information is stored in a `capture_map`.
The `Binder` has been updated to use this new `capture_map` instead of a
`HashSet` of boxed declarations. The `DefLocal` node in `bound_nodes.rs`
now stores a `captured_by` field, which is a list of lambda identities
that capture the local variable.
A new `dumper` module has been added to provide a human-readable
representation of the bound AST, including information about captured
variables.
The `VM` has been updated to use the `captured_by` field to determine if
a local variable needs to be stored in a `Cell`, rather than relying on
a boolean `is_boxed` flag.
An example script `extreme_capture.myc` has been added to test deep
nesting and variable capture.
A new "Dump AST" button has been added to the UI, which uses the new
`Dumper` to display the bound AST.
Michael Schimmel