The `Parameter` kind in `UntypedKind` was only used for identifiers that
were being declared or referenced. This commit renames it to
`Identifier` and updates all the necessary code to reflect this change.
This simplifies the AST and makes it more consistent.
Additionally, a new macro `repeat` has been added to
`src/ast/system.myc`.
- Add `rtl/system.myc` as the system library, containing core utilities
like `while` and `cache`.
- Remove the duplicate `prelude.myc` from `src/ast/rtl/`.
- Modify `Environment::collect_dependencies` to implicitly add `#use
system` when necessary.
- Update `Environment::with_cwd` to also add the `rtl` subdirectory to
search paths.
- Add `target/` to `.geminiignore` to exclude build artifacts.
- Adjust example `sma.myc` to use the new `cache` macro and reduce
sample data size.
- Update `rtl/sma.myc` to correctly initialize the `history` series with
its `length`.
- Add `preload_dependencies` calls to `dump_ast` and `run_script` for
proper module loading.
The `PipelineNode` has been refactored into `StreamNode`. This commit
updates the example files to reflect this change and also updates the
`series` constructor to accept a lookback parameter.
The `PipelineNode` was an artifact of a previous implementation and is
no longer needed. The `StreamNode` is the appropriate type for
representing the output of a pipe operation.
The `series` function now requires a `lookback` argument to specify the
maximum number of items to store. This makes the behavior of series more
explicit and prevents accidental unbounded memory growth.
This commit extracts the logic for inlining function calls into a new
private
method, `try_inline`. This improves code organization and reduces
duplication in the `Optimizer`'s `visit_node` method.
The extracted method handles the preparation of substitutions and the
recursive visiting of the inlined node. It also ensures that the
substitution map state is correctly synchronized back to the calling
context.
The type checker now correctly handles unwrapping `Stream(T)` types for
lambda arguments.
A new `build_map_stream` function has been added to `rtl/streams.rs` to
facilitate field access on streams.
The `create-random-ohlc` function now returns a `Stream` instead of a
`Series`.
Examples have been updated to reflect these changes and demonstrate
stream usage.
The `#use` directive now supports referencing entire directories,
automatically including all `.myc` files within them. Additionally,
environments are now initialized with the current working directory as a
default search path, simplifying module resolution.
Introduces the `#use` preprocessor directive, allowing Myc scripts to
explicitly declare dependencies on other Myc libraries. This change
moves dependency management from a command-line argument to an in-script
declaration, ensuring scripts are self-contained and can correctly
resolve macros and other symbols.
Key features include:
- Declarations at the beginning of a file, evaluated before parsing.
- Support for relative paths and a new `->` separator.
- Automatic resolution of dependencies from specified search paths.
- Idempotent loading to prevent duplicate parsing and evaluation.
- No AST pollution; dependency management is a compile-time concern.
The compiler's lexer has been updated to recognize `#` as a comment
character, and the environment now manages search paths and loaded
modules. This lays the groundwork for more complex library structures
and improved code organization.
Move the SMA implementation from examples/err.myc and examples/sma.myc
to a new dedicated file in lib/. This change also cleans up unused code
and improves the SMA logic.
This commit enhances the series type handling in `src/ast/rtl/series.rs`
by introducing helper methods `as_float` and `as_int` in
`src/ast/types.rs` to manage type conversions more robustly.
It also updates the function overload resolution in `src/ast/types.rs`
to prioritize exact parameter matches before falling back to implicit
coercions.
Finally, the example `err.myc` has been updated to reflect a more
accurate implementation of a Simple Moving Average (SMA) calculation.
The tail call resolution logic was duplicated in `Environment::call` and
`VM::run`. This commit extracts the tail call resolution logic into a
single method `VM::resolve_tail_calls` and uses it in both places.
Additionally, this commit adds support for series indexing as a form of
tail call, allowing for direct access to series elements through the
`series(index)` syntax. This is useful for back-referencing in
time-series data.
A new example `err.myc` is added to demonstrate basic series usage and
error handling.
This commit introduces the BNF for the Myc language, along with its core
semantics, data types, and evaluation logic. It also details the macro
system and provides an overview of the standard library. This document
serves as a foundational context for LLMs to understand and generate Myc
code.
Introduce new indicators for Simple Moving Average (SMA) and Weighted
Moving Average (WMA), along with their Hull Moving Average (HMA)
derivative.
Also refactors series implementation to use `RefCell` for interior
mutability and adds `SeriesMember` trait for better dynamic series
access. Updates `soa_series.myc` example to reflect new series creation
and push syntax.
This commit updates the benchmark and repeat counts for various example
files. These changes reflect potential performance improvements or
variations observed during testing.
Add `Output` comments to the example files to reflect the actual output
of the programs. This helps with verifying correctness and understanding
example behavior.
Introduces type specialization for reactive pipeline nodes and their
data buffers. This eliminates the overhead of generic `Value` enums and
`SharedValueSeries` when dealing with known scalar or record types.
The architecture shifts buffer instantiation from the VM to the runtime
(RTL), leveraging type information from the AST. A new `out_type` field
is added to `BoundKind::Pipe` to store the static type of the pipeline's
output, determined by the type checker.
This enables the creation of specialized `RingBuffer<T>` and
`SharedRecordSeries` (for Struct-of-Arrays layout) when the output type
is known, significantly improving memory usage and processing speed for
time-series data, especially in financial analysis.
- Use a `make-random` factory to create isolated random number
generators.
- Remove the global `prng` from the `Environment`.
- Ensure `make-random` can be called with or without a seed.
Implements the `create-ticker` function, which allows for the creation
of a ticker stream based on a provided condition closure. This function
is crucial for synchronizing pipeline execution based on specific events
or conditions.
Adds support for pipelines, allowing streams to be chained together and
transformed.
This includes new types for `PipeStream` and `SourceAdapter`, along with
modifications
to the `Environment` to manage pipeline execution. A new example
`pipeline.myc`
demonstrates its usage.
Introduces `StaticType::Optional` to represent values that can be either
of a specific type `T` or `Void`. This is crucial for handling
situations in filter pipes where an expression might not always produce
a value.
The type checker now correctly deduces and propagates this `Optional`
type, and the pipeline operator (`pipe`) is updated to unwrap
`Optional(T)` results, yielding `T`. This ensures that the type system
accurately reflects the potential absence of values in intermediate
pipeline steps.
Also includes a minor rename in the tuple-struct example from `pipe` to
`p` for clarity, and registers the `streams` module.
This commit introduces the `get_record` method to `RecordSeries`,
allowing for the dynamic reconstruction of a full record from its
constituent fields at a given index. This enables idiomatic access to
series data using indexing like `(my_ticks 0)`.
The `push-series` function has also been updated to correctly handle the
structure of records when pushing data into a `RecordSeries`.
Introduces a new RTL module for handling series data, enabling efficient
storage and retrieval of time-series data in a Struct-of-Arrays (SoA)
format.
Includes `SeriesView`, a zero-copy wrapper that provides a fast,
read-only interface to individual columns within a `RecordSeries`,
optimizing performance for data access in the VM.
Registers new native functions `create-series` and `push-series` for
managing series data.
This commit refactors the internal representation of `BoundKind::Record`
to store a `RecordLayout` and a `Vec` of values, rather than a `Vec` of
key-value pairs. This change simplifies the representation and improves
efficiency by decoupling the record's structure from its specific values
during compilation and analysis.
The `RecordLayout` now defines the structure of the record, and the
values are stored in a separate vector, ordered according to the layout.
This allows for better optimization and type checking, as the record's
shape is explicitly defined and immutable once created.
These changes update the benchmark numbers and repeat counts in the
example files. The benchmark results have slightly varied, and these
updates reflect the most recent measurements.
This commit updates the benchmark results for various examples. The
benchmark times and repeat counts have been adjusted to reflect current
performance characteristics.
This commit introduces a new prelude file that defines the `while` macro
and registers it during environment bootstrapping. The `again` macro has
been updated to accept multiple arguments for its recursive call, and
several examples have been adjusted to reflect this change.
Additionally, the `MacroRegistry` in the compiler is now cloneable and
can be moved out of the `MacroExpander`.
Add a new Markdown document detailing the AST node structure as it
evolves through the compiler's different stages. This includes a diagram
illustrating the data flow and relationships between the generic
`Node<K, T>` structure and its specialized forms (`UntypedNode`,
`BoundNode`, `TypedNode`, `AnalyzedNode`). The document explains the
`kind` and `ty` payloads for each stage and their significance.
Also removes a deleted file related to a previous optimization plan and
adds new files for future optimization plans.
Introduces a new `RecordLayout` system for efficient and type-safe
record handling.
Key changes:
- `RecordLayout` struct with `O(1)` field lookup using FMap
optimization.
- Field accessors (`.name`) are now first-class callable values.
- Parser recognizes dot-prefixed identifiers as field accessors.
- Binder and TypeChecker handle field accessors.
- Optimizer transforms field accessor calls into specialized `GetField`
nodes.
- VM executes `GetField` efficiently by directly accessing record
values.
- Adds a new `records.myc` example showcasing record features.
- Improves comparison logic for records to use pointer equality for
layout.
These tests were for record destructuring, which has been removed.
The code for record destructuring was also removed from the type checker
and types modules.
This commit introduces newtype wrappers for `LocalSlot`, `UpvalueIdx`,
and `GlobalIdx` to improve type safety and clarity. These wrappers
replace direct use of `u32` for indices, making the code more robust and
easier to understand.
The changes include:
- Defining `LocalSlot`, `UpvalueIdx`, and `GlobalIdx` structs.
- Implementing `Display` for these new types to provide user-friendly
output.
- Updating the `Address` enum to use these new types.
- Modifying various compiler components (analyzer, binder, optimizer,
type checker, environment, VM) to use the new types.
- Adjusting tests to accommodate the changes.
This commit consolidates `DefLocal` and `DefGlobal` into a single
`Define` bound kind. This simplifies the AST and makes it more
consistent.
It also introduces `DeclarationKind` to differentiate between variable
and parameter definitions.
The `UsageInfo` struct has been updated to include tracking for
`used_upvalues` and `assigned_upvalues`. The `collect_usage` function
has been modified to handle these new fields when encountering
`Address::Upvalue`.
The `map_params_to_args` function now takes `body_usage` as an argument
to ensure that parameters assigned to or used in the body are correctly
substituted. A new helper function, `collect_parameter_slots_set`, has
been added to gather all parameter slots within a pattern.
The inlining logic in `Optimizer::inline_call` has been enhanced to
prevent inlining if a parameter is used or assigned in the function body
but cannot be substituted. This addresses a bug where aggressive
inlining could lead to incorrect code generation when parameters were
reassigned.
A new integration test, `test_closure_reassignment_optimization_bug`,
has been added to specifically target and verify the fix for this
inlining issue.
Refine the type checker to correctly handle destructuring of various
collection types (tuples, vectors, matrices, lists, records).
Additionally, prevent implicit type coercion for function arguments,
ensuring that only tuples are passed to functions expecting tuple
arguments. This avoids unexpected behavior where vectors or matrices
might be treated as tuples.
Add a new example file demonstrating pattern matching and destructuring
rules.
The `flatten_tuple` function was unnecessarily recursive. It can now
simply return the elements of the tuple directly. The VM's destructuring
logic has been updated to handle nested destructuring more efficiently.
A new integration test case for multi-level destructuring has been
added.
This commit introduces optimizations for nested destructuring, allowing
tuples and records to be flattened and matched directly against function
arguments. This significantly improves performance by enabling more
constant folding and reducing intermediate allocations.
The changes include:
- Modifying the `Binder` to correctly count nested parameters.
- Enhancing `flatten_tuple` in the `Optimizer` to handle records and NOP
nodes.
- Updating `map_params_to_args` to recursively destructure nested
compound arguments.
- Adding integration tests to verify the correctness of tuple-to-tuple
and record-to-tuple destructuring optimizations.
This commit updates the benchmark numbers in various example files. The
changes reflect recent performance optimizations or adjustments to the
benchmarking environment.
This commit updates the benchmark metadata in several example files. The
benchmark runs and repeat counts have been slightly adjusted due to
optimizations or minor variations in execution.
This commit introduces purity inference to the compiler's optimizer.
This allows for more aggressive constant folding and dead code
elimination by tracking which functions and global variables
are free of side effects.
Key changes include:
- Added `global_purity` field to `Optimizer` and `Environment`.
- Modified `Optimizer::is_pure` to recursively determine if an
AST node represents a pure computation.
- Introduced `Optimizer::try_fold_pure` to replace the old
`try_fold_intrinsic`, enabling folding of pure function calls
with constant arguments.
- Updated `Environment::register_native` and
`Environment::register_constant`
to optionally record purity.
- Added purity flags to several built-in functions in `core.rs` and
`datetime.rs`.
- A new example `optimizer_purity.myc` demonstrates the new feature.
Add a new example file `examples/def_local_inlining.myc` to demonstrate
and test local inlining.
Refine the optimizer to handle local inlining more robustly by:
- Passing a `SubstitutionMap` to `visit_node` to track local variable
substitutions.
- Enabling constant propagation for local variables by checking
`sub.locals` in `BoundKind::Get`.
- Updating `BoundKind::DefLocal` and `BoundKind::Set` to maintain the
substitution map for local variables.
- Adjusting `try_beta_reduce` to use the optimizer's `visit_node` with
the substitution map for inlining.
- Re-indexing upvalues correctly when inlining captured variables into
lambdas.
Introduces a new optimizer pass that can "crack" closures, allowing for
more aggressive specialization. It also enables inlining of upvalues
that point to immutable global variables. This removes overhead for
higher-order functions and currying when arguments are statically
resolvable.
Michael Schimmel