The 'again' macro is a new addition to the macro system, allowing for
recursive macro expansion. This commit introduces the necessary AST
nodes, macro expansion logic, and an integration test to ensure its
functionality. The `soa_series.myc` example has also been updated to
demonstrate its usage.
The `Parameter` kind in `UntypedKind` was only used for identifiers that
were being declared or referenced. This commit renames it to
`Identifier` and updates all the necessary code to reflect this change.
This simplifies the AST and makes it more consistent.
Additionally, a new macro `repeat` has been added to
`src/ast/system.myc`.
The `system.myc` file, containing macros and core utilities, has been
removed as its functionality is now handled by an embedded string. This
simplifies the build process and eliminates a file that was no longer
strictly necessary.
Additionally, the `with_cwd()` method on the `Environment` has been
removed. The environment now automatically adds the current working
directory and its `rtl` subdirectory to the search paths during
initialization. This streamlines the setup for file-based usage and
ensures standard search paths are always considered.
- Add `rtl/system.myc` as the system library, containing core utilities
like `while` and `cache`.
- Remove the duplicate `prelude.myc` from `src/ast/rtl/`.
- Modify `Environment::collect_dependencies` to implicitly add `#use
system` when necessary.
- Update `Environment::with_cwd` to also add the `rtl` subdirectory to
search paths.
- Add `target/` to `.geminiignore` to exclude build artifacts.
- Adjust example `sma.myc` to use the new `cache` macro and reduce
sample data size.
- Update `rtl/sma.myc` to correctly initialize the `history` series with
its `length`.
- Add `preload_dependencies` calls to `dump_ast` and `run_script` for
proper module loading.
The `PipelineNode` has been refactored into `StreamNode`. This commit
updates the example files to reflect this change and also updates the
`series` constructor to accept a lookback parameter.
The `PipelineNode` was an artifact of a previous implementation and is
no longer needed. The `StreamNode` is the appropriate type for
representing the output of a pipe operation.
The `series` function now requires a `lookback` argument to specify the
maximum number of items to store. This makes the behavior of series more
explicit and prevents accidental unbounded memory growth.
This commit extracts the logic for inlining function calls into a new
private
method, `try_inline`. This improves code organization and reduces
duplication in the `Optimizer`'s `visit_node` method.
The extracted method handles the preparation of substitutions and the
recursive visiting of the inlined node. It also ensures that the
substitution map state is correctly synchronized back to the calling
context.
The type checker now correctly handles unwrapping `Stream(T)` types for
lambda arguments.
A new `build_map_stream` function has been added to `rtl/streams.rs` to
facilitate field access on streams.
The `create-random-ohlc` function now returns a `Stream` instead of a
`Series`.
Examples have been updated to reflect these changes and demonstrate
stream usage.
The type checker's fallback logic for non-lambda nodes was unnecessarily
complex. This commit simplifies it by directly checking the node within
a new, basic type context.
Additionally, the environment now explicitly wraps non-lambda AST nodes
in a lambda before type checking, ensuring consistent handling. This
aligns the type checking process for all top-level expressions.
The `#use` directive now supports referencing entire directories,
automatically including all `.myc` files within them. Additionally,
environments are now initialized with the current working directory as a
default search path, simplifying module resolution.
Introduces the `#use` preprocessor directive, allowing Myc scripts to
explicitly declare dependencies on other Myc libraries. This change
moves dependency management from a command-line argument to an in-script
declaration, ensuring scripts are self-contained and can correctly
resolve macros and other symbols.
Key features include:
- Declarations at the beginning of a file, evaluated before parsing.
- Support for relative paths and a new `->` separator.
- Automatic resolution of dependencies from specified search paths.
- Idempotent loading to prevent duplicate parsing and evaluation.
- No AST pollution; dependency management is a compile-time concern.
The compiler's lexer has been updated to recognize `#` as a comment
character, and the environment now manages search paths and loaded
modules. This lays the groundwork for more complex library structures
and improved code organization.
Move the SMA implementation from examples/err.myc and examples/sma.myc
to a new dedicated file in lib/. This change also cleans up unused code
and improves the SMA logic.
This commit enhances the series type handling in `src/ast/rtl/series.rs`
by introducing helper methods `as_float` and `as_int` in
`src/ast/types.rs` to manage type conversions more robustly.
It also updates the function overload resolution in `src/ast/types.rs`
to prioritize exact parameter matches before falling back to implicit
coercions.
Finally, the example `err.myc` has been updated to reflect a more
accurate implementation of a Simple Moving Average (SMA) calculation.
The `LambdaCollector` was cloning nodes and wrapping them in `Rc`.
However, the `registry` itself is already holding `Rc` clones of the
nodes. This commit removes the redundant `Rc::new` to simplify the code.
This change also addresses a performance regression observed in
benchmarks due to unnecessary atomic overhead from `Arc` in the
`Keyword` type.
The `resolve_tail_calls` function is being called twice in some
execution paths, which is unnecessary and can lead to incorrect
behavior. This commit removes the redundant calls and simplifies the
execution flow.
The tail call resolution logic was duplicated in `Environment::call` and
`VM::run`. This commit extracts the tail call resolution logic into a
single method `VM::resolve_tail_calls` and uses it in both places.
Additionally, this commit adds support for series indexing as a form of
tail call, allowing for direct access to series elements through the
`series(index)` syntax. This is useful for back-referencing in
time-series data.
A new example `err.myc` is added to demonstrate basic series usage and
error handling.
order
- Add new_for_inlining to SubstitutionMap to preserve the next_slot
counter, preventing local slot collisions when merging
scopes.
- Update Inliner to request fresh slots for parameters during beta
reduction instead of forcing identity mappings.
- Fix execution order in Optimizer::visit_node for closure inlining
to run prepare_beta_reduction before visiting the function
body, preventing corrupted upvalue captures.
- Fix CLI --no-opt flag to correctly disable the optimizer in the ast
binary.
- Add integration test to prevent future slot clash regressions.
This commit introduces the BNF for the Myc language, along with its core
semantics, data types, and evaluation logic. It also details the macro
system and provides an overview of the standard library. This document
serves as a foundational context for LLMs to understand and generate Myc
code.
The specializer now correctly identifies when a compiled value is a
scalar and avoids folding it into a constant node. Instead, it updates
the callee to the specialized version if it's an object. This ensures
that scalar folding does not interfere with the program's execution.
Introduce new indicators for Simple Moving Average (SMA) and Weighted
Moving Average (WMA), along with their Hull Moving Average (HMA)
derivative.
Also refactors series implementation to use `RefCell` for interior
mutability and adds `SeriesMember` trait for better dynamic series
access. Updates `soa_series.myc` example to reflect new series creation
and push syntax.
This commit updates the benchmark and repeat counts for various example
files. These changes reflect potential performance improvements or
variations observed during testing.
This commit refactors various AST node types to use `Rc` (Reference
Counting) instead of `Box`. This change is primarily driven by the need
for shared ownership and more efficient handling of potentially
recursive or shared data structures within the AST.
The main benefits of this change include:
- **Reduced cloning:** `Rc` allows multiple parts of the AST to refer to
the same node without needing to clone the entire node. This is
particularly beneficial for optimization passes where nodes might be
visited multiple times.
- **Enabling sharing:** It makes it easier to represent scenarios where
a single AST node might be a child of multiple parent nodes.
- **Performance improvements:** By avoiding unnecessary deep copies of
AST nodes, this change can lead to performance gains, especially
during complex compilation phases.
This change introduces Rc<MacroMap> to allow for copy-on-write semantics
when defining macros in nested scopes. When a macro is defined in a
scope that is already shared, the MacroMap is cloned before
modification, ensuring that changes to one scope do not affect others
that share the same map.
Introduces `GlobalFunctionRegistry` and `GlobalAnalyzedRegistry` type
aliases to clarify the usage of `HashMap`s for storing global functions
and their analyzed versions. This change also refactors the
`LambdaCollector` and `Optimizer` to use these new aliases, improving
code readability and maintainability.
Add `Output` comments to the example files to reflect the actual output
of the programs. This helps with verifying correctness and understanding
example behavior.
The Binder and its helper functions have been updated to accept and
propagate a `Diagnostics` struct. This allows for better error reporting
during the binding phase, enabling the compiler to continue processing
even after encountering errors and collect all issues before halting.
The `BoundKind::Error` node and `StaticType::Error` are introduced as
"poison" nodes/types. These nodes indicate that an error occurred during
compilation, preventing further valid processing of that specific AST
fragment but allowing the compiler to continue with other parts of the
code.
The `Parser` has also been updated to return a `Diagnostics` struct,
enabling it to report errors during the initial parsing stage while
still attempting to build a partial AST. This adheres to the same error
recovery strategy.
Introduces type specialization for reactive pipeline nodes and their
data buffers. This eliminates the overhead of generic `Value` enums and
`SharedValueSeries` when dealing with known scalar or record types.
The architecture shifts buffer instantiation from the VM to the runtime
(RTL), leveraging type information from the AST. A new `out_type` field
is added to `BoundKind::Pipe` to store the static type of the pipeline's
output, determined by the type checker.
This enables the creation of specialized `RingBuffer<T>` and
`SharedRecordSeries` (for Struct-of-Arrays layout) when the output type
is known, significantly improving memory usage and processing speed for
time-series data, especially in financial analysis.
- Use a `make-random` factory to create isolated random number
generators.
- Remove the global `prng` from the `Environment`.
- Ensure `make-random` can be called with or without a seed.
Implements the `create-ticker` function, which allows for the creation
of a ticker stream based on a provided condition closure. This function
is crucial for synchronizing pipeline execution based on specific events
or conditions.
Adds support for pipelines, allowing streams to be chained together and
transformed.
This includes new types for `PipeStream` and `SourceAdapter`, along with
modifications
to the `Environment` to manage pipeline execution. A new example
`pipeline.myc`
demonstrates its usage.
Introduces `StaticType::Optional` to represent values that can be either
of a specific type `T` or `Void`. This is crucial for handling
situations in filter pipes where an expression might not always produce
a value.
The type checker now correctly deduces and propagates this `Optional`
type, and the pipeline operator (`pipe`) is updated to unwrap
`Optional(T)` results, yielding `T`. This ensures that the type system
accurately reflects the potential absence of values in intermediate
pipeline steps.
Also includes a minor rename in the tuple-struct example from `pipe` to
`p` for clarity, and registers the `streams` module.
Introduces `SharedValueSeries` for read-only access to generic `Value`
buffers, and `PipelineNode` which combines an `ObservableStream` with a
`Series` view. This is a foundational step for implementing the `pipe`
operator.
Introduce a new field to `Environment` to store a list of pipeline
generator functions.
Add a `run_pipeline` method to `Environment` that iterates through and
executes all registered generators until they are exhausted.
Implement `ObservableStream` trait for `RootStream` and `PipeStream`.
Add a `StreamNode` wrapper for `ObservableStream` to be used as a script
object.
Register `create-random-ohlc` as a native function that creates a
`RootStream`, sets up a OHLC record layout, and registers a stateful
generator closure in the environment's pipeline generators.
Introduces a new `Pipe` node to the Abstract Syntax Tree (AST).
This node represents a functional composition pattern, allowing
for chaining of operations.
The changes include:
- Defining the `Pipe` variant in `BoundKind` and `UntypedKind`.
- Implementing parsing logic for the `pipe` keyword.
- Adding handling for the `Pipe` node in various compiler passes
(analyzer, binder, captures, dumper, macros, optimizer, TCO,
type checker).
- Including a placeholder implementation for `Pipe` execution in the
VM.
- Updating integration tests to use the new `pipe` syntax.
Introduces the streams module, defining core reactive primitives like
Signal,
Stream, Observer, RootStream, and PipeStream. This module lays the
groundwork
for building reactive data processing pipelines.
Includes new types for managing reactive data flow and a basic test
suite to
verify the functionality of RootStream and PipeStream.
This commit introduces the `Series` trait to provide a unified interface
for accessing indexed data across different series types (RecordSeries,
SeriesView, and future SharedSeries). This simplifies the VM's indexer
logic and lays the groundwork for the dual series architecture.
Michael Schimmel