The type checker's fallback logic for non-lambda nodes was unnecessarily
complex. This commit simplifies it by directly checking the node within
a new, basic type context.
Additionally, the environment now explicitly wraps non-lambda AST nodes
in a lambda before type checking, ensuring consistent handling. This
aligns the type checking process for all top-level expressions.
The `#use` directive now supports referencing entire directories,
automatically including all `.myc` files within them. Additionally,
environments are now initialized with the current working directory as a
default search path, simplifying module resolution.
Introduces the `#use` preprocessor directive, allowing Myc scripts to
explicitly declare dependencies on other Myc libraries. This change
moves dependency management from a command-line argument to an in-script
declaration, ensuring scripts are self-contained and can correctly
resolve macros and other symbols.
Key features include:
- Declarations at the beginning of a file, evaluated before parsing.
- Support for relative paths and a new `->` separator.
- Automatic resolution of dependencies from specified search paths.
- Idempotent loading to prevent duplicate parsing and evaluation.
- No AST pollution; dependency management is a compile-time concern.
The compiler's lexer has been updated to recognize `#` as a comment
character, and the environment now manages search paths and loaded
modules. This lays the groundwork for more complex library structures
and improved code organization.
Move the SMA implementation from examples/err.myc and examples/sma.myc
to a new dedicated file in lib/. This change also cleans up unused code
and improves the SMA logic.
This commit enhances the series type handling in `src/ast/rtl/series.rs`
by introducing helper methods `as_float` and `as_int` in
`src/ast/types.rs` to manage type conversions more robustly.
It also updates the function overload resolution in `src/ast/types.rs`
to prioritize exact parameter matches before falling back to implicit
coercions.
Finally, the example `err.myc` has been updated to reflect a more
accurate implementation of a Simple Moving Average (SMA) calculation.
The `LambdaCollector` was cloning nodes and wrapping them in `Rc`.
However, the `registry` itself is already holding `Rc` clones of the
nodes. This commit removes the redundant `Rc::new` to simplify the code.
This change also addresses a performance regression observed in
benchmarks due to unnecessary atomic overhead from `Arc` in the
`Keyword` type.
The `resolve_tail_calls` function is being called twice in some
execution paths, which is unnecessary and can lead to incorrect
behavior. This commit removes the redundant calls and simplifies the
execution flow.
The tail call resolution logic was duplicated in `Environment::call` and
`VM::run`. This commit extracts the tail call resolution logic into a
single method `VM::resolve_tail_calls` and uses it in both places.
Additionally, this commit adds support for series indexing as a form of
tail call, allowing for direct access to series elements through the
`series(index)` syntax. This is useful for back-referencing in
time-series data.
A new example `err.myc` is added to demonstrate basic series usage and
error handling.
order
- Add new_for_inlining to SubstitutionMap to preserve the next_slot
counter, preventing local slot collisions when merging
scopes.
- Update Inliner to request fresh slots for parameters during beta
reduction instead of forcing identity mappings.
- Fix execution order in Optimizer::visit_node for closure inlining
to run prepare_beta_reduction before visiting the function
body, preventing corrupted upvalue captures.
- Fix CLI --no-opt flag to correctly disable the optimizer in the ast
binary.
- Add integration test to prevent future slot clash regressions.
This commit introduces the BNF for the Myc language, along with its core
semantics, data types, and evaluation logic. It also details the macro
system and provides an overview of the standard library. This document
serves as a foundational context for LLMs to understand and generate Myc
code.
The specializer now correctly identifies when a compiled value is a
scalar and avoids folding it into a constant node. Instead, it updates
the callee to the specialized version if it's an object. This ensures
that scalar folding does not interfere with the program's execution.
Introduce new indicators for Simple Moving Average (SMA) and Weighted
Moving Average (WMA), along with their Hull Moving Average (HMA)
derivative.
Also refactors series implementation to use `RefCell` for interior
mutability and adds `SeriesMember` trait for better dynamic series
access. Updates `soa_series.myc` example to reflect new series creation
and push syntax.
This commit updates the benchmark and repeat counts for various example
files. These changes reflect potential performance improvements or
variations observed during testing.
This commit refactors various AST node types to use `Rc` (Reference
Counting) instead of `Box`. This change is primarily driven by the need
for shared ownership and more efficient handling of potentially
recursive or shared data structures within the AST.
The main benefits of this change include:
- **Reduced cloning:** `Rc` allows multiple parts of the AST to refer to
the same node without needing to clone the entire node. This is
particularly beneficial for optimization passes where nodes might be
visited multiple times.
- **Enabling sharing:** It makes it easier to represent scenarios where
a single AST node might be a child of multiple parent nodes.
- **Performance improvements:** By avoiding unnecessary deep copies of
AST nodes, this change can lead to performance gains, especially
during complex compilation phases.
This change introduces Rc<MacroMap> to allow for copy-on-write semantics
when defining macros in nested scopes. When a macro is defined in a
scope that is already shared, the MacroMap is cloned before
modification, ensuring that changes to one scope do not affect others
that share the same map.
Introduces `GlobalFunctionRegistry` and `GlobalAnalyzedRegistry` type
aliases to clarify the usage of `HashMap`s for storing global functions
and their analyzed versions. This change also refactors the
`LambdaCollector` and `Optimizer` to use these new aliases, improving
code readability and maintainability.
Add `Output` comments to the example files to reflect the actual output
of the programs. This helps with verifying correctness and understanding
example behavior.
The Binder and its helper functions have been updated to accept and
propagate a `Diagnostics` struct. This allows for better error reporting
during the binding phase, enabling the compiler to continue processing
even after encountering errors and collect all issues before halting.
The `BoundKind::Error` node and `StaticType::Error` are introduced as
"poison" nodes/types. These nodes indicate that an error occurred during
compilation, preventing further valid processing of that specific AST
fragment but allowing the compiler to continue with other parts of the
code.
The `Parser` has also been updated to return a `Diagnostics` struct,
enabling it to report errors during the initial parsing stage while
still attempting to build a partial AST. This adheres to the same error
recovery strategy.
Introduces type specialization for reactive pipeline nodes and their
data buffers. This eliminates the overhead of generic `Value` enums and
`SharedValueSeries` when dealing with known scalar or record types.
The architecture shifts buffer instantiation from the VM to the runtime
(RTL), leveraging type information from the AST. A new `out_type` field
is added to `BoundKind::Pipe` to store the static type of the pipeline's
output, determined by the type checker.
This enables the creation of specialized `RingBuffer<T>` and
`SharedRecordSeries` (for Struct-of-Arrays layout) when the output type
is known, significantly improving memory usage and processing speed for
time-series data, especially in financial analysis.
- Use a `make-random` factory to create isolated random number
generators.
- Remove the global `prng` from the `Environment`.
- Ensure `make-random` can be called with or without a seed.
Implements the `create-ticker` function, which allows for the creation
of a ticker stream based on a provided condition closure. This function
is crucial for synchronizing pipeline execution based on specific events
or conditions.
Adds support for pipelines, allowing streams to be chained together and
transformed.
This includes new types for `PipeStream` and `SourceAdapter`, along with
modifications
to the `Environment` to manage pipeline execution. A new example
`pipeline.myc`
demonstrates its usage.
Introduces `StaticType::Optional` to represent values that can be either
of a specific type `T` or `Void`. This is crucial for handling
situations in filter pipes where an expression might not always produce
a value.
The type checker now correctly deduces and propagates this `Optional`
type, and the pipeline operator (`pipe`) is updated to unwrap
`Optional(T)` results, yielding `T`. This ensures that the type system
accurately reflects the potential absence of values in intermediate
pipeline steps.
Also includes a minor rename in the tuple-struct example from `pipe` to
`p` for clarity, and registers the `streams` module.
Introduces `SharedValueSeries` for read-only access to generic `Value`
buffers, and `PipelineNode` which combines an `ObservableStream` with a
`Series` view. This is a foundational step for implementing the `pipe`
operator.
Introduce a new field to `Environment` to store a list of pipeline
generator functions.
Add a `run_pipeline` method to `Environment` that iterates through and
executes all registered generators until they are exhausted.
Implement `ObservableStream` trait for `RootStream` and `PipeStream`.
Add a `StreamNode` wrapper for `ObservableStream` to be used as a script
object.
Register `create-random-ohlc` as a native function that creates a
`RootStream`, sets up a OHLC record layout, and registers a stateful
generator closure in the environment's pipeline generators.
Introduces a new `Pipe` node to the Abstract Syntax Tree (AST).
This node represents a functional composition pattern, allowing
for chaining of operations.
The changes include:
- Defining the `Pipe` variant in `BoundKind` and `UntypedKind`.
- Implementing parsing logic for the `pipe` keyword.
- Adding handling for the `Pipe` node in various compiler passes
(analyzer, binder, captures, dumper, macros, optimizer, TCO,
type checker).
- Including a placeholder implementation for `Pipe` execution in the
VM.
- Updating integration tests to use the new `pipe` syntax.
Introduces the streams module, defining core reactive primitives like
Signal,
Stream, Observer, RootStream, and PipeStream. This module lays the
groundwork
for building reactive data processing pipelines.
Includes new types for managing reactive data flow and a basic test
suite to
verify the functionality of RootStream and PipeStream.
This commit introduces the `Series` trait to provide a unified interface
for accessing indexed data across different series types (RecordSeries,
SeriesView, and future SharedSeries). This simplifies the VM's indexer
logic and lays the groundwork for the dual series architecture.
This commit introduces a detailed analysis of the Delphi Reactive Stream
Pipeline architecture, contrasting it with the current Rust
implementation. It outlines the core concepts of Delphi's stateless
streams, `TStreamSignal`, `TRootStream`, `TPipeSource`, `TScalarSeries`,
and `TPipeStream`, emphasizing their roles in event propagation, state
management, and synchronization.
The document also details the integration of the `pipe` statement within
the Delphi AST and compiler pipeline, covering the parser, AST
structure, type checker, and evaluator.
Finally, it proposes a "Dual Series Architecture" for Rust,
differentiating between script-local `RecordSeries` (for accumulators)
and reactive `SharedSeries` (for the pipeline), and outlines a concrete
implementation plan.
This commit introduces the `get_record` method to `RecordSeries`,
allowing for the dynamic reconstruction of a full record from its
constituent fields at a given index. This enables idiomatic access to
series data using indexing like `(my_ticks 0)`.
The `push-series` function has also been updated to correctly handle the
structure of records when pushing data into a `RecordSeries`.
Introduces a new RTL module for handling series data, enabling efficient
storage and retrieval of time-series data in a Struct-of-Arrays (SoA)
format.
Includes `SeriesView`, a zero-copy wrapper that provides a fast,
read-only interface to individual columns within a `RecordSeries`,
optimizing performance for data access in the VM.
Registers new native functions `create-series` and `push-series` for
managing series data.
Introduce the `series` module to handle time-series data storage and
manipulation. This module includes:
- `RingBuffer`: An efficient ring buffer for storing time-series data
with a configurable lookback.
- `ScalarSeries`: A type-safe series for primitive scalar values (f64,
i64, bool) optimized for performance.
- `ValueSeries`: A fallback series for non-scalar or mixed data types.
- `RecordSeries`: A "Struct of Arrays" implementation for records,
splitting fields into parallel series for efficient access.
This commit refactors the internal representation of `BoundKind::Record`
to store a `RecordLayout` and a `Vec` of values, rather than a `Vec` of
key-value pairs. This change simplifies the representation and improves
efficiency by decoupling the record's structure from its specific values
during compilation and analysis.
The `RecordLayout` now defines the structure of the record, and the
values are stored in a separate vector, ordered according to the layout.
This allows for better optimization and type checking, as the record's
shape is explicitly defined and immutable once created.
These changes update the benchmark numbers and repeat counts in the
example files. The benchmark results have slightly varied, and these
updates reflect the most recent measurements.
The `SubstitutionMap::new_inner()` method provides a more robust way to
create nested substitution maps, ensuring that only global substitutions
are inherited. This prevents potential issues with overlapping local and
upvalue addresses in nested scopes, such as during lambda inlining.
This change also includes:
- A new `try_fold_record` method in `folder.rs` to optimize record
literals into constant values.
- Updates to `inliner.rs` to recognize `Value::Record` for inlining.
- Various test cases to verify record optimization and constant folding.
This commit refactors the `execute_trace` function to utilize the
`run_debug` method on the `Environment`. This simplifies the execution
flow and leverages existing functionality for tracing and error
handling.
This commit updates the benchmark results for various examples. The
benchmark times and repeat counts have been adjusted to reflect current
performance characteristics.
The `CallFrame` struct and various VM methods now use `Rc<dyn Object>`
to hold closures, allowing for more flexibility and avoiding unnecessary
cloning.
This change also addresses the performance concern regarding deep copies
of closures, preferring `Rc<dyn Trait>` with local `downcast_ref` where
appropriate.
Additionally, a documentation note has been added to `gemini.md`
regarding this performance preference in Rust.
This commit introduces a new prelude file that defines the `while` macro
and registers it during environment bootstrapping. The `again` macro has
been updated to accept multiple arguments for its recursive call, and
several examples have been adjusted to reflect this change.
Additionally, the `MacroRegistry` in the compiler is now cloneable and
can be moved out of the `MacroExpander`.
Michael Schimmel