Introduce `StaticType::Forall` to represent universally quantified
types. This enables Hindley-Milner's let-polymorphism, allowing
functions to be generalized and reused with different concrete types.
- **Generalization at `def`:** Function values are now generalized using
`self.generalize` when they are defined, wrapping their type in
`Forall`. This occurs at `def` boundaries.
- **Instantiation at use:** Function types are instantiated with fresh
type variables at each call site using `self.instantiate`. This
ensures that different uses of the same polymorphic function do not
interfere with each other's type inference.
- **Value restriction:** Only function-typed definitions are
generalized. Mutable state like series and scalars remain monomorphic
to prevent unexpected behavior.
- **Type context awareness:** The `generalize` function considers the
current type context to avoid quantifying type variables that are
still in use elsewhere.
Feat: Add Forall type for polymorphism
Introduces the `Forall` type to support polymorphic functions and enable
let-polymorphism.
- `Forall(vars, body)`: Represents a universally quantified type where
`vars` are the type variables bound by this quantification, and `body`
is the type itself.
- `TypeChecker::generalize`: Wraps a type in `Forall` when a `def`
boundary is encountered for function-typed values. This ensures that
each use site of a polymorphic function gets its own instantiation.
- `TypeChecker::instantiate`: Replaces the type variables within a
`Forall` type with fresh ones. This is performed at each call site of
a polymorphic function.
- Updates `TypeChecker::unify` and `TypeChecker::apply_subst` to
correctly handle `Forall` types, ensuring that bound variables within
a `Forall` are not affected by global substitutions and that `Forall`
types are instantiated before unification.
- Adds a new example script `examples/Propa.myc` to demonstrate basic
usage.
- Includes a regression test
`test_let_poly_non_series_callable_no_false_positive` to ensure that
passing non-series callables to generic functions does not lead to
type errors.
- Introduces `TypeChecker::bind_var` to handle lazy propagation of
index-call constraints, crucial for correctly typing series lookups
like `(s 0)`.
The `series` constructor now only accepts the `lookback` limit. The
element type is inferred by the type checker and implicitly added as a
schema argument during compilation. This simplifies the API and
centralizes type inference.
This change also includes:
- Updates to example scripts (`HMA.myc`, `sma.myc`, `soa_series.myc`) to
reflect the new `series` signature.
- Modifications to the `TypeChecker` to handle the inferred schema
injection.
- An addition of a regression test for type inference through nested
closures with `series`.
- Removal of `#[allow(dead_code)]` from several `TypeChecker` methods as
they are now used.
- Update to `rtl/prelude.myc` macro `cache`.
- Update to `rtl/series/mod.rs` to reflect new signature.
- Update to `ast/types.rs` to allow `series` to be indexed with an
integer to retrieve its element type.
This commit updates all example files to use `assert_eq!` for verifying
output, replacing the previous `Output:` comments. This change makes the
examples more robust and self-testing.
The benchmark results in some examples have also been slightly adjusted,
reflecting minor performance variations after the refactoring.
Additionally, a runtime panic handling mechanism has been introduced in
the VM for function calls, which improves error reporting for unexpected
panics.
This commit introduces a new example script, `HMA.myc`, which
demonstrates the implementation and usage of the Hull Moving Average
(HMA) indicator.
The script includes factory functions for Simple Moving Average
(`make-sma`), Weighted Moving Average (`make-wma`), and the Hull Moving
Average itself (`make-hma`). It also includes a small example loop to
initialize and print HMA values.
This commit introduces support for comments within the AST nodes.
Comments are now parsed by the lexer and stored within the `Token`
struct.
These comments are then propagated through various compiler phases,
including
the `Node` struct, ensuring they are preserved in the Abstract Syntax
Tree.
This change enhances the AST's ability to retain source code
information,
which can be valuable for debugging and analysis.
The Binder and related types have been refactored to use the new
`NodeKind` enum instead of the previous `BoundKind`. This commit updates
all references to use the new structure, ensuring consistency across the
compiler's AST representation.
Key changes include:
- Replacing `BoundKind` with `NodeKind` in the Binder's `bind` and
`visit` methods.
- Updating pattern matching and field access to reflect the new enum
variants (e.g., `NodeKind::Def` instead of `BoundKind::Define`).
- Adjusting identifier bindings to use the new `IdentifierBinding` enum.
- Reflecting changes in `DefBinding`, `AssignBinding`, and
`LambdaBinding` structures.
- Ensuring all newly created nodes use `NodeKind` and the appropriate
metadata.
Implement strict block scoping and distinguish between statements and
expressions.
This change introduces hierarchical scopes for functions and blocks,
enforces that `def` is a statement with no return value, and prevents
statements from appearing in expression positions or as the last element
of a block. A new example `design-flaw.myc` is added to demonstrate
the uninitialized variable issue solved by these changes.
The `Analyzer` is now responsible for determining the purity of `Define`
bindings. Previously, this responsibility was left to the caller.
Added a new example `design-flaw.myc` to test this change.
Updated `soa_series.myc` due to the purity change.
The 'again' macro is a new addition to the macro system, allowing for
recursive macro expansion. This commit introduces the necessary AST
nodes, macro expansion logic, and an integration test to ensure its
functionality. The `soa_series.myc` example has also been updated to
demonstrate its usage.
The `Parameter` kind in `UntypedKind` was only used for identifiers that
were being declared or referenced. This commit renames it to
`Identifier` and updates all the necessary code to reflect this change.
This simplifies the AST and makes it more consistent.
Additionally, a new macro `repeat` has been added to
`src/ast/system.myc`.
- Add `rtl/system.myc` as the system library, containing core utilities
like `while` and `cache`.
- Remove the duplicate `prelude.myc` from `src/ast/rtl/`.
- Modify `Environment::collect_dependencies` to implicitly add `#use
system` when necessary.
- Update `Environment::with_cwd` to also add the `rtl` subdirectory to
search paths.
- Add `target/` to `.geminiignore` to exclude build artifacts.
- Adjust example `sma.myc` to use the new `cache` macro and reduce
sample data size.
- Update `rtl/sma.myc` to correctly initialize the `history` series with
its `length`.
- Add `preload_dependencies` calls to `dump_ast` and `run_script` for
proper module loading.
The `PipelineNode` has been refactored into `StreamNode`. This commit
updates the example files to reflect this change and also updates the
`series` constructor to accept a lookback parameter.
The `PipelineNode` was an artifact of a previous implementation and is
no longer needed. The `StreamNode` is the appropriate type for
representing the output of a pipe operation.
The `series` function now requires a `lookback` argument to specify the
maximum number of items to store. This makes the behavior of series more
explicit and prevents accidental unbounded memory growth.
This commit extracts the logic for inlining function calls into a new
private
method, `try_inline`. This improves code organization and reduces
duplication in the `Optimizer`'s `visit_node` method.
The extracted method handles the preparation of substitutions and the
recursive visiting of the inlined node. It also ensures that the
substitution map state is correctly synchronized back to the calling
context.
The type checker now correctly handles unwrapping `Stream(T)` types for
lambda arguments.
A new `build_map_stream` function has been added to `rtl/streams.rs` to
facilitate field access on streams.
The `create-random-ohlc` function now returns a `Stream` instead of a
`Series`.
Examples have been updated to reflect these changes and demonstrate
stream usage.
The `#use` directive now supports referencing entire directories,
automatically including all `.myc` files within them. Additionally,
environments are now initialized with the current working directory as a
default search path, simplifying module resolution.
Introduces the `#use` preprocessor directive, allowing Myc scripts to
explicitly declare dependencies on other Myc libraries. This change
moves dependency management from a command-line argument to an in-script
declaration, ensuring scripts are self-contained and can correctly
resolve macros and other symbols.
Key features include:
- Declarations at the beginning of a file, evaluated before parsing.
- Support for relative paths and a new `->` separator.
- Automatic resolution of dependencies from specified search paths.
- Idempotent loading to prevent duplicate parsing and evaluation.
- No AST pollution; dependency management is a compile-time concern.
The compiler's lexer has been updated to recognize `#` as a comment
character, and the environment now manages search paths and loaded
modules. This lays the groundwork for more complex library structures
and improved code organization.
Move the SMA implementation from examples/err.myc and examples/sma.myc
to a new dedicated file in lib/. This change also cleans up unused code
and improves the SMA logic.
This commit enhances the series type handling in `src/ast/rtl/series.rs`
by introducing helper methods `as_float` and `as_int` in
`src/ast/types.rs` to manage type conversions more robustly.
It also updates the function overload resolution in `src/ast/types.rs`
to prioritize exact parameter matches before falling back to implicit
coercions.
Finally, the example `err.myc` has been updated to reflect a more
accurate implementation of a Simple Moving Average (SMA) calculation.
The tail call resolution logic was duplicated in `Environment::call` and
`VM::run`. This commit extracts the tail call resolution logic into a
single method `VM::resolve_tail_calls` and uses it in both places.
Additionally, this commit adds support for series indexing as a form of
tail call, allowing for direct access to series elements through the
`series(index)` syntax. This is useful for back-referencing in
time-series data.
A new example `err.myc` is added to demonstrate basic series usage and
error handling.
This commit introduces the BNF for the Myc language, along with its core
semantics, data types, and evaluation logic. It also details the macro
system and provides an overview of the standard library. This document
serves as a foundational context for LLMs to understand and generate Myc
code.
Introduce new indicators for Simple Moving Average (SMA) and Weighted
Moving Average (WMA), along with their Hull Moving Average (HMA)
derivative.
Also refactors series implementation to use `RefCell` for interior
mutability and adds `SeriesMember` trait for better dynamic series
access. Updates `soa_series.myc` example to reflect new series creation
and push syntax.
This commit updates the benchmark and repeat counts for various example
files. These changes reflect potential performance improvements or
variations observed during testing.
Add `Output` comments to the example files to reflect the actual output
of the programs. This helps with verifying correctness and understanding
example behavior.
Introduces type specialization for reactive pipeline nodes and their
data buffers. This eliminates the overhead of generic `Value` enums and
`SharedValueSeries` when dealing with known scalar or record types.
The architecture shifts buffer instantiation from the VM to the runtime
(RTL), leveraging type information from the AST. A new `out_type` field
is added to `BoundKind::Pipe` to store the static type of the pipeline's
output, determined by the type checker.
This enables the creation of specialized `RingBuffer<T>` and
`SharedRecordSeries` (for Struct-of-Arrays layout) when the output type
is known, significantly improving memory usage and processing speed for
time-series data, especially in financial analysis.
- Use a `make-random` factory to create isolated random number
generators.
- Remove the global `prng` from the `Environment`.
- Ensure `make-random` can be called with or without a seed.
Implements the `create-ticker` function, which allows for the creation
of a ticker stream based on a provided condition closure. This function
is crucial for synchronizing pipeline execution based on specific events
or conditions.
Adds support for pipelines, allowing streams to be chained together and
transformed.
This includes new types for `PipeStream` and `SourceAdapter`, along with
modifications
to the `Environment` to manage pipeline execution. A new example
`pipeline.myc`
demonstrates its usage.
Introduces `StaticType::Optional` to represent values that can be either
of a specific type `T` or `Void`. This is crucial for handling
situations in filter pipes where an expression might not always produce
a value.
The type checker now correctly deduces and propagates this `Optional`
type, and the pipeline operator (`pipe`) is updated to unwrap
`Optional(T)` results, yielding `T`. This ensures that the type system
accurately reflects the potential absence of values in intermediate
pipeline steps.
Also includes a minor rename in the tuple-struct example from `pipe` to
`p` for clarity, and registers the `streams` module.
This commit introduces the `get_record` method to `RecordSeries`,
allowing for the dynamic reconstruction of a full record from its
constituent fields at a given index. This enables idiomatic access to
series data using indexing like `(my_ticks 0)`.
The `push-series` function has also been updated to correctly handle the
structure of records when pushing data into a `RecordSeries`.
Introduces a new RTL module for handling series data, enabling efficient
storage and retrieval of time-series data in a Struct-of-Arrays (SoA)
format.
Includes `SeriesView`, a zero-copy wrapper that provides a fast,
read-only interface to individual columns within a `RecordSeries`,
optimizing performance for data access in the VM.
Registers new native functions `create-series` and `push-series` for
managing series data.
This commit refactors the internal representation of `BoundKind::Record`
to store a `RecordLayout` and a `Vec` of values, rather than a `Vec` of
key-value pairs. This change simplifies the representation and improves
efficiency by decoupling the record's structure from its specific values
during compilation and analysis.
The `RecordLayout` now defines the structure of the record, and the
values are stored in a separate vector, ordered according to the layout.
This allows for better optimization and type checking, as the record's
shape is explicitly defined and immutable once created.
These changes update the benchmark numbers and repeat counts in the
example files. The benchmark results have slightly varied, and these
updates reflect the most recent measurements.
This commit updates the benchmark results for various examples. The
benchmark times and repeat counts have been adjusted to reflect current
performance characteristics.
This commit introduces a new prelude file that defines the `while` macro
and registers it during environment bootstrapping. The `again` macro has
been updated to accept multiple arguments for its recursive call, and
several examples have been adjusted to reflect this change.
Additionally, the `MacroRegistry` in the compiler is now cloneable and
can be moved out of the `MacroExpander`.
Add a new Markdown document detailing the AST node structure as it
evolves through the compiler's different stages. This includes a diagram
illustrating the data flow and relationships between the generic
`Node<K, T>` structure and its specialized forms (`UntypedNode`,
`BoundNode`, `TypedNode`, `AnalyzedNode`). The document explains the
`kind` and `ty` payloads for each stage and their significance.
Also removes a deleted file related to a previous optimization plan and
adds new files for future optimization plans.
Introduces a new `RecordLayout` system for efficient and type-safe
record handling.
Key changes:
- `RecordLayout` struct with `O(1)` field lookup using FMap
optimization.
- Field accessors (`.name`) are now first-class callable values.
- Parser recognizes dot-prefixed identifiers as field accessors.
- Binder and TypeChecker handle field accessors.
- Optimizer transforms field accessor calls into specialized `GetField`
nodes.
- VM executes `GetField` efficiently by directly accessing record
values.
- Adds a new `records.myc` example showcasing record features.
- Improves comparison logic for records to use pointer equality for
layout.
These tests were for record destructuring, which has been removed.
The code for record destructuring was also removed from the type checker
and types modules.
This commit introduces newtype wrappers for `LocalSlot`, `UpvalueIdx`,
and `GlobalIdx` to improve type safety and clarity. These wrappers
replace direct use of `u32` for indices, making the code more robust and
easier to understand.
The changes include:
- Defining `LocalSlot`, `UpvalueIdx`, and `GlobalIdx` structs.
- Implementing `Display` for these new types to provide user-friendly
output.
- Updating the `Address` enum to use these new types.
- Modifying various compiler components (analyzer, binder, optimizer,
type checker, environment, VM) to use the new types.
- Adjusting tests to accommodate the changes.
This commit consolidates `DefLocal` and `DefGlobal` into a single
`Define` bound kind. This simplifies the AST and makes it more
consistent.
It also introduces `DeclarationKind` to differentiate between variable
and parameter definitions.
The `UsageInfo` struct has been updated to include tracking for
`used_upvalues` and `assigned_upvalues`. The `collect_usage` function
has been modified to handle these new fields when encountering
`Address::Upvalue`.
The `map_params_to_args` function now takes `body_usage` as an argument
to ensure that parameters assigned to or used in the body are correctly
substituted. A new helper function, `collect_parameter_slots_set`, has
been added to gather all parameter slots within a pattern.
The inlining logic in `Optimizer::inline_call` has been enhanced to
prevent inlining if a parameter is used or assigned in the function body
but cannot be substituted. This addresses a bug where aggressive
inlining could lead to incorrect code generation when parameters were
reassigned.
A new integration test, `test_closure_reassignment_optimization_bug`,
has been added to specifically target and verify the fix for this
inlining issue.
Refine the type checker to correctly handle destructuring of various
collection types (tuples, vectors, matrices, lists, records).
Additionally, prevent implicit type coercion for function arguments,
ensuring that only tuples are passed to functions expecting tuple
arguments. This avoids unexpected behavior where vectors or matrices
might be treated as tuples.
Add a new example file demonstrating pattern matching and destructuring
rules.
Brummel
Michael Schimmel