When a generic function indexes the same TypeVar parameter multiple
times,
all resulting TypeVars must share a single element type. This change
ensures
that subsequent indexing operations unify their fresh TypeVar with an
existing
one if the parameter has already been indexed, preventing incorrect type
inference and false negatives.
A new test, `test_let_poly_multi_index_all_results_typed`, is added to
verify this behavior.
Introduce `StaticType::Forall` to represent universally quantified
types. This enables Hindley-Milner's let-polymorphism, allowing
functions to be generalized and reused with different concrete types.
- **Generalization at `def`:** Function values are now generalized using
`self.generalize` when they are defined, wrapping their type in
`Forall`. This occurs at `def` boundaries.
- **Instantiation at use:** Function types are instantiated with fresh
type variables at each call site using `self.instantiate`. This
ensures that different uses of the same polymorphic function do not
interfere with each other's type inference.
- **Value restriction:** Only function-typed definitions are
generalized. Mutable state like series and scalars remain monomorphic
to prevent unexpected behavior.
- **Type context awareness:** The `generalize` function considers the
current type context to avoid quantifying type variables that are
still in use elsewhere.
Feat: Add Forall type for polymorphism
Introduces the `Forall` type to support polymorphic functions and enable
let-polymorphism.
- `Forall(vars, body)`: Represents a universally quantified type where
`vars` are the type variables bound by this quantification, and `body`
is the type itself.
- `TypeChecker::generalize`: Wraps a type in `Forall` when a `def`
boundary is encountered for function-typed values. This ensures that
each use site of a polymorphic function gets its own instantiation.
- `TypeChecker::instantiate`: Replaces the type variables within a
`Forall` type with fresh ones. This is performed at each call site of
a polymorphic function.
- Updates `TypeChecker::unify` and `TypeChecker::apply_subst` to
correctly handle `Forall` types, ensuring that bound variables within
a `Forall` are not affected by global substitutions and that `Forall`
types are instantiated before unification.
- Adds a new example script `examples/Propa.myc` to demonstrate basic
usage.
- Includes a regression test
`test_let_poly_non_series_callable_no_false_positive` to ensure that
passing non-series callables to generic functions does not lead to
type errors.
- Introduces `TypeChecker::bind_var` to handle lazy propagation of
index-call constraints, crucial for correctly typing series lookups
like `(s 0)`.
This commit introduces a new test suite for HM Let-Polymorphism.
These tests verify that user-defined generic functions work correctly
across multiple series element types without type conflicts. This is
crucial for ensuring that the type inference system can handle
polymorphic functions properly, especially when they are applied to
different types. The tests cover scenarios like generic accessors,
runtime correctness with different series types, record series, and
higher-order functions. They also include a test for the value
restriction to ensure that series bindings are not generalized, which
would lead to incorrect type errors.
This commit introduces implicit numeric widening from Int to Float and a
corresponding record promotion mechanism.
When pushing values into a series, if the series' element type is a
TypeVar,
and the pushed value is of a different numeric type (e.g., Int and
Float),
the TypeVar will be unified with the wider type (Float). This ensures
that
operations on series elements handle type differences gracefully.
The record promotion handles cases where two records are involved, and
their
fields have compatible types, either identical or through numeric
widening.
This allows for more flexible type inference in complex structures.
When an identifier is used as an upvalue within a nested scope (e.g., a
`while` loop), the type checker needs to apply the global substitution
map to resolve `TypeVar`s that might have been determined in outer
scopes. This ensures that the correct type is used even if
`ctx.set_type` couldn't propagate the change directly through the
upvalue address.
This fix also includes a regression test to specifically cover this
scenario with series and closures.
Introduce `unify_matched_overload` to handle type variable propagation
for overloaded functions. This ensures that when an overload is chosen,
its concrete parameter types are unified with the actual argument types.
This is crucial for resolving type variables nested within closures
called through overloaded functions.
Additionally, this commit:
- Allows `series` to infer schema type from `push` calls, simplifying
its signature.
- Adds a fallback to `ValueSeries` in `rtl::series` if schema injection
fails.
- Updates `StaticType::TypeVar` to return `Any` when accessed,
simplifying field access logic.
- Adjusts tests to reflect the simplified `series` signature.
The `series` constructor now only accepts the `lookback` limit. The
element type is inferred by the type checker and implicitly added as a
schema argument during compilation. This simplifies the API and
centralizes type inference.
This change also includes:
- Updates to example scripts (`HMA.myc`, `sma.myc`, `soa_series.myc`) to
reflect the new `series` signature.
- Modifications to the `TypeChecker` to handle the inferred schema
injection.
- An addition of a regression test for type inference through nested
closures with `series`.
- Removal of `#[allow(dead_code)]` from several `TypeChecker` methods as
they are now used.
- Update to `rtl/prelude.myc` macro `cache`.
- Update to `rtl/series/mod.rs` to reflect new signature.
- Update to `ast/types.rs` to allow `series` to be indexed with an
integer to retrieve its element type.
This commit refactors the `Environment` struct to better manage the Rust
Runtime Library (RTL) bootstrap state.
A new `Rtl` struct is introduced to hold a frozen snapshot of the
initial RTL state, including scopes, types, purity, values, and
documentation. This snapshot is created once after `rtl::register()`
completes.
The `Environment::new()` constructor is updated to perform a two-phase
bootstrap: first, it registers RTL functions, and then it freezes the
scope 0 and captures the RTL state into the new `Rtl` struct.
A new constructor, `Environment::from_rtl()`, is added. This allows
creating independent execution environments that all share the same
underlying RTL state. This is crucial for isolation between different
execution contexts, as user definitions and loaded modules will not leak
between environments created from the same `Rtl` snapshot.
The `Environment::list_bindings` and `Environment::list_rtl_docs`
methods are updated to use the `rtl` field for accessing the RTL scope
and documentation, reinforcing the concept of a shared, immutable RTL
state.
A new test, `test_environments_from_shared_rtl_are_independent`, is
added to verify that environments created from the same `Rtl` snapshot
are indeed independent and do not leak user-defined bindings.
The `instantiate` method in `Environment` has been refactored to return
`Result<Rc<Closure>, String>` instead of `Rc<NativeFunction>`. This
change separates the creation of a `Closure` from its execution,
allowing the caller to manage the `VM` lifecycle and choose the
appropriate execution strategy (e.g., single run vs. repeated benchmark
iterations).
The `NativeFunction` type is now exclusively used for true Rust
intrinsics. The benchmark runner has also been updated to reuse a single
`VM` across iterations, improving performance by avoiding repeated VM
allocations.
Introduce `ModuleLoader` to handle `#use` directives and dependency
resolution. This involves adding `tempfile` dependency, modifying
`Cargo.toml` and `Cargo.lock`, creating a new `module_loader.rs` file,
and updating `environment.rs` to use the new module.
Also, add comprehensive tests for `Diagnostics::format_errors`,
`CompilationResult`, and the new `ModuleLoader` functionality. These
tests cover various scenarios including empty diagnostics, multiple
errors, error formatting, compilation success/failure, and complex
dependency loading with topological ordering and search path resolution.
This commit introduces support for unhygienic macro expansion using the
`~ident` syntax. When an identifier is prefixed with a tilde (`~`)
within a macro template, it is no longer subject to hygienic renaming.
Instead, it directly refers to a binding in the call site's environment.
This allows macros to interact with variables defined outside the
macro's scope, such as:
- Modifying call-site variables.
- Capturing call-site variables as upvalues in closures.
- Defining new variables that are visible in the call site.
A new test case `test_macro_tilde_nonparam_assign` has been added to
verify this functionality. Previously, unhygienic escapes were
implicitly handled by `SyntaxKind::Identifier` if they were not macro
parameters. This change makes this behavior explicit and correctly
handles non-parameter identifiers by returning a bare symbol.
Removes outdated tests for macro expansion, destructuring, and if branch
scope leaks. These tests were either superseded by newer, more accurate
tests or are no longer relevant to the current implementation of the
compiler's scoping rules. The remaining test correctly asserts that
defining the same variable in different 'if' branches is functional.
Introduce `free_slots` in `StackAllocator` to keep track of physical
slots that have been freed. When mapping a new virtual slot, the
allocator first attempts to reuse a slot from the `free_slots` vector
before allocating a new one.
This change also includes a new function `collect_scope_locals` that
identifies non-captured local variables defined within a block. After
these expressions are lowered, the allocator reclaims the physical slots
associated with these locals, making them available for reuse by
subsequent blocks.
A new test `test_slot_reuse_non_overlapping_scopes` is added to verify
that distinct scopes correctly reuse stack slots, ensuring optimal stack
usage.
The optimizer now correctly handles empty blocks by collapsing them into
Nop nodes. It also unwraps blocks containing a single expression, making
the AST more concise. This change improves AST simplification by
ensuring that redundant block structures are removed.
Introduces a new AST emitter module responsible for serializing AST
nodes back into Myc source code. This emitter preserves leading comments
and whitespace, enabling a round-trip guarantee for the parser and
emitter.
The documentation in `docs/BNF.md` has been updated to reflect the new
syntax rules for comments, including single-line comments (`;`) and
documentation comments (`;;`). The rules clarify that comments must be
whole lines and that inline comments are disallowed.
The lexer and AST node structures have been updated to accommodate
`CommentLine` variants, allowing for different types of comments
(regular, documentation, and blank lines) to be stored and processed.
A new test suite `tests/comment_roundtrip.rs` has been added to verify
the round-trip property of the parser and emitter across all example
`.myc` files. This test ensures that parsing source code, emitting it,
and then parsing the emitted code again results in the same output
string.
The `ast.rs` binary now includes a `--emit` flag that uses the new
emitter to output the parsed source code, facilitating debugging and
testing of the emitter itself.
This commit introduces bidirectional type inference, enabling the
compiler to infer lambda parameter types based on the context of their
usage. This is particularly useful for functions like `pipe`, where a
lambda's signature can be deduced from the types of the streams it
operates on.
Key changes include:
- **`extract_lambda_param_hints` function:** This new helper function in
`type_checker.rs` is responsible for extracting expected parameter
types for a lambda based on the callee's signature and already known
argument types.
- **`check_lambda_with_param_hints` function:** This function allows
type-checking a lambda node using provided parameter type hints,
preserving upvalue types from the enclosing scope.
- **Call expression type checking:** The `check_call` function now
phases its argument type checking. Non-lambda arguments are typed
first, and then lambda arguments are typed using hints derived from
`extract_lambda_param_hints`.
- **`pipe_arg_hint_resolver`:** This new resolver for the `pipe`
function's `PolymorphicFn` type allows it to provide specific type
hints for its lambda argument based on the types of its stream inputs
(single stream, vector of streams, or tuple of streams).
- **`StaticType::PolymorphicFn` update:** The `PolymorphicFn` enum
variant has been extended to include `resolve_arg_hints`, enabling
functions to provide lambda parameter hints during bidirectional type
inference.
- **New tests:** Added tests to verify the correct inference of lambda
parameters for `pipe` with vector and tuple inputs, ensuring that
inner stream types are correctly propagated. Also, a test to confirm
`pipe` with an optional return still produces `Stream(Float)`.
The `bound_nodes.rs` file has been removed and its contents have been
moved to `src/ast/nodes.rs`. This consolidates all AST node definitions
into a single module, improving organization and maintainability.
The `compiler` modules now import these definitions from
`crate::ast::nodes` instead of `crate::ast::compiler::bound_nodes`.
This commit refactors the test suite by removing the monolithic
`integration_test.rs` file.
The tests have been categorized and moved into dedicated modules within
the `tests/` directory:
- `tests/parser.rs`: Contains tests for parsing various literal values.
- `tests/destructuring.rs`: Houses tests related to destructuring
syntax.
- `tests/closures.rs`: Includes tests for closure behavior and related
optimizations.
- `tests/optimizer.rs`: Contains tests specifically targeting optimizer
logic and regressions.
- `tests/records.rs`: Holds tests for record syntax and associated
optimizations.
- `tests/rtl.rs`: Contains tests for runtime library operators and
functions.
- `tests/pipeline.rs`: Tests for pipeline functionality.
- `tests/macros.rs`: Tests for macro expansion and related issues.
- `tests/error_recovery.rs`: Includes tests for compiler error handling
and recovery mechanisms.
- `tests/examples.rs`: Verifies the execution of example scripts.
This reorganization improves test discoverability and maintainability.
The `lib.rs` file has also been updated to reflect these changes by
removing the reference to the old `integration_test` module.
The `type_checker.rs` file has had some tests removed, as they are now
covered by the more specific tests in `tests/error_recovery.rs`.
Brummel