Refactor: Implement Record Layouts and Optimized Field Access
Introduces a new `RecordLayout` system for efficient and type-safe record handling. Key changes: - `RecordLayout` struct with `O(1)` field lookup using FMap optimization. - Field accessors (`.name`) are now first-class callable values. - Parser recognizes dot-prefixed identifiers as field accessors. - Binder and TypeChecker handle field accessors. - Optimizer transforms field accessor calls into specialized `GetField` nodes. - VM executes `GetField` efficiently by directly accessing record values. - Adds a new `records.myc` example showcasing record features. - Improves comparison logic for records to use pointer equality for layout.
This commit is contained in:
Michael Schimmel
@@ -0,0 +1,55 @@
|
||||
# Plan: Record Layout & Optimized Field Access
|
||||
|
||||
This document outlines the implementation plan for porting Delphi's `TKeywordMappingRegistry` optimizations to the Rust-based Myc AST compiler and integrating first-class field accessors.
|
||||
|
||||
## 1. Core Data Structures (`src/ast/types.rs`)
|
||||
|
||||
### RecordLayout (The Schema)
|
||||
- **Concept:** Replaces `IKeywordMapping<IStaticType>`.
|
||||
- **Interning:** Managed via a global `OnceLock<Mutex<HashMap<..., Arc<RecordLayout>>>>`.
|
||||
- **FMap Optimization:**
|
||||
- Stores `fields: Vec<(Keyword, StaticType)>`.
|
||||
- Maintains a lookup array `Vec<i32>` mapping `Keyword.idx - first_key` to the index in the fields array.
|
||||
- Provides `index_of(Keyword) -> Option<usize>` with **O(1)** complexity.
|
||||
- **Thread Safety:** Uses `Arc` for global sharing across script threads.
|
||||
|
||||
### Value & StaticType Updates
|
||||
- `Value::Record` points to an `Arc<RecordLayout>` and a flat `Rc<Vec<Value>>`.
|
||||
- `Value::FieldAccessor(Keyword)` added as a first-class callable value (e.g., `.name`).
|
||||
- `StaticType::Record` now holds `Arc<RecordLayout>`.
|
||||
|
||||
## 2. Compiler Pipeline
|
||||
|
||||
### Parser (`src/ast/parser.rs`)
|
||||
- Identifiers starting with `.` (e.g., `.age`) are parsed as `UntypedKind::FieldAccessor(Keyword)`.
|
||||
- This ensures interning happens immediately; strings are never used for field access in subsequent phases.
|
||||
|
||||
### Binder (`src/ast/compiler/binder.rs`)
|
||||
- Maps `UntypedKind::FieldAccessor` to `BoundKind::FieldAccessor`.
|
||||
|
||||
### TypeChecker (`src/ast/compiler/type_checker.rs`)
|
||||
- **Analysis Only:** When encountering `Call { callee: FieldAccessor, args: [rec] }`:
|
||||
- Validates that `rec` is a Record.
|
||||
- Uses `RecordLayout::index_of` to find the field's type.
|
||||
- Sets the call's return type accordingly.
|
||||
- **Crucial:** Does NOT modify the AST structure. It remains a `Call`.
|
||||
|
||||
### Optimizer (`src/ast/compiler/optimizer/engine.rs`)
|
||||
- **Transformation:** Recognizes the `Call { callee: FieldAccessor, ... }` pattern.
|
||||
- Transforms it into a specialized `BoundKind::GetField { rec, field: Keyword }` node.
|
||||
|
||||
## 3. Execution (`src/ast/vm.rs`)
|
||||
|
||||
### Optimized Path (`GetField`)
|
||||
- The VM executes `GetField` by:
|
||||
1. Evaluating the record.
|
||||
2. Calling `layout.index_of(field)` (O(1)).
|
||||
3. Fetching the value from the flat array.
|
||||
|
||||
### Functional Path (`FieldAccessor`)
|
||||
- If `.field` is passed as a value (e.g., `(map .name users)`), the VM treats the `FieldAccessor` value as a standard function that takes one argument and performs the lookup.
|
||||
|
||||
## 4. Benefits
|
||||
- **Type Identity:** `Arc::ptr_eq` allows instant type comparison.
|
||||
- **Performance:** Field access in loops is O(1) without hash lookups.
|
||||
- **Safety:** The TypeChecker remains a pure analysis phase as per design requirements.
|
||||
@@ -1,5 +1,5 @@
|
||||
;; Benchmark: 351ns
|
||||
;; Benchmark-Repeat: 5716
|
||||
;; Benchmark: 401ns
|
||||
;; Benchmark-Repeat: 5016
|
||||
;; Financial DSL Macro example
|
||||
;; Demonstrates generating complex records from simple parameters.
|
||||
;; Output: {:price 100, :size 2, :total 200}
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
;; Benchmark: 3.4us
|
||||
;; Benchmark-Repeat: 573
|
||||
;; Output: ["Alice" 101 :admin "Zürich" ["Alice" "Bob"] true]
|
||||
|
||||
; ---------------------------------------------------------
|
||||
; Records & Optimized Field Access Showcase
|
||||
; ---------------------------------------------------------
|
||||
|
||||
(do
|
||||
; 1. Record Creation
|
||||
; Records use interned layouts for O(1) access and low memory overhead.
|
||||
(def user {:id 101 :name "Alice" :role :admin})
|
||||
|
||||
; 2. Optimized Field Access
|
||||
; These calls are transformed by the optimizer into specialized instructions.
|
||||
(def name (.name user))
|
||||
(def id (.id user))
|
||||
|
||||
; 3. First-Class Field Accessors
|
||||
; Accessors start with a dot and can be passed around like functions.
|
||||
(def get-role .role)
|
||||
(def role (get-role user))
|
||||
|
||||
; 4. Nested Records
|
||||
(def company {
|
||||
:name "Myc Corp"
|
||||
:location {:city "Zürich" :zip 8000}
|
||||
:employees [user {:id 102 :name "Bob" :role :dev}]
|
||||
})
|
||||
|
||||
; Deep access (also optimized)
|
||||
(def city (.city (.location company)))
|
||||
|
||||
; 5. Higher-Order usage
|
||||
; Accessors are perfect for mapping over data.
|
||||
(def names ((fn [f list]
|
||||
(do
|
||||
(def [e1 e2] list)
|
||||
[(f e1) (f e2)]
|
||||
))
|
||||
.name (.employees company)))
|
||||
|
||||
; 6. Structural Identity
|
||||
; Records with the same layout share memory; equality checks values.
|
||||
(def is-equal (= {:x 1 :y 2} {:x 1 :y 2}))
|
||||
|
||||
; Return a summary of all tested features
|
||||
[name id role city names is-equal]
|
||||
)
|
||||
@@ -83,6 +83,20 @@ impl<'a> Analyzer<'a> {
|
||||
)
|
||||
}
|
||||
|
||||
BoundKind::FieldAccessor(k) => (BoundKind::FieldAccessor(*k), Purity::Pure),
|
||||
|
||||
BoundKind::GetField { rec, field } => {
|
||||
let rec_m = self.visit(Rc::new((**rec).clone()));
|
||||
let p = rec_m.ty.purity;
|
||||
(
|
||||
BoundKind::GetField {
|
||||
rec: Box::new(rec_m),
|
||||
field: *field,
|
||||
},
|
||||
p,
|
||||
)
|
||||
}
|
||||
|
||||
BoundKind::Set { addr, value } => {
|
||||
let val_m = self.visit(Rc::new((**value).clone()));
|
||||
(
|
||||
@@ -313,6 +327,9 @@ impl NodeExt for BoundKind<crate::ast::types::StaticType> {
|
||||
BoundKind::Define { value, .. } | BoundKind::Set { value, .. } => {
|
||||
f(value);
|
||||
}
|
||||
BoundKind::GetField { rec, .. } => {
|
||||
f(rec);
|
||||
}
|
||||
BoundKind::Lambda { params, body, .. } => {
|
||||
f(params);
|
||||
f(body);
|
||||
|
||||
@@ -189,6 +189,10 @@ impl Binder {
|
||||
Err("Unexpected 'Parameter' node in general binder context. This should be handled via 'bind_pattern'.".to_string())
|
||||
}
|
||||
|
||||
UntypedKind::FieldAccessor(k) => {
|
||||
Ok(self.make_node(node.identity.clone(), BoundKind::FieldAccessor(*k)))
|
||||
}
|
||||
|
||||
UntypedKind::If {
|
||||
cond,
|
||||
then_br,
|
||||
|
||||
@@ -97,6 +97,15 @@ pub enum BoundKind<T = ()> {
|
||||
captured_by: Vec<Identity>,
|
||||
},
|
||||
|
||||
/// A first-class field accessor (e.g. .name)
|
||||
FieldAccessor(crate::ast::types::Keyword),
|
||||
|
||||
/// Specialized field access (O(1) via RecordLayout)
|
||||
GetField {
|
||||
rec: Box<BoundNode<T>>,
|
||||
field: crate::ast::types::Keyword,
|
||||
},
|
||||
|
||||
If {
|
||||
cond: Box<BoundNode<T>>,
|
||||
then_br: Box<BoundNode<T>>,
|
||||
@@ -188,6 +197,10 @@ where
|
||||
captured_by: cb,
|
||||
},
|
||||
) => na == nb && aa == ab && ka == kb && va == vb && ca == cb,
|
||||
(BoundKind::FieldAccessor(a), BoundKind::FieldAccessor(b)) => a == b,
|
||||
(BoundKind::GetField { rec: ra, field: fa }, BoundKind::GetField { rec: rb, field: fb }) => {
|
||||
ra == rb && fa == fb
|
||||
}
|
||||
(
|
||||
BoundKind::If {
|
||||
cond: ca,
|
||||
@@ -273,6 +286,8 @@ impl<T> BoundKind<T> {
|
||||
};
|
||||
format!("DEF_{}({}, {:?})", k_str, name.name, addr)
|
||||
}
|
||||
BoundKind::FieldAccessor(k) => format!("FIELD_ACCESSOR(.{})", k.name()),
|
||||
BoundKind::GetField { field, .. } => format!("GET_FIELD(.{})", field.name()),
|
||||
BoundKind::If { .. } => "IF".to_string(),
|
||||
BoundKind::Destructure { .. } => "DESTRUCTURE".to_string(),
|
||||
BoundKind::Lambda {
|
||||
|
||||
@@ -47,6 +47,15 @@ impl CapturePass {
|
||||
};
|
||||
}
|
||||
|
||||
BoundKind::FieldAccessor(_) => {}
|
||||
|
||||
BoundKind::GetField { rec, field } => {
|
||||
node.kind = BoundKind::GetField {
|
||||
rec: Box::new(Self::transform(*rec, capture_map)),
|
||||
field,
|
||||
};
|
||||
}
|
||||
|
||||
BoundKind::Destructure { pattern, value } => {
|
||||
node.kind = BoundKind::Destructure {
|
||||
pattern: Box::new(Self::transform(*pattern, capture_map)),
|
||||
|
||||
@@ -66,6 +66,17 @@ impl Dumper {
|
||||
self.log(&format!("Get: {} ({:?})", name.name, addr), node)
|
||||
}
|
||||
|
||||
BoundKind::FieldAccessor(k) => {
|
||||
self.log(&format!("FieldAccessor: .{}", k.name()), node)
|
||||
}
|
||||
|
||||
BoundKind::GetField { rec, field } => {
|
||||
self.log(&format!("GetField: .{}", field.name()), node);
|
||||
self.indent += 1;
|
||||
self.visit(rec);
|
||||
self.indent -= 1;
|
||||
}
|
||||
|
||||
BoundKind::Set { addr, value } => {
|
||||
self.log(&format!("Set: {:?}", addr), node);
|
||||
self.indent += 1;
|
||||
|
||||
@@ -112,6 +112,27 @@ impl Optimizer {
|
||||
)
|
||||
}
|
||||
|
||||
BoundKind::FieldAccessor(k) => (BoundKind::FieldAccessor(k), node.ty.clone()),
|
||||
|
||||
BoundKind::GetField { ref rec, field } => {
|
||||
let rec_opt = self.visit_node((**rec).clone(), sub, path);
|
||||
|
||||
// Constant folding for Field Access
|
||||
if let BoundKind::Constant(Value::Record(layout, values)) = &rec_opt.kind
|
||||
&& let Some(idx) = layout.index_of(field)
|
||||
{
|
||||
return folder.make_constant_node(values[idx].clone(), &node);
|
||||
}
|
||||
|
||||
(
|
||||
BoundKind::GetField {
|
||||
rec: Box::new(rec_opt),
|
||||
field,
|
||||
},
|
||||
node.ty.clone(),
|
||||
)
|
||||
}
|
||||
|
||||
BoundKind::Set { addr, value } => {
|
||||
let value = Box::new(self.visit_node(*value, sub, path));
|
||||
if let BoundKind::Constant(val) = &value.kind {
|
||||
@@ -175,6 +196,23 @@ impl Optimizer {
|
||||
let args = self.visit_node(*args, sub, path);
|
||||
|
||||
if self.enabled {
|
||||
// Optimized Field Access Transformation
|
||||
if let BoundKind::FieldAccessor(k) = &callee.kind
|
||||
&& let BoundKind::Tuple { elements } = &args.kind
|
||||
&& elements.len() == 1
|
||||
{
|
||||
let rec = elements[0].clone();
|
||||
let metrics = node.ty.clone();
|
||||
return Node {
|
||||
identity: node.identity,
|
||||
kind: BoundKind::GetField {
|
||||
rec: Box::new(rec),
|
||||
field: *k,
|
||||
},
|
||||
ty: metrics,
|
||||
};
|
||||
}
|
||||
|
||||
let mut arg_nodes = Vec::new();
|
||||
self.flatten_tuple(args.clone(), &mut arg_nodes);
|
||||
|
||||
@@ -542,11 +580,6 @@ impl Optimizer {
|
||||
self.flatten_tuple(el, into);
|
||||
}
|
||||
}
|
||||
BoundKind::Record { fields } => {
|
||||
for (_, v) in fields {
|
||||
self.flatten_tuple(v, into);
|
||||
}
|
||||
}
|
||||
BoundKind::Nop => {}
|
||||
_ => into.push(node),
|
||||
}
|
||||
|
||||
@@ -68,6 +68,9 @@ impl UsageInfo {
|
||||
self.assigned.insert(*addr);
|
||||
self.collect(value);
|
||||
}
|
||||
BoundKind::GetField { rec, .. } => {
|
||||
self.collect(rec);
|
||||
}
|
||||
BoundKind::Lambda {
|
||||
params,
|
||||
body,
|
||||
@@ -147,7 +150,10 @@ impl UsageInfo {
|
||||
BoundKind::Expansion { bound_expanded, .. } => {
|
||||
self.collect(bound_expanded);
|
||||
}
|
||||
_ => {}
|
||||
BoundKind::Again { args } => {
|
||||
self.collect(args);
|
||||
}
|
||||
BoundKind::Nop | BoundKind::FieldAccessor(_) | BoundKind::Extension(_) => {}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -141,6 +141,11 @@ impl TCO {
|
||||
addr: *addr,
|
||||
name: name.clone(),
|
||||
},
|
||||
BoundKind::FieldAccessor(k) => BoundKind::FieldAccessor(*k),
|
||||
BoundKind::GetField { rec, field } => BoundKind::GetField {
|
||||
rec: Box::new(Self::transform(Rc::new((**rec).clone()), false)),
|
||||
field: *field,
|
||||
},
|
||||
BoundKind::Nop => BoundKind::Nop,
|
||||
BoundKind::Expansion {
|
||||
original_call,
|
||||
|
||||
@@ -226,9 +226,10 @@ impl TypeChecker {
|
||||
StaticType::Vector(inner, _) => (**inner).clone(),
|
||||
StaticType::Matrix(inner, _) => (**inner).clone(), // Matrix flat iteration or row? Wait. Matrix destructuring logic is tricky. But for now let's focus on Record.
|
||||
StaticType::List(inner) => (**inner).clone(),
|
||||
StaticType::Record(fields) => fields
|
||||
StaticType::Record(layout) => layout
|
||||
.fields
|
||||
.get(i)
|
||||
.map(|(_, ty)| ty.clone())
|
||||
.map(|(_, ty): &(crate::ast::types::Keyword, StaticType)| ty.clone())
|
||||
.unwrap_or(StaticType::Any),
|
||||
_ => StaticType::Any,
|
||||
};
|
||||
@@ -296,6 +297,36 @@ impl TypeChecker {
|
||||
)
|
||||
}
|
||||
|
||||
BoundKind::FieldAccessor(k) => (BoundKind::FieldAccessor(k), StaticType::FieldAccessor(k)),
|
||||
|
||||
BoundKind::GetField { rec, field } => {
|
||||
let rec_typed = self.check_node(*rec, ctx)?;
|
||||
let field_ty = match &rec_typed.ty {
|
||||
StaticType::Record(layout) => {
|
||||
if let Some(idx) = layout.index_of(field) {
|
||||
layout.fields[idx].1.clone()
|
||||
} else {
|
||||
return Err(format!("Record does not have field :{}", field.name()));
|
||||
}
|
||||
}
|
||||
StaticType::Any => StaticType::Any,
|
||||
_ => {
|
||||
return Err(format!(
|
||||
"Cannot access field :{} on non-record type {}",
|
||||
field.name(),
|
||||
rec_typed.ty
|
||||
));
|
||||
}
|
||||
};
|
||||
(
|
||||
BoundKind::GetField {
|
||||
rec: Box::new(rec_typed),
|
||||
field,
|
||||
},
|
||||
field_ty,
|
||||
)
|
||||
}
|
||||
|
||||
BoundKind::Set { addr, value } => {
|
||||
let val_typed = self.check_node(*value, ctx)?;
|
||||
let ty = val_typed.ty.clone();
|
||||
@@ -545,11 +576,12 @@ impl TypeChecker {
|
||||
|
||||
typed_fields.push((kt, vt));
|
||||
}
|
||||
let layout = crate::ast::types::RecordLayout::get_or_create(fields_ty);
|
||||
(
|
||||
BoundKind::Record {
|
||||
fields: typed_fields,
|
||||
},
|
||||
StaticType::Record(Rc::new(fields_ty)),
|
||||
StaticType::Record(layout),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -801,12 +833,12 @@ mod tests {
|
||||
#[test]
|
||||
fn test_inference_record() {
|
||||
use crate::ast::types::Keyword;
|
||||
let typed = check_source("{:x 1, :y 0.3}");
|
||||
let typed = check_source("{:x 1 :y 0.3}");
|
||||
let ty = get_ret_type(&typed);
|
||||
if let StaticType::Record(fields) = ty {
|
||||
assert_eq!(fields.len(), 2);
|
||||
assert_eq!(fields[0], (Keyword::intern("x"), StaticType::Int));
|
||||
assert_eq!(fields[1], (Keyword::intern("y"), StaticType::Float));
|
||||
if let StaticType::Record(layout) = ty {
|
||||
assert_eq!(layout.fields.len(), 2);
|
||||
assert_eq!(layout.fields[0], (Keyword::intern("x"), StaticType::Int));
|
||||
assert_eq!(layout.fields[1], (Keyword::intern("y"), StaticType::Float));
|
||||
} else {
|
||||
panic!("Expected Record, got {:?}", ty);
|
||||
}
|
||||
|
||||
@@ -194,6 +194,16 @@ impl Environment {
|
||||
let (bound_ast, captures) = Binder::bind_root(self.global_names.clone(), &expanded_ast)?;
|
||||
let bound_ast = crate::ast::compiler::captures::CapturePass::apply(bound_ast, &captures);
|
||||
|
||||
// Pre-allocate global slots to prevent out-of-bounds during specialization/optimization
|
||||
{
|
||||
let mut values = self.global_values.borrow_mut();
|
||||
let names = self.global_names.borrow();
|
||||
let max_idx = names.values().map(|(idx, _)| idx.0).max().unwrap_or(0);
|
||||
if (max_idx as usize) >= values.len() {
|
||||
values.resize((max_idx + 1) as usize, Value::Void);
|
||||
}
|
||||
}
|
||||
|
||||
LambdaCollector::collect(&bound_ast, &mut self.function_registry.borrow_mut());
|
||||
|
||||
let checker = TypeChecker::new(self.global_types.clone());
|
||||
|
||||
@@ -65,6 +65,8 @@ pub enum UntypedKind {
|
||||
Constant(Value),
|
||||
Identifier(Symbol),
|
||||
Parameter(Symbol),
|
||||
/// A first-class field accessor (e.g. .name)
|
||||
FieldAccessor(crate::ast::types::Keyword),
|
||||
If {
|
||||
cond: Box<Node<UntypedKind>>,
|
||||
then_br: Box<Node<UntypedKind>>,
|
||||
|
||||
@@ -100,6 +100,9 @@ impl<'a> Parser<'a> {
|
||||
TokenKind::Keyword(k) => UntypedKind::Constant(Value::Keyword(Keyword::intern(&k))),
|
||||
TokenKind::Identifier(id) => match id.as_ref() {
|
||||
"..." => UntypedKind::Nop,
|
||||
s if s.starts_with('.') && s.len() > 1 => {
|
||||
UntypedKind::FieldAccessor(Keyword::intern(&s[1..]))
|
||||
}
|
||||
_ => UntypedKind::Identifier(id.into()),
|
||||
},
|
||||
_ => {
|
||||
|
||||
@@ -366,6 +366,9 @@ fn register_comparison(env: &Environment) {
|
||||
(Value::Bool(a), Value::Bool(b)) => Value::Bool(a == b),
|
||||
(Value::Keyword(a), Value::Keyword(b)) => Value::Bool(a == b),
|
||||
(Value::DateTime(a), Value::DateTime(b)) => Value::Bool(a == b),
|
||||
(Value::Record(la, va), Value::Record(lb, vb)) => {
|
||||
Value::Bool(std::sync::Arc::ptr_eq(la, lb) && va == vb)
|
||||
}
|
||||
(Value::Void, Value::Void) => Value::Bool(true),
|
||||
_ => Value::Bool(false),
|
||||
}
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
use crate::ast::types::{Keyword, Purity, Signature, StaticType, Value};
|
||||
use std::any::TypeId;
|
||||
use std::collections::HashMap;
|
||||
use std::rc::Rc;
|
||||
|
||||
/// Represents a Rust type that can be exposed to the script environment.
|
||||
pub trait Scriptable: 'static + Sized {
|
||||
@@ -149,7 +148,7 @@ impl TypeBuilder {
|
||||
}
|
||||
|
||||
pub fn build(self) -> StaticType {
|
||||
StaticType::Record(Rc::new(self.fields))
|
||||
StaticType::Record(crate::ast::types::RecordLayout::get_or_create(self.fields))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -202,20 +201,16 @@ mod tests {
|
||||
|
||||
// Check static type
|
||||
let st = TypeRegistry::resolve_type::<Person>(®istry);
|
||||
if let StaticType::Record(fields) = st {
|
||||
assert!(fields.iter().any(|(k, _)| *k == Keyword::intern("greet")));
|
||||
if let StaticType::Record(layout) = st {
|
||||
assert!(layout.fields.iter().any(|(k, _)| *k == Keyword::intern("greet")));
|
||||
} else {
|
||||
panic!("Expected Record type");
|
||||
}
|
||||
|
||||
// Check runtime behavior
|
||||
if let Value::Record(r) = wrapped {
|
||||
let idx = r
|
||||
.keys
|
||||
.iter()
|
||||
.position(|k| *k == Keyword::intern("greet"))
|
||||
.unwrap();
|
||||
let greet_fn = &r.values[idx];
|
||||
if let Value::Record(layout, values) = wrapped {
|
||||
let idx = layout.index_of(Keyword::intern("greet")).unwrap();
|
||||
let greet_fn = &values[idx];
|
||||
|
||||
if let Value::Function(f) = greet_fn {
|
||||
let res = (f.func)(vec![]);
|
||||
@@ -251,13 +246,9 @@ mod tests {
|
||||
|
||||
if let Value::Function(f) = factory_val {
|
||||
let instance = (f.func)(vec![Value::Text("Bob".into()), Value::Int(40)]);
|
||||
if let Value::Record(r) = instance {
|
||||
let idx = r
|
||||
.keys
|
||||
.iter()
|
||||
.position(|k| *k == Keyword::intern("greet"))
|
||||
.unwrap();
|
||||
let greet_fn = &r.values[idx];
|
||||
if let Value::Record(layout, values) = instance {
|
||||
let idx = layout.index_of(Keyword::intern("greet")).unwrap();
|
||||
let greet_fn = &values[idx];
|
||||
|
||||
if let Value::Function(gf) = greet_fn {
|
||||
let res = (gf.func)(vec![]);
|
||||
|
||||
+126
-29
@@ -98,6 +98,84 @@ impl Keyword {
|
||||
}
|
||||
}
|
||||
|
||||
/// A shared schema for Records, providing O(1) field lookup via FMap optimization.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RecordLayout {
|
||||
pub fields: Vec<(Keyword, StaticType)>,
|
||||
/// Optimization: maps (Keyword.idx - min_idx) to index in fields.
|
||||
fmap: Vec<i32>,
|
||||
min_idx: u32,
|
||||
}
|
||||
|
||||
impl PartialEq for RecordLayout {
|
||||
fn eq(&self, other: &Self) -> bool {
|
||||
self.fields == other.fields
|
||||
}
|
||||
}
|
||||
|
||||
impl Eq for RecordLayout {}
|
||||
|
||||
impl std::hash::Hash for RecordLayout {
|
||||
fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
|
||||
self.fields.hash(state);
|
||||
}
|
||||
}
|
||||
|
||||
type LayoutMap = HashMap<Vec<(Keyword, StaticType)>, std::sync::Arc<RecordLayout>>;
|
||||
|
||||
static LAYOUT_REGISTRY: OnceLock<Mutex<LayoutMap>> = OnceLock::new();
|
||||
|
||||
impl RecordLayout {
|
||||
pub fn get_or_create(fields: Vec<(Keyword, StaticType)>) -> std::sync::Arc<Self> {
|
||||
let mut reg = LAYOUT_REGISTRY
|
||||
.get_or_init(|| Mutex::new(HashMap::new()))
|
||||
.lock()
|
||||
.unwrap();
|
||||
|
||||
if let Some(layout) = reg.get(&fields) {
|
||||
return layout.clone();
|
||||
}
|
||||
|
||||
let mut min_idx = u32::MAX;
|
||||
let mut max_idx = 0;
|
||||
for (k, _) in &fields {
|
||||
min_idx = min_idx.min(k.0);
|
||||
max_idx = max_idx.max(k.0);
|
||||
}
|
||||
|
||||
let mut fmap = vec![-1; (max_idx - min_idx + 1) as usize];
|
||||
for (i, (k, _)) in fields.iter().enumerate() {
|
||||
fmap[(k.0 - min_idx) as usize] = i as i32;
|
||||
}
|
||||
|
||||
let layout = std::sync::Arc::new(RecordLayout {
|
||||
fields: fields.clone(),
|
||||
fmap,
|
||||
min_idx,
|
||||
});
|
||||
|
||||
reg.insert(fields, layout.clone());
|
||||
layout
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn index_of(&self, key: Keyword) -> Option<usize> {
|
||||
if key.0 < self.min_idx {
|
||||
return None;
|
||||
}
|
||||
let offset = (key.0 - self.min_idx) as usize;
|
||||
if offset >= self.fmap.len() {
|
||||
return None;
|
||||
}
|
||||
let idx = self.fmap[offset];
|
||||
if idx < 0 {
|
||||
None
|
||||
} else {
|
||||
Some(idx as usize)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Interface for custom objects (Closures, Series, Streams)
|
||||
pub trait Object: fmt::Debug {
|
||||
fn type_name(&self) -> &'static str;
|
||||
@@ -107,15 +185,6 @@ pub trait Object: fmt::Debug {
|
||||
/// A shared sequence of values, used by both Tuples and Records.
|
||||
pub type ValueList = Rc<Vec<Value>>;
|
||||
|
||||
/// Internal storage for Records to allow sharing schema (keys) between instances.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct RecordData {
|
||||
/// Names for slots.
|
||||
pub keys: Rc<Vec<Keyword>>,
|
||||
/// The actual values, potentially shared with a Tuple.
|
||||
pub values: ValueList,
|
||||
}
|
||||
|
||||
/// Purity level of an expression or function.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
|
||||
pub enum Purity {
|
||||
@@ -150,7 +219,8 @@ pub enum Value {
|
||||
Text(Rc<str>),
|
||||
Keyword(Keyword),
|
||||
Tuple(ValueList),
|
||||
Record(Rc<RecordData>),
|
||||
Record(std::sync::Arc<RecordLayout>, ValueList),
|
||||
FieldAccessor(Keyword),
|
||||
Function(Rc<NativeFunction>),
|
||||
Object(Rc<dyn Object>), // For compiled Closures and other opaque types
|
||||
Cell(Rc<RefCell<Value>>), // Boxed value for captures
|
||||
@@ -168,7 +238,10 @@ impl PartialEq for Value {
|
||||
(Value::Text(a), Value::Text(b)) => a == b,
|
||||
(Value::Keyword(a), Value::Keyword(b)) => a == b,
|
||||
(Value::Tuple(a), Value::Tuple(b)) => a == b,
|
||||
(Value::Record(a), Value::Record(b)) => a == b,
|
||||
(Value::Record(la, va), Value::Record(lb, vb)) => {
|
||||
std::sync::Arc::ptr_eq(la, lb) && va == vb
|
||||
}
|
||||
(Value::FieldAccessor(a), Value::FieldAccessor(b)) => a == b,
|
||||
(Value::Function(a), Value::Function(b)) => Rc::ptr_eq(a, b),
|
||||
(Value::Object(a), Value::Object(b)) => Rc::ptr_eq(a, b),
|
||||
(Value::Cell(a), Value::Cell(b)) => Rc::ptr_eq(a, b),
|
||||
@@ -214,7 +287,8 @@ pub enum StaticType {
|
||||
Tuple(Vec<StaticType>), // Heterogeneous fixed-size
|
||||
Vector(Box<StaticType>, usize), // Homogeneous fixed-size
|
||||
Matrix(Box<StaticType>, Vec<usize>), // Multi-dimensional homogeneous
|
||||
Record(Rc<Vec<(Keyword, StaticType)>>),
|
||||
Record(std::sync::Arc<RecordLayout>),
|
||||
FieldAccessor(Keyword),
|
||||
Function(Box<Signature>),
|
||||
FunctionOverloads(Vec<Signature>),
|
||||
Object(&'static str),
|
||||
@@ -253,9 +327,9 @@ impl fmt::Display for StaticType {
|
||||
}
|
||||
write!(f, "]>")
|
||||
}
|
||||
StaticType::Record(fields) => {
|
||||
StaticType::Record(layout) => {
|
||||
write!(f, "{{")?;
|
||||
for (i, (k, v)) in fields.iter().enumerate() {
|
||||
for (i, (k, v)) in layout.fields.iter().enumerate() {
|
||||
if i > 0 {
|
||||
write!(f, ", ")?;
|
||||
}
|
||||
@@ -263,6 +337,7 @@ impl fmt::Display for StaticType {
|
||||
}
|
||||
write!(f, "}}")
|
||||
}
|
||||
StaticType::FieldAccessor(k) => write!(f, ".{}", k.name()),
|
||||
StaticType::Function(sig) => {
|
||||
write!(f, "fn({}) -> {}", sig.params, sig.ret)
|
||||
}
|
||||
@@ -312,6 +387,10 @@ impl StaticType {
|
||||
inner.is_assignable_from(&StaticType::Matrix(m_inner.clone(), sub_shape))
|
||||
}
|
||||
}
|
||||
// Records are assignable if their layouts match (Structural identity via interning)
|
||||
(StaticType::Record(a), StaticType::Record(b)) => {
|
||||
std::sync::Arc::ptr_eq(a, b)
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
@@ -320,6 +399,26 @@ impl StaticType {
|
||||
pub fn resolve_call(&self, args_ty: &StaticType) -> Option<StaticType> {
|
||||
match self {
|
||||
StaticType::Any => Some(StaticType::Any),
|
||||
StaticType::FieldAccessor(k) => {
|
||||
// (.name { :name val })
|
||||
// The args_ty should be a Tuple of [Record]
|
||||
let rec_ty = if let StaticType::Tuple(t) = args_ty {
|
||||
t.first()?
|
||||
} else {
|
||||
args_ty
|
||||
};
|
||||
|
||||
match rec_ty {
|
||||
StaticType::Record(layout) => {
|
||||
if let Some(idx) = layout.index_of(*k) {
|
||||
return Some(layout.fields[idx].1.clone());
|
||||
}
|
||||
}
|
||||
StaticType::Any => return Some(StaticType::Any),
|
||||
_ => {}
|
||||
}
|
||||
None
|
||||
}
|
||||
StaticType::Function(sig) => {
|
||||
if sig.params.is_assignable_from(args_ty) {
|
||||
Some(sig.ret.clone())
|
||||
@@ -370,10 +469,12 @@ impl Value {
|
||||
}
|
||||
|
||||
pub fn make_record(keys: Vec<Keyword>, values: Vec<Value>) -> Self {
|
||||
Value::Record(Rc::new(RecordData {
|
||||
keys: Rc::new(keys),
|
||||
values: Rc::new(values),
|
||||
}))
|
||||
let fields: Vec<_> = keys
|
||||
.into_iter()
|
||||
.zip(values.iter().map(|v| v.static_type()))
|
||||
.collect();
|
||||
let layout = RecordLayout::get_or_create(fields);
|
||||
Value::Record(layout, Rc::new(values))
|
||||
}
|
||||
|
||||
pub fn make_function(purity: Purity, func: impl Fn(Vec<Value>) -> Value + 'static) -> Self {
|
||||
@@ -420,14 +521,9 @@ impl Value {
|
||||
StaticType::Tuple(element_types)
|
||||
}
|
||||
}
|
||||
Value::Record(r) => {
|
||||
let mut fields = Vec::with_capacity(r.values.len());
|
||||
for (i, v) in r.values.iter().enumerate() {
|
||||
fields.push((r.keys[i], v.static_type()));
|
||||
}
|
||||
StaticType::Record(Rc::new(fields))
|
||||
}
|
||||
Value::Function(_) => StaticType::Any, // Dynamic function
|
||||
Value::Record(layout, _) => StaticType::Record(layout.clone()),
|
||||
Value::FieldAccessor(k) => StaticType::FieldAccessor(*k),
|
||||
Value::Function(_) => StaticType::Any, // Dynamic function
|
||||
Value::Object(o) => StaticType::Object(o.type_name()),
|
||||
Value::Cell(c) => c.borrow().static_type(),
|
||||
Value::TailCallRequest(_) => StaticType::Any, // Internal state, but typable as Any
|
||||
@@ -460,16 +556,17 @@ impl fmt::Display for Value {
|
||||
}
|
||||
write!(f, "]")
|
||||
}
|
||||
Value::Record(r) => {
|
||||
Value::Record(layout, values) => {
|
||||
write!(f, "{{")?;
|
||||
for i in 0..r.values.len() {
|
||||
for i in 0..values.len() {
|
||||
if i > 0 {
|
||||
write!(f, ", ")?;
|
||||
}
|
||||
write!(f, ":{} {}", r.keys[i].name(), r.values[i])?;
|
||||
write!(f, ":{} {}", layout.fields[i].0.name(), values[i])?;
|
||||
}
|
||||
write!(f, "}}")
|
||||
}
|
||||
Value::FieldAccessor(k) => write!(f, ".{}", k.name()),
|
||||
Value::Function(f_meta) => write!(f, "<native fn, {:?}>", f_meta.purity),
|
||||
Value::Object(o) => write!(f, "<{}>", o.type_name()),
|
||||
Value::Cell(c) => write!(f, "{}", c.borrow()),
|
||||
|
||||
@@ -294,6 +294,22 @@ impl VM {
|
||||
Ok(val)
|
||||
}
|
||||
BoundKind::Get { addr, .. } => self.get_value(*addr),
|
||||
|
||||
BoundKind::FieldAccessor(k) => Ok(Value::FieldAccessor(*k)),
|
||||
|
||||
BoundKind::GetField { rec, field } => {
|
||||
let rec_val = self.eval_internal(obs, rec)?;
|
||||
if let Value::Record(layout, values) = rec_val {
|
||||
if let Some(idx) = layout.index_of(*field) {
|
||||
Ok(values[idx].clone())
|
||||
} else {
|
||||
Err(format!("Record does not have field :{}", field.name()))
|
||||
}
|
||||
} else {
|
||||
Err(format!("Attempt to access field on non-record: {}", rec_val))
|
||||
}
|
||||
}
|
||||
|
||||
BoundKind::Set { addr, value } => {
|
||||
let val = self.eval_internal(obs, value)?;
|
||||
self.set_value(*addr, val.clone())?;
|
||||
@@ -368,6 +384,29 @@ impl VM {
|
||||
return Ok(Value::TailCallRequest(Box::new((obj, arg_vals))));
|
||||
}
|
||||
Value::Function(f) => return Ok((f.func)(arg_vals)),
|
||||
Value::FieldAccessor(k) => {
|
||||
if arg_vals.len() != 1 {
|
||||
return Err(format!(
|
||||
"Field accessor .{} expects exactly 1 argument, got {}",
|
||||
k.name(),
|
||||
arg_vals.len()
|
||||
));
|
||||
}
|
||||
let rec = &arg_vals[0];
|
||||
if let Value::Record(layout, values) = rec {
|
||||
if let Some(idx) = layout.index_of(k) {
|
||||
return Ok(values[idx].clone());
|
||||
} else {
|
||||
return Err(format!("Record does not have field :{}", k.name()));
|
||||
}
|
||||
} else {
|
||||
return Err(format!(
|
||||
"Field accessor .{} expects a record, got {}",
|
||||
k.name(),
|
||||
rec
|
||||
));
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
return Err(format!(
|
||||
"Tail call target is not a function: {}",
|
||||
@@ -380,6 +419,29 @@ impl VM {
|
||||
loop {
|
||||
match func_val {
|
||||
Value::Function(f) => break Ok((f.func)(arg_vals)),
|
||||
Value::FieldAccessor(k) => {
|
||||
if arg_vals.len() != 1 {
|
||||
break Err(format!(
|
||||
"Field accessor .{} expects exactly 1 argument, got {}",
|
||||
k.name(),
|
||||
arg_vals.len()
|
||||
));
|
||||
}
|
||||
let rec = &arg_vals[0];
|
||||
if let Value::Record(layout, values) = rec {
|
||||
if let Some(idx) = layout.index_of(k) {
|
||||
break Ok(values[idx].clone());
|
||||
} else {
|
||||
break Err(format!("Record does not have field :{}", k.name()));
|
||||
}
|
||||
} else {
|
||||
break Err(format!(
|
||||
"Field accessor .{} expects a record, got {}",
|
||||
k.name(),
|
||||
rec
|
||||
));
|
||||
}
|
||||
}
|
||||
Value::Object(obj) => {
|
||||
if let Some(closure) = obj.as_any().downcast_ref::<Closure>() {
|
||||
let old_stack_top = self.stack.len();
|
||||
|
||||
@@ -426,4 +426,85 @@ mod tests {
|
||||
"Definitions should be removed by DCE"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_record_basics() {
|
||||
let env = Environment::new();
|
||||
let source = r#"
|
||||
((fn [user] [(.name user) (.age user)])
|
||||
{:name "Alice" :age 30})
|
||||
"#;
|
||||
let res = env.run_script(source).unwrap();
|
||||
assert_eq!(format!("{}", res), "[\"Alice\" 30]");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_record_optimized_access() {
|
||||
let env = Environment::new();
|
||||
let source = "(.price {:id 1 :price 99.5})";
|
||||
|
||||
// 1. Check result
|
||||
let res = env.run_script(source).unwrap();
|
||||
assert_eq!(format!("{}", res), "99.5");
|
||||
|
||||
// 2. Verify optimization to GET_FIELD
|
||||
let dump = env.dump_ast(source).unwrap();
|
||||
assert!(dump.contains("GetField: .price"), "Should be optimized to GetField. Dump:\n{}", dump);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_first_class_field_accessor() {
|
||||
let env = Environment::new();
|
||||
let source = r#"
|
||||
(do
|
||||
(def get-name .name)
|
||||
; Dynamic call to field accessor
|
||||
(get-name {:name "Alice"}))
|
||||
"#;
|
||||
let res = env.run_script(source).unwrap();
|
||||
assert_eq!(format!("{}", res), "\"Alice\"");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_record_constant_folding() {
|
||||
let env = Environment::new();
|
||||
// Optimizer should fold (.x {:x 10}) into 10
|
||||
let source = "(.x {:x 10 :y 20})";
|
||||
let dump = env.dump_ast(source).unwrap();
|
||||
assert!(dump.contains("Constant: 10"), "Should be constant folded. Dump:\n{}", dump);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_record_errors() {
|
||||
let env = Environment::new();
|
||||
|
||||
// Both cases are reported by the TypeChecker since it can't resolve the call
|
||||
// for a missing field or a non-record argument.
|
||||
|
||||
// 1. Missing field
|
||||
let res_missing = env.run_script("(.missing {:a 1})");
|
||||
assert!(res_missing.is_err());
|
||||
assert!(res_missing.unwrap_err().contains("Invalid arguments"));
|
||||
|
||||
// 2. Not a record
|
||||
let res_not_rec = env.run_script("(.name 123)");
|
||||
assert!(res_not_rec.is_err());
|
||||
assert!(res_not_rec.unwrap_err().contains("Invalid arguments"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_record_layout_interning() {
|
||||
let env = Environment::new();
|
||||
let source = r#"
|
||||
(do
|
||||
(def r1 {:a 1 :b 2})
|
||||
(def r2 {:a 10 :b 20})
|
||||
; Identical layouts result in identical types
|
||||
(= r1 r2))
|
||||
"#;
|
||||
// This will be false because values differ, but let's just check if it compiles and runs.
|
||||
// To really test interning, we'd need a way to check if layouts are the same Arc.
|
||||
let res = env.run_script(source).unwrap();
|
||||
assert_eq!(format!("{}", res), "false");
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user