Brummel
63a474030d
This commit introduces two new files: - `.claudeignore`: Specifies files and directories that should be ignored by Claude, such as snapshot files and the `target` directory. - `docs/bytecode-architecture.md`: Documents the architecture for bytecode generation, outlining the hybrid execution model, the transformation pipeline (Specializer and Lowering), and the role of Tail Call Optimization (TCO).
82 lines
3.8 KiB
Markdown
82 lines
3.8 KiB
Markdown
# Bytecode Generation Architecture
|
|
|
|
This document describes the transition from a pure tree-walking AST interpreter to a hybrid execution model using specialized bytecode.
|
|
|
|
## 1. Overview: The Hybrid Model
|
|
|
|
The MYC VM operates in a hybrid mode. While the Abstract Syntax Tree (AST) is the first-class citizen and primary representation, "hot" paths—specifically monomorphized functions and financial pipeline lambdas—are lowered into a linear bytecode representation.
|
|
|
|
---
|
|
|
|
## 2. The Integrated Transformation Pipeline
|
|
|
|
To ensure maximum performance and correct Tail Call Optimization (TCO), bytecode generation is integrated into the structural optimization phase.
|
|
|
|
### Step 1: Specializer (Structural Optimization & Emission)
|
|
The **Specializer** is the core engine for code improvement.
|
|
- **Inlining & Monomorphization:** It first performs all structural changes (e.g., inlining function bodies). This is critical because inlining changes what constitutes a "tail position."
|
|
- **Symbolic Emission:** Once the structure of a function/block is final, the Specializer converts it into **Symbolic Bytecode**.
|
|
- **Integrated TCO Analysis:** During emission, the Specializer tracks the tail-call state locally. It emits `OP_RECUR` or `OP_TAIL_CALL` based on the final, inlined structure.
|
|
- **Variable Mapping:** Instructions still use `VirtualId` (e.g., `OP_GET_VIRTUAL(V102)`).
|
|
|
|
### Step 2: Lowering (Address Resolution & Linking)
|
|
The **Lowering** pass acts as the "Linker". It does not change the structure anymore.
|
|
- **Address Resolution:** It walks the symbolic bytecode and replaces `VirtualId` with concrete `StackOffset` values.
|
|
- **Stack Allocation:** It calculates the final `stack_size` for the bytecode block.
|
|
- **Finalization:** It "freezes" the instruction vector for the `ExecNode`.
|
|
|
|
---
|
|
|
|
## 3. Why This Order? (Inlining vs. TCO)
|
|
|
|
TCO analysis must occur **after** structural optimizations like inlining.
|
|
If TCO were analyzed before inlining, a tail-call in a small function might lose its tail position once embedded into a larger caller. By performing TCO analysis **during** bytecode emission (after inlining), we guarantee that the `is_tail` flags are always semantically correct.
|
|
|
|
---
|
|
|
|
## 4. Instruction Set Architecture (ISA) & TCO
|
|
|
|
The ISA uses specialized instructions to bypass the heavy tree-walking logic:
|
|
|
|
1. **OP_RECUR:**
|
|
- Used for the `again` keyword.
|
|
- Triggers an immediate jump to PC 0 within the same bytecode executor.
|
|
- Repacks the current stack frame with new arguments.
|
|
|
|
2. **OP_TAIL_CALL(Callee):**
|
|
- Used for function calls in tail positions.
|
|
- If the callee is bytecode, it performs a fast-swap of the instruction vector.
|
|
- If the callee is native/tree, it returns a `TailCallRequest` to the VM dispatcher.
|
|
|
|
---
|
|
|
|
## 5. The Hybrid VM Dispatcher
|
|
|
|
The VM's `eval_internal` remains the master controller. It dispatches to the Bytecode Executor when encountering a `BoundKind::Bytecode` node. The executor can "escape" back to the tree-walker for any operation it cannot handle natively (e.g., calling an opaque extension).
|
|
|
|
```rust
|
|
fn run_bytecode(instructions: &[Instruction], pc: &mut usize) -> Value {
|
|
loop {
|
|
match instructions[*pc] {
|
|
Instruction::PushConst(v) => stack.push(v),
|
|
Instruction::AddInt => {
|
|
let b = stack.pop();
|
|
let a = stack.pop();
|
|
stack.push(a + b);
|
|
}
|
|
Instruction::TailCall(callee) => {
|
|
// Return to dispatcher if it's a cross-world call
|
|
return self.handle_tail_call(callee);
|
|
}
|
|
Instruction::CallTree(node) => {
|
|
// Re-enter tree-walking for specific node
|
|
let res = self.eval_internal(node)?;
|
|
stack.push(res);
|
|
}
|
|
// ...
|
|
}
|
|
pc += 1;
|
|
}
|
|
}
|
|
```
|