feat: Add AST analysis pass for purity and recursion

This commit introduces a new AST analysis pass that identifies function
purity and recursion. This information is then used by the optimizer and
specializer to make more informed decisions, particularly regarding
inlining.

The `Analyzer` struct and its associated `Analysis` struct are
responsible for traversing the AST and collecting this data.

Key changes include:
- A new `analyzer` module is added to `ast::compiler`.
- `Analyzer::analyze` performs a two-pass traversal to collect
  global-to-lambda mappings and then analyze purity and recursion.
- The `Optimizer` and `Specializer` are updated to accept and utilize
  the `Analysis` data.
- Recursion checks in `Optimizer` and `Specializer` are replaced with
  checks against the pre-computed `Analysis.is_recursive` set.
- The `Environment` now stores and passes the `Analysis` results to the
  compiler stages.

docs/Optimization_Analysis.md

@@ -0,0 +1,48 @@
+# Analyse der AST-Optimierungen und Spezialisierung
+
+*Datum: 23.05.2024*
+*Tool: `src/bin/ast.rs` mit `--dump`*
+
+## 1. Motivation
+Diese Analyse untersucht die Effektivität der Rust-Portierung des Optimizers. Ziel war es, die theoretischen Konzepte aus den Design-Dokumenten (wie "Closure Cracking" und "Beta-Reduction") empirisch am lebenden Objekt zu verifizieren.
+
+## 2. Kern-Erkenntnisse
+
+### A. Aggressives Constant Folding (The "Zero-Cost" Promise)
+Der Optimizer ist extrem aggressiv bei der Vorkalkulation von Werten. Ein komplexer Ausdruck wie:
+```lisp
+(do
+  (def MYPI 3.14)
+  (def radius 10)
+  (* MYPI (* radius radius)))
+```
+wird im optimierten AST zu einer einzigen Konstante reduziert: `Constant: 314`. 
+**Erstaunlich:** Nicht nur die Arithmetik wird gefaltet, sondern auch die `def`-Bindungen werden komplett eliminiert (Dead Code Elimination), da sie nach dem Inlining nicht mehr benötigt werden.
+
+### B. Beta-Reduction & Lambda-Inlining
+Aufrufe von Lambda-Literalen werden bereits zur Compile-Zeit "geknackt" (Cracking). 
+*   **Beispiel:** `((fn [x] (+ x 1)) 10)` -> `Constant: 11`.
+*   **Verschachtelung:** Selbst bei verschachtelten Lambdas wie `((fn [x] (fn [] x)) 10)` erkennt der Optimizer die Konstante und reduziert den äußeren Aufruf auf ein Lambda, das lediglich `Constant: 10` zurückgibt.
+
+### C. Die Rolle der Spezialisierung (Monomorphisierung)
+Eine der wichtigsten architektonischen Erkenntnisse war das Verhalten des `Specializer`. Selbst wenn das Tool mit `--no-opt` aufgerufen wird, tauchen im Dump oft spezialisierte Closures auf.
+*   **Grund:** Die Spezialisierung ist in `Environment::link` fest verdrahtet, da sie für die Korrektheit (z. B. Overload-Resolution von Operatoren wie `+`) notwendig ist.
+*   **Effekt:** Ein globaler Aufruf von `f` mit einem `Int` wird zu einem Aufruf einer spezialisierten Version von `f`, die bereits auf `Int` optimiert wurde. Der "echte" Optimizer baut darauf auf und führt dann das Inlining durch.
+
+## 3. Überraschungen & Kuriositäten
+
+### Der Rekursions-Schutz
+Beim Testen einer rekursiven Funktion `(def f (fn [x] (f x)))` zeigte der Dump ein interessantes Muster:
+*   Der Optimizer führt genau eine Ebene des Inlinings durch, bevor der `inlining_stack` greift. 
+*   Im Dump resultiert dies in einem "Callee", der selbst ein Lambda ist, welches wiederum die globale Funktion aufruft. 
+*   Dabei tauchten spezialisierte Typen wie `Vector(Any, 1)` oder `Matrix(Any, [1, 1])` auf. Dies sind interne Artefakte der Monomorphisierung, die zeigen, wie das System versucht, Argumentlisten in einheitliche Tupel-Strukturen zu zwingen (Unification).
+
+### String-Folding
+Der Optimizer beherrscht das Zusammenfügen von Strings via `+` bereits zur Compile-Zeit. 
+*   `(+ "Hello, " "Rust!")` wird im AST direkt zu `"Hello, Rust!"`. 
+*   Dies ist besonders wertvoll für Makros, die Code-Teile oder Bezeichner generieren.
+
+## 4. Fazit für die Entwicklung
+Die Pipeline (Expand -> Bind -> Specialize -> Optimize -> TCO) ist robust. Die Entscheidung, die Spezialisierung vor den Optimizer zu schalten, erweist sich als goldrichtig, da der Optimizer so auf einem bereits typsicheren und "aufgecrackten" Baum operieren kann.
+
+Das Tool `ast.exe --dump` bleibt das wichtigste Instrument, um sicherzustellen, dass neue Sprachfeatures nicht den "Fast Path" des Compilers verlassen.

src/ast/compiler/analyzer.rs

@@ -0,0 +1,173 @@
+use crate::ast::compiler::bound_nodes::{Address, BoundKind, TypedNode};
+use crate::ast::types::{Identity, Purity, StaticType};
+use std::collections::{HashMap, HashSet};
+
+#[derive(Debug, Clone, Default)]
+pub struct Analysis {
+    pub purity: HashMap<Identity, Purity>,
+    pub is_recursive: HashSet<Identity>,
+}
+
+pub struct Analyzer<'a> {
+    global_purity: &'a HashMap<u32, Purity>,
+    results: Analysis,
+    /// Stack of currently visiting lambdas to detect direct recursion.
+    lambda_stack: Vec<Identity>,
+    /// Map of global index to its Lambda identity if known.
+    globals_to_lambdas: HashMap<u32, Identity>,
+}
+
+impl<'a> Analyzer<'a> {
+    pub fn analyze(node: &TypedNode, global_purity: &'a HashMap<u32, Purity>) -> Analysis {
+        let mut analyzer = Self {
+            global_purity,
+            results: Analysis::default(),
+            lambda_stack: Vec::new(),
+            globals_to_lambdas: HashMap::new(),
+        };
+        
+        // First pass: map globals to their lambda identities
+        analyzer.collect_globals(node);
+        
+        // Second pass: full analysis
+        analyzer.visit(node);
+        analyzer.results
+    }
+
+    fn collect_globals(&mut self, node: &TypedNode) {
+        match &node.kind {
+            BoundKind::DefGlobal { global_index, value, .. } => {
+                if let BoundKind::Lambda { .. } = &value.kind {
+                    self.globals_to_lambdas.insert(*global_index, value.identity.clone());
+                }
+                self.collect_globals(value);
+            }
+            BoundKind::Block { exprs } => {
+                for e in exprs { self.collect_globals(e); }
+            }
+            _ => {
+                // Simplified traversal for global collection
+                node.kind.for_each_child(|child| self.collect_globals(child));
+            }
+        }
+    }
+
+    fn visit(&mut self, node: &TypedNode) -> Purity {
+        let purity = match &node.kind {
+            BoundKind::Constant(_) | BoundKind::Nop | BoundKind::Parameter { .. } => Purity::Pure,
+            
+            BoundKind::Get { addr, .. } => match addr {
+                Address::Global(idx) => self.global_purity.get(idx).cloned().unwrap_or(Purity::Pure),
+                _ => Purity::Pure, // Locals are considered pure access in this model
+            },
+
+            BoundKind::Set { .. } => Purity::Impure,
+
+            BoundKind::DefLocal { value, .. } | BoundKind::DefGlobal { value, .. } => {
+                self.visit(value)
+            }
+
+            BoundKind::If { cond, then_br, else_br } => {
+                let p_cond = self.visit(cond);
+                let p_then = self.visit(then_br);
+                let p_else = else_br.as_ref().map(|e| self.visit(e)).unwrap_or(Purity::Pure);
+                p_cond.min(p_then).min(p_else)
+            }
+
+            BoundKind::Lambda { body, .. } => {
+                self.lambda_stack.push(node.identity.clone());
+                self.visit(body);
+                self.lambda_stack.pop();
+                Purity::Pure // Creating a lambda is pure
+            }
+
+            BoundKind::Call { callee, args } => {
+                let p_callee = self.visit(callee);
+                let p_args = self.visit(args);
+                
+                // Detect recursion
+                if let BoundKind::Get { addr: Address::Global(idx), .. } = &callee.kind
+                    && let Some(lambda_id) = self.globals_to_lambdas.get(idx)
+                    && self.lambda_stack.contains(lambda_id)
+                {
+                    self.results.is_recursive.insert(lambda_id.clone());
+                    // Also mark the call itself if needed
+                    self.results.is_recursive.insert(node.identity.clone());
+                }
+
+                // For purity, we'd need to know the function's purity.
+                // For now, if it's a call, we conservatively check if it's a known pure global.
+                let p_func = if let BoundKind::Get { addr: Address::Global(idx), .. } = &callee.kind {
+                    self.global_purity.get(idx).cloned().unwrap_or(Purity::Impure)
+                } else {
+                    Purity::Impure 
+                };
+
+                p_callee.min(p_args).min(p_func)
+            }
+
+            BoundKind::Block { exprs } => {
+                let mut p = Purity::Pure;
+                for e in exprs {
+                    p = p.min(self.visit(e));
+                }
+                p
+            }
+
+            BoundKind::Tuple { elements } => {
+                let mut p = Purity::Pure;
+                for e in elements {
+                    p = p.min(self.visit(e));
+                }
+                p
+            }
+
+            BoundKind::Record { fields } => {
+                let mut p = Purity::Pure;
+                for (k, v) in fields {
+                    p = p.min(self.visit(k)).min(self.visit(v));
+                }
+                p
+            }
+            
+            _ => Purity::Impure,
+        };
+
+        self.results.purity.insert(node.identity.clone(), purity);
+        purity
+    }
+}
+
+/// Extension trait to make traversal easier
+trait NodeExt {
+    fn for_each_child<F: FnMut(&TypedNode)>(&self, f: F);
+}
+
+impl NodeExt for BoundKind<StaticType> {
+    fn for_each_child<F: FnMut(&TypedNode)>(&self, mut f: F) {
+        match self {
+            BoundKind::If { cond, then_br, else_br } => {
+                f(cond); f(then_br); if let Some(e) = else_br { f(e); }
+            }
+            BoundKind::DefLocal { value, .. } | BoundKind::DefGlobal { value, .. } | BoundKind::Set { value, .. } => {
+                f(value);
+            }
+            BoundKind::Lambda { params, body, .. } => {
+                f(params); f(body);
+            }
+            BoundKind::Call { callee, args } => {
+                f(callee); f(args);
+            }
+            BoundKind::Block { exprs } => {
+                for e in exprs { f(e); } // Block
+            }
+            BoundKind::Tuple { elements } => {
+                for e in elements { f(e); } // Tuple
+            }
+            BoundKind::Record { fields } => {
+                for (k, v) in fields { f(k); f(v); }
+            }
+            _ => {}
+        }
+    }
+}

src/ast/compiler/mod.rs

@@ -1,3 +1,4 @@
+pub mod analyzer;
 pub mod binder;
 pub mod bound_nodes;
 pub mod dumper;

src/ast/compiler/optimizer.rs

@@ -1,3 +1,4 @@
+use crate::ast::compiler::analyzer::Analysis;
 use crate::ast::compiler::bound_nodes::{Address, BoundKind, TypedNode};
 use crate::ast::nodes::Node;
 use crate::ast::types::{Purity, StaticType, Value};
@@ -14,6 +15,7 @@ pub struct Optimizer {
     pub globals: Option<Rc<RefCell<Vec<Value>>>>,
     pub global_purity: Option<Rc<RefCell<HashMap<u32, Purity>>>>,
     pub lambda_registry: Option<Rc<RefCell<HashMap<u32, TypedNode>>>>,
+    pub analysis: Analysis,
 }
 
 /// Global state for tracking the optimization path (recursion, depth).
@@ -64,9 +66,15 @@ impl Optimizer {
             globals: None,
             global_purity: None,
             lambda_registry: None,
+            analysis: Analysis::default(),
         }
     }
 
+    pub fn with_analysis(mut self, analysis: Analysis) -> Self {
+        self.analysis = analysis;
+        self
+    }
+
     pub fn with_globals(mut self, globals: Rc<RefCell<Vec<Value>>>) -> Self {
         self.globals = Some(globals);
         self
@@ -100,34 +108,6 @@ impl Optimizer {
         current
     }
 
-    fn is_recursive(
-        &self,
-        node: &TypedNode,
-        global_idx: Option<u32>,
-        local_slot: Option<u32>,
-        identity: Option<&crate::ast::types::Identity>,
-    ) -> bool {
-        let mut info = UsageInfo::default();
-        self.collect_usage(node, &mut info);
-
-        if let Some(idx) = global_idx
-            && info.used_globals.contains(&idx)
-        {
-            return true;
-        }
-        if let Some(slot) = local_slot
-            && info.used_locals.contains(&slot)
-        {
-            return true;
-        }
-        if let Some(id) = identity
-            && info.used_identities.contains(id)
-        {
-            return true;
-        }
-        false
-    }
-
     fn visit_node(
         &self,
         node: TypedNode,
@@ -257,8 +237,8 @@ impl Optimizer {
                         && positional_count.is_some()
                         && path.inlining_depth < 5
                     {
-                        // STRICT RECURSION CHECK: Don't inline if recursive
+                        // USE STATIC ANALYSIS: Don't inline if the function is known to be recursive
-                        if !self.is_recursive(body, None, None, Some(&callee.identity))
+                        if !self.analysis.is_recursive.contains(&callee.identity)
                             && path.enter_lambda(&callee.identity)
                         {
                             path.inlining_depth += 1;
@@ -298,13 +278,8 @@ impl Optimizer {
                             && upvalues.is_empty()
                             && positional_count.is_some()
                         {
-                            // STRICT RECURSION CHECK: Don't inline if recursive (idx or identity)
+                            // USE STATIC ANALYSIS: Don't inline if the function is known to be recursive
-                            if !self.is_recursive(
+                            if !self.analysis.is_recursive.contains(&lambda_node.identity) {
-                                body,
-                                Some(*idx),
-                                None,
-                                Some(&lambda_node.identity),
-                            ) {
                                 let mut inner_sub = SubstitutionMap::new();
                                 path.inlining_stack.insert(*idx);
                                 path.inlining_depth += 1;
@@ -332,13 +307,9 @@ impl Optimizer {
                         && (closure.upvalues.is_empty()
                             || self.purity_of(&closure.function_node) >= Purity::SideEffectFree)
                     {
-                        // STRICT RECURSION CHECK: Don't inline if closure body contains its own identity
+                        // USE STATIC ANALYSIS: Don't inline if the function is known to be recursive
-                        if !self.is_recursive(
+                        if !self.analysis.is_recursive.contains(&closure.function_node.identity)
-                            &closure.function_node,
+                            && path.enter_lambda(&closure.function_node.identity)
-                            None,
-                            None,
-                            Some(&closure.function_node.identity),
-                        ) && path.enter_lambda(&closure.function_node.identity)
                         {
                             let mut closure_sub = SubstitutionMap::new();
                             for (i, cell) in closure.upvalues.iter().enumerate() {
@@ -755,7 +726,7 @@ impl Optimizer {
         }
     }
 
-    fn is_inlinable_value(&self, val: &Value, global_idx: Option<u32>) -> bool {
+    fn is_inlinable_value(&self, val: &Value, _global_idx: Option<u32>) -> bool {
         match val {
             Value::Int(_)
             | Value::Float(_)
@@ -767,12 +738,7 @@ impl Optimizer {
                 if let Some(closure) = obj.as_any().downcast_ref::<Closure>() {
                     // A closure is only inlinable if it's not recursive
                     closure.upvalues.is_empty()
-                        && !self.is_recursive(
+                        && !self.analysis.is_recursive.contains(&closure.function_node.identity)
-                            &closure.function_node,
-                            global_idx,
-                            None,
-                            Some(&closure.function_node.identity),
-                        )
                 } else {
                     false
                 }
@@ -782,6 +748,10 @@ impl Optimizer {
     }
 
     fn purity_of(&self, node: &TypedNode) -> Purity {
+        if let Some(p) = self.analysis.purity.get(&node.identity) {
+            return *p;
+        }
+
         match &node.kind {
             BoundKind::Constant(_) | BoundKind::Parameter { .. } | BoundKind::Nop => Purity::Pure,
             // Defining a lambda is pure; only calling it may have side effects.

src/ast/compiler/specializer.rs

@@ -1,3 +1,4 @@
+use crate::ast::compiler::analyzer::Analysis;
 use crate::ast::compiler::bound_nodes::{Address, BoundKind, BoundNode, TypedNode};
 use crate::ast::nodes::Node;
 use crate::ast::types::{Signature, StaticType, Value};
@@ -22,6 +23,7 @@ pub type MonoCache = HashMap<MonoCacheKey, (Value, StaticType)>;
 
 pub struct Specializer {
     pub cache: Rc<RefCell<MonoCache>>,
+    pub analysis: Analysis,
     registry: Option<Rc<dyn FunctionRegistry>>,
     compiler: Option<CompileFunc>,
     rtl_lookup: Option<RtlLookupFunc>,
@@ -33,9 +35,11 @@ impl Specializer {
         compiler: Option<CompileFunc>,
         rtl_lookup: Option<RtlLookupFunc>,
         cache: Option<Rc<RefCell<MonoCache>>>,
+        analysis: Analysis,
     ) -> Self {
         Self {
             cache: cache.unwrap_or_else(|| Rc::new(RefCell::new(HashMap::new()))),
+            analysis,
             registry,
             compiler,
             rtl_lookup,
@@ -245,6 +249,11 @@ impl Specializer {
 
         // 6. Resolve Function Definition
         if let Some(func_node) = self.registry.as_ref().and_then(|r| r.resolve(address)) {
+            // Check for recursion from pre-pass.
+            if self.analysis.is_recursive.contains(&func_node.identity) {
+                return (new_callee, new_args, original_ty);
+            }
+
             // Check constraints (no closures with state)
             if let BoundKind::Lambda { upvalues, .. } = &func_node.kind {
                 if !upvalues.is_empty() {

src/ast/environment.rs

@@ -1,3 +1,4 @@
+use crate::ast::compiler::analyzer::{Analysis, Analyzer};
 use crate::ast::compiler::binder::Binder;
 use crate::ast::compiler::{TypeChecker, TypedNode};
 use crate::ast::nodes::{Node, Symbol, UntypedKind};
@@ -29,6 +30,7 @@ pub struct Environment {
     pub monomorph_cache: Rc<RefCell<MonoCache>>,
     pub debug_mode: bool,
     pub optimization: bool,
+    pub last_analysis: RefCell<Analysis>,
 }
 
 struct EnvFunctionRegistry {
@@ -96,6 +98,7 @@ impl Environment {
             monomorph_cache: Rc::new(RefCell::new(HashMap::new())),
             debug_mode: false,
             optimization: true,
+            last_analysis: RefCell::new(Analysis::default()),
         };
         env.register_stdlib();
         env
@@ -204,6 +207,10 @@ impl Environment {
         // 7. Collect Typed Lambdas
         LambdaCollector::collect(&typed_ast, &mut self.typed_function_registry.borrow_mut());
 
+        // 8. Analyze (Purity, Recursion)
+        let analysis = Analyzer::analyze(&typed_ast, &self.global_purity.borrow());
+        *self.last_analysis.borrow_mut() = analysis;
+
         Ok(typed_ast)
     }
 
@@ -216,7 +223,8 @@ impl Environment {
         let optimizer = Optimizer::new(self.optimization)
             .with_globals(self.global_values.clone())
             .with_purity(self.global_purity.clone())
-            .with_registry(self.typed_function_registry.clone());
+            .with_registry(self.typed_function_registry.clone())
+            .with_analysis(self.last_analysis.borrow().clone());
         let optimized = optimizer.optimize(specialized);
 
         // 3. TCO (Always performed, converts to ExecNode)
@@ -302,7 +310,9 @@ impl Environment {
         let global_types = self.global_types.clone();
         let global_purity = self.global_purity.clone();
         let optimization = self.optimization;
+        let analysis = self.last_analysis.borrow().clone();
 
+        let compiler_analysis = analysis.clone();
         let compiler = Rc::new(
             move |func_template: BoundNode,
                   arg_types: &[StaticType]|
@@ -323,6 +333,7 @@ impl Environment {
                     None,
                     Some(sub_rtl_lookup),
                     Some(mono_cache.clone()),
+                    compiler_analysis.clone(),
                 );
 
                 let specialized_ast = sub_specializer.specialize(retyped_ast);
@@ -330,7 +341,8 @@ impl Environment {
                 // 3. Optimize (Phase 2: Cracking & Folding)
                 let optimizer = Optimizer::new(optimization)
                     .with_globals(global_values.clone())
-                    .with_purity(global_purity.clone());
+                    .with_purity(global_purity.clone())
+                    .with_analysis(compiler_analysis.clone());
                 let optimized_ast = optimizer.optimize(specialized_ast);
 
                 // 4. TCO (converts to ExecNode)
@@ -359,6 +371,7 @@ impl Environment {
             Some(compiler),
             Some(rtl_lookup),
             Some(self.monomorph_cache.clone()),
+            analysis,
         );
 
         specializer.specialize(node)