Old Docs added

docs/White Paper Macro Hygiene.md

@@ -0,0 +1,119 @@
+# White Paper: A Hybrid Approach to Macro Hygiene
+
+**Date:** 09.11.2025
+**Status:** Final
+
+## 1\. Executive Summary
+
+The macro system is a cornerstone of the compiler, enabling powerful syntactic abstraction. However, designing a macro system requires solving the fundamental conflict between **safety** (preventing accidental variable conflicts) and **power** (allowing macros to interact with their calling context).
+
+This document outlines the rationale for the implemented hybrid macro system. This system is designed to provide the best of both worlds without burdening the macro author with manual hygiene management (such as `gensym` or special syntax).
+
+Our system operates on two simple, deterministic rules:
+
+1.  **Automatic Hygiene for Definitions:** All symbols *defined* within a macro template are automatically renamed to be unique, preventing conflicts with user code or nested macro calls.
+2.  **Unhygienic Fallback for Free Symbols:** All *free symbols* (those used but not defined within the template) are left untouched. They are resolved by the binder in the **call-site scope** (the scope where the macro was invoked).
+
+This hybrid model ensures that internal macro variables are always safe, while simultaneously permitting powerful, context-aware macros.
+
+-----
+
+## 2\. The Core Challenge: Safety vs. Context
+
+A macro, by definition, injects code into a foreign scope. This creates two distinct, opposing requirements.
+
+### Scenario A: The Need for Safety (Hygienic Definitions)
+
+This is the classic hygiene problem. A macro must manage its own internal state without interfering with the user's code.
+
+Consider a simple `stopwatch` macro:
+
+```lisp
+(* Macro Definition *)
+(defmacro stopwatch [body]
+  `(do
+     (def start-time (timestamp))
+     (def result ~body)
+     (print "Time: " (- (timestamp) start-time))
+     result))
+
+(* User Code *)
+(def start-time "Important User Data")
+(stopwatch (expensive-call))
+(print start-time)
+```
+
+A naive (fully unhygienic) expansion would redefine the user's `start-time` variable, corrupting their program. This is unacceptable. The macro's internal variables (`start-time`, `result`) must be **hygienic**—that is, isolated from the call-site scope.
+
+### Scenario B: The Need for Context (Unhygienic Free Symbols)
+
+Macros derive their power from interacting with the context in which they are called. A macro may need to read variables from the user's scope.
+
+Consider a `debug-print` macro:
+
+```lisp
+(* Macro Definition *)
+(defmacro debug-print [msg]
+  `(if *debug-mode*
+       (print msg)))
+
+(* User Code *)
+(do
+  (def *debug-mode* true)  ; User-defined context variable
+  (debug-print "Test message")
+)
+```
+
+In this case, the macro *must* access the user's `*debug-mode*` variable from the call-site scope. A "fully hygienic" system (which binds *all* symbols to the macro's *definition-site scope*) would fail, as it would be unable to see the user's local `*debug-mode*` variable.
+
+-----
+
+## 3\. Rejected Alternatives
+
+To solve this conflict, several common designs were considered and rejected for failing one of the two core scenarios.
+
+  * **Fully Unhygienic (The `$` Suffix):** This approach makes all symbols unhygienic by default and requires the author to manually mark internal variables (e.g., `start-time$`) for special handling. This inverts the desired default (safety) and fails to solve nested macro conflicts.
+  * **Fully Hygienic (Strict Academic):** This system binds *all* symbols (defined or free) to the macro's definition-site scope. This perfectly solves Scenario A but makes Scenario B impossible.
+  * **Explicit Gensym (The Lisp Way):** This requires the macro author to manually generate unique symbols (`(let [start-sym (gensym)] ...)`). This is syntactically complex, error-prone, and places an unnecessary burden on the author.
+
+-----
+
+## 4\. The Implemented Solution: Hybrid Hygiene
+
+Our system resolves the conflict by treating definitions and free variables differently. The logic is handled entirely by the macro expander, requiring no special syntax from the macro author.
+
+### Rule 1: Automatic Renaming of Definitions
+
+When the macro expander processes a template, it performs a pre-pass to identify all symbols being *defined*. This includes `def` forms, `fn` parameters, and `let` bindings.
+
+  * For each defined symbol (e.g., `start-time`), the expander generates a unique, internal-only name (e.g., `start-time_G123`).
+  * It stores this in an expansion-local rename map.
+  * It then replaces all occurrences of that symbol *within the template* with the new name.
+
+This automatically and transparently solves **Scenario A**. The `stopwatch` macro's `start-time` becomes `start-time_G123` and cannot possibly conflict with the user's `start-time`. This also solves the nested macro problem, as each expansion generates new unique names.
+
+### Rule 2: Unhygienic Fallback for Free Symbols
+
+Any symbol in the macro template that is *not* part of a definition (a "free symbol") is left untouched by the expander.
+
+  * In `stopwatch`, the symbols `do`, `timestamp`, `print`, and `-` are free symbols.
+  * In `debug-print`, the symbols `if`, `*debug-mode*`, and `print` are free symbols.
+
+The expander passes these symbols directly to the next compiler stage (the binder). The binder then resolves them, as it would any normal code, within the **call-site scope**.
+
+This solves **Scenario B**. The binder finds `*debug-mode*` in the user's `do` block, exactly as intended.
+
+-----
+
+## 5\. Conclusion and Trade-Offs
+
+This hybrid design provides "implicit hygiene" for the common case (internal variables) while defaulting to "unhygienic" behavior for external symbols, which provides maximum power and flexibility.
+
+The primary trade-off of this design is that **typographical errors in free symbols are caught late**.
+
+For example, if the `stopwatch` macro misspelled `print` as `prnit`:
+
+1.  The expander would see `prnit` as a free symbol (Rule 2) and leave it untouched.
+2.  The binder would then fail to find `prnit` in the *user's* call-site scope.
+
+The resulting error (`Undefined symbol: prnit`) will point to the user's code where `stopwatch` was *called*, not the macro *definition*. This is a minor, acceptable trade-off for a system that achieves both safety and power without syntactic overhead.