MycLib/Doc/White Paper Macro Hygiene.md

White Paper: A Hybrid Approach to Macro Hygiene

Date: 09.11.2025 Status: Final

1. Executive Summary

The macro system is a cornerstone of the compiler, enabling powerful syntactic abstraction. However, designing a macro system requires solving the fundamental conflict between safety (preventing accidental variable conflicts) and power (allowing macros to interact with their calling context).

This document outlines the rationale for the implemented hybrid macro system. This system is designed to provide the best of both worlds without burdening the macro author with manual hygiene management (such as gensym or special syntax).

Our system operates on two simple, deterministic rules:

1. Automatic Hygiene for Definitions: All symbols defined within a macro template are automatically renamed to be unique, preventing conflicts with user code or nested macro calls.
2. Unhygienic Fallback for Free Symbols: All free symbols (those used but not defined within the template) are left untouched. They are resolved by the binder in the call-site scope (the scope where the macro was invoked).

This hybrid model ensures that internal macro variables are always safe, while simultaneously permitting powerful, context-aware macros.

---

2. The Core Challenge: Safety vs. Context

A macro, by definition, injects code into a foreign scope. This creates two distinct, opposing requirements.

Scenario A: The Need for Safety (Hygienic Definitions)

This is the classic hygiene problem. A macro must manage its own internal state without interfering with the user's code.

Consider a simple stopwatch macro:

(* Macro Definition *)
(defmacro stopwatch [body]
  `(do
     (def start-time (timestamp))
     (def result ~body)
     (print "Time: " (- (timestamp) start-time))
     result))

(* User Code *)
(def start-time "Important User Data")
(stopwatch (expensive-call))
(print start-time)

A naive (fully unhygienic) expansion would redefine the user's start-time variable, corrupting their program. This is unacceptable. The macro's internal variables (start-time, result) must be hygienic—that is, isolated from the call-site scope.

Scenario B: The Need for Context (Unhygienic Free Symbols)

Macros derive their power from interacting with the context in which they are called. A macro may need to read variables from the user's scope.

Consider a debug-print macro:

(* Macro Definition *)
(defmacro debug-print [msg]
  `(if *debug-mode*
       (print msg)))

(* User Code *)
(do
  (def *debug-mode* true)  ; User-defined context variable
  (debug-print "Test message")
)

In this case, the macro must access the user's *debug-mode* variable from the call-site scope. A "fully hygienic" system (which binds all symbols to the macro's definition-site scope) would fail, as it would be unable to see the user's local *debug-mode* variable.

---

3. Rejected Alternatives

To solve this conflict, several common designs were considered and rejected for failing one of the two core scenarios.

• Fully Unhygienic (The $ Suffix): This approach makes all symbols unhygienic by default and requires the author to manually mark internal variables (e.g., start-time$) for special handling. This inverts the desired default (safety) and fails to solve nested macro conflicts.
• Fully Hygienic (Strict Academic): This system binds all symbols (defined or free) to the macro's definition-site scope. This perfectly solves Scenario A but makes Scenario B impossible.
• Explicit Gensym (The Lisp Way): This requires the macro author to manually generate unique symbols ((let [start-sym (gensym)] ...)). This is syntactically complex, error-prone, and places an unnecessary burden on the author.

---

4. The Implemented Solution: Hybrid Hygiene

Our system resolves the conflict by treating definitions and free variables differently. The logic is handled entirely by the macro expander, requiring no special syntax from the macro author.

Rule 1: Automatic Renaming of Definitions

When the macro expander processes a template, it performs a pre-pass to identify all symbols being defined. This includes def forms, fn parameters, and let bindings.

• For each defined symbol (e.g., start-time), the expander generates a unique, internal-only name (e.g., start-time_G123).
• It stores this in an expansion-local rename map.
• It then replaces all occurrences of that symbol within the template with the new name.

This automatically and transparently solves Scenario A. The stopwatch macro's start-time becomes start-time_G123 and cannot possibly conflict with the user's start-time. This also solves the nested macro problem, as each expansion generates new unique names.

Rule 2: Unhygienic Fallback for Free Symbols

Any symbol in the macro template that is not part of a definition (a "free symbol") is left untouched by the expander.

• In stopwatch, the symbols do, timestamp, print, and - are free symbols.
• In debug-print, the symbols if, *debug-mode*, and print are free symbols.

The expander passes these symbols directly to the next compiler stage (the binder). The binder then resolves them, as it would any normal code, within the call-site scope.

This solves Scenario B. The binder finds *debug-mode* in the user's do block, exactly as intended.

---

5. Conclusion and Trade-Offs

This hybrid design provides "implicit hygiene" for the common case (internal variables) while defaulting to "unhygienic" behavior for external symbols, which provides maximum power and flexibility.

The primary trade-off of this design is that typographical errors in free symbols are caught late.

For example, if the stopwatch macro misspelled print as prnit:

1. The expander would see prnit as a free symbol (Rule 2) and leave it untouched.
2. The binder would then fail to find prnit in the user's call-site scope.

The resulting error (Undefined symbol: prnit) will point to the user's code where stopwatch was called, not the macro definition. This is a minor, acceptable trade-off for a system that achieves both safety and power without syntactic overhead.