Hygienic macro

In programming languages that have non-hygienic macro systems, it is possible for existing variable bindings to be hidden from a macro by variable bindings that are created during its expansion. In C, this problem can be illustrated by the following fragment:

#define INCI(i) { int a=0; ++i; }
int main(void)
{
  int a = 4, b = 8;
  INCI(a);
  INCI(b);
  printf("a is now %d, b is now %d\n", a, b);
  return 0;
}

Running the above through the C preprocessor produces:

int main(void)
{
    int a = 4, b = 8;
    { int a = 0; ++a; };
    { int a = 0; ++b; };
    printf("a is now %d, b is now %d\n", a, b);
    return 0;
}

The variable a declared in the top scope is shadowed by the a variable in the macro, which introduces a new scope. As a result, a is never altered by the execution of the program, as the output of the compiled program shows:

a is now 4, b is now 9

The hygiene problem can extend beyond variable bindings. Consider this Common Lisp macro:

(defmacro my-unless (condition &body body)
 `(if (not ,condition)
    (progn
      ,@body)))

While there are no references to variables in this macro, it assumes the symbols "if", "not", and "progn" are all bound to their usual definitions in the standard library. If, however the above macro is used in the following code:

(flet ((not (x) x))
  (my-unless t
    (format t "This should not be printed!")))

The definition of "not" has been locally altered and so the expansion of my-unless changes.

Of course, the problem can occur for program-defined functions in a similar way:

(defun user-defined-operator (cond)
  (not cond))

(defmacro my-unless (condition &body body)
 `(if (user-defined-operator ,condition)
    (progn
      ,@body)))

; ... later ...

(flet ((user-defined-operator (x) x))
  (my-unless t
    (format t "This should not be printed!")))

The use site redefines user-defined-operator and hence changes the behavior of the macro.

The simplest solution, if temporary storage is needed during macro expansion, is to use unusual variables names in the macro in hope that the same names will never be used by the rest of the program.

#define INCI(i) { int INCIa = 0; ++i; }
int main(void)
{
    int a = 4, b = 8;
    INCI(a);
    INCI(b);
    printf("a is now %d, b is now %d\n", a, b);
    return 0;
}

Until a variable named INCIa is created, this solution produces the correct output:

a is now 5, b is now 9

The problem is solved for the current program, but this solution is not robust. The variables used inside the macro and those in the rest of the program have to be kept in sync by the programmer. Specifically, using the macro INCI on a variable INCIa is going to fail in the same way that the original macro failed on a variable a.

In some programming languages, it is possible for a new variable name, or symbol, to be generated and bound to a temporary location. The language processing system ensures that this never clashes with another name or location in the execution environment. The responsibility for choosing to use this feature within the body of a macro definition is left to the programmer. This method was used in MacLisp, where a function named gensym could be used to generate a new symbol name. Similar functions (usually named gensym as well) exist in many Lisp-like languages, including the widely implemented Common Lisp standard[4] and Elisp.

Although symbol creation solves the variable shadowing issue, it does not directly solve the issue of function redefinition.[5] However, gensym, macro facilities, and standard library functions are sufficient to embed hygienic macros in an unhygienic language.[6]

This is similar to obfuscation in that a single name is shared by multiple expansions of the same macro. Unlike an unusual name, however, a read time uninterned symbol is used (denoted by the #: notation), for which it is impossible to occur outside of the macro, similar to gensym.

Using packages such as in Common Lisp, the macro simply uses a private symbol from the package in which the macro is defined. The symbol will not accidentally occur in user code. User code would have to reach inside the package using the double colon (::) notation to give itself permission to use the private symbol, for instance cool-macros::secret-sym. At that point, the issue of accidental lack of hygiene is moot. Furthermore the ANSI Common Lisp standard categorizes redefining standard functions and operators, globally or locally, as invoking undefined behavior. Such usage can be thus diagnosed by the implementation as erroneous. Thus the Lisp package system provide a viable, complete solution to the macro hygiene problem, which can be regarded as an instance of name clashing.

For example, in the program-defined function redefinition example, the my-unless macro can reside in its own package, where user-defined-operator is a private symbol in that package. The symbol user-defined-operator occurring in the user code will then be a different symbol, unrelated to the one used in the definition of the my-unless macro.

In some languages the expansion of a macro does not need to correspond to textual code; rather than expanding to an expression containing the symbol f, a macro may produce an expansion containing the actual object referred to by f. Similarly if the macro needs to use local variables or objects defined in the macro's package, it can expand to an invocation of a closure object whose enclosing lexical environment is that of the macro definition.

Syntax-rules is a high-level pattern matching facility that attempts to make macros easier to write. However, syntax-rules is not able to succinctly describe certain classes of macros and is insufficient to express other macro systems. Syntax-rules was described in the R4RS document in an appendix but not mandated. Later, R5RS adopted it as a standard macro facility. Here is an example syntax-rules macro that swaps the value of two variables:

(define-syntax swap!
  (syntax-rules ()
    ((_ a b)
     (let ((temp a))
       (set! a b)
       (set! b temp)))))

Due to the deficiencies of a purely syntax-rules based macro system, the R6RS Scheme standard adopted the syntax-case macro system.[10] Unlike syntax-rules, syntax-case contains both a pattern matching language and a low-level facility for writing macros. The former allows macros to be written declaratively, while the latter allows the implementation of alternative frontends for writing macros. The swap example from before is nearly identical in syntax-case because the pattern matching language is similar:

(define-syntax swap!
  (lambda (stx)
    (syntax-case stx ()
      ((_ a b)
       (syntax
        (let ((temp a))
          (set! a b)
          (set! b temp)))))))

However, syntax-case is more powerful than syntax-rules. For example, syntax-case macros can specify side-conditions on its pattern matching rules via arbitrary Scheme functions. Alternatively, a macro writer can choose not to use the pattern matching frontend and manipulate the syntax directly. Using the datum->syntax function, syntax-case macros can also intentionally capture identifiers, thus breaking hygiene.

Other macro systems have also been proposed and implemented for Scheme. Syntactic closures and explicit renaming[11] are two alternative macro systems. Both systems are lower-level than syntax-rules and leave the enforcement of hygiene to the macro writer. This differs from both syntax-rules and syntax-case, which automatically enforce hygiene by default. The swap examples from above are shown here using a syntactic closure and explicit renaming implementation respectively:

;; syntactic closures
(define-syntax swap!
   (sc-macro-transformer
    (lambda (form environment)
      (let ((a (close-syntax (cadr form) environment))
            (b (close-syntax (caddr form) environment)))
        `(let ((temp ,a))
           (set! ,a ,b)
           (set! ,b temp))))))

;; explicit renaming
(define-syntax swap!
 (er-macro-transformer
  (lambda (form rename compare)
    (let ((a (cadr form))
          (b (caddr form))
          (temp (rename 'temp)))
      `(,(rename 'let) ((,temp ,a))
           (,(rename 'set!) ,a ,b)
           (,(rename 'set!) ,b ,temp))))))

• Scheme – syntax-rules, syntax-case, syntactic closures, and others.
• Racket – a Scheme variant, its macro system was originally based on syntax-case, but now has more features.
• Nemerle[12]
• Dylan
• Elixir[13]
• Nim
• Rust
• Haxe
• Mary2 – scoped macro bodies in an ALGOL 68-derivative language circa 1978
• Julia[14]
• Raku – supports both hygienic and unhygienic macros[15]