scheme shell
about
download
support
resources
docu
links
 
scsh.net

Hygiene Versus Gensym

GENSYM does not provide all the protection that hygiene does. This brief explanation shows why.

Take the example of a SWAP! macro, which should have a definition somewhat like this:

      (SWAP! <var1> <var2>)
        --expands--to-->
      (let ((value VAR1))
        (set! VAR1 VAR2)
        (set! VAR2 value))

Suppose we wrote a very naive definition, using DEFMACRO with Common Lisp syntax, but expanding to Scheme:

      (defmacro naive-swap! (var1 var2)
        `(let ((value ,var1))
           (set! ,var1 ,var2)
           (set! ,var2 value)))

Any Common Lisp programmer will tell you that that is a very vulnerable macro; if you were to try either (naive-swap! value something) or (naive-swap! something value), VALUE would be inadvertently captured, and the macro would break. That same Common Lisp programmer would probably tell you to rewrite it as:

      (defmacro cl-swap! (var1 var2)
        (let ((value-symbol (gensym)))
          `(let ((,value-symbol ,var1))
             (set! ,var1 ,var2)
             (set! ,var2 ,value-symbol))))

This new definition is greatly improved: VALUE is not captured at all; the example that broke the old definition will now work perfectly file. Indeed, to many, it would seem that CL-SWAP! is good enough as it is and there are no more problems with it. But unfortunately that idea is wrong. Suppose someone wrote something like this:

      (let ((set! display))
        (cl-swap! a b))

CL-SWAP! would gleefully expand to

      (let ((set! display))
        (let ((value a))
          (set! a b)
          (set! b value)))

Performing a simple beta-substitution, we get:

      (let ((value a))
        (display a b)
        (display b value))

Oops. That's not quite what SWAP!, in any variation, should do at all.

Now, this may not seem particularly relevant, because, really, how often do you locally shadow SET!? However, think of this problem in the case of other situations; you have the same sorts of problems with dynamic scoping. Indeed, imagine that you wrote a code snippet like this, in a language that looks like Scheme but is dynamically scoped:

      Module A:

(define v ...) (define (f ...) ...v...(set! v ...)...v...)

Module B:

(define (g <args>) ...(let ((v <some-local-value>)) ...(f ...)...))

Whoops. F gets G's local V. The author of B didn't even know about V, as it's a hidden part of the implementation of module A. Code breaks unexpectedly just as it would with dynamic scoping. This is exactly the same problem. Now, you could solve it by using unnecessarily verbose identifiers, which gets to be a pain. Hygiene already provides you with the solution, though: LET & SET! from the original SWAP! example just get renamed by the macro system; they new refer exactly to the LET & SET! that the SCHEME package provides; no matter what else is called LET & SET! in the same location, those three uses of the two forms will always refer to the right binding.

Here is the definition of HYGIENIC-SWAP!, which hygienically renames all of the identifiers it introduces uses of, in the explicit renaming macro system -- described in [1] --, which we use to make explicit where hygiene 'happens:'

      (define-syntax hygienic-swap!
        (transformer ; Drop this for Scheme48.
          (lambda (form rename compare)
            (destructure ; Open the DESTRUCTURING package first, of course.
                (((swap! var1 var2) form))
              `(,(rename 'let) ; Be sure to rename LET so we get SCHEME's LET.
                   ((,(rename 'value) ; Rename VALUE so we don't inadvertently
                                      ; and unhygienically capture it.
                     ,var1))
                 (,(rename 'set!) ; Rename SET!, too, to get SCHEME's SET!.
                   ,var1
                   ,var2)
                 (,(rename 'set!)
                   ,var2
                   ,(rename 'value) ; It's safe to apply RENAME to the symbol
                                    ; VALUE more than once, because RENAME is
                                    ; always referentially transparent.
                   ))))))

Now, let us observe a sample output. <name>~<colour> identifies NAME with a special colour:

      (hygienic-swap! value something)
        --expands--to-->
      (let ((value~0 value~1))
        (set! value~1 something~0)
        (set! something~0 value~0))

It looks like we fixed the problem NAIVE-SWAP! had with inadvertently capturing VALUE; let's try it with the problem CL-SWAP! has. In this output, rather than using colour, the package that a variable reference (rather than an ordinary symbol) refers to is made explicit; <package>##<name> refers to the binding of NAME in PACKAGE:

      (let ((set! display))
        (hygienic-swap! a b))
        --expands--to-->
      (scheme##let ((set! scheme##display))
        (scheme##let ((value a))
          (scheme##set! a b)
          (scheme##set! b value)))

Now the user of HYGIENIC-SWAP! locally binds SET!, not SCHEME##SET!; thus our macro HYGIENIC-SWAP! doesn't inadvertently use the SET! that the user of the macro introduced.

There is also another problem with inserting symbols directly; this example implementation of FORCE & DELAY demonstrates it quite well:

      (defmacro cl-delay (expression)
        `(make-promise (lambda () ,expression)))

(define (force promise) (promise))

(define (make-promise thunk) ...)

The problem is this: MAKE-PROMISE is not exported from any module; the only interface to lazy evaluation is FORCE & DELAY. So when you use CL-DELAY, it expands to a use of MAKE-PROMISE, but MAKE-PROMISE is unbound. Again, hygiene saves you here:

      (define-syntax hygienic-delay
        (transformer
          (lambda (form rename compare)
            `(,(rename 'make-promise) ; Rename MAKE-PROMISE so we get a binding
                                      ; to the MAKE-PROMISE that our module for
                                      ; lazy evaluation does not export.
              (,(rename 'lambda) ; Rename LAMBDA, too, just in case. (You never
                                 ; know what those whackos might do to LAMBDA!)
                  ()
                ,(cadr form))))))

Conclusion: GENSYM is not a suitable replacement for real hygiene. Common Lisp's package system solves the problem of CL-DELAY, though the merit of that solution is disputable, and I don't want to get into debates about module systems between Common Lisp and Scheme (it's bad enough when only Scheme is involved!).

[1] William D. Clinger. Hygienic macros through explicit renaming. Lisp Pointers IV(4), 25-28, December 1991. http://www.bloodandcoffee.net/campbell/papers/clinger91exrename.ps


HygieneVersusGensym - raw wiki source | code snippets archive