CS 2360 Lecture 17: February 9

Macros

One of the key components in extending LISP or building new languages on top of LISP is the "macro". A macro works like a function, except that when LISP sees a call to a macro in some code that is being executed, the definition of the macro is inserted in place of the macro call. This is in contrast to a "normal" function call which results in a copy of the current environment being saved on the program stack, followed by the execution of the function definition, which in turn is followed by the "popping" of the program stack. Another way to think of the difference between functions and macros is this: A function produces results, while a macro produces expressions which, when evaluated, produce results.

We've seen lots of macros in use already: dolist, dotimes, setf, do, defun, case, etc. You should look at pp. 1026-27 of Steele for an exhaustive list of predefined LISP macros.

Macros and their arguments

One big important feature of macros is that they allow you to create constructs which do not evaluate their arguments. The importance of this may not be immediately obvious. An example will help you see the light. Say that you had a version of LISP in which there was no "if" form, and you wanted to create one. How would you do it?

Your first thought might be to create an ordinary function:

(defun my-if (test then-part else-part) (cond (test then-part) (T else-part))) That looks reasonable, no? Let's test it:

? (my-if t 2 3) 2 ? (my-if nil 2 3) 3 ? Perfect. Exactly what we want. Let's test it just a little bit more:

? (my-if t (print "yes") (print "no")) "yes" "no" "yes" ? OOPS!! What happened here? Why does "my-if" do that? The problem is that "my-if" evaluates its arguments, and these particular arguments have side-effects when evaluated. So, even before the "my-if" definition is applied to the arguments, the two "print" expressions are evaluated, resulting in both "yes" and "no" being printed on the terminal. Only then is the function applied to the arguments, and the result of evaluating (print "yes") is returned. But that's not what "if" does, is it?

How then does "if" work? It's actually something called a special form, hard-coded into the LISP interpreter. You can't write a special form; they're written only by the folks who created the interpreter. But, "if" could be implemented as a macro, and that's something you could do yourself. How? By using another macro called "defmacro":

(defmacro my-if (test then-part else-part) (list 'cond (list test then-part) (list 'T else-part))) When LISP then sees:

(my-if nil (print "yes") (print "no")) it first "expands the macro", replacing the macro call with:

(cond (nil (print "yes")) (T (print "no"))) In other words, anything that's quoted is used literally, and anything that's not quoted is replaced by the value it's bound to at the time the macro call is encountered.

Backquote-comma syntax

We could write our macros as shown above, but the macros that result don't look much like the code they're going to expand into. This makes macros hard to read and debug. So a syntax for macro creation has been invented, which when used makes these things look more like templates for the code to be generated.

The backquote-comma syntax introduces two new operators: the backquote (`) and the comma (,). What did you expect? The backquote tells LISP to quote everything in the list that follows (so in that sense it works just like quote), except for those items immediately preceded by a comma.

The comma works only within the scope of a backquote. It says "turn off the quoting for now". When it appears in front of a list element, it cancels out the quote that would have been in effect otherwise. So instead of the literal symbol being put into that template, the value of that symbol at the time of the macroexpansion is put there.

In other words, backquote makes a template while comma makes a slot in the template.

Here's "my-if" again, using backquote-comma syntax:

(defmacro my-if (test then-part else-part) `(cond (,test ,then-part) (T ,else-part))) Note: In the LISP version of "if", the "else-part" is actually an optional argument. We could get that same effect here by doing the following:

(defmacro my-if (test then-part &optional else-part) : : In either case, this expands exactly the same way as our previous macro definition, but it's much easier to read because it looks much more like the code that it will turn into, no?

How to define simple macros (taken from "On LISP" by Paul Graham)

Let's walk through another macro example in gory detail and, along the way, we'll outline a step-by-step process for defining macros.

You're familiar with "member", right? This function uses "eql" to test for equality by default. But you could use an optional argument to use a different test:

(member item my-list :test #'eq) If we use this a lot, we could make our own variant of "member" that always uses "eq". In older dialects of LISP, this was called "memq":

(memq item my-list) We could define "memq" as an ordinary function:

(defun memq (item my-list) (member item my-list :test #'eq)) but that wouldn't tell us much about macros, would it? Let's define "memq" as a macro. Graham tells us that the first step in macro definition is to begin with a typical call to the macro that we want to define, and write it on a piece of paper. Below that, we should write the expression that we want the macro call to expand into:

call: (memq item my-list)

expansion: (member item my-list :test #'eq)

Then, we begin the macro definition like this:

(defmacro memq And we construct the argument list for the macro definition from the parameters in the macro call. To avoid confusion, we invent new names for each of these arguments:

(defmacro memq (obj lst) Now we go back to the call and expansion that we wrote down. For each argument in the macro call, we draw a line connecting it with the place it appears in the expansion: 

call:        (memq item my-list)
                     \     \
                      \     \
expansion:   (member item my-list :test #'eq)
Now we look at the expansion, and while reading the expansion, we start constructing the macro body. We begin with a backquote:

(defmacro memq (obj lst) ` Then, whenever we see a parenthesis in the expansion that isn't part of an argument in the macro call, we put that same parenthesis in the macro definition:

(defmacro memq (obj lst) `( And then, for each expression in the expansion, we do the following:

  1. If there is no line connecting it with the macro call, we write down the expression itself, as is.
  2. If there is a connection to one of the arguments in the macro call, we write down the symbol which occurs in the corresponding position in the macro parameter list, immediately preceded by a comma (i.e., we don't want this quoted...instead, we want to pass a value through here).
Since there's no connecting line to the first element, "member", we write it down as is:

(defmacro memq (obj lst) `(member On the other hand, "item" does have a connecting line, so in the macro body we insert the corresponding parameter, preceded by a comma:

(defmacro memq (obj lst) `(member ,obj And we do this stuff until we're done:

(defmacro memq (obj lst) `(member ,obj ,lst :test #'eq)) That was easy, wasn't it?

The LISP evaluator, revisited

All this macro stuff suggests that our concept of what the LISP evaluator does isn't quite complete. Let's update that---here's how it really works (taken from "Common LISP Programming for Artificial Intelligence" by Hasemer and Domingue):

  1. If the expression is an atom, return its value.
  2. If the expression is a list, then either
    1. If the first member of the list is a special form, handle it and its arguments accordingly.
    2. If the first member of the list is the name of a macro, call the function "macroexpand" to expand it. Then go to step 1.
    3. If the first member of the list is the name of a function, then retrieve the function body and
      • Find the values of the remaining members of the list by applying these rules to each of them in turn.
      • Apply the function from (2.3) to the arguments from the rule above and return the result.
Things to remember: First, as noted before, macro expansion does not evaluate the macro's arguments. The macro is expanded before the arguments are evaluated by the evaluator. Second, macro expansion is performed as long as the first element of the expression is the name of a macro.

The "macroexpand" function

The key to macro expansion in LISP is the "macroexpand" function. This function takes one argument, which is quoted. And that argument is a macro call. So let's say we define our "my-if" macro:

Welcome to Macintosh Common Lisp Version 2.0! ? (defmacro my-if (test then-part else-part) `(cond (,test ,then-part) (T ,else-part))) MY-IF ? If we then want to see how a macro call to "my-if" will expand, we can do this:

? (macroexpand '(my-if (eql 1 2) (print "yes") (print "no"))) (IF (EQL 1 2) (PROGN (PRINT "yes")) (COND (T (PRINT "no")))) T ? Note that "macroexpand" returns not one result, but two. Weird, eh? Read pp. 179-187 of Steele for more about this. In any case, the first value is the expanded code, and the second result tells us whether the first element of the argument was the name of a macro. If it was, then the second value returned will be T, and the first value will be different from the input argument. If the first element was not the name of a macro, the second value returned will be nil, and the first value will be the unchanged input argument.

Also note that the result of the expansion above isn't what we expected, at least based on our macro definition. That's because we put a macro call inside our macro definition---"cond" is the name of a macro. So "macroexpand" did one expansion of the call to "my-if", and the result of that was an expression beginning with "cond". Since "cond" is the name of a macro, "macroexpand" was called again, resulting in yet another expansion. This new expression began with "if", which is a special form, not a macro, so macroexpansion ceased at that point.

If we want to see the result of just one level of expansion so we can check to see if our macro expands the way we intended it to, we use another function called "macroexpand-1". Here's the result of one level of expansion:

? (macroexpand-1 '(my-if (eql 1 2) (print "yes") (print "no"))) (COND ((EQL 1 2) (PRINT "yes")) (T (PRINT "no"))) T Here's the what happens when we expand the result of the previous expansion:

? (macroexpand-1 '(COND ((EQL 1 2) (PRINT "yes")) (T (PRINT "no")))) (IF (EQL 1 2) (PROGN (PRINT "yes")) (COND (T (PRINT "no")))) T And here's what happens when we try to expand that result, which starts with the name of a special form, not a macro:

? (macroexpand-1 '(IF (EQL 1 2) (PROGN (PRINT "yes")) (COND (T (PRINT "no"))))) (IF (EQL 1 2) (PROGN (PRINT "yes")) (COND (T (PRINT "no")))) NIL

Advantages and disadvantages of macros

There are two key advantages to using macros:

  1. The arguments to macros are not evaluated. Thus, new syntactic constructs can be defined with macros. This is essential to defining new languages on top of LISP, especially languages that aren't constrained in ways that functional programming languages often are.
  2. Macros produce faster (but more) code than functions because they avoid the overhead of function calls.
But there are also disadvantages:

  1. If you compile macros, you can't detect them with your usual debugging tools, so "trace" and "step" won't work.
  2. Since macros aren't functions, they can't be used with "funcall", "apply", or the mapping functions.
  3. Again, if you compile macros, when you redefine a macro, you must recompile every function whose sourdce contains a call to the macro. If you don't, your source code may contain the results of expanding old macro definitions instead of the new ones. You may think the new macro has been expanded where the macro call occurs, but it won't be. This can be especially aggravating if the old macro definition resulted in code which didn't blow up. So your program does something, it just doesn't do what your new macro definition says it should do. You can spend lots of hours spinning wheels trying to debug that kind of thing.
See you at the exam... Friday, first period (ugh) 8 am.

Back To The CS2360 Home Page

Ian Smith (iansmith@cc.gatech.edu)
Last Modified 27 Feb 95 by Ian Smith (iansmith@cc.gatech.edu)