What makes Lisp macros so special?

What programming languages are you familiar with already?

One example that I feel most devs can appreciate is the elimination of "boilerplate" code. How often in your non-macro language do you find yourself repeating the same lines of code with minute differences?

Think about JavaScript before and after destructuring was implemented. How many lines of code and mental effort are saved!

In Common Lisp this is just a macro, DESTRUCTURING-BIND (http://www.lispworks.com/documentation/HyperSpec/Body/m_dest...). Not a fan of the syntax? Just write a macro to modify how you use it.

Macros allow you the programmer to do similar things, but with anything you want. You don't have to wait for a language committee to agree on something, and then a third party to write a convoluted pre-processing system to 'transpile' the code to work. It's built into the language.

pepper_sauce's example is a good one, the ability to mold the language to what features you need today is a big point.

But even beyond that I think the big purpose of macros is the ability to mold the language the application you're applying it to. This is the Domain Specific Language (DSL) idea.

For instance, take Protocol Buffers. The .proto format is a language that lets you define your messages and denote which fields are required, optional, etc. You run a compiler against these .proto files that spits out a generated library for the platform you're on.

This sort of workflow isn't as onerous as it used to be with all the CI/CD push to have tools that build your application in one step. But it is still a separate language with its own compiler that must be maintained entirely outside of your project or language ecosystem.

If, in a hypothetical universe where Lisp dominated Google, protocol buffers had been written in Lisp you wouldn't need a separate compiler and language. A library of macros like defpackage and defmessage could be written that would generate the code for sending and receiving each message type you define inline with the rest of your application. It would all be Lisp code and would benefit from all of your Lisp tools, like linting, debugging, and testing.

And if it was written in a Lisp-y tradition, such a DSL would likely be riddled with hooks for your application code to be able to attach to dynamically, say if you would like every receipt of a message to be logged or if a message fails validation you would want to page the on-call to figure out who's sending bad messages and why.

So, in short, macros allow you to extend the language for reasons even beyond syntactical conveniences like DESTRUCTURING-BIND. It allows you to really shape your tools to the application at hand, and allows others to shape the tools you share to their applications.

18-19 years ago, I didn't switch to Lisp because of macros.

I was hooked by the list representation: how cons cells can be treated as immutable to implement list operations by always allocating new cells that are garbage collected.

I was hooked by the uniform syntax: good-bye ambiguities requiring associativity and precedence rules.

I was charmed by the empty list being nil which is also Boolean false, and the convenience of (car nil) just returning nil: the resulting code looked slick and elegant to me.

I thought it was great how you can just quote any part of the syntax and you get an expression which gives you that syntax as a piece of literal data.

Eventually I learned about defmacro, too, by which time I was productively using Lisp.

I learned about backquote before macros: wow, how great, I thought: like "foo $bar" string interpolation in the shell but with actual data structure.

I got into Lisp not because of some online hype about macros, but because I happened on a copy of Stuart Shapiro's Common Lisp: An Interactive Approach and worked through that. Now that book can be downloaded in electronic form.

One thing that charmed me was the Common Lisp read-table driven dispatch. Knuth's TeX does something very similar. I imitated Knuth's idea in a macro preprocessor that I wrote in 1999 called mpp; the preprocessor was able to redefine its read syntax by re-assigning the categories of characters. When I discovered that Lisp has a character-categorizing dispatch table as the basis of its scanning, I was really bowled over: it was something I had already liked to the point of having implemented it. (Today, I'm not so hot on read tables any more; my TXR Lisp doesn't have the feature.)