Ask HN: What Is the Lisp “Enlightment”?

I think that the enlightenment phase of Lisp is different for everyone, so you're going to get some varied responses.

Learning Lisp certainly improved the way I write code in other languages and moved me to a more functional style of programming. So while I don't regularly code in Lisp, I do use a lot of concepts that can be traced back to Lisp.

For me Lisp enlightenment was when I realized everything is lambda all the way down (except of course for lists and atoms). I mean we don't really need all the various fancy let, let*, defun expressions to do programming. let, defun are all just abstractions on top of LAMBDA expressions. That means that we don't need some 30 odd keywords to bootstrap a language like we do in C. We can bootstrap a language implementation with some 8 or so special forms and everything else can be defined in terms of those. Isn't that beautiful?

For me, it was the realization that with LISP, you don't really build much of anything because you're caught in the blinding light of beauty, and the perfect program is the one that never gets written.

That code is just data.

In lispalikes, code has the same "shape" as any other data.

The compiler / language environment and the runtime environment are "blended" in a much more natural way than is usual.

Thinking about ASSERTS FILL-IN-THE-BLANKS requires more meditation.

You have not yet reached enlightenment.

    A koan is incomplete.

Please meditate on the following code:

    File "koans/asserts.lisp"

    Koan "FILL-IN-THE-BLANKS"

    Current koan assert status is "(INCOMPLETE INCOMPLETE INCOMPLETE)"

You are now 0/198 koans and 0/31 lessons closer to reaching enlightenment.

Lisp is 64 years old. In some ways its reputation is oversold. A lot of features (GC, varargs, lexical scope, closures, higher-order functions, dynamic method dispatch, reflection) first gained prominence in Lisp, despite being extravagantly expensive. Some optimizations were found, computers became vastly more powerful, and now those innovations are merely table stakes for any modern language.

IMHO the biggest exception is macros. Lisp isn’t the only language that has them, but it’s still the easiest by far if you ever want a domain-specific language to concisely express some problem. The grammar of its special forms is simple enough that we can easily use the symbol and linked-list builtins to generate boilerplate expressions or functions for the compiler without fighting with in-memory parse tree nodes or generating entire files of syntactically valid source.

At the end of the 70's I bought a HP67 pocket calculator and discovered that adding 1 and 2 was done using the strange reverse Polish notation, "1 2 +". Much later I came across LISP and discovered prefixed parenthesized expressions, {+ 1 2}, i.e. the normal polish notation in parentheses. Then I discovered that an HTML expression like "<b><i><u>Hello World</u></i></b>", requiring that closing brackets be written in the reverse order, would be better written in a LISP style, {b {i {u Hello World}}}, without boring anymore with the order of closing brackets. Finally I found that if writing {u 1 2 3 4 5 6} displayed "1 2 3 4 5 6" in underline style, writing {* 1 2 3 4 5 6} displayed the product, "720", and writing {u {* 1 2 3 4 5 6}} displayed "720" in underline style.

Mixing pure text and code became obvious. This made me want to write a text editor with a programming language and its current state is in this wiki - http://lambdaway.free.fr/lambdawalks

For me - I'm just an "amateur", it was that the Lisp "Enlightment", just to give me a way to implement from scratch a small engine to go down to the foundations of code, discover the lambda-calculus, and have a chance to understand and enjoy coding.

Homoiconicity

Before Lisp, write code, solve problems. After Lisp, write code, solve problems.

Lisp/Clojure allows you to solve/think about problems at a higher level, do a hacker rank puzzle in java/C++ and then do the same using clojure.

i dunno, i like keyword parameters

While it's part of homoiconicity, I'd stress that a great part of the language is that it grows without warts. Because the "syntax" is so insanely regular, new layers of abstraction look just like the builtin layers.

With something like SBCL, you can literally write assembly in lisp notation (called VOP in SBCL), then layer the Common Lisp language on top seamlessly, then layer your domain-specific code on top with your code looking and acting just like it is part of the native language.

When you're done, you've built language primitives up to deal with your specific problem domain then you can solve the problem with a language custom-made for your needs.

I've found this essay to be a good shortcut to Lisp "enlightenment" without having to put in the effort of actually learning/using Lisp:

https://stopa.io/post/265

The big topic is meta-programming with symbolic expressions

1) one can generate code and evaluate it

2) one can write simple interpreters, see McCarthy's first Lisp interpreter in Lisp -> your own eval

3) one can write procedural source code transformations, where the code is Lisp data -> transforming Lisp programs by Lisp programs -> apply the same development tools to the meta-level -> integrate it directly into the evaluator

The englightment is understanding how a minimal Lisp machinery for this works and that things fit together quite well.

4) how to apply source code to various software problems, for example in declarative programming or how to provide simpler interfaces to more complex, say, object-oriented mechanisms (-> example CLOS)

then there are a large number of consequences / further directions

* addressing the deficiencies of that model

* compilation

* language building (embedded DSLs)

* editor interfaces

* interactive programming

* UI representations as data

* machine models

A fundamentally interactive programming paradigm. This function contains the heart of lisp "enlightenment": http://clhs.lisp.se/Body/f_upda_1.htm

Common Lisp has many features that newer languages like Python have adapted in the last 15 years.

And yet, it is dramatically simpler.

I have been using Common Lisp professionally since 1982, and other Lisps before that. It has always been my language for thinking and experiments.

If you are just starting, maybe you would be better off using Racket Scheme - it has a modern and rich ecosystem around it.

If you do go with Common Lisp, you can grab a free copy of my book by setting the price to $0.00 https://leanpub.com/lovinglisp There are better introductory Common Lisp books, but mine has many fun example programs that you might enjoy if your technical interests overlap mine.

In my experience people are talking about a couple of different things when saying Lisp enlightened them:

1. The realization that code and data are the same kind of entity. From this realization springs the realization that you can use data structures for managing code and that leads you to Lisp macros - which is one of the superpowers of Lisp. Lisp stands for LISt Processor and Lists are the primary data structure (okay, technically CONSes are, but let's not confuse things) for storing both data and code. Why all the funny parenthesis in Lisp? Because that's the syntax for a list data structure! '(1 2 3) is a list of the first three positive integers and (add 1 2 3) is a list whose first member is the symbol 'add' - which is a function. When evaluated it will yield the value 6. There - now you know Lisp syntax!

2. The realization that many algorithms can be easily implemented using a list data structure having two simple operators: first and rest (CAR and CDR in Lisp parlance) and recursion. This is why you see so many articles about implementing Lisp in <insert popular language here>. It doesn't take a lot to get a Lisp interpreter working, in fact it used to be a routine exercise for undergraduate CS students. Evaluating (car '(1 2 3)) yields 1, and (cdr '(1 2 3)) yields '(2 3). A recursive function in Lisp then typically has two parameters, a list and an accumulator. If the list is NIL then you return the accumulator, otherwise you update the accumulator by applying some operation to the CAR of the list and the input accumulator and recursively invoke the function with the CDR of the list and the updated accumulator. Simple. Tail-call optimization also makes this highly-performant.

3. The difference between symbols in Lisp and variables in other programming languages. In many commonly-used programming languages variables refer to memory locations. That's not strictly true in Lisp. Symbols are an item in a symbol table and the current symbol table is context-sensitive. That means symbols have properties, including user-defined properties, and the same code may be executed with different symbol tables in effect - which is another part of how macros can be so powerful.

4. Functions are first-class objects. This is also different from many other commonly-used programming languages, though that has been changing. A symbol can evaluate to a function rather than a value, functions can be created on the fly (lambdas) and passed as parameters to other functions. You can have a list containing functions (such as (add 1 2 3) from above) and evaluate those functions and so forth. A function is simply another data type. This was more of a big deal 10 or more years ago, now many languages have been adding this capability.

5. The REPL. It's because code and data are the same construct that the REPL as you know it can exist. Even better is when you realize not all functions in the REPL have to be interpreted - you can compile them! You can freely intermix interpreted and compiled code together. You can load compiled code into your session, add new code, modify existing code - whatever you need to do. It's an extremely creative and productive development environment.

That's it! That's the Lisp enlightenment.

I'd say that Common Lisp is still the ultimate weapon of a solipsistic developer, with unprecedented raw power for reinventing the world from scratch without collaborating with others. For a cohort of people, this feeling can be very liberating and enlightening.

In my opinion, most modern dynamically-typed languages are actually better than Common Lisp in almost all regards that matter for social, collective programming:

- they are usually much more functional and declarative: more immutability, more value-oriented approach (Erlang, Clojure), less unpredictable mutation, more useful abstractions built into the language. They provide modern data structures in standard libraries in a useful and uniform way.

- they provide a builtin class/object/prototype common language for their ecosystems (CLOS in Common Lisp and Moose in Perl 5 are examples of too flexible opt-in OOP systems that fragment ecosystems instead of unifying them); this also gives decent namespaces, which are crucial for cooperation between teams.

- they usually have a good async story (iterators/generators/coroutines, promises/futures, async/await), again, built into the language and enabling async ecosystems on top of them.

- they have good(-ish) story about package management. Lispers seem to like to write everything from scratch.

- they have good story about testing infrastructure. Lispers are usually too proud of keeping the world in their heads and of their REPL-driven bug-fixing dopamine cycle to actually think about having computer test their code. The ability to change a running program from the inside is actually a detriment to abstract thinking about what code is meant to do and describing that in more declarative invariants like type systems and tests.

- tooling has finally caught up: with LSP and treesitter, programmers do not have to bear the pain of S-expressions to have structural editing anymore. A sufficiently educated programmer recognizes syntax trees as a valid data structure in any language. If really needed, modern languages do provide abstractions to manipulate their AST (`import ast, dis` in Python, `erl_syntax` in Erlang, `ast` in Go, declarative and procedural macros in Rust, Template Haskell, etc).

The only thing which I think Common Lisp still does better is conditions/restarts, which seems a better way of error handling than just catching exceptions.

Related: Lisp, Smalltalk, and the Power of Symmetry - https://medium.com/smalltalk-talk/lisp-smalltalk-and-the-pow...

This is what enlightenment looks like: https://aphyr.com/posts/353-rewriting-the-technical-intervie...

At the risk of explaining the joke and ruining it: the author is a transcendental being doing a fizz-buzz coding interview. The interviewer doesn't like the fact that the author copied and paste in a solution....so the author then goes off and writes an interpreter in Clojure (Clojure is a Lisp) that actually interprets the original code snippet.

---

What I actually love and enjoy about Lisp (Clojure in particular) is how malleable it is. Functional Programming is the paradigm you hear about the most, but you can write object-oriented code (lambdas simulate objects very easily, see SICP) or Logical programming (clojure/core.logic) or, ultimately, imperative programming.

Adding new "features" to the language is something every Lisp intro book should include. Don't have an `until` operator? (the opposite of a `while`) bam: macro. It's trivial.

Want to have goroutines? There's a library: core.async. Languages like Go get popularity because of their features like goroutines, Lisps can just import a library and get the same (or similar).

I'll fully admit, I bounced off Clojure for the first three times I tried learning it. But when it clicks is when you assemble a handful of map/reduce/apply and transform your data with ease, instead of shoving it around. Lisp makes your functions and data sing. Lisps allow you to find higher levels of composing functions, e.g. Transducers (https://clojure.org/reference/transducers)

Eval