Hy – A dialect of Lisp that’s embedded in Python (docs.hylang.org)

Serendipity :)

It's a kind of rite of passage, I think, to implement a Lisp. I wrote half (the easy half) of ISLISP that ran on the Python VM back in 2005 :)

Helluw, one of the core devs here. This was originally an idea just to display whats possible with the Python AST. It's pretty neat. The result is an kinda fancy lisp variant with bidirectional interop, macros and all the fancy lisp stuff.

Being an Emacs head, I like sexp based syntax for edition and navigation. Now I can edit hy like any lisp and push it to python hosting servers.

my 2 cents.

If you are a lisp-enthusiast, Python comes with strong support for data science and web applications, if nothing else.

If you are a Python enthusiast, lisp-like languages can be used to write DSLs or cut down on boilerplate code in a way which isn't as easy in Python or similar languages.

But yes, in general, a lisp dialect on Python has the problem that Python itself is a great language.

Python has one of the biggest standard libraries there is, there is a lot of useful stuff in there already provided, plus there is a humongous amount of third-party packages available through pip. It also generally has a pretty good C FFI, so there's a lot more support of C-libs in Python.

It's not so much about any specific libraries as it is about the volume of support one gets 'for free,' and as well I think Python syntax and semantics alike really get along better with Lisp than some of the other dialects that rely on FFIs (like Clojure to Java/JavaScript, or the number of C FFIs available for Schemes).

Lisps in general don't tend to include much of a standard library, opting for a 'batteries not included' model, with Racket being perhaps the most prominent (and quite deliberate) exception. So the idea of getting the whole Python ecosystem 'free' is potentially very appealing, especially to someone with a Python background (and there are a lot of those someones around, especially with it increasingly being a more popular teaching language than anything else except JavaScript).

This comes with any Python, out of the box: https://docs.python.org/2/py-modindex.html

What stands out will be different for different people. I use the ftp and telnet libs a lot, I've used the imap and email libs.

Here's a repackaging of a large number of 3rd party libs, packaged as an alternative python distribution, which I use at work: http://docs.continuum.io/anaconda/pkg-docs.html

I've explored twisted (networking), numpy and scipy (math and scientific), and sympy (symbolic math).

IPython's shell and notebook are great work environment enhancers. http://ipython.org/

Here's Python's larger set of 3rd parties: https://pypi.python.org/pypi?%3Aaction=browse

What about Clojure seems dense or implicit to you? Clojure and Python have a similar feel to me. Both are well-designed in the sense they strive for a small set of primitives useful in solving a broad range of problems.

Python natively supports this sort of list comprehension (https://docs.python.org/2/tutorial/datastructures.html#list-...), and the Lisp source is being translated into a Python AST, so that snippet is probably being directly mapped into whatever AST form is necessary to invoke the list comprehension.

You can create a list comprehension using a macro. In common lisp, lisp comprehension is implemented by the loop macro. http://www.gigamonkeys.com/book/loop-for-black-belts.html

It is a little more involved, but basically you define a syntax for list comprehension and then parse it using the macro to generate the final result.

"Real" List Comprehensions in 24 Lines of Lisp[0]

[0]: http://lisp-univ-etc.blogspot.com/2013/01/real-list-comprehe...

There's an excellent paper on how to build list comprehensions (in Common Lisp) on top of simpler iteration constructs:

http://www.ai.sri.com/~latendre/listCompFinal.pdf

I have a (very loose) implementation of this approach at:

https://github.com/TBRSS/cl-lc

Frankly, though, once you've learned the trick, list comprehensions lose a lot of their appeal.

How to implement list comprehensions in Lisp is an open-ended topic.

For instance, I did it once in a "researchy" way by implementing monads. With the help of CLOS, I was able to define various monads such as the list monad, identity monad, or state transformer monad (the map, join and unit being methods dispatched on the monad type).

Then a generic monadic comprehension macro provides different functionality based on which monad is selected. The list monad gives rise to a list comprehension. The identity monad gives rise to just sequential variable binding. And the state transformer monad ... to something like a pipeline of succcessive state transformations, I think.

http://www.kylheku.com/cgit/lisp-snippets/tree/monads.lisp

If you just want a list comprehension, it's not very difficult. Basically it has to expand to code which (for example) expresses a nested loop that steps every variable over its respective list. The expression being collected by the comprehension is stuck into the middle of this loop, inside a piece of code which collects its value into a list, which is stored in a hidden local variable. (Perhaps two local variables are used, to keep track of the head and tail of the list for the sake of efficiency.)

ANSI Lisp has functional applicators that process multiple lists. For instance if you want to add together values from two lists, you can do this:

  (mapcar '+ '(1 2 3) '(10 20 30)) 

   -> (11 22 33)

If you wanted to form a cross product of the two lists instead, the obvious thing would be to write a mapcar-like function:

  (mapcar-cross '+ '(1 2 3) '(10 20 30))

   -> '(11 21 31 12 22 32 13 23 33)

This function could be used as the target syntax for a comprehension macro so that, say:

  (list-comprehend (+ b a) (a '(1 2 3)) (b '(10 20 30)))

   -> '(11 21 31 12 22 32 13 23 33)

is macro-expanded into

  (mapcar-cross (lambda (a b) (+ b a)) '(1 2 3) '(10 20 30))

Translating (transliterating, really) the list-comprehend syntax into mapcar-cross is not very difficult: it's just some very straightforward nested list manipulation. The macro has to collect the list of variables from the trailing arguments after the expression, to form the (a b) argument list of the lambda. It has to remove the variables from those trailing arguments to produce the list expressions like '(1 2 3) and '(10 20 30) that will be applied to mapcar-cross. Then it has to assemble the mapcar-cross syntax.

Of course, you also have to write mapcar-cross, which is your "run-time support function" for your list-comprehend syntax (usable directly without that syntax, too).

You can create macros that look like functions, but the macro actually has access to the the "program code", so you can do arbitrary transformations.

The implementation is easier if you assume that you have exactly two generators. Here is an example in Racket. It uses define-syntax-rule that is the method to define simple macros (there are more advanced options).

(The program also define string->list/string that splits a list in a list of one-character string, because the result of string->list is a list of chars. You can ignore it safety.)

  #lang racket

  (define (string->list/string x)
    (map (lambda (c) (list->string (list c)))
         (string->list x)))

  (define-syntax-rule (list-comp2 (proc v ...)
                        [u1 gen1]
                        [u2 gen2])
    (map (lambda (u1 u2) (proc v ...))
         gen1
         gen2))
  
  (list-comp2 (list x y)
    [x (range 8)]
    [y (string->list/string "ABCDEFGH")])

  ;==> '((0 "A") (1 "B") (2 "C") (3 "D") (4 "E") (5 "F") (6 "G") (7 "H"))

Fundamentally it's macros and homoiconicity of code and data that give lisp it's expressive power. Combined they grant you the power to inject new syntax / domain specific languages (DSLs) directly into your code. Any new syntax you create can look like normal lisp or can have different syntax evaluation rules like infix math notation [1]. A number of years ago I did a short write-up on stack overflow giving a basic explanation of how macros work in common lisp which may be worth a read [2].

Macros in general work by being a syntax tree -> syntax tree transformation. The neat trick is it's not a static pattern like C - you can leverage functions, conditions, etc to build that syntax tree.

As for comprehensions themselves, if you want to see the power of lisp look at the loop macro. It's a single macro yet gives you the power of normal control loops, list comprehensions, for-each all rolled into one package. [3]

[1] http://data-sorcery.org/2010/05/14/infix-math/

[2] http://www.gigamonkeys.com/book/loop-for-black-belts.html

[3] http://stackoverflow.com/questions/267862/what-makes-lisp-ma...

For me, one of the simplest explanations of the difference between macros and functions is the short-circuiting and.

In many languages, and does not evaluate all of its arguments, only as many as it needs to until it finds #f. This sounds like an easy thing to implement, but in most languages it actually isn't, because order of nested operations means that without being asked, it will evaluate it's arguments before our and.

So we can write this broken-and easily enough:

  (define (broken-and . rest)
    (cond
      [(null? rest) #t]
      [(car rest) (apply broken-and (cdr rest))]
      [else #f]))

But give it the following, and you get an error:

  > (broken-and (zero? 0) (string? "dave") (null? '(is fat)) (/ 5 0))

  . . /: division by zero

And that will work so long as all your args are valid. But what will work better is to write a macro. Macros aren't evaluated, they merely splice in their syntax, but we can write them a lot like code, using all our control structures and so forth, and in Scheme/Racket, also some cool pattern matching stuff):

  (define-syntax my-and.v2
    (syntax-rules ()
      [(_) #t]
      [(_ a) a]
      [(_ a b ...) (if a
                       (my-and.v2 b ...)
                       #f)]))

Now we can try our input from before, and get what we want, because the macro expands recursively and won't ever get to our divide by zero, at least unless all preceding args are true:

  > (my-and.v2 (zero? 0) (string? "dave") (null? '(is fat)) (/ 5 0))
  #f

This property of only expanding, not evaluating, is why macros can write all kinds of clever control structures, because you're mangling syntax, not evaluating code.

You don't need to translate it -- you can import modules both ways just fine -- THAT's the killer feature.

By python-sh do you mean http://amoffat.github.io/sh/ ?

Argument's invalid as Rebol and Red are both members of the lisp family. It is like comparing the similarities of a circle with a ellipse. https://news.ycombinator.com/item?id=8613377

Also related : f-expr, a historical attempt at unifying them http://en.wikipedia.org/wiki/Fexpr .

Abandoned because the near impossibility of compilation when using them ( http://stackoverflow.com/questions/18324743/why-were-fexprs-... ).

And more recently http://web.cs.wpi.edu/~jshutt/kernel.html .

That's fascinating. What is performance like?

After a quick IRC discussion; you wrote it wrong :D! Using hy --spy so you can see the equivalent python code:

    => (list-comp (, x y) [x (range 5) y (range 5)] (> 2 x))   
    from hy.core.language import range   
    [(x, y) for x in range(5) for y in range(5) if (2 > x)]    
    [(0, 0), (0, 1), (0, 2), (0, 3), (0, 4), (1, 0), (1, 1) .....]

Can submit a PR and fix the docs so this is better displayed!

    r = []
    for x in range(5):
      if 2>x:
        for y in range(5):
          r.append((x,y))

    r = []
    for x in range(5):
      for y in range(5):
        if 2>x:
          r.append((x,y))

i started migrating a project from clojure to common lisp so that i could move the ui from swing to qt. i should really give hy + pyqt a try as well.

There is nothing implicit about that. You are instantiating an anonymous class (that's the reify) implementing 3 interfaces (presumably you looked these up since they're user libraries) that you are defining inline.

Compare this to something like a Django Rest Framework:

    class SnippetList(APIView):
        """
        List all snippets, or create a new snippet.
        """
        def get(self, request, format=None):
            snippets = Snippet.objects.all()
            serializer = SnippetSerializer(snippets, many=True)
            return Response(serializer.data)

        def post(self, request, format=None):
            serializer = SnippetSerializer(data=request.data)
            if serializer.is_valid():
                serializer.save()
                return Response(serializer.data, status=status.HTTP_201_CREATED)
            return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

reify doesn't require any special knowledge that class doesn't. The Python code here is subclassing a single abstract class (APIView) instead of 3, but it has the same "problem" in that you have to know what methods to override. Again this is user code so you probably looked up what APIView needs to work just like you looked up with InitState, IDidMount, and IRenderState.

reify vs class in Clojure vs Python is a matter of idioms. You can definitely create an actual class instead of reifying in clojure, it just isn't necessary. Likewise, you could create an anonymous type in python via type(), but you'd probably get fired and/or shot in most circles for doing so.

        serializer = SnippetSerializer(data=request.data)
            if serializer.is_valid():
                serializer.save()
                return Response(serializer.data, status=status.HTTP_201_CREATED)
            return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

        (let [events (sub (:notif-chan (om/get-shared owner)) :hello (chan))]
            (go
              (loop [e (<! events)]
                (om/set-state! owner :message (:data e))
                (recur (<! events)))))))

It will import cython stuff, and vice-versa.

It dosn't stop there. Why use the decorators directly when you can get Hy on Meth? https://github.com/hylang/hy/blob/master/hy/contrib/meth.hy

The entry is misleading. We don't really do TCO. It's just a trampoline calling the loop/recur form until there is nothing left to produce.

Same solution clojure has, as the JVM dosn't allow for TCO.

Well, that most likely won't change. Hy's goal is to be bidirectional, so there are a few things here and there that doesn't feel as lispy as it should. Like having data returned when you use assoc as an example. The scoping is an issue aswell, like let-over-lambdas doesn't work, and iteration over maps isn't as fluid as it is in Clojure.