I can not believe for once that this guy obtained hacking nirvana through Arc. Not once! Why? because the Arc documentation is so meager as to be non-existent. You will need serious feet-wetting with more established Lisp dialects before you can "get there" with Arc.
All the epiphanies he lists are easily experienced with most modern Lisp dialects; either he "got it" before and just needed to extend the credit to Arc, for some reason .. or he isn't there yet, at least not to its full potential.
It had to be said, as much as I hate to be rude and blunt like this :-|
This is why I stopped reading C.L.L. - The notion that it's inconceivable to genuinely like any other language than Common Lisp. It's a bizarre idea, and one that carries much more fervor than I think CL has earned. If this were still 1990, then I suppose it would have a stronger leg to stand on.
Don't get me wrong, I really like Common Lisp (well, all I've ever played with is SBCL, so maybe I would like another variant more or less). I just don't see why it's impossible for someone to actually like Arc, or Clojure, or PLT Scheme or any other little-L lisp instead of Common Lisp.
So it's a personal-preference choice made after sufficient deliberation and experience in all the options on the table, I hope.
Ok. I will accept that.
Here is my take on your personal essay, it's also a matter of personal opinion made after sufficient technical evaluation of the matter: I think you need to be a little more critical of your own reasoning, specially when writing for programmers. The spirit of your essay has been largely positive, IMO, but go easy on the snide remarks and the dismissive attitude of other technologies you have very little experience in (yeah, it shows.) Otherwise a much less sympathetic reader than I, and a far more competent one, will come aboard and tear your argument to pieces.
A better way to frame your enthusiastic writing would be "I can do X in Arc, I really wish ${LISP_DIALECT} would let me do the same". This is both curious and also accepting of one's own self-limitations. Someone will come aboard and show you what you have been missing, and you might as well get pointers to the literature that further your understanding of Arc or whatever other technology.
Can you give me an example of a snide remark that could be rewritten?
(I appreciate your suggestion to say that I wish another Lisp dialect was more like Arc, but I'm not sure how I'd choose one since I don't really have a favorite aside from Arc).
> Can you give me an example of a snide remark that could be rewritten?
Here is a quick list of what I think needs further deliberation and scrutiny on your part.
> I’ve long liked Lisp, but hadn’t been programming in Lisp for a long time because I hadn’t found any features in Lisp compelling enough to overcome the advantage other languages had in having large libraries available and such like.
[You came to the wrong Lisp then, as Arc has no libraries compared to all the mainstream lisp dialects]
> I found the [Scheme] rationale of hygienic macros -- avoiding all possibility of conflicting identifiers in macro expansions -- to be compelling. After all, who would want to write a macro that breaks just because the user happens to be using a variable of the same name as one used in the macro?
[Actually, scheme's hygienic macros shift the burden of capture-avoidance to the compiler. It's CL that has to use gensym.]
>I now understand that all these language features that people work hard to create solve some problem, and it can be a useful and important problem to solve, and yet it can be a problem I don’t care about, a problem I don’t have. That doesn’t necessarily make it a bad language feature, but it can make it one that I don’t want to pay the cost of using.
[This is fraught with reasoning errors. If you want a language that only has the features you want, create your own DSL. And even after that, you might apply that a la carte reasoning to the runtime and question why you might need a certain garbage collector, if any at all, or why you might need your runtime to have bindings for the OS system calls when you don't use them. Slippery slope. Richness is usually a good thing, not bad.]
>Yet in Java, it’s hard to get anything shorter than ten or twenty lines of code. See some code duplication, and by the time you’ve written the classes and interfaces and methods you need to write to remove the duplication, the code is just as long as it was before.
[A personal opinion that will send java programmers in uproar. not that java doesn't suck ;-]
>Languages written by hackers for hackers like Perl and Ruby and Python are lots better (and, I find for myself, a lot more productive for me for the kinds of projects I work on), and I can write shorter programs in them, yet they don’t help me go further
[and now the Perl, Python and Ruby programmers feel trolled and will jump at your neck. You requirements are of the DWIM kind.]
>they don’t help me make my program shorter.
[Yes, and an Aston Martin can't tow a boat. Right tool for the job]
>Now I see some pattern in my code, if I’m doing more typing than I want to be doing, BAM I write a macro and use it. Just like that
[and the Lisp programmer collective is surprised by your surprise.]
[You came to the wrong Lisp then, as Arc has no libraries compared to all the mainstream lisp dialects]
I'm not saying that Arc has better libraries than other Lisps, I'm saying that for me the advantages of Arc outweigh for me the disadvantages of not having the libraries that other languages do.
[Actually, scheme's hygienic macros shift the burden of capture-avoidance to the compiler. It's CL that has to use gensym.]
Yes, what I was saying was that when I learned about hygienic macros I thought they were a good idea for that reason.
Richness is usually a good thing
If I can choose to use a feature or not (as it typical with libraries: I can choose to use a library or not) then the cost to me of a feature that I don't want is zero; if I have to use a feature or it is harder to avoid then it matters to me whether the cost to me of that feature outweighs the benefit to me.
A personal opinion
Not at all. While it wasn't the purpose of my thank-you to do such a demonstration, it's easy to compare the relative length of code needed to do a particular implementation in different languages.
feel trolled
Yes, perhaps it is common for people who like X to be mad if I say that I prefer Y.
DWIM kind
I'm not following you.
and the Lisp programmer collective is surprised by your surprise
Perhaps they have had better luck than I have at writing hygienic macros.
His intent has been "largely positive", but what do you expect people to say after his "Perl, Python, Ruby, Java, Scheme and Common Lisp suck because they didn't allow me to write compact code" remarks?
Here is what will happen: Perl and Ruby programmers will ask you to show code and they will rewrite it in one-liners. Java and Python programmers will stuff code into packages/modules and write a tiny 5-line "client" for it. And Scheme and Common Lisp programmers will take absolute pleasure in skinning your argument about macrology and hygiene, the Common Lispers keeping a bucket of salt nearby for the rubbing.
"Yet in Java, it’s hard to get anything shorter than ten or twenty lines of code."
I am a Java programmer, I find much to like about Java, and I agree totally with this statement. I tried writing some functionality in Clojure, then re-writing in Java. It was a factor of 2 or 3 increase in lines of code.
He does say that he has long liked Lisp, and that he generally liked Scheme, so he already had his feet wet. And although the documentation is, well, scarce, he did read through the source code file arc.arc.
Read what he wrote again and you will be surprised by the logic:
"I’ve long liked Lisp, but hadn’t been programming in Lisp for a long time because I hadn’t found any features in Lisp compelling enough to overcome the advantage other languages had in having large libraries available and such like."
I mean, come ON. If Lisp wasn't compelling enough because of the perceived lack of libraries, then Arc is the last way to come back to Lisp as it has NIL.
It's completely acceptable to have a preferred Lisp, but it's unacceptable to do so for irrational reasons ;-) Every time you need a new library/feature in Arc you're most likely to call out to the MzScheme runtime and FFI. It's highly hypocritical of you to dismiss a language for lack of libraries, then jump on another language it hosts.
For me it's a cost/benefit analysis. Programming in MzScheme or Arc has a cost to me in that the libraries I typically use for my projects are mostly available in other languages, so they are harder to get to. I may decide to program in a language despite such a cost if it has other benefits for me that I find outweigh the cost. While I liked MzScheme, I didn't find its benefit to me great enough to outweigh the inconvenience of the library issue. The scale tipped in the other direction for me with Arc, where I'm happy enough with the benefits I find in Arc that they outweigh the other disadvantages for me.
Your article doesn't list any of the library features that tipped the scale in favor of Arc. It's unfair to the MzScheme developers and users to dismiss their platform without telling them what's lacking. Please revise the article and flesh it further, if you have the time, because as it stands, it doesn't stand well under scrutiny.
Public technical rationales aught to be a little more than "just cuz" and personal preference. If you wanted a discussion that is ..
Your article doesn't list any of the library features that tipped the scale in favor of Arc.
The features of Arc are well described in the Arc tutorial, http://ycombinator.com/arc/tut.txt would you recommend that I include a link to that?
it doesn't stand well under scrutiny ... public technical rationales aught to be a little more than ...
Well, I had posted it to the Arc forum, where people are already familiar with Arc. I expect that your criticism is valid if it had been intended to be a technical article.
I'm sorry but this seems an aggressive comment to me. CatDancer did mention the preference of simple macros over MzScheme's more complicated hygienic macros. Beyond that, Arc is different from Scheme and everyone knows it. The differences are documented and they are the reason why Arc is not Scheme. You do not have to challenge the piece just because you don't particularly like the changes Arc made in comparison to Scheme.
If Lisp wasn't compelling enough because of the perceived lack of libraries, then Arc is the last way to come back to Lisp as it has NIL.
I think you've misunderstood this sentence, which explains why you don't understand the logic.
He is dismissing Lisp because [library pain] is not alleviated by [compelling language features]. For him, in Arc, [compelling language features] do alleviate [library pain], so he is happy to use Arc. This logic is fine even if the [library pain] in Arc is actually worse than the [library pain] he experienced in Lisp, because [compelling language features] differs too.
So, no hypocrisy. You could still argue that he should have experienced less library pain in Lisp than he did experience, or that Lisp might have compelling features that he overlooked, or even that library pain cannot in principle be alleviated by language features, though I won't comment there.
I don't know about 'hacking nirvana', but I found the Arc tutorial coupled with help from an appropriate newsgroup to grok macros (#arc? I don't remember now) was enough to write a ray tracer (LightMakesRight in the anarki distribution). Arc was my first exposure to a lisp dialect.
I have a lot of experience with Arc, Scheme, and Common Lisp, and I find Arc significantly more pleasant to program in. When I have to shift into Scheme (some parts of Arc are written in Scheme), it feels like there is a lot of extra drag.
It was noticing this difference that made me decide Arc was ready to release. If it (a) felt significantly better to me, who (b) am experienced with existing Lisp dialects, and (c) writing a type of application (HN) that's not wildly specialized, then it had the "quantum of utility" I usually use as the test for releasing a version 1.
Any chance of releasing a book or spec alongside Arc at some point? Or are you encouraging people to just play around with it for now? I'm just wondering whether the idea here is to encourage exploration through scarcity, or that there's insufficient resources to produce the documentation, or if it's just too early on at this stage.
Paul, you're fit to prefer Arc over the rest because: 1) you're an experienced Lisper, and 2) it's your own creation, you know it inside and out and the language spec, manual, tutorial and research papers are somewhere between your ears (the most convenient type of bookshelf ;-)
However, it's IMO, unacceptable for a fairly new Lisp programmer to cast aside 50 years of Lisp research, development and deployment while brushing aside the industry's backbone languages, and herald Arc as the best thing ever when it isn't even finished.
I can accept personal choice and opinion, if kept to one's self and made in private. Even if someone sings Arc, or Clojure or Common Lisp or Scheme, praises, I am fine with it. But to say "Languages X, Y, Z suck because they don't allow me to write succinct programs", well, you're just trolling for X, Y and Z programmers to come out and straighten you out. The sort of macrology he is after (code-compacting type, not language extension) is pretty basic and could be done with the C pre-processor and M4. You can even write compact assembly programs with GASP, HLA, and NASM macros; it's not rocket science.
I reserve my judgment for Arc until it's done. There is no manual yet to hold the compiler accountable, and to save me from guess-work. However, what I can't ignore is blind fanboyism; if everybody here sang Arc praises, when it is just arc-tutorial.txt and arc.arc, and there were no "dissenting" honest voices, well, the place just wouldn't be the same.
I think you're being unnecessarily argumentative in this discussion. The author likes Arc, feel is makes him more productive, and he wanted to share that with the community.
He did not say other languages suck. What he said was he prefers Arc over those languages, and explained why.
The guy expressed an opinion... this isn't a Draft Proposal For The Official Unanimous Worldwide Opinion On The Best Programming Language Ever And For All Time.
I can summarize the whole thing as:
"I really like working with arc, and it makes me happy enough to blog about it."
So yeah, while I didn't find it particularly worth upvoting, I'm not sure why you are trying to convince this guy he is somehow _wrong_ about being happy, and telling him he shouldn't dare write about it. You can no more do that than convince someone who claims their favorite color is red that it is in fact blue.
I will be the first to admit that this has been an unnecessary verbal body-slam on my part and I do owe the poster an apology, but:
> The guy expressed an opinion...
In technical circles even opinions have to be defensible. You can't just accept "X sucks and Y rocks" as an opinion and let it pass. The whole place will become abuzz with irrational thought, and overrun with "we are all right" group-think liberal-arts type BS.
> You can't just accept "X sucks and Y rocks" as an opinion and let it pass.
Why not? The only reason that 'the whole place will become abuzz with irrational thought, and overrun with "we are all right" group-think liberal-arts type BS' is because people can't just accept things like this as somebody's opinion... they have to create a language war out of simple blog post about liking something.
(To everyone else, I'll quit now I promise, obviously I'm not helping the matter!)
I see CatDancer's touched a bit of a raw nerve. You've been compelled to respond with about a dozen (didn't count them) posts all quite attacking of both CatDancer and Arc.
What the hell is your problem with a) someone creating their own Lisp implementation, and b) someone else enjoying using it?
you wouldn't understand it, and it's not a personal problem of mine: just consider my behavior in this discussion a throw-back to a long-dead form of communication from when programmers where held accountable to their words :-)
I should have been nice, we're all equally-right in some way. hugs.
>> "The second half of the book is how to apply the principles to Java code. Which is horrendous! Not the book. If you should be so unfortunate to be stuck trying to write good code in Java, the book is a great, lucid description of how to do so. Yet in Java, it’s hard to get anything shorter than ten or twenty lines of code. See some code duplication, and by the time you’ve written the classes and interfaces and methods you need to write to remove the duplication, the code is just as long as it was before."
Yeah you're doing something wrong there. I know it's fashionable to bash Java and say how lame and verbose it is, but it does wear a bit thin. I actually enjoy writing Java. Perhaps due to me having grown up programming assembly - I don't mind verbosity. Java is an extremely solid fast language.
And a big part of the value proposition of Clojure is that it gives you Lisp's ability to eliminate almost all duplication with macros alongside most of the benefits of Java.
Right - Java as a word is heavily overloaded ;). For me, that's the bit that matters - the JVM. Java is just something I use because it's easier than writing bytecode :/
Java as a language, the syntax, is for me sane. It gets a lot of things right, and some things wrong, but on balance, it's sane enough to use. Just like English as a language is sane enough to use, whilst having lots of stupid inconsistencies in spelling/pronunciation etc.
And it's likely to be the fastest/most optimized of any JVM target language.
np :) and apologies for taking the comments off topic.
Do linguists agonize over language efficiency as much as some programmers do? For example, writing English is maybe far more letters/strokes than Japanese (idk), but we don't all start learning Japanese and bash English.
If the Sapir-Whorf hypothesis is true, then they should if they don't already. If the Sapir-Whorf hypothesis is true for programming languages, then our agonizing is perfectly reasonable.
I often get jumped on for my views, but S-W is true, and untrue. Untrue in that absence of a direct expression implies absence of concept; true in that absence of a direct expression impedes such an expression, such that it is less likely to get used, and hence, less efficient, and less likely to be conceived.
And I certainly agree that S-W applies for human and computer languages alike.
From what I have seen (not much, but probably more than most programmers), they don't. I suspect the reason is because everybody has at least one native spoken language, but programming languages are not native in the same way; they are acquired, and so are not perceived as a necessity.
And as such, for most people it is much harder to have a "favorite spoken language" that they can use as a platform to bash other languages. The way that programmers, text editor users, and coffee drinkers bash each other is bashing someone for an acquired taste. Bashing other languages is almost like bashing someone's physical appearance, and the immaturity becomes very obvious a lot quicker.
Also, regarding compactness of the spoken language, on average, English is longer than, say, Chinese. But brevity is NOT the goal. The tradeoff is redundancy, which English has much more of (e.g. parallelism and verb/plurality agreement). Old Chinese, with all its homophones, has plenty of texts written so compactly that if you don't read the text you will not understand it (at least for modern pupils).
I'm just imagining an anti-English troll, extolling the virtues of Japanese. Actually, Outliers makes the point that Korean has logical words for numbers, which makes maths a bit more accessible. I have to agree, I mean "eleven, twelve". I was irritated with the nonsensical naming of numbers as a child, especially when people said that mathematics was logical.
Appreciated. BTW I thought that was the only part of your post that was putting down another language. But it didn't seem deliberate, just a casual and unconscious going-with-the-flow put-down of java. As axon said, many people do it. I was struck by your preface + the following statement:
My statements that I prefer Arc for the kind of programming I do was not intended to be a put-down of other languages.
If you should be so unfortunate to be stuck trying to write good code in Java, ...
One solution is to remove the connotations, eg:
If you are writing code in Java, ...
Note: this version is more concise :-)
But please keep the factual points, such that Java's verbosity can mean that removing duplication doesn't make it shorter. I thought that was an insightful and pithy expression of the idea. But I also note that that's not the only benefit of removing duplication.
I haven't used arc in a while, but if I remember correctly it has a Common Lisp style defmacro that anyone can figure out:
(defmacro foo (args go here) `(code goes ,here))
On the other hand, Scheme has this monstrosity for defining macros that I can't describe because I never managed to keep straight in my head how it was supposed to work.
Hygiene is a red-herring. It's easy enough to achieve with a defmacro-style macro system - just use gensyms.
Ah, this has been repeated many times, but I can't help to click "reply"...
> It's easy enough to achieve with a defmacro-style macro system - just use gensyms.
Gensym cannot prevent global identifiers from being shadowed in macro use environment.
(define-macro (my-if a b c) `(if ,a ,b ,c))
(define (boo)
(let ((if list)) ; if is shadowed
(my-if 'x 'y 'z))) ; evaluates to (x y z), not y
In CL you don't need to worry much, since (a) it is Lisp-2 so variables won't shadow operators, and (b) using packages greatly reduces accidental shadowing. In Scheme the chance is much higher; you probably don't bind 'if' locally, but you can't check every user-defined global procedures the macro inserts into the output.
It is unfortunate for abstractbill to use this incorrect explanation, because hygienic macros are still a red herring - a decent module system and a bit of restraint gets you there.
(define-macro (my-if a b c) `(std.if ,a ,b ,c))
(define (boo)
(let ((if list)) ; if is shadowed
(my-if 'x 'y 'z))) ; gives (std.if x y z)
(define (boo2)
(let ((std.if list)) ; error! cannot shadow module vars.
(my-if 'x 'y 'z)))
I've posted this around a few times asking why this isn't already used in Lisps and never seem to get a convincing argument back. The only conclusion I can draw from this is that CL-ers are ashamed of the fact that multiple namespaces aren't actually necessary and Schemers are rightly ashamed of hygienic macros. Please convincingly prove me wrong!
(ps. This ofcourse would not be limited to the languages std module but would also apply to any exported module symbols.)
(mac awhen (expr . body)
`(std.let it ,expr (std.if it (std.do ,@body))))
This assumes that awhen is being defined and exported from the std module too, so that it can also be used in expansions. Slightly more verbose, but that's optional hygiene if you want it.
That's exactly what I meant in "In CL ... (b) using packages greatly reduces accidental shadowing." So I'm not sure what you are asking to be convinced. CL certainly does not prevent you from messing with package-private symbols, but usually a convention is enough to avoid disaster. I mean, I won't write (let ((your-package::private-symbol list)) (your-macro ...) ...) unless there's absolute need to do, such as emergency workaround of a bug in a third-party library.
On the other hand, one of the reasons Scheme avoids explicit module prefix is, I think, that it breaks abstraction. Suppose my-module defines my-macro and my-function. My-macro expands into a call to my-function. But I don't really want my module users to call my-function directly, so I only export my-macro and keep my-function private.
In your proposal, my-macro will expands into prefixed-symbol like this:
But... if we allow module-private function to be called with module prefix, we can't prevent ordinary code from calling my-function as well.
This is a language design choice. Some may not mind others to mess with module-private bindings; CLers certainly would not. But if you want to hide module-private symbols, and want to allow macros expand into them, how will you do that? One idea is to mark the output of macro expansion so that the compiler can distinguish whether the reference is from legitimate macro expansion or not... well, that's what hygienic macro is doing.
Thanks for the explanation. I hadn't thought of the my-macro problem because I tend to think of macros exactly as a clever copy paste mechanism, so would expect it to always use public variables, but I guess some people might have a problem with that in some cases.
In the first point I was asking to be convinced that you can't have hygiene solely through the use of a package / module system and without needing multiple namespaces.
Although your workaround argument (for allowing variables to be shadowed) is convincing, in that case the package system wouldn't really be providing any safety at all and only the convention would be helping. I guess we can sum it up as CLers really don't see a problem at all, and Schemers require macros to be completely abstract.
> Hygiene is a red-herring. It's easy enough to achieve with a defmacro-style macro system - just use gensyms.
Hygiene can also be baked-in to a nice lispy defmacro-style system; there's no reason that hygienic macros have to be scheme-style. Variable capture can be permitted with a (capture x) special form. (I like to use ^x as a shorthand for that).
At that point, macros start behaving scope-wise similarly to lexical functions, and captured variables behave kind of like dynamic variables in a function.
It suddenly occurs to me that this form of hygiene could be implemented in the expansion of let, lambda and defun forms, at which point (someone yell at me please if I'm wrong), plain-ol' defmacros would be hygienic, unless there's a type of variable capture that I'm forgetting about right now...
Hmm, I don't know if I'm going to be able to explain it very well without an example. My general experience was that I'd start with syntax-rules, which would turn out to not be powerful enough for the particular macro I wanted to write, so I'd turn to the more complex but more powerful syntax-case. Which then might turn out to not be powerful enough either, or else it might be powerful enough but I'm struggling to understand how to implement my macro using it, plowing my way through pages of documentation and reading examples.
Then there's a macro in Arc, for example,
(mac awhen (expr . body)
`(let it ,expr (if it (do ,@body))))
which, on the one hand, yes it is true it's not hygienic. But, on the other hand, I can bang out in seconds. So now I'm moving ahead writing my program instead of struggling to write a macro. And the lack of hygiene has never been an actual problem for me in practice.
I imagine that a Scheme programmer who was experienced in writing hygienic macros and so could write them quickly, and who was more bothered by the lack of hygiene than I am, could well reach a different cost/benefit decision than I do and decide they'd rather go with hygienic macros.
I understand what the guy is saying, but this is equivalent to art students discussing the technicality and construction of canvases. Even if you select the best canvas in the world, it does not make your painting better.
I think that too much of writing about programming is focused on the details of the programming act, and too little on the result of the programming.
Reading about finding and learning a programming language is like reading about an artist choosing a brush. Reading about the technical details of someone scaling a Django app on a single server to serve 1000 concurrent requests per second is like seeing the finished artwork.
> Reading about finding and learning a programming language is like reading about an artist choosing a brush.
Wands are a much better metaphor to make. Or perhaps, what if you had a magic brush that could at any time be extremely fine or large or anywhere in between depending on how you felt? That would be some brush, especially when you compare it to the many medium sized brushes which aren't so fine when you need them to be.
Okay, so maybe the metaphor is still stretching a bit, but these are no ordinary brushes; they are worth writing about.
It's funny, just today I finished up some work merging a hygienic expander into the core of Guile. I've been writing defmacros for a few years now, and while things are not all lovely in the world of syntax-rules or syntax-case, it is so much more lovely to write a macro that you know catches the important cases, and that will compose well with other code, and /especially/ code from other modules.
From what this guy is saying, the power of Lisp style Macros is akin to what I can do with the syntactic Rewrite Tool and Blocks (Lambdas) in Smalltalk. I've always been led to believe that Lisp Macros are something even more powerful. (I haven't had the time to really delve into them.)
That was comparing Smalltalk to C#, but it is a similar idea. Smalltalk and Lisp belong to that elite fraternity of programming languages where the ability to easily extend the syntax is built into the language. The mechanisms are different, but the expressive power is similar.
Well, I've heard that Blocks get you most of the way to Macros, but not quite. I'm wondering if the "not quite" part is something beyond what's afforded by the Rewrite Tool.
The Rewrite Tool is not dynamic. Perhaps this is the key? But I could imagine a Smalltalk function that takes a method, applies a Rewrite Tool syntactic transformation to its source, compiles the method as a Block, then calls this Block with the arguments that would've been send to the original method.
"Order of Blocks reversed"
``@rcv
ifTrue: `Block2
ifFalse: `Block1
Just be careful to cache the result of the compilation by method name and Class, and this wouldn't be too much slower than a conventional #perform:
This particular example would let you dynamically call any method, but with all of the if-then-else equivalents with the then and else clauses swapped.
"I could imagine a Smalltalk function that takes a method, applies a Rewrite Tool syntactic transformation to its source, compiles the method as a Block, then calls this Block with the arguments that would've been send to the original method."
I don't understand what you said, but in non-hygenic Lisp macros (the kind you get in CL, Arc, and Clojure) a macro is simply a function that expands into code before being compiled or evaluated. You can almost think of it as an extremely clever query-replace functionality in your text editor that always happens before you compile or run your program.
This is why so many Lispers claim that Lisp is at the top of the heap when it comes to language power. If you can arbitrarily expand your code into some other form at compile time, you can pretty much do anything any other language can do. A canonical example of this is that the object system added to Common Lisp is just a bunch of macros. Name your programming paradigm, and there's a very good chance someone has implemented it in Lisp, with macros.
What you describe sounds more complex than Lisp macros, which brings us to the necessity of all the parentheses. The idea of macros only works well when the language is built up out of a data-structure supported by the language. In Lisp, of course, programs are lists and macros let you do anything to your program that you can do to a list. When people try to add "macros" to languages where the code is not a data structure, there usually needs to be some intermediate representation of the parse tree, and then things tend to start getting ugly as the relation to what the source code would look like is quickly lost.
Having said all that, I'm curious if anyone that knows both Smalltalk and Lisp can name concrete examples of programming in Smalltalk and finding something you were unable to do without macros.
The Rewrite Tool meta-syntax allows you to do a few nifty things. It is also "an extremely clever query-replace functionality" and in the scheme I proposed, you could "decorate" a function, such that a transformed version is called at runtime.
The Rewrite Tool also gives you access to the Smalltalk Parse Tree, though this isn't quite as pretty as Lisp's. However, you can very easily apply a Visitor to your AST. Perhaps even better, there's also a { : ... } syntax for executing arbitrary Smalltalk code to do your transform.
which brings us to the necessity of all the parentheses
Yes, Lisp is it's own AST! That's amazingly brilliant! Makes software toolsmithing equivalent to just plain programming.
OK, I have a HN etiquette question. I don't post much on HN, and I wasn't the one who posted the link to my thank-you here on HN. I've had a HN account for a while, but I'm still pretty much a newbie to HN.
I have a desire to not just stand here and take it on the chin as mahmud publicly proclaims that I'm pandering, a Lisp newbie, irrational, a blind fanboy, hypocritical, dismissive, and unfair. On the other hand I don't want to waste everyone's time with a tedious flamewar.
He was surprisingly nasty about it, but I wouldn't bother fighting back. A lot of the people who voted up his comment didn't do it because they agreed with him so much as because they're worried you'll start a trend of people posting stuff trying to suck up to us. Which is actually a reasonable thing to worry about, considering that a significant number of HN users think they might one day apply to YC. So you were basically just unlucky.
It happens to us all I guess, sometimes it's too much coffee too early in the day or someone hits a soft spot.
It's easy to misinterpret what people say on the net.
FWIW I too had a very positive superficial read of Arc when it was first published and I'm neither a fanboy nor a Lisp newbie. Scheme is one of my true loves.
First I figure out how to do what I need to do, then I translate that into the constructs of the language, then I try it and it doesn’t work, and then I debug or figure it out :-).
It's a wonderful thing to have a language that matches how you think, an extension of your mind analogous to how physical tools can be an extension of your body, with a heft, weight and balance that causes them to leap to hand, and guide you at times.
But my particular tasks at the moment involve so much more work at the drawing board (the figuring stage) that having better tools doesn't actually help me - much as I would love them to.
It's very true that the tools you need depend on your personal way of thinking, the task, the stage of the task, and how many times you've done it.
Hi. My experience lies in Ruby and in JavaScript (these days) so I can see these worlds however close, differently. In Ruby I have all kinds of gimmicks that help with cutting the chase when that's what I want, but I also can make it a little harder when I am interested in something else, in experimenting with some other approach.
What I find a little disgusting is when my "meta-programming" in Ruby starts doing things like when an exception occurs, it points to a line number slightly different than where it really is, or when the backtrace becomes all the more "hacked" with harder to understand things thrown in there. It serves to remind us that there are tradeoffs involved. Also there is the issue creating shareable modules and when you start hacking it away it can make it all the more harder to share. But it's more than fun, it does work, it's just that one can have mixed feelings because for ever cool finding or development, there can exist a little nuisance as well or something that you cannot quite do anymore because you deviated from mainstream...
86 comments
[ 3.1 ms ] story [ 103 ms ] threadI can not believe for once that this guy obtained hacking nirvana through Arc. Not once! Why? because the Arc documentation is so meager as to be non-existent. You will need serious feet-wetting with more established Lisp dialects before you can "get there" with Arc.
All the epiphanies he lists are easily experienced with most modern Lisp dialects; either he "got it" before and just needed to extend the credit to Arc, for some reason .. or he isn't there yet, at least not to its full potential.
It had to be said, as much as I hate to be rude and blunt like this :-|
I hope any pre-conceived notion of Lisper hostility isn't getting in the way you read my prose above.
Demand for rigor in thought shouldn't be mistaken for hostility.
Ok. I will accept that.
Here is my take on your personal essay, it's also a matter of personal opinion made after sufficient technical evaluation of the matter: I think you need to be a little more critical of your own reasoning, specially when writing for programmers. The spirit of your essay has been largely positive, IMO, but go easy on the snide remarks and the dismissive attitude of other technologies you have very little experience in (yeah, it shows.) Otherwise a much less sympathetic reader than I, and a far more competent one, will come aboard and tear your argument to pieces.
A better way to frame your enthusiastic writing would be "I can do X in Arc, I really wish ${LISP_DIALECT} would let me do the same". This is both curious and also accepting of one's own self-limitations. Someone will come aboard and show you what you have been missing, and you might as well get pointers to the literature that further your understanding of Arc or whatever other technology.
Regards.
(I appreciate your suggestion to say that I wish another Lisp dialect was more like Arc, but I'm not sure how I'd choose one since I don't really have a favorite aside from Arc).
Here is a quick list of what I think needs further deliberation and scrutiny on your part.
> I’ve long liked Lisp, but hadn’t been programming in Lisp for a long time because I hadn’t found any features in Lisp compelling enough to overcome the advantage other languages had in having large libraries available and such like.
[You came to the wrong Lisp then, as Arc has no libraries compared to all the mainstream lisp dialects]
> I found the [Scheme] rationale of hygienic macros -- avoiding all possibility of conflicting identifiers in macro expansions -- to be compelling. After all, who would want to write a macro that breaks just because the user happens to be using a variable of the same name as one used in the macro?
[Actually, scheme's hygienic macros shift the burden of capture-avoidance to the compiler. It's CL that has to use gensym.]
>I now understand that all these language features that people work hard to create solve some problem, and it can be a useful and important problem to solve, and yet it can be a problem I don’t care about, a problem I don’t have. That doesn’t necessarily make it a bad language feature, but it can make it one that I don’t want to pay the cost of using.
[This is fraught with reasoning errors. If you want a language that only has the features you want, create your own DSL. And even after that, you might apply that a la carte reasoning to the runtime and question why you might need a certain garbage collector, if any at all, or why you might need your runtime to have bindings for the OS system calls when you don't use them. Slippery slope. Richness is usually a good thing, not bad.]
>Yet in Java, it’s hard to get anything shorter than ten or twenty lines of code. See some code duplication, and by the time you’ve written the classes and interfaces and methods you need to write to remove the duplication, the code is just as long as it was before.
[A personal opinion that will send java programmers in uproar. not that java doesn't suck ;-]
>Languages written by hackers for hackers like Perl and Ruby and Python are lots better (and, I find for myself, a lot more productive for me for the kinds of projects I work on), and I can write shorter programs in them, yet they don’t help me go further
[and now the Perl, Python and Ruby programmers feel trolled and will jump at your neck. You requirements are of the DWIM kind.]
>they don’t help me make my program shorter.
[Yes, and an Aston Martin can't tow a boat. Right tool for the job]
>Now I see some pattern in my code, if I’m doing more typing than I want to be doing, BAM I write a macro and use it. Just like that
[and the Lisp programmer collective is surprised by your surprise.]
and so on and so forth.
I'm not saying that Arc has better libraries than other Lisps, I'm saying that for me the advantages of Arc outweigh for me the disadvantages of not having the libraries that other languages do.
[Actually, scheme's hygienic macros shift the burden of capture-avoidance to the compiler. It's CL that has to use gensym.]
Yes, what I was saying was that when I learned about hygienic macros I thought they were a good idea for that reason.
Richness is usually a good thing
If I can choose to use a feature or not (as it typical with libraries: I can choose to use a library or not) then the cost to me of a feature that I don't want is zero; if I have to use a feature or it is harder to avoid then it matters to me whether the cost to me of that feature outweighs the benefit to me.
A personal opinion
Not at all. While it wasn't the purpose of my thank-you to do such a demonstration, it's easy to compare the relative length of code needed to do a particular implementation in different languages.
feel trolled
Yes, perhaps it is common for people who like X to be mad if I say that I prefer Y.
DWIM kind
I'm not following you.
and the Lisp programmer collective is surprised by your surprise
Perhaps they have had better luck than I have at writing hygienic macros.
Here is what will happen: Perl and Ruby programmers will ask you to show code and they will rewrite it in one-liners. Java and Python programmers will stuff code into packages/modules and write a tiny 5-line "client" for it. And Scheme and Common Lisp programmers will take absolute pleasure in skinning your argument about macrology and hygiene, the Common Lispers keeping a bucket of salt nearby for the rubbing.
I am a Java programmer, I find much to like about Java, and I agree totally with this statement. I tried writing some functionality in Clojure, then re-writing in Java. It was a factor of 2 or 3 increase in lines of code.
"I’ve long liked Lisp, but hadn’t been programming in Lisp for a long time because I hadn’t found any features in Lisp compelling enough to overcome the advantage other languages had in having large libraries available and such like."
I mean, come ON. If Lisp wasn't compelling enough because of the perceived lack of libraries, then Arc is the last way to come back to Lisp as it has NIL.
It's completely acceptable to have a preferred Lisp, but it's unacceptable to do so for irrational reasons ;-) Every time you need a new library/feature in Arc you're most likely to call out to the MzScheme runtime and FFI. It's highly hypocritical of you to dismiss a language for lack of libraries, then jump on another language it hosts.
Public technical rationales aught to be a little more than "just cuz" and personal preference. If you wanted a discussion that is ..
The features of Arc are well described in the Arc tutorial, http://ycombinator.com/arc/tut.txt would you recommend that I include a link to that?
it doesn't stand well under scrutiny ... public technical rationales aught to be a little more than ...
Well, I had posted it to the Arc forum, where people are already familiar with Arc. I expect that your criticism is valid if it had been intended to be a technical article.
I think you've misunderstood this sentence, which explains why you don't understand the logic.
He is dismissing Lisp because [library pain] is not alleviated by [compelling language features]. For him, in Arc, [compelling language features] do alleviate [library pain], so he is happy to use Arc. This logic is fine even if the [library pain] in Arc is actually worse than the [library pain] he experienced in Lisp, because [compelling language features] differs too.
So, no hypocrisy. You could still argue that he should have experienced less library pain in Lisp than he did experience, or that Lisp might have compelling features that he overlooked, or even that library pain cannot in principle be alleviated by language features, though I won't comment there.
It was noticing this difference that made me decide Arc was ready to release. If it (a) felt significantly better to me, who (b) am experienced with existing Lisp dialects, and (c) writing a type of application (HN) that's not wildly specialized, then it had the "quantum of utility" I usually use as the test for releasing a version 1.
Any chance of releasing a book or spec alongside Arc at some point? Or are you encouraging people to just play around with it for now? I'm just wondering whether the idea here is to encourage exploration through scarcity, or that there's insufficient resources to produce the documentation, or if it's just too early on at this stage.
However, it's IMO, unacceptable for a fairly new Lisp programmer to cast aside 50 years of Lisp research, development and deployment while brushing aside the industry's backbone languages, and herald Arc as the best thing ever when it isn't even finished.
I can accept personal choice and opinion, if kept to one's self and made in private. Even if someone sings Arc, or Clojure or Common Lisp or Scheme, praises, I am fine with it. But to say "Languages X, Y, Z suck because they don't allow me to write succinct programs", well, you're just trolling for X, Y and Z programmers to come out and straighten you out. The sort of macrology he is after (code-compacting type, not language extension) is pretty basic and could be done with the C pre-processor and M4. You can even write compact assembly programs with GASP, HLA, and NASM macros; it's not rocket science.
I reserve my judgment for Arc until it's done. There is no manual yet to hold the compiler accountable, and to save me from guess-work. However, what I can't ignore is blind fanboyism; if everybody here sang Arc praises, when it is just arc-tutorial.txt and arc.arc, and there were no "dissenting" honest voices, well, the place just wouldn't be the same.
He did not say other languages suck. What he said was he prefers Arc over those languages, and explained why.
Lesson learned. Modded you up for the truth. Thank you :-)
Actually it didn't, but that hasn't stopped you before;
I can summarize the whole thing as:
"I really like working with arc, and it makes me happy enough to blog about it."
So yeah, while I didn't find it particularly worth upvoting, I'm not sure why you are trying to convince this guy he is somehow _wrong_ about being happy, and telling him he shouldn't dare write about it. You can no more do that than convince someone who claims their favorite color is red that it is in fact blue.
> The guy expressed an opinion...
In technical circles even opinions have to be defensible. You can't just accept "X sucks and Y rocks" as an opinion and let it pass. The whole place will become abuzz with irrational thought, and overrun with "we are all right" group-think liberal-arts type BS.
Why not? The only reason that 'the whole place will become abuzz with irrational thought, and overrun with "we are all right" group-think liberal-arts type BS' is because people can't just accept things like this as somebody's opinion... they have to create a language war out of simple blog post about liking something.
(To everyone else, I'll quit now I promise, obviously I'm not helping the matter!)
What the hell is your problem with a) someone creating their own Lisp implementation, and b) someone else enjoying using it?
I should have been nice, we're all equally-right in some way. hugs.
Yeah you're doing something wrong there. I know it's fashionable to bash Java and say how lame and verbose it is, but it does wear a bit thin. I actually enjoy writing Java. Perhaps due to me having grown up programming assembly - I don't mind verbosity. Java is an extremely solid fast language.
You mean "the JVM is an extremely solid fast runtime" - languages by themselves aren't fast or slow ;-)
Java as a language, the syntax, is for me sane. It gets a lot of things right, and some things wrong, but on balance, it's sane enough to use. Just like English as a language is sane enough to use, whilst having lots of stupid inconsistencies in spelling/pronunciation etc.
And it's likely to be the fastest/most optimized of any JVM target language.
Do linguists agonize over language efficiency as much as some programmers do? For example, writing English is maybe far more letters/strokes than Japanese (idk), but we don't all start learning Japanese and bash English.
http://en.wikipedia.org/wiki/Sapir–Whorf_hypothesis
And I certainly agree that S-W applies for human and computer languages alike.
And as such, for most people it is much harder to have a "favorite spoken language" that they can use as a platform to bash other languages. The way that programmers, text editor users, and coffee drinkers bash each other is bashing someone for an acquired taste. Bashing other languages is almost like bashing someone's physical appearance, and the immaturity becomes very obvious a lot quicker.
Also, regarding compactness of the spoken language, on average, English is longer than, say, Chinese. But brevity is NOT the goal. The tradeoff is redundancy, which English has much more of (e.g. parallelism and verb/plurality agreement). Old Chinese, with all its homophones, has plenty of texts written so compactly that if you don't read the text you will not understand it (at least for modern pupils).
My statements that I prefer Arc for the kind of programming I do was not intended to be a put-down of other languages.
If you should be so unfortunate to be stuck trying to write good code in Java, ...
One solution is to remove the connotations, eg:
If you are writing code in Java, ...
Note: this version is more concise :-)
But please keep the factual points, such that Java's verbosity can mean that removing duplication doesn't make it shorter. I thought that was an insightful and pithy expression of the idea. But I also note that that's not the only benefit of removing duplication.
Thanks Anarki and arcfn.com!*
They were both tremendously useful/helpful when I last used arc.
*All creators and contributors.
Hygiene is a red-herring. It's easy enough to achieve with a defmacro-style macro system - just use gensyms.
> It's easy enough to achieve with a defmacro-style macro system - just use gensyms.
Gensym cannot prevent global identifiers from being shadowed in macro use environment.
In CL you don't need to worry much, since (a) it is Lisp-2 so variables won't shadow operators, and (b) using packages greatly reduces accidental shadowing. In Scheme the chance is much higher; you probably don't bind 'if' locally, but you can't check every user-defined global procedures the macro inserts into the output.(ps. This ofcourse would not be limited to the languages std module but would also apply to any exported module symbols.)
On the other hand, one of the reasons Scheme avoids explicit module prefix is, I think, that it breaks abstraction. Suppose my-module defines my-macro and my-function. My-macro expands into a call to my-function. But I don't really want my module users to call my-function directly, so I only export my-macro and keep my-function private.
In your proposal, my-macro will expands into prefixed-symbol like this:
But... if we allow module-private function to be called with module prefix, we can't prevent ordinary code from calling my-function as well.This is a language design choice. Some may not mind others to mess with module-private bindings; CLers certainly would not. But if you want to hide module-private symbols, and want to allow macros expand into them, how will you do that? One idea is to mark the output of macro expansion so that the compiler can distinguish whether the reference is from legitimate macro expansion or not... well, that's what hygienic macro is doing.
In the first point I was asking to be convinced that you can't have hygiene solely through the use of a package / module system and without needing multiple namespaces.
Although your workaround argument (for allowing variables to be shadowed) is convincing, in that case the package system wouldn't really be providing any safety at all and only the convention would be helping. I guess we can sum it up as CLers really don't see a problem at all, and Schemers require macros to be completely abstract.
Hygiene can also be baked-in to a nice lispy defmacro-style system; there's no reason that hygienic macros have to be scheme-style. Variable capture can be permitted with a (capture x) special form. (I like to use ^x as a shorthand for that).
At that point, macros start behaving scope-wise similarly to lexical functions, and captured variables behave kind of like dynamic variables in a function.
It suddenly occurs to me that this form of hygiene could be implemented in the expansion of let, lambda and defun forms, at which point (someone yell at me please if I'm wrong), plain-ol' defmacros would be hygienic, unless there's a type of variable capture that I'm forgetting about right now...
Then there's a macro in Arc, for example,
which, on the one hand, yes it is true it's not hygienic. But, on the other hand, I can bang out in seconds. So now I'm moving ahead writing my program instead of struggling to write a macro. And the lack of hygiene has never been an actual problem for me in practice.I imagine that a Scheme programmer who was experienced in writing hygienic macros and so could write them quickly, and who was more bothered by the lack of hygiene than I am, could well reach a different cost/benefit decision than I do and decide they'd rather go with hygienic macros.
I think that too much of writing about programming is focused on the details of the programming act, and too little on the result of the programming.
Reading about finding and learning a programming language is like reading about an artist choosing a brush. Reading about the technical details of someone scaling a Django app on a single server to serve 1000 concurrent requests per second is like seeing the finished artwork.
Wands are a much better metaphor to make. Or perhaps, what if you had a magic brush that could at any time be extremely fine or large or anywhere in between depending on how you felt? That would be some brush, especially when you compare it to the many medium sized brushes which aren't so fine when you need them to be.
Okay, so maybe the metaphor is still stretching a bit, but these are no ordinary brushes; they are worth writing about.
That said, have fun with Arc ;-)
http://blog.bitquabit.com/2009/05/20/your-language-features-...
That was comparing Smalltalk to C#, but it is a similar idea. Smalltalk and Lisp belong to that elite fraternity of programming languages where the ability to easily extend the syntax is built into the language. The mechanisms are different, but the expressive power is similar.
The Rewrite Tool is not dynamic. Perhaps this is the key? But I could imagine a Smalltalk function that takes a method, applies a Rewrite Tool syntactic transformation to its source, compiles the method as a Block, then calls this Block with the arguments that would've been send to the original method.
Something like:
Where aMatchingPattern would be something like: Then aRewritePattern could be something like: Just be careful to cache the result of the compilation by method name and Class, and this wouldn't be too much slower than a conventional #perform:This particular example would let you dynamically call any method, but with all of the if-then-else equivalents with the then and else clauses swapped.
Would this constitute Smalltalk Macros?
I don't understand what you said, but in non-hygenic Lisp macros (the kind you get in CL, Arc, and Clojure) a macro is simply a function that expands into code before being compiled or evaluated. You can almost think of it as an extremely clever query-replace functionality in your text editor that always happens before you compile or run your program.
This is why so many Lispers claim that Lisp is at the top of the heap when it comes to language power. If you can arbitrarily expand your code into some other form at compile time, you can pretty much do anything any other language can do. A canonical example of this is that the object system added to Common Lisp is just a bunch of macros. Name your programming paradigm, and there's a very good chance someone has implemented it in Lisp, with macros.
What you describe sounds more complex than Lisp macros, which brings us to the necessity of all the parentheses. The idea of macros only works well when the language is built up out of a data-structure supported by the language. In Lisp, of course, programs are lists and macros let you do anything to your program that you can do to a list. When people try to add "macros" to languages where the code is not a data structure, there usually needs to be some intermediate representation of the parse tree, and then things tend to start getting ugly as the relation to what the source code would look like is quickly lost.
Having said all that, I'm curious if anyone that knows both Smalltalk and Lisp can name concrete examples of programming in Smalltalk and finding something you were unable to do without macros.
The Rewrite Tool also gives you access to the Smalltalk Parse Tree, though this isn't quite as pretty as Lisp's. However, you can very easily apply a Visitor to your AST. Perhaps even better, there's also a { : ... } syntax for executing arbitrary Smalltalk code to do your transform.
which brings us to the necessity of all the parentheses
Yes, Lisp is it's own AST! That's amazingly brilliant! Makes software toolsmithing equivalent to just plain programming.
http://arclanguage.org/item?id=9257
One of the things I want to add next is graphics libraries. Does anyone have any recommendations for what to plug into to munge images?
ImageMagick (http://www.imagemagick.org/) has worked well for us. There are bindings out there for many languages, including Common Lisp (http://common-lisp.net/project/cl-magick/); you might want to look there for inspiration.
http://arclanguage.org/item?id=4070
Are you planning to do another poll ?
http://rhodesmill.org/brandon/2009/graphicsmagick-saved-the-...
I have a desire to not just stand here and take it on the chin as mahmud publicly proclaims that I'm pandering, a Lisp newbie, irrational, a blind fanboy, hypocritical, dismissive, and unfair. On the other hand I don't want to waste everyone's time with a tedious flamewar.
Thoughts?
It's easy to misinterpret what people say on the net.
FWIW I too had a very positive superficial read of Arc when it was first published and I'm neither a fanboy nor a Lisp newbie. Scheme is one of my true loves.
It's a wonderful thing to have a language that matches how you think, an extension of your mind analogous to how physical tools can be an extension of your body, with a heft, weight and balance that causes them to leap to hand, and guide you at times.
But my particular tasks at the moment involve so much more work at the drawing board (the figuring stage) that having better tools doesn't actually help me - much as I would love them to.
It's very true that the tools you need depend on your personal way of thinking, the task, the stage of the task, and how many times you've done it.
What I find a little disgusting is when my "meta-programming" in Ruby starts doing things like when an exception occurs, it points to a line number slightly different than where it really is, or when the backtrace becomes all the more "hacked" with harder to understand things thrown in there. It serves to remind us that there are tradeoffs involved. Also there is the issue creating shareable modules and when you start hacking it away it can make it all the more harder to share. But it's more than fun, it does work, it's just that one can have mixed feelings because for ever cool finding or development, there can exist a little nuisance as well or something that you cannot quite do anymore because you deviated from mainstream...
I gotta go have lunch!
Be well.