Ask HN: Who's using Clojure, and to do what?
Hello everyone,
I've seen a lot of articles on HN recently talking about Clojure. I had a quick look at a few pages (Wikipedia, etc...), and I can't help but think that it sounds like the new cool language, pretty much like Ruby in 2007.
But to get a better idea, I'd like to hear the feedback from the community: Who is using Clojure (in real-life cases), and what are you doing with it?
Bonus points if you can tell what specific features made it a better language for the task.
70 comments
[ 3.3 ms ] story [ 130 ms ] threadOh, and, as I told "First Blood", quoting from Watchmen, "you think i am in here with you, but, really, you are in here with me."
http://www.backtype.com/opensource
http://github.com/nathanmarz/cascalog
The big reason I used Clojure was that it was easy to parallelize my program; however, running independent trials with many different parameter-sets is one of those "embarrassingly parallel" tasks.
How do you think it would hold up to larger, say intraday data?
There are two reasons why:
1. There would be more data to load into memory, and that process of loading the price history from a CSV file into memory would take a little longer simply because there is more data to deal with.
2. The data structure I'm using to index the historic price information by timestamp is the Java 6 implementation of TreeMap. The TreeMap class is an implementation of a Red-Black tree, and provides a guarantee that lookup operations (even when the key is not present in the collection) are O(log n). Since the time to find a key in the collection is O(log n), there would be a small performance hit when adding more data to the collection.
I actually began using intraday price history that I had downloaded from TradeStation, but I ran into the problem that I only had intraday price history over a 1-year period. I needed to be able to run 1-year trials over the course of a much larger period of time, so I switched to EOD data. Switching to EOD data had the nice side effect of speeding up my program (mainly due to the fact that loading all that intraday price history into memory took several seconds - up to 30 seconds or so).
Not that I had the foresight to pick Clojure because of this, but after I started working with Clojure I noticed that change became much easier than with the languages I was using before (one of which was C++).
When my thesis advisor would tell me, "you should go do X" (fill in the blank), I'd do what he said, and then a week later he'd have changed his mind and he would say "now go do Y", instead of X. Those changes would kill my enthusiasm and productivity when I was using C++, but once I began using Clojure those changes no longer felt like a bombshell. Change became relatively easy.
Now, I'm not saying that change is trivial, but compared to C++ (in my experience), changes are much easier to deal with using Clojure.
I'd love to read your thesis and code, if they're available.
edit: Cleaned up my bad grammar.
http://cran.r-project.org/web/packages/multicore/index.html
My worry with parallel loops in R would be inadvertent race conditions. For example, is the random number generator in R thread safe? I doubt it.
One of the reasons I picked Clojure over the other languages is that I knew that I could use code from the Incanter project if I needed some statistical functions.
A link to my content-complete thesis is: https://docs.google.com/document/edit?id=1kOKjY265a3F5SbN25d...
The thesis is long and boring to read. Let me save you some time. Here it is in a nutshell: Genetic algorithms (GAs) can be used, somewhat successfully, to identify good parameter-sets for technical trading strategies; however, even the "best" GA-identified parameter-sets in my research failed to outperform the buy and hold strategy when trading, with EOD prices, over the course of a randomly picked 1-year period taken from the interval starting January 1, 1985 and ending May 1, 2010.
My code is not available yet because according to the university it belongs to them (ridiculous, I know). I'll have to ask my advisor whether or not I can make it publicly available.
Enjoying you're thesis, btw, especially the section on Bollinger bands. The Turtle Traders (http://bit.ly/by1j2M ) seemed to have used this successfully with commodities. Did you reference any of the Turtle traders stuff when you were designing your system?
Thanks for the comment about my thesis. I hadn't ever heard of The Turtle Traders, but I read a few pages of the preview that you linked to, and it looked interesting enough to buy a copy. It will be an interesting read. Thanks for the tip.
Also, I bought and read Way of the Turtle. I really enjoyed it. Thanks for the suggestion.
http://www.etsy.com/explorer
It's used for the autosuggest and facet suggestion on the sidebar.
We'll be posting more about this on our tech blog sometime soon.
I'm also and to liberate data from a MeetingMaker server that's only available via a Java api and make it available in Rails apps. Obviously the superior expressiveness of Clojure makes the code base a lot smaller and makes more sophisticated processing of the data far simpler than it would be in Java.
Hopefully more soon.
https://docuharvest.com
Short reason: JVM perf/libs + persistent data structures + lisp goodness (macros etc.) is a helluva package deal.
Plus, honestly, it's a lot of fun to use.
Next up is some data analysis: I'm looking into the excellent Incanter for that.
Great libs, great community and a fun, fun language that's practical and elegant.
EDIT: to expand on what I like about the language; it's a lisp, so everything that comes with that. It's also got some syntax that simply makes it more pleasant to use on a day-to-day basis. And so much more!
(excuse typos: my phone is acting up today/can't handle HN for long comments.)
> Many fewer lines of code than
This grammar bugs me. I feel like this should be "Much fewer" or similar, but I'm not totally sure it's wrong as posted. Opinions?
I do agree with your assessment of the awkwardness of the phrase though.
"Much fewer" is definitely wrong.
"Far fewer" would be my choice.
Normally, grammar nazis at least know something about, which you clearly do not. Why does someone of such low caliber try to correct others?
I'm looking at several possible applications, the one I find most interesting being a tool to catch students who pay others to write their papers.
Clojure has a few features that have been a big help. The rich set of tree-based data structures is especially helpful because it makes splitting and merging data inexpensive. The ability to inexpensively switch to mutable versions and back where appropriate for performance reasons is also quite nice, and absent from most other functional languages. The fact that its a Lisp has advantages that should be well-known by most people here.
Edit: the fact that it's fast is also important for my application. I need to be able to do classification on a large number of possible categories at web speed.
There are a lot things to like about clojure, but the reason I chose it for this task was that it was the only language on the JVM that had a usable (imho) repl.
Macros were essential to what I wanted in a PEG-based system. I wanted to use as few extra characters as possible to increase readability, since fewer characters meant less clutter around the Non-Terminals. I also wanted a custom syntax that leveraged the basic approach of the parent language to minimize the learning curve. Macros allowed me to easily introduce those features inline with the rest of the language.
I've also wanted to use a language that was moving toward platform independence so that my efforts would last longer. Clojure isn't far (a year or so) from being more easily implemented on platforms other than Java. I'm fairly convinced that they'll make it and clj-peg will take advantage of that if it can.
While Common Lisp, among other languages, meets the macro feature, and I developed in Common Lisp for a while, bringing the Java ecosystem into the picture solves other problems I had had with common lisp.
In addition to clj-peg, there are other projects that are in-process, but I'd rather not bring them up here since they don't have the resources that clj-peg currently has.
To sum up: Macros and the Java ecosystem.
Please consider moving the project to a hosting site like github which would make it easier to track updates to it and easier to find in the future. Thanks again!
In my case: I reported a bug in the clojure.contrib XML library, on the google group. It's a trivial fix, but not worth the hassle of signing a contributor agreement. So although I'd fixed it myself, I couldn't send a patch. All I could do was report it - and the report was ignored.
Some time later, I reported another, deeper bug that I didn't know how to begin fixing (but did know a workaround for). It got ignored too.
I like the language, and I'm not going to say my experience was typical of the community. I spent some time hanging out on the group and the people there seemed friendly and helpful. But that's one reason I no longer use it. (Another reason is "no reason at all", and I don't know the relative weights of these reasons.)
This feels a little like whining to me, but feedback should be both positive and negative.
Hopefully this will improve with the creation of the Clojure/core group. Having every commit need to be approved by a single person is absolutely unsustainable in the long run, and we're definitely already at to the point where it's causing issues.
That said, it certainly doesn't stop me from using Clojure every day; it just makes me prefer decentralization (using 3rd-party libraries rather than contrib, for example) where possible.
But now it's had more more pedestrian nuts-and-bolts kind of libraries added like clojure.test and clojure.main. These are included out of necessity, but they are very easy to understand and fairly easy to patch and improve as well. It would be reasonable if the original authors were allowed to improve them and accept patches from the community for them even after they've been promoted from contrib.
I went back through the mailing list and think I have found the issues you are referring to here. I have asked Chouser for feedback on the XML issue, and have responded to the "deeper" question myself (http://groups.google.com/group/clojure/browse_thread/thread/...) Let me know (preferably on email) if I found the right ones, and if the answer is satisfactory.
Clojure code services the HTTP service calls (multimethods are very nice here) and also does the job of converting spreadsheets or CSV files into XML.
I found this great series by Rich Hickey last night: http://clojure.blip.tv/
http://cloudhoist.com/
Lisp in general is amazing, and being able to deploy to the JVM is a huge win for Clojure.
I really like the language. Am I productive with it? Not really - as a guess, I'm about 3 times faster with .NET / C# (with which I'm very experienced). I might be able to get to the same level of productivity, but it'd certainly take me a while to get there.
http://github.com/gcv/dgraph
I also wrote an embedded database library for Clojure, based on Berkeley DB.
http://github.com/gcv/cupboard
Which specific features make Clojure a good choice? Interactive development and macros.
We are using Clojure since the early days and we are extremely happy with it.
We used Clojure to build a finance engine which aggregates financial info from various places, stores them in the DB and serves them over a RESTful HTTP API.
Clojure's awesome concurrency semantics and the overall power of Lisp combined with the vast JVM ecosystem has helped us a lot.
We are Infinitely Beta (http://infinitelybeta.com). We are launching in a month, I will announce when we do.
Pros: wonderful integration with Java, good support for concurrent programming, conciseness, macros.
Cons: numerically intensive code can be a bit slow (but it is going to improve significantly with 1.2, and you can always do that in Java), prefix notation can be quite hard to read if you are dealing with complicated math expressions.
We use the power of lisp to extend the language for specific tasks, like accessing the distributed key/value datastore or building an expert-system shell.
http://project-overtone.org
Clojure has without a doubt been the most eye opening language I've learned in the last 10 years. I highly recommend giving it a try. It has powerful yet incredibly simple to use constructs for dealing with state in a multi-core world, and I think it far surpasses everything else out there today for parallel programming. The centerpiece being a built-in software transactional memory system that enables composable, deadlock free, multi-threaded coordination.
Beyond solid support for concurrency, the standard library provides an elegant set of programming tools that will make you realize how even expressive languages like python and ruby make you jump through lots of unnecessary hoops. In Clojure you represent everything with basic maps, sets and sequences, and the standard library has extensive tools for manipulating these things. (By the way, your code is also available for manipulation with these same tools.)
Macros let you peel away every last bit of boiler plate, and what you realize is that in the end all of programming is just arithmetic, iteration, branching, and assignment. Everything else is just getting in the way of your core logic. I think Clojure lets you get to the center of your problem with the least overhead possible.