HN Explorer

16 comments

[ 2.4 ms ] story [ 47.1 ms ] thread
soumyaskartha
Clojure never got the data science crowd even though the language is genuinely good for it. Always felt like a distribution problem more than a technical one.
ertucetin
I’ve built many different kinds of software (backend, frontend, 3D games, cli tools, code editor, and more) with Clojure and have been using it for over a decade now.

I can confidently say that, among the list I mentioned, it’s the best for data manipulation/transformation. Thanks to the author for presenting it clearly and showing how the libraries and code look across different languages, all of which do a great job.

But Clojure has its own special place (maybe in my heart as well :). I think Clojure should be used more in the data science space. Thanks to the JVM, it can be very performant (I’m looking at you, Python).

hatmatrix
There was XLISP-STAT before R, but the scientists have spoken. They don't like the parentheses.
__mharrison__
Good pandas and polars code should also be written in an immutable way...
olivia-banks
Having "NA" being treated as nil/null/None by default seems like it would cause the Namibia problem!
zmmmmm
Seems like it's going to be a tough sell to get people to want to write

    (tc/select-rows ds #(> (% "year") 2008))
instead of

    filter(ds, year > 2008)
They seem to ignore the existance of Spark, so even if you specifically want to use JVM it feels clearer and simpler:

    ds.filter(r => r.year > 2008)
condwanaland
Couldn't agree more. R and dplyrs ability to pass column names as unquoted objects actually reduces cognitive load for new people so much (pure anecdata, nothing to back this up except lots of teaching people).

And that's on top of the vastly simpler syntax compared to what's being shown here

geokon
In my experience the advantage comes when you have a few more lines of code

The Clojure pipelining makes code much more readable. Granted dplyr has them too, but tidyverse pipes always felt like a hack on top of R (though my experience is dated here). While in Clojure I always feel like I'm playing with the fundamental language data-types/protocols. I can extend things in any way I want

aphyr
You're right, that is longer! I get why though; `filter` is a clojure.core function name people don't necessarily feel comfortable shadowing, and the Clojure and Spark versions make it clear what's a symbol in local scope versus a field in the dataset. I don't think it'd be hard to make a little wrapper for this sort of thing though! Here's an example which turns any symbols not in local scope into field lookups on an implicit row variable.

    (require '[clojure.walk :refer [postwalk]])

    (defmacro filter
      [ds & anaphoric-pred]
      (let [row-name (gensym 'row)
            pred     (postwalk (fn [form]
                                 (if (and (symbol? form) (nil? (resolve form)))
                                   `(get ~row-name ~(str form))
                                   form))
                       anaphoric-pred)]
      `(tc/select-rows ds (fn [~row-name] ~@pred))))
Now you can write

    (filter ds (> year 2008))
And it'll expand to the ts form:

    (pprint (macroexpand '(filter ds (> year 2008))))
    => (tc/select-rows ds (fn [row2411] (> (get row2411 "year") 2008)))
hrrld
In practice we use `ds/filter-column` and `ds/filter` much more than `select-rows`.

The sell isn't about typing a few more or a few less characters, it's about doing data science functionally.

teleforce
All the comparisons are with scripting and untyped languages perhaps for faster development and more intuitive eco-system to increase developer productivity.

In the age of IntelliSense, auto-completion and AI assisted coding, does the choice of scripting and untyped language justifiable for increased in productivity at the expense of safety and reliability?

If you're building data system not just for exploratory, surely modern compiled and typed system languages like Rust and D language make more sense for safety and reliability for the end users?

Even more so with D language where you can even have scripting capability for exploratory and protyping stage with its built-in REPL facility [1],[2]. This is feasible due to its very fast compile time unlike Rust. It has more intuitive "Phytonic" syntax compared to other typed languages [3]. You can also program with GC on by default if you choose to. Apparently, you can have your cake and eat it too.

[1] drepl:

https://github.com/dlang-community/drepl

[2] Why I use the D programming language for scripting:

https://opensource.com/article/21/1/d-scripting

[3] All in on DLang: Why I pivoted to D for web, teaching, and graphics in 2025 and beyond! [PDF]

https://dconf.org/2025/slides/shah.pdf

manudaro
The Clojure tablecloth performance numbers here are pretty surprising, usually see Python/polars dominating these benchmarks. Been running similar transformations on transit data feeds and polars consistently outperforms pandas by 3x-5x on the group-by operations, but hadn't considered Clojure for the pipeline. Anyone actually using tablecloth in production data workflows?
daslu
Great post.

The way Tablecloth unifies column processing and row processing in a functional way is so elegant.