>If we used interop for everything mundane, Clojure would really just be an S-expression shaped husk over Java code. Not a very good solution.
IMO some programmers pursue purity (no pun intended) over pragmatism, at their own expense. I understand the appeal, it's like hand planing a board with your vintage No. 8 instead of sending it through the Powermatic.
It's a lot more like buying a board planing robot cell, replacing the robot with a human operator because you couldn't get a teach pendant for it, replacing the automated part loading/unloading because you're now running parts that your fixtures can't handle, and ultimately replacing your low-cycle-time planer with a Powermatic because the vendor went under and you can't get any spare parts. Now you just have a Powermatic with a human operator and a distant memory of your initial machine.
You (at least should be) picking oddball stuff like Clojure because the unique advantages outweigh the problems. If you're just using it as a wrapper over another language that doesn't provide those advantages, you're taking the balls off the bull, and you probably should have just used Java, C#, Go, etc.
And clojure is actually in a better position than other niche languages because _at least_ you have the option of calling the boring and maintained java library to ship _something_ (it might be something non lispy, maybe a bit less efficient and maintainable but at least you solved the business issue.)
Try doing that in some other languages were your underlying platform has even less libraries than a JVM...
It’s very easy, as someone who went from Ruby to Elixir and had little understanding (to start) about Erlang.
It might be more natural than going down to Java because you’re mostly dealing with immutable data and Elixir is very consciously building on top of Erlang. In comparison, Clojure has always struck me as wanting to build on top of JVM but being an entirely different thing.
I did it for some zip stuff that was already implemented in the Erlang stdlib, it was surprisingly easy. I assume the very small numbers of types in Elixir helps with that.
Yes, you get the eco system. But to what extend, the very thing mentioned in article rails-like framework is not really accessible right? I am ok with clojure not backing web framework, but also this one place where interop with springboot would be nice is not there (right?).
At some point, do we have to consider that the problem might be Clojure itself? Does LISP lend itself to "Rails" -- a Rails that people want to work with? I ask, genuinely, as a near-total outsider.
The comparison to Elixir got me really thinking. I appreciate your "call to action", but community consensus gathering wasn't necessary for Phoenix to emerge as an option-of-choice, it gained traction because it is _good_.
Surely there are large organizations using full stack Clojure, similar to the Lawrence World-Journal. Is there something about LISP that makes it hard for them to abstract their "framework" bits into an open source package?
The something is probably more philosophical within lisp communities. I’m relatively new to lisp (writing clojure/cl for about a year) and from what I’ve seen people prefer to bolt on small things instead of big frameworks. The full stack frameworks exist but they’re hardly frameworks. They’re closer to project templates than anything.
It might be that simply committing strongly to a language, any language, has benefits? Years ago, a long term consulting customer hired me on a short fire-drill job. He had also hired some Clojure celebrities like Stuart Sierra and Stuart Holloway. They were incredible productive and fast. It was a very long time ago, but as far a I remember they had a small set up libraries they used. BTW, I was hired for a few hours a week to help with dev ops, not code.
EDIT: I am currently working (part time) with an all in Common Lisp company. It is really nice to see a group of people who all love a programming language using it. As much as I like virtually ALL Lisp languages, I do sometimes feel guilty not just using Python when there are much better libraries available (e.g., machine learning, linked data, etc.). When a programming language has 1000x as many developers you simply get a richer ecosystem.
> When a programming language has 1000x as many developers you simply get a richer ecosystem
Interesting comment given the article. That's the magic of Clojure being hosted, you get the rich Java ecosystem and get to use your Lisp that you love.
The downside is you still need to deal with the rich ecosystem that isn't as nice as if there was a rich one in Clojure.
You are correct, Clojure does get Java’s rich ecosystem, and Python’s via pythonlib. There is some impedance mismatch that you don’t get using, for example, Python directly.
It will be interesting to see what languages win out long term. I really don’t like programming with Rust but it is becoming wildly popular. I do like Swift, the language is intuitive to me, has good REPL support, and has awesome standard libraries for machine learning and just about everything else.
All that said, I am an old man, I like Lisp languages, and that works for me.
That's fair, I didn't intend to sound entitled, but "let's all decide what Rails-like to use" doesn't seem to be working for Clojurists, so I was hoping that poking at other avenues might spark some productive conversation.
It's not that it doesn't work, it's that most Clojure programmers do not see the benefit since the selection of data-oriented libraries we have is much superior in our minds.
Some people (a small minority) are trying to "solve" this, but it garners little interest since most people don't see any problem.
"community consensus gathering wasn't necessary for Phoenix to emerge as an option-of-choice, it gained traction because it is _good_"
I think it also because there's two camps of elixir devs - those who came from Ruby and those who came from Erlang. The Rubyists have a fairly easy time being productive quickly with Phoenix because it (at a glance) is very similar to Rails. From there you learn about supervisor trees, genserver, etc.
> Surely there are large organizations using full stack Clojure, similar to the Lawrence World-Journal. Is there something about LISP that makes it hard for them to abstract their "framework" bits into an open source package?
There's a fundamental model shift that's not as compatible to frameworks.
Lisp optimizes for it to be really fast to implement perfectly tailored custom solutions, with your own DSL (domain specific language) that best fit your problem. Iterations are quick, development is fast, you can automate away repetitions and ceremony trivially.
The need for scaffolding and for complicated code-gen, which is half of what most web-frameworks provide, is just not there, because Lisps are already meta-programmable.
The other thing is Clojure's functional nature. Objects and Actors and Modules, these things naturally expand to become Singleton state or behavior containers that you call upon. They also naturally lend to a plugin architecture, where configuration setups a chain of implementing interfaces called into by standard modules from the framework, etc.
The functional, higher order, immutable, and data driven nature of Clojure doesn't fit well here either.
What happens instead is that libraries, not frameworks, tend to work better in Clojure. Things you can call and use as you want, instead of things that call you and ask of very specific extension and configuration from your part.
So the Clojure web stack is still full of reusable code, they're just exposed as libraries.
When you know what to do, you can easily put together a Clojure website together, in no less time then it takes using a framework in another language, and you similarly reuse almost as much code as in any other language.
The difference is that because you need to use the libraries, you also need to figure out how to use them and where and what to do to glue all of that stuff together.
In a framework, you can't choose how to glue them all, it's forced on you, that's both the issue with frameworks, and the benefit. So when you don't know how to build a website, a framework really helps, it guides you, and you just follow its strict structure and plug what needs to be plugged, and configure what needs to be configured.
With the library approach, that's left up to you, much harder to get started with, but not any less productive ounce you know.
Finally, a framework forces everything to be a monolith, that is built and designed within the rules of the framework itself. All behavior you'd want to reuse needs to be framework aware and compatible. This means that slowly people grow an ecosystem of compatible libraries all around the framework.
Again, with the library approach, the various libraries you choose to use were all independently designed and built, they weren't built to work together, so again, you have to find how to glue and connect them and map between them yourself.
You could still have a framework, but the mental model a framework follows is often contrary to the mental model Clojure enables, people often choose Clojure because you can quickly customize things to your liking and connect various independent building blocks together in the way you prefer and that works best for you.
That means whenever someone made their code structure available to others as a framework, everyone else went, I don't like the way you did it, so nevermind, I'll just use my way.
If the libraries all only worked with their structure though, it be too much effort to go your own way, but not so in Clojure, because all the building blocks are exposed as libraries, so you can easily structure it however you prefer.
> If we used interop for everything mundane, Clojure would really just be an S-expression shaped husk over Java code. Not a very good solution.
I don't have a lot of experience with Clojure, but speaking as someone who uses Kotlin on the server side, this attitude strikes me as odd. Most of the reason to use a JVM language is to take advantage of the mature and well-maintained Java libraries for basically everything.
If interop is considered a last resort (to the point where you reach for a 3-year-old unofficial Stripe library over the officially maintained Java one), what's the point of running on the JVM at all?
One problem with interop is you import the design/ergonomic considerations that may come from other languages, while neglecting what is uniquely powerful about your own language.
“Idiomatic Clojure code calls Java libraries directly and doesn’t try to wrap everything under the sun to look like Lisp… Where Java isn’t broken, Clojure doesn’t fix it.”
Perhaps that stance has changed over time. In Clojure, there's the clojure.string namespace just functions wrapping java.lang.String methods https://clojuredocs.org/clojure.string
This is much more than "just wrapping". It allows strings to be used more readily with existing Clojure abstractions and/or allows idiomatic use without the need for being on guard to avoid reflection.
Notably, Alex didn't write this library and only has a minor contribution to it.
I think the problem is that if you want to write code portable between clj and cljs, you’re gonna need a lightweight abstraction. With that said, I’m still not sure how exactly the maintainers feel about portable code, having written a portable http library.
There's two cases where you'll see pragmatic wrapping:
1. Higher order usage. When functions would really benefit from being used in a higher order way, passed around as values, you'll see some of the Java methods wrapped so they can more easily be used as such, since you can't use Java methods in a higher order way. This would count as "can do it better", and it's the case for clojure.string.
2. For portability with Clojurescript. This is a more recent thing, but as Clojurescript usage grows, and code reuse between backed and frontend more common, there are certain basic methods Java interop was totally fine with that now benefits from being wrapped. This is the case for clojure.math.
Clojure is intended as a hosted language and embracing the host is the philosophy, so interop with the host is never a bandage.
Again, I am curious where you get all these impressions that are contrary to most people in the community take for granted, such as:
* Clojurians prefer libraries over framework, but you said we spent too much effort in developing "big web framework".
* Clojure is intended to be a hosted language, so seamless interop with the host is the norm, but you thought interop is a bandage until Clojure matures.
Really it's not. The interop is a conscious decision core to the language rationale.
https://clojure.org/about/rationale
> Customers and stakeholders have substantial investments in, and are comfortable with the performance, security and stability of, industry-standard platforms like the JVM. While Java developers may envy the succinctness, flexibility and productivity of dynamic languages, they have concerns about running on customer-approved infrastructure, access to their existing code base and libraries, and performance
> symbiotic with an established Platform
> VMs, not OSes, are the platforms of the future, providing: [...]
* Libraries
* Abstract away OSes
* Huge set of facilities
* Built-in and 3rd-party
Where Clojure had nothing to bring, nothing was implemented, because interop is considered just as good, that's why there's not a lot of IO functions in core, and why math was missing for a long time.
A good wrapper (like the Clojure version of java-time) can be a real lifesaver for those of us without Java backgrounds who don’t like having to learn deep and often baroque Java apis.
I also use a pretty half baked wrapper someone wrote because I just about lost my mind trying to make sense of the class hierarchy it was wrapping.
I think for people like Rich and Alex wrappers may look pointless because they already have invested in learning many of these APIs, know the patterns already, etc. For people like me from different language backgrounds they keep me from having to learn APIs with mutable state and deep class hierarchies and to me that’s a big win.
(I rewrote this whole comment to hopefully be more concise.)
Just a nit-pick: clojure.java-time wraps Java's own _immutable_ java.time.* classes and it really doesn't provide much value add. I initially thought it did and introduced it at work but we're taking it out of a lot of our code now and using plain interop -- it's only slightly more verbose, it's already mostly "functional" in style, and it gives us more precise control over conversions between different types of date/time values.
Your main point about interop as a barrier to non-Java people is reasonable -- although if you learn a hosted language, you're going to have to learn stuff about the ecosystem on which it is hosted (so, not Java itself, but some of the JVM stuff and the docs for any libraries you interop with). And this is true of the JS ecosystem supporting ClojureScript (and the .NET ecosystem supporting ClojureCLR).
Thanks, I’ll check out the raw java-time next time I have a minute.
You’re right about needing to learn some of the host in a hosted language. I’ve enjoyed for example Jsoup and I use some of the Math stuff inline with no issue. Even with the java time wrapper I’ve dropped down to the original to do certain things involving locales. I definitely knew going in to Clojure I’d need to pick up some Java.
Where I really like wrappers is where the underlying java API is not great in some way. (Which now that I’ve seen the original puredanger quote seems to be what he’s saying.)
> although if you learn a hosted language, you're going to have to learn stuff about the ecosystem on which it is hosted
I thought this at first, but then I thought if you take Python, which is hosted on the OS directly through CPython, so we can say Python is hosted on Windows or Linux.
Well it turns out, most people never need to interop directly with Windows or Linux APIs or any other C interop.
Python seems to have a different mindset here, where most of its value proposition are providing wrappers over C/Windows/Linux/Mac APIs.
Why does so much effort get put in maintaining all these wrappers in Python? Is it just attitude? Is it that those APIs are much worse then Java's enough that even C programmers using Python want to wrap them over using them directly?
This is the most illustrative analogy. I suppose the answer is a little bit of both.
Python numeric performance is a really interesting case. Those folks would certainly drop down to C. But my casual observation is that they would build the bridge when they came back above water. A lot of work went into this because Python performance has historically been quite bad for a Data Science/ML language.
Part of this is the size of the community.
But what is we also consider the Common Lisp community? Common Lisp is incredibly performant. A lot of work went into the underlying C libraries. Perhaps most of that was when the community was more vibrant. But still, a casual observation is that CL folk live in CL.
Very good point. I have no rebutal, you're absolutely right on that front.
I just don't know how to address this problem. It seems the people in a position to build good wrappers or Clojure implementations are also the people who know Java and Clojure, and maybe came to Clojure from Java. For them, it's easy to interop directly, so they take the path of least effort, and don't bother making a Clojure lib.
In a way, if Clojure couldn't use Java, while it would mean I probably wouldn't be able to use it at my work and would be forced to still work in Java, it would also mean maybe for my personal stuff or for certain companies who'd embrace it anyways, I would and they'd be forced to go ahead and put the effort to build Clojure variants of all these libraries, and that would make it friendlier to people learning Clojure that don't know Java.
I have to admit, I always react a bit defensive on articles like this one, and it's mainly because I don't see the problem, I see people complaining about things that are non-issue for me. But your point hits me in a different way, you make a good point, having to learn a language and a half (all of Clojure and half of Java) is definitely not as good as it should be.
I think normally people come at it like saying how interop is bad, difficult, not idiomatic, or that there's no libraries so they had to use Go-lang, where there were clearly options to use a Java lib from Clojure. But if the article just said, hey, using Java is unapproachable to a lot of folks, learning to understand Java APIs and semantics is a lot of work, and it's turning people away from Clojure otherwise, now that's something worth discussing.
Ya I hear you and agree with most all of this. I did choose Clojure in part because it had Java interop so on some level I knew I’d need to learn a bunch of Java and APIs to take advantage of that. I think it’s a great design decision but also am actually kind of a fan of wrappers - I think being able to use raw Java is great and sometimes the right choice but I also appreciate even the half baked wrappers sometimes. As to the original article I should probably read it :-) I was jumping into the discussion more to address this chip on my shoulder about wrappers.
> Really it's not. The interop is a conscious decision core to the language rationale.
I'm aware. I was expressing an opinion to a twitter friend, but I should have foreseen the replies of some people taking my opinions as me promoting fact. That's my fault for not being clear.
You should take Alex's words contextually, and with a grain of salt. You can easily see his commits in this project, for example: https://github.com/cognitect-labs/aws-api
This library doesn't wrap the AWS SDK, explicitly calls out existing Clojure wrappers that do in the README, and none of his commits seem to conflict with the referenced Tweet. Am I missing something?
When I started out with Clojure, I was using wrappers everywhere, but after about a year or so realised that they added very little value, hurt performance, and made the code less simple. On top of that, wrapper libraries were often not complete in functionality.
The pivotal moment for me with this was when dealing with Kafka, just using the “raw” Java library ended up being so incredibly much simpler than any of the wrapper libraries.
It's been more than a decade since I've used Clojure so I say this with a big grain of salt.
Where there weren't good wrappers, just a few functions "wrapping" Java libraries for my project's needs always felt lightweight and lispy. The little internal APIs I made almost always passed around the library's raw Java objects, but with more Clojure idiomatic ways to construct them, or slapping some seq on the library outputs.
I have a github repo where I implement a feature identical microservice in various tech stacks. I started that repo with a Clojure version that used community provided wrappers. See https://github.com/gengstrand/clojure-news-feed/blob/master/... as an example of calling Cassandra. Recently, I added another implementation with Clojure that just called the Java drivers directly. See https://github.com/gengstrand/clojure-news-feed/blob/master/... for that version of the same call. In the end, I decided to forego wrappers and make the calls directly because you end up with fewer dependencies and are more likely to be able to use latest versions of everything.
Why does he say this? I would think the opposite and in a professional environment I've gone in the other direction of regretting not writing _more_ wrappers for external libs.
It Kotlin it makes a lot more sense to use Java libraries, because for most purposes, the interfaces you'd write in Kotlin and the ones in Java are not all that different, other than coroutines. From Scala or Clojure, the differences in interface design are far larger. You are going from languages where mutable state is an optimization you want to hide, to libraries full of setters that return void. A land of monads, and maybe macros vs imperative programming. This makes the ergonomics of many mature java libraries just too much of a bother to use directly. There are cases where the effort to rewrite is bonkers (say, ApachePOI), and others where wrapping is sensible (see a variety of json serialization libraries which support different parsing backends, some of which will be straight java), but there are many libraries that are sensible from Kotlin, but are worse than a rewrite if you are using an FP language.
Still, I think you are undervaluing the JVM itself, which is almost magical when it comes to production observability. It'd be a whole lot of work to come anywhere near that tooling support for ops if you were avoiding the JVM. Scala, for instance, can also transpile into Javascript if you feel like it, and there's a native project that compiles to LLVM, but the JVM has the vast majority of the use cases, and it's not due to the libraries.
Yeah, I get that. Even in Kotlin there definitely a number of places where I prefer the Kotlin library to the Java, and I can see that it would happen more often in Clojure.
That said, the example in the article is the Stripe API, and I cannot see a good reason not to just use the official Java one. I've used it a ton, and it's as simple as can be—you pass a map in and get back an object. If that's a problem, either Clojure has a way worse interop story than necessary or the author is suffering from a variant of not-invented-here syndrome.
There is a difference between a library and a framework, framework dictate how you work, library is an external, isolated to some extent, entity. I can see why someone would like to have a more native framework but will be ok to use an external library for purposes like stripe.
Agreed, but again, the author isn't actually talking about frameworks except at the very end. Most of the article is about API wrappers like Stripe:
> My theory is, this causes a lack of effort expended on mundane libraries. It's not the most fun thing to build. I get it. Really, I do, but lack of these API-wrapping and protocol-handling libraries put Clojure at a significant disadvantage in the business of software.
The kinds of libraries the author misses in Clojure exist in abundance in Java, and the most likely explanation for their absence in Clojure is that most Clojure developers don't avoid interop as much as the author thinks they do.
To expand a little on this, here's Rich Hickey on the subject [0]:
> I dislike writing Java/C# as much as anyone, and Clojure is my ticket
to writing much less of it, while leveraging the efforts of
multitudes. As I've said in my talks, most Clojure users go from "eww,
Java libs" to "ooh, Java libs", leveraging the fact there there is
already a lib for almost anything they need to do. And using the lib
is completely transparent, idiomatic and wrapper free. OTOH, if they
want to customize their interface to a lib, it is trivial to do so in
Clojure.
> If, however, the stigma of Java extends to libraries written in it,
even when consumed from languages like JRuby and Clojure, and people
think it's cooler to reinvent than reuse, they need a new definition
of cool.
Interop with Java libs is decidedly not pleasant in many cases. I haven't used Clojure before, but I've worked on services built in Scala and JRuby over the years.
There are a few challenges you run into:
1. Many libs written in Java follow the worst of Java's design patterns and idioms. Rather than calling a function and passing it some data as an argument, you instead have to instantiate a WhateverClient that itself takes a WhateverThingyStrategy instance and a WhateverConfig class instance, then call a .executeCommand method with a new IWhateverCommand anonymous class instance that implements the execute, doCatch, and doFinally methods. And of course, you can't forget whatever subclass of AbstractOrigamiWhateverResponseMarshaller you need next. Oh, and because all these instances need to be defined as beans that will be DI-friendly later, there's even more boilerplate to look forward to.
2. In comparison to languages like Scala, Java's type system is often quite lacking. You can never trust that a non-Option type is not null.
3. If the rest of your code is free-floating with structural types in some other language, you inevitably need to write a ton of glue code for adapting these structural data type objects to whatever blessed POJO interface Java expects.
All that being said, I still would prefer to use the more mature Java lib if the more native choice isn't as well maintained. But there's no denying that it's painful. The ergonomic hit is just as bad (if not more than) execing a child process to run code in another well-suported target like Rust/Go/Node/whatever that maybe has more sensible idioms. There are of course a lot of reasons why that'd be a bad idea, but still...
> The ergonomic hit is just as bad (if not more than) execing a child process to run code in another well-suported target like Rust/Go/Node/whatever that maybe has more sensible idioms. There are of course a lot of reasons why that'd be a bad idea, but still...
Come on, we both know this is false and it is only your feeling, nothing factual. How is IPC with all of its intricacies anywhere close to as ergonomic as calling one more method in the host language when all of your objects are also of said host language?
Frankly enough, creating a wrapper should not be hard at all, no matter the language. Especially not for libs that are mostly unidirectional in communication.
> Come on, we both know this is false and it is only your feeling, nothing factual. How is IPC with all of its intricacies anywhere close to as ergonomic as calling one more method in the host language when all of your objects are also of said host language?
Oh, it's definitely not false. The reality is you can't just call one more method in many cases. There is a ton of ceremony involved because enterprise quality coding[0] is often seen as a desirable attribute of the software written in Java. The linked example is an exaggeration, but it's sadly not too far off from reality in some cases.
I'd also counter that IPC is often dead-simple in many contexts. Sure, a long-lived process like a server might have intricacies you need to consider, but outside of that, it's pretty straight-forward. CLI tools are another example of this, but the JVM startup times generally make it a bad choice for CLI tools to begin with.
> Frankly enough, creating a wrapper should not be hard at all, no matter the language. Especially not for libs that are mostly unidirectional in communication.
This depends on the surface area of the library you're using. And maintaining such a wrapper is not a one-off activity. It's not an issue of being "hard", but tedious and time-consuming.
> Sure, a long-lived process like a server might have intricacies you need to consider, but outside of that, it's pretty straight-forward.
The rest of your comment is fluff around this admission that the person you are responding to is correct. I'm sure you're aware this is the primary use-case for Clojure.
In the very very worst case, you can just write a saner Java wrapper over the Java lib (chances are you are not needing every option in the original), and use that from the guest language of your choice.
Suzi just needs to find an excuse to spend a couple years building a serious thing entirely in Clojure, no matter what. And then to disentangle all of the logic particular to her thing from the application framework she's built for this (or the existing one she used as a starting point), as well as disentangle anything that she thinks should be an external library, and release it all for free.
Suzi may need to wait to do this until after her current gig has won the Startup Lottery. Or perhaps she may need to quit her current gig and figure out how to get a bunch of other Clojure-lovers to pay her rent while she does this. Good luck.
Good luck! Getting this post on the front of Hacker News is probably a step in the direction of finding a way to pay your bills while spending a year or two building Canals For Clojure to the point where it attracts other people to build libraries and plugins for it.
I think this article vastly undersells the value of Java(script) interop. The ability to call out to a well-maintained library in two of the most widely-used languages in the industry is one of the major selling points of Clojure as a pragmatic LISP, versus e.g. Racket or CL. The reason why there are so many half-baked wrappers around popular Java libraries is that it’s reasonably trivial to write one yourself on demand, so that’s what people do.
It’s an interesting argument. However, I’m not sure about it. Consider Rust doesn’t have a Rails per se but the ecosystem it is thriving. It seems more like an indication that the Clojure community of developers is either focused on things other than the article is concerned with, or its period of popularity has waned and the libraries that do exist from that time are slowly rotting. I think about Erlang before Elixir. In its niche Erlang had a very good set of libraries, primarily centered around lower level network communication type things. But for integration with popular APIs etc. not so much. After Elixir and an influx of new people the off-the-shelf library situation got significantly better. I don’t think that’s because they were doing Rails type things necessarily but more because there was a more diverse pool of developer community, whereas the Erlang community was slowly shrinking through attrition.
If you want Clojure to thrive again you’ll have to stop the attrition of its developer community. Maybe that’s by building a Rails but even your example in-article where Go is more suitable isn’t really a web type problem. What problems can Clojure solve that other languages can’t, in a way that is worth the effort for someone new to the ecosystem?
I think you've answered you're own question here, in a sense.
Erlang was created by Ericson for use in telecom systems. (OTP stands for Open Telecom Platform) which gives credence to the idea that Erlang didn't need a Rails. Once Elixir came along, being a bit more general purpose, people started building those things.
Clojure, from the beginning, was intended to be a general purpose language, not a web language. So I think it would be fine to expect Clojure to be able to solve non-web type problems. Maybe the community has attrition, but I don't think so. The Clojurians slack seems bigger than it ever was, more companies are using Clojure than I've ever seen.
Minor quibble, OTP _used_ to stand for Open Telecom Platform but that acronym was removed in the push to open-source. Ericson had hoped to sell the Open Telecom Platform to other telecoms but that didn't work out, hence the modern Erlang/OTP. Anyhow, Erlang's niche was and is fault-tolerant middleware, yeah, the kind of thing you see deep in the bowels of systems. The community built up a sizable bulk of code to support the kind of middleware you'd find in Enterprise-y type places in the early 2000s and then slowly started to fall off, leaving an ecosystem that slowly bit-rotted in much the same way you describe in your article. That, specifically, is what I'm referring to as the off-the-shelf rot problem. Perhaps I could have been more clear.
I agree with the argument that Clojure was intended to be a general purpose kind of language, but I think you article also suggests that -- unless you're tied to a JVM model _and_ you refuse to use Java -- that Clojure gets out-competed by languages like Go in that kind of work. It's not clear to me how or why building a Rails kind of thing for Clojure would reverse this situation. I think, also, the size of an online community and tally of companies can be misleading. Erlang's IRC and company use was at an all-time high even while the actual pool of developers had reached its peak and was declining, slowly but surely. It's argument from analogy, sure, but what you're describing in the article for Clojure looks damn familiar from my time with Erlang, pre-Elixir.
> Clojure, from the beginning, was intended to be a general purpose language, not a web language
i don’t think that is strictly true, clojure is for “situated programs” which is basically a perfect description of cloud information systems and user interfaces.
immutability and macros is not what you want for games, for example. and data science is very SQL/struct oriented, it doesn’t hit clojure’s high notes. mobile is very platform interop heavy and best done natively without adding intermediate abstraction layers. databases are written in lower level langs for performance (Datomic is not known for lightning fast queries). what’s left?
I think the lack of a Rails or other flagship framework has arguably hurt Rust if you compare it's trajectory to that of Go. Go had Docker and Kubernetes as early flag bearers to indicate it's a "real language", while Rust has only really had the critical mass to start getting treated as one in the last two years.
Ah yeah, very fair if we’re talking “Rails” as a signifier of a big project that gives a language and ecosystem is “flavor”. For what it’s worth, I imagine Go has a broader remit than Rust even today. Go is a very good update on the pain points of Python for domains where integration with existing C libraries can be a little wonky, resource consumption isn’t a major concern. Rust puts resource consumption and fitting into existing systems projects as a first class design concern, which is great for me because that’s the kind of thing I work on, but I do admit it makes for a more challenging language to use.
Clojure typically doesn't attract the kind of developers who want to work on "boring" things
So there is an imbalance there
On one hand I'm kind of happy there's no one true way in Clojure to do web dev, because as I think Rich said, it's not a solved problem
So if we couple too heavily to one opinion on web dev then it's going to fail us as a community on a big scale and be hard to move away from
Sets of libraries are typically easier to swap out
In reality I think most Clojure businesses end up on reagent or and reframe and they are the defacto ways of doing web dev
For "solved" problems like stripe calls it would be nice if we had more people working on maintaining libraries but as a Clojurian myself I know I wouldn't do that myself so I only have me and people like me to blame
> Clojure typically doesn't attract the kind of developers who want to work on "boring" things
I'm quite the opposite. I love Clojure, just because it is "boring". Maybe not "boring" as in well-known, but "boring" as in the core language and it's runtime doesn't change and is stable, like really stable. I've upgraded projects using "ancient" Clojure versions (released +5 years ago) and not having to almost do any changes, as both the language, the runtime and most libraries are so stable over time. It simply just works. If people want to introduce breaking changes, they tend to prefer to create entirely new libraries instead.
Could you elaborate on that? I don't do ClojureScript, but I've dabbled with it on and off since 2013 (I've been doing Clojure in production for over eleven years). When I first started looking at ClojureScript, Om was "the thing" and Reagent was the new kid. I built a prototype at work with Om and then rebuilt it with Reagent and found the latter much more pleasant and idiomatic to work with.
Last year, I dipped back into ClojureScript and built some simple projects with re-frame and I really liked it. I can definitely see why re-frame has become so popular.
> Clojure typically doesn't attract the kind of developers who want to work on "boring" things
I kind of have the opposite opinion. I worked on a large Clojure codebase for several years. There was so much effort put into building tooling and libraries in Clojure because the ecosystem isn't huge, (and some of its bigger pieces are just a misadventure, ie spec). It was really frustrating to me because I felt like we were spending 50% of our time writing inferior versions of code that Python, Ruby, or even JavaScript already had as libraries. I'd have preferred that time to be spent on interesting things like business concerns. Of course, you can use java interop, but it's much more natural in Scala and Kotlin.
I have a little trouble remembering all the details, and I'm not sure I can speak to where things are now. Maybe it's more helpful to mention where I thought we got off-the-rails with building our own stuff?
- unit tests with ephemeral dbs
- db migrations
- ORM library
Many of these things exist in Java and in retrospect, were probably the way to go. One other pain point was that Clojure libraries would stop being updated, lag too far behind the java libs, or lack features. I remember this being the case with Google Cloud and AWS, and I think Elastic Search. I remember a number of times just completely swapping out the Clojure wrapper lib for the java lib.
Most of these things do have Clojure libraries. For example:
- unit tests with ephemeral dbs: we use com.opentable.components/otj-pg-embedded for postgres.
- db migrations: there are Migratus and many others, we also just use flyway in app startup shell script
Some are not relevant if you write proper Clojure.
- ORM library: I am not sure you need these in Clojure. Clojure is data-oriented, not object-oriented, so you don't need to map data to objects. Clojure is good at transforming data. Just keep data as data.
I don't recall us spent too much effort in building Clojure libraries of our own for something missing.
For my over a decade of Clojure career, I only built two Clojure libraries of my own. Both were not missing, just that I didn't like the existing ones. One is a Clojure data diff library (Editscript), and another is a database (Datalevin).
In general, if one cannot find a Clojure library (very rare), we tend to use Java interop. As mentioned repeatedly, Clojure is a hosted language and we embrace the platform.
I don’t understand this at all. Clojure’s all over the place in banks and fintech startups and they’re all boring as hell. People don’t spend a lot of time on random proprietary wrappers because they don’t provide a lot of leverage, not because they’re ‘boring’.
This is the second appearance of "boring" in this thread. There is also "boring" libraries. Somewhere someone should-be/is writing a thesis on the psychology of programming and programming languages.
But API wrappers handling things like retries, pagination, client-side business logic, and coercion from JSON. Having to account for all of that for two API calls is somewhat tedious and repetitive, and I'm sure we've all solved those problems at least a dozen times.
Retries, JSON coercions and so on do have Clojure libraries for them, so I am not sure why you want to solve these problems yourself, let alone a dozen times. Really not sure what your exact point is.
Writing high quality ergonomic API bindings is a surprising amount of work, especially in statically typed languages.
Every API has its quirks, sometimes returns different data, has edge cases for certain calls, has different error reporting patterns, rate limiting, authentication methods, sometimes has intended or unintended breaking changes, ....
Sure, you can usually just manually whip up the code for an endpoint or two in a few minutes , but then you need to continually add endpoins, fix the edge cases, adapt to upstream changes, ...
Correct. If we take their point to an extreme, HTTP calls are just bytes sent and received over TCP, so what's the use of an HTTP client library? Just open up a TCP connection and send/receive text to/from the server!
Clojure libraries target microservices with a precision that no other language ecosystem has. In essence, Clojure web services developers rail against Rails and other bloated, unnecessarily complected (https://www.infoq.com/presentations/Simple-Made-Easy/) frameworks of the 90s. As a Clojurist I too rail against Rails. I don't think that expansive model fits the problem space. I have had painful experiences in the past maintaining Rails projects wondering why they didn't know of DRY. If there is a essence within Rails that you feel could be distilled into a lean Clojure model, build it out in a library and share it.
I'm not advocating for a Clojure Rails because I love Rails. I've never used Rails to be honest. I'm arguing open-source efforts are repeatedly being spent on trying to build the next web framework/library/toolkit (Rails) for Clojure and not much else, so it would be great if there was one, so we can get on with filling the gaps in the Clojure ecosystem.
"open-source efforts are repeatedly being spent on trying to build the next web framework/library/toolkit (Rails) for Clojure and not much else", really?
That's contrary to what most of us know. There are some efforts in the Web front, but not much. At least, the community is not paying much attention to these efforts.
Let's look at the list of community funded projects, e.g. those in Clojure Together: the only Web related projects funded were clj-http in 2018, ring, re-frame and reagent in 2020. None of these are Web frameworks, and the rest of the funded projects are not Web related at all.
> Clojure libraries target microservices with a precision that ..
I’m not sure what that means exactly, but no one is using Rails only for building out micro services. They’re using Rails to go from nothing to a production ready web application, with all the incidental complexity taken care of, in a very short time.
It’s been a few years since I was involved with Clojure ecosystem - what is the Clojure experience equivalent to say, the original Rails demo from back in 2005? The last I tried, all the parts seemed to be there, but much painful assembly/“composing” was required and not all the parts fitted which ended up producing a lot of awkward complexity needing desperate decomplecting.
I also thought "Clojure needs a Rails" some 5 years ago. Nowadays (multiple, real-world Clojure jobs later) I don't.
I also learned that trying to second-guess why developers spend their time in certain libraries and not in other endeavors (like frameworks) comes across as pretty disrespectful and actually ignorant (considering that those people are very often more skilled/experienced than most of us).
(Good related read: Open Source Is Not About You)
So I'd recommend to anyone, open your mind, be ready to have some preconceptions changed, and contribute a significant deal to (Clojure) OSS to truly appreciate what we have. And write more interop ;)
I see exactly the same thing in the Scala community. Your first instinct when you need a library is to find a pure Scala one, when many times there are far superior (in terms of maturity, stability and documentation) in the Java word. Sometimes it makes sense to write an intermediate wrapper but not always.
Stop trying to make Rails in things that can’t be rails. Rails is Rails uniquely because it is RUBY on Rails. It plays uniquely into Rubys strengths, that’s what makes it what it is.
What would a 'Clojure on Rails' framework look like? I think that's the primary reason why such a framework has yet to find traction; there's no single, obvious way to go about it.
The efforts made so far in this space have taken many different approaches, because they're all experiments. Explorations of what an idiomatic Clojure web framework might potentially look like. Clojure has been around for over a decade now, and it's fairly mature as a language; but it's ideology is young in comparison to more established paradigms like OOP.
To make matters more complex, Rails came to prominence in an era where web applications were mostly of one type: server-side HTML with a sprinkling of JavaScript. Nowadays there tends to be more options, e.g. a web application may consist of a single HTML shim, with a thick React client that talks to a GraphQL back end.
So on the one hand we have a language that's still exploring its identity; on the other we have a rapidly changing idea of what a web application should be.
My guess is that an idiomatic Clojure web service would look like a client-accessible relational database with a strong security model and datalog queries, along with some system for adding in side-effectful hooks to respond to data change. In other words, something rather different in design to Rails.
To clarify, I'm not advocating to re-create Rails for Clojure, rather, I'm arguing open-source efforts are too focused on the search for Clojure's Big Web Framework TM. However, I do like your guess as to what a Clojure Web service might look like.
Your observation is simply not true. To the contrary, the consensus of the community seems to be that there's no need for a rails-like framework for Clojure. Most of Clojurians are satisfied with what we have in the Web front, and are not worrying about the lack of "big Web framework" at all. I don't know where you got your impression from.
I got that impression from the relentless amount of web frameworks for Clojure that keep popping up. Anytime I see Clojure on the front page of HN, it's about some new Clojure web framework, so this post is a protest of that in a sense.
The big web framework is an idea, one where people won't reinvent the wheel over and over again, so that we can have a lot more non-web packages for Clojure. I'm quite literally saying, the effort could be better spent elsewhere.
I don't think the HN mentions of Clojure is representative of what the Clojure community thinks. If you are talking about the frontpage of HN, it is even less representative.
The Clojurians slack, Clojure Reddits and Clojurverse are more representative, and you can hardly see anything about Web frameworks in those places.
I must admit, I do not understand why (some) people seem to get so excited about the latest "Web Framework" for Clojure, over and over again.
Yes, there seem to be a lot of them but they are mostly failures -- they rarely get any traction, and the creators often quickly move on (several to other tech altogether).
I don't think those people would have directed their efforts into other things -- libraries -- to any degree that would mitigate some of the other points in your article. People tend to work on what interests them and if someone wants to design and build a web framework, persuading them not to isn't going to encourage them to work on a "more useful" library I suspect.
I think the abandoned API library problem is real but partly because they were wrong-headed in the first place: Clojure is designed as a hosted language, specifically to take advantage of the vast, mature ecosystem that already exists on the JVM. When we got started with Clojure at work over a decade ago, it was common to see "all-Clojure" as a mindset and reject interop as a solution. I think that has changed a lot over the years and people now leverage interop and Java libraries as a "first solution" these days, perhaps with a thin wrapper around those libraries just to provide a more fluid, more functional approach.
At work, we've certainly taken a conscious decision to switch away from "all-Clojure" where there are mature Java libraries that are reasonable to use via interop (unless, of course, the "all-Clojure" equivalent is very well-maintained and really adds a lot of value).
I am quite interested reading all this from both sides.. I saw a talk where Rich talked about clojure avoiding the HttpServletRequest calling it life sucking, and that one huge advantage clojure has is it's all data... but without wrappers, don't we have to then use these life sucking things? So many java api's are like this, the reason I use clojure is to not have these things, so I actually prefer the all clojure approach..
As noted elsewhere, it's not either/or here. Some wrappers are worth having and add huge value: Ring vs HttpServletRequest is a classic example, Cognitect's AWS libraries -- built on the same principle -- are another.
Wrapping Java Time, on the other hand, is a bit pointless because it's already a value-based system with a "mostly functional" approach that is straightforward to use via interop.
On the other hand, if you want to write portable -- .cljc -- code that manipulates date/time values, you _do_ want a wrapper so that your code works on both the JVM (using Java Time under the hood) and in JS (using whatever is the appropriate JS lib behind the scenes).
yes makes perfect sense everything you just said..so I suppose this is one reason why clojure doesn't have one library/framework to rule them all, Rich saying he gave us a tool to do what we want/need, and clojurists do just that! Do what makes best sense in your situation etc
unrelated sorta..
Really makes me wonder if the lisp curse is really not a curse, now that I also google it I see Eric Normand has something to say about it https://ericnormand.me/podcast/what-is-the-curse-of-lisp so maybe i should give that a listen!
This is kinda like when go developers said the go community doesn't want generics. The people that really want these things will self select out of your community.
But I was merely pointing out a false premise. The article was saying too much focus was spent on "big web frameworks" in Clojure, but the reality is that most people in the community do not care about these at all.
I would agree, except this is different, anyone can be the one to provide this framework, you don't depend on the language stewards for it.
With Generics in Go, it's more about asking for more power to the community itself to be able to extend the language.
Clojure is strange in a way how much it can be extended by the community itself.
Now, it might still self-select out people who don't want to contribute to a big web framework and just want one they can use right away. But it wouldn't self-select against people wanting to build a web framework.
> the consensus of the community seems to be that there's no need for a rails-like framework for Clojure
Maybe (I’m not really sure how you judge community consensus) but, tangibly, the problem of libraries being old, broken and unmaintained while people go off and build new things is real.
Clojure has simultaneously tried to distance itself from the JVM and but also eat the cake of “just use the Java library for that”.
I personally think the “you don’t need to use the Java runtime” has seen a de facto split in the ecosystem which is also partially to blame.
Kotlin has been very successful at what is basically the same model, and it is different how? Stronger guiding principles? Found a silver bullet use case in android apps? Benefit of strongly opinionated steering from jetbrains?
I don’t think is technically more sophisticated than clojure, but it’s definitely more successful and popular.
I think we have to face the facts; clojure is flat if not already in decline. I know, usage goes up every year in the survey, but the ecosystem isn’t healthy. Especially in the web space.
> Most of Clojurians are satisfied with what we have in the Web front
Most? Geez. I think that’s a big call. I literally only ever hear people complain about this.
I am not satisfied by the current state of affairs. :/
I think this attitude in the Clojure community is sealing Clojure's fate. You discover the incredible genius of Clojure's simplicity then hit this brick wall of intransigence when you search for a framework which will give you leverage in the business world. Sad.
Something like Phoenix? Elixir is a newer language, with a lot of similarities to Clojure. Phoenix even made the most loved web framework in the Stack Overflow 2022 survey.
Why did they converge on something so, apparently, great, in such a low amount of time? And why is this so hard for Clojure?
Not taking sides or trying to be negative, just honestly curious what the driving factor is.
I haven't used Elixir enough to be particularly confident in an opinion about it. However, Phoenix appears to have a similar design to many other MVC frameworks, and if a conventional design works for a language, there's less need to experiment. Initial development can converge around something that's tried and tested.
A conventional framework doesn't fit Clojure particularly well, so the community has been forced to tread new ground to figure out what works with the language.
Elixir is a functional language as well, and borrows inspiration from Clojure in particular.
I haven't dug into Clojure enough, but I'm curious why you think a conventional MVC-style framework wouldn't be a good fit (not that the article is making this argument).
Elixir is still Actor based though, and relies on modeling behavior through something very akin to objects. You could even call it OOP if you went back to the more original message passing/late binding roots of OOP.
Kinda? I guess it depends upon what exactly you mean. If you mean behaviours, structs, protocols, modules and maps, then I can kinda see that, although Clojure has most of these too. But if you mean actors? Well...
I view Elixir as two languages in one: an actor language, and a functional programming language. Sometimes you gotta use the actor language (processes) to model behavior, but it's far better to leave that to the FP aspects.
The problem with using processes is there are a lot of concerns that only they can handle.
To be fair, I'm only a hobbyist Elixir dev. But there's lots of misinformation about how easy Elixir makes certain things. In my experience, it's not easy to build a process tree that factors in concurrency and synchronization, much less fault tolerance on top of that. Nevermind performance and gc isolation. Adding modelling on top of that just makes the job way harder.
The only real thing processes have in common with objects is they both hold state. Messages are kinda similar to methods if you squint hard enough, but there's more differences than similarities. Everything else listed above has nothing in common with any given object in your average object oriented programming language.
> If you mean behaviours, structs, protocols, modules and maps, then I can kinda see that, although Clojure has most of these too. But if you mean actors? Well
I kind of implied all of that, but I was specifically referring to the concept of modules and all its feature-sets, which together, in my opinion, does lead to a more OO like style of programming, especially structs, processes, behaviors and use, those things I feel are really similar to classes and inheritance. And there's also the dotted syntax for both map and struct access.
While yes, it's different than your common OO, in practice, you end up using these things similarly.
In Pheonix, simply put, almost all extension points relies on the __using__ macro, __MODULE__ compile time macro and use keyword. So it ends up very similar to an OOP framework, you basically extend classes, but you do so by using `use` instead of using `extend`, and a macro that copies over code and dynamically set the module name, instead of having an inherited lookup table.
How it's achieved is very different, but as a user of Pheonix, the ergonomics end up very similar.
Look at this:
defmodule TodoWeb.ItemsController do
use TodoWeb, :controller
alias Todo.Items
def index(conn, _params) do
items = Items.list_items()
render(conn, "index.html", items: items)
end
end
Yes, this isn't exactly a class, but like:
package TodoWeb;
import Todo.Items;
class ItemsController extends TodoWebController
public index(conn, params) {
var items = Items.listItems();
this.render(conn, "index.html", items);
}
}
How similar is the user experience?
And TodoWebController similary uses Phoenix.Controller, etc. It gives you a very similar feel to an inheritance tree, and pluging-in behavior in the framework by class extension.
Namespaces in Clojure really don't have this feel to them at all, now maybe you could shenaningan something similar with some similar macro convention over namespaces in Clojure, but it definitely isn't encouraged by the language or community the way it is in Elixir.
So in general, I feel modules in Elixir have a much closer feel to them to classes/objects in OO languages, and you end up using them similarly, where-as that's not the case for Clojure namespaces.
This is a topic that I think requires a more in depth article, but I'll attempt to justify myself as best I can in the limited space of a comment.
In most languages, related data is generally represented by some manner of closed record, struct or object. In Clojure, this data is represented by open maps instead.
This distinction may seem minor or even irrelevant, but in my view it's the key difference between Clojure and other languages, and the reason why the MVC pattern has never quite fit comfortably in Clojure.
In a language that uses closed records, complex data tends to align itself in terms of fixed hierarchies. Think of how the models in Rails are typically laid out. You have a few dozen models, with a handful of fields each, arranged in some sort of static tree or graph.
In Clojure, data doesn't tend to align itself in the same way. Data doesn't need to be artificially grouped if there's no benefit in doing so; its structure can be much more amorphous.
There's a similar problem at the "view" end. REST tends to assume a hierarchy of resources, when that may not be the best way for the client to get the data it wants, or for the server to represent the data.
> Data doesn't need to be artificially grouped if there's no benefit in doing so; its structure can be much more amorphous.
I'm curious, just because Clojure offers that flexibility to a developer when writing the logic in their app, why does this flexibility need to "interfere" (for lack of a better word) with the design of a web framework?
For the same reason you wouldn't design a web framework in Ruby that doesn't use classes. Technically possible, but you'd be fighting against the design of the language.
In Clojure, this data is represented by open maps instead.
Is Clojure unique here? seems this is the norm for Javascript applications too, that are done mostly with a FP style, instead of open clojure maps you have open JS objects. Most JS SDKs and libraries I have work with is just passing JS objects between them and your code.
JS offers classes (or a similar construct pre-ES2015) and you can’t easily merge two objects (there’s the spread syntax, but that doesn’t bring along the prototype).
So you’re mostly rewarded by keeping object «types» seperate
Javascript objects have some differences to Clojure maps that make them harder to use in the same way.
For example, you don't have to store functions in objects in Javascript, but the language design encourages it. This is problematic because we can't pull data out of an object without considering the functions that rely on it.
Another issue is that keys are not unique. Two objects might contain an "id" key, for example, that identifies them in different ways. This is problematic if you want to arbitrarily merge objects together.
Idiomatic Javascript might look like this:
user.orders.total_cost()
But if we want to program in the same way as Clojure, we'd perhaps be writing something more like:
Orders.total_cost(user["user/orders"]);
Not impossible, but certainly more inconvenient.
The more you force rigid categorisations and behaviour on data, the harder it is to manipulate and organise in different ways. In-memory data is almost always ordered with a hierarchical structure, because that's what closed records gravitate toward. But if we take a look at databases, its rare to find any organised along strict hierarchical lines.
Another issue is that keys are not unique. Two objects might contain an "id" key, for example, that identifies them in different ways. This is problematic if you want to arbitrarily merge objects together.
Don't exactly understand this part, if one of your objects has its unique identifier as "userID" and the other as "invoiceID" you can merge them without conflict or what I'm missing? "id" is indeed use widely in blog examples/tutorials and even books but it's not the norm for enterprise data.
If you ensure that all object keys are uniquely named, then merging objects wouldn't be a problem. But unless you use some sort of naming convention to enforce it, the more keys you have, the more likely it is for naming clashes to occur.
Maybe it's because Clojure has typically attracted a demographic who are more shy about self-promoting and marketing their new ideas and tools. Photon is an exciting (and relevant) example defying that trend though: https://www.hytradboi.com/2022/uis-are-streaming-dags
Case in point. Ripley looks promising but it's just one dev and his single page Github doc. Useless to a newcomer or business looking for a framework for the long haul. Clojure has dozens of these projects.
In Clojure, you can make a website many other ways, use any existing Java server, implement your own server quickly, use any templating language you want, implement your own quickly, use whatever routing lib you want, implement your own quickly, etc.
Not only are there existing options from Java, but implementing your own is approachable to your average dev, and can be done in a reasonably short amount of time.
Maybe it's a bit of the Lisp curse afterall. A language too productive, that no one bothers settling for the one framework.
I also think Elixirs Ruby friendly syntax brought in a lot of Rubyists with Rails experience, and that attracted people who used Rails to Phoenix, giving it that inertia.
Chris McCord used to be a Rails developer, so he came directly from Rails, so it made sense to rebuild a similar framework for Elixir, and a lot of people followed from Rails to Elixir as a better ruby with better scaling.
Erlang is not commonly used for web applications though, with Clojure, you had existing Java shops with Java web app who transitioned to using Clojure, simply keeping their Java web framework and writing their business logic in Clojure instead.
If you adopted Elixir, you most likely were starting from scratch, a green field project. Since Erlang is already great at concurrent distributed systems, Elixir's niche became web application development, and it didn't have much existing libraries or frameworks to leverage, so I think it makes more sense for an initial popular framework to come to existence and rally others around.
It still built on Erlang fundamentals, its web server is Cowboy for example, but there wasn't much on top to help with other parts of a web app.
> In Clojure, you can make a website many other ways, use any existing Java server, implement your own server quickly, use any templating language you want, implement your own quickly, use whatever routing lib you want, implement your own quickly, etc.
The thing is, many (most?) developers are interested in building the end product, not building or heavily researching the ideal framework/library/stack itself. Without something easy to use, widely understood, and that has all the pieces hooked together, a lot of developers will get frustrated and seek different pastures so they can just concentrate on a product. That's why Rails and other similar libraries in other languages are so successful and make those languages successful, because a key layer is handled for you so you can just get higher-level work done.
I used Clojure many years full-time and liked so many things about it, but the myriad ways to do the same thing does not appeal to everyone. This also extends to other aspects of the language -- the many different development environments and tooling, hooking together all the various ways to actually write, develop Clojure code. Many developers also don't want a lot of their time spent building a development workflow.
Maybe since I consider myself more of a software engineer than a developer, I have a different viewpoint, but to me, as a SE, my job is not just to deliver the end product, but actually to produce the version of the end product that is also of as good quality as it can be within the budget and constraints of the business, as well as guarantee enough architectural runway to meet the business future goals and needs without increasing the future cost or lowering the delivery velocity over time. Often, I even contribute to what the end product should be, by consulting and informing on what can or cannot be achieved with technologies, or how by changing certain aspect of the product we can enable a lot more functionality in other ways.
That means, throwing a scaffolded template and a prefab framework might be good for an initial proof of concept and v0 testing the market launch, but with Clojure you can often deliver POCs and v0s just as fast, but then have the flexibility to grow it as it needs too, where frameworks often become big drag on what you can do and how quickly you can do so as you need to grow the product.
Basically, the inner working matters to the end product, especially when products don't have an end, but are really a lifetime of support and continued changes in feature-set and compatibility/scale.
Valid perspective for some developers, yet thousands of software engineers enjoy creative, satisfying careers building projects without concerns for the inner workings of a web library or the extra effort to engineer the stack directly. They just want to build the product using an established framework.
I totally respect that, and Clojure could invest more in offering frameworks or even no-code platforms or such features, but the truth is it doesn't. The language very much targets the people who are interested in not just the functional requirements, but also the non-functional requirements of performance, scale, architectural runway, future extensibility, operations, maintainability, correctness, re-usability, etc. Especially, Clojure targets those who believe a balance between all these and functional requirements is the holy grail. That's why it won't be the most correct, the most performant, the most productive, but a pragmatic balance of all these in almost equal parts.
Maybe it should also embrace the people looking to get a product out by simply using a framework, and I'd say there's more of that in Clojure today than ever before, but the community I think is more composed of the former people that I describe, which is why you don't see any attempted framework take hold in the community, because most current members are not in the group that "just want to build the product using an established framework".
That said, since the community is more composed of people like me, you don't see a mass move of every Clojurian switching to one of those.
So it creates some questions?
1. Is it a problem that the language targets engineers more interested in a balance between non-functional and functional?
2. Should it be mutually exclusive, or can Clojure equally serve both niche? And if so, should it, why?
3. Is the claim that you can be as productive and it is just as easy to build a product without using a framework in Clojure true? Does this apply to everyone, or only certain personalities or people with certain amount of lower level knowledge?
4. Is Clojure's marketing misleading? Are people looking to just "build the product using an established framework" mislead in thinking Clojure will offer them salvation?
5. Where do most developer fall in, if they don't fall in the category Clojure currently targets, then does that mean Clojure cannot become mainstream? To go mainstream does it mean you have to target frameworks because there are more developers looking to just make a product using a framework?
I don't have answers to these, I'm just trying to define the current state and what the problem with it might be, or if it even is a problem.
So, for example, email verification. There could be a watch that triggers when a new user is inserted into the database, and this triggers a side-effectful function that sends an email.
> Nowadays there tends to be more options, e.g. a web application may consist of a single HTML shim, with a thick React client that talks to a GraphQL back end.
Best part about Rails is that you can do this as well. Rails has done a great job of being adaptable to new web technologies.
Im sorry I couldn't get past a hypothetical Org allowing a single developer to yolo-yeet bespoke languages, frameworks, tooling, into internal-critical-business-project after internal-critical-business-project.
If your Org behaves like the above you could have the best Rails-Closure framework in the world but you're still destined for implosion.
I think this is just a cultural thing where the ecosystem prefers to compose things together. The Node ecosystem also hasn't centralized around a Rails. People tend to just use Express.
Also, the syslog server example doesn't seem like something a Rails would solve.
> can we please just get together at the next Conj conference decide what our "Rails" is going to be?
I think these days we, OP included, tend to use "Rails" to not just mean a monolithic framework, but also one that monopolizes the ecosystem which is one thing that made Rails good.
But since it's been tried before in the Clojure and Node ecosystems, maybe people just don't care for it.
The blog post mentions Elixir and while Elixir has its own mini-Rails with a lot less opinions, you'll still run into the same problems as Clojure around things like wanting a Stripe supported library, or PayPal, or Braintree, or Paddle or a ton of other popular 3rd party SAAS SDKs that exist in Ruby, Python, Node, PHP, Go, etc..
It's one of the reasons I stopped using Elixir. When I reached out to Stripe (and other 3rd party tools) they said the demand wasn't there to justify the dev time to maintain an official library internally. I think this is the reality to expect when using a more niche language. That may be ok if you're ok with that but it's important to know that's what you're getting into when you choose languages like this. It's a personal preference.
This brings up another point I hope someone tries to solve in programming. Every time a new language comes out we have to recreate millions of baseline libraries and it just sucks. As a dev I want to be able to make use of great libraries oblivious to what they are created with.
> Every time a new language comes out we have to recreate millions of baseline libraries and it just sucks. As a dev I want to be able to make use of great libraries oblivious to what they are created with.
Technically this tool mostly does exist already with the OpenAPI specification if we're talking about REST APIs. If you as the API provider put in the leg work to create a very detailed specification which is a YAML file, you can generate programming language specific SDKs out of it as long as the language has an OpenAPI library to consume this spec in an accurate way.
Stripe has publicly mentioned[0] they mostly use this spec to generate their SDKs (even as of a few years ago), but I guess auto-generated code still requires some developer time and there's a quality assurance level of "hey we're dedicated to internally supporting this". It's a huge deal having a provider internally support your language's SDK.
Honestly, as a rubyist, I more and more often run into new "SDKs" that don't have an official ruby version supported! Ruby is no longer always considered popular enough to get an SDK from a vendor.
I think the fundamental truth here is that so much of writing software these days is having open source to rely on. That applies to things like an SDK for a vendor API, but also applies to things like consolidating community effort on one approach to back-end web app -- instead of having a bunch of not really mature options, and also so effort can be consolidated on building additional things that work with that popular tool.
I don't think these two areas are actually related like the OP does -- if we can just reach consensus on a web framework, then somehow we'll get things like a Stripe SDK, I'm not sure about that.... but I think the general challenge of "how do we get a sustainable, supported, mature/polished open source ecosystem of software that we can use that works together" is kind something we don't really understand, how does that happen (or not) for a new language? I don't think it's just "well, if we only get popular enough it'll happen on it's own", and of course there is a chicken and egg issue there too.
> I think the general challenge of "how do we get a sustainable, supported, mature/polished open source ecosystem of software that we can use that works together" is kind something we don't really understand
I think one problem is people who tend to flock to niche languages like to write libraries so what happens is you get a random person on the internet who makes a library, then they eventually stop using it personally and the project dies because they were the sole maintainer championing the project. You have this happen in parallel and the next thing you know you have 3-4 libraries that are either missing big features, fall out of date, go into maintenance mode, etc.. I saw this a lot with Elixir's community libraries (Stripe, AWS SDK, pagination, etc.).
There is a chicken / egg aspect to it because I think the fix here is to eventually get popular enough where you have a large enough of a community around the language and ecosystem that the most popular projects for a specific feature get so popular that they can't die because too many people depend on it and then a bunch of folks start to contribute to it and the community informally blesses 1 solution as "the" solution. The chicken / egg problem here is that a lot of people want to use Stripe, not create a library to use Stripe, etc. so it never gets mass adopted.
I think the best way for this to happen is organically. Trying to create a committee with an org with specific contributors trying to invent solutions is going about it in the opposite way. The best libraries tend to be extracted out of real projects and grow to become popular because the person or people behind it have a skillset around not only creating high quality software but also a dedication to create great documentation, can handle support, have lots of tests and generally have created a maintainable code base that makes it easy for others to contribute.
> Ruby is no longer always considered popular enough to get an SDK from a vendor.
Do you have a bunch of examples here? I haven't come across that with the usual libs I use (Stripe, Braintree, PayPal, Sentry, Datadog, Mux, AWS, etc.).
I feel like seeing the ruby open-source-developing community shrink a bit, and formerly community-supported libraries starting to become more abandoned or have more trouble staying maintained, is giving me a different perspective on "if the language just gets popular enough this will take care of itself."
But also just seeing in general more people trying to make money with open source. I think the way open source "naturally" gets produced for free is not what it was 15+ years ago.
> Do you have a bunch of examples here
I don't recall any off the top of my head, because of course it's nothing I use, working in ruby. Most of the ones you mention have been around with ruby SDK's for over 10 years already, had them released when ruby was still at the top. And certainly an SDK is still more likely to be available in ruby than elixir or clojure. Let me see if I can find a few examples remembering the sorts of things I was seeing... OK some examples that don't have vendor-supplied ruby SDK but do for other languages:
In some cases there are third-party open source libraries available, often with limited coverage and/or kind of abandoned. Vimeo is the only one I really dug into, the third-party open source "sdks" did not seem to meet my needs or do much.
Ah, I'm surprised Slack doesn't support Ruby, but to be fair they are pretty tight on official support with only Python, Node and Java.
Fortunately there's https://github.com/slack-ruby/slack-ruby-client, but it's not official. Although it seems to be a good example of a community driven library coming together. I haven't used it personally (I never interfaced with Slack in a Ruby app) but at a glance it looks like it has really good docs, a decent amount of contributors, well maintained, etc.. If I were building some Rails app that used Slack I'd likely reach for this and not feel bad about it.
Kind of a bummer on the other 2 tho. Thankfully I wouldn't be building too many apps using those tools, but I get the point you're making. In the grand scheme of things I think this also shows how much more popular Python is than Ruby.
I'm actually surprised there's even a Java SDK for Slack -- their API is really straightforward to use via plain old HTTP (which is how we interact with it at work, from Clojure).
The whole thing with niche languages remaining niche because they don't get access to popular libraries has been a thing for a very long time. It was actually worse before OSS became dominant, because proprietary libraries generally shipped as binaries, and so you'd be limited not only to specific languages, but to specific implementations and versions thereof that'd support the ABI.
I think the thing people most struggle with in Clojure is that it looks like it should be an incredibly pure, deeply opinionated language but it’s actually extremely pragmatic. There is never going to be a long lived, vertically integrated, do everything framework for Clojure. If it was going to happen it would have already. Rails took off because burnt out enterprise Java developers wanted to quit and have fun again. Clojure happened because enterprise Java developers wanted to do enterprise Java better.
This is true, clojure is not super opinionated as people tend to think. Its nice, as you get to ramp up easily. There seems to be a steady path where you start with writing "java" in lisp and end up with pure functions and 1 reduce + something to trigger side-effects. I have worked with 4 codebases and they differed vastly.
Apples to oranges. You're comparing a framework to a language. Ruby itself was born out of the desire for developer happiness. It doesn't follow logically that momentum for a large framework could not have evolved out of Clojure's origins.
In my experience Clojure teams use high quality Java and JavaScript libraries all the time. It's more often than not a piece of cake.
'Clojure needs a rails' has come up perennially for that last 10 years. I don't disagree, as I think convergence on one web solution rather than a constant churn would have great benefits. The idea that you can't use sysdig, or whatever, because there is no Clojure library, is nonsense. Interop is good and Clojure is nicer than Java for writing Java :)
Also, you can overload functions with further aritiss in protocols just fine. Nothing to do with variadics.
I never said it can't be done. I did write this in a hurry, so I may have muddled my central thesis.
I'm arguing two things:
First, Clojure has missing or rotting libraries, requiring re-implementing logic we've all done at least once because people are fixated building the Rails for Clojure.
Second, that the business case for using Clojure becomes weaker because the lack of open-source efforts on this front (due to the 'Rails Vacuum') causes either interal library building (extra time) or, more likely, interop. Interop, to _some_ management, sounds an awful lot like using Java, but with very expensive Clojure developers to do it.
What are the "missing or rotting libraries"? Maybe a few examples might be helpful. In my over 10 years of using Clojure in production, I am not sure I have "re-implemented" a lot of logic that are not already covered by some Clojure libraries. Such examples are rare.
Sure, I have used Java interop for working with Stripe, but that's because it is futile to maintain a 3rd party Clojure library if Stripe decided not to do it themselves. These proprietary APIs are constantly changing. It is perfectly OK to do Java interop in these cases. Any niche language will have the same problem, and Clojure interop with Java is quite pleasant to use, compared with using Java itself.
As far as I can tell, using ring and related things are fairly standard on the server side, and re-frame and related things are fairly standard on the client side, so I am honestly not sure what is missing for Web development in Clojure, as Web development is the most common use case of Clojure.
It seems unlikely to me that efforts to build web frameworks are causing a decrease in time spent building wrapper libraries. Developer time (particularly, open-source developer time) isn't a homogeneous commodity; people tend to work on things that they personally need. Anecdotally I can say that if I were to cease development of my own Clojure web framework, I would not immediately start writing a Syslog library instead :).
I developed for fun in Clojure and ClojureScript probably for 3 years, it was tremendous fun and got me into Lisps. In many ways Clojure does still hit the sweet spots from a language design perspective for me.
I have revisited Clojure every now and then after and IMHO its in a weird state, too popular and successful to completely die, but also limited to reach wider adoption.
When revisiting old projects I have seen countless of Clojure dependencies having died in the meantime. I have replaced utility libraries with more recent utility libraries sometimes several times. Very annoying. Paradoxically the core API is really stable to a degree that bugs aren't fixed that are a behaviour userland code might rely on.
But its overall I think more of a symptom than a cause for frictions when developing Clojure. Community management is bad, even worse that RH nourishes this elitist aura around himself. Then the JVM integration is of course the blessing that led to Clojure's initial success but is at the same time limiting. The Java community by majority seems to gravitate towards Scala (weird scala version incompatibilities incoming, Scala DLL Hell) and Kotlin.
So overall Clojure is in a strange situation. For picking it up on-the-job its not the responsible choice probably for most situations I work in. For my private tinkering the JVM dependency feels just a little too heavyweight. I don't see that Clojure will hit an inflection point of adoption in any way in the future that would make it a viable candidate for me to use at work.
I really would hope that a Clojure-on-guile or Clojure-on-racket implementation becomes ready at some point that would make me want to switch from Scheme to Clojure for the toy projects at least.
> Paradoxically the core API is really stable to a degree that bugs aren't fixed that are a behaviour userland code might rely on.
Can you give an example? As a medium user of Clojure I struggle to think of anything in the core that could rightly be considered a "bug" and fits this description.
I assume this is on community management. I am not taking part in Clojure surveys so its fairly obvious to me why my opinion is not reflected in the community surveys.
I got to admit that maybe community management might have improved in the last few years.
It seems to come down to different views on what this means. Some people want leadership who will listen to user demands and apply pressure to the community to push them to work on things in those directions.
Clojure has more leadership along the lines of "hey, other devs, I am open sourcing this cool stuff that I think is good. Here is my philosophy on why I like these types of libraries and would love contributions if you agree with the direction. I'm happy there are other directions being taken as well, that's great. Oh, also, some non-contributors have been making demands that I spend time on their problems and my answer is a firm no."
I think it is a hugely positive aspect, but I also open source my own libraries and have the same feelings towards people who make demands of me without being willing to contribute so I'm certainly biased.
I think RH emits fun grumpy vibes, the good kind. Seasoned dev who is tired that his way of dealing with programming is not more prevalent. Very down to earth, lets just do it better.
> When revisiting old projects I have seen countless of Clojure dependencies having died in the meantime. I have replaced utility libraries with more recent utility libraries sometimes several times. Very annoying.
Yes, the "best in class" libraries change every few years in Clojure's universe, but they always bring huge improvements that require completely new APIs.
> So overall Clojure is in a strange situation. For picking it up on-the-job its not the responsible choice probably for most situations I work in. For my private tinkering the JVM dependency feels just a little too heavyweight. I don't see that Clojure will hit an inflection point of adoption in any way in the future that would make it a viable candidate for me to use at work.
I think about programming languages as tools for different bottlenecks.
– Computational: Rust
– Networking: Go
– Business logic: Clojure / Python
And for many people Python has everything that Clojure is missing: a very low barrier to enter, and a very stable and well-documented ecosystem of frameworks and libraries.
What I meant, is frameworks like Django, Flask, and PyTorch. Also, libraries like Pandas, SciPy, and NumPy. That's what Clojure is missing the most for an average software engineer: simple and easy choices, that can remain relevant for decades.
So what you mean by "stable" is in the names only, but not in substance.
What I mean by stable, is that my code remains working after I upgrade the dependencies, which is definitely NOT the case for python frameworks, but it is mostly the case for Clojure libraries.
In Clojure, Web related things have remained relevant for a long time as well. For example, I do not anticipate that the standard things such as Ring, Reagent, Re-frame, etc. are going to change any time soon.
If you have not used Clojure, you just have not experienced what a stable ecosystem looks like. You have too used to changing your own code just because of a dependency bump. This is very rare in the Clojure world, but a norm outside.
Yes, these are two different meanings of "stable".
In Clojure, "stable" means that your code remains working after the upgrades of its dependencies.
In Python, "stable" means that the dependencies themselves remain upgraded.
I am not suggesting that one thing is better than the other. What I meant, is that one thing is more attractive than the other for an average developer.
Currently, there are very few Clojure libraries from a decade ago that are still being actively maintained today.
Clojure itself is not much more than a decade years old. Even that, there are quite many libraries that have stayed all these year, e.g. the standard things I mentioned, such as Ring, Reagent, etc.
As I have already mentioned, "actively maintained" is not a requirement for many Clojure libraries. Many are just done, as in "nothing needs to be changed".
As long as they are still working, why do you want to update them? As I mentioned, many production code bases in Clojure contain libraries that have not been updated for ages, but they still work. Why fixing things that are not broken?
Do you want to back that up with something else than the Python2-to-3 migration? Python 3.4 was released in 2014 and ever since stability was never an issue for me.
What exact does it mean by "died"? It does not work any more? Very unlikely.
In the Clojure world, it is the norm that a 5 years old library works as well as a one-week old one.
If you mean "died" by "not updated any more", sure that's the norm in Clojure as well. Some libraries are done, and some authors moved on. But most likely, these libraries can still be used just fine.
My production code base contains plenty of libraries that are not updated for ages, but I consider that a benefit, not a fault.
From everything I've seen these past years, Scala is on the same path to extinction as Clojure is.
Java keeps getting more popular, pushed by the very solid innovations that Oracle has been trickling into it every release, and Kotlin has certainly picked up a lot of momentum these past five years.
240 comments
[ 3.5 ms ] story [ 248 ms ] threadIMO some programmers pursue purity (no pun intended) over pragmatism, at their own expense. I understand the appeal, it's like hand planing a board with your vintage No. 8 instead of sending it through the Powermatic.
You (at least should be) picking oddball stuff like Clojure because the unique advantages outweigh the problems. If you're just using it as a wrapper over another language that doesn't provide those advantages, you're taking the balls off the bull, and you probably should have just used Java, C#, Go, etc.
Try doing that in some other languages were your underlying platform has even less libraries than a JVM...
It might be more natural than going down to Java because you’re mostly dealing with immutable data and Elixir is very consciously building on top of Erlang. In comparison, Clojure has always struck me as wanting to build on top of JVM but being an entirely different thing.
It's just that there are even _less_ library to call :P
The comparison to Elixir got me really thinking. I appreciate your "call to action", but community consensus gathering wasn't necessary for Phoenix to emerge as an option-of-choice, it gained traction because it is _good_.
Surely there are large organizations using full stack Clojure, similar to the Lawrence World-Journal. Is there something about LISP that makes it hard for them to abstract their "framework" bits into an open source package?
EDIT: I am currently working (part time) with an all in Common Lisp company. It is really nice to see a group of people who all love a programming language using it. As much as I like virtually ALL Lisp languages, I do sometimes feel guilty not just using Python when there are much better libraries available (e.g., machine learning, linked data, etc.). When a programming language has 1000x as many developers you simply get a richer ecosystem.
Interesting comment given the article. That's the magic of Clojure being hosted, you get the rich Java ecosystem and get to use your Lisp that you love.
The downside is you still need to deal with the rich ecosystem that isn't as nice as if there was a rich one in Clojure.
It will be interesting to see what languages win out long term. I really don’t like programming with Rust but it is becoming wildly popular. I do like Swift, the language is intuitive to me, has good REPL support, and has awesome standard libraries for machine learning and just about everything else.
All that said, I am an old man, I like Lisp languages, and that works for me.
No. I don't know what else to tell you.
Some people (a small minority) are trying to "solve" this, but it garners little interest since most people don't see any problem.
I think it also because there's two camps of elixir devs - those who came from Ruby and those who came from Erlang. The Rubyists have a fairly easy time being productive quickly with Phoenix because it (at a glance) is very similar to Rails. From there you learn about supervisor trees, genserver, etc.
There's a fundamental model shift that's not as compatible to frameworks.
Lisp optimizes for it to be really fast to implement perfectly tailored custom solutions, with your own DSL (domain specific language) that best fit your problem. Iterations are quick, development is fast, you can automate away repetitions and ceremony trivially.
The need for scaffolding and for complicated code-gen, which is half of what most web-frameworks provide, is just not there, because Lisps are already meta-programmable.
The other thing is Clojure's functional nature. Objects and Actors and Modules, these things naturally expand to become Singleton state or behavior containers that you call upon. They also naturally lend to a plugin architecture, where configuration setups a chain of implementing interfaces called into by standard modules from the framework, etc.
The functional, higher order, immutable, and data driven nature of Clojure doesn't fit well here either.
What happens instead is that libraries, not frameworks, tend to work better in Clojure. Things you can call and use as you want, instead of things that call you and ask of very specific extension and configuration from your part.
So the Clojure web stack is still full of reusable code, they're just exposed as libraries.
When you know what to do, you can easily put together a Clojure website together, in no less time then it takes using a framework in another language, and you similarly reuse almost as much code as in any other language.
The difference is that because you need to use the libraries, you also need to figure out how to use them and where and what to do to glue all of that stuff together.
In a framework, you can't choose how to glue them all, it's forced on you, that's both the issue with frameworks, and the benefit. So when you don't know how to build a website, a framework really helps, it guides you, and you just follow its strict structure and plug what needs to be plugged, and configure what needs to be configured.
With the library approach, that's left up to you, much harder to get started with, but not any less productive ounce you know.
Finally, a framework forces everything to be a monolith, that is built and designed within the rules of the framework itself. All behavior you'd want to reuse needs to be framework aware and compatible. This means that slowly people grow an ecosystem of compatible libraries all around the framework.
Again, with the library approach, the various libraries you choose to use were all independently designed and built, they weren't built to work together, so again, you have to find how to glue and connect them and map between them yourself.
You could still have a framework, but the mental model a framework follows is often contrary to the mental model Clojure enables, people often choose Clojure because you can quickly customize things to your liking and connect various independent building blocks together in the way you prefer and that works best for you.
That means whenever someone made their code structure available to others as a framework, everyone else went, I don't like the way you did it, so nevermind, I'll just use my way.
If the libraries all only worked with their structure though, it be too much effort to go your own way, but not so in Clojure, because all the building blocks are exposed as libraries, so you can easily structure it however you prefer.
I don't have a lot of experience with Clojure, but speaking as someone who uses Kotlin on the server side, this attitude strikes me as odd. Most of the reason to use a JVM language is to take advantage of the mature and well-maintained Java libraries for basically everything.
If interop is considered a last resort (to the point where you reach for a 3-year-old unofficial Stripe library over the officially maintained Java one), what's the point of running on the JVM at all?
I am not sure if this really answers your comment, but I really liked his comment because I tend to write wrappers and re-evaluated my habits.
But if Alex said that, then maybe it's not. ¯\_(ツ)_/¯
Notably, Alex didn't write this library and only has a minor contribution to it.
1. Higher order usage. When functions would really benefit from being used in a higher order way, passed around as values, you'll see some of the Java methods wrapped so they can more easily be used as such, since you can't use Java methods in a higher order way. This would count as "can do it better", and it's the case for clojure.string.
2. For portability with Clojurescript. This is a more recent thing, but as Clojurescript usage grows, and code reuse between backed and frontend more common, there are certain basic methods Java interop was totally fine with that now benefits from being wrapped. This is the case for clojure.math.
Again, I am curious where you get all these impressions that are contrary to most people in the community take for granted, such as:
* Clojurians prefer libraries over framework, but you said we spent too much effort in developing "big web framework".
* Clojure is intended to be a hosted language, so seamless interop with the host is the norm, but you thought interop is a bandage until Clojure matures.
It's very interesting.
https://clojure.org/about/rationale > Customers and stakeholders have substantial investments in, and are comfortable with the performance, security and stability of, industry-standard platforms like the JVM. While Java developers may envy the succinctness, flexibility and productivity of dynamic languages, they have concerns about running on customer-approved infrastructure, access to their existing code base and libraries, and performance
> symbiotic with an established Platform
> VMs, not OSes, are the platforms of the future, providing: [...]
https://clojure.org/about/jvm_hosted > Clojure is designed to be a hosted language, sharing the JVM type system, GC, threads etc.> Clojure is a great Java library consumer
Where Clojure had nothing to bring, nothing was implemented, because interop is considered just as good, that's why there's not a lot of IO functions in core, and why math was missing for a long time.
I also use a pretty half baked wrapper someone wrote because I just about lost my mind trying to make sense of the class hierarchy it was wrapping.
I think for people like Rich and Alex wrappers may look pointless because they already have invested in learning many of these APIs, know the patterns already, etc. For people like me from different language backgrounds they keep me from having to learn APIs with mutable state and deep class hierarchies and to me that’s a big win.
(I rewrote this whole comment to hopefully be more concise.)
Your main point about interop as a barrier to non-Java people is reasonable -- although if you learn a hosted language, you're going to have to learn stuff about the ecosystem on which it is hosted (so, not Java itself, but some of the JVM stuff and the docs for any libraries you interop with). And this is true of the JS ecosystem supporting ClojureScript (and the .NET ecosystem supporting ClojureCLR).
You’re right about needing to learn some of the host in a hosted language. I’ve enjoyed for example Jsoup and I use some of the Math stuff inline with no issue. Even with the java time wrapper I’ve dropped down to the original to do certain things involving locales. I definitely knew going in to Clojure I’d need to pick up some Java.
Where I really like wrappers is where the underlying java API is not great in some way. (Which now that I’ve seen the original puredanger quote seems to be what he’s saying.)
I thought this at first, but then I thought if you take Python, which is hosted on the OS directly through CPython, so we can say Python is hosted on Windows or Linux.
Well it turns out, most people never need to interop directly with Windows or Linux APIs or any other C interop.
Python seems to have a different mindset here, where most of its value proposition are providing wrappers over C/Windows/Linux/Mac APIs.
Why does so much effort get put in maintaining all these wrappers in Python? Is it just attitude? Is it that those APIs are much worse then Java's enough that even C programmers using Python want to wrap them over using them directly?
Python numeric performance is a really interesting case. Those folks would certainly drop down to C. But my casual observation is that they would build the bridge when they came back above water. A lot of work went into this because Python performance has historically been quite bad for a Data Science/ML language.
Part of this is the size of the community.
But what is we also consider the Common Lisp community? Common Lisp is incredibly performant. A lot of work went into the underlying C libraries. Perhaps most of that was when the community was more vibrant. But still, a casual observation is that CL folk live in CL.
I just don't know how to address this problem. It seems the people in a position to build good wrappers or Clojure implementations are also the people who know Java and Clojure, and maybe came to Clojure from Java. For them, it's easy to interop directly, so they take the path of least effort, and don't bother making a Clojure lib.
In a way, if Clojure couldn't use Java, while it would mean I probably wouldn't be able to use it at my work and would be forced to still work in Java, it would also mean maybe for my personal stuff or for certain companies who'd embrace it anyways, I would and they'd be forced to go ahead and put the effort to build Clojure variants of all these libraries, and that would make it friendlier to people learning Clojure that don't know Java.
I have to admit, I always react a bit defensive on articles like this one, and it's mainly because I don't see the problem, I see people complaining about things that are non-issue for me. But your point hits me in a different way, you make a good point, having to learn a language and a half (all of Clojure and half of Java) is definitely not as good as it should be.
I think normally people come at it like saying how interop is bad, difficult, not idiomatic, or that there's no libraries so they had to use Go-lang, where there were clearly options to use a Java lib from Clojure. But if the article just said, hey, using Java is unapproachable to a lot of folks, learning to understand Java APIs and semantics is a lot of work, and it's turning people away from Clojure otherwise, now that's something worth discussing.
I'm aware. I was expressing an opinion to a twitter friend, but I should have foreseen the replies of some people taking my opinions as me promoting fact. That's my fault for not being clear.
The pivotal moment for me with this was when dealing with Kafka, just using the “raw” Java library ended up being so incredibly much simpler than any of the wrapper libraries.
Where there weren't good wrappers, just a few functions "wrapping" Java libraries for my project's needs always felt lightweight and lispy. The little internal APIs I made almost always passed around the library's raw Java objects, but with more Clojure idiomatic ways to construct them, or slapping some seq on the library outputs.
For example you hand craft a wrapper for lib-version1. lib-version2 is released but you do not need it any more. So wrapper is outdated.
I prefer recipes to call other language code (or framework) instead of wrappers.
E.g. Call a C library from Dlang instead of waiting??? for some wrapper.
Still, I think you are undervaluing the JVM itself, which is almost magical when it comes to production observability. It'd be a whole lot of work to come anywhere near that tooling support for ops if you were avoiding the JVM. Scala, for instance, can also transpile into Javascript if you feel like it, and there's a native project that compiles to LLVM, but the JVM has the vast majority of the use cases, and it's not due to the libraries.
That said, the example in the article is the Stripe API, and I cannot see a good reason not to just use the official Java one. I've used it a ton, and it's as simple as can be—you pass a map in and get back an object. If that's a problem, either Clojure has a way worse interop story than necessary or the author is suffering from a variant of not-invented-here syndrome.
> My theory is, this causes a lack of effort expended on mundane libraries. It's not the most fun thing to build. I get it. Really, I do, but lack of these API-wrapping and protocol-handling libraries put Clojure at a significant disadvantage in the business of software.
The kinds of libraries the author misses in Clojure exist in abundance in Java, and the most likely explanation for their absence in Clojure is that most Clojure developers don't avoid interop as much as the author thinks they do.
That stuff removes 90% of the arguments against using another language in a Java shop.
> I dislike writing Java/C# as much as anyone, and Clojure is my ticket to writing much less of it, while leveraging the efforts of multitudes. As I've said in my talks, most Clojure users go from "eww, Java libs" to "ooh, Java libs", leveraging the fact there there is already a lib for almost anything they need to do. And using the lib is completely transparent, idiomatic and wrapper free. OTOH, if they want to customize their interface to a lib, it is trivial to do so in Clojure.
> If, however, the stigma of Java extends to libraries written in it, even when consumed from languages like JRuby and Clojure, and people think it's cooler to reinvent than reuse, they need a new definition of cool.
[0] https://groups.google.com/g/clojure/c/W_FTvyX6swY/m/UDhgIbkh...
There are a few challenges you run into:
1. Many libs written in Java follow the worst of Java's design patterns and idioms. Rather than calling a function and passing it some data as an argument, you instead have to instantiate a WhateverClient that itself takes a WhateverThingyStrategy instance and a WhateverConfig class instance, then call a .executeCommand method with a new IWhateverCommand anonymous class instance that implements the execute, doCatch, and doFinally methods. And of course, you can't forget whatever subclass of AbstractOrigamiWhateverResponseMarshaller you need next. Oh, and because all these instances need to be defined as beans that will be DI-friendly later, there's even more boilerplate to look forward to.
2. In comparison to languages like Scala, Java's type system is often quite lacking. You can never trust that a non-Option type is not null.
3. If the rest of your code is free-floating with structural types in some other language, you inevitably need to write a ton of glue code for adapting these structural data type objects to whatever blessed POJO interface Java expects.
All that being said, I still would prefer to use the more mature Java lib if the more native choice isn't as well maintained. But there's no denying that it's painful. The ergonomic hit is just as bad (if not more than) execing a child process to run code in another well-suported target like Rust/Go/Node/whatever that maybe has more sensible idioms. There are of course a lot of reasons why that'd be a bad idea, but still...
Come on, we both know this is false and it is only your feeling, nothing factual. How is IPC with all of its intricacies anywhere close to as ergonomic as calling one more method in the host language when all of your objects are also of said host language?
Frankly enough, creating a wrapper should not be hard at all, no matter the language. Especially not for libs that are mostly unidirectional in communication.
Oh, it's definitely not false. The reality is you can't just call one more method in many cases. There is a ton of ceremony involved because enterprise quality coding[0] is often seen as a desirable attribute of the software written in Java. The linked example is an exaggeration, but it's sadly not too far off from reality in some cases.
I'd also counter that IPC is often dead-simple in many contexts. Sure, a long-lived process like a server might have intricacies you need to consider, but outside of that, it's pretty straight-forward. CLI tools are another example of this, but the JVM startup times generally make it a bad choice for CLI tools to begin with.
> Frankly enough, creating a wrapper should not be hard at all, no matter the language. Especially not for libs that are mostly unidirectional in communication.
This depends on the surface area of the library you're using. And maintaining such a wrapper is not a one-off activity. It's not an issue of being "hard", but tedious and time-consuming.
[0]: https://github.com/EnterpriseQualityCoding
The rest of your comment is fluff around this admission that the person you are responding to is correct. I'm sure you're aware this is the primary use-case for Clojure.
Suzi may need to wait to do this until after her current gig has won the Startup Lottery. Or perhaps she may need to quit her current gig and figure out how to get a bunch of other Clojure-lovers to pay her rent while she does this. Good luck.
If you want Clojure to thrive again you’ll have to stop the attrition of its developer community. Maybe that’s by building a Rails but even your example in-article where Go is more suitable isn’t really a web type problem. What problems can Clojure solve that other languages can’t, in a way that is worth the effort for someone new to the ecosystem?
Erlang was created by Ericson for use in telecom systems. (OTP stands for Open Telecom Platform) which gives credence to the idea that Erlang didn't need a Rails. Once Elixir came along, being a bit more general purpose, people started building those things.
Clojure, from the beginning, was intended to be a general purpose language, not a web language. So I think it would be fine to expect Clojure to be able to solve non-web type problems. Maybe the community has attrition, but I don't think so. The Clojurians slack seems bigger than it ever was, more companies are using Clojure than I've ever seen.
edit: general not generate
I agree with the argument that Clojure was intended to be a general purpose kind of language, but I think you article also suggests that -- unless you're tied to a JVM model _and_ you refuse to use Java -- that Clojure gets out-competed by languages like Go in that kind of work. It's not clear to me how or why building a Rails kind of thing for Clojure would reverse this situation. I think, also, the size of an online community and tally of companies can be misleading. Erlang's IRC and company use was at an all-time high even while the actual pool of developers had reached its peak and was declining, slowly but surely. It's argument from analogy, sure, but what you're describing in the article for Clojure looks damn familiar from my time with Erlang, pre-Elixir.
i don’t think that is strictly true, clojure is for “situated programs” which is basically a perfect description of cloud information systems and user interfaces.
immutability and macros is not what you want for games, for example. and data science is very SQL/struct oriented, it doesn’t hit clojure’s high notes. mobile is very platform interop heavy and best done natively without adding intermediate abstraction layers. databases are written in lower level langs for performance (Datomic is not known for lightning fast queries). what’s left?
So there is an imbalance there
On one hand I'm kind of happy there's no one true way in Clojure to do web dev, because as I think Rich said, it's not a solved problem
So if we couple too heavily to one opinion on web dev then it's going to fail us as a community on a big scale and be hard to move away from
Sets of libraries are typically easier to swap out
In reality I think most Clojure businesses end up on reagent or and reframe and they are the defacto ways of doing web dev
For "solved" problems like stripe calls it would be nice if we had more people working on maintaining libraries but as a Clojurian myself I know I wouldn't do that myself so I only have me and people like me to blame
I'm quite the opposite. I love Clojure, just because it is "boring". Maybe not "boring" as in well-known, but "boring" as in the core language and it's runtime doesn't change and is stable, like really stable. I've upgraded projects using "ancient" Clojure versions (released +5 years ago) and not having to almost do any changes, as both the language, the runtime and most libraries are so stable over time. It simply just works. If people want to introduce breaking changes, they tend to prefer to create entirely new libraries instead.
I thought about touching on this in the article. Re-frame and reagent have become the default for Clojurescript development, but I have no idea how.
Last year, I dipped back into ClojureScript and built some simple projects with re-frame and I really liked it. I can definitely see why re-frame has become so popular.
I kind of have the opposite opinion. I worked on a large Clojure codebase for several years. There was so much effort put into building tooling and libraries in Clojure because the ecosystem isn't huge, (and some of its bigger pieces are just a misadventure, ie spec). It was really frustrating to me because I felt like we were spending 50% of our time writing inferior versions of code that Python, Ruby, or even JavaScript already had as libraries. I'd have preferred that time to be spent on interesting things like business concerns. Of course, you can use java interop, but it's much more natural in Scala and Kotlin.
I can hardly recall a case where I could not find a Clojure libraries for. The only exception would be wrappers for proprietary APIs, such as Stripe.
- unit tests with ephemeral dbs
- db migrations
- ORM library
Many of these things exist in Java and in retrospect, were probably the way to go. One other pain point was that Clojure libraries would stop being updated, lag too far behind the java libs, or lack features. I remember this being the case with Google Cloud and AWS, and I think Elastic Search. I remember a number of times just completely swapping out the Clojure wrapper lib for the java lib.
- unit tests with ephemeral dbs: we use com.opentable.components/otj-pg-embedded for postgres.
- db migrations: there are Migratus and many others, we also just use flyway in app startup shell script
Some are not relevant if you write proper Clojure.
- ORM library: I am not sure you need these in Clojure. Clojure is data-oriented, not object-oriented, so you don't need to map data to objects. Clojure is good at transforming data. Just keep data as data.
I don't recall us spent too much effort in building Clojure libraries of our own for something missing.
For my over a decade of Clojure career, I only built two Clojure libraries of my own. Both were not missing, just that I didn't like the existing ones. One is a Clojure data diff library (Editscript), and another is a database (Datalevin).
In general, if one cannot find a Clojure library (very rare), we tend to use Java interop. As mentioned repeatedly, Clojure is a hosted language and we embrace the platform.
But API wrappers handling things like retries, pagination, client-side business logic, and coercion from JSON. Having to account for all of that for two API calls is somewhat tedious and repetitive, and I'm sure we've all solved those problems at least a dozen times.
Every API has its quirks, sometimes returns different data, has edge cases for certain calls, has different error reporting patterns, rate limiting, authentication methods, sometimes has intended or unintended breaking changes, ....
Sure, you can usually just manually whip up the code for an endpoint or two in a few minutes , but then you need to continually add endpoins, fix the edge cases, adapt to upstream changes, ...
Good bindings make your life a whole lot easier.
It’s nothing specific to the language or framework.
I'm not advocating for a Clojure Rails because I love Rails. I've never used Rails to be honest. I'm arguing open-source efforts are repeatedly being spent on trying to build the next web framework/library/toolkit (Rails) for Clojure and not much else, so it would be great if there was one, so we can get on with filling the gaps in the Clojure ecosystem.
That's contrary to what most of us know. There are some efforts in the Web front, but not much. At least, the community is not paying much attention to these efforts.
Let's look at the list of community funded projects, e.g. those in Clojure Together: the only Web related projects funded were clj-http in 2018, ring, re-frame and reagent in 2020. None of these are Web frameworks, and the rest of the funded projects are not Web related at all.
I’m not sure what that means exactly, but no one is using Rails only for building out micro services. They’re using Rails to go from nothing to a production ready web application, with all the incidental complexity taken care of, in a very short time.
It’s been a few years since I was involved with Clojure ecosystem - what is the Clojure experience equivalent to say, the original Rails demo from back in 2005? The last I tried, all the parts seemed to be there, but much painful assembly/“composing” was required and not all the parts fitted which ended up producing a lot of awkward complexity needing desperate decomplecting.
I also learned that trying to second-guess why developers spend their time in certain libraries and not in other endeavors (like frameworks) comes across as pretty disrespectful and actually ignorant (considering that those people are very often more skilled/experienced than most of us).
(Good related read: Open Source Is Not About You)
So I'd recommend to anyone, open your mind, be ready to have some preconceptions changed, and contribute a significant deal to (Clojure) OSS to truly appreciate what we have. And write more interop ;)
The efforts made so far in this space have taken many different approaches, because they're all experiments. Explorations of what an idiomatic Clojure web framework might potentially look like. Clojure has been around for over a decade now, and it's fairly mature as a language; but it's ideology is young in comparison to more established paradigms like OOP.
To make matters more complex, Rails came to prominence in an era where web applications were mostly of one type: server-side HTML with a sprinkling of JavaScript. Nowadays there tends to be more options, e.g. a web application may consist of a single HTML shim, with a thick React client that talks to a GraphQL back end.
So on the one hand we have a language that's still exploring its identity; on the other we have a rapidly changing idea of what a web application should be.
My guess is that an idiomatic Clojure web service would look like a client-accessible relational database with a strong security model and datalog queries, along with some system for adding in side-effectful hooks to respond to data change. In other words, something rather different in design to Rails.
I agree with everything you said.
To clarify, I'm not advocating to re-create Rails for Clojure, rather, I'm arguing open-source efforts are too focused on the search for Clojure's Big Web Framework TM. However, I do like your guess as to what a Clojure Web service might look like.
The big web framework is an idea, one where people won't reinvent the wheel over and over again, so that we can have a lot more non-web packages for Clojure. I'm quite literally saying, the effort could be better spent elsewhere.
The Clojurians slack, Clojure Reddits and Clojurverse are more representative, and you can hardly see anything about Web frameworks in those places.
Yes, there seem to be a lot of them but they are mostly failures -- they rarely get any traction, and the creators often quickly move on (several to other tech altogether).
I don't think those people would have directed their efforts into other things -- libraries -- to any degree that would mitigate some of the other points in your article. People tend to work on what interests them and if someone wants to design and build a web framework, persuading them not to isn't going to encourage them to work on a "more useful" library I suspect.
I think the abandoned API library problem is real but partly because they were wrong-headed in the first place: Clojure is designed as a hosted language, specifically to take advantage of the vast, mature ecosystem that already exists on the JVM. When we got started with Clojure at work over a decade ago, it was common to see "all-Clojure" as a mindset and reject interop as a solution. I think that has changed a lot over the years and people now leverage interop and Java libraries as a "first solution" these days, perhaps with a thin wrapper around those libraries just to provide a more fluid, more functional approach.
At work, we've certainly taken a conscious decision to switch away from "all-Clojure" where there are mature Java libraries that are reasonable to use via interop (unless, of course, the "all-Clojure" equivalent is very well-maintained and really adds a lot of value).
You're probably right about people working on what interests them.
I may have started Clojure during the "all-Clojure" years and failed to recognize the shift.
Wrapping Java Time, on the other hand, is a bit pointless because it's already a value-based system with a "mostly functional" approach that is straightforward to use via interop.
On the other hand, if you want to write portable -- .cljc -- code that manipulates date/time values, you _do_ want a wrapper so that your code works on both the JVM (using Java Time under the hood) and in JS (using whatever is the appropriate JS lib behind the scenes).
unrelated sorta..
Really makes me wonder if the lisp curse is really not a curse, now that I also google it I see Eric Normand has something to say about it https://ericnormand.me/podcast/what-is-the-curse-of-lisp so maybe i should give that a listen!
But I was merely pointing out a false premise. The article was saying too much focus was spent on "big web frameworks" in Clojure, but the reality is that most people in the community do not care about these at all.
With Generics in Go, it's more about asking for more power to the community itself to be able to extend the language.
Clojure is strange in a way how much it can be extended by the community itself.
Now, it might still self-select out people who don't want to contribute to a big web framework and just want one they can use right away. But it wouldn't self-select against people wanting to build a web framework.
Maybe (I’m not really sure how you judge community consensus) but, tangibly, the problem of libraries being old, broken and unmaintained while people go off and build new things is real.
It’s also undeniable that (see https://www.reddit.com/r/Clojure/comments/w9roqv/is_there_an... for example) that there are tonne of half baked efforts in the web space.
Maybe they’re unrelated.
There are lots of argument you could make;
Clojure has simultaneously tried to distance itself from the JVM and but also eat the cake of “just use the Java library for that”.
I personally think the “you don’t need to use the Java runtime” has seen a de facto split in the ecosystem which is also partially to blame.
Kotlin has been very successful at what is basically the same model, and it is different how? Stronger guiding principles? Found a silver bullet use case in android apps? Benefit of strongly opinionated steering from jetbrains?
I don’t think is technically more sophisticated than clojure, but it’s definitely more successful and popular.
I think we have to face the facts; clojure is flat if not already in decline. I know, usage goes up every year in the survey, but the ecosystem isn’t healthy. Especially in the web space.
> Most of Clojurians are satisfied with what we have in the Web front
Most? Geez. I think that’s a big call. I literally only ever hear people complain about this.
I am not satisfied by the current state of affairs. :/
Why did they converge on something so, apparently, great, in such a low amount of time? And why is this so hard for Clojure?
Not taking sides or trying to be negative, just honestly curious what the driving factor is.
A conventional framework doesn't fit Clojure particularly well, so the community has been forced to tread new ground to figure out what works with the language.
I haven't dug into Clojure enough, but I'm curious why you think a conventional MVC-style framework wouldn't be a good fit (not that the article is making this argument).
I view Elixir as two languages in one: an actor language, and a functional programming language. Sometimes you gotta use the actor language (processes) to model behavior, but it's far better to leave that to the FP aspects.
The problem with using processes is there are a lot of concerns that only they can handle.
To be fair, I'm only a hobbyist Elixir dev. But there's lots of misinformation about how easy Elixir makes certain things. In my experience, it's not easy to build a process tree that factors in concurrency and synchronization, much less fault tolerance on top of that. Nevermind performance and gc isolation. Adding modelling on top of that just makes the job way harder.
The only real thing processes have in common with objects is they both hold state. Messages are kinda similar to methods if you squint hard enough, but there's more differences than similarities. Everything else listed above has nothing in common with any given object in your average object oriented programming language.
I kind of implied all of that, but I was specifically referring to the concept of modules and all its feature-sets, which together, in my opinion, does lead to a more OO like style of programming, especially structs, processes, behaviors and use, those things I feel are really similar to classes and inheritance. And there's also the dotted syntax for both map and struct access.
While yes, it's different than your common OO, in practice, you end up using these things similarly.
In Pheonix, simply put, almost all extension points relies on the __using__ macro, __MODULE__ compile time macro and use keyword. So it ends up very similar to an OOP framework, you basically extend classes, but you do so by using `use` instead of using `extend`, and a macro that copies over code and dynamically set the module name, instead of having an inherited lookup table.
How it's achieved is very different, but as a user of Pheonix, the ergonomics end up very similar.
Look at this:
Yes, this isn't exactly a class, but like: How similar is the user experience?And TodoWebController similary uses Phoenix.Controller, etc. It gives you a very similar feel to an inheritance tree, and pluging-in behavior in the framework by class extension.
Namespaces in Clojure really don't have this feel to them at all, now maybe you could shenaningan something similar with some similar macro convention over namespaces in Clojure, but it definitely isn't encouraged by the language or community the way it is in Elixir.
So in general, I feel modules in Elixir have a much closer feel to them to classes/objects in OO languages, and you end up using them similarly, where-as that's not the case for Clojure namespaces.
In most languages, related data is generally represented by some manner of closed record, struct or object. In Clojure, this data is represented by open maps instead.
This distinction may seem minor or even irrelevant, but in my view it's the key difference between Clojure and other languages, and the reason why the MVC pattern has never quite fit comfortably in Clojure.
In a language that uses closed records, complex data tends to align itself in terms of fixed hierarchies. Think of how the models in Rails are typically laid out. You have a few dozen models, with a handful of fields each, arranged in some sort of static tree or graph.
In Clojure, data doesn't tend to align itself in the same way. Data doesn't need to be artificially grouped if there's no benefit in doing so; its structure can be much more amorphous.
There's a similar problem at the "view" end. REST tends to assume a hierarchy of resources, when that may not be the best way for the client to get the data it wants, or for the server to represent the data.
I'm curious, just because Clojure offers that flexibility to a developer when writing the logic in their app, why does this flexibility need to "interfere" (for lack of a better word) with the design of a web framework?
Is Clojure unique here? seems this is the norm for Javascript applications too, that are done mostly with a FP style, instead of open clojure maps you have open JS objects. Most JS SDKs and libraries I have work with is just passing JS objects between them and your code.
So you’re mostly rewarded by keeping object «types» seperate
For example, you don't have to store functions in objects in Javascript, but the language design encourages it. This is problematic because we can't pull data out of an object without considering the functions that rely on it.
Another issue is that keys are not unique. Two objects might contain an "id" key, for example, that identifies them in different ways. This is problematic if you want to arbitrarily merge objects together.
Idiomatic Javascript might look like this:
But if we want to program in the same way as Clojure, we'd perhaps be writing something more like: Not impossible, but certainly more inconvenient.The more you force rigid categorisations and behaviour on data, the harder it is to manipulate and organise in different ways. In-memory data is almost always ordered with a hierarchical structure, because that's what closed records gravitate toward. But if we take a look at databases, its rare to find any organised along strict hierarchical lines.
Don't exactly understand this part, if one of your objects has its unique identifier as "userID" and the other as "invoiceID" you can merge them without conflict or what I'm missing? "id" is indeed use widely in blog examples/tutorials and even books but it's not the norm for enterprise data.
Also relevant as a Phoenix-like alternative for Clojure: https://github.com/tatut/ripley
In Clojure, you can make a website many other ways, use any existing Java server, implement your own server quickly, use any templating language you want, implement your own quickly, use whatever routing lib you want, implement your own quickly, etc.
Not only are there existing options from Java, but implementing your own is approachable to your average dev, and can be done in a reasonably short amount of time.
Maybe it's a bit of the Lisp curse afterall. A language too productive, that no one bothers settling for the one framework.
I also think Elixirs Ruby friendly syntax brought in a lot of Rubyists with Rails experience, and that attracted people who used Rails to Phoenix, giving it that inertia.
Chris McCord used to be a Rails developer, so he came directly from Rails, so it made sense to rebuild a similar framework for Elixir, and a lot of people followed from Rails to Elixir as a better ruby with better scaling.
If you adopted Elixir, you most likely were starting from scratch, a green field project. Since Erlang is already great at concurrent distributed systems, Elixir's niche became web application development, and it didn't have much existing libraries or frameworks to leverage, so I think it makes more sense for an initial popular framework to come to existence and rally others around.
It still built on Erlang fundamentals, its web server is Cowboy for example, but there wasn't much on top to help with other parts of a web app.
The thing is, many (most?) developers are interested in building the end product, not building or heavily researching the ideal framework/library/stack itself. Without something easy to use, widely understood, and that has all the pieces hooked together, a lot of developers will get frustrated and seek different pastures so they can just concentrate on a product. That's why Rails and other similar libraries in other languages are so successful and make those languages successful, because a key layer is handled for you so you can just get higher-level work done.
I used Clojure many years full-time and liked so many things about it, but the myriad ways to do the same thing does not appeal to everyone. This also extends to other aspects of the language -- the many different development environments and tooling, hooking together all the various ways to actually write, develop Clojure code. Many developers also don't want a lot of their time spent building a development workflow.
That means, throwing a scaffolded template and a prefab framework might be good for an initial proof of concept and v0 testing the market launch, but with Clojure you can often deliver POCs and v0s just as fast, but then have the flexibility to grow it as it needs too, where frameworks often become big drag on what you can do and how quickly you can do so as you need to grow the product.
Basically, the inner working matters to the end product, especially when products don't have an end, but are really a lifetime of support and continued changes in feature-set and compatibility/scale.
Maybe it should also embrace the people looking to get a product out by simply using a framework, and I'd say there's more of that in Clojure today than ever before, but the community I think is more composed of the former people that I describe, which is why you don't see any attempted framework take hold in the community, because most current members are not in the group that "just want to build the product using an established framework".
I think the community has settled, ounce again, on a bit of a balanced approach, Kit (https://github.com/kit-clj/kit) and Edge (https://github.com/juxt/edge) are such hybrids. And some more direct viable frameworks have come along like Biff (https://biffweb.com/) and Fulcro (https://fulcro.fulcrologic.com/).
That said, since the community is more composed of people like me, you don't see a mass move of every Clojurian switching to one of those.
So it creates some questions?
1. Is it a problem that the language targets engineers more interested in a balance between non-functional and functional?
2. Should it be mutually exclusive, or can Clojure equally serve both niche? And if so, should it, why?
3. Is the claim that you can be as productive and it is just as easy to build a product without using a framework in Clojure true? Does this apply to everyone, or only certain personalities or people with certain amount of lower level knowledge?
4. Is Clojure's marketing misleading? Are people looking to just "build the product using an established framework" mislead in thinking Clojure will offer them salvation?
5. Where do most developer fall in, if they don't fall in the category Clojure currently targets, then does that mean Clojure cannot become mainstream? To go mainstream does it mean you have to target frameworks because there are more developers looking to just make a product using a framework?
I don't have answers to these, I'm just trying to define the current state and what the problem with it might be, or if it even is a problem.
Curious, can you elaborate on what you mean by this?
Best part about Rails is that you can do this as well. Rails has done a great job of being adaptable to new web technologies.
If your Org behaves like the above you could have the best Rails-Closure framework in the world but you're still destined for implosion.
Also, the syslog server example doesn't seem like something a Rails would solve.
> can we please just get together at the next Conj conference decide what our "Rails" is going to be?
I think these days we, OP included, tend to use "Rails" to not just mean a monolithic framework, but also one that monopolizes the ecosystem which is one thing that made Rails good.
But since it's been tried before in the Clojure and Node ecosystems, maybe people just don't care for it.
It's one of the reasons I stopped using Elixir. When I reached out to Stripe (and other 3rd party tools) they said the demand wasn't there to justify the dev time to maintain an official library internally. I think this is the reality to expect when using a more niche language. That may be ok if you're ok with that but it's important to know that's what you're getting into when you choose languages like this. It's a personal preference.
GraalVM is trying to solve this with its polygot VM idea https://www.graalvm.org/22.1/reference-manual/polyglot-progr...
Technically this tool mostly does exist already with the OpenAPI specification if we're talking about REST APIs. If you as the API provider put in the leg work to create a very detailed specification which is a YAML file, you can generate programming language specific SDKs out of it as long as the language has an OpenAPI library to consume this spec in an accurate way.
Stripe has publicly mentioned[0] they mostly use this spec to generate their SDKs (even as of a few years ago), but I guess auto-generated code still requires some developer time and there's a quality assurance level of "hey we're dedicated to internally supporting this". It's a huge deal having a provider internally support your language's SDK.
[0]: https://github.com/stripe/stripe-python/issues/694#issuecomm...
e.g https://github.com/cognitect-labs/aws-api
I think the fundamental truth here is that so much of writing software these days is having open source to rely on. That applies to things like an SDK for a vendor API, but also applies to things like consolidating community effort on one approach to back-end web app -- instead of having a bunch of not really mature options, and also so effort can be consolidated on building additional things that work with that popular tool.
I don't think these two areas are actually related like the OP does -- if we can just reach consensus on a web framework, then somehow we'll get things like a Stripe SDK, I'm not sure about that.... but I think the general challenge of "how do we get a sustainable, supported, mature/polished open source ecosystem of software that we can use that works together" is kind something we don't really understand, how does that happen (or not) for a new language? I don't think it's just "well, if we only get popular enough it'll happen on it's own", and of course there is a chicken and egg issue there too.
I think one problem is people who tend to flock to niche languages like to write libraries so what happens is you get a random person on the internet who makes a library, then they eventually stop using it personally and the project dies because they were the sole maintainer championing the project. You have this happen in parallel and the next thing you know you have 3-4 libraries that are either missing big features, fall out of date, go into maintenance mode, etc.. I saw this a lot with Elixir's community libraries (Stripe, AWS SDK, pagination, etc.).
There is a chicken / egg aspect to it because I think the fix here is to eventually get popular enough where you have a large enough of a community around the language and ecosystem that the most popular projects for a specific feature get so popular that they can't die because too many people depend on it and then a bunch of folks start to contribute to it and the community informally blesses 1 solution as "the" solution. The chicken / egg problem here is that a lot of people want to use Stripe, not create a library to use Stripe, etc. so it never gets mass adopted.
I think the best way for this to happen is organically. Trying to create a committee with an org with specific contributors trying to invent solutions is going about it in the opposite way. The best libraries tend to be extracted out of real projects and grow to become popular because the person or people behind it have a skillset around not only creating high quality software but also a dedication to create great documentation, can handle support, have lots of tests and generally have created a maintainable code base that makes it easy for others to contribute.
> Ruby is no longer always considered popular enough to get an SDK from a vendor.
Do you have a bunch of examples here? I haven't come across that with the usual libs I use (Stripe, Braintree, PayPal, Sentry, Datadog, Mux, AWS, etc.).
But also just seeing in general more people trying to make money with open source. I think the way open source "naturally" gets produced for free is not what it was 15+ years ago.
> Do you have a bunch of examples here
I don't recall any off the top of my head, because of course it's nothing I use, working in ruby. Most of the ones you mention have been around with ruby SDK's for over 10 years already, had them released when ruby was still at the top. And certainly an SDK is still more likely to be available in ruby than elixir or clojure. Let me see if I can find a few examples remembering the sorts of things I was seeing... OK some examples that don't have vendor-supplied ruby SDK but do for other languages:
* vimeo: https://developer.vimeo.com/api/libraries
* slack: https://api.slack.com/tools
* dropbox: https://www.dropbox.com/developers/documentation
In some cases there are third-party open source libraries available, often with limited coverage and/or kind of abandoned. Vimeo is the only one I really dug into, the third-party open source "sdks" did not seem to meet my needs or do much.
Fortunately there's https://github.com/slack-ruby/slack-ruby-client, but it's not official. Although it seems to be a good example of a community driven library coming together. I haven't used it personally (I never interfaced with Slack in a Ruby app) but at a glance it looks like it has really good docs, a decent amount of contributors, well maintained, etc.. If I were building some Rails app that used Slack I'd likely reach for this and not feel bad about it.
Kind of a bummer on the other 2 tho. Thankfully I wouldn't be building too many apps using those tools, but I get the point you're making. In the grand scheme of things I think this also shows how much more popular Python is than Ruby.
i work in rails, and my main hobby lang is clojure. i hate the thinking patterns of OO more and more every day.
'Clojure needs a rails' has come up perennially for that last 10 years. I don't disagree, as I think convergence on one web solution rather than a constant churn would have great benefits. The idea that you can't use sysdig, or whatever, because there is no Clojure library, is nonsense. Interop is good and Clojure is nicer than Java for writing Java :)
Also, you can overload functions with further aritiss in protocols just fine. Nothing to do with variadics.
I'm arguing two things:
First, Clojure has missing or rotting libraries, requiring re-implementing logic we've all done at least once because people are fixated building the Rails for Clojure.
Second, that the business case for using Clojure becomes weaker because the lack of open-source efforts on this front (due to the 'Rails Vacuum') causes either interal library building (extra time) or, more likely, interop. Interop, to _some_ management, sounds an awful lot like using Java, but with very expensive Clojure developers to do it.
Sure, I have used Java interop for working with Stripe, but that's because it is futile to maintain a 3rd party Clojure library if Stripe decided not to do it themselves. These proprietary APIs are constantly changing. It is perfectly OK to do Java interop in these cases. Any niche language will have the same problem, and Clojure interop with Java is quite pleasant to use, compared with using Java itself.
As far as I can tell, using ring and related things are fairly standard on the server side, and re-frame and related things are fairly standard on the client side, so I am honestly not sure what is missing for Web development in Clojure, as Web development is the most common use case of Clojure.
I am honestly confused.
I have revisited Clojure every now and then after and IMHO its in a weird state, too popular and successful to completely die, but also limited to reach wider adoption.
When revisiting old projects I have seen countless of Clojure dependencies having died in the meantime. I have replaced utility libraries with more recent utility libraries sometimes several times. Very annoying. Paradoxically the core API is really stable to a degree that bugs aren't fixed that are a behaviour userland code might rely on.
But its overall I think more of a symptom than a cause for frictions when developing Clojure. Community management is bad, even worse that RH nourishes this elitist aura around himself. Then the JVM integration is of course the blessing that led to Clojure's initial success but is at the same time limiting. The Java community by majority seems to gravitate towards Scala (weird scala version incompatibilities incoming, Scala DLL Hell) and Kotlin.
So overall Clojure is in a strange situation. For picking it up on-the-job its not the responsible choice probably for most situations I work in. For my private tinkering the JVM dependency feels just a little too heavyweight. I don't see that Clojure will hit an inflection point of adoption in any way in the future that would make it a viable candidate for me to use at work.
I really would hope that a Clojure-on-guile or Clojure-on-racket implementation becomes ready at some point that would make me want to switch from Scheme to Clojure for the toy projects at least.
Can you give an example? As a medium user of Clojure I struggle to think of anything in the core that could rightly be considered a "bug" and fits this description.
That's a weird take, considering it's consistently called out as one of the most positive aspects of Clojure on surveys.
I got to admit that maybe community management might have improved in the last few years.
Clojure has more leadership along the lines of "hey, other devs, I am open sourcing this cool stuff that I think is good. Here is my philosophy on why I like these types of libraries and would love contributions if you agree with the direction. I'm happy there are other directions being taken as well, that's great. Oh, also, some non-contributors have been making demands that I spend time on their problems and my answer is a firm no."
I think it is a hugely positive aspect, but I also open source my own libraries and have the same feelings towards people who make demands of me without being willing to contribute so I'm certainly biased.
I have never seen Rich say or do anything that matches this description.
Yes, the "best in class" libraries change every few years in Clojure's universe, but they always bring huge improvements that require completely new APIs.
For instance:
https://github.com/plumatic/schema -> https://github.com/metosin/malli
https://github.com/juxt/bidi -> https://github.com/metosin/reitit
https://github.com/bhauman/lein-figwheel -> https://github.com/thheller/shadow-cljs
https://github.com/stuartsierra/component -> https://github.com/weavejester/integrant
https://github.com/technomancy/leiningen -> https://github.com/practicalli/clojure-deps-edn
> So overall Clojure is in a strange situation. For picking it up on-the-job its not the responsible choice probably for most situations I work in. For my private tinkering the JVM dependency feels just a little too heavyweight. I don't see that Clojure will hit an inflection point of adoption in any way in the future that would make it a viable candidate for me to use at work.
I think about programming languages as tools for different bottlenecks.
– Computational: Rust
– Networking: Go
– Business logic: Clojure / Python
And for many people Python has everything that Clojure is missing: a very low barrier to enter, and a very stable and well-documented ecosystem of frameworks and libraries.
What I mean by stable, is that my code remains working after I upgrade the dependencies, which is definitely NOT the case for python frameworks, but it is mostly the case for Clojure libraries.
In Clojure, Web related things have remained relevant for a long time as well. For example, I do not anticipate that the standard things such as Ring, Reagent, Re-frame, etc. are going to change any time soon.
If you have not used Clojure, you just have not experienced what a stable ecosystem looks like. You have too used to changing your own code just because of a dependency bump. This is very rare in the Clojure world, but a norm outside.
In Clojure, "stable" means that your code remains working after the upgrades of its dependencies.
In Python, "stable" means that the dependencies themselves remain upgraded.
I am not suggesting that one thing is better than the other. What I meant, is that one thing is more attractive than the other for an average developer.
Currently, there are very few Clojure libraries from a decade ago that are still being actively maintained today.
As I have already mentioned, "actively maintained" is not a requirement for many Clojure libraries. Many are just done, as in "nothing needs to be changed".
As long as they are still working, why do you want to update them? As I mentioned, many production code bases in Clojure contain libraries that have not been updated for ages, but they still work. Why fixing things that are not broken?
In fact, it is rare that I do not need to change code after I upgrade python dependencies. Most likely I have to change my code.
I’m going to steal this!
What exact does it mean by "died"? It does not work any more? Very unlikely.
In the Clojure world, it is the norm that a 5 years old library works as well as a one-week old one.
If you mean "died" by "not updated any more", sure that's the norm in Clojure as well. Some libraries are done, and some authors moved on. But most likely, these libraries can still be used just fine.
My production code base contains plenty of libraries that are not updated for ages, but I consider that a benefit, not a fault.
Java keeps getting more popular, pushed by the very solid innovations that Oracle has been trickling into it every release, and Kotlin has certainly picked up a lot of momentum these past five years.