I wonder if he’ll still give talks. I don’t blame him if not, but he’s really good at it.
“Simple made easy” was a classic even if everyone proceeded to ignore the practical advice contained therein. And “Maybe not” is my personal favorite, a great discussion of requirements/provisions and the downsides of option types.
Because of your comment, I watched this for the first time. I agree that it deserves a spot in the classics.
For those who haven't seen it, in the presentation Rich talks about figuring out the problem before you work on a solution. It's definitely something that could be improved in myself and in a lot of people and organizations involved in software development.
I'll never forget "gem install hairball" during his talk "Simplicity Matters" at Ruby Conf 2012 where he basically told the Ruby community they were doing it wrong.
Yep, the guy has a gift. I've never touched a line of Clojure or looked at Datomic, but I'm repeatedly inspired by Rich's talks, which I come back to often.
Maybe Not is an excellent video in some ways, but I just watched it a few years later and it reminded me that as much as I love almost every Rich video, I do wish he'd spend some time reading a little deeper on type systems and theory.
For example his note about "Categorical descriptions" and then the explanation about maps/and spec at 25:00[1] seems to indicate he's unfamiliar with the difference between extensional and intensional type theory.
The core difference between the two is extensional type systems decide equality on the observable behavior of the output. For example if two functions take the same input, and produce the same output extensional type systems/theories say they are the same. The problem with this is type equality is then undecidable.
Extensional type systems also struggle because there are more than one way things might be equal! For example two string might be equal if you ignore case, but unequal if you don't. So if we want our type system to check if two functions are equal under case insensitivity an extensional type system will struggle with this.[2][3]
Now in fairness to Rich, these concepts come more from the math community, especially the notion of types being equal under different paths, and in the case of Homotopy type theory are burred in unapproachable language and concepts... even by mathematicians standards.
He also talks about select from spec, and it not preventing you from passing types broader types, and the necessity of specifying deeper type.[4] But, that's just an eliminator defined on the type[5].
Now most strongly typed programming languages suffer from the issue he identifies when talking about not making brittle systems [6] and the issues you get with coupling around taking a map/struct/product type C = A X B (e.g a map C = {A: something, B: something}), and you require A so you're function should be good but you fail to compile because the function says A, and now you're passing C.
It's frankly pretty annoying in places like Java that if you have a Point = Int x Int y you can't use it in functions which take a Tuple = Int x Int y. I'd actually say that's the common observation that both Rich, the Go Inventors, and the TypeScript folks have all made. You need to care about the extensional type of your input (and output) fairly often. And carring around the intensional type causes both dependency issues and just some general PITA's.
Don't think Rich Hickey is "retiring" retiring, just that he's retiring from Nubank and "commerical software development".
TFA: I look forward to continuing to lead ongoing work maintaining and enhancing Clojure with Alex, Stu, Fogus, and many others, as an independent developer once again... Retirement returns me to the freedom and independence I had when originally developing Clojure. The journey continues!
Rich Hickey's talks inspired me to explore Clojure and functional programming in college. That led to a part of my career where I got to help advance an FP language (F#). Can't say it was a raging success, but it wasn't a raging failure either. Really happy things worked out the way they did, though. I still look back at my senior project, written in Clojure, from time to time and find it elegant in a unique way.
My experience was similar. I've built far more impactful things since then, but at the end of the day, the software that I'm most proud of was a calculator, of all things, that I made in Clojurescript. It was loosely based on the final chapter of SICP, and implemented an hilariously metacircular approach to transitioning the state of the register machine. Since the implementation was way more interesting than the actual application, I used Processing to visualize the register machine as if it were a rudimentary ALU (which it basically was).
If it hadn't been for Rich Hickey, or Clojure, I'm not sure I'd have the slightest inkling of how joyful and creative it can be to write software. It was an A+ experience, even if I'd be swiftly fired for trying to push logic like that into production.
FWIW, I started programming F# during your tenure at MS and in a couple of years it became a really capable cross platform option - partly MS, partly community, but definitely helped out by you. Thanks!
This is awesome! One, it reinforces the ongoing maturing of Clojure, the language as a team project. Two, it portends cool new things to come; the kind that can only come from a place of personal liberty. Three, BDFL is retiring from an employer, headlong into his life's work. I for one am cheering.
This is probably great news for clojure devs. It sounds like Rich just wants his autonomy back. Being able to choose exactly what you do, and do not, work on is precious.
While I never took the time to learn Clojure in depth or use it for any serious project, I feel like I've been deeply inspired by Hickey's talks. A lot of the principles he promoted helped me so much in my career as a developer.
Congrats Rich! I met you at Stu's house during a rousing game of Marvel Champions. Thank you for Clojure, Datomic, Cognitect, and your talks. Your creations have enriched my life in many ways. I hope retirement affords you ample hammock time!
The word congratulations came into English from Latin, where it was formed by combining prefix com-, meaning with, to gratulari, meaning "give thanks" or "show joy." Gratulari is derived from gratus, as is gratitude. Gratus means "pleasing," "thankful."
Having only had the good fortune to meet you in person a few times:
Congratulations, Rich. You have my deepest gratitude. It fills me with great joy to know you are taking this step. I look forward to what your next stage brings.
Side question: Rich seems to use etymology as a tool for original thinking and clear explanations. Are there other people who do this as well as he does?
I don't know about technologists who use it in writing or speaking like Rich, but you can trace writers' use of etymology to provoke thinking at least back to Plato. More recent, prominent examples would be Hannah Arendt's acceptance speech in 1975 receiving Denmark's Sonning Prize, in which she uses the etymology of "person", and at least Martin Heidegger's essay The Question Concerning Technology in which he plays with several, including the etymology of "technology" itself.
I’ve always been fascinated with language, and long had a hobby of looking up etymologies (my high school Latin teacher was great at pointing these out in class), but Rich’s style definitely pushed me to use it as a tool for thinking. Naming things is (famously) hard, but a strong command of language (and willingness to dig through the thesaurus) has lit the way through many difficult situations.
I use etymonline for this kind of thing often. Our complex words are compositions of lots of forgotten meaning... I find it helps to retrace those steps.
Funnily enough, your complaint here is that although it may be simple, it is not necessarily easy, while the talk is an argument that we should prefer simplicity even at the cost of some (temporary) un-ease.
I'm not sure how you got that impression from my comment.
But for the sake of argument, I'm saying "complect" is neither simple *nor* easy. It's obviously not easy since it's unfamiliar. But nor is it simple, or to be really precise, no simpler than alternatives, since it doesn't conceptually improve on plain words and phrases like "complicate", "intertwine", "mix up", "separate", "tease apart", etc. It adds nothing, really.
Your comment kind of highlights some of the errors many take away from "Simple Made Easy": that ease is in opposition to simplicity and/or that they are completely orthogonal.
Few people capture this spirit like the classic "Congratulations" video from Big Man Tyrone.
While it has gone through the revolutions from sincere, to sarcastic in intent, to back again, I believe the inherently joyful attitude underpinning the original video carries the buoyishly optimism of sincere joy.
Clojure has changed my whole career, and many of Rich's talks directly inspired my thinking about large systems. I hope retirement gives Rich the best Hammock Time!
I, for one, and super excited to see how Rich and Clojure evolve post-Nubank. Clojure and Rich's talks greatly inspired me and improved my software thinking skills - I can't say thank you enough!
Amazing news. I saw a video cast of Rick during a meeting @ NuBank, and it looked like corporate life had sucked the life out of him [1]. So, I'm glad he got out of there, and I hope he's able to shake it off like a bad dream. Building something like Clojure require radical thinking, and creative freedom. I can only imagine how trying to fit that into your weekly scrum sprint would be draining. And in the monthly meetings where they recite the corporate values.. shivers
Observing a public figure and expressing concern that their workplace doesn't align with their passions isn't prying into their personal life, or speculating about their mental health. What I'm saying is: NuBank is where hopes and dreams go to die, and corporate life more generally. It's not about being "on" or "off" but about being in an environment one can thrive.
After two years at Cognitect (pre-nubank), I can say that his talks are more subdued than his usual way of speaking (in slack, at least). He's not as mild-mannered as he appears.
Funny, I too thought of that very video the moment I’d read the news. Rich was barely managing to feign just enough engagement without appearing rude. Happy he has escaped.
i’m very grateful to rich for clojure, cognitect’s libraries/tooling, and datomic. all of these have been well maintained, stable, and backwards compatible over the years.
although he’s no longer working on datomic, i really hope cognitect values growing it. it feels like such an improvement over sql to work with.
Could not disagree more strongly that Datomic is well maintained. I'd view it as a significant liability in any organization using it without very good reason. My experience operationally supporting Datomic at even a moderate scale was a total nightmare and soured me on the Clojure ecosystem as a whole.
may i ask what you consider moderate scale in terms of daily/concurrent users? i’ve had no issue with small projects, but i’m curious to know where it starts falling over
Without Clojure and the immense effort of the community around it and ClojureScript, OrgPad would not be possible and the individuals on the team would probably be stuck in some academic or corporate structures instead of having basically a two-families startup/ company. So thank you Rich and thank you to everybody else, who supported You on the journey to create something that makes coding predictable, productive and almost fun instead of laborious work. ;-)
I wrote my first trading system in Clojure because a few of the libraries we relied on were in java.
It was 2011 and I'd had about 3 years of lisp experience. I got a bit of side eye from people when I told them I was using a relatively new programming language but the fact that it was based on the JVM, which alot of HFT firms were using helped make the case.
We didn't use if for more than a few years before it was retired and rewritten, though that was due to new requirements that included C++ interop.
In the end tracking memory usage and allocations got too hard and if you've ever written something that is time sensitive you'll know just how slow memory allocation is, so you could argue I made a poor choice but for the rewrite was easy to reuse the java libraries when we moved to java.
The harder part was porting the algorithm implementation code.
The reason was that I chose a common lisp trick of first writing a DSL in Clojure that was then used to write the algos.
That was the one part that seemed worse in the rewrite as it became far more verbose and clunky.
But man was it fun, I learned more in a month than I often do in a year now.
Thanks Rich!! It's not often you get to experiment with cool tech and make lots of money doing it. I appreciate your work.
> The harder part was porting the algorithm implementation code. The reason was that I chose a common lisp trick of first writing a DSL in Clojure that was then used to write the algos.
Doesn't it mean that Lisp macros should really be avoided at all costs?
Can't say for sure, given that my sample size is one.
The upsides of DSL(Domain specific languages) and macros in general is that they cut down on code duplication and give everyone using them a common language and set of tools on which to build an app. That part is awesome and I think really helped to speed up development of the algos that were built on top of the DSL.
The downside is, that you need to learn the DSL but that's a very small impact in my experience. THe larger downside as you and I point out is that those tools aren't really there for other languages so if you do plan on porting the code base to another language then maybe steer clear of macros.
But again, I wouldn't say I have anywhere near enough experience to claim to be an expert in that.
> So you had to switch to C++ because Clojure / Java wasn't performant enough?
Clojure yes, java no. Many HFT firms build their systems in java and run on the jvm(specialized hardware aside), its plenty fast if you avoid memory allocations.
It is also possible Clojure is more than up to the task now and its also reasonable to believe that I wasn't a strong enough clojure developer to fix those issues at the time and a more competent developer could have made it work.
But like almost all things if you chose a very out of the mainstream language and start to have issues, most developers will point fingers at the language first.
Side note, its a miracle that Jane Street kept using OCaml after the first few years and points to how strong their core tech team was.
C++ was an external dependency brought on by other systems that needed to be integrated and is a far safer language to use. I'd imagine that if you say HFT to most developers, their first thought would be C++ even if they haven't worked in the domain before.
It's like choosing Julia for your machine learning platform and then switching to python.
It's a shame you couldn't have pitched Common Lisp. Maybe could have ported your DSL pretty quickly and wound up having SBCL output nice compiled assembly instructions and shown x% speed improvement without much effort by using a "sister language" of sorts? ABCL in 2011 would have been a harder sell, it only had its 1.0 release that October.
A few years ago another commenter noted they had developed a trading system in Lisp and C https://news.ycombinator.com/item?id=25222297 (further comments elaborate a bit) before being asked to rewrite it in Java. What keeps less popular languages alive at companies, when there's no forcing reason to get rid of it like really unacceptable performance, is a conviction (borne out as far as I can tell) that it's ok to hire people who don't know the language -- they'll get up to speed more than fast enough.
> ... its also reasonable to believe that I wasn't a strong enough clojure developer to fix those issues...
Technically it can likely be done but in practice pushing Clojure code for performance (eg, [0]) doesn't follow the usual language idioms and relies on a sophisticated understanding of the interactions between program design, Clojure, the Clojure compiler and the JVM. I'd suggest pointing more at the language than the programmer. In a situation where direct memory management is required Clojure is a weaker choice.
Although in Clojure's defence it leverages immutability to squeeze out some surprising performance benefits without much effort. I would expect Clojure code to be fast by default but challenging to hand-optimise further if that extra level of control is required. Clojure makes it easy to write code that performs well, it is a stronger language for situations where ordinary business logic changes are the productivity bottleneck.
For extremely low latency, Clojure can be an awkward fit. By default there's a lot of sugar in the syntax, so if you need to be precise about the underlying types of your data and the exact datastructure, you need to toss out most of the standard datatypes and datastructures. It's small details like: "I definitely have an array of unboxed integers, and I need to be sure this one operation won't accidentally allocate a new list of boxed integers, and that this function I'm calling doesn't dispatch dynamically" or things like that. You start using `deftype` everywhere, sometimes dipping into Java, etc. You may as well just write java at that point. However from a pure Clojure standpoint, the author of Neanderthal has done great work on making Clojure viable for high performance numerical computing.
Languages aren't "performant". You can write good or bad code in most languages, or code that isn't a good fit for a language.
That said, there are languages which will make you incur certain penalties, as there is a price to be paid for automatic memory management with garbage collection, for example. These penalties generally do not matter except for edge cases (and HFT trading might very well be one!).
There are also languages which help you write much faster code, but on a higher level. Clojure transducers are a good example: pipelines built of composable transformations, where data sequences are not fully realized between the transformations. These can provide significant performance (and memory allocation) improvements, while still letting you write high-level code.
Yes. And it really doesn't matter in practice. In a larger system this kind of performance won't matter. What will matter is design decisions, data structures, databases and access patterns — things that are much more high-level than "language performance".
Just a small nitpick, but allocation is definitely not slow in case of the JVM, it is often faster than manually managed languages. It is a pointer bump only.
All the necessary mechanisms of a GC does have an overhead, so your point stands, but not for the mentioned reason.
we used clojure on a pretty big project. We made the decision to rewrite in java after it became clear that some fairly big libraries we were using weren't being maintained.
It was contentious, but I'm glad we dumped clojure.
No matter how good a project is, it needs loads of support behind it
157 comments
[ 3.1 ms ] story [ 238 ms ] thread“Simple made easy” was a classic even if everyone proceeded to ignore the practical advice contained therein. And “Maybe not” is my personal favorite, a great discussion of requirements/provisions and the downsides of option types.
[0]: https://www.youtube.com/watch?v=f84n5oFoZBc
For those who haven't seen it, in the presentation Rich talks about figuring out the problem before you work on a solution. It's definitely something that could be improved in myself and in a lot of people and organizations involved in software development.
For example his note about "Categorical descriptions" and then the explanation about maps/and spec at 25:00[1] seems to indicate he's unfamiliar with the difference between extensional and intensional type theory.
The core difference between the two is extensional type systems decide equality on the observable behavior of the output. For example if two functions take the same input, and produce the same output extensional type systems/theories say they are the same. The problem with this is type equality is then undecidable.
Extensional type systems also struggle because there are more than one way things might be equal! For example two string might be equal if you ignore case, but unequal if you don't. So if we want our type system to check if two functions are equal under case insensitivity an extensional type system will struggle with this.[2][3]
Now in fairness to Rich, these concepts come more from the math community, especially the notion of types being equal under different paths, and in the case of Homotopy type theory are burred in unapproachable language and concepts... even by mathematicians standards.
He also talks about select from spec, and it not preventing you from passing types broader types, and the necessity of specifying deeper type.[4] But, that's just an eliminator defined on the type[5].
Now most strongly typed programming languages suffer from the issue he identifies when talking about not making brittle systems [6] and the issues you get with coupling around taking a map/struct/product type C = A X B (e.g a map C = {A: something, B: something}), and you require A so you're function should be good but you fail to compile because the function says A, and now you're passing C.
It's frankly pretty annoying in places like Java that if you have a Point = Int x Int y you can't use it in functions which take a Tuple = Int x Int y. I'd actually say that's the common observation that both Rich, the Go Inventors, and the TypeScript folks have all made. You need to care about the extensional type of your input (and output) fairly often. And carring around the intensional type causes both dependency issues and just some general PITA's.
[1] https://www.youtube.com/watch?v=YR5WdGrpoug&t=1504s [2] https://math.stackexchange.com/questions/4486995/what-is-an-... [3] https://en.wikipedia.org/wiki/Intuitionistic_type_theory#Ext... [4] https://youtu.be/YR5WdGrpoug?t=2668 [5] https://www.quora.com/In-type-theory-what-is-an-eliminator-a... [6] https://youtu.be/YR5WdGrpoug?t=2823
TFA: I look forward to continuing to lead ongoing work maintaining and enhancing Clojure with Alex, Stu, Fogus, and many others, as an independent developer once again... Retirement returns me to the freedom and independence I had when originally developing Clojure. The journey continues!
If it hadn't been for Rich Hickey, or Clojure, I'm not sure I'd have the slightest inkling of how joyful and creative it can be to write software. It was an A+ experience, even if I'd be swiftly fired for trying to push logic like that into production.
https://cognitect.com/blog/2020/07/23/Cognitect-Joins-Nubank
https://news.ycombinator.com/item?id=23926407
https://building.nubank.com.br/welcoming-cognitect-nubank/
Very interesting story!
edit: not one year ago at all
(edit: typos, formatting)
Having only had the good fortune to meet you in person a few times:
Congratulations, Rich. You have my deepest gratitude. It fills me with great joy to know you are taking this step. I look forward to what your next stage brings.
Side question: Rich seems to use etymology as a tool for original thinking and clear explanations. Are there other people who do this as well as he does?
e.g. https://www.etymonline.com/word/system - see how it's about composition, not so much about ways of doing things.
But for the sake of argument, I'm saying "complect" is neither simple *nor* easy. It's obviously not easy since it's unfamiliar. But nor is it simple, or to be really precise, no simpler than alternatives, since it doesn't conceptually improve on plain words and phrases like "complicate", "intertwine", "mix up", "separate", "tease apart", etc. It adds nothing, really.
Your comment kind of highlights some of the errors many take away from "Simple Made Easy": that ease is in opposition to simplicity and/or that they are completely orthogonal.
https://youtu.be/SxdOUGdseq4
One of the funniest Conj talks ever.
Perhaps you've never listened to a Rich Hickey talk before but he's notorious for riffing on the meaning of words.
Anyways thanks for the explanation. I wasn't in on the meme
While it has gone through the revolutions from sincere, to sarcastic in intent, to back again, I believe the inherently joyful attitude underpinning the original video carries the buoyishly optimism of sincere joy.
[1] https://news.ycombinator.com/item?id=34781607
It's a bit distasteful to virtually follow him around and speculate about his mental health.
although he’s no longer working on datomic, i really hope cognitect values growing it. it feels like such an improvement over sql to work with.
A true computing thought leader. All the best Rich. My only complaint about you was not open-sourcing datomic and making it free for developers.
- https://blog.datomic.com/2023/04/datomic-is-free.html - https://blog.datomic.com/2023/06/datomic-cloud-is-free.html
It was 2011 and I'd had about 3 years of lisp experience. I got a bit of side eye from people when I told them I was using a relatively new programming language but the fact that it was based on the JVM, which alot of HFT firms were using helped make the case.
We didn't use if for more than a few years before it was retired and rewritten, though that was due to new requirements that included C++ interop.
In the end tracking memory usage and allocations got too hard and if you've ever written something that is time sensitive you'll know just how slow memory allocation is, so you could argue I made a poor choice but for the rewrite was easy to reuse the java libraries when we moved to java.
The harder part was porting the algorithm implementation code. The reason was that I chose a common lisp trick of first writing a DSL in Clojure that was then used to write the algos.
That was the one part that seemed worse in the rewrite as it became far more verbose and clunky.
But man was it fun, I learned more in a month than I often do in a year now.
Thanks Rich!! It's not often you get to experiment with cool tech and make lots of money doing it. I appreciate your work.
Doesn't it mean that Lisp macros should really be avoided at all costs?
The upsides of DSL(Domain specific languages) and macros in general is that they cut down on code duplication and give everyone using them a common language and set of tools on which to build an app. That part is awesome and I think really helped to speed up development of the algos that were built on top of the DSL.
The downside is, that you need to learn the DSL but that's a very small impact in my experience. THe larger downside as you and I point out is that those tools aren't really there for other languages so if you do plan on porting the code base to another language then maybe steer clear of macros.
But again, I wouldn't say I have anywhere near enough experience to claim to be an expert in that.
Clojure yes, java no. Many HFT firms build their systems in java and run on the jvm(specialized hardware aside), its plenty fast if you avoid memory allocations.
It is also possible Clojure is more than up to the task now and its also reasonable to believe that I wasn't a strong enough clojure developer to fix those issues at the time and a more competent developer could have made it work.
But like almost all things if you chose a very out of the mainstream language and start to have issues, most developers will point fingers at the language first.
Side note, its a miracle that Jane Street kept using OCaml after the first few years and points to how strong their core tech team was.
C++ was an external dependency brought on by other systems that needed to be integrated and is a far safer language to use. I'd imagine that if you say HFT to most developers, their first thought would be C++ even if they haven't worked in the domain before.
It's like choosing Julia for your machine learning platform and then switching to python.
A few years ago another commenter noted they had developed a trading system in Lisp and C https://news.ycombinator.com/item?id=25222297 (further comments elaborate a bit) before being asked to rewrite it in Java. What keeps less popular languages alive at companies, when there's no forcing reason to get rid of it like really unacceptable performance, is a conviction (borne out as far as I can tell) that it's ok to hire people who don't know the language -- they'll get up to speed more than fast enough.
Technically it can likely be done but in practice pushing Clojure code for performance (eg, [0]) doesn't follow the usual language idioms and relies on a sophisticated understanding of the interactions between program design, Clojure, the Clojure compiler and the JVM. I'd suggest pointing more at the language than the programmer. In a situation where direct memory management is required Clojure is a weaker choice.
Although in Clojure's defence it leverages immutability to squeeze out some surprising performance benefits without much effort. I would expect Clojure code to be fast by default but challenging to hand-optimise further if that extra level of control is required. Clojure makes it easy to write code that performs well, it is a stronger language for situations where ordinary business logic changes are the productivity bottleneck.
[0] https://blog.redplanetlabs.com/2020/09/02/clojure-faster/
Languages aren't "performant". You can write good or bad code in most languages, or code that isn't a good fit for a language.
That said, there are languages which will make you incur certain penalties, as there is a price to be paid for automatic memory management with garbage collection, for example. These penalties generally do not matter except for edge cases (and HFT trading might very well be one!).
There are also languages which help you write much faster code, but on a higher level. Clojure transducers are a good example: pipelines built of composable transformations, where data sequences are not fully realized between the transformations. These can provide significant performance (and memory allocation) improvements, while still letting you write high-level code.
Just a small nitpick, but allocation is definitely not slow in case of the JVM, it is often faster than manually managed languages. It is a pointer bump only.
All the necessary mechanisms of a GC does have an overhead, so your point stands, but not for the mentioned reason.
Typically in HFT We just want to avoid any full sweep garbage collection between 8am and 6pm
It was contentious, but I'm glad we dumped clojure.
No matter how good a project is, it needs loads of support behind it